[RFC 2.6.10 2/22] xfrm: Add xfrm offload management calls to struct netdevice

[RFC 2.6.10 2/22] xfrm: Add xfrm offload management calls to struct netdevice

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:28:25-05:00 dave@thedillows.org 
#   Add the xfrm offload management calls to struct netdevice.
#   
#   xfrm_state_add() is called for inbound xfrm states
#   xfrm_bundle_add() is called for outbound xfrm bundles
#   xfrm_state_del() is called for all offloaded xfrms,
#   	inbound or outbound.
#   
#   If a driver adds NETIF_F_IPSEC to its features, it must
#   provide all three callbacks.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/linux/netdevice.h
#   2004/12/30 00:28:07-05:00 dave@thedillows.org +11 -0
#   Add the xfrm offload management calls to struct netdevice.
#   
#   xfrm_state_add() is called for inbound xfrm states
#   xfrm_bundle_add() is called for outbound xfrm bundles
#   xfrm_state_del() is called for all offloaded xfrms,
#   	inbound or outbound.
#   
#   If a driver adds NETIF_F_IPSEC to its features, it must
#   provide all three callbacks.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/linux/netdevice.h b/include/linux/netdevice.h
--- a/include/linux/netdevice.h	2004-12-30 01:11:56 -05:00
+++ b/include/linux/netdevice.h	2004-12-30 01:11:56 -05:00
@@ -250,6 +250,9 @@
 };
 #define NETDEV_BOOT_SETUP_MAX 8
 
+struct xfrm_state;
+struct xfrm_offload;
+struct xfrm_bundle_list;
 
 /*
  *	The DEVICE structure.
@@ -415,6 +418,7 @@
 #define NETIF_F_VLAN_CHALLENGED	1024	/* Device cannot handle VLAN packets */
 #define NETIF_F_TSO		2048	/* Can offload TCP/IP segmentation */
 #define NETIF_F_LLTX		4096	/* LockLess TX */
+#define NETIF_F_IPSEC		8192	/* Can offload IPSEC crypto */
 
 	/* Called after device is detached from network. */
 	void			(*uninit)(struct net_device *dev);
@@ -464,6 +468,13 @@
 						   unsigned short vid);
 	void			(*vlan_rx_kill_vid)(struct net_device *dev,
 						    unsigned short vid);
+
+	void			(*xfrm_state_add)(struct net_device *dev,
+						  struct xfrm_state *x);
+	void			(*xfrm_bundle_add)(struct net_device *dev,
+						   struct xfrm_bundle_list *bl);
+	void			(*xfrm_state_del)(struct net_device *dev,
+						  struct xfrm_offload *xol);
 
 	int			(*hard_header_parse)(struct sk_buff *skb,
 						     unsigned char *haddr);

[RFC 2.6.10 0/22] Add hardware assist for IPSEC crypto

[David Dillow]

The following patch set adds hardware offload of the crypto operations for
IPv4 IPSEC processing. It gives a noticible speedup on my (admittedly older)
hardware, but given the recent numbers posted, can be a speedup even for
more recent hardware.

There are a few known issues with the current patchset, but I think it is
ready for wider review.

* Only the 3Com 3CR990 family of NICs are supported. I don't have hardware
	or documentation for the Intel cards.
* Doesn't do IPv6. Need someone to implement map_direction(), and
	AH/ESP handling, as well as come up with a card that supports it.
* The use of GFP_ATOMIC in xfrm_offload_alloc() is probably not a good idea.
* linux/skbuff.h cannot include net/xfrm.h currently, so there are
	redundant defines (requires some header cleanup, which I'm not
	very inclined to tackle at the moment.)
* TCP Segmentation offload seems broken by firmware 03.001.008. It could be
	my changes to support the offload, but that seems unlikely. I will
	have to investigate this.
* Latency suffers somewhat on smaller packets, it may be advisable to have
	a minimum packet size to offload.
* No real feedback on which xfrm_states have been offloaded or not.

The patch set will be sent as follow-ups to this post, or is available via:

	bk pull http://typhoon.bkbits.net/ipsec-2.6

It will update the following files:

 Documentation/networking/netdevices.txt |   16 
 drivers/net/typhoon.c                   |  687 +++++++++++++++++++++++++++++++-
 drivers/net/typhoon.h                   |   38 +
 include/linux/ethtool.h                 |    8 
 include/linux/netdevice.h               |   11 
 include/linux/skbuff.h                  |   55 ++
 include/net/dst.h                       |    1 
 include/net/xfrm.h                      |  108 +++++
 net/core/ethtool.c                      |   54 ++
 net/core/skbuff.c                       |   31 +
 net/ipv4/ah4.c                          |   99 ++--
 net/ipv4/esp4.c                         |  102 +++-
 net/ipv4/xfrm4_state.c                  |   10 
 net/ipv6/xfrm6_state.c                  |    9 
 net/xfrm/xfrm_export.c                  |    4 
 net/xfrm/xfrm_policy.c                  |   64 ++
 net/xfrm/xfrm_state.c                   |  101 ++++
 17 files changed, 1284 insertions(+), 114 deletions(-)

If you work from the mailed patches, you will want the netdev-2.6 updates
to the typhoon driver, as the 3CR990B series needs the newest firmware to
correctly offload IPSEC processing. That patch is available from

http://www.thedillows.org/typhoon-netdev-2.6.patch.bz2


The following results were generated using a dual processor PIII 1GHz/512MB
with a 3CR990SVR97 (ori) and an Athlon 550 MHz/256MB with a 3CR990B (tank).
Latency testing was performed with lmbench's lat_tcp, and bandwith testing
was performed with Andrew Morton's zcc/zcs/cyclesoak. I ran the tests
multiple times, and picked the median results to report. There was not much
deviation in the results (+/- 1.5 us +/- 50KBytes/s +/- 1.5% CPU usage).


TCP Latency tests (1 byte msg)

Config					Latency
No IPSEC				196 us
AH/SHA1 (sw)				256 us
AH/SHA1 (hw)				317 us
ESP/3DES,SHA1 (sw)			333 us
ESP/3DES,SHA1 (hw)			347 us
ESP-AH/3DES,SHA1-SHA1 (sw)		387 us
ESP-AH/3DES,SHA1-SHA1 (hw)		467 us

TCP Latency tests (1024 byte msg)

Config					Latency
No IPSEC				625 us
AH/SHA1 (sw)				771 us
AH/SHA1 (hw)				858 us
ESP/3DES,SHA1 (sw)			1999 us
ESP/3DES,SHA1 (hw)			902 us
ESP-AH/3DES,SHA1-SHA1 (sw)		2140 us
ESP-AH/3DES,SHA1-SHA1 (hw)		1131 us

Bandwidth tests

Config (sender -> receiver)		Bandwidth	ori CPU	  tank CPU
No IPSEC (tank->ori)			11494 KB/s	11.9%	  18.7%
No IPSEC (ori->tank)			11492 KB/s	9.5%	  34.3%

AH/SHA1 (sw) (tank->ori)		11303 KB/s	29.2%	  79.3%
AH/SHA1 (sw) (ori->tank)		11302 KB/s	28.6%	  91.1%
ESP/3DES,SHA1 (sw) (tank->ori)		2130 KB/s	29.6%	  100%
ESP/3DES,SHA1 (sw) (ori->tank)		2263 KB/s	29.3%	  99.7%
ESP-AH/3DES,SHA1-SHA1 (sw) (tank->ori)	1906 KB/s	29.1%	  100%
ESP-AH/3DES,SHA1-SHA1 (sw) (ori->tank)	2051 KB/s	29.3%	  99.7%

AH/SHA1 (hw) (tank->ori)		11303 KB/s	14.0%	  30.2%
AH/SHA1 (hw) (ori->tank)		11301 KB/s	14.1%	  39.8%
ESP/3DES,SHA1 (hw) (tank->ori)		11221 KB/s	15.4%	  44.9%
ESP/3DES,SHA1 (hw) (ori->tank)		11220 KB/s	21.5%	  48.1%
ESP-AH/3DES,SHA1-SHA1 (hw) (tank->ori)	5920 KB/s	10.8%	  35.9%
ESP-AH/3DES,SHA1-SHA1 (hw) (ori->tank)	7189 KB/s	14.3%	  35.4%


The last line seems suspicious, and should probably be retested.

[RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:27:15-05:00 dave@thedillows.org 
#   Add direction information to xfrm_state. This will be needed to
#   offload xfrm processing to the NIC.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_state.c
#   2004/12/30 00:25:42-05:00 dave@thedillows.org +5 -0
#   Add direction information to xfrm_state. This will be needed to
#   offload xfrm processing to the NIC.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/ipv6/xfrm6_state.c
#   2004/12/30 00:25:42-05:00 dave@thedillows.org +9 -0
#   Place holder for adding IPv6 direction mapping routine.
# 
# net/ipv4/xfrm4_state.c
#   2004/12/30 00:25:42-05:00 dave@thedillows.org +10 -0
#   Add direction information to xfrm_state. This will be needed to
#   offload xfrm processing to the NIC.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/net/xfrm.h
#   2004/12/30 00:25:42-05:00 dave@thedillows.org +10 -0
#   Add direction information to xfrm_state. This will be needed to
#   offload xfrm processing to the NIC.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
--- a/include/net/xfrm.h	2004-12-30 01:12:08 -05:00
+++ b/include/net/xfrm.h	2004-12-30 01:12:08 -05:00
@@ -146,6 +146,9 @@
 	/* Private data of this transformer, format is opaque,
 	 * interpreted by xfrm_type methods. */
 	void			*data;
+
+	/* Intended direction of this state, used for offloading */
+	int			dir;
 };
 
 enum {
@@ -157,6 +160,12 @@
 	XFRM_STATE_DEAD
 };
 
+enum {
+	XFRM_STATE_DIR_UNKNOWN,
+	XFRM_STATE_DIR_IN,
+	XFRM_STATE_DIR_OUT,
+};
+
 struct xfrm_type;
 struct xfrm_dst;
 struct xfrm_policy_afinfo {
@@ -194,6 +203,7 @@
 	struct xfrm_state	*(*find_acq)(u8 mode, u32 reqid, u8 proto, 
 					     xfrm_address_t *daddr, xfrm_address_t *saddr, 
 					     int create);
+	void			(*map_direction)(struct xfrm_state *xfrm);
 };
 
 extern int xfrm_state_register_afinfo(struct xfrm_state_afinfo *afinfo);
diff -Nru a/net/ipv4/xfrm4_state.c b/net/ipv4/xfrm4_state.c
--- a/net/ipv4/xfrm4_state.c	2004-12-30 01:12:08 -05:00
+++ b/net/ipv4/xfrm4_state.c	2004-12-30 01:12:08 -05:00
@@ -106,12 +106,22 @@
 	return x0;
 }
 
+static void
+__xfrm4_map_direction(struct xfrm_state *x)
+{
+	if (inet_addr_type(x->id.daddr.a4) == RTN_LOCAL)
+		x->dir = XFRM_STATE_DIR_IN;
+	else
+		x->dir = XFRM_STATE_DIR_OUT;
+}
+
 static struct xfrm_state_afinfo xfrm4_state_afinfo = {
 	.family			= AF_INET,
 	.lock			= RW_LOCK_UNLOCKED,
 	.init_tempsel		= __xfrm4_init_tempsel,
 	.state_lookup		= __xfrm4_state_lookup,
 	.find_acq		= __xfrm4_find_acq,
+	.map_direction		= __xfrm4_map_direction,
 };
 
 void __init xfrm4_state_init(void)
diff -Nru a/net/ipv6/xfrm6_state.c b/net/ipv6/xfrm6_state.c
--- a/net/ipv6/xfrm6_state.c	2004-12-30 01:12:08 -05:00
+++ b/net/ipv6/xfrm6_state.c	2004-12-30 01:12:08 -05:00
@@ -116,12 +116,21 @@
 	return x0;
 }
 
+static void
+__xfrm6_map_direction(struct xfrm_state *x)
+{
+	/* XXX This needs to be implemented by someone who knows
+	 * IPv6 better then I.
+	 */
+}
+
 static struct xfrm_state_afinfo xfrm6_state_afinfo = {
 	.family			= AF_INET6,
 	.lock			= RW_LOCK_UNLOCKED,
 	.init_tempsel		= __xfrm6_init_tempsel,
 	.state_lookup		= __xfrm6_state_lookup,
 	.find_acq		= __xfrm6_find_acq,
+	.map_direction		= __xfrm6_map_direction,
 };
 
 void __init xfrm6_state_init(void)
diff -Nru a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
--- a/net/xfrm/xfrm_state.c	2004-12-30 01:12:08 -05:00
+++ b/net/xfrm/xfrm_state.c	2004-12-30 01:12:08 -05:00
@@ -186,6 +186,7 @@
 		x->lft.soft_packet_limit = XFRM_INF;
 		x->lft.hard_byte_limit = XFRM_INF;
 		x->lft.hard_packet_limit = XFRM_INF;
+		x->dir = XFRM_STATE_DIR_UNKNOWN;
 		spin_lock_init(&x->lock);
 	}
 	return x;
@@ -404,6 +405,8 @@
 	if (unlikely(afinfo == NULL))
 		return -EAFNOSUPPORT;
 
+	afinfo->map_direction(x);
+
 	spin_lock_bh(&xfrm_state_lock);
 
 	x1 = afinfo->state_lookup(&x->id.daddr, x->id.spi, x->id.proto);
@@ -451,6 +454,8 @@
 	afinfo = xfrm_state_get_afinfo(x->props.family);
 	if (unlikely(afinfo == NULL))
 		return -EAFNOSUPPORT;
+
+	afinfo->map_direction(x);
 
 	spin_lock_bh(&xfrm_state_lock);
 	x1 = afinfo->state_lookup(&x->id.daddr, x->id.spi, x->id.proto);

[RFC 2.6.10 4/22] xfrm: Try to offload inbound xfrm_states

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:33:11-05:00 dave@thedillows.org 
#   Plumb in offloading of inbound xfrm_states.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_state.c
#   2004/12/30 00:32:54-05:00 dave@thedillows.org +28 -1
#   Try to offload an inbound xfrm_state when it is added or updated.
#   Since it could potentially come in from any interface, try to
#   offload it on all devices that support it.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
--- a/net/xfrm/xfrm_state.c	2004-12-30 01:11:30 -05:00
+++ b/net/xfrm/xfrm_state.c	2004-12-30 01:11:30 -05:00
@@ -398,6 +398,26 @@
 	spin_unlock_bh(&xfrm_state_lock);
 }
 
+static void xfrm_state_inbound_accel(struct xfrm_state *x)
+{
+	/* Only called for an inbound xfrm_state. Since it could
+	 * possibly arrive on any interface, try to offload it
+	 * on all devices that are capable.
+	 */
+	struct net_device *dev;
+
+	rtnl_lock();
+	read_lock(&dev_base_lock);
+	dev = dev_base;
+	while (dev) {
+		if (netif_running(dev) && (dev->features & NETIF_F_IPSEC))
+			dev->xfrm_state_add(dev, x);
+		dev = dev->next;
+	}
+	read_unlock(&dev_base_lock);
+	rtnl_unlock();
+}
+
 static struct xfrm_state *__xfrm_find_acq_byseq(u32 seq);
 
 int xfrm_state_add(struct xfrm_state *x)
@@ -444,6 +464,9 @@
 	spin_unlock_bh(&xfrm_state_lock);
 	xfrm_state_put_afinfo(afinfo);
 
+	if (!err && x->dir == XFRM_STATE_DIR_IN)
+		xfrm_state_inbound_accel(x);
+
 	if (x1) {
 		xfrm_state_delete(x1);
 		xfrm_state_put(x1);
@@ -455,7 +478,7 @@
 int xfrm_state_update(struct xfrm_state *x)
 {
 	struct xfrm_state_afinfo *afinfo;
-	struct xfrm_state *x1;
+	struct xfrm_state *x1, *accel = NULL;
 	int err;
 
 	afinfo = xfrm_state_get_afinfo(x->props.family);
@@ -479,6 +502,7 @@
 
 	if (x1->km.state == XFRM_STATE_ACQ) {
 		__xfrm_state_insert(x);
+		accel = x;
 		x = NULL;
 	}
 	err = 0;
@@ -489,6 +513,9 @@
 
 	if (err)
 		return err;
+
+	if (accel && accel->dir == XFRM_STATE_DIR_IN)
+		xfrm_state_inbound_accel(accel);
 
 	if (!x) {
 		xfrm_state_delete(x1);

[RFC 2.6.10 5/22] xfrm: Attempt to offload bundled xfrm_states for outbound xfrms

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:34:46-05:00 dave@thedillows.org 
#   Plumb in offloading new bundles for outgoing packets.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_policy.c
#   2004/12/30 00:34:28-05:00 dave@thedillows.org +28 -0
#   When we create a new bundle for an outbound flow, try to
#   offload as much as the destination driver will allow.
#   
#   Don't forget to clean up....
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/net/xfrm.h
#   2004/12/30 00:34:28-05:00 dave@thedillows.org +6 -0
#   A convenience structure for offloading bundles.
#   
#   The dst->child field gives us a singly linked list
#   from upper protocols to outer transforms. Drivers, however,
#   will likely have a limited number of offloads they can
#   perform on a particular packet, so they need to offload
#   the bundle from the outside in. This list makes it easier
#   for them.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/net/dst.h
#   2004/12/30 00:34:28-05:00 dave@thedillows.org +1 -0
#   Add a field to store the offload information for this part
#   of the outgoing bundle (non-NULL if this dst is offloaded.)
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/net/dst.h b/include/net/dst.h
--- a/include/net/dst.h	2004-12-30 01:11:18 -05:00
+++ b/include/net/dst.h	2004-12-30 01:11:18 -05:00
@@ -65,6 +65,7 @@
 	struct neighbour	*neighbour;
 	struct hh_cache		*hh;
 	struct xfrm_state	*xfrm;
+	struct xfrm_offload	*xfrm_offload;
 
 	int			(*input)(struct sk_buff*);
 	int			(*output)(struct sk_buff*);
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
--- a/include/net/xfrm.h	2004-12-30 01:11:18 -05:00
+++ b/include/net/xfrm.h	2004-12-30 01:11:18 -05:00
@@ -178,6 +178,12 @@
 	atomic_t		refcnt;
 };
 
+struct xfrm_bundle_list
+{
+	struct list_head	node;
+	struct dst_entry *	dst;
+};
+
 struct xfrm_type;
 struct xfrm_dst;
 struct xfrm_policy_afinfo {
diff -Nru a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
--- a/net/xfrm/xfrm_policy.c	2004-12-30 01:11:18 -05:00
+++ b/net/xfrm/xfrm_policy.c	2004-12-30 01:11:18 -05:00
@@ -705,6 +705,31 @@
 	};
 }
 
+static void xfrm_accel_bundle(struct dst_entry *dst)
+{
+	struct xfrm_bundle_list bundle, *xbl, *tmp;
+	struct net_device *dev = dst->dev;
+	INIT_LIST_HEAD(&bundle.node);
+
+	if (dev && netif_running(dev) && (dev->features & NETIF_F_IPSEC)) {
+		while (dst) {
+			xbl = kmalloc(sizeof(*xbl), GFP_ATOMIC);
+			if (!xbl)
+				goto out;
+
+			xbl->dst = dst;
+			list_add_tail(&xbl->node, &bundle.node);
+			dst = dst->child;
+		}
+
+		dev->xfrm_bundle_add(dev, &bundle);
+	}
+
+out:
+	list_for_each_entry_safe(xbl, tmp, &bundle.node, node)
+		kfree(xbl);
+}
+
 static int stale_bundle(struct dst_entry *dst);
 
 /* Main function: finds/creates a bundle for given flow.
@@ -833,6 +858,7 @@
 		policy->bundles = dst;
 		dst_hold(dst);
 		write_unlock_bh(&policy->lock);
+		xfrm_accel_bundle(dst);
 	}
 	*dst_p = dst;
 	dst_release(dst_orig);
@@ -1023,8 +1049,10 @@
 {
 	if (!dst->xfrm)
 		return;
+	xfrm_offload_release(dst->xfrm_offload);
 	xfrm_state_put(dst->xfrm);
 	dst->xfrm = NULL;
+	dst->xfrm_offload = NULL;
 }
 
 static void xfrm_link_failure(struct sk_buff *skb)

[RFC 2.6.10 3/22] xfrm: Add offload management routines

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:31:03-05:00 dave@thedillows.org 
#   Add offload management to xfrm_state.
#   
#   xfrm_offload_alloc() creates a new xfrm_offload, with a private
#   	part to be used by the driver (ala net_device->priv)
#   	The returned offload may be kfree'd if it has not been
#   	added to a xfrm_state using xfrm_state_offload_add().
#   xfrm_offload_priv() returns a pointer to the private area of
#   	the xfrm_offload. This will be 8-byte aligned.
#   xfrm_offload_hold()/xfrm_offload_release() do the reference
#   	counting of the xfrm_offload
#   xfrm_offload_get() looks up the xfrm_offload from a given device
#   	The caller should call xfrm_offload_release() when it is
#   	finished with this offload.
#   xfrm_state_offload_add() adds a new offload to the xfrm_state,
#   	replacing any existing offload for the device that may
#   	exist for this xfrm_state.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_state.c
#   2004/12/30 00:30:46-05:00 dave@thedillows.org +28 -0
#   Clean up any offloads on destruction of an xfrm_state, and
#   allow the addition of new xfrm_offloads to a xfrm_state.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_export.c
#   2004/12/30 00:30:46-05:00 dave@thedillows.org +2 -0
#   Export xfrm_state_offload_add() to drivers.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/net/xfrm.h
#   2004/12/30 00:30:46-05:00 dave@thedillows.org +79 -0
#   Add offload management to xfrm_state.
#   
#   Add xfrm_offload_alloc(), xfrm_offload_priv(), xfrm_offload_hold()
#   xfrm_offload_release(), and xfrm_offload_get().
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
--- a/include/net/xfrm.h	2004-12-30 01:11:43 -05:00
+++ b/include/net/xfrm.h	2004-12-30 01:11:43 -05:00
@@ -81,6 +81,8 @@
       metrics. Plus, it will be made via sk->sk_dst_cache. Solved.
  */
 
+struct xfrm_offload;
+
 /* Full description of state of transformer. */
 struct xfrm_state
 {
@@ -149,6 +151,9 @@
 
 	/* Intended direction of this state, used for offloading */
 	int			dir;
+
+	/* List of offload cookies, per device */
+	struct list_head	offloads;
 };
 
 enum {
@@ -166,6 +171,13 @@
 	XFRM_STATE_DIR_OUT,
 };
 
+struct xfrm_offload
+{
+	struct list_head	bydev;
+	struct net_device *	dev;
+	atomic_t		refcnt;
+};
+
 struct xfrm_type;
 struct xfrm_dst;
 struct xfrm_policy_afinfo {
@@ -911,5 +923,72 @@
 				     (struct in6_addr *)b);
 	}
 }
+
+#define XFRM_OFFLOAD_ALIGN		8
+#define XFRM_OFFLOAD_ALIGN_CONST	(XFRM_OFFLOAD_ALIGN - 1)
+
+static inline struct xfrm_offload *
+xfrm_offload_alloc(int sizeof_priv, struct net_device *dev)
+{
+	struct xfrm_offload *xol;
+	int alloc_size;
+	
+	alloc_size = (sizeof(*xol) + XFRM_OFFLOAD_ALIGN_CONST)
+				& ~XFRM_OFFLOAD_ALIGN_CONST;
+	alloc_size += sizeof_priv;
+	xol = kmalloc(alloc_size, GFP_ATOMIC);
+	if (xol) {
+		memset(xol, 0, alloc_size);
+		INIT_LIST_HEAD(&xol->bydev);
+		atomic_set(&xol->refcnt, 1);
+		xol->dev = dev;
+	}
+
+	return xol;
+}
+
+static inline void *xfrm_offload_priv(struct xfrm_offload *xol)
+{
+	return (char *)xol + ((sizeof(*xol) + XFRM_OFFLOAD_ALIGN_CONST)
+					& ~XFRM_OFFLOAD_ALIGN_CONST);
+}
+
+static inline void xfrm_offload_hold(struct xfrm_offload *xol)
+{
+	atomic_inc(&xol->refcnt);
+}
+
+static inline void xfrm_offload_release(struct xfrm_offload *xol)
+{
+	if (xol) {
+		WARN_ON(atomic_read(&xol->refcnt) < 1);
+		if (atomic_dec_and_test(&xol->refcnt)) {
+			xol->dev->xfrm_state_del(xol->dev, xol);
+			dev_put(xol->dev);
+			kfree(xol);
+		}
+	}
+}
+
+static inline struct xfrm_offload *xfrm_offload_get(struct xfrm_state *x,
+                                               struct net_device *dev)
+{
+	struct xfrm_offload *xol, *ret = NULL;
+
+	spin_lock(&x->lock);
+	list_for_each_entry(xol, &x->offloads, bydev) {
+		if (xol->dev == dev) {
+			xfrm_offload_hold(xol);
+			ret = xol;
+			break;
+		}
+	}
+	spin_unlock(&x->lock);
+
+	return ret;
+}
+
+extern void xfrm_state_offload_add(struct xfrm_state *x,
+					struct xfrm_offload *xol);
 
 #endif	/* _NET_XFRM_H */
diff -Nru a/net/xfrm/xfrm_export.c b/net/xfrm/xfrm_export.c
--- a/net/xfrm/xfrm_export.c	2004-12-30 01:11:43 -05:00
+++ b/net/xfrm/xfrm_export.c	2004-12-30 01:11:43 -05:00
@@ -61,3 +61,5 @@
 EXPORT_SYMBOL_GPL(xfrm_ealg_get_byname);
 EXPORT_SYMBOL_GPL(xfrm_calg_get_byname);
 EXPORT_SYMBOL_GPL(skb_icv_walk);
+
+EXPORT_SYMBOL_GPL(xfrm_state_offload_add);
diff -Nru a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
--- a/net/xfrm/xfrm_state.c	2004-12-30 01:11:43 -05:00
+++ b/net/xfrm/xfrm_state.c	2004-12-30 01:11:43 -05:00
@@ -53,6 +53,12 @@
 
 static void xfrm_state_gc_destroy(struct xfrm_state *x)
 {
+	if (!list_empty(&x->offloads)) {
+		struct xfrm_offload *xol, *next;
+
+		list_for_each_entry_safe(xol, next, &x->offloads, bydev)
+			xfrm_offload_release(xol);
+	}
 	if (del_timer(&x->timer))
 		BUG();
 	if (x->aalg)
@@ -178,6 +184,7 @@
 		atomic_set(&x->tunnel_users, 0);
 		INIT_LIST_HEAD(&x->bydst);
 		INIT_LIST_HEAD(&x->byspi);
+		INIT_LIST_HEAD(&x->offloads);
 		init_timer(&x->timer);
 		x->timer.function = xfrm_timer_handler;
 		x->timer.data	  = (unsigned long)x;
@@ -941,6 +948,27 @@
 		xfrm_state_put(t);
 		x->tunnel = NULL;
 	}
+}
+
+void xfrm_state_offload_add(struct xfrm_state *x, struct xfrm_offload *xol)
+{
+	struct xfrm_offload *entry, *old = NULL;
+
+	spin_lock(&x->lock);
+	list_for_each_entry(entry, &x->offloads, bydev) {
+		if (entry->dev == xol->dev) {
+			list_del(&entry->bydev);
+			old = entry;
+			break;
+		}
+	}
+
+	dev_hold(xol->dev);
+	list_add_tail(&xol->bydev, &x->offloads);
+	spin_unlock(&x->lock);
+
+	if(old)
+		xfrm_offload_release(old);
 }
 
 void __init xfrm_state_init(void)

[RFC 2.6.10 8/22] skbuff: Add routines to manage applied offloads per skb

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:40:20-05:00 dave@thedillows.org 
#   Add fields to sk_buff to track offloaded xfrm_states for this
#   packet. On Tx, these will be pointers to struct xfrm_offload.
#   On Rx, these will be a 4 bit bitfield indicating what operations
#   were performed, and the result of those operations.
#   
#   skb_push_xfrm_offload() records an offloaded xfrm on Tx. It will
#   	return an error code if it is unable to record the offload.
#   skb_get_xfrm_offload() returns the xfrm_offload struct at the
#   	given position on the stack. It will return NULL if there
#   	are no more offloads available.
#   skb_has_xfrm_offload() returns true if the sk_buff has offload
#   	information available.
#   skb_put_xfrm_result() records an offload result on Rx at the given
#   	index. It will return an error code if it is unable to
#   	record the result.
#   skb_pop_xfrm_result() pops the current offload result from the.
#   	stack. If there are no more results, it will return
#   	XFRM_OFFLOAD_NONE.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/core/skbuff.c
#   2004/12/30 00:40:02-05:00 dave@thedillows.org +31 -0
#   When an sk_buff is cloned, we must gain a reference to each
#   xfrm_offload that it references.
#   
#   When an sk_buff is freed, we must release our references
#   to the xfrm_offloads attached to it.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/net/xfrm.h
#   2004/12/30 00:40:02-05:00 dave@thedillows.org +9 -0
#   Add the values for the result bitfield.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/linux/skbuff.h
#   2004/12/30 00:40:02-05:00 dave@thedillows.org +55 -0
#   Add the fields and functins to track offloads and results, as
#   well as the current position in the stack.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/linux/skbuff.h b/include/linux/skbuff.h
--- a/include/linux/skbuff.h	2004-12-30 01:10:39 -05:00
+++ b/include/linux/skbuff.h	2004-12-30 01:10:39 -05:00
@@ -146,6 +146,14 @@
 	skb_frag_t	frags[MAX_SKB_FRAGS];
 };
 
+/* XXX UGH. We cannot include <net/xfrm.h> in this file without some
+ * header file surgery, so define our own max xfrm depth. This should
+ * be kept >= XFRM_MAX_DEPTH until we fix the includes, and it can
+ * go away.
+ */
+#define SKB_XFRM_MAX_DEPTH 4
+struct xfrm_offload;
+
 /** 
  *	struct sk_buff - socket buffer
  *	@next: Next buffer in list
@@ -187,6 +195,8 @@
  *	@nf_bridge: Saved data about a bridged frame - see br_netfilter.c
  *      @private: Data which is private to the HIPPI implementation
  *	@tc_index: Traffic control index
+ *	@xfrm_offload: Tx offload info, Rx offload results
+ *	@xfrm_offload_idx: The number of cookies/results stored currently
  */
 
 struct sk_buff {
@@ -272,6 +282,8 @@
 
 #endif
 
+	int			xfrm_offload_idx;
+	struct xfrm_offload *	xfrm_offload[SKB_XFRM_MAX_DEPTH];
 
 	/* These elements must be at the end, see alloc_skb() for details.  */
 	unsigned int		truesize;
@@ -1178,6 +1190,49 @@
 #else /* CONFIG_NETFILTER */
 static inline void nf_reset(struct sk_buff *skb) {}
 #endif /* CONFIG_NETFILTER */
+
+static inline int skb_push_xfrm_offload(struct sk_buff *skb,
+					struct xfrm_offload *xol)
+{
+	if (likely(skb->xfrm_offload_idx < SKB_XFRM_MAX_DEPTH)) {
+		skb->xfrm_offload[skb->xfrm_offload_idx++] = xol;
+		return 0;
+	}
+
+	return -ENOMEM;
+}
+
+static inline struct xfrm_offload *
+skb_get_xfrm_offload(const struct sk_buff *skb, int idx)
+{
+	if (likely(idx < skb->xfrm_offload_idx))
+		return skb->xfrm_offload[idx];
+	else
+		return NULL;
+}
+
+static inline int skb_has_xfrm_offload(const struct sk_buff *skb)
+{
+	return !!skb_get_xfrm_offload(skb, 0);
+}
+
+static inline int skb_put_xfrm_result(struct sk_buff *skb, int result, int idx)
+{
+	if (likely(idx < SKB_XFRM_MAX_DEPTH)) {
+		skb->xfrm_offload[idx] = (struct xfrm_offload *) result;
+		return 0;
+	}
+	return -ENOMEM;
+}
+
+static inline int skb_pop_xfrm_result(struct sk_buff *skb)
+{
+	/* XXX XFRM_OFFLOAD_NONE == 0, but cannot include <net/xfrm.h> */
+	int res = 0;
+	if (likely(skb->xfrm_offload_idx < SKB_XFRM_MAX_DEPTH))
+		res = (int) skb->xfrm_offload[skb->xfrm_offload_idx++];
+	return res;
+}
 
 #endif	/* __KERNEL__ */
 #endif	/* _LINUX_SKBUFF_H */
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
--- a/include/net/xfrm.h	2004-12-30 01:10:39 -05:00
+++ b/include/net/xfrm.h	2004-12-30 01:10:39 -05:00
@@ -171,6 +171,15 @@
 	XFRM_STATE_DIR_OUT,
 };
 
+enum {
+	XFRM_OFFLOAD_NONE = 0,
+	XFRM_OFFLOAD_CONF = 1,
+	XFRM_OFFLOAD_AUTH = 2,
+	XFRM_OFFLOAD_AUTH_OK = 4,
+	XFRM_OFFLOAD_AUTH_FAIL = 8,
+	XFRM_OFFLOAD_FIELD = 0x0f
+};
+
 struct xfrm_offload
 {
 	struct list_head	bydev;
diff -Nru a/net/core/skbuff.c b/net/core/skbuff.c
--- a/net/core/skbuff.c	2004-12-30 01:10:39 -05:00
+++ b/net/core/skbuff.c	2004-12-30 01:10:39 -05:00
@@ -230,6 +230,14 @@
 
 	dst_release(skb->dst);
 #ifdef CONFIG_XFRM
+	{
+		int i = 0;
+		struct xfrm_offload *xol;
+		while ((xol = skb_get_xfrm_offload(skb, i++)) != NULL) {
+			if ((unsigned long) xol > XFRM_OFFLOAD_FIELD)
+				xfrm_offload_release(xol);
+		}
+	}
 	secpath_put(skb->sp);
 #endif
 	if(skb->destructor) {
@@ -334,6 +342,17 @@
 #endif
 
 #endif
+	C(xfrm_offload_idx);
+	memcpy(n->xfrm_offload, skb->xfrm_offload,
+			sizeof(struct xfrm_offload *) * SKB_XFRM_MAX_DEPTH);
+	{
+		int i = 0;
+		struct xfrm_offload *xol;
+		while ((xol = skb_get_xfrm_offload(skb, i++)) != NULL) {
+			if ((unsigned long) xol > XFRM_OFFLOAD_FIELD)
+				xfrm_offload_hold(xol);
+		}
+	}
 	C(truesize);
 	atomic_set(&n->users, 1);
 	C(head);
@@ -396,6 +415,18 @@
 	atomic_set(&new->users, 1);
 	skb_shinfo(new)->tso_size = skb_shinfo(old)->tso_size;
 	skb_shinfo(new)->tso_segs = skb_shinfo(old)->tso_segs;
+
+	new->xfrm_offload_idx = old->xfrm_offload_idx;
+	memcpy(new->xfrm_offload, old->xfrm_offload,
+			sizeof(struct xfrm_offload *) * SKB_XFRM_MAX_DEPTH);
+	{
+		int i = 0;
+		struct xfrm_offload *xol;
+		while ((xol = skb_get_xfrm_offload(old, i++)) != NULL) {
+			if ((unsigned long) xol > XFRM_OFFLOAD_FIELD)
+				xfrm_offload_hold(xol);
+		}
+	}
 }
 
 /**

[RFC 2.6.10 7/22] xfrm: Allow device drivers to force recalculation of offloads

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:37:44-05:00 dave@thedillows.org 
#   Give device drivers a method to allow the use of crypto
#   offload features for existing xfrm_states and bundles, as
#   well as dynamically remove crypto offload capabilities.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_state.c
#   2004/12/30 00:37:26-05:00 dave@thedillows.org +39 -0
#   When we've been informed of a new device that can offload
#   xfrm crypto operations, go ahead and offload existing inbound
#   xfrm_states to it.
#   
#   When we're removing crypto offload capabilities, remove every
#   offload instance for that device.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_policy.c
#   2004/12/30 00:37:26-05:00 dave@thedillows.org +17 -0
#   When adding/removing xfrm offload capable device, give the xfrm_state
#   engine a chance to make the changes it needs, then flush any existing
#   bundles that use the device so that future flows get a chance to use
#   the offload features (for add), or resume using the software crypto
#   routines (for remove).
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_export.c
#   2004/12/30 00:37:26-05:00 dave@thedillows.org +2 -0
#   Export the driver-visible API.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/net/xfrm.h
#   2004/12/30 00:37:26-05:00 dave@thedillows.org +4 -0
#   Prototypes for the new routines.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
--- a/include/net/xfrm.h	2004-12-30 01:10:52 -05:00
+++ b/include/net/xfrm.h	2004-12-30 01:10:52 -05:00
@@ -833,6 +833,8 @@
 extern struct xfrm_state *xfrm_find_acq_byseq(u32 seq);
 extern void xfrm_state_delete(struct xfrm_state *x);
 extern void xfrm_state_flush(u8 proto);
+extern void xfrm_state_accel_add(struct net_device *dev);
+extern void xfrm_state_accel_flush(struct net_device *dev);
 extern int xfrm_replay_check(struct xfrm_state *x, u32 seq);
 extern void xfrm_replay_advance(struct xfrm_state *x, u32 seq);
 extern int xfrm_state_check(struct xfrm_state *x, struct sk_buff *skb);
@@ -888,6 +890,8 @@
 extern int xfrm_sk_policy_insert(struct sock *sk, int dir, struct xfrm_policy *pol);
 extern struct xfrm_policy *xfrm_sk_policy_lookup(struct sock *sk, int dir, struct flowi *fl);
 extern int xfrm_flush_bundles(void);
+extern void xfrm_accel_add(struct net_device *dev);
+extern void xfrm_accel_flush(struct net_device *dev);
 
 extern wait_queue_head_t km_waitq;
 extern void km_state_expired(struct xfrm_state *x, int hard);
diff -Nru a/net/xfrm/xfrm_export.c b/net/xfrm/xfrm_export.c
--- a/net/xfrm/xfrm_export.c	2004-12-30 01:10:52 -05:00
+++ b/net/xfrm/xfrm_export.c	2004-12-30 01:10:52 -05:00
@@ -63,3 +63,5 @@
 EXPORT_SYMBOL_GPL(skb_icv_walk);
 
 EXPORT_SYMBOL_GPL(xfrm_state_offload_add);
+EXPORT_SYMBOL_GPL(xfrm_accel_add);
+EXPORT_SYMBOL_GPL(xfrm_accel_flush);
diff -Nru a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
--- a/net/xfrm/xfrm_policy.c	2004-12-30 01:10:52 -05:00
+++ b/net/xfrm/xfrm_policy.c	2004-12-30 01:10:52 -05:00
@@ -1121,6 +1121,23 @@
 	return 0;
 }
 
+static int bundle_uses_dev(struct dst_entry *dst, void *dev)
+{
+	return (dst->dev == dev);
+}
+
+void xfrm_accel_add(struct net_device *dev)
+{
+	xfrm_state_accel_add(dev);
+	xfrm_prune_bundles(bundle_uses_dev, dev);
+}
+
+void xfrm_accel_flush(struct net_device *dev)
+{
+	xfrm_state_accel_flush(dev);
+	xfrm_prune_bundles(bundle_uses_dev, dev);
+}
+
 /* Well... that's _TASK_. We need to scan through transformation
  * list and figure out what mss tcp should generate in order to
  * final datagram fit to mtu. Mama mia... :-)
diff -Nru a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
--- a/net/xfrm/xfrm_state.c	2004-12-30 01:10:52 -05:00
+++ b/net/xfrm/xfrm_state.c	2004-12-30 01:10:52 -05:00
@@ -998,6 +998,45 @@
 		xfrm_offload_release(old);
 }
 
+static int try_new_accel(struct xfrm_state *x, int unused, void *data)
+{
+	struct net_device *dev = data;
+
+	if (x->dir == XFRM_STATE_DIR_IN)
+		dev->xfrm_state_add(dev, x);
+	return 0;
+}
+
+static int remove_stale_accel(struct xfrm_state *x, int unused, void *dev)
+{
+	struct xfrm_offload *xol, *entry = NULL;
+
+	spin_lock(&x->lock);
+	list_for_each_entry(xol, &x->offloads, bydev) {
+		if (xol->dev == dev) {
+			list_del(&xol->bydev);
+			entry = xol;
+			break;
+		}
+	}
+	spin_unlock(&x->lock);
+
+	if (entry)
+		xfrm_offload_release(entry);
+
+	return 0;
+}
+
+void xfrm_state_accel_add(struct net_device *dev)
+{
+	xfrm_state_walk(IPSEC_PROTO_ANY, try_new_accel, dev);
+}
+
+void xfrm_state_accel_flush(struct net_device *dev)
+{
+	xfrm_state_walk(IPSEC_PROTO_ANY, remove_stale_accel, dev);
+}
+
 void __init xfrm_state_init(void)
 {
 	int i;

[RFC 2.6.10 6/22] xfrm: add a parameter to xfrm_prune_bundles()

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:35:34-05:00 dave@thedillows.org 
#   Add a parameter to the decision function(s) used by
#   xfrm_prune_bundles(). This will allow us to have more
#   fine grained selection of bundles pruned (like, say,
#   per device.)
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/xfrm/xfrm_policy.c
#   2004/12/30 00:35:16-05:00 dave@thedillows.org +10 -9
#   Add a parameter to the decision function(s) used by
#   xfrm_prune_bundles(). This will allow us to have more
#   fine grained selection of bundles pruned (like, say,
#   per device.)
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
--- a/net/xfrm/xfrm_policy.c	2004-12-30 01:11:05 -05:00
+++ b/net/xfrm/xfrm_policy.c	2004-12-30 01:11:05 -05:00
@@ -730,7 +730,7 @@
 		kfree(xbl);
 }
 
-static int stale_bundle(struct dst_entry *dst);
+static int stale_bundle(struct dst_entry *dst, void *unused);
 
 /* Main function: finds/creates a bundle for given flow.
  *
@@ -841,7 +841,7 @@
 		}
 
 		write_lock_bh(&policy->lock);
-		if (unlikely(policy->dead || stale_bundle(dst))) {
+		if (unlikely(policy->dead || stale_bundle(dst, NULL))) {
 			/* Wow! While we worked on resolving, this
 			 * policy has gone. Retry. It is not paranoia,
 			 * we just cannot enlist new bundle to dead object.
@@ -1022,14 +1022,14 @@
 
 static struct dst_entry *xfrm_dst_check(struct dst_entry *dst, u32 cookie)
 {
-	if (!stale_bundle(dst))
+	if (!stale_bundle(dst, NULL))
 		return dst;
 
 	dst_release(dst);
 	return NULL;
 }
 
-static int stale_bundle(struct dst_entry *dst)
+static int stale_bundle(struct dst_entry *dst, void *unused)
 {
 	struct dst_entry *child = dst;
 
@@ -1072,7 +1072,8 @@
 	return dst;
 }
 
-static void xfrm_prune_bundles(int (*func)(struct dst_entry *))
+static void xfrm_prune_bundles(int (*func)(struct dst_entry *, void *),
+					void *data)
 {
 	int i;
 	struct xfrm_policy *pol;
@@ -1084,7 +1085,7 @@
 			write_lock(&pol->lock);
 			dstp = &pol->bundles;
 			while ((dst=*dstp) != NULL) {
-				if (func(dst)) {
+				if (func(dst, data)) {
 					*dstp = dst->next;
 					dst->next = gc_list;
 					gc_list = dst;
@@ -1104,19 +1105,19 @@
 	}
 }
 
-static int unused_bundle(struct dst_entry *dst)
+static int unused_bundle(struct dst_entry *dst, void *unused)
 {
 	return !atomic_read(&dst->__refcnt);
 }
 
 static void __xfrm_garbage_collect(void)
 {
-	xfrm_prune_bundles(unused_bundle);
+	xfrm_prune_bundles(unused_bundle, NULL);
 }
 
 int xfrm_flush_bundles(void)
 {
-	xfrm_prune_bundles(stale_bundle);
+	xfrm_prune_bundles(stale_bundle, NULL);
 	return 0;
 }
 

[RFC 2.6.10 10/22] AH, ESP: Add offloading of outbound packets

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:44:50-05:00 dave@thedillows.org 
#   Add crypto processing for outbound AH and ESP xfrms (IPv4).
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/ipv4/esp4.c
#   2004/12/30 00:44:32-05:00 dave@thedillows.org +35 -21
#   Add crypto offload for outbound ESP (IPv4) xfrms. Note that we always
#   generate a random IV, as we are not guaranteed to have any state in
#   the software crypto engine (we may have always been offloaded), and
#   we cannot rely on secure IV generation by the NIC driver/hw.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/ipv4/ah4.c
#   2004/12/30 00:44:32-05:00 dave@thedillows.org +31 -21
#   Add crypto offload for outbound AH (IPv4) xfrms. Note that the NIC
#   driver/hw is responsible for zeroing the mutable IP header fields.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/net/ipv4/ah4.c b/net/ipv4/ah4.c
--- a/net/ipv4/ah4.c	2004-12-30 01:10:14 -05:00
+++ b/net/ipv4/ah4.c	2004-12-30 01:10:14 -05:00
@@ -83,31 +83,41 @@
 	ah->spi = x->id.spi;
 	ah->seq_no = htonl(x->replay.oseq + 1);
 
-	iph->tos = top_iph->tos;
-	iph->ttl = top_iph->ttl;
-	iph->frag_off = top_iph->frag_off;
-
-	if (top_iph->ihl != 5) {
-		iph->daddr = top_iph->daddr;
-		memcpy(iph+1, top_iph+1, top_iph->ihl*4 - sizeof(struct iphdr));
-		err = ip_clear_mutable_options(top_iph, &top_iph->daddr);
-		if (err)
+	if (dst->xfrm_offload) {
+		err = -ENOMEM;
+		xfrm_offload_hold(dst->xfrm_offload);
+		if (skb_push_xfrm_offload(skb, dst->xfrm_offload)) {
+			xfrm_offload_release(dst->xfrm_offload);
 			goto error;
-	}
+		}
+	} else {
+		/* Not offloaded, manually calculate the auth hash */
+		iph->tos = top_iph->tos;
+		iph->ttl = top_iph->ttl;
+		iph->frag_off = top_iph->frag_off;
+
+		if (top_iph->ihl != 5) {
+			iph->daddr = top_iph->daddr;
+			memcpy(iph+1, top_iph+1, top_iph->ihl*4 - sizeof(struct iphdr));
+			err = ip_clear_mutable_options(top_iph, &top_iph->daddr);
+			if (err)
+				goto error;
+		}
 
-	top_iph->tos = 0;
-	top_iph->frag_off = 0;
-	top_iph->ttl = 0;
-	top_iph->check = 0;
+		top_iph->tos = 0;
+		top_iph->frag_off = 0;
+		top_iph->ttl = 0;
+		top_iph->check = 0;
 
-	ahp->icv(ahp, skb, ah->auth_data);
+		ahp->icv(ahp, skb, ah->auth_data);
 
-	top_iph->tos = iph->tos;
-	top_iph->ttl = iph->ttl;
-	top_iph->frag_off = iph->frag_off;
-	if (top_iph->ihl != 5) {
-		top_iph->daddr = iph->daddr;
-		memcpy(top_iph+1, iph+1, top_iph->ihl*4 - sizeof(struct iphdr));
+		top_iph->tos = iph->tos;
+		top_iph->ttl = iph->ttl;
+		top_iph->frag_off = iph->frag_off;
+		if (top_iph->ihl != 5) {
+			top_iph->daddr = iph->daddr;
+			memcpy(top_iph+1, iph+1, top_iph->ihl*4 - sizeof(struct iphdr));
+		}
 	}
 
 	/* Delay incrementing the replay sequence until we know we're going
diff -Nru a/net/ipv4/esp4.c b/net/ipv4/esp4.c
--- a/net/ipv4/esp4.c	2004-12-30 01:10:14 -05:00
+++ b/net/ipv4/esp4.c	2004-12-30 01:10:14 -05:00
@@ -98,33 +98,47 @@
 	esph->spi = x->id.spi;
 	esph->seq_no = htonl(++x->replay.oseq);
 
-	if (esp->conf.ivlen)
-		crypto_cipher_set_iv(tfm, esp->conf.ivec, crypto_tfm_alg_ivsize(tfm));
+	if (dst->xfrm_offload) {
+		xfrm_offload_hold(dst->xfrm_offload);
+		if (skb_push_xfrm_offload(skb, dst->xfrm_offload)) {
+			xfrm_offload_release(dst->xfrm_offload);
+			goto error;
+		}
+
+		if (esp->conf.ivlen)
+			get_random_bytes(esph->enc_data, esp->conf.ivlen);
+	} else {
+		if (esp->conf.ivlen)
+			crypto_cipher_set_iv(tfm, esp->conf.ivec, crypto_tfm_alg_ivsize(tfm));
+
+		do {
+			struct scatterlist *sg = &esp->sgbuf[0];
 
-	do {
-		struct scatterlist *sg = &esp->sgbuf[0];
+			if (unlikely(nfrags > ESP_NUM_FAST_SG)) {
+				sg = kmalloc(sizeof(struct scatterlist)*nfrags, GFP_ATOMIC);
+				if (!sg)
+					goto error;
+			}
+			skb_to_sgvec(skb, sg, esph->enc_data+esp->conf.ivlen-skb->data, clen);
+			crypto_cipher_encrypt(tfm, sg, sg, clen);
+			if (unlikely(sg != &esp->sgbuf[0]))
+				kfree(sg);
+		} while (0);
 
-		if (unlikely(nfrags > ESP_NUM_FAST_SG)) {
-			sg = kmalloc(sizeof(struct scatterlist)*nfrags, GFP_ATOMIC);
-			if (!sg)
-				goto error;
+		if (esp->conf.ivlen) {
+			memcpy(esph->enc_data, esp->conf.ivec, crypto_tfm_alg_ivsize(tfm));
+			crypto_cipher_get_iv(tfm, esp->conf.ivec, crypto_tfm_alg_ivsize(tfm));
+		}
+
+		if (esp->auth.icv_full_len) {
+			esp->auth.icv(esp, skb, (u8*)esph-skb->data,
+		              	sizeof(struct ip_esp_hdr) + esp->conf.ivlen+clen, trailer->tail);
 		}
-		skb_to_sgvec(skb, sg, esph->enc_data+esp->conf.ivlen-skb->data, clen);
-		crypto_cipher_encrypt(tfm, sg, sg, clen);
-		if (unlikely(sg != &esp->sgbuf[0]))
-			kfree(sg);
-	} while (0);
-
-	if (esp->conf.ivlen) {
-		memcpy(esph->enc_data, esp->conf.ivec, crypto_tfm_alg_ivsize(tfm));
-		crypto_cipher_get_iv(tfm, esp->conf.ivec, crypto_tfm_alg_ivsize(tfm));
 	}
 
-	if (esp->auth.icv_full_len) {
-		esp->auth.icv(esp, skb, (u8*)esph-skb->data,
-		              sizeof(struct ip_esp_hdr) + esp->conf.ivlen+clen, trailer->tail);
+	/* Need to account for auth data, offloading or not... */
+	if (esp->auth.icv_full_len)
 		pskb_put(skb, trailer, alen);
-	}
 
 	ip_send_check(top_iph);
 

[RFC 2.6.10 14/22] typhoon: add inbound offload result processing

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:54:09-05:00 dave@thedillows.org 
#   Add inbound packet crypto result processing to the Typhoon driver.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 00:53:50-05:00 dave@thedillows.org +42 -0
#   Add inbound packet crypto result processing to the Typhoon driver.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:09:23 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:09:23 -05:00
@@ -131,6 +131,7 @@
 #include <asm/checksum.h>
 #include <linux/version.h>
 #include <linux/dma-mapping.h>
+#include <net/xfrm.h>
 
 #include "typhoon.h"
 #include "typhoon-firmware.h"
@@ -1681,6 +1682,43 @@
 	return 0;
 }
 
+static inline void
+typhoon_ipsec_rx(struct sk_buff *skb, u16 results)
+{
+#define CHECK_OFFLOAD(good, bad) \
+	do { if(results & (good|bad)) { \
+		unsigned int tmp = XFRM_OFFLOAD_CONF | XFRM_OFFLOAD_AUTH; \
+		tmp |= (results & good) ?  XFRM_OFFLOAD_AUTH_OK : \
+					   XFRM_OFFLOAD_AUTH_FAIL; \
+		if(skb_put_xfrm_result(skb, tmp, i)) \
+				return; \
+		i++; \
+	} } while(0)
+
+	/* We have no way to determine what the order of the SAs were on
+	 * the wire, just the 1st AH seen, the 1st ESP seen, etc.
+	 *
+	 * We just walk the stack, and pretend that AH SAs get decypted
+	 * so that if we get the order wrong, the worst case scenerio is
+	 * that we indicate the failure on the wrong SA, since we'll need
+	 * to match all SAs against the policy.
+	 *
+	 * We get a "ESP good" indication for null auth hash on ESP.
+	 */
+	/* XXX think more about security indications -- can I craft a
+	 * packet to do bad things -- maybe a NULL auth ESP packet,
+	 * and a failed AH packet?
+	 */
+	int i = 0;
+
+	CHECK_OFFLOAD(TYPHOON_RX_AH1_GOOD, TYPHOON_RX_AH1_FAIL);
+	CHECK_OFFLOAD(TYPHOON_RX_ESP1_GOOD, TYPHOON_RX_ESP1_FAIL);
+	CHECK_OFFLOAD(TYPHOON_RX_AH2_GOOD, TYPHOON_RX_AH2_FAIL);
+	CHECK_OFFLOAD(TYPHOON_RX_ESP2_GOOD, TYPHOON_RX_ESP2_FAIL);
+
+#undef CHECK_OFFLOAD
+}
+
 static int
 typhoon_rx(struct typhoon *tp, struct basic_ring *rxRing, volatile u32 * ready,
 	   volatile u32 * cleared, int budget)
@@ -1745,6 +1783,10 @@
 			new_skb->ip_summed = CHECKSUM_UNNECESSARY;
 		} else
 			new_skb->ip_summed = CHECKSUM_NONE;
+
+		if((rx->rxStatus & TYPHOON_RX_IPSEC) &&
+				!(rx->rxStatus & TYPHOON_RX_IP_FRAG))
+			typhoon_ipsec_rx(new_skb, rx->ipsecResults);
 
 		spin_lock(&tp->state_lock);
 		if(tp->vlgrp != NULL && rx->rxStatus & TYPHOON_RX_VLAN)

[RFC 2.6.10 12/22] ethtool: Add support for crypto offload

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:51:19-05:00 dave@thedillows.org 
#   Add support for querying and changing the status of the
#   IPSEC crypto offload feature of a NIC.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/core/ethtool.c
#   2004/12/30 00:51:00-05:00 dave@thedillows.org +54 -0
#   Add support for querying and changing the status of the IPSEC
#   crypto offload feature of a NIC.
#   
#   Turn on/off the feature flag before informing the xfrm engine
#   of the change so that existing xfrms get the new settings.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# include/linux/ethtool.h
#   2004/12/30 00:51:00-05:00 dave@thedillows.org +8 -0
#   Add support for querying and changing the status of the
#   IPSEC crypto offload feature of a NIC.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/include/linux/ethtool.h b/include/linux/ethtool.h
--- a/include/linux/ethtool.h	2004-12-30 01:09:49 -05:00
+++ b/include/linux/ethtool.h	2004-12-30 01:09:49 -05:00
@@ -260,6 +260,8 @@
 int ethtool_op_set_sg(struct net_device *dev, u32 data);
 u32 ethtool_op_get_tso(struct net_device *dev);
 int ethtool_op_set_tso(struct net_device *dev, u32 data);
+u32 ethtool_op_get_ipsec(struct net_device *dev);
+int ethtool_op_set_ipsec(struct net_device *dev, u32 data);
 
 /**
  * &ethtool_ops - Alter and report network device settings
@@ -293,6 +295,8 @@
  * get_strings: Return a set of strings that describe the requested objects 
  * phys_id: Identify the device
  * get_stats: Return statistics about the device
+ * get_ipsec: Report whether IPSEC crypto offload is enabled
+ * set_ipsec: Turn IPSEC crypto offload on or off
  *
  * Description:
  *
@@ -345,6 +349,8 @@
 	int	(*set_sg)(struct net_device *, u32);
 	u32	(*get_tso)(struct net_device *);
 	int	(*set_tso)(struct net_device *, u32);
+	u32	(*get_ipsec)(struct net_device *);
+	int	(*set_ipsec)(struct net_device *, u32);
 	int	(*self_test_count)(struct net_device *);
 	void	(*self_test)(struct net_device *, struct ethtool_test *, u64 *);
 	void	(*get_strings)(struct net_device *, u32 stringset, u8 *);
@@ -388,6 +394,8 @@
 #define ETHTOOL_GSTATS		0x0000001d /* get NIC-specific statistics */
 #define ETHTOOL_GTSO		0x0000001e /* Get TSO enable (ethtool_value) */
 #define ETHTOOL_STSO		0x0000001f /* Set TSO enable (ethtool_value) */
+#define ETHTOOL_GIPSEC		0x00000020 /* Get IPSEC enable (ethtool_value) */
+#define ETHTOOL_SIPSEC		0x00000021 /* Set IPSEC enable (ethtool_value) */
 
 /* compatibility with older code */
 #define SPARC_ETH_GSET		ETHTOOL_GSET
diff -Nru a/net/core/ethtool.c b/net/core/ethtool.c
--- a/net/core/ethtool.c	2004-12-30 01:09:49 -05:00
+++ b/net/core/ethtool.c	2004-12-30 01:09:49 -05:00
@@ -14,6 +14,7 @@
 #include <linux/errno.h>
 #include <linux/ethtool.h>
 #include <linux/netdevice.h>
+#include <net/xfrm.h>
 #include <asm/uaccess.h>
 
 /* 
@@ -72,6 +73,24 @@
 	return 0;
 }
 
+u32 ethtool_op_get_ipsec(struct net_device *dev)
+{
+	return (dev->features & NETIF_F_IPSEC) != 0;
+}
+
+int ethtool_op_set_ipsec(struct net_device *dev, u32 data)
+{
+	if (data) {
+		dev->features |= NETIF_F_IPSEC;
+		xfrm_accel_add(dev);
+	} else {
+		dev->features &= ~NETIF_F_IPSEC;
+		xfrm_accel_flush(dev);
+	}
+
+	return 0;
+}
+
 /* Handlers for each ethtool command */
 
 static int ethtool_get_settings(struct net_device *dev, void __user *useraddr)
@@ -548,6 +567,33 @@
 	return dev->ethtool_ops->set_tso(dev, edata.data);
 }
 
+static int ethtool_get_ipsec(struct net_device *dev, char __user *useraddr)
+{
+	struct ethtool_value edata = { ETHTOOL_GIPSEC };
+
+	if (!dev->ethtool_ops->get_ipsec)
+		return -EOPNOTSUPP;
+
+	edata.data = dev->ethtool_ops->get_ipsec(dev);
+
+	if (copy_to_user(useraddr, &edata, sizeof(edata)))
+		return -EFAULT;
+	return 0;
+}
+
+static int ethtool_set_ipsec(struct net_device *dev, char __user *useraddr)
+{
+	struct ethtool_value edata;
+
+	if (!dev->ethtool_ops->set_ipsec)
+		return -EOPNOTSUPP;
+
+	if (copy_from_user(&edata, useraddr, sizeof(edata)))
+		return -EFAULT;
+
+	return dev->ethtool_ops->set_ipsec(dev, edata.data);
+}
+
 static int ethtool_self_test(struct net_device *dev, char __user *useraddr)
 {
 	struct ethtool_test test;
@@ -783,6 +829,12 @@
 	case ETHTOOL_STSO:
 		rc = ethtool_set_tso(dev, useraddr);
 		break;
+	case ETHTOOL_GIPSEC:
+		rc = ethtool_get_ipsec(dev, useraddr);
+		break;
+	case ETHTOOL_SIPSEC:
+		rc = ethtool_set_ipsec(dev, useraddr);
+		break;
 	case ETHTOOL_TEST:
 		rc = ethtool_self_test(dev, useraddr);
 		break;
@@ -813,7 +865,9 @@
 EXPORT_SYMBOL(ethtool_op_get_link);
 EXPORT_SYMBOL(ethtool_op_get_sg);
 EXPORT_SYMBOL(ethtool_op_get_tso);
+EXPORT_SYMBOL(ethtool_op_get_ipsec);
 EXPORT_SYMBOL(ethtool_op_get_tx_csum);
 EXPORT_SYMBOL(ethtool_op_set_sg);
 EXPORT_SYMBOL(ethtool_op_set_tso);
+EXPORT_SYMBOL(ethtool_op_set_ipsec);
 EXPORT_SYMBOL(ethtool_op_set_tx_csum);

[RFC 2.6.10 15/22] typhoon: add outbound offload processing

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:55:05-05:00 dave@thedillows.org 
#   Add outbound xfrm crypto offload processing to the packet path.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 00:54:46-05:00 dave@thedillows.org +130 -0
#   Add outbound xfrm crypto offload processing to the packet path.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:09:10 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:09:10 -05:00
@@ -352,6 +352,15 @@
 #define TSO_OFFLOAD_ON		0
 #endif
 
+#define IPSEC_NUM_DESCRIPTORS	1
+
+struct typhoon_xfrm_offload {
+	u16	sa_cookie;
+	u16	tunnel:1,
+		ah:1,
+		inbound:1;
+};
+
 static inline void
 typhoon_inc_index(u32 *index, const int count, const int num_entries)
 {
@@ -779,12 +788,115 @@
 	tcpd->status = 0;
 }
 
+static inline int
+typhoon_ipsec_fill(struct typhoon *tp, struct sk_buff *skb,
+				struct transmit_ring *txRing)
+{
+	struct xfrm_offload *xol;
+	struct typhoon_xfrm_offload *txo;
+	struct ipsec_desc *ipsec;
+	int last_was_esp = 0;
+	int i, entry;
+	u32 sa[3];
+
+	ipsec = (struct ipsec_desc *) (txRing->ringBase + txRing->lastWrite);
+	typhoon_inc_tx_index(&txRing->lastWrite, 1);
+
+	ipsec->flags = TYPHOON_OPT_DESC | TYPHOON_OPT_IPSEC;
+	ipsec->numDesc = 1;
+	ipsec->ipsecFlags = TYPHOON_IPSEC_USE_IV;
+	ipsec->reserved = 0;
+	sa[0] = sa[1] = sa[2] = 0;
+
+	/* Fill the offload descriptor with the cookies to indicate
+	 * which key set to use when. While we're looping through the
+	 * offloaded xfrms, if the last xfrm was ESP, and we're doing
+	 * AH now, * then we can move the ESP part to the top of the
+	 * descriptor. Otherwise, we'll need to move to the next one.
+	 * We overrun into sa[2] to prevent needing to check the entry
+	 * limit in the middile of things.
+	 */
+	entry = i = 0;
+	xol = skb_get_xfrm_offload(skb, i++);
+	while(xol && entry < 2) {
+		xfrm_offload_hold(xol);
+		txo = xfrm_offload_priv(xol);
+		if(sa[entry] && txo->tunnel)
+			entry++;
+		if(sa[entry] & 0xffff) {
+			if(last_was_esp && txo->ah)
+				sa[entry] <<= 16;
+			else
+				entry++;
+		}
+
+		sa[entry] |= txo->sa_cookie;
+		last_was_esp = !txo->ah;
+
+		xol = skb_get_xfrm_offload(skb, i++);
+	}
+
+	/* Make sure we used all of the xfrms that were offloaded.
+	 */
+	if(unlikely(entry == 2 && xol)) {
+		if(net_ratelimit())
+			printk(KERN_ERR "%s: failing to offload IPSEC packet "
+					"with too many xfrms!\n", tp->name);
+		goto bad_packet;
+	}
+
+	ipsec->sa[0] = cpu_to_le16(sa[0] & 0xffff);
+	ipsec->sa[1] = cpu_to_le16(sa[0] >> 16);
+	ipsec->sa[2] = cpu_to_le16(sa[1] & 0xffff);
+	ipsec->sa[3] = cpu_to_le16(sa[1] >> 16);
+
+	/* The current 3XP firmware seems to hang if we try to feed it
+	 * the same (non-zero) SA twice on the same packet. So, detect
+	 * and drop those packets as it is likely a stack bug, or
+	 * misconfiguration of policy.
+	 *
+	 * I.e., we should never hit this.
+	 */
+	if(unlikely(ipsec->sa[2])) {
+		if(unlikely(ipsec->sa[2] == ipsec->sa[3]))
+			goto avoiding_sa_hang;
+		if(unlikely(ipsec->sa[2] == ipsec->sa[0] ||
+					ipsec->sa[2] == ipsec->sa[1]))
+			goto avoiding_sa_hang;
+		if(unlikely(ipsec->sa[3] && (ipsec->sa[3] == ipsec->sa[0] ||
+					ipsec->sa[3] == ipsec->sa[1])))
+			goto avoiding_sa_hang;
+	}
+
+	if(unlikely(ipsec->sa[1] && ipsec->sa[0] == ipsec->sa[1]))
+		goto avoiding_sa_hang;
+
+	return 0;
+
+avoiding_sa_hang:
+	if(net_ratelimit())
+		printk(KERN_ERR "%s: failing attempted IPSEC offload with "
+				"duplicate SAs %08x %08x\n", tp->name,
+				sa[0], sa[1]);
+
+bad_packet:
+	/* Any xfrm_offloads we've attached to this skb will be
+	 * released for us when typhoon_start_tx() calls dev_kfree_skb_any()
+	 * on it.
+	 *
+	 * Return an error to indicate this packet cannot be offloaded as
+	 * specified and should never make it to the wire.
+	 */
+	return -EINVAL;
+}
+
 static int
 typhoon_start_tx(struct sk_buff *skb, struct net_device *dev)
 {
 	struct typhoon *tp = netdev_priv(dev);
 	struct transmit_ring *txRing;
 	struct tx_desc *txd, *first_txd;
+	u32 origLastWrite;
 	dma_addr_t skb_dma;
 	int numDesc;
 
@@ -811,6 +923,9 @@
 	if(skb_tso_size(skb))
 		numDesc++;
 
+	if(skb_has_xfrm_offload(skb))
+		numDesc++;
+
 	/* When checking for free space in the ring, we need to also
 	 * account for the initial Tx descriptor, and we always must leave
 	 * at least one descriptor unused in the ring so that it doesn't
@@ -823,6 +938,7 @@
 	while(unlikely(typhoon_num_free_tx(txRing) < (numDesc + 2)))
 		smp_rmb();
 
+	origLastWrite = txRing->lastWrite;
 	first_txd = (struct tx_desc *) (txRing->ringBase + txRing->lastWrite);
 	typhoon_inc_tx_index(&txRing->lastWrite, 1);
 
@@ -855,6 +971,14 @@
 		typhoon_tso_fill(skb, txRing, tp->txlo_dma_addr);
 	}
 
+	if(skb_has_xfrm_offload(skb)) {
+		first_txd->processFlags |= TYPHOON_TX_PF_IPSEC;
+		first_txd->numDesc++;
+
+		if(typhoon_ipsec_fill(tp, skb, txRing))
+			goto error;
+	}
+
 	txd = (struct tx_desc *) (txRing->ringBase + txRing->lastWrite);
 	typhoon_inc_tx_index(&txRing->lastWrite, 1);
 
@@ -915,6 +1039,7 @@
 	 * Tx header.
 	 */
 	numDesc = MAX_SKB_FRAGS + TSO_NUM_DESCRIPTORS + 1;
+	numDesc += IPSEC_NUM_DESCRIPTORS;
 
 	if(typhoon_num_free_tx(txRing) < (numDesc + 2)) {
 		netif_stop_queue(dev);
@@ -927,6 +1052,11 @@
 			netif_wake_queue(dev);
 	}
 
+	return 0;
+
+error:
+	txRing->lastWrite = origLastWrite;
+	dev_kfree_skb_any(skb);
 	return 0;
 }
 

[RFC 2.6.10 9/22] AH: Split header initialization from zeroing of mutable fields

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:42:33-05:00 dave@thedillows.org 
#   Seperate AH header initialization from the zeroing of mutable
#   IP header fields in preparation for offloading the crypto
#   processing of the packet.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/ipv4/ah4.c
#   2004/12/30 00:42:15-05:00 dave@thedillows.org +18 -12
#   Seperate AH header initialization from the zeroing of mutable
#   IP header fields in preparation for offloading the crypto
#   processing of the packet.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/net/ipv4/ah4.c b/net/ipv4/ah4.c
--- a/net/ipv4/ah4.c	2004-12-30 01:10:27 -05:00
+++ b/net/ipv4/ah4.c	2004-12-30 01:10:27 -05:00
@@ -69,6 +69,20 @@
 	top_iph = skb->nh.iph;
 	iph = &tmp_iph.iph;
 
+	ah = (struct ip_auth_hdr *)((char *)top_iph+top_iph->ihl*4);
+	ah->nexthdr = top_iph->protocol;
+
+	top_iph->tot_len = htons(skb->len);
+	top_iph->protocol = IPPROTO_AH;
+
+	ahp = x->data;
+	ah->hdrlen  = (XFRM_ALIGN8(sizeof(struct ip_auth_hdr) + 
+				   ahp->icv_trunc_len) >> 2) - 2;
+
+	ah->reserved = 0;
+	ah->spi = x->id.spi;
+	ah->seq_no = htonl(x->replay.oseq + 1);
+
 	iph->tos = top_iph->tos;
 	iph->ttl = top_iph->ttl;
 	iph->frag_off = top_iph->frag_off;
@@ -81,23 +95,11 @@
 			goto error;
 	}
 
-	ah = (struct ip_auth_hdr *)((char *)top_iph+top_iph->ihl*4);
-	ah->nexthdr = top_iph->protocol;
-
 	top_iph->tos = 0;
-	top_iph->tot_len = htons(skb->len);
 	top_iph->frag_off = 0;
 	top_iph->ttl = 0;
-	top_iph->protocol = IPPROTO_AH;
 	top_iph->check = 0;
 
-	ahp = x->data;
-	ah->hdrlen  = (XFRM_ALIGN8(sizeof(struct ip_auth_hdr) + 
-				   ahp->icv_trunc_len) >> 2) - 2;
-
-	ah->reserved = 0;
-	ah->spi = x->id.spi;
-	ah->seq_no = htonl(++x->replay.oseq);
 	ahp->icv(ahp, skb, ah->auth_data);
 
 	top_iph->tos = iph->tos;
@@ -108,6 +110,10 @@
 		memcpy(top_iph+1, iph+1, top_iph->ihl*4 - sizeof(struct iphdr));
 	}
 
+	/* Delay incrementing the replay sequence until we know we're going
+	 * to send this packet to prevent gaps.
+	 */
+	x->replay.oseq++;
 	ip_send_check(top_iph);
 
 	err = 0;

[RFC 2.6.10 11/22] AH, ESP: Add offloading of inbound packets

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:47:54-05:00 dave@thedillows.org 
#   Add crypto offload for inbound IPv4 AH xfrms.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/ipv4/esp4.c
#   2004/12/30 00:47:36-05:00 dave@thedillows.org +30 -16
#   Add crypto offload for inbound IPv4 AH xfrms.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# net/ipv4/ah4.c
#   2004/12/30 00:47:36-05:00 dave@thedillows.org +13 -4
#   Add crypto offload for inbound IPv4 AH xfrms.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/net/ipv4/ah4.c b/net/ipv4/ah4.c
--- a/net/ipv4/ah4.c	2004-12-30 01:10:02 -05:00
+++ b/net/ipv4/ah4.c	2004-12-30 01:10:02 -05:00
@@ -138,6 +138,7 @@
 	struct iphdr *iph;
 	struct ip_auth_hdr *ah;
 	struct ah_data *ahp;
+	int offload;
 	char work_buf[60];
 
 	if (!pskb_may_pull(skb, sizeof(struct ip_auth_hdr)))
@@ -164,6 +165,7 @@
 
 	ah = (struct ip_auth_hdr*)skb->data;
 	iph = skb->nh.iph;
+	offload = skb_pop_xfrm_result(skb);
 
 	memcpy(work_buf, iph, iph->ihl*4);
 
@@ -181,10 +183,17 @@
 		
 		memcpy(auth_data, ah->auth_data, ahp->icv_trunc_len);
 		skb_push(skb, skb->data - skb->nh.raw);
-		ahp->icv(ahp, skb, ah->auth_data);
-		if (memcmp(ah->auth_data, auth_data, ahp->icv_trunc_len)) {
-			x->stats.integrity_failed++;
-			goto out;
+		if (offload & XFRM_OFFLOAD_AUTH) {
+			if (unlikely(offload & XFRM_OFFLOAD_AUTH_FAIL)) {
+				x->stats.integrity_failed++;
+				goto out;
+			}
+		} else {
+			ahp->icv(ahp, skb, ah->auth_data);
+			if (memcmp(ah->auth_data, auth_data, ahp->icv_trunc_len)) {
+				x->stats.integrity_failed++;
+				goto out;
+			}
 		}
 	}
 	((struct iphdr*)work_buf)->protocol = ah->nexthdr;
diff -Nru a/net/ipv4/esp4.c b/net/ipv4/esp4.c
--- a/net/ipv4/esp4.c	2004-12-30 01:10:02 -05:00
+++ b/net/ipv4/esp4.c	2004-12-30 01:10:02 -05:00
@@ -164,6 +164,7 @@
 	int elen = skb->len - sizeof(struct ip_esp_hdr) - esp->conf.ivlen - alen;
 	int nfrags;
 	int encap_len = 0;
+	int offload;
 
 	if (!pskb_may_pull(skb, sizeof(struct ip_esp_hdr)))
 		goto out;
@@ -171,22 +172,32 @@
 	if (elen <= 0 || (elen & (blksize-1)))
 		goto out;
 
+	offload = skb_pop_xfrm_result(skb);
+
 	/* If integrity check is required, do this. */
 	if (esp->auth.icv_full_len) {
-		u8 sum[esp->auth.icv_full_len];
-		u8 sum1[alen];
+		if (unlikely(offload & XFRM_OFFLOAD_AUTH_FAIL)) {
+			x->stats.integrity_failed++;
+			goto out;
+		}
+
+		if (!(offload & XFRM_OFFLOAD_AUTH)) {
+			u8 sum[esp->auth.icv_full_len];
+			u8 sum1[alen];
 		
-		esp->auth.icv(esp, skb, 0, skb->len-alen, sum);
+			esp->auth.icv(esp, skb, 0, skb->len-alen, sum);
 
-		if (skb_copy_bits(skb, skb->len-alen, sum1, alen))
-			BUG();
+			if (skb_copy_bits(skb, skb->len-alen, sum1, alen))
+				BUG();
 
-		if (unlikely(memcmp(sum, sum1, alen))) {
-			x->stats.integrity_failed++;
-			goto out;
+			if (unlikely(memcmp(sum, sum1, alen))) {
+				x->stats.integrity_failed++;
+				goto out;
+			}
 		}
 	}
 
+	/* XXX I think this can be moved to the !offload case */
 	if ((nfrags = skb_cow_data(skb, 0, &trailer)) < 0)
 		goto out;
 
@@ -195,15 +206,12 @@
 	esph = (struct ip_esp_hdr*)skb->data;
 	iph = skb->nh.iph;
 
-	/* Get ivec. This can be wrong, check against another impls. */
-	if (esp->conf.ivlen)
-		crypto_cipher_set_iv(esp->conf.tfm, esph->enc_data, crypto_tfm_alg_ivsize(esp->conf.tfm));
-
-        {
-		u8 nexthdr[2];
+	if (!(offload & XFRM_OFFLOAD_CONF)) {
 		struct scatterlist *sg = &esp->sgbuf[0];
-		u8 workbuf[60];
-		int padlen;
+
+		/* Get ivec. This can be wrong, check against another impls. */
+		if (esp->conf.ivlen)
+			crypto_cipher_set_iv(esp->conf.tfm, esph->enc_data, crypto_tfm_alg_ivsize(esp->conf.tfm));
 
 		if (unlikely(nfrags > ESP_NUM_FAST_SG)) {
 			sg = kmalloc(sizeof(struct scatterlist)*nfrags, GFP_ATOMIC);
@@ -214,6 +222,12 @@
 		crypto_cipher_decrypt(esp->conf.tfm, sg, sg, elen);
 		if (unlikely(sg != &esp->sgbuf[0]))
 			kfree(sg);
+	}
+
+        {
+		u8 nexthdr[2];
+		u8 workbuf[60];
+		int padlen;
 
 		if (skb_copy_bits(skb, skb->len-alen-2, nexthdr, 2))
 			BUG();

[RFC 2.6.10 13/22] typhoon: Make the ipsec descriptor match actual usage

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:53:15-05:00 dave@thedillows.org 
#   Make the crypto structures better match actual usage.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.h
#   2004/12/30 00:52:57-05:00 dave@thedillows.org +13 -11
#   Make the crypto structures better match actual usage.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.h b/drivers/net/typhoon.h
--- a/drivers/net/typhoon.h	2004-12-30 01:09:36 -05:00
+++ b/drivers/net/typhoon.h	2004-12-30 01:09:36 -05:00
@@ -210,7 +210,10 @@
  * flags:	descriptor type
  * numDesc:	must be 1
  * ipsecFlags:	bit 0: 0 -- generate IV, 1 -- use supplied IV
- * sa1, sa2:	Security Association IDs for this packet
+ * sa[0]:	the inner AH header offload cookie (or ESP if no AH)
+ * sa[1]:	the inner ESP header offload cookie (or 0 if no AH)
+ * sa[2]:	the outer AH header offload cookie (or ESP if no AH)
+ * sa[3]:	the outer ESP header offload cookie (or 0 if no AH)
  * reserved:	set to 0
  */
 struct ipsec_desc {
@@ -219,8 +222,7 @@
 	u16 ipsecFlags;
 #define TYPHOON_IPSEC_GEN_IV	__constant_cpu_to_le16(0x0000)
 #define TYPHOON_IPSEC_USE_IV	__constant_cpu_to_le16(0x0001)
-	u32 sa1;
-	u32 sa2;
+	u16 sa[4];
 	u32 reserved;
 } __attribute__ ((packed));
 
@@ -268,14 +270,14 @@
 #define TYPHOON_RX_FILTER_MASK		__constant_cpu_to_le16(0x7fff)
 #define TYPHOON_RX_FILTERED		__constant_cpu_to_le16(0x8000)
 	u16 ipsecResults;
-#define TYPHOON_RX_OUTER_AH_GOOD	__constant_cpu_to_le16(0x0001)
-#define TYPHOON_RX_OUTER_ESP_GOOD	__constant_cpu_to_le16(0x0002)
-#define TYPHOON_RX_INNER_AH_GOOD	__constant_cpu_to_le16(0x0004)
-#define TYPHOON_RX_INNER_ESP_GOOD	__constant_cpu_to_le16(0x0008)
-#define TYPHOON_RX_OUTER_AH_FAIL	__constant_cpu_to_le16(0x0010)
-#define TYPHOON_RX_OUTER_ESP_FAIL	__constant_cpu_to_le16(0x0020)
-#define TYPHOON_RX_INNER_AH_FAIL	__constant_cpu_to_le16(0x0040)
-#define TYPHOON_RX_INNER_ESP_FAIL	__constant_cpu_to_le16(0x0080)
+#define TYPHOON_RX_AH1_GOOD		__constant_cpu_to_le16(0x0001)
+#define TYPHOON_RX_ESP1_GOOD		__constant_cpu_to_le16(0x0002)
+#define TYPHOON_RX_AH2_GOOD		__constant_cpu_to_le16(0x0004)
+#define TYPHOON_RX_ESP2_GOOD		__constant_cpu_to_le16(0x0008)
+#define TYPHOON_RX_AH1_FAIL		__constant_cpu_to_le16(0x0010)
+#define TYPHOON_RX_ESP1_FAIL		__constant_cpu_to_le16(0x0020)
+#define TYPHOON_RX_AH2_FAIL		__constant_cpu_to_le16(0x0040)
+#define TYPHOON_RX_ESP2_FAIL		__constant_cpu_to_le16(0x0080)
 #define TYPHOON_RX_UNKNOWN_SA		__constant_cpu_to_le16(0x0100)
 #define TYPHOON_RX_ESP_FORMAT_ERR	__constant_cpu_to_le16(0x0200)
 	u32 vlanTag;

[RFC 2.6.10 17/22] typhoon: split out setting of offloaded tasks

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:57:35-05:00 dave@thedillows.org 
#   Move the setting of the currently offloaded tasks to its own
#   function, as we'll be making use of it to change the crypto
#   offload status when adding/removing xfrms.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 00:57:17-05:00 dave@thedillows.org +26 -15
#   Move the setting of the currently offloaded tasks to its own
#   function, as we'll be making use of it to change the crypto
#   offload status when adding/removing xfrms.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:08:45 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:08:45 -05:00
@@ -304,6 +304,7 @@
 	u16			xcvr_select;
 	u16			wol_events;
 	u32			offload;
+	spinlock_t		offload_lock;
 
 	u16			tx_sa_max;
 	u16			rx_sa_max;
@@ -725,11 +726,28 @@
 	return err;
 }
 
+static int
+typhoon_set_offload(struct typhoon *tp)
+{
+	/* Caller should hold tp->offload_lock, or otherwise guarantee
+         * exclusitivity to this routine.
+         */
+	struct cmd_desc xp_cmd;
+
+	smp_rmb();
+	if(tp->card_state != Running)
+		return 0;
+
+	INIT_COMMAND_WITH_RESPONSE(&xp_cmd, TYPHOON_CMD_SET_OFFLOAD_TASKS);
+	xp_cmd.parm2 = tp->offload;
+	xp_cmd.parm3 = tp->offload;
+	return typhoon_issue_command(tp, 1, &xp_cmd, 0, NULL);
+}
+
 static void
 typhoon_vlan_rx_register(struct net_device *dev, struct vlan_group *grp)
 {
 	struct typhoon *tp = netdev_priv(dev);
-	struct cmd_desc xp_cmd;
 	int err;
 
 	spin_lock_bh(&tp->state_lock);
@@ -737,25 +755,16 @@
 		/* We've either been turned on for the first time, or we've
 		 * been turned off. Update the 3XP.
 		 */
+		spin_lock_bh(&tp->offload_lock);
 		if(grp)
 			tp->offload |= TYPHOON_OFFLOAD_VLAN;
 		else
 			tp->offload &= ~TYPHOON_OFFLOAD_VLAN;
+		err = typhoon_set_offload(tp);
+		spin_unlock_bh(&tp->offload_lock);
 
-		/* If the interface is up, the runtime is running -- and we
-		 * must be up for the vlan core to call us.
-		 *
-		 * Do the command outside of the spin lock, as it is slow.
-		 */
-		INIT_COMMAND_WITH_RESPONSE(&xp_cmd,
-					TYPHOON_CMD_SET_OFFLOAD_TASKS);
-		xp_cmd.parm2 = tp->offload;
-		xp_cmd.parm3 = tp->offload;
-		spin_unlock_bh(&tp->state_lock);
-		err = typhoon_issue_command(tp, 1, &xp_cmd, 0, NULL);
 		if(err < 0)
 			printk("%s: vlan offload error %d\n", tp->name, -err);
-		spin_lock_bh(&tp->state_lock);
 	}
 
 	/* now make the change visible */
@@ -1486,6 +1495,7 @@
 
 	spin_lock_init(&tp->command_lock);
 	spin_lock_init(&tp->state_lock);
+	spin_lock_init(&tp->offload_lock);
 }
 
 static void
@@ -2218,12 +2228,13 @@
 	if(err < 0)
 		goto error_out;
 
+	/* tp->card_state != Running, so nothing will change this out
+	 * from under us.
+	 */
 	INIT_COMMAND_NO_RESPONSE(&xp_cmd, TYPHOON_CMD_SET_OFFLOAD_TASKS);
-	spin_lock_bh(&tp->state_lock);
 	xp_cmd.parm2 = tp->offload;
 	xp_cmd.parm3 = tp->offload;
 	err = typhoon_issue_command(tp, 1, &xp_cmd, 0, NULL);
-	spin_unlock_bh(&tp->state_lock);
 	if(err < 0)
 		goto error_out;
 

[RFC 2.6.10 19/22] typhoon: add loading of xfrm_states to hardware

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 01:02:32-05:00 dave@thedillows.org 
#   Teach the Typhoon driver how to add and remove xfrm_states to
#   the 3XP for later packet processing.
#   
#   When the first xfrm_state is added, we turn on IPSEC offloads
#   for the 3XP, and we turn it off when the last one is removed.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 01:02:14-05:00 dave@thedillows.org +167 -0
#   Teach the Typhoon driver how to add and remove xfrm_states to
#   the 3XP for later packet processing.
#   
#   When the first xfrm_state is added, we turn on IPSEC offloads
#   for the 3XP, and we turn it off when the last one is removed.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:08:19 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:08:19 -05:00
@@ -2420,6 +2420,173 @@
 #undef REQUIRED
 #undef UNSUPPORTED
 
+static struct xfrm_offload *
+typhoon_offload_ipsec(struct typhoon *tp, struct xfrm_state *x)
+{
+	struct cmd_desc xp_cmd[5];
+	struct resp_desc xp_resp;
+	struct sa_descriptor *sa = (struct sa_descriptor *)xp_cmd;
+	struct xfrm_offload *xol;
+	struct typhoon_xfrm_offload *txo;
+	u16 *dir_sa_avail = &tp->rx_sa_avail;
+	u16 cookie;
+	int keylen, err;
+
+	if(!typhoon_validate_xfrm(tp, x))
+		goto error;
+
+	memset(xp_cmd, 0, 5 * sizeof(xp_cmd[0]));
+	INIT_COMMAND_WITH_RESPONSE(xp_cmd, TYPHOON_CMD_CREATE_SA);
+	sa->numDesc = 4;
+
+	sa->mode = TYPHOON_SA_MODE_AH;
+	if(x->type->proto == IPPROTO_ESP)
+		sa->mode = TYPHOON_SA_MODE_ESP;
+
+	if(x->dir == XFRM_STATE_DIR_OUT) {
+		sa->direction = TYPHOON_SA_DIR_TX;
+		dir_sa_avail = &tp->tx_sa_avail;
+	}
+
+	spin_lock_bh(&tp->offload_lock);
+	if(!*dir_sa_avail) {
+		spin_unlock_bh(&tp->offload_lock);
+		goto error;
+	}
+	*dir_sa_avail--;
+	if(!tp->sa_count++) {
+		tp->offload |= TYPHOON_OFFLOAD_IPSEC;
+		err = typhoon_set_offload(tp);
+		if(err < 0) {
+			spin_unlock_bh(&tp->offload_lock);
+			printk(KERN_ERR "%s: unable to enable IPSEC "
+					"offload (%d)\n", tp->name, -err);
+			goto error_counted;
+		}
+	}
+	spin_unlock_bh(&tp->offload_lock);
+
+	if(x->props.aalgo != SADB_X_AALG_NULL && x->aalg) {
+		keylen = (x->aalg->alg_key_len + 7) / 8;
+
+		sa->hashFlags = TYPHOON_SA_HASH_SHA1;
+		if(x->props.aalgo == SADB_AALG_MD5HMAC)
+			sa->hashFlags = TYPHOON_SA_HASH_MD5;
+		sa->hashFlags |= TYPHOON_SA_HASH_ENABLE;
+
+		memcpy(sa->integKey, x->aalg->alg_key, keylen);
+	}
+
+	if(x->props.ealgo != SADB_EALG_NULL && x->ealg) {
+		keylen = (x->ealg->alg_key_len + 7) / 8;
+
+		sa->encryptionFlags = TYPHOON_SA_ENCRYPT_ENABLE |
+						TYPHOON_SA_ENCRYPT_CBC;
+		if(x->props.ealgo == SADB_EALG_DESCBC)
+			sa->encryptionFlags |= TYPHOON_SA_ENCRYPT_DES;
+		else if(x->ealg->alg_key_len == 192)
+			sa->encryptionFlags |= TYPHOON_SA_ENCRYPT_3DES_3KEY;
+		else {
+			sa->encryptionFlags |= TYPHOON_SA_ENCRYPT_3DES_2KEY;
+			memcpy(&sa->confKey[16], x->ealg->alg_key, 8);
+		}
+
+		memcpy(sa->confKey, x->ealg->alg_key, keylen);
+	}
+
+	/* The 3XP expects the SPI to be in host order, litte endian.
+	 * It expects the address to be in network order.
+	 */
+	sa->SPI = cpu_to_le32(ntohl(x->id.spi));
+	sa->destAddr = x->id.daddr.a4;
+	sa->destMask = (u32) ~0UL;
+
+	err = typhoon_issue_command(tp, 5, xp_cmd, 1, &xp_resp);
+	cookie = le16_to_cpu(xp_resp.parm1);
+	if(err < 0 || !cookie || cookie == 0xffff)
+		goto error_counted;
+
+	xol = xfrm_offload_alloc(sizeof(*txo), tp->dev);
+	if(!xol)
+		goto error_cookie;
+
+	txo = xfrm_offload_priv(xol);
+	txo->sa_cookie = cookie;
+	txo->tunnel = !!x->props.mode;
+	txo->ah = (x->id.proto == IPPROTO_AH);
+	txo->inbound = (x->dir == XFRM_STATE_DIR_IN);
+
+	xfrm_state_offload_add(x, xol);
+
+	return xol;
+
+error_cookie:
+	INIT_COMMAND_NO_RESPONSE(xp_cmd, TYPHOON_CMD_DELETE_SA);
+	xp_cmd[0].parm1 = xp_resp.parm1;
+	typhoon_issue_command(tp, 1, xp_cmd, 0, NULL);
+
+error_counted:
+	spin_lock_bh(&tp->offload_lock);
+	*dir_sa_avail++;
+	tp->sa_count--;
+	if(!tp->sa_count) {
+		tp->offload &= ~TYPHOON_OFFLOAD_IPSEC;
+		err = typhoon_set_offload(tp);
+		if(err < 0)
+			printk(KERN_ERR "%s: unable to disable IPSEC "
+					"offload (%d)\n", tp->name, -err);
+	}
+	spin_unlock_bh(&tp->offload_lock);
+
+error:
+	return NULL;
+}
+
+static void
+typhoon_xfrm_state_add(struct net_device *dev, struct xfrm_state *x)
+{
+	struct typhoon *tp = netdev_priv(dev);
+
+	smp_rmb();
+	if(tp->card_state == Running)
+		typhoon_offload_ipsec(tp, x);
+}
+
+static void
+typhoon_xfrm_state_del(struct net_device *dev, struct xfrm_offload *xol)
+{
+	struct typhoon *tp = netdev_priv(dev);
+	struct typhoon_xfrm_offload *txo = xfrm_offload_priv(xol);
+	struct cmd_desc xp_cmd;
+	int err;
+
+	smp_rmb();
+	if(tp->card_state != Running)
+		return;
+
+	INIT_COMMAND_NO_RESPONSE(&xp_cmd, TYPHOON_CMD_DELETE_SA);
+	xp_cmd.parm1 = cpu_to_le16(txo->sa_cookie);
+	if(typhoon_issue_command(tp, 1, &xp_cmd, 0, NULL) < 0) {
+		printk(KERN_ERR "%s: unable to remove offloaded SA 0x%04x\n",
+				tp->name,  txo->sa_cookie);
+	}
+
+	spin_lock_bh(&tp->offload_lock);
+	if(txo->inbound)
+		tp->rx_sa_avail++;
+	else
+		tp->tx_sa_avail++;
+	tp->sa_count--;
+	if(!tp->sa_count) {
+		tp->offload &= ~TYPHOON_OFFLOAD_IPSEC;
+		err = typhoon_set_offload(tp);
+		if(err < 0)
+			printk(KERN_ERR "%s: unable to disable IPSEC "
+					"offload (%d)\n", tp->name, -err);
+	}
+	spin_unlock_bh(&tp->offload_lock);
+}
+
 static void
 typhoon_tx_timeout(struct net_device *dev)
 {

[RFC 2.6.10 21/22] typhoon: add callbacks to support crypto offload

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 01:05:45-05:00 dave@thedillows.org 
#   Export the xfrm offload callbacks, and let the world know we
#   support IPSEC offload.
#   
#   While we're at it, allow this to controlled by ethtool.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 01:05:27-05:00 dave@thedillows.org +23 -4
#   Export the xfrm offload callbacks, and let the world know we
#   support IPSEC offload.
#   
#   While we're at it, allow this to controlled by ethtool.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:07:53 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:07:53 -05:00
@@ -33,9 +33,12 @@
 	*) Waiting for a command response takes 8ms due to non-preemptable
 		polling. Only significant for getting stats and creating
 		SAs, but an ugly wart never the less.
+	*) Inbound IPSEC packets of the form outer ESP transport, inner
+		ESP tunnel seems to fail the hash on the inner ESP
+	*) Inbound IPSEC packets of the form outer AH transport, inner
+		AH tunnel seems to fail the hash on the outer AH
 
 	TODO:
-	*) Doesn't do IPSEC offloading. Yet. Keep yer pants on, it's coming.
 	*) Add more support for ethtool (especially for NIC stats)
 	*) Allow disabling of RX checksum offloading
 	*) Fix MAC changing to work while the interface is up
@@ -100,8 +103,8 @@
 #define PKT_BUF_SZ		1536
 
 #define DRV_MODULE_NAME		"typhoon"
-#define DRV_MODULE_VERSION 	"1.5.6"
-#define DRV_MODULE_RELDATE	"04/12/17"
+#define DRV_MODULE_VERSION 	"1.5.6-ipsec"
+#define DRV_MODULE_RELDATE	"04/12/19"
 #define PFX			DRV_MODULE_NAME ": "
 #define ERR_PFX			KERN_ERR PFX
 
@@ -1406,6 +1409,8 @@
 	.get_tso		= ethtool_op_get_tso,
 	.set_tso		= ethtool_op_set_tso,
 	.get_ringparam		= typhoon_get_ringparam,
+	.get_ipsec		= ethtool_op_get_ipsec,
+	.set_ipsec		= ethtool_op_set_ipsec,
 };
 
 static int
@@ -2253,6 +2258,9 @@
 	tp->card_state = Running;
 	smp_wmb();
 
+	if(dev->features & NETIF_F_IPSEC)
+		xfrm_accel_add(dev);
+
 	iowrite32(TYPHOON_INTR_ENABLE_ALL, ioaddr + TYPHOON_REG_INTR_ENABLE);
 	iowrite32(TYPHOON_INTR_NONE, ioaddr + TYPHOON_REG_INTR_MASK);
 	typhoon_post_pci_writes(ioaddr);
@@ -2327,6 +2335,14 @@
 		typhoon_clean_tx(tp, &tp->txLoRing, &indexes->txLoCleared);
 	}
 
+	if(tp->dev->features & NETIF_F_IPSEC)
+		xfrm_accel_flush(tp->dev);
+
+	/* tp->card_state != Running, so nothing will change this out
+         * from under us.
+         */
+	tp->offload &= ~TYPHOON_OFFLOAD_IPSEC;
+
 	return 0;
 }
 
@@ -3183,6 +3199,9 @@
 	dev->set_mac_address	= typhoon_set_mac_address;
 	dev->vlan_rx_register	= typhoon_vlan_rx_register;
 	dev->vlan_rx_kill_vid	= typhoon_vlan_rx_kill_vid;
+	dev->xfrm_state_add	= typhoon_xfrm_state_add;
+	dev->xfrm_state_del	= typhoon_xfrm_state_del;
+	dev->xfrm_bundle_add	= typhoon_xfrm_bundle_add;
 	SET_ETHTOOL_OPS(dev, &typhoon_ethtool_ops);
 
 	/* We can handle scatter gather, up to 16 entries, and
@@ -3190,7 +3209,7 @@
 	 */
 	dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM;
 	dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
-	dev->features |= NETIF_F_TSO;
+	dev->features |= NETIF_F_TSO | NETIF_F_IPSEC;
 
 	if(register_netdev(dev) < 0)
 		goto error_out_reset;

[RFC 2.6.10 18/22] typhoon: add validation of offloaded xfrm_states

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 01:00:43-05:00 dave@thedillows.org 
#   Add routines to validate that the xfrm_state passed to them is
#   one that we can offload to the 3XP.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 01:00:25-05:00 dave@thedillows.org +90 -0
#   Add routines to validate that the xfrm_state passed to them is
#   one that we can offload to the 3XP.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:08:32 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:08:32 -05:00
@@ -2330,6 +2330,96 @@
 	return 0;
 }
 
+#define UNSUPPORTED	goto unsupported
+#define REQUIRED(x)	if(!(x)) goto unsupported
+
+static inline int
+typhoon_validate_ealgo(struct typhoon *tp, struct xfrm_state *x)
+{
+	switch(x->props.ealgo) {
+	case SADB_EALG_NULL:
+		break;
+	case SADB_EALG_DESCBC:
+		REQUIRED(x->ealg);
+		REQUIRED(tp->capabilities & TYPHOON_CRYPTO_DES);
+		REQUIRED(x->ealg->alg_key_len == 64);
+		break;
+	case SADB_EALG_3DESCBC:
+		REQUIRED(x->ealg);
+		REQUIRED(tp->capabilities & TYPHOON_CRYPTO_3DES);
+		REQUIRED(x->ealg->alg_key_len == 128 ||
+					x->ealg->alg_key_len == 192);
+		break;
+	default:
+		UNSUPPORTED;
+	}
+
+	return 1;
+
+unsupported:
+	return 0;
+}
+
+static inline int
+typhoon_validate_aalgo(struct typhoon *tp, struct xfrm_state *x)
+{
+	switch(x->props.aalgo) {
+	case SADB_X_AALG_NULL:
+		break;
+	case SADB_AALG_MD5HMAC:
+		REQUIRED(x->aalg);
+		REQUIRED(x->aalg->alg_key_len == 128);
+		break;
+	case SADB_AALG_SHA1HMAC:
+		REQUIRED(x->aalg);
+		REQUIRED(x->aalg->alg_key_len == 160);
+		break;
+	default:
+		UNSUPPORTED;
+	}
+
+	return 1;
+
+unsupported:
+	return 0;
+}
+
+static inline int
+typhoon_validate_xfrm(struct typhoon *tp, struct xfrm_state *x)
+{
+	u8 ealgo, aalgo, need_auth = 1;
+
+	REQUIRED(x->props.family == AF_INET);
+	REQUIRED(x->dir == XFRM_STATE_DIR_OUT || x->dir == XFRM_STATE_DIR_IN);
+	REQUIRED(!x->encap);
+
+	aalgo = x->props.aalgo;
+	ealgo = x->props.ealgo;
+
+	switch(x->type->proto) {
+	case IPPROTO_ESP:
+		need_auth = 0;
+		REQUIRED(aalgo != SADB_X_AALG_NULL || ealgo != SADB_EALG_NULL);
+		REQUIRED(typhoon_validate_ealgo(tp, x));
+		/* fall through to validate auth algorithm */
+	case IPPROTO_AH:
+		REQUIRED(typhoon_validate_aalgo(tp, x));
+		if(need_auth)
+			REQUIRED(aalgo != SADB_X_AALG_NULL);
+		break;
+	default:
+		UNSUPPORTED;
+	}
+
+	return 1;
+
+unsupported:
+	return 0;
+}
+
+#undef REQUIRED
+#undef UNSUPPORTED
+
 static void
 typhoon_tx_timeout(struct net_device *dev)
 {

[RFC 2.6.10 16/22] typhoon: collect crypto offload capabilities

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 00:56:40-05:00 dave@thedillows.org 
#   Collect some information about the Typhoon's offload capabilities,
#   and store it for future use.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.h
#   2004/12/30 00:56:22-05:00 dave@thedillows.org +14 -0
#   Add the reply message format for the crypto capability query command.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 00:56:22-05:00 dave@thedillows.org +56 -0
#   Collect some information about the Typhoon's offload capabilities,
#   and store it for future use.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:08:58 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:08:58 -05:00
@@ -305,6 +305,12 @@
 	u16			wol_events;
 	u32			offload;
 
+	u16			tx_sa_max;
+	u16			rx_sa_max;
+	u16			tx_sa_avail;
+	u16			rx_sa_avail;
+	int			sa_count;
+
 	/* unused stuff (future use) */
 	int			capabilities;
 	struct transmit_ring 	txHiRing;
@@ -2105,6 +2111,53 @@
 	return 0;
 }
 
+static inline int
+typhoon_ipsec_init(struct typhoon *tp)
+{
+	struct cmd_desc xp_cmd;
+	struct resp_desc xp_resp;
+	struct ipsec_info_desc *info = (struct ipsec_info_desc *) &xp_resp;
+	u16 last_tx, last_rx, last_cap;
+	int err;
+
+	last_tx = tp->tx_sa_max;
+	last_rx = tp->rx_sa_max;
+	last_cap = tp->capabilities;
+
+	INIT_COMMAND_WITH_RESPONSE(&xp_cmd, TYPHOON_CMD_READ_IPSEC_INFO);
+	err = typhoon_issue_command(tp, 1, &xp_cmd, 1, &xp_resp);
+	if(err < 0)
+		goto out;
+
+	/* We're not up yet, so no need to lock this -- we cannot modify
+	 * these fields yet.
+	 */
+	tp->tx_sa_avail = tp->tx_sa_max = le16_to_cpu(info->tx_sa_max);
+	tp->rx_sa_avail = tp->rx_sa_max = le16_to_cpu(info->rx_sa_max);
+	tp->sa_count = 0;
+
+	/* Typhoon2 was originally going to have variable crypto capabilities,
+	 * subject to registration with 3Com. It appears they have decided
+	 * to just enable 3DES as well.
+	 */
+	if(tp->capabilities & TYPHOON_CRYPTO_VARIABLE) {
+		tp->capabilities &= ~TYPHOON_CRYPTO_VARIABLE;
+		tp->capabilities |= TYPHOON_CRYPTO_DES | TYPHOON_CRYPTO_3DES;
+	}
+
+	if(last_tx != tp->tx_sa_max || last_rx != tp->rx_sa_max ||
+					last_cap != tp->capabilities) {
+		printk(KERN_INFO "%s: IPSEC offload %s%s %d Tx %d Rx\n",
+			tp->name,
+			tp->capabilities & TYPHOON_CRYPTO_DES ? "DES " : "",
+			tp->capabilities & TYPHOON_CRYPTO_3DES ? "3DES" : "",
+			tp->tx_sa_max, tp->rx_sa_max);
+	}
+
+out:
+	return err;
+}
+
 static int
 typhoon_start_runtime(struct typhoon *tp)
 {
@@ -2127,6 +2180,9 @@
 		err = -EIO;
 		goto error_out;
 	}
+
+	if(typhoon_ipsec_init(tp))
+		goto error_out;
 
 	INIT_COMMAND_NO_RESPONSE(&xp_cmd, TYPHOON_CMD_SET_MAX_PKT_SIZE);
 	xp_cmd.parm1 = cpu_to_le16(PKT_BUF_SZ);
diff -Nru a/drivers/net/typhoon.h b/drivers/net/typhoon.h
--- a/drivers/net/typhoon.h	2004-12-30 01:08:58 -05:00
+++ b/drivers/net/typhoon.h	2004-12-30 01:08:58 -05:00
@@ -487,6 +487,20 @@
 	u32 unused2;
 } __attribute__ ((packed));
 
+/* TYPHOON_CMD_READ_IPSEC_INFO response descriptor
+ */
+struct ipsec_info_desc {
+	u8  flags;
+	u8  numDesc;
+	u16 cmd;
+	u16 seqNo;
+	u16 des_enabled;
+	u16 tx_sa_max;
+	u16 rx_sa_max;
+	u16 tx_sa_count;
+	u16 rx_sa_count;
+} __attribute__ ((packed));
+
 /* TYPHOON_CMD_SET_OFFLOAD_TASKS bits (cmd.parm2 (Tx) & cmd.parm3 (Rx))
  * This is all for IPv4.
  */

[RFC 2.6.10 20/22] typhoon: add management of outbound bundles

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 01:04:39-05:00 dave@thedillows.org 
#   Add the offloading of outbound bundles.
#   
#   This is a tricky business, because there are restrictions on
#   the types and order of the xfrms we can offload. Some combinations
#   also yield incorrect results, so we have to reduce the amount of
#   offloading we do in those cases.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# drivers/net/typhoon.c
#   2004/12/30 01:04:20-05:00 dave@thedillows.org +134 -0
#   Add the offloading of outbound bundles.
#   
#   This is a tricky business, because there are restrictions on
#   the types and order of the xfrms we can offload. Some combinations
#   also yield incorrect results, so we have to reduce the amount of
#   offloading we do in those cases.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
--- a/drivers/net/typhoon.c	2004-12-30 01:08:06 -05:00
+++ b/drivers/net/typhoon.c	2004-12-30 01:08:06 -05:00
@@ -2587,6 +2587,140 @@
 	spin_unlock_bh(&tp->offload_lock);
 }
 
+static inline int
+typhoon_max_offload(struct xfrm_bundle_list *xbl)
+{
+	/* Pre-scan the bundle to avoid offloading problematic sequences.
+	 * Only reduces the offload level to keep as much advantage as
+	 * possible.
+	 *
+	 * For 03.001.002 -- still problematic for 03.001.008, but need
+	 *	re-verify symptoms.
+	 *
+	 * inner AH tunnel, outer AH transport
+	 *	--> 3XP seems to put the inner hash at the wrong location
+	 * inner AH tunnel, outer ESP tunnel
+	 *	--> 3XP corrupts outer hash, maybe wrong place?
+	 * inner AH transport, outer ESP tunnel
+	 *	--> 3XP seems to encrypt the wrong portion of the packet
+	 * inner ESP transport, outer AH tunnel
+	 *	--> 3XP lockup, requires reset
+	 */
+	struct xfrm_bundle_list *bundle;
+	struct dst_entry *dst;
+	struct xfrm_state *x;
+	int last_was_ah = 0, last_was_tunnel = 0;
+	int max_level = 2;
+	int proto;
+
+	list_for_each_entry_reverse(bundle, &xbl->node, node) {
+		dst = bundle->dst;
+		x = dst->xfrm;
+
+		proto = x ? x->type->proto : IPPROTO_IP;
+
+		if(proto == IPPROTO_AH && x->props.mode &&
+					(last_was_ah ^ last_was_tunnel))
+			goto problem_offload;
+
+		if(proto == IPPROTO_AH && !x->props.mode &&
+					(!last_was_ah && last_was_tunnel))
+			goto problem_offload;
+
+		if(proto == IPPROTO_ESP && last_was_ah && last_was_tunnel)
+			goto problem_offload;
+
+		last_was_ah = (proto == IPPROTO_AH) ? 1 : 0;
+		last_was_tunnel = x ? x->props.mode : 0;
+		continue;
+
+problem_offload:
+		max_level--;
+		break;
+	}
+
+	return max_level;
+}
+
+static void
+typhoon_xfrm_bundle_add(struct net_device *dev, struct xfrm_bundle_list *xbl)
+{
+	/* Walk from the outermost dst back up the chain, offloading
+	 * until we hit something we cannot deal with.
+	 */
+	struct typhoon *tp = netdev_priv(dev);
+	struct xfrm_bundle_list *bundle;
+	struct dst_entry *dst;
+	struct xfrm_state *x;
+	struct xfrm_offload *xol;
+	struct typhoon_xfrm_offload *txo;
+	int proto;
+	int level = 0, max_level;
+	int last = -1;
+
+	smp_rmb();
+	if(tp->card_state != Running)
+		return;
+
+	max_level = typhoon_max_offload(xbl);
+
+	list_for_each_entry_reverse(bundle, &xbl->node, node) {
+		dst = bundle->dst;
+		x = dst->xfrm;
+
+		/* Only support IPv4 */
+		if(dst->ops->family != AF_INET)
+			goto cannot_offload;
+
+		proto = x ? x->type->proto : IPPROTO_IP;
+
+		switch(proto) {
+		case IPPROTO_IP:
+		case IPPROTO_IPIP:
+			if(last == IPPROTO_IP || last == IPPROTO_IPIP)
+				goto cannot_offload;
+			if(level)
+				level++;
+			last = proto;
+			continue;
+		case IPPROTO_ESP:
+			if(last != IPPROTO_AH)
+				level++;
+			break;
+		case IPPROTO_AH:
+			level++;
+			break;
+		default:
+			goto cannot_offload;
+		}
+
+		last = proto;
+		if(level > max_level)
+			goto cannot_offload;
+
+		if(dst->xfrm_offload)
+			continue;
+
+		xol = xfrm_offload_get(x, dev);
+		if(!xol) {
+			xol = typhoon_offload_ipsec(tp, x);
+			if(xol)
+				xfrm_offload_hold(xol);
+		}
+
+		if(!xol)
+			goto cannot_offload;
+
+		dst->xfrm_offload = xol;
+		txo = xfrm_offload_priv(xol);
+		if(txo->tunnel)
+			last = IPPROTO_IPIP;
+	}
+
+cannot_offload:
+	return;
+}
+
 static void
 typhoon_tx_timeout(struct net_device *dev)
 {

[RFC 2.6.10 22/22] Add some documentation for the IPSEC crypto offload

[David Dillow]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/12/30 01:06:37-05:00 dave@thedillows.org 
#   Add some documentation for the IPSEC crypto offload.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
# Documentation/networking/netdevices.txt
#   2004/12/30 01:06:19-05:00 dave@thedillows.org +16 -0
#   Add some documentation for the IPSEC crypto offload.
#   
#   Signed-off-by: David Dillow <dave@thedillows.org>
# 
diff -Nru a/Documentation/networking/netdevices.txt b/Documentation/networking/netdevices.txt
--- a/Documentation/networking/netdevices.txt	2004-12-30 01:07:40 -05:00
+++ b/Documentation/networking/netdevices.txt	2004-12-30 01:07:40 -05:00
@@ -73,3 +73,19 @@
 		dev_close code and comments in net/core/dev.c for more info.
 	Context: softirq
 
+dev->xfrm_state_add:
+	Synchronization: None, but can be called inside dev_base_lock rwlock
+	Context: nominally process, but don't sleep inside an rwlock
+	Notes: Only called for inbound xfrm_state(s). Can be invoked during
+       		xfrm_accel_add() call.
+
+dev->xfrm_state_del:
+	Synchronization: None, but can be called inside dev->xmit_lock spinlock.
+	Context: BHs disabled/softirq
+	Notes: Called for all offloaded xfrm_state(s). Can be invoked during
+		xfrm_accel_flush() call.
+
+dev->xfrm_bundle_add:
+	Synchronization: None
+	Context: softirq/process
+	Notes: Called for newly created outbound xfrm bundles.

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

On Thu, 2004-12-30 03:48:34 -0500, David Dillow <dave@thedillows.org>
wrote in message <20041230035000.10@ori.thedillows.org>:
> diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
> --- a/include/net/xfrm.h	2004-12-30 01:12:08 -05:00
> +++ b/include/net/xfrm.h	2004-12-30 01:12:08 -05:00
> @@ -146,6 +146,9 @@
>  	/* Private data of this transformer, format is opaque,
>  	 * interpreted by xfrm_type methods. */
>  	void			*data;
> +
> +	/* Intended direction of this state, used for offloading */
> +	int			dir;
>  };
>  
>  enum {
> @@ -157,6 +160,12 @@
>  	XFRM_STATE_DEAD
>  };
>  
> +enum {
> +	XFRM_STATE_DIR_UNKNOWN,
> +	XFRM_STATE_DIR_IN,
> +	XFRM_STATE_DIR_OUT,
> +};

Any specific reason to first define such a nice enum and then using int
in the struct?

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[Ingo Oeser]

Hi David,

I'm happy to see a framework and example driver for this.

David Dillow schrieb:
> diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
> --- a/include/net/xfrm.h 2004-12-30 01:12:08 -05:00
> +++ b/include/net/xfrm.h 2004-12-30 01:12:08 -05:00
> @@ -194,6 +203,7 @@
>   struct xfrm_state *(*find_acq)(u8 mode, u32 reqid, u8 proto,
>            xfrm_address_t *daddr, xfrm_address_t *saddr,
>            int create);
> + void   (*map_direction)(struct xfrm_state *xfrm);
>  };
>

Please don't build modifiers, but build functions instead.

e.g. 

xfrm->direction = map_direction(xfrm)

That way you don't hide the assignment and thus code becomes much clearer and
can be called multiple times without risk.


Regards

Ingo Oeser

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[Dave Dillow]

On Thu, 2004-12-30 at 04:48, Jan-Benedict Glaw wrote:
> On Thu, 2004-12-30 03:48:34 -0500, David Dillow <dave@thedillows.org>
> wrote in message <20041230035000.10@ori.thedillows.org>:
> > diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
> > --- a/include/net/xfrm.h	2004-12-30 01:12:08 -05:00
> > +++ b/include/net/xfrm.h	2004-12-30 01:12:08 -05:00
> > @@ -146,6 +146,9 @@
> >  	/* Private data of this transformer, format is opaque,
> >  	 * interpreted by xfrm_type methods. */
> >  	void			*data;
> > +
> > +	/* Intended direction of this state, used for offloading */
> > +	int			dir;
> >  };
> >  
> >  enum {
> > @@ -157,6 +160,12 @@
> >  	XFRM_STATE_DEAD
> >  };
> >  
> > +enum {
> > +	XFRM_STATE_DIR_UNKNOWN,
> > +	XFRM_STATE_DIR_IN,
> > +	XFRM_STATE_DIR_OUT,
> > +};
> 
> Any specific reason to first define such a nice enum and then using int
> in the struct?

Just following the current style in net/xfrm.h, see xfrm_state.km.state
and XFRM_STATE_*.

Though, I probably should have used a u8; easily changed if it is an
issue.
-- 
Dave Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[Dave Dillow]

[readding netdev to the cc list]

On Thu, 2004-12-30 at 08:36, Ingo Oeser wrote:
> Hi David,
> 
> I'm happy to see a framework and example driver for this.

Thanks, I'm just glad it works.

> David Dillow schrieb:
> > diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
> > --- a/include/net/xfrm.h 2004-12-30 01:12:08 -05:00
> > +++ b/include/net/xfrm.h 2004-12-30 01:12:08 -05:00
> > @@ -194,6 +203,7 @@
> >   struct xfrm_state *(*find_acq)(u8 mode, u32 reqid, u8 proto,
> >            xfrm_address_t *daddr, xfrm_address_t *saddr,
> >            int create);
> > + void   (*map_direction)(struct xfrm_state *xfrm);
> >  };
> >
> 
> Please don't build modifiers, but build functions instead.
> 
> e.g. 
> 
> xfrm->direction = map_direction(xfrm)
> 
> That way you don't hide the assignment and thus code becomes much clearer and
> can be called multiple times without risk.

I'll make the change for the next iteration.
-- 
Dave Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1037 bytes --]

On Thu, 2004-12-30 11:16:49 -0500, Dave Dillow <dave@thedillows.org>
wrote in message <1104423409.23254.9.camel@dillow.idleaire.com>:
> On Thu, 2004-12-30 at 04:48, Jan-Benedict Glaw wrote:
> > On Thu, 2004-12-30 03:48:34 -0500, David Dillow <dave@thedillows.org>
> > wrote in message <20041230035000.10@ori.thedillows.org>:
> > > +enum {
> > > +	XFRM_STATE_DIR_UNKNOWN,
> > > +	XFRM_STATE_DIR_IN,
> > > +	XFRM_STATE_DIR_OUT,
> > > +};
> > 
> > Any specific reason to first define such a nice enum and then using int
> > in the struct?
> 
> Just following the current style in net/xfrm.h, see xfrm_state.km.state
> and XFRM_STATE_*.

Hmmm... Maybe I'd prepare patches then :)

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC 2.6.10 5/22] xfrm: Attempt to offload bundled xfrm_states for outbound xfrms

[Francois Romieu]

David Dillow <dave@thedillows.org> :
[...]
> diff -Nru a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> --- a/net/xfrm/xfrm_policy.c	2004-12-30 01:11:18 -05:00
> +++ b/net/xfrm/xfrm_policy.c	2004-12-30 01:11:18 -05:00
> @@ -705,6 +705,31 @@
>  	};
>  }
>  
> +static void xfrm_accel_bundle(struct dst_entry *dst)
> +{
> +	struct xfrm_bundle_list bundle, *xbl, *tmp;
> +	struct net_device *dev = dst->dev;
> +	INIT_LIST_HEAD(&bundle.node);
> +
> +	if (dev && netif_running(dev) && (dev->features & NETIF_F_IPSEC)) {
> +		while (dst) {
> +			xbl = kmalloc(sizeof(*xbl), GFP_ATOMIC);
> +			if (!xbl)
> +				goto out;
> +
> +			xbl->dst = dst;
> +			list_add_tail(&xbl->node, &bundle.node);
> +			dst = dst->child;
> +		}
> +
> +		dev->xfrm_bundle_add(dev, &bundle);
> +	}
> +
> +out:
> +	list_for_each_entry_safe(xbl, tmp, &bundle.node, node)
> +		kfree(xbl);
> +}

If the driver knows the max depth which is allowed, why not have it
allocate its own bundle-like struct during initialization one for once ?
Instead of pushing the bundle list, dst is walked by the code of
the device's own xyz_xfrm_bundle_add into the said circular list,
entries get overwriten if the dst chain is longer and when the end of
dst is reached, the bundle-like list is walked in reverse order.
It avoids a few failure points imho.

--
Ueimor

Re: [RFC 2.6.10 5/22] xfrm: Attempt to offload bundled xfrm_states for outbound xfrms

[David Dillow]

On Fri, 2004-12-31 at 00:34 +0100, Francois Romieu wrote:
> David Dillow <dave@thedillows.org> :
> [...]
> > diff -Nru a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> > --- a/net/xfrm/xfrm_policy.c	2004-12-30 01:11:18 -05:00
> > +++ b/net/xfrm/xfrm_policy.c	2004-12-30 01:11:18 -05:00
> > @@ -705,6 +705,31 @@
> >  	};
> >  }
> >  
> > +static void xfrm_accel_bundle(struct dst_entry *dst)
> > +{
> > +	struct xfrm_bundle_list bundle, *xbl, *tmp;
> > +	struct net_device *dev = dst->dev;
> > +	INIT_LIST_HEAD(&bundle.node);
> > +
> > +	if (dev && netif_running(dev) && (dev->features & NETIF_F_IPSEC)) {
> > +		while (dst) {
> > +			xbl = kmalloc(sizeof(*xbl), GFP_ATOMIC);
> > +			if (!xbl)
> > +				goto out;
> > +
> > +			xbl->dst = dst;
> > +			list_add_tail(&xbl->node, &bundle.node);
> > +			dst = dst->child;
> > +		}
> > +
> > +		dev->xfrm_bundle_add(dev, &bundle);
> > +	}
> > +
> > +out:
> > +	list_for_each_entry_safe(xbl, tmp, &bundle.node, node)
> > +		kfree(xbl);
> > +}
> 
> If the driver knows the max depth which is allowed, why not have it
> allocate its own bundle-like struct during initialization one for once ?
> Instead of pushing the bundle list, dst is walked by the code of
> the device's own xyz_xfrm_bundle_add into the said circular list,
> entries get overwriten if the dst chain is longer and when the end of
> dst is reached, the bundle-like list is walked in reverse order.
> It avoids a few failure points imho.

Good idea! I'll see if I can't code it up. I definitely want to get rid
of that GFP_ATOMIC allocation.
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 0/22] Add hardware assist for IPSEC crypto

[David S. Miller]

On Thu, 30 Dec 2004 03:48:34 -0500
David Dillow <dave@thedillows.org> wrote:

> Bandwidth tests
> 
> Config (sender -> receiver)		Bandwidth	ori CPU	  tank CPU
> No IPSEC (tank->ori)			11494 KB/s	11.9%	  18.7%
> No IPSEC (ori->tank)			11492 KB/s	9.5%	  34.3%
> 
> AH/SHA1 (sw) (tank->ori)		11303 KB/s	29.2%	  79.3%
> AH/SHA1 (sw) (ori->tank)		11302 KB/s	28.6%	  91.1%
> ESP/3DES,SHA1 (sw) (tank->ori)		2130 KB/s	29.6%	  100%
> ESP/3DES,SHA1 (sw) (ori->tank)		2263 KB/s	29.3%	  99.7%
> ESP-AH/3DES,SHA1-SHA1 (sw) (tank->ori)	1906 KB/s	29.1%	  100%
> ESP-AH/3DES,SHA1-SHA1 (sw) (ori->tank)	2051 KB/s	29.3%	  99.7%

It's a shame we don't have an assembler optimized 3DES at least for
x86 in the crypto library of the kernel, even if just for tests such
as this.

I'll start reviewing your patches right now.

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[David S. Miller]

On Thu, 30 Dec 2004 03:48:34 -0500
David Dillow <dave@thedillows.org> wrote:

> # This is a BitKeeper generated diff -Nru style patch.
> #
> # ChangeSet
> #   2004/12/30 00:27:15-05:00 dave@thedillows.org 
> #   Add direction information to xfrm_state. This will be needed to
> #   offload xfrm processing to the NIC.

And xfrm_state, when actually used, does not exist in a vacuum.
It is being used in terms of a particular xfrm_policy which has
a direction indicator already.

Given that, it seems the xfrm_state is not only unnecessary, but
also slightly illogical in fact.  If you need the direction, you
should make sure that at those places you can get back to the
policy in order to obtain that piece of information.

Re: [RFC 2.6.10 2/22] xfrm: Add xfrm offload management calls to struct netdevice

[David S. Miller]

On Thu, 30 Dec 2004 03:48:34 -0500
David Dillow <dave@thedillows.org> wrote:

> # This is a BitKeeper generated diff -Nru style patch.
> #
> # ChangeSet
> #   2004/12/30 00:28:25-05:00 dave@thedillows.org 
> #   Add the xfrm offload management calls to struct netdevice.
> #   
> #   xfrm_state_add() is called for inbound xfrm states
> #   xfrm_bundle_add() is called for outbound xfrm bundles
> #   xfrm_state_del() is called for all offloaded xfrms,
> #   	inbound or outbound.
> #   
> #   If a driver adds NETIF_F_IPSEC to its features, it must
> #   provide all three callbacks.
> #   
> #   Signed-off-by: David Dillow <dave@thedillows.org>

I have no problems with this interface.  In fact, just about
whatever works for the card you have would be fine for an
initial implementation.

We know this interface will likely need to change over time
so we should explicitly mark these callbacks as in a state
of flux so that nobody who tries to play with this gets surprised
when we tweak it.

Re: [RFC 2.6.10 3/22] xfrm: Add offload management routines

[David S. Miller]

On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow <dave@thedillows.org> wrote:

> +static inline struct xfrm_offload *
> +xfrm_offload_alloc(int sizeof_priv, struct net_device *dev)

This whole scheme looks buggy.  The intent is to 8-byte align
the object, but look at what the code is actually doing.

Whatever kmalloc() returns to xfrm_offload_alloc() is directly
used as the xfrm_offload pointer, and the members are initialized.

Then xfrm_offload_priv() does the alignments.

It is clear that kmalloc() is always giving you 8-byte aligned
data else the first time xfrm_offload_priv() is used you'd
get a bogus pointer since xfrm_offload_alloc() initialized
the object without first aligning the pointer.

We do something similar when we allocate netdevs, so have a look
at how net/core/dev.c:alloc_netdev() works.

Re: [RFC 2.6.10 4/22] xfrm: Try to offload inbound xfrm_states

[David S. Miller]

On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow <dave@thedillows.org> wrote:

> # This is a BitKeeper generated diff -Nru style patch.
> #
> # ChangeSet
> #   2004/12/30 00:33:11-05:00 dave@thedillows.org 
> #   Plumb in offloading of inbound xfrm_states.
> #   
> #   Signed-off-by: David Dillow <dave@thedillows.org>

Hmmm, this seems to deadlock.  xfrm_state_add() is invoked
with the RTNL semaphore already acquired.  For example, via
xfrm_user.c:xfrm_add_sa()

Re: [RFC 2.6.10 5/22] xfrm: Attempt to offload bundled xfrm_states for outbound xfrms

[David S. Miller]

On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow <dave@thedillows.org> wrote:

> +static void xfrm_accel_bundle(struct dst_entry *dst)
> +{
> +	struct xfrm_bundle_list bundle, *xbl, *tmp;
> +	struct net_device *dev = dst->dev;
> +	INIT_LIST_HEAD(&bundle.node);
> +
> +	if (dev && netif_running(dev) && (dev->features & NETIF_F_IPSEC)) {

netif_running() is only steady while the RTNL semaphore is held,
which is not necessarily true when xfrm_lookup() is invoked.

Re: [RFC 2.6.10 6/22] xfrm: add a parameter to xfrm_prune_bundles()

[David S. Miller]

On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow <dave@thedillows.org> wrote:

> # This is a BitKeeper generated diff -Nru style patch.
> #
> # ChangeSet
> #   2004/12/30 00:35:34-05:00 dave@thedillows.org 
> #   Add a parameter to the decision function(s) used by
> #   xfrm_prune_bundles(). This will allow us to have more
> #   fine grained selection of bundles pruned (like, say,
> #   per device.)
> #   
> #   Signed-off-by: David Dillow <dave@thedillows.org>

This one looks OK.

Re: [RFC 2.6.10 7/22] xfrm: Allow device drivers to force recalculation of offloads

[David S. Miller]

On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow <dave@thedillows.org> wrote:

> # This is a BitKeeper generated diff -Nru style patch.
> #
> # ChangeSet
> #   2004/12/30 00:37:44-05:00 dave@thedillows.org 
> #   Give device drivers a method to allow the use of crypto
> #   offload features for existing xfrm_states and bundles, as
> #   well as dynamically remove crypto offload capabilities.
> #   
> #   Signed-off-by: David Dillow <dave@thedillows.org>

This change looks fine, but it reminds me of something.

We have to make sure there is no way for xfrm_offloads to
refer to a device when it is brought down.

One way to ensure that, is to make each xfrm_offload attached
grab a reference to the device it is for.  The refcounting will
help catch cases where stale xfrm_offload are around at
unregister_netdev() time.

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[David Dillow]

On Fri, 2005-01-21 at 14:38 -0800, David S. Miller wrote: 
> On Thu, 30 Dec 2004 03:48:34 -0500
> David Dillow <dave@thedillows.org> wrote:
> 
> > # This is a BitKeeper generated diff -Nru style patch.
> > #
> > # ChangeSet
> > #   2004/12/30 00:27:15-05:00 dave@thedillows.org 
> > #   Add direction information to xfrm_state. This will be needed to
> > #   offload xfrm processing to the NIC.
> 
> And xfrm_state, when actually used, does not exist in a vacuum.
> It is being used in terms of a particular xfrm_policy which has
> a direction indicator already.
> 
> Given that, it seems the xfrm_state is not only unnecessary, but
> also slightly illogical in fact.  If you need the direction, you
> should make sure that at those places you can get back to the
> policy in order to obtain that piece of information.

Hmm, I can see having netdev->xfrm_bundle_add() assume a outbound state,
since it's only called for Tx.

Rx offloading could be interesting, but it looks like I could hook into
net/xfrm/xfrm_policy.c:__xfrm_policy_check(), and add the xfrm_states
seen there to a work queue for offloading. netdev->xfrm_state_add()
would then only see inbound states.

Sound sane?
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 4/22] xfrm: Try to offload inbound xfrm_states

[David Dillow]

On Fri, 2005-01-21 at 14:56 -0800, David S. Miller wrote: 
> On Thu, 30 Dec 2004 03:48:35 -0500
> David Dillow <dave@thedillows.org> wrote:
> 
> > # This is a BitKeeper generated diff -Nru style patch.
> > #
> > # ChangeSet
> > #   2004/12/30 00:33:11-05:00 dave@thedillows.org 
> > #   Plumb in offloading of inbound xfrm_states.
> > #   
> > #   Signed-off-by: David Dillow <dave@thedillows.org>
> 
> Hmmm, this seems to deadlock.  xfrm_state_add() is invoked
> with the RTNL semaphore already acquired.  For example, via
> xfrm_user.c:xfrm_add_sa()

Are you sure? It worked just fine during testing, and I don't see where
in the call chain from xfrm_netlink_rcv() to xfrm_state_add() the RTNL
sema is taken. Perhaps you were thinking of xfrm_cfg_sem?
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 5/22] xfrm: Attempt to offload bundled xfrm_states for outbound xfrms

[David Dillow]

On Fri, 2005-01-21 at 15:20 -0800, David S. Miller wrote: 
> On Thu, 30 Dec 2004 03:48:35 -0500
> David Dillow <dave@thedillows.org> wrote:
> 
> > +static void xfrm_accel_bundle(struct dst_entry *dst)
> > +{
> > +	struct xfrm_bundle_list bundle, *xbl, *tmp;
> > +	struct net_device *dev = dst->dev;
> > +	INIT_LIST_HEAD(&bundle.node);
> > +
> > +	if (dev && netif_running(dev) && (dev->features & NETIF_F_IPSEC)) {
> 
> netif_running() is only steady while the RTNL semaphore is held,
> which is not necessarily true when xfrm_lookup() is invoked.

I've been reworking this area already to add the ability for the user to
control which devices/states combinations can be offloaded, so this code
will change. What do you think about just having the driver be
responsible for ignoring offload requests when they don't make sense,
like when the device is down, or in sleep mode?

BTW, xfrm_lookup() also calls stale_bundle(), which also relies on
netif_running(), among other tests.
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 7/22] xfrm: Allow device drivers to force recalculation of offloads

[David Dillow]

On Fri, 2005-01-21 at 15:23 -0800, David S. Miller wrote: 
> On Thu, 30 Dec 2004 03:48:35 -0500
> David Dillow <dave@thedillows.org> wrote:
> 
> > # This is a BitKeeper generated diff -Nru style patch.
> > #
> > # ChangeSet
> > #   2004/12/30 00:37:44-05:00 dave@thedillows.org 
> > #   Give device drivers a method to allow the use of crypto
> > #   offload features for existing xfrm_states and bundles, as
> > #   well as dynamically remove crypto offload capabilities.
> > #   
> > #   Signed-off-by: David Dillow <dave@thedillows.org>
> 
> This change looks fine, but it reminds me of something.
> 
> We have to make sure there is no way for xfrm_offloads to
> refer to a device when it is brought down.
> 
> One way to ensure that, is to make each xfrm_offload attached
> grab a reference to the device it is for.  The refcounting will
> help catch cases where stale xfrm_offload are around at
> unregister_netdev() time.

You're right, it would be a problem, and I have the oopsen to prove
it. :)

In patch 3 of the series, xfrm_offload_add() grabs a reference to the
device, and xfrm_offload_release() releases it when the offload's own
reference count hits zero.
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 3/22] xfrm: Add offload management routines

[David Dillow]

[ Sorry, David, if you get this multiple times. Had a doh! moment... ]

On Fri, 2005-01-21 at 14:47 -0800, David S. Miller wrote: 
> On Thu, 30 Dec 2004 03:48:35 -0500
> David Dillow <dave@thedillows.org> wrote:
> 
> > +static inline struct xfrm_offload *
> > +xfrm_offload_alloc(int sizeof_priv, struct net_device *dev)
> 
> This whole scheme looks buggy.  The intent is to 8-byte align
> the object, but look at what the code is actually doing.
> 
> Whatever kmalloc() returns to xfrm_offload_alloc() is directly
> used as the xfrm_offload pointer, and the members are initialized.
> 
> Then xfrm_offload_priv() does the alignments.

Right, I'm not worried about the alignment of the xfrm_offload, as I'm
assuming that kmalloc will give us something suitable. However, I do
want to make sure that the private section used by the driver is 8 byte
aligned in the face of a potentially oddly sized xfrm_offload struct.

Am I making a bad assumption that kmalloc() will give me a suitable
alignment for the base struct?

> It is clear that kmalloc() is always giving you 8-byte aligned
> data else the first time xfrm_offload_priv() is used you'd
> get a bogus pointer since xfrm_offload_alloc() initialized
> the object without first aligning the pointer.

The driver calls xfrm_offload_alloc(), then calls xfrm_offload_priv() to
get it's private struct, and inits that. I just want to make sure the
private struct is 8 byte aligned, which I assume (wrongly?) will be
sufficient to prevent misaligned accesses.

> We do something similar when we allocate netdevs, so have a look
> at how net/core/dev.c:alloc_netdev() works.

I copied part of that code to xfrm_offload_alloc(), removing the part
that aligned the base xfrm_offload struct. 32 byte alignment seemed a
bit much, do you think it is needed? I'll be happy to add that if you
want.
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 7/22] xfrm: Allow device drivers to force recalculation of offloads

[David S. Miller]

On Sat, 22 Jan 2005 00:53:36 -0500
David Dillow <dave@thedillows.org> wrote:

> In patch 3 of the series, xfrm_offload_add() grabs a reference to the
> device, and xfrm_offload_release() releases it when the offload's own
> reference count hits zero.

Perfect.

Re: [RFC 2.6.10 5/22] xfrm: Attempt to offload bundled xfrm_states for outbound xfrms

[David S. Miller]

On Sat, 22 Jan 2005 00:53:01 -0500
David Dillow <dave@thedillows.org> wrote:

> I've been reworking this area already to add the ability for the user to
> control which devices/states combinations can be offloaded, so this code
> will change. What do you think about just having the driver be
> responsible for ignoring offload requests when they don't make sense,
> like when the device is down, or in sleep mode?

That would be fine.

> BTW, xfrm_lookup() also calls stale_bundle(), which also relies on
> netif_running(), among other tests.

Good catch, hmmm... let me think about this case.

Re: [RFC 2.6.10 4/22] xfrm: Try to offload inbound xfrm_states

[David S. Miller]

On Sat, 22 Jan 2005 00:52:13 -0500
David Dillow <dave@thedillows.org> wrote:

> > Hmmm, this seems to deadlock.  xfrm_state_add() is invoked
> > with the RTNL semaphore already acquired.  For example, via
> > xfrm_user.c:xfrm_add_sa()
> 
> Are you sure? It worked just fine during testing, and I don't see where
> in the call chain from xfrm_netlink_rcv() to xfrm_state_add() the RTNL
> sema is taken. Perhaps you were thinking of xfrm_cfg_sem?

My bad, you're right.

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[David S. Miller]

On Sat, 22 Jan 2005 00:50:22 -0500
David Dillow <dave@thedillows.org> wrote:

> Hmm, I can see having netdev->xfrm_bundle_add() assume a outbound state,
> since it's only called for Tx.
> 
> Rx offloading could be interesting, but it looks like I could hook into
> net/xfrm/xfrm_policy.c:__xfrm_policy_check(), and add the xfrm_states
> seen there to a work queue for offloading. netdev->xfrm_state_add()
> would then only see inbound states.
> 
> Sound sane?

You really can't get at the policies at these places somehow?
Even by passing down the xfrm_policy pointer into these
call chains?

If you could get at the xfrm_policy, then you could get the
direction.

Re: [RFC 2.6.10 1/22] xfrm: Add direction information to xfrm_state

[David Dillow]

On Tue, 2005-01-25 at 22:17 -0800, David S. Miller wrote:
> On Sat, 22 Jan 2005 00:50:22 -0500
> David Dillow <dave@thedillows.org> wrote:
> 
> > Hmm, I can see having netdev->xfrm_bundle_add() assume a outbound state,
> > since it's only called for Tx.
> > 
> > Rx offloading could be interesting, but it looks like I could hook into
> > net/xfrm/xfrm_policy.c:__xfrm_policy_check(), and add the xfrm_states
> > seen there to a work queue for offloading. netdev->xfrm_state_add()
> > would then only see inbound states.
> > 
> > Sound sane?
> 
> You really can't get at the policies at these places somehow?
> Even by passing down the xfrm_policy pointer into these
> call chains?

By these places, you mean xfrm_lookup() and __xfrm_policy_check()?

There I absolutely have access to the policy, but even better, I only
get to those places for one purpose -- xfrm_lookup() on output,
__xfrm_policy_check() on input.

Even in the first versions of the offload code, netdev->xfrm_state_add()
and netdev->xfrm_bundle_add() were solely called for inbound and
outbound xfrm_states, respectively. The problem was that I had no access
to the policy at xfrm_state.c::xfrm_state_add(), since it wouldn't be
called in the context of a policy. 

By moving the offload calls/queuing to __xfrm_policy_check(), I have the
policy available, and I *know* it is inbound. If I offload SAs there, I
can use that knowledge, and I get the benefit of lazy offload -- I'll
only be offloading inbound SAs to netdev's that they've actually been
received on, rather than to all capable devices, as the first version
did.

And, by making the decision to offload at the policy check means I get
more info to use in that decision -- I'll probably change netdev-
>xfrm_state_add() into netdev->xfrm_secpath_add(), and have it take the
inbound secpath as an argument. This will let the driver see a more
complete picture of the inbound xfrms, and allow it to not offload
problematic combinations (the typhoon firmware has issues with certain
combinations.)

Now I'm working on a nice cheap way to decide if the inbound packet is a
candidate for offloading, and avoid trying the same SA stack over and
over and over...
-- 
David Dillow <dave@thedillows.org>

Re: [RFC 2.6.10 3/22] xfrm: Add offload management routines

[David Dillow]

On Tue, 2005-01-25 at 22:16 -0800, David S. Miller wrote:
> On Sat, 22 Jan 2005 00:50:38 -0500
> David Dillow <dave@thedillows.org> wrote:
> >
> Note that kmalloc() and SLAB do guarentee whatever alignment is
> necessary to access the largest aligned C data type on a platform.
> So on sparc32 this is 8 bytes, on some it is 4 bytes.

I must be missing something, since I only need to ensure the items in
struct xfrm_offload are aligned correctly -- the pointers in the first
list_head need to be aligned for the platform. I'm assuming the compiler
will align them on a proper offset from the beginning of the struct, and
as long as kmalloc() gives me the proper alignment for a pointer, then I
don't see the problem.

The hardware will never DMA the base xfrm_offload struct, and only
possibly it's private area -- which I agree would need proper alignment.

Just for reference, here's xfrm_offload:

struct xfrm_offload
{
        struct list_head        bydev;
        struct net_device *     dev;
        atomic_t                refcnt;
};

I'm making the assumption that kmalloc() will give me back a pointer
that will have an acceptable alignment for the platform for all of the
pointers, as well as the refcnt.

> > I copied part of that code to xfrm_offload_alloc(), removing the part
> > that aligned the base xfrm_offload struct. 32 byte alignment seemed a
> > bit much, do you think it is needed? I'll be happy to add that if you
> > want.
> 
> Depends upon what you really need the alignment for.  If you need it
> due to some hardware restriction or whatever, that would be determined
> per device I guess.

Per above, it seems like I only need kmalloc()'s alignment for struct
xfrm_offload. I don't know what the user will put in his private area,
but I wanted to try to align it such that it had the same guarantee that
kmalloc() had. That was the purpose of doing the alignment math only in
xfrm_offload_priv().
-- 
David Dillow <dave@thedillows.org>