[PATCH] net: do not disable sg for AoE

[PATCH] net: do not disable sg for AoE

[Ed Cashin]

A change in a series of VLAN-related changes appears to have
inadvertently disabled the use of the scatter gather feature of
network cards for transmission of non-IP ethernet protocols like ATA
over Ethernet (AoE).  Below is a reference to the commit that
introduces a "harmonize_features" function that turns off scatter
gather when the NIC does not support hardware checksumming for the
ethernet protocol of an sk buff.

  commit f01a5236bd4b140198fbcc550f085e8361fd73fa
  Author: Jesse Gross <jesse@nicira.com>
  Date:   Sun Jan 9 06:23:31 2011 +0000

      net offloading: Generalize netif_get_vlan_features().

The can_checksum_protocol function is not equipped to consider a
protocol that does not require checksumming.  Calling it for a
protocol that requires no checksum is inappropriate.

The patch below has harmonize_features call can_checksum_protocol when
the protocol needs a checksum, so that the network layer is not forced
to perform unnecessary skb linearization on the transmission of AoE
packets.  Unnecessary linearization results in decreased performance
and increased memory pressure, as reported here:

  http://www.spinics.net/lists/linux-mm/msg15184.html

The problem has probably not been widely experienced yet, because
only recently has the kernel.org-distributed aoe driver acquired the
ability to use payloads of over a page in size, with the patchset
recently included in the mm tree:

  https://lkml.org/lkml/2012/8/28/140

The coraid.com-distributed aoe driver already could use payloads of
greater than a page in size, but its users generally do not use the
newest kernels.

Signed-off-by: Ed Cashin <ecashin@coraid.com>
---
 net/core/dev.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/net/core/dev.c b/net/core/dev.c
index d7fe32c..5531159 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2134,7 +2134,8 @@ static bool can_checksum_protocol(netdev_features_t features, __be16 protocol)
 static netdev_features_t harmonize_features(struct sk_buff *skb,
 	__be16 protocol, netdev_features_t features)
 {
-	if (!can_checksum_protocol(features, protocol)) {
+	if (protocol != htons(ETH_P_AOE) &&
+	    !can_checksum_protocol(features, protocol)) {
 		features &= ~NETIF_F_ALL_CSUM;
 		features &= ~NETIF_F_SG;
 	} else if (illegal_highdma(skb->dev, skb)) {
-- 
1.7.2.5

Re: [PATCH] net: do not disable sg for AoE

[Ben Hutchings]

On Tue, 2012-09-18 at 20:20 -0400, Ed Cashin wrote:
> A change in a series of VLAN-related changes appears to have
> inadvertently disabled the use of the scatter gather feature of
> network cards for transmission of non-IP ethernet protocols like ATA
> over Ethernet (AoE).  Below is a reference to the commit that
> introduces a "harmonize_features" function that turns off scatter
> gather when the NIC does not support hardware checksumming for the
> ethernet protocol of an sk buff.
> 
>   commit f01a5236bd4b140198fbcc550f085e8361fd73fa
>   Author: Jesse Gross <jesse@nicira.com>
>   Date:   Sun Jan 9 06:23:31 2011 +0000
> 
>       net offloading: Generalize netif_get_vlan_features().
> 
> The can_checksum_protocol function is not equipped to consider a
> protocol that does not require checksumming.  Calling it for a
> protocol that requires no checksum is inappropriate.

Right, but...

[...]
> diff --git a/net/core/dev.c b/net/core/dev.c
> index d7fe32c..5531159 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -2134,7 +2134,8 @@ static bool can_checksum_protocol(netdev_features_t features, __be16 protocol)
>  static netdev_features_t harmonize_features(struct sk_buff *skb,
>  	__be16 protocol, netdev_features_t features)
>  {
> -	if (!can_checksum_protocol(features, protocol)) {
> +	if (protocol != htons(ETH_P_AOE) &&

the generic way to check that would be skb->ip_summed != CHECKSUM_NONE.

Ben.

> +	    !can_checksum_protocol(features, protocol)) {
>  		features &= ~NETIF_F_ALL_CSUM;
>  		features &= ~NETIF_F_SG;
>  	} else if (illegal_highdma(skb->dev, skb)) {

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH 0/2] do not disable sg when packet requires no checksum

[Ed Cashin]

This two-part patchset replaces an earlier net-only patch that
added an explicit check for the AoE protocol to harmonize_features
in net/core/dev.c.

Following the suggestion of Ben Hutchings, this patchset makes
the decision in the network layer protocol agnostic instead of
using ETH_P_AOE as a special case.

Ed L. Cashin (2):
  aoe: mark AoE packets as requiring no checksumming
  net: do not disable sg for packets requiring no checksum

 drivers/block/aoe/aoecmd.c |    1 +
 net/core/dev.c             |    3 ++-
 2 files changed, 3 insertions(+), 1 deletions(-)

-- 
1.7.2.5

[PATCH 1/2] aoe: mark AoE packets as requiring no checksumming

[Ed Cashin]

In order for the network layer to see that AoE requires
no checksumming in a generic way, packets should be marked
CHECKSUM_NONE.

Rather than relying on the current behavior of alloc_skb,
this change causes the aoe driver to explicitly mark its
packets as requiring no checksum.

Signed-off-by: Ed Cashin <ecashin@coraid.com>
---
 drivers/block/aoe/aoecmd.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c
index de0435e..0ba1b63 100644
--- a/drivers/block/aoe/aoecmd.c
+++ b/drivers/block/aoe/aoecmd.c
@@ -35,6 +35,7 @@ new_skb(ulong len)
 		skb_reset_mac_header(skb);
 		skb_reset_network_header(skb);
 		skb->protocol = __constant_htons(ETH_P_AOE);
+		skb->ip_summed = CHECKSUM_NONE;
 	}
 	return skb;
 }
-- 
1.7.2.5

Re: [PATCH 1/2] aoe: mark AoE packets as requiring no checksumming

[Ben Hutchings]

On Wed, 2012-09-19 at 18:17 -0700, Ed Cashin wrote:
> In order for the network layer to see that AoE requires
> no checksumming in a generic way, packets should be marked
> CHECKSUM_NONE.

Which they are by default.

> Rather than relying on the current behavior of alloc_skb,
> this change causes the aoe driver to explicitly mark its
> packets as requiring no checksum.
>
> Signed-off-by: Ed Cashin <ecashin@coraid.com>
> ---
>  drivers/block/aoe/aoecmd.c |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c
> index de0435e..0ba1b63 100644
> --- a/drivers/block/aoe/aoecmd.c
> +++ b/drivers/block/aoe/aoecmd.c
> @@ -35,6 +35,7 @@ new_skb(ulong len)
>  		skb_reset_mac_header(skb);
>  		skb_reset_network_header(skb);
>  		skb->protocol = __constant_htons(ETH_P_AOE);
> +		skb->ip_summed = CHECKSUM_NONE;

Can be written as:
		skb_checksum_none_assert(skb);
if you really want to be sure.

Ben.

>  	}
>  	return skb;
>  }

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [PATCH 1/2] aoe: mark AoE packets as requiring no checksumming

[Ed Cashin]

On Sep 19, 2012, at 9:25 PM, Ben Hutchings wrote:

> Can be written as:
> 		skb_checksum_none_assert(skb);

Another good idea.  I'll resend.

-- 
  Ed Cashin
  ecashin@coraid.com

[PATCH 2/2] net: do not disable sg for packets requiring no checksum

[Ed Cashin]

A change in a series of VLAN-related changes appears to have
inadvertently disabled the use of the scatter gather feature of
network cards for transmission of non-IP ethernet protocols like ATA
over Ethernet (AoE).  Below is a reference to the commit that
introduces a "harmonize_features" function that turns off scatter
gather when the NIC does not support hardware checksumming for the
ethernet protocol of an sk buff.

  commit f01a5236bd4b140198fbcc550f085e8361fd73fa
  Author: Jesse Gross <jesse@nicira.com>
  Date:   Sun Jan 9 06:23:31 2011 +0000

      net offloading: Generalize netif_get_vlan_features().

The can_checksum_protocol function is not equipped to consider a
protocol that does not require checksumming.  Calling it for a
protocol that requires no checksum is inappropriate.

The patch below has harmonize_features call can_checksum_protocol when
the protocol needs a checksum, so that the network layer is not forced
to perform unnecessary skb linearization on the transmission of AoE
packets.  Unnecessary linearization results in decreased performance
and increased memory pressure, as reported here:

  http://www.spinics.net/lists/linux-mm/msg15184.html

The problem has probably not been widely experienced yet, because
only recently has the kernel.org-distributed aoe driver acquired the
ability to use payloads of over a page in size, with the patchset
recently included in the mm tree:

  https://lkml.org/lkml/2012/8/28/140

The coraid.com-distributed aoe driver already could use payloads of
greater than a page in size, but its users generally do not use the
newest kernels.

Signed-off-by: Ed Cashin <ecashin@coraid.com>
---
 net/core/dev.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/net/core/dev.c b/net/core/dev.c
index d7fe32c..9b934d4 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2134,7 +2134,8 @@ static bool can_checksum_protocol(netdev_features_t features, __be16 protocol)
 static netdev_features_t harmonize_features(struct sk_buff *skb,
 	__be16 protocol, netdev_features_t features)
 {
-	if (!can_checksum_protocol(features, protocol)) {
+	if (skb->ip_summed != CHECKSUM_NONE &&
+	    !can_checksum_protocol(features, protocol)) {
 		features &= ~NETIF_F_ALL_CSUM;
 		features &= ~NETIF_F_SG;
 	} else if (illegal_highdma(skb->dev, skb)) {
-- 
1.7.2.5

Re: [PATCH] net: do not disable sg for AoE

[David Miller]

> Date: Tue, 18 Sep 2012 20:20:36 -0400

Please do not incorporate the date of the commit in your local tree
into the Date: field advertised in your outgoing emails when you post
patches.

You did not post this on September 18th, you posted it on September
19th.

When you post patches the way you did, it screws up the ordering
of my patch queue at:

	http://patchwork.ozlabs.org/project/netdev/list/

since it is sorted by the Date: in the patch's email.

Therefore if you just use the normal email posting date it makes
my life a lot easier and it makes sure patches will be looked at
and processed in the correct order.

Thanks.

Re: [PATCH] net: do not disable sg for AoE

[Ed Cashin]

On Sep 19, 2012, at 4:00 PM, David Miller wrote:

> Please do not incorporate the date of the commit in your local tree
> into the Date: field advertised in your outgoing emails when you post
> patches.

Noted.   I will remove the Date field added by git-format-patch from now on.

Sorry for the inconvenience.

-- 
  Ed Cashin
  ecashin@coraid.com