netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH net-next] net: ipv6/gre: Add GRO support
@ 2018-04-01 13:17 Tariq Toukan
  2018-04-01 16:35 ` Eric Dumazet
  0 siblings, 1 reply; 7+ messages in thread
From: Tariq Toukan @ 2018-04-01 13:17 UTC (permalink / raw)
  To: David S. Miller; +Cc: netdev, Eran Ben Elisha, Tariq Toukan, Eric Dumazet

From: Eran Ben Elisha <eranbe@mellanox.com>

Add GRO capability for IPv6 GRE tunnel and ip6erspan tap, via gro_cells
infrastructure.

Performance testing: 55% higher badwidth.
Measuring bandwidth of 1 thread IPv4 TCP traffic over IPv6 GRE tunnel
while GRO on the physical interface is disabled.
CPU: Intel Xeon E312xx (Sandy Bridge)
NIC: Mellanox Technologies MT27700 Family [ConnectX-4]
Before (GRO not working in tunnel) : 2.47 Gbits/sec
After  (GRO working in tunnel)     : 3.85 Gbits/sec

Signed-off-by: Eran Ben Elisha <eranbe@mellanox.com>
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
CC: Eric Dumazet <edumazet@google.com>
---
 net/ipv6/ip6_gre.c | 37 +++++++++++++++++++++++++++----------
 1 file changed, 27 insertions(+), 10 deletions(-)

diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 22e86557aca4..8abc9d72993e 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -1306,6 +1306,7 @@ static void ip6gre_dev_free(struct net_device *dev)
 {
 	struct ip6_tnl *t = netdev_priv(dev);
 
+	gro_cells_destroy(&t->gro_cells);
 	dst_cache_destroy(&t->dst_cache);
 	free_percpu(dev->tstats);
 }
@@ -1373,11 +1374,12 @@ static int ip6gre_tunnel_init_common(struct net_device *dev)
 		return -ENOMEM;
 
 	ret = dst_cache_init(&tunnel->dst_cache, GFP_KERNEL);
-	if (ret) {
-		free_percpu(dev->tstats);
-		dev->tstats = NULL;
-		return ret;
-	}
+	if (ret)
+		goto cleanup_alloc_pcpu_stats;
+
+	ret = gro_cells_init(&tunnel->gro_cells, dev);
+	if (ret)
+		goto cleanup_dst_cache_init;
 
 	tunnel->tun_hlen = gre_calc_hlen(tunnel->parms.o_flags);
 	tunnel->hlen = tunnel->tun_hlen + tunnel->encap_hlen;
@@ -1397,6 +1399,13 @@ static int ip6gre_tunnel_init_common(struct net_device *dev)
 	ip6gre_tnl_init_features(dev);
 
 	return 0;
+
+cleanup_dst_cache_init:
+	dst_cache_destroy(&tunnel->dst_cache);
+cleanup_alloc_pcpu_stats:
+	free_percpu(dev->tstats);
+	dev->tstats = NULL;
+	return ret;
 }
 
 static int ip6gre_tunnel_init(struct net_device *dev)
@@ -1743,11 +1752,12 @@ static int ip6erspan_tap_init(struct net_device *dev)
 		return -ENOMEM;
 
 	ret = dst_cache_init(&tunnel->dst_cache, GFP_KERNEL);
-	if (ret) {
-		free_percpu(dev->tstats);
-		dev->tstats = NULL;
-		return ret;
-	}
+	if (ret)
+		goto cleanup_alloc_pcpu_stats;
+
+	ret = gro_cells_init(&tunnel->gro_cells, dev);
+	if (ret)
+		goto cleanup_dst_cache_init;
 
 	tunnel->tun_hlen = 8;
 	tunnel->hlen = tunnel->tun_hlen + tunnel->encap_hlen +
@@ -1766,6 +1776,13 @@ static int ip6erspan_tap_init(struct net_device *dev)
 	ip6gre_tnl_link_config(tunnel, 1);
 
 	return 0;
+
+cleanup_dst_cache_init:
+	dst_cache_destroy(&tunnel->dst_cache);
+cleanup_alloc_pcpu_stats:
+	free_percpu(dev->tstats);
+	dev->tstats = NULL;
+	return ret;
 }
 
 static const struct net_device_ops ip6erspan_netdev_ops = {
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH net-next] net: ipv6/gre: Add GRO support
  2018-04-01 13:17 [PATCH net-next] net: ipv6/gre: Add GRO support Tariq Toukan
@ 2018-04-01 16:35 ` Eric Dumazet
  2018-04-02 12:40   ` Eran Ben Elisha
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Dumazet @ 2018-04-01 16:35 UTC (permalink / raw)
  To: Tariq Toukan, David S. Miller; +Cc: netdev, Eran Ben Elisha, Eric Dumazet



On 04/01/2018 06:17 AM, Tariq Toukan wrote:
> From: Eran Ben Elisha <eranbe@mellanox.com>
> 
> Add GRO capability for IPv6 GRE tunnel and ip6erspan tap, via gro_cells
> infrastructure.
> 
> Performance testing: 55% higher badwidth.
> Measuring bandwidth of 1 thread IPv4 TCP traffic over IPv6 GRE tunnel
> while GRO on the physical interface is disabled.
> CPU: Intel Xeon E312xx (Sandy Bridge)
> NIC: Mellanox Technologies MT27700 Family [ConnectX-4]
> Before (GRO not working in tunnel) : 2.47 Gbits/sec
> After  (GRO working in tunnel)     : 3.85 Gbits/sec
> 
> Signed-off-by: Eran Ben Elisha <eranbe@mellanox.com>
> Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
> CC: Eric Dumazet <edumazet@google.com>
> ---


Seems good, but why isn't this handled directly in GRO native layer ?

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH net-next] net: ipv6/gre: Add GRO support
  2018-04-01 16:35 ` Eric Dumazet
@ 2018-04-02 12:40   ` Eran Ben Elisha
  2018-04-02 14:42     ` Eric Dumazet
  0 siblings, 1 reply; 7+ messages in thread
From: Eran Ben Elisha @ 2018-04-02 12:40 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Tariq Toukan, David S. Miller, Linux Netdev List,
	Eran Ben Elisha, Eric Dumazet

On Sun, Apr 1, 2018 at 7:35 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
>
> On 04/01/2018 06:17 AM, Tariq Toukan wrote:
>> From: Eran Ben Elisha <eranbe@mellanox.com>
>>
>> Add GRO capability for IPv6 GRE tunnel and ip6erspan tap, via gro_cells
>> infrastructure.
>>
>> Performance testing: 55% higher badwidth.
>> Measuring bandwidth of 1 thread IPv4 TCP traffic over IPv6 GRE tunnel
>> while GRO on the physical interface is disabled.
>> CPU: Intel Xeon E312xx (Sandy Bridge)
>> NIC: Mellanox Technologies MT27700 Family [ConnectX-4]
>> Before (GRO not working in tunnel) : 2.47 Gbits/sec
>> After  (GRO working in tunnel)     : 3.85 Gbits/sec
>>
>> Signed-off-by: Eran Ben Elisha <eranbe@mellanox.com>
>> Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
>> CC: Eric Dumazet <edumazet@google.com>
>> ---
>
>
> Seems good, but why isn't this handled directly in GRO native layer ?
ip6_tunnel and ip6_gre do not share initialization flow functions (unlike ipv4).
Changing the ipv6 init infrastructure should not be part of this
patch. we prefer to keep this one minimal, simple and safe.
>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH net-next] net: ipv6/gre: Add GRO support
  2018-04-02 12:40   ` Eran Ben Elisha
@ 2018-04-02 14:42     ` Eric Dumazet
  2018-04-02 15:00       ` Eran Ben Elisha
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Dumazet @ 2018-04-02 14:42 UTC (permalink / raw)
  To: Eran Ben Elisha
  Cc: Tariq Toukan, David S. Miller, Linux Netdev List,
	Eran Ben Elisha, Eric Dumazet



On 04/02/2018 05:40 AM, Eran Ben Elisha wrote:
> On Sun, Apr 1, 2018 at 7:35 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>
>>
>> On 04/01/2018 06:17 AM, Tariq Toukan wrote:
>>> From: Eran Ben Elisha <eranbe@mellanox.com>
>>>
>>> Add GRO capability for IPv6 GRE tunnel and ip6erspan tap, via gro_cells
>>> infrastructure.
>>>
>>> Performance testing: 55% higher badwidth.
>>> Measuring bandwidth of 1 thread IPv4 TCP traffic over IPv6 GRE tunnel
>>> while GRO on the physical interface is disabled.
>>> CPU: Intel Xeon E312xx (Sandy Bridge)
>>> NIC: Mellanox Technologies MT27700 Family [ConnectX-4]
>>> Before (GRO not working in tunnel) : 2.47 Gbits/sec
>>> After  (GRO working in tunnel)     : 3.85 Gbits/sec
>>>
>>> Signed-off-by: Eran Ben Elisha <eranbe@mellanox.com>
>>> Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
>>> CC: Eric Dumazet <edumazet@google.com>
>>> ---
>>
>>
>> Seems good, but why isn't this handled directly in GRO native layer ?
> ip6_tunnel and ip6_gre do not share initialization flow functions (unlike ipv4).
> Changing the ipv6 init infrastructure should not be part of this
> patch. we prefer to keep this one minimal, simple and safe.



Looking at gre_gro_receive() and gre_gro_complete() I could not see why they
could not be copied/pasted to IPv6.

Maybe give more details on the changelog, it is really not obvious.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH net-next] net: ipv6/gre: Add GRO support
  2018-04-02 14:42     ` Eric Dumazet
@ 2018-04-02 15:00       ` Eran Ben Elisha
  2018-04-02 15:19         ` Eric Dumazet
  0 siblings, 1 reply; 7+ messages in thread
From: Eran Ben Elisha @ 2018-04-02 15:00 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Tariq Toukan, David S. Miller, Linux Netdev List,
	Eran Ben Elisha, Eric Dumazet

>>> Seems good, but why isn't this handled directly in GRO native layer ?
>> ip6_tunnel and ip6_gre do not share initialization flow functions (unlike ipv4).
>> Changing the ipv6 init infrastructure should not be part of this
>> patch. we prefer to keep this one minimal, simple and safe.
>
>
>
> Looking at gre_gro_receive() and gre_gro_complete() I could not see why they
> could not be copied/pasted to IPv6.

These functions to handle GRO over GRE are already assigned in
gre_offload_init() (in net/ipv4/gre_offload.c under CONFIG_IPV6).
However without initializing the gro_cells, the receive path will not
go via napi_gro_receive path, but directly to netif_rx.
So AFAIU, only gcells->cells was missing for gro_cells_receive to
really go via GRO flow.

>
> Maybe give more details on the changelog, it is really not obvious.
Hopefully the above filled this request.
>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH net-next] net: ipv6/gre: Add GRO support
  2018-04-02 15:00       ` Eran Ben Elisha
@ 2018-04-02 15:19         ` Eric Dumazet
  2018-04-02 15:45           ` Tariq Toukan
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Dumazet @ 2018-04-02 15:19 UTC (permalink / raw)
  To: Eran Ben Elisha, Eric Dumazet
  Cc: Tariq Toukan, David S. Miller, Linux Netdev List,
	Eran Ben Elisha, Eric Dumazet



On 04/02/2018 08:00 AM, Eran Ben Elisha wrote:
>>>> Seems good, but why isn't this handled directly in GRO native layer ?
>>> ip6_tunnel and ip6_gre do not share initialization flow functions (unlike ipv4).
>>> Changing the ipv6 init infrastructure should not be part of this
>>> patch. we prefer to keep this one minimal, simple and safe.
>>
>>
>>
>> Looking at gre_gro_receive() and gre_gro_complete() I could not see why they
>> could not be copied/pasted to IPv6.
> 
> These functions to handle GRO over GRE are already assigned in
> gre_offload_init() (in net/ipv4/gre_offload.c under CONFIG_IPV6).
> However without initializing the gro_cells, the receive path will not
> go via napi_gro_receive path, but directly to netif_rx.
> So AFAIU, only gcells->cells was missing for gro_cells_receive to
> really go via GRO flow.
> 
>>
>> Maybe give more details on the changelog, it is really not obvious.
> Hopefully the above filled this request.
>>

Not really :/

gro_cells_receive() is not really useful with native GRO, since packet is already
a GRO packet by the time it reaches ip_tunnel_rcv() or __ip6_tnl_rcv()

Sure, it might be usefull if native GRO (happening on eth0 if you prefer) did not
handle a particular encapsulation.

gro_cell was a work around before we extended GRO to be able to decap some tunnel headers.

It seems we have to extend this to also support GRE6.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH net-next] net: ipv6/gre: Add GRO support
  2018-04-02 15:19         ` Eric Dumazet
@ 2018-04-02 15:45           ` Tariq Toukan
  0 siblings, 0 replies; 7+ messages in thread
From: Tariq Toukan @ 2018-04-02 15:45 UTC (permalink / raw)
  To: Eric Dumazet, Eran Ben Elisha
  Cc: David S. Miller, Linux Netdev List, Eran Ben Elisha, Eric Dumazet



On 02/04/2018 6:19 PM, Eric Dumazet wrote:
> 
> 
> On 04/02/2018 08:00 AM, Eran Ben Elisha wrote:
>>>>> Seems good, but why isn't this handled directly in GRO native layer ?
>>>> ip6_tunnel and ip6_gre do not share initialization flow functions (unlike ipv4).
>>>> Changing the ipv6 init infrastructure should not be part of this
>>>> patch. we prefer to keep this one minimal, simple and safe.
>>>
>>>
>>>
>>> Looking at gre_gro_receive() and gre_gro_complete() I could not see why they
>>> could not be copied/pasted to IPv6.
>>
>> These functions to handle GRO over GRE are already assigned in
>> gre_offload_init() (in net/ipv4/gre_offload.c under CONFIG_IPV6).
>> However without initializing the gro_cells, the receive path will not
>> go via napi_gro_receive path, but directly to netif_rx.
>> So AFAIU, only gcells->cells was missing for gro_cells_receive to
>> really go via GRO flow.
>>
>>>
>>> Maybe give more details on the changelog, it is really not obvious.
>> Hopefully the above filled this request.
>>>
> 
> Not really :/
> 

So you're referring to native interface. We thought you meant native IP 
module.


> gro_cells_receive() is not really useful with native GRO, since packet is already
> a GRO packet by the time it reaches ip_tunnel_rcv() or __ip6_tnl_rcv()
> 

Right. If GRO on native interface is ON, our patch doesn't help much.
The case we improve here is:
Native has GRO OFF, GRE has GRO ON.

Before this patch there were no GRO packets at all in this case, only 
MTU packets went up the stack.

> Sure, it might be usefull if native GRO (happening on eth0 if you prefer) did not
> handle a particular encapsulation.
> 

Or it is turned OFF.

> gro_cell was a work around before we extended GRO to be able to decap some tunnel headers.
> 
> It seems we have to extend this to also support GRE6.
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2018-04-02 15:45 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-01 13:17 [PATCH net-next] net: ipv6/gre: Add GRO support Tariq Toukan
2018-04-01 16:35 ` Eric Dumazet
2018-04-02 12:40   ` Eran Ben Elisha
2018-04-02 14:42     ` Eric Dumazet
2018-04-02 15:00       ` Eran Ben Elisha
2018-04-02 15:19         ` Eric Dumazet
2018-04-02 15:45           ` Tariq Toukan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).