* [RFC PATCH 0/5] Add GSO Partial support for devices with existing tunnel offloads @ 2016-04-19 19:05 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:05 UTC (permalink / raw) To: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, mchan This patch set is meant to extend tunnel offload support for devices that already support UDP or GRE tunnels to include support for those type of tunnels with an outer checksum. To do this I am enabling GSO partial support for these types of tunnels which should work assuming the devices supported by these drivers are either overwriting or ignoring checksum and length fields contained in the IPv4/v6 and ignoring the checksum contained in the outer L4 tunnel headers. I don't have any of the hardware that uses the drivers that these patches are applied do. That was one of the motivations for submitting them as RFC. My hope is that somebody out there with this hardware is willing to pull in the patch and test it out in order to determine if this approach actually works for these devices. --- Alexander Duyck (5): mlx4: Add support for UDP tunnel segmentation with outer checksum offload mlx5: Add support for UDP tunnel segmentation with outer checksum offload bnx2x: Add support for segmentation of tunnels with outer checksums bnxt: Add support for segmentation of tunnels with outer checksums fm10k: Add support for UDP tunnel segmentation with outer checksum offload drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 13 ++++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++++++-- drivers/net/ethernet/intel/fm10k/fm10k_netdev.c | 14 ++++++++++---- drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 17 +++++++++++++---- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- 5 files changed, 46 insertions(+), 14 deletions(-) ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 0/5] Add GSO Partial support for devices with existing tunnel offloads @ 2016-04-19 19:05 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:05 UTC (permalink / raw) To: intel-wired-lan This patch set is meant to extend tunnel offload support for devices that already support UDP or GRE tunnels to include support for those type of tunnels with an outer checksum. To do this I am enabling GSO partial support for these types of tunnels which should work assuming the devices supported by these drivers are either overwriting or ignoring checksum and length fields contained in the IPv4/v6 and ignoring the checksum contained in the outer L4 tunnel headers. I don't have any of the hardware that uses the drivers that these patches are applied do. That was one of the motivations for submitting them as RFC. My hope is that somebody out there with this hardware is willing to pull in the patch and test it out in order to determine if this approach actually works for these devices. --- Alexander Duyck (5): mlx4: Add support for UDP tunnel segmentation with outer checksum offload mlx5: Add support for UDP tunnel segmentation with outer checksum offload bnx2x: Add support for segmentation of tunnels with outer checksums bnxt: Add support for segmentation of tunnels with outer checksums fm10k: Add support for UDP tunnel segmentation with outer checksum offload drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 13 ++++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++++++-- drivers/net/ethernet/intel/fm10k/fm10k_netdev.c | 14 ++++++++++---- drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 17 +++++++++++++---- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- 5 files changed, 46 insertions(+), 14 deletions(-) -- ^ permalink raw reply [flat|nested] 50+ messages in thread
* [RFC PATCH 1/5] mlx4: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-19 19:05 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:05 UTC (permalink / raw) To: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, mchan This patch assumes that the mlx4 hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP headers. I have no means of testing this as I do not have any mlx4 hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment UDP tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c index b4b258c8ca47..cb6a8a522857 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c @@ -2356,7 +2356,9 @@ out: /* set offloads */ priv->dev->hw_enc_features |= NETIF_F_IP_CSUM | NETIF_F_RXCSUM | - NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL; + NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; } static void mlx4_en_del_vxlan_offloads(struct work_struct *work) @@ -2366,7 +2368,9 @@ static void mlx4_en_del_vxlan_offloads(struct work_struct *work) vxlan_del_task); /* unset offloads */ priv->dev->hw_enc_features &= ~(NETIF_F_IP_CSUM | NETIF_F_RXCSUM | - NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL); + NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL); ret = mlx4_SET_PORT_VXLAN(priv->mdev->dev, priv->port, VXLAN_STEER_BY_OUTER_MAC, 0); @@ -2990,8 +2994,13 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, } if (mdev->dev->caps.tunnel_offload_mode == MLX4_TUNNEL_OFFLOAD_MODE_VXLAN) { - dev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; - dev->features |= NETIF_F_GSO_UDP_TUNNEL; + dev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + dev->features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; } mdev->pndev[port] = dev; ^ permalink raw reply related [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 1/5] mlx4: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-19 19:05 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:05 UTC (permalink / raw) To: intel-wired-lan This patch assumes that the mlx4 hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP headers. I have no means of testing this as I do not have any mlx4 hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment UDP tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c index b4b258c8ca47..cb6a8a522857 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c @@ -2356,7 +2356,9 @@ out: /* set offloads */ priv->dev->hw_enc_features |= NETIF_F_IP_CSUM | NETIF_F_RXCSUM | - NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL; + NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; } static void mlx4_en_del_vxlan_offloads(struct work_struct *work) @@ -2366,7 +2368,9 @@ static void mlx4_en_del_vxlan_offloads(struct work_struct *work) vxlan_del_task); /* unset offloads */ priv->dev->hw_enc_features &= ~(NETIF_F_IP_CSUM | NETIF_F_RXCSUM | - NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL); + NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL); ret = mlx4_SET_PORT_VXLAN(priv->mdev->dev, priv->port, VXLAN_STEER_BY_OUTER_MAC, 0); @@ -2990,8 +2994,13 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, } if (mdev->dev->caps.tunnel_offload_mode == MLX4_TUNNEL_OFFLOAD_MODE_VXLAN) { - dev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; - dev->features |= NETIF_F_GSO_UDP_TUNNEL; + dev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + dev->features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; } mdev->pndev[port] = dev; ^ permalink raw reply related [flat|nested] 50+ messages in thread
* [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-19 19:06 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, mchan This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP headers. I have no means of testing this as I do not have any mlx5 hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment UDP tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index e0adb604f461..57d8da796d50 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; if (mlx5e_vxlan_allowed(mdev)) { - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; netdev->hw_enc_features |= NETIF_F_IP_CSUM; netdev->hw_enc_features |= NETIF_F_RXCSUM; netdev->hw_enc_features |= NETIF_F_TSO; netdev->hw_enc_features |= NETIF_F_TSO6; netdev->hw_enc_features |= NETIF_F_RXHASH; netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; } netdev->features = netdev->hw_features; ^ permalink raw reply related [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-19 19:06 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: intel-wired-lan This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP headers. I have no means of testing this as I do not have any mlx5 hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment UDP tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index e0adb604f461..57d8da796d50 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; if (mlx5e_vxlan_allowed(mdev)) { - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; netdev->hw_enc_features |= NETIF_F_IP_CSUM; netdev->hw_enc_features |= NETIF_F_RXCSUM; netdev->hw_enc_features |= NETIF_F_TSO; netdev->hw_enc_features |= NETIF_F_TSO6; netdev->hw_enc_features |= NETIF_F_RXHASH; netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; } netdev->features = netdev->hw_features; ^ permalink raw reply related [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-20 17:40 ` Saeed Mahameed -1 siblings, 0 replies; 50+ messages in thread From: Saeed Mahameed @ 2016-04-20 17:40 UTC (permalink / raw) To: Alexander Duyck Cc: eugenia, bruce.w.allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, ariel.elior, mchan, Matthew Finlay On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: > This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 > header fields for length and checksum as well as the length and checksum > fields for outer UDP headers. > > I have no means of testing this as I do not have any mlx5 hardware but > thought I would submit it as an RFC to see if anyone out there wants to > test this and see if this does in fact enable this functionality allowing > us to to segment UDP tunneled frames that have an outer checksum. > > Signed-off-by: Alexander Duyck <aduyck@mirantis.com> > --- > drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index e0adb604f461..57d8da796d50 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) > netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; > > if (mlx5e_vxlan_allowed(mdev)) { > - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; > + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | > + NETIF_F_GSO_UDP_TUNNEL_CSUM | > + NETIF_F_GSO_PARTIAL; > netdev->hw_enc_features |= NETIF_F_IP_CSUM; > netdev->hw_enc_features |= NETIF_F_RXCSUM; > netdev->hw_enc_features |= NETIF_F_TSO; > netdev->hw_enc_features |= NETIF_F_TSO6; > netdev->hw_enc_features |= NETIF_F_RXHASH; > netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; > + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | > + NETIF_F_GSO_PARTIAL; > + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; > } > > netdev->features = netdev->hw_features; > Hi Alex, Adding Matt, VxLAN feature owner from Mellanox, Matt please correct me if am wrong, but We already tested GSO VxLAN and we saw the TCP/IP checksum offloads for both inner and outer headers handled by the hardware. And looking at mlx5e_sq_xmit: if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; if (skb->encapsulation) { eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | MLX5_ETH_WQE_L4_INNER_CSUM; sq->stats.csum_offload_inner++; } else { eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; } We enable inner/outer hardware checksumming unconditionally without looking at the features Alex is suggesting in this patch, Alex, can you elaborate more on the meaning of those features ? and why would it work for us without declaring them ? ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-20 17:40 ` Saeed Mahameed 0 siblings, 0 replies; 50+ messages in thread From: Saeed Mahameed @ 2016-04-20 17:40 UTC (permalink / raw) To: intel-wired-lan On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: > This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 > header fields for length and checksum as well as the length and checksum > fields for outer UDP headers. > > I have no means of testing this as I do not have any mlx5 hardware but > thought I would submit it as an RFC to see if anyone out there wants to > test this and see if this does in fact enable this functionality allowing > us to to segment UDP tunneled frames that have an outer checksum. > > Signed-off-by: Alexander Duyck <aduyck@mirantis.com> > --- > drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index e0adb604f461..57d8da796d50 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) > netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; > > if (mlx5e_vxlan_allowed(mdev)) { > - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; > + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | > + NETIF_F_GSO_UDP_TUNNEL_CSUM | > + NETIF_F_GSO_PARTIAL; > netdev->hw_enc_features |= NETIF_F_IP_CSUM; > netdev->hw_enc_features |= NETIF_F_RXCSUM; > netdev->hw_enc_features |= NETIF_F_TSO; > netdev->hw_enc_features |= NETIF_F_TSO6; > netdev->hw_enc_features |= NETIF_F_RXHASH; > netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; > + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | > + NETIF_F_GSO_PARTIAL; > + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; > } > > netdev->features = netdev->hw_features; > Hi Alex, Adding Matt, VxLAN feature owner from Mellanox, Matt please correct me if am wrong, but We already tested GSO VxLAN and we saw the TCP/IP checksum offloads for both inner and outer headers handled by the hardware. And looking at mlx5e_sq_xmit: if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; if (skb->encapsulation) { eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | MLX5_ETH_WQE_L4_INNER_CSUM; sq->stats.csum_offload_inner++; } else { eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; } We enable inner/outer hardware checksumming unconditionally without looking at the features Alex is suggesting in this patch, Alex, can you elaborate more on the meaning of those features ? and why would it work for us without declaring them ? ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-20 17:40 ` [Intel-wired-lan] " Saeed Mahameed @ 2016-04-20 18:06 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-20 18:06 UTC (permalink / raw) To: Saeed Mahameed Cc: Alexander Duyck, eugenia, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan, Matthew Finlay On Wed, Apr 20, 2016 at 10:40 AM, Saeed Mahameed <saeedm@dev.mellanox.co.il> wrote: > On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >> This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 >> header fields for length and checksum as well as the length and checksum >> fields for outer UDP headers. >> >> I have no means of testing this as I do not have any mlx5 hardware but >> thought I would submit it as an RFC to see if anyone out there wants to >> test this and see if this does in fact enable this functionality allowing >> us to to segment UDP tunneled frames that have an outer checksum. >> >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >> --- >> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> index e0adb604f461..57d8da796d50 100644 >> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) >> netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; >> >> if (mlx5e_vxlan_allowed(mdev)) { >> - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; >> + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | >> + NETIF_F_GSO_UDP_TUNNEL_CSUM | >> + NETIF_F_GSO_PARTIAL; >> netdev->hw_enc_features |= NETIF_F_IP_CSUM; >> netdev->hw_enc_features |= NETIF_F_RXCSUM; >> netdev->hw_enc_features |= NETIF_F_TSO; >> netdev->hw_enc_features |= NETIF_F_TSO6; >> netdev->hw_enc_features |= NETIF_F_RXHASH; >> netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >> + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | >> + NETIF_F_GSO_PARTIAL; >> + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; >> } >> >> netdev->features = netdev->hw_features; >> > > Hi Alex, > > Adding Matt, VxLAN feature owner from Mellanox, > Matt please correct me if am wrong, but We already tested GSO VxLAN > and we saw the TCP/IP checksum offloads for both inner and outer > headers handled by the hardware. > > And looking at mlx5e_sq_xmit: > > if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { > eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; > if (skb->encapsulation) { > eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | > MLX5_ETH_WQE_L4_INNER_CSUM; > sq->stats.csum_offload_inner++; > } else { > eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; > } > > We enable inner/outer hardware checksumming unconditionally without > looking at the features Alex is suggesting in this patch, > Alex, can you elaborate more on the meaning of those features ? and > why would it work for us without declaring them ? Well right now the feature list exposed by the device indicates that TSO is not used if a VxLAN tunnel has a checksum in an outer header. Since that is not exposed currently that is completely offloaded in software via GSO. What the GSO partial does is allow us to treat GSO for tunnels with checksum like it is GSO for tunnels without checksum by precomputing the UDP checksum as though the frame had already been segmented and restricts us to an even multiple of MSS bytes that are to be segmented between all the frames. One side effect though is that all of the IP and UDP header fields are also precomputed, but from what I can tell it looks like the values that would be changed by a change in length are ignored or overwritten by the hardware and driver anyway. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-20 18:06 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-20 18:06 UTC (permalink / raw) To: intel-wired-lan On Wed, Apr 20, 2016 at 10:40 AM, Saeed Mahameed <saeedm@dev.mellanox.co.il> wrote: > On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >> This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 >> header fields for length and checksum as well as the length and checksum >> fields for outer UDP headers. >> >> I have no means of testing this as I do not have any mlx5 hardware but >> thought I would submit it as an RFC to see if anyone out there wants to >> test this and see if this does in fact enable this functionality allowing >> us to to segment UDP tunneled frames that have an outer checksum. >> >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >> --- >> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> index e0adb604f461..57d8da796d50 100644 >> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) >> netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; >> >> if (mlx5e_vxlan_allowed(mdev)) { >> - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; >> + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | >> + NETIF_F_GSO_UDP_TUNNEL_CSUM | >> + NETIF_F_GSO_PARTIAL; >> netdev->hw_enc_features |= NETIF_F_IP_CSUM; >> netdev->hw_enc_features |= NETIF_F_RXCSUM; >> netdev->hw_enc_features |= NETIF_F_TSO; >> netdev->hw_enc_features |= NETIF_F_TSO6; >> netdev->hw_enc_features |= NETIF_F_RXHASH; >> netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >> + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | >> + NETIF_F_GSO_PARTIAL; >> + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; >> } >> >> netdev->features = netdev->hw_features; >> > > Hi Alex, > > Adding Matt, VxLAN feature owner from Mellanox, > Matt please correct me if am wrong, but We already tested GSO VxLAN > and we saw the TCP/IP checksum offloads for both inner and outer > headers handled by the hardware. > > And looking at mlx5e_sq_xmit: > > if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { > eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; > if (skb->encapsulation) { > eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | > MLX5_ETH_WQE_L4_INNER_CSUM; > sq->stats.csum_offload_inner++; > } else { > eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; > } > > We enable inner/outer hardware checksumming unconditionally without > looking at the features Alex is suggesting in this patch, > Alex, can you elaborate more on the meaning of those features ? and > why would it work for us without declaring them ? Well right now the feature list exposed by the device indicates that TSO is not used if a VxLAN tunnel has a checksum in an outer header. Since that is not exposed currently that is completely offloaded in software via GSO. What the GSO partial does is allow us to treat GSO for tunnels with checksum like it is GSO for tunnels without checksum by precomputing the UDP checksum as though the frame had already been segmented and restricts us to an even multiple of MSS bytes that are to be segmented between all the frames. One side effect though is that all of the IP and UDP header fields are also precomputed, but from what I can tell it looks like the values that would be changed by a change in length are ignored or overwritten by the hardware and driver anyway. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-20 18:06 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-28 21:43 ` Matthew Finlay -1 siblings, 0 replies; 50+ messages in thread From: Matthew Finlay @ 2016-04-28 21:43 UTC (permalink / raw) To: Alexander Duyck, Saeed Mahameed Cc: Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan On 4/20/16, 11:06 AM, "Alexander Duyck" <alexander.duyck@gmail.com> wrote: >On Wed, Apr 20, 2016 at 10:40 AM, Saeed Mahameed ><saeedm@dev.mellanox.co.il> wrote: >> On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>> This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 >>> header fields for length and checksum as well as the length and checksum >>> fields for outer UDP headers. >>> >>> I have no means of testing this as I do not have any mlx5 hardware but >>> thought I would submit it as an RFC to see if anyone out there wants to >>> test this and see if this does in fact enable this functionality allowing >>> us to to segment UDP tunneled frames that have an outer checksum. >>> >>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>> --- >>> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- >>> 1 file changed, 6 insertions(+), 1 deletion(-) >>> >>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> index e0adb604f461..57d8da796d50 100644 >>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) >>> netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; >>> >>> if (mlx5e_vxlan_allowed(mdev)) { >>> - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; >>> + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | >>> + NETIF_F_GSO_UDP_TUNNEL_CSUM | >>> + NETIF_F_GSO_PARTIAL; >>> netdev->hw_enc_features |= NETIF_F_IP_CSUM; >>> netdev->hw_enc_features |= NETIF_F_RXCSUM; >>> netdev->hw_enc_features |= NETIF_F_TSO; >>> netdev->hw_enc_features |= NETIF_F_TSO6; >>> netdev->hw_enc_features |= NETIF_F_RXHASH; >>> netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >>> + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | >>> + NETIF_F_GSO_PARTIAL; >>> + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; >>> } >>> >>> netdev->features = netdev->hw_features; >>> >> >> Hi Alex, >> >> Adding Matt, VxLAN feature owner from Mellanox, >> Matt please correct me if am wrong, but We already tested GSO VxLAN >> and we saw the TCP/IP checksum offloads for both inner and outer >> headers handled by the hardware. >> >> And looking at mlx5e_sq_xmit: >> >> if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { >> eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; >> if (skb->encapsulation) { >> eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | >> MLX5_ETH_WQE_L4_INNER_CSUM; >> sq->stats.csum_offload_inner++; >> } else { >> eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; >> } >> >> We enable inner/outer hardware checksumming unconditionally without >> looking at the features Alex is suggesting in this patch, >> Alex, can you elaborate more on the meaning of those features ? and >> why would it work for us without declaring them ? > >Well right now the feature list exposed by the device indicates that >TSO is not used if a VxLAN tunnel has a checksum in an outer header. >Since that is not exposed currently that is completely offloaded in >software via GSO. The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. > >What the GSO partial does is allow us to treat GSO for tunnels with >checksum like it is GSO for tunnels without checksum by precomputing >the UDP checksum as though the frame had already been segmented and >restricts us to an even multiple of MSS bytes that are to be segmented >between all the frames. One side effect though is that all of the IP >and UDP header fields are also precomputed, but from what I can tell >it looks like the values that would be changed by a change in length >are ignored or overwritten by the hardware and driver anyway. > >- Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-28 21:43 ` Matthew Finlay 0 siblings, 0 replies; 50+ messages in thread From: Matthew Finlay @ 2016-04-28 21:43 UTC (permalink / raw) To: intel-wired-lan On 4/20/16, 11:06 AM, "Alexander Duyck" <alexander.duyck@gmail.com> wrote: >On Wed, Apr 20, 2016 at 10:40 AM, Saeed Mahameed ><saeedm@dev.mellanox.co.il> wrote: >> On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>> This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 >>> header fields for length and checksum as well as the length and checksum >>> fields for outer UDP headers. >>> >>> I have no means of testing this as I do not have any mlx5 hardware but >>> thought I would submit it as an RFC to see if anyone out there wants to >>> test this and see if this does in fact enable this functionality allowing >>> us to to segment UDP tunneled frames that have an outer checksum. >>> >>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>> --- >>> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- >>> 1 file changed, 6 insertions(+), 1 deletion(-) >>> >>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> index e0adb604f461..57d8da796d50 100644 >>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) >>> netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; >>> >>> if (mlx5e_vxlan_allowed(mdev)) { >>> - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; >>> + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | >>> + NETIF_F_GSO_UDP_TUNNEL_CSUM | >>> + NETIF_F_GSO_PARTIAL; >>> netdev->hw_enc_features |= NETIF_F_IP_CSUM; >>> netdev->hw_enc_features |= NETIF_F_RXCSUM; >>> netdev->hw_enc_features |= NETIF_F_TSO; >>> netdev->hw_enc_features |= NETIF_F_TSO6; >>> netdev->hw_enc_features |= NETIF_F_RXHASH; >>> netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >>> + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | >>> + NETIF_F_GSO_PARTIAL; >>> + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; >>> } >>> >>> netdev->features = netdev->hw_features; >>> >> >> Hi Alex, >> >> Adding Matt, VxLAN feature owner from Mellanox, >> Matt please correct me if am wrong, but We already tested GSO VxLAN >> and we saw the TCP/IP checksum offloads for both inner and outer >> headers handled by the hardware. >> >> And looking at mlx5e_sq_xmit: >> >> if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { >> eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; >> if (skb->encapsulation) { >> eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | >> MLX5_ETH_WQE_L4_INNER_CSUM; >> sq->stats.csum_offload_inner++; >> } else { >> eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; >> } >> >> We enable inner/outer hardware checksumming unconditionally without >> looking at the features Alex is suggesting in this patch, >> Alex, can you elaborate more on the meaning of those features ? and >> why would it work for us without declaring them ? > >Well right now the feature list exposed by the device indicates that >TSO is not used if a VxLAN tunnel has a checksum in an outer header. >Since that is not exposed currently that is completely offloaded in >software via GSO. The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. > >What the GSO partial does is allow us to treat GSO for tunnels with >checksum like it is GSO for tunnels without checksum by precomputing >the UDP checksum as though the frame had already been segmented and >restricts us to an even multiple of MSS bytes that are to be segmented >between all the frames. One side effect though is that all of the IP >and UDP header fields are also precomputed, but from what I can tell >it looks like the values that would be changed by a change in length >are ignored or overwritten by the hardware and driver anyway. > >- Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-28 21:43 ` [Intel-wired-lan] " Matthew Finlay @ 2016-04-28 22:04 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-28 22:04 UTC (permalink / raw) To: Matthew Finlay Cc: Saeed Mahameed, Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan On Thu, Apr 28, 2016 at 2:43 PM, Matthew Finlay <Matt@mellanox.com> wrote: > > > > > > On 4/20/16, 11:06 AM, "Alexander Duyck" <alexander.duyck@gmail.com> wrote: > >>On Wed, Apr 20, 2016 at 10:40 AM, Saeed Mahameed >><saeedm@dev.mellanox.co.il> wrote: >>> On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>> This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 >>>> header fields for length and checksum as well as the length and checksum >>>> fields for outer UDP headers. >>>> >>>> I have no means of testing this as I do not have any mlx5 hardware but >>>> thought I would submit it as an RFC to see if anyone out there wants to >>>> test this and see if this does in fact enable this functionality allowing >>>> us to to segment UDP tunneled frames that have an outer checksum. >>>> >>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>>> --- >>>> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- >>>> 1 file changed, 6 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>>> index e0adb604f461..57d8da796d50 100644 >>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>>> @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) >>>> netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; >>>> >>>> if (mlx5e_vxlan_allowed(mdev)) { >>>> - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; >>>> + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | >>>> + NETIF_F_GSO_UDP_TUNNEL_CSUM | >>>> + NETIF_F_GSO_PARTIAL; >>>> netdev->hw_enc_features |= NETIF_F_IP_CSUM; >>>> netdev->hw_enc_features |= NETIF_F_RXCSUM; >>>> netdev->hw_enc_features |= NETIF_F_TSO; >>>> netdev->hw_enc_features |= NETIF_F_TSO6; >>>> netdev->hw_enc_features |= NETIF_F_RXHASH; >>>> netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >>>> + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | >>>> + NETIF_F_GSO_PARTIAL; >>>> + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; >>>> } >>>> >>>> netdev->features = netdev->hw_features; >>>> >>> >>> Hi Alex, >>> >>> Adding Matt, VxLAN feature owner from Mellanox, >>> Matt please correct me if am wrong, but We already tested GSO VxLAN >>> and we saw the TCP/IP checksum offloads for both inner and outer >>> headers handled by the hardware. >>> >>> And looking at mlx5e_sq_xmit: >>> >>> if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { >>> eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; >>> if (skb->encapsulation) { >>> eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | >>> MLX5_ETH_WQE_L4_INNER_CSUM; >>> sq->stats.csum_offload_inner++; >>> } else { >>> eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; >>> } >>> >>> We enable inner/outer hardware checksumming unconditionally without >>> looking at the features Alex is suggesting in this patch, >>> Alex, can you elaborate more on the meaning of those features ? and >>> why would it work for us without declaring them ? >> >>Well right now the feature list exposed by the device indicates that >>TSO is not used if a VxLAN tunnel has a checksum in an outer header. >>Since that is not exposed currently that is completely offloaded in >>software via GSO. > > The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. The Intel documentation said the same thing. That was due to the fact that the hardware didn't computer the outer UDP header checksum. I suspect the Mellanox hardware has the same issue. Also I have tested on a ConnectX-4 board with the latest firmware and what I am seeing is that with my patches applied the outer checksum is being correctly applied for segmentation offloads. My thought is that that the hardware appears to ignore the UDP checksum so if it is non-zero you cannot guarantee the checksum would be correct on the last frame if it is a different size than the rest of the segments. In the case of these patches that issue has been resolved as I have precomputed the UDP checksum for the outer UDP header and all of the segments will be the same length so there should be no variation in the UDP checksum of the outer header. Unless you can tell my exactly the reason why we cannot provide the outer UDP checksum I would assume that the reason is due to the fact that the hardware doesn't compute it so you cannot handle a fragment on the end which is resolved already via GSO_PARTIAL. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-28 22:04 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-28 22:04 UTC (permalink / raw) To: intel-wired-lan On Thu, Apr 28, 2016 at 2:43 PM, Matthew Finlay <Matt@mellanox.com> wrote: > > > > > > On 4/20/16, 11:06 AM, "Alexander Duyck" <alexander.duyck@gmail.com> wrote: > >>On Wed, Apr 20, 2016 at 10:40 AM, Saeed Mahameed >><saeedm@dev.mellanox.co.il> wrote: >>> On Tue, Apr 19, 2016 at 10:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>> This patch assumes that the mlx5 hardware will ignore existing IPv4/v6 >>>> header fields for length and checksum as well as the length and checksum >>>> fields for outer UDP headers. >>>> >>>> I have no means of testing this as I do not have any mlx5 hardware but >>>> thought I would submit it as an RFC to see if anyone out there wants to >>>> test this and see if this does in fact enable this functionality allowing >>>> us to to segment UDP tunneled frames that have an outer checksum. >>>> >>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>>> --- >>>> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 ++++++- >>>> 1 file changed, 6 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>>> index e0adb604f461..57d8da796d50 100644 >>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>>> @@ -2390,13 +2390,18 @@ static void mlx5e_build_netdev(struct net_device *netdev) >>>> netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; >>>> >>>> if (mlx5e_vxlan_allowed(mdev)) { >>>> - netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL; >>>> + netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL | >>>> + NETIF_F_GSO_UDP_TUNNEL_CSUM | >>>> + NETIF_F_GSO_PARTIAL; >>>> netdev->hw_enc_features |= NETIF_F_IP_CSUM; >>>> netdev->hw_enc_features |= NETIF_F_RXCSUM; >>>> netdev->hw_enc_features |= NETIF_F_TSO; >>>> netdev->hw_enc_features |= NETIF_F_TSO6; >>>> netdev->hw_enc_features |= NETIF_F_RXHASH; >>>> netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >>>> + netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM | >>>> + NETIF_F_GSO_PARTIAL; >>>> + netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; >>>> } >>>> >>>> netdev->features = netdev->hw_features; >>>> >>> >>> Hi Alex, >>> >>> Adding Matt, VxLAN feature owner from Mellanox, >>> Matt please correct me if am wrong, but We already tested GSO VxLAN >>> and we saw the TCP/IP checksum offloads for both inner and outer >>> headers handled by the hardware. >>> >>> And looking at mlx5e_sq_xmit: >>> >>> if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { >>> eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; >>> if (skb->encapsulation) { >>> eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM | >>> MLX5_ETH_WQE_L4_INNER_CSUM; >>> sq->stats.csum_offload_inner++; >>> } else { >>> eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM; >>> } >>> >>> We enable inner/outer hardware checksumming unconditionally without >>> looking at the features Alex is suggesting in this patch, >>> Alex, can you elaborate more on the meaning of those features ? and >>> why would it work for us without declaring them ? >> >>Well right now the feature list exposed by the device indicates that >>TSO is not used if a VxLAN tunnel has a checksum in an outer header. >>Since that is not exposed currently that is completely offloaded in >>software via GSO. > > The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. The Intel documentation said the same thing. That was due to the fact that the hardware didn't computer the outer UDP header checksum. I suspect the Mellanox hardware has the same issue. Also I have tested on a ConnectX-4 board with the latest firmware and what I am seeing is that with my patches applied the outer checksum is being correctly applied for segmentation offloads. My thought is that that the hardware appears to ignore the UDP checksum so if it is non-zero you cannot guarantee the checksum would be correct on the last frame if it is a different size than the rest of the segments. In the case of these patches that issue has been resolved as I have precomputed the UDP checksum for the outer UDP header and all of the segments will be the same length so there should be no variation in the UDP checksum of the outer header. Unless you can tell my exactly the reason why we cannot provide the outer UDP checksum I would assume that the reason is due to the fact that the hardware doesn't compute it so you cannot handle a fragment on the end which is resolved already via GSO_PARTIAL. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-28 22:04 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-29 1:18 ` Matthew Finlay -1 siblings, 0 replies; 50+ messages in thread From: Matthew Finlay @ 2016-04-29 1:18 UTC (permalink / raw) To: Alexander Duyck Cc: Saeed Mahameed, Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan >> >> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. > >The Intel documentation said the same thing. That was due to the fact >that the hardware didn't computer the outer UDP header checksum. I >suspect the Mellanox hardware has the same issue. Also I have tested >on a ConnectX-4 board with the latest firmware and what I am seeing is >that with my patches applied the outer checksum is being correctly >applied for segmentation offloads. > >My thought is that that the hardware appears to ignore the UDP >checksum so if it is non-zero you cannot guarantee the checksum would >be correct on the last frame if it is a different size than the rest >of the segments. In the case of these patches that issue has been >resolved as I have precomputed the UDP checksum for the outer UDP >header and all of the segments will be the same length so there should >be no variation in the UDP checksum of the outer header. Unless you >can tell my exactly the reason why we cannot provide the outer UDP >checksum I would assume that the reason is due to the fact that the >hardware doesn't compute it so you cannot handle a fragment on the end >which is resolved already via GSO_PARTIAL. I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. > >- Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-29 1:18 ` Matthew Finlay 0 siblings, 0 replies; 50+ messages in thread From: Matthew Finlay @ 2016-04-29 1:18 UTC (permalink / raw) To: intel-wired-lan >> >> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. > >The Intel documentation said the same thing. That was due to the fact >that the hardware didn't computer the outer UDP header checksum. I >suspect the Mellanox hardware has the same issue. Also I have tested >on a ConnectX-4 board with the latest firmware and what I am seeing is >that with my patches applied the outer checksum is being correctly >applied for segmentation offloads. > >My thought is that that the hardware appears to ignore the UDP >checksum so if it is non-zero you cannot guarantee the checksum would >be correct on the last frame if it is a different size than the rest >of the segments. In the case of these patches that issue has been >resolved as I have precomputed the UDP checksum for the outer UDP >header and all of the segments will be the same length so there should >be no variation in the UDP checksum of the outer header. Unless you >can tell my exactly the reason why we cannot provide the outer UDP >checksum I would assume that the reason is due to the fact that the >hardware doesn't compute it so you cannot handle a fragment on the end >which is resolved already via GSO_PARTIAL. I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. > >- Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-29 1:18 ` [Intel-wired-lan] " Matthew Finlay @ 2016-04-29 1:59 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 1:59 UTC (permalink / raw) To: Matthew Finlay Cc: Saeed Mahameed, Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan On Thu, Apr 28, 2016 at 6:18 PM, Matthew Finlay <Matt@mellanox.com> wrote: > > > > > >>> >>> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. >> >>The Intel documentation said the same thing. That was due to the fact >>that the hardware didn't computer the outer UDP header checksum. I >>suspect the Mellanox hardware has the same issue. Also I have tested >>on a ConnectX-4 board with the latest firmware and what I am seeing is >>that with my patches applied the outer checksum is being correctly >>applied for segmentation offloads. >> >>My thought is that that the hardware appears to ignore the UDP >>checksum so if it is non-zero you cannot guarantee the checksum would >>be correct on the last frame if it is a different size than the rest >>of the segments. In the case of these patches that issue has been >>resolved as I have precomputed the UDP checksum for the outer UDP >>header and all of the segments will be the same length so there should >>be no variation in the UDP checksum of the outer header. Unless you >>can tell my exactly the reason why we cannot provide the outer UDP >>checksum I would assume that the reason is due to the fact that the >>hardware doesn't compute it so you cannot handle a fragment on the end >>which is resolved already via GSO_PARTIAL. > > I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. Thanks. Any idea how long it should be. I know I was getting a auto-reply about people being out until May 1st due to a holiday so I am just wondering if we should have Dave drop this patch set and I submit a v2 when you can get me the feedback next week, or if we run with the patches as-is for now and be prepared to revert if anything should come up. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-29 1:59 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 1:59 UTC (permalink / raw) To: intel-wired-lan On Thu, Apr 28, 2016 at 6:18 PM, Matthew Finlay <Matt@mellanox.com> wrote: > > > > > >>> >>> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. >> >>The Intel documentation said the same thing. That was due to the fact >>that the hardware didn't computer the outer UDP header checksum. I >>suspect the Mellanox hardware has the same issue. Also I have tested >>on a ConnectX-4 board with the latest firmware and what I am seeing is >>that with my patches applied the outer checksum is being correctly >>applied for segmentation offloads. >> >>My thought is that that the hardware appears to ignore the UDP >>checksum so if it is non-zero you cannot guarantee the checksum would >>be correct on the last frame if it is a different size than the rest >>of the segments. In the case of these patches that issue has been >>resolved as I have precomputed the UDP checksum for the outer UDP >>header and all of the segments will be the same length so there should >>be no variation in the UDP checksum of the outer header. Unless you >>can tell my exactly the reason why we cannot provide the outer UDP >>checksum I would assume that the reason is due to the fact that the >>hardware doesn't compute it so you cannot handle a fragment on the end >>which is resolved already via GSO_PARTIAL. > > I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. Thanks. Any idea how long it should be. I know I was getting a auto-reply about people being out until May 1st due to a holiday so I am just wondering if we should have Dave drop this patch set and I submit a v2 when you can get me the feedback next week, or if we run with the patches as-is for now and be prepared to revert if anything should come up. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-29 1:59 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-29 19:27 ` Saeed Mahameed -1 siblings, 0 replies; 50+ messages in thread From: Saeed Mahameed @ 2016-04-29 19:27 UTC (permalink / raw) To: Alexander Duyck Cc: Matthew Finlay, Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan On Fri, Apr 29, 2016 at 4:59 AM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > On Thu, Apr 28, 2016 at 6:18 PM, Matthew Finlay <Matt@mellanox.com> wrote: >> >> >> >> >> >>>> >>>> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. >>> >>>The Intel documentation said the same thing. That was due to the fact >>>that the hardware didn't computer the outer UDP header checksum. I >>>suspect the Mellanox hardware has the same issue. Also I have tested >>>on a ConnectX-4 board with the latest firmware and what I am seeing is >>>that with my patches applied the outer checksum is being correctly >>>applied for segmentation offloads. >>> >>>My thought is that that the hardware appears to ignore the UDP >>>checksum so if it is non-zero you cannot guarantee the checksum would >>>be correct on the last frame if it is a different size than the rest >>>of the segments. In the case of these patches that issue has been >>>resolved as I have precomputed the UDP checksum for the outer UDP >>>header and all of the segments will be the same length so there should >>>be no variation in the UDP checksum of the outer header. Unless you >>>can tell my exactly the reason why we cannot provide the outer UDP >>>checksum I would assume that the reason is due to the fact that the >>>hardware doesn't compute it so you cannot handle a fragment on the end >>>which is resolved already via GSO_PARTIAL. >> >> I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. > > Thanks. Any idea how long it should be. I know I was getting a > auto-reply about people being out until May 1st due to a holiday so I > am just wondering if we should have Dave drop this patch set and I > submit a v2 when you can get me the feedback next week, or if we run > with the patches as-is for now and be prepared to revert if anything > should come up. > Alex, I reviewed your patches and other than the claim about mlx4, everything seems to be ok. I took the liberty to apply your patches and play around with them only on mlx5 for now, It seems that it works and HW do correctly calculate the IPv6 outer UDP sum as illustrated in the log from the tcpdump on the receiver [1], I also noticed that the GSO also works on the xmitter and the HW ignores the outer sum as expected. Matt, I think you still need to do the homework and see if we didn't miss anything here, but theoretically and practically what Alex did here should work. Tariq will also check what went wrong with mlx4 outer ipv6 support, but this shouldn't block this series. Side notes: - Alex, you will need to rebase the series, it didn't apply smoothly. - BTW mlx5 on RX side will provide csum complete and not csum unnecessary. Saeed. [1] 21:28:15.474287 e4:1d:2d:5c:f2:e8 > e4:1d:2d:5c:f2:d4, ethertype IPv6 (0x86dd), length 1514: (hlim 64, next-header UDP (17) payload length: 1460) 2001::37:4.58006 > 2001::36:4.4789: [udp sum ok] VXLAN, flags [I] (0x08), vni 46 66:cb:6e:a4:31:4e > b6:bf:9b:61:31:62, ethertype IPv4 (0x0800), length 1444: (tos 0x0, ttl 64, id 64920, offset 0, flags [DF], proto TCP (6), length 1430) 56.0.37.4.53614 > 56.0.36.4.commplex-link: Flags [.], cksum 0xd6cc (correct), seq 1911713070:1911714448, ack 1, win 218, options [nop,nop,TS val 183853011 ecr 183840106], length 1378 ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-29 19:27 ` Saeed Mahameed 0 siblings, 0 replies; 50+ messages in thread From: Saeed Mahameed @ 2016-04-29 19:27 UTC (permalink / raw) To: intel-wired-lan On Fri, Apr 29, 2016 at 4:59 AM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > On Thu, Apr 28, 2016 at 6:18 PM, Matthew Finlay <Matt@mellanox.com> wrote: >> >> >> >> >> >>>> >>>> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. >>> >>>The Intel documentation said the same thing. That was due to the fact >>>that the hardware didn't computer the outer UDP header checksum. I >>>suspect the Mellanox hardware has the same issue. Also I have tested >>>on a ConnectX-4 board with the latest firmware and what I am seeing is >>>that with my patches applied the outer checksum is being correctly >>>applied for segmentation offloads. >>> >>>My thought is that that the hardware appears to ignore the UDP >>>checksum so if it is non-zero you cannot guarantee the checksum would >>>be correct on the last frame if it is a different size than the rest >>>of the segments. In the case of these patches that issue has been >>>resolved as I have precomputed the UDP checksum for the outer UDP >>>header and all of the segments will be the same length so there should >>>be no variation in the UDP checksum of the outer header. Unless you >>>can tell my exactly the reason why we cannot provide the outer UDP >>>checksum I would assume that the reason is due to the fact that the >>>hardware doesn't compute it so you cannot handle a fragment on the end >>>which is resolved already via GSO_PARTIAL. >> >> I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. > > Thanks. Any idea how long it should be. I know I was getting a > auto-reply about people being out until May 1st due to a holiday so I > am just wondering if we should have Dave drop this patch set and I > submit a v2 when you can get me the feedback next week, or if we run > with the patches as-is for now and be prepared to revert if anything > should come up. > Alex, I reviewed your patches and other than the claim about mlx4, everything seems to be ok. I took the liberty to apply your patches and play around with them only on mlx5 for now, It seems that it works and HW do correctly calculate the IPv6 outer UDP sum as illustrated in the log from the tcpdump on the receiver [1], I also noticed that the GSO also works on the xmitter and the HW ignores the outer sum as expected. Matt, I think you still need to do the homework and see if we didn't miss anything here, but theoretically and practically what Alex did here should work. Tariq will also check what went wrong with mlx4 outer ipv6 support, but this shouldn't block this series. Side notes: - Alex, you will need to rebase the series, it didn't apply smoothly. - BTW mlx5 on RX side will provide csum complete and not csum unnecessary. Saeed. [1] 21:28:15.474287 e4:1d:2d:5c:f2:e8 > e4:1d:2d:5c:f2:d4, ethertype IPv6 (0x86dd), length 1514: (hlim 64, next-header UDP (17) payload length: 1460) 2001::37:4.58006 > 2001::36:4.4789: [udp sum ok] VXLAN, flags [I] (0x08), vni 46 66:cb:6e:a4:31:4e > b6:bf:9b:61:31:62, ethertype IPv4 (0x0800), length 1444: (tos 0x0, ttl 64, id 64920, offset 0, flags [DF], proto TCP (6), length 1430) 56.0.37.4.53614 > 56.0.36.4.commplex-link: Flags [.], cksum 0xd6cc (correct), seq 1911713070:1911714448, ack 1, win 218, options [nop,nop,TS val 183853011 ecr 183840106], length 1378 ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-29 19:27 ` [Intel-wired-lan] " Saeed Mahameed @ 2016-04-29 19:36 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 19:36 UTC (permalink / raw) To: Saeed Mahameed Cc: Matthew Finlay, Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Linux Netdev List, intel-wired-lan, Ariel Elior, Michael Chan On Fri, Apr 29, 2016 at 12:27 PM, Saeed Mahameed <saeedm@dev.mellanox.co.il> wrote: > On Fri, Apr 29, 2016 at 4:59 AM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: >> On Thu, Apr 28, 2016 at 6:18 PM, Matthew Finlay <Matt@mellanox.com> wrote: >>> >>> >>> >>> >>> >>>>> >>>>> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. >>>> >>>>The Intel documentation said the same thing. That was due to the fact >>>>that the hardware didn't computer the outer UDP header checksum. I >>>>suspect the Mellanox hardware has the same issue. Also I have tested >>>>on a ConnectX-4 board with the latest firmware and what I am seeing is >>>>that with my patches applied the outer checksum is being correctly >>>>applied for segmentation offloads. >>>> >>>>My thought is that that the hardware appears to ignore the UDP >>>>checksum so if it is non-zero you cannot guarantee the checksum would >>>>be correct on the last frame if it is a different size than the rest >>>>of the segments. In the case of these patches that issue has been >>>>resolved as I have precomputed the UDP checksum for the outer UDP >>>>header and all of the segments will be the same length so there should >>>>be no variation in the UDP checksum of the outer header. Unless you >>>>can tell my exactly the reason why we cannot provide the outer UDP >>>>checksum I would assume that the reason is due to the fact that the >>>>hardware doesn't compute it so you cannot handle a fragment on the end >>>>which is resolved already via GSO_PARTIAL. >>> >>> I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. >> >> Thanks. Any idea how long it should be. I know I was getting a >> auto-reply about people being out until May 1st due to a holiday so I >> am just wondering if we should have Dave drop this patch set and I >> submit a v2 when you can get me the feedback next week, or if we run >> with the patches as-is for now and be prepared to revert if anything >> should come up. >> > > Alex, > > I reviewed your patches and other than the claim about mlx4, > everything seems to be ok. > I took the liberty to apply your patches and play around with them > only on mlx5 for now, > It seems that it works and HW do correctly calculate the IPv6 outer > UDP sum as illustrated in the log from the tcpdump on the receiver > [1], I also noticed that the GSO also works on the xmitter and the HW > ignores the outer sum as expected. > > Matt, I think you still need to do the homework and see if we didn't > miss anything here, but theoretically and practically what Alex did > here should work. > Tariq will also check what went wrong with mlx4 outer ipv6 support, > but this shouldn't block this series. > > Side notes: > - Alex, you will need to rebase the series, it didn't apply smoothly. > - BTW mlx5 on RX side will provide csum complete and not csum unnecessary. > > Saeed. > > [1] > 21:28:15.474287 e4:1d:2d:5c:f2:e8 > e4:1d:2d:5c:f2:d4, ethertype IPv6 > (0x86dd), length 1514: (hlim 64, next-header UDP (17) payload length: > 1460) 2001::37:4.58006 > 2001::36:4.4789: [udp sum ok] VXLAN, flags > [I] (0x08), vni 46 > 66:cb:6e:a4:31:4e > b6:bf:9b:61:31:62, ethertype IPv4 (0x0800), length > 1444: (tos 0x0, ttl 64, id 64920, offset 0, flags [DF], proto TCP (6), > length 1430) > 56.0.37.4.53614 > 56.0.36.4.commplex-link: Flags [.], cksum 0xd6cc > (correct), seq 1911713070:1911714448, ack 1, win 218, options > [nop,nop,TS val 183853011 ecr 183840106], length 1378 Thanks. I'll probably resubmit next week after your latest set of 12 patches gets applied. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 2/5] mlx5: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-29 19:36 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 19:36 UTC (permalink / raw) To: intel-wired-lan On Fri, Apr 29, 2016 at 12:27 PM, Saeed Mahameed <saeedm@dev.mellanox.co.il> wrote: > On Fri, Apr 29, 2016 at 4:59 AM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: >> On Thu, Apr 28, 2016 at 6:18 PM, Matthew Finlay <Matt@mellanox.com> wrote: >>> >>> >>> >>> >>> >>>>> >>>>> The mlx5 hardware requires the outer UDP checksum is not set when offloading encapsulated packets. >>>> >>>>The Intel documentation said the same thing. That was due to the fact >>>>that the hardware didn't computer the outer UDP header checksum. I >>>>suspect the Mellanox hardware has the same issue. Also I have tested >>>>on a ConnectX-4 board with the latest firmware and what I am seeing is >>>>that with my patches applied the outer checksum is being correctly >>>>applied for segmentation offloads. >>>> >>>>My thought is that that the hardware appears to ignore the UDP >>>>checksum so if it is non-zero you cannot guarantee the checksum would >>>>be correct on the last frame if it is a different size than the rest >>>>of the segments. In the case of these patches that issue has been >>>>resolved as I have precomputed the UDP checksum for the outer UDP >>>>header and all of the segments will be the same length so there should >>>>be no variation in the UDP checksum of the outer header. Unless you >>>>can tell my exactly the reason why we cannot provide the outer UDP >>>>checksum I would assume that the reason is due to the fact that the >>>>hardware doesn't compute it so you cannot handle a fragment on the end >>>>which is resolved already via GSO_PARTIAL. >>> >>> I will check internally and verify there are no unforeseen issues with setting the outer UDP checksum in this scenario. >> >> Thanks. Any idea how long it should be. I know I was getting a >> auto-reply about people being out until May 1st due to a holiday so I >> am just wondering if we should have Dave drop this patch set and I >> submit a v2 when you can get me the feedback next week, or if we run >> with the patches as-is for now and be prepared to revert if anything >> should come up. >> > > Alex, > > I reviewed your patches and other than the claim about mlx4, > everything seems to be ok. > I took the liberty to apply your patches and play around with them > only on mlx5 for now, > It seems that it works and HW do correctly calculate the IPv6 outer > UDP sum as illustrated in the log from the tcpdump on the receiver > [1], I also noticed that the GSO also works on the xmitter and the HW > ignores the outer sum as expected. > > Matt, I think you still need to do the homework and see if we didn't > miss anything here, but theoretically and practically what Alex did > here should work. > Tariq will also check what went wrong with mlx4 outer ipv6 support, > but this shouldn't block this series. > > Side notes: > - Alex, you will need to rebase the series, it didn't apply smoothly. > - BTW mlx5 on RX side will provide csum complete and not csum unnecessary. > > Saeed. > > [1] > 21:28:15.474287 e4:1d:2d:5c:f2:e8 > e4:1d:2d:5c:f2:d4, ethertype IPv6 > (0x86dd), length 1514: (hlim 64, next-header UDP (17) payload length: > 1460) 2001::37:4.58006 > 2001::36:4.4789: [udp sum ok] VXLAN, flags > [I] (0x08), vni 46 > 66:cb:6e:a4:31:4e > b6:bf:9b:61:31:62, ethertype IPv4 (0x0800), length > 1444: (tos 0x0, ttl 64, id 64920, offset 0, flags [DF], proto TCP (6), > length 1430) > 56.0.37.4.53614 > 56.0.36.4.commplex-link: Flags [.], cksum 0xd6cc > (correct), seq 1911713070:1911714448, ack 1, win 218, options > [nop,nop,TS val 183853011 ecr 183840106], length 1378 Thanks. I'll probably resubmit next week after your latest set of 12 patches gets applied. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-19 19:06 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, mchan This patch assumes that the bnx2x hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP and GRE headers. I have no means of testing this as I do not have any bnx2x hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c index d465bd721146..e3459524fb0e 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c @@ -13258,14 +13258,21 @@ static int bnx2x_init_dev(struct bnx2x *bp, struct pci_dev *pdev, NETIF_F_RXCSUM | NETIF_F_LRO | NETIF_F_GRO | NETIF_F_RXHASH | NETIF_F_HW_VLAN_CTAG_TX; if (!chip_is_e1x) { - dev->hw_features |= NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL | - NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT; + dev->hw_features |= NETIF_F_GSO_GRE | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | + NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; dev->hw_enc_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6 | NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | - NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL; + NETIF_F_GSO_GRE | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + dev->gso_partial_features = NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_UDP_TUNNEL_CSUM; } dev->vlan_features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | ^ permalink raw reply related [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums @ 2016-04-19 19:06 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: intel-wired-lan This patch assumes that the bnx2x hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP and GRE headers. I have no means of testing this as I do not have any bnx2x hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c index d465bd721146..e3459524fb0e 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c @@ -13258,14 +13258,21 @@ static int bnx2x_init_dev(struct bnx2x *bp, struct pci_dev *pdev, NETIF_F_RXCSUM | NETIF_F_LRO | NETIF_F_GRO | NETIF_F_RXHASH | NETIF_F_HW_VLAN_CTAG_TX; if (!chip_is_e1x) { - dev->hw_features |= NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL | - NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT; + dev->hw_features |= NETIF_F_GSO_GRE | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | + NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; dev->hw_enc_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6 | NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | - NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL; + NETIF_F_GSO_GRE | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + dev->gso_partial_features = NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_UDP_TUNNEL_CSUM; } dev->vlan_features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | ^ permalink raw reply related [flat|nested] 50+ messages in thread
* RE: [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck (?) @ 2016-08-24 12:33 ` Yuval Mintz 2016-08-24 15:39 ` Alexander Duyck -1 siblings, 1 reply; 50+ messages in thread From: Yuval Mintz @ 2016-08-24 12:33 UTC (permalink / raw) To: Alexander Duyck, netdev > This patch assumes that the bnx2x hardware will ignore existing IPv4/v6 header > fields for length and checksum as well as the length and checksum fields for > outer UDP and GRE headers. > > I have no means of testing this as I do not have any bnx2x hardware but thought > I would submit it as an RFC to see if anyone out there wants to test this and see > if this does in fact enable this functionality allowing us to to segment tunneled > frames that have an outer checksum. > > Signed-off-by: Alexander Duyck <aduyck@mirantis.com> So it took me some [well, a lot] time to reach this, but I've finally gave it a try. I saw a performance boost with the partial support - Throughput for vxlan tunnels with and without udpcsum were almost identical after this, whereas without this patch the udpcsum prevented GSO and a TCP/IPv4 connection on top of it got roughly half the throughput. However, I did encounter one oddity I couldn't explain - After I've disabled tx-udp_tnl-segmentation via ethtool on the base interface, got left with: tx-gso-partial: on tx-udp_tnl-segmentation: off tx-udp_tnl-csum-segmentation: on When I ran traffic over both vxlan tunnels the one with the udpcsum was still Passing gso aggregations to base device to transmit [and the throughput was same as before], where's the tunnel without the udpcsum showed only MTU-sized packets reaching the base interface for transmission [which is what I've expected] Any idea why that happened? ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-08-24 12:33 ` Yuval Mintz @ 2016-08-24 15:39 ` Alexander Duyck 2016-08-25 5:06 ` Yuval Mintz 0 siblings, 1 reply; 50+ messages in thread From: Alexander Duyck @ 2016-08-24 15:39 UTC (permalink / raw) To: Yuval Mintz; +Cc: netdev On Wed, Aug 24, 2016 at 5:33 AM, Yuval Mintz <Yuval.Mintz@qlogic.com> wrote: >> This patch assumes that the bnx2x hardware will ignore existing IPv4/v6 header >> fields for length and checksum as well as the length and checksum fields for >> outer UDP and GRE headers. >> >> I have no means of testing this as I do not have any bnx2x hardware but thought >> I would submit it as an RFC to see if anyone out there wants to test this and see >> if this does in fact enable this functionality allowing us to to segment tunneled >> frames that have an outer checksum. >> >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> > > So it took me some [well, a lot] time to reach this, but I've finally gave it a try. > I saw a performance boost with the partial support - > Throughput for vxlan tunnels with and without udpcsum were almost identical > after this, whereas without this patch the udpcsum prevented GSO and > a TCP/IPv4 connection on top of it got roughly half the throughput. > > However, I did encounter one oddity I couldn't explain - > After I've disabled tx-udp_tnl-segmentation via ethtool on the base interface, > got left with: > tx-gso-partial: on > tx-udp_tnl-segmentation: off > tx-udp_tnl-csum-segmentation: on > > When I ran traffic over both vxlan tunnels the one with the udpcsum was still > Passing gso aggregations to base device to transmit [and the throughput was > same as before], where's the tunnel without the udpcsum showed only > MTU-sized packets reaching the base interface for transmission [which is what > I've expected] > > Any idea why that happened? So the way they are implemented tx-udp_tnl-segmentation and tx-udp_tnl-csum-segmentation are treated as two separate features. The kernel currently gives them the same treatment as NETIF_F_TSO and NETIF_F_TSO6. You can disable one and the other still functions. Now if you disable tx-gso-partial you should expect to see tx-udp_tnl-csum-segmentation be disabled because it is dependent on the partial GSO offload. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* RE: [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-08-24 15:39 ` Alexander Duyck @ 2016-08-25 5:06 ` Yuval Mintz 2016-08-25 18:26 ` Alexander Duyck 0 siblings, 1 reply; 50+ messages in thread From: Yuval Mintz @ 2016-08-25 5:06 UTC (permalink / raw) To: Alexander Duyck; +Cc: netdev > >> This patch assumes that the bnx2x hardware will ignore existing > >> IPv4/v6 header fields for length and checksum as well as the length > >> and checksum fields for outer UDP and GRE headers. > >> > >> I have no means of testing this as I do not have any bnx2x hardware > >> but thought I would submit it as an RFC to see if anyone out there > >> wants to test this and see if this does in fact enable this > >> functionality allowing us to to segment tunneled frames that have an outer > checksum. > >> > >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> > > > > So it took me some [well, a lot] time to reach this, but I've finally gave it a try. > > I saw a performance boost with the partial support - Throughput for > > vxlan tunnels with and without udpcsum were almost identical after > > this, whereas without this patch the udpcsum prevented GSO and a > > TCP/IPv4 connection on top of it got roughly half the throughput. > > > > However, I did encounter one oddity I couldn't explain - After I've > > disabled tx-udp_tnl-segmentation via ethtool on the base interface, > > got left with: > > tx-gso-partial: on > > tx-udp_tnl-segmentation: off > > tx-udp_tnl-csum-segmentation: on > > > > When I ran traffic over both vxlan tunnels the one with the udpcsum > > was still Passing gso aggregations to base device to transmit [and the > > throughput was same as before], where's the tunnel without the udpcsum > > showed only MTU-sized packets reaching the base interface for > > transmission [which is what I've expected] > > > > Any idea why that happened? > > So the way they are implemented tx-udp_tnl-segmentation and tx-udp_tnl- > csum-segmentation are treated as two separate features. > The kernel currently gives them the same treatment as NETIF_F_TSO and > NETIF_F_TSO6. You can disable one and the other still functions. > > Now if you disable tx-gso-partial you should expect to see tx-udp_tnl-csum- > segmentation be disabled because it is dependent on the partial GSO offload. > > - Alex O.k., thanks. Then I'll run some more testing scenarios, and assuming everything works fine I'll re-send this. Alex - should I place you at the 'from' field? ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-08-25 5:06 ` Yuval Mintz @ 2016-08-25 18:26 ` Alexander Duyck 2016-08-31 11:31 ` Yuval Mintz 0 siblings, 1 reply; 50+ messages in thread From: Alexander Duyck @ 2016-08-25 18:26 UTC (permalink / raw) To: Yuval Mintz; +Cc: netdev On Wed, Aug 24, 2016 at 10:06 PM, Yuval Mintz <Yuval.Mintz@qlogic.com> wrote: >> >> This patch assumes that the bnx2x hardware will ignore existing >> >> IPv4/v6 header fields for length and checksum as well as the length >> >> and checksum fields for outer UDP and GRE headers. >> >> >> >> I have no means of testing this as I do not have any bnx2x hardware >> >> but thought I would submit it as an RFC to see if anyone out there >> >> wants to test this and see if this does in fact enable this >> >> functionality allowing us to to segment tunneled frames that have an outer >> checksum. >> >> >> >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >> > >> > So it took me some [well, a lot] time to reach this, but I've finally gave it a try. >> > I saw a performance boost with the partial support - Throughput for >> > vxlan tunnels with and without udpcsum were almost identical after >> > this, whereas without this patch the udpcsum prevented GSO and a >> > TCP/IPv4 connection on top of it got roughly half the throughput. >> > >> > However, I did encounter one oddity I couldn't explain - After I've >> > disabled tx-udp_tnl-segmentation via ethtool on the base interface, >> > got left with: >> > tx-gso-partial: on >> > tx-udp_tnl-segmentation: off >> > tx-udp_tnl-csum-segmentation: on >> > >> > When I ran traffic over both vxlan tunnels the one with the udpcsum >> > was still Passing gso aggregations to base device to transmit [and the >> > throughput was same as before], where's the tunnel without the udpcsum >> > showed only MTU-sized packets reaching the base interface for >> > transmission [which is what I've expected] >> > >> > Any idea why that happened? >> >> So the way they are implemented tx-udp_tnl-segmentation and tx-udp_tnl- >> csum-segmentation are treated as two separate features. >> The kernel currently gives them the same treatment as NETIF_F_TSO and >> NETIF_F_TSO6. You can disable one and the other still functions. >> >> Now if you disable tx-gso-partial you should expect to see tx-udp_tnl-csum- >> segmentation be disabled because it is dependent on the partial GSO offload. >> >> - Alex > > O.k., thanks. > Then I'll run some more testing scenarios, and assuming everything works > fine I'll re-send this. Alex - should I place you at the 'from' field? If you could do that, then that would be great. I'm not working at Mirantis anymore, but I don't believe that the account is setup to bounce the email so it shouldn't generate any noise for Dave. If not you can just leave my original signed off by and add yours if you want. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* RE: [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-08-25 18:26 ` Alexander Duyck @ 2016-08-31 11:31 ` Yuval Mintz 2016-08-31 16:13 ` Alexander Duyck 0 siblings, 1 reply; 50+ messages in thread From: Yuval Mintz @ 2016-08-31 11:31 UTC (permalink / raw) To: Alexander Duyck; +Cc: netdev One question I have regarding the feature, regarding the partial offload compatible with ndo_features_check(). Consider the following example - Let's assume my adapter is capable of doing outer-csum validation for vxlan packets, but only if inner network protocol is IPv4, while at the same time it's capable of doing gso offloading for any vxlan-encapsulated packet. In that case, I'll publish NETIF_F_GSO_UDP_TUNNEL_CSUM as part of my offload capability [& encap capabilities], and in my driver's implementation of ndo_features_check() I'd remove the feature in case I'd find out SKB is vxlan whose inner network protocol is ipv6. Is there a way I could have benefited from the partial offload in that case? If I understand the feature correctly, if I were to publish UDP_TUNNEL_CSUM under partial-gso, it would mean that stack would peel the feature off until it would decide there's actual need for SW GSO, thereby denying me the ability of utilizing the HW offload capability for CSUM. Am I reading it wrong? Or does this trade-off exist? ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums 2016-08-31 11:31 ` Yuval Mintz @ 2016-08-31 16:13 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-08-31 16:13 UTC (permalink / raw) To: Yuval Mintz; +Cc: netdev On Wed, Aug 31, 2016 at 4:31 AM, Yuval Mintz <Yuval.Mintz@qlogic.com> wrote: > One question I have regarding the feature, regarding the > partial offload compatible with ndo_features_check(). > > Consider the following example - > Let's assume my adapter is capable of doing outer-csum validation > for vxlan packets, but only if inner network protocol is IPv4, > while at the same time it's capable of doing gso offloading for any > vxlan-encapsulated packet. When you say validation do you mean generating an outer checksum? If I recall ndo_features_check applies to Tx features not Rx features. I'll assume you mean Tx checksum offload and not validation. > In that case, I'll publish NETIF_F_GSO_UDP_TUNNEL_CSUM as > part of my offload capability [& encap capabilities], and in my driver's > implementation of ndo_features_check() I'd remove the feature in case > I'd find out SKB is vxlan whose inner network protocol is ipv6. Right. That sounds correct so far. > Is there a way I could have benefited from the partial offload in that > case? If I understand the feature correctly, if I were to publish > UDP_TUNNEL_CSUM under partial-gso, it would mean that stack would > peel the feature off until it would decide there's actual need for SW GSO, > thereby denying me the ability of utilizing the HW offload capability for CSUM. > > Am I reading it wrong? Or does this trade-off exist? I think you may have that correct. Basically any feature you advertise as partially supporting via GSO_PARTIAL ends up being tied to the partial offload after that. Although from what I have seen the difference isn't really all that great between GSO_PARTIAL versus TSO. In many cases the stack generates MSS aligned frames anyway so all GSO_PARTIAL is really doing is unsharing the header and updating the header fields. ^ permalink raw reply [flat|nested] 50+ messages in thread
* [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-19 19:06 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, mchan This patch assumes that the bnxt hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP and GRE headers. I have no means of testing this as I do not have any bnx2x hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 4645c44e7c15..ae668476fff0 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -6194,14 +6194,19 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_GRE | NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | - NETIF_F_RXHASH | + NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_PARTIAL | NETIF_F_RXHASH | NETIF_F_RXCSUM | NETIF_F_LRO | NETIF_F_GRO; dev->hw_enc_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_GRE | - NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT; + NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | + NETIF_F_GSO_PARTIAL; + dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_GRE_CSUM; dev->vlan_features = dev->hw_features | NETIF_F_HIGHDMA; dev->hw_features |= NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_STAG_TX; ^ permalink raw reply related [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-19 19:06 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: intel-wired-lan This patch assumes that the bnxt hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP and GRE headers. I have no means of testing this as I do not have any bnx2x hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 4645c44e7c15..ae668476fff0 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -6194,14 +6194,19 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_GRE | NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | - NETIF_F_RXHASH | + NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_PARTIAL | NETIF_F_RXHASH | NETIF_F_RXCSUM | NETIF_F_LRO | NETIF_F_GRO; dev->hw_enc_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_GRE | - NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT; + NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_GSO_GRE_CSUM | + NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT | + NETIF_F_GSO_PARTIAL; + dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_GRE_CSUM; dev->vlan_features = dev->hw_features | NETIF_F_HIGHDMA; dev->hw_features |= NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_STAG_TX; ^ permalink raw reply related [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-27 5:55 ` Michael Chan -1 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-27 5:55 UTC (permalink / raw) To: Alexander Duyck Cc: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, Michael Chan On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: > This patch assumes that the bnxt hardware will ignore existing IPv4/v6 > header fields for length and checksum as well as the length and checksum > fields for outer UDP and GRE headers. > > I have no means of testing this as I do not have any bnx2x hardware but > thought I would submit it as an RFC to see if anyone out there wants to > test this and see if this does in fact enable this functionality allowing > us to to segment tunneled frames that have an outer checksum. > > Signed-off-by: Alexander Duyck <aduyck@mirantis.com> Hi Alex, I just did a very quick test of this patch on our bnxt hardware and it seemed to work. I created a vxlan endpoint with udpcsum enabled and I saw TSO packets getting through. I've verified that our hardware can be programmed to either ignore outer UDP checksum or to calculate it. Current default is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. Thanks. > --- > drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-27 5:55 ` Michael Chan 0 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-27 5:55 UTC (permalink / raw) To: intel-wired-lan On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: > This patch assumes that the bnxt hardware will ignore existing IPv4/v6 > header fields for length and checksum as well as the length and checksum > fields for outer UDP and GRE headers. > > I have no means of testing this as I do not have any bnx2x hardware but > thought I would submit it as an RFC to see if anyone out there wants to > test this and see if this does in fact enable this functionality allowing > us to to segment tunneled frames that have an outer checksum. > > Signed-off-by: Alexander Duyck <aduyck@mirantis.com> Hi Alex, I just did a very quick test of this patch on our bnxt hardware and it seemed to work. I created a vxlan endpoint with udpcsum enabled and I saw TSO packets getting through. I've verified that our hardware can be programmed to either ignore outer UDP checksum or to calculate it. Current default is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. Thanks. > --- > drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-27 5:55 ` [Intel-wired-lan] " Michael Chan @ 2016-04-27 15:21 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-27 15:21 UTC (permalink / raw) To: Michael Chan Cc: Alexander Duyck, eugenia, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >> header fields for length and checksum as well as the length and checksum >> fields for outer UDP and GRE headers. >> >> I have no means of testing this as I do not have any bnx2x hardware but >> thought I would submit it as an RFC to see if anyone out there wants to >> test this and see if this does in fact enable this functionality allowing >> us to to segment tunneled frames that have an outer checksum. >> >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> > > Hi Alex, I just did a very quick test of this patch on our bnxt > hardware and it seemed to work. > > I created a vxlan endpoint with udpcsum enabled and I saw TSO packets > getting through. I've verified that our hardware can be programmed to > either ignore outer UDP checksum or to calculate it. Current default > is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. > Thanks. Are you saying you can natively support UDP tunnel with outer checksum offload then? I'm just trying to sort out if you actually need to have the partial segmentation offload support or if we can handle it in hardware. Also is there any documentation you could point me to that might help to clarify what the hardware does/doesn't support so that I could improve upon this patch in order to make sure we are getting the most bang for the buck in terms of the features that can be offloaded by hardware? Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-27 15:21 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-27 15:21 UTC (permalink / raw) To: intel-wired-lan On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >> header fields for length and checksum as well as the length and checksum >> fields for outer UDP and GRE headers. >> >> I have no means of testing this as I do not have any bnx2x hardware but >> thought I would submit it as an RFC to see if anyone out there wants to >> test this and see if this does in fact enable this functionality allowing >> us to to segment tunneled frames that have an outer checksum. >> >> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> > > Hi Alex, I just did a very quick test of this patch on our bnxt > hardware and it seemed to work. > > I created a vxlan endpoint with udpcsum enabled and I saw TSO packets > getting through. I've verified that our hardware can be programmed to > either ignore outer UDP checksum or to calculate it. Current default > is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. > Thanks. Are you saying you can natively support UDP tunnel with outer checksum offload then? I'm just trying to sort out if you actually need to have the partial segmentation offload support or if we can handle it in hardware. Also is there any documentation you could point me to that might help to clarify what the hardware does/doesn't support so that I could improve upon this patch in order to make sure we are getting the most bang for the buck in terms of the features that can be offloaded by hardware? Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-27 15:21 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-28 4:32 ` Michael Chan -1 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-28 4:32 UTC (permalink / raw) To: Alexander Duyck Cc: Alexander Duyck, eugenia, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan > <michael.chan@broadcom.com> wrote: >> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>> header fields for length and checksum as well as the length and checksum >>> fields for outer UDP and GRE headers. >>> >>> I have no means of testing this as I do not have any bnx2x hardware but >>> thought I would submit it as an RFC to see if anyone out there wants to >>> test this and see if this does in fact enable this functionality allowing >>> us to to segment tunneled frames that have an outer checksum. >>> >>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >> >> Hi Alex, I just did a very quick test of this patch on our bnxt >> hardware and it seemed to work. >> >> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >> getting through. I've verified that our hardware can be programmed to >> either ignore outer UDP checksum or to calculate it. Current default >> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >> Thanks. > > Are you saying you can natively support UDP tunnel with outer checksum > offload then? Yes. Calculate or ignore the outer UDP checksum. > > I'm just trying to sort out if you actually need to have the partial > segmentation offload support or if we can handle it in hardware. Also > is there any documentation you could point me to that might help to > clarify what the hardware does/doesn't support so that I could improve > upon this patch in order to make sure we are getting the most bang for > the buck in terms of the features that can be offloaded by hardware? No public documentation yet. I think the plan is to publish the programmer's reference on our website at some point in the future. ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-28 4:32 ` Michael Chan 0 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-28 4:32 UTC (permalink / raw) To: intel-wired-lan On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan > <michael.chan@broadcom.com> wrote: >> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>> header fields for length and checksum as well as the length and checksum >>> fields for outer UDP and GRE headers. >>> >>> I have no means of testing this as I do not have any bnx2x hardware but >>> thought I would submit it as an RFC to see if anyone out there wants to >>> test this and see if this does in fact enable this functionality allowing >>> us to to segment tunneled frames that have an outer checksum. >>> >>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >> >> Hi Alex, I just did a very quick test of this patch on our bnxt >> hardware and it seemed to work. >> >> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >> getting through. I've verified that our hardware can be programmed to >> either ignore outer UDP checksum or to calculate it. Current default >> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >> Thanks. > > Are you saying you can natively support UDP tunnel with outer checksum > offload then? Yes. Calculate or ignore the outer UDP checksum. > > I'm just trying to sort out if you actually need to have the partial > segmentation offload support or if we can handle it in hardware. Also > is there any documentation you could point me to that might help to > clarify what the hardware does/doesn't support so that I could improve > upon this patch in order to make sure we are getting the most bang for > the buck in terms of the features that can be offloaded by hardware? No public documentation yet. I think the plan is to publish the programmer's reference on our website at some point in the future. ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-28 4:32 ` [Intel-wired-lan] " Michael Chan @ 2016-04-29 21:17 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 21:17 UTC (permalink / raw) To: Michael Chan Cc: Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Wed, Apr 27, 2016 at 9:32 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: >> On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan >> <michael.chan@broadcom.com> wrote: >>> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>>> header fields for length and checksum as well as the length and checksum >>>> fields for outer UDP and GRE headers. >>>> >>>> I have no means of testing this as I do not have any bnx2x hardware but >>>> thought I would submit it as an RFC to see if anyone out there wants to >>>> test this and see if this does in fact enable this functionality allowing >>>> us to to segment tunneled frames that have an outer checksum. >>>> >>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>> >>> Hi Alex, I just did a very quick test of this patch on our bnxt >>> hardware and it seemed to work. >>> >>> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >>> getting through. I've verified that our hardware can be programmed to >>> either ignore outer UDP checksum or to calculate it. Current default >>> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >>> Thanks. >> >> Are you saying you can natively support UDP tunnel with outer checksum >> offload then? > > Yes. Calculate or ignore the outer UDP checksum. I was just thinking about this. When you say you compute the IPv6 checksum how is it you are specifying to the hardware that you want to do that? Is it something you can configure per packet or is it something that is configured for the VXLAN flow? I just want to make sure you aren't adding checksums to IPv6 tunnels that specify that they don't want a checksum, or stripping them from v4 tunnels that do want a checksum. Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-29 21:17 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 21:17 UTC (permalink / raw) To: intel-wired-lan On Wed, Apr 27, 2016 at 9:32 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: >> On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan >> <michael.chan@broadcom.com> wrote: >>> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>>> header fields for length and checksum as well as the length and checksum >>>> fields for outer UDP and GRE headers. >>>> >>>> I have no means of testing this as I do not have any bnx2x hardware but >>>> thought I would submit it as an RFC to see if anyone out there wants to >>>> test this and see if this does in fact enable this functionality allowing >>>> us to to segment tunneled frames that have an outer checksum. >>>> >>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>> >>> Hi Alex, I just did a very quick test of this patch on our bnxt >>> hardware and it seemed to work. >>> >>> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >>> getting through. I've verified that our hardware can be programmed to >>> either ignore outer UDP checksum or to calculate it. Current default >>> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >>> Thanks. >> >> Are you saying you can natively support UDP tunnel with outer checksum >> offload then? > > Yes. Calculate or ignore the outer UDP checksum. I was just thinking about this. When you say you compute the IPv6 checksum how is it you are specifying to the hardware that you want to do that? Is it something you can configure per packet or is it something that is configured for the VXLAN flow? I just want to make sure you aren't adding checksums to IPv6 tunnels that specify that they don't want a checksum, or stripping them from v4 tunnels that do want a checksum. Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-29 21:17 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-29 21:29 ` Michael Chan -1 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-29 21:29 UTC (permalink / raw) To: Alexander Duyck Cc: Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Fri, Apr 29, 2016 at 2:17 PM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > On Wed, Apr 27, 2016 at 9:32 PM, Michael Chan <michael.chan@broadcom.com> wrote: >> On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck >> <alexander.duyck@gmail.com> wrote: >>> On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan >>> <michael.chan@broadcom.com> wrote: >>>> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>>>> header fields for length and checksum as well as the length and checksum >>>>> fields for outer UDP and GRE headers. >>>>> >>>>> I have no means of testing this as I do not have any bnx2x hardware but >>>>> thought I would submit it as an RFC to see if anyone out there wants to >>>>> test this and see if this does in fact enable this functionality allowing >>>>> us to to segment tunneled frames that have an outer checksum. >>>>> >>>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>>> >>>> Hi Alex, I just did a very quick test of this patch on our bnxt >>>> hardware and it seemed to work. >>>> >>>> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >>>> getting through. I've verified that our hardware can be programmed to >>>> either ignore outer UDP checksum or to calculate it. Current default >>>> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >>>> Thanks. >>> >>> Are you saying you can natively support UDP tunnel with outer checksum >>> offload then? >> >> Yes. Calculate or ignore the outer UDP checksum. > > I was just thinking about this. When you say you compute the IPv6 > checksum how is it you are specifying to the hardware that you want to > do that? Is it something you can configure per packet or is it > something that is configured for the VXLAN flow? In the current version of the hardware, it is a global (chip-wide) setting. 1 bit to control outer ipv4 vxlan and 1 bit for outer ipv6 vxlan. > > I just want to make sure you aren't adding checksums to IPv6 tunnels > that specify that they don't want a checksum, or stripping them from > v4 tunnels that do want a checksum. If the global setting has outer UDP checksum enabled, it will be calculated no matter what. If the setting is disabled, it will just ignore it without overwriting it. Thanks. ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-29 21:29 ` Michael Chan 0 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-29 21:29 UTC (permalink / raw) To: intel-wired-lan On Fri, Apr 29, 2016 at 2:17 PM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > On Wed, Apr 27, 2016 at 9:32 PM, Michael Chan <michael.chan@broadcom.com> wrote: >> On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck >> <alexander.duyck@gmail.com> wrote: >>> On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan >>> <michael.chan@broadcom.com> wrote: >>>> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>>>> header fields for length and checksum as well as the length and checksum >>>>> fields for outer UDP and GRE headers. >>>>> >>>>> I have no means of testing this as I do not have any bnx2x hardware but >>>>> thought I would submit it as an RFC to see if anyone out there wants to >>>>> test this and see if this does in fact enable this functionality allowing >>>>> us to to segment tunneled frames that have an outer checksum. >>>>> >>>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>>> >>>> Hi Alex, I just did a very quick test of this patch on our bnxt >>>> hardware and it seemed to work. >>>> >>>> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >>>> getting through. I've verified that our hardware can be programmed to >>>> either ignore outer UDP checksum or to calculate it. Current default >>>> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >>>> Thanks. >>> >>> Are you saying you can natively support UDP tunnel with outer checksum >>> offload then? >> >> Yes. Calculate or ignore the outer UDP checksum. > > I was just thinking about this. When you say you compute the IPv6 > checksum how is it you are specifying to the hardware that you want to > do that? Is it something you can configure per packet or is it > something that is configured for the VXLAN flow? In the current version of the hardware, it is a global (chip-wide) setting. 1 bit to control outer ipv4 vxlan and 1 bit for outer ipv6 vxlan. > > I just want to make sure you aren't adding checksums to IPv6 tunnels > that specify that they don't want a checksum, or stripping them from > v4 tunnels that do want a checksum. If the global setting has outer UDP checksum enabled, it will be calculated no matter what. If the setting is disabled, it will just ignore it without overwriting it. Thanks. ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-29 21:29 ` [Intel-wired-lan] " Michael Chan @ 2016-04-29 21:31 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 21:31 UTC (permalink / raw) To: Michael Chan Cc: Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Fri, Apr 29, 2016 at 2:29 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Fri, Apr 29, 2016 at 2:17 PM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: >> On Wed, Apr 27, 2016 at 9:32 PM, Michael Chan <michael.chan@broadcom.com> wrote: >>> On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck >>> <alexander.duyck@gmail.com> wrote: >>>> On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan >>>> <michael.chan@broadcom.com> wrote: >>>>> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>>>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>>>>> header fields for length and checksum as well as the length and checksum >>>>>> fields for outer UDP and GRE headers. >>>>>> >>>>>> I have no means of testing this as I do not have any bnx2x hardware but >>>>>> thought I would submit it as an RFC to see if anyone out there wants to >>>>>> test this and see if this does in fact enable this functionality allowing >>>>>> us to to segment tunneled frames that have an outer checksum. >>>>>> >>>>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>>>> >>>>> Hi Alex, I just did a very quick test of this patch on our bnxt >>>>> hardware and it seemed to work. >>>>> >>>>> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >>>>> getting through. I've verified that our hardware can be programmed to >>>>> either ignore outer UDP checksum or to calculate it. Current default >>>>> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >>>>> Thanks. >>>> >>>> Are you saying you can natively support UDP tunnel with outer checksum >>>> offload then? >>> >>> Yes. Calculate or ignore the outer UDP checksum. >> >> I was just thinking about this. When you say you compute the IPv6 >> checksum how is it you are specifying to the hardware that you want to >> do that? Is it something you can configure per packet or is it >> something that is configured for the VXLAN flow? > > In the current version of the hardware, it is a global (chip-wide) > setting. 1 bit to control outer ipv4 vxlan and 1 bit for outer ipv6 > vxlan. > >> >> I just want to make sure you aren't adding checksums to IPv6 tunnels >> that specify that they don't want a checksum, or stripping them from >> v4 tunnels that do want a checksum. > > If the global setting has outer UDP checksum enabled, it will be > calculated no matter what. If the setting is disabled, it will just > ignore it without overwriting it. Okay so if that is the case we may want to make it so that we ignore checksum for both IPv4 and IPv6 and then we can just provide it via GSO_PARTIAL in the case we want it. Otherwise you are technically mangling the frames by inserting a checksum on the outer header even though the tunnel was not configured for it. If you can point me toward the point in the code where that is happening I can probably make it a part of this patch. Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-29 21:31 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 21:31 UTC (permalink / raw) To: intel-wired-lan On Fri, Apr 29, 2016 at 2:29 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Fri, Apr 29, 2016 at 2:17 PM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: >> On Wed, Apr 27, 2016 at 9:32 PM, Michael Chan <michael.chan@broadcom.com> wrote: >>> On Wed, Apr 27, 2016 at 8:21 AM, Alexander Duyck >>> <alexander.duyck@gmail.com> wrote: >>>> On Tue, Apr 26, 2016 at 10:55 PM, Michael Chan >>>> <michael.chan@broadcom.com> wrote: >>>>> On Tue, Apr 19, 2016 at 12:06 PM, Alexander Duyck <aduyck@mirantis.com> wrote: >>>>>> This patch assumes that the bnxt hardware will ignore existing IPv4/v6 >>>>>> header fields for length and checksum as well as the length and checksum >>>>>> fields for outer UDP and GRE headers. >>>>>> >>>>>> I have no means of testing this as I do not have any bnx2x hardware but >>>>>> thought I would submit it as an RFC to see if anyone out there wants to >>>>>> test this and see if this does in fact enable this functionality allowing >>>>>> us to to segment tunneled frames that have an outer checksum. >>>>>> >>>>>> Signed-off-by: Alexander Duyck <aduyck@mirantis.com> >>>>> >>>>> Hi Alex, I just did a very quick test of this patch on our bnxt >>>>> hardware and it seemed to work. >>>>> >>>>> I created a vxlan endpoint with udpcsum enabled and I saw TSO packets >>>>> getting through. I've verified that our hardware can be programmed to >>>>> either ignore outer UDP checksum or to calculate it. Current default >>>>> is to ignore ipv4 UDP checksum and calculate ipv6 UDP checksum. >>>>> Thanks. >>>> >>>> Are you saying you can natively support UDP tunnel with outer checksum >>>> offload then? >>> >>> Yes. Calculate or ignore the outer UDP checksum. >> >> I was just thinking about this. When you say you compute the IPv6 >> checksum how is it you are specifying to the hardware that you want to >> do that? Is it something you can configure per packet or is it >> something that is configured for the VXLAN flow? > > In the current version of the hardware, it is a global (chip-wide) > setting. 1 bit to control outer ipv4 vxlan and 1 bit for outer ipv6 > vxlan. > >> >> I just want to make sure you aren't adding checksums to IPv6 tunnels >> that specify that they don't want a checksum, or stripping them from >> v4 tunnels that do want a checksum. > > If the global setting has outer UDP checksum enabled, it will be > calculated no matter what. If the setting is disabled, it will just > ignore it without overwriting it. Okay so if that is the case we may want to make it so that we ignore checksum for both IPv4 and IPv6 and then we can just provide it via GSO_PARTIAL in the case we want it. Otherwise you are technically mangling the frames by inserting a checksum on the outer header even though the tunnel was not configured for it. If you can point me toward the point in the code where that is happening I can probably make it a part of this patch. Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-29 21:31 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-29 23:29 ` Michael Chan -1 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-29 23:29 UTC (permalink / raw) To: Alexander Duyck Cc: Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Fri, Apr 29, 2016 at 2:31 PM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > Okay so if that is the case we may want to make it so that we ignore > checksum for both IPv4 and IPv6 and then we can just provide it via > GSO_PARTIAL in the case we want it. Otherwise you are technically > mangling the frames by inserting a checksum on the outer header even > though the tunnel was not configured for it. If you can point me > toward the point in the code where that is happening I can probably > make it a part of this patch. > All the chip settings are controlled by firmware. I will check with the firmware team to disable them if they are not already disabled. When first developing the driver, before all the recent proposals, the intention was to not advertise NETIF_F_GSO_UDP_TUNNEL_CSUM and not support TSO with outer UDP checksum enabled. Thanks. ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-29 23:29 ` Michael Chan 0 siblings, 0 replies; 50+ messages in thread From: Michael Chan @ 2016-04-29 23:29 UTC (permalink / raw) To: intel-wired-lan On Fri, Apr 29, 2016 at 2:31 PM, Alexander Duyck <alexander.duyck@gmail.com> wrote: > Okay so if that is the case we may want to make it so that we ignore > checksum for both IPv4 and IPv6 and then we can just provide it via > GSO_PARTIAL in the case we want it. Otherwise you are technically > mangling the frames by inserting a checksum on the outer header even > though the tunnel was not configured for it. If you can point me > toward the point in the code where that is happening I can probably > make it a part of this patch. > All the chip settings are controlled by firmware. I will check with the firmware team to disable them if they are not already disabled. When first developing the driver, before all the recent proposals, the intention was to not advertise NETIF_F_GSO_UDP_TUNNEL_CSUM and not support TSO with outer UDP checksum enabled. Thanks. ^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums 2016-04-29 23:29 ` [Intel-wired-lan] " Michael Chan @ 2016-04-29 23:39 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 23:39 UTC (permalink / raw) To: Michael Chan Cc: Alexander Duyck, Eugenia Emantayev, Bruce W Allan, Saeed Mahameed, Netdev, intel-wired-lan, Ariel Elior, Michael Chan On Fri, Apr 29, 2016 at 4:29 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Fri, Apr 29, 2016 at 2:31 PM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: > >> Okay so if that is the case we may want to make it so that we ignore >> checksum for both IPv4 and IPv6 and then we can just provide it via >> GSO_PARTIAL in the case we want it. Otherwise you are technically >> mangling the frames by inserting a checksum on the outer header even >> though the tunnel was not configured for it. If you can point me >> toward the point in the code where that is happening I can probably >> make it a part of this patch. >> > > All the chip settings are controlled by firmware. I will check with > the firmware team to disable them if they are not already disabled. > When first developing the driver, before all the recent proposals, the > intention was to not advertise NETIF_F_GSO_UDP_TUNNEL_CSUM and not > support TSO with outer UDP checksum enabled. Thanks. Right. But adding a checksum where there wasn't one could theoretically be problematic if there was a implementation out there where somebody was mandating that the tunnel checksum must be 0. The i40e driver was enabling the same thing in the driver for an upcoming device that supported an outer checksum offload until I went in and fixed it. Generally if we can match what is expected that is preferred so we don't have any unexpected conflicts in the event that a VTEP expects the packets to come in with checksums or without. I have submitted the patch and when the firmware gets updated the behavior will be cleared up so you can have it either way depending on what the tunnel itself requested. Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 4/5] bnxt: Add support for segmentation of tunnels with outer checksums @ 2016-04-29 23:39 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-29 23:39 UTC (permalink / raw) To: intel-wired-lan On Fri, Apr 29, 2016 at 4:29 PM, Michael Chan <michael.chan@broadcom.com> wrote: > On Fri, Apr 29, 2016 at 2:31 PM, Alexander Duyck > <alexander.duyck@gmail.com> wrote: > >> Okay so if that is the case we may want to make it so that we ignore >> checksum for both IPv4 and IPv6 and then we can just provide it via >> GSO_PARTIAL in the case we want it. Otherwise you are technically >> mangling the frames by inserting a checksum on the outer header even >> though the tunnel was not configured for it. If you can point me >> toward the point in the code where that is happening I can probably >> make it a part of this patch. >> > > All the chip settings are controlled by firmware. I will check with > the firmware team to disable them if they are not already disabled. > When first developing the driver, before all the recent proposals, the > intention was to not advertise NETIF_F_GSO_UDP_TUNNEL_CSUM and not > support TSO with outer UDP checksum enabled. Thanks. Right. But adding a checksum where there wasn't one could theoretically be problematic if there was a implementation out there where somebody was mandating that the tunnel checksum must be 0. The i40e driver was enabling the same thing in the driver for an upcoming device that supported an outer checksum offload until I went in and fixed it. Generally if we can match what is expected that is preferred so we don't have any unexpected conflicts in the event that a VTEP expects the packets to come in with checksums or without. I have submitted the patch and when the firmware gets updated the behavior will be cleared up so you can have it either way depending on what the tunnel itself requested. Thanks. - Alex ^ permalink raw reply [flat|nested] 50+ messages in thread
* [RFC PATCH 5/5] fm10k: Add support for UDP tunnel segmentation with outer checksum offload 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck @ 2016-04-19 19:06 ` Alexander Duyck -1 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: eugenia, bruce.w.allan, saeedm, netdev, intel-wired-lan, ariel.elior, mchan This patch assumes that the fm10k hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP headers. I have no means of testing this as I do not have any mlx5 hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment UDP tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/intel/fm10k/fm10k_netdev.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c index 2a08d3f5b6df..dc4901def72d 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c @@ -1417,15 +1417,21 @@ struct net_device *fm10k_alloc_netdev(const struct fm10k_info *info) /* Only the PF can support VXLAN and NVGRE tunnel offloads */ if (info->mac == fm10k_mac_pf) { - dev->hw_enc_features = NETIF_F_IP_CSUM | + dev->hw_enc_features = NETIF_F_SG | + NETIF_F_IP_CSUM | + NETIF_F_IPV6_CSUM | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_TSO_ECN | NETIF_F_GSO_UDP_TUNNEL | - NETIF_F_IPV6_CSUM | - NETIF_F_SG; + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + + dev->features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; - dev->features |= NETIF_F_GSO_UDP_TUNNEL; + dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; } /* all features defined to this point should be changeable */ ^ permalink raw reply related [flat|nested] 50+ messages in thread
* [Intel-wired-lan] [RFC PATCH 5/5] fm10k: Add support for UDP tunnel segmentation with outer checksum offload @ 2016-04-19 19:06 ` Alexander Duyck 0 siblings, 0 replies; 50+ messages in thread From: Alexander Duyck @ 2016-04-19 19:06 UTC (permalink / raw) To: intel-wired-lan This patch assumes that the fm10k hardware will ignore existing IPv4/v6 header fields for length and checksum as well as the length and checksum fields for outer UDP headers. I have no means of testing this as I do not have any mlx5 hardware but thought I would submit it as an RFC to see if anyone out there wants to test this and see if this does in fact enable this functionality allowing us to to segment UDP tunneled frames that have an outer checksum. Signed-off-by: Alexander Duyck <aduyck@mirantis.com> --- drivers/net/ethernet/intel/fm10k/fm10k_netdev.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c index 2a08d3f5b6df..dc4901def72d 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c @@ -1417,15 +1417,21 @@ struct net_device *fm10k_alloc_netdev(const struct fm10k_info *info) /* Only the PF can support VXLAN and NVGRE tunnel offloads */ if (info->mac == fm10k_mac_pf) { - dev->hw_enc_features = NETIF_F_IP_CSUM | + dev->hw_enc_features = NETIF_F_SG | + NETIF_F_IP_CSUM | + NETIF_F_IPV6_CSUM | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_TSO_ECN | NETIF_F_GSO_UDP_TUNNEL | - NETIF_F_IPV6_CSUM | - NETIF_F_SG; + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; + + dev->features |= NETIF_F_GSO_UDP_TUNNEL | + NETIF_F_GSO_UDP_TUNNEL_CSUM | + NETIF_F_GSO_PARTIAL; - dev->features |= NETIF_F_GSO_UDP_TUNNEL; + dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM; } /* all features defined to this point should be changeable */ ^ permalink raw reply related [flat|nested] 50+ messages in thread
end of thread, other threads:[~2016-08-31 16:15 UTC | newest] Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-04-19 19:05 [RFC PATCH 0/5] Add GSO Partial support for devices with existing tunnel offloads Alexander Duyck 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck 2016-04-19 19:05 ` [RFC PATCH 1/5] mlx4: Add support for UDP tunnel segmentation with outer checksum offload Alexander Duyck 2016-04-19 19:05 ` [Intel-wired-lan] " Alexander Duyck 2016-04-19 19:06 ` [RFC PATCH 2/5] mlx5: " Alexander Duyck 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck 2016-04-20 17:40 ` Saeed Mahameed 2016-04-20 17:40 ` [Intel-wired-lan] " Saeed Mahameed 2016-04-20 18:06 ` Alexander Duyck 2016-04-20 18:06 ` [Intel-wired-lan] " Alexander Duyck 2016-04-28 21:43 ` Matthew Finlay 2016-04-28 21:43 ` [Intel-wired-lan] " Matthew Finlay 2016-04-28 22:04 ` Alexander Duyck 2016-04-28 22:04 ` [Intel-wired-lan] " Alexander Duyck 2016-04-29 1:18 ` Matthew Finlay 2016-04-29 1:18 ` [Intel-wired-lan] " Matthew Finlay 2016-04-29 1:59 ` Alexander Duyck 2016-04-29 1:59 ` [Intel-wired-lan] " Alexander Duyck 2016-04-29 19:27 ` Saeed Mahameed 2016-04-29 19:27 ` [Intel-wired-lan] " Saeed Mahameed 2016-04-29 19:36 ` Alexander Duyck 2016-04-29 19:36 ` [Intel-wired-lan] " Alexander Duyck 2016-04-19 19:06 ` [RFC PATCH 3/5] bnx2x: Add support for segmentation of tunnels with outer checksums Alexander Duyck 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck 2016-08-24 12:33 ` Yuval Mintz 2016-08-24 15:39 ` Alexander Duyck 2016-08-25 5:06 ` Yuval Mintz 2016-08-25 18:26 ` Alexander Duyck 2016-08-31 11:31 ` Yuval Mintz 2016-08-31 16:13 ` Alexander Duyck 2016-04-19 19:06 ` [RFC PATCH 4/5] bnxt: " Alexander Duyck 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck 2016-04-27 5:55 ` Michael Chan 2016-04-27 5:55 ` [Intel-wired-lan] " Michael Chan 2016-04-27 15:21 ` Alexander Duyck 2016-04-27 15:21 ` [Intel-wired-lan] " Alexander Duyck 2016-04-28 4:32 ` Michael Chan 2016-04-28 4:32 ` [Intel-wired-lan] " Michael Chan 2016-04-29 21:17 ` Alexander Duyck 2016-04-29 21:17 ` [Intel-wired-lan] " Alexander Duyck 2016-04-29 21:29 ` Michael Chan 2016-04-29 21:29 ` [Intel-wired-lan] " Michael Chan 2016-04-29 21:31 ` Alexander Duyck 2016-04-29 21:31 ` [Intel-wired-lan] " Alexander Duyck 2016-04-29 23:29 ` Michael Chan 2016-04-29 23:29 ` [Intel-wired-lan] " Michael Chan 2016-04-29 23:39 ` Alexander Duyck 2016-04-29 23:39 ` [Intel-wired-lan] " Alexander Duyck 2016-04-19 19:06 ` [RFC PATCH 5/5] fm10k: Add support for UDP tunnel segmentation with outer checksum offload Alexander Duyck 2016-04-19 19:06 ` [Intel-wired-lan] " Alexander Duyck
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.