Re: [PATCH net-next] gso: enable udp gso for virtual devices

[Jason Baron <jbaron@akamai.com>]

On 6/13/19 1:15 PM, Alexander Duyck wrote:
> On Wed, Jun 12, 2019 at 4:14 PM Jason Baron <jbaron@akamai.com> wrote:
>>
>> Now that the stack supports UDP GRO, we can enable udp gso for virtual
>> devices. If packets are looped back locally, and UDP GRO is not enabled
>> then they will be segmented to gso_size via udp_rcv_segment(). This
>> essentiallly just reverts: 8eea1ca gso: limit udp gso to egress-only
>> virtual devices.
>>
>> Tested by connecting two namespaces via macvlan and then ran
>> udpgso_bench_tx:
>>
>> before:
>> udp tx:   2068 MB/s    35085 calls/s  35085 msg/s
>>
>> after (no UDP_GRO):
>> udp tx:   3438 MB/s    58319 calls/s  58319 msg/s
>>
>> after (UDP_GRO):
>> udp tx:   8037 MB/s   136314 calls/s 136314 msg/s
>>
>> Signed-off-by: Jason Baron <jbaron@akamai.com>
>> Co-developed-by: Joshua Hunt <johunt@akamai.com>
>> Signed-off-by: Joshua Hunt <johunt@akamai.com>
>> Cc: Alexander Duyck <alexander.duyck@gmail.com>
>> Cc: Willem de Bruijn <willemb@google.com>
>> Cc: Paolo Abeni <pabeni@redhat.com>
>> ---
>>  drivers/net/bonding/bond_main.c | 5 ++---
>>  drivers/net/team/team.c         | 5 ++---
>>  include/linux/netdev_features.h | 1 +
>>  3 files changed, 5 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
>> index 4f5b3ba..c4260be 100644
>> --- a/drivers/net/bonding/bond_main.c
>> +++ b/drivers/net/bonding/bond_main.c
>> @@ -1120,8 +1120,7 @@ static void bond_compute_features(struct bonding *bond)
>>
>>  done:
>>         bond_dev->vlan_features = vlan_features;
>> -       bond_dev->hw_enc_features = enc_features | NETIF_F_GSO_ENCAP_ALL |
>> -                                   NETIF_F_GSO_UDP_L4;
>> +       bond_dev->hw_enc_features = enc_features | NETIF_F_GSO_ENCAP_ALL;
>>         bond_dev->mpls_features = mpls_features;
>>         bond_dev->gso_max_segs = gso_max_segs;
>>         netif_set_gso_max_size(bond_dev, gso_max_size);
>> @@ -4308,7 +4307,7 @@ void bond_setup(struct net_device *bond_dev)
>>                                 NETIF_F_HW_VLAN_CTAG_RX |
>>                                 NETIF_F_HW_VLAN_CTAG_FILTER;
>>
>> -       bond_dev->hw_features |= NETIF_F_GSO_ENCAP_ALL | NETIF_F_GSO_UDP_L4;
>> +       bond_dev->hw_features |= NETIF_F_GSO_ENCAP_ALL;
>>         bond_dev->features |= bond_dev->hw_features;
>>  }
>>
>> diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c
>> index b48006e..30299e3 100644
>> --- a/drivers/net/team/team.c
>> +++ b/drivers/net/team/team.c
>> @@ -1003,8 +1003,7 @@ static void __team_compute_features(struct team *team)
>>         }
>>
>>         team->dev->vlan_features = vlan_features;
>> -       team->dev->hw_enc_features = enc_features | NETIF_F_GSO_ENCAP_ALL |
>> -                                    NETIF_F_GSO_UDP_L4;
>> +       team->dev->hw_enc_features = enc_features | NETIF_F_GSO_ENCAP_ALL;
>>         team->dev->hard_header_len = max_hard_header_len;
>>
>>         team->dev->priv_flags &= ~IFF_XMIT_DST_RELEASE;
>> @@ -2132,7 +2131,7 @@ static void team_setup(struct net_device *dev)
>>                            NETIF_F_HW_VLAN_CTAG_RX |
>>                            NETIF_F_HW_VLAN_CTAG_FILTER;
>>
>> -       dev->hw_features |= NETIF_F_GSO_ENCAP_ALL | NETIF_F_GSO_UDP_L4;
>> +       dev->hw_features |= NETIF_F_GSO_ENCAP_ALL;
>>         dev->features |= dev->hw_features;
>>  }
>>
>> diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
>> index 4b19c54..188127c 100644
>> --- a/include/linux/netdev_features.h
>> +++ b/include/linux/netdev_features.h
>> @@ -237,6 +237,7 @@ static inline int find_next_netdev_feature(u64 feature, unsigned long start)
>>                                  NETIF_F_GSO_GRE_CSUM |                 \
>>                                  NETIF_F_GSO_IPXIP4 |                   \
>>                                  NETIF_F_GSO_IPXIP6 |                   \
>> +                                NETIF_F_GSO_UDP_L4 |                   \
>>                                  NETIF_F_GSO_UDP_TUNNEL |               \
>>                                  NETIF_F_GSO_UDP_TUNNEL_CSUM)
> 
> Are you adding this to NETIF_F_GSO_ENCAP_ALL? Wouldn't it make more
> sense to add it to NETIF_F_GSO_SOFTWARE?
> 

Yes, I'm adding to NETIF_F_GSO_ENCAP_ALL (not very clear from the
context). I will fix the commit log.

In: 83aa025 udp: add gso support to virtual devices, the support was
also added to NETIF_F_GSO_ENCAP_ALL (although subsequently reverted due
to UDP GRO not being in place), so I wonder what the reason was for that?

I agree that NETIF_F_GSO_SOFTWARE seems conceptually more logical and
further I think it adds support for more 'virtual' devices. For example,
I tested loopback with NETIF_F_GSO_UDP_L4 being added to
NETIF_F_GSO_SOFTWARE and it shows a nice performance gain, whereas
NETIF_F_GSO_ENCAP_ALL isn't included for loopback.

Thanks,

-Jason