Re: [PATCH net-next v2 0/3] VGT+ support

[Jakub Kicinski <jakub.kicinski@netronome.com>]

On Mon, 4 Nov 2019 18:47:32 -0700, David Ahern wrote:
> On 11/4/19 6:38 PM, Saeed Mahameed wrote:
> > On Fri, 2019-11-01 at 17:21 -0700, Jakub Kicinski wrote:  
> >> On Fri, 1 Nov 2019 21:28:22 +0000, Saeed Mahameed wrote:  
> >>> Bottom line, we tried to push this feature a couple of years ago,
> >>> and
> >>> due to some internal issues this submission ignored for a while,
> >>> now as
> >>> the legacy sriov customers are moving towards upstream, which is
> >>> for me
> >>> a great progress I think this feature worth the shot, also as Ariel
> >>> pointed out, VF vlan filter is really a gap that should be closed.
> >>>
> >>> For all other features it is true that the user must consider
> >>> moving to
> >>> witchdev mode or find a another community for support.
> >>>
> >>> Our policy is still strong regarding obsoleting legacy mode and
> >>> pushing
> >>> all new feature to switchdev mode, but looking at the facts here  I
> >>> do
> >>> think there is a point here and ROI to close this gap in legacy
> >>> mode.
> >>>
> >>> I hope this all make sense.   
> >>
> >> I understand and sympathize, you know full well the benefits of
> >> working
> >> upstream-first...
> >>
> >> I won't reiterate the entire response from my previous email, but the
> >> bottom line for me is that we haven't added a single legacy VF NDO
> >> since 2016, I was hoping we never will add more and I was trying to
> >> stop anyone who tried.
> >>  
> > 
> > The NDO is not the problem here, we can perfectly extend the current
> > set_vf_vlan_ndo to achieve the same goal with minimal or even NO kernel
> > changes, but first you have to look at this from my angel, i have been
> > doing lots of research and there are many points for why this should be
> > added to legacy mode:
> > 
> > 1) Switchdev mode can't replace legacy mode with a press of a button,
> > many missing pieces.
> > 
> > 2) Upstream Legacy SRIOV is incomplete since it goes together with
> > flexible vf vlan configuration, most of mlx5 legacy sriov users are
> > using customized kernels and external drivers, since upstream is
> > lacking this one basic vlan filtering feature, and many users decline
> > switching to upstream kernel due to this missing features.
> > 
> > 3) Many other vendors have this feature in customized drivers/kernels,
> > and many vendors/drivers don't even support switchdev mode (mlx4 for
> > example), we can't just tell the users of such device we are not
> > supporting basic sriov legacy mode features in upstream kernel.
> > 
> > 4) the motivation for this is to slowly move sriov users to upstream
> > and eventually to switchdev mode.  
> 
> If the legacy freeze started in 2016 and we are at the end of 2019, what
> is the migration path?

The migration path is to implement basic bridge offload which can take
care of this trivially.

Problem is people equate switchdev with OvS offload today, so bridge
offload is not actually implemented. It's really hard to convince
product marketing that it's work worth taking on.

Adding incremental features to legacy API is always going to be cheaper
than migrating controllers to switchdev.

IDK if you remember the net_failover argument about in-kernel VF to
virtio bonding. The controllers are doing the bare minimum and it's 
hard for HW vendors to justify the expense of moving forward all parts 
of the stack.

Which means SR-IOV is either stuck in pure-L2 middle ages or requires
all the SDN complexity and overhead. Switchdev bridge offload can be
trivially extended to support eVPN and simplify so many deployments,
sigh..

> > Now if the only remaining problem is the uAPI, we can minimize kernel
> > impact or even make no kernel changes at all, only ip route2 and
> > drivers, by reusing the current set_vf_vlan_ndo.  
> 
> And this caught my eye as well -- iproute2 does not need the baggage either.
> 
> Is there any reason this continued support for legacy sriov can not be
> done out of tree?

Exactly. Moving to upstream is only valuable if it doesn't require
brining all the out-of-tree baggage.