Re: [RFC] current devlink extension plan for NICs

[Jakub Kicinski <kuba@kernel.org>]

On Mon, 23 Mar 2020 19:06:05 -0300 Jason Gunthorpe wrote:
> On Mon, Mar 23, 2020 at 12:21:23PM -0700, Jakub Kicinski wrote:
> > > >I see so you want the creation to be controlled by the same entity that
> > > >controls the eswitch..
> > > >
> > > >To me the creation should be on the side that actually needs/will use
> > > >the new port. And if it's not eswitch manager then eswitch manager
> > > >needs to ack it.    
> > > 
> > > Hmm. The question is, is it worth to complicate things in this way?
> > > I don't know. I see a lot of potential misunderstandings :/  
> > 
> > I'd see requesting SFs over devlink/sysfs as simplification, if
> > anything.  
> 
> We looked at it for a while, working the communication such that the
> 'untrusted' side could request a port be created with certain
> parameters and the 'secure eswitch' could know those parameters to
> authorize and wire it up was super complicated and very hard to do
> without races.
> 
> Since it is a security sensitive operation it seems like a much more
> secure design to have the secure side do all the creation and present
> the fully operational object to the insecure side.
> 
> To draw a parallel to qemu & kvm, the untrusted guest VM can't request
> that qemu create a virtio-net for it. Those are always hot plugged in
> by the secure side. Same flow here.

Could you tell us a little more about the races? Other than the
communication channel what changes between issuing from cloud API
vs devlink?

Side note - there is no communication channel between VM and hypervisor
right now, which is the cause for weird designs e.g. the failover/auto
bond mechanism.

> > > >The precedence is probably not a strong argument, but that'd be the
> > > >same way VFs work.. I don't think you can change how VFs work, right?    
> > > 
> > > You can't, but since the VF model is not optimal as it turned out, do we
> > > want to stick with it for the SFs too?  
> > 
> > The VF model is not optimal in what sense? I thought the main reason
> > for SFs is that they are significantly more light weight on Si side.  
> 
> Not entirely really, the biggest reason for SF is because VF have to
> be pr-eallocated and have a number limited by PCI.
> 
> The VF model is poor because the VF is just a dummy stub until the
> representor/eswitch side is fully configured. There is no way for the
> Linux driver to know if the VF is operational or not, so we get weird
> artifacts where we sometimes bind a driver to a VF (and get a
> non-working ethXX) and sometimes we don't. 

Sounds like an implementation issue :S

> The only reason it is like this is because of how SRIOV requires
> everything to be preallocated.

SF also requires pre-allocated resources, so you're not talking about
PCI mem space etc. here I assume.

> The SFs can't even exist until they are configured, so there is no
> state where a driver is connected to an inoperative SF.

You mean it doesn't exist in terms of sysfs device entry?

Okay, if we limit ourselves to sysfs as device interface then sure.
We have far richer interfaces in networking world, so it's a little 
hard to sympathize.

> So it would be nice if VF and SF had the same flow, the SF flow is
> better, lets fix the VF flow to match it.