Re: [RFC] current devlink extension plan for NICs

From: Jakub Kicinski <kuba@kernel.org>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Parav Pandit <parav@mellanox.com>, Jiri Pirko <jiri@resnulli.us>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	Yuval Avnery <yuvalav@mellanox.com>,
	Saeed Mahameed <saeedm@mellanox.com>,
	"leon@kernel.org" <leon@kernel.org>,
	"andrew.gospodarek@broadcom.com" <andrew.gospodarek@broadcom.com>,
	"michael.chan@broadcom.com" <michael.chan@broadcom.com>,
	Moshe Shemesh <moshe@mellanox.com>, Aya Levin <ayal@mellanox.com>,
	Eran Ben Elisha <eranbe@mellanox.com>,
	Vlad Buslov <vladbu@mellanox.com>,
	Yevgeny Kliteynik <kliteyn@mellanox.com>,
	"dchickles@marvell.com" <dchickles@marvell.com>,
	"sburla@marvell.com" <sburla@marvell.com>,
	"fmanlunas@marvell.com" <fmanlunas@marvell.com>,
	Tariq Toukan <tariqt@mellanox.com>,
	"oss-drivers@netronome.com" <oss-drivers@netronome.com>,
	"snelson@pensando.io" <snelson@pensando.io>,
	"drivers@pensando.io" <drivers@pensando.io>,
	"aelior@marvell.com" <aelior@marvell.com>,
	"GR-everest-linux-l2@marvell.com"
	<GR-everest-linux-l2@marvell.com>,
	"grygorii.strashko@ti.com" <grygorii.strashko@ti.com>,
	mlxsw <mlxsw@mellanox.com>, Ido Schimmel <idosch@mellanox.com>,
	Mark Zhang <markz@mellanox.com>,
	"jacob.e.keller@intel.com" <jacob.e.keller@intel.com>,
	Alex Vesker <valex@mellanox.com>,
	"linyunsheng@huawei.com" <linyunsheng@huawei.com>,
	"lihong.yang@intel.com" <lihong.yang@intel.com>,
	"vikas.gupta@broadcom.com" <vikas.gupta@broadcom.com>,
	"magnus.karlsson@intel.com" <magnus.karlsson@intel.com>
Subject: Re: [RFC] current devlink extension plan for NICs
Date: Mon, 23 Mar 2020 20:41:16 -0700	[thread overview]
Message-ID: <20200323204116.7c2f6e46@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <20200323225009.GA1839@ziepe.ca>

On Mon, 23 Mar 2020 19:50:09 -0300 Jason Gunthorpe wrote:
> On Mon, Mar 23, 2020 at 12:31:16PM -0700, Jakub Kicinski wrote:
> 
> > Right, that is the point. It's the host admin that wants the new
> > entity, so if possible it'd be better if they could just ask for it 
> > via devlink rather than some cloud API. Not that I'm completely opposed
> > to a cloud API - just seems unnecessary here.  
> 
> The cloud API provides all the permissions checks and security
> elements. It cannot be avoided.

Ack, the question is just who consults the cloud API, the Host or the
SmartNIC (latter would abstract differences between cloud APIs).

> If you try to do it as you say then it is weird. You have to use the
> cloud API to authorize the VM to touch a certain network, then the VM
> has to somehow take that network ID and use devlink to get a netdev
> for it. And the cloud side has to protect against a hostile VM sending
> garbage along this communication channel.

I don't understand how the VM needs to know the network ID, quite the
opposite, the Network ID should be gettable/settable by the hypervisor/
/PF. 

If VF starts requesting nested network IDs those should be in a
separate namespace from the the "outer" ones, no?

> vs simply host plugging in the correct network fully operational when
> the cloud API connects the VM to the network.

That means the user has to pre-allocate the device ID, or query the
cloud API after the device is created about its attributes (in the case
of two interfaces being requested simultaneously).

I don't feel very strongly about this, but given how many Linux
instances run in the cloud it'd seem nice if we had some APIs to meet
their basic needs.