From: Jason Gunthorpe <jgg@ziepe.ca>
To: Jakub Kicinski <kuba@kernel.org>
Cc: Jiri Pirko <jiri@resnulli.us>,
netdev@vger.kernel.org, davem@davemloft.net, parav@mellanox.com,
yuvalav@mellanox.com, saeedm@mellanox.com, leon@kernel.org,
andrew.gospodarek@broadcom.com, michael.chan@broadcom.com,
moshe@mellanox.com, ayal@mellanox.com, eranbe@mellanox.com,
vladbu@mellanox.com, kliteyn@mellanox.com, dchickles@marvell.com,
sburla@marvell.com, fmanlunas@marvell.com, tariqt@mellanox.com,
oss-drivers@netronome.com, snelson@pensando.io,
drivers@pensando.io, aelior@marvell.com,
GR-everest-linux-l2@marvell.com, grygorii.strashko@ti.com,
mlxsw@mellanox.com, idosch@mellanox.com, markz@mellanox.com,
jacob.e.keller@intel.com, valex@mellanox.com,
linyunsheng@huawei.com, lihong.yang@intel.com,
vikas.gupta@broadcom.com, magnus.karlsson@intel.com
Subject: Re: [RFC] current devlink extension plan for NICs
Date: Tue, 24 Mar 2020 10:20:44 -0300 [thread overview]
Message-ID: <20200324132044.GI20941@ziepe.ca> (raw)
In-Reply-To: <20200323205619.2f957f1a@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
On Mon, Mar 23, 2020 at 08:56:19PM -0700, Jakub Kicinski wrote:
> On Mon, 23 Mar 2020 19:06:05 -0300 Jason Gunthorpe wrote:
> > On Mon, Mar 23, 2020 at 12:21:23PM -0700, Jakub Kicinski wrote:
> > > > >I see so you want the creation to be controlled by the same entity that
> > > > >controls the eswitch..
> > > > >
> > > > >To me the creation should be on the side that actually needs/will use
> > > > >the new port. And if it's not eswitch manager then eswitch manager
> > > > >needs to ack it.
> > > >
> > > > Hmm. The question is, is it worth to complicate things in this way?
> > > > I don't know. I see a lot of potential misunderstandings :/
> > >
> > > I'd see requesting SFs over devlink/sysfs as simplification, if
> > > anything.
> >
> > We looked at it for a while, working the communication such that the
> > 'untrusted' side could request a port be created with certain
> > parameters and the 'secure eswitch' could know those parameters to
> > authorize and wire it up was super complicated and very hard to do
> > without races.
> >
> > Since it is a security sensitive operation it seems like a much more
> > secure design to have the secure side do all the creation and present
> > the fully operational object to the insecure side.
> >
> > To draw a parallel to qemu & kvm, the untrusted guest VM can't request
> > that qemu create a virtio-net for it. Those are always hot plugged in
> > by the secure side. Same flow here.
>
> Could you tell us a little more about the races? Other than the
> communication channel what changes between issuing from cloud API
> vs devlink?
If I recall the problems came when trying to work with existing cloud
infrastructure that doesn't assume this operating model. You need to
somehow adapt an async APIs of secure/insecure communication with an
async API inside the cloud world. It was a huge mess.
> Side note - there is no communication channel between VM and hypervisor
> right now, which is the cause for weird designs e.g. the failover/auto
> bond mechanism.
Right, and considering the security concerns building one hidden
inside a driver seems like a poor idea..
> > The VF model is poor because the VF is just a dummy stub until the
> > representor/eswitch side is fully configured. There is no way for the
> > Linux driver to know if the VF is operational or not, so we get weird
> > artifacts where we sometimes bind a driver to a VF (and get a
> > non-working ethXX) and sometimes we don't.
>
> Sounds like an implementation issue :S
How so?
> > The only reason it is like this is because of how SRIOV requires
> > everything to be preallocated.
>
> SF also requires pre-allocated resources, so you're not talking about
> PCI mem space etc. here I assume.
It isn't pre-allocated, the usage of the BAR space is dynamic.
> > The SFs can't even exist until they are configured, so there is no
> > state where a driver is connected to an inoperative SF.
>
> You mean it doesn't exist in terms of sysfs device entry?
I mean literally do not exist at the HW level.
Jason
next prev parent reply other threads:[~2020-03-24 13:20 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-19 19:27 [RFC] current devlink extension plan for NICs Jiri Pirko
2020-03-20 3:32 ` Jakub Kicinski
2020-03-20 7:35 ` Jiri Pirko
2020-03-20 21:25 ` Jakub Kicinski
2020-03-21 9:07 ` Parav Pandit
2020-03-23 19:31 ` Jakub Kicinski
2020-03-23 22:50 ` Jason Gunthorpe
2020-03-24 3:41 ` Jakub Kicinski
2020-03-24 13:43 ` Jason Gunthorpe
2020-03-24 5:36 ` Parav Pandit
2020-03-21 9:35 ` Jiri Pirko
2020-03-23 19:21 ` Jakub Kicinski
2020-03-23 22:06 ` Jason Gunthorpe
2020-03-24 3:56 ` Jakub Kicinski
2020-03-24 13:20 ` Jason Gunthorpe [this message]
2020-03-26 14:37 ` Jiri Pirko
2020-03-26 14:43 ` Jiri Pirko
2020-03-26 14:47 ` Jiri Pirko
2020-03-26 14:51 ` Jiri Pirko
2020-03-26 20:30 ` Jakub Kicinski
2020-03-27 7:47 ` Jiri Pirko
2020-03-27 16:38 ` Jakub Kicinski
2020-03-27 18:49 ` Samudrala, Sridhar
2020-03-27 19:10 ` Jakub Kicinski
2020-03-27 19:45 ` Saeed Mahameed
2020-03-27 20:42 ` Jakub Kicinski
2020-03-30 9:07 ` Parav Pandit
2020-04-08 6:10 ` Parav Pandit
2020-03-27 20:47 ` Samudrala, Sridhar
2020-03-27 20:59 ` Jakub Kicinski
2020-03-30 7:09 ` Parav Pandit
2020-03-30 7:48 ` Parav Pandit
2020-03-30 19:36 ` Jakub Kicinski
2020-03-31 7:45 ` Parav Pandit
2020-03-31 17:32 ` Jakub Kicinski
2020-04-01 7:32 ` Parav Pandit
2020-04-01 20:12 ` Jakub Kicinski
2020-04-02 6:16 ` Jiri Pirko
2020-04-08 5:10 ` Parav Pandit
2020-04-08 5:07 ` Parav Pandit
2020-04-08 16:59 ` Jakub Kicinski
2020-04-08 18:13 ` Parav Pandit
2020-04-09 2:07 ` Jakub Kicinski
2020-04-09 6:43 ` Parav Pandit
2020-03-30 5:30 ` Parav Pandit
2020-03-26 14:59 ` Jiri Pirko
2020-03-23 23:32 ` Andy Gospodarek
2020-03-24 0:11 ` Jason Gunthorpe
2020-03-24 5:53 ` Parav Pandit
2020-03-23 21:32 ` Andy Gospodarek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200324132044.GI20941@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=GR-everest-linux-l2@marvell.com \
--cc=aelior@marvell.com \
--cc=andrew.gospodarek@broadcom.com \
--cc=ayal@mellanox.com \
--cc=davem@davemloft.net \
--cc=dchickles@marvell.com \
--cc=drivers@pensando.io \
--cc=eranbe@mellanox.com \
--cc=fmanlunas@marvell.com \
--cc=grygorii.strashko@ti.com \
--cc=idosch@mellanox.com \
--cc=jacob.e.keller@intel.com \
--cc=jiri@resnulli.us \
--cc=kliteyn@mellanox.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=lihong.yang@intel.com \
--cc=linyunsheng@huawei.com \
--cc=magnus.karlsson@intel.com \
--cc=markz@mellanox.com \
--cc=michael.chan@broadcom.com \
--cc=mlxsw@mellanox.com \
--cc=moshe@mellanox.com \
--cc=netdev@vger.kernel.org \
--cc=oss-drivers@netronome.com \
--cc=parav@mellanox.com \
--cc=saeedm@mellanox.com \
--cc=sburla@marvell.com \
--cc=snelson@pensando.io \
--cc=tariqt@mellanox.com \
--cc=valex@mellanox.com \
--cc=vikas.gupta@broadcom.com \
--cc=vladbu@mellanox.com \
--cc=yuvalav@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).