RE: [net-next v5 05/15] devlink: Support get and set state of port function

From: Parav Pandit <parav@nvidia.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: Saeed Mahameed <saeed@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Jason Gunthorpe <jgg@nvidia.com>,
	Leon Romanovsky <leonro@nvidia.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	David Ahern <dsahern@kernel.org>,
	Jacob Keller <jacob.e.keller@intel.com>,
	"Sridhar Samudrala" <sridhar.samudrala@intel.com>,
	"david.m.ertman@intel.com" <david.m.ertman@intel.com>,
	"dan.j.williams@intel.com" <dan.j.williams@intel.com>,
	"kiran.patil@intel.com" <kiran.patil@intel.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	Jiri Pirko <jiri@nvidia.com>, Vu Pham <vuhuong@nvidia.com>,
	Saeed Mahameed <saeedm@nvidia.com>
Subject: RE: [net-next v5 05/15] devlink: Support get and set state of port function
Date: Sat, 19 Dec 2020 05:06:08 +0000	[thread overview]
Message-ID: <BY5PR12MB43229C7664D0DE29FCB68CE6DCC20@BY5PR12MB4322.namprd12.prod.outlook.com> (raw)
In-Reply-To: <20201218115110.33701ded@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>

> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Saturday, December 19, 2020 1:21 AM
> 
> On Thu, 17 Dec 2020 05:46:45 +0000 Parav Pandit wrote:
> > > From: Jakub Kicinski <kuba@kernel.org>
> > > Sent: Thursday, December 17, 2020 5:39 AM
> > >
> > > On Wed, 16 Dec 2020 05:15:04 +0000 Parav Pandit wrote:
> > > > > From: Jakub Kicinski <kuba@kernel.org>
> > > > > Sent: Wednesday, December 16, 2020 6:08 AM
> > > > >
> > > > > On Tue, 15 Dec 2020 01:03:48 -0800 Saeed Mahameed wrote:
> > > > > > From: Parav Pandit <parav@nvidia.com>
> > > > > >
> > > > > > devlink port function can be in active or inactive state.
> > > > > > Allow users to get and set port function's state.
> > > > > >
> > > > > > When the port function it activated, its operational state may
> > > > > > change after a while when the device is created and driver binds to
> it.
> > > > > > Similarly on deactivation flow.
> > > > >
> > > > > So what's the flow device should implement?
> > > > >
> > > > > User requests deactivated, the device sends a notification to
> > > > > the driver bound to the device. What if the driver ignores it?
> > > > >
> > > > If driver ignores it, those devices are marked unusable for new
> allocation.
> > > > Device becomes usable only after it has act on the event.
> > >
> > > But the device remains fully operational?
> > >
> > > So if I'm an admin who wants to unplug a misbehaving "entity"[1] the
> > > deactivate is not gonna help me, it's just a graceful hint?
> > Right.
> > > Is there no need for a forceful shutdown?
> > In this patchset, no. I didn't add the knob for it. It is already at 15 patches.
> > But yes, forceful shutdown extension can be done by the admin in
> > future patchset as,
> >
> > $ devlink port del pci/0000:06:00.0/<port_index> force true
> >
> > ^^^^^^^^ Above will be the extension in control of the admin.
> 
> Can we come up with operational states that would encompass that?
> 
Operational state is read only. Adding more states will likely make user job harder, unless its absolute necessary.
Currently state and operational state definitions cover all the scenario needed.
Only exception is user doesn't have the ability of force delete.
$ devlink port shutdown pci/0000:03:00.0/port_index
Above command will attempt graceful port deletion.

$ devlink port del pci/0000:03:00.0/port_index
Above command will do force deletion.

> The "force true" does not look too clean.
>
I think notion of force to user is more intuitive than above two commands, as its exist for other parts of the system for example 'reboot'.
So no need for flag as true/false. Just force if user wish to do force removal.

> And let's document meaning of the states. We don't want the next vendor to
> just "assume" the states match their own interpretation.
Oh yes, I did document it in the UAPI header file which will be the first place for vendors to look on how to implement get/set.
But I will add this to patch_14 in the devlink 'subfunctions' section documentation.