Re: [PATCH net-next] devlink: Require devlink lock during device reload

From: Jiri Pirko <jiri@resnulli.us>
To: Jakub Kicinski <kuba@kernel.org>
Cc: Jason Gunthorpe <jgg@nvidia.com>,
	Leon Romanovsky <leon@kernel.org>, Jiri Pirko <jiri@nvidia.com>,
	Ido Schimmel <idosch@idosch.org>,
	"David S . Miller" <davem@davemloft.net>,
	linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
	edwin.peer@broadcom.com
Subject: Re: [PATCH net-next] devlink: Require devlink lock during device reload
Date: Tue, 16 Nov 2021 08:00:10 +0100	[thread overview]
Message-ID: <YZNW+mtcXe2kjJlY@nanopsycho> (raw)
In-Reply-To: <20211115072206.72435d60@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>

Mon, Nov 15, 2021 at 04:22:06PM CET, kuba@kernel.org wrote:
>On Mon, 15 Nov 2021 11:09:31 -0400 Jason Gunthorpe wrote:
>> On Mon, Nov 15, 2021 at 03:42:58PM +0100, Jiri Pirko wrote:
>> > >Sorry, I don't agree that registering a net notifier in an aux device
>> > >probe function is non-standard or wrong.  
>> > 
>> > Listening to events which happen in different namespaces and react to
>> > them is the non-standard behaviour which I refered to. If you would not
>> > need to do it, you could just use netns notofier which would solve your
>> > issue. You know it.  
>> 
>> Huh?
>> 
>> It calls the bog standard
>> 
>>  register_netdevice_notifier() 
>> 
>> Like hundreds of other drivers do from their probe functions
>> 
>> Which does:
>> 
>> int register_netdevice_notifier(struct notifier_block *nb)
>> {
>> 	struct net *net;
>> 	int err;
>> 
>> 	/* Close race with setup_net() and cleanup_net() */
>> 	down_write(&pernet_ops_rwsem);
>> 
>> And deadlocks because devlink hols the pernet_ops_rwsem when it
>> triggers reload in some paths.
>> 
>> There is nothing wrong with a driver doing this standard pattern.
>> 
>> There is only one place in the entire kernel calling the per-ns
>> register_netdevice_notifier_dev_net() and it is burred inside another
>> part of mlx5 for some reason..
>> 
>> I believe Parav already looked at using that in rdma and it didn't
>> work for some reason I've forgotten. 
>> 
>> It is not that we care about events in different namespaces, it is
>> that rdma, like everything else, doesn't care about namespaces and
>> wants events from the netdev no matter where it is located.
>
>devlink now allows drivers to be net ns-aware, and they should 
>obey if they declare support.  Can we add a flag / capability 
>to devlink and make it an explicit opt-in for drivers who care?

Looks kind of odd to me. It is also not possible for netdevice to say if
it should be net namespace aware or not. The driver should not be the
one to decide this.