Re: [PATCH net 1/6] ice: avoid bonding causing auxiliary plug/unplug under RTNL lock

From: Tony Nguyen <anthony.l.nguyen@intel.com>
To: Leon Romanovsky <leonro@nvidia.com>
Cc: "davem@davemloft.net" <davem@davemloft.net>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"pabeni@redhat.com" <pabeni@redhat.com>,
	"edumazet@google.com" <edumazet@google.com>,
	"Ertman, David M" <david.m.ertman@intel.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	poros <poros@redhat.com>, ivecera <ivecera@redhat.com>,
	"Saleem, Shiraz" <shiraz.saleem@intel.com>,
	"Ismail, Mustafa" <mustafa.ismail@intel.com>,
	"jgg@nvidia.com" <jgg@nvidia.com>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>,
	Michal Swiatkowski <michal.swiatkowski@linux.intel.com>,
	"G, GurucharanX" <GurucharanX.G@intel.com>
Subject: Re: [PATCH net 1/6] ice: avoid bonding causing auxiliary plug/unplug under RTNL lock
Date: Mon, 6 Feb 2023 15:12:29 -0800	[thread overview]
Message-ID: <cb31a911-ba80-e2dc-231f-851757cfd0b8@intel.com> (raw)
In-Reply-To: <Y9o1wbLykLodmbDd@unreal>

On 2/1/2023 1:49 AM, Leon Romanovsky wrote:
> On Tue, Jan 31, 2023 at 01:36:58PM -0800, Tony Nguyen wrote:
>> From: Dave Ertman <david.m.ertman@intel.com>
>>
>> RDMA is not supported in ice on a PF that has been added to a bonded
>> interface. To enforce this, when an interface enters a bond, we unplug
>> the auxiliary device that supports RDMA functionality.  This unplug
>> currently happens in the context of handling the netdev bonding event.
>> This event is sent to the ice driver under RTNL context.  This is causing
>> a deadlock where the RDMA driver is waiting for the RTNL lock to complete
>> the removal.
>>
>> Defer the unplugging/re-plugging of the auxiliary device to the service
>> task so that it is not performed under the RTNL lock context.
>>
>> Reported-by: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
>> Link: https://lore.kernel.org/linux-rdma/68b14b11-d0c7-65c9-4eeb-0487c95e395d@leemhuis.info/
>> Fixes: 5cb1ebdbc434 ("ice: Fix race condition during interface enslave")
>> Fixes: 4eace75e0853 ("RDMA/irdma: Report the correct link speed")
>> Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
>> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
>> Tested-by: Gurucharan G <gurucharanx.g@intel.com> (A Contingent worker at Intel)
>> Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
>> ---
>>   drivers/net/ethernet/intel/ice/ice.h      | 14 +++++---------
>>   drivers/net/ethernet/intel/ice/ice_main.c | 17 +++++++----------
>>   2 files changed, 12 insertions(+), 19 deletions(-)
> 
> <...>
> 
>> index 5f86e4111fa9..055494dbcce0 100644
>> --- a/drivers/net/ethernet/intel/ice/ice_main.c
>> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
>> @@ -2290,18 +2290,15 @@ static void ice_service_task(struct work_struct *work)
>>   		}
>>   	}
>>   
>> -	if (test_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags)) {
>> -		/* Plug aux device per request */
>> +	/* Plug aux device per request */
>> +	if (test_and_clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags))
> 
> Very interesting pattern. You are not holding any locks while running
> ice_service_task() and clear bits before you actually performed requested
> operation.
> 
> How do you protect from races while testing bits in other places of ice
> driver?

I haven't heard from Dave so I'm going to drop this from the series so 
that the other patches can move on.

Thanks,
Tony