From: Leon Romanovsky <leonro@nvidia.com>
To: "Ertman, David M" <david.m.ertman@intel.com>
Cc: "Nguyen, Anthony L" <anthony.l.nguyen@intel.com>,
"davem@davemloft.net" <davem@davemloft.net>,
"kuba@kernel.org" <kuba@kernel.org>,
"pabeni@redhat.com" <pabeni@redhat.com>,
"edumazet@google.com" <edumazet@google.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
poros <poros@redhat.com>, ivecera <ivecera@redhat.com>,
"Saleem, Shiraz" <shiraz.saleem@intel.com>,
"Ismail, Mustafa" <mustafa.ismail@intel.com>,
"jgg@nvidia.com" <jgg@nvidia.com>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>,
Michal Swiatkowski <michal.swiatkowski@linux.intel.com>,
"G, GurucharanX" <gurucharanx.g@intel.com>
Subject: Re: [PATCH net 1/6] ice: avoid bonding causing auxiliary plug/unplug under RTNL lock
Date: Wed, 15 Feb 2023 14:13:33 +0200 [thread overview]
Message-ID: <Y+zMbR7aBs8tQrrN@unreal> (raw)
In-Reply-To: <MW5PR11MB58119C1D62DAC020FB32ED00DDA29@MW5PR11MB5811.namprd11.prod.outlook.com>
On Tue, Feb 14, 2023 at 10:24:04PM +0000, Ertman, David M wrote:
> > -----Original Message-----
> > From: Leon Romanovsky <leonro@nvidia.com>
> > Sent: Wednesday, February 1, 2023 1:50 AM
> > Subject: Re: [PATCH net 1/6] ice: avoid bonding causing auxiliary plug/unplug
> > under RTNL lock
> >
> > On Tue, Jan 31, 2023 at 01:36:58PM -0800, Tony Nguyen wrote:
> > > From: Dave Ertman <david.m.ertman@intel.com>
> > >
> > > RDMA is not supported in ice on a PF that has been added to a bonded
> > > interface. To enforce this, when an interface enters a bond, we unplug
> > > the auxiliary device that supports RDMA functionality. This unplug
> > > currently happens in the context of handling the netdev bonding event.
> > > This event is sent to the ice driver under RTNL context. This is causing
> > > a deadlock where the RDMA driver is waiting for the RTNL lock to complete
> > > the removal.
> > >
> > > Defer the unplugging/re-plugging of the auxiliary device to the service
> > > task so that it is not performed under the RTNL lock context.
> > >
> > > Reported-by: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
> > > Link: https://lore.kernel.org/linux-rdma/68b14b11-d0c7-65c9-4eeb-
> > 0487c95e395d@leemhuis.info/
> > > Fixes: 5cb1ebdbc434 ("ice: Fix race condition during interface enslave")
> > > Fixes: 4eace75e0853 ("RDMA/irdma: Report the correct link speed")
> > > Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
> > > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > > Tested-by: Gurucharan G <gurucharanx.g@intel.com> (A Contingent
> > worker at Intel)
> > > Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
> > > ---
> > > drivers/net/ethernet/intel/ice/ice.h | 14 +++++---------
> > > drivers/net/ethernet/intel/ice/ice_main.c | 17 +++++++----------
> > > 2 files changed, 12 insertions(+), 19 deletions(-)
> >
> > <...>
> >
> > > index 5f86e4111fa9..055494dbcce0 100644
> > > --- a/drivers/net/ethernet/intel/ice/ice_main.c
> > > +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> > > @@ -2290,18 +2290,15 @@ static void ice_service_task(struct work_struct
> > *work)
> > > }
> > > }
> > >
> > > - if (test_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags)) {
> > > - /* Plug aux device per request */
> > > + /* Plug aux device per request */
> > > + if (test_and_clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags))
> >
> > Very interesting pattern. You are not holding any locks while running
> > ice_service_task() and clear bits before you actually performed requested
> > operation.
> >
> > How do you protect from races while testing bits in other places of ice
> > driver?
>
> Leon,
>
> Thanks for the review and sorry for the late reply, got sidetracked into another project.
>
> Your review caused us to re-evaluate the plug/unplug flow, and since these bits are only set/cleared in
> the bonding event flow, and the UNPLUG bit set clears the PLUG bit, we attain the desired outcome
> in all cases if we swap the order that we evaluate the bits in the service task.
I afraid that it won't make ice state machine more understandable. :)
Thanks
>
> Any multi-event situation that happens between or during service task will be handled in the expected way.
>
> DaveE
>
> >
> > Thanks
> >
> > > ice_plug_aux_dev(pf);
> > >
> > > - /* Mark plugging as done but check whether unplug was
> > > - * requested during ice_plug_aux_dev() call
> > > - * (e.g. from ice_clear_rdma_cap()) and if so then
> > > - * plug aux device.
> > > - */
> > > - if (!test_and_clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf-
> > >flags))
> > > - ice_unplug_aux_dev(pf);
> > > - }
> > > + /* unplug aux dev per request, if an unplug request came in
> > > + * while processing a plug request, this will handle it
> > > + */
> > > + if (test_and_clear_bit(ICE_FLAG_UNPLUG_AUX_DEV, pf->flags))
> > > + ice_unplug_aux_dev(pf);
> > >
> > > if (test_and_clear_bit(ICE_FLAG_MTU_CHANGED, pf->flags)) {
> > > struct iidc_event *event;
> > > --
> > > 2.38.1
> > >
next prev parent reply other threads:[~2023-02-15 12:13 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-31 21:36 [PATCH net 0/6][pull request] Intel Wired LAN Driver Updates 2023-01-31 (ice) Tony Nguyen
2023-01-31 21:36 ` [PATCH net 1/6] ice: avoid bonding causing auxiliary plug/unplug under RTNL lock Tony Nguyen
2023-02-01 9:49 ` Leon Romanovsky
2023-02-06 23:12 ` Tony Nguyen
2023-02-14 22:24 ` Ertman, David M
2023-02-15 12:13 ` Leon Romanovsky [this message]
2023-01-31 21:36 ` [PATCH net 2/6] ice: Do not use WQ_MEM_RECLAIM flag for workqueue Tony Nguyen
2023-02-01 9:51 ` Leon Romanovsky
2023-01-31 21:37 ` [PATCH net 3/6] ice: fix out-of-bounds KASAN warning in virtchnl Tony Nguyen
2023-02-01 9:52 ` Leon Romanovsky
2023-01-31 21:37 ` [PATCH net 4/6] ice: Fix disabling Rx VLAN filtering with port VLAN enabled Tony Nguyen
2023-01-31 21:37 ` [PATCH net 5/6] ice: Fix off by one in ice_tc_forward_to_queue() Tony Nguyen
2023-02-01 9:52 ` Leon Romanovsky
2023-01-31 21:37 ` [PATCH net 6/6] ice: switch: fix potential memleak in ice_add_adv_recipe() Tony Nguyen
2023-02-01 9:55 ` Leon Romanovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y+zMbR7aBs8tQrrN@unreal \
--to=leonro@nvidia.com \
--cc=anthony.l.nguyen@intel.com \
--cc=davem@davemloft.net \
--cc=david.m.ertman@intel.com \
--cc=edumazet@google.com \
--cc=gurucharanx.g@intel.com \
--cc=ivecera@redhat.com \
--cc=jaroslav.pulchart@gooddata.com \
--cc=jgg@nvidia.com \
--cc=kuba@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=michal.swiatkowski@linux.intel.com \
--cc=mustafa.ismail@intel.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=poros@redhat.com \
--cc=shiraz.saleem@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).