Re: [Intel-wired-lan] [PATCH net v2] ice: avoid bonding causing auxiliary plug/unplug under RTNL lock

[Petr Oros <poros@redhat.com>]

Hi Dave,

The first version was sent and commented on upstream ml.
Why do you omit cc on upstream (netdev@vger.kernel.org
<netdev@vger.kernel.org>)?
Can you resend this to upstream ml to take valuable feedback for this
critical fix?

Many Thanks,
Petr


Dave Ertman píše v St 15. 02. 2023 v 11:17 -0800:
> RDMA is not supported in ice on a PF that has been added to a bonded
> interface. To enforce this, when an interface enters a bond, we
> unplug
> the auxiliary device that supports RDMA functionality.  This unplug
> currently happens in the context of handling the netdev bonding
> event.
> This event is sent to the ice driver under RTNL context.  This is
> causing
> a deadlock where the RDMA driver is waiting for the RTNL lock to
> complete
> the removal.
> 
> Defer the unplugging/re-plugging of the auxiliary device to the
> service
> task so that it is not performed under the RTNL lock context.
> 
> Reported-by: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
> Link:
> https://lore.kernel.org/linux-rdma/68b14b11-d0c7-65c9-4eeb-0487c95e395d@leemhuis.info/
> Fixes: 5cb1ebdbc434 ("ice: Fix race condition during interface
> enslave")
> Fixes: 425c9bd06b7a ("RDMA/irdma: Report the correct link speed")
> Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
> ---
> Changes since v1:
> Reversed order of bit processing in ice_service_task for PLUG/UNPLUG
> 
>  drivers/net/ethernet/intel/ice/ice.h      | 14 +++++---------
>  drivers/net/ethernet/intel/ice/ice_main.c | 19 ++++++++-----------
>  2 files changed, 13 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/net/ethernet/intel/ice/ice.h
> b/drivers/net/ethernet/intel/ice/ice.h
> index 2f0b604abc5e..0ad9bab84617 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -506,6 +506,7 @@ enum ice_pf_flags {
>         ICE_FLAG_VF_VLAN_PRUNING,
>         ICE_FLAG_LINK_LENIENT_MODE_ENA,
>         ICE_FLAG_PLUG_AUX_DEV,
> +       ICE_FLAG_UNPLUG_AUX_DEV,
>         ICE_FLAG_MTU_CHANGED,
>         ICE_FLAG_GNSS,                  /* GNSS successfully
> initialized */
>         ICE_PF_FLAGS_NBITS              /* must be last */
> @@ -950,16 +951,11 @@ static inline void ice_set_rdma_cap(struct
> ice_pf *pf)
>   */
>  static inline void ice_clear_rdma_cap(struct ice_pf *pf)
>  {
> -       /* We can directly unplug aux device here only if the flag
> bit
> -        * ICE_FLAG_PLUG_AUX_DEV is not set because
> ice_unplug_aux_dev()
> -        * could race with ice_plug_aux_dev() called from
> -        * ice_service_task(). In this case we only clear that bit
> now and
> -        * aux device will be unplugged later once
> ice_plug_aux_device()
> -        * called from ice_service_task() finishes (see
> ice_service_task()).
> +       /* defer unplug to service task to avoid RTNL lock and
> +        * clear PLUG bit so that pending plugs don't interfere
>          */
> -       if (!test_and_clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags))
> -               ice_unplug_aux_dev(pf);
> -
> +       clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags);
> +       set_bit(ICE_FLAG_UNPLUG_AUX_DEV, pf->flags);
>         clear_bit(ICE_FLAG_RDMA_ENA, pf->flags);
>  }
>  #endif /* _ICE_H_ */
> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c
> b/drivers/net/ethernet/intel/ice/ice_main.c
> index a9a7f8b52140..b62c2f18eb5f 100644
> --- a/drivers/net/ethernet/intel/ice/ice_main.c
> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> @@ -2290,18 +2290,15 @@ static void ice_service_task(struct
> work_struct *work)
>                 }
>         }
>  
> -       if (test_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags)) {
> -               /* Plug aux device per request */
> -               ice_plug_aux_dev(pf);
> +       /* unplug aux dev per request, if an unplug request came in
> +        * while processing a plug request, this will handle it
> +        */
> +       if (test_and_clear_bit(ICE_FLAG_UNPLUG_AUX_DEV, pf->flags))
> +               ice_unplug_aux_dev(pf);
>  
> -               /* Mark plugging as done but check whether unplug was
> -                * requested during ice_plug_aux_dev() call
> -                * (e.g. from ice_clear_rdma_cap()) and if so then
> -                * plug aux device.
> -                */
> -               if (!test_and_clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf-
> >flags))
> -                       ice_unplug_aux_dev(pf);
> -       }
> +       /* Plug aux device per request */
> +       if (test_and_clear_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags))
> +               ice_plug_aux_dev(pf);
>  
>         if (test_and_clear_bit(ICE_FLAG_MTU_CHANGED, pf->flags)) {
>                 struct iidc_event *event;

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan