* [PATCH net] iavf: Fix deadlock in initialization
@ 2022-08-08 17:58 Ivan Vecera
2022-08-12 9:59 ` [Intel-wired-lan] " Jankowski, Konrad0
0 siblings, 1 reply; 2+ messages in thread
From: Ivan Vecera @ 2022-08-08 17:58 UTC (permalink / raw)
To: netdev
Cc: Jesse Brandeburg, Tony Nguyen, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Stefan Assmann,
moderated list:INTEL ETHERNET DRIVERS, open list
Fix deadlock that occurs when iavf interface is a part of failover
configuration.
1. Mutex crit_lock is taken at the beginning of iavf_watchdog_task()
2. Function iavf_init_config_adapter() is called when adapter
state is __IAVF_INIT_CONFIG_ADAPTER
3. iavf_init_config_adapter() calls register_netdevice() that emits
NETDEV_REGISTER event
4. Notifier function failover_event() then calls
net_failover_slave_register() that calls dev_open()
5. dev_open() calls iavf_open() that tries to take crit_lock in
end-less loop
Stack trace:
...
[ 790.251876] usleep_range_state+0x5b/0x80
[ 790.252547] iavf_open+0x37/0x1d0 [iavf]
[ 790.253139] __dev_open+0xcd/0x160
[ 790.253699] dev_open+0x47/0x90
[ 790.254323] net_failover_slave_register+0x122/0x220 [net_failover]
[ 790.255213] failover_slave_register.part.7+0xd2/0x180 [failover]
[ 790.256050] failover_event+0x122/0x1ab [failover]
[ 790.256821] notifier_call_chain+0x47/0x70
[ 790.257510] register_netdevice+0x20f/0x550
[ 790.258263] iavf_watchdog_task+0x7c8/0xea0 [iavf]
[ 790.259009] process_one_work+0x1a7/0x360
[ 790.259705] worker_thread+0x30/0x390
To fix the situation we should check the current adapter state after
first unsuccessful mutex_trylock() and return with -EBUSY if it is
__IAVF_INIT_CONFIG_ADAPTER.
Fixes: 226d528512cf ("iavf: fix locking of critical sections")
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
---
drivers/net/ethernet/intel/iavf/iavf_main.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 45d097a164ad..f9dcaadc7ea0 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -4085,8 +4085,17 @@ static int iavf_open(struct net_device *netdev)
return -EIO;
}
- while (!mutex_trylock(&adapter->crit_lock))
+ while (!mutex_trylock(&adapter->crit_lock)) {
+ /* If we are in __IAVF_INIT_CONFIG_ADAPTER state the crit_lock
+ * is already taken and iavf_open is called from an upper
+ * device's notifier reacting on NETDEV_REGISTER event.
+ * We have to leave here to avoid dead lock.
+ */
+ if (adapter->state == __IAVF_INIT_CONFIG_ADAPTER)
+ return -EBUSY;
+
usleep_range(500, 1000);
+ }
if (adapter->state != __IAVF_DOWN) {
err = -EBUSY;
--
2.35.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* RE: [Intel-wired-lan] [PATCH net] iavf: Fix deadlock in initialization
2022-08-08 17:58 [PATCH net] iavf: Fix deadlock in initialization Ivan Vecera
@ 2022-08-12 9:59 ` Jankowski, Konrad0
0 siblings, 0 replies; 2+ messages in thread
From: Jankowski, Konrad0 @ 2022-08-12 9:59 UTC (permalink / raw)
To: ivecera, netdev
Cc: moderated list:INTEL ETHERNET DRIVERS, Brandeburg, Jesse,
open list, Stefan Assmann, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, David S. Miller
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> Ivan Vecera
> Sent: Monday, August 8, 2022 7:59 PM
> To: netdev@vger.kernel.org
> Cc: moderated list:INTEL ETHERNET DRIVERS <intel-wired-
> lan@lists.osuosl.org>; Brandeburg, Jesse <jesse.brandeburg@intel.com>;
> open list <linux-kernel@vger.kernel.org>; Stefan Assmann
> <sassmann@kpanic.de>; Eric Dumazet <edumazet@google.com>; Jakub
> Kicinski <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>; David S.
> Miller <davem@davemloft.net>
> Subject: [Intel-wired-lan] [PATCH net] iavf: Fix deadlock in initialization
>
> Fix deadlock that occurs when iavf interface is a part of failover configuration.
>
> 1. Mutex crit_lock is taken at the beginning of iavf_watchdog_task() 2.
> Function iavf_init_config_adapter() is called when adapter
> state is __IAVF_INIT_CONFIG_ADAPTER
> 3. iavf_init_config_adapter() calls register_netdevice() that emits
> NETDEV_REGISTER event
> 4. Notifier function failover_event() then calls
> net_failover_slave_register() that calls dev_open() 5. dev_open() calls
> iavf_open() that tries to take crit_lock in
> end-less loop
>
> Stack trace:
> ...
> [ 790.251876] usleep_range_state+0x5b/0x80 [ 790.252547]
> iavf_open+0x37/0x1d0 [iavf] [ 790.253139] __dev_open+0xcd/0x160 [
> 790.253699] dev_open+0x47/0x90 [ 790.254323]
> net_failover_slave_register+0x122/0x220 [net_failover] [ 790.255213]
> failover_slave_register.part.7+0xd2/0x180 [failover] [ 790.256050]
> failover_event+0x122/0x1ab [failover] [ 790.256821]
> notifier_call_chain+0x47/0x70 [ 790.257510]
> register_netdevice+0x20f/0x550 [ 790.258263]
> iavf_watchdog_task+0x7c8/0xea0 [iavf] [ 790.259009]
> process_one_work+0x1a7/0x360 [ 790.259705] worker_thread+0x30/0x390
>
> To fix the situation we should check the current adapter state after first
> unsuccessful mutex_trylock() and return with -EBUSY if it is
> __IAVF_INIT_CONFIG_ADAPTER.
>
> Fixes: 226d528512cf ("iavf: fix locking of critical sections")
> Signed-off-by: Ivan Vecera <ivecera@redhat.com>
> ---
> drivers/net/ethernet/intel/iavf/iavf_main.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c
> b/drivers/net/ethernet/intel/iavf/iavf_main.c
> index 45d097a164ad..f9dcaadc7ea0 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_main.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
> @@ -4085,8 +4085,17 @@ static int iavf_open(struct net_device *netdev)
Tested-by: Konrad Jankowski <konrad0.jankowski@intel.com>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-08-12 9:59 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-08 17:58 [PATCH net] iavf: Fix deadlock in initialization Ivan Vecera
2022-08-12 9:59 ` [Intel-wired-lan] " Jankowski, Konrad0
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).