* [PATCH nf] netfilter: nfnetlink: avoid deadlock due to synchronous request_module
@ 2019-07-02 19:41 Florian Westphal
2019-07-03 11:57 ` Pablo Neira Ayuso
0 siblings, 1 reply; 2+ messages in thread
From: Florian Westphal @ 2019-07-02 19:41 UTC (permalink / raw)
To: netfilter-devel; +Cc: Florian Westphal, Thomas Jarosch, Juliana Rodrigueiro
Thomas and Juliana report a deadlock when running:
(rmmod nf_conntrack_netlink/xfrm_user)
conntrack -e NEW -E &
modprobe -v xfrm_user
They provided following analysis:
conntrack -e NEW -E
netlink_bind()
netlink_lock_table() -> increases "nl_table_users"
nfnetlink_bind()
# does not unlock the table as it's locked by netlink_bind()
__request_module()
call_usermodehelper_exec()
This triggers "modprobe nf_conntrack_netlink" from kernel, netlink_bind()
won't return until modprobe process is done.
"modprobe xfrm_user":
xfrm_user_init()
register_pernet_subsys()
-> grab pernet_ops_rwsem
..
netlink_table_grab()
calls schedule() as "nl_table_users" is non-zero
so modprobe is blocked because netlink_bind() increased
nl_table_users while also holding pernet_ops_rwsem.
"modprobe nf_conntrack_netlink" runs and inits nf_conntrack_netlink:
ctnetlink_init()
register_pernet_subsys()
-> blocks on "pernet_ops_rwsem" thanks to xfrm_user module
both modprobe processes wait on one another -- neither can make
progress.
Switch netlink_bind() to "nowait" modprobe -- this releases the netlink
table lock, which then allows both modprobe instances to complete.
Reported-by: Thomas Jarosch <thomas.jarosch@intra2net.com>
Reported-by: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
---
net/netfilter/nfnetlink.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/netfilter/nfnetlink.c b/net/netfilter/nfnetlink.c
index 92077d459109..4abbb452cf6c 100644
--- a/net/netfilter/nfnetlink.c
+++ b/net/netfilter/nfnetlink.c
@@ -578,7 +578,7 @@ static int nfnetlink_bind(struct net *net, int group)
ss = nfnetlink_get_subsys(type << 8);
rcu_read_unlock();
if (!ss)
- request_module("nfnetlink-subsys-%d", type);
+ request_module_nowait("nfnetlink-subsys-%d", type);
return 0;
}
#endif
--
2.21.0
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH nf] netfilter: nfnetlink: avoid deadlock due to synchronous request_module
2019-07-02 19:41 [PATCH nf] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Florian Westphal
@ 2019-07-03 11:57 ` Pablo Neira Ayuso
0 siblings, 0 replies; 2+ messages in thread
From: Pablo Neira Ayuso @ 2019-07-03 11:57 UTC (permalink / raw)
To: Florian Westphal; +Cc: netfilter-devel, Thomas Jarosch, Juliana Rodrigueiro
On Tue, Jul 02, 2019 at 09:41:40PM +0200, Florian Westphal wrote:
> Thomas and Juliana report a deadlock when running:
>
> (rmmod nf_conntrack_netlink/xfrm_user)
>
> conntrack -e NEW -E &
> modprobe -v xfrm_user
>
> They provided following analysis:
>
> conntrack -e NEW -E
> netlink_bind()
> netlink_lock_table() -> increases "nl_table_users"
> nfnetlink_bind()
> # does not unlock the table as it's locked by netlink_bind()
> __request_module()
> call_usermodehelper_exec()
>
> This triggers "modprobe nf_conntrack_netlink" from kernel, netlink_bind()
> won't return until modprobe process is done.
>
> "modprobe xfrm_user":
> xfrm_user_init()
> register_pernet_subsys()
> -> grab pernet_ops_rwsem
> ..
> netlink_table_grab()
> calls schedule() as "nl_table_users" is non-zero
>
> so modprobe is blocked because netlink_bind() increased
> nl_table_users while also holding pernet_ops_rwsem.
>
> "modprobe nf_conntrack_netlink" runs and inits nf_conntrack_netlink:
> ctnetlink_init()
> register_pernet_subsys()
> -> blocks on "pernet_ops_rwsem" thanks to xfrm_user module
>
> both modprobe processes wait on one another -- neither can make
> progress.
>
> Switch netlink_bind() to "nowait" modprobe -- this releases the netlink
> table lock, which then allows both modprobe instances to complete.
Applied, thanks.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2019-07-03 11:58 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-02 19:41 [PATCH nf] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Florian Westphal
2019-07-03 11:57 ` Pablo Neira Ayuso
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).