* [Linux-kernel-mentees] [PATCH net 1/2 RESEND] ipmr: Fix RCU list debugging warning @ 2020-05-09 7:22 Amol Grover 2020-05-09 7:22 ` [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro Amol Grover 0 siblings, 1 reply; 9+ messages in thread From: Amol Grover @ 2020-05-09 7:22 UTC (permalink / raw) To: David S . Miller, Alexey Kuznetsov, Hideaki YOSHIFUJI, Jakub Kicinski Cc: Paul E . McKenney, netdev, linux-kernel, Madhuparna Bhowmik, Joel Fernandes, linux-kernel-mentees ipmr_for_each_table() macro uses list_for_each_entry_rcu() for traversing outside of an RCU read side critical section but under the protection of rtnl_mutex. Hence, add the corresponding lockdep expression to silence the following false-positive warning at boot: [ 4.319347] ============================= [ 4.319349] WARNING: suspicious RCU usage [ 4.319351] 5.5.4-stable #17 Tainted: G E [ 4.319352] ----------------------------- [ 4.319354] net/ipv4/ipmr.c:1757 RCU-list traversed in non-reader section!! Signed-off-by: Amol Grover <frextrite@gmail.com> --- net/ipv4/ipmr.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c index 6e68def66822..99c864eb6e34 100644 --- a/net/ipv4/ipmr.c +++ b/net/ipv4/ipmr.c @@ -110,7 +110,8 @@ static void ipmr_expire_process(struct timer_list *t); #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES #define ipmr_for_each_table(mrt, net) \ - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list) + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ + lockdep_rtnl_is_held()) static struct mr_table *ipmr_mr_table_iter(struct net *net, struct mr_table *mrt) -- 2.24.1 _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-09 7:22 [Linux-kernel-mentees] [PATCH net 1/2 RESEND] ipmr: Fix RCU list debugging warning Amol Grover @ 2020-05-09 7:22 ` Amol Grover 2020-05-09 21:19 ` Jakub Kicinski 0 siblings, 1 reply; 9+ messages in thread From: Amol Grover @ 2020-05-09 7:22 UTC (permalink / raw) To: David S . Miller, Alexey Kuznetsov, Hideaki YOSHIFUJI, Jakub Kicinski Cc: Paul E . McKenney, netdev, linux-kernel, Madhuparna Bhowmik, Joel Fernandes, linux-kernel-mentees ipmr_for_each_table() uses list_for_each_entry_rcu() for traversing outside of an RCU read-side critical section but under the protection of pernet_ops_rwsem. Hence add the corresponding lockdep expression to silence the following false-positive warning at boot: [ 0.645292] ============================= [ 0.645294] WARNING: suspicious RCU usage [ 0.645296] 5.5.4-stable #17 Not tainted [ 0.645297] ----------------------------- [ 0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!! Signed-off-by: Amol Grover <frextrite@gmail.com> --- net/ipv4/ipmr.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c index 99c864eb6e34..950ffe9943da 100644 --- a/net/ipv4/ipmr.c +++ b/net/ipv4/ipmr.c @@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags); static void ipmr_expire_process(struct timer_list *t); #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES -#define ipmr_for_each_table(mrt, net) \ - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ - lockdep_rtnl_is_held()) +#define ipmr_for_each_table(mrt, net) \ + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ + lockdep_rtnl_is_held() || \ + lockdep_is_held(&pernet_ops_rwsem)) static struct mr_table *ipmr_mr_table_iter(struct net *net, struct mr_table *mrt) -- 2.24.1 _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-09 7:22 ` [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro Amol Grover @ 2020-05-09 21:19 ` Jakub Kicinski 2020-05-12 5:17 ` Madhuparna Bhowmik 0 siblings, 1 reply; 9+ messages in thread From: Jakub Kicinski @ 2020-05-09 21:19 UTC (permalink / raw) To: Amol Grover Cc: Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, Madhuparna Bhowmik, Joel Fernandes, Alexey Kuznetsov, linux-kernel-mentees, David S . Miller On Sat, 9 May 2020 12:52:44 +0530 Amol Grover wrote: > ipmr_for_each_table() uses list_for_each_entry_rcu() for > traversing outside of an RCU read-side critical section but > under the protection of pernet_ops_rwsem. Hence add the > corresponding lockdep expression to silence the following > false-positive warning at boot: Thanks for the fix, the warning has been annoying me as well! > [ 0.645292] ============================= > [ 0.645294] WARNING: suspicious RCU usage > [ 0.645296] 5.5.4-stable #17 Not tainted > [ 0.645297] ----------------------------- > [ 0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!! please provide a fuller stack trace, it would have helped the review > Signed-off-by: Amol Grover <frextrite@gmail.com> > --- > net/ipv4/ipmr.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c > index 99c864eb6e34..950ffe9943da 100644 > --- a/net/ipv4/ipmr.c > +++ b/net/ipv4/ipmr.c > @@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags); > static void ipmr_expire_process(struct timer_list *t); > > #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES > -#define ipmr_for_each_table(mrt, net) \ > - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > - lockdep_rtnl_is_held()) > +#define ipmr_for_each_table(mrt, net) \ > + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > + lockdep_rtnl_is_held() || \ > + lockdep_is_held(&pernet_ops_rwsem)) This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks.. I think what is happening is this: ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be: #define ipmr_for_each_table(mrt, net) \ list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ lockdep_rtnl_is_held() || \ list_empty(&net->ipv4.mr_tables)) Thoughts? _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-09 21:19 ` Jakub Kicinski @ 2020-05-12 5:17 ` Madhuparna Bhowmik 2020-05-12 16:32 ` Jakub Kicinski ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Madhuparna Bhowmik @ 2020-05-12 5:17 UTC (permalink / raw) To: sfr Cc: Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, Madhuparna Bhowmik, Alexey Kuznetsov, Joel Fernandes, kuba, linux-kernel-mentees, David S . Miller On Sat, May 09, 2020 at 02:19:38PM -0700, Jakub Kicinski wrote: > On Sat, 9 May 2020 12:52:44 +0530 Amol Grover wrote: > > ipmr_for_each_table() uses list_for_each_entry_rcu() for > > traversing outside of an RCU read-side critical section but > > under the protection of pernet_ops_rwsem. Hence add the > > corresponding lockdep expression to silence the following > > false-positive warning at boot: > > Thanks for the fix, the warning has been annoying me as well! > > > [ 0.645292] ============================= > > [ 0.645294] WARNING: suspicious RCU usage > > [ 0.645296] 5.5.4-stable #17 Not tainted > > [ 0.645297] ----------------------------- > > [ 0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!! > > please provide a fuller stack trace, it would have helped the review > > > Signed-off-by: Amol Grover <frextrite@gmail.com> > > --- > > net/ipv4/ipmr.c | 7 ++++--- > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c > > index 99c864eb6e34..950ffe9943da 100644 > > --- a/net/ipv4/ipmr.c > > +++ b/net/ipv4/ipmr.c > > @@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags); > > static void ipmr_expire_process(struct timer_list *t); > > > > #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES > > -#define ipmr_for_each_table(mrt, net) \ > > - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > - lockdep_rtnl_is_held()) > > +#define ipmr_for_each_table(mrt, net) \ > > + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > + lockdep_rtnl_is_held() || \ > > + lockdep_is_held(&pernet_ops_rwsem)) > > This is a strange condition, IMHO. How can we be fine with either > lock.. This is supposed to be the writer side lock, one can't have > two writer side locks.. > > I think what is happening is this: > > ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() > > ipmr_new_table() returns an existing table if there is one, but > obviously none can exist at init. So a better fix would be: > > #define ipmr_for_each_table(mrt, net) \ > list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > lockdep_rtnl_is_held() || \ > list_empty(&net->ipv4.mr_tables)) > (adding Stephen) Hi Jakub, Thank you for your suggestion about this patch. Here is a stack trace for ipmr.c: [ 1.515015] TCP: Hash tables configured (established 8192 bind 8192) [ 1.516790] UDP hash table entries: 512 (order: 3, 49152 bytes, linear) [ 1.518177] UDP-Lite hash table entries: 512 (order: 3, 49152 bytes, linear) [ 1.519805] [ 1.520178] ============================= [ 1.520982] WARNING: suspicious RCU usage [ 1.521798] 5.7.0-rc2-00006-gb35af6a26b7c6f #1 Not tainted [ 1.522910] ----------------------------- [ 1.523671] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!! [ 1.525218] [ 1.525218] other info that might help us debug this: [ 1.525218] [ 1.526731] [ 1.526731] rcu_scheduler_active = 2, debug_locks = 1 [ 1.528004] 1 lock held by swapper/1: [ 1.528714] #0: c20be1d8 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0xd/0x30 [ 1.530433] [ 1.530433] stack backtrace: [ 1.531262] CPU: 0 PID: 1 Comm: swapper Not tainted 5.7.0-rc2-00006-gb35af6a26b7c6f #1 [ 1.532729] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 1.534305] Call Trace: [ 1.534758] ? ipmr_get_table+0x3c/0x70 [ 1.535430] ? ipmr_new_table+0x1c/0x60 [ 1.536173] ? ipmr_net_init+0x7b/0x170 [ 1.536923] ? register_pernet_subsys+0xd/0x30 [ 1.537810] ? ops_init+0x1a0/0x1e0 [ 1.538518] ? kmem_cache_create_usercopy+0x28a/0x350 [ 1.539752] ? register_pernet_operations+0xc9/0x1c0 [ 1.540630] ? ipv4_offload_init+0x65/0x65 [ 1.541451] ? register_pernet_subsys+0x19/0x30 [ 1.542357] ? ip_mr_init+0x28/0xff [ 1.543079] ? inet_init+0x17b/0x249 [ 1.543773] ? do_one_initcall+0xc5/0x240 [ 1.544532] ? parse_args+0x192/0x350 [ 1.545266] ? rcu_read_lock_sched_held+0x2f/0x60 [ 1.546180] ? trace_initcall_level+0x61/0x93 [ 1.547061] ? kernel_init_freeable+0x112/0x18a [ 1.547978] ? kernel_init_freeable+0x12b/0x18a [ 1.548974] ? rest_init+0x220/0x220 [ 1.549792] ? kernel_init+0x8/0x100 [ 1.550548] ? rest_init+0x220/0x220 [ 1.551288] ? schedule_tail_wrapper+0x6/0x8 [ 1.552136] ? rest_init+0x220/0x220 [ 1.552873] ? ret_from_fork+0x2e/0x38 ALso, there is a similar warning for ip6mr.c : ============================= WARNING: suspicious RCU usage 5.7.0-rc4-next-20200507-syzkaller #0 Not tainted ----------------------------- net/ipv6/ip6mr.c:124 RCU-list traversed in non-reader section!! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by swapper/0/1: #0: ffffffff8a7aae30 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0x16/0x40 net/core/net_namespace.c:1257 stack backtrace: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc4-next-20200507-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x18f/0x20d lib/dump_stack.c:118 ip6mr_get_table+0x153/0x180 net/ipv6/ip6mr.c:124 ip6mr_new_table+0x1b/0x70 net/ipv6/ip6mr.c:382 ip6mr_rules_init net/ipv6/ip6mr.c:236 [inline] ip6mr_net_init+0x133/0x3f0 net/ipv6/ip6mr.c:1310 ops_init+0xaf/0x420 net/core/net_namespace.c:151 __register_pernet_operations net/core/net_namespace.c:1140 [inline] register_pernet_operations+0x346/0x840 net/core/net_namespace.c:1217 register_pernet_subsys+0x25/0x40 net/core/net_namespace.c:1258 ip6_mr_init+0x49/0x152 net/ipv6/ip6mr.c:1363 inet6_init+0x1d7/0x6dc net/ipv6/af_inet6.c:1037 do_one_initcall+0x10a/0x7d0 init/main.c:1159 do_initcall_level init/main.c:1232 [inline] do_initcalls init/main.c:1248 [inline] do_basic_setup init/main.c:1268 [inline] kernel_init_freeable+0x501/0x5ae init/main.c:1454 kernel_init+0xd/0x1bb init/main.c:1359 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:351 Segment Routing with IPv6 mip6: Mobile IPv6 sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver ip6_gre: GRE over IPv6 tunneling driver > Thoughts? Do you think a similar fix (the one you suggested) is also applicable in the ip6mr case. Thank you, Madhuparna _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-12 5:17 ` Madhuparna Bhowmik @ 2020-05-12 16:32 ` Jakub Kicinski 2020-05-13 5:34 ` Madhuparna Bhowmik 2020-05-12 17:17 ` Amol Grover 2020-05-13 11:54 ` Stephen Rothwell 2 siblings, 1 reply; 9+ messages in thread From: Jakub Kicinski @ 2020-05-12 16:32 UTC (permalink / raw) To: Madhuparna Bhowmik Cc: sfr, Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, Joel Fernandes, Alexey Kuznetsov, linux-kernel-mentees, David S . Miller On Tue, 12 May 2020 10:47:05 +0530 Madhuparna Bhowmik wrote: > > > #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES > > > -#define ipmr_for_each_table(mrt, net) \ > > > - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > - lockdep_rtnl_is_held()) > > > +#define ipmr_for_each_table(mrt, net) \ > > > + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > + lockdep_rtnl_is_held() || \ > > > + lockdep_is_held(&pernet_ops_rwsem)) > > > > This is a strange condition, IMHO. How can we be fine with either > > lock.. This is supposed to be the writer side lock, one can't have > > two writer side locks.. > > > > I think what is happening is this: > > > > ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() > > > > ipmr_new_table() returns an existing table if there is one, but > > obviously none can exist at init. So a better fix would be: > > > > #define ipmr_for_each_table(mrt, net) \ > > list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > lockdep_rtnl_is_held() || \ > > list_empty(&net->ipv4.mr_tables)) > > > (adding Stephen) > > Hi Jakub, > > Thank you for your suggestion about this patch. > Here is a stack trace for ipmr.c: > > [...] Thanks! > > Thoughts? > > Do you think a similar fix (the one you suggested) is also applicable > in the ip6mr case. Yes, looking at the code it seems ip6mr has the exact same flow for netns init. _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-12 16:32 ` Jakub Kicinski @ 2020-05-13 5:34 ` Madhuparna Bhowmik 0 siblings, 0 replies; 9+ messages in thread From: Madhuparna Bhowmik @ 2020-05-13 5:34 UTC (permalink / raw) To: Jakub Kicinski Cc: sfr, Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, Madhuparna Bhowmik, Joel Fernandes, Alexey Kuznetsov, linux-kernel-mentees, David S . Miller On Tue, May 12, 2020 at 09:32:31AM -0700, Jakub Kicinski wrote: > On Tue, 12 May 2020 10:47:05 +0530 Madhuparna Bhowmik wrote: > > > > #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES > > > > -#define ipmr_for_each_table(mrt, net) \ > > > > - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > > - lockdep_rtnl_is_held()) > > > > +#define ipmr_for_each_table(mrt, net) \ > > > > + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > > + lockdep_rtnl_is_held() || \ > > > > + lockdep_is_held(&pernet_ops_rwsem)) > > > > > > This is a strange condition, IMHO. How can we be fine with either > > > lock.. This is supposed to be the writer side lock, one can't have > > > two writer side locks.. > > > > > > I think what is happening is this: > > > > > > ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() > > > > > > ipmr_new_table() returns an existing table if there is one, but > > > obviously none can exist at init. So a better fix would be: > > > > > > #define ipmr_for_each_table(mrt, net) \ > > > list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > lockdep_rtnl_is_held() || \ > > > list_empty(&net->ipv4.mr_tables)) > > > > > (adding Stephen) > > > > Hi Jakub, > > > > Thank you for your suggestion about this patch. > > Here is a stack trace for ipmr.c: > > > > [...] > > Thanks! > > > > Thoughts? > > > > Do you think a similar fix (the one you suggested) is also applicable > > in the ip6mr case. > > Yes, looking at the code it seems ip6mr has the exact same flow for > netns init. Alright, thanks a lot. I will send a patch for ip6mr.c soon. Thank you, Madhuparna _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-12 5:17 ` Madhuparna Bhowmik 2020-05-12 16:32 ` Jakub Kicinski @ 2020-05-12 17:17 ` Amol Grover 2020-05-12 21:59 ` Jakub Kicinski 2020-05-13 11:54 ` Stephen Rothwell 2 siblings, 1 reply; 9+ messages in thread From: Amol Grover @ 2020-05-12 17:17 UTC (permalink / raw) To: Madhuparna Bhowmik Cc: sfr, Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, kuba, Joel Fernandes, Alexey Kuznetsov, linux-kernel-mentees, David S . Miller On Tue, May 12, 2020 at 10:47:05AM +0530, Madhuparna Bhowmik wrote: > On Sat, May 09, 2020 at 02:19:38PM -0700, Jakub Kicinski wrote: > > On Sat, 9 May 2020 12:52:44 +0530 Amol Grover wrote: > > > ipmr_for_each_table() uses list_for_each_entry_rcu() for > > > traversing outside of an RCU read-side critical section but > > > under the protection of pernet_ops_rwsem. Hence add the > > > corresponding lockdep expression to silence the following > > > false-positive warning at boot: > > > > Thanks for the fix, the warning has been annoying me as well! > > > > > [ 0.645292] ============================= > > > [ 0.645294] WARNING: suspicious RCU usage > > > [ 0.645296] 5.5.4-stable #17 Not tainted > > > [ 0.645297] ----------------------------- > > > [ 0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!! > > > > please provide a fuller stack trace, it would have helped the review > > > > > Signed-off-by: Amol Grover <frextrite@gmail.com> > > > --- > > > net/ipv4/ipmr.c | 7 ++++--- > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c > > > index 99c864eb6e34..950ffe9943da 100644 > > > --- a/net/ipv4/ipmr.c > > > +++ b/net/ipv4/ipmr.c > > > @@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags); > > > static void ipmr_expire_process(struct timer_list *t); > > > > > > #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES > > > -#define ipmr_for_each_table(mrt, net) \ > > > - list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > - lockdep_rtnl_is_held()) > > > +#define ipmr_for_each_table(mrt, net) \ > > > + list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > + lockdep_rtnl_is_held() || \ > > > + lockdep_is_held(&pernet_ops_rwsem)) > > > > This is a strange condition, IMHO. How can we be fine with either > > lock.. This is supposed to be the writer side lock, one can't have > > two writer side locks.. > > > > I think what is happening is this: > > > > ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() > > > > ipmr_new_table() returns an existing table if there is one, but > > obviously none can exist at init. So a better fix would be: > > > > #define ipmr_for_each_table(mrt, net) \ > > list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > lockdep_rtnl_is_held() || \ > > list_empty(&net->ipv4.mr_tables)) > > Jakub, I agree, this condition looks better (and correct) than the one I proposed. I'll do the changes as necessary. Also, do you want me to add the full trace to the git commit body as well? I omitted it on purpose to not make it messy. > (adding Stephen) > > Hi Jakub, > > Thank you for your suggestion about this patch. > Here is a stack trace for ipmr.c: > > [ 1.515015] TCP: Hash tables configured (established 8192 bind 8192) > [ 1.516790] UDP hash table entries: 512 (order: 3, 49152 bytes, linear) > [ 1.518177] UDP-Lite hash table entries: 512 (order: 3, 49152 bytes, linear) > [ 1.519805] > [ 1.520178] ============================= > [ 1.520982] WARNING: suspicious RCU usage > [ 1.521798] 5.7.0-rc2-00006-gb35af6a26b7c6f #1 Not tainted > [ 1.522910] ----------------------------- > [ 1.523671] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!! > [ 1.525218] > [ 1.525218] other info that might help us debug this: > [ 1.525218] > [ 1.526731] > [ 1.526731] rcu_scheduler_active = 2, debug_locks = 1 > [ 1.528004] 1 lock held by swapper/1: > [ 1.528714] #0: c20be1d8 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0xd/0x30 > [ 1.530433] > [ 1.530433] stack backtrace: > [ 1.531262] CPU: 0 PID: 1 Comm: swapper Not tainted 5.7.0-rc2-00006-gb35af6a26b7c6f #1 > [ 1.532729] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 > [ 1.534305] Call Trace: > [ 1.534758] ? ipmr_get_table+0x3c/0x70 > [ 1.535430] ? ipmr_new_table+0x1c/0x60 > [ 1.536173] ? ipmr_net_init+0x7b/0x170 > [ 1.536923] ? register_pernet_subsys+0xd/0x30 > [ 1.537810] ? ops_init+0x1a0/0x1e0 > [ 1.538518] ? kmem_cache_create_usercopy+0x28a/0x350 > [ 1.539752] ? register_pernet_operations+0xc9/0x1c0 > [ 1.540630] ? ipv4_offload_init+0x65/0x65 > [ 1.541451] ? register_pernet_subsys+0x19/0x30 > [ 1.542357] ? ip_mr_init+0x28/0xff > [ 1.543079] ? inet_init+0x17b/0x249 > [ 1.543773] ? do_one_initcall+0xc5/0x240 > [ 1.544532] ? parse_args+0x192/0x350 > [ 1.545266] ? rcu_read_lock_sched_held+0x2f/0x60 > [ 1.546180] ? trace_initcall_level+0x61/0x93 > [ 1.547061] ? kernel_init_freeable+0x112/0x18a > [ 1.547978] ? kernel_init_freeable+0x12b/0x18a > [ 1.548974] ? rest_init+0x220/0x220 > [ 1.549792] ? kernel_init+0x8/0x100 > [ 1.550548] ? rest_init+0x220/0x220 > [ 1.551288] ? schedule_tail_wrapper+0x6/0x8 > [ 1.552136] ? rest_init+0x220/0x220 > [ 1.552873] ? ret_from_fork+0x2e/0x38 > Thank you for the stacktrace Madhuparna. > ALso, there is a similar warning for ip6mr.c : > > ============================= > WARNING: suspicious RCU usage > 5.7.0-rc4-next-20200507-syzkaller #0 Not tainted > ----------------------------- > net/ipv6/ip6mr.c:124 RCU-list traversed in non-reader section!! > > other info that might help us debug this: > > rcu_scheduler_active = 2, debug_locks = 1 > 1 lock held by swapper/0/1: > #0: ffffffff8a7aae30 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0x16/0x40 net/core/net_namespace.c:1257 > > stack backtrace: > CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc4-next-20200507-syzkaller #0 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 > Call Trace: > __dump_stack lib/dump_stack.c:77 [inline] > dump_stack+0x18f/0x20d lib/dump_stack.c:118 > ip6mr_get_table+0x153/0x180 net/ipv6/ip6mr.c:124 > ip6mr_new_table+0x1b/0x70 net/ipv6/ip6mr.c:382 > ip6mr_rules_init net/ipv6/ip6mr.c:236 [inline] > ip6mr_net_init+0x133/0x3f0 net/ipv6/ip6mr.c:1310 > ops_init+0xaf/0x420 net/core/net_namespace.c:151 > __register_pernet_operations net/core/net_namespace.c:1140 [inline] > register_pernet_operations+0x346/0x840 net/core/net_namespace.c:1217 > register_pernet_subsys+0x25/0x40 net/core/net_namespace.c:1258 > ip6_mr_init+0x49/0x152 net/ipv6/ip6mr.c:1363 > inet6_init+0x1d7/0x6dc net/ipv6/af_inet6.c:1037 > do_one_initcall+0x10a/0x7d0 init/main.c:1159 > do_initcall_level init/main.c:1232 [inline] > do_initcalls init/main.c:1248 [inline] > do_basic_setup init/main.c:1268 [inline] > kernel_init_freeable+0x501/0x5ae init/main.c:1454 > kernel_init+0xd/0x1bb init/main.c:1359 > ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:351 > Segment Routing with IPv6 > mip6: Mobile IPv6 > sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver > ip6_gre: GRE over IPv6 tunneling driver > > > Thoughts? > > Do you think a similar fix (the one you suggested) is also applicable > in the ip6mr case. > > Thank you, > Madhuparna _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-12 17:17 ` Amol Grover @ 2020-05-12 21:59 ` Jakub Kicinski 0 siblings, 0 replies; 9+ messages in thread From: Jakub Kicinski @ 2020-05-12 21:59 UTC (permalink / raw) To: Amol Grover Cc: sfr, Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, Madhuparna Bhowmik, Joel Fernandes, Alexey Kuznetsov, linux-kernel-mentees, David S . Miller On Tue, 12 May 2020 22:47:10 +0530 Amol Grover wrote: > > > This is a strange condition, IMHO. How can we be fine with either > > > lock.. This is supposed to be the writer side lock, one can't have > > > two writer side locks.. > > > > > > I think what is happening is this: > > > > > > ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() > > > > > > ipmr_new_table() returns an existing table if there is one, but > > > obviously none can exist at init. So a better fix would be: > > > > > > #define ipmr_for_each_table(mrt, net) \ > > > list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > > lockdep_rtnl_is_held() || \ > > > list_empty(&net->ipv4.mr_tables)) > > > > > Jakub, I agree, this condition looks better (and correct) than the one I > proposed. I'll do the changes as necessary. Also, do you want me to add > the full trace to the git commit body as well? I omitted it on purpose > to not make it messy. In this case we can leave it at the depth of IPMR code + the caller, so: [ 1.534758] ? ipmr_get_table+0x3c/0x70 [ 1.535430] ? ipmr_new_table+0x1c/0x60 [ 1.536173] ? ipmr_net_init+0x7b/0x170 [ 1.536923] ? register_pernet_subsys+0xd/0x30 This makes it clear that the problem happens at net namespace init. Thanks! _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro 2020-05-12 5:17 ` Madhuparna Bhowmik 2020-05-12 16:32 ` Jakub Kicinski 2020-05-12 17:17 ` Amol Grover @ 2020-05-13 11:54 ` Stephen Rothwell 2 siblings, 0 replies; 9+ messages in thread From: Stephen Rothwell @ 2020-05-13 11:54 UTC (permalink / raw) To: Madhuparna Bhowmik Cc: Paul E . McKenney, Hideaki YOSHIFUJI, netdev, linux-kernel, Alexey Kuznetsov, Joel Fernandes, kuba, linux-kernel-mentees, David S . Miller [-- Attachment #1.1: Type: text/plain, Size: 676 bytes --] Hi all, On Tue, 12 May 2020 10:47:05 +0530 Madhuparna Bhowmik <madhuparnabhowmik10@gmail.com> wrote: > > > > I think what is happening is this: > > > > ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table() > > > > ipmr_new_table() returns an existing table if there is one, but > > obviously none can exist at init. So a better fix would be: > > > > #define ipmr_for_each_table(mrt, net) \ > > list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \ > > lockdep_rtnl_is_held() || \ > > list_empty(&net->ipv4.mr_tables)) > > > (adding Stephen) I have changed the patch in my fixes tree to this. -- Cheers, Stephen Rothwell [-- Attachment #1.2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 201 bytes --] _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-05-13 11:54 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-05-09 7:22 [Linux-kernel-mentees] [PATCH net 1/2 RESEND] ipmr: Fix RCU list debugging warning Amol Grover 2020-05-09 7:22 ` [Linux-kernel-mentees] [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro Amol Grover 2020-05-09 21:19 ` Jakub Kicinski 2020-05-12 5:17 ` Madhuparna Bhowmik 2020-05-12 16:32 ` Jakub Kicinski 2020-05-13 5:34 ` Madhuparna Bhowmik 2020-05-12 17:17 ` Amol Grover 2020-05-12 21:59 ` Jakub Kicinski 2020-05-13 11:54 ` Stephen Rothwell
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).