From: "Zhang, Qiang1" <qiang1.zhang@intel.com>
To: "paulmck@kernel.org" <paulmck@kernel.org>
Cc: "frederic@kernel.org" <frederic@kernel.org>,
"joel@joelfernandes.org" <joel@joelfernandes.org>,
"rcu@vger.kernel.org" <rcu@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH] srcu: Fix flush sup work warning in cleanup_srcu_struct()
Date: Wed, 22 Mar 2023 04:38:29 +0000 [thread overview]
Message-ID: <PH0PR11MB5880CAE4D4A58BD71AE984CFDA869@PH0PR11MB5880.namprd11.prod.outlook.com> (raw)
In-Reply-To: <f13e878f-65c3-40cb-aa3f-eb87ae6c9d6b@paulmck-laptop>
> > insmod rcutorture.ko
> > rmmod rcutorture.ko
> >
> > [ 209.437327] WARNING: CPU: 0 PID: 508 at kernel/workqueue.c:3167
> > __flush_work+0x50a/0x540 [ 209.437346] Modules linked in:
> > rcutorture(-) torture [last unloaded: rcutorture] [ 209.437382]
> > CPU: 0 PID: 508 Comm: rmmod Tainted: G W 6.3.0-rc1-yocto-standard+
> > [ 209.437406] RIP: 0010:__flush_work+0x50a/0x540 .....
> > [ 209.437758] flush_delayed_work+0x36/0x90 [ 209.437776]
> > cleanup_srcu_struct+0x68/0x2e0 [ 209.437817]
> > srcu_module_notify+0x71/0x140 [ 209.437854]
> > blocking_notifier_call_chain+0x9d/0xd0
> > [ 209.437880] __x64_sys_delete_module+0x223/0x2e0
> > [ 209.438046] do_syscall_64+0x43/0x90 [ 209.438062]
> > entry_SYSCALL_64_after_hwframe+0x72/0xdc
> >
> > For srcu objects defined with DEFINE_SRCU() or DEFINE_STATIC_SRCU(),
> > when compiling and loading as modules, the srcu_module_coming() is
> > invoked, allocate memory for srcu structure's->sda and initialize
> > sda structure, due to not fully initialize srcu structure's->sup, so
> > at this time the sup structure's->delaywork.func is null, if not
> > invoke init_srcu_struct_fields() before unloading modules, in
> > srcu_module_going() the __flush_work() find
> > work->func is empty, will raise the warning above.
> >
> > This commit add init_srcu_struct_fields() to initialize srcu
> > structure's->sup, in srcu_module_coming().
> >
> > Signed-off-by: Zqiang <qiang1.zhang@intel.com>
> >
> >Good catch, and thank you for testing the in-module case!
> >
> >One question below...
> >
> > Thanx, Paul
> >
> > ---
> > kernel/rcu/srcutree.c | 11 ++++++++---
> > 1 file changed, 8 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index
> > 1fb078abbdc9..42d8720e016c 100644
> > --- a/kernel/rcu/srcutree.c
> > +++ b/kernel/rcu/srcutree.c
> > @@ -1921,7 +1921,8 @@ static int srcu_module_coming(struct module *mod)
> > ssp->sda = alloc_percpu(struct srcu_data);
> > if (WARN_ON_ONCE(!ssp->sda))
> > return -ENOMEM;
> > - init_srcu_struct_data(ssp);
> > + if (WARN_ON_ONCE(init_srcu_struct_fields(ssp, true)))
> > + return -ENOMEM;
> >
> >Wouldn't it be better to simply delete the init_srcu_struct_data()?
> >
> >Then the first call to check_init_srcu_struct() would take care of
> >the initialization, just as for the non-module case. Or am I missing
> >something subtle?
>
> Hi Paul
>
> Maybe the check_init_srcu_struct() is always not invoked, for example,
> In rcutorture.c, here is such a definition DEFINE_STATIC_SRCU(srcu_ctl),
> but we use torture_type=rcu to test, there will not be any interface
> calling
> check_init_srcu_struct() to initialize srcu_ctl and set
> structure's->delaywork.func is process_srcu().
> when we unload the rcutorture module, invoke cleanup_srcu_struct() to
> flush sup structure's->delaywork.func, due to the func pointer is not
> initialize, it's null, will trigger warning.
>
> About kernel/workqueue.c:3167
>
> __flush_work
> if (WARN_ON(!work->func)) <---------trigger waning
> return false;
>
>
> and in init_srcu_struct_fields(ssp, true), wil set
> srcu_sup->sda_is_static is true and set srcu_sup-> srcu_gp_seq_needed
> is zero, after that when we call
> check_init_srcu_struct() again, it not be initialized again.
>
>
>Good point! In the non-module statically allocated case there is never a call to cleanup_srcu_struct().
>
>So suppose the code in srcu_module_coming() only did the alloc_percpu(), and then the
>code in srcu_module_going() only did the the matching
>free_percpu() instead of the current cleanup_srcu_struct()?
But in modules, for srcu objects defined with DEFINE_SRCU() or DEFINE_STATIC_SRCU(),
when a module is unloaded, we usually don't call cleanup_srcu_struct() in the module
unload function.
If in srcu_module_going() only do free_percpu(), the srcu_sup->node memory maybe
can not free and also lost the opportunity to refresh the running work.
Thanks
Zqiang
>
> Thanx, Paul
>
> Thanks
> Zqiang
>
> >
> >It should also be possible to eliminate duplicate code between the
> >in-module and non-module statically allocated initialization cases,
> >come to think of it.
> >
> > }
> > return 0;
> > }
> > @@ -1931,9 +1932,13 @@ static void srcu_module_going(struct module
> > *mod) {
> > int i;
> > struct srcu_struct **sspp = mod->srcu_struct_ptrs;
> > + struct srcu_struct *ssp;
> >
> > - for (i = 0; i < mod->num_srcu_structs; i++)
> > - cleanup_srcu_struct(*(sspp++));
> > + for (i = 0; i < mod->num_srcu_structs; i++) {
> > + ssp = *(sspp++);
> > + cleanup_srcu_struct(ssp);
> > + free_percpu(ssp->sda);
> > + }
> >
> >And good catch on another memory leak with this one, looks proper to
> >me.
> >
> > }
> >
> > /* Handle one module, either coming or going. */
> > --
> > 2.25.1
> >
next prev parent reply other threads:[~2023-03-22 4:38 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-21 8:13 [PATCH] srcu: Fix flush sup work warning in cleanup_srcu_struct() Zqiang
2023-03-21 15:04 ` Paul E. McKenney
2023-03-21 15:06 ` Paul E. McKenney
2023-03-22 1:15 ` Zhang, Qiang1
2023-03-22 3:54 ` Paul E. McKenney
2023-03-22 4:38 ` Zhang, Qiang1 [this message]
2023-03-22 14:57 ` Paul E. McKenney
2023-03-22 22:08 ` Zhang, Qiang1
2023-03-22 23:10 ` Paul E. McKenney
2023-03-22 23:45 ` Zhang, Qiang1
2023-03-22 23:52 ` Zhang, Qiang1
2023-03-23 0:07 ` Paul E. McKenney
2023-03-23 0:40 ` Zhang, Qiang1
2023-03-23 3:39 ` Zhang, Qiang1
2023-03-23 5:07 ` Paul E. McKenney
2023-03-23 6:21 ` Zhang, Qiang1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=PH0PR11MB5880CAE4D4A58BD71AE984CFDA869@PH0PR11MB5880.namprd11.prod.outlook.com \
--to=qiang1.zhang@intel.com \
--cc=frederic@kernel.org \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=rcu@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).