Re: [PATCH v2] rcu: Make tiny RCU support leak callbacks for debug-object errors

["Paul E. McKenney" <paulmck@kernel.org>]

On Wed, Jul 06, 2022 at 04:24:03AM +0000, Zhang, Qiang1 wrote:
> 
> On Wed, Jul 06, 2022 at 02:00:51AM +0000, Zhang, Qiang1 wrote:
> > On Fri, Jul 01, 2022 at 10:44:04AM +0800, Zqiang wrote:
> > > Currently, only tree RCU support leak callbacks setting when do
> > > duplicate call_rcu(). this commit add leak callbacks setting when
> > > fo duplicate call_rcu() for tiny RCU.
> > > 
> > > Signed-off-by: Zqiang <qiang1.zhang@intel.com>
> > 
> > >This does look plausible, thank you!
> > >
> > >What testing have you done?
> > >
> > >One important test for Tiny RCU is that the size of the kernel not
> > >grow without a very good reason.  In this case, the added code should
> > >be dead code in a production build (CONFIG_DEBUG_OBJECTS_RCU_HEAD=n),
> > >but it is good to check.
> > >
> > >It is of course also good to check that the messages print as expected,
> > >which is what rcutorture.object_debug is there to help with.
> > 
> > In the condition that the CONFIG_DEBUG_OBJECTS_RCU_HEAD=n, the function directly returns zero.
> > 
> > #else   /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */
> > static inline int debug_rcu_head_queue(struct rcu_head *head)
> > {
> >         return 0;
> > }
> >
> >Yes, like I said, the added code -should- be dead code.  But there is
> >often a gap between "should" and "is", for example, compilers don't
> >always do what we would like them to.  So please use the "size vmlinux"
> >command with and without your patch for a kernel built (both times)
> >with CONFIG_TINY_RCU=y and CONFIG_DEBUG_OBJECTS_RCU_HEAD==n.
> >
> >The rest of the test results look good, thank you!
> 
> Hi Paul
> 
> 1. CONFIG_TINY_RCU=y and CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
> 
> Original:
> text                      data                   bss                   dec                     hex       filename
> 26291319        20160143        15212544        61664006        3aceb06    vmlinux
> 
> Applay patch:
> text                       data                  bss                     dec                 hex    filename
> 26291319        20160431        15212544        61664294        3acec26 vmlinux
> 
> 2. CONFIG_TINY_RCU=y and CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
> 
> Original:
> text                    data                   bss                          dec     hex filename
> 26290663        20159823        15212544        61663030        3ace736 vmlinux
> 
> Applay patch:
> text                     data                     bss                    dec     hex filename
> 26290663        20159823        15212544        61663030        3ace736 vmlinux

Much better, thank you!

Please see below for the commit updated with this information and
wordsmithed a bit.  As always, please let me know if I messed something
up.

							Thanx, Paul

------------------------------------------------------------------------

commit 88cea4e18ed430aa1187063450236fc00408eaac
Author: Zqiang <qiang1.zhang@intel.com>
Date:   Fri Jul 1 10:44:04 2022 +0800

    rcu: Make tiny RCU support leak callbacks for debug-object errors
    
    Currently, only Tree RCU leaks callbacks setting when it detects a
    duplicate call_rcu().  This commit causes Tiny RCU to also leak
    callbacks in this situation.
    
    Because this is Tiny RCU, kernel size is important:
    
    1. CONFIG_TINY_RCU=y and CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
       (Production kernel)
    
        Original:
        text      data      bss       dec       hex     filename
        26290663  20159823  15212544  61663030  3ace736 vmlinux
    
        With this commit:
        text      data      bss       dec       hex     filename
        26290663  20159823  15212544  61663030  3ace736 vmlinux
    
    2. CONFIG_TINY_RCU=y and CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
       (Debugging kernel)
    
        Original:
        text      data      bss       dec       hex     filename
        26291319  20160143  15212544  61664006  3aceb06 vmlinux
    
        With this commit:
        text      data      bss       dec       hex     filename
        26291319  20160431  15212544  61664294  3acec26 vmlinux
    
    These results show that the kernel size is unchanged for production
    kernels, as desired.
    
    Signed-off-by: Zqiang <qiang1.zhang@intel.com>
    Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

diff --git a/kernel/rcu/tiny.c b/kernel/rcu/tiny.c
index f0561ee16b9c2..943d431b908f6 100644
--- a/kernel/rcu/tiny.c
+++ b/kernel/rcu/tiny.c
@@ -158,6 +158,10 @@ void synchronize_rcu(void)
 }
 EXPORT_SYMBOL_GPL(synchronize_rcu);
 
+static void tiny_rcu_leak_callback(struct rcu_head *rhp)
+{
+}
+
 /*
  * Post an RCU callback to be invoked after the end of an RCU grace
  * period.  But since we have but one CPU, that would be after any
@@ -165,9 +169,20 @@ EXPORT_SYMBOL_GPL(synchronize_rcu);
  */
 void call_rcu(struct rcu_head *head, rcu_callback_t func)
 {
+	static atomic_t doublefrees;
 	unsigned long flags;
 
-	debug_rcu_head_queue(head);
+	if (debug_rcu_head_queue(head)) {
+		if (atomic_inc_return(&doublefrees) < 4) {
+			pr_err("%s(): Double-freed CB %p->%pS()!!!  ", __func__, head, head->func);
+			mem_dump_obj(head);
+		}
+
+		if (!__is_kvfree_rcu_offset((unsigned long)head->func))
+			WRITE_ONCE(head->func, tiny_rcu_leak_callback);
+		return;
+	}
+
 	head->func = func;
 	head->next = NULL;