[PATCH 0/3] rcu,sched: noinstr fixes

[PATCH 0/3] rcu,sched: noinstr fixes

[Peter Zijlstra]

A few noinstr fixes.. I'll stick the sched one in sched/urgent, Paul could you pick up the other two?

[PATCH 1/3] rcu: Always inline rcu_dynticks_task*_{enter,exit}()

[Peter Zijlstra]

RCU managed to grow a few noinstr violations:

  vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x0: call to rcu_dynticks_task_trace_enter() leaves .noinstr.text section
  vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0xe: call to rcu_dynticks_task_trace_exit() leaves .noinstr.text section

Fix them by adding __always_inline to the relevant trivial functions.

Also replace the noinstr with __always_inline for the existing
rcu_dynticks_task_*() functions since noinstr would force noinline
them, even when empty, which seems silly.

Fixes: 7d0c9c50c5a1 ("rcu-tasks: Avoid IPIing userspace/idle tasks if kernel is so built")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/rcu/tree_plugin.h |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1479,7 +1479,7 @@ static void rcu_bind_gp_kthread(void)
 }
 
 /* Record the current task on dyntick-idle entry. */
-static void noinstr rcu_dynticks_task_enter(void)
+static __always_inline void rcu_dynticks_task_enter(void)
 {
 #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
 	WRITE_ONCE(current->rcu_tasks_idle_cpu, smp_processor_id());
@@ -1487,7 +1487,7 @@ static void noinstr rcu_dynticks_task_en
 }
 
 /* Record no current task on dyntick-idle exit. */
-static void noinstr rcu_dynticks_task_exit(void)
+static __always_inline void rcu_dynticks_task_exit(void)
 {
 #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
 	WRITE_ONCE(current->rcu_tasks_idle_cpu, -1);
@@ -1495,7 +1495,7 @@ static void noinstr rcu_dynticks_task_ex
 }
 
 /* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
-static void rcu_dynticks_task_trace_enter(void)
+static __always_inline void rcu_dynticks_task_trace_enter(void)
 {
 #ifdef CONFIG_TASKS_TRACE_RCU
 	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
@@ -1504,7 +1504,7 @@ static void rcu_dynticks_task_trace_ente
 }
 
 /* Turn off heavyweight RCU tasks trace readers on idle/user exit. */
-static void rcu_dynticks_task_trace_exit(void)
+static __always_inline void rcu_dynticks_task_trace_exit(void)
 {
 #ifdef CONFIG_TASKS_TRACE_RCU
 	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))

[PATCH 2/3] rcu: Fix rcu_dynticks_curr_cpu_in_eqs() vs noinstr

[Peter Zijlstra]

  vmlinux.o: warning: objtool: rcu_nmi_enter()+0x36: call to __kasan_check_read() leaves .noinstr.text section

noinstr cannot have atomic_*() functions in because they're explicitly
annotated, use arch_atomic_*().

Fixes: 2be57f732889 ("rcu: Weaken ->dynticks accesses and updates")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/rcu/tree.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -327,7 +327,7 @@ static void rcu_dynticks_eqs_online(void
  */
 static __always_inline bool rcu_dynticks_curr_cpu_in_eqs(void)
 {
-	return !(atomic_read(this_cpu_ptr(&rcu_data.dynticks)) & 0x1);
+	return !(arch_atomic_read(this_cpu_ptr(&rcu_data.dynticks)) & 0x1);
 }
 
 /*

[PATCH 3/3] sched: Always inline is_percpu_thread()

[Peter Zijlstra]

  vmlinux.o: warning: objtool: check_preemption_disabled()+0x81: call to is_percpu_thread() leaves .noinstr.text section

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 include/linux/sched.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1730,7 +1730,7 @@ extern struct pid *cad_pid;
 #define tsk_used_math(p)			((p)->flags & PF_USED_MATH)
 #define used_math()				tsk_used_math(current)
 
-static inline bool is_percpu_thread(void)
+static __always_inline bool is_percpu_thread(void)
 {
 #ifdef CONFIG_SMP
 	return (current->flags & PF_NO_SETAFFINITY) &&

Re: [PATCH 0/3] rcu,sched: noinstr fixes

[Paul E. McKenney]

On Tue, Sep 28, 2021 at 10:40:20AM +0200, Peter Zijlstra wrote:
> A few noinstr fixes.. I'll stick the sched one in sched/urgent, Paul could you pick up the other two?

I queued those two, thank you!  I am guessing that you are looking for
me to send them ASAP.  If so, not a problem, but please confirm.

After all, yes, these are simple and low-risk, but in the absence of
any need to hurry them, extended testing is even lower risk.  ;-)

							Thanx, Paul

Re: [PATCH 1/3] rcu: Always inline rcu_dynticks_task*_{enter,exit}()

[Thomas Gleixner]

On Tue, Sep 28 2021 at 10:40, Peter Zijlstra wrote:

> RCU managed to grow a few noinstr violations:
>
>   vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x0: call to rcu_dynticks_task_trace_enter() leaves .noinstr.text section
>   vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0xe: call to rcu_dynticks_task_trace_exit() leaves .noinstr.text section
>
> Fix them by adding __always_inline to the relevant trivial functions.
>
> Also replace the noinstr with __always_inline for the existing
> rcu_dynticks_task_*() functions since noinstr would force noinline
> them, even when empty, which seems silly.
>
> Fixes: 7d0c9c50c5a1 ("rcu-tasks: Avoid IPIing userspace/idle tasks if kernel is so built")
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH 2/3] rcu: Fix rcu_dynticks_curr_cpu_in_eqs() vs noinstr

[Thomas Gleixner]

On Tue, Sep 28 2021 at 10:40, Peter Zijlstra wrote:

>   vmlinux.o: warning: objtool: rcu_nmi_enter()+0x36: call to __kasan_check_read() leaves .noinstr.text section
>
> noinstr cannot have atomic_*() functions in because they're explicitly
> annotated, use arch_atomic_*().
>
> Fixes: 2be57f732889 ("rcu: Weaken ->dynticks accesses and updates")
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH 3/3] sched: Always inline is_percpu_thread()

[Thomas Gleixner]

On Tue, Sep 28 2021 at 10:40, Peter Zijlstra wrote:

>   vmlinux.o: warning: objtool: check_preemption_disabled()+0x81: call to is_percpu_thread() leaves .noinstr.text section
>
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH 1/3] rcu: Always inline rcu_dynticks_task*_{enter,exit}()

[Paul E. McKenney]

On Wed, Sep 29, 2021 at 12:35:14AM +0200, Thomas Gleixner wrote:
> On Tue, Sep 28 2021 at 10:40, Peter Zijlstra wrote:
> 
> > RCU managed to grow a few noinstr violations:
> >
> >   vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x0: call to rcu_dynticks_task_trace_enter() leaves .noinstr.text section
> >   vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0xe: call to rcu_dynticks_task_trace_exit() leaves .noinstr.text section
> >
> > Fix them by adding __always_inline to the relevant trivial functions.
> >
> > Also replace the noinstr with __always_inline for the existing
> > rcu_dynticks_task_*() functions since noinstr would force noinline
> > them, even when empty, which seems silly.
> >
> > Fixes: 7d0c9c50c5a1 ("rcu-tasks: Avoid IPIing userspace/idle tasks if kernel is so built")
> > Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> 
> Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

Thank you!  I will apply to the pair of RCU commits at the next rebase.

							Thanx, Paul

Re: [PATCH 0/3] rcu,sched: noinstr fixes

[Peter Zijlstra]

On Tue, Sep 28, 2021 at 11:15:59AM -0700, Paul E. McKenney wrote:
> On Tue, Sep 28, 2021 at 10:40:20AM +0200, Peter Zijlstra wrote:
> > A few noinstr fixes.. I'll stick the sched one in sched/urgent, Paul could you pick up the other two?
> 
> I queued those two, thank you!  I am guessing that you are looking for
> me to send them ASAP.  If so, not a problem, but please confirm.

Either way works, but given syzcaller could potentially trigger these
holes (it did find a whole bunch of 'interesting' problems in the early
noinstr days) I'd prefer them earlier rather than later.

OTOH, the objtool patches that keep triggering them all over the place
are still in objtool/core for the next cycle.

Re: [PATCH 0/3] rcu,sched: noinstr fixes

[Paul E. McKenney]

On Wed, Sep 29, 2021 at 10:39:02AM +0200, Peter Zijlstra wrote:
> On Tue, Sep 28, 2021 at 11:15:59AM -0700, Paul E. McKenney wrote:
> > On Tue, Sep 28, 2021 at 10:40:20AM +0200, Peter Zijlstra wrote:
> > > A few noinstr fixes.. I'll stick the sched one in sched/urgent, Paul could you pick up the other two?
> > 
> > I queued those two, thank you!  I am guessing that you are looking for
> > me to send them ASAP.  If so, not a problem, but please confirm.
> 
> Either way works, but given syzcaller could potentially trigger these
> holes (it did find a whole bunch of 'interesting' problems in the early
> noinstr days) I'd prefer them earlier rather than later.
> 
> OTOH, the objtool patches that keep triggering them all over the place
> are still in objtool/core for the next cycle.

OK, let's split the difference, then.  Instead of the default v5.17
merge window, I will submit them to the upcoming v5.16 merge window.

							Thanx, Paul

[tip: sched/urgent] sched: Always inline is_percpu_thread()

[tip-bot2 for Peter Zijlstra]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     83d40a61046f73103b4e5d8f1310261487ff63b0
Gitweb:        https://git.kernel.org/tip/83d40a61046f73103b4e5d8f1310261487ff63b0
Author:        Peter Zijlstra <peterz@infradead.org>
AuthorDate:    Mon, 20 Sep 2021 15:31:11 +02:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Fri, 01 Oct 2021 13:57:57 +02:00

sched: Always inline is_percpu_thread()

  vmlinux.o: warning: objtool: check_preemption_disabled()+0x81: call to is_percpu_thread() leaves .noinstr.text section

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210928084218.063371959@infradead.org
---
 include/linux/sched.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 39039ce..c1a927d 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1720,7 +1720,7 @@ extern struct pid *cad_pid;
 #define tsk_used_math(p)			((p)->flags & PF_USED_MATH)
 #define used_math()				tsk_used_math(current)
 
-static inline bool is_percpu_thread(void)
+static __always_inline bool is_percpu_thread(void)
 {
 #ifdef CONFIG_SMP
 	return (current->flags & PF_NO_SETAFFINITY) &&