From: Thomas Gleixner <tglx@linutronix.de>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
Paul McKenney <paulmck@kernel.org>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Frederic Weisbecker <frederic@kernel.org>
Subject: [patch V3 3/6] softirq: Move various protections into inline helpers
Date: Tue, 09 Mar 2021 09:55:55 +0100 [thread overview]
Message-ID: <20210309085727.310118772@linutronix.de> (raw)
In-Reply-To: 20210309085552.815026890@linutronix.de
To allow reuse of the bulk of softirq processing code for RT and to avoid
#ifdeffery all over the place, split protections for various code sections
out into inline helpers so the RT variant can just replace them in one go.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
kernel/softirq.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -207,6 +207,32 @@ void __local_bh_enable_ip(unsigned long
}
EXPORT_SYMBOL(__local_bh_enable_ip);
+static inline void softirq_handle_begin(void)
+{
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_OFFSET);
+}
+
+static inline void softirq_handle_end(void)
+{
+ __local_bh_enable(SOFTIRQ_OFFSET);
+ WARN_ON_ONCE(in_interrupt());
+}
+
+static inline void ksoftirqd_run_begin(void)
+{
+ local_irq_disable();
+}
+
+static inline void ksoftirqd_run_end(void)
+{
+ local_irq_enable();
+}
+
+static inline bool should_wake_ksoftirqd(void)
+{
+ return true;
+}
+
static inline void invoke_softirq(void)
{
if (ksoftirqd_running(local_softirq_pending()))
@@ -319,7 +345,7 @@ asmlinkage __visible void __softirq_entr
pending = local_softirq_pending();
- __local_bh_disable_ip(_RET_IP_, SOFTIRQ_OFFSET);
+ softirq_handle_begin();
in_hardirq = lockdep_softirq_start();
account_softirq_enter(current);
@@ -370,8 +396,7 @@ asmlinkage __visible void __softirq_entr
account_softirq_exit(current);
lockdep_softirq_end(in_hardirq);
- __local_bh_enable(SOFTIRQ_OFFSET);
- WARN_ON_ONCE(in_interrupt());
+ softirq_handle_end();
current_restore_flags(old_flags, PF_MEMALLOC);
}
@@ -466,7 +491,7 @@ inline void raise_softirq_irqoff(unsigne
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
- if (!in_interrupt())
+ if (!in_interrupt() && should_wake_ksoftirqd())
wakeup_softirqd();
}
@@ -694,18 +719,18 @@ static int ksoftirqd_should_run(unsigned
static void run_ksoftirqd(unsigned int cpu)
{
- local_irq_disable();
+ ksoftirqd_run_begin();
if (local_softirq_pending()) {
/*
* We can safely run softirq on inline stack, as we are not deep
* in the task stack here.
*/
__do_softirq();
- local_irq_enable();
+ ksoftirqd_run_end();
cond_resched();
return;
}
- local_irq_enable();
+ ksoftirqd_run_end();
}
#ifdef CONFIG_HOTPLUG_CPU
next prev parent reply other threads:[~2021-03-09 8:58 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-09 8:55 [patch V3 0/6] softirq: Add RT specific softirq accounting Thomas Gleixner
2021-03-09 8:55 ` [patch V3 1/6] " Thomas Gleixner
2021-03-17 15:48 ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-03-09 8:55 ` [patch V3 2/6] irqtime: Make accounting correct on RT Thomas Gleixner
2021-03-17 15:48 ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-03-09 8:55 ` Thomas Gleixner [this message]
2021-03-17 15:48 ` [tip: irq/core] softirq: Move various protections into inline helpers tip-bot2 for Thomas Gleixner
2021-03-09 8:55 ` [patch V3 4/6] softirq: Make softirq control and processing RT aware Thomas Gleixner
2021-03-17 15:48 ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-03-09 8:55 ` [patch V3 5/6] tick/sched: Prevent false positive softirq pending warnings on RT Thomas Gleixner
2021-03-17 15:48 ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-03-09 8:55 ` [patch V3 6/6] rcu: Prevent false positive softirq warning " Thomas Gleixner
2021-03-09 13:06 ` Frederic Weisbecker
2021-03-17 15:48 ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-03-09 14:07 ` [patch V3 0/6] softirq: Add RT specific softirq accounting Peter Zijlstra
2021-03-09 21:32 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210309085727.310118772@linutronix.de \
--to=tglx@linutronix.de \
--cc=bigeasy@linutronix.de \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.