From: Thomas Gleixner <tglx@linutronix.de>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
Frederic Weisbecker <frederic@kernel.org>,
Paul McKenney <paulmck@kernel.org>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: [patch V2 3/9] softirq: Move various protections into inline helpers
Date: Fri, 04 Dec 2020 18:01:54 +0100 [thread overview]
Message-ID: <20201204170805.004816169@linutronix.de> (raw)
In-Reply-To: 20201204170151.960336698@linutronix.de
To allow reuse of the bulk of softirq processing code for RT and to avoid
#ifdeffery all over the place, split protections for various code sections
out into inline helpers so the RT variant can just replace them in one go.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
V2: Adapt to Frederics rework
---
kernel/softirq.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -204,6 +204,32 @@ void __local_bh_enable_ip(unsigned long
}
EXPORT_SYMBOL(__local_bh_enable_ip);
+static inline void softirq_handle_begin(void)
+{
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_OFFSET);
+}
+
+static inline void softirq_handle_end(void)
+{
+ __local_bh_enable(SOFTIRQ_OFFSET);
+ WARN_ON_ONCE(in_interrupt());
+}
+
+static inline void ksoftirqd_run_begin(void)
+{
+ local_irq_disable();
+}
+
+static inline void ksoftirqd_run_end(void)
+{
+ local_irq_enable();
+}
+
+static inline bool should_wake_ksoftirqd(void)
+{
+ return true;
+}
+
static inline void invoke_softirq(void)
{
if (ksoftirqd_running(local_softirq_pending()))
@@ -316,7 +342,7 @@ asmlinkage __visible void __softirq_entr
pending = local_softirq_pending();
- __local_bh_disable_ip(_RET_IP_, SOFTIRQ_OFFSET);
+ softirq_handle_begin();
in_hardirq = lockdep_softirq_start();
account_softirq_enter(current);
@@ -367,8 +393,7 @@ asmlinkage __visible void __softirq_entr
account_softirq_exit(current);
lockdep_softirq_end(in_hardirq);
- __local_bh_enable(SOFTIRQ_OFFSET);
- WARN_ON_ONCE(in_interrupt());
+ softirq_handle_end();
current_restore_flags(old_flags, PF_MEMALLOC);
}
@@ -463,7 +488,7 @@ inline void raise_softirq_irqoff(unsigne
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
- if (!in_interrupt())
+ if (!in_interrupt() && should_wake_ksoftirqd())
wakeup_softirqd();
}
@@ -641,18 +666,18 @@ static int ksoftirqd_should_run(unsigned
static void run_ksoftirqd(unsigned int cpu)
{
- local_irq_disable();
+ ksoftirqd_run_begin();
if (local_softirq_pending()) {
/*
* We can safely run softirq on inline stack, as we are not deep
* in the task stack here.
*/
__do_softirq();
- local_irq_enable();
+ ksoftirqd_run_end();
cond_resched();
return;
}
- local_irq_enable();
+ ksoftirqd_run_end();
}
#ifdef CONFIG_HOTPLUG_CPU
next prev parent reply other threads:[~2020-12-04 17:10 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-04 17:01 [patch V2 0/9] softirq: Make it RT aware Thomas Gleixner
2020-12-04 17:01 ` [patch V2 1/9] softirq: Add RT specific softirq accounting Thomas Gleixner
2020-12-07 13:06 ` Frederic Weisbecker
2020-12-04 17:01 ` [patch V2 2/9] irqtime: Make accounting correct on RT Thomas Gleixner
2020-12-07 0:23 ` Frederic Weisbecker
2020-12-07 0:57 ` Thomas Gleixner
2020-12-07 1:14 ` Frederic Weisbecker
2020-12-07 13:27 ` Frederic Weisbecker
2020-12-07 14:44 ` Thomas Gleixner
2020-12-04 17:01 ` Thomas Gleixner [this message]
2020-12-07 13:37 ` [patch V2 3/9] softirq: Move various protections into inline helpers Frederic Weisbecker
2020-12-04 17:01 ` [patch V2 4/9] softirq: Make softirq control and processing RT aware Thomas Gleixner
2020-12-07 14:16 ` Frederic Weisbecker
2020-12-07 15:08 ` Thomas Gleixner
2020-12-08 0:08 ` Frederic Weisbecker
2020-12-09 10:11 ` Peter Zijlstra
2020-12-09 12:36 ` Thomas Gleixner
2020-12-09 12:42 ` Peter Zijlstra
2020-12-09 13:30 ` Thomas Gleixner
2020-12-09 10:34 ` Peter Zijlstra
2020-12-04 17:01 ` [patch V2 5/9] tick/sched: Prevent false positive softirq pending warnings on RT Thomas Gleixner
2020-12-08 12:23 ` Frederic Weisbecker
2020-12-04 17:01 ` [patch V2 6/9] rcu: Prevent false positive softirq warning " Thomas Gleixner
2020-12-04 17:59 ` Paul E. McKenney
2020-12-04 17:01 ` [patch V2 7/9] softirq: Replace barrier() with cpu_relax() in tasklet_unlock_wait() Thomas Gleixner
2020-12-07 11:39 ` Peter Zijlstra
2020-12-07 15:21 ` Thomas Gleixner
2020-12-04 17:01 ` [patch V2 8/9] tasklets: Use static inlines for stub implementations Thomas Gleixner
2020-12-04 17:02 ` [patch V2 9/9] tasklets: Prevent kill/unlock_wait deadlock on RT Thomas Gleixner
2020-12-07 11:47 ` Peter Zijlstra
2020-12-07 14:00 ` Sebastian Andrzej Siewior
2020-12-07 14:27 ` Peter Zijlstra
2020-12-07 17:55 ` Thomas Gleixner
2020-12-07 15:22 ` Thomas Gleixner
2020-12-07 15:39 ` Sebastian Andrzej Siewior
2020-12-07 17:49 ` Thomas Gleixner
2020-12-07 17:50 ` Thomas Gleixner
2020-12-06 10:05 ` [patch V2 0/9] softirq: Make it RT aware Sebastian Andrzej Siewior
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201204170805.004816169@linutronix.de \
--to=tglx@linutronix.de \
--cc=bigeasy@linutronix.de \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).