From: Frederic Weisbecker <frederic@kernel.org>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
"David S . Miller" <davem@davemloft.net>,
Linus Torvalds <torvalds@linux-foundation.org>,
Thomas Gleixner <tglx@linutronix.de>,
Frederic Weisbecker <frederic@kernel.org>,
"Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
Ingo Molnar <mingo@kernel.org>,
Mauro Carvalho Chehab <mchehab@s-opensource.com>
Subject: [RFC PATCH 29/30] softirq: Make softirq processing softinterruptible
Date: Thu, 11 Oct 2018 01:12:16 +0200 [thread overview]
Message-ID: <1539213137-13953-30-git-send-email-frederic@kernel.org> (raw)
In-Reply-To: <1539213137-13953-1-git-send-email-frederic@kernel.org>
From: Frederic Weisbecker <fweisbec@gmail.com>
Make do_softirq() re-entrant and allow a vector, being either processed
or disabled, to be interrupted by another vector. This way a vector
won't be able to monopolize the CPU for a long while at the expense of
the others that may rely on some predictable latency, especially on
softirq disabled sections that used to disable all vectors.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: David S. Miller <davem@davemloft.net>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/softirq.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 457bf60..f4cb1ea 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -195,7 +195,7 @@ void __local_bh_enable_ip(unsigned long ip, unsigned int cnt, unsigned int bh)
if (cnt)
preempt_count_sub(cnt - 1);
- if (unlikely(!in_interrupt() && local_softirq_pending())) {
+ if (unlikely(!in_irq() && (local_softirq_pending() & local_softirq_enabled()))) {
/*
* Run softirq if any pending. And do it in its own stack
* as we may be calling this deep in a task call stack already.
@@ -387,7 +387,7 @@ asmlinkage __visible void __softirq_entry __do_softirq(void)
lockdep_softirq_end(in_hardirq);
account_irq_exit_time(current);
local_bh_exit();
- WARN_ON_ONCE(in_interrupt());
+ WARN_ON_ONCE(in_irq());
current_restore_flags(old_flags, PF_MEMALLOC);
}
@@ -396,12 +396,12 @@ asmlinkage __visible void do_softirq(void)
__u32 pending;
unsigned long flags;
- if (in_interrupt())
+ if (in_irq())
return;
local_irq_save(flags);
- pending = local_softirq_pending();
+ pending = local_softirq_pending() & local_softirq_enabled();
if (pending && !ksoftirqd_running(pending))
do_softirq_own_stack();
@@ -432,7 +432,7 @@ void irq_enter(void)
static inline void invoke_softirq(void)
{
- if (ksoftirqd_running(local_softirq_pending()))
+ if (ksoftirqd_running(local_softirq_pending() & local_softirq_enabled()))
return;
if (!force_irqthreads) {
@@ -481,7 +481,7 @@ void irq_exit(void)
#endif
account_irq_exit_time(current);
preempt_count_sub(HARDIRQ_OFFSET);
- if (!in_interrupt() && local_softirq_pending())
+ if (!in_irq() && (local_softirq_pending() & local_softirq_enabled()))
invoke_softirq();
tick_irq_exit();
@@ -712,13 +712,13 @@ void __init softirq_init(void)
static int ksoftirqd_should_run(unsigned int cpu)
{
- return local_softirq_pending();
+ return local_softirq_pending() & local_softirq_enabled();
}
static void run_ksoftirqd(unsigned int cpu)
{
local_irq_disable();
- if (local_softirq_pending()) {
+ if (local_softirq_pending() & local_softirq_enabled()) {
/*
* We can safely run softirq on inline stack, as we are not deep
* in the task stack here.
--
2.7.4
next prev parent reply other threads:[~2018-10-10 23:14 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-10 23:11 [RFC PATCH 00/30] softirq: Make softirqs soft-interruptible (+ per vector disablement) Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 01/30] x86: Revert "x86/irq: Demote irq_cpustat_t::__softirq_pending to u16" Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 02/30] arch/softirq: Rename softirq_pending fields to softirq_data Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 03/30] softirq: Implement local_softirq_pending() below softirq vector definition Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 04/30] softirq: Normalize softirq_pending naming scheme Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 05/30] softirq: Convert softirq_pending_set() to softirq_pending_nand() Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 06/30] softirq: Introduce disabled softirq vectors bits Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 07/30] softirq: Rename _local_bh_enable() to local_bh_enable_no_softirq() Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 08/30] softirq: Move vectors bits to bottom_half.h Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 09/30] x86: Init softirq enabled field Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 10/30] softirq: Check enabled bits on the softirq loop Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 11/30] net: Prepare netif_tx_lock_bh/netif_tx_unlock_bh() for handling softirq mask Frederic Weisbecker
2018-10-10 23:11 ` [RFC PATCH 12/30] rcu: Prepare rcu_read_[un]lock_bh() " Frederic Weisbecker
2018-10-16 5:28 ` Joel Fernandes
2018-10-17 0:44 ` Frederic Weisbecker
2018-10-17 0:55 ` Joel Fernandes
2018-10-10 23:12 ` [RFC PATCH 13/30] net: Prepare tcp_get_md5sig_pool() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 14/30] softirq: Introduce local_bh_disable_all() Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 15/30] net: Prepare [un]lock_sock_fast() for handling softirq mask Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 16/30] net: Prepare nf_log_buf_open() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 17/30] isdn: Prepare isdn_net_get_locked_lp() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 18/30] softirq: Prepare local_bh_disable() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 19/30] diva: Prepare diva_os_enter_spin_lock() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 20/30] tg3: Prepare tg3_full_[un]lock() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 21/30] locking: Prepare spin_lock_bh() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 22/30] seqlock: Prepare write_seq[un]lock_bh() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 23/30] rwlock: Prepare write_[un]lock_bh() " Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 24/30] softirq: Introduce Local_bh_enter/exit() Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 25/30] softirq: Push down softirq mask to __local_bh_disable_ip() Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 26/30] softirq: Increment the softirq offset on top of enabled bits Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 27/30] softirq: Swap softirq serving VS disable on preempt mask layout Frederic Weisbecker
2018-10-10 23:12 ` [RFC PATCH 28/30] softirq: Disable vector on execution Frederic Weisbecker
2018-10-10 23:12 ` Frederic Weisbecker [this message]
2018-10-16 4:15 ` [RFC PATCH 29/30] softirq: Make softirq processing softinterruptible Pavan Kondeti
2018-10-17 0:26 ` Frederic Weisbecker
2018-10-22 8:12 ` Pavan Kondeti
2018-10-10 23:12 ` [RFC PATCH 30/30] softirq: Tasklet/net-rx fixup Frederic Weisbecker
2018-10-16 22:03 ` [RFC PATCH 00/30] softirq: Make softirqs soft-interruptible (+ per vector disablement) Jonathan Corbet
2018-10-16 23:37 ` Richard Cochran
2018-10-17 1:20 ` Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1539213137-13953-30-git-send-email-frederic@kernel.org \
--to=frederic@kernel.org \
--cc=bigeasy@linutronix.de \
--cc=davem@davemloft.net \
--cc=fweisbec@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mchehab@s-opensource.com \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).