* [PATCH] x86/preempt: do not clear PREEMPT_NEED_RESCHED on preempt count reset
@ 2016-11-07 13:01 Martin Schwidefsky
2016-11-11 17:22 ` Peter Zijlstra
2016-11-16 12:12 ` [tip:sched/core] sched/x86: Do " tip-bot for Martin Schwidefsky
0 siblings, 2 replies; 3+ messages in thread
From: Martin Schwidefsky @ 2016-11-07 13:01 UTC (permalink / raw)
To: Thomas Gleixner, Peter Zijlstra, linux-kernel; +Cc: Martin Schwidefsky
The per-cpu preempt count of x86 contains two values, the actual preempt
count and the inverted PREEMPT_NEED_RESCHED bit. If a corrupted preempt
count is detected the preempt_count_set function is used to reset the
preempt count.
In case the inverted PREEMPT_NEED_RESCHED bit is zero at the time of the
reset, the preemption indication is lost. Use raw_cpu_cmpxchg_4 to reset
only the count part and leave the PREEMPT_NEED_RESCHED bit as it is.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
---
arch/x86/include/asm/preempt.h | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h
index 17f2186..ec1f3c6 100644
--- a/arch/x86/include/asm/preempt.h
+++ b/arch/x86/include/asm/preempt.h
@@ -24,7 +24,13 @@ static __always_inline int preempt_count(void)
static __always_inline void preempt_count_set(int pc)
{
- raw_cpu_write_4(__preempt_count, pc);
+ int old, new;
+
+ do {
+ old = raw_cpu_read_4(__preempt_count);
+ new = (old & PREEMPT_NEED_RESCHED) |
+ (pc & ~PREEMPT_NEED_RESCHED);
+ } while (raw_cpu_cmpxchg_4(__preempt_count, old, new) != old);
}
/*
--
1.9.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] x86/preempt: do not clear PREEMPT_NEED_RESCHED on preempt count reset
2016-11-07 13:01 [PATCH] x86/preempt: do not clear PREEMPT_NEED_RESCHED on preempt count reset Martin Schwidefsky
@ 2016-11-11 17:22 ` Peter Zijlstra
2016-11-16 12:12 ` [tip:sched/core] sched/x86: Do " tip-bot for Martin Schwidefsky
1 sibling, 0 replies; 3+ messages in thread
From: Peter Zijlstra @ 2016-11-11 17:22 UTC (permalink / raw)
To: Martin Schwidefsky; +Cc: Thomas Gleixner, linux-kernel
On Mon, Nov 07, 2016 at 02:01:00PM +0100, Martin Schwidefsky wrote:
> The per-cpu preempt count of x86 contains two values, the actual preempt
> count and the inverted PREEMPT_NEED_RESCHED bit. If a corrupted preempt
> count is detected the preempt_count_set function is used to reset the
> preempt count.
>
> In case the inverted PREEMPT_NEED_RESCHED bit is zero at the time of the
> reset, the preemption indication is lost. Use raw_cpu_cmpxchg_4 to reset
> only the count part and leave the PREEMPT_NEED_RESCHED bit as it is.
>
> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Thanks Martin!
I don't suppose this really hurts too much; if we hit a warn where this
restore is needed there are bigger problems, but given you've done the
patch already and it does improve consistency, I've taken it.
> ---
> arch/x86/include/asm/preempt.h | 8 +++++++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h
> index 17f2186..ec1f3c6 100644
> --- a/arch/x86/include/asm/preempt.h
> +++ b/arch/x86/include/asm/preempt.h
> @@ -24,7 +24,13 @@ static __always_inline int preempt_count(void)
>
> static __always_inline void preempt_count_set(int pc)
> {
> - raw_cpu_write_4(__preempt_count, pc);
> + int old, new;
> +
> + do {
> + old = raw_cpu_read_4(__preempt_count);
> + new = (old & PREEMPT_NEED_RESCHED) |
> + (pc & ~PREEMPT_NEED_RESCHED);
> + } while (raw_cpu_cmpxchg_4(__preempt_count, old, new) != old);
> }
>
> /*
> --
> 1.9.1
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* [tip:sched/core] sched/x86: Do not clear PREEMPT_NEED_RESCHED on preempt count reset
2016-11-07 13:01 [PATCH] x86/preempt: do not clear PREEMPT_NEED_RESCHED on preempt count reset Martin Schwidefsky
2016-11-11 17:22 ` Peter Zijlstra
@ 2016-11-16 12:12 ` tip-bot for Martin Schwidefsky
1 sibling, 0 replies; 3+ messages in thread
From: tip-bot for Martin Schwidefsky @ 2016-11-16 12:12 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, schwidefsky, hpa, tglx, peterz, torvalds, mingo
Commit-ID: f285144f81e814f39342dbf5321d6ba939890b1b
Gitweb: http://git.kernel.org/tip/f285144f81e814f39342dbf5321d6ba939890b1b
Author: Martin Schwidefsky <schwidefsky@de.ibm.com>
AuthorDate: Mon, 7 Nov 2016 14:01:00 +0100
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Wed, 16 Nov 2016 10:29:04 +0100
sched/x86: Do not clear PREEMPT_NEED_RESCHED on preempt count reset
The per-cpu preempt count of x86 contains two values, the actual preempt
count and the inverted PREEMPT_NEED_RESCHED bit. If a corrupted preempt
count is detected the preempt_count_set() function is used to reset the
preempt count.
In case the inverted PREEMPT_NEED_RESCHED bit is zero at the time of the
reset, the preemption indication is lost. Use raw_cpu_cmpxchg_4() to reset
only the count part and leave the PREEMPT_NEED_RESCHED bit as it is.
This improves the kernel's behavior when it runs into preempt count leaks
and tries to fix them up.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1478523660-733-1-git-send-email-schwidefsky@de.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/preempt.h | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h
index 17f2186..ec1f3c6 100644
--- a/arch/x86/include/asm/preempt.h
+++ b/arch/x86/include/asm/preempt.h
@@ -24,7 +24,13 @@ static __always_inline int preempt_count(void)
static __always_inline void preempt_count_set(int pc)
{
- raw_cpu_write_4(__preempt_count, pc);
+ int old, new;
+
+ do {
+ old = raw_cpu_read_4(__preempt_count);
+ new = (old & PREEMPT_NEED_RESCHED) |
+ (pc & ~PREEMPT_NEED_RESCHED);
+ } while (raw_cpu_cmpxchg_4(__preempt_count, old, new) != old);
}
/*
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-11-16 12:12 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-07 13:01 [PATCH] x86/preempt: do not clear PREEMPT_NEED_RESCHED on preempt count reset Martin Schwidefsky
2016-11-11 17:22 ` Peter Zijlstra
2016-11-16 12:12 ` [tip:sched/core] sched/x86: Do " tip-bot for Martin Schwidefsky
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).