From: "Paul E. McKenney" <paulmck@kernel.org>
To: linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com,
kernel-team@fb.com, mingo@kernel.org
Cc: elver@google.com, andreyknvl@google.com, glider@google.com,
dvyukov@google.com, cai@lca.pw, boqun.feng@gmail.com,
Mark Rutland <mark.rutland@arm.com>,
"Paul E . McKenney" <paulmck@kernel.org>
Subject: [PATCH kcsan 4/8] kcsan: Reduce get_ctx() uses in kcsan_found_watchpoint()
Date: Wed, 21 Jul 2021 14:08:08 -0700 [thread overview]
Message-ID: <20210721210812.844740-4-paulmck@kernel.org> (raw)
In-Reply-To: <20210721210726.GA828672@paulmck-ThinkPad-P17-Gen-1>
From: Marco Elver <elver@google.com>
There are a number get_ctx() calls that are close to each other, which
results in poor codegen (repeated preempt_count loads).
Specifically in kcsan_found_watchpoint() (even though it's a slow-path)
it is beneficial to keep the race-window small until the watchpoint has
actually been consumed to avoid missed opportunities to report a race.
Let's clean it up a bit before we add more code in
kcsan_found_watchpoint().
Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/kcsan/core.c | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/kernel/kcsan/core.c b/kernel/kcsan/core.c
index d92977ede7e17..906100923b888 100644
--- a/kernel/kcsan/core.c
+++ b/kernel/kcsan/core.c
@@ -301,9 +301,9 @@ static inline void reset_kcsan_skip(void)
this_cpu_write(kcsan_skip, skip_count);
}
-static __always_inline bool kcsan_is_enabled(void)
+static __always_inline bool kcsan_is_enabled(struct kcsan_ctx *ctx)
{
- return READ_ONCE(kcsan_enabled) && get_ctx()->disable_count == 0;
+ return READ_ONCE(kcsan_enabled) && !ctx->disable_count;
}
/* Introduce delay depending on context and configuration. */
@@ -353,10 +353,17 @@ static noinline void kcsan_found_watchpoint(const volatile void *ptr,
atomic_long_t *watchpoint,
long encoded_watchpoint)
{
+ struct kcsan_ctx *ctx = get_ctx();
unsigned long flags;
bool consumed;
- if (!kcsan_is_enabled())
+ /*
+ * We know a watchpoint exists. Let's try to keep the race-window
+ * between here and finally consuming the watchpoint below as small as
+ * possible -- avoid unneccessarily complex code until consumed.
+ */
+
+ if (!kcsan_is_enabled(ctx))
return;
/*
@@ -364,14 +371,12 @@ static noinline void kcsan_found_watchpoint(const volatile void *ptr,
* reporting a race where e.g. the writer set up the watchpoint, but the
* reader has access_mask!=0, we have to ignore the found watchpoint.
*/
- if (get_ctx()->access_mask != 0)
+ if (ctx->access_mask)
return;
/*
- * Consume the watchpoint as soon as possible, to minimize the chances
- * of !consumed. Consuming the watchpoint must always be guarded by
- * kcsan_is_enabled() check, as otherwise we might erroneously
- * triggering reports when disabled.
+ * Consuming the watchpoint must be guarded by kcsan_is_enabled() to
+ * avoid erroneously triggering reports if the context is disabled.
*/
consumed = try_consume_watchpoint(watchpoint, encoded_watchpoint);
@@ -409,6 +414,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)
unsigned long access_mask;
enum kcsan_value_change value_change = KCSAN_VALUE_CHANGE_MAYBE;
unsigned long ua_flags = user_access_save();
+ struct kcsan_ctx *ctx = get_ctx();
unsigned long irq_flags = 0;
/*
@@ -417,7 +423,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)
*/
reset_kcsan_skip();
- if (!kcsan_is_enabled())
+ if (!kcsan_is_enabled(ctx))
goto out;
/*
@@ -489,7 +495,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)
* Re-read value, and check if it is as expected; if not, we infer a
* racy access.
*/
- access_mask = get_ctx()->access_mask;
+ access_mask = ctx->access_mask;
new = 0;
switch (size) {
case 1:
--
2.31.1.189.g2e36527f23
next prev parent reply other threads:[~2021-07-21 21:08 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-21 21:07 [PATCH kcsan 0/8] KCSAN updates for v5.15 Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 1/8] kcsan: Improve some Kconfig comments Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 2/8] kcsan: Remove CONFIG_KCSAN_DEBUG Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 3/8] kcsan: Introduce CONFIG_KCSAN_STRICT Paul E. McKenney
2021-07-21 21:08 ` Paul E. McKenney [this message]
2021-07-21 21:08 ` [PATCH kcsan 5/8] kcsan: Rework atomic.h into permissive.h Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 6/8] kcsan: Print if strict or non-strict during init Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 7/8] kcsan: permissive: Ignore data-racy 1-bit value changes Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 8/8] kcsan: Make strict mode imply interruptible watchers Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 1/4] tools/memory-model: Make read_foo_diagnostic() more clearly diagnostic Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 2/4] tools/memory-model: Add example for heuristic lockless reads Paul E. McKenney
2021-07-23 2:08 ` Alan Stern
2021-07-23 6:52 ` Manfred Spraul
2021-07-23 13:05 ` Alan Stern
2021-07-23 13:57 ` Manfred Spraul
2021-07-23 16:30 ` Paul E. McKenney
2021-07-23 17:08 ` Alan Stern
2021-07-23 20:32 ` Paul E. McKenney
2021-07-23 21:03 ` Alan Stern
2021-07-23 22:29 ` Paul E. McKenney
2021-07-23 16:24 ` Paul E. McKenney
2021-07-23 16:59 ` Alan Stern
2021-07-23 17:30 ` Paul E. McKenney
2021-07-23 18:11 ` Alan Stern
2021-07-23 20:28 ` Paul E. McKenney
2021-07-28 17:40 ` [PATCH v2 " Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 3/4] tools/memory-model: Heuristics using data_race() must handle all values Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 4/4] tools/memory-model: Document data_race(READ_ONCE()) Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210721210812.844740-4-paulmck@kernel.org \
--to=paulmck@kernel.org \
--cc=andreyknvl@google.com \
--cc=boqun.feng@gmail.com \
--cc=cai@lca.pw \
--cc=dvyukov@google.com \
--cc=elver@google.com \
--cc=glider@google.com \
--cc=kasan-dev@googlegroups.com \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).