From: Marco Elver <elver@google.com>
To: Will Deacon <will@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
Eric Dumazet <edumazet@google.com>, Jann Horn <jannh@google.com>,
Kees Cook <keescook@chromium.org>,
Maddie Stone <maddiestone@google.com>,
"Paul E . McKenney" <paulmck@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Thomas Gleixner <tglx@linutronix.de>,
kernel-team@android.com, kernel-hardening@lists.openwall.com
Subject: Re: [RFC PATCH 03/21] list: Annotate lockless list primitives with data_race()
Date: Tue, 24 Mar 2020 17:23:30 +0100 [thread overview]
Message-ID: <CANpmjNPWpkxqZQJJOwmx0oqvzfcxhtqErjCzjRO_y0BQSmre8A@mail.gmail.com> (raw)
In-Reply-To: <20200324153643.15527-4-will@kernel.org>
On Tue, 24 Mar 2020 at 16:37, Will Deacon <will@kernel.org> wrote:
>
> Some list predicates can be used locklessly even with the non-RCU list
> implementations, since they effectively boil down to a test against
> NULL. For example, checking whether or not a list is empty is safe even
> in the presence of a concurrent, tearing write to the list head pointer.
> Similarly, checking whether or not an hlist node has been hashed is safe
> as well.
>
> Annotate these lockless list predicates with data_race() and READ_ONCE()
> so that KCSAN and the compiler are aware of what's going on. The writer
> side can then avoid having to use WRITE_ONCE() in the non-RCU
> implementation.
>
> Cc: Paul E. McKenney <paulmck@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Marco Elver <elver@google.com>
> Signed-off-by: Will Deacon <will@kernel.org>
> ---
> include/linux/list.h | 10 +++++-----
> include/linux/list_bl.h | 5 +++--
> include/linux/list_nulls.h | 6 +++---
> include/linux/llist.h | 2 +-
> 4 files changed, 12 insertions(+), 11 deletions(-)
>
> diff --git a/include/linux/list.h b/include/linux/list.h
> index 4fed5a0f9b77..4d9f5f9ed1a8 100644
> --- a/include/linux/list.h
> +++ b/include/linux/list.h
> @@ -279,7 +279,7 @@ static inline int list_is_last(const struct list_head *list,
> */
> static inline int list_empty(const struct list_head *head)
> {
> - return READ_ONCE(head->next) == head;
> + return data_race(READ_ONCE(head->next) == head);
Double-marking should never be necessary, at least if you want to make
KCSAN happy. From what I gather there is an unmarked write somewhere,
correct? In that case, KCSAN will still complain because if it sees a
race between this read and the other write, then at least one is still
plain (the write).
Then, my suggestion would be to mark the write with data_race() and
just leave this as a READ_ONCE(). Having a data_race() somewhere only
makes KCSAN stop reporting the race if the paired access is also
marked (be it with data_race() or _ONCE, etc.).
Alternatively, if marking the write is impossible, you can surround
the access with kcsan_disable_current()/kcsan_enable_current(). Or, as
a last resort, just leaving as-is is fine too, because KCSAN's default
config (still) has KCSAN_ASSUME_PLAIN_WRITES_ATOMIC selected.
Thanks,
-- Marco
> }
>
> /**
> @@ -524,7 +524,7 @@ static inline void list_splice_tail_init(struct list_head *list,
> */
> #define list_first_entry_or_null(ptr, type, member) ({ \
> struct list_head *head__ = (ptr); \
> - struct list_head *pos__ = READ_ONCE(head__->next); \
> + struct list_head *pos__ = data_race(READ_ONCE(head__->next)); \
> pos__ != head__ ? list_entry(pos__, type, member) : NULL; \
> })
>
> @@ -772,13 +772,13 @@ static inline void INIT_HLIST_NODE(struct hlist_node *h)
> * hlist_unhashed - Has node been removed from list and reinitialized?
> * @h: Node to be checked
> *
> - * Not that not all removal functions will leave a node in unhashed
> + * Note that not all removal functions will leave a node in unhashed
> * state. For example, hlist_nulls_del_init_rcu() does leave the
> * node in unhashed state, but hlist_nulls_del() does not.
> */
> static inline int hlist_unhashed(const struct hlist_node *h)
> {
> - return !READ_ONCE(h->pprev);
> + return data_race(!READ_ONCE(h->pprev));
> }
>
> /**
> @@ -787,7 +787,7 @@ static inline int hlist_unhashed(const struct hlist_node *h)
> */
> static inline int hlist_empty(const struct hlist_head *h)
> {
> - return !READ_ONCE(h->first);
> + return data_race(!READ_ONCE(h->first));
> }
>
> static inline void __hlist_del(struct hlist_node *n)
> diff --git a/include/linux/list_bl.h b/include/linux/list_bl.h
> index ae1b541446c9..1804fdb84dda 100644
> --- a/include/linux/list_bl.h
> +++ b/include/linux/list_bl.h
> @@ -51,7 +51,7 @@ static inline void INIT_HLIST_BL_NODE(struct hlist_bl_node *h)
>
> static inline bool hlist_bl_unhashed(const struct hlist_bl_node *h)
> {
> - return !h->pprev;
> + return data_race(!READ_ONCE(h->pprev));
> }
>
> static inline struct hlist_bl_node *hlist_bl_first(struct hlist_bl_head *h)
> @@ -71,7 +71,8 @@ static inline void hlist_bl_set_first(struct hlist_bl_head *h,
>
> static inline bool hlist_bl_empty(const struct hlist_bl_head *h)
> {
> - return !((unsigned long)READ_ONCE(h->first) & ~LIST_BL_LOCKMASK);
> + unsigned long first = data_race((unsigned long)READ_ONCE(h->first));
> + return !(first & ~LIST_BL_LOCKMASK);
> }
>
> static inline void hlist_bl_add_head(struct hlist_bl_node *n,
> diff --git a/include/linux/list_nulls.h b/include/linux/list_nulls.h
> index 3a9ff01e9a11..fa51a801bf32 100644
> --- a/include/linux/list_nulls.h
> +++ b/include/linux/list_nulls.h
> @@ -60,18 +60,18 @@ static inline unsigned long get_nulls_value(const struct hlist_nulls_node *ptr)
> * hlist_nulls_unhashed - Has node been removed and reinitialized?
> * @h: Node to be checked
> *
> - * Not that not all removal functions will leave a node in unhashed state.
> + * Note that not all removal functions will leave a node in unhashed state.
> * For example, hlist_del_init_rcu() leaves the node in unhashed state,
> * but hlist_nulls_del() does not.
> */
> static inline int hlist_nulls_unhashed(const struct hlist_nulls_node *h)
> {
> - return !READ_ONCE(h->pprev);
> + return data_race(!READ_ONCE(h->pprev));
> }
>
> static inline int hlist_nulls_empty(const struct hlist_nulls_head *h)
> {
> - return is_a_nulls(READ_ONCE(h->first));
> + return data_race(is_a_nulls(READ_ONCE(h->first)));
> }
>
> static inline void hlist_nulls_add_head(struct hlist_nulls_node *n,
> diff --git a/include/linux/llist.h b/include/linux/llist.h
> index 2e9c7215882b..c7f042b73899 100644
> --- a/include/linux/llist.h
> +++ b/include/linux/llist.h
> @@ -186,7 +186,7 @@ static inline void init_llist_head(struct llist_head *list)
> */
> static inline bool llist_empty(const struct llist_head *head)
> {
> - return READ_ONCE(head->first) == NULL;
> + return data_race(READ_ONCE(head->first) == NULL);
> }
>
> static inline struct llist_node *llist_next(struct llist_node *node)
> --
> 2.20.1
>
next prev parent reply other threads:[~2020-03-24 18:18 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-24 15:36 [RFC PATCH 00/21] Improve list integrity checking Will Deacon
2020-03-24 15:36 ` [RFC PATCH 01/21] list: Remove hlist_unhashed_lockless() Will Deacon
2020-03-24 16:27 ` Greg KH
2020-03-30 23:05 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 02/21] list: Remove hlist_nulls_unhashed_lockless() Will Deacon
2020-03-24 16:27 ` Greg KH
2020-03-30 23:07 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 03/21] list: Annotate lockless list primitives with data_race() Will Deacon
2020-03-24 16:20 ` Jann Horn
2020-03-24 16:26 ` Greg KH
2020-03-24 16:38 ` Jann Horn
2020-03-24 16:59 ` Greg KH
2020-03-24 18:22 ` Jann Horn
2020-03-24 16:23 ` Marco Elver [this message]
2020-03-24 21:33 ` Will Deacon
2020-03-31 13:10 ` Will Deacon
2020-04-01 6:34 ` Marco Elver
2020-04-01 8:40 ` Will Deacon
2020-03-24 16:51 ` Peter Zijlstra
2020-03-24 16:56 ` Jann Horn
2020-03-24 21:32 ` Will Deacon
2020-03-30 23:13 ` Paul E. McKenney
2020-04-24 17:39 ` Will Deacon
2020-04-27 19:24 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 04/21] timers: Use hlist_unhashed() instead of open-coding in timer_pending() Will Deacon
2020-03-24 16:30 ` Greg KH
2020-03-24 15:36 ` [RFC PATCH 05/21] list: Comment missing WRITE_ONCE() in __list_del() Will Deacon
2020-03-30 23:14 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 06/21] list: Remove superfluous WRITE_ONCE() from hlist_nulls implementation Will Deacon
2020-03-30 23:21 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 07/21] Revert "list: Use WRITE_ONCE() when adding to lists and hlists" Will Deacon
2020-03-30 23:19 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 08/21] Revert "list: Use WRITE_ONCE() when initializing list_head structures" Will Deacon
2020-03-30 23:25 ` Paul E. McKenney
2020-03-31 13:11 ` Will Deacon
2020-03-31 13:47 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 09/21] list: Remove unnecessary WRITE_ONCE() from hlist_bl_add_before() Will Deacon
2020-03-30 23:30 ` Paul E. McKenney
2020-03-31 12:37 ` Will Deacon
2020-03-24 15:36 ` [RFC PATCH 10/21] kernel-hacking: Make DEBUG_{LIST,PLIST,SG,NOTIFIERS} non-debug options Will Deacon
2020-03-24 16:42 ` Greg KH
2020-03-24 15:36 ` [RFC PATCH 11/21] list: Add integrity checking to hlist implementation Will Deacon
2020-03-24 15:36 ` [RFC PATCH 12/21] list: Poison ->next pointer for non-RCU deletion of 'hlist_nulls_node' Will Deacon
2020-03-30 23:32 ` Paul E. McKenney
2020-03-24 15:36 ` [RFC PATCH 13/21] list: Add integrity checking to hlist_nulls implementation Will Deacon
2020-03-24 15:36 ` [RFC PATCH 14/21] plist: Use CHECK_DATA_CORRUPTION instead of explicit {BUG,WARN}_ON() Will Deacon
2020-03-24 16:42 ` Greg KH
2020-03-24 15:36 ` [RFC PATCH 15/21] list_bl: Use CHECK_DATA_CORRUPTION instead of custom BUG_ON() wrapper Will Deacon
2020-03-24 15:36 ` [RFC PATCH 16/21] list_bl: Extend integrity checking in deletion routines Will Deacon
2020-03-24 15:36 ` [RFC PATCH 17/21] linux/bit_spinlock.h: Include linux/processor.h Will Deacon
2020-03-24 16:28 ` Greg KH
2020-03-24 21:08 ` Will Deacon
2020-03-24 15:36 ` [RFC PATCH 18/21] list_bl: Move integrity checking out of line Will Deacon
2020-03-24 15:36 ` [RFC PATCH 19/21] list_bl: Extend integrity checking to cover the same cases as 'hlist' Will Deacon
2020-03-24 15:36 ` [RFC PATCH 20/21] list: Format CHECK_DATA_CORRUPTION error messages consistently Will Deacon
2020-03-24 16:40 ` Greg KH
2020-03-24 15:36 ` [RFC PATCH 21/21] lkdtm: Extend list corruption checks Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CANpmjNPWpkxqZQJJOwmx0oqvzfcxhtqErjCzjRO_y0BQSmre8A@mail.gmail.com \
--to=elver@google.com \
--cc=edumazet@google.com \
--cc=jannh@google.com \
--cc=keescook@chromium.org \
--cc=kernel-hardening@lists.openwall.com \
--cc=kernel-team@android.com \
--cc=linux-kernel@vger.kernel.org \
--cc=maddiestone@google.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).