From: Mark Rutland <mark.rutland@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, aryabinin@virtuozzo.com, boqun.feng@gmail.com, catalin.marinas@arm.com, dvyukov@google.com, mark.rutland@arm.com, mingo@kernel.org, peterz@infradead.org, will.deacon@arm.com Subject: [PATCH 2/6] locking/atomic, asm-generic: instrument atomic*andnot*() Date: Fri, 4 May 2018 18:39:33 +0100 [thread overview] Message-ID: <20180504173937.25300-3-mark.rutland@arm.com> (raw) In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com> We don't currently define instrumentation wrappers for the various forms of atomic*andnot*(), as these aren't implemented directly by x86. So that we can instrument architectures which provide these, let's define wrappers for all the variants of these atomics. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Will Deacon <will.deacon@arm.com> --- include/asm-generic/atomic-instrumented.h | 112 ++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h index 26f0e3098442..b1920f0f64ab 100644 --- a/include/asm-generic/atomic-instrumented.h +++ b/include/asm-generic/atomic-instrumented.h @@ -498,6 +498,62 @@ INSTR_ATOMIC64_AND(_release) #define atomic64_and_release atomic64_and_release #endif +#define INSTR_ATOMIC_ANDNOT(order) \ +static __always_inline void \ +atomic_andnot##order(int i, atomic_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + arch_atomic_andnot##order(i, v); \ +} + +#ifdef arch_atomic_andnot +INSTR_ATOMIC_ANDNOT() +#define atomic_andnot atomic_andnot +#endif + +#ifdef arch_atomic_andnot_relaxed +INSTR_ATOMIC_ANDNOT(_relaxed) +#define atomic_andnot_relaxed atomic_andnot_relaxed +#endif + +#ifdef arch_atomic_andnot_acquire +INSTR_ATOMIC_ANDNOT(_acquire) +#define atomic_andnot_acquire atomic_andnot_acquire +#endif + +#ifdef arch_atomic_andnot_release +INSTR_ATOMIC_ANDNOT(_release) +#define atomic_andnot_release atomic_andnot_release +#endif + +#define INSTR_ATOMIC64_ANDNOT(order) \ +static __always_inline void \ +atomic64_andnot##order(s64 i, atomic64_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + arch_atomic64_andnot##order(i, v); \ +} + +#ifdef arch_atomic64_andnot +INSTR_ATOMIC64_ANDNOT() +#define atomic64_andnot atomic64_andnot +#endif + +#ifdef arch_atomic64_andnot_relaxed +INSTR_ATOMIC64_ANDNOT(_relaxed) +#define atomic64_andnot_relaxed atomic64_andnot_relaxed +#endif + +#ifdef arch_atomic64_andnot_acquire +INSTR_ATOMIC64_ANDNOT(_acquire) +#define atomic64_andnot_acquire atomic64_andnot_acquire +#endif + +#ifdef arch_atomic64_andnot_release +INSTR_ATOMIC64_ANDNOT(_release) +#define atomic64_andnot_release atomic64_andnot_release +#endif + #define INSTR_ATOMIC_OR(order) \ static __always_inline void \ atomic_or##order(int i, atomic_t *v) \ @@ -984,6 +1040,62 @@ INSTR_ATOMIC64_FETCH_AND(_release) #define atomic64_fetch_and_release atomic64_fetch_and_release #endif +#define INSTR_ATOMIC_FETCH_ANDNOT(order) \ +static __always_inline int \ +atomic_fetch_andnot##order(int i, atomic_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + return arch_atomic_fetch_andnot##order(i, v); \ +} + +#ifdef arch_atomic_fetch_andnot +INSTR_ATOMIC_FETCH_ANDNOT() +#define atomic_fetch_andnot atomic_fetch_andnot +#endif + +#ifdef arch_atomic_fetch_andnot_relaxed +INSTR_ATOMIC_FETCH_ANDNOT(_relaxed) +#define atomic_fetch_andnot_relaxed atomic_fetch_andnot_relaxed +#endif + +#ifdef arch_atomic_fetch_andnot_acquire +INSTR_ATOMIC_FETCH_ANDNOT(_acquire) +#define atomic_fetch_andnot_acquire atomic_fetch_andnot_acquire +#endif + +#ifdef arch_atomic_fetch_andnot_release +INSTR_ATOMIC_FETCH_ANDNOT(_release) +#define atomic_fetch_andnot_release atomic_fetch_andnot_release +#endif + +#define INSTR_ATOMIC64_FETCH_ANDNOT(order) \ +static __always_inline s64 \ +atomic64_fetch_andnot##order(s64 i, atomic64_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + return arch_atomic64_fetch_andnot##order(i, v); \ +} + +#ifdef arch_atomic64_fetch_andnot +INSTR_ATOMIC64_FETCH_ANDNOT() +#define atomic64_fetch_andnot atomic64_fetch_andnot +#endif + +#ifdef arch_atomic64_fetch_andnot_relaxed +INSTR_ATOMIC64_FETCH_ANDNOT(_relaxed) +#define atomic64_fetch_andnot_relaxed atomic64_fetch_andnot_relaxed +#endif + +#ifdef arch_atomic64_fetch_andnot_acquire +INSTR_ATOMIC64_FETCH_ANDNOT(_acquire) +#define atomic64_fetch_andnot_acquire atomic64_fetch_andnot_acquire +#endif + +#ifdef arch_atomic64_fetch_andnot_release +INSTR_ATOMIC64_FETCH_ANDNOT(_release) +#define atomic64_fetch_andnot_release atomic64_fetch_andnot_release +#endif + #define INSTR_ATOMIC_FETCH_OR(order) \ static __always_inline int \ atomic_fetch_or##order(int i, atomic_t *v) \ -- 2.11.0
WARNING: multiple messages have this Message-ID (diff)
From: mark.rutland@arm.com (Mark Rutland) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/6] locking/atomic, asm-generic: instrument atomic*andnot*() Date: Fri, 4 May 2018 18:39:33 +0100 [thread overview] Message-ID: <20180504173937.25300-3-mark.rutland@arm.com> (raw) In-Reply-To: <20180504173937.25300-1-mark.rutland@arm.com> We don't currently define instrumentation wrappers for the various forms of atomic*andnot*(), as these aren't implemented directly by x86. So that we can instrument architectures which provide these, let's define wrappers for all the variants of these atomics. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Will Deacon <will.deacon@arm.com> --- include/asm-generic/atomic-instrumented.h | 112 ++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h index 26f0e3098442..b1920f0f64ab 100644 --- a/include/asm-generic/atomic-instrumented.h +++ b/include/asm-generic/atomic-instrumented.h @@ -498,6 +498,62 @@ INSTR_ATOMIC64_AND(_release) #define atomic64_and_release atomic64_and_release #endif +#define INSTR_ATOMIC_ANDNOT(order) \ +static __always_inline void \ +atomic_andnot##order(int i, atomic_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + arch_atomic_andnot##order(i, v); \ +} + +#ifdef arch_atomic_andnot +INSTR_ATOMIC_ANDNOT() +#define atomic_andnot atomic_andnot +#endif + +#ifdef arch_atomic_andnot_relaxed +INSTR_ATOMIC_ANDNOT(_relaxed) +#define atomic_andnot_relaxed atomic_andnot_relaxed +#endif + +#ifdef arch_atomic_andnot_acquire +INSTR_ATOMIC_ANDNOT(_acquire) +#define atomic_andnot_acquire atomic_andnot_acquire +#endif + +#ifdef arch_atomic_andnot_release +INSTR_ATOMIC_ANDNOT(_release) +#define atomic_andnot_release atomic_andnot_release +#endif + +#define INSTR_ATOMIC64_ANDNOT(order) \ +static __always_inline void \ +atomic64_andnot##order(s64 i, atomic64_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + arch_atomic64_andnot##order(i, v); \ +} + +#ifdef arch_atomic64_andnot +INSTR_ATOMIC64_ANDNOT() +#define atomic64_andnot atomic64_andnot +#endif + +#ifdef arch_atomic64_andnot_relaxed +INSTR_ATOMIC64_ANDNOT(_relaxed) +#define atomic64_andnot_relaxed atomic64_andnot_relaxed +#endif + +#ifdef arch_atomic64_andnot_acquire +INSTR_ATOMIC64_ANDNOT(_acquire) +#define atomic64_andnot_acquire atomic64_andnot_acquire +#endif + +#ifdef arch_atomic64_andnot_release +INSTR_ATOMIC64_ANDNOT(_release) +#define atomic64_andnot_release atomic64_andnot_release +#endif + #define INSTR_ATOMIC_OR(order) \ static __always_inline void \ atomic_or##order(int i, atomic_t *v) \ @@ -984,6 +1040,62 @@ INSTR_ATOMIC64_FETCH_AND(_release) #define atomic64_fetch_and_release atomic64_fetch_and_release #endif +#define INSTR_ATOMIC_FETCH_ANDNOT(order) \ +static __always_inline int \ +atomic_fetch_andnot##order(int i, atomic_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + return arch_atomic_fetch_andnot##order(i, v); \ +} + +#ifdef arch_atomic_fetch_andnot +INSTR_ATOMIC_FETCH_ANDNOT() +#define atomic_fetch_andnot atomic_fetch_andnot +#endif + +#ifdef arch_atomic_fetch_andnot_relaxed +INSTR_ATOMIC_FETCH_ANDNOT(_relaxed) +#define atomic_fetch_andnot_relaxed atomic_fetch_andnot_relaxed +#endif + +#ifdef arch_atomic_fetch_andnot_acquire +INSTR_ATOMIC_FETCH_ANDNOT(_acquire) +#define atomic_fetch_andnot_acquire atomic_fetch_andnot_acquire +#endif + +#ifdef arch_atomic_fetch_andnot_release +INSTR_ATOMIC_FETCH_ANDNOT(_release) +#define atomic_fetch_andnot_release atomic_fetch_andnot_release +#endif + +#define INSTR_ATOMIC64_FETCH_ANDNOT(order) \ +static __always_inline s64 \ +atomic64_fetch_andnot##order(s64 i, atomic64_t *v) \ +{ \ + kasan_check_write(v, sizeof(*v)); \ + return arch_atomic64_fetch_andnot##order(i, v); \ +} + +#ifdef arch_atomic64_fetch_andnot +INSTR_ATOMIC64_FETCH_ANDNOT() +#define atomic64_fetch_andnot atomic64_fetch_andnot +#endif + +#ifdef arch_atomic64_fetch_andnot_relaxed +INSTR_ATOMIC64_FETCH_ANDNOT(_relaxed) +#define atomic64_fetch_andnot_relaxed atomic64_fetch_andnot_relaxed +#endif + +#ifdef arch_atomic64_fetch_andnot_acquire +INSTR_ATOMIC64_FETCH_ANDNOT(_acquire) +#define atomic64_fetch_andnot_acquire atomic64_fetch_andnot_acquire +#endif + +#ifdef arch_atomic64_fetch_andnot_release +INSTR_ATOMIC64_FETCH_ANDNOT(_release) +#define atomic64_fetch_andnot_release atomic64_fetch_andnot_release +#endif + #define INSTR_ATOMIC_FETCH_OR(order) \ static __always_inline int \ atomic_fetch_or##order(int i, atomic_t *v) \ -- 2.11.0
next prev parent reply other threads:[~2018-05-04 17:39 UTC|newest] Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-05-04 17:39 [PATCH 0/6] arm64: add instrumented atomics Mark Rutland 2018-05-04 17:39 ` Mark Rutland 2018-05-04 17:39 ` [PATCH 1/6] locking/atomic, asm-generic: instrument ordering variants Mark Rutland 2018-05-04 17:39 ` Mark Rutland 2018-05-04 18:01 ` Peter Zijlstra 2018-05-04 18:01 ` Peter Zijlstra 2018-05-04 18:09 ` Mark Rutland 2018-05-04 18:09 ` Mark Rutland 2018-05-04 18:24 ` Peter Zijlstra 2018-05-04 18:24 ` Peter Zijlstra 2018-05-05 9:12 ` Mark Rutland 2018-05-05 9:12 ` Mark Rutland 2018-05-05 8:11 ` [PATCH] locking/atomics: Clean up the atomic.h maze of #defines Ingo Molnar 2018-05-05 8:11 ` Ingo Molnar 2018-05-05 8:36 ` [PATCH] locking/atomics: Simplify the op definitions in atomic.h some more Ingo Molnar 2018-05-05 8:36 ` Ingo Molnar 2018-05-05 8:54 ` [PATCH] locking/atomics: Combine the atomic_andnot() and atomic64_andnot() API definitions Ingo Molnar 2018-05-05 8:54 ` Ingo Molnar 2018-05-06 12:15 ` [tip:locking/core] " tip-bot for Ingo Molnar 2018-05-06 14:15 ` [PATCH] " Andrea Parri 2018-05-06 14:15 ` Andrea Parri 2018-05-06 12:14 ` [tip:locking/core] locking/atomics: Simplify the op definitions in atomic.h some more tip-bot for Ingo Molnar 2018-05-09 7:33 ` Peter Zijlstra 2018-05-09 13:03 ` Will Deacon 2018-05-15 8:54 ` Ingo Molnar 2018-05-15 8:35 ` Ingo Molnar 2018-05-15 11:41 ` Peter Zijlstra 2018-05-15 12:13 ` Peter Zijlstra 2018-05-15 15:43 ` Mark Rutland 2018-05-15 17:10 ` Peter Zijlstra 2018-05-15 17:53 ` Mark Rutland 2018-05-15 18:11 ` Peter Zijlstra 2018-05-15 18:15 ` Peter Zijlstra 2018-05-15 18:52 ` Linus Torvalds 2018-05-15 19:39 ` Peter Zijlstra 2018-05-21 17:12 ` Mark Rutland 2018-05-06 14:12 ` [PATCH] " Andrea Parri 2018-05-06 14:12 ` Andrea Parri 2018-05-06 14:57 ` Ingo Molnar 2018-05-06 14:57 ` Ingo Molnar 2018-05-07 9:54 ` Andrea Parri 2018-05-07 9:54 ` Andrea Parri 2018-05-18 18:43 ` Palmer Dabbelt 2018-05-18 18:43 ` Palmer Dabbelt 2018-05-05 8:47 ` [PATCH] locking/atomics: Clean up the atomic.h maze of #defines Peter Zijlstra 2018-05-05 8:47 ` Peter Zijlstra 2018-05-05 9:04 ` Ingo Molnar 2018-05-05 9:04 ` Ingo Molnar 2018-05-05 9:24 ` Peter Zijlstra 2018-05-05 9:24 ` Peter Zijlstra 2018-05-05 9:38 ` Ingo Molnar 2018-05-05 9:38 ` Ingo Molnar 2018-05-05 10:00 ` [RFC PATCH] locking/atomics/powerpc: Introduce optimized cmpxchg_release() family of APIs for PowerPC Ingo Molnar 2018-05-05 10:00 ` Ingo Molnar 2018-05-05 10:26 ` Boqun Feng 2018-05-05 10:26 ` Boqun Feng 2018-05-06 1:56 ` Benjamin Herrenschmidt 2018-05-06 1:56 ` Benjamin Herrenschmidt 2018-05-05 10:16 ` [PATCH] locking/atomics: Clean up the atomic.h maze of #defines Boqun Feng 2018-05-05 10:16 ` Boqun Feng 2018-05-05 10:35 ` [RFC PATCH] locking/atomics/powerpc: Clarify why the cmpxchg_relaxed() family of APIs falls back to full cmpxchg() Ingo Molnar 2018-05-05 10:35 ` Ingo Molnar 2018-05-05 11:28 ` Boqun Feng 2018-05-05 11:28 ` Boqun Feng 2018-05-05 13:27 ` [PATCH] locking/atomics/powerpc: Move cmpxchg helpers to asm/cmpxchg.h and define the full set of cmpxchg APIs Ingo Molnar 2018-05-05 13:27 ` Ingo Molnar 2018-05-05 14:03 ` Boqun Feng 2018-05-05 14:03 ` Boqun Feng 2018-05-06 12:11 ` Ingo Molnar 2018-05-06 12:11 ` Ingo Molnar 2018-05-07 1:04 ` Boqun Feng 2018-05-07 1:04 ` Boqun Feng 2018-05-07 6:50 ` Ingo Molnar 2018-05-07 6:50 ` Ingo Molnar 2018-05-06 12:13 ` [tip:locking/core] " tip-bot for Boqun Feng 2018-05-07 13:31 ` [PATCH v2] " Boqun Feng 2018-05-07 13:31 ` Boqun Feng 2018-05-05 9:05 ` [PATCH] locking/atomics: Clean up the atomic.h maze of #defines Dmitry Vyukov 2018-05-05 9:05 ` Dmitry Vyukov 2018-05-05 9:32 ` Peter Zijlstra 2018-05-05 9:32 ` Peter Zijlstra 2018-05-07 6:43 ` [RFC PATCH] locking/atomics/x86/64: Clean up and fix details of <asm/atomic64_64.h> Ingo Molnar 2018-05-07 6:43 ` Ingo Molnar 2018-05-05 9:09 ` [PATCH] locking/atomics: Clean up the atomic.h maze of #defines Ingo Molnar 2018-05-05 9:09 ` Ingo Molnar 2018-05-05 9:29 ` Peter Zijlstra 2018-05-05 9:29 ` Peter Zijlstra 2018-05-05 10:48 ` [PATCH] locking/atomics: Shorten the __atomic_op() defines to __op() Ingo Molnar 2018-05-05 10:48 ` Ingo Molnar 2018-05-05 10:59 ` Ingo Molnar 2018-05-05 10:59 ` Ingo Molnar 2018-05-06 12:15 ` [tip:locking/core] " tip-bot for Ingo Molnar 2018-05-06 12:14 ` [tip:locking/core] locking/atomics: Clean up the atomic.h maze of #defines tip-bot for Ingo Molnar 2018-05-04 17:39 ` Mark Rutland [this message] 2018-05-04 17:39 ` [PATCH 2/6] locking/atomic, asm-generic: instrument atomic*andnot*() Mark Rutland 2018-05-04 17:39 ` [PATCH 3/6] arm64: use <linux/atomic.h> for cmpxchg Mark Rutland 2018-05-04 17:39 ` Mark Rutland 2018-05-04 17:39 ` [PATCH 4/6] arm64: fix assembly constraints " Mark Rutland 2018-05-04 17:39 ` Mark Rutland 2018-05-04 17:39 ` [PATCH 5/6] arm64: use instrumented atomics Mark Rutland 2018-05-04 17:39 ` Mark Rutland 2018-05-04 17:39 ` [PATCH 6/6] arm64: instrument smp_{load_acquire,store_release} Mark Rutland 2018-05-04 17:39 ` Mark Rutland
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180504173937.25300-3-mark.rutland@arm.com \ --to=mark.rutland@arm.com \ --cc=aryabinin@virtuozzo.com \ --cc=boqun.feng@gmail.com \ --cc=catalin.marinas@arm.com \ --cc=dvyukov@google.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@kernel.org \ --cc=peterz@infradead.org \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.