From: Waiman Long <waiman.long@hpe.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: <linux-kernel@vger.kernel.org>, <torvalds@linux-foundation.org>,
<manfred@colorfullife.com>, <dave@stgolabs.net>,
<paulmck@linux.vnet.ibm.com>, <will.deacon@arm.com>,
<boqun.feng@gmail.com>, <tj@kernel.org>, <pablo@netfilter.org>,
<kaber@trash.net>, <davem@davemloft.net>, <oleg@redhat.com>,
<netfilter-devel@vger.kernel.org>, <sasha.levin@oracle.com>,
<hofrat@osadl.org>
Subject: Re: [PATCH -v3 7/8] locking: Move smp_cond_load_acquire() and friends into asm-generic/barrier.h
Date: Tue, 31 May 2016 16:01:06 -0400 [thread overview]
Message-ID: <574DED82.9080200@hpe.com> (raw)
In-Reply-To: <20160531094844.282806055@infradead.org>
On 05/31/2016 05:41 AM, Peter Zijlstra wrote:
> Since all asm/barrier.h should/must include asm-generic/barrier.h the
> latter is a good place for generic infrastructure like this.
>
> This also allows archs to override the new
> smp_acquire__after_ctrl_dep().
>
> Signed-off-by: Peter Zijlstra (Intel)<peterz@infradead.org>
>
> --- a/include/asm-generic/barrier.h
> +++ b/include/asm-generic/barrier.h
> @@ -194,7 +194,7 @@ do { \
> })
> #endif
>
> -#endif
> +#endif /* CONFIG_SMP */
>
> /* Barriers for virtual machine guests when talking to an SMP host */
> #define virt_mb() __smp_mb()
> @@ -207,5 +207,61 @@ do { \
> #define virt_store_release(p, v) __smp_store_release(p, v)
> #define virt_load_acquire(p) __smp_load_acquire(p)
>
> +/**
> + * smp_acquire__after_ctrl_dep() - Provide ACQUIRE ordering after a control dependency
> + *
> + * A control dependency provides a LOAD->STORE order, the additional RMB
> + * provides LOAD->LOAD order, together they provide LOAD->{LOAD,STORE} order,
> + * aka. (load)-ACQUIRE.
> + *
> + * Architectures that do not do load speculation can have this be barrier().
> + */
> +#ifndef smp_acquire__after_ctrl_dep
> +#define smp_acquire__after_ctrl_dep() smp_rmb()
> +#endif
> +
> +/**
> + * cmpwait - compare and wait for a variable to change
> + * @ptr: pointer to the variable to wait on
> + * @val: the value it should change from
> + *
> + * A simple constuct that waits for a variable to change from a known
> + * value; some architectures can do this in hardware.
> + */
> +#ifndef cmpwait
> +#define cmpwait(ptr, val) do { \
> + typeof (ptr) __ptr = (ptr); \
> + typeof (val) __val = (val); \
> + while (READ_ONCE(*__ptr) == __val) \
> + cpu_relax(); \
> +} while (0)
> +#endif
> +
> +/**
> + * smp_cond_load_acquire() - (Spin) wait for cond with ACQUIRE ordering
> + * @ptr: pointer to the variable to wait on
> + * @cond: boolean expression to wait for
> + *
> + * Equivalent to using smp_load_acquire() on the condition variable but employs
> + * the control dependency of the wait to reduce the barrier on many platforms.
> + *
> + * Due to C lacking lambda expressions we load the value of *ptr into a
> + * pre-named variable @VAL to be used in @cond.
> + */
> +#ifndef smp_cond_load_acquire
> +#define smp_cond_load_acquire(ptr, cond_expr) ({ \
> + typeof(ptr) __PTR = (ptr); \
> + typeof(*ptr) VAL; \
> + for (;;) { \
> + VAL = READ_ONCE(*__PTR); \
> + if (cond_expr) \
> + break; \
> + cmpwait(__PTR, VAL); \
> + } \
> + smp_acquire__after_ctrl_dep(); \
> + VAL; \
> +})
> +#endif
>
You are doing two READ_ONCE's in the smp_cond_load_acquire loop. Can we
change it to do just one READ_ONCE, like
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -229,12 +229,18 @@ do {
* value; some architectures can do this in hardware.
*/
#ifndef cmpwait
-#define cmpwait(ptr, val) do { \
+#define cmpwait(ptr, val) ({ \
typeof (ptr) __ptr = (ptr); \
- typeof (val) __val = (val); \
- while (READ_ONCE(*__ptr) == __val) \
+ typeof (val) __old = (val); \
+ typeof (val) __new; \
+ for (;;) { \
+ __new = READ_ONCE(*__ptr); \
+ if (__new != __old) \
+ break; \
cpu_relax(); \
-} while (0)
+ } \
+ __new; \
+})
#endif
/**
@@ -251,12 +257,11 @@ do {
#ifndef smp_cond_load_acquire
#define smp_cond_load_acquire(ptr, cond_expr) ({ \
typeof(ptr) __PTR = (ptr); \
- typeof(*ptr) VAL; \
+ typeof(*ptr) VAL = READ_ONCE(*__PTR); \
for (;;) { \
- VAL = READ_ONCE(*__PTR); \
if (cond_expr) \
break; \
- cmpwait(__PTR, VAL); \
+ VAL = cmpwait(__PTR, VAL); \
} \
smp_acquire__after_ctrl_dep(); \
VAL; \
Cheers,
Longman
next prev parent reply other threads:[~2016-05-31 20:01 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-31 9:41 [PATCH -v3 0/8] spin_unlock_wait borkage and assorted bits Peter Zijlstra
2016-05-31 9:41 ` [PATCH -v3 1/8] locking: Replace smp_cond_acquire with smp_cond_load_acquire Peter Zijlstra
2016-05-31 9:41 ` [PATCH -v3 2/8] locking: Introduce cmpwait() Peter Zijlstra
2016-05-31 9:41 ` [PATCH -v3 3/8] locking: Introduce smp_acquire__after_ctrl_dep Peter Zijlstra
2016-06-01 13:52 ` Boqun Feng
2016-06-01 16:22 ` Peter Zijlstra
2016-06-01 23:19 ` Boqun Feng
2016-05-31 9:41 ` [PATCH -v3 4/8] locking, arch: Update spin_unlock_wait() Peter Zijlstra
2016-06-01 11:24 ` Will Deacon
2016-06-01 11:37 ` Peter Zijlstra
2016-05-31 9:41 ` [PATCH -v3 5/8] locking: Update spin_unlock_wait users Peter Zijlstra
2016-05-31 9:41 ` [PATCH -v3 6/8] locking,netfilter: Fix nf_conntrack_lock() Peter Zijlstra
2016-05-31 9:41 ` [PATCH -v3 7/8] locking: Move smp_cond_load_acquire() and friends into asm-generic/barrier.h Peter Zijlstra
2016-05-31 20:01 ` Waiman Long [this message]
2016-06-01 9:31 ` Peter Zijlstra
2016-06-01 12:00 ` Will Deacon
2016-06-01 12:06 ` Peter Zijlstra
2016-06-01 12:13 ` Will Deacon
2016-06-01 12:45 ` Peter Zijlstra
2016-06-01 14:07 ` Will Deacon
2016-06-01 17:13 ` Peter Zijlstra
2016-06-01 16:53 ` Waiman Long
2016-05-31 9:41 ` [PATCH -v3 8/8] locking, tile: Provide TILE specific smp_acquire__after_ctrl_dep Peter Zijlstra
2016-05-31 15:32 ` Chris Metcalf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=574DED82.9080200@hpe.com \
--to=waiman.long@hpe.com \
--cc=boqun.feng@gmail.com \
--cc=dave@stgolabs.net \
--cc=davem@davemloft.net \
--cc=hofrat@osadl.org \
--cc=kaber@trash.net \
--cc=linux-kernel@vger.kernel.org \
--cc=manfred@colorfullife.com \
--cc=netfilter-devel@vger.kernel.org \
--cc=oleg@redhat.com \
--cc=pablo@netfilter.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=sasha.levin@oracle.com \
--cc=tj@kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).