[PATCH tip/core/rcu 0/4] Documentation changes for 3.17

[PATCH tip/core/rcu 0/4] Documentation changes for 3.17

[Paul E. McKenney]

Hello!

This series provides a few documentation updates:

1.	Clarify that if there is nothing awakened, then there is no
	memory barrier.

2.	Fix broken RTFP URL, courtesy of Pranith Kumar.

3.	Add acquire/release barrier to memory-barrier pairing rules.

4.	Add pointer to percpu-ref in rcuref.txt documentation.

							Thanx, Paul

------------------------------------------------------------------------

 b/Documentation/RCU/RTFP.txt        |    4 ++--
 b/Documentation/RCU/rcuref.txt      |    9 +++++++++
 b/Documentation/memory-barriers.txt |   25 +++++++++++++++++++++----
 3 files changed, 32 insertions(+), 6 deletions(-)

[PATCH tip/core/rcu 1/4] documentation: Clarify wake-up/memory-barrier relationship

[Paul E. McKenney]

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

This commit adds an example demonstrating that if a wake_up() doesn't
actually wake something up, no memory ordering is provided.

Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/memory-barriers.txt | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index f1dc4a215593..a6ca533a73fc 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -1893,6 +1893,21 @@ between the STORE to indicate the event and the STORE to set TASK_RUNNING:
 	    <general barrier>		  STORE current->state
 	LOAD event_indicated
 
+To repeat, this write memory barrier is present if and only if something
+is actually awakened.  To see this, consider the following sequence of
+events, where X and Y are both initially zero:
+
+	CPU 1				CPU 2
+	===============================	===============================
+	X = 1;				STORE event_indicated
+	smp_mb();			wake_up();
+	Y = 1;				wait_event(wq, Y == 1);
+	wake_up();			  load from Y sees 1, no memory barrier
+					load from X might see 0
+
+In contrast, if a wakeup does occur, CPU 2's load from X would be guaranteed
+to see 1.
+
 The available waker functions include:
 
 	complete();
-- 
1.8.1.5

[PATCH tip/core/rcu 2/4] documentation: Update reference, kerneltrap.org no longer works

[Paul E. McKenney]

From: Pranith Kumar <bobby.prani@gmail.com>

The kerneltrap.org site no longer works, so this commit updates it to
a working reference, namely gmane.

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/RTFP.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/RCU/RTFP.txt b/Documentation/RCU/RTFP.txt
index 2f0fcb2112d2..f29bcbc463e7 100644
--- a/Documentation/RCU/RTFP.txt
+++ b/Documentation/RCU/RTFP.txt
@@ -2451,8 +2451,8 @@ lot of {Linux} into your technology!!!"
 ,month="February"
 ,year="2010"
 ,note="Available:
-\url{http://kerneltrap.com/mailarchive/linux-netdev/2010/2/26/6270589}
-[Viewed March 20, 2011]"
+\url{http://thread.gmane.org/gmane.linux.network/153338}
+[Viewed June 9, 2014]"
 ,annotation={
 	Use a pair of list_head structures to support RCU-protected
 	resizable hash tables.
-- 
1.8.1.5

[PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Paul E. McKenney]

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

It is possible to pair acquire and release barriers with other barriers,
so this commit adds them to the list in the SMP barrier pairing section.

Reported-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Tejun Heo <tj@kernel.org>
---
 Documentation/memory-barriers.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index a6ca533a73fc..2a7c3c4fb53f 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -757,10 +757,12 @@ SMP BARRIER PAIRING
 When dealing with CPU-CPU interactions, certain types of memory barrier should
 always be paired.  A lack of appropriate pairing is almost certainly an error.
 
-A write barrier should always be paired with a data dependency barrier or read
-barrier, though a general barrier would also be viable.  Similarly a read
-barrier or a data dependency barrier should always be paired with at least an
-write barrier, though, again, a general barrier is viable:
+A write barrier should always be paired with a data dependency barrier,
+acquire barrier, release barrier, or read barrier, though a general
+barrier would also be viable.  Similarly a read barrier or a data
+dependency barrier should always be paired with at least a write barrier,
+an acquire barrier, or a release barrier, though, again, a general
+barrier is viable:
 
 	CPU 1		      CPU 2
 	===============	      ===============
-- 
1.8.1.5

[PATCH tip/core/rcu 4/4] documentation: Add pointer to percpu-ref for RCU and refcount

[Paul E. McKenney]

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/rcuref.txt | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Documentation/RCU/rcuref.txt b/Documentation/RCU/rcuref.txt
index 141d531aa14b..613033ff2b9b 100644
--- a/Documentation/RCU/rcuref.txt
+++ b/Documentation/RCU/rcuref.txt
@@ -1,5 +1,14 @@
 Reference-count design for elements of lists/arrays protected by RCU.
 
+
+Please note that the percpu-ref feature is likely your first
+stop if you need to combine reference counts and RCU.  Please see
+include/linux/percpu-refcount.h for more information.  However, in
+those unusual cases where percpu-ref would consume too much memory,
+please read on.
+
+------------------------------------------------------------------------
+
 Reference counting on elements of lists which are protected by traditional
 reader/writer spinlocks or semaphores are straightforward:
 
-- 
1.8.1.5

Re: [PATCH tip/core/rcu 0/4] Documentation changes for 3.17

[Josh Triplett]

On Mon, Jul 07, 2014 at 03:23:45PM -0700, Paul E. McKenney wrote:
> Hello!
> 
> This series provides a few documentation updates:
> 
> 1.	Clarify that if there is nothing awakened, then there is no
> 	memory barrier.
> 
> 2.	Fix broken RTFP URL, courtesy of Pranith Kumar.
> 
> 3.	Add acquire/release barrier to memory-barrier pairing rules.
> 
> 4.	Add pointer to percpu-ref in rcuref.txt documentation.

For all four:

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

Re: [PATCH tip/core/rcu 1/4] documentation: Clarify wake-up/memory-barrier relationship

[Peter Zijlstra]

[-- Attachment #1: Type: text/plain, Size: 448 bytes --]

On Mon, Jul 07, 2014 at 03:24:19PM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> This commit adds an example demonstrating that if a wake_up() doesn't
> actually wake something up, no memory ordering is provided.
> 
> Reported-by: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Thanks,

Acked-by: Peter Zijlstra <peterz@infradead.org>

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Peter Zijlstra]

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]

On Mon, Jul 07, 2014 at 03:24:21PM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> It is possible to pair acquire and release barriers with other barriers,
> so this commit adds them to the list in the SMP barrier pairing section.
> 
> Reported-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Reviewed-by: Tejun Heo <tj@kernel.org>


> +A write barrier should always be paired with a data dependency barrier,
> +acquire barrier, release barrier, or read barrier, though a general
> +barrier would also be viable.

   Similarly a read barrier or a data
> +dependency barrier should always be paired with at least a write barrier,
> +an acquire barrier, or a release barrier, though, again, a general
> +barrier is viable:

When I first read the Changelog I though you were going to add things
like:

  An acquire barrier should be paired with a release barrier, however
  .... barrier is also viable.

  A release barrier should be paired with an acquire barrier,... etc.

Now the above does seem to imply such rules but it isn't explicit in
them, since it only lists the requirements for read/write. Now since the
entire thing is indeed symmetric the implications are fairly strong,
still.

Also, it might be good to have a section on the ramifications of pairing
acquire/release with other than themselves, I have the feeling there's
subtle things there.

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH tip/core/rcu 0/4] Documentation changes for 3.17

[Lai Jiangshan]

On 07/08/2014 08:14 AM, Josh Triplett wrote:
> On Mon, Jul 07, 2014 at 03:23:45PM -0700, Paul E. McKenney wrote:
>> Hello!
>>
>> This series provides a few documentation updates:
>>
>> 1.	Clarify that if there is nothing awakened, then there is no
>> 	memory barrier.
>>
>> 2.	Fix broken RTFP URL, courtesy of Pranith Kumar.
>>
>> 3.	Add acquire/release barrier to memory-barrier pairing rules.
>>
>> 4.	Add pointer to percpu-ref in rcuref.txt documentation.
> 
> For all four:
> 
> Reviewed-by: Josh Triplett <josh@joshtriplett.org>
> .
> 

For patch 1 ("documentation: Clarify wake-up/memory-barrier relationship")
and patch 4 ("documentation: Add pointer to percpu-ref for RCU and refcount")

Reviewed-by: Lai Jiangshan <laijs@cn.fujitsu.com>

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Paul E. McKenney]

On Tue, Jul 08, 2014 at 09:59:02AM +0200, Peter Zijlstra wrote:
> On Mon, Jul 07, 2014 at 03:24:21PM -0700, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > 
> > It is possible to pair acquire and release barriers with other barriers,
> > so this commit adds them to the list in the SMP barrier pairing section.
> > 
> > Reported-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > Reviewed-by: Tejun Heo <tj@kernel.org>
> 
> 
> > +A write barrier should always be paired with a data dependency barrier,
> > +acquire barrier, release barrier, or read barrier, though a general
> > +barrier would also be viable.
> 
>    Similarly a read barrier or a data
> > +dependency barrier should always be paired with at least a write barrier,
> > +an acquire barrier, or a release barrier, though, again, a general
> > +barrier is viable:
> 
> When I first read the Changelog I though you were going to add things
> like:
> 
>   An acquire barrier should be paired with a release barrier, however
>   .... barrier is also viable.
> 
>   A release barrier should be paired with an acquire barrier,... etc.
> 
> Now the above does seem to imply such rules but it isn't explicit in
> them, since it only lists the requirements for read/write. Now since the
> entire thing is indeed symmetric the implications are fairly strong,
> still.

Good point, how about the following?

	General barriers pair with each other, though they also pair
	with most other types of barriers, albeit without transitivity.
	An acquire barrier pairs with a release barrier, but both may also
	pair with other barriers, including of course general barriers.
	A write barrier pairs with a data dependency barrier, an acquire
	barrier, a release barrier, a read barrier, or a general barrier.
	Similarly a read barrier or a data dependency barrier pairs
	with a write barrier, an acquire barrier, a release barrier,
	or a general barrier:

> Also, it might be good to have a section on the ramifications of pairing
> acquire/release with other than themselves, I have the feeling there's
> subtle things there.

It can get quite subtle.  For the time being, I am dodging this subtlety
by saying that only general barriers provide transitivity (see the
"TRANSITIVITY" section).

To give but one example of the subtlety, given X, Y, and Z all initially
zero where it matters:

	X=2;		Y=2;		Z=2;
	smp_wmb();	smp_wmb();	smp_wmb();
	Y=1;		Z=1;		X=1;

	BUG_ON(X==2 && Y==2 && Z==2); /* Never triggers. */

But:

	X=2;		Y=2;		Z=2;
	smp_wmb();	smp_wmb();	smp_mb();
	Y=1;		Z=1;		r1=X;

	BUG_ON(r1==0 && Y==2 && Z==2); /* Can trigger!!! */

Maybe some day we should capture this subtlety in memory-barriers.txt,
but we will first need a new generation of small children who are not
scared by the current document.  ;-)

								Thanx, Paul

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Peter Zijlstra]

[-- Attachment #1: Type: text/plain, Size: 1854 bytes --]

On Tue, Jul 08, 2014 at 08:31:17AM -0700, Paul E. McKenney wrote:

> Good point, how about the following?
> 
> 	General barriers pair with each other, though they also pair
> 	with most other types of barriers, albeit without transitivity.

> 	An acquire barrier pairs with a release barrier, but both may also
> 	pair with other barriers, including of course general barriers.

> 	A write barrier pairs with a data dependency barrier, an acquire
> 	barrier, a release barrier, a read barrier, or a general barrier.

> 	Similarly a read barrier or a data dependency barrier pairs
> 	with a write barrier, an acquire barrier, a release barrier,
> 	or a general barrier:

It might be clearer with the added whitespace, or as an explicit list  I
suppose, but yes.

> > Also, it might be good to have a section on the ramifications of pairing
> > acquire/release with other than themselves, I have the feeling there's
> > subtle things there.
> 
> It can get quite subtle.  For the time being, I am dodging this subtlety
> by saying that only general barriers provide transitivity (see the
> "TRANSITIVITY" section).

Ah, I was more thinking of the fact that ACQUIRE/RELEASE are
semi-permeable while READ/WRITE are memop dependent.

So any combination will be a semi-permeable memop dependent thing,
which is the most narrow barrier possible.

So if we thing of ACQUIRE/RELEASE as being 'half' a full barrier,
separated in direction, and READ/WRITE as being 'half' a full barrier
separated on type, then the combination is a 'quarter' barrier.

Not arguing they're not useful, just saying we need to be extra careful.

> Maybe some day we should capture this subtlety in memory-barriers.txt,
> but we will first need a new generation of small children who are not
> scared by the current document.  ;-)

Lolz :-)

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Paul E. McKenney]

On Mon, Jul 14, 2014 at 01:57:38PM +0200, Peter Zijlstra wrote:
> On Tue, Jul 08, 2014 at 08:31:17AM -0700, Paul E. McKenney wrote:
> 
> > Good point, how about the following?
> > 
> > 	General barriers pair with each other, though they also pair
> > 	with most other types of barriers, albeit without transitivity.
> 
> > 	An acquire barrier pairs with a release barrier, but both may also
> > 	pair with other barriers, including of course general barriers.
> 
> > 	A write barrier pairs with a data dependency barrier, an acquire
> > 	barrier, a release barrier, a read barrier, or a general barrier.
> 
> > 	Similarly a read barrier or a data dependency barrier pairs
> > 	with a write barrier, an acquire barrier, a release barrier,
> > 	or a general barrier:
> 
> It might be clearer with the added whitespace, or as an explicit list  I
> suppose, but yes.

If I get ambitious, I might try making a table out of it, but I am not
yet sure how I would set that up.  Something about having to say a lot
in each cell, but with only a small amount of room in which to say it.

> > > Also, it might be good to have a section on the ramifications of pairing
> > > acquire/release with other than themselves, I have the feeling there's
> > > subtle things there.
> > 
> > It can get quite subtle.  For the time being, I am dodging this subtlety
> > by saying that only general barriers provide transitivity (see the
> > "TRANSITIVITY" section).
> 
> Ah, I was more thinking of the fact that ACQUIRE/RELEASE are
> semi-permeable while READ/WRITE are memop dependent.
> 
> So any combination will be a semi-permeable memop dependent thing,
> which is the most narrow barrier possible.
> 
> So if we thing of ACQUIRE/RELEASE as being 'half' a full barrier,
> separated in direction, and READ/WRITE as being 'half' a full barrier
> separated on type, then the combination is a 'quarter' barrier.
> 
> Not arguing they're not useful, just saying we need to be extra careful.

I do agree completely about the need for extra care!

For whatever it is worth, the permeability and read-write properties
are isolated to each barrier in the pair.  For example, with "a" and
"b" both initially zero:

	CPU 1				CPU 2
	-----				-----
	ACCESS_ONCE(a) = 1;		r1 = b;
	smp_store_release(&b, 1);	smp_rmb();
	ACCESS_ONCE(c) = 1;		r2 = a;
					ACCESS_ONCE(c) = 2;

The outcome r1==1&&r2==0 is prohibited, but the ordering of the stores
to "c" are not ordered: CPU 1's smp_store_release() does not affect
later accesses, and CPU 2's smp_rmb() does not order stores.

Not sure that it is worth adding this sort of example, though.

							Thanx, Paul

> > Maybe some day we should capture this subtlety in memory-barriers.txt,
> > but we will first need a new generation of small children who are not
> > scared by the current document.  ;-)
> 
> Lolz :-)

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Peter Zijlstra]

[-- Attachment #1: Type: text/plain, Size: 3459 bytes --]

On Wed, Jul 16, 2014 at 05:16:26AM -0700, Paul E. McKenney wrote:
> On Mon, Jul 14, 2014 at 01:57:38PM +0200, Peter Zijlstra wrote:
> > On Tue, Jul 08, 2014 at 08:31:17AM -0700, Paul E. McKenney wrote:
> > 
> > > Good point, how about the following?
> > > 
> > > 	General barriers pair with each other, though they also pair
> > > 	with most other types of barriers, albeit without transitivity.
> > 
> > > 	An acquire barrier pairs with a release barrier, but both may also
> > > 	pair with other barriers, including of course general barriers.
> > 
> > > 	A write barrier pairs with a data dependency barrier, an acquire
> > > 	barrier, a release barrier, a read barrier, or a general barrier.
> > 
> > > 	Similarly a read barrier or a data dependency barrier pairs
> > > 	with a write barrier, an acquire barrier, a release barrier,
> > > 	or a general barrier:
> > 
> > It might be clearer with the added whitespace, or as an explicit list  I
> > suppose, but yes.
> 
> If I get ambitious, I might try making a table out of it, but I am not
> yet sure how I would set that up.  Something about having to say a lot
> in each cell, but with only a small amount of room in which to say it.


      |  mb | wmb | rmb | rbd | acq | rel |
 -----+-----+-----+-----+-----+-----+-----+
   mb |  X  |  X  |  X  |  X  |  X  |  X  |
 -----+-----+-----+-----+-----+-----+-----+
  wmb |  X  |     |  X  |  X  |     |     |
 -----+-----+-----+-----+-----+-----+-----+
  rmb |  X  |  X  |     |     |     |     |
 -----+-----+-----+-----+-----+-----+-----+
  rbd |  X  |  X  |     |     |     |     |
 -----+-----+-----+-----+-----+-----+-----+
  acq |  X  |     |     |     |     |  X  |
 -----+-----+-----+-----+-----+-----+-----+
  rel |  X  |     |     |     |  X  |     |
 -----+-----+-----+-----+-----+-----+-----+

(where rbd is read_barrier_depends).

Which is not entirely filled out, in particular I didn't do the creative
acq/rel bits.

> > Ah, I was more thinking of the fact that ACQUIRE/RELEASE are
> > semi-permeable while READ/WRITE are memop dependent.
> > 
> > So any combination will be a semi-permeable memop dependent thing,
> > which is the most narrow barrier possible.
> > 
> > So if we thing of ACQUIRE/RELEASE as being 'half' a full barrier,
> > separated in direction, and READ/WRITE as being 'half' a full barrier
> > separated on type, then the combination is a 'quarter' barrier.
> > 
> > Not arguing they're not useful, just saying we need to be extra careful.
> 
> I do agree completely about the need for extra care!
> 
> For whatever it is worth, the permeability and read-write properties
> are isolated to each barrier in the pair.  For example, with "a" and
> "b" both initially zero:
> 
> 	CPU 1				CPU 2
> 	-----				-----
> 	ACCESS_ONCE(a) = 1;		r1 = b;
> 	smp_store_release(&b, 1);	smp_rmb();
> 	ACCESS_ONCE(c) = 1;		r2 = a;
> 					ACCESS_ONCE(c) = 2;
> 
> The outcome r1==1&&r2==0 is prohibited, but the ordering of the stores
> to "c" are not ordered: CPU 1's smp_store_release() does not affect
> later accesses, and CPU 2's smp_rmb() does not order stores.
> 
> Not sure that it is worth adding this sort of example, though.

Yeah, not sure either. Maybe just a big fat caution if you pair acq/rel
with anything other than its opposite or a general barrier.

Maybe use small 'x' for acq/rel + rmb/wmb and put a caution in the
'legend' for 'x'.

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Paul E. McKenney]

On Wed, Jul 16, 2014 at 03:05:34PM +0200, Peter Zijlstra wrote:
> On Wed, Jul 16, 2014 at 05:16:26AM -0700, Paul E. McKenney wrote:
> > On Mon, Jul 14, 2014 at 01:57:38PM +0200, Peter Zijlstra wrote:
> > > On Tue, Jul 08, 2014 at 08:31:17AM -0700, Paul E. McKenney wrote:
> > > 
> > > > Good point, how about the following?
> > > > 
> > > > 	General barriers pair with each other, though they also pair
> > > > 	with most other types of barriers, albeit without transitivity.
> > > 
> > > > 	An acquire barrier pairs with a release barrier, but both may also
> > > > 	pair with other barriers, including of course general barriers.
> > > 
> > > > 	A write barrier pairs with a data dependency barrier, an acquire
> > > > 	barrier, a release barrier, a read barrier, or a general barrier.
> > > 
> > > > 	Similarly a read barrier or a data dependency barrier pairs
> > > > 	with a write barrier, an acquire barrier, a release barrier,
> > > > 	or a general barrier:
> > > 
> > > It might be clearer with the added whitespace, or as an explicit list  I
> > > suppose, but yes.
> > 
> > If I get ambitious, I might try making a table out of it, but I am not
> > yet sure how I would set that up.  Something about having to say a lot
> > in each cell, but with only a small amount of room in which to say it.
> 
> 
>       |  mb | wmb | rmb | rbd | acq | rel |
>  -----+-----+-----+-----+-----+-----+-----+
>    mb |  X  |  X  |  X  |  X  |  X  |  X  |
>  -----+-----+-----+-----+-----+-----+-----+
>   wmb |  X  |     |  X  |  X  |     |     |
>  -----+-----+-----+-----+-----+-----+-----+
>   rmb |  X  |  X  |     |     |     |     |
>  -----+-----+-----+-----+-----+-----+-----+
>   rbd |  X  |  X  |     |     |     |     |
>  -----+-----+-----+-----+-----+-----+-----+
>   acq |  X  |     |     |     |     |  X  |
>  -----+-----+-----+-----+-----+-----+-----+
>   rel |  X  |     |     |     |  X  |     |
>  -----+-----+-----+-----+-----+-----+-----+
> 
> (where rbd is read_barrier_depends).
> 
> Which is not entirely filled out, in particular I didn't do the creative
> acq/rel bits.

Also needs to reflect that wmb really can pair with wmb.  See for example
Scenario 15 of https://lwn.net/Articles/573436/.  Then again, your point
might be that Scenario 15 is not all that useful, which is true in my
experience -- though a good way to cause someone to find a use is to leave
it out of such a table...

> > > Ah, I was more thinking of the fact that ACQUIRE/RELEASE are
> > > semi-permeable while READ/WRITE are memop dependent.
> > > 
> > > So any combination will be a semi-permeable memop dependent thing,
> > > which is the most narrow barrier possible.
> > > 
> > > So if we thing of ACQUIRE/RELEASE as being 'half' a full barrier,
> > > separated in direction, and READ/WRITE as being 'half' a full barrier
> > > separated on type, then the combination is a 'quarter' barrier.
> > > 
> > > Not arguing they're not useful, just saying we need to be extra careful.
> > 
> > I do agree completely about the need for extra care!
> > 
> > For whatever it is worth, the permeability and read-write properties
> > are isolated to each barrier in the pair.  For example, with "a" and
> > "b" both initially zero:
> > 
> > 	CPU 1				CPU 2
> > 	-----				-----
> > 	ACCESS_ONCE(a) = 1;		r1 = b;
> > 	smp_store_release(&b, 1);	smp_rmb();
> > 	ACCESS_ONCE(c) = 1;		r2 = a;
> > 					ACCESS_ONCE(c) = 2;
> > 
> > The outcome r1==1&&r2==0 is prohibited, but the ordering of the stores
> > to "c" are not ordered: CPU 1's smp_store_release() does not affect
> > later accesses, and CPU 2's smp_rmb() does not order stores.
> > 
> > Not sure that it is worth adding this sort of example, though.
> 
> Yeah, not sure either. Maybe just a big fat caution if you pair acq/rel
> with anything other than its opposite or a general barrier.
> 
> Maybe use small 'x' for acq/rel + rmb/wmb and put a caution in the
> 'legend' for 'x'.

When I expand things out, I end up wanting each cell to hold a
scenario from the tables in https://lwn.net/Articles/573436/ and
https://lwn.net/Articles/573497/, probably along with some cautions
on what it does not do as well.  :-(

							Thanx, Paul

Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules

[Mathieu Desnoyers]

----- Original Message -----
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> To: "Peter Zijlstra" <peterz@infradead.org>
> Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, laijs@cn.fujitsu.com, dipankar@in.ibm.com,
> akpm@linux-foundation.org, "mathieu desnoyers" <mathieu.desnoyers@efficios.com>, josh@joshtriplett.org,
> niv@us.ibm.com, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
> dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, sbw@mit.edu
> Sent: Wednesday, July 16, 2014 9:18:55 AM
> Subject: Re: [PATCH tip/core/rcu 3/4] documentation: Add acquire/release barriers to pairing rules
> 
> On Wed, Jul 16, 2014 at 03:05:34PM +0200, Peter Zijlstra wrote:
> > On Wed, Jul 16, 2014 at 05:16:26AM -0700, Paul E. McKenney wrote:
> > > On Mon, Jul 14, 2014 at 01:57:38PM +0200, Peter Zijlstra wrote:
> > > > On Tue, Jul 08, 2014 at 08:31:17AM -0700, Paul E. McKenney wrote:
> > > > 
> > > > > Good point, how about the following?
> > > > > 
> > > > > 	General barriers pair with each other, though they also pair
> > > > > 	with most other types of barriers, albeit without transitivity.
> > > > 
> > > > > 	An acquire barrier pairs with a release barrier, but both may also
> > > > > 	pair with other barriers, including of course general barriers.
> > > > 
> > > > > 	A write barrier pairs with a data dependency barrier, an acquire
> > > > > 	barrier, a release barrier, a read barrier, or a general barrier.
> > > > 
> > > > > 	Similarly a read barrier or a data dependency barrier pairs
> > > > > 	with a write barrier, an acquire barrier, a release barrier,
> > > > > 	or a general barrier:
> > > > 
> > > > It might be clearer with the added whitespace, or as an explicit list
> > > > I
> > > > suppose, but yes.
> > > 
> > > If I get ambitious, I might try making a table out of it, but I am not
> > > yet sure how I would set that up.  Something about having to say a lot
> > > in each cell, but with only a small amount of room in which to say it.
> > 
> > 
> >       |  mb | wmb | rmb | rbd | acq | rel |
> >  -----+-----+-----+-----+-----+-----+-----+
> >    mb |  X  |  X  |  X  |  X  |  X  |  X  |
> >  -----+-----+-----+-----+-----+-----+-----+
> >   wmb |  X  |     |  X  |  X  |     |     |
> >  -----+-----+-----+-----+-----+-----+-----+
> >   rmb |  X  |  X  |     |     |     |     |
> >  -----+-----+-----+-----+-----+-----+-----+
> >   rbd |  X  |  X  |     |     |     |     |
> >  -----+-----+-----+-----+-----+-----+-----+
> >   acq |  X  |     |     |     |     |  X  |
> >  -----+-----+-----+-----+-----+-----+-----+
> >   rel |  X  |     |     |     |  X  |     |
> >  -----+-----+-----+-----+-----+-----+-----+
> > 
> > (where rbd is read_barrier_depends).
> > 
> > Which is not entirely filled out, in particular I didn't do the creative
> > acq/rel bits.
> 
> Also needs to reflect that wmb really can pair with wmb.  See for example
> Scenario 15 of https://lwn.net/Articles/573436/.  Then again, your point
> might be that Scenario 15 is not all that useful, which is true in my
> experience -- though a good way to cause someone to find a use is to leave
> it out of such a table...
> 
> > > > Ah, I was more thinking of the fact that ACQUIRE/RELEASE are
> > > > semi-permeable while READ/WRITE are memop dependent.
> > > > 
> > > > So any combination will be a semi-permeable memop dependent thing,
> > > > which is the most narrow barrier possible.
> > > > 
> > > > So if we thing of ACQUIRE/RELEASE as being 'half' a full barrier,
> > > > separated in direction, and READ/WRITE as being 'half' a full barrier
> > > > separated on type, then the combination is a 'quarter' barrier.
> > > > 
> > > > Not arguing they're not useful, just saying we need to be extra
> > > > careful.
> > > 
> > > I do agree completely about the need for extra care!
> > > 
> > > For whatever it is worth, the permeability and read-write properties
> > > are isolated to each barrier in the pair.  For example, with "a" and
> > > "b" both initially zero:
> > > 
> > > 	CPU 1				CPU 2
> > > 	-----				-----
> > > 	ACCESS_ONCE(a) = 1;		r1 = b;
> > > 	smp_store_release(&b, 1);	smp_rmb();
> > > 	ACCESS_ONCE(c) = 1;		r2 = a;
> > > 					ACCESS_ONCE(c) = 2;
> > > 
> > > The outcome r1==1&&r2==0 is prohibited, but the ordering of the stores
> > > to "c" are not ordered: CPU 1's smp_store_release() does not affect
> > > later accesses, and CPU 2's smp_rmb() does not order stores.
> > > 
> > > Not sure that it is worth adding this sort of example, though.
> > 
> > Yeah, not sure either. Maybe just a big fat caution if you pair acq/rel
> > with anything other than its opposite or a general barrier.
> > 
> > Maybe use small 'x' for acq/rel + rmb/wmb and put a caution in the
> > 'legend' for 'x'.
> 
> When I expand things out, I end up wanting each cell to hold a
> scenario from the tables in https://lwn.net/Articles/573436/ and
> https://lwn.net/Articles/573497/, probably along with some cautions
> on what it does not do as well.  :-(

The table you just proposed seems to fit well as a "memory barrier
cheat sheet" to remind the usual pairing that can be explained in
a single sentence each.

Then you could point out that this table is indeed incomplete, and
point to a separate URL for detailed explanation of the more elaborate,
seldom used pairing.

Thoughts ?

Thanks,

Mathieu

> 
> 							Thanx, Paul
> 
> 

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com