* Re: [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence [not found] <1392672413-5114-2-git-send-email-paulmck () linux ! vnet ! ibm ! com> @ 2020-04-24 3:36 ` Jon Masters 2020-04-27 22:59 ` Paul E. McKenney 0 siblings, 1 reply; 5+ messages in thread From: Jon Masters @ 2020-04-24 3:36 UTC (permalink / raw) To: Paul E. McKenney, linux-kernel Hi Paul, On 2/17/14 4:26 PM, Paul E. McKenney wrote: > The ACCESS_ONCE() primitive provides cache coherence, but the > documentation does not clearly state this. This commit therefore upgrades > the documentation. <snip> > + In short, ACCESS_ONCE() provides "cache coherence" for accesses from > + multiple CPUs to a single variable. (ACCESS_ONCE is now READ_ONCE/WRITE_ONCE but the above added the original language around cache coherence) I would argue that we might want to avoid describing it in this manner. The hardware provides cache coherency in order to keep a single memory location coherent between multiple observers. These kernel macros only tell the compiler to perform the load once. They take advantage of the properties of coherence in the presence of multiple observers. Jon. -- Computer Architect ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence 2020-04-24 3:36 ` [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence Jon Masters @ 2020-04-27 22:59 ` Paul E. McKenney 0 siblings, 0 replies; 5+ messages in thread From: Paul E. McKenney @ 2020-04-27 22:59 UTC (permalink / raw) To: Jon Masters; +Cc: linux-kernel On Thu, Apr 23, 2020 at 11:36:19PM -0400, Jon Masters wrote: > Hi Paul, > > On 2/17/14 4:26 PM, Paul E. McKenney wrote: > > > The ACCESS_ONCE() primitive provides cache coherence, but the > > documentation does not clearly state this. This commit therefore upgrades > > the documentation. > > <snip> > > > + In short, ACCESS_ONCE() provides "cache coherence" for accesses from > > + multiple CPUs to a single variable. > > (ACCESS_ONCE is now READ_ONCE/WRITE_ONCE but the above added the original > language around cache coherence) > > I would argue that we might want to avoid describing it in this manner. The > hardware provides cache coherency in order to keep a single memory location > coherent between multiple observers. These kernel macros only tell the > compiler to perform the load once. They take advantage of the properties of > coherence in the presence of multiple observers. You lost me on this one. Are you advocating that this be described as constraining the compiler from invalidating the cache coherency (single-variable sequential consistency) provided by modern hardware? Just for background, my view is that "cache coherence", like "real time", is a property of the system that can be destroyed by any component. Thanx, Paul ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH tip/core/rcu 0/6] Documentation changes for 3.15 @ 2014-02-17 21:26 Paul E. McKenney 2014-02-17 21:26 ` [PATCH tip/core/rcu 1/6] documentation: Document call_rcu() safety mechanisms and limitations Paul E. McKenney 0 siblings, 1 reply; 5+ messages in thread From: Paul E. McKenney @ 2014-02-17 21:26 UTC (permalink / raw) To: linux-kernel Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, niv, tglx, peterz, rostedt, dhowells, edumazet, darren, fweisbec, oleg, sbw Hello! This series provides a variety of documentation updates: 1. Update the documentation on actions to take to avoid frenetic call_rcu() invocations from exhausting memory, including a mention of internal-to-RCU avoidance measures. 2. Add a note to Documentation/memory-barriers.txt stating that ACCESS_ONCE() provides cache coherence for accesses to any single given variable. 3. Add an explicit statement that in control dependencies, the condition must include the load in question. 4. Add WQ_SYSFS workqueues as a work-offloading option in Documentation/kernel-per-CPU-kthreads.txt. 5. It turns out that some types of control dependencies require memory barriers, most notably when the exact same store is at the beginning of both the then-clause and the else-clause. Document this. 6. Fixups to RTFP.txt Thanx, Paul ------------------------------------------------------------------------ Documentation/memory-barriers.txt | 29 +++-- b/Documentation/RCU/RTFP.txt | 149 +++++++++++++++++++++++----- b/Documentation/RCU/checklist.txt | 19 ++- b/Documentation/kernel-per-CPU-kthreads.txt | 13 ++ b/Documentation/memory-barriers.txt | 17 +++ 5 files changed, 189 insertions(+), 38 deletions(-) ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH tip/core/rcu 1/6] documentation: Document call_rcu() safety mechanisms and limitations 2014-02-17 21:26 [PATCH tip/core/rcu 0/6] Documentation changes for 3.15 Paul E. McKenney @ 2014-02-17 21:26 ` Paul E. McKenney 2014-02-17 21:26 ` [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence Paul E. McKenney 0 siblings, 1 reply; 5+ messages in thread From: Paul E. McKenney @ 2014-02-17 21:26 UTC (permalink / raw) To: linux-kernel Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, niv, tglx, peterz, rostedt, dhowells, edumazet, darren, fweisbec, oleg, sbw, Paul E. McKenney From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> The call_rcu() family of primitives will take action to accelerate grace periods when the number of callbacks pending on a given CPU becomes excessive. Although this safety mechanism can be useful, it is no substitute for users of call_rcu() having rate-limit controls in place. This commit adds this nuance to the documentation. Reported-by: "Michael S. Tsirkin" <mst@redhat.com> Reported-by: Gleb Natapov <gleb@redhat.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> --- Documentation/RCU/checklist.txt | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt index 91266193b8f4..5733e31836b5 100644 --- a/Documentation/RCU/checklist.txt +++ b/Documentation/RCU/checklist.txt @@ -256,10 +256,11 @@ over a rather long period of time, but improvements are always welcome! variations on this theme. b. Limiting update rate. For example, if updates occur only - once per hour, then no explicit rate limiting is required, - unless your system is already badly broken. The dcache - subsystem takes this approach -- updates are guarded - by a global lock, limiting their rate. + once per hour, then no explicit rate limiting is + required, unless your system is already badly broken. + Older versions of the dcache subsystem takes this + approach -- updates were guarded by a global lock, + limiting their rate. c. Trusted update -- if updates can only be done manually by superuser or some other trusted user, then it might not @@ -268,7 +269,8 @@ over a rather long period of time, but improvements are always welcome! the machine. d. Use call_rcu_bh() rather than call_rcu(), in order to take - advantage of call_rcu_bh()'s faster grace periods. + advantage of call_rcu_bh()'s faster grace periods. (This + is only a partial solution, though.) e. Periodically invoke synchronize_rcu(), permitting a limited number of updates per grace period. @@ -276,6 +278,13 @@ over a rather long period of time, but improvements are always welcome! The same cautions apply to call_rcu_bh(), call_rcu_sched(), call_srcu(), and kfree_rcu(). + Note that although these primitives do take action to avoid memory + exhaustion when any given CPU has too many callbacks, a determined + user could still exhaust memory. This is especially the case + if a system with a large number of CPUs has been configured to + offload all of its RCU callbacks onto a single CPU, or if the + system has relatively little free memory. + 9. All RCU list-traversal primitives, which include rcu_dereference(), list_for_each_entry_rcu(), and list_for_each_safe_rcu(), must be either within an RCU read-side -- 1.8.1.5 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence 2014-02-17 21:26 ` [PATCH tip/core/rcu 1/6] documentation: Document call_rcu() safety mechanisms and limitations Paul E. McKenney @ 2014-02-17 21:26 ` Paul E. McKenney 2014-02-17 21:40 ` Josh Triplett 0 siblings, 1 reply; 5+ messages in thread From: Paul E. McKenney @ 2014-02-17 21:26 UTC (permalink / raw) To: linux-kernel Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, niv, tglx, peterz, rostedt, dhowells, edumazet, darren, fweisbec, oleg, sbw, Paul E. McKenney From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> The ACCESS_ONCE() primitive provides cache coherence, but the documentation does not clearly state this. This commit therefore upgrades the documentation. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> --- Documentation/memory-barriers.txt | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 102dc19c4119..ad6db1d48f1f 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -1249,6 +1249,23 @@ The ACCESS_ONCE() function can prevent any number of optimizations that, while perfectly safe in single-threaded code, can be fatal in concurrent code. Here are some examples of these sorts of optimizations: + (*) The compiler is within its rights to reorder loads and stores + to the same variable, and in some cases, the CPU is within its + rights to reorder loads to the same variable. This means that + the following code: + + a[0] = x; + a[1] = x; + + Might result in an older value of x stored in a[1] than in a[0]. + Prevent both the compiler and the CPU from doing this as follows: + + a[0] = ACCESS_ONCE(x); + a[1] = ACCESS_ONCE(x); + + In short, ACCESS_ONCE() provides "cache coherence" for accesses from + multiple CPUs to a single variable. + (*) The compiler is within its rights to merge successive loads from the same variable. Such merging can cause the compiler to "optimize" the following code: -- 1.8.1.5 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence 2014-02-17 21:26 ` [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence Paul E. McKenney @ 2014-02-17 21:40 ` Josh Triplett 2014-02-17 22:52 ` Paul E. McKenney 0 siblings, 1 reply; 5+ messages in thread From: Josh Triplett @ 2014-02-17 21:40 UTC (permalink / raw) To: Paul E. McKenney Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, niv, tglx, peterz, rostedt, dhowells, edumazet, darren, fweisbec, oleg, sbw On Mon, Feb 17, 2014 at 01:26:49PM -0800, Paul E. McKenney wrote: > From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > The ACCESS_ONCE() primitive provides cache coherence, but the > documentation does not clearly state this. This commit therefore upgrades > the documentation. > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Punctuation nit below; otherwise: Reviewed-by: Josh Triplett <josh@joshtriplett.org> > Documentation/memory-barriers.txt | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt > index 102dc19c4119..ad6db1d48f1f 100644 > --- a/Documentation/memory-barriers.txt > +++ b/Documentation/memory-barriers.txt > @@ -1249,6 +1249,23 @@ The ACCESS_ONCE() function can prevent any number of optimizations that, > while perfectly safe in single-threaded code, can be fatal in concurrent > code. Here are some examples of these sorts of optimizations: > > + (*) The compiler is within its rights to reorder loads and stores > + to the same variable, and in some cases, the CPU is within its > + rights to reorder loads to the same variable. This means that > + the following code: > + > + a[0] = x; > + a[1] = x; > + > + Might result in an older value of x stored in a[1] than in a[0]. > + Prevent both the compiler and the CPU from doing this as follows: > + > + a[0] = ACCESS_ONCE(x); > + a[1] = ACCESS_ONCE(x); > + > + In short, ACCESS_ONCE() provides "cache coherence" for accesses from > + multiple CPUs to a single variable. You don't need to "quote" the well-established term "cache coherence". > (*) The compiler is within its rights to merge successive loads from > the same variable. Such merging can cause the compiler to "optimize" > the following code: > -- > 1.8.1.5 > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence 2014-02-17 21:40 ` Josh Triplett @ 2014-02-17 22:52 ` Paul E. McKenney 0 siblings, 0 replies; 5+ messages in thread From: Paul E. McKenney @ 2014-02-17 22:52 UTC (permalink / raw) To: Josh Triplett Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, niv, tglx, peterz, rostedt, dhowells, edumazet, darren, fweisbec, oleg, sbw On Mon, Feb 17, 2014 at 01:40:56PM -0800, Josh Triplett wrote: > On Mon, Feb 17, 2014 at 01:26:49PM -0800, Paul E. McKenney wrote: > > From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > > > The ACCESS_ONCE() primitive provides cache coherence, but the > > documentation does not clearly state this. This commit therefore upgrades > > the documentation. > > > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> > > Punctuation nit below; otherwise: > Reviewed-by: Josh Triplett <josh@joshtriplett.org> > > > Documentation/memory-barriers.txt | 17 +++++++++++++++++ > > 1 file changed, 17 insertions(+) > > > > diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt > > index 102dc19c4119..ad6db1d48f1f 100644 > > --- a/Documentation/memory-barriers.txt > > +++ b/Documentation/memory-barriers.txt > > @@ -1249,6 +1249,23 @@ The ACCESS_ONCE() function can prevent any number of optimizations that, > > while perfectly safe in single-threaded code, can be fatal in concurrent > > code. Here are some examples of these sorts of optimizations: > > > > + (*) The compiler is within its rights to reorder loads and stores > > + to the same variable, and in some cases, the CPU is within its > > + rights to reorder loads to the same variable. This means that > > + the following code: > > + > > + a[0] = x; > > + a[1] = x; > > + > > + Might result in an older value of x stored in a[1] than in a[0]. > > + Prevent both the compiler and the CPU from doing this as follows: > > + > > + a[0] = ACCESS_ONCE(x); > > + a[1] = ACCESS_ONCE(x); > > + > > + In short, ACCESS_ONCE() provides "cache coherence" for accesses from > > + multiple CPUs to a single variable. > > You don't need to "quote" the well-established term "cache coherence". Good point, fixed and applied your Reviewed-by, thank you! Thanx, Paul > > (*) The compiler is within its rights to merge successive loads from > > the same variable. Such merging can cause the compiler to "optimize" > > the following code: > > -- > > 1.8.1.5 > > > ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-04-27 22:59 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <1392672413-5114-2-git-send-email-paulmck () linux ! vnet ! ibm ! com> 2020-04-24 3:36 ` [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence Jon Masters 2020-04-27 22:59 ` Paul E. McKenney 2014-02-17 21:26 [PATCH tip/core/rcu 0/6] Documentation changes for 3.15 Paul E. McKenney 2014-02-17 21:26 ` [PATCH tip/core/rcu 1/6] documentation: Document call_rcu() safety mechanisms and limitations Paul E. McKenney 2014-02-17 21:26 ` [PATCH tip/core/rcu 2/6] Documentation/memory-barriers.txt: ACCESS_ONCE() provides cache coherence Paul E. McKenney 2014-02-17 21:40 ` Josh Triplett 2014-02-17 22:52 ` Paul E. McKenney
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.