From: "Paul E. McKenney" <paulmck@kernel.org>
To: Alan Stern <stern@rowland.harvard.edu>
Cc: Manfred Spraul <manfred@colorfullife.com>,
linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org,
kernel-team@fb.com, mingo@kernel.org, parri.andrea@gmail.com,
will@kernel.org, peterz@infradead.org, boqun.feng@gmail.com,
npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk,
luc.maranget@inria.fr, akiyks@gmail.com
Subject: Re: [PATCH memory-model 2/4] tools/memory-model: Add example for heuristic lockless reads
Date: Fri, 23 Jul 2021 13:32:48 -0700 [thread overview]
Message-ID: <20210723203248.GL4397@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <20210723170820.GB46562@rowland.harvard.edu>
On Fri, Jul 23, 2021 at 01:08:20PM -0400, Alan Stern wrote:
> On Fri, Jul 23, 2021 at 09:30:08AM -0700, Paul E. McKenney wrote:
> > How about like this?
> >
> > Thanx, Paul
>
> Generally a lot better, but still at least one issue.
>
> > ------------------------------------------------------------------------
> >
> > Lock-Protected Writes With Heuristic Lockless Reads
> > ---------------------------------------------------
> >
> > For another example, suppose that the code can normally make use of
> > a per-data-structure lock, but there are times when a global lock
> > is required. These times are indicated via a global flag. The code
> > might look as follows, and is based loosely on nf_conntrack_lock(),
> > nf_conntrack_all_lock(), and nf_conntrack_all_unlock():
> >
> > bool global_flag;
> > DEFINE_SPINLOCK(global_lock);
> > struct foo {
> > spinlock_t f_lock;
> > int f_data;
> > };
> >
> > /* All foo structures are in the following array. */
> > int nfoo;
> > struct foo *foo_array;
> >
> > void do_something_locked(struct foo *fp)
> > {
> > /* IMPORTANT: Heuristic plus spin_lock()! */
> > if (!data_race(global_flag)) {
> > spin_lock(&fp->f_lock);
> > if (!smp_load_acquire(&global_flag)) {
> > do_something(fp);
> > spin_unlock(&fp->f_lock);
> > return;
> > }
> > spin_unlock(&fp->f_lock);
> > }
> > spin_lock(&global_lock);
> > /* global_lock held, thus global flag cannot be set. */
> > spin_lock(&fp->f_lock);
> > spin_unlock(&global_lock);
> > /*
> > * global_flag might be set here, but begin_global()
> > * will wait for ->f_lock to be released.
> > */
> > do_something(fp);
> > spin_lock(&fp->f_lock);
>
> spin_unlock.
Good eyes, fixed.
> > }
> >
> > void begin_global(void)
> > {
> > int i;
> >
> > spin_lock(&global_lock);
> > WRITE_ONCE(global_flag, true);
> > for (i = 0; i < nfoo; i++) {
> > /*
> > * Wait for pre-existing local locks. One at
> > * a time to avoid lockdep limitations.
> > */
> > spin_lock(&fp->f_lock);
> > spin_unlock(&fp->f_lock);
> > }
> > }
> >
> > void end_global(void)
> > {
> > smp_store_release(&global_flag, false);
> > spin_unlock(&global_lock);
> > }
> >
> > All code paths leading from the do_something_locked() function's first
> > read from global_flag acquire a lock, so endless load fusing cannot
> > happen.
> >
> > If the value read from global_flag is true, then global_flag is
> > rechecked while holding ->f_lock, which, if global_flag is now false,
> > prevents begin_global() from completing. It is therefore safe to invoke
> > do_something().
> >
> > Otherwise, if either value read from global_flag is true, then after
> > global_lock is acquired global_flag must be false. The acquisition of
> > ->f_lock will prevent any call to begin_global() from returning, which
> > means that it is safe to release global_lock and invoke do_something().
> >
> > For this to work, only those foo structures in foo_array[] may be passed
> > to do_something_locked(). The reason for this is that the synchronization
> > with begin_global() relies on momentarily holding the lock of each and
> > every foo structure.
>
> This doesn't mention the reason for the acquire-release
> synchronization of global_flag. It's needed because work done between
> begin_global() and end_global() can affect a foo structure without
> holding its private f_lock member, and we want all such work to be
> visible to other threads when they call do_something_locked() later.
Like this added paragraph at the end?
The smp_load_acquire() and smp_store_release() are required
because changes to a foo structure between calls to begin_global()
and end_global() are carried out without holding that structure's
->f_lock. The smp_load_acquire() and smp_store_release()
ensure that the next invocation of do_something() from the call
to do_something_locked() that acquires that ->f_lock will see
those changes.
Thanx, Paul
next prev parent reply other threads:[~2021-07-23 20:32 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-21 21:07 [PATCH kcsan 0/8] KCSAN updates for v5.15 Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 1/8] kcsan: Improve some Kconfig comments Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 2/8] kcsan: Remove CONFIG_KCSAN_DEBUG Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 3/8] kcsan: Introduce CONFIG_KCSAN_STRICT Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 4/8] kcsan: Reduce get_ctx() uses in kcsan_found_watchpoint() Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 5/8] kcsan: Rework atomic.h into permissive.h Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 6/8] kcsan: Print if strict or non-strict during init Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 7/8] kcsan: permissive: Ignore data-racy 1-bit value changes Paul E. McKenney
2021-07-21 21:08 ` [PATCH kcsan 8/8] kcsan: Make strict mode imply interruptible watchers Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 1/4] tools/memory-model: Make read_foo_diagnostic() more clearly diagnostic Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 2/4] tools/memory-model: Add example for heuristic lockless reads Paul E. McKenney
2021-07-23 2:08 ` Alan Stern
2021-07-23 6:52 ` Manfred Spraul
2021-07-23 13:05 ` Alan Stern
2021-07-23 13:57 ` Manfred Spraul
2021-07-23 16:30 ` Paul E. McKenney
2021-07-23 17:08 ` Alan Stern
2021-07-23 20:32 ` Paul E. McKenney [this message]
2021-07-23 21:03 ` Alan Stern
2021-07-23 22:29 ` Paul E. McKenney
2021-07-23 16:24 ` Paul E. McKenney
2021-07-23 16:59 ` Alan Stern
2021-07-23 17:30 ` Paul E. McKenney
2021-07-23 18:11 ` Alan Stern
2021-07-23 20:28 ` Paul E. McKenney
2021-07-28 17:40 ` [PATCH v2 " Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 3/4] tools/memory-model: Heuristics using data_race() must handle all values Paul E. McKenney
2021-07-21 21:10 ` [PATCH memory-model 4/4] tools/memory-model: Document data_race(READ_ONCE()) Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210723203248.GL4397@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=akiyks@gmail.com \
--cc=boqun.feng@gmail.com \
--cc=dhowells@redhat.com \
--cc=j.alglave@ucl.ac.uk \
--cc=kernel-team@fb.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luc.maranget@inria.fr \
--cc=manfred@colorfullife.com \
--cc=mingo@kernel.org \
--cc=npiggin@gmail.com \
--cc=parri.andrea@gmail.com \
--cc=peterz@infradead.org \
--cc=stern@rowland.harvard.edu \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).