Re: [RFC PATCH] LKMM: Add ctrl_dep() macro for control dependency

From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Will Deacon <will@kernel.org>, paulmck <paulmck@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Segher Boessenkool <segher@kernel.crashing.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Alan Stern <stern@rowland.harvard.edu>,
	Andrea Parri <parri.andrea@gmail.com>,
	Boqun Feng <boqun.feng@gmail.com>,
	Nicholas Piggin <npiggin@gmail.com>,
	David Howells <dhowells@redhat.com>,
	j alglave <j.alglave@ucl.ac.uk>,
	luc maranget <luc.maranget@inria.fr>, akiyks <akiyks@gmail.com>,
	linux-toolchains <linux-toolchains@vger.kernel.org>,
	linux-arch <linux-arch@vger.kernel.org>
Subject: Re: [RFC PATCH] LKMM: Add ctrl_dep() macro for control dependency
Date: Wed, 29 Sep 2021 15:27:06 -0400 (EDT)	[thread overview]
Message-ID: <1882826966.44389.1632943626923.JavaMail.zimbra@efficios.com> (raw)
In-Reply-To: <CAHk-=wg23CqjGWjjxDQ7yxrb+eF5at2KFU03GZa18Znx=+Xvow@mail.gmail.com>

----- On Sep 29, 2021, at 10:47 AM, Linus Torvalds torvalds@linux-foundation.org wrote:

> On Tue, Sep 28, 2021 at 2:15 PM Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
>>
>> Introduce the ctrl_dep macro in the generic headers, and use it
>> everywhere it appears relevant.
> 
> The control dependency is so subtle - just see our discussions - that
> I really think every single use of it needs to have a comment about
> why it's needed.

I agree with you on thorough documentation of each control dependency,
perhaps just not about documentation of all compiler optimizations
affecting each of them.

> 
> Right now, that patch seems to just sprinkle them more or less
> randomly. That's absolutely not what I want. It will just mean that
> other people start sprinkling them randomly even more, and nobody will
> dare remove them.

Note that I have not found that many uses of control dependencies in the
kernel tree. When they are used, this happens to be code where speed
really matters though.

> So I'd literally want a comment about "this needs a control
> dependency, because otherwise the compiler could merge the two
> identical stores X and Y".

My hope with this ctrl_dep() macro is to remove at least some of
the caveats to keep in mind when using control dependency ordering.
Requiring to keep track of all relevant compiler optimizations on all
architectures while reasoning about memory barriers is error-prone.

> When you have a READ_ONCE() in the conditional, and a WRITE_ONCE() in
> the statement protected by the conditional, there is *no* need to
> randomly sprinkle noise that doesn't matter.

The main advantage in doing so would be documentation, both in terms of
letting the compiler know that this control dependency matters for
ordering, and in terms of simplifying the set of caveats to document
in Documentation/memory-barriers.txt.

> And if there *is* need ("look, we have that same store in both the if-
> and the else-statement" or whatever), then say so, and state that
> thing.

If we go for only using ctrl_dep() for scenarios which require it for
documented reasons, then we would need to leave in place all the
caveats details in Documentation/memory-barriers.txt, and document
that in those scenarios ctrl_dep() should be used. This would be a
starting point I guess.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com