Re: [RFC PATCH 0/2] x86: Fix missing core serialization on migration

From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andy Lutomirski <luto@kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	linux-api <linux-api@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Boqun Feng <boqun.feng@gmail.com>, Andrew Hunter <ahh@google.com>,
	maged michael <maged.michael@gmail.com>,
	Avi Kivity <avi@scylladb.com>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Dave Watson <davejwatson@fb.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
	Andrea Parri <parri.andrea@gmail.com>,
	"Russell King, ARM Linux" <linux@armlinux.org.uk>,
	Greg Hackmann <ghackmann@google.com>,
	Will Deacon <will.deacon@arm.com>, David Sehr <sehr@google.com>,
	x86 <x86@kernel.org>
Subject: Re: [RFC PATCH 0/2] x86: Fix missing core serialization on migration
Date: Fri, 10 Nov 2017 21:57:14 +0000 (UTC)	[thread overview]
Message-ID: <885227610.13045.1510351034488.JavaMail.zimbra@efficios.com> (raw)
In-Reply-To: <CA+55aFzbroWqi+FTdYhRVSwUZ-M0wDVxjXqDbh40JEnXc2LdgQ@mail.gmail.com>

----- On Nov 10, 2017, at 4:36 PM, Linus Torvalds torvalds@linux-foundation.org wrote:

> On Fri, Nov 10, 2017 at 1:12 PM, Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
>> x86 can return to user-space through sysexit and sysretq, which are not
>> core serializing. This breaks expectations from user-space about
>> sequential consistency from a single-threaded self-modifying program
>> point of view in specific migration patterns.
>>
>> Feedback is welcome,
> 
> We should check with Intel. I would actually be surprised if the I$
> can be out of sync with the D$ after a sysretq.  It would actually
> break things like "read code from disk" too in theory.

That core serializing instruction is not that much about I$ vs D$
consistency, but rather about the processor speculatively executing code
ahead of its retirement point. Ref. Intel Architecture Software Developer's
Manual, Volume 3: System Programming.

7.1.3. "Handling Self- and Cross-Modifying Code":

"The act of a processor writing data into a currently executing code segment with the intent of
executing that data as code is called self-modifying code. Intel Architecture processors exhibit
model-specific behavior when executing self-modified code, depending upon how far ahead of
the current execution pointer the code has been modified. As processor architectures become
more complex and start to speculatively execute code ahead of the retirement point (as in the P6
family processors), the rules regarding which code should execute, pre- or post-modification,
become blurred. [...]"

AFAIU, this core serializing instruction seems to be needed for use-cases of
self-modifying code, but not for the initial load of a program from disk,
as the processor has no way to have speculatively executed any of its
instructions.

Hopefully hpa can tell us more about this,

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com