Re: Redoing eXclusive Page Frame Ownership (XPFO) with isolated CPUs in mind (for KVM to isolate its guests per CPU)

From: Julian Stecklina <jsteckli@amazon.de>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: David Woodhouse <dwmw@amazon.co.uk>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	juerg.haefliger@hpe.com, deepa.srinivasan@oracle.com,
	Jim Mattson <jmattson@google.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	linux-mm <linux-mm@kvack.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	joao.m.martins@oracle.com, pradeep.vincent@oracle.com,
	Andi Kleen <ak@linux.intel.com>,
	Khalid Aziz <khalid.aziz@oracle.com>,
	kanth.ghatraju@oracle.com, Liran Alon <liran.alon@oracle.com>,
	Kees Cook <keescook@google.com>,
	Kernel Hardening <kernel-hardening@lists.openwall.com>,
	chris.hyser@oracle.com, Tyler Hicks <tyhicks@canonical.com>,
	John Haxby <john.haxby@oracle.com>, Jon Masters <jcm@redhat.com>
Subject: Re: Redoing eXclusive Page Frame Ownership (XPFO) with isolated CPUs in mind (for KVM to isolate its guests per CPU)
Date: Thu, 30 Aug 2018 18:00:51 +0200	[thread overview]
Message-ID: <ciirm8a7p3alos.fsf@u54ee758033e858cfa736.ant.amazon.com> (raw)
In-Reply-To: <CA+55aFxyUdhYjnQdnmWAt8tTwn4HQ1xz3SAMZJiawkLpMiJ_+w@mail.gmail.com>

Hey everyone,

On Mon, 20 Aug 2018 15:27 Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Mon, Aug 20, 2018 at 3:02 PM Woodhouse, David <dwmw@amazon.co.uk> wrote:
>>
>> It's the *kernel* we don't want being able to access those pages,
>> because of the multitude of unfixable cache load gadgets.
>
> Ahh.
> 
> I guess the proof is in the pudding. Did somebody try to forward-port
> that patch set and see what the performance is like?

I've been spending some cycles on the XPFO patch set this week. For the
patch set as it was posted for v4.13, the performance overhead of
compiling a Linux kernel is ~40% on x86_64[1]. The overhead comes almost
completely from TLB flushing. If we can live with stale TLB entries
allowing temporary access (which I think is reasonable), we can remove
all TLB flushing (on x86). This reduces the overhead to 2-3% for
kernel compile.

There were no problems in forward-porting the patch set to master.
You can find the result here, including a patch makes the TLB flushing
configurable:
http://git.infradead.org/users/jsteckli/linux-xpfo.git/shortlog/refs/heads/xpfo-master

It survived some casual stress-ng runs. I can rerun the benchmarks on
this version, but I doubt there is any change.

> It used to be just 500 LOC. Was that because they took horrible
> shortcuts?

The patch is still fairly small. As for the horrible shortcuts, I let
others comment on that.

HTH,
Julian

[1] Measured on my quad-core (8 hyperthreads) Kaby Lake desktop building
Linux 4.18 with the Phoronix Test Suite.

--
Amazon Development Center Germany GmbH
Berlin - Dresden - Aachen
main office: Krausenstr. 38, 10117 Berlin
Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
Ust-ID: DE289237879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B