Re: [PATCH 3/5] KVM: PPC: Book3S HV: Handle memory slot deletion and modification correctly

From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
To: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Avi Kivity <avi@redhat.com>, Paul Mackerras <paulus@samba.org>,
	Gleb Natapov <gleb@redhat.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>,
	Alexander Graf <agraf@suse.de>,
	kvm-ppc@vger.kernel.org, kvm@vger.kernel.org
Subject: Re: [PATCH 3/5] KVM: PPC: Book3S HV: Handle memory slot deletion and modification correctly
Date: Fri, 17 Aug 2012 17:06:18 +1000	[thread overview]
Message-ID: <1345187178.11751.49.camel@pasglop> (raw)
In-Reply-To: <20120815175920.GB18452@amt.cnet>

On Wed, 2012-08-15 at 14:59 -0300, Marcelo Tosatti wrote:
> 
> The guest should not expect memory accesses to an address
> to behave sanely while changing a BAR anyway.
> 
> Yes, compatibility for change of GPA base can be done in the
> kernel. I can look into it next week if nobody has done so at
> that point. 

There's one thing to be extra careful about here is if we start
doing that for normal memory (in case we start breaking it up
in slots, such as NUMA setups etc...).

The problem is that we must not allow normal memory accesses to be
handled via the "emulation" code (ie MMIO emulation or load/store
emulation, whatever we call it).

Part of the issues is that on architectures that don't use IPIs for
TLB invalidations but instead use some form of HW broadcast such as
PowerPC or ARM, there is an inherent race in that the emulation code can
keep a guest physical address (and perform the relevant access to the
corresponding memory region) way beyond the point where the guest
virtual->physical translation leading to that address has been
invalidated.

This doesn't happen on x86 because essentially the completion of the
invalidation IPI has to wait for all VCPUs to "respond" and thus to
finish whatever emulation they are doing. This is not the case on archs
with a HW invalidate broadcast.

This is a nasty race, and while we more/less decided that it was
survivable as long as we only go through emulation for devices (as we
don't play swapping games with them in the guest kernel), the minute we
allow normal guest memory access to "slip through", we have broken the
guest virtual memory model.

So if we are manipulated memory slots used for guest RAM we must -not-
break atomicity, since during the time the slot is gone, it will
fallback to emulation, introducing the above race (at least on PowerPC
and ARM).

Cheers,
Ben.