RE: [for-6.0 v5 11/13] spapr: PEF: prevent migration

From: Ram Pai <linuxram@us.ibm.com>
To: Halil Pasic <pasic@linux.ibm.com>
Cc: Cornelia Huck <cohuck@redhat.com>, Greg Kurz <groug@kaod.org>,
	pair@us.ibm.com, brijesh.singh@amd.com, kvm@vger.kernel.org,
	"Michael S. Tsirkin" <mst@redhat.com>,
	qemu-devel@nongnu.org, frankja@linux.ibm.com, david@redhat.com,
	mdroth@linux.vnet.ibm.com, borntraeger@de.ibm.com,
	David Gibson <david@gibson.dropbear.id.au>,
	thuth@redhat.com, Eduardo Habkost <ehabkost@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	dgilbert@redhat.com, qemu-s390x@nongnu.org, rth@twiddle.net,
	berrange@redhat.com, Marcelo Tosatti <mtosatti@redhat.com>,
	qemu-ppc@nongnu.org, pbonzini@redhat.com
Subject: RE: [for-6.0 v5 11/13] spapr: PEF: prevent migration
Date: Mon, 4 Jan 2021 10:40:26 -0800	[thread overview]
Message-ID: <20210104184026.GD4102@ram-ibm-com.ibm.com> (raw)
In-Reply-To: <20210104134629.49997b53.pasic@linux.ibm.com>

On Mon, Jan 04, 2021 at 01:46:29PM +0100, Halil Pasic wrote:
> On Sun, 3 Jan 2021 23:15:50 -0800
> Ram Pai <linuxram@us.ibm.com> wrote:
> 
> > On Fri, Dec 18, 2020 at 12:41:11PM +0100, Cornelia Huck wrote:
> > > On Thu, 17 Dec 2020 15:15:30 +0100
> [..]
> > > > > > > > +int kvmppc_svm_init(SecurableGuestMemory *sgm, Error **errp)
> > > > > > > >  {
> > > > > > > >      if (!kvm_check_extension(kvm_state, KVM_CAP_PPC_SECURABLE_GUEST)) {
> > > > > > > >          error_setg(errp,
> > > > > > > > @@ -54,6 +58,11 @@ static int kvmppc_svm_init(Error **errp)
> > > > > > > >          }
> > > > > > > >      }
> > > > > > > >  
> > > > > > > > +    /* add migration blocker */
> > > > > > > > +    error_setg(&pef_mig_blocker, "PEF: Migration is not implemented");
> > > > > > > > +    /* NB: This can fail if --only-migratable is used */
> > > > > > > > +    migrate_add_blocker(pef_mig_blocker, &error_fatal);      
> > > > > > > 
> > > > > > > Just so that I understand: is PEF something that is enabled by the host
> > > > > > > (and the guest is either secured or doesn't start), or is it using a
> > > > > > > model like s390x PV where the guest initiates the transition into
> > > > > > > secured mode?      
> > > > > > 
> > > > > > Like s390x PV it's initiated by the guest.
> > > > > >     
> > > > > > > Asking because s390x adds the migration blocker only when the
> > > > > > > transition is actually happening (i.e. guests that do not transition
> > > > > > > into secure mode remain migratable.) This has the side effect that you
> > > > > > > might be able to start a machine with --only-migratable that
> > > > > > > transitions into a non-migratable machine via a guest action, if I'm
> > > > > > > not mistaken. Without the new object, I don't see a way to block with
> > > > > > > --only-migratable; with it, we should be able to do that. Not sure what
> > > > > > > the desirable behaviour is here.      
> > > > > >     
> > > > 
> > > > The purpose of --only-migratable is specifically to prevent the machine
> > > > to transition to a non-migrate state IIUC. The guest transition to
> > > > secure mode should be nacked in this case.  
> > > 
> > > Yes, that's what happens for s390x: The guest tries to transition, QEMU
> > > can't add a migration blocker and fails the instruction used for
> > > transitioning, the guest sees the error.
> > > 
> > > The drawback is that we see the failure only when we already launched
> > > the machine and the guest tries to transition. If I start QEMU with
> > > --only-migratable, it will refuse to start when non-migratable devices
> > > are configured in the command line, so I see the issue right from the
> > > start. (For s390x, that would possibly mean that we should not even
> > > present the cpu feature bit when only_migratable is set?)  
> > 
> > What happens in s390x,  if the guest tries to transition to secure, when
> > the secure object is NOT configured on the machine?
> > 
> 
> Nothing in particular.
> 
> > On PEF systems, the transition fails and the guest is terminated.
> > 
> > My point is -- QEMU will not be able to predict in advance, what the
> > guest might or might not do, regardless of what devices and objects are
> > configured in the machine.   If the guest does something unexpected, it
> > has to be terminated.
> 
> We can't fail transition to secure when the secure object is not
> configured on the machine, because that would break pre-existing
> setups.

So the instruction to switch-to-secure; which I believe is a ultracall
on S390,  will return success even though the switch-to-secure has failed?
Will the guest continue as a normal guest or as a secure guest?

> This feature is still to be shipped, but secure execution has
> already been shipped, but without migration support.
> 
> That's why when you have both the secure object configured, and mandate
> migratability, the we can fail. Actually we should fail now, because the
> two options are not compatible: you can't have a qemu that is guaranteed
> to be migratable, and guaranteed to be able to operate in secure
> execution mode today. Failing early, and not on the guests opt-in would
> be preferable.
> 
> After migration support is added, the combo should be fine, and probably
> also the default for secure execution machines.
> 
> > 
> > So one possible design choice is to let the guest know that migration
> > must be facilitated. It can then decide if it wants to continue as a
> > normal VM or terminate itself, or take the plunge and switch to secure.
> > A well behaving guest will not switch to secure.
> > 
> 
> I don't understand this point. Sorry.

Qemu will present the 'must-support-migrate' and the 'secure-object' capability
to the guest.

The secure-aware guest, has three choices
   (a) terminate itself. OR
   (b) not call the switch-to-secure ucall, and continue as normal guest. OR
   (c) call the switch-to-secure ucall.

Legacy guests which are not aware of secure-object, will continue to do
(b).   New Guests which are secure-object aware, will observe that 
'must-support-migrate' and 'secure-object' capabilities are
incompatible.  Hence will choose (a) or (b), but will never choose
(c).

The main difference between my proposal and the other proposal is...

  In my proposal the guest makes the compatibility decision and acts
  accordingly.  In the other proposal QEMU makes the compatibility
  decision and acts accordingly. I argue that QEMU cannot make a good
  compatibility decision, because it wont know in advance, if the guest
  will or will-not switch-to-secure.

RP