Re: [EXTERNAL] Re: [for-6.0 v5 11/13] spapr: PEF: prevent migration

[Halil Pasic <pasic@linux.ibm.com>]

On Mon, 4 Jan 2021 10:40:26 -0800
Ram Pai <linuxram@us.ibm.com> wrote:

> On Mon, Jan 04, 2021 at 01:46:29PM +0100, Halil Pasic wrote:
> > On Sun, 3 Jan 2021 23:15:50 -0800
> > Ram Pai <linuxram@us.ibm.com> wrote:
> >   
> > > On Fri, Dec 18, 2020 at 12:41:11PM +0100, Cornelia Huck wrote:  
> > > > On Thu, 17 Dec 2020 15:15:30 +0100  
> > [..]  
> > > > > > > > > +int kvmppc_svm_init(SecurableGuestMemory *sgm, Error **errp)
> > > > > > > > >  {
> > > > > > > > >      if (!kvm_check_extension(kvm_state, KVM_CAP_PPC_SECURABLE_GUEST)) {
> > > > > > > > >          error_setg(errp,
> > > > > > > > > @@ -54,6 +58,11 @@ static int kvmppc_svm_init(Error **errp)
> > > > > > > > >          }
> > > > > > > > >      }
> > > > > > > > >  
> > > > > > > > > +    /* add migration blocker */
> > > > > > > > > +    error_setg(&pef_mig_blocker, "PEF: Migration is not implemented");
> > > > > > > > > +    /* NB: This can fail if --only-migratable is used */
> > > > > > > > > +    migrate_add_blocker(pef_mig_blocker, &error_fatal);        
> > > > > > > > 
> > > > > > > > Just so that I understand: is PEF something that is enabled by the host
> > > > > > > > (and the guest is either secured or doesn't start), or is it using a
> > > > > > > > model like s390x PV where the guest initiates the transition into
> > > > > > > > secured mode?        
> > > > > > > 
> > > > > > > Like s390x PV it's initiated by the guest.
> > > > > > >       
> > > > > > > > Asking because s390x adds the migration blocker only when the
> > > > > > > > transition is actually happening (i.e. guests that do not transition
> > > > > > > > into secure mode remain migratable.) This has the side effect that you
> > > > > > > > might be able to start a machine with --only-migratable that
> > > > > > > > transitions into a non-migratable machine via a guest action, if I'm
> > > > > > > > not mistaken. Without the new object, I don't see a way to block with
> > > > > > > > --only-migratable; with it, we should be able to do that. Not sure what
> > > > > > > > the desirable behaviour is here.        
> > > > > > >       
> > > > > 
> > > > > The purpose of --only-migratable is specifically to prevent the machine
> > > > > to transition to a non-migrate state IIUC. The guest transition to
> > > > > secure mode should be nacked in this case.    
> > > > 
> > > > Yes, that's what happens for s390x: The guest tries to transition, QEMU
> > > > can't add a migration blocker and fails the instruction used for
> > > > transitioning, the guest sees the error.
> > > > 
> > > > The drawback is that we see the failure only when we already launched
> > > > the machine and the guest tries to transition. If I start QEMU with
> > > > --only-migratable, it will refuse to start when non-migratable devices
> > > > are configured in the command line, so I see the issue right from the
> > > > start. (For s390x, that would possibly mean that we should not even
> > > > present the cpu feature bit when only_migratable is set?)    
> > > 
> > > What happens in s390x,  if the guest tries to transition to secure, when
> > > the secure object is NOT configured on the machine?
> > >   
> > 
> > Nothing in particular.
> >   
> > > On PEF systems, the transition fails and the guest is terminated.
> > > 
> > > My point is -- QEMU will not be able to predict in advance, what the
> > > guest might or might not do, regardless of what devices and objects are
> > > configured in the machine.   If the guest does something unexpected, it
> > > has to be terminated.  
> > 
> > We can't fail transition to secure when the secure object is not
> > configured on the machine, because that would break pre-existing
> > setups.  
> 
> So the instruction to switch-to-secure; which I believe is a ultracall
> on S390,  

Yes it is an ultravisor call. 

> will return success even though the switch-to-secure has failed?

No, I don't think so.

> Will the guest continue as a normal guest or as a secure guest?
> 

I think the guest will give up. It definitely can't continue as secure
because the conversion to secure failed. And it should not continue as
non-secure because that's not what the user asked for.

I'm not sure you got my point. My point is: we may not break existing
setups when adding new features. Secure execution can work without secure
object today, and what works today shall keep working tomorrow and
beyond.

> > This feature is still to be shipped, but secure execution has
> > already been shipped, but without migration support.
> > 
> > That's why when you have both the secure object configured, and mandate
> > migratability, the we can fail. Actually we should fail now, because the
> > two options are not compatible: you can't have a qemu that is guaranteed
> > to be migratable, and guaranteed to be able to operate in secure
> > execution mode today. Failing early, and not on the guests opt-in would
> > be preferable.
> > 
> > After migration support is added, the combo should be fine, and probably
> > also the default for secure execution machines.
> >   
> > > 
> > > So one possible design choice is to let the guest know that migration
> > > must be facilitated. It can then decide if it wants to continue as a
> > > normal VM or terminate itself, or take the plunge and switch to secure.
> > > A well behaving guest will not switch to secure.
> > >   
> > 
> > I don't understand this point. Sorry.  
> 
> Qemu will present the 'must-support-migrate' and the 'secure-object' capability
> to the guest.

How does the qemu preset the 'must-support-migrate' and the
'secure-object' capability to the guest on (PPC and especially on s390)? And
please clarify what do you mean by 'secure-object'. I used to believe I
understood, but now I have the feeling I don't understand.

> 
> The secure-aware guest, has three choices
>    (a) terminate itself. OR
>    (b) not call the switch-to-secure ucall, and continue as normal guest. OR
>    (c) call the switch-to-secure ucall.
> 
> Legacy guests which are not aware of secure-object, will continue to do
> (b).   
> New Guests which are secure-object aware, will observe that 
> 'must-support-migrate' and 'secure-object' capabilities are
> incompatible.  Hence will choose (a) or (b), but will never choose
> (c).
> 

The first problem is, IMHO, that you want to expose QEMU internals to the
guest. For the guest, there is no such thing as 'must-support-migrate'
(AFAIK).

The other problem is, that migration and secure are not inherently
incompatible. On s390x it is the property of the current host
implementation, that we can't do migration for secure. But this can
change in the future. 

> 
> 
> The main difference between my proposal and the other proposal is...
> 
>   In my proposal the guest makes the compatibility decision and acts
>   accordingly.  In the other proposal QEMU makes the compatibility
>   decision and acts accordingly. I argue that QEMU cannot make a good
>   compatibility decision, because it wont know in advance, if the guest
>   will or will-not switch-to-secure.
> 

You have a point there when you say that QEMU does not know in advance,
if the guest will or will-not switch-to-secure. I made that argument
regarding VIRTIO_F_ACCESS_PLATFORM (iommu_platform) myself. My idea
was to flip that property on demand when the conversion occurs. David
explained to me that this is not possible for ppc, and that having the
"securable-guest-memory" property (or whatever the name will be)
specified is a strong indication, that the VM is intended to be used as
a secure VM (thus it is OK to hurt the case where the guest does not
try to transition). That argument applies here as well.

But more importantly, as I explained above, the guest does not know if
migration and secure are incompatible or not. So the guest can't make a
good decision.

Regards,
Halil