[Qemu-devel] libvirt/QEMU/SEV interaction

[Qemu-devel] libvirt/QEMU/SEV interaction

[Brijesh Singh]

Hi All,

(sorry for the long message)

CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
feature - the feature allows running encrypted VMs. To enable the feature,
I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
We have been making some good progress in getting patches accepted upstream
in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
feature -- SME support just got pulled into 4.14 merge window. The base
SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
I am getting ready to take off "RFC" tag from remaining patches to get them
reviewed and accepted.

The boot flow for launching an SEV guest is a bit different from a typical
guest launch. In order to launch SEV guest from virt-manager or other
high-level VM management tools, we need to design and implement new
interface between libvirt and qemu, and probably add new APIs in libvirt
to be used by VM management tools. I am new to the libvirt and need some
expert advice while designing this interface. A pictorial representation
for a SEV guest launch flow is available in SEV Spec Appendix A [4].

A typical flow looks like this:

1. Guest owner (GO) asks the cloud provider to launch SEV guest.
2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
3. libvirt opens /dev/sev device to get its PDH and return the blob to the
   caller.
4. VM tool gives its PDH to GO.
5. GO provides its DH key, session-info and guest policy.
6. VM tool somehow communicates the GO provided information to libvirt.
7. libvirt adds "sev-guest" object in its xml file with all the information
   obtained from #5

   (currently my xml file looks like this)

   <qemu:arg value='-object'>
   <qemu:arg
value='sev-guest,id=sev0,policy=<GO_policy>,dh-key-file=<filename>,session-file=<filename>/>
   <qemu:arg value='-machine'/>
   <qemu:arg value='memory-encryption=sev0'/>

8. libvirt launches the guest with "-S"
9. While creating the SEV guest qemu does the following
 i) create encryption context using GO's DH, session-info and guest policy
    (LAUNCH_START)
 ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
 iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
10. By some interface we must propagate the measurement all the way to GO
  before libvirt starts the guest.
11. GO verifies the measurement and if measurement matches then it may
 give a secret blob -- which must be injected into the guest before
 libvirt starts the VM. If verification failed, GO will request cloud
 provider to destroy the VM.
12. After secret blob is injected into guest, we call LAUNCH_FINISH
  to destory the encryption context.
13. libvirt issues "continue" command to resume the guest boot.

Please note that the measurement value is protected with transport
encryption key (TIK) and it changes on each run. Similarly the secret blob
provided by GO does not need to be protected using libvirt/qemu APIs. The
secret is protected by TIK. From qemu and libvirt point of view these are
blobs and must be passed as-is to the SEV FW.

Questions:
a) Do we need to add a new set of APIs in libvirt to return the PDH from
libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
opaque blobs ? (this is mainly for step 3 and 6)
b) do we need to define a new xml tag to for memory-encryption ? or just
use the qemu:args tag ? (step 6)
c) what existing communicate interface can be used between libvirt and qemu
to get the measurement ? can we add a new qemu monitor command
'get_sev_measurement' to get the measurement ? (step 10)
d) how to pass the secret blob from libvirt to qemu ? should we consider
adding a new object (sev-guest-secret) -- libvirt can add the object through
qemu monitor.


[1] https://marc.info/?l=kvm&m=150092661105069&w=2
[2] https://marc.info/?l=qemu-devel&m=148901186615642&w=2
[3] https://lists.01.org/pipermail/edk2-devel/2017-July/012220.html
[4] http://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf

Thanks

Brijesh

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Daniel P. Berrange]

On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
> Hi All,
> 
> (sorry for the long message)
> 
> CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
> feature - the feature allows running encrypted VMs. To enable the feature,
> I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
> We have been making some good progress in getting patches accepted upstream
> in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
> feature -- SME support just got pulled into 4.14 merge window. The base
> SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
> I am getting ready to take off "RFC" tag from remaining patches to get them
> reviewed and accepted.
> 
> The boot flow for launching an SEV guest is a bit different from a typical
> guest launch. In order to launch SEV guest from virt-manager or other
> high-level VM management tools, we need to design and implement new
> interface between libvirt and qemu, and probably add new APIs in libvirt
> to be used by VM management tools. I am new to the libvirt and need some
> expert advice while designing this interface. A pictorial representation
> for a SEV guest launch flow is available in SEV Spec Appendix A [4].
> 
> A typical flow looks like this:
> 
> 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
> 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
> 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
>    caller.

What sort of size are we talking about for the PDH ?

There's a few ways libvirt could report it

 1. As an XML element in the host capabilities XML
 2. As an XML element in the emulator capabilities XML
 3. Via a newly added host API

> 4. VM tool gives its PDH to GO.
> 5. GO provides its DH key, session-info and guest policy.

Are steps 4 & 5 strictly required to be in this order, or is it
possible

What are the security requirements around the DH key, session info
and guest policy ? Are any of them sensitive data which needs to be
kept private from untrustworthy users. Also are all three of these
items different for every guest launched, or are some of them
likely to be the same for every guest ?

eg, would the same guest policy blob be used for every guest, with
only session info changing ?

Also what sort of size are we talking about for each of these
data items, KBs, 10's of KB, 100's of KBs or larger ?

The security and data size can influence our design approach from
the libvirt POV.

> 6. VM tool somehow communicates the GO provided information to libvirt.

Essentially we have two choices

 1. inline in the guest XML config description passed to libvirt
    when defining the guest XML

 2. out of band, ahead of time, via some other API prior to defining
    the guest XML, which is then referenced in guest XML. THis could
    be done using the virSecret APIs, where we create 3 secrets, one
    each for DH key, session-info and guets policy. The UUID of the
    secret could be specified in the guest XML. This has flexibility
    of allowing the same secrets ot be used for many guests (if this
    is valid for SEV)


> 7. libvirt adds "sev-guest" object in its xml file with all the information
>    obtained from #5
> 
>    (currently my xml file looks like this)
> 
>    <qemu:arg value='-object'>
>    <qemu:arg
> value='sev-guest,id=sev0,policy=<GO_policy>,dh-key-file=<filename>,session-file=<filename>/>
>    <qemu:arg value='-machine'/>
>    <qemu:arg value='memory-encryption=sev0'/>
> 
> 8. libvirt launches the guest with "-S"

All libvirt guests get launched with -S, to give libvirt chance to do some
setup before starting vCPUs. Normally vCPUs are started by default, but
the VIR_DOMAIN_START_PAUSED flag allows the mgmt app to tell libvirt to
leave vCPUS paused.

Alternatively, if libvirt sees presencese of 'sev' config for the guest,
it could automatically leave it in PAUSED state regardless of the
VIR_DOMAIN_START_PAUSED flag.

> 9. While creating the SEV guest qemu does the following
>  i) create encryption context using GO's DH, session-info and guest policy
>     (LAUNCH_START)
>  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
>  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
> 10. By some interface we must propagate the measurement all the way to GO
>   before libvirt starts the guest.

Again, what kind of size data are we talking about for athe "measurement"
blob ? a KB, 10's of KB, or more ?

My first gut instinct would be for QEMU to emit a QMP event when it has
the measurement available. The event could include the actual data blob,
or we can could add an explicit QMP command to fetch the data blob.

Libvirt could listen for this QEMU event, and in turn emit an event from
libvirt with the same data, which the mgmt tool can finally give to the
GO.

> 11. GO verifies the measurement and if measurement matches then it may
>  give a secret blob -- which must be injected into the guest before
>  libvirt starts the VM. If verification failed, GO will request cloud
>  provider to destroy the VM.

So we need some mechanism to provide the secret blob.  This could be
done via a new libvirt API.  Alternatively, if we're using virSecret
for the other stuff, the guest config XML could include the UUID of
a 4th  secret. Libvirt would then watch to see when that secret has
a value set, and pass that onto QEMU.

> 12. After secret blob is injected into guest, we call LAUNCH_FINISH
>   to destory the encryption context.
> 13. libvirt issues "continue" command to resume the guest boot.

This is as simple as the mgmt tool calling virDomainResume to unpause
CPUs. We could have it such that virDomainResume checks whether the
virSecret has been populated secret blob by GO, and pass it onto
QEMU at this time.

> Please note that the measurement value is protected with transport
> encryption key (TIK) and it changes on each run. Similarly the secret blob
> provided by GO does not need to be protected using libvirt/qemu APIs. The
> secret is protected by TIK. From qemu and libvirt point of view these are
> blobs and must be passed as-is to the SEV FW.
> 
> Questions:
> a) Do we need to add a new set of APIs in libvirt to return the PDH from
> libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
> opaque blobs ? (this is mainly for step 3 and 6)
> b) do we need to define a new xml tag to for memory-encryption ? or just
> use the qemu:args tag ? (step 6)

<qemu:args> is explicitly only ever for ad-hoc testing.

For anything that is to be used in production deployment we must
explicitly model it in the XML. So we definitely need new XML
defined.

> c) what existing communicate interface can be used between libvirt and qemu
> to get the measurement ? can we add a new qemu monitor command
> 'get_sev_measurement' to get the measurement ? (step 10)

Yes, QMP commands seeem most likely.

> d) how to pass the secret blob from libvirt to qemu ? should we consider
> adding a new object (sev-guest-secret) -- libvirt can add the object through
> qemu monitor.

Yeah, that looks like a viable option too.

> 
> 
> [1] https://marc.info/?l=kvm&m=150092661105069&w=2
> [2] https://marc.info/?l=qemu-devel&m=148901186615642&w=2
> [3] https://lists.01.org/pipermail/edk2-devel/2017-July/012220.html
> [4] http://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Relph, Richard]

A few answers in line…

On 9/8/17, 8:16 AM, "Daniel P. Berrange" <berrange@redhat.com> wrote:

    On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
    > Hi All,
    > 
    > (sorry for the long message)
    > 
    > CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
    > feature - the feature allows running encrypted VMs. To enable the feature,
    > I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
    > We have been making some good progress in getting patches accepted upstream
    > in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
    > feature -- SME support just got pulled into 4.14 merge window. The base
    > SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
    > I am getting ready to take off "RFC" tag from remaining patches to get them
    > reviewed and accepted.
    > 
    > The boot flow for launching an SEV guest is a bit different from a typical
    > guest launch. In order to launch SEV guest from virt-manager or other
    > high-level VM management tools, we need to design and implement new
    > interface between libvirt and qemu, and probably add new APIs in libvirt
    > to be used by VM management tools. I am new to the libvirt and need some
    > expert advice while designing this interface. A pictorial representation
    > for a SEV guest launch flow is available in SEV Spec Appendix A [4].
    > 
    > A typical flow looks like this:
    > 
    > 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
    > 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
    > 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
    >    caller.
    
    What sort of size are we talking about for the PDH ?

The PDH blob is described in reference 4. It’s 0x824 bytes long… a bit over 2K bytes.
PDH is “Platform Diffie-Hellman” key, public portion.
    
    There's a few ways libvirt could report it
    
     1. As an XML element in the host capabilities XML
     2. As an XML element in the emulator capabilities XML
     3. Via a newly added host API
    
    > 4. VM tool gives its PDH to GO.
    > 5. GO provides its DH key, session-info and guest policy.
    
    Are steps 4 & 5 strictly required to be in this order, or is it
    possible

Steps 4 and 5 must occur in that order. The data sent by the GO in the “session info” is encrypted and integrity protected with keys that the GO derives from the GO private Diffie-Hellman key and the PDH public Diffie-Hellman key.
    
    What are the security requirements around the DH key, session info
    and guest policy ? Are any of them sensitive data which needs to be
    kept private from untrustworthy users. Also are all three of these
    items different for every guest launched, or are some of them
    likely to be the same for every guest ?

The PDH is not sensitive. It is relatively static for the platform. (It only changes when the platform owner chooses to change the apparent identity of the platform that will actually run the virtual machine.)
The 128-byte session info data is encrypted and integrity protected by the GO. It need not be additionally encrypted or integrity protected. It should vary for EVERY guest launch.
The 4-byte guest policy must be sent in the clear as some components may want to observe the policy bits. It may change from guest to guest, but there will likely only be a few common values.
    
    eg, would the same guest policy blob be used for every guest, with
    only session info changing ?

Also what sort of size are we talking about for each of these
    data items, KBs, 10's of KB, 100's of KBs or larger ?
    
    The security and data size can influence our design approach from
    the libvirt POV.
    
    > 6. VM tool somehow communicates the GO provided information to libvirt.
    
    Essentially we have two choices
    
     1. inline in the guest XML config description passed to libvirt
        when defining the guest XML
    
     2. out of band, ahead of time, via some other API prior to defining
        the guest XML, which is then referenced in guest XML. THis could
        be done using the virSecret APIs, where we create 3 secrets, one
        each for DH key, session-info and guets policy. The UUID of the
        secret could be specified in the guest XML. This has flexibility
        of allowing the same secrets ot be used for many guests (if this
        is valid for SEV)
    
    
    > 7. libvirt adds "sev-guest" object in its xml file with all the information
    >    obtained from #5
    > 
    >    (currently my xml file looks like this)
    > 
    >    <qemu:arg value='-object'>
    >    <qemu:arg
    > value='sev-guest,id=sev0,policy=<GO_policy>,dh-key-file=<filename>,session-file=<filename>/>
    >    <qemu:arg value='-machine'/>
    >    <qemu:arg value='memory-encryption=sev0'/>
    > 
    > 8. libvirt launches the guest with "-S"
    
    All libvirt guests get launched with -S, to give libvirt chance to do some
    setup before starting vCPUs. Normally vCPUs are started by default, but
    the VIR_DOMAIN_START_PAUSED flag allows the mgmt app to tell libvirt to
    leave vCPUS paused.
    
    Alternatively, if libvirt sees presencese of 'sev' config for the guest,
    it could automatically leave it in PAUSED state regardless of the
    VIR_DOMAIN_START_PAUSED flag.

While using the LAUNCH_MEASURE and LAUNCH_SECRET operations is expected to be the common case, they are optional. I would recommend requiring the upstream software to explicitly set the VIR_DOMAIN_START_PAUSED flag, if only to minimize the dependencies…
    
    > 9. While creating the SEV guest qemu does the following
    >  i) create encryption context using GO's DH, session-info and guest policy
    >     (LAUNCH_START)
    >  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
    >  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
    > 10. By some interface we must propagate the measurement all the way to GO
    >   before libvirt starts the guest.
    
    Again, what kind of size data are we talking about for athe "measurement"
    blob ? a KB, 10's of KB, or more ?

The measurement is 48 bytes…
    
    My first gut instinct would be for QEMU to emit a QMP event when it has
    the measurement available. The event could include the actual data blob,
    or we can could add an explicit QMP command to fetch the data blob.
    
    Libvirt could listen for this QEMU event, and in turn emit an event from
    libvirt with the same data, which the mgmt tool can finally give to the
    GO.
    
    > 11. GO verifies the measurement and if measurement matches then it may
    >  give a secret blob -- which must be injected into the guest before
    >  libvirt starts the VM. If verification failed, GO will request cloud
    >  provider to destroy the VM.
    
    So we need some mechanism to provide the secret blob.  This could be
    done via a new libvirt API.  Alternatively, if we're using virSecret
    for the other stuff, the guest config XML could include the UUID of
    a 4th  secret. Libvirt would then watch to see when that secret has
    a value set, and pass that onto QEMU.

The secret is optional… it is up to 16KB, already encrypted, and integrity protected with an IV and MAC value passed with the secret. The guest address the secret should be deposited at is 
Technically, the SEV FW API allows there to be more than one secret… I don’t see a strong reason to require libvirt to support more than one, though.
    
    > 12. After secret blob is injected into guest, we call LAUNCH_FINISH
    >   to destory the encryption context.
    > 13. libvirt issues "continue" command to resume the guest boot.
    
    This is as simple as the mgmt tool calling virDomainResume to unpause
    CPUs. We could have it such that virDomainResume checks whether the
    virSecret has been populated secret blob by GO, and pass it onto
    QEMU at this time.
    
    > Please note that the measurement value is protected with transport
    > encryption key (TIK) and it changes on each run. Similarly the secret blob
    > provided by GO does not need to be protected using libvirt/qemu APIs. The
    > secret is protected by TIK. From qemu and libvirt point of view these are
    > blobs and must be passed as-is to the SEV FW.
    > 
    > Questions:
    > a) Do we need to add a new set of APIs in libvirt to return the PDH from
    > libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
    > opaque blobs ? (this is mainly for step 3 and 6)
    > b) do we need to define a new xml tag to for memory-encryption ? or just
    > use the qemu:args tag ? (step 6)
    
    <qemu:args> is explicitly only ever for ad-hoc testing.
    
    For anything that is to be used in production deployment we must
    explicitly model it in the XML. So we definitely need new XML
    defined.
    
    > c) what existing communicate interface can be used between libvirt and qemu
    > to get the measurement ? can we add a new qemu monitor command
    > 'get_sev_measurement' to get the measurement ? (step 10)
    
    Yes, QMP commands seeem most likely.
    
    > d) how to pass the secret blob from libvirt to qemu ? should we consider
    > adding a new object (sev-guest-secret) -- libvirt can add the object through
    > qemu monitor.
    
    Yeah, that looks like a viable option too.
    
    > 
    > 
    > [1] https://marc.info/?l=kvm&m=150092661105069&w=2
    > [2] https://marc.info/?l=qemu-devel&m=148901186615642&w=2
    > [3] https://lists.01.org/pipermail/edk2-devel/2017-July/012220.html
    > [4] http://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf
    
    Regards,
    Daniel
    -- 
    |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
    |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
    |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
    

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Daniel P. Berrange]

On Fri, Sep 08, 2017 at 01:45:06PM +0000, Relph, Richard wrote:
> A few answers in line…
> 
> On 9/8/17, 8:16 AM, "Daniel P. Berrange" <berrange@redhat.com> wrote:
> 
>     On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
>     > Hi All,
>     > 
>     > (sorry for the long message)
>     > 
>     > CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
>     > feature - the feature allows running encrypted VMs. To enable the feature,
>     > I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
>     > We have been making some good progress in getting patches accepted upstream
>     > in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
>     > feature -- SME support just got pulled into 4.14 merge window. The base
>     > SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
>     > I am getting ready to take off "RFC" tag from remaining patches to get them
>     > reviewed and accepted.
>     > 
>     > The boot flow for launching an SEV guest is a bit different from a typical
>     > guest launch. In order to launch SEV guest from virt-manager or other
>     > high-level VM management tools, we need to design and implement new
>     > interface between libvirt and qemu, and probably add new APIs in libvirt
>     > to be used by VM management tools. I am new to the libvirt and need some
>     > expert advice while designing this interface. A pictorial representation
>     > for a SEV guest launch flow is available in SEV Spec Appendix A [4].
>     > 
>     > A typical flow looks like this:
>     > 
>     > 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
>     > 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
>     > 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
>     >    caller.
>     
>     What sort of size are we talking about for the PDH ?
> 
> The PDH blob is described in reference 4. It’s 0x824 bytes long… a bit over 2K bytes.
> PDH is “Platform Diffie-Hellman” key, public portion.
>     
>     There's a few ways libvirt could report it
>     
>      1. As an XML element in the host capabilities XML
>      2. As an XML element in the emulator capabilities XML
>      3. Via a newly added host API
>     
>     > 4. VM tool gives its PDH to GO.
>     > 5. GO provides its DH key, session-info and guest policy.
>     
>     Are steps 4 & 5 strictly required to be in this order, or is it
>     possible
> 
> Steps 4 and 5 must occur in that order. The data sent by the GO in the
> “session info” is encrypted and integrity protected with keys that the
> GO derives from the GO private Diffie-Hellman key and the PDH public
> Diffie-Hellman key.
>     
>     What are the security requirements around the DH key, session info
>     and guest policy ? Are any of them sensitive data which needs to be
>     kept private from untrustworthy users. Also are all three of these
>     items different for every guest launched, or are some of them
>     likely to be the same for every guest ?
> 
> The PDH is not sensitive. It is relatively static for the platform.
> (It only changes when the platform owner chooses to change the apparent
> identity of the platform that will actually run the virtual machine.)
> The 128-byte session info data is encrypted and integrity protected by
> the GO. It need not be additionally encrypted or integrity protected.
> It should vary for EVERY guest launch.
> The 4-byte guest policy must be sent in the clear as some components
> may want to observe the policy bits. It may change from guest to guest,
> but there will likely only be a few common values.

Given this, and the pretty small data sizes, I think this info could
in fact all be provided inline in the XML config - either hex or base64
encoded for the binary blobs.



>     > 8. libvirt launches the guest with "-S"
>     
>     All libvirt guests get launched with -S, to give libvirt chance to do some
>     setup before starting vCPUs. Normally vCPUs are started by default, but
>     the VIR_DOMAIN_START_PAUSED flag allows the mgmt app to tell libvirt to
>     leave vCPUS paused.
>     
>     Alternatively, if libvirt sees presencese of 'sev' config for the guest,
>     it could automatically leave it in PAUSED state regardless of the
>     VIR_DOMAIN_START_PAUSED flag.
> 
> While using the LAUNCH_MEASURE and LAUNCH_SECRET operations is expected
> to be the common case, they are optional. I would recommend requiring
> the upstream software to explicitly set the VIR_DOMAIN_START_PAUSED flag,
> if only to minimize the dependencies…

Ok.

>     > 9. While creating the SEV guest qemu does the following
>     >  i) create encryption context using GO's DH, session-info and guest policy
>     >     (LAUNCH_START)
>     >  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
>     >  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
>     > 10. By some interface we must propagate the measurement all the way to GO
>     >   before libvirt starts the guest.
>     
>     Again, what kind of size data are we talking about for athe "measurement"
>     blob ? a KB, 10's of KB, or more ?
> 
> The measurement is 48 bytes…

For that small size we can definitely provide is inline in the event
payload then.

>     My first gut instinct would be for QEMU to emit a QMP event when it has
>     the measurement available. The event could include the actual data blob,
>     or we can could add an explicit QMP command to fetch the data blob.
>     
>     Libvirt could listen for this QEMU event, and in turn emit an event from
>     libvirt with the same data, which the mgmt tool can finally give to the
>     GO.
>     
>     > 11. GO verifies the measurement and if measurement matches then it may
>     >  give a secret blob -- which must be injected into the guest before
>     >  libvirt starts the VM. If verification failed, GO will request cloud
>     >  provider to destroy the VM.
>     
>     So we need some mechanism to provide the secret blob.  This could be
>     done via a new libvirt API.  Alternatively, if we're using virSecret
>     for the other stuff, the guest config XML could include the UUID of
>     a 4th  secret. Libvirt would then watch to see when that secret has
>     a value set, and pass that onto QEMU.
> 
> The secret is optional… it is up to 16KB, already encrypted, and integrity
> protected with an IV and MAC value passed with the secret. The guest
> address the secret should be deposited at is 
> Technically, the SEV FW API allows there to be more than one secret…
> I don’t see a strong reason to require libvirt to support more than one,
> though.

So I think we could simply add a new API to libvirt to provide this
data item.

>     > 12. After secret blob is injected into guest, we call LAUNCH_FINISH
>     >   to destory the encryption context.
>     > 13. libvirt issues "continue" command to resume the guest boot.
>     
>     This is as simple as the mgmt tool calling virDomainResume to unpause
>     CPUs. We could have it such that virDomainResume checks whether the
>     virSecret has been populated secret blob by GO, and pass it onto
>     QEMU at this time.
>     
>     > Please note that the measurement value is protected with transport
>     > encryption key (TIK) and it changes on each run. Similarly the secret blob
>     > provided by GO does not need to be protected using libvirt/qemu APIs. The
>     > secret is protected by TIK. From qemu and libvirt point of view these are
>     > blobs and must be passed as-is to the SEV FW.
>     > 
>     > Questions:
>     > a) Do we need to add a new set of APIs in libvirt to return the PDH from
>     > libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
>     > opaque blobs ? (this is mainly for step 3 and 6)
>     > b) do we need to define a new xml tag to for memory-encryption ? or just
>     > use the qemu:args tag ? (step 6)
>     
>     <qemu:args> is explicitly only ever for ad-hoc testing.
>     
>     For anything that is to be used in production deployment we must
>     explicitly model it in the XML. So we definitely need new XML
>     defined.
>     
>     > c) what existing communicate interface can be used between libvirt and qemu
>     > to get the measurement ? can we add a new qemu monitor command
>     > 'get_sev_measurement' to get the measurement ? (step 10)
>     
>     Yes, QMP commands seeem most likely.
>     
>     > d) how to pass the secret blob from libvirt to qemu ? should we consider
>     > adding a new object (sev-guest-secret) -- libvirt can add the object through
>     > qemu monitor.
>     
>     Yeah, that looks like a viable option too.

So I could see a flow like the following:


  1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
     document that includes the following

      <host>
         ...other bits...
        <sev>
	  <platform-key>...hex encoded PDH key...</platform-key>
	</sev>
      </host>

  2. mgmt tool requests to start a guest calling virCreateXML(),
     passing VIR_DOMAIN_START_PAUSED. The XML would include

      <sev>
        <owner-key>...hex encode DH key...</owner-key>
	<session-info>..hex encode info...</session-info>
	<policy>...int32 value..</policy>
      </sev>


     if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
     libvirt would report an error and refuse to start the guest

  3. Libvirt generates the QEMU cli arg to enable SEV using
     the XML data and starts QEMU, leaving CPUs paused

  4. QEMU emits a SEV_MEASURE event containing the measurement
     blob

  5. Libvirt catches the QEMU event and emits its own
     VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
     the measurement blob

  6. GO does its validation of the measurement

  7a  If validation failed, then virDomainDestroy() to stop QEMU

  7b  If validation succeeed

     Optionally call

         virDomainSetSEVSecret()

     providing the optional secret, then

         virDomainResume()

     to let QEMU continue




Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Brijesh Singh]

Hi Daniel,


On 09/08/2017 09:52 AM, Daniel P. Berrange wrote:
> On Fri, Sep 08, 2017 at 01:45:06PM +0000, Relph, Richard wrote:
>> A few answers in line…
>>
>> On 9/8/17, 8:16 AM, "Daniel P. Berrange" <berrange@redhat.com> wrote:
>>
>>      On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
>>      > Hi All,
>>      >
>>      > (sorry for the long message)
>>      >
>>      > CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
>>      > feature - the feature allows running encrypted VMs. To enable the feature,
>>      > I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
>>      > We have been making some good progress in getting patches accepted upstream
>>      > in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
>>      > feature -- SME support just got pulled into 4.14 merge window. The base
>>      > SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
>>      > I am getting ready to take off "RFC" tag from remaining patches to get them
>>      > reviewed and accepted.
>>      >
>>      > The boot flow for launching an SEV guest is a bit different from a typical
>>      > guest launch. In order to launch SEV guest from virt-manager or other
>>      > high-level VM management tools, we need to design and implement new
>>      > interface between libvirt and qemu, and probably add new APIs in libvirt
>>      > to be used by VM management tools. I am new to the libvirt and need some
>>      > expert advice while designing this interface. A pictorial representation
>>      > for a SEV guest launch flow is available in SEV Spec Appendix A [4].
>>      >
>>      > A typical flow looks like this:
>>      >
>>      > 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
>>      > 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
>>      > 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
>>      >    caller.
>>      
>>      What sort of size are we talking about for the PDH ?
>>
>> The PDH blob is described in reference 4. It’s 0x824 bytes long… a bit over 2K bytes.
>> PDH is “Platform Diffie-Hellman” key, public portion.
>>      
>>      There's a few ways libvirt could report it
>>      
>>       1. As an XML element in the host capabilities XML
>>       2. As an XML element in the emulator capabilities XML
>>       3. Via a newly added host API
>>      
>>      > 4. VM tool gives its PDH to GO.
>>      > 5. GO provides its DH key, session-info and guest policy.
>>      
>>      Are steps 4 & 5 strictly required to be in this order, or is it
>>      possible
>>
>> Steps 4 and 5 must occur in that order. The data sent by the GO in the
>> “session info” is encrypted and integrity protected with keys that the
>> GO derives from the GO private Diffie-Hellman key and the PDH public
>> Diffie-Hellman key.
>>      
>>      What are the security requirements around the DH key, session info
>>      and guest policy ? Are any of them sensitive data which needs to be
>>      kept private from untrustworthy users. Also are all three of these
>>      items different for every guest launched, or are some of them
>>      likely to be the same for every guest ?
>>
>> The PDH is not sensitive. It is relatively static for the platform.
>> (It only changes when the platform owner chooses to change the apparent
>> identity of the platform that will actually run the virtual machine.)
>> The 128-byte session info data is encrypted and integrity protected by
>> the GO. It need not be additionally encrypted or integrity protected.
>> It should vary for EVERY guest launch.
>> The 4-byte guest policy must be sent in the clear as some components
>> may want to observe the policy bits. It may change from guest to guest,
>> but there will likely only be a few common values.
> 
> Given this, and the pretty small data sizes, I think this info could
> in fact all be provided inline in the XML config - either hex or base64
> encoded for the binary blobs.
> 
> 
> 
>>      > 8. libvirt launches the guest with "-S"
>>      
>>      All libvirt guests get launched with -S, to give libvirt chance to do some
>>      setup before starting vCPUs. Normally vCPUs are started by default, but
>>      the VIR_DOMAIN_START_PAUSED flag allows the mgmt app to tell libvirt to
>>      leave vCPUS paused.
>>      
>>      Alternatively, if libvirt sees presencese of 'sev' config for the guest,
>>      it could automatically leave it in PAUSED state regardless of the
>>      VIR_DOMAIN_START_PAUSED flag.
>>
>> While using the LAUNCH_MEASURE and LAUNCH_SECRET operations is expected
>> to be the common case, they are optional. I would recommend requiring
>> the upstream software to explicitly set the VIR_DOMAIN_START_PAUSED flag,
>> if only to minimize the dependencies…
> 
> Ok.
> 
>>      > 9. While creating the SEV guest qemu does the following
>>      >  i) create encryption context using GO's DH, session-info and guest policy
>>      >     (LAUNCH_START)
>>      >  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
>>      >  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
>>      > 10. By some interface we must propagate the measurement all the way to GO
>>      >   before libvirt starts the guest.
>>      
>>      Again, what kind of size data are we talking about for athe "measurement"
>>      blob ? a KB, 10's of KB, or more ?
>>
>> The measurement is 48 bytes…
> 
> For that small size we can definitely provide is inline in the event
> payload then.
> 
>>      My first gut instinct would be for QEMU to emit a QMP event when it has
>>      the measurement available. The event could include the actual data blob,
>>      or we can could add an explicit QMP command to fetch the data blob.
>>      
>>      Libvirt could listen for this QEMU event, and in turn emit an event from
>>      libvirt with the same data, which the mgmt tool can finally give to the
>>      GO.
>>      
>>      > 11. GO verifies the measurement and if measurement matches then it may
>>      >  give a secret blob -- which must be injected into the guest before
>>      >  libvirt starts the VM. If verification failed, GO will request cloud
>>      >  provider to destroy the VM.
>>      
>>      So we need some mechanism to provide the secret blob.  This could be
>>      done via a new libvirt API.  Alternatively, if we're using virSecret
>>      for the other stuff, the guest config XML could include the UUID of
>>      a 4th  secret. Libvirt would then watch to see when that secret has
>>      a value set, and pass that onto QEMU.
>>
>> The secret is optional… it is up to 16KB, already encrypted, and integrity
>> protected with an IV and MAC value passed with the secret. The guest
>> address the secret should be deposited at is
>> Technically, the SEV FW API allows there to be more than one secret…
>> I don’t see a strong reason to require libvirt to support more than one,
>> though.
> 
> So I think we could simply add a new API to libvirt to provide this
> data item.
> 
>>      > 12. After secret blob is injected into guest, we call LAUNCH_FINISH
>>      >   to destory the encryption context.
>>      > 13. libvirt issues "continue" command to resume the guest boot.
>>      
>>      This is as simple as the mgmt tool calling virDomainResume to unpause
>>      CPUs. We could have it such that virDomainResume checks whether the
>>      virSecret has been populated secret blob by GO, and pass it onto
>>      QEMU at this time.
>>      
>>      > Please note that the measurement value is protected with transport
>>      > encryption key (TIK) and it changes on each run. Similarly the secret blob
>>      > provided by GO does not need to be protected using libvirt/qemu APIs. The
>>      > secret is protected by TIK. From qemu and libvirt point of view these are
>>      > blobs and must be passed as-is to the SEV FW.
>>      >
>>      > Questions:
>>      > a) Do we need to add a new set of APIs in libvirt to return the PDH from
>>      > libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
>>      > opaque blobs ? (this is mainly for step 3 and 6)
>>      > b) do we need to define a new xml tag to for memory-encryption ? or just
>>      > use the qemu:args tag ? (step 6)
>>      
>>      <qemu:args> is explicitly only ever for ad-hoc testing.
>>      
>>      For anything that is to be used in production deployment we must
>>      explicitly model it in the XML. So we definitely need new XML
>>      defined.
>>      
>>      > c) what existing communicate interface can be used between libvirt and qemu
>>      > to get the measurement ? can we add a new qemu monitor command
>>      > 'get_sev_measurement' to get the measurement ? (step 10)
>>      
>>      Yes, QMP commands seeem most likely.
>>      
>>      > d) how to pass the secret blob from libvirt to qemu ? should we consider
>>      > adding a new object (sev-guest-secret) -- libvirt can add the object through
>>      > qemu monitor.
>>      
>>      Yeah, that looks like a viable option too.
> 
> So I could see a flow like the following:


The flow looks good

> 
> 
>    1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
>       document that includes the following
> 
>        <host>
>           ...other bits...
>          <sev>
> 	  <platform-key>...hex encoded PDH key...</platform-key>
> 	</sev>
>        </host>
> 
>    2. mgmt tool requests to start a guest calling virCreateXML(),
>       passing VIR_DOMAIN_START_PAUSED. The XML would include
> 
>        <sev>
>          <owner-key>...hex encode DH key...</owner-key>
> 	<session-info>..hex encode info...</session-info>
> 	<policy>...int32 value..</policy>
>        </sev>
> 
> 
>       if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
>       libvirt would report an error and refuse to start the guest
> 


One thing which is not clear to me is, how do we know that we are asked
to launch SEV guest? Are you thinking that <sev> tag in the XML will
hint libvirt that GO has asked to launch a SEV guest?


>    3. Libvirt generates the QEMU cli arg to enable SEV using
>       the XML data and starts QEMU, leaving CPUs paused
> 


I am looking at [1] to get the feel for how do we model it in the XML.
As you can see I am using ad-hoc <qemu:args> to create the sev-guest
object. Currently, sev-guest object accepts the following properties:

dh-cert-file: <file containing the GO DH key>
session-info-file: <file contain the GO session info>
policy: <int32 GO policy>

I believe the new XML model will influence the property input type,
Any recommendation on how do model this part ? thank you so much.

[1] https://libvirt.org/formatdomain.html#elementsCPU


>    4. QEMU emits a SEV_MEASURE event containing the measurement
>       blob
> 
>    5. Libvirt catches the QEMU event and emits its own
>       VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
>       the measurement blob
> 
>    6. GO does its validation of the measurement
> 
>    7a  If validation failed, then virDomainDestroy() to stop QEMU
> 
>    7b  If validation succeeed
> 
>       Optionally call
> 
>           virDomainSetSEVSecret()
> 
>       providing the optional secret, then
> 
>           virDomainResume()
> 
>       to let QEMU continue
> 
> 
> 
> 
> Regards,
> Daniel
> 

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Daniel P. Berrange]

On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:
> > So I could see a flow like the following:
> 
> 
> The flow looks good
> 
> > 
> > 
> >    1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
> >       document that includes the following
> > 
> >        <host>
> >           ...other bits...
> >          <sev>
> > 	  <platform-key>...hex encoded PDH key...</platform-key>
> > 	</sev>
> >        </host>
> > 
> >    2. mgmt tool requests to start a guest calling virCreateXML(),
> >       passing VIR_DOMAIN_START_PAUSED. The XML would include
> > 
> >        <sev>
> >          <owner-key>...hex encode DH key...</owner-key>
> > 	<session-info>..hex encode info...</session-info>
> > 	<policy>...int32 value..</policy>
> >        </sev>
> > 
> > 
> >       if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
> >       libvirt would report an error and refuse to start the guest
> > 
> 
> 
> One thing which is not clear to me is, how do we know that we are asked
> to launch SEV guest? Are you thinking that <sev> tag in the XML will
> hint libvirt that GO has asked to launch a SEV guest?

Yes, the existance of the <sev> tag is the indicator that informs
libvirt that SEV *must* be used for the guest.

> >    3. Libvirt generates the QEMU cli arg to enable SEV using
> >       the XML data and starts QEMU, leaving CPUs paused
> > 
> 
> 
> I am looking at [1] to get the feel for how do we model it in the XML.
> As you can see I am using ad-hoc <qemu:args> to create the sev-guest
> object. Currently, sev-guest object accepts the following properties:
> 
> dh-cert-file: <file containing the GO DH key>
> session-info-file: <file contain the GO session info>
> policy: <int32 GO policy>
> 
> I believe the new XML model will influence the property input type,
> Any recommendation on how do model this part ? thank you so much.

That looks ok to me - even if QEMU wants the data provided in
files on disk, libvirt can just create the files on the fly
from the data it has in the <sev> element in the XML file.
Since they're only needed during startup, libvirt can then
easily delete the files the moment QEMU has completed its
startup.

> 
> [1] https://libvirt.org/formatdomain.html#elementsCPU
> 
> 
> >    4. QEMU emits a SEV_MEASURE event containing the measurement
> >       blob
> > 
> >    5. Libvirt catches the QEMU event and emits its own
> >       VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
> >       the measurement blob
> > 
> >    6. GO does its validation of the measurement
> > 
> >    7a  If validation failed, then virDomainDestroy() to stop QEMU
> > 
> >    7b  If validation succeeed
> > 
> >       Optionally call
> > 
> >           virDomainSetSEVSecret()
> > 
> >       providing the optional secret, then
> > 
> >           virDomainResume()
> > 
> >       to let QEMU continue
> > 
> > 
> > 
> > 
> > Regards,
> > Daniel
> > 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Brijesh Singh]

On 09/08/2017 10:51 AM, Daniel P. Berrange wrote:
> On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:
>>> So I could see a flow like the following:
>>
>>
>> The flow looks good
>>
>>>
>>>
>>>     1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
>>>        document that includes the following
>>>
>>>         <host>
>>>            ...other bits...
>>>           <sev>
>>> 	  <platform-key>...hex encoded PDH key...</platform-key>
>>> 	</sev>
>>>         </host>
>>>
>>>     2. mgmt tool requests to start a guest calling virCreateXML(),
>>>        passing VIR_DOMAIN_START_PAUSED. The XML would include
>>>
>>>         <sev>
>>>           <owner-key>...hex encode DH key...</owner-key>
>>> 	<session-info>..hex encode info...</session-info>
>>> 	<policy>...int32 value..</policy>
>>>         </sev>
>>>
>>>
>>>        if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
>>>        libvirt would report an error and refuse to start the guest
>>>
>>
>>
>> One thing which is not clear to me is, how do we know that we are asked
>> to launch SEV guest? Are you thinking that <sev> tag in the XML will
>> hint libvirt that GO has asked to launch a SEV guest?
> 
> Yes, the existance of the <sev> tag is the indicator that informs
> libvirt that SEV *must* be used for the guest.


Thanks for confirming.


> 
>>>     3. Libvirt generates the QEMU cli arg to enable SEV using
>>>        the XML data and starts QEMU, leaving CPUs paused
>>>
>>
>>
>> I am looking at [1] to get the feel for how do we model it in the XML.
>> As you can see I am using ad-hoc <qemu:args> to create the sev-guest
>> object. Currently, sev-guest object accepts the following properties:
>>
>> dh-cert-file: <file containing the GO DH key>
>> session-info-file: <file contain the GO session info>
>> policy: <int32 GO policy>
>>
>> I believe the new XML model will influence the property input type,
>> Any recommendation on how do model this part ? thank you so much.
> 
> That looks ok to me - even if QEMU wants the data provided in
> files on disk, libvirt can just create the files on the fly
> from the data it has in the <sev> element in the XML file.
> Since they're only needed during startup, libvirt can then
> easily delete the files the moment QEMU has completed its
> startup.
> 

Perfect! works well with me.

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Laszlo Ersek]

On 09/08/17 17:51, Daniel P. Berrange wrote:
> On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:

>> I am looking at [1] to get the feel for how do we model it in the XML.
>> As you can see I am using ad-hoc <qemu:args> to create the sev-guest
>> object. Currently, sev-guest object accepts the following properties:
>>
>> dh-cert-file: <file containing the GO DH key>
>> session-info-file: <file contain the GO session info>
>> policy: <int32 GO policy>
>>
>> I believe the new XML model will influence the property input type,
>> Any recommendation on how do model this part ? thank you so much.
> 
> That looks ok to me - even if QEMU wants the data provided in
> files on disk, libvirt can just create the files on the fly
> from the data it has in the <sev> element in the XML file.
> Since they're only needed during startup, libvirt can then
> easily delete the files the moment QEMU has completed its
> startup.

/dev/fd/N filenames could be used for poor man's fd passing, I think.

(/dev/fd is a symlink to the /proc/self/fd directory)

proc(5) has documentation on this.

Thanks,
Laszlo

Re: [Qemu-devel] [libvirt] libvirt/QEMU/SEV interaction

[Erik Skultety]

[...]

> >
> >     > c) what existing communicate interface can be used between libvirt and qemu
> >     > to get the measurement ? can we add a new qemu monitor command
> >     > 'get_sev_measurement' to get the measurement ? (step 10)
> >
> >     Yes, QMP commands seeem most likely.
> >
> >     > d) how to pass the secret blob from libvirt to qemu ? should we consider
> >     > adding a new object (sev-guest-secret) -- libvirt can add the object through
> >     > qemu monitor.
> >
> >     Yeah, that looks like a viable option too.
>
> So I could see a flow like the following:
>
>
>   1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
>      document that includes the following
>
>       <host>
>          ...other bits...
>         <sev>
> 	  <platform-key>...hex encoded PDH key...</platform-key>
> 	</sev>
>       </host>
>
>   2. mgmt tool requests to start a guest calling virCreateXML(),
>      passing VIR_DOMAIN_START_PAUSED. The XML would include
>
>       <sev>
>         <owner-key>...hex encode DH key...</owner-key>
> 	<session-info>..hex encode info...</session-info>
> 	<policy>...int32 value..</policy>
>       </sev>
>
>
>      if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
>      libvirt would report an error and refuse to start the guest
>
>   3. Libvirt generates the QEMU cli arg to enable SEV using
>      the XML data and starts QEMU, leaving CPUs paused
>
>   4. QEMU emits a SEV_MEASURE event containing the measurement
>      blob

Speaking of which, I expect QEMU to have a QMP command to retrieve the
measurement, in which case I think libvirt has to provide an API for the user
to retrieve the measurement in case libvirtd crashes somewhere between setting
up QEMU and waiting for the measurement event from QEMU, or simply because the
GO missed the event for some unspecified reason.

>
>   5. Libvirt catches the QEMU event and emits its own
>      VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
>      the measurement blob
>
>   6. GO does its validation of the measurement
>
>   7a  If validation failed, then virDomainDestroy() to stop QEMU
>
>   7b  If validation succeeed
>
>      Optionally call
>
>          virDomainSetSEVSecret()

Given the fact that we're likely introducing a new <sev> element to the XML
config, I'm more inclined to utilizing the existing virSecret interfaces (as
was originally suggested) instead of creating a vendor-specific API. You could
have an optional secret sub-element within the <sev> element and libvirt would
simply check if that secret has a value set, once the GO issues
virDomainResume(). Any particular reason for having a specific API for this that
I'm missing?

Other than that I like the initial proposal for the design.

Erik

Re: [Qemu-devel] [libvirt] libvirt/QEMU/SEV interaction

[Daniel P. Berrange]

On Mon, Sep 18, 2017 at 11:43:57AM +0200, Erik Skultety wrote:
> [...]
> 
> > >
> > >     > c) what existing communicate interface can be used between libvirt and qemu
> > >     > to get the measurement ? can we add a new qemu monitor command
> > >     > 'get_sev_measurement' to get the measurement ? (step 10)
> > >
> > >     Yes, QMP commands seeem most likely.
> > >
> > >     > d) how to pass the secret blob from libvirt to qemu ? should we consider
> > >     > adding a new object (sev-guest-secret) -- libvirt can add the object through
> > >     > qemu monitor.
> > >
> > >     Yeah, that looks like a viable option too.
> >
> > So I could see a flow like the following:
> >
> >
> >   1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
> >      document that includes the following
> >
> >       <host>
> >          ...other bits...
> >         <sev>
> > 	  <platform-key>...hex encoded PDH key...</platform-key>
> > 	</sev>
> >       </host>
> >
> >   2. mgmt tool requests to start a guest calling virCreateXML(),
> >      passing VIR_DOMAIN_START_PAUSED. The XML would include
> >
> >       <sev>
> >         <owner-key>...hex encode DH key...</owner-key>
> > 	<session-info>..hex encode info...</session-info>
> > 	<policy>...int32 value..</policy>
> >       </sev>
> >
> >
> >      if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
> >      libvirt would report an error and refuse to start the guest
> >
> >   3. Libvirt generates the QEMU cli arg to enable SEV using
> >      the XML data and starts QEMU, leaving CPUs paused
> >
> >   4. QEMU emits a SEV_MEASURE event containing the measurement
> >      blob
> 
> Speaking of which, I expect QEMU to have a QMP command to retrieve the
> measurement, in which case I think libvirt has to provide an API for the user
> to retrieve the measurement in case libvirtd crashes somewhere between setting
> up QEMU and waiting for the measurement event from QEMU, or simply because the
> GO missed the event for some unspecified reason.

Yeah, that's a good point - we also ought to have a pause-reason that
reflects that it is paused due to waiting for SEV secrets.

> 
> >
> >   5. Libvirt catches the QEMU event and emits its own
> >      VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
> >      the measurement blob
> >
> >   6. GO does its validation of the measurement
> >
> >   7a  If validation failed, then virDomainDestroy() to stop QEMU
> >
> >   7b  If validation succeeed
> >
> >      Optionally call
> >
> >          virDomainSetSEVSecret()
> 
> Given the fact that we're likely introducing a new <sev> element to the XML
> config, I'm more inclined to utilizing the existing virSecret interfaces (as
> was originally suggested) instead of creating a vendor-specific API. You could
> have an optional secret sub-element within the <sev> element and libvirt would
> simply check if that secret has a value set, once the GO issues
> virDomainResume(). Any particular reason for having a specific API for this that
> I'm missing?

Initially I was intending to suggest extensive use of virSecret, but it
turns out that despite being called a "secret", none of the SEV data we are
passing around needs protection. Either it is safe to be public, or it is
already encrypted.  So essentially we just have some data blobs we need to
pass into QEMU. I didn't feel we ought to be abusing virSecret as a
general purpose mechanism for passing in opaque data blobs which do not
need any kind of protection.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [libvirt] libvirt/QEMU/SEV interaction

[Richard Relph]

On 9/18/17 4:47 AM, Daniel P. Berrange wrote:
> On Mon, Sep 18, 2017 at 11:43:57AM +0200, Erik Skultety wrote:
>> [...]
>>
>>>>      > c) what existing communicate interface can be used between libvirt and qemu
>>>>      > to get the measurement ? can we add a new qemu monitor command
>>>>      > 'get_sev_measurement' to get the measurement ? (step 10)
>>>>
>>>>      Yes, QMP commands seeem most likely.
>>>>
>>>>      > d) how to pass the secret blob from libvirt to qemu ? should we consider
>>>>      > adding a new object (sev-guest-secret) -- libvirt can add the object through
>>>>      > qemu monitor.
>>>>
>>>>      Yeah, that looks like a viable option too.
>>> So I could see a flow like the following:
>>>
>>>
>>>    1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
>>>       document that includes the following
>>>
>>>        <host>
>>>           ...other bits...
>>>          <sev>
>>> 	  <platform-key>...hex encoded PDH key...</platform-key>
>>> 	</sev>
>>>        </host>
>>>
>>>    2. mgmt tool requests to start a guest calling virCreateXML(),
>>>       passing VIR_DOMAIN_START_PAUSED. The XML would include
>>>
>>>        <sev>
>>>          <owner-key>...hex encode DH key...</owner-key>
>>> 	<session-info>..hex encode info...</session-info>
>>> 	<policy>...int32 value..</policy>
>>>        </sev>
>>>
>>>
>>>       if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
>>>       libvirt would report an error and refuse to start the guest
For ease of use, I would not add this conditional to libvirt. If <sev> 
is provided and VIR_DOMAIN_START_PAUSED is missing, I’d just send the 
"GO" command as it would naturally occur.
Unless that would confuse things inside libvirt or QEMU in relation to 
the measurement and secret…
Many of our existing tests focus on other aspects of SEV functionality 
and so they skip the MEASURE/SECRET phase of launch and just go 
immediately from LAUNCH_UPDATE_DATA (or VMSA) to LAUNCH_FINISH.
I guess the key question will be how will QEMU know when to get the 
MEASUREMENT and wait for a LAUNCH_SECRET before doing a LAUNCH_FINISH 
when connected to libvirt.
Brijesh, this is your area. It feels to me like QEMU will have to wait 
to do the LAUNCH_FINISH until it gets the first “go” from libvirt. If 
that’s right, and assuming the same “go” comes from libvirt with or 
without VIR_DOMAIN_START_PAUSED, then I’d simply exclude the conditional 
check. QEMU would get the measurement when it is done sending the data.
Though in “real world” uses, I think the conditional is perfectly OK.
>>>
>>>    3. Libvirt generates the QEMU cli arg to enable SEV using
>>>       the XML data and starts QEMU, leaving CPUs paused
>>>
>>>    4. QEMU emits a SEV_MEASURE event containing the measurement
>>>       blob
>> Speaking of which, I expect QEMU to have a QMP command to retrieve the
>> measurement, in which case I think libvirt has to provide an API for the user
>> to retrieve the measurement in case libvirtd crashes somewhere between setting
>> up QEMU and waiting for the measurement event from QEMU, or simply because the
>> GO missed the event for some unspecified reason.
> Yeah, that's a good point - we also ought to have a pause-reason that
> reflects that it is paused due to waiting for SEV secrets.
>
>>>    5. Libvirt catches the QEMU event and emits its own
>>>       VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
>>>       the measurement blob
>>>
>>>    6. GO does its validation of the measurement
>>>
>>>    7a  If validation failed, then virDomainDestroy() to stop QEMU
>>>
>>>    7b  If validation succeeed
>>>
>>>       Optionally call
>>>
>>>           virDomainSetSEVSecret()
>> Given the fact that we're likely introducing a new <sev> element to the XML
>> config, I'm more inclined to utilizing the existing virSecret interfaces (as
>> was originally suggested) instead of creating a vendor-specific API. You could
>> have an optional secret sub-element within the <sev> element and libvirt would
>> simply check if that secret has a value set, once the GO issues
>> virDomainResume(). Any particular reason for having a specific API for this that
>> I'm missing?
> Initially I was intending to suggest extensive use of virSecret, but it
> turns out that despite being called a "secret", none of the SEV data we are
> passing around needs protection. Either it is safe to be public, or it is
> already encrypted.  So essentially we just have some data blobs we need to
> pass into QEMU. I didn't feel we ought to be abusing virSecret as a
> general purpose mechanism for passing in opaque data blobs which do not
> need any kind of protection.
All of the above looks really good to me.
While I agree with Daniel’s analysis of the need for “secret”, I do like 
using virSecret to convey the notion of secrecy. But it isn’t necessary. 
The end points are the SEV FW and the guest owner and all secrets they 
share are already encrypted. Embedding it in the “GO” command feels 
equally OK to me.
Note that sending a secret with a “GO” other than the first one is an 
error… I don’t think libvirt needs to catch that, though. The SEV FW will.

> Regards,
> Daniel
Thanks,
Richard

Re: [Qemu-devel] [libvirt] libvirt/QEMU/SEV interaction

[Erik Skultety]

On Mon, Sep 18, 2017 at 07:41:09AM -0500, Richard Relph wrote:
>
>
> On 9/18/17 4:47 AM, Daniel P. Berrange wrote:
> > On Mon, Sep 18, 2017 at 11:43:57AM +0200, Erik Skultety wrote:
> > > [...]
> > >
> > > > >      > c) what existing communicate interface can be used between libvirt and qemu
> > > > >      > to get the measurement ? can we add a new qemu monitor command
> > > > >      > 'get_sev_measurement' to get the measurement ? (step 10)
> > > > >
> > > > >      Yes, QMP commands seeem most likely.
> > > > >
> > > > >      > d) how to pass the secret blob from libvirt to qemu ? should we consider
> > > > >      > adding a new object (sev-guest-secret) -- libvirt can add the object through
> > > > >      > qemu monitor.
> > > > >
> > > > >      Yeah, that looks like a viable option too.
> > > > So I could see a flow like the following:
> > > >
> > > >
> > > >    1. mgmt tool calls  virConnectGetCapabilities. This returns an XML
> > > >       document that includes the following
> > > >
> > > >        <host>
> > > >           ...other bits...
> > > >          <sev>
> > > > 	  <platform-key>...hex encoded PDH key...</platform-key>
> > > > 	</sev>
> > > >        </host>
> > > >
> > > >    2. mgmt tool requests to start a guest calling virCreateXML(),
> > > >       passing VIR_DOMAIN_START_PAUSED. The XML would include
> > > >
> > > >        <sev>
> > > >          <owner-key>...hex encode DH key...</owner-key>
> > > > 	<session-info>..hex encode info...</session-info>
> > > > 	<policy>...int32 value..</policy>
> > > >        </sev>
> > > >
> > > >
> > > >       if <sev> is provided and VIR_DOMAIN_START_PAUSED is missing,
> > > >       libvirt would report an error and refuse to start the guest
> For ease of use, I would not add this conditional to libvirt. If <sev> is
> provided and VIR_DOMAIN_START_PAUSED is missing, I’d just send the "GO"

I also feel that the presence of the <sev> element might determine the usage of
the VIR_DOMAIN_START_PAUSED flag implicitly.

> command as it would naturally occur.
> Unless that would confuse things inside libvirt or QEMU in relation to the
> measurement and secret…
> Many of our existing tests focus on other aspects of SEV functionality and
> so they skip the MEASURE/SECRET phase of launch and just go immediately from
> LAUNCH_UPDATE_DATA (or VMSA) to LAUNCH_FINISH.
> I guess the key question will be how will QEMU know when to get the
> MEASUREMENT and wait for a LAUNCH_SECRET before doing a LAUNCH_FINISH when
> connected to libvirt.
> Brijesh, this is your area. It feels to me like QEMU will have to wait to do
> the LAUNCH_FINISH until it gets the first “go” from libvirt. If that’s
> right, and assuming the same “go” comes from libvirt with or without
> VIR_DOMAIN_START_PAUSED, then I’d simply exclude the conditional check. QEMU
> would get the measurement when it is done sending the data.
> Though in “real world” uses, I think the conditional is perfectly OK.
> > > >
> > > >    3. Libvirt generates the QEMU cli arg to enable SEV using
> > > >       the XML data and starts QEMU, leaving CPUs paused
> > > >
> > > >    4. QEMU emits a SEV_MEASURE event containing the measurement
> > > >       blob
> > > Speaking of which, I expect QEMU to have a QMP command to retrieve the
> > > measurement, in which case I think libvirt has to provide an API for the user
> > > to retrieve the measurement in case libvirtd crashes somewhere between setting
> > > up QEMU and waiting for the measurement event from QEMU, or simply because the
> > > GO missed the event for some unspecified reason.
> > Yeah, that's a good point - we also ought to have a pause-reason that
> > reflects that it is paused due to waiting for SEV secrets.
> >
> > > >    5. Libvirt catches the QEMU event and emits its own
> > > >       VIR_CONNECT_DOMAIN_EVENT_SEV_MEASURE event containing
> > > >       the measurement blob
> > > >
> > > >    6. GO does its validation of the measurement
> > > >
> > > >    7a  If validation failed, then virDomainDestroy() to stop QEMU
> > > >
> > > >    7b  If validation succeeed
> > > >
> > > >       Optionally call
> > > >
> > > >           virDomainSetSEVSecret()
> > > Given the fact that we're likely introducing a new <sev> element to the XML
> > > config, I'm more inclined to utilizing the existing virSecret interfaces (as
> > > was originally suggested) instead of creating a vendor-specific API. You could
> > > have an optional secret sub-element within the <sev> element and libvirt would
> > > simply check if that secret has a value set, once the GO issues
> > > virDomainResume(). Any particular reason for having a specific API for this that
> > > I'm missing?
> > Initially I was intending to suggest extensive use of virSecret, but it
> > turns out that despite being called a "secret", none of the SEV data we are
> > passing around needs protection. Either it is safe to be public, or it is
> > already encrypted.  So essentially we just have some data blobs we need to
> > pass into QEMU. I didn't feel we ought to be abusing virSecret as a
> > general purpose mechanism for passing in opaque data blobs which do not
> > need any kind of protection.
> All of the above looks really good to me.
> While I agree with Daniel’s analysis of the need for “secret”, I do like
> using virSecret to convey the notion of secrecy. But it isn’t necessary. The
> end points are the SEV FW and the guest owner and all secrets they share are
> already encrypted. Embedding it in the “GO” command feels equally OK to me.
> Note that sending a secret with a “GO” other than the first one is an error…
> I don’t think libvirt needs to catch that, though. The SEV FW will.

Setting a secret and resuming a domain are two separate actions, as resuming a
domain doesn't take any arbitrary data (ambiguity of the opaque data).
This would especially be an issue if we allowed multiple secrets to be passed to
the guest. Therefore such a check is transparent to libvirt, as libvirt can't
reliably tell how many secrets are there necessary to be passed to the guest
(only guest owner knows that) - libvirt would only fail setting the same secret
multiple times with QEMU error which in turn would get it from SEV FW).

Additionally, virDomainResume() can be called multiple times on the same domain
safely, since the domain to be resumed is going to be in the 'running' in all
but the first time you call the function.

Erik

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
> Hi All,

Sorry if below comment doesn't make sense, I might be misunderstanding
something basic about SEV. Also sorry about the delay, I've been on
vacation.


> (sorry for the long message)
> 
> CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
> feature - the feature allows running encrypted VMs. To enable the feature,
> I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
> We have been making some good progress in getting patches accepted upstream
> in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
> feature -- SME support just got pulled into 4.14 merge window. The base
> SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
> I am getting ready to take off "RFC" tag from remaining patches to get them
> reviewed and accepted.
> 
> The boot flow for launching an SEV guest is a bit different from a typical
> guest launch. In order to launch SEV guest from virt-manager or other
> high-level VM management tools, we need to design and implement new
> interface between libvirt and qemu, and probably add new APIs in libvirt
> to be used by VM management tools. I am new to the libvirt and need some
> expert advice while designing this interface. A pictorial representation
> for a SEV guest launch flow is available in SEV Spec Appendix A [4].
> 
> A typical flow looks like this:
> 
> 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
> 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
> 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
>    caller.
> 4. VM tool gives its PDH to GO.
> 5. GO provides its DH key, session-info and guest policy.
> 6. VM tool somehow communicates the GO provided information to libvirt.
> 7. libvirt adds "sev-guest" object in its xml file with all the information
>    obtained from #5
> 
>    (currently my xml file looks like this)
> 
>    <qemu:arg value='-object'>
>    <qemu:arg
> value='sev-guest,id=sev0,policy=<GO_policy>,dh-key-file=<filename>,session-file=<filename>/>
>    <qemu:arg value='-machine'/>
>    <qemu:arg value='memory-encryption=sev0'/>
> 
> 8. libvirt launches the guest with "-S"
> 9. While creating the SEV guest qemu does the following
>  i) create encryption context using GO's DH, session-info and guest policy
>     (LAUNCH_START)
>  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
>  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement

This part troubles me. This seems to mean that the guest being launched
must know what the measurement of the bios is going to be.  This means
that the cloud provider can not update the bios without breaking guests.
Also, while in practice you typically can run an old bios image on a new
qemu instance, this is not really tested so would be very hard to
support properly in QEMU.


And this looks like a fundamental problem with the hash based
measurement that's in hardware. So below I suggest that we layer
some software on top to rely on the hash as little as possible.



> 10. By some interface we must propagate the measurement all the way to GO
>   before libvirt starts the guest.
> 11. GO verifies the measurement and if measurement matches then it may
>  give a secret blob -- which must be injected into the guest before
>  libvirt starts the VM. If verification failed, GO will request cloud
>  provider to destroy the VM.
> 12. After secret blob is injected into guest, we call LAUNCH_FINISH
>   to destory the encryption context.
> 13. libvirt issues "continue" command to resume the guest boot.
> 
> Please note that the measurement value is protected with transport
> encryption key (TIK) and it changes on each run. Similarly the secret blob
> provided by GO does not need to be protected using libvirt/qemu APIs. The
> secret is protected by TIK. From qemu and libvirt point of view these are
> blobs and must be passed as-is to the SEV FW.

So here's an alternative idea for starting guests:

How about building a minimal shim firmware that
runs on a single CPU and uses no hardware at all,
it just contains the secret blob.

That firmware just immediately stops and signals
hypervisor that it is ready to be run in the cloud.

Have user generate and start this shim firmware as a guest in a private
setup, then export it out using SEND_* commands.

Then instead of asking to launch guest, you ask provider
to load it with RECEIVE_* commands.

Unlike bios the shim firmware
can hopefully be static so supporting it across qemu
versions should be easy.

The shim firmware then loads bios from qemu, verifies
it in any way it sees fit (e.g. it could check a signature, version, etc:
it is not limited to a hardware hash anymore).
It then jumps to the bios.


While not exactly the same, there is some similarity
here with how people solved the issues around secureboot -
by using a shim.

Thanks!


> Questions:
> a) Do we need to add a new set of APIs in libvirt to return the PDH from
> libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
> opaque blobs ? (this is mainly for step 3 and 6)
> b) do we need to define a new xml tag to for memory-encryption ? or just
> use the qemu:args tag ? (step 6)
> c) what existing communicate interface can be used between libvirt and qemu
> to get the measurement ? can we add a new qemu monitor command
> 'get_sev_measurement' to get the measurement ? (step 10)
> d) how to pass the secret blob from libvirt to qemu ? should we consider
> adding a new object (sev-guest-secret) -- libvirt can add the object through
> qemu monitor.
> 
> 
> [1] https://marc.info/?l=kvm&m=150092661105069&w=2
> [2] https://marc.info/?l=qemu-devel&m=148901186615642&w=2
> [3] https://lists.01.org/pipermail/edk2-devel/2017-July/012220.html
> [4] http://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf
> 
> Thanks
> 
> Brijesh
> 

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Dr. David Alan Gilbert]

* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
> > Hi All,
> 
> Sorry if below comment doesn't make sense, I might be misunderstanding
> something basic about SEV. Also sorry about the delay, I've been on
> vacation.
> 
> 
> > (sorry for the long message)
> > 
> > CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
> > feature - the feature allows running encrypted VMs. To enable the feature,
> > I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
> > We have been making some good progress in getting patches accepted upstream
> > in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
> > feature -- SME support just got pulled into 4.14 merge window. The base
> > SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
> > I am getting ready to take off "RFC" tag from remaining patches to get them
> > reviewed and accepted.
> > 
> > The boot flow for launching an SEV guest is a bit different from a typical
> > guest launch. In order to launch SEV guest from virt-manager or other
> > high-level VM management tools, we need to design and implement new
> > interface between libvirt and qemu, and probably add new APIs in libvirt
> > to be used by VM management tools. I am new to the libvirt and need some
> > expert advice while designing this interface. A pictorial representation
> > for a SEV guest launch flow is available in SEV Spec Appendix A [4].
> > 
> > A typical flow looks like this:
> > 
> > 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
> > 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
> > 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
> >    caller.
> > 4. VM tool gives its PDH to GO.
> > 5. GO provides its DH key, session-info and guest policy.
> > 6. VM tool somehow communicates the GO provided information to libvirt.
> > 7. libvirt adds "sev-guest" object in its xml file with all the information
> >    obtained from #5
> > 
> >    (currently my xml file looks like this)
> > 
> >    <qemu:arg value='-object'>
> >    <qemu:arg
> > value='sev-guest,id=sev0,policy=<GO_policy>,dh-key-file=<filename>,session-file=<filename>/>
> >    <qemu:arg value='-machine'/>
> >    <qemu:arg value='memory-encryption=sev0'/>
> > 
> > 8. libvirt launches the guest with "-S"
> > 9. While creating the SEV guest qemu does the following
> >  i) create encryption context using GO's DH, session-info and guest policy
> >     (LAUNCH_START)
> >  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
> >  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
> 
> This part troubles me. This seems to mean that the guest being launched
> must know what the measurement of the bios is going to be.  This means
> that the cloud provider can not update the bios without breaking guests.
> Also, while in practice you typically can run an old bios image on a new
> qemu instance, this is not really tested so would be very hard to
> support properly in QEMU.
> 
> 
> And this looks like a fundamental problem with the hash based
> measurement that's in hardware. So below I suggest that we layer
> some software on top to rely on the hash as little as possible.

I think the normal way to solve this is that the Distro would
provide a list of the signatures, and the GO would check the bios
measurement against that list.  The GO needs to keep that list
up to date; either directly downloading it from the Distro or
a copy signed by the Distro.

Dave

> 
> > 10. By some interface we must propagate the measurement all the way to GO
> >   before libvirt starts the guest.
> > 11. GO verifies the measurement and if measurement matches then it may
> >  give a secret blob -- which must be injected into the guest before
> >  libvirt starts the VM. If verification failed, GO will request cloud
> >  provider to destroy the VM.
> > 12. After secret blob is injected into guest, we call LAUNCH_FINISH
> >   to destory the encryption context.
> > 13. libvirt issues "continue" command to resume the guest boot.
> > 
> > Please note that the measurement value is protected with transport
> > encryption key (TIK) and it changes on each run. Similarly the secret blob
> > provided by GO does not need to be protected using libvirt/qemu APIs. The
> > secret is protected by TIK. From qemu and libvirt point of view these are
> > blobs and must be passed as-is to the SEV FW.
> 
> So here's an alternative idea for starting guests:
> 
> How about building a minimal shim firmware that
> runs on a single CPU and uses no hardware at all,
> it just contains the secret blob.
> 
> That firmware just immediately stops and signals
> hypervisor that it is ready to be run in the cloud.
> 
> Have user generate and start this shim firmware as a guest in a private
> setup, then export it out using SEND_* commands.
> 
> Then instead of asking to launch guest, you ask provider
> to load it with RECEIVE_* commands.
> 
> Unlike bios the shim firmware
> can hopefully be static so supporting it across qemu
> versions should be easy.
> 
> The shim firmware then loads bios from qemu, verifies
> it in any way it sees fit (e.g. it could check a signature, version, etc:
> it is not limited to a hardware hash anymore).
> It then jumps to the bios.
> 
> 
> While not exactly the same, there is some similarity
> here with how people solved the issues around secureboot -
> by using a shim.
> 
> Thanks!
> 
> 
> > Questions:
> > a) Do we need to add a new set of APIs in libvirt to return the PDH from
> > libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
> > opaque blobs ? (this is mainly for step 3 and 6)
> > b) do we need to define a new xml tag to for memory-encryption ? or just
> > use the qemu:args tag ? (step 6)
> > c) what existing communicate interface can be used between libvirt and qemu
> > to get the measurement ? can we add a new qemu monitor command
> > 'get_sev_measurement' to get the measurement ? (step 10)
> > d) how to pass the secret blob from libvirt to qemu ? should we consider
> > adding a new object (sev-guest-secret) -- libvirt can add the object through
> > qemu monitor.
> > 
> > 
> > [1] https://marc.info/?l=kvm&m=150092661105069&w=2
> > [2] https://marc.info/?l=qemu-devel&m=148901186615642&w=2
> > [3] https://lists.01.org/pipermail/edk2-devel/2017-July/012220.html
> > [4] http://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf
> > 
> > Thanks
> > 
> > Brijesh
> > 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Brijesh Singh]

Hi Michael,


On 09/26/2017 09:36 AM, Michael S. Tsirkin wrote:

...

>> 8. libvirt launches the guest with "-S"
>> 9. While creating the SEV guest qemu does the following
>>   i) create encryption context using GO's DH, session-info and guest policy
>>      (LAUNCH_START)
>>   ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
>>   iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
> 
> This part troubles me. This seems to mean that the guest being launched
> must know what the measurement of the bios is going to be.  This means
> that the cloud provider can not update the bios without breaking guests.
> Also, while in practice you typically can run an old bios image on a new
> qemu instance, this is not really tested so would be very hard to
> support properly in QEMU.
> 
> 


The guest itself does not need to know the measurement of the bios --
the SEV launch flow empowers the GO to validate the bootstrap code (bios)
before GO can provide a confidential information to the guest. Please note
that the validating of the measurement flow is optional. GO can ask cloud
provider to ignore the measurement all together and boot the SEV guest.

The measurement flow can be useful when GO decides to provide a custom
bios and want to know that his bios is used for booting the guest. In this
case, since the guest owner knows the initial contents of the guest at boot,
he can request the measurement from the cloud provider and compare it with
what the guest owner expects.



> And this looks like a fundamental problem with the hash based
> measurement that's in hardware. So below I suggest that we layer
> some software on top to rely on the hash as little as possible.
> 
> 
> 
>> 10. By some interface we must propagate the measurement all the way to GO
>>    before libvirt starts the guest.
>> 11. GO verifies the measurement and if measurement matches then it may
>>   give a secret blob -- which must be injected into the guest before
>>   libvirt starts the VM. If verification failed, GO will request cloud
>>   provider to destroy the VM.
>> 12. After secret blob is injected into guest, we call LAUNCH_FINISH
>>    to destory the encryption context.
>> 13. libvirt issues "continue" command to resume the guest boot.
>>
>> Please note that the measurement value is protected with transport
>> encryption key (TIK) and it changes on each run. Similarly the secret blob
>> provided by GO does not need to be protected using libvirt/qemu APIs. The
>> secret is protected by TIK. From qemu and libvirt point of view these are
>> blobs and must be passed as-is to the SEV FW.
> 
> So here's an alternative idea for starting guests:
> 
> How about building a minimal shim firmware that
> runs on a single CPU and uses no hardware at all,
> it just contains the secret blob.
> 
> That firmware just immediately stops and signals
> hypervisor that it is ready to be run in the cloud.
> 
> Have user generate and start this shim firmware as a guest in a private
> setup, then export it out using SEND_* commands.
> 
> Then instead of asking to launch guest, you ask provider
> to load it with RECEIVE_* commands.
> 
> Unlike bios the shim firmware
> can hopefully be static so supporting it across qemu
> versions should be easy.
> 
> The shim firmware then loads bios from qemu, verifies
> it in any way it sees fit (e.g. it could check a signature, version, etc:
> it is not limited to a hardware hash anymore).
> It then jumps to the bios.
> 
> 
> While not exactly the same, there is some similarity
> here with how people solved the issues around secureboot -
> by using a shim.
> 

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Wed, Sep 27, 2017 at 08:39:24AM -0500, Brijesh Singh wrote:
> Hi Michael,
> 
> 
> On 09/26/2017 09:36 AM, Michael S. Tsirkin wrote:
> 
> ...
> 
> > > 8. libvirt launches the guest with "-S"
> > > 9. While creating the SEV guest qemu does the following
> > >   i) create encryption context using GO's DH, session-info and guest policy
> > >      (LAUNCH_START)
> > >   ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
> > >   iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
> > 
> > This part troubles me. This seems to mean that the guest being launched
> > must know what the measurement of the bios is going to be.  This means
> > that the cloud provider can not update the bios without breaking guests.
> > Also, while in practice you typically can run an old bios image on a new
> > qemu instance, this is not really tested so would be very hard to
> > support properly in QEMU.
> > 
> > 
> 
> 
> The guest itself does not need to know the measurement of the bios --
> the SEV launch flow empowers the GO to validate the bootstrap code (bios)
> before GO can provide a confidential information to the guest. Please note
> that the validating of the measurement flow is optional. GO can ask cloud
> provider to ignore the measurement all together and boot the SEV guest.

As the OS runs on top of the bios, it does not seem prudent to boot the
SEV guest without a way to verify that the bios is safe to use.
Linux generally trusts the firmware it runs on. Are you looking for
examples of how a malicious firmware can leak info out of the guest?

> The measurement flow can be useful when GO decides to provide a custom
> bios and want to know that his bios is used for booting the guest. In this
> case, since the guest owner knows the initial contents of the guest at boot,
> he can request the measurement from the cloud provider and compare it with
> what the guest owner expects.

I agree the measurement works for this, but If that's the only case, I'd
say it's not all that interesting since most people will use a standard
bios.

I do think something needs to vaidate the bios though, and I do not
think naively measuring the hash of the full bios is a way to do this
that we can support well long term.

> 
> 
> > And this looks like a fundamental problem with the hash based
> > measurement that's in hardware. So below I suggest that we layer
> > some software on top to rely on the hash as little as possible.
> > 
> > 
> > 
> > > 10. By some interface we must propagate the measurement all the way to GO
> > >    before libvirt starts the guest.
> > > 11. GO verifies the measurement and if measurement matches then it may
> > >   give a secret blob -- which must be injected into the guest before
> > >   libvirt starts the VM. If verification failed, GO will request cloud
> > >   provider to destroy the VM.
> > > 12. After secret blob is injected into guest, we call LAUNCH_FINISH
> > >    to destory the encryption context.
> > > 13. libvirt issues "continue" command to resume the guest boot.
> > > 
> > > Please note that the measurement value is protected with transport
> > > encryption key (TIK) and it changes on each run. Similarly the secret blob
> > > provided by GO does not need to be protected using libvirt/qemu APIs. The
> > > secret is protected by TIK. From qemu and libvirt point of view these are
> > > blobs and must be passed as-is to the SEV FW.
> > 
> > So here's an alternative idea for starting guests:
> > 
> > How about building a minimal shim firmware that
> > runs on a single CPU and uses no hardware at all,
> > it just contains the secret blob.
> > 
> > That firmware just immediately stops and signals
> > hypervisor that it is ready to be run in the cloud.
> > 
> > Have user generate and start this shim firmware as a guest in a private
> > setup, then export it out using SEND_* commands.
> > 
> > Then instead of asking to launch guest, you ask provider
> > to load it with RECEIVE_* commands.
> > 
> > Unlike bios the shim firmware
> > can hopefully be static so supporting it across qemu
> > versions should be easy.
> > 
> > The shim firmware then loads bios from qemu, verifies
> > it in any way it sees fit (e.g. it could check a signature, version, etc:
> > it is not limited to a hardware hash anymore).
> > It then jumps to the bios.
> > 
> > 
> > While not exactly the same, there is some similarity
> > here with how people solved the issues around secureboot -
> > by using a shim.
> > 

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Richard Relph]

Forgive the top post... some of the conversation has been trimmed, but I 
need to go back to first principles of SEV in order to make sure we all 
have a clear understanding of what the goal is.

The goal - for BOTH guest owner and cloud provider - is to get to a VM 
where ONLY the guest owner (GO) has access to the GO "secrets". 
Legitimate cloud providers (ie, those that wish to not retain a back 
door in to their customer's VMs) want this every bit as much as GOs want 
it. It is this privacy concern that some believe holds back broader 
adoption of public cloud for sensitive applications.

To provide this additional privacy will require some changes to the 
"untrustable" model that seems to be in place now. There is value for 
everyone in creating a "trustable" model.

Given that, the root of the problem for the GO is trust. How can the GO 
know that every instruction in their VM is "theirs"? AMD Epyc CPUs 
decrypt every instruction fetch (and guest page table walk) in an SEV 
guest VM with that VM's random memory encryption key. (Data can either 
be encrypted or not, at the guest's choosing.) Only the SEV FW and the 
guest itself can encrypt memory with that key. The SEV FW measures every 
byte it encrypts for the guest and provides that measurement to the GO. 
The GO is free to ignore the value and run the guest "as-is". But 
recommended practice will be to inspect the measurement, verify it, and 
only then provide the guest VM with "secrets" necessary to decrypt 
disks, connect to privileged network resources, etc.

As has been observed, the BIOS has a great deal of power. It is 
impossible to maintain the GO's privacy in a VM where the BIOS (or any 
other code, for that matter) is "unknown". It simply is a violation of 
the trust model both the GO and the CP want to have to allow unknown 
code from the CP to enter the GO's VM. (Yes, there are LOTS of other 
ways for untrusted code to get in and secrets to get out... we're only 
trying to close this door between the CP and the guest VM at this time. ;-)

I anticipate that legitimate cloud providers will be happy (or at least 
willing) to share with customers the source for their BIOS. The GO can 
inspect the source, build the binary from that source, and generate the 
required hash. Or they may just trust that someone else has done that 
work and accept the hash the CP posts on their BIOS image. (Note that 
when the hash is returned by the SEV FW, it is in HMAC form, with a 
nonce that the GO can compute, and a key the GO provided at launch time.)

Whether the "BIOS" is a "static shim" as Michael suggests, or a full 
BIOS, or even a BIOS+kernel+initrd is really not too significant. What 
is significant is that the GO has a basis for trusting all code that is 
imported in to their VM by the CP. And that NONE of the code provided by 
the CP is "unknown" and unauditable by the GO. If the CP has a way to 
inject code unknown to the GO in to the guest VM, the trust model is 
broken and both GO and CP suffer the consequences.

When the CP needs to update the BIOS image, they will have to inform the 
GO and allow the GO to establish trust in the CP's new BIOS image somehow.

I hope that helps outline what we're doing, and why.

Richard


On 9/27/17 11:12 AM, Michael S. Tsirkin wrote:
> On Wed, Sep 27, 2017 at 08:39:24AM -0500, Brijesh Singh wrote:
>> Hi Michael,
>>
>>
>> On 09/26/2017 09:36 AM, Michael S. Tsirkin wrote:
>>
>> ...
>>
>>>> 8. libvirt launches the guest with "-S"
>>>> 9. While creating the SEV guest qemu does the following
>>>>    i) create encryption context using GO's DH, session-info and guest policy
>>>>       (LAUNCH_START)
>>>>    ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
>>>>    iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
>>>
>>> This part troubles me. This seems to mean that the guest being launched
>>> must know what the measurement of the bios is going to be.  This means
>>> that the cloud provider can not update the bios without breaking guests.
>>> Also, while in practice you typically can run an old bios image on a new
>>> qemu instance, this is not really tested so would be very hard to
>>> support properly in QEMU.
>>>
>>>
>>
>>
>> The guest itself does not need to know the measurement of the bios --
>> the SEV launch flow empowers the GO to validate the bootstrap code (bios)
>> before GO can provide a confidential information to the guest. Please note
>> that the validating of the measurement flow is optional. GO can ask cloud
>> provider to ignore the measurement all together and boot the SEV guest.
> 
> As the OS runs on top of the bios, it does not seem prudent to boot the
> SEV guest without a way to verify that the bios is safe to use.
> Linux generally trusts the firmware it runs on. Are you looking for
> examples of how a malicious firmware can leak info out of the guest?
> 
>> The measurement flow can be useful when GO decides to provide a custom
>> bios and want to know that his bios is used for booting the guest. In this
>> case, since the guest owner knows the initial contents of the guest at boot,
>> he can request the measurement from the cloud provider and compare it with
>> what the guest owner expects.
> 
> I agree the measurement works for this, but If that's the only case, I'd
> say it's not all that interesting since most people will use a standard
> bios.
> 
> I do think something needs to vaidate the bios though, and I do not
> think naively measuring the hash of the full bios is a way to do this
> that we can support well long term.
> 
>>
>>
>>> And this looks like a fundamental problem with the hash based
>>> measurement that's in hardware. So below I suggest that we layer
>>> some software on top to rely on the hash as little as possible.
>>>
>>>
>>>
>>>> 10. By some interface we must propagate the measurement all the way to GO
>>>>     before libvirt starts the guest.
>>>> 11. GO verifies the measurement and if measurement matches then it may
>>>>    give a secret blob -- which must be injected into the guest before
>>>>    libvirt starts the VM. If verification failed, GO will request cloud
>>>>    provider to destroy the VM.
>>>> 12. After secret blob is injected into guest, we call LAUNCH_FINISH
>>>>     to destory the encryption context.
>>>> 13. libvirt issues "continue" command to resume the guest boot.
>>>>
>>>> Please note that the measurement value is protected with transport
>>>> encryption key (TIK) and it changes on each run. Similarly the secret blob
>>>> provided by GO does not need to be protected using libvirt/qemu APIs. The
>>>> secret is protected by TIK. From qemu and libvirt point of view these are
>>>> blobs and must be passed as-is to the SEV FW.
>>>
>>> So here's an alternative idea for starting guests:
>>>
>>> How about building a minimal shim firmware that
>>> runs on a single CPU and uses no hardware at all,
>>> it just contains the secret blob.
>>>
>>> That firmware just immediately stops and signals
>>> hypervisor that it is ready to be run in the cloud.
>>>
>>> Have user generate and start this shim firmware as a guest in a private
>>> setup, then export it out using SEND_* commands.
>>>
>>> Then instead of asking to launch guest, you ask provider
>>> to load it with RECEIVE_* commands.
>>>
>>> Unlike bios the shim firmware
>>> can hopefully be static so supporting it across qemu
>>> versions should be easy.
>>>
>>> The shim firmware then loads bios from qemu, verifies
>>> it in any way it sees fit (e.g. it could check a signature, version, etc:
>>> it is not limited to a hardware hash anymore).
>>> It then jumps to the bios.
>>>
>>>
>>> While not exactly the same, there is some similarity
>>> here with how people solved the issues around secureboot -
>>> by using a shim.
>>>

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
> Whether the "BIOS" is a "static shim" as Michael suggests, or a full BIOS,
> or even a BIOS+kernel+initrd is really not too significant. What is
> significant is that the GO has a basis for trusting all code that is
> imported in to their VM by the CP. And that NONE of the code provided by the
> CP is "unknown" and unauditable by the GO. If the CP has a way to inject
> code unknown to the GO in to the guest VM, the trust model is broken and
> both GO and CP suffer the consequences.

Absolutely.

> When the CP needs to update the BIOS image, they will have to inform the GO
> and allow the GO to establish trust in the CP's new BIOS image somehow.

This GO update on every BIOS change is imho is not a workable model. You
want something like checking the BIOS signature instead. And since
hardware is all hash based, you need the shim to do it in software.

-- 
MST

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Richard Relph]

On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
> On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
>> Whether the "BIOS" is a "static shim" as Michael suggests, or a full BIOS,
>> or even a BIOS+kernel+initrd is really not too significant. What is
>> significant is that the GO has a basis for trusting all code that is
>> imported in to their VM by the CP. And that NONE of the code provided by the
>> CP is "unknown" and unauditable by the GO. If the CP has a way to inject
>> code unknown to the GO in to the guest VM, the trust model is broken and
>> both GO and CP suffer the consequences.
> 
> Absolutely.
> 
>> When the CP needs to update the BIOS image, they will have to inform the GO
>> and allow the GO to establish trust in the CP's new BIOS image somehow.
> 
> This GO update on every BIOS change is imho is not a workable model. You
> want something like checking the BIOS signature instead. And since
> hardware is all hash based, you need the shim to do it in software.

A BIOS "signed" by the CP doesn't meet the security requirement. It is 
code that is "unknown" to the GO.

The (legitimate) CP does NOT want to be in that position of trust. If 
they are, then some government somewhere is going to insist that they 
sign a BIOS that allows the government to spy on the GO's VMs, and steal 
secrets from it. Or some hacker admin will do it "for fun".

How often do large public CPs really change their BIOSes? My sense is 
that large public CPs prefer stability over "latest and greatest".

And, perhaps more importantly, if a CP are able to sell a "more secure" 
VM, one that justifies a higher price per vCPU hour, wouldn't that 
warrant some changes in the "insecure" model being used today?

Richard

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Richard Relph]

On 9/29/17 2:48 PM, Richard Relph wrote:
> On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
>> On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
>>> Whether the "BIOS" is a "static shim" as Michael suggests, or a full 
>>> BIOS,
>>> or even a BIOS+kernel+initrd is really not too significant. What is
>>> significant is that the GO has a basis for trusting all code that is
>>> imported in to their VM by the CP. And that NONE of the code provided 
>>> by the
>>> CP is "unknown" and unauditable by the GO. If the CP has a way to inject
>>> code unknown to the GO in to the guest VM, the trust model is broken and
>>> both GO and CP suffer the consequences.
>>
>> Absolutely.
>>
>>> When the CP needs to update the BIOS image, they will have to inform 
>>> the GO
>>> and allow the GO to establish trust in the CP's new BIOS image somehow.
>>
>> This GO update on every BIOS change is imho is not a workable model. You
>> want something like checking the BIOS signature instead. And since
>> hardware is all hash based, you need the shim to do it in software.
> 
> A BIOS "signed" by the CP doesn't meet the security requirement. It is 
> code that is "unknown" to the GO.
> 
> The (legitimate) CP does NOT want to be in that position of trust. If 
> they are, then some government somewhere is going to insist that they 
> sign a BIOS that allows the government to spy on the GO's VMs, and steal 
> secrets from it. Or some hacker admin will do it "for fun".
> 
> How often do large public CPs really change their BIOSes? My sense is 
> that large public CPs prefer stability over "latest and greatest".
> 
> And, perhaps more importantly, if a CP are able to sell a "more secure" 
> VM, one that justifies a higher price per vCPU hour, wouldn't that 
> warrant some changes in the "insecure" model being used today?

Ultimately, I think both approaches are "doable". It will be a CP and GO 
decision. If the GO trusts the CP, the shim+signed BIOS will work fine. 
If GO requires a more secure VM and the CP wants to offer it, the CP 
will figure out a way to satisfy the GO's "trust issue" that the BIOS 
can't be used to circumvent SEV's protections. Depending on your level 
of paranoia, that may require advance notice of BIOS changes, or even 
allowing the GO to provide the BIOS themselves, written to a spec 
supported by the CP's HV, and/or based on BIOS code provided by the CP.

It's a business decision and I think SEV can support both. That said, 
AMD currently has no plans to write a shim that can verify the signature 
on a CP-provided BIOS image.

Richard

> 
> Richard

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Fri, Sep 29, 2017 at 02:48:45PM -0500, Richard Relph wrote:
> On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
> > On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
> > > Whether the "BIOS" is a "static shim" as Michael suggests, or a full BIOS,
> > > or even a BIOS+kernel+initrd is really not too significant. What is
> > > significant is that the GO has a basis for trusting all code that is
> > > imported in to their VM by the CP. And that NONE of the code provided by the
> > > CP is "unknown" and unauditable by the GO. If the CP has a way to inject
> > > code unknown to the GO in to the guest VM, the trust model is broken and
> > > both GO and CP suffer the consequences.
> > 
> > Absolutely.
> > 
> > > When the CP needs to update the BIOS image, they will have to inform the GO
> > > and allow the GO to establish trust in the CP's new BIOS image somehow.
> > 
> > This GO update on every BIOS change is imho is not a workable model. You
> > want something like checking the BIOS signature instead. And since
> > hardware is all hash based, you need the shim to do it in software.
> 
> A BIOS "signed" by the CP doesn't meet the security requirement. It is code
> that is "unknown" to the GO.

There is a misunderstanding here.

BIOS would not be signed by a CP. It would be signed by a trusted
software vendor e.g. by Red Hat.

> The (legitimate) CP does NOT want to be in that position of trust. If they
> are, then some government somewhere is going to insist that they sign a BIOS
> that allows the government to spy on the GO's VMs, and steal secrets from
> it. Or some hacker admin will do it "for fun".
> 
> How often do large public CPs really change their BIOSes? My sense is that
> large public CPs prefer stability over "latest and greatest".

CPs just do dnf update. Software vendors change BIOSes.

And we do change them. Look at number of revisions for seabios in e.g.
Fedora. More importantly we might need to change them quickly e.g.
because of a security issue. Adding the need to coordinate with all GOs
is not going to work. Neither can QEMU support booting old BIOS
versions on new machine types indefinitely.

> And, perhaps more importantly, if a CP are able to sell a "more secure" VM,
> one that justifies a higher price per vCPU hour, wouldn't that warrant some
> changes in the "insecure" model being used today?
> 
> Richard

Absolutely. CPs have no business signing images. But it is just not
really feasible for software vendors to distribute hashes.

-- 
MST

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Fri, Sep 29, 2017 at 03:07:40PM -0500, Richard Relph wrote:
> On 9/29/17 2:48 PM, Richard Relph wrote:
> > On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
> > > On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
> > > > Whether the "BIOS" is a "static shim" as Michael suggests, or a
> > > > full BIOS,
> > > > or even a BIOS+kernel+initrd is really not too significant. What is
> > > > significant is that the GO has a basis for trusting all code that is
> > > > imported in to their VM by the CP. And that NONE of the code
> > > > provided by the
> > > > CP is "unknown" and unauditable by the GO. If the CP has a way to inject
> > > > code unknown to the GO in to the guest VM, the trust model is broken and
> > > > both GO and CP suffer the consequences.
> > > 
> > > Absolutely.
> > > 
> > > > When the CP needs to update the BIOS image, they will have to
> > > > inform the GO
> > > > and allow the GO to establish trust in the CP's new BIOS image somehow.
> > > 
> > > This GO update on every BIOS change is imho is not a workable model. You
> > > want something like checking the BIOS signature instead. And since
> > > hardware is all hash based, you need the shim to do it in software.
> > 
> > A BIOS "signed" by the CP doesn't meet the security requirement. It is
> > code that is "unknown" to the GO.
> > 
> > The (legitimate) CP does NOT want to be in that position of trust. If
> > they are, then some government somewhere is going to insist that they
> > sign a BIOS that allows the government to spy on the GO's VMs, and steal
> > secrets from it. Or some hacker admin will do it "for fun".
> > 
> > How often do large public CPs really change their BIOSes? My sense is
> > that large public CPs prefer stability over "latest and greatest".
> > 
> > And, perhaps more importantly, if a CP are able to sell a "more secure"
> > VM, one that justifies a higher price per vCPU hour, wouldn't that
> > warrant some changes in the "insecure" model being used today?
> 
> Ultimately, I think both approaches are "doable". It will be a CP and GO
> decision. If the GO trusts the CP, the shim+signed BIOS will work fine.

I think there's a misunderstanding. A trusted software vendor would
sign the BIOS. GO would verify it. Trusting the CP is not required.

> If
> GO requires a more secure VM and the CP wants to offer it, the CP will
> figure out a way to satisfy the GO's "trust issue" that the BIOS can't be
> used to circumvent SEV's protections. Depending on your level of paranoia,
> that may require advance notice of BIOS changes, or even allowing the GO to
> provide the BIOS themselves, written to a spec supported by the CP's HV,
> and/or based on BIOS code provided by the CP.

We are discussing this on a qemu mailing list, aren't we? And from QEMU
point of view,  I think it won't be able to support a requirement
to boot ancient bios versions on new machine types indefinitely
with good performance and also fix security issues in them
in a timely manner somehow.


> 
> It's a business decision and I think SEV can support both. That said, AMD
> currently has no plans to write a shim that can verify the signature on a
> CP-provided BIOS image.
> 
> Richard

Someone else will have to work on a supportable solution for QEMU then.

> > 
> > Richard

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Laszlo Ersek]

Side topic; sorry if it has been mentioned elsewhere:

On 09/27/17 21:06, Richard Relph wrote:

> Whether the "BIOS" is a "static shim" as Michael suggests, or a full
> BIOS, or even a BIOS+kernel+initrd is really not too significant. What
> is significant is that the GO has a basis for trusting all code that is
> imported in to their VM by the CP. And that NONE of the code provided by
> the CP is "unknown" and unauditable by the GO. If the CP has a way to
> inject code unknown to the GO in to the guest VM, the trust model is
> broken and both GO and CP suffer the consequences.

The expansion ROMs (containing UEFI drivers) of emulated PCI devices,
and the same of assigned physical PCI devices, constitute another
channel through which code enters the guest from the outside (i.e., from
the Cloud Provider). The ROM BARs from which the guest firmware reads
the UEFI binaries are not guest RAM, they are MMIO. (For execution, the
drivers are copied into encrypted guest RAM.)

If the guest has Secure Boot enabled, then the oproms are verified[*]
(and not launched if verification fails), but this is slightly different
from what I understand under audit-by-GO. It means the GO wouldn't get a
measurement of the oproms for one-by-one clearing, when about to
green-light a guest startup. Instead the GO would ensure that Secure
Boot be enabled with the right certificates (and/or executable hashes)
enrolled off the bat, and then implicitly trust all oprom drivers
accepted by those certs / hashes. It's another layer of indirection.

This is likely nothing new qualitatively, but "the devil is in the
details", so I thought it was worth raising.

[*] For edk2 / OvmfPkg specifics, I'll mention

  gEfiSecurityPkgTokenSpaceGuid.PcdOptionRomImageVerificationPolicy

The SecurityPkg default is 0x04 ("Deny execution when there is security
violation"). However, OVMF sets it to 0x00 ("Always trust the image").
Please see the following commit for the reasons:

  https://github.com/tianocore/edk2/commit/1fea9ddb4e3fd

Brijesh, for SEV guests, we likely want to flip this PCD to 0x04, in the
AmdSevInitialize() function, in "OvmfPkg/PlatformPei/AmdSev.c". For that
we'll also have to change the PCD from fixed-at-build to dynamic, but
that in turn will require a change to "SecurityPkg.dec" itself
(currently it only allows fixed-at-build or patchable, not dynamic). Do
you want me to file a BZ in the TianoCore tracker for this, and assign
it to you? If you don't have time for writing the patch, I'm glad to do
it too, but then the review could be slower; both other OvmfPkg
co-maintainers are busy with other things.)

Thanks!
Laszlo

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Laszlo Ersek]

On 09/29/17 23:16, Michael S. Tsirkin wrote:
> On Fri, Sep 29, 2017 at 02:48:45PM -0500, Richard Relph wrote:
>> On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
>>> On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
>>>> Whether the "BIOS" is a "static shim" as Michael suggests, or a full BIOS,
>>>> or even a BIOS+kernel+initrd is really not too significant. What is
>>>> significant is that the GO has a basis for trusting all code that is
>>>> imported in to their VM by the CP. And that NONE of the code provided by the
>>>> CP is "unknown" and unauditable by the GO. If the CP has a way to inject
>>>> code unknown to the GO in to the guest VM, the trust model is broken and
>>>> both GO and CP suffer the consequences.
>>>
>>> Absolutely.
>>>
>>>> When the CP needs to update the BIOS image, they will have to inform the GO
>>>> and allow the GO to establish trust in the CP's new BIOS image somehow.
>>>
>>> This GO update on every BIOS change is imho is not a workable model. You
>>> want something like checking the BIOS signature instead. And since
>>> hardware is all hash based, you need the shim to do it in software.
>>
>> A BIOS "signed" by the CP doesn't meet the security requirement. It is code
>> that is "unknown" to the GO.
> 
> There is a misunderstanding here.
> 
> BIOS would not be signed by a CP. It would be signed by a trusted
> software vendor e.g. by Red Hat.
> 
>> The (legitimate) CP does NOT want to be in that position of trust. If they
>> are, then some government somewhere is going to insist that they sign a BIOS
>> that allows the government to spy on the GO's VMs, and steal secrets from
>> it. Or some hacker admin will do it "for fun".
>>
>> How often do large public CPs really change their BIOSes? My sense is that
>> large public CPs prefer stability over "latest and greatest".
> 
> CPs just do dnf update. Software vendors change BIOSes.
> 
> And we do change them. Look at number of revisions for seabios in e.g.
> Fedora. More importantly we might need to change them quickly e.g.
> because of a security issue. Adding the need to coordinate with all GOs
> is not going to work. Neither can QEMU support booting old BIOS
> versions on new machine types indefinitely.
> 
>> And, perhaps more importantly, if a CP are able to sell a "more secure" VM,
>> one that justifies a higher price per vCPU hour, wouldn't that warrant some
>> changes in the "insecure" model being used today?
>>
>> Richard
> 
> Absolutely. CPs have no business signing images. But it is just not
> really feasible for software vendors to distribute hashes.

Can this be helped by "reproducible builds"? Like,

- guest firmware vendor publishes the source package,
- GO asynchronously audits the source package (most likely
incrementally, reviewing the new, broken-out patches in the source package),
- GO builds the binary package,
- GO verifies that it bit-wise matches the guest fw vendor's binary package,
- GO adds the fw binary's hash to their own whitelist.

By virtue of releasing the source package of the guest firmware (and by
announcing it via another erratum), the guest fw vendor implicitly
notifies all GOs. It is then up to the individual GOs to evaluate the
changes on their own time, and to switch to the new fw binary.

It would even be OK if the GO built their own fw binary from the source
package, and submitted that to the CP, as part of the guest payload. The
guest firmware vendor would still be able to support this build, since
it would match the vendor's own binary 100%.

Thanks
Laszlo

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Brijesh Singh]

On 9/29/17 4:58 PM, Laszlo Ersek wrote:
...
> The expansion ROMs (containing UEFI drivers) of emulated PCI devices,
> and the same of assigned physical PCI devices, constitute another
> channel through which code enters the guest from the outside (i.e., from
> the Cloud Provider). The ROM BARs from which the guest firmware reads
> the UEFI binaries are not guest RAM, they are MMIO. (For execution, the
> drivers are copied into encrypted guest RAM.)
>
> If the guest has Secure Boot enabled, then the oproms are verified[*]
> (and not launched if verification fails), but this is slightly different
> from what I understand under audit-by-GO. It means the GO wouldn't get a
> measurement of the oproms for one-by-one clearing, when about to
> green-light a guest startup. Instead the GO would ensure that Secure
> Boot be enabled with the right certificates (and/or executable hashes)
> enrolled off the bat, and then implicitly trust all oprom drivers
> accepted by those certs / hashes. It's another layer of indirection.
>
> This is likely nothing new qualitatively, but "the devil is in the
> details", so I thought it was worth raising.
>
> [*] For edk2 / OvmfPkg specifics, I'll mention
>
>   gEfiSecurityPkgTokenSpaceGuid.PcdOptionRomImageVerificationPolicy
>
> The SecurityPkg default is 0x04 ("Deny execution when there is security
> violation"). However, OVMF sets it to 0x00 ("Always trust the image").
> Please see the following commit for the reasons:
>
>   https://github.com/tianocore/edk2/commit/1fea9ddb4e3fd
>
> Brijesh, for SEV guests, we likely want to flip this PCD to 0x04, in the
> AmdSevInitialize() function, in "OvmfPkg/PlatformPei/AmdSev.c". For that
> we'll also have to change the PCD from fixed-at-build to dynamic, but
> that in turn will require a change to "SecurityPkg.dec" itself
> (currently it only allows fixed-at-build or patchable, not dynamic). Do
> you want me to file a BZ in the TianoCore tracker for this, and assign
> it to you? If you don't have time for writing the patch, I'm glad to do
> it too, but then the review could be slower; both other OvmfPkg
> co-maintainers are busy with other things.)

Very good point Laszlo. Please submit the BZ and assign it me - thank
you. we will take a look at implementing the required support.

-Brijesh

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Fri, Sep 29, 2017 at 03:07:40PM -0500, Richard Relph wrote:
> Depending on your level of paranoia,
> that may require advance notice of BIOS changes, or even allowing the GO to
> provide the BIOS themselves, written to a spec supported by the CP's HV,
> and/or based on BIOS code provided by the CP.

BTW this last most secure option is easy to implement with the shim
because the shim is using very little in terms of the HV interface.

User can then easily build it from source.

-- 
MST

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Fri, Sep 29, 2017 at 03:07:40PM -0500, Richard Relph wrote:
> It's a business decision and I think SEV can support both.

I think what has been missed in the noise is the fact that
with VM launch, key distribution is a huge problem.

With the shim the key distribution problem can go completely away, as
you just start it in the private cloud and include the key with the
shim, then use the send/recv machinery to migrate to the public one.

-- 
MST

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Laszlo Ersek]

On 10/01/17 02:09, Brijesh Singh wrote:
> 
> 
> On 9/29/17 4:58 PM, Laszlo Ersek wrote:
> ...
>> The expansion ROMs (containing UEFI drivers) of emulated PCI devices,
>> and the same of assigned physical PCI devices, constitute another
>> channel through which code enters the guest from the outside (i.e., from
>> the Cloud Provider). The ROM BARs from which the guest firmware reads
>> the UEFI binaries are not guest RAM, they are MMIO. (For execution, the
>> drivers are copied into encrypted guest RAM.)
>>
>> If the guest has Secure Boot enabled, then the oproms are verified[*]
>> (and not launched if verification fails), but this is slightly different
>> from what I understand under audit-by-GO. It means the GO wouldn't get a
>> measurement of the oproms for one-by-one clearing, when about to
>> green-light a guest startup. Instead the GO would ensure that Secure
>> Boot be enabled with the right certificates (and/or executable hashes)
>> enrolled off the bat, and then implicitly trust all oprom drivers
>> accepted by those certs / hashes. It's another layer of indirection.
>>
>> This is likely nothing new qualitatively, but "the devil is in the
>> details", so I thought it was worth raising.
>>
>> [*] For edk2 / OvmfPkg specifics, I'll mention
>>
>>   gEfiSecurityPkgTokenSpaceGuid.PcdOptionRomImageVerificationPolicy
>>
>> The SecurityPkg default is 0x04 ("Deny execution when there is security
>> violation"). However, OVMF sets it to 0x00 ("Always trust the image").
>> Please see the following commit for the reasons:
>>
>>   https://github.com/tianocore/edk2/commit/1fea9ddb4e3fd
>>
>> Brijesh, for SEV guests, we likely want to flip this PCD to 0x04, in the
>> AmdSevInitialize() function, in "OvmfPkg/PlatformPei/AmdSev.c". For that
>> we'll also have to change the PCD from fixed-at-build to dynamic, but
>> that in turn will require a change to "SecurityPkg.dec" itself
>> (currently it only allows fixed-at-build or patchable, not dynamic). Do
>> you want me to file a BZ in the TianoCore tracker for this, and assign
>> it to you? If you don't have time for writing the patch, I'm glad to do
>> it too, but then the review could be slower; both other OvmfPkg
>> co-maintainers are busy with other things.)
> 
> Very good point Laszlo. Please submit the BZ and assign it me - thank
> you. we will take a look at implementing the required support.

I've filed:

https://bugzilla.tianocore.org/show_bug.cgi?id=728

Another thought: once we delegate part of the protection that SEV
provides to the Secure Boot implementation in the guest, the pflash chip
that stores the UEFI variables (i.e., certificates for SB as well)
becomes a target of attacks not just from within the guest, but also
from the host.

This is a big problem because the pflash chip is MMIO, nor RAM. So it is
not encrypted, and the Cloud Provider can tamper with it at will -- even
if the variable store file that backs the pflash chip (with the
certificates pre-enrolled) comes from the Guest Owner initially, and/or
passes the GO's measurement check.

Put differently, even if we set PcdOptionRomImageVerificationPolicy to
0x04, it doesn't help if the CP can poke any UEFI executable hash or
self-signed certificate into the varstore, which in turn will "verify"
the CP's malicious oprom driver.

The varstore pflash chip has never been considered to be under attack
from the *host* side; all the guest-SMM work was focused on protecting
pflash (and guest firmware in general) from the *guest* OS (and 3rd
party *guest* UEFI apps).

Theoretically, I see three ways to mitigate this:

(1) Eliminate pflash from OVMF altogether. This would be a huge project,
affecting OVMF, QEMU and libvirt. It would also throw the current
in-guest protections provided by SB out the window.

(2) Keep pflash as-is, but do not off-load any SEV protections to
guest-side Secure Boot. This means that the guest firmware must be
comprehensively restricted to load binaries only from media that the GO
measured and the CP cannot tamper with. No option ROMs, no PXE boot, no
plaintext ISOs, no plaintext hard disk images. (And booting from
encrypted ISOs / hard disk images would require a LUKS driver for OVMF.)

(3) Implement SEV encryption for pflash. A pflash chip can be in one of
two modes: (a) it reads and executes as ROM, or (b) it behaves like a
programmable (r/w) device with MMIO registers. Switching between both
modes is explicit (see
"OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.c"); perhaps some SEV
controls could be hooked in there.

Thanks
Laszlo

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Laszlo Ersek]

On 10/01/17 11:17, Laszlo Ersek wrote:

> (3) Implement SEV encryption for pflash. A pflash chip can be in one of
> two modes: (a) it reads and executes as ROM, or (b) it behaves like a
> programmable (r/w) device with MMIO registers. Switching between both
> modes is explicit (see
> "OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.c"); perhaps some SEV
> controls could be hooked in there.

Hmmm wait a second, we *already* execute code from pflash (namely SEC
code from OVMF_CODE.fd), and instruction fetch always entails decryption.

Furthermore, in SEC, we decompress PEI and DXE FVs from pflash
(OVMF_CODE.fd) to RAM -- the decompression code runs in long mode, plus
you modified the OVMF/X64 reset vector to set the C-bit in all PTEs,
covering the low 4GB of guest RAM.

This tells me that at least read and execute from pflash work with SEV
decryption already. Is this expected?

However, in PlatformPei, we don't report the varstore pflash range via
any resource descriptor HOB (the varstore chip is at [0xffc00000,
0xFFC84000) in the 4MB build).  Consequently, AmdSevDxe clears the C-bit
under the "NonExistent" branch:

AmdSevDxe:SetMemoryEncDec: Cr3Base=0x0 Physical=0xFEF00000
Length=0x1100000 Mode=Decrypt CacheFlush=0

Subsequently, "OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockService.c"
adds the pflash range as EFI_MEMORY_UC | EFI_MEMORY_RUNTIME. The C-bit
remains clear, AIUI.

OK, I'm totally confused now. Looks like SEV decryption already works
for pflash (from SEC's example), and transparently at that (???), but we
fail to use it in QemuFlashFvbServicesRuntimeDxe?

Laszlo

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Daniel P. Berrange]

On Fri, Sep 29, 2017 at 02:48:45PM -0500, Richard Relph wrote:
> On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
> > On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
> > > Whether the "BIOS" is a "static shim" as Michael suggests, or a full BIOS,
> > > or even a BIOS+kernel+initrd is really not too significant. What is
> > > significant is that the GO has a basis for trusting all code that is
> > > imported in to their VM by the CP. And that NONE of the code provided by the
> > > CP is "unknown" and unauditable by the GO. If the CP has a way to inject
> > > code unknown to the GO in to the guest VM, the trust model is broken and
> > > both GO and CP suffer the consequences.
> > 
> > Absolutely.
> > 
> > > When the CP needs to update the BIOS image, they will have to inform the GO
> > > and allow the GO to establish trust in the CP's new BIOS image somehow.
> > 
> > This GO update on every BIOS change is imho is not a workable model. You
> > want something like checking the BIOS signature instead. And since
> > hardware is all hash based, you need the shim to do it in software.
> 
> A BIOS "signed" by the CP doesn't meet the security requirement. It is code
> that is "unknown" to the GO.
> 
> The (legitimate) CP does NOT want to be in that position of trust. If they
> are, then some government somewhere is going to insist that they sign a BIOS
> that allows the government to spy on the GO's VMs, and steal secrets from
> it. Or some hacker admin will do it "for fun".
> 
> How often do large public CPs really change their BIOSes? My sense is that
> large public CPs prefer stability over "latest and greatest".

It is hard to generalize, but from a RHEL POV, we typically do major updates
of the virt stack every ~6 months, and these will include BIOS updates. So if
a cloud vendor is following the RHEL update stream actively that's the kind
of cadence you'd expect.

The gotcha would come if there were out-of-band security updates for BIOS
which caused it to be updated before the 6 month window. Fortunately I've
not see these happen often, so I don't think its a fatal problem.

IOW, I tend to agree with you that this is not really a blocking problem to
the use of SEV in cloud. 

> And, perhaps more importantly, if a CP are able to sell a "more secure" VM,
> one that justifies a higher price per vCPU hour, wouldn't that warrant some
> changes in the "insecure" model being used today?

Yes.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Daniel P. Berrange]

On Sat, Sep 30, 2017 at 12:16:55AM +0300, Michael S. Tsirkin wrote:
> On Fri, Sep 29, 2017 at 02:48:45PM -0500, Richard Relph wrote:
> > On 9/29/17 2:34 PM, Michael S. Tsirkin wrote:
> > > On Wed, Sep 27, 2017 at 02:06:10PM -0500, Richard Relph wrote:
> > > > Whether the "BIOS" is a "static shim" as Michael suggests, or a full BIOS,
> > > > or even a BIOS+kernel+initrd is really not too significant. What is
> > > > significant is that the GO has a basis for trusting all code that is
> > > > imported in to their VM by the CP. And that NONE of the code provided by the
> > > > CP is "unknown" and unauditable by the GO. If the CP has a way to inject
> > > > code unknown to the GO in to the guest VM, the trust model is broken and
> > > > both GO and CP suffer the consequences.
> > > 
> > > Absolutely.
> > > 
> > > > When the CP needs to update the BIOS image, they will have to inform the GO
> > > > and allow the GO to establish trust in the CP's new BIOS image somehow.
> > > 
> > > This GO update on every BIOS change is imho is not a workable model. You
> > > want something like checking the BIOS signature instead. And since
> > > hardware is all hash based, you need the shim to do it in software.
> > 
> > A BIOS "signed" by the CP doesn't meet the security requirement. It is code
> > that is "unknown" to the GO.
> 
> There is a misunderstanding here.
> 
> BIOS would not be signed by a CP. It would be signed by a trusted
> software vendor e.g. by Red Hat.
> 
> > The (legitimate) CP does NOT want to be in that position of trust. If they
> > are, then some government somewhere is going to insist that they sign a BIOS
> > that allows the government to spy on the GO's VMs, and steal secrets from
> > it. Or some hacker admin will do it "for fun".
> > 
> > How often do large public CPs really change their BIOSes? My sense is that
> > large public CPs prefer stability over "latest and greatest".
> 
> CPs just do dnf update. Software vendors change BIOSes.

No they really don't do that.   Cloud providers will stick on a golden image
of RHEL. If and when they need to do an upgrade, they won't usually do a
'dnf update' because that imposes risk of breaking VMs on that node with no
fallback path. Instead they would usually deploy brand new nodes with the new
software version, and then live migrate VMs off the old hosts.

> And we do change them. Look at number of revisions for seabios in e.g.
> Fedora. More importantly we might need to change them quickly e.g.
> because of a security issue. Adding the need to coordinate with all GOs
> is not going to work. Neither can QEMU support booting old BIOS
> versions on new machine types indefinitely.

I don't think Fedora is a good fit for SEV - a long term supported distro
like RHEL is the more apt target, and there you are looking at 6 months
lifetime, except in rare cases of having a BIOS security flaw to pyush
out.

> 
> > And, perhaps more importantly, if a CP are able to sell a "more secure" VM,
> > one that justifies a higher price per vCPU hour, wouldn't that warrant some
> > changes in the "insecure" model being used today?
> > 
> > Richard
> 
> Absolutely. CPs have no business signing images. But it is just not
> really feasible for software vendors to distribute hashes.

Why can't software vendors distribute hashes - they are best placed to,
particularly when a single vendor is supplying the whole stack of base
OS, virt and OpenStack as a single coordinated product.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Brijesh Singh]

Hi Laszlo,


On 10/01/2017 04:56 AM, Laszlo Ersek wrote:
> On 10/01/17 11:17, Laszlo Ersek wrote:
> 
>> (3) Implement SEV encryption for pflash. A pflash chip can be in one of
>> two modes: (a) it reads and executes as ROM, or (b) it behaves like a
>> programmable (r/w) device with MMIO registers. Switching between both
>> modes is explicit (see
>> "OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.c"); perhaps some SEV
>> controls could be hooked in there.
> 
> Hmmm wait a second, we *already* execute code from pflash (namely SEC
> code from OVMF_CODE.fd), and instruction fetch always entails decryption.
> 
> Furthermore, in SEC, we decompress PEI and DXE FVs from pflash
> (OVMF_CODE.fd) to RAM -- the decompression code runs in long mode, plus
> you modified the OVMF/X64 reset vector to set the C-bit in all PTEs,
> covering the low 4GB of guest RAM.
> 
> This tells me that at least read and execute from pflash work with SEV
> decryption already. Is this expected?
> 


Yes, this is expected. During the machine_init, qemu creates two pflash units.
The pflash unit#0 contains OVMF_CODE.fd and unit #1 points to OVMF_VARS.fd.
As part of machine creation, we encrypt the unit#0 using KVM_SEV_LAUNCH_UPDATE
command. The unit#0 is mapped at the x86 reset vector. On vm_start, guest
fetches the code from the unit#0. Since the code was already encrypted hence
guest was able to read and execute the initial bootstrap code just fine.

When OVMF is build with OvmfPkgX64.dsc, SetCr3ForPageTables64 builds the
initial page table. When SEV is active, we add the C-bit pte mask [1] -- it
will ensures that when decompression happen then data get written as
encrypted in the guest RAM.

[1] https://github.com/tianocore/edk2/commit/e60af8a1ebb15bfcbf2ecc4afb6cf35084c847aa


When OVMF is build with OvmfPkgIa32X64.dsc, the SEC phase runs in 32-bit mode
hence SEV hardware forces the C-bit to 1.  This will ensure that decompression
routines writes the data as encrypted.



> However, in PlatformPei, we don't report the varstore pflash range via
> any resource descriptor HOB (the varstore chip is at [0xffc00000,
> 0xFFC84000) in the 4MB build).  Consequently, AmdSevDxe clears the C-bit
> under the "NonExistent" branch:
> 
> AmdSevDxe:SetMemoryEncDec: Cr3Base=0x0 Physical=0xFEF00000
> Length=0x1100000 Mode=Decrypt CacheFlush=0
> 
> Subsequently, "OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockService.c"
> adds the pflash range as EFI_MEMORY_UC | EFI_MEMORY_RUNTIME. The C-bit
> remains clear, AIUI.
> 
> OK, I'm totally confused now. Looks like SEV decryption already works
> for pflash (from SEC's example), and transparently at that (???), but we
> fail to use it in QemuFlashFvbServicesRuntimeDxe?
> 


AmdSevDxe clears the C-bit of pflash range in PlatformPei because
QemuFlashFvbServicesRuntimeDxe switches the pflash out from ROMD mode; i.e
it become MMIO. The MMIO regions must be mapped as unencrypted.

SEV hardware engine uses a random encryption key on each run. Since
OVMF_VAR.fd is used for storing the UEFI persistent variable hence we
will not be able to store the data encrypted using the SEV engine. Any
persistent storage should be protected using a disk encryption technologies
(like luks, dm-crypt etc).

Regarding optionROM comments, I will propose something on edk-devel
list. I am thinking at least we should start by adding some kind of bug_on or
warning when a SEV guest BIOS is attempting to copy the option ROM.
I am hoping that once we have secure boot working then we should be able to
handle the signed option ROM, grub.efi or kernel.efi etc.


-Brijesh

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:
> > >      > 11. GO verifies the measurement and if measurement matches then it may
> > >      >  give a secret blob -- which must be injected into the guest before
> > >      >  libvirt starts the VM. If verification failed, GO will request cloud
> > >      >  provider to destroy the VM.

I realised I'm missing something here: how does GO limit the
secret to the specific VM? For example, what prevents hypervisor
from launching two VMs with the same GO's DH, getting measurement
from 1st one but injecting the secret into the second one?

Thanks,

-- 
MST

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Dr. David Alan Gilbert]

* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:
> > > >      > 11. GO verifies the measurement and if measurement matches then it may
> > > >      >  give a secret blob -- which must be injected into the guest before
> > > >      >  libvirt starts the VM. If verification failed, GO will request cloud
> > > >      >  provider to destroy the VM.
> 
> I realised I'm missing something here: how does GO limit the
> secret to the specific VM? For example, what prevents hypervisor
> from launching two VMs with the same GO's DH, getting measurement
> from 1st one but injecting the secret into the second one?

Isn't that the 'trusted channel nonce currently associated with the
guest' in the guest context?

Dave

> Thanks,
> 
> -- 
> MST
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Michael S. Tsirkin]

On Wed, Oct 18, 2017 at 08:18:48PM +0100, Dr. David Alan Gilbert wrote:
> * Michael S. Tsirkin (mst@redhat.com) wrote:
> > On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:
> > > > >      > 11. GO verifies the measurement and if measurement matches then it may
> > > > >      >  give a secret blob -- which must be injected into the guest before
> > > > >      >  libvirt starts the VM. If verification failed, GO will request cloud
> > > > >      >  provider to destroy the VM.
> > 
> > I realised I'm missing something here: how does GO limit the
> > secret to the specific VM? For example, what prevents hypervisor
> > from launching two VMs with the same GO's DH, getting measurement
> > from 1st one but injecting the secret into the second one?
> 
> Isn't that the 'trusted channel nonce currently associated with the
> guest' in the guest context?
> 
> Dave

Let me try to clarify the question. I understand that sometimes a key
is shared between VMs. If this is allowed, it seems that a hypervisor
can run any number of VMs with the same key. An unauthorised VM
will not get a measurement that guest owner authorizes, but
can the hypervisor get secret intended for an authorized VM and
then inject it into an unauthorized one sharing the same key?

> > Thanks,
> > 
> > -- 
> > MST
> > 
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Richard Relph]

On 10/18/17 8:35 PM, Michael S. Tsirkin wrote:
> On Wed, Oct 18, 2017 at 08:18:48PM +0100, Dr. David Alan Gilbert wrote:
>> * Michael S. Tsirkin (mst@redhat.com) wrote:
>>> On Fri, Sep 08, 2017 at 10:48:10AM -0500, Brijesh Singh wrote:
>>>>>>       > 11. GO verifies the measurement and if measurement matches then it may
>>>>>>       >  give a secret blob -- which must be injected into the guest before
>>>>>>       >  libvirt starts the VM. If verification failed, GO will request cloud
>>>>>>       >  provider to destroy the VM.
>>>
>>> I realised I'm missing something here: how does GO limit the
>>> secret to the specific VM? For example, what prevents hypervisor
>>> from launching two VMs with the same GO's DH, getting measurement
>>> from 1st one but injecting the secret into the second one?
>>
>> Isn't that the 'trusted channel nonce currently associated with the
>> guest' in the guest context?
>>
>> Dave
> 
> Let me try to clarify the question. I understand that sometimes a key
> is shared between VMs. If this is allowed, it seems that a hypervisor
> can run any number of VMs with the same key. An unauthorised VM
> will not get a measurement that guest owner authorizes, but
> can the hypervisor get secret intended for an authorized VM and
> then inject it into an unauthorized one sharing the same key?

Michael,
     Yes, that's possible. This is why we recommend against a guest 
owner authorizing key sharing. There's no way for the guest owner to 
control what other guests the HV might install using the same key and 
mapping the same memory to.
     So a security-conscious customer, especially in a cloud 
environment, should never enable key sharing.

Richard

> 
>>> Thanks,
>>>
>>> -- 
>>> MST
>>>
>> --
>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK