Re: [PATCH v1 00/15] Add support for Nitro Enclaves

[Alexander Graf <graf@amazon.com>]

On 23.04.20 23:18, Paolo Bonzini wrote:
> 
> 
> On 23/04/20 22:56, Alexander Graf wrote:
>>>
>>> CPL3 is how the user application run, but does the enclave's Linux boot
>>> process start in real mode at the reset vector (0xfffffff0), in 16-bit
>>> protected mode at the Linux bzImage entry point, or at the ELF entry
>>> point?
>>
>> There is no "entry point" per se. You prepopulate at target bzImage into
>> the enclave memory on boot which then follows the standard boot
>> protocol. Everything
> 
> There's still a "where" missing in that sentence. :)  I assume you put
> it at 0x10000 (and so the entry point at 0x10200)?  That should be
> documented because that is absolutely not what the KVM API looks like.

Yes, that part is not documented in the patch set, correct. I would 
personally just make an example user space binary the documentation for 
now. Later we will publish a proper device specification outside of the 
Linux ecosystem which will describe the register layout and image 
loading semantics in verbatim, so that other OSs can implement the 
driver too.

To answer the question though, the target file is in a newly invented 
file format called "EIF" and it needs to be loaded at offset 0x800000 of 
the address space donated to the enclave.

> 
>> before that (enclave firmware, etc.) is provided by
>> the enclave environment.
>>
>> Think of it like a mechanism to launch a second QEMU instance on the
>> host, but all you can actually control are the -smp, -m, -kernel and
>> -initrd parameters.
> 
> Are there requirements on how to populate the memory to ensure that the
> host firmware doesn't crash and burn?  E.g. some free memory right below
> 4GiB (for the firmware, the LAPIC/IOAPIC or any other special MMIO
> devices you have, PCI BARs, and the like)?

No, the target memory layout is currently disconnected from the memory 
layout defined through the KVM_SET_USER_MEMORY_REGION ioctl. While we do 
check that guest_phys_addr is contiguous, the underlying device API does 
not have any notion of a "guest address" - all it gets is a 
scatter-gather sliced bucket of memory.

>> The only I/O channel you have between your VM and
>> that new VM is a vsock channel which is configured by the host on your
>> behalf.
> 
> Is this virtio-mmio or virtio-pci, and what other emulated devices are
> there and how do you discover them?  Are there any ISA devices
> (RTC/PIC/PIT), and are there SMBIOS/RSDP/MP tables in the F segment?

It is virtio-mmio for the enclave and virtio-pci for the parent. The 
enclave is a microvm.

For more details on the enclave device topology, we'll have to wait for 
the public documentation that describes the enclave view of the world 
though. I don't think that one's public quite yet. This patch set is 
about the parent's view.


Alex



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879