LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Alexander Graf <graf@amazon.com>
To: Paolo Bonzini <pbonzini@redhat.com>,
	"Paraschiv, Andra-Irina" <andraprs@amazon.com>,
	<linux-kernel@vger.kernel.org>
Cc: Anthony Liguori <aliguori@amazon.com>,
	Benjamin Herrenschmidt <benh@amazon.com>,
	Colm MacCarthaigh <colmmacc@amazon.com>,
	Bjoern Doebel <doebel@amazon.de>,
	David Woodhouse <dwmw@amazon.co.uk>,
	Frank van der Linden <fllinden@amazon.com>,
	Martin Pohlack <mpohlack@amazon.de>, Matt Wilson <msw@amazon.com>,
	Balbir Singh <sblbir@amazon.com>,
	Stewart Smith <trawets@amazon.com>,
	Uwe Dannowski <uwed@amazon.de>, <kvm@vger.kernel.org>,
	<ne-devel-upstream@amazon.com>
Subject: Re: [PATCH v1 00/15] Add support for Nitro Enclaves
Date: Tue, 28 Apr 2020 17:07:34 +0200
Message-ID: <0a4c7a95-af86-270f-6770-0a283cec30df@amazon.com> (raw)
In-Reply-To: <fb0bfd95-4732-f3c6-4a59-7227cf50356c@redhat.com>



On 25.04.20 18:05, Paolo Bonzini wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
> On 24/04/20 21:11, Alexander Graf wrote:
>> What I was saying above is that maybe code is easier to transfer that
>> than a .txt file that gets lost somewhere in the Documentation directory
>> :).
> 
> whynotboth.jpg :D

Uh, sure? :)

Let's first hammer out what we really want for the UABI though. Then we 
can document it.

>>>> To answer the question though, the target file is in a newly invented
>>>> file format called "EIF" and it needs to be loaded at offset 0x800000 of
>>>> the address space donated to the enclave.
>>>
>>> What is this EIF?
>>
>> It's just a very dumb container format that has a trivial header, a
>> section with the bzImage and one to many sections of initramfs.
>>
>> As mentioned earlier in this thread, it really is just "-kernel" and
>> "-initrd", packed into a single binary for transmission to the host.
> 
> Okay, got it.  So, correct me if this is wrong, the information that is
> needed to boot the enclave is:
> 
> * the kernel, in bzImage format
> 
> * the initrd

It's a single EIF file for a good reason. There are checksums in there 
and potentially signatures too, so that you can the enclave can attest 
itself. For the sake of the user space API, the enclave image really 
should just be considered a blob.

> 
> * a consecutive amount of memory, to be mapped with
> KVM_SET_USER_MEMORY_REGION
> 
> Off list, Alex and I discussed having a struct that points to kernel and
> initrd off enclave memory, and have the driver build EIF at the
> appropriate point in enclave memory (the 8 MiB ofset that you mentioned).
> 
> This however has two disadvantages:
> 
> 1) having the kernel and initrd loaded by the parent VM in enclave
> memory has the advantage that you save memory outside the enclave memory
> for something that is only needed inside the enclave
> 
> 2) it is less extensible (what if you want to use PVH in the future for
> example) and puts in the driver policy that should be in userspace.
> 
> 
> So why not just start running the enclave at 0xfffffff0 in real mode?
> Yes everybody hates it, but that's what OSes are written against.  In
> the simplest example, the parent enclave can load bzImage and initrd at
> 0x10000 and place firmware tables (MPTable and DMI) somewhere at
> 0xf0000; the firmware would just be a few movs to segment registers
> followed by a long jmp.

There is a bit of initial attestation flow in the enclave, so that you 
can be sure that the code that is running is actually what you wanted to 
run.

I would also in general prefer to disconnect the notion of "enclave 
memory" as much as possible from a memory location view. User space 
shouldn't be in the business of knowing location of its donated memory 
ended up at which enclave memory position. By disconnecting the view of 
the memory world, we can do some more optimizations, such as compact 
memory ranges more efficiently in kernel space.

> If you want to keep EIF, we measured in QEMU that there is no measurable
> difference between loading the kernel in the host and doing it in the
> guest, so Amazon could provide an EIF loader stub at 0xfffffff0 for
> backwards compatibility.

It's not about performance :).

So the other thing we discussed was whether the KVM API really turned 
out to be a good fit here. After all, today we merely call:

   * CREATE_VM
   * SET_MEMORY_RANGE
   * CREATE_VCPU
   * START_ENCLAVE

where we even butcher up CREATE_VCPU into a meaningless blob of overhead 
for no good reason.

Why don't we build something like the following instead?

   vm = ne_create(vcpus = 4)
   ne_set_memory(vm, hva, len)
   ne_load_image(vm, addr, len)
   ne_start(vm)

That way we would get the EIF loading into kernel space. "LOAD_IMAGE" 
would only be available in the time window between set_memory and start. 
It basically implements a memcpy(), but it would completely hide the 
hidden semantics of where an EIF has to go, so future device versions 
(or even other enclave implementers) could change the logic.

I think it also makes sense to just allocate those 4 ioctls from 
scratch. Paolo, would you still want to "donate" KVM ioctl space in that 
case?

Overall, the above should address most of the concerns you raised in 
this mail, right? It still requires copying, but at least we don't have 
to keep the copy in kernel space.


Alex



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



  parent reply index

Thread overview: 77+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-21 18:41 Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 01/15] nitro_enclaves: Add ioctl interface definition Andra Paraschiv
2020-04-21 18:47   ` Randy Dunlap
2020-04-21 21:45     ` Paolo Bonzini
2020-04-22 15:49       ` Paraschiv, Andra-Irina
2020-04-21 18:41 ` [PATCH v1 02/15] nitro_enclaves: Define the PCI device interface Andra Paraschiv
2020-04-21 21:22   ` Paolo Bonzini
2020-04-23 13:37     ` Paraschiv, Andra-Irina
2020-04-24 15:10       ` Paraschiv, Andra-Irina
2020-04-21 18:41 ` [PATCH v1 03/15] nitro_enclaves: Define enclave info for internal bookkeeping Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 04/15] nitro_enclaves: Init PCI device driver Andra Paraschiv
2020-04-25 14:25   ` Liran Alon
2020-04-29 16:31     ` Paraschiv, Andra-Irina
2020-04-21 18:41 ` [PATCH v1 05/15] nitro_enclaves: Handle PCI device command requests Andra Paraschiv
2020-04-25 14:52   ` Liran Alon
2020-04-29 17:00     ` Paraschiv, Andra-Irina
2020-04-21 18:41 ` [PATCH v1 06/15] nitro_enclaves: Handle out-of-band PCI device events Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 07/15] nitro_enclaves: Init misc device providing the ioctl interface Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 08/15] nitro_enclaves: Add logic for enclave vm creation Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 09/15] nitro_enclaves: Add logic for enclave vcpu creation Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 10/15] nitro_enclaves: Add logic for enclave memory region set Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 11/15] nitro_enclaves: Add logic for enclave start Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 12/15] nitro_enclaves: Add logic for enclave termination Andra Paraschiv
2020-04-21 18:41 ` [PATCH v1 13/15] nitro_enclaves: Add Kconfig for the Nitro Enclaves driver Andra Paraschiv
2020-04-21 18:50   ` Randy Dunlap
2020-04-22 14:35     ` Paraschiv, Andra-Irina
2020-04-21 18:41 ` [PATCH v1 14/15] nitro_enclaves: Add Makefile " Andra Paraschiv
2020-04-23  8:12   ` kbuild test robot
2020-04-24 17:00     ` Paraschiv, Andra-Irina
2020-04-23  8:43   ` kbuild test robot
2020-04-24 15:27     ` Paraschiv, Andra-Irina
2020-04-21 18:41 ` [PATCH v1 15/15] MAINTAINERS: Add entry " Andra Paraschiv
2020-04-21 21:46 ` [PATCH v1 00/15] Add support for Nitro Enclaves Paolo Bonzini
2020-04-23 13:19   ` Paraschiv, Andra-Irina
2020-04-23 13:42     ` Paolo Bonzini
2020-04-23 17:42       ` Paraschiv, Andra-Irina
2020-04-23 17:51         ` Paolo Bonzini
2020-04-23 20:56           ` Alexander Graf
2020-04-23 21:18             ` Paolo Bonzini
2020-04-24 12:56               ` Alexander Graf
2020-04-24 16:27                 ` Paolo Bonzini
2020-04-24 19:11                   ` Alexander Graf
2020-04-25 16:05                     ` Paolo Bonzini
2020-04-27  9:15                       ` Paraschiv, Andra-Irina
2020-04-27  9:22                       ` Paraschiv, Andra-Irina
2020-04-27  9:46                         ` Paolo Bonzini
2020-04-27 10:00                           ` Paraschiv, Andra-Irina
2020-04-28 15:07                       ` Alexander Graf [this message]
2020-04-29 13:20                         ` Paolo Bonzini
2020-04-30 13:59                           ` Paraschiv, Andra-Irina
2020-04-30 10:34                         ` Paolo Bonzini
2020-04-30 11:21                           ` Alexander Graf
2020-04-30 11:38                             ` Paolo Bonzini
2020-04-30 11:47                               ` Alexander Graf
2020-04-30 11:58                                 ` Paolo Bonzini
2020-04-30 12:19                                   ` Alexander Graf
2020-05-07 17:44       ` Pavel Machek
2020-05-08  7:00         ` Paraschiv, Andra-Irina
2020-05-09 19:21           ` Pavel Machek
2020-05-10 11:02             ` Herrenschmidt, Benjamin
2020-05-11 10:49               ` Paraschiv, Andra-Irina
2020-05-11 13:49               ` Stefan Hajnoczi
2020-04-24  3:04     ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2020-04-24  8:19       ` Paraschiv, Andra-Irina
2020-04-24  9:54         ` Paraschiv, Andra-Irina
2020-04-26  1:55           ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2020-04-27 18:39             ` Paraschiv, Andra-Irina
2020-04-24  9:59     ` Tian, Kevin
2020-04-24 13:59       ` Paraschiv, Andra-Irina
2020-04-26  8:16         ` Tian, Kevin
2020-04-27 19:05           ` Paraschiv, Andra-Irina
     [not found]         ` <CAKXe6SLonLQLAOY9Q_2AzTeg4uJxiknsAWnJpTF0hMcXEG5Tew@mail.gmail.com>
2020-05-11 12:05           ` Paraschiv, Andra-Irina
2020-04-25 15:25     ` Liran Alon
2020-04-27  7:56       ` Paraschiv, Andra-Irina
2020-04-27 11:44         ` Liran Alon
2020-04-28 15:25           ` Alexander Graf
2020-04-28 16:01             ` Liran Alon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0a4c7a95-af86-270f-6770-0a283cec30df@amazon.com \
    --to=graf@amazon.com \
    --cc=aliguori@amazon.com \
    --cc=andraprs@amazon.com \
    --cc=benh@amazon.com \
    --cc=colmmacc@amazon.com \
    --cc=doebel@amazon.de \
    --cc=dwmw@amazon.co.uk \
    --cc=fllinden@amazon.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mpohlack@amazon.de \
    --cc=msw@amazon.com \
    --cc=ne-devel-upstream@amazon.com \
    --cc=pbonzini@redhat.com \
    --cc=sblbir@amazon.com \
    --cc=trawets@amazon.com \
    --cc=uwed@amazon.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git