Linux-Sgx Archive on lore.kernel.org
 help / color / Atom feed
From: Nathaniel McCallum <npmccallum@redhat.com>
To: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Jethro Beekman <jethro@fortanix.com>,
	Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
	linux-kernel@vger.kernel.org, x86@kernel.org,
	linux-sgx@vger.kernel.org, akpm@linux-foundation.org,
	dave.hansen@intel.com, Neil Horman <nhorman@redhat.com>,
	"Huang, Haitao" <haitao.huang@intel.com>,
	andriy.shevchenko@linux.intel.com, tglx@linutronix.de, "Svahn,
	Kai" <kai.svahn@intel.com>,
	bp@alien8.de, Josh Triplett <josh@joshtriplett.org>,
	luto@kernel.org, kai.huang@intel.com,
	David Rientjes <rientjes@google.com>,
	cedric.xing@intel.com, Patrick Uiterwijk <puiterwijk@redhat.com>,
	Andy Lutomirski <luto@amacapital.net>,
	Connor Kuehl <ckuehl@redhat.com>,
	Harald Hoyer <harald@redhat.com>,
	Lily Sturmann <lsturman@redhat.com>
Subject: Re: [PATCH v28 21/22] x86/vdso: Implement a vDSO for Intel SGX enclave call
Date: Fri, 13 Mar 2020 16:14:01 -0400
Message-ID: <CAOASepP_oGOenjCvAvLg+e+=fz4H0X=cyD+v5ywD0peeXEEmYg@mail.gmail.com> (raw)
In-Reply-To: <20200313184452.GD5181@linux.intel.com>

On Fri, Mar 13, 2020 at 2:45 PM Sean Christopherson
<sean.j.christopherson@intel.com> wrote:
>
> On Fri, Mar 13, 2020 at 02:32:29PM -0400, Nathaniel McCallum wrote:
> > On Fri, Mar 13, 2020 at 12:46 PM Sean Christopherson
> > <sean.j.christopherson@intel.com> wrote:
> > >
> > > On Fri, Mar 13, 2020 at 11:48:54AM -0400, Nathaniel McCallum wrote:
> > > > Therefore, I'd like to propose that __vdso_sgx_enter_enclave():
> > > >   * Preserve %rbx.
> > >
> > > At first glance, that looks sane.  Being able to call __vdso... from C
> > > would certainly be nice.
> >
> > Agreed. I think ergonomically we want __vdso...() to be called from C
> > and the handler to be implemented in asm (optionally); without
> > breaking the ability to call __vdso..() from asm in special cases.
> >
> > I think all ergonomic issues get solved by the following:
> >    * Pass a void * into the handler from C through __vdso...().
> >    * Allow the handler to pop parameters off of the output stack without hacks.
> >
> > This allows the handler to pop extra arguments off the stack and write
> > them into the memory at the void *. Then the handler can be very small
> > and pass logic back to the caller of __vdso...().
> >
> > Here's what this all means for the enclave. For maximum usability, the
> > enclave should preserve all callee-saved registers (except %rbx, which
> > is preserved by __vdso..()). For each ABI rule that the enclave
> > breaks, you need logic in a handler to fix it. So if you push return
> > params on the stack, the handler needs to undo that.
>
> Or the untrusted runtime needs to wrap the __vdso() to save state that is
> clobbered by the enclave.  Just want to make it crystal clear that using a
> handler is only required for stack shenanigans.

Yup.

> > This doesn't compromise the ability to treat __vsdo...() like ENCLU if
> > you need the full power. But it does make it significantly easier to
> > consume when you don't have special needs. So as I see it, __vdso...()
> > should:
> >
> > 1. preserve %rbx
> > 2. take leaf in %rcx
> > 3. gain a void* stack param which is passed to the handler
>
> Unless I'm misunderstanding the request, this already exists.  %rsp at the
> time of EEXIT is passed to the handler.

Sorry, different stack parameter. I mean this:

typedef int (*sgx_enclave_exit_handler_t)(
    long rdi,
    long rsi,
    long rdx,
    long ursp,
    long r8,
    long r9,
    int ret,
    void *tcs,
    struct sgx_enclave_exception *e,
    void *misc
);

int __vdso_sgx_enter_enclave(
    long rdi,
    long rsi,
    long rdx,
    int leaf,
    long r8,
    long r9,
    void *tcs,
    struct sgx_enclave_exception *e,
    void *misc,
    sgx_enclave_exit_handler_t handler
);

This is so that the caller of __vdso...() can pass context into the
handler. Note that I've also reordered the stack parameters so that
the stack order can be reused.

> > 4. sub/add to %rsp rather than save/restore
>
> Can you elaborate on why you want to sub/add to %rsp instead of having the
> enclave unwind the stack?  Preserving %rsp across EEXIT/ERESUME seems more
> in line with function call semantics, which I assume is desirable?  E.g.
>
>   push param3
>   push param2
>   push param1
>
>   enclu[EEXIT]
>
>   add $0x18, %rsp

Before enclave EEXIT, the enclave restores %rsp to the value it had
before EENTER was called. Then it pushes additional output arguments
onto the stack. The enclave calls EENCLU[EEXIT].

We are now in __vdso...() on the way back to the caller. However, %rsp
has a different value than we entered the function with. This breaks
x86_64 ABI, obviously. The handler needs to fix this up: how does it
do so?

In the current code, __vdso..() saves the value of %rsp, calls the
handler and then restores %rsp. The handler can fix up the stack by
setting the correct value to %rbx and returning without restoring it.
But this requires internal knowledge of the __vdso...() function,
which could theoretically change in the future.

If instead the __vdso...() only did add/sub, then the handler could do:
1. pop return address
2. pop handler stack params
3. pop enclave additional output stack params
4. push handler stack params
5. push return address

While this is more work, it is standard calling convention work that
doesn't require internal knowledge of __vdso..(). Alternatively, if we
don't like the extra work, we can document the %rbx hack explicitly
into the handler documentation and make it part of the interface. But
we need some explicit way for the handler to pop enclave output stack
params that doesn't depend on internal knowledge of the __vdso...()
invariants.

> > That would make this a very usable and fast interface without
> > sacrificing any of its current power.
>


  reply index

Thread overview: 109+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-03 23:35 [PATCH v28 00/22] Intel SGX foundations Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 01/22] x86/sgx: Update MAINTAINERS Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 02/22] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 03/22] x86/cpufeatures: x86/msr: Intel SGX Launch Control " Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 04/22] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 05/22] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 06/22] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2020-03-09 21:14   ` Sean Christopherson
2020-03-03 23:35 ` [PATCH v28 07/22] x86/cpu/intel: Detect SGX supprt Jarkko Sakkinen
2020-03-09 21:56   ` Sean Christopherson
2020-03-11 17:03     ` Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 08/22] x86/sgx: Enumerate and track EPC sections Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 09/22] x86/sgx: Add functions to allocate and free EPC pages Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 10/22] mm: Introduce vm_ops->may_mprotect() Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 11/22] x86/sgx: Linux Enclave Driver Jarkko Sakkinen
2020-03-05 17:40   ` Sean Christopherson
2020-03-05 18:24     ` Jethro Beekman
2020-03-05 19:04       ` Sean Christopherson
2020-03-06 19:00       ` Jarkko Sakkinen
2020-03-19 18:22         ` Dr. Greg
2020-03-06 18:58     ` Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 12/22] docs: x86/sgx: Document SGX micro architecture and kernel internals Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 13/22] selftests/x86: Recurse into subdirectories Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 14/22] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2020-03-04 19:27   ` Nathaniel McCallum
2020-03-05 11:33     ` Jarkko Sakkinen
2020-03-06 15:42       ` Dr. Greg
2020-03-06 19:07         ` Jarkko Sakkinen
2020-03-07 17:42           ` Dr. Greg
2020-03-10 13:08             ` Jarkko Sakkinen
2020-03-11 13:28       ` Jarkko Sakkinen
     [not found]         ` <20200311164047.GG21852@linux.intel.com>
2020-03-13 19:24           ` Jarkko Sakkinen
2020-03-04 19:44   ` Nathaniel McCallum
2020-03-04 19:51   ` Nathaniel McCallum
2020-03-06  5:32   ` Dr. Greg
2020-03-06 19:04     ` Jarkko Sakkinen
2020-03-10 19:29     ` Haitao Huang
2020-03-11  9:13       ` Dr. Greg
2020-03-11 17:15         ` Haitao Huang
2020-03-17  1:07       ` Dr. Greg
2020-03-03 23:36 ` [PATCH v28 15/22] x86/sgx: Add provisioning Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 16/22] x86/sgx: Add a page reclaimer Jarkko Sakkinen
2020-03-05 19:03   ` Sean Christopherson
2020-03-06 18:47     ` Jarkko Sakkinen
2020-03-12 18:38       ` Sean Christopherson
2020-03-15  0:27         ` Jarkko Sakkinen
2020-03-15  1:17           ` Jarkko Sakkinen
2020-03-09 21:16   ` Sean Christopherson
2020-03-03 23:36 ` [PATCH v28 17/22] x86/sgx: ptrace() support for the SGX driver Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 18/22] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 19/22] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 20/22] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 21/22] x86/vdso: Implement a vDSO for Intel SGX enclave call Jarkko Sakkinen
2020-03-11 17:30   ` Nathaniel McCallum
2020-03-11 17:38     ` Jethro Beekman
2020-03-11 19:15       ` Nathaniel McCallum
2020-03-13 15:48       ` Nathaniel McCallum
2020-03-13 16:46         ` Sean Christopherson
2020-03-13 18:32           ` Nathaniel McCallum
2020-03-13 18:44             ` Sean Christopherson
2020-03-13 20:14               ` Nathaniel McCallum [this message]
2020-03-13 22:08                 ` Sean Christopherson
2020-03-14 14:10                   ` Nathaniel McCallum
2020-03-18 23:40                     ` Sean Christopherson
2020-03-19  0:38                       ` Xing, Cedric
2020-03-19  1:03                         ` Sean Christopherson
2020-03-20 13:55                       ` Nathaniel McCallum
2020-03-15  1:25     ` Jarkko Sakkinen
2020-03-15 17:53       ` Nathaniel McCallum
2020-03-16 13:31         ` Jethro Beekman
2020-03-16 13:57           ` Nathaniel McCallum
2020-03-16 13:59             ` Jethro Beekman
2020-03-16 14:03               ` Nathaniel McCallum
2020-03-16 17:17                 ` Sean Christopherson
2020-03-16 21:27             ` Jarkko Sakkinen
2020-03-16 21:29               ` Jarkko Sakkinen
2020-03-16 22:55           ` Sean Christopherson
2020-03-16 23:56             ` Xing, Cedric
2020-03-18 22:01               ` Jarkko Sakkinen
2020-03-18 22:18                 ` Jarkko Sakkinen
2020-03-16 13:56         ` Jarkko Sakkinen
2020-03-16 14:01           ` Nathaniel McCallum
2020-03-16 21:38             ` Jarkko Sakkinen
2020-03-16 22:53               ` Sean Christopherson
2020-03-16 23:50                 ` Xing, Cedric
2020-03-16 23:59                   ` Sean Christopherson
2020-03-17  0:18                     ` Xing, Cedric
2020-03-17  0:27                       ` Sean Christopherson
2020-03-17 16:37                         ` Nathaniel McCallum
2020-03-17 16:50                       ` Nathaniel McCallum
2020-03-17 21:40                         ` Xing, Cedric
2020-03-17 22:09                           ` Sean Christopherson
2020-03-17 22:36                             ` Xing, Cedric
2020-03-17 23:57                               ` Sean Christopherson
2020-03-17 22:23                         ` Xing, Cedric
2020-03-18 13:01                           ` Nathaniel McCallum
2020-03-20 15:53                             ` Nathaniel McCallum
2020-03-17 16:28                 ` Nathaniel McCallum
2020-03-18 22:58                   ` Jarkko Sakkinen
2020-03-18 22:39                 ` Jarkko Sakkinen
2020-03-11 19:30   ` Nathaniel McCallum
2020-03-13  0:52     ` Sean Christopherson
2020-03-13 16:07       ` Nathaniel McCallum
2020-03-13 16:33         ` Sean Christopherson
2020-03-03 23:36 ` [PATCH v28 22/22] selftests/x86: Add vDSO selftest for SGX Jarkko Sakkinen
2020-03-04 19:24 ` [PATCH v28 00/22] Intel SGX foundations Nathaniel McCallum
2020-03-17 16:00 ` Jordan Hand
2020-03-18 21:56   ` Jarkko Sakkinen
2020-03-19 17:16   ` Dr. Greg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAOASepP_oGOenjCvAvLg+e+=fz4H0X=cyD+v5ywD0peeXEEmYg@mail.gmail.com' \
    --to=npmccallum@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=bp@alien8.de \
    --cc=cedric.xing@intel.com \
    --cc=ckuehl@redhat.com \
    --cc=dave.hansen@intel.com \
    --cc=haitao.huang@intel.com \
    --cc=harald@redhat.com \
    --cc=jarkko.sakkinen@linux.intel.com \
    --cc=jethro@fortanix.com \
    --cc=josh@joshtriplett.org \
    --cc=kai.huang@intel.com \
    --cc=kai.svahn@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-sgx@vger.kernel.org \
    --cc=lsturman@redhat.com \
    --cc=luto@amacapital.net \
    --cc=luto@kernel.org \
    --cc=nhorman@redhat.com \
    --cc=puiterwijk@redhat.com \
    --cc=rientjes@google.com \
    --cc=sean.j.christopherson@intel.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Sgx Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-sgx/0 linux-sgx/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-sgx linux-sgx/ https://lore.kernel.org/linux-sgx \
		linux-sgx@vger.kernel.org
	public-inbox-index linux-sgx

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-sgx


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git