Linux-Sgx Archive on lore.kernel.org
 help / color / Atom feed
From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Nathaniel McCallum <npmccallum@redhat.com>
Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
	x86@kernel.org, linux-sgx@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Andy Lutomirski <luto@amacapital.net>,
	Jethro Beekman <jethro@fortanix.com>,
	Cedric Xing <cedric.xing@intel.com>,
	akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com,
	asapek@google.com, bp@alien8.de, chenalexchen@google.com,
	conradparker@google.com, cyhanish@google.com,
	dave.hansen@intel.com, "Huang, Haitao" <haitao.huang@intel.com>,
	Josh Triplett <josh@joshtriplett.org>,
	kai.huang@intel.com, "Svahn, Kai" <kai.svahn@intel.com>,
	kmoy@google.com, ludloff@google.com, luto@kernel.org,
	Neil Horman <nhorman@redhat.com>,
	Patrick Uiterwijk <puiterwijk@redhat.com>,
	David Rientjes <rientjes@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	yaozhangx@google.com
Subject: Re: [PATCH v36 21/24] x86/vdso: Implement a vDSO for Intel SGX enclave call
Date: Mon, 10 Aug 2020 15:23:17 -0700
Message-ID: <20200810222317.GG14724@linux.intel.com> (raw)
In-Reply-To: <CAOASepOqRfUafSv_qjUv-jW_6n8G7kZ9yh-2z_Z9sjL_2zqNCg@mail.gmail.com>

On Thu, Aug 06, 2020 at 10:55:43AM -0400, Nathaniel McCallum wrote:
> In a past revision of this patch, I had requested a void *misc
> parameter that could be passed through vdso_sgx_enter_enclave_t into
> sgx_enclave_exit_handler_t. This request encountered some push back
> and I dropped the issue. However, I'd like to revisit it or something
> similar.
> 
> One way to create a generic interface to SGX is to pass a structure
> that captures the relevant CPU state from the handler so that it can
> be evaluated in C code before reentry. Precedent for this approach can
> be found in struct kvm_run[0]. Currently, however, there is no way to
> pass a pointer to such a structure directly into the handler.

The context switching aspect of kvm_run isn't a great template.  kvm_run
allows the VMM to get/set a limited amount of vCPU state without having to
invoke separate ioctls(), i.e. it's it's purely an optimization.  KVM also
needs to context switch guests state regardless of the ability to get/set
state via kvm_run, whereas this vDSO case doesn't _need_ to insert itself
between the runtime and its enclave.

The flow control and exit reporting aspect of kvm_run are relevant though.
More thoughts on that part at the end.

> This can be done implicitly by wrapping the struct
> sgx_enclave_exception in another structure and then using techniques
> like container_of() to find another field. However, this is made more
> difficult by the fact that the sgx_enclave_exit_handler_t is not
> really using the x86_64 sysv calling convention. Therefore, the
> sgx_enclave_exit_handler_t MUST be written in assembly.

What bits of the x86-64 ABI require writing the handler in assembly?  There
are certainly restrictions on what the handler can do without needing an
assembly trampoline, but I was under the impression that vanilla C code is
compatible with the exit handler patch.  Is Rust more picky about calling
convention?

Side topic, the documentation for vdso_sgx_enter_enclave_t is wrong, it
states the EFLAGS.DF is not cleared before invoking the handler, but that's
a lie.

> This also implies that we can't use techniques like container_of() and must
> calculate all the offsets manually, which is tedious, error prone and
> fragile.
> 
> This is further complicated by the fact that I'm using Rust (as are a
> number of other consumers), which has no native offsetof support (yes,
> there is a crate for it, but it requires a number of complex
> strategies to defeat the compiler which aren't great) and therefore no
> container_of() support.
> 
> We could design a standard struct for this (similar to struct
> kvm_run). But in order to keep performance in check we'd have to
> define a limited ABI surface (to avoid things like xsave) which
> wouldn't have the full flexibility of the current approach. This would
> allow for a kernel provided vDSO function with a normal calling
> convention, however (which does have some non-trivial value). I think
> this is a trade-off we should consider (perhaps making it optional?).
> 
> But at the least, allowing a pass-through void *misc would reduce the
> complexity of the assembly calculations.

I'm not opposed to adding a pass-through param, it's literally one line
and an extra PUSH <mem> in the exit handler path. 

Another thought would be to wrap sgx_enclave_exception in a struct to give
room for supporting additional exit information (if such a thing ever pops
up) and to allow the caller to opt in to select behavior, e.g. Jethro's
request to invoke the exit handler on IRQ exits.  This is basically the
equivalent of "struct kvm_run", minus the vCPU/enclave state.

Such a struct could also be used to avoid using -EFAULT for the "fault in
enclave" exit path, which I believe Andy isn't a fan of, by having an
explicit "exit_reason" field with arbitrary, dedicated exit codes, and
defining "success" as making it to ENCLU, i.e. returning '0' when there is
no exit handler if ENCLU is attempted.

E.g.:

struct sgx_enter_enclave {
        __u64 tcs;
        __u64 flags;

	__u32 exit_leaf;	/* output only */
	__u32 exit_reason;

	__u64 user_handler;
	__u64 user_data;

        union {
                struct sgx_enclave_exception {
                        __u16 trapnr;
                        __u16 error_code;
                        __u32 reserved32;
                        __u64 address;
                };

                __u8 pad[256];  /* 100% arbitrary */
        };
}

typedef int (*vdso_sgx_enter_enclave_t)(unsigned long rdi, unsigned long rsi,
                                        unsigned long rdx, unsigned int leaf,
                                        unsigned long r8,  unsigned long r9,
                                        struct sgx_enter_enclave *e);


The exit handler could then be:

typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx,
                                          long ursp, long r8, long r9,
                                          struct sgx_enter_enclave *e);

or if Rust doesn't like casting user_data:

typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx,
                                          long ursp, long r8, long r9,
                                          struct sgx_enter_enclave *e
					  void *user_data);

  reply index

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-16 13:52 [PATCH v36 00/24] Intel SGX foundations Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 01/24] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen
2020-08-06 13:13   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 02/24] x86/cpufeatures: x86/msr: Add Intel SGX Launch Control " Jarkko Sakkinen
2020-08-06 13:14   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 03/24] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen
2020-08-06 13:16   ` Darren Kenny
2020-08-20 15:31   ` Borislav Petkov
2020-08-21 17:35     ` Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 04/24] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen
2020-08-06 13:14   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 05/24] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2020-08-07  9:37   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 06/24] x86/cpu/intel: Detect SGX support Jarkko Sakkinen
2020-08-06 13:17   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 07/24] x86/cpu/intel: Add nosgx kernel parameter Jarkko Sakkinen
2020-08-06 13:18   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 08/24] x86/sgx: Initialize metadata for Enclave Page Cache (EPC) sections Jarkko Sakkinen
2020-08-06 13:27   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 09/24] x86/sgx: Add __sgx_alloc_epc_page() and sgx_free_epc_page() Jarkko Sakkinen
2020-08-06 13:29   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 10/24] mm: Add vm_ops->mprotect() Jarkko Sakkinen
2020-08-06 13:35   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 11/24] x86/sgx: Add SGX enclave driver Jarkko Sakkinen
2020-08-06 13:59   ` Darren Kenny
2020-08-25 16:44   ` Borislav Petkov
2020-08-26 13:46     ` Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 12/24] x86/sgx: Add SGX_IOC_ENCLAVE_CREATE Jarkko Sakkinen
2020-08-06 15:40   ` Darren Kenny
2020-08-26 14:52   ` Borislav Petkov
2020-08-27 13:24     ` Jarkko Sakkinen
2020-08-27 16:15       ` Borislav Petkov
2020-08-28 23:39         ` Jarkko Sakkinen
2020-08-29  0:21       ` Jarkko Sakkinen
2020-09-01 16:41   ` Haitao Huang
2020-09-04 11:55     ` Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 13/24] x86/sgx: Add SGX_IOC_ENCLAVE_ADD_PAGES Jarkko Sakkinen
2020-08-06 16:29   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 14/24] x86/sgx: Add SGX_IOC_ENCLAVE_INIT Jarkko Sakkinen
2020-08-06 16:40   ` Darren Kenny
2020-07-16 13:52 ` [PATCH v36 15/24] x86/sgx: Allow a limited use of ATTRIBUTE.PROVISIONKEY for attestation Jarkko Sakkinen
2020-08-06 17:00   ` Darren Kenny
2020-08-18 13:30     ` Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 16/24] x86/sgx: Add a page reclaimer Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 17/24] x86/sgx: ptrace() support for the SGX driver Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 18/24] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 19/24] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2020-07-16 13:52 ` [PATCH v36 20/24] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2020-07-16 13:53 ` [PATCH v36 21/24] x86/vdso: Implement a vDSO for Intel SGX enclave call Jarkko Sakkinen
2020-08-06 14:55   ` Nathaniel McCallum
2020-08-10 22:23     ` Sean Christopherson [this message]
2020-08-11  7:16       ` Jethro Beekman
2020-08-11 14:54         ` Sean Christopherson
2020-08-18 14:52       ` Jarkko Sakkinen
2020-08-18 15:06         ` Jarkko Sakkinen
2020-08-18 15:15           ` Nathaniel McCallum
2020-08-18 16:43             ` Jarkko Sakkinen
2020-08-19 13:33               ` Nathaniel McCallum
2020-08-19 14:00                 ` Jethro Beekman
2020-08-19 21:23                 ` Jarkko Sakkinen
2020-08-10 23:08     ` Andy Lutomirski
2020-08-10 23:48       ` Sean Christopherson
2020-08-11  0:52         ` Andy Lutomirski
2020-08-11 15:16           ` Andy Lutomirski
2020-08-13 19:38             ` Sean Christopherson
2020-08-17 13:12       ` Nathaniel McCallum
2020-08-17 15:01         ` Andy Lutomirski
2020-08-18 15:15       ` Jarkko Sakkinen
2020-08-20  0:19         ` Andy Lutomirski
2020-08-18 14:26     ` Jarkko Sakkinen
2020-07-16 13:53 ` [PATCH v36 22/24] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2020-08-27  4:47   ` Nathaniel McCallum
2020-08-27 15:20     ` Sean Christopherson
2020-08-28 23:27       ` Jarkko Sakkinen
2020-07-16 13:53 ` [PATCH v36 23/24] docs: x86/sgx: Document SGX micro architecture and kernel internals Jarkko Sakkinen
2020-07-28 21:35   ` Pavel Machek
2020-08-06 10:21     ` Dr. Greg
2020-08-08 22:18       ` Pavel Machek
2020-08-19 20:55     ` Jarkko Sakkinen
2020-07-16 13:53 ` [PATCH v36 24/24] x86/sgx: Update MAINTAINERS Jarkko Sakkinen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200810222317.GG14724@linux.intel.com \
    --to=sean.j.christopherson@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=asapek@google.com \
    --cc=bp@alien8.de \
    --cc=cedric.xing@intel.com \
    --cc=chenalexchen@google.com \
    --cc=conradparker@google.com \
    --cc=cyhanish@google.com \
    --cc=dave.hansen@intel.com \
    --cc=haitao.huang@intel.com \
    --cc=jarkko.sakkinen@linux.intel.com \
    --cc=jethro@fortanix.com \
    --cc=josh@joshtriplett.org \
    --cc=kai.huang@intel.com \
    --cc=kai.svahn@intel.com \
    --cc=kmoy@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-sgx@vger.kernel.org \
    --cc=ludloff@google.com \
    --cc=luto@amacapital.net \
    --cc=luto@kernel.org \
    --cc=nhorman@redhat.com \
    --cc=npmccallum@redhat.com \
    --cc=puiterwijk@redhat.com \
    --cc=rientjes@google.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    --cc=yaozhangx@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Sgx Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-sgx/0 linux-sgx/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-sgx linux-sgx/ https://lore.kernel.org/linux-sgx \
		linux-sgx@vger.kernel.org
	public-inbox-index linux-sgx

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-sgx


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git