From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Fri, 2 Nov 2018 13:32:05 -0400 From: Rich Felker To: Andy Lutomirski CC: Jethro Beekman , "Christopherson, Sean J" , Linus Torvalds , Jann Horn , Dave Hansen , Jarkko Sakkinen , Florian Weimer , Linux API , X86 ML , linux-arch , LKML , "Peter Zijlstra" , , , "Ayoun, Serge" , , , Andy Shevchenko , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "Carlos O'Donell" , Subject: Re: RFC: userspace exception fixups Message-ID: <20181102173205.GM5150@brightrain.aerifal.cx> References: <20181101193107.GE5150@brightrain.aerifal.cx> <20181102163034.GB7393@linux.intel.com> <7e14ee0e-ce15-1e88-7ae9-4d0f40cb3d84@fortanix.com> <20181102165204.GC7393@linux.intel.com> <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> Content-Type: text/plain; charset="us-ascii" In-Reply-To: Sender: Rich Felker Return-Path: dalias@aerifal.cx MIME-Version: 1.0 List-ID: On Fri, Nov 02, 2018 at 10:16:02AM -0700, Andy Lutomirski wrote: > On Fri, Nov 2, 2018 at 10:05 AM Jethro Beekman wrote: > > > > On 2018-11-02 10:01, Andy Lutomirski wrote: > > > On Fri, Nov 2, 2018 at 9:56 AM Jethro Beekman wrote: > > >> > > >> On 2018-11-02 09:52, Sean Christopherson wrote: > > >>> On Fri, Nov 02, 2018 at 04:37:10PM +0000, Jethro Beekman wrote: > > >>>> On 2018-11-02 09:30, Sean Christopherson wrote: > > >>>>> ... The intended convention for EENTER is to have an ENCLU at the AEX target ... > > >>>>> > > >>>>> ... to further enforce that the AEX target needs to be ENCLU. > > >>>> > > >>>> Some SGX runtimes may want to use a different AEX target. > > >>> > > >>> To what end? Userspace gets no indication as to why the AEX occurred. > > >>> And if exceptions are getting transfered to userspace the trampoline > > >>> would effectively be handling only INTR, NMI, #MC and EPC #PF. > > >>> > > >> > > >> Various reasons... > > >> > > >> Userspace may have established an exception handling convention with the > > >> enclave (by setting TCS.NSSA > 1) and may want to call EENTER instead of > > >> ERESUME. > > >> > > > > > > Ugh, > > > > > > I sincerely hope that a future ISA extension lets the kernel return > > > directly back to enclave mode so that AEX events become entirely > > > invisible to user code. > > > > Can you explain how this would work for things like #BR/#DE/#UD that > > need to be fixed up by code running in the enclave before it can be resumed? > > > > Sure. A better enclave entry function would complete in one of two ways: > > 1. The enclave exited normally. Some register output would indicate this. > > 2. The enclave existed due to an exception or interrupt. The kernel > would be entered directly and notified of what happened. The kernel > would fix it up if needed (#PF), handle an interrupt (for en enclave > exit due to an interrupt) and reenter the enclave. If, of the error > is not kernel-fixable-up, it would return back to userspace with some > explanation of what happened. Kind of like normal user code. > > Alternatively, the CPU could directly distinguish between exceptions > that need the enclave's attention (#BR) and those that don't. > > The fact that user code is involved in resuming an enclave when a > hardware interrupt occurs is silly IMO. Agreed absolutely. If this is necessary, it seems like there should be an agreed-upon protocol such that the kernel can make it happen via returning to code in the vdso that performs the actual resume, so that the application never sees it. Rich From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 710CBC65C22 for ; Fri, 2 Nov 2018 17:34:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 45F1A2082D for ; Fri, 2 Nov 2018 17:34:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 45F1A2082D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=libc.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-sgx-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728113AbeKCCm3 (ORCPT ); Fri, 2 Nov 2018 22:42:29 -0400 Received: from 216-12-86-13.cv.mvl.ntelos.net ([216.12.86.13]:58004 "EHLO brightrain.aerifal.cx" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726085AbeKCCm3 (ORCPT ); Fri, 2 Nov 2018 22:42:29 -0400 Received: from dalias by brightrain.aerifal.cx with local (Exim 3.15 #2) id 1gIdIj-0000Fq-00; Fri, 02 Nov 2018 17:32:05 +0000 Date: Fri, 2 Nov 2018 13:32:05 -0400 From: Rich Felker To: Andy Lutomirski Cc: Jethro Beekman , "Christopherson, Sean J" , Linus Torvalds , Jann Horn , Dave Hansen , Jarkko Sakkinen , Florian Weimer , Linux API , X86 ML , linux-arch , LKML , Peter Zijlstra , nhorman@redhat.com, npmccallum@redhat.com, "Ayoun, Serge" , shay.katz-zamir@intel.com, linux-sgx@vger.kernel.org, Andy Shevchenko , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Carlos O'Donell , adhemerval.zanella@linaro.org Subject: Re: RFC: userspace exception fixups Message-ID: <20181102173205.GM5150@brightrain.aerifal.cx> References: <20181101193107.GE5150@brightrain.aerifal.cx> <20181102163034.GB7393@linux.intel.com> <7e14ee0e-ce15-1e88-7ae9-4d0f40cb3d84@fortanix.com> <20181102165204.GC7393@linux.intel.com> <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Message-ID: <20181102173205.c5hPxAm7wbOkqGjZP96TmGrALotqRf-K5kgE-Q283HA@z> On Fri, Nov 02, 2018 at 10:16:02AM -0700, Andy Lutomirski wrote: > On Fri, Nov 2, 2018 at 10:05 AM Jethro Beekman wrote: > > > > On 2018-11-02 10:01, Andy Lutomirski wrote: > > > On Fri, Nov 2, 2018 at 9:56 AM Jethro Beekman wrote: > > >> > > >> On 2018-11-02 09:52, Sean Christopherson wrote: > > >>> On Fri, Nov 02, 2018 at 04:37:10PM +0000, Jethro Beekman wrote: > > >>>> On 2018-11-02 09:30, Sean Christopherson wrote: > > >>>>> ... The intended convention for EENTER is to have an ENCLU at the AEX target ... > > >>>>> > > >>>>> ... to further enforce that the AEX target needs to be ENCLU. > > >>>> > > >>>> Some SGX runtimes may want to use a different AEX target. > > >>> > > >>> To what end? Userspace gets no indication as to why the AEX occurred. > > >>> And if exceptions are getting transfered to userspace the trampoline > > >>> would effectively be handling only INTR, NMI, #MC and EPC #PF. > > >>> > > >> > > >> Various reasons... > > >> > > >> Userspace may have established an exception handling convention with the > > >> enclave (by setting TCS.NSSA > 1) and may want to call EENTER instead of > > >> ERESUME. > > >> > > > > > > Ugh, > > > > > > I sincerely hope that a future ISA extension lets the kernel return > > > directly back to enclave mode so that AEX events become entirely > > > invisible to user code. > > > > Can you explain how this would work for things like #BR/#DE/#UD that > > need to be fixed up by code running in the enclave before it can be resumed? > > > > Sure. A better enclave entry function would complete in one of two ways: > > 1. The enclave exited normally. Some register output would indicate this. > > 2. The enclave existed due to an exception or interrupt. The kernel > would be entered directly and notified of what happened. The kernel > would fix it up if needed (#PF), handle an interrupt (for en enclave > exit due to an interrupt) and reenter the enclave. If, of the error > is not kernel-fixable-up, it would return back to userspace with some > explanation of what happened. Kind of like normal user code. > > Alternatively, the CPU could directly distinguish between exceptions > that need the enclave's attention (#BR) and those that don't. > > The fact that user code is involved in resuming an enclave when a > hardware interrupt occurs is silly IMO. Agreed absolutely. If this is necessary, it seems like there should be an agreed-upon protocol such that the kernel can make it happen via returning to code in the vdso that performs the actual resume, so that the application never sees it. Rich