From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_PASS,T_DKIMWL_WL_HIGH,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F7CAC65C22 for ; Fri, 2 Nov 2018 17:16:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DD41920831 for ; Fri, 2 Nov 2018 17:16:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="2M0FlpJH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD41920831 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728044AbeKCCYI (ORCPT ); Fri, 2 Nov 2018 22:24:08 -0400 Received: from mail.kernel.org ([198.145.29.99]:39742 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727859AbeKCCYI (ORCPT ); Fri, 2 Nov 2018 22:24:08 -0400 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1FFA12087A for ; Fri, 2 Nov 2018 17:16:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1541178976; bh=BS5f9ZQlmBWVfhhyRJ68tO2jQTR8yTfLegJeHBMczWk=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=2M0FlpJH731Rnx0iZkOjQKz5YjqvjW5Aubpr2sPVGYWFDqKS1wP31eJ2ftY178yQK cWkMMO1V1OnbwkkV6VNKTrdQhsqJ8wibkd6QCT6LEwKRdOleEm49phiilvrrB5oclr dmSmKwVLN0vYgKwO9tJIDAH/pMgVYexytHZxiOU8= Received: by mail-wm1-f50.google.com with SMTP id f1-v6so1651394wmg.1 for ; Fri, 02 Nov 2018 10:16:16 -0700 (PDT) X-Gm-Message-State: AGRZ1gJN81IR7a3iCz8uGbbFCWq1W4NS0LxbTuOznLNFMvCVg3Ezb5OK IMQ5+xQtRdAoZrT+ReG7k+Nu79xJ7UkNYF833rUvwg== X-Google-Smtp-Source: AJdET5diC8EDhjOonsIYn/AMl15Vhm4bsEAdeE4ktLY7wiHe4Oo9hMpIvgCnCDiKrQFJFoR5kCkdoNst374IyklkCS4= X-Received: by 2002:a1c:d4b:: with SMTP id 72-v6mr108795wmn.102.1541178974377; Fri, 02 Nov 2018 10:16:14 -0700 (PDT) MIME-Version: 1.0 References: <20181101185225.GC5150@brightrain.aerifal.cx> <20181101193107.GE5150@brightrain.aerifal.cx> <20181102163034.GB7393@linux.intel.com> <7e14ee0e-ce15-1e88-7ae9-4d0f40cb3d84@fortanix.com> <20181102165204.GC7393@linux.intel.com> <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> In-Reply-To: <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> From: Andy Lutomirski Date: Fri, 2 Nov 2018 10:16:02 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: RFC: userspace exception fixups To: Jethro Beekman Cc: Andrew Lutomirski , "Christopherson, Sean J" , Linus Torvalds , Rich Felker , Jann Horn , Dave Hansen , Jarkko Sakkinen , Florian Weimer , Linux API , X86 ML , linux-arch , LKML , Peter Zijlstra , nhorman@redhat.com, npmccallum@redhat.com, "Ayoun, Serge" , shay.katz-zamir@intel.com, linux-sgx@vger.kernel.org, Andy Shevchenko , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "Carlos O'Donell" , adhemerval.zanella@linaro.org Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Nov 2, 2018 at 10:05 AM Jethro Beekman wrote: > > On 2018-11-02 10:01, Andy Lutomirski wrote: > > On Fri, Nov 2, 2018 at 9:56 AM Jethro Beekman wrote: > >> > >> On 2018-11-02 09:52, Sean Christopherson wrote: > >>> On Fri, Nov 02, 2018 at 04:37:10PM +0000, Jethro Beekman wrote: > >>>> On 2018-11-02 09:30, Sean Christopherson wrote: > >>>>> ... The intended convention for EENTER is to have an ENCLU at the AEX target ... > >>>>> > >>>>> ... to further enforce that the AEX target needs to be ENCLU. > >>>> > >>>> Some SGX runtimes may want to use a different AEX target. > >>> > >>> To what end? Userspace gets no indication as to why the AEX occurred. > >>> And if exceptions are getting transfered to userspace the trampoline > >>> would effectively be handling only INTR, NMI, #MC and EPC #PF. > >>> > >> > >> Various reasons... > >> > >> Userspace may have established an exception handling convention with the > >> enclave (by setting TCS.NSSA > 1) and may want to call EENTER instead of > >> ERESUME. > >> > > > > Ugh, > > > > I sincerely hope that a future ISA extension lets the kernel return > > directly back to enclave mode so that AEX events become entirely > > invisible to user code. > > Can you explain how this would work for things like #BR/#DE/#UD that > need to be fixed up by code running in the enclave before it can be resumed? > Sure. A better enclave entry function would complete in one of two ways: 1. The enclave exited normally. Some register output would indicate this. 2. The enclave existed due to an exception or interrupt. The kernel would be entered directly and notified of what happened. The kernel would fix it up if needed (#PF), handle an interrupt (for en enclave exit due to an interrupt) and reenter the enclave. If, of the error is not kernel-fixable-up, it would return back to userspace with some explanation of what happened. Kind of like normal user code. Alternatively, the CPU could directly distinguish between exceptions that need the enclave's attention (#BR) and those that don't. The fact that user code is involved in resuming an enclave when a hardware interrupt occurs is silly IMO. From mboxrd@z Thu Jan 1 00:00:00 1970 References: <20181101185225.GC5150@brightrain.aerifal.cx> <20181101193107.GE5150@brightrain.aerifal.cx> <20181102163034.GB7393@linux.intel.com> <7e14ee0e-ce15-1e88-7ae9-4d0f40cb3d84@fortanix.com> <20181102165204.GC7393@linux.intel.com> <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> In-Reply-To: <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> From: Andy Lutomirski Date: Fri, 2 Nov 2018 10:16:02 -0700 Message-ID: Subject: Re: RFC: userspace exception fixups To: Jethro Beekman CC: Andrew Lutomirski , "Christopherson, Sean J" , Linus Torvalds , Rich Felker , Jann Horn , Dave Hansen , "Jarkko Sakkinen" , Florian Weimer , Linux API , X86 ML , linux-arch , LKML , Peter Zijlstra , , , "Ayoun, Serge" , , , Andy Shevchenko , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "Carlos O'Donell" , Content-Type: text/plain; charset="UTF-8" Return-Path: luto@kernel.org MIME-Version: 1.0 List-ID: On Fri, Nov 2, 2018 at 10:05 AM Jethro Beekman wrote: > > On 2018-11-02 10:01, Andy Lutomirski wrote: > > On Fri, Nov 2, 2018 at 9:56 AM Jethro Beekman wrote: > >> > >> On 2018-11-02 09:52, Sean Christopherson wrote: > >>> On Fri, Nov 02, 2018 at 04:37:10PM +0000, Jethro Beekman wrote: > >>>> On 2018-11-02 09:30, Sean Christopherson wrote: > >>>>> ... The intended convention for EENTER is to have an ENCLU at the AEX target ... > >>>>> > >>>>> ... to further enforce that the AEX target needs to be ENCLU. > >>>> > >>>> Some SGX runtimes may want to use a different AEX target. > >>> > >>> To what end? Userspace gets no indication as to why the AEX occurred. > >>> And if exceptions are getting transfered to userspace the trampoline > >>> would effectively be handling only INTR, NMI, #MC and EPC #PF. > >>> > >> > >> Various reasons... > >> > >> Userspace may have established an exception handling convention with the > >> enclave (by setting TCS.NSSA > 1) and may want to call EENTER instead of > >> ERESUME. > >> > > > > Ugh, > > > > I sincerely hope that a future ISA extension lets the kernel return > > directly back to enclave mode so that AEX events become entirely > > invisible to user code. > > Can you explain how this would work for things like #BR/#DE/#UD that > need to be fixed up by code running in the enclave before it can be resumed? > Sure. A better enclave entry function would complete in one of two ways: 1. The enclave exited normally. Some register output would indicate this. 2. The enclave existed due to an exception or interrupt. The kernel would be entered directly and notified of what happened. The kernel would fix it up if needed (#PF), handle an interrupt (for en enclave exit due to an interrupt) and reenter the enclave. If, of the error is not kernel-fixable-up, it would return back to userspace with some explanation of what happened. Kind of like normal user code. Alternatively, the CPU could directly distinguish between exceptions that need the enclave's attention (#BR) and those that don't. The fact that user code is involved in resuming an enclave when a hardware interrupt occurs is silly IMO. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andy Lutomirski Subject: Re: RFC: userspace exception fixups Date: Fri, 2 Nov 2018 10:16:02 -0700 Message-ID: References: <20181101185225.GC5150@brightrain.aerifal.cx> <20181101193107.GE5150@brightrain.aerifal.cx> <20181102163034.GB7393@linux.intel.com> <7e14ee0e-ce15-1e88-7ae9-4d0f40cb3d84@fortanix.com> <20181102165204.GC7393@linux.intel.com> <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: In-Reply-To: <1b87048e-7ed8-14a1-572f-3cd825319f8c@fortanix.com> Sender: linux-kernel-owner@vger.kernel.org To: Jethro Beekman Cc: Andrew Lutomirski , "Christopherson, Sean J" , Linus Torvalds , Rich Felker , Jann Horn , Dave Hansen , Jarkko Sakkinen , Florian Weimer , Linux API , X86 ML , linux-arch , LKML , Peter Zijlstra , nhorman@redhat.com, npmccallum@redhat.com, "Ayoun, Serge" , shay.katz-zamir@intel.com, linux-sgx@vger.kernel.org, Andy Shevchenko , Thomas Gleixner , Ingo Molnar List-Id: linux-api@vger.kernel.org On Fri, Nov 2, 2018 at 10:05 AM Jethro Beekman wrote: > > On 2018-11-02 10:01, Andy Lutomirski wrote: > > On Fri, Nov 2, 2018 at 9:56 AM Jethro Beekman wrote: > >> > >> On 2018-11-02 09:52, Sean Christopherson wrote: > >>> On Fri, Nov 02, 2018 at 04:37:10PM +0000, Jethro Beekman wrote: > >>>> On 2018-11-02 09:30, Sean Christopherson wrote: > >>>>> ... The intended convention for EENTER is to have an ENCLU at the AEX target ... > >>>>> > >>>>> ... to further enforce that the AEX target needs to be ENCLU. > >>>> > >>>> Some SGX runtimes may want to use a different AEX target. > >>> > >>> To what end? Userspace gets no indication as to why the AEX occurred. > >>> And if exceptions are getting transfered to userspace the trampoline > >>> would effectively be handling only INTR, NMI, #MC and EPC #PF. > >>> > >> > >> Various reasons... > >> > >> Userspace may have established an exception handling convention with the > >> enclave (by setting TCS.NSSA > 1) and may want to call EENTER instead of > >> ERESUME. > >> > > > > Ugh, > > > > I sincerely hope that a future ISA extension lets the kernel return > > directly back to enclave mode so that AEX events become entirely > > invisible to user code. > > Can you explain how this would work for things like #BR/#DE/#UD that > need to be fixed up by code running in the enclave before it can be resumed? > Sure. A better enclave entry function would complete in one of two ways: 1. The enclave exited normally. Some register output would indicate this. 2. The enclave existed due to an exception or interrupt. The kernel would be entered directly and notified of what happened. The kernel would fix it up if needed (#PF), handle an interrupt (for en enclave exit due to an interrupt) and reenter the enclave. If, of the error is not kernel-fixable-up, it would return back to userspace with some explanation of what happened. Kind of like normal user code. Alternatively, the CPU could directly distinguish between exceptions that need the enclave's attention (#BR) and those that don't. The fact that user code is involved in resuming an enclave when a hardware interrupt occurs is silly IMO.