From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E51A3C43612 for ; Tue, 18 Dec 2018 03:27:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD56D2145D for ; Tue, 18 Dec 2018 03:27:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726518AbeLRD1O (ORCPT ); Mon, 17 Dec 2018 22:27:14 -0500 Received: from mga14.intel.com ([192.55.52.115]:21605 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726305AbeLRD1O (ORCPT ); Mon, 17 Dec 2018 22:27:14 -0500 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Dec 2018 19:27:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,367,1539673200"; d="scan'208";a="303033774" Received: from zhangxi4-mobl5.ccr.corp.intel.com (HELO localhost) ([10.249.254.207]) by fmsmga006.fm.intel.com with ESMTP; 17 Dec 2018 19:27:03 -0800 Date: Tue, 18 Dec 2018 05:27:02 +0200 From: Jarkko Sakkinen To: Sean Christopherson Cc: Andy Lutomirski , Dave Hansen , X86 ML , Platform Driver , linux-sgx@vger.kernel.org, nhorman@redhat.com, npmccallum@redhat.com, "Ayoun, Serge" , shay.katz-zamir@intel.com, Haitao Huang , Andy Shevchenko , Thomas Gleixner , "Svahn, Kai" , mark.shanahan@intel.com, Suresh Siddha , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Darren Hart , Andy Shevchenko , "open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Subject: Re: [PATCH v17 18/23] platform/x86: Intel SGX driver Message-ID: <20181218032702.GA2903@linux.intel.com> References: <20181116010412.23967-1-jarkko.sakkinen@linux.intel.com> <20181116010412.23967-19-jarkko.sakkinen@linux.intel.com> <7d5cde02-4649-546b-0f03-2d6414bb80b5@intel.com> <20181217180102.GA12560@linux.intel.com> <20181217183613.GD12491@linux.intel.com> <20181217184333.GA26920@linux.intel.com> <20181217222047.GG12491@linux.intel.com> <20181218013918.GC333@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20181218013918.GC333@linux.intel.com> Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 18, 2018 at 03:39:18AM +0200, Jarkko Sakkinen wrote: > On Mon, Dec 17, 2018 at 02:20:48PM -0800, Sean Christopherson wrote: > > The only potential hiccup I can see is the build flow. Currently, > > EADD+EEXTEND is done via a work queue to avoid major performance issues > > (10x regression) when userspace is building multiple enclaves in parallel > > using goroutines to wrap Cgo (the issue might apply to any M:N scheduler, > > but I've only confirmed the Golang case). The issue is that allocating > > an EPC page acts like a blocking syscall when the EPC is under pressure, > > i.e. an EPC page isn't immediately available. This causes Go's scheduler > > to thrash and tank performance[1]. > > I don't see any major issues having that kthread. All the code that > maps the enclave would be removed. > > I would only allow to map enclave to process address space after the > enclave has been initialized i.e. SGX_IOC_ENCLAVE_ATTACH. Some refined thoughts. PTE insertion can done in the #PF handler. In fact, we can PoC this already with the current architecture (and I will right after sending v18). The backing space is a bit more nasty issue in the add pager thread. The previous shmem swapping would have been a better fit. Maybe that should be reconsidered? If shmem was used, all the commits up to "SGX Enclave Driver" could be reworked to the new model. When we think about the swapping code, there uprises some difficulties. Namely, when a page is swapped, the enclave must unmap the PTE from all processes that have it mapped. I have a one compromise solution for the problem above: make enclaves shared BUT mutually exclusive. When you attach an enclave it gets detached from the previous process that had it. This would still fully implement the daemon example that Andy gave in earlier response. /Jarkko