From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CEEAC4332F for ; Fri, 18 Nov 2022 01:36:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30A7A6B0075; Thu, 17 Nov 2022 20:36:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E1C58E0001; Thu, 17 Nov 2022 20:36:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 182F86B007B; Thu, 17 Nov 2022 20:36:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 095C76B0075 for ; Thu, 17 Nov 2022 20:36:40 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D946580279 for ; Fri, 18 Nov 2022 01:36:39 +0000 (UTC) X-FDA: 80144848518.15.E60F83C Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf22.hostedemail.com (Postfix) with ESMTP id 4580CC0010 for ; Fri, 18 Nov 2022 01:36:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668735398; x=1700271398; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:content-transfer-encoding:in-reply-to; bh=PGqeIE9D9mQ0SzWgaCrOjd5GhXHcGRQuoepeW+csvHw=; b=K6hcVBAfU28dnlb8quqk/4hPn8anjPxcyX3C1ZT3GF4s7716cb+ivkQ8 /9Sv/dVu+5P3GFM8bczkj9WJQuP12MtIFI3uPj0YVZk9MCCJW17b/k7uX +qcC+qIVX7/HQZuIT8spJsHt2J9n1X0YE6Ew9qGtMRkEhoLSIsomxrsAB aidT0Of+hRBtPXEin3disVRZdWHB81pYkx4+oPHIqG18JmsOgsWjfjPiY 8aCyLsQpwX6VVnBo3HEhM+3Vusz/cFqkIEtRUS2PD3//cFV9BhlBMVq2s BaLk5VhwusPdKZ2Ud4UXmbnMzspOQgn4xmQxHKP+FL6L/jzX9xXhIPp4i A==; X-IronPort-AV: E=McAfee;i="6500,9779,10534"; a="314843077" X-IronPort-AV: E=Sophos;i="5.96,172,1665471600"; d="scan'208";a="314843077" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Nov 2022 17:36:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10534"; a="703558926" X-IronPort-AV: E=Sophos;i="5.96,172,1665471600"; d="scan'208";a="703558926" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.193.75]) by fmsmga008.fm.intel.com with ESMTP; 17 Nov 2022 17:36:26 -0800 Date: Fri, 18 Nov 2022 09:32:01 +0800 From: Chao Peng To: Alex =?iso-8859-1?Q?Benn=E9e?= Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , tabba@google.com, Michael Roth , mhocko@suse.com, Muchun Song , wei.w.wang@intel.com Subject: Re: [PATCH v9 3/8] KVM: Add KVM_EXIT_MEMORY_FAULT exit Message-ID: <20221118013201.GA456562@chaop.bj.intel.com> Reply-To: Chao Peng References: <20221025151344.3784230-1-chao.p.peng@linux.intel.com> <20221025151344.3784230-4-chao.p.peng@linux.intel.com> <87cz9o9mr8.fsf@linaro.org> <20221116031441.GA364614@chaop.bj.intel.com> <87mt8q90rw.fsf@linaro.org> <20221117134520.GD422408@chaop.bj.intel.com> <87a64p8vof.fsf@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87a64p8vof.fsf@linaro.org> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668735398; a=rsa-sha256; cv=none; b=bXqSwn1nZkxMU/rILaWdwye/TBkMg0VhfIP1d/9jqapECdo6X+CWPBjH0FnvLTjcaMiYhJ ijr5tYLNebmpwUITLZW5YS0dO5KfxSKcfALKEwX70lOdhyeJEIBPGdKEl9FK4rnhm2U8+U OUBZdgwQETtqslLgfVsGj0woQHOAsYA= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=K6hcVBAf; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none); spf=none (imf22.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.65) smtp.mailfrom=chao.p.peng@linux.intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668735398; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NKRVqbiOtTg8V2ts5H1w/J0rs2e0Q+g/sNIXagqZYBU=; b=KHpAwmVdRhUXPyoZp1BoigPgoFQrEUm1lp/wVH2lvChGOBGP4dZoUW4CzzXAkxLsR+Ia73 sOxTn6qP1xAmnlSwqDmmKBXmgkO6G/o8THegaPcziM6jkFcgmUk62tVzq9DzSKUev9uzTg hOazZgdKJqRgNoI0+3WpwaF8RWJ0Qtk= X-Stat-Signature: cho7c4834w63u4g6cw78rjekeshaam8u X-Rspamd-Queue-Id: 4580CC0010 X-Rspam-User: Authentication-Results: imf22.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=K6hcVBAf; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none); spf=none (imf22.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.65) smtp.mailfrom=chao.p.peng@linux.intel.com X-Rspamd-Server: rspam09 X-HE-Tag: 1668735398-348475 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Nov 17, 2022 at 03:08:17PM +0000, Alex Bennée wrote: > > Chao Peng writes: > > > On Wed, Nov 16, 2022 at 07:03:49PM +0000, Alex Bennée wrote: > >> > >> Chao Peng writes: > >> > >> > On Tue, Nov 15, 2022 at 04:56:12PM +0000, Alex Bennée wrote: > >> >> > >> >> Chao Peng writes: > >> >> > >> >> > This new KVM exit allows userspace to handle memory-related errors. It > >> >> > indicates an error happens in KVM at guest memory range [gpa, gpa+size). > >> >> > The flags includes additional information for userspace to handle the > >> >> > error. Currently bit 0 is defined as 'private memory' where '1' > >> >> > indicates error happens due to private memory access and '0' indicates > >> >> > error happens due to shared memory access. > >> >> > > >> >> > When private memory is enabled, this new exit will be used for KVM to > >> >> > exit to userspace for shared <-> private memory conversion in memory > >> >> > encryption usage. In such usage, typically there are two kind of memory > >> >> > conversions: > >> >> > - explicit conversion: happens when guest explicitly calls into KVM > >> >> > to map a range (as private or shared), KVM then exits to userspace > >> >> > to perform the map/unmap operations. > >> >> > - implicit conversion: happens in KVM page fault handler where KVM > >> >> > exits to userspace for an implicit conversion when the page is in a > >> >> > different state than requested (private or shared). > >> >> > > >> >> > Suggested-by: Sean Christopherson > >> >> > Co-developed-by: Yu Zhang > >> >> > Signed-off-by: Yu Zhang > >> >> > Signed-off-by: Chao Peng > >> >> > --- > >> >> > Documentation/virt/kvm/api.rst | 23 +++++++++++++++++++++++ > >> >> > include/uapi/linux/kvm.h | 9 +++++++++ > >> >> > 2 files changed, 32 insertions(+) > >> >> > > >> >> > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > >> >> > index f3fa75649a78..975688912b8c 100644 > >> >> > --- a/Documentation/virt/kvm/api.rst > >> >> > +++ b/Documentation/virt/kvm/api.rst > >> >> > @@ -6537,6 +6537,29 @@ array field represents return values. The userspace should update the return > >> >> > values of SBI call before resuming the VCPU. For more details on RISC-V SBI > >> >> > spec refer, https://github.com/riscv/riscv-sbi-doc. > >> >> > > >> >> > +:: > >> >> > + > >> >> > + /* KVM_EXIT_MEMORY_FAULT */ > >> >> > + struct { > >> >> > + #define KVM_MEMORY_EXIT_FLAG_PRIVATE (1 << 0) > >> >> > + __u32 flags; > >> >> > + __u32 padding; > >> >> > + __u64 gpa; > >> >> > + __u64 size; > >> >> > + } memory; > >> >> > + > >> >> > +If exit reason is KVM_EXIT_MEMORY_FAULT then it indicates that the VCPU has > >> >> > +encountered a memory error which is not handled by KVM kernel module and > >> >> > +userspace may choose to handle it. The 'flags' field indicates the memory > >> >> > +properties of the exit. > >> >> > + > >> >> > + - KVM_MEMORY_EXIT_FLAG_PRIVATE - indicates the memory error is caused by > >> >> > + private memory access when the bit is set. Otherwise the memory error is > >> >> > + caused by shared memory access when the bit is clear. > >> >> > >> >> What does a shared memory access failure entail? > >> > > >> > In the context of confidential computing usages, guest can issue a > >> > shared memory access while the memory is actually private from the host > >> > point of view. This exit with bit 0 cleared gives userspace a chance to > >> > convert the private memory to shared memory on host. > >> > >> I think this should be explicit rather than implied by the absence of > >> another flag. Sean suggested you might want flags for RWX failures so > >> maybe something like: > >> > >> KVM_MEMORY_EXIT_SHARED_FLAG_READ (1 << 0) > >> KVM_MEMORY_EXIT_SHARED_FLAG_WRITE (1 << 1) > >> KVM_MEMORY_EXIT_SHARED_FLAG_EXECUTE (1 << 2) > >> KVM_MEMORY_EXIT_FLAG_PRIVATE (1 << 3) > > > > Yes, but I would not add 'SHARED' to RWX, they are not share memory > > specific, private memory can also set them once introduced. > > OK so how about: > > KVM_MEMORY_EXIT_FLAG_READ (1 << 0) > KVM_MEMORY_EXIT_FLAG_WRITE (1 << 1) > KVM_MEMORY_EXIT_FLAG_EXECUTE (1 << 2) > KVM_MEMORY_EXIT_FLAG_SHARED (1 << 3) > KVM_MEMORY_EXIT_FLAG_PRIVATE (1 << 4) We don't actually need a new bit, the opposite side of private is shared, i.e. flags with KVM_MEMORY_EXIT_FLAG_PRIVATE cleared expresses 'shared'. Chao > > > > > Thanks, > > Chao > >> > >> which would allow you to signal the various failure modes of the shared > >> region, or that you had accessed private memory. > >> > >> > > >> >> > >> >> If you envision any other failure modes it might be worth making it > >> >> explicit with additional flags. > >> > > >> > Sean mentioned some more usages[1][]2] other than the memory conversion > >> > for confidential usage. But I would leave those flags being added in the > >> > future after those usages being well discussed. > >> > > >> > [1] https://lkml.kernel.org/r/20200617230052.GB27751@linux.intel.com > >> > [2] https://lore.kernel.org/all/YKxJLcg%2FWomPE422@google.com > >> > > >> >> I also wonder if a bitmask makes sense if > >> >> there can only be one reason for a failure? Maybe all that is needed is > >> >> a reason enum? > >> > > >> > Tough we only have one reason right now but we still want to leave room > >> > for future extension. Enum can express a single value at once well but > >> > bitmask makes it possible to express multiple orthogonal flags. > >> > >> I agree if multiple orthogonal failures can occur at once a bitmask is > >> the right choice. > >> > >> > > >> > Chao > >> >> > >> >> > + > >> >> > +'gpa' and 'size' indicate the memory range the error occurs at. The userspace > >> >> > +may handle the error and return to KVM to retry the previous memory access. > >> >> > + > >> >> > :: > >> >> > > >> >> > /* KVM_EXIT_NOTIFY */ > >> >> > diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h > >> >> > index f1ae45c10c94..fa60b032a405 100644 > >> >> > --- a/include/uapi/linux/kvm.h > >> >> > +++ b/include/uapi/linux/kvm.h > >> >> > @@ -300,6 +300,7 @@ struct kvm_xen_exit { > >> >> > #define KVM_EXIT_RISCV_SBI 35 > >> >> > #define KVM_EXIT_RISCV_CSR 36 > >> >> > #define KVM_EXIT_NOTIFY 37 > >> >> > +#define KVM_EXIT_MEMORY_FAULT 38 > >> >> > > >> >> > /* For KVM_EXIT_INTERNAL_ERROR */ > >> >> > /* Emulate instruction failed. */ > >> >> > @@ -538,6 +539,14 @@ struct kvm_run { > >> >> > #define KVM_NOTIFY_CONTEXT_INVALID (1 << 0) > >> >> > __u32 flags; > >> >> > } notify; > >> >> > + /* KVM_EXIT_MEMORY_FAULT */ > >> >> > + struct { > >> >> > +#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1 << 0) > >> >> > + __u32 flags; > >> >> > + __u32 padding; > >> >> > + __u64 gpa; > >> >> > + __u64 size; > >> >> > + } memory; > >> >> > /* Fix the size of the union. */ > >> >> > char padding[256]; > >> >> > }; > >> >> > >> >> > >> >> -- > >> >> Alex Bennée > >> > >> > >> -- > >> Alex Bennée > > > -- > Alex Bennée