From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 684A4C433FE for ; Fri, 8 Apr 2022 13:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236498AbiDHNtJ (ORCPT ); Fri, 8 Apr 2022 09:49:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235681AbiDHNtG (ORCPT ); Fri, 8 Apr 2022 09:49:06 -0400 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61B6EECDB2; Fri, 8 Apr 2022 06:47:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649425621; x=1680961621; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=Z2KoO20U5Gou3STOQQVbT5tvwIoJoZ5aoHHnzZT/K3Y=; b=lnGV7cx3CcSc5Tu8yNYSycT8snaDKM1NRA++nIJD8GQdPQSjWmBPG/Lk sa5pO9+DQybtJj9KxYhuuS62+mC96IZcFH3PG2pnJJsg+IytU+J4IgyGY wWygylp9bfl+0zqlGF36Qr3gPvY7TsNRWORWNJ3ddkPwpD4QRdltfRYJd 3S2IApXEtDGitCaVKu9qZWOQCjMbGaPN+YGkuaqb3kEdMQPftx6Xe+1q+ 7Gjz6rKYuWdD4t4uCTfSNaBKYXsPgxpWwpWSujld66RZS3QeqvJFE7M6b vKegfmJA2YfYOSWwiLgsWVl4U7KKsm1sCUZC4Oy0b8OXuizPZajsm+iv4 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10310"; a="322283260" X-IronPort-AV: E=Sophos;i="5.90,245,1643702400"; d="scan'208";a="322283260" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2022 06:47:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,245,1643702400"; d="scan'208";a="698187600" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.192.101]) by fmsmga001.fm.intel.com with ESMTP; 08 Apr 2022 06:46:52 -0700 Date: Fri, 8 Apr 2022 21:46:41 +0800 From: Chao Peng To: Sean Christopherson Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com Subject: Re: [PATCH v5 05/13] KVM: Extend the memslot to support fd-based private memory Message-ID: <20220408134641.GD57095@chaop.bj.intel.com> Reply-To: Chao Peng References: <20220310140911.50924-1-chao.p.peng@linux.intel.com> <20220310140911.50924-6-chao.p.peng@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 28, 2022 at 09:56:33PM +0000, Sean Christopherson wrote: > On Thu, Mar 10, 2022, Chao Peng wrote: > > Extend the memslot definition to provide fd-based private memory support > > by adding two new fields (private_fd/private_offset). The memslot then > > can maintain memory for both shared pages and private pages in a single > > memslot. Shared pages are provided by existing userspace_addr(hva) field > > and private pages are provided through the new private_fd/private_offset > > fields. > > > > Since there is no 'hva' concept anymore for private memory so we cannot > > rely on get_user_pages() to get a pfn, instead we use the newly added > > memfile_notifier to complete the same job. > > > > This new extension is indicated by a new flag KVM_MEM_PRIVATE. > > > > Signed-off-by: Yu Zhang > > Signed-off-by: Chao Peng > > --- > > Documentation/virt/kvm/api.rst | 37 +++++++++++++++++++++++++++------- > > include/linux/kvm_host.h | 7 +++++++ > > include/uapi/linux/kvm.h | 8 ++++++++ > > 3 files changed, 45 insertions(+), 7 deletions(-) > > > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > > index 3acbf4d263a5..f76ac598606c 100644 > > --- a/Documentation/virt/kvm/api.rst > > +++ b/Documentation/virt/kvm/api.rst > > @@ -1307,7 +1307,7 @@ yet and must be cleared on entry. > > :Capability: KVM_CAP_USER_MEMORY > > :Architectures: all > > :Type: vm ioctl > > -:Parameters: struct kvm_userspace_memory_region (in) > > +:Parameters: struct kvm_userspace_memory_region(_ext) (in) > > :Returns: 0 on success, -1 on error > > > > :: > > @@ -1320,9 +1320,17 @@ yet and must be cleared on entry. > > __u64 userspace_addr; /* start of the userspace allocated memory */ > > }; > > > > + struct kvm_userspace_memory_region_ext { > > + struct kvm_userspace_memory_region region; > > Peeking ahead, the partial switch to the _ext variant is rather gross. I would > prefer that KVM use an entirely different, but binary compatible, struct internally. > And once the kernel supports C11[*], I'm pretty sure we can make the "region" in > _ext an anonymous struct, and make KVM's internal struct a #define of _ext. That > should minimize the churn (no need to get the embedded "region" field), reduce > line lengths, and avoid confusion due to some flows taking the _ext but others > dealing with only the "base" struct. Will try that. > > Maybe kvm_user_memory_region or kvm_user_mem_region? Though it's tempting to be > evil and usurp the old kvm_memory_region :-) > > E.g. pre-C11 do > > struct kvm_userspace_memory_region_ext { > struct kvm_userspace_memory_region region; > __u64 private_offset; > __u32 private_fd; > __u32 padding[5]; > }; > > #ifdef __KERNEL__ > struct kvm_user_mem_region { > __u32 slot; > __u32 flags; > __u64 guest_phys_addr; > __u64 memory_size; /* bytes */ > __u64 userspace_addr; /* start of the userspace allocated memory */ > __u64 private_offset; > __u32 private_fd; > __u32 padding[5]; > }; > #endif > > and then post-C11 do > > struct kvm_userspace_memory_region_ext { > #ifdef __KERNEL__ Is this #ifndef? As I think anonymous struct is only for kernel? Thanks, Chao > struct kvm_userspace_memory_region region; > #else > struct kvm_userspace_memory_region; > #endif > __u64 private_offset; > __u32 private_fd; > __u32 padding[5]; > }; > > #ifdef __KERNEL__ > #define kvm_user_mem_region kvm_userspace_memory_region_ext > #endif > > [*] https://lore.kernel.org/all/20220301145233.3689119-1-arnd@kernel.org > > > + __u64 private_offset; > > + __u32 private_fd; > > + __u32 padding[5]; > > +}; From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 729ADC433F5 for ; Fri, 8 Apr 2022 13:56:40 +0000 (UTC) Received: from localhost ([::1]:60864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ncp6N-0001zi-Af for qemu-devel@archiver.kernel.org; Fri, 08 Apr 2022 09:56:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ncoxK-0004th-1u for qemu-devel@nongnu.org; Fri, 08 Apr 2022 09:47:18 -0400 Received: from mga01.intel.com ([192.55.52.88]:34796) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ncoxA-0005br-05 for qemu-devel@nongnu.org; Fri, 08 Apr 2022 09:47:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649425628; x=1680961628; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=Z2KoO20U5Gou3STOQQVbT5tvwIoJoZ5aoHHnzZT/K3Y=; b=UJbi0GxgiqeRxKwGBdT5x/A4+JPBR8+j/0hAwcDwWjV8uHeFVpFCAEiB Nyphs/WmTJec0Pfw91K3PWSDX8oPA0fH1Ui1zvesHWcrfvNAO99vgyyz0 f5tYj3VOaVnkfDAUImTLMYKYL5hPss0oNMv/r1HTxeYtubGuQy7yQayTE N+37CYP2a22K2Ey/pK6hfMJ7OTe1xJBrdYsAganrFfPnFCJcehRFF1VyN 1Be+VsZqAIU/eg2BHAlX8f892iUj6x4bGXZHKN9FOVuB6CyglEy4uPX42 ZC9y/voC16Li2T196ECpav1Sd69EZTzxNn9UTUFFqeOFIQCrCFfwOfqvt g==; X-IronPort-AV: E=McAfee;i="6400,9594,10310"; a="286584555" X-IronPort-AV: E=Sophos;i="5.90,245,1643702400"; d="scan'208";a="286584555" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2022 06:47:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,245,1643702400"; d="scan'208";a="698187600" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.192.101]) by fmsmga001.fm.intel.com with ESMTP; 08 Apr 2022 06:46:52 -0700 Date: Fri, 8 Apr 2022 21:46:41 +0800 From: Chao Peng To: Sean Christopherson Subject: Re: [PATCH v5 05/13] KVM: Extend the memslot to support fd-based private memory Message-ID: <20220408134641.GD57095@chaop.bj.intel.com> References: <20220310140911.50924-1-chao.p.peng@linux.intel.com> <20220310140911.50924-6-chao.p.peng@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Received-SPF: none client-ip=192.55.52.88; envelope-from=chao.p.peng@linux.intel.com; helo=mga01.intel.com X-Spam_score_int: -70 X-Spam_score: -7.1 X-Spam_bar: ------- X-Spam_report: (-7.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Chao Peng Cc: Wanpeng Li , jun.nakajima@intel.com, kvm@vger.kernel.org, david@redhat.com, qemu-devel@nongnu.org, "J . Bruce Fields" , linux-mm@kvack.org, "H . Peter Anvin" , ak@linux.intel.com, Jonathan Corbet , Joerg Roedel , x86@kernel.org, Hugh Dickins , Steven Price , Ingo Molnar , "Maciej S . Szmigiero" , Borislav Petkov , luto@kernel.org, Thomas Gleixner , Vitaly Kuznetsov , Vlastimil Babka , Jim Mattson , dave.hansen@intel.com, linux-api@vger.kernel.org, Jeff Layton , linux-kernel@vger.kernel.org, Yu Zhang , "Kirill A . Shutemov" , linux-fsdevel@vger.kernel.org, Paolo Bonzini , Andrew Morton , Vishal Annapurve , Mike Rapoport Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Mon, Mar 28, 2022 at 09:56:33PM +0000, Sean Christopherson wrote: > On Thu, Mar 10, 2022, Chao Peng wrote: > > Extend the memslot definition to provide fd-based private memory support > > by adding two new fields (private_fd/private_offset). The memslot then > > can maintain memory for both shared pages and private pages in a single > > memslot. Shared pages are provided by existing userspace_addr(hva) field > > and private pages are provided through the new private_fd/private_offset > > fields. > > > > Since there is no 'hva' concept anymore for private memory so we cannot > > rely on get_user_pages() to get a pfn, instead we use the newly added > > memfile_notifier to complete the same job. > > > > This new extension is indicated by a new flag KVM_MEM_PRIVATE. > > > > Signed-off-by: Yu Zhang > > Signed-off-by: Chao Peng > > --- > > Documentation/virt/kvm/api.rst | 37 +++++++++++++++++++++++++++------- > > include/linux/kvm_host.h | 7 +++++++ > > include/uapi/linux/kvm.h | 8 ++++++++ > > 3 files changed, 45 insertions(+), 7 deletions(-) > > > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > > index 3acbf4d263a5..f76ac598606c 100644 > > --- a/Documentation/virt/kvm/api.rst > > +++ b/Documentation/virt/kvm/api.rst > > @@ -1307,7 +1307,7 @@ yet and must be cleared on entry. > > :Capability: KVM_CAP_USER_MEMORY > > :Architectures: all > > :Type: vm ioctl > > -:Parameters: struct kvm_userspace_memory_region (in) > > +:Parameters: struct kvm_userspace_memory_region(_ext) (in) > > :Returns: 0 on success, -1 on error > > > > :: > > @@ -1320,9 +1320,17 @@ yet and must be cleared on entry. > > __u64 userspace_addr; /* start of the userspace allocated memory */ > > }; > > > > + struct kvm_userspace_memory_region_ext { > > + struct kvm_userspace_memory_region region; > > Peeking ahead, the partial switch to the _ext variant is rather gross. I would > prefer that KVM use an entirely different, but binary compatible, struct internally. > And once the kernel supports C11[*], I'm pretty sure we can make the "region" in > _ext an anonymous struct, and make KVM's internal struct a #define of _ext. That > should minimize the churn (no need to get the embedded "region" field), reduce > line lengths, and avoid confusion due to some flows taking the _ext but others > dealing with only the "base" struct. Will try that. > > Maybe kvm_user_memory_region or kvm_user_mem_region? Though it's tempting to be > evil and usurp the old kvm_memory_region :-) > > E.g. pre-C11 do > > struct kvm_userspace_memory_region_ext { > struct kvm_userspace_memory_region region; > __u64 private_offset; > __u32 private_fd; > __u32 padding[5]; > }; > > #ifdef __KERNEL__ > struct kvm_user_mem_region { > __u32 slot; > __u32 flags; > __u64 guest_phys_addr; > __u64 memory_size; /* bytes */ > __u64 userspace_addr; /* start of the userspace allocated memory */ > __u64 private_offset; > __u32 private_fd; > __u32 padding[5]; > }; > #endif > > and then post-C11 do > > struct kvm_userspace_memory_region_ext { > #ifdef __KERNEL__ Is this #ifndef? As I think anonymous struct is only for kernel? Thanks, Chao > struct kvm_userspace_memory_region region; > #else > struct kvm_userspace_memory_region; > #endif > __u64 private_offset; > __u32 private_fd; > __u32 padding[5]; > }; > > #ifdef __KERNEL__ > #define kvm_user_mem_region kvm_userspace_memory_region_ext > #endif > > [*] https://lore.kernel.org/all/20220301145233.3689119-1-arnd@kernel.org > > > + __u64 private_offset; > > + __u32 private_fd; > > + __u32 padding[5]; > > +};