From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DBFBC07E95 for ; Tue, 20 Jul 2021 14:02:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F322661165 for ; Tue, 20 Jul 2021 14:02:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239509AbhGTNTS (ORCPT ); Tue, 20 Jul 2021 09:19:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238928AbhGTNIK (ORCPT ); Tue, 20 Jul 2021 09:08:10 -0400 Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD0B9C061574 for ; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) Received: by mail-ot1-x335.google.com with SMTP id 59-20020a9d0ac10000b0290462f0ab0800so21555720otq.11 for ; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sSKYKbxnVu+G8KsuMx0V0FlK4uTjY08r+UWrZ/z8LHQ=; b=utAjJcc9cGMaMXp3rIBJ1ZRTSBKvYt0c86Q62CMxLwEWhTH+xS0N+cbAwxz7Y2TFM4 dp6GRnKGtA6N6wQuQ+FTqHC5WYixuviEtvzFioOV8p6mJoFir6Qc99xo2H87a8BznM6T SmXVeUKSAJ2YxDebBssr4xVaAO78BHlhlYwsMlGNwZ8wmSgRCDB9iH5Ar+9VJBzUU4ga 7M6VNPrGVXhUCdtAr+Dmzm7WHV7bhTh/T6OMjEb6K2tww/GGztX2oZoTDDfyGdZ24ZfN h6PgRlr2wokfmTq8lbrr7qoeplfwRm/eGNhXml8CRF+5rGCpdRCemqSuz/HNtGv1TRK/ 8DJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sSKYKbxnVu+G8KsuMx0V0FlK4uTjY08r+UWrZ/z8LHQ=; b=YA5jslb8Gh/8P7BNjm9wFX76X+t79g5EHnIdrqJfk5pXJoQT8D90sZbhODpQDo/+1e Aza5IY5g48G1IfF60JUe8gzzL9IUxnJAaPUdMksSR4mDbfSZd9cjNpPzKm9gStsT5ZVe PKlHRUi7qKEM4DENet6piY2q7ic2QOwrTjlCS5klGSvZ1DdabZ710XTPMZH+ACcbK6ne swwOplzq+MqH6gc7WclX9QbIVXjptwc4JTjtYLPor4AqmOqr8itQaajG6CE8oG3liZgM AhN0BVJTQH97ztKXVDGDFTv7HzxmVQqh3Ar3adzueRl3uQw4Y4krf+U0Rv9eOMxaJpdL mlRg== X-Gm-Message-State: AOAM531xoUOdSC1kLYCbKuz6frNaxLxRopcV8Q547BFxEDYHS59AWrky YaChYzqEqO/no8etNBwh6M98lAUrU4JTaMkykja5dg== X-Google-Smtp-Source: ABdhPJwO2Xb6g/ZfTRHpSfHvJvHBkk79kp1jTLisPNfqjoFjNOCC/oC/AfEZFLcKbXu8Y1Xt91MqVh+c+yhVsZgW1nw= X-Received: by 2002:a05:6830:1455:: with SMTP id w21mr22257855otp.365.1626788926039; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) MIME-Version: 1.0 References: <20210719104735.3681732-1-qperret@google.com> <20210719104735.3681732-9-qperret@google.com> In-Reply-To: <20210719104735.3681732-9-qperret@google.com> From: Fuad Tabba Date: Tue, 20 Jul 2021 14:48:09 +0100 Message-ID: Subject: Re: [PATCH 08/14] KVM: arm64: Add support for tagging shared pages in page-table To: Quentin Perret Cc: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, dbrazdil@google.com, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Quentin, On Mon, Jul 19, 2021 at 11:47 AM Quentin Perret wrote: > > The hypervisor will soon be in charge of tracking ownership of all > memory pages in the system. The current page-tracking infrastructure at > EL2 only allows binary states: a page is either owned or not by an > entity. But a number of use-cases will require more complex states for > pages that are shared between two entities (host, hypervisor, or guests). > > In preparation for supporting these use-cases, introduce in the KVM > page-table library some infrastructure allowing to tag shared pages > using ignored bits (a.k.a. software bits) in PTEs. > > Signed-off-by: Quentin Perret > --- > arch/arm64/include/asm/kvm_pgtable.h | 5 +++++ > arch/arm64/kvm/hyp/pgtable.c | 25 +++++++++++++++++++++++++ > 2 files changed, 30 insertions(+) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index dd72653314c7..f6d3d5c8910d 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -81,6 +81,8 @@ enum kvm_pgtable_stage2_flags { > * @KVM_PGTABLE_PROT_W: Write permission. > * @KVM_PGTABLE_PROT_R: Read permission. > * @KVM_PGTABLE_PROT_DEVICE: Device attributes. > + * @KVM_PGTABLE_STATE_SHARED: Page shared with another entity. > + * @KVM_PGTABLE_STATE_BORROWED: Page borrowed from another entity. > */ > enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_X = BIT(0), > @@ -88,6 +90,9 @@ enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_R = BIT(2), > > KVM_PGTABLE_PROT_DEVICE = BIT(3), > + > + KVM_PGTABLE_STATE_SHARED = BIT(4), > + KVM_PGTABLE_STATE_BORROWED = BIT(5), > }; > > #define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index 5bdbe7a31551..51598b79dafc 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -211,6 +211,29 @@ static kvm_pte_t kvm_init_invalid_leaf_owner(u8 owner_id) > return FIELD_PREP(KVM_INVALID_PTE_OWNER_MASK, owner_id); > } > > +static kvm_pte_t pte_ignored_bit_prot(enum kvm_pgtable_prot prot) > +{ > + kvm_pte_t ignored_bits = 0; > + > + /* > + * Ignored bits 0 and 1 are reserved to track the memory ownership > + * state of each page: > + * 00: The page is owned solely by the page-table owner. > + * 01: The page is owned by the page-table owner, but is shared > + * with another entity. > + * 10: The page is shared with, but not owned by the page-table owner. > + * 11: Reserved for future use (lending). > + */ > + if (prot & KVM_PGTABLE_STATE_SHARED) { > + if (prot & KVM_PGTABLE_STATE_BORROWED) > + ignored_bits |= BIT(1); > + else > + ignored_bits |= BIT(0); > + } This might tie in to Marc's comments for using enums, but consolidating the translation between prot and ignored/software bits in one place would be good: thinking about patch 10 as well, where you get the prot from the ignored bits. Even though you have documented it, I'm finding the part where a field can be borrowed and shared as opposed to being only shared not very intuitive, and I need to reread the comment here to remember the difference while going through the code. You also mention lending as potentially reserved for the future, but I think that lending is the other side of borrowing (depends on who's doing the giving/taking). I wonder if in this case it would be clearer to describe it in terms of whether it's exclusively owned vs owned but shared (for the owner), and just shared for the sharer... Thanks, /fuad > + return FIELD_PREP(KVM_PTE_LEAF_ATTR_IGNORED, ignored_bits); > +} > + > static int kvm_pgtable_visitor_cb(struct kvm_pgtable_walk_data *data, u64 addr, > u32 level, kvm_pte_t *ptep, > enum kvm_pgtable_walk_flags flag) > @@ -357,6 +380,7 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_AP, ap); > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_SH, sh); > attr |= KVM_PTE_LEAF_ATTR_LO_S1_AF; > + attr |= pte_ignored_bit_prot(prot); > *ptep = attr; > > return 0; > @@ -558,6 +582,7 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p > > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh); > attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF; > + attr |= pte_ignored_bit_prot(prot); > *ptep = attr; > > return 0; > -- > 2.32.0.402.g57bb445576-goog > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21F8CC07E9B for ; Tue, 20 Jul 2021 13:48:51 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 9ECD761186 for ; Tue, 20 Jul 2021 13:48:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9ECD761186 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 1A01D4B0C7; Tue, 20 Jul 2021 09:48:50 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cx6sh9iwI1E6; Tue, 20 Jul 2021 09:48:49 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id F380849D50; Tue, 20 Jul 2021 09:48:48 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id EC5EC40808 for ; Tue, 20 Jul 2021 09:48:47 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1TVggj+cfzrg for ; Tue, 20 Jul 2021 09:48:46 -0400 (EDT) Received: from mail-ot1-f50.google.com (mail-ot1-f50.google.com [209.85.210.50]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id D8361407EF for ; Tue, 20 Jul 2021 09:48:46 -0400 (EDT) Received: by mail-ot1-f50.google.com with SMTP id o72-20020a9d224e0000b02904bb9756274cso21582007ota.6 for ; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sSKYKbxnVu+G8KsuMx0V0FlK4uTjY08r+UWrZ/z8LHQ=; b=utAjJcc9cGMaMXp3rIBJ1ZRTSBKvYt0c86Q62CMxLwEWhTH+xS0N+cbAwxz7Y2TFM4 dp6GRnKGtA6N6wQuQ+FTqHC5WYixuviEtvzFioOV8p6mJoFir6Qc99xo2H87a8BznM6T SmXVeUKSAJ2YxDebBssr4xVaAO78BHlhlYwsMlGNwZ8wmSgRCDB9iH5Ar+9VJBzUU4ga 7M6VNPrGVXhUCdtAr+Dmzm7WHV7bhTh/T6OMjEb6K2tww/GGztX2oZoTDDfyGdZ24ZfN h6PgRlr2wokfmTq8lbrr7qoeplfwRm/eGNhXml8CRF+5rGCpdRCemqSuz/HNtGv1TRK/ 8DJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sSKYKbxnVu+G8KsuMx0V0FlK4uTjY08r+UWrZ/z8LHQ=; b=S8m9YZRO9dr6xPcsxoc24Yb8ir+cYXNbx+792Dl1yU+h6ck6I+y6A8+POVfhW+wdMy 0S1xVVpDOWFXHM2ZmvELdp/NXg5cW8Y3aV7HbdkJC+x+2JuGywu8pKPYayGs34jSd2PA Yk5fpjp2lDZqD2MWfgJay0By7DHbGx1xCIkeXXbeHR7uSPlVuxlfbITjg+0svIMmr9Qk axvUofWjYoqGivo3wZr8mvCCDJeWWRFQV/fRglcB/62fuwsoy/8NLvQ1G3vSPxXOMeYj sWMYOWF4++qDSWdqVdNWVY8fbh+ylLKKwqXiRUCcfSjbdyjG7WdOeaKagaZKmFlSxTGi ZRiw== X-Gm-Message-State: AOAM533TYnNKfeuexF0Bb5r+OTv19MQ5ssBCzemlBRXX49koITL3RrpC tthjbBhq9t3OVjHKgxHGg+WR4LZKshJv/AYXx45o+g== X-Google-Smtp-Source: ABdhPJwO2Xb6g/ZfTRHpSfHvJvHBkk79kp1jTLisPNfqjoFjNOCC/oC/AfEZFLcKbXu8Y1Xt91MqVh+c+yhVsZgW1nw= X-Received: by 2002:a05:6830:1455:: with SMTP id w21mr22257855otp.365.1626788926039; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) MIME-Version: 1.0 References: <20210719104735.3681732-1-qperret@google.com> <20210719104735.3681732-9-qperret@google.com> In-Reply-To: <20210719104735.3681732-9-qperret@google.com> From: Fuad Tabba Date: Tue, 20 Jul 2021 14:48:09 +0100 Message-ID: Subject: Re: [PATCH 08/14] KVM: arm64: Add support for tagging shared pages in page-table To: Quentin Perret Cc: kernel-team@android.com, qwandor@google.com, maz@kernel.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hi Quentin, On Mon, Jul 19, 2021 at 11:47 AM Quentin Perret wrote: > > The hypervisor will soon be in charge of tracking ownership of all > memory pages in the system. The current page-tracking infrastructure at > EL2 only allows binary states: a page is either owned or not by an > entity. But a number of use-cases will require more complex states for > pages that are shared between two entities (host, hypervisor, or guests). > > In preparation for supporting these use-cases, introduce in the KVM > page-table library some infrastructure allowing to tag shared pages > using ignored bits (a.k.a. software bits) in PTEs. > > Signed-off-by: Quentin Perret > --- > arch/arm64/include/asm/kvm_pgtable.h | 5 +++++ > arch/arm64/kvm/hyp/pgtable.c | 25 +++++++++++++++++++++++++ > 2 files changed, 30 insertions(+) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index dd72653314c7..f6d3d5c8910d 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -81,6 +81,8 @@ enum kvm_pgtable_stage2_flags { > * @KVM_PGTABLE_PROT_W: Write permission. > * @KVM_PGTABLE_PROT_R: Read permission. > * @KVM_PGTABLE_PROT_DEVICE: Device attributes. > + * @KVM_PGTABLE_STATE_SHARED: Page shared with another entity. > + * @KVM_PGTABLE_STATE_BORROWED: Page borrowed from another entity. > */ > enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_X = BIT(0), > @@ -88,6 +90,9 @@ enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_R = BIT(2), > > KVM_PGTABLE_PROT_DEVICE = BIT(3), > + > + KVM_PGTABLE_STATE_SHARED = BIT(4), > + KVM_PGTABLE_STATE_BORROWED = BIT(5), > }; > > #define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index 5bdbe7a31551..51598b79dafc 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -211,6 +211,29 @@ static kvm_pte_t kvm_init_invalid_leaf_owner(u8 owner_id) > return FIELD_PREP(KVM_INVALID_PTE_OWNER_MASK, owner_id); > } > > +static kvm_pte_t pte_ignored_bit_prot(enum kvm_pgtable_prot prot) > +{ > + kvm_pte_t ignored_bits = 0; > + > + /* > + * Ignored bits 0 and 1 are reserved to track the memory ownership > + * state of each page: > + * 00: The page is owned solely by the page-table owner. > + * 01: The page is owned by the page-table owner, but is shared > + * with another entity. > + * 10: The page is shared with, but not owned by the page-table owner. > + * 11: Reserved for future use (lending). > + */ > + if (prot & KVM_PGTABLE_STATE_SHARED) { > + if (prot & KVM_PGTABLE_STATE_BORROWED) > + ignored_bits |= BIT(1); > + else > + ignored_bits |= BIT(0); > + } This might tie in to Marc's comments for using enums, but consolidating the translation between prot and ignored/software bits in one place would be good: thinking about patch 10 as well, where you get the prot from the ignored bits. Even though you have documented it, I'm finding the part where a field can be borrowed and shared as opposed to being only shared not very intuitive, and I need to reread the comment here to remember the difference while going through the code. You also mention lending as potentially reserved for the future, but I think that lending is the other side of borrowing (depends on who's doing the giving/taking). I wonder if in this case it would be clearer to describe it in terms of whether it's exclusively owned vs owned but shared (for the owner), and just shared for the sharer... Thanks, /fuad > + return FIELD_PREP(KVM_PTE_LEAF_ATTR_IGNORED, ignored_bits); > +} > + > static int kvm_pgtable_visitor_cb(struct kvm_pgtable_walk_data *data, u64 addr, > u32 level, kvm_pte_t *ptep, > enum kvm_pgtable_walk_flags flag) > @@ -357,6 +380,7 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_AP, ap); > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_SH, sh); > attr |= KVM_PTE_LEAF_ATTR_LO_S1_AF; > + attr |= pte_ignored_bit_prot(prot); > *ptep = attr; > > return 0; > @@ -558,6 +582,7 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p > > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh); > attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF; > + attr |= pte_ignored_bit_prot(prot); > *ptep = attr; > > return 0; > -- > 2.32.0.402.g57bb445576-goog > _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1613BC636C9 for ; Tue, 20 Jul 2021 13:53:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D628A606A5 for ; Tue, 20 Jul 2021 13:53:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D628A606A5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ROH81OizNgqsNEjYdLq5bnnfPCrChi+orKR4hnEBLjs=; b=NfxppsIUOSGDWy TqJUcvmCpQ3UnJ/+p014PeYZpsD1Ue3DfkdkoafDgRvg0FPJRXcOX5o8fsZZZibo5/mH4OB/vUJQC N45a4WeemB41gnNEPbCIUgmmTIvu0njMTEGNAniPE07TP2zFL9Q2/VwycmkDyLbKpk3PFQAozvS2F lfygagcGArUs7zopsCOOrrbuRiOlgR3dGMQJS5UnYIaZ7r1V/7RhSuvgYbeguhokR9mMjoGPGpJ7V Cg1aam8T8BihSj9mcpjKOS/Xh5wtp1EJ4tCgRtDs91JPinVxoX3OulmiHtPHrb/9A3jZfzBGb6QUj k1lsSc1Ba3Im8W+G5iCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m5q9S-00D0zY-0e; Tue, 20 Jul 2021 13:51:14 +0000 Received: from mail-ot1-x32a.google.com ([2607:f8b0:4864:20::32a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m5q77-00D00k-TR for linux-arm-kernel@lists.infradead.org; Tue, 20 Jul 2021 13:48:51 +0000 Received: by mail-ot1-x32a.google.com with SMTP id a17-20020a9d3e110000b02904ce97efee36so9798411otd.7 for ; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sSKYKbxnVu+G8KsuMx0V0FlK4uTjY08r+UWrZ/z8LHQ=; b=utAjJcc9cGMaMXp3rIBJ1ZRTSBKvYt0c86Q62CMxLwEWhTH+xS0N+cbAwxz7Y2TFM4 dp6GRnKGtA6N6wQuQ+FTqHC5WYixuviEtvzFioOV8p6mJoFir6Qc99xo2H87a8BznM6T SmXVeUKSAJ2YxDebBssr4xVaAO78BHlhlYwsMlGNwZ8wmSgRCDB9iH5Ar+9VJBzUU4ga 7M6VNPrGVXhUCdtAr+Dmzm7WHV7bhTh/T6OMjEb6K2tww/GGztX2oZoTDDfyGdZ24ZfN h6PgRlr2wokfmTq8lbrr7qoeplfwRm/eGNhXml8CRF+5rGCpdRCemqSuz/HNtGv1TRK/ 8DJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sSKYKbxnVu+G8KsuMx0V0FlK4uTjY08r+UWrZ/z8LHQ=; b=TY++vPOiRoo0vYkbtcxbGzOWbrnv1LeG+6rjlFu4pXQ2WnBU2oGGqYHHLWf+pPJV7Y Lcp1823nxpN6QWB8d6QflzuPyjDA3Z4fzioPN3rB/nMPEjBlMS1MBxNXW4VQXbNZazg9 XMPKCVaXwvOrwu0I+fpXCy9p0tRRRJk2vV6uVeA1ITPLLHa3+BXRf+qJiP9GG108fXiF p4zBBU/aHvFmk8vlvuIWUE7cIljXVcY+FVt8HGbvlVt+Xda964JePg8ghHOrIfFrzYYB 14O3Y5iQjuH7Wpp2Q17gwWftXAo3KZrXrq1H4v3WFCNGQRRbhhp6x05/465luMsFYatW 3gwg== X-Gm-Message-State: AOAM533FqJvOWlMmPayVKhHB1qJZxejYzgqo8JBsgV5pHQRvvIFOm/07 qWS8fereuXxa7CdAV7LUfbOp/tCJrauzGy9sXl+YOw== X-Google-Smtp-Source: ABdhPJwO2Xb6g/ZfTRHpSfHvJvHBkk79kp1jTLisPNfqjoFjNOCC/oC/AfEZFLcKbXu8Y1Xt91MqVh+c+yhVsZgW1nw= X-Received: by 2002:a05:6830:1455:: with SMTP id w21mr22257855otp.365.1626788926039; Tue, 20 Jul 2021 06:48:46 -0700 (PDT) MIME-Version: 1.0 References: <20210719104735.3681732-1-qperret@google.com> <20210719104735.3681732-9-qperret@google.com> In-Reply-To: <20210719104735.3681732-9-qperret@google.com> From: Fuad Tabba Date: Tue, 20 Jul 2021 14:48:09 +0100 Message-ID: Subject: Re: [PATCH 08/14] KVM: arm64: Add support for tagging shared pages in page-table To: Quentin Perret Cc: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, dbrazdil@google.com, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210720_064850_010318_0F045360 X-CRM114-Status: GOOD ( 31.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Quentin, On Mon, Jul 19, 2021 at 11:47 AM Quentin Perret wrote: > > The hypervisor will soon be in charge of tracking ownership of all > memory pages in the system. The current page-tracking infrastructure at > EL2 only allows binary states: a page is either owned or not by an > entity. But a number of use-cases will require more complex states for > pages that are shared between two entities (host, hypervisor, or guests). > > In preparation for supporting these use-cases, introduce in the KVM > page-table library some infrastructure allowing to tag shared pages > using ignored bits (a.k.a. software bits) in PTEs. > > Signed-off-by: Quentin Perret > --- > arch/arm64/include/asm/kvm_pgtable.h | 5 +++++ > arch/arm64/kvm/hyp/pgtable.c | 25 +++++++++++++++++++++++++ > 2 files changed, 30 insertions(+) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index dd72653314c7..f6d3d5c8910d 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -81,6 +81,8 @@ enum kvm_pgtable_stage2_flags { > * @KVM_PGTABLE_PROT_W: Write permission. > * @KVM_PGTABLE_PROT_R: Read permission. > * @KVM_PGTABLE_PROT_DEVICE: Device attributes. > + * @KVM_PGTABLE_STATE_SHARED: Page shared with another entity. > + * @KVM_PGTABLE_STATE_BORROWED: Page borrowed from another entity. > */ > enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_X = BIT(0), > @@ -88,6 +90,9 @@ enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_R = BIT(2), > > KVM_PGTABLE_PROT_DEVICE = BIT(3), > + > + KVM_PGTABLE_STATE_SHARED = BIT(4), > + KVM_PGTABLE_STATE_BORROWED = BIT(5), > }; > > #define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index 5bdbe7a31551..51598b79dafc 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -211,6 +211,29 @@ static kvm_pte_t kvm_init_invalid_leaf_owner(u8 owner_id) > return FIELD_PREP(KVM_INVALID_PTE_OWNER_MASK, owner_id); > } > > +static kvm_pte_t pte_ignored_bit_prot(enum kvm_pgtable_prot prot) > +{ > + kvm_pte_t ignored_bits = 0; > + > + /* > + * Ignored bits 0 and 1 are reserved to track the memory ownership > + * state of each page: > + * 00: The page is owned solely by the page-table owner. > + * 01: The page is owned by the page-table owner, but is shared > + * with another entity. > + * 10: The page is shared with, but not owned by the page-table owner. > + * 11: Reserved for future use (lending). > + */ > + if (prot & KVM_PGTABLE_STATE_SHARED) { > + if (prot & KVM_PGTABLE_STATE_BORROWED) > + ignored_bits |= BIT(1); > + else > + ignored_bits |= BIT(0); > + } This might tie in to Marc's comments for using enums, but consolidating the translation between prot and ignored/software bits in one place would be good: thinking about patch 10 as well, where you get the prot from the ignored bits. Even though you have documented it, I'm finding the part where a field can be borrowed and shared as opposed to being only shared not very intuitive, and I need to reread the comment here to remember the difference while going through the code. You also mention lending as potentially reserved for the future, but I think that lending is the other side of borrowing (depends on who's doing the giving/taking). I wonder if in this case it would be clearer to describe it in terms of whether it's exclusively owned vs owned but shared (for the owner), and just shared for the sharer... Thanks, /fuad > + return FIELD_PREP(KVM_PTE_LEAF_ATTR_IGNORED, ignored_bits); > +} > + > static int kvm_pgtable_visitor_cb(struct kvm_pgtable_walk_data *data, u64 addr, > u32 level, kvm_pte_t *ptep, > enum kvm_pgtable_walk_flags flag) > @@ -357,6 +380,7 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_AP, ap); > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_SH, sh); > attr |= KVM_PTE_LEAF_ATTR_LO_S1_AF; > + attr |= pte_ignored_bit_prot(prot); > *ptep = attr; > > return 0; > @@ -558,6 +582,7 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p > > attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh); > attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF; > + attr |= pte_ignored_bit_prot(prot); > *ptep = attr; > > return 0; > -- > 2.32.0.402.g57bb445576-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel