linux-coco.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: Suzuki K Poulose <suzuki.poulose@arm.com>
To: Steven Price <steven.price@arm.com>,
	kvm@vger.kernel.org, kvmarm@lists.linux.dev
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Will Deacon <will@kernel.org>,
	James Morse <james.morse@arm.com>,
	Oliver Upton <oliver.upton@linux.dev>,
	Zenghui Yu <yuzenghui@huawei.com>,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, Joey Gouly <joey.gouly@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Christoffer Dall <christoffer.dall@arm.com>,
	Fuad Tabba <tabba@google.com>,
	linux-coco@lists.linux.dev,
	Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
Subject: Re: [PATCH v2 17/43] arm64: RME: Allow VMM to set RIPAS
Date: Thu, 2 May 2024 11:16:56 +0100	[thread overview]
Message-ID: <85a4bbda-4516-4c2a-ae51-5e8f8c451e3a@arm.com> (raw)
In-Reply-To: <98ef5e64-2d1e-4fa2-a3da-210ea180f20f@arm.com>

On 01/05/2024 16:47, Steven Price wrote:
> On 19/04/2024 10:34, Suzuki K Poulose wrote:
>> On 12/04/2024 09:42, Steven Price wrote:
>>> Each page within the protected region of the realm guest can be marked
>>> as either RAM or EMPTY. Allow the VMM to control this before the guest
>>> has started and provide the equivalent functions to change this (with
>>> the guest's approval) at runtime.
>>>
>>> When transitioning from RIPAS RAM (1) to RIPAS EMPTY (0) the memory is
>>> unmapped from the guest and undelegated allowing the memory to be reused
>>> by the host. When transitioning to RIPAS RAM the actual population of
>>> the leaf RTTs is done later on stage 2 fault, however it may be
>>> necessary to allocate additional RTTs to represent the range requested.
>>
>> minor nit: To give a bit more context:
>>
>> "however it may be necessary to allocate additional RTTs in order for
>> the RMM to track the RIPAS for the requested range".
> 
> That is what I meant - but your wording is probably clearer ;)
> Technically there's also the case where a RIPAS change will cause a
> block mapping to be split which isn't just about tracking RIPAS but also
> about breaking up the block mapping for the pages that remain.
> 
>>>
>>> When freeing a block mapping it is necessary to temporarily unfold the
>>> RTT which requires delegating an extra page to the RMM, this page can
>>> then be recovered once the contents of the block mapping have been
>>> freed. A spare, delegated page (spare_page) is used for this purpose.
>>>
>>> Signed-off-by: Steven Price <steven.price@arm.com>
>>> ---
>>>    arch/arm64/include/asm/kvm_rme.h |  16 ++
>>>    arch/arm64/kvm/mmu.c             |   8 +-
>>>    arch/arm64/kvm/rme.c             | 390 +++++++++++++++++++++++++++++++
>>>    3 files changed, 411 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/kvm_rme.h
>>> b/arch/arm64/include/asm/kvm_rme.h
>>> index 915e76068b00..cc8f81cfc3c0 100644
>>> --- a/arch/arm64/include/asm/kvm_rme.h
>>> +++ b/arch/arm64/include/asm/kvm_rme.h
>>> @@ -96,6 +96,14 @@ void kvm_realm_destroy_rtts(struct kvm *kvm, u32
>>> ia_bits);
>>>    int kvm_create_rec(struct kvm_vcpu *vcpu);
>>>    void kvm_destroy_rec(struct kvm_vcpu *vcpu);
>>>    +void kvm_realm_unmap_range(struct kvm *kvm,
>>> +               unsigned long ipa,
>>> +               u64 size,
>>> +               bool unmap_private);
>>> +int realm_set_ipa_state(struct kvm_vcpu *vcpu,
>>> +            unsigned long addr, unsigned long end,
>>> +            unsigned long ripas);
>>> +
>>>    #define RME_RTT_BLOCK_LEVEL    2
>>>    #define RME_RTT_MAX_LEVEL    3
>>>    @@ -114,4 +122,12 @@ static inline unsigned long
>>> rme_rtt_level_mapsize(int level)
>>>        return (1UL << RME_RTT_LEVEL_SHIFT(level));
>>>    }
>>>    +static inline bool realm_is_addr_protected(struct realm *realm,
>>> +                       unsigned long addr)
>>> +{
>>> +    unsigned int ia_bits = realm->ia_bits;
>>> +
>>> +    return !(addr & ~(BIT(ia_bits - 1) - 1));
>>> +}
>>> +
>>>    #endif
>>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>>> index 46f0c4e80ace..8a7b5449697f 100644
>>> --- a/arch/arm64/kvm/mmu.c
>>> +++ b/arch/arm64/kvm/mmu.c
>>> @@ -310,6 +310,7 @@ static void invalidate_icache_guest_page(void *va,
>>> size_t size)
>>>     * @start: The intermediate physical base address of the range to unmap
>>>     * @size:  The size of the area to unmap
>>>     * @may_block: Whether or not we are permitted to block
>>> + * @only_shared: If true then protected mappings should not be unmapped
>>>     *
>>>     * Clear a range of stage-2 mappings, lowering the various
>>> ref-counts.  Must
>>>     * be called while holding mmu_lock (unless for freeing the stage2
>>> pgd before
>>> @@ -317,7 +318,7 @@ static void invalidate_icache_guest_page(void *va,
>>> size_t size)
>>>     * with things behind our backs.
>>>     */
>>>    static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t
>>> start, u64 size,
>>> -                 bool may_block)
>>> +                 bool may_block, bool only_shared)
>>>    {
>>>        struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
>>>        phys_addr_t end = start + size;
>>> @@ -330,7 +331,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu
>>> *mmu, phys_addr_t start, u64
>>>      static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t
>>> start, u64 size)
>>>    {
>>> -    __unmap_stage2_range(mmu, start, size, true);
>>> +    __unmap_stage2_range(mmu, start, size, true, false);
>>>    }
>>>      static void stage2_flush_memslot(struct kvm *kvm,
>>> @@ -1771,7 +1772,8 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct
>>> kvm_gfn_range *range)
>>>          __unmap_stage2_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
>>>                     (range->end - range->start) << PAGE_SHIFT,
>>> -                 range->may_block);
>>> +                 range->may_block,
>>> +                 range->only_shared);
>>>          return false;
>>>    }
>>> diff --git a/arch/arm64/kvm/rme.c b/arch/arm64/kvm/rme.c
>>> index 629a095bea61..9e5983c51393 100644
>>> --- a/arch/arm64/kvm/rme.c
>>> +++ b/arch/arm64/kvm/rme.c
>>> @@ -79,6 +79,12 @@ static phys_addr_t __alloc_delegated_page(struct
>>> realm *realm,
>>>        return phys;
>>>    }
>>>    +static phys_addr_t alloc_delegated_page(struct realm *realm,
>>> +                    struct kvm_mmu_memory_cache *mc)
>>> +{
>>> +    return __alloc_delegated_page(realm, mc, GFP_KERNEL);
>>> +}
>>> +
>>>    static void free_delegated_page(struct realm *realm, phys_addr_t phys)
>>>    {
>>>        if (realm->spare_page == PHYS_ADDR_MAX) {
>>> @@ -94,6 +100,151 @@ static void free_delegated_page(struct realm
>>> *realm, phys_addr_t phys)
>>>        free_page((unsigned long)phys_to_virt(phys));
>>>    }
>>>    +static int realm_rtt_create(struct realm *realm,
>>> +                unsigned long addr,
>>> +                int level,
>>> +                phys_addr_t phys)
>>> +{
>>> +    addr = ALIGN_DOWN(addr, rme_rtt_level_mapsize(level - 1));
>>> +    return rmi_rtt_create(virt_to_phys(realm->rd), phys, addr, level);
>>> +}
>>> +
>>> +static int realm_rtt_fold(struct realm *realm,
>>> +              unsigned long addr,
>>> +              int level,
>>> +              phys_addr_t *rtt_granule)
>>> +{
>>> +    unsigned long out_rtt;
>>> +    int ret;
>>> +
>>> +    ret = rmi_rtt_fold(virt_to_phys(realm->rd), addr, level, &out_rtt);
>>> +
>>> +    if (RMI_RETURN_STATUS(ret) == RMI_SUCCESS && rtt_granule)
>>> +        *rtt_granule = out_rtt;
>>> +
>>> +    return ret;
>>> +}
>>> +
>>> +static int realm_destroy_protected(struct realm *realm,
>>> +                   unsigned long ipa,
>>> +                   unsigned long *next_addr)
>>> +{
>>> +    unsigned long rd = virt_to_phys(realm->rd);
>>> +    unsigned long addr;
>>> +    phys_addr_t rtt;
>>> +    int ret;
>>> +
>>> +loop:
>>> +    ret = rmi_data_destroy(rd, ipa, &addr, next_addr);
>>> +    if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT) {
>>> +        if (*next_addr > ipa)
>>> +            return 0; /* UNASSIGNED */
>>> +        rtt = alloc_delegated_page(realm, NULL);
>>> +        if (WARN_ON(rtt == PHYS_ADDR_MAX))
>>> +            return -1;
>>> +        /* ASSIGNED - ipa is mapped as a block, so split */
>>> +        ret = realm_rtt_create(realm, ipa,
>>> +                       RMI_RETURN_INDEX(ret) + 1, rtt);
>>
>> Could we not go all the way to L3 (rather than 1 level deeper) and try
>> again ? That way, we are covered for block mappings at L1 (1G).
> 
> I don't think this situation can happen. The spec states that for
> RMI_ERROR_RTT either top>ipa (UNASSIGNED, in which case we've bailed out
> above) or top==ipa (ASSIGNED) in which case RMI_RETURN_INDEX(ret) must
> equal 2.

It could be 1 as RMM spec allows Block mapping at L1. But if we don't
fold to that level in KVM, the current code may be fine.

> 
> So in this case realm_rtt_create() will always be called with level=3.
> 
> I can simplify this by explicitly passing 3 and adding a WARN_ON() for
> RMI_RETURN_INDEX(ret) != 2.

Yes please, if we don't fold all the way up to L1.

Thanks
Suzuki



  reply	other threads:[~2024-05-02 10:17 UTC|newest]

Thread overview: 124+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-12  8:40 [v2] Support for Arm CCA VMs on Linux Steven Price
2024-04-11 18:54 ` Itaru Kitayama
2024-04-15  8:14   ` Steven Price
2024-04-12  8:41 ` [PATCH v2 00/14] arm64: Support for running as a guest in Arm CCA Steven Price
2024-04-12  8:42   ` [PATCH v2 01/14] arm64: rsi: Add RSI definitions Steven Price
2024-04-12  8:42   ` [PATCH v2 02/14] arm64: Detect if in a realm and set RIPAS RAM Steven Price
2024-05-10 17:35     ` Catalin Marinas
2024-05-14 10:18       ` Suzuki K Poulose
2024-05-16 14:32         ` Catalin Marinas
2024-05-15 15:03       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 03/14] arm64: realm: Query IPA size from the RMM Steven Price
2024-05-13 14:03     ` Catalin Marinas
2024-05-16 15:13       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 04/14] arm64: Mark all I/O as non-secure shared Steven Price
2024-04-12  8:42   ` [PATCH v2 05/14] fixmap: Allow architecture overriding set_fixmap_io Steven Price
2024-04-12  8:42   ` [PATCH v2 06/14] arm64: Override set_fixmap_io Steven Price
2024-05-13 16:14     ` Catalin Marinas
2024-05-14 10:21       ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 07/14] arm64: Make the PHYS_MASK_SHIFT dynamic Steven Price
2024-05-13 16:38     ` Catalin Marinas
2024-05-16 15:34       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 08/14] arm64: Enforce bounce buffers for realm DMA Steven Price
2024-05-13 16:56     ` Catalin Marinas
2024-04-12  8:42   ` [PATCH v2 09/14] arm64: Enable memory encrypt for Realms Steven Price
2024-04-15  3:13     ` kernel test robot
2024-04-25 13:42       ` Suzuki K Poulose
2024-04-25 15:52         ` Steven Price
2024-04-25 16:29         ` Suzuki K Poulose
2024-04-25 18:16           ` Emanuele Rocca
2024-05-14 18:00     ` Catalin Marinas
2024-05-15 10:47       ` Suzuki K Poulose
2024-05-16  7:48         ` Catalin Marinas
2024-05-16  9:06           ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 10/14] arm64: Force device mappings to be non-secure shared Steven Price
2024-05-15  9:01     ` Catalin Marinas
2024-05-15 11:00       ` Suzuki K Poulose
2024-05-17  9:34         ` Catalin Marinas
2024-04-12  8:42   ` [PATCH v2 11/14] efi: arm64: Map Device with Prot Shared Steven Price
2024-04-12  8:42   ` [PATCH v2 12/14] arm64: realm: Support nonsecure ITS emulation shared Steven Price
2024-05-15 11:01     ` Catalin Marinas
2024-04-12  8:42   ` [PATCH v2 13/14] arm64: rsi: Interfaces to query attestation token Steven Price
2024-05-15 11:10     ` Catalin Marinas
2024-04-12  8:42   ` [PATCH v2 14/14] virt: arm-cca-guest: TSM_REPORT support for realms Steven Price
2024-04-24 13:06     ` Thomas Fossati
2024-04-24 13:27       ` Suzuki K Poulose
2024-04-24 13:19     ` Suzuki K Poulose
2024-04-12  8:42 ` [PATCH v2 00/43] arm64: Support for Arm CCA in KVM Steven Price
2024-04-12  8:42   ` [PATCH v2 01/43] KVM: Prepare for handling only shared mappings in mmu_notifier events Steven Price
2024-04-25  9:48     ` Fuad Tabba
2024-04-25 15:58       ` Steven Price
2024-04-25 22:56         ` Sean Christopherson
2024-04-12  8:42   ` [PATCH v2 02/43] kvm: arm64: pgtable: Track the number of pages in the entry level Steven Price
2024-04-12  8:42   ` [PATCH v2 03/43] kvm: arm64: Include kvm_emulate.h in kvm/arm_psci.h Steven Price
2024-04-12  8:42   ` [PATCH v2 04/43] arm64: RME: Handle Granule Protection Faults (GPFs) Steven Price
2024-04-16 11:17     ` Suzuki K Poulose
2024-04-18 13:17       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 05/43] arm64: RME: Add SMC definitions for calling the RMM Steven Price
2024-04-16 12:38     ` Suzuki K Poulose
2024-04-18 13:17       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 06/43] arm64: RME: Add wrappers for RMI calls Steven Price
2024-04-16 13:14     ` Suzuki K Poulose
2024-04-19 11:18       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 07/43] arm64: RME: Check for RME support at KVM init Steven Price
2024-04-16 13:30     ` Suzuki K Poulose
2024-04-22 15:39       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 08/43] arm64: RME: Define the user ABI Steven Price
2024-04-12  8:42   ` [PATCH v2 09/43] arm64: RME: ioctls to create and configure realms Steven Price
2024-04-17  9:51     ` Suzuki K Poulose
2024-04-22 16:33       ` Steven Price
2024-04-18 16:04     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 10/43] kvm: arm64: Expose debug HW register numbers for Realm Steven Price
2024-04-12  8:42   ` [PATCH v2 11/43] arm64: kvm: Allow passing machine type in KVM creation Steven Price
2024-04-17 10:20     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 12/43] arm64: RME: Keep a spare page delegated to the RMM Steven Price
2024-04-17 10:19     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 13/43] arm64: RME: RTT handling Steven Price
2024-04-17 13:37     ` Suzuki K Poulose
2024-04-24 10:59       ` Steven Price
2024-04-12  8:42   ` [PATCH v2 14/43] arm64: RME: Allocate/free RECs to match vCPUs Steven Price
2024-04-18  9:23     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 15/43] arm64: RME: Support for the VGIC in realms Steven Price
2024-04-12  8:42   ` [PATCH v2 16/43] KVM: arm64: Support timers in realm RECs Steven Price
2024-04-18  9:30     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 17/43] arm64: RME: Allow VMM to set RIPAS Steven Price
2024-04-19  9:34     ` Suzuki K Poulose
2024-04-19 10:20       ` Suzuki K Poulose
2024-05-01 15:47       ` Steven Price
2024-05-02 10:16         ` Suzuki K Poulose [this message]
2024-04-25  9:53     ` Fuad Tabba
2024-05-01 14:27     ` Jean-Philippe Brucker
2024-05-01 14:56       ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 18/43] arm64: RME: Handle realm enter/exit Steven Price
2024-04-19 13:00     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 19/43] KVM: arm64: Handle realm MMIO emulation Steven Price
2024-04-12  8:42   ` [PATCH v2 20/43] arm64: RME: Allow populating initial contents Steven Price
2024-04-19 13:17     ` Suzuki K Poulose
2024-04-12  8:42   ` [PATCH v2 21/43] arm64: RME: Runtime faulting of memory Steven Price
2024-04-25 10:43     ` Fuad Tabba
2024-04-12  8:42   ` [PATCH v2 22/43] KVM: arm64: Handle realm VCPU load Steven Price
2024-04-12  8:42   ` [PATCH v2 23/43] KVM: arm64: Validate register access for a Realm VM Steven Price
2024-04-12  8:42   ` [PATCH v2 24/43] KVM: arm64: Handle Realm PSCI requests Steven Price
2024-04-12  8:42   ` [PATCH v2 25/43] KVM: arm64: WARN on injected undef exceptions Steven Price
2024-04-12  8:42   ` [PATCH v2 26/43] arm64: Don't expose stolen time for realm guests Steven Price
2024-04-12  8:42   ` [PATCH v2 27/43] arm64: rme: allow userspace to inject aborts Steven Price
2024-04-12  8:42   ` [PATCH v2 28/43] arm64: rme: support RSI_HOST_CALL Steven Price
2024-04-12  8:42   ` [PATCH v2 29/43] arm64: rme: Allow checking SVE on VM instance Steven Price
2024-04-12  8:42   ` [PATCH v2 30/43] arm64: RME: Always use 4k pages for realms Steven Price
2024-04-12  8:42   ` [PATCH v2 31/43] arm64: rme: Prevent Device mappings for Realms Steven Price
2024-04-12  8:42   ` [PATCH v2 32/43] arm_pmu: Provide a mechanism for disabling the physical IRQ Steven Price
2024-04-12  8:42   ` [PATCH v2 33/43] arm64: rme: Enable PMU support with a realm guest Steven Price
2024-04-13 23:44     ` kernel test robot
2024-04-18 16:06       ` Suzuki K Poulose
2024-04-12  8:43   ` [PATCH v2 34/43] kvm: rme: Hide KVM_CAP_READONLY_MEM for realm guests Steven Price
2024-04-12  8:43   ` [PATCH v2 35/43] arm64: RME: Propagate number of breakpoints and watchpoints to userspace Steven Price
2024-04-12  8:43   ` [PATCH v2 36/43] arm64: RME: Set breakpoint parameters through SET_ONE_REG Steven Price
2024-04-12  8:43   ` [PATCH v2 37/43] arm64: RME: Initialize PMCR.N with number counter supported by RMM Steven Price
2024-04-12  8:43   ` [PATCH v2 38/43] arm64: RME: Propagate max SVE vector length from RMM Steven Price
2024-04-12  8:43   ` [PATCH v2 39/43] arm64: RME: Configure max SVE vector length for a Realm Steven Price
2024-04-12  8:43   ` [PATCH v2 40/43] arm64: RME: Provide register list for unfinalized RME RECs Steven Price
2024-04-12  8:43   ` [PATCH v2 41/43] arm64: RME: Provide accurate register list Steven Price
2024-04-12  8:43   ` [PATCH v2 42/43] arm64: kvm: Expose support for private memory Steven Price
2024-04-25 14:44     ` Fuad Tabba
2024-04-12  8:43   ` [PATCH v2 43/43] KVM: arm64: Allow activating realms Steven Price
2024-04-12 16:52 ` [v2] Support for Arm CCA VMs on Linux Jean-Philippe Brucker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=85a4bbda-4516-4c2a-ae51-5e8f8c451e3a@arm.com \
    --to=suzuki.poulose@arm.com \
    --cc=alexandru.elisei@arm.com \
    --cc=catalin.marinas@arm.com \
    --cc=christoffer.dall@arm.com \
    --cc=gankulkarni@os.amperecomputing.com \
    --cc=james.morse@arm.com \
    --cc=joey.gouly@arm.com \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=oliver.upton@linux.dev \
    --cc=steven.price@arm.com \
    --cc=tabba@google.com \
    --cc=will@kernel.org \
    --cc=yuzenghui@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).