From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 595CDC433E0 for ; Fri, 5 Mar 2021 09:15:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1734164F53 for ; Fri, 5 Mar 2021 09:15:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229497AbhCEJPU (ORCPT ); Fri, 5 Mar 2021 04:15:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229611AbhCEJOt (ORCPT ); Fri, 5 Mar 2021 04:14:49 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5536EC061574 for ; Fri, 5 Mar 2021 01:14:49 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id w203-20020a1c49d40000b029010c706d0642so5682688wma.0 for ; Fri, 05 Mar 2021 01:14:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=NiGdJYWOwk1ggBwvtk4IUuiZlQG7Jo/ZpYTDhsXbulA=; b=oygQwv/uuVsd5nAZHELl0vYZAwL1bEnGaj9F3FXEjO1aDtyUsCLlSZH4VFoz/BZ23Z 7JIaatV3Cn/gtHUE7gdSWrBwMvwL/NvBQK1PMENx0Wir+Tc6ixUoZc4M5AY2FQw9GKY4 W8HXdZ8mlFrB0aEfVyFtMhMjYrgcDEXeh9i4/7fOtRGgGfEEK5hhkjJCRtSKHd7Swo47 z8/GZ+YMn11E+eKb7xKCqzlCyzhXX3fVVGmHj3/EGo42mar1WRyA3AstJ3TTywM7X1ZK 87XFFzISGMbXifIvG3xDD9g9dMcbvJlF67H3BiIcQHg6xDxPNr6QrjHzDHGWfdaeeJEa eR8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=NiGdJYWOwk1ggBwvtk4IUuiZlQG7Jo/ZpYTDhsXbulA=; b=RP8YW94YkI6rlDuvMJdxYWhhWeJWSJvKYzExe6et0LdzjUX1fPWL9t6Q2fEBY2aKOL HHlMu+2x1B+OFrkIJF26j+Ycj7mdsJPOO3ggmh3JPgbxNQM+jVq4cdnMBiTEyTZTgubl XodTDB2hymzfxSU8wtXaDFrKt6+ECh1v8MKqJ4g7masbaqi4klxVbp3LayEdKWCX8ln/ AFBEEkOW9d+va+41aJdDGjYqlvFhW1zZY1kuaysAGIeZQv+DH3JJEG6KiALdg9HMNgoK TIc8Ui/IIE9qOLV3GleRACfa1II6qRxF3yy+KLw+zKmQaVbh+gq3zTfpd6YW8NhNKgov rRlw== X-Gm-Message-State: AOAM532zCcxRQJfTwEN2AjwsCjidKcXoK5gjLGEBAhEhb6db1untUgyn ekosF/6bpFCfgrO3sz98FNFtew== X-Google-Smtp-Source: ABdhPJyhmE/fuJuG/SeC3BKpW8cT6v1DcHKNo/NjdhJ2SUdl9YVtwJAipHAm0jYNcM/x6WsfP8i8wQ== X-Received: by 2002:a1c:17:: with SMTP id 23mr7928664wma.6.1614935687986; Fri, 05 Mar 2021 01:14:47 -0800 (PST) Received: from google.com (230.69.233.35.bc.googleusercontent.com. [35.233.69.230]) by smtp.gmail.com with ESMTPSA id m3sm3355609wmc.48.2021.03.05.01.14.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Mar 2021 01:14:47 -0800 (PST) Date: Fri, 5 Mar 2021 09:14:45 +0000 From: Quentin Perret To: Will Deacon Cc: catalin.marinas@arm.com, maz@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, linux-kernel@vger.kernel.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, tabba@google.com, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com, seanjc@google.com, robh+dt@kernel.org Subject: Re: [PATCH v3 16/32] KVM: arm64: Elevate hypervisor mappings creation at EL2 Message-ID: References: <20210302150002.3685113-1-qperret@google.com> <20210302150002.3685113-17-qperret@google.com> <20210304192540.GE21795@willie-the-truck> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210304192540.GE21795@willie-the-truck> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thursday 04 Mar 2021 at 19:25:41 (+0000), Will Deacon wrote: > > +static int do_pkvm_init(u32 hyp_va_bits) > > +{ > > + void *per_cpu_base = kvm_ksym_ref(kvm_arm_hyp_percpu_base); > > + int ret; > > + > > + preempt_disable(); > > + hyp_install_host_vector(); > > It's a shame we need this both here _and_ on the reinit path, but it looks > like it's necessary. Right and I want this before the KVM vectors are installed on secondary CPUs, to make sure they get the new pgtable from the start. Otherwise I'd need to do the same dance on all of them to go a switch TTBR0_EL2 and such. > > + ret = kvm_call_hyp_nvhe(__pkvm_init, hyp_mem_base, hyp_mem_size, > > + num_possible_cpus(), kern_hyp_va(per_cpu_base), > > + hyp_va_bits); > > + preempt_enable(); > > + > > + return ret; > > +} > > [...] > > > /** > > * Inits Hyp-mode on all online CPUs > > */ > > static int init_hyp_mode(void) > > { > > + u32 hyp_va_bits; > > int cpu; > > - int err = 0; > > + int err = -ENOMEM; > > + > > + /* > > + * The protected Hyp-mode cannot be initialized if the memory pool > > + * allocation has failed. > > + */ > > + if (is_protected_kvm_enabled() && !hyp_mem_base) > > + return err; > > This skips the error message you get on the out_err path. Ack, I'll fix. > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > > index 4d41d7838d53..9d331bf262d2 100644 > > --- a/arch/arm64/kvm/mmu.c > > +++ b/arch/arm64/kvm/mmu.c > > @@ -221,15 +221,39 @@ void free_hyp_pgds(void) > > if (hyp_pgtable) { > > kvm_pgtable_hyp_destroy(hyp_pgtable); > > kfree(hyp_pgtable); > > + hyp_pgtable = NULL; > > } > > mutex_unlock(&kvm_hyp_pgd_mutex); > > } > > > > +static bool kvm_host_owns_hyp_mappings(void) > > +{ > > + if (static_branch_likely(&kvm_protected_mode_initialized)) > > + return false; > > + > > + /* > > + * This can happen at boot time when __create_hyp_mappings() is called > > + * after the hyp protection has been enabled, but the static key has > > + * not been flipped yet. > > + */ > > + if (!hyp_pgtable && is_protected_kvm_enabled()) > > + return false; > > + > > + WARN_ON(!hyp_pgtable); > > + > > + return true; > > return !(WARN_ON(!hyp_pgtable) && is_protected_kvm_enabled()); Wouldn't this WARN when I have !hyp_pgtable && is_protected_kvm_enabled() but the static key is still off (which can happen, sadly, as per the comment above)? Thanks, Quentin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82A3CC433E0 for ; Fri, 5 Mar 2021 09:14:53 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id CDCB664F53 for ; Fri, 5 Mar 2021 09:14:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDCB664F53 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 51D9B4B468; Fri, 5 Mar 2021 04:14:52 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nbJ08Rb+TQhm; Fri, 5 Mar 2021 04:14:51 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 3DB5A4B3D4; Fri, 5 Mar 2021 04:14:51 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 3BDD04B39A for ; Fri, 5 Mar 2021 04:14:50 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PbA4H146dbtl for ; Fri, 5 Mar 2021 04:14:49 -0500 (EST) Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 15F0D4B38A for ; Fri, 5 Mar 2021 04:14:49 -0500 (EST) Received: by mail-wm1-f44.google.com with SMTP id n22so769685wmc.2 for ; Fri, 05 Mar 2021 01:14:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=NiGdJYWOwk1ggBwvtk4IUuiZlQG7Jo/ZpYTDhsXbulA=; b=oygQwv/uuVsd5nAZHELl0vYZAwL1bEnGaj9F3FXEjO1aDtyUsCLlSZH4VFoz/BZ23Z 7JIaatV3Cn/gtHUE7gdSWrBwMvwL/NvBQK1PMENx0Wir+Tc6ixUoZc4M5AY2FQw9GKY4 W8HXdZ8mlFrB0aEfVyFtMhMjYrgcDEXeh9i4/7fOtRGgGfEEK5hhkjJCRtSKHd7Swo47 z8/GZ+YMn11E+eKb7xKCqzlCyzhXX3fVVGmHj3/EGo42mar1WRyA3AstJ3TTywM7X1ZK 87XFFzISGMbXifIvG3xDD9g9dMcbvJlF67H3BiIcQHg6xDxPNr6QrjHzDHGWfdaeeJEa eR8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=NiGdJYWOwk1ggBwvtk4IUuiZlQG7Jo/ZpYTDhsXbulA=; b=DitFo38FHVjGruDBbEK/ZKPzv3ak6JqF6JjugRV7JFYADCT0hl1pZy2T2uvQgD1BnE 6mXHGcZz/tNWuWhv7k3TJeYauP0K9vhdFqpHo77djdi20mWab/I34RGDqR9fUJHC4mRA igEBCSFEoidBRVMmWKqDklbzPurp8gnx9Wcg0L/KYt5RQKr2JCmVRTEtLmkweu6spC3r 2hRLnHnx9XCDH2ehvdUFkoPC6SNMmFanuHwFurjk4oRyqyIdX8nV9FfHeCawARWFmvih POm7C8EhWGV1sKcwpPxjXGrxXX++J+yJkBmU/HAqPv43e6u0j6/pDGjHCavIhC+ZPeGK on1w== X-Gm-Message-State: AOAM530JsUJopMZqVYPeybqoREqP5NqhU0uyfGeKlx0b9KfUW735cvr3 mVPiKry6C3XprtwC5lIEgeh7QQ== X-Google-Smtp-Source: ABdhPJyhmE/fuJuG/SeC3BKpW8cT6v1DcHKNo/NjdhJ2SUdl9YVtwJAipHAm0jYNcM/x6WsfP8i8wQ== X-Received: by 2002:a1c:17:: with SMTP id 23mr7928664wma.6.1614935687986; Fri, 05 Mar 2021 01:14:47 -0800 (PST) Received: from google.com (230.69.233.35.bc.googleusercontent.com. [35.233.69.230]) by smtp.gmail.com with ESMTPSA id m3sm3355609wmc.48.2021.03.05.01.14.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Mar 2021 01:14:47 -0800 (PST) Date: Fri, 5 Mar 2021 09:14:45 +0000 From: Quentin Perret To: Will Deacon Subject: Re: [PATCH v3 16/32] KVM: arm64: Elevate hypervisor mappings creation at EL2 Message-ID: References: <20210302150002.3685113-1-qperret@google.com> <20210302150002.3685113-17-qperret@google.com> <20210304192540.GE21795@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210304192540.GE21795@willie-the-truck> Cc: android-kvm@google.com, catalin.marinas@arm.com, mate.toth-pal@arm.com, seanjc@google.com, tabba@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Thursday 04 Mar 2021 at 19:25:41 (+0000), Will Deacon wrote: > > +static int do_pkvm_init(u32 hyp_va_bits) > > +{ > > + void *per_cpu_base = kvm_ksym_ref(kvm_arm_hyp_percpu_base); > > + int ret; > > + > > + preempt_disable(); > > + hyp_install_host_vector(); > > It's a shame we need this both here _and_ on the reinit path, but it looks > like it's necessary. Right and I want this before the KVM vectors are installed on secondary CPUs, to make sure they get the new pgtable from the start. Otherwise I'd need to do the same dance on all of them to go a switch TTBR0_EL2 and such. > > + ret = kvm_call_hyp_nvhe(__pkvm_init, hyp_mem_base, hyp_mem_size, > > + num_possible_cpus(), kern_hyp_va(per_cpu_base), > > + hyp_va_bits); > > + preempt_enable(); > > + > > + return ret; > > +} > > [...] > > > /** > > * Inits Hyp-mode on all online CPUs > > */ > > static int init_hyp_mode(void) > > { > > + u32 hyp_va_bits; > > int cpu; > > - int err = 0; > > + int err = -ENOMEM; > > + > > + /* > > + * The protected Hyp-mode cannot be initialized if the memory pool > > + * allocation has failed. > > + */ > > + if (is_protected_kvm_enabled() && !hyp_mem_base) > > + return err; > > This skips the error message you get on the out_err path. Ack, I'll fix. > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > > index 4d41d7838d53..9d331bf262d2 100644 > > --- a/arch/arm64/kvm/mmu.c > > +++ b/arch/arm64/kvm/mmu.c > > @@ -221,15 +221,39 @@ void free_hyp_pgds(void) > > if (hyp_pgtable) { > > kvm_pgtable_hyp_destroy(hyp_pgtable); > > kfree(hyp_pgtable); > > + hyp_pgtable = NULL; > > } > > mutex_unlock(&kvm_hyp_pgd_mutex); > > } > > > > +static bool kvm_host_owns_hyp_mappings(void) > > +{ > > + if (static_branch_likely(&kvm_protected_mode_initialized)) > > + return false; > > + > > + /* > > + * This can happen at boot time when __create_hyp_mappings() is called > > + * after the hyp protection has been enabled, but the static key has > > + * not been flipped yet. > > + */ > > + if (!hyp_pgtable && is_protected_kvm_enabled()) > > + return false; > > + > > + WARN_ON(!hyp_pgtable); > > + > > + return true; > > return !(WARN_ON(!hyp_pgtable) && is_protected_kvm_enabled()); Wouldn't this WARN when I have !hyp_pgtable && is_protected_kvm_enabled() but the static key is still off (which can happen, sadly, as per the comment above)? Thanks, Quentin _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02A66C433E0 for ; Fri, 5 Mar 2021 09:17:07 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E93264DD1 for ; Fri, 5 Mar 2021 09:17:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E93264DD1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/0BmGhPgscEUEnpoSUt4cFiaX4894CSjtJAdO5gHVto=; b=IsMcmbJwYV794kQsuwL3t/4OH 6ixoT9Qlg8vDhY4doaPbnA/EgFFJ8ShNf7bY4XHqnXud8EyzdmSBKsMAZEEIFpnR688LNdoHDNmC6 sLHlOVNHm64wY5jsyo9YYyEMM2+sy9JimPTccd/CnclSgOOneXS3mCTSMyJ0pVAbj8P2S85hSugIO 9iOk32fsgztG4NN9oDdsByXWS5ZoR3jRjR2GAulcdYVboXon3E4EGJntkNMw9paAxTrkZ6FdPSW7M COTt9MfDcFMB17FW1HRFqJ6CpFp4GKeqco5oO0dA/YbccyRvOdUMZCDaG5pnQC+hnGXK5AY3F17VH 07SKIkC1A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lI6Yc-00ECxP-1p; Fri, 05 Mar 2021 09:15:38 +0000 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lI6Xo-00ECMN-PC for linux-arm-kernel@lists.infradead.org; Fri, 05 Mar 2021 09:14:52 +0000 Received: by mail-wm1-x333.google.com with SMTP id e23so762265wmh.3 for ; Fri, 05 Mar 2021 01:14:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=NiGdJYWOwk1ggBwvtk4IUuiZlQG7Jo/ZpYTDhsXbulA=; b=oygQwv/uuVsd5nAZHELl0vYZAwL1bEnGaj9F3FXEjO1aDtyUsCLlSZH4VFoz/BZ23Z 7JIaatV3Cn/gtHUE7gdSWrBwMvwL/NvBQK1PMENx0Wir+Tc6ixUoZc4M5AY2FQw9GKY4 W8HXdZ8mlFrB0aEfVyFtMhMjYrgcDEXeh9i4/7fOtRGgGfEEK5hhkjJCRtSKHd7Swo47 z8/GZ+YMn11E+eKb7xKCqzlCyzhXX3fVVGmHj3/EGo42mar1WRyA3AstJ3TTywM7X1ZK 87XFFzISGMbXifIvG3xDD9g9dMcbvJlF67H3BiIcQHg6xDxPNr6QrjHzDHGWfdaeeJEa eR8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=NiGdJYWOwk1ggBwvtk4IUuiZlQG7Jo/ZpYTDhsXbulA=; b=D5iDShTNtvxOtHpOB60isq1NSachEJZBD6k0IIVhVh4zwq52/BMTcrwEms6JWmyuXk ag5gt3uJNP+UjySnGN8bOSFC/VAaTQaScnHVskV1JODbS9Sry5CFPBhwRmNUw+IV/hDL yHVnadCys9+oCAq71dRhMzIv8tisx34cDah22f/AG4+wELTTddqEUzeWeylY3nDCYPbk 8iwMGR1qvSHZraFXQNO5U8Nik/904BUUpeWU0KXIMzH+0hrdubCWnpDLKz9g9IstZMMO Dhh2fHibJ/OjcLh+hB/B7sMXTbm1l9Z1FTXQKpoNIA/msVkIzOSz0/o0oSqcBO7z3t3E z6OQ== X-Gm-Message-State: AOAM532qcD/NASi0kmAW1T1FiTGs8G/oAsaTqYLkRWlGd3Ze43RSEYmZ 6GIZvL65HQxDz/h2aTR8WAxDhQ== X-Google-Smtp-Source: ABdhPJyhmE/fuJuG/SeC3BKpW8cT6v1DcHKNo/NjdhJ2SUdl9YVtwJAipHAm0jYNcM/x6WsfP8i8wQ== X-Received: by 2002:a1c:17:: with SMTP id 23mr7928664wma.6.1614935687986; Fri, 05 Mar 2021 01:14:47 -0800 (PST) Received: from google.com (230.69.233.35.bc.googleusercontent.com. [35.233.69.230]) by smtp.gmail.com with ESMTPSA id m3sm3355609wmc.48.2021.03.05.01.14.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Mar 2021 01:14:47 -0800 (PST) Date: Fri, 5 Mar 2021 09:14:45 +0000 From: Quentin Perret To: Will Deacon Cc: catalin.marinas@arm.com, maz@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, linux-kernel@vger.kernel.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, tabba@google.com, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com, seanjc@google.com, robh+dt@kernel.org Subject: Re: [PATCH v3 16/32] KVM: arm64: Elevate hypervisor mappings creation at EL2 Message-ID: References: <20210302150002.3685113-1-qperret@google.com> <20210302150002.3685113-17-qperret@google.com> <20210304192540.GE21795@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210304192540.GE21795@willie-the-truck> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210305_091449_939854_F4CAC617 X-CRM114-Status: GOOD ( 24.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thursday 04 Mar 2021 at 19:25:41 (+0000), Will Deacon wrote: > > +static int do_pkvm_init(u32 hyp_va_bits) > > +{ > > + void *per_cpu_base = kvm_ksym_ref(kvm_arm_hyp_percpu_base); > > + int ret; > > + > > + preempt_disable(); > > + hyp_install_host_vector(); > > It's a shame we need this both here _and_ on the reinit path, but it looks > like it's necessary. Right and I want this before the KVM vectors are installed on secondary CPUs, to make sure they get the new pgtable from the start. Otherwise I'd need to do the same dance on all of them to go a switch TTBR0_EL2 and such. > > + ret = kvm_call_hyp_nvhe(__pkvm_init, hyp_mem_base, hyp_mem_size, > > + num_possible_cpus(), kern_hyp_va(per_cpu_base), > > + hyp_va_bits); > > + preempt_enable(); > > + > > + return ret; > > +} > > [...] > > > /** > > * Inits Hyp-mode on all online CPUs > > */ > > static int init_hyp_mode(void) > > { > > + u32 hyp_va_bits; > > int cpu; > > - int err = 0; > > + int err = -ENOMEM; > > + > > + /* > > + * The protected Hyp-mode cannot be initialized if the memory pool > > + * allocation has failed. > > + */ > > + if (is_protected_kvm_enabled() && !hyp_mem_base) > > + return err; > > This skips the error message you get on the out_err path. Ack, I'll fix. > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > > index 4d41d7838d53..9d331bf262d2 100644 > > --- a/arch/arm64/kvm/mmu.c > > +++ b/arch/arm64/kvm/mmu.c > > @@ -221,15 +221,39 @@ void free_hyp_pgds(void) > > if (hyp_pgtable) { > > kvm_pgtable_hyp_destroy(hyp_pgtable); > > kfree(hyp_pgtable); > > + hyp_pgtable = NULL; > > } > > mutex_unlock(&kvm_hyp_pgd_mutex); > > } > > > > +static bool kvm_host_owns_hyp_mappings(void) > > +{ > > + if (static_branch_likely(&kvm_protected_mode_initialized)) > > + return false; > > + > > + /* > > + * This can happen at boot time when __create_hyp_mappings() is called > > + * after the hyp protection has been enabled, but the static key has > > + * not been flipped yet. > > + */ > > + if (!hyp_pgtable && is_protected_kvm_enabled()) > > + return false; > > + > > + WARN_ON(!hyp_pgtable); > > + > > + return true; > > return !(WARN_ON(!hyp_pgtable) && is_protected_kvm_enabled()); Wouldn't this WARN when I have !hyp_pgtable && is_protected_kvm_enabled() but the static key is still off (which can happen, sadly, as per the comment above)? Thanks, Quentin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel