From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CB88C433E0 for ; Mon, 22 Feb 2021 11:05:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B9AF064E5C for ; Mon, 22 Feb 2021 11:05:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B9AF064E5C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VZMHXLFYqU4G9BVFUfY3acA1WOh7Zdo97n/EK/XA7/0=; b=XLsy9pu+ws50Nf+90KHkZ9v95 rKnOn50H5Khnu+3ANJGnbIATjP12DADSKacK6OQhi9kvriywec+82UBkcG1r3x5vruss62NWaonx3 OromLvn4KVPgy/2HFFMSzB1G/8l4qCaGuURmkP1gQwP33TQTxIQNFsFzo1xUbqhP6BhziISsIAfJ0 J/8LxXV+SG3JPPiSR7osSqUoqWo1cLb4fbJJvW0KU/5EED4bVhiWfrhakm+Ag9MtiKFkcvaJljEDU ErzptJxaFqTlJgRiW0ulV3Mie0F6Fv2Vg7Tl0qoj3KFbQ4Z8Oy8RnULEtz1cylQOL/T4FvB+ToIK+ gNDWaFA9Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lE90s-0002ew-W0; Mon, 22 Feb 2021 11:04:27 +0000 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lE90m-0002e4-Sg for linux-arm-kernel@lists.infradead.org; Mon, 22 Feb 2021 11:04:25 +0000 Received: by mail-wr1-x42c.google.com with SMTP id h98so13780121wrh.11 for ; Mon, 22 Feb 2021 03:04:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=w6npELERY6rB3Dp9A7WDHsdNDwVeXG2MhW1a6MllviU=; b=AGxhojrCDx89PT+ATbFNrC9o05X4CdmNDewjWlY4kRAotuz6K4g8lTe6F7MI7TAsd/ t1UhE4GmUc8UU2jRgOT0UAim4SDtwchNBcEbJ2sIByMB5Tz+tmZ+4ScgArPT/rJ8b4p3 ktA6fXbESdmdl6LnOCWbo/LB1Vwth9A4NTM7YLsuGSRnNz2GRmy1Xm4dL4KHhs/rNzzY WwrSZ3pPjl4Jfc5qxHuPLE66PdyE5HIjrFQ18myghhwepi7VO5/ZOpnlmktzJrlhgp20 vR0P9zbdP1TmxvGqisnhieWerU/Zekskp/sE9zs/6zhpfmIcWerjSFCzxJMYQsLcOV+M XCqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=w6npELERY6rB3Dp9A7WDHsdNDwVeXG2MhW1a6MllviU=; b=d6R8K7RyMxtqzQbcBZG5tMEpVSCtcIBWNw+twH43ROsSz3w1ZGn3geTv1857gaWJiE 7PsPPPVWyhj7JtFeTnxP75/eavNYjFsehS6eci0sLibjpYiaaTuFdVLAA3dPvkJTbmwq kTZ7kGp8MIZq4dfvPxCvG0UdGwkXqXuygBwA4K2950nm5ZWri+mIi1J71W6sEBkYtrr1 yfpcNtNnq1BAnD4TzYsH0boLNYyMA/8+PDSlriaz2lT75BSt8WVvy8dYHREZx50BNNkl lGSY7NnKyIINIHNvBGl5JTOkYCj0JTXGSE2TaYd3g6FW/Vis1bhVIcQqNzL1V35rcmSA JQ6w== X-Gm-Message-State: AOAM533GYXz+X3hcj6QqWVfDo+4GD+u0fDTbCwakF7G6+ZmGm4FQ6pkt R+LV543ryuufn9oVBapvZyXcWQ== X-Google-Smtp-Source: ABdhPJxgFDcmq5dsILmm6KR9g+MsM/rBJTmBsyfH/l7oU2+CjNz1rIn8Osx2uhrrwoGXu4U3HE8c4w== X-Received: by 2002:a5d:6148:: with SMTP id y8mr21007272wrt.238.1613991859512; Mon, 22 Feb 2021 03:04:19 -0800 (PST) Received: from google.com (230.69.233.35.bc.googleusercontent.com. [35.233.69.230]) by smtp.gmail.com with ESMTPSA id z21sm20005568wma.29.2021.02.22.03.04.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Feb 2021 03:04:19 -0800 (PST) Date: Mon, 22 Feb 2021 11:04:16 +0000 From: Quentin Perret To: Sean Christopherson Subject: Re: [RFC PATCH v2 16/26] KVM: arm64: Prepare Hyp memory protection Message-ID: References: <20210108121524.656872-1-qperret@google.com> <20210108121524.656872-17-qperret@google.com> <20210203143709.GA18907@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210222_060420_987599_8FC3467E X-CRM114-Status: GOOD ( 37.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , devicetree@vger.kernel.org, kernel-team@android.com, Frank Rowand , Suzuki K Poulose , android-kvm@google.com, Catalin Marinas , Fuad Tabba , linux-kernel@vger.kernel.org, Rob Herring , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , kvmarm@lists.cs.columbia.edu, Julien Thierry Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Sean, On Friday 19 Feb 2021 at 10:32:58 (-0800), Sean Christopherson wrote: > On Wed, Feb 03, 2021, Will Deacon wrote: > > On Fri, Jan 08, 2021 at 12:15:14PM +0000, Quentin Perret wrote: > > ... > > > > +static inline unsigned long hyp_s1_pgtable_size(void) > > > +{ > > ... > > > > + res += nr_pages << PAGE_SHIFT; > > > + } > > > + > > > + /* Allow 1 GiB for private mappings */ > > > + nr_pages = (1 << 30) >> PAGE_SHIFT; > > > > SZ_1G >> PAGE_SHIFT > > Where does the 1gb magic number come from? Admittedly it is arbitrary. It needs to be enough to cover all the so-called 'private' mappings that EL2 needs, and which can vary a little depending on the hardware. > IIUC, this is calculating the number > of pages needed for the hypervisor's Stage-1 page tables. Correct. The thing worth noting is that the hypervisor VA space is essentially split in half. One half is reserved to map portions of memory with a fixed offset, and the other half is used for a whole bunch of other things: we have a vmemmap, the 'private' mappings and the idmap page. > The amount of memory > needed for those page tables should be easily calculated As mentioned above, that is true for pretty much everything in the hyp VA space except the private mappings as that depends on e.g. the CPU uarch and such. > and assuming huge pages can be used, should be far less the 1gb. Ack, though this is no supported for the EL2 mappings yet. Historically the amount of contiguous portions of memory mapped at EL2 has been rather small, so there wasn't really a need, but we might want to revisit this at some point. > > > + nr_pages = __hyp_pgtable_max_pages(nr_pages); > > > + res += nr_pages << PAGE_SHIFT; > > > + > > > + return res; > > ... > > > > +void __init kvm_hyp_reserve(void) > > > +{ > > > + u64 nr_pages, prev; > > > + > > > + if (!is_hyp_mode_available() || is_kernel_in_hyp_mode()) > > > + return; > > > + > > > + if (kvm_get_mode() != KVM_MODE_PROTECTED) > > > + return; > > > + > > > + if (kvm_nvhe_sym(hyp_memblock_nr) < 0) { > > > + kvm_err("Failed to register hyp memblocks\n"); > > > + return; > > > + } > > > + > > > + sort_memblock_regions(); > > > + > > > + /* > > > + * We don't know the number of possible CPUs yet, so allocate for the > > > + * worst case. > > > + */ > > > + hyp_mem_size += NR_CPUS << PAGE_SHIFT; > > Is this for per-cpu stack? Correct. > If so, what guarantees a single page is sufficient? Mostly a curiosity question, > since it looks like this is an existing assumption by init_hyp_mode(). Shouldn't > the required stack size be defined in bytes and converted to pages, or is there a > guarantee that 64kb pages will be used? Nope, we have no such guarantees, but 4K has been more than enough for EL2 so far. The hyp code doesn't use recursion much (I think the only occurence we have is Will's pgtable code, and that is architecturally limited to 4 levels of recursion for obvious reasons) and doesn't have use stack allocations. It's on my todo list to remap the stack pages in the 'private' range, to surround them with guard pages so we can at least run-time check this assumption, so stay tuned :) > > There was a recent patch bumping NR_CPUs to 512, so this would be 32MB > > with 64k pages. Is it possible to return memory to the host later on once > > we have a better handle on the number of CPUs in the system? > > Does kvm_hyp_reserve() really need to be called during bootmem_init()? What > prevents doing the reservation during init_hyp_mode()? If the problem is that > pKVM needs a single contiguous chunk of memory, then it might be worth solving > _that_ problem, e.g. letting the host donate memory in N-byte chunks instead of > requiring a single huge blob of memory. Right, I've been thinking about this over the weekend and that might actually be fairly straightforward for stack pages. I'll try to move this allocation to init_hyp_mode() where it belongs (or better, re-use the existing one) in the nest version. Thanks, Quentin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel