From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30E4AC433EF for ; Tue, 14 Jun 2022 12:02:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CD138D0243; Tue, 14 Jun 2022 08:02:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 86A4B8D0244; Tue, 14 Jun 2022 08:02:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 562108D0243; Tue, 14 Jun 2022 08:02:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 396078D023D for ; Tue, 14 Jun 2022 08:02:45 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0847935EF1 for ; Tue, 14 Jun 2022 12:02:45 +0000 (UTC) X-FDA: 79576704690.12.8431F61 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf02.hostedemail.com (Postfix) with ESMTP id 00547800A2 for ; Tue, 14 Jun 2022 12:02:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655208164; x=1686744164; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bd6faJOVW2FfIjzDQn03hQW9BQt9XmcRmtaVqSAHsus=; b=VI/ZaUFEirvPHOeGDgj/T3vS1hOTnEG5htdXshXnabHXtmg3DuX4v+Yi 0DuWAN+/4PvyV7pvi4VRaZb3gqwXuWUoB+WimDgtBIMLtU3r19dz0Orne 7tQTaAgaPxDnOAOkxkkyxAZToWwzjHsbAXY3eaw8iXdpOjXWo0q5HNqvD y+eJchmBfr9TD28QFod4WKILWYl9WFyKGNZPAx0XxRAozHkWKrChtP0rd EnQcyWAwgZ8U4fYRK9AM8+CAJqp7uOs5iwtuMiYCZQe9h6mAADzUZLCR4 LrZCD1hWW3uH0hveANRRbA05fUNOO9CQmHi6D89+IrxsEUw+0+YRSdn8j w==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276138240" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276138240" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 05:02:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="535534975" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga003.jf.intel.com with ESMTP; 14 Jun 2022 05:02:36 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id D40B792D; Tue, 14 Jun 2022 15:02:32 +0300 (EEST) From: "Kirill A. Shutemov" To: Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel , Ard Biesheuvel Cc: Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , Dave Hansen , Mike Rapoport , David Hildenbrand , marcelo.cerri@canonical.com, tim.gardner@canonical.com, khalid.elmously@canonical.com, philip.cox@canonical.com, x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv7 10/14] x86/mm: Avoid load_unaligned_zeropad() stepping into unaccepted memory Date: Tue, 14 Jun 2022 15:02:27 +0300 Message-Id: <20220614120231.48165-11-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220614120231.48165-1-kirill.shutemov@linux.intel.com> References: <20220614120231.48165-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655208164; a=rsa-sha256; cv=none; b=gonY1WNNlts5Otr6MruZmle8zEM//sBa6TpiD6NyBDmEH3IwRI3Sbfyf6tZ2KT3tzMMXUH iLWxuqI5MhVz9v+KXa81+3HUo5VoI8gan0uP0IB+tdKiyzBURHGQ9yJTTBUtBFwuehiGAZ UcT6cG7qn7Dh5pujGvjWSnITPhJlaJQ= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="VI/ZaUFE"; spf=none (imf02.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.93) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655208164; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kmfRzBMfXCTLbCAYkUrsy7+bxCVXQn86pog+30PajRc=; b=5A8W+qG14ZIYwCZVPA0faSY/h278/zCn9maFhL5nfFHvy3in9Ia2kwD6N/TAUucRPjWa29 YC9452lKGTX3tqCTL+qjcMKlMTSxfDMVirnWFP5VDE1U8HwMHUifkLnMCTy0P/R/ZWkTSE q5uqwUYj31oG16ImnkS7SABBrRBMKSI= X-Stat-Signature: whhrjm9i7wrfaias74pqjyue3dip6gyb X-Rspamd-Queue-Id: 00547800A2 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="VI/ZaUFE"; spf=none (imf02.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.93) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1655208163-631350 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: load_unaligned_zeropad() can lead to unwanted loads across page boundaries. The unwanted loads are typically harmless. But, they might be made to totally unrelated or even unmapped memory. load_unaligned_zeropad() relies on exception fixup (#PF, #GP and now #VE) to recover from these unwanted loads. But, this approach does not work for unaccepted memory. For TDX, a load from unaccepted memory will not lead to a recoverable exception within the guest. The guest will exit to the VMM where the only recourse is to terminate the guest. There are three parts to fix this issue and comprehensively avoid access to unaccepted memory. Together these ensure that an extra “guard” page is accepted in addition to the memory that needs to be used. 1. Implicitly extend the range_contains_unaccepted_memory(start, end) checks up to end+2M if ‘end’ is aligned on a 2M boundary. 2. Implicitly extend accept_memory(start, end) to end+2M if ‘end’ is aligned on a 2M boundary. 3. Set PageUnaccepted() on both memory that itself needs to be accepted *and* memory where the next page needs to be accepted. Essentially, make PageUnaccepted(page) a marker for whether work needs to be done to make ‘page’ usable. That work might include accepting pages in addition to ‘page’ itself. Side note: This leads to something strange. Pages which were accepted at boot, marked by the firmware as accepted and will never _need_ to be accepted might have PageUnaccepted() set on them. PageUnaccepted(page) is a cue to ensure that the next page is accepted before ‘page’ can be used. This is an actual, real-world problem which was discovered during TDX testing. Signed-off-by: Kirill A. Shutemov --- arch/x86/mm/unaccepted_memory.c | 36 +++++++++++++++++++++++++ drivers/firmware/efi/libstub/x86-stub.c | 7 +++++ 2 files changed, 43 insertions(+) diff --git a/arch/x86/mm/unaccepted_memory.c b/arch/x86/mm/unaccepted_memory.c index 1df918b21469..bcd56fe82b9e 100644 --- a/arch/x86/mm/unaccepted_memory.c +++ b/arch/x86/mm/unaccepted_memory.c @@ -23,6 +23,38 @@ void accept_memory(phys_addr_t start, phys_addr_t end) bitmap = __va(boot_params.unaccepted_memory); range_start = start / PMD_SIZE; + /* + * load_unaligned_zeropad() can lead to unwanted loads across page + * boundaries. The unwanted loads are typically harmless. But, they + * might be made to totally unrelated or even unmapped memory. + * load_unaligned_zeropad() relies on exception fixup (#PF, #GP and now + * #VE) to recover from these unwanted loads. + * + * But, this approach does not work for unaccepted memory. For TDX, a + * load from unaccepted memory will not lead to a recoverable exception + * within the guest. The guest will exit to the VMM where the only + * recourse is to terminate the guest. + * + * There are three parts to fix this issue and comprehensively avoid + * access to unaccepted memory. Together these ensure that an extra + * “guard” page is accepted in addition to the memory that needs to be + * used: + * + * 1. Implicitly extend the range_contains_unaccepted_memory(start, end) + * checks up to end+2M if ‘end’ is aligned on a 2M boundary. + * + * 2. Implicitly extend accept_memory(start, end) to end+2M if ‘end’ is + * aligned on a 2M boundary. + * + * 3. Set PageUnaccepted() on both memory that itself needs to be + * accepted *and* memory where the next page needs to be accepted. + * Essentially, make PageUnaccepted(page) a marker for whether work + * needs to be done to make ‘page’ usable. That work might include + * accepting pages in addition to ‘page’ itself. + */ + if (!(end % PMD_SIZE)) + end += PMD_SIZE; + spin_lock_irqsave(&unaccepted_memory_lock, flags); for_each_set_bitrange_from(range_start, range_end, bitmap, DIV_ROUND_UP(end, PMD_SIZE)) { @@ -46,6 +78,10 @@ bool range_contains_unaccepted_memory(phys_addr_t start, phys_addr_t end) bitmap = __va(boot_params.unaccepted_memory); + /* See comment on load_unaligned_zeropad() in accept_memory() */ + if (!(end % PMD_SIZE)) + end += PMD_SIZE; + spin_lock_irqsave(&unaccepted_memory_lock, flags); while (start < end) { if (test_bit(start / PMD_SIZE, bitmap)) { diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index b91c89100b2d..bc1110509de4 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -709,6 +709,13 @@ static efi_status_t allocate_unaccepted_memory(struct boot_params *params, return EFI_SUCCESS; } + /* + * range_contains_unaccepted_memory() may need to check one 2M chunk + * beyond the end of RAM to deal with load_unaligned_zeropad(). Make + * sure that the bitmap is large enough handle it. + */ + max_addr += PMD_SIZE; + /* * If unaccepted memory is present allocate a bitmap to track what * memory has to be accepted before access. -- 2.35.1