From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DD477A70A for ; Wed, 24 Jan 2024 12:56:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=134.134.136.31 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101014; cv=none; b=Ud6KIvEXQZMy8HGcEwu2rnLz1PcZWkk25DaDgtP0X2q4r/nD95H7Evgwog434MFpajCLWuTpesEOOYynsYWM4GPZv9ysZKKwcV5hC4LbVWqqL320/FJaoxOuvEJkNWejisTcVRaJKC1XfZbxHxHhrhb2ivPYAEM+b1aqZ2Jhy9A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101014; c=relaxed/simple; bh=R0O6sMf8J5nkax+miymCAK7F6PZpI1weCFL0hJKn81Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Mb+TYQNO3Yf4sf3Kb08dP6Z2umJoa8e6mpfE5K71ugxGkdANiD7T6qVeSpBuYOMSLanDJISevIRfHG4kvjc7oxUNF8HF3bZmiWKdDtdS4gFNLJSsInRwMHgtMBZGpuWtOawlw3DPBUcTb46Nj3KwxktktgMKIph+kB1xMJu2b60= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QfFO3UYo; arc=none smtp.client-ip=134.134.136.31 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QfFO3UYo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706101013; x=1737637013; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R0O6sMf8J5nkax+miymCAK7F6PZpI1weCFL0hJKn81Q=; b=QfFO3UYoayln5w+eTghgxtGNB/fWxZ8Zi0/O1YWF9bvdmF3ALG0KYbZq 9jT7snsnNYc4qw+DAiXEecrzWIfrNY+tj6Tx5EJPTYygv95CYYQKfcSGj sP27lNsMg11SqWfvKYCZ/Wzpkc6ud4ZRk7Ynen9MxidLVTmi2tjNxNXj8 zrrPJeJsCo2uOqxDF2gTkS1nDHWKV/10nwxNlaqW2qB1vrtGKTygPHNZF kvvrIDklZrqAFlL4vaI+axaVyaLFYAgkBFctLAyTz5cXAOYif05Pw0aw+ v9vXHzte50WKxu24TRvBgNqcrUXjiBxM4AvM8DfZ6rCcyN8Aa0xfYskZr w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="466110104" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="466110104" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2024 04:56:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="735924071" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="735924071" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga003.jf.intel.com with ESMTP; 24 Jan 2024 04:56:47 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 8B190832; Wed, 24 Jan 2024 14:56:02 +0200 (EET) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Cc: "Rafael J. Wysocki" , Peter Zijlstra , Adrian Hunter , Kuppuswamy Sathyanarayanan , Elena Reshetova , Jun Nakajima , Rick Edgecombe , Tom Lendacky , "Kalra, Ashish" , Sean Christopherson , "Huang, Kai" , Baoquan He , kexec@lists.infradead.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv6 09/16] x86/mm: Adding callbacks to prepare encrypted memory for kexec Date: Wed, 24 Jan 2024 14:55:50 +0200 Message-ID: <20240124125557.493675-10-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124125557.493675-1-kirill.shutemov@linux.intel.com> References: <20240124125557.493675-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit AMD SEV and Intel TDX guests allocate shared buffers for performing I/O. This is done by allocating pages normally from the buddy allocator and then converting them to shared using set_memory_decrypted(). On kexec, the second kernel is unaware of which memory has been converted in this manner. It only sees E820_TYPE_RAM. Accessing shared memory as private is fatal. Therefore, the memory state must be reset to its original state before starting the new kernel with kexec. The process of converting shared memory back to private occurs in two steps: - enc_kexec_stop_conversion() stops new conversions. - enc_kexec_unshare_mem() unshares all existing shared memory, reverting it back to private. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/x86_init.h | 2 ++ arch/x86/kernel/crash.c | 5 +++++ arch/x86/kernel/reboot.c | 12 ++++++++++++ arch/x86/kernel/x86_init.c | 4 ++++ 4 files changed, 23 insertions(+) diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h index c9503fe2d13a..3196ff20a29e 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -154,6 +154,8 @@ struct x86_guest { int (*enc_status_change_finish)(unsigned long vaddr, int npages, bool enc); bool (*enc_tlb_flush_required)(bool enc); bool (*enc_cache_flush_required)(void); + void (*enc_kexec_stop_conversion)(bool crash); + void (*enc_kexec_unshare_mem)(void); }; /** diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index b6b044356f1b..6585a5f2c2ba 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -107,6 +107,11 @@ void native_machine_crash_shutdown(struct pt_regs *regs) crash_smp_send_stop(); + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { + x86_platform.guest.enc_kexec_stop_conversion(true); + x86_platform.guest.enc_kexec_unshare_mem(); + } + cpu_emergency_disable_virtualization(); /* diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 830425e6d38e..0574d4ad6b41 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -716,6 +717,14 @@ static void native_machine_emergency_restart(void) void native_machine_shutdown(void) { + /* + * Call enc_kexec_stop_conversion() while all CPUs are still active and + * interrupts are enabled. This will allow all in-flight memory + * conversions to finish cleanly. + */ + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) && kexec_in_progress) + x86_platform.guest.enc_kexec_stop_conversion(false); + /* Stop the cpus and apics */ #ifdef CONFIG_X86_IO_APIC /* @@ -752,6 +761,9 @@ void native_machine_shutdown(void) #ifdef CONFIG_X86_64 x86_platform.iommu_shutdown(); #endif + + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) && kexec_in_progress) + x86_platform.guest.enc_kexec_unshare_mem(); } static void __machine_emergency_restart(int emergency) diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index f0f54e109eb9..b95206ebc621 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -135,6 +135,8 @@ static int enc_status_change_prepare_noop(unsigned long vaddr, int npages, bool static int enc_status_change_finish_noop(unsigned long vaddr, int npages, bool enc) { return 0; } static bool enc_tlb_flush_required_noop(bool enc) { return false; } static bool enc_cache_flush_required_noop(void) { return false; } +static void enc_kexec_stop_conversion_noop(bool crash) {} +static void enc_kexec_unshare_mem_noop(void) {} static bool is_private_mmio_noop(u64 addr) {return false; } struct x86_platform_ops x86_platform __ro_after_init = { @@ -158,6 +160,8 @@ struct x86_platform_ops x86_platform __ro_after_init = { .enc_status_change_finish = enc_status_change_finish_noop, .enc_tlb_flush_required = enc_tlb_flush_required_noop, .enc_cache_flush_required = enc_cache_flush_required_noop, + .enc_kexec_stop_conversion = enc_kexec_stop_conversion_noop, + .enc_kexec_unshare_mem = enc_kexec_unshare_mem_noop, }, }; -- 2.43.0