[PATCH] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[PATCH] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[Kirill A. Shutemov]

RDMSR in the trampoline code overrides EDX, but we use the register to
indicate if 5-level paging has to enabled. It leads to failure to boot
on a 5-level paging machine.

Preserve EDX on the stack while we are dealing with EFER.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Fixes: b677dfae5aa1 ("x86/boot/compressed/64: Set EFER.LME=1 in 32-bit trampoline before returning to long mode")
Reported-by: Kyle D Pelton <kyle.d.pelton@intel.com>
Cc: Wei Huang <wei@redhat.com>
---
 arch/x86/boot/compressed/head_64.S | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index f105ae8651c9..f62e347862cc 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -602,10 +602,12 @@ ENTRY(trampoline_32bit_src)
 3:
 	/* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */
 	pushl	%ecx
+	pushl	%edx
 	movl	$MSR_EFER, %ecx
 	rdmsr
 	btsl	$_EFER_LME, %eax
 	wrmsr
+	popl	%edx
 	popl	%ecx
 
 	/* Enable PAE and LA57 (if required) paging modes */
-- 
2.20.1

Re: [PATCH] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[Borislav Petkov]

On Wed, Feb 06, 2019 at 02:52:53PM +0300, Kirill A. Shutemov wrote:
> RDMSR in the trampoline code overrides EDX, but we use the register to
> indicate if 5-level paging has to enabled. It leads to failure to boot
> on a 5-level paging machine.
> 
> Preserve EDX on the stack while we are dealing with EFER.

Comment says:

 * Non zero RDX on return means we need to enable 5-level paging.

Is that per-chance refering to struct paging_config which
paging_prepare() returns and on that return rdx contains
paging_config.l5_required which is 1 when 5 level is to be enabled?

If so, is that written somewhere explicitly? Because it is not
immediately clear at least to me why that RDX is live...

Thx.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

Re: [PATCH] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[Kirill A. Shutemov]

On Wed, Feb 06, 2019 at 03:21:41PM +0000, Borislav Petkov wrote:
> On Wed, Feb 06, 2019 at 02:52:53PM +0300, Kirill A. Shutemov wrote:
> > RDMSR in the trampoline code overrides EDX, but we use the register to
> > indicate if 5-level paging has to enabled. It leads to failure to boot
> > on a 5-level paging machine.
> > 
> > Preserve EDX on the stack while we are dealing with EFER.
> 
> Comment says:
> 
>  * Non zero RDX on return means we need to enable 5-level paging.
> 
> Is that per-chance refering to struct paging_config which
> paging_prepare() returns and on that return rdx contains
> paging_config.l5_required which is 1 when 5 level is to be enabled?

Yes.

> If so, is that written somewhere explicitly? Because it is not
> immediately clear at least to me why that RDX is live...

What about this:

From d43b9d157c574baf782f6d9982fe6f2c1f918c0e Mon Sep 17 00:00:00 2001
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Date: Wed, 6 Feb 2019 18:29:08 +0300
Subject: [PATCH] x86/boot/compressed/64: Fix confusing comment for
 paging_config()

paging_prepare() returns two-quadword structure which lands
into RDX:RAX:
  - Address of the trampoline is returned in RAX.
  - Non zero RDX means trampoline needs to enable 5-level paging.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/boot/compressed/head_64.S | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index f62e347862cc..87509a3f00f4 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -358,8 +358,11 @@ ENTRY(startup_64)
 	 * paging_prepare() sets up the trampoline and checks if we need to
 	 * enable 5-level paging.
 	 *
-	 * Address of the trampoline is returned in RAX.
-	 * Non zero RDX on return means we need to enable 5-level paging.
+	 * paging_prepare() returns two-quadword structure which lands
+	 * into RDX:RAX:
+	 *   - Address of the trampoline is returned in RAX.
+	 *   - Non zero RDX means trampoline needs to enable 5-level
+	 *     paging.
 	 *
 	 * RSI holds real mode data and needs to be preserved across
 	 * this function call.
@@ -565,7 +568,7 @@ adjust_got:
  *
  * RDI contains the return address (might be above 4G).
  * ECX contains the base address of the trampoline memory.
- * Non zero RDX on return means we need to enable 5-level paging.
+ * Non zero RDX means trampoline needs to enable 5-level paging.
  */
 ENTRY(trampoline_32bit_src)
 	/* Set up data and stack segments */
-- 
 Kirill A. Shutemov

[tip:x86/urgent] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[tip-bot for Kirill A. Shutemov]

Commit-ID:  45b13b424faafb81c8c44541f093a682fdabdefc
Gitweb:     https://git.kernel.org/tip/45b13b424faafb81c8c44541f093a682fdabdefc
Author:     Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
AuthorDate: Wed, 6 Feb 2019 14:52:53 +0300
Committer:  Borislav Petkov <bp@suse.de>
CommitDate: Wed, 6 Feb 2019 18:56:18 +0100

x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

RDMSR in the trampoline code overwrites EDX but that register is used
to indicate whether 5-level paging has to be enabled and if clobbered,
leads to failure to boot on a 5-level paging machine.

Preserve EDX on the stack while we are dealing with EFER.

Fixes: b677dfae5aa1 ("x86/boot/compressed/64: Set EFER.LME=1 in 32-bit trampoline before returning to long mode")
Reported-by: Kyle D Pelton <kyle.d.pelton@intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: dave.hansen@linux.intel.com
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wei Huang <wei@redhat.com>
Cc: x86-ml <x86@kernel.org>
Link: https://lkml.kernel.org/r/20190206115253.1907-1-kirill.shutemov@linux.intel.com
---
 arch/x86/boot/compressed/head_64.S | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index f105ae8651c9..f62e347862cc 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -602,10 +602,12 @@ ENTRY(trampoline_32bit_src)
 3:
 	/* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */
 	pushl	%ecx
+	pushl	%edx
 	movl	$MSR_EFER, %ecx
 	rdmsr
 	btsl	$_EFER_LME, %eax
 	wrmsr
+	popl	%edx
 	popl	%ecx
 
 	/* Enable PAE and LA57 (if required) paging modes */

Re: [PATCH] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[Borislav Petkov]

On Wed, Feb 06, 2019 at 06:47:57PM +0300, Kirill A. Shutemov wrote:
> What about this:

Yap, thanks!

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

[tip:x86/cleanups] x86/boot/compressed/64: Explain paging_prepare()'s return value

[tip-bot for Kirill A. Shutemov]

Commit-ID:  82434d23f36de42f70925f70d645ed3b1394361b
Gitweb:     https://git.kernel.org/tip/82434d23f36de42f70925f70d645ed3b1394361b
Author:     Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
AuthorDate: Wed, 6 Feb 2019 18:29:08 +0300
Committer:  Borislav Petkov <bp@suse.de>
CommitDate: Wed, 6 Feb 2019 19:08:34 +0100

x86/boot/compressed/64: Explain paging_prepare()'s return value

paging_prepare() returns a two-quadword structure which lands
into RDX:RAX:

  - Address of the trampoline is returned in RAX.
  - Non zero RDX means trampoline needs to enable 5-level paging.

Document that explicitly.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: dave.hansen@linux.intel.com
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kyle D Pelton <kyle.d.pelton@intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wei Huang <wei@redhat.com>
Cc: x86-ml <x86@kernel.org>
Link: https://lkml.kernel.org/r/20190206154756.matwldebbxkmlnae@black.fi.intel.com
---
 arch/x86/boot/compressed/head_64.S | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index b27b338d2f6d..73b9d7e91a9c 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -358,8 +358,11 @@ ENTRY(startup_64)
 	 * paging_prepare() sets up the trampoline and checks if we need to
 	 * enable 5-level paging.
 	 *
-	 * Address of the trampoline is returned in RAX.
-	 * Non zero RDX on return means we need to enable 5-level paging.
+	 * paging_prepare() returns a two-quadword structure which lands
+	 * into RDX:RAX:
+	 *   - Address of the trampoline is returned in RAX.
+	 *   - Non zero RDX means trampoline needs to enable 5-level
+	 *     paging.
 	 *
 	 * RSI holds real mode data and needs to be preserved across
 	 * this function call.
@@ -565,7 +568,7 @@ adjust_got:
  *
  * RDI contains the return address (might be above 4G).
  * ECX contains the base address of the trampoline memory.
- * Non zero RDX on return means we need to enable 5-level paging.
+ * Non zero RDX means trampoline needs to enable 5-level paging.
  */
 ENTRY(trampoline_32bit_src)
 	/* Set up data and stack segments */

Re: [PATCH] x86/boot/compressed/64: Do not corrupt EDX on EFER.LME=1 setting

[Wei Huang]

On 2/6/19 5:52 AM, Kirill A. Shutemov wrote:
> RDMSR in the trampoline code overrides EDX, but we use the register to
> indicate if 5-level paging has to enabled. It leads to failure to boot
> on a 5-level paging machine.
> 
> Preserve EDX on the stack while we are dealing with EFER.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: b677dfae5aa1 ("x86/boot/compressed/64: Set EFER.LME=1 in 32-bit trampoline before returning to long mode")
> Reported-by: Kyle D Pelton <kyle.d.pelton@intel.com>
> Cc: Wei Huang <wei@redhat.com>
> ---
>  arch/x86/boot/compressed/head_64.S | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
> index f105ae8651c9..f62e347862cc 100644
> --- a/arch/x86/boot/compressed/head_64.S
> +++ b/arch/x86/boot/compressed/head_64.S
> @@ -602,10 +602,12 @@ ENTRY(trampoline_32bit_src)
>  3:
>  	/* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */
>  	pushl	%ecx
> +	pushl	%edx
>  	movl	$MSR_EFER, %ecx
>  	rdmsr
>  	btsl	$_EFER_LME, %eax
>  	wrmsr
> +	popl	%edx
>  	popl	%ecx
>  
>  	/* Enable PAE and LA57 (if required) paging modes */
> 

Oops, rdmsr indeed corrupts EDX.

Acked-by: Wei Huang <wei@redhat.com>