[PATCH v2] x86/setup: call early_reserve_memory() earlier

[PATCH v2] x86/setup: call early_reserve_memory() earlier

[Juergen Gross]

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
reservations") introduced early_reserve_memory() to do all needed
initial memblock_reserve() calls in one function. Unfortunately the
call of early_reserve_memory() is done too late for Xen dom0, as in
some cases a Xen hook called by e820__memory_setup() will need those
memory reservations to have happened already.

Move the call of early_reserve_memory() before the call of
e820__memory_setup() in order to avoid such problems.

Cc: stable@vger.kernel.org
Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)
---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 79f164141116..40ed44ead063 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();
@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
-- 
2.26.2

Re: [PATCH v2] x86/setup: call early_reserve_memory() earlier

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 2959 bytes --]

On Mon, Sep 20, 2021 at 02:04:21PM +0200, Juergen Gross wrote:
> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> reservations") introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately the
> call of early_reserve_memory() is done too late for Xen dom0, as in
> some cases a Xen hook called by e820__memory_setup() will need those
> memory reservations to have happened already.
> 
> Move the call of early_reserve_memory() before the call of
> e820__memory_setup() in order to avoid such problems.
> 
> Cc: stable@vger.kernel.org
> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
> Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>

I confirm this fixes my boot issue too.

Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> ---
> V2:
> - update comment (Jan Beulich, Boris Petkov)
> - move call down in setup_arch() (Mike Galbraith)
> ---
>  arch/x86/kernel/setup.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f164141116..40ed44ead063 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
>  
>  	x86_init.oem.arch_setup();
>  
> +	/*
> +	 * Do some memory reservations *before* memory is added to memblock, so
> +	 * memblock allocations won't overwrite it.
> +	 *
> +	 * After this point, everything still needed from the boot loader or
> +	 * firmware or kernel text should be early reserved or marked not RAM in
> +	 * e820. All other memory is free game.
> +	 *
> +	 * This call needs to happen before e820__memory_setup() which calls the
> +	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
> +	 * early reservations have happened already.
> +	 */
> +	early_reserve_memory();
> +
>  	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
>  	e820__memory_setup();
>  	parse_setup_data();
> @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
>  
>  	parse_early_param();
>  
> -	/*
> -	 * Do some memory reservations *before* memory is added to
> -	 * memblock, so memblock allocations won't overwrite it.
> -	 * Do it after early param, so we could get (unlikely) panic from
> -	 * serial.
> -	 *
> -	 * After this point everything still needed from the boot loader or
> -	 * firmware or kernel text should be early reserved or marked not
> -	 * RAM in e820. All other memory is free game.
> -	 */
> -	early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
> -- 
> 2.26.2
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2] x86/setup: call early_reserve_memory() earlier

[Nathan Chancellor]

On Mon, Sep 20, 2021 at 02:04:21PM +0200, Juergen Gross wrote:
> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> reservations") introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately the
> call of early_reserve_memory() is done too late for Xen dom0, as in
> some cases a Xen hook called by e820__memory_setup() will need those
> memory reservations to have happened already.
> 
> Move the call of early_reserve_memory() before the call of
> e820__memory_setup() in order to avoid such problems.
> 
> Cc: stable@vger.kernel.org
> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
> Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>

I had issues on an AMD Ryzen 3 4300G based system with v1. v2 does not
trigger any boot issues on that same machine or an Intel i5-4210U based
system that I also test with.

Tested-by: Nathan Chancellor <nathan@kernel.org>

> ---
> V2:
> - update comment (Jan Beulich, Boris Petkov)
> - move call down in setup_arch() (Mike Galbraith)
> ---
>  arch/x86/kernel/setup.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f164141116..40ed44ead063 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
>  
>  	x86_init.oem.arch_setup();
>  
> +	/*
> +	 * Do some memory reservations *before* memory is added to memblock, so
> +	 * memblock allocations won't overwrite it.
> +	 *
> +	 * After this point, everything still needed from the boot loader or
> +	 * firmware or kernel text should be early reserved or marked not RAM in
> +	 * e820. All other memory is free game.
> +	 *
> +	 * This call needs to happen before e820__memory_setup() which calls the
> +	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
> +	 * early reservations have happened already.
> +	 */
> +	early_reserve_memory();
> +
>  	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
>  	e820__memory_setup();
>  	parse_setup_data();
> @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
>  
>  	parse_early_param();
>  
> -	/*
> -	 * Do some memory reservations *before* memory is added to
> -	 * memblock, so memblock allocations won't overwrite it.
> -	 * Do it after early param, so we could get (unlikely) panic from
> -	 * serial.
> -	 *
> -	 * After this point everything still needed from the boot loader or
> -	 * firmware or kernel text should be early reserved or marked not
> -	 * RAM in e820. All other memory is free game.
> -	 */
> -	early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
> -- 
> 2.26.2
> 
> 

[tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[tip-bot2 for Juergen Gross]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     8aa83e6395ce047a506f0b16edca45f36c1ae7f8
Gitweb:        https://git.kernel.org/tip/8aa83e6395ce047a506f0b16edca45f36c1ae7f8
Author:        Juergen Gross <jgross@suse.com>
AuthorDate:    Mon, 20 Sep 2021 14:04:21 +02:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Tue, 21 Sep 2021 09:52:08 +02:00

x86/setup: Call early_reserve_memory() earlier

Commit in Fixes introduced early_reserve_memory() to do all needed
initial memblock_reserve() calls in one function. Unfortunately, the call
of early_reserve_memory() is done too late for Xen dom0, as in some
cases a Xen hook called by e820__memory_setup() will need those memory
reservations to have happened already.

Move the call of early_reserve_memory() before the call of
e820__memory_setup() in order to avoid such problems.

Fixes: a799c2bd29d1 ("x86/setup: Consolidate early memory reservations")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com
---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 79f1641..40ed44e 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();
@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Williams, Dan J]

On Wed, 2021-09-22 at 17:18 +0000, tip-bot2 for Juergen Gross wrote:
> The following commit has been merged into the x86/urgent branch of
> tip:
> 
> Commit-ID:     8aa83e6395ce047a506f0b16edca45f36c1ae7f8
> Gitweb:       
> https://git.kernel.org/tip/8aa83e6395ce047a506f0b16edca45f36c1ae7f8
> Author:        Juergen Gross <jgross@suse.com>
> AuthorDate:    Mon, 20 Sep 2021 14:04:21 +02:00
> Committer:     Borislav Petkov <bp@suse.de>
> CommitterDate: Tue, 21 Sep 2021 09:52:08 +02:00
> 
> x86/setup: Call early_reserve_memory() earlier

Hi,

I got a report from Anjaneya that starting with the v5.15 kernel he can
no longer use the "efi=nosoftreserve" kernel command line parameter to
suppress "soft reservation" behavior. Recall that "soft reserved" is
the Linux designation for memory that is marked with the EFI "Special
Purpose" attribute.

By inspection, this commit looks like the source of the problem because
early_reserve_memory() now runs before parse_early_param(). I suspect
that this also affects memmap= since it is also an early_param(). I'll
verify that tomorrow when I'm more awake, but wanted to give a heads up
in the meantime.


> 
> Commit in Fixes introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately, the call
> of early_reserve_memory() is done too late for Xen dom0, as in some
> cases a Xen hook called by e820__memory_setup() will need those memory
> reservations to have happened already.
> 
> Move the call of early_reserve_memory() before the call of
> e820__memory_setup() in order to avoid such problems.
> 
> Fixes: a799c2bd29d1 ("x86/setup: Consolidate early memory reservations")
> Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Signed-off-by: Borislav Petkov <bp@suse.de>
> Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
> Cc: stable@vger.kernel.org
> Link: https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com
> ---
>  arch/x86/kernel/setup.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f1641..40ed44e 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
>  
>         x86_init.oem.arch_setup();
>  
> +       /*
> +        * Do some memory reservations *before* memory is added to memblock, so
> +        * memblock allocations won't overwrite it.
> +        *
> +        * After this point, everything still needed from the boot loader or
> +        * firmware or kernel text should be early reserved or marked not RAM in
> +        * e820. All other memory is free game.
> +        *
> +        * This call needs to happen before e820__memory_setup() which calls the
> +        * xen_memory_setup() on Xen dom0 which relies on the fact that those
> +        * early reservations have happened already.
> +        */
> +       early_reserve_memory();
> +
>         iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
>         e820__memory_setup();
>         parse_setup_data();
> @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
>  
>         parse_early_param();
>  
> -       /*
> -        * Do some memory reservations *before* memory is added to
> -        * memblock, so memblock allocations won't overwrite it.
> -        * Do it after early param, so we could get (unlikely) panic from
> -        * serial.
> -        *
> -        * After this point everything still needed from the boot loader or
> -        * firmware or kernel text should be early reserved or marked not
> -        * RAM in e820. All other memory is free game.
> -        */
> -       early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>         /*
>          * Memory used by the kernel cannot be hot-removed because Linux

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Borislav Petkov]

+ Mike.

On Thu, Nov 04, 2021 at 05:38:54AM +0000, Williams, Dan J wrote:
> By inspection, this commit looks like the source of the problem because
> early_reserve_memory() now runs before parse_early_param().

Yah, I see it too:

early_reserve_memory
|-> efi_memblock_x86_reserve_range
    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed
"nosoftreserve".

And parse_early_param() happens later.

Uuuf, that early memory reservation dance is not going to be over,
ever...

Well, I guess we can do something like this below. The intent being to
carve out the command line preparation into a separate function which
does the early param parsing too. So that it all goes together.

And then call that function before early_reserve_memory() so that the
params have been parsed by then.

Looking at the changed flow, I think we should be ok but I've said that
a bunch of times already regarding this memory reservation early and
something in our house of cards called early boot order always broke.

Damn.

---
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 40ed44ead063..05f69e7d84dc 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
+static char *prepare_command_line(void)
+{
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	parse_early_param();
+
+	return command_line;
+}
+
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures
@@ -830,6 +852,23 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * x86_configure_nx() is called before parse_early_param() (called by
+	 * prepare_command_line()) to detect whether hardware doesn't support
+	 * NX (so that the early EHCI debug console setup can safely call
+	 * set_fixmap()). It may then be called again from within noexec_setup()
+	 * during parsing early parameters to honor the respective command line
+	 * option.
+	 */
+	x86_configure_nx();
+
+	/*
+	 * This parses early params and it needs to run before
+	 * early_reserve_memory() because latter relies on such settings
+	 * supplies as early params.
+	 */
+	*cmdline_p = prepare_command_line();
+
 	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.
@@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-	*cmdline_p = command_line;
-
-	/*
-	 * x86_configure_nx() is called before parse_early_param() to detect
-	 * whether hardware doesn't support NX (so that the early EHCI debug
-	 * console setup can safely call set_fixmap()). It may then be called
-	 * again from within noexec_setup() during parsing early parameters
-	 * to honor the respective command line option.
-	 */
-	x86_configure_nx();
-
-	parse_early_param();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux


-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Dan Williams]

On Thu, Nov 4, 2021 at 4:40 AM Borislav Petkov <bp@alien8.de> wrote:
>
> + Mike.
>
> On Thu, Nov 04, 2021 at 05:38:54AM +0000, Williams, Dan J wrote:
> > By inspection, this commit looks like the source of the problem because
> > early_reserve_memory() now runs before parse_early_param().
>
> Yah, I see it too:
>
> early_reserve_memory
> |-> efi_memblock_x86_reserve_range
>     |-> efi_fake_memmap_early
>
> which does
>
>         if (!efi_soft_reserve_enabled())
>                 return;
>
> and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed
> "nosoftreserve".
>
> And parse_early_param() happens later.
>
> Uuuf, that early memory reservation dance is not going to be over,
> ever...
>
> Well, I guess we can do something like this below. The intent being to
> carve out the command line preparation into a separate function which
> does the early param parsing too. So that it all goes together.
>
> And then call that function before early_reserve_memory() so that the
> params have been parsed by then.
>
> Looking at the changed flow, I think we should be ok but I've said that
> a bunch of times already regarding this memory reservation early and
> something in our house of cards called early boot order always broke.
>
> Damn.

Thanks Boris!

You can add:

Tested-by: Anjaneya Chagam <anjaneya.chagam@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>

>
> ---
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 40ed44ead063..05f69e7d84dc 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
>         return 0;
>  }
>
> +static char *prepare_command_line(void)
> +{
> +#ifdef CONFIG_CMDLINE_BOOL
> +#ifdef CONFIG_CMDLINE_OVERRIDE
> +       strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
> +#else
> +       if (builtin_cmdline[0]) {
> +               /* append boot loader cmdline to builtin */
> +               strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
> +               strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
> +               strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
> +       }
> +#endif
> +#endif
> +
> +       strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
> +
> +       parse_early_param();
> +
> +       return command_line;
> +}
> +
>  /*
>   * Determine if we were loaded by an EFI loader.  If so, then we have also been
>   * passed the efi memmap, systab, etc., so we should use these data structures
> @@ -830,6 +852,23 @@ void __init setup_arch(char **cmdline_p)
>
>         x86_init.oem.arch_setup();
>
> +       /*
> +        * x86_configure_nx() is called before parse_early_param() (called by
> +        * prepare_command_line()) to detect whether hardware doesn't support
> +        * NX (so that the early EHCI debug console setup can safely call
> +        * set_fixmap()). It may then be called again from within noexec_setup()
> +        * during parsing early parameters to honor the respective command line
> +        * option.
> +        */
> +       x86_configure_nx();
> +
> +       /*
> +        * This parses early params and it needs to run before
> +        * early_reserve_memory() because latter relies on such settings
> +        * supplies as early params.
> +        */
> +       *cmdline_p = prepare_command_line();
> +
>         /*
>          * Do some memory reservations *before* memory is added to memblock, so
>          * memblock allocations won't overwrite it.
> @@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p)
>         bss_resource.start = __pa_symbol(__bss_start);
>         bss_resource.end = __pa_symbol(__bss_stop)-1;
>
> -#ifdef CONFIG_CMDLINE_BOOL
> -#ifdef CONFIG_CMDLINE_OVERRIDE
> -       strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
> -#else
> -       if (builtin_cmdline[0]) {
> -               /* append boot loader cmdline to builtin */
> -               strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
> -               strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
> -               strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
> -       }
> -#endif
> -#endif
> -
> -       strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
> -       *cmdline_p = command_line;
> -
> -       /*
> -        * x86_configure_nx() is called before parse_early_param() to detect
> -        * whether hardware doesn't support NX (so that the early EHCI debug
> -        * console setup can safely call set_fixmap()). It may then be called
> -        * again from within noexec_setup() during parsing early parameters
> -        * to honor the respective command line option.
> -        */
> -       x86_configure_nx();
> -
> -       parse_early_param();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>         /*
>          * Memory used by the kernel cannot be hot-removed because Linux
>
>
> --
> Regards/Gruss,
>     Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette

[tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[tip-bot2 for Borislav Petkov]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     8d48bf8206f77aa8687f0e241e901e5197e52423
Gitweb:        https://git.kernel.org/tip/8d48bf8206f77aa8687f0e241e901e5197e52423
Author:        Borislav Petkov <bp@suse.de>
AuthorDate:    Fri, 05 Nov 2021 10:41:51 +01:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Mon, 15 Nov 2021 12:27:40 +01:00

x86/boot: Pull up cmdline preparation and early param parsing

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve
kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range
    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed
"nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot
cmdline not being taken into account.

Therefore, carve out the command line preparation into a separate
function which does the early param parsing too. So that it all goes
together.

And then call that function before early_reserve_memory() so that the
params would have been parsed by then.

Fixes: 8aa83e6395ce ("x86/setup: Call early_reserve_memory() earlier")
Reported-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Tested-by: Anjaneya Chagam <anjaneya.chagam@intel.com>
Cc: <stable@vger.kernel.org>
Link: https://lore.kernel.org/r/e8dd8993c38702ee6dd73b3c11f158617e665607.camel@intel.com
---
 arch/x86/kernel/setup.c | 66 +++++++++++++++++++++++-----------------
 1 file changed, 39 insertions(+), 27 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 49b596d..c410be7 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
+static char *prepare_command_line(void)
+{
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	parse_early_param();
+
+	return command_line;
+}
+
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures
@@ -831,6 +853,23 @@ void __init setup_arch(char **cmdline_p)
 	x86_init.oem.arch_setup();
 
 	/*
+	 * x86_configure_nx() is called before parse_early_param() (called by
+	 * prepare_command_line()) to detect whether hardware doesn't support
+	 * NX (so that the early EHCI debug console setup can safely call
+	 * set_fixmap()). It may then be called again from within noexec_setup()
+	 * during parsing early parameters to honor the respective command line
+	 * option.
+	 */
+	x86_configure_nx();
+
+	/*
+	 * This parses early params and it needs to run before
+	 * early_reserve_memory() because latter relies on such settings
+	 * supplied as early params.
+	 */
+	*cmdline_p = prepare_command_line();
+
+	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.
 	 *
@@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-	*cmdline_p = command_line;
-
-	/*
-	 * x86_configure_nx() is called before parse_early_param() to detect
-	 * whether hardware doesn't support NX (so that the early EHCI debug
-	 * console setup can safely call set_fixmap()). It may then be called
-	 * again from within noexec_setup() during parsing early parameters
-	 * to honor the respective command line option.
-	 */
-	x86_configure_nx();
-
-	parse_early_param();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[John Dorminy]

Greetings;

It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately... 

As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.

I apologize that I don't know how to fix this, but I'm happy to test patches.

Typical dmesg output showing the lack of effect, built from the prior change and this change:

With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working):
[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G
...
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
...
[    0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)

Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable
...
[    0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

+ Hugh and Patrick.

On Thu, Dec 09, 2021 at 09:38:10AM -0500, John Dorminy wrote:
> Greetings;
> 
> It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately... 
> 
> As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
> 
> I apologize that I don't know how to fix this, but I'm happy to test patches.

Yeah, people have been reporting boot failures with mem= on the cmdline.

I think I see why, can you try this one:

---
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 6a190c7f4d71..6db971e61e4b 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p)
 	 */
 	x86_configure_nx();
 
+	e820__memory_setup();
+
 	/*
 	 * This parses early params and it needs to run before
 	 * early_reserve_memory() because latter relies on such settings
@@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p)
 	early_reserve_memory();
 
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
-	e820__memory_setup();
 	parse_setup_data();
 
 	copy_edd();
---

Leaving in the rest for the newly added folks.

> Typical dmesg output showing the lack of effect, built from the prior change and this change:
> 
> With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working):
> [    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G
> ...
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
> [    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
> [    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
> [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
> [    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
> [    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
> [    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
> [    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
> [    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
> [    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
> [    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
> [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
> [    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
> [    0.000000] NX (Execute Disable) protection: active
> [    0.000000] user-defined physical RAM map:
> [    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
> [    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
> [    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
> [    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
> [    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
> [    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
> [    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
> [    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
> [    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
> [    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
> [    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
> ...
> [    0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)
> 
> Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):
> 
> [    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
> [    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
> [    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
> [    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
> [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
> [    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
> [    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
> [    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
> [    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
> [    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
> [    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
> [    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
> [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
> [    0.000000] NX (Execute Disable) protection: active
> [    0.000000] user-defined physical RAM map:
> [    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
> [    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
> [    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
> [    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
> [    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
> [    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
> [    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
> [    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
> [    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
> [    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
> [    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
> [    0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable
> ...
> [    0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)
> 

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 7145 bytes --]

On 09.12.21 16:19, Borislav Petkov wrote:
> + Hugh and Patrick.
> 
> On Thu, Dec 09, 2021 at 09:38:10AM -0500, John Dorminy wrote:
>> Greetings;
>>
>> It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
>>
>> As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
>>
>> I apologize that I don't know how to fix this, but I'm happy to test patches.
> 
> Yeah, people have been reporting boot failures with mem= on the cmdline.
> 
> I think I see why, can you try this one:

Sigh. This will break Xen PV. Again. The comment above the call of
early_reserve_memory() tells you why.

Juergen

> 
> ---
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 6a190c7f4d71..6db971e61e4b 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p)
>   	 */
>   	x86_configure_nx();
>   
> +	e820__memory_setup();
> +
>   	/*
>   	 * This parses early params and it needs to run before
>   	 * early_reserve_memory() because latter relies on such settings
> @@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p)
>   	early_reserve_memory();
>   
>   	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
> -	e820__memory_setup();
>   	parse_setup_data();
>   
>   	copy_edd();
> ---
> 
> Leaving in the rest for the newly added folks.
> 
>> Typical dmesg output showing the lack of effect, built from the prior change and this change:
>>
>> With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working):
>> [    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G
>> ...
>> [    0.000000] BIOS-provided physical RAM map:
>> [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
>> [    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
>> [    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
>> [    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
>> [    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
>> [    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
>> [    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
>> [    0.000000] NX (Execute Disable) protection: active
>> [    0.000000] user-defined physical RAM map:
>> [    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
>> [    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
>> [    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
>> [    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
>> [    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
>> [    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
>> [    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
>> [    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
>> [    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
>> [    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
>> [    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
>> ...
>> [    0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)
>>
>> Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):
>>
>> [    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
>> [    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
>> [    0.000000] BIOS-provided physical RAM map:
>> [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
>> [    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
>> [    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
>> [    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
>> [    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
>> [    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
>> [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
>> [    0.000000] NX (Execute Disable) protection: active
>> [    0.000000] user-defined physical RAM map:
>> [    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
>> [    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
>> [    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
>> [    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
>> [    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
>> [    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
>> [    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
>> [    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
>> [    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
>> [    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
>> [    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
>> [    0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable
>> ...
>> [    0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)
>>
> 


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Thu, Dec 09, 2021 at 04:26:55PM +0100, Juergen Gross wrote:
> Sigh. This will break Xen PV. Again. The comment above the call of
> early_reserve_memory() tells you why.

I know. I was just looking at how to fix that particular thing and was
going to find you on IRC to talk to you about it...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Mike Rapoport]

On Thu, Dec 09, 2021 at 04:28:48PM +0100, Borislav Petkov wrote:
> On Thu, Dec 09, 2021 at 04:26:55PM +0100, Juergen Gross wrote:
> > Sigh. This will break Xen PV. Again. The comment above the call of
> > early_reserve_memory() tells you why.
> 
> I know. I was just looking at how to fix that particular thing and was
> going to find you on IRC to talk to you about it...

The memory reservation in arch/x86/platform/efi/efi.c depends on at least
two command line parameters, I think it's better put it back later in the
boot process and move efi_memblock_x86_reserve_range() out of
early_memory_reserve().

I.e. revert c0f2077baa41 ("x86/boot: Mark prepare_command_line() __init")
and 8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param
parsing") and add the patch below on top.

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 49b596db5631..da36b8f8430b 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();
@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT)) {
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

-- 
Sincerely yours,
Mike.

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Thu, Dec 09, 2021 at 06:29:27PM +0200, Mike Rapoport wrote:
> On Thu, Dec 09, 2021 at 04:28:48PM +0100, Borislav Petkov wrote:
> > On Thu, Dec 09, 2021 at 04:26:55PM +0100, Juergen Gross wrote:
> > > Sigh. This will break Xen PV. Again. The comment above the call of
> > > early_reserve_memory() tells you why.
> > 
> > I know. I was just looking at how to fix that particular thing and was
> > going to find you on IRC to talk to you about it...
> 
> The memory reservation in arch/x86/platform/efi/efi.c depends on at least
> two command line parameters, I think it's better put it back later in the
> boot process and move efi_memblock_x86_reserve_range() out of
> early_memory_reserve().
> 
> I.e. revert c0f2077baa41 ("x86/boot: Mark prepare_command_line() __init")
> and 8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param
> parsing") and add the patch below on top.
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 49b596db5631..da36b8f8430b 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
>  
>  	early_reserve_initrd();
>  
> -	if (efi_enabled(EFI_BOOT))
> -		efi_memblock_x86_reserve_range();
> -
>  	memblock_x86_reserve_range_setup_data();
>  
>  	reserve_ibft_region();
> @@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
>  
>  	parse_early_param();
>  
> +	if (efi_enabled(EFI_BOOT)) {
> +		efi_memblock_x86_reserve_range();
> +
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
> 
> -- 

Jürgen and I were thinking about a different fix but that's probably
ok too. But I've said that already about this mess and there's always
something we haven't thought about.

Whatever we do, it needs to be tested by all folks on Cc who already
reported regressions, i.e., Anjaneya, Hugh, John and Patrick.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
> Whatever we do, it needs to be tested by all folks on Cc who already
> reported regressions, i.e., Anjaneya, Hugh, John and Patrick.

Ok, Mike is busy so here are some patches for testing:

https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot

I'd appreciate it if folks who reported an issue, verify those.

The first two are reverts which should address the issues with mem=
folks have reported. And the last one should address Anjaneya's issue.

I guess doing it the way as Mike suggested is cleaner/better.

Thx!

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Hugh Dickins]

On Fri, 10 Dec 2021, Borislav Petkov wrote:
> On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
> > Whatever we do, it needs to be tested by all folks on Cc who already
> > reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
> 
> Ok, Mike is busy so here are some patches for testing:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
> 
> I'd appreciate it if folks who reported an issue, verify those.
> 
> The first two are reverts which should address the issues with mem=
> folks have reported. And the last one should address Anjaneya's issue.
> 
> I guess doing it the way as Mike suggested is cleaner/better.

Yes, mem= works fine for me, on both machines, 64-bit and 32-bit,
thanks; but I'm not exercising the troublesome EFI case at all.

Hugh

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Fri, Dec 10, 2021 at 12:11:02PM -0800, Hugh Dickins wrote:
> Yes, mem= works fine for me, on both machines, 64-bit and 32-bit,
> thanks;

Thanks!

> but I'm not exercising the troublesome EFI case at all.

Yeah, I added some debug printks in a VM yesterday to confirm the
ordering. But will give Anjaneya some more time to verify, before I
queue them next week.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Patrick J. Volkerding]

On Fri, Dec 10, 2021 at 5:28 AM Borislav Petkov <bp@alien8.de> wrote:
>
> On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
> > Whatever we do, it needs to be tested by all folks on Cc who already
> > reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
>
> Ok, Mike is busy so here are some patches for testing:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
>
> I'd appreciate it if folks who reported an issue, verify those.
>
> The first two are reverts which should address the issues with mem=
> folks have reported. And the last one should address Anjaneya's issue.

I applied the two revert patches to 5.15.7 (the last one won't apply
so I skipped it) and the resulting x86 32-bit kernel boots fine here
on the Thinkpad X1E that was having issues previously.

Then I tested an unpatched 5.16-rc4, which (as expected) got the boot
hang on the affected machine. Applied the three patches, and the
resulting kernel boots fine.

Take care,

Pat

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Fri, Dec 10, 2021 at 02:32:38PM -0600, Patrick J. Volkerding wrote:
> I applied the two revert patches to 5.15.7 (the last one won't apply
> so I skipped it)

Looks like it is only a contextual conflict.

> and the resulting x86 32-bit kernel boots fine here on the Thinkpad
> X1E that was having issues previously.

Good.

> Then I tested an unpatched 5.16-rc4, which (as expected) got the boot
> hang on the affected machine. Applied the three patches, and the
> resulting kernel boots fine.

Thanks for testing!

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[John Dorminy]

Apologies for delay; my dev machine was broken much of today. But I
have tested this patch under the same conditions as previously, and
agree with Hugh that the patches make mem= work correctly.

Thanks!

John Dorminy

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Sat, Dec 11, 2021 at 12:24:25AM -0500, John Dorminy wrote:
> Apologies for delay; my dev machine was broken much of today. But I
> have tested this patch under the same conditions as previously, and
> agree with Hugh that the patches make mem= work correctly.
> 
> Thanks!

Thanks for testing!

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Mike Rapoport]

On Fri, Dec 10, 2021 at 12:28:09PM +0100, Borislav Petkov wrote:
> On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
> > Whatever we do, it needs to be tested by all folks on Cc who already
> > reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
> 
> Ok, Mike is busy so here are some patches for testing:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot

Thanks for taking care of this!

-- 
Sincerely yours,
Mike.

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Mon, Dec 13, 2021 at 10:20:03AM +0200, Mike Rapoport wrote:
> Thanks for taking care of this!

Sure, no probs.

Lemme send them out officially so they're on the list. Will queue them
this week.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

[PATCH 0/3] x86: Fix boot ordering issues yet again

[Borislav Petkov]

From: Borislav Petkov <bp@suse.de>

Hi,

here are the three I'm going to queue this week. Thanks again to
everyone for the quick testing.

Borislav Petkov (2):
  Revert "x86/boot: Mark prepare_command_line() __init"
  Revert "x86/boot: Pull up cmdline preparation and early param parsing"

Mike Rapoport (1):
  x86/boot: Move EFI range reservation after cmdline parsing

 arch/x86/kernel/setup.c | 72 +++++++++++++++++------------------------
 1 file changed, 30 insertions(+), 42 deletions(-)

-- 
2.29.2

[PATCH 1/3] Revert "x86/boot: Mark prepare_command_line() __init"

[Borislav Petkov]

From: Borislav Petkov <bp@suse.de>

This reverts commit c0f2077baa4113f38f008b8e912b9fb3ff8d43df.

Signed-off-by: Borislav Petkov <bp@suse.de>
---
 arch/x86/kernel/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 6a190c7f4d71..c410be738ae7 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -742,7 +742,7 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char * __init prepare_command_line(void)
+static char *prepare_command_line(void)
 {
 #ifdef CONFIG_CMDLINE_BOOL
 #ifdef CONFIG_CMDLINE_OVERRIDE
-- 
2.29.2

[tip: x86/urgent] Revert "x86/boot: Mark prepare_command_line() __init"

[tip-bot2 for Borislav Petkov]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     58e138d62476fc5f889252dcf73848beeaa54789
Gitweb:        https://git.kernel.org/tip/58e138d62476fc5f889252dcf73848beeaa54789
Author:        Borislav Petkov <bp@suse.de>
AuthorDate:    Mon, 13 Dec 2021 12:27:55 +01:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Wed, 15 Dec 2021 11:14:28 +01:00

Revert "x86/boot: Mark prepare_command_line() __init"

This reverts commit c0f2077baa4113f38f008b8e912b9fb3ff8d43df.

Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/20211213112757.2612-2-bp@alien8.de
---
 arch/x86/kernel/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 6a190c7..c410be7 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -742,7 +742,7 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char * __init prepare_command_line(void)
+static char *prepare_command_line(void)
 {
 #ifdef CONFIG_CMDLINE_BOOL
 #ifdef CONFIG_CMDLINE_OVERRIDE

[PATCH 2/3] Revert "x86/boot: Pull up cmdline preparation and early param parsing"

[Borislav Petkov]

From: Borislav Petkov <bp@suse.de>

This reverts commit 8d48bf8206f77aa8687f0e241e901e5197e52423.

It turned out to be a bad idea as it broke supplying mem= cmdline
parameters due to parse_memopt() requiring preparatory work like setting
up the e820 table in e820__memory_setup() in order to be able to exclude
the range specified by mem=.

Pulling that up would've broken Xen PV again, see threads at

  https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com

due to xen_memory_setup() needing the first reservations in
early_reserve_memory() - kernel and initrd - to have happened already.

This could be fixed again by having Xen do those reservations itself...

Long story short, revert this and do a simpler fix in a later patch.

Signed-off-by: Borislav Petkov <bp@suse.de>
---
 arch/x86/kernel/setup.c | 66 +++++++++++++++++------------------------
 1 file changed, 27 insertions(+), 39 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index c410be738ae7..49b596db5631 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -742,28 +742,6 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char *prepare_command_line(void)
-{
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-
-	parse_early_param();
-
-	return command_line;
-}
-
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures
@@ -852,23 +830,6 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
-	/*
-	 * x86_configure_nx() is called before parse_early_param() (called by
-	 * prepare_command_line()) to detect whether hardware doesn't support
-	 * NX (so that the early EHCI debug console setup can safely call
-	 * set_fixmap()). It may then be called again from within noexec_setup()
-	 * during parsing early parameters to honor the respective command line
-	 * option.
-	 */
-	x86_configure_nx();
-
-	/*
-	 * This parses early params and it needs to run before
-	 * early_reserve_memory() because latter relies on such settings
-	 * supplied as early params.
-	 */
-	*cmdline_p = prepare_command_line();
-
 	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.
@@ -902,6 +863,33 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+	*cmdline_p = command_line;
+
+	/*
+	 * x86_configure_nx() is called before parse_early_param() to detect
+	 * whether hardware doesn't support NX (so that the early EHCI debug
+	 * console setup can safely call set_fixmap()). It may then be called
+	 * again from within noexec_setup() during parsing early parameters
+	 * to honor the respective command line option.
+	 */
+	x86_configure_nx();
+
+	parse_early_param();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
-- 
2.29.2

[tip: x86/urgent] Revert "x86/boot: Pull up cmdline preparation and early param parsing"

[tip-bot2 for Borislav Petkov]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     fbe6183998546f8896ee0b620ece86deff5a2fd1
Gitweb:        https://git.kernel.org/tip/fbe6183998546f8896ee0b620ece86deff5a2fd1
Author:        Borislav Petkov <bp@suse.de>
AuthorDate:    Mon, 13 Dec 2021 12:27:56 +01:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Wed, 15 Dec 2021 11:38:57 +01:00

Revert "x86/boot: Pull up cmdline preparation and early param parsing"

This reverts commit 8d48bf8206f77aa8687f0e241e901e5197e52423.

It turned out to be a bad idea as it broke supplying mem= cmdline
parameters due to parse_memopt() requiring preparatory work like setting
up the e820 table in e820__memory_setup() in order to be able to exclude
the range specified by mem=.

Pulling that up would've broken Xen PV again, see threads at

  https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com

due to xen_memory_setup() needing the first reservations in
early_reserve_memory() - kernel and initrd - to have happened already.

This could be fixed again by having Xen do those reservations itself...

Long story short, revert this and do a simpler fix in a later patch.

Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/20211213112757.2612-3-bp@alien8.de
---
 arch/x86/kernel/setup.c | 66 ++++++++++++++++------------------------
 1 file changed, 27 insertions(+), 39 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index c410be7..49b596d 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -742,28 +742,6 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char *prepare_command_line(void)
-{
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-
-	parse_early_param();
-
-	return command_line;
-}
-
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures
@@ -853,23 +831,6 @@ void __init setup_arch(char **cmdline_p)
 	x86_init.oem.arch_setup();
 
 	/*
-	 * x86_configure_nx() is called before parse_early_param() (called by
-	 * prepare_command_line()) to detect whether hardware doesn't support
-	 * NX (so that the early EHCI debug console setup can safely call
-	 * set_fixmap()). It may then be called again from within noexec_setup()
-	 * during parsing early parameters to honor the respective command line
-	 * option.
-	 */
-	x86_configure_nx();
-
-	/*
-	 * This parses early params and it needs to run before
-	 * early_reserve_memory() because latter relies on such settings
-	 * supplied as early params.
-	 */
-	*cmdline_p = prepare_command_line();
-
-	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.
 	 *
@@ -902,6 +863,33 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+	*cmdline_p = command_line;
+
+	/*
+	 * x86_configure_nx() is called before parse_early_param() to detect
+	 * whether hardware doesn't support NX (so that the early EHCI debug
+	 * console setup can safely call set_fixmap()). It may then be called
+	 * again from within noexec_setup() during parsing early parameters
+	 * to honor the respective command line option.
+	 */
+	x86_configure_nx();
+
+	parse_early_param();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

[PATCH 3/3] x86/boot: Move EFI range reservation after cmdline parsing

[Borislav Petkov]

From: Mike Rapoport <rppt@kernel.org>

The memory reservation in arch/x86/platform/efi/efi.c depends on at
least two command line parameters. Put it back later in the boot process
and move efi_memblock_x86_reserve_range() out of early_memory_reserve().

An attempt to fix this was done in

  8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing")

but that caused other troubles so it got reverted.

The bug this is addressing is:

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve
kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range
    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed
"nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot
cmdline not being taken into account.

 [ bp: Productize into a proper patch. ]

Signed-off-by: Mike Rapoport <rppt@kernel.org>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/e8dd8993c38702ee6dd73b3c11f158617e665607.camel@intel.com
---
 arch/x86/kernel/setup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 49b596db5631..e04f5e6eb33f 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();
@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
-- 
2.29.2

[tip: x86/urgent] x86/boot: Move EFI range reservation after cmdline parsing

[tip-bot2 for Mike Rapoport]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     2f5b3514c33fecad4003ce0f22ca9691492d310b
Gitweb:        https://git.kernel.org/tip/2f5b3514c33fecad4003ce0f22ca9691492d310b
Author:        Mike Rapoport <rppt@kernel.org>
AuthorDate:    Mon, 13 Dec 2021 12:27:57 +01:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Wed, 15 Dec 2021 14:07:54 +01:00

x86/boot: Move EFI range reservation after cmdline parsing

The memory reservation in arch/x86/platform/efi/efi.c depends on at
least two command line parameters. Put it back later in the boot process
and move efi_memblock_x86_reserve_range() out of early_memory_reserve().

An attempt to fix this was done in

  8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing")

but that caused other troubles so it got reverted.

The bug this is addressing is:

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve
kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

  early_reserve_memory
  |-> efi_memblock_x86_reserve_range
      |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed
"nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot
cmdline not being taken into account.

See also https://lore.kernel.org/r/e8dd8993c38702ee6dd73b3c11f158617e665607.camel@intel.com

  [ bp: Turn into a proper patch. ]

Signed-off-by: Mike Rapoport <rppt@kernel.org>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/20211213112757.2612-4-bp@alien8.de
---
 arch/x86/kernel/setup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 49b596d..e04f5e6 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();
@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Patrick J. Volkerding]

Trying again since gmail didn't use plain text and the message got rejected.

We're waiting for these patches to appear in a 5.15 kernel so that we
can ship with an unpatched kernel. Will they be queued for the stable
kernels sometime soon?

Thanks,

Pat


On Mon, Dec 20, 2021 at 12:43 PM Patrick J. Volkerding
<volkerdi@gmail.com> wrote:
>
> Will these patches be queued for the stable kernels soon?
>
> Thanks,
>
> Pat
>
> On Mon, Dec 13, 2021, 03:33 Borislav Petkov <bp@alien8.de> wrote:
>>
>> On Mon, Dec 13, 2021 at 10:20:03AM +0200, Mike Rapoport wrote:
>> > Thanks for taking care of this!
>>
>> Sure, no probs.
>>
>> Lemme send them out officially so they're on the list. Will queue them
>> this week.
>>
>> Thx.
>>
>> --
>> Regards/Gruss,
>>     Boris.
>>
>> https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Mon, Dec 20, 2021 at 12:49:52PM -0600, Patrick J. Volkerding wrote:
> Trying again since gmail didn't use plain text and the message got rejected.
> 
> We're waiting for these patches to appear in a 5.15 kernel so that we
> can ship with an unpatched kernel. Will they be queued for the stable
> kernels sometime soon?

Yes, they're currently queued here:

https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/urgent

and will go to Linus on the weekend.

Which reminds me - they need stable tags. Lemme add those.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Mon, Dec 20, 2021 at 07:59:49PM +0100, Borislav Petkov wrote:
> Which reminds me - they need stable tags. Lemme add those.

Actually, it'll be a lot easier if I send backporting instructions to
the stable@ folks next week. Yap, I'll do that.

/me adds a TODO.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[John Dorminy]

On Thu, Dec 9, 2021 at 10:19 AM Borislav Petkov <bp@alien8.de> wrote:
>
> + Hugh and Patrick.
>
> On Thu, Dec 09, 2021 at 09:38:10AM -0500, John Dorminy wrote:
> > Greetings;
> >
> > It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
> >
> > As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
> >
> > I apologize that I don't know how to fix this, but I'm happy to test patches.
>
> Yeah, people have been reporting boot failures with mem= on the cmdline.
>
> I think I see why, can you try this one:
>
> ---
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 6a190c7f4d71..6db971e61e4b 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p)
>          */
>         x86_configure_nx();
>
> +       e820__memory_setup();
> +
>         /*
>          * This parses early params and it needs to run before
>          * early_reserve_memory() because latter relies on such settings
> @@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p)
>         early_reserve_memory();
>
>         iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
> -       e820__memory_setup();
>         parse_setup_data();
>
>         copy_edd();
> ---
>
Confirmed that that patch makes mem= work again:

[    0.000000] Command line:
BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e
750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
...
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
...
[    0.025520] Memory: 1762976K/2061136K available (16394K kernel
code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 297900K
reserved, 0K cma-reserved)

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

[Borislav Petkov]

On Thu, Dec 09, 2021 at 10:49:06AM -0500, John Dorminy wrote:
> Confirmed that that patch makes mem= work again:

Thanks!

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

[tip: x86/urgent] x86/boot: Move EFI range reservation after cmdline parsing

[tip-bot2 for Mike Rapoport]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     83757bbb9fe029b704fb28e80c3f2b92f23a1994
Gitweb:        https://git.kernel.org/tip/83757bbb9fe029b704fb28e80c3f2b92f23a1994
Author:        Mike Rapoport <rppt@kernel.org>
AuthorDate:    Mon, 13 Dec 2021 12:27:57 +01:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Wed, 15 Dec 2021 12:36:47 +01:00

x86/boot: Move EFI range reservation after cmdline parsing

The memory reservation in arch/x86/platform/efi/efi.c depends on at
least two command line parameters. Put it back later in the boot process
and move efi_memblock_x86_reserve_range() out of early_memory_reserve().

An attempt to fix this was done in

  8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing")

but that caused other troubles so it got reverted.

The bug this is addressing is:

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve
kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range
    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed
"nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot
cmdline not being taken into account.

 [ bp: Produce into a proper patch. ]

Signed-off-by: Borislav Petkov <bp@suse.de>
Signed-off-by: Mike Rapoport <rppt@kernel.org>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/e8dd8993c38702ee6dd73b3c11f158617e665607.camel@intel.com
Link: https://lore.kernel.org/r/20211213112757.2612-4-bp@alien8.de
---
 arch/x86/kernel/setup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 49b596d..e04f5e6 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();
@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

[PATCH] x86/setup: call early_reserve_memory() earlier

[Juergen Gross]

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
reservations") introduced early_reserve_memory() to do all needed
initial memblock_reserve() calls in one function. Unfortunately the
call of early_reserve_memory() is done too late for Xen dom0, as in
some cases a Xen hook called by e820__memory_setup() will need those
memory reservations to have happened already.

Move the call of early_reserve_memory() to the beginning of
setup_arch() in order to avoid such problems.

Cc: stable@vger.kernel.org
Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/kernel/setup.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 79f164141116..f369c51ec580 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 
 void __init setup_arch(char **cmdline_p)
 {
+	/*
+	 * Do some memory reservations *before* memory is added to
+	 * memblock, so memblock allocations won't overwrite it.
+	 * Do it after early param, so we could get (unlikely) panic from
+	 * serial.
+	 *
+	 * After this point everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not
+	 * RAM in e820. All other memory is free game.
+	 */
+	early_reserve_memory();
+
 #ifdef CONFIG_X86_32
 	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
 
@@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
-- 
2.26.2

[tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[tip-bot2 for Juergen Gross]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c1046581f1a3809e075669a3df0191869d96dd1
Gitweb:        https://git.kernel.org/tip/1c1046581f1a3809e075669a3df0191869d96dd1
Author:        Juergen Gross <jgross@suse.com>
AuthorDate:    Tue, 14 Sep 2021 11:41:08 +02:00
Committer:     Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 16 Sep 2021 12:38:05 +02:00

x86/setup: Call early_reserve_memory() earlier

Commit in Fixes introduced early_reserve_memory() to do all needed
initial memblock_reserve() calls in one function. Unfortunately, the call
of early_reserve_memory() is done too late for Xen dom0, as in some
cases a Xen hook called by e820__memory_setup() will need those memory
reservations to have happened already.

Move the call of early_reserve_memory() to the beginning of setup_arch()
in order to avoid such problems.

Fixes: a799c2bd29d1 ("x86/setup: Consolidate early memory reservations")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20210914094108.22482-1-jgross@suse.com
---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index bff3a78..9095158 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -766,6 +766,20 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 
 void __init setup_arch(char **cmdline_p)
 {
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 #ifdef CONFIG_X86_32
 	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
 
@@ -885,18 +899,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Mike Galbraith]

On Thu, 2021-09-16 at 10:50 +0000, tip-bot2 for Juergen Gross wrote:
> The following commit has been merged into the x86/urgent branch of
> tip:
>
> Commit-ID:     1c1046581f1a3809e075669a3df0191869d96dd1
> Gitweb:       
> https://git.kernel.org/tip/1c1046581f1a3809e075669a3df0191869d96dd1
> Author:        Juergen Gross <jgross@suse.com>
> AuthorDate:    Tue, 14 Sep 2021 11:41:08 +02:00
> Committer:     Borislav Petkov <bp@suse.de>
> CommitterDate: Thu, 16 Sep 2021 12:38:05 +02:00
>
> x86/setup: Call early_reserve_memory() earlier

This commit rendered tip toxic to my i4790 desktop box and i5-6200U
lappy.  Boot for both is instantly over without so much as a twitch.

Post bisect revert made both all better.


	-Mike

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Mike Rapoport]

On Sun, Sep 19, 2021 at 06:55:16PM +0200, Mike Galbraith wrote:
> On Thu, 2021-09-16 at 10:50 +0000, tip-bot2 for Juergen Gross wrote:
> > The following commit has been merged into the x86/urgent branch of
> > tip:
> >
> > Commit-ID:     1c1046581f1a3809e075669a3df0191869d96dd1
> > Gitweb:       
> > https://git.kernel.org/tip/1c1046581f1a3809e075669a3df0191869d96dd1
> > Author:        Juergen Gross <jgross@suse.com>
> > AuthorDate:    Tue, 14 Sep 2021 11:41:08 +02:00
> > Committer:     Borislav Petkov <bp@suse.de>
> > CommitterDate: Thu, 16 Sep 2021 12:38:05 +02:00
> >
> > x86/setup: Call early_reserve_memory() earlier
> 
> This commit rendered tip toxic to my i4790 desktop box and i5-6200U
> lappy.  Boot for both is instantly over without so much as a twitch.
> 
> Post bisect revert made both all better.

Can you please send the boot log of a working kernel up to 

"Memory: %luK/%luK available"

line for both of them?
 
> 
> 	-Mike

-- 
Sincerely yours,
Mike.

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Mike Galbraith]

On Sun, 2021-09-19 at 20:04 +0300, Mike Rapoport wrote:
>
> Can you please send the boot log of a working kernel up to
>
> "Memory: %luK/%luK available"
>
> line for both of them?

Desktop box:
[    0.000000] microcode: microcode updated early to revision 0x28, date = 2019-11-12
[    0.000000] Linux version 5.15.0.g02770d1-tip (root@homer) (gcc (SUSE Linux) 7.5.0, GNU ld (GNU Binutils; SUSE Linux Enterprise 15) 2.35.1.20201123-7.18) #46 SMP Sun Sep 19 18:42:41 CEST 2021
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip root=UUID=891c2a1f-cc1a-464b-a529-ab6add65aa21 scsi_mod.use_blk_mq=1 ftrace_dump_on_oops skew_tick=1 nortsched nodelayacct audit=0 cgroup_disable=memory cgroup_hide=all mitigations=off noresume panic=60 ignore_loglevel showopts crashkernel=204M
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.000000] signal: max sigframe size: 1776
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x0000000000057fff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000058000-0x0000000000058fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000059000-0x000000000009dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000cf9b7fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000cf9b8000-0x00000000cf9befff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x00000000cf9bf000-0x00000000cfdfdfff] usable
[    0.000000] BIOS-e820: [mem 0x00000000cfdfe000-0x00000000d00befff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000d00bf000-0x00000000de787fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000de788000-0x00000000de811fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000de812000-0x00000000de828fff] ACPI data
[    0.000000] BIOS-e820: [mem 0x00000000de829000-0x00000000de98dfff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x00000000de98e000-0x00000000deffefff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000defff000-0x00000000deffffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000041effffff] usable
[    0.000000] printk: debug: ignoring loglevel setting.
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] e820: update [mem 0xcf673018-0xcf694a57] usable ==> usable
[    0.000000] e820: update [mem 0xcf673018-0xcf694a57] usable ==> usable
[    0.000000] extended physical RAM map:
[    0.000000] reserve setup_data: [mem 0x0000000000000000-0x0000000000057fff] usable
[    0.000000] reserve setup_data: [mem 0x0000000000058000-0x0000000000058fff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000059000-0x000000000009dfff] usable
[    0.000000] reserve setup_data: [mem 0x000000000009e000-0x000000000009ffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000100000-0x00000000cf673017] usable
[    0.000000] reserve setup_data: [mem 0x00000000cf673018-0x00000000cf694a57] usable
[    0.000000] reserve setup_data: [mem 0x00000000cf694a58-0x00000000cf9b7fff] usable
[    0.000000] reserve setup_data: [mem 0x00000000cf9b8000-0x00000000cf9befff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x00000000cf9bf000-0x00000000cfdfdfff] usable
[    0.000000] reserve setup_data: [mem 0x00000000cfdfe000-0x00000000d00befff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000d00bf000-0x00000000de787fff] usable
[    0.000000] reserve setup_data: [mem 0x00000000de788000-0x00000000de811fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000de812000-0x00000000de828fff] ACPI data
[    0.000000] reserve setup_data: [mem 0x00000000de829000-0x00000000de98dfff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x00000000de98e000-0x00000000deffefff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000defff000-0x00000000deffffff] usable
[    0.000000] reserve setup_data: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000100000000-0x000000041effffff] usable
[    0.000000] efi: EFI v2.31 by American Megatrends
[    0.000000] efi: ESRT=0xdef87998 ACPI=0xde816000 ACPI 2.0=0xde816000 SMBIOS=0xdef87598
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 3591.714 MHz processor
[    0.000092] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000094] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000098] last_pfn = 0x41f000 max_arch_pfn = 0x400000000
[    0.000169] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT
[    0.000869] e820: update [mem 0xe0000000-0xffffffff] usable ==> reserved
[    0.000874] last_pfn = 0xdf000 max_arch_pfn = 0x400000000
[    0.005979] esrt: Reserving ESRT space from 0x00000000def87998 to 0x00000000def879d0.
[    0.005985] Kernel/User page tables isolation: disabled on command line.
[    0.005987] Using GB pages for direct mapping
[    0.006571] Secure boot disabled
[    0.006571] RAMDISK: [mem 0x3f214000-0x3fffafff]
[    0.006575] ACPI: Early table checksum verification disabled
[    0.006577] ACPI: RSDP 0x00000000DE816000 000024 (v02 MEDION)
[    0.006580] ACPI: XSDT 0x00000000DE816088 00008C (v01 MEDION MEDIONAG 11112011 AMI  00010013)
[    0.006583] ACPI: FACP 0x00000000DE822CC8 00010C (v05 MEDION MEDIONAG 11112011 AMI  00010013)
[    0.006587] ACPI: DSDT 0x00000000DE8161A0 00CB24 (v02 MEDION MEDIONAG 00000028 INTL 20120711)
[    0.006589] ACPI: FACS 0x00000000DE98C080 000040
[    0.006591] ACPI: APIC 0x00000000DE822DD8 000092 (v03 MEDION MEDIONAG 11112011 AMI  00010013)
[    0.006593] ACPI: FPDT 0x00000000DE822E70 000044 (v01 MEDION MEDIONAG 11112011 AMI  00010013)
[    0.006595] ACPI: MSDM 0x00000000DE822EB8 000055 (v03 MEDION MEDIONAG 11112011 AMI  00010013)
[    0.006597] ACPI: LPIT 0x00000000DE822F10 00005C (v01 MEDION MEDIONAG 00000000 AMI. 00000005)
[    0.006599] ACPI: SSDT 0x00000000DE822F70 000539 (v01 PmRef  Cpu0Ist  00003000 INTL 20120711)
[    0.006602] ACPI: SSDT 0x00000000DE8234B0 000AD8 (v01 PmRef  CpuPm    00003000 INTL 20120711)
[    0.006604] ACPI: MCFG 0x00000000DE823F88 00003C (v01 MEDION MEDIONAG 11112011 MSFT 00000097)
[    0.006606] ACPI: HPET 0x00000000DE823FC8 000038 (v01 MEDION MEDIONAG 11112011 AMI. 00000005)
[    0.006608] ACPI: SSDT 0x00000000DE824000 000443 (v01 SataRe SataTabl 00001000 INTL 20120711)
[    0.006610] ACPI: SSDT 0x00000000DE824448 0033CE (v01 SaSsdt SaSsdt   00003000 INTL 20091112)
[    0.006612] ACPI: DMAR 0x00000000DE827818 000080 (v01 INTEL  HSW      00000001 INTL 00000001)
[    0.006614] ACPI: SSDT 0x00000000DE827898 000A26 (v01 Intel_ IsctTabl 00001000 INTL 20120711)
[    0.006616] ACPI: Reserving FACP table memory at [mem 0xde822cc8-0xde822dd3]
[    0.006617] ACPI: Reserving DSDT table memory at [mem 0xde8161a0-0xde822cc3]
[    0.006618] ACPI: Reserving FACS table memory at [mem 0xde98c080-0xde98c0bf]
[    0.006619] ACPI: Reserving APIC table memory at [mem 0xde822dd8-0xde822e69]
[    0.006620] ACPI: Reserving FPDT table memory at [mem 0xde822e70-0xde822eb3]
[    0.006621] ACPI: Reserving MSDM table memory at [mem 0xde822eb8-0xde822f0c]
[    0.006622] ACPI: Reserving LPIT table memory at [mem 0xde822f10-0xde822f6b]
[    0.006622] ACPI: Reserving SSDT table memory at [mem 0xde822f70-0xde8234a8]
[    0.006623] ACPI: Reserving SSDT table memory at [mem 0xde8234b0-0xde823f87]
[    0.006624] ACPI: Reserving MCFG table memory at [mem 0xde823f88-0xde823fc3]
[    0.006625] ACPI: Reserving HPET table memory at [mem 0xde823fc8-0xde823fff]
[    0.006626] ACPI: Reserving SSDT table memory at [mem 0xde824000-0xde824442]
[    0.006627] ACPI: Reserving SSDT table memory at [mem 0xde824448-0xde827815]
[    0.006627] ACPI: Reserving DMAR table memory at [mem 0xde827818-0xde827897]
[    0.006628] ACPI: Reserving SSDT table memory at [mem 0xde827898-0xde8282bd]
[    0.006689] No NUMA configuration found
[    0.006690] Faking a node at [mem 0x0000000000000000-0x000000041effffff]
[    0.006692] NODE_DATA(0) allocated [mem 0x41eff8000-0x41effbfff]
[    0.006697] Reserving 204MB of memory at 3104MB for crashkernel (System RAM: 16340MB)
[    0.006711] Zone ranges:
[    0.006712]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.006713]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.006714]   Normal   [mem 0x0000000100000000-0x000000041effffff]
[    0.006716] Movable zone start for each node
[    0.006716] Early memory node ranges
[    0.006717]   node   0: [mem 0x0000000000001000-0x0000000000057fff]
[    0.006718]   node   0: [mem 0x0000000000059000-0x000000000009dfff]
[    0.006719]   node   0: [mem 0x0000000000100000-0x00000000cf9b7fff]
[    0.006720]   node   0: [mem 0x00000000cf9bf000-0x00000000cfdfdfff]
[    0.006721]   node   0: [mem 0x00000000d00bf000-0x00000000de787fff]
[    0.006721]   node   0: [mem 0x00000000defff000-0x00000000deffffff]
[    0.006722]   node   0: [mem 0x0000000100000000-0x000000041effffff]
[    0.006724] Initmem setup node 0 [mem 0x0000000000001000-0x000000041effffff]
[    0.006727] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.006728] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.006757] On node 0, zone DMA: 98 pages in unavailable ranges
[    0.012880] On node 0, zone DMA32: 7 pages in unavailable ranges
[    0.013356] On node 0, zone DMA32: 705 pages in unavailable ranges
[    0.013376] On node 0, zone DMA32: 2167 pages in unavailable ranges
[    0.013672] On node 0, zone Normal: 4096 pages in unavailable ranges
[    0.013704] On node 0, zone Normal: 4096 pages in unavailable ranges
[    0.013777] ACPI: PM-Timer IO Port: 0x1808
[    0.013783] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[    0.013792] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
[    0.013794] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.013795] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.013798] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.013799] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.013802] TSC deadline timer available
[    0.013803] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
[    0.013818] [mem 0xdf000000-0xf7ffffff] available for PCI devices
[    0.013819] Booting paravirtualized kernel on bare hardware
[    0.013821] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.016320] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.016472] percpu: Embedded 52 pages/cpu s173592 r8192 d31208 u262144
[    0.016476] pcpu-alloc: s173592 r8192 d31208 u262144 alloc=1*2097152
[    0.016478] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
[    0.016496] Built 1 zonelists, mobility grouping on.  Total pages: 4117613
[    0.016497] Policy zone: Normal
[    0.016499] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip root=UUID=891c2a1f-cc1a-464b-a529-ab6add65aa21 scsi_mod.use_blk_mq=1 ftrace_dump_on_oops skew_tick=1 nortsched nodelayacct audit=0 cgroup_disable=memory cgroup_hide=all mitigations=off noresume panic=60 ignore_loglevel showopts crashkernel=204M
[    0.016631] Unknown command line parameters: nodelayacct noresume showopts BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip audit=0 crashkernel=204M
[    0.016632] printk: log_buf_len individual max cpu contribution: 32768 bytes
[    0.016633] printk: log_buf_len total cpu_extra contributions: 229376 bytes
[    0.016634] printk: log_buf_len min size: 262144 bytes
[    0.016824] printk: log_buf_len: 524288 bytes
[    0.016825] printk: early log buf free: 250392(95%)
[    0.017475] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.017810] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.017854] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.037819] Memory: 3431140K/16732532K available (10248K kernel code, 2187K rwdata, 3396K rodata, 1628K init, 1084K bss, 632956K reserved, 0K cma-reserved)

Lappy:
[    0.000000] microcode: microcode updated early to revision 0xea, date = 2021-01-25
[    0.000000] Linux version 5.15.0.g02770d1-tip (root@homer) (gcc (SUSE Linux) 7.5.0, GNU ld (GNU Binutils; SUSE Linux Enterprise 15) 2.35.1.20201123-7.18) #46 SMP Sun Sep 19 18:42:41 CEST 2021
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip root=UUID=fa9ef4c7-201b-4d4e-a975-d7042e8c8e92 ntremap=no_x2apic_optout acpi_backlight=vendor "acpi_osi=!Windows 2013" "acpi_osi=!Windows 2012" nortsched ftrace_dump_on_oops audit=0 nodelayacct scsi_mod.use_blk_mq=1 mitigations=off noresume panic=60 ignore_loglevel call_trace=old noresume crashkernel=211M,high crashkernel=72M,low
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
[    0.000000] x86/fpu: Enabled xstate features 0x1f, context size is 960 bytes, using 'compacted' format.
[    0.000000] signal: max sigframe size: 2032
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x0000000000057fff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000058000-0x0000000000058fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000059000-0x000000000009dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000008454cfff] usable
[    0.000000] BIOS-e820: [mem 0x000000008454d000-0x000000008454dfff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000008454e000-0x0000000084577fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000084578000-0x000000008b7aafff] usable
[    0.000000] BIOS-e820: [mem 0x000000008b7ab000-0x000000008c4aefff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008c4af000-0x000000008c4fbfff] ACPI data
[    0.000000] BIOS-e820: [mem 0x000000008c4fc000-0x000000008ce3dfff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000008ce3e000-0x000000008d2fefff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008d2ff000-0x000000008d2fffff] usable
[    0.000000] BIOS-e820: [mem 0x000000008d300000-0x000000008d3fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fe000000-0x00000000fe010fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000026effffff] usable
[    0.000000] printk: debug: ignoring loglevel setting.
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] e820: update [mem 0x831d6018-0x831e6057] usable ==> usable
[    0.000000] e820: update [mem 0x831d6018-0x831e6057] usable ==> usable
[    0.000000] extended physical RAM map:
[    0.000000] reserve setup_data: [mem 0x0000000000000000-0x0000000000057fff] usable
[    0.000000] reserve setup_data: [mem 0x0000000000058000-0x0000000000058fff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000059000-0x000000000009dfff] usable
[    0.000000] reserve setup_data: [mem 0x000000000009e000-0x000000000009ffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000100000-0x00000000831d6017] usable
[    0.000000] reserve setup_data: [mem 0x00000000831d6018-0x00000000831e6057] usable
[    0.000000] reserve setup_data: [mem 0x00000000831e6058-0x000000008454cfff] usable
[    0.000000] reserve setup_data: [mem 0x000000008454d000-0x000000008454dfff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x000000008454e000-0x0000000084577fff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000084578000-0x000000008b7aafff] usable
[    0.000000] reserve setup_data: [mem 0x000000008b7ab000-0x000000008c4aefff] reserved
[    0.000000] reserve setup_data: [mem 0x000000008c4af000-0x000000008c4fbfff] ACPI data
[    0.000000] reserve setup_data: [mem 0x000000008c4fc000-0x000000008ce3dfff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x000000008ce3e000-0x000000008d2fefff] reserved
[    0.000000] reserve setup_data: [mem 0x000000008d2ff000-0x000000008d2fffff] usable
[    0.000000] reserve setup_data: [mem 0x000000008d300000-0x000000008d3fffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fe000000-0x00000000fe010fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000100000000-0x000000026effffff] usable
[    0.000000] efi: EFI v2.40 by American Megatrends
[    0.000000] efi: ESRT=0x8d265118 ACPI=0x8c4c6000 ACPI 2.0=0x8c4c6000 SMBIOS=0x8a64d010 TPMEventLog=0x845e9018
[    0.000000] TPM Final Events table not present
[    0.000000] SMBIOS 2.8 present.
[    0.000000] DMI: HP HP Spectre x360 Convertible/804F, BIOS F.47 11/22/2017
[    0.000000] tsc: Detected 2400.000 MHz processor
[    0.000725] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000728] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000734] last_pfn = 0x26f000 max_arch_pfn = 0x400000000
[    0.000836] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT
[    0.001875] last_pfn = 0x8d300 max_arch_pfn = 0x400000000
[    0.009164] esrt: Reserving ESRT space from 0x000000008d265118 to 0x000000008d265150.
[    0.009173] Kernel/User page tables isolation: disabled on command line.
[    0.009174] Using GB pages for direct mapping
[    0.009885] Secure boot disabled
[    0.009886] RAMDISK: [mem 0x3f1c1000-0x3fffafff]
[    0.009890] ACPI: Early table checksum verification disabled
[    0.009893] ACPI: RSDP 0x000000008C4C6000 000024 (v02 HPQOEM)
[    0.009897] ACPI: XSDT 0x000000008C4C60A8 0000CC (v01 HPQOEM SLIC-MPC 01072009 HP   00010013)
[    0.009902] ACPI: FACP 0x000000008C4F2DF0 00010C (v05 HPQOEM SLIC-MPC 01072009 HP   00010013)
[    0.009907] ACPI: DSDT 0x000000008C4C6200 02CBE9 (v02 HPQOEM 804F     01072009 ACPI 20120913)
[    0.009910] ACPI: FACS 0x000000008CE3BF80 000040
[    0.009913] ACPI: APIC 0x000000008C4F2F00 000084 (v03 HPQOEM 804F     01072009 HP   00010013)
[    0.009915] ACPI: FPDT 0x000000008C4F2F88 000044 (v01 HPQOEM 804F     01072009 HP   00010013)
[    0.009918] ACPI: FIDT 0x000000008C4F2FD0 00009C (v01 HPQOEM 804F     01072009 HP   00010013)
[    0.009921] ACPI: MCFG 0x000000008C4F3070 00003C (v01 HPQOEM 804F     01072009 HP   00000097)
[    0.009924] ACPI: HPET 0x000000008C4F30B0 000038 (v01 HPQOEM 804F     01072009 HP   0005000B)
[    0.009926] ACPI: LPIT 0x000000008C4F30E8 000094 (v01 HPQOEM 804F     00000000 HP   0000005F)
[    0.009929] ACPI: SSDT 0x000000008C4F3180 000248 (v02 HPQOEM 804F     00000000 ACPI 20120913)
[    0.009932] ACPI: SSDT 0x000000008C4F33C8 0011A8 (v02 HPQOEM 804F     00001000 ACPI 20120913)
[    0.009935] ACPI: SSDT 0x000000008C4F4570 0004A3 (v02 HPQOEM 804F     00001000 ACPI 20120913)
[    0.009938] ACPI: DBGP 0x000000008C4F4A18 000034 (v01 HPQOEM 804F     00000000 HP   0000005F)
[    0.009941] ACPI: DBG2 0x000000008C4F4A50 000054 (v00 HPQOEM 804F     00000000 HP   0000005F)
[    0.009943] ACPI: SSDT 0x000000008C4F4AA8 000024 (v02 HPQOEM 804F     00000000 ACPI 20120913)
[    0.009946] ACPI: MSDM 0x000000008C4F4AD0 000055 (v03 HPQOEM SLIC-MPC 00000001 HP   00010013)
[    0.009949] ACPI: SSDT 0x000000008C4F4B28 00546C (v02 HPQOEM 804F     00003000 ACPI 20120913)
[    0.009951] ACPI: UEFI 0x000000008C4F9F98 000042 (v01 HPQOEM 804F     00000000 HP   00000000)
[    0.009955] ACPI: SSDT 0x000000008C4F9FE0 000E73 (v02 HPQOEM 804F     00003000 ACPI 20120913)
[    0.009958] ACPI: DMAR 0x000000008C4FAE58 0000A8 (v01 HPQOEM 804F     00000001 HP   00000001)
[    0.009960] ACPI: TPM2 0x000000008C4FAF00 000034 (v03 HPQOEM 804F     00000001 HP   00000000)
[    0.009963] ACPI: ASF! 0x000000008C4FAF38 0000A5 (v32 HPQOEM 804F     00000001 HP   000F4240)
[    0.009966] ACPI: BGRT 0x000000008C4FAFE0 000038 (v01 HPQOEM 804F     01072009 HP   00010013)
[    0.009969] ACPI: Reserving FACP table memory at [mem 0x8c4f2df0-0x8c4f2efb]
[    0.009971] ACPI: Reserving DSDT table memory at [mem 0x8c4c6200-0x8c4f2de8]
[    0.009972] ACPI: Reserving FACS table memory at [mem 0x8ce3bf80-0x8ce3bfbf]
[    0.009973] ACPI: Reserving APIC table memory at [mem 0x8c4f2f00-0x8c4f2f83]
[    0.009974] ACPI: Reserving FPDT table memory at [mem 0x8c4f2f88-0x8c4f2fcb]
[    0.009975] ACPI: Reserving FIDT table memory at [mem 0x8c4f2fd0-0x8c4f306b]
[    0.009976] ACPI: Reserving MCFG table memory at [mem 0x8c4f3070-0x8c4f30ab]
[    0.009978] ACPI: Reserving HPET table memory at [mem 0x8c4f30b0-0x8c4f30e7]
[    0.009979] ACPI: Reserving LPIT table memory at [mem 0x8c4f30e8-0x8c4f317b]
[    0.009980] ACPI: Reserving SSDT table memory at [mem 0x8c4f3180-0x8c4f33c7]
[    0.009981] ACPI: Reserving SSDT table memory at [mem 0x8c4f33c8-0x8c4f456f]
[    0.009982] ACPI: Reserving SSDT table memory at [mem 0x8c4f4570-0x8c4f4a12]
[    0.009983] ACPI: Reserving DBGP table memory at [mem 0x8c4f4a18-0x8c4f4a4b]
[    0.009984] ACPI: Reserving DBG2 table memory at [mem 0x8c4f4a50-0x8c4f4aa3]
[    0.009985] ACPI: Reserving SSDT table memory at [mem 0x8c4f4aa8-0x8c4f4acb]
[    0.009986] ACPI: Reserving MSDM table memory at [mem 0x8c4f4ad0-0x8c4f4b24]
[    0.009988] ACPI: Reserving SSDT table memory at [mem 0x8c4f4b28-0x8c4f9f93]
[    0.009989] ACPI: Reserving UEFI table memory at [mem 0x8c4f9f98-0x8c4f9fd9]
[    0.009990] ACPI: Reserving SSDT table memory at [mem 0x8c4f9fe0-0x8c4fae52]
[    0.009991] ACPI: Reserving DMAR table memory at [mem 0x8c4fae58-0x8c4faeff]
[    0.009992] ACPI: Reserving TPM2 table memory at [mem 0x8c4faf00-0x8c4faf33]
[    0.009994] ACPI: Reserving ASF! table memory at [mem 0x8c4faf38-0x8c4fafdc]
[    0.009995] ACPI: Reserving BGRT table memory at [mem 0x8c4fafe0-0x8c4fb017]
[    0.010282] No NUMA configuration found
[    0.010283] Faking a node at [mem 0x0000000000000000-0x000000026effffff]
[    0.010287] NODE_DATA(0) allocated [mem 0x26effa000-0x26effdfff]
[    0.010298] Reserving 72MB of low memory at 2016MB for crashkernel (low RAM limit: 4096MB)
[    0.010300] Reserving 211MB of memory at 9744MB for crashkernel (System RAM: 8103MB)
[    0.010313] Zone ranges:
[    0.010314]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.010316]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.010318]   Normal   [mem 0x0000000100000000-0x000000026effffff]
[    0.010320] Movable zone start for each node
[    0.010321] Early memory node ranges
[    0.010322]   node   0: [mem 0x0000000000001000-0x0000000000057fff]
[    0.010323]   node   0: [mem 0x0000000000059000-0x000000000009dfff]
[    0.010325]   node   0: [mem 0x0000000000100000-0x000000008454cfff]
[    0.010326]   node   0: [mem 0x0000000084578000-0x000000008b7aafff]
[    0.010327]   node   0: [mem 0x000000008d2ff000-0x000000008d2fffff]
[    0.010328]   node   0: [mem 0x0000000100000000-0x000000026effffff]
[    0.010330] Initmem setup node 0 [mem 0x0000000000001000-0x000000026effffff]
[    0.010334] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.010336] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.010373] On node 0, zone DMA: 98 pages in unavailable ranges
[    0.016133] On node 0, zone DMA32: 43 pages in unavailable ranges
[    0.016202] On node 0, zone DMA32: 6996 pages in unavailable ranges
[    0.016653] On node 0, zone Normal: 11520 pages in unavailable ranges
[    0.016695] On node 0, zone Normal: 4096 pages in unavailable ranges
[    0.016704] Reserving Intel graphics memory at [mem 0x8e000000-0x8fffffff]
[    0.017024] ACPI: PM-Timer IO Port: 0x1808
[    0.017030] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[    0.017032] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
[    0.017033] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
[    0.017034] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
[    0.017060] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
[    0.017063] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.017065] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.017068] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.017069] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.017075] e820: update [mem 0x89ade000-0x89b00fff] usable ==> reserved
[    0.017084] TSC deadline timer available
[    0.017085] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.017104] [mem 0x90000000-0xdfffffff] available for PCI devices
[    0.017105] Booting paravirtualized kernel on bare hardware
[    0.017108] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.020679] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:4 nr_node_ids:1
[    0.020813] percpu: Embedded 52 pages/cpu s173592 r8192 d31208 u524288
[    0.020819] pcpu-alloc: s173592 r8192 d31208 u524288 alloc=1*2097152
[    0.020822] pcpu-alloc: [0] 0 1 2 3
[    0.020842] Built 1 zonelists, mobility grouping on.  Total pages: 2041826
[    0.020844] Policy zone: Normal
[    0.020846] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip root=UUID=fa9ef4c7-201b-4d4e-a975-d7042e8c8e92 ntremap=no_x2apic_optout acpi_backlight=vendor "acpi_osi=!Windows 2013" "acpi_osi=!Windows 2012" nortsched ftrace_dump_on_oops audit=0 nodelayacct scsi_mod.use_blk_mq=1 mitigations=off noresume panic=60 ignore_loglevel call_trace=old noresume crashkernel=211M,high crashkernel=72M,low
[    0.021078] Unknown command line parameters: nodelayacct noresume noresume BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip ntremap=no_x2apic_optout audit=0 call_trace=old crashkernel=72M,low
[    0.021465] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.021661] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.021701] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.041321] Memory: 2104580K/8297588K available (10248K kernel code, 2187K rwdata, 3396K rodata, 1628K init, 1084K bss, 672016K reserved, 0K cma-reserved)

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Mike Rapoport]

Hi,

On Mon, Sep 20, 2021 at 02:56:15AM +0200, Mike Galbraith wrote:
> On Sun, 2021-09-19 at 20:04 +0300, Mike Rapoport wrote:
> >
> > Can you please send the boot log of a working kernel up to
> >
> > "Memory: %luK/%luK available"
> >
> > line for both of them?
> 
> Desktop box:
> [    0.000000] microcode: microcode updated early to revision 0x28, date = 2019-11-12
> [    0.000000] Linux version 5.15.0.g02770d1-tip (root@homer) (gcc (SUSE Linux) 7.5.0, GNU ld (GNU Binutils; SUSE Linux Enterprise 15) 2.35.1.20201123-7.18) #46 SMP Sun Sep 19 18:42:41 CEST 2021
> [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0.g02770d1-tip root=UUID=891c2a1f-cc1a-464b-a529-ab6add65aa21 scsi_mod.use_blk_mq=1 ftrace_dump_on_oops skew_tick=1 nortsched nodelayacct audit=0 cgroup_disable=memory cgroup_hide=all mitigations=off noresume panic=60 ignore_loglevel showopts crashkernel=204M

Thanks!
Can't say anything caught my eye, except the early microcode update.
Just to rule that out, can you try booting without the early microcode
update?

And, to check Juergen's suggestion about failure in
memblock_x86_reserve_range_setup_data(), can you try this patch on top of
the failing tip:

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 25425edc81a4..78162d9e90cf 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -716,8 +716,6 @@ static void __init early_reserve_memory(void)
 	if (efi_enabled(EFI_BOOT))
 		efi_memblock_x86_reserve_range();
 
-	memblock_x86_reserve_range_setup_data();
-
 	reserve_ibft_region();
 	reserve_bios_regions();
 	trim_snb_memory();
@@ -888,6 +886,8 @@ void __init setup_arch(char **cmdline_p)
 	 */
 	x86_configure_nx();
 
+	memblock_x86_reserve_range_setup_data();
+
 	parse_early_param();
 
 #ifdef CONFIG_MEMORY_HOTPLUG

-- 
Sincerely yours,
Mike.

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Borislav Petkov]

On Mon, Sep 20, 2021 at 12:26:24PM +0300, Mike Rapoport wrote:
> Can't say anything caught my eye, except the early microcode update.
> Just to rule that out, can you try booting without the early microcode
> update?

That should be unrelated but sure, worth a try, you can boot with
"dis_ucode_ldr" on the cmdline.

-- 
Regards/Gruss,
    Boris.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Mike Galbraith]

On Mon, 2021-09-20 at 12:26 +0300, Mike Rapoport wrote:
>
> Can't say anything caught my eye, except the early microcode update.
> Just to rule that out, can you try booting without the early microcode
> update?

Nogo.

> And, to check Juergen's suggestion about failure in
> memblock_x86_reserve_range_setup_data(), can you try this patch on top of
> the failing tip:

Yup, patchlet detoxified it for both boxen.

> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 25425edc81a4..78162d9e90cf 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -716,8 +716,6 @@ static void __init early_reserve_memory(void)
>         if (efi_enabled(EFI_BOOT))
>                 efi_memblock_x86_reserve_range();
>  
> -       memblock_x86_reserve_range_setup_data();
> -
>         reserve_ibft_region();
>         reserve_bios_regions();
>         trim_snb_memory();
> @@ -888,6 +886,8 @@ void __init setup_arch(char **cmdline_p)
>          */
>         x86_configure_nx();
>  
> +       memblock_x86_reserve_range_setup_data();
> +
>         parse_early_param();
>  
>  #ifdef CONFIG_MEMORY_HOTPLUG
>

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 684 bytes --]

On 20.09.21 13:25, Mike Galbraith wrote:
> On Mon, 2021-09-20 at 12:26 +0300, Mike Rapoport wrote:
>>
>> Can't say anything caught my eye, except the early microcode update.
>> Just to rule that out, can you try booting without the early microcode
>> update?
> 
> Nogo.
> 
>> And, to check Juergen's suggestion about failure in
>> memblock_x86_reserve_range_setup_data(), can you try this patch on top of
>> the failing tip:
> 
> Yup, patchlet detoxified it for both boxen.

Yay!

Will respin my patch moving the call of early_reserve_memory() just
before the call of e820__memory_setup().

Thanks for reporting the issue and verifying my suspicion.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Borislav Petkov]

On Sun, Sep 19, 2021 at 06:55:16PM +0200, Mike Galbraith wrote:
> On Thu, 2021-09-16 at 10:50 +0000, tip-bot2 for Juergen Gross wrote:
> > The following commit has been merged into the x86/urgent branch of
> > tip:
> >
> > Commit-ID:     1c1046581f1a3809e075669a3df0191869d96dd1
> > Gitweb:       
> > https://git.kernel.org/tip/1c1046581f1a3809e075669a3df0191869d96dd1
> > Author:        Juergen Gross <jgross@suse.com>
> > AuthorDate:    Tue, 14 Sep 2021 11:41:08 +02:00
> > Committer:     Borislav Petkov <bp@suse.de>
> > CommitterDate: Thu, 16 Sep 2021 12:38:05 +02:00
> >
> > x86/setup: Call early_reserve_memory() earlier
> 
> This commit rendered tip toxic to my i4790 desktop box and i5-6200U
> lappy.  Boot for both is instantly over without so much as a twitch.
> 
> Post bisect revert made both all better.

I had a suspicion that moving stuff around like that would not just
simply work in all cases, as our boot order is very lovely and fragile.

And it booted just fine on my machines here.

;-\

Anyway, commit zapped from the x86/urgent lineup. We'll have to have a
third try later.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 2202 bytes --]

On 19.09.21 19:15, Borislav Petkov wrote:
> On Sun, Sep 19, 2021 at 06:55:16PM +0200, Mike Galbraith wrote:
>> On Thu, 2021-09-16 at 10:50 +0000, tip-bot2 for Juergen Gross wrote:
>>> The following commit has been merged into the x86/urgent branch of
>>> tip:
>>>
>>> Commit-ID:     1c1046581f1a3809e075669a3df0191869d96dd1
>>> Gitweb:
>>> https://git.kernel.org/tip/1c1046581f1a3809e075669a3df0191869d96dd1
>>> Author:        Juergen Gross <jgross@suse.com>
>>> AuthorDate:    Tue, 14 Sep 2021 11:41:08 +02:00
>>> Committer:     Borislav Petkov <bp@suse.de>
>>> CommitterDate: Thu, 16 Sep 2021 12:38:05 +02:00
>>>
>>> x86/setup: Call early_reserve_memory() earlier
>>
>> This commit rendered tip toxic to my i4790 desktop box and i5-6200U
>> lappy.  Boot for both is instantly over without so much as a twitch.
>>
>> Post bisect revert made both all better.
> 
> I had a suspicion that moving stuff around like that would not just
> simply work in all cases, as our boot order is very lovely and fragile.
> 
> And it booted just fine on my machines here.
> 
> ;-\
> 
> Anyway, commit zapped from the x86/urgent lineup. We'll have to have a
> third try later.
> 

How will that try look like? I'm seeing the following alternatives:

1. Just revert a799c2bd29d19c565 ("x86/setup: Consolidate early memory
    reservations").

2. Try to move the call of early_reserve_memory() just before the call
    of e820__memory_setup().

3. Split early_reserve_memory() into two parts, with the first part
    doing the memblock_reserve() calls for the kernel text, initrd and
    page 0 right at the start of setup_arch(), and the second part for
    the rest at the same place it is handled now.

4. Analyze why Mike's systems fail to boot and try to fix his issue(s)
    (probably via one of the above ways).

Looking at the calls done in early_reserve_memory() I have my doubts
that memblock_x86_reserve_range_setup_data() will work before
early_ioremap_init() has been called, as it is using early_memremap().
This would suggest variant 2 could be a working solution.

In case I'm wrong with my doubts I'd prefer variant 3.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Mike Rapoport]

Hi,

On Mon, Sep 20, 2021 at 08:00:31AM +0200, Juergen Gross wrote:
> On 19.09.21 19:15, Borislav Petkov wrote:
> > On Sun, Sep 19, 2021 at 06:55:16PM +0200, Mike Galbraith wrote:
> > > On Thu, 2021-09-16 at 10:50 +0000, tip-bot2 for Juergen Gross wrote:
> > > > The following commit has been merged into the x86/urgent branch of
> > > > tip:
> > > > 
> > > > x86/setup: Call early_reserve_memory() earlier
> > > 
> > > This commit rendered tip toxic to my i4790 desktop box and i5-6200U
> > > lappy.  Boot for both is instantly over without so much as a twitch.
> > > 
> > > Post bisect revert made both all better.
> > 
> > I had a suspicion that moving stuff around like that would not just
> > simply work in all cases, as our boot order is very lovely and fragile.
> > 
> > And it booted just fine on my machines here.
> > 
> > ;-\
> > 
> > Anyway, commit zapped from the x86/urgent lineup. We'll have to have a
> > third try later.
> > 
> 
> How will that try look like? I'm seeing the following alternatives:
> 
> 1. Just revert a799c2bd29d19c565 ("x86/setup: Consolidate early memory
>    reservations").
> 
> 2. Try to move the call of early_reserve_memory() just before the call
>    of e820__memory_setup().
> 
> 3. Split early_reserve_memory() into two parts, with the first part
>    doing the memblock_reserve() calls for the kernel text, initrd and
>    page 0 right at the start of setup_arch(), and the second part for
>    the rest at the same place it is handled now.
> 
> 4. Analyze why Mike's systems fail to boot and try to fix his issue(s)
>    (probably via one of the above ways).
> 
> Looking at the calls done in early_reserve_memory() I have my doubts
> that memblock_x86_reserve_range_setup_data() will work before
> early_ioremap_init() has been called, as it is using early_memremap().
> This would suggest variant 2 could be a working solution.
> 
> In case I'm wrong with my doubts I'd prefer variant 3.

I actually prefer variant 2 to keep the early memory reservations together
as much as possible. 

I tend to agree with your doubts about
memblock_x86_reserve_range_setup_data(), but it should be fine to move
early_reserve_memory() just before e820__memory_setup().
  
Anyway, I'd like to understand why Mike's systems fail before moving on
with the fixes.

-- 
Sincerely yours,
Mike.

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Nathan Chancellor]

On Sun, Sep 19, 2021 at 07:15:30PM +0200, Borislav Petkov wrote:
> On Sun, Sep 19, 2021 at 06:55:16PM +0200, Mike Galbraith wrote:
> > On Thu, 2021-09-16 at 10:50 +0000, tip-bot2 for Juergen Gross wrote:
> > > The following commit has been merged into the x86/urgent branch of
> > > tip:
> > >
> > > Commit-ID:     1c1046581f1a3809e075669a3df0191869d96dd1
> > > Gitweb:       
> > > https://git.kernel.org/tip/1c1046581f1a3809e075669a3df0191869d96dd1
> > > Author:        Juergen Gross <jgross@suse.com>
> > > AuthorDate:    Tue, 14 Sep 2021 11:41:08 +02:00
> > > Committer:     Borislav Petkov <bp@suse.de>
> > > CommitterDate: Thu, 16 Sep 2021 12:38:05 +02:00
> > >
> > > x86/setup: Call early_reserve_memory() earlier
> > 
> > This commit rendered tip toxic to my i4790 desktop box and i5-6200U
> > lappy.  Boot for both is instantly over without so much as a twitch.
> > 
> > Post bisect revert made both all better.
> 
> I had a suspicion that moving stuff around like that would not just
> simply work in all cases, as our boot order is very lovely and fragile.
> 
> And it booted just fine on my machines here.
> 
> ;-\
> 
> Anyway, commit zapped from the x86/urgent lineup. We'll have to have a
> third try later.

Could auto-latest get updated too so that it does not show up in -next?
I just spent a solid chunk of my day bisecting a boot failure on one of
my test boxes on -next down to this change, only to find out it was
already reported :/

Cheers,
Nathan

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Borislav Petkov]

On Mon, Sep 20, 2021 at 03:48:18PM -0700, Nathan Chancellor wrote:
> Could auto-latest get updated too so that it does not show up in -next?
> I just spent a solid chunk of my day bisecting a boot failure on one of
> my test boxes on -next down to this change, only to find out it was
> already reported :/

Sorry about that - commit is zapped from tip/master and tip/auto-latest.

But your effort hasn't been in vain - you have a box which triggers this
boot issue and I haven't found one yet.

Can you please test on that exact test box whether the new version of
that commit works?

That one:

https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com

It would be much appreciated.

Thx!

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Nathan Chancellor]

On Tue, Sep 21, 2021 at 05:38:40AM +0200, Borislav Petkov wrote:
> On Mon, Sep 20, 2021 at 03:48:18PM -0700, Nathan Chancellor wrote:
> > Could auto-latest get updated too so that it does not show up in -next?
> > I just spent a solid chunk of my day bisecting a boot failure on one of
> > my test boxes on -next down to this change, only to find out it was
> > already reported :/
> 
> Sorry about that - commit is zapped from tip/master and tip/auto-latest.

Thank you!

> But your effort hasn't been in vain - you have a box which triggers this
> boot issue and I haven't found one yet.
> 
> Can you please test on that exact test box whether the new version of
> that commit works?
> 
> That one:
> 
> https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com
> 
> It would be much appreciated.

Sure thing. I tested both of my test systems and added a tested-by tag
to that thread. Glad to hear it was not in vain :)

Cheers,
Nathan

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

[Borislav Petkov]

On Mon, Sep 20, 2021 at 08:59:40PM -0700, Nathan Chancellor wrote:
> Sure thing. I tested both of my test systems and added a tested-by tag
> to that thread. Glad to hear it was not in vain :)

Thanks!

:-)

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette