[PATCH v1] x86/apic: Fix kernel panic when "intremap=off" and "x2apic_phys" are set

[PATCH v1] x86/apic: Fix kernel panic when "intremap=off" and "x2apic_phys" are set

[Dheeraj Kumar Srivastava]

x2APIC mode requires "Interrupt Remapping" to be enabled and the
"physical x2apic" driver can be used only when x2APIC mode is enabled.
However when "intremap=off" and "x2apic_phys" kernel command line
parameters are passed, "physical x2apic" driver is being used even when
x2APIC mode is disabled ("intremap=off" disables x2APIC mode).
This results in the below kernel panic:

  unchecked MSR access error: RDMSR from 0x80f at rIP: 0xffffffff87eab4ec
  (native_read_msr+0xc/0x40)
  Call Trace:
  <TASK>
  native_apic_msr_read+0x1f/0x30
  setup_local_APIC+0x4e/0x380
  ? zen_untrain_ret+0x1/0x1
  ? enable_IR_x2apic+0xe8/0x250
  apic_intr_mode_init+0x4c/0x120
  x86_late_time_init+0x28/0x40
  start_kernel+0x626/0xa80
  x86_64_start_reservations+0x1c/0x30
  x86_64_start_kernel+0xbf/0x110
  secondary_startup_64_no_verify+0x10b/0x10b
  </TASK>

This is due to an incorrect conditional check in x2apic_phys_probe().
Fix it here by returning 0 when "x2apic_mode" is not set in
x2apic_phys_probe(). This now prevents default_setup_apic_routing() from
selecting "physical x2apic" driver.

Fixes: 9ebd680bd029 ("x86, apic: Use probe routines to simplify apic selection")
Reviewed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
 arch/x86/kernel/apic/x2apic_phys.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/x2apic_phys.c b/arch/x86/kernel/apic/x2apic_phys.c
index 6bde05a86b4e..896bc41cb2ba 100644
--- a/arch/x86/kernel/apic/x2apic_phys.c
+++ b/arch/x86/kernel/apic/x2apic_phys.c
@@ -97,7 +97,10 @@ static void init_x2apic_ldr(void)
 
 static int x2apic_phys_probe(void)
 {
-	if (x2apic_mode && (x2apic_phys || x2apic_fadt_phys()))
+	if (!x2apic_mode)
+		return 0;
+
+	if (x2apic_phys || x2apic_fadt_phys())
 		return 1;
 
 	return apic == &apic_x2apic_phys;
-- 
2.25.1

Re: [PATCH v1] x86/apic: Fix kernel panic when "intremap=off" and "x2apic_phys" are set

[Cyrill Gorcunov]

On Sat, Jun 17, 2023 at 02:52:36AM +0530, Dheeraj Kumar Srivastava wrote:
> x2APIC mode requires "Interrupt Remapping" to be enabled and the
> "physical x2apic" driver can be used only when x2APIC mode is enabled.
> However when "intremap=off" and "x2apic_phys" kernel command line
> parameters are passed, "physical x2apic" driver is being used even when
> x2APIC mode is disabled ("intremap=off" disables x2APIC mode).
> This results in the below kernel panic:
...

Hi! Good catch! In long term I think we could switch to use x2apic_state variable
instead since at the moment the code is somehow hard to read and remember which exactly
deps are to be satisfied to enable x2apic mode.

Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>

Re: [PATCH v1] x86/apic: Fix kernel panic when "intremap=off" and "x2apic_phys" are set

[Borislav Petkov]

On Sat, Jun 17, 2023 at 02:52:36AM +0530, Dheeraj Kumar Srivastava wrote:
> x2APIC mode requires "Interrupt Remapping" to be enabled and the
> "physical x2apic" driver can be used only when x2APIC mode is enabled.
> However when "intremap=off" and "x2apic_phys" kernel command line
> parameters are passed, "physical x2apic" driver is being used even when
> x2APIC mode is disabled ("intremap=off" disables x2APIC mode).
> This results in the below kernel panic:
> 
>   unchecked MSR access error: RDMSR from 0x80f at rIP: 0xffffffff87eab4ec

Not a kernel panic - that's a warning about accessing a non-existent
MSR.

Can you send full dmesg? Privately is fine too.

How exactly do you trigger this? What hw?

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v1] x86/apic: Fix kernel panic when "intremap=off" and "x2apic_phys" are set

[Thomas Gleixner]

On Mon, Jun 19 2023 at 01:24, Cyrill Gorcunov wrote:
>
> Hi! Good catch! In long term I think we could switch to use
> x2apic_state variable instead since at the moment the code is somehow
> hard to read and remember which exactly deps are to be satisfied to
> enable x2apic mode.

It's on my todo list for a very long time. Unfortunately that list is
growing only into one direction...

Re: [PATCH v1] x86/apic: Fix kernel panic when "intremap=off" and "x2apic_phys" are set

[Thomas Gleixner]

On Sat, Jun 17 2023 at 02:52, Dheeraj Kumar Srivastava wrote:
> x2APIC mode requires "Interrupt Remapping" to be enabled and the
> "physical x2apic" driver can be used only when x2APIC mode is enabled.
> However when "intremap=off" and "x2apic_phys" kernel command line
> parameters are passed, "physical x2apic" driver is being used even when
> x2APIC mode is disabled ("intremap=off" disables x2APIC mode).
> This results in the below kernel panic:
>
>   unchecked MSR access error: RDMSR from 0x80f at rIP: 0xffffffff87eab4ec
>   (native_read_msr+0xc/0x40)
>   Call Trace:
>   <TASK>
>   native_apic_msr_read+0x1f/0x30
>   setup_local_APIC+0x4e/0x380
>   ? zen_untrain_ret+0x1/0x1
>   ? enable_IR_x2apic+0xe8/0x250
>   apic_intr_mode_init+0x4c/0x120
>   x86_late_time_init+0x28/0x40
>   start_kernel+0x626/0xa80
>   x86_64_start_reservations+0x1c/0x30
>   x86_64_start_kernel+0xbf/0x110
>   secondary_startup_64_no_verify+0x10b/0x10b
>   </TASK>
>
> This is due to an incorrect conditional check in x2apic_phys_probe().
> Fix it here by returning 0 when "x2apic_mode" is not set in
> x2apic_phys_probe(). This now prevents default_setup_apic_routing() from
> selecting "physical x2apic" driver.
>
> Fixes: 9ebd680bd029 ("x86, apic: Use probe routines to simplify apic selection")
> Reviewed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
> Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

[tip: x86/urgent] x86/apic: Fix kernel panic when booting with intremap=off and x2apic_phys

[tip-bot2 for Dheeraj Kumar Srivastava]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     85d38d5810e285d5aec7fb5283107d1da70c12a9
Gitweb:        https://git.kernel.org/tip/85d38d5810e285d5aec7fb5283107d1da70c12a9
Author:        Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
AuthorDate:    Sat, 17 Jun 2023 02:52:36 +05:30
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Mon, 19 Jun 2023 20:59:40 +02:00

x86/apic: Fix kernel panic when booting with intremap=off and x2apic_phys

When booting with "intremap=off" and "x2apic_phys" on the kernel command
line, the physical x2APIC driver ends up being used even when x2APIC
mode is disabled ("intremap=off" disables x2APIC mode). This happens
because the first compound condition check in x2apic_phys_probe() is
false due to x2apic_mode == 0 and so the following one returns true
after default_acpi_madt_oem_check() having already selected the physical
x2APIC driver.

This results in the following panic:

   kernel BUG at arch/x86/kernel/apic/io_apic.c:2409!
   invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
   CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.4.0-rc2-ver4.1rc2 #2
   Hardware name: Dell Inc. PowerEdge R6515/07PXPY, BIOS 2.3.6 07/06/2021
   RIP: 0010:setup_IO_APIC+0x9c/0xaf0
   Call Trace:
    <TASK>
    ? native_read_msr
    apic_intr_mode_init
    x86_late_time_init
    start_kernel
    x86_64_start_reservations
    x86_64_start_kernel
    secondary_startup_64_no_verify
    </TASK>

which is:

setup_IO_APIC:
  apic_printk(APIC_VERBOSE, "ENABLING IO-APIC IRQs\n");
  for_each_ioapic(ioapic)
  	BUG_ON(mp_irqdomain_create(ioapic));

Return 0 to denote that x2APIC has not been enabled when probing the
physical x2APIC driver.

  [ bp: Massage commit message heavily. ]

Fixes: 9ebd680bd029 ("x86, apic: Use probe routines to simplify apic selection")
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20230616212236.1389-1-dheerajkumar.srivastava@amd.com
---
 arch/x86/kernel/apic/x2apic_phys.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/x2apic_phys.c b/arch/x86/kernel/apic/x2apic_phys.c
index 6bde05a..896bc41 100644
--- a/arch/x86/kernel/apic/x2apic_phys.c
+++ b/arch/x86/kernel/apic/x2apic_phys.c
@@ -97,7 +97,10 @@ static void init_x2apic_ldr(void)
 
 static int x2apic_phys_probe(void)
 {
-	if (x2apic_mode && (x2apic_phys || x2apic_fadt_phys()))
+	if (!x2apic_mode)
+		return 0;
+
+	if (x2apic_phys || x2apic_fadt_phys())
 		return 1;
 
 	return apic == &apic_x2apic_phys;