* [PATCH v5 1/6] x86/apic: Split out restore_boot_irq_mode from disable_IO_APIC
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
@ 2018-02-14 5:46 ` Baoquan He
2018-02-17 11:41 ` [tip:x86/apic] x86/apic: Split out restore_boot_irq_mode() from disable_IO_APIC() tip-bot for Baoquan He
2018-02-14 5:46 ` [PATCH v5 2/6] x86/apic: Replace disable_IO_APIC for KEXEC_JUMP Baoquan He
` (5 subsequent siblings)
6 siblings, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-14 5:46 UTC (permalink / raw)
To: linux-kernel, ebiederm
Cc: mingo, tglx, x86, douly.fnst, joro, uobergfe, prarit, Baoquan He
This is a preparation patch. Split out the code which restores boot
irq mode from disable_IO_APIC() and wrap into a new function
restore_boot_irq_mode(). No functional change.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/include/asm/io_apic.h | 1 +
arch/x86/kernel/apic/io_apic.c | 5 +++++
2 files changed, 6 insertions(+)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index a8834dd546cd..558d1a6a13ad 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -193,6 +193,7 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
extern void disable_IO_APIC(void);
+extern void restore_boot_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
extern void print_IO_APICs(void);
#else /* !CONFIG_X86_IO_APIC */
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 8ad2e410974f..7b73b6b9b4b6 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1448,6 +1448,11 @@ void disable_IO_APIC(void)
*/
clear_IO_APIC();
+ restore_boot_irq_mode();
+}
+
+void restore_boot_irq_mode(void)
+{
if (!nr_legacy_irqs())
return;
--
2.13.6
^ permalink raw reply related [flat|nested] 19+ messages in thread* [tip:x86/apic] x86/apic: Split out restore_boot_irq_mode() from disable_IO_APIC()
2018-02-14 5:46 ` [PATCH v5 1/6] x86/apic: Split out restore_boot_irq_mode from disable_IO_APIC Baoquan He
@ 2018-02-17 11:41 ` tip-bot for Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: tip-bot for Baoquan He @ 2018-02-17 11:41 UTC (permalink / raw)
To: linux-tip-commits
Cc: tglx, mingo, torvalds, ebiederm, peterz, linux-kernel, bhe, hpa
Commit-ID: ce279cdc04aafd5c41ae49f941ee2c3342e35e3e
Gitweb: https://git.kernel.org/tip/ce279cdc04aafd5c41ae49f941ee2c3342e35e3e
Author: Baoquan He <bhe@redhat.com>
AuthorDate: Wed, 14 Feb 2018 13:46:51 +0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 17 Feb 2018 11:47:29 +0100
x86/apic: Split out restore_boot_irq_mode() from disable_IO_APIC()
This is a preparation patch. Split out the code which restores boot
irq mode from disable_IO_APIC() into the new restore_boot_irq_mode()
function.
No functional changes.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: douly.fnst@cn.fujitsu.com
Cc: joro@8bytes.org
Cc: prarit@redhat.com
Cc: uobergfe@redhat.com
Link: http://lkml.kernel.org/r/20180214054656.3780-2-bhe@redhat.com
[ Build fix for !CONFIG_IO_APIC and rewrote the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/io_apic.h | 2 ++
arch/x86/kernel/apic/io_apic.c | 5 +++++
2 files changed, 7 insertions(+)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index a8834dd..4e3bb13 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -193,6 +193,7 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
extern void disable_IO_APIC(void);
+extern void restore_boot_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
extern void print_IO_APICs(void);
#else /* !CONFIG_X86_IO_APIC */
@@ -232,6 +233,7 @@ static inline void io_apic_init_mappings(void) { }
static inline void setup_IO_APIC(void) { }
static inline void enable_IO_APIC(void) { }
+static inline void restore_boot_irq_mode(void) { }
#endif
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 8ad2e41..7b73b6b 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1448,6 +1448,11 @@ void disable_IO_APIC(void)
*/
clear_IO_APIC();
+ restore_boot_irq_mode();
+}
+
+void restore_boot_irq_mode(void)
+{
if (!nr_legacy_irqs())
return;
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v5 2/6] x86/apic: Replace disable_IO_APIC for KEXEC_JUMP
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
2018-02-14 5:46 ` [PATCH v5 1/6] x86/apic: Split out restore_boot_irq_mode from disable_IO_APIC Baoquan He
@ 2018-02-14 5:46 ` Baoquan He
2018-02-17 11:41 ` [tip:x86/apic] x86/apic: Split disable_IO_APIC() into two functions to fix CONFIG_KEXEC_JUMP=y tip-bot for Baoquan He
2018-02-14 5:46 ` [PATCH v5 3/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
` (4 subsequent siblings)
6 siblings, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-14 5:46 UTC (permalink / raw)
To: linux-kernel, ebiederm
Cc: mingo, tglx, x86, douly.fnst, joro, uobergfe, prarit, Baoquan He
Later disable_IO_APIC() will be broken down into clear_IO_APIC()
and restore_boot_irq_mode(). These two functions will be called
separately where they are needed to fix a regression introduced
by commit 522e66464467 ("x86/apic: Disable I/O APIC before
shutdown of the local APIC").
While KEXEC_JUMP code doesn't call lapic_shutdown() before jump
like kexec/kdump, so it's not impacted by commit 522e66464467.
Hence here change clear_IO_APIC() as public, and replace disable_IO_APIC()
with clear_IO_APIC() and restore_boot_irq_mode() to keep KEXEC_JUMP
code unchanged in essence. No functional change.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
v4->v5:
Make this patch to replace disable_IO_APIC() with clear_IO_APIC
and restore_boot_irq_mode() for KEXEC_JUMP path only. This makes
patch easier to review according to Eric's suggestion..
arch/x86/include/asm/io_apic.h | 1 +
arch/x86/kernel/apic/io_apic.c | 2 +-
arch/x86/kernel/machine_kexec_32.c | 3 ++-
arch/x86/kernel/machine_kexec_64.c | 3 ++-
4 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 558d1a6a13ad..0fa95bfacb39 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -193,6 +193,7 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
extern void disable_IO_APIC(void);
+extern void clear_IO_APIC(void);
extern void restore_boot_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
extern void print_IO_APICs(void);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 7b73b6b9b4b6..2d7cd2db77f5 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -587,7 +587,7 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
mpc_ioapic_id(apic), pin);
}
-static void clear_IO_APIC (void)
+void clear_IO_APIC (void)
{
int apic, pin;
diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c
index edfede768688..4cd79d88a4ac 100644
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -199,7 +199,8 @@ void machine_kexec(struct kimage *image)
* one form or other. kexec jump path also need
* one.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
+ restore_boot_irq_mode();
#endif
}
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 1f790cf9d38f..2ab14b9c1a89 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -297,7 +297,8 @@ void machine_kexec(struct kimage *image)
* one form or other. kexec jump path also need
* one.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
+ restore_boot_irq_mode();
#endif
}
--
2.13.6
^ permalink raw reply related [flat|nested] 19+ messages in thread* [tip:x86/apic] x86/apic: Split disable_IO_APIC() into two functions to fix CONFIG_KEXEC_JUMP=y
2018-02-14 5:46 ` [PATCH v5 2/6] x86/apic: Replace disable_IO_APIC for KEXEC_JUMP Baoquan He
@ 2018-02-17 11:41 ` tip-bot for Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: tip-bot for Baoquan He @ 2018-02-17 11:41 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, torvalds, bhe, tglx, mingo, peterz, hpa, ebiederm
Commit-ID: 3c9e76dbea004b2c7c3ce872022ceaf5ff0dae79
Gitweb: https://git.kernel.org/tip/3c9e76dbea004b2c7c3ce872022ceaf5ff0dae79
Author: Baoquan He <bhe@redhat.com>
AuthorDate: Wed, 14 Feb 2018 13:46:52 +0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 17 Feb 2018 11:47:44 +0100
x86/apic: Split disable_IO_APIC() into two functions to fix CONFIG_KEXEC_JUMP=y
Split following patches disable_IO_APIC() will be broken up into
clear_IO_APIC() and restore_boot_irq_mode().
These two functions will be called separately where they are needed
to fix a regression introduced by:
522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC").
While the CONFIG_KEXEC_JUMP=y code doesn't call lapic_shutdown() before jump
like kexec/kdump, so it's not impacted by commit 522e66464467.
Hence here change clear_IO_APIC() as public, and replace disable_IO_APIC()
with clear_IO_APIC() and restore_boot_irq_mode() to keep CONFIG_KEXEC_JUMP=y
code unchanged in essence. No functional change.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: douly.fnst@cn.fujitsu.com
Cc: joro@8bytes.org
Cc: prarit@redhat.com
Cc: uobergfe@redhat.com
Link: http://lkml.kernel.org/r/20180214054656.3780-3-bhe@redhat.com
[ Rewrote the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/io_apic.h | 1 +
arch/x86/kernel/apic/io_apic.c | 2 +-
arch/x86/kernel/machine_kexec_32.c | 3 ++-
arch/x86/kernel/machine_kexec_64.c | 3 ++-
4 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 4e3bb13..2ae1b424c 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -193,6 +193,7 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
extern void disable_IO_APIC(void);
+extern void clear_IO_APIC(void);
extern void restore_boot_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
extern void print_IO_APICs(void);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 7b73b6b..2d7cd2d 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -587,7 +587,7 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
mpc_ioapic_id(apic), pin);
}
-static void clear_IO_APIC (void)
+void clear_IO_APIC (void)
{
int apic, pin;
diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c
index edfede7..4cd79d8 100644
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -199,7 +199,8 @@ void machine_kexec(struct kimage *image)
* one form or other. kexec jump path also need
* one.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
+ restore_boot_irq_mode();
#endif
}
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 1f790cf..2ab14b9 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -297,7 +297,8 @@ void machine_kexec(struct kimage *image)
* one form or other. kexec jump path also need
* one.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
+ restore_boot_irq_mode();
#endif
}
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v5 3/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
2018-02-14 5:46 ` [PATCH v5 1/6] x86/apic: Split out restore_boot_irq_mode from disable_IO_APIC Baoquan He
2018-02-14 5:46 ` [PATCH v5 2/6] x86/apic: Replace disable_IO_APIC for KEXEC_JUMP Baoquan He
@ 2018-02-14 5:46 ` Baoquan He
2018-02-17 11:42 ` [tip:x86/apic] x86/apic: Fix restoring boot IRQ " tip-bot for Baoquan He
2018-02-14 5:46 ` [PATCH v5 4/6] x86/apic: Remove useless disable_IO_APIC Baoquan He
` (3 subsequent siblings)
6 siblings, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-14 5:46 UTC (permalink / raw)
To: linux-kernel, ebiederm
Cc: mingo, tglx, x86, douly.fnst, joro, uobergfe, prarit, Baoquan He, stable
This is a regression fix.
Before, to fix erratum AVR31, commit 522e66464467 ("x86/apic: Disable
I/O APIC before shutdown of the local APIC") moved lapic_shutdown()
calling after disable_IO_APIC() in reboot and kexec/kdump code path.
This introdued a regression. The root cause is that disable_IO_APIC()
not only clears IO_APIC, also restore boot irq mode by setting
LAPIC/APIC/IMCR, calling lapic_shutdown() after disable_IO_APIC() will
disable LAPIC and ruin the possible virtual wire mode setting which
the code has been trying to do all along.
The consequence is, in KVM guest kernel always prints warning as below
during kexec/kdump kernel boots up. That happened in setup_local_APIC()
since 'do { xxx } while (queued && max_loops > 0)' loop does not function
well any more if pending irq exists in APIC IRR after LAPIC is disabled.
[ 0.001000] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/apic/apic.c:1467 setup_local_APIC+0x228/0x330
[ 0.001000] Modules linked in:
[ 0.001000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc5+ #3
[ 0.001000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1.fc26 04/01/2014
[ 0.001000] RIP: 0010:setup_local_APIC+0x228/0x330
[ 0.001000] RSP: 0000:ffffffffb6e03eb8 EFLAGS: 00010286
[ 0.001000] RAX: 0000009edb4c4d84 RBX: 0000000000000000 RCX: 00000000b099d800
[ 0.001000] RDX: 0000009e00000000 RSI: 0000000000000000 RDI: 0000000000000810
[ 0.001000] RBP: 0000000000000000 R08: ffffffffffffffff R09: 0000000000000001
[ 0.001000] R10: ffff98ce6a801c00 R11: 0761076d072f0776 R12: 0000000000000001
[ 0.001000] R13: 00000000000000f0 R14: 0000000000004000 R15: ffffffffffffc6ff
[ 0.001000] FS: 0000000000000000(0000) GS:ffff98ce6bc00000(0000) knlGS:0000000000000000
[ 0.001000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.001000] CR2: 00000000ffffffff CR3: 0000000022209000 CR4: 00000000000406b0
[ 0.001000] Call Trace:
[ 0.001000] apic_bsp_setup+0x56/0x74
[ 0.001000] x86_late_time_init+0x11/0x16
[ 0.001000] start_kernel+0x3c9/0x486
[ 0.001000] secondary_startup_64+0xa5/0xb0
[ 0.001000] Code: 00 85 c9 74 2d 0f 31 c1 e1 0a 48 c1 e2 20 41 89 cf 4c 03 7c 24 08 48 09 d0 49 29 c7 4c 89 3c 24 48 83 3c 24 00 0f 8f 8f fe ff
ff <0f> ff e9 10 ff ff ff 48 83 2c 24 01 eb e7 48 83 c4 18 5b 5d 41
[ 0.001000] ---[ end trace b88e71b9a6ebebdd ]---
[ 0.001000] masked ExtINT on CPU#0
To fix this, just call clear_IO_APIC() to stop IO_APIC where
disable_IO_APIC() was called, and call restore_boot_irq_mode() to
restore boot irq mode before reboot or kexec/kdump jump.
Signed-off-by: Baoquan He <bhe@redhat.com>
Fixes: commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
Cc: stable@vger.kernel.org
---
v4->v5:
Take the change related to KEXEC_JUMP out to new patch 0002.
v4->v3:
Eric pointed out the change related to KEXEC_JUMP is not right.
Correct it.
Add Fixes tag and Cc to stable.
arch/x86/kernel/crash.c | 3 ++-
arch/x86/kernel/reboot.c | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 10e74d4778a1..1f6680427ff0 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -199,9 +199,10 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
#ifdef CONFIG_X86_IO_APIC
/* Prevent crash_kexec() from deadlocking on ioapic_lock. */
ioapic_zap_locks();
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
#endif
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 2126b9d27c34..725624b6c0c0 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -666,7 +666,7 @@ void native_machine_shutdown(void)
* Even without the erratum, it still makes sense to quiet IO APIC
* before disabling Local APIC.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
#ifdef CONFIG_SMP
@@ -680,6 +680,7 @@ void native_machine_shutdown(void)
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
--
2.13.6
^ permalink raw reply related [flat|nested] 19+ messages in thread* [tip:x86/apic] x86/apic: Fix restoring boot IRQ mode in reboot and kexec/kdump
2018-02-14 5:46 ` [PATCH v5 3/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
@ 2018-02-17 11:42 ` tip-bot for Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: tip-bot for Baoquan He @ 2018-02-17 11:42 UTC (permalink / raw)
To: linux-tip-commits
Cc: mingo, ebiederm, bhe, tglx, linux-kernel, hpa, torvalds, peterz
Commit-ID: 339b2ae0cd5d4a58f9efe06e4ee36adbeca59228
Gitweb: https://git.kernel.org/tip/339b2ae0cd5d4a58f9efe06e4ee36adbeca59228
Author: Baoquan He <bhe@redhat.com>
AuthorDate: Wed, 14 Feb 2018 13:46:53 +0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 17 Feb 2018 11:47:45 +0100
x86/apic: Fix restoring boot IRQ mode in reboot and kexec/kdump
This is a regression fix.
Before, to fix erratum AVR31, the following commit:
522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
... moved the lapic_shutdown() call to after disable_IO_APIC() in the reboot
and kexec/kdump code paths.
This introduced the following regression: disable_IO_APIC() not only clears
the IO-APIC, but it also restores boot IRQ mode by setting the
LAPIC/APIC/IMCR, calling lapic_shutdown() after disable_IO_APIC() will
disable LAPIC and ruin the possible virtual wire mode setting which
the code has been trying to do all along.
The consequence is that a KVM guest kernel always prints the warning below
during kexec/kdump as the kernel boots up:
[ 0.001000] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/apic/apic.c:1467 setup_local_APIC+0x228/0x330
[ ........]
[ 0.001000] Call Trace:
[ 0.001000] apic_bsp_setup+0x56/0x74
[ 0.001000] x86_late_time_init+0x11/0x16
[ 0.001000] start_kernel+0x3c9/0x486
[ 0.001000] secondary_startup_64+0xa5/0xb0
[ ........]
[ 0.001000] masked ExtINT on CPU#0
To fix this, just call clear_IO_APIC() to stop the IO-APIC where
disable_IO_APIC() was called, and call restore_boot_irq_mode() to
restore boot IRQ mode before a reboot or a kexec/kdump jump.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: douly.fnst@cn.fujitsu.com
Cc: joro@8bytes.org
Cc: prarit@redhat.com
Cc: stable@vger.kernel.org
Cc: uobergfe@redhat.com
Fixes: commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
Link: http://lkml.kernel.org/r/20180214054656.3780-4-bhe@redhat.com
[ Rewrote the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/kernel/crash.c | 3 ++-
arch/x86/kernel/reboot.c | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 10e74d4..1f66804 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -199,9 +199,10 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
#ifdef CONFIG_X86_IO_APIC
/* Prevent crash_kexec() from deadlocking on ioapic_lock. */
ioapic_zap_locks();
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
#endif
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 2126b9d..725624b 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -666,7 +666,7 @@ void native_machine_shutdown(void)
* Even without the erratum, it still makes sense to quiet IO APIC
* before disabling Local APIC.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
#ifdef CONFIG_SMP
@@ -680,6 +680,7 @@ void native_machine_shutdown(void)
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v5 4/6] x86/apic: Remove useless disable_IO_APIC
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
` (2 preceding siblings ...)
2018-02-14 5:46 ` [PATCH v5 3/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
@ 2018-02-14 5:46 ` Baoquan He
2018-02-17 11:42 ` [tip:x86/apic] x86/apic: Remove the (now) unused disable_IO_APIC() function tip-bot for Baoquan He
2018-02-14 5:46 ` [PATCH v5 5/6] x86/apic: Rename variable/function related to x86_io_apic_ops Baoquan He
` (2 subsequent siblings)
6 siblings, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-14 5:46 UTC (permalink / raw)
To: linux-kernel, ebiederm
Cc: mingo, tglx, x86, douly.fnst, joro, uobergfe, prarit, Baoquan He
No one uses it anymore.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/include/asm/io_apic.h | 1 -
arch/x86/kernel/apic/io_apic.c | 13 -------------
arch/x86/kernel/machine_kexec_32.c | 5 ++---
arch/x86/kernel/machine_kexec_64.c | 5 ++---
4 files changed, 4 insertions(+), 20 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 0fa95bfacb39..5e389145d808 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -192,7 +192,6 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
-extern void disable_IO_APIC(void);
extern void clear_IO_APIC(void);
extern void restore_boot_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 2d7cd2db77f5..9d86b10c2121 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1438,19 +1438,6 @@ void native_disable_io_apic(void)
disconnect_bsp_APIC(ioapic_i8259.pin != -1);
}
-/*
- * Not an __init, needed by the reboot code
- */
-void disable_IO_APIC(void)
-{
- /*
- * Clear the IO-APIC before rebooting:
- */
- clear_IO_APIC();
-
- restore_boot_irq_mode();
-}
-
void restore_boot_irq_mode(void)
{
if (!nr_legacy_irqs())
diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c
index 4cd79d88a4ac..60cdec6628b0 100644
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -195,9 +195,8 @@ void machine_kexec(struct kimage *image)
/*
* We need to put APICs in legacy mode so that we can
* get timer interrupts in second kernel. kexec/kdump
- * paths already have calls to disable_IO_APIC() in
- * one form or other. kexec jump path also need
- * one.
+ * paths already have calls to restore_boot_irq_mode()
+ * in one form or other. kexec jump path also need one.
*/
clear_IO_APIC();
restore_boot_irq_mode();
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 2ab14b9c1a89..5ffbc55ea80f 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -293,9 +293,8 @@ void machine_kexec(struct kimage *image)
/*
* We need to put APICs in legacy mode so that we can
* get timer interrupts in second kernel. kexec/kdump
- * paths already have calls to disable_IO_APIC() in
- * one form or other. kexec jump path also need
- * one.
+ * paths already have calls to restore_boot_irq_mode()
+ * in one form or other. kexec jump path also need one.
*/
clear_IO_APIC();
restore_boot_irq_mode();
--
2.13.6
^ permalink raw reply related [flat|nested] 19+ messages in thread* [tip:x86/apic] x86/apic: Remove the (now) unused disable_IO_APIC() function
2018-02-14 5:46 ` [PATCH v5 4/6] x86/apic: Remove useless disable_IO_APIC Baoquan He
@ 2018-02-17 11:42 ` tip-bot for Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: tip-bot for Baoquan He @ 2018-02-17 11:42 UTC (permalink / raw)
To: linux-tip-commits
Cc: tglx, linux-kernel, torvalds, mingo, ebiederm, bhe, hpa, peterz
Commit-ID: 50374b96d2d30c03c8d42b3f8846d8938748d454
Gitweb: https://git.kernel.org/tip/50374b96d2d30c03c8d42b3f8846d8938748d454
Author: Baoquan He <bhe@redhat.com>
AuthorDate: Wed, 14 Feb 2018 13:46:54 +0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 17 Feb 2018 11:47:45 +0100
x86/apic: Remove the (now) unused disable_IO_APIC() function
No one uses it anymore.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: douly.fnst@cn.fujitsu.com
Cc: joro@8bytes.org
Cc: prarit@redhat.com
Cc: uobergfe@redhat.com
Link: http://lkml.kernel.org/r/20180214054656.3780-5-bhe@redhat.com
[ Rewrote the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/io_apic.h | 1 -
arch/x86/kernel/apic/io_apic.c | 13 -------------
arch/x86/kernel/machine_kexec_32.c | 5 ++---
arch/x86/kernel/machine_kexec_64.c | 5 ++---
4 files changed, 4 insertions(+), 20 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 2ae1b424c..8018fc4 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -192,7 +192,6 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
-extern void disable_IO_APIC(void);
extern void clear_IO_APIC(void);
extern void restore_boot_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 2d7cd2d..9d86b10 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1438,19 +1438,6 @@ void native_disable_io_apic(void)
disconnect_bsp_APIC(ioapic_i8259.pin != -1);
}
-/*
- * Not an __init, needed by the reboot code
- */
-void disable_IO_APIC(void)
-{
- /*
- * Clear the IO-APIC before rebooting:
- */
- clear_IO_APIC();
-
- restore_boot_irq_mode();
-}
-
void restore_boot_irq_mode(void)
{
if (!nr_legacy_irqs())
diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c
index 4cd79d8..60cdec6 100644
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -195,9 +195,8 @@ void machine_kexec(struct kimage *image)
/*
* We need to put APICs in legacy mode so that we can
* get timer interrupts in second kernel. kexec/kdump
- * paths already have calls to disable_IO_APIC() in
- * one form or other. kexec jump path also need
- * one.
+ * paths already have calls to restore_boot_irq_mode()
+ * in one form or other. kexec jump path also need one.
*/
clear_IO_APIC();
restore_boot_irq_mode();
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 2ab14b9..5ffbc55 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -293,9 +293,8 @@ void machine_kexec(struct kimage *image)
/*
* We need to put APICs in legacy mode so that we can
* get timer interrupts in second kernel. kexec/kdump
- * paths already have calls to disable_IO_APIC() in
- * one form or other. kexec jump path also need
- * one.
+ * paths already have calls to restore_boot_irq_mode()
+ * in one form or other. kexec jump path also need one.
*/
clear_IO_APIC();
restore_boot_irq_mode();
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v5 5/6] x86/apic: Rename variable/function related to x86_io_apic_ops
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
` (3 preceding siblings ...)
2018-02-14 5:46 ` [PATCH v5 4/6] x86/apic: Remove useless disable_IO_APIC Baoquan He
@ 2018-02-14 5:46 ` Baoquan He
2018-02-17 11:43 ` [tip:x86/apic] x86/apic: Rename variables and functions " tip-bot for Baoquan He
2018-02-14 5:46 ` [PATCH v5 6/6] x86/apic: Set up through-local-APIC on boot CPU if 'noapic' specified Baoquan He
2018-02-16 9:38 ` [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Ingo Molnar
6 siblings, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-14 5:46 UTC (permalink / raw)
To: linux-kernel, ebiederm
Cc: mingo, tglx, x86, douly.fnst, joro, uobergfe, prarit, Baoquan He
The names of x86_io_apic_ops and its two member variables, are
misleading. The .read member is to read IO_APIC reg, while .disable
which hook native_disable_io_apic/irq_remapping_disable_io_apic
is actually used to restore boot irq mode, not disable IO_APIC.
So rename x86_io_apic_ops to x86_apic_ops since it doesn't only
handle IO_APIC, also LAPIC.
And also rename its member variables and the related hooks.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/include/asm/io_apic.h | 6 +++---
arch/x86/include/asm/x86_init.h | 8 ++++----
arch/x86/kernel/apic/io_apic.c | 4 ++--
arch/x86/kernel/x86_init.c | 6 +++---
arch/x86/xen/apic.c | 2 +-
drivers/iommu/irq_remapping.c | 4 ++--
6 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 5e389145d808..06fec4426458 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -183,11 +183,11 @@ extern void disable_ioapic_support(void);
extern void __init io_apic_init_mappings(void);
extern unsigned int native_io_apic_read(unsigned int apic, unsigned int reg);
-extern void native_disable_io_apic(void);
+extern void native_restore_boot_irq_mode(void);
static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
{
- return x86_io_apic_ops.read(apic, reg);
+ return x86_apic_ops.io_apic_read(apic, reg);
}
extern void setup_IO_APIC(void);
@@ -229,7 +229,7 @@ static inline void mp_save_irq(struct mpc_intsrc *m) { }
static inline void disable_ioapic_support(void) { }
static inline void io_apic_init_mappings(void) { }
#define native_io_apic_read NULL
-#define native_disable_io_apic NULL
+#define native_restore_boot_irq_mode NULL
static inline void setup_IO_APIC(void) { }
static inline void enable_IO_APIC(void) { }
diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h
index fc2f082ac635..88306054bd98 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -274,16 +274,16 @@ struct x86_msi_ops {
void (*restore_msi_irqs)(struct pci_dev *dev);
};
-struct x86_io_apic_ops {
- unsigned int (*read) (unsigned int apic, unsigned int reg);
- void (*disable)(void);
+struct x86_apic_ops {
+ unsigned int (*io_apic_read) (unsigned int apic, unsigned int reg);
+ void (*restore)(void);
};
extern struct x86_init_ops x86_init;
extern struct x86_cpuinit_ops x86_cpuinit;
extern struct x86_platform_ops x86_platform;
extern struct x86_msi_ops x86_msi;
-extern struct x86_io_apic_ops x86_io_apic_ops;
+extern struct x86_apic_ops x86_apic_ops;
extern void x86_early_init_platform_quirks(void);
extern void x86_init_noop(void);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 9d86b10c2121..68129f11e7db 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1410,7 +1410,7 @@ void __init enable_IO_APIC(void)
clear_IO_APIC();
}
-void native_disable_io_apic(void)
+void native_restore_boot_irq_mode(void)
{
/*
* If the i8259 is routed through an IOAPIC
@@ -1443,7 +1443,7 @@ void restore_boot_irq_mode(void)
if (!nr_legacy_irqs())
return;
- x86_io_apic_ops.disable();
+ x86_apic_ops.restore();
}
#ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 1151ccd72ce9..2bccd03bd654 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -146,7 +146,7 @@ void arch_restore_msi_irqs(struct pci_dev *dev)
}
#endif
-struct x86_io_apic_ops x86_io_apic_ops __ro_after_init = {
- .read = native_io_apic_read,
- .disable = native_disable_io_apic,
+struct x86_apic_ops x86_apic_ops __ro_after_init = {
+ .io_apic_read = native_io_apic_read,
+ .restore = native_restore_boot_irq_mode,
};
diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c
index de58533d3664..2163888497d3 100644
--- a/arch/x86/xen/apic.c
+++ b/arch/x86/xen/apic.c
@@ -215,7 +215,7 @@ static void __init xen_apic_check(void)
}
void __init xen_init_apic(void)
{
- x86_io_apic_ops.read = xen_io_apic_read;
+ x86_apic_ops.io_apic_read = xen_io_apic_read;
/* On PV guests the APIC CPUID bit is disabled so none of the
* routines end up executing. */
if (!xen_initial_domain())
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index 49721b4e1975..496deee3ae3a 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -27,7 +27,7 @@ int disable_irq_post = 0;
static int disable_irq_remap;
static struct irq_remap_ops *remap_ops;
-static void irq_remapping_disable_io_apic(void)
+static void irq_remapping_restore_boot_irq_mode(void)
{
/*
* With interrupt-remapping, for now we will use virtual wire A
@@ -42,7 +42,7 @@ static void irq_remapping_disable_io_apic(void)
static void __init irq_remapping_modify_x86_ops(void)
{
- x86_io_apic_ops.disable = irq_remapping_disable_io_apic;
+ x86_apic_ops.restore = irq_remapping_restore_boot_irq_mode;
}
static __init int setup_nointremap(char *str)
--
2.13.6
^ permalink raw reply related [flat|nested] 19+ messages in thread* [tip:x86/apic] x86/apic: Rename variables and functions related to x86_io_apic_ops
2018-02-14 5:46 ` [PATCH v5 5/6] x86/apic: Rename variable/function related to x86_io_apic_ops Baoquan He
@ 2018-02-17 11:43 ` tip-bot for Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: tip-bot for Baoquan He @ 2018-02-17 11:43 UTC (permalink / raw)
To: linux-tip-commits
Cc: peterz, ebiederm, tglx, bhe, linux-kernel, hpa, torvalds, mingo
Commit-ID: 51b146c572201e3c368e0baa3e565760aefcf25f
Gitweb: https://git.kernel.org/tip/51b146c572201e3c368e0baa3e565760aefcf25f
Author: Baoquan He <bhe@redhat.com>
AuthorDate: Wed, 14 Feb 2018 13:46:55 +0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 17 Feb 2018 11:47:45 +0100
x86/apic: Rename variables and functions related to x86_io_apic_ops
The names of x86_io_apic_ops and its two member variables are
misleading:
The ->read() member is to read IO_APIC reg, while ->disable()
which is called by native_disable_io_apic()/irq_remapping_disable_io_apic()
is actually used to restore boot IRQ mode, not to disable the IO-APIC.
So rename x86_io_apic_ops to 'x86_apic_ops' since it doesn't only
handle the IO-APIC, but also the local APIC.
Also rename its member variables and the related callbacks.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: douly.fnst@cn.fujitsu.com
Cc: joro@8bytes.org
Cc: prarit@redhat.com
Cc: uobergfe@redhat.com
Link: http://lkml.kernel.org/r/20180214054656.3780-6-bhe@redhat.com
[ Rewrote the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/io_apic.h | 6 +++---
arch/x86/include/asm/x86_init.h | 8 ++++----
arch/x86/kernel/apic/io_apic.c | 4 ++--
arch/x86/kernel/x86_init.c | 6 +++---
arch/x86/xen/apic.c | 2 +-
drivers/iommu/irq_remapping.c | 4 ++--
6 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 8018fc4..fd20a23 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -183,11 +183,11 @@ extern void disable_ioapic_support(void);
extern void __init io_apic_init_mappings(void);
extern unsigned int native_io_apic_read(unsigned int apic, unsigned int reg);
-extern void native_disable_io_apic(void);
+extern void native_restore_boot_irq_mode(void);
static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
{
- return x86_io_apic_ops.read(apic, reg);
+ return x86_apic_ops.io_apic_read(apic, reg);
}
extern void setup_IO_APIC(void);
@@ -229,7 +229,7 @@ static inline void mp_save_irq(struct mpc_intsrc *m) { }
static inline void disable_ioapic_support(void) { }
static inline void io_apic_init_mappings(void) { }
#define native_io_apic_read NULL
-#define native_disable_io_apic NULL
+#define native_restore_boot_irq_mode NULL
static inline void setup_IO_APIC(void) { }
static inline void enable_IO_APIC(void) { }
diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h
index fc2f082..8830605 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -274,16 +274,16 @@ struct x86_msi_ops {
void (*restore_msi_irqs)(struct pci_dev *dev);
};
-struct x86_io_apic_ops {
- unsigned int (*read) (unsigned int apic, unsigned int reg);
- void (*disable)(void);
+struct x86_apic_ops {
+ unsigned int (*io_apic_read) (unsigned int apic, unsigned int reg);
+ void (*restore)(void);
};
extern struct x86_init_ops x86_init;
extern struct x86_cpuinit_ops x86_cpuinit;
extern struct x86_platform_ops x86_platform;
extern struct x86_msi_ops x86_msi;
-extern struct x86_io_apic_ops x86_io_apic_ops;
+extern struct x86_apic_ops x86_apic_ops;
extern void x86_early_init_platform_quirks(void);
extern void x86_init_noop(void);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 9d86b10..68129f1 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1410,7 +1410,7 @@ void __init enable_IO_APIC(void)
clear_IO_APIC();
}
-void native_disable_io_apic(void)
+void native_restore_boot_irq_mode(void)
{
/*
* If the i8259 is routed through an IOAPIC
@@ -1443,7 +1443,7 @@ void restore_boot_irq_mode(void)
if (!nr_legacy_irqs())
return;
- x86_io_apic_ops.disable();
+ x86_apic_ops.restore();
}
#ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 1151ccd..2bccd03 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -146,7 +146,7 @@ void arch_restore_msi_irqs(struct pci_dev *dev)
}
#endif
-struct x86_io_apic_ops x86_io_apic_ops __ro_after_init = {
- .read = native_io_apic_read,
- .disable = native_disable_io_apic,
+struct x86_apic_ops x86_apic_ops __ro_after_init = {
+ .io_apic_read = native_io_apic_read,
+ .restore = native_restore_boot_irq_mode,
};
diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c
index de58533..2163888 100644
--- a/arch/x86/xen/apic.c
+++ b/arch/x86/xen/apic.c
@@ -215,7 +215,7 @@ static void __init xen_apic_check(void)
}
void __init xen_init_apic(void)
{
- x86_io_apic_ops.read = xen_io_apic_read;
+ x86_apic_ops.io_apic_read = xen_io_apic_read;
/* On PV guests the APIC CPUID bit is disabled so none of the
* routines end up executing. */
if (!xen_initial_domain())
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index 49721b4..496deee 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -27,7 +27,7 @@ int disable_irq_post = 0;
static int disable_irq_remap;
static struct irq_remap_ops *remap_ops;
-static void irq_remapping_disable_io_apic(void)
+static void irq_remapping_restore_boot_irq_mode(void)
{
/*
* With interrupt-remapping, for now we will use virtual wire A
@@ -42,7 +42,7 @@ static void irq_remapping_disable_io_apic(void)
static void __init irq_remapping_modify_x86_ops(void)
{
- x86_io_apic_ops.disable = irq_remapping_disable_io_apic;
+ x86_apic_ops.restore = irq_remapping_restore_boot_irq_mode;
}
static __init int setup_nointremap(char *str)
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v5 6/6] x86/apic: Set up through-local-APIC on boot CPU if 'noapic' specified
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
` (4 preceding siblings ...)
2018-02-14 5:46 ` [PATCH v5 5/6] x86/apic: Rename variable/function related to x86_io_apic_ops Baoquan He
@ 2018-02-14 5:46 ` Baoquan He
2018-02-17 11:43 ` [tip:x86/apic] x86/apic: Set up through-local-APIC mode on the " tip-bot for Baoquan He
2018-02-16 9:38 ` [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Ingo Molnar
6 siblings, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-14 5:46 UTC (permalink / raw)
To: linux-kernel, ebiederm
Cc: mingo, tglx, x86, douly.fnst, joro, uobergfe, prarit, Baoquan He
Currently kdump kernel becomes very slow if 'noapic' is specified.
Normal kernel won't.
Kernel parameter 'noapic' is used to disable IO-APIC in system for
testing or special purpose. Here the root cause is that in kdump
kernel LAPIC is disabled since commit 522e664644
("x86/apic: Disable I/O APIC before shutdown of the local APIC").
In this case We need set up through-local-APIC on boot CPU in
setup_local_APIC().
While In normal kernel the legacy irq mode is enabled in BIOS. If
it is virtual wire mode, the local-APIC has been enabled and set as
through-local-APIC.
Though we fix the regression introduced by criminal commit 522e664644,
for safety and clarity, better set up through-local-APIC explicitly,
but not rely on the default boot irq mode.
Do it now.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/kernel/apic/apic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 25ddf02598d2..3fc259b4dd2d 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1570,7 +1570,7 @@ void setup_local_APIC(void)
* TODO: set up through-local-APIC from through-I/O-APIC? --macro
*/
value = apic_read(APIC_LVT0) & APIC_LVT_MASKED;
- if (!cpu && (pic_mode || !value)) {
+ if (!cpu && (pic_mode || !value || skip_ioapic_setup)) {
value = APIC_DM_EXTINT;
apic_printk(APIC_VERBOSE, "enabled ExtINT on CPU#%d\n", cpu);
} else {
--
2.13.6
^ permalink raw reply related [flat|nested] 19+ messages in thread* [tip:x86/apic] x86/apic: Set up through-local-APIC mode on the boot CPU if 'noapic' specified
2018-02-14 5:46 ` [PATCH v5 6/6] x86/apic: Set up through-local-APIC on boot CPU if 'noapic' specified Baoquan He
@ 2018-02-17 11:43 ` tip-bot for Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: tip-bot for Baoquan He @ 2018-02-17 11:43 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, torvalds, bhe, hpa, mingo, ebiederm, tglx, peterz
Commit-ID: bee3204ec3c49f6f53add9c3962c9012a5c036fa
Gitweb: https://git.kernel.org/tip/bee3204ec3c49f6f53add9c3962c9012a5c036fa
Author: Baoquan He <bhe@redhat.com>
AuthorDate: Wed, 14 Feb 2018 13:46:56 +0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 17 Feb 2018 11:47:46 +0100
x86/apic: Set up through-local-APIC mode on the boot CPU if 'noapic' specified
Currently the kdump kernel becomes very slow if 'noapic' is specified.
Normal kernel doesn't have this bug.
Kernel parameter 'noapic' is used to disable IO-APIC in system for
testing or special purpose. Here the root cause is that in kdump
kernel LAPIC is disabled since commit:
522e664644 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
In this case we need set up through-local-APIC on boot CPU in
setup_local_APIC().
In normal kernel the legacy irq mode is enabled by the BIOS. If
it is virtual wire mode, the local-APIC has been enabled and set as
through-local-APIC.
Though we fixed the regression introduced by commit 522e664644,
to further improve robustness set up the through-local-APIC mode
explicitly, do not rely on the default boot IRQ mode.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: douly.fnst@cn.fujitsu.com
Cc: joro@8bytes.org
Cc: prarit@redhat.com
Cc: uobergfe@redhat.com
Link: http://lkml.kernel.org/r/20180214054656.3780-7-bhe@redhat.com
[ Rewrote the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/kernel/apic/apic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 871018d..2ceac9f 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1570,7 +1570,7 @@ static void setup_local_APIC(void)
* TODO: set up through-local-APIC from through-I/O-APIC? --macro
*/
value = apic_read(APIC_LVT0) & APIC_LVT_MASKED;
- if (!cpu && (pic_mode || !value)) {
+ if (!cpu && (pic_mode || !value || skip_ioapic_setup)) {
value = APIC_DM_EXTINT;
apic_printk(APIC_VERBOSE, "enabled ExtINT on CPU#%d\n", cpu);
} else {
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-14 5:46 [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Baoquan He
` (5 preceding siblings ...)
2018-02-14 5:46 ` [PATCH v5 6/6] x86/apic: Set up through-local-APIC on boot CPU if 'noapic' specified Baoquan He
@ 2018-02-16 9:38 ` Ingo Molnar
2018-02-16 10:24 ` Baoquan He
2018-02-16 19:16 ` Eric W. Biederman
6 siblings, 2 replies; 19+ messages in thread
From: Ingo Molnar @ 2018-02-16 9:38 UTC (permalink / raw)
To: Baoquan He, ebiederm
Cc: linux-kernel, ebiederm, tglx, x86, douly.fnst, joro, uobergfe, prarit
* Baoquan He <bhe@redhat.com> wrote:
> This is v5 post. Newly added patch 0002 includes the change
> related to KEXEC_JUMP path. Patch 0003 only includes the
> regression fix.
>
> A regression bug was introduced in below commit.
> commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
>
> It caused the action to fail that we try to restore boot irq mode
> in reboot and kexec/kdump. Details can be seen in patch 0003.
>
> Warning can always be seen during kdump kernel boot on qemu/kvm
> platform. Our customer even saw casual kdump kernel hang once in
> ~30 attempts during stress testing of kdump on KVM machine.
>
> v4->v5:
> Take out the change related to KEXEC_JUMP to a new patch 0002
> according to Eric's suggestion.
> Patch 0003 in this series only includes the regression fix.
>
> v3->v4:
> Eric pointed out that in patch 0002 the change related to
> KEXEC_JUMP is not right.
> Correct it.
>
> Add Fixes tag and Cc to stable.
Eric, are these patches looking good to you now?
Thanks,
Ingo
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-16 9:38 ` [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Ingo Molnar
@ 2018-02-16 10:24 ` Baoquan He
2018-02-17 10:46 ` Ingo Molnar
2018-02-16 19:16 ` Eric W. Biederman
1 sibling, 1 reply; 19+ messages in thread
From: Baoquan He @ 2018-02-16 10:24 UTC (permalink / raw)
To: ebiederm, Ingo Molnar
Cc: linux-kernel, tglx, x86, douly.fnst, joro, uobergfe, prarit
Hi Ingo, Eric,
On 02/16/18 at 10:38am, Ingo Molnar wrote:
>
> * Baoquan He <bhe@redhat.com> wrote:
>
> > This is v5 post. Newly added patch 0002 includes the change
> > related to KEXEC_JUMP path. Patch 0003 only includes the
> > regression fix.
> >
> > A regression bug was introduced in below commit.
> > commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
> >
> > It caused the action to fail that we try to restore boot irq mode
> > in reboot and kexec/kdump. Details can be seen in patch 0003.
> >
> > Warning can always be seen during kdump kernel boot on qemu/kvm
> > platform. Our customer even saw casual kdump kernel hang once in
> > ~30 attempts during stress testing of kdump on KVM machine.
> >
> > v4->v5:
> > Take out the change related to KEXEC_JUMP to a new patch 0002
> > according to Eric's suggestion.
> > Patch 0003 in this series only includes the regression fix.
> >
> > v3->v4:
> > Eric pointed out that in patch 0002 the change related to
> > KEXEC_JUMP is not right.
> > Correct it.
> >
> > Add Fixes tag and Cc to stable.
>
> Eric, are these patches looking good to you now?
Thanks for checking this!
I got warning message from kbuild test robot on previous v3 and v4. I
guess this v5 post also has the issue since the code is the same, just
patch format is changed.
Will report status after investigation.
Thanks
Baoquan
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-16 10:24 ` Baoquan He
@ 2018-02-17 10:46 ` Ingo Molnar
2018-02-17 11:28 ` Baoquan He
0 siblings, 1 reply; 19+ messages in thread
From: Ingo Molnar @ 2018-02-17 10:46 UTC (permalink / raw)
To: Baoquan He
Cc: ebiederm, linux-kernel, tglx, x86, douly.fnst, joro, uobergfe, prarit
* Baoquan He <bhe@redhat.com> wrote:
> Thanks for checking this!
>
> I got warning message from kbuild test robot on previous v3 and v4. I
> guess this v5 post also has the issue since the code is the same, just
> patch format is changed.
>
> Will report status after investigation.
Well, your patches didn't build when CONFIG_IO_APIC is disabled - I have fixed
this by the patch below.
Thanks,
Ingo
---
arch/x86/include/asm/io_apic.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 06fec4426458..fd20a2334885 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -233,6 +233,7 @@ static inline void io_apic_init_mappings(void) { }
static inline void setup_IO_APIC(void) { }
static inline void enable_IO_APIC(void) { }
+static inline void restore_boot_irq_mode(void) { }
#endif
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-17 10:46 ` Ingo Molnar
@ 2018-02-17 11:28 ` Baoquan He
0 siblings, 0 replies; 19+ messages in thread
From: Baoquan He @ 2018-02-17 11:28 UTC (permalink / raw)
To: Ingo Molnar
Cc: ebiederm, linux-kernel, tglx, x86, douly.fnst, joro, uobergfe, prarit
On 02/17/18 at 11:46am, Ingo Molnar wrote:
>
> * Baoquan He <bhe@redhat.com> wrote:
>
> > Thanks for checking this!
> >
> > I got warning message from kbuild test robot on previous v3 and v4. I
> > guess this v5 post also has the issue since the code is the same, just
> > patch format is changed.
> >
> > Will report status after investigation.
>
> Well, your patches didn't build when CONFIG_IO_APIC is disabled - I have fixed
> this by the patch below.
>
> Thanks,
>
> Ingo
>
> ---
> arch/x86/include/asm/io_apic.h | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
> index 06fec4426458..fd20a2334885 100644
> --- a/arch/x86/include/asm/io_apic.h
> +++ b/arch/x86/include/asm/io_apic.h
> @@ -233,6 +233,7 @@ static inline void io_apic_init_mappings(void) { }
>
> static inline void setup_IO_APIC(void) { }
> static inline void enable_IO_APIC(void) { }
> +static inline void restore_boot_irq_mode(void) { }
Ah, yes, thanks. Ack this patch.
Acked-by: Baoquan He <bhe@redhat.com>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-16 9:38 ` [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump Ingo Molnar
2018-02-16 10:24 ` Baoquan He
@ 2018-02-16 19:16 ` Eric W. Biederman
2018-02-17 10:17 ` Ingo Molnar
1 sibling, 1 reply; 19+ messages in thread
From: Eric W. Biederman @ 2018-02-16 19:16 UTC (permalink / raw)
To: Ingo Molnar
Cc: Baoquan He, linux-kernel, tglx, x86, douly.fnst, joro, uobergfe, prarit
Ingo Molnar <mingo@kernel.org> writes:
> * Baoquan He <bhe@redhat.com> wrote:
>
>> This is v5 post. Newly added patch 0002 includes the change
>> related to KEXEC_JUMP path. Patch 0003 only includes the
>> regression fix.
>>
>> A regression bug was introduced in below commit.
>> commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
>>
>> It caused the action to fail that we try to restore boot irq mode
>> in reboot and kexec/kdump. Details can be seen in patch 0003.
>>
>> Warning can always be seen during kdump kernel boot on qemu/kvm
>> platform. Our customer even saw casual kdump kernel hang once in
>> ~30 attempts during stress testing of kdump on KVM machine.
>>
>> v4->v5:
>> Take out the change related to KEXEC_JUMP to a new patch 0002
>> according to Eric's suggestion.
>> Patch 0003 in this series only includes the regression fix.
>>
>> v3->v4:
>> Eric pointed out that in patch 0002 the change related to
>> KEXEC_JUMP is not right.
>> Correct it.
>>
>> Add Fixes tag and Cc to stable.
>
> Eric, are these patches looking good to you now?
The result of applying the patches looks good.
Barring whatever fix to header files that kbuild seems to find necessary.
I wish patches 1 2 and 4 were all the same patch. That I think would
make reading the patches a bit easier, and make the backports clearer.
But at this point that is just me bike-shedding.
Reviewed-by: "Eric W. Biederman" <ebiederm@xmission.com>
Eric
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH v5 0/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
2018-02-16 19:16 ` Eric W. Biederman
@ 2018-02-17 10:17 ` Ingo Molnar
0 siblings, 0 replies; 19+ messages in thread
From: Ingo Molnar @ 2018-02-17 10:17 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Baoquan He, linux-kernel, tglx, x86, douly.fnst, joro, uobergfe, prarit
* Eric W. Biederman <ebiederm@xmission.com> wrote:
> Ingo Molnar <mingo@kernel.org> writes:
>
> > * Baoquan He <bhe@redhat.com> wrote:
> >
> >> This is v5 post. Newly added patch 0002 includes the change
> >> related to KEXEC_JUMP path. Patch 0003 only includes the
> >> regression fix.
> >>
> >> A regression bug was introduced in below commit.
> >> commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC")
> >>
> >> It caused the action to fail that we try to restore boot irq mode
> >> in reboot and kexec/kdump. Details can be seen in patch 0003.
> >>
> >> Warning can always be seen during kdump kernel boot on qemu/kvm
> >> platform. Our customer even saw casual kdump kernel hang once in
> >> ~30 attempts during stress testing of kdump on KVM machine.
> >>
> >> v4->v5:
> >> Take out the change related to KEXEC_JUMP to a new patch 0002
> >> according to Eric's suggestion.
> >> Patch 0003 in this series only includes the regression fix.
> >>
> >> v3->v4:
> >> Eric pointed out that in patch 0002 the change related to
> >> KEXEC_JUMP is not right.
> >> Correct it.
> >>
> >> Add Fixes tag and Cc to stable.
> >
> > Eric, are these patches looking good to you now?
>
> The result of applying the patches looks good.
> Barring whatever fix to header files that kbuild seems to find necessary.
>
> I wish patches 1 2 and 4 were all the same patch. That I think would
> make reading the patches a bit easier, and make the backports clearer.
> But at this point that is just me bike-shedding.
>
> Reviewed-by: "Eric W. Biederman" <ebiederm@xmission.com>
Thank you!
Ingo
^ permalink raw reply [flat|nested] 19+ messages in thread