* kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-17 7:22 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-17 7:22 UTC (permalink / raw)
To: Yu Chen
Cc: Juergen Gross, thomas.lendacky, Tony Luck, bhe, kexec, mingo,
Dan Williams, linux-kernel, Rui Zhang, ebiederm, Borislav Petkov,
torvalds, Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
[Modify the subject since this is a new problem, original io vector
issue has been fixed with one commit from Thomas]
Add more cc according to below old discussion:
https://lkml.org/lkml/2017/7/27/574
Tom, I'm not sure why you finally did not dynamically run wbinvd?
On 01/04/18 at 11:15am, Dave Young wrote:
> On 12/14/17 at 05:24pm, Dave Young wrote:
> > On 12/13/17 at 11:57pm, Yu Chen wrote:
> > > On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > > > Hi,
> > > >
> > > > Kexec reboot and kdump has broken on my laptop for long time with
> > > > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > > > https://patchwork.kernel.org/patch/10084289/
> > > >
> > > > But still can not get a successful reboot, it looked like graphic
> > > > issue, but after bisecting the kernel, I got below:
> > > >
> > > > [dyoung@dhcp-*-* linux]$ git bisect good
> > > > There are only 'skip'ped commits left to test.
> > > > The first bad commit could be any of:
> > > > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > > > 4900be83602b6be07366d3e69f756c1959f4169a
> > > > We cannot bisect more!
> > > >
> > > > These two commits can no be reverted because of code conflicts, thus
> > > > I reverted the whole series from Thomas (below commits), with those
> > > > x86/vector changes reverted, kexec reboot works fine.
> > > >
> > > > Could you help to take a look, any thoughts? I can do the test
> > > > if you have some debug patch to try.
> > > Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> > > what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> > > directly)
> >
> > Thanks for the reply.
> >
> > For kdump, yes, for kexec, I'm not sure.
> >
> > Here is some kexec kernel boot log:
> > http://people.redhat.com/~ruyang/misc/kexec-regression.txt
> >
> > Copy the lockup call trace here:
> > [ 23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0
> > [ 23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
> > 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
> > tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
> > ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
> > _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
> > ci pps_core snd soundcore rfkill video
> > [ 23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378
> > [ 23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42
> > ) 03/03/2017
> > [ 23.779312] RIP: 0010:poll_idle+0x2f/0x5f
> > [ 23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246
> > [ 23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
> > [ 23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
> > [ 23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
> > [ 23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
> > [ 23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
> > [ 23.779319] FS: 0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
> > 00000000000
> > [ 23.779320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
> > [ 23.779322] Call Trace:
> > [ 23.779328] cpuidle_enter_state+0x6a/0x2c0
> > [ 23.779333] do_idle+0x17b/0x1d0
> > [ 23.779335] cpu_startup_entry+0x6f/0x80
> > [ 23.779338] start_kernel+0x431/0x451
> > [ 23.779342] secondary_startup_64+0xa5/0xb0
> > [ 23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
> > 02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
> > a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
> >
>
> Followup this issue, seems another commit from Thomas partially fixed
> this, kexec/kdump boot up successfully for me, but kexec after kexec
> (2nd kexec reboot cycle) failed, kernel hung early
The above kexec reboot hang is another problem, so Thomas has fully
fixed previous report, thanks!
For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
then kexec works fine. like this:
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 832a6acd730f..6d7499730b27 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
disable_local_APIC();
mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
- for (;;) {
- /*
- * Use wbinvd followed by hlt to stop the processor. This
- * provides support for kexec on a processor that supports
- * SME. With kexec, going from SME inactive to SME active
- * requires clearing cache entries so that addresses without
- * the encryption bit set don't corrupt the same physical
- * address that has the encryption bit set when caches are
- * flushed. To achieve this a wbinvd is performed followed by
- * a hlt. Even if the processor is not in the kexec/SME
- * scenario this only adds a wbinvd to a halting processor.
- */
- asm volatile("wbinvd; hlt" : : : "memory");
- }
+ for (;;)
+ halt();
}
/*
But I have no idea why though, seeking for help and thoughts..
>
> commit bc976233a872c0f20f018fb1e89264a541584e25
> Author: Thomas Gleixner <tglx@linutronix.de>
> Date: Fri Dec 29 10:47:22 2017 +0100
>
> genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
>
> Thanks
> Dave
Thanks
Dave
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 7:22 ` Dave Young
@ 2018-01-17 15:06 ` Tom Lendacky
-1 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-17 15:06 UTC (permalink / raw)
To: Dave Young, Yu Chen
Cc: Thomas Gleixner, Juergen Gross, Tony Luck, Boris Ostrovsky,
Borislav Petkov, Rui Zhang, Arjan van de Ven, Dan Williams,
mingo, kexec, linux-kernel, ebiederm, bhe, torvalds
On 1/17/2018 1:22 AM, Dave Young wrote:
> [Modify the subject since this is a new problem, original io vector
> issue has been fixed with one commit from Thomas]
>
> Add more cc according to below old discussion:
> https://lkml.org/lkml/2017/7/27/574
>
> Tom, I'm not sure why you finally did not dynamically run wbinvd?
That discussion was aimed at the wbinvd that was being performed
in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
run based on a flag.
> On 01/04/18 at 11:15am, Dave Young wrote:
>> On 12/14/17 at 05:24pm, Dave Young wrote:
>>> On 12/13/17 at 11:57pm, Yu Chen wrote:
>>>> On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
>>>>> Hi,
>>>>>
>>>>> Kexec reboot and kdump has broken on my laptop for long time with
>>>>> 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
>>>>> https://patchwork.kernel.org/patch/10084289/
>>>>>
>>>>> But still can not get a successful reboot, it looked like graphic
>>>>> issue, but after bisecting the kernel, I got below:
>>>>>
>>>>> [dyoung@dhcp-*-* linux]$ git bisect good
>>>>> There are only 'skip'ped commits left to test.
>>>>> The first bad commit could be any of:
>>>>> 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
>>>>> 4900be83602b6be07366d3e69f756c1959f4169a
>>>>> We cannot bisect more!
>>>>>
>>>>> These two commits can no be reverted because of code conflicts, thus
>>>>> I reverted the whole series from Thomas (below commits), with those
>>>>> x86/vector changes reverted, kexec reboot works fine.
>>>>>
>>>>> Could you help to take a look, any thoughts? I can do the test
>>>>> if you have some debug patch to try.
>>>> Is it possible that the "second" kernel runs on non-zero CPU? If yes,
>>>> what if some irqs are only delivered to cpu0? (use cpumask_of(0)
>>>> directly)
>>>
>>> Thanks for the reply.
>>>
>>> For kdump, yes, for kexec, I'm not sure.
>>>
>>> Here is some kexec kernel boot log:
>>> http://people.redhat.com/~ruyang/misc/kexec-regression.txt
>>>
>>> Copy the lockup call trace here:
>>> [ 23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0
>>> [ 23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
>>> 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
>>> tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
>>> ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
>>> _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
>>> ci pps_core snd soundcore rfkill video
>>> [ 23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378
>>> [ 23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42
>>> ) 03/03/2017
>>> [ 23.779312] RIP: 0010:poll_idle+0x2f/0x5f
>>> [ 23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246
>>> [ 23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
>>> [ 23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
>>> [ 23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
>>> [ 23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
>>> [ 23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
>>> [ 23.779319] FS: 0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
>>> 00000000000
>>> [ 23.779320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [ 23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
>>> [ 23.779322] Call Trace:
>>> [ 23.779328] cpuidle_enter_state+0x6a/0x2c0
>>> [ 23.779333] do_idle+0x17b/0x1d0
>>> [ 23.779335] cpu_startup_entry+0x6f/0x80
>>> [ 23.779338] start_kernel+0x431/0x451
>>> [ 23.779342] secondary_startup_64+0xa5/0xb0
>>> [ 23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
>>> 02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
>>> a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
>>>
>>
>> Followup this issue, seems another commit from Thomas partially fixed
>> this, kexec/kdump boot up successfully for me, but kexec after kexec
>> (2nd kexec reboot cycle) failed, kernel hung early
>
> The above kexec reboot hang is another problem, so Thomas has fully
> fixed previous report, thanks!
>
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:
>
> diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
> index 832a6acd730f..6d7499730b27 100644
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
> disable_local_APIC();
> mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
>
> - for (;;) {
> - /*
> - * Use wbinvd followed by hlt to stop the processor. This
> - * provides support for kexec on a processor that supports
> - * SME. With kexec, going from SME inactive to SME active
> - * requires clearing cache entries so that addresses without
> - * the encryption bit set don't corrupt the same physical
> - * address that has the encryption bit set when caches are
> - * flushed. To achieve this a wbinvd is performed followed by
> - * a hlt. Even if the processor is not in the kexec/SME
> - * scenario this only adds a wbinvd to a halting processor.
> - */
> - asm volatile("wbinvd; hlt" : : : "memory");
> - }
> + for (;;)
> + halt();
> }
>
> /*
>
> But I have no idea why though, seeking for help and thoughts..
Yeah, I don't know why that works either.
Thanks,
Tom
>
>>
>> commit bc976233a872c0f20f018fb1e89264a541584e25
>> Author: Thomas Gleixner <tglx@linutronix.de>
>> Date: Fri Dec 29 10:47:22 2017 +0100
>>
>> genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
>>
>> Thanks
>> Dave
>
> Thanks
> Dave
>
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-17 15:06 ` Tom Lendacky
0 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-17 15:06 UTC (permalink / raw)
To: Dave Young, Yu Chen
Cc: Juergen Gross, Tony Luck, bhe, kexec, mingo, Dan Williams,
linux-kernel, Rui Zhang, ebiederm, Borislav Petkov, torvalds,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 1/17/2018 1:22 AM, Dave Young wrote:
> [Modify the subject since this is a new problem, original io vector
> issue has been fixed with one commit from Thomas]
>
> Add more cc according to below old discussion:
> https://lkml.org/lkml/2017/7/27/574
>
> Tom, I'm not sure why you finally did not dynamically run wbinvd?
That discussion was aimed at the wbinvd that was being performed
in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
run based on a flag.
> On 01/04/18 at 11:15am, Dave Young wrote:
>> On 12/14/17 at 05:24pm, Dave Young wrote:
>>> On 12/13/17 at 11:57pm, Yu Chen wrote:
>>>> On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
>>>>> Hi,
>>>>>
>>>>> Kexec reboot and kdump has broken on my laptop for long time with
>>>>> 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
>>>>> https://patchwork.kernel.org/patch/10084289/
>>>>>
>>>>> But still can not get a successful reboot, it looked like graphic
>>>>> issue, but after bisecting the kernel, I got below:
>>>>>
>>>>> [dyoung@dhcp-*-* linux]$ git bisect good
>>>>> There are only 'skip'ped commits left to test.
>>>>> The first bad commit could be any of:
>>>>> 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
>>>>> 4900be83602b6be07366d3e69f756c1959f4169a
>>>>> We cannot bisect more!
>>>>>
>>>>> These two commits can no be reverted because of code conflicts, thus
>>>>> I reverted the whole series from Thomas (below commits), with those
>>>>> x86/vector changes reverted, kexec reboot works fine.
>>>>>
>>>>> Could you help to take a look, any thoughts? I can do the test
>>>>> if you have some debug patch to try.
>>>> Is it possible that the "second" kernel runs on non-zero CPU? If yes,
>>>> what if some irqs are only delivered to cpu0? (use cpumask_of(0)
>>>> directly)
>>>
>>> Thanks for the reply.
>>>
>>> For kdump, yes, for kexec, I'm not sure.
>>>
>>> Here is some kexec kernel boot log:
>>> http://people.redhat.com/~ruyang/misc/kexec-regression.txt
>>>
>>> Copy the lockup call trace here:
>>> [ 23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0
>>> [ 23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
>>> 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
>>> tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
>>> ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
>>> _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
>>> ci pps_core snd soundcore rfkill video
>>> [ 23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378
>>> [ 23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42
>>> ) 03/03/2017
>>> [ 23.779312] RIP: 0010:poll_idle+0x2f/0x5f
>>> [ 23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246
>>> [ 23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
>>> [ 23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
>>> [ 23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
>>> [ 23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
>>> [ 23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
>>> [ 23.779319] FS: 0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
>>> 00000000000
>>> [ 23.779320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [ 23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
>>> [ 23.779322] Call Trace:
>>> [ 23.779328] cpuidle_enter_state+0x6a/0x2c0
>>> [ 23.779333] do_idle+0x17b/0x1d0
>>> [ 23.779335] cpu_startup_entry+0x6f/0x80
>>> [ 23.779338] start_kernel+0x431/0x451
>>> [ 23.779342] secondary_startup_64+0xa5/0xb0
>>> [ 23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
>>> 02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
>>> a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
>>>
>>
>> Followup this issue, seems another commit from Thomas partially fixed
>> this, kexec/kdump boot up successfully for me, but kexec after kexec
>> (2nd kexec reboot cycle) failed, kernel hung early
>
> The above kexec reboot hang is another problem, so Thomas has fully
> fixed previous report, thanks!
>
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:
>
> diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
> index 832a6acd730f..6d7499730b27 100644
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
> disable_local_APIC();
> mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
>
> - for (;;) {
> - /*
> - * Use wbinvd followed by hlt to stop the processor. This
> - * provides support for kexec on a processor that supports
> - * SME. With kexec, going from SME inactive to SME active
> - * requires clearing cache entries so that addresses without
> - * the encryption bit set don't corrupt the same physical
> - * address that has the encryption bit set when caches are
> - * flushed. To achieve this a wbinvd is performed followed by
> - * a hlt. Even if the processor is not in the kexec/SME
> - * scenario this only adds a wbinvd to a halting processor.
> - */
> - asm volatile("wbinvd; hlt" : : : "memory");
> - }
> + for (;;)
> + halt();
> }
>
> /*
>
> But I have no idea why though, seeking for help and thoughts..
Yeah, I don't know why that works either.
Thanks,
Tom
>
>>
>> commit bc976233a872c0f20f018fb1e89264a541584e25
>> Author: Thomas Gleixner <tglx@linutronix.de>
>> Date: Fri Dec 29 10:47:22 2017 +0100
>>
>> genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
>>
>> Thanks
>> Dave
>
> Thanks
> Dave
>
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 15:06 ` Tom Lendacky
@ 2018-01-18 1:50 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 1:50 UTC (permalink / raw)
To: Tom Lendacky
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, mingo, kexec, linux-kernel, ebiederm, bhe,
torvalds
On 01/17/18 at 09:06am, Tom Lendacky wrote:
> On 1/17/2018 1:22 AM, Dave Young wrote:
> > [Modify the subject since this is a new problem, original io vector
> > issue has been fixed with one commit from Thomas]
> >
> > Add more cc according to below old discussion:
> > https://lkml.org/lkml/2017/7/27/574
> >
> > Tom, I'm not sure why you finally did not dynamically run wbinvd?
>
> That discussion was aimed at the wbinvd that was being performed
> in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
> run based on a flag.
Oh, I did not notice that, sorry for referring a wrong discussion.
[snip]
Thanks
Dave
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 1:50 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 1:50 UTC (permalink / raw)
To: Tom Lendacky
Cc: Juergen Gross, Tony Luck, Yu Chen, bhe, kexec, mingo,
Dan Williams, linux-kernel, Rui Zhang, ebiederm, Borislav Petkov,
torvalds, Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 01/17/18 at 09:06am, Tom Lendacky wrote:
> On 1/17/2018 1:22 AM, Dave Young wrote:
> > [Modify the subject since this is a new problem, original io vector
> > issue has been fixed with one commit from Thomas]
> >
> > Add more cc according to below old discussion:
> > https://lkml.org/lkml/2017/7/27/574
> >
> > Tom, I'm not sure why you finally did not dynamically run wbinvd?
>
> That discussion was aimed at the wbinvd that was being performed
> in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
> run based on a flag.
Oh, I did not notice that, sorry for referring a wrong discussion.
[snip]
Thanks
Dave
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 7:22 ` Dave Young
@ 2018-01-17 19:42 ` Linus Torvalds
-1 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-17 19:42 UTC (permalink / raw)
To: Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:
Honestly, I think we should apply that patch regardless.
Using 'wbinvd' should not be some "just because of random reasons".
There are CPU's with errata on wbinvd, and the thing in general is
slow and nasty.
Doing the wbinvd in a loop sounds even stranger.
If we're only doing it because of some SME issue, why isn't it
dependent on SME? And why is it inside that loop at all?
Anyway, does it work for you if you just do the wbinvd() once, outside
the loop? Admittedly the loop shouldn't actually loop (hlt with
interrupts disabled), but who the hell knows.. Some of the errata
around SME have been about machine check exceptions or something.
See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
play_dead loop") for another example where wbinvd was inside a loop
and apparently caused some odd issues.
Linus
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-17 19:42 ` Linus Torvalds
0 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-17 19:42 UTC (permalink / raw)
To: Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:
Honestly, I think we should apply that patch regardless.
Using 'wbinvd' should not be some "just because of random reasons".
There are CPU's with errata on wbinvd, and the thing in general is
slow and nasty.
Doing the wbinvd in a loop sounds even stranger.
If we're only doing it because of some SME issue, why isn't it
dependent on SME? And why is it inside that loop at all?
Anyway, does it work for you if you just do the wbinvd() once, outside
the loop? Admittedly the loop shouldn't actually loop (hlt with
interrupts disabled), but who the hell knows.. Some of the errata
around SME have been about machine check exceptions or something.
See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
play_dead loop") for another example where wbinvd was inside a loop
and apparently caused some odd issues.
Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 19:42 ` Linus Torvalds
@ 2018-01-17 20:01 ` Tom Lendacky
-1 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-17 20:01 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Baoquan He
On 1/17/2018 1:42 PM, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>
>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>> then kexec works fine. like this:
>
> Honestly, I think we should apply that patch regardless.
>
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
>
> Doing the wbinvd in a loop sounds even stranger.
>
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?
My original patches did check for X86_FEATURE_SME and only do the
wbinvd if SME was supported (although still in the loop). The general
consensus was to just do the wbinvd no matter what and so it is as it is
today.
It can probably be outside of the loop. The issue I was seeing was
memory corruption from the stack when using halt() with paravirt ops
enabled. So a native_halt() should be used.
>
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.
I think that should work as long as it's a native_wbinvd() call and it
can also be conditional on boot_cpu_has(X86_FEATURE_SME).
I'll do some testing.
Thanks,
Tom
>
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
>
> Linus
>
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-17 20:01 ` Tom Lendacky
0 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-17 20:01 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Juergen Gross, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 1/17/2018 1:42 PM, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>
>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>> then kexec works fine. like this:
>
> Honestly, I think we should apply that patch regardless.
>
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
>
> Doing the wbinvd in a loop sounds even stranger.
>
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?
My original patches did check for X86_FEATURE_SME and only do the
wbinvd if SME was supported (although still in the loop). The general
consensus was to just do the wbinvd no matter what and so it is as it is
today.
It can probably be outside of the loop. The issue I was seeing was
memory corruption from the stack when using halt() with paravirt ops
enabled. So a native_halt() should be used.
>
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.
I think that should work as long as it's a native_wbinvd() call and it
can also be conditional on boot_cpu_has(X86_FEATURE_SME).
I'll do some testing.
Thanks,
Tom
>
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
>
> Linus
>
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 20:01 ` Tom Lendacky
@ 2018-01-17 22:53 ` Tom Lendacky
-1 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-17 22:53 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Baoquan He
On 1/17/2018 2:01 PM, Tom Lendacky wrote:
> On 1/17/2018 1:42 PM, Linus Torvalds wrote:
>> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>>> then kexec works fine. like this:
>>
>> Honestly, I think we should apply that patch regardless.
>>
>> Using 'wbinvd' should not be some "just because of random reasons".
>> There are CPU's with errata on wbinvd, and the thing in general is
>> slow and nasty.
>>
>> Doing the wbinvd in a loop sounds even stranger.
>>
>> If we're only doing it because of some SME issue, why isn't it
>> dependent on SME? And why is it inside that loop at all?
>
> My original patches did check for X86_FEATURE_SME and only do the
> wbinvd if SME was supported (although still in the loop). The general
> consensus was to just do the wbinvd no matter what and so it is as it is
> today.
>
> It can probably be outside of the loop. The issue I was seeing was
> memory corruption from the stack when using halt() with paravirt ops
> enabled. So a native_halt() should be used.
>
>>
>> Anyway, does it work for you if you just do the wbinvd() once, outside
>> the loop? Admittedly the loop shouldn't actually loop (hlt with
>> interrupts disabled), but who the hell knows.. Some of the errata
>> around SME have been about machine check exceptions or something.
>
> I think that should work as long as it's a native_wbinvd() call and it
> can also be conditional on boot_cpu_has(X86_FEATURE_SME).
>
> I'll do some testing.
Looks like everything is good with the suggested changes. Patch to follow
shortly.
Thanks,
Tom
>
> Thanks,
> Tom
>
>>
>> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
>> play_dead loop") for another example where wbinvd was inside a loop
>> and apparently caused some odd issues.
>>
>> Linus
>>
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-17 22:53 ` Tom Lendacky
0 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-17 22:53 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Juergen Gross, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 1/17/2018 2:01 PM, Tom Lendacky wrote:
> On 1/17/2018 1:42 PM, Linus Torvalds wrote:
>> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>>> then kexec works fine. like this:
>>
>> Honestly, I think we should apply that patch regardless.
>>
>> Using 'wbinvd' should not be some "just because of random reasons".
>> There are CPU's with errata on wbinvd, and the thing in general is
>> slow and nasty.
>>
>> Doing the wbinvd in a loop sounds even stranger.
>>
>> If we're only doing it because of some SME issue, why isn't it
>> dependent on SME? And why is it inside that loop at all?
>
> My original patches did check for X86_FEATURE_SME and only do the
> wbinvd if SME was supported (although still in the loop). The general
> consensus was to just do the wbinvd no matter what and so it is as it is
> today.
>
> It can probably be outside of the loop. The issue I was seeing was
> memory corruption from the stack when using halt() with paravirt ops
> enabled. So a native_halt() should be used.
>
>>
>> Anyway, does it work for you if you just do the wbinvd() once, outside
>> the loop? Admittedly the loop shouldn't actually loop (hlt with
>> interrupts disabled), but who the hell knows.. Some of the errata
>> around SME have been about machine check exceptions or something.
>
> I think that should work as long as it's a native_wbinvd() call and it
> can also be conditional on boot_cpu_has(X86_FEATURE_SME).
>
> I'll do some testing.
Looks like everything is good with the suggested changes. Patch to follow
shortly.
Thanks,
Tom
>
> Thanks,
> Tom
>
>>
>> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
>> play_dead loop") for another example where wbinvd was inside a loop
>> and apparently caused some odd issues.
>>
>> Linus
>>
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 19:42 ` Linus Torvalds
@ 2018-01-17 20:10 ` Linus Torvalds
-1 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-17 20:10 UTC (permalink / raw)
To: Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On Wed, Jan 17, 2018 at 11:42 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> [ .. ] Some of the errata
> around SME have been about machine check exceptions or something.
That should be "some of the errata around wbinvd". I have no idea if
there have been SME issues.
That said, the really bad old wbinvd bug (which hung the system iirc)
was for some old PPro CPU's. Googling around, the errata I find seem
either irrelevant (eip corruption in 16-bit mode) or fairly mild (odd
behavior wrt parity errors).
So I have this memory of wbinvd having problems, but that memory does
seem to be largely historical. But Dave Young clearly sees *something*
odd going on.
Linus
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-17 20:10 ` Linus Torvalds
0 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-17 20:10 UTC (permalink / raw)
To: Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On Wed, Jan 17, 2018 at 11:42 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> [ .. ] Some of the errata
> around SME have been about machine check exceptions or something.
That should be "some of the errata around wbinvd". I have no idea if
there have been SME issues.
That said, the really bad old wbinvd bug (which hung the system iirc)
was for some old PPro CPU's. Googling around, the errata I find seem
either irrelevant (eip corruption in 16-bit mode) or fairly mild (odd
behavior wrt parity errors).
So I have this memory of wbinvd having problems, but that memory does
seem to be largely historical. But Dave Young clearly sees *something*
odd going on.
Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-17 19:42 ` Linus Torvalds
@ 2018-01-18 1:47 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 1:47 UTC (permalink / raw)
To: Linus Torvalds
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On 01/17/18 at 11:42am, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> > then kexec works fine. like this:
>
> Honestly, I think we should apply that patch regardless.
>
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
>
> Doing the wbinvd in a loop sounds even stranger.
>
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?
>
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.
It does not work with just once wbinvd(), and it only works with
removing the wbinvd() for me. Tom's new post works for me as well
since my cpu is an Intel i5-4200U.
>
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
>
> Linus
Thanks for the help! Finally I get my laptop kexec & kdump works again
after solving these issues.
Thanks
Dave
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 1:47 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 1:47 UTC (permalink / raw)
To: Linus Torvalds
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 01/17/18 at 11:42am, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> > then kexec works fine. like this:
>
> Honestly, I think we should apply that patch regardless.
>
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
>
> Doing the wbinvd in a loop sounds even stranger.
>
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?
>
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.
It does not work with just once wbinvd(), and it only works with
removing the wbinvd() for me. Tom's new post works for me as well
since my cpu is an Intel i5-4200U.
>
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
>
> Linus
Thanks for the help! Finally I get my laptop kexec & kdump works again
after solving these issues.
Thanks
Dave
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 1:47 ` Dave Young
@ 2018-01-18 2:14 ` Linus Torvalds
-1 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-18 2:14 UTC (permalink / raw)
To: Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>
> It does not work with just once wbinvd(), and it only works with
> removing the wbinvd() for me. Tom's new post works for me as well
> since my cpu is an Intel i5-4200U.
Intriguing.
It's not like the wbinvd really should be that much of a deal.
I think Tom's patch is fine and should be applied, but it does worry
me a bit that even a single wbinvd makes that much of a difference for
you. There is very little logical reason I can think of that a wbinvd
should make any difference what-so-ever on an i5-4200U.
I wonder if you have some system issues, and wbinvd just happens to
trigger them. But I think we do wbinvd before a suspend-to-RAM too
(it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
Would you mind experimenting a bit with that wbinvd?
In particular, what happens if you enable it (so it's not hidden by
the SME check), but you move it up to before interrupts are disabled?
I'm wondering if there is some issue with MCE generation and wbinvd
and whatever, and doing it when the CPU is down and interrupts are
disabled causes some system issue..
Does anybody have any other ideas?
Linus
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 2:14 ` Linus Torvalds
0 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-18 2:14 UTC (permalink / raw)
To: Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>
> It does not work with just once wbinvd(), and it only works with
> removing the wbinvd() for me. Tom's new post works for me as well
> since my cpu is an Intel i5-4200U.
Intriguing.
It's not like the wbinvd really should be that much of a deal.
I think Tom's patch is fine and should be applied, but it does worry
me a bit that even a single wbinvd makes that much of a difference for
you. There is very little logical reason I can think of that a wbinvd
should make any difference what-so-ever on an i5-4200U.
I wonder if you have some system issues, and wbinvd just happens to
trigger them. But I think we do wbinvd before a suspend-to-RAM too
(it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
Would you mind experimenting a bit with that wbinvd?
In particular, what happens if you enable it (so it's not hidden by
the SME check), but you move it up to before interrupts are disabled?
I'm wondering if there is some issue with MCE generation and wbinvd
and whatever, and doing it when the CPU is down and interrupts are
disabled causes some system issue..
Does anybody have any other ideas?
Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:14 ` Linus Torvalds
@ 2018-01-18 2:29 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 2:29 UTC (permalink / raw)
To: Linus Torvalds
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me. Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
>
> Intriguing.
>
> It's not like the wbinvd really should be that much of a deal.
>
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
>
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>
> Would you mind experimenting a bit with that wbinvd?
>
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?
Will play with it more. Actually I found the hang seems happens
in code of arch/x86/kernel/relocate_kernel_64.S, there is another
wbinvd there as well.
>
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
>
> Does anybody have any other ideas?
>
> Linus
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 2:29 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 2:29 UTC (permalink / raw)
To: Linus Torvalds
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me. Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
>
> Intriguing.
>
> It's not like the wbinvd really should be that much of a deal.
>
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
>
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>
> Would you mind experimenting a bit with that wbinvd?
>
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?
Will play with it more. Actually I found the hang seems happens
in code of arch/x86/kernel/relocate_kernel_64.S, there is another
wbinvd there as well.
>
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
>
> Does anybody have any other ideas?
>
> Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:29 ` Dave Young
@ 2018-01-18 5:14 ` Tom Lendacky
-1 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-18 5:14 UTC (permalink / raw)
To: Dave Young, Linus Torvalds
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Baoquan He
On 1/17/2018 8:29 PM, Dave Young wrote:
> On 01/17/18 at 06:14pm, Linus Torvalds wrote:
>> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> It does not work with just once wbinvd(), and it only works with
>>> removing the wbinvd() for me. Tom's new post works for me as well
>>> since my cpu is an Intel i5-4200U.
>>
>> Intriguing.
>>
>> It's not like the wbinvd really should be that much of a deal.
>>
>> I think Tom's patch is fine and should be applied, but it does worry
>> me a bit that even a single wbinvd makes that much of a difference for
>> you. There is very little logical reason I can think of that a wbinvd
>> should make any difference what-so-ever on an i5-4200U.
>>
>> I wonder if you have some system issues, and wbinvd just happens to
>> trigger them. But I think we do wbinvd before a suspend-to-RAM too
>> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
>> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>>
>> Would you mind experimenting a bit with that wbinvd?
>>
>> In particular, what happens if you enable it (so it's not hidden by
>> the SME check), but you move it up to before interrupts are disabled?
>
> Will play with it more. Actually I found the hang seems happens
> in code of arch/x86/kernel/relocate_kernel_64.S, there is another
> wbinvd there as well.
The wbinvd in arch/x86/kernel/relocate_kernel_64.S is only performed if
SME is active, so that one won't be executed on an Intel chip.
Thanks,
Tom
>
>>
>> I'm wondering if there is some issue with MCE generation and wbinvd
>> and whatever, and doing it when the CPU is down and interrupts are
>> disabled causes some system issue..
>>
>> Does anybody have any other ideas?
>>
>> Linus
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 5:14 ` Tom Lendacky
0 siblings, 0 replies; 46+ messages in thread
From: Tom Lendacky @ 2018-01-18 5:14 UTC (permalink / raw)
To: Dave Young, Linus Torvalds
Cc: Juergen Gross, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 1/17/2018 8:29 PM, Dave Young wrote:
> On 01/17/18 at 06:14pm, Linus Torvalds wrote:
>> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> It does not work with just once wbinvd(), and it only works with
>>> removing the wbinvd() for me. Tom's new post works for me as well
>>> since my cpu is an Intel i5-4200U.
>>
>> Intriguing.
>>
>> It's not like the wbinvd really should be that much of a deal.
>>
>> I think Tom's patch is fine and should be applied, but it does worry
>> me a bit that even a single wbinvd makes that much of a difference for
>> you. There is very little logical reason I can think of that a wbinvd
>> should make any difference what-so-ever on an i5-4200U.
>>
>> I wonder if you have some system issues, and wbinvd just happens to
>> trigger them. But I think we do wbinvd before a suspend-to-RAM too
>> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
>> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>>
>> Would you mind experimenting a bit with that wbinvd?
>>
>> In particular, what happens if you enable it (so it's not hidden by
>> the SME check), but you move it up to before interrupts are disabled?
>
> Will play with it more. Actually I found the hang seems happens
> in code of arch/x86/kernel/relocate_kernel_64.S, there is another
> wbinvd there as well.
The wbinvd in arch/x86/kernel/relocate_kernel_64.S is only performed if
SME is active, so that one won't be executed on an Intel chip.
Thanks,
Tom
>
>>
>> I'm wondering if there is some issue with MCE generation and wbinvd
>> and whatever, and doing it when the CPU is down and interrupts are
>> disabled causes some system issue..
>>
>> Does anybody have any other ideas?
>>
>> Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:14 ` Linus Torvalds
@ 2018-01-18 2:47 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 2:47 UTC (permalink / raw)
To: Linus Torvalds
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me. Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
>
> Intriguing.
>
> It's not like the wbinvd really should be that much of a deal.
>
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
>
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>
> Would you mind experimenting a bit with that wbinvd?
>
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?
Did several quick tests, probably need more tests, but till now the
results are:
void stop_this_cpu(void *dummy)
{
=====> add wbinvd here: kexec works
local_irq_disable();
=====> add wbinvd here: kexec works
/*
* Remove this CPU:
*/
set_cpu_online(smp_processor_id(), false);
=====> add wbinvd here: kexec does not work
disable_local_APIC();
mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
[snip]
So it seems that it will not work after cpu offined..
>
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
>
> Does anybody have any other ideas?
>
> Linus
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 2:47 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 2:47 UTC (permalink / raw)
To: Linus Torvalds
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me. Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
>
> Intriguing.
>
> It's not like the wbinvd really should be that much of a deal.
>
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
>
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>
> Would you mind experimenting a bit with that wbinvd?
>
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?
Did several quick tests, probably need more tests, but till now the
results are:
void stop_this_cpu(void *dummy)
{
=====> add wbinvd here: kexec works
local_irq_disable();
=====> add wbinvd here: kexec works
/*
* Remove this CPU:
*/
set_cpu_online(smp_processor_id(), false);
=====> add wbinvd here: kexec does not work
disable_local_APIC();
mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
[snip]
So it seems that it will not work after cpu offined..
>
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
>
> Does anybody have any other ideas?
>
> Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:47 ` Dave Young
@ 2018-01-18 2:56 ` Linus Torvalds
-1 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-18 2:56 UTC (permalink / raw)
To: Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Arjan van de Ven,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On Wed, Jan 17, 2018 at 6:47 PM, Dave Young <dyoung@redhat.com> wrote:
> Did several quick tests, probably need more tests, but till now the
> results are:
>
> void stop_this_cpu(void *dummy)
> {
> =====> add wbinvd here: kexec works
> local_irq_disable();
> =====> add wbinvd here: kexec works
> /*
> * Remove this CPU:
> */
> set_cpu_online(smp_processor_id(), false);
> =====> add wbinvd here: kexec does not work
Funky.
> So it seems that it will not work after cpu offined..
Well, that set_cpu_online() call really just clears a bit in our
'__cpu_online_mask' CPU mask. It doesn't really do anything to the
*hardware*.
But I do wonder if the wbinvd causes an SMI or something on your
system. I _think_ wbinvd causes some external pin to be wiggled just
to tell possible external cache hardware to flush too, and on a system
level that could be tied to some random thing.
And then if we get an SMI/NMI when we've marked the system offline,
maybe we do something odd.
Very odd. But maybe this makes somebody go "Duh, that's because of xyz.."
Linus
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 2:56 ` Linus Torvalds
0 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-18 2:56 UTC (permalink / raw)
To: Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On Wed, Jan 17, 2018 at 6:47 PM, Dave Young <dyoung@redhat.com> wrote:
> Did several quick tests, probably need more tests, but till now the
> results are:
>
> void stop_this_cpu(void *dummy)
> {
> =====> add wbinvd here: kexec works
> local_irq_disable();
> =====> add wbinvd here: kexec works
> /*
> * Remove this CPU:
> */
> set_cpu_online(smp_processor_id(), false);
> =====> add wbinvd here: kexec does not work
Funky.
> So it seems that it will not work after cpu offined..
Well, that set_cpu_online() call really just clears a bit in our
'__cpu_online_mask' CPU mask. It doesn't really do anything to the
*hardware*.
But I do wonder if the wbinvd causes an SMI or something on your
system. I _think_ wbinvd causes some external pin to be wiggled just
to tell possible external cache hardware to flush too, and on a system
level that could be tied to some random thing.
And then if we get an SMI/NMI when we've marked the system offline,
maybe we do something odd.
Very odd. But maybe this makes somebody go "Duh, that's because of xyz.."
Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:14 ` Linus Torvalds
@ 2018-01-18 2:53 ` Arjan van de Ven
-1 siblings, 0 replies; 46+ messages in thread
From: Arjan van de Ven @ 2018-01-18 2:53 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Dan Williams,
Ingo Molnar, Kexec Mailing List, Linux Kernel Mailing List,
ebiederm, Tom Lendacky, Baoquan He
>
> Does anybody have any other ideas?
wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.
I can only imagine a machine check case where a write back to a bad cell causes some parity error
or something... but it's odd that no other machine checks are reported?
(can the user check for this please)
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 2:53 ` Arjan van de Ven
0 siblings, 0 replies; 46+ messages in thread
From: Arjan van de Ven @ 2018-01-18 2:53 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Boris Ostrovsky
>
> Does anybody have any other ideas?
wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.
I can only imagine a machine check case where a write back to a bad cell causes some parity error
or something... but it's odd that no other machine checks are reported?
(can the user check for this please)
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:53 ` Arjan van de Ven
@ 2018-01-18 2:57 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 2:57 UTC (permalink / raw)
To: Arjan van de Ven
Cc: Linus Torvalds, Yu Chen, Thomas Gleixner, Juergen Gross,
Tony Luck, Boris Ostrovsky, Borislav Petkov, Rui Zhang,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On 01/17/18 at 06:53pm, Arjan van de Ven wrote:
> >
> > Does anybody have any other ideas?
>
> wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
> and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.
>
> I can only imagine a machine check case where a write back to a bad cell causes some parity error
> or something... but it's odd that no other machine checks are reported?
> (can the user check for this please)
Could you say more about how to check this? My .config disabled
CONFIG_X86_MCE, should this be enabled?
Thanks
Dave
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 2:57 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 2:57 UTC (permalink / raw)
To: Arjan van de Ven
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Linus Torvalds, Boris Ostrovsky
On 01/17/18 at 06:53pm, Arjan van de Ven wrote:
> >
> > Does anybody have any other ideas?
>
> wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
> and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.
>
> I can only imagine a machine check case where a write back to a bad cell causes some parity error
> or something... but it's odd that no other machine checks are reported?
> (can the user check for this please)
Could you say more about how to check this? My .config disabled
CONFIG_X86_MCE, should this be enabled?
Thanks
Dave
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:57 ` Dave Young
@ 2018-01-18 3:00 ` Linus Torvalds
-1 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-18 3:00 UTC (permalink / raw)
To: Dave Young
Cc: Arjan van de Ven, Yu Chen, Thomas Gleixner, Juergen Gross,
Tony Luck, Boris Ostrovsky, Borislav Petkov, Rui Zhang,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
>
> Could you say more about how to check this? My .config disabled
> CONFIG_X86_MCE, should this be enabled?
By all means, try it.
Some pending machine check exception that we have *not* reacted to,
and that is pending around kexec boot could very possibly be relevant.
I forget your exact symptoms. Was it that the _first_ kexec works, but
the second one hangs? Or was that the other unrelated issue? Or have I
confused this with some other report entirely?
Linus
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 3:00 ` Linus Torvalds
0 siblings, 0 replies; 46+ messages in thread
From: Linus Torvalds @ 2018-01-18 3:00 UTC (permalink / raw)
To: Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
>
> Could you say more about how to check this? My .config disabled
> CONFIG_X86_MCE, should this be enabled?
By all means, try it.
Some pending machine check exception that we have *not* reacted to,
and that is pending around kexec boot could very possibly be relevant.
I forget your exact symptoms. Was it that the _first_ kexec works, but
the second one hangs? Or was that the other unrelated issue? Or have I
confused this with some other report entirely?
Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 3:00 ` Linus Torvalds
@ 2018-01-18 3:04 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 3:04 UTC (permalink / raw)
To: Linus Torvalds
Cc: Arjan van de Ven, Yu Chen, Thomas Gleixner, Juergen Gross,
Tony Luck, Boris Ostrovsky, Borislav Petkov, Rui Zhang,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > Could you say more about how to check this? My .config disabled
> > CONFIG_X86_MCE, should this be enabled?
>
> By all means, try it.
>
> Some pending machine check exception that we have *not* reacted to,
> and that is pending around kexec boot could very possibly be relevant.
>
> I forget your exact symptoms. Was it that the _first_ kexec works, but
> the second one hangs? Or was that the other unrelated issue? Or have I
> confused this with some other report entirely?
Yes, 1st kexec works, 2nd hangs.
Rebuilding with MCE enabled now.
Thanks
Dave
>
> Linus
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 3:04 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 3:04 UTC (permalink / raw)
To: Linus Torvalds
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > Could you say more about how to check this? My .config disabled
> > CONFIG_X86_MCE, should this be enabled?
>
> By all means, try it.
>
> Some pending machine check exception that we have *not* reacted to,
> and that is pending around kexec boot could very possibly be relevant.
>
> I forget your exact symptoms. Was it that the _first_ kexec works, but
> the second one hangs? Or was that the other unrelated issue? Or have I
> confused this with some other report entirely?
Yes, 1st kexec works, 2nd hangs.
Rebuilding with MCE enabled now.
Thanks
Dave
>
> Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 3:04 ` Dave Young
@ 2018-01-18 3:31 ` Dave Young
-1 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 3:31 UTC (permalink / raw)
To: Linus Torvalds
Cc: Arjan van de Ven, Yu Chen, Thomas Gleixner, Juergen Gross,
Tony Luck, Boris Ostrovsky, Borislav Petkov, Rui Zhang,
Dan Williams, Ingo Molnar, Kexec Mailing List,
Linux Kernel Mailing List, ebiederm, Tom Lendacky, Baoquan He
On 01/18/18 at 11:04am, Dave Young wrote:
> On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> > On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> > >
> > > Could you say more about how to check this? My .config disabled
> > > CONFIG_X86_MCE, should this be enabled?
> >
> > By all means, try it.
> >
> > Some pending machine check exception that we have *not* reacted to,
> > and that is pending around kexec boot could very possibly be relevant.
> >
> > I forget your exact symptoms. Was it that the _first_ kexec works, but
> > the second one hangs? Or was that the other unrelated issue? Or have I
> > confused this with some other report entirely?
>
> Yes, 1st kexec works, 2nd hangs.
>
> Rebuilding with MCE enabled now.
dmesg only shows "mce: CPU supports 7 MCE banks"
Nothing special happens with MCE enabled , anyway thank all for
the hints, I will continue to think about this and maybe try more experiments.
>
> Thanks
> Dave
> >
> > Linus
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 3:31 ` Dave Young
0 siblings, 0 replies; 46+ messages in thread
From: Dave Young @ 2018-01-18 3:31 UTC (permalink / raw)
To: Linus Torvalds
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Arjan van de Ven, Boris Ostrovsky
On 01/18/18 at 11:04am, Dave Young wrote:
> On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> > On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> > >
> > > Could you say more about how to check this? My .config disabled
> > > CONFIG_X86_MCE, should this be enabled?
> >
> > By all means, try it.
> >
> > Some pending machine check exception that we have *not* reacted to,
> > and that is pending around kexec boot could very possibly be relevant.
> >
> > I forget your exact symptoms. Was it that the _first_ kexec works, but
> > the second one hangs? Or was that the other unrelated issue? Or have I
> > confused this with some other report entirely?
>
> Yes, 1st kexec works, 2nd hangs.
>
> Rebuilding with MCE enabled now.
dmesg only shows "mce: CPU supports 7 MCE banks"
Nothing special happens with MCE enabled , anyway thank all for
the hints, I will continue to think about this and maybe try more experiments.
>
> Thanks
> Dave
> >
> > Linus
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
2018-01-18 2:14 ` Linus Torvalds
@ 2018-01-18 3:00 ` Arjan van de Ven
-1 siblings, 0 replies; 46+ messages in thread
From: Arjan van de Ven @ 2018-01-18 3:00 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Yu Chen, Thomas Gleixner, Juergen Gross, Tony Luck,
Boris Ostrovsky, Borislav Petkov, Rui Zhang, Dan Williams,
Ingo Molnar, Kexec Mailing List, Linux Kernel Mailing List,
ebiederm, Tom Lendacky, Baoquan He
> Does anybody have any other ideas?
the only other weird case that comes to mind; what happens if there's a line dirty in the caches,
but the memory is now mapped uncached. (Which could happen if kexec does muck with MTRRs, CR0 or other similar
things in weird ways)... not sure what happens in CPU, a machine check for cache inclusion violations
is not beyond the imagination and might be lethal
this would explain a kexec specific angle versus general normal (but rare) use of wbinvd.
other weird case could be cached mmio (not common, but some gpus and the like can do it)
with iommu/VT-D in the middle, and during kexec VT-D shutting
down the iommu before the wbinvd. This would be... highly odd... but this report already is in highly odd space.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: kexec reboot fails with extra wbinvd introduced for AME SME
@ 2018-01-18 3:00 ` Arjan van de Ven
0 siblings, 0 replies; 46+ messages in thread
From: Arjan van de Ven @ 2018-01-18 3:00 UTC (permalink / raw)
To: Linus Torvalds, Dave Young
Cc: Juergen Gross, Tom Lendacky, Tony Luck, Yu Chen, Baoquan He,
Kexec Mailing List, Ingo Molnar, Dan Williams,
Linux Kernel Mailing List, Rui Zhang, ebiederm, Borislav Petkov,
Thomas Gleixner, Boris Ostrovsky
> Does anybody have any other ideas?
the only other weird case that comes to mind; what happens if there's a line dirty in the caches,
but the memory is now mapped uncached. (Which could happen if kexec does muck with MTRRs, CR0 or other similar
things in weird ways)... not sure what happens in CPU, a machine check for cache inclusion violations
is not beyond the imagination and might be lethal
this would explain a kexec specific angle versus general normal (but rare) use of wbinvd.
other weird case could be cached mmio (not common, but some gpus and the like can do it)
with iommu/VT-D in the middle, and during kexec VT-D shutting
down the iommu before the wbinvd. This would be... highly odd... but this report already is in highly odd space.
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 46+ messages in thread