Regression: kexec/kdump boot hangs with x86/vector commits

Regression: kexec/kdump boot hangs with x86/vector commits

[Dave Young]

Hi,

Kexec reboot and kdump has broken on my laptop for long time with
4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
https://patchwork.kernel.org/patch/10084289/

But still can not get a successful reboot, it looked like graphic
issue, but after bisecting the kernel, I got below:

[dyoung@dhcp-*-* linux]$ git bisect good
There are only 'skip'ped commits left to test.
The first bad commit could be any of:
2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
4900be83602b6be07366d3e69f756c1959f4169a
We cannot bisect more!

These two commits can no be reverted because of code conflicts, thus
I reverted the whole series from Thomas (below commits), with those
x86/vector changes reverted, kexec reboot works fine.

Could you help to take a look, any thoughts?  I can do the test
if you have some debug patch to try.

commit 90ad9e2d91067983f3328e21b306323877e5f48a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:49 2017 +0200

    x86/io_apic: Reevaluate vector configuration on activate()

commit 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:50 2017 +0200

    x86/vector: Handle managed interrupts proper

commit 4900be83602b6be07366d3e69f756c1959f4169a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:51 2017 +0200

    x86/vector/msi: Switch to global reservation mode

commit 464d12309e1b5829597793db551ae8ecaecf4036
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:52 2017 +0200

    x86/vector: Switch IOAPIC to global reservation mod

commit 2cffad7bad83157f89332872015f4305d2ac09ac
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:53 2017 +0200

    x86/irq: Simplify hotplug vector accounting

commit d6ffc6ac83b1f9f12652d89b9cb5bcbfbea7796c
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:54 2017 +0200

    x86/vector: Respect affinity mask in irq descriptor

commit 02edee152d6ea325c88898f3a702f5db2d78de7a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Thu Oct 12 11:05:28 2017 +0200

    x86/apic/vector: Ignore set_affinity call for inactive interrupts

commit 0696d059f23c05f2dbc3b19ef50e5bdd175b782b
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Mon Oct 16 16:16:19 2017 +0200

    x86/vector: Use correct per cpu variable in free_moved_vector()
----------------

BTW, my git HEAD is below:
commit 43570f0383d6d5879ae585e6c3cf027ba321546f (origin/master, origin/HEAD, bisect)
Merge: 43f462f1c2e1 c14ca8386539
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Tue Nov 28 16:22:10 2017 -0800

    Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6



Thanks
Dave

Regression: kexec/kdump boot hangs with x86/vector commits

[Dave Young]

Hi,

Kexec reboot and kdump has broken on my laptop for long time with
4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
https://patchwork.kernel.org/patch/10084289/

But still can not get a successful reboot, it looked like graphic
issue, but after bisecting the kernel, I got below:

[dyoung@dhcp-*-* linux]$ git bisect good
There are only 'skip'ped commits left to test.
The first bad commit could be any of:
2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
4900be83602b6be07366d3e69f756c1959f4169a
We cannot bisect more!

These two commits can no be reverted because of code conflicts, thus
I reverted the whole series from Thomas (below commits), with those
x86/vector changes reverted, kexec reboot works fine.

Could you help to take a look, any thoughts?  I can do the test
if you have some debug patch to try.

commit 90ad9e2d91067983f3328e21b306323877e5f48a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:49 2017 +0200

    x86/io_apic: Reevaluate vector configuration on activate()

commit 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:50 2017 +0200

    x86/vector: Handle managed interrupts proper

commit 4900be83602b6be07366d3e69f756c1959f4169a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:51 2017 +0200

    x86/vector/msi: Switch to global reservation mode

commit 464d12309e1b5829597793db551ae8ecaecf4036
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:52 2017 +0200

    x86/vector: Switch IOAPIC to global reservation mod

commit 2cffad7bad83157f89332872015f4305d2ac09ac
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:53 2017 +0200

    x86/irq: Simplify hotplug vector accounting

commit d6ffc6ac83b1f9f12652d89b9cb5bcbfbea7796c
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Sep 13 23:29:54 2017 +0200

    x86/vector: Respect affinity mask in irq descriptor

commit 02edee152d6ea325c88898f3a702f5db2d78de7a
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Thu Oct 12 11:05:28 2017 +0200

    x86/apic/vector: Ignore set_affinity call for inactive interrupts

commit 0696d059f23c05f2dbc3b19ef50e5bdd175b782b
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Mon Oct 16 16:16:19 2017 +0200

    x86/vector: Use correct per cpu variable in free_moved_vector()
----------------

BTW, my git HEAD is below:
commit 43570f0383d6d5879ae585e6c3cf027ba321546f (origin/master, origin/HEAD, bisect)
Merge: 43f462f1c2e1 c14ca8386539
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Tue Nov 28 16:22:10 2017 -0800

    Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6



Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Regression: kexec/kdump boot hangs with x86/vector commits

[Yu Chen]

On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> Hi,
> 
> Kexec reboot and kdump has broken on my laptop for long time with
> 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> https://patchwork.kernel.org/patch/10084289/
> 
> But still can not get a successful reboot, it looked like graphic
> issue, but after bisecting the kernel, I got below:
> 
> [dyoung@dhcp-*-* linux]$ git bisect good
> There are only 'skip'ped commits left to test.
> The first bad commit could be any of:
> 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> 4900be83602b6be07366d3e69f756c1959f4169a
> We cannot bisect more!
> 
> These two commits can no be reverted because of code conflicts, thus
> I reverted the whole series from Thomas (below commits), with those
> x86/vector changes reverted, kexec reboot works fine.
> 
> Could you help to take a look, any thoughts?  I can do the test
> if you have some debug patch to try.
Is it possible that the "second" kernel runs on non-zero CPU? If yes,
what if some irqs are only delivered to cpu0? (use cpumask_of(0)
directly)

Thanks,
	Yu

Re: Regression: kexec/kdump boot hangs with x86/vector commits

[Yu Chen]

On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> Hi,
> 
> Kexec reboot and kdump has broken on my laptop for long time with
> 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> https://patchwork.kernel.org/patch/10084289/
> 
> But still can not get a successful reboot, it looked like graphic
> issue, but after bisecting the kernel, I got below:
> 
> [dyoung@dhcp-*-* linux]$ git bisect good
> There are only 'skip'ped commits left to test.
> The first bad commit could be any of:
> 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> 4900be83602b6be07366d3e69f756c1959f4169a
> We cannot bisect more!
> 
> These two commits can no be reverted because of code conflicts, thus
> I reverted the whole series from Thomas (below commits), with those
> x86/vector changes reverted, kexec reboot works fine.
> 
> Could you help to take a look, any thoughts?  I can do the test
> if you have some debug patch to try.
Is it possible that the "second" kernel runs on non-zero CPU? If yes,
what if some irqs are only delivered to cpu0? (use cpumask_of(0)
directly)

Thanks,
	Yu


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Regression: kexec/kdump boot hangs with x86/vector commits

[Dave Young]

On 12/13/17 at 11:57pm, Yu Chen wrote:
> On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > Hi,
> > 
> > Kexec reboot and kdump has broken on my laptop for long time with
> > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > https://patchwork.kernel.org/patch/10084289/
> > 
> > But still can not get a successful reboot, it looked like graphic
> > issue, but after bisecting the kernel, I got below:
> > 
> > [dyoung@dhcp-*-* linux]$ git bisect good
> > There are only 'skip'ped commits left to test.
> > The first bad commit could be any of:
> > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > 4900be83602b6be07366d3e69f756c1959f4169a
> > We cannot bisect more!
> > 
> > These two commits can no be reverted because of code conflicts, thus
> > I reverted the whole series from Thomas (below commits), with those
> > x86/vector changes reverted, kexec reboot works fine.
> > 
> > Could you help to take a look, any thoughts?  I can do the test
> > if you have some debug patch to try.
> Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> directly)

Thanks for the reply.

For kdump, yes, for kexec, I'm not sure.  

Here is some kexec kernel boot log:
http://people.redhat.com/~ruyang/misc/kexec-regression.txt

Copy the lockup call trace here:
[   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
[   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
_device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
ci pps_core snd soundcore rfkill video                                          
[   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
[   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
) 03/03/2017                                                                    
[   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
[   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
[   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
[   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
[   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
[   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
[   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
[   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
00000000000                                                                     
[   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
[   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
[   23.779322] Call Trace:                                                      
[   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
[   23.779333]  do_idle+0x17b/0x1d0                                             
[   23.779335]  cpu_startup_entry+0x6f/0x80                                     
[   23.779338]  start_kernel+0x431/0x451                                        
[   23.779342]  secondary_startup_64+0xa5/0xb0                                  
[   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
 02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
 a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df

Thanks
Dave
> 
> Thanks,
> 	Yu
> 

Re: Regression: kexec/kdump boot hangs with x86/vector commits

[Dave Young]

On 12/13/17 at 11:57pm, Yu Chen wrote:
> On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > Hi,
> > 
> > Kexec reboot and kdump has broken on my laptop for long time with
> > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > https://patchwork.kernel.org/patch/10084289/
> > 
> > But still can not get a successful reboot, it looked like graphic
> > issue, but after bisecting the kernel, I got below:
> > 
> > [dyoung@dhcp-*-* linux]$ git bisect good
> > There are only 'skip'ped commits left to test.
> > The first bad commit could be any of:
> > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > 4900be83602b6be07366d3e69f756c1959f4169a
> > We cannot bisect more!
> > 
> > These two commits can no be reverted because of code conflicts, thus
> > I reverted the whole series from Thomas (below commits), with those
> > x86/vector changes reverted, kexec reboot works fine.
> > 
> > Could you help to take a look, any thoughts?  I can do the test
> > if you have some debug patch to try.
> Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> directly)

Thanks for the reply.

For kdump, yes, for kexec, I'm not sure.  

Here is some kexec kernel boot log:
http://people.redhat.com/~ruyang/misc/kexec-regression.txt

Copy the lockup call trace here:
[   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
[   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
_device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
ci pps_core snd soundcore rfkill video                                          
[   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
[   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
) 03/03/2017                                                                    
[   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
[   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
[   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
[   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
[   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
[   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
[   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
[   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
00000000000                                                                     
[   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
[   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
[   23.779322] Call Trace:                                                      
[   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
[   23.779333]  do_idle+0x17b/0x1d0                                             
[   23.779335]  cpu_startup_entry+0x6f/0x80                                     
[   23.779338]  start_kernel+0x431/0x451                                        
[   23.779342]  secondary_startup_64+0xa5/0xb0                                  
[   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
 02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
 a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df

Thanks
Dave
> 
> Thanks,
> 	Yu
> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Regression: kexec/kdump boot hangs with x86/vector commits

[Dave Young]

On 12/14/17 at 05:24pm, Dave Young wrote:
> On 12/13/17 at 11:57pm, Yu Chen wrote:
> > On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > > Hi,
> > > 
> > > Kexec reboot and kdump has broken on my laptop for long time with
> > > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > > https://patchwork.kernel.org/patch/10084289/
> > > 
> > > But still can not get a successful reboot, it looked like graphic
> > > issue, but after bisecting the kernel, I got below:
> > > 
> > > [dyoung@dhcp-*-* linux]$ git bisect good
> > > There are only 'skip'ped commits left to test.
> > > The first bad commit could be any of:
> > > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > > 4900be83602b6be07366d3e69f756c1959f4169a
> > > We cannot bisect more!
> > > 
> > > These two commits can no be reverted because of code conflicts, thus
> > > I reverted the whole series from Thomas (below commits), with those
> > > x86/vector changes reverted, kexec reboot works fine.
> > > 
> > > Could you help to take a look, any thoughts?  I can do the test
> > > if you have some debug patch to try.
> > Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> > what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> > directly)
> 
> Thanks for the reply.
> 
> For kdump, yes, for kexec, I'm not sure.  
> 
> Here is some kexec kernel boot log:
> http://people.redhat.com/~ruyang/misc/kexec-regression.txt
> 
> Copy the lockup call trace here:
> [   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
> [   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
> 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
> tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
> ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
> _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
> ci pps_core snd soundcore rfkill video                                          
> [   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
> [   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
> ) 03/03/2017                                                                    
> [   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
> [   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
> [   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
> [   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
> [   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
> [   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
> [   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
> [   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
> 00000000000                                                                     
> [   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
> [   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
> [   23.779322] Call Trace:                                                      
> [   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
> [   23.779333]  do_idle+0x17b/0x1d0                                             
> [   23.779335]  cpu_startup_entry+0x6f/0x80                                     
> [   23.779338]  start_kernel+0x431/0x451                                        
> [   23.779342]  secondary_startup_64+0xa5/0xb0                                  
> [   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
>  02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
>  a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
> 

Followup this issue, seems another commit from Thomas partially fixed
this, kexec/kdump boot up successfully for me, but kexec after kexec
(2nd kexec reboot cycle) failed, kernel hung early

commit bc976233a872c0f20f018fb1e89264a541584e25
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Fri Dec 29 10:47:22 2017 +0100

    genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI

Thanks
Dave

Re: Regression: kexec/kdump boot hangs with x86/vector commits

[Dave Young]

On 12/14/17 at 05:24pm, Dave Young wrote:
> On 12/13/17 at 11:57pm, Yu Chen wrote:
> > On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > > Hi,
> > > 
> > > Kexec reboot and kdump has broken on my laptop for long time with
> > > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > > https://patchwork.kernel.org/patch/10084289/
> > > 
> > > But still can not get a successful reboot, it looked like graphic
> > > issue, but after bisecting the kernel, I got below:
> > > 
> > > [dyoung@dhcp-*-* linux]$ git bisect good
> > > There are only 'skip'ped commits left to test.
> > > The first bad commit could be any of:
> > > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > > 4900be83602b6be07366d3e69f756c1959f4169a
> > > We cannot bisect more!
> > > 
> > > These two commits can no be reverted because of code conflicts, thus
> > > I reverted the whole series from Thomas (below commits), with those
> > > x86/vector changes reverted, kexec reboot works fine.
> > > 
> > > Could you help to take a look, any thoughts?  I can do the test
> > > if you have some debug patch to try.
> > Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> > what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> > directly)
> 
> Thanks for the reply.
> 
> For kdump, yes, for kexec, I'm not sure.  
> 
> Here is some kexec kernel boot log:
> http://people.redhat.com/~ruyang/misc/kexec-regression.txt
> 
> Copy the lockup call trace here:
> [   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
> [   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
> 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
> tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
> ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
> _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
> ci pps_core snd soundcore rfkill video                                          
> [   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
> [   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
> ) 03/03/2017                                                                    
> [   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
> [   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
> [   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
> [   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
> [   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
> [   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
> [   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
> [   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
> 00000000000                                                                     
> [   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
> [   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
> [   23.779322] Call Trace:                                                      
> [   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
> [   23.779333]  do_idle+0x17b/0x1d0                                             
> [   23.779335]  cpu_startup_entry+0x6f/0x80                                     
> [   23.779338]  start_kernel+0x431/0x451                                        
> [   23.779342]  secondary_startup_64+0xa5/0xb0                                  
> [   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
>  02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
>  a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
> 

Followup this issue, seems another commit from Thomas partially fixed
this, kexec/kdump boot up successfully for me, but kexec after kexec
(2nd kexec reboot cycle) failed, kernel hung early

commit bc976233a872c0f20f018fb1e89264a541584e25
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Fri Dec 29 10:47:22 2017 +0100

    genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

[Modify the subject since this is a new problem, original io vector
issue has been fixed with one commit from Thomas]

Add more cc according to below old discussion:
https://lkml.org/lkml/2017/7/27/574

Tom, I'm not sure why you finally did not dynamically run wbinvd?
On 01/04/18 at 11:15am, Dave Young wrote:
> On 12/14/17 at 05:24pm, Dave Young wrote:
> > On 12/13/17 at 11:57pm, Yu Chen wrote:
> > > On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > > > Hi,
> > > > 
> > > > Kexec reboot and kdump has broken on my laptop for long time with
> > > > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > > > https://patchwork.kernel.org/patch/10084289/
> > > > 
> > > > But still can not get a successful reboot, it looked like graphic
> > > > issue, but after bisecting the kernel, I got below:
> > > > 
> > > > [dyoung@dhcp-*-* linux]$ git bisect good
> > > > There are only 'skip'ped commits left to test.
> > > > The first bad commit could be any of:
> > > > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > > > 4900be83602b6be07366d3e69f756c1959f4169a
> > > > We cannot bisect more!
> > > > 
> > > > These two commits can no be reverted because of code conflicts, thus
> > > > I reverted the whole series from Thomas (below commits), with those
> > > > x86/vector changes reverted, kexec reboot works fine.
> > > > 
> > > > Could you help to take a look, any thoughts?  I can do the test
> > > > if you have some debug patch to try.
> > > Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> > > what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> > > directly)
> > 
> > Thanks for the reply.
> > 
> > For kdump, yes, for kexec, I'm not sure.  
> > 
> > Here is some kexec kernel boot log:
> > http://people.redhat.com/~ruyang/misc/kexec-regression.txt
> > 
> > Copy the lockup call trace here:
> > [   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
> > [   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
> > 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
> > tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
> > ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
> > _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
> > ci pps_core snd soundcore rfkill video                                          
> > [   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
> > [   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
> > ) 03/03/2017                                                                    
> > [   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
> > [   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
> > [   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
> > [   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
> > [   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
> > [   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
> > [   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
> > [   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
> > 00000000000                                                                     
> > [   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
> > [   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
> > [   23.779322] Call Trace:                                                      
> > [   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
> > [   23.779333]  do_idle+0x17b/0x1d0                                             
> > [   23.779335]  cpu_startup_entry+0x6f/0x80                                     
> > [   23.779338]  start_kernel+0x431/0x451                                        
> > [   23.779342]  secondary_startup_64+0xa5/0xb0                                  
> > [   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
> >  02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
> >  a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
> > 
> 
> Followup this issue, seems another commit from Thomas partially fixed
> this, kexec/kdump boot up successfully for me, but kexec after kexec
> (2nd kexec reboot cycle) failed, kernel hung early

The above kexec reboot hang is another problem, so Thomas has fully
fixed previous report, thanks!

For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
then kexec works fine. like this:
 
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 832a6acd730f..6d7499730b27 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
 	disable_local_APIC();
 	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
 
-	for (;;) {
-		/*
-		 * Use wbinvd followed by hlt to stop the processor. This
-		 * provides support for kexec on a processor that supports
-		 * SME. With kexec, going from SME inactive to SME active
-		 * requires clearing cache entries so that addresses without
-		 * the encryption bit set don't corrupt the same physical
-		 * address that has the encryption bit set when caches are
-		 * flushed. To achieve this a wbinvd is performed followed by
-		 * a hlt. Even if the processor is not in the kexec/SME
-		 * scenario this only adds a wbinvd to a halting processor.
-		 */
-		asm volatile("wbinvd; hlt" : : : "memory");
-	}
+	for (;;)
+		halt();
 }
 
 /*

But I have no idea why though, seeking for help and thoughts..

> 
> commit bc976233a872c0f20f018fb1e89264a541584e25
> Author: Thomas Gleixner <tglx@linutronix.de>
> Date:   Fri Dec 29 10:47:22 2017 +0100
> 
>     genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
> 
> Thanks
> Dave

Thanks
Dave

kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

[Modify the subject since this is a new problem, original io vector
issue has been fixed with one commit from Thomas]

Add more cc according to below old discussion:
https://lkml.org/lkml/2017/7/27/574

Tom, I'm not sure why you finally did not dynamically run wbinvd?
On 01/04/18 at 11:15am, Dave Young wrote:
> On 12/14/17 at 05:24pm, Dave Young wrote:
> > On 12/13/17 at 11:57pm, Yu Chen wrote:
> > > On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
> > > > Hi,
> > > > 
> > > > Kexec reboot and kdump has broken on my laptop for long time with
> > > > 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
> > > > https://patchwork.kernel.org/patch/10084289/
> > > > 
> > > > But still can not get a successful reboot, it looked like graphic
> > > > issue, but after bisecting the kernel, I got below:
> > > > 
> > > > [dyoung@dhcp-*-* linux]$ git bisect good
> > > > There are only 'skip'ped commits left to test.
> > > > The first bad commit could be any of:
> > > > 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
> > > > 4900be83602b6be07366d3e69f756c1959f4169a
> > > > We cannot bisect more!
> > > > 
> > > > These two commits can no be reverted because of code conflicts, thus
> > > > I reverted the whole series from Thomas (below commits), with those
> > > > x86/vector changes reverted, kexec reboot works fine.
> > > > 
> > > > Could you help to take a look, any thoughts?  I can do the test
> > > > if you have some debug patch to try.
> > > Is it possible that the "second" kernel runs on non-zero CPU? If yes,
> > > what if some irqs are only delivered to cpu0? (use cpumask_of(0)
> > > directly)
> > 
> > Thanks for the reply.
> > 
> > For kdump, yes, for kexec, I'm not sure.  
> > 
> > Here is some kexec kernel boot log:
> > http://people.redhat.com/~ruyang/misc/kexec-regression.txt
> > 
> > Copy the lockup call trace here:
> > [   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
> > [   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
> > 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
> > tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
> > ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
> > _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
> > ci pps_core snd soundcore rfkill video                                          
> > [   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
> > [   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
> > ) 03/03/2017                                                                    
> > [   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
> > [   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
> > [   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
> > [   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
> > [   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
> > [   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
> > [   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
> > [   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
> > 00000000000                                                                     
> > [   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
> > [   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
> > [   23.779322] Call Trace:                                                      
> > [   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
> > [   23.779333]  do_idle+0x17b/0x1d0                                             
> > [   23.779335]  cpu_startup_entry+0x6f/0x80                                     
> > [   23.779338]  start_kernel+0x431/0x451                                        
> > [   23.779342]  secondary_startup_64+0xa5/0xb0                                  
> > [   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
> >  02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
> >  a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
> > 
> 
> Followup this issue, seems another commit from Thomas partially fixed
> this, kexec/kdump boot up successfully for me, but kexec after kexec
> (2nd kexec reboot cycle) failed, kernel hung early

The above kexec reboot hang is another problem, so Thomas has fully
fixed previous report, thanks!

For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
then kexec works fine. like this:
 
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 832a6acd730f..6d7499730b27 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
 	disable_local_APIC();
 	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
 
-	for (;;) {
-		/*
-		 * Use wbinvd followed by hlt to stop the processor. This
-		 * provides support for kexec on a processor that supports
-		 * SME. With kexec, going from SME inactive to SME active
-		 * requires clearing cache entries so that addresses without
-		 * the encryption bit set don't corrupt the same physical
-		 * address that has the encryption bit set when caches are
-		 * flushed. To achieve this a wbinvd is performed followed by
-		 * a hlt. Even if the processor is not in the kexec/SME
-		 * scenario this only adds a wbinvd to a halting processor.
-		 */
-		asm volatile("wbinvd; hlt" : : : "memory");
-	}
+	for (;;)
+		halt();
 }
 
 /*

But I have no idea why though, seeking for help and thoughts..

> 
> commit bc976233a872c0f20f018fb1e89264a541584e25
> Author: Thomas Gleixner <tglx@linutronix.de>
> Date:   Fri Dec 29 10:47:22 2017 +0100
> 
>     genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
> 
> Thanks
> Dave

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 1:22 AM, Dave Young wrote:
> [Modify the subject since this is a new problem, original io vector
> issue has been fixed with one commit from Thomas]
> 
> Add more cc according to below old discussion:
> https://lkml.org/lkml/2017/7/27/574
> 
> Tom, I'm not sure why you finally did not dynamically run wbinvd?

That discussion was aimed at the wbinvd that was being performed
in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
run based on a flag.

> On 01/04/18 at 11:15am, Dave Young wrote:
>> On 12/14/17 at 05:24pm, Dave Young wrote:
>>> On 12/13/17 at 11:57pm, Yu Chen wrote:
>>>> On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
>>>>> Hi,
>>>>>
>>>>> Kexec reboot and kdump has broken on my laptop for long time with
>>>>> 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
>>>>> https://patchwork.kernel.org/patch/10084289/
>>>>>
>>>>> But still can not get a successful reboot, it looked like graphic
>>>>> issue, but after bisecting the kernel, I got below:
>>>>>
>>>>> [dyoung@dhcp-*-* linux]$ git bisect good
>>>>> There are only 'skip'ped commits left to test.
>>>>> The first bad commit could be any of:
>>>>> 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
>>>>> 4900be83602b6be07366d3e69f756c1959f4169a
>>>>> We cannot bisect more!
>>>>>
>>>>> These two commits can no be reverted because of code conflicts, thus
>>>>> I reverted the whole series from Thomas (below commits), with those
>>>>> x86/vector changes reverted, kexec reboot works fine.
>>>>>
>>>>> Could you help to take a look, any thoughts?  I can do the test
>>>>> if you have some debug patch to try.
>>>> Is it possible that the "second" kernel runs on non-zero CPU? If yes,
>>>> what if some irqs are only delivered to cpu0? (use cpumask_of(0)
>>>> directly)
>>>
>>> Thanks for the reply.
>>>
>>> For kdump, yes, for kexec, I'm not sure.  
>>>
>>> Here is some kexec kernel boot log:
>>> http://people.redhat.com/~ruyang/misc/kexec-regression.txt
>>>
>>> Copy the lockup call trace here:
>>> [   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
>>> [   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
>>> 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
>>> tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
>>> ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
>>> _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
>>> ci pps_core snd soundcore rfkill video                                          
>>> [   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
>>> [   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
>>> ) 03/03/2017                                                                    
>>> [   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
>>> [   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
>>> [   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
>>> [   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
>>> [   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
>>> [   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
>>> [   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
>>> [   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
>>> 00000000000                                                                     
>>> [   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
>>> [   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
>>> [   23.779322] Call Trace:                                                      
>>> [   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
>>> [   23.779333]  do_idle+0x17b/0x1d0                                             
>>> [   23.779335]  cpu_startup_entry+0x6f/0x80                                     
>>> [   23.779338]  start_kernel+0x431/0x451                                        
>>> [   23.779342]  secondary_startup_64+0xa5/0xb0                                  
>>> [   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
>>>  02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
>>>  a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
>>>
>>
>> Followup this issue, seems another commit from Thomas partially fixed
>> this, kexec/kdump boot up successfully for me, but kexec after kexec
>> (2nd kexec reboot cycle) failed, kernel hung early
> 
> The above kexec reboot hang is another problem, so Thomas has fully
> fixed previous report, thanks!
> 
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:
>  
> diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
> index 832a6acd730f..6d7499730b27 100644
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
>  	disable_local_APIC();
>  	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
>  
> -	for (;;) {
> -		/*
> -		 * Use wbinvd followed by hlt to stop the processor. This
> -		 * provides support for kexec on a processor that supports
> -		 * SME. With kexec, going from SME inactive to SME active
> -		 * requires clearing cache entries so that addresses without
> -		 * the encryption bit set don't corrupt the same physical
> -		 * address that has the encryption bit set when caches are
> -		 * flushed. To achieve this a wbinvd is performed followed by
> -		 * a hlt. Even if the processor is not in the kexec/SME
> -		 * scenario this only adds a wbinvd to a halting processor.
> -		 */
> -		asm volatile("wbinvd; hlt" : : : "memory");
> -	}
> +	for (;;)
> +		halt();
>  }
>  
>  /*
> 
> But I have no idea why though, seeking for help and thoughts..

Yeah, I don't know why that works either.

Thanks,
Tom

> 
>>
>> commit bc976233a872c0f20f018fb1e89264a541584e25
>> Author: Thomas Gleixner <tglx@linutronix.de>
>> Date:   Fri Dec 29 10:47:22 2017 +0100
>>
>>     genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
>>
>> Thanks
>> Dave
> 
> Thanks
> Dave
> 

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 1:22 AM, Dave Young wrote:
> [Modify the subject since this is a new problem, original io vector
> issue has been fixed with one commit from Thomas]
> 
> Add more cc according to below old discussion:
> https://lkml.org/lkml/2017/7/27/574
> 
> Tom, I'm not sure why you finally did not dynamically run wbinvd?

That discussion was aimed at the wbinvd that was being performed
in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
run based on a flag.

> On 01/04/18 at 11:15am, Dave Young wrote:
>> On 12/14/17 at 05:24pm, Dave Young wrote:
>>> On 12/13/17 at 11:57pm, Yu Chen wrote:
>>>> On Wed, Dec 13, 2017 at 10:52:56AM +0800, Dave Young wrote:
>>>>> Hi,
>>>>>
>>>>> Kexec reboot and kdump has broken on my laptop for long time with
>>>>> 4.15.0-rc1+ kernels. With the patch below an early panic been fixed:
>>>>> https://patchwork.kernel.org/patch/10084289/
>>>>>
>>>>> But still can not get a successful reboot, it looked like graphic
>>>>> issue, but after bisecting the kernel, I got below:
>>>>>
>>>>> [dyoung@dhcp-*-* linux]$ git bisect good
>>>>> There are only 'skip'ped commits left to test.
>>>>> The first bad commit could be any of:
>>>>> 2db1f959d9dc16035f2eb44ed5fdb2789b754d6a
>>>>> 4900be83602b6be07366d3e69f756c1959f4169a
>>>>> We cannot bisect more!
>>>>>
>>>>> These two commits can no be reverted because of code conflicts, thus
>>>>> I reverted the whole series from Thomas (below commits), with those
>>>>> x86/vector changes reverted, kexec reboot works fine.
>>>>>
>>>>> Could you help to take a look, any thoughts?  I can do the test
>>>>> if you have some debug patch to try.
>>>> Is it possible that the "second" kernel runs on non-zero CPU? If yes,
>>>> what if some irqs are only delivered to cpu0? (use cpumask_of(0)
>>>> directly)
>>>
>>> Thanks for the reply.
>>>
>>> For kdump, yes, for kexec, I'm not sure.  
>>>
>>> Here is some kexec kernel boot log:
>>> http://people.redhat.com/~ruyang/misc/kexec-regression.txt
>>>
>>> Copy the lockup call trace here:
>>> [   23.779285] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0             
>>> [   23.779285] Modules linked in: arc4 rtsx_pci_sdmmc i915 iwlmvm kvm_intel mac8
>>> 0211 kvm irqbypass btusb btrtl btbcm intel_gtt btintel drm_kms_helper snd_hda_in
>>> tel syscopyarea bluetooth iwlwifi snd_hda_codec snd_hwdep snd_hda_core sysfillre
>>> ct snd_seq sysimgblt input_leds fb_sys_fops e1000e ecdh_generic cfg80211 snd_seq
>>> _device drm snd_pcm serio_raw ptp pcspkr thinkpad_acpi i2c_i801 snd_timer rtsx_p
>>> ci pps_core snd soundcore rfkill video                                          
>>> [   23.779307] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc3+ #378       
>>> [   23.779308] Hardware name: LENOVO 20ARS1BJ02/20ARS1BJ02, BIOS GJET92WW (2.42 
>>> ) 03/03/2017                                                                    
>>> [   23.779312] RIP: 0010:poll_idle+0x2f/0x5f                                    
>>> [   23.779313] RSP: 0018:ffffffff81c03e80 EFLAGS: 00000246                      
>>> [   23.779314] RAX: ffffffff81c0f4c0 RBX: ffffffff81c6db80 RCX: 0000000000000000
>>> [   23.779315] RDX: 0000000000000000 RSI: ffffffff81c6db80 RDI: ffff88021f2201e8
>>> [   23.779316] RBP: ffff88021f2201e8 R08: 000000349a65b7dd R09: ffff88021f216db4
>>> [   23.779317] R10: ffffffff81c03e68 R11: 0000000000000000 R12: 0000000000000000
>>> [   23.779318] R13: ffffffff81c6db98 R14: 0000000000000000 R15: 0000000578a065b1
>>> [   23.779319] FS:  0000000000000000(0000) GS:ffff88021f200000(0000) knlGS:00000
>>> 00000000000                                                                     
>>> [   23.779320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
>>> [   23.779321] CR2: 00007ffed1d0ee60 CR3: 000000021ec0a006 CR4: 00000000001606b0
>>> [   23.779322] Call Trace:                                                      
>>> [   23.779328]  cpuidle_enter_state+0x6a/0x2c0                                  
>>> [   23.779333]  do_idle+0x17b/0x1d0                                             
>>> [   23.779335]  cpu_startup_entry+0x6f/0x80                                     
>>> [   23.779338]  start_kernel+0x431/0x451                                        
>>> [   23.779342]  secondary_startup_64+0xa5/0xb0                                  
>>> [   23.779344] Code: 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 40 c4 00 00 f0 80 48
>>>  02 20 48 8b 08 83 e1 08 74 0d eb 12 f3 90 65 48 8b 04 25 40 c4 00 00 <48> 8b 00
>>>  a8 08 74 ee 65 48 8b 04 25 40 c4 00 00 f0 80 60 02 df
>>>
>>
>> Followup this issue, seems another commit from Thomas partially fixed
>> this, kexec/kdump boot up successfully for me, but kexec after kexec
>> (2nd kexec reboot cycle) failed, kernel hung early
> 
> The above kexec reboot hang is another problem, so Thomas has fully
> fixed previous report, thanks!
> 
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:
>  
> diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
> index 832a6acd730f..6d7499730b27 100644
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -380,20 +380,8 @@ void stop_this_cpu(void *dummy)
>  	disable_local_APIC();
>  	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
>  
> -	for (;;) {
> -		/*
> -		 * Use wbinvd followed by hlt to stop the processor. This
> -		 * provides support for kexec on a processor that supports
> -		 * SME. With kexec, going from SME inactive to SME active
> -		 * requires clearing cache entries so that addresses without
> -		 * the encryption bit set don't corrupt the same physical
> -		 * address that has the encryption bit set when caches are
> -		 * flushed. To achieve this a wbinvd is performed followed by
> -		 * a hlt. Even if the processor is not in the kexec/SME
> -		 * scenario this only adds a wbinvd to a halting processor.
> -		 */
> -		asm volatile("wbinvd; hlt" : : : "memory");
> -	}
> +	for (;;)
> +		halt();
>  }
>  
>  /*
> 
> But I have no idea why though, seeking for help and thoughts..

Yeah, I don't know why that works either.

Thanks,
Tom

> 
>>
>> commit bc976233a872c0f20f018fb1e89264a541584e25
>> Author: Thomas Gleixner <tglx@linutronix.de>
>> Date:   Fri Dec 29 10:47:22 2017 +0100
>>
>>     genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
>>
>> Thanks
>> Dave
> 
> Thanks
> Dave
> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:

Honestly, I think we should apply that patch regardless.

Using 'wbinvd' should not be some "just because of random reasons".
There are CPU's with errata on wbinvd, and the thing in general is
slow and nasty.

Doing the wbinvd in a loop sounds even stranger.

If we're only doing it because of some SME issue, why isn't it
dependent on SME? And why is it inside that loop at all?

Anyway, does it work for you if you just do the wbinvd() once, outside
the loop? Admittedly the loop shouldn't actually loop (hlt with
interrupts disabled), but who the hell knows.. Some of the errata
around SME have been about machine check exceptions or something.

See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
play_dead loop") for another example where wbinvd was inside a loop
and apparently caused some odd issues.

              Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>
> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> then kexec works fine. like this:

Honestly, I think we should apply that patch regardless.

Using 'wbinvd' should not be some "just because of random reasons".
There are CPU's with errata on wbinvd, and the thing in general is
slow and nasty.

Doing the wbinvd in a loop sounds even stranger.

If we're only doing it because of some SME issue, why isn't it
dependent on SME? And why is it inside that loop at all?

Anyway, does it work for you if you just do the wbinvd() once, outside
the loop? Admittedly the loop shouldn't actually loop (hlt with
interrupts disabled), but who the hell knows.. Some of the errata
around SME have been about machine check exceptions or something.

See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
play_dead loop") for another example where wbinvd was inside a loop
and apparently caused some odd issues.

              Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 1:42 PM, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>
>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>> then kexec works fine. like this:
> 
> Honestly, I think we should apply that patch regardless.
> 
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
> 
> Doing the wbinvd in a loop sounds even stranger.
> 
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?

My original patches did check for X86_FEATURE_SME and only do the
wbinvd if SME was supported (although still in the loop).  The general
consensus was to just do the wbinvd no matter what and so it is as it is
today.

It can probably be outside of the loop.  The issue I was seeing was
memory corruption from the stack when using halt() with paravirt ops
enabled.  So a native_halt() should be used.

> 
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.

I think that should work as long as it's a native_wbinvd() call and it
can also be conditional on boot_cpu_has(X86_FEATURE_SME).

I'll do some testing.

Thanks,
Tom

> 
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
> 
>               Linus
> 

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 1:42 PM, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>
>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>> then kexec works fine. like this:
> 
> Honestly, I think we should apply that patch regardless.
> 
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
> 
> Doing the wbinvd in a loop sounds even stranger.
> 
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?

My original patches did check for X86_FEATURE_SME and only do the
wbinvd if SME was supported (although still in the loop).  The general
consensus was to just do the wbinvd no matter what and so it is as it is
today.

It can probably be outside of the loop.  The issue I was seeing was
memory corruption from the stack when using halt() with paravirt ops
enabled.  So a native_halt() should be used.

> 
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.

I think that should work as long as it's a native_wbinvd() call and it
can also be conditional on boot_cpu_has(X86_FEATURE_SME).

I'll do some testing.

Thanks,
Tom

> 
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
> 
>               Linus
> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 11:42 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>  [ .. ]        Some of the errata
> around SME have been about machine check exceptions or something.

That should be "some of the errata around wbinvd". I have no idea if
there have been SME issues.

That said, the really bad old wbinvd bug (which hung the system iirc)
was for some old PPro CPU's. Googling around, the errata I find seem
either irrelevant (eip corruption in 16-bit mode) or fairly mild (odd
behavior wrt parity errors).

So I have this memory of wbinvd having problems, but that memory does
seem to be largely historical. But Dave Young clearly sees *something*
odd going on.

               Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 11:42 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>  [ .. ]        Some of the errata
> around SME have been about machine check exceptions or something.

That should be "some of the errata around wbinvd". I have no idea if
there have been SME issues.

That said, the really bad old wbinvd bug (which hung the system iirc)
was for some old PPro CPU's. Googling around, the errata I find seem
either irrelevant (eip corruption in 16-bit mode) or fairly mild (odd
behavior wrt parity errors).

So I have this memory of wbinvd having problems, but that memory does
seem to be largely historical. But Dave Young clearly sees *something*
odd going on.

               Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 2:01 PM, Tom Lendacky wrote:
> On 1/17/2018 1:42 PM, Linus Torvalds wrote:
>> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>>> then kexec works fine. like this:
>>
>> Honestly, I think we should apply that patch regardless.
>>
>> Using 'wbinvd' should not be some "just because of random reasons".
>> There are CPU's with errata on wbinvd, and the thing in general is
>> slow and nasty.
>>
>> Doing the wbinvd in a loop sounds even stranger.
>>
>> If we're only doing it because of some SME issue, why isn't it
>> dependent on SME? And why is it inside that loop at all?
> 
> My original patches did check for X86_FEATURE_SME and only do the
> wbinvd if SME was supported (although still in the loop).  The general
> consensus was to just do the wbinvd no matter what and so it is as it is
> today.
> 
> It can probably be outside of the loop.  The issue I was seeing was
> memory corruption from the stack when using halt() with paravirt ops
> enabled.  So a native_halt() should be used.
> 
>>
>> Anyway, does it work for you if you just do the wbinvd() once, outside
>> the loop? Admittedly the loop shouldn't actually loop (hlt with
>> interrupts disabled), but who the hell knows.. Some of the errata
>> around SME have been about machine check exceptions or something.
> 
> I think that should work as long as it's a native_wbinvd() call and it
> can also be conditional on boot_cpu_has(X86_FEATURE_SME).
> 
> I'll do some testing.

Looks like everything is good with the suggested changes.  Patch to follow
shortly.

Thanks,
Tom

> 
> Thanks,
> Tom
> 
>>
>> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
>> play_dead loop") for another example where wbinvd was inside a loop
>> and apparently caused some odd issues.
>>
>>               Linus
>>

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 2:01 PM, Tom Lendacky wrote:
> On 1/17/2018 1:42 PM, Linus Torvalds wrote:
>> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
>>> then kexec works fine. like this:
>>
>> Honestly, I think we should apply that patch regardless.
>>
>> Using 'wbinvd' should not be some "just because of random reasons".
>> There are CPU's with errata on wbinvd, and the thing in general is
>> slow and nasty.
>>
>> Doing the wbinvd in a loop sounds even stranger.
>>
>> If we're only doing it because of some SME issue, why isn't it
>> dependent on SME? And why is it inside that loop at all?
> 
> My original patches did check for X86_FEATURE_SME and only do the
> wbinvd if SME was supported (although still in the loop).  The general
> consensus was to just do the wbinvd no matter what and so it is as it is
> today.
> 
> It can probably be outside of the loop.  The issue I was seeing was
> memory corruption from the stack when using halt() with paravirt ops
> enabled.  So a native_halt() should be used.
> 
>>
>> Anyway, does it work for you if you just do the wbinvd() once, outside
>> the loop? Admittedly the loop shouldn't actually loop (hlt with
>> interrupts disabled), but who the hell knows.. Some of the errata
>> around SME have been about machine check exceptions or something.
> 
> I think that should work as long as it's a native_wbinvd() call and it
> can also be conditional on boot_cpu_has(X86_FEATURE_SME).
> 
> I'll do some testing.

Looks like everything is good with the suggested changes.  Patch to follow
shortly.

Thanks,
Tom

> 
> Thanks,
> Tom
> 
>>
>> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
>> play_dead loop") for another example where wbinvd was inside a loop
>> and apparently caused some odd issues.
>>
>>               Linus
>>

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 11:42am, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> > then kexec works fine. like this:
> 
> Honestly, I think we should apply that patch regardless.
> 
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
> 
> Doing the wbinvd in a loop sounds even stranger.
> 
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?
> 
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.

It does not work with just once wbinvd(), and it only works with
removing the wbinvd() for me.  Tom's new post works for me as well
since my cpu is an Intel i5-4200U.

> 
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
> 
>               Linus

Thanks for the help! Finally I get my laptop kexec & kdump works again
after solving these issues.

Thanks
Dave

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 11:42am, Linus Torvalds wrote:
> On Tue, Jan 16, 2018 at 11:22 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > For the kexec reboot hang, if I remove the wbinvd in stop_this_cpu()
> > then kexec works fine. like this:
> 
> Honestly, I think we should apply that patch regardless.
> 
> Using 'wbinvd' should not be some "just because of random reasons".
> There are CPU's with errata on wbinvd, and the thing in general is
> slow and nasty.
> 
> Doing the wbinvd in a loop sounds even stranger.
> 
> If we're only doing it because of some SME issue, why isn't it
> dependent on SME? And why is it inside that loop at all?
> 
> Anyway, does it work for you if you just do the wbinvd() once, outside
> the loop? Admittedly the loop shouldn't actually loop (hlt with
> interrupts disabled), but who the hell knows.. Some of the errata
> around SME have been about machine check exceptions or something.

It does not work with just once wbinvd(), and it only works with
removing the wbinvd() for me.  Tom's new post works for me as well
since my cpu is an Intel i5-4200U.

> 
> See commit a68e5c94f7d3 ("x86, hotplug: Move WBINVD back outside the
> play_dead loop") for another example where wbinvd was inside a loop
> and apparently caused some odd issues.
> 
>               Linus

Thanks for the help! Finally I get my laptop kexec & kdump works again
after solving these issues.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 09:06am, Tom Lendacky wrote:
> On 1/17/2018 1:22 AM, Dave Young wrote:
> > [Modify the subject since this is a new problem, original io vector
> > issue has been fixed with one commit from Thomas]
> > 
> > Add more cc according to below old discussion:
> > https://lkml.org/lkml/2017/7/27/574
> > 
> > Tom, I'm not sure why you finally did not dynamically run wbinvd?
> 
> That discussion was aimed at the wbinvd that was being performed
> in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
> run based on a flag.

Oh, I did not notice that, sorry for referring a wrong discussion.

[snip]

Thanks
Dave

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 09:06am, Tom Lendacky wrote:
> On 1/17/2018 1:22 AM, Dave Young wrote:
> > [Modify the subject since this is a new problem, original io vector
> > issue has been fixed with one commit from Thomas]
> > 
> > Add more cc according to below old discussion:
> > https://lkml.org/lkml/2017/7/27/574
> > 
> > Tom, I'm not sure why you finally did not dynamically run wbinvd?
> 
> That discussion was aimed at the wbinvd that was being performed
> in arch/x86/kernel/relocate_kernel_64.S, which is dynamically
> run based on a flag.

Oh, I did not notice that, sorry for referring a wrong discussion.

[snip]

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>
> It does not work with just once wbinvd(), and it only works with
> removing the wbinvd() for me.  Tom's new post works for me as well
> since my cpu is an Intel i5-4200U.

Intriguing.

It's not like the wbinvd really should be that much of a deal.

I think Tom's patch is fine and should be applied, but it does worry
me a bit that even a single wbinvd makes that much of a difference for
you. There is very little logical reason I can think of that a wbinvd
should make any difference what-so-ever on an i5-4200U.

I wonder if you have some system issues, and wbinvd just happens to
trigger them. But I think we do wbinvd before a suspend-to-RAM too
(it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
dioes "wbinvd_on_all_cpus()" which does a cross-call etc.

Would you mind experimenting a bit with that wbinvd?

In particular, what happens if you enable it (so it's not hidden by
the SME check), but you move it up to before interrupts are disabled?

I'm wondering if there is some issue with MCE generation and wbinvd
and whatever, and doing it when the CPU is down and interrupts are
disabled causes some system issue..

Does anybody have any other ideas?

                Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>
> It does not work with just once wbinvd(), and it only works with
> removing the wbinvd() for me.  Tom's new post works for me as well
> since my cpu is an Intel i5-4200U.

Intriguing.

It's not like the wbinvd really should be that much of a deal.

I think Tom's patch is fine and should be applied, but it does worry
me a bit that even a single wbinvd makes that much of a difference for
you. There is very little logical reason I can think of that a wbinvd
should make any difference what-so-ever on an i5-4200U.

I wonder if you have some system issues, and wbinvd just happens to
trigger them. But I think we do wbinvd before a suspend-to-RAM too
(it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
dioes "wbinvd_on_all_cpus()" which does a cross-call etc.

Would you mind experimenting a bit with that wbinvd?

In particular, what happens if you enable it (so it's not hidden by
the SME check), but you move it up to before interrupts are disabled?

I'm wondering if there is some issue with MCE generation and wbinvd
and whatever, and doing it when the CPU is down and interrupts are
disabled causes some system issue..

Does anybody have any other ideas?

                Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me.  Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
> 
> Intriguing.
> 
> It's not like the wbinvd really should be that much of a deal.
> 
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
> 
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
> 
> Would you mind experimenting a bit with that wbinvd?
> 
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?

Will play with it more.  Actually I found the hang seems happens
in code of arch/x86/kernel/relocate_kernel_64.S, there is another
wbinvd there as well.

> 
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
> 
> Does anybody have any other ideas?
> 
>                 Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me.  Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
> 
> Intriguing.
> 
> It's not like the wbinvd really should be that much of a deal.
> 
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
> 
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
> 
> Would you mind experimenting a bit with that wbinvd?
> 
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?

Will play with it more.  Actually I found the hang seems happens
in code of arch/x86/kernel/relocate_kernel_64.S, there is another
wbinvd there as well.

> 
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
> 
> Does anybody have any other ideas?
> 
>                 Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me.  Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
> 
> Intriguing.
> 
> It's not like the wbinvd really should be that much of a deal.
> 
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
> 
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
> 
> Would you mind experimenting a bit with that wbinvd?
> 
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?

Did several quick tests, probably need more tests, but till now the
results are:

void stop_this_cpu(void *dummy)
{
=====> add wbinvd here: kexec works
        local_irq_disable();
=====> add wbinvd here: kexec works
        /*
         * Remove this CPU:
         */
        set_cpu_online(smp_processor_id(), false);
=====> add wbinvd here: kexec does not work
 
        disable_local_APIC();
        mcheck_cpu_clear(this_cpu_ptr(&cpu_info));

[snip]

So it seems that it will not work after cpu offined..

> 
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
> 
> Does anybody have any other ideas?
> 
>                 Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 06:14pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > It does not work with just once wbinvd(), and it only works with
> > removing the wbinvd() for me.  Tom's new post works for me as well
> > since my cpu is an Intel i5-4200U.
> 
> Intriguing.
> 
> It's not like the wbinvd really should be that much of a deal.
> 
> I think Tom's patch is fine and should be applied, but it does worry
> me a bit that even a single wbinvd makes that much of a difference for
> you. There is very little logical reason I can think of that a wbinvd
> should make any difference what-so-ever on an i5-4200U.
> 
> I wonder if you have some system issues, and wbinvd just happens to
> trigger them. But I think we do wbinvd before a suspend-to-RAM too
> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
> 
> Would you mind experimenting a bit with that wbinvd?
> 
> In particular, what happens if you enable it (so it's not hidden by
> the SME check), but you move it up to before interrupts are disabled?

Did several quick tests, probably need more tests, but till now the
results are:

void stop_this_cpu(void *dummy)
{
=====> add wbinvd here: kexec works
        local_irq_disable();
=====> add wbinvd here: kexec works
        /*
         * Remove this CPU:
         */
        set_cpu_online(smp_processor_id(), false);
=====> add wbinvd here: kexec does not work
 
        disable_local_APIC();
        mcheck_cpu_clear(this_cpu_ptr(&cpu_info));

[snip]

So it seems that it will not work after cpu offined..

> 
> I'm wondering if there is some issue with MCE generation and wbinvd
> and whatever, and doing it when the CPU is down and interrupts are
> disabled causes some system issue..
> 
> Does anybody have any other ideas?
> 
>                 Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Arjan van de Ven]

> 
> Does anybody have any other ideas?

wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.

I can only imagine a machine check case where a write back to a bad cell causes some parity error
or something...  but it's odd that no other machine checks are reported?
(can the user check for this please)

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Arjan van de Ven]

> 
> Does anybody have any other ideas?

wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.

I can only imagine a machine check case where a write back to a bad cell causes some parity error
or something...  but it's odd that no other machine checks are reported?
(can the user check for this please)

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 6:47 PM, Dave Young <dyoung@redhat.com> wrote:
> Did several quick tests, probably need more tests, but till now the
> results are:
>
> void stop_this_cpu(void *dummy)
> {
> =====> add wbinvd here: kexec works
>         local_irq_disable();
> =====> add wbinvd here: kexec works
>         /*
>          * Remove this CPU:
>          */
>         set_cpu_online(smp_processor_id(), false);
> =====> add wbinvd here: kexec does not work

Funky.

> So it seems that it will not work after cpu offined..

Well, that set_cpu_online() call really just clears a bit in our
'__cpu_online_mask' CPU mask. It doesn't really do anything to the
*hardware*.

But I do wonder if the wbinvd causes an SMI or something on your
system. I _think_ wbinvd causes some external pin to be wiggled just
to tell possible external cache hardware to flush too, and on a system
level that could be tied to some random thing.

And then if we get an SMI/NMI when we've marked the system offline,
maybe we do something odd.

Very odd. But maybe this makes somebody go "Duh, that's because of xyz.."

                Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 6:47 PM, Dave Young <dyoung@redhat.com> wrote:
> Did several quick tests, probably need more tests, but till now the
> results are:
>
> void stop_this_cpu(void *dummy)
> {
> =====> add wbinvd here: kexec works
>         local_irq_disable();
> =====> add wbinvd here: kexec works
>         /*
>          * Remove this CPU:
>          */
>         set_cpu_online(smp_processor_id(), false);
> =====> add wbinvd here: kexec does not work

Funky.

> So it seems that it will not work after cpu offined..

Well, that set_cpu_online() call really just clears a bit in our
'__cpu_online_mask' CPU mask. It doesn't really do anything to the
*hardware*.

But I do wonder if the wbinvd causes an SMI or something on your
system. I _think_ wbinvd causes some external pin to be wiggled just
to tell possible external cache hardware to flush too, and on a system
level that could be tied to some random thing.

And then if we get an SMI/NMI when we've marked the system offline,
maybe we do something odd.

Very odd. But maybe this makes somebody go "Duh, that's because of xyz.."

                Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 06:53pm, Arjan van de Ven wrote:
> > 
> > Does anybody have any other ideas?
> 
> wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
> and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.
> 
> I can only imagine a machine check case where a write back to a bad cell causes some parity error
> or something...  but it's odd that no other machine checks are reported?
> (can the user check for this please)

Could you say more about how to check this? My .config disabled
CONFIG_X86_MCE, should this be enabled?

Thanks
Dave

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 06:53pm, Arjan van de Ven wrote:
> > 
> > Does anybody have any other ideas?
> 
> wbinvd is thankfully not common, but also not rare (MTRR setup and a bunch of other cases)
> and in some other operating systems it happens even more than on Linux.. it's generally not totally broken like this.
> 
> I can only imagine a machine check case where a write back to a bad cell causes some parity error
> or something...  but it's odd that no other machine checks are reported?
> (can the user check for this please)

Could you say more about how to check this? My .config disabled
CONFIG_X86_MCE, should this be enabled?

Thanks
Dave


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
>
> Could you say more about how to check this? My .config disabled
> CONFIG_X86_MCE, should this be enabled?

By all means, try it.

Some pending machine check exception that we have *not* reacted to,
and that is pending around kexec boot could very possibly be relevant.

I forget your exact symptoms. Was it that the _first_ kexec works, but
the second one hangs? Or was that the other unrelated issue? Or have I
confused this with some other report entirely?

               Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Arjan van de Ven]

> Does anybody have any other ideas?

the only other weird case that comes to mind; what happens if there's a line dirty in the caches,
but the memory is now mapped uncached. (Which could happen if kexec does muck with  MTRRs, CR0 or other similar
things in weird ways)... not sure what happens in CPU, a machine check for cache inclusion violations
is not beyond the imagination and might be lethal

this would explain a kexec specific angle versus general normal (but rare) use of wbinvd.


other weird case could be cached mmio (not common, but some gpus and the like can do it)
with iommu/VT-D in the middle, and during kexec VT-D shutting
down the iommu before the wbinvd. This would be... highly odd... but this report already is in highly odd space.

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Linus Torvalds]

On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
>
> Could you say more about how to check this? My .config disabled
> CONFIG_X86_MCE, should this be enabled?

By all means, try it.

Some pending machine check exception that we have *not* reacted to,
and that is pending around kexec boot could very possibly be relevant.

I forget your exact symptoms. Was it that the _first_ kexec works, but
the second one hangs? Or was that the other unrelated issue? Or have I
confused this with some other report entirely?

               Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Arjan van de Ven]

> Does anybody have any other ideas?

the only other weird case that comes to mind; what happens if there's a line dirty in the caches,
but the memory is now mapped uncached. (Which could happen if kexec does muck with  MTRRs, CR0 or other similar
things in weird ways)... not sure what happens in CPU, a machine check for cache inclusion violations
is not beyond the imagination and might be lethal

this would explain a kexec specific angle versus general normal (but rare) use of wbinvd.


other weird case could be cached mmio (not common, but some gpus and the like can do it)
with iommu/VT-D in the middle, and during kexec VT-D shutting
down the iommu before the wbinvd. This would be... highly odd... but this report already is in highly odd space.

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > Could you say more about how to check this? My .config disabled
> > CONFIG_X86_MCE, should this be enabled?
> 
> By all means, try it.
> 
> Some pending machine check exception that we have *not* reacted to,
> and that is pending around kexec boot could very possibly be relevant.
> 
> I forget your exact symptoms. Was it that the _first_ kexec works, but
> the second one hangs? Or was that the other unrelated issue? Or have I
> confused this with some other report entirely?

Yes, 1st kexec works, 2nd hangs.

Rebuilding with MCE enabled now.

Thanks
Dave
> 
>                Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> >
> > Could you say more about how to check this? My .config disabled
> > CONFIG_X86_MCE, should this be enabled?
> 
> By all means, try it.
> 
> Some pending machine check exception that we have *not* reacted to,
> and that is pending around kexec boot could very possibly be relevant.
> 
> I forget your exact symptoms. Was it that the _first_ kexec works, but
> the second one hangs? Or was that the other unrelated issue? Or have I
> confused this with some other report entirely?

Yes, 1st kexec works, 2nd hangs.

Rebuilding with MCE enabled now.

Thanks
Dave
> 
>                Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/18/18 at 11:04am, Dave Young wrote:
> On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> > On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> > >
> > > Could you say more about how to check this? My .config disabled
> > > CONFIG_X86_MCE, should this be enabled?
> > 
> > By all means, try it.
> > 
> > Some pending machine check exception that we have *not* reacted to,
> > and that is pending around kexec boot could very possibly be relevant.
> > 
> > I forget your exact symptoms. Was it that the _first_ kexec works, but
> > the second one hangs? Or was that the other unrelated issue? Or have I
> > confused this with some other report entirely?
> 
> Yes, 1st kexec works, 2nd hangs.
> 
> Rebuilding with MCE enabled now.

dmesg only shows "mce: CPU supports 7 MCE banks"
Nothing special happens with MCE enabled , anyway thank all for
the hints, I will continue to think about this and maybe try more experiments.

> 
> Thanks
> Dave
> > 
> >                Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Dave Young]

On 01/18/18 at 11:04am, Dave Young wrote:
> On 01/17/18 at 07:00pm, Linus Torvalds wrote:
> > On Wed, Jan 17, 2018 at 6:57 PM, Dave Young <dyoung@redhat.com> wrote:
> > >
> > > Could you say more about how to check this? My .config disabled
> > > CONFIG_X86_MCE, should this be enabled?
> > 
> > By all means, try it.
> > 
> > Some pending machine check exception that we have *not* reacted to,
> > and that is pending around kexec boot could very possibly be relevant.
> > 
> > I forget your exact symptoms. Was it that the _first_ kexec works, but
> > the second one hangs? Or was that the other unrelated issue? Or have I
> > confused this with some other report entirely?
> 
> Yes, 1st kexec works, 2nd hangs.
> 
> Rebuilding with MCE enabled now.

dmesg only shows "mce: CPU supports 7 MCE banks"
Nothing special happens with MCE enabled , anyway thank all for
the hints, I will continue to think about this and maybe try more experiments.

> 
> Thanks
> Dave
> > 
> >                Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 8:29 PM, Dave Young wrote:
> On 01/17/18 at 06:14pm, Linus Torvalds wrote:
>> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> It does not work with just once wbinvd(), and it only works with
>>> removing the wbinvd() for me.  Tom's new post works for me as well
>>> since my cpu is an Intel i5-4200U.
>>
>> Intriguing.
>>
>> It's not like the wbinvd really should be that much of a deal.
>>
>> I think Tom's patch is fine and should be applied, but it does worry
>> me a bit that even a single wbinvd makes that much of a difference for
>> you. There is very little logical reason I can think of that a wbinvd
>> should make any difference what-so-ever on an i5-4200U.
>>
>> I wonder if you have some system issues, and wbinvd just happens to
>> trigger them. But I think we do wbinvd before a suspend-to-RAM too
>> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
>> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>>
>> Would you mind experimenting a bit with that wbinvd?
>>
>> In particular, what happens if you enable it (so it's not hidden by
>> the SME check), but you move it up to before interrupts are disabled?
> 
> Will play with it more.  Actually I found the hang seems happens
> in code of arch/x86/kernel/relocate_kernel_64.S, there is another
> wbinvd there as well.

The wbinvd in arch/x86/kernel/relocate_kernel_64.S is only performed if
SME is active, so that one won't be executed on an Intel chip.

Thanks,
Tom

> 
>>
>> I'm wondering if there is some issue with MCE generation and wbinvd
>> and whatever, and doing it when the CPU is down and interrupts are
>> disabled causes some system issue..
>>
>> Does anybody have any other ideas?
>>
>>                 Linus

Re: kexec reboot fails with extra wbinvd introduced for AME SME

[Tom Lendacky]

On 1/17/2018 8:29 PM, Dave Young wrote:
> On 01/17/18 at 06:14pm, Linus Torvalds wrote:
>> On Wed, Jan 17, 2018 at 5:47 PM, Dave Young <dyoung@redhat.com> wrote:
>>>
>>> It does not work with just once wbinvd(), and it only works with
>>> removing the wbinvd() for me.  Tom's new post works for me as well
>>> since my cpu is an Intel i5-4200U.
>>
>> Intriguing.
>>
>> It's not like the wbinvd really should be that much of a deal.
>>
>> I think Tom's patch is fine and should be applied, but it does worry
>> me a bit that even a single wbinvd makes that much of a difference for
>> you. There is very little logical reason I can think of that a wbinvd
>> should make any difference what-so-ever on an i5-4200U.
>>
>> I wonder if you have some system issues, and wbinvd just happens to
>> trigger them. But I think we do wbinvd before a suspend-to-RAM too
>> (it's "ACPI_FLUSH_CPU_CACHE()" in the ACPI code). And the dmr code
>> dioes "wbinvd_on_all_cpus()" which does a cross-call etc.
>>
>> Would you mind experimenting a bit with that wbinvd?
>>
>> In particular, what happens if you enable it (so it's not hidden by
>> the SME check), but you move it up to before interrupts are disabled?
> 
> Will play with it more.  Actually I found the hang seems happens
> in code of arch/x86/kernel/relocate_kernel_64.S, there is another
> wbinvd there as well.

The wbinvd in arch/x86/kernel/relocate_kernel_64.S is only performed if
SME is active, so that one won't be executed on an Intel chip.

Thanks,
Tom

> 
>>
>> I'm wondering if there is some issue with MCE generation and wbinvd
>> and whatever, and doing it when the CPU is down and interrupts are
>> disabled causes some system issue..
>>
>> Does anybody have any other ideas?
>>
>>                 Linus

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec