[FSL P50x0] KVM HV doesn't work anymore

[FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

Hi All,

The RC1 of kernel 5.13 doesn't boot in a virtual e5500 QEMU machine with 
KVM HV anymore. I see in the serial console that the uImage doesn't 
load. I use the following QEMU command for booting:

qemu-system-ppc64 -M ppce500 -cpu e5500 -enable-kvm -m 1024 -kernel 
uImage-5.13 -drive format=raw,file=MintPPC32-X5000.img,index=0,if=virtio 
-netdev user,id=mynet0 -device e1000,netdev=mynet0 -append "rw 
root=/dev/vda" -device virtio-vga -device virtio-mouse-pci -device 
virtio-keyboard-pci -device pci-ohci,id=newusb -device 
usb-audio,bus=newusb.0 -smp 4

The kernel boots without KVM HV.

Have you already tested KVM HV with the kernel 5.13?

Thanks,
Christian

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

Hi All,

I bisected today [1] and the bisecting itself was OK but the reverting 
of the bad commit doesn't solve the issue. Do you have an idea which 
commit could be resposible for this issue? Maybe the bisecting wasn't 
successful. I will look in the kernel git log. Maybe there is a commit 
that affected KVM HV on FSL P50x0 machines.

Thanks,
Christian

[1] https://forum.hyperion-entertainment.com/viewtopic.php?p=53209#p53209

On 14 May 2021 at 10:10 am, Christian Zigotzky wrote:
> Hi All,
>
> The RC1 of kernel 5.13 doesn't boot in a virtual e5500 QEMU machine 
> with KVM HV anymore. I see in the serial console that the uImage 
> doesn't load. I use the following QEMU command for booting:
>
> qemu-system-ppc64 -M ppce500 -cpu e5500 -enable-kvm -m 1024 -kernel 
> uImage-5.13 -drive 
> format=raw,file=MintPPC32-X5000.img,index=0,if=virtio -netdev 
> user,id=mynet0 -device e1000,netdev=mynet0 -append "rw root=/dev/vda" 
> -device virtio-vga -device virtio-mouse-pci -device 
> virtio-keyboard-pci -device pci-ohci,id=newusb -device 
> usb-audio,bus=newusb.0 -smp 4
>
> The kernel boots without KVM HV.
>
> Have you already tested KVM HV with the kernel 5.13?
>
> Thanks,
> Christian
>
>

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christophe Leroy]

Le 15/05/2021 à 11:48, Christian Zigotzky a écrit :
> Hi All,
> 
> I bisected today [1] and the bisecting itself was OK but the reverting of the bad commit doesn't 
> solve the issue. Do you have an idea which commit could be resposible for this issue? Maybe the 
> bisecting wasn't successful. I will look in the kernel git log. Maybe there is a commit that 
> affected KVM HV on FSL P50x0 machines.

If the uImage doesn't load, it may be because of the size of uImage.

See https://github.com/linuxppc/issues/issues/208

Is there a significant size difference with and without KVM HV ?

Maybe you can try to remove another option to reduce the size of the uImage.

Or if you are using gzipped uImage you can try with an lzma uImage. You can find a way to get an 
lzma uImage here: https://github.com/linuxppc/issues/issues/208#issuecomment-477479951

Christophe

> 
> Thanks,
> Christian
> 
> [1] https://forum.hyperion-entertainment.com/viewtopic.php?p=53209#p53209
> 
> On 14 May 2021 at 10:10 am, Christian Zigotzky wrote:
>> Hi All,
>>
>> The RC1 of kernel 5.13 doesn't boot in a virtual e5500 QEMU machine with KVM HV anymore. I see in 
>> the serial console that the uImage doesn't load. I use the following QEMU command for booting:
>>
>> qemu-system-ppc64 -M ppce500 -cpu e5500 -enable-kvm -m 1024 -kernel uImage-5.13 -drive 
>> format=raw,file=MintPPC32-X5000.img,index=0,if=virtio -netdev user,id=mynet0 -device 
>> e1000,netdev=mynet0 -append "rw root=/dev/vda" -device virtio-vga -device virtio-mouse-pci -device 
>> virtio-keyboard-pci -device pci-ohci,id=newusb -device usb-audio,bus=newusb.0 -smp 4
>>
>> The kernel boots without KVM HV.
>>
>> Have you already tested KVM HV with the kernel 5.13?
>>
>> Thanks,
>> Christian
>>
>>

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

On 15 May 2021 at 12:08pm Christophe Leroy wrote:
>
>
> Le 15/05/2021 à 11:48, Christian Zigotzky a écrit :
>> Hi All,
>>
>> I bisected today [1] and the bisecting itself was OK but the 
>> reverting of the bad commit doesn't solve the issue. Do you have an 
>> idea which commit could be resposible for this issue? Maybe the 
>> bisecting wasn't successful. I will look in the kernel git log. Maybe 
>> there is a commit that affected KVM HV on FSL P50x0 machines.
>
> If the uImage doesn't load, it may be because of the size of uImage.
>
> See https://github.com/linuxppc/issues/issues/208
>
> Is there a significant size difference with and without KVM HV ?
>
> Maybe you can try to remove another option to reduce the size of the 
> uImage.
I tried it but it doesn't solve the issue. The uImage works without KVM 
HV in a virtual e5500 QEMU machine.

-Christian
>
> Or if you are using gzipped uImage you can try with an lzma uImage. 
> You can find a way to get an lzma uImage here: 
> https://github.com/linuxppc/issues/issues/208#issuecomment-477479951
>
> Christophe
>
>>
>> Thanks,
>> Christian
>>
>> [1] 
>> https://forum.hyperion-entertainment.com/viewtopic.php?p=53209#p53209
>>
>> On 14 May 2021 at 10:10 am, Christian Zigotzky wrote:
>>> Hi All,
>>>
>>> The RC1 of kernel 5.13 doesn't boot in a virtual e5500 QEMU machine 
>>> with KVM HV anymore. I see in the serial console that the uImage 
>>> doesn't load. I use the following QEMU command for booting:
>>>
>>> qemu-system-ppc64 -M ppce500 -cpu e5500 -enable-kvm -m 1024 -kernel 
>>> uImage-5.13 -drive 
>>> format=raw,file=MintPPC32-X5000.img,index=0,if=virtio -netdev 
>>> user,id=mynet0 -device e1000,netdev=mynet0 -append "rw 
>>> root=/dev/vda" -device virtio-vga -device virtio-mouse-pci -device 
>>> virtio-keyboard-pci -device pci-ohci,id=newusb -device 
>>> usb-audio,bus=newusb.0 -smp 4
>>>
>>> The kernel boots without KVM HV.
>>>
>>> Have you already tested KVM HV with the kernel 5.13?
>>>
>>> Thanks,
>>> Christian
>>>
>>>

Re: [FSL P50x0] KVM HV doesn't work anymore

[Nicholas Piggin]

Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
> On 15 May 2021 at 12:08pm Christophe Leroy wrote:
>>
>>
>> Le 15/05/2021 à 11:48, Christian Zigotzky a écrit :
>>> Hi All,
>>>
>>> I bisected today [1] and the bisecting itself was OK but the 
>>> reverting of the bad commit doesn't solve the issue. Do you have an 
>>> idea which commit could be resposible for this issue? Maybe the 
>>> bisecting wasn't successful. I will look in the kernel git log. Maybe 
>>> there is a commit that affected KVM HV on FSL P50x0 machines.
>>
>> If the uImage doesn't load, it may be because of the size of uImage.
>>
>> See https://github.com/linuxppc/issues/issues/208
>>
>> Is there a significant size difference with and without KVM HV ?
>>
>> Maybe you can try to remove another option to reduce the size of the 
>> uImage.
> I tried it but it doesn't solve the issue. The uImage works without KVM 
> HV in a virtual e5500 QEMU machine.

Any more progress with this? I would say that bisect might have just
been a bit unstable and maybe by chance some things did not crash so
it's pointing to the wrong patch.

Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?

Between that looks like some KVM MMU rework. You could try the patch
before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU 
notifier callbacks"). That won't revert cleanly so just try run the
tree at that point. If it works, test the patch and see if it fails.

Thanks,
Nick

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

On 17 May 2021 at 09:42am, Nicholas Piggin wrote:
> Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
>> On 15 May 2021 at 12:08pm Christophe Leroy wrote:
>>>
>>> Le 15/05/2021 à 11:48, Christian Zigotzky a écrit :
>>>> Hi All,
>>>>
>>>> I bisected today [1] and the bisecting itself was OK but the
>>>> reverting of the bad commit doesn't solve the issue. Do you have an
>>>> idea which commit could be resposible for this issue? Maybe the
>>>> bisecting wasn't successful. I will look in the kernel git log. Maybe
>>>> there is a commit that affected KVM HV on FSL P50x0 machines.
>>> If the uImage doesn't load, it may be because of the size of uImage.
>>>
>>> See https://github.com/linuxppc/issues/issues/208
>>>
>>> Is there a significant size difference with and without KVM HV ?
>>>
>>> Maybe you can try to remove another option to reduce the size of the
>>> uImage.
>> I tried it but it doesn't solve the issue. The uImage works without KVM
>> HV in a virtual e5500 QEMU machine.
> Any more progress with this? I would say that bisect might have just
> been a bit unstable and maybe by chance some things did not crash so
> it's pointing to the wrong patch.
>
> Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?
>
> Between that looks like some KVM MMU rework. You could try the patch
> before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU
> notifier callbacks"). That won't revert cleanly so just try run the
> tree at that point. If it works, test the patch and see if it fails.
>
> Thanks,
> Nick
Hi Nick,

Thanks a lot for your answer. Yes, there is a little bit of progress. 
The RC2 of kernel 5.13 successfully boots with -smp 3 in a virtual e5500 
QEMU machine.
-smp 4 doesn't work anymore since the PowerPC updates 5.13-2. I used 
-smp 4 before 5.13 because my FSL P5040 machine has 4 cores.

Could you please post a patch for reverting the commit before 
b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU notifier callbacks")?

Thanks in advance,

Christian

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

> On 17. May 2021, at 11:43, Christian Zigotzky <chzigotzky@xenosoft.de> wrote:
> 
> On 17 May 2021 at 09:42am, Nicholas Piggin wrote:
>> Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
>>> On 15 May 2021 at 12:08pm Christophe Leroy wrote:
>>>> 
>>>>> Le 15/05/2021 à 11:48, Christian Zigotzky a écrit :
>>>>>> Hi All,
>>>>>> 
>>>>>> I bisected today [1] and the bisecting itself was OK but the
>>>>>> reverting of the bad commit doesn't solve the issue. Do you have an
>>>>>> idea which commit could be resposible for this issue? Maybe the
>>>>>> bisecting wasn't successful. I will look in the kernel git log. Maybe
>>>>>> there is a commit that affected KVM HV on FSL P50x0 machines.
>>>>> If the uImage doesn't load, it may be because of the size of uImage.
>>>>> 
>>>>> See https://github.com/linuxppc/issues/issues/208
>>>>> 
>>>>> Is there a significant size difference with and without KVM HV ?
>>>>> 
>>>>> Maybe you can try to remove another option to reduce the size of the
>>>>> uImage.
>>> I tried it but it doesn't solve the issue. The uImage works without KVM
>>> HV in a virtual e5500 QEMU machine.
>> Any more progress with this? I would say that bisect might have just
>> been a bit unstable and maybe by chance some things did not crash so
>> it's pointing to the wrong patch.
>> 
>> Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?
>> 
>> Between that looks like some KVM MMU rework. You could try the patch
>> before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU
>> notifier callbacks"). That won't revert cleanly so just try run the
>> tree at that point. If it works, test the patch and see if it fails.
>> 
>> Thanks,
>> Nick
> Hi Nick,
> 
> Thanks a lot for your answer. Yes, there is a little bit of progress. The RC2 of kernel 5.13 successfully boots with -smp 3 in a virtual e5500 QEMU machine.
> -smp 4 doesn't work anymore since the PowerPC updates 5.13-2. I used -smp 4 before 5.13 because my FSL P5040 machine has 4 cores.
> 
> Could you please post a patch for reverting the commit before b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU notifier callbacks")?
> 
> Thanks in advance,
> 
> Christian
> 
> 
For me it is ok to work with -smp 1, 2, and 3 but I am curious why -smp 4 doesn’t work.

-Christian

Re: [FSL P50x0] KVM HV doesn't work anymore

[Nicholas Piggin]

Excerpts from Christian Zigotzky's message of May 17, 2021 7:42 pm:
> On 17 May 2021 at 09:42am, Nicholas Piggin wrote:
>> Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
>>> On 15 May 2021 at 12:08pm Christophe Leroy wrote:
>>>>
>>>> Le 15/05/2021 à 11:48, Christian Zigotzky a écrit :
>>>>> Hi All,
>>>>>
>>>>> I bisected today [1] and the bisecting itself was OK but the
>>>>> reverting of the bad commit doesn't solve the issue. Do you have an
>>>>> idea which commit could be resposible for this issue? Maybe the
>>>>> bisecting wasn't successful. I will look in the kernel git log. Maybe
>>>>> there is a commit that affected KVM HV on FSL P50x0 machines.
>>>> If the uImage doesn't load, it may be because of the size of uImage.
>>>>
>>>> See https://github.com/linuxppc/issues/issues/208
>>>>
>>>> Is there a significant size difference with and without KVM HV ?
>>>>
>>>> Maybe you can try to remove another option to reduce the size of the
>>>> uImage.
>>> I tried it but it doesn't solve the issue. The uImage works without KVM
>>> HV in a virtual e5500 QEMU machine.
>> Any more progress with this? I would say that bisect might have just
>> been a bit unstable and maybe by chance some things did not crash so
>> it's pointing to the wrong patch.
>>
>> Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?
>>
>> Between that looks like some KVM MMU rework. You could try the patch
>> before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU
>> notifier callbacks"). That won't revert cleanly so just try run the
>> tree at that point. If it works, test the patch and see if it fails.
>>
>> Thanks,
>> Nick
> Hi Nick,
> 
> Thanks a lot for your answer. Yes, there is a little bit of progress. 
> The RC2 of kernel 5.13 successfully boots with -smp 3 in a virtual e5500 
> QEMU machine.
> -smp 4 doesn't work anymore since the PowerPC updates 5.13-2. I used 
> -smp 4 before 5.13 because my FSL P5040 machine has 4 cores.
> 
> Could you please post a patch for reverting the commit before 
> b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU notifier callbacks")?

You could `git checkout b1c5356e873c~1`

Thanks,
Nick
> 

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

On 19 May 2021 at 09:57 am, Nicholas Piggin wrote:
> Excerpts from Christian Zigotzky's message of May 17, 2021 7:42 pm:
>> On 17 May 2021 at 09:42am, Nicholas Piggin wrote:
>>> Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
>>>> I tried it but it doesn't solve the issue. The uImage works without 
>>>> KVM
>>>> HV in a virtual e5500 QEMU machine.
>>> Any more progress with this? I would say that bisect might have just
>>> been a bit unstable and maybe by chance some things did not crash so
>>> it's pointing to the wrong patch.
>>>
>>> Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?
>>>
>>> Between that looks like some KVM MMU rework. You could try the patch
>>> before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU
>>> notifier callbacks"). That won't revert cleanly so just try run the
>>> tree at that point. If it works, test the patch and see if it fails.
>>>
>>> Thanks,
>>> Nick
>> Hi Nick,
>>
>> Thanks a lot for your answer. Yes, there is a little bit of progress.
>> The RC2 of kernel 5.13 successfully boots with -smp 3 in a virtual e5500
>> QEMU machine.
>> -smp 4 doesn't work anymore since the PowerPC updates 5.13-2. I used
>> -smp 4 before 5.13 because my FSL P5040 machine has 4 cores.
>>
>> Could you please post a patch for reverting the commit before
>> b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU notifier callbacks")?
> You could `git checkout b1c5356e873c~1`
>
> Thanks,
> Nick
Hi Nick,

Thanks for your answer. I checked out the commit b1c5356e873c~1 (HEAD is 
now at d923ff258423 KVM: MIPS/MMU: Convert to the gfn-based MMU notifier 
callbacks).
The kernel boots with '-smp 4' without any problems.
After that I patched with the probable first bad commit (KVM: PPC: 
Convert to the gfn-based MMU notifier callbacks). The kernel also boots 
with this patch. That means, this isn't the first bad commit.
Further information: 
https://forum.hyperion-entertainment.com/viewtopic.php?p=53267#p53267

Cheers,
Christian

Re: [FSL P50x0] KVM HV doesn't work anymore

[Nicholas Piggin]

Excerpts from Christian Zigotzky's message of May 19, 2021 9:52 pm:
> On 19 May 2021 at 09:57 am, Nicholas Piggin wrote:
>> Excerpts from Christian Zigotzky's message of May 17, 2021 7:42 pm:
>>> On 17 May 2021 at 09:42am, Nicholas Piggin wrote:
>>>> Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
>>>>> I tried it but it doesn't solve the issue. The uImage works without 
>>>>> KVM
>>>>> HV in a virtual e5500 QEMU machine.
>>>> Any more progress with this? I would say that bisect might have just
>>>> been a bit unstable and maybe by chance some things did not crash so
>>>> it's pointing to the wrong patch.
>>>>
>>>> Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?
>>>>
>>>> Between that looks like some KVM MMU rework. You could try the patch
>>>> before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU
>>>> notifier callbacks"). That won't revert cleanly so just try run the
>>>> tree at that point. If it works, test the patch and see if it fails.
>>>>
>>>> Thanks,
>>>> Nick
>>> Hi Nick,
>>>
>>> Thanks a lot for your answer. Yes, there is a little bit of progress.
>>> The RC2 of kernel 5.13 successfully boots with -smp 3 in a virtual e5500
>>> QEMU machine.
>>> -smp 4 doesn't work anymore since the PowerPC updates 5.13-2. I used
>>> -smp 4 before 5.13 because my FSL P5040 machine has 4 cores.
>>>
>>> Could you please post a patch for reverting the commit before
>>> b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU notifier callbacks")?
>> You could `git checkout b1c5356e873c~1`
>>
>> Thanks,
>> Nick
> Hi Nick,
> 
> Thanks for your answer. I checked out the commit b1c5356e873c~1 (HEAD is 
> now at d923ff258423 KVM: MIPS/MMU: Convert to the gfn-based MMU notifier 
> callbacks).
> The kernel boots with '-smp 4' without any problems.
> After that I patched with the probable first bad commit (KVM: PPC: 
> Convert to the gfn-based MMU notifier callbacks). The kernel also boots 
> with this patch. That means, this isn't the first bad commit.
> Further information: 
> https://forum.hyperion-entertainment.com/viewtopic.php?p=53267#p53267

Hmm, okay that probably rules out those notifier changes then.

Can you remind me were you able to rule these out as suspects?

8f6cc75a97d1 powerpc: move norestart trap flag to bit 0
8dc7f0229b78 powerpc: remove partial register save logic
c45ba4f44f6b powerpc: clean up do_page_fault
d738ee8d56de powerpc/64e/interrupt: handle bad_page_fault in C
ceff77efa4f8 powerpc/64e/interrupt: Use new interrupt context tracking scheme
097157e16cf8 powerpc/64e/interrupt: reconcile irq soft-mask state in C
3db8aa10de9a powerpc/64e/interrupt: NMI save irq soft-mask state in C
0c2472de23ae powerpc/64e/interrupt: use new interrupt return
dc6231821a14 powerpc/interrupt: update common interrupt code for
4228b2c3d20e powerpc/64e/interrupt: always save nvgprs on interrupt
5a5a893c4ad8 powerpc/syscall: switch user_exit_irqoff and trace_hardirqs_off order

Thanks,
Nick

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

On 20 May 21 at 01:07am, Nicholas Piggin wrote:
> Excerpts from Christian Zigotzky's message of May 19, 2021 9:52 pm:
>> On 19 May 2021 at 09:57 am, Nicholas Piggin wrote:
>>> Excerpts from Christian Zigotzky's message of May 17, 2021 7:42 pm:
>>>> On 17 May 2021 at 09:42am, Nicholas Piggin wrote:
>>>>> Excerpts from Christian Zigotzky's message of May 15, 2021 11:46 pm:
>>>>>> I tried it but it doesn't solve the issue. The uImage works without
>>>>>> KVM
>>>>>> HV in a virtual e5500 QEMU machine.
>>>>> Any more progress with this? I would say that bisect might have just
>>>>> been a bit unstable and maybe by chance some things did not crash so
>>>>> it's pointing to the wrong patch.
>>>>>
>>>>> Upstream merge of powerpc-5.13-1 was good and powerpc-5.13-2 was bad?
>>>>>
>>>>> Between that looks like some KVM MMU rework. You could try the patch
>>>>> before this one b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU
>>>>> notifier callbacks"). That won't revert cleanly so just try run the
>>>>> tree at that point. If it works, test the patch and see if it fails.
>>>>>
>>>>> Thanks,
>>>>> Nick
>>>> Hi Nick,
>>>>
>>>> Thanks a lot for your answer. Yes, there is a little bit of progress.
>>>> The RC2 of kernel 5.13 successfully boots with -smp 3 in a virtual e5500
>>>> QEMU machine.
>>>> -smp 4 doesn't work anymore since the PowerPC updates 5.13-2. I used
>>>> -smp 4 before 5.13 because my FSL P5040 machine has 4 cores.
>>>>
>>>> Could you please post a patch for reverting the commit before
>>>> b1c5356e873c ("KVM: PPC: Convert to the gfn-based MMU notifier callbacks")?
>>> You could `git checkout b1c5356e873c~1`
>>>
>>> Thanks,
>>> Nick
>> Hi Nick,
>>
>> Thanks for your answer. I checked out the commit b1c5356e873c~1 (HEAD is
>> now at d923ff258423 KVM: MIPS/MMU: Convert to the gfn-based MMU notifier
>> callbacks).
>> The kernel boots with '-smp 4' without any problems.
>> After that I patched with the probable first bad commit (KVM: PPC:
>> Convert to the gfn-based MMU notifier callbacks). The kernel also boots
>> with this patch. That means, this isn't the first bad commit.
>> Further information:
>> https://forum.hyperion-entertainment.com/viewtopic.php?p=53267#p53267
> Hmm, okay that probably rules out those notifier changes then.
>
> Can you remind me were you able to rule these out as suspects?
>
> 8f6cc75a97d1 powerpc: move norestart trap flag to bit 0
> 8dc7f0229b78 powerpc: remove partial register save logic
> c45ba4f44f6b powerpc: clean up do_page_fault
> d738ee8d56de powerpc/64e/interrupt: handle bad_page_fault in C
> ceff77efa4f8 powerpc/64e/interrupt: Use new interrupt context tracking scheme
> 097157e16cf8 powerpc/64e/interrupt: reconcile irq soft-mask state in C
> 3db8aa10de9a powerpc/64e/interrupt: NMI save irq soft-mask state in C
> 0c2472de23ae powerpc/64e/interrupt: use new interrupt return
> dc6231821a14 powerpc/interrupt: update common interrupt code for
> 4228b2c3d20e powerpc/64e/interrupt: always save nvgprs on interrupt
> 5a5a893c4ad8 powerpc/syscall: switch user_exit_irqoff and trace_hardirqs_off order
>
> Thanks,
> Nick
Hi Nick,

Thanks for your answer. Smp 4 still doesn't work on quad core e5500 
CPUs. I use -smp 3 currently. Shall I checkout the commits above (in 
your last answer) and test them? Do you prefer a commit for testing?

Thanks,
Christian

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

On 20 May 2021 at 01:07am, Nicholas Piggin wrote:
> Hmm, okay that probably rules out those notifier changes then.
> Can you remind me were you able to rule these out as suspects?
>
> 8f6cc75a97d1 powerpc: move norestart trap flag to bit 0
> 8dc7f0229b78 powerpc: remove partial register save logic
> c45ba4f44f6b powerpc: clean up do_page_fault
> d738ee8d56de powerpc/64e/interrupt: handle bad_page_fault in C
> ceff77efa4f8 powerpc/64e/interrupt: Use new interrupt context tracking scheme
> 097157e16cf8 powerpc/64e/interrupt: reconcile irq soft-mask state in C
> 3db8aa10de9a powerpc/64e/interrupt: NMI save irq soft-mask state in C
> 0c2472de23ae powerpc/64e/interrupt: use new interrupt return
> dc6231821a14 powerpc/interrupt: update common interrupt code for
> 4228b2c3d20e powerpc/64e/interrupt: always save nvgprs on interrupt
> 5a5a893c4ad8 powerpc/syscall: switch user_exit_irqoff and trace_hardirqs_off order
>
> Thanks,
> Nick
Hi Nick,

I tested these commits above today and all works with -smp 4. [1]

Smp 4 still doesn't work with the RC4 of kernel 5.13 on quad core e5500 
CPUs with KVM HV. I use -smp 3 currently.

What shall I test next?

Thanks,
Christian

[1] https://forum.hyperion-entertainment.com/viewtopic.php?p=53367#p53367

Re: [FSL P50x0] KVM HV doesn't work anymore

[Christian Zigotzky]

On 02 June 2021 at 01:26pm, Christian Zigotzky wrote:
> On 20 May 2021 at 01:07am, Nicholas Piggin wrote:
>> Hmm, okay that probably rules out those notifier changes then.
>> Can you remind me were you able to rule these out as suspects?
>>
>> 8f6cc75a97d1 powerpc: move norestart trap flag to bit 0
>> 8dc7f0229b78 powerpc: remove partial register save logic
>> c45ba4f44f6b powerpc: clean up do_page_fault
>> d738ee8d56de powerpc/64e/interrupt: handle bad_page_fault in C
>> ceff77efa4f8 powerpc/64e/interrupt: Use new interrupt context 
>> tracking scheme
>> 097157e16cf8 powerpc/64e/interrupt: reconcile irq soft-mask state in C
>> 3db8aa10de9a powerpc/64e/interrupt: NMI save irq soft-mask state in C
>> 0c2472de23ae powerpc/64e/interrupt: use new interrupt return
>> dc6231821a14 powerpc/interrupt: update common interrupt code for
>> 4228b2c3d20e powerpc/64e/interrupt: always save nvgprs on interrupt
>> 5a5a893c4ad8 powerpc/syscall: switch user_exit_irqoff and 
>> trace_hardirqs_off order
>>
>> Thanks,
>> Nick
> Hi Nick,
>
> I tested these commits above today and all works with -smp 4. [1]
>
> Smp 4 still doesn't work with the RC4 of kernel 5.13 on quad core 
> e5500 CPUs with KVM HV. I use -smp 3 currently.
>
> What shall I test next?
>
> Thanks,
> Christian
>
> [1] https://forum.hyperion-entertainment.com/viewtopic.php?p=53367#p53367
Hi All,

I tested the RC5 of kernel 5.13 today. Unfortunately the KVM HV issue 
still exists.
I also figured out, that '-smp 2' doesn't work either.

Summary:

-smp 1 -> works
-smp 2 -> doesn't work
-smp 3 -> works
-smp 4 -> doesn't work

Cheers,
Christian

Re: [FSL P50x0] KVM HV doesn't work anymore

[Nicholas Piggin]

Excerpts from Christian Zigotzky's message of June 7, 2021 5:21 pm:
> On 02 June 2021 at 01:26pm, Christian Zigotzky wrote:
>> On 20 May 2021 at 01:07am, Nicholas Piggin wrote:
>>> Hmm, okay that probably rules out those notifier changes then.
>>> Can you remind me were you able to rule these out as suspects?
>>>
>>> 8f6cc75a97d1 powerpc: move norestart trap flag to bit 0
>>> 8dc7f0229b78 powerpc: remove partial register save logic
>>> c45ba4f44f6b powerpc: clean up do_page_fault
>>> d738ee8d56de powerpc/64e/interrupt: handle bad_page_fault in C
>>> ceff77efa4f8 powerpc/64e/interrupt: Use new interrupt context 
>>> tracking scheme
>>> 097157e16cf8 powerpc/64e/interrupt: reconcile irq soft-mask state in C
>>> 3db8aa10de9a powerpc/64e/interrupt: NMI save irq soft-mask state in C
>>> 0c2472de23ae powerpc/64e/interrupt: use new interrupt return
>>> dc6231821a14 powerpc/interrupt: update common interrupt code for
>>> 4228b2c3d20e powerpc/64e/interrupt: always save nvgprs on interrupt
>>> 5a5a893c4ad8 powerpc/syscall: switch user_exit_irqoff and 
>>> trace_hardirqs_off order
>>>
>>> Thanks,
>>> Nick
>> Hi Nick,
>>
>> I tested these commits above today and all works with -smp 4. [1]
>>
>> Smp 4 still doesn't work with the RC4 of kernel 5.13 on quad core 
>> e5500 CPUs with KVM HV. I use -smp 3 currently.
>>
>> What shall I test next?
>>
>> Thanks,
>> Christian
>>
>> [1] https://forum.hyperion-entertainment.com/viewtopic.php?p=53367#p53367
> Hi All,
> 
> I tested the RC5 of kernel 5.13 today. Unfortunately the KVM HV issue 
> still exists.
> I also figured out, that '-smp 2' doesn't work either.
> 
> Summary:
> 
> -smp 1 -> works
> -smp 2 -> doesn't work
> -smp 3 -> works
> -smp 4 -> doesn't work

Sorry, I'm not able to see anything, if the KVM patches were okay and 
the 64e interrupt series. I don't know why the -smp behaviour would make

I can't think of why the -smp behaviour would make a difference except 
for a strange race. Doing another bisect might be the only way to get
to the bottom of it.

But before that you could try get some data about why the guest stops?
Get some samples of CPU registers when it gets stuck and see if you can
see if it is stuck in a loop of interupts or something.

I don't know if qemu can log much from KVM execution so you might have
to just run info registers a dozen times on each CPU (`cpu 1` will 
change to CPU 1 in the qemu monitor).

Thanks,
Nick