* Boot issue with the latest Git kernel
@ 2020-06-04 14:16 Christian Zigotzky
2020-06-04 14:26 ` Christophe Leroy
0 siblings, 1 reply; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-04 14:16 UTC (permalink / raw)
To: linuxppc-dev, Christoph Hellwig, Darren Stevens; +Cc: R.T.Dickinson
Hi All,
I tested the latest Git kernel today. [1]. Unfortunately it doesn't boot
on my PowerPC machines.
Could you please test the latest Git kernel with your PowerPC machine?
BTW, it doesn't boot in a virtual QEMU PowerPC machine either.
Thanks,
Christian
[1]
https://forum.hyperion-entertainment.com/viewtopic.php?f=58&p=50758&sid=3f816f078869510dea9fe4baca3605db#p50758
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-04 14:16 Boot issue with the latest Git kernel Christian Zigotzky
@ 2020-06-04 14:26 ` Christophe Leroy
2020-06-04 14:29 ` Christophe Leroy
0 siblings, 1 reply; 29+ messages in thread
From: Christophe Leroy @ 2020-06-04 14:26 UTC (permalink / raw)
To: Christian Zigotzky, linuxppc-dev, Christoph Hellwig, Darren Stevens
Cc: R.T.Dickinson
Hi,
Le 04/06/2020 à 16:16, Christian Zigotzky a écrit :
> Hi All,
>
> I tested the latest Git kernel today. [1]. Unfortunately it doesn't boot
> on my PowerPC machines.
>
> Could you please test the latest Git kernel with your PowerPC machine?
>
> BTW, it doesn't boot in a virtual QEMU PowerPC machine either.
>
Which machine/platform ? Which defconfig are you using ?
Christophe
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-04 14:26 ` Christophe Leroy
@ 2020-06-04 14:29 ` Christophe Leroy
2020-06-04 15:46 ` Christian Zigotzky
2020-06-04 15:53 ` Christian Zigotzky
0 siblings, 2 replies; 29+ messages in thread
From: Christophe Leroy @ 2020-06-04 14:29 UTC (permalink / raw)
To: Christian Zigotzky, linuxppc-dev, Christoph Hellwig, Darren Stevens
Cc: R.T.Dickinson
Le 04/06/2020 à 16:26, Christophe Leroy a écrit :
> Hi,
>
>
> Le 04/06/2020 à 16:16, Christian Zigotzky a écrit :
>> Hi All,
>>
>> I tested the latest Git kernel today. [1]. Unfortunately it doesn't
>> boot on my PowerPC machines.
>>
>> Could you please test the latest Git kernel with your PowerPC machine?
>>
>> BTW, it doesn't boot in a virtual QEMU PowerPC machine either.
>>
>
> Which machine/platform ? Which defconfig are you using ?
>
And are you able to perform a 'git bisect' to identify the guilty commit ?
Thanks
Christophe
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-04 14:29 ` Christophe Leroy
@ 2020-06-04 15:46 ` Christian Zigotzky
2020-06-04 15:53 ` Christian Zigotzky
1 sibling, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-04 15:46 UTC (permalink / raw)
To: Christophe Leroy
Cc: Darren Stevens, linuxppc-dev, Christoph Hellwig, R.T.Dickinson
[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]
Hello Christophe,
I tested it on my Nemo board with a P.A. Semi PA6T CPU [1], on my Cyrus+ board with a FSL P5040 CPU [2] and in a virtual e5500 QEMU machine. You can find the kernel configs in the following package.
Link: http://www.xenosoft.de/linux-image-5.8-alpha1-X1000_X5000.tar.gz
Cheers,
Christian
[1] https://en.m.wikipedia.org/wiki/AmigaOne_X1000
[2] https://www.amigaos.net/hardware/133/amigaone-x5000
> On 4. Jun 2020, at 16:29, Christophe Leroy <christophe.leroy@csgroup.eu> wrote:
>
>
>
>> Le 04/06/2020 à 16:26, Christophe Leroy a écrit :
>> Hi,
>>> Le 04/06/2020 à 16:16, Christian Zigotzky a écrit :
>>> Hi All,
>>>
>>> I tested the latest Git kernel today. [1]. Unfortunately it doesn't boot on my PowerPC machines.
>>>
>>> Could you please test the latest Git kernel with your PowerPC machine?
>>>
>>> BTW, it doesn't boot in a virtual QEMU PowerPC machine either.
>>>
>> Which machine/platform ? Which defconfig are you using ?
>
>
> And are you able to perform a 'git bisect' to identify the guilty commit ?
>
> Thanks
> Christophe
[-- Attachment #2: Type: text/html, Size: 2635 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-04 14:29 ` Christophe Leroy
2020-06-04 15:46 ` Christian Zigotzky
@ 2020-06-04 15:53 ` Christian Zigotzky
2020-06-04 17:15 ` Christophe Leroy
1 sibling, 1 reply; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-04 15:53 UTC (permalink / raw)
To: Christophe Leroy
Cc: Darren Stevens, linuxppc-dev, Christoph Hellwig, R.T.Dickinson
> On 4. Jun 2020, at 16:29, Christophe Leroy <christophe.leroy@csgroup.eu> wrote:
>
> And are you able to perform a 'git bisect' to identify the guilty commit ?
>
> Thanks
> Christophe
Hello Christophe,
Unfortunately I haven’t had time to bisect the latest Git kernel. Does it boot on your PowerPC machine?
Thanks,
Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-04 15:53 ` Christian Zigotzky
@ 2020-06-04 17:15 ` Christophe Leroy
2020-06-05 16:23 ` Christian Zigotzky
0 siblings, 1 reply; 29+ messages in thread
From: Christophe Leroy @ 2020-06-04 17:15 UTC (permalink / raw)
To: Christian Zigotzky
Cc: Darren Stevens, linuxppc-dev, Christoph Hellwig, R.T.Dickinson
Le 04/06/2020 à 17:53, Christian Zigotzky a écrit :
>
>
>> On 4. Jun 2020, at 16:29, Christophe Leroy <christophe.leroy@csgroup.eu> wrote:
>>
>> And are you able to perform a 'git bisect' to identify the guilty commit ?
>>
>> Thanks
>> Christophe
>
> Hello Christophe,
>
> Unfortunately I haven’t had time to bisect the latest Git kernel. Does it boot on your PowerPC machine?
>
Yes today's linux-next boots on my powerpc 8xx board.
Christophe
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-04 17:15 ` Christophe Leroy
@ 2020-06-05 16:23 ` Christian Zigotzky
2020-06-07 13:27 ` Christian Zigotzky
2020-06-07 14:07 ` Aneesh Kumar K.V
0 siblings, 2 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-05 16:23 UTC (permalink / raw)
To: Christophe Leroy, linuxppc-dev, jroedel
Cc: Darren Stevens, Christoph Hellwig, R.T.Dickinson, Christian Zigotzky
On 04 June 2020 at 7:15 pm, Christophe Leroy wrote:
> Yes today's linux-next boots on my powerpc 8xx board.
>
> Christophe
Hello Christophe,
Thanks for testing.
I was able to perform a 'git bisect' [1] and identified the bad commit.
[2] I reverted this commit and after that the kernel boots and works
without any problems.
Could you please check this commit?
Thanks,
Christian
[1] https://forum.hyperion-entertainment.com/viewtopic.php?p=50772#p50772
[2]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2ba3e6947aed9bb9575eb1603c0ac6e39185d32a
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-05 16:23 ` Christian Zigotzky
@ 2020-06-07 13:27 ` Christian Zigotzky
2020-06-07 14:07 ` Aneesh Kumar K.V
1 sibling, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-07 13:27 UTC (permalink / raw)
To: Christophe Leroy, linuxppc-dev, jroedel
Cc: Darren Stevens, Christoph Hellwig, R.T.Dickinson, Christian Zigotzky
Hi All,
It seems, someone has fixed the boot issue. The latest Git kernel boots
on my PowerPC machines.
Thanks,
Christian
On 05 June 2020 at 6:23 pm, Christian Zigotzky wrote:
> On 04 June 2020 at 7:15 pm, Christophe Leroy wrote:
>> Yes today's linux-next boots on my powerpc 8xx board.
>>
>> Christophe
> Hello Christophe,
>
> Thanks for testing.
>
> I was able to perform a 'git bisect' [1] and identified the bad
> commit. [2] I reverted this commit and after that the kernel boots and
> works without any problems.
>
> Could you please check this commit?
>
> Thanks,
> Christian
>
>
> [1] https://forum.hyperion-entertainment.com/viewtopic.php?p=50772#p50772
> [2]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2ba3e6947aed9bb9575eb1603c0ac6e39185d32a
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Boot issue with the latest Git kernel
2020-06-05 16:23 ` Christian Zigotzky
2020-06-07 13:27 ` Christian Zigotzky
@ 2020-06-07 14:07 ` Aneesh Kumar K.V
2020-06-09 22:18 ` PowerPC KVM-PR issue Christian Zigotzky
1 sibling, 1 reply; 29+ messages in thread
From: Aneesh Kumar K.V @ 2020-06-07 14:07 UTC (permalink / raw)
To: Christian Zigotzky, Christophe Leroy, linuxppc-dev, jroedel
Cc: Darren Stevens, Christoph Hellwig, R.T.Dickinson, Christian Zigotzky
Christian Zigotzky <chzigotzky@xenosoft.de> writes:
> On 04 June 2020 at 7:15 pm, Christophe Leroy wrote:
>> Yes today's linux-next boots on my powerpc 8xx board.
>>
>> Christophe
> Hello Christophe,
>
> Thanks for testing.
>
> I was able to perform a 'git bisect' [1] and identified the bad commit.
> [2] I reverted this commit and after that the kernel boots and works
> without any problems.
>
> Could you please check this commit?
>
> Thanks,
> Christian
>
>
> [1] https://forum.hyperion-entertainment.com/viewtopic.php?p=50772#p50772
> [2]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2ba3e6947aed9bb9575eb1603c0ac6e39185d32a
This was also reported here.
https://lore.kernel.org/linuxppc-dev/1591181457.9020.13.camel@abdul
-aneesh
^ permalink raw reply [flat|nested] 29+ messages in thread
* PowerPC KVM-PR issue
2020-06-07 14:07 ` Aneesh Kumar K.V
@ 2020-06-09 22:18 ` Christian Zigotzky
2020-06-10 9:06 ` Christian Zigotzky
0 siblings, 1 reply; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-09 22:18 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Christian Zigotzky
Hello,
KVM-PR doesn't work anymore on my Nemo board [1]. I figured out that the
Git kernels and the kernel 5.7 are affected.
Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at 700
failed (00000000)
I can boot virtual QEMU PowerPC machines with KVM-PR with the kernel 5.6
without any problems on my Nemo board.
I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
Could you please check KVM-PR on your PowerPC machine?
Thanks,
Christian
[1] https://en.wikipedia.org/wiki/AmigaOne_X1000
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-09 22:18 ` PowerPC KVM-PR issue Christian Zigotzky
@ 2020-06-10 9:06 ` Christian Zigotzky
2020-06-10 11:23 ` Christian Zigotzky
0 siblings, 1 reply; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-10 9:06 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
> Hello,
>
> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out that
> the Git kernels and the kernel 5.7 are affected.
>
> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at 700
> failed (00000000)
>
> I can boot virtual QEMU PowerPC machines with KVM-PR with the kernel
> 5.6 without any problems on my Nemo board.
>
> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>
> Could you please check KVM-PR on your PowerPC machine?
>
> Thanks,
> Christian
>
> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
I figured out that the PowerPC updates 5.7-1 [1] are responsible for the
KVM-PR issue. Please test KVM-PR on your PowerPC machines and check the
PowerPC updates 5.7-1 [1].
Thanks
[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-10 9:06 ` Christian Zigotzky
@ 2020-06-10 11:23 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-10 11:23 UTC (permalink / raw)
To: linuxppc-dev, Alexander Graf, kvm-ppc
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>> Hello,
>>
>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out that
>> the Git kernels and the kernel 5.7 are affected.
>>
>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>> 700 failed (00000000)
>>
>> I can boot virtual QEMU PowerPC machines with KVM-PR with the kernel
>> 5.6 without any problems on my Nemo board.
>>
>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>
>> Could you please check KVM-PR on your PowerPC machine?
>>
>> Thanks,
>> Christian
>>
>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>
> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
> check the PowerPC updates 5.7-1 [1].
>
> Thanks
>
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>
>
I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
Unfortunately I can't use KVM-PR with MoL anymore because of this issue
(see screenshots [1]). Please check the PowerPC updates 5.7-1.
Thanks
[1]
-
https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
-
https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-10 11:23 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-10 11:23 UTC (permalink / raw)
To: linuxppc-dev, Alexander Graf, kvm-ppc
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>> Hello,
>>
>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out that
>> the Git kernels and the kernel 5.7 are affected.
>>
>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>> 700 failed (00000000)
>>
>> I can boot virtual QEMU PowerPC machines with KVM-PR with the kernel
>> 5.6 without any problems on my Nemo board.
>>
>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>
>> Could you please check KVM-PR on your PowerPC machine?
>>
>> Thanks,
>> Christian
>>
>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>
> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
> check the PowerPC updates 5.7-1 [1].
>
> Thanks
>
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>
>
I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
Unfortunately I can't use KVM-PR with MoL anymore because of this issue
(see screenshots [1]). Please check the PowerPC updates 5.7-1.
Thanks
[1]
-
https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
-
https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-10 11:23 ` Christian Zigotzky
@ 2020-06-11 14:47 ` Christian Zigotzky
-1 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-11 14:47 UTC (permalink / raw)
To: linuxppc-dev, npiggin, kvm-ppc
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>> Hello,
>>>
>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out that
>>> the Git kernels and the kernel 5.7 are affected.
>>>
>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>> 700 failed (00000000)
>>>
>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the kernel
>>> 5.6 without any problems on my Nemo board.
>>>
>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>
>>> Could you please check KVM-PR on your PowerPC machine?
>>>
>>> Thanks,
>>> Christian
>>>
>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>
>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>> check the PowerPC updates 5.7-1 [1].
>>
>> Thanks
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>
>>
> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
> Unfortunately I can't use KVM-PR with MoL anymore because of this
> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>
> Thanks
>
> [1]
> -
> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
> -
> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>
Hi All,
I bisected today because of the KVM-PR issue.
Result:
9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
Author: Nicholas Piggin <npiggin@gmail.com>
Date: Wed Feb 26 03:35:21 2020 +1000
powerpc/64s/exception: Move KVM test to common code
This allows more code to be moved out of unrelocated regions. The
system call KVMTEST is changed to be open-coded and remain in the
tramp area to avoid having to move it to entry_64.S. The custom nature
of the system call entry code means the hcall case can be made more
streamlined than regular interrupt handlers.
mpe: Incorporate fix from Nick:
Moving KVM test to the common entry code missed the case of HMI and
MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
switch to virt mode).
This means a MCE or HMI exception that is taken while KVM is running a
guest context will not be switched out of that context, and KVM won't
be notified. Found by running sigfuz in guest with patched host on
POWER9 DD2.3, which causes some TM related HMI interrupts (which are
expected and supposed to be handled by KVM).
This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
the KVM test. This makes them look a little more like other handlers
that all use __GEN_COMMON_ENTRY.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link:
https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
:040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
-----
The following commit is the problem: powerpc/64s/exception: Move KVM
test to common code [1]
These changes were included in the PowerPC updates 5.7-1. [2]
Another test:
git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
5.7-1 [2] ) -> KVM-PR doesn't work.
After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1 ->
KVM-PR works.
Could you please check the first bad commit? [1]
Thanks,
Christian
[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
[2]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-11 14:47 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-11 14:47 UTC (permalink / raw)
To: linuxppc-dev, npiggin, kvm-ppc
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>> Hello,
>>>
>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out that
>>> the Git kernels and the kernel 5.7 are affected.
>>>
>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>> 700 failed (00000000)
>>>
>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the kernel
>>> 5.6 without any problems on my Nemo board.
>>>
>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>
>>> Could you please check KVM-PR on your PowerPC machine?
>>>
>>> Thanks,
>>> Christian
>>>
>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>
>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>> check the PowerPC updates 5.7-1 [1].
>>
>> Thanks
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>
>>
> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
> Unfortunately I can't use KVM-PR with MoL anymore because of this
> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>
> Thanks
>
> [1]
> -
> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
> -
> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>
Hi All,
I bisected today because of the KVM-PR issue.
Result:
9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
Author: Nicholas Piggin <npiggin@gmail.com>
Date:Â Â Wed Feb 26 03:35:21 2020 +1000
   powerpc/64s/exception: Move KVM test to common code
   This allows more code to be moved out of unrelocated regions. The
   system call KVMTEST is changed to be open-coded and remain in the
   tramp area to avoid having to move it to entry_64.S. The custom nature
   of the system call entry code means the hcall case can be made more
   streamlined than regular interrupt handlers.
   mpe: Incorporate fix from Nick:
   Moving KVM test to the common entry code missed the case of HMI and
   MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
   switch to virt mode).
   This means a MCE or HMI exception that is taken while KVM is running a
   guest context will not be switched out of that context, and KVM won't
   be notified. Found by running sigfuz in guest with patched host on
   POWER9 DD2.3, which causes some TM related HMI interrupts (which are
   expected and supposed to be handled by KVM).
   This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
   the KVM test. This makes them look a little more like other handlers
   that all use __GEN_COMMON_ENTRY.
   Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
   Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
   Link:
https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
:040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
87dd49a9cd7202ec79350e8ca26cea01f1dbd93d MÂ Â Â arch
-----
The following commit is the problem: powerpc/64s/exception: Move KVM
test to common code [1]
These changes were included in the PowerPC updates 5.7-1. [2]
Another test:
git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
5.7-1 [2] ) -> KVM-PR doesn't work.
After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1 ->
KVM-PR works.
Could you please check the first bad commit? [1]
Thanks,
Christian
[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id–00f261acaaabd476d7833cec2dd20f2919f1a0
[2]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-11 14:47 ` Christian Zigotzky
@ 2020-06-12 13:01 ` Christian Zigotzky
-1 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-12 13:01 UTC (permalink / raw)
To: linuxppc-dev, npiggin, kvm-ppc
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>> Hello,
>>>>
>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>
>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>> 700 failed (00000000)
>>>>
>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>> kernel 5.6 without any problems on my Nemo board.
>>>>
>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>
>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>
>>>> Thanks,
>>>> Christian
>>>>
>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>
>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>> check the PowerPC updates 5.7-1 [1].
>>>
>>> Thanks
>>>
>>> [1]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>
>>>
>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>
>> Thanks
>>
>> [1]
>> -
>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>> -
>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>
> Hi All,
>
> I bisected today because of the KVM-PR issue.
>
> Result:
>
> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
> Author: Nicholas Piggin <npiggin@gmail.com>
> Date: Wed Feb 26 03:35:21 2020 +1000
>
> powerpc/64s/exception: Move KVM test to common code
>
> This allows more code to be moved out of unrelocated regions. The
> system call KVMTEST is changed to be open-coded and remain in the
> tramp area to avoid having to move it to entry_64.S. The custom
> nature
> of the system call entry code means the hcall case can be made more
> streamlined than regular interrupt handlers.
>
> mpe: Incorporate fix from Nick:
>
> Moving KVM test to the common entry code missed the case of HMI and
> MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
> switch to virt mode).
>
> This means a MCE or HMI exception that is taken while KVM is
> running a
> guest context will not be switched out of that context, and KVM won't
> be notified. Found by running sigfuz in guest with patched host on
> POWER9 DD2.3, which causes some TM related HMI interrupts (which are
> expected and supposed to be handled by KVM).
>
> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
> the KVM test. This makes them look a little more like other handlers
> that all use __GEN_COMMON_ENTRY.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> Link:
> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>
> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>
> -----
>
> The following commit is the problem: powerpc/64s/exception: Move KVM
> test to common code [1]
>
> These changes were included in the PowerPC updates 5.7-1. [2]
>
> Another test:
>
> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
> 5.7-1 [2] ) -> KVM-PR doesn't work.
>
> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
> -> KVM-PR works.
>
> Could you please check the first bad commit? [1]
>
> Thanks,
> Christian
>
>
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
> [2]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
Hi All,
I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
kernel and for the stable kernel 5.7.2 but without any success. There
was lot of restructuring work during the kernel 5.7 development time in
the PowerPC area so it isn't possible reactivate the old code. That
means we have lost the whole KVM-PR support. I also reported this issue
to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
broken. Have you ever made a bisect to see where the problem comes from?"
Please check the KVM-PR code.
Thanks,
Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-12 13:01 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-12 13:01 UTC (permalink / raw)
To: linuxppc-dev, npiggin, kvm-ppc
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>> Hello,
>>>>
>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>
>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>> 700 failed (00000000)
>>>>
>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>> kernel 5.6 without any problems on my Nemo board.
>>>>
>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>
>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>
>>>> Thanks,
>>>> Christian
>>>>
>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>
>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>> check the PowerPC updates 5.7-1 [1].
>>>
>>> Thanks
>>>
>>> [1]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>
>>>
>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>
>> Thanks
>>
>> [1]
>> -
>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>> -
>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>
> Hi All,
>
> I bisected today because of the KVM-PR issue.
>
> Result:
>
> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
> Author: Nicholas Piggin <npiggin@gmail.com>
> Date:Â Â Wed Feb 26 03:35:21 2020 +1000
>
> Â Â Â powerpc/64s/exception: Move KVM test to common code
>
> Â Â Â This allows more code to be moved out of unrelocated regions. The
> Â Â Â system call KVMTEST is changed to be open-coded and remain in the
> Â Â Â tramp area to avoid having to move it to entry_64.S. The custom
> nature
> Â Â Â of the system call entry code means the hcall case can be made more
> Â Â Â streamlined than regular interrupt handlers.
>
> Â Â Â mpe: Incorporate fix from Nick:
>
> Â Â Â Moving KVM test to the common entry code missed the case of HMI and
> Â Â Â MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
> Â Â Â switch to virt mode).
>
> Â Â Â This means a MCE or HMI exception that is taken while KVM is
> running a
> Â Â Â guest context will not be switched out of that context, and KVM won't
> Â Â Â be notified. Found by running sigfuz in guest with patched host on
> Â Â Â POWER9 DD2.3, which causes some TM related HMI interrupts (which are
> Â Â Â expected and supposed to be handled by KVM).
>
> Â Â Â This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
> Â Â Â the KVM test. This makes them look a little more like other handlers
> Â Â Â that all use __GEN_COMMON_ENTRY.
>
> Â Â Â Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> Â Â Â Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> Â Â Â Link:
> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>
> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d MÂ Â Â arch
>
> -----
>
> The following commit is the problem: powerpc/64s/exception: Move KVM
> test to common code [1]
>
> These changes were included in the PowerPC updates 5.7-1. [2]
>
> Another test:
>
> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
> 5.7-1 [2] ) -> KVM-PR doesn't work.
>
> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
> -> KVM-PR works.
>
> Could you please check the first bad commit? [1]
>
> Thanks,
> Christian
>
>
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id–00f261acaaabd476d7833cec2dd20f2919f1a0
> [2]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
Hi All,
I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
kernel and for the stable kernel 5.7.2 but without any success. There
was lot of restructuring work during the kernel 5.7 development time in
the PowerPC area so it isn't possible reactivate the old code. That
means we have lost the whole KVM-PR support. I also reported this issue
to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
broken. Have you ever made a bisect to see where the problem comes from?"
Please check the KVM-PR code.
Thanks,
Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-12 13:01 ` Christian Zigotzky
@ 2020-06-14 8:50 ` Nicholas Piggin
-1 siblings, 0 replies; 29+ messages in thread
From: Nicholas Piggin @ 2020-06-14 8:50 UTC (permalink / raw)
To: Christian Zigotzky, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>> Hello,
>>>>>
>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>
>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>> 700 failed (00000000)
>>>>>
>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>
>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>
>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>
>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>> check the PowerPC updates 5.7-1 [1].
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>>>
>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>
>>> Thanks
>>>
>>> [1]
>>> -
>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>> -
>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>
>> Hi All,
>>
>> I bisected today because of the KVM-PR issue.
>>
>> Result:
>>
>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>> Author: Nicholas Piggin <npiggin@gmail.com>
>> Date: Wed Feb 26 03:35:21 2020 +1000
>>
>> powerpc/64s/exception: Move KVM test to common code
>>
>> This allows more code to be moved out of unrelocated regions. The
>> system call KVMTEST is changed to be open-coded and remain in the
>> tramp area to avoid having to move it to entry_64.S. The custom
>> nature
>> of the system call entry code means the hcall case can be made more
>> streamlined than regular interrupt handlers.
>>
>> mpe: Incorporate fix from Nick:
>>
>> Moving KVM test to the common entry code missed the case of HMI and
>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>> switch to virt mode).
>>
>> This means a MCE or HMI exception that is taken while KVM is
>> running a
>> guest context will not be switched out of that context, and KVM won't
>> be notified. Found by running sigfuz in guest with patched host on
>> POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>> expected and supposed to be handled by KVM).
>>
>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>> the KVM test. This makes them look a little more like other handlers
>> that all use __GEN_COMMON_ENTRY.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>> Link:
>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>
>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>
>> -----
>>
>> The following commit is the problem: powerpc/64s/exception: Move KVM
>> test to common code [1]
>>
>> These changes were included in the PowerPC updates 5.7-1. [2]
>>
>> Another test:
>>
>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>
>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>> -> KVM-PR works.
>>
>> Could you please check the first bad commit? [1]
>>
>> Thanks,
>> Christian
>>
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>> [2]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>
> Hi All,
>
> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
> kernel and for the stable kernel 5.7.2 but without any success. There
> was lot of restructuring work during the kernel 5.7 development time in
> the PowerPC area so it isn't possible reactivate the old code. That
> means we have lost the whole KVM-PR support. I also reported this issue
> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
> broken. Have you ever made a bisect to see where the problem comes from?"
>
> Please check the KVM-PR code.
Hey, thanks for debugging it and reporting. I'm looking into it, will
try to get a fix soon.
Thanks,
Nick
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-14 8:50 ` Nicholas Piggin
0 siblings, 0 replies; 29+ messages in thread
From: Nicholas Piggin @ 2020-06-14 8:50 UTC (permalink / raw)
To: Christian Zigotzky, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>> Hello,
>>>>>
>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>
>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>> 700 failed (00000000)
>>>>>
>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>
>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>
>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>
>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>> check the PowerPC updates 5.7-1 [1].
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>>>
>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>
>>> Thanks
>>>
>>> [1]
>>> -
>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>> -
>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>
>> Hi All,
>>
>> I bisected today because of the KVM-PR issue.
>>
>> Result:
>>
>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>> Author: Nicholas Piggin <npiggin@gmail.com>
>> Date: Wed Feb 26 03:35:21 2020 +1000
>>
>> powerpc/64s/exception: Move KVM test to common code
>>
>> This allows more code to be moved out of unrelocated regions. The
>> system call KVMTEST is changed to be open-coded and remain in the
>> tramp area to avoid having to move it to entry_64.S. The custom
>> nature
>> of the system call entry code means the hcall case can be made more
>> streamlined than regular interrupt handlers.
>>
>> mpe: Incorporate fix from Nick:
>>
>> Moving KVM test to the common entry code missed the case of HMI and
>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>> switch to virt mode).
>>
>> This means a MCE or HMI exception that is taken while KVM is
>> running a
>> guest context will not be switched out of that context, and KVM won't
>> be notified. Found by running sigfuz in guest with patched host on
>> POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>> expected and supposed to be handled by KVM).
>>
>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>> the KVM test. This makes them look a little more like other handlers
>> that all use __GEN_COMMON_ENTRY.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>> Link:
>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>
>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>
>> -----
>>
>> The following commit is the problem: powerpc/64s/exception: Move KVM
>> test to common code [1]
>>
>> These changes were included in the PowerPC updates 5.7-1. [2]
>>
>> Another test:
>>
>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>
>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>> -> KVM-PR works.
>>
>> Could you please check the first bad commit? [1]
>>
>> Thanks,
>> Christian
>>
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>> [2]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>
> Hi All,
>
> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
> kernel and for the stable kernel 5.7.2 but without any success. There
> was lot of restructuring work during the kernel 5.7 development time in
> the PowerPC area so it isn't possible reactivate the old code. That
> means we have lost the whole KVM-PR support. I also reported this issue
> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
> broken. Have you ever made a bisect to see where the problem comes from?"
>
> Please check the KVM-PR code.
Hey, thanks for debugging it and reporting. I'm looking into it, will
try to get a fix soon.
Thanks,
Nick
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-12 13:01 ` Christian Zigotzky
@ 2020-06-14 12:53 ` Nicholas Piggin
-1 siblings, 0 replies; 29+ messages in thread
From: Nicholas Piggin @ 2020-06-14 12:53 UTC (permalink / raw)
To: Christian Zigotzky, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>> Hello,
>>>>>
>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>
>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>> 700 failed (00000000)
>>>>>
>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>
>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>
>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>
>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>> check the PowerPC updates 5.7-1 [1].
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>>>
>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>
>>> Thanks
>>>
>>> [1]
>>> -
>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>> -
>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>
>> Hi All,
>>
>> I bisected today because of the KVM-PR issue.
>>
>> Result:
>>
>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>> Author: Nicholas Piggin <npiggin@gmail.com>
>> Date: Wed Feb 26 03:35:21 2020 +1000
>>
>> powerpc/64s/exception: Move KVM test to common code
>>
>> This allows more code to be moved out of unrelocated regions. The
>> system call KVMTEST is changed to be open-coded and remain in the
>> tramp area to avoid having to move it to entry_64.S. The custom
>> nature
>> of the system call entry code means the hcall case can be made more
>> streamlined than regular interrupt handlers.
>>
>> mpe: Incorporate fix from Nick:
>>
>> Moving KVM test to the common entry code missed the case of HMI and
>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>> switch to virt mode).
>>
>> This means a MCE or HMI exception that is taken while KVM is
>> running a
>> guest context will not be switched out of that context, and KVM won't
>> be notified. Found by running sigfuz in guest with patched host on
>> POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>> expected and supposed to be handled by KVM).
>>
>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>> the KVM test. This makes them look a little more like other handlers
>> that all use __GEN_COMMON_ENTRY.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>> Link:
>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>
>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>
>> -----
>>
>> The following commit is the problem: powerpc/64s/exception: Move KVM
>> test to common code [1]
>>
>> These changes were included in the PowerPC updates 5.7-1. [2]
>>
>> Another test:
>>
>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>
>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>> -> KVM-PR works.
>>
>> Could you please check the first bad commit? [1]
>>
>> Thanks,
>> Christian
>>
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>> [2]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>
> Hi All,
>
> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
> kernel and for the stable kernel 5.7.2 but without any success. There
> was lot of restructuring work during the kernel 5.7 development time in
> the PowerPC area so it isn't possible reactivate the old code. That
> means we have lost the whole KVM-PR support. I also reported this issue
> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
> broken. Have you ever made a bisect to see where the problem comes from?"
>
> Please check the KVM-PR code.
Does this patch fix it for you?
The CTR register reload in the KVM interrupt path used the wrong save
area for SLB (and NMI) interrupts.
Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common code")
Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kernel/exceptions-64s.S | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index e70ebb5c318c..fa080694e581 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
.endif
- ld r10,PACA_EXGEN+EX_CTR(r13)
+ ld r10,IAREA+EX_CTR(r13)
mtctr r10
BEGIN_FTR_SECTION
ld r10,IAREA+EX_PPR(r13)
@@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
.if IKVM_SKIP
89: mtocrf 0x80,r9
- ld r10,PACA_EXGEN+EX_CTR(r13)
+ ld r10,IAREA+EX_CTR(r13)
mtctr r10
ld r9,IAREA+EX_R9(r13)
ld r10,IAREA+EX_R10(r13)
--
2.23.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-14 12:53 ` Nicholas Piggin
0 siblings, 0 replies; 29+ messages in thread
From: Nicholas Piggin @ 2020-06-14 12:53 UTC (permalink / raw)
To: Christian Zigotzky, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>> Hello,
>>>>>
>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>
>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>> 700 failed (00000000)
>>>>>
>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>
>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>
>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>
>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>> check the PowerPC updates 5.7-1 [1].
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>>>
>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>
>>> Thanks
>>>
>>> [1]
>>> -
>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>> -
>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>
>> Hi All,
>>
>> I bisected today because of the KVM-PR issue.
>>
>> Result:
>>
>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>> Author: Nicholas Piggin <npiggin@gmail.com>
>> Date: Wed Feb 26 03:35:21 2020 +1000
>>
>> powerpc/64s/exception: Move KVM test to common code
>>
>> This allows more code to be moved out of unrelocated regions. The
>> system call KVMTEST is changed to be open-coded and remain in the
>> tramp area to avoid having to move it to entry_64.S. The custom
>> nature
>> of the system call entry code means the hcall case can be made more
>> streamlined than regular interrupt handlers.
>>
>> mpe: Incorporate fix from Nick:
>>
>> Moving KVM test to the common entry code missed the case of HMI and
>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>> switch to virt mode).
>>
>> This means a MCE or HMI exception that is taken while KVM is
>> running a
>> guest context will not be switched out of that context, and KVM won't
>> be notified. Found by running sigfuz in guest with patched host on
>> POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>> expected and supposed to be handled by KVM).
>>
>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>> the KVM test. This makes them look a little more like other handlers
>> that all use __GEN_COMMON_ENTRY.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>> Link:
>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>
>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>
>> -----
>>
>> The following commit is the problem: powerpc/64s/exception: Move KVM
>> test to common code [1]
>>
>> These changes were included in the PowerPC updates 5.7-1. [2]
>>
>> Another test:
>>
>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>
>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>> -> KVM-PR works.
>>
>> Could you please check the first bad commit? [1]
>>
>> Thanks,
>> Christian
>>
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>> [2]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>
> Hi All,
>
> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
> kernel and for the stable kernel 5.7.2 but without any success. There
> was lot of restructuring work during the kernel 5.7 development time in
> the PowerPC area so it isn't possible reactivate the old code. That
> means we have lost the whole KVM-PR support. I also reported this issue
> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
> broken. Have you ever made a bisect to see where the problem comes from?"
>
> Please check the KVM-PR code.
Does this patch fix it for you?
The CTR register reload in the KVM interrupt path used the wrong save
area for SLB (and NMI) interrupts.
Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common code")
Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kernel/exceptions-64s.S | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index e70ebb5c318c..fa080694e581 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
.endif
- ld r10,PACA_EXGEN+EX_CTR(r13)
+ ld r10,IAREA+EX_CTR(r13)
mtctr r10
BEGIN_FTR_SECTION
ld r10,IAREA+EX_PPR(r13)
@@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
.if IKVM_SKIP
89: mtocrf 0x80,r9
- ld r10,PACA_EXGEN+EX_CTR(r13)
+ ld r10,IAREA+EX_CTR(r13)
mtctr r10
ld r9,IAREA+EX_R9(r13)
ld r10,IAREA+EX_R10(r13)
--
2.23.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-14 12:53 ` Nicholas Piggin
@ 2020-06-14 14:52 ` Christian Zigotzky
-1 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-14 14:52 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>> Hello,
>>>>>>
>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>
>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>> 700 failed (00000000)
>>>>>>
>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>
>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>
>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>
>>>>>> Thanks,
>>>>>> Christian
>>>>>>
>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>
>>>>> Thanks
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>>>
>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> -
>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>> -
>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>
>>> Hi All,
>>>
>>> I bisected today because of the KVM-PR issue.
>>>
>>> Result:
>>>
>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>> Date: Wed Feb 26 03:35:21 2020 +1000
>>>
>>> powerpc/64s/exception: Move KVM test to common code
>>>
>>> This allows more code to be moved out of unrelocated regions. The
>>> system call KVMTEST is changed to be open-coded and remain in the
>>> tramp area to avoid having to move it to entry_64.S. The custom
>>> nature
>>> of the system call entry code means the hcall case can be made more
>>> streamlined than regular interrupt handlers.
>>>
>>> mpe: Incorporate fix from Nick:
>>>
>>> Moving KVM test to the common entry code missed the case of HMI and
>>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>>> switch to virt mode).
>>>
>>> This means a MCE or HMI exception that is taken while KVM is
>>> running a
>>> guest context will not be switched out of that context, and KVM won't
>>> be notified. Found by running sigfuz in guest with patched host on
>>> POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>>> expected and supposed to be handled by KVM).
>>>
>>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>>> the KVM test. This makes them look a little more like other handlers
>>> that all use __GEN_COMMON_ENTRY.
>>>
>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>> Link:
>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>
>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>>
>>> -----
>>>
>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>> test to common code [1]
>>>
>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>
>>> Another test:
>>>
>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>
>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>> -> KVM-PR works.
>>>
>>> Could you please check the first bad commit? [1]
>>>
>>> Thanks,
>>> Christian
>>>
>>>
>>> [1]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>>> [2]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>> Hi All,
>>
>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
>> kernel and for the stable kernel 5.7.2 but without any success. There
>> was lot of restructuring work during the kernel 5.7 development time in
>> the PowerPC area so it isn't possible reactivate the old code. That
>> means we have lost the whole KVM-PR support. I also reported this issue
>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>> broken. Have you ever made a bisect to see where the problem comes from?"
>>
>> Please check the KVM-PR code.
> Does this patch fix it for you?
>
> The CTR register reload in the KVM interrupt path used the wrong save
> area for SLB (and NMI) interrupts.
>
> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common code")
> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
> arch/powerpc/kernel/exceptions-64s.S | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
> index e70ebb5c318c..fa080694e581 100644
> --- a/arch/powerpc/kernel/exceptions-64s.S
> +++ b/arch/powerpc/kernel/exceptions-64s.S
> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
> END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
> .endif
>
> - ld r10,PACA_EXGEN+EX_CTR(r13)
> + ld r10,IAREA+EX_CTR(r13)
> mtctr r10
> BEGIN_FTR_SECTION
> ld r10,IAREA+EX_PPR(r13)
> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>
> .if IKVM_SKIP
> 89: mtocrf 0x80,r9
> - ld r10,PACA_EXGEN+EX_CTR(r13)
> + ld r10,IAREA+EX_CTR(r13)
> mtctr r10
> ld r9,IAREA+EX_R9(r13)
> ld r10,IAREA+EX_R10(r13)
Many thanks for the fix! I will test it with the RC1 tomorrow.
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-14 14:52 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-14 14:52 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>> Hello,
>>>>>>
>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>
>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>> 700 failed (00000000)
>>>>>>
>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>
>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>
>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>
>>>>>> Thanks,
>>>>>> Christian
>>>>>>
>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>
>>>>> Thanks
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>>>
>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> -
>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>> -
>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>
>>> Hi All,
>>>
>>> I bisected today because of the KVM-PR issue.
>>>
>>> Result:
>>>
>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>> Date:Â Â Wed Feb 26 03:35:21 2020 +1000
>>>
>>> Â Â Â powerpc/64s/exception: Move KVM test to common code
>>>
>>> Â Â Â This allows more code to be moved out of unrelocated regions. The
>>> Â Â Â system call KVMTEST is changed to be open-coded and remain in the
>>> Â Â Â tramp area to avoid having to move it to entry_64.S. The custom
>>> nature
>>> Â Â Â of the system call entry code means the hcall case can be made more
>>> Â Â Â streamlined than regular interrupt handlers.
>>>
>>> Â Â Â mpe: Incorporate fix from Nick:
>>>
>>> Â Â Â Moving KVM test to the common entry code missed the case of HMI and
>>> Â Â Â MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>>> Â Â Â switch to virt mode).
>>>
>>> Â Â Â This means a MCE or HMI exception that is taken while KVM is
>>> running a
>>> Â Â Â guest context will not be switched out of that context, and KVM won't
>>> Â Â Â be notified. Found by running sigfuz in guest with patched host on
>>> Â Â Â POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>>> Â Â Â expected and supposed to be handled by KVM).
>>>
>>> Â Â Â This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>>> Â Â Â the KVM test. This makes them look a little more like other handlers
>>> Â Â Â that all use __GEN_COMMON_ENTRY.
>>>
>>> Â Â Â Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> Â Â Â Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>> Â Â Â Link:
>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>
>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d MÂ Â Â arch
>>>
>>> -----
>>>
>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>> test to common code [1]
>>>
>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>
>>> Another test:
>>>
>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>
>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>> -> KVM-PR works.
>>>
>>> Could you please check the first bad commit? [1]
>>>
>>> Thanks,
>>> Christian
>>>
>>>
>>> [1]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id–00f261acaaabd476d7833cec2dd20f2919f1a0
>>> [2]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>> Hi All,
>>
>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
>> kernel and for the stable kernel 5.7.2 but without any success. There
>> was lot of restructuring work during the kernel 5.7 development time in
>> the PowerPC area so it isn't possible reactivate the old code. That
>> means we have lost the whole KVM-PR support. I also reported this issue
>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>> broken. Have you ever made a bisect to see where the problem comes from?"
>>
>> Please check the KVM-PR code.
> Does this patch fix it for you?
>
> The CTR register reload in the KVM interrupt path used the wrong save
> area for SLB (and NMI) interrupts.
>
> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common code")
> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
> arch/powerpc/kernel/exceptions-64s.S | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
> index e70ebb5c318c..fa080694e581 100644
> --- a/arch/powerpc/kernel/exceptions-64s.S
> +++ b/arch/powerpc/kernel/exceptions-64s.S
> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
> END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
> .endif
>
> - ld r10,PACA_EXGEN+EX_CTR(r13)
> + ld r10,IAREA+EX_CTR(r13)
> mtctr r10
> BEGIN_FTR_SECTION
> ld r10,IAREA+EX_PPR(r13)
> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>
> .if IKVM_SKIP
> 89: mtocrf 0x80,r9
> - ld r10,PACA_EXGEN+EX_CTR(r13)
> + ld r10,IAREA+EX_CTR(r13)
> mtctr r10
> ld r9,IAREA+EX_R9(r13)
> ld r10,IAREA+EX_R10(r13)
Many thanks for the fix! I will test it with the RC1 tomorrow.
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-14 14:52 ` Christian Zigotzky
@ 2020-06-14 23:39 ` Christian Zigotzky
-1 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-14 23:39 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 14 June 2020 at 04:52 pm, Christian Zigotzky wrote:
> On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
>> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>>
>>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>>> 700 failed (00000000)
>>>>>>>
>>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>>
>>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>>
>>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Christian
>>>>>>>
>>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> [1]
>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>>
>>>>>>
>>>>>>
>>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>>
>>>>> Thanks
>>>>>
>>>>> [1]
>>>>> -
>>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>>>
>>>>> -
>>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>>
>>>>>
>>>> Hi All,
>>>>
>>>> I bisected today because of the KVM-PR issue.
>>>>
>>>> Result:
>>>>
>>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>>> Date: Wed Feb 26 03:35:21 2020 +1000
>>>>
>>>> powerpc/64s/exception: Move KVM test to common code
>>>>
>>>> This allows more code to be moved out of unrelocated regions. The
>>>> system call KVMTEST is changed to be open-coded and remain in the
>>>> tramp area to avoid having to move it to entry_64.S. The custom
>>>> nature
>>>> of the system call entry code means the hcall case can be made
>>>> more
>>>> streamlined than regular interrupt handlers.
>>>>
>>>> mpe: Incorporate fix from Nick:
>>>>
>>>> Moving KVM test to the common entry code missed the case of
>>>> HMI and
>>>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't
>>>> want to
>>>> switch to virt mode).
>>>>
>>>> This means a MCE or HMI exception that is taken while KVM is
>>>> running a
>>>> guest context will not be switched out of that context, and
>>>> KVM won't
>>>> be notified. Found by running sigfuz in guest with patched
>>>> host on
>>>> POWER9 DD2.3, which causes some TM related HMI interrupts
>>>> (which are
>>>> expected and supposed to be handled by KVM).
>>>>
>>>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers
>>>> to add
>>>> the KVM test. This makes them look a little more like other
>>>> handlers
>>>> that all use __GEN_COMMON_ENTRY.
>>>>
>>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>> Link:
>>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>>
>>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>>>
>>>> -----
>>>>
>>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>>> test to common code [1]
>>>>
>>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>>
>>>> Another test:
>>>>
>>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>>
>>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>>> -> KVM-PR works.
>>>>
>>>> Could you please check the first bad commit? [1]
>>>>
>>>> Thanks,
>>>> Christian
>>>>
>>>>
>>>> [1]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>
>>>> [2]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>> Hi All,
>>>
>>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest
>>> Git
>>> kernel and for the stable kernel 5.7.2 but without any success. There
>>> was lot of restructuring work during the kernel 5.7 development time in
>>> the PowerPC area so it isn't possible reactivate the old code. That
>>> means we have lost the whole KVM-PR support. I also reported this issue
>>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>>> broken. Have you ever made a bisect to see where the problem comes
>>> from?"
>>>
>>> Please check the KVM-PR code.
>> Does this patch fix it for you?
>>
>> The CTR register reload in the KVM interrupt path used the wrong save
>> area for SLB (and NMI) interrupts.
>>
>> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common
>> code")
>> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>> arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/powerpc/kernel/exceptions-64s.S
>> b/arch/powerpc/kernel/exceptions-64s.S
>> index e70ebb5c318c..fa080694e581 100644
>> --- a/arch/powerpc/kernel/exceptions-64s.S
>> +++ b/arch/powerpc/kernel/exceptions-64s.S
>> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>> END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>> .endif
>> - ld r10,PACA_EXGEN+EX_CTR(r13)
>> + ld r10,IAREA+EX_CTR(r13)
>> mtctr r10
>> BEGIN_FTR_SECTION
>> ld r10,IAREA+EX_PPR(r13)
>> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>> .if IKVM_SKIP
>> 89: mtocrf 0x80,r9
>> - ld r10,PACA_EXGEN+EX_CTR(r13)
>> + ld r10,IAREA+EX_CTR(r13)
>> mtctr r10
>> ld r9,IAREA+EX_R9(r13)
>> ld r10,IAREA+EX_R10(r13)
> Many thanks for the fix! I will test it with the RC1 tomorrow.
>
> -- Christian
It works! :-) Thanks a lot! Screenshot:
https://i.pinimg.com/originals/5d/5f/e5/5d5fe584db474dc88bcc641450b2a7e0.png
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-14 23:39 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-14 23:39 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 14 June 2020 at 04:52 pm, Christian Zigotzky wrote:
> On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
>> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>>
>>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>>> 700 failed (00000000)
>>>>>>>
>>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>>
>>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>>
>>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Christian
>>>>>>>
>>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> [1]
>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>>
>>>>>>
>>>>>>
>>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>>
>>>>> Thanks
>>>>>
>>>>> [1]
>>>>> -
>>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>>>
>>>>> -
>>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>>
>>>>>
>>>> Hi All,
>>>>
>>>> I bisected today because of the KVM-PR issue.
>>>>
>>>> Result:
>>>>
>>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>>> Date:Â Â Wed Feb 26 03:35:21 2020 +1000
>>>>
>>>> Â Â Â Â powerpc/64s/exception: Move KVM test to common code
>>>>
>>>> Â Â Â Â This allows more code to be moved out of unrelocated regions. The
>>>> Â Â Â Â system call KVMTEST is changed to be open-coded and remain in the
>>>> Â Â Â Â tramp area to avoid having to move it to entry_64.S. The custom
>>>> nature
>>>> Â Â Â Â of the system call entry code means the hcall case can be made
>>>> more
>>>> Â Â Â Â streamlined than regular interrupt handlers.
>>>>
>>>> Â Â Â Â mpe: Incorporate fix from Nick:
>>>>
>>>> Â Â Â Â Moving KVM test to the common entry code missed the case of
>>>> HMI and
>>>> Â Â Â Â MCE, which do not do __GEN_COMMON_ENTRY (because they don't
>>>> want to
>>>> Â Â Â Â switch to virt mode).
>>>>
>>>> Â Â Â Â This means a MCE or HMI exception that is taken while KVM is
>>>> running a
>>>> Â Â Â Â guest context will not be switched out of that context, and
>>>> KVM won't
>>>> Â Â Â Â be notified. Found by running sigfuz in guest with patched
>>>> host on
>>>> Â Â Â Â POWER9 DD2.3, which causes some TM related HMI interrupts
>>>> (which are
>>>> Â Â Â Â expected and supposed to be handled by KVM).
>>>>
>>>> Â Â Â Â This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers
>>>> to add
>>>> Â Â Â Â the KVM test. This makes them look a little more like other
>>>> handlers
>>>> Â Â Â Â that all use __GEN_COMMON_ENTRY.
>>>>
>>>> Â Â Â Â Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>> Â Â Â Â Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>> Â Â Â Â Link:
>>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>>
>>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d MÂ Â Â arch
>>>>
>>>> -----
>>>>
>>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>>> test to common code [1]
>>>>
>>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>>
>>>> Another test:
>>>>
>>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>>
>>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>>> -> KVM-PR works.
>>>>
>>>> Could you please check the first bad commit? [1]
>>>>
>>>> Thanks,
>>>> Christian
>>>>
>>>>
>>>> [1]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id–00f261acaaabd476d7833cec2dd20f2919f1a0
>>>>
>>>> [2]
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>> Hi All,
>>>
>>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest
>>> Git
>>> kernel and for the stable kernel 5.7.2 but without any success. There
>>> was lot of restructuring work during the kernel 5.7 development time in
>>> the PowerPC area so it isn't possible reactivate the old code. That
>>> means we have lost the whole KVM-PR support. I also reported this issue
>>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>>> broken. Have you ever made a bisect to see where the problem comes
>>> from?"
>>>
>>> Please check the KVM-PR code.
>> Does this patch fix it for you?
>>
>> The CTR register reload in the KVM interrupt path used the wrong save
>> area for SLB (and NMI) interrupts.
>>
>> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common
>> code")
>> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>> Â arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>> Â 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/powerpc/kernel/exceptions-64s.S
>> b/arch/powerpc/kernel/exceptions-64s.S
>> index e70ebb5c318c..fa080694e581 100644
>> --- a/arch/powerpc/kernel/exceptions-64s.S
>> +++ b/arch/powerpc/kernel/exceptions-64s.S
>> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>> Â END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>> Â Â Â Â Â .endif
>>  -   ld   r10,PACA_EXGEN+EX_CTR(r13)
>> +   ld   r10,IAREA+EX_CTR(r13)
>>      mtctr   r10
>> Â BEGIN_FTR_SECTION
>>      ld   r10,IAREA+EX_PPR(r13)
>> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>> Â Â Â Â Â Â .if IKVM_SKIP
>>  89:   mtocrf   0x80,r9
>> -   ld   r10,PACA_EXGEN+EX_CTR(r13)
>> +   ld   r10,IAREA+EX_CTR(r13)
>>      mtctr   r10
>>      ld   r9,IAREA+EX_R9(r13)
>>      ld   r10,IAREA+EX_R10(r13)
> Many thanks for the fix! I will test it with the RC1 tomorrow.
>
> -- Christian
It works! :-) Thanks a lot! Screenshot:
https://i.pinimg.com/originals/5d/5f/e5/5d5fe584db474dc88bcc641450b2a7e0.png
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-14 23:39 ` Christian Zigotzky
@ 2020-06-15 7:34 ` Christian Zigotzky
-1 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-15 7:34 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 15 June 2020 at 01:39 am, Christian Zigotzky wrote:
> On 14 June 2020 at 04:52 pm, Christian Zigotzky wrote:
>> On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
>>> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>>>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>>>
>>>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>>>> 700 failed (00000000)
>>>>>>>>
>>>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>>>
>>>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>>>
>>>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Christian
>>>>>>>>
>>>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible
>>>>>>> for
>>>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> [1]
>>>>>> -
>>>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>>>>
>>>>>> -
>>>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>>>
>>>>>>
>>>>> Hi All,
>>>>>
>>>>> I bisected today because of the KVM-PR issue.
>>>>>
>>>>> Result:
>>>>>
>>>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>>>> Date: Wed Feb 26 03:35:21 2020 +1000
>>>>>
>>>>> powerpc/64s/exception: Move KVM test to common code
>>>>>
>>>>> This allows more code to be moved out of unrelocated regions.
>>>>> The
>>>>> system call KVMTEST is changed to be open-coded and remain in
>>>>> the
>>>>> tramp area to avoid having to move it to entry_64.S. The custom
>>>>> nature
>>>>> of the system call entry code means the hcall case can be
>>>>> made more
>>>>> streamlined than regular interrupt handlers.
>>>>>
>>>>> mpe: Incorporate fix from Nick:
>>>>>
>>>>> Moving KVM test to the common entry code missed the case of
>>>>> HMI and
>>>>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't
>>>>> want to
>>>>> switch to virt mode).
>>>>>
>>>>> This means a MCE or HMI exception that is taken while KVM is
>>>>> running a
>>>>> guest context will not be switched out of that context, and
>>>>> KVM won't
>>>>> be notified. Found by running sigfuz in guest with patched
>>>>> host on
>>>>> POWER9 DD2.3, which causes some TM related HMI interrupts
>>>>> (which are
>>>>> expected and supposed to be handled by KVM).
>>>>>
>>>>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those
>>>>> handlers to add
>>>>> the KVM test. This makes them look a little more like other
>>>>> handlers
>>>>> that all use __GEN_COMMON_ENTRY.
>>>>>
>>>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>>> Link:
>>>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>>>
>>>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>>>>
>>>>> -----
>>>>>
>>>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>>>> test to common code [1]
>>>>>
>>>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>>>
>>>>> Another test:
>>>>>
>>>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC
>>>>> updates
>>>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>>>
>>>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>>>> -> KVM-PR works.
>>>>>
>>>>> Could you please check the first bad commit? [1]
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>>
>>>>> [2]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>> Hi All,
>>>>
>>>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the
>>>> latest Git
>>>> kernel and for the stable kernel 5.7.2 but without any success. There
>>>> was a lot of restructuring work during the kernel 5.7 development
>>>> time in
>>>> the PowerPC area so it isn't possible to reactivate the old code. That
>>>> means we have lost the whole KVM-PR support. I also reported this
>>>> issue
>>>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>>>> broken. Have you ever made a bisect to see where the problem comes
>>>> from?"
>>>>
>>>> Please check the KVM-PR code.
>>> Does this patch fix it for you?
>>>
>>> The CTR register reload in the KVM interrupt path used the wrong save
>>> area for SLB (and NMI) interrupts.
>>>
>>> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to
>>> common code")
>>> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> ---
>>> arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/powerpc/kernel/exceptions-64s.S
>>> b/arch/powerpc/kernel/exceptions-64s.S
>>> index e70ebb5c318c..fa080694e581 100644
>>> --- a/arch/powerpc/kernel/exceptions-64s.S
>>> +++ b/arch/powerpc/kernel/exceptions-64s.S
>>> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>>> END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>>> .endif
>>> - ld r10,PACA_EXGEN+EX_CTR(r13)
>>> + ld r10,IAREA+EX_CTR(r13)
>>> mtctr r10
>>> BEGIN_FTR_SECTION
>>> ld r10,IAREA+EX_PPR(r13)
>>> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>>> .if IKVM_SKIP
>>> 89: mtocrf 0x80,r9
>>> - ld r10,PACA_EXGEN+EX_CTR(r13)
>>> + ld r10,IAREA+EX_CTR(r13)
>>> mtctr r10
>>> ld r9,IAREA+EX_R9(r13)
>>> ld r10,IAREA+EX_R10(r13)
>> Many thanks for the fix! I will test it with the RC1 tomorrow.
>>
>> -- Christian
>
> It works! :-) Thanks a lot! Screenshot:
> https://i.pinimg.com/originals/5d/5f/e5/5d5fe584db474dc88bcc641450b2a7e0.png
>
> -- Christian
I also successfully tested it with Mac-on-Linux/KVM-PR today.
Screenshot:
https://i.pinimg.com/originals/d6/5e/3c/d65e3c694ca22996569a193dd0aabbdc.png
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-15 7:34 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-15 7:34 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, R.T.Dickinson, Christian Zigotzky
On 15 June 2020 at 01:39 am, Christian Zigotzky wrote:
> On 14 June 2020 at 04:52 pm, Christian Zigotzky wrote:
>> On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
>>> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>>>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>>>
>>>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>>>> 700 failed (00000000)
>>>>>>>>
>>>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>>>
>>>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>>>
>>>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Christian
>>>>>>>>
>>>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible
>>>>>>> for
>>>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> [1]
>>>>>> -
>>>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>>>>
>>>>>> -
>>>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>>>
>>>>>>
>>>>> Hi All,
>>>>>
>>>>> I bisected today because of the KVM-PR issue.
>>>>>
>>>>> Result:
>>>>>
>>>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>>>> Date:Â Â Wed Feb 26 03:35:21 2020 +1000
>>>>>
>>>>> Â Â Â Â powerpc/64s/exception: Move KVM test to common code
>>>>>
>>>>> Â Â Â Â This allows more code to be moved out of unrelocated regions.
>>>>> The
>>>>> Â Â Â Â system call KVMTEST is changed to be open-coded and remain in
>>>>> the
>>>>> Â Â Â Â tramp area to avoid having to move it to entry_64.S. The custom
>>>>> nature
>>>>> Â Â Â Â of the system call entry code means the hcall case can be
>>>>> made more
>>>>> Â Â Â Â streamlined than regular interrupt handlers.
>>>>>
>>>>> Â Â Â Â mpe: Incorporate fix from Nick:
>>>>>
>>>>> Â Â Â Â Moving KVM test to the common entry code missed the case of
>>>>> HMI and
>>>>> Â Â Â Â MCE, which do not do __GEN_COMMON_ENTRY (because they don't
>>>>> want to
>>>>> Â Â Â Â switch to virt mode).
>>>>>
>>>>> Â Â Â Â This means a MCE or HMI exception that is taken while KVM is
>>>>> running a
>>>>> Â Â Â Â guest context will not be switched out of that context, and
>>>>> KVM won't
>>>>> Â Â Â Â be notified. Found by running sigfuz in guest with patched
>>>>> host on
>>>>> Â Â Â Â POWER9 DD2.3, which causes some TM related HMI interrupts
>>>>> (which are
>>>>> Â Â Â Â expected and supposed to be handled by KVM).
>>>>>
>>>>> Â Â Â Â This fix adds a __GEN_REALMODE_COMMON_ENTRY for those
>>>>> handlers to add
>>>>> Â Â Â Â the KVM test. This makes them look a little more like other
>>>>> handlers
>>>>> Â Â Â Â that all use __GEN_COMMON_ENTRY.
>>>>>
>>>>> Â Â Â Â Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>>> Â Â Â Â Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>>> Â Â Â Â Link:
>>>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>>>
>>>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d MÂ Â Â arch
>>>>>
>>>>> -----
>>>>>
>>>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>>>> test to common code [1]
>>>>>
>>>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>>>
>>>>> Another test:
>>>>>
>>>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC
>>>>> updates
>>>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>>>
>>>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>>>> -> KVM-PR works.
>>>>>
>>>>> Could you please check the first bad commit? [1]
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id–00f261acaaabd476d7833cec2dd20f2919f1a0
>>>>>
>>>>> [2]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>> Hi All,
>>>>
>>>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the
>>>> latest Git
>>>> kernel and for the stable kernel 5.7.2 but without any success. There
>>>> was a lot of restructuring work during the kernel 5.7 development
>>>> time in
>>>> the PowerPC area so it isn't possible to reactivate the old code. That
>>>> means we have lost the whole KVM-PR support. I also reported this
>>>> issue
>>>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>>>> broken. Have you ever made a bisect to see where the problem comes
>>>> from?"
>>>>
>>>> Please check the KVM-PR code.
>>> Does this patch fix it for you?
>>>
>>> The CTR register reload in the KVM interrupt path used the wrong save
>>> area for SLB (and NMI) interrupts.
>>>
>>> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to
>>> common code")
>>> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> ---
>>> Â arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>>> Â 1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/powerpc/kernel/exceptions-64s.S
>>> b/arch/powerpc/kernel/exceptions-64s.S
>>> index e70ebb5c318c..fa080694e581 100644
>>> --- a/arch/powerpc/kernel/exceptions-64s.S
>>> +++ b/arch/powerpc/kernel/exceptions-64s.S
>>> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>>> Â END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>>> Â Â Â Â Â .endif
>>>  -   ld   r10,PACA_EXGEN+EX_CTR(r13)
>>> +   ld   r10,IAREA+EX_CTR(r13)
>>>      mtctr   r10
>>> Â BEGIN_FTR_SECTION
>>>      ld   r10,IAREA+EX_PPR(r13)
>>> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>>> Â Â Â Â Â Â .if IKVM_SKIP
>>>  89:   mtocrf   0x80,r9
>>> -   ld   r10,PACA_EXGEN+EX_CTR(r13)
>>> +   ld   r10,IAREA+EX_CTR(r13)
>>>      mtctr   r10
>>>      ld   r9,IAREA+EX_R9(r13)
>>>      ld   r10,IAREA+EX_R10(r13)
>> Many thanks for the fix! I will test it with the RC1 tomorrow.
>>
>> -- Christian
>
> It works! :-) Thanks a lot! Screenshot:
> https://i.pinimg.com/originals/5d/5f/e5/5d5fe584db474dc88bcc641450b2a7e0.png
>
> -- Christian
I also successfully tested it with Mac-on-Linux/KVM-PR today.
Screenshot:
https://i.pinimg.com/originals/d6/5e/3c/d65e3c694ca22996569a193dd0aabbdc.png
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
2020-06-14 23:39 ` Christian Zigotzky
@ 2020-06-25 9:38 ` Christian Zigotzky
-1 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-25 9:38 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, mad skateman, R.T.Dickinson, Christian Zigotzky
On 15 June 2020 at 01:39 pm, Christian Zigotzky wrote:
> On 14 June 2020 at 04:52 pm, Christian Zigotzky wrote:
>> On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
>>> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>>>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>>>
>>>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>>>> 700 failed (00000000)
>>>>>>>>
>>>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>>>
>>>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>>>
>>>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Christian
>>>>>>>>
>>>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible
>>>>>>> for
>>>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> [1]
>>>>>> -
>>>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>>>>
>>>>>> -
>>>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>>>
>>>>>>
>>>>> Hi All,
>>>>>
>>>>> I bisected today because of the KVM-PR issue.
>>>>>
>>>>> Result:
>>>>>
>>>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>>>> Date: Wed Feb 26 03:35:21 2020 +1000
>>>>>
>>>>> powerpc/64s/exception: Move KVM test to common code
>>>>>
>>>>> This allows more code to be moved out of unrelocated regions.
>>>>> The
>>>>> system call KVMTEST is changed to be open-coded and remain in
>>>>> the
>>>>> tramp area to avoid having to move it to entry_64.S. The custom
>>>>> nature
>>>>> of the system call entry code means the hcall case can be
>>>>> made more
>>>>> streamlined than regular interrupt handlers.
>>>>>
>>>>> mpe: Incorporate fix from Nick:
>>>>>
>>>>> Moving KVM test to the common entry code missed the case of
>>>>> HMI and
>>>>> MCE, which do not do __GEN_COMMON_ENTRY (because they don't
>>>>> want to
>>>>> switch to virt mode).
>>>>>
>>>>> This means a MCE or HMI exception that is taken while KVM is
>>>>> running a
>>>>> guest context will not be switched out of that context, and
>>>>> KVM won't
>>>>> be notified. Found by running sigfuz in guest with patched
>>>>> host on
>>>>> POWER9 DD2.3, which causes some TM related HMI interrupts
>>>>> (which are
>>>>> expected and supposed to be handled by KVM).
>>>>>
>>>>> This fix adds a __GEN_REALMODE_COMMON_ENTRY for those
>>>>> handlers to add
>>>>> the KVM test. This makes them look a little more like other
>>>>> handlers
>>>>> that all use __GEN_COMMON_ENTRY.
>>>>>
>>>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>>> Link:
>>>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>>>
>>>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M arch
>>>>>
>>>>> -----
>>>>>
>>>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>>>> test to common code [1]
>>>>>
>>>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>>>
>>>>> Another test:
>>>>>
>>>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC
>>>>> updates
>>>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>>>
>>>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>>>> -> KVM-PR works.
>>>>>
>>>>> Could you please check the first bad commit? [1]
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>>
>>>>> [2]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>> Hi All,
>>>>
>>>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the
>>>> latest Git
>>>> kernel and for the stable kernel 5.7.2 but without any success. There
>>>> was lot of restructuring work during the kernel 5.7 development
>>>> time in
>>>> the PowerPC area so it isn't possible reactivate the old code. That
>>>> means we have lost the whole KVM-PR support. I also reported this
>>>> issue
>>>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>>>> broken. Have you ever made a bisect to see where the problem comes
>>>> from?"
>>>>
>>>> Please check the KVM-PR code.
>>> Does this patch fix it for you?
>>>
>>> The CTR register reload in the KVM interrupt path used the wrong save
>>> area for SLB (and NMI) interrupts.
>>>
>>> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to
>>> common code")
>>> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> ---
>>> arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/powerpc/kernel/exceptions-64s.S
>>> b/arch/powerpc/kernel/exceptions-64s.S
>>> index e70ebb5c318c..fa080694e581 100644
>>> --- a/arch/powerpc/kernel/exceptions-64s.S
>>> +++ b/arch/powerpc/kernel/exceptions-64s.S
>>> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>>> END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>>> .endif
>>> - ld r10,PACA_EXGEN+EX_CTR(r13)
>>> + ld r10,IAREA+EX_CTR(r13)
>>> mtctr r10
>>> BEGIN_FTR_SECTION
>>> ld r10,IAREA+EX_PPR(r13)
>>> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>>> .if IKVM_SKIP
>>> 89: mtocrf 0x80,r9
>>> - ld r10,PACA_EXGEN+EX_CTR(r13)
>>> + ld r10,IAREA+EX_CTR(r13)
>>> mtctr r10
>>> ld r9,IAREA+EX_R9(r13)
>>> ld r10,IAREA+EX_R10(r13)
>> Many thanks for the fix! I will test it with the RC1 tomorrow.
>>
>> -- Christian
>
> It works! :-) Thanks a lot! Screenshot:
> https://i.pinimg.com/originals/5d/5f/e5/5d5fe584db474dc88bcc641450b2a7e0.png
>
> -- Christian
Just for info: I successfully tested KVM-PR with the stable kernel 5.7.6
and with the RC2 of kernel 5.8 today. Thanks a lot for fixing the issue.
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: PowerPC KVM-PR issue
@ 2020-06-25 9:38 ` Christian Zigotzky
0 siblings, 0 replies; 29+ messages in thread
From: Christian Zigotzky @ 2020-06-25 9:38 UTC (permalink / raw)
To: Nicholas Piggin, kvm-ppc, linuxppc-dev
Cc: Darren Stevens, mad skateman, R.T.Dickinson, Christian Zigotzky
On 15 June 2020 at 01:39 pm, Christian Zigotzky wrote:
> On 14 June 2020 at 04:52 pm, Christian Zigotzky wrote:
>> On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
>>> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>>>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>>>
>>>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>>>> 700 failed (00000000)
>>>>>>>>
>>>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>>>
>>>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>>>
>>>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Christian
>>>>>>>>
>>>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible
>>>>>>> for
>>>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> [1]
>>>>>> -
>>>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>>>>
>>>>>> -
>>>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>>>
>>>>>>
>>>>> Hi All,
>>>>>
>>>>> I bisected today because of the KVM-PR issue.
>>>>>
>>>>> Result:
>>>>>
>>>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>>>> Date:Â Â Wed Feb 26 03:35:21 2020 +1000
>>>>>
>>>>> Â Â Â Â powerpc/64s/exception: Move KVM test to common code
>>>>>
>>>>> Â Â Â Â This allows more code to be moved out of unrelocated regions.
>>>>> The
>>>>> Â Â Â Â system call KVMTEST is changed to be open-coded and remain in
>>>>> the
>>>>> Â Â Â Â tramp area to avoid having to move it to entry_64.S. The custom
>>>>> nature
>>>>> Â Â Â Â of the system call entry code means the hcall case can be
>>>>> made more
>>>>> Â Â Â Â streamlined than regular interrupt handlers.
>>>>>
>>>>> Â Â Â Â mpe: Incorporate fix from Nick:
>>>>>
>>>>> Â Â Â Â Moving KVM test to the common entry code missed the case of
>>>>> HMI and
>>>>> Â Â Â Â MCE, which do not do __GEN_COMMON_ENTRY (because they don't
>>>>> want to
>>>>> Â Â Â Â switch to virt mode).
>>>>>
>>>>> Â Â Â Â This means a MCE or HMI exception that is taken while KVM is
>>>>> running a
>>>>> Â Â Â Â guest context will not be switched out of that context, and
>>>>> KVM won't
>>>>> Â Â Â Â be notified. Found by running sigfuz in guest with patched
>>>>> host on
>>>>> Â Â Â Â POWER9 DD2.3, which causes some TM related HMI interrupts
>>>>> (which are
>>>>> Â Â Â Â expected and supposed to be handled by KVM).
>>>>>
>>>>> Â Â Â Â This fix adds a __GEN_REALMODE_COMMON_ENTRY for those
>>>>> handlers to add
>>>>> Â Â Â Â the KVM test. This makes them look a little more like other
>>>>> handlers
>>>>> Â Â Â Â that all use __GEN_COMMON_ENTRY.
>>>>>
>>>>> Â Â Â Â Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>>> Â Â Â Â Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>>> Â Â Â Â Link:
>>>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>>>
>>>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d MÂ Â Â arch
>>>>>
>>>>> -----
>>>>>
>>>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>>>> test to common code [1]
>>>>>
>>>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>>>
>>>>> Another test:
>>>>>
>>>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC
>>>>> updates
>>>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>>>
>>>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>>>> -> KVM-PR works.
>>>>>
>>>>> Could you please check the first bad commit? [1]
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id–00f261acaaabd476d7833cec2dd20f2919f1a0
>>>>>
>>>>> [2]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?idÓ8c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>> Hi All,
>>>>
>>>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the
>>>> latest Git
>>>> kernel and for the stable kernel 5.7.2 but without any success. There
>>>> was lot of restructuring work during the kernel 5.7 development
>>>> time in
>>>> the PowerPC area so it isn't possible reactivate the old code. That
>>>> means we have lost the whole KVM-PR support. I also reported this
>>>> issue
>>>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>>>> broken. Have you ever made a bisect to see where the problem comes
>>>> from?"
>>>>
>>>> Please check the KVM-PR code.
>>> Does this patch fix it for you?
>>>
>>> The CTR register reload in the KVM interrupt path used the wrong save
>>> area for SLB (and NMI) interrupts.
>>>
>>> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to
>>> common code")
>>> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> ---
>>> Â arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>>> Â 1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/powerpc/kernel/exceptions-64s.S
>>> b/arch/powerpc/kernel/exceptions-64s.S
>>> index e70ebb5c318c..fa080694e581 100644
>>> --- a/arch/powerpc/kernel/exceptions-64s.S
>>> +++ b/arch/powerpc/kernel/exceptions-64s.S
>>> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>>> Â END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>>> Â Â Â Â Â .endif
>>>  -   ld   r10,PACA_EXGEN+EX_CTR(r13)
>>> +   ld   r10,IAREA+EX_CTR(r13)
>>>      mtctr   r10
>>> Â BEGIN_FTR_SECTION
>>>      ld   r10,IAREA+EX_PPR(r13)
>>> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>>> Â Â Â Â Â Â .if IKVM_SKIP
>>>  89:   mtocrf   0x80,r9
>>> -   ld   r10,PACA_EXGEN+EX_CTR(r13)
>>> +   ld   r10,IAREA+EX_CTR(r13)
>>>      mtctr   r10
>>>      ld   r9,IAREA+EX_R9(r13)
>>>      ld   r10,IAREA+EX_R10(r13)
>> Many thanks for the fix! I will test it with the RC1 tomorrow.
>>
>> -- Christian
>
> It works! :-) Thanks a lot! Screenshot:
> https://i.pinimg.com/originals/5d/5f/e5/5d5fe584db474dc88bcc641450b2a7e0.png
>
> -- Christian
Just for info: I successfully tested KVM-PR with the stable kernel 5.7.6
and with the RC2 of kernel 5.8 today. Thanks a lot for fixing the issue.
-- Christian
^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2020-06-25 9:41 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-04 14:16 Boot issue with the latest Git kernel Christian Zigotzky
2020-06-04 14:26 ` Christophe Leroy
2020-06-04 14:29 ` Christophe Leroy
2020-06-04 15:46 ` Christian Zigotzky
2020-06-04 15:53 ` Christian Zigotzky
2020-06-04 17:15 ` Christophe Leroy
2020-06-05 16:23 ` Christian Zigotzky
2020-06-07 13:27 ` Christian Zigotzky
2020-06-07 14:07 ` Aneesh Kumar K.V
2020-06-09 22:18 ` PowerPC KVM-PR issue Christian Zigotzky
2020-06-10 9:06 ` Christian Zigotzky
2020-06-10 11:23 ` Christian Zigotzky
2020-06-10 11:23 ` Christian Zigotzky
2020-06-11 14:47 ` Christian Zigotzky
2020-06-11 14:47 ` Christian Zigotzky
2020-06-12 13:01 ` Christian Zigotzky
2020-06-12 13:01 ` Christian Zigotzky
2020-06-14 8:50 ` Nicholas Piggin
2020-06-14 8:50 ` Nicholas Piggin
2020-06-14 12:53 ` Nicholas Piggin
2020-06-14 12:53 ` Nicholas Piggin
2020-06-14 14:52 ` Christian Zigotzky
2020-06-14 14:52 ` Christian Zigotzky
2020-06-14 23:39 ` Christian Zigotzky
2020-06-14 23:39 ` Christian Zigotzky
2020-06-15 7:34 ` Christian Zigotzky
2020-06-15 7:34 ` Christian Zigotzky
2020-06-25 9:38 ` Christian Zigotzky
2020-06-25 9:38 ` Christian Zigotzky
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.