[RFC PATCH] Disable Book-E KVM support?

[RFC PATCH] Disable Book-E KVM support?

[Nicholas Piggin]

BookE KVM is in a deep maintenance state, I'm not sure how much testing
it gets. I don't have a test setup, and it does not look like QEMU has
any HV architecture enabled. It hasn't been too painful but there are
some cases where it causes a bit of problem not being able to test, e.g.,

https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html

Time to begin removal process, or are there still people using it? I'm
happy to to keep making occasional patches to try keep it going if
there are people testing upstream. Getting HV support into QEMU would
help with long term support, not sure how big of a job that would be.

Thanks,
Nick
---
 arch/powerpc/kvm/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index a9f57dad6d91..6c9458741cb3 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -191,6 +191,7 @@ config KVM_EXIT_TIMING
 
 config KVM_E500V2
 	bool "KVM support for PowerPC E500v2 processors"
+	depends on false
 	depends on PPC_E500 && !PPC_E500MC
 	depends on !CONTEXT_TRACKING_USER
 	select KVM
@@ -207,6 +208,7 @@ config KVM_E500V2
 
 config KVM_E500MC
 	bool "KVM support for PowerPC E500MC/E5500/E6500 processors"
+	depends on false
 	depends on PPC_E500MC
 	depends on !CONTEXT_TRACKING_USER
 	select KVM
-- 
2.37.2

Re: [RFC PATCH] Disable Book-E KVM support?

[Crystal Wood]

On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:
> BookE KVM is in a deep maintenance state, I'm not sure how much testing
> it gets. I don't have a test setup, and it does not look like QEMU has
> any HV architecture enabled. It hasn't been too painful but there are
> some cases where it causes a bit of problem not being able to test, e.g.,
> 
> https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html
> 
> Time to begin removal process, or are there still people using it? I'm
> happy to to keep making occasional patches to try keep it going if
> there are people testing upstream. Getting HV support into QEMU would
> help with long term support, not sure how big of a job that would be.

Not sure what you mean about QEMU not having e500 HV support?  I don't know if
it's bitrotted, but it's there.

I don't know whether anyone is still using this, but if they are, it's
probably e500mc and not e500v2 (which involved a bunch of hacks to get almost-
sorta-usable performance out of hardware not designed for virtualization).  I
do see that there have been a few recent patches on QEMU e500 (beyond the
treewide cleanup type stuff), though I don't know if they're using KVM.  CCing
them and the QEMU list.

I have an e6500 I could occasionally test on, if it turns out people do still
care about this.  Don't count me as the use case, though. :-)

FWIW, as far as the RECONCILE_IRQ_STATE issue, that used to be done in
kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 to be "cleaner and
faster". :-P

-Crystal

Re: [RFC PATCH] Disable Book-E KVM support?

[Nicholas Piggin]

On Thu Dec 1, 2022 at 6:45 AM AEST, Crystal Wood wrote:
> On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:
> > BookE KVM is in a deep maintenance state, I'm not sure how much testing
> > it gets. I don't have a test setup, and it does not look like QEMU has
> > any HV architecture enabled. It hasn't been too painful but there are
> > some cases where it causes a bit of problem not being able to test, e.g.,
> > 
> > https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html
> > 
> > Time to begin removal process, or are there still people using it? I'm
> > happy to to keep making occasional patches to try keep it going if
> > there are people testing upstream. Getting HV support into QEMU would
> > help with long term support, not sure how big of a job that would be.
>
> Not sure what you mean about QEMU not having e500 HV support?  I don't know if
> it's bitrotted, but it's there.
>
> I don't know whether anyone is still using this, but if they are, it's
> probably e500mc and not e500v2 (which involved a bunch of hacks to get almost-
> sorta-usable performance out of hardware not designed for virtualization).  I
> do see that there have been a few recent patches on QEMU e500 (beyond the
> treewide cleanup type stuff), though I don't know if they're using KVM.  CCing
> them and the QEMU list.

Well I could be wrong about it, but it doesn't look it implements LPIDR
or GSPRs. The only use of MSR_GS seems to be a couple of places
including an instruction that aborts because no HV implementation. It
does have an MMU index selector but before d764184ddb22 that apparently
didn't really work.

QEMU probably should be able to run BookE KVM in PR mode at least.

> I have an e6500 I could occasionally test on, if it turns out people do still
> care about this.  Don't count me as the use case, though. :-)

Do you have a KVM setup on it? And it works with recentish upstream?

> FWIW, as far as the RECONCILE_IRQ_STATE issue, that used to be done in
> kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 to be "cleaner and
> faster". :-P

Yeah that was probably reasonable at the time, that was the common way
to do it and thie patch avoids an unnecessary expensive write to MSR
(which my patch retains).

I think it must have always clobbered r4 though. It's possible it wasn't
tested with the right build option, or the right tracer active, or maybe
the call was simple enough that it was lucky and the compiler didn't use
r4. Easy bug to miss when it's not obvious that macro can call into C.

Thanks,
Nick

Re: [RFC PATCH] Disable Book-E KVM support?

[Bernhard Beschow]

Am 1. Dezember 2022 06:06:16 UTC schrieb Nicholas Piggin <npiggin@gmail.com>:
>On Thu Dec 1, 2022 at 6:45 AM AEST, Crystal Wood wrote:
>> On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:
>> > BookE KVM is in a deep maintenance state, I'm not sure how much testing
>> > it gets. I don't have a test setup, and it does not look like QEMU has
>> > any HV architecture enabled. It hasn't been too painful but there are
>> > some cases where it causes a bit of problem not being able to test, e.g.,
>> > 
>> > https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html
>> > 
>> > Time to begin removal process, or are there still people using it? I'm
>> > happy to to keep making occasional patches to try keep it going if
>> > there are people testing upstream. Getting HV support into QEMU would
>> > help with long term support, not sure how big of a job that would be.
>>
>> Not sure what you mean about QEMU not having e500 HV support?  I don't know if
>> it's bitrotted, but it's there.
>>
>> I don't know whether anyone is still using this, but if they are, it's
>> probably e500mc and not e500v2 (which involved a bunch of hacks to get almost-
>> sorta-usable performance out of hardware not designed for virtualization).  I
>> do see that there have been a few recent patches on QEMU e500 (beyond the
>> treewide cleanup type stuff), though I don't know if they're using KVM.  CCing
>> them and the QEMU list.

Thanks for CCing!

No, I'm not using KVM on e500. The goal of my patches is to run software in QEMU on an x86_64 host rather than on a real PPC machine to optimize our development process.

Best regards,
Bernhard

>Well I could be wrong about it, but it doesn't look it implements LPIDR
>or GSPRs. The only use of MSR_GS seems to be a couple of places
>including an instruction that aborts because no HV implementation. It
>does have an MMU index selector but before d764184ddb22 that apparently
>didn't really work.
>
>QEMU probably should be able to run BookE KVM in PR mode at least.
>
>> I have an e6500 I could occasionally test on, if it turns out people do still
>> care about this.  Don't count me as the use case, though. :-)
>
>Do you have a KVM setup on it? And it works with recentish upstream?
>
>> FWIW, as far as the RECONCILE_IRQ_STATE issue, that used to be done in
>> kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 to be "cleaner and
>> faster". :-P
>
>Yeah that was probably reasonable at the time, that was the common way
>to do it and thie patch avoids an unnecessary expensive write to MSR
>(which my patch retains).
>
>I think it must have always clobbered r4 though. It's possible it wasn't
>tested with the right build option, or the right tracer active, or maybe
>the call was simple enough that it was lucky and the compiler didn't use
>r4. Easy bug to miss when it's not obvious that macro can call into C.
>
>Thanks,
>Nick

Re: [RFC PATCH] Disable Book-E KVM support?

[Daniel Henrique Barboza]

On 11/30/22 17:45, Crystal Wood wrote:
> On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:
>> BookE KVM is in a deep maintenance state, I'm not sure how much testing
>> it gets. I don't have a test setup, and it does not look like QEMU has
>> any HV architecture enabled. It hasn't been too painful but there are
>> some cases where it causes a bit of problem not being able to test, e.g.,
>>
>> https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html
>>
>> Time to begin removal process, or are there still people using it? I'm
>> happy to to keep making occasional patches to try keep it going if
>> there are people testing upstream. Getting HV support into QEMU would
>> help with long term support, not sure how big of a job that would be.
> 
> Not sure what you mean about QEMU not having e500 HV support?  I don't know if
> it's bitrotted, but it's there.

AFAIK all QEMU ppc boards, aside from pSeries and the Mac ones, are always used in
emulated mode in an use case similar to what Bernhard described in his reply (run
in x86 due to lack of ppc hardware).

I am not aware of e500 KVM support in QEMU since I never attempted it. But yes,
it is present, but poorly tested - if tested at all. And the reason why there's
no push on our side to removed it from QEMU is because its code is so entwined
with pSeries KVM that it would take too much effort.

Do not take the presence of e500 KVM support in QEMU as a blocker to disabled it in
the kernel. As far as the current QEMU usage goes e500 KVM can be removed without
too much drama from our side.


Cedric, do you have any opinions about it?



Daniel


> 
> I don't know whether anyone is still using this, but if they are, it's
> probably e500mc and not e500v2 (which involved a bunch of hacks to get almost-
> sorta-usable performance out of hardware not designed for virtualization).  I
> do see that there have been a few recent patches on QEMU e500 (beyond the
> treewide cleanup type stuff), though I don't know if they're using KVM.  CCing
> them and the QEMU list.
> 
> I have an e6500 I could occasionally test on, if it turns out people do still
> care about this.  Don't count me as the use case, though. :-)
> 
> FWIW, as far as the RECONCILE_IRQ_STATE issue, that used to be done in
> kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 to be "cleaner and
> faster". :-P
> 
> -Crystal
> 
> 

Re: [RFC PATCH] Disable Book-E KVM support?

[Cédric Le Goater]

On 12/2/22 12:04, Daniel Henrique Barboza wrote:
> 
> 
> On 11/30/22 17:45, Crystal Wood wrote:
>> On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:
>>> BookE KVM is in a deep maintenance state, I'm not sure how much testing
>>> it gets. I don't have a test setup, and it does not look like QEMU has
>>> any HV architecture enabled. It hasn't been too painful but there are
>>> some cases where it causes a bit of problem not being able to test, e.g.,
>>>
>>> https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html
>>>
>>> Time to begin removal process, or are there still people using it? I'm
>>> happy to to keep making occasional patches to try keep it going if
>>> there are people testing upstream. Getting HV support into QEMU would
>>> help with long term support, not sure how big of a job that would be.
>>
>> Not sure what you mean about QEMU not having e500 HV support?  I don't know if
>> it's bitrotted, but it's there.
> 
> AFAIK all QEMU ppc boards, aside from pSeries and the Mac ones, are always used in
> emulated mode in an use case similar to what Bernhard described in his reply (run
> in x86 due to lack of ppc hardware).
> 
> I am not aware of e500 KVM support in QEMU since I never attempted it. But yes,
> it is present, but poorly tested - if tested at all. And the reason why there's
> no push on our side to removed it from QEMU is because its code is so entwined
> with pSeries KVM that it would take too much effort.
>
> Do not take the presence of e500 KVM support in QEMU as a blocker to disabled it in
> the kernel. As far as the current QEMU usage goes e500 KVM can be removed without
> too much drama from our side.
> 
> Cedric, do you have any opinions about it?

I can not tell how much e500 KVM is used. The last report we had
on the topic was :

   https://lore.kernel.org/all/R4OPHT$7F12C66D1107397991E0E4C978FE6AF1@locati.it/

and the last commit mentioning e500 VMs I could find is cb3778a045,
which brings us back to QEMU 2.2 or so.

It would be nice to 'quickly' check the state of the KVM stack on
such boards and, may be, plan for more cleanups.


Thanks,

C.

> 
> 
> 
> Daniel
> 
> 
>>
>> I don't know whether anyone is still using this, but if they are, it's
>> probably e500mc and not e500v2 (which involved a bunch of hacks to get almost-
>> sorta-usable performance out of hardware not designed for virtualization).  I
>> do see that there have been a few recent patches on QEMU e500 (beyond the
>> treewide cleanup type stuff), though I don't know if they're using KVM.  CCing
>> them and the QEMU list.
>>
>> I have an e6500 I could occasionally test on, if it turns out people do still
>> care about this.  Don't count me as the use case, though. :-)
>>
>> FWIW, as far as the RECONCILE_IRQ_STATE issue, that used to be done in
>> kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 to be "cleaner and
>> faster". :-P
>>
>> -Crystal
>>
>>

Re: [RFC PATCH] Disable Book-E KVM support?

[Nicholas Piggin]

On Sun Dec 4, 2022 at 9:33 PM AEST, Christian Zigotzky wrote:
> Hi All,
>
> We regularly use QEMU with KVM HV on our A-EON AmigaOne X5000 machines 
> (book3e). It works fast and without any problems.
>
> Screenshot tour of QEMU/KVM HV on our AmigaOnes:
>
> - https://i.ibb.co/m4vgwNT/Kernel-6-1-rc3-Power-PC.png
> - https://i.ibb.co/Fwdjf7Z/Kernel-6-0-rc6-Power-PC.png
> - https://i.ibb.co/LYnJGdF/Kernel-5-19-rc5-Power-PC-2.png
> - https://i.ibb.co/vz1Wm5z/QEMU-with-9p-and-USB-sound.png
> - https://i.ibb.co/ScMjtp7/Kernel-5-17-alpha5-Power-PC.png
> - https://i.ibb.co/LQryFcK/Kernel-5-17-alpha4-Power-PC.png
> - https://i.ibb.co/kKLx9mf/Kernel-5-10-89-Power-PC.png
> - https://i.ibb.co/LRG1RDV/Kernel-5-10-89-Power-PC-2.png
> - https://i.ibb.co/NCFqY0k/QEMU-USB-Audio-on-Void-PPC.png
> - https://i.ibb.co/N1vL5Kd/Kernel-5-16-alpha3-Power-PC.png
> - https://i.ibb.co/SwjTyJk/Kernel-5-16-alpha1-Power-PC.png
> - https://i.ibb.co/LkpWNPx/Kernel-5-15-rc5-Power-PC.png
> - https://i.ibb.co/F8q1jDR/Kernel-5-15-rc4-Power-PC.png
> - https://i.ibb.co/zZxrbhV/Kernel-5-15-alpha6-Power-PC.png
> - 
> https://i.pinimg.com/originals/a8/8b/42/a88b422870201887fc01ef44ddc1a235.png
> - 
> https://i.pinimg.com/originals/57/d9/83/57d98324cd055b7ae00a87ad5a45a42f.png
> - 
> https://i.pinimg.com/originals/f2/a5/e3/f2a5e34e2015381b0cb87cc51232a8bc.png
> - 
> https://i.pinimg.com/originals/c5/0d/85/c50d85d7e8f20b4caa1a439faf751964.png
> - 
> https://i.pinimg.com/originals/6e/3b/59/6e3b59fe10276c5644b15622a81f43f1.png
>
> We solved some issues:
>
> - https://forum.hyperion-entertainment.com/viewtopic.php?p=54357#p54357
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-November/236307.html
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-September/249021.html
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-May/229103.html
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-January/223342.html
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216379.html
> - 
> https://forum.hyperion-entertainment.com/viewtopic.php?f=58&t=4655&p=53393&hilit=KVM#p53393
> - https://forum.hyperion-entertainment.com/viewtopic.php?p=53209#p53209
>
> Please, do not remove KVM support from Book3e because it works without 
> any problems and fast. We need it for our work.

Hey Christian,

I knew you do a lot of testing with BookE, I didn't know you used KVM
there. Good to know it's working for you. In that case I withdraw my
question.

BookE KVM has not been too much trouble, but some things come up that
really need testing before patching, so a QEMU HV implementation would
be good to have (I mean QEMU running the host). mpe has a hardware but
probably no time$ to do much testing (especially of other peoples'
untested patches). So having that would help keep kernel support going.

Thanks,
Nick

Re: [RFC PATCH] Disable Book-E KVM support?

[Christian Zigotzky]

Further information: 
https://lists.nongnu.org/archive/html/qemu-ppc/2022-12/msg00000.html

-- Christian


On 04 December 2022 at 12:33 pm, Christian Zigotzky wrote:
> Hi All,
>
> We regularly use QEMU with KVM HV on our A-EON AmigaOne X5000 machines 
> (book3e). It works fast and without any problems.
>
> Screenshot tour of QEMU/KVM HV on our AmigaOnes:
>
> - https://i.ibb.co/m4vgwNT/Kernel-6-1-rc3-Power-PC.png
> - https://i.ibb.co/Fwdjf7Z/Kernel-6-0-rc6-Power-PC.png
> - https://i.ibb.co/LYnJGdF/Kernel-5-19-rc5-Power-PC-2.png
> - https://i.ibb.co/vz1Wm5z/QEMU-with-9p-and-USB-sound.png
> - https://i.ibb.co/ScMjtp7/Kernel-5-17-alpha5-Power-PC.png
> - https://i.ibb.co/LQryFcK/Kernel-5-17-alpha4-Power-PC.png
> - https://i.ibb.co/kKLx9mf/Kernel-5-10-89-Power-PC.png
> - https://i.ibb.co/LRG1RDV/Kernel-5-10-89-Power-PC-2.png
> - https://i.ibb.co/NCFqY0k/QEMU-USB-Audio-on-Void-PPC.png
> - https://i.ibb.co/N1vL5Kd/Kernel-5-16-alpha3-Power-PC.png
> - https://i.ibb.co/SwjTyJk/Kernel-5-16-alpha1-Power-PC.png
> - https://i.ibb.co/LkpWNPx/Kernel-5-15-rc5-Power-PC.png
> - https://i.ibb.co/F8q1jDR/Kernel-5-15-rc4-Power-PC.png
> - https://i.ibb.co/zZxrbhV/Kernel-5-15-alpha6-Power-PC.png
> - 
> https://i.pinimg.com/originals/a8/8b/42/a88b422870201887fc01ef44ddc1a235.png
> - 
> https://i.pinimg.com/originals/57/d9/83/57d98324cd055b7ae00a87ad5a45a42f.png
> - 
> https://i.pinimg.com/originals/f2/a5/e3/f2a5e34e2015381b0cb87cc51232a8bc.png
> - 
> https://i.pinimg.com/originals/c5/0d/85/c50d85d7e8f20b4caa1a439faf751964.png
> - 
> https://i.pinimg.com/originals/6e/3b/59/6e3b59fe10276c5644b15622a81f43f1.png
>
> We solved some issues:
>
> - https://forum.hyperion-entertainment.com/viewtopic.php?p=54357#p54357
> - 
> https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-November/236307.html
> - 
> https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-September/249021.html
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-May/229103.html
> - 
> https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-January/223342.html
> - https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216379.html
> - 
> https://forum.hyperion-entertainment.com/viewtopic.php?f=58&t=4655&p=53393&hilit=KVM#p53393
> - https://forum.hyperion-entertainment.com/viewtopic.php?p=53209#p53209
>
> Please, do not remove KVM support from Book3e because it works without 
> any problems and fast. We need it for our work.
>
> Thanks,
> Christian
>
>
>
>
>     On 12/2/22 12:04, Daniel Henrique Barboza wrote:
>
>         On 11/30/22 17:45, Crystal Wood wrote:
>
>             On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:
>
>                 BookE KVM is in a deep maintenance state, I'm not sure 
> how much testing
>                 it gets. I don't have a test setup, and it does not 
> look like QEMU has
>                 any HV architecture enabled. It hasn't been too 
> painful but there are
>                 some cases where it causes a bit of problem not being 
> able to test, e.g.,
>
> https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html
>
>                 Time to begin removal process, or are there still 
> people using it? I'm
>                 happy to to keep making occasional patches to try keep 
> it going if
>                 there are people testing upstream. Getting HV support 
> into QEMU would
>                 help with long term support, not sure how big of a job 
> that would be.
>
>
>             Not sure what you mean about QEMU not having e500 HV 
> support?  I don't know if
>             it's bitrotted, but it's there.
>
>
>         AFAIK all QEMU ppc boards, aside from pSeries and the Mac 
> ones, are always used
>         in
>         emulated mode in an use case similar to what Bernhard 
> described in his reply
>         (run
>         in x86 due to lack of ppc hardware).
>
>         I am not aware of e500 KVM support in QEMU since I never 
> attempted it. But yes,
>         it is present, but poorly tested - if tested at all. And the 
> reason why there's
>         no push on our side to removed it from QEMU is because its 
> code is so entwined
>         with pSeries KVM that it would take too much effort.
>
>         Do not take the presence of e500 KVM support in QEMU as a 
> blocker to disabled
>         it in
>         the kernel. As far as the current QEMU usage goes e500 KVM can 
> be removed
>         without
>         too much drama from our side.
>
>         Cedric, do you have any opinions about it?
>
>
>
>     I can not tell how much e500 KVM is used. The last report we had
>     on the topic was :
>
> https://lore.kernel.org/all/R4OPHT$7F12C66D1107397991E0E4C978FE6AF1@locati.it/ 
>
>
>     and the last commit mentioning e500 VMs I could find is cb3778a045,
>     which brings us back to QEMU 2.2 or so.
>
>     It would be nice to 'quickly' check the state of the KVM stack on
>     such boards and, may be, plan for more cleanups.
>
>
>     Thanks,
>
>     C.
>
>
>
>         Daniel
>
>
>
>             I don't know whether anyone is still using this, but if 
> they are, it's
>             probably e500mc and not e500v2 (which involved a bunch of 
> hacks to get almost-
>             sorta-usable performance out of hardware not designed for 
> virtualization).  I
>             do see that there have been a few recent patches on QEMU 
> e500 (beyond the
>             treewide cleanup type stuff), though I don't know if 
> they're using KVM.  CCing
>             them and the QEMU list.
>
>             I have an e6500 I could occasionally test on, if it turns 
> out people do still
>             care about this.  Don't count me as the use case, though. :-)
>
>             FWIW, as far as the RECONCILE_IRQ_STATE issue, that used 
> to be done in
>             kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 
> to be "cleaner and
>             faster". :-P
>
>             -Crystal
>
>
>
> Dear all,
> please, do not proceed removing KVM support to Book3e.
>
> The PowerProgressCommunity, our not-for-profit organisation of PowerPC 
> and alternative platforms enthusiasts is in the process of hardware 
> testing our new “Powerboard Tyche”, a fully open hardware motherboard 
> for a laptop based on a NXP T2080 which is a Book3e e6500 CPU (4 
> physical cores, 8 logical cores), see our recent blog post about it
> https://www.powerpc-notebook.org/2022/12/prototypes-produced-lets-go-on-hardware-tests/ 
>
>
> With such a board we hope to have a new small niche group of users of 
> a PowerPC Book3e platform that will be more than happy to keep using 
> QEMU with KVM enabled as using VMs without KVM is way too slow.
>
> At the moment we still have a working NXP DevKit based on the same 
> T2080 CPU and thank to a patch submitted back in December 2021 we can 
> use QEMU with KVM enable on it.
> The only issue we have so far is that it only starts when setting it 
> as “e5500” (that does not have altivec) because starting it with e6500 
> (that has altivec) it does not start.
>
> We published a blog post about QEMU with KVM enable back in December 
> 2021, I made a screenshot with multiple QEMU instances
> https://www.powerpc-notebook.org/2021/12/december-2021-updates-facing-electronic-components-shortages/ 
>
>
> I am also aware of users of the A-Eon AmigaOne X5000 computers (still 
> being produced and sold) that successfully use QEMU with KVM enabled, 
> their system are based either on a NXP P5020 or P5040, both Book3e 
> e5500, so without altivec.
>
> So, again, please, do not remove KVM support to Book3e we need that!
>
> Regards,
> Mario
>

Re: [RFC PATCH] Disable Book-E KVM support?

[Christian Zigotzky]

Hi All,

We regularly use QEMU with KVM HV on our A-EON AmigaOne X5000 machines 
(book3e). It works fast and without any problems.

Screenshot tour of QEMU/KVM HV on our AmigaOnes:

- https://i.ibb.co/m4vgwNT/Kernel-6-1-rc3-Power-PC.png
- https://i.ibb.co/Fwdjf7Z/Kernel-6-0-rc6-Power-PC.png
- https://i.ibb.co/LYnJGdF/Kernel-5-19-rc5-Power-PC-2.png
- https://i.ibb.co/vz1Wm5z/QEMU-with-9p-and-USB-sound.png
- https://i.ibb.co/ScMjtp7/Kernel-5-17-alpha5-Power-PC.png
- https://i.ibb.co/LQryFcK/Kernel-5-17-alpha4-Power-PC.png
- https://i.ibb.co/kKLx9mf/Kernel-5-10-89-Power-PC.png
- https://i.ibb.co/LRG1RDV/Kernel-5-10-89-Power-PC-2.png
- https://i.ibb.co/NCFqY0k/QEMU-USB-Audio-on-Void-PPC.png
- https://i.ibb.co/N1vL5Kd/Kernel-5-16-alpha3-Power-PC.png
- https://i.ibb.co/SwjTyJk/Kernel-5-16-alpha1-Power-PC.png
- https://i.ibb.co/LkpWNPx/Kernel-5-15-rc5-Power-PC.png
- https://i.ibb.co/F8q1jDR/Kernel-5-15-rc4-Power-PC.png
- https://i.ibb.co/zZxrbhV/Kernel-5-15-alpha6-Power-PC.png
- 
https://i.pinimg.com/originals/a8/8b/42/a88b422870201887fc01ef44ddc1a235.png
- 
https://i.pinimg.com/originals/57/d9/83/57d98324cd055b7ae00a87ad5a45a42f.png
- 
https://i.pinimg.com/originals/f2/a5/e3/f2a5e34e2015381b0cb87cc51232a8bc.png
- 
https://i.pinimg.com/originals/c5/0d/85/c50d85d7e8f20b4caa1a439faf751964.png
- 
https://i.pinimg.com/originals/6e/3b/59/6e3b59fe10276c5644b15622a81f43f1.png

We solved some issues:

- https://forum.hyperion-entertainment.com/viewtopic.php?p=54357#p54357
- https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-November/236307.html
- https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-September/249021.html
- https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-May/229103.html
- https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-January/223342.html
- https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216379.html
- 
https://forum.hyperion-entertainment.com/viewtopic.php?f=58&t=4655&p=53393&hilit=KVM#p53393
- https://forum.hyperion-entertainment.com/viewtopic.php?p=53209#p53209

Please, do not remove KVM support from Book3e because it works without 
any problems and fast. We need it for our work.

Thanks,
Christian




     On 12/2/22 12:04, Daniel Henrique Barboza wrote:

         On 11/30/22 17:45, Crystal Wood wrote:

             On Mon, 2022-11-28 at 14:36 +1000, Nicholas Piggin wrote:

                 BookE KVM is in a deep maintenance state, I'm not sure 
how much testing
                 it gets. I don't have a test setup, and it does not 
look like QEMU has
                 any HV architecture enabled. It hasn't been too painful 
but there are
                 some cases where it causes a bit of problem not being 
able to test, e.g.,

https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-November/251452.html

                 Time to begin removal process, or are there still 
people using it? I'm
                 happy to to keep making occasional patches to try keep 
it going if
                 there are people testing upstream. Getting HV support 
into QEMU would
                 help with long term support, not sure how big of a job 
that would be.


             Not sure what you mean about QEMU not having e500 HV 
support?  I don't know if
             it's bitrotted, but it's there.


         AFAIK all QEMU ppc boards, aside from pSeries and the Mac ones, 
are always used
         in
         emulated mode in an use case similar to what Bernhard described 
in his reply
         (run
         in x86 due to lack of ppc hardware).

         I am not aware of e500 KVM support in QEMU since I never 
attempted it. But yes,
         it is present, but poorly tested - if tested at all. And the 
reason why there's
         no push on our side to removed it from QEMU is because its code 
is so entwined
         with pSeries KVM that it would take too much effort.

         Do not take the presence of e500 KVM support in QEMU as a 
blocker to disabled
         it in
         the kernel. As far as the current QEMU usage goes e500 KVM can 
be removed
         without
         too much drama from our side.

         Cedric, do you have any opinions about it?



     I can not tell how much e500 KVM is used. The last report we had
     on the topic was :

https://lore.kernel.org/all/R4OPHT$7F12C66D1107397991E0E4C978FE6AF1@locati.it/

     and the last commit mentioning e500 VMs I could find is cb3778a045,
     which brings us back to QEMU 2.2 or so.

     It would be nice to 'quickly' check the state of the KVM stack on
     such boards and, may be, plan for more cleanups.


     Thanks,

     C.



         Daniel



             I don't know whether anyone is still using this, but if 
they are, it's
             probably e500mc and not e500v2 (which involved a bunch of 
hacks to get almost-
             sorta-usable performance out of hardware not designed for 
virtualization).  I
             do see that there have been a few recent patches on QEMU 
e500 (beyond the
             treewide cleanup type stuff), though I don't know if 
they're using KVM.  CCing
             them and the QEMU list.

             I have an e6500 I could occasionally test on, if it turns 
out people do still
             care about this.  Don't count me as the use case, though. :-)

             FWIW, as far as the RECONCILE_IRQ_STATE issue, that used to 
be done in
             kvmppc_handle_exit(), but was moved in commit 9bd880a2c882 
to be "cleaner and
             faster". :-P

             -Crystal



Dear all,
please, do not proceed removing KVM support to Book3e.

The PowerProgressCommunity, our not-for-profit organisation of PowerPC 
and alternative platforms enthusiasts is in the process of hardware 
testing our new “Powerboard Tyche”, a fully open hardware motherboard 
for a laptop based on a NXP T2080 which is a Book3e e6500 CPU (4 
physical cores, 8 logical cores), see our recent blog post about it
https://www.powerpc-notebook.org/2022/12/prototypes-produced-lets-go-on-hardware-tests/

With such a board we hope to have a new small niche group of users of a 
PowerPC Book3e platform that will be more than happy to keep using QEMU 
with KVM enabled as using VMs without KVM is way too slow.

At the moment we still have a working NXP DevKit based on the same T2080 
CPU and thank to a patch submitted back in December 2021 we can use QEMU 
with KVM enable on it.
The only issue we have so far is that it only starts when setting it as 
“e5500” (that does not have altivec) because starting it with e6500 
(that has altivec) it does not start.

We published a blog post about QEMU with KVM enable back in December 
2021, I made a screenshot with multiple QEMU instances
https://www.powerpc-notebook.org/2021/12/december-2021-updates-facing-electronic-components-shortages/

I am also aware of users of the A-Eon AmigaOne X5000 computers (still 
being produced and sold) that successfully use QEMU with KVM enabled, 
their system are based either on a NXP P5020 or P5040, both Book3e 
e5500, so without altivec.

So, again, please, do not remove KVM support to Book3e we need that!

Regards,
Mario