* [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN @ 2022-09-26 16:51 Paolo Bonzini 2022-09-27 17:07 ` Sean Christopherson 0 siblings, 1 reply; 18+ messages in thread From: Paolo Bonzini @ 2022-09-26 16:51 UTC (permalink / raw) To: linux-kernel, kvm; +Cc: seanjc 32-bit KVM has extra complications in the code due to: - different ways to write 64-bit values in VMCS - different handling of DS and ES selectors as well as FS/GS bases - lack of CR8 and EFER - lack of XFD - impossibility of writing 64-bit PTEs atomically The last is the big one, because it prevents from using the TDP MMU unconditionally. Since 32-bit processors with virtualization extensions are a historical curiosity, and even 32-bit userspace can only deal with small guests due to limited address space, it makes sense to restrict KVM to 64-bit hosts and kernels. In preparation for that make 32-bit KVM depend on CONFIG_BROKEN, and opportunistically rephrase the help message---processors with virtualization extensions are supported even if they are not "fairly recent". Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- arch/x86/kvm/Kconfig | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index e3cbd7706136..a107df22ffee 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -21,6 +21,7 @@ if VIRTUALIZATION config KVM tristate "Kernel-based Virtual Machine (KVM) support" depends on HAVE_KVM + depends on X86_64 || BROKEN depends on HIGH_RES_TIMERS depends on X86_LOCAL_APIC select PREEMPT_NOTIFIERS @@ -50,12 +51,13 @@ config KVM select HAVE_KVM_PM_NOTIFIER if PM help Support hosting fully virtualized guest machines using hardware - virtualization extensions. You will need a fairly recent - processor equipped with virtualization extensions. You will also - need to select one or more of the processor modules below. + virtualization extensions. You will also need to select one or + more of the processor modules below. This module provides access to the hardware capabilities through - a character device node named /dev/kvm. + a character device node named /dev/kvm. You might have to enable + virtualization extensions in the firmware setup if the device is + not visible. To compile this as a module, choose M here: the module will be called kvm. -- 2.31.1 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-26 16:51 [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN Paolo Bonzini @ 2022-09-27 17:07 ` Sean Christopherson 2022-09-28 7:10 ` Maxim Levitsky 2022-09-28 10:04 ` Paolo Bonzini 0 siblings, 2 replies; 18+ messages in thread From: Sean Christopherson @ 2022-09-27 17:07 UTC (permalink / raw) To: Paolo Bonzini; +Cc: linux-kernel, kvm On Mon, Sep 26, 2022, Paolo Bonzini wrote: > 32-bit KVM has extra complications in the code due to: > > - different ways to write 64-bit values in VMCS > > - different handling of DS and ES selectors as well as FS/GS bases > > - lack of CR8 and EFER > > - lack of XFD > More for the list: - SVM is effectively restricted to PAE kernels due to NX requirements > - impossibility of writing 64-bit PTEs atomically It's not impossible, just ugly. KVM could use CMPXCHG8B to do all of the accesses for the TDP MMU, including the non-atomic reads and writes. > The last is the big one, because it prevents from using the TDP MMU > unconditionally. As above, if the TDP MMU really is the sticking point, that's solvable. The real justification for deprecating 32-bit KVM is that, outside of KVM developers, literally no one uses 32-bit KVM. I.e. any amount of effort that is required to continue supporting 32-bit kernels is a complete waste of resources. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-27 17:07 ` Sean Christopherson @ 2022-09-28 7:10 ` Maxim Levitsky 2022-09-28 9:55 ` Paolo Bonzini 2022-09-28 10:04 ` Paolo Bonzini 1 sibling, 1 reply; 18+ messages in thread From: Maxim Levitsky @ 2022-09-28 7:10 UTC (permalink / raw) To: Sean Christopherson, Paolo Bonzini; +Cc: linux-kernel, kvm On Tue, 2022-09-27 at 17:07 +0000, Sean Christopherson wrote: > On Mon, Sep 26, 2022, Paolo Bonzini wrote: > > 32-bit KVM has extra complications in the code due to: > > > > - different ways to write 64-bit values in VMCS > > > > - different handling of DS and ES selectors as well as FS/GS bases > > > > - lack of CR8 and EFER > > > > - lack of XFD > > > > More for the list: > > - SVM is effectively restricted to PAE kernels due to NX requirements > > > - impossibility of writing 64-bit PTEs atomically > > It's not impossible, just ugly. KVM could use CMPXCHG8B to do all of the accesses > for the TDP MMU, including the non-atomic reads and writes. > > > The last is the big one, because it prevents from using the TDP MMU > > unconditionally. > > As above, if the TDP MMU really is the sticking point, that's solvable. > > The real justification for deprecating 32-bit KVM is that, outside of KVM developers, > literally no one uses 32-bit KVM. I.e. any amount of effort that is required to > continue supporting 32-bit kernels is a complete waste of resources. > I also think that outside KVM developers nobody should be using KVM on 32 bit host. However for _developement_ I think that 32 bit KVM support is very useful, as it allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while, and can even probably be useful to some users (e.g running some legacy stuff in a VM, which includes a hypervisor, especially to run really legacy OSes / custom bare metal software, using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but other 32 bit nested hypervisors can be useful. Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long as current kernel works, it is useful to use the same kernel on host and guest. Best regards, Maxim Levitsky ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-28 7:10 ` Maxim Levitsky @ 2022-09-28 9:55 ` Paolo Bonzini 2022-09-28 16:12 ` Sean Christopherson 0 siblings, 1 reply; 18+ messages in thread From: Paolo Bonzini @ 2022-09-28 9:55 UTC (permalink / raw) To: Maxim Levitsky, Sean Christopherson; +Cc: linux-kernel, kvm On 9/28/22 09:10, Maxim Levitsky wrote: > I also think that outside KVM developers nobody should be using KVM on 32 bit host. > > However for_developement_ I think that 32 bit KVM support is very useful, as it > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while, > and can even probably be useful to some users (e.g running some legacy stuff in a VM, > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software, > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but > other 32 bit nested hypervisors can be useful. > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long > as current kernel works, it is useful to use the same kernel on host and guest. Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT reinjection. :) But really the ultimate solution to this would be to improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit. Paolo ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-28 9:55 ` Paolo Bonzini @ 2022-09-28 16:12 ` Sean Christopherson 2022-09-28 17:43 ` Maxim Levitsky 2022-09-28 17:44 ` Maxim Levitsky 0 siblings, 2 replies; 18+ messages in thread From: Sean Christopherson @ 2022-09-28 16:12 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Maxim Levitsky, linux-kernel, kvm On Wed, Sep 28, 2022, Paolo Bonzini wrote: > On 9/28/22 09:10, Maxim Levitsky wrote: > > I also think that outside KVM developers nobody should be using KVM on 32 bit host. > > > > However for_developement_ I think that 32 bit KVM support is very useful, as it > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while, > > and can even probably be useful to some users (e.g running some legacy stuff in a VM, > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software, > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but > > other 32 bit nested hypervisors can be useful. > > > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long > > as current kernel works, it is useful to use the same kernel on host and guest. > > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT > reinjection. :) But really the ultimate solution to this would be to > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit. Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors, but the maintenance cost is painfully high. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-28 16:12 ` Sean Christopherson @ 2022-09-28 17:43 ` Maxim Levitsky 2022-09-28 17:44 ` Maxim Levitsky 1 sibling, 0 replies; 18+ messages in thread From: Maxim Levitsky @ 2022-09-28 17:43 UTC (permalink / raw) To: Sean Christopherson, Paolo Bonzini; +Cc: linux-kernel, kvm On Wed, 2022-09-28 at 16:12 +0000, Sean Christopherson wrote: > On Wed, Sep 28, 2022, Paolo Bonzini wrote: > > On 9/28/22 09:10, Maxim Levitsky wrote: > > > I also think that outside KVM developers nobody should be using KVM on 32 bit host. > > > > > > However for_developement_ I think that 32 bit KVM support is very useful, as it > > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while, > > > and can even probably be useful to some users (e.g running some legacy stuff in a VM, > > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software, > > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but > > > other 32 bit nested hypervisors can be useful. > > > > > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long > > > as current kernel works, it is useful to use the same kernel on host and guest. > > > > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT > > reinjection. :) But really the ultimate solution to this would be to > > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit. > > Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors, > but the maintenance cost is painfully high. > But is is actually? I test it routinely and it it does work quite well IMHO. As far as my opinion goes I do volunteer to test this code more often, and I do not want to see the 32 bit KVM support be removed *yet*. Best regards, Maxim Levitsky ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-28 16:12 ` Sean Christopherson 2022-09-28 17:43 ` Maxim Levitsky @ 2022-09-28 17:44 ` Maxim Levitsky 2022-09-28 17:55 ` Sean Christopherson 1 sibling, 1 reply; 18+ messages in thread From: Maxim Levitsky @ 2022-09-28 17:44 UTC (permalink / raw) To: Sean Christopherson, Paolo Bonzini; +Cc: linux-kernel, kvm On Wed, 2022-09-28 at 16:12 +0000, Sean Christopherson wrote: > On Wed, Sep 28, 2022, Paolo Bonzini wrote: > > On 9/28/22 09:10, Maxim Levitsky wrote: > > > I also think that outside KVM developers nobody should be using KVM on 32 bit host. > > > > > > However for_developement_ I think that 32 bit KVM support is very useful, as it > > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while, > > > and can even probably be useful to some users (e.g running some legacy stuff in a VM, > > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software, > > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but > > > other 32 bit nested hypervisors can be useful. > > > > > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long > > > as current kernel works, it is useful to use the same kernel on host and guest. > > > > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT > > reinjection. :) But really the ultimate solution to this would be to > > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit. > > Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors, > but the maintenance cost is painfully high. > But is it actually? I test it routinely and it it does work quite well IMHO. I don't remember that there were that much breakage lately in this area. As far as my opinion goes I do volunteer to test this code more often, and I do not want to see the 32 bit KVM support be removed *yet*. Best regards, Maxim Levitsky ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-28 17:44 ` Maxim Levitsky @ 2022-09-28 17:55 ` Sean Christopherson 2022-09-29 13:26 ` Paolo Bonzini 0 siblings, 1 reply; 18+ messages in thread From: Sean Christopherson @ 2022-09-28 17:55 UTC (permalink / raw) To: Maxim Levitsky; +Cc: Paolo Bonzini, linux-kernel, kvm On Wed, Sep 28, 2022, Maxim Levitsky wrote: > On Wed, 2022-09-28 at 16:12 +0000, Sean Christopherson wrote: > > On Wed, Sep 28, 2022, Paolo Bonzini wrote: > > > On 9/28/22 09:10, Maxim Levitsky wrote: > > > > I also think that outside KVM developers nobody should be using KVM on 32 bit host. > > > > > > > > However for_developement_ I think that 32 bit KVM support is very useful, as it > > > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while, > > > > and can even probably be useful to some users (e.g running some legacy stuff in a VM, > > > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software, > > > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but > > > > other 32 bit nested hypervisors can be useful. > > > > > > > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long > > > > as current kernel works, it is useful to use the same kernel on host and guest. > > > > > > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT > > > reinjection. :) But really the ultimate solution to this would be to > > > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit. > > > > Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors, > > but the maintenance cost is painfully high. > > > > But is it actually? I test it routinely and it it does work quite well IMHO. > I don't remember that there were that much breakage lately in this area. Oh, I didn't mean that it actually requires a lot of attention in terms of bug fixes, what I meant by "maintenance cost" is the cost of testing that all the flavors of 32-bit KVM actually work. That can be automated to some extent, but there's a non-trivial cost to maintaining all that automation. > As far as my opinion goes I do volunteer to test this code more often, > and I do not want to see the 32 bit KVM support be removed *yet*. Yeah, I 100% agree that it shouldn't be removed until we have equivalent test coverage. But I do think it should an "off-by-default" sort of thing. Maybe BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-28 17:55 ` Sean Christopherson @ 2022-09-29 13:26 ` Paolo Bonzini 2022-09-29 13:52 ` Maxim Levitsky 0 siblings, 1 reply; 18+ messages in thread From: Paolo Bonzini @ 2022-09-29 13:26 UTC (permalink / raw) To: Sean Christopherson, Maxim Levitsky; +Cc: linux-kernel, kvm On 9/28/22 19:55, Sean Christopherson wrote: >> As far as my opinion goes I do volunteer to test this code more often, >> and I do not want to see the 32 bit KVM support be removed*yet*. > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test > coverage. But I do think it should an "off-by-default" sort of thing. Maybe > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? Yeah, maybe EXPERT is better but I'm not sure of the equivalent test coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but what's wrong with booting an older guest? Paolo ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-29 13:26 ` Paolo Bonzini @ 2022-09-29 13:52 ` Maxim Levitsky 2022-09-29 15:07 ` Paolo Bonzini 2023-02-17 19:36 ` Thomas Huth 0 siblings, 2 replies; 18+ messages in thread From: Maxim Levitsky @ 2022-09-29 13:52 UTC (permalink / raw) To: Paolo Bonzini, Sean Christopherson; +Cc: linux-kernel, kvm On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: > On 9/28/22 19:55, Sean Christopherson wrote: > > > As far as my opinion goes I do volunteer to test this code more often, > > > and I do not want to see the 32 bit KVM support be removed*yet*. > > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but > what's wrong with booting an older guest? From my point of view, using the same kernel source for host and the guest is easier because you know that both kernels behave the same. About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway one needs to compile a recent 32 bit kernel manually - thus IMHO whoever these days compiles a 32 bit kernel, knows what they are doing. I personally would wait few more releases when there is a pressing reason to remove this support. AFAIK, it is not really possible to remove most of the legacy direct mmu because shadowing mmu still can use it (I think Sean told me that once). Best regards, Maxim Levitsky > > Paolo > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-29 13:52 ` Maxim Levitsky @ 2022-09-29 15:07 ` Paolo Bonzini 2023-02-17 19:36 ` Thomas Huth 1 sibling, 0 replies; 18+ messages in thread From: Paolo Bonzini @ 2022-09-29 15:07 UTC (permalink / raw) To: Maxim Levitsky, Sean Christopherson; +Cc: linux-kernel, kvm On 9/29/22 15:52, Maxim Levitsky wrote: > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: >> On 9/28/22 19:55, Sean Christopherson wrote: >>>> As far as my opinion goes I do volunteer to test this code more often, >>>> and I do not want to see the 32 bit KVM support be removed*yet*. >>> >>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test >>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe >>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? >> >> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test >> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but >> what's wrong with booting an older guest? > > From my point of view, using the same kernel source for host and the guest > is easier because you know that both kernels behave the same. It is certainly easier, but it is less correct. You don't cover anything that KVM doesn't use. > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever > these days compiles a 32 bit kernel, knows what they are doing. > > I personally would wait few more releases when there is a pressing reason to remove > this support. > > AFAIK, it is not really possible to remove most of the legacy direct mmu > because shadowing mmu still can use it (I think Sean told me that once). Yeah, it won't let us remove a lot of code but there are several logic cleanups that become possible if the TDP case can just assume the TDP MMU is there. For example, there is no reason to have a cpu_role (as opposed to an mmu_page_role for the root) if you are building HPA->GPA page tables. (Which reminds me that toggling CR0.WP is still a hog with the TDP MMU). Paolo ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-29 13:52 ` Maxim Levitsky 2022-09-29 15:07 ` Paolo Bonzini @ 2023-02-17 19:36 ` Thomas Huth 2023-02-22 22:27 ` Sean Christopherson 1 sibling, 1 reply; 18+ messages in thread From: Thomas Huth @ 2023-02-17 19:36 UTC (permalink / raw) To: Maxim Levitsky, Paolo Bonzini, Sean Christopherson Cc: linux-kernel, kvm, Daniel P. Berrange On 29/09/2022 15.52, Maxim Levitsky wrote: > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: >> On 9/28/22 19:55, Sean Christopherson wrote: >>>> As far as my opinion goes I do volunteer to test this code more often, >>>> and I do not want to see the 32 bit KVM support be removed*yet*. >>> >>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test >>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe >>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? >> >> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test >> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but >> what's wrong with booting an older guest? > >>From my point of view, using the same kernel source for host and the guest > is easier because you know that both kernels behave the same. > > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever > these days compiles a 32 bit kernel, knows what they are doing. > > I personally would wait few more releases when there is a pressing reason to remove > this support. FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit KVM support from the kernel. The QEMU project currently struggles badly with keeping everything tested in the CI in a reasonable amount of time. The 32-bit KVM kernel support is the only reason to keep the qemu-system-i386 binary around - everything else can be covered with the qemu-system-x86_64 binary that is a superset of the -i386 variant (except for the KVM part as far as I know). Sure, we could also drop qemu-system-i386 from the CI without dropping the 32-bit KVM code in the kernel, but I guess things will rather bitrot there even faster in that case, so I'd appreciate if the kernel could drop the 32-bit in the near future, too. Thomas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2023-02-17 19:36 ` Thomas Huth @ 2023-02-22 22:27 ` Sean Christopherson 2023-02-23 7:01 ` Thomas Huth 0 siblings, 1 reply; 18+ messages in thread From: Sean Christopherson @ 2023-02-22 22:27 UTC (permalink / raw) To: Thomas Huth Cc: Maxim Levitsky, Paolo Bonzini, linux-kernel, kvm, Daniel P. Berrange On Fri, Feb 17, 2023, Thomas Huth wrote: > On 29/09/2022 15.52, Maxim Levitsky wrote: > > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: > > > On 9/28/22 19:55, Sean Christopherson wrote: > > > > > As far as my opinion goes I do volunteer to test this code more often, > > > > > and I do not want to see the 32 bit KVM support be removed*yet*. > > > > > > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test > > > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe > > > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? > > > > > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test > > > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but > > > what's wrong with booting an older guest? > > > > > From my point of view, using the same kernel source for host and the guest > > is easier because you know that both kernels behave the same. > > > > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway > > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever > > these days compiles a 32 bit kernel, knows what they are doing. > > > > I personally would wait few more releases when there is a pressing reason to remove > > this support. > > FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit > KVM support from the kernel. The QEMU project currently struggles badly with > keeping everything tested in the CI in a reasonable amount of time. The > 32-bit KVM kernel support is the only reason to keep the qemu-system-i386 > binary around - everything else can be covered with the qemu-system-x86_64 > binary that is a superset of the -i386 variant (except for the KVM part as > far as I know). > Sure, we could also drop qemu-system-i386 from the CI without dropping the > 32-bit KVM code in the kernel, but I guess things will rather bitrot there > even faster in that case, so I'd appreciate if the kernel could drop the > 32-bit in the near future, too. Ya, I would happily drop support for 32-bit kernels today, the only sticking point is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather large point. :-( ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2023-02-22 22:27 ` Sean Christopherson @ 2023-02-23 7:01 ` Thomas Huth 2023-02-23 8:33 ` Maxim Levitsky 0 siblings, 1 reply; 18+ messages in thread From: Thomas Huth @ 2023-02-23 7:01 UTC (permalink / raw) To: Sean Christopherson Cc: Maxim Levitsky, Paolo Bonzini, linux-kernel, kvm, Daniel P. Berrange On 22/02/2023 23.27, Sean Christopherson wrote: > On Fri, Feb 17, 2023, Thomas Huth wrote: >> On 29/09/2022 15.52, Maxim Levitsky wrote: >>> On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: >>>> On 9/28/22 19:55, Sean Christopherson wrote: >>>>>> As far as my opinion goes I do volunteer to test this code more often, >>>>>> and I do not want to see the 32 bit KVM support be removed*yet*. >>>>> >>>>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test >>>>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe >>>>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? >>>> >>>> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test >>>> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but >>>> what's wrong with booting an older guest? >>> >>>> From my point of view, using the same kernel source for host and the guest >>> is easier because you know that both kernels behave the same. >>> >>> About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway >>> one needs to compile a recent 32 bit kernel manually - thus IMHO whoever >>> these days compiles a 32 bit kernel, knows what they are doing. >>> >>> I personally would wait few more releases when there is a pressing reason to remove >>> this support. >> >> FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit >> KVM support from the kernel. The QEMU project currently struggles badly with >> keeping everything tested in the CI in a reasonable amount of time. The >> 32-bit KVM kernel support is the only reason to keep the qemu-system-i386 >> binary around - everything else can be covered with the qemu-system-x86_64 >> binary that is a superset of the -i386 variant (except for the KVM part as >> far as I know). >> Sure, we could also drop qemu-system-i386 from the CI without dropping the >> 32-bit KVM code in the kernel, but I guess things will rather bitrot there >> even faster in that case, so I'd appreciate if the kernel could drop the >> 32-bit in the near future, too. > > Ya, I would happily drop support for 32-bit kernels today, the only sticking point > is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather > large point. :-( From your point of view, would it be OK if QEMU dropped qemu-system-i386? I.e. would it be fine to use older versions of QEMU only for that test coverage (or do you even use a different userspace for testing that)? Thomas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2023-02-23 7:01 ` Thomas Huth @ 2023-02-23 8:33 ` Maxim Levitsky 2023-02-23 22:10 ` Sean Christopherson 0 siblings, 1 reply; 18+ messages in thread From: Maxim Levitsky @ 2023-02-23 8:33 UTC (permalink / raw) To: Thomas Huth, Sean Christopherson Cc: Paolo Bonzini, linux-kernel, kvm, Daniel P. Berrange On Thu, 2023-02-23 at 08:01 +0100, Thomas Huth wrote: > On 22/02/2023 23.27, Sean Christopherson wrote: > > On Fri, Feb 17, 2023, Thomas Huth wrote: > > > On 29/09/2022 15.52, Maxim Levitsky wrote: > > > > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: > > > > > On 9/28/22 19:55, Sean Christopherson wrote: > > > > > > > As far as my opinion goes I do volunteer to test this code more often, > > > > > > > and I do not want to see the 32 bit KVM support be removed*yet*. > > > > > > > > > > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test > > > > > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe > > > > > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? > > > > > > > > > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test > > > > > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but > > > > > what's wrong with booting an older guest? > > > > > From my point of view, using the same kernel source for host and the guest > > > > is easier because you know that both kernels behave the same. > > > > > > > > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway > > > > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever > > > > these days compiles a 32 bit kernel, knows what they are doing. > > > > > > > > I personally would wait few more releases when there is a pressing reason to remove > > > > this support. > > > > > > FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit > > > KVM support from the kernel. The QEMU project currently struggles badly with > > > keeping everything tested in the CI in a reasonable amount of time. The > > > 32-bit KVM kernel support is the only reason to keep the qemu-system-i386 > > > binary around - everything else can be covered with the qemu-system-x86_64 > > > binary that is a superset of the -i386 variant (except for the KVM part as > > > far as I know). > > > Sure, we could also drop qemu-system-i386 from the CI without dropping the > > > 32-bit KVM code in the kernel, but I guess things will rather bitrot there > > > even faster in that case, so I'd appreciate if the kernel could drop the > > > 32-bit in the near future, too. > > > > Ya, I would happily drop support for 32-bit kernels today, the only sticking point > > is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather > > large point. :-( > > From your point of view, would it be OK if QEMU dropped qemu-system-i386? > I.e. would it be fine to use older versions of QEMU only for that test > coverage (or do you even use a different userspace for testing that)? > > Thomas > From my point of view qemu-system-x86_64 does run 32 bit guests just fine. The only exception that I know is that gdbstub is somewhat broken, but that can be probably fixed. Best regards, Maxim Levitsky ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2023-02-23 8:33 ` Maxim Levitsky @ 2023-02-23 22:10 ` Sean Christopherson 2023-02-24 6:28 ` Thomas Huth 0 siblings, 1 reply; 18+ messages in thread From: Sean Christopherson @ 2023-02-23 22:10 UTC (permalink / raw) To: Maxim Levitsky Cc: Thomas Huth, Paolo Bonzini, linux-kernel, kvm, Daniel P. Berrange On Thu, Feb 23, 2023, Maxim Levitsky wrote: > On Thu, 2023-02-23 at 08:01 +0100, Thomas Huth wrote: > > On 22/02/2023 23.27, Sean Christopherson wrote: > > > On Fri, Feb 17, 2023, Thomas Huth wrote: > > > > On 29/09/2022 15.52, Maxim Levitsky wrote: > > > > > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: > > > > > > On 9/28/22 19:55, Sean Christopherson wrote: > > > > > > > > As far as my opinion goes I do volunteer to test this code more often, > > > > > > > > and I do not want to see the 32 bit KVM support be removed*yet*. > > > > > > > > > > > > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test > > > > > > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe > > > > > > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? > > > > > > > > > > > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test > > > > > > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but > > > > > > what's wrong with booting an older guest? > > > > > > From my point of view, using the same kernel source for host and the guest > > > > > is easier because you know that both kernels behave the same. > > > > > > > > > > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway > > > > > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever > > > > > these days compiles a 32 bit kernel, knows what they are doing. > > > > > > > > > > I personally would wait few more releases when there is a pressing reason to remove > > > > > this support. > > > > > > > > FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit > > > > KVM support from the kernel. The QEMU project currently struggles badly with > > > > keeping everything tested in the CI in a reasonable amount of time. The > > > > 32-bit KVM kernel support is the only reason to keep the qemu-system-i386 > > > > binary around - everything else can be covered with the qemu-system-x86_64 > > > > binary that is a superset of the -i386 variant (except for the KVM part as > > > > far as I know). > > > > Sure, we could also drop qemu-system-i386 from the CI without dropping the > > > > 32-bit KVM code in the kernel, but I guess things will rather bitrot there > > > > even faster in that case, so I'd appreciate if the kernel could drop the > > > > 32-bit in the near future, too. > > > > > > Ya, I would happily drop support for 32-bit kernels today, the only sticking point > > > is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather > > > large point. :-( > > > > From your point of view, would it be OK if QEMU dropped qemu-system-i386? > > I.e. would it be fine to use older versions of QEMU only for that test > > coverage (or do you even use a different userspace for testing that)? For me personally, I have no objection to dropping qemu-system-i386 support in future QEMU releases. I update my 32-bit images very, very infrequently, so I probably wouldn't even notice for like 5 years :-) > From my point of view qemu-system-x86_64 does run 32 bit guests just fine. Right, but unless I seriously misunderstand what qemu-system-x86_64 ecompasses, it can't be used to run guests of 32-bit _hosts_, which is what we need to test shadowing of 32-bit NPT. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2023-02-23 22:10 ` Sean Christopherson @ 2023-02-24 6:28 ` Thomas Huth 0 siblings, 0 replies; 18+ messages in thread From: Thomas Huth @ 2023-02-24 6:28 UTC (permalink / raw) To: Sean Christopherson, Maxim Levitsky Cc: Paolo Bonzini, linux-kernel, kvm, Daniel P. Berrange On 23/02/2023 23.10, Sean Christopherson wrote: > On Thu, Feb 23, 2023, Maxim Levitsky wrote: >> On Thu, 2023-02-23 at 08:01 +0100, Thomas Huth wrote: >>> On 22/02/2023 23.27, Sean Christopherson wrote: >>>> On Fri, Feb 17, 2023, Thomas Huth wrote: >>>>> On 29/09/2022 15.52, Maxim Levitsky wrote: >>>>>> On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote: >>>>>>> On 9/28/22 19:55, Sean Christopherson wrote: >>>>>>>>> As far as my opinion goes I do volunteer to test this code more often, >>>>>>>>> and I do not want to see the 32 bit KVM support be removed*yet*. >>>>>>>> >>>>>>>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test >>>>>>>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe >>>>>>>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option? >>>>>>> >>>>>>> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test >>>>>>> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but >>>>>>> what's wrong with booting an older guest? >>>>>>> From my point of view, using the same kernel source for host and the guest >>>>>> is easier because you know that both kernels behave the same. >>>>>> >>>>>> About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway >>>>>> one needs to compile a recent 32 bit kernel manually - thus IMHO whoever >>>>>> these days compiles a 32 bit kernel, knows what they are doing. >>>>>> >>>>>> I personally would wait few more releases when there is a pressing reason to remove >>>>>> this support. >>>>> >>>>> FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit >>>>> KVM support from the kernel. The QEMU project currently struggles badly with >>>>> keeping everything tested in the CI in a reasonable amount of time. The >>>>> 32-bit KVM kernel support is the only reason to keep the qemu-system-i386 >>>>> binary around - everything else can be covered with the qemu-system-x86_64 >>>>> binary that is a superset of the -i386 variant (except for the KVM part as >>>>> far as I know). >>>>> Sure, we could also drop qemu-system-i386 from the CI without dropping the >>>>> 32-bit KVM code in the kernel, but I guess things will rather bitrot there >>>>> even faster in that case, so I'd appreciate if the kernel could drop the >>>>> 32-bit in the near future, too. >>>> >>>> Ya, I would happily drop support for 32-bit kernels today, the only sticking point >>>> is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather >>>> large point. :-( >>> >>> From your point of view, would it be OK if QEMU dropped qemu-system-i386? >>> I.e. would it be fine to use older versions of QEMU only for that test >>> coverage (or do you even use a different userspace for testing that)? > > For me personally, I have no objection to dropping qemu-system-i386 support in > future QEMU releases. I update my 32-bit images very, very infrequently, so I > probably wouldn't even notice for like 5 years :-) > >> From my point of view qemu-system-x86_64 does run 32 bit guests just fine. > > Right, but unless I seriously misunderstand what qemu-system-x86_64 ecompasses, > it can't be used to run guests of 32-bit _hosts_, which is what we need to test > shadowing of 32-bit NPT. That's what I've been told in the past, too, and that's why I asked. Thanks for the clarification! To summarize: My takeaway is that nobody really needs qemu-system-i386 anymore for recent development - the remaining 32-bit KVM use cases can be done with older versions of QEMU instead, thus it should be fine for the QEMU project to drop qemu-system-i386 nowadays. Thanks, Thomas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN 2022-09-27 17:07 ` Sean Christopherson 2022-09-28 7:10 ` Maxim Levitsky @ 2022-09-28 10:04 ` Paolo Bonzini 1 sibling, 0 replies; 18+ messages in thread From: Paolo Bonzini @ 2022-09-28 10:04 UTC (permalink / raw) To: Sean Christopherson; +Cc: linux-kernel, kvm On 9/27/22 19:07, Sean Christopherson wrote: > On Mon, Sep 26, 2022, Paolo Bonzini wrote: >> 32-bit KVM has extra complications in the code due to: >> >> - different ways to write 64-bit values in VMCS >> >> - different handling of DS and ES selectors as well as FS/GS bases >> >> - lack of CR8 and EFER >> >> - lack of XFD >> > > More for the list: > > - SVM is effectively restricted to PAE kernels due to NX requirements True, but I'm not sure how it complicates the code? Do you mean having to kmap/kunmap, and if so are you thinking of making KVM depend on !HIGHMEM? >> - impossibility of writing 64-bit PTEs atomically > > It's not impossible, just ugly. KVM could use CMPXCHG8B to do all of the accesses > for the TDP MMU, including the non-atomic reads and writes. Ok, rephrased: ========== Breakage in 32-bit KVM is somewhat rare, but it did happen and developers themselves found out a few months later. This means that it is very unlikely that it has any users. 32-bit processors with virtualization extensions are a historical curiosity; 32-bit userspace can only deal with small guests due to limited address space. Hence, it makes sense to restrict x86 KVM to 64-bit hosts and kernels. This removes all the conditional code to deal with the above differences, and it also enables unconditional use of the TDP MMU: making it work on 32-bit systems would require contortions using CMPXCHG8B to write 64-bit PTEs atomically, and they are simply not worth it. ========== >> The last is the big one, because it prevents from using the TDP MMU >> unconditionally. > > As above, if the TDP MMU really is the sticking point, that's solvable. Yeah, but until now the maintainability cost of keeping 32-bit on life support was small. Using CMPXCHG8B in the TDP MMU is possibly worse than having two MMUs for the two-dimensional paging case, therefore dropping the old x86 direct MMU for me is what tips the balance in favor of removal. (Once David submits his MMU callbacks work, I still want to see if it's possible to preserve the "old" x86 direct MMU, for example for use in pKVM; however, it would not be in arch/x86/). Paolo > The real justification for deprecating 32-bit KVM is that, outside of KVM developers, > literally no one uses 32-bit KVM. I.e. any amount of effort that is required to > continue supporting 32-bit kernels is a complete waste of resources. > ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2023-02-24 6:29 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-09-26 16:51 [PATCH] KVM: x86: disable on 32-bit unless CONFIG_BROKEN Paolo Bonzini 2022-09-27 17:07 ` Sean Christopherson 2022-09-28 7:10 ` Maxim Levitsky 2022-09-28 9:55 ` Paolo Bonzini 2022-09-28 16:12 ` Sean Christopherson 2022-09-28 17:43 ` Maxim Levitsky 2022-09-28 17:44 ` Maxim Levitsky 2022-09-28 17:55 ` Sean Christopherson 2022-09-29 13:26 ` Paolo Bonzini 2022-09-29 13:52 ` Maxim Levitsky 2022-09-29 15:07 ` Paolo Bonzini 2023-02-17 19:36 ` Thomas Huth 2023-02-22 22:27 ` Sean Christopherson 2023-02-23 7:01 ` Thomas Huth 2023-02-23 8:33 ` Maxim Levitsky 2023-02-23 22:10 ` Sean Christopherson 2023-02-24 6:28 ` Thomas Huth 2022-09-28 10:04 ` Paolo Bonzini
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).