* About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-11 20:13 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-11 20:13 UTC (permalink / raw) To: kvm, qemu-devel Cc: Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral, Maxim Levitsky Hi all, I hope you are very well! May I know whether it is possible to enable two-dimensional page translation (e.g., Intel EPT) mechanisms and shadow page table mechanisms in Linux QEMU/KVM at the same time on a physical server? For example, if the physical server has 80 cores, is it possible to let 40 cores use Intel EPT mechanisms for page translation and the other 40 cores use shadow page table mechanisms? Thanks! Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-11 20:13 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-11 20:13 UTC (permalink / raw) To: kvm, qemu-devel Cc: Paolo Bonzini, mathieu.tarral, stefanha, Sean Christopherson, Maxim Levitsky Hi all, I hope you are very well! May I know whether it is possible to enable two-dimensional page translation (e.g., Intel EPT) mechanisms and shadow page table mechanisms in Linux QEMU/KVM at the same time on a physical server? For example, if the physical server has 80 cores, is it possible to let 40 cores use Intel EPT mechanisms for page translation and the other 40 cores use shadow page table mechanisms? Thanks! Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-11 20:13 ` harry harry @ 2021-07-12 9:49 ` Maxim Levitsky -1 siblings, 0 replies; 25+ messages in thread From: Maxim Levitsky @ 2021-07-12 9:49 UTC (permalink / raw) To: harry harry, kvm, qemu-devel Cc: Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > Hi all, > > I hope you are very well! May I know whether it is possible to enable > two-dimensional page translation (e.g., Intel EPT) mechanisms and > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > physical server? For example, if the physical server has 80 cores, is > it possible to let 40 cores use Intel EPT mechanisms for page > translation and the other 40 cores use shadow page table mechanisms? > Thanks! Nope sadly. EPT/NPT is enabled by a module param. Best regards, Maxim Levitsky > > Best, > Harry > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-12 9:49 ` Maxim Levitsky 0 siblings, 0 replies; 25+ messages in thread From: Maxim Levitsky @ 2021-07-12 9:49 UTC (permalink / raw) To: harry harry, kvm, qemu-devel Cc: Paolo Bonzini, mathieu.tarral, stefanha, Sean Christopherson On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > Hi all, > > I hope you are very well! May I know whether it is possible to enable > two-dimensional page translation (e.g., Intel EPT) mechanisms and > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > physical server? For example, if the physical server has 80 cores, is > it possible to let 40 cores use Intel EPT mechanisms for page > translation and the other 40 cores use shadow page table mechanisms? > Thanks! Nope sadly. EPT/NPT is enabled by a module param. Best regards, Maxim Levitsky > > Best, > Harry > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-12 9:49 ` Maxim Levitsky @ 2021-07-12 13:02 ` harry harry -1 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-12 13:02 UTC (permalink / raw) To: Maxim Levitsky Cc: kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Dear Maxim, Thanks for your reply. I knew, in our current design/implementation, EPT/NPT is enabled by a module param. I think it is possible to modify the QEMU/KVM code to let it support EPT/NPT and show page table (SPT) simultaneously (e.g., for an 80-core server, 40 cores use EPT/NPT and the other 40 cores use SPT). What do you think? Thanks! Best regards, Harry On Mon, Jul 12, 2021 at 4:49 AM Maxim Levitsky <mlevitsk@redhat.com> wrote: > > On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > > Hi all, > > > > I hope you are very well! May I know whether it is possible to enable > > two-dimensional page translation (e.g., Intel EPT) mechanisms and > > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > > physical server? For example, if the physical server has 80 cores, is > > it possible to let 40 cores use Intel EPT mechanisms for page > > translation and the other 40 cores use shadow page table mechanisms? > > Thanks! > > Nope sadly. EPT/NPT is enabled by a module param. > > Best regards, > Maxim Levitsky > > > > > Best, > > Harry > > > > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-12 13:02 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-12 13:02 UTC (permalink / raw) To: Maxim Levitsky Cc: kvm, qemu-devel, Sean Christopherson, mathieu.tarral, stefanha, Paolo Bonzini Dear Maxim, Thanks for your reply. I knew, in our current design/implementation, EPT/NPT is enabled by a module param. I think it is possible to modify the QEMU/KVM code to let it support EPT/NPT and show page table (SPT) simultaneously (e.g., for an 80-core server, 40 cores use EPT/NPT and the other 40 cores use SPT). What do you think? Thanks! Best regards, Harry On Mon, Jul 12, 2021 at 4:49 AM Maxim Levitsky <mlevitsk@redhat.com> wrote: > > On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > > Hi all, > > > > I hope you are very well! May I know whether it is possible to enable > > two-dimensional page translation (e.g., Intel EPT) mechanisms and > > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > > physical server? For example, if the physical server has 80 cores, is > > it possible to let 40 cores use Intel EPT mechanisms for page > > translation and the other 40 cores use shadow page table mechanisms? > > Thanks! > > Nope sadly. EPT/NPT is enabled by a module param. > > Best regards, > Maxim Levitsky > > > > > Best, > > Harry > > > > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-12 13:02 ` harry harry @ 2021-07-12 13:11 ` Maxim Levitsky -1 siblings, 0 replies; 25+ messages in thread From: Maxim Levitsky @ 2021-07-12 13:11 UTC (permalink / raw) To: harry harry Cc: kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Mon, 2021-07-12 at 08:02 -0500, harry harry wrote: > Dear Maxim, > > Thanks for your reply. I knew, in our current design/implementation, > EPT/NPT is enabled by a module param. I think it is possible to modify > the QEMU/KVM code to let it support EPT/NPT and show page table (SPT) > simultaneously (e.g., for an 80-core server, 40 cores use EPT/NPT and > the other 40 cores use SPT). What do you think? Thanks! > > Best regards, > Harry > > On Mon, Jul 12, 2021 at 4:49 AM Maxim Levitsky <mlevitsk@redhat.com> wrote: > > On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > > > Hi all, > > > > > > I hope you are very well! May I know whether it is possible to enable > > > two-dimensional page translation (e.g., Intel EPT) mechanisms and > > > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > > > physical server? For example, if the physical server has 80 cores, is > > > it possible to let 40 cores use Intel EPT mechanisms for page > > > translation and the other 40 cores use shadow page table mechanisms? > > > Thanks! > > > > Nope sadly. EPT/NPT is enabled by a module param. > > > > Best regards, > > Maxim Levitsky > > > > > Best, > > > Harry > > > For same VM, I don't think it is feasable. For multiple VMs make some use NPT/EPT and some don't, this should be possible to implement. Why do you need it? Best regards, Maxim Levitsky ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-12 13:11 ` Maxim Levitsky 0 siblings, 0 replies; 25+ messages in thread From: Maxim Levitsky @ 2021-07-12 13:11 UTC (permalink / raw) To: harry harry Cc: kvm, qemu-devel, Sean Christopherson, mathieu.tarral, stefanha, Paolo Bonzini On Mon, 2021-07-12 at 08:02 -0500, harry harry wrote: > Dear Maxim, > > Thanks for your reply. I knew, in our current design/implementation, > EPT/NPT is enabled by a module param. I think it is possible to modify > the QEMU/KVM code to let it support EPT/NPT and show page table (SPT) > simultaneously (e.g., for an 80-core server, 40 cores use EPT/NPT and > the other 40 cores use SPT). What do you think? Thanks! > > Best regards, > Harry > > On Mon, Jul 12, 2021 at 4:49 AM Maxim Levitsky <mlevitsk@redhat.com> wrote: > > On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > > > Hi all, > > > > > > I hope you are very well! May I know whether it is possible to enable > > > two-dimensional page translation (e.g., Intel EPT) mechanisms and > > > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > > > physical server? For example, if the physical server has 80 cores, is > > > it possible to let 40 cores use Intel EPT mechanisms for page > > > translation and the other 40 cores use shadow page table mechanisms? > > > Thanks! > > > > Nope sadly. EPT/NPT is enabled by a module param. > > > > Best regards, > > Maxim Levitsky > > > > > Best, > > > Harry > > > For same VM, I don't think it is feasable. For multiple VMs make some use NPT/EPT and some don't, this should be possible to implement. Why do you need it? Best regards, Maxim Levitsky ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-12 13:11 ` Maxim Levitsky (?) @ 2021-07-12 14:56 ` Sean Christopherson 2021-07-14 5:30 ` harry harry -1 siblings, 1 reply; 25+ messages in thread From: Sean Christopherson @ 2021-07-12 14:56 UTC (permalink / raw) To: Maxim Levitsky Cc: harry harry, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Mon, Jul 12, 2021, Maxim Levitsky wrote: > On Mon, 2021-07-12 at 08:02 -0500, harry harry wrote: > > Dear Maxim, > > > > Thanks for your reply. I knew, in our current design/implementation, > > EPT/NPT is enabled by a module param. I think it is possible to modify > > the QEMU/KVM code to let it support EPT/NPT and show page table (SPT) > > simultaneously (e.g., for an 80-core server, 40 cores use EPT/NPT and > > the other 40 cores use SPT). What do you think? Thanks! > > > > Best regards, > > Harry > > > > On Mon, Jul 12, 2021 at 4:49 AM Maxim Levitsky <mlevitsk@redhat.com> wrote: > > > On Sun, 2021-07-11 at 15:13 -0500, harry harry wrote: > > > > Hi all, > > > > > > > > I hope you are very well! May I know whether it is possible to enable > > > > two-dimensional page translation (e.g., Intel EPT) mechanisms and > > > > shadow page table mechanisms in Linux QEMU/KVM at the same time on a > > > > physical server? For example, if the physical server has 80 cores, is > > > > it possible to let 40 cores use Intel EPT mechanisms for page > > > > translation and the other 40 cores use shadow page table mechanisms? > > > > Thanks! > > > > > > Nope sadly. EPT/NPT is enabled by a module param. > > > > For same VM, I don't think it is feasable. Heh, because the MMUs are all per-vCPU, it actually wouldn't be that much effort beyond supporting !TDP and TDP for different VMs... > For multiple VMs make some use NPT/EPT and some don't, > this should be possible to implement. ...but supporting !TDP and TDP in a single KVM instance isn't going to happen. It's certainly possible, but comes with a very high complexity cost, and likely even performance costs. The more sane way to support !TDP and TDP on a single host would be to support multiple instances of KVM, e.g. /dev/kvm0, /dev/kvm1, etc... Being able to use !TDP and TDP isn't strong justification for the work required, but supporting multiple KVM instances would allow upgrading KVM without having to migrate VMs off the host, which is very desirable. If multiple KVM instances are supported, running !TDP and TDP KVM instances should Just Work. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-12 14:56 ` Sean Christopherson @ 2021-07-14 5:30 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-14 5:30 UTC (permalink / raw) To: Sean Christopherson Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Dear Sean, Thanks for the comments! > Heh, because the MMUs are all per-vCPU, it actually wouldn't be that much effort > beyond supporting !TDP and TDP for different VMs... > Sorry, may I know what do you mean by "MMUs are all per-vCPU"? Do you mean the MMUs walk the page tables of each vCPU? > ...but supporting !TDP and TDP in a single KVM instance isn't going to happen. > It's certainly possible, but comes with a very high complexity cost, and likely > even performance costs. For one KVM instance, I think it might be possible to let several physical cores use !TDP and other cores use TDP but I am not sure about the implementation complexity. > > The more sane way to support !TDP and TDP on a single host would be to support > multiple instances of KVM, e.g. /dev/kvm0, /dev/kvm1, etc... Being able to use > !TDP and TDP isn't strong justification for the work required, but supporting > multiple KVM instances would allow upgrading KVM without having to migrate VMs > off the host, which is very desirable. If multiple KVM instances are supported, > running !TDP and TDP KVM instances should Just Work. Yes, for different KVM instances, it may be much easier but there might be some other issues, e.g., communication overhead between different instances. I think the upgrading idea is great but is very limited to local upgrading. Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-14 5:30 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-14 5:30 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, qemu-devel, Sean Christopherson, Maxim Levitsky, mathieu.tarral, stefanha, Paolo Bonzini Dear Sean, Thanks for the comments! > Heh, because the MMUs are all per-vCPU, it actually wouldn't be that much effort > beyond supporting !TDP and TDP for different VMs... > Sorry, may I know what do you mean by "MMUs are all per-vCPU"? Do you mean the MMUs walk the page tables of each vCPU? > ...but supporting !TDP and TDP in a single KVM instance isn't going to happen. > It's certainly possible, but comes with a very high complexity cost, and likely > even performance costs. For one KVM instance, I think it might be possible to let several physical cores use !TDP and other cores use TDP but I am not sure about the implementation complexity. > > The more sane way to support !TDP and TDP on a single host would be to support > multiple instances of KVM, e.g. /dev/kvm0, /dev/kvm1, etc... Being able to use > !TDP and TDP isn't strong justification for the work required, but supporting > multiple KVM instances would allow upgrading KVM without having to migrate VMs > off the host, which is very desirable. If multiple KVM instances are supported, > running !TDP and TDP KVM instances should Just Work. Yes, for different KVM instances, it may be much easier but there might be some other issues, e.g., communication overhead between different instances. I think the upgrading idea is great but is very limited to local upgrading. Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-14 5:30 ` harry harry (?) @ 2021-07-14 17:47 ` Sean Christopherson 2021-07-15 5:49 ` harry harry -1 siblings, 1 reply; 25+ messages in thread From: Sean Christopherson @ 2021-07-14 17:47 UTC (permalink / raw) To: harry harry Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Wed, Jul 14, 2021, harry harry wrote: > > Heh, because the MMUs are all per-vCPU, it actually wouldn't be that much effort > > beyond supporting !TDP and TDP for different VMs... > > Sorry, may I know what do you mean by "MMUs are all per-vCPU"? Do you > mean the MMUs walk the page tables of each vCPU? No, each vCPU has its own MMU instance, where an "MMU instance" is (mostly) a KVM construct. Per-vCPU MMU instances are necessary because each vCPU has its own relevant state, e.g. CR0, CR4, EFER, etc..., that affects the MMU instance in some way. E.g. the MMU instance is used to walk guest page tables when translating GVA->GPA for emulation, so per-vCPU MMUs are necessary even when using TDP. However, shadow/TDP PTEs are shared between compatible MMU instances. E.g. in the common case where all vCPUs in a VM use identical settings, there will effectively be a single set of TDP page tables shared by all vCPUs. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-14 17:47 ` Sean Christopherson @ 2021-07-15 5:49 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-15 5:49 UTC (permalink / raw) To: Sean Christopherson Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Hi Sean, > No, each vCPU has its own MMU instance, where an "MMU instance" is (mostly) a KVM > construct. Per-vCPU MMU instances are necessary because each vCPU has its own > relevant state, e.g. CR0, CR4, EFER, etc..., that affects the MMU instance in > some way. E.g. the MMU instance is used to walk guest page tables when > translating GVA->GPA for emulation, so per-vCPU MMUs are necessary even when > using TDP. > > However, shadow/TDP PTEs are shared between compatible MMU instances. E.g. in > the common case where all vCPUs in a VM use identical settings, there will > effectively be a single set of TDP page tables shared by all vCPUs. What do you mean by "MMU instance"? Do you mean VMCS? MMU is hardware. Could you please share me the code of the MMU instance in KVM? Thanks! ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-15 5:49 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-15 5:49 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, qemu-devel, Sean Christopherson, Maxim Levitsky, mathieu.tarral, stefanha, Paolo Bonzini Hi Sean, > No, each vCPU has its own MMU instance, where an "MMU instance" is (mostly) a KVM > construct. Per-vCPU MMU instances are necessary because each vCPU has its own > relevant state, e.g. CR0, CR4, EFER, etc..., that affects the MMU instance in > some way. E.g. the MMU instance is used to walk guest page tables when > translating GVA->GPA for emulation, so per-vCPU MMUs are necessary even when > using TDP. > > However, shadow/TDP PTEs are shared between compatible MMU instances. E.g. in > the common case where all vCPUs in a VM use identical settings, there will > effectively be a single set of TDP page tables shared by all vCPUs. What do you mean by "MMU instance"? Do you mean VMCS? MMU is hardware. Could you please share me the code of the MMU instance in KVM? Thanks! ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-15 5:49 ` harry harry (?) @ 2021-07-15 22:24 ` Sean Christopherson 2021-07-16 3:20 ` harry harry -1 siblings, 1 reply; 25+ messages in thread From: Sean Christopherson @ 2021-07-15 22:24 UTC (permalink / raw) To: harry harry Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Thu, Jul 15, 2021, harry harry wrote: > Hi Sean, > > > No, each vCPU has its own MMU instance, where an "MMU instance" is (mostly) a KVM > > construct. Per-vCPU MMU instances are necessary because each vCPU has its own > > relevant state, e.g. CR0, CR4, EFER, etc..., that affects the MMU instance in > > some way. E.g. the MMU instance is used to walk guest page tables when > > translating GVA->GPA for emulation, so per-vCPU MMUs are necessary even when > > using TDP. > > > > However, shadow/TDP PTEs are shared between compatible MMU instances. E.g. in > > the common case where all vCPUs in a VM use identical settings, there will > > effectively be a single set of TDP page tables shared by all vCPUs. > > What do you mean by "MMU instance"? Do you mean VMCS? MMU is hardware. No, an MMU is not a hardware-exclusive term, e.g. a software emulator will have an MMU to emulate the MMU of the target hardware. The terminology we use in KVM is roughly that a KVM MMU is KVM's presentation of a hardware MMU to the guest. E.g. when shadow paging is used, there is both the hardware MMU that is stuffed with KVM's shadow PTEs, and the KVM MMU that models the guest's MMU (the guest thinks its configuring a hardware MMU, but in reality KVM is intercepting (some) guest PTE modifications). When TDP (EPT) is used, the hardware MMU has two parts: the TDP PTEs that are controlled by KVM, and the IA32 PTEs that are controlled by the guest. And there's still a KVM MMU for the guest; the KVM MMU in that case knows how to connfigure the TDP PTEs in hardware _and_ walk the guest IA32 PTEs, e.g. to handle memory accesses during emulation. Even more fun, when nested TDP is used, there is a KVM MMU for L1, a KVM MMU for L1's EPT for L2, a KVM MMU for L2 (L2's legacy page tables), and the hardware MMU. > Could you please share me the code of the MMU instance in KVM? Thanks! struct kvm_mmu, and generally speaking everything under arch/x86/kvm/mmu/. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-15 22:24 ` Sean Christopherson @ 2021-07-16 3:20 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-16 3:20 UTC (permalink / raw) To: Sean Christopherson Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Hi Sean, Thanks for the explanations. Please see my comments below. Thanks! > When TDP (EPT) is used, the > hardware MMU has two parts: the TDP PTEs that are controlled by KVM, and the IA32 > PTEs that are controlled by the guest. And there's still a KVM MMU for the guest; > the KVM MMU in that case knows how to connfigure the TDP PTEs in hardware _and_ > walk the guest IA32 PTEs, e.g. to handle memory accesses during emulation. Sorry, I could not understand why the emulated MMU is still needed when TDP (e.g., Intel EPT) is used? In particular, in what situations, we need the emulated MMU to configure the TDP PTEs in hardware and walk the guest IA32 PTEs? Why do we need the emulated MMU in these situations? Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-16 3:20 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-16 3:20 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, qemu-devel, Sean Christopherson, Maxim Levitsky, mathieu.tarral, stefanha, Paolo Bonzini Hi Sean, Thanks for the explanations. Please see my comments below. Thanks! > When TDP (EPT) is used, the > hardware MMU has two parts: the TDP PTEs that are controlled by KVM, and the IA32 > PTEs that are controlled by the guest. And there's still a KVM MMU for the guest; > the KVM MMU in that case knows how to connfigure the TDP PTEs in hardware _and_ > walk the guest IA32 PTEs, e.g. to handle memory accesses during emulation. Sorry, I could not understand why the emulated MMU is still needed when TDP (e.g., Intel EPT) is used? In particular, in what situations, we need the emulated MMU to configure the TDP PTEs in hardware and walk the guest IA32 PTEs? Why do we need the emulated MMU in these situations? Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-16 3:20 ` harry harry (?) @ 2021-07-21 21:00 ` Sean Christopherson 2021-07-28 19:00 ` harry harry -1 siblings, 1 reply; 25+ messages in thread From: Sean Christopherson @ 2021-07-21 21:00 UTC (permalink / raw) To: harry harry Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Thu, Jul 15, 2021, harry harry wrote: > Hi Sean, > > Thanks for the explanations. Please see my comments below. Thanks! > > > When TDP (EPT) is used, the hardware MMU has two parts: the TDP PTEs that > > are controlled by KVM, and the IA32 PTEs that are controlled by the guest. > > And there's still a KVM MMU for the guest; the KVM MMU in that case knows > > how to connfigure the TDP PTEs in hardware _and_ walk the guest IA32 PTEs, > > e.g. to handle memory accesses during emulation. > > Sorry, I could not understand why the emulated MMU is still needed > when TDP (e.g., Intel EPT) is used? > In particular, in what situations, we need the emulated MMU to > configure the TDP PTEs in hardware and walk the guest IA32 PTEs? Ignoring some weird corner cases that blur the lines between emulation and hardware configuration, the emulated IA32 MMU isn't used to configure TDP PTEs in hardware, it's only used to walk the the guest page tables. > Why do we need the emulated MMU in these situations? For emulation of any instruction/flow that starts with a guest virtual address. On Intel CPUs, that includes quite literally any "full" instruction emulation, since KVM needs to translate CS:RIP to a guest physical address in order to fetch the guest's code stream. KVM can't avoid "full" emulation unless the guest is heavily enlightened, e.g. to avoid string I/O, among many other things. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-21 21:00 ` Sean Christopherson @ 2021-07-28 19:00 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-28 19:00 UTC (permalink / raw) To: Sean Christopherson Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Sean, sorry for the late reply. Thanks for your careful explanations. > For emulation of any instruction/flow that starts with a guest virtual address. > On Intel CPUs, that includes quite literally any "full" instruction emulation, > since KVM needs to translate CS:RIP to a guest physical address in order to fetch > the guest's code stream. KVM can't avoid "full" emulation unless the guest is > heavily enlightened, e.g. to avoid string I/O, among many other things. Do you mean the emulated MMU is needed when it *only* wants to translate GVAs to GPAs in the guest level? In such cases, the hardware MMU cannot be used because hardware MMU can only translate GVAs to HPAs, right? ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-28 19:00 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-28 19:00 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, qemu-devel, Sean Christopherson, Maxim Levitsky, mathieu.tarral, stefanha, Paolo Bonzini Sean, sorry for the late reply. Thanks for your careful explanations. > For emulation of any instruction/flow that starts with a guest virtual address. > On Intel CPUs, that includes quite literally any "full" instruction emulation, > since KVM needs to translate CS:RIP to a guest physical address in order to fetch > the guest's code stream. KVM can't avoid "full" emulation unless the guest is > heavily enlightened, e.g. to avoid string I/O, among many other things. Do you mean the emulated MMU is needed when it *only* wants to translate GVAs to GPAs in the guest level? In such cases, the hardware MMU cannot be used because hardware MMU can only translate GVAs to HPAs, right? ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-28 19:00 ` harry harry (?) @ 2021-07-28 20:01 ` Sean Christopherson 2021-08-05 19:42 ` harry harry -1 siblings, 1 reply; 25+ messages in thread From: Sean Christopherson @ 2021-07-28 20:01 UTC (permalink / raw) To: harry harry Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral On Wed, Jul 28, 2021, harry harry wrote: > Sean, sorry for the late reply. Thanks for your careful explanations. > > > For emulation of any instruction/flow that starts with a guest virtual address. > > On Intel CPUs, that includes quite literally any "full" instruction emulation, > > since KVM needs to translate CS:RIP to a guest physical address in order to fetch > > the guest's code stream. KVM can't avoid "full" emulation unless the guest is > > heavily enlightened, e.g. to avoid string I/O, among many other things. > > Do you mean the emulated MMU is needed when it *only* wants to > translate GVAs to GPAs in the guest level? Not quite, though gva_to_gpa() is the main use. The emulated MMU is also used to inject guest #PF and to load/store guest PDTPRs. > In such cases, the hardware MMU cannot be used because hardware MMU > can only translate GVAs to HPAs, right? Sort of. The hardware MMU does translate GVA to GPA, but the GPA value is not visible to software (unless the GPA->HPA translation faults). That's also true for VA to PA (and GVA to HPA). Irrespective of virtualization, x86 ISA doesn't provide an instruction to retrive the PA for a given VA. If such an instruction did exist, and it was to be usable for a VMM to do a GVA->GPA translation, the magic instruction would need to take all MMU params as operands, e.g. CR0, CR3, CR4, and EFER. When KVM is active (not the guest), the hardware MMU is loaded with the host MMU configuration, not the guest. In both VMX and SVM, vCPU state is mostly ephemeral in the sense that it ceases to exist in hardware when the vCPU exits to the host. Some state is retained in hardware, e.g. TLB and cache entries, but those are associated with select properties of the vCPU, e.g. EPTP, CR3, etc..., not with the vCPU itself, i.e. not with the VMCS (VMX) / VMCB (SVM). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-28 20:01 ` Sean Christopherson @ 2021-08-05 19:42 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-08-05 19:42 UTC (permalink / raw) To: Sean Christopherson Cc: Maxim Levitsky, kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Sean, understood with many thanks! Good luck, Harry On Wed, Jul 28, 2021 at 3:01 PM Sean Christopherson <seanjc@google.com> wrote: > > On Wed, Jul 28, 2021, harry harry wrote: > > Sean, sorry for the late reply. Thanks for your careful explanations. > > > > > For emulation of any instruction/flow that starts with a guest virtual address. > > > On Intel CPUs, that includes quite literally any "full" instruction emulation, > > > since KVM needs to translate CS:RIP to a guest physical address in order to fetch > > > the guest's code stream. KVM can't avoid "full" emulation unless the guest is > > > heavily enlightened, e.g. to avoid string I/O, among many other things. > > > > Do you mean the emulated MMU is needed when it *only* wants to > > translate GVAs to GPAs in the guest level? > > Not quite, though gva_to_gpa() is the main use. The emulated MMU is also used to > inject guest #PF and to load/store guest PDTPRs. > > > In such cases, the hardware MMU cannot be used because hardware MMU > > can only translate GVAs to HPAs, right? > > Sort of. The hardware MMU does translate GVA to GPA, but the GPA value is not > visible to software (unless the GPA->HPA translation faults). That's also true > for VA to PA (and GVA to HPA). Irrespective of virtualization, x86 ISA doesn't > provide an instruction to retrive the PA for a given VA. > > If such an instruction did exist, and it was to be usable for a VMM to do a > GVA->GPA translation, the magic instruction would need to take all MMU params as > operands, e.g. CR0, CR3, CR4, and EFER. When KVM is active (not the guest), the > hardware MMU is loaded with the host MMU configuration, not the guest. In both > VMX and SVM, vCPU state is mostly ephemeral in the sense that it ceases to exist > in hardware when the vCPU exits to the host. Some state is retained in hardware, > e.g. TLB and cache entries, but those are associated with select properties of > the vCPU, e.g. EPTP, CR3, etc..., not with the vCPU itself, i.e. not with the > VMCS (VMX) / VMCB (SVM). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-08-05 19:42 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-08-05 19:42 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, qemu-devel, Sean Christopherson, Maxim Levitsky, mathieu.tarral, stefanha, Paolo Bonzini Sean, understood with many thanks! Good luck, Harry On Wed, Jul 28, 2021 at 3:01 PM Sean Christopherson <seanjc@google.com> wrote: > > On Wed, Jul 28, 2021, harry harry wrote: > > Sean, sorry for the late reply. Thanks for your careful explanations. > > > > > For emulation of any instruction/flow that starts with a guest virtual address. > > > On Intel CPUs, that includes quite literally any "full" instruction emulation, > > > since KVM needs to translate CS:RIP to a guest physical address in order to fetch > > > the guest's code stream. KVM can't avoid "full" emulation unless the guest is > > > heavily enlightened, e.g. to avoid string I/O, among many other things. > > > > Do you mean the emulated MMU is needed when it *only* wants to > > translate GVAs to GPAs in the guest level? > > Not quite, though gva_to_gpa() is the main use. The emulated MMU is also used to > inject guest #PF and to load/store guest PDTPRs. > > > In such cases, the hardware MMU cannot be used because hardware MMU > > can only translate GVAs to HPAs, right? > > Sort of. The hardware MMU does translate GVA to GPA, but the GPA value is not > visible to software (unless the GPA->HPA translation faults). That's also true > for VA to PA (and GVA to HPA). Irrespective of virtualization, x86 ISA doesn't > provide an instruction to retrive the PA for a given VA. > > If such an instruction did exist, and it was to be usable for a VMM to do a > GVA->GPA translation, the magic instruction would need to take all MMU params as > operands, e.g. CR0, CR3, CR4, and EFER. When KVM is active (not the guest), the > hardware MMU is loaded with the host MMU configuration, not the guest. In both > VMX and SVM, vCPU state is mostly ephemeral in the sense that it ceases to exist > in hardware when the vCPU exits to the host. Some state is retained in hardware, > e.g. TLB and cache entries, but those are associated with select properties of > the vCPU, e.g. EPTP, CR3, etc..., not with the vCPU itself, i.e. not with the > VMCS (VMX) / VMCB (SVM). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM 2021-07-12 13:11 ` Maxim Levitsky @ 2021-07-14 5:22 ` harry harry -1 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-14 5:22 UTC (permalink / raw) To: Maxim Levitsky Cc: kvm, qemu-devel, Sean Christopherson, Paolo Bonzini, stefanha, mathieu.tarral Dear Maxim, Thanks for your reply! > For same VM, I don't think it is feasable. > > For multiple VMs make some use NPT/EPT and some don't, > this should be possible to implement. > > Why do you need it? > I am just curious about it :). Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM @ 2021-07-14 5:22 ` harry harry 0 siblings, 0 replies; 25+ messages in thread From: harry harry @ 2021-07-14 5:22 UTC (permalink / raw) To: Maxim Levitsky Cc: kvm, qemu-devel, Sean Christopherson, mathieu.tarral, stefanha, Paolo Bonzini Dear Maxim, Thanks for your reply! > For same VM, I don't think it is feasable. > > For multiple VMs make some use NPT/EPT and some don't, > this should be possible to implement. > > Why do you need it? > I am just curious about it :). Best, Harry ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2021-08-05 19:43 UTC | newest] Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-07-11 20:13 About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM harry harry 2021-07-11 20:13 ` harry harry 2021-07-12 9:49 ` Maxim Levitsky 2021-07-12 9:49 ` Maxim Levitsky 2021-07-12 13:02 ` harry harry 2021-07-12 13:02 ` harry harry 2021-07-12 13:11 ` Maxim Levitsky 2021-07-12 13:11 ` Maxim Levitsky 2021-07-12 14:56 ` Sean Christopherson 2021-07-14 5:30 ` harry harry 2021-07-14 5:30 ` harry harry 2021-07-14 17:47 ` Sean Christopherson 2021-07-15 5:49 ` harry harry 2021-07-15 5:49 ` harry harry 2021-07-15 22:24 ` Sean Christopherson 2021-07-16 3:20 ` harry harry 2021-07-16 3:20 ` harry harry 2021-07-21 21:00 ` Sean Christopherson 2021-07-28 19:00 ` harry harry 2021-07-28 19:00 ` harry harry 2021-07-28 20:01 ` Sean Christopherson 2021-08-05 19:42 ` harry harry 2021-08-05 19:42 ` harry harry 2021-07-14 5:22 ` harry harry 2021-07-14 5:22 ` harry harry
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.