From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Nadav Har'El" Subject: [PATCH 0/29] nVMX: Nested VMX, v8 Date: Thu, 27 Jan 2011 10:29:48 +0200 Message-ID: <1296116987-nyh@il.ibm.com> Cc: gleb@redhat.com, avi@redhat.com To: kvm@vger.kernel.org Return-path: Received: from mtagate4.uk.ibm.com ([194.196.100.164]:54481 "EHLO mtagate4.uk.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751358Ab1A0I34 (ORCPT ); Thu, 27 Jan 2011 03:29:56 -0500 Received: from d06nrmr1507.portsmouth.uk.ibm.com (d06nrmr1507.portsmouth.uk.ibm.com [9.149.38.233]) by mtagate4.uk.ibm.com (8.13.1/8.13.1) with ESMTP id p0R8Tt27004001 for ; Thu, 27 Jan 2011 08:29:55 GMT Received: from d06av03.portsmouth.uk.ibm.com (d06av03.portsmouth.uk.ibm.com [9.149.37.213]) by d06nrmr1507.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p0R8Tv0R1388550 for ; Thu, 27 Jan 2011 08:29:57 GMT Received: from d06av03.portsmouth.uk.ibm.com (localhost.localdomain [127.0.0.1]) by d06av03.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p0R8TssB029307 for ; Thu, 27 Jan 2011 01:29:54 -0700 Sender: kvm-owner@vger.kernel.org List-ID: Hi, This is the eighth iteration of the nested VMX patch set. This iteration solves a number of bugs and issues that bothered the reviewers. Some more issues raised in the previous review remain open, but don't worry - I *am* working to resolve all of them. The biggest improvement in this version is that SMP finally works: You can now run nested VMX on an SMP host - The "nosmp" kernel option is no longer required. You can also have SMP L1s and L2s, although in this version, SMP L2 support is still somewhat buggy and should be made more stable in the next version. The "vpid=0" option that used to be required is also no longer required. Other improvements include: * #GP on writing read-only VMX MSRs, don't save/restore them, and don't print annoying and incorrect messages on startup. * Cleanup free_l1_state() and renamed it free_nested(). * Removed guest expoitable printk()s. * Finally got rid of the l1_state structure and all its redundant fields. * Moved cpu and launched fields out of the (guest memory) vmcs12, and moved to a new structure (in host memory) saved_vmcs. Avi, you asked if and why these two fields are really needed - and they are needed, and I explained why in a comment. * Moved kunmap() out of nested_release_page() and into callers. * Made vmcs_field_to_offset_table initialization more readable. * Moved constants in vmx.c and to include files, as requested. * Fixed wrong MOV_SS check in handle_launch_or_resume(). * Fixed page leak in nested_vmx_exit_handled_msr(). * Removed redundant if(nested) check. * Allow turning off nested VMX for one guest (by removing VMX from cpuid). * Fixed the EFER handling code. This new set of patches applys to the current KVM trunk (I checked with 844e6679184180cffa7aca014d672545941ed78e). If you wish, you can also check out an already-patched version of KVM from the repository git://github.com/nyh/kvm-nested-vmx.git - take the branch "nvmx8". About nested VMX: ----------------- The following 29 patches implement nested VMX support. This feature enables a guest to use the VMX APIs in order to run its own nested guests. In other words, it allows running hypervisors (that use VMX) under KVM. Multiple guest hypervisors can be run concurrently, and each of those can in turn host multiple guests. The theory behind this work, our implementation, and its performance characteristics were presented in OSDI 2010 (the USENIX Symposium on Operating Systems Design and Implementation). Our paper was titled "The Turtles Project: Design and Implementation of Nested Virtualization", and was awarded "Jay Lepreau Best Paper". The paper is available online, at: http://www.usenix.org/events/osdi10/tech/full_papers/Ben-Yehuda.pdf This patch set does not include all the features described in the paper. In particular, this patch set is missing nested EPT (L1 can't use EPT and must use shadow page tables). It is also missing some features required to run VMWare hypervisors as a guest. These missing features will be sent as follow-on patchs. Running nested VMX: ------------------ The nested VMX feature is currently disabled by default. It must be explicitly enabled with the "nested=1" option to the kvm-intel module. No modifications are required to user space (qemu). However, qemu's default emulated CPU type (qemu64) does not list the "VMX" CPU feature, so it must be explicitly enabled, by giving qemu one of the following options: -cpu host (emulated CPU has all features of the real CPU) -cpu qemu64,+vmx (add just the vmx feature to a named CPU type) This version was only tested with KVM (64-bit) as a guest hypervisor, and Linux as a nested guest. Patch statistics: ----------------- Documentation/kvm/nested-vmx.txt | 241 ++ arch/x86/include/asm/kvm_host.h | 2 arch/x86/include/asm/msr-index.h | 9 arch/x86/include/asm/vmx.h | 31 arch/x86/kvm/svm.c | 6 arch/x86/kvm/vmx.c | 2496 ++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 10 arch/x86/kvm/x86.h | 6 8 files changed, 2760 insertions(+), 41 deletions(-) -- Nadav Har'El IBM Haifa Research Lab