From mboxrd@z Thu Jan 1 00:00:00 1970 From: Valentine Sinitsyn Subject: Re: Nested paging in nested SVM setup Date: Thu, 21 Aug 2014 14:48:16 +0600 Message-ID: <53F5B250.3000206@gmail.com> References: <53A179C4.9060203@gmail.com> <53A18A6D.2050407@siemens.com> <53F44440.6070408@gmail.com> <53F44645.2000708@redhat.com> <53F45036.1070309@gmail.com> <53F45832.5080708@redhat.com> <53F59192.3070500@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit To: Paolo Bonzini , Jan Kiszka , kvm@vger.kernel.org Return-path: Received: from mail-lb0-f180.google.com ([209.85.217.180]:57966 "EHLO mail-lb0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753468AbaHUIsV (ORCPT ); Thu, 21 Aug 2014 04:48:21 -0400 Received: by mail-lb0-f180.google.com with SMTP id v6so7666123lbi.25 for ; Thu, 21 Aug 2014 01:48:19 -0700 (PDT) In-Reply-To: <53F59192.3070500@gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On 21.08.2014 12:28, Valentine Sinitsyn wrote: > KVM seems to work properly (no weird exceptions injected), although my > code now freezes (quick look on the trace suggests it's looping reading > APIC). Not sure whose bug is it, will look further. Looks like the problem is that if nested page tables maps some GPA to 0xfee00000 HPA, it's really mapped to this HPA, and not intercepted with KVM's virtual LAPIC implementation. Consider the following trace: > qemu-system-x86-344 [000] 644.974072: kvm_entry: vcpu 0 > qemu-system-x86-344 [000] 644.974075: kvm_exit: reason npf rip 0xffffffff8104e883 info 10000000d fee000f0 > qemu-system-x86-344 [000] 644.974075: kvm_page_fault: address fee000f0 error_code d > qemu-system-x86-344 [000] 644.974077: kvm_emulate_insn: 0:ffffffff8104e883:8b 87 00 b0 5f ff (prot64) > qemu-system-x86-344 [000] 644.974078: kvm_apic: apic_read APIC_SPIV = 0xf > qemu-system-x86-344 [000] 644.974079: kvm_mmio: mmio read len 4 gpa 0xfee000f0 val 0x72007200000000f > qemu-system-x86-344 [000] 644.974081: kvm_entry: vcpu 0 Here, I set up NPT so that any access to 0xfee00000 nested guest physical address cause VM exit. Then, my code writes or reads register that is mapped to 0xfee00000 KVM's GPA. kvm_apic is called, and everything works as expected. However, if I set up NTP to make 0xfee00000 nested guest physical address reads don't cause nested VM exit (by simply clearing U/S flag in the NPTE), I get: > qemu-system-x86-1066 [003] 1105.864286: kvm_exit: reason npf rip 0xffffffff8104eaa4 info 10000000f fee00310 > qemu-system-x86-1066 [003] 1105.864287: kvm_nested_vmexit: rip: 0xffffffff8104eaa4 reason: npf ext_inf1: 0x000000010000000f ext_inf2: 0x00000000fee00310 ext_int: 0x00000000 ext_int_err: 0x00000000 > qemu-system-x86-1066 [003] 1105.864287: kvm_page_fault: address fee00310 error_code f > qemu-system-x86-1064 [001] 1105.864288: kvm_exit: reason npf rip 0xffffffff8104e876 info 10000000f fee000b0 > qemu-system-x86-1066 [003] 1105.864289: kvm_emulate_insn: 0:ffffffff8104eaa4:89 14 25 10 b3 5f ff (prot64) > qemu-system-x86-1064 [001] 1105.864289: kvm_nested_vmexit: rip: 0xffffffff8104e876 reason: npf ext_inf1: 0x000000010000000f ext_inf2: 0x00000000fee000b0 ext_int: 0x00000000 ext_int_err: 0x00000000 > qemu-system-x86-1064 [001] 1105.864289: kvm_page_fault: address fee000b0 error_code f > qemu-system-x86-1064 [001] 1105.864291: kvm_emulate_insn: 0:ffffffff8104e876:89 b7 00 b0 5f ff (prot64) > qemu-system-x86-1066 [003] 1105.864292: kvm_inj_exception: e (0x2) > qemu-system-x86-1066 [003] 1105.864293: kvm_entry: vcpu 3 > qemu-system-x86-1064 [001] 1105.864294: kvm_inj_exception: e (0x2) > qemu-system-x86-1064 [001] 1105.864295: kvm_entry: vcpu 1 No kvm_apic: after NPTs are set up, no page faults caused by register read (error_code: d), to trap and emulate APIC access. So I'm returning to my original question: is this an intended behavior of KVM that APIC access on nested page tables level are not trapped, or is this a bug? Valentine