From: Bandan Das <bsd@redhat.com>
To: Jintack Lim <jintack.lim@linaro.org>
Cc: kvmarm@lists.cs.columbia.edu,
"Christoffer Dall" <christoffer.dall@linaro.org>,
"Marc Zyngier" <marc.zyngier@arm.com>,
corbet@lwn.net, "Paolo Bonzini" <pbonzini@redhat.com>,
"Radim Krčmář" <rkrcmar@redhat.com>,
linux@armlinux.org.uk,
"Catalin Marinas" <catalin.marinas@arm.com>,
"Will Deacon" <will.deacon@arm.com>,
akpm@linux-foundation.org, mchehab@kernel.org,
cov@codeaurora.org, daniel.lezcano@linaro.org,
david.daney@cavium.com, mark.rutland@arm.com,
"Suzuki K Poulose" <suzuki.poulose@arm.com>,
stefan@hello-penguin.com, andy.gross@linaro.org,
wcohen@redhat.com, ard.biesheuvel@linaro.org,
shankerd@codeaurora.org, vladimir.murzin@arm.com,
james.morse@arm.com, linux-doc@vger.kernel.org,
"lkml - Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"KVM General" <kvm@vg>
Subject: Re: [RFC PATCH v2 00/38] Nested Virtualization on KVM/ARM
Date: Fri, 28 Jul 2017 16:13:04 -0400 [thread overview]
Message-ID: <jpgshhg7227.fsf@linux.bootlegged.copy> (raw)
In-Reply-To: CAHyh4xjkT+JNJL0kxmEjz4qU8kD2-ZwYgVB8VgwsSYjpjTc3LA@mail.gmail.com
Jintack Lim <jintack.lim@linaro.org> writes:
...
>>
>> I'll share my experiment setup shortly.
>
> I summarized my experiment setup here.
>
> https://github.com/columbia/nesting-pub/wiki/Nested-virtualization-on-ARM-setup
Thanks Jintack! I was able to test L2 boot up with these instructions.
Next, I will try to run some simple tests. Any suggestions on reducing the L2 bootup
time in my test setup ? I think I will try to make the L2 kernel print
less messages; and maybe just get rid of some of the userspace services.
I also applied the patch to reduce the timer frequency btw.
Bandan
>>
>> Even though this work has some limitations and TODOs, I'd appreciate early
>> feedback on this RFC. Specifically, I'm interested in:
>>
>> - Overall design to manage vcpu context for the virtual EL2
>> - Verifying correct EL2 register configurations such as HCR_EL2, CPTR_EL2
>> (Patch 30 and 32)
>> - Patch organization and coding style
>
> I also wonder if the hardware and/or KVM do not support nested
> virtualization but the userspace uses nested virtualization option,
> which one is better: giving an error or launching a regular VM
> silently.
>
>>
>> This patch series is based on kvm/next d38338e.
>> The whole patch series including memory, VGIC, and timer patches is available
>> here:
>>
>> git@github.com:columbia/nesting-pub.git rfc-v2
>>
>> Limitations:
>> - There are some cases that the target exception level of a VM is ambiguous when
>> emulating eret instruction. I'm discussing this issue with Christoffer and
>> Marc. Meanwhile, I added a temporary patch (not included in this
>> series. f1beaba in the repo) and used 4.10.0 kernel when testing the guest
>> hypervisor with VHE.
>> - Recursive nested virtualization is not tested yet.
>> - Other hypervisors (such as Xen) on KVM are not tested.
>>
>> TODO:
>> - Submit memory, VGIC, and timer patches
>> - Evaluate regular VM performance to see if there's a negative impact.
>> - Test other hypervisors such as Xen on KVM
>> - Test recursive nested virtualization
>>
>> v1-->v2:
>> - Added support for the virtual EL2 with VHE
>> - Rewrote commit messages and comments from the perspective of supporting
>> execution environments to VMs, rather than from the perspective of the guest
>> hypervisor running in them.
>> - Fixed a few bugs to make it run on the FastModel.
>> - Tested on ARMv8.3 with four configurations. (host/guest. with/without VHE.)
>> - Rebased to kvm/next
>>
>> [1] https://www.community.arm.com/processors/b/blog/posts/armv8-a-architecture-2016-additions
>>
>> Christoffer Dall (7):
>> KVM: arm64: Add KVM nesting feature
>> KVM: arm64: Allow userspace to set PSR_MODE_EL2x
>> KVM: arm64: Add vcpu_mode_el2 primitive to support nesting
>> KVM: arm/arm64: Add a framework to prepare virtual EL2 execution
>> arm64: Add missing TCR hw defines
>> KVM: arm64: Create shadow EL1 registers
>> KVM: arm64: Trap EL1 VM register accesses in virtual EL2
>>
>> Jintack Lim (31):
>> arm64: Add ARM64_HAS_NESTED_VIRT feature
>> KVM: arm/arm64: Enable nested virtualization via command-line
>> KVM: arm/arm64: Check if nested virtualization is in use
>> KVM: arm64: Add EL2 system registers to vcpu context
>> KVM: arm64: Add EL2 special registers to vcpu context
>> KVM: arm64: Add the shadow context for virtual EL2 execution
>> KVM: arm64: Set vcpu context depending on the guest exception level
>> KVM: arm64: Synchronize EL1 system registers on virtual EL2 entry and
>> exit
>> KVM: arm64: Move exception macros and enums to a common file
>> KVM: arm64: Support to inject exceptions to the virtual EL2
>> KVM: arm64: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
>> KVM: arm64: Trap CPACR_EL1 access in virtual EL2
>> KVM: arm64: Handle eret instruction traps
>> KVM: arm64: Set a handler for the system instruction traps
>> KVM: arm64: Handle PSCI call via smc from the guest
>> KVM: arm64: Inject HVC exceptions to the virtual EL2
>> KVM: arm64: Respect virtual HCR_EL2.TWX setting
>> KVM: arm64: Respect virtual CPTR_EL2.TFP setting
>> KVM: arm64: Add macros to support the virtual EL2 with VHE
>> KVM: arm64: Add EL2 registers defined in ARMv8.1 to vcpu context
>> KVM: arm64: Emulate EL12 register accesses from the virtual EL2
>> KVM: arm64: Support a VM with VHE considering EL0 of the VHE host
>> KVM: arm64: Allow the virtual EL2 to access EL2 states without trap
>> KVM: arm64: Manage the shadow states when virtual E2H bit enabled
>> KVM: arm64: Trap and emulate CPTR_EL2 accesses via CPACR_EL1 from the
>> virtual EL2 with VHE
>> KVM: arm64: Emulate appropriate VM control system registers
>> KVM: arm64: Respect the virtual HCR_EL2.NV bit setting
>> KVM: arm64: Respect the virtual HCR_EL2.NV bit setting for EL12
>> register traps
>> KVM: arm64: Respect virtual HCR_EL2.TVM and TRVM settings
>> KVM: arm64: Respect the virtual HCR_EL2.NV1 bit setting
>> KVM: arm64: Respect the virtual CPTR_EL2.TCPAC setting
>>
>> Documentation/admin-guide/kernel-parameters.txt | 4 +
>> arch/arm/include/asm/kvm_emulate.h | 17 ++
>> arch/arm/include/asm/kvm_host.h | 15 +
>> arch/arm64/include/asm/cpucaps.h | 3 +-
>> arch/arm64/include/asm/esr.h | 1 +
>> arch/arm64/include/asm/kvm_arm.h | 2 +
>> arch/arm64/include/asm/kvm_coproc.h | 3 +-
>> arch/arm64/include/asm/kvm_emulate.h | 56 ++++
>> arch/arm64/include/asm/kvm_host.h | 64 ++++-
>> arch/arm64/include/asm/kvm_hyp.h | 24 --
>> arch/arm64/include/asm/pgtable-hwdef.h | 6 +
>> arch/arm64/include/asm/sysreg.h | 70 +++++
>> arch/arm64/include/uapi/asm/kvm.h | 1 +
>> arch/arm64/kernel/asm-offsets.c | 1 +
>> arch/arm64/kernel/cpufeature.c | 11 +
>> arch/arm64/kvm/Makefile | 5 +-
>> arch/arm64/kvm/context.c | 346 +++++++++++++++++++++++
>> arch/arm64/kvm/emulate-nested.c | 83 ++++++
>> arch/arm64/kvm/guest.c | 2 +
>> arch/arm64/kvm/handle_exit.c | 89 +++++-
>> arch/arm64/kvm/hyp/entry.S | 13 +
>> arch/arm64/kvm/hyp/hyp-entry.S | 2 +-
>> arch/arm64/kvm/hyp/switch.c | 33 ++-
>> arch/arm64/kvm/hyp/sysreg-sr.c | 117 ++++----
>> arch/arm64/kvm/inject_fault.c | 12 -
>> arch/arm64/kvm/nested.c | 63 +++++
>> arch/arm64/kvm/reset.c | 8 +
>> arch/arm64/kvm/sys_regs.c | 359 +++++++++++++++++++++++-
>> arch/arm64/kvm/sys_regs.h | 8 +
>> arch/arm64/kvm/trace.h | 43 ++-
>> virt/kvm/arm/arm.c | 20 ++
>> 31 files changed, 1363 insertions(+), 118 deletions(-)
>> create mode 100644 arch/arm64/kvm/context.c
>> create mode 100644 arch/arm64/kvm/emulate-nested.c
>> create mode 100644 arch/arm64/kvm/nested.c
>>
>> --
>> 1.9.1
>>
next prev parent reply other threads:[~2017-07-28 20:13 UTC|newest]
Thread overview: 77+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-18 16:58 [RFC PATCH v2 00/38] Nested Virtualization on KVM/ARM Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 01/38] arm64: Add ARM64_HAS_NESTED_VIRT feature Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 02/38] KVM: arm/arm64: Enable nested virtualization via command-line Jintack Lim
2017-07-30 19:59 ` Christoffer Dall
2017-08-01 13:56 ` Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 03/38] KVM: arm64: Add KVM nesting feature Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 04/38] KVM: arm/arm64: Check if nested virtualization is in use Jintack Lim
2017-07-30 19:59 ` Christoffer Dall
2017-08-01 13:59 ` Jintack Lim
2017-07-30 19:59 ` Christoffer Dall
2017-08-01 14:07 ` Jintack Lim
2017-08-01 14:58 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 05/38] KVM: arm64: Allow userspace to set PSR_MODE_EL2x Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 06/38] KVM: arm64: Add vcpu_mode_el2 primitive to support nesting Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 07/38] KVM: arm64: Add EL2 system registers to vcpu context Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 08/38] KVM: arm64: Add EL2 special " Jintack Lim
2017-07-30 19:59 ` Christoffer Dall
2017-08-01 14:08 ` Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 09/38] KVM: arm64: Add the shadow context for virtual EL2 execution Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 10/38] KVM: arm/arm64: Add a framework to prepare " Jintack Lim
2017-07-30 12:02 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 11/38] KVM: arm64: Set vcpu context depending on the guest exception level Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 12/38] arm64: Add missing TCR hw defines Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 13/38] KVM: arm64: Create shadow EL1 registers Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 14/38] KVM: arm64: Synchronize EL1 system registers on virtual EL2 entry and exit Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 15/38] KVM: arm64: Move exception macros and enums to a common file Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 16/38] KVM: arm64: Support to inject exceptions to the virtual EL2 Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 17/38] KVM: arm64: Trap EL1 VM register accesses in " Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 18/38] KVM: arm64: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from " Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 19/38] KVM: arm64: Trap CPACR_EL1 access in " Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 20/38] KVM: arm64: Handle eret instruction traps Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-08-01 14:11 ` Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 21/38] KVM: arm64: Set a handler for the system " Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 22/38] KVM: arm64: Handle PSCI call via smc from the guest Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 23/38] KVM: arm64: Inject HVC exceptions to the virtual EL2 Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 24/38] KVM: arm64: Respect virtual HCR_EL2.TWX setting Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 25/38] KVM: arm64: Respect virtual CPTR_EL2.TFP setting Jintack Lim
2017-07-30 20:00 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 26/38] KVM: arm64: Add macros to support the virtual EL2 with VHE Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 27/38] KVM: arm64: Add EL2 registers defined in ARMv8.1 to vcpu context Jintack Lim
2017-07-18 16:58 ` [RFC PATCH v2 28/38] KVM: arm64: Emulate EL12 register accesses from the virtual EL2 Jintack Lim
2017-07-31 8:44 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 29/38] KVM: arm64: Support a VM with VHE considering EL0 of the VHE host Jintack Lim
2017-07-31 9:01 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 30/38] KVM: arm64: Allow the virtual EL2 to access EL2 states without trap Jintack Lim
2017-07-31 9:37 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 31/38] KVM: arm64: Manage the shadow states when virtual E2H bit enabled Jintack Lim
2017-07-31 9:57 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 32/38] KVM: arm64: Trap and emulate CPTR_EL2 accesses via CPACR_EL1 from the virtual EL2 with VHE Jintack Lim
2017-07-31 12:04 ` Christoffer Dall
2017-07-18 16:58 ` [RFC PATCH v2 33/38] KVM: arm64: Emulate appropriate VM control system registers Jintack Lim
2017-07-31 12:09 ` Christoffer Dall
2017-07-18 16:59 ` [RFC PATCH v2 34/38] KVM: arm64: Respect the virtual HCR_EL2.NV bit setting Jintack Lim
2017-07-18 16:59 ` [RFC PATCH v2 35/38] KVM: arm64: Respect the virtual HCR_EL2.NV bit setting for EL12 register traps Jintack Lim
2017-07-31 12:39 ` Christoffer Dall
2017-07-18 16:59 ` [RFC PATCH v2 36/38] KVM: arm64: Respect virtual HCR_EL2.TVM and TRVM settings Jintack Lim
2017-07-31 12:42 ` Christoffer Dall
2017-07-18 16:59 ` [RFC PATCH v2 37/38] KVM: arm64: Respect the virtual HCR_EL2.NV1 bit setting Jintack Lim
2017-07-19 2:24 ` Jintack Lim
2017-07-31 12:53 ` Christoffer Dall
2017-07-18 16:59 ` [RFC PATCH v2 38/38] KVM: arm64: Respect the virtual CPTR_EL2.TCPAC setting Jintack Lim
2017-07-31 12:59 ` Christoffer Dall
2017-08-01 11:03 ` Jintack Lim
2017-08-01 11:20 ` Christoffer Dall
2017-07-19 2:23 ` [RFC PATCH v2 00/38] Nested Virtualization on KVM/ARM Jintack Lim
2017-07-19 8:49 ` Christoffer Dall
2017-07-19 14:35 ` Jintack Lim
2017-07-28 20:13 ` Bandan Das [this message]
2017-07-28 21:45 ` Jintack Lim
2017-07-31 13:00 ` Christoffer Dall
2017-08-01 10:48 ` Jintack Lim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=jpgshhg7227.fsf@linux.bootlegged.copy \
--to=bsd@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=andy.gross@linaro.org \
--cc=ard.biesheuvel@linaro.org \
--cc=catalin.marinas@arm.com \
--cc=christoffer.dall@linaro.org \
--cc=corbet@lwn.net \
--cc=cov@codeaurora.org \
--cc=daniel.lezcano@linaro.org \
--cc=david.daney@cavium.com \
--cc=james.morse@arm.com \
--cc=jintack.lim@linaro.org \
--cc=kvm@vg \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=marc.zyngier@arm.com \
--cc=mark.rutland@arm.com \
--cc=mchehab@kernel.org \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=shankerd@codeaurora.org \
--cc=stefan@hello-penguin.com \
--cc=suzuki.poulose@arm.com \
--cc=vladimir.murzin@arm.com \
--cc=wcohen@redhat.com \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).