KVM ARM Archive on lore.kernel.org
 help / color / Atom feed
From: Marc Zyngier <marc.zyngier@arm.com>
To: Tomasz Nowicki <tn@semihalf.com>,
	linux-arm-kernel@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org
Cc: Andre Przywara <andre.przywara@arm.com>,
	Dave Martin <Dave.Martin@arm.com>
Subject: Re: [PATCH 43/59] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
Date: Wed, 24 Jul 2019 13:39:14 +0100
Message-ID: <327dfc0e-e7c9-0d18-2615-1db5b05121f4@arm.com> (raw)
In-Reply-To: <0411c636-adbd-98d0-5191-2b073daaf68e@semihalf.com>

On 24/07/2019 11:25, Tomasz Nowicki wrote:
> On 21.06.2019 11:38, Marc Zyngier wrote:
>> From: Jintack Lim <jintack.lim@linaro.org>
>>
>> When supporting nested virtualization a guest hypervisor executing AT
>> instructions must be trapped and emulated by the host hypervisor,
>> because untrapped AT instructions operating on S1E1 will use the wrong
>> translation regieme (the one used to emulate virtual EL2 in EL1 instead
>> of virtual EL1) and AT instructions operating on S12 will not work from
>> EL1.
>>
>> This patch does several things.
>>
>> 1. List and define all AT system instructions to emulate and document
>> the emulation design.
>>
>> 2. Implement AT instruction handling logic in EL2. This will be used to
>> emulate AT instructions executed in the virtual EL2.
>>
>> AT instruction emulation works by loading the proper processor
>> context, which depends on the trapped instruction and the virtual
>> HCR_EL2, to the EL1 virtual memory control registers and executing AT
>> instructions. Note that ctxt->hw_sys_regs is expected to have the
>> proper processor context before calling the handling
>> function(__kvm_at_insn) implemented in this patch.
>>
>> 4. Emulate AT S1E[01] instructions by issuing the same instructions in
>> EL2. We set the physical EL1 registers, NV and NV1 bits as described in
>> the AT instruction emulation overview.
>>
>> 5. Emulate AT A12E[01] instructions in two steps: First, do the stage-1
>> translation by reusing the existing AT emulation functions.  Second, do
>> the stage-2 translation by walking the guest hypervisor's stage-2 page
>> table in software. Record the translation result to PAR_EL1.
>>
>> 6. Emulate AT S1E2 instructions by issuing the corresponding S1E1
>> instructions in EL2. We set the physical EL1 registers and the HCR_EL2
>> register as described in the AT instruction emulation overview.
>>
>> 7. Forward system instruction traps to the virtual EL2 if the corresponding
>> virtual AT bit is set in the virtual HCR_EL2.
>>
>>    [ Much logic above has been reworked by Marc Zyngier ]
>>
>> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
>> ---
>>   arch/arm64/include/asm/kvm_arm.h |   2 +
>>   arch/arm64/include/asm/kvm_asm.h |   2 +
>>   arch/arm64/include/asm/sysreg.h  |  17 +++
>>   arch/arm64/kvm/hyp/Makefile      |   1 +
>>   arch/arm64/kvm/hyp/at.c          | 217 +++++++++++++++++++++++++++++++
>>   arch/arm64/kvm/hyp/switch.c      |  13 +-
>>   arch/arm64/kvm/sys_regs.c        | 202 +++++++++++++++++++++++++++-
>>   7 files changed, 450 insertions(+), 4 deletions(-)
>>   create mode 100644 arch/arm64/kvm/hyp/at.c
>>
> 
> [...]
> 
>> +
>> +void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
>> +{
>> +	struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
>> +	struct mmu_config config;
>> +	struct kvm_s2_mmu *mmu;
>> +
>> +	/*
>> +	 * We can only get here when trapping from vEL2, so we're
>> +	 * translating a guest guest VA.
>> +	 *
>> +	 * FIXME: Obtaining the S2 MMU for a a guest guest is horribly
>> +	 * racy, and we may not find it.
>> +	 */
>> +	spin_lock(&vcpu->kvm->mmu_lock);
>> +
>> +	mmu = lookup_s2_mmu(vcpu->kvm,
>> +			    vcpu_read_sys_reg(vcpu, VTTBR_EL2),
>> +			    vcpu_read_sys_reg(vcpu, HCR_EL2));
>> +
>> +	if (WARN_ON(!mmu))
>> +		goto out;
>> +
>> +	/* We've trapped, so everything is live on the CPU. */
>> +	__mmu_config_save(&config);
>> +
>> +	write_sysreg_el1(ctxt->sys_regs[TTBR0_EL1],	SYS_TTBR0);
>> +	write_sysreg_el1(ctxt->sys_regs[TTBR1_EL1],	SYS_TTBR1);
>> +	write_sysreg_el1(ctxt->sys_regs[TCR_EL1],	SYS_TCR);
>> +	write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1],	SYS_SCTLR);
>> +	write_sysreg(kvm_get_vttbr(mmu),		vttbr_el2);
>> +	/* FIXME: write S2 MMU VTCR_EL2 */
>> +	write_sysreg(config.hcr & ~HCR_TGE,		hcr_el2);
> 
> All below AT S1E* operations may initiate stage-1 PTW. And stage-1 table 
> walk addresses are themselves subject to stage-2 translation.
> 
> So should we enable stage-2 translation here by setting HCR_VM bit?

Hopefully that's already the case, otherwise nothing works. Have you
verified that it is actually clear at this stage?

It's a bit of a moot point as we need a full S1 PTW in SW anyway (so
most of that code will eventually be removed), but at least this should
be fixed if it needs to.

Thanks,

	M. (who really needs to get on top of these review comments)
-- 
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

  reply index

Thread overview: 177+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-21  9:37 [PATCH 00/59] KVM: arm64: ARMv8.3 Nested Virtualization support Marc Zyngier
2019-06-21  9:37 ` [PATCH 01/59] KVM: arm64: Migrate _elx sysreg accessors to msr_s/mrs_s Marc Zyngier
2019-06-24 11:16   ` Dave Martin
2019-06-24 12:59   ` Alexandru Elisei
2019-07-03 12:32     ` Marc Zyngier
2019-06-21  9:37 ` [PATCH 02/59] KVM: arm64: Move __load_guest_stage2 to kvm_mmu.h Marc Zyngier
2019-06-24 11:19   ` Dave Martin
2019-07-03  9:30     ` Marc Zyngier
2019-07-03 16:13       ` Dave Martin
2019-06-21  9:37 ` [PATCH 03/59] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature Marc Zyngier
2019-06-21 13:08   ` Julien Thierry
2019-06-21 13:22     ` Marc Zyngier
2019-06-21 13:44   ` Suzuki K Poulose
2019-06-24 11:24   ` Dave Martin
2019-06-21  9:37 ` [PATCH 04/59] KVM: arm64: nv: Introduce nested virtualization VCPU feature Marc Zyngier
2019-06-21 13:08   ` Julien Thierry
2019-06-24 11:28   ` Dave Martin
2019-07-03 11:53     ` Marc Zyngier
2019-07-03 16:27       ` Dave Martin
2019-06-24 11:43   ` Dave Martin
2019-07-03 11:56     ` Marc Zyngier
2019-07-03 16:24       ` Dave Martin
2019-06-21  9:37 ` [PATCH 05/59] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set Marc Zyngier
2019-06-24 10:19   ` Suzuki K Poulose
2019-06-24 11:38   ` Dave Martin
2019-06-21  9:37 ` [PATCH 06/59] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x Marc Zyngier
2019-06-21 13:24   ` Julien Thierry
2019-06-21 13:50     ` Marc Zyngier
2019-06-24 12:48       ` Dave Martin
2019-07-03  9:21         ` Marc Zyngier
2019-07-04 10:00           ` Dave Martin
2019-06-21  9:37 ` [PATCH 07/59] KVM: arm64: nv: Add EL2 system registers to vcpu context Marc Zyngier
2019-06-24 12:54   ` Dave Martin
2019-07-03 12:20     ` Marc Zyngier
2019-07-03 16:31       ` Dave Martin
2019-06-24 15:47   ` Alexandru Elisei
2019-07-03 13:20     ` Marc Zyngier
2019-07-03 16:01       ` Marc Zyngier
2019-07-01 16:36   ` Suzuki K Poulose
2019-06-21  9:37 ` [PATCH 08/59] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values Marc Zyngier
2019-06-24 12:59   ` Dave Martin
2019-06-21  9:37 ` [PATCH 09/59] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state Marc Zyngier
2019-06-24 13:08   ` Dave Martin
2019-06-21  9:37 ` [PATCH 10/59] KVM: arm64: nv: Support virtual EL2 exceptions Marc Zyngier
2019-07-08 13:56   ` Steven Price
2019-06-21  9:37 ` [PATCH 11/59] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2 Marc Zyngier
2019-06-25 13:13   ` Alexandru Elisei
2019-07-03 14:16     ` Marc Zyngier
2019-07-30 14:08     ` Alexandru Elisei
2019-06-21  9:37 ` [PATCH 12/59] KVM: arm64: nv: Handle trapped ERET from " Marc Zyngier
2019-07-02 12:00   ` Alexandru Elisei
2019-06-21  9:37 ` [PATCH 13/59] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg() Marc Zyngier
2019-06-24 12:42   ` Julien Thierry
2019-06-25 14:02     ` Alexandru Elisei
2019-07-03 12:15     ` Marc Zyngier
2019-07-03 15:21       ` Julien Thierry
2019-06-25 15:18   ` Alexandru Elisei
2019-07-01  9:58     ` Alexandru Elisei
2019-07-03 15:59     ` Marc Zyngier
2019-07-03 16:32       ` Alexandru Elisei
2019-07-04 14:39         ` Marc Zyngier
2019-06-26 15:04   ` Alexandru Elisei
2019-07-04 15:05     ` Marc Zyngier
2019-07-01 12:10   ` Alexandru Elisei
2019-06-21  9:37 ` [PATCH 14/59] KVM: arm64: nv: Handle SPSR_EL2 specially Marc Zyngier
2019-06-21  9:37 ` [PATCH 15/59] KVM: arm64: nv: Refactor vcpu_{read,write}_sys_reg Marc Zyngier
2019-06-24 15:07   ` Julien Thierry
2019-07-03 13:09     ` Marc Zyngier
2019-06-27  9:21   ` Alexandru Elisei
2019-07-04 15:15     ` Marc Zyngier
2019-06-21  9:38 ` [PATCH 16/59] KVM: arm64: nv: Save/Restore vEL2 sysregs Marc Zyngier
2019-06-25  8:48   ` Julien Thierry
2019-07-03 13:42     ` Marc Zyngier
2019-07-01 12:09   ` Alexandru Elisei
2019-08-21 11:57   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 17/59] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor Marc Zyngier
2019-06-21  9:38 ` [PATCH 18/59] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2 Marc Zyngier
2019-07-01 16:12   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 19/59] KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from " Marc Zyngier
2019-06-21  9:38 ` [PATCH 20/59] KVM: arm64: nv: Trap CPACR_EL1 access in " Marc Zyngier
2019-07-01 16:40   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 21/59] KVM: arm64: nv: Set a handler for the system instruction traps Marc Zyngier
2019-06-25 12:55   ` Julien Thierry
2019-07-03 14:15     ` Marc Zyngier
2019-06-21  9:38 ` [PATCH 22/59] KVM: arm64: nv: Handle PSCI call via smc from the guest Marc Zyngier
2019-06-21  9:38 ` [PATCH 23/59] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting Marc Zyngier
2019-06-25 14:19   ` Julien Thierry
2019-07-02 12:54     ` Alexandru Elisei
2019-07-03 14:18     ` Marc Zyngier
2019-06-21  9:38 ` [PATCH 24/59] KVM: arm64: nv: Respect virtual CPTR_EL2.TFP setting Marc Zyngier
2019-06-21  9:38 ` [PATCH 25/59] KVM: arm64: nv: Don't expose SVE to nested guests Marc Zyngier
2019-06-21  9:38 ` [PATCH 26/59] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting Marc Zyngier
2019-06-26  5:31   ` Julien Thierry
2019-07-03 16:31     ` Marc Zyngier
2019-06-21  9:38 ` [PATCH 27/59] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings Marc Zyngier
2019-06-26  6:55   ` Julien Thierry
2019-07-04 14:57     ` Marc Zyngier
2019-06-21  9:38 ` [PATCH 28/59] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting Marc Zyngier
2019-06-26  7:23   ` Julien Thierry
2019-07-02 16:32   ` Alexandru Elisei
2019-07-03  9:10     ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 29/59] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2 Marc Zyngier
2019-07-03  9:16   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 30/59] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization Marc Zyngier
2019-06-21  9:38 ` [PATCH 31/59] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes Marc Zyngier
2019-06-21  9:38 ` [PATCH 32/59] KVM: arm64: nv: Hide RAS from nested guests Marc Zyngier
2019-07-03 13:59   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 33/59] KVM: arm64: nv: Pretend we only support larger-than-host page sizes Marc Zyngier
2019-07-03 14:13   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 34/59] KVM: arm/arm64: nv: Factor out stage 2 page table data from struct kvm Marc Zyngier
2019-07-03 15:52   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 35/59] KVM: arm/arm64: nv: Support multiple nested stage 2 mmu structures Marc Zyngier
2019-06-25 12:19   ` Alexandru Elisei
2019-07-03 13:47     ` Marc Zyngier
2019-06-27 13:15   ` Julien Thierry
2019-07-04 15:51   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 36/59] KVM: arm64: nv: Implement nested Stage-2 page table walk logic Marc Zyngier
2019-06-21  9:38 ` [PATCH 37/59] KVM: arm64: nv: Handle shadow stage 2 page faults Marc Zyngier
2019-07-05 14:28   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 38/59] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables Marc Zyngier
2019-07-01  8:03   ` Julien Thierry
2019-06-21  9:38 ` [PATCH 39/59] KVM: arm64: nv: Move last_vcpu_ran to be per s2 mmu Marc Zyngier
2019-07-01  9:10   ` Julien Thierry
2019-07-05 15:28   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 40/59] KVM: arm64: nv: Don't always start an S2 MMU search from the beginning Marc Zyngier
2019-07-09  9:59   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 41/59] KVM: arm64: nv: Introduce sys_reg_desc.forward_trap Marc Zyngier
2019-06-21  9:38 ` [PATCH 42/59] KVM: arm64: nv: Rework the system instruction emulation framework Marc Zyngier
2019-06-21  9:38 ` [PATCH 43/59] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2 Marc Zyngier
2019-07-01 15:45   ` Julien Thierry
2019-07-09 13:20   ` Alexandru Elisei
2019-07-18 12:13     ` Tomasz Nowicki
2019-07-18 12:36       ` Alexandru Elisei
2019-07-18 12:56         ` Alexandru Elisei
2019-07-18 12:59         ` Tomasz Nowicki
2019-07-24 10:25   ` Tomasz Nowicki
2019-07-24 12:39     ` Marc Zyngier [this message]
2019-07-24 13:56       ` Tomasz Nowicki
2019-06-21  9:38 ` [PATCH 44/59] KVM: arm64: nv: Trap and emulate TLBI " Marc Zyngier
2019-07-02 12:37   ` Julien Thierry
2019-07-10 10:15   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 45/59] KVM: arm64: nv: Handle traps for timer _EL02 and _EL2 sysregs accessors Marc Zyngier
2019-06-21  9:38 ` [PATCH 46/59] KVM: arm64: nv: arch_timer: Support hyp timer emulation Marc Zyngier
2019-07-10 16:23   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 47/59] KVM: arm64: nv: Propagate CNTVOFF_EL2 to the virtual EL1 timer Marc Zyngier
2019-08-08  9:34   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 48/59] KVM: arm64: nv: Load timer before the GIC Marc Zyngier
2019-07-11 13:17   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 49/59] KVM: arm64: nv: vgic-v3: Take cpu_if pointer directly instead of vcpu Marc Zyngier
2019-06-21  9:38 ` [PATCH 50/59] KVM: arm64: nv: Nested GICv3 Support Marc Zyngier
2019-07-16 11:41   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 51/59] KVM: arm64: nv: vgic: Emulate the HW bit in software Marc Zyngier
2019-06-21  9:38 ` [PATCH 52/59] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ Marc Zyngier
2019-07-04  7:38   ` Julien Thierry
2019-07-04  9:01     ` Andre Przywara
2019-07-04  9:04       ` Julien Thierry
2019-06-21  9:38 ` [PATCH 53/59] KVM: arm64: nv: Implement maintenance interrupt forwarding Marc Zyngier
2019-07-04  8:06   ` Julien Thierry
2019-07-16 16:35   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 54/59] KVM: arm64: nv: Add nested GICv3 tracepoints Marc Zyngier
2019-06-21  9:38 ` [PATCH 55/59] arm64: KVM: nv: Add handling of EL2-specific timer registers Marc Zyngier
2019-07-11 12:35   ` Alexandru Elisei
2019-07-17 10:19   ` Alexandru Elisei
2019-06-21  9:38 ` [PATCH 56/59] arm64: KVM: nv: Honor SCTLR_EL2.SPAN on entering vEL2 Marc Zyngier
2019-06-21  9:38 ` [PATCH 57/59] arm64: KVM: nv: Handle SCTLR_EL2 RES0/RES1 bits Marc Zyngier
2019-06-21  9:38 ` [PATCH 58/59] arm64: KVM: nv: Restrict S2 RD/WR permissions to match the guest's Marc Zyngier
2019-06-21  9:38 ` [PATCH 59/59] arm64: KVM: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT Marc Zyngier
2019-06-21  9:57 ` [PATCH 00/59] KVM: arm64: ARMv8.3 Nested Virtualization support Itaru Kitayama
2019-06-21 11:21   ` Marc Zyngier
2019-08-02 10:11 ` Alexandru Elisei
2019-08-02 10:30   ` Andrew Jones
2019-08-09 10:01   ` Alexandru Elisei
2019-08-09 11:44     ` Andrew Jones
2019-08-09 12:00       ` Alexandru Elisei
2019-08-09 13:00         ` Andrew Jones
2019-08-22 11:57     ` Alexandru Elisei
2019-08-22 15:32       ` Alexandru Elisei

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=327dfc0e-e7c9-0d18-2615-1db5b05121f4@arm.com \
    --to=marc.zyngier@arm.com \
    --cc=Dave.Martin@arm.com \
    --cc=andre.przywara@arm.com \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=tn@semihalf.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM ARM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvmarm/0 kvmarm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvmarm kvmarm/ https://lore.kernel.org/kvmarm \
		kvmarm@lists.cs.columbia.edu
	public-inbox-index kvmarm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/edu.columbia.cs.lists.kvmarm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git