* [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
This is a rework of the NV series that I posted 10 months ago[1], as a
lot of the KVM code has changed since, and the series apply anymore
(not that anybody really cares as the the HW is, as usual, made of
unobtainium...).
From the previous version:
- Integration with the new page-table code
- New exception injection code
- No more messing with the nVHE code
- No AArch32!!!!
- Rebased on v5.10-rc4 + kvmarm/next for 5.11
From a functionality perspective, you can expect a L2 guest to work,
but don't even think of L3, as we only partially emulate the
ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
as well as anything that would require a Stage-1 PTW. What we want to
achieve is that with NV disabled, there is no performance overhead and
no regression.
The series is roughly divided in 5 parts: exception handling, memory
virtualization, interrupts and timers for ARMv8.3, followed by the
ARMv8.4 support. There are of course some dependencies, but you'll
hopefully get the gist of it.
For the most courageous of you, I've put out a branch[2]. Of course,
you'll need some userspace. Andre maintains a hacked version of
kvmtool[3] that takes a --nested option, allowing the guest to be
started at EL2. You can run the whole stack in the Foundation
model. Don't be in a hurry ;-).
And to be clear: although Jintack and Christoffer have written tons of
the stuff originaly, I'm the one responsible for breaking it!
[1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
[2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
[3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
Andre Przywara (1):
KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
Christoffer Dall (15):
KVM: arm64: nv: Introduce nested virtualization VCPU feature
KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
KVM: arm64: nv: Handle trapped ERET from virtual EL2
KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
changes
KVM: arm64: nv: Implement nested Stage-2 page table walk logic
KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
KVM: arm64: nv: arch_timer: Support hyp timer emulation
KVM: arm64: nv: vgic: Emulate the HW bit in software
KVM: arm64: nv: Add nested GICv3 tracepoints
KVM: arm64: nv: Sync nested timer state with ARMv8.4
Jintack Lim (19):
arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
KVM: arm64: nv: Handle HCR_EL2.NV system register traps
KVM: arm64: nv: Support virtual EL2 exceptions
KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
KVM: arm64: nv: Handle PSCI call via smc from the guest
KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
KVM: arm64: nv: Set a handler for the system instruction traps
KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
KVM: arm64: nv: Nested GICv3 Support
Marc Zyngier (31):
KVM: arm64: nv: Add EL2 system registers to vcpu context
KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
KVM: arm64: nv: Handle virtual EL2 registers in
vcpu_read/write_sys_reg()
KVM: arm64: nv: Handle SPSR_EL2 specially
KVM: arm64: nv: Handle HCR_EL2.E2H specially
KVM: arm64: nv: Save/Restore vEL2 sysregs
KVM: arm64: nv: Forward debug traps to the nested guest
KVM: arm64: nv: Filter out unsupported features from ID regs
KVM: arm64: nv: Hide RAS from nested guests
KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
KVM: arm64: nv: Handle shadow stage 2 page faults
KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
KVM: arm64: nv: Add handling of EL2-specific timer registers
KVM: arm64: nv: Load timer before the GIC
KVM: arm64: nv: Don't load the GICv4 context on entering a nested
guest
KVM: arm64: nv: Implement maintenance interrupt forwarding
KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
information
KVM: arm64: Allow populating S2 SW bits
KVM: arm64: nv: Tag shadow S2 entries with nested level
KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
KVM: arm64: Map VNCR-capable registers to a separate page
KVM: arm64: nv: Move nested vgic state into the sysreg file
KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
KVM: arm64: nv: Synchronize PSTATE early on exit
KVM: arm64: nv: Allocate VNCR page when required
KVM: arm64: nv: Enable ARMv8.4-NV support
KVM: arm64: nv: Fast-track 'InHost' exception returns
KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
.../admin-guide/kernel-parameters.txt | 4 +
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
arch/arm64/include/asm/cpucaps.h | 2 +
arch/arm64/include/asm/esr.h | 6 +
arch/arm64/include/asm/kvm_arm.h | 28 +-
arch/arm64/include/asm/kvm_asm.h | 4 +
arch/arm64/include/asm/kvm_emulate.h | 145 +-
arch/arm64/include/asm/kvm_host.h | 175 ++-
arch/arm64/include/asm/kvm_hyp.h | 2 +
arch/arm64/include/asm/kvm_mmu.h | 17 +-
arch/arm64/include/asm/kvm_nested.h | 152 ++
arch/arm64/include/asm/kvm_pgtable.h | 10 +
arch/arm64/include/asm/sysreg.h | 104 +-
arch/arm64/include/asm/vncr_mapping.h | 73 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kernel/cpufeature.c | 35 +
arch/arm64/kvm/Makefile | 4 +-
arch/arm64/kvm/arch_timer.c | 189 ++-
arch/arm64/kvm/arm.c | 34 +-
arch/arm64/kvm/at.c | 231 ++++
arch/arm64/kvm/emulate-nested.c | 186 +++
arch/arm64/kvm/guest.c | 6 +
arch/arm64/kvm/handle_exit.c | 81 +-
arch/arm64/kvm/hyp/exception.c | 44 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/pgtable.c | 6 +
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
arch/arm64/kvm/inject_fault.c | 62 +-
arch/arm64/kvm/mmu.c | 183 ++-
arch/arm64/kvm/nested.c | 908 ++++++++++++
arch/arm64/kvm/reset.c | 14 +-
arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
arch/arm64/kvm/sys_regs.h | 6 +
arch/arm64/kvm/trace_arm.h | 65 +-
arch/arm64/kvm/vgic/vgic-init.c | 30 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
arch/arm64/kvm/vgic/vgic.c | 44 +
arch/arm64/kvm/vgic/vgic.h | 10 +
include/kvm/arm_arch_timer.h | 7 +
include/kvm/arm_vgic.h | 16 +
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
50 files changed, 4890 insertions(+), 160 deletions(-)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
create mode 100644 arch/arm64/kvm/at.c
create mode 100644 arch/arm64/kvm/emulate-nested.c
create mode 100644 arch/arm64/kvm/nested.c
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
--
2.29.2
^ permalink raw reply [flat|nested] 249+ messages in thread
* [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
This is a rework of the NV series that I posted 10 months ago[1], as a
lot of the KVM code has changed since, and the series apply anymore
(not that anybody really cares as the the HW is, as usual, made of
unobtainium...).
From the previous version:
- Integration with the new page-table code
- New exception injection code
- No more messing with the nVHE code
- No AArch32!!!!
- Rebased on v5.10-rc4 + kvmarm/next for 5.11
From a functionality perspective, you can expect a L2 guest to work,
but don't even think of L3, as we only partially emulate the
ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
as well as anything that would require a Stage-1 PTW. What we want to
achieve is that with NV disabled, there is no performance overhead and
no regression.
The series is roughly divided in 5 parts: exception handling, memory
virtualization, interrupts and timers for ARMv8.3, followed by the
ARMv8.4 support. There are of course some dependencies, but you'll
hopefully get the gist of it.
For the most courageous of you, I've put out a branch[2]. Of course,
you'll need some userspace. Andre maintains a hacked version of
kvmtool[3] that takes a --nested option, allowing the guest to be
started at EL2. You can run the whole stack in the Foundation
model. Don't be in a hurry ;-).
And to be clear: although Jintack and Christoffer have written tons of
the stuff originaly, I'm the one responsible for breaking it!
[1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
[2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
[3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
Andre Przywara (1):
KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
Christoffer Dall (15):
KVM: arm64: nv: Introduce nested virtualization VCPU feature
KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
KVM: arm64: nv: Handle trapped ERET from virtual EL2
KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
changes
KVM: arm64: nv: Implement nested Stage-2 page table walk logic
KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
KVM: arm64: nv: arch_timer: Support hyp timer emulation
KVM: arm64: nv: vgic: Emulate the HW bit in software
KVM: arm64: nv: Add nested GICv3 tracepoints
KVM: arm64: nv: Sync nested timer state with ARMv8.4
Jintack Lim (19):
arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
KVM: arm64: nv: Handle HCR_EL2.NV system register traps
KVM: arm64: nv: Support virtual EL2 exceptions
KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
KVM: arm64: nv: Handle PSCI call via smc from the guest
KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
KVM: arm64: nv: Set a handler for the system instruction traps
KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
KVM: arm64: nv: Nested GICv3 Support
Marc Zyngier (31):
KVM: arm64: nv: Add EL2 system registers to vcpu context
KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
KVM: arm64: nv: Handle virtual EL2 registers in
vcpu_read/write_sys_reg()
KVM: arm64: nv: Handle SPSR_EL2 specially
KVM: arm64: nv: Handle HCR_EL2.E2H specially
KVM: arm64: nv: Save/Restore vEL2 sysregs
KVM: arm64: nv: Forward debug traps to the nested guest
KVM: arm64: nv: Filter out unsupported features from ID regs
KVM: arm64: nv: Hide RAS from nested guests
KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
KVM: arm64: nv: Handle shadow stage 2 page faults
KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
KVM: arm64: nv: Add handling of EL2-specific timer registers
KVM: arm64: nv: Load timer before the GIC
KVM: arm64: nv: Don't load the GICv4 context on entering a nested
guest
KVM: arm64: nv: Implement maintenance interrupt forwarding
KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
information
KVM: arm64: Allow populating S2 SW bits
KVM: arm64: nv: Tag shadow S2 entries with nested level
KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
KVM: arm64: Map VNCR-capable registers to a separate page
KVM: arm64: nv: Move nested vgic state into the sysreg file
KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
KVM: arm64: nv: Synchronize PSTATE early on exit
KVM: arm64: nv: Allocate VNCR page when required
KVM: arm64: nv: Enable ARMv8.4-NV support
KVM: arm64: nv: Fast-track 'InHost' exception returns
KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
.../admin-guide/kernel-parameters.txt | 4 +
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
arch/arm64/include/asm/cpucaps.h | 2 +
arch/arm64/include/asm/esr.h | 6 +
arch/arm64/include/asm/kvm_arm.h | 28 +-
arch/arm64/include/asm/kvm_asm.h | 4 +
arch/arm64/include/asm/kvm_emulate.h | 145 +-
arch/arm64/include/asm/kvm_host.h | 175 ++-
arch/arm64/include/asm/kvm_hyp.h | 2 +
arch/arm64/include/asm/kvm_mmu.h | 17 +-
arch/arm64/include/asm/kvm_nested.h | 152 ++
arch/arm64/include/asm/kvm_pgtable.h | 10 +
arch/arm64/include/asm/sysreg.h | 104 +-
arch/arm64/include/asm/vncr_mapping.h | 73 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kernel/cpufeature.c | 35 +
arch/arm64/kvm/Makefile | 4 +-
arch/arm64/kvm/arch_timer.c | 189 ++-
arch/arm64/kvm/arm.c | 34 +-
arch/arm64/kvm/at.c | 231 ++++
arch/arm64/kvm/emulate-nested.c | 186 +++
arch/arm64/kvm/guest.c | 6 +
arch/arm64/kvm/handle_exit.c | 81 +-
arch/arm64/kvm/hyp/exception.c | 44 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/pgtable.c | 6 +
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
arch/arm64/kvm/inject_fault.c | 62 +-
arch/arm64/kvm/mmu.c | 183 ++-
arch/arm64/kvm/nested.c | 908 ++++++++++++
arch/arm64/kvm/reset.c | 14 +-
arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
arch/arm64/kvm/sys_regs.h | 6 +
arch/arm64/kvm/trace_arm.h | 65 +-
arch/arm64/kvm/vgic/vgic-init.c | 30 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
arch/arm64/kvm/vgic/vgic.c | 44 +
arch/arm64/kvm/vgic/vgic.h | 10 +
include/kvm/arm_arch_timer.h | 7 +
include/kvm/arm_vgic.h | 16 +
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
50 files changed, 4890 insertions(+), 160 deletions(-)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
create mode 100644 arch/arm64/kvm/at.c
create mode 100644 arch/arm64/kvm/emulate-nested.c
create mode 100644 arch/arm64/kvm/nested.c
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
This is a rework of the NV series that I posted 10 months ago[1], as a
lot of the KVM code has changed since, and the series apply anymore
(not that anybody really cares as the the HW is, as usual, made of
unobtainium...).
From the previous version:
- Integration with the new page-table code
- New exception injection code
- No more messing with the nVHE code
- No AArch32!!!!
- Rebased on v5.10-rc4 + kvmarm/next for 5.11
From a functionality perspective, you can expect a L2 guest to work,
but don't even think of L3, as we only partially emulate the
ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
as well as anything that would require a Stage-1 PTW. What we want to
achieve is that with NV disabled, there is no performance overhead and
no regression.
The series is roughly divided in 5 parts: exception handling, memory
virtualization, interrupts and timers for ARMv8.3, followed by the
ARMv8.4 support. There are of course some dependencies, but you'll
hopefully get the gist of it.
For the most courageous of you, I've put out a branch[2]. Of course,
you'll need some userspace. Andre maintains a hacked version of
kvmtool[3] that takes a --nested option, allowing the guest to be
started at EL2. You can run the whole stack in the Foundation
model. Don't be in a hurry ;-).
And to be clear: although Jintack and Christoffer have written tons of
the stuff originaly, I'm the one responsible for breaking it!
[1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
[2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
[3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
Andre Przywara (1):
KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
Christoffer Dall (15):
KVM: arm64: nv: Introduce nested virtualization VCPU feature
KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
KVM: arm64: nv: Handle trapped ERET from virtual EL2
KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
changes
KVM: arm64: nv: Implement nested Stage-2 page table walk logic
KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
KVM: arm64: nv: arch_timer: Support hyp timer emulation
KVM: arm64: nv: vgic: Emulate the HW bit in software
KVM: arm64: nv: Add nested GICv3 tracepoints
KVM: arm64: nv: Sync nested timer state with ARMv8.4
Jintack Lim (19):
arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
KVM: arm64: nv: Handle HCR_EL2.NV system register traps
KVM: arm64: nv: Support virtual EL2 exceptions
KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
KVM: arm64: nv: Handle PSCI call via smc from the guest
KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
KVM: arm64: nv: Set a handler for the system instruction traps
KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
KVM: arm64: nv: Nested GICv3 Support
Marc Zyngier (31):
KVM: arm64: nv: Add EL2 system registers to vcpu context
KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
KVM: arm64: nv: Handle virtual EL2 registers in
vcpu_read/write_sys_reg()
KVM: arm64: nv: Handle SPSR_EL2 specially
KVM: arm64: nv: Handle HCR_EL2.E2H specially
KVM: arm64: nv: Save/Restore vEL2 sysregs
KVM: arm64: nv: Forward debug traps to the nested guest
KVM: arm64: nv: Filter out unsupported features from ID regs
KVM: arm64: nv: Hide RAS from nested guests
KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
KVM: arm64: nv: Handle shadow stage 2 page faults
KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
KVM: arm64: nv: Add handling of EL2-specific timer registers
KVM: arm64: nv: Load timer before the GIC
KVM: arm64: nv: Don't load the GICv4 context on entering a nested
guest
KVM: arm64: nv: Implement maintenance interrupt forwarding
KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
information
KVM: arm64: Allow populating S2 SW bits
KVM: arm64: nv: Tag shadow S2 entries with nested level
KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
KVM: arm64: Map VNCR-capable registers to a separate page
KVM: arm64: nv: Move nested vgic state into the sysreg file
KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
KVM: arm64: nv: Synchronize PSTATE early on exit
KVM: arm64: nv: Allocate VNCR page when required
KVM: arm64: nv: Enable ARMv8.4-NV support
KVM: arm64: nv: Fast-track 'InHost' exception returns
KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
.../admin-guide/kernel-parameters.txt | 4 +
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
arch/arm64/include/asm/cpucaps.h | 2 +
arch/arm64/include/asm/esr.h | 6 +
arch/arm64/include/asm/kvm_arm.h | 28 +-
arch/arm64/include/asm/kvm_asm.h | 4 +
arch/arm64/include/asm/kvm_emulate.h | 145 +-
arch/arm64/include/asm/kvm_host.h | 175 ++-
arch/arm64/include/asm/kvm_hyp.h | 2 +
arch/arm64/include/asm/kvm_mmu.h | 17 +-
arch/arm64/include/asm/kvm_nested.h | 152 ++
arch/arm64/include/asm/kvm_pgtable.h | 10 +
arch/arm64/include/asm/sysreg.h | 104 +-
arch/arm64/include/asm/vncr_mapping.h | 73 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kernel/cpufeature.c | 35 +
arch/arm64/kvm/Makefile | 4 +-
arch/arm64/kvm/arch_timer.c | 189 ++-
arch/arm64/kvm/arm.c | 34 +-
arch/arm64/kvm/at.c | 231 ++++
arch/arm64/kvm/emulate-nested.c | 186 +++
arch/arm64/kvm/guest.c | 6 +
arch/arm64/kvm/handle_exit.c | 81 +-
arch/arm64/kvm/hyp/exception.c | 44 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/pgtable.c | 6 +
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
arch/arm64/kvm/inject_fault.c | 62 +-
arch/arm64/kvm/mmu.c | 183 ++-
arch/arm64/kvm/nested.c | 908 ++++++++++++
arch/arm64/kvm/reset.c | 14 +-
arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
arch/arm64/kvm/sys_regs.h | 6 +
arch/arm64/kvm/trace_arm.h | 65 +-
arch/arm64/kvm/vgic/vgic-init.c | 30 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
arch/arm64/kvm/vgic/vgic.c | 44 +
arch/arm64/kvm/vgic/vgic.h | 10 +
include/kvm/arm_arch_timer.h | 7 +
include/kvm/arm_vgic.h | 16 +
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
50 files changed, 4890 insertions(+), 160 deletions(-)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
create mode 100644 arch/arm64/kvm/at.c
create mode 100644 arch/arm64/kvm/emulate-nested.c
create mode 100644 arch/arm64/kvm/nested.c
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* [PATCH v3 01/66] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:58 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Add a new ARM64_HAS_NESTED_VIRT feature to indicate that the
CPU has the ARMv8.3 nested virtualization capability.
This will be used to support nested virtualization in KVM.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 4 +++
arch/arm64/include/asm/cpucaps.h | 1 +
arch/arm64/kernel/cpufeature.c | 25 +++++++++++++++++++
3 files changed, 30 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ee9f13776388..727e49640124 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2285,6 +2285,10 @@
[KVM,ARM] Allow use of GICv4 for direct injection of
LPIs.
+ kvm-arm.nested=
+ [KVM,ARM] Allow nested virtualization in KVM/ARM.
+ Default is 0 (disabled)
+
kvm_cma_resv_ratio=n [PPC]
Reserves given percentage from system memory area for
contiguous memory allocation for KVM hash pagetable
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index b77d997b173b..6c5b84692a01 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -16,6 +16,7 @@
#define ARM64_WORKAROUND_CAVIUM_23154 6
#define ARM64_WORKAROUND_834220 7
#define ARM64_HAS_NO_HW_PREFETCH 8
+#define ARM64_HAS_NESTED_VIRT 9
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d96f4554282d..eaeea8a88292 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1577,6 +1577,21 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN))
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
}
+
+static bool nested_param;
+static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,
+ int scope)
+{
+ return has_cpuid_feature(cap, scope) &&
+ nested_param;
+}
+
+static int __init kvmarm_nested_cfg(char *buf)
+{
+ return strtobool(buf, &nested_param);
+}
+
+early_param("kvm-arm.nested", kvmarm_nested_cfg);
#endif
static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused)
@@ -1790,6 +1805,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
+ {
+ .desc = "Nested Virtualization Support",
+ .capability = ARM64_HAS_NESTED_VIRT,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_nested_virt_support,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_NV_SHIFT,
+ .min_field_value = 1,
+ },
#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 01/66] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Add a new ARM64_HAS_NESTED_VIRT feature to indicate that the
CPU has the ARMv8.3 nested virtualization capability.
This will be used to support nested virtualization in KVM.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 4 +++
arch/arm64/include/asm/cpucaps.h | 1 +
arch/arm64/kernel/cpufeature.c | 25 +++++++++++++++++++
3 files changed, 30 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ee9f13776388..727e49640124 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2285,6 +2285,10 @@
[KVM,ARM] Allow use of GICv4 for direct injection of
LPIs.
+ kvm-arm.nested=
+ [KVM,ARM] Allow nested virtualization in KVM/ARM.
+ Default is 0 (disabled)
+
kvm_cma_resv_ratio=n [PPC]
Reserves given percentage from system memory area for
contiguous memory allocation for KVM hash pagetable
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index b77d997b173b..6c5b84692a01 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -16,6 +16,7 @@
#define ARM64_WORKAROUND_CAVIUM_23154 6
#define ARM64_WORKAROUND_834220 7
#define ARM64_HAS_NO_HW_PREFETCH 8
+#define ARM64_HAS_NESTED_VIRT 9
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d96f4554282d..eaeea8a88292 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1577,6 +1577,21 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN))
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
}
+
+static bool nested_param;
+static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,
+ int scope)
+{
+ return has_cpuid_feature(cap, scope) &&
+ nested_param;
+}
+
+static int __init kvmarm_nested_cfg(char *buf)
+{
+ return strtobool(buf, &nested_param);
+}
+
+early_param("kvm-arm.nested", kvmarm_nested_cfg);
#endif
static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused)
@@ -1790,6 +1805,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
+ {
+ .desc = "Nested Virtualization Support",
+ .capability = ARM64_HAS_NESTED_VIRT,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_nested_virt_support,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_NV_SHIFT,
+ .min_field_value = 1,
+ },
#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 01/66] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
Add a new ARM64_HAS_NESTED_VIRT feature to indicate that the
CPU has the ARMv8.3 nested virtualization capability.
This will be used to support nested virtualization in KVM.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 4 +++
arch/arm64/include/asm/cpucaps.h | 1 +
arch/arm64/kernel/cpufeature.c | 25 +++++++++++++++++++
3 files changed, 30 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ee9f13776388..727e49640124 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2285,6 +2285,10 @@
[KVM,ARM] Allow use of GICv4 for direct injection of
LPIs.
+ kvm-arm.nested=
+ [KVM,ARM] Allow nested virtualization in KVM/ARM.
+ Default is 0 (disabled)
+
kvm_cma_resv_ratio=n [PPC]
Reserves given percentage from system memory area for
contiguous memory allocation for KVM hash pagetable
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index b77d997b173b..6c5b84692a01 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -16,6 +16,7 @@
#define ARM64_WORKAROUND_CAVIUM_23154 6
#define ARM64_WORKAROUND_834220 7
#define ARM64_HAS_NO_HW_PREFETCH 8
+#define ARM64_HAS_NESTED_VIRT 9
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d96f4554282d..eaeea8a88292 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1577,6 +1577,21 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN))
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
}
+
+static bool nested_param;
+static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,
+ int scope)
+{
+ return has_cpuid_feature(cap, scope) &&
+ nested_param;
+}
+
+static int __init kvmarm_nested_cfg(char *buf)
+{
+ return strtobool(buf, &nested_param);
+}
+
+early_param("kvm-arm.nested", kvmarm_nested_cfg);
#endif
static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused)
@@ -1790,6 +1805,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
+ {
+ .desc = "Nested Virtualization Support",
+ .capability = ARM64_HAS_NESTED_VIRT,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_nested_virt_support,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_NV_SHIFT,
+ .min_field_value = 1,
+ },
#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 02/66] KVM: arm64: nv: Introduce nested virtualization VCPU feature
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:58 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
Introduce the feature bit and a primitive that checks if the feature is
set behind a static key check based on the cpus_have_const_cap check.
Checking nested_virt_in_use() on systems without nested virt enabled
should have neglgible overhead.
We don't yet allow userspace to actually set this feature.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 14 ++++++++++++++
arch/arm64/include/uapi/asm/kvm.h | 1 +
2 files changed, 15 insertions(+)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
new file mode 100644
index 000000000000..1028ac65a897
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARM64_KVM_NESTED_H
+#define __ARM64_KVM_NESTED_H
+
+#include <linux/kvm_host.h>
+
+static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
+{
+ return (!__is_defined(__KVM_NVHE_HYPERVISOR__) &&
+ cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+ test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
+}
+
+#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 24223adae150..fe3cb67f0d26 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -106,6 +106,7 @@ struct kvm_regs {
#define KVM_ARM_VCPU_SVE 4 /* enable SVE for this CPU */
#define KVM_ARM_VCPU_PTRAUTH_ADDRESS 5 /* VCPU uses address authentication */
#define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */
+#define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */
struct kvm_vcpu_init {
__u32 target;
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 02/66] KVM: arm64: nv: Introduce nested virtualization VCPU feature
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
Introduce the feature bit and a primitive that checks if the feature is
set behind a static key check based on the cpus_have_const_cap check.
Checking nested_virt_in_use() on systems without nested virt enabled
should have neglgible overhead.
We don't yet allow userspace to actually set this feature.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 14 ++++++++++++++
arch/arm64/include/uapi/asm/kvm.h | 1 +
2 files changed, 15 insertions(+)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
new file mode 100644
index 000000000000..1028ac65a897
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARM64_KVM_NESTED_H
+#define __ARM64_KVM_NESTED_H
+
+#include <linux/kvm_host.h>
+
+static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
+{
+ return (!__is_defined(__KVM_NVHE_HYPERVISOR__) &&
+ cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+ test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
+}
+
+#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 24223adae150..fe3cb67f0d26 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -106,6 +106,7 @@ struct kvm_regs {
#define KVM_ARM_VCPU_SVE 4 /* enable SVE for this CPU */
#define KVM_ARM_VCPU_PTRAUTH_ADDRESS 5 /* VCPU uses address authentication */
#define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */
+#define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */
struct kvm_vcpu_init {
__u32 target;
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 02/66] KVM: arm64: nv: Introduce nested virtualization VCPU feature
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
Introduce the feature bit and a primitive that checks if the feature is
set behind a static key check based on the cpus_have_const_cap check.
Checking nested_virt_in_use() on systems without nested virt enabled
should have neglgible overhead.
We don't yet allow userspace to actually set this feature.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 14 ++++++++++++++
arch/arm64/include/uapi/asm/kvm.h | 1 +
2 files changed, 15 insertions(+)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
new file mode 100644
index 000000000000..1028ac65a897
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARM64_KVM_NESTED_H
+#define __ARM64_KVM_NESTED_H
+
+#include <linux/kvm_host.h>
+
+static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
+{
+ return (!__is_defined(__KVM_NVHE_HYPERVISOR__) &&
+ cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+ test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
+}
+
+#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 24223adae150..fe3cb67f0d26 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -106,6 +106,7 @@ struct kvm_regs {
#define KVM_ARM_VCPU_SVE 4 /* enable SVE for this CPU */
#define KVM_ARM_VCPU_PTRAUTH_ADDRESS 5 /* VCPU uses address authentication */
#define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */
+#define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */
struct kvm_vcpu_init {
__u32 target;
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 03/66] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:58 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
Reset the VCPU with PSTATE.M = EL2h when the nested virtualization
feature is enabled on the VCPU.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: rework register reset not to use empty data structures]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/reset.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 47f3f035f3ea..2d2c780e6c69 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -27,6 +27,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/virt.h>
/* Maximum phys_shift supported for any VM on this host */
@@ -38,6 +39,9 @@ static u32 kvm_ipa_limit;
#define VCPU_RESET_PSTATE_EL1 (PSR_MODE_EL1h | PSR_A_BIT | PSR_I_BIT | \
PSR_F_BIT | PSR_D_BIT)
+#define VCPU_RESET_PSTATE_EL2 (PSR_MODE_EL2h | PSR_A_BIT | PSR_I_BIT | \
+ PSR_F_BIT | PSR_D_BIT)
+
#define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \
PSR_AA32_I_BIT | PSR_AA32_F_BIT)
@@ -224,11 +228,18 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
switch (vcpu->arch.target) {
default:
if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features)) {
- if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1)) {
+ /*
+ * The CPU must support 32bit EL1, and 32bit
+ * NV is just not a thing...
+ */
+ if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1) ||
+ nested_virt_in_use(vcpu)) {
ret = -EINVAL;
goto out;
}
pstate = VCPU_RESET_PSTATE_SVC;
+ } else if (nested_virt_in_use(vcpu)) {
+ pstate = VCPU_RESET_PSTATE_EL2;
} else {
pstate = VCPU_RESET_PSTATE_EL1;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 03/66] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
Reset the VCPU with PSTATE.M = EL2h when the nested virtualization
feature is enabled on the VCPU.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: rework register reset not to use empty data structures]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/reset.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 47f3f035f3ea..2d2c780e6c69 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -27,6 +27,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/virt.h>
/* Maximum phys_shift supported for any VM on this host */
@@ -38,6 +39,9 @@ static u32 kvm_ipa_limit;
#define VCPU_RESET_PSTATE_EL1 (PSR_MODE_EL1h | PSR_A_BIT | PSR_I_BIT | \
PSR_F_BIT | PSR_D_BIT)
+#define VCPU_RESET_PSTATE_EL2 (PSR_MODE_EL2h | PSR_A_BIT | PSR_I_BIT | \
+ PSR_F_BIT | PSR_D_BIT)
+
#define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \
PSR_AA32_I_BIT | PSR_AA32_F_BIT)
@@ -224,11 +228,18 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
switch (vcpu->arch.target) {
default:
if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features)) {
- if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1)) {
+ /*
+ * The CPU must support 32bit EL1, and 32bit
+ * NV is just not a thing...
+ */
+ if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1) ||
+ nested_virt_in_use(vcpu)) {
ret = -EINVAL;
goto out;
}
pstate = VCPU_RESET_PSTATE_SVC;
+ } else if (nested_virt_in_use(vcpu)) {
+ pstate = VCPU_RESET_PSTATE_EL2;
} else {
pstate = VCPU_RESET_PSTATE_EL1;
}
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 03/66] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
@ 2020-12-10 15:58 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:58 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
Reset the VCPU with PSTATE.M = EL2h when the nested virtualization
feature is enabled on the VCPU.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: rework register reset not to use empty data structures]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/reset.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 47f3f035f3ea..2d2c780e6c69 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -27,6 +27,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/virt.h>
/* Maximum phys_shift supported for any VM on this host */
@@ -38,6 +39,9 @@ static u32 kvm_ipa_limit;
#define VCPU_RESET_PSTATE_EL1 (PSR_MODE_EL1h | PSR_A_BIT | PSR_I_BIT | \
PSR_F_BIT | PSR_D_BIT)
+#define VCPU_RESET_PSTATE_EL2 (PSR_MODE_EL2h | PSR_A_BIT | PSR_I_BIT | \
+ PSR_F_BIT | PSR_D_BIT)
+
#define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \
PSR_AA32_I_BIT | PSR_AA32_F_BIT)
@@ -224,11 +228,18 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
switch (vcpu->arch.target) {
default:
if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features)) {
- if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1)) {
+ /*
+ * The CPU must support 32bit EL1, and 32bit
+ * NV is just not a thing...
+ */
+ if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1) ||
+ nested_virt_in_use(vcpu)) {
ret = -EINVAL;
goto out;
}
pstate = VCPU_RESET_PSTATE_SVC;
+ } else if (nested_virt_in_use(vcpu)) {
+ pstate = VCPU_RESET_PSTATE_EL2;
} else {
pstate = VCPU_RESET_PSTATE_EL1;
}
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 04/66] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Christoffer Dall
From: Christoffer Dall <christoffer.dall@linaro.org>
We were not allowing userspace to set a more privileged mode for the VCPU
than EL1, but we should allow this when nested virtualization is enabled
for the VCPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/guest.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 9bbd30e62799..171936da30b1 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -24,6 +24,7 @@
#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
#include <asm/sigcontext.h>
#include "trace.h"
@@ -242,6 +243,11 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
if (vcpu_el1_is_32bit(vcpu))
return -EINVAL;
break;
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ if (vcpu_el1_is_32bit(vcpu) || !nested_virt_in_use(vcpu))
+ return -EINVAL;
+ break;
default:
err = -EINVAL;
goto out;
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 04/66] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Andre Przywara, Christoffer Dall
From: Christoffer Dall <christoffer.dall@linaro.org>
We were not allowing userspace to set a more privileged mode for the VCPU
than EL1, but we should allow this when nested virtualization is enabled
for the VCPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/guest.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 9bbd30e62799..171936da30b1 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -24,6 +24,7 @@
#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
#include <asm/sigcontext.h>
#include "trace.h"
@@ -242,6 +243,11 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
if (vcpu_el1_is_32bit(vcpu))
return -EINVAL;
break;
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ if (vcpu_el1_is_32bit(vcpu) || !nested_virt_in_use(vcpu))
+ return -EINVAL;
+ break;
default:
err = -EINVAL;
goto out;
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 04/66] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Christoffer Dall, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Christoffer Dall <christoffer.dall@linaro.org>
We were not allowing userspace to set a more privileged mode for the VCPU
than EL1, but we should allow this when nested virtualization is enabled
for the VCPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/guest.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 9bbd30e62799..171936da30b1 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -24,6 +24,7 @@
#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
#include <asm/sigcontext.h>
#include "trace.h"
@@ -242,6 +243,11 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
if (vcpu_el1_is_32bit(vcpu))
return -EINVAL;
break;
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ if (vcpu_el1_is_32bit(vcpu) || !nested_virt_in_use(vcpu))
+ return -EINVAL;
+ break;
default:
err = -EINVAL;
goto out;
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Add the minimal set of EL2 system registers to the vcpu context.
Nothing uses them just yet.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 34 ++++++++++++++++++++++++++++++-
1 file changed, 33 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 11beda85ee7e..d731cf7a56cb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -206,12 +206,44 @@ enum vcpu_sysreg {
CNTP_CVAL_EL0,
CNTP_CTL_EL0,
- /* 32bit specific registers. Keep them at the end of the range */
+ /* 32bit specific registers. */
DACR32_EL2, /* Domain Access Control Register */
IFSR32_EL2, /* Instruction Fault Status Register */
FPEXC32_EL2, /* Floating-Point Exception Control Register */
DBGVCR32_EL2, /* Debug Vector Catch Register */
+ /* EL2 registers */
+ VPIDR_EL2, /* Virtualization Processor ID Register */
+ VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
+ SCTLR_EL2, /* System Control Register (EL2) */
+ ACTLR_EL2, /* Auxiliary Control Register (EL2) */
+ HCR_EL2, /* Hypervisor Configuration Register */
+ MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
+ CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
+ HSTR_EL2, /* Hypervisor System Trap Register */
+ HACR_EL2, /* Hypervisor Auxiliary Control Register */
+ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
+ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
+ TCR_EL2, /* Translation Control Register (EL2) */
+ VTTBR_EL2, /* Virtualization Translation Table Base Register */
+ VTCR_EL2, /* Virtualization Translation Control Register */
+ SPSR_EL2, /* EL2 saved program status register */
+ ELR_EL2, /* EL2 exception link register */
+ AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
+ AFSR1_EL2, /* Auxiliary Fault Status Register 1 (EL2) */
+ ESR_EL2, /* Exception Syndrome Register (EL2) */
+ FAR_EL2, /* Hypervisor IPA Fault Address Register */
+ HPFAR_EL2, /* Hypervisor IPA Fault Address Register */
+ MAIR_EL2, /* Memory Attribute Indirection Register (EL2) */
+ AMAIR_EL2, /* Auxiliary Memory Attribute Indirection Register (EL2) */
+ VBAR_EL2, /* Vector Base Address Register (EL2) */
+ RVBAR_EL2, /* Reset Vector Base Address Register */
+ RMR_EL2, /* Reset Management Register */
+ CONTEXTIDR_EL2, /* Context ID Register (EL2) */
+ TPIDR_EL2, /* EL2 Software Thread ID Register */
+ CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
+ SP_EL2, /* EL2 Stack Pointer */
+
NR_SYS_REGS /* Nothing after this line! */
};
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Add the minimal set of EL2 system registers to the vcpu context.
Nothing uses them just yet.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 34 ++++++++++++++++++++++++++++++-
1 file changed, 33 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 11beda85ee7e..d731cf7a56cb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -206,12 +206,44 @@ enum vcpu_sysreg {
CNTP_CVAL_EL0,
CNTP_CTL_EL0,
- /* 32bit specific registers. Keep them at the end of the range */
+ /* 32bit specific registers. */
DACR32_EL2, /* Domain Access Control Register */
IFSR32_EL2, /* Instruction Fault Status Register */
FPEXC32_EL2, /* Floating-Point Exception Control Register */
DBGVCR32_EL2, /* Debug Vector Catch Register */
+ /* EL2 registers */
+ VPIDR_EL2, /* Virtualization Processor ID Register */
+ VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
+ SCTLR_EL2, /* System Control Register (EL2) */
+ ACTLR_EL2, /* Auxiliary Control Register (EL2) */
+ HCR_EL2, /* Hypervisor Configuration Register */
+ MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
+ CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
+ HSTR_EL2, /* Hypervisor System Trap Register */
+ HACR_EL2, /* Hypervisor Auxiliary Control Register */
+ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
+ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
+ TCR_EL2, /* Translation Control Register (EL2) */
+ VTTBR_EL2, /* Virtualization Translation Table Base Register */
+ VTCR_EL2, /* Virtualization Translation Control Register */
+ SPSR_EL2, /* EL2 saved program status register */
+ ELR_EL2, /* EL2 exception link register */
+ AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
+ AFSR1_EL2, /* Auxiliary Fault Status Register 1 (EL2) */
+ ESR_EL2, /* Exception Syndrome Register (EL2) */
+ FAR_EL2, /* Hypervisor IPA Fault Address Register */
+ HPFAR_EL2, /* Hypervisor IPA Fault Address Register */
+ MAIR_EL2, /* Memory Attribute Indirection Register (EL2) */
+ AMAIR_EL2, /* Auxiliary Memory Attribute Indirection Register (EL2) */
+ VBAR_EL2, /* Vector Base Address Register (EL2) */
+ RVBAR_EL2, /* Reset Vector Base Address Register */
+ RMR_EL2, /* Reset Management Register */
+ CONTEXTIDR_EL2, /* Context ID Register (EL2) */
+ TPIDR_EL2, /* EL2 Software Thread ID Register */
+ CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
+ SP_EL2, /* EL2 Stack Pointer */
+
NR_SYS_REGS /* Nothing after this line! */
};
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Add the minimal set of EL2 system registers to the vcpu context.
Nothing uses them just yet.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 34 ++++++++++++++++++++++++++++++-
1 file changed, 33 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 11beda85ee7e..d731cf7a56cb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -206,12 +206,44 @@ enum vcpu_sysreg {
CNTP_CVAL_EL0,
CNTP_CTL_EL0,
- /* 32bit specific registers. Keep them at the end of the range */
+ /* 32bit specific registers. */
DACR32_EL2, /* Domain Access Control Register */
IFSR32_EL2, /* Instruction Fault Status Register */
FPEXC32_EL2, /* Floating-Point Exception Control Register */
DBGVCR32_EL2, /* Debug Vector Catch Register */
+ /* EL2 registers */
+ VPIDR_EL2, /* Virtualization Processor ID Register */
+ VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
+ SCTLR_EL2, /* System Control Register (EL2) */
+ ACTLR_EL2, /* Auxiliary Control Register (EL2) */
+ HCR_EL2, /* Hypervisor Configuration Register */
+ MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
+ CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
+ HSTR_EL2, /* Hypervisor System Trap Register */
+ HACR_EL2, /* Hypervisor Auxiliary Control Register */
+ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
+ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
+ TCR_EL2, /* Translation Control Register (EL2) */
+ VTTBR_EL2, /* Virtualization Translation Table Base Register */
+ VTCR_EL2, /* Virtualization Translation Control Register */
+ SPSR_EL2, /* EL2 saved program status register */
+ ELR_EL2, /* EL2 exception link register */
+ AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
+ AFSR1_EL2, /* Auxiliary Fault Status Register 1 (EL2) */
+ ESR_EL2, /* Exception Syndrome Register (EL2) */
+ FAR_EL2, /* Hypervisor IPA Fault Address Register */
+ HPFAR_EL2, /* Hypervisor IPA Fault Address Register */
+ MAIR_EL2, /* Memory Attribute Indirection Register (EL2) */
+ AMAIR_EL2, /* Auxiliary Memory Attribute Indirection Register (EL2) */
+ VBAR_EL2, /* Vector Base Address Register (EL2) */
+ RVBAR_EL2, /* Reset Vector Base Address Register */
+ RMR_EL2, /* Reset Management Register */
+ CONTEXTIDR_EL2, /* Context ID Register (EL2) */
+ TPIDR_EL2, /* EL2 Software Thread ID Register */
+ CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
+ SP_EL2, /* EL2 Stack Pointer */
+
NR_SYS_REGS /* Nothing after this line! */
};
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 06/66] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
When running a nested hypervisor we commonly have to figure out if
the VCPU mode is running in the context of a guest hypervisor or guest
guest, or just a normal guest.
Add convenient primitives for this.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 55 ++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index c8f550a53516..e3e22d364dc9 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -173,6 +173,61 @@ static __always_inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
vcpu_gp_regs(vcpu)->regs[reg_num] = val;
}
+static inline bool vcpu_mode_el2_ctxt(const struct kvm_cpu_context *ctxt)
+{
+ unsigned long cpsr = ctxt->regs.pstate;
+
+ switch (cpsr & (PSR_MODE32_BIT | PSR_MODE_MASK)) {
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static inline bool vcpu_mode_el2(const struct kvm_vcpu *vcpu)
+{
+ return vcpu_mode_el2_ctxt(&vcpu->arch.ctxt);
+}
+
+static inline bool __vcpu_el2_e2h_is_set(const struct kvm_cpu_context *ctxt)
+{
+ return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_E2H;
+}
+
+static inline bool vcpu_el2_e2h_is_set(const struct kvm_vcpu *vcpu)
+{
+ return __vcpu_el2_e2h_is_set(&vcpu->arch.ctxt);
+}
+
+static inline bool __vcpu_el2_tge_is_set(const struct kvm_cpu_context *ctxt)
+{
+ return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_TGE;
+}
+
+static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)
+{
+ return __vcpu_el2_tge_is_set(&vcpu->arch.ctxt);
+}
+
+static inline bool __is_hyp_ctxt(const struct kvm_cpu_context *ctxt)
+{
+ /*
+ * We are in a hypervisor context if the vcpu mode is EL2 or
+ * E2H and TGE bits are set. The latter means we are in the user space
+ * of the VHE kernel. ARMv8.1 ARM describes this as 'InHost'
+ */
+ return vcpu_mode_el2_ctxt(ctxt) ||
+ (__vcpu_el2_e2h_is_set(ctxt) && __vcpu_el2_tge_is_set(ctxt)) ||
+ WARN_ON(__vcpu_el2_tge_is_set(ctxt));
+}
+
+static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
+{
+ return __is_hyp_ctxt(&vcpu->arch.ctxt);
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 06/66] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
When running a nested hypervisor we commonly have to figure out if
the VCPU mode is running in the context of a guest hypervisor or guest
guest, or just a normal guest.
Add convenient primitives for this.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 55 ++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index c8f550a53516..e3e22d364dc9 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -173,6 +173,61 @@ static __always_inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
vcpu_gp_regs(vcpu)->regs[reg_num] = val;
}
+static inline bool vcpu_mode_el2_ctxt(const struct kvm_cpu_context *ctxt)
+{
+ unsigned long cpsr = ctxt->regs.pstate;
+
+ switch (cpsr & (PSR_MODE32_BIT | PSR_MODE_MASK)) {
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static inline bool vcpu_mode_el2(const struct kvm_vcpu *vcpu)
+{
+ return vcpu_mode_el2_ctxt(&vcpu->arch.ctxt);
+}
+
+static inline bool __vcpu_el2_e2h_is_set(const struct kvm_cpu_context *ctxt)
+{
+ return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_E2H;
+}
+
+static inline bool vcpu_el2_e2h_is_set(const struct kvm_vcpu *vcpu)
+{
+ return __vcpu_el2_e2h_is_set(&vcpu->arch.ctxt);
+}
+
+static inline bool __vcpu_el2_tge_is_set(const struct kvm_cpu_context *ctxt)
+{
+ return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_TGE;
+}
+
+static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)
+{
+ return __vcpu_el2_tge_is_set(&vcpu->arch.ctxt);
+}
+
+static inline bool __is_hyp_ctxt(const struct kvm_cpu_context *ctxt)
+{
+ /*
+ * We are in a hypervisor context if the vcpu mode is EL2 or
+ * E2H and TGE bits are set. The latter means we are in the user space
+ * of the VHE kernel. ARMv8.1 ARM describes this as 'InHost'
+ */
+ return vcpu_mode_el2_ctxt(ctxt) ||
+ (__vcpu_el2_e2h_is_set(ctxt) && __vcpu_el2_tge_is_set(ctxt)) ||
+ WARN_ON(__vcpu_el2_tge_is_set(ctxt));
+}
+
+static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
+{
+ return __is_hyp_ctxt(&vcpu->arch.ctxt);
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 06/66] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
When running a nested hypervisor we commonly have to figure out if
the VCPU mode is running in the context of a guest hypervisor or guest
guest, or just a normal guest.
Add convenient primitives for this.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 55 ++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index c8f550a53516..e3e22d364dc9 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -173,6 +173,61 @@ static __always_inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
vcpu_gp_regs(vcpu)->regs[reg_num] = val;
}
+static inline bool vcpu_mode_el2_ctxt(const struct kvm_cpu_context *ctxt)
+{
+ unsigned long cpsr = ctxt->regs.pstate;
+
+ switch (cpsr & (PSR_MODE32_BIT | PSR_MODE_MASK)) {
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static inline bool vcpu_mode_el2(const struct kvm_vcpu *vcpu)
+{
+ return vcpu_mode_el2_ctxt(&vcpu->arch.ctxt);
+}
+
+static inline bool __vcpu_el2_e2h_is_set(const struct kvm_cpu_context *ctxt)
+{
+ return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_E2H;
+}
+
+static inline bool vcpu_el2_e2h_is_set(const struct kvm_vcpu *vcpu)
+{
+ return __vcpu_el2_e2h_is_set(&vcpu->arch.ctxt);
+}
+
+static inline bool __vcpu_el2_tge_is_set(const struct kvm_cpu_context *ctxt)
+{
+ return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_TGE;
+}
+
+static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)
+{
+ return __vcpu_el2_tge_is_set(&vcpu->arch.ctxt);
+}
+
+static inline bool __is_hyp_ctxt(const struct kvm_cpu_context *ctxt)
+{
+ /*
+ * We are in a hypervisor context if the vcpu mode is EL2 or
+ * E2H and TGE bits are set. The latter means we are in the user space
+ * of the VHE kernel. ARMv8.1 ARM describes this as 'InHost'
+ */
+ return vcpu_mode_el2_ctxt(ctxt) ||
+ (__vcpu_el2_e2h_is_set(ctxt) && __vcpu_el2_tge_is_set(ctxt)) ||
+ WARN_ON(__vcpu_el2_tge_is_set(ctxt));
+}
+
+static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
+{
+ return __is_hyp_ctxt(&vcpu->arch.ctxt);
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When
this bit is set, accessing EL2 registers in EL1 traps to EL2. In
addition, executing the following instructions in EL1 will trap to EL2:
tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the
instructions that trap to EL2 with the NV bit were undef at EL1 prior to
ARM v8.3. The only instruction that was not undef is eret.
This patch sets up a handler for EL2 registers and SP_EL1 register
accesses at EL1. The host hypervisor keeps those register values in
memory, and will emulate their behavior.
This patch doesn't set the NV bit yet. It will be set in a later patch
once nested virtualization support is completed.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: added SCTLR_EL2 RES0/RES1 handling]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/sysreg.h | 44 ++++++++++++-
arch/arm64/kvm/sys_regs.c | 112 ++++++++++++++++++++++++++++++--
2 files changed, 150 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 82521cdbfc1c..05b49eafbb49 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -469,19 +469,50 @@
#define SYS_PMCCFILTR_EL0 sys_reg(3, 3, 14, 15, 7)
+#define SYS_VPIDR_EL2 sys_reg(3, 4, 0, 0, 0)
+#define SYS_VMPIDR_EL2 sys_reg(3, 4, 0, 0, 5)
+
#define SYS_SCTLR_EL2 sys_reg(3, 4, 1, 0, 0)
+#define SYS_ACTLR_EL2 sys_reg(3, 4, 1, 0, 1)
+#define SYS_HCR_EL2 sys_reg(3, 4, 1, 1, 0)
+#define SYS_MDCR_EL2 sys_reg(3, 4, 1, 1, 1)
+#define SYS_CPTR_EL2 sys_reg(3, 4, 1, 1, 2)
+#define SYS_HSTR_EL2 sys_reg(3, 4, 1, 1, 3)
+#define SYS_HACR_EL2 sys_reg(3, 4, 1, 1, 7)
+
#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
+
+#define SYS_TTBR0_EL2 sys_reg(3, 4, 2, 0, 0)
+#define SYS_TTBR1_EL2 sys_reg(3, 4, 2, 0, 1)
+#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
+#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
+#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
+
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
+
#define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0)
#define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1)
+#define SYS_SP_EL1 sys_reg(3, 4, 4, 1, 0)
+
#define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
+#define SYS_AFSR0_EL2 sys_reg(3, 4, 5, 1, 0)
+#define SYS_AFSR1_EL2 sys_reg(3, 4, 5, 1, 1)
#define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0)
#define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3)
#define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
#define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0)
#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
-#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
+#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
+#define SYS_HPFAR_EL2 sys_reg(3, 4, 6, 0, 4)
+
+#define SYS_MAIR_EL2 sys_reg(3, 4, 10, 2, 0)
+#define SYS_AMAIR_EL2 sys_reg(3, 4, 10, 3, 0)
+
+#define SYS_VBAR_EL2 sys_reg(3, 4, 12, 0, 0)
+#define SYS_RVBAR_EL2 sys_reg(3, 4, 12, 0, 1)
+#define SYS_RMR_EL2 sys_reg(3, 4, 12, 0, 2)
+#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
#define __SYS__AP0Rx_EL2(x) sys_reg(3, 4, 12, 8, x)
#define SYS_ICH_AP0R0_EL2 __SYS__AP0Rx_EL2(0)
#define SYS_ICH_AP0R1_EL2 __SYS__AP0Rx_EL2(1)
@@ -523,15 +554,24 @@
#define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
#define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
+#define SYS_CONTEXTIDR_EL2 sys_reg(3, 4, 13, 0, 1)
+#define SYS_TPIDR_EL2 sys_reg(3, 4, 13, 0, 2)
+
+#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
+#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
+
/* VHE encodings for architectural EL0/1 system registers */
#define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
#define SYS_CPACR_EL12 sys_reg(3, 5, 1, 0, 2)
#define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
+
#define SYS_TTBR0_EL12 sys_reg(3, 5, 2, 0, 0)
#define SYS_TTBR1_EL12 sys_reg(3, 5, 2, 0, 1)
#define SYS_TCR_EL12 sys_reg(3, 5, 2, 0, 2)
+
#define SYS_SPSR_EL12 sys_reg(3, 5, 4, 0, 0)
#define SYS_ELR_EL12 sys_reg(3, 5, 4, 0, 1)
+
#define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0)
#define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1)
#define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0)
@@ -549,6 +589,8 @@
#define SYS_CNTV_CTL_EL02 sys_reg(3, 5, 14, 3, 1)
#define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
+#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 3313dedfa505..c049867a39bc 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -104,6 +104,57 @@ static u32 get_ccsidr(u32 csselr)
return ccsidr;
}
+static bool access_rw(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, r->reg);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+
+ return true;
+}
+
+static bool access_wi(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return ignore_write(vcpu, p);
+
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ return true;
+}
+
+static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write) {
+ u64 val = p->regval;
+
+ if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) {
+ val &= ~(GENMASK_ULL(63,45) | GENMASK_ULL(34, 32) |
+ BIT_ULL(17));
+ val |= SCTLR_EL1_RES1;
+ } else {
+ val &= ~(GENMASK_ULL(63,45) | BIT_ULL(42) |
+ GENMASK_ULL(39, 38) | GENMASK_ULL(35, 32) |
+ BIT_ULL(26) | BIT_ULL(24) | BIT_ULL(20) |
+ BIT_ULL(17) | GENMASK_ULL(15, 14) |
+ GENMASK(10, 7));
+ val |= SCTLR_EL2_RES1;
+ }
+
+ vcpu_write_sys_reg(vcpu, val, r->reg);
+ } else {
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ }
+
+ return true;
+}
+
/*
* See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
*/
@@ -342,12 +393,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- if (p->is_write) {
- vcpu_write_sys_reg(vcpu, p->regval, r->reg);
+ access_rw(vcpu, p, r);
+ if (p->is_write)
vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
- } else {
- p->regval = vcpu_read_sys_reg(vcpu, r->reg);
- }
trace_trap_reg(__func__, r->reg, p->is_write, p->regval);
@@ -1314,6 +1362,18 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
.set_user = set_raz_id_reg, \
}
+static bool access_sp_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ __vcpu_sys_reg(vcpu, SP_EL1) = p->regval;
+ else
+ p->regval = __vcpu_sys_reg(vcpu, SP_EL1);
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1692,9 +1752,51 @@ static const struct sys_reg_desc sys_reg_descs[] = {
*/
{ SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
+ { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
+ { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
+
+ { SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
+ { SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
+ { SYS_DESC(SYS_HCR_EL2), access_rw, reset_val, HCR_EL2, 0 },
+ { SYS_DESC(SYS_MDCR_EL2), access_rw, reset_val, MDCR_EL2, 0 },
+ { SYS_DESC(SYS_CPTR_EL2), access_rw, reset_val, CPTR_EL2, CPTR_EL2_RES1 },
+ { SYS_DESC(SYS_HSTR_EL2), access_rw, reset_val, HSTR_EL2, 0 },
+ { SYS_DESC(SYS_HACR_EL2), access_rw, reset_val, HACR_EL2, 0 },
+
+ { SYS_DESC(SYS_TTBR0_EL2), access_rw, reset_val, TTBR0_EL2, 0 },
+ { SYS_DESC(SYS_TTBR1_EL2), access_rw, reset_val, TTBR1_EL2, 0 },
+ { SYS_DESC(SYS_TCR_EL2), access_rw, reset_val, TCR_EL2, TCR_EL2_RES1 },
+ { SYS_DESC(SYS_VTTBR_EL2), access_rw, reset_val, VTTBR_EL2, 0 },
+ { SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
+
{ SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
+ { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
+ { SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
+ { SYS_DESC(SYS_SP_EL1), access_sp_el1},
+
{ SYS_DESC(SYS_IFSR32_EL2), NULL, reset_unknown, IFSR32_EL2 },
+ { SYS_DESC(SYS_AFSR0_EL2), access_rw, reset_val, AFSR0_EL2, 0 },
+ { SYS_DESC(SYS_AFSR1_EL2), access_rw, reset_val, AFSR1_EL2, 0 },
+ { SYS_DESC(SYS_ESR_EL2), access_rw, reset_val, ESR_EL2, 0 },
{ SYS_DESC(SYS_FPEXC32_EL2), NULL, reset_val, FPEXC32_EL2, 0x700 },
+
+ { SYS_DESC(SYS_FAR_EL2), access_rw, reset_val, FAR_EL2, 0 },
+ { SYS_DESC(SYS_HPFAR_EL2), access_rw, reset_val, HPFAR_EL2, 0 },
+
+ { SYS_DESC(SYS_MAIR_EL2), access_rw, reset_val, MAIR_EL2, 0 },
+ { SYS_DESC(SYS_AMAIR_EL2), access_rw, reset_val, AMAIR_EL2, 0 },
+
+ { SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
+ { SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
+ { SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
+
+ { SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
+ { SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
+
+ { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
+ { SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+
+ { SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When
this bit is set, accessing EL2 registers in EL1 traps to EL2. In
addition, executing the following instructions in EL1 will trap to EL2:
tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the
instructions that trap to EL2 with the NV bit were undef at EL1 prior to
ARM v8.3. The only instruction that was not undef is eret.
This patch sets up a handler for EL2 registers and SP_EL1 register
accesses at EL1. The host hypervisor keeps those register values in
memory, and will emulate their behavior.
This patch doesn't set the NV bit yet. It will be set in a later patch
once nested virtualization support is completed.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: added SCTLR_EL2 RES0/RES1 handling]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/sysreg.h | 44 ++++++++++++-
arch/arm64/kvm/sys_regs.c | 112 ++++++++++++++++++++++++++++++--
2 files changed, 150 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 82521cdbfc1c..05b49eafbb49 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -469,19 +469,50 @@
#define SYS_PMCCFILTR_EL0 sys_reg(3, 3, 14, 15, 7)
+#define SYS_VPIDR_EL2 sys_reg(3, 4, 0, 0, 0)
+#define SYS_VMPIDR_EL2 sys_reg(3, 4, 0, 0, 5)
+
#define SYS_SCTLR_EL2 sys_reg(3, 4, 1, 0, 0)
+#define SYS_ACTLR_EL2 sys_reg(3, 4, 1, 0, 1)
+#define SYS_HCR_EL2 sys_reg(3, 4, 1, 1, 0)
+#define SYS_MDCR_EL2 sys_reg(3, 4, 1, 1, 1)
+#define SYS_CPTR_EL2 sys_reg(3, 4, 1, 1, 2)
+#define SYS_HSTR_EL2 sys_reg(3, 4, 1, 1, 3)
+#define SYS_HACR_EL2 sys_reg(3, 4, 1, 1, 7)
+
#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
+
+#define SYS_TTBR0_EL2 sys_reg(3, 4, 2, 0, 0)
+#define SYS_TTBR1_EL2 sys_reg(3, 4, 2, 0, 1)
+#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
+#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
+#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
+
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
+
#define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0)
#define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1)
+#define SYS_SP_EL1 sys_reg(3, 4, 4, 1, 0)
+
#define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
+#define SYS_AFSR0_EL2 sys_reg(3, 4, 5, 1, 0)
+#define SYS_AFSR1_EL2 sys_reg(3, 4, 5, 1, 1)
#define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0)
#define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3)
#define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
#define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0)
#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
-#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
+#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
+#define SYS_HPFAR_EL2 sys_reg(3, 4, 6, 0, 4)
+
+#define SYS_MAIR_EL2 sys_reg(3, 4, 10, 2, 0)
+#define SYS_AMAIR_EL2 sys_reg(3, 4, 10, 3, 0)
+
+#define SYS_VBAR_EL2 sys_reg(3, 4, 12, 0, 0)
+#define SYS_RVBAR_EL2 sys_reg(3, 4, 12, 0, 1)
+#define SYS_RMR_EL2 sys_reg(3, 4, 12, 0, 2)
+#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
#define __SYS__AP0Rx_EL2(x) sys_reg(3, 4, 12, 8, x)
#define SYS_ICH_AP0R0_EL2 __SYS__AP0Rx_EL2(0)
#define SYS_ICH_AP0R1_EL2 __SYS__AP0Rx_EL2(1)
@@ -523,15 +554,24 @@
#define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
#define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
+#define SYS_CONTEXTIDR_EL2 sys_reg(3, 4, 13, 0, 1)
+#define SYS_TPIDR_EL2 sys_reg(3, 4, 13, 0, 2)
+
+#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
+#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
+
/* VHE encodings for architectural EL0/1 system registers */
#define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
#define SYS_CPACR_EL12 sys_reg(3, 5, 1, 0, 2)
#define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
+
#define SYS_TTBR0_EL12 sys_reg(3, 5, 2, 0, 0)
#define SYS_TTBR1_EL12 sys_reg(3, 5, 2, 0, 1)
#define SYS_TCR_EL12 sys_reg(3, 5, 2, 0, 2)
+
#define SYS_SPSR_EL12 sys_reg(3, 5, 4, 0, 0)
#define SYS_ELR_EL12 sys_reg(3, 5, 4, 0, 1)
+
#define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0)
#define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1)
#define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0)
@@ -549,6 +589,8 @@
#define SYS_CNTV_CTL_EL02 sys_reg(3, 5, 14, 3, 1)
#define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
+#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 3313dedfa505..c049867a39bc 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -104,6 +104,57 @@ static u32 get_ccsidr(u32 csselr)
return ccsidr;
}
+static bool access_rw(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, r->reg);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+
+ return true;
+}
+
+static bool access_wi(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return ignore_write(vcpu, p);
+
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ return true;
+}
+
+static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write) {
+ u64 val = p->regval;
+
+ if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) {
+ val &= ~(GENMASK_ULL(63,45) | GENMASK_ULL(34, 32) |
+ BIT_ULL(17));
+ val |= SCTLR_EL1_RES1;
+ } else {
+ val &= ~(GENMASK_ULL(63,45) | BIT_ULL(42) |
+ GENMASK_ULL(39, 38) | GENMASK_ULL(35, 32) |
+ BIT_ULL(26) | BIT_ULL(24) | BIT_ULL(20) |
+ BIT_ULL(17) | GENMASK_ULL(15, 14) |
+ GENMASK(10, 7));
+ val |= SCTLR_EL2_RES1;
+ }
+
+ vcpu_write_sys_reg(vcpu, val, r->reg);
+ } else {
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ }
+
+ return true;
+}
+
/*
* See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
*/
@@ -342,12 +393,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- if (p->is_write) {
- vcpu_write_sys_reg(vcpu, p->regval, r->reg);
+ access_rw(vcpu, p, r);
+ if (p->is_write)
vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
- } else {
- p->regval = vcpu_read_sys_reg(vcpu, r->reg);
- }
trace_trap_reg(__func__, r->reg, p->is_write, p->regval);
@@ -1314,6 +1362,18 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
.set_user = set_raz_id_reg, \
}
+static bool access_sp_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ __vcpu_sys_reg(vcpu, SP_EL1) = p->regval;
+ else
+ p->regval = __vcpu_sys_reg(vcpu, SP_EL1);
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1692,9 +1752,51 @@ static const struct sys_reg_desc sys_reg_descs[] = {
*/
{ SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
+ { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
+ { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
+
+ { SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
+ { SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
+ { SYS_DESC(SYS_HCR_EL2), access_rw, reset_val, HCR_EL2, 0 },
+ { SYS_DESC(SYS_MDCR_EL2), access_rw, reset_val, MDCR_EL2, 0 },
+ { SYS_DESC(SYS_CPTR_EL2), access_rw, reset_val, CPTR_EL2, CPTR_EL2_RES1 },
+ { SYS_DESC(SYS_HSTR_EL2), access_rw, reset_val, HSTR_EL2, 0 },
+ { SYS_DESC(SYS_HACR_EL2), access_rw, reset_val, HACR_EL2, 0 },
+
+ { SYS_DESC(SYS_TTBR0_EL2), access_rw, reset_val, TTBR0_EL2, 0 },
+ { SYS_DESC(SYS_TTBR1_EL2), access_rw, reset_val, TTBR1_EL2, 0 },
+ { SYS_DESC(SYS_TCR_EL2), access_rw, reset_val, TCR_EL2, TCR_EL2_RES1 },
+ { SYS_DESC(SYS_VTTBR_EL2), access_rw, reset_val, VTTBR_EL2, 0 },
+ { SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
+
{ SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
+ { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
+ { SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
+ { SYS_DESC(SYS_SP_EL1), access_sp_el1},
+
{ SYS_DESC(SYS_IFSR32_EL2), NULL, reset_unknown, IFSR32_EL2 },
+ { SYS_DESC(SYS_AFSR0_EL2), access_rw, reset_val, AFSR0_EL2, 0 },
+ { SYS_DESC(SYS_AFSR1_EL2), access_rw, reset_val, AFSR1_EL2, 0 },
+ { SYS_DESC(SYS_ESR_EL2), access_rw, reset_val, ESR_EL2, 0 },
{ SYS_DESC(SYS_FPEXC32_EL2), NULL, reset_val, FPEXC32_EL2, 0x700 },
+
+ { SYS_DESC(SYS_FAR_EL2), access_rw, reset_val, FAR_EL2, 0 },
+ { SYS_DESC(SYS_HPFAR_EL2), access_rw, reset_val, HPFAR_EL2, 0 },
+
+ { SYS_DESC(SYS_MAIR_EL2), access_rw, reset_val, MAIR_EL2, 0 },
+ { SYS_DESC(SYS_AMAIR_EL2), access_rw, reset_val, AMAIR_EL2, 0 },
+
+ { SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
+ { SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
+ { SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
+
+ { SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
+ { SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
+
+ { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
+ { SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+
+ { SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When
this bit is set, accessing EL2 registers in EL1 traps to EL2. In
addition, executing the following instructions in EL1 will trap to EL2:
tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the
instructions that trap to EL2 with the NV bit were undef at EL1 prior to
ARM v8.3. The only instruction that was not undef is eret.
This patch sets up a handler for EL2 registers and SP_EL1 register
accesses at EL1. The host hypervisor keeps those register values in
memory, and will emulate their behavior.
This patch doesn't set the NV bit yet. It will be set in a later patch
once nested virtualization support is completed.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: added SCTLR_EL2 RES0/RES1 handling]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/sysreg.h | 44 ++++++++++++-
arch/arm64/kvm/sys_regs.c | 112 ++++++++++++++++++++++++++++++--
2 files changed, 150 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 82521cdbfc1c..05b49eafbb49 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -469,19 +469,50 @@
#define SYS_PMCCFILTR_EL0 sys_reg(3, 3, 14, 15, 7)
+#define SYS_VPIDR_EL2 sys_reg(3, 4, 0, 0, 0)
+#define SYS_VMPIDR_EL2 sys_reg(3, 4, 0, 0, 5)
+
#define SYS_SCTLR_EL2 sys_reg(3, 4, 1, 0, 0)
+#define SYS_ACTLR_EL2 sys_reg(3, 4, 1, 0, 1)
+#define SYS_HCR_EL2 sys_reg(3, 4, 1, 1, 0)
+#define SYS_MDCR_EL2 sys_reg(3, 4, 1, 1, 1)
+#define SYS_CPTR_EL2 sys_reg(3, 4, 1, 1, 2)
+#define SYS_HSTR_EL2 sys_reg(3, 4, 1, 1, 3)
+#define SYS_HACR_EL2 sys_reg(3, 4, 1, 1, 7)
+
#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
+
+#define SYS_TTBR0_EL2 sys_reg(3, 4, 2, 0, 0)
+#define SYS_TTBR1_EL2 sys_reg(3, 4, 2, 0, 1)
+#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
+#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
+#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
+
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
+
#define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0)
#define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1)
+#define SYS_SP_EL1 sys_reg(3, 4, 4, 1, 0)
+
#define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
+#define SYS_AFSR0_EL2 sys_reg(3, 4, 5, 1, 0)
+#define SYS_AFSR1_EL2 sys_reg(3, 4, 5, 1, 1)
#define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0)
#define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3)
#define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
#define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0)
#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
-#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
+#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
+#define SYS_HPFAR_EL2 sys_reg(3, 4, 6, 0, 4)
+
+#define SYS_MAIR_EL2 sys_reg(3, 4, 10, 2, 0)
+#define SYS_AMAIR_EL2 sys_reg(3, 4, 10, 3, 0)
+
+#define SYS_VBAR_EL2 sys_reg(3, 4, 12, 0, 0)
+#define SYS_RVBAR_EL2 sys_reg(3, 4, 12, 0, 1)
+#define SYS_RMR_EL2 sys_reg(3, 4, 12, 0, 2)
+#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
#define __SYS__AP0Rx_EL2(x) sys_reg(3, 4, 12, 8, x)
#define SYS_ICH_AP0R0_EL2 __SYS__AP0Rx_EL2(0)
#define SYS_ICH_AP0R1_EL2 __SYS__AP0Rx_EL2(1)
@@ -523,15 +554,24 @@
#define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
#define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
+#define SYS_CONTEXTIDR_EL2 sys_reg(3, 4, 13, 0, 1)
+#define SYS_TPIDR_EL2 sys_reg(3, 4, 13, 0, 2)
+
+#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
+#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
+
/* VHE encodings for architectural EL0/1 system registers */
#define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
#define SYS_CPACR_EL12 sys_reg(3, 5, 1, 0, 2)
#define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
+
#define SYS_TTBR0_EL12 sys_reg(3, 5, 2, 0, 0)
#define SYS_TTBR1_EL12 sys_reg(3, 5, 2, 0, 1)
#define SYS_TCR_EL12 sys_reg(3, 5, 2, 0, 2)
+
#define SYS_SPSR_EL12 sys_reg(3, 5, 4, 0, 0)
#define SYS_ELR_EL12 sys_reg(3, 5, 4, 0, 1)
+
#define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0)
#define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1)
#define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0)
@@ -549,6 +589,8 @@
#define SYS_CNTV_CTL_EL02 sys_reg(3, 5, 14, 3, 1)
#define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
+#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 3313dedfa505..c049867a39bc 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -104,6 +104,57 @@ static u32 get_ccsidr(u32 csselr)
return ccsidr;
}
+static bool access_rw(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, r->reg);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+
+ return true;
+}
+
+static bool access_wi(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return ignore_write(vcpu, p);
+
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ return true;
+}
+
+static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write) {
+ u64 val = p->regval;
+
+ if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) {
+ val &= ~(GENMASK_ULL(63,45) | GENMASK_ULL(34, 32) |
+ BIT_ULL(17));
+ val |= SCTLR_EL1_RES1;
+ } else {
+ val &= ~(GENMASK_ULL(63,45) | BIT_ULL(42) |
+ GENMASK_ULL(39, 38) | GENMASK_ULL(35, 32) |
+ BIT_ULL(26) | BIT_ULL(24) | BIT_ULL(20) |
+ BIT_ULL(17) | GENMASK_ULL(15, 14) |
+ GENMASK(10, 7));
+ val |= SCTLR_EL2_RES1;
+ }
+
+ vcpu_write_sys_reg(vcpu, val, r->reg);
+ } else {
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ }
+
+ return true;
+}
+
/*
* See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
*/
@@ -342,12 +393,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- if (p->is_write) {
- vcpu_write_sys_reg(vcpu, p->regval, r->reg);
+ access_rw(vcpu, p, r);
+ if (p->is_write)
vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
- } else {
- p->regval = vcpu_read_sys_reg(vcpu, r->reg);
- }
trace_trap_reg(__func__, r->reg, p->is_write, p->regval);
@@ -1314,6 +1362,18 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
.set_user = set_raz_id_reg, \
}
+static bool access_sp_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ __vcpu_sys_reg(vcpu, SP_EL1) = p->regval;
+ else
+ p->regval = __vcpu_sys_reg(vcpu, SP_EL1);
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1692,9 +1752,51 @@ static const struct sys_reg_desc sys_reg_descs[] = {
*/
{ SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
+ { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
+ { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
+
+ { SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
+ { SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
+ { SYS_DESC(SYS_HCR_EL2), access_rw, reset_val, HCR_EL2, 0 },
+ { SYS_DESC(SYS_MDCR_EL2), access_rw, reset_val, MDCR_EL2, 0 },
+ { SYS_DESC(SYS_CPTR_EL2), access_rw, reset_val, CPTR_EL2, CPTR_EL2_RES1 },
+ { SYS_DESC(SYS_HSTR_EL2), access_rw, reset_val, HSTR_EL2, 0 },
+ { SYS_DESC(SYS_HACR_EL2), access_rw, reset_val, HACR_EL2, 0 },
+
+ { SYS_DESC(SYS_TTBR0_EL2), access_rw, reset_val, TTBR0_EL2, 0 },
+ { SYS_DESC(SYS_TTBR1_EL2), access_rw, reset_val, TTBR1_EL2, 0 },
+ { SYS_DESC(SYS_TCR_EL2), access_rw, reset_val, TCR_EL2, TCR_EL2_RES1 },
+ { SYS_DESC(SYS_VTTBR_EL2), access_rw, reset_val, VTTBR_EL2, 0 },
+ { SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
+
{ SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
+ { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
+ { SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
+ { SYS_DESC(SYS_SP_EL1), access_sp_el1},
+
{ SYS_DESC(SYS_IFSR32_EL2), NULL, reset_unknown, IFSR32_EL2 },
+ { SYS_DESC(SYS_AFSR0_EL2), access_rw, reset_val, AFSR0_EL2, 0 },
+ { SYS_DESC(SYS_AFSR1_EL2), access_rw, reset_val, AFSR1_EL2, 0 },
+ { SYS_DESC(SYS_ESR_EL2), access_rw, reset_val, ESR_EL2, 0 },
{ SYS_DESC(SYS_FPEXC32_EL2), NULL, reset_val, FPEXC32_EL2, 0x700 },
+
+ { SYS_DESC(SYS_FAR_EL2), access_rw, reset_val, FAR_EL2, 0 },
+ { SYS_DESC(SYS_HPFAR_EL2), access_rw, reset_val, HPFAR_EL2, 0 },
+
+ { SYS_DESC(SYS_MAIR_EL2), access_rw, reset_val, MAIR_EL2, 0 },
+ { SYS_DESC(SYS_AMAIR_EL2), access_rw, reset_val, AMAIR_EL2, 0 },
+
+ { SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
+ { SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
+ { SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
+
+ { SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
+ { SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
+
+ { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
+ { SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+
+ { SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 08/66] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
The VMPIDR_EL2 and VPIDR_EL2 are architecturally UNKNOWN at reset, but
let's be nice to a guest hypervisor behaving foolishly and reset these
to something reasonable anyway.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c049867a39bc..cc7a86624e36 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -621,7 +621,7 @@ static void reset_actlr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
vcpu_write_sys_reg(vcpu, actlr, ACTLR_EL1);
}
-static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+static u64 compute_reset_mpidr(struct kvm_vcpu *vcpu)
{
u64 mpidr;
@@ -635,7 +635,24 @@ static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
mpidr = (vcpu->vcpu_id & 0x0f) << MPIDR_LEVEL_SHIFT(0);
mpidr |= ((vcpu->vcpu_id >> 4) & 0xff) << MPIDR_LEVEL_SHIFT(1);
mpidr |= ((vcpu->vcpu_id >> 12) & 0xff) << MPIDR_LEVEL_SHIFT(2);
- vcpu_write_sys_reg(vcpu, (1ULL << 31) | mpidr, MPIDR_EL1);
+ mpidr |= (1ULL << 31);
+
+ return mpidr;
+}
+
+static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, compute_reset_mpidr(vcpu), MPIDR_EL1);
+}
+
+static void reset_vmpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, compute_reset_mpidr(vcpu), VMPIDR_EL2);
+}
+
+static void reset_vpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, read_cpuid_id(), VPIDR_EL2);
}
static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
@@ -1752,8 +1769,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
*/
{ SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
- { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
- { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
+ { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_vpidr, VPIDR_EL2 },
+ { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_vmpidr, VMPIDR_EL2 },
{ SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
{ SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 08/66] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
The VMPIDR_EL2 and VPIDR_EL2 are architecturally UNKNOWN at reset, but
let's be nice to a guest hypervisor behaving foolishly and reset these
to something reasonable anyway.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c049867a39bc..cc7a86624e36 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -621,7 +621,7 @@ static void reset_actlr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
vcpu_write_sys_reg(vcpu, actlr, ACTLR_EL1);
}
-static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+static u64 compute_reset_mpidr(struct kvm_vcpu *vcpu)
{
u64 mpidr;
@@ -635,7 +635,24 @@ static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
mpidr = (vcpu->vcpu_id & 0x0f) << MPIDR_LEVEL_SHIFT(0);
mpidr |= ((vcpu->vcpu_id >> 4) & 0xff) << MPIDR_LEVEL_SHIFT(1);
mpidr |= ((vcpu->vcpu_id >> 12) & 0xff) << MPIDR_LEVEL_SHIFT(2);
- vcpu_write_sys_reg(vcpu, (1ULL << 31) | mpidr, MPIDR_EL1);
+ mpidr |= (1ULL << 31);
+
+ return mpidr;
+}
+
+static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, compute_reset_mpidr(vcpu), MPIDR_EL1);
+}
+
+static void reset_vmpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, compute_reset_mpidr(vcpu), VMPIDR_EL2);
+}
+
+static void reset_vpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, read_cpuid_id(), VPIDR_EL2);
}
static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
@@ -1752,8 +1769,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
*/
{ SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
- { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
- { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
+ { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_vpidr, VPIDR_EL2 },
+ { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_vmpidr, VMPIDR_EL2 },
{ SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
{ SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 08/66] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
The VMPIDR_EL2 and VPIDR_EL2 are architecturally UNKNOWN at reset, but
let's be nice to a guest hypervisor behaving foolishly and reset these
to something reasonable anyway.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c049867a39bc..cc7a86624e36 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -621,7 +621,7 @@ static void reset_actlr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
vcpu_write_sys_reg(vcpu, actlr, ACTLR_EL1);
}
-static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+static u64 compute_reset_mpidr(struct kvm_vcpu *vcpu)
{
u64 mpidr;
@@ -635,7 +635,24 @@ static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
mpidr = (vcpu->vcpu_id & 0x0f) << MPIDR_LEVEL_SHIFT(0);
mpidr |= ((vcpu->vcpu_id >> 4) & 0xff) << MPIDR_LEVEL_SHIFT(1);
mpidr |= ((vcpu->vcpu_id >> 12) & 0xff) << MPIDR_LEVEL_SHIFT(2);
- vcpu_write_sys_reg(vcpu, (1ULL << 31) | mpidr, MPIDR_EL1);
+ mpidr |= (1ULL << 31);
+
+ return mpidr;
+}
+
+static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, compute_reset_mpidr(vcpu), MPIDR_EL1);
+}
+
+static void reset_vmpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, compute_reset_mpidr(vcpu), VMPIDR_EL2);
+}
+
+static void reset_vpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
+{
+ vcpu_write_sys_reg(vcpu, read_cpuid_id(), VPIDR_EL2);
}
static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
@@ -1752,8 +1769,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
*/
{ SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
- { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
- { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
+ { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_vpidr, VPIDR_EL2 },
+ { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_vmpidr, VMPIDR_EL2 },
{ SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
{ SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Support injecting exceptions and performing exception returns to and
from virtual EL2. This must be done entirely in software except when
taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE}
== {1,1} (a VHE guest hypervisor).
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: switch to common exception injection framework]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 17 +++
arch/arm64/include/asm/kvm_emulate.h | 10 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/emulate-nested.c | 176 +++++++++++++++++++++++++++
arch/arm64/kvm/hyp/exception.c | 44 +++++--
arch/arm64/kvm/inject_fault.c | 62 ++++++++--
arch/arm64/kvm/trace_arm.h | 59 +++++++++
7 files changed, 352 insertions(+), 18 deletions(-)
create mode 100644 arch/arm64/kvm/emulate-nested.c
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4e90c2debf70..e9e10e498785 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -332,4 +332,21 @@
#define CPACR_EL1_TTA (1 << 28)
#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
+#define kvm_mode_names \
+ { PSR_MODE_EL0t, "EL0t" }, \
+ { PSR_MODE_EL1t, "EL1t" }, \
+ { PSR_MODE_EL1h, "EL1h" }, \
+ { PSR_MODE_EL2t, "EL2t" }, \
+ { PSR_MODE_EL2h, "EL2h" }, \
+ { PSR_MODE_EL3t, "EL3t" }, \
+ { PSR_MODE_EL3h, "EL3h" }, \
+ { PSR_AA32_MODE_USR, "32-bit USR" }, \
+ { PSR_AA32_MODE_FIQ, "32-bit FIQ" }, \
+ { PSR_AA32_MODE_IRQ, "32-bit IRQ" }, \
+ { PSR_AA32_MODE_SVC, "32-bit SVC" }, \
+ { PSR_AA32_MODE_ABT, "32-bit ABT" }, \
+ { PSR_AA32_MODE_HYP, "32-bit HYP" }, \
+ { PSR_AA32_MODE_UND, "32-bit UND" }, \
+ { PSR_AA32_MODE_SYS, "32-bit SYS" }
+
#endif /* __ARM64_KVM_ARM_H__ */
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index e3e22d364dc9..deffd3cc4443 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -33,6 +33,12 @@ enum exception_type {
except_type_serror = 0x180,
};
+#define kvm_exception_type_names \
+ { except_type_sync, "SYNC" }, \
+ { except_type_irq, "IRQ" }, \
+ { except_type_fiq, "FIQ" }, \
+ { except_type_serror, "SERROR" }
+
bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);
void kvm_skip_instr32(struct kvm_vcpu *vcpu);
@@ -41,6 +47,10 @@ void kvm_inject_vabt(struct kvm_vcpu *vcpu);
void kvm_inject_dabt(struct kvm_vcpu *vcpu, unsigned long addr);
void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
+void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu);
+int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2);
+int kvm_inject_nested_irq(struct kvm_vcpu *vcpu);
+
static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
{
return !(vcpu->arch.hcr_el2 & HCR_RW);
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 60fd181df624..672acdf1e993 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,11 +17,11 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
+ emulate-nested.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
vgic/vgic-its.o vgic/vgic-debug.o
-
kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
new file mode 100644
index 000000000000..ee91bcd925d8
--- /dev/null
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -0,0 +1,176 @@
+/*
+ * Copyright (C) 2016 - Linaro and Columbia University
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
+
+#include "hyp/include/hyp/adjust_pc.h"
+
+#include "trace.h"
+
+void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
+{
+ u64 spsr, elr, mode;
+ bool direct_eret;
+
+ /*
+ * Going through the whole put/load motions is a waste of time
+ * if this is a VHE guest hypervisor returning to its own
+ * userspace, or the hypervisor performing a local exception
+ * return. No need to save/restore registers, no need to
+ * switch S2 MMU. Just do the canonical ERET.
+ */
+ spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
+ mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ direct_eret = (mode == PSR_MODE_EL0t &&
+ vcpu_el2_e2h_is_set(vcpu) &&
+ vcpu_el2_tge_is_set(vcpu));
+ direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
+
+ if (direct_eret) {
+ *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
+ *vcpu_cpsr(vcpu) = spsr;
+ trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
+ return;
+ }
+
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+
+ elr = __vcpu_sys_reg(vcpu, ELR_EL2);
+
+ trace_kvm_nested_eret(vcpu, elr, spsr);
+
+ /*
+ * Note that the current exception level is always the virtual EL2,
+ * since we set HCR_EL2.NV bit only when entering the virtual EL2.
+ */
+ *vcpu_pc(vcpu) = elr;
+ *vcpu_cpsr(vcpu) = spsr;
+
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+}
+
+static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,
+ enum exception_type type)
+{
+ trace_kvm_inject_nested_exception(vcpu, esr_el2, type);
+
+ switch (type) {
+ case except_type_sync:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_SYNC;
+ break;
+ case except_type_irq:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_IRQ;
+ break;
+ default:
+ WARN_ONCE(1, "Unsupported EL2 exception injection %d\n", type);
+ }
+
+ vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL2 |
+ KVM_ARM64_PENDING_EXCEPTION);
+
+ vcpu_write_sys_reg(vcpu, esr_el2, ESR_EL2);
+}
+
+/*
+ * Emulate taking an exception to EL2.
+ * See ARM ARM J8.1.2 AArch64.TakeException()
+ */
+static int kvm_inject_nested(struct kvm_vcpu *vcpu, u64 esr_el2,
+ enum exception_type type)
+{
+ u64 pstate, mode;
+ bool direct_inject;
+
+ if (!nested_virt_in_use(vcpu)) {
+ kvm_err("Unexpected call to %s for the non-nesting configuration\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ /*
+ * As for ERET, we can avoid doing too much on the injection path by
+ * checking that we either took the exception from a VHE host
+ * userspace or from vEL2. In these cases, there is no change in
+ * translation regime (or anything else), so let's do as little as
+ * possible.
+ */
+ pstate = *vcpu_cpsr(vcpu);
+ mode = pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ direct_inject = (mode == PSR_MODE_EL0t &&
+ vcpu_el2_e2h_is_set(vcpu) &&
+ vcpu_el2_tge_is_set(vcpu));
+ direct_inject |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
+
+ if (direct_inject) {
+ kvm_inject_el2_exception(vcpu, esr_el2, type);
+ return 1;
+ }
+
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+
+ kvm_inject_el2_exception(vcpu, esr_el2, type);
+
+ /*
+ * A hard requirement is that a switch between EL1 and EL2
+ * contexts has to happen between a put/load, so that we can
+ * pick the correct timer and interrupt configuration, among
+ * other things.
+ *
+ * Make sure the exception actually took place before we load
+ * the new context.
+ */
+ __adjust_pc(vcpu);
+
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+
+ return 1;
+}
+
+int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2)
+{
+ return kvm_inject_nested(vcpu, esr_el2, except_type_sync);
+}
+
+int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)
+{
+ /*
+ * Do not inject an irq if the:
+ * - Current exception level is EL2, and
+ * - virtual HCR_EL2.TGE == 0
+ * - virtual HCR_EL2.IMO == 0
+ *
+ * See Table D1-17 "Physical interrupt target and masking when EL3 is
+ * not implemented and EL2 is implemented" in ARM DDI 0487C.a.
+ */
+
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_tge_is_set(vcpu) &&
+ !(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO))
+ return 1;
+
+ /* esr_el2 value doesn't matter for exits due to irqs. */
+ return kvm_inject_nested(vcpu, 0, except_type_irq);
+}
diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
index 73629094f903..7c8b3681dafd 100644
--- a/arch/arm64/kvm/hyp/exception.c
+++ b/arch/arm64/kvm/hyp/exception.c
@@ -22,7 +22,9 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
{
u64 val;
- if (__vcpu_read_sys_reg_from_cpu(reg, &val))
+ if (unlikely(nested_virt_in_use(vcpu)))
+ return vcpu_read_sys_reg(vcpu, reg);
+ else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
return val;
return __vcpu_sys_reg(vcpu, reg);
@@ -30,14 +32,26 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
{
- if (__vcpu_write_sys_reg_to_cpu(val, reg))
+ if (unlikely(nested_virt_in_use(vcpu)))
+ vcpu_write_sys_reg(vcpu, val, reg);
+ else if (__vcpu_write_sys_reg_to_cpu(val, reg))
return;
__vcpu_sys_reg(vcpu, reg) = val;
}
-static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, u64 val)
+static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode,
+ u64 val)
{
+ if (unlikely(nested_virt_in_use(vcpu))) {
+ if (target_mode == PSR_MODE_EL1h)
+ vcpu_write_sys_reg(vcpu, val, SPSR_EL1);
+ else
+ vcpu_write_sys_reg(vcpu, val, SPSR_EL2);
+
+ return;
+ }
+
write_sysreg_el1(val, SYS_SPSR);
}
@@ -97,6 +111,11 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL1);
__vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL1);
break;
+ case PSR_MODE_EL2h:
+ vbar = __vcpu_read_sys_reg(vcpu, VBAR_EL2);
+ sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL2);
+ __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL2);
+ break;
default:
/* Don't do that */
BUG();
@@ -148,7 +167,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
new |= target_mode;
*vcpu_cpsr(vcpu) = new;
- __vcpu_write_spsr(vcpu, old);
+ __vcpu_write_spsr(vcpu, target_mode, old);
}
/*
@@ -319,11 +338,22 @@ void kvm_inject_exception(struct kvm_vcpu *vcpu)
KVM_ARM64_EXCEPT_AA64_EL1):
enter_exception64(vcpu, PSR_MODE_EL1h, except_type_sync);
break;
+
+ case (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
+ KVM_ARM64_EXCEPT_AA64_EL2):
+ enter_exception64(vcpu, PSR_MODE_EL2h, except_type_sync);
+ break;
+
+ case (KVM_ARM64_EXCEPT_AA64_ELx_IRQ |
+ KVM_ARM64_EXCEPT_AA64_EL2):
+ enter_exception64(vcpu, PSR_MODE_EL2h, except_type_irq);
+ break;
+
default:
/*
- * Only EL1_SYNC makes sense so far, EL2_{SYNC,IRQ}
- * will be implemented at some point. Everything
- * else gets silently ignored.
+ * Only EL1_SYNC and EL2_{SYNC,IRQ} makes
+ * sense so far. Everything else gets silently
+ * ignored.
*/
break;
}
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index b47df73e98d7..530562eb09fd 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -14,17 +14,50 @@
#include <asm/kvm_emulate.h>
#include <asm/esr.h>
+static void pend_sync_exception(struct kvm_vcpu *vcpu)
+{
+ vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
+ KVM_ARM64_PENDING_EXCEPTION);
+
+ /* If not nesting, EL1 is the only possible exception target */
+ if (likely(!nested_virt_in_use(vcpu))) {
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ return;
+ }
+
+ /*
+ * With NV, we need to pick between EL1 and EL2. Note that we
+ * never deal with a nesting exception here, hence never
+ * changing context, and the exception itself can be delayed
+ * until the next entry.
+ */
+ switch(*vcpu_cpsr(vcpu) & PSR_MODE_MASK) {
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
+ break;
+ case PSR_MODE_EL1h:
+ case PSR_MODE_EL1t:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ break;
+ case PSR_MODE_EL0t:
+ if (vcpu_el2_tge_is_set(vcpu) & HCR_TGE)
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
+ else
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ break;
+ default:
+ BUG();
+ }
+}
+
static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
{
unsigned long cpsr = *vcpu_cpsr(vcpu);
bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
u32 esr = 0;
- vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
- KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
- KVM_ARM64_PENDING_EXCEPTION);
-
- vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
+ pend_sync_exception(vcpu);
/*
* Build an {i,d}abort, depending on the level and the
@@ -45,16 +78,22 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
if (!is_iabt)
esr |= ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT;
- vcpu_write_sys_reg(vcpu, esr | ESR_ELx_FSC_EXTABT, ESR_EL1);
+ esr |= ESR_ELx_FSC_EXTABT;
+
+ if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1) {
+ vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ } else {
+ vcpu_write_sys_reg(vcpu, addr, FAR_EL2);
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
+ }
}
static void inject_undef64(struct kvm_vcpu *vcpu)
{
u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
- vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
- KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
- KVM_ARM64_PENDING_EXCEPTION);
+ pend_sync_exception(vcpu);
/*
* Build an unknown exception, depending on the instruction
@@ -63,7 +102,10 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
if (kvm_vcpu_trap_il_is32bit(vcpu))
esr |= ESR_ELx_IL;
- vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1)
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ else
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
}
#define DFSR_FSC_EXTABT_LPAE 0x10
diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
index ff0444352bba..5707011c4f47 100644
--- a/arch/arm64/kvm/trace_arm.h
+++ b/arch/arm64/kvm/trace_arm.h
@@ -2,6 +2,7 @@
#if !defined(_TRACE_ARM_ARM64_KVM_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_ARM_ARM64_KVM_H
+#include <asm/kvm_emulate.h>
#include <kvm/arm_arch_timer.h>
#include <linux/tracepoint.h>
@@ -367,6 +368,64 @@ TRACE_EVENT(kvm_timer_emulate,
__entry->timer_idx, __entry->should_fire)
);
+TRACE_EVENT(kvm_nested_eret,
+ TP_PROTO(struct kvm_vcpu *vcpu, unsigned long elr_el2,
+ unsigned long spsr_el2),
+ TP_ARGS(vcpu, elr_el2, spsr_el2),
+
+ TP_STRUCT__entry(
+ __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, elr_el2)
+ __field(unsigned long, spsr_el2)
+ __field(unsigned long, target_mode)
+ __field(unsigned long, hcr_el2)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->elr_el2 = elr_el2;
+ __entry->spsr_el2 = spsr_el2;
+ __entry->target_mode = spsr_el2 & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+ ),
+
+ TP_printk("elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
+ __entry->elr_el2, __entry->spsr_el2,
+ __print_symbolic(__entry->target_mode, kvm_mode_names),
+ __entry->hcr_el2)
+);
+
+TRACE_EVENT(kvm_inject_nested_exception,
+ TP_PROTO(struct kvm_vcpu *vcpu, u64 esr_el2, int type),
+ TP_ARGS(vcpu, esr_el2, type),
+
+ TP_STRUCT__entry(
+ __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, esr_el2)
+ __field(int, type)
+ __field(unsigned long, spsr_el2)
+ __field(unsigned long, pc)
+ __field(unsigned long, source_mode)
+ __field(unsigned long, hcr_el2)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->esr_el2 = esr_el2;
+ __entry->type = type;
+ __entry->spsr_el2 = *vcpu_cpsr(vcpu);
+ __entry->pc = *vcpu_pc(vcpu);
+ __entry->source_mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+ ),
+
+ TP_printk("%s: esr_el2 0x%lx elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
+ __print_symbolic(__entry->type, kvm_exception_type_names),
+ __entry->esr_el2, __entry->pc, __entry->spsr_el2,
+ __print_symbolic(__entry->source_mode, kvm_mode_names),
+ __entry->hcr_el2)
+);
+
#endif /* _TRACE_ARM_ARM64_KVM_H */
#undef TRACE_INCLUDE_PATH
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Support injecting exceptions and performing exception returns to and
from virtual EL2. This must be done entirely in software except when
taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE}
== {1,1} (a VHE guest hypervisor).
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: switch to common exception injection framework]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 17 +++
arch/arm64/include/asm/kvm_emulate.h | 10 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/emulate-nested.c | 176 +++++++++++++++++++++++++++
arch/arm64/kvm/hyp/exception.c | 44 +++++--
arch/arm64/kvm/inject_fault.c | 62 ++++++++--
arch/arm64/kvm/trace_arm.h | 59 +++++++++
7 files changed, 352 insertions(+), 18 deletions(-)
create mode 100644 arch/arm64/kvm/emulate-nested.c
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4e90c2debf70..e9e10e498785 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -332,4 +332,21 @@
#define CPACR_EL1_TTA (1 << 28)
#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
+#define kvm_mode_names \
+ { PSR_MODE_EL0t, "EL0t" }, \
+ { PSR_MODE_EL1t, "EL1t" }, \
+ { PSR_MODE_EL1h, "EL1h" }, \
+ { PSR_MODE_EL2t, "EL2t" }, \
+ { PSR_MODE_EL2h, "EL2h" }, \
+ { PSR_MODE_EL3t, "EL3t" }, \
+ { PSR_MODE_EL3h, "EL3h" }, \
+ { PSR_AA32_MODE_USR, "32-bit USR" }, \
+ { PSR_AA32_MODE_FIQ, "32-bit FIQ" }, \
+ { PSR_AA32_MODE_IRQ, "32-bit IRQ" }, \
+ { PSR_AA32_MODE_SVC, "32-bit SVC" }, \
+ { PSR_AA32_MODE_ABT, "32-bit ABT" }, \
+ { PSR_AA32_MODE_HYP, "32-bit HYP" }, \
+ { PSR_AA32_MODE_UND, "32-bit UND" }, \
+ { PSR_AA32_MODE_SYS, "32-bit SYS" }
+
#endif /* __ARM64_KVM_ARM_H__ */
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index e3e22d364dc9..deffd3cc4443 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -33,6 +33,12 @@ enum exception_type {
except_type_serror = 0x180,
};
+#define kvm_exception_type_names \
+ { except_type_sync, "SYNC" }, \
+ { except_type_irq, "IRQ" }, \
+ { except_type_fiq, "FIQ" }, \
+ { except_type_serror, "SERROR" }
+
bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);
void kvm_skip_instr32(struct kvm_vcpu *vcpu);
@@ -41,6 +47,10 @@ void kvm_inject_vabt(struct kvm_vcpu *vcpu);
void kvm_inject_dabt(struct kvm_vcpu *vcpu, unsigned long addr);
void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
+void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu);
+int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2);
+int kvm_inject_nested_irq(struct kvm_vcpu *vcpu);
+
static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
{
return !(vcpu->arch.hcr_el2 & HCR_RW);
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 60fd181df624..672acdf1e993 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,11 +17,11 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
+ emulate-nested.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
vgic/vgic-its.o vgic/vgic-debug.o
-
kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
new file mode 100644
index 000000000000..ee91bcd925d8
--- /dev/null
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -0,0 +1,176 @@
+/*
+ * Copyright (C) 2016 - Linaro and Columbia University
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
+
+#include "hyp/include/hyp/adjust_pc.h"
+
+#include "trace.h"
+
+void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
+{
+ u64 spsr, elr, mode;
+ bool direct_eret;
+
+ /*
+ * Going through the whole put/load motions is a waste of time
+ * if this is a VHE guest hypervisor returning to its own
+ * userspace, or the hypervisor performing a local exception
+ * return. No need to save/restore registers, no need to
+ * switch S2 MMU. Just do the canonical ERET.
+ */
+ spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
+ mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ direct_eret = (mode == PSR_MODE_EL0t &&
+ vcpu_el2_e2h_is_set(vcpu) &&
+ vcpu_el2_tge_is_set(vcpu));
+ direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
+
+ if (direct_eret) {
+ *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
+ *vcpu_cpsr(vcpu) = spsr;
+ trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
+ return;
+ }
+
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+
+ elr = __vcpu_sys_reg(vcpu, ELR_EL2);
+
+ trace_kvm_nested_eret(vcpu, elr, spsr);
+
+ /*
+ * Note that the current exception level is always the virtual EL2,
+ * since we set HCR_EL2.NV bit only when entering the virtual EL2.
+ */
+ *vcpu_pc(vcpu) = elr;
+ *vcpu_cpsr(vcpu) = spsr;
+
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+}
+
+static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,
+ enum exception_type type)
+{
+ trace_kvm_inject_nested_exception(vcpu, esr_el2, type);
+
+ switch (type) {
+ case except_type_sync:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_SYNC;
+ break;
+ case except_type_irq:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_IRQ;
+ break;
+ default:
+ WARN_ONCE(1, "Unsupported EL2 exception injection %d\n", type);
+ }
+
+ vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL2 |
+ KVM_ARM64_PENDING_EXCEPTION);
+
+ vcpu_write_sys_reg(vcpu, esr_el2, ESR_EL2);
+}
+
+/*
+ * Emulate taking an exception to EL2.
+ * See ARM ARM J8.1.2 AArch64.TakeException()
+ */
+static int kvm_inject_nested(struct kvm_vcpu *vcpu, u64 esr_el2,
+ enum exception_type type)
+{
+ u64 pstate, mode;
+ bool direct_inject;
+
+ if (!nested_virt_in_use(vcpu)) {
+ kvm_err("Unexpected call to %s for the non-nesting configuration\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ /*
+ * As for ERET, we can avoid doing too much on the injection path by
+ * checking that we either took the exception from a VHE host
+ * userspace or from vEL2. In these cases, there is no change in
+ * translation regime (or anything else), so let's do as little as
+ * possible.
+ */
+ pstate = *vcpu_cpsr(vcpu);
+ mode = pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ direct_inject = (mode == PSR_MODE_EL0t &&
+ vcpu_el2_e2h_is_set(vcpu) &&
+ vcpu_el2_tge_is_set(vcpu));
+ direct_inject |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
+
+ if (direct_inject) {
+ kvm_inject_el2_exception(vcpu, esr_el2, type);
+ return 1;
+ }
+
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+
+ kvm_inject_el2_exception(vcpu, esr_el2, type);
+
+ /*
+ * A hard requirement is that a switch between EL1 and EL2
+ * contexts has to happen between a put/load, so that we can
+ * pick the correct timer and interrupt configuration, among
+ * other things.
+ *
+ * Make sure the exception actually took place before we load
+ * the new context.
+ */
+ __adjust_pc(vcpu);
+
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+
+ return 1;
+}
+
+int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2)
+{
+ return kvm_inject_nested(vcpu, esr_el2, except_type_sync);
+}
+
+int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)
+{
+ /*
+ * Do not inject an irq if the:
+ * - Current exception level is EL2, and
+ * - virtual HCR_EL2.TGE == 0
+ * - virtual HCR_EL2.IMO == 0
+ *
+ * See Table D1-17 "Physical interrupt target and masking when EL3 is
+ * not implemented and EL2 is implemented" in ARM DDI 0487C.a.
+ */
+
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_tge_is_set(vcpu) &&
+ !(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO))
+ return 1;
+
+ /* esr_el2 value doesn't matter for exits due to irqs. */
+ return kvm_inject_nested(vcpu, 0, except_type_irq);
+}
diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
index 73629094f903..7c8b3681dafd 100644
--- a/arch/arm64/kvm/hyp/exception.c
+++ b/arch/arm64/kvm/hyp/exception.c
@@ -22,7 +22,9 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
{
u64 val;
- if (__vcpu_read_sys_reg_from_cpu(reg, &val))
+ if (unlikely(nested_virt_in_use(vcpu)))
+ return vcpu_read_sys_reg(vcpu, reg);
+ else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
return val;
return __vcpu_sys_reg(vcpu, reg);
@@ -30,14 +32,26 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
{
- if (__vcpu_write_sys_reg_to_cpu(val, reg))
+ if (unlikely(nested_virt_in_use(vcpu)))
+ vcpu_write_sys_reg(vcpu, val, reg);
+ else if (__vcpu_write_sys_reg_to_cpu(val, reg))
return;
__vcpu_sys_reg(vcpu, reg) = val;
}
-static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, u64 val)
+static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode,
+ u64 val)
{
+ if (unlikely(nested_virt_in_use(vcpu))) {
+ if (target_mode == PSR_MODE_EL1h)
+ vcpu_write_sys_reg(vcpu, val, SPSR_EL1);
+ else
+ vcpu_write_sys_reg(vcpu, val, SPSR_EL2);
+
+ return;
+ }
+
write_sysreg_el1(val, SYS_SPSR);
}
@@ -97,6 +111,11 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL1);
__vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL1);
break;
+ case PSR_MODE_EL2h:
+ vbar = __vcpu_read_sys_reg(vcpu, VBAR_EL2);
+ sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL2);
+ __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL2);
+ break;
default:
/* Don't do that */
BUG();
@@ -148,7 +167,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
new |= target_mode;
*vcpu_cpsr(vcpu) = new;
- __vcpu_write_spsr(vcpu, old);
+ __vcpu_write_spsr(vcpu, target_mode, old);
}
/*
@@ -319,11 +338,22 @@ void kvm_inject_exception(struct kvm_vcpu *vcpu)
KVM_ARM64_EXCEPT_AA64_EL1):
enter_exception64(vcpu, PSR_MODE_EL1h, except_type_sync);
break;
+
+ case (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
+ KVM_ARM64_EXCEPT_AA64_EL2):
+ enter_exception64(vcpu, PSR_MODE_EL2h, except_type_sync);
+ break;
+
+ case (KVM_ARM64_EXCEPT_AA64_ELx_IRQ |
+ KVM_ARM64_EXCEPT_AA64_EL2):
+ enter_exception64(vcpu, PSR_MODE_EL2h, except_type_irq);
+ break;
+
default:
/*
- * Only EL1_SYNC makes sense so far, EL2_{SYNC,IRQ}
- * will be implemented at some point. Everything
- * else gets silently ignored.
+ * Only EL1_SYNC and EL2_{SYNC,IRQ} makes
+ * sense so far. Everything else gets silently
+ * ignored.
*/
break;
}
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index b47df73e98d7..530562eb09fd 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -14,17 +14,50 @@
#include <asm/kvm_emulate.h>
#include <asm/esr.h>
+static void pend_sync_exception(struct kvm_vcpu *vcpu)
+{
+ vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
+ KVM_ARM64_PENDING_EXCEPTION);
+
+ /* If not nesting, EL1 is the only possible exception target */
+ if (likely(!nested_virt_in_use(vcpu))) {
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ return;
+ }
+
+ /*
+ * With NV, we need to pick between EL1 and EL2. Note that we
+ * never deal with a nesting exception here, hence never
+ * changing context, and the exception itself can be delayed
+ * until the next entry.
+ */
+ switch(*vcpu_cpsr(vcpu) & PSR_MODE_MASK) {
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
+ break;
+ case PSR_MODE_EL1h:
+ case PSR_MODE_EL1t:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ break;
+ case PSR_MODE_EL0t:
+ if (vcpu_el2_tge_is_set(vcpu) & HCR_TGE)
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
+ else
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ break;
+ default:
+ BUG();
+ }
+}
+
static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
{
unsigned long cpsr = *vcpu_cpsr(vcpu);
bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
u32 esr = 0;
- vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
- KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
- KVM_ARM64_PENDING_EXCEPTION);
-
- vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
+ pend_sync_exception(vcpu);
/*
* Build an {i,d}abort, depending on the level and the
@@ -45,16 +78,22 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
if (!is_iabt)
esr |= ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT;
- vcpu_write_sys_reg(vcpu, esr | ESR_ELx_FSC_EXTABT, ESR_EL1);
+ esr |= ESR_ELx_FSC_EXTABT;
+
+ if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1) {
+ vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ } else {
+ vcpu_write_sys_reg(vcpu, addr, FAR_EL2);
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
+ }
}
static void inject_undef64(struct kvm_vcpu *vcpu)
{
u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
- vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
- KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
- KVM_ARM64_PENDING_EXCEPTION);
+ pend_sync_exception(vcpu);
/*
* Build an unknown exception, depending on the instruction
@@ -63,7 +102,10 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
if (kvm_vcpu_trap_il_is32bit(vcpu))
esr |= ESR_ELx_IL;
- vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1)
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ else
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
}
#define DFSR_FSC_EXTABT_LPAE 0x10
diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
index ff0444352bba..5707011c4f47 100644
--- a/arch/arm64/kvm/trace_arm.h
+++ b/arch/arm64/kvm/trace_arm.h
@@ -2,6 +2,7 @@
#if !defined(_TRACE_ARM_ARM64_KVM_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_ARM_ARM64_KVM_H
+#include <asm/kvm_emulate.h>
#include <kvm/arm_arch_timer.h>
#include <linux/tracepoint.h>
@@ -367,6 +368,64 @@ TRACE_EVENT(kvm_timer_emulate,
__entry->timer_idx, __entry->should_fire)
);
+TRACE_EVENT(kvm_nested_eret,
+ TP_PROTO(struct kvm_vcpu *vcpu, unsigned long elr_el2,
+ unsigned long spsr_el2),
+ TP_ARGS(vcpu, elr_el2, spsr_el2),
+
+ TP_STRUCT__entry(
+ __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, elr_el2)
+ __field(unsigned long, spsr_el2)
+ __field(unsigned long, target_mode)
+ __field(unsigned long, hcr_el2)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->elr_el2 = elr_el2;
+ __entry->spsr_el2 = spsr_el2;
+ __entry->target_mode = spsr_el2 & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+ ),
+
+ TP_printk("elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
+ __entry->elr_el2, __entry->spsr_el2,
+ __print_symbolic(__entry->target_mode, kvm_mode_names),
+ __entry->hcr_el2)
+);
+
+TRACE_EVENT(kvm_inject_nested_exception,
+ TP_PROTO(struct kvm_vcpu *vcpu, u64 esr_el2, int type),
+ TP_ARGS(vcpu, esr_el2, type),
+
+ TP_STRUCT__entry(
+ __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, esr_el2)
+ __field(int, type)
+ __field(unsigned long, spsr_el2)
+ __field(unsigned long, pc)
+ __field(unsigned long, source_mode)
+ __field(unsigned long, hcr_el2)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->esr_el2 = esr_el2;
+ __entry->type = type;
+ __entry->spsr_el2 = *vcpu_cpsr(vcpu);
+ __entry->pc = *vcpu_pc(vcpu);
+ __entry->source_mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+ ),
+
+ TP_printk("%s: esr_el2 0x%lx elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
+ __print_symbolic(__entry->type, kvm_exception_type_names),
+ __entry->esr_el2, __entry->pc, __entry->spsr_el2,
+ __print_symbolic(__entry->source_mode, kvm_mode_names),
+ __entry->hcr_el2)
+);
+
#endif /* _TRACE_ARM_ARM64_KVM_H */
#undef TRACE_INCLUDE_PATH
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
Support injecting exceptions and performing exception returns to and
from virtual EL2. This must be done entirely in software except when
taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE}
== {1,1} (a VHE guest hypervisor).
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: switch to common exception injection framework]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 17 +++
arch/arm64/include/asm/kvm_emulate.h | 10 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/emulate-nested.c | 176 +++++++++++++++++++++++++++
arch/arm64/kvm/hyp/exception.c | 44 +++++--
arch/arm64/kvm/inject_fault.c | 62 ++++++++--
arch/arm64/kvm/trace_arm.h | 59 +++++++++
7 files changed, 352 insertions(+), 18 deletions(-)
create mode 100644 arch/arm64/kvm/emulate-nested.c
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4e90c2debf70..e9e10e498785 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -332,4 +332,21 @@
#define CPACR_EL1_TTA (1 << 28)
#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
+#define kvm_mode_names \
+ { PSR_MODE_EL0t, "EL0t" }, \
+ { PSR_MODE_EL1t, "EL1t" }, \
+ { PSR_MODE_EL1h, "EL1h" }, \
+ { PSR_MODE_EL2t, "EL2t" }, \
+ { PSR_MODE_EL2h, "EL2h" }, \
+ { PSR_MODE_EL3t, "EL3t" }, \
+ { PSR_MODE_EL3h, "EL3h" }, \
+ { PSR_AA32_MODE_USR, "32-bit USR" }, \
+ { PSR_AA32_MODE_FIQ, "32-bit FIQ" }, \
+ { PSR_AA32_MODE_IRQ, "32-bit IRQ" }, \
+ { PSR_AA32_MODE_SVC, "32-bit SVC" }, \
+ { PSR_AA32_MODE_ABT, "32-bit ABT" }, \
+ { PSR_AA32_MODE_HYP, "32-bit HYP" }, \
+ { PSR_AA32_MODE_UND, "32-bit UND" }, \
+ { PSR_AA32_MODE_SYS, "32-bit SYS" }
+
#endif /* __ARM64_KVM_ARM_H__ */
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index e3e22d364dc9..deffd3cc4443 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -33,6 +33,12 @@ enum exception_type {
except_type_serror = 0x180,
};
+#define kvm_exception_type_names \
+ { except_type_sync, "SYNC" }, \
+ { except_type_irq, "IRQ" }, \
+ { except_type_fiq, "FIQ" }, \
+ { except_type_serror, "SERROR" }
+
bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);
void kvm_skip_instr32(struct kvm_vcpu *vcpu);
@@ -41,6 +47,10 @@ void kvm_inject_vabt(struct kvm_vcpu *vcpu);
void kvm_inject_dabt(struct kvm_vcpu *vcpu, unsigned long addr);
void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
+void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu);
+int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2);
+int kvm_inject_nested_irq(struct kvm_vcpu *vcpu);
+
static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
{
return !(vcpu->arch.hcr_el2 & HCR_RW);
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 60fd181df624..672acdf1e993 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,11 +17,11 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
+ emulate-nested.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
vgic/vgic-its.o vgic/vgic-debug.o
-
kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
new file mode 100644
index 000000000000..ee91bcd925d8
--- /dev/null
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -0,0 +1,176 @@
+/*
+ * Copyright (C) 2016 - Linaro and Columbia University
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
+
+#include "hyp/include/hyp/adjust_pc.h"
+
+#include "trace.h"
+
+void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
+{
+ u64 spsr, elr, mode;
+ bool direct_eret;
+
+ /*
+ * Going through the whole put/load motions is a waste of time
+ * if this is a VHE guest hypervisor returning to its own
+ * userspace, or the hypervisor performing a local exception
+ * return. No need to save/restore registers, no need to
+ * switch S2 MMU. Just do the canonical ERET.
+ */
+ spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
+ mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ direct_eret = (mode == PSR_MODE_EL0t &&
+ vcpu_el2_e2h_is_set(vcpu) &&
+ vcpu_el2_tge_is_set(vcpu));
+ direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
+
+ if (direct_eret) {
+ *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
+ *vcpu_cpsr(vcpu) = spsr;
+ trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
+ return;
+ }
+
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+
+ elr = __vcpu_sys_reg(vcpu, ELR_EL2);
+
+ trace_kvm_nested_eret(vcpu, elr, spsr);
+
+ /*
+ * Note that the current exception level is always the virtual EL2,
+ * since we set HCR_EL2.NV bit only when entering the virtual EL2.
+ */
+ *vcpu_pc(vcpu) = elr;
+ *vcpu_cpsr(vcpu) = spsr;
+
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+}
+
+static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,
+ enum exception_type type)
+{
+ trace_kvm_inject_nested_exception(vcpu, esr_el2, type);
+
+ switch (type) {
+ case except_type_sync:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_SYNC;
+ break;
+ case except_type_irq:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_IRQ;
+ break;
+ default:
+ WARN_ONCE(1, "Unsupported EL2 exception injection %d\n", type);
+ }
+
+ vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL2 |
+ KVM_ARM64_PENDING_EXCEPTION);
+
+ vcpu_write_sys_reg(vcpu, esr_el2, ESR_EL2);
+}
+
+/*
+ * Emulate taking an exception to EL2.
+ * See ARM ARM J8.1.2 AArch64.TakeException()
+ */
+static int kvm_inject_nested(struct kvm_vcpu *vcpu, u64 esr_el2,
+ enum exception_type type)
+{
+ u64 pstate, mode;
+ bool direct_inject;
+
+ if (!nested_virt_in_use(vcpu)) {
+ kvm_err("Unexpected call to %s for the non-nesting configuration\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ /*
+ * As for ERET, we can avoid doing too much on the injection path by
+ * checking that we either took the exception from a VHE host
+ * userspace or from vEL2. In these cases, there is no change in
+ * translation regime (or anything else), so let's do as little as
+ * possible.
+ */
+ pstate = *vcpu_cpsr(vcpu);
+ mode = pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ direct_inject = (mode == PSR_MODE_EL0t &&
+ vcpu_el2_e2h_is_set(vcpu) &&
+ vcpu_el2_tge_is_set(vcpu));
+ direct_inject |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
+
+ if (direct_inject) {
+ kvm_inject_el2_exception(vcpu, esr_el2, type);
+ return 1;
+ }
+
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+
+ kvm_inject_el2_exception(vcpu, esr_el2, type);
+
+ /*
+ * A hard requirement is that a switch between EL1 and EL2
+ * contexts has to happen between a put/load, so that we can
+ * pick the correct timer and interrupt configuration, among
+ * other things.
+ *
+ * Make sure the exception actually took place before we load
+ * the new context.
+ */
+ __adjust_pc(vcpu);
+
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+
+ return 1;
+}
+
+int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2)
+{
+ return kvm_inject_nested(vcpu, esr_el2, except_type_sync);
+}
+
+int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)
+{
+ /*
+ * Do not inject an irq if the:
+ * - Current exception level is EL2, and
+ * - virtual HCR_EL2.TGE == 0
+ * - virtual HCR_EL2.IMO == 0
+ *
+ * See Table D1-17 "Physical interrupt target and masking when EL3 is
+ * not implemented and EL2 is implemented" in ARM DDI 0487C.a.
+ */
+
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_tge_is_set(vcpu) &&
+ !(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO))
+ return 1;
+
+ /* esr_el2 value doesn't matter for exits due to irqs. */
+ return kvm_inject_nested(vcpu, 0, except_type_irq);
+}
diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
index 73629094f903..7c8b3681dafd 100644
--- a/arch/arm64/kvm/hyp/exception.c
+++ b/arch/arm64/kvm/hyp/exception.c
@@ -22,7 +22,9 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
{
u64 val;
- if (__vcpu_read_sys_reg_from_cpu(reg, &val))
+ if (unlikely(nested_virt_in_use(vcpu)))
+ return vcpu_read_sys_reg(vcpu, reg);
+ else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
return val;
return __vcpu_sys_reg(vcpu, reg);
@@ -30,14 +32,26 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
{
- if (__vcpu_write_sys_reg_to_cpu(val, reg))
+ if (unlikely(nested_virt_in_use(vcpu)))
+ vcpu_write_sys_reg(vcpu, val, reg);
+ else if (__vcpu_write_sys_reg_to_cpu(val, reg))
return;
__vcpu_sys_reg(vcpu, reg) = val;
}
-static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, u64 val)
+static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode,
+ u64 val)
{
+ if (unlikely(nested_virt_in_use(vcpu))) {
+ if (target_mode == PSR_MODE_EL1h)
+ vcpu_write_sys_reg(vcpu, val, SPSR_EL1);
+ else
+ vcpu_write_sys_reg(vcpu, val, SPSR_EL2);
+
+ return;
+ }
+
write_sysreg_el1(val, SYS_SPSR);
}
@@ -97,6 +111,11 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL1);
__vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL1);
break;
+ case PSR_MODE_EL2h:
+ vbar = __vcpu_read_sys_reg(vcpu, VBAR_EL2);
+ sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL2);
+ __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL2);
+ break;
default:
/* Don't do that */
BUG();
@@ -148,7 +167,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
new |= target_mode;
*vcpu_cpsr(vcpu) = new;
- __vcpu_write_spsr(vcpu, old);
+ __vcpu_write_spsr(vcpu, target_mode, old);
}
/*
@@ -319,11 +338,22 @@ void kvm_inject_exception(struct kvm_vcpu *vcpu)
KVM_ARM64_EXCEPT_AA64_EL1):
enter_exception64(vcpu, PSR_MODE_EL1h, except_type_sync);
break;
+
+ case (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
+ KVM_ARM64_EXCEPT_AA64_EL2):
+ enter_exception64(vcpu, PSR_MODE_EL2h, except_type_sync);
+ break;
+
+ case (KVM_ARM64_EXCEPT_AA64_ELx_IRQ |
+ KVM_ARM64_EXCEPT_AA64_EL2):
+ enter_exception64(vcpu, PSR_MODE_EL2h, except_type_irq);
+ break;
+
default:
/*
- * Only EL1_SYNC makes sense so far, EL2_{SYNC,IRQ}
- * will be implemented at some point. Everything
- * else gets silently ignored.
+ * Only EL1_SYNC and EL2_{SYNC,IRQ} makes
+ * sense so far. Everything else gets silently
+ * ignored.
*/
break;
}
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index b47df73e98d7..530562eb09fd 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -14,17 +14,50 @@
#include <asm/kvm_emulate.h>
#include <asm/esr.h>
+static void pend_sync_exception(struct kvm_vcpu *vcpu)
+{
+ vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
+ KVM_ARM64_PENDING_EXCEPTION);
+
+ /* If not nesting, EL1 is the only possible exception target */
+ if (likely(!nested_virt_in_use(vcpu))) {
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ return;
+ }
+
+ /*
+ * With NV, we need to pick between EL1 and EL2. Note that we
+ * never deal with a nesting exception here, hence never
+ * changing context, and the exception itself can be delayed
+ * until the next entry.
+ */
+ switch(*vcpu_cpsr(vcpu) & PSR_MODE_MASK) {
+ case PSR_MODE_EL2h:
+ case PSR_MODE_EL2t:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
+ break;
+ case PSR_MODE_EL1h:
+ case PSR_MODE_EL1t:
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ break;
+ case PSR_MODE_EL0t:
+ if (vcpu_el2_tge_is_set(vcpu) & HCR_TGE)
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
+ else
+ vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
+ break;
+ default:
+ BUG();
+ }
+}
+
static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
{
unsigned long cpsr = *vcpu_cpsr(vcpu);
bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
u32 esr = 0;
- vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
- KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
- KVM_ARM64_PENDING_EXCEPTION);
-
- vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
+ pend_sync_exception(vcpu);
/*
* Build an {i,d}abort, depending on the level and the
@@ -45,16 +78,22 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
if (!is_iabt)
esr |= ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT;
- vcpu_write_sys_reg(vcpu, esr | ESR_ELx_FSC_EXTABT, ESR_EL1);
+ esr |= ESR_ELx_FSC_EXTABT;
+
+ if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1) {
+ vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ } else {
+ vcpu_write_sys_reg(vcpu, addr, FAR_EL2);
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
+ }
}
static void inject_undef64(struct kvm_vcpu *vcpu)
{
u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
- vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
- KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
- KVM_ARM64_PENDING_EXCEPTION);
+ pend_sync_exception(vcpu);
/*
* Build an unknown exception, depending on the instruction
@@ -63,7 +102,10 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
if (kvm_vcpu_trap_il_is32bit(vcpu))
esr |= ESR_ELx_IL;
- vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1)
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
+ else
+ vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
}
#define DFSR_FSC_EXTABT_LPAE 0x10
diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
index ff0444352bba..5707011c4f47 100644
--- a/arch/arm64/kvm/trace_arm.h
+++ b/arch/arm64/kvm/trace_arm.h
@@ -2,6 +2,7 @@
#if !defined(_TRACE_ARM_ARM64_KVM_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_ARM_ARM64_KVM_H
+#include <asm/kvm_emulate.h>
#include <kvm/arm_arch_timer.h>
#include <linux/tracepoint.h>
@@ -367,6 +368,64 @@ TRACE_EVENT(kvm_timer_emulate,
__entry->timer_idx, __entry->should_fire)
);
+TRACE_EVENT(kvm_nested_eret,
+ TP_PROTO(struct kvm_vcpu *vcpu, unsigned long elr_el2,
+ unsigned long spsr_el2),
+ TP_ARGS(vcpu, elr_el2, spsr_el2),
+
+ TP_STRUCT__entry(
+ __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, elr_el2)
+ __field(unsigned long, spsr_el2)
+ __field(unsigned long, target_mode)
+ __field(unsigned long, hcr_el2)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->elr_el2 = elr_el2;
+ __entry->spsr_el2 = spsr_el2;
+ __entry->target_mode = spsr_el2 & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+ ),
+
+ TP_printk("elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
+ __entry->elr_el2, __entry->spsr_el2,
+ __print_symbolic(__entry->target_mode, kvm_mode_names),
+ __entry->hcr_el2)
+);
+
+TRACE_EVENT(kvm_inject_nested_exception,
+ TP_PROTO(struct kvm_vcpu *vcpu, u64 esr_el2, int type),
+ TP_ARGS(vcpu, esr_el2, type),
+
+ TP_STRUCT__entry(
+ __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, esr_el2)
+ __field(int, type)
+ __field(unsigned long, spsr_el2)
+ __field(unsigned long, pc)
+ __field(unsigned long, source_mode)
+ __field(unsigned long, hcr_el2)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->esr_el2 = esr_el2;
+ __entry->type = type;
+ __entry->spsr_el2 = *vcpu_cpsr(vcpu);
+ __entry->pc = *vcpu_pc(vcpu);
+ __entry->source_mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+ ),
+
+ TP_printk("%s: esr_el2 0x%lx elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
+ __print_symbolic(__entry->type, kvm_exception_type_names),
+ __entry->esr_el2, __entry->pc, __entry->spsr_el2,
+ __print_symbolic(__entry->source_mode, kvm_mode_names),
+ __entry->hcr_el2)
+);
+
#endif /* _TRACE_ARM_ARM64_KVM_H */
#undef TRACE_INCLUDE_PATH
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 10/66] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
As we expect all PSCI calls from the L1 hypervisor to be performed
using SMC when nested virtualization is enabled, it is clear that
all HVC instruction from the VM (including from the virtual EL2)
are supposed to handled in the virtual EL2.
Forward these to EL2 as required.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: add handling of HCR_EL2.HCD]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/handle_exit.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index cebe39f3b1b6..5245ac818e4f 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -16,6 +16,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/debug-monitors.h>
#include <asm/traps.h>
@@ -40,6 +41,16 @@ static int handle_hvc(struct kvm_vcpu *vcpu)
kvm_vcpu_hvc_get_imm(vcpu));
vcpu->stat.hvc_exit_stat++;
+ /* Forward hvc instructions to the virtual EL2 if the guest has EL2. */
+ if (nested_virt_in_use(vcpu)) {
+ if (vcpu_read_sys_reg(vcpu, HCR_EL2) & HCR_HCD)
+ kvm_inject_undefined(vcpu);
+ else
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ return 1;
+ }
+
ret = kvm_hvc_call_handler(vcpu);
if (ret < 0) {
vcpu_set_reg(vcpu, 0, ~0UL);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 10/66] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
As we expect all PSCI calls from the L1 hypervisor to be performed
using SMC when nested virtualization is enabled, it is clear that
all HVC instruction from the VM (including from the virtual EL2)
are supposed to handled in the virtual EL2.
Forward these to EL2 as required.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: add handling of HCR_EL2.HCD]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/handle_exit.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index cebe39f3b1b6..5245ac818e4f 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -16,6 +16,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/debug-monitors.h>
#include <asm/traps.h>
@@ -40,6 +41,16 @@ static int handle_hvc(struct kvm_vcpu *vcpu)
kvm_vcpu_hvc_get_imm(vcpu));
vcpu->stat.hvc_exit_stat++;
+ /* Forward hvc instructions to the virtual EL2 if the guest has EL2. */
+ if (nested_virt_in_use(vcpu)) {
+ if (vcpu_read_sys_reg(vcpu, HCR_EL2) & HCR_HCD)
+ kvm_inject_undefined(vcpu);
+ else
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ return 1;
+ }
+
ret = kvm_hvc_call_handler(vcpu);
if (ret < 0) {
vcpu_set_reg(vcpu, 0, ~0UL);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 10/66] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
As we expect all PSCI calls from the L1 hypervisor to be performed
using SMC when nested virtualization is enabled, it is clear that
all HVC instruction from the VM (including from the virtual EL2)
are supposed to handled in the virtual EL2.
Forward these to EL2 as required.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: add handling of HCR_EL2.HCD]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/handle_exit.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index cebe39f3b1b6..5245ac818e4f 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -16,6 +16,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/debug-monitors.h>
#include <asm/traps.h>
@@ -40,6 +41,16 @@ static int handle_hvc(struct kvm_vcpu *vcpu)
kvm_vcpu_hvc_get_imm(vcpu));
vcpu->stat.hvc_exit_stat++;
+ /* Forward hvc instructions to the virtual EL2 if the guest has EL2. */
+ if (nested_virt_in_use(vcpu)) {
+ if (vcpu_read_sys_reg(vcpu, HCR_EL2) & HCR_HCD)
+ kvm_inject_undefined(vcpu);
+ else
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ return 1;
+ }
+
ret = kvm_hvc_call_handler(vcpu);
if (ret < 0) {
vcpu_set_reg(vcpu, 0, ~0UL);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 11/66] KVM: arm64: nv: Handle trapped ERET from virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
When a guest hypervisor running virtual EL2 in EL1 executes an ERET
instruction, we will have set HCR_EL2.NV which traps ERET to EL2, so
that we can emulate the exception return in software.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/esr.h | 5 +++++
arch/arm64/include/asm/kvm_arm.h | 2 +-
arch/arm64/kvm/handle_exit.c | 10 ++++++++++
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index 22c81f1edda2..ab63b7b3ea14 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -251,6 +251,11 @@
(((e) & ESR_ELx_SYS64_ISS_OP2_MASK) >> \
ESR_ELx_SYS64_ISS_OP2_SHIFT))
+/* ISS field definitions for ERET/ERETAA/ERETAB trapping */
+
+#define ESR_ELx_ERET_ISS_ERET_ERETAx 0x2
+#define ESR_ELx_ERET_ISS_ERETA_ERATAB 0x1
+
/*
* ISS field definitions for floating-point exception traps
* (FP_EXC_32/FP_EXC_64).
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index e9e10e498785..4cf745f79985 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -326,7 +326,7 @@
ECN(SP_ALIGN), ECN(FP_EXC32), ECN(FP_EXC64), ECN(SERROR), \
ECN(BREAKPT_LOW), ECN(BREAKPT_CUR), ECN(SOFTSTP_LOW), \
ECN(SOFTSTP_CUR), ECN(WATCHPT_LOW), ECN(WATCHPT_CUR), \
- ECN(BKPT32), ECN(VECTOR32), ECN(BRK64)
+ ECN(BKPT32), ECN(VECTOR32), ECN(BRK64), ECN(ERET)
#define CPACR_EL1_FPEN (3 << 20)
#define CPACR_EL1_TTA (1 << 28)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 5245ac818e4f..56f8a77b9d4a 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -183,6 +183,15 @@ static int kvm_handle_ptrauth(struct kvm_vcpu *vcpu)
return 1;
}
+static int kvm_handle_eret(struct kvm_vcpu *vcpu)
+{
+ if (kvm_vcpu_get_esr(vcpu) & ESR_ELx_ERET_ISS_ERET_ERETAx)
+ return kvm_handle_ptrauth(vcpu);
+
+ kvm_emulate_nested_eret(vcpu);
+ return 1;
+}
+
static exit_handle_fn arm_exit_handlers[] = {
[0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
[ESR_ELx_EC_WFx] = kvm_handle_wfx,
@@ -197,6 +206,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_SMC64] = handle_smc,
[ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
[ESR_ELx_EC_SVE] = handle_sve,
+ [ESR_ELx_EC_ERET] = kvm_handle_eret,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 11/66] KVM: arm64: nv: Handle trapped ERET from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
When a guest hypervisor running virtual EL2 in EL1 executes an ERET
instruction, we will have set HCR_EL2.NV which traps ERET to EL2, so
that we can emulate the exception return in software.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/esr.h | 5 +++++
arch/arm64/include/asm/kvm_arm.h | 2 +-
arch/arm64/kvm/handle_exit.c | 10 ++++++++++
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index 22c81f1edda2..ab63b7b3ea14 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -251,6 +251,11 @@
(((e) & ESR_ELx_SYS64_ISS_OP2_MASK) >> \
ESR_ELx_SYS64_ISS_OP2_SHIFT))
+/* ISS field definitions for ERET/ERETAA/ERETAB trapping */
+
+#define ESR_ELx_ERET_ISS_ERET_ERETAx 0x2
+#define ESR_ELx_ERET_ISS_ERETA_ERATAB 0x1
+
/*
* ISS field definitions for floating-point exception traps
* (FP_EXC_32/FP_EXC_64).
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index e9e10e498785..4cf745f79985 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -326,7 +326,7 @@
ECN(SP_ALIGN), ECN(FP_EXC32), ECN(FP_EXC64), ECN(SERROR), \
ECN(BREAKPT_LOW), ECN(BREAKPT_CUR), ECN(SOFTSTP_LOW), \
ECN(SOFTSTP_CUR), ECN(WATCHPT_LOW), ECN(WATCHPT_CUR), \
- ECN(BKPT32), ECN(VECTOR32), ECN(BRK64)
+ ECN(BKPT32), ECN(VECTOR32), ECN(BRK64), ECN(ERET)
#define CPACR_EL1_FPEN (3 << 20)
#define CPACR_EL1_TTA (1 << 28)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 5245ac818e4f..56f8a77b9d4a 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -183,6 +183,15 @@ static int kvm_handle_ptrauth(struct kvm_vcpu *vcpu)
return 1;
}
+static int kvm_handle_eret(struct kvm_vcpu *vcpu)
+{
+ if (kvm_vcpu_get_esr(vcpu) & ESR_ELx_ERET_ISS_ERET_ERETAx)
+ return kvm_handle_ptrauth(vcpu);
+
+ kvm_emulate_nested_eret(vcpu);
+ return 1;
+}
+
static exit_handle_fn arm_exit_handlers[] = {
[0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
[ESR_ELx_EC_WFx] = kvm_handle_wfx,
@@ -197,6 +206,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_SMC64] = handle_smc,
[ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
[ESR_ELx_EC_SVE] = handle_sve,
+ [ESR_ELx_EC_ERET] = kvm_handle_eret,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 11/66] KVM: arm64: nv: Handle trapped ERET from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
When a guest hypervisor running virtual EL2 in EL1 executes an ERET
instruction, we will have set HCR_EL2.NV which traps ERET to EL2, so
that we can emulate the exception return in software.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/esr.h | 5 +++++
arch/arm64/include/asm/kvm_arm.h | 2 +-
arch/arm64/kvm/handle_exit.c | 10 ++++++++++
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index 22c81f1edda2..ab63b7b3ea14 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -251,6 +251,11 @@
(((e) & ESR_ELx_SYS64_ISS_OP2_MASK) >> \
ESR_ELx_SYS64_ISS_OP2_SHIFT))
+/* ISS field definitions for ERET/ERETAA/ERETAB trapping */
+
+#define ESR_ELx_ERET_ISS_ERET_ERETAx 0x2
+#define ESR_ELx_ERET_ISS_ERETA_ERATAB 0x1
+
/*
* ISS field definitions for floating-point exception traps
* (FP_EXC_32/FP_EXC_64).
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index e9e10e498785..4cf745f79985 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -326,7 +326,7 @@
ECN(SP_ALIGN), ECN(FP_EXC32), ECN(FP_EXC64), ECN(SERROR), \
ECN(BREAKPT_LOW), ECN(BREAKPT_CUR), ECN(SOFTSTP_LOW), \
ECN(SOFTSTP_CUR), ECN(WATCHPT_LOW), ECN(WATCHPT_CUR), \
- ECN(BKPT32), ECN(VECTOR32), ECN(BRK64)
+ ECN(BKPT32), ECN(VECTOR32), ECN(BRK64), ECN(ERET)
#define CPACR_EL1_FPEN (3 << 20)
#define CPACR_EL1_TTA (1 << 28)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 5245ac818e4f..56f8a77b9d4a 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -183,6 +183,15 @@ static int kvm_handle_ptrauth(struct kvm_vcpu *vcpu)
return 1;
}
+static int kvm_handle_eret(struct kvm_vcpu *vcpu)
+{
+ if (kvm_vcpu_get_esr(vcpu) & ESR_ELx_ERET_ISS_ERET_ERETAx)
+ return kvm_handle_ptrauth(vcpu);
+
+ kvm_emulate_nested_eret(vcpu);
+ return 1;
+}
+
static exit_handle_fn arm_exit_handlers[] = {
[0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
[ESR_ELx_EC_WFx] = kvm_handle_wfx,
@@ -197,6 +206,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_SMC64] = handle_smc,
[ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
[ESR_ELx_EC_SVE] = handle_sve,
+ [ESR_ELx_EC_ERET] = kvm_handle_eret,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 12/66] KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Some EL2 system registers immediately affect the current execution
of the system, so we need to use their respective EL1 counterparts.
For this we need to define a mapping between the two. In general,
this only affects non-VHE guest hypervisors, as VHE system registers
are compatible with the EL1 counterparts.
These helpers will get used in subsequent patches.
Co-developed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 50 +++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 1028ac65a897..67a2c0d05233 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -2,6 +2,7 @@
#ifndef __ARM64_KVM_NESTED_H
#define __ARM64_KVM_NESTED_H
+#include <linux/bitfield.h>
#include <linux/kvm_host.h>
static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
@@ -11,4 +12,53 @@ static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
}
+/* Translation helpers from non-VHE EL2 to EL1 */
+static inline u64 tcr_el2_ips_to_tcr_el1_ps(u64 tcr_el2)
+{
+ return (u64)FIELD_GET(TCR_EL2_PS_MASK, tcr_el2) << TCR_IPS_SHIFT;
+}
+
+static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr)
+{
+ return TCR_EPD1_MASK | /* disable TTBR1_EL1 */
+ ((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) |
+ tcr_el2_ips_to_tcr_el1_ps(tcr) |
+ (tcr & TCR_EL2_TG0_MASK) |
+ (tcr & TCR_EL2_ORGN0_MASK) |
+ (tcr & TCR_EL2_IRGN0_MASK) |
+ (tcr & TCR_EL2_T0SZ_MASK);
+}
+
+static inline u64 translate_cptr_el2_to_cpacr_el1(u64 cptr_el2)
+{
+ u64 cpacr_el1 = 0;
+
+ if (!(cptr_el2 & CPTR_EL2_TFP))
+ cpacr_el1 |= CPACR_EL1_FPEN;
+ if (cptr_el2 & CPTR_EL2_TTA)
+ cpacr_el1 |= CPACR_EL1_TTA;
+ if (!(cptr_el2 & CPTR_EL2_TZ))
+ cpacr_el1 |= CPACR_EL1_ZEN;
+
+ return cpacr_el1;
+}
+
+static inline u64 translate_sctlr_el2_to_sctlr_el1(u64 sctlr)
+{
+ /* Bit 20 is RES1 in SCTLR_EL1, but RES0 in SCTLR_EL2 */
+ return sctlr | BIT(20);
+}
+
+static inline u64 translate_ttbr0_el2_to_ttbr0_el1(u64 ttbr0)
+{
+ /* Force ASID to 0 (ASID 0 or RES0) */
+ return ttbr0 & ~GENMASK_ULL(63, 48);
+}
+
+static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
+{
+ return ((FIELD_GET(CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN, cnthctl) << 10) |
+ (cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
+}
+
#endif /* __ARM64_KVM_NESTED_H */
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 12/66] KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Some EL2 system registers immediately affect the current execution
of the system, so we need to use their respective EL1 counterparts.
For this we need to define a mapping between the two. In general,
this only affects non-VHE guest hypervisors, as VHE system registers
are compatible with the EL1 counterparts.
These helpers will get used in subsequent patches.
Co-developed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 50 +++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 1028ac65a897..67a2c0d05233 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -2,6 +2,7 @@
#ifndef __ARM64_KVM_NESTED_H
#define __ARM64_KVM_NESTED_H
+#include <linux/bitfield.h>
#include <linux/kvm_host.h>
static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
@@ -11,4 +12,53 @@ static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
}
+/* Translation helpers from non-VHE EL2 to EL1 */
+static inline u64 tcr_el2_ips_to_tcr_el1_ps(u64 tcr_el2)
+{
+ return (u64)FIELD_GET(TCR_EL2_PS_MASK, tcr_el2) << TCR_IPS_SHIFT;
+}
+
+static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr)
+{
+ return TCR_EPD1_MASK | /* disable TTBR1_EL1 */
+ ((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) |
+ tcr_el2_ips_to_tcr_el1_ps(tcr) |
+ (tcr & TCR_EL2_TG0_MASK) |
+ (tcr & TCR_EL2_ORGN0_MASK) |
+ (tcr & TCR_EL2_IRGN0_MASK) |
+ (tcr & TCR_EL2_T0SZ_MASK);
+}
+
+static inline u64 translate_cptr_el2_to_cpacr_el1(u64 cptr_el2)
+{
+ u64 cpacr_el1 = 0;
+
+ if (!(cptr_el2 & CPTR_EL2_TFP))
+ cpacr_el1 |= CPACR_EL1_FPEN;
+ if (cptr_el2 & CPTR_EL2_TTA)
+ cpacr_el1 |= CPACR_EL1_TTA;
+ if (!(cptr_el2 & CPTR_EL2_TZ))
+ cpacr_el1 |= CPACR_EL1_ZEN;
+
+ return cpacr_el1;
+}
+
+static inline u64 translate_sctlr_el2_to_sctlr_el1(u64 sctlr)
+{
+ /* Bit 20 is RES1 in SCTLR_EL1, but RES0 in SCTLR_EL2 */
+ return sctlr | BIT(20);
+}
+
+static inline u64 translate_ttbr0_el2_to_ttbr0_el1(u64 ttbr0)
+{
+ /* Force ASID to 0 (ASID 0 or RES0) */
+ return ttbr0 & ~GENMASK_ULL(63, 48);
+}
+
+static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
+{
+ return ((FIELD_GET(CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN, cnthctl) << 10) |
+ (cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
+}
+
#endif /* __ARM64_KVM_NESTED_H */
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 12/66] KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Some EL2 system registers immediately affect the current execution
of the system, so we need to use their respective EL1 counterparts.
For this we need to define a mapping between the two. In general,
this only affects non-VHE guest hypervisors, as VHE system registers
are compatible with the EL1 counterparts.
These helpers will get used in subsequent patches.
Co-developed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 50 +++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 1028ac65a897..67a2c0d05233 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -2,6 +2,7 @@
#ifndef __ARM64_KVM_NESTED_H
#define __ARM64_KVM_NESTED_H
+#include <linux/bitfield.h>
#include <linux/kvm_host.h>
static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
@@ -11,4 +12,53 @@ static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
}
+/* Translation helpers from non-VHE EL2 to EL1 */
+static inline u64 tcr_el2_ips_to_tcr_el1_ps(u64 tcr_el2)
+{
+ return (u64)FIELD_GET(TCR_EL2_PS_MASK, tcr_el2) << TCR_IPS_SHIFT;
+}
+
+static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr)
+{
+ return TCR_EPD1_MASK | /* disable TTBR1_EL1 */
+ ((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) |
+ tcr_el2_ips_to_tcr_el1_ps(tcr) |
+ (tcr & TCR_EL2_TG0_MASK) |
+ (tcr & TCR_EL2_ORGN0_MASK) |
+ (tcr & TCR_EL2_IRGN0_MASK) |
+ (tcr & TCR_EL2_T0SZ_MASK);
+}
+
+static inline u64 translate_cptr_el2_to_cpacr_el1(u64 cptr_el2)
+{
+ u64 cpacr_el1 = 0;
+
+ if (!(cptr_el2 & CPTR_EL2_TFP))
+ cpacr_el1 |= CPACR_EL1_FPEN;
+ if (cptr_el2 & CPTR_EL2_TTA)
+ cpacr_el1 |= CPACR_EL1_TTA;
+ if (!(cptr_el2 & CPTR_EL2_TZ))
+ cpacr_el1 |= CPACR_EL1_ZEN;
+
+ return cpacr_el1;
+}
+
+static inline u64 translate_sctlr_el2_to_sctlr_el1(u64 sctlr)
+{
+ /* Bit 20 is RES1 in SCTLR_EL1, but RES0 in SCTLR_EL2 */
+ return sctlr | BIT(20);
+}
+
+static inline u64 translate_ttbr0_el2_to_ttbr0_el1(u64 ttbr0)
+{
+ /* Force ASID to 0 (ASID 0 or RES0) */
+ return ttbr0 & ~GENMASK_ULL(63, 48);
+}
+
+static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
+{
+ return ((FIELD_GET(CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN, cnthctl) << 10) |
+ (cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
+}
+
#endif /* __ARM64_KVM_NESTED_H */
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 13/66] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg()
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
KVM internally uses accessor functions when reading or writing the
guest's system registers. This takes care of accessing either the stored
copy or using the "live" EL1 system registers when the host uses VHE.
With the introduction of virtual EL2 we add a bunch of EL2 system
registers, which now must also be taken care of:
- If the guest is running in vEL2, and we access an EL1 sysreg, we must
revert to the stored version of that, and not use the CPU's copy.
- If the guest is running in vEL1, and we access an EL2 sysreg, we must
also use the stored version, since the CPU carries the EL1 copy.
- Some EL2 system registers are supposed to affect the current execution
of the system, so we need to put them into their respective EL1
counterparts. For this we need to define a mapping between the two.
This is done using the newly introduced struct el2_sysreg_map.
- Some EL2 system registers have a different format than their EL1
counterpart, so we need to translate them before writing them to the
CPU. This is done using an (optional) translate function in the map.
- There are the three special registers SP_EL2, SPSR_EL2 and ELR_EL2,
which need some separate handling (SPSR_EL2 is being handled in a
separate patch).
All of these cases are now wrapped into the existing accessor functions,
so KVM users wouldn't need to care whether they access EL2 or EL1
registers and also which state the guest is in.
This handles what was formerly known as the "shadow state" dynamically,
without requiring a separate copy for each vCPU EL.
Co-developed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 144 ++++++++++++++++++++++++++++++++++++--
1 file changed, 140 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index cc7a86624e36..d5a855cb65ca 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -23,6 +23,7 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/perf_event.h>
#include <asm/sysreg.h>
@@ -63,23 +64,158 @@ static bool write_to_read_only(struct kvm_vcpu *vcpu,
return false;
}
+#define PURE_EL2_SYSREG(el2) \
+ case el2: { \
+ *el1r = el2; \
+ return true; \
+ }
+
+#define MAPPED_EL2_SYSREG(el2, el1, fn) \
+ case el2: { \
+ *xlate = fn; \
+ *el1r = el1; \
+ return true; \
+ }
+
+static bool get_el2_mapping(unsigned int reg,
+ unsigned int *el1r, u64 (**xlate)(u64))
+{
+ switch (reg) {
+ PURE_EL2_SYSREG( VPIDR_EL2 );
+ PURE_EL2_SYSREG( VMPIDR_EL2 );
+ PURE_EL2_SYSREG( ACTLR_EL2 );
+ PURE_EL2_SYSREG( HCR_EL2 );
+ PURE_EL2_SYSREG( MDCR_EL2 );
+ PURE_EL2_SYSREG( HSTR_EL2 );
+ PURE_EL2_SYSREG( HACR_EL2 );
+ PURE_EL2_SYSREG( VTTBR_EL2 );
+ PURE_EL2_SYSREG( VTCR_EL2 );
+ PURE_EL2_SYSREG( RVBAR_EL2 );
+ PURE_EL2_SYSREG( RMR_EL2 );
+ PURE_EL2_SYSREG( TPIDR_EL2 );
+ PURE_EL2_SYSREG( HPFAR_EL2 );
+ PURE_EL2_SYSREG( ELR_EL2 );
+ PURE_EL2_SYSREG( SPSR_EL2 );
+ MAPPED_EL2_SYSREG(SCTLR_EL2, SCTLR_EL1,
+ translate_sctlr_el2_to_sctlr_el1 );
+ MAPPED_EL2_SYSREG(CPTR_EL2, CPACR_EL1,
+ translate_cptr_el2_to_cpacr_el1 );
+ MAPPED_EL2_SYSREG(TTBR0_EL2, TTBR0_EL1,
+ translate_ttbr0_el2_to_ttbr0_el1 );
+ MAPPED_EL2_SYSREG(TTBR1_EL2, TTBR1_EL1, NULL );
+ MAPPED_EL2_SYSREG(TCR_EL2, TCR_EL1,
+ translate_tcr_el2_to_tcr_el1 );
+ MAPPED_EL2_SYSREG(VBAR_EL2, VBAR_EL1, NULL );
+ MAPPED_EL2_SYSREG(AFSR0_EL2, AFSR0_EL1, NULL );
+ MAPPED_EL2_SYSREG(AFSR1_EL2, AFSR1_EL1, NULL );
+ MAPPED_EL2_SYSREG(ESR_EL2, ESR_EL1, NULL );
+ MAPPED_EL2_SYSREG(FAR_EL2, FAR_EL1, NULL );
+ MAPPED_EL2_SYSREG(MAIR_EL2, MAIR_EL1, NULL );
+ MAPPED_EL2_SYSREG(AMAIR_EL2, AMAIR_EL1, NULL );
+ MAPPED_EL2_SYSREG(CNTHCTL_EL2, CNTKCTL_EL1,
+ translate_cnthctl_el2_to_cntkctl_el1 );
+ default:
+ return false;
+ }
+}
+
u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
{
u64 val = 0x8badf00d8badf00d;
+ u64 (*xlate)(u64) = NULL;
+ unsigned int el1r;
+
+ if (!vcpu->arch.sysregs_loaded_on_cpu)
+ goto memory_read;
+
+ if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ if (!is_hyp_ctxt(vcpu))
+ goto memory_read;
+
+ /*
+ * ELR_EL2 is special cased for now.
+ */
+ switch (reg) {
+ case ELR_EL2:
+ return read_sysreg_el1(SYS_ELR);
+ }
+
+ /*
+ * If this register does not have an EL1 counterpart,
+ * then read the stored EL2 version.
+ */
+ if (reg == el1r)
+ goto memory_read;
+
+ /*
+ * If we have a non-VHE guest and that the sysreg
+ * requires translation to be used at EL1, use the
+ * in-memory copy instead.
+ */
+ if (!vcpu_el2_e2h_is_set(vcpu) && xlate)
+ goto memory_read;
+
+ /* Get the current version of the EL1 counterpart. */
+ WARN_ON(!__vcpu_read_sys_reg_from_cpu(el1r, &val));
+ return val;
+ }
+
+ /* EL1 register can't be on the CPU if the guest is in vEL2. */
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ goto memory_read;
- if (vcpu->arch.sysregs_loaded_on_cpu &&
- __vcpu_read_sys_reg_from_cpu(reg, &val))
+ if (__vcpu_read_sys_reg_from_cpu(reg, &val))
return val;
+memory_read:
return __vcpu_sys_reg(vcpu, reg);
}
void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
{
- if (vcpu->arch.sysregs_loaded_on_cpu &&
- __vcpu_write_sys_reg_to_cpu(val, reg))
+ u64 (*xlate)(u64) = NULL;
+ unsigned int el1r;
+
+ if (!vcpu->arch.sysregs_loaded_on_cpu)
+ goto memory_write;
+
+ if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ if (!is_hyp_ctxt(vcpu))
+ goto memory_write;
+
+ /*
+ * Always store a copy of the write to memory to avoid having
+ * to reverse-translate virtual EL2 system registers for a
+ * non-VHE guest hypervisor.
+ */
+ __vcpu_sys_reg(vcpu, reg) = val;
+
+ switch (reg) {
+ case ELR_EL2:
+ write_sysreg_el1(val, SYS_ELR);
+ return;
+ }
+
+ /* No EL1 counterpart? We're done here.? */
+ if (reg == el1r)
+ return;
+
+ if (!vcpu_el2_e2h_is_set(vcpu) && xlate)
+ val = xlate(val);
+
+ /* Redirect this to the EL1 version of the register. */
+ WARN_ON(!__vcpu_write_sys_reg_to_cpu(val, el1r));
+ return;
+ }
+
+ /* EL1 register can't be on the CPU if the guest is in vEL2. */
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ goto memory_write;
+
+ if (__vcpu_write_sys_reg_to_cpu(val, reg))
return;
+memory_write:
__vcpu_sys_reg(vcpu, reg) = val;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 13/66] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg()
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
KVM internally uses accessor functions when reading or writing the
guest's system registers. This takes care of accessing either the stored
copy or using the "live" EL1 system registers when the host uses VHE.
With the introduction of virtual EL2 we add a bunch of EL2 system
registers, which now must also be taken care of:
- If the guest is running in vEL2, and we access an EL1 sysreg, we must
revert to the stored version of that, and not use the CPU's copy.
- If the guest is running in vEL1, and we access an EL2 sysreg, we must
also use the stored version, since the CPU carries the EL1 copy.
- Some EL2 system registers are supposed to affect the current execution
of the system, so we need to put them into their respective EL1
counterparts. For this we need to define a mapping between the two.
This is done using the newly introduced struct el2_sysreg_map.
- Some EL2 system registers have a different format than their EL1
counterpart, so we need to translate them before writing them to the
CPU. This is done using an (optional) translate function in the map.
- There are the three special registers SP_EL2, SPSR_EL2 and ELR_EL2,
which need some separate handling (SPSR_EL2 is being handled in a
separate patch).
All of these cases are now wrapped into the existing accessor functions,
so KVM users wouldn't need to care whether they access EL2 or EL1
registers and also which state the guest is in.
This handles what was formerly known as the "shadow state" dynamically,
without requiring a separate copy for each vCPU EL.
Co-developed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 144 ++++++++++++++++++++++++++++++++++++--
1 file changed, 140 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index cc7a86624e36..d5a855cb65ca 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -23,6 +23,7 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/perf_event.h>
#include <asm/sysreg.h>
@@ -63,23 +64,158 @@ static bool write_to_read_only(struct kvm_vcpu *vcpu,
return false;
}
+#define PURE_EL2_SYSREG(el2) \
+ case el2: { \
+ *el1r = el2; \
+ return true; \
+ }
+
+#define MAPPED_EL2_SYSREG(el2, el1, fn) \
+ case el2: { \
+ *xlate = fn; \
+ *el1r = el1; \
+ return true; \
+ }
+
+static bool get_el2_mapping(unsigned int reg,
+ unsigned int *el1r, u64 (**xlate)(u64))
+{
+ switch (reg) {
+ PURE_EL2_SYSREG( VPIDR_EL2 );
+ PURE_EL2_SYSREG( VMPIDR_EL2 );
+ PURE_EL2_SYSREG( ACTLR_EL2 );
+ PURE_EL2_SYSREG( HCR_EL2 );
+ PURE_EL2_SYSREG( MDCR_EL2 );
+ PURE_EL2_SYSREG( HSTR_EL2 );
+ PURE_EL2_SYSREG( HACR_EL2 );
+ PURE_EL2_SYSREG( VTTBR_EL2 );
+ PURE_EL2_SYSREG( VTCR_EL2 );
+ PURE_EL2_SYSREG( RVBAR_EL2 );
+ PURE_EL2_SYSREG( RMR_EL2 );
+ PURE_EL2_SYSREG( TPIDR_EL2 );
+ PURE_EL2_SYSREG( HPFAR_EL2 );
+ PURE_EL2_SYSREG( ELR_EL2 );
+ PURE_EL2_SYSREG( SPSR_EL2 );
+ MAPPED_EL2_SYSREG(SCTLR_EL2, SCTLR_EL1,
+ translate_sctlr_el2_to_sctlr_el1 );
+ MAPPED_EL2_SYSREG(CPTR_EL2, CPACR_EL1,
+ translate_cptr_el2_to_cpacr_el1 );
+ MAPPED_EL2_SYSREG(TTBR0_EL2, TTBR0_EL1,
+ translate_ttbr0_el2_to_ttbr0_el1 );
+ MAPPED_EL2_SYSREG(TTBR1_EL2, TTBR1_EL1, NULL );
+ MAPPED_EL2_SYSREG(TCR_EL2, TCR_EL1,
+ translate_tcr_el2_to_tcr_el1 );
+ MAPPED_EL2_SYSREG(VBAR_EL2, VBAR_EL1, NULL );
+ MAPPED_EL2_SYSREG(AFSR0_EL2, AFSR0_EL1, NULL );
+ MAPPED_EL2_SYSREG(AFSR1_EL2, AFSR1_EL1, NULL );
+ MAPPED_EL2_SYSREG(ESR_EL2, ESR_EL1, NULL );
+ MAPPED_EL2_SYSREG(FAR_EL2, FAR_EL1, NULL );
+ MAPPED_EL2_SYSREG(MAIR_EL2, MAIR_EL1, NULL );
+ MAPPED_EL2_SYSREG(AMAIR_EL2, AMAIR_EL1, NULL );
+ MAPPED_EL2_SYSREG(CNTHCTL_EL2, CNTKCTL_EL1,
+ translate_cnthctl_el2_to_cntkctl_el1 );
+ default:
+ return false;
+ }
+}
+
u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
{
u64 val = 0x8badf00d8badf00d;
+ u64 (*xlate)(u64) = NULL;
+ unsigned int el1r;
+
+ if (!vcpu->arch.sysregs_loaded_on_cpu)
+ goto memory_read;
+
+ if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ if (!is_hyp_ctxt(vcpu))
+ goto memory_read;
+
+ /*
+ * ELR_EL2 is special cased for now.
+ */
+ switch (reg) {
+ case ELR_EL2:
+ return read_sysreg_el1(SYS_ELR);
+ }
+
+ /*
+ * If this register does not have an EL1 counterpart,
+ * then read the stored EL2 version.
+ */
+ if (reg == el1r)
+ goto memory_read;
+
+ /*
+ * If we have a non-VHE guest and that the sysreg
+ * requires translation to be used at EL1, use the
+ * in-memory copy instead.
+ */
+ if (!vcpu_el2_e2h_is_set(vcpu) && xlate)
+ goto memory_read;
+
+ /* Get the current version of the EL1 counterpart. */
+ WARN_ON(!__vcpu_read_sys_reg_from_cpu(el1r, &val));
+ return val;
+ }
+
+ /* EL1 register can't be on the CPU if the guest is in vEL2. */
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ goto memory_read;
- if (vcpu->arch.sysregs_loaded_on_cpu &&
- __vcpu_read_sys_reg_from_cpu(reg, &val))
+ if (__vcpu_read_sys_reg_from_cpu(reg, &val))
return val;
+memory_read:
return __vcpu_sys_reg(vcpu, reg);
}
void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
{
- if (vcpu->arch.sysregs_loaded_on_cpu &&
- __vcpu_write_sys_reg_to_cpu(val, reg))
+ u64 (*xlate)(u64) = NULL;
+ unsigned int el1r;
+
+ if (!vcpu->arch.sysregs_loaded_on_cpu)
+ goto memory_write;
+
+ if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ if (!is_hyp_ctxt(vcpu))
+ goto memory_write;
+
+ /*
+ * Always store a copy of the write to memory to avoid having
+ * to reverse-translate virtual EL2 system registers for a
+ * non-VHE guest hypervisor.
+ */
+ __vcpu_sys_reg(vcpu, reg) = val;
+
+ switch (reg) {
+ case ELR_EL2:
+ write_sysreg_el1(val, SYS_ELR);
+ return;
+ }
+
+ /* No EL1 counterpart? We're done here.? */
+ if (reg == el1r)
+ return;
+
+ if (!vcpu_el2_e2h_is_set(vcpu) && xlate)
+ val = xlate(val);
+
+ /* Redirect this to the EL1 version of the register. */
+ WARN_ON(!__vcpu_write_sys_reg_to_cpu(val, el1r));
+ return;
+ }
+
+ /* EL1 register can't be on the CPU if the guest is in vEL2. */
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ goto memory_write;
+
+ if (__vcpu_write_sys_reg_to_cpu(val, reg))
return;
+memory_write:
__vcpu_sys_reg(vcpu, reg) = val;
}
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 13/66] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg()
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
KVM internally uses accessor functions when reading or writing the
guest's system registers. This takes care of accessing either the stored
copy or using the "live" EL1 system registers when the host uses VHE.
With the introduction of virtual EL2 we add a bunch of EL2 system
registers, which now must also be taken care of:
- If the guest is running in vEL2, and we access an EL1 sysreg, we must
revert to the stored version of that, and not use the CPU's copy.
- If the guest is running in vEL1, and we access an EL2 sysreg, we must
also use the stored version, since the CPU carries the EL1 copy.
- Some EL2 system registers are supposed to affect the current execution
of the system, so we need to put them into their respective EL1
counterparts. For this we need to define a mapping between the two.
This is done using the newly introduced struct el2_sysreg_map.
- Some EL2 system registers have a different format than their EL1
counterpart, so we need to translate them before writing them to the
CPU. This is done using an (optional) translate function in the map.
- There are the three special registers SP_EL2, SPSR_EL2 and ELR_EL2,
which need some separate handling (SPSR_EL2 is being handled in a
separate patch).
All of these cases are now wrapped into the existing accessor functions,
so KVM users wouldn't need to care whether they access EL2 or EL1
registers and also which state the guest is in.
This handles what was formerly known as the "shadow state" dynamically,
without requiring a separate copy for each vCPU EL.
Co-developed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 144 ++++++++++++++++++++++++++++++++++++--
1 file changed, 140 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index cc7a86624e36..d5a855cb65ca 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -23,6 +23,7 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/perf_event.h>
#include <asm/sysreg.h>
@@ -63,23 +64,158 @@ static bool write_to_read_only(struct kvm_vcpu *vcpu,
return false;
}
+#define PURE_EL2_SYSREG(el2) \
+ case el2: { \
+ *el1r = el2; \
+ return true; \
+ }
+
+#define MAPPED_EL2_SYSREG(el2, el1, fn) \
+ case el2: { \
+ *xlate = fn; \
+ *el1r = el1; \
+ return true; \
+ }
+
+static bool get_el2_mapping(unsigned int reg,
+ unsigned int *el1r, u64 (**xlate)(u64))
+{
+ switch (reg) {
+ PURE_EL2_SYSREG( VPIDR_EL2 );
+ PURE_EL2_SYSREG( VMPIDR_EL2 );
+ PURE_EL2_SYSREG( ACTLR_EL2 );
+ PURE_EL2_SYSREG( HCR_EL2 );
+ PURE_EL2_SYSREG( MDCR_EL2 );
+ PURE_EL2_SYSREG( HSTR_EL2 );
+ PURE_EL2_SYSREG( HACR_EL2 );
+ PURE_EL2_SYSREG( VTTBR_EL2 );
+ PURE_EL2_SYSREG( VTCR_EL2 );
+ PURE_EL2_SYSREG( RVBAR_EL2 );
+ PURE_EL2_SYSREG( RMR_EL2 );
+ PURE_EL2_SYSREG( TPIDR_EL2 );
+ PURE_EL2_SYSREG( HPFAR_EL2 );
+ PURE_EL2_SYSREG( ELR_EL2 );
+ PURE_EL2_SYSREG( SPSR_EL2 );
+ MAPPED_EL2_SYSREG(SCTLR_EL2, SCTLR_EL1,
+ translate_sctlr_el2_to_sctlr_el1 );
+ MAPPED_EL2_SYSREG(CPTR_EL2, CPACR_EL1,
+ translate_cptr_el2_to_cpacr_el1 );
+ MAPPED_EL2_SYSREG(TTBR0_EL2, TTBR0_EL1,
+ translate_ttbr0_el2_to_ttbr0_el1 );
+ MAPPED_EL2_SYSREG(TTBR1_EL2, TTBR1_EL1, NULL );
+ MAPPED_EL2_SYSREG(TCR_EL2, TCR_EL1,
+ translate_tcr_el2_to_tcr_el1 );
+ MAPPED_EL2_SYSREG(VBAR_EL2, VBAR_EL1, NULL );
+ MAPPED_EL2_SYSREG(AFSR0_EL2, AFSR0_EL1, NULL );
+ MAPPED_EL2_SYSREG(AFSR1_EL2, AFSR1_EL1, NULL );
+ MAPPED_EL2_SYSREG(ESR_EL2, ESR_EL1, NULL );
+ MAPPED_EL2_SYSREG(FAR_EL2, FAR_EL1, NULL );
+ MAPPED_EL2_SYSREG(MAIR_EL2, MAIR_EL1, NULL );
+ MAPPED_EL2_SYSREG(AMAIR_EL2, AMAIR_EL1, NULL );
+ MAPPED_EL2_SYSREG(CNTHCTL_EL2, CNTKCTL_EL1,
+ translate_cnthctl_el2_to_cntkctl_el1 );
+ default:
+ return false;
+ }
+}
+
u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
{
u64 val = 0x8badf00d8badf00d;
+ u64 (*xlate)(u64) = NULL;
+ unsigned int el1r;
+
+ if (!vcpu->arch.sysregs_loaded_on_cpu)
+ goto memory_read;
+
+ if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ if (!is_hyp_ctxt(vcpu))
+ goto memory_read;
+
+ /*
+ * ELR_EL2 is special cased for now.
+ */
+ switch (reg) {
+ case ELR_EL2:
+ return read_sysreg_el1(SYS_ELR);
+ }
+
+ /*
+ * If this register does not have an EL1 counterpart,
+ * then read the stored EL2 version.
+ */
+ if (reg == el1r)
+ goto memory_read;
+
+ /*
+ * If we have a non-VHE guest and that the sysreg
+ * requires translation to be used at EL1, use the
+ * in-memory copy instead.
+ */
+ if (!vcpu_el2_e2h_is_set(vcpu) && xlate)
+ goto memory_read;
+
+ /* Get the current version of the EL1 counterpart. */
+ WARN_ON(!__vcpu_read_sys_reg_from_cpu(el1r, &val));
+ return val;
+ }
+
+ /* EL1 register can't be on the CPU if the guest is in vEL2. */
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ goto memory_read;
- if (vcpu->arch.sysregs_loaded_on_cpu &&
- __vcpu_read_sys_reg_from_cpu(reg, &val))
+ if (__vcpu_read_sys_reg_from_cpu(reg, &val))
return val;
+memory_read:
return __vcpu_sys_reg(vcpu, reg);
}
void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
{
- if (vcpu->arch.sysregs_loaded_on_cpu &&
- __vcpu_write_sys_reg_to_cpu(val, reg))
+ u64 (*xlate)(u64) = NULL;
+ unsigned int el1r;
+
+ if (!vcpu->arch.sysregs_loaded_on_cpu)
+ goto memory_write;
+
+ if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ if (!is_hyp_ctxt(vcpu))
+ goto memory_write;
+
+ /*
+ * Always store a copy of the write to memory to avoid having
+ * to reverse-translate virtual EL2 system registers for a
+ * non-VHE guest hypervisor.
+ */
+ __vcpu_sys_reg(vcpu, reg) = val;
+
+ switch (reg) {
+ case ELR_EL2:
+ write_sysreg_el1(val, SYS_ELR);
+ return;
+ }
+
+ /* No EL1 counterpart? We're done here.? */
+ if (reg == el1r)
+ return;
+
+ if (!vcpu_el2_e2h_is_set(vcpu) && xlate)
+ val = xlate(val);
+
+ /* Redirect this to the EL1 version of the register. */
+ WARN_ON(!__vcpu_write_sys_reg_to_cpu(val, el1r));
+ return;
+ }
+
+ /* EL1 register can't be on the CPU if the guest is in vEL2. */
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ goto memory_write;
+
+ if (__vcpu_write_sys_reg_to_cpu(val, reg))
return;
+memory_write:
__vcpu_sys_reg(vcpu, reg) = val;
}
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 14/66] KVM: arm64: nv: Handle SPSR_EL2 specially
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
SPSR_EL2 needs special attention when running nested on ARMv8.3:
If taking an exception while running at vEL2 (actually EL1), the
HW will update the SPSR_EL1 register with the EL1 mode. We need
to track this in order to make sure that accesses to the virtual
view of SPSR_EL2 is correct.
To do so, we place an illegal value in SPSR_EL1.M, and patch it
accordingly if required when accessing it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 37 ++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 23 +++++++++++++++--
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index deffd3cc4443..5c7dc659e8c4 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -238,6 +238,43 @@ static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
return __is_hyp_ctxt(&vcpu->arch.ctxt);
}
+static inline u64 __fixup_spsr_el2_write(struct kvm_cpu_context *ctxt, u64 val)
+{
+ if (!__vcpu_el2_e2h_is_set(ctxt)) {
+ /*
+ * Clear the .M field when writing SPSR to the CPU, so that we
+ * can detect when the CPU clobbered our SPSR copy during a
+ * local exception.
+ */
+ val &= ~0xc;
+ }
+
+ return val;
+}
+
+static inline u64 __fixup_spsr_el2_read(const struct kvm_cpu_context *ctxt, u64 val)
+{
+ if (__vcpu_el2_e2h_is_set(ctxt))
+ return val;
+
+ /*
+ * SPSR.M == 0 means the CPU has not touched the SPSR, so the
+ * register has still the value we saved on the last write.
+ */
+ if ((val & 0xc) == 0)
+ return ctxt_sys_reg(ctxt, SPSR_EL2);
+
+ /*
+ * Otherwise there was a "local" exception on the CPU,
+ * which from the guest's point of view was being taken from
+ * EL2 to EL2, although it actually happened to be from
+ * EL1 to EL1.
+ * So we need to fix the .M field in SPSR, to make it look
+ * like EL2, which is what the guest would expect.
+ */
+ return (val & ~0x0c) | CurrentEL_EL2;
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d5a855cb65ca..77a2d452b79d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -133,11 +133,14 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
goto memory_read;
/*
- * ELR_EL2 is special cased for now.
+ * ELR_EL2 and SPSR_EL2 are special cased for now.
*/
switch (reg) {
case ELR_EL2:
return read_sysreg_el1(SYS_ELR);
+ case SPSR_EL2:
+ val = read_sysreg_el1(SYS_SPSR);
+ return __fixup_spsr_el2_read(&vcpu->arch.ctxt, val);
}
/*
@@ -194,6 +197,10 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
case ELR_EL2:
write_sysreg_el1(val, SYS_ELR);
return;
+ case SPSR_EL2:
+ val = __fixup_spsr_el2_write(&vcpu->arch.ctxt, val);
+ write_sysreg_el1(val, SYS_SPSR);
+ return;
}
/* No EL1 counterpart? We're done here.? */
@@ -1527,6 +1534,18 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_spsr_el2(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, SPSR_EL2);
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1923,7 +1942,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
{ SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
- { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
+ { SYS_DESC(SYS_SPSR_EL2), access_spsr_el2, reset_val, SPSR_EL2, 0 },
{ SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
{ SYS_DESC(SYS_SP_EL1), access_sp_el1},
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 14/66] KVM: arm64: nv: Handle SPSR_EL2 specially
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
SPSR_EL2 needs special attention when running nested on ARMv8.3:
If taking an exception while running at vEL2 (actually EL1), the
HW will update the SPSR_EL1 register with the EL1 mode. We need
to track this in order to make sure that accesses to the virtual
view of SPSR_EL2 is correct.
To do so, we place an illegal value in SPSR_EL1.M, and patch it
accordingly if required when accessing it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 37 ++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 23 +++++++++++++++--
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index deffd3cc4443..5c7dc659e8c4 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -238,6 +238,43 @@ static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
return __is_hyp_ctxt(&vcpu->arch.ctxt);
}
+static inline u64 __fixup_spsr_el2_write(struct kvm_cpu_context *ctxt, u64 val)
+{
+ if (!__vcpu_el2_e2h_is_set(ctxt)) {
+ /*
+ * Clear the .M field when writing SPSR to the CPU, so that we
+ * can detect when the CPU clobbered our SPSR copy during a
+ * local exception.
+ */
+ val &= ~0xc;
+ }
+
+ return val;
+}
+
+static inline u64 __fixup_spsr_el2_read(const struct kvm_cpu_context *ctxt, u64 val)
+{
+ if (__vcpu_el2_e2h_is_set(ctxt))
+ return val;
+
+ /*
+ * SPSR.M == 0 means the CPU has not touched the SPSR, so the
+ * register has still the value we saved on the last write.
+ */
+ if ((val & 0xc) == 0)
+ return ctxt_sys_reg(ctxt, SPSR_EL2);
+
+ /*
+ * Otherwise there was a "local" exception on the CPU,
+ * which from the guest's point of view was being taken from
+ * EL2 to EL2, although it actually happened to be from
+ * EL1 to EL1.
+ * So we need to fix the .M field in SPSR, to make it look
+ * like EL2, which is what the guest would expect.
+ */
+ return (val & ~0x0c) | CurrentEL_EL2;
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d5a855cb65ca..77a2d452b79d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -133,11 +133,14 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
goto memory_read;
/*
- * ELR_EL2 is special cased for now.
+ * ELR_EL2 and SPSR_EL2 are special cased for now.
*/
switch (reg) {
case ELR_EL2:
return read_sysreg_el1(SYS_ELR);
+ case SPSR_EL2:
+ val = read_sysreg_el1(SYS_SPSR);
+ return __fixup_spsr_el2_read(&vcpu->arch.ctxt, val);
}
/*
@@ -194,6 +197,10 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
case ELR_EL2:
write_sysreg_el1(val, SYS_ELR);
return;
+ case SPSR_EL2:
+ val = __fixup_spsr_el2_write(&vcpu->arch.ctxt, val);
+ write_sysreg_el1(val, SYS_SPSR);
+ return;
}
/* No EL1 counterpart? We're done here.? */
@@ -1527,6 +1534,18 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_spsr_el2(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, SPSR_EL2);
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1923,7 +1942,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
{ SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
- { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
+ { SYS_DESC(SYS_SPSR_EL2), access_spsr_el2, reset_val, SPSR_EL2, 0 },
{ SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
{ SYS_DESC(SYS_SP_EL1), access_sp_el1},
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 14/66] KVM: arm64: nv: Handle SPSR_EL2 specially
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
SPSR_EL2 needs special attention when running nested on ARMv8.3:
If taking an exception while running at vEL2 (actually EL1), the
HW will update the SPSR_EL1 register with the EL1 mode. We need
to track this in order to make sure that accesses to the virtual
view of SPSR_EL2 is correct.
To do so, we place an illegal value in SPSR_EL1.M, and patch it
accordingly if required when accessing it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 37 ++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 23 +++++++++++++++--
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index deffd3cc4443..5c7dc659e8c4 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -238,6 +238,43 @@ static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
return __is_hyp_ctxt(&vcpu->arch.ctxt);
}
+static inline u64 __fixup_spsr_el2_write(struct kvm_cpu_context *ctxt, u64 val)
+{
+ if (!__vcpu_el2_e2h_is_set(ctxt)) {
+ /*
+ * Clear the .M field when writing SPSR to the CPU, so that we
+ * can detect when the CPU clobbered our SPSR copy during a
+ * local exception.
+ */
+ val &= ~0xc;
+ }
+
+ return val;
+}
+
+static inline u64 __fixup_spsr_el2_read(const struct kvm_cpu_context *ctxt, u64 val)
+{
+ if (__vcpu_el2_e2h_is_set(ctxt))
+ return val;
+
+ /*
+ * SPSR.M == 0 means the CPU has not touched the SPSR, so the
+ * register has still the value we saved on the last write.
+ */
+ if ((val & 0xc) == 0)
+ return ctxt_sys_reg(ctxt, SPSR_EL2);
+
+ /*
+ * Otherwise there was a "local" exception on the CPU,
+ * which from the guest's point of view was being taken from
+ * EL2 to EL2, although it actually happened to be from
+ * EL1 to EL1.
+ * So we need to fix the .M field in SPSR, to make it look
+ * like EL2, which is what the guest would expect.
+ */
+ return (val & ~0x0c) | CurrentEL_EL2;
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d5a855cb65ca..77a2d452b79d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -133,11 +133,14 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
goto memory_read;
/*
- * ELR_EL2 is special cased for now.
+ * ELR_EL2 and SPSR_EL2 are special cased for now.
*/
switch (reg) {
case ELR_EL2:
return read_sysreg_el1(SYS_ELR);
+ case SPSR_EL2:
+ val = read_sysreg_el1(SYS_SPSR);
+ return __fixup_spsr_el2_read(&vcpu->arch.ctxt, val);
}
/*
@@ -194,6 +197,10 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
case ELR_EL2:
write_sysreg_el1(val, SYS_ELR);
return;
+ case SPSR_EL2:
+ val = __fixup_spsr_el2_write(&vcpu->arch.ctxt, val);
+ write_sysreg_el1(val, SYS_SPSR);
+ return;
}
/* No EL1 counterpart? We're done here.? */
@@ -1527,6 +1534,18 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_spsr_el2(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, SPSR_EL2);
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1923,7 +1942,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
{ SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
- { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
+ { SYS_DESC(SYS_SPSR_EL2), access_spsr_el2, reset_val, SPSR_EL2, 0 },
{ SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
{ SYS_DESC(SYS_SP_EL1), access_sp_el1},
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 15/66] KVM: arm64: nv: Handle HCR_EL2.E2H specially
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
HCR_EL2.E2H is nasty, as a flip of this bit completely changes the way
we deal with a lot of the state. So when the guest flips this bit
(sysregs are live), do the put/load dance so that we have a consistent
state.
Yes, this is slow. Don't do it.
Suggested-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 77a2d452b79d..e7db4d809674 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -183,9 +183,24 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
goto memory_write;
if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ bool need_put_load;
+
if (!is_hyp_ctxt(vcpu))
goto memory_write;
+ /*
+ * HCR_EL2.E2H is nasty: it changes the way we interpret a
+ * lot of the EL2 state, so treat is as a full state
+ * transition.
+ */
+ need_put_load = ((reg == HCR_EL2) &&
+ vcpu_el2_e2h_is_set(vcpu) != !!(val & HCR_E2H));
+
+ if (need_put_load) {
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+ }
+
/*
* Always store a copy of the write to memory to avoid having
* to reverse-translate virtual EL2 system registers for a
@@ -193,6 +208,11 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
*/
__vcpu_sys_reg(vcpu, reg) = val;
+ if (need_put_load) {
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+ }
+
switch (reg) {
case ELR_EL2:
write_sysreg_el1(val, SYS_ELR);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 15/66] KVM: arm64: nv: Handle HCR_EL2.E2H specially
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
HCR_EL2.E2H is nasty, as a flip of this bit completely changes the way
we deal with a lot of the state. So when the guest flips this bit
(sysregs are live), do the put/load dance so that we have a consistent
state.
Yes, this is slow. Don't do it.
Suggested-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 77a2d452b79d..e7db4d809674 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -183,9 +183,24 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
goto memory_write;
if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ bool need_put_load;
+
if (!is_hyp_ctxt(vcpu))
goto memory_write;
+ /*
+ * HCR_EL2.E2H is nasty: it changes the way we interpret a
+ * lot of the EL2 state, so treat is as a full state
+ * transition.
+ */
+ need_put_load = ((reg == HCR_EL2) &&
+ vcpu_el2_e2h_is_set(vcpu) != !!(val & HCR_E2H));
+
+ if (need_put_load) {
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+ }
+
/*
* Always store a copy of the write to memory to avoid having
* to reverse-translate virtual EL2 system registers for a
@@ -193,6 +208,11 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
*/
__vcpu_sys_reg(vcpu, reg) = val;
+ if (need_put_load) {
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+ }
+
switch (reg) {
case ELR_EL2:
write_sysreg_el1(val, SYS_ELR);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 15/66] KVM: arm64: nv: Handle HCR_EL2.E2H specially
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
HCR_EL2.E2H is nasty, as a flip of this bit completely changes the way
we deal with a lot of the state. So when the guest flips this bit
(sysregs are live), do the put/load dance so that we have a consistent
state.
Yes, this is slow. Don't do it.
Suggested-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 77a2d452b79d..e7db4d809674 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -183,9 +183,24 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
goto memory_write;
if (unlikely(get_el2_mapping(reg, &el1r, &xlate))) {
+ bool need_put_load;
+
if (!is_hyp_ctxt(vcpu))
goto memory_write;
+ /*
+ * HCR_EL2.E2H is nasty: it changes the way we interpret a
+ * lot of the EL2 state, so treat is as a full state
+ * transition.
+ */
+ need_put_load = ((reg == HCR_EL2) &&
+ vcpu_el2_e2h_is_set(vcpu) != !!(val & HCR_E2H));
+
+ if (need_put_load) {
+ preempt_disable();
+ kvm_arch_vcpu_put(vcpu);
+ }
+
/*
* Always store a copy of the write to memory to avoid having
* to reverse-translate virtual EL2 system registers for a
@@ -193,6 +208,11 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
*/
__vcpu_sys_reg(vcpu, reg) = val;
+ if (need_put_load) {
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ preempt_enable();
+ }
+
switch (reg) {
case ELR_EL2:
write_sysreg_el1(val, SYS_ELR);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 16/66] KVM: arm64: nv: Save/Restore vEL2 sysregs
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Whenever we need to restore the guest's system registers to the CPU, we
now need to take care of the EL2 system registers as well. Most of them
are accessed via traps only, but some have an immediate effect and also
a guest running in VHE mode would expect them to be accessible via their
EL1 encoding, which we do not trap.
For vEL2 we write the virtual EL2 registers with an identical format directly
into their EL1 counterpart, and translate the few registers that have a
different format for the same effect on the execution when running a
non-VHE guest guest hypervisor.
Based on an initial patch from Andre Przywara, rewritten many times
since.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 5 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 ++++++++++++++++++++-
3 files changed, 127 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index cce43bfe158f..e3901c73893e 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -71,9 +71,10 @@ static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0);
}
-static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
+static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
+ u64 mpidr)
{
- write_sysreg(ctxt_sys_reg(ctxt, MPIDR_EL1), vmpidr_el2);
+ write_sysreg(mpidr, vmpidr_el2);
write_sysreg(ctxt_sys_reg(ctxt, CSSELR_EL1), csselr_el1);
if (has_vhe() ||
diff --git a/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c b/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
index 29305022bc04..dba101565de3 100644
--- a/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
@@ -28,7 +28,7 @@ void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_el1_state(ctxt, ctxt_sys_reg(ctxt, MPIDR_EL1));
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
__sysreg_restore_el2_return_state(ctxt);
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index 2a0b8c88d74f..53835fcc0ac6 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -13,6 +13,96 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
+
+static void __sysreg_save_vel2_state(struct kvm_cpu_context *ctxt)
+{
+ /* These registers are common with EL1 */
+ ctxt_sys_reg(ctxt, CSSELR_EL1) = read_sysreg(csselr_el1);
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+ ctxt_sys_reg(ctxt, TPIDR_EL1) = read_sysreg(tpidr_el1);
+
+ ctxt_sys_reg(ctxt, ESR_EL2) = read_sysreg_el1(SYS_ESR);
+ ctxt_sys_reg(ctxt, AFSR0_EL2) = read_sysreg_el1(SYS_AFSR0);
+ ctxt_sys_reg(ctxt, AFSR1_EL2) = read_sysreg_el1(SYS_AFSR1);
+ ctxt_sys_reg(ctxt, FAR_EL2) = read_sysreg_el1(SYS_FAR);
+ ctxt_sys_reg(ctxt, MAIR_EL2) = read_sysreg_el1(SYS_MAIR);
+ ctxt_sys_reg(ctxt, VBAR_EL2) = read_sysreg_el1(SYS_VBAR);
+ ctxt_sys_reg(ctxt, CONTEXTIDR_EL2) = read_sysreg_el1(SYS_CONTEXTIDR);
+ ctxt_sys_reg(ctxt, AMAIR_EL2) = read_sysreg_el1(SYS_AMAIR);
+
+ /*
+ * In VHE mode those registers are compatible between EL1 and EL2,
+ * and the guest uses the _EL1 versions on the CPU naturally.
+ * So we save them into their _EL2 versions here.
+ * For nVHE mode we trap accesses to those registers, so our
+ * _EL2 copy in sys_regs[] is always up-to-date and we don't need
+ * to save anything here.
+ */
+ if (__vcpu_el2_e2h_is_set(ctxt)) {
+ ctxt_sys_reg(ctxt, SCTLR_EL2) = read_sysreg_el1(SYS_SCTLR);
+ ctxt_sys_reg(ctxt, CPTR_EL2) = read_sysreg_el1(SYS_CPACR);
+ ctxt_sys_reg(ctxt, TTBR0_EL2) = read_sysreg_el1(SYS_TTBR0);
+ ctxt_sys_reg(ctxt, TTBR1_EL2) = read_sysreg_el1(SYS_TTBR1);
+ ctxt_sys_reg(ctxt, TCR_EL2) = read_sysreg_el1(SYS_TCR);
+ ctxt_sys_reg(ctxt, CNTHCTL_EL2) = read_sysreg_el1(SYS_CNTKCTL);
+ }
+
+ ctxt_sys_reg(ctxt, SP_EL2) = read_sysreg(sp_el1);
+ ctxt_sys_reg(ctxt, ELR_EL2) = read_sysreg_el1(SYS_ELR);
+ ctxt_sys_reg(ctxt, SPSR_EL2) = __fixup_spsr_el2_read(ctxt, read_sysreg_el1(SYS_SPSR));
+}
+
+static void __sysreg_restore_vel2_state(struct kvm_cpu_context *ctxt)
+{
+ u64 val;
+
+ /* These registers are common with EL1 */
+ write_sysreg(ctxt_sys_reg(ctxt, CSSELR_EL1), csselr_el1);
+ write_sysreg(ctxt_sys_reg(ctxt, PAR_EL1), par_el1);
+ write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL1), tpidr_el1);
+
+ write_sysreg(read_cpuid_id(), vpidr_el2);
+ write_sysreg(ctxt_sys_reg(ctxt, MPIDR_EL1), vmpidr_el2);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, MAIR_EL2), SYS_MAIR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, VBAR_EL2), SYS_VBAR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CONTEXTIDR_EL2),SYS_CONTEXTIDR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AMAIR_EL2), SYS_AMAIR);
+
+ if (__vcpu_el2_e2h_is_set(ctxt)) {
+ /*
+ * In VHE mode those registers are compatible between
+ * EL1 and EL2.
+ */
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL2), SYS_SCTLR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CPTR_EL2), SYS_CPACR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL2), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL2), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CNTHCTL_EL2), SYS_CNTKCTL);
+ } else {
+ val = translate_sctlr_el2_to_sctlr_el1(ctxt_sys_reg(ctxt, SCTLR_EL2));
+ write_sysreg_el1(val, SYS_SCTLR);
+ val = translate_cptr_el2_to_cpacr_el1(ctxt_sys_reg(ctxt, CPTR_EL2));
+ write_sysreg_el1(val, SYS_CPACR);
+ val = translate_ttbr0_el2_to_ttbr0_el1(ctxt_sys_reg(ctxt, TTBR0_EL2));
+ write_sysreg_el1(val, SYS_TTBR0);
+ val = translate_tcr_el2_to_tcr_el1(ctxt_sys_reg(ctxt, TCR_EL2));
+ write_sysreg_el1(val, SYS_TCR);
+ val = translate_cnthctl_el2_to_cntkctl_el1(ctxt_sys_reg(ctxt, CNTHCTL_EL2));
+ write_sysreg_el1(val, SYS_CNTKCTL);
+ }
+
+ write_sysreg_el1(ctxt_sys_reg(ctxt, ESR_EL2), SYS_ESR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AFSR0_EL2), SYS_AFSR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AFSR1_EL2), SYS_AFSR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, FAR_EL2), SYS_FAR);
+ write_sysreg(ctxt_sys_reg(ctxt, SP_EL2), sp_el1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, ELR_EL2), SYS_ELR);
+
+ val = __fixup_spsr_el2_write(ctxt, ctxt_sys_reg(ctxt, SPSR_EL2));
+ write_sysreg_el1(val, SYS_SPSR);
+}
/*
* VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and
@@ -65,6 +155,7 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
struct kvm_cpu_context *host_ctxt;
+ u64 mpidr;
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
__sysreg_save_user_state(host_ctxt);
@@ -77,7 +168,29 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu)
*/
__sysreg32_restore_state(vcpu);
__sysreg_restore_user_state(guest_ctxt);
- __sysreg_restore_el1_state(guest_ctxt);
+
+ if (unlikely(__is_hyp_ctxt(guest_ctxt))) {
+ __sysreg_restore_vel2_state(guest_ctxt);
+ } else {
+ if (nested_virt_in_use(vcpu)) {
+ /*
+ * Only set VPIDR_EL2 for nested VMs, as this is the
+ * only time it changes. We'll restore the MIDR_EL1
+ * view on put.
+ */
+ write_sysreg(ctxt_sys_reg(guest_ctxt, VPIDR_EL2), vpidr_el2);
+
+ /*
+ * As we're restoring a nested guest, set the value
+ * provided by the guest hypervisor.
+ */
+ mpidr = ctxt_sys_reg(guest_ctxt, VMPIDR_EL2);
+ } else {
+ mpidr = ctxt_sys_reg(guest_ctxt, MPIDR_EL1);
+ }
+
+ __sysreg_restore_el1_state(guest_ctxt, mpidr);
+ }
vcpu->arch.sysregs_loaded_on_cpu = true;
@@ -103,12 +216,20 @@ void kvm_vcpu_put_sysregs_vhe(struct kvm_vcpu *vcpu)
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
deactivate_traps_vhe_put();
- __sysreg_save_el1_state(guest_ctxt);
+ if (unlikely(__is_hyp_ctxt(guest_ctxt)))
+ __sysreg_save_vel2_state(guest_ctxt);
+ else
+ __sysreg_save_el1_state(guest_ctxt);
+
__sysreg_save_user_state(guest_ctxt);
__sysreg32_save_state(vcpu);
/* Restore host user state */
__sysreg_restore_user_state(host_ctxt);
+ /* If leaving a nesting guest, restore MPIDR_EL1 default view */
+ if (nested_virt_in_use(vcpu))
+ write_sysreg(read_cpuid_id(), vpidr_el2);
+
vcpu->arch.sysregs_loaded_on_cpu = false;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 16/66] KVM: arm64: nv: Save/Restore vEL2 sysregs
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Whenever we need to restore the guest's system registers to the CPU, we
now need to take care of the EL2 system registers as well. Most of them
are accessed via traps only, but some have an immediate effect and also
a guest running in VHE mode would expect them to be accessible via their
EL1 encoding, which we do not trap.
For vEL2 we write the virtual EL2 registers with an identical format directly
into their EL1 counterpart, and translate the few registers that have a
different format for the same effect on the execution when running a
non-VHE guest guest hypervisor.
Based on an initial patch from Andre Przywara, rewritten many times
since.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 5 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 ++++++++++++++++++++-
3 files changed, 127 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index cce43bfe158f..e3901c73893e 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -71,9 +71,10 @@ static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0);
}
-static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
+static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
+ u64 mpidr)
{
- write_sysreg(ctxt_sys_reg(ctxt, MPIDR_EL1), vmpidr_el2);
+ write_sysreg(mpidr, vmpidr_el2);
write_sysreg(ctxt_sys_reg(ctxt, CSSELR_EL1), csselr_el1);
if (has_vhe() ||
diff --git a/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c b/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
index 29305022bc04..dba101565de3 100644
--- a/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
@@ -28,7 +28,7 @@ void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_el1_state(ctxt, ctxt_sys_reg(ctxt, MPIDR_EL1));
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
__sysreg_restore_el2_return_state(ctxt);
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index 2a0b8c88d74f..53835fcc0ac6 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -13,6 +13,96 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
+
+static void __sysreg_save_vel2_state(struct kvm_cpu_context *ctxt)
+{
+ /* These registers are common with EL1 */
+ ctxt_sys_reg(ctxt, CSSELR_EL1) = read_sysreg(csselr_el1);
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+ ctxt_sys_reg(ctxt, TPIDR_EL1) = read_sysreg(tpidr_el1);
+
+ ctxt_sys_reg(ctxt, ESR_EL2) = read_sysreg_el1(SYS_ESR);
+ ctxt_sys_reg(ctxt, AFSR0_EL2) = read_sysreg_el1(SYS_AFSR0);
+ ctxt_sys_reg(ctxt, AFSR1_EL2) = read_sysreg_el1(SYS_AFSR1);
+ ctxt_sys_reg(ctxt, FAR_EL2) = read_sysreg_el1(SYS_FAR);
+ ctxt_sys_reg(ctxt, MAIR_EL2) = read_sysreg_el1(SYS_MAIR);
+ ctxt_sys_reg(ctxt, VBAR_EL2) = read_sysreg_el1(SYS_VBAR);
+ ctxt_sys_reg(ctxt, CONTEXTIDR_EL2) = read_sysreg_el1(SYS_CONTEXTIDR);
+ ctxt_sys_reg(ctxt, AMAIR_EL2) = read_sysreg_el1(SYS_AMAIR);
+
+ /*
+ * In VHE mode those registers are compatible between EL1 and EL2,
+ * and the guest uses the _EL1 versions on the CPU naturally.
+ * So we save them into their _EL2 versions here.
+ * For nVHE mode we trap accesses to those registers, so our
+ * _EL2 copy in sys_regs[] is always up-to-date and we don't need
+ * to save anything here.
+ */
+ if (__vcpu_el2_e2h_is_set(ctxt)) {
+ ctxt_sys_reg(ctxt, SCTLR_EL2) = read_sysreg_el1(SYS_SCTLR);
+ ctxt_sys_reg(ctxt, CPTR_EL2) = read_sysreg_el1(SYS_CPACR);
+ ctxt_sys_reg(ctxt, TTBR0_EL2) = read_sysreg_el1(SYS_TTBR0);
+ ctxt_sys_reg(ctxt, TTBR1_EL2) = read_sysreg_el1(SYS_TTBR1);
+ ctxt_sys_reg(ctxt, TCR_EL2) = read_sysreg_el1(SYS_TCR);
+ ctxt_sys_reg(ctxt, CNTHCTL_EL2) = read_sysreg_el1(SYS_CNTKCTL);
+ }
+
+ ctxt_sys_reg(ctxt, SP_EL2) = read_sysreg(sp_el1);
+ ctxt_sys_reg(ctxt, ELR_EL2) = read_sysreg_el1(SYS_ELR);
+ ctxt_sys_reg(ctxt, SPSR_EL2) = __fixup_spsr_el2_read(ctxt, read_sysreg_el1(SYS_SPSR));
+}
+
+static void __sysreg_restore_vel2_state(struct kvm_cpu_context *ctxt)
+{
+ u64 val;
+
+ /* These registers are common with EL1 */
+ write_sysreg(ctxt_sys_reg(ctxt, CSSELR_EL1), csselr_el1);
+ write_sysreg(ctxt_sys_reg(ctxt, PAR_EL1), par_el1);
+ write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL1), tpidr_el1);
+
+ write_sysreg(read_cpuid_id(), vpidr_el2);
+ write_sysreg(ctxt_sys_reg(ctxt, MPIDR_EL1), vmpidr_el2);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, MAIR_EL2), SYS_MAIR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, VBAR_EL2), SYS_VBAR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CONTEXTIDR_EL2),SYS_CONTEXTIDR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AMAIR_EL2), SYS_AMAIR);
+
+ if (__vcpu_el2_e2h_is_set(ctxt)) {
+ /*
+ * In VHE mode those registers are compatible between
+ * EL1 and EL2.
+ */
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL2), SYS_SCTLR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CPTR_EL2), SYS_CPACR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL2), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL2), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CNTHCTL_EL2), SYS_CNTKCTL);
+ } else {
+ val = translate_sctlr_el2_to_sctlr_el1(ctxt_sys_reg(ctxt, SCTLR_EL2));
+ write_sysreg_el1(val, SYS_SCTLR);
+ val = translate_cptr_el2_to_cpacr_el1(ctxt_sys_reg(ctxt, CPTR_EL2));
+ write_sysreg_el1(val, SYS_CPACR);
+ val = translate_ttbr0_el2_to_ttbr0_el1(ctxt_sys_reg(ctxt, TTBR0_EL2));
+ write_sysreg_el1(val, SYS_TTBR0);
+ val = translate_tcr_el2_to_tcr_el1(ctxt_sys_reg(ctxt, TCR_EL2));
+ write_sysreg_el1(val, SYS_TCR);
+ val = translate_cnthctl_el2_to_cntkctl_el1(ctxt_sys_reg(ctxt, CNTHCTL_EL2));
+ write_sysreg_el1(val, SYS_CNTKCTL);
+ }
+
+ write_sysreg_el1(ctxt_sys_reg(ctxt, ESR_EL2), SYS_ESR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AFSR0_EL2), SYS_AFSR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AFSR1_EL2), SYS_AFSR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, FAR_EL2), SYS_FAR);
+ write_sysreg(ctxt_sys_reg(ctxt, SP_EL2), sp_el1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, ELR_EL2), SYS_ELR);
+
+ val = __fixup_spsr_el2_write(ctxt, ctxt_sys_reg(ctxt, SPSR_EL2));
+ write_sysreg_el1(val, SYS_SPSR);
+}
/*
* VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and
@@ -65,6 +155,7 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
struct kvm_cpu_context *host_ctxt;
+ u64 mpidr;
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
__sysreg_save_user_state(host_ctxt);
@@ -77,7 +168,29 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu)
*/
__sysreg32_restore_state(vcpu);
__sysreg_restore_user_state(guest_ctxt);
- __sysreg_restore_el1_state(guest_ctxt);
+
+ if (unlikely(__is_hyp_ctxt(guest_ctxt))) {
+ __sysreg_restore_vel2_state(guest_ctxt);
+ } else {
+ if (nested_virt_in_use(vcpu)) {
+ /*
+ * Only set VPIDR_EL2 for nested VMs, as this is the
+ * only time it changes. We'll restore the MIDR_EL1
+ * view on put.
+ */
+ write_sysreg(ctxt_sys_reg(guest_ctxt, VPIDR_EL2), vpidr_el2);
+
+ /*
+ * As we're restoring a nested guest, set the value
+ * provided by the guest hypervisor.
+ */
+ mpidr = ctxt_sys_reg(guest_ctxt, VMPIDR_EL2);
+ } else {
+ mpidr = ctxt_sys_reg(guest_ctxt, MPIDR_EL1);
+ }
+
+ __sysreg_restore_el1_state(guest_ctxt, mpidr);
+ }
vcpu->arch.sysregs_loaded_on_cpu = true;
@@ -103,12 +216,20 @@ void kvm_vcpu_put_sysregs_vhe(struct kvm_vcpu *vcpu)
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
deactivate_traps_vhe_put();
- __sysreg_save_el1_state(guest_ctxt);
+ if (unlikely(__is_hyp_ctxt(guest_ctxt)))
+ __sysreg_save_vel2_state(guest_ctxt);
+ else
+ __sysreg_save_el1_state(guest_ctxt);
+
__sysreg_save_user_state(guest_ctxt);
__sysreg32_save_state(vcpu);
/* Restore host user state */
__sysreg_restore_user_state(host_ctxt);
+ /* If leaving a nesting guest, restore MPIDR_EL1 default view */
+ if (nested_virt_in_use(vcpu))
+ write_sysreg(read_cpuid_id(), vpidr_el2);
+
vcpu->arch.sysregs_loaded_on_cpu = false;
}
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 16/66] KVM: arm64: nv: Save/Restore vEL2 sysregs
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Whenever we need to restore the guest's system registers to the CPU, we
now need to take care of the EL2 system registers as well. Most of them
are accessed via traps only, but some have an immediate effect and also
a guest running in VHE mode would expect them to be accessible via their
EL1 encoding, which we do not trap.
For vEL2 we write the virtual EL2 registers with an identical format directly
into their EL1 counterpart, and translate the few registers that have a
different format for the same effect on the execution when running a
non-VHE guest guest hypervisor.
Based on an initial patch from Andre Przywara, rewritten many times
since.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 5 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 ++++++++++++++++++++-
3 files changed, 127 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index cce43bfe158f..e3901c73893e 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -71,9 +71,10 @@ static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0);
}
-static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
+static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
+ u64 mpidr)
{
- write_sysreg(ctxt_sys_reg(ctxt, MPIDR_EL1), vmpidr_el2);
+ write_sysreg(mpidr, vmpidr_el2);
write_sysreg(ctxt_sys_reg(ctxt, CSSELR_EL1), csselr_el1);
if (has_vhe() ||
diff --git a/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c b/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
index 29305022bc04..dba101565de3 100644
--- a/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
@@ -28,7 +28,7 @@ void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_el1_state(ctxt, ctxt_sys_reg(ctxt, MPIDR_EL1));
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
__sysreg_restore_el2_return_state(ctxt);
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index 2a0b8c88d74f..53835fcc0ac6 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -13,6 +13,96 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
+
+static void __sysreg_save_vel2_state(struct kvm_cpu_context *ctxt)
+{
+ /* These registers are common with EL1 */
+ ctxt_sys_reg(ctxt, CSSELR_EL1) = read_sysreg(csselr_el1);
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+ ctxt_sys_reg(ctxt, TPIDR_EL1) = read_sysreg(tpidr_el1);
+
+ ctxt_sys_reg(ctxt, ESR_EL2) = read_sysreg_el1(SYS_ESR);
+ ctxt_sys_reg(ctxt, AFSR0_EL2) = read_sysreg_el1(SYS_AFSR0);
+ ctxt_sys_reg(ctxt, AFSR1_EL2) = read_sysreg_el1(SYS_AFSR1);
+ ctxt_sys_reg(ctxt, FAR_EL2) = read_sysreg_el1(SYS_FAR);
+ ctxt_sys_reg(ctxt, MAIR_EL2) = read_sysreg_el1(SYS_MAIR);
+ ctxt_sys_reg(ctxt, VBAR_EL2) = read_sysreg_el1(SYS_VBAR);
+ ctxt_sys_reg(ctxt, CONTEXTIDR_EL2) = read_sysreg_el1(SYS_CONTEXTIDR);
+ ctxt_sys_reg(ctxt, AMAIR_EL2) = read_sysreg_el1(SYS_AMAIR);
+
+ /*
+ * In VHE mode those registers are compatible between EL1 and EL2,
+ * and the guest uses the _EL1 versions on the CPU naturally.
+ * So we save them into their _EL2 versions here.
+ * For nVHE mode we trap accesses to those registers, so our
+ * _EL2 copy in sys_regs[] is always up-to-date and we don't need
+ * to save anything here.
+ */
+ if (__vcpu_el2_e2h_is_set(ctxt)) {
+ ctxt_sys_reg(ctxt, SCTLR_EL2) = read_sysreg_el1(SYS_SCTLR);
+ ctxt_sys_reg(ctxt, CPTR_EL2) = read_sysreg_el1(SYS_CPACR);
+ ctxt_sys_reg(ctxt, TTBR0_EL2) = read_sysreg_el1(SYS_TTBR0);
+ ctxt_sys_reg(ctxt, TTBR1_EL2) = read_sysreg_el1(SYS_TTBR1);
+ ctxt_sys_reg(ctxt, TCR_EL2) = read_sysreg_el1(SYS_TCR);
+ ctxt_sys_reg(ctxt, CNTHCTL_EL2) = read_sysreg_el1(SYS_CNTKCTL);
+ }
+
+ ctxt_sys_reg(ctxt, SP_EL2) = read_sysreg(sp_el1);
+ ctxt_sys_reg(ctxt, ELR_EL2) = read_sysreg_el1(SYS_ELR);
+ ctxt_sys_reg(ctxt, SPSR_EL2) = __fixup_spsr_el2_read(ctxt, read_sysreg_el1(SYS_SPSR));
+}
+
+static void __sysreg_restore_vel2_state(struct kvm_cpu_context *ctxt)
+{
+ u64 val;
+
+ /* These registers are common with EL1 */
+ write_sysreg(ctxt_sys_reg(ctxt, CSSELR_EL1), csselr_el1);
+ write_sysreg(ctxt_sys_reg(ctxt, PAR_EL1), par_el1);
+ write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL1), tpidr_el1);
+
+ write_sysreg(read_cpuid_id(), vpidr_el2);
+ write_sysreg(ctxt_sys_reg(ctxt, MPIDR_EL1), vmpidr_el2);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, MAIR_EL2), SYS_MAIR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, VBAR_EL2), SYS_VBAR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CONTEXTIDR_EL2),SYS_CONTEXTIDR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AMAIR_EL2), SYS_AMAIR);
+
+ if (__vcpu_el2_e2h_is_set(ctxt)) {
+ /*
+ * In VHE mode those registers are compatible between
+ * EL1 and EL2.
+ */
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL2), SYS_SCTLR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CPTR_EL2), SYS_CPACR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL2), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL2), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, CNTHCTL_EL2), SYS_CNTKCTL);
+ } else {
+ val = translate_sctlr_el2_to_sctlr_el1(ctxt_sys_reg(ctxt, SCTLR_EL2));
+ write_sysreg_el1(val, SYS_SCTLR);
+ val = translate_cptr_el2_to_cpacr_el1(ctxt_sys_reg(ctxt, CPTR_EL2));
+ write_sysreg_el1(val, SYS_CPACR);
+ val = translate_ttbr0_el2_to_ttbr0_el1(ctxt_sys_reg(ctxt, TTBR0_EL2));
+ write_sysreg_el1(val, SYS_TTBR0);
+ val = translate_tcr_el2_to_tcr_el1(ctxt_sys_reg(ctxt, TCR_EL2));
+ write_sysreg_el1(val, SYS_TCR);
+ val = translate_cnthctl_el2_to_cntkctl_el1(ctxt_sys_reg(ctxt, CNTHCTL_EL2));
+ write_sysreg_el1(val, SYS_CNTKCTL);
+ }
+
+ write_sysreg_el1(ctxt_sys_reg(ctxt, ESR_EL2), SYS_ESR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AFSR0_EL2), SYS_AFSR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, AFSR1_EL2), SYS_AFSR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, FAR_EL2), SYS_FAR);
+ write_sysreg(ctxt_sys_reg(ctxt, SP_EL2), sp_el1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, ELR_EL2), SYS_ELR);
+
+ val = __fixup_spsr_el2_write(ctxt, ctxt_sys_reg(ctxt, SPSR_EL2));
+ write_sysreg_el1(val, SYS_SPSR);
+}
/*
* VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and
@@ -65,6 +155,7 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
struct kvm_cpu_context *host_ctxt;
+ u64 mpidr;
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
__sysreg_save_user_state(host_ctxt);
@@ -77,7 +168,29 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu)
*/
__sysreg32_restore_state(vcpu);
__sysreg_restore_user_state(guest_ctxt);
- __sysreg_restore_el1_state(guest_ctxt);
+
+ if (unlikely(__is_hyp_ctxt(guest_ctxt))) {
+ __sysreg_restore_vel2_state(guest_ctxt);
+ } else {
+ if (nested_virt_in_use(vcpu)) {
+ /*
+ * Only set VPIDR_EL2 for nested VMs, as this is the
+ * only time it changes. We'll restore the MIDR_EL1
+ * view on put.
+ */
+ write_sysreg(ctxt_sys_reg(guest_ctxt, VPIDR_EL2), vpidr_el2);
+
+ /*
+ * As we're restoring a nested guest, set the value
+ * provided by the guest hypervisor.
+ */
+ mpidr = ctxt_sys_reg(guest_ctxt, VMPIDR_EL2);
+ } else {
+ mpidr = ctxt_sys_reg(guest_ctxt, MPIDR_EL1);
+ }
+
+ __sysreg_restore_el1_state(guest_ctxt, mpidr);
+ }
vcpu->arch.sysregs_loaded_on_cpu = true;
@@ -103,12 +216,20 @@ void kvm_vcpu_put_sysregs_vhe(struct kvm_vcpu *vcpu)
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
deactivate_traps_vhe_put();
- __sysreg_save_el1_state(guest_ctxt);
+ if (unlikely(__is_hyp_ctxt(guest_ctxt)))
+ __sysreg_save_vel2_state(guest_ctxt);
+ else
+ __sysreg_save_el1_state(guest_ctxt);
+
__sysreg_save_user_state(guest_ctxt);
__sysreg32_save_state(vcpu);
/* Restore host user state */
__sysreg_restore_user_state(host_ctxt);
+ /* If leaving a nesting guest, restore MPIDR_EL1 default view */
+ if (nested_virt_in_use(vcpu))
+ write_sysreg(read_cpuid_id(), vpidr_el2);
+
vcpu->arch.sysregs_loaded_on_cpu = false;
}
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 17/66] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
We can no longer blindly copy the VCPU's PSTATE into SPSR_EL2 and return
to the guest and vice versa when taking an exception to the hypervisor,
because we emulate virtual EL2 in EL1 and therefore have to translate
the mode field from EL2 to EL1 and vice versa.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 43 +++++++++++++++++++++-
1 file changed, 41 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index e3901c73893e..92715fa01e88 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -51,10 +51,32 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt_sys_reg(ctxt, SPSR_EL1) = read_sysreg_el1(SYS_SPSR);
}
+static inline u64 from_hw_pstate(const struct kvm_cpu_context *ctxt)
+{
+ u64 reg = read_sysreg_el2(SYS_SPSR);
+
+ if (__is_hyp_ctxt(ctxt)) {
+ u64 mode = reg & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL1t:
+ mode = PSR_MODE_EL2t;
+ break;
+ case PSR_MODE_EL1h:
+ mode = PSR_MODE_EL2h;
+ break;
+ }
+
+ return (reg & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+ }
+
+ return reg;
+}
+
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
{
ctxt->regs.pc = read_sysreg_el2(SYS_ELR);
- ctxt->regs.pstate = read_sysreg_el2(SYS_SPSR);
+ ctxt->regs.pstate = from_hw_pstate(ctxt);
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
@@ -131,9 +153,26 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
write_sysreg_el1(ctxt_sys_reg(ctxt, SPSR_EL1), SYS_SPSR);
}
+/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
+static inline u64 to_hw_pstate(const struct kvm_cpu_context *ctxt)
+{
+ u64 mode = ctxt->regs.pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL2t:
+ mode = PSR_MODE_EL1t;
+ break;
+ case PSR_MODE_EL2h:
+ mode = PSR_MODE_EL1h;
+ break;
+ }
+
+ return (ctxt->regs.pstate & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+}
+
static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
{
- u64 pstate = ctxt->regs.pstate;
+ u64 pstate = to_hw_pstate(ctxt);
u64 mode = pstate & PSR_AA32_MODE_MASK;
/*
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 17/66] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
We can no longer blindly copy the VCPU's PSTATE into SPSR_EL2 and return
to the guest and vice versa when taking an exception to the hypervisor,
because we emulate virtual EL2 in EL1 and therefore have to translate
the mode field from EL2 to EL1 and vice versa.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 43 +++++++++++++++++++++-
1 file changed, 41 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index e3901c73893e..92715fa01e88 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -51,10 +51,32 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt_sys_reg(ctxt, SPSR_EL1) = read_sysreg_el1(SYS_SPSR);
}
+static inline u64 from_hw_pstate(const struct kvm_cpu_context *ctxt)
+{
+ u64 reg = read_sysreg_el2(SYS_SPSR);
+
+ if (__is_hyp_ctxt(ctxt)) {
+ u64 mode = reg & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL1t:
+ mode = PSR_MODE_EL2t;
+ break;
+ case PSR_MODE_EL1h:
+ mode = PSR_MODE_EL2h;
+ break;
+ }
+
+ return (reg & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+ }
+
+ return reg;
+}
+
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
{
ctxt->regs.pc = read_sysreg_el2(SYS_ELR);
- ctxt->regs.pstate = read_sysreg_el2(SYS_SPSR);
+ ctxt->regs.pstate = from_hw_pstate(ctxt);
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
@@ -131,9 +153,26 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
write_sysreg_el1(ctxt_sys_reg(ctxt, SPSR_EL1), SYS_SPSR);
}
+/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
+static inline u64 to_hw_pstate(const struct kvm_cpu_context *ctxt)
+{
+ u64 mode = ctxt->regs.pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL2t:
+ mode = PSR_MODE_EL1t;
+ break;
+ case PSR_MODE_EL2h:
+ mode = PSR_MODE_EL1h;
+ break;
+ }
+
+ return (ctxt->regs.pstate & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+}
+
static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
{
- u64 pstate = ctxt->regs.pstate;
+ u64 pstate = to_hw_pstate(ctxt);
u64 mode = pstate & PSR_AA32_MODE_MASK;
/*
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 17/66] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
We can no longer blindly copy the VCPU's PSTATE into SPSR_EL2 and return
to the guest and vice versa when taking an exception to the hypervisor,
because we emulate virtual EL2 in EL1 and therefore have to translate
the mode field from EL2 to EL1 and vice versa.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 43 +++++++++++++++++++++-
1 file changed, 41 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index e3901c73893e..92715fa01e88 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -51,10 +51,32 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt_sys_reg(ctxt, SPSR_EL1) = read_sysreg_el1(SYS_SPSR);
}
+static inline u64 from_hw_pstate(const struct kvm_cpu_context *ctxt)
+{
+ u64 reg = read_sysreg_el2(SYS_SPSR);
+
+ if (__is_hyp_ctxt(ctxt)) {
+ u64 mode = reg & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL1t:
+ mode = PSR_MODE_EL2t;
+ break;
+ case PSR_MODE_EL1h:
+ mode = PSR_MODE_EL2h;
+ break;
+ }
+
+ return (reg & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+ }
+
+ return reg;
+}
+
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
{
ctxt->regs.pc = read_sysreg_el2(SYS_ELR);
- ctxt->regs.pstate = read_sysreg_el2(SYS_SPSR);
+ ctxt->regs.pstate = from_hw_pstate(ctxt);
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
@@ -131,9 +153,26 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
write_sysreg_el1(ctxt_sys_reg(ctxt, SPSR_EL1), SYS_SPSR);
}
+/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
+static inline u64 to_hw_pstate(const struct kvm_cpu_context *ctxt)
+{
+ u64 mode = ctxt->regs.pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL2t:
+ mode = PSR_MODE_EL1t;
+ break;
+ case PSR_MODE_EL2h:
+ mode = PSR_MODE_EL1h;
+ break;
+ }
+
+ return (ctxt->regs.pstate & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+}
+
static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
{
- u64 pstate = ctxt->regs.pstate;
+ u64 pstate = to_hw_pstate(ctxt);
u64 mode = pstate & PSR_AA32_MODE_MASK;
/*
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 18/66] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Christoffer Dall
From: Christoffer Dall <christoffer.dall@linaro.org>
When running in virtual EL2 mode, we actually run the hardware in EL1
and therefore have to use the EL1 registers to ensure correct operation.
By setting the HCR.TVM and HCR.TVRM we ensure that the virtual EL2 mode
doesn't shoot itself in the foot when setting up what it believes to be
a different mode's system register state (for example when preparing to
switch to a VM).
We can leverage the existing sysregs infrastructure to support trapped
accesses to these registers.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +---
arch/arm64/kvm/hyp/nvhe/switch.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 7 ++++++-
arch/arm64/kvm/sys_regs.c | 19 ++++++++++++++++---
4 files changed, 24 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 84473574c2e7..424b074e4377 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -101,10 +101,8 @@ static inline void __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
-static inline void ___activate_traps(struct kvm_vcpu *vcpu)
+static inline void ___activate_traps(struct kvm_vcpu *vcpu, u64 hcr)
{
- u64 hcr = vcpu->arch.hcr_el2;
-
if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM))
hcr |= HCR_TVM;
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f3d0e9eca56c..07bf5c03631b 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -37,7 +37,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
{
u64 val;
- ___activate_traps(vcpu);
+ ___activate_traps(vcpu, vcpu->arch.hcr_el2);
__activate_traps_common(vcpu);
val = CPTR_EL2_DEFAULT;
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index af8e940d0f03..3d8a4402b52c 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -36,9 +36,14 @@ DEFINE_PER_CPU(unsigned long, kvm_hyp_vector);
static void __activate_traps(struct kvm_vcpu *vcpu)
{
+ u64 hcr = vcpu->arch.hcr_el2;
u64 val;
- ___activate_traps(vcpu);
+ /* Trap VM sysreg accesses if an EL2 guest is not using VHE. */
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
+ hcr |= HCR_TVM | HCR_TRVM;
+
+ ___activate_traps(vcpu, hcr);
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e7db4d809674..540a7faaaee3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -361,8 +361,15 @@ static void get_access_mask(const struct sys_reg_desc *r, u64 *mask, u64 *shift)
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
- * is set. If the guest enables the MMU, we stop trapping the VM
- * sys_regs and leave it in complete control of the caches.
+ * is set.
+ *
+ * This is set in two cases: either (1) we're running at vEL2, or (2)
+ * we're running at EL1 and the guest has its MMU off.
+ *
+ * (1) TVM/TRVM is set, as we need to virtualise some of the VM
+ * registers for the guest hypervisor
+ * (2) Once the guest enables the MMU, we stop trapping the VM sys_regs
+ * and leave it in complete control of the caches.
*/
static bool access_vm_reg(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
@@ -371,7 +378,13 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
bool was_enabled = vcpu_has_cache_enabled(vcpu);
u64 val, mask, shift;
- BUG_ON(!p->is_write);
+ /* We don't expect TRVM on the host */
+ BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
+
+ if (!p->is_write) {
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ return true;
+ }
get_access_mask(r, &mask, &shift);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 18/66] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Andre Przywara, Christoffer Dall
From: Christoffer Dall <christoffer.dall@linaro.org>
When running in virtual EL2 mode, we actually run the hardware in EL1
and therefore have to use the EL1 registers to ensure correct operation.
By setting the HCR.TVM and HCR.TVRM we ensure that the virtual EL2 mode
doesn't shoot itself in the foot when setting up what it believes to be
a different mode's system register state (for example when preparing to
switch to a VM).
We can leverage the existing sysregs infrastructure to support trapped
accesses to these registers.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +---
arch/arm64/kvm/hyp/nvhe/switch.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 7 ++++++-
arch/arm64/kvm/sys_regs.c | 19 ++++++++++++++++---
4 files changed, 24 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 84473574c2e7..424b074e4377 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -101,10 +101,8 @@ static inline void __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
-static inline void ___activate_traps(struct kvm_vcpu *vcpu)
+static inline void ___activate_traps(struct kvm_vcpu *vcpu, u64 hcr)
{
- u64 hcr = vcpu->arch.hcr_el2;
-
if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM))
hcr |= HCR_TVM;
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f3d0e9eca56c..07bf5c03631b 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -37,7 +37,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
{
u64 val;
- ___activate_traps(vcpu);
+ ___activate_traps(vcpu, vcpu->arch.hcr_el2);
__activate_traps_common(vcpu);
val = CPTR_EL2_DEFAULT;
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index af8e940d0f03..3d8a4402b52c 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -36,9 +36,14 @@ DEFINE_PER_CPU(unsigned long, kvm_hyp_vector);
static void __activate_traps(struct kvm_vcpu *vcpu)
{
+ u64 hcr = vcpu->arch.hcr_el2;
u64 val;
- ___activate_traps(vcpu);
+ /* Trap VM sysreg accesses if an EL2 guest is not using VHE. */
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
+ hcr |= HCR_TVM | HCR_TRVM;
+
+ ___activate_traps(vcpu, hcr);
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e7db4d809674..540a7faaaee3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -361,8 +361,15 @@ static void get_access_mask(const struct sys_reg_desc *r, u64 *mask, u64 *shift)
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
- * is set. If the guest enables the MMU, we stop trapping the VM
- * sys_regs and leave it in complete control of the caches.
+ * is set.
+ *
+ * This is set in two cases: either (1) we're running at vEL2, or (2)
+ * we're running at EL1 and the guest has its MMU off.
+ *
+ * (1) TVM/TRVM is set, as we need to virtualise some of the VM
+ * registers for the guest hypervisor
+ * (2) Once the guest enables the MMU, we stop trapping the VM sys_regs
+ * and leave it in complete control of the caches.
*/
static bool access_vm_reg(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
@@ -371,7 +378,13 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
bool was_enabled = vcpu_has_cache_enabled(vcpu);
u64 val, mask, shift;
- BUG_ON(!p->is_write);
+ /* We don't expect TRVM on the host */
+ BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
+
+ if (!p->is_write) {
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ return true;
+ }
get_access_mask(r, &mask, &shift);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 18/66] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Christoffer Dall, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Christoffer Dall <christoffer.dall@linaro.org>
When running in virtual EL2 mode, we actually run the hardware in EL1
and therefore have to use the EL1 registers to ensure correct operation.
By setting the HCR.TVM and HCR.TVRM we ensure that the virtual EL2 mode
doesn't shoot itself in the foot when setting up what it believes to be
a different mode's system register state (for example when preparing to
switch to a VM).
We can leverage the existing sysregs infrastructure to support trapped
accesses to these registers.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +---
arch/arm64/kvm/hyp/nvhe/switch.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 7 ++++++-
arch/arm64/kvm/sys_regs.c | 19 ++++++++++++++++---
4 files changed, 24 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 84473574c2e7..424b074e4377 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -101,10 +101,8 @@ static inline void __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
-static inline void ___activate_traps(struct kvm_vcpu *vcpu)
+static inline void ___activate_traps(struct kvm_vcpu *vcpu, u64 hcr)
{
- u64 hcr = vcpu->arch.hcr_el2;
-
if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM))
hcr |= HCR_TVM;
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f3d0e9eca56c..07bf5c03631b 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -37,7 +37,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
{
u64 val;
- ___activate_traps(vcpu);
+ ___activate_traps(vcpu, vcpu->arch.hcr_el2);
__activate_traps_common(vcpu);
val = CPTR_EL2_DEFAULT;
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index af8e940d0f03..3d8a4402b52c 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -36,9 +36,14 @@ DEFINE_PER_CPU(unsigned long, kvm_hyp_vector);
static void __activate_traps(struct kvm_vcpu *vcpu)
{
+ u64 hcr = vcpu->arch.hcr_el2;
u64 val;
- ___activate_traps(vcpu);
+ /* Trap VM sysreg accesses if an EL2 guest is not using VHE. */
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
+ hcr |= HCR_TVM | HCR_TRVM;
+
+ ___activate_traps(vcpu, hcr);
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e7db4d809674..540a7faaaee3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -361,8 +361,15 @@ static void get_access_mask(const struct sys_reg_desc *r, u64 *mask, u64 *shift)
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
- * is set. If the guest enables the MMU, we stop trapping the VM
- * sys_regs and leave it in complete control of the caches.
+ * is set.
+ *
+ * This is set in two cases: either (1) we're running at vEL2, or (2)
+ * we're running at EL1 and the guest has its MMU off.
+ *
+ * (1) TVM/TRVM is set, as we need to virtualise some of the VM
+ * registers for the guest hypervisor
+ * (2) Once the guest enables the MMU, we stop trapping the VM sys_regs
+ * and leave it in complete control of the caches.
*/
static bool access_vm_reg(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
@@ -371,7 +378,13 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
bool was_enabled = vcpu_has_cache_enabled(vcpu);
u64 val, mask, shift;
- BUG_ON(!p->is_write);
+ /* We don't expect TRVM on the host */
+ BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
+
+ if (!p->is_write) {
+ p->regval = vcpu_read_sys_reg(vcpu, r->reg);
+ return true;
+ }
get_access_mask(r, &mask, &shift);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 19/66] KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
For the same reason we trap virtual memory register accesses at virtual
EL2, we need to trap SPSR_EL1, ELR_EL1 and VBAR_EL1 accesses. ARM v8.3
introduces the HCR_EL2.NV1 bit to be able to trap on those register
accesses in EL1. Do not set this bit until the whole nesting support is
completed.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 29 ++++++++++++++++++++++++++++-
1 file changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 540a7faaaee3..5185f148f25e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1567,6 +1567,30 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_elr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, ELR_EL1);
+
+ return true;
+}
+
+static bool access_spsr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ __vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
+ else
+ p->regval = __vcpu_sys_reg(vcpu, SPSR_EL1);
+
+ return true;
+}
+
static bool access_spsr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1728,6 +1752,9 @@ static const struct sys_reg_desc sys_reg_descs[] = {
PTRAUTH_KEY(APDB),
PTRAUTH_KEY(APGA),
+ { SYS_DESC(SYS_SPSR_EL1), access_spsr},
+ { SYS_DESC(SYS_ELR_EL1), access_elr},
+
{ SYS_DESC(SYS_AFSR0_EL1), access_vm_reg, reset_unknown, AFSR0_EL1 },
{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
@@ -1759,7 +1786,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_LORC_EL1), trap_loregion },
{ SYS_DESC(SYS_LORID_EL1), trap_loregion },
- { SYS_DESC(SYS_VBAR_EL1), NULL, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_VBAR_EL1), access_rw, reset_val, VBAR_EL1, 0 },
{ SYS_DESC(SYS_DISR_EL1), NULL, reset_val, DISR_EL1, 0 },
{ SYS_DESC(SYS_ICC_IAR0_EL1), write_to_read_only },
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 19/66] KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
For the same reason we trap virtual memory register accesses at virtual
EL2, we need to trap SPSR_EL1, ELR_EL1 and VBAR_EL1 accesses. ARM v8.3
introduces the HCR_EL2.NV1 bit to be able to trap on those register
accesses in EL1. Do not set this bit until the whole nesting support is
completed.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 29 ++++++++++++++++++++++++++++-
1 file changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 540a7faaaee3..5185f148f25e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1567,6 +1567,30 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_elr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, ELR_EL1);
+
+ return true;
+}
+
+static bool access_spsr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ __vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
+ else
+ p->regval = __vcpu_sys_reg(vcpu, SPSR_EL1);
+
+ return true;
+}
+
static bool access_spsr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1728,6 +1752,9 @@ static const struct sys_reg_desc sys_reg_descs[] = {
PTRAUTH_KEY(APDB),
PTRAUTH_KEY(APGA),
+ { SYS_DESC(SYS_SPSR_EL1), access_spsr},
+ { SYS_DESC(SYS_ELR_EL1), access_elr},
+
{ SYS_DESC(SYS_AFSR0_EL1), access_vm_reg, reset_unknown, AFSR0_EL1 },
{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
@@ -1759,7 +1786,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_LORC_EL1), trap_loregion },
{ SYS_DESC(SYS_LORID_EL1), trap_loregion },
- { SYS_DESC(SYS_VBAR_EL1), NULL, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_VBAR_EL1), access_rw, reset_val, VBAR_EL1, 0 },
{ SYS_DESC(SYS_DISR_EL1), NULL, reset_val, DISR_EL1, 0 },
{ SYS_DESC(SYS_ICC_IAR0_EL1), write_to_read_only },
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 19/66] KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
For the same reason we trap virtual memory register accesses at virtual
EL2, we need to trap SPSR_EL1, ELR_EL1 and VBAR_EL1 accesses. ARM v8.3
introduces the HCR_EL2.NV1 bit to be able to trap on those register
accesses in EL1. Do not set this bit until the whole nesting support is
completed.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 29 ++++++++++++++++++++++++++++-
1 file changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 540a7faaaee3..5185f148f25e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1567,6 +1567,30 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_elr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
+ else
+ p->regval = vcpu_read_sys_reg(vcpu, ELR_EL1);
+
+ return true;
+}
+
+static bool access_spsr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ __vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
+ else
+ p->regval = __vcpu_sys_reg(vcpu, SPSR_EL1);
+
+ return true;
+}
+
static bool access_spsr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1728,6 +1752,9 @@ static const struct sys_reg_desc sys_reg_descs[] = {
PTRAUTH_KEY(APDB),
PTRAUTH_KEY(APGA),
+ { SYS_DESC(SYS_SPSR_EL1), access_spsr},
+ { SYS_DESC(SYS_ELR_EL1), access_elr},
+
{ SYS_DESC(SYS_AFSR0_EL1), access_vm_reg, reset_unknown, AFSR0_EL1 },
{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
@@ -1759,7 +1786,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_LORC_EL1), trap_loregion },
{ SYS_DESC(SYS_LORID_EL1), trap_loregion },
- { SYS_DESC(SYS_VBAR_EL1), NULL, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_VBAR_EL1), access_rw, reset_val, VBAR_EL1, 0 },
{ SYS_DESC(SYS_DISR_EL1), NULL, reset_val, DISR_EL1, 0 },
{ SYS_DESC(SYS_ICC_IAR0_EL1), write_to_read_only },
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 20/66] KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
For the same reason we trap virtual memory register accesses in virtual
EL2, we trap CPACR_EL1 access too; We allow the virtual EL2 mode to
access EL1 system register state instead of the virtual EL2 one.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 3 +++
arch/arm64/kvm/sys_regs.c | 2 +-
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4cf745f79985..3bd29fd01d80 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -269,7 +269,7 @@
#define CPTR_EL2_TFP_SHIFT 10
/* Hyp Coprocessor Trap Register */
-#define CPTR_EL2_TCPAC (1 << 31)
+#define CPTR_EL2_TCPAC (1U << 31)
#define CPTR_EL2_TAM (1 << 30)
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 3d8a4402b52c..66358f8ec5ad 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -68,6 +68,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
__activate_traps_fpsimd32(vcpu);
}
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
+ val |= CPTR_EL2_TCPAC;
+
write_sysreg(val, cpacr_el1);
write_sysreg(__this_cpu_read(kvm_hyp_vector), vbar_el1);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5185f148f25e..4410d399f8e2 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1736,7 +1736,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 },
- { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
+ { SYS_DESC(SYS_CPACR_EL1), access_rw, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_RGSR_EL1), undef_access },
{ SYS_DESC(SYS_GCR_EL1), undef_access },
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 20/66] KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
For the same reason we trap virtual memory register accesses in virtual
EL2, we trap CPACR_EL1 access too; We allow the virtual EL2 mode to
access EL1 system register state instead of the virtual EL2 one.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 3 +++
arch/arm64/kvm/sys_regs.c | 2 +-
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4cf745f79985..3bd29fd01d80 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -269,7 +269,7 @@
#define CPTR_EL2_TFP_SHIFT 10
/* Hyp Coprocessor Trap Register */
-#define CPTR_EL2_TCPAC (1 << 31)
+#define CPTR_EL2_TCPAC (1U << 31)
#define CPTR_EL2_TAM (1 << 30)
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 3d8a4402b52c..66358f8ec5ad 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -68,6 +68,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
__activate_traps_fpsimd32(vcpu);
}
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
+ val |= CPTR_EL2_TCPAC;
+
write_sysreg(val, cpacr_el1);
write_sysreg(__this_cpu_read(kvm_hyp_vector), vbar_el1);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5185f148f25e..4410d399f8e2 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1736,7 +1736,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 },
- { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
+ { SYS_DESC(SYS_CPACR_EL1), access_rw, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_RGSR_EL1), undef_access },
{ SYS_DESC(SYS_GCR_EL1), undef_access },
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 20/66] KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
For the same reason we trap virtual memory register accesses in virtual
EL2, we trap CPACR_EL1 access too; We allow the virtual EL2 mode to
access EL1 system register state instead of the virtual EL2 one.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 3 +++
arch/arm64/kvm/sys_regs.c | 2 +-
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4cf745f79985..3bd29fd01d80 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -269,7 +269,7 @@
#define CPTR_EL2_TFP_SHIFT 10
/* Hyp Coprocessor Trap Register */
-#define CPTR_EL2_TCPAC (1 << 31)
+#define CPTR_EL2_TCPAC (1U << 31)
#define CPTR_EL2_TAM (1 << 30)
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 3d8a4402b52c..66358f8ec5ad 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -68,6 +68,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
__activate_traps_fpsimd32(vcpu);
}
+ if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
+ val |= CPTR_EL2_TCPAC;
+
write_sysreg(val, cpacr_el1);
write_sysreg(__this_cpu_read(kvm_hyp_vector), vbar_el1);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5185f148f25e..4410d399f8e2 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1736,7 +1736,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 },
- { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
+ { SYS_DESC(SYS_CPACR_EL1), access_rw, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_RGSR_EL1), undef_access },
{ SYS_DESC(SYS_GCR_EL1), undef_access },
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 21/66] KVM: arm64: nv: Handle PSCI call via smc from the guest
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
VMs used to execute hvc #0 for the psci call if EL3 is not implemented.
However, when we come to provide the virtual EL2 mode to the VM, the
host OS inside the VM calls kvm_call_hyp() which is also hvc #0. So,
it's hard to differentiate between them from the host hypervisor's point
of view.
So, let the VM execute smc instruction for the psci call. On ARMv8.3,
even if EL3 is not implemented, a smc instruction executed at non-secure
EL1 is trapped to EL2 if HCR_EL2.TSC==1, rather than being treated as
UNDEFINED. So, the host hypervisor can handle this psci call without any
confusion.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/handle_exit.c | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 56f8a77b9d4a..fe1dc0551991 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -62,6 +62,8 @@ static int handle_hvc(struct kvm_vcpu *vcpu)
static int handle_smc(struct kvm_vcpu *vcpu)
{
+ int ret;
+
/*
* "If an SMC instruction executed at Non-secure EL1 is
* trapped to EL2 because HCR_EL2.TSC is 1, the exception is a
@@ -69,10 +71,28 @@ static int handle_smc(struct kvm_vcpu *vcpu)
*
* We need to advance the PC after the trap, as it would
* otherwise return to the same address...
+ *
+ * If imm is non-zero, it's not defined, so just skip it.
+ */
+ if (kvm_vcpu_hvc_get_imm(vcpu)) {
+ vcpu_set_reg(vcpu, 0, ~0UL);
+ kvm_incr_pc(vcpu);
+ return 1;
+ }
+
+ /*
+ * If imm is zero, it's a psci call.
+ * Note that on ARMv8.3, even if EL3 is not implemented, SMC executed
+ * at Non-secure EL1 is trapped to EL2 if HCR_EL2.TSC==1, rather than
+ * being treated as UNDEFINED.
*/
- vcpu_set_reg(vcpu, 0, ~0UL);
+ ret = kvm_hvc_call_handler(vcpu);
+ if (ret < 0)
+ vcpu_set_reg(vcpu, 0, ~0UL);
+
kvm_incr_pc(vcpu);
- return 1;
+
+ return ret;
}
/*
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 21/66] KVM: arm64: nv: Handle PSCI call via smc from the guest
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
VMs used to execute hvc #0 for the psci call if EL3 is not implemented.
However, when we come to provide the virtual EL2 mode to the VM, the
host OS inside the VM calls kvm_call_hyp() which is also hvc #0. So,
it's hard to differentiate between them from the host hypervisor's point
of view.
So, let the VM execute smc instruction for the psci call. On ARMv8.3,
even if EL3 is not implemented, a smc instruction executed at non-secure
EL1 is trapped to EL2 if HCR_EL2.TSC==1, rather than being treated as
UNDEFINED. So, the host hypervisor can handle this psci call without any
confusion.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/handle_exit.c | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 56f8a77b9d4a..fe1dc0551991 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -62,6 +62,8 @@ static int handle_hvc(struct kvm_vcpu *vcpu)
static int handle_smc(struct kvm_vcpu *vcpu)
{
+ int ret;
+
/*
* "If an SMC instruction executed at Non-secure EL1 is
* trapped to EL2 because HCR_EL2.TSC is 1, the exception is a
@@ -69,10 +71,28 @@ static int handle_smc(struct kvm_vcpu *vcpu)
*
* We need to advance the PC after the trap, as it would
* otherwise return to the same address...
+ *
+ * If imm is non-zero, it's not defined, so just skip it.
+ */
+ if (kvm_vcpu_hvc_get_imm(vcpu)) {
+ vcpu_set_reg(vcpu, 0, ~0UL);
+ kvm_incr_pc(vcpu);
+ return 1;
+ }
+
+ /*
+ * If imm is zero, it's a psci call.
+ * Note that on ARMv8.3, even if EL3 is not implemented, SMC executed
+ * at Non-secure EL1 is trapped to EL2 if HCR_EL2.TSC==1, rather than
+ * being treated as UNDEFINED.
*/
- vcpu_set_reg(vcpu, 0, ~0UL);
+ ret = kvm_hvc_call_handler(vcpu);
+ if (ret < 0)
+ vcpu_set_reg(vcpu, 0, ~0UL);
+
kvm_incr_pc(vcpu);
- return 1;
+
+ return ret;
}
/*
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 21/66] KVM: arm64: nv: Handle PSCI call via smc from the guest
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
VMs used to execute hvc #0 for the psci call if EL3 is not implemented.
However, when we come to provide the virtual EL2 mode to the VM, the
host OS inside the VM calls kvm_call_hyp() which is also hvc #0. So,
it's hard to differentiate between them from the host hypervisor's point
of view.
So, let the VM execute smc instruction for the psci call. On ARMv8.3,
even if EL3 is not implemented, a smc instruction executed at non-secure
EL1 is trapped to EL2 if HCR_EL2.TSC==1, rather than being treated as
UNDEFINED. So, the host hypervisor can handle this psci call without any
confusion.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/handle_exit.c | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 56f8a77b9d4a..fe1dc0551991 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -62,6 +62,8 @@ static int handle_hvc(struct kvm_vcpu *vcpu)
static int handle_smc(struct kvm_vcpu *vcpu)
{
+ int ret;
+
/*
* "If an SMC instruction executed at Non-secure EL1 is
* trapped to EL2 because HCR_EL2.TSC is 1, the exception is a
@@ -69,10 +71,28 @@ static int handle_smc(struct kvm_vcpu *vcpu)
*
* We need to advance the PC after the trap, as it would
* otherwise return to the same address...
+ *
+ * If imm is non-zero, it's not defined, so just skip it.
+ */
+ if (kvm_vcpu_hvc_get_imm(vcpu)) {
+ vcpu_set_reg(vcpu, 0, ~0UL);
+ kvm_incr_pc(vcpu);
+ return 1;
+ }
+
+ /*
+ * If imm is zero, it's a psci call.
+ * Note that on ARMv8.3, even if EL3 is not implemented, SMC executed
+ * at Non-secure EL1 is trapped to EL2 if HCR_EL2.TSC==1, rather than
+ * being treated as UNDEFINED.
*/
- vcpu_set_reg(vcpu, 0, ~0UL);
+ ret = kvm_hvc_call_handler(vcpu);
+ if (ret < 0)
+ vcpu_set_reg(vcpu, 0, ~0UL);
+
kvm_incr_pc(vcpu);
- return 1;
+
+ return ret;
}
/*
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 22/66] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward exceptions due to WFI or WFE instructions to the virtual EL2 if
they are not coming from the virtual EL2 and virtual HCR_EL2.TWX is set.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/handle_exit.c | 11 +++++++-
arch/arm64/kvm/nested.c | 40 +++++++++++++++++++++++++++++
4 files changed, 53 insertions(+), 2 deletions(-)
create mode 100644 arch/arm64/kvm/nested.c
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 67a2c0d05233..4c2ac9650a3e 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -61,4 +61,6 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
(cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
}
+int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 672acdf1e993..7bb1d1314cad 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,7 +17,7 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
- emulate-nested.o \
+ emulate-nested.o nested.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index fe1dc0551991..a2437b2380f1 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -119,7 +119,16 @@ static int handle_no_fpsimd(struct kvm_vcpu *vcpu)
*/
static int kvm_handle_wfx(struct kvm_vcpu *vcpu)
{
- if (kvm_vcpu_get_esr(vcpu) & ESR_ELx_WFx_ISS_WFE) {
+ bool is_wfe = !!(kvm_vcpu_get_esr(vcpu) & ESR_ELx_WFx_ISS_WFE);
+
+ if (nested_virt_in_use(vcpu)) {
+ int ret = handle_wfx_nested(vcpu, is_wfe);
+
+ if (ret != -EINVAL)
+ return ret;
+ }
+
+ if (is_wfe) {
trace_kvm_wfx_arm64(*vcpu_pc(vcpu), true);
vcpu->stat.wfe_exit_stat++;
kvm_vcpu_on_spin(vcpu, vcpu_mode_priv(vcpu));
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
new file mode 100644
index 000000000000..42a96c8d2adc
--- /dev/null
+++ b/arch/arm64/kvm/nested.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 - Columbia University and Linaro Ltd.
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_emulate.h>
+
+/*
+ * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
+ * the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
+ * handle this.
+ */
+int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
+{
+ u64 hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+
+ if (vcpu_mode_el2(vcpu))
+ return -EINVAL;
+
+ if ((is_wfe && (hcr_el2 & HCR_TWE)) || (!is_wfe && (hcr_el2 & HCR_TWI)))
+ return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ return -EINVAL;
+}
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 22/66] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward exceptions due to WFI or WFE instructions to the virtual EL2 if
they are not coming from the virtual EL2 and virtual HCR_EL2.TWX is set.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/handle_exit.c | 11 +++++++-
arch/arm64/kvm/nested.c | 40 +++++++++++++++++++++++++++++
4 files changed, 53 insertions(+), 2 deletions(-)
create mode 100644 arch/arm64/kvm/nested.c
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 67a2c0d05233..4c2ac9650a3e 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -61,4 +61,6 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
(cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
}
+int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 672acdf1e993..7bb1d1314cad 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,7 +17,7 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
- emulate-nested.o \
+ emulate-nested.o nested.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index fe1dc0551991..a2437b2380f1 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -119,7 +119,16 @@ static int handle_no_fpsimd(struct kvm_vcpu *vcpu)
*/
static int kvm_handle_wfx(struct kvm_vcpu *vcpu)
{
- if (kvm_vcpu_get_esr(vcpu) & ESR_ELx_WFx_ISS_WFE) {
+ bool is_wfe = !!(kvm_vcpu_get_esr(vcpu) & ESR_ELx_WFx_ISS_WFE);
+
+ if (nested_virt_in_use(vcpu)) {
+ int ret = handle_wfx_nested(vcpu, is_wfe);
+
+ if (ret != -EINVAL)
+ return ret;
+ }
+
+ if (is_wfe) {
trace_kvm_wfx_arm64(*vcpu_pc(vcpu), true);
vcpu->stat.wfe_exit_stat++;
kvm_vcpu_on_spin(vcpu, vcpu_mode_priv(vcpu));
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
new file mode 100644
index 000000000000..42a96c8d2adc
--- /dev/null
+++ b/arch/arm64/kvm/nested.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 - Columbia University and Linaro Ltd.
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_emulate.h>
+
+/*
+ * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
+ * the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
+ * handle this.
+ */
+int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
+{
+ u64 hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+
+ if (vcpu_mode_el2(vcpu))
+ return -EINVAL;
+
+ if ((is_wfe && (hcr_el2 & HCR_TWE)) || (!is_wfe && (hcr_el2 & HCR_TWI)))
+ return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ return -EINVAL;
+}
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 22/66] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
Forward exceptions due to WFI or WFE instructions to the virtual EL2 if
they are not coming from the virtual EL2 and virtual HCR_EL2.TWX is set.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/handle_exit.c | 11 +++++++-
arch/arm64/kvm/nested.c | 40 +++++++++++++++++++++++++++++
4 files changed, 53 insertions(+), 2 deletions(-)
create mode 100644 arch/arm64/kvm/nested.c
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 67a2c0d05233..4c2ac9650a3e 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -61,4 +61,6 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
(cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
}
+int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 672acdf1e993..7bb1d1314cad 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,7 +17,7 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
- emulate-nested.o \
+ emulate-nested.o nested.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index fe1dc0551991..a2437b2380f1 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -119,7 +119,16 @@ static int handle_no_fpsimd(struct kvm_vcpu *vcpu)
*/
static int kvm_handle_wfx(struct kvm_vcpu *vcpu)
{
- if (kvm_vcpu_get_esr(vcpu) & ESR_ELx_WFx_ISS_WFE) {
+ bool is_wfe = !!(kvm_vcpu_get_esr(vcpu) & ESR_ELx_WFx_ISS_WFE);
+
+ if (nested_virt_in_use(vcpu)) {
+ int ret = handle_wfx_nested(vcpu, is_wfe);
+
+ if (ret != -EINVAL)
+ return ret;
+ }
+
+ if (is_wfe) {
trace_kvm_wfx_arm64(*vcpu_pc(vcpu), true);
vcpu->stat.wfe_exit_stat++;
kvm_vcpu_on_spin(vcpu, vcpu_mode_priv(vcpu));
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
new file mode 100644
index 000000000000..42a96c8d2adc
--- /dev/null
+++ b/arch/arm64/kvm/nested.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 - Columbia University and Linaro Ltd.
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_emulate.h>
+
+/*
+ * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
+ * the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
+ * handle this.
+ */
+int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
+{
+ u64 hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+
+ if (vcpu_mode_el2(vcpu))
+ return -EINVAL;
+
+ if ((is_wfe && (hcr_el2 & HCR_TWE)) || (!is_wfe && (hcr_el2 & HCR_TWI)))
+ return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ return -EINVAL;
+}
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 23/66] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward traps due to FP/ASIMD register accesses to the virtual EL2
if virtual CPTR_EL2.TFP is set (with HCR_EL2.E2H == 0) or
CPTR_EL2.FPEN is configure to do so (with HCR_EL2.E2h == 1).
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: account for HCR_EL2.E2H when testing for TFP/FPEN, with
all the hard work actually being done by Chase Conklin]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 26 +++++++++++++++++++++++++
arch/arm64/kvm/handle_exit.c | 16 +++++++++++----
arch/arm64/kvm/hyp/include/hyp/switch.h | 11 +++++++++--
3 files changed, 47 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5c7dc659e8c4..a000e6e05091 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -11,12 +11,14 @@
#ifndef __ARM64_KVM_EMULATE_H__
#define __ARM64_KVM_EMULATE_H__
+#include <linux/bitfield.h>
#include <linux/kvm_host.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/kvm_arm.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
#include <asm/ptrace.h>
#include <asm/cputype.h>
#include <asm/virt.h>
@@ -321,6 +323,30 @@ static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
return mode != PSR_MODE_EL0t;
}
+static inline bool guest_hyp_fpsimd_traps_enabled(const struct kvm_vcpu *vcpu)
+{
+ u64 val;
+
+ if (!nested_virt_in_use(vcpu))
+ return false;
+
+ val = vcpu_read_sys_reg(vcpu, CPTR_EL2);
+
+ if (!vcpu_el2_e2h_is_set(vcpu))
+ return (val & CPTR_EL2_TFP);
+
+ switch (FIELD_GET(CPACR_EL1_FPEN, val)) {
+ case 0b00:
+ case 0b10:
+ return true;
+ case 0b01:
+ return vcpu_el2_tge_is_set(vcpu) && !vcpu_mode_el2(vcpu);
+ case 0b11:
+ default: /* GCC is dumb */
+ return false;
+ }
+}
+
static __always_inline u32 kvm_vcpu_get_esr(const struct kvm_vcpu *vcpu)
{
return vcpu->arch.fault.esr_el2;
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index a2437b2380f1..5276e8563367 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -96,11 +96,19 @@ static int handle_smc(struct kvm_vcpu *vcpu)
}
/*
- * Guest access to FP/ASIMD registers are routed to this handler only
- * when the system doesn't support FP/ASIMD.
+ * This handles the cases where the system does not support FP/ASIMD or when
+ * we are running nested virtualization and the guest hypervisor is trapping
+ * FP/ASIMD accesses by its guest guest.
+ *
+ * All other handling of guest vs. host FP/ASIMD register state is handled in
+ * fixup_guest_exit().
*/
-static int handle_no_fpsimd(struct kvm_vcpu *vcpu)
+static int kvm_handle_fpasimd(struct kvm_vcpu *vcpu)
{
+ if (guest_hyp_fpsimd_traps_enabled(vcpu))
+ return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ /* This is the case when the system doesn't support FP/ASIMD. */
kvm_inject_undefined(vcpu);
return 1;
}
@@ -243,7 +251,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_BREAKPT_LOW]= kvm_handle_guest_debug,
[ESR_ELx_EC_BKPT32] = kvm_handle_guest_debug,
[ESR_ELx_EC_BRK64] = kvm_handle_guest_debug,
- [ESR_ELx_EC_FP_ASIMD] = handle_no_fpsimd,
+ [ESR_ELx_EC_FP_ASIMD] = kvm_handle_fpasimd,
[ESR_ELx_EC_PAC] = kvm_handle_ptrauth,
};
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 424b074e4377..e5e201314c87 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -223,11 +223,18 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
esr_ec != ESR_ELx_EC_SVE)
return false;
- /* Don't handle SVE traps for non-SVE vcpus here: */
- if (!sve_guest)
+ /*
+ * Don't handle SVE traps for non-SVE vcpus here. This
+ * includes NV guests for the time being.
+ */
+ if (!sve_guest) {
if (esr_ec != ESR_ELx_EC_FP_ASIMD)
return false;
+ if (guest_hyp_fpsimd_traps_enabled(vcpu))
+ return false;
+ }
+
/* Valid trap. Switch the context: */
if (vhe) {
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 23/66] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP, FPEN} settings
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward traps due to FP/ASIMD register accesses to the virtual EL2
if virtual CPTR_EL2.TFP is set (with HCR_EL2.E2H == 0) or
CPTR_EL2.FPEN is configure to do so (with HCR_EL2.E2h == 1).
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: account for HCR_EL2.E2H when testing for TFP/FPEN, with
all the hard work actually being done by Chase Conklin]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 26 +++++++++++++++++++++++++
arch/arm64/kvm/handle_exit.c | 16 +++++++++++----
arch/arm64/kvm/hyp/include/hyp/switch.h | 11 +++++++++--
3 files changed, 47 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5c7dc659e8c4..a000e6e05091 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -11,12 +11,14 @@
#ifndef __ARM64_KVM_EMULATE_H__
#define __ARM64_KVM_EMULATE_H__
+#include <linux/bitfield.h>
#include <linux/kvm_host.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/kvm_arm.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
#include <asm/ptrace.h>
#include <asm/cputype.h>
#include <asm/virt.h>
@@ -321,6 +323,30 @@ static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
return mode != PSR_MODE_EL0t;
}
+static inline bool guest_hyp_fpsimd_traps_enabled(const struct kvm_vcpu *vcpu)
+{
+ u64 val;
+
+ if (!nested_virt_in_use(vcpu))
+ return false;
+
+ val = vcpu_read_sys_reg(vcpu, CPTR_EL2);
+
+ if (!vcpu_el2_e2h_is_set(vcpu))
+ return (val & CPTR_EL2_TFP);
+
+ switch (FIELD_GET(CPACR_EL1_FPEN, val)) {
+ case 0b00:
+ case 0b10:
+ return true;
+ case 0b01:
+ return vcpu_el2_tge_is_set(vcpu) && !vcpu_mode_el2(vcpu);
+ case 0b11:
+ default: /* GCC is dumb */
+ return false;
+ }
+}
+
static __always_inline u32 kvm_vcpu_get_esr(const struct kvm_vcpu *vcpu)
{
return vcpu->arch.fault.esr_el2;
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index a2437b2380f1..5276e8563367 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -96,11 +96,19 @@ static int handle_smc(struct kvm_vcpu *vcpu)
}
/*
- * Guest access to FP/ASIMD registers are routed to this handler only
- * when the system doesn't support FP/ASIMD.
+ * This handles the cases where the system does not support FP/ASIMD or when
+ * we are running nested virtualization and the guest hypervisor is trapping
+ * FP/ASIMD accesses by its guest guest.
+ *
+ * All other handling of guest vs. host FP/ASIMD register state is handled in
+ * fixup_guest_exit().
*/
-static int handle_no_fpsimd(struct kvm_vcpu *vcpu)
+static int kvm_handle_fpasimd(struct kvm_vcpu *vcpu)
{
+ if (guest_hyp_fpsimd_traps_enabled(vcpu))
+ return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ /* This is the case when the system doesn't support FP/ASIMD. */
kvm_inject_undefined(vcpu);
return 1;
}
@@ -243,7 +251,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_BREAKPT_LOW]= kvm_handle_guest_debug,
[ESR_ELx_EC_BKPT32] = kvm_handle_guest_debug,
[ESR_ELx_EC_BRK64] = kvm_handle_guest_debug,
- [ESR_ELx_EC_FP_ASIMD] = handle_no_fpsimd,
+ [ESR_ELx_EC_FP_ASIMD] = kvm_handle_fpasimd,
[ESR_ELx_EC_PAC] = kvm_handle_ptrauth,
};
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 424b074e4377..e5e201314c87 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -223,11 +223,18 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
esr_ec != ESR_ELx_EC_SVE)
return false;
- /* Don't handle SVE traps for non-SVE vcpus here: */
- if (!sve_guest)
+ /*
+ * Don't handle SVE traps for non-SVE vcpus here. This
+ * includes NV guests for the time being.
+ */
+ if (!sve_guest) {
if (esr_ec != ESR_ELx_EC_FP_ASIMD)
return false;
+ if (guest_hyp_fpsimd_traps_enabled(vcpu))
+ return false;
+ }
+
/* Valid trap. Switch the context: */
if (vhe) {
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 23/66] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP, FPEN} settings
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
Forward traps due to FP/ASIMD register accesses to the virtual EL2
if virtual CPTR_EL2.TFP is set (with HCR_EL2.E2H == 0) or
CPTR_EL2.FPEN is configure to do so (with HCR_EL2.E2h == 1).
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: account for HCR_EL2.E2H when testing for TFP/FPEN, with
all the hard work actually being done by Chase Conklin]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 26 +++++++++++++++++++++++++
arch/arm64/kvm/handle_exit.c | 16 +++++++++++----
arch/arm64/kvm/hyp/include/hyp/switch.h | 11 +++++++++--
3 files changed, 47 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5c7dc659e8c4..a000e6e05091 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -11,12 +11,14 @@
#ifndef __ARM64_KVM_EMULATE_H__
#define __ARM64_KVM_EMULATE_H__
+#include <linux/bitfield.h>
#include <linux/kvm_host.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/kvm_arm.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
#include <asm/ptrace.h>
#include <asm/cputype.h>
#include <asm/virt.h>
@@ -321,6 +323,30 @@ static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
return mode != PSR_MODE_EL0t;
}
+static inline bool guest_hyp_fpsimd_traps_enabled(const struct kvm_vcpu *vcpu)
+{
+ u64 val;
+
+ if (!nested_virt_in_use(vcpu))
+ return false;
+
+ val = vcpu_read_sys_reg(vcpu, CPTR_EL2);
+
+ if (!vcpu_el2_e2h_is_set(vcpu))
+ return (val & CPTR_EL2_TFP);
+
+ switch (FIELD_GET(CPACR_EL1_FPEN, val)) {
+ case 0b00:
+ case 0b10:
+ return true;
+ case 0b01:
+ return vcpu_el2_tge_is_set(vcpu) && !vcpu_mode_el2(vcpu);
+ case 0b11:
+ default: /* GCC is dumb */
+ return false;
+ }
+}
+
static __always_inline u32 kvm_vcpu_get_esr(const struct kvm_vcpu *vcpu)
{
return vcpu->arch.fault.esr_el2;
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index a2437b2380f1..5276e8563367 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -96,11 +96,19 @@ static int handle_smc(struct kvm_vcpu *vcpu)
}
/*
- * Guest access to FP/ASIMD registers are routed to this handler only
- * when the system doesn't support FP/ASIMD.
+ * This handles the cases where the system does not support FP/ASIMD or when
+ * we are running nested virtualization and the guest hypervisor is trapping
+ * FP/ASIMD accesses by its guest guest.
+ *
+ * All other handling of guest vs. host FP/ASIMD register state is handled in
+ * fixup_guest_exit().
*/
-static int handle_no_fpsimd(struct kvm_vcpu *vcpu)
+static int kvm_handle_fpasimd(struct kvm_vcpu *vcpu)
{
+ if (guest_hyp_fpsimd_traps_enabled(vcpu))
+ return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+
+ /* This is the case when the system doesn't support FP/ASIMD. */
kvm_inject_undefined(vcpu);
return 1;
}
@@ -243,7 +251,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_BREAKPT_LOW]= kvm_handle_guest_debug,
[ESR_ELx_EC_BKPT32] = kvm_handle_guest_debug,
[ESR_ELx_EC_BRK64] = kvm_handle_guest_debug,
- [ESR_ELx_EC_FP_ASIMD] = handle_no_fpsimd,
+ [ESR_ELx_EC_FP_ASIMD] = kvm_handle_fpasimd,
[ESR_ELx_EC_PAC] = kvm_handle_ptrauth,
};
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 424b074e4377..e5e201314c87 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -223,11 +223,18 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
esr_ec != ESR_ELx_EC_SVE)
return false;
- /* Don't handle SVE traps for non-SVE vcpus here: */
- if (!sve_guest)
+ /*
+ * Don't handle SVE traps for non-SVE vcpus here. This
+ * includes NV guests for the time being.
+ */
+ if (!sve_guest) {
if (esr_ec != ESR_ELx_EC_FP_ASIMD)
return false;
+ if (guest_hyp_fpsimd_traps_enabled(vcpu))
+ return false;
+ }
+
/* Valid trap. Switch the context: */
if (vhe) {
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 24/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward traps due to HCR_EL2.NV bit to the virtual EL2 if they are not
coming from the virtual EL2 and the virtual HCR_EL2.NV bit is set.
In addition to EL2 register accesses, setting NV bit will also make EL12
register accesses trap to EL2. To emulate this for the virtual EL2,
forword traps due to EL12 register accessses to the virtual EL2 if the
virtual HCR_EL2.NV bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[Moved code to emulate-nested.c]
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/emulate-nested.c | 27 +++++++++++++++++++++++++++
arch/arm64/kvm/handle_exit.c | 7 +++++++
arch/arm64/kvm/sys_regs.c | 24 ++++++++++++++++++++++++
5 files changed, 61 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3bd29fd01d80..c18edb1b35bf 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
#define HCR_APK (UL(1) << 40)
#define HCR_TEA (UL(1) << 37)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 4c2ac9650a3e..26cba7b4d743 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -62,5 +62,7 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
}
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
+extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index ee91bcd925d8..feb9b5eded96 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -25,11 +25,38 @@
#include "trace.h"
+bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+{
+ bool control_bit_set;
+
+ if (!nested_virt_in_use(vcpu))
+ return false;
+
+ control_bit_set = __vcpu_sys_reg(vcpu, HCR_EL2) & control_bit;
+ if (!vcpu_mode_el2(vcpu) && control_bit_set) {
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+ return true;
+ }
+ return false;
+}
+
+bool forward_nv_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_NV);
+}
+
void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
{
u64 spsr, elr, mode;
bool direct_eret;
+ /*
+ * Forward this trap to the virtual EL2 if the virtual
+ * HCR_EL2.NV bit is set and this is coming from !EL2.
+ */
+ if (forward_nv_traps(vcpu))
+ return;
+
/*
* Going through the whole put/load motions is a waste of time
* if this is a VHE guest hypervisor returning to its own
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 5276e8563367..0ad20457d0f1 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -64,6 +64,13 @@ static int handle_smc(struct kvm_vcpu *vcpu)
{
int ret;
+ /*
+ * Forward this trapped smc instruction to the virtual EL2 if
+ * the guest has asked for it.
+ */
+ if (forward_traps(vcpu, HCR_TSC))
+ return 1;
+
/*
* "If an SMC instruction executed at Non-secure EL1 is
* trapped to EL2 because HCR_EL2.TSC is 1, the exception is a
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 4410d399f8e2..8b0be7f0f44d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -267,10 +267,19 @@ static u32 get_ccsidr(u32 csselr)
return ccsidr;
}
+static bool el12_reg(struct sys_reg_params *p)
+{
+ /* All *_EL12 registers have Op1=5. */
+ return (p->Op1 == 5);
+}
+
static bool access_rw(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, r->reg);
else
@@ -294,6 +303,9 @@ static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write) {
u64 val = p->regval;
@@ -378,6 +390,9 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
bool was_enabled = vcpu_has_cache_enabled(vcpu);
u64 val, mask, shift;
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
/* We don't expect TRVM on the host */
BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
@@ -1571,6 +1586,9 @@ static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
else
@@ -1583,6 +1601,9 @@ static bool access_spsr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
__vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
else
@@ -1595,6 +1616,9 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
else
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 24/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward traps due to HCR_EL2.NV bit to the virtual EL2 if they are not
coming from the virtual EL2 and the virtual HCR_EL2.NV bit is set.
In addition to EL2 register accesses, setting NV bit will also make EL12
register accesses trap to EL2. To emulate this for the virtual EL2,
forword traps due to EL12 register accessses to the virtual EL2 if the
virtual HCR_EL2.NV bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[Moved code to emulate-nested.c]
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/emulate-nested.c | 27 +++++++++++++++++++++++++++
arch/arm64/kvm/handle_exit.c | 7 +++++++
arch/arm64/kvm/sys_regs.c | 24 ++++++++++++++++++++++++
5 files changed, 61 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3bd29fd01d80..c18edb1b35bf 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
#define HCR_APK (UL(1) << 40)
#define HCR_TEA (UL(1) << 37)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 4c2ac9650a3e..26cba7b4d743 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -62,5 +62,7 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
}
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
+extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index ee91bcd925d8..feb9b5eded96 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -25,11 +25,38 @@
#include "trace.h"
+bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+{
+ bool control_bit_set;
+
+ if (!nested_virt_in_use(vcpu))
+ return false;
+
+ control_bit_set = __vcpu_sys_reg(vcpu, HCR_EL2) & control_bit;
+ if (!vcpu_mode_el2(vcpu) && control_bit_set) {
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+ return true;
+ }
+ return false;
+}
+
+bool forward_nv_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_NV);
+}
+
void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
{
u64 spsr, elr, mode;
bool direct_eret;
+ /*
+ * Forward this trap to the virtual EL2 if the virtual
+ * HCR_EL2.NV bit is set and this is coming from !EL2.
+ */
+ if (forward_nv_traps(vcpu))
+ return;
+
/*
* Going through the whole put/load motions is a waste of time
* if this is a VHE guest hypervisor returning to its own
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 5276e8563367..0ad20457d0f1 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -64,6 +64,13 @@ static int handle_smc(struct kvm_vcpu *vcpu)
{
int ret;
+ /*
+ * Forward this trapped smc instruction to the virtual EL2 if
+ * the guest has asked for it.
+ */
+ if (forward_traps(vcpu, HCR_TSC))
+ return 1;
+
/*
* "If an SMC instruction executed at Non-secure EL1 is
* trapped to EL2 because HCR_EL2.TSC is 1, the exception is a
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 4410d399f8e2..8b0be7f0f44d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -267,10 +267,19 @@ static u32 get_ccsidr(u32 csselr)
return ccsidr;
}
+static bool el12_reg(struct sys_reg_params *p)
+{
+ /* All *_EL12 registers have Op1=5. */
+ return (p->Op1 == 5);
+}
+
static bool access_rw(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, r->reg);
else
@@ -294,6 +303,9 @@ static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write) {
u64 val = p->regval;
@@ -378,6 +390,9 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
bool was_enabled = vcpu_has_cache_enabled(vcpu);
u64 val, mask, shift;
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
/* We don't expect TRVM on the host */
BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
@@ -1571,6 +1586,9 @@ static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
else
@@ -1583,6 +1601,9 @@ static bool access_spsr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
__vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
else
@@ -1595,6 +1616,9 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
else
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 24/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
Forward traps due to HCR_EL2.NV bit to the virtual EL2 if they are not
coming from the virtual EL2 and the virtual HCR_EL2.NV bit is set.
In addition to EL2 register accesses, setting NV bit will also make EL12
register accesses trap to EL2. To emulate this for the virtual EL2,
forword traps due to EL12 register accessses to the virtual EL2 if the
virtual HCR_EL2.NV bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[Moved code to emulate-nested.c]
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/emulate-nested.c | 27 +++++++++++++++++++++++++++
arch/arm64/kvm/handle_exit.c | 7 +++++++
arch/arm64/kvm/sys_regs.c | 24 ++++++++++++++++++++++++
5 files changed, 61 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3bd29fd01d80..c18edb1b35bf 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
#define HCR_APK (UL(1) << 40)
#define HCR_TEA (UL(1) << 37)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 4c2ac9650a3e..26cba7b4d743 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -62,5 +62,7 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
}
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
+extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index ee91bcd925d8..feb9b5eded96 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -25,11 +25,38 @@
#include "trace.h"
+bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+{
+ bool control_bit_set;
+
+ if (!nested_virt_in_use(vcpu))
+ return false;
+
+ control_bit_set = __vcpu_sys_reg(vcpu, HCR_EL2) & control_bit;
+ if (!vcpu_mode_el2(vcpu) && control_bit_set) {
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+ return true;
+ }
+ return false;
+}
+
+bool forward_nv_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_NV);
+}
+
void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
{
u64 spsr, elr, mode;
bool direct_eret;
+ /*
+ * Forward this trap to the virtual EL2 if the virtual
+ * HCR_EL2.NV bit is set and this is coming from !EL2.
+ */
+ if (forward_nv_traps(vcpu))
+ return;
+
/*
* Going through the whole put/load motions is a waste of time
* if this is a VHE guest hypervisor returning to its own
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 5276e8563367..0ad20457d0f1 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -64,6 +64,13 @@ static int handle_smc(struct kvm_vcpu *vcpu)
{
int ret;
+ /*
+ * Forward this trapped smc instruction to the virtual EL2 if
+ * the guest has asked for it.
+ */
+ if (forward_traps(vcpu, HCR_TSC))
+ return 1;
+
/*
* "If an SMC instruction executed at Non-secure EL1 is
* trapped to EL2 because HCR_EL2.TSC is 1, the exception is a
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 4410d399f8e2..8b0be7f0f44d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -267,10 +267,19 @@ static u32 get_ccsidr(u32 csselr)
return ccsidr;
}
+static bool el12_reg(struct sys_reg_params *p)
+{
+ /* All *_EL12 registers have Op1=5. */
+ return (p->Op1 == 5);
+}
+
static bool access_rw(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, r->reg);
else
@@ -294,6 +303,9 @@ static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write) {
u64 val = p->regval;
@@ -378,6 +390,9 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
bool was_enabled = vcpu_has_cache_enabled(vcpu);
u64 val, mask, shift;
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
/* We don't expect TRVM on the host */
BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
@@ -1571,6 +1586,9 @@ static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
else
@@ -1583,6 +1601,9 @@ static bool access_spsr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
__vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
else
@@ -1595,6 +1616,9 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (el12_reg(p) && forward_nv_traps(vcpu))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
else
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 25/66] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward the EL1 virtual memory register traps to the virtual EL2 if they
are not coming from the virtual EL2 and the virtual HCR_EL2.TVM or TRVM
bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 8b0be7f0f44d..d09fda0fd76f 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -393,6 +393,13 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p)) {
+ u64 bit = p->is_write ? HCR_TVM : HCR_TRVM;
+
+ if (forward_traps(vcpu, bit))
+ return false;
+ }
+
/* We don't expect TRVM on the host */
BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 25/66] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
Forward the EL1 virtual memory register traps to the virtual EL2 if they
are not coming from the virtual EL2 and the virtual HCR_EL2.TVM or TRVM
bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 8b0be7f0f44d..d09fda0fd76f 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -393,6 +393,13 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p)) {
+ u64 bit = p->is_write ? HCR_TVM : HCR_TRVM;
+
+ if (forward_traps(vcpu, bit))
+ return false;
+ }
+
/* We don't expect TRVM on the host */
BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 25/66] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
Forward the EL1 virtual memory register traps to the virtual EL2 if they
are not coming from the virtual EL2 and the virtual HCR_EL2.TVM or TRVM
bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 8b0be7f0f44d..d09fda0fd76f 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -393,6 +393,13 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p)) {
+ u64 bit = p->is_write ? HCR_TVM : HCR_TRVM;
+
+ if (forward_traps(vcpu, bit))
+ return false;
+ }
+
/* We don't expect TRVM on the host */
BUG_ON(!vcpu_mode_el2(vcpu) && !p->is_write);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 26/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Jintack Lim <jintack@cs.columbia.edu>
Forward ELR_EL1, SPSR_EL1 and VBAR_EL1 traps to the virtual EL2 if the
virtual HCR_EL2.NV bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/kvm/sys_regs.c | 28 +++++++++++++++++++++++++++-
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index c18edb1b35bf..3d2abc40d3a2 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
#define HCR_APK (UL(1) << 40)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d09fda0fd76f..ea511e569ee0 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -299,6 +299,22 @@ static bool access_wi(struct kvm_vcpu *vcpu,
return true;
}
+/* This function is to support the recursive nested virtualization */
+static bool forward_nv1_traps(struct kvm_vcpu *vcpu, struct sys_reg_params *p)
+{
+ return forward_traps(vcpu, HCR_NV1);
+}
+
+static bool access_vbar_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (forward_nv1_traps(vcpu, p))
+ return false;
+
+ return access_rw(vcpu, p, r);
+}
+
static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1589,6 +1605,7 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1596,6 +1613,9 @@ static bool access_elr(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
else
@@ -1611,6 +1631,9 @@ static bool access_spsr(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
__vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
else
@@ -1626,6 +1649,9 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
else
@@ -1817,7 +1843,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_LORC_EL1), trap_loregion },
{ SYS_DESC(SYS_LORID_EL1), trap_loregion },
- { SYS_DESC(SYS_VBAR_EL1), access_rw, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_VBAR_EL1), access_vbar_el1, reset_val, VBAR_EL1, 0 },
{ SYS_DESC(SYS_DISR_EL1), NULL, reset_val, DISR_EL1, 0 },
{ SYS_DESC(SYS_ICC_IAR0_EL1), write_to_read_only },
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 26/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Jintack Lim <jintack@cs.columbia.edu>
Forward ELR_EL1, SPSR_EL1 and VBAR_EL1 traps to the virtual EL2 if the
virtual HCR_EL2.NV bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/kvm/sys_regs.c | 28 +++++++++++++++++++++++++++-
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index c18edb1b35bf..3d2abc40d3a2 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
#define HCR_APK (UL(1) << 40)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d09fda0fd76f..ea511e569ee0 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -299,6 +299,22 @@ static bool access_wi(struct kvm_vcpu *vcpu,
return true;
}
+/* This function is to support the recursive nested virtualization */
+static bool forward_nv1_traps(struct kvm_vcpu *vcpu, struct sys_reg_params *p)
+{
+ return forward_traps(vcpu, HCR_NV1);
+}
+
+static bool access_vbar_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (forward_nv1_traps(vcpu, p))
+ return false;
+
+ return access_rw(vcpu, p, r);
+}
+
static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1589,6 +1605,7 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1596,6 +1613,9 @@ static bool access_elr(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
else
@@ -1611,6 +1631,9 @@ static bool access_spsr(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
__vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
else
@@ -1626,6 +1649,9 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
else
@@ -1817,7 +1843,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_LORC_EL1), trap_loregion },
{ SYS_DESC(SYS_LORID_EL1), trap_loregion },
- { SYS_DESC(SYS_VBAR_EL1), access_rw, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_VBAR_EL1), access_vbar_el1, reset_val, VBAR_EL1, 0 },
{ SYS_DESC(SYS_DISR_EL1), NULL, reset_val, DISR_EL1, 0 },
{ SYS_DESC(SYS_ICC_IAR0_EL1), write_to_read_only },
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 26/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Jintack Lim <jintack@cs.columbia.edu>
Forward ELR_EL1, SPSR_EL1 and VBAR_EL1 traps to the virtual EL2 if the
virtual HCR_EL2.NV bit is set.
This is for recursive nested virtualization.
Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/kvm/sys_regs.c | 28 +++++++++++++++++++++++++++-
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index c18edb1b35bf..3d2abc40d3a2 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
#define HCR_APK (UL(1) << 40)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d09fda0fd76f..ea511e569ee0 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -299,6 +299,22 @@ static bool access_wi(struct kvm_vcpu *vcpu,
return true;
}
+/* This function is to support the recursive nested virtualization */
+static bool forward_nv1_traps(struct kvm_vcpu *vcpu, struct sys_reg_params *p)
+{
+ return forward_traps(vcpu, HCR_NV1);
+}
+
+static bool access_vbar_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (forward_nv1_traps(vcpu, p))
+ return false;
+
+ return access_rw(vcpu, p, r);
+}
+
static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1589,6 +1605,7 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -1596,6 +1613,9 @@ static bool access_elr(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, ELR_EL1);
else
@@ -1611,6 +1631,9 @@ static bool access_spsr(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
__vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval;
else
@@ -1626,6 +1649,9 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
if (el12_reg(p) && forward_nv_traps(vcpu))
return false;
+ if (!el12_reg(p) && forward_nv1_traps(vcpu, p))
+ return false;
+
if (p->is_write)
vcpu_write_sys_reg(vcpu, p->regval, SPSR_EL2);
else
@@ -1817,7 +1843,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_LORC_EL1), trap_loregion },
{ SYS_DESC(SYS_LORID_EL1), trap_loregion },
- { SYS_DESC(SYS_VBAR_EL1), access_rw, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_VBAR_EL1), access_vbar_el1, reset_val, VBAR_EL1, 0 },
{ SYS_DESC(SYS_DISR_EL1), NULL, reset_val, DISR_EL1, 0 },
{ SYS_DESC(SYS_ICC_IAR0_EL1), write_to_read_only },
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 27/66] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
With HCR_EL2.NV bit set, accesses to EL12 registers in the virtual EL2
trap to EL2. Handle those traps just like we do for EL1 registers.
One exception is CNTKCTL_EL12. We don't trap on CNTKCTL_EL1 for non-VHE
virtual EL2 because we don't have to. However, accessing CNTKCTL_EL12
will trap since it's one of the EL12 registers controlled by HCR_EL2.NV
bit. Therefore, add a handler for it and don't treat it as a
non-trap-registers when preparing a shadow context.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index ea511e569ee0..dc8a33ebad5f 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2085,6 +2085,23 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+ { SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
+ { SYS_DESC(SYS_CPACR_EL12), access_rw, reset_val, CPACR_EL1, 0 },
+ { SYS_DESC(SYS_TTBR0_EL12), access_vm_reg, reset_unknown, TTBR0_EL1 },
+ { SYS_DESC(SYS_TTBR1_EL12), access_vm_reg, reset_unknown, TTBR1_EL1 },
+ { SYS_DESC(SYS_TCR_EL12), access_vm_reg, reset_val, TCR_EL1, 0 },
+ { SYS_DESC(SYS_SPSR_EL12), access_spsr},
+ { SYS_DESC(SYS_ELR_EL12), access_elr},
+ { SYS_DESC(SYS_AFSR0_EL12), access_vm_reg, reset_unknown, AFSR0_EL1 },
+ { SYS_DESC(SYS_AFSR1_EL12), access_vm_reg, reset_unknown, AFSR1_EL1 },
+ { SYS_DESC(SYS_ESR_EL12), access_vm_reg, reset_unknown, ESR_EL1 },
+ { SYS_DESC(SYS_FAR_EL12), access_vm_reg, reset_unknown, FAR_EL1 },
+ { SYS_DESC(SYS_MAIR_EL12), access_vm_reg, reset_unknown, MAIR_EL1 },
+ { SYS_DESC(SYS_AMAIR_EL12), access_vm_reg, reset_amair_el1, AMAIR_EL1 },
+ { SYS_DESC(SYS_VBAR_EL12), access_rw, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_CONTEXTIDR_EL12), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
+ { SYS_DESC(SYS_CNTKCTL_EL12), access_rw, reset_val, CNTKCTL_EL1, 0 },
+
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 27/66] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
With HCR_EL2.NV bit set, accesses to EL12 registers in the virtual EL2
trap to EL2. Handle those traps just like we do for EL1 registers.
One exception is CNTKCTL_EL12. We don't trap on CNTKCTL_EL1 for non-VHE
virtual EL2 because we don't have to. However, accessing CNTKCTL_EL12
will trap since it's one of the EL12 registers controlled by HCR_EL2.NV
bit. Therefore, add a handler for it and don't treat it as a
non-trap-registers when preparing a shadow context.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index ea511e569ee0..dc8a33ebad5f 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2085,6 +2085,23 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+ { SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
+ { SYS_DESC(SYS_CPACR_EL12), access_rw, reset_val, CPACR_EL1, 0 },
+ { SYS_DESC(SYS_TTBR0_EL12), access_vm_reg, reset_unknown, TTBR0_EL1 },
+ { SYS_DESC(SYS_TTBR1_EL12), access_vm_reg, reset_unknown, TTBR1_EL1 },
+ { SYS_DESC(SYS_TCR_EL12), access_vm_reg, reset_val, TCR_EL1, 0 },
+ { SYS_DESC(SYS_SPSR_EL12), access_spsr},
+ { SYS_DESC(SYS_ELR_EL12), access_elr},
+ { SYS_DESC(SYS_AFSR0_EL12), access_vm_reg, reset_unknown, AFSR0_EL1 },
+ { SYS_DESC(SYS_AFSR1_EL12), access_vm_reg, reset_unknown, AFSR1_EL1 },
+ { SYS_DESC(SYS_ESR_EL12), access_vm_reg, reset_unknown, ESR_EL1 },
+ { SYS_DESC(SYS_FAR_EL12), access_vm_reg, reset_unknown, FAR_EL1 },
+ { SYS_DESC(SYS_MAIR_EL12), access_vm_reg, reset_unknown, MAIR_EL1 },
+ { SYS_DESC(SYS_AMAIR_EL12), access_vm_reg, reset_amair_el1, AMAIR_EL1 },
+ { SYS_DESC(SYS_VBAR_EL12), access_rw, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_CONTEXTIDR_EL12), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
+ { SYS_DESC(SYS_CNTKCTL_EL12), access_rw, reset_val, CNTKCTL_EL1, 0 },
+
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 27/66] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
With HCR_EL2.NV bit set, accesses to EL12 registers in the virtual EL2
trap to EL2. Handle those traps just like we do for EL1 registers.
One exception is CNTKCTL_EL12. We don't trap on CNTKCTL_EL1 for non-VHE
virtual EL2 because we don't have to. However, accessing CNTKCTL_EL12
will trap since it's one of the EL12 registers controlled by HCR_EL2.NV
bit. Therefore, add a handler for it and don't treat it as a
non-trap-registers when preparing a shadow context.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index ea511e569ee0..dc8a33ebad5f 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2085,6 +2085,23 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+ { SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
+ { SYS_DESC(SYS_CPACR_EL12), access_rw, reset_val, CPACR_EL1, 0 },
+ { SYS_DESC(SYS_TTBR0_EL12), access_vm_reg, reset_unknown, TTBR0_EL1 },
+ { SYS_DESC(SYS_TTBR1_EL12), access_vm_reg, reset_unknown, TTBR1_EL1 },
+ { SYS_DESC(SYS_TCR_EL12), access_vm_reg, reset_val, TCR_EL1, 0 },
+ { SYS_DESC(SYS_SPSR_EL12), access_spsr},
+ { SYS_DESC(SYS_ELR_EL12), access_elr},
+ { SYS_DESC(SYS_AFSR0_EL12), access_vm_reg, reset_unknown, AFSR0_EL1 },
+ { SYS_DESC(SYS_AFSR1_EL12), access_vm_reg, reset_unknown, AFSR1_EL1 },
+ { SYS_DESC(SYS_ESR_EL12), access_vm_reg, reset_unknown, ESR_EL1 },
+ { SYS_DESC(SYS_FAR_EL12), access_vm_reg, reset_unknown, FAR_EL1 },
+ { SYS_DESC(SYS_MAIR_EL12), access_vm_reg, reset_unknown, MAIR_EL1 },
+ { SYS_DESC(SYS_AMAIR_EL12), access_vm_reg, reset_amair_el1, AMAIR_EL1 },
+ { SYS_DESC(SYS_VBAR_EL12), access_rw, reset_val, VBAR_EL1, 0 },
+ { SYS_DESC(SYS_CONTEXTIDR_EL12), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
+ { SYS_DESC(SYS_CNTKCTL_EL12), access_rw, reset_val, CNTKCTL_EL1, 0 },
+
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 28/66] KVM: arm64: nv: Forward debug traps to the nested guest
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
On handling a debug trap, check whether we need to forward it to the
guest before handling it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/emulate-nested.c | 9 +++++++--
arch/arm64/kvm/sys_regs.c | 3 +++
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 26cba7b4d743..07c15f51cf86 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -62,6 +62,8 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
}
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
+ u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index feb9b5eded96..df4661515183 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -25,14 +25,14 @@
#include "trace.h"
-bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg, u64 control_bit)
{
bool control_bit_set;
if (!nested_virt_in_use(vcpu))
return false;
- control_bit_set = __vcpu_sys_reg(vcpu, HCR_EL2) & control_bit;
+ control_bit_set = __vcpu_sys_reg(vcpu, reg) & control_bit;
if (!vcpu_mode_el2(vcpu) && control_bit_set) {
kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
return true;
@@ -40,6 +40,11 @@ bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
return false;
}
+bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+{
+ return __forward_traps(vcpu, HCR_EL2, control_bit);
+}
+
bool forward_nv_traps(struct kvm_vcpu *vcpu)
{
return forward_traps(vcpu, HCR_NV);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index dc8a33ebad5f..80cf0c0761b9 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -607,6 +607,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (__forward_traps(vcpu, MDCR_EL2, MDCR_EL2_TDA | MDCR_EL2_TDE))
+ return false;
+
access_rw(vcpu, p, r);
if (p->is_write)
vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 28/66] KVM: arm64: nv: Forward debug traps to the nested guest
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
On handling a debug trap, check whether we need to forward it to the
guest before handling it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/emulate-nested.c | 9 +++++++--
arch/arm64/kvm/sys_regs.c | 3 +++
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 26cba7b4d743..07c15f51cf86 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -62,6 +62,8 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
}
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
+ u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index feb9b5eded96..df4661515183 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -25,14 +25,14 @@
#include "trace.h"
-bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg, u64 control_bit)
{
bool control_bit_set;
if (!nested_virt_in_use(vcpu))
return false;
- control_bit_set = __vcpu_sys_reg(vcpu, HCR_EL2) & control_bit;
+ control_bit_set = __vcpu_sys_reg(vcpu, reg) & control_bit;
if (!vcpu_mode_el2(vcpu) && control_bit_set) {
kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
return true;
@@ -40,6 +40,11 @@ bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
return false;
}
+bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+{
+ return __forward_traps(vcpu, HCR_EL2, control_bit);
+}
+
bool forward_nv_traps(struct kvm_vcpu *vcpu)
{
return forward_traps(vcpu, HCR_NV);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index dc8a33ebad5f..80cf0c0761b9 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -607,6 +607,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (__forward_traps(vcpu, MDCR_EL2, MDCR_EL2_TDA | MDCR_EL2_TDE))
+ return false;
+
access_rw(vcpu, p, r);
if (p->is_write)
vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 28/66] KVM: arm64: nv: Forward debug traps to the nested guest
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
On handling a debug trap, check whether we need to forward it to the
guest before handling it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 2 ++
arch/arm64/kvm/emulate-nested.c | 9 +++++++--
arch/arm64/kvm/sys_regs.c | 3 +++
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 26cba7b4d743..07c15f51cf86 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -62,6 +62,8 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
}
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
+extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
+ u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index feb9b5eded96..df4661515183 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -25,14 +25,14 @@
#include "trace.h"
-bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg, u64 control_bit)
{
bool control_bit_set;
if (!nested_virt_in_use(vcpu))
return false;
- control_bit_set = __vcpu_sys_reg(vcpu, HCR_EL2) & control_bit;
+ control_bit_set = __vcpu_sys_reg(vcpu, reg) & control_bit;
if (!vcpu_mode_el2(vcpu) && control_bit_set) {
kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
return true;
@@ -40,6 +40,11 @@ bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
return false;
}
+bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit)
+{
+ return __forward_traps(vcpu, HCR_EL2, control_bit);
+}
+
bool forward_nv_traps(struct kvm_vcpu *vcpu)
{
return forward_traps(vcpu, HCR_NV);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index dc8a33ebad5f..80cf0c0761b9 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -607,6 +607,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ if (__forward_traps(vcpu, MDCR_EL2, MDCR_EL2_TDA | MDCR_EL2_TDE))
+ return false;
+
access_rw(vcpu, p, r);
if (p->is_write)
vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 29/66] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
We enable nested virtualization by setting the HCR NV and NV1 bit.
When the virtual E2H bit is set, we can support EL2 register accesses
via EL1 registers from the virtual EL2 by doing trap-and-emulate. A
better alternative, however, is to allow the virtual EL2 to access EL2
register states without trap. This can be easily achieved by not traping
EL1 registers since those registers already have EL2 register states.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 36 ++++++++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 66358f8ec5ad..940d88ffd53e 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -39,9 +39,39 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
u64 hcr = vcpu->arch.hcr_el2;
u64 val;
- /* Trap VM sysreg accesses if an EL2 guest is not using VHE. */
- if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
- hcr |= HCR_TVM | HCR_TRVM;
+ if (is_hyp_ctxt(vcpu)) {
+ hcr |= HCR_NV;
+
+ if (!vcpu_el2_e2h_is_set(vcpu)) {
+ /*
+ * For a guest hypervisor on v8.0, trap and emulate
+ * the EL1 virtual memory control register accesses.
+ */
+ hcr |= HCR_TVM | HCR_TRVM | HCR_NV1;
+ } else {
+ /*
+ * For a guest hypervisor on v8.1 (VHE), allow to
+ * access the EL1 virtual memory control registers
+ * natively. These accesses are to access EL2 register
+ * states.
+ * Note that we still need to respect the virtual
+ * HCR_EL2 state.
+ */
+ u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+
+ /*
+ * We already set TVM to handle set/way cache maint
+ * ops traps, this somewhat collides with the nested
+ * virt trapping for nVHE. So turn this off for now
+ * here, in the hope that VHE guests won't ever do this.
+ * TODO: find out whether it's worth to support both
+ * cases at the same time.
+ */
+ hcr &= ~HCR_TVM;
+
+ hcr |= vhcr_el2 & (HCR_TVM | HCR_TRVM);
+ }
+ }
___activate_traps(vcpu, hcr);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 29/66] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
We enable nested virtualization by setting the HCR NV and NV1 bit.
When the virtual E2H bit is set, we can support EL2 register accesses
via EL1 registers from the virtual EL2 by doing trap-and-emulate. A
better alternative, however, is to allow the virtual EL2 to access EL2
register states without trap. This can be easily achieved by not traping
EL1 registers since those registers already have EL2 register states.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 36 ++++++++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 66358f8ec5ad..940d88ffd53e 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -39,9 +39,39 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
u64 hcr = vcpu->arch.hcr_el2;
u64 val;
- /* Trap VM sysreg accesses if an EL2 guest is not using VHE. */
- if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
- hcr |= HCR_TVM | HCR_TRVM;
+ if (is_hyp_ctxt(vcpu)) {
+ hcr |= HCR_NV;
+
+ if (!vcpu_el2_e2h_is_set(vcpu)) {
+ /*
+ * For a guest hypervisor on v8.0, trap and emulate
+ * the EL1 virtual memory control register accesses.
+ */
+ hcr |= HCR_TVM | HCR_TRVM | HCR_NV1;
+ } else {
+ /*
+ * For a guest hypervisor on v8.1 (VHE), allow to
+ * access the EL1 virtual memory control registers
+ * natively. These accesses are to access EL2 register
+ * states.
+ * Note that we still need to respect the virtual
+ * HCR_EL2 state.
+ */
+ u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+
+ /*
+ * We already set TVM to handle set/way cache maint
+ * ops traps, this somewhat collides with the nested
+ * virt trapping for nVHE. So turn this off for now
+ * here, in the hope that VHE guests won't ever do this.
+ * TODO: find out whether it's worth to support both
+ * cases at the same time.
+ */
+ hcr &= ~HCR_TVM;
+
+ hcr |= vhcr_el2 & (HCR_TVM | HCR_TRVM);
+ }
+ }
___activate_traps(vcpu, hcr);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 29/66] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
We enable nested virtualization by setting the HCR NV and NV1 bit.
When the virtual E2H bit is set, we can support EL2 register accesses
via EL1 registers from the virtual EL2 by doing trap-and-emulate. A
better alternative, however, is to allow the virtual EL2 to access EL2
register states without trap. This can be easily achieved by not traping
EL1 registers since those registers already have EL2 register states.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 36 ++++++++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 66358f8ec5ad..940d88ffd53e 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -39,9 +39,39 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
u64 hcr = vcpu->arch.hcr_el2;
u64 val;
- /* Trap VM sysreg accesses if an EL2 guest is not using VHE. */
- if (vcpu_mode_el2(vcpu) && !vcpu_el2_e2h_is_set(vcpu))
- hcr |= HCR_TVM | HCR_TRVM;
+ if (is_hyp_ctxt(vcpu)) {
+ hcr |= HCR_NV;
+
+ if (!vcpu_el2_e2h_is_set(vcpu)) {
+ /*
+ * For a guest hypervisor on v8.0, trap and emulate
+ * the EL1 virtual memory control register accesses.
+ */
+ hcr |= HCR_TVM | HCR_TRVM | HCR_NV1;
+ } else {
+ /*
+ * For a guest hypervisor on v8.1 (VHE), allow to
+ * access the EL1 virtual memory control registers
+ * natively. These accesses are to access EL2 register
+ * states.
+ * Note that we still need to respect the virtual
+ * HCR_EL2 state.
+ */
+ u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
+
+ /*
+ * We already set TVM to handle set/way cache maint
+ * ops traps, this somewhat collides with the nested
+ * virt trapping for nVHE. So turn this off for now
+ * here, in the hope that VHE guests won't ever do this.
+ * TODO: find out whether it's worth to support both
+ * cases at the same time.
+ */
+ hcr &= ~HCR_TVM;
+
+ hcr |= vhcr_el2 & (HCR_TVM | HCR_TRVM);
+ }
+ }
___activate_traps(vcpu, hcr);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 30/66] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Christoffer Dall, Jintack Lim
From: Christoffer Dall <christoffer.dall@linaro.org>
So far we were flushing almost the entire universe whenever a VM would
load/unload the SCTLR_EL1 and the two versions of that register had
different MMU enabled settings. This turned out to be so slow that it
prevented forward progress for a nested VM, because a scheduler timer
tick interrupt would always be pending when we reached the nested VM.
To avoid this problem, we consider the SCTLR_EL2 when evaluating if
caches are on or off when entering virtual EL2 (because this is the
value that we end up shadowing onto the hardware EL1 register).
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_mmu.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index e52d82aeadca..76a8a0ca45b8 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -125,6 +125,7 @@ alternative_cb_end
#include <asm/cache.h>
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
+#include <asm/kvm_emulate.h>
void kvm_update_va_mask(struct alt_instr *alt,
__le32 *origptr, __le32 *updptr, int nr_inst);
@@ -201,7 +202,10 @@ struct kvm;
static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
{
- return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
+ if (vcpu_mode_el2(vcpu))
+ return (__vcpu_sys_reg(vcpu, SCTLR_EL2) & 0b101) == 0b101;
+ else
+ return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
}
static inline void __clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size)
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 30/66] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Andre Przywara, Christoffer Dall, Jintack Lim
From: Christoffer Dall <christoffer.dall@linaro.org>
So far we were flushing almost the entire universe whenever a VM would
load/unload the SCTLR_EL1 and the two versions of that register had
different MMU enabled settings. This turned out to be so slow that it
prevented forward progress for a nested VM, because a scheduler timer
tick interrupt would always be pending when we reached the nested VM.
To avoid this problem, we consider the SCTLR_EL2 when evaluating if
caches are on or off when entering virtual EL2 (because this is the
value that we end up shadowing onto the hardware EL1 register).
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_mmu.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index e52d82aeadca..76a8a0ca45b8 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -125,6 +125,7 @@ alternative_cb_end
#include <asm/cache.h>
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
+#include <asm/kvm_emulate.h>
void kvm_update_va_mask(struct alt_instr *alt,
__le32 *origptr, __le32 *updptr, int nr_inst);
@@ -201,7 +202,10 @@ struct kvm;
static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
{
- return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
+ if (vcpu_mode_el2(vcpu))
+ return (__vcpu_sys_reg(vcpu, SCTLR_EL2) & 0b101) == 0b101;
+ else
+ return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
}
static inline void __clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size)
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 30/66] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Christoffer Dall, Jintack Lim, Alexandru Elisei,
Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@linaro.org>
So far we were flushing almost the entire universe whenever a VM would
load/unload the SCTLR_EL1 and the two versions of that register had
different MMU enabled settings. This turned out to be so slow that it
prevented forward progress for a nested VM, because a scheduler timer
tick interrupt would always be pending when we reached the nested VM.
To avoid this problem, we consider the SCTLR_EL2 when evaluating if
caches are on or off when entering virtual EL2 (because this is the
value that we end up shadowing onto the hardware EL1 register).
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_mmu.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index e52d82aeadca..76a8a0ca45b8 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -125,6 +125,7 @@ alternative_cb_end
#include <asm/cache.h>
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
+#include <asm/kvm_emulate.h>
void kvm_update_va_mask(struct alt_instr *alt,
__le32 *origptr, __le32 *updptr, int nr_inst);
@@ -201,7 +202,10 @@ struct kvm;
static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
{
- return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
+ if (vcpu_mode_el2(vcpu))
+ return (__vcpu_sys_reg(vcpu, SCTLR_EL2) & 0b101) == 0b101;
+ else
+ return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
}
static inline void __clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 31/66] KVM: arm64: nv: Filter out unsupported features from ID regs
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
As there is a number of features that we either can't support,
or don't want to support right away with NV, let's add some
basic filtering so that we don't advertize silly things to the
EL2 guest.
Whilst we are at it, avertize ARMv8.4-TTL as well as ARMv8.5-GTG.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 6 ++
arch/arm64/kvm/nested.c | 152 ++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 4 +-
3 files changed, 161 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 07c15f51cf86..026ddaad972c 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -67,4 +67,10 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+struct sys_reg_params;
+struct sys_reg_desc;
+
+void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
+ const struct sys_reg_desc *r);
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 42a96c8d2adc..9fb44bc7db3f 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -20,6 +20,10 @@
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
+#include <asm/sysreg.h>
+
+#include "sys_regs.h"
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
@@ -38,3 +42,151 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
return -EINVAL;
}
+
+#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
+
+/*
+ * Our emulated CPU doesn't support all the possible features. For the
+ * sake of simplicity (and probably mental sanity), wipe out a number
+ * of feature bits we don't intend to support for the time being.
+ * This list should get updated as new features get added to the NV
+ * support, and new extension to the architecture.
+ */
+void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
+ (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+ u64 val, tmp;
+
+ if (!nested_virt_in_use(v))
+ return;
+
+ val = p->regval;
+
+ switch (id) {
+ case SYS_ID_AA64ISAR0_EL1:
+ /* Support everything but O.S. and Range TLBIs */
+ val &= ~(FEATURE(ID_AA64ISAR0_TLB) |
+ GENMASK_ULL(27, 24) |
+ GENMASK_ULL(3, 0));
+ break;
+
+ case SYS_ID_AA64ISAR1_EL1:
+ /* Support everything but PtrAuth and Spec Invalidation */
+ val &= ~(GENMASK_ULL(63, 56) |
+ FEATURE(ID_AA64ISAR1_SPECRES) |
+ FEATURE(ID_AA64ISAR1_GPI) |
+ FEATURE(ID_AA64ISAR1_GPA) |
+ FEATURE(ID_AA64ISAR1_API) |
+ FEATURE(ID_AA64ISAR1_APA));
+ break;
+
+ case SYS_ID_AA64PFR0_EL1:
+ /* No AMU, MPAM, S-EL2, RAS or SVE */
+ val &= ~(GENMASK_ULL(55, 52) |
+ FEATURE(ID_AA64PFR0_AMU) |
+ FEATURE(ID_AA64PFR0_MPAM) |
+ FEATURE(ID_AA64PFR0_SEL2) |
+ FEATURE(ID_AA64PFR0_RAS) |
+ FEATURE(ID_AA64PFR0_SVE) |
+ FEATURE(ID_AA64PFR0_EL2));
+ /* 64bit EL2 only */
+ val |= FIELD_PREP(FEATURE(ID_AA64PFR0_EL2), 0b0001);
+ break;
+
+ case SYS_ID_AA64PFR1_EL1:
+ /* Only support SSBS */
+ val &= FEATURE(ID_AA64PFR1_SSBS);
+ break;
+
+ case SYS_ID_AA64MMFR0_EL1:
+ /* Hide ECV, FGT, ExS, Secure Memory */
+ val &= ~(GENMASK_ULL(63, 43) |
+ FEATURE(ID_AA64MMFR0_TGRAN4_2) |
+ FEATURE(ID_AA64MMFR0_TGRAN16_2) |
+ FEATURE(ID_AA64MMFR0_TGRAN64_2) |
+ FEATURE(ID_AA64MMFR0_SNSMEM));
+
+ /* Disallow unsupported S2 page sizes */
+ switch (PAGE_SIZE) {
+ case SZ_64K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0001);
+ /* Fall through */
+ case SZ_16K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0001);
+ /* Fall through */
+ case SZ_4K:
+ /* Support everything */
+ break;
+ }
+ /* Advertize supported S2 page sizes */
+ switch (PAGE_SIZE) {
+ case SZ_4K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0010);
+ /* Fall through */
+ case SZ_16K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0010);
+ /* Fall through */
+ case SZ_64K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN64_2), 0b0010);
+ break;
+ }
+ /* Cap PARange to 40bits */
+ tmp = FIELD_GET(FEATURE(ID_AA64MMFR0_PARANGE), val);
+ if (tmp > 0b0010) {
+ val &= ~FEATURE(ID_AA64MMFR0_PARANGE);
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_PARANGE), 0b0010);
+ }
+ break;
+
+ case SYS_ID_AA64MMFR1_EL1:
+ val &= (FEATURE(ID_AA64MMFR1_PAN) |
+ FEATURE(ID_AA64MMFR1_LOR) |
+ FEATURE(ID_AA64MMFR1_HPD) |
+ FEATURE(ID_AA64MMFR1_VHE) |
+ FEATURE(ID_AA64MMFR1_VMIDBITS));
+ break;
+
+ case SYS_ID_AA64MMFR2_EL1:
+ val &= ~(FEATURE(ID_AA64MMFR2_EVT) |
+ FEATURE(ID_AA64MMFR2_BBM) |
+ FEATURE(ID_AA64MMFR2_TTL) |
+ GENMASK_ULL(47, 44) |
+ FEATURE(ID_AA64MMFR2_ST) |
+ FEATURE(ID_AA64MMFR2_CCIDX) |
+ FEATURE(ID_AA64MMFR2_LVA));
+
+ /* Force TTL support */
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR2_TTL), 0b0001);
+ break;
+
+ case SYS_ID_AA64DFR0_EL1:
+ /* Only limited support for PMU, Debug, BPs and WPs */
+ val &= (FEATURE(ID_AA64DFR0_PMSVER) |
+ FEATURE(ID_AA64DFR0_WRPS) |
+ FEATURE(ID_AA64DFR0_BRPS) |
+ FEATURE(ID_AA64DFR0_DEBUGVER));
+
+ /* Cap PMU to ARMv8.1 */
+ tmp = FIELD_GET(FEATURE(ID_AA64DFR0_PMUVER), val);
+ if (tmp > 0b0100) {
+ val &= ~FEATURE(ID_AA64DFR0_PMUVER);
+ val |= FIELD_PREP(FEATURE(ID_AA64DFR0_PMUVER), 0b0100);
+ }
+ /* Cap Debug to ARMv8.1 */
+ tmp = FIELD_GET(FEATURE(ID_AA64DFR0_DEBUGVER), val);
+ if (tmp > 0b0111) {
+ val &= ~FEATURE(ID_AA64DFR0_DEBUGVER);
+ val |= FIELD_PREP(FEATURE(ID_AA64DFR0_DEBUGVER), 0b0111);
+ }
+ break;
+
+ default:
+ /* Unknown register, just wipe it clean */
+ val = 0;
+ break;
+ }
+
+ p->regval = val;
+}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 80cf0c0761b9..d011b8156c52 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1372,8 +1372,10 @@ static bool access_id_reg(struct kvm_vcpu *vcpu,
const struct sys_reg_desc *r)
{
bool raz = sysreg_visible_as_raz(vcpu, r);
+ bool ret = __access_id_reg(vcpu, p, r, raz);
- return __access_id_reg(vcpu, p, r, raz);
+ access_nested_id_reg(vcpu, p, r);
+ return ret;
}
static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 31/66] KVM: arm64: nv: Filter out unsupported features from ID regs
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
As there is a number of features that we either can't support,
or don't want to support right away with NV, let's add some
basic filtering so that we don't advertize silly things to the
EL2 guest.
Whilst we are at it, avertize ARMv8.4-TTL as well as ARMv8.5-GTG.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 6 ++
arch/arm64/kvm/nested.c | 152 ++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 4 +-
3 files changed, 161 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 07c15f51cf86..026ddaad972c 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -67,4 +67,10 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+struct sys_reg_params;
+struct sys_reg_desc;
+
+void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
+ const struct sys_reg_desc *r);
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 42a96c8d2adc..9fb44bc7db3f 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -20,6 +20,10 @@
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
+#include <asm/sysreg.h>
+
+#include "sys_regs.h"
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
@@ -38,3 +42,151 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
return -EINVAL;
}
+
+#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
+
+/*
+ * Our emulated CPU doesn't support all the possible features. For the
+ * sake of simplicity (and probably mental sanity), wipe out a number
+ * of feature bits we don't intend to support for the time being.
+ * This list should get updated as new features get added to the NV
+ * support, and new extension to the architecture.
+ */
+void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
+ (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+ u64 val, tmp;
+
+ if (!nested_virt_in_use(v))
+ return;
+
+ val = p->regval;
+
+ switch (id) {
+ case SYS_ID_AA64ISAR0_EL1:
+ /* Support everything but O.S. and Range TLBIs */
+ val &= ~(FEATURE(ID_AA64ISAR0_TLB) |
+ GENMASK_ULL(27, 24) |
+ GENMASK_ULL(3, 0));
+ break;
+
+ case SYS_ID_AA64ISAR1_EL1:
+ /* Support everything but PtrAuth and Spec Invalidation */
+ val &= ~(GENMASK_ULL(63, 56) |
+ FEATURE(ID_AA64ISAR1_SPECRES) |
+ FEATURE(ID_AA64ISAR1_GPI) |
+ FEATURE(ID_AA64ISAR1_GPA) |
+ FEATURE(ID_AA64ISAR1_API) |
+ FEATURE(ID_AA64ISAR1_APA));
+ break;
+
+ case SYS_ID_AA64PFR0_EL1:
+ /* No AMU, MPAM, S-EL2, RAS or SVE */
+ val &= ~(GENMASK_ULL(55, 52) |
+ FEATURE(ID_AA64PFR0_AMU) |
+ FEATURE(ID_AA64PFR0_MPAM) |
+ FEATURE(ID_AA64PFR0_SEL2) |
+ FEATURE(ID_AA64PFR0_RAS) |
+ FEATURE(ID_AA64PFR0_SVE) |
+ FEATURE(ID_AA64PFR0_EL2));
+ /* 64bit EL2 only */
+ val |= FIELD_PREP(FEATURE(ID_AA64PFR0_EL2), 0b0001);
+ break;
+
+ case SYS_ID_AA64PFR1_EL1:
+ /* Only support SSBS */
+ val &= FEATURE(ID_AA64PFR1_SSBS);
+ break;
+
+ case SYS_ID_AA64MMFR0_EL1:
+ /* Hide ECV, FGT, ExS, Secure Memory */
+ val &= ~(GENMASK_ULL(63, 43) |
+ FEATURE(ID_AA64MMFR0_TGRAN4_2) |
+ FEATURE(ID_AA64MMFR0_TGRAN16_2) |
+ FEATURE(ID_AA64MMFR0_TGRAN64_2) |
+ FEATURE(ID_AA64MMFR0_SNSMEM));
+
+ /* Disallow unsupported S2 page sizes */
+ switch (PAGE_SIZE) {
+ case SZ_64K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0001);
+ /* Fall through */
+ case SZ_16K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0001);
+ /* Fall through */
+ case SZ_4K:
+ /* Support everything */
+ break;
+ }
+ /* Advertize supported S2 page sizes */
+ switch (PAGE_SIZE) {
+ case SZ_4K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0010);
+ /* Fall through */
+ case SZ_16K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0010);
+ /* Fall through */
+ case SZ_64K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN64_2), 0b0010);
+ break;
+ }
+ /* Cap PARange to 40bits */
+ tmp = FIELD_GET(FEATURE(ID_AA64MMFR0_PARANGE), val);
+ if (tmp > 0b0010) {
+ val &= ~FEATURE(ID_AA64MMFR0_PARANGE);
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_PARANGE), 0b0010);
+ }
+ break;
+
+ case SYS_ID_AA64MMFR1_EL1:
+ val &= (FEATURE(ID_AA64MMFR1_PAN) |
+ FEATURE(ID_AA64MMFR1_LOR) |
+ FEATURE(ID_AA64MMFR1_HPD) |
+ FEATURE(ID_AA64MMFR1_VHE) |
+ FEATURE(ID_AA64MMFR1_VMIDBITS));
+ break;
+
+ case SYS_ID_AA64MMFR2_EL1:
+ val &= ~(FEATURE(ID_AA64MMFR2_EVT) |
+ FEATURE(ID_AA64MMFR2_BBM) |
+ FEATURE(ID_AA64MMFR2_TTL) |
+ GENMASK_ULL(47, 44) |
+ FEATURE(ID_AA64MMFR2_ST) |
+ FEATURE(ID_AA64MMFR2_CCIDX) |
+ FEATURE(ID_AA64MMFR2_LVA));
+
+ /* Force TTL support */
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR2_TTL), 0b0001);
+ break;
+
+ case SYS_ID_AA64DFR0_EL1:
+ /* Only limited support for PMU, Debug, BPs and WPs */
+ val &= (FEATURE(ID_AA64DFR0_PMSVER) |
+ FEATURE(ID_AA64DFR0_WRPS) |
+ FEATURE(ID_AA64DFR0_BRPS) |
+ FEATURE(ID_AA64DFR0_DEBUGVER));
+
+ /* Cap PMU to ARMv8.1 */
+ tmp = FIELD_GET(FEATURE(ID_AA64DFR0_PMUVER), val);
+ if (tmp > 0b0100) {
+ val &= ~FEATURE(ID_AA64DFR0_PMUVER);
+ val |= FIELD_PREP(FEATURE(ID_AA64DFR0_PMUVER), 0b0100);
+ }
+ /* Cap Debug to ARMv8.1 */
+ tmp = FIELD_GET(FEATURE(ID_AA64DFR0_DEBUGVER), val);
+ if (tmp > 0b0111) {
+ val &= ~FEATURE(ID_AA64DFR0_DEBUGVER);
+ val |= FIELD_PREP(FEATURE(ID_AA64DFR0_DEBUGVER), 0b0111);
+ }
+ break;
+
+ default:
+ /* Unknown register, just wipe it clean */
+ val = 0;
+ break;
+ }
+
+ p->regval = val;
+}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 80cf0c0761b9..d011b8156c52 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1372,8 +1372,10 @@ static bool access_id_reg(struct kvm_vcpu *vcpu,
const struct sys_reg_desc *r)
{
bool raz = sysreg_visible_as_raz(vcpu, r);
+ bool ret = __access_id_reg(vcpu, p, r, raz);
- return __access_id_reg(vcpu, p, r, raz);
+ access_nested_id_reg(vcpu, p, r);
+ return ret;
}
static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 31/66] KVM: arm64: nv: Filter out unsupported features from ID regs
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
As there is a number of features that we either can't support,
or don't want to support right away with NV, let's add some
basic filtering so that we don't advertize silly things to the
EL2 guest.
Whilst we are at it, avertize ARMv8.4-TTL as well as ARMv8.5-GTG.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 6 ++
arch/arm64/kvm/nested.c | 152 ++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 4 +-
3 files changed, 161 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 07c15f51cf86..026ddaad972c 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -67,4 +67,10 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+struct sys_reg_params;
+struct sys_reg_desc;
+
+void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
+ const struct sys_reg_desc *r);
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 42a96c8d2adc..9fb44bc7db3f 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -20,6 +20,10 @@
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
+#include <asm/sysreg.h>
+
+#include "sys_regs.h"
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
@@ -38,3 +42,151 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
return -EINVAL;
}
+
+#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
+
+/*
+ * Our emulated CPU doesn't support all the possible features. For the
+ * sake of simplicity (and probably mental sanity), wipe out a number
+ * of feature bits we don't intend to support for the time being.
+ * This list should get updated as new features get added to the NV
+ * support, and new extension to the architecture.
+ */
+void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
+ (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+ u64 val, tmp;
+
+ if (!nested_virt_in_use(v))
+ return;
+
+ val = p->regval;
+
+ switch (id) {
+ case SYS_ID_AA64ISAR0_EL1:
+ /* Support everything but O.S. and Range TLBIs */
+ val &= ~(FEATURE(ID_AA64ISAR0_TLB) |
+ GENMASK_ULL(27, 24) |
+ GENMASK_ULL(3, 0));
+ break;
+
+ case SYS_ID_AA64ISAR1_EL1:
+ /* Support everything but PtrAuth and Spec Invalidation */
+ val &= ~(GENMASK_ULL(63, 56) |
+ FEATURE(ID_AA64ISAR1_SPECRES) |
+ FEATURE(ID_AA64ISAR1_GPI) |
+ FEATURE(ID_AA64ISAR1_GPA) |
+ FEATURE(ID_AA64ISAR1_API) |
+ FEATURE(ID_AA64ISAR1_APA));
+ break;
+
+ case SYS_ID_AA64PFR0_EL1:
+ /* No AMU, MPAM, S-EL2, RAS or SVE */
+ val &= ~(GENMASK_ULL(55, 52) |
+ FEATURE(ID_AA64PFR0_AMU) |
+ FEATURE(ID_AA64PFR0_MPAM) |
+ FEATURE(ID_AA64PFR0_SEL2) |
+ FEATURE(ID_AA64PFR0_RAS) |
+ FEATURE(ID_AA64PFR0_SVE) |
+ FEATURE(ID_AA64PFR0_EL2));
+ /* 64bit EL2 only */
+ val |= FIELD_PREP(FEATURE(ID_AA64PFR0_EL2), 0b0001);
+ break;
+
+ case SYS_ID_AA64PFR1_EL1:
+ /* Only support SSBS */
+ val &= FEATURE(ID_AA64PFR1_SSBS);
+ break;
+
+ case SYS_ID_AA64MMFR0_EL1:
+ /* Hide ECV, FGT, ExS, Secure Memory */
+ val &= ~(GENMASK_ULL(63, 43) |
+ FEATURE(ID_AA64MMFR0_TGRAN4_2) |
+ FEATURE(ID_AA64MMFR0_TGRAN16_2) |
+ FEATURE(ID_AA64MMFR0_TGRAN64_2) |
+ FEATURE(ID_AA64MMFR0_SNSMEM));
+
+ /* Disallow unsupported S2 page sizes */
+ switch (PAGE_SIZE) {
+ case SZ_64K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0001);
+ /* Fall through */
+ case SZ_16K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0001);
+ /* Fall through */
+ case SZ_4K:
+ /* Support everything */
+ break;
+ }
+ /* Advertize supported S2 page sizes */
+ switch (PAGE_SIZE) {
+ case SZ_4K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0010);
+ /* Fall through */
+ case SZ_16K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0010);
+ /* Fall through */
+ case SZ_64K:
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN64_2), 0b0010);
+ break;
+ }
+ /* Cap PARange to 40bits */
+ tmp = FIELD_GET(FEATURE(ID_AA64MMFR0_PARANGE), val);
+ if (tmp > 0b0010) {
+ val &= ~FEATURE(ID_AA64MMFR0_PARANGE);
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_PARANGE), 0b0010);
+ }
+ break;
+
+ case SYS_ID_AA64MMFR1_EL1:
+ val &= (FEATURE(ID_AA64MMFR1_PAN) |
+ FEATURE(ID_AA64MMFR1_LOR) |
+ FEATURE(ID_AA64MMFR1_HPD) |
+ FEATURE(ID_AA64MMFR1_VHE) |
+ FEATURE(ID_AA64MMFR1_VMIDBITS));
+ break;
+
+ case SYS_ID_AA64MMFR2_EL1:
+ val &= ~(FEATURE(ID_AA64MMFR2_EVT) |
+ FEATURE(ID_AA64MMFR2_BBM) |
+ FEATURE(ID_AA64MMFR2_TTL) |
+ GENMASK_ULL(47, 44) |
+ FEATURE(ID_AA64MMFR2_ST) |
+ FEATURE(ID_AA64MMFR2_CCIDX) |
+ FEATURE(ID_AA64MMFR2_LVA));
+
+ /* Force TTL support */
+ val |= FIELD_PREP(FEATURE(ID_AA64MMFR2_TTL), 0b0001);
+ break;
+
+ case SYS_ID_AA64DFR0_EL1:
+ /* Only limited support for PMU, Debug, BPs and WPs */
+ val &= (FEATURE(ID_AA64DFR0_PMSVER) |
+ FEATURE(ID_AA64DFR0_WRPS) |
+ FEATURE(ID_AA64DFR0_BRPS) |
+ FEATURE(ID_AA64DFR0_DEBUGVER));
+
+ /* Cap PMU to ARMv8.1 */
+ tmp = FIELD_GET(FEATURE(ID_AA64DFR0_PMUVER), val);
+ if (tmp > 0b0100) {
+ val &= ~FEATURE(ID_AA64DFR0_PMUVER);
+ val |= FIELD_PREP(FEATURE(ID_AA64DFR0_PMUVER), 0b0100);
+ }
+ /* Cap Debug to ARMv8.1 */
+ tmp = FIELD_GET(FEATURE(ID_AA64DFR0_DEBUGVER), val);
+ if (tmp > 0b0111) {
+ val &= ~FEATURE(ID_AA64DFR0_DEBUGVER);
+ val |= FIELD_PREP(FEATURE(ID_AA64DFR0_DEBUGVER), 0b0111);
+ }
+ break;
+
+ default:
+ /* Unknown register, just wipe it clean */
+ val = 0;
+ break;
+ }
+
+ p->regval = val;
+}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 80cf0c0761b9..d011b8156c52 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1372,8 +1372,10 @@ static bool access_id_reg(struct kvm_vcpu *vcpu,
const struct sys_reg_desc *r)
{
bool raz = sysreg_visible_as_raz(vcpu, r);
+ bool ret = __access_id_reg(vcpu, p, r, raz);
- return __access_id_reg(vcpu, p, r, raz);
+ access_nested_id_reg(vcpu, p, r);
+ return ret;
}
static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 32/66] KVM: arm64: nv: Hide RAS from nested guests
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
We don't want to expose complicated features to guests until we have
a good grasp on the basic CPU emulation. So let's pretend that RAS,
doesn't exist in a nested guest. We already hide the feature bits,
let's now make sure VDISR_EL1 will UNDEF.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d011b8156c52..090ce14306ba 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -527,6 +527,14 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
return read_zero(vcpu, p);
}
+static bool trap_undef(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ kvm_inject_undefined(vcpu);
+ return false;
+}
+
/*
* ARMv8.1 mandates at least a trivial LORegion implementation, where all the
* RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
@@ -2083,6 +2091,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
{ SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
{ SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
+ { SYS_DESC(SYS_VDISR_EL2), trap_undef },
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 32/66] KVM: arm64: nv: Hide RAS from nested guests
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
We don't want to expose complicated features to guests until we have
a good grasp on the basic CPU emulation. So let's pretend that RAS,
doesn't exist in a nested guest. We already hide the feature bits,
let's now make sure VDISR_EL1 will UNDEF.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d011b8156c52..090ce14306ba 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -527,6 +527,14 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
return read_zero(vcpu, p);
}
+static bool trap_undef(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ kvm_inject_undefined(vcpu);
+ return false;
+}
+
/*
* ARMv8.1 mandates at least a trivial LORegion implementation, where all the
* RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
@@ -2083,6 +2091,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
{ SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
{ SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
+ { SYS_DESC(SYS_VDISR_EL2), trap_undef },
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 32/66] KVM: arm64: nv: Hide RAS from nested guests
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
We don't want to expose complicated features to guests until we have
a good grasp on the basic CPU emulation. So let's pretend that RAS,
doesn't exist in a nested guest. We already hide the feature bits,
let's now make sure VDISR_EL1 will UNDEF.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index d011b8156c52..090ce14306ba 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -527,6 +527,14 @@ static bool trap_raz_wi(struct kvm_vcpu *vcpu,
return read_zero(vcpu, p);
}
+static bool trap_undef(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ kvm_inject_undefined(vcpu);
+ return false;
+}
+
/*
* ARMv8.1 mandates at least a trivial LORegion implementation, where all the
* RW registers are RES0 (which we can implement as RAZ/WI). On an ARMv8.0
@@ -2083,6 +2091,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
{ SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
{ SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
+ { SYS_DESC(SYS_VDISR_EL2), trap_undef },
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Add Stage-2 mmu data structures for virtual EL2 and for nested guests.
We don't yet populate shadow Stage-2 page tables, but we now have a
framework for getting to a shadow Stage-2 pgd.
We allocate twice the number of vcpus as Stage-2 mmu structures because
that's sufficient for each vcpu running two translation regimes without
having to flush the Stage-2 page tables.
Co-developed-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 29 +++++
arch/arm64/include/asm/kvm_mmu.h | 8 ++
arch/arm64/include/asm/kvm_nested.h | 7 ++
arch/arm64/kvm/arm.c | 16 ++-
arch/arm64/kvm/mmu.c | 18 ++-
arch/arm64/kvm/nested.c | 183 ++++++++++++++++++++++++++++
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d731cf7a56cb..d99e51e7cbee 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -95,14 +95,43 @@ struct kvm_s2_mmu {
int __percpu *last_vcpu_ran;
struct kvm *kvm;
+
+ /*
+ * For a shadow stage-2 MMU, the virtual vttbr programmed by the guest
+ * hypervisor. Unused for kvm_arch->mmu. Set to 1 when the structure
+ * contains no valid information.
+ */
+ u64 vttbr;
+
+ /* true when this represents a nested context where virtual HCR_EL2.VM == 1 */
+ bool nested_stage2_enabled;
+
+ /*
+ * 0: Nobody is currently using this, check vttbr for validity
+ * >0: Somebody is actively using this.
+ */
+ atomic_t refcnt;
};
+static inline bool kvm_s2_mmu_valid(struct kvm_s2_mmu *mmu)
+{
+ return !(mmu->vttbr & 1);
+}
+
struct kvm_arch_memory_slot {
};
struct kvm_arch {
struct kvm_s2_mmu mmu;
+ /*
+ * Stage 2 paging stage for VMs with nested virtual using a virtual
+ * VMID.
+ */
+ struct kvm_s2_mmu *nested_mmus;
+ size_t nested_mmus_size;
+ int nested_mmus_next;
+
/* VTCR_EL2 value for this VM */
u64 vtcr;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 76a8a0ca45b8..ec39015bb2a6 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -126,6 +126,7 @@ alternative_cb_end
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
void kvm_update_va_mask(struct alt_instr *alt,
__le32 *origptr, __le32 *updptr, int nr_inst);
@@ -184,6 +185,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
void **haddr);
void free_hyp_pgds(void);
+void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
void stage2_unmap_vm(struct kvm *kvm);
int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
@@ -306,5 +308,11 @@ static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu)
asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
}
+static inline u64 get_vmid(u64 vttbr)
+{
+ return (vttbr & VTTBR_VMID_MASK(kvm_get_vmid_bits())) >>
+ VTTBR_VMID_SHIFT;
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 026ddaad972c..473ecd1d60d0 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -61,6 +61,13 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
(cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
}
+extern void kvm_init_nested(struct kvm *kvm);
+extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
+extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
+extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
+extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
+extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 6e637d2b4cfb..1656dd80bbc4 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -35,6 +35,7 @@
#include <asm/kvm_arm.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/kvm_emulate.h>
#include <asm/sections.h>
@@ -142,6 +143,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
if (ret)
return ret;
+ kvm_init_nested(kvm);
+
ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
if (ret)
goto out_free_stage2_pgd;
@@ -385,6 +388,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
struct kvm_s2_mmu *mmu;
int *last_ran;
+ if (nested_virt_in_use(vcpu))
+ kvm_vcpu_load_hw_mmu(vcpu);
+
mmu = vcpu->arch.hw_mmu;
last_ran = this_cpu_ptr(mmu->last_vcpu_ran);
@@ -426,6 +432,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
kvm_vgic_put(vcpu);
kvm_vcpu_pmu_restore_host(vcpu);
+ if (nested_virt_in_use(vcpu))
+ kvm_vcpu_put_hw_mmu(vcpu);
+
vcpu->cpu = -1;
}
@@ -1026,8 +1035,13 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
vcpu->arch.target = phys_target;
+ /* Prepare for nested if required */
+ ret = kvm_vcpu_init_nested(vcpu);
+
/* Now we know what it is, we can reset it. */
- ret = kvm_reset_vcpu(vcpu);
+ if (!ret)
+ ret = kvm_reset_vcpu(vcpu);
+
if (ret) {
vcpu->arch.target = -1;
bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 1f41173e6149..2f0302211af3 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -113,7 +113,7 @@ static bool kvm_is_device_pfn(unsigned long pfn)
* does.
*/
/**
- * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
+ * kvm_unmap_stage2_range -- Clear stage2 page table entries to unmap a range
* @mmu: The KVM stage-2 MMU pointer
* @start: The intermediate physical base address of the range to unmap
* @size: The size of the area to unmap
@@ -136,7 +136,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
may_block));
}
-static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
+void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
{
__unmap_stage2_range(mmu, start, size, true);
}
@@ -391,6 +391,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
mmu->pgt = pgt;
mmu->pgd_phys = __pa(pgt->pgd);
mmu->vmid.vmid_gen = 0;
+
+ kvm_init_nested_s2_mmu(mmu);
+
return 0;
out_destroy_pgtable:
@@ -435,7 +438,7 @@ static void stage2_unmap_memslot(struct kvm *kvm,
if (!(vma->vm_flags & VM_PFNMAP)) {
gpa_t gpa = addr + (vm_start - memslot->userspace_addr);
- unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
}
hva = vm_end;
} while (hva < reg_end);
@@ -1360,7 +1363,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
spin_lock(&kvm->mmu_lock);
if (ret)
- unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
stage2_flush_memslot(kvm, memslot);
spin_unlock(&kvm->mmu_lock);
@@ -1377,11 +1380,6 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
{
}
-void kvm_arch_flush_shadow_all(struct kvm *kvm)
-{
- kvm_free_stage2_pgd(&kvm->arch.mmu);
-}
-
void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
struct kvm_memory_slot *slot)
{
@@ -1389,7 +1387,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
phys_addr_t size = slot->npages << PAGE_SHIFT;
spin_lock(&kvm->mmu_lock);
- unmap_stage2_range(&kvm->arch.mmu, gpa, size);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9fb44bc7db3f..8e85d2ef24d9 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -19,12 +19,177 @@
#include <linux/kvm.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_arm.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_mmu.h>
#include <asm/kvm_nested.h>
#include <asm/sysreg.h>
#include "sys_regs.h"
+void kvm_init_nested(struct kvm *kvm)
+{
+ kvm->arch.nested_mmus = NULL;
+ kvm->arch.nested_mmus_size = 0;
+}
+
+int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_s2_mmu *tmp;
+ int num_mmus;
+ int ret = -ENOMEM;
+
+ if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
+ return 0;
+
+ if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
+ return -EINVAL;
+
+ mutex_lock(&kvm->lock);
+
+ /*
+ * Let's treat memory allocation failures as benign: If we fail to
+ * allocate anything, return an error and keep the allocated array
+ * alive. Userspace may try to recover by intializing the vcpu
+ * again, and there is no reason to affect the whole VM for this.
+ */
+ num_mmus = atomic_read(&kvm->online_vcpus) * 2;
+ tmp = krealloc(kvm->arch.nested_mmus,
+ num_mmus * sizeof(*kvm->arch.nested_mmus),
+ GFP_KERNEL | __GFP_ZERO);
+ if (tmp) {
+ if (kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 1]) ||
+ kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
+ kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
+ kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
+ } else {
+ kvm->arch.nested_mmus_size = num_mmus;
+ ret = 0;
+ }
+
+ kvm->arch.nested_mmus = tmp;
+ }
+
+ mutex_unlock(&kvm->lock);
+ return ret;
+}
+
+/* Must be called with kvm->lock held */
+struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
+{
+ bool nested_stage2_enabled = hcr & HCR_VM;
+ int i;
+
+ /* Don't consider the CnP bit for the vttbr match */
+ vttbr = vttbr & ~VTTBR_CNP_BIT;
+
+ /*
+ * Two possibilities when looking up a S2 MMU context:
+ *
+ * - either S2 is enabled in the guest, and we need a context that
+ * is S2-enabled and matches the full VTTBR (VMID+BADDR), which
+ * makes it safe from a TLB conflict perspective (a broken guest
+ * won't be able to generate them),
+ *
+ * - or S2 is disabled, and we need a context that is S2-disabled
+ * and matches the VMID only, as all TLBs are tagged by VMID even
+ * if S2 translation is enabled.
+ */
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (!kvm_s2_mmu_valid(mmu))
+ continue;
+
+ if (nested_stage2_enabled &&
+ mmu->nested_stage2_enabled &&
+ vttbr == mmu->vttbr)
+ return mmu;
+
+ if (!nested_stage2_enabled &&
+ !mmu->nested_stage2_enabled &&
+ get_vmid(vttbr) == get_vmid(mmu->vttbr))
+ return mmu;
+ }
+ return NULL;
+}
+
+static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ u64 hcr= vcpu_read_sys_reg(vcpu, HCR_EL2);
+ struct kvm_s2_mmu *s2_mmu;
+ int i;
+
+ s2_mmu = lookup_s2_mmu(kvm, vttbr, hcr);
+ if (s2_mmu)
+ goto out;
+
+ /*
+ * Make sure we don't always search from the same point, or we
+ * will always reuse a potentially active context, leaving
+ * free contexts unused.
+ */
+ for (i = kvm->arch.nested_mmus_next;
+ i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
+ i++) {
+ s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
+
+ if (atomic_read(&s2_mmu->refcnt) == 0)
+ break;
+ }
+ BUG_ON(atomic_read(&s2_mmu->refcnt)); /* We have struct MMUs to spare */
+
+ /* Set the scene for the next search */
+ kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
+
+ if (kvm_s2_mmu_valid(s2_mmu)) {
+ /* Clear the old state */
+ kvm_unmap_stage2_range(s2_mmu, 0, kvm_phys_size(kvm));
+ if (s2_mmu->vmid.vmid_gen)
+ kvm_call_hyp(__kvm_tlb_flush_vmid, s2_mmu);
+ }
+
+ /*
+ * The virtual VMID (modulo CnP) will be used as a key when matching
+ * an existing kvm_s2_mmu.
+ */
+ s2_mmu->vttbr = vttbr & ~VTTBR_CNP_BIT;
+ s2_mmu->nested_stage2_enabled = hcr & HCR_VM;
+
+out:
+ atomic_inc(&s2_mmu->refcnt);
+ return s2_mmu;
+}
+
+void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)
+{
+ mmu->vttbr = 1;
+ mmu->nested_stage2_enabled = false;
+ atomic_set(&mmu->refcnt, 0);
+}
+
+void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
+{
+ if (is_hyp_ctxt(vcpu)) {
+ vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
+ } else {
+ spin_lock(&vcpu->kvm->mmu_lock);
+ vcpu->arch.hw_mmu = get_s2_mmu_nested(vcpu);
+ spin_unlock(&vcpu->kvm->mmu_lock);
+ }
+}
+
+void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu) {
+ atomic_dec(&vcpu->arch.hw_mmu->refcnt);
+ vcpu->arch.hw_mmu = NULL;
+ }
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
@@ -43,6 +208,24 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
return -EINVAL;
}
+void kvm_arch_flush_shadow_all(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ WARN_ON(atomic_read(&mmu->refcnt));
+
+ if (!atomic_read(&mmu->refcnt))
+ kvm_free_stage2_pgd(mmu);
+ }
+ kfree(kvm->arch.nested_mmus);
+ kvm->arch.nested_mmus = NULL;
+ kvm->arch.nested_mmus_size = 0;
+ kvm_free_stage2_pgd(&kvm->arch.mmu);
+}
+
#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
/*
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Add Stage-2 mmu data structures for virtual EL2 and for nested guests.
We don't yet populate shadow Stage-2 page tables, but we now have a
framework for getting to a shadow Stage-2 pgd.
We allocate twice the number of vcpus as Stage-2 mmu structures because
that's sufficient for each vcpu running two translation regimes without
having to flush the Stage-2 page tables.
Co-developed-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 29 +++++
arch/arm64/include/asm/kvm_mmu.h | 8 ++
arch/arm64/include/asm/kvm_nested.h | 7 ++
arch/arm64/kvm/arm.c | 16 ++-
arch/arm64/kvm/mmu.c | 18 ++-
arch/arm64/kvm/nested.c | 183 ++++++++++++++++++++++++++++
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d731cf7a56cb..d99e51e7cbee 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -95,14 +95,43 @@ struct kvm_s2_mmu {
int __percpu *last_vcpu_ran;
struct kvm *kvm;
+
+ /*
+ * For a shadow stage-2 MMU, the virtual vttbr programmed by the guest
+ * hypervisor. Unused for kvm_arch->mmu. Set to 1 when the structure
+ * contains no valid information.
+ */
+ u64 vttbr;
+
+ /* true when this represents a nested context where virtual HCR_EL2.VM == 1 */
+ bool nested_stage2_enabled;
+
+ /*
+ * 0: Nobody is currently using this, check vttbr for validity
+ * >0: Somebody is actively using this.
+ */
+ atomic_t refcnt;
};
+static inline bool kvm_s2_mmu_valid(struct kvm_s2_mmu *mmu)
+{
+ return !(mmu->vttbr & 1);
+}
+
struct kvm_arch_memory_slot {
};
struct kvm_arch {
struct kvm_s2_mmu mmu;
+ /*
+ * Stage 2 paging stage for VMs with nested virtual using a virtual
+ * VMID.
+ */
+ struct kvm_s2_mmu *nested_mmus;
+ size_t nested_mmus_size;
+ int nested_mmus_next;
+
/* VTCR_EL2 value for this VM */
u64 vtcr;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 76a8a0ca45b8..ec39015bb2a6 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -126,6 +126,7 @@ alternative_cb_end
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
void kvm_update_va_mask(struct alt_instr *alt,
__le32 *origptr, __le32 *updptr, int nr_inst);
@@ -184,6 +185,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
void **haddr);
void free_hyp_pgds(void);
+void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
void stage2_unmap_vm(struct kvm *kvm);
int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
@@ -306,5 +308,11 @@ static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu)
asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
}
+static inline u64 get_vmid(u64 vttbr)
+{
+ return (vttbr & VTTBR_VMID_MASK(kvm_get_vmid_bits())) >>
+ VTTBR_VMID_SHIFT;
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 026ddaad972c..473ecd1d60d0 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -61,6 +61,13 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
(cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
}
+extern void kvm_init_nested(struct kvm *kvm);
+extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
+extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
+extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
+extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
+extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 6e637d2b4cfb..1656dd80bbc4 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -35,6 +35,7 @@
#include <asm/kvm_arm.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/kvm_emulate.h>
#include <asm/sections.h>
@@ -142,6 +143,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
if (ret)
return ret;
+ kvm_init_nested(kvm);
+
ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
if (ret)
goto out_free_stage2_pgd;
@@ -385,6 +388,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
struct kvm_s2_mmu *mmu;
int *last_ran;
+ if (nested_virt_in_use(vcpu))
+ kvm_vcpu_load_hw_mmu(vcpu);
+
mmu = vcpu->arch.hw_mmu;
last_ran = this_cpu_ptr(mmu->last_vcpu_ran);
@@ -426,6 +432,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
kvm_vgic_put(vcpu);
kvm_vcpu_pmu_restore_host(vcpu);
+ if (nested_virt_in_use(vcpu))
+ kvm_vcpu_put_hw_mmu(vcpu);
+
vcpu->cpu = -1;
}
@@ -1026,8 +1035,13 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
vcpu->arch.target = phys_target;
+ /* Prepare for nested if required */
+ ret = kvm_vcpu_init_nested(vcpu);
+
/* Now we know what it is, we can reset it. */
- ret = kvm_reset_vcpu(vcpu);
+ if (!ret)
+ ret = kvm_reset_vcpu(vcpu);
+
if (ret) {
vcpu->arch.target = -1;
bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 1f41173e6149..2f0302211af3 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -113,7 +113,7 @@ static bool kvm_is_device_pfn(unsigned long pfn)
* does.
*/
/**
- * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
+ * kvm_unmap_stage2_range -- Clear stage2 page table entries to unmap a range
* @mmu: The KVM stage-2 MMU pointer
* @start: The intermediate physical base address of the range to unmap
* @size: The size of the area to unmap
@@ -136,7 +136,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
may_block));
}
-static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
+void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
{
__unmap_stage2_range(mmu, start, size, true);
}
@@ -391,6 +391,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
mmu->pgt = pgt;
mmu->pgd_phys = __pa(pgt->pgd);
mmu->vmid.vmid_gen = 0;
+
+ kvm_init_nested_s2_mmu(mmu);
+
return 0;
out_destroy_pgtable:
@@ -435,7 +438,7 @@ static void stage2_unmap_memslot(struct kvm *kvm,
if (!(vma->vm_flags & VM_PFNMAP)) {
gpa_t gpa = addr + (vm_start - memslot->userspace_addr);
- unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
}
hva = vm_end;
} while (hva < reg_end);
@@ -1360,7 +1363,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
spin_lock(&kvm->mmu_lock);
if (ret)
- unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
stage2_flush_memslot(kvm, memslot);
spin_unlock(&kvm->mmu_lock);
@@ -1377,11 +1380,6 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
{
}
-void kvm_arch_flush_shadow_all(struct kvm *kvm)
-{
- kvm_free_stage2_pgd(&kvm->arch.mmu);
-}
-
void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
struct kvm_memory_slot *slot)
{
@@ -1389,7 +1387,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
phys_addr_t size = slot->npages << PAGE_SHIFT;
spin_lock(&kvm->mmu_lock);
- unmap_stage2_range(&kvm->arch.mmu, gpa, size);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9fb44bc7db3f..8e85d2ef24d9 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -19,12 +19,177 @@
#include <linux/kvm.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_arm.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_mmu.h>
#include <asm/kvm_nested.h>
#include <asm/sysreg.h>
#include "sys_regs.h"
+void kvm_init_nested(struct kvm *kvm)
+{
+ kvm->arch.nested_mmus = NULL;
+ kvm->arch.nested_mmus_size = 0;
+}
+
+int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_s2_mmu *tmp;
+ int num_mmus;
+ int ret = -ENOMEM;
+
+ if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
+ return 0;
+
+ if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
+ return -EINVAL;
+
+ mutex_lock(&kvm->lock);
+
+ /*
+ * Let's treat memory allocation failures as benign: If we fail to
+ * allocate anything, return an error and keep the allocated array
+ * alive. Userspace may try to recover by intializing the vcpu
+ * again, and there is no reason to affect the whole VM for this.
+ */
+ num_mmus = atomic_read(&kvm->online_vcpus) * 2;
+ tmp = krealloc(kvm->arch.nested_mmus,
+ num_mmus * sizeof(*kvm->arch.nested_mmus),
+ GFP_KERNEL | __GFP_ZERO);
+ if (tmp) {
+ if (kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 1]) ||
+ kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
+ kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
+ kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
+ } else {
+ kvm->arch.nested_mmus_size = num_mmus;
+ ret = 0;
+ }
+
+ kvm->arch.nested_mmus = tmp;
+ }
+
+ mutex_unlock(&kvm->lock);
+ return ret;
+}
+
+/* Must be called with kvm->lock held */
+struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
+{
+ bool nested_stage2_enabled = hcr & HCR_VM;
+ int i;
+
+ /* Don't consider the CnP bit for the vttbr match */
+ vttbr = vttbr & ~VTTBR_CNP_BIT;
+
+ /*
+ * Two possibilities when looking up a S2 MMU context:
+ *
+ * - either S2 is enabled in the guest, and we need a context that
+ * is S2-enabled and matches the full VTTBR (VMID+BADDR), which
+ * makes it safe from a TLB conflict perspective (a broken guest
+ * won't be able to generate them),
+ *
+ * - or S2 is disabled, and we need a context that is S2-disabled
+ * and matches the VMID only, as all TLBs are tagged by VMID even
+ * if S2 translation is enabled.
+ */
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (!kvm_s2_mmu_valid(mmu))
+ continue;
+
+ if (nested_stage2_enabled &&
+ mmu->nested_stage2_enabled &&
+ vttbr == mmu->vttbr)
+ return mmu;
+
+ if (!nested_stage2_enabled &&
+ !mmu->nested_stage2_enabled &&
+ get_vmid(vttbr) == get_vmid(mmu->vttbr))
+ return mmu;
+ }
+ return NULL;
+}
+
+static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ u64 hcr= vcpu_read_sys_reg(vcpu, HCR_EL2);
+ struct kvm_s2_mmu *s2_mmu;
+ int i;
+
+ s2_mmu = lookup_s2_mmu(kvm, vttbr, hcr);
+ if (s2_mmu)
+ goto out;
+
+ /*
+ * Make sure we don't always search from the same point, or we
+ * will always reuse a potentially active context, leaving
+ * free contexts unused.
+ */
+ for (i = kvm->arch.nested_mmus_next;
+ i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
+ i++) {
+ s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
+
+ if (atomic_read(&s2_mmu->refcnt) == 0)
+ break;
+ }
+ BUG_ON(atomic_read(&s2_mmu->refcnt)); /* We have struct MMUs to spare */
+
+ /* Set the scene for the next search */
+ kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
+
+ if (kvm_s2_mmu_valid(s2_mmu)) {
+ /* Clear the old state */
+ kvm_unmap_stage2_range(s2_mmu, 0, kvm_phys_size(kvm));
+ if (s2_mmu->vmid.vmid_gen)
+ kvm_call_hyp(__kvm_tlb_flush_vmid, s2_mmu);
+ }
+
+ /*
+ * The virtual VMID (modulo CnP) will be used as a key when matching
+ * an existing kvm_s2_mmu.
+ */
+ s2_mmu->vttbr = vttbr & ~VTTBR_CNP_BIT;
+ s2_mmu->nested_stage2_enabled = hcr & HCR_VM;
+
+out:
+ atomic_inc(&s2_mmu->refcnt);
+ return s2_mmu;
+}
+
+void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)
+{
+ mmu->vttbr = 1;
+ mmu->nested_stage2_enabled = false;
+ atomic_set(&mmu->refcnt, 0);
+}
+
+void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
+{
+ if (is_hyp_ctxt(vcpu)) {
+ vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
+ } else {
+ spin_lock(&vcpu->kvm->mmu_lock);
+ vcpu->arch.hw_mmu = get_s2_mmu_nested(vcpu);
+ spin_unlock(&vcpu->kvm->mmu_lock);
+ }
+}
+
+void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu) {
+ atomic_dec(&vcpu->arch.hw_mmu->refcnt);
+ vcpu->arch.hw_mmu = NULL;
+ }
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
@@ -43,6 +208,24 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
return -EINVAL;
}
+void kvm_arch_flush_shadow_all(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ WARN_ON(atomic_read(&mmu->refcnt));
+
+ if (!atomic_read(&mmu->refcnt))
+ kvm_free_stage2_pgd(mmu);
+ }
+ kfree(kvm->arch.nested_mmus);
+ kvm->arch.nested_mmus = NULL;
+ kvm->arch.nested_mmus_size = 0;
+ kvm_free_stage2_pgd(&kvm->arch.mmu);
+}
+
#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
/*
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Add Stage-2 mmu data structures for virtual EL2 and for nested guests.
We don't yet populate shadow Stage-2 page tables, but we now have a
framework for getting to a shadow Stage-2 pgd.
We allocate twice the number of vcpus as Stage-2 mmu structures because
that's sufficient for each vcpu running two translation regimes without
having to flush the Stage-2 page tables.
Co-developed-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 29 +++++
arch/arm64/include/asm/kvm_mmu.h | 8 ++
arch/arm64/include/asm/kvm_nested.h | 7 ++
arch/arm64/kvm/arm.c | 16 ++-
arch/arm64/kvm/mmu.c | 18 ++-
arch/arm64/kvm/nested.c | 183 ++++++++++++++++++++++++++++
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d731cf7a56cb..d99e51e7cbee 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -95,14 +95,43 @@ struct kvm_s2_mmu {
int __percpu *last_vcpu_ran;
struct kvm *kvm;
+
+ /*
+ * For a shadow stage-2 MMU, the virtual vttbr programmed by the guest
+ * hypervisor. Unused for kvm_arch->mmu. Set to 1 when the structure
+ * contains no valid information.
+ */
+ u64 vttbr;
+
+ /* true when this represents a nested context where virtual HCR_EL2.VM == 1 */
+ bool nested_stage2_enabled;
+
+ /*
+ * 0: Nobody is currently using this, check vttbr for validity
+ * >0: Somebody is actively using this.
+ */
+ atomic_t refcnt;
};
+static inline bool kvm_s2_mmu_valid(struct kvm_s2_mmu *mmu)
+{
+ return !(mmu->vttbr & 1);
+}
+
struct kvm_arch_memory_slot {
};
struct kvm_arch {
struct kvm_s2_mmu mmu;
+ /*
+ * Stage 2 paging stage for VMs with nested virtual using a virtual
+ * VMID.
+ */
+ struct kvm_s2_mmu *nested_mmus;
+ size_t nested_mmus_size;
+ int nested_mmus_next;
+
/* VTCR_EL2 value for this VM */
u64 vtcr;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 76a8a0ca45b8..ec39015bb2a6 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -126,6 +126,7 @@ alternative_cb_end
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_nested.h>
void kvm_update_va_mask(struct alt_instr *alt,
__le32 *origptr, __le32 *updptr, int nr_inst);
@@ -184,6 +185,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
void **haddr);
void free_hyp_pgds(void);
+void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
void stage2_unmap_vm(struct kvm *kvm);
int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
@@ -306,5 +308,11 @@ static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu)
asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
}
+static inline u64 get_vmid(u64 vttbr)
+{
+ return (vttbr & VTTBR_VMID_MASK(kvm_get_vmid_bits())) >>
+ VTTBR_VMID_SHIFT;
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 026ddaad972c..473ecd1d60d0 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -61,6 +61,13 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
(cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
}
+extern void kvm_init_nested(struct kvm *kvm);
+extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
+extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
+extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
+extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
+extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 6e637d2b4cfb..1656dd80bbc4 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -35,6 +35,7 @@
#include <asm/kvm_arm.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/kvm_emulate.h>
#include <asm/sections.h>
@@ -142,6 +143,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
if (ret)
return ret;
+ kvm_init_nested(kvm);
+
ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
if (ret)
goto out_free_stage2_pgd;
@@ -385,6 +388,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
struct kvm_s2_mmu *mmu;
int *last_ran;
+ if (nested_virt_in_use(vcpu))
+ kvm_vcpu_load_hw_mmu(vcpu);
+
mmu = vcpu->arch.hw_mmu;
last_ran = this_cpu_ptr(mmu->last_vcpu_ran);
@@ -426,6 +432,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
kvm_vgic_put(vcpu);
kvm_vcpu_pmu_restore_host(vcpu);
+ if (nested_virt_in_use(vcpu))
+ kvm_vcpu_put_hw_mmu(vcpu);
+
vcpu->cpu = -1;
}
@@ -1026,8 +1035,13 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
vcpu->arch.target = phys_target;
+ /* Prepare for nested if required */
+ ret = kvm_vcpu_init_nested(vcpu);
+
/* Now we know what it is, we can reset it. */
- ret = kvm_reset_vcpu(vcpu);
+ if (!ret)
+ ret = kvm_reset_vcpu(vcpu);
+
if (ret) {
vcpu->arch.target = -1;
bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 1f41173e6149..2f0302211af3 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -113,7 +113,7 @@ static bool kvm_is_device_pfn(unsigned long pfn)
* does.
*/
/**
- * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
+ * kvm_unmap_stage2_range -- Clear stage2 page table entries to unmap a range
* @mmu: The KVM stage-2 MMU pointer
* @start: The intermediate physical base address of the range to unmap
* @size: The size of the area to unmap
@@ -136,7 +136,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
may_block));
}
-static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
+void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
{
__unmap_stage2_range(mmu, start, size, true);
}
@@ -391,6 +391,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
mmu->pgt = pgt;
mmu->pgd_phys = __pa(pgt->pgd);
mmu->vmid.vmid_gen = 0;
+
+ kvm_init_nested_s2_mmu(mmu);
+
return 0;
out_destroy_pgtable:
@@ -435,7 +438,7 @@ static void stage2_unmap_memslot(struct kvm *kvm,
if (!(vma->vm_flags & VM_PFNMAP)) {
gpa_t gpa = addr + (vm_start - memslot->userspace_addr);
- unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
}
hva = vm_end;
} while (hva < reg_end);
@@ -1360,7 +1363,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
spin_lock(&kvm->mmu_lock);
if (ret)
- unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
stage2_flush_memslot(kvm, memslot);
spin_unlock(&kvm->mmu_lock);
@@ -1377,11 +1380,6 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
{
}
-void kvm_arch_flush_shadow_all(struct kvm *kvm)
-{
- kvm_free_stage2_pgd(&kvm->arch.mmu);
-}
-
void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
struct kvm_memory_slot *slot)
{
@@ -1389,7 +1387,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
phys_addr_t size = slot->npages << PAGE_SHIFT;
spin_lock(&kvm->mmu_lock);
- unmap_stage2_range(&kvm->arch.mmu, gpa, size);
+ kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9fb44bc7db3f..8e85d2ef24d9 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -19,12 +19,177 @@
#include <linux/kvm.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_arm.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_mmu.h>
#include <asm/kvm_nested.h>
#include <asm/sysreg.h>
#include "sys_regs.h"
+void kvm_init_nested(struct kvm *kvm)
+{
+ kvm->arch.nested_mmus = NULL;
+ kvm->arch.nested_mmus_size = 0;
+}
+
+int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_s2_mmu *tmp;
+ int num_mmus;
+ int ret = -ENOMEM;
+
+ if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
+ return 0;
+
+ if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
+ return -EINVAL;
+
+ mutex_lock(&kvm->lock);
+
+ /*
+ * Let's treat memory allocation failures as benign: If we fail to
+ * allocate anything, return an error and keep the allocated array
+ * alive. Userspace may try to recover by intializing the vcpu
+ * again, and there is no reason to affect the whole VM for this.
+ */
+ num_mmus = atomic_read(&kvm->online_vcpus) * 2;
+ tmp = krealloc(kvm->arch.nested_mmus,
+ num_mmus * sizeof(*kvm->arch.nested_mmus),
+ GFP_KERNEL | __GFP_ZERO);
+ if (tmp) {
+ if (kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 1]) ||
+ kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
+ kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
+ kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
+ } else {
+ kvm->arch.nested_mmus_size = num_mmus;
+ ret = 0;
+ }
+
+ kvm->arch.nested_mmus = tmp;
+ }
+
+ mutex_unlock(&kvm->lock);
+ return ret;
+}
+
+/* Must be called with kvm->lock held */
+struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
+{
+ bool nested_stage2_enabled = hcr & HCR_VM;
+ int i;
+
+ /* Don't consider the CnP bit for the vttbr match */
+ vttbr = vttbr & ~VTTBR_CNP_BIT;
+
+ /*
+ * Two possibilities when looking up a S2 MMU context:
+ *
+ * - either S2 is enabled in the guest, and we need a context that
+ * is S2-enabled and matches the full VTTBR (VMID+BADDR), which
+ * makes it safe from a TLB conflict perspective (a broken guest
+ * won't be able to generate them),
+ *
+ * - or S2 is disabled, and we need a context that is S2-disabled
+ * and matches the VMID only, as all TLBs are tagged by VMID even
+ * if S2 translation is enabled.
+ */
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (!kvm_s2_mmu_valid(mmu))
+ continue;
+
+ if (nested_stage2_enabled &&
+ mmu->nested_stage2_enabled &&
+ vttbr == mmu->vttbr)
+ return mmu;
+
+ if (!nested_stage2_enabled &&
+ !mmu->nested_stage2_enabled &&
+ get_vmid(vttbr) == get_vmid(mmu->vttbr))
+ return mmu;
+ }
+ return NULL;
+}
+
+static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ u64 hcr= vcpu_read_sys_reg(vcpu, HCR_EL2);
+ struct kvm_s2_mmu *s2_mmu;
+ int i;
+
+ s2_mmu = lookup_s2_mmu(kvm, vttbr, hcr);
+ if (s2_mmu)
+ goto out;
+
+ /*
+ * Make sure we don't always search from the same point, or we
+ * will always reuse a potentially active context, leaving
+ * free contexts unused.
+ */
+ for (i = kvm->arch.nested_mmus_next;
+ i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
+ i++) {
+ s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
+
+ if (atomic_read(&s2_mmu->refcnt) == 0)
+ break;
+ }
+ BUG_ON(atomic_read(&s2_mmu->refcnt)); /* We have struct MMUs to spare */
+
+ /* Set the scene for the next search */
+ kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
+
+ if (kvm_s2_mmu_valid(s2_mmu)) {
+ /* Clear the old state */
+ kvm_unmap_stage2_range(s2_mmu, 0, kvm_phys_size(kvm));
+ if (s2_mmu->vmid.vmid_gen)
+ kvm_call_hyp(__kvm_tlb_flush_vmid, s2_mmu);
+ }
+
+ /*
+ * The virtual VMID (modulo CnP) will be used as a key when matching
+ * an existing kvm_s2_mmu.
+ */
+ s2_mmu->vttbr = vttbr & ~VTTBR_CNP_BIT;
+ s2_mmu->nested_stage2_enabled = hcr & HCR_VM;
+
+out:
+ atomic_inc(&s2_mmu->refcnt);
+ return s2_mmu;
+}
+
+void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)
+{
+ mmu->vttbr = 1;
+ mmu->nested_stage2_enabled = false;
+ atomic_set(&mmu->refcnt, 0);
+}
+
+void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
+{
+ if (is_hyp_ctxt(vcpu)) {
+ vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
+ } else {
+ spin_lock(&vcpu->kvm->mmu_lock);
+ vcpu->arch.hw_mmu = get_s2_mmu_nested(vcpu);
+ spin_unlock(&vcpu->kvm->mmu_lock);
+ }
+}
+
+void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu) {
+ atomic_dec(&vcpu->arch.hw_mmu->refcnt);
+ vcpu->arch.hw_mmu = NULL;
+ }
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
@@ -43,6 +208,24 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
return -EINVAL;
}
+void kvm_arch_flush_shadow_all(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ WARN_ON(atomic_read(&mmu->refcnt));
+
+ if (!atomic_read(&mmu->refcnt))
+ kvm_free_stage2_pgd(mmu);
+ }
+ kfree(kvm->arch.nested_mmus);
+ kvm->arch.nested_mmus = NULL;
+ kvm->arch.nested_mmus_size = 0;
+ kvm_free_stage2_pgd(&kvm->arch.mmu);
+}
+
#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
/*
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 34/66] KVM: arm64: nv: Implement nested Stage-2 page table walk logic
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Christoffer Dall, Jintack Lim
From: Christoffer Dall <christoffer.dall@linaro.org>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software
page table walker.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: heavily reworked for future ARMv8.4-TTL support]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/esr.h | 1 +
arch/arm64/include/asm/kvm_arm.h | 2 +
arch/arm64/include/asm/kvm_nested.h | 13 ++
arch/arm64/kvm/nested.c | 276 ++++++++++++++++++++++++++++
4 files changed, 292 insertions(+)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index ab63b7b3ea14..e099eba5e616 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -127,6 +127,7 @@
#define ESR_ELx_CM (UL(1) << ESR_ELx_CM_SHIFT)
/* ISS field definitions for exceptions taken in to Hyp */
+#define ESR_ELx_FSC_ADDRSZ (0x00)
#define ESR_ELx_CV (UL(1) << 24)
#define ESR_ELx_COND_SHIFT (20)
#define ESR_ELx_COND_MASK (UL(0xF) << ESR_ELx_COND_SHIFT)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3d2abc40d3a2..347e9e9b015c 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -264,6 +264,8 @@
#define VTTBR_VMID_SHIFT (UL(48))
#define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT)
+#define SCTLR_EE (UL(1) << 25)
+
/* Hyp System Trap Register */
#define HSTR_EL2_T(x) (1 << x)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 473ecd1d60d0..b784d7891851 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -68,6 +68,19 @@ extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+struct kvm_s2_trans {
+ phys_addr_t output;
+ unsigned long block_size;
+ bool writable;
+ bool readable;
+ int level;
+ u32 esr;
+ u64 upper_attr;
+};
+
+extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
+ struct kvm_s2_trans *result);
+
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 8e85d2ef24d9..ff970221a219 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -75,6 +75,282 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return ret;
}
+struct s2_walk_info {
+ int (*read_desc)(phys_addr_t pa, u64 *desc, void *data);
+ void *data;
+ u64 baddr;
+ unsigned int max_pa_bits;
+ unsigned int pgshift;
+ unsigned int pgsize;
+ unsigned int ps;
+ unsigned int sl;
+ unsigned int t0sz;
+ bool be;
+ bool el1_aarch32;
+};
+
+static unsigned int ps_to_output_size(unsigned int ps)
+{
+ switch (ps) {
+ case 0: return 32;
+ case 1: return 36;
+ case 2: return 40;
+ case 3: return 42;
+ case 4: return 44;
+ case 5:
+ default:
+ return 48;
+ }
+}
+
+static u32 compute_fsc(int level, u32 fsc)
+{
+ return fsc | (level & 0x3);
+}
+
+static int esr_s2_fault(struct kvm_vcpu *vcpu, int level, u32 fsc)
+{
+ u32 esr;
+
+ esr = kvm_vcpu_get_esr(vcpu) & ~ESR_ELx_FSC;
+ esr |= compute_fsc(level, fsc);
+ return esr;
+}
+
+static int check_base_s2_limits(struct s2_walk_info *wi,
+ int level, int input_size, int stride)
+{
+ int start_size;
+
+ /* Check translation limits */
+ switch (wi->pgsize) {
+ case SZ_64K:
+ if (level == 0 || (level == 1 && wi->max_pa_bits <= 42))
+ return -EFAULT;
+ break;
+ case SZ_16K:
+ if (level == 0 || (level == 1 && wi->max_pa_bits <= 40))
+ return -EFAULT;
+ break;
+ case SZ_4K:
+ if (level < 0 || (level == 0 && wi->max_pa_bits <= 42))
+ return -EFAULT;
+ break;
+ }
+
+ /* Check input size limits */
+ if (input_size > wi->max_pa_bits &&
+ (!wi->el1_aarch32 || input_size > 40))
+ return -EFAULT;
+
+ /* Check number of entries in starting level table */
+ start_size = input_size - ((3 - level) * stride + wi->pgshift);
+ if (start_size < 1 || start_size > stride + 4)
+ return -EFAULT;
+
+ return 0;
+}
+
+/* Check if output is within boundaries */
+static int check_output_size(struct s2_walk_info *wi, phys_addr_t output)
+{
+ unsigned int output_size = ps_to_output_size(wi->ps);
+
+ if (output_size > wi->max_pa_bits)
+ output_size = wi->max_pa_bits;
+
+ if (output_size != 48 && (output & GENMASK_ULL(47, output_size)))
+ return -1;
+
+ return 0;
+}
+
+/*
+ * This is essentially a C-version of the pseudo code from the ARM ARM
+ * AArch64.TranslationTableWalk function. I strongly recommend looking at
+ * that pseudocode in trying to understand this.
+ *
+ * Must be called with the kvm->srcu read lock held
+ */
+static int walk_nested_s2_pgd(phys_addr_t ipa,
+ struct s2_walk_info *wi, struct kvm_s2_trans *out)
+{
+ int first_block_level, level, stride, input_size, base_lower_bound;
+ phys_addr_t base_addr;
+ unsigned int addr_top, addr_bottom;
+ u64 desc; /* page table entry */
+ int ret;
+ phys_addr_t paddr;
+
+ switch (wi->pgsize) {
+ case SZ_64K:
+ case SZ_16K:
+ level = 3 - wi->sl;
+ first_block_level = 2;
+ break;
+ case SZ_4K:
+ level = 2 - wi->sl;
+ first_block_level = 1;
+ break;
+ default:
+ /* GCC is braindead */
+ unreachable();
+ }
+
+ stride = wi->pgshift - 3;
+ input_size = 64 - wi->t0sz;
+ if (input_size > 48 || input_size < 25)
+ return -EFAULT;
+
+ ret = check_base_s2_limits(wi, level, input_size, stride);
+ if (WARN_ON(ret))
+ return ret;
+
+ base_lower_bound = 3 + input_size - ((3 - level) * stride +
+ wi->pgshift);
+ base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound);
+
+ if (check_output_size(wi, base_addr)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ return 1;
+ }
+
+ addr_top = input_size - 1;
+
+ while (1) {
+ phys_addr_t index;
+
+ addr_bottom = (3 - level) * stride + wi->pgshift;
+ index = (ipa & GENMASK_ULL(addr_top, addr_bottom))
+ >> (addr_bottom - 3);
+
+ paddr = base_addr | index;
+ ret = wi->read_desc(paddr, &desc, wi->data);
+ if (ret < 0)
+ return ret;
+
+ /*
+ * Handle reversedescriptors if endianness differs between the
+ * host and the guest hypervisor.
+ */
+ if (wi->be)
+ desc = be64_to_cpu(desc);
+ else
+ desc = le64_to_cpu(desc);
+
+ /* Check for valid descriptor at this point */
+ if (!(desc & 1) || ((desc & 3) == 1 && level == 3)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /* We're at the final level or block translation level */
+ if ((desc & 3) == 1 || level == 3)
+ break;
+
+ if (check_output_size(wi, desc)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ base_addr = desc & GENMASK_ULL(47, wi->pgshift);
+
+ level += 1;
+ addr_top = addr_bottom - 1;
+ }
+
+ if (level < first_block_level) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /*
+ * We don't use the contiguous bit in the stage-2 ptes, so skip check
+ * for misprogramming of the contiguous bit.
+ */
+
+ if (check_output_size(wi, desc)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ if (!(desc & BIT(10))) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ACCESS);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /* Calculate and return the result */
+ paddr = (desc & GENMASK_ULL(47, addr_bottom)) |
+ (ipa & GENMASK_ULL(addr_bottom - 1, 0));
+ out->output = paddr;
+ out->block_size = 1UL << ((3 - level) * stride + wi->pgshift);
+ out->readable = desc & (0b01 << 6);
+ out->writable = desc & (0b10 << 6);
+ out->level = level;
+ out->upper_attr = desc & GENMASK_ULL(63, 52);
+ return 0;
+}
+
+static int read_guest_s2_desc(phys_addr_t pa, u64 *desc, void *data)
+{
+ struct kvm_vcpu *vcpu = data;
+
+ return kvm_read_guest(vcpu->kvm, pa, desc, sizeof(*desc));
+}
+
+static void vtcr_to_walk_info(u64 vtcr, struct s2_walk_info *wi)
+{
+ wi->t0sz = vtcr & TCR_EL2_T0SZ_MASK;
+
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ wi->pgshift = 12; break;
+ case VTCR_EL2_TG0_16K:
+ wi->pgshift = 14; break;
+ case VTCR_EL2_TG0_64K:
+ default:
+ wi->pgshift = 16; break;
+ }
+
+ wi->pgsize = 1UL << wi->pgshift;
+ wi->ps = (vtcr & VTCR_EL2_PS_MASK) >> VTCR_EL2_PS_SHIFT;
+ wi->sl = (vtcr & VTCR_EL2_SL0_MASK) >> VTCR_EL2_SL0_SHIFT;
+ wi->max_pa_bits = VTCR_EL2_IPA(vtcr);
+}
+
+int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
+ struct kvm_s2_trans *result)
+{
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct s2_walk_info wi;
+ int ret;
+
+ result->esr = 0;
+
+ if (!nested_virt_in_use(vcpu))
+ return 0;
+
+ wi.read_desc = read_guest_s2_desc;
+ wi.data = vcpu;
+ wi.baddr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+
+ vtcr_to_walk_info(vtcr, &wi);
+
+ wi.be = vcpu_read_sys_reg(vcpu, SCTLR_EL2) & SCTLR_EE;
+ wi.el1_aarch32 = vcpu_mode_is_32bit(vcpu);
+
+ ret = walk_nested_s2_pgd(gipa, &wi, result);
+ if (ret)
+ result->esr |= (kvm_vcpu_get_esr(vcpu) & ~ESR_ELx_FSC);
+
+ return ret;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 34/66] KVM: arm64: nv: Implement nested Stage-2 page table walk logic
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Andre Przywara, Christoffer Dall, Jintack Lim
From: Christoffer Dall <christoffer.dall@linaro.org>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software
page table walker.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: heavily reworked for future ARMv8.4-TTL support]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/esr.h | 1 +
arch/arm64/include/asm/kvm_arm.h | 2 +
arch/arm64/include/asm/kvm_nested.h | 13 ++
arch/arm64/kvm/nested.c | 276 ++++++++++++++++++++++++++++
4 files changed, 292 insertions(+)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index ab63b7b3ea14..e099eba5e616 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -127,6 +127,7 @@
#define ESR_ELx_CM (UL(1) << ESR_ELx_CM_SHIFT)
/* ISS field definitions for exceptions taken in to Hyp */
+#define ESR_ELx_FSC_ADDRSZ (0x00)
#define ESR_ELx_CV (UL(1) << 24)
#define ESR_ELx_COND_SHIFT (20)
#define ESR_ELx_COND_MASK (UL(0xF) << ESR_ELx_COND_SHIFT)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3d2abc40d3a2..347e9e9b015c 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -264,6 +264,8 @@
#define VTTBR_VMID_SHIFT (UL(48))
#define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT)
+#define SCTLR_EE (UL(1) << 25)
+
/* Hyp System Trap Register */
#define HSTR_EL2_T(x) (1 << x)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 473ecd1d60d0..b784d7891851 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -68,6 +68,19 @@ extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+struct kvm_s2_trans {
+ phys_addr_t output;
+ unsigned long block_size;
+ bool writable;
+ bool readable;
+ int level;
+ u32 esr;
+ u64 upper_attr;
+};
+
+extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
+ struct kvm_s2_trans *result);
+
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 8e85d2ef24d9..ff970221a219 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -75,6 +75,282 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return ret;
}
+struct s2_walk_info {
+ int (*read_desc)(phys_addr_t pa, u64 *desc, void *data);
+ void *data;
+ u64 baddr;
+ unsigned int max_pa_bits;
+ unsigned int pgshift;
+ unsigned int pgsize;
+ unsigned int ps;
+ unsigned int sl;
+ unsigned int t0sz;
+ bool be;
+ bool el1_aarch32;
+};
+
+static unsigned int ps_to_output_size(unsigned int ps)
+{
+ switch (ps) {
+ case 0: return 32;
+ case 1: return 36;
+ case 2: return 40;
+ case 3: return 42;
+ case 4: return 44;
+ case 5:
+ default:
+ return 48;
+ }
+}
+
+static u32 compute_fsc(int level, u32 fsc)
+{
+ return fsc | (level & 0x3);
+}
+
+static int esr_s2_fault(struct kvm_vcpu *vcpu, int level, u32 fsc)
+{
+ u32 esr;
+
+ esr = kvm_vcpu_get_esr(vcpu) & ~ESR_ELx_FSC;
+ esr |= compute_fsc(level, fsc);
+ return esr;
+}
+
+static int check_base_s2_limits(struct s2_walk_info *wi,
+ int level, int input_size, int stride)
+{
+ int start_size;
+
+ /* Check translation limits */
+ switch (wi->pgsize) {
+ case SZ_64K:
+ if (level == 0 || (level == 1 && wi->max_pa_bits <= 42))
+ return -EFAULT;
+ break;
+ case SZ_16K:
+ if (level == 0 || (level == 1 && wi->max_pa_bits <= 40))
+ return -EFAULT;
+ break;
+ case SZ_4K:
+ if (level < 0 || (level == 0 && wi->max_pa_bits <= 42))
+ return -EFAULT;
+ break;
+ }
+
+ /* Check input size limits */
+ if (input_size > wi->max_pa_bits &&
+ (!wi->el1_aarch32 || input_size > 40))
+ return -EFAULT;
+
+ /* Check number of entries in starting level table */
+ start_size = input_size - ((3 - level) * stride + wi->pgshift);
+ if (start_size < 1 || start_size > stride + 4)
+ return -EFAULT;
+
+ return 0;
+}
+
+/* Check if output is within boundaries */
+static int check_output_size(struct s2_walk_info *wi, phys_addr_t output)
+{
+ unsigned int output_size = ps_to_output_size(wi->ps);
+
+ if (output_size > wi->max_pa_bits)
+ output_size = wi->max_pa_bits;
+
+ if (output_size != 48 && (output & GENMASK_ULL(47, output_size)))
+ return -1;
+
+ return 0;
+}
+
+/*
+ * This is essentially a C-version of the pseudo code from the ARM ARM
+ * AArch64.TranslationTableWalk function. I strongly recommend looking at
+ * that pseudocode in trying to understand this.
+ *
+ * Must be called with the kvm->srcu read lock held
+ */
+static int walk_nested_s2_pgd(phys_addr_t ipa,
+ struct s2_walk_info *wi, struct kvm_s2_trans *out)
+{
+ int first_block_level, level, stride, input_size, base_lower_bound;
+ phys_addr_t base_addr;
+ unsigned int addr_top, addr_bottom;
+ u64 desc; /* page table entry */
+ int ret;
+ phys_addr_t paddr;
+
+ switch (wi->pgsize) {
+ case SZ_64K:
+ case SZ_16K:
+ level = 3 - wi->sl;
+ first_block_level = 2;
+ break;
+ case SZ_4K:
+ level = 2 - wi->sl;
+ first_block_level = 1;
+ break;
+ default:
+ /* GCC is braindead */
+ unreachable();
+ }
+
+ stride = wi->pgshift - 3;
+ input_size = 64 - wi->t0sz;
+ if (input_size > 48 || input_size < 25)
+ return -EFAULT;
+
+ ret = check_base_s2_limits(wi, level, input_size, stride);
+ if (WARN_ON(ret))
+ return ret;
+
+ base_lower_bound = 3 + input_size - ((3 - level) * stride +
+ wi->pgshift);
+ base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound);
+
+ if (check_output_size(wi, base_addr)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ return 1;
+ }
+
+ addr_top = input_size - 1;
+
+ while (1) {
+ phys_addr_t index;
+
+ addr_bottom = (3 - level) * stride + wi->pgshift;
+ index = (ipa & GENMASK_ULL(addr_top, addr_bottom))
+ >> (addr_bottom - 3);
+
+ paddr = base_addr | index;
+ ret = wi->read_desc(paddr, &desc, wi->data);
+ if (ret < 0)
+ return ret;
+
+ /*
+ * Handle reversedescriptors if endianness differs between the
+ * host and the guest hypervisor.
+ */
+ if (wi->be)
+ desc = be64_to_cpu(desc);
+ else
+ desc = le64_to_cpu(desc);
+
+ /* Check for valid descriptor at this point */
+ if (!(desc & 1) || ((desc & 3) == 1 && level == 3)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /* We're at the final level or block translation level */
+ if ((desc & 3) == 1 || level == 3)
+ break;
+
+ if (check_output_size(wi, desc)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ base_addr = desc & GENMASK_ULL(47, wi->pgshift);
+
+ level += 1;
+ addr_top = addr_bottom - 1;
+ }
+
+ if (level < first_block_level) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /*
+ * We don't use the contiguous bit in the stage-2 ptes, so skip check
+ * for misprogramming of the contiguous bit.
+ */
+
+ if (check_output_size(wi, desc)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ if (!(desc & BIT(10))) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ACCESS);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /* Calculate and return the result */
+ paddr = (desc & GENMASK_ULL(47, addr_bottom)) |
+ (ipa & GENMASK_ULL(addr_bottom - 1, 0));
+ out->output = paddr;
+ out->block_size = 1UL << ((3 - level) * stride + wi->pgshift);
+ out->readable = desc & (0b01 << 6);
+ out->writable = desc & (0b10 << 6);
+ out->level = level;
+ out->upper_attr = desc & GENMASK_ULL(63, 52);
+ return 0;
+}
+
+static int read_guest_s2_desc(phys_addr_t pa, u64 *desc, void *data)
+{
+ struct kvm_vcpu *vcpu = data;
+
+ return kvm_read_guest(vcpu->kvm, pa, desc, sizeof(*desc));
+}
+
+static void vtcr_to_walk_info(u64 vtcr, struct s2_walk_info *wi)
+{
+ wi->t0sz = vtcr & TCR_EL2_T0SZ_MASK;
+
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ wi->pgshift = 12; break;
+ case VTCR_EL2_TG0_16K:
+ wi->pgshift = 14; break;
+ case VTCR_EL2_TG0_64K:
+ default:
+ wi->pgshift = 16; break;
+ }
+
+ wi->pgsize = 1UL << wi->pgshift;
+ wi->ps = (vtcr & VTCR_EL2_PS_MASK) >> VTCR_EL2_PS_SHIFT;
+ wi->sl = (vtcr & VTCR_EL2_SL0_MASK) >> VTCR_EL2_SL0_SHIFT;
+ wi->max_pa_bits = VTCR_EL2_IPA(vtcr);
+}
+
+int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
+ struct kvm_s2_trans *result)
+{
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct s2_walk_info wi;
+ int ret;
+
+ result->esr = 0;
+
+ if (!nested_virt_in_use(vcpu))
+ return 0;
+
+ wi.read_desc = read_guest_s2_desc;
+ wi.data = vcpu;
+ wi.baddr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+
+ vtcr_to_walk_info(vtcr, &wi);
+
+ wi.be = vcpu_read_sys_reg(vcpu, SCTLR_EL2) & SCTLR_EE;
+ wi.el1_aarch32 = vcpu_mode_is_32bit(vcpu);
+
+ ret = walk_nested_s2_pgd(gipa, &wi, result);
+ if (ret)
+ result->esr |= (kvm_vcpu_get_esr(vcpu) & ~ESR_ELx_FSC);
+
+ return ret;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 34/66] KVM: arm64: nv: Implement nested Stage-2 page table walk logic
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Christoffer Dall, Jintack Lim, Alexandru Elisei,
Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@linaro.org>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software
page table walker.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: heavily reworked for future ARMv8.4-TTL support]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/esr.h | 1 +
arch/arm64/include/asm/kvm_arm.h | 2 +
arch/arm64/include/asm/kvm_nested.h | 13 ++
arch/arm64/kvm/nested.c | 276 ++++++++++++++++++++++++++++
4 files changed, 292 insertions(+)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index ab63b7b3ea14..e099eba5e616 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -127,6 +127,7 @@
#define ESR_ELx_CM (UL(1) << ESR_ELx_CM_SHIFT)
/* ISS field definitions for exceptions taken in to Hyp */
+#define ESR_ELx_FSC_ADDRSZ (0x00)
#define ESR_ELx_CV (UL(1) << 24)
#define ESR_ELx_COND_SHIFT (20)
#define ESR_ELx_COND_MASK (UL(0xF) << ESR_ELx_COND_SHIFT)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3d2abc40d3a2..347e9e9b015c 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -264,6 +264,8 @@
#define VTTBR_VMID_SHIFT (UL(48))
#define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT)
+#define SCTLR_EE (UL(1) << 25)
+
/* Hyp System Trap Register */
#define HSTR_EL2_T(x) (1 << x)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 473ecd1d60d0..b784d7891851 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -68,6 +68,19 @@ extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+struct kvm_s2_trans {
+ phys_addr_t output;
+ unsigned long block_size;
+ bool writable;
+ bool readable;
+ int level;
+ u32 esr;
+ u64 upper_attr;
+};
+
+extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
+ struct kvm_s2_trans *result);
+
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 8e85d2ef24d9..ff970221a219 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -75,6 +75,282 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return ret;
}
+struct s2_walk_info {
+ int (*read_desc)(phys_addr_t pa, u64 *desc, void *data);
+ void *data;
+ u64 baddr;
+ unsigned int max_pa_bits;
+ unsigned int pgshift;
+ unsigned int pgsize;
+ unsigned int ps;
+ unsigned int sl;
+ unsigned int t0sz;
+ bool be;
+ bool el1_aarch32;
+};
+
+static unsigned int ps_to_output_size(unsigned int ps)
+{
+ switch (ps) {
+ case 0: return 32;
+ case 1: return 36;
+ case 2: return 40;
+ case 3: return 42;
+ case 4: return 44;
+ case 5:
+ default:
+ return 48;
+ }
+}
+
+static u32 compute_fsc(int level, u32 fsc)
+{
+ return fsc | (level & 0x3);
+}
+
+static int esr_s2_fault(struct kvm_vcpu *vcpu, int level, u32 fsc)
+{
+ u32 esr;
+
+ esr = kvm_vcpu_get_esr(vcpu) & ~ESR_ELx_FSC;
+ esr |= compute_fsc(level, fsc);
+ return esr;
+}
+
+static int check_base_s2_limits(struct s2_walk_info *wi,
+ int level, int input_size, int stride)
+{
+ int start_size;
+
+ /* Check translation limits */
+ switch (wi->pgsize) {
+ case SZ_64K:
+ if (level == 0 || (level == 1 && wi->max_pa_bits <= 42))
+ return -EFAULT;
+ break;
+ case SZ_16K:
+ if (level == 0 || (level == 1 && wi->max_pa_bits <= 40))
+ return -EFAULT;
+ break;
+ case SZ_4K:
+ if (level < 0 || (level == 0 && wi->max_pa_bits <= 42))
+ return -EFAULT;
+ break;
+ }
+
+ /* Check input size limits */
+ if (input_size > wi->max_pa_bits &&
+ (!wi->el1_aarch32 || input_size > 40))
+ return -EFAULT;
+
+ /* Check number of entries in starting level table */
+ start_size = input_size - ((3 - level) * stride + wi->pgshift);
+ if (start_size < 1 || start_size > stride + 4)
+ return -EFAULT;
+
+ return 0;
+}
+
+/* Check if output is within boundaries */
+static int check_output_size(struct s2_walk_info *wi, phys_addr_t output)
+{
+ unsigned int output_size = ps_to_output_size(wi->ps);
+
+ if (output_size > wi->max_pa_bits)
+ output_size = wi->max_pa_bits;
+
+ if (output_size != 48 && (output & GENMASK_ULL(47, output_size)))
+ return -1;
+
+ return 0;
+}
+
+/*
+ * This is essentially a C-version of the pseudo code from the ARM ARM
+ * AArch64.TranslationTableWalk function. I strongly recommend looking at
+ * that pseudocode in trying to understand this.
+ *
+ * Must be called with the kvm->srcu read lock held
+ */
+static int walk_nested_s2_pgd(phys_addr_t ipa,
+ struct s2_walk_info *wi, struct kvm_s2_trans *out)
+{
+ int first_block_level, level, stride, input_size, base_lower_bound;
+ phys_addr_t base_addr;
+ unsigned int addr_top, addr_bottom;
+ u64 desc; /* page table entry */
+ int ret;
+ phys_addr_t paddr;
+
+ switch (wi->pgsize) {
+ case SZ_64K:
+ case SZ_16K:
+ level = 3 - wi->sl;
+ first_block_level = 2;
+ break;
+ case SZ_4K:
+ level = 2 - wi->sl;
+ first_block_level = 1;
+ break;
+ default:
+ /* GCC is braindead */
+ unreachable();
+ }
+
+ stride = wi->pgshift - 3;
+ input_size = 64 - wi->t0sz;
+ if (input_size > 48 || input_size < 25)
+ return -EFAULT;
+
+ ret = check_base_s2_limits(wi, level, input_size, stride);
+ if (WARN_ON(ret))
+ return ret;
+
+ base_lower_bound = 3 + input_size - ((3 - level) * stride +
+ wi->pgshift);
+ base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound);
+
+ if (check_output_size(wi, base_addr)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ return 1;
+ }
+
+ addr_top = input_size - 1;
+
+ while (1) {
+ phys_addr_t index;
+
+ addr_bottom = (3 - level) * stride + wi->pgshift;
+ index = (ipa & GENMASK_ULL(addr_top, addr_bottom))
+ >> (addr_bottom - 3);
+
+ paddr = base_addr | index;
+ ret = wi->read_desc(paddr, &desc, wi->data);
+ if (ret < 0)
+ return ret;
+
+ /*
+ * Handle reversedescriptors if endianness differs between the
+ * host and the guest hypervisor.
+ */
+ if (wi->be)
+ desc = be64_to_cpu(desc);
+ else
+ desc = le64_to_cpu(desc);
+
+ /* Check for valid descriptor at this point */
+ if (!(desc & 1) || ((desc & 3) == 1 && level == 3)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /* We're at the final level or block translation level */
+ if ((desc & 3) == 1 || level == 3)
+ break;
+
+ if (check_output_size(wi, desc)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ base_addr = desc & GENMASK_ULL(47, wi->pgshift);
+
+ level += 1;
+ addr_top = addr_bottom - 1;
+ }
+
+ if (level < first_block_level) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /*
+ * We don't use the contiguous bit in the stage-2 ptes, so skip check
+ * for misprogramming of the contiguous bit.
+ */
+
+ if (check_output_size(wi, desc)) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ if (!(desc & BIT(10))) {
+ out->esr = compute_fsc(level, ESR_ELx_FSC_ACCESS);
+ out->upper_attr = desc;
+ return 1;
+ }
+
+ /* Calculate and return the result */
+ paddr = (desc & GENMASK_ULL(47, addr_bottom)) |
+ (ipa & GENMASK_ULL(addr_bottom - 1, 0));
+ out->output = paddr;
+ out->block_size = 1UL << ((3 - level) * stride + wi->pgshift);
+ out->readable = desc & (0b01 << 6);
+ out->writable = desc & (0b10 << 6);
+ out->level = level;
+ out->upper_attr = desc & GENMASK_ULL(63, 52);
+ return 0;
+}
+
+static int read_guest_s2_desc(phys_addr_t pa, u64 *desc, void *data)
+{
+ struct kvm_vcpu *vcpu = data;
+
+ return kvm_read_guest(vcpu->kvm, pa, desc, sizeof(*desc));
+}
+
+static void vtcr_to_walk_info(u64 vtcr, struct s2_walk_info *wi)
+{
+ wi->t0sz = vtcr & TCR_EL2_T0SZ_MASK;
+
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ wi->pgshift = 12; break;
+ case VTCR_EL2_TG0_16K:
+ wi->pgshift = 14; break;
+ case VTCR_EL2_TG0_64K:
+ default:
+ wi->pgshift = 16; break;
+ }
+
+ wi->pgsize = 1UL << wi->pgshift;
+ wi->ps = (vtcr & VTCR_EL2_PS_MASK) >> VTCR_EL2_PS_SHIFT;
+ wi->sl = (vtcr & VTCR_EL2_SL0_MASK) >> VTCR_EL2_SL0_SHIFT;
+ wi->max_pa_bits = VTCR_EL2_IPA(vtcr);
+}
+
+int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
+ struct kvm_s2_trans *result)
+{
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct s2_walk_info wi;
+ int ret;
+
+ result->esr = 0;
+
+ if (!nested_virt_in_use(vcpu))
+ return 0;
+
+ wi.read_desc = read_guest_s2_desc;
+ wi.data = vcpu;
+ wi.baddr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+
+ vtcr_to_walk_info(vtcr, &wi);
+
+ wi.be = vcpu_read_sys_reg(vcpu, SCTLR_EL2) & SCTLR_EE;
+ wi.el1_aarch32 = vcpu_mode_is_32bit(vcpu);
+
+ ret = walk_nested_s2_pgd(gipa, &wi, result);
+ if (ret)
+ result->esr |= (kvm_vcpu_get_esr(vcpu) & ~ESR_ELx_FSC);
+
+ return ret;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 35/66] KVM: arm64: nv: Handle shadow stage 2 page faults
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Christoffer Dall, Jintack Lim
If we are faulting on a shadow stage 2 translation, we first walk the
guest hypervisor's stage 2 page table to see if it has a mapping. If
not, we inject a stage 2 page fault to the virtual EL2. Otherwise, we
create a mapping in the shadow stage 2 page table.
Note that we have to deal with two IPAs when we got a shadow stage 2
page fault. One is the address we faulted on, and is in the L2 guest
phys space. The other is from the guest stage-2 page table walk, and is
in the L1 guest phys space. To differentiate them, we rename variables
so that fault_ipa is used for the former and ipa is used for the latter.
Co-developed-by: Christoffer Dall <christoffer.dall@linaro.org>
Co-developed-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: rewrote this multiple times...]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 6 ++
arch/arm64/include/asm/kvm_nested.h | 18 ++++++
arch/arm64/kvm/mmu.c | 93 ++++++++++++++++++++++------
arch/arm64/kvm/nested.c | 39 ++++++++++++
4 files changed, 138 insertions(+), 18 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index a000e6e05091..b447817d9dee 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -586,4 +586,10 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
vcpu->arch.flags |= KVM_ARM64_INCREMENT_PC;
}
+static inline bool kvm_is_shadow_s2_fault(struct kvm_vcpu *vcpu)
+{
+ return (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu &&
+ vcpu->arch.hw_mmu->nested_stage2_enabled);
+}
+
#endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index b784d7891851..4f93a5dab183 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -78,9 +78,27 @@ struct kvm_s2_trans {
u64 upper_attr;
};
+static inline phys_addr_t kvm_s2_trans_output(struct kvm_s2_trans *trans)
+{
+ return trans->output;
+}
+
+static inline unsigned long kvm_s2_trans_size(struct kvm_s2_trans *trans)
+{
+ return trans->block_size;
+}
+
+static inline u32 kvm_s2_trans_esr(struct kvm_s2_trans *trans)
+{
+ return trans->esr;
+}
+
extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
struct kvm_s2_trans *result);
+extern int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu,
+ struct kvm_s2_trans *trans);
+extern int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2);
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 2f0302211af3..aa8e2ed7acfe 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -699,7 +699,7 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
static unsigned long
transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
unsigned long hva, kvm_pfn_t *pfnp,
- phys_addr_t *ipap)
+ phys_addr_t *ipap, phys_addr_t *fault_ipap)
{
kvm_pfn_t pfn = *pfnp;
@@ -728,6 +728,7 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
* to PG_head and switch the pfn from a tail page to the head
* page accordingly.
*/
+ *fault_ipap &= PMD_MASK;
*ipap &= PMD_MASK;
kvm_release_pfn_clean(pfn);
pfn &= ~(PTRS_PER_PMD - 1);
@@ -742,14 +743,16 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
}
static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
- struct kvm_memory_slot *memslot, unsigned long hva,
- unsigned long fault_status)
+ struct kvm_s2_trans *nested,
+ struct kvm_memory_slot *memslot,
+ unsigned long hva, unsigned long fault_status)
{
int ret = 0;
- bool write_fault, writable, force_pte = false;
+ bool write_fault, writable;
bool exec_fault;
bool device = false;
unsigned long mmu_seq;
+ phys_addr_t ipa = fault_ipa;
struct kvm *kvm = vcpu->kvm;
struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
struct vm_area_struct *vma;
@@ -760,6 +763,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
unsigned long vma_pagesize;
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt;
+ unsigned long max_map_size = PUD_SIZE;
write_fault = kvm_is_write_fault(vcpu);
exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
@@ -786,7 +790,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (logging_active ||
(vma->vm_flags & VM_PFNMAP)) {
- force_pte = true;
+ max_map_size = vma_pagesize = PAGE_SIZE;
vma_shift = PAGE_SHIFT;
}
@@ -806,7 +810,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
fallthrough;
case CONT_PTE_SHIFT:
vma_shift = PAGE_SHIFT;
- force_pte = true;
+ max_map_size = PAGE_SIZE;
fallthrough;
case PAGE_SHIFT:
break;
@@ -815,10 +819,25 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
}
vma_pagesize = 1UL << vma_shift;
+
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ ipa = kvm_s2_trans_output(nested);
+
+ /*
+ * If we're about to create a shadow stage 2 entry, then we
+ * can only create a block mapping if the guest stage 2 page
+ * table uses at least as big a mapping.
+ */
+ max_map_size = min(kvm_s2_trans_size(nested), max_map_size);
+ }
+
+ vma_pagesize = min(vma_pagesize, max_map_size);
+
if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
fault_ipa &= ~(vma_pagesize - 1);
- gfn = fault_ipa >> PAGE_SHIFT;
+ gfn = ipa >> PAGE_SHIFT;
+
mmap_read_unlock(current->mm);
/*
@@ -856,7 +875,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (kvm_is_device_pfn(pfn)) {
device = true;
- force_pte = true;
+ max_map_size = PAGE_SIZE;
} else if (logging_active && !write_fault) {
/*
* Only actually map the page as writable if this was a write
@@ -877,9 +896,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
* If we are not forced to use page mapping, check if we are
* backed by a THP and thus use block mapping if possible.
*/
- if (vma_pagesize == PAGE_SIZE && !force_pte)
- vma_pagesize = transparent_hugepage_adjust(memslot, hva,
- &pfn, &fault_ipa);
+ if (vma_pagesize == PAGE_SIZE && max_map_size >= PMD_SIZE)
+ vma_pagesize = transparent_hugepage_adjust(memslot, hva, &pfn,
+ &ipa, &fault_ipa);
if (writable) {
prot |= KVM_PGTABLE_PROT_W;
kvm_set_pfn_dirty(pfn);
@@ -947,8 +966,10 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
{
unsigned long fault_status;
- phys_addr_t fault_ipa;
+ phys_addr_t fault_ipa; /* The address we faulted on */
+ phys_addr_t ipa; /* Always the IPA in the L1 guest phys space */
struct kvm_memory_slot *memslot;
+ struct kvm_s2_trans nested_trans;
unsigned long hva;
bool is_iabt, write_fault, writable;
gfn_t gfn;
@@ -956,7 +977,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
- fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
+ ipa = fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
is_iabt = kvm_vcpu_trap_is_iabt(vcpu);
/* Synchronous External Abort? */
@@ -977,6 +998,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
/* Check the stage-2 fault is trans. fault or write fault */
if (fault_status != FSC_FAULT && fault_status != FSC_PERM &&
fault_status != FSC_ACCESS) {
+ /*
+ * We must never see an address size fault on shadow stage 2
+ * page table walk, because we would have injected an addr
+ * size fault when we walked the nested s2 page and not
+ * create the shadow entry.
+ */
kvm_err("Unsupported FSC: EC=%#x xFSC=%#lx ESR_EL2=%#lx\n",
kvm_vcpu_trap_get_class(vcpu),
(unsigned long)kvm_vcpu_trap_get_fault(vcpu),
@@ -986,7 +1013,36 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
idx = srcu_read_lock(&vcpu->kvm->srcu);
- gfn = fault_ipa >> PAGE_SHIFT;
+ /*
+ * We may have faulted on a shadow stage 2 page table if we are
+ * running a nested guest. In this case, we have to resolve the L2
+ * IPA to the L1 IPA first, before knowing what kind of memory should
+ * back the L1 IPA.
+ *
+ * If the shadow stage 2 page table walk faults, then we simply inject
+ * this to the guest and carry on.
+ */
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ u32 esr;
+
+ ret = kvm_walk_nested_s2(vcpu, fault_ipa, &nested_trans);
+ esr = kvm_s2_trans_esr(&nested_trans);
+ if (esr)
+ kvm_inject_s2_fault(vcpu, esr);
+ if (ret)
+ goto out_unlock;
+
+ ret = kvm_s2_handle_perm_fault(vcpu, &nested_trans);
+ esr = kvm_s2_trans_esr(&nested_trans);
+ if (esr)
+ kvm_inject_s2_fault(vcpu, esr);
+ if (ret)
+ goto out_unlock;
+
+ ipa = kvm_s2_trans_output(&nested_trans);
+ }
+
+ gfn = ipa >> PAGE_SHIFT;
memslot = gfn_to_memslot(vcpu->kvm, gfn);
hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
write_fault = kvm_is_write_fault(vcpu);
@@ -1030,13 +1086,13 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
* faulting VA. This is always 12 bits, irrespective
* of the page size.
*/
- fault_ipa |= kvm_vcpu_get_hfar(vcpu) & ((1 << 12) - 1);
- ret = io_mem_abort(vcpu, fault_ipa);
+ ipa |= kvm_vcpu_get_hfar(vcpu) & ((1 << 12) - 1);
+ ret = io_mem_abort(vcpu, ipa);
goto out_unlock;
}
/* Userspace should not be able to register out-of-bounds IPAs */
- VM_BUG_ON(fault_ipa >= kvm_phys_size(vcpu->kvm));
+ VM_BUG_ON(ipa >= kvm_phys_size(vcpu->kvm));
if (fault_status == FSC_ACCESS) {
handle_access_fault(vcpu, fault_ipa);
@@ -1044,7 +1100,8 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
goto out_unlock;
}
- ret = user_mem_abort(vcpu, fault_ipa, memslot, hva, fault_status);
+ ret = user_mem_abort(vcpu, fault_ipa, &nested_trans,
+ memslot, hva, fault_status);
if (ret == 0)
ret = 1;
out:
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index ff970221a219..a64895ec6907 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -466,6 +466,45 @@ void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
}
}
+/*
+ * Returns non-zero if permission fault is handled by injecting it to the next
+ * level hypervisor.
+ */
+int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu, struct kvm_s2_trans *trans)
+{
+ unsigned long fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
+ bool forward_fault = false;
+
+ trans->esr = 0;
+
+ if (fault_status != FSC_PERM)
+ return 0;
+
+ if (kvm_vcpu_trap_is_iabt(vcpu)) {
+ forward_fault = (trans->upper_attr & BIT(54));
+ } else {
+ bool write_fault = kvm_is_write_fault(vcpu);
+
+ forward_fault = ((write_fault && !trans->writable) ||
+ (!write_fault && !trans->readable));
+ }
+
+ if (forward_fault) {
+ trans->esr = esr_s2_fault(vcpu, trans->level, ESR_ELx_FSC_PERM);
+ return 1;
+ }
+
+ return 0;
+}
+
+int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2)
+{
+ vcpu_write_sys_reg(vcpu, vcpu->arch.fault.far_el2, FAR_EL2);
+ vcpu_write_sys_reg(vcpu, vcpu->arch.fault.hpfar_el2, HPFAR_EL2);
+
+ return kvm_inject_nested_sync(vcpu, esr_el2);
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 35/66] KVM: arm64: nv: Handle shadow stage 2 page faults
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Andre Przywara, Christoffer Dall, Jintack Lim
If we are faulting on a shadow stage 2 translation, we first walk the
guest hypervisor's stage 2 page table to see if it has a mapping. If
not, we inject a stage 2 page fault to the virtual EL2. Otherwise, we
create a mapping in the shadow stage 2 page table.
Note that we have to deal with two IPAs when we got a shadow stage 2
page fault. One is the address we faulted on, and is in the L2 guest
phys space. The other is from the guest stage-2 page table walk, and is
in the L1 guest phys space. To differentiate them, we rename variables
so that fault_ipa is used for the former and ipa is used for the latter.
Co-developed-by: Christoffer Dall <christoffer.dall@linaro.org>
Co-developed-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: rewrote this multiple times...]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 6 ++
arch/arm64/include/asm/kvm_nested.h | 18 ++++++
arch/arm64/kvm/mmu.c | 93 ++++++++++++++++++++++------
arch/arm64/kvm/nested.c | 39 ++++++++++++
4 files changed, 138 insertions(+), 18 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index a000e6e05091..b447817d9dee 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -586,4 +586,10 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
vcpu->arch.flags |= KVM_ARM64_INCREMENT_PC;
}
+static inline bool kvm_is_shadow_s2_fault(struct kvm_vcpu *vcpu)
+{
+ return (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu &&
+ vcpu->arch.hw_mmu->nested_stage2_enabled);
+}
+
#endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index b784d7891851..4f93a5dab183 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -78,9 +78,27 @@ struct kvm_s2_trans {
u64 upper_attr;
};
+static inline phys_addr_t kvm_s2_trans_output(struct kvm_s2_trans *trans)
+{
+ return trans->output;
+}
+
+static inline unsigned long kvm_s2_trans_size(struct kvm_s2_trans *trans)
+{
+ return trans->block_size;
+}
+
+static inline u32 kvm_s2_trans_esr(struct kvm_s2_trans *trans)
+{
+ return trans->esr;
+}
+
extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
struct kvm_s2_trans *result);
+extern int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu,
+ struct kvm_s2_trans *trans);
+extern int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2);
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 2f0302211af3..aa8e2ed7acfe 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -699,7 +699,7 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
static unsigned long
transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
unsigned long hva, kvm_pfn_t *pfnp,
- phys_addr_t *ipap)
+ phys_addr_t *ipap, phys_addr_t *fault_ipap)
{
kvm_pfn_t pfn = *pfnp;
@@ -728,6 +728,7 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
* to PG_head and switch the pfn from a tail page to the head
* page accordingly.
*/
+ *fault_ipap &= PMD_MASK;
*ipap &= PMD_MASK;
kvm_release_pfn_clean(pfn);
pfn &= ~(PTRS_PER_PMD - 1);
@@ -742,14 +743,16 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
}
static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
- struct kvm_memory_slot *memslot, unsigned long hva,
- unsigned long fault_status)
+ struct kvm_s2_trans *nested,
+ struct kvm_memory_slot *memslot,
+ unsigned long hva, unsigned long fault_status)
{
int ret = 0;
- bool write_fault, writable, force_pte = false;
+ bool write_fault, writable;
bool exec_fault;
bool device = false;
unsigned long mmu_seq;
+ phys_addr_t ipa = fault_ipa;
struct kvm *kvm = vcpu->kvm;
struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
struct vm_area_struct *vma;
@@ -760,6 +763,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
unsigned long vma_pagesize;
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt;
+ unsigned long max_map_size = PUD_SIZE;
write_fault = kvm_is_write_fault(vcpu);
exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
@@ -786,7 +790,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (logging_active ||
(vma->vm_flags & VM_PFNMAP)) {
- force_pte = true;
+ max_map_size = vma_pagesize = PAGE_SIZE;
vma_shift = PAGE_SHIFT;
}
@@ -806,7 +810,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
fallthrough;
case CONT_PTE_SHIFT:
vma_shift = PAGE_SHIFT;
- force_pte = true;
+ max_map_size = PAGE_SIZE;
fallthrough;
case PAGE_SHIFT:
break;
@@ -815,10 +819,25 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
}
vma_pagesize = 1UL << vma_shift;
+
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ ipa = kvm_s2_trans_output(nested);
+
+ /*
+ * If we're about to create a shadow stage 2 entry, then we
+ * can only create a block mapping if the guest stage 2 page
+ * table uses at least as big a mapping.
+ */
+ max_map_size = min(kvm_s2_trans_size(nested), max_map_size);
+ }
+
+ vma_pagesize = min(vma_pagesize, max_map_size);
+
if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
fault_ipa &= ~(vma_pagesize - 1);
- gfn = fault_ipa >> PAGE_SHIFT;
+ gfn = ipa >> PAGE_SHIFT;
+
mmap_read_unlock(current->mm);
/*
@@ -856,7 +875,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (kvm_is_device_pfn(pfn)) {
device = true;
- force_pte = true;
+ max_map_size = PAGE_SIZE;
} else if (logging_active && !write_fault) {
/*
* Only actually map the page as writable if this was a write
@@ -877,9 +896,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
* If we are not forced to use page mapping, check if we are
* backed by a THP and thus use block mapping if possible.
*/
- if (vma_pagesize == PAGE_SIZE && !force_pte)
- vma_pagesize = transparent_hugepage_adjust(memslot, hva,
- &pfn, &fault_ipa);
+ if (vma_pagesize == PAGE_SIZE && max_map_size >= PMD_SIZE)
+ vma_pagesize = transparent_hugepage_adjust(memslot, hva, &pfn,
+ &ipa, &fault_ipa);
if (writable) {
prot |= KVM_PGTABLE_PROT_W;
kvm_set_pfn_dirty(pfn);
@@ -947,8 +966,10 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
{
unsigned long fault_status;
- phys_addr_t fault_ipa;
+ phys_addr_t fault_ipa; /* The address we faulted on */
+ phys_addr_t ipa; /* Always the IPA in the L1 guest phys space */
struct kvm_memory_slot *memslot;
+ struct kvm_s2_trans nested_trans;
unsigned long hva;
bool is_iabt, write_fault, writable;
gfn_t gfn;
@@ -956,7 +977,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
- fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
+ ipa = fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
is_iabt = kvm_vcpu_trap_is_iabt(vcpu);
/* Synchronous External Abort? */
@@ -977,6 +998,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
/* Check the stage-2 fault is trans. fault or write fault */
if (fault_status != FSC_FAULT && fault_status != FSC_PERM &&
fault_status != FSC_ACCESS) {
+ /*
+ * We must never see an address size fault on shadow stage 2
+ * page table walk, because we would have injected an addr
+ * size fault when we walked the nested s2 page and not
+ * create the shadow entry.
+ */
kvm_err("Unsupported FSC: EC=%#x xFSC=%#lx ESR_EL2=%#lx\n",
kvm_vcpu_trap_get_class(vcpu),
(unsigned long)kvm_vcpu_trap_get_fault(vcpu),
@@ -986,7 +1013,36 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
idx = srcu_read_lock(&vcpu->kvm->srcu);
- gfn = fault_ipa >> PAGE_SHIFT;
+ /*
+ * We may have faulted on a shadow stage 2 page table if we are
+ * running a nested guest. In this case, we have to resolve the L2
+ * IPA to the L1 IPA first, before knowing what kind of memory should
+ * back the L1 IPA.
+ *
+ * If the shadow stage 2 page table walk faults, then we simply inject
+ * this to the guest and carry on.
+ */
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ u32 esr;
+
+ ret = kvm_walk_nested_s2(vcpu, fault_ipa, &nested_trans);
+ esr = kvm_s2_trans_esr(&nested_trans);
+ if (esr)
+ kvm_inject_s2_fault(vcpu, esr);
+ if (ret)
+ goto out_unlock;
+
+ ret = kvm_s2_handle_perm_fault(vcpu, &nested_trans);
+ esr = kvm_s2_trans_esr(&nested_trans);
+ if (esr)
+ kvm_inject_s2_fault(vcpu, esr);
+ if (ret)
+ goto out_unlock;
+
+ ipa = kvm_s2_trans_output(&nested_trans);
+ }
+
+ gfn = ipa >> PAGE_SHIFT;
memslot = gfn_to_memslot(vcpu->kvm, gfn);
hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
write_fault = kvm_is_write_fault(vcpu);
@@ -1030,13 +1086,13 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
* faulting VA. This is always 12 bits, irrespective
* of the page size.
*/
- fault_ipa |= kvm_vcpu_get_hfar(vcpu) & ((1 << 12) - 1);
- ret = io_mem_abort(vcpu, fault_ipa);
+ ipa |= kvm_vcpu_get_hfar(vcpu) & ((1 << 12) - 1);
+ ret = io_mem_abort(vcpu, ipa);
goto out_unlock;
}
/* Userspace should not be able to register out-of-bounds IPAs */
- VM_BUG_ON(fault_ipa >= kvm_phys_size(vcpu->kvm));
+ VM_BUG_ON(ipa >= kvm_phys_size(vcpu->kvm));
if (fault_status == FSC_ACCESS) {
handle_access_fault(vcpu, fault_ipa);
@@ -1044,7 +1100,8 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
goto out_unlock;
}
- ret = user_mem_abort(vcpu, fault_ipa, memslot, hva, fault_status);
+ ret = user_mem_abort(vcpu, fault_ipa, &nested_trans,
+ memslot, hva, fault_status);
if (ret == 0)
ret = 1;
out:
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index ff970221a219..a64895ec6907 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -466,6 +466,45 @@ void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
}
}
+/*
+ * Returns non-zero if permission fault is handled by injecting it to the next
+ * level hypervisor.
+ */
+int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu, struct kvm_s2_trans *trans)
+{
+ unsigned long fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
+ bool forward_fault = false;
+
+ trans->esr = 0;
+
+ if (fault_status != FSC_PERM)
+ return 0;
+
+ if (kvm_vcpu_trap_is_iabt(vcpu)) {
+ forward_fault = (trans->upper_attr & BIT(54));
+ } else {
+ bool write_fault = kvm_is_write_fault(vcpu);
+
+ forward_fault = ((write_fault && !trans->writable) ||
+ (!write_fault && !trans->readable));
+ }
+
+ if (forward_fault) {
+ trans->esr = esr_s2_fault(vcpu, trans->level, ESR_ELx_FSC_PERM);
+ return 1;
+ }
+
+ return 0;
+}
+
+int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2)
+{
+ vcpu_write_sys_reg(vcpu, vcpu->arch.fault.far_el2, FAR_EL2);
+ vcpu_write_sys_reg(vcpu, vcpu->arch.fault.hpfar_el2, HPFAR_EL2);
+
+ return kvm_inject_nested_sync(vcpu, esr_el2);
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 35/66] KVM: arm64: nv: Handle shadow stage 2 page faults
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Christoffer Dall, Jintack Lim, Alexandru Elisei,
Jintack Lim, Julien Thierry
If we are faulting on a shadow stage 2 translation, we first walk the
guest hypervisor's stage 2 page table to see if it has a mapping. If
not, we inject a stage 2 page fault to the virtual EL2. Otherwise, we
create a mapping in the shadow stage 2 page table.
Note that we have to deal with two IPAs when we got a shadow stage 2
page fault. One is the address we faulted on, and is in the L2 guest
phys space. The other is from the guest stage-2 page table walk, and is
in the L1 guest phys space. To differentiate them, we rename variables
so that fault_ipa is used for the former and ipa is used for the latter.
Co-developed-by: Christoffer Dall <christoffer.dall@linaro.org>
Co-developed-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: rewrote this multiple times...]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 6 ++
arch/arm64/include/asm/kvm_nested.h | 18 ++++++
arch/arm64/kvm/mmu.c | 93 ++++++++++++++++++++++------
arch/arm64/kvm/nested.c | 39 ++++++++++++
4 files changed, 138 insertions(+), 18 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index a000e6e05091..b447817d9dee 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -586,4 +586,10 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
vcpu->arch.flags |= KVM_ARM64_INCREMENT_PC;
}
+static inline bool kvm_is_shadow_s2_fault(struct kvm_vcpu *vcpu)
+{
+ return (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu &&
+ vcpu->arch.hw_mmu->nested_stage2_enabled);
+}
+
#endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index b784d7891851..4f93a5dab183 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -78,9 +78,27 @@ struct kvm_s2_trans {
u64 upper_attr;
};
+static inline phys_addr_t kvm_s2_trans_output(struct kvm_s2_trans *trans)
+{
+ return trans->output;
+}
+
+static inline unsigned long kvm_s2_trans_size(struct kvm_s2_trans *trans)
+{
+ return trans->block_size;
+}
+
+static inline u32 kvm_s2_trans_esr(struct kvm_s2_trans *trans)
+{
+ return trans->esr;
+}
+
extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
struct kvm_s2_trans *result);
+extern int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu,
+ struct kvm_s2_trans *trans);
+extern int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2);
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 2f0302211af3..aa8e2ed7acfe 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -699,7 +699,7 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
static unsigned long
transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
unsigned long hva, kvm_pfn_t *pfnp,
- phys_addr_t *ipap)
+ phys_addr_t *ipap, phys_addr_t *fault_ipap)
{
kvm_pfn_t pfn = *pfnp;
@@ -728,6 +728,7 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
* to PG_head and switch the pfn from a tail page to the head
* page accordingly.
*/
+ *fault_ipap &= PMD_MASK;
*ipap &= PMD_MASK;
kvm_release_pfn_clean(pfn);
pfn &= ~(PTRS_PER_PMD - 1);
@@ -742,14 +743,16 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,
}
static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
- struct kvm_memory_slot *memslot, unsigned long hva,
- unsigned long fault_status)
+ struct kvm_s2_trans *nested,
+ struct kvm_memory_slot *memslot,
+ unsigned long hva, unsigned long fault_status)
{
int ret = 0;
- bool write_fault, writable, force_pte = false;
+ bool write_fault, writable;
bool exec_fault;
bool device = false;
unsigned long mmu_seq;
+ phys_addr_t ipa = fault_ipa;
struct kvm *kvm = vcpu->kvm;
struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
struct vm_area_struct *vma;
@@ -760,6 +763,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
unsigned long vma_pagesize;
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt;
+ unsigned long max_map_size = PUD_SIZE;
write_fault = kvm_is_write_fault(vcpu);
exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
@@ -786,7 +790,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (logging_active ||
(vma->vm_flags & VM_PFNMAP)) {
- force_pte = true;
+ max_map_size = vma_pagesize = PAGE_SIZE;
vma_shift = PAGE_SHIFT;
}
@@ -806,7 +810,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
fallthrough;
case CONT_PTE_SHIFT:
vma_shift = PAGE_SHIFT;
- force_pte = true;
+ max_map_size = PAGE_SIZE;
fallthrough;
case PAGE_SHIFT:
break;
@@ -815,10 +819,25 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
}
vma_pagesize = 1UL << vma_shift;
+
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ ipa = kvm_s2_trans_output(nested);
+
+ /*
+ * If we're about to create a shadow stage 2 entry, then we
+ * can only create a block mapping if the guest stage 2 page
+ * table uses at least as big a mapping.
+ */
+ max_map_size = min(kvm_s2_trans_size(nested), max_map_size);
+ }
+
+ vma_pagesize = min(vma_pagesize, max_map_size);
+
if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
fault_ipa &= ~(vma_pagesize - 1);
- gfn = fault_ipa >> PAGE_SHIFT;
+ gfn = ipa >> PAGE_SHIFT;
+
mmap_read_unlock(current->mm);
/*
@@ -856,7 +875,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (kvm_is_device_pfn(pfn)) {
device = true;
- force_pte = true;
+ max_map_size = PAGE_SIZE;
} else if (logging_active && !write_fault) {
/*
* Only actually map the page as writable if this was a write
@@ -877,9 +896,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
* If we are not forced to use page mapping, check if we are
* backed by a THP and thus use block mapping if possible.
*/
- if (vma_pagesize == PAGE_SIZE && !force_pte)
- vma_pagesize = transparent_hugepage_adjust(memslot, hva,
- &pfn, &fault_ipa);
+ if (vma_pagesize == PAGE_SIZE && max_map_size >= PMD_SIZE)
+ vma_pagesize = transparent_hugepage_adjust(memslot, hva, &pfn,
+ &ipa, &fault_ipa);
if (writable) {
prot |= KVM_PGTABLE_PROT_W;
kvm_set_pfn_dirty(pfn);
@@ -947,8 +966,10 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
{
unsigned long fault_status;
- phys_addr_t fault_ipa;
+ phys_addr_t fault_ipa; /* The address we faulted on */
+ phys_addr_t ipa; /* Always the IPA in the L1 guest phys space */
struct kvm_memory_slot *memslot;
+ struct kvm_s2_trans nested_trans;
unsigned long hva;
bool is_iabt, write_fault, writable;
gfn_t gfn;
@@ -956,7 +977,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
- fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
+ ipa = fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
is_iabt = kvm_vcpu_trap_is_iabt(vcpu);
/* Synchronous External Abort? */
@@ -977,6 +998,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
/* Check the stage-2 fault is trans. fault or write fault */
if (fault_status != FSC_FAULT && fault_status != FSC_PERM &&
fault_status != FSC_ACCESS) {
+ /*
+ * We must never see an address size fault on shadow stage 2
+ * page table walk, because we would have injected an addr
+ * size fault when we walked the nested s2 page and not
+ * create the shadow entry.
+ */
kvm_err("Unsupported FSC: EC=%#x xFSC=%#lx ESR_EL2=%#lx\n",
kvm_vcpu_trap_get_class(vcpu),
(unsigned long)kvm_vcpu_trap_get_fault(vcpu),
@@ -986,7 +1013,36 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
idx = srcu_read_lock(&vcpu->kvm->srcu);
- gfn = fault_ipa >> PAGE_SHIFT;
+ /*
+ * We may have faulted on a shadow stage 2 page table if we are
+ * running a nested guest. In this case, we have to resolve the L2
+ * IPA to the L1 IPA first, before knowing what kind of memory should
+ * back the L1 IPA.
+ *
+ * If the shadow stage 2 page table walk faults, then we simply inject
+ * this to the guest and carry on.
+ */
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ u32 esr;
+
+ ret = kvm_walk_nested_s2(vcpu, fault_ipa, &nested_trans);
+ esr = kvm_s2_trans_esr(&nested_trans);
+ if (esr)
+ kvm_inject_s2_fault(vcpu, esr);
+ if (ret)
+ goto out_unlock;
+
+ ret = kvm_s2_handle_perm_fault(vcpu, &nested_trans);
+ esr = kvm_s2_trans_esr(&nested_trans);
+ if (esr)
+ kvm_inject_s2_fault(vcpu, esr);
+ if (ret)
+ goto out_unlock;
+
+ ipa = kvm_s2_trans_output(&nested_trans);
+ }
+
+ gfn = ipa >> PAGE_SHIFT;
memslot = gfn_to_memslot(vcpu->kvm, gfn);
hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
write_fault = kvm_is_write_fault(vcpu);
@@ -1030,13 +1086,13 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
* faulting VA. This is always 12 bits, irrespective
* of the page size.
*/
- fault_ipa |= kvm_vcpu_get_hfar(vcpu) & ((1 << 12) - 1);
- ret = io_mem_abort(vcpu, fault_ipa);
+ ipa |= kvm_vcpu_get_hfar(vcpu) & ((1 << 12) - 1);
+ ret = io_mem_abort(vcpu, ipa);
goto out_unlock;
}
/* Userspace should not be able to register out-of-bounds IPAs */
- VM_BUG_ON(fault_ipa >= kvm_phys_size(vcpu->kvm));
+ VM_BUG_ON(ipa >= kvm_phys_size(vcpu->kvm));
if (fault_status == FSC_ACCESS) {
handle_access_fault(vcpu, fault_ipa);
@@ -1044,7 +1100,8 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
goto out_unlock;
}
- ret = user_mem_abort(vcpu, fault_ipa, memslot, hva, fault_status);
+ ret = user_mem_abort(vcpu, fault_ipa, &nested_trans,
+ memslot, hva, fault_status);
if (ret == 0)
ret = 1;
out:
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index ff970221a219..a64895ec6907 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -466,6 +466,45 @@ void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
}
}
+/*
+ * Returns non-zero if permission fault is handled by injecting it to the next
+ * level hypervisor.
+ */
+int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu, struct kvm_s2_trans *trans)
+{
+ unsigned long fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
+ bool forward_fault = false;
+
+ trans->esr = 0;
+
+ if (fault_status != FSC_PERM)
+ return 0;
+
+ if (kvm_vcpu_trap_is_iabt(vcpu)) {
+ forward_fault = (trans->upper_attr & BIT(54));
+ } else {
+ bool write_fault = kvm_is_write_fault(vcpu);
+
+ forward_fault = ((write_fault && !trans->writable) ||
+ (!write_fault && !trans->readable));
+ }
+
+ if (forward_fault) {
+ trans->esr = esr_s2_fault(vcpu, trans->level, ESR_ELx_FSC_PERM);
+ return 1;
+ }
+
+ return 0;
+}
+
+int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2)
+{
+ vcpu_write_sys_reg(vcpu, vcpu->arch.fault.far_el2, FAR_EL2);
+ vcpu_write_sys_reg(vcpu, vcpu->arch.fault.hpfar_el2, HPFAR_EL2);
+
+ return kvm_inject_nested_sync(vcpu, esr_el2);
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 36/66] KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
When mapping a page in a shadow stage-2, special care must be
taken not to be more permissive than the guest is (writable or
readable page when the guest hasn't set that permission).
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 15 +++++++++++++++
arch/arm64/kvm/mmu.c | 14 +++++++++++++-
arch/arm64/kvm/nested.c | 2 +-
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 4f93a5dab183..3f3d8e10bd99 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -93,6 +93,21 @@ static inline u32 kvm_s2_trans_esr(struct kvm_s2_trans *trans)
return trans->esr;
}
+static inline bool kvm_s2_trans_readable(struct kvm_s2_trans *trans)
+{
+ return trans->readable;
+}
+
+static inline bool kvm_s2_trans_writable(struct kvm_s2_trans *trans)
+{
+ return trans->writable;
+}
+
+static inline bool kvm_s2_trans_executable(struct kvm_s2_trans *trans)
+{
+ return !(trans->upper_attr & BIT(54));
+}
+
extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
struct kvm_s2_trans *result);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index aa8e2ed7acfe..6f973efb2cc3 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -887,6 +887,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (exec_fault && device)
return -ENOEXEC;
+ /*
+ * Potentially reduce shadow S2 permissions to match the guest's own
+ * S2. For exec faults, we'd only reach this point if the guest
+ * actually allowed it (see kvm_s2_handle_perm_fault).
+ */
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ writable &= kvm_s2_trans_writable(nested);
+ if (!kvm_s2_trans_readable(nested))
+ prot &= ~KVM_PGTABLE_PROT_R;
+ }
+
spin_lock(&kvm->mmu_lock);
pgt = vcpu->arch.hw_mmu->pgt;
if (mmu_notifier_retry(kvm, mmu_seq))
@@ -915,7 +926,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (device)
prot |= KVM_PGTABLE_PROT_DEVICE;
- else if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC))
+ else if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC) &&
+ kvm_s2_trans_executable(nested))
prot |= KVM_PGTABLE_PROT_X;
if (fault_status == FSC_PERM && !(logging_active && writable)) {
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index a64895ec6907..551aee363cc3 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -481,7 +481,7 @@ int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu, struct kvm_s2_trans *trans)
return 0;
if (kvm_vcpu_trap_is_iabt(vcpu)) {
- forward_fault = (trans->upper_attr & BIT(54));
+ forward_fault = !kvm_s2_trans_executable(trans);
} else {
bool write_fault = kvm_is_write_fault(vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 36/66] KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
When mapping a page in a shadow stage-2, special care must be
taken not to be more permissive than the guest is (writable or
readable page when the guest hasn't set that permission).
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 15 +++++++++++++++
arch/arm64/kvm/mmu.c | 14 +++++++++++++-
arch/arm64/kvm/nested.c | 2 +-
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 4f93a5dab183..3f3d8e10bd99 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -93,6 +93,21 @@ static inline u32 kvm_s2_trans_esr(struct kvm_s2_trans *trans)
return trans->esr;
}
+static inline bool kvm_s2_trans_readable(struct kvm_s2_trans *trans)
+{
+ return trans->readable;
+}
+
+static inline bool kvm_s2_trans_writable(struct kvm_s2_trans *trans)
+{
+ return trans->writable;
+}
+
+static inline bool kvm_s2_trans_executable(struct kvm_s2_trans *trans)
+{
+ return !(trans->upper_attr & BIT(54));
+}
+
extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
struct kvm_s2_trans *result);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index aa8e2ed7acfe..6f973efb2cc3 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -887,6 +887,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (exec_fault && device)
return -ENOEXEC;
+ /*
+ * Potentially reduce shadow S2 permissions to match the guest's own
+ * S2. For exec faults, we'd only reach this point if the guest
+ * actually allowed it (see kvm_s2_handle_perm_fault).
+ */
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ writable &= kvm_s2_trans_writable(nested);
+ if (!kvm_s2_trans_readable(nested))
+ prot &= ~KVM_PGTABLE_PROT_R;
+ }
+
spin_lock(&kvm->mmu_lock);
pgt = vcpu->arch.hw_mmu->pgt;
if (mmu_notifier_retry(kvm, mmu_seq))
@@ -915,7 +926,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (device)
prot |= KVM_PGTABLE_PROT_DEVICE;
- else if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC))
+ else if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC) &&
+ kvm_s2_trans_executable(nested))
prot |= KVM_PGTABLE_PROT_X;
if (fault_status == FSC_PERM && !(logging_active && writable)) {
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index a64895ec6907..551aee363cc3 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -481,7 +481,7 @@ int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu, struct kvm_s2_trans *trans)
return 0;
if (kvm_vcpu_trap_is_iabt(vcpu)) {
- forward_fault = (trans->upper_attr & BIT(54));
+ forward_fault = !kvm_s2_trans_executable(trans);
} else {
bool write_fault = kvm_is_write_fault(vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 36/66] KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
When mapping a page in a shadow stage-2, special care must be
taken not to be more permissive than the guest is (writable or
readable page when the guest hasn't set that permission).
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 15 +++++++++++++++
arch/arm64/kvm/mmu.c | 14 +++++++++++++-
arch/arm64/kvm/nested.c | 2 +-
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 4f93a5dab183..3f3d8e10bd99 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -93,6 +93,21 @@ static inline u32 kvm_s2_trans_esr(struct kvm_s2_trans *trans)
return trans->esr;
}
+static inline bool kvm_s2_trans_readable(struct kvm_s2_trans *trans)
+{
+ return trans->readable;
+}
+
+static inline bool kvm_s2_trans_writable(struct kvm_s2_trans *trans)
+{
+ return trans->writable;
+}
+
+static inline bool kvm_s2_trans_executable(struct kvm_s2_trans *trans)
+{
+ return !(trans->upper_attr & BIT(54));
+}
+
extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
struct kvm_s2_trans *result);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index aa8e2ed7acfe..6f973efb2cc3 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -887,6 +887,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (exec_fault && device)
return -ENOEXEC;
+ /*
+ * Potentially reduce shadow S2 permissions to match the guest's own
+ * S2. For exec faults, we'd only reach this point if the guest
+ * actually allowed it (see kvm_s2_handle_perm_fault).
+ */
+ if (kvm_is_shadow_s2_fault(vcpu)) {
+ writable &= kvm_s2_trans_writable(nested);
+ if (!kvm_s2_trans_readable(nested))
+ prot &= ~KVM_PGTABLE_PROT_R;
+ }
+
spin_lock(&kvm->mmu_lock);
pgt = vcpu->arch.hw_mmu->pgt;
if (mmu_notifier_retry(kvm, mmu_seq))
@@ -915,7 +926,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (device)
prot |= KVM_PGTABLE_PROT_DEVICE;
- else if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC))
+ else if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC) &&
+ kvm_s2_trans_executable(nested))
prot |= KVM_PGTABLE_PROT_X;
if (fault_status == FSC_PERM && !(logging_active && writable)) {
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index a64895ec6907..551aee363cc3 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -481,7 +481,7 @@ int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu, struct kvm_s2_trans *trans)
return 0;
if (kvm_vcpu_trap_is_iabt(vcpu)) {
- forward_fault = (trans->upper_attr & BIT(54));
+ forward_fault = !kvm_s2_trans_executable(trans);
} else {
bool write_fault = kvm_is_write_fault(vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 37/66] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Christoffer Dall, Jintack Lim
From: Christoffer Dall <christoffer.dall@linaro.org>
Unmap/flush shadow stage 2 page tables for the nested VMs as well as the
stage 2 page table for the guest hypervisor.
Note: A bunch of the code in mmu.c relating to MMU notifiers is
currently dealt with in an extremely abrupt way, for example by clearing
out an entire shadow stage-2 table. This will be handled in a more
efficient way using the reverse mapping feature in a later version of
the patch series.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_mmu.h | 3 +++
arch/arm64/include/asm/kvm_nested.h | 3 +++
arch/arm64/kvm/mmu.c | 34 ++++++++++++++++++++++---
arch/arm64/kvm/nested.c | 39 +++++++++++++++++++++++++++++
4 files changed, 75 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index ec39015bb2a6..e2c58ad46bd1 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -183,6 +183,8 @@ int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size,
void __iomem **haddr);
int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
void **haddr);
+void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu,
+ phys_addr_t addr, phys_addr_t end);
void free_hyp_pgds(void);
void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
@@ -191,6 +193,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
phys_addr_t pa, unsigned long size, bool writable);
+void kvm_stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end);
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 3f3d8e10bd99..2987806850f0 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -114,6 +114,9 @@ extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
extern int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu,
struct kvm_s2_trans *trans);
extern int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2);
+extern void kvm_nested_s2_wp(struct kvm *kvm);
+extern void kvm_nested_s2_clear(struct kvm *kvm);
+extern void kvm_nested_s2_flush(struct kvm *kvm);
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6f973efb2cc3..36cb9fa22153 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -141,13 +141,20 @@ void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
__unmap_stage2_range(mmu, start, size, true);
}
+void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu,
+ phys_addr_t addr, phys_addr_t end)
+{
+ stage2_apply_range_resched(mmu->kvm, addr, end, kvm_pgtable_stage2_flush);
+}
+
static void stage2_flush_memslot(struct kvm *kvm,
struct kvm_memory_slot *memslot)
{
phys_addr_t addr = memslot->base_gfn << PAGE_SHIFT;
phys_addr_t end = addr + PAGE_SIZE * memslot->npages;
+ struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
- stage2_apply_range_resched(kvm, addr, end, kvm_pgtable_stage2_flush);
+ kvm_stage2_flush_range(mmu, addr, end);
}
/**
@@ -170,6 +177,8 @@ static void stage2_flush_vm(struct kvm *kvm)
kvm_for_each_memslot(memslot, slots)
stage2_flush_memslot(kvm, memslot);
+ kvm_nested_s2_flush(kvm);
+
spin_unlock(&kvm->mmu_lock);
srcu_read_unlock(&kvm->srcu, idx);
}
@@ -465,6 +474,8 @@ void stage2_unmap_vm(struct kvm *kvm)
kvm_for_each_memslot(memslot, slots)
stage2_unmap_memslot(kvm, memslot);
+ kvm_nested_s2_clear(kvm);
+
spin_unlock(&kvm->mmu_lock);
mmap_read_unlock(current->mm);
srcu_read_unlock(&kvm->srcu, idx);
@@ -539,7 +550,7 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
* @addr: Start address of range
* @end: End address of range
*/
-static void stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
+void kvm_stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
{
struct kvm *kvm = mmu->kvm;
stage2_apply_range_resched(kvm, addr, end, kvm_pgtable_stage2_wrprotect);
@@ -571,7 +582,8 @@ void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot)
end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
spin_lock(&kvm->mmu_lock);
- stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_nested_s2_wp(kvm);
spin_unlock(&kvm->mmu_lock);
kvm_flush_remote_tlbs(kvm);
}
@@ -595,7 +607,7 @@ static void kvm_mmu_write_protect_pt_masked(struct kvm *kvm,
phys_addr_t start = (base_gfn + __ffs(mask)) << PAGE_SHIFT;
phys_addr_t end = (base_gfn + __fls(mask) + 1) << PAGE_SHIFT;
- stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_stage2_wp_range(&kvm->arch.mmu, start, end);
}
/*
@@ -610,6 +622,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
gfn_t gfn_offset, unsigned long mask)
{
kvm_mmu_write_protect_pt_masked(kvm, slot, gfn_offset, mask);
+ kvm_nested_s2_wp(kvm);
}
static void clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size)
@@ -1164,6 +1177,7 @@ static int kvm_unmap_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *dat
bool may_block = flags & MMU_NOTIFIER_RANGE_BLOCKABLE;
__unmap_stage2_range(&kvm->arch.mmu, gpa, size, may_block);
+ kvm_nested_s2_clear(kvm);
return 0;
}
@@ -1192,6 +1206,7 @@ static int kvm_set_spte_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data
*/
kvm_pgtable_stage2_map(kvm->arch.mmu.pgt, gpa, PAGE_SIZE,
__pfn_to_phys(*pfn), KVM_PGTABLE_PROT_R, NULL);
+ kvm_nested_s2_clear(kvm);
return 0;
}
@@ -1223,12 +1238,22 @@ static int kvm_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data)
kpte = kvm_pgtable_stage2_mkold(kvm->arch.mmu.pgt, gpa);
pte = __pte(kpte);
return pte_valid(pte) && pte_young(pte);
+
+ /*
+ * TODO: Handle nested_mmu structures here using the reverse mapping in
+ * a later version of patch series.
+ */
}
static int kvm_test_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data)
{
WARN_ON(size != PAGE_SIZE && size != PMD_SIZE && size != PUD_SIZE);
return kvm_pgtable_stage2_is_young(kvm->arch.mmu.pgt, gpa);
+
+ /*
+ * TODO: Handle nested_mmu structures here using the reverse mapping in
+ * a later version of patch series.
+ */
}
int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end)
@@ -1457,6 +1482,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
spin_lock(&kvm->mmu_lock);
kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
+ kvm_nested_s2_clear(kvm);
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 551aee363cc3..e78c6c093afc 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -505,6 +505,45 @@ int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2)
return kvm_inject_nested_sync(vcpu, esr_el2);
}
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_wp(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_stage2_wp_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_clear(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_flush(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_stage2_flush_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 37/66] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Andre Przywara, Christoffer Dall, Jintack Lim
From: Christoffer Dall <christoffer.dall@linaro.org>
Unmap/flush shadow stage 2 page tables for the nested VMs as well as the
stage 2 page table for the guest hypervisor.
Note: A bunch of the code in mmu.c relating to MMU notifiers is
currently dealt with in an extremely abrupt way, for example by clearing
out an entire shadow stage-2 table. This will be handled in a more
efficient way using the reverse mapping feature in a later version of
the patch series.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_mmu.h | 3 +++
arch/arm64/include/asm/kvm_nested.h | 3 +++
arch/arm64/kvm/mmu.c | 34 ++++++++++++++++++++++---
arch/arm64/kvm/nested.c | 39 +++++++++++++++++++++++++++++
4 files changed, 75 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index ec39015bb2a6..e2c58ad46bd1 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -183,6 +183,8 @@ int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size,
void __iomem **haddr);
int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
void **haddr);
+void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu,
+ phys_addr_t addr, phys_addr_t end);
void free_hyp_pgds(void);
void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
@@ -191,6 +193,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
phys_addr_t pa, unsigned long size, bool writable);
+void kvm_stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end);
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 3f3d8e10bd99..2987806850f0 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -114,6 +114,9 @@ extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
extern int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu,
struct kvm_s2_trans *trans);
extern int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2);
+extern void kvm_nested_s2_wp(struct kvm *kvm);
+extern void kvm_nested_s2_clear(struct kvm *kvm);
+extern void kvm_nested_s2_flush(struct kvm *kvm);
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6f973efb2cc3..36cb9fa22153 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -141,13 +141,20 @@ void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
__unmap_stage2_range(mmu, start, size, true);
}
+void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu,
+ phys_addr_t addr, phys_addr_t end)
+{
+ stage2_apply_range_resched(mmu->kvm, addr, end, kvm_pgtable_stage2_flush);
+}
+
static void stage2_flush_memslot(struct kvm *kvm,
struct kvm_memory_slot *memslot)
{
phys_addr_t addr = memslot->base_gfn << PAGE_SHIFT;
phys_addr_t end = addr + PAGE_SIZE * memslot->npages;
+ struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
- stage2_apply_range_resched(kvm, addr, end, kvm_pgtable_stage2_flush);
+ kvm_stage2_flush_range(mmu, addr, end);
}
/**
@@ -170,6 +177,8 @@ static void stage2_flush_vm(struct kvm *kvm)
kvm_for_each_memslot(memslot, slots)
stage2_flush_memslot(kvm, memslot);
+ kvm_nested_s2_flush(kvm);
+
spin_unlock(&kvm->mmu_lock);
srcu_read_unlock(&kvm->srcu, idx);
}
@@ -465,6 +474,8 @@ void stage2_unmap_vm(struct kvm *kvm)
kvm_for_each_memslot(memslot, slots)
stage2_unmap_memslot(kvm, memslot);
+ kvm_nested_s2_clear(kvm);
+
spin_unlock(&kvm->mmu_lock);
mmap_read_unlock(current->mm);
srcu_read_unlock(&kvm->srcu, idx);
@@ -539,7 +550,7 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
* @addr: Start address of range
* @end: End address of range
*/
-static void stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
+void kvm_stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
{
struct kvm *kvm = mmu->kvm;
stage2_apply_range_resched(kvm, addr, end, kvm_pgtable_stage2_wrprotect);
@@ -571,7 +582,8 @@ void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot)
end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
spin_lock(&kvm->mmu_lock);
- stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_nested_s2_wp(kvm);
spin_unlock(&kvm->mmu_lock);
kvm_flush_remote_tlbs(kvm);
}
@@ -595,7 +607,7 @@ static void kvm_mmu_write_protect_pt_masked(struct kvm *kvm,
phys_addr_t start = (base_gfn + __ffs(mask)) << PAGE_SHIFT;
phys_addr_t end = (base_gfn + __fls(mask) + 1) << PAGE_SHIFT;
- stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_stage2_wp_range(&kvm->arch.mmu, start, end);
}
/*
@@ -610,6 +622,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
gfn_t gfn_offset, unsigned long mask)
{
kvm_mmu_write_protect_pt_masked(kvm, slot, gfn_offset, mask);
+ kvm_nested_s2_wp(kvm);
}
static void clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size)
@@ -1164,6 +1177,7 @@ static int kvm_unmap_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *dat
bool may_block = flags & MMU_NOTIFIER_RANGE_BLOCKABLE;
__unmap_stage2_range(&kvm->arch.mmu, gpa, size, may_block);
+ kvm_nested_s2_clear(kvm);
return 0;
}
@@ -1192,6 +1206,7 @@ static int kvm_set_spte_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data
*/
kvm_pgtable_stage2_map(kvm->arch.mmu.pgt, gpa, PAGE_SIZE,
__pfn_to_phys(*pfn), KVM_PGTABLE_PROT_R, NULL);
+ kvm_nested_s2_clear(kvm);
return 0;
}
@@ -1223,12 +1238,22 @@ static int kvm_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data)
kpte = kvm_pgtable_stage2_mkold(kvm->arch.mmu.pgt, gpa);
pte = __pte(kpte);
return pte_valid(pte) && pte_young(pte);
+
+ /*
+ * TODO: Handle nested_mmu structures here using the reverse mapping in
+ * a later version of patch series.
+ */
}
static int kvm_test_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data)
{
WARN_ON(size != PAGE_SIZE && size != PMD_SIZE && size != PUD_SIZE);
return kvm_pgtable_stage2_is_young(kvm->arch.mmu.pgt, gpa);
+
+ /*
+ * TODO: Handle nested_mmu structures here using the reverse mapping in
+ * a later version of patch series.
+ */
}
int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end)
@@ -1457,6 +1482,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
spin_lock(&kvm->mmu_lock);
kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
+ kvm_nested_s2_clear(kvm);
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 551aee363cc3..e78c6c093afc 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -505,6 +505,45 @@ int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2)
return kvm_inject_nested_sync(vcpu, esr_el2);
}
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_wp(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_stage2_wp_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_clear(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_flush(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_stage2_flush_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 37/66] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Christoffer Dall, Jintack Lim, Alexandru Elisei,
Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@linaro.org>
Unmap/flush shadow stage 2 page tables for the nested VMs as well as the
stage 2 page table for the guest hypervisor.
Note: A bunch of the code in mmu.c relating to MMU notifiers is
currently dealt with in an extremely abrupt way, for example by clearing
out an entire shadow stage-2 table. This will be handled in a more
efficient way using the reverse mapping feature in a later version of
the patch series.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_mmu.h | 3 +++
arch/arm64/include/asm/kvm_nested.h | 3 +++
arch/arm64/kvm/mmu.c | 34 ++++++++++++++++++++++---
arch/arm64/kvm/nested.c | 39 +++++++++++++++++++++++++++++
4 files changed, 75 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index ec39015bb2a6..e2c58ad46bd1 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -183,6 +183,8 @@ int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size,
void __iomem **haddr);
int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
void **haddr);
+void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu,
+ phys_addr_t addr, phys_addr_t end);
void free_hyp_pgds(void);
void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
@@ -191,6 +193,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
phys_addr_t pa, unsigned long size, bool writable);
+void kvm_stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end);
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 3f3d8e10bd99..2987806850f0 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -114,6 +114,9 @@ extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
extern int kvm_s2_handle_perm_fault(struct kvm_vcpu *vcpu,
struct kvm_s2_trans *trans);
extern int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2);
+extern void kvm_nested_s2_wp(struct kvm *kvm);
+extern void kvm_nested_s2_clear(struct kvm *kvm);
+extern void kvm_nested_s2_flush(struct kvm *kvm);
int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6f973efb2cc3..36cb9fa22153 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -141,13 +141,20 @@ void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
__unmap_stage2_range(mmu, start, size, true);
}
+void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu,
+ phys_addr_t addr, phys_addr_t end)
+{
+ stage2_apply_range_resched(mmu->kvm, addr, end, kvm_pgtable_stage2_flush);
+}
+
static void stage2_flush_memslot(struct kvm *kvm,
struct kvm_memory_slot *memslot)
{
phys_addr_t addr = memslot->base_gfn << PAGE_SHIFT;
phys_addr_t end = addr + PAGE_SIZE * memslot->npages;
+ struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
- stage2_apply_range_resched(kvm, addr, end, kvm_pgtable_stage2_flush);
+ kvm_stage2_flush_range(mmu, addr, end);
}
/**
@@ -170,6 +177,8 @@ static void stage2_flush_vm(struct kvm *kvm)
kvm_for_each_memslot(memslot, slots)
stage2_flush_memslot(kvm, memslot);
+ kvm_nested_s2_flush(kvm);
+
spin_unlock(&kvm->mmu_lock);
srcu_read_unlock(&kvm->srcu, idx);
}
@@ -465,6 +474,8 @@ void stage2_unmap_vm(struct kvm *kvm)
kvm_for_each_memslot(memslot, slots)
stage2_unmap_memslot(kvm, memslot);
+ kvm_nested_s2_clear(kvm);
+
spin_unlock(&kvm->mmu_lock);
mmap_read_unlock(current->mm);
srcu_read_unlock(&kvm->srcu, idx);
@@ -539,7 +550,7 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
* @addr: Start address of range
* @end: End address of range
*/
-static void stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
+void kvm_stage2_wp_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
{
struct kvm *kvm = mmu->kvm;
stage2_apply_range_resched(kvm, addr, end, kvm_pgtable_stage2_wrprotect);
@@ -571,7 +582,8 @@ void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot)
end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
spin_lock(&kvm->mmu_lock);
- stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_nested_s2_wp(kvm);
spin_unlock(&kvm->mmu_lock);
kvm_flush_remote_tlbs(kvm);
}
@@ -595,7 +607,7 @@ static void kvm_mmu_write_protect_pt_masked(struct kvm *kvm,
phys_addr_t start = (base_gfn + __ffs(mask)) << PAGE_SHIFT;
phys_addr_t end = (base_gfn + __fls(mask) + 1) << PAGE_SHIFT;
- stage2_wp_range(&kvm->arch.mmu, start, end);
+ kvm_stage2_wp_range(&kvm->arch.mmu, start, end);
}
/*
@@ -610,6 +622,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
gfn_t gfn_offset, unsigned long mask)
{
kvm_mmu_write_protect_pt_masked(kvm, slot, gfn_offset, mask);
+ kvm_nested_s2_wp(kvm);
}
static void clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size)
@@ -1164,6 +1177,7 @@ static int kvm_unmap_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *dat
bool may_block = flags & MMU_NOTIFIER_RANGE_BLOCKABLE;
__unmap_stage2_range(&kvm->arch.mmu, gpa, size, may_block);
+ kvm_nested_s2_clear(kvm);
return 0;
}
@@ -1192,6 +1206,7 @@ static int kvm_set_spte_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data
*/
kvm_pgtable_stage2_map(kvm->arch.mmu.pgt, gpa, PAGE_SIZE,
__pfn_to_phys(*pfn), KVM_PGTABLE_PROT_R, NULL);
+ kvm_nested_s2_clear(kvm);
return 0;
}
@@ -1223,12 +1238,22 @@ static int kvm_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data)
kpte = kvm_pgtable_stage2_mkold(kvm->arch.mmu.pgt, gpa);
pte = __pte(kpte);
return pte_valid(pte) && pte_young(pte);
+
+ /*
+ * TODO: Handle nested_mmu structures here using the reverse mapping in
+ * a later version of patch series.
+ */
}
static int kvm_test_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data)
{
WARN_ON(size != PAGE_SIZE && size != PMD_SIZE && size != PUD_SIZE);
return kvm_pgtable_stage2_is_young(kvm->arch.mmu.pgt, gpa);
+
+ /*
+ * TODO: Handle nested_mmu structures here using the reverse mapping in
+ * a later version of patch series.
+ */
}
int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end)
@@ -1457,6 +1482,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
spin_lock(&kvm->mmu_lock);
kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
+ kvm_nested_s2_clear(kvm);
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 551aee363cc3..e78c6c093afc 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -505,6 +505,45 @@ int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2)
return kvm_inject_nested_sync(vcpu, esr_el2);
}
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_wp(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_stage2_wp_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_clear(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
+/* expects kvm->mmu_lock to be held */
+void kvm_nested_s2_flush(struct kvm *kvm)
+{
+ int i;
+
+ for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
+ struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
+
+ if (kvm_s2_mmu_valid(mmu))
+ kvm_stage2_flush_range(mmu, 0, kvm_phys_size(kvm));
+ }
+}
+
/*
* Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
* the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 38/66] KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
This introduces a function prototype to determine if we need to forward
system instruction traps to the virtual EL2. The implementation of
forward_trap functions for each system instruction will be added in
later patches.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 8 ++++++++
arch/arm64/kvm/sys_regs.h | 6 ++++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 090ce14306ba..8f1378dfec6c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2490,6 +2490,14 @@ static void perform_access(struct kvm_vcpu *vcpu,
*/
BUG_ON(!r->access);
+ /*
+ * Forward this trap to the virtual EL2 if the guest hypervisor has
+ * configured to trap the current instruction.
+ */
+ if (nested_virt_in_use(vcpu) && r->forward_trap
+ && unlikely(r->forward_trap(vcpu)))
+ return;
+
/* Skip instruction if instructed so */
if (likely(r->access(vcpu, params, r)))
kvm_incr_pc(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index 9d0621417c2a..62ca4119069a 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -58,6 +58,12 @@ struct sys_reg_desc {
int (*set_user)(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr);
+ /*
+ * Forward the trap to the virtual EL2 if the guest hypervisor has
+ * configured to trap the current instruction.
+ */
+ bool (*forward_trap)(struct kvm_vcpu *vcpu);
+
/* Return mask of REG_* runtime visibility overrides */
unsigned int (*visibility)(const struct kvm_vcpu *vcpu,
const struct sys_reg_desc *rd);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 38/66] KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
This introduces a function prototype to determine if we need to forward
system instruction traps to the virtual EL2. The implementation of
forward_trap functions for each system instruction will be added in
later patches.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 8 ++++++++
arch/arm64/kvm/sys_regs.h | 6 ++++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 090ce14306ba..8f1378dfec6c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2490,6 +2490,14 @@ static void perform_access(struct kvm_vcpu *vcpu,
*/
BUG_ON(!r->access);
+ /*
+ * Forward this trap to the virtual EL2 if the guest hypervisor has
+ * configured to trap the current instruction.
+ */
+ if (nested_virt_in_use(vcpu) && r->forward_trap
+ && unlikely(r->forward_trap(vcpu)))
+ return;
+
/* Skip instruction if instructed so */
if (likely(r->access(vcpu, params, r)))
kvm_incr_pc(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index 9d0621417c2a..62ca4119069a 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -58,6 +58,12 @@ struct sys_reg_desc {
int (*set_user)(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr);
+ /*
+ * Forward the trap to the virtual EL2 if the guest hypervisor has
+ * configured to trap the current instruction.
+ */
+ bool (*forward_trap)(struct kvm_vcpu *vcpu);
+
/* Return mask of REG_* runtime visibility overrides */
unsigned int (*visibility)(const struct kvm_vcpu *vcpu,
const struct sys_reg_desc *rd);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 38/66] KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
This introduces a function prototype to determine if we need to forward
system instruction traps to the virtual EL2. The implementation of
forward_trap functions for each system instruction will be added in
later patches.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 8 ++++++++
arch/arm64/kvm/sys_regs.h | 6 ++++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 090ce14306ba..8f1378dfec6c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2490,6 +2490,14 @@ static void perform_access(struct kvm_vcpu *vcpu,
*/
BUG_ON(!r->access);
+ /*
+ * Forward this trap to the virtual EL2 if the guest hypervisor has
+ * configured to trap the current instruction.
+ */
+ if (nested_virt_in_use(vcpu) && r->forward_trap
+ && unlikely(r->forward_trap(vcpu)))
+ return;
+
/* Skip instruction if instructed so */
if (likely(r->access(vcpu, params, r)))
kvm_incr_pc(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index 9d0621417c2a..62ca4119069a 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -58,6 +58,12 @@ struct sys_reg_desc {
int (*set_user)(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr);
+ /*
+ * Forward the trap to the virtual EL2 if the guest hypervisor has
+ * configured to trap the current instruction.
+ */
+ bool (*forward_trap)(struct kvm_vcpu *vcpu);
+
/* Return mask of REG_* runtime visibility overrides */
unsigned int (*visibility)(const struct kvm_vcpu *vcpu,
const struct sys_reg_desc *rd);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 39/66] KVM: arm64: nv: Set a handler for the system instruction traps
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
When HCR.NV bit is set, execution of the EL2 translation regime address
aranslation instructions and TLB maintenance instructions are trapped to
EL2. In addition, execution of the EL1 translation regime address
aranslation instructions and TLB maintenance instructions that are only
accessible from EL2 and above are trapped to EL2. In these cases,
ESR_EL2.EC will be set to 0x18.
Rework the system instruction emulation framework to handle potentially
all system instruction traps other than MSR/MRS instructions. Those
system instructions would be AT and TLBI instructions controlled by
HCR_EL2.NV, AT, and TTLB bits.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: squashed two patches together, redispatched various bits around]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +--
arch/arm64/kvm/handle_exit.c | 2 +-
arch/arm64/kvm/sys_regs.c | 48 +++++++++++++++++++++++++------
3 files changed, 42 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d99e51e7cbee..bf60c50eec17 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -369,7 +369,7 @@ struct kvm_vcpu_arch {
/*
* Guest registers we preserve during guest debugging.
*
- * These shadow registers are updated by the kvm_handle_sys_reg
+ * These shadow registers are updated by the kvm_handle_sys
* trap handler if the guest accesses or updates them while we
* are using guest debug.
*/
@@ -681,7 +681,7 @@ int kvm_handle_cp14_32(struct kvm_vcpu *vcpu);
int kvm_handle_cp14_64(struct kvm_vcpu *vcpu);
int kvm_handle_cp15_32(struct kvm_vcpu *vcpu);
int kvm_handle_cp15_64(struct kvm_vcpu *vcpu);
-int kvm_handle_sys_reg(struct kvm_vcpu *vcpu);
+int kvm_handle_sys(struct kvm_vcpu *vcpu);
void kvm_reset_sys_regs(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 0ad20457d0f1..6e1be999bb00 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -248,7 +248,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_SMC32] = handle_smc,
[ESR_ELx_EC_HVC64] = handle_hvc,
[ESR_ELx_EC_SMC64] = handle_smc,
- [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
+ [ESR_ELx_EC_SYS64] = kvm_handle_sys,
[ESR_ELx_EC_SVE] = handle_sve,
[ESR_ELx_EC_ERET] = kvm_handle_eret,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 8f1378dfec6c..35aa1a51aa7d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1685,10 +1685,6 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
* more demanding guest...
*/
static const struct sys_reg_desc sys_reg_descs[] = {
- { SYS_DESC(SYS_DC_ISW), access_dcsw },
- { SYS_DESC(SYS_DC_CSW), access_dcsw },
- { SYS_DESC(SYS_DC_CISW), access_dcsw },
-
DBG_BCR_BVR_WCR_WVR_EL1(0),
DBG_BCR_BVR_WCR_WVR_EL1(1),
{ SYS_DESC(SYS_MDCCINT_EL1), trap_debug_regs, reset_val, MDCCINT_EL1, 0 },
@@ -2119,6 +2115,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
+#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
+ { SYS_DESC((insn)), (access_fn), NULL, 0, 0, NULL, NULL, (forward_fn) }
+static struct sys_reg_desc sys_insn_descs[] = {
+ { SYS_DESC(SYS_DC_ISW), access_dcsw },
+ { SYS_DESC(SYS_DC_CSW), access_dcsw },
+ { SYS_DESC(SYS_DC_CISW), access_dcsw },
+};
+
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -2686,6 +2690,24 @@ static int emulate_sys_reg(struct kvm_vcpu *vcpu,
return 1;
}
+static int emulate_sys_instr(struct kvm_vcpu *vcpu, struct sys_reg_params *p)
+{
+ const struct sys_reg_desc *r;
+
+ /* Search from the system instruction table. */
+ r = find_reg(p, sys_insn_descs, ARRAY_SIZE(sys_insn_descs));
+
+ if (likely(r)) {
+ perform_access(vcpu, p, r);
+ } else {
+ kvm_err("Unsupported guest sys instruction at: %lx\n",
+ *vcpu_pc(vcpu));
+ print_sys_reg_instr(p);
+ kvm_inject_undefined(vcpu);
+ }
+ return 1;
+}
+
/**
* kvm_reset_sys_regs - sets system registers to reset value
* @vcpu: The VCPU pointer
@@ -2703,10 +2725,11 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu)
}
/**
- * kvm_handle_sys_reg -- handles a mrs/msr trap on a guest sys_reg access
+ * kvm_handle_sys-- handles a system instruction or mrs/msr instruction trap
+ on a guest execution
* @vcpu: The VCPU pointer
*/
-int kvm_handle_sys_reg(struct kvm_vcpu *vcpu)
+int kvm_handle_sys(struct kvm_vcpu *vcpu)
{
struct sys_reg_params params;
unsigned long esr = kvm_vcpu_get_esr(vcpu);
@@ -2723,10 +2746,16 @@ int kvm_handle_sys_reg(struct kvm_vcpu *vcpu)
params.regval = vcpu_get_reg(vcpu, Rt);
params.is_write = !(esr & 1);
- ret = emulate_sys_reg(vcpu, ¶ms);
+ if (params.Op0 == 1) {
+ /* System instructions */
+ ret = emulate_sys_instr(vcpu, ¶ms);
+ } else {
+ /* MRS/MSR instructions */
+ ret = emulate_sys_reg(vcpu, ¶ms);
+ if (!params.is_write)
+ vcpu_set_reg(vcpu, Rt, params.regval);
+ }
- if (!params.is_write)
- vcpu_set_reg(vcpu, Rt, params.regval);
return ret;
}
@@ -3142,6 +3171,7 @@ void kvm_sys_reg_table_init(void)
BUG_ON(check_sysreg_table(cp15_regs, ARRAY_SIZE(cp15_regs), true));
BUG_ON(check_sysreg_table(cp15_64_regs, ARRAY_SIZE(cp15_64_regs), true));
BUG_ON(check_sysreg_table(invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs), false));
+ BUG_ON(check_sysreg_table(sys_insn_descs, ARRAY_SIZE(sys_insn_descs), false));
/* We abuse the reset function to overwrite the table itself. */
for (i = 0; i < ARRAY_SIZE(invariant_sys_regs); i++)
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 39/66] KVM: arm64: nv: Set a handler for the system instruction traps
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
When HCR.NV bit is set, execution of the EL2 translation regime address
aranslation instructions and TLB maintenance instructions are trapped to
EL2. In addition, execution of the EL1 translation regime address
aranslation instructions and TLB maintenance instructions that are only
accessible from EL2 and above are trapped to EL2. In these cases,
ESR_EL2.EC will be set to 0x18.
Rework the system instruction emulation framework to handle potentially
all system instruction traps other than MSR/MRS instructions. Those
system instructions would be AT and TLBI instructions controlled by
HCR_EL2.NV, AT, and TTLB bits.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: squashed two patches together, redispatched various bits around]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +--
arch/arm64/kvm/handle_exit.c | 2 +-
arch/arm64/kvm/sys_regs.c | 48 +++++++++++++++++++++++++------
3 files changed, 42 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d99e51e7cbee..bf60c50eec17 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -369,7 +369,7 @@ struct kvm_vcpu_arch {
/*
* Guest registers we preserve during guest debugging.
*
- * These shadow registers are updated by the kvm_handle_sys_reg
+ * These shadow registers are updated by the kvm_handle_sys
* trap handler if the guest accesses or updates them while we
* are using guest debug.
*/
@@ -681,7 +681,7 @@ int kvm_handle_cp14_32(struct kvm_vcpu *vcpu);
int kvm_handle_cp14_64(struct kvm_vcpu *vcpu);
int kvm_handle_cp15_32(struct kvm_vcpu *vcpu);
int kvm_handle_cp15_64(struct kvm_vcpu *vcpu);
-int kvm_handle_sys_reg(struct kvm_vcpu *vcpu);
+int kvm_handle_sys(struct kvm_vcpu *vcpu);
void kvm_reset_sys_regs(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 0ad20457d0f1..6e1be999bb00 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -248,7 +248,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_SMC32] = handle_smc,
[ESR_ELx_EC_HVC64] = handle_hvc,
[ESR_ELx_EC_SMC64] = handle_smc,
- [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
+ [ESR_ELx_EC_SYS64] = kvm_handle_sys,
[ESR_ELx_EC_SVE] = handle_sve,
[ESR_ELx_EC_ERET] = kvm_handle_eret,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 8f1378dfec6c..35aa1a51aa7d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1685,10 +1685,6 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
* more demanding guest...
*/
static const struct sys_reg_desc sys_reg_descs[] = {
- { SYS_DESC(SYS_DC_ISW), access_dcsw },
- { SYS_DESC(SYS_DC_CSW), access_dcsw },
- { SYS_DESC(SYS_DC_CISW), access_dcsw },
-
DBG_BCR_BVR_WCR_WVR_EL1(0),
DBG_BCR_BVR_WCR_WVR_EL1(1),
{ SYS_DESC(SYS_MDCCINT_EL1), trap_debug_regs, reset_val, MDCCINT_EL1, 0 },
@@ -2119,6 +2115,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
+#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
+ { SYS_DESC((insn)), (access_fn), NULL, 0, 0, NULL, NULL, (forward_fn) }
+static struct sys_reg_desc sys_insn_descs[] = {
+ { SYS_DESC(SYS_DC_ISW), access_dcsw },
+ { SYS_DESC(SYS_DC_CSW), access_dcsw },
+ { SYS_DESC(SYS_DC_CISW), access_dcsw },
+};
+
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -2686,6 +2690,24 @@ static int emulate_sys_reg(struct kvm_vcpu *vcpu,
return 1;
}
+static int emulate_sys_instr(struct kvm_vcpu *vcpu, struct sys_reg_params *p)
+{
+ const struct sys_reg_desc *r;
+
+ /* Search from the system instruction table. */
+ r = find_reg(p, sys_insn_descs, ARRAY_SIZE(sys_insn_descs));
+
+ if (likely(r)) {
+ perform_access(vcpu, p, r);
+ } else {
+ kvm_err("Unsupported guest sys instruction at: %lx\n",
+ *vcpu_pc(vcpu));
+ print_sys_reg_instr(p);
+ kvm_inject_undefined(vcpu);
+ }
+ return 1;
+}
+
/**
* kvm_reset_sys_regs - sets system registers to reset value
* @vcpu: The VCPU pointer
@@ -2703,10 +2725,11 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu)
}
/**
- * kvm_handle_sys_reg -- handles a mrs/msr trap on a guest sys_reg access
+ * kvm_handle_sys-- handles a system instruction or mrs/msr instruction trap
+ on a guest execution
* @vcpu: The VCPU pointer
*/
-int kvm_handle_sys_reg(struct kvm_vcpu *vcpu)
+int kvm_handle_sys(struct kvm_vcpu *vcpu)
{
struct sys_reg_params params;
unsigned long esr = kvm_vcpu_get_esr(vcpu);
@@ -2723,10 +2746,16 @@ int kvm_handle_sys_reg(struct kvm_vcpu *vcpu)
params.regval = vcpu_get_reg(vcpu, Rt);
params.is_write = !(esr & 1);
- ret = emulate_sys_reg(vcpu, ¶ms);
+ if (params.Op0 == 1) {
+ /* System instructions */
+ ret = emulate_sys_instr(vcpu, ¶ms);
+ } else {
+ /* MRS/MSR instructions */
+ ret = emulate_sys_reg(vcpu, ¶ms);
+ if (!params.is_write)
+ vcpu_set_reg(vcpu, Rt, params.regval);
+ }
- if (!params.is_write)
- vcpu_set_reg(vcpu, Rt, params.regval);
return ret;
}
@@ -3142,6 +3171,7 @@ void kvm_sys_reg_table_init(void)
BUG_ON(check_sysreg_table(cp15_regs, ARRAY_SIZE(cp15_regs), true));
BUG_ON(check_sysreg_table(cp15_64_regs, ARRAY_SIZE(cp15_64_regs), true));
BUG_ON(check_sysreg_table(invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs), false));
+ BUG_ON(check_sysreg_table(sys_insn_descs, ARRAY_SIZE(sys_insn_descs), false));
/* We abuse the reset function to overwrite the table itself. */
for (i = 0; i < ARRAY_SIZE(invariant_sys_regs); i++)
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 39/66] KVM: arm64: nv: Set a handler for the system instruction traps
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
When HCR.NV bit is set, execution of the EL2 translation regime address
aranslation instructions and TLB maintenance instructions are trapped to
EL2. In addition, execution of the EL1 translation regime address
aranslation instructions and TLB maintenance instructions that are only
accessible from EL2 and above are trapped to EL2. In these cases,
ESR_EL2.EC will be set to 0x18.
Rework the system instruction emulation framework to handle potentially
all system instruction traps other than MSR/MRS instructions. Those
system instructions would be AT and TLBI instructions controlled by
HCR_EL2.NV, AT, and TTLB bits.
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
[maz: squashed two patches together, redispatched various bits around]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +--
arch/arm64/kvm/handle_exit.c | 2 +-
arch/arm64/kvm/sys_regs.c | 48 +++++++++++++++++++++++++------
3 files changed, 42 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d99e51e7cbee..bf60c50eec17 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -369,7 +369,7 @@ struct kvm_vcpu_arch {
/*
* Guest registers we preserve during guest debugging.
*
- * These shadow registers are updated by the kvm_handle_sys_reg
+ * These shadow registers are updated by the kvm_handle_sys
* trap handler if the guest accesses or updates them while we
* are using guest debug.
*/
@@ -681,7 +681,7 @@ int kvm_handle_cp14_32(struct kvm_vcpu *vcpu);
int kvm_handle_cp14_64(struct kvm_vcpu *vcpu);
int kvm_handle_cp15_32(struct kvm_vcpu *vcpu);
int kvm_handle_cp15_64(struct kvm_vcpu *vcpu);
-int kvm_handle_sys_reg(struct kvm_vcpu *vcpu);
+int kvm_handle_sys(struct kvm_vcpu *vcpu);
void kvm_reset_sys_regs(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 0ad20457d0f1..6e1be999bb00 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -248,7 +248,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_SMC32] = handle_smc,
[ESR_ELx_EC_HVC64] = handle_hvc,
[ESR_ELx_EC_SMC64] = handle_smc,
- [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
+ [ESR_ELx_EC_SYS64] = kvm_handle_sys,
[ESR_ELx_EC_SVE] = handle_sve,
[ESR_ELx_EC_ERET] = kvm_handle_eret,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 8f1378dfec6c..35aa1a51aa7d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1685,10 +1685,6 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
* more demanding guest...
*/
static const struct sys_reg_desc sys_reg_descs[] = {
- { SYS_DESC(SYS_DC_ISW), access_dcsw },
- { SYS_DESC(SYS_DC_CSW), access_dcsw },
- { SYS_DESC(SYS_DC_CISW), access_dcsw },
-
DBG_BCR_BVR_WCR_WVR_EL1(0),
DBG_BCR_BVR_WCR_WVR_EL1(1),
{ SYS_DESC(SYS_MDCCINT_EL1), trap_debug_regs, reset_val, MDCCINT_EL1, 0 },
@@ -2119,6 +2115,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
+#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
+ { SYS_DESC((insn)), (access_fn), NULL, 0, 0, NULL, NULL, (forward_fn) }
+static struct sys_reg_desc sys_insn_descs[] = {
+ { SYS_DESC(SYS_DC_ISW), access_dcsw },
+ { SYS_DESC(SYS_DC_CSW), access_dcsw },
+ { SYS_DESC(SYS_DC_CISW), access_dcsw },
+};
+
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -2686,6 +2690,24 @@ static int emulate_sys_reg(struct kvm_vcpu *vcpu,
return 1;
}
+static int emulate_sys_instr(struct kvm_vcpu *vcpu, struct sys_reg_params *p)
+{
+ const struct sys_reg_desc *r;
+
+ /* Search from the system instruction table. */
+ r = find_reg(p, sys_insn_descs, ARRAY_SIZE(sys_insn_descs));
+
+ if (likely(r)) {
+ perform_access(vcpu, p, r);
+ } else {
+ kvm_err("Unsupported guest sys instruction at: %lx\n",
+ *vcpu_pc(vcpu));
+ print_sys_reg_instr(p);
+ kvm_inject_undefined(vcpu);
+ }
+ return 1;
+}
+
/**
* kvm_reset_sys_regs - sets system registers to reset value
* @vcpu: The VCPU pointer
@@ -2703,10 +2725,11 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu)
}
/**
- * kvm_handle_sys_reg -- handles a mrs/msr trap on a guest sys_reg access
+ * kvm_handle_sys-- handles a system instruction or mrs/msr instruction trap
+ on a guest execution
* @vcpu: The VCPU pointer
*/
-int kvm_handle_sys_reg(struct kvm_vcpu *vcpu)
+int kvm_handle_sys(struct kvm_vcpu *vcpu)
{
struct sys_reg_params params;
unsigned long esr = kvm_vcpu_get_esr(vcpu);
@@ -2723,10 +2746,16 @@ int kvm_handle_sys_reg(struct kvm_vcpu *vcpu)
params.regval = vcpu_get_reg(vcpu, Rt);
params.is_write = !(esr & 1);
- ret = emulate_sys_reg(vcpu, ¶ms);
+ if (params.Op0 == 1) {
+ /* System instructions */
+ ret = emulate_sys_instr(vcpu, ¶ms);
+ } else {
+ /* MRS/MSR instructions */
+ ret = emulate_sys_reg(vcpu, ¶ms);
+ if (!params.is_write)
+ vcpu_set_reg(vcpu, Rt, params.regval);
+ }
- if (!params.is_write)
- vcpu_set_reg(vcpu, Rt, params.regval);
return ret;
}
@@ -3142,6 +3171,7 @@ void kvm_sys_reg_table_init(void)
BUG_ON(check_sysreg_table(cp15_regs, ARRAY_SIZE(cp15_regs), true));
BUG_ON(check_sysreg_table(cp15_64_regs, ARRAY_SIZE(cp15_64_regs), true));
BUG_ON(check_sysreg_table(invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs), false));
+ BUG_ON(check_sysreg_table(sys_insn_descs, ARRAY_SIZE(sys_insn_descs), false));
/* We abuse the reset function to overwrite the table itself. */
for (i = 0; i < ARRAY_SIZE(invariant_sys_regs); i++)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 40/66] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
When supporting nested virtualization a guest hypervisor executing AT
instructions must be trapped and emulated by the host hypervisor,
because untrapped AT instructions operating on S1E1 will use the wrong
translation regieme (the one used to emulate virtual EL2 in EL1 instead
of virtual EL1) and AT instructions operating on S12 will not work from
EL1.
This patch does several things.
1. List and define all AT system instructions to emulate and document
the emulation design.
2. Implement AT instruction handling logic in EL2. This will be used to
emulate AT instructions executed in the virtual EL2.
AT instruction emulation works by loading the proper processor
context, which depends on the trapped instruction and the virtual
HCR_EL2, to the EL1 virtual memory control registers and executing AT
instructions. Note that ctxt->hw_sys_regs is expected to have the
proper processor context before calling the handling
function(__kvm_at_insn) implemented in this patch.
4. Emulate AT S1E[01] instructions by issuing the same instructions in
EL2. We set the physical EL1 registers, NV and NV1 bits as described in
the AT instruction emulation overview.
5. Emulate AT A12E[01] instructions in two steps: First, do the stage-1
translation by reusing the existing AT emulation functions. Second, do
the stage-2 translation by walking the guest hypervisor's stage-2 page
table in software. Record the translation result to PAR_EL1.
6. Emulate AT S1E2 instructions by issuing the corresponding S1E1
instructions in EL2. We set the physical EL1 registers and the HCR_EL2
register as described in the AT instruction emulation overview.
7. Forward system instruction traps to the virtual EL2 if the corresponding
virtual AT bit is set in the virtual HCR_EL2.
[ Much logic above has been reworked by Marc Zyngier ]
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
---
arch/arm64/include/asm/kvm_arm.h | 2 +
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/sysreg.h | 17 +++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/at.c | 231 +++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/vhe/switch.c | 13 +-
arch/arm64/kvm/sys_regs.c | 201 ++++++++++++++++++++++++++-
7 files changed, 463 insertions(+), 5 deletions(-)
create mode 100644 arch/arm64/kvm/at.c
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 347e9e9b015c..0d88a7c51dec 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_AT (UL(1) << 44)
#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
@@ -110,6 +111,7 @@
#define VTCR_EL2_TG0_16K TCR_TG0_16K
#define VTCR_EL2_TG0_64K TCR_TG0_64K
#define VTCR_EL2_SH0_MASK TCR_SH0_MASK
+#define VTCR_EL2_SH0_SHIFT TCR_SH0_SHIFT
#define VTCR_EL2_SH0_INNER TCR_SH0_INNER
#define VTCR_EL2_ORGN0_MASK TCR_ORGN0_MASK
#define VTCR_EL2_ORGN0_WBWA TCR_ORGN0_WBWA
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index fa8e886998a3..8bda33d026ca 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -189,6 +189,8 @@ extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_timer_set_cntvoff(u64 cntvoff);
+extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
+extern void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 05b49eafbb49..4131a0c5df6e 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -591,6 +591,23 @@
#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
+/* AT instructions */
+#define AT_Op0 1
+#define AT_CRn 7
+
+#define OP_AT_S1E1R sys_insn(AT_Op0, 0, AT_CRn, 8, 0)
+#define OP_AT_S1E1W sys_insn(AT_Op0, 0, AT_CRn, 8, 1)
+#define OP_AT_S1E0R sys_insn(AT_Op0, 0, AT_CRn, 8, 2)
+#define OP_AT_S1E0W sys_insn(AT_Op0, 0, AT_CRn, 8, 3)
+#define OP_AT_S1E1RP sys_insn(AT_Op0, 0, AT_CRn, 9, 0)
+#define OP_AT_S1E1WP sys_insn(AT_Op0, 0, AT_CRn, 9, 1)
+#define OP_AT_S1E2R sys_insn(AT_Op0, 4, AT_CRn, 8, 0)
+#define OP_AT_S1E2W sys_insn(AT_Op0, 4, AT_CRn, 8, 1)
+#define OP_AT_S12E1R sys_insn(AT_Op0, 4, AT_CRn, 8, 4)
+#define OP_AT_S12E1W sys_insn(AT_Op0, 4, AT_CRn, 8, 5)
+#define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6)
+#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 7bb1d1314cad..8a23991f82b2 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,7 +17,7 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
- emulate-nested.o nested.o \
+ emulate-nested.o nested.o at.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
new file mode 100644
index 000000000000..c345ef98ca1e
--- /dev/null
+++ b/arch/arm64/kvm/at.c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 - Linaro Ltd
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/kvm_hyp.h>
+#include <asm/kvm_mmu.h>
+
+struct mmu_config {
+ u64 ttbr0;
+ u64 ttbr1;
+ u64 tcr;
+ u64 sctlr;
+ u64 vttbr;
+ u64 vtcr;
+ u64 hcr;
+};
+
+static void __mmu_config_save(struct mmu_config *config)
+{
+ config->ttbr0 = read_sysreg_el1(SYS_TTBR0);
+ config->ttbr1 = read_sysreg_el1(SYS_TTBR1);
+ config->tcr = read_sysreg_el1(SYS_TCR);
+ config->sctlr = read_sysreg_el1(SYS_SCTLR);
+ config->vttbr = read_sysreg(vttbr_el2);
+ config->vtcr = read_sysreg(vtcr_el2);
+ config->hcr = read_sysreg(hcr_el2);
+}
+
+static void __mmu_config_restore(struct mmu_config *config)
+{
+ write_sysreg_el1(config->ttbr0, SYS_TTBR0);
+ write_sysreg_el1(config->ttbr1, SYS_TTBR1);
+ write_sysreg_el1(config->tcr, SYS_TCR);
+ write_sysreg_el1(config->sctlr, SYS_SCTLR);
+ write_sysreg(config->vttbr, vttbr_el2);
+ write_sysreg(config->vtcr, vtcr_el2);
+ write_sysreg(config->hcr, hcr_el2);
+
+ isb();
+}
+
+void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ struct mmu_config config;
+ struct kvm_s2_mmu *mmu;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ /*
+ * If HCR_EL2.{E2H,TGE} == {1,1}, the MMU context is already
+ * the right one (as we trapped from vEL2).
+ */
+ if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu))
+ goto skip_mmu_switch;
+
+ /*
+ * FIXME: Obtaining the S2 MMU for a guest guest is horribly
+ * racy, and we may not find it (evicted by another vcpu, for
+ * example).
+ */
+ mmu = lookup_s2_mmu(vcpu->kvm,
+ vcpu_read_sys_reg(vcpu, VTTBR_EL2),
+ vcpu_read_sys_reg(vcpu, HCR_EL2));
+
+ if (WARN_ON(!mmu))
+ goto out;
+
+ /* We've trapped, so everything is live on the CPU. */
+ __mmu_config_save(&config);
+
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL1), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL1), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL1), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL1), SYS_SCTLR);
+ write_sysreg(kvm_get_vttbr(mmu), vttbr_el2);
+ /*
+ * REVISIT: do we need anything from the guest's VTCR_EL2? If
+ * looks like keeping the hosts configuration is the right
+ * thing to do at this stage (and we could avoid save/restore
+ * it. Keep the host's version for now.
+ */
+ write_sysreg((config.hcr & ~HCR_TGE) | HCR_VM, hcr_el2);
+
+ isb();
+
+skip_mmu_switch:
+
+ switch (op) {
+ case OP_AT_S1E1R:
+ case OP_AT_S1E1RP:
+ asm volatile("at s1e1r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E1W:
+ case OP_AT_S1E1WP:
+ asm volatile("at s1e1w, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E0R:
+ asm volatile("at s1e0r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E0W:
+ asm volatile("at s1e0w, %0" : : "r" (vaddr));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ isb();
+
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+
+ /*
+ * Failed? let's leave the building now.
+ *
+ * FIXME: how about a failed translation because the shadow S2
+ * wasn't populated? We may need to perform a SW PTW,
+ * populating our shadow S2 and retry the instruction.
+ */
+ if (ctxt_sys_reg(ctxt, PAR_EL1) & 1)
+ goto nopan;
+
+ /* No PAN? No problem. */
+ if (!(*vcpu_cpsr(vcpu) & PSR_PAN_BIT))
+ goto nopan;
+
+ /*
+ * For PAN-involved AT operations, perform the same
+ * translation, using EL0 this time.
+ */
+ switch (op) {
+ case OP_AT_S1E1RP:
+ asm volatile("at s1e0r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E1WP:
+ asm volatile("at s1e0w, %0" : : "r" (vaddr));
+ break;
+ default:
+ goto nopan;
+ }
+
+ /*
+ * If the EL0 translation has succeeded, we need to pretend
+ * the AT operation has failed, as the PAN setting forbids
+ * such a translation.
+ *
+ * FIXME: we hardcode a Level-3 permission fault. We really
+ * should return the real fault level.
+ */
+ if (!(read_sysreg(par_el1) & 1))
+ ctxt_sys_reg(ctxt, PAR_EL1) = 0x1f;
+
+nopan:
+ if (!(vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)))
+ __mmu_config_restore(&config);
+
+out:
+ spin_unlock(&vcpu->kvm->mmu_lock);
+}
+
+void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ struct mmu_config config;
+ struct kvm_s2_mmu *mmu;
+ u64 val;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = &vcpu->kvm->arch.mmu;
+
+ /* We've trapped, so everything is live on the CPU. */
+ __mmu_config_save(&config);
+
+ if (vcpu_el2_e2h_is_set(vcpu)) {
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL2), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL2), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL2), SYS_SCTLR);
+
+ val = config.hcr;
+ } else {
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ val = translate_tcr_el2_to_tcr_el1(ctxt_sys_reg(ctxt, TCR_EL2));
+ write_sysreg_el1(val, SYS_TCR);
+ val = translate_sctlr_el2_to_sctlr_el1(ctxt_sys_reg(ctxt, SCTLR_EL2));
+ write_sysreg_el1(val, SYS_SCTLR);
+
+ val = config.hcr | HCR_NV | HCR_NV1;
+ }
+
+ write_sysreg(kvm_get_vttbr(mmu), vttbr_el2);
+ /* FIXME: write S2 MMU VTCR_EL2? */
+ write_sysreg((val & ~HCR_TGE) | HCR_VM, hcr_el2);
+
+ isb();
+
+ switch (op) {
+ case OP_AT_S1E2R:
+ asm volatile("at s1e1r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E2W:
+ asm volatile("at s1e1w, %0" : : "r" (vaddr));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ isb();
+
+ /* FIXME: handle failed translation due to shadow S2 */
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+
+ __mmu_config_restore(&config);
+ spin_unlock(&vcpu->kvm->mmu_lock);
+}
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 940d88ffd53e..396d329e015d 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -45,9 +45,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_e2h_is_set(vcpu)) {
/*
* For a guest hypervisor on v8.0, trap and emulate
- * the EL1 virtual memory control register accesses.
+ * the EL1 virtual memory control register accesses
+ * as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_NV1;
+ hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -70,6 +71,14 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
hcr &= ~HCR_TVM;
hcr |= vhcr_el2 & (HCR_TVM | HCR_TRVM);
+
+ /*
+ * If we're using the EL1 translation regime
+ * (TGE clear), then ensure that AT S1 ops are
+ * trapped too.
+ */
+ if (!vcpu_el2_tge_is_set(vcpu))
+ hcr |= HCR_AT;
}
}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 35aa1a51aa7d..6b7a7ce9b1a3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1618,6 +1618,10 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool forward_at_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_AT);
+}
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
@@ -2115,12 +2119,205 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
-#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
- { SYS_DESC((insn)), (access_fn), NULL, 0, 0, NULL, NULL, (forward_fn) }
+static bool handle_s1e01(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ __kvm_at_s1e01(vcpu, sys_encoding, p->regval);
+
+ return true;
+}
+
+static bool handle_s1e2(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ __kvm_at_s1e2(vcpu, sys_encoding, p->regval);
+
+ return true;
+}
+
+static u64 setup_par_aborted(u32 esr)
+{
+ u64 par = 0;
+
+ /* S [9]: fault in the stage 2 translation */
+ par |= (1 << 9);
+ /* FST [6:1]: Fault status code */
+ par |= (esr << 1);
+ /* F [0]: translation is aborted */
+ par |= 1;
+
+ return par;
+}
+
+static u64 setup_par_completed(struct kvm_vcpu *vcpu, struct kvm_s2_trans *out)
+{
+ u64 par, vtcr_sh0;
+
+ /* F [0]: Translation is completed successfully */
+ par = 0;
+ /* ATTR [63:56] */
+ par |= out->upper_attr;
+ /* PA [47:12] */
+ par |= out->output & GENMASK_ULL(11, 0);
+ /* RES1 [11] */
+ par |= (1UL << 11);
+ /* SH [8:7]: Shareability attribute */
+ vtcr_sh0 = vcpu_read_sys_reg(vcpu, VTCR_EL2) & VTCR_EL2_SH0_MASK;
+ par |= (vtcr_sh0 >> VTCR_EL2_SH0_SHIFT) << 7;
+
+ return par;
+}
+
+static bool handle_s12(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r, bool write)
+{
+ u64 par, va;
+ u32 esr;
+ phys_addr_t ipa;
+ struct kvm_s2_trans out;
+ int ret;
+
+ /* Do the stage-1 translation */
+ handle_s1e01(vcpu, p, r);
+ par = vcpu_read_sys_reg(vcpu, PAR_EL1);
+ if (par & 1) {
+ /* The stage-1 translation aborted */
+ return true;
+ }
+
+ /* Do the stage-2 translation */
+ va = p->regval;
+ ipa = (par & GENMASK_ULL(47, 12)) | (va & GENMASK_ULL(11, 0));
+ out.esr = 0;
+ ret = kvm_walk_nested_s2(vcpu, ipa, &out);
+ if (ret < 0)
+ return false;
+
+ /* Check if the stage-2 PTW is aborted */
+ if (out.esr) {
+ esr = out.esr;
+ goto s2_trans_abort;
+ }
+
+ /* Check the access permission */
+ if ((!write && !out.readable) || (write && !out.writable)) {
+ esr = ESR_ELx_FSC_PERM;
+ esr |= out.level & 0x3;
+ goto s2_trans_abort;
+ }
+
+ vcpu_write_sys_reg(vcpu, setup_par_completed(vcpu, &out), PAR_EL1);
+ return true;
+
+s2_trans_abort:
+ vcpu_write_sys_reg(vcpu, setup_par_aborted(esr), PAR_EL1);
+ return true;
+}
+
+static bool handle_s12r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return handle_s12(vcpu, p, r, false);
+}
+
+static bool handle_s12w(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return handle_s12(vcpu, p, r, true);
+}
+
+/*
+ * AT instruction emulation
+ *
+ * We emulate AT instructions executed in the virtual EL2.
+ * Basic strategy for the stage-1 translation emulation is to load proper
+ * context, which depends on the trapped instruction and the virtual HCR_EL2,
+ * to the EL1 virtual memory control registers and execute S1E[01] instructions
+ * in EL2. See below for more detail.
+ *
+ * For the stage-2 translation, which is necessary for S12E[01] emulation,
+ * we walk the guest hypervisor's stage-2 page table in software.
+ *
+ * The stage-1 translation emulations can be divided into two groups depending
+ * on the translation regime.
+ *
+ * 1. EL2 AT instructions: S1E2x
+ * +-----------------------------------------------------------------------+
+ * | | Setting for the emulation |
+ * | Virtual HCR_EL2.E2H on trap |-----------------------------------------+
+ * | | Phys EL1 regs | Phys NV, NV1 | Phys TGE |
+ * |-----------------------------------------------------------------------|
+ * | 0 | vEL2 | (1, 1) | 0 |
+ * | 1 | vEL2 | (0, 0) | 0 |
+ * +-----------------------------------------------------------------------+
+ *
+ * We emulate the EL2 AT instructions by loading virtual EL2 context
+ * to the EL1 virtual memory control registers and executing corresponding
+ * EL1 AT instructions.
+ *
+ * We set physical NV and NV1 bits to use EL2 page table format for non-VHE
+ * guest hypervisor (i.e. HCR_EL2.E2H == 0). As a VHE guest hypervisor uses the
+ * EL1 page table format, we don't set those bits.
+ *
+ * We should clear physical TGE bit not to use the EL2 translation regime when
+ * the host uses the VHE feature.
+ *
+ *
+ * 2. EL0/EL1 AT instructions: S1E[01]x, S12E1x
+ * +----------------------------------------------------------------------+
+ * | Virtual HCR_EL2 on trap | Setting for the emulation |
+ * |----------------------------------------------------------------------+
+ * | (vE2H, vTGE) | (vNV, vNV1) | Phys EL1 regs | Phys NV, NV1 | Phys TGE |
+ * |----------------------------------------------------------------------|
+ * | (0, 0)* | (0, 0) | vEL1 | (0, 0) | 0 |
+ * | (0, 0) | (1, 1) | vEL1 | (1, 1) | 0 |
+ * | (1, 1) | (0, 0) | vEL2 | (0, 0) | 0 |
+ * | (1, 1) | (1, 1) | vEL2 | (1, 1) | 0 |
+ * +----------------------------------------------------------------------+
+ *
+ * *For (0, 0) in the 'Virtual HCR_EL2 on trap' column, it actually means
+ * (1, 1). Keep them (0, 0) just for the readability.
+ *
+ * We set physical EL1 virtual memory control registers depending on
+ * (vE2H, vTGE) pair. When the pair is (0, 0) where AT instructions are
+ * supposed to use EL0/EL1 translation regime, we load the EL1 registers with
+ * the virtual EL1 registers (i.e. EL1 registers from the guest hypervisor's
+ * point of view). When the pair is (1, 1), however, AT instructions are defined
+ * to apply EL2 translation regime. To emulate this behavior, we load the EL1
+ * registers with the virtual EL2 context. (i.e the shadow registers)
+ *
+ * We respect the virtual NV and NV1 bit for the emulation. When those bits are
+ * set, it means that a guest hypervisor would like to use EL2 page table format
+ * for the EL1 translation regime. We emulate this by setting the physical
+ * NV and NV1 bits.
+ */
+
+#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
+ { SYS_DESC(OP_##insn), (access_fn), NULL, 0, 0, \
+ NULL, NULL, (forward_fn) }
static struct sys_reg_desc sys_insn_descs[] = {
{ SYS_DESC(SYS_DC_ISW), access_dcsw },
+
+ SYS_INSN_TO_DESC(AT_S1E1R, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1W, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E0R, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E0W, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1RP, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1WP, handle_s1e01, forward_at_traps),
+
{ SYS_DESC(SYS_DC_CSW), access_dcsw },
{ SYS_DESC(SYS_DC_CISW), access_dcsw },
+
+ SYS_INSN_TO_DESC(AT_S1E2R, handle_s1e2, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S1E2W, handle_s1e2, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E1R, handle_s12r, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E1W, handle_s12w, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E0R, handle_s12r, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E0W, handle_s12w, forward_nv_traps),
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 40/66] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
When supporting nested virtualization a guest hypervisor executing AT
instructions must be trapped and emulated by the host hypervisor,
because untrapped AT instructions operating on S1E1 will use the wrong
translation regieme (the one used to emulate virtual EL2 in EL1 instead
of virtual EL1) and AT instructions operating on S12 will not work from
EL1.
This patch does several things.
1. List and define all AT system instructions to emulate and document
the emulation design.
2. Implement AT instruction handling logic in EL2. This will be used to
emulate AT instructions executed in the virtual EL2.
AT instruction emulation works by loading the proper processor
context, which depends on the trapped instruction and the virtual
HCR_EL2, to the EL1 virtual memory control registers and executing AT
instructions. Note that ctxt->hw_sys_regs is expected to have the
proper processor context before calling the handling
function(__kvm_at_insn) implemented in this patch.
4. Emulate AT S1E[01] instructions by issuing the same instructions in
EL2. We set the physical EL1 registers, NV and NV1 bits as described in
the AT instruction emulation overview.
5. Emulate AT A12E[01] instructions in two steps: First, do the stage-1
translation by reusing the existing AT emulation functions. Second, do
the stage-2 translation by walking the guest hypervisor's stage-2 page
table in software. Record the translation result to PAR_EL1.
6. Emulate AT S1E2 instructions by issuing the corresponding S1E1
instructions in EL2. We set the physical EL1 registers and the HCR_EL2
register as described in the AT instruction emulation overview.
7. Forward system instruction traps to the virtual EL2 if the corresponding
virtual AT bit is set in the virtual HCR_EL2.
[ Much logic above has been reworked by Marc Zyngier ]
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
---
arch/arm64/include/asm/kvm_arm.h | 2 +
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/sysreg.h | 17 +++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/at.c | 231 +++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/vhe/switch.c | 13 +-
arch/arm64/kvm/sys_regs.c | 201 ++++++++++++++++++++++++++-
7 files changed, 463 insertions(+), 5 deletions(-)
create mode 100644 arch/arm64/kvm/at.c
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 347e9e9b015c..0d88a7c51dec 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_AT (UL(1) << 44)
#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
@@ -110,6 +111,7 @@
#define VTCR_EL2_TG0_16K TCR_TG0_16K
#define VTCR_EL2_TG0_64K TCR_TG0_64K
#define VTCR_EL2_SH0_MASK TCR_SH0_MASK
+#define VTCR_EL2_SH0_SHIFT TCR_SH0_SHIFT
#define VTCR_EL2_SH0_INNER TCR_SH0_INNER
#define VTCR_EL2_ORGN0_MASK TCR_ORGN0_MASK
#define VTCR_EL2_ORGN0_WBWA TCR_ORGN0_WBWA
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index fa8e886998a3..8bda33d026ca 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -189,6 +189,8 @@ extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_timer_set_cntvoff(u64 cntvoff);
+extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
+extern void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 05b49eafbb49..4131a0c5df6e 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -591,6 +591,23 @@
#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
+/* AT instructions */
+#define AT_Op0 1
+#define AT_CRn 7
+
+#define OP_AT_S1E1R sys_insn(AT_Op0, 0, AT_CRn, 8, 0)
+#define OP_AT_S1E1W sys_insn(AT_Op0, 0, AT_CRn, 8, 1)
+#define OP_AT_S1E0R sys_insn(AT_Op0, 0, AT_CRn, 8, 2)
+#define OP_AT_S1E0W sys_insn(AT_Op0, 0, AT_CRn, 8, 3)
+#define OP_AT_S1E1RP sys_insn(AT_Op0, 0, AT_CRn, 9, 0)
+#define OP_AT_S1E1WP sys_insn(AT_Op0, 0, AT_CRn, 9, 1)
+#define OP_AT_S1E2R sys_insn(AT_Op0, 4, AT_CRn, 8, 0)
+#define OP_AT_S1E2W sys_insn(AT_Op0, 4, AT_CRn, 8, 1)
+#define OP_AT_S12E1R sys_insn(AT_Op0, 4, AT_CRn, 8, 4)
+#define OP_AT_S12E1W sys_insn(AT_Op0, 4, AT_CRn, 8, 5)
+#define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6)
+#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 7bb1d1314cad..8a23991f82b2 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,7 +17,7 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
- emulate-nested.o nested.o \
+ emulate-nested.o nested.o at.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
new file mode 100644
index 000000000000..c345ef98ca1e
--- /dev/null
+++ b/arch/arm64/kvm/at.c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 - Linaro Ltd
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/kvm_hyp.h>
+#include <asm/kvm_mmu.h>
+
+struct mmu_config {
+ u64 ttbr0;
+ u64 ttbr1;
+ u64 tcr;
+ u64 sctlr;
+ u64 vttbr;
+ u64 vtcr;
+ u64 hcr;
+};
+
+static void __mmu_config_save(struct mmu_config *config)
+{
+ config->ttbr0 = read_sysreg_el1(SYS_TTBR0);
+ config->ttbr1 = read_sysreg_el1(SYS_TTBR1);
+ config->tcr = read_sysreg_el1(SYS_TCR);
+ config->sctlr = read_sysreg_el1(SYS_SCTLR);
+ config->vttbr = read_sysreg(vttbr_el2);
+ config->vtcr = read_sysreg(vtcr_el2);
+ config->hcr = read_sysreg(hcr_el2);
+}
+
+static void __mmu_config_restore(struct mmu_config *config)
+{
+ write_sysreg_el1(config->ttbr0, SYS_TTBR0);
+ write_sysreg_el1(config->ttbr1, SYS_TTBR1);
+ write_sysreg_el1(config->tcr, SYS_TCR);
+ write_sysreg_el1(config->sctlr, SYS_SCTLR);
+ write_sysreg(config->vttbr, vttbr_el2);
+ write_sysreg(config->vtcr, vtcr_el2);
+ write_sysreg(config->hcr, hcr_el2);
+
+ isb();
+}
+
+void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ struct mmu_config config;
+ struct kvm_s2_mmu *mmu;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ /*
+ * If HCR_EL2.{E2H,TGE} == {1,1}, the MMU context is already
+ * the right one (as we trapped from vEL2).
+ */
+ if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu))
+ goto skip_mmu_switch;
+
+ /*
+ * FIXME: Obtaining the S2 MMU for a guest guest is horribly
+ * racy, and we may not find it (evicted by another vcpu, for
+ * example).
+ */
+ mmu = lookup_s2_mmu(vcpu->kvm,
+ vcpu_read_sys_reg(vcpu, VTTBR_EL2),
+ vcpu_read_sys_reg(vcpu, HCR_EL2));
+
+ if (WARN_ON(!mmu))
+ goto out;
+
+ /* We've trapped, so everything is live on the CPU. */
+ __mmu_config_save(&config);
+
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL1), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL1), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL1), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL1), SYS_SCTLR);
+ write_sysreg(kvm_get_vttbr(mmu), vttbr_el2);
+ /*
+ * REVISIT: do we need anything from the guest's VTCR_EL2? If
+ * looks like keeping the hosts configuration is the right
+ * thing to do at this stage (and we could avoid save/restore
+ * it. Keep the host's version for now.
+ */
+ write_sysreg((config.hcr & ~HCR_TGE) | HCR_VM, hcr_el2);
+
+ isb();
+
+skip_mmu_switch:
+
+ switch (op) {
+ case OP_AT_S1E1R:
+ case OP_AT_S1E1RP:
+ asm volatile("at s1e1r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E1W:
+ case OP_AT_S1E1WP:
+ asm volatile("at s1e1w, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E0R:
+ asm volatile("at s1e0r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E0W:
+ asm volatile("at s1e0w, %0" : : "r" (vaddr));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ isb();
+
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+
+ /*
+ * Failed? let's leave the building now.
+ *
+ * FIXME: how about a failed translation because the shadow S2
+ * wasn't populated? We may need to perform a SW PTW,
+ * populating our shadow S2 and retry the instruction.
+ */
+ if (ctxt_sys_reg(ctxt, PAR_EL1) & 1)
+ goto nopan;
+
+ /* No PAN? No problem. */
+ if (!(*vcpu_cpsr(vcpu) & PSR_PAN_BIT))
+ goto nopan;
+
+ /*
+ * For PAN-involved AT operations, perform the same
+ * translation, using EL0 this time.
+ */
+ switch (op) {
+ case OP_AT_S1E1RP:
+ asm volatile("at s1e0r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E1WP:
+ asm volatile("at s1e0w, %0" : : "r" (vaddr));
+ break;
+ default:
+ goto nopan;
+ }
+
+ /*
+ * If the EL0 translation has succeeded, we need to pretend
+ * the AT operation has failed, as the PAN setting forbids
+ * such a translation.
+ *
+ * FIXME: we hardcode a Level-3 permission fault. We really
+ * should return the real fault level.
+ */
+ if (!(read_sysreg(par_el1) & 1))
+ ctxt_sys_reg(ctxt, PAR_EL1) = 0x1f;
+
+nopan:
+ if (!(vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)))
+ __mmu_config_restore(&config);
+
+out:
+ spin_unlock(&vcpu->kvm->mmu_lock);
+}
+
+void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ struct mmu_config config;
+ struct kvm_s2_mmu *mmu;
+ u64 val;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = &vcpu->kvm->arch.mmu;
+
+ /* We've trapped, so everything is live on the CPU. */
+ __mmu_config_save(&config);
+
+ if (vcpu_el2_e2h_is_set(vcpu)) {
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL2), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL2), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL2), SYS_SCTLR);
+
+ val = config.hcr;
+ } else {
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ val = translate_tcr_el2_to_tcr_el1(ctxt_sys_reg(ctxt, TCR_EL2));
+ write_sysreg_el1(val, SYS_TCR);
+ val = translate_sctlr_el2_to_sctlr_el1(ctxt_sys_reg(ctxt, SCTLR_EL2));
+ write_sysreg_el1(val, SYS_SCTLR);
+
+ val = config.hcr | HCR_NV | HCR_NV1;
+ }
+
+ write_sysreg(kvm_get_vttbr(mmu), vttbr_el2);
+ /* FIXME: write S2 MMU VTCR_EL2? */
+ write_sysreg((val & ~HCR_TGE) | HCR_VM, hcr_el2);
+
+ isb();
+
+ switch (op) {
+ case OP_AT_S1E2R:
+ asm volatile("at s1e1r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E2W:
+ asm volatile("at s1e1w, %0" : : "r" (vaddr));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ isb();
+
+ /* FIXME: handle failed translation due to shadow S2 */
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+
+ __mmu_config_restore(&config);
+ spin_unlock(&vcpu->kvm->mmu_lock);
+}
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 940d88ffd53e..396d329e015d 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -45,9 +45,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_e2h_is_set(vcpu)) {
/*
* For a guest hypervisor on v8.0, trap and emulate
- * the EL1 virtual memory control register accesses.
+ * the EL1 virtual memory control register accesses
+ * as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_NV1;
+ hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -70,6 +71,14 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
hcr &= ~HCR_TVM;
hcr |= vhcr_el2 & (HCR_TVM | HCR_TRVM);
+
+ /*
+ * If we're using the EL1 translation regime
+ * (TGE clear), then ensure that AT S1 ops are
+ * trapped too.
+ */
+ if (!vcpu_el2_tge_is_set(vcpu))
+ hcr |= HCR_AT;
}
}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 35aa1a51aa7d..6b7a7ce9b1a3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1618,6 +1618,10 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool forward_at_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_AT);
+}
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
@@ -2115,12 +2119,205 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
-#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
- { SYS_DESC((insn)), (access_fn), NULL, 0, 0, NULL, NULL, (forward_fn) }
+static bool handle_s1e01(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ __kvm_at_s1e01(vcpu, sys_encoding, p->regval);
+
+ return true;
+}
+
+static bool handle_s1e2(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ __kvm_at_s1e2(vcpu, sys_encoding, p->regval);
+
+ return true;
+}
+
+static u64 setup_par_aborted(u32 esr)
+{
+ u64 par = 0;
+
+ /* S [9]: fault in the stage 2 translation */
+ par |= (1 << 9);
+ /* FST [6:1]: Fault status code */
+ par |= (esr << 1);
+ /* F [0]: translation is aborted */
+ par |= 1;
+
+ return par;
+}
+
+static u64 setup_par_completed(struct kvm_vcpu *vcpu, struct kvm_s2_trans *out)
+{
+ u64 par, vtcr_sh0;
+
+ /* F [0]: Translation is completed successfully */
+ par = 0;
+ /* ATTR [63:56] */
+ par |= out->upper_attr;
+ /* PA [47:12] */
+ par |= out->output & GENMASK_ULL(11, 0);
+ /* RES1 [11] */
+ par |= (1UL << 11);
+ /* SH [8:7]: Shareability attribute */
+ vtcr_sh0 = vcpu_read_sys_reg(vcpu, VTCR_EL2) & VTCR_EL2_SH0_MASK;
+ par |= (vtcr_sh0 >> VTCR_EL2_SH0_SHIFT) << 7;
+
+ return par;
+}
+
+static bool handle_s12(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r, bool write)
+{
+ u64 par, va;
+ u32 esr;
+ phys_addr_t ipa;
+ struct kvm_s2_trans out;
+ int ret;
+
+ /* Do the stage-1 translation */
+ handle_s1e01(vcpu, p, r);
+ par = vcpu_read_sys_reg(vcpu, PAR_EL1);
+ if (par & 1) {
+ /* The stage-1 translation aborted */
+ return true;
+ }
+
+ /* Do the stage-2 translation */
+ va = p->regval;
+ ipa = (par & GENMASK_ULL(47, 12)) | (va & GENMASK_ULL(11, 0));
+ out.esr = 0;
+ ret = kvm_walk_nested_s2(vcpu, ipa, &out);
+ if (ret < 0)
+ return false;
+
+ /* Check if the stage-2 PTW is aborted */
+ if (out.esr) {
+ esr = out.esr;
+ goto s2_trans_abort;
+ }
+
+ /* Check the access permission */
+ if ((!write && !out.readable) || (write && !out.writable)) {
+ esr = ESR_ELx_FSC_PERM;
+ esr |= out.level & 0x3;
+ goto s2_trans_abort;
+ }
+
+ vcpu_write_sys_reg(vcpu, setup_par_completed(vcpu, &out), PAR_EL1);
+ return true;
+
+s2_trans_abort:
+ vcpu_write_sys_reg(vcpu, setup_par_aborted(esr), PAR_EL1);
+ return true;
+}
+
+static bool handle_s12r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return handle_s12(vcpu, p, r, false);
+}
+
+static bool handle_s12w(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return handle_s12(vcpu, p, r, true);
+}
+
+/*
+ * AT instruction emulation
+ *
+ * We emulate AT instructions executed in the virtual EL2.
+ * Basic strategy for the stage-1 translation emulation is to load proper
+ * context, which depends on the trapped instruction and the virtual HCR_EL2,
+ * to the EL1 virtual memory control registers and execute S1E[01] instructions
+ * in EL2. See below for more detail.
+ *
+ * For the stage-2 translation, which is necessary for S12E[01] emulation,
+ * we walk the guest hypervisor's stage-2 page table in software.
+ *
+ * The stage-1 translation emulations can be divided into two groups depending
+ * on the translation regime.
+ *
+ * 1. EL2 AT instructions: S1E2x
+ * +-----------------------------------------------------------------------+
+ * | | Setting for the emulation |
+ * | Virtual HCR_EL2.E2H on trap |-----------------------------------------+
+ * | | Phys EL1 regs | Phys NV, NV1 | Phys TGE |
+ * |-----------------------------------------------------------------------|
+ * | 0 | vEL2 | (1, 1) | 0 |
+ * | 1 | vEL2 | (0, 0) | 0 |
+ * +-----------------------------------------------------------------------+
+ *
+ * We emulate the EL2 AT instructions by loading virtual EL2 context
+ * to the EL1 virtual memory control registers and executing corresponding
+ * EL1 AT instructions.
+ *
+ * We set physical NV and NV1 bits to use EL2 page table format for non-VHE
+ * guest hypervisor (i.e. HCR_EL2.E2H == 0). As a VHE guest hypervisor uses the
+ * EL1 page table format, we don't set those bits.
+ *
+ * We should clear physical TGE bit not to use the EL2 translation regime when
+ * the host uses the VHE feature.
+ *
+ *
+ * 2. EL0/EL1 AT instructions: S1E[01]x, S12E1x
+ * +----------------------------------------------------------------------+
+ * | Virtual HCR_EL2 on trap | Setting for the emulation |
+ * |----------------------------------------------------------------------+
+ * | (vE2H, vTGE) | (vNV, vNV1) | Phys EL1 regs | Phys NV, NV1 | Phys TGE |
+ * |----------------------------------------------------------------------|
+ * | (0, 0)* | (0, 0) | vEL1 | (0, 0) | 0 |
+ * | (0, 0) | (1, 1) | vEL1 | (1, 1) | 0 |
+ * | (1, 1) | (0, 0) | vEL2 | (0, 0) | 0 |
+ * | (1, 1) | (1, 1) | vEL2 | (1, 1) | 0 |
+ * +----------------------------------------------------------------------+
+ *
+ * *For (0, 0) in the 'Virtual HCR_EL2 on trap' column, it actually means
+ * (1, 1). Keep them (0, 0) just for the readability.
+ *
+ * We set physical EL1 virtual memory control registers depending on
+ * (vE2H, vTGE) pair. When the pair is (0, 0) where AT instructions are
+ * supposed to use EL0/EL1 translation regime, we load the EL1 registers with
+ * the virtual EL1 registers (i.e. EL1 registers from the guest hypervisor's
+ * point of view). When the pair is (1, 1), however, AT instructions are defined
+ * to apply EL2 translation regime. To emulate this behavior, we load the EL1
+ * registers with the virtual EL2 context. (i.e the shadow registers)
+ *
+ * We respect the virtual NV and NV1 bit for the emulation. When those bits are
+ * set, it means that a guest hypervisor would like to use EL2 page table format
+ * for the EL1 translation regime. We emulate this by setting the physical
+ * NV and NV1 bits.
+ */
+
+#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
+ { SYS_DESC(OP_##insn), (access_fn), NULL, 0, 0, \
+ NULL, NULL, (forward_fn) }
static struct sys_reg_desc sys_insn_descs[] = {
{ SYS_DESC(SYS_DC_ISW), access_dcsw },
+
+ SYS_INSN_TO_DESC(AT_S1E1R, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1W, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E0R, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E0W, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1RP, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1WP, handle_s1e01, forward_at_traps),
+
{ SYS_DESC(SYS_DC_CSW), access_dcsw },
{ SYS_DESC(SYS_DC_CISW), access_dcsw },
+
+ SYS_INSN_TO_DESC(AT_S1E2R, handle_s1e2, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S1E2W, handle_s1e2, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E1R, handle_s12r, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E1W, handle_s12w, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E0R, handle_s12r, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E0W, handle_s12w, forward_nv_traps),
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 40/66] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
When supporting nested virtualization a guest hypervisor executing AT
instructions must be trapped and emulated by the host hypervisor,
because untrapped AT instructions operating on S1E1 will use the wrong
translation regieme (the one used to emulate virtual EL2 in EL1 instead
of virtual EL1) and AT instructions operating on S12 will not work from
EL1.
This patch does several things.
1. List and define all AT system instructions to emulate and document
the emulation design.
2. Implement AT instruction handling logic in EL2. This will be used to
emulate AT instructions executed in the virtual EL2.
AT instruction emulation works by loading the proper processor
context, which depends on the trapped instruction and the virtual
HCR_EL2, to the EL1 virtual memory control registers and executing AT
instructions. Note that ctxt->hw_sys_regs is expected to have the
proper processor context before calling the handling
function(__kvm_at_insn) implemented in this patch.
4. Emulate AT S1E[01] instructions by issuing the same instructions in
EL2. We set the physical EL1 registers, NV and NV1 bits as described in
the AT instruction emulation overview.
5. Emulate AT A12E[01] instructions in two steps: First, do the stage-1
translation by reusing the existing AT emulation functions. Second, do
the stage-2 translation by walking the guest hypervisor's stage-2 page
table in software. Record the translation result to PAR_EL1.
6. Emulate AT S1E2 instructions by issuing the corresponding S1E1
instructions in EL2. We set the physical EL1 registers and the HCR_EL2
register as described in the AT instruction emulation overview.
7. Forward system instruction traps to the virtual EL2 if the corresponding
virtual AT bit is set in the virtual HCR_EL2.
[ Much logic above has been reworked by Marc Zyngier ]
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
---
arch/arm64/include/asm/kvm_arm.h | 2 +
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/sysreg.h | 17 +++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/at.c | 231 +++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/vhe/switch.c | 13 +-
arch/arm64/kvm/sys_regs.c | 201 ++++++++++++++++++++++++++-
7 files changed, 463 insertions(+), 5 deletions(-)
create mode 100644 arch/arm64/kvm/at.c
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 347e9e9b015c..0d88a7c51dec 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_AT (UL(1) << 44)
#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
#define HCR_API (UL(1) << 41)
@@ -110,6 +111,7 @@
#define VTCR_EL2_TG0_16K TCR_TG0_16K
#define VTCR_EL2_TG0_64K TCR_TG0_64K
#define VTCR_EL2_SH0_MASK TCR_SH0_MASK
+#define VTCR_EL2_SH0_SHIFT TCR_SH0_SHIFT
#define VTCR_EL2_SH0_INNER TCR_SH0_INNER
#define VTCR_EL2_ORGN0_MASK TCR_ORGN0_MASK
#define VTCR_EL2_ORGN0_WBWA TCR_ORGN0_WBWA
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index fa8e886998a3..8bda33d026ca 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -189,6 +189,8 @@ extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_timer_set_cntvoff(u64 cntvoff);
+extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
+extern void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 05b49eafbb49..4131a0c5df6e 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -591,6 +591,23 @@
#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
+/* AT instructions */
+#define AT_Op0 1
+#define AT_CRn 7
+
+#define OP_AT_S1E1R sys_insn(AT_Op0, 0, AT_CRn, 8, 0)
+#define OP_AT_S1E1W sys_insn(AT_Op0, 0, AT_CRn, 8, 1)
+#define OP_AT_S1E0R sys_insn(AT_Op0, 0, AT_CRn, 8, 2)
+#define OP_AT_S1E0W sys_insn(AT_Op0, 0, AT_CRn, 8, 3)
+#define OP_AT_S1E1RP sys_insn(AT_Op0, 0, AT_CRn, 9, 0)
+#define OP_AT_S1E1WP sys_insn(AT_Op0, 0, AT_CRn, 9, 1)
+#define OP_AT_S1E2R sys_insn(AT_Op0, 4, AT_CRn, 8, 0)
+#define OP_AT_S1E2W sys_insn(AT_Op0, 4, AT_CRn, 8, 1)
+#define OP_AT_S12E1R sys_insn(AT_Op0, 4, AT_CRn, 8, 4)
+#define OP_AT_S12E1W sys_insn(AT_Op0, 4, AT_CRn, 8, 5)
+#define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6)
+#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 7bb1d1314cad..8a23991f82b2 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -17,7 +17,7 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
guest.o debug.o reset.o sys_regs.o \
vgic-sys-reg-v3.o fpsimd.o pmu.o \
arch_timer.o \
- emulate-nested.o nested.o \
+ emulate-nested.o nested.o at.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
new file mode 100644
index 000000000000..c345ef98ca1e
--- /dev/null
+++ b/arch/arm64/kvm/at.c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 - Linaro Ltd
+ * Author: Jintack Lim <jintack.lim@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/kvm_hyp.h>
+#include <asm/kvm_mmu.h>
+
+struct mmu_config {
+ u64 ttbr0;
+ u64 ttbr1;
+ u64 tcr;
+ u64 sctlr;
+ u64 vttbr;
+ u64 vtcr;
+ u64 hcr;
+};
+
+static void __mmu_config_save(struct mmu_config *config)
+{
+ config->ttbr0 = read_sysreg_el1(SYS_TTBR0);
+ config->ttbr1 = read_sysreg_el1(SYS_TTBR1);
+ config->tcr = read_sysreg_el1(SYS_TCR);
+ config->sctlr = read_sysreg_el1(SYS_SCTLR);
+ config->vttbr = read_sysreg(vttbr_el2);
+ config->vtcr = read_sysreg(vtcr_el2);
+ config->hcr = read_sysreg(hcr_el2);
+}
+
+static void __mmu_config_restore(struct mmu_config *config)
+{
+ write_sysreg_el1(config->ttbr0, SYS_TTBR0);
+ write_sysreg_el1(config->ttbr1, SYS_TTBR1);
+ write_sysreg_el1(config->tcr, SYS_TCR);
+ write_sysreg_el1(config->sctlr, SYS_SCTLR);
+ write_sysreg(config->vttbr, vttbr_el2);
+ write_sysreg(config->vtcr, vtcr_el2);
+ write_sysreg(config->hcr, hcr_el2);
+
+ isb();
+}
+
+void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ struct mmu_config config;
+ struct kvm_s2_mmu *mmu;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ /*
+ * If HCR_EL2.{E2H,TGE} == {1,1}, the MMU context is already
+ * the right one (as we trapped from vEL2).
+ */
+ if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu))
+ goto skip_mmu_switch;
+
+ /*
+ * FIXME: Obtaining the S2 MMU for a guest guest is horribly
+ * racy, and we may not find it (evicted by another vcpu, for
+ * example).
+ */
+ mmu = lookup_s2_mmu(vcpu->kvm,
+ vcpu_read_sys_reg(vcpu, VTTBR_EL2),
+ vcpu_read_sys_reg(vcpu, HCR_EL2));
+
+ if (WARN_ON(!mmu))
+ goto out;
+
+ /* We've trapped, so everything is live on the CPU. */
+ __mmu_config_save(&config);
+
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL1), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL1), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL1), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL1), SYS_SCTLR);
+ write_sysreg(kvm_get_vttbr(mmu), vttbr_el2);
+ /*
+ * REVISIT: do we need anything from the guest's VTCR_EL2? If
+ * looks like keeping the hosts configuration is the right
+ * thing to do at this stage (and we could avoid save/restore
+ * it. Keep the host's version for now.
+ */
+ write_sysreg((config.hcr & ~HCR_TGE) | HCR_VM, hcr_el2);
+
+ isb();
+
+skip_mmu_switch:
+
+ switch (op) {
+ case OP_AT_S1E1R:
+ case OP_AT_S1E1RP:
+ asm volatile("at s1e1r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E1W:
+ case OP_AT_S1E1WP:
+ asm volatile("at s1e1w, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E0R:
+ asm volatile("at s1e0r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E0W:
+ asm volatile("at s1e0w, %0" : : "r" (vaddr));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ isb();
+
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+
+ /*
+ * Failed? let's leave the building now.
+ *
+ * FIXME: how about a failed translation because the shadow S2
+ * wasn't populated? We may need to perform a SW PTW,
+ * populating our shadow S2 and retry the instruction.
+ */
+ if (ctxt_sys_reg(ctxt, PAR_EL1) & 1)
+ goto nopan;
+
+ /* No PAN? No problem. */
+ if (!(*vcpu_cpsr(vcpu) & PSR_PAN_BIT))
+ goto nopan;
+
+ /*
+ * For PAN-involved AT operations, perform the same
+ * translation, using EL0 this time.
+ */
+ switch (op) {
+ case OP_AT_S1E1RP:
+ asm volatile("at s1e0r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E1WP:
+ asm volatile("at s1e0w, %0" : : "r" (vaddr));
+ break;
+ default:
+ goto nopan;
+ }
+
+ /*
+ * If the EL0 translation has succeeded, we need to pretend
+ * the AT operation has failed, as the PAN setting forbids
+ * such a translation.
+ *
+ * FIXME: we hardcode a Level-3 permission fault. We really
+ * should return the real fault level.
+ */
+ if (!(read_sysreg(par_el1) & 1))
+ ctxt_sys_reg(ctxt, PAR_EL1) = 0x1f;
+
+nopan:
+ if (!(vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)))
+ __mmu_config_restore(&config);
+
+out:
+ spin_unlock(&vcpu->kvm->mmu_lock);
+}
+
+void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ struct mmu_config config;
+ struct kvm_s2_mmu *mmu;
+ u64 val;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = &vcpu->kvm->arch.mmu;
+
+ /* We've trapped, so everything is live on the CPU. */
+ __mmu_config_save(&config);
+
+ if (vcpu_el2_e2h_is_set(vcpu)) {
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR1_EL2), SYS_TTBR1);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TCR_EL2), SYS_TCR);
+ write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR_EL2), SYS_SCTLR);
+
+ val = config.hcr;
+ } else {
+ write_sysreg_el1(ctxt_sys_reg(ctxt, TTBR0_EL2), SYS_TTBR0);
+ val = translate_tcr_el2_to_tcr_el1(ctxt_sys_reg(ctxt, TCR_EL2));
+ write_sysreg_el1(val, SYS_TCR);
+ val = translate_sctlr_el2_to_sctlr_el1(ctxt_sys_reg(ctxt, SCTLR_EL2));
+ write_sysreg_el1(val, SYS_SCTLR);
+
+ val = config.hcr | HCR_NV | HCR_NV1;
+ }
+
+ write_sysreg(kvm_get_vttbr(mmu), vttbr_el2);
+ /* FIXME: write S2 MMU VTCR_EL2? */
+ write_sysreg((val & ~HCR_TGE) | HCR_VM, hcr_el2);
+
+ isb();
+
+ switch (op) {
+ case OP_AT_S1E2R:
+ asm volatile("at s1e1r, %0" : : "r" (vaddr));
+ break;
+ case OP_AT_S1E2W:
+ asm volatile("at s1e1w, %0" : : "r" (vaddr));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ isb();
+
+ /* FIXME: handle failed translation due to shadow S2 */
+ ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg(par_el1);
+
+ __mmu_config_restore(&config);
+ spin_unlock(&vcpu->kvm->mmu_lock);
+}
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 940d88ffd53e..396d329e015d 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -45,9 +45,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_e2h_is_set(vcpu)) {
/*
* For a guest hypervisor on v8.0, trap and emulate
- * the EL1 virtual memory control register accesses.
+ * the EL1 virtual memory control register accesses
+ * as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_NV1;
+ hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -70,6 +71,14 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
hcr &= ~HCR_TVM;
hcr |= vhcr_el2 & (HCR_TVM | HCR_TRVM);
+
+ /*
+ * If we're using the EL1 translation regime
+ * (TGE clear), then ensure that AT S1 ops are
+ * trapped too.
+ */
+ if (!vcpu_el2_tge_is_set(vcpu))
+ hcr |= HCR_AT;
}
}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 35aa1a51aa7d..6b7a7ce9b1a3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1618,6 +1618,10 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu,
return true;
}
+static bool forward_at_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_AT);
+}
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
@@ -2115,12 +2119,205 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
-#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
- { SYS_DESC((insn)), (access_fn), NULL, 0, 0, NULL, NULL, (forward_fn) }
+static bool handle_s1e01(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ __kvm_at_s1e01(vcpu, sys_encoding, p->regval);
+
+ return true;
+}
+
+static bool handle_s1e2(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ __kvm_at_s1e2(vcpu, sys_encoding, p->regval);
+
+ return true;
+}
+
+static u64 setup_par_aborted(u32 esr)
+{
+ u64 par = 0;
+
+ /* S [9]: fault in the stage 2 translation */
+ par |= (1 << 9);
+ /* FST [6:1]: Fault status code */
+ par |= (esr << 1);
+ /* F [0]: translation is aborted */
+ par |= 1;
+
+ return par;
+}
+
+static u64 setup_par_completed(struct kvm_vcpu *vcpu, struct kvm_s2_trans *out)
+{
+ u64 par, vtcr_sh0;
+
+ /* F [0]: Translation is completed successfully */
+ par = 0;
+ /* ATTR [63:56] */
+ par |= out->upper_attr;
+ /* PA [47:12] */
+ par |= out->output & GENMASK_ULL(11, 0);
+ /* RES1 [11] */
+ par |= (1UL << 11);
+ /* SH [8:7]: Shareability attribute */
+ vtcr_sh0 = vcpu_read_sys_reg(vcpu, VTCR_EL2) & VTCR_EL2_SH0_MASK;
+ par |= (vtcr_sh0 >> VTCR_EL2_SH0_SHIFT) << 7;
+
+ return par;
+}
+
+static bool handle_s12(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r, bool write)
+{
+ u64 par, va;
+ u32 esr;
+ phys_addr_t ipa;
+ struct kvm_s2_trans out;
+ int ret;
+
+ /* Do the stage-1 translation */
+ handle_s1e01(vcpu, p, r);
+ par = vcpu_read_sys_reg(vcpu, PAR_EL1);
+ if (par & 1) {
+ /* The stage-1 translation aborted */
+ return true;
+ }
+
+ /* Do the stage-2 translation */
+ va = p->regval;
+ ipa = (par & GENMASK_ULL(47, 12)) | (va & GENMASK_ULL(11, 0));
+ out.esr = 0;
+ ret = kvm_walk_nested_s2(vcpu, ipa, &out);
+ if (ret < 0)
+ return false;
+
+ /* Check if the stage-2 PTW is aborted */
+ if (out.esr) {
+ esr = out.esr;
+ goto s2_trans_abort;
+ }
+
+ /* Check the access permission */
+ if ((!write && !out.readable) || (write && !out.writable)) {
+ esr = ESR_ELx_FSC_PERM;
+ esr |= out.level & 0x3;
+ goto s2_trans_abort;
+ }
+
+ vcpu_write_sys_reg(vcpu, setup_par_completed(vcpu, &out), PAR_EL1);
+ return true;
+
+s2_trans_abort:
+ vcpu_write_sys_reg(vcpu, setup_par_aborted(esr), PAR_EL1);
+ return true;
+}
+
+static bool handle_s12r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return handle_s12(vcpu, p, r, false);
+}
+
+static bool handle_s12w(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return handle_s12(vcpu, p, r, true);
+}
+
+/*
+ * AT instruction emulation
+ *
+ * We emulate AT instructions executed in the virtual EL2.
+ * Basic strategy for the stage-1 translation emulation is to load proper
+ * context, which depends on the trapped instruction and the virtual HCR_EL2,
+ * to the EL1 virtual memory control registers and execute S1E[01] instructions
+ * in EL2. See below for more detail.
+ *
+ * For the stage-2 translation, which is necessary for S12E[01] emulation,
+ * we walk the guest hypervisor's stage-2 page table in software.
+ *
+ * The stage-1 translation emulations can be divided into two groups depending
+ * on the translation regime.
+ *
+ * 1. EL2 AT instructions: S1E2x
+ * +-----------------------------------------------------------------------+
+ * | | Setting for the emulation |
+ * | Virtual HCR_EL2.E2H on trap |-----------------------------------------+
+ * | | Phys EL1 regs | Phys NV, NV1 | Phys TGE |
+ * |-----------------------------------------------------------------------|
+ * | 0 | vEL2 | (1, 1) | 0 |
+ * | 1 | vEL2 | (0, 0) | 0 |
+ * +-----------------------------------------------------------------------+
+ *
+ * We emulate the EL2 AT instructions by loading virtual EL2 context
+ * to the EL1 virtual memory control registers and executing corresponding
+ * EL1 AT instructions.
+ *
+ * We set physical NV and NV1 bits to use EL2 page table format for non-VHE
+ * guest hypervisor (i.e. HCR_EL2.E2H == 0). As a VHE guest hypervisor uses the
+ * EL1 page table format, we don't set those bits.
+ *
+ * We should clear physical TGE bit not to use the EL2 translation regime when
+ * the host uses the VHE feature.
+ *
+ *
+ * 2. EL0/EL1 AT instructions: S1E[01]x, S12E1x
+ * +----------------------------------------------------------------------+
+ * | Virtual HCR_EL2 on trap | Setting for the emulation |
+ * |----------------------------------------------------------------------+
+ * | (vE2H, vTGE) | (vNV, vNV1) | Phys EL1 regs | Phys NV, NV1 | Phys TGE |
+ * |----------------------------------------------------------------------|
+ * | (0, 0)* | (0, 0) | vEL1 | (0, 0) | 0 |
+ * | (0, 0) | (1, 1) | vEL1 | (1, 1) | 0 |
+ * | (1, 1) | (0, 0) | vEL2 | (0, 0) | 0 |
+ * | (1, 1) | (1, 1) | vEL2 | (1, 1) | 0 |
+ * +----------------------------------------------------------------------+
+ *
+ * *For (0, 0) in the 'Virtual HCR_EL2 on trap' column, it actually means
+ * (1, 1). Keep them (0, 0) just for the readability.
+ *
+ * We set physical EL1 virtual memory control registers depending on
+ * (vE2H, vTGE) pair. When the pair is (0, 0) where AT instructions are
+ * supposed to use EL0/EL1 translation regime, we load the EL1 registers with
+ * the virtual EL1 registers (i.e. EL1 registers from the guest hypervisor's
+ * point of view). When the pair is (1, 1), however, AT instructions are defined
+ * to apply EL2 translation regime. To emulate this behavior, we load the EL1
+ * registers with the virtual EL2 context. (i.e the shadow registers)
+ *
+ * We respect the virtual NV and NV1 bit for the emulation. When those bits are
+ * set, it means that a guest hypervisor would like to use EL2 page table format
+ * for the EL1 translation regime. We emulate this by setting the physical
+ * NV and NV1 bits.
+ */
+
+#define SYS_INSN_TO_DESC(insn, access_fn, forward_fn) \
+ { SYS_DESC(OP_##insn), (access_fn), NULL, 0, 0, \
+ NULL, NULL, (forward_fn) }
static struct sys_reg_desc sys_insn_descs[] = {
{ SYS_DESC(SYS_DC_ISW), access_dcsw },
+
+ SYS_INSN_TO_DESC(AT_S1E1R, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1W, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E0R, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E0W, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1RP, handle_s1e01, forward_at_traps),
+ SYS_INSN_TO_DESC(AT_S1E1WP, handle_s1e01, forward_at_traps),
+
{ SYS_DESC(SYS_DC_CSW), access_dcsw },
{ SYS_DESC(SYS_DC_CISW), access_dcsw },
+
+ SYS_INSN_TO_DESC(AT_S1E2R, handle_s1e2, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S1E2W, handle_s1e2, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E1R, handle_s12r, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E1W, handle_s12w, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E0R, handle_s12r, forward_nv_traps),
+ SYS_INSN_TO_DESC(AT_S12E0W, handle_s12w, forward_nv_traps),
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 41/66] KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team,
Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
When supporting nested virtualization a guest hypervisor executing TLBI
instructions must be trapped and emulated by the host hypervisor,
because the guest hypervisor can only affect physical TLB entries
relating to its own execution environment (virtual EL2 in EL1) but not
to the nested guests as required by the semantics of the instructions
and TLBI instructions might also result in updates (invalidations) to
shadow page tables.
This patch does several things.
1. List and define all TLBI system instructions to emulate.
2. Emulate TLBI ALLE2(IS) instruction executed in the virtual EL2. Since
we emulate the virtual EL2 in the EL1, we invalidate EL1&0 regime stage
1 TLB entries with setting vttbr_el2 having the VMID of the virtual EL2.
3. Emulate TLBI VAE2* instruction executed in the virtual EL2. Based on the
same principle as TLBI ALLE2 instruction, we can simply emulate those
instructions by executing corresponding VAE1* instructions with the
virtual EL2's VMID assigned by the host hypervisor.
Note that we are able to emulate TLBI ALLE2IS precisely by only
invalidating stage 1 TLB entries via TLBI VMALL1IS instruction, but to
make it simeple, we reuse the existing function, __kvm_tlb_flush_vmid(),
which invalidates both of stage 1 and 2 TLB entries.
4. TLBI ALLE1(IS) instruction invalidates all EL1&0 regime stage 1 and 2
TLB entries (on all PEs in the same Inner Shareable domain). To emulate
these instructions, we first need to clear all the mappings in the
shadow page tables since executing those instructions implies the change
of mappings in the stage 2 page tables maintained by the guest
hypervisor. We then need to invalidate all EL1&0 regime stage 1 and 2
TLB entries of all VMIDs, which are assigned by the host hypervisor, for
this VM.
5. Based on the same principle as TLBI ALLE1(IS) emulation, we clear the
mappings in the shadow stage-2 page tables and invalidate TLB entries.
But this time we do it only for the current VMID from the guest
hypervisor's perspective, not for all VMIDs.
6. Based on the same principle as TLBI ALLE1(IS) and TLBI VMALLS12E1(IS)
emulation, we clear the mappings in the shadow stage-2 page tables and
invalidate TLB entries. We do it only for one mapping for the current
VMID from the guest hypervisor's view.
7. Forward system instruction traps to the virtual EL2 if a
corresponding bit in the virtual HCR_EL2 is set.
8. Even though a guest hypervisor can execute TLBI instructions that are
accesible at EL1 without trap, it's wrong; All those TLBI instructions
work based on current VMID, and when running a guest hypervisor current
VMID is the one for itself, not the one from the virtual vttbr_el2. So
letting a guest hypervisor execute those TLBI instructions results in
invalidating its own TLB entries and leaving invalid TLB entries
unhandled.
Therefore we trap and emulate those TLBI instructions. The emulation is
simple; we find a shadow VMID mapped to the virtual vttbr_el2, set it in
the physical vttbr_el2, then execute the same instruction in EL2.
We don't set HCR_EL2.TTLB bit yet.
[ Changes performed by Marc Zynger:
The TLBI handling code more or less directly execute the same
instruction that has been trapped (with an EL2->EL1 conversion
in the case of an EL2 TLBI), but that's unfortunately not enough:
- TLBIs must be upgraded to the Inner Shareable domain to account
for vcpu migration, just like we already have with HCR_EL2.FB.
- The DSB instruction that synchronises these must thus be on
the Inner Shareable domain as well.
- Prior to executing the TLBI, we need another DSB ISHST to make
sure that the update to the page tables is now visible.
Ordering of system instructions fixed
- The current TLB invalidation code is pretty buggy, as it assume a
page mapping. On the contrary, it is likely that TLB invalidation
will cover more than a single page, and the size should be decided
by the guests configuration (and not the host's).
Since we don't cache the guest mapping sizes in the shadow PT yet,
let's assume the worse case (a block mapping) and invalidate that.
Take this opportunity to fix the decoding of the parameter (it
isn't a straight IPA).
- In general, we always emulate local TBL invalidations as being
as upgraded to the Inner Shareable domain so that we can easily
deal with vcpu migration. This is consistent with the fact that
we set HCR_EL2.FB when running non-nested VMs.
So let's emulate TLBI ALLE2 as ALLE2IS.
]
[ Changes performed by Christoffer Dall:
Sometimes when we are invalidating the TLB for a certain S2 MMU
context, this context can also have EL2 context associated with it
and we have to invalidate this too.
]
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/sysreg.h | 36 ++++++
arch/arm64/kvm/hyp/vhe/switch.c | 8 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 81 ++++++++++++
arch/arm64/kvm/mmu.c | 18 ++-
arch/arm64/kvm/sys_regs.c | 212 +++++++++++++++++++++++++++++++
6 files changed, 352 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 8bda33d026ca..2fae66fb2c86 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -187,6 +187,8 @@ extern void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa,
int level);
extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu);
+extern void __kvm_tlb_vae2is(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding);
+extern void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding);
extern void __kvm_timer_set_cntvoff(u64 cntvoff);
extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 4131a0c5df6e..814fb57b55d9 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -608,6 +608,42 @@
#define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6)
#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)
+/* TLBI instructions */
+#define TLBI_Op0 1
+#define TLBI_Op1_EL1 0 /* Accessible from EL1 or higher */
+#define TLBI_Op1_EL2 4 /* Accessible from EL2 or higher */
+#define TLBI_CRn 8
+#define tlbi_insn_el1(CRm, Op2) sys_insn(TLBI_Op0, TLBI_Op1_EL1, TLBI_CRn, (CRm), (Op2))
+#define tlbi_insn_el2(CRm, Op2) sys_insn(TLBI_Op0, TLBI_Op1_EL2, TLBI_CRn, (CRm), (Op2))
+
+#define OP_TLBI_VMALLE1IS tlbi_insn_el1(3, 0)
+#define OP_TLBI_VAE1IS tlbi_insn_el1(3, 1)
+#define OP_TLBI_ASIDE1IS tlbi_insn_el1(3, 2)
+#define OP_TLBI_VAAE1IS tlbi_insn_el1(3, 3)
+#define OP_TLBI_VALE1IS tlbi_insn_el1(3, 5)
+#define OP_TLBI_VAALE1IS tlbi_insn_el1(3, 7)
+#define OP_TLBI_VMALLE1 tlbi_insn_el1(7, 0)
+#define OP_TLBI_VAE1 tlbi_insn_el1(7, 1)
+#define OP_TLBI_ASIDE1 tlbi_insn_el1(7, 2)
+#define OP_TLBI_VAAE1 tlbi_insn_el1(7, 3)
+#define OP_TLBI_VALE1 tlbi_insn_el1(7, 5)
+#define OP_TLBI_VAALE1 tlbi_insn_el1(7, 7)
+
+#define OP_TLBI_IPAS2E1IS tlbi_insn_el2(0, 1)
+#define OP_TLBI_IPAS2LE1IS tlbi_insn_el2(0, 5)
+#define OP_TLBI_ALLE2IS tlbi_insn_el2(3, 0)
+#define OP_TLBI_VAE2IS tlbi_insn_el2(3, 1)
+#define OP_TLBI_ALLE1IS tlbi_insn_el2(3, 4)
+#define OP_TLBI_VALE2IS tlbi_insn_el2(3, 5)
+#define OP_TLBI_VMALLS12E1IS tlbi_insn_el2(3, 6)
+#define OP_TLBI_IPAS2E1 tlbi_insn_el2(4, 1)
+#define OP_TLBI_IPAS2LE1 tlbi_insn_el2(4, 5)
+#define OP_TLBI_ALLE2 tlbi_insn_el2(7, 0)
+#define OP_TLBI_VAE2 tlbi_insn_el2(7, 1)
+#define OP_TLBI_ALLE1 tlbi_insn_el2(7, 4)
+#define OP_TLBI_VALE2 tlbi_insn_el2(7, 5)
+#define OP_TLBI_VMALLS12E1 tlbi_insn_el2(7, 6)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 396d329e015d..bac6ccc2595c 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -48,7 +48,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
* the EL1 virtual memory control register accesses
* as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_NV1;
+ hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_TTLB | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -74,11 +74,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
/*
* If we're using the EL1 translation regime
- * (TGE clear), then ensure that AT S1 ops are
- * trapped too.
+ * (TGE clear), then ensure that AT S1 and
+ * TLBI E1 ops are trapped too.
*/
if (!vcpu_el2_tge_is_set(vcpu))
- hcr |= HCR_AT;
+ hcr |= HCR_AT | HCR_TTLB;
}
}
diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c
index fd7895945bbc..52fda1d61308 100644
--- a/arch/arm64/kvm/hyp/vhe/tlb.c
+++ b/arch/arm64/kvm/hyp/vhe/tlb.c
@@ -160,3 +160,84 @@ void __kvm_flush_vm_context(void)
dsb(ish);
}
+
+void __kvm_tlb_vae2is(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)
+{
+ struct tlb_inv_context cxt;
+
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ __tlb_switch_to_guest(mmu, &cxt);
+
+ /*
+ * Execute the EL1 version of TLBI VAE2* instruction, forcing
+ * an upgrade to the Inner Shareable domain in order to
+ * perform the invalidation on all CPUs.
+ */
+ switch (sys_encoding) {
+ case OP_TLBI_VAE2:
+ case OP_TLBI_VAE2IS:
+ __tlbi(vae1is, va);
+ break;
+ case OP_TLBI_VALE2:
+ case OP_TLBI_VALE2IS:
+ __tlbi(vale1is, va);
+ break;
+ default:
+ break;
+ }
+ dsb(ish);
+ isb();
+
+ __tlb_switch_to_host(&cxt);
+}
+
+void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
+{
+ struct tlb_inv_context cxt;
+
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ __tlb_switch_to_guest(mmu, &cxt);
+
+ /*
+ * Execute the same instruction as the guest hypervisor did,
+ * expanding the scope of local TLB invalidations to the Inner
+ * Shareable domain so that it takes place on all CPUs. This
+ * is equivalent to having HCR_EL2.FB set.
+ */
+ switch (sys_encoding) {
+ case OP_TLBI_VMALLE1:
+ case OP_TLBI_VMALLE1IS:
+ __tlbi(vmalle1is);
+ break;
+ case OP_TLBI_VAE1:
+ case OP_TLBI_VAE1IS:
+ __tlbi(vae1is, val);
+ break;
+ case OP_TLBI_ASIDE1:
+ case OP_TLBI_ASIDE1IS:
+ __tlbi(aside1is, val);
+ break;
+ case OP_TLBI_VAAE1:
+ case OP_TLBI_VAAE1IS:
+ __tlbi(vaae1is, val);
+ break;
+ case OP_TLBI_VALE1:
+ case OP_TLBI_VALE1IS:
+ __tlbi(vale1is, val);
+ break;
+ case OP_TLBI_VAALE1:
+ case OP_TLBI_VAALE1IS:
+ __tlbi(vaale1is, val);
+ break;
+ default:
+ break;
+ }
+ dsb(ish);
+ isb();
+
+ __tlb_switch_to_host(&cxt);
+}
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 36cb9fa22153..5fb13995190e 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -80,7 +80,23 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
*/
void kvm_flush_remote_tlbs(struct kvm *kvm)
{
- kvm_call_hyp(__kvm_tlb_flush_vmid, &kvm->arch.mmu);
+ struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
+
+ if (mmu == &kvm->arch.mmu) {
+ /*
+ * For a normal (i.e. non-nested) guest, flush entries for the
+ * given VMID *
+ */
+ kvm_call_hyp(__kvm_tlb_flush_vmid, mmu);
+ } else {
+ /*
+ * When supporting nested virtualization, we can have multiple
+ * VMIDs in play for each VCPU in the VM, so it's really not
+ * worth it to try to quiesce the system and flush all the
+ * VMIDs that may be in use, instead just nuke the whole thing.
+ */
+ kvm_call_hyp(__kvm_flush_vm_context);
+ }
}
static bool kvm_is_device_pfn(unsigned long pfn)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6b7a7ce9b1a3..ecd61a6b06d1 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1623,6 +1623,11 @@ static bool forward_at_traps(struct kvm_vcpu *vcpu)
return forward_traps(vcpu, HCR_AT);
}
+static bool forward_ttlb_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_TTLB);
+}
+
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -2230,6 +2235,185 @@ static bool handle_s12w(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return handle_s12(vcpu, p, r, true);
}
+static bool handle_alle2is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ /*
+ * To emulate invalidating all EL2 regime stage 1 TLB entries for all
+ * PEs, executing TLBI VMALLE1IS is enough. But reuse the existing
+ * interface for the simplicity; invalidating stage 2 entries doesn't
+ * affect the correctness.
+ */
+ __kvm_tlb_flush_vmid(&vcpu->kvm->arch.mmu);
+ return true;
+}
+
+static bool handle_vae2is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ /*
+ * Based on the same principle as TLBI ALLE2 instruction
+ * emulation, we emulate TLBI VAE2* instructions by executing
+ * corresponding TLBI VAE1* instructions with the virtual
+ * EL2's VMID assigned by the host hypervisor.
+ */
+ __kvm_tlb_vae2is(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
+ return true;
+}
+
+static bool handle_alle1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct kvm_s2_mmu *mmu = &vcpu->kvm->arch.mmu;
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ /*
+ * Clear all mappings in the shadow page tables and invalidate the stage
+ * 1 and 2 TLB entries via kvm_tlb_flush_vmid_ipa().
+ */
+ kvm_nested_s2_clear(vcpu->kvm);
+
+ if (mmu->vmid.vmid_gen) {
+ /*
+ * Invalidate the stage 1 and 2 TLB entries for the host OS
+ * in a VM only if there is one.
+ */
+ __kvm_tlb_flush_vmid(mmu);
+ }
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(vcpu->kvm));
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(vcpu->kvm));
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct kvm_s2_mmu *mmu;
+ u64 base_addr;
+ int max_size;
+
+ /*
+ * We drop a number of things from the supplied value:
+ *
+ * - NS bit: we're non-secure only.
+ *
+ * - TTL field: We already have the granule size from the
+ * VTCR_EL2.TG0 field, and the level is only relevant to the
+ * guest's S2PT.
+ *
+ * - IPA[51:48]: We don't support 52bit IPA just yet...
+ *
+ * And of course, adjust the IPA to be on an actual address.
+ */
+ base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
+
+ /* Compute the maximum extent of the invalidation */
+ switch ((vtcr & VTCR_EL2_TG0_MASK)) {
+ case VTCR_EL2_TG0_4K:
+ max_size = SZ_1G;
+ break;
+ case VTCR_EL2_TG0_16K:
+ max_size = SZ_32M;
+ break;
+ case VTCR_EL2_TG0_64K:
+ /*
+ * No, we do not support 52bit IPA in nested yet. Once
+ * we do, this should be 4TB.
+ */
+ /* FIXME: remove the 52bit PA support from the IDregs */
+ max_size = SZ_512M;
+ break;
+ default:
+ BUG();
+ }
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, base_addr, max_size);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, base_addr, max_size);
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u32 sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ /*
+ * If we're here, this is because we've trapped on a EL1 TLBI
+ * instruction that affects the EL1 translation regime while
+ * we're running in a context that doesn't allow us to let the
+ * HW do its thing (aka vEL2):
+ *
+ * - HCR_EL2.E2H == 0 : a non-VHE guest
+ * - HCR_EL2.{E2H,TGE} == { 1, 0 } : a VHE guest in guest mode
+ *
+ * We don't expect these helpers to ever be called when running
+ * in a vEL1 context.
+ */
+
+ WARN_ON(!vcpu_mode_el2(vcpu));
+
+ mutex_lock(&vcpu->kvm->lock);
+
+ if ((__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
+ u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
+
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+ } else {
+ /*
+ * ARMv8.4-NV allows the guest to change TGE behind
+ * our back, so we always trap EL1 TLBIs from vEL2...
+ */
+ __kvm_tlb_el1_instr(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
+ }
+
+ mutex_unlock(&vcpu->kvm->lock);
+
+ return true;
+}
+
/*
* AT instruction emulation
*
@@ -2312,12 +2496,40 @@ static struct sys_reg_desc sys_insn_descs[] = {
{ SYS_DESC(SYS_DC_CSW), access_dcsw },
{ SYS_DESC(SYS_DC_CISW), access_dcsw },
+ SYS_INSN_TO_DESC(TLBI_VMALLE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_ASIDE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAAE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAALE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_ASIDE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAAE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAALE1, handle_tlbi_el1, forward_ttlb_traps),
+
SYS_INSN_TO_DESC(AT_S1E2R, handle_s1e2, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S1E2W, handle_s1e2, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E1R, handle_s12r, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E1W, handle_s12w, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E0R, handle_s12r, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E0W, handle_s12w, forward_nv_traps),
+
+ SYS_INSN_TO_DESC(TLBI_IPAS2E1IS, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2LE1IS, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE2IS, handle_alle2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE2IS, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE1IS, handle_alle1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE2IS, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLS12E1IS, handle_vmalls12e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2E1, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2LE1, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE2, handle_alle2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE2, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE1, handle_alle1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE2, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLS12E1, handle_vmalls12e1is, forward_nv_traps),
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 41/66] KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara, Jintack Lim
From: Jintack Lim <jintack.lim@linaro.org>
When supporting nested virtualization a guest hypervisor executing TLBI
instructions must be trapped and emulated by the host hypervisor,
because the guest hypervisor can only affect physical TLB entries
relating to its own execution environment (virtual EL2 in EL1) but not
to the nested guests as required by the semantics of the instructions
and TLBI instructions might also result in updates (invalidations) to
shadow page tables.
This patch does several things.
1. List and define all TLBI system instructions to emulate.
2. Emulate TLBI ALLE2(IS) instruction executed in the virtual EL2. Since
we emulate the virtual EL2 in the EL1, we invalidate EL1&0 regime stage
1 TLB entries with setting vttbr_el2 having the VMID of the virtual EL2.
3. Emulate TLBI VAE2* instruction executed in the virtual EL2. Based on the
same principle as TLBI ALLE2 instruction, we can simply emulate those
instructions by executing corresponding VAE1* instructions with the
virtual EL2's VMID assigned by the host hypervisor.
Note that we are able to emulate TLBI ALLE2IS precisely by only
invalidating stage 1 TLB entries via TLBI VMALL1IS instruction, but to
make it simeple, we reuse the existing function, __kvm_tlb_flush_vmid(),
which invalidates both of stage 1 and 2 TLB entries.
4. TLBI ALLE1(IS) instruction invalidates all EL1&0 regime stage 1 and 2
TLB entries (on all PEs in the same Inner Shareable domain). To emulate
these instructions, we first need to clear all the mappings in the
shadow page tables since executing those instructions implies the change
of mappings in the stage 2 page tables maintained by the guest
hypervisor. We then need to invalidate all EL1&0 regime stage 1 and 2
TLB entries of all VMIDs, which are assigned by the host hypervisor, for
this VM.
5. Based on the same principle as TLBI ALLE1(IS) emulation, we clear the
mappings in the shadow stage-2 page tables and invalidate TLB entries.
But this time we do it only for the current VMID from the guest
hypervisor's perspective, not for all VMIDs.
6. Based on the same principle as TLBI ALLE1(IS) and TLBI VMALLS12E1(IS)
emulation, we clear the mappings in the shadow stage-2 page tables and
invalidate TLB entries. We do it only for one mapping for the current
VMID from the guest hypervisor's view.
7. Forward system instruction traps to the virtual EL2 if a
corresponding bit in the virtual HCR_EL2 is set.
8. Even though a guest hypervisor can execute TLBI instructions that are
accesible at EL1 without trap, it's wrong; All those TLBI instructions
work based on current VMID, and when running a guest hypervisor current
VMID is the one for itself, not the one from the virtual vttbr_el2. So
letting a guest hypervisor execute those TLBI instructions results in
invalidating its own TLB entries and leaving invalid TLB entries
unhandled.
Therefore we trap and emulate those TLBI instructions. The emulation is
simple; we find a shadow VMID mapped to the virtual vttbr_el2, set it in
the physical vttbr_el2, then execute the same instruction in EL2.
We don't set HCR_EL2.TTLB bit yet.
[ Changes performed by Marc Zynger:
The TLBI handling code more or less directly execute the same
instruction that has been trapped (with an EL2->EL1 conversion
in the case of an EL2 TLBI), but that's unfortunately not enough:
- TLBIs must be upgraded to the Inner Shareable domain to account
for vcpu migration, just like we already have with HCR_EL2.FB.
- The DSB instruction that synchronises these must thus be on
the Inner Shareable domain as well.
- Prior to executing the TLBI, we need another DSB ISHST to make
sure that the update to the page tables is now visible.
Ordering of system instructions fixed
- The current TLB invalidation code is pretty buggy, as it assume a
page mapping. On the contrary, it is likely that TLB invalidation
will cover more than a single page, and the size should be decided
by the guests configuration (and not the host's).
Since we don't cache the guest mapping sizes in the shadow PT yet,
let's assume the worse case (a block mapping) and invalidate that.
Take this opportunity to fix the decoding of the parameter (it
isn't a straight IPA).
- In general, we always emulate local TBL invalidations as being
as upgraded to the Inner Shareable domain so that we can easily
deal with vcpu migration. This is consistent with the fact that
we set HCR_EL2.FB when running non-nested VMs.
So let's emulate TLBI ALLE2 as ALLE2IS.
]
[ Changes performed by Christoffer Dall:
Sometimes when we are invalidating the TLB for a certain S2 MMU
context, this context can also have EL2 context associated with it
and we have to invalidate this too.
]
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/sysreg.h | 36 ++++++
arch/arm64/kvm/hyp/vhe/switch.c | 8 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 81 ++++++++++++
arch/arm64/kvm/mmu.c | 18 ++-
arch/arm64/kvm/sys_regs.c | 212 +++++++++++++++++++++++++++++++
6 files changed, 352 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 8bda33d026ca..2fae66fb2c86 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -187,6 +187,8 @@ extern void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa,
int level);
extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu);
+extern void __kvm_tlb_vae2is(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding);
+extern void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding);
extern void __kvm_timer_set_cntvoff(u64 cntvoff);
extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 4131a0c5df6e..814fb57b55d9 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -608,6 +608,42 @@
#define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6)
#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)
+/* TLBI instructions */
+#define TLBI_Op0 1
+#define TLBI_Op1_EL1 0 /* Accessible from EL1 or higher */
+#define TLBI_Op1_EL2 4 /* Accessible from EL2 or higher */
+#define TLBI_CRn 8
+#define tlbi_insn_el1(CRm, Op2) sys_insn(TLBI_Op0, TLBI_Op1_EL1, TLBI_CRn, (CRm), (Op2))
+#define tlbi_insn_el2(CRm, Op2) sys_insn(TLBI_Op0, TLBI_Op1_EL2, TLBI_CRn, (CRm), (Op2))
+
+#define OP_TLBI_VMALLE1IS tlbi_insn_el1(3, 0)
+#define OP_TLBI_VAE1IS tlbi_insn_el1(3, 1)
+#define OP_TLBI_ASIDE1IS tlbi_insn_el1(3, 2)
+#define OP_TLBI_VAAE1IS tlbi_insn_el1(3, 3)
+#define OP_TLBI_VALE1IS tlbi_insn_el1(3, 5)
+#define OP_TLBI_VAALE1IS tlbi_insn_el1(3, 7)
+#define OP_TLBI_VMALLE1 tlbi_insn_el1(7, 0)
+#define OP_TLBI_VAE1 tlbi_insn_el1(7, 1)
+#define OP_TLBI_ASIDE1 tlbi_insn_el1(7, 2)
+#define OP_TLBI_VAAE1 tlbi_insn_el1(7, 3)
+#define OP_TLBI_VALE1 tlbi_insn_el1(7, 5)
+#define OP_TLBI_VAALE1 tlbi_insn_el1(7, 7)
+
+#define OP_TLBI_IPAS2E1IS tlbi_insn_el2(0, 1)
+#define OP_TLBI_IPAS2LE1IS tlbi_insn_el2(0, 5)
+#define OP_TLBI_ALLE2IS tlbi_insn_el2(3, 0)
+#define OP_TLBI_VAE2IS tlbi_insn_el2(3, 1)
+#define OP_TLBI_ALLE1IS tlbi_insn_el2(3, 4)
+#define OP_TLBI_VALE2IS tlbi_insn_el2(3, 5)
+#define OP_TLBI_VMALLS12E1IS tlbi_insn_el2(3, 6)
+#define OP_TLBI_IPAS2E1 tlbi_insn_el2(4, 1)
+#define OP_TLBI_IPAS2LE1 tlbi_insn_el2(4, 5)
+#define OP_TLBI_ALLE2 tlbi_insn_el2(7, 0)
+#define OP_TLBI_VAE2 tlbi_insn_el2(7, 1)
+#define OP_TLBI_ALLE1 tlbi_insn_el2(7, 4)
+#define OP_TLBI_VALE2 tlbi_insn_el2(7, 5)
+#define OP_TLBI_VMALLS12E1 tlbi_insn_el2(7, 6)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 396d329e015d..bac6ccc2595c 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -48,7 +48,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
* the EL1 virtual memory control register accesses
* as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_NV1;
+ hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_TTLB | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -74,11 +74,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
/*
* If we're using the EL1 translation regime
- * (TGE clear), then ensure that AT S1 ops are
- * trapped too.
+ * (TGE clear), then ensure that AT S1 and
+ * TLBI E1 ops are trapped too.
*/
if (!vcpu_el2_tge_is_set(vcpu))
- hcr |= HCR_AT;
+ hcr |= HCR_AT | HCR_TTLB;
}
}
diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c
index fd7895945bbc..52fda1d61308 100644
--- a/arch/arm64/kvm/hyp/vhe/tlb.c
+++ b/arch/arm64/kvm/hyp/vhe/tlb.c
@@ -160,3 +160,84 @@ void __kvm_flush_vm_context(void)
dsb(ish);
}
+
+void __kvm_tlb_vae2is(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)
+{
+ struct tlb_inv_context cxt;
+
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ __tlb_switch_to_guest(mmu, &cxt);
+
+ /*
+ * Execute the EL1 version of TLBI VAE2* instruction, forcing
+ * an upgrade to the Inner Shareable domain in order to
+ * perform the invalidation on all CPUs.
+ */
+ switch (sys_encoding) {
+ case OP_TLBI_VAE2:
+ case OP_TLBI_VAE2IS:
+ __tlbi(vae1is, va);
+ break;
+ case OP_TLBI_VALE2:
+ case OP_TLBI_VALE2IS:
+ __tlbi(vale1is, va);
+ break;
+ default:
+ break;
+ }
+ dsb(ish);
+ isb();
+
+ __tlb_switch_to_host(&cxt);
+}
+
+void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
+{
+ struct tlb_inv_context cxt;
+
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ __tlb_switch_to_guest(mmu, &cxt);
+
+ /*
+ * Execute the same instruction as the guest hypervisor did,
+ * expanding the scope of local TLB invalidations to the Inner
+ * Shareable domain so that it takes place on all CPUs. This
+ * is equivalent to having HCR_EL2.FB set.
+ */
+ switch (sys_encoding) {
+ case OP_TLBI_VMALLE1:
+ case OP_TLBI_VMALLE1IS:
+ __tlbi(vmalle1is);
+ break;
+ case OP_TLBI_VAE1:
+ case OP_TLBI_VAE1IS:
+ __tlbi(vae1is, val);
+ break;
+ case OP_TLBI_ASIDE1:
+ case OP_TLBI_ASIDE1IS:
+ __tlbi(aside1is, val);
+ break;
+ case OP_TLBI_VAAE1:
+ case OP_TLBI_VAAE1IS:
+ __tlbi(vaae1is, val);
+ break;
+ case OP_TLBI_VALE1:
+ case OP_TLBI_VALE1IS:
+ __tlbi(vale1is, val);
+ break;
+ case OP_TLBI_VAALE1:
+ case OP_TLBI_VAALE1IS:
+ __tlbi(vaale1is, val);
+ break;
+ default:
+ break;
+ }
+ dsb(ish);
+ isb();
+
+ __tlb_switch_to_host(&cxt);
+}
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 36cb9fa22153..5fb13995190e 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -80,7 +80,23 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
*/
void kvm_flush_remote_tlbs(struct kvm *kvm)
{
- kvm_call_hyp(__kvm_tlb_flush_vmid, &kvm->arch.mmu);
+ struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
+
+ if (mmu == &kvm->arch.mmu) {
+ /*
+ * For a normal (i.e. non-nested) guest, flush entries for the
+ * given VMID *
+ */
+ kvm_call_hyp(__kvm_tlb_flush_vmid, mmu);
+ } else {
+ /*
+ * When supporting nested virtualization, we can have multiple
+ * VMIDs in play for each VCPU in the VM, so it's really not
+ * worth it to try to quiesce the system and flush all the
+ * VMIDs that may be in use, instead just nuke the whole thing.
+ */
+ kvm_call_hyp(__kvm_flush_vm_context);
+ }
}
static bool kvm_is_device_pfn(unsigned long pfn)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6b7a7ce9b1a3..ecd61a6b06d1 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1623,6 +1623,11 @@ static bool forward_at_traps(struct kvm_vcpu *vcpu)
return forward_traps(vcpu, HCR_AT);
}
+static bool forward_ttlb_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_TTLB);
+}
+
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -2230,6 +2235,185 @@ static bool handle_s12w(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return handle_s12(vcpu, p, r, true);
}
+static bool handle_alle2is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ /*
+ * To emulate invalidating all EL2 regime stage 1 TLB entries for all
+ * PEs, executing TLBI VMALLE1IS is enough. But reuse the existing
+ * interface for the simplicity; invalidating stage 2 entries doesn't
+ * affect the correctness.
+ */
+ __kvm_tlb_flush_vmid(&vcpu->kvm->arch.mmu);
+ return true;
+}
+
+static bool handle_vae2is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ /*
+ * Based on the same principle as TLBI ALLE2 instruction
+ * emulation, we emulate TLBI VAE2* instructions by executing
+ * corresponding TLBI VAE1* instructions with the virtual
+ * EL2's VMID assigned by the host hypervisor.
+ */
+ __kvm_tlb_vae2is(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
+ return true;
+}
+
+static bool handle_alle1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct kvm_s2_mmu *mmu = &vcpu->kvm->arch.mmu;
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ /*
+ * Clear all mappings in the shadow page tables and invalidate the stage
+ * 1 and 2 TLB entries via kvm_tlb_flush_vmid_ipa().
+ */
+ kvm_nested_s2_clear(vcpu->kvm);
+
+ if (mmu->vmid.vmid_gen) {
+ /*
+ * Invalidate the stage 1 and 2 TLB entries for the host OS
+ * in a VM only if there is one.
+ */
+ __kvm_tlb_flush_vmid(mmu);
+ }
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(vcpu->kvm));
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(vcpu->kvm));
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct kvm_s2_mmu *mmu;
+ u64 base_addr;
+ int max_size;
+
+ /*
+ * We drop a number of things from the supplied value:
+ *
+ * - NS bit: we're non-secure only.
+ *
+ * - TTL field: We already have the granule size from the
+ * VTCR_EL2.TG0 field, and the level is only relevant to the
+ * guest's S2PT.
+ *
+ * - IPA[51:48]: We don't support 52bit IPA just yet...
+ *
+ * And of course, adjust the IPA to be on an actual address.
+ */
+ base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
+
+ /* Compute the maximum extent of the invalidation */
+ switch ((vtcr & VTCR_EL2_TG0_MASK)) {
+ case VTCR_EL2_TG0_4K:
+ max_size = SZ_1G;
+ break;
+ case VTCR_EL2_TG0_16K:
+ max_size = SZ_32M;
+ break;
+ case VTCR_EL2_TG0_64K:
+ /*
+ * No, we do not support 52bit IPA in nested yet. Once
+ * we do, this should be 4TB.
+ */
+ /* FIXME: remove the 52bit PA support from the IDregs */
+ max_size = SZ_512M;
+ break;
+ default:
+ BUG();
+ }
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, base_addr, max_size);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, base_addr, max_size);
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u32 sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ /*
+ * If we're here, this is because we've trapped on a EL1 TLBI
+ * instruction that affects the EL1 translation regime while
+ * we're running in a context that doesn't allow us to let the
+ * HW do its thing (aka vEL2):
+ *
+ * - HCR_EL2.E2H == 0 : a non-VHE guest
+ * - HCR_EL2.{E2H,TGE} == { 1, 0 } : a VHE guest in guest mode
+ *
+ * We don't expect these helpers to ever be called when running
+ * in a vEL1 context.
+ */
+
+ WARN_ON(!vcpu_mode_el2(vcpu));
+
+ mutex_lock(&vcpu->kvm->lock);
+
+ if ((__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
+ u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
+
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+ } else {
+ /*
+ * ARMv8.4-NV allows the guest to change TGE behind
+ * our back, so we always trap EL1 TLBIs from vEL2...
+ */
+ __kvm_tlb_el1_instr(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
+ }
+
+ mutex_unlock(&vcpu->kvm->lock);
+
+ return true;
+}
+
/*
* AT instruction emulation
*
@@ -2312,12 +2496,40 @@ static struct sys_reg_desc sys_insn_descs[] = {
{ SYS_DESC(SYS_DC_CSW), access_dcsw },
{ SYS_DESC(SYS_DC_CISW), access_dcsw },
+ SYS_INSN_TO_DESC(TLBI_VMALLE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_ASIDE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAAE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAALE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_ASIDE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAAE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAALE1, handle_tlbi_el1, forward_ttlb_traps),
+
SYS_INSN_TO_DESC(AT_S1E2R, handle_s1e2, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S1E2W, handle_s1e2, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E1R, handle_s12r, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E1W, handle_s12w, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E0R, handle_s12r, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E0W, handle_s12w, forward_nv_traps),
+
+ SYS_INSN_TO_DESC(TLBI_IPAS2E1IS, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2LE1IS, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE2IS, handle_alle2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE2IS, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE1IS, handle_alle1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE2IS, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLS12E1IS, handle_vmalls12e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2E1, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2LE1, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE2, handle_alle2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE2, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE1, handle_alle1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE2, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLS12E1, handle_vmalls12e1is, forward_nv_traps),
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 41/66] KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Jintack Lim, Alexandru Elisei, Jintack Lim,
Julien Thierry
From: Jintack Lim <jintack.lim@linaro.org>
When supporting nested virtualization a guest hypervisor executing TLBI
instructions must be trapped and emulated by the host hypervisor,
because the guest hypervisor can only affect physical TLB entries
relating to its own execution environment (virtual EL2 in EL1) but not
to the nested guests as required by the semantics of the instructions
and TLBI instructions might also result in updates (invalidations) to
shadow page tables.
This patch does several things.
1. List and define all TLBI system instructions to emulate.
2. Emulate TLBI ALLE2(IS) instruction executed in the virtual EL2. Since
we emulate the virtual EL2 in the EL1, we invalidate EL1&0 regime stage
1 TLB entries with setting vttbr_el2 having the VMID of the virtual EL2.
3. Emulate TLBI VAE2* instruction executed in the virtual EL2. Based on the
same principle as TLBI ALLE2 instruction, we can simply emulate those
instructions by executing corresponding VAE1* instructions with the
virtual EL2's VMID assigned by the host hypervisor.
Note that we are able to emulate TLBI ALLE2IS precisely by only
invalidating stage 1 TLB entries via TLBI VMALL1IS instruction, but to
make it simeple, we reuse the existing function, __kvm_tlb_flush_vmid(),
which invalidates both of stage 1 and 2 TLB entries.
4. TLBI ALLE1(IS) instruction invalidates all EL1&0 regime stage 1 and 2
TLB entries (on all PEs in the same Inner Shareable domain). To emulate
these instructions, we first need to clear all the mappings in the
shadow page tables since executing those instructions implies the change
of mappings in the stage 2 page tables maintained by the guest
hypervisor. We then need to invalidate all EL1&0 regime stage 1 and 2
TLB entries of all VMIDs, which are assigned by the host hypervisor, for
this VM.
5. Based on the same principle as TLBI ALLE1(IS) emulation, we clear the
mappings in the shadow stage-2 page tables and invalidate TLB entries.
But this time we do it only for the current VMID from the guest
hypervisor's perspective, not for all VMIDs.
6. Based on the same principle as TLBI ALLE1(IS) and TLBI VMALLS12E1(IS)
emulation, we clear the mappings in the shadow stage-2 page tables and
invalidate TLB entries. We do it only for one mapping for the current
VMID from the guest hypervisor's view.
7. Forward system instruction traps to the virtual EL2 if a
corresponding bit in the virtual HCR_EL2 is set.
8. Even though a guest hypervisor can execute TLBI instructions that are
accesible at EL1 without trap, it's wrong; All those TLBI instructions
work based on current VMID, and when running a guest hypervisor current
VMID is the one for itself, not the one from the virtual vttbr_el2. So
letting a guest hypervisor execute those TLBI instructions results in
invalidating its own TLB entries and leaving invalid TLB entries
unhandled.
Therefore we trap and emulate those TLBI instructions. The emulation is
simple; we find a shadow VMID mapped to the virtual vttbr_el2, set it in
the physical vttbr_el2, then execute the same instruction in EL2.
We don't set HCR_EL2.TTLB bit yet.
[ Changes performed by Marc Zynger:
The TLBI handling code more or less directly execute the same
instruction that has been trapped (with an EL2->EL1 conversion
in the case of an EL2 TLBI), but that's unfortunately not enough:
- TLBIs must be upgraded to the Inner Shareable domain to account
for vcpu migration, just like we already have with HCR_EL2.FB.
- The DSB instruction that synchronises these must thus be on
the Inner Shareable domain as well.
- Prior to executing the TLBI, we need another DSB ISHST to make
sure that the update to the page tables is now visible.
Ordering of system instructions fixed
- The current TLB invalidation code is pretty buggy, as it assume a
page mapping. On the contrary, it is likely that TLB invalidation
will cover more than a single page, and the size should be decided
by the guests configuration (and not the host's).
Since we don't cache the guest mapping sizes in the shadow PT yet,
let's assume the worse case (a block mapping) and invalidate that.
Take this opportunity to fix the decoding of the parameter (it
isn't a straight IPA).
- In general, we always emulate local TBL invalidations as being
as upgraded to the Inner Shareable domain so that we can easily
deal with vcpu migration. This is consistent with the fact that
we set HCR_EL2.FB when running non-nested VMs.
So let's emulate TLBI ALLE2 as ALLE2IS.
]
[ Changes performed by Christoffer Dall:
Sometimes when we are invalidating the TLB for a certain S2 MMU
context, this context can also have EL2 context associated with it
and we have to invalidate this too.
]
Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/sysreg.h | 36 ++++++
arch/arm64/kvm/hyp/vhe/switch.c | 8 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 81 ++++++++++++
arch/arm64/kvm/mmu.c | 18 ++-
arch/arm64/kvm/sys_regs.c | 212 +++++++++++++++++++++++++++++++
6 files changed, 352 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 8bda33d026ca..2fae66fb2c86 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -187,6 +187,8 @@ extern void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa,
int level);
extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu);
extern void __kvm_tlb_flush_local_vmid(struct kvm_s2_mmu *mmu);
+extern void __kvm_tlb_vae2is(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding);
+extern void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding);
extern void __kvm_timer_set_cntvoff(u64 cntvoff);
extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 4131a0c5df6e..814fb57b55d9 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -608,6 +608,42 @@
#define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6)
#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)
+/* TLBI instructions */
+#define TLBI_Op0 1
+#define TLBI_Op1_EL1 0 /* Accessible from EL1 or higher */
+#define TLBI_Op1_EL2 4 /* Accessible from EL2 or higher */
+#define TLBI_CRn 8
+#define tlbi_insn_el1(CRm, Op2) sys_insn(TLBI_Op0, TLBI_Op1_EL1, TLBI_CRn, (CRm), (Op2))
+#define tlbi_insn_el2(CRm, Op2) sys_insn(TLBI_Op0, TLBI_Op1_EL2, TLBI_CRn, (CRm), (Op2))
+
+#define OP_TLBI_VMALLE1IS tlbi_insn_el1(3, 0)
+#define OP_TLBI_VAE1IS tlbi_insn_el1(3, 1)
+#define OP_TLBI_ASIDE1IS tlbi_insn_el1(3, 2)
+#define OP_TLBI_VAAE1IS tlbi_insn_el1(3, 3)
+#define OP_TLBI_VALE1IS tlbi_insn_el1(3, 5)
+#define OP_TLBI_VAALE1IS tlbi_insn_el1(3, 7)
+#define OP_TLBI_VMALLE1 tlbi_insn_el1(7, 0)
+#define OP_TLBI_VAE1 tlbi_insn_el1(7, 1)
+#define OP_TLBI_ASIDE1 tlbi_insn_el1(7, 2)
+#define OP_TLBI_VAAE1 tlbi_insn_el1(7, 3)
+#define OP_TLBI_VALE1 tlbi_insn_el1(7, 5)
+#define OP_TLBI_VAALE1 tlbi_insn_el1(7, 7)
+
+#define OP_TLBI_IPAS2E1IS tlbi_insn_el2(0, 1)
+#define OP_TLBI_IPAS2LE1IS tlbi_insn_el2(0, 5)
+#define OP_TLBI_ALLE2IS tlbi_insn_el2(3, 0)
+#define OP_TLBI_VAE2IS tlbi_insn_el2(3, 1)
+#define OP_TLBI_ALLE1IS tlbi_insn_el2(3, 4)
+#define OP_TLBI_VALE2IS tlbi_insn_el2(3, 5)
+#define OP_TLBI_VMALLS12E1IS tlbi_insn_el2(3, 6)
+#define OP_TLBI_IPAS2E1 tlbi_insn_el2(4, 1)
+#define OP_TLBI_IPAS2LE1 tlbi_insn_el2(4, 5)
+#define OP_TLBI_ALLE2 tlbi_insn_el2(7, 0)
+#define OP_TLBI_VAE2 tlbi_insn_el2(7, 1)
+#define OP_TLBI_ALLE1 tlbi_insn_el2(7, 4)
+#define OP_TLBI_VALE2 tlbi_insn_el2(7, 5)
+#define OP_TLBI_VMALLS12E1 tlbi_insn_el2(7, 6)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (BIT(44))
#define SCTLR_ELx_ATA (BIT(43))
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 396d329e015d..bac6ccc2595c 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -48,7 +48,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
* the EL1 virtual memory control register accesses
* as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_NV1;
+ hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_TTLB | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -74,11 +74,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
/*
* If we're using the EL1 translation regime
- * (TGE clear), then ensure that AT S1 ops are
- * trapped too.
+ * (TGE clear), then ensure that AT S1 and
+ * TLBI E1 ops are trapped too.
*/
if (!vcpu_el2_tge_is_set(vcpu))
- hcr |= HCR_AT;
+ hcr |= HCR_AT | HCR_TTLB;
}
}
diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c
index fd7895945bbc..52fda1d61308 100644
--- a/arch/arm64/kvm/hyp/vhe/tlb.c
+++ b/arch/arm64/kvm/hyp/vhe/tlb.c
@@ -160,3 +160,84 @@ void __kvm_flush_vm_context(void)
dsb(ish);
}
+
+void __kvm_tlb_vae2is(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)
+{
+ struct tlb_inv_context cxt;
+
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ __tlb_switch_to_guest(mmu, &cxt);
+
+ /*
+ * Execute the EL1 version of TLBI VAE2* instruction, forcing
+ * an upgrade to the Inner Shareable domain in order to
+ * perform the invalidation on all CPUs.
+ */
+ switch (sys_encoding) {
+ case OP_TLBI_VAE2:
+ case OP_TLBI_VAE2IS:
+ __tlbi(vae1is, va);
+ break;
+ case OP_TLBI_VALE2:
+ case OP_TLBI_VALE2IS:
+ __tlbi(vale1is, va);
+ break;
+ default:
+ break;
+ }
+ dsb(ish);
+ isb();
+
+ __tlb_switch_to_host(&cxt);
+}
+
+void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
+{
+ struct tlb_inv_context cxt;
+
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ __tlb_switch_to_guest(mmu, &cxt);
+
+ /*
+ * Execute the same instruction as the guest hypervisor did,
+ * expanding the scope of local TLB invalidations to the Inner
+ * Shareable domain so that it takes place on all CPUs. This
+ * is equivalent to having HCR_EL2.FB set.
+ */
+ switch (sys_encoding) {
+ case OP_TLBI_VMALLE1:
+ case OP_TLBI_VMALLE1IS:
+ __tlbi(vmalle1is);
+ break;
+ case OP_TLBI_VAE1:
+ case OP_TLBI_VAE1IS:
+ __tlbi(vae1is, val);
+ break;
+ case OP_TLBI_ASIDE1:
+ case OP_TLBI_ASIDE1IS:
+ __tlbi(aside1is, val);
+ break;
+ case OP_TLBI_VAAE1:
+ case OP_TLBI_VAAE1IS:
+ __tlbi(vaae1is, val);
+ break;
+ case OP_TLBI_VALE1:
+ case OP_TLBI_VALE1IS:
+ __tlbi(vale1is, val);
+ break;
+ case OP_TLBI_VAALE1:
+ case OP_TLBI_VAALE1IS:
+ __tlbi(vaale1is, val);
+ break;
+ default:
+ break;
+ }
+ dsb(ish);
+ isb();
+
+ __tlb_switch_to_host(&cxt);
+}
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 36cb9fa22153..5fb13995190e 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -80,7 +80,23 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
*/
void kvm_flush_remote_tlbs(struct kvm *kvm)
{
- kvm_call_hyp(__kvm_tlb_flush_vmid, &kvm->arch.mmu);
+ struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
+
+ if (mmu == &kvm->arch.mmu) {
+ /*
+ * For a normal (i.e. non-nested) guest, flush entries for the
+ * given VMID *
+ */
+ kvm_call_hyp(__kvm_tlb_flush_vmid, mmu);
+ } else {
+ /*
+ * When supporting nested virtualization, we can have multiple
+ * VMIDs in play for each VCPU in the VM, so it's really not
+ * worth it to try to quiesce the system and flush all the
+ * VMIDs that may be in use, instead just nuke the whole thing.
+ */
+ kvm_call_hyp(__kvm_flush_vm_context);
+ }
}
static bool kvm_is_device_pfn(unsigned long pfn)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6b7a7ce9b1a3..ecd61a6b06d1 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1623,6 +1623,11 @@ static bool forward_at_traps(struct kvm_vcpu *vcpu)
return forward_traps(vcpu, HCR_AT);
}
+static bool forward_ttlb_traps(struct kvm_vcpu *vcpu)
+{
+ return forward_traps(vcpu, HCR_TTLB);
+}
+
static bool access_elr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
@@ -2230,6 +2235,185 @@ static bool handle_s12w(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return handle_s12(vcpu, p, r, true);
}
+static bool handle_alle2is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ /*
+ * To emulate invalidating all EL2 regime stage 1 TLB entries for all
+ * PEs, executing TLBI VMALLE1IS is enough. But reuse the existing
+ * interface for the simplicity; invalidating stage 2 entries doesn't
+ * affect the correctness.
+ */
+ __kvm_tlb_flush_vmid(&vcpu->kvm->arch.mmu);
+ return true;
+}
+
+static bool handle_vae2is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ int sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ /*
+ * Based on the same principle as TLBI ALLE2 instruction
+ * emulation, we emulate TLBI VAE2* instructions by executing
+ * corresponding TLBI VAE1* instructions with the virtual
+ * EL2's VMID assigned by the host hypervisor.
+ */
+ __kvm_tlb_vae2is(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
+ return true;
+}
+
+static bool handle_alle1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct kvm_s2_mmu *mmu = &vcpu->kvm->arch.mmu;
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ /*
+ * Clear all mappings in the shadow page tables and invalidate the stage
+ * 1 and 2 TLB entries via kvm_tlb_flush_vmid_ipa().
+ */
+ kvm_nested_s2_clear(vcpu->kvm);
+
+ if (mmu->vmid.vmid_gen) {
+ /*
+ * Invalidate the stage 1 and 2 TLB entries for the host OS
+ * in a VM only if there is one.
+ */
+ __kvm_tlb_flush_vmid(mmu);
+ }
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(vcpu->kvm));
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, 0, kvm_phys_size(vcpu->kvm));
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct kvm_s2_mmu *mmu;
+ u64 base_addr;
+ int max_size;
+
+ /*
+ * We drop a number of things from the supplied value:
+ *
+ * - NS bit: we're non-secure only.
+ *
+ * - TTL field: We already have the granule size from the
+ * VTCR_EL2.TG0 field, and the level is only relevant to the
+ * guest's S2PT.
+ *
+ * - IPA[51:48]: We don't support 52bit IPA just yet...
+ *
+ * And of course, adjust the IPA to be on an actual address.
+ */
+ base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
+
+ /* Compute the maximum extent of the invalidation */
+ switch ((vtcr & VTCR_EL2_TG0_MASK)) {
+ case VTCR_EL2_TG0_4K:
+ max_size = SZ_1G;
+ break;
+ case VTCR_EL2_TG0_16K:
+ max_size = SZ_32M;
+ break;
+ case VTCR_EL2_TG0_64K:
+ /*
+ * No, we do not support 52bit IPA in nested yet. Once
+ * we do, this should be 4TB.
+ */
+ /* FIXME: remove the 52bit PA support from the IDregs */
+ max_size = SZ_512M;
+ break;
+ default:
+ BUG();
+ }
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, base_addr, max_size);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
+ if (mmu)
+ kvm_unmap_stage2_range(mmu, base_addr, max_size);
+
+ spin_unlock(&vcpu->kvm->mmu_lock);
+
+ return true;
+}
+
+static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ u32 sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+
+ /*
+ * If we're here, this is because we've trapped on a EL1 TLBI
+ * instruction that affects the EL1 translation regime while
+ * we're running in a context that doesn't allow us to let the
+ * HW do its thing (aka vEL2):
+ *
+ * - HCR_EL2.E2H == 0 : a non-VHE guest
+ * - HCR_EL2.{E2H,TGE} == { 1, 0 } : a VHE guest in guest mode
+ *
+ * We don't expect these helpers to ever be called when running
+ * in a vEL1 context.
+ */
+
+ WARN_ON(!vcpu_mode_el2(vcpu));
+
+ mutex_lock(&vcpu->kvm->lock);
+
+ if ((__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
+ u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
+
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+ } else {
+ /*
+ * ARMv8.4-NV allows the guest to change TGE behind
+ * our back, so we always trap EL1 TLBIs from vEL2...
+ */
+ __kvm_tlb_el1_instr(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
+ }
+
+ mutex_unlock(&vcpu->kvm->lock);
+
+ return true;
+}
+
/*
* AT instruction emulation
*
@@ -2312,12 +2496,40 @@ static struct sys_reg_desc sys_insn_descs[] = {
{ SYS_DESC(SYS_DC_CSW), access_dcsw },
{ SYS_DESC(SYS_DC_CISW), access_dcsw },
+ SYS_INSN_TO_DESC(TLBI_VMALLE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_ASIDE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAAE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAALE1IS, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_ASIDE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAAE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE1, handle_tlbi_el1, forward_ttlb_traps),
+ SYS_INSN_TO_DESC(TLBI_VAALE1, handle_tlbi_el1, forward_ttlb_traps),
+
SYS_INSN_TO_DESC(AT_S1E2R, handle_s1e2, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S1E2W, handle_s1e2, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E1R, handle_s12r, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E1W, handle_s12w, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E0R, handle_s12r, forward_nv_traps),
SYS_INSN_TO_DESC(AT_S12E0W, handle_s12w, forward_nv_traps),
+
+ SYS_INSN_TO_DESC(TLBI_IPAS2E1IS, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2LE1IS, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE2IS, handle_alle2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE2IS, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE1IS, handle_alle1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE2IS, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLS12E1IS, handle_vmalls12e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2E1, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_IPAS2LE1, handle_ipas2e1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE2, handle_alle2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VAE2, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_ALLE1, handle_alle1is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VALE2, handle_vae2is, forward_nv_traps),
+ SYS_INSN_TO_DESC(TLBI_VMALLS12E1, handle_vmalls12e1is, forward_nv_traps),
};
static bool trap_dbgidr(struct kvm_vcpu *vcpu,
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 42/66] KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
When entering a L2 guest (nested virt enabled, but not in hypervisor
context), we need to honor the traps the L1 guest has asked enabled.
For now, just OR the guest's HCR_EL2 into the host's. We may have to do
some filtering in the future though.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index bac6ccc2595c..d9dc470c7790 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -80,6 +80,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_tge_is_set(vcpu))
hcr |= HCR_AT | HCR_TTLB;
}
+ } else if (nested_virt_in_use(vcpu)) {
+ hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
}
___activate_traps(vcpu, hcr);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 42/66] KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
When entering a L2 guest (nested virt enabled, but not in hypervisor
context), we need to honor the traps the L1 guest has asked enabled.
For now, just OR the guest's HCR_EL2 into the host's. We may have to do
some filtering in the future though.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index bac6ccc2595c..d9dc470c7790 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -80,6 +80,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_tge_is_set(vcpu))
hcr |= HCR_AT | HCR_TTLB;
}
+ } else if (nested_virt_in_use(vcpu)) {
+ hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
}
___activate_traps(vcpu, hcr);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 42/66] KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
When entering a L2 guest (nested virt enabled, but not in hypervisor
context), we need to honor the traps the L1 guest has asked enabled.
For now, just OR the guest's HCR_EL2 into the host's. We may have to do
some filtering in the future though.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index bac6ccc2595c..d9dc470c7790 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -80,6 +80,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_tge_is_set(vcpu))
hcr |= HCR_AT | HCR_TTLB;
}
+ } else if (nested_virt_in_use(vcpu)) {
+ hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
}
___activate_traps(vcpu, hcr);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 43/66] KVM: arm64: nv: arch_timer: Support hyp timer emulation
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
Emulating EL2 also means emulating the EL2 timers. To do so, we expand
our timer framework to deal with at most 4 timers. At any given time,
two timers are using the HW timers, and the two others are purely
emulated.
The role of deciding which is which at any given time is left to a
mapping function which is called every time we need to make such a
decision.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: added CNTVOFF support, general reworking for v4.8]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +
arch/arm64/kvm/arch_timer.c | 152 +++++++++++++++++++++++++++++-
arch/arm64/kvm/sys_regs.c | 7 +-
arch/arm64/kvm/trace_arm.h | 6 +-
arch/arm64/kvm/vgic/vgic.c | 15 +++
include/kvm/arm_arch_timer.h | 6 ++
include/kvm/arm_vgic.h | 1 +
7 files changed, 186 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bf60c50eec17..f3743e45ded1 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -272,6 +272,10 @@ enum vcpu_sysreg {
TPIDR_EL2, /* EL2 Software Thread ID Register */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
SP_EL2, /* EL2 Stack Pointer */
+ CNTHP_CTL_EL2,
+ CNTHP_CVAL_EL2,
+ CNTHV_CTL_EL2,
+ CNTHV_CVAL_EL2,
NR_SYS_REGS /* Nothing after this line! */
};
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 32ba6fbc3814..1c2be6391951 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -15,6 +15,7 @@
#include <asm/arch_timer.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
#include <kvm/arm_vgic.h>
#include <kvm/arm_arch_timer.h>
@@ -39,6 +40,16 @@ static const struct kvm_irq_level default_vtimer_irq = {
.level = 1,
};
+static const struct kvm_irq_level default_hptimer_irq = {
+ .irq = 26,
+ .level = 1,
+};
+
+static const struct kvm_irq_level default_hvtimer_irq = {
+ .irq = 28,
+ .level = 1,
+};
+
static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx);
static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
struct arch_timer_context *timer_ctx);
@@ -60,6 +71,10 @@ u32 timer_get_ctl(struct arch_timer_context *ctxt)
return __vcpu_sys_reg(vcpu, CNTV_CTL_EL0);
case TIMER_PTIMER:
return __vcpu_sys_reg(vcpu, CNTP_CTL_EL0);
+ case TIMER_HVTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2);
+ case TIMER_HPTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2);
default:
WARN_ON(1);
return 0;
@@ -75,6 +90,10 @@ u64 timer_get_cval(struct arch_timer_context *ctxt)
return __vcpu_sys_reg(vcpu, CNTV_CVAL_EL0);
case TIMER_PTIMER:
return __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0);
+ case TIMER_HVTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2);
+ case TIMER_HPTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2);
default:
WARN_ON(1);
return 0;
@@ -104,6 +123,12 @@ static void timer_set_ctl(struct arch_timer_context *ctxt, u32 ctl)
case TIMER_PTIMER:
__vcpu_sys_reg(vcpu, CNTP_CTL_EL0) = ctl;
break;
+ case TIMER_HVTIMER:
+ __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2) = ctl;
+ break;
+ case TIMER_HPTIMER:
+ __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2) = ctl;
+ break;
default:
WARN_ON(1);
}
@@ -120,6 +145,12 @@ static void timer_set_cval(struct arch_timer_context *ctxt, u64 cval)
case TIMER_PTIMER:
__vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = cval;
break;
+ case TIMER_HVTIMER:
+ __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2) = cval;
+ break;
+ case TIMER_HPTIMER:
+ __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2) = cval;
+ break;
default:
WARN_ON(1);
}
@@ -145,13 +176,27 @@ u64 kvm_phys_timer_read(void)
static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map)
{
- if (has_vhe()) {
+ if (nested_virt_in_use(vcpu)) {
+ if (is_hyp_ctxt(vcpu)) {
+ map->direct_vtimer = vcpu_hvtimer(vcpu);
+ map->direct_ptimer = vcpu_hptimer(vcpu);
+ map->emul_vtimer = vcpu_vtimer(vcpu);
+ map->emul_ptimer = vcpu_ptimer(vcpu);
+ } else {
+ map->direct_vtimer = vcpu_vtimer(vcpu);
+ map->direct_ptimer = vcpu_ptimer(vcpu);
+ map->emul_vtimer = vcpu_hvtimer(vcpu);
+ map->emul_ptimer = vcpu_hptimer(vcpu);
+ }
+ } else if (has_vhe()) {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = vcpu_ptimer(vcpu);
+ map->emul_vtimer = NULL;
map->emul_ptimer = NULL;
} else {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = NULL;
+ map->emul_vtimer = NULL;
map->emul_ptimer = vcpu_ptimer(vcpu);
}
@@ -324,9 +369,11 @@ static bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
cnt_ctl = read_sysreg_el0(SYS_CNTV_CTL);
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
cnt_ctl = read_sysreg_el0(SYS_CNTP_CTL);
break;
case NR_KVM_TIMERS:
@@ -357,6 +404,7 @@ bool kvm_timer_is_pending(struct kvm_vcpu *vcpu)
return kvm_timer_should_fire(map.direct_vtimer) ||
kvm_timer_should_fire(map.direct_ptimer) ||
+ kvm_timer_should_fire(map.emul_vtimer) ||
kvm_timer_should_fire(map.emul_ptimer);
}
@@ -437,6 +485,7 @@ static void timer_save_state(struct arch_timer_context *ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL));
timer_set_cval(ctx, read_sysreg_el0(SYS_CNTV_CVAL));
@@ -446,6 +495,7 @@ static void timer_save_state(struct arch_timer_context *ctx)
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL));
timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL));
@@ -483,6 +533,7 @@ static void kvm_timer_blocking(struct kvm_vcpu *vcpu)
*/
if (!kvm_timer_irq_can_fire(map.direct_vtimer) &&
!kvm_timer_irq_can_fire(map.direct_ptimer) &&
+ !kvm_timer_irq_can_fire(map.emul_vtimer) &&
!kvm_timer_irq_can_fire(map.emul_ptimer))
return;
@@ -516,11 +567,13 @@ static void timer_restore_state(struct arch_timer_context *ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
write_sysreg_el0(timer_get_cval(ctx), SYS_CNTV_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTV_CTL);
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
write_sysreg_el0(timer_get_cval(ctx), SYS_CNTP_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL);
@@ -597,6 +650,40 @@ static void kvm_timer_vcpu_load_nogic(struct kvm_vcpu *vcpu)
enable_percpu_irq(host_vtimer_irq, host_vtimer_irq_flags);
}
+static void kvm_timer_vcpu_load_nested_switch(struct kvm_vcpu *vcpu,
+ struct timer_map *map)
+{
+ int hw, ret;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return;
+
+ /*
+ * We only ever unmap the vtimer irq on a VHE system that runs nested
+ * virtualization, in which case we have both a valid emul_vtimer,
+ * emul_ptimer, direct_vtimer, and direct_ptimer.
+ *
+ * Since this is called from kvm_timer_vcpu_load(), a change between
+ * vEL2 and vEL1/0 will have just happened, and the timer_map will
+ * represent this, and therefore we switch the emul/direct mappings
+ * below.
+ */
+ hw = kvm_vgic_get_map(vcpu, map->direct_vtimer->irq.irq);
+ if (hw < 0) {
+ kvm_vgic_unmap_phys_irq(vcpu, map->emul_vtimer->irq.irq);
+ kvm_vgic_unmap_phys_irq(vcpu, map->emul_ptimer->irq.irq);
+
+ ret = kvm_vgic_map_phys_irq(vcpu,
+ map->direct_vtimer->host_timer_irq,
+ map->direct_vtimer->irq.irq,
+ kvm_arch_timer_get_input_level);
+ ret = kvm_vgic_map_phys_irq(vcpu,
+ map->direct_ptimer->host_timer_irq,
+ map->direct_ptimer->irq.irq,
+ kvm_arch_timer_get_input_level);
+ }
+}
+
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
{
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
@@ -608,6 +695,9 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
get_timer_map(vcpu, &map);
if (static_branch_likely(&has_gic_active_state)) {
+ if (nested_virt_in_use(vcpu))
+ kvm_timer_vcpu_load_nested_switch(vcpu, &map);
+
kvm_timer_vcpu_load_gic(map.direct_vtimer);
if (map.direct_ptimer)
kvm_timer_vcpu_load_gic(map.direct_ptimer);
@@ -623,6 +713,8 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
if (map.direct_ptimer)
timer_restore_state(map.direct_ptimer);
+ if (map.emul_vtimer)
+ timer_emulate(map.emul_vtimer);
if (map.emul_ptimer)
timer_emulate(map.emul_ptimer);
}
@@ -668,6 +760,8 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
* In any case, we re-schedule the hrtimer for the physical timer when
* coming back to the VCPU thread in kvm_timer_vcpu_load().
*/
+ if (map.emul_vtimer)
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
if (map.emul_ptimer)
soft_timer_cancel(&map.emul_ptimer->hrtimer);
@@ -728,10 +822,14 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
*/
timer_set_ctl(vcpu_vtimer(vcpu), 0);
timer_set_ctl(vcpu_ptimer(vcpu), 0);
+ timer_set_ctl(vcpu_hvtimer(vcpu), 0);
+ timer_set_ctl(vcpu_hptimer(vcpu), 0);
if (timer->enabled) {
kvm_timer_update_irq(vcpu, false, vcpu_vtimer(vcpu));
kvm_timer_update_irq(vcpu, false, vcpu_ptimer(vcpu));
+ kvm_timer_update_irq(vcpu, false, vcpu_hvtimer(vcpu));
+ kvm_timer_update_irq(vcpu, false, vcpu_hptimer(vcpu));
if (irqchip_in_kernel(vcpu->kvm)) {
kvm_vgic_reset_mapped_irq(vcpu, map.direct_vtimer->irq.irq);
@@ -740,6 +838,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
}
}
+ if (map.emul_vtimer)
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
if (map.emul_ptimer)
soft_timer_cancel(&map.emul_ptimer->hrtimer);
@@ -770,30 +870,47 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+ struct arch_timer_context *hvtimer = vcpu_hvtimer(vcpu);
+ struct arch_timer_context *hptimer = vcpu_hptimer(vcpu);
vtimer->vcpu = vcpu;
ptimer->vcpu = vcpu;
+ hvtimer->vcpu = vcpu;
+ hptimer->vcpu = vcpu;
/* Synchronize cntvoff across all vtimers of a VM. */
update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
timer_set_offset(ptimer, 0);
+ timer_set_offset(hvtimer, 0);
+ timer_set_offset(hptimer, 0);
hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
timer->bg_timer.function = kvm_bg_timer_expire;
hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+ hrtimer_init(&hvtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+ hrtimer_init(&hptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+
vtimer->hrtimer.function = kvm_hrtimer_expire;
ptimer->hrtimer.function = kvm_hrtimer_expire;
+ hvtimer->hrtimer.function = kvm_hrtimer_expire;
+ hptimer->hrtimer.function = kvm_hrtimer_expire;
vtimer->irq.irq = default_vtimer_irq.irq;
ptimer->irq.irq = default_ptimer_irq.irq;
+ hvtimer->irq.irq = default_hvtimer_irq.irq;
+ hptimer->irq.irq = default_hptimer_irq.irq;
vtimer->host_timer_irq = host_vtimer_irq;
ptimer->host_timer_irq = host_ptimer_irq;
+ hvtimer->host_timer_irq = host_vtimer_irq;
+ hptimer->host_timer_irq = host_ptimer_irq;
vtimer->host_timer_irq_flags = host_vtimer_irq_flags;
ptimer->host_timer_irq_flags = host_ptimer_irq_flags;
+ hvtimer->host_timer_irq_flags = host_vtimer_irq_flags;
+ hptimer->host_timer_irq_flags = host_ptimer_irq_flags;
}
static void kvm_timer_init_interrupt(void *info)
@@ -900,6 +1017,10 @@ static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu,
val = kvm_phys_timer_read() - timer_get_offset(timer);
break;
+ case TIMER_REG_VOFF:
+ val = timer_get_offset(timer);
+ break;
+
default:
BUG();
}
@@ -942,6 +1063,10 @@ static void kvm_arm_timer_write(struct kvm_vcpu *vcpu,
timer_set_cval(timer, val);
break;
+ case TIMER_REG_VOFF:
+ timer_set_offset(timer, val);
+ break;
+
default:
BUG();
}
@@ -1079,7 +1204,7 @@ void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)
static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu)
{
- int vtimer_irq, ptimer_irq;
+ int vtimer_irq, ptimer_irq, hvtimer_irq, hptimer_irq;
int i, ret;
vtimer_irq = vcpu_vtimer(vcpu)->irq.irq;
@@ -1092,9 +1217,21 @@ static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu)
if (ret)
return false;
+ hvtimer_irq = vcpu_hvtimer(vcpu)->irq.irq;
+ ret = kvm_vgic_set_owner(vcpu, hvtimer_irq, vcpu_hvtimer(vcpu));
+ if (ret)
+ return false;
+
+ hptimer_irq = vcpu_hptimer(vcpu)->irq.irq;
+ ret = kvm_vgic_set_owner(vcpu, hptimer_irq, vcpu_hptimer(vcpu));
+ if (ret)
+ return false;
+
kvm_for_each_vcpu(i, vcpu, vcpu->kvm) {
if (vcpu_vtimer(vcpu)->irq.irq != vtimer_irq ||
- vcpu_ptimer(vcpu)->irq.irq != ptimer_irq)
+ vcpu_ptimer(vcpu)->irq.irq != ptimer_irq ||
+ vcpu_hvtimer(vcpu)->irq.irq != hvtimer_irq ||
+ vcpu_hptimer(vcpu)->irq.irq != hptimer_irq)
return false;
}
@@ -1110,6 +1247,10 @@ bool kvm_arch_timer_get_input_level(int vintid)
timer = vcpu_vtimer(vcpu);
else if (vintid == vcpu_ptimer(vcpu)->irq.irq)
timer = vcpu_ptimer(vcpu);
+ else if (vintid == vcpu_hvtimer(vcpu)->irq.irq)
+ timer = vcpu_hvtimer(vcpu);
+ else if (vintid == vcpu_hptimer(vcpu)->irq.irq)
+ timer = vcpu_hptimer(vcpu);
else
BUG();
@@ -1191,6 +1332,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq)
kvm_for_each_vcpu(i, vcpu, kvm) {
vcpu_vtimer(vcpu)->irq.irq = vtimer_irq;
vcpu_ptimer(vcpu)->irq.irq = ptimer_irq;
+ /* TODO: Add support for hv/hp timers */
}
}
@@ -1201,6 +1343,8 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
int irq;
+ /* TODO: Add support for hv/hp timers */
+
if (!irqchip_in_kernel(vcpu->kvm))
return -EINVAL;
@@ -1233,6 +1377,8 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
struct arch_timer_context *timer;
int irq;
+ /* TODO: Add support for hv/hp timers */
+
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
timer = vcpu_vtimer(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index ecd61a6b06d1..98ff5b42a6b5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1290,6 +1290,11 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
tmr = TIMER_PTIMER;
treg = TIMER_REG_CVAL;
break;
+ case SYS_CNTVOFF_EL2:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_VOFF;
+ break;
+
default:
BUG();
}
@@ -2101,7 +2106,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
- { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
+ { SYS_DESC(SYS_CNTVOFF_EL2), access_arch_timer },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
{ SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
index 5707011c4f47..49f4cb52d35d 100644
--- a/arch/arm64/kvm/trace_arm.h
+++ b/arch/arm64/kvm/trace_arm.h
@@ -272,6 +272,7 @@ TRACE_EVENT(kvm_get_timer_map,
__field( unsigned long, vcpu_id )
__field( int, direct_vtimer )
__field( int, direct_ptimer )
+ __field( int, emul_vtimer )
__field( int, emul_ptimer )
),
@@ -280,14 +281,17 @@ TRACE_EVENT(kvm_get_timer_map,
__entry->direct_vtimer = arch_timer_ctx_index(map->direct_vtimer);
__entry->direct_ptimer =
(map->direct_ptimer) ? arch_timer_ctx_index(map->direct_ptimer) : -1;
+ __entry->emul_vtimer =
+ (map->emul_vtimer) ? arch_timer_ctx_index(map->emul_vtimer) : -1;
__entry->emul_ptimer =
(map->emul_ptimer) ? arch_timer_ctx_index(map->emul_ptimer) : -1;
),
- TP_printk("VCPU: %ld, dv: %d, dp: %d, ep: %d",
+ TP_printk("VCPU: %ld, dv: %d, dp: %d, ev: %d, ep: %d",
__entry->vcpu_id,
__entry->direct_vtimer,
__entry->direct_ptimer,
+ __entry->emul_vtimer,
__entry->emul_ptimer)
);
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 1c597c9885fa..2af93dc80689 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -573,6 +573,21 @@ int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
return 0;
}
+int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid)
+{
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+ unsigned long flags;
+ int ret = -1;
+
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+ if (irq->hw)
+ ret = irq->hwintid;
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ return ret;
+}
+
/**
* kvm_vgic_set_owner - Set the owner of an interrupt for a VM
*
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 51c19381108c..063f613fbc7e 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -13,6 +13,8 @@
enum kvm_arch_timers {
TIMER_PTIMER,
TIMER_VTIMER,
+ TIMER_HVTIMER,
+ TIMER_HPTIMER,
NR_KVM_TIMERS
};
@@ -21,6 +23,7 @@ enum kvm_arch_timer_regs {
TIMER_REG_CVAL,
TIMER_REG_TVAL,
TIMER_REG_CTL,
+ TIMER_REG_VOFF,
};
struct arch_timer_context {
@@ -47,6 +50,7 @@ struct arch_timer_context {
struct timer_map {
struct arch_timer_context *direct_vtimer;
struct arch_timer_context *direct_ptimer;
+ struct arch_timer_context *emul_vtimer;
struct arch_timer_context *emul_ptimer;
};
@@ -91,6 +95,8 @@ bool kvm_arch_timer_get_input_level(int vintid);
#define vcpu_get_timer(v,t) (&vcpu_timer(v)->timers[(t)])
#define vcpu_vtimer(v) (&(v)->arch.timer_cpu.timers[TIMER_VTIMER])
#define vcpu_ptimer(v) (&(v)->arch.timer_cpu.timers[TIMER_PTIMER])
+#define vcpu_hvtimer(v) (&(v)->arch.timer_cpu.timers[TIMER_HVTIMER])
+#define vcpu_hptimer(v) (&(v)->arch.timer_cpu.timers[TIMER_HPTIMER])
#define arch_timer_ctx_index(ctx) ((ctx) - vcpu_timer((ctx)->vcpu)->timers)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 3d74f1060bd1..4749c4748e5a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -353,6 +353,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
u32 vintid, bool (*get_input_level)(int vindid));
int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
+int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid);
bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 43/66] KVM: arm64: nv: arch_timer: Support hyp timer emulation
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
Emulating EL2 also means emulating the EL2 timers. To do so, we expand
our timer framework to deal with at most 4 timers. At any given time,
two timers are using the HW timers, and the two others are purely
emulated.
The role of deciding which is which at any given time is left to a
mapping function which is called every time we need to make such a
decision.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: added CNTVOFF support, general reworking for v4.8]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +
arch/arm64/kvm/arch_timer.c | 152 +++++++++++++++++++++++++++++-
arch/arm64/kvm/sys_regs.c | 7 +-
arch/arm64/kvm/trace_arm.h | 6 +-
arch/arm64/kvm/vgic/vgic.c | 15 +++
include/kvm/arm_arch_timer.h | 6 ++
include/kvm/arm_vgic.h | 1 +
7 files changed, 186 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bf60c50eec17..f3743e45ded1 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -272,6 +272,10 @@ enum vcpu_sysreg {
TPIDR_EL2, /* EL2 Software Thread ID Register */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
SP_EL2, /* EL2 Stack Pointer */
+ CNTHP_CTL_EL2,
+ CNTHP_CVAL_EL2,
+ CNTHV_CTL_EL2,
+ CNTHV_CVAL_EL2,
NR_SYS_REGS /* Nothing after this line! */
};
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 32ba6fbc3814..1c2be6391951 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -15,6 +15,7 @@
#include <asm/arch_timer.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
#include <kvm/arm_vgic.h>
#include <kvm/arm_arch_timer.h>
@@ -39,6 +40,16 @@ static const struct kvm_irq_level default_vtimer_irq = {
.level = 1,
};
+static const struct kvm_irq_level default_hptimer_irq = {
+ .irq = 26,
+ .level = 1,
+};
+
+static const struct kvm_irq_level default_hvtimer_irq = {
+ .irq = 28,
+ .level = 1,
+};
+
static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx);
static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
struct arch_timer_context *timer_ctx);
@@ -60,6 +71,10 @@ u32 timer_get_ctl(struct arch_timer_context *ctxt)
return __vcpu_sys_reg(vcpu, CNTV_CTL_EL0);
case TIMER_PTIMER:
return __vcpu_sys_reg(vcpu, CNTP_CTL_EL0);
+ case TIMER_HVTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2);
+ case TIMER_HPTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2);
default:
WARN_ON(1);
return 0;
@@ -75,6 +90,10 @@ u64 timer_get_cval(struct arch_timer_context *ctxt)
return __vcpu_sys_reg(vcpu, CNTV_CVAL_EL0);
case TIMER_PTIMER:
return __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0);
+ case TIMER_HVTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2);
+ case TIMER_HPTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2);
default:
WARN_ON(1);
return 0;
@@ -104,6 +123,12 @@ static void timer_set_ctl(struct arch_timer_context *ctxt, u32 ctl)
case TIMER_PTIMER:
__vcpu_sys_reg(vcpu, CNTP_CTL_EL0) = ctl;
break;
+ case TIMER_HVTIMER:
+ __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2) = ctl;
+ break;
+ case TIMER_HPTIMER:
+ __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2) = ctl;
+ break;
default:
WARN_ON(1);
}
@@ -120,6 +145,12 @@ static void timer_set_cval(struct arch_timer_context *ctxt, u64 cval)
case TIMER_PTIMER:
__vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = cval;
break;
+ case TIMER_HVTIMER:
+ __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2) = cval;
+ break;
+ case TIMER_HPTIMER:
+ __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2) = cval;
+ break;
default:
WARN_ON(1);
}
@@ -145,13 +176,27 @@ u64 kvm_phys_timer_read(void)
static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map)
{
- if (has_vhe()) {
+ if (nested_virt_in_use(vcpu)) {
+ if (is_hyp_ctxt(vcpu)) {
+ map->direct_vtimer = vcpu_hvtimer(vcpu);
+ map->direct_ptimer = vcpu_hptimer(vcpu);
+ map->emul_vtimer = vcpu_vtimer(vcpu);
+ map->emul_ptimer = vcpu_ptimer(vcpu);
+ } else {
+ map->direct_vtimer = vcpu_vtimer(vcpu);
+ map->direct_ptimer = vcpu_ptimer(vcpu);
+ map->emul_vtimer = vcpu_hvtimer(vcpu);
+ map->emul_ptimer = vcpu_hptimer(vcpu);
+ }
+ } else if (has_vhe()) {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = vcpu_ptimer(vcpu);
+ map->emul_vtimer = NULL;
map->emul_ptimer = NULL;
} else {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = NULL;
+ map->emul_vtimer = NULL;
map->emul_ptimer = vcpu_ptimer(vcpu);
}
@@ -324,9 +369,11 @@ static bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
cnt_ctl = read_sysreg_el0(SYS_CNTV_CTL);
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
cnt_ctl = read_sysreg_el0(SYS_CNTP_CTL);
break;
case NR_KVM_TIMERS:
@@ -357,6 +404,7 @@ bool kvm_timer_is_pending(struct kvm_vcpu *vcpu)
return kvm_timer_should_fire(map.direct_vtimer) ||
kvm_timer_should_fire(map.direct_ptimer) ||
+ kvm_timer_should_fire(map.emul_vtimer) ||
kvm_timer_should_fire(map.emul_ptimer);
}
@@ -437,6 +485,7 @@ static void timer_save_state(struct arch_timer_context *ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL));
timer_set_cval(ctx, read_sysreg_el0(SYS_CNTV_CVAL));
@@ -446,6 +495,7 @@ static void timer_save_state(struct arch_timer_context *ctx)
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL));
timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL));
@@ -483,6 +533,7 @@ static void kvm_timer_blocking(struct kvm_vcpu *vcpu)
*/
if (!kvm_timer_irq_can_fire(map.direct_vtimer) &&
!kvm_timer_irq_can_fire(map.direct_ptimer) &&
+ !kvm_timer_irq_can_fire(map.emul_vtimer) &&
!kvm_timer_irq_can_fire(map.emul_ptimer))
return;
@@ -516,11 +567,13 @@ static void timer_restore_state(struct arch_timer_context *ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
write_sysreg_el0(timer_get_cval(ctx), SYS_CNTV_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTV_CTL);
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
write_sysreg_el0(timer_get_cval(ctx), SYS_CNTP_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL);
@@ -597,6 +650,40 @@ static void kvm_timer_vcpu_load_nogic(struct kvm_vcpu *vcpu)
enable_percpu_irq(host_vtimer_irq, host_vtimer_irq_flags);
}
+static void kvm_timer_vcpu_load_nested_switch(struct kvm_vcpu *vcpu,
+ struct timer_map *map)
+{
+ int hw, ret;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return;
+
+ /*
+ * We only ever unmap the vtimer irq on a VHE system that runs nested
+ * virtualization, in which case we have both a valid emul_vtimer,
+ * emul_ptimer, direct_vtimer, and direct_ptimer.
+ *
+ * Since this is called from kvm_timer_vcpu_load(), a change between
+ * vEL2 and vEL1/0 will have just happened, and the timer_map will
+ * represent this, and therefore we switch the emul/direct mappings
+ * below.
+ */
+ hw = kvm_vgic_get_map(vcpu, map->direct_vtimer->irq.irq);
+ if (hw < 0) {
+ kvm_vgic_unmap_phys_irq(vcpu, map->emul_vtimer->irq.irq);
+ kvm_vgic_unmap_phys_irq(vcpu, map->emul_ptimer->irq.irq);
+
+ ret = kvm_vgic_map_phys_irq(vcpu,
+ map->direct_vtimer->host_timer_irq,
+ map->direct_vtimer->irq.irq,
+ kvm_arch_timer_get_input_level);
+ ret = kvm_vgic_map_phys_irq(vcpu,
+ map->direct_ptimer->host_timer_irq,
+ map->direct_ptimer->irq.irq,
+ kvm_arch_timer_get_input_level);
+ }
+}
+
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
{
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
@@ -608,6 +695,9 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
get_timer_map(vcpu, &map);
if (static_branch_likely(&has_gic_active_state)) {
+ if (nested_virt_in_use(vcpu))
+ kvm_timer_vcpu_load_nested_switch(vcpu, &map);
+
kvm_timer_vcpu_load_gic(map.direct_vtimer);
if (map.direct_ptimer)
kvm_timer_vcpu_load_gic(map.direct_ptimer);
@@ -623,6 +713,8 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
if (map.direct_ptimer)
timer_restore_state(map.direct_ptimer);
+ if (map.emul_vtimer)
+ timer_emulate(map.emul_vtimer);
if (map.emul_ptimer)
timer_emulate(map.emul_ptimer);
}
@@ -668,6 +760,8 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
* In any case, we re-schedule the hrtimer for the physical timer when
* coming back to the VCPU thread in kvm_timer_vcpu_load().
*/
+ if (map.emul_vtimer)
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
if (map.emul_ptimer)
soft_timer_cancel(&map.emul_ptimer->hrtimer);
@@ -728,10 +822,14 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
*/
timer_set_ctl(vcpu_vtimer(vcpu), 0);
timer_set_ctl(vcpu_ptimer(vcpu), 0);
+ timer_set_ctl(vcpu_hvtimer(vcpu), 0);
+ timer_set_ctl(vcpu_hptimer(vcpu), 0);
if (timer->enabled) {
kvm_timer_update_irq(vcpu, false, vcpu_vtimer(vcpu));
kvm_timer_update_irq(vcpu, false, vcpu_ptimer(vcpu));
+ kvm_timer_update_irq(vcpu, false, vcpu_hvtimer(vcpu));
+ kvm_timer_update_irq(vcpu, false, vcpu_hptimer(vcpu));
if (irqchip_in_kernel(vcpu->kvm)) {
kvm_vgic_reset_mapped_irq(vcpu, map.direct_vtimer->irq.irq);
@@ -740,6 +838,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
}
}
+ if (map.emul_vtimer)
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
if (map.emul_ptimer)
soft_timer_cancel(&map.emul_ptimer->hrtimer);
@@ -770,30 +870,47 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+ struct arch_timer_context *hvtimer = vcpu_hvtimer(vcpu);
+ struct arch_timer_context *hptimer = vcpu_hptimer(vcpu);
vtimer->vcpu = vcpu;
ptimer->vcpu = vcpu;
+ hvtimer->vcpu = vcpu;
+ hptimer->vcpu = vcpu;
/* Synchronize cntvoff across all vtimers of a VM. */
update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
timer_set_offset(ptimer, 0);
+ timer_set_offset(hvtimer, 0);
+ timer_set_offset(hptimer, 0);
hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
timer->bg_timer.function = kvm_bg_timer_expire;
hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+ hrtimer_init(&hvtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+ hrtimer_init(&hptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+
vtimer->hrtimer.function = kvm_hrtimer_expire;
ptimer->hrtimer.function = kvm_hrtimer_expire;
+ hvtimer->hrtimer.function = kvm_hrtimer_expire;
+ hptimer->hrtimer.function = kvm_hrtimer_expire;
vtimer->irq.irq = default_vtimer_irq.irq;
ptimer->irq.irq = default_ptimer_irq.irq;
+ hvtimer->irq.irq = default_hvtimer_irq.irq;
+ hptimer->irq.irq = default_hptimer_irq.irq;
vtimer->host_timer_irq = host_vtimer_irq;
ptimer->host_timer_irq = host_ptimer_irq;
+ hvtimer->host_timer_irq = host_vtimer_irq;
+ hptimer->host_timer_irq = host_ptimer_irq;
vtimer->host_timer_irq_flags = host_vtimer_irq_flags;
ptimer->host_timer_irq_flags = host_ptimer_irq_flags;
+ hvtimer->host_timer_irq_flags = host_vtimer_irq_flags;
+ hptimer->host_timer_irq_flags = host_ptimer_irq_flags;
}
static void kvm_timer_init_interrupt(void *info)
@@ -900,6 +1017,10 @@ static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu,
val = kvm_phys_timer_read() - timer_get_offset(timer);
break;
+ case TIMER_REG_VOFF:
+ val = timer_get_offset(timer);
+ break;
+
default:
BUG();
}
@@ -942,6 +1063,10 @@ static void kvm_arm_timer_write(struct kvm_vcpu *vcpu,
timer_set_cval(timer, val);
break;
+ case TIMER_REG_VOFF:
+ timer_set_offset(timer, val);
+ break;
+
default:
BUG();
}
@@ -1079,7 +1204,7 @@ void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)
static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu)
{
- int vtimer_irq, ptimer_irq;
+ int vtimer_irq, ptimer_irq, hvtimer_irq, hptimer_irq;
int i, ret;
vtimer_irq = vcpu_vtimer(vcpu)->irq.irq;
@@ -1092,9 +1217,21 @@ static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu)
if (ret)
return false;
+ hvtimer_irq = vcpu_hvtimer(vcpu)->irq.irq;
+ ret = kvm_vgic_set_owner(vcpu, hvtimer_irq, vcpu_hvtimer(vcpu));
+ if (ret)
+ return false;
+
+ hptimer_irq = vcpu_hptimer(vcpu)->irq.irq;
+ ret = kvm_vgic_set_owner(vcpu, hptimer_irq, vcpu_hptimer(vcpu));
+ if (ret)
+ return false;
+
kvm_for_each_vcpu(i, vcpu, vcpu->kvm) {
if (vcpu_vtimer(vcpu)->irq.irq != vtimer_irq ||
- vcpu_ptimer(vcpu)->irq.irq != ptimer_irq)
+ vcpu_ptimer(vcpu)->irq.irq != ptimer_irq ||
+ vcpu_hvtimer(vcpu)->irq.irq != hvtimer_irq ||
+ vcpu_hptimer(vcpu)->irq.irq != hptimer_irq)
return false;
}
@@ -1110,6 +1247,10 @@ bool kvm_arch_timer_get_input_level(int vintid)
timer = vcpu_vtimer(vcpu);
else if (vintid == vcpu_ptimer(vcpu)->irq.irq)
timer = vcpu_ptimer(vcpu);
+ else if (vintid == vcpu_hvtimer(vcpu)->irq.irq)
+ timer = vcpu_hvtimer(vcpu);
+ else if (vintid == vcpu_hptimer(vcpu)->irq.irq)
+ timer = vcpu_hptimer(vcpu);
else
BUG();
@@ -1191,6 +1332,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq)
kvm_for_each_vcpu(i, vcpu, kvm) {
vcpu_vtimer(vcpu)->irq.irq = vtimer_irq;
vcpu_ptimer(vcpu)->irq.irq = ptimer_irq;
+ /* TODO: Add support for hv/hp timers */
}
}
@@ -1201,6 +1343,8 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
int irq;
+ /* TODO: Add support for hv/hp timers */
+
if (!irqchip_in_kernel(vcpu->kvm))
return -EINVAL;
@@ -1233,6 +1377,8 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
struct arch_timer_context *timer;
int irq;
+ /* TODO: Add support for hv/hp timers */
+
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
timer = vcpu_vtimer(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index ecd61a6b06d1..98ff5b42a6b5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1290,6 +1290,11 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
tmr = TIMER_PTIMER;
treg = TIMER_REG_CVAL;
break;
+ case SYS_CNTVOFF_EL2:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_VOFF;
+ break;
+
default:
BUG();
}
@@ -2101,7 +2106,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
- { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
+ { SYS_DESC(SYS_CNTVOFF_EL2), access_arch_timer },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
{ SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
index 5707011c4f47..49f4cb52d35d 100644
--- a/arch/arm64/kvm/trace_arm.h
+++ b/arch/arm64/kvm/trace_arm.h
@@ -272,6 +272,7 @@ TRACE_EVENT(kvm_get_timer_map,
__field( unsigned long, vcpu_id )
__field( int, direct_vtimer )
__field( int, direct_ptimer )
+ __field( int, emul_vtimer )
__field( int, emul_ptimer )
),
@@ -280,14 +281,17 @@ TRACE_EVENT(kvm_get_timer_map,
__entry->direct_vtimer = arch_timer_ctx_index(map->direct_vtimer);
__entry->direct_ptimer =
(map->direct_ptimer) ? arch_timer_ctx_index(map->direct_ptimer) : -1;
+ __entry->emul_vtimer =
+ (map->emul_vtimer) ? arch_timer_ctx_index(map->emul_vtimer) : -1;
__entry->emul_ptimer =
(map->emul_ptimer) ? arch_timer_ctx_index(map->emul_ptimer) : -1;
),
- TP_printk("VCPU: %ld, dv: %d, dp: %d, ep: %d",
+ TP_printk("VCPU: %ld, dv: %d, dp: %d, ev: %d, ep: %d",
__entry->vcpu_id,
__entry->direct_vtimer,
__entry->direct_ptimer,
+ __entry->emul_vtimer,
__entry->emul_ptimer)
);
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 1c597c9885fa..2af93dc80689 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -573,6 +573,21 @@ int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
return 0;
}
+int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid)
+{
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+ unsigned long flags;
+ int ret = -1;
+
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+ if (irq->hw)
+ ret = irq->hwintid;
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ return ret;
+}
+
/**
* kvm_vgic_set_owner - Set the owner of an interrupt for a VM
*
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 51c19381108c..063f613fbc7e 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -13,6 +13,8 @@
enum kvm_arch_timers {
TIMER_PTIMER,
TIMER_VTIMER,
+ TIMER_HVTIMER,
+ TIMER_HPTIMER,
NR_KVM_TIMERS
};
@@ -21,6 +23,7 @@ enum kvm_arch_timer_regs {
TIMER_REG_CVAL,
TIMER_REG_TVAL,
TIMER_REG_CTL,
+ TIMER_REG_VOFF,
};
struct arch_timer_context {
@@ -47,6 +50,7 @@ struct arch_timer_context {
struct timer_map {
struct arch_timer_context *direct_vtimer;
struct arch_timer_context *direct_ptimer;
+ struct arch_timer_context *emul_vtimer;
struct arch_timer_context *emul_ptimer;
};
@@ -91,6 +95,8 @@ bool kvm_arch_timer_get_input_level(int vintid);
#define vcpu_get_timer(v,t) (&vcpu_timer(v)->timers[(t)])
#define vcpu_vtimer(v) (&(v)->arch.timer_cpu.timers[TIMER_VTIMER])
#define vcpu_ptimer(v) (&(v)->arch.timer_cpu.timers[TIMER_PTIMER])
+#define vcpu_hvtimer(v) (&(v)->arch.timer_cpu.timers[TIMER_HVTIMER])
+#define vcpu_hptimer(v) (&(v)->arch.timer_cpu.timers[TIMER_HPTIMER])
#define arch_timer_ctx_index(ctx) ((ctx) - vcpu_timer((ctx)->vcpu)->timers)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 3d74f1060bd1..4749c4748e5a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -353,6 +353,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
u32 vintid, bool (*get_input_level)(int vindid));
int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
+int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid);
bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 43/66] KVM: arm64: nv: arch_timer: Support hyp timer emulation
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
Emulating EL2 also means emulating the EL2 timers. To do so, we expand
our timer framework to deal with at most 4 timers. At any given time,
two timers are using the HW timers, and the two others are purely
emulated.
The role of deciding which is which at any given time is left to a
mapping function which is called every time we need to make such a
decision.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
[maz: added CNTVOFF support, general reworking for v4.8]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +
arch/arm64/kvm/arch_timer.c | 152 +++++++++++++++++++++++++++++-
arch/arm64/kvm/sys_regs.c | 7 +-
arch/arm64/kvm/trace_arm.h | 6 +-
arch/arm64/kvm/vgic/vgic.c | 15 +++
include/kvm/arm_arch_timer.h | 6 ++
include/kvm/arm_vgic.h | 1 +
7 files changed, 186 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bf60c50eec17..f3743e45ded1 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -272,6 +272,10 @@ enum vcpu_sysreg {
TPIDR_EL2, /* EL2 Software Thread ID Register */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
SP_EL2, /* EL2 Stack Pointer */
+ CNTHP_CTL_EL2,
+ CNTHP_CVAL_EL2,
+ CNTHV_CTL_EL2,
+ CNTHV_CVAL_EL2,
NR_SYS_REGS /* Nothing after this line! */
};
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 32ba6fbc3814..1c2be6391951 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -15,6 +15,7 @@
#include <asm/arch_timer.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_nested.h>
#include <kvm/arm_vgic.h>
#include <kvm/arm_arch_timer.h>
@@ -39,6 +40,16 @@ static const struct kvm_irq_level default_vtimer_irq = {
.level = 1,
};
+static const struct kvm_irq_level default_hptimer_irq = {
+ .irq = 26,
+ .level = 1,
+};
+
+static const struct kvm_irq_level default_hvtimer_irq = {
+ .irq = 28,
+ .level = 1,
+};
+
static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx);
static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
struct arch_timer_context *timer_ctx);
@@ -60,6 +71,10 @@ u32 timer_get_ctl(struct arch_timer_context *ctxt)
return __vcpu_sys_reg(vcpu, CNTV_CTL_EL0);
case TIMER_PTIMER:
return __vcpu_sys_reg(vcpu, CNTP_CTL_EL0);
+ case TIMER_HVTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2);
+ case TIMER_HPTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2);
default:
WARN_ON(1);
return 0;
@@ -75,6 +90,10 @@ u64 timer_get_cval(struct arch_timer_context *ctxt)
return __vcpu_sys_reg(vcpu, CNTV_CVAL_EL0);
case TIMER_PTIMER:
return __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0);
+ case TIMER_HVTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2);
+ case TIMER_HPTIMER:
+ return __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2);
default:
WARN_ON(1);
return 0;
@@ -104,6 +123,12 @@ static void timer_set_ctl(struct arch_timer_context *ctxt, u32 ctl)
case TIMER_PTIMER:
__vcpu_sys_reg(vcpu, CNTP_CTL_EL0) = ctl;
break;
+ case TIMER_HVTIMER:
+ __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2) = ctl;
+ break;
+ case TIMER_HPTIMER:
+ __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2) = ctl;
+ break;
default:
WARN_ON(1);
}
@@ -120,6 +145,12 @@ static void timer_set_cval(struct arch_timer_context *ctxt, u64 cval)
case TIMER_PTIMER:
__vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = cval;
break;
+ case TIMER_HVTIMER:
+ __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2) = cval;
+ break;
+ case TIMER_HPTIMER:
+ __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2) = cval;
+ break;
default:
WARN_ON(1);
}
@@ -145,13 +176,27 @@ u64 kvm_phys_timer_read(void)
static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map)
{
- if (has_vhe()) {
+ if (nested_virt_in_use(vcpu)) {
+ if (is_hyp_ctxt(vcpu)) {
+ map->direct_vtimer = vcpu_hvtimer(vcpu);
+ map->direct_ptimer = vcpu_hptimer(vcpu);
+ map->emul_vtimer = vcpu_vtimer(vcpu);
+ map->emul_ptimer = vcpu_ptimer(vcpu);
+ } else {
+ map->direct_vtimer = vcpu_vtimer(vcpu);
+ map->direct_ptimer = vcpu_ptimer(vcpu);
+ map->emul_vtimer = vcpu_hvtimer(vcpu);
+ map->emul_ptimer = vcpu_hptimer(vcpu);
+ }
+ } else if (has_vhe()) {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = vcpu_ptimer(vcpu);
+ map->emul_vtimer = NULL;
map->emul_ptimer = NULL;
} else {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = NULL;
+ map->emul_vtimer = NULL;
map->emul_ptimer = vcpu_ptimer(vcpu);
}
@@ -324,9 +369,11 @@ static bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
cnt_ctl = read_sysreg_el0(SYS_CNTV_CTL);
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
cnt_ctl = read_sysreg_el0(SYS_CNTP_CTL);
break;
case NR_KVM_TIMERS:
@@ -357,6 +404,7 @@ bool kvm_timer_is_pending(struct kvm_vcpu *vcpu)
return kvm_timer_should_fire(map.direct_vtimer) ||
kvm_timer_should_fire(map.direct_ptimer) ||
+ kvm_timer_should_fire(map.emul_vtimer) ||
kvm_timer_should_fire(map.emul_ptimer);
}
@@ -437,6 +485,7 @@ static void timer_save_state(struct arch_timer_context *ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL));
timer_set_cval(ctx, read_sysreg_el0(SYS_CNTV_CVAL));
@@ -446,6 +495,7 @@ static void timer_save_state(struct arch_timer_context *ctx)
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL));
timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL));
@@ -483,6 +533,7 @@ static void kvm_timer_blocking(struct kvm_vcpu *vcpu)
*/
if (!kvm_timer_irq_can_fire(map.direct_vtimer) &&
!kvm_timer_irq_can_fire(map.direct_ptimer) &&
+ !kvm_timer_irq_can_fire(map.emul_vtimer) &&
!kvm_timer_irq_can_fire(map.emul_ptimer))
return;
@@ -516,11 +567,13 @@ static void timer_restore_state(struct arch_timer_context *ctx)
switch (index) {
case TIMER_VTIMER:
+ case TIMER_HVTIMER:
write_sysreg_el0(timer_get_cval(ctx), SYS_CNTV_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTV_CTL);
break;
case TIMER_PTIMER:
+ case TIMER_HPTIMER:
write_sysreg_el0(timer_get_cval(ctx), SYS_CNTP_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL);
@@ -597,6 +650,40 @@ static void kvm_timer_vcpu_load_nogic(struct kvm_vcpu *vcpu)
enable_percpu_irq(host_vtimer_irq, host_vtimer_irq_flags);
}
+static void kvm_timer_vcpu_load_nested_switch(struct kvm_vcpu *vcpu,
+ struct timer_map *map)
+{
+ int hw, ret;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return;
+
+ /*
+ * We only ever unmap the vtimer irq on a VHE system that runs nested
+ * virtualization, in which case we have both a valid emul_vtimer,
+ * emul_ptimer, direct_vtimer, and direct_ptimer.
+ *
+ * Since this is called from kvm_timer_vcpu_load(), a change between
+ * vEL2 and vEL1/0 will have just happened, and the timer_map will
+ * represent this, and therefore we switch the emul/direct mappings
+ * below.
+ */
+ hw = kvm_vgic_get_map(vcpu, map->direct_vtimer->irq.irq);
+ if (hw < 0) {
+ kvm_vgic_unmap_phys_irq(vcpu, map->emul_vtimer->irq.irq);
+ kvm_vgic_unmap_phys_irq(vcpu, map->emul_ptimer->irq.irq);
+
+ ret = kvm_vgic_map_phys_irq(vcpu,
+ map->direct_vtimer->host_timer_irq,
+ map->direct_vtimer->irq.irq,
+ kvm_arch_timer_get_input_level);
+ ret = kvm_vgic_map_phys_irq(vcpu,
+ map->direct_ptimer->host_timer_irq,
+ map->direct_ptimer->irq.irq,
+ kvm_arch_timer_get_input_level);
+ }
+}
+
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
{
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
@@ -608,6 +695,9 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
get_timer_map(vcpu, &map);
if (static_branch_likely(&has_gic_active_state)) {
+ if (nested_virt_in_use(vcpu))
+ kvm_timer_vcpu_load_nested_switch(vcpu, &map);
+
kvm_timer_vcpu_load_gic(map.direct_vtimer);
if (map.direct_ptimer)
kvm_timer_vcpu_load_gic(map.direct_ptimer);
@@ -623,6 +713,8 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
if (map.direct_ptimer)
timer_restore_state(map.direct_ptimer);
+ if (map.emul_vtimer)
+ timer_emulate(map.emul_vtimer);
if (map.emul_ptimer)
timer_emulate(map.emul_ptimer);
}
@@ -668,6 +760,8 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
* In any case, we re-schedule the hrtimer for the physical timer when
* coming back to the VCPU thread in kvm_timer_vcpu_load().
*/
+ if (map.emul_vtimer)
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
if (map.emul_ptimer)
soft_timer_cancel(&map.emul_ptimer->hrtimer);
@@ -728,10 +822,14 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
*/
timer_set_ctl(vcpu_vtimer(vcpu), 0);
timer_set_ctl(vcpu_ptimer(vcpu), 0);
+ timer_set_ctl(vcpu_hvtimer(vcpu), 0);
+ timer_set_ctl(vcpu_hptimer(vcpu), 0);
if (timer->enabled) {
kvm_timer_update_irq(vcpu, false, vcpu_vtimer(vcpu));
kvm_timer_update_irq(vcpu, false, vcpu_ptimer(vcpu));
+ kvm_timer_update_irq(vcpu, false, vcpu_hvtimer(vcpu));
+ kvm_timer_update_irq(vcpu, false, vcpu_hptimer(vcpu));
if (irqchip_in_kernel(vcpu->kvm)) {
kvm_vgic_reset_mapped_irq(vcpu, map.direct_vtimer->irq.irq);
@@ -740,6 +838,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
}
}
+ if (map.emul_vtimer)
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
if (map.emul_ptimer)
soft_timer_cancel(&map.emul_ptimer->hrtimer);
@@ -770,30 +870,47 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+ struct arch_timer_context *hvtimer = vcpu_hvtimer(vcpu);
+ struct arch_timer_context *hptimer = vcpu_hptimer(vcpu);
vtimer->vcpu = vcpu;
ptimer->vcpu = vcpu;
+ hvtimer->vcpu = vcpu;
+ hptimer->vcpu = vcpu;
/* Synchronize cntvoff across all vtimers of a VM. */
update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
timer_set_offset(ptimer, 0);
+ timer_set_offset(hvtimer, 0);
+ timer_set_offset(hptimer, 0);
hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
timer->bg_timer.function = kvm_bg_timer_expire;
hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+ hrtimer_init(&hvtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+ hrtimer_init(&hptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
+
vtimer->hrtimer.function = kvm_hrtimer_expire;
ptimer->hrtimer.function = kvm_hrtimer_expire;
+ hvtimer->hrtimer.function = kvm_hrtimer_expire;
+ hptimer->hrtimer.function = kvm_hrtimer_expire;
vtimer->irq.irq = default_vtimer_irq.irq;
ptimer->irq.irq = default_ptimer_irq.irq;
+ hvtimer->irq.irq = default_hvtimer_irq.irq;
+ hptimer->irq.irq = default_hptimer_irq.irq;
vtimer->host_timer_irq = host_vtimer_irq;
ptimer->host_timer_irq = host_ptimer_irq;
+ hvtimer->host_timer_irq = host_vtimer_irq;
+ hptimer->host_timer_irq = host_ptimer_irq;
vtimer->host_timer_irq_flags = host_vtimer_irq_flags;
ptimer->host_timer_irq_flags = host_ptimer_irq_flags;
+ hvtimer->host_timer_irq_flags = host_vtimer_irq_flags;
+ hptimer->host_timer_irq_flags = host_ptimer_irq_flags;
}
static void kvm_timer_init_interrupt(void *info)
@@ -900,6 +1017,10 @@ static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu,
val = kvm_phys_timer_read() - timer_get_offset(timer);
break;
+ case TIMER_REG_VOFF:
+ val = timer_get_offset(timer);
+ break;
+
default:
BUG();
}
@@ -942,6 +1063,10 @@ static void kvm_arm_timer_write(struct kvm_vcpu *vcpu,
timer_set_cval(timer, val);
break;
+ case TIMER_REG_VOFF:
+ timer_set_offset(timer, val);
+ break;
+
default:
BUG();
}
@@ -1079,7 +1204,7 @@ void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)
static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu)
{
- int vtimer_irq, ptimer_irq;
+ int vtimer_irq, ptimer_irq, hvtimer_irq, hptimer_irq;
int i, ret;
vtimer_irq = vcpu_vtimer(vcpu)->irq.irq;
@@ -1092,9 +1217,21 @@ static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu)
if (ret)
return false;
+ hvtimer_irq = vcpu_hvtimer(vcpu)->irq.irq;
+ ret = kvm_vgic_set_owner(vcpu, hvtimer_irq, vcpu_hvtimer(vcpu));
+ if (ret)
+ return false;
+
+ hptimer_irq = vcpu_hptimer(vcpu)->irq.irq;
+ ret = kvm_vgic_set_owner(vcpu, hptimer_irq, vcpu_hptimer(vcpu));
+ if (ret)
+ return false;
+
kvm_for_each_vcpu(i, vcpu, vcpu->kvm) {
if (vcpu_vtimer(vcpu)->irq.irq != vtimer_irq ||
- vcpu_ptimer(vcpu)->irq.irq != ptimer_irq)
+ vcpu_ptimer(vcpu)->irq.irq != ptimer_irq ||
+ vcpu_hvtimer(vcpu)->irq.irq != hvtimer_irq ||
+ vcpu_hptimer(vcpu)->irq.irq != hptimer_irq)
return false;
}
@@ -1110,6 +1247,10 @@ bool kvm_arch_timer_get_input_level(int vintid)
timer = vcpu_vtimer(vcpu);
else if (vintid == vcpu_ptimer(vcpu)->irq.irq)
timer = vcpu_ptimer(vcpu);
+ else if (vintid == vcpu_hvtimer(vcpu)->irq.irq)
+ timer = vcpu_hvtimer(vcpu);
+ else if (vintid == vcpu_hptimer(vcpu)->irq.irq)
+ timer = vcpu_hptimer(vcpu);
else
BUG();
@@ -1191,6 +1332,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq)
kvm_for_each_vcpu(i, vcpu, kvm) {
vcpu_vtimer(vcpu)->irq.irq = vtimer_irq;
vcpu_ptimer(vcpu)->irq.irq = ptimer_irq;
+ /* TODO: Add support for hv/hp timers */
}
}
@@ -1201,6 +1343,8 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
int irq;
+ /* TODO: Add support for hv/hp timers */
+
if (!irqchip_in_kernel(vcpu->kvm))
return -EINVAL;
@@ -1233,6 +1377,8 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
struct arch_timer_context *timer;
int irq;
+ /* TODO: Add support for hv/hp timers */
+
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
timer = vcpu_vtimer(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index ecd61a6b06d1..98ff5b42a6b5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1290,6 +1290,11 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
tmr = TIMER_PTIMER;
treg = TIMER_REG_CVAL;
break;
+ case SYS_CNTVOFF_EL2:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_VOFF;
+ break;
+
default:
BUG();
}
@@ -2101,7 +2106,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
- { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
+ { SYS_DESC(SYS_CNTVOFF_EL2), access_arch_timer },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
{ SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
index 5707011c4f47..49f4cb52d35d 100644
--- a/arch/arm64/kvm/trace_arm.h
+++ b/arch/arm64/kvm/trace_arm.h
@@ -272,6 +272,7 @@ TRACE_EVENT(kvm_get_timer_map,
__field( unsigned long, vcpu_id )
__field( int, direct_vtimer )
__field( int, direct_ptimer )
+ __field( int, emul_vtimer )
__field( int, emul_ptimer )
),
@@ -280,14 +281,17 @@ TRACE_EVENT(kvm_get_timer_map,
__entry->direct_vtimer = arch_timer_ctx_index(map->direct_vtimer);
__entry->direct_ptimer =
(map->direct_ptimer) ? arch_timer_ctx_index(map->direct_ptimer) : -1;
+ __entry->emul_vtimer =
+ (map->emul_vtimer) ? arch_timer_ctx_index(map->emul_vtimer) : -1;
__entry->emul_ptimer =
(map->emul_ptimer) ? arch_timer_ctx_index(map->emul_ptimer) : -1;
),
- TP_printk("VCPU: %ld, dv: %d, dp: %d, ep: %d",
+ TP_printk("VCPU: %ld, dv: %d, dp: %d, ev: %d, ep: %d",
__entry->vcpu_id,
__entry->direct_vtimer,
__entry->direct_ptimer,
+ __entry->emul_vtimer,
__entry->emul_ptimer)
);
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 1c597c9885fa..2af93dc80689 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -573,6 +573,21 @@ int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
return 0;
}
+int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid)
+{
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+ unsigned long flags;
+ int ret = -1;
+
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+ if (irq->hw)
+ ret = irq->hwintid;
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ return ret;
+}
+
/**
* kvm_vgic_set_owner - Set the owner of an interrupt for a VM
*
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 51c19381108c..063f613fbc7e 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -13,6 +13,8 @@
enum kvm_arch_timers {
TIMER_PTIMER,
TIMER_VTIMER,
+ TIMER_HVTIMER,
+ TIMER_HPTIMER,
NR_KVM_TIMERS
};
@@ -21,6 +23,7 @@ enum kvm_arch_timer_regs {
TIMER_REG_CVAL,
TIMER_REG_TVAL,
TIMER_REG_CTL,
+ TIMER_REG_VOFF,
};
struct arch_timer_context {
@@ -47,6 +50,7 @@ struct arch_timer_context {
struct timer_map {
struct arch_timer_context *direct_vtimer;
struct arch_timer_context *direct_ptimer;
+ struct arch_timer_context *emul_vtimer;
struct arch_timer_context *emul_ptimer;
};
@@ -91,6 +95,8 @@ bool kvm_arch_timer_get_input_level(int vintid);
#define vcpu_get_timer(v,t) (&vcpu_timer(v)->timers[(t)])
#define vcpu_vtimer(v) (&(v)->arch.timer_cpu.timers[TIMER_VTIMER])
#define vcpu_ptimer(v) (&(v)->arch.timer_cpu.timers[TIMER_PTIMER])
+#define vcpu_hvtimer(v) (&(v)->arch.timer_cpu.timers[TIMER_HVTIMER])
+#define vcpu_hptimer(v) (&(v)->arch.timer_cpu.timers[TIMER_HPTIMER])
#define arch_timer_ctx_index(ctx) ((ctx) - vcpu_timer((ctx)->vcpu)->timers)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 3d74f1060bd1..4749c4748e5a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -353,6 +353,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
u32 vintid, bool (*get_input_level)(int vindid));
int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
+int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid);
bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 44/66] KVM: arm64: nv: Add handling of EL2-specific timer registers
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Add the required handling for EL2 and EL02 registers, as
well as EL1 registers used in the E2H context.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/sysreg.h | 6 +++
arch/arm64/kvm/sys_regs.c | 87 +++++++++++++++++++++++++++++++++
2 files changed, 93 insertions(+)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 814fb57b55d9..d047954400d4 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -559,6 +559,12 @@
#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
+#define SYS_CNTHP_TVAL_EL2 sys_reg(3, 4, 14, 2, 0)
+#define SYS_CNTHP_CTL_EL2 sys_reg(3, 4, 14, 2, 1)
+#define SYS_CNTHP_CVAL_EL2 sys_reg(3, 4, 14, 2, 2)
+#define SYS_CNTHV_TVAL_EL2 sys_reg(3, 4, 14, 3, 0)
+#define SYS_CNTHV_CTL_EL2 sys_reg(3, 4, 14, 3, 1)
+#define SYS_CNTHV_CVAL_EL2 sys_reg(3, 4, 14, 3, 2)
/* VHE encodings for architectural EL0/1 system registers */
#define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 98ff5b42a6b5..560b44325e98 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1276,20 +1276,92 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
switch (reg) {
case SYS_CNTP_TVAL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
case SYS_AARCH32_CNTP_TVAL:
+ case SYS_CNTP_TVAL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_TVAL;
break;
+
+ case SYS_CNTV_TVAL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
+ case SYS_CNTHP_TVAL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
+ case SYS_CNTHV_TVAL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
case SYS_CNTP_CTL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
case SYS_AARCH32_CNTP_CTL:
+ case SYS_CNTP_CTL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_CTL;
break;
+
+ case SYS_CNTV_CTL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
+ case SYS_CNTHP_CTL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
+ case SYS_CNTHV_CTL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
case SYS_CNTP_CVAL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
case SYS_AARCH32_CNTP_CVAL:
+ case SYS_CNTP_CVAL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_CVAL;
break;
+
+ case SYS_CNTV_CVAL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
+ case SYS_CNTHP_CVAL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
+ case SYS_CNTHV_CVAL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
case SYS_CNTVOFF_EL2:
tmr = TIMER_VTIMER;
treg = TIMER_REG_VOFF;
@@ -2109,6 +2181,13 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CNTVOFF_EL2), access_arch_timer },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+ { SYS_DESC(SYS_CNTHP_TVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHP_CTL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHP_CVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_TVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_CTL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_CVAL_EL2), access_arch_timer },
+
{ SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_CPACR_EL12), access_rw, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_TTBR0_EL12), access_vm_reg, reset_unknown, TTBR0_EL1 },
@@ -2126,6 +2205,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CONTEXTIDR_EL12), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
{ SYS_DESC(SYS_CNTKCTL_EL12), access_rw, reset_val, CNTKCTL_EL1, 0 },
+ { SYS_DESC(SYS_CNTP_TVAL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTP_CTL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTP_CVAL_EL02), access_arch_timer },
+
+ { SYS_DESC(SYS_CNTV_TVAL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTV_CTL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTV_CVAL_EL02), access_arch_timer },
+
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 44/66] KVM: arm64: nv: Add handling of EL2-specific timer registers
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Add the required handling for EL2 and EL02 registers, as
well as EL1 registers used in the E2H context.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/sysreg.h | 6 +++
arch/arm64/kvm/sys_regs.c | 87 +++++++++++++++++++++++++++++++++
2 files changed, 93 insertions(+)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 814fb57b55d9..d047954400d4 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -559,6 +559,12 @@
#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
+#define SYS_CNTHP_TVAL_EL2 sys_reg(3, 4, 14, 2, 0)
+#define SYS_CNTHP_CTL_EL2 sys_reg(3, 4, 14, 2, 1)
+#define SYS_CNTHP_CVAL_EL2 sys_reg(3, 4, 14, 2, 2)
+#define SYS_CNTHV_TVAL_EL2 sys_reg(3, 4, 14, 3, 0)
+#define SYS_CNTHV_CTL_EL2 sys_reg(3, 4, 14, 3, 1)
+#define SYS_CNTHV_CVAL_EL2 sys_reg(3, 4, 14, 3, 2)
/* VHE encodings for architectural EL0/1 system registers */
#define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 98ff5b42a6b5..560b44325e98 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1276,20 +1276,92 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
switch (reg) {
case SYS_CNTP_TVAL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
case SYS_AARCH32_CNTP_TVAL:
+ case SYS_CNTP_TVAL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_TVAL;
break;
+
+ case SYS_CNTV_TVAL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
+ case SYS_CNTHP_TVAL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
+ case SYS_CNTHV_TVAL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
case SYS_CNTP_CTL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
case SYS_AARCH32_CNTP_CTL:
+ case SYS_CNTP_CTL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_CTL;
break;
+
+ case SYS_CNTV_CTL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
+ case SYS_CNTHP_CTL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
+ case SYS_CNTHV_CTL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
case SYS_CNTP_CVAL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
case SYS_AARCH32_CNTP_CVAL:
+ case SYS_CNTP_CVAL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_CVAL;
break;
+
+ case SYS_CNTV_CVAL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
+ case SYS_CNTHP_CVAL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
+ case SYS_CNTHV_CVAL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
case SYS_CNTVOFF_EL2:
tmr = TIMER_VTIMER;
treg = TIMER_REG_VOFF;
@@ -2109,6 +2181,13 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CNTVOFF_EL2), access_arch_timer },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+ { SYS_DESC(SYS_CNTHP_TVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHP_CTL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHP_CVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_TVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_CTL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_CVAL_EL2), access_arch_timer },
+
{ SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_CPACR_EL12), access_rw, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_TTBR0_EL12), access_vm_reg, reset_unknown, TTBR0_EL1 },
@@ -2126,6 +2205,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CONTEXTIDR_EL12), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
{ SYS_DESC(SYS_CNTKCTL_EL12), access_rw, reset_val, CNTKCTL_EL1, 0 },
+ { SYS_DESC(SYS_CNTP_TVAL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTP_CTL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTP_CVAL_EL02), access_arch_timer },
+
+ { SYS_DESC(SYS_CNTV_TVAL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTV_CTL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTV_CVAL_EL02), access_arch_timer },
+
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 44/66] KVM: arm64: nv: Add handling of EL2-specific timer registers
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Add the required handling for EL2 and EL02 registers, as
well as EL1 registers used in the E2H context.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/sysreg.h | 6 +++
arch/arm64/kvm/sys_regs.c | 87 +++++++++++++++++++++++++++++++++
2 files changed, 93 insertions(+)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 814fb57b55d9..d047954400d4 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -559,6 +559,12 @@
#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
+#define SYS_CNTHP_TVAL_EL2 sys_reg(3, 4, 14, 2, 0)
+#define SYS_CNTHP_CTL_EL2 sys_reg(3, 4, 14, 2, 1)
+#define SYS_CNTHP_CVAL_EL2 sys_reg(3, 4, 14, 2, 2)
+#define SYS_CNTHV_TVAL_EL2 sys_reg(3, 4, 14, 3, 0)
+#define SYS_CNTHV_CTL_EL2 sys_reg(3, 4, 14, 3, 1)
+#define SYS_CNTHV_CVAL_EL2 sys_reg(3, 4, 14, 3, 2)
/* VHE encodings for architectural EL0/1 system registers */
#define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 98ff5b42a6b5..560b44325e98 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1276,20 +1276,92 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu,
switch (reg) {
case SYS_CNTP_TVAL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
case SYS_AARCH32_CNTP_TVAL:
+ case SYS_CNTP_TVAL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_TVAL;
break;
+
+ case SYS_CNTV_TVAL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
+ case SYS_CNTHP_TVAL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
+ case SYS_CNTHV_TVAL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_TVAL;
+ break;
+
case SYS_CNTP_CTL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
case SYS_AARCH32_CNTP_CTL:
+ case SYS_CNTP_CTL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_CTL;
break;
+
+ case SYS_CNTV_CTL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
+ case SYS_CNTHP_CTL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
+ case SYS_CNTHV_CTL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_CTL;
+ break;
+
case SYS_CNTP_CVAL_EL0:
+ if (vcpu_mode_el2(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ tmr = TIMER_HPTIMER;
+ else
+ tmr = TIMER_PTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
case SYS_AARCH32_CNTP_CVAL:
+ case SYS_CNTP_CVAL_EL02:
tmr = TIMER_PTIMER;
treg = TIMER_REG_CVAL;
break;
+
+ case SYS_CNTV_CVAL_EL02:
+ tmr = TIMER_VTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
+ case SYS_CNTHP_CVAL_EL2:
+ tmr = TIMER_HPTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
+ case SYS_CNTHV_CVAL_EL2:
+ tmr = TIMER_HVTIMER;
+ treg = TIMER_REG_CVAL;
+ break;
+
case SYS_CNTVOFF_EL2:
tmr = TIMER_VTIMER;
treg = TIMER_REG_VOFF;
@@ -2109,6 +2181,13 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CNTVOFF_EL2), access_arch_timer },
{ SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
+ { SYS_DESC(SYS_CNTHP_TVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHP_CTL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHP_CVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_TVAL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_CTL_EL2), access_arch_timer },
+ { SYS_DESC(SYS_CNTHV_CVAL_EL2), access_arch_timer },
+
{ SYS_DESC(SYS_SCTLR_EL12), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_CPACR_EL12), access_rw, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_TTBR0_EL12), access_vm_reg, reset_unknown, TTBR0_EL1 },
@@ -2126,6 +2205,14 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_CONTEXTIDR_EL12), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
{ SYS_DESC(SYS_CNTKCTL_EL12), access_rw, reset_val, CNTKCTL_EL1, 0 },
+ { SYS_DESC(SYS_CNTP_TVAL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTP_CTL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTP_CVAL_EL02), access_arch_timer },
+
+ { SYS_DESC(SYS_CNTV_TVAL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTV_CTL_EL02), access_arch_timer },
+ { SYS_DESC(SYS_CNTV_CVAL_EL02), access_arch_timer },
+
{ SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
};
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 45/66] KVM: arm64: nv: Load timer before the GIC
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
In order for vgic_v3_load_nested to be able to observe which timer
interrupts have the HW bit set for the current context, the timers
must have been loaded in the new mode and the right timer mapped
to their corresponding HW IRQs.
At the moment, we load the GIC first, meaning that timer interrupts
injected to an L2 guest will never have the HW bit set (we see the
old configuration).
Swapping the two loads solves this particular problem.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 1656dd80bbc4..2e042185e625 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -405,8 +405,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
vcpu->cpu = cpu;
- kvm_vgic_load(vcpu);
kvm_timer_vcpu_load(vcpu);
+ kvm_vgic_load(vcpu);
if (has_vhe())
kvm_vcpu_load_sysregs_vhe(vcpu);
kvm_arch_vcpu_load_fp(vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 45/66] KVM: arm64: nv: Load timer before the GIC
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
In order for vgic_v3_load_nested to be able to observe which timer
interrupts have the HW bit set for the current context, the timers
must have been loaded in the new mode and the right timer mapped
to their corresponding HW IRQs.
At the moment, we load the GIC first, meaning that timer interrupts
injected to an L2 guest will never have the HW bit set (we see the
old configuration).
Swapping the two loads solves this particular problem.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 1656dd80bbc4..2e042185e625 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -405,8 +405,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
vcpu->cpu = cpu;
- kvm_vgic_load(vcpu);
kvm_timer_vcpu_load(vcpu);
+ kvm_vgic_load(vcpu);
if (has_vhe())
kvm_vcpu_load_sysregs_vhe(vcpu);
kvm_arch_vcpu_load_fp(vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 45/66] KVM: arm64: nv: Load timer before the GIC
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
In order for vgic_v3_load_nested to be able to observe which timer
interrupts have the HW bit set for the current context, the timers
must have been loaded in the new mode and the right timer mapped
to their corresponding HW IRQs.
At the moment, we load the GIC first, meaning that timer interrupts
injected to an L2 guest will never have the HW bit set (we see the
old configuration).
Swapping the two loads solves this particular problem.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 1656dd80bbc4..2e042185e625 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -405,8 +405,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
vcpu->cpu = cpu;
- kvm_vgic_load(vcpu);
kvm_timer_vcpu_load(vcpu);
+ kvm_vgic_load(vcpu);
if (has_vhe())
kvm_vcpu_load_sysregs_vhe(vcpu);
kvm_arch_vcpu_load_fp(vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 46/66] KVM: arm64: nv: Nested GICv3 Support
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Jintack Lim <jintack@cs.columbia.edu>
When entering a nested VM, we set up the hypervisor control interface
based on what the guest hypervisor has set. Especially, we investigate
each list register written by the guest hypervisor whether HW bit is
set. If so, we translate hw irq number from the guest's point of view
to the real hardware irq number if there is a mapping.
Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
[Rewritten to support GICv3 instead of GICv2]
Signed-off-by: Marc Zyngier <maz@kernel.org>
[Redesigned execution flow around vcpu load/put]
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
---
arch/arm64/include/asm/kvm_emulate.h | 8 +-
arch/arm64/include/asm/kvm_host.h | 9 +-
arch/arm64/include/asm/kvm_nested.h | 1 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 13 +-
arch/arm64/kvm/nested.c | 16 +++
arch/arm64/kvm/sys_regs.c | 179 ++++++++++++++++++++++++++-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 178 ++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3.c | 26 ++++
arch/arm64/kvm/vgic/vgic.c | 27 ++++
include/kvm/arm_vgic.h | 18 +++
11 files changed, 468 insertions(+), 9 deletions(-)
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index b447817d9dee..44b395854430 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -497,7 +497,13 @@ static inline bool kvm_is_write_fault(struct kvm_vcpu *vcpu)
static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
{
- return vcpu_read_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
+ /*
+ * Use the in-memory view for MPIDR_EL1. It can't be changed by the
+ * guest, and is also accessed from the context of *another* vcpu,
+ * so anything using some other state (such as the NV state that is
+ * used by vcpu_read_sys_reg) will eventually go wrong.
+ */
+ return __vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
}
static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index f3743e45ded1..bd91b4219d5e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -42,10 +42,11 @@
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
-#define KVM_REQ_IRQ_PENDING KVM_ARCH_REQ(1)
-#define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2)
-#define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3)
-#define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4)
+#define KVM_REQ_IRQ_PENDING KVM_ARCH_REQ(1)
+#define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2)
+#define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3)
+#define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4)
+#define KVM_REQ_GUEST_HYP_IRQ_PENDING KVM_ARCH_REQ(5)
#define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \
KVM_DIRTY_LOG_INITIALLY_SET)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 2987806850f0..5fa3c634c8e1 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -67,6 +67,7 @@ extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+extern void check_nested_vcpu_requests(struct kvm_vcpu *vcpu);
struct kvm_s2_trans {
phys_addr_t output;
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 8a23991f82b2..0c15321c0827 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -23,5 +23,5 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
- vgic/vgic-its.o vgic/vgic-debug.o
+ vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o
kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2e042185e625..818ec91801e2 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -692,6 +692,8 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu)
vgic_v4_load(vcpu);
preempt_enable();
}
+
+ check_nested_vcpu_requests(vcpu);
}
}
@@ -738,9 +740,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
*/
cond_resched();
- update_vmid(&vcpu->arch.hw_mmu->vmid);
-
+ /*
+ * A nested exeption triggered by a vcpu request (such
+ * as an interrupt injected in a guest hypervisor) can
+ * change the currently used VMID (by switching to a
+ * different translation regime. It is thus necesary
+ * to update the VMID *after* all requests have been
+ * processed.
+ */
check_vcpu_requests(vcpu);
+ update_vmid(&vcpu->arch.hw_mmu->vmid);
/*
* Preparing the interrupts to be injected also
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index e78c6c093afc..9574fde823e6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -580,6 +580,22 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm)
kvm_free_stage2_pgd(&kvm->arch.mmu);
}
+bool vgic_state_is_nested(struct kvm_vcpu *vcpu)
+{
+ bool imo = __vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO;
+ bool fmo = __vcpu_sys_reg(vcpu, HCR_EL2) & HCR_FMO;
+
+ WARN_ONCE(imo != fmo, "Separate virtual IRQ/FIQ settings not supported\n");
+
+ return nested_virt_in_use(vcpu) && imo && fmo && !is_hyp_ctxt(vcpu);
+}
+
+void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)
+{
+ if (kvm_check_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu))
+ kvm_inject_nested_irq(vcpu);
+}
+
#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
/*
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 560b44325e98..60d2fb6edc93 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -15,6 +15,8 @@
#include <linux/printk.h>
#include <linux/uaccess.h>
+#include <linux/irqchip/arm-gic-v3.h>
+
#include <asm/cacheflush.h>
#include <asm/cputype.h>
#include <asm/debug-monitors.h>
@@ -455,6 +457,19 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
return true;
}
+/*
+ * The architecture says that non-secure write accesses to this register from
+ * EL1 are trapped to EL2, if either:
+ * - HCR_EL2.FMO==1, or
+ * - HCR_EL2.IMO==1
+ */
+static bool sgi_traps_to_vel2(struct kvm_vcpu *vcpu)
+{
+ return (nested_virt_in_use(vcpu) &&
+ !vcpu_mode_el2(vcpu) &&
+ !!(vcpu_read_sys_reg(vcpu, HCR_EL2) & (HCR_IMO | HCR_FMO)));
+}
+
/*
* Trap handler for the GICv3 SGI generation system register.
* Forward the request to the VGIC emulation.
@@ -470,6 +485,11 @@ static bool access_gic_sgi(struct kvm_vcpu *vcpu,
if (!p->is_write)
return read_from_write_only(vcpu, p, r);
+ if (sgi_traps_to_vel2(vcpu)) {
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+ return false;
+ }
+
/*
* In a system where GICD_CTLR.DS=1, a ICC_SGI0R_EL1 access generates
* Group0 SGIs only, while ICC_SGI1R_EL1 can generate either group,
@@ -513,7 +533,13 @@ static bool access_gic_sre(struct kvm_vcpu *vcpu,
if (p->is_write)
return ignore_write(vcpu, p);
- p->regval = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+ if (p->Op1 == 4) { /* ICC_SRE_EL2 */
+ p->regval = (ICC_SRE_EL2_ENABLE | ICC_SRE_EL2_SRE |
+ ICC_SRE_EL1_DIB | ICC_SRE_EL1_DFB);
+ } else { /* ICC_SRE_EL1 */
+ p->regval = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+ }
+
return true;
}
@@ -1759,6 +1785,122 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_gic_apr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ u32 index, *base;
+
+ index = r->Op2;
+ if (r->CRm == 8)
+ base = cpu_if->vgic_ap0r;
+ else
+ base = cpu_if->vgic_ap1r;
+
+ if (p->is_write)
+ base[index] = p->regval;
+ else
+ p->regval = base[index];
+
+ return true;
+}
+
+static bool access_gic_hcr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
+ if (p->is_write)
+ cpu_if->vgic_hcr = p->regval;
+ else
+ p->regval = cpu_if->vgic_hcr;
+
+ return true;
+}
+
+static bool access_gic_vtr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = kvm_vgic_global_state.ich_vtr_el2;
+
+ return true;
+}
+
+static bool access_gic_misr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_misr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_eisr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_eisr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_elrsr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_elrsr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_vmcr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
+ if (p->is_write)
+ cpu_if->vgic_vmcr = p->regval;
+ else
+ p->regval = cpu_if->vgic_vmcr;
+
+ return true;
+}
+
+static bool access_gic_lr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ u32 index;
+
+ index = p->Op2;
+ if (p->CRm == 13)
+ index += 8;
+
+ if (p->is_write)
+ cpu_if->vgic_lr[index] = p->regval;
+ else
+ p->regval = cpu_if->vgic_lr[index];
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -2175,6 +2317,41 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
{ SYS_DESC(SYS_VDISR_EL2), trap_undef },
+ { SYS_DESC(SYS_ICH_AP0R0_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R1_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R2_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R3_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R0_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R1_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R2_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R3_EL2), access_gic_apr },
+
+ { SYS_DESC(SYS_ICC_SRE_EL2), access_gic_sre },
+
+ { SYS_DESC(SYS_ICH_HCR_EL2), access_gic_hcr },
+ { SYS_DESC(SYS_ICH_VTR_EL2), access_gic_vtr },
+ { SYS_DESC(SYS_ICH_MISR_EL2), access_gic_misr },
+ { SYS_DESC(SYS_ICH_EISR_EL2), access_gic_eisr },
+ { SYS_DESC(SYS_ICH_ELRSR_EL2), access_gic_elrsr },
+ { SYS_DESC(SYS_ICH_VMCR_EL2), access_gic_vmcr },
+
+ { SYS_DESC(SYS_ICH_LR0_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR1_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR2_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR3_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR4_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR5_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR6_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR7_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR8_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR9_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR10_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR11_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR12_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR13_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR14_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR15_EL2), access_gic_lr },
+
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
new file mode 100644
index 000000000000..bfe0ccde514f
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -0,0 +1,178 @@
+#include <linux/cpu.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/uaccess.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <kvm/arm_vgic.h>
+
+#include "vgic.h"
+
+static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->arch.vgic_cpu.nested_vgic_v3;
+}
+
+static inline struct vgic_v3_cpu_if *vcpu_shadow_if(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+}
+
+static inline bool lr_triggers_eoi(u64 lr)
+{
+ return !(lr & (ICH_LR_STATE | ICH_LR_HW)) && (lr & ICH_LR_EOI);
+}
+
+u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ u16 reg = 0;
+ int i;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ if (lr_triggers_eoi(cpu_if->vgic_lr[i]))
+ reg |= BIT(i);
+ }
+
+ return reg;
+}
+
+u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ u16 reg = 0;
+ int i;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ if (!(cpu_if->vgic_lr[i] & ICH_LR_STATE))
+ reg |= BIT(i);
+ }
+
+ return reg;
+}
+
+u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ int nr_lr = kvm_vgic_global_state.nr_lr;
+ u64 reg = 0;
+
+ if (vgic_v3_get_eisr(vcpu))
+ reg |= ICH_MISR_EOI;
+
+ if (cpu_if->vgic_hcr & ICH_HCR_UIE) {
+ int used_lrs;
+
+ used_lrs = nr_lr - hweight16(vgic_v3_get_elrsr(vcpu));
+ if (used_lrs <= 1)
+ reg |= ICH_MISR_U;
+ }
+
+ /* TODO: Support remaining bits in this register */
+ return reg;
+}
+
+/*
+ * For LRs which have HW bit set such as timer interrupts, we modify them to
+ * have the host hardware interrupt number instead of the virtual one programmed
+ * by the guest hypervisor.
+ */
+static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ struct vgic_irq *irq;
+ int i, used_lrs = 0;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ u64 lr = cpu_if->vgic_lr[i];
+ int l1_irq;
+
+ if (!(lr & ICH_LR_HW))
+ goto next;
+
+ /* We have the HW bit set */
+ l1_irq = (lr & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT;
+ irq = vgic_get_irq(vcpu->kvm, vcpu, l1_irq);
+
+ if (!irq || !irq->hw) {
+ /* There was no real mapping, so nuke the HW bit */
+ lr &= ~ICH_LR_HW;
+ if (irq)
+ vgic_put_irq(vcpu->kvm, irq);
+ goto next;
+ }
+
+ /* Translate the virtual mapping to the real one */
+ lr &= ~ICH_LR_EOI; /* Why? */
+ lr &= ~ICH_LR_PHYS_ID_MASK;
+ lr |= (u64)irq->hwintid << ICH_LR_PHYS_ID_SHIFT;
+ vgic_put_irq(vcpu->kvm, irq);
+
+next:
+ s_cpu_if->vgic_lr[i] = lr;
+ used_lrs = i + 1;
+ }
+
+ s_cpu_if->used_lrs = used_lrs;
+}
+
+/*
+ * Change the shadow HWIRQ field back to the virtual value before copying over
+ * the entire shadow struct to the nested state.
+ */
+static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ int lr;
+
+ for (lr = 0; lr < kvm_vgic_global_state.nr_lr; lr++) {
+ s_cpu_if->vgic_lr[lr] &= ~ICH_LR_PHYS_ID_MASK;
+ s_cpu_if->vgic_lr[lr] |= cpu_if->vgic_lr[lr] & ICH_LR_PHYS_ID_MASK;
+ }
+}
+
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
+ vgic_v3_create_shadow_lr(vcpu);
+ __vgic_v3_restore_state(vcpu_shadow_if(vcpu));
+}
+
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ __vgic_v3_save_state(vcpu_shadow_if(vcpu));
+
+ /*
+ * Translate the shadow state HW fields back to the virtual ones
+ * before copying the shadow struct back to the nested one.
+ */
+ vgic_v3_fixup_shadow_lr_state(vcpu);
+ vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+}
+
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+
+ /*
+ * If we exit a nested VM with a pending maintenance interrupt from the
+ * GIC, then we need to forward this to the guest hypervisor so that it
+ * can re-sync the appropriate LRs and sample level triggered interrupts
+ * again.
+ */
+ if (vgic_state_is_nested(vcpu) &&
+ (cpu_if->vgic_hcr & ICH_HCR_EN) &&
+ vgic_v3_get_misr(vcpu))
+ kvm_inject_nested_irq(vcpu);
+}
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index 9cdf39a94a63..e4c5c27552da 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -6,6 +6,7 @@
#include <kvm/arm_vgic.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/kvm_asm.h>
#include "vgic.h"
@@ -289,6 +290,12 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
vgic_v3->vgic_sre = (ICC_SRE_EL1_DIB |
ICC_SRE_EL1_DFB |
ICC_SRE_EL1_SRE);
+ /*
+ * If nesting is allowed, force GICv3 onto the nested
+ * guests as well.
+ */
+ if (nested_virt_in_use(vcpu))
+ vcpu->arch.vgic_cpu.nested_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
@@ -654,6 +661,13 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ /*
+ * vgic_v3_load_nested only affects the LRs in the shadow
+ * state, so it is fine to pass the nested state around.
+ */
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
/*
* If dealing with a GICv2 emulation on GICv3, VMCR_EL2.VFIQen
* is dependent on ICC_SRE_EL1.SRE, and we have to perform the
@@ -667,6 +681,9 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
if (has_vhe())
__vgic_v3_activate_traps(cpu_if);
+ if (vgic_state_is_nested(vcpu))
+ vgic_v3_load_nested(vcpu);
+
WARN_ON(vgic_v4_load(vcpu));
}
@@ -674,6 +691,9 @@ void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+
if (likely(cpu_if->vgic_sre))
cpu_if->vgic_vmcr = kvm_call_hyp_ret(__vgic_v3_read_vmcr);
}
@@ -686,8 +706,14 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
vgic_v3_vmcr_sync(vcpu);
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+
kvm_call_hyp(__vgic_v3_save_aprs, cpu_if);
if (has_vhe())
__vgic_v3_deactivate_traps(cpu_if);
+
+ if (vgic_state_is_nested(vcpu))
+ vgic_v3_put_nested(vcpu);
}
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 2af93dc80689..eae63e5c8322 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -876,6 +876,10 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
int used_lrs;
+ /* If nesting, this is a load/put affair, not flush/sync. */
+ if (vgic_state_is_nested(vcpu))
+ return;
+
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
@@ -920,6 +924,29 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
!vgic_supports_direct_msis(vcpu->kvm))
return;
+ /*
+ * If in a nested state, we must return early. Two possibilities:
+ *
+ * - If we have any pending IRQ for the guest and the guest
+ * expects IRQs to be handled in its virtual EL2 mode (the
+ * virtual IMO bit is set) and it is not already running in
+ * virtual EL2 mode, then we have to emulate an IRQ
+ * exception to virtual EL2.
+ *
+ * We do that by placing a request to ourselves which will
+ * abort the entry procedure and inject the exception at the
+ * beginning of the run loop.
+ *
+ * - Otherwise, do exactly *NOTHING*. The guest state is
+ * already loaded, and we can carry on with running it.
+ */
+ if (vgic_state_is_nested(vcpu)) {
+ if (kvm_vgic_vcpu_pending_irq(vcpu))
+ kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
+
+ return;
+ }
+
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
if (!list_empty(&vcpu->arch.vgic_cpu.ap_list_head)) {
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4749c4748e5a..4c6e61398f63 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -306,6 +306,15 @@ struct vgic_cpu {
struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
+ /* CPU vif control registers for the virtual GICH interface */
+ struct vgic_v3_cpu_if nested_vgic_v3;
+
+ /*
+ * The shadow vif control register loaded to the hardware when
+ * running a nested L2 guest with the virtual IMO/FMO bit set.
+ */
+ struct vgic_v3_cpu_if shadow_vgic_v3;
+
raw_spinlock_t ap_list_lock; /* Protects the ap_list */
/*
@@ -362,6 +371,13 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
+u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu);
+u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu);
+u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu);
+
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k) ((k)->arch.vgic.initialized)
#define vgic_ready(k) ((k)->arch.vgic.ready)
@@ -406,4 +422,6 @@ int vgic_v4_load(struct kvm_vcpu *vcpu);
void vgic_v4_commit(struct kvm_vcpu *vcpu);
int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
+
#endif /* __KVM_ARM_VGIC_H */
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 46/66] KVM: arm64: nv: Nested GICv3 Support
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Jintack Lim <jintack@cs.columbia.edu>
When entering a nested VM, we set up the hypervisor control interface
based on what the guest hypervisor has set. Especially, we investigate
each list register written by the guest hypervisor whether HW bit is
set. If so, we translate hw irq number from the guest's point of view
to the real hardware irq number if there is a mapping.
Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
[Rewritten to support GICv3 instead of GICv2]
Signed-off-by: Marc Zyngier <maz@kernel.org>
[Redesigned execution flow around vcpu load/put]
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
---
arch/arm64/include/asm/kvm_emulate.h | 8 +-
arch/arm64/include/asm/kvm_host.h | 9 +-
arch/arm64/include/asm/kvm_nested.h | 1 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 13 +-
arch/arm64/kvm/nested.c | 16 +++
arch/arm64/kvm/sys_regs.c | 179 ++++++++++++++++++++++++++-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 178 ++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3.c | 26 ++++
arch/arm64/kvm/vgic/vgic.c | 27 ++++
include/kvm/arm_vgic.h | 18 +++
11 files changed, 468 insertions(+), 9 deletions(-)
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index b447817d9dee..44b395854430 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -497,7 +497,13 @@ static inline bool kvm_is_write_fault(struct kvm_vcpu *vcpu)
static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
{
- return vcpu_read_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
+ /*
+ * Use the in-memory view for MPIDR_EL1. It can't be changed by the
+ * guest, and is also accessed from the context of *another* vcpu,
+ * so anything using some other state (such as the NV state that is
+ * used by vcpu_read_sys_reg) will eventually go wrong.
+ */
+ return __vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
}
static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index f3743e45ded1..bd91b4219d5e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -42,10 +42,11 @@
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
-#define KVM_REQ_IRQ_PENDING KVM_ARCH_REQ(1)
-#define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2)
-#define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3)
-#define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4)
+#define KVM_REQ_IRQ_PENDING KVM_ARCH_REQ(1)
+#define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2)
+#define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3)
+#define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4)
+#define KVM_REQ_GUEST_HYP_IRQ_PENDING KVM_ARCH_REQ(5)
#define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \
KVM_DIRTY_LOG_INITIALLY_SET)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 2987806850f0..5fa3c634c8e1 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -67,6 +67,7 @@ extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+extern void check_nested_vcpu_requests(struct kvm_vcpu *vcpu);
struct kvm_s2_trans {
phys_addr_t output;
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 8a23991f82b2..0c15321c0827 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -23,5 +23,5 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
- vgic/vgic-its.o vgic/vgic-debug.o
+ vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o
kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2e042185e625..818ec91801e2 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -692,6 +692,8 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu)
vgic_v4_load(vcpu);
preempt_enable();
}
+
+ check_nested_vcpu_requests(vcpu);
}
}
@@ -738,9 +740,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
*/
cond_resched();
- update_vmid(&vcpu->arch.hw_mmu->vmid);
-
+ /*
+ * A nested exeption triggered by a vcpu request (such
+ * as an interrupt injected in a guest hypervisor) can
+ * change the currently used VMID (by switching to a
+ * different translation regime. It is thus necesary
+ * to update the VMID *after* all requests have been
+ * processed.
+ */
check_vcpu_requests(vcpu);
+ update_vmid(&vcpu->arch.hw_mmu->vmid);
/*
* Preparing the interrupts to be injected also
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index e78c6c093afc..9574fde823e6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -580,6 +580,22 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm)
kvm_free_stage2_pgd(&kvm->arch.mmu);
}
+bool vgic_state_is_nested(struct kvm_vcpu *vcpu)
+{
+ bool imo = __vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO;
+ bool fmo = __vcpu_sys_reg(vcpu, HCR_EL2) & HCR_FMO;
+
+ WARN_ONCE(imo != fmo, "Separate virtual IRQ/FIQ settings not supported\n");
+
+ return nested_virt_in_use(vcpu) && imo && fmo && !is_hyp_ctxt(vcpu);
+}
+
+void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)
+{
+ if (kvm_check_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu))
+ kvm_inject_nested_irq(vcpu);
+}
+
#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
/*
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 560b44325e98..60d2fb6edc93 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -15,6 +15,8 @@
#include <linux/printk.h>
#include <linux/uaccess.h>
+#include <linux/irqchip/arm-gic-v3.h>
+
#include <asm/cacheflush.h>
#include <asm/cputype.h>
#include <asm/debug-monitors.h>
@@ -455,6 +457,19 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
return true;
}
+/*
+ * The architecture says that non-secure write accesses to this register from
+ * EL1 are trapped to EL2, if either:
+ * - HCR_EL2.FMO==1, or
+ * - HCR_EL2.IMO==1
+ */
+static bool sgi_traps_to_vel2(struct kvm_vcpu *vcpu)
+{
+ return (nested_virt_in_use(vcpu) &&
+ !vcpu_mode_el2(vcpu) &&
+ !!(vcpu_read_sys_reg(vcpu, HCR_EL2) & (HCR_IMO | HCR_FMO)));
+}
+
/*
* Trap handler for the GICv3 SGI generation system register.
* Forward the request to the VGIC emulation.
@@ -470,6 +485,11 @@ static bool access_gic_sgi(struct kvm_vcpu *vcpu,
if (!p->is_write)
return read_from_write_only(vcpu, p, r);
+ if (sgi_traps_to_vel2(vcpu)) {
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+ return false;
+ }
+
/*
* In a system where GICD_CTLR.DS=1, a ICC_SGI0R_EL1 access generates
* Group0 SGIs only, while ICC_SGI1R_EL1 can generate either group,
@@ -513,7 +533,13 @@ static bool access_gic_sre(struct kvm_vcpu *vcpu,
if (p->is_write)
return ignore_write(vcpu, p);
- p->regval = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+ if (p->Op1 == 4) { /* ICC_SRE_EL2 */
+ p->regval = (ICC_SRE_EL2_ENABLE | ICC_SRE_EL2_SRE |
+ ICC_SRE_EL1_DIB | ICC_SRE_EL1_DFB);
+ } else { /* ICC_SRE_EL1 */
+ p->regval = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+ }
+
return true;
}
@@ -1759,6 +1785,122 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_gic_apr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ u32 index, *base;
+
+ index = r->Op2;
+ if (r->CRm == 8)
+ base = cpu_if->vgic_ap0r;
+ else
+ base = cpu_if->vgic_ap1r;
+
+ if (p->is_write)
+ base[index] = p->regval;
+ else
+ p->regval = base[index];
+
+ return true;
+}
+
+static bool access_gic_hcr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
+ if (p->is_write)
+ cpu_if->vgic_hcr = p->regval;
+ else
+ p->regval = cpu_if->vgic_hcr;
+
+ return true;
+}
+
+static bool access_gic_vtr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = kvm_vgic_global_state.ich_vtr_el2;
+
+ return true;
+}
+
+static bool access_gic_misr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_misr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_eisr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_eisr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_elrsr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_elrsr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_vmcr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
+ if (p->is_write)
+ cpu_if->vgic_vmcr = p->regval;
+ else
+ p->regval = cpu_if->vgic_vmcr;
+
+ return true;
+}
+
+static bool access_gic_lr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ u32 index;
+
+ index = p->Op2;
+ if (p->CRm == 13)
+ index += 8;
+
+ if (p->is_write)
+ cpu_if->vgic_lr[index] = p->regval;
+ else
+ p->regval = cpu_if->vgic_lr[index];
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -2175,6 +2317,41 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
{ SYS_DESC(SYS_VDISR_EL2), trap_undef },
+ { SYS_DESC(SYS_ICH_AP0R0_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R1_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R2_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R3_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R0_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R1_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R2_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R3_EL2), access_gic_apr },
+
+ { SYS_DESC(SYS_ICC_SRE_EL2), access_gic_sre },
+
+ { SYS_DESC(SYS_ICH_HCR_EL2), access_gic_hcr },
+ { SYS_DESC(SYS_ICH_VTR_EL2), access_gic_vtr },
+ { SYS_DESC(SYS_ICH_MISR_EL2), access_gic_misr },
+ { SYS_DESC(SYS_ICH_EISR_EL2), access_gic_eisr },
+ { SYS_DESC(SYS_ICH_ELRSR_EL2), access_gic_elrsr },
+ { SYS_DESC(SYS_ICH_VMCR_EL2), access_gic_vmcr },
+
+ { SYS_DESC(SYS_ICH_LR0_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR1_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR2_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR3_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR4_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR5_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR6_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR7_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR8_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR9_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR10_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR11_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR12_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR13_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR14_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR15_EL2), access_gic_lr },
+
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
new file mode 100644
index 000000000000..bfe0ccde514f
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -0,0 +1,178 @@
+#include <linux/cpu.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/uaccess.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <kvm/arm_vgic.h>
+
+#include "vgic.h"
+
+static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->arch.vgic_cpu.nested_vgic_v3;
+}
+
+static inline struct vgic_v3_cpu_if *vcpu_shadow_if(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+}
+
+static inline bool lr_triggers_eoi(u64 lr)
+{
+ return !(lr & (ICH_LR_STATE | ICH_LR_HW)) && (lr & ICH_LR_EOI);
+}
+
+u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ u16 reg = 0;
+ int i;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ if (lr_triggers_eoi(cpu_if->vgic_lr[i]))
+ reg |= BIT(i);
+ }
+
+ return reg;
+}
+
+u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ u16 reg = 0;
+ int i;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ if (!(cpu_if->vgic_lr[i] & ICH_LR_STATE))
+ reg |= BIT(i);
+ }
+
+ return reg;
+}
+
+u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ int nr_lr = kvm_vgic_global_state.nr_lr;
+ u64 reg = 0;
+
+ if (vgic_v3_get_eisr(vcpu))
+ reg |= ICH_MISR_EOI;
+
+ if (cpu_if->vgic_hcr & ICH_HCR_UIE) {
+ int used_lrs;
+
+ used_lrs = nr_lr - hweight16(vgic_v3_get_elrsr(vcpu));
+ if (used_lrs <= 1)
+ reg |= ICH_MISR_U;
+ }
+
+ /* TODO: Support remaining bits in this register */
+ return reg;
+}
+
+/*
+ * For LRs which have HW bit set such as timer interrupts, we modify them to
+ * have the host hardware interrupt number instead of the virtual one programmed
+ * by the guest hypervisor.
+ */
+static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ struct vgic_irq *irq;
+ int i, used_lrs = 0;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ u64 lr = cpu_if->vgic_lr[i];
+ int l1_irq;
+
+ if (!(lr & ICH_LR_HW))
+ goto next;
+
+ /* We have the HW bit set */
+ l1_irq = (lr & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT;
+ irq = vgic_get_irq(vcpu->kvm, vcpu, l1_irq);
+
+ if (!irq || !irq->hw) {
+ /* There was no real mapping, so nuke the HW bit */
+ lr &= ~ICH_LR_HW;
+ if (irq)
+ vgic_put_irq(vcpu->kvm, irq);
+ goto next;
+ }
+
+ /* Translate the virtual mapping to the real one */
+ lr &= ~ICH_LR_EOI; /* Why? */
+ lr &= ~ICH_LR_PHYS_ID_MASK;
+ lr |= (u64)irq->hwintid << ICH_LR_PHYS_ID_SHIFT;
+ vgic_put_irq(vcpu->kvm, irq);
+
+next:
+ s_cpu_if->vgic_lr[i] = lr;
+ used_lrs = i + 1;
+ }
+
+ s_cpu_if->used_lrs = used_lrs;
+}
+
+/*
+ * Change the shadow HWIRQ field back to the virtual value before copying over
+ * the entire shadow struct to the nested state.
+ */
+static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ int lr;
+
+ for (lr = 0; lr < kvm_vgic_global_state.nr_lr; lr++) {
+ s_cpu_if->vgic_lr[lr] &= ~ICH_LR_PHYS_ID_MASK;
+ s_cpu_if->vgic_lr[lr] |= cpu_if->vgic_lr[lr] & ICH_LR_PHYS_ID_MASK;
+ }
+}
+
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
+ vgic_v3_create_shadow_lr(vcpu);
+ __vgic_v3_restore_state(vcpu_shadow_if(vcpu));
+}
+
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ __vgic_v3_save_state(vcpu_shadow_if(vcpu));
+
+ /*
+ * Translate the shadow state HW fields back to the virtual ones
+ * before copying the shadow struct back to the nested one.
+ */
+ vgic_v3_fixup_shadow_lr_state(vcpu);
+ vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+}
+
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+
+ /*
+ * If we exit a nested VM with a pending maintenance interrupt from the
+ * GIC, then we need to forward this to the guest hypervisor so that it
+ * can re-sync the appropriate LRs and sample level triggered interrupts
+ * again.
+ */
+ if (vgic_state_is_nested(vcpu) &&
+ (cpu_if->vgic_hcr & ICH_HCR_EN) &&
+ vgic_v3_get_misr(vcpu))
+ kvm_inject_nested_irq(vcpu);
+}
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index 9cdf39a94a63..e4c5c27552da 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -6,6 +6,7 @@
#include <kvm/arm_vgic.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/kvm_asm.h>
#include "vgic.h"
@@ -289,6 +290,12 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
vgic_v3->vgic_sre = (ICC_SRE_EL1_DIB |
ICC_SRE_EL1_DFB |
ICC_SRE_EL1_SRE);
+ /*
+ * If nesting is allowed, force GICv3 onto the nested
+ * guests as well.
+ */
+ if (nested_virt_in_use(vcpu))
+ vcpu->arch.vgic_cpu.nested_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
@@ -654,6 +661,13 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ /*
+ * vgic_v3_load_nested only affects the LRs in the shadow
+ * state, so it is fine to pass the nested state around.
+ */
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
/*
* If dealing with a GICv2 emulation on GICv3, VMCR_EL2.VFIQen
* is dependent on ICC_SRE_EL1.SRE, and we have to perform the
@@ -667,6 +681,9 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
if (has_vhe())
__vgic_v3_activate_traps(cpu_if);
+ if (vgic_state_is_nested(vcpu))
+ vgic_v3_load_nested(vcpu);
+
WARN_ON(vgic_v4_load(vcpu));
}
@@ -674,6 +691,9 @@ void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+
if (likely(cpu_if->vgic_sre))
cpu_if->vgic_vmcr = kvm_call_hyp_ret(__vgic_v3_read_vmcr);
}
@@ -686,8 +706,14 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
vgic_v3_vmcr_sync(vcpu);
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+
kvm_call_hyp(__vgic_v3_save_aprs, cpu_if);
if (has_vhe())
__vgic_v3_deactivate_traps(cpu_if);
+
+ if (vgic_state_is_nested(vcpu))
+ vgic_v3_put_nested(vcpu);
}
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 2af93dc80689..eae63e5c8322 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -876,6 +876,10 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
int used_lrs;
+ /* If nesting, this is a load/put affair, not flush/sync. */
+ if (vgic_state_is_nested(vcpu))
+ return;
+
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
@@ -920,6 +924,29 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
!vgic_supports_direct_msis(vcpu->kvm))
return;
+ /*
+ * If in a nested state, we must return early. Two possibilities:
+ *
+ * - If we have any pending IRQ for the guest and the guest
+ * expects IRQs to be handled in its virtual EL2 mode (the
+ * virtual IMO bit is set) and it is not already running in
+ * virtual EL2 mode, then we have to emulate an IRQ
+ * exception to virtual EL2.
+ *
+ * We do that by placing a request to ourselves which will
+ * abort the entry procedure and inject the exception at the
+ * beginning of the run loop.
+ *
+ * - Otherwise, do exactly *NOTHING*. The guest state is
+ * already loaded, and we can carry on with running it.
+ */
+ if (vgic_state_is_nested(vcpu)) {
+ if (kvm_vgic_vcpu_pending_irq(vcpu))
+ kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
+
+ return;
+ }
+
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
if (!list_empty(&vcpu->arch.vgic_cpu.ap_list_head)) {
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4749c4748e5a..4c6e61398f63 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -306,6 +306,15 @@ struct vgic_cpu {
struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
+ /* CPU vif control registers for the virtual GICH interface */
+ struct vgic_v3_cpu_if nested_vgic_v3;
+
+ /*
+ * The shadow vif control register loaded to the hardware when
+ * running a nested L2 guest with the virtual IMO/FMO bit set.
+ */
+ struct vgic_v3_cpu_if shadow_vgic_v3;
+
raw_spinlock_t ap_list_lock; /* Protects the ap_list */
/*
@@ -362,6 +371,13 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
+u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu);
+u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu);
+u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu);
+
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k) ((k)->arch.vgic.initialized)
#define vgic_ready(k) ((k)->arch.vgic.ready)
@@ -406,4 +422,6 @@ int vgic_v4_load(struct kvm_vcpu *vcpu);
void vgic_v4_commit(struct kvm_vcpu *vcpu);
int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
+
#endif /* __KVM_ARM_VGIC_H */
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 46/66] KVM: arm64: nv: Nested GICv3 Support
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Jintack Lim <jintack@cs.columbia.edu>
When entering a nested VM, we set up the hypervisor control interface
based on what the guest hypervisor has set. Especially, we investigate
each list register written by the guest hypervisor whether HW bit is
set. If so, we translate hw irq number from the guest's point of view
to the real hardware irq number if there is a mapping.
Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
[Rewritten to support GICv3 instead of GICv2]
Signed-off-by: Marc Zyngier <maz@kernel.org>
[Redesigned execution flow around vcpu load/put]
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
---
arch/arm64/include/asm/kvm_emulate.h | 8 +-
arch/arm64/include/asm/kvm_host.h | 9 +-
arch/arm64/include/asm/kvm_nested.h | 1 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 13 +-
arch/arm64/kvm/nested.c | 16 +++
arch/arm64/kvm/sys_regs.c | 179 ++++++++++++++++++++++++++-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 178 ++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3.c | 26 ++++
arch/arm64/kvm/vgic/vgic.c | 27 ++++
include/kvm/arm_vgic.h | 18 +++
11 files changed, 468 insertions(+), 9 deletions(-)
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index b447817d9dee..44b395854430 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -497,7 +497,13 @@ static inline bool kvm_is_write_fault(struct kvm_vcpu *vcpu)
static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
{
- return vcpu_read_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
+ /*
+ * Use the in-memory view for MPIDR_EL1. It can't be changed by the
+ * guest, and is also accessed from the context of *another* vcpu,
+ * so anything using some other state (such as the NV state that is
+ * used by vcpu_read_sys_reg) will eventually go wrong.
+ */
+ return __vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
}
static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index f3743e45ded1..bd91b4219d5e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -42,10 +42,11 @@
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
-#define KVM_REQ_IRQ_PENDING KVM_ARCH_REQ(1)
-#define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2)
-#define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3)
-#define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4)
+#define KVM_REQ_IRQ_PENDING KVM_ARCH_REQ(1)
+#define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2)
+#define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3)
+#define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4)
+#define KVM_REQ_GUEST_HYP_IRQ_PENDING KVM_ARCH_REQ(5)
#define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \
KVM_DIRTY_LOG_INITIALLY_SET)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 2987806850f0..5fa3c634c8e1 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -67,6 +67,7 @@ extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
+extern void check_nested_vcpu_requests(struct kvm_vcpu *vcpu);
struct kvm_s2_trans {
phys_addr_t output;
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 8a23991f82b2..0c15321c0827 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -23,5 +23,5 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
vgic/vgic-v3.o vgic/vgic-v4.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
- vgic/vgic-its.o vgic/vgic-debug.o
+ vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o
kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2e042185e625..818ec91801e2 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -692,6 +692,8 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu)
vgic_v4_load(vcpu);
preempt_enable();
}
+
+ check_nested_vcpu_requests(vcpu);
}
}
@@ -738,9 +740,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
*/
cond_resched();
- update_vmid(&vcpu->arch.hw_mmu->vmid);
-
+ /*
+ * A nested exeption triggered by a vcpu request (such
+ * as an interrupt injected in a guest hypervisor) can
+ * change the currently used VMID (by switching to a
+ * different translation regime. It is thus necesary
+ * to update the VMID *after* all requests have been
+ * processed.
+ */
check_vcpu_requests(vcpu);
+ update_vmid(&vcpu->arch.hw_mmu->vmid);
/*
* Preparing the interrupts to be injected also
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index e78c6c093afc..9574fde823e6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -580,6 +580,22 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm)
kvm_free_stage2_pgd(&kvm->arch.mmu);
}
+bool vgic_state_is_nested(struct kvm_vcpu *vcpu)
+{
+ bool imo = __vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO;
+ bool fmo = __vcpu_sys_reg(vcpu, HCR_EL2) & HCR_FMO;
+
+ WARN_ONCE(imo != fmo, "Separate virtual IRQ/FIQ settings not supported\n");
+
+ return nested_virt_in_use(vcpu) && imo && fmo && !is_hyp_ctxt(vcpu);
+}
+
+void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)
+{
+ if (kvm_check_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu))
+ kvm_inject_nested_irq(vcpu);
+}
+
#define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
/*
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 560b44325e98..60d2fb6edc93 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -15,6 +15,8 @@
#include <linux/printk.h>
#include <linux/uaccess.h>
+#include <linux/irqchip/arm-gic-v3.h>
+
#include <asm/cacheflush.h>
#include <asm/cputype.h>
#include <asm/debug-monitors.h>
@@ -455,6 +457,19 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
return true;
}
+/*
+ * The architecture says that non-secure write accesses to this register from
+ * EL1 are trapped to EL2, if either:
+ * - HCR_EL2.FMO==1, or
+ * - HCR_EL2.IMO==1
+ */
+static bool sgi_traps_to_vel2(struct kvm_vcpu *vcpu)
+{
+ return (nested_virt_in_use(vcpu) &&
+ !vcpu_mode_el2(vcpu) &&
+ !!(vcpu_read_sys_reg(vcpu, HCR_EL2) & (HCR_IMO | HCR_FMO)));
+}
+
/*
* Trap handler for the GICv3 SGI generation system register.
* Forward the request to the VGIC emulation.
@@ -470,6 +485,11 @@ static bool access_gic_sgi(struct kvm_vcpu *vcpu,
if (!p->is_write)
return read_from_write_only(vcpu, p, r);
+ if (sgi_traps_to_vel2(vcpu)) {
+ kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu));
+ return false;
+ }
+
/*
* In a system where GICD_CTLR.DS=1, a ICC_SGI0R_EL1 access generates
* Group0 SGIs only, while ICC_SGI1R_EL1 can generate either group,
@@ -513,7 +533,13 @@ static bool access_gic_sre(struct kvm_vcpu *vcpu,
if (p->is_write)
return ignore_write(vcpu, p);
- p->regval = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+ if (p->Op1 == 4) { /* ICC_SRE_EL2 */
+ p->regval = (ICC_SRE_EL2_ENABLE | ICC_SRE_EL2_SRE |
+ ICC_SRE_EL1_DIB | ICC_SRE_EL1_DFB);
+ } else { /* ICC_SRE_EL1 */
+ p->regval = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+ }
+
return true;
}
@@ -1759,6 +1785,122 @@ static bool access_spsr_el2(struct kvm_vcpu *vcpu,
return true;
}
+static bool access_gic_apr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ u32 index, *base;
+
+ index = r->Op2;
+ if (r->CRm == 8)
+ base = cpu_if->vgic_ap0r;
+ else
+ base = cpu_if->vgic_ap1r;
+
+ if (p->is_write)
+ base[index] = p->regval;
+ else
+ p->regval = base[index];
+
+ return true;
+}
+
+static bool access_gic_hcr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
+ if (p->is_write)
+ cpu_if->vgic_hcr = p->regval;
+ else
+ p->regval = cpu_if->vgic_hcr;
+
+ return true;
+}
+
+static bool access_gic_vtr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = kvm_vgic_global_state.ich_vtr_el2;
+
+ return true;
+}
+
+static bool access_gic_misr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_misr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_eisr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_eisr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_elrsr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = vgic_v3_get_elrsr(vcpu);
+
+ return true;
+}
+
+static bool access_gic_vmcr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
+ if (p->is_write)
+ cpu_if->vgic_vmcr = p->regval;
+ else
+ p->regval = cpu_if->vgic_vmcr;
+
+ return true;
+}
+
+static bool access_gic_lr(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ u32 index;
+
+ index = p->Op2;
+ if (p->CRm == 13)
+ index += 8;
+
+ if (p->is_write)
+ cpu_if->vgic_lr[index] = p->regval;
+ else
+ p->regval = cpu_if->vgic_lr[index];
+
+ return true;
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -2175,6 +2317,41 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
{ SYS_DESC(SYS_VDISR_EL2), trap_undef },
+ { SYS_DESC(SYS_ICH_AP0R0_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R1_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R2_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP0R3_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R0_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R1_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R2_EL2), access_gic_apr },
+ { SYS_DESC(SYS_ICH_AP1R3_EL2), access_gic_apr },
+
+ { SYS_DESC(SYS_ICC_SRE_EL2), access_gic_sre },
+
+ { SYS_DESC(SYS_ICH_HCR_EL2), access_gic_hcr },
+ { SYS_DESC(SYS_ICH_VTR_EL2), access_gic_vtr },
+ { SYS_DESC(SYS_ICH_MISR_EL2), access_gic_misr },
+ { SYS_DESC(SYS_ICH_EISR_EL2), access_gic_eisr },
+ { SYS_DESC(SYS_ICH_ELRSR_EL2), access_gic_elrsr },
+ { SYS_DESC(SYS_ICH_VMCR_EL2), access_gic_vmcr },
+
+ { SYS_DESC(SYS_ICH_LR0_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR1_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR2_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR3_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR4_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR5_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR6_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR7_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR8_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR9_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR10_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR11_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR12_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR13_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR14_EL2), access_gic_lr },
+ { SYS_DESC(SYS_ICH_LR15_EL2), access_gic_lr },
+
{ SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
{ SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
new file mode 100644
index 000000000000..bfe0ccde514f
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -0,0 +1,178 @@
+#include <linux/cpu.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/uaccess.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <kvm/arm_vgic.h>
+
+#include "vgic.h"
+
+static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->arch.vgic_cpu.nested_vgic_v3;
+}
+
+static inline struct vgic_v3_cpu_if *vcpu_shadow_if(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+}
+
+static inline bool lr_triggers_eoi(u64 lr)
+{
+ return !(lr & (ICH_LR_STATE | ICH_LR_HW)) && (lr & ICH_LR_EOI);
+}
+
+u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ u16 reg = 0;
+ int i;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ if (lr_triggers_eoi(cpu_if->vgic_lr[i]))
+ reg |= BIT(i);
+ }
+
+ return reg;
+}
+
+u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ u16 reg = 0;
+ int i;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ if (!(cpu_if->vgic_lr[i] & ICH_LR_STATE))
+ reg |= BIT(i);
+ }
+
+ return reg;
+}
+
+u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ int nr_lr = kvm_vgic_global_state.nr_lr;
+ u64 reg = 0;
+
+ if (vgic_v3_get_eisr(vcpu))
+ reg |= ICH_MISR_EOI;
+
+ if (cpu_if->vgic_hcr & ICH_HCR_UIE) {
+ int used_lrs;
+
+ used_lrs = nr_lr - hweight16(vgic_v3_get_elrsr(vcpu));
+ if (used_lrs <= 1)
+ reg |= ICH_MISR_U;
+ }
+
+ /* TODO: Support remaining bits in this register */
+ return reg;
+}
+
+/*
+ * For LRs which have HW bit set such as timer interrupts, we modify them to
+ * have the host hardware interrupt number instead of the virtual one programmed
+ * by the guest hypervisor.
+ */
+static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ struct vgic_irq *irq;
+ int i, used_lrs = 0;
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ u64 lr = cpu_if->vgic_lr[i];
+ int l1_irq;
+
+ if (!(lr & ICH_LR_HW))
+ goto next;
+
+ /* We have the HW bit set */
+ l1_irq = (lr & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT;
+ irq = vgic_get_irq(vcpu->kvm, vcpu, l1_irq);
+
+ if (!irq || !irq->hw) {
+ /* There was no real mapping, so nuke the HW bit */
+ lr &= ~ICH_LR_HW;
+ if (irq)
+ vgic_put_irq(vcpu->kvm, irq);
+ goto next;
+ }
+
+ /* Translate the virtual mapping to the real one */
+ lr &= ~ICH_LR_EOI; /* Why? */
+ lr &= ~ICH_LR_PHYS_ID_MASK;
+ lr |= (u64)irq->hwintid << ICH_LR_PHYS_ID_SHIFT;
+ vgic_put_irq(vcpu->kvm, irq);
+
+next:
+ s_cpu_if->vgic_lr[i] = lr;
+ used_lrs = i + 1;
+ }
+
+ s_cpu_if->used_lrs = used_lrs;
+}
+
+/*
+ * Change the shadow HWIRQ field back to the virtual value before copying over
+ * the entire shadow struct to the nested state.
+ */
+static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ int lr;
+
+ for (lr = 0; lr < kvm_vgic_global_state.nr_lr; lr++) {
+ s_cpu_if->vgic_lr[lr] &= ~ICH_LR_PHYS_ID_MASK;
+ s_cpu_if->vgic_lr[lr] |= cpu_if->vgic_lr[lr] & ICH_LR_PHYS_ID_MASK;
+ }
+}
+
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
+ vgic_v3_create_shadow_lr(vcpu);
+ __vgic_v3_restore_state(vcpu_shadow_if(vcpu));
+}
+
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ __vgic_v3_save_state(vcpu_shadow_if(vcpu));
+
+ /*
+ * Translate the shadow state HW fields back to the virtual ones
+ * before copying the shadow struct back to the nested one.
+ */
+ vgic_v3_fixup_shadow_lr_state(vcpu);
+ vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+}
+
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+
+ /*
+ * If we exit a nested VM with a pending maintenance interrupt from the
+ * GIC, then we need to forward this to the guest hypervisor so that it
+ * can re-sync the appropriate LRs and sample level triggered interrupts
+ * again.
+ */
+ if (vgic_state_is_nested(vcpu) &&
+ (cpu_if->vgic_hcr & ICH_HCR_EN) &&
+ vgic_v3_get_misr(vcpu))
+ kvm_inject_nested_irq(vcpu);
+}
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index 9cdf39a94a63..e4c5c27552da 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -6,6 +6,7 @@
#include <kvm/arm_vgic.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include <asm/kvm_asm.h>
#include "vgic.h"
@@ -289,6 +290,12 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
vgic_v3->vgic_sre = (ICC_SRE_EL1_DIB |
ICC_SRE_EL1_DFB |
ICC_SRE_EL1_SRE);
+ /*
+ * If nesting is allowed, force GICv3 onto the nested
+ * guests as well.
+ */
+ if (nested_virt_in_use(vcpu))
+ vcpu->arch.vgic_cpu.nested_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
@@ -654,6 +661,13 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ /*
+ * vgic_v3_load_nested only affects the LRs in the shadow
+ * state, so it is fine to pass the nested state around.
+ */
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+
/*
* If dealing with a GICv2 emulation on GICv3, VMCR_EL2.VFIQen
* is dependent on ICC_SRE_EL1.SRE, and we have to perform the
@@ -667,6 +681,9 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
if (has_vhe())
__vgic_v3_activate_traps(cpu_if);
+ if (vgic_state_is_nested(vcpu))
+ vgic_v3_load_nested(vcpu);
+
WARN_ON(vgic_v4_load(vcpu));
}
@@ -674,6 +691,9 @@ void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+
if (likely(cpu_if->vgic_sre))
cpu_if->vgic_vmcr = kvm_call_hyp_ret(__vgic_v3_read_vmcr);
}
@@ -686,8 +706,14 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
vgic_v3_vmcr_sync(vcpu);
+ if (vgic_state_is_nested(vcpu))
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+
kvm_call_hyp(__vgic_v3_save_aprs, cpu_if);
if (has_vhe())
__vgic_v3_deactivate_traps(cpu_if);
+
+ if (vgic_state_is_nested(vcpu))
+ vgic_v3_put_nested(vcpu);
}
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 2af93dc80689..eae63e5c8322 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -876,6 +876,10 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
int used_lrs;
+ /* If nesting, this is a load/put affair, not flush/sync. */
+ if (vgic_state_is_nested(vcpu))
+ return;
+
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
@@ -920,6 +924,29 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
!vgic_supports_direct_msis(vcpu->kvm))
return;
+ /*
+ * If in a nested state, we must return early. Two possibilities:
+ *
+ * - If we have any pending IRQ for the guest and the guest
+ * expects IRQs to be handled in its virtual EL2 mode (the
+ * virtual IMO bit is set) and it is not already running in
+ * virtual EL2 mode, then we have to emulate an IRQ
+ * exception to virtual EL2.
+ *
+ * We do that by placing a request to ourselves which will
+ * abort the entry procedure and inject the exception at the
+ * beginning of the run loop.
+ *
+ * - Otherwise, do exactly *NOTHING*. The guest state is
+ * already loaded, and we can carry on with running it.
+ */
+ if (vgic_state_is_nested(vcpu)) {
+ if (kvm_vgic_vcpu_pending_irq(vcpu))
+ kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
+
+ return;
+ }
+
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
if (!list_empty(&vcpu->arch.vgic_cpu.ap_list_head)) {
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4749c4748e5a..4c6e61398f63 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -306,6 +306,15 @@ struct vgic_cpu {
struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
+ /* CPU vif control registers for the virtual GICH interface */
+ struct vgic_v3_cpu_if nested_vgic_v3;
+
+ /*
+ * The shadow vif control register loaded to the hardware when
+ * running a nested L2 guest with the virtual IMO/FMO bit set.
+ */
+ struct vgic_v3_cpu_if shadow_vgic_v3;
+
raw_spinlock_t ap_list_lock; /* Protects the ap_list */
/*
@@ -362,6 +371,13 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
+u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu);
+u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu);
+u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu);
+
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k) ((k)->arch.vgic.initialized)
#define vgic_ready(k) ((k)->arch.vgic.ready)
@@ -406,4 +422,6 @@ int vgic_v4_load(struct kvm_vcpu *vcpu);
void vgic_v4_commit(struct kvm_vcpu *vcpu);
int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
+
#endif /* __KVM_ARM_VGIC_H */
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 47/66] KVM: arm64: nv: Don't load the GICv4 context on entering a nested guest
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
When entering a nested guest (vgic_state_is_nested() == true),
special care must be taken *not* to make the vPE resident, as
these are interrupts targetting the L1 guest, and not any
nested guest.
By not making the vPE resident, we guarantee that the delivery
of an vLPI will result in a doorbell, forcing an exit from the
nested guest and a switch to the L1 guest to handle the interrupt.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-v3.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index e4c5c27552da..47ea2a0f3ae4 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -683,8 +683,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
if (vgic_state_is_nested(vcpu))
vgic_v3_load_nested(vcpu);
-
- WARN_ON(vgic_v4_load(vcpu));
+ else
+ WARN_ON(vgic_v4_load(vcpu));
}
void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
@@ -702,6 +702,12 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ /*
+ * vgic_v4_put will do nothing if we were not resident. This
+ * covers both the cases where we've blocked (we already have
+ * done a vgic_v4_put) and when running a nested guest (the
+ * vPE was never resident in order to generate a doorbell).
+ */
WARN_ON(vgic_v4_put(vcpu, false));
vgic_v3_vmcr_sync(vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 47/66] KVM: arm64: nv: Don't load the GICv4 context on entering a nested guest
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
When entering a nested guest (vgic_state_is_nested() == true),
special care must be taken *not* to make the vPE resident, as
these are interrupts targetting the L1 guest, and not any
nested guest.
By not making the vPE resident, we guarantee that the delivery
of an vLPI will result in a doorbell, forcing an exit from the
nested guest and a switch to the L1 guest to handle the interrupt.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-v3.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index e4c5c27552da..47ea2a0f3ae4 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -683,8 +683,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
if (vgic_state_is_nested(vcpu))
vgic_v3_load_nested(vcpu);
-
- WARN_ON(vgic_v4_load(vcpu));
+ else
+ WARN_ON(vgic_v4_load(vcpu));
}
void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
@@ -702,6 +702,12 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ /*
+ * vgic_v4_put will do nothing if we were not resident. This
+ * covers both the cases where we've blocked (we already have
+ * done a vgic_v4_put) and when running a nested guest (the
+ * vPE was never resident in order to generate a doorbell).
+ */
WARN_ON(vgic_v4_put(vcpu, false));
vgic_v3_vmcr_sync(vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 47/66] KVM: arm64: nv: Don't load the GICv4 context on entering a nested guest
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
When entering a nested guest (vgic_state_is_nested() == true),
special care must be taken *not* to make the vPE resident, as
these are interrupts targetting the L1 guest, and not any
nested guest.
By not making the vPE resident, we guarantee that the delivery
of an vLPI will result in a doorbell, forcing an exit from the
nested guest and a switch to the L1 guest to handle the interrupt.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-v3.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index e4c5c27552da..47ea2a0f3ae4 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -683,8 +683,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
if (vgic_state_is_nested(vcpu))
vgic_v3_load_nested(vcpu);
-
- WARN_ON(vgic_v4_load(vcpu));
+ else
+ WARN_ON(vgic_v4_load(vcpu));
}
void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
@@ -702,6 +702,12 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ /*
+ * vgic_v4_put will do nothing if we were not resident. This
+ * covers both the cases where we've blocked (we already have
+ * done a vgic_v4_put) and when running a nested guest (the
+ * vPE was never resident in order to generate a doorbell).
+ */
WARN_ON(vgic_v4_put(vcpu, false));
vgic_v3_vmcr_sync(vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 48/66] KVM: arm64: nv: vgic: Emulate the HW bit in software
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
Should the guest hypervisor use the HW bit in the LRs, we need to
emulate the deactivation from the L2 guest into the L1 distributor
emulation, which is handled by L0.
It's all good fun.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_hyp.h | 2 ++
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 32 ++++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic.c | 6 ++++--
include/kvm/arm_vgic.h | 1 +
5 files changed, 40 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index c0450828378b..ce6880280e92 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -57,6 +57,8 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
+u64 __gic_v3_get_lr(unsigned int lr);
+
void __vgic_v3_save_state(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_restore_state(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_activate_traps(struct vgic_v3_cpu_if *cpu_if);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 80406f463c28..b44c8977b4f3 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -18,7 +18,7 @@
#define vtr_to_nr_pre_bits(v) ((((u32)(v) >> 26) & 7) + 1)
#define vtr_to_nr_apr_regs(v) (1 << (vtr_to_nr_pre_bits(v) - 5))
-static u64 __gic_v3_get_lr(unsigned int lr)
+u64 __gic_v3_get_lr(unsigned int lr)
{
switch (lr & 0xf) {
case 0:
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index bfe0ccde514f..3ee3438b5e22 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -138,6 +138,38 @@ static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
}
}
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ struct vgic_irq *irq;
+ int i;
+
+ for (i = 0; i < s_cpu_if->used_lrs; i++) {
+ u64 lr = cpu_if->vgic_lr[i];
+ int l1_irq;
+
+ if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE))
+ continue;
+
+ /*
+ * If we had a HW lr programmed by the guest hypervisor, we
+ * need to emulate the HW effect between the guest hypervisor
+ * and the nested guest.
+ */
+ l1_irq = (lr & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT;
+ irq = vgic_get_irq(vcpu->kvm, vcpu, l1_irq);
+ if (!irq)
+ continue; /* oh well, the guest hyp is broken */
+
+ lr = __gic_v3_get_lr(i);
+ if (!(lr & ICH_LR_STATE))
+ irq->active = false;
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index eae63e5c8322..e1527ea60486 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -876,9 +876,11 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
int used_lrs;
- /* If nesting, this is a load/put affair, not flush/sync. */
- if (vgic_state_is_nested(vcpu))
+ /* If nesting, emulate the HW effect from L0 to L1 */
+ if (vgic_state_is_nested(vcpu)) {
+ vgic_v3_sync_nested(vcpu);
return;
+ }
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4c6e61398f63..bf08135a0b78 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -371,6 +371,7 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 48/66] KVM: arm64: nv: vgic: Emulate the HW bit in software
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
Should the guest hypervisor use the HW bit in the LRs, we need to
emulate the deactivation from the L2 guest into the L1 distributor
emulation, which is handled by L0.
It's all good fun.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_hyp.h | 2 ++
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 32 ++++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic.c | 6 ++++--
include/kvm/arm_vgic.h | 1 +
5 files changed, 40 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index c0450828378b..ce6880280e92 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -57,6 +57,8 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
+u64 __gic_v3_get_lr(unsigned int lr);
+
void __vgic_v3_save_state(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_restore_state(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_activate_traps(struct vgic_v3_cpu_if *cpu_if);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 80406f463c28..b44c8977b4f3 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -18,7 +18,7 @@
#define vtr_to_nr_pre_bits(v) ((((u32)(v) >> 26) & 7) + 1)
#define vtr_to_nr_apr_regs(v) (1 << (vtr_to_nr_pre_bits(v) - 5))
-static u64 __gic_v3_get_lr(unsigned int lr)
+u64 __gic_v3_get_lr(unsigned int lr)
{
switch (lr & 0xf) {
case 0:
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index bfe0ccde514f..3ee3438b5e22 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -138,6 +138,38 @@ static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
}
}
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ struct vgic_irq *irq;
+ int i;
+
+ for (i = 0; i < s_cpu_if->used_lrs; i++) {
+ u64 lr = cpu_if->vgic_lr[i];
+ int l1_irq;
+
+ if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE))
+ continue;
+
+ /*
+ * If we had a HW lr programmed by the guest hypervisor, we
+ * need to emulate the HW effect between the guest hypervisor
+ * and the nested guest.
+ */
+ l1_irq = (lr & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT;
+ irq = vgic_get_irq(vcpu->kvm, vcpu, l1_irq);
+ if (!irq)
+ continue; /* oh well, the guest hyp is broken */
+
+ lr = __gic_v3_get_lr(i);
+ if (!(lr & ICH_LR_STATE))
+ irq->active = false;
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index eae63e5c8322..e1527ea60486 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -876,9 +876,11 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
int used_lrs;
- /* If nesting, this is a load/put affair, not flush/sync. */
- if (vgic_state_is_nested(vcpu))
+ /* If nesting, emulate the HW effect from L0 to L1 */
+ if (vgic_state_is_nested(vcpu)) {
+ vgic_v3_sync_nested(vcpu);
return;
+ }
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4c6e61398f63..bf08135a0b78 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -371,6 +371,7 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 48/66] KVM: arm64: nv: vgic: Emulate the HW bit in software
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
Should the guest hypervisor use the HW bit in the LRs, we need to
emulate the deactivation from the L2 guest into the L1 distributor
emulation, which is handled by L0.
It's all good fun.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_hyp.h | 2 ++
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 32 ++++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic.c | 6 ++++--
include/kvm/arm_vgic.h | 1 +
5 files changed, 40 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index c0450828378b..ce6880280e92 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -57,6 +57,8 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
+u64 __gic_v3_get_lr(unsigned int lr);
+
void __vgic_v3_save_state(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_restore_state(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_activate_traps(struct vgic_v3_cpu_if *cpu_if);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 80406f463c28..b44c8977b4f3 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -18,7 +18,7 @@
#define vtr_to_nr_pre_bits(v) ((((u32)(v) >> 26) & 7) + 1)
#define vtr_to_nr_apr_regs(v) (1 << (vtr_to_nr_pre_bits(v) - 5))
-static u64 __gic_v3_get_lr(unsigned int lr)
+u64 __gic_v3_get_lr(unsigned int lr)
{
switch (lr & 0xf) {
case 0:
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index bfe0ccde514f..3ee3438b5e22 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -138,6 +138,38 @@ static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
}
}
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ struct vgic_irq *irq;
+ int i;
+
+ for (i = 0; i < s_cpu_if->used_lrs; i++) {
+ u64 lr = cpu_if->vgic_lr[i];
+ int l1_irq;
+
+ if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE))
+ continue;
+
+ /*
+ * If we had a HW lr programmed by the guest hypervisor, we
+ * need to emulate the HW effect between the guest hypervisor
+ * and the nested guest.
+ */
+ l1_irq = (lr & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT;
+ irq = vgic_get_irq(vcpu->kvm, vcpu, l1_irq);
+ if (!irq)
+ continue; /* oh well, the guest hyp is broken */
+
+ lr = __gic_v3_get_lr(i);
+ if (!(lr & ICH_LR_STATE))
+ irq->active = false;
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index eae63e5c8322..e1527ea60486 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -876,9 +876,11 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
int used_lrs;
- /* If nesting, this is a load/put affair, not flush/sync. */
- if (vgic_state_is_nested(vcpu))
+ /* If nesting, emulate the HW effect from L0 to L1 */
+ if (vgic_state_is_nested(vcpu)) {
+ vgic_v3_sync_nested(vcpu);
return;
+ }
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4c6e61398f63..bf08135a0b78 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -371,6 +371,7 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 49/66] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Andre Przywara <andre.przywara@arm.com>
The VGIC maintenance IRQ signals various conditions about the LRs, when
the GIC's virtualization extension is used.
So far we didn't need it, but nested virtualization needs to know about
this interrupt, so add a userland interface to setup the IRQ number.
The architecture mandates that it must be a PPI, on top of that this code
only exports a per-device option, so the PPI is the same on all VCPUs.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
[added some bits of documentation]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +++++++++-
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +++++++++++++++++++
include/kvm/arm_vgic.h | 3 +++
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
5 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/Documentation/virt/kvm/devices/arm-vgic-v3.rst b/Documentation/virt/kvm/devices/arm-vgic-v3.rst
index 5dd3bff51978..da48bc9a6046 100644
--- a/Documentation/virt/kvm/devices/arm-vgic-v3.rst
+++ b/Documentation/virt/kvm/devices/arm-vgic-v3.rst
@@ -284,8 +284,18 @@ Groups:
| Aff3 | Aff2 | Aff1 | Aff0 |
Errors:
-
======= =============================================
-EINVAL vINTID is not multiple of 32 or info field is
not VGIC_LEVEL_INFO_LINE_LEVEL
======= =============================================
+
+ KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ
+ Attributes:
+
+ The attr field of kvm_device_attr encodes the following values:
+
+ bits: | 31 .... 5 | 4 .... 0 |
+ values: | RES0 | vINTID |
+
+ The vINTID specifies which interrupt is generated when the vGIC
+ must generate a maintenance interrupt. This must be a PPI.
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index fe3cb67f0d26..24c5752b1cc1 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -335,6 +335,7 @@ struct kvm_vcpu_events {
#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
+#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
diff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c
index 44419679f91a..dfb1d7cc66b3 100644
--- a/arch/arm64/kvm/vgic/vgic-kvm-device.c
+++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c
@@ -241,6 +241,12 @@ static int vgic_get_common_attr(struct kvm_device *dev,
VGIC_NR_PRIVATE_IRQS, uaddr);
break;
}
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+
+ r = put_user(dev->kvm->arch.vgic.maint_irq, uaddr);
+ break;
+ }
}
return r;
@@ -627,6 +633,21 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
reg = tmp32;
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 val;
+
+ if (get_user(val, uaddr))
+ return -EFAULT;
+
+ /* Must be a PPI. */
+ if ((val >= VGIC_NR_PRIVATE_IRQS) || (val < VGIC_NR_SGIS))
+ return -EINVAL;
+
+ dev->kvm->arch.vgic.maint_irq = val;
+
+ return 0;
+ }
case KVM_DEV_ARM_VGIC_GRP_CTRL: {
int ret;
@@ -712,6 +733,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ:
return 0;
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index bf08135a0b78..e02bc45e9508 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -219,6 +219,9 @@ struct vgic_dist {
int nr_spis;
+ /* The GIC maintenance IRQ for nested hypervisors. */
+ u32 maint_irq;
+
/* base addresses in guest physical address space: */
gpa_t vgic_dist_base; /* distributor */
union {
diff --git a/tools/arch/arm/include/uapi/asm/kvm.h b/tools/arch/arm/include/uapi/asm/kvm.h
index 03cd7c19a683..d5dd96902817 100644
--- a/tools/arch/arm/include/uapi/asm/kvm.h
+++ b/tools/arch/arm/include/uapi/asm/kvm.h
@@ -246,6 +246,7 @@ struct kvm_vcpu_events {
#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
+#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 49/66] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Andre Przywara <andre.przywara@arm.com>
The VGIC maintenance IRQ signals various conditions about the LRs, when
the GIC's virtualization extension is used.
So far we didn't need it, but nested virtualization needs to know about
this interrupt, so add a userland interface to setup the IRQ number.
The architecture mandates that it must be a PPI, on top of that this code
only exports a per-device option, so the PPI is the same on all VCPUs.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
[added some bits of documentation]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +++++++++-
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +++++++++++++++++++
include/kvm/arm_vgic.h | 3 +++
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
5 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/Documentation/virt/kvm/devices/arm-vgic-v3.rst b/Documentation/virt/kvm/devices/arm-vgic-v3.rst
index 5dd3bff51978..da48bc9a6046 100644
--- a/Documentation/virt/kvm/devices/arm-vgic-v3.rst
+++ b/Documentation/virt/kvm/devices/arm-vgic-v3.rst
@@ -284,8 +284,18 @@ Groups:
| Aff3 | Aff2 | Aff1 | Aff0 |
Errors:
-
======= =============================================
-EINVAL vINTID is not multiple of 32 or info field is
not VGIC_LEVEL_INFO_LINE_LEVEL
======= =============================================
+
+ KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ
+ Attributes:
+
+ The attr field of kvm_device_attr encodes the following values:
+
+ bits: | 31 .... 5 | 4 .... 0 |
+ values: | RES0 | vINTID |
+
+ The vINTID specifies which interrupt is generated when the vGIC
+ must generate a maintenance interrupt. This must be a PPI.
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index fe3cb67f0d26..24c5752b1cc1 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -335,6 +335,7 @@ struct kvm_vcpu_events {
#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
+#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
diff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c
index 44419679f91a..dfb1d7cc66b3 100644
--- a/arch/arm64/kvm/vgic/vgic-kvm-device.c
+++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c
@@ -241,6 +241,12 @@ static int vgic_get_common_attr(struct kvm_device *dev,
VGIC_NR_PRIVATE_IRQS, uaddr);
break;
}
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+
+ r = put_user(dev->kvm->arch.vgic.maint_irq, uaddr);
+ break;
+ }
}
return r;
@@ -627,6 +633,21 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
reg = tmp32;
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 val;
+
+ if (get_user(val, uaddr))
+ return -EFAULT;
+
+ /* Must be a PPI. */
+ if ((val >= VGIC_NR_PRIVATE_IRQS) || (val < VGIC_NR_SGIS))
+ return -EINVAL;
+
+ dev->kvm->arch.vgic.maint_irq = val;
+
+ return 0;
+ }
case KVM_DEV_ARM_VGIC_GRP_CTRL: {
int ret;
@@ -712,6 +733,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ:
return 0;
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index bf08135a0b78..e02bc45e9508 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -219,6 +219,9 @@ struct vgic_dist {
int nr_spis;
+ /* The GIC maintenance IRQ for nested hypervisors. */
+ u32 maint_irq;
+
/* base addresses in guest physical address space: */
gpa_t vgic_dist_base; /* distributor */
union {
diff --git a/tools/arch/arm/include/uapi/asm/kvm.h b/tools/arch/arm/include/uapi/asm/kvm.h
index 03cd7c19a683..d5dd96902817 100644
--- a/tools/arch/arm/include/uapi/asm/kvm.h
+++ b/tools/arch/arm/include/uapi/asm/kvm.h
@@ -246,6 +246,7 @@ struct kvm_vcpu_events {
#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
+#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 49/66] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Andre Przywara <andre.przywara@arm.com>
The VGIC maintenance IRQ signals various conditions about the LRs, when
the GIC's virtualization extension is used.
So far we didn't need it, but nested virtualization needs to know about
this interrupt, so add a userland interface to setup the IRQ number.
The architecture mandates that it must be a PPI, on top of that this code
only exports a per-device option, so the PPI is the same on all VCPUs.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
[added some bits of documentation]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +++++++++-
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +++++++++++++++++++
include/kvm/arm_vgic.h | 3 +++
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
5 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/Documentation/virt/kvm/devices/arm-vgic-v3.rst b/Documentation/virt/kvm/devices/arm-vgic-v3.rst
index 5dd3bff51978..da48bc9a6046 100644
--- a/Documentation/virt/kvm/devices/arm-vgic-v3.rst
+++ b/Documentation/virt/kvm/devices/arm-vgic-v3.rst
@@ -284,8 +284,18 @@ Groups:
| Aff3 | Aff2 | Aff1 | Aff0 |
Errors:
-
======= =============================================
-EINVAL vINTID is not multiple of 32 or info field is
not VGIC_LEVEL_INFO_LINE_LEVEL
======= =============================================
+
+ KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ
+ Attributes:
+
+ The attr field of kvm_device_attr encodes the following values:
+
+ bits: | 31 .... 5 | 4 .... 0 |
+ values: | RES0 | vINTID |
+
+ The vINTID specifies which interrupt is generated when the vGIC
+ must generate a maintenance interrupt. This must be a PPI.
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index fe3cb67f0d26..24c5752b1cc1 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -335,6 +335,7 @@ struct kvm_vcpu_events {
#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
+#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
diff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c
index 44419679f91a..dfb1d7cc66b3 100644
--- a/arch/arm64/kvm/vgic/vgic-kvm-device.c
+++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c
@@ -241,6 +241,12 @@ static int vgic_get_common_attr(struct kvm_device *dev,
VGIC_NR_PRIVATE_IRQS, uaddr);
break;
}
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+
+ r = put_user(dev->kvm->arch.vgic.maint_irq, uaddr);
+ break;
+ }
}
return r;
@@ -627,6 +633,21 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
reg = tmp32;
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 val;
+
+ if (get_user(val, uaddr))
+ return -EFAULT;
+
+ /* Must be a PPI. */
+ if ((val >= VGIC_NR_PRIVATE_IRQS) || (val < VGIC_NR_SGIS))
+ return -EINVAL;
+
+ dev->kvm->arch.vgic.maint_irq = val;
+
+ return 0;
+ }
case KVM_DEV_ARM_VGIC_GRP_CTRL: {
int ret;
@@ -712,6 +733,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
+ case KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ:
return 0;
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index bf08135a0b78..e02bc45e9508 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -219,6 +219,9 @@ struct vgic_dist {
int nr_spis;
+ /* The GIC maintenance IRQ for nested hypervisors. */
+ u32 maint_irq;
+
/* base addresses in guest physical address space: */
gpa_t vgic_dist_base; /* distributor */
union {
diff --git a/tools/arch/arm/include/uapi/asm/kvm.h b/tools/arch/arm/include/uapi/asm/kvm.h
index 03cd7c19a683..d5dd96902817 100644
--- a/tools/arch/arm/include/uapi/asm/kvm.h
+++ b/tools/arch/arm/include/uapi/asm/kvm.h
@@ -246,6 +246,7 @@ struct kvm_vcpu_events {
#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
+#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 50/66] KVM: arm64: nv: Implement maintenance interrupt forwarding
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
When we take a maintenance interrupt, we need to decide whether
it is generated on an action from the guest, or if it is something
that needs to be forwarded to the guest hypervisor.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-init.c | 30 ++++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 25 +++++++++++++++++++----
2 files changed, 51 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c
index 32e32d67a127..4cfaa36b7f91 100644
--- a/arch/arm64/kvm/vgic/vgic-init.c
+++ b/arch/arm64/kvm/vgic/vgic-init.c
@@ -6,10 +6,12 @@
#include <linux/uaccess.h>
#include <linux/interrupt.h>
#include <linux/cpu.h>
+#include <linux/irq.h>
#include <linux/kvm_host.h>
#include <kvm/arm_vgic.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include "vgic.h"
/*
@@ -221,6 +223,16 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
if (!irqchip_in_kernel(vcpu->kvm))
return 0;
+ if (nested_virt_in_use(vcpu)) {
+ /* FIXME: remove this hack */
+ if (vcpu->kvm->arch.vgic.maint_irq == 0)
+ vcpu->kvm->arch.vgic.maint_irq = kvm_vgic_global_state.maint_irq;
+ ret = kvm_vgic_set_owner(vcpu, vcpu->kvm->arch.vgic.maint_irq,
+ vcpu);
+ if (ret)
+ return ret;
+ }
+
/*
* If we are creating a VCPU with a GICv3 we must also register the
* KVM io device for the redistributor that belongs to this VCPU.
@@ -462,12 +474,23 @@ static int vgic_init_cpu_dying(unsigned int cpu)
static irqreturn_t vgic_maintenance_handler(int irq, void *data)
{
+ struct kvm_vcpu *vcpu = *(struct kvm_vcpu **)data;
+
/*
* We cannot rely on the vgic maintenance interrupt to be
* delivered synchronously. This means we can only use it to
* exit the VM, and we perform the handling of EOIed
* interrupts on the exit path (see vgic_fold_lr_state).
*/
+
+ /* If not nested, deactivate */
+ if (!vcpu || !vgic_state_is_nested(vcpu)) {
+ irq_set_irqchip_state(irq, IRQCHIP_STATE_ACTIVE, false);
+ return IRQ_HANDLED;
+ }
+
+ /* Assume nested from now */
+ vgic_v3_handle_nested_maint_irq(vcpu);
return IRQ_HANDLED;
}
@@ -538,6 +561,13 @@ int kvm_vgic_hyp_init(void)
return ret;
}
+ ret = irq_set_vcpu_affinity(kvm_vgic_global_state.maint_irq,
+ kvm_get_running_vcpus());
+ if (ret) {
+ kvm_err("Error setting vcpu affinity\n");
+ goto out_free_irq;
+ }
+
ret = cpuhp_setup_state(CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
"kvm/arm/vgic:starting",
vgic_init_cpu_starting, vgic_init_cpu_dying);
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 3ee3438b5e22..4ba426e2324d 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -173,10 +173,20 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_irq *irq;
+ unsigned long flags;
vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
vgic_v3_create_shadow_lr(vcpu);
__vgic_v3_restore_state(vcpu_shadow_if(vcpu));
+
+ irq = vgic_get_irq(vcpu->kvm, vcpu, vcpu->kvm->arch.vgic.maint_irq);
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+ if (irq->line_level || irq->active)
+ irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
+ IRQCHIP_STATE_ACTIVE, true);
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+ vgic_put_irq(vcpu->kvm, irq);
}
void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
@@ -191,11 +201,14 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
*/
vgic_v3_fixup_shadow_lr_state(vcpu);
vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+ irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
+ IRQCHIP_STATE_ACTIVE, false);
}
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ bool state;
/*
* If we exit a nested VM with a pending maintenance interrupt from the
@@ -203,8 +216,12 @@ void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
* can re-sync the appropriate LRs and sample level triggered interrupts
* again.
*/
- if (vgic_state_is_nested(vcpu) &&
- (cpu_if->vgic_hcr & ICH_HCR_EN) &&
- vgic_v3_get_misr(vcpu))
- kvm_inject_nested_irq(vcpu);
+ if (!vgic_state_is_nested(vcpu))
+ return;
+
+ state = cpu_if->vgic_hcr & ICH_HCR_EN;
+ state &= vgic_v3_get_misr(vcpu);
+
+ kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
+ vcpu->kvm->arch.vgic.maint_irq, state, vcpu);
}
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 50/66] KVM: arm64: nv: Implement maintenance interrupt forwarding
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
When we take a maintenance interrupt, we need to decide whether
it is generated on an action from the guest, or if it is something
that needs to be forwarded to the guest hypervisor.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-init.c | 30 ++++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 25 +++++++++++++++++++----
2 files changed, 51 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c
index 32e32d67a127..4cfaa36b7f91 100644
--- a/arch/arm64/kvm/vgic/vgic-init.c
+++ b/arch/arm64/kvm/vgic/vgic-init.c
@@ -6,10 +6,12 @@
#include <linux/uaccess.h>
#include <linux/interrupt.h>
#include <linux/cpu.h>
+#include <linux/irq.h>
#include <linux/kvm_host.h>
#include <kvm/arm_vgic.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include "vgic.h"
/*
@@ -221,6 +223,16 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
if (!irqchip_in_kernel(vcpu->kvm))
return 0;
+ if (nested_virt_in_use(vcpu)) {
+ /* FIXME: remove this hack */
+ if (vcpu->kvm->arch.vgic.maint_irq == 0)
+ vcpu->kvm->arch.vgic.maint_irq = kvm_vgic_global_state.maint_irq;
+ ret = kvm_vgic_set_owner(vcpu, vcpu->kvm->arch.vgic.maint_irq,
+ vcpu);
+ if (ret)
+ return ret;
+ }
+
/*
* If we are creating a VCPU with a GICv3 we must also register the
* KVM io device for the redistributor that belongs to this VCPU.
@@ -462,12 +474,23 @@ static int vgic_init_cpu_dying(unsigned int cpu)
static irqreturn_t vgic_maintenance_handler(int irq, void *data)
{
+ struct kvm_vcpu *vcpu = *(struct kvm_vcpu **)data;
+
/*
* We cannot rely on the vgic maintenance interrupt to be
* delivered synchronously. This means we can only use it to
* exit the VM, and we perform the handling of EOIed
* interrupts on the exit path (see vgic_fold_lr_state).
*/
+
+ /* If not nested, deactivate */
+ if (!vcpu || !vgic_state_is_nested(vcpu)) {
+ irq_set_irqchip_state(irq, IRQCHIP_STATE_ACTIVE, false);
+ return IRQ_HANDLED;
+ }
+
+ /* Assume nested from now */
+ vgic_v3_handle_nested_maint_irq(vcpu);
return IRQ_HANDLED;
}
@@ -538,6 +561,13 @@ int kvm_vgic_hyp_init(void)
return ret;
}
+ ret = irq_set_vcpu_affinity(kvm_vgic_global_state.maint_irq,
+ kvm_get_running_vcpus());
+ if (ret) {
+ kvm_err("Error setting vcpu affinity\n");
+ goto out_free_irq;
+ }
+
ret = cpuhp_setup_state(CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
"kvm/arm/vgic:starting",
vgic_init_cpu_starting, vgic_init_cpu_dying);
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 3ee3438b5e22..4ba426e2324d 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -173,10 +173,20 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_irq *irq;
+ unsigned long flags;
vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
vgic_v3_create_shadow_lr(vcpu);
__vgic_v3_restore_state(vcpu_shadow_if(vcpu));
+
+ irq = vgic_get_irq(vcpu->kvm, vcpu, vcpu->kvm->arch.vgic.maint_irq);
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+ if (irq->line_level || irq->active)
+ irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
+ IRQCHIP_STATE_ACTIVE, true);
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+ vgic_put_irq(vcpu->kvm, irq);
}
void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
@@ -191,11 +201,14 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
*/
vgic_v3_fixup_shadow_lr_state(vcpu);
vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+ irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
+ IRQCHIP_STATE_ACTIVE, false);
}
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ bool state;
/*
* If we exit a nested VM with a pending maintenance interrupt from the
@@ -203,8 +216,12 @@ void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
* can re-sync the appropriate LRs and sample level triggered interrupts
* again.
*/
- if (vgic_state_is_nested(vcpu) &&
- (cpu_if->vgic_hcr & ICH_HCR_EN) &&
- vgic_v3_get_misr(vcpu))
- kvm_inject_nested_irq(vcpu);
+ if (!vgic_state_is_nested(vcpu))
+ return;
+
+ state = cpu_if->vgic_hcr & ICH_HCR_EN;
+ state &= vgic_v3_get_misr(vcpu);
+
+ kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
+ vcpu->kvm->arch.vgic.maint_irq, state, vcpu);
}
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 50/66] KVM: arm64: nv: Implement maintenance interrupt forwarding
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
When we take a maintenance interrupt, we need to decide whether
it is generated on an action from the guest, or if it is something
that needs to be forwarded to the guest hypervisor.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-init.c | 30 ++++++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 25 +++++++++++++++++++----
2 files changed, 51 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c
index 32e32d67a127..4cfaa36b7f91 100644
--- a/arch/arm64/kvm/vgic/vgic-init.c
+++ b/arch/arm64/kvm/vgic/vgic-init.c
@@ -6,10 +6,12 @@
#include <linux/uaccess.h>
#include <linux/interrupt.h>
#include <linux/cpu.h>
+#include <linux/irq.h>
#include <linux/kvm_host.h>
#include <kvm/arm_vgic.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_nested.h>
#include "vgic.h"
/*
@@ -221,6 +223,16 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
if (!irqchip_in_kernel(vcpu->kvm))
return 0;
+ if (nested_virt_in_use(vcpu)) {
+ /* FIXME: remove this hack */
+ if (vcpu->kvm->arch.vgic.maint_irq == 0)
+ vcpu->kvm->arch.vgic.maint_irq = kvm_vgic_global_state.maint_irq;
+ ret = kvm_vgic_set_owner(vcpu, vcpu->kvm->arch.vgic.maint_irq,
+ vcpu);
+ if (ret)
+ return ret;
+ }
+
/*
* If we are creating a VCPU with a GICv3 we must also register the
* KVM io device for the redistributor that belongs to this VCPU.
@@ -462,12 +474,23 @@ static int vgic_init_cpu_dying(unsigned int cpu)
static irqreturn_t vgic_maintenance_handler(int irq, void *data)
{
+ struct kvm_vcpu *vcpu = *(struct kvm_vcpu **)data;
+
/*
* We cannot rely on the vgic maintenance interrupt to be
* delivered synchronously. This means we can only use it to
* exit the VM, and we perform the handling of EOIed
* interrupts on the exit path (see vgic_fold_lr_state).
*/
+
+ /* If not nested, deactivate */
+ if (!vcpu || !vgic_state_is_nested(vcpu)) {
+ irq_set_irqchip_state(irq, IRQCHIP_STATE_ACTIVE, false);
+ return IRQ_HANDLED;
+ }
+
+ /* Assume nested from now */
+ vgic_v3_handle_nested_maint_irq(vcpu);
return IRQ_HANDLED;
}
@@ -538,6 +561,13 @@ int kvm_vgic_hyp_init(void)
return ret;
}
+ ret = irq_set_vcpu_affinity(kvm_vgic_global_state.maint_irq,
+ kvm_get_running_vcpus());
+ if (ret) {
+ kvm_err("Error setting vcpu affinity\n");
+ goto out_free_irq;
+ }
+
ret = cpuhp_setup_state(CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
"kvm/arm/vgic:starting",
vgic_init_cpu_starting, vgic_init_cpu_dying);
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 3ee3438b5e22..4ba426e2324d 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -173,10 +173,20 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_irq *irq;
+ unsigned long flags;
vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
vgic_v3_create_shadow_lr(vcpu);
__vgic_v3_restore_state(vcpu_shadow_if(vcpu));
+
+ irq = vgic_get_irq(vcpu->kvm, vcpu, vcpu->kvm->arch.vgic.maint_irq);
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+ if (irq->line_level || irq->active)
+ irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
+ IRQCHIP_STATE_ACTIVE, true);
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+ vgic_put_irq(vcpu->kvm, irq);
}
void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
@@ -191,11 +201,14 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
*/
vgic_v3_fixup_shadow_lr_state(vcpu);
vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+ irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
+ IRQCHIP_STATE_ACTIVE, false);
}
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
+ bool state;
/*
* If we exit a nested VM with a pending maintenance interrupt from the
@@ -203,8 +216,12 @@ void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
* can re-sync the appropriate LRs and sample level triggered interrupts
* again.
*/
- if (vgic_state_is_nested(vcpu) &&
- (cpu_if->vgic_hcr & ICH_HCR_EN) &&
- vgic_v3_get_misr(vcpu))
- kvm_inject_nested_irq(vcpu);
+ if (!vgic_state_is_nested(vcpu))
+ return;
+
+ state = cpu_if->vgic_hcr & ICH_HCR_EN;
+ state &= vgic_v3_get_misr(vcpu);
+
+ kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
+ vcpu->kvm->arch.vgic.maint_irq, state, vcpu);
}
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 51/66] KVM: arm64: nv: Add nested GICv3 tracepoints
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
Adding tracepoints to be able to peek into the shadow LRs used when
running a guest guest.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 13 ++-
2 files changed, 149 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
diff --git a/arch/arm64/kvm/vgic/vgic-nested-trace.h b/arch/arm64/kvm/vgic/vgic-nested-trace.h
new file mode 100644
index 000000000000..f1a074c791a6
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-nested-trace.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#if !defined(_TRACE_VGIC_NESTED_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_VGIC_NESTED_H
+
+#include <linux/tracepoint.h>
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM kvm
+
+#define SLR_ENTRY_VALS(x) \
+ " ", \
+ !!(__entry->lrs[x] & ICH_LR_HW), \
+ !!(__entry->lrs[x] & ICH_LR_PENDING_BIT), \
+ !!(__entry->lrs[x] & ICH_LR_ACTIVE_BIT), \
+ __entry->lrs[x] & ICH_LR_VIRTUAL_ID_MASK, \
+ (__entry->lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT, \
+ (__entry->orig_lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT
+
+TRACE_EVENT(vgic_create_shadow_lrs,
+ TP_PROTO(struct kvm_vcpu *vcpu, int nr_lr, u64 *lrs, u64 *orig_lrs),
+ TP_ARGS(vcpu, nr_lr, lrs, orig_lrs),
+
+ TP_STRUCT__entry(
+ __field( int, nr_lr )
+ __array( u64, lrs, 16 )
+ __array( u64, orig_lrs, 16 )
+ ),
+
+ TP_fast_assign(
+ __entry->nr_lr = nr_lr;
+ memcpy(__entry->lrs, lrs, 16 * sizeof(u64));
+ memcpy(__entry->orig_lrs, orig_lrs, 16 * sizeof(u64));
+ ),
+
+ TP_printk("nr_lr: %d\n"
+ "%50sLR[ 0]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 1]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 2]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 3]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 4]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 5]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 6]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 7]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 8]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 9]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[10]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[11]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[12]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[13]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[14]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[15]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)",
+ __entry->nr_lr,
+ SLR_ENTRY_VALS(0), SLR_ENTRY_VALS(1), SLR_ENTRY_VALS(2),
+ SLR_ENTRY_VALS(3), SLR_ENTRY_VALS(4), SLR_ENTRY_VALS(5),
+ SLR_ENTRY_VALS(6), SLR_ENTRY_VALS(7), SLR_ENTRY_VALS(8),
+ SLR_ENTRY_VALS(9), SLR_ENTRY_VALS(10), SLR_ENTRY_VALS(11),
+ SLR_ENTRY_VALS(12), SLR_ENTRY_VALS(13), SLR_ENTRY_VALS(14),
+ SLR_ENTRY_VALS(15))
+);
+
+#define LR_ENTRY_VALS(x) \
+ " ", \
+ !!(__entry->lrs[x] & ICH_LR_HW), \
+ !!(__entry->lrs[x] & ICH_LR_PENDING_BIT), \
+ !!(__entry->lrs[x] & ICH_LR_ACTIVE_BIT), \
+ __entry->lrs[x] & ICH_LR_VIRTUAL_ID_MASK, \
+ (__entry->lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT
+
+TRACE_EVENT(vgic_put_nested,
+ TP_PROTO(struct kvm_vcpu *vcpu, int nr_lr, u64 *lrs),
+ TP_ARGS(vcpu, nr_lr, lrs),
+
+ TP_STRUCT__entry(
+ __field( int, nr_lr )
+ __array( u64, lrs, 16 )
+ ),
+
+ TP_fast_assign(
+ __entry->nr_lr = nr_lr;
+ memcpy(__entry->lrs, lrs, 16 * sizeof(u64));
+ ),
+
+ TP_printk("nr_lr: %d\n"
+ "%50sLR[ 0]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 1]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 2]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 3]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 4]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 5]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 6]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 7]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 8]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 9]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[10]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[11]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[12]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[13]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[14]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[15]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu",
+ __entry->nr_lr,
+ LR_ENTRY_VALS(0), LR_ENTRY_VALS(1), LR_ENTRY_VALS(2),
+ LR_ENTRY_VALS(3), LR_ENTRY_VALS(4), LR_ENTRY_VALS(5),
+ LR_ENTRY_VALS(6), LR_ENTRY_VALS(7), LR_ENTRY_VALS(8),
+ LR_ENTRY_VALS(9), LR_ENTRY_VALS(10), LR_ENTRY_VALS(11),
+ LR_ENTRY_VALS(12), LR_ENTRY_VALS(13), LR_ENTRY_VALS(14),
+ LR_ENTRY_VALS(15))
+);
+
+TRACE_EVENT(vgic_nested_hw_emulate,
+ TP_PROTO(int lr, u64 lr_val, u32 l1_intid),
+ TP_ARGS(lr, lr_val, l1_intid),
+
+ TP_STRUCT__entry(
+ __field( int, lr )
+ __field( u64, lr_val )
+ __field( u32, l1_intid )
+ ),
+
+ TP_fast_assign(
+ __entry->lr = lr;
+ __entry->lr_val = lr_val;
+ __entry->l1_intid = l1_intid;
+ ),
+
+ TP_printk("lr: %d LR %llx L1 INTID: %u\n",
+ __entry->lr, __entry->lr_val, __entry->l1_intid)
+);
+
+#endif /* _TRACE_VGIC_NESTED_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH vgic/
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE vgic-nested-trace
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 4ba426e2324d..94b1edb67011 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -13,6 +13,9 @@
#include "vgic.h"
+#define CREATE_TRACE_POINTS
+#include "vgic-nested-trace.h"
+
static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.vgic_cpu.nested_vgic_v3;
@@ -119,6 +122,9 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
used_lrs = i + 1;
}
+ trace_vgic_create_shadow_lrs(vcpu, kvm_vgic_global_state.nr_lr,
+ s_cpu_if->vgic_lr, cpu_if->vgic_lr);
+
s_cpu_if->used_lrs = used_lrs;
}
@@ -163,8 +169,10 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
continue; /* oh well, the guest hyp is broken */
lr = __gic_v3_get_lr(i);
- if (!(lr & ICH_LR_STATE))
+ if (!(lr & ICH_LR_STATE)) {
+ trace_vgic_nested_hw_emulate(i, lr, l1_irq);
irq->active = false;
+ }
vgic_put_irq(vcpu->kvm, irq);
}
@@ -195,6 +203,9 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
__vgic_v3_save_state(vcpu_shadow_if(vcpu));
+ trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr,
+ vcpu_shadow_if(vcpu)->vgic_lr);
+
/*
* Translate the shadow state HW fields back to the virtual ones
* before copying the shadow struct back to the nested one.
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 51/66] KVM: arm64: nv: Add nested GICv3 tracepoints
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
Adding tracepoints to be able to peek into the shadow LRs used when
running a guest guest.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 13 ++-
2 files changed, 149 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
diff --git a/arch/arm64/kvm/vgic/vgic-nested-trace.h b/arch/arm64/kvm/vgic/vgic-nested-trace.h
new file mode 100644
index 000000000000..f1a074c791a6
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-nested-trace.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#if !defined(_TRACE_VGIC_NESTED_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_VGIC_NESTED_H
+
+#include <linux/tracepoint.h>
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM kvm
+
+#define SLR_ENTRY_VALS(x) \
+ " ", \
+ !!(__entry->lrs[x] & ICH_LR_HW), \
+ !!(__entry->lrs[x] & ICH_LR_PENDING_BIT), \
+ !!(__entry->lrs[x] & ICH_LR_ACTIVE_BIT), \
+ __entry->lrs[x] & ICH_LR_VIRTUAL_ID_MASK, \
+ (__entry->lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT, \
+ (__entry->orig_lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT
+
+TRACE_EVENT(vgic_create_shadow_lrs,
+ TP_PROTO(struct kvm_vcpu *vcpu, int nr_lr, u64 *lrs, u64 *orig_lrs),
+ TP_ARGS(vcpu, nr_lr, lrs, orig_lrs),
+
+ TP_STRUCT__entry(
+ __field( int, nr_lr )
+ __array( u64, lrs, 16 )
+ __array( u64, orig_lrs, 16 )
+ ),
+
+ TP_fast_assign(
+ __entry->nr_lr = nr_lr;
+ memcpy(__entry->lrs, lrs, 16 * sizeof(u64));
+ memcpy(__entry->orig_lrs, orig_lrs, 16 * sizeof(u64));
+ ),
+
+ TP_printk("nr_lr: %d\n"
+ "%50sLR[ 0]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 1]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 2]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 3]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 4]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 5]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 6]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 7]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 8]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 9]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[10]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[11]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[12]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[13]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[14]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[15]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)",
+ __entry->nr_lr,
+ SLR_ENTRY_VALS(0), SLR_ENTRY_VALS(1), SLR_ENTRY_VALS(2),
+ SLR_ENTRY_VALS(3), SLR_ENTRY_VALS(4), SLR_ENTRY_VALS(5),
+ SLR_ENTRY_VALS(6), SLR_ENTRY_VALS(7), SLR_ENTRY_VALS(8),
+ SLR_ENTRY_VALS(9), SLR_ENTRY_VALS(10), SLR_ENTRY_VALS(11),
+ SLR_ENTRY_VALS(12), SLR_ENTRY_VALS(13), SLR_ENTRY_VALS(14),
+ SLR_ENTRY_VALS(15))
+);
+
+#define LR_ENTRY_VALS(x) \
+ " ", \
+ !!(__entry->lrs[x] & ICH_LR_HW), \
+ !!(__entry->lrs[x] & ICH_LR_PENDING_BIT), \
+ !!(__entry->lrs[x] & ICH_LR_ACTIVE_BIT), \
+ __entry->lrs[x] & ICH_LR_VIRTUAL_ID_MASK, \
+ (__entry->lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT
+
+TRACE_EVENT(vgic_put_nested,
+ TP_PROTO(struct kvm_vcpu *vcpu, int nr_lr, u64 *lrs),
+ TP_ARGS(vcpu, nr_lr, lrs),
+
+ TP_STRUCT__entry(
+ __field( int, nr_lr )
+ __array( u64, lrs, 16 )
+ ),
+
+ TP_fast_assign(
+ __entry->nr_lr = nr_lr;
+ memcpy(__entry->lrs, lrs, 16 * sizeof(u64));
+ ),
+
+ TP_printk("nr_lr: %d\n"
+ "%50sLR[ 0]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 1]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 2]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 3]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 4]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 5]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 6]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 7]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 8]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 9]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[10]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[11]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[12]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[13]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[14]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[15]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu",
+ __entry->nr_lr,
+ LR_ENTRY_VALS(0), LR_ENTRY_VALS(1), LR_ENTRY_VALS(2),
+ LR_ENTRY_VALS(3), LR_ENTRY_VALS(4), LR_ENTRY_VALS(5),
+ LR_ENTRY_VALS(6), LR_ENTRY_VALS(7), LR_ENTRY_VALS(8),
+ LR_ENTRY_VALS(9), LR_ENTRY_VALS(10), LR_ENTRY_VALS(11),
+ LR_ENTRY_VALS(12), LR_ENTRY_VALS(13), LR_ENTRY_VALS(14),
+ LR_ENTRY_VALS(15))
+);
+
+TRACE_EVENT(vgic_nested_hw_emulate,
+ TP_PROTO(int lr, u64 lr_val, u32 l1_intid),
+ TP_ARGS(lr, lr_val, l1_intid),
+
+ TP_STRUCT__entry(
+ __field( int, lr )
+ __field( u64, lr_val )
+ __field( u32, l1_intid )
+ ),
+
+ TP_fast_assign(
+ __entry->lr = lr;
+ __entry->lr_val = lr_val;
+ __entry->l1_intid = l1_intid;
+ ),
+
+ TP_printk("lr: %d LR %llx L1 INTID: %u\n",
+ __entry->lr, __entry->lr_val, __entry->l1_intid)
+);
+
+#endif /* _TRACE_VGIC_NESTED_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH vgic/
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE vgic-nested-trace
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 4ba426e2324d..94b1edb67011 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -13,6 +13,9 @@
#include "vgic.h"
+#define CREATE_TRACE_POINTS
+#include "vgic-nested-trace.h"
+
static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.vgic_cpu.nested_vgic_v3;
@@ -119,6 +122,9 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
used_lrs = i + 1;
}
+ trace_vgic_create_shadow_lrs(vcpu, kvm_vgic_global_state.nr_lr,
+ s_cpu_if->vgic_lr, cpu_if->vgic_lr);
+
s_cpu_if->used_lrs = used_lrs;
}
@@ -163,8 +169,10 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
continue; /* oh well, the guest hyp is broken */
lr = __gic_v3_get_lr(i);
- if (!(lr & ICH_LR_STATE))
+ if (!(lr & ICH_LR_STATE)) {
+ trace_vgic_nested_hw_emulate(i, lr, l1_irq);
irq->active = false;
+ }
vgic_put_irq(vcpu->kvm, irq);
}
@@ -195,6 +203,9 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
__vgic_v3_save_state(vcpu_shadow_if(vcpu));
+ trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr,
+ vcpu_shadow_if(vcpu)->vgic_lr);
+
/*
* Translate the shadow state HW fields back to the virtual ones
* before copying the shadow struct back to the nested one.
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 51/66] KVM: arm64: nv: Add nested GICv3 tracepoints
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
Adding tracepoints to be able to peek into the shadow LRs used when
running a guest guest.
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++++++++++++++++++++++++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 13 ++-
2 files changed, 149 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
diff --git a/arch/arm64/kvm/vgic/vgic-nested-trace.h b/arch/arm64/kvm/vgic/vgic-nested-trace.h
new file mode 100644
index 000000000000..f1a074c791a6
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-nested-trace.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#if !defined(_TRACE_VGIC_NESTED_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_VGIC_NESTED_H
+
+#include <linux/tracepoint.h>
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM kvm
+
+#define SLR_ENTRY_VALS(x) \
+ " ", \
+ !!(__entry->lrs[x] & ICH_LR_HW), \
+ !!(__entry->lrs[x] & ICH_LR_PENDING_BIT), \
+ !!(__entry->lrs[x] & ICH_LR_ACTIVE_BIT), \
+ __entry->lrs[x] & ICH_LR_VIRTUAL_ID_MASK, \
+ (__entry->lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT, \
+ (__entry->orig_lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT
+
+TRACE_EVENT(vgic_create_shadow_lrs,
+ TP_PROTO(struct kvm_vcpu *vcpu, int nr_lr, u64 *lrs, u64 *orig_lrs),
+ TP_ARGS(vcpu, nr_lr, lrs, orig_lrs),
+
+ TP_STRUCT__entry(
+ __field( int, nr_lr )
+ __array( u64, lrs, 16 )
+ __array( u64, orig_lrs, 16 )
+ ),
+
+ TP_fast_assign(
+ __entry->nr_lr = nr_lr;
+ memcpy(__entry->lrs, lrs, 16 * sizeof(u64));
+ memcpy(__entry->orig_lrs, orig_lrs, 16 * sizeof(u64));
+ ),
+
+ TP_printk("nr_lr: %d\n"
+ "%50sLR[ 0]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 1]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 2]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 3]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 4]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 5]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 6]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 7]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 8]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[ 9]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[10]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[11]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[12]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[13]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[14]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)\n"
+ "%50sLR[15]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu (%5llu)",
+ __entry->nr_lr,
+ SLR_ENTRY_VALS(0), SLR_ENTRY_VALS(1), SLR_ENTRY_VALS(2),
+ SLR_ENTRY_VALS(3), SLR_ENTRY_VALS(4), SLR_ENTRY_VALS(5),
+ SLR_ENTRY_VALS(6), SLR_ENTRY_VALS(7), SLR_ENTRY_VALS(8),
+ SLR_ENTRY_VALS(9), SLR_ENTRY_VALS(10), SLR_ENTRY_VALS(11),
+ SLR_ENTRY_VALS(12), SLR_ENTRY_VALS(13), SLR_ENTRY_VALS(14),
+ SLR_ENTRY_VALS(15))
+);
+
+#define LR_ENTRY_VALS(x) \
+ " ", \
+ !!(__entry->lrs[x] & ICH_LR_HW), \
+ !!(__entry->lrs[x] & ICH_LR_PENDING_BIT), \
+ !!(__entry->lrs[x] & ICH_LR_ACTIVE_BIT), \
+ __entry->lrs[x] & ICH_LR_VIRTUAL_ID_MASK, \
+ (__entry->lrs[x] & ICH_LR_PHYS_ID_MASK) >> ICH_LR_PHYS_ID_SHIFT
+
+TRACE_EVENT(vgic_put_nested,
+ TP_PROTO(struct kvm_vcpu *vcpu, int nr_lr, u64 *lrs),
+ TP_ARGS(vcpu, nr_lr, lrs),
+
+ TP_STRUCT__entry(
+ __field( int, nr_lr )
+ __array( u64, lrs, 16 )
+ ),
+
+ TP_fast_assign(
+ __entry->nr_lr = nr_lr;
+ memcpy(__entry->lrs, lrs, 16 * sizeof(u64));
+ ),
+
+ TP_printk("nr_lr: %d\n"
+ "%50sLR[ 0]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 1]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 2]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 3]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 4]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 5]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 6]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 7]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 8]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[ 9]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[10]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[11]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[12]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[13]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[14]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu\n"
+ "%50sLR[15]: HW: %d P: %d: A: %d vINTID: %5llu pINTID: %5llu",
+ __entry->nr_lr,
+ LR_ENTRY_VALS(0), LR_ENTRY_VALS(1), LR_ENTRY_VALS(2),
+ LR_ENTRY_VALS(3), LR_ENTRY_VALS(4), LR_ENTRY_VALS(5),
+ LR_ENTRY_VALS(6), LR_ENTRY_VALS(7), LR_ENTRY_VALS(8),
+ LR_ENTRY_VALS(9), LR_ENTRY_VALS(10), LR_ENTRY_VALS(11),
+ LR_ENTRY_VALS(12), LR_ENTRY_VALS(13), LR_ENTRY_VALS(14),
+ LR_ENTRY_VALS(15))
+);
+
+TRACE_EVENT(vgic_nested_hw_emulate,
+ TP_PROTO(int lr, u64 lr_val, u32 l1_intid),
+ TP_ARGS(lr, lr_val, l1_intid),
+
+ TP_STRUCT__entry(
+ __field( int, lr )
+ __field( u64, lr_val )
+ __field( u32, l1_intid )
+ ),
+
+ TP_fast_assign(
+ __entry->lr = lr;
+ __entry->lr_val = lr_val;
+ __entry->l1_intid = l1_intid;
+ ),
+
+ TP_printk("lr: %d LR %llx L1 INTID: %u\n",
+ __entry->lr, __entry->lr_val, __entry->l1_intid)
+);
+
+#endif /* _TRACE_VGIC_NESTED_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH vgic/
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE vgic-nested-trace
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 4ba426e2324d..94b1edb67011 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -13,6 +13,9 @@
#include "vgic.h"
+#define CREATE_TRACE_POINTS
+#include "vgic-nested-trace.h"
+
static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.vgic_cpu.nested_vgic_v3;
@@ -119,6 +122,9 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
used_lrs = i + 1;
}
+ trace_vgic_create_shadow_lrs(vcpu, kvm_vgic_global_state.nr_lr,
+ s_cpu_if->vgic_lr, cpu_if->vgic_lr);
+
s_cpu_if->used_lrs = used_lrs;
}
@@ -163,8 +169,10 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
continue; /* oh well, the guest hyp is broken */
lr = __gic_v3_get_lr(i);
- if (!(lr & ICH_LR_STATE))
+ if (!(lr & ICH_LR_STATE)) {
+ trace_vgic_nested_hw_emulate(i, lr, l1_irq);
irq->active = false;
+ }
vgic_put_irq(vcpu->kvm, irq);
}
@@ -195,6 +203,9 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
__vgic_v3_save_state(vcpu_shadow_if(vcpu));
+ trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr,
+ vcpu_shadow_if(vcpu)->vgic_lr);
+
/*
* Translate the shadow state HW fields back to the virtual ones
* before copying the shadow struct back to the nested one.
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 52/66] KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Since we're (almost) feature complete, let's allow userspace to
request KVM_ARM_VCPU_NESTED_VIRT by bumping the KVM_VCPU_MAX_FEATURES
up.
It's going to be great...
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bd91b4219d5e..9e190a65e78b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -38,7 +38,7 @@
#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
-#define KVM_VCPU_MAX_FEATURES 7
+#define KVM_VCPU_MAX_FEATURES 8
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 52/66] KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Since we're (almost) feature complete, let's allow userspace to
request KVM_ARM_VCPU_NESTED_VIRT by bumping the KVM_VCPU_MAX_FEATURES
up.
It's going to be great...
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bd91b4219d5e..9e190a65e78b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -38,7 +38,7 @@
#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
-#define KVM_VCPU_MAX_FEATURES 7
+#define KVM_VCPU_MAX_FEATURES 8
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 52/66] KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Since we're (almost) feature complete, let's allow userspace to
request KVM_ARM_VCPU_NESTED_VIRT by bumping the KVM_VCPU_MAX_FEATURES
up.
It's going to be great...
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bd91b4219d5e..9e190a65e78b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -38,7 +38,7 @@
#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
-#define KVM_VCPU_MAX_FEATURES 7
+#define KVM_VCPU_MAX_FEATURES 8
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 53/66] KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Support guest-provided information information to find out about
the range of required invalidation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 1 +
arch/arm64/kvm/nested.c | 57 +++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 79 +++++++++++++++++++----------
3 files changed, 109 insertions(+), 28 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 5fa3c634c8e1..7c47ad655e2e 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -123,6 +123,7 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+unsigned int ttl_to_size(u8 ttl);
struct sys_reg_params;
struct sys_reg_desc;
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9574fde823e6..05b711e730b6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -351,6 +351,63 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
return ret;
}
+
+unsigned int ttl_to_size(u8 ttl)
+{
+ int level = ttl & 3;
+ int gran = (ttl >> 2) & 3;
+ unsigned int max_size = 0;
+
+ switch (gran) {
+ case TLBI_TTL_TG_4K:
+ switch (level) {
+ case 0:
+ break;
+ case 1:
+ max_size = SZ_1G;
+ break;
+ case 2:
+ max_size = SZ_2M;
+ break;
+ case 3:
+ max_size = SZ_4K;
+ break;
+ }
+ break;
+ case TLBI_TTL_TG_16K:
+ switch (level) {
+ case 0:
+ case 1:
+ break;
+ case 2:
+ max_size = SZ_32M;
+ break;
+ case 3:
+ max_size = SZ_16K;
+ break;
+ }
+ break;
+ case TLBI_TTL_TG_64K:
+ switch (level) {
+ case 0:
+ case 1:
+ /* No 52bit IPA support */
+ break;
+ case 2:
+ max_size = SZ_512M;
+ break;
+ case 3:
+ max_size = SZ_64K;
+ break;
+ }
+ break;
+ default: /* No size information */
+ break;
+ }
+
+ return max_size;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 60d2fb6edc93..61094f5e955e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -9,6 +9,7 @@
* Christoffer Dall <c.dall@virtualopensystems.com>
*/
+#include <linux/bitfield.h>
#include <linux/bsearch.h>
#include <linux/kvm_host.h>
#include <linux/mm.h>
@@ -2578,59 +2579,81 @@ static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return true;
}
+static unsigned long compute_tlb_inval_range(struct kvm_vcpu *vcpu,
+ struct kvm_s2_mmu *mmu,
+ u64 val)
+{
+ unsigned long max_size;
+ u8 ttl = 0;
+
+ if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL)) {
+ ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+ }
+
+ max_size = ttl_to_size(ttl);
+
+ if (!max_size) {
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+
+ /* Compute the maximum extent of the invalidation */
+ switch ((vtcr & VTCR_EL2_TG0_MASK)) {
+ case VTCR_EL2_TG0_4K:
+ max_size = SZ_1G;
+ break;
+ case VTCR_EL2_TG0_16K:
+ max_size = SZ_32M;
+ break;
+ case VTCR_EL2_TG0_64K:
+ /*
+ * No, we do not support 52bit IPA in nested yet. Once
+ * we do, this should be 4TB.
+ */
+ /* FIXME: remove the 52bit PA support from the IDregs */
+ max_size = SZ_512M;
+ break;
+ default:
+ BUG();
+ }
+ }
+
+ WARN_ON(!max_size);
+ return max_size;
+}
+
static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
- u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
struct kvm_s2_mmu *mmu;
u64 base_addr;
- int max_size;
+ unsigned long max_size;
/*
* We drop a number of things from the supplied value:
*
* - NS bit: we're non-secure only.
*
- * - TTL field: We already have the granule size from the
- * VTCR_EL2.TG0 field, and the level is only relevant to the
- * guest's S2PT.
- *
* - IPA[51:48]: We don't support 52bit IPA just yet...
*
* And of course, adjust the IPA to be on an actual address.
*/
base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
- /* Compute the maximum extent of the invalidation */
- switch ((vtcr & VTCR_EL2_TG0_MASK)) {
- case VTCR_EL2_TG0_4K:
- max_size = SZ_1G;
- break;
- case VTCR_EL2_TG0_16K:
- max_size = SZ_32M;
- break;
- case VTCR_EL2_TG0_64K:
- /*
- * No, we do not support 52bit IPA in nested yet. Once
- * we do, this should be 4TB.
- */
- /* FIXME: remove the 52bit PA support from the IDregs */
- max_size = SZ_512M;
- break;
- default:
- BUG();
- }
-
spin_lock(&vcpu->kvm->mmu_lock);
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
- if (mmu)
+ if (mmu) {
+ max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
+ base_addr &= ~(max_size - 1);
kvm_unmap_stage2_range(mmu, base_addr, max_size);
+ }
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
- if (mmu)
+ if (mmu) {
+ max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
+ base_addr &= ~(max_size - 1);
kvm_unmap_stage2_range(mmu, base_addr, max_size);
+ }
spin_unlock(&vcpu->kvm->mmu_lock);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 53/66] KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Support guest-provided information information to find out about
the range of required invalidation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 1 +
arch/arm64/kvm/nested.c | 57 +++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 79 +++++++++++++++++++----------
3 files changed, 109 insertions(+), 28 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 5fa3c634c8e1..7c47ad655e2e 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -123,6 +123,7 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+unsigned int ttl_to_size(u8 ttl);
struct sys_reg_params;
struct sys_reg_desc;
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9574fde823e6..05b711e730b6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -351,6 +351,63 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
return ret;
}
+
+unsigned int ttl_to_size(u8 ttl)
+{
+ int level = ttl & 3;
+ int gran = (ttl >> 2) & 3;
+ unsigned int max_size = 0;
+
+ switch (gran) {
+ case TLBI_TTL_TG_4K:
+ switch (level) {
+ case 0:
+ break;
+ case 1:
+ max_size = SZ_1G;
+ break;
+ case 2:
+ max_size = SZ_2M;
+ break;
+ case 3:
+ max_size = SZ_4K;
+ break;
+ }
+ break;
+ case TLBI_TTL_TG_16K:
+ switch (level) {
+ case 0:
+ case 1:
+ break;
+ case 2:
+ max_size = SZ_32M;
+ break;
+ case 3:
+ max_size = SZ_16K;
+ break;
+ }
+ break;
+ case TLBI_TTL_TG_64K:
+ switch (level) {
+ case 0:
+ case 1:
+ /* No 52bit IPA support */
+ break;
+ case 2:
+ max_size = SZ_512M;
+ break;
+ case 3:
+ max_size = SZ_64K;
+ break;
+ }
+ break;
+ default: /* No size information */
+ break;
+ }
+
+ return max_size;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 60d2fb6edc93..61094f5e955e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -9,6 +9,7 @@
* Christoffer Dall <c.dall@virtualopensystems.com>
*/
+#include <linux/bitfield.h>
#include <linux/bsearch.h>
#include <linux/kvm_host.h>
#include <linux/mm.h>
@@ -2578,59 +2579,81 @@ static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return true;
}
+static unsigned long compute_tlb_inval_range(struct kvm_vcpu *vcpu,
+ struct kvm_s2_mmu *mmu,
+ u64 val)
+{
+ unsigned long max_size;
+ u8 ttl = 0;
+
+ if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL)) {
+ ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+ }
+
+ max_size = ttl_to_size(ttl);
+
+ if (!max_size) {
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+
+ /* Compute the maximum extent of the invalidation */
+ switch ((vtcr & VTCR_EL2_TG0_MASK)) {
+ case VTCR_EL2_TG0_4K:
+ max_size = SZ_1G;
+ break;
+ case VTCR_EL2_TG0_16K:
+ max_size = SZ_32M;
+ break;
+ case VTCR_EL2_TG0_64K:
+ /*
+ * No, we do not support 52bit IPA in nested yet. Once
+ * we do, this should be 4TB.
+ */
+ /* FIXME: remove the 52bit PA support from the IDregs */
+ max_size = SZ_512M;
+ break;
+ default:
+ BUG();
+ }
+ }
+
+ WARN_ON(!max_size);
+ return max_size;
+}
+
static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
- u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
struct kvm_s2_mmu *mmu;
u64 base_addr;
- int max_size;
+ unsigned long max_size;
/*
* We drop a number of things from the supplied value:
*
* - NS bit: we're non-secure only.
*
- * - TTL field: We already have the granule size from the
- * VTCR_EL2.TG0 field, and the level is only relevant to the
- * guest's S2PT.
- *
* - IPA[51:48]: We don't support 52bit IPA just yet...
*
* And of course, adjust the IPA to be on an actual address.
*/
base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
- /* Compute the maximum extent of the invalidation */
- switch ((vtcr & VTCR_EL2_TG0_MASK)) {
- case VTCR_EL2_TG0_4K:
- max_size = SZ_1G;
- break;
- case VTCR_EL2_TG0_16K:
- max_size = SZ_32M;
- break;
- case VTCR_EL2_TG0_64K:
- /*
- * No, we do not support 52bit IPA in nested yet. Once
- * we do, this should be 4TB.
- */
- /* FIXME: remove the 52bit PA support from the IDregs */
- max_size = SZ_512M;
- break;
- default:
- BUG();
- }
-
spin_lock(&vcpu->kvm->mmu_lock);
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
- if (mmu)
+ if (mmu) {
+ max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
+ base_addr &= ~(max_size - 1);
kvm_unmap_stage2_range(mmu, base_addr, max_size);
+ }
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
- if (mmu)
+ if (mmu) {
+ max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
+ base_addr &= ~(max_size - 1);
kvm_unmap_stage2_range(mmu, base_addr, max_size);
+ }
spin_unlock(&vcpu->kvm->mmu_lock);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 53/66] KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Support guest-provided information information to find out about
the range of required invalidation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 1 +
arch/arm64/kvm/nested.c | 57 +++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 79 +++++++++++++++++++----------
3 files changed, 109 insertions(+), 28 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 5fa3c634c8e1..7c47ad655e2e 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -123,6 +123,7 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+unsigned int ttl_to_size(u8 ttl);
struct sys_reg_params;
struct sys_reg_desc;
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9574fde823e6..05b711e730b6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -351,6 +351,63 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
return ret;
}
+
+unsigned int ttl_to_size(u8 ttl)
+{
+ int level = ttl & 3;
+ int gran = (ttl >> 2) & 3;
+ unsigned int max_size = 0;
+
+ switch (gran) {
+ case TLBI_TTL_TG_4K:
+ switch (level) {
+ case 0:
+ break;
+ case 1:
+ max_size = SZ_1G;
+ break;
+ case 2:
+ max_size = SZ_2M;
+ break;
+ case 3:
+ max_size = SZ_4K;
+ break;
+ }
+ break;
+ case TLBI_TTL_TG_16K:
+ switch (level) {
+ case 0:
+ case 1:
+ break;
+ case 2:
+ max_size = SZ_32M;
+ break;
+ case 3:
+ max_size = SZ_16K;
+ break;
+ }
+ break;
+ case TLBI_TTL_TG_64K:
+ switch (level) {
+ case 0:
+ case 1:
+ /* No 52bit IPA support */
+ break;
+ case 2:
+ max_size = SZ_512M;
+ break;
+ case 3:
+ max_size = SZ_64K;
+ break;
+ }
+ break;
+ default: /* No size information */
+ break;
+ }
+
+ return max_size;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 60d2fb6edc93..61094f5e955e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -9,6 +9,7 @@
* Christoffer Dall <c.dall@virtualopensystems.com>
*/
+#include <linux/bitfield.h>
#include <linux/bsearch.h>
#include <linux/kvm_host.h>
#include <linux/mm.h>
@@ -2578,59 +2579,81 @@ static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return true;
}
+static unsigned long compute_tlb_inval_range(struct kvm_vcpu *vcpu,
+ struct kvm_s2_mmu *mmu,
+ u64 val)
+{
+ unsigned long max_size;
+ u8 ttl = 0;
+
+ if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL)) {
+ ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+ }
+
+ max_size = ttl_to_size(ttl);
+
+ if (!max_size) {
+ u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+
+ /* Compute the maximum extent of the invalidation */
+ switch ((vtcr & VTCR_EL2_TG0_MASK)) {
+ case VTCR_EL2_TG0_4K:
+ max_size = SZ_1G;
+ break;
+ case VTCR_EL2_TG0_16K:
+ max_size = SZ_32M;
+ break;
+ case VTCR_EL2_TG0_64K:
+ /*
+ * No, we do not support 52bit IPA in nested yet. Once
+ * we do, this should be 4TB.
+ */
+ /* FIXME: remove the 52bit PA support from the IDregs */
+ max_size = SZ_512M;
+ break;
+ default:
+ BUG();
+ }
+ }
+
+ WARN_ON(!max_size);
+ return max_size;
+}
+
static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
- u64 vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
struct kvm_s2_mmu *mmu;
u64 base_addr;
- int max_size;
+ unsigned long max_size;
/*
* We drop a number of things from the supplied value:
*
* - NS bit: we're non-secure only.
*
- * - TTL field: We already have the granule size from the
- * VTCR_EL2.TG0 field, and the level is only relevant to the
- * guest's S2PT.
- *
* - IPA[51:48]: We don't support 52bit IPA just yet...
*
* And of course, adjust the IPA to be on an actual address.
*/
base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
- /* Compute the maximum extent of the invalidation */
- switch ((vtcr & VTCR_EL2_TG0_MASK)) {
- case VTCR_EL2_TG0_4K:
- max_size = SZ_1G;
- break;
- case VTCR_EL2_TG0_16K:
- max_size = SZ_32M;
- break;
- case VTCR_EL2_TG0_64K:
- /*
- * No, we do not support 52bit IPA in nested yet. Once
- * we do, this should be 4TB.
- */
- /* FIXME: remove the 52bit PA support from the IDregs */
- max_size = SZ_512M;
- break;
- default:
- BUG();
- }
-
spin_lock(&vcpu->kvm->mmu_lock);
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
- if (mmu)
+ if (mmu) {
+ max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
+ base_addr &= ~(max_size - 1);
kvm_unmap_stage2_range(mmu, base_addr, max_size);
+ }
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, 0);
- if (mmu)
+ if (mmu) {
+ max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
+ base_addr &= ~(max_size - 1);
kvm_unmap_stage2_range(mmu, base_addr, max_size);
+ }
spin_unlock(&vcpu->kvm->mmu_lock);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 54/66] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
In order to be able to make S2 TLB invalidations more performant on NV,
let's use a scheme derived from the ARMv8.4 TTL extension.
If bits [56:55] in the descriptor are non-zero, they indicate a level
which can be used as an invalidation range.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 4 ++
arch/arm64/kvm/nested.c | 98 +++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 13 ++--
3 files changed, 110 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 7c47ad655e2e..34499c496ae6 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -123,6 +123,8 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+u8 get_guest_mapping_ttl(struct kvm_vcpu *vcpu, struct kvm_s2_mmu *mmu,
+ u64 addr);
unsigned int ttl_to_size(u8 ttl);
struct sys_reg_params;
@@ -131,4 +133,6 @@ struct sys_reg_desc;
void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
const struct sys_reg_desc *r);
+#define KVM_NV_GUEST_MAP_SZ GENMASK_ULL(56, 55)
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 05b711e730b6..eef8f9873814 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -16,6 +16,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/bitfield.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
@@ -351,6 +352,29 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
return ret;
}
+static int read_host_s2_desc(phys_addr_t pa, u64 *desc, void *data)
+{
+ u64 *va = phys_to_virt(pa);
+
+ *desc = *va;
+
+ return 0;
+}
+
+static int kvm_walk_shadow_s2(struct kvm_s2_mmu *mmu, phys_addr_t gipa,
+ struct kvm_s2_trans *result)
+{
+ struct s2_walk_info wi = { };
+
+ wi.read_desc = read_host_s2_desc;
+ wi.baddr = mmu->pgd_phys;
+
+ vtcr_to_walk_info(mmu->kvm->arch.vtcr, &wi);
+
+ wi.be = IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
+
+ return walk_nested_s2_pgd(gipa, &wi, result);
+}
unsigned int ttl_to_size(u8 ttl)
{
@@ -408,6 +432,80 @@ unsigned int ttl_to_size(u8 ttl)
return max_size;
}
+/*
+ * Compute the equivalent of the TTL field by parsing the shadow PT.
+ * The granule size is extracted from VTCR_EL2.TG0 while the level is
+ * retrieved from first entry carrying the level as a tag.
+ */
+u8 get_guest_mapping_ttl(struct kvm_vcpu *vcpu, struct kvm_s2_mmu *mmu,
+ u64 addr)
+{
+ u64 tmp, sz = 0, vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct kvm_s2_trans out;
+ u8 ttl, level;
+
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ ttl = (1 << 2);
+ break;
+ case VTCR_EL2_TG0_16K:
+ ttl = (2 << 2);
+ break;
+ case VTCR_EL2_TG0_64K:
+ ttl = (3 << 2);
+ break;
+ default:
+ BUG();
+ }
+
+ tmp = addr;
+
+again:
+ /* Iteratively compute the block sizes for a particular granule size */
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ if (sz < SZ_4K) sz = SZ_4K;
+ else if (sz < SZ_2M) sz = SZ_2M;
+ else if (sz < SZ_1G) sz = SZ_1G;
+ else sz = 0;
+ break;
+ case VTCR_EL2_TG0_16K:
+ if (sz < SZ_16K) sz = SZ_16K;
+ else if (sz < SZ_32M) sz = SZ_32M;
+ else sz = 0;
+ break;
+ case VTCR_EL2_TG0_64K:
+ if (sz < SZ_64K) sz = SZ_64K;
+ else if (sz < SZ_512M) sz = SZ_512M;
+ else sz = 0;
+ break;
+ default:
+ BUG();
+ }
+
+ if (sz == 0)
+ return 0;
+
+ tmp &= ~(sz - 1);
+ out = (struct kvm_s2_trans) { };
+ kvm_walk_shadow_s2(mmu, tmp, &out);
+ level = FIELD_GET(KVM_NV_GUEST_MAP_SZ, out.upper_attr);
+ if (!level)
+ goto again;
+
+ ttl |= level;
+
+ /*
+ * We now have found some level information in the shadow S2. Check
+ * that the resulting range is actually including the original IPA.
+ */
+ sz = ttl_to_size(ttl);
+ if (addr < (tmp + sz))
+ return ttl;
+
+ return 0;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 61094f5e955e..bd0a3e861ca5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2584,10 +2584,13 @@ static unsigned long compute_tlb_inval_range(struct kvm_vcpu *vcpu,
u64 val)
{
unsigned long max_size;
- u8 ttl = 0;
+ u8 ttl;
- if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL)) {
- ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+ ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+
+ if (!(cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL) && ttl)) {
+ u64 addr = (val & GENMASK_ULL(35, 0)) << 12;
+ ttl = get_guest_mapping_ttl(vcpu, mmu, addr);
}
max_size = ttl_to_size(ttl);
@@ -2628,6 +2631,8 @@ static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
u64 base_addr;
unsigned long max_size;
+ spin_lock(&vcpu->kvm->mmu_lock);
+
/*
* We drop a number of things from the supplied value:
*
@@ -2639,8 +2644,6 @@ static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
*/
base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
- spin_lock(&vcpu->kvm->mmu_lock);
-
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
if (mmu) {
max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 54/66] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
In order to be able to make S2 TLB invalidations more performant on NV,
let's use a scheme derived from the ARMv8.4 TTL extension.
If bits [56:55] in the descriptor are non-zero, they indicate a level
which can be used as an invalidation range.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 4 ++
arch/arm64/kvm/nested.c | 98 +++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 13 ++--
3 files changed, 110 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 7c47ad655e2e..34499c496ae6 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -123,6 +123,8 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+u8 get_guest_mapping_ttl(struct kvm_vcpu *vcpu, struct kvm_s2_mmu *mmu,
+ u64 addr);
unsigned int ttl_to_size(u8 ttl);
struct sys_reg_params;
@@ -131,4 +133,6 @@ struct sys_reg_desc;
void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
const struct sys_reg_desc *r);
+#define KVM_NV_GUEST_MAP_SZ GENMASK_ULL(56, 55)
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 05b711e730b6..eef8f9873814 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -16,6 +16,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/bitfield.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
@@ -351,6 +352,29 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
return ret;
}
+static int read_host_s2_desc(phys_addr_t pa, u64 *desc, void *data)
+{
+ u64 *va = phys_to_virt(pa);
+
+ *desc = *va;
+
+ return 0;
+}
+
+static int kvm_walk_shadow_s2(struct kvm_s2_mmu *mmu, phys_addr_t gipa,
+ struct kvm_s2_trans *result)
+{
+ struct s2_walk_info wi = { };
+
+ wi.read_desc = read_host_s2_desc;
+ wi.baddr = mmu->pgd_phys;
+
+ vtcr_to_walk_info(mmu->kvm->arch.vtcr, &wi);
+
+ wi.be = IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
+
+ return walk_nested_s2_pgd(gipa, &wi, result);
+}
unsigned int ttl_to_size(u8 ttl)
{
@@ -408,6 +432,80 @@ unsigned int ttl_to_size(u8 ttl)
return max_size;
}
+/*
+ * Compute the equivalent of the TTL field by parsing the shadow PT.
+ * The granule size is extracted from VTCR_EL2.TG0 while the level is
+ * retrieved from first entry carrying the level as a tag.
+ */
+u8 get_guest_mapping_ttl(struct kvm_vcpu *vcpu, struct kvm_s2_mmu *mmu,
+ u64 addr)
+{
+ u64 tmp, sz = 0, vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct kvm_s2_trans out;
+ u8 ttl, level;
+
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ ttl = (1 << 2);
+ break;
+ case VTCR_EL2_TG0_16K:
+ ttl = (2 << 2);
+ break;
+ case VTCR_EL2_TG0_64K:
+ ttl = (3 << 2);
+ break;
+ default:
+ BUG();
+ }
+
+ tmp = addr;
+
+again:
+ /* Iteratively compute the block sizes for a particular granule size */
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ if (sz < SZ_4K) sz = SZ_4K;
+ else if (sz < SZ_2M) sz = SZ_2M;
+ else if (sz < SZ_1G) sz = SZ_1G;
+ else sz = 0;
+ break;
+ case VTCR_EL2_TG0_16K:
+ if (sz < SZ_16K) sz = SZ_16K;
+ else if (sz < SZ_32M) sz = SZ_32M;
+ else sz = 0;
+ break;
+ case VTCR_EL2_TG0_64K:
+ if (sz < SZ_64K) sz = SZ_64K;
+ else if (sz < SZ_512M) sz = SZ_512M;
+ else sz = 0;
+ break;
+ default:
+ BUG();
+ }
+
+ if (sz == 0)
+ return 0;
+
+ tmp &= ~(sz - 1);
+ out = (struct kvm_s2_trans) { };
+ kvm_walk_shadow_s2(mmu, tmp, &out);
+ level = FIELD_GET(KVM_NV_GUEST_MAP_SZ, out.upper_attr);
+ if (!level)
+ goto again;
+
+ ttl |= level;
+
+ /*
+ * We now have found some level information in the shadow S2. Check
+ * that the resulting range is actually including the original IPA.
+ */
+ sz = ttl_to_size(ttl);
+ if (addr < (tmp + sz))
+ return ttl;
+
+ return 0;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 61094f5e955e..bd0a3e861ca5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2584,10 +2584,13 @@ static unsigned long compute_tlb_inval_range(struct kvm_vcpu *vcpu,
u64 val)
{
unsigned long max_size;
- u8 ttl = 0;
+ u8 ttl;
- if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL)) {
- ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+ ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+
+ if (!(cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL) && ttl)) {
+ u64 addr = (val & GENMASK_ULL(35, 0)) << 12;
+ ttl = get_guest_mapping_ttl(vcpu, mmu, addr);
}
max_size = ttl_to_size(ttl);
@@ -2628,6 +2631,8 @@ static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
u64 base_addr;
unsigned long max_size;
+ spin_lock(&vcpu->kvm->mmu_lock);
+
/*
* We drop a number of things from the supplied value:
*
@@ -2639,8 +2644,6 @@ static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
*/
base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
- spin_lock(&vcpu->kvm->mmu_lock);
-
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
if (mmu) {
max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 54/66] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
In order to be able to make S2 TLB invalidations more performant on NV,
let's use a scheme derived from the ARMv8.4 TTL extension.
If bits [56:55] in the descriptor are non-zero, they indicate a level
which can be used as an invalidation range.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 4 ++
arch/arm64/kvm/nested.c | 98 +++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 13 ++--
3 files changed, 110 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 7c47ad655e2e..34499c496ae6 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -123,6 +123,8 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
u64 control_bit);
extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit);
extern bool forward_nv_traps(struct kvm_vcpu *vcpu);
+u8 get_guest_mapping_ttl(struct kvm_vcpu *vcpu, struct kvm_s2_mmu *mmu,
+ u64 addr);
unsigned int ttl_to_size(u8 ttl);
struct sys_reg_params;
@@ -131,4 +133,6 @@ struct sys_reg_desc;
void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
const struct sys_reg_desc *r);
+#define KVM_NV_GUEST_MAP_SZ GENMASK_ULL(56, 55)
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 05b711e730b6..eef8f9873814 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -16,6 +16,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/bitfield.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
@@ -351,6 +352,29 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,
return ret;
}
+static int read_host_s2_desc(phys_addr_t pa, u64 *desc, void *data)
+{
+ u64 *va = phys_to_virt(pa);
+
+ *desc = *va;
+
+ return 0;
+}
+
+static int kvm_walk_shadow_s2(struct kvm_s2_mmu *mmu, phys_addr_t gipa,
+ struct kvm_s2_trans *result)
+{
+ struct s2_walk_info wi = { };
+
+ wi.read_desc = read_host_s2_desc;
+ wi.baddr = mmu->pgd_phys;
+
+ vtcr_to_walk_info(mmu->kvm->arch.vtcr, &wi);
+
+ wi.be = IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
+
+ return walk_nested_s2_pgd(gipa, &wi, result);
+}
unsigned int ttl_to_size(u8 ttl)
{
@@ -408,6 +432,80 @@ unsigned int ttl_to_size(u8 ttl)
return max_size;
}
+/*
+ * Compute the equivalent of the TTL field by parsing the shadow PT.
+ * The granule size is extracted from VTCR_EL2.TG0 while the level is
+ * retrieved from first entry carrying the level as a tag.
+ */
+u8 get_guest_mapping_ttl(struct kvm_vcpu *vcpu, struct kvm_s2_mmu *mmu,
+ u64 addr)
+{
+ u64 tmp, sz = 0, vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
+ struct kvm_s2_trans out;
+ u8 ttl, level;
+
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ ttl = (1 << 2);
+ break;
+ case VTCR_EL2_TG0_16K:
+ ttl = (2 << 2);
+ break;
+ case VTCR_EL2_TG0_64K:
+ ttl = (3 << 2);
+ break;
+ default:
+ BUG();
+ }
+
+ tmp = addr;
+
+again:
+ /* Iteratively compute the block sizes for a particular granule size */
+ switch (vtcr & VTCR_EL2_TG0_MASK) {
+ case VTCR_EL2_TG0_4K:
+ if (sz < SZ_4K) sz = SZ_4K;
+ else if (sz < SZ_2M) sz = SZ_2M;
+ else if (sz < SZ_1G) sz = SZ_1G;
+ else sz = 0;
+ break;
+ case VTCR_EL2_TG0_16K:
+ if (sz < SZ_16K) sz = SZ_16K;
+ else if (sz < SZ_32M) sz = SZ_32M;
+ else sz = 0;
+ break;
+ case VTCR_EL2_TG0_64K:
+ if (sz < SZ_64K) sz = SZ_64K;
+ else if (sz < SZ_512M) sz = SZ_512M;
+ else sz = 0;
+ break;
+ default:
+ BUG();
+ }
+
+ if (sz == 0)
+ return 0;
+
+ tmp &= ~(sz - 1);
+ out = (struct kvm_s2_trans) { };
+ kvm_walk_shadow_s2(mmu, tmp, &out);
+ level = FIELD_GET(KVM_NV_GUEST_MAP_SZ, out.upper_attr);
+ if (!level)
+ goto again;
+
+ ttl |= level;
+
+ /*
+ * We now have found some level information in the shadow S2. Check
+ * that the resulting range is actually including the original IPA.
+ */
+ sz = ttl_to_size(ttl);
+ if (addr < (tmp + sz))
+ return ttl;
+
+ return 0;
+}
+
/* Must be called with kvm->lock held */
struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
{
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 61094f5e955e..bd0a3e861ca5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2584,10 +2584,13 @@ static unsigned long compute_tlb_inval_range(struct kvm_vcpu *vcpu,
u64 val)
{
unsigned long max_size;
- u8 ttl = 0;
+ u8 ttl;
- if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL)) {
- ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+ ttl = FIELD_GET(GENMASK_ULL(47, 44), val);
+
+ if (!(cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL) && ttl)) {
+ u64 addr = (val & GENMASK_ULL(35, 0)) << 12;
+ ttl = get_guest_mapping_ttl(vcpu, mmu, addr);
}
max_size = ttl_to_size(ttl);
@@ -2628,6 +2631,8 @@ static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
u64 base_addr;
unsigned long max_size;
+ spin_lock(&vcpu->kvm->mmu_lock);
+
/*
* We drop a number of things from the supplied value:
*
@@ -2639,8 +2644,6 @@ static bool handle_ipas2e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
*/
base_addr = (p->regval & GENMASK_ULL(35, 0)) << 12;
- spin_lock(&vcpu->kvm->mmu_lock);
-
mmu = lookup_s2_mmu(vcpu->kvm, vttbr, HCR_VM);
if (mmu) {
max_size = compute_tlb_inval_range(vcpu, mmu, p->regval);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 55/66] KVM: arm64: Allow populating S2 SW bits
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
The S2 page table code doesn't use the SW bits yet, but we are about
to need them to encode some guest Stage-2 information (its mapping size
in the form of the TTL encoding).
Propagate the SW bits specified by the caller, and store them into
the corresponding entry.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_pgtable.h | 10 ++++++++++
arch/arm64/kvm/hyp/pgtable.c | 6 ++++++
2 files changed, 16 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 52ab38db04c7..90a369185702 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -35,6 +35,10 @@ struct kvm_pgtable {
* @KVM_PGTABLE_PROT_W: Write permission.
* @KVM_PGTABLE_PROT_R: Read permission.
* @KVM_PGTABLE_PROT_DEVICE: Device attributes.
+ * @KVM_PGTABLE_PROT_S2_SW0: SW bit 0.
+ * @KVM_PGTABLE_PROT_S2_SW1: SW bit 1.
+ * @KVM_PGTABLE_PROT_S2_SW2: SW bit 2.
+ * @KVM_PGTABLE_PROT_S2_SW3: SW bit 3.
*/
enum kvm_pgtable_prot {
KVM_PGTABLE_PROT_X = BIT(0),
@@ -42,6 +46,12 @@ enum kvm_pgtable_prot {
KVM_PGTABLE_PROT_R = BIT(2),
KVM_PGTABLE_PROT_DEVICE = BIT(3),
+
+ /* Cunningly, this matches the PTE bits... */
+ KVM_PGTABLE_PROT_S2_SW0 = BIT(55),
+ KVM_PGTABLE_PROT_S2_SW1 = BIT(56),
+ KVM_PGTABLE_PROT_S2_SW2 = BIT(57),
+ KVM_PGTABLE_PROT_S2_SW3 = BIT(58),
};
#define PAGE_HYP (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W)
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 0271b4a3b9fe..9ee53e71a939 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -44,6 +44,7 @@
#define KVM_PTE_LEAF_ATTR_HI_S1_XN BIT(54)
#define KVM_PTE_LEAF_ATTR_HI_S2_XN BIT(54)
+#define KVM_PTE_LEAF_ATTR_HI_S2_SW GENMASK(58, 55)
struct kvm_pgtable_walk_data {
struct kvm_pgtable *pgt;
@@ -457,6 +458,7 @@ static int stage2_map_set_prot_attr(enum kvm_pgtable_prot prot,
attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh);
attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF;
+ attr |= prot & KVM_PTE_LEAF_ATTR_HI_S2_SW;
data->attr = attr;
return 0;
}
@@ -805,6 +807,10 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr,
if (prot & KVM_PGTABLE_PROT_X)
clr |= KVM_PTE_LEAF_ATTR_HI_S2_XN;
+ /* Always propagate the SW bits */
+ clr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_HI_S2_SW, 0xf);
+ set |= prot & KVM_PTE_LEAF_ATTR_HI_S2_SW;
+
ret = stage2_update_leaf_attrs(pgt, addr, 1, set, clr, NULL, &level);
if (!ret)
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, pgt->mmu, addr, level);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 55/66] KVM: arm64: Allow populating S2 SW bits
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
The S2 page table code doesn't use the SW bits yet, but we are about
to need them to encode some guest Stage-2 information (its mapping size
in the form of the TTL encoding).
Propagate the SW bits specified by the caller, and store them into
the corresponding entry.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_pgtable.h | 10 ++++++++++
arch/arm64/kvm/hyp/pgtable.c | 6 ++++++
2 files changed, 16 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 52ab38db04c7..90a369185702 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -35,6 +35,10 @@ struct kvm_pgtable {
* @KVM_PGTABLE_PROT_W: Write permission.
* @KVM_PGTABLE_PROT_R: Read permission.
* @KVM_PGTABLE_PROT_DEVICE: Device attributes.
+ * @KVM_PGTABLE_PROT_S2_SW0: SW bit 0.
+ * @KVM_PGTABLE_PROT_S2_SW1: SW bit 1.
+ * @KVM_PGTABLE_PROT_S2_SW2: SW bit 2.
+ * @KVM_PGTABLE_PROT_S2_SW3: SW bit 3.
*/
enum kvm_pgtable_prot {
KVM_PGTABLE_PROT_X = BIT(0),
@@ -42,6 +46,12 @@ enum kvm_pgtable_prot {
KVM_PGTABLE_PROT_R = BIT(2),
KVM_PGTABLE_PROT_DEVICE = BIT(3),
+
+ /* Cunningly, this matches the PTE bits... */
+ KVM_PGTABLE_PROT_S2_SW0 = BIT(55),
+ KVM_PGTABLE_PROT_S2_SW1 = BIT(56),
+ KVM_PGTABLE_PROT_S2_SW2 = BIT(57),
+ KVM_PGTABLE_PROT_S2_SW3 = BIT(58),
};
#define PAGE_HYP (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W)
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 0271b4a3b9fe..9ee53e71a939 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -44,6 +44,7 @@
#define KVM_PTE_LEAF_ATTR_HI_S1_XN BIT(54)
#define KVM_PTE_LEAF_ATTR_HI_S2_XN BIT(54)
+#define KVM_PTE_LEAF_ATTR_HI_S2_SW GENMASK(58, 55)
struct kvm_pgtable_walk_data {
struct kvm_pgtable *pgt;
@@ -457,6 +458,7 @@ static int stage2_map_set_prot_attr(enum kvm_pgtable_prot prot,
attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh);
attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF;
+ attr |= prot & KVM_PTE_LEAF_ATTR_HI_S2_SW;
data->attr = attr;
return 0;
}
@@ -805,6 +807,10 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr,
if (prot & KVM_PGTABLE_PROT_X)
clr |= KVM_PTE_LEAF_ATTR_HI_S2_XN;
+ /* Always propagate the SW bits */
+ clr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_HI_S2_SW, 0xf);
+ set |= prot & KVM_PTE_LEAF_ATTR_HI_S2_SW;
+
ret = stage2_update_leaf_attrs(pgt, addr, 1, set, clr, NULL, &level);
if (!ret)
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, pgt->mmu, addr, level);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 55/66] KVM: arm64: Allow populating S2 SW bits
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
The S2 page table code doesn't use the SW bits yet, but we are about
to need them to encode some guest Stage-2 information (its mapping size
in the form of the TTL encoding).
Propagate the SW bits specified by the caller, and store them into
the corresponding entry.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_pgtable.h | 10 ++++++++++
arch/arm64/kvm/hyp/pgtable.c | 6 ++++++
2 files changed, 16 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 52ab38db04c7..90a369185702 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -35,6 +35,10 @@ struct kvm_pgtable {
* @KVM_PGTABLE_PROT_W: Write permission.
* @KVM_PGTABLE_PROT_R: Read permission.
* @KVM_PGTABLE_PROT_DEVICE: Device attributes.
+ * @KVM_PGTABLE_PROT_S2_SW0: SW bit 0.
+ * @KVM_PGTABLE_PROT_S2_SW1: SW bit 1.
+ * @KVM_PGTABLE_PROT_S2_SW2: SW bit 2.
+ * @KVM_PGTABLE_PROT_S2_SW3: SW bit 3.
*/
enum kvm_pgtable_prot {
KVM_PGTABLE_PROT_X = BIT(0),
@@ -42,6 +46,12 @@ enum kvm_pgtable_prot {
KVM_PGTABLE_PROT_R = BIT(2),
KVM_PGTABLE_PROT_DEVICE = BIT(3),
+
+ /* Cunningly, this matches the PTE bits... */
+ KVM_PGTABLE_PROT_S2_SW0 = BIT(55),
+ KVM_PGTABLE_PROT_S2_SW1 = BIT(56),
+ KVM_PGTABLE_PROT_S2_SW2 = BIT(57),
+ KVM_PGTABLE_PROT_S2_SW3 = BIT(58),
};
#define PAGE_HYP (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W)
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 0271b4a3b9fe..9ee53e71a939 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -44,6 +44,7 @@
#define KVM_PTE_LEAF_ATTR_HI_S1_XN BIT(54)
#define KVM_PTE_LEAF_ATTR_HI_S2_XN BIT(54)
+#define KVM_PTE_LEAF_ATTR_HI_S2_SW GENMASK(58, 55)
struct kvm_pgtable_walk_data {
struct kvm_pgtable *pgt;
@@ -457,6 +458,7 @@ static int stage2_map_set_prot_attr(enum kvm_pgtable_prot prot,
attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh);
attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF;
+ attr |= prot & KVM_PTE_LEAF_ATTR_HI_S2_SW;
data->attr = attr;
return 0;
}
@@ -805,6 +807,10 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr,
if (prot & KVM_PGTABLE_PROT_X)
clr |= KVM_PTE_LEAF_ATTR_HI_S2_XN;
+ /* Always propagate the SW bits */
+ clr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_HI_S2_SW, 0xf);
+ set |= prot & KVM_PTE_LEAF_ATTR_HI_S2_SW;
+
ret = stage2_update_leaf_attrs(pgt, addr, 1, set, clr, NULL, &level);
if (!ret)
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, pgt->mmu, addr, level);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 56/66] KVM: arm64: nv: Tag shadow S2 entries with nested level
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Populate bits [56:55] of the leaf entry with the level provided
by the guest's S2 translation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 8 ++++++++
arch/arm64/kvm/mmu.c | 6 ++++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 34499c496ae6..36f2cd2c6fdf 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -5,6 +5,8 @@
#include <linux/bitfield.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_pgtable.h>
+
static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
{
return (!__is_defined(__KVM_NVHE_HYPERVISOR__) &&
@@ -135,4 +137,10 @@ void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
#define KVM_NV_GUEST_MAP_SZ GENMASK_ULL(56, 55)
+static inline u64 kvm_encode_nested_level(struct kvm_s2_trans *trans)
+{
+ return FIELD_PREP(KVM_PGTABLE_PROT_S2_SW1 | KVM_PGTABLE_PROT_S2_SW0,
+ trans->level);
+}
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 5fb13995190e..0994b3264d28 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -920,11 +920,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
* Potentially reduce shadow S2 permissions to match the guest's own
* S2. For exec faults, we'd only reach this point if the guest
* actually allowed it (see kvm_s2_handle_perm_fault).
+ *
+ * Also encode the level of the nested translation in the SW bits of
+ * the PTE/PMD/PUD. This will be retrived on TLB invalidation from
+ * the guest.
*/
if (kvm_is_shadow_s2_fault(vcpu)) {
writable &= kvm_s2_trans_writable(nested);
if (!kvm_s2_trans_readable(nested))
prot &= ~KVM_PGTABLE_PROT_R;
+
+ prot |= kvm_encode_nested_level(nested);
}
spin_lock(&kvm->mmu_lock);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 56/66] KVM: arm64: nv: Tag shadow S2 entries with nested level
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Populate bits [56:55] of the leaf entry with the level provided
by the guest's S2 translation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 8 ++++++++
arch/arm64/kvm/mmu.c | 6 ++++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 34499c496ae6..36f2cd2c6fdf 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -5,6 +5,8 @@
#include <linux/bitfield.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_pgtable.h>
+
static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
{
return (!__is_defined(__KVM_NVHE_HYPERVISOR__) &&
@@ -135,4 +137,10 @@ void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
#define KVM_NV_GUEST_MAP_SZ GENMASK_ULL(56, 55)
+static inline u64 kvm_encode_nested_level(struct kvm_s2_trans *trans)
+{
+ return FIELD_PREP(KVM_PGTABLE_PROT_S2_SW1 | KVM_PGTABLE_PROT_S2_SW0,
+ trans->level);
+}
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 5fb13995190e..0994b3264d28 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -920,11 +920,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
* Potentially reduce shadow S2 permissions to match the guest's own
* S2. For exec faults, we'd only reach this point if the guest
* actually allowed it (see kvm_s2_handle_perm_fault).
+ *
+ * Also encode the level of the nested translation in the SW bits of
+ * the PTE/PMD/PUD. This will be retrived on TLB invalidation from
+ * the guest.
*/
if (kvm_is_shadow_s2_fault(vcpu)) {
writable &= kvm_s2_trans_writable(nested);
if (!kvm_s2_trans_readable(nested))
prot &= ~KVM_PGTABLE_PROT_R;
+
+ prot |= kvm_encode_nested_level(nested);
}
spin_lock(&kvm->mmu_lock);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 56/66] KVM: arm64: nv: Tag shadow S2 entries with nested level
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Populate bits [56:55] of the leaf entry with the level provided
by the guest's S2 translation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_nested.h | 8 ++++++++
arch/arm64/kvm/mmu.c | 6 ++++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 34499c496ae6..36f2cd2c6fdf 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -5,6 +5,8 @@
#include <linux/bitfield.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_pgtable.h>
+
static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
{
return (!__is_defined(__KVM_NVHE_HYPERVISOR__) &&
@@ -135,4 +137,10 @@ void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p,
#define KVM_NV_GUEST_MAP_SZ GENMASK_ULL(56, 55)
+static inline u64 kvm_encode_nested_level(struct kvm_s2_trans *trans)
+{
+ return FIELD_PREP(KVM_PGTABLE_PROT_S2_SW1 | KVM_PGTABLE_PROT_S2_SW0,
+ trans->level);
+}
+
#endif /* __ARM64_KVM_NESTED_H */
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 5fb13995190e..0994b3264d28 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -920,11 +920,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
* Potentially reduce shadow S2 permissions to match the guest's own
* S2. For exec faults, we'd only reach this point if the guest
* actually allowed it (see kvm_s2_handle_perm_fault).
+ *
+ * Also encode the level of the nested translation in the SW bits of
+ * the PTE/PMD/PUD. This will be retrived on TLB invalidation from
+ * the guest.
*/
if (kvm_is_shadow_s2_fault(vcpu)) {
writable &= kvm_s2_trans_writable(nested);
if (!kvm_s2_trans_readable(nested))
prot &= ~KVM_PGTABLE_PROT_R;
+
+ prot |= kvm_encode_nested_level(nested);
}
spin_lock(&kvm->mmu_lock);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 57/66] KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
VNCR_EL2 points to a page containing a number of system registers
accessed by a guest hypervisor when ARMv8.4-NV is enabled.
Let's document the offsets in that page, as we are going to use
this layout.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/vncr_mapping.h | 73 +++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h
new file mode 100644
index 000000000000..64c46d658fc8
--- /dev/null
+++ b/arch/arm64/include/asm/vncr_mapping.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * System register offsets in the VNCR page
+ * All offsets are *byte* displacements!
+ */
+
+#ifndef __ARM64_VNCR_MAPPING_H__
+#define __ARM64_VNCR_MAPPING_H__
+
+#define VNCR_VTTBR_EL2 0x020
+#define VNCR_VTCR_EL2 0x040
+#define VNCR_VMPIDR_EL2 0x050
+#define VNCR_CNTVOFF_EL2 0x060
+#define VNCR_HCR_EL2 0x078
+#define VNCR_HSTR_EL2 0x080
+#define VNCR_VPIDR_EL2 0x088
+#define VNCR_TPIDR_EL2 0x090
+#define VNCR_VNCR_EL2 0x0B0
+#define VNCR_CPACR_EL1 0x100
+#define VNCR_CONTEXTIDR_EL1 0x108
+#define VNCR_SCTLR_EL1 0x110
+#define VNCR_ACTLR_EL1 0x118
+#define VNCR_TCR_EL1 0x120
+#define VNCR_AFSR0_EL1 0x128
+#define VNCR_AFSR1_EL1 0x130
+#define VNCR_ESR_EL1 0x138
+#define VNCR_MAIR_EL1 0x140
+#define VNCR_AMAIR_EL1 0x148
+#define VNCR_MDSCR_EL1 0x158
+#define VNCR_SPSR_EL1 0x160
+#define VNCR_CNTV_CVAL_EL0 0x168
+#define VNCR_CNTV_CTL_EL0 0x170
+#define VNCR_CNTP_CVAL_EL0 0x178
+#define VNCR_CNTP_CTL_EL0 0x180
+#define VNCR_SCXTNUM_EL1 0x188
+#define VNCR_ZCR_EL1 0x1E0
+#define VNCR_TTBR0_EL1 0x200
+#define VNCR_TTBR1_EL1 0x210
+#define VNCR_FAR_EL1 0x220
+#define VNCR_ELR_EL1 0x230
+#define VNCR_SP_EL1 0x240
+#define VNCR_VBAR_EL1 0x250
+#define VNCR_ICH_LR0_EL2 0x400
+// VNCR_ICH_LRN_EL2(n) VNCR_ICH_LR0_EL2+8*((n) & 7)
+#define VNCR_ICH_AP0R0_EL2 0x480
+// VNCR_ICH_AP0RN_EL2(n) VNCR_ICH_AP0R0_EL2+8*((n) & 3)
+#define VNCR_ICH_AP1R0_EL2 0x4A0
+// VNCR_ICH_AP1RN_EL2(n) VNCR_ICH_AP1R0_EL2+8*((n) & 3)
+#define VNCR_ICH_HCR_EL2 0x4C0
+#define VNCR_ICH_VMCR_EL2 0x4C8
+#define VNCR_VDISR_EL2 0x500
+#define VNCR_PMBLIMITR_EL1 0x800
+#define VNCR_PMBPTR_EL1 0x810
+#define VNCR_PMBSR_EL1 0x820
+#define VNCR_PMSCR_EL1 0x828
+#define VNCR_PMSEVFR_EL1 0x830
+#define VNCR_PMSICR_EL1 0x838
+#define VNCR_PMSIRR_EL1 0x840
+#define VNCR_PMSLATFR_EL1 0x848
+#define VNCR_TRFCR_EL1 0x880
+#define VNCR_MPAM1_EL1 0x900
+#define VNCR_MPAMHCR_EL2 0x930
+#define VNCR_MPAMVPMV_EL2 0x938
+#define VNCR_MPAMVPM0_EL2 0x940
+#define VNCR_MPAMVPM1_EL2 0x948
+#define VNCR_MPAMVPM2_EL2 0x950
+#define VNCR_MPAMVPM3_EL2 0x958
+#define VNCR_MPAMVPM4_EL2 0x960
+#define VNCR_MPAMVPM5_EL2 0x968
+#define VNCR_MPAMVPM6_EL2 0x970
+#define VNCR_MPAMVPM7_EL2 0x978
+
+#endif /* __ARM64_VNCR_MAPPING_H__ */
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 57/66] KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
VNCR_EL2 points to a page containing a number of system registers
accessed by a guest hypervisor when ARMv8.4-NV is enabled.
Let's document the offsets in that page, as we are going to use
this layout.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/vncr_mapping.h | 73 +++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h
new file mode 100644
index 000000000000..64c46d658fc8
--- /dev/null
+++ b/arch/arm64/include/asm/vncr_mapping.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * System register offsets in the VNCR page
+ * All offsets are *byte* displacements!
+ */
+
+#ifndef __ARM64_VNCR_MAPPING_H__
+#define __ARM64_VNCR_MAPPING_H__
+
+#define VNCR_VTTBR_EL2 0x020
+#define VNCR_VTCR_EL2 0x040
+#define VNCR_VMPIDR_EL2 0x050
+#define VNCR_CNTVOFF_EL2 0x060
+#define VNCR_HCR_EL2 0x078
+#define VNCR_HSTR_EL2 0x080
+#define VNCR_VPIDR_EL2 0x088
+#define VNCR_TPIDR_EL2 0x090
+#define VNCR_VNCR_EL2 0x0B0
+#define VNCR_CPACR_EL1 0x100
+#define VNCR_CONTEXTIDR_EL1 0x108
+#define VNCR_SCTLR_EL1 0x110
+#define VNCR_ACTLR_EL1 0x118
+#define VNCR_TCR_EL1 0x120
+#define VNCR_AFSR0_EL1 0x128
+#define VNCR_AFSR1_EL1 0x130
+#define VNCR_ESR_EL1 0x138
+#define VNCR_MAIR_EL1 0x140
+#define VNCR_AMAIR_EL1 0x148
+#define VNCR_MDSCR_EL1 0x158
+#define VNCR_SPSR_EL1 0x160
+#define VNCR_CNTV_CVAL_EL0 0x168
+#define VNCR_CNTV_CTL_EL0 0x170
+#define VNCR_CNTP_CVAL_EL0 0x178
+#define VNCR_CNTP_CTL_EL0 0x180
+#define VNCR_SCXTNUM_EL1 0x188
+#define VNCR_ZCR_EL1 0x1E0
+#define VNCR_TTBR0_EL1 0x200
+#define VNCR_TTBR1_EL1 0x210
+#define VNCR_FAR_EL1 0x220
+#define VNCR_ELR_EL1 0x230
+#define VNCR_SP_EL1 0x240
+#define VNCR_VBAR_EL1 0x250
+#define VNCR_ICH_LR0_EL2 0x400
+// VNCR_ICH_LRN_EL2(n) VNCR_ICH_LR0_EL2+8*((n) & 7)
+#define VNCR_ICH_AP0R0_EL2 0x480
+// VNCR_ICH_AP0RN_EL2(n) VNCR_ICH_AP0R0_EL2+8*((n) & 3)
+#define VNCR_ICH_AP1R0_EL2 0x4A0
+// VNCR_ICH_AP1RN_EL2(n) VNCR_ICH_AP1R0_EL2+8*((n) & 3)
+#define VNCR_ICH_HCR_EL2 0x4C0
+#define VNCR_ICH_VMCR_EL2 0x4C8
+#define VNCR_VDISR_EL2 0x500
+#define VNCR_PMBLIMITR_EL1 0x800
+#define VNCR_PMBPTR_EL1 0x810
+#define VNCR_PMBSR_EL1 0x820
+#define VNCR_PMSCR_EL1 0x828
+#define VNCR_PMSEVFR_EL1 0x830
+#define VNCR_PMSICR_EL1 0x838
+#define VNCR_PMSIRR_EL1 0x840
+#define VNCR_PMSLATFR_EL1 0x848
+#define VNCR_TRFCR_EL1 0x880
+#define VNCR_MPAM1_EL1 0x900
+#define VNCR_MPAMHCR_EL2 0x930
+#define VNCR_MPAMVPMV_EL2 0x938
+#define VNCR_MPAMVPM0_EL2 0x940
+#define VNCR_MPAMVPM1_EL2 0x948
+#define VNCR_MPAMVPM2_EL2 0x950
+#define VNCR_MPAMVPM3_EL2 0x958
+#define VNCR_MPAMVPM4_EL2 0x960
+#define VNCR_MPAMVPM5_EL2 0x968
+#define VNCR_MPAMVPM6_EL2 0x970
+#define VNCR_MPAMVPM7_EL2 0x978
+
+#endif /* __ARM64_VNCR_MAPPING_H__ */
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 57/66] KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
VNCR_EL2 points to a page containing a number of system registers
accessed by a guest hypervisor when ARMv8.4-NV is enabled.
Let's document the offsets in that page, as we are going to use
this layout.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/vncr_mapping.h | 73 +++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h
new file mode 100644
index 000000000000..64c46d658fc8
--- /dev/null
+++ b/arch/arm64/include/asm/vncr_mapping.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * System register offsets in the VNCR page
+ * All offsets are *byte* displacements!
+ */
+
+#ifndef __ARM64_VNCR_MAPPING_H__
+#define __ARM64_VNCR_MAPPING_H__
+
+#define VNCR_VTTBR_EL2 0x020
+#define VNCR_VTCR_EL2 0x040
+#define VNCR_VMPIDR_EL2 0x050
+#define VNCR_CNTVOFF_EL2 0x060
+#define VNCR_HCR_EL2 0x078
+#define VNCR_HSTR_EL2 0x080
+#define VNCR_VPIDR_EL2 0x088
+#define VNCR_TPIDR_EL2 0x090
+#define VNCR_VNCR_EL2 0x0B0
+#define VNCR_CPACR_EL1 0x100
+#define VNCR_CONTEXTIDR_EL1 0x108
+#define VNCR_SCTLR_EL1 0x110
+#define VNCR_ACTLR_EL1 0x118
+#define VNCR_TCR_EL1 0x120
+#define VNCR_AFSR0_EL1 0x128
+#define VNCR_AFSR1_EL1 0x130
+#define VNCR_ESR_EL1 0x138
+#define VNCR_MAIR_EL1 0x140
+#define VNCR_AMAIR_EL1 0x148
+#define VNCR_MDSCR_EL1 0x158
+#define VNCR_SPSR_EL1 0x160
+#define VNCR_CNTV_CVAL_EL0 0x168
+#define VNCR_CNTV_CTL_EL0 0x170
+#define VNCR_CNTP_CVAL_EL0 0x178
+#define VNCR_CNTP_CTL_EL0 0x180
+#define VNCR_SCXTNUM_EL1 0x188
+#define VNCR_ZCR_EL1 0x1E0
+#define VNCR_TTBR0_EL1 0x200
+#define VNCR_TTBR1_EL1 0x210
+#define VNCR_FAR_EL1 0x220
+#define VNCR_ELR_EL1 0x230
+#define VNCR_SP_EL1 0x240
+#define VNCR_VBAR_EL1 0x250
+#define VNCR_ICH_LR0_EL2 0x400
+// VNCR_ICH_LRN_EL2(n) VNCR_ICH_LR0_EL2+8*((n) & 7)
+#define VNCR_ICH_AP0R0_EL2 0x480
+// VNCR_ICH_AP0RN_EL2(n) VNCR_ICH_AP0R0_EL2+8*((n) & 3)
+#define VNCR_ICH_AP1R0_EL2 0x4A0
+// VNCR_ICH_AP1RN_EL2(n) VNCR_ICH_AP1R0_EL2+8*((n) & 3)
+#define VNCR_ICH_HCR_EL2 0x4C0
+#define VNCR_ICH_VMCR_EL2 0x4C8
+#define VNCR_VDISR_EL2 0x500
+#define VNCR_PMBLIMITR_EL1 0x800
+#define VNCR_PMBPTR_EL1 0x810
+#define VNCR_PMBSR_EL1 0x820
+#define VNCR_PMSCR_EL1 0x828
+#define VNCR_PMSEVFR_EL1 0x830
+#define VNCR_PMSICR_EL1 0x838
+#define VNCR_PMSIRR_EL1 0x840
+#define VNCR_PMSLATFR_EL1 0x848
+#define VNCR_TRFCR_EL1 0x880
+#define VNCR_MPAM1_EL1 0x900
+#define VNCR_MPAMHCR_EL2 0x930
+#define VNCR_MPAMVPMV_EL2 0x938
+#define VNCR_MPAMVPM0_EL2 0x940
+#define VNCR_MPAMVPM1_EL2 0x948
+#define VNCR_MPAMVPM2_EL2 0x950
+#define VNCR_MPAMVPM3_EL2 0x958
+#define VNCR_MPAMVPM4_EL2 0x960
+#define VNCR_MPAMVPM5_EL2 0x968
+#define VNCR_MPAMVPM6_EL2 0x970
+#define VNCR_MPAMVPM7_EL2 0x978
+
+#endif /* __ARM64_VNCR_MAPPING_H__ */
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 58/66] KVM: arm64: Map VNCR-capable registers to a separate page
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
With ARMv8.4-NV, registers that can be directly accessed in memory
by the guest have to live at architected offsets in a special page.
Let's annotate the sysreg enum to reflect the offset at which they
are in this page, whith a little twist:
If running on HW that doesn't have the ARMv8.4-NV feature, or even
a VM that doesn't use NV, we store all the system registers in the
usual sys_regs array. The only difference with the pre-8.4
situation is that VNCR-capable registers are at a "similar" offset
as in the VNCR page (we can compute the actual offset at compile
time), and that the sys_regs array is both bigger and sparse.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 97 ++++++++++++++++++++-----------
1 file changed, 63 insertions(+), 34 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9e190a65e78b..a4832df71a52 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -26,6 +26,7 @@
#include <asm/kvm.h>
#include <asm/kvm_asm.h>
#include <asm/thread_info.h>
+#include <asm/vncr_mapping.h>
#define __KVM_HAVE_ARCH_INTC_INITIALIZED
@@ -171,31 +172,32 @@ struct kvm_vcpu_fault_info {
u64 disr_el1; /* Deferred [SError] Status Register */
};
+/*
+ * VNCR() just places the VNCR_capable registers in the enum after
+ * __VNCR_START__, and the value (after correction) to be an 8-byte offset
+ * from the VNCR base. As we don't require the enum to be otherwise ordered,
+ * we need the terrible hack below to ensure that we correctly size the
+ * sys_regs array, no matter what.
+ *
+ * The __MAX__ macro has been lifted from Sean Eron Anderson's wonderful
+ * treasure trove of bit hacks:
+ * https://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+ */
+#define __MAX__(x,y) ((x) ^ (((x) ^ (y)) & -((x) < (y))))
+#define VNCR(r) \
+ __before_##r, \
+ r = __VNCR_START__ + ((VNCR_ ## r) / 8), \
+ __after_##r = __MAX__(__before_##r - 1, r)
+
enum vcpu_sysreg {
__INVALID_SYSREG__, /* 0 is reserved as an invalid value */
MPIDR_EL1, /* MultiProcessor Affinity Register */
CSSELR_EL1, /* Cache Size Selection Register */
- SCTLR_EL1, /* System Control Register */
- ACTLR_EL1, /* Auxiliary Control Register */
- CPACR_EL1, /* Coprocessor Access Control */
- ZCR_EL1, /* SVE Control */
- TTBR0_EL1, /* Translation Table Base Register 0 */
- TTBR1_EL1, /* Translation Table Base Register 1 */
- TCR_EL1, /* Translation Control Register */
- ESR_EL1, /* Exception Syndrome Register */
- AFSR0_EL1, /* Auxiliary Fault Status Register 0 */
- AFSR1_EL1, /* Auxiliary Fault Status Register 1 */
- FAR_EL1, /* Fault Address Register */
- MAIR_EL1, /* Memory Attribute Indirection Register */
- VBAR_EL1, /* Vector Base Address Register */
- CONTEXTIDR_EL1, /* Context ID Register */
TPIDR_EL0, /* Thread ID, User R/W */
TPIDRRO_EL0, /* Thread ID, User R/O */
TPIDR_EL1, /* Thread ID, Privileged */
- AMAIR_EL1, /* Aux Memory Attribute Indirection Register */
CNTKCTL_EL1, /* Timer Control Register (EL1) */
PAR_EL1, /* Physical Address Register */
- MDSCR_EL1, /* Monitor Debug System Control Register */
MDCCINT_EL1, /* Monitor Debug Comms Channel Interrupt Enable Reg */
DISR_EL1, /* Deferred Interrupt Status Register */
@@ -226,16 +228,6 @@ enum vcpu_sysreg {
APGAKEYLO_EL1,
APGAKEYHI_EL1,
- ELR_EL1,
- SP_EL1,
- SPSR_EL1,
-
- CNTVOFF_EL2,
- CNTV_CVAL_EL0,
- CNTV_CTL_EL0,
- CNTP_CVAL_EL0,
- CNTP_CTL_EL0,
-
/* 32bit specific registers. */
DACR32_EL2, /* Domain Access Control Register */
IFSR32_EL2, /* Instruction Fault Status Register */
@@ -243,20 +235,14 @@ enum vcpu_sysreg {
DBGVCR32_EL2, /* Debug Vector Catch Register */
/* EL2 registers */
- VPIDR_EL2, /* Virtualization Processor ID Register */
- VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
SCTLR_EL2, /* System Control Register (EL2) */
ACTLR_EL2, /* Auxiliary Control Register (EL2) */
- HCR_EL2, /* Hypervisor Configuration Register */
MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
- HSTR_EL2, /* Hypervisor System Trap Register */
HACR_EL2, /* Hypervisor Auxiliary Control Register */
TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
TCR_EL2, /* Translation Control Register (EL2) */
- VTTBR_EL2, /* Virtualization Translation Table Base Register */
- VTCR_EL2, /* Virtualization Translation Control Register */
SPSR_EL2, /* EL2 saved program status register */
ELR_EL2, /* EL2 exception link register */
AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
@@ -270,7 +256,6 @@ enum vcpu_sysreg {
RVBAR_EL2, /* Reset Vector Base Address Register */
RMR_EL2, /* Reset Management Register */
CONTEXTIDR_EL2, /* Context ID Register (EL2) */
- TPIDR_EL2, /* EL2 Software Thread ID Register */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
SP_EL2, /* EL2 Stack Pointer */
CNTHP_CTL_EL2,
@@ -278,6 +263,41 @@ enum vcpu_sysreg {
CNTHV_CTL_EL2,
CNTHV_CVAL_EL2,
+ __VNCR_START__, /* Any VNCR-capable reg goes after this point */
+
+ VNCR(SCTLR_EL1),/* System Control Register */
+ VNCR(ACTLR_EL1),/* Auxiliary Control Register */
+ VNCR(CPACR_EL1),/* Coprocessor Access Control */
+ VNCR(ZCR_EL1), /* SVE Control */
+ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */
+ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */
+ VNCR(TCR_EL1), /* Translation Control Register */
+ VNCR(ESR_EL1), /* Exception Syndrome Register */
+ VNCR(AFSR0_EL1),/* Auxiliary Fault Status Register 0 */
+ VNCR(AFSR1_EL1),/* Auxiliary Fault Status Register 1 */
+ VNCR(FAR_EL1), /* Fault Address Register */
+ VNCR(MAIR_EL1), /* Memory Attribute Indirection Register */
+ VNCR(VBAR_EL1), /* Vector Base Address Register */
+ VNCR(CONTEXTIDR_EL1), /* Context ID Register */
+ VNCR(AMAIR_EL1),/* Aux Memory Attribute Indirection Register */
+ VNCR(MDSCR_EL1),/* Monitor Debug System Control Register */
+ VNCR(ELR_EL1),
+ VNCR(SP_EL1),
+ VNCR(SPSR_EL1),
+ VNCR(VPIDR_EL2),/* Virtualization Processor ID Register */
+ VNCR(VMPIDR_EL2),/* Virtualization Multiprocessor ID Register */
+ VNCR(HCR_EL2), /* Hypervisor Configuration Register */
+ VNCR(HSTR_EL2), /* Hypervisor System Trap Register */
+ VNCR(VTTBR_EL2),/* Virtualization Translation Table Base Register */
+ VNCR(VTCR_EL2), /* Virtualization Translation Control Register */
+ VNCR(TPIDR_EL2),/* EL2 Software Thread ID Register */
+
+ VNCR(CNTVOFF_EL2),
+ VNCR(CNTV_CVAL_EL0),
+ VNCR(CNTV_CTL_EL0),
+ VNCR(CNTP_CVAL_EL0),
+ VNCR(CNTP_CTL_EL0),
+
NR_SYS_REGS /* Nothing after this line! */
};
@@ -294,6 +314,9 @@ struct kvm_cpu_context {
u64 sys_regs[NR_SYS_REGS];
struct kvm_vcpu *__hyp_running_vcpu;
+
+ /* This pointer has to be 4kB aligned. */
+ u64 *vncr_array;
};
struct kvm_pmu_events {
@@ -489,7 +512,13 @@ struct kvm_vcpu_arch {
* for system registers that are never context switched, but only
* emulated.
*/
-#define __ctxt_sys_reg(c,r) (&(c)->sys_regs[(r)])
+static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
+{
+ if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
+ return &ctxt->vncr_array[r - __VNCR_START__];
+
+ return (u64 *)&ctxt->sys_regs[r];
+}
#define ctxt_sys_reg(c,r) (*__ctxt_sys_reg(c,r))
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 58/66] KVM: arm64: Map VNCR-capable registers to a separate page
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
With ARMv8.4-NV, registers that can be directly accessed in memory
by the guest have to live at architected offsets in a special page.
Let's annotate the sysreg enum to reflect the offset at which they
are in this page, whith a little twist:
If running on HW that doesn't have the ARMv8.4-NV feature, or even
a VM that doesn't use NV, we store all the system registers in the
usual sys_regs array. The only difference with the pre-8.4
situation is that VNCR-capable registers are at a "similar" offset
as in the VNCR page (we can compute the actual offset at compile
time), and that the sys_regs array is both bigger and sparse.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 97 ++++++++++++++++++++-----------
1 file changed, 63 insertions(+), 34 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9e190a65e78b..a4832df71a52 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -26,6 +26,7 @@
#include <asm/kvm.h>
#include <asm/kvm_asm.h>
#include <asm/thread_info.h>
+#include <asm/vncr_mapping.h>
#define __KVM_HAVE_ARCH_INTC_INITIALIZED
@@ -171,31 +172,32 @@ struct kvm_vcpu_fault_info {
u64 disr_el1; /* Deferred [SError] Status Register */
};
+/*
+ * VNCR() just places the VNCR_capable registers in the enum after
+ * __VNCR_START__, and the value (after correction) to be an 8-byte offset
+ * from the VNCR base. As we don't require the enum to be otherwise ordered,
+ * we need the terrible hack below to ensure that we correctly size the
+ * sys_regs array, no matter what.
+ *
+ * The __MAX__ macro has been lifted from Sean Eron Anderson's wonderful
+ * treasure trove of bit hacks:
+ * https://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+ */
+#define __MAX__(x,y) ((x) ^ (((x) ^ (y)) & -((x) < (y))))
+#define VNCR(r) \
+ __before_##r, \
+ r = __VNCR_START__ + ((VNCR_ ## r) / 8), \
+ __after_##r = __MAX__(__before_##r - 1, r)
+
enum vcpu_sysreg {
__INVALID_SYSREG__, /* 0 is reserved as an invalid value */
MPIDR_EL1, /* MultiProcessor Affinity Register */
CSSELR_EL1, /* Cache Size Selection Register */
- SCTLR_EL1, /* System Control Register */
- ACTLR_EL1, /* Auxiliary Control Register */
- CPACR_EL1, /* Coprocessor Access Control */
- ZCR_EL1, /* SVE Control */
- TTBR0_EL1, /* Translation Table Base Register 0 */
- TTBR1_EL1, /* Translation Table Base Register 1 */
- TCR_EL1, /* Translation Control Register */
- ESR_EL1, /* Exception Syndrome Register */
- AFSR0_EL1, /* Auxiliary Fault Status Register 0 */
- AFSR1_EL1, /* Auxiliary Fault Status Register 1 */
- FAR_EL1, /* Fault Address Register */
- MAIR_EL1, /* Memory Attribute Indirection Register */
- VBAR_EL1, /* Vector Base Address Register */
- CONTEXTIDR_EL1, /* Context ID Register */
TPIDR_EL0, /* Thread ID, User R/W */
TPIDRRO_EL0, /* Thread ID, User R/O */
TPIDR_EL1, /* Thread ID, Privileged */
- AMAIR_EL1, /* Aux Memory Attribute Indirection Register */
CNTKCTL_EL1, /* Timer Control Register (EL1) */
PAR_EL1, /* Physical Address Register */
- MDSCR_EL1, /* Monitor Debug System Control Register */
MDCCINT_EL1, /* Monitor Debug Comms Channel Interrupt Enable Reg */
DISR_EL1, /* Deferred Interrupt Status Register */
@@ -226,16 +228,6 @@ enum vcpu_sysreg {
APGAKEYLO_EL1,
APGAKEYHI_EL1,
- ELR_EL1,
- SP_EL1,
- SPSR_EL1,
-
- CNTVOFF_EL2,
- CNTV_CVAL_EL0,
- CNTV_CTL_EL0,
- CNTP_CVAL_EL0,
- CNTP_CTL_EL0,
-
/* 32bit specific registers. */
DACR32_EL2, /* Domain Access Control Register */
IFSR32_EL2, /* Instruction Fault Status Register */
@@ -243,20 +235,14 @@ enum vcpu_sysreg {
DBGVCR32_EL2, /* Debug Vector Catch Register */
/* EL2 registers */
- VPIDR_EL2, /* Virtualization Processor ID Register */
- VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
SCTLR_EL2, /* System Control Register (EL2) */
ACTLR_EL2, /* Auxiliary Control Register (EL2) */
- HCR_EL2, /* Hypervisor Configuration Register */
MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
- HSTR_EL2, /* Hypervisor System Trap Register */
HACR_EL2, /* Hypervisor Auxiliary Control Register */
TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
TCR_EL2, /* Translation Control Register (EL2) */
- VTTBR_EL2, /* Virtualization Translation Table Base Register */
- VTCR_EL2, /* Virtualization Translation Control Register */
SPSR_EL2, /* EL2 saved program status register */
ELR_EL2, /* EL2 exception link register */
AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
@@ -270,7 +256,6 @@ enum vcpu_sysreg {
RVBAR_EL2, /* Reset Vector Base Address Register */
RMR_EL2, /* Reset Management Register */
CONTEXTIDR_EL2, /* Context ID Register (EL2) */
- TPIDR_EL2, /* EL2 Software Thread ID Register */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
SP_EL2, /* EL2 Stack Pointer */
CNTHP_CTL_EL2,
@@ -278,6 +263,41 @@ enum vcpu_sysreg {
CNTHV_CTL_EL2,
CNTHV_CVAL_EL2,
+ __VNCR_START__, /* Any VNCR-capable reg goes after this point */
+
+ VNCR(SCTLR_EL1),/* System Control Register */
+ VNCR(ACTLR_EL1),/* Auxiliary Control Register */
+ VNCR(CPACR_EL1),/* Coprocessor Access Control */
+ VNCR(ZCR_EL1), /* SVE Control */
+ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */
+ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */
+ VNCR(TCR_EL1), /* Translation Control Register */
+ VNCR(ESR_EL1), /* Exception Syndrome Register */
+ VNCR(AFSR0_EL1),/* Auxiliary Fault Status Register 0 */
+ VNCR(AFSR1_EL1),/* Auxiliary Fault Status Register 1 */
+ VNCR(FAR_EL1), /* Fault Address Register */
+ VNCR(MAIR_EL1), /* Memory Attribute Indirection Register */
+ VNCR(VBAR_EL1), /* Vector Base Address Register */
+ VNCR(CONTEXTIDR_EL1), /* Context ID Register */
+ VNCR(AMAIR_EL1),/* Aux Memory Attribute Indirection Register */
+ VNCR(MDSCR_EL1),/* Monitor Debug System Control Register */
+ VNCR(ELR_EL1),
+ VNCR(SP_EL1),
+ VNCR(SPSR_EL1),
+ VNCR(VPIDR_EL2),/* Virtualization Processor ID Register */
+ VNCR(VMPIDR_EL2),/* Virtualization Multiprocessor ID Register */
+ VNCR(HCR_EL2), /* Hypervisor Configuration Register */
+ VNCR(HSTR_EL2), /* Hypervisor System Trap Register */
+ VNCR(VTTBR_EL2),/* Virtualization Translation Table Base Register */
+ VNCR(VTCR_EL2), /* Virtualization Translation Control Register */
+ VNCR(TPIDR_EL2),/* EL2 Software Thread ID Register */
+
+ VNCR(CNTVOFF_EL2),
+ VNCR(CNTV_CVAL_EL0),
+ VNCR(CNTV_CTL_EL0),
+ VNCR(CNTP_CVAL_EL0),
+ VNCR(CNTP_CTL_EL0),
+
NR_SYS_REGS /* Nothing after this line! */
};
@@ -294,6 +314,9 @@ struct kvm_cpu_context {
u64 sys_regs[NR_SYS_REGS];
struct kvm_vcpu *__hyp_running_vcpu;
+
+ /* This pointer has to be 4kB aligned. */
+ u64 *vncr_array;
};
struct kvm_pmu_events {
@@ -489,7 +512,13 @@ struct kvm_vcpu_arch {
* for system registers that are never context switched, but only
* emulated.
*/
-#define __ctxt_sys_reg(c,r) (&(c)->sys_regs[(r)])
+static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
+{
+ if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
+ return &ctxt->vncr_array[r - __VNCR_START__];
+
+ return (u64 *)&ctxt->sys_regs[r];
+}
#define ctxt_sys_reg(c,r) (*__ctxt_sys_reg(c,r))
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 58/66] KVM: arm64: Map VNCR-capable registers to a separate page
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
With ARMv8.4-NV, registers that can be directly accessed in memory
by the guest have to live at architected offsets in a special page.
Let's annotate the sysreg enum to reflect the offset at which they
are in this page, whith a little twist:
If running on HW that doesn't have the ARMv8.4-NV feature, or even
a VM that doesn't use NV, we store all the system registers in the
usual sys_regs array. The only difference with the pre-8.4
situation is that VNCR-capable registers are at a "similar" offset
as in the VNCR page (we can compute the actual offset at compile
time), and that the sys_regs array is both bigger and sparse.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 97 ++++++++++++++++++++-----------
1 file changed, 63 insertions(+), 34 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9e190a65e78b..a4832df71a52 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -26,6 +26,7 @@
#include <asm/kvm.h>
#include <asm/kvm_asm.h>
#include <asm/thread_info.h>
+#include <asm/vncr_mapping.h>
#define __KVM_HAVE_ARCH_INTC_INITIALIZED
@@ -171,31 +172,32 @@ struct kvm_vcpu_fault_info {
u64 disr_el1; /* Deferred [SError] Status Register */
};
+/*
+ * VNCR() just places the VNCR_capable registers in the enum after
+ * __VNCR_START__, and the value (after correction) to be an 8-byte offset
+ * from the VNCR base. As we don't require the enum to be otherwise ordered,
+ * we need the terrible hack below to ensure that we correctly size the
+ * sys_regs array, no matter what.
+ *
+ * The __MAX__ macro has been lifted from Sean Eron Anderson's wonderful
+ * treasure trove of bit hacks:
+ * https://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+ */
+#define __MAX__(x,y) ((x) ^ (((x) ^ (y)) & -((x) < (y))))
+#define VNCR(r) \
+ __before_##r, \
+ r = __VNCR_START__ + ((VNCR_ ## r) / 8), \
+ __after_##r = __MAX__(__before_##r - 1, r)
+
enum vcpu_sysreg {
__INVALID_SYSREG__, /* 0 is reserved as an invalid value */
MPIDR_EL1, /* MultiProcessor Affinity Register */
CSSELR_EL1, /* Cache Size Selection Register */
- SCTLR_EL1, /* System Control Register */
- ACTLR_EL1, /* Auxiliary Control Register */
- CPACR_EL1, /* Coprocessor Access Control */
- ZCR_EL1, /* SVE Control */
- TTBR0_EL1, /* Translation Table Base Register 0 */
- TTBR1_EL1, /* Translation Table Base Register 1 */
- TCR_EL1, /* Translation Control Register */
- ESR_EL1, /* Exception Syndrome Register */
- AFSR0_EL1, /* Auxiliary Fault Status Register 0 */
- AFSR1_EL1, /* Auxiliary Fault Status Register 1 */
- FAR_EL1, /* Fault Address Register */
- MAIR_EL1, /* Memory Attribute Indirection Register */
- VBAR_EL1, /* Vector Base Address Register */
- CONTEXTIDR_EL1, /* Context ID Register */
TPIDR_EL0, /* Thread ID, User R/W */
TPIDRRO_EL0, /* Thread ID, User R/O */
TPIDR_EL1, /* Thread ID, Privileged */
- AMAIR_EL1, /* Aux Memory Attribute Indirection Register */
CNTKCTL_EL1, /* Timer Control Register (EL1) */
PAR_EL1, /* Physical Address Register */
- MDSCR_EL1, /* Monitor Debug System Control Register */
MDCCINT_EL1, /* Monitor Debug Comms Channel Interrupt Enable Reg */
DISR_EL1, /* Deferred Interrupt Status Register */
@@ -226,16 +228,6 @@ enum vcpu_sysreg {
APGAKEYLO_EL1,
APGAKEYHI_EL1,
- ELR_EL1,
- SP_EL1,
- SPSR_EL1,
-
- CNTVOFF_EL2,
- CNTV_CVAL_EL0,
- CNTV_CTL_EL0,
- CNTP_CVAL_EL0,
- CNTP_CTL_EL0,
-
/* 32bit specific registers. */
DACR32_EL2, /* Domain Access Control Register */
IFSR32_EL2, /* Instruction Fault Status Register */
@@ -243,20 +235,14 @@ enum vcpu_sysreg {
DBGVCR32_EL2, /* Debug Vector Catch Register */
/* EL2 registers */
- VPIDR_EL2, /* Virtualization Processor ID Register */
- VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
SCTLR_EL2, /* System Control Register (EL2) */
ACTLR_EL2, /* Auxiliary Control Register (EL2) */
- HCR_EL2, /* Hypervisor Configuration Register */
MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
- HSTR_EL2, /* Hypervisor System Trap Register */
HACR_EL2, /* Hypervisor Auxiliary Control Register */
TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
TCR_EL2, /* Translation Control Register (EL2) */
- VTTBR_EL2, /* Virtualization Translation Table Base Register */
- VTCR_EL2, /* Virtualization Translation Control Register */
SPSR_EL2, /* EL2 saved program status register */
ELR_EL2, /* EL2 exception link register */
AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
@@ -270,7 +256,6 @@ enum vcpu_sysreg {
RVBAR_EL2, /* Reset Vector Base Address Register */
RMR_EL2, /* Reset Management Register */
CONTEXTIDR_EL2, /* Context ID Register (EL2) */
- TPIDR_EL2, /* EL2 Software Thread ID Register */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
SP_EL2, /* EL2 Stack Pointer */
CNTHP_CTL_EL2,
@@ -278,6 +263,41 @@ enum vcpu_sysreg {
CNTHV_CTL_EL2,
CNTHV_CVAL_EL2,
+ __VNCR_START__, /* Any VNCR-capable reg goes after this point */
+
+ VNCR(SCTLR_EL1),/* System Control Register */
+ VNCR(ACTLR_EL1),/* Auxiliary Control Register */
+ VNCR(CPACR_EL1),/* Coprocessor Access Control */
+ VNCR(ZCR_EL1), /* SVE Control */
+ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */
+ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */
+ VNCR(TCR_EL1), /* Translation Control Register */
+ VNCR(ESR_EL1), /* Exception Syndrome Register */
+ VNCR(AFSR0_EL1),/* Auxiliary Fault Status Register 0 */
+ VNCR(AFSR1_EL1),/* Auxiliary Fault Status Register 1 */
+ VNCR(FAR_EL1), /* Fault Address Register */
+ VNCR(MAIR_EL1), /* Memory Attribute Indirection Register */
+ VNCR(VBAR_EL1), /* Vector Base Address Register */
+ VNCR(CONTEXTIDR_EL1), /* Context ID Register */
+ VNCR(AMAIR_EL1),/* Aux Memory Attribute Indirection Register */
+ VNCR(MDSCR_EL1),/* Monitor Debug System Control Register */
+ VNCR(ELR_EL1),
+ VNCR(SP_EL1),
+ VNCR(SPSR_EL1),
+ VNCR(VPIDR_EL2),/* Virtualization Processor ID Register */
+ VNCR(VMPIDR_EL2),/* Virtualization Multiprocessor ID Register */
+ VNCR(HCR_EL2), /* Hypervisor Configuration Register */
+ VNCR(HSTR_EL2), /* Hypervisor System Trap Register */
+ VNCR(VTTBR_EL2),/* Virtualization Translation Table Base Register */
+ VNCR(VTCR_EL2), /* Virtualization Translation Control Register */
+ VNCR(TPIDR_EL2),/* EL2 Software Thread ID Register */
+
+ VNCR(CNTVOFF_EL2),
+ VNCR(CNTV_CVAL_EL0),
+ VNCR(CNTV_CTL_EL0),
+ VNCR(CNTP_CVAL_EL0),
+ VNCR(CNTP_CTL_EL0),
+
NR_SYS_REGS /* Nothing after this line! */
};
@@ -294,6 +314,9 @@ struct kvm_cpu_context {
u64 sys_regs[NR_SYS_REGS];
struct kvm_vcpu *__hyp_running_vcpu;
+
+ /* This pointer has to be 4kB aligned. */
+ u64 *vncr_array;
};
struct kvm_pmu_events {
@@ -489,7 +512,13 @@ struct kvm_vcpu_arch {
* for system registers that are never context switched, but only
* emulated.
*/
-#define __ctxt_sys_reg(c,r) (&(c)->sys_regs[(r)])
+static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
+{
+ if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
+ return &ctxt->vncr_array[r - __VNCR_START__];
+
+ return (u64 *)&ctxt->sys_regs[r];
+}
#define ctxt_sys_reg(c,r) (*__ctxt_sys_reg(c,r))
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 59/66] KVM: arm64: nv: Move nested vgic state into the sysreg file
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
The vgic nested state needs to be accessible from the VNCR page, and
thus needs to be part of the normal sysreg file. Let's move it there.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 9 +++
arch/arm64/kvm/sys_regs.c | 53 +++++++++++------
arch/arm64/kvm/vgic/vgic-v3-nested.c | 88 ++++++++++++++--------------
arch/arm64/kvm/vgic/vgic-v3.c | 17 ++++--
arch/arm64/kvm/vgic/vgic.h | 10 ++++
include/kvm/arm_vgic.h | 7 ---
6 files changed, 110 insertions(+), 74 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index a4832df71a52..78630bd5124d 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -298,6 +298,15 @@ enum vcpu_sysreg {
VNCR(CNTP_CVAL_EL0),
VNCR(CNTP_CTL_EL0),
+ VNCR(ICH_LR0_EL2),
+ ICH_LR15_EL2 = ICH_LR0_EL2 + 15,
+ VNCR(ICH_AP0R0_EL2),
+ ICH_AP0R3_EL2 = ICH_AP0R0_EL2 + 3,
+ VNCR(ICH_AP1R0_EL2),
+ ICH_AP1R3_EL2 = ICH_AP1R0_EL2 + 3,
+ VNCR(ICH_HCR_EL2),
+ VNCR(ICH_VMCR_EL2),
+
NR_SYS_REGS /* Nothing after this line! */
};
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index bd0a3e861ca5..c7b004982ea9 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1790,17 +1790,17 @@ static bool access_gic_apr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
- u32 index, *base;
+ u64 *base;
+ u8 index;
index = r->Op2;
if (r->CRm == 8)
- base = cpu_if->vgic_ap0r;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_AP0R0_EL2);
else
- base = cpu_if->vgic_ap1r;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_AP1R0_EL2);
if (p->is_write)
- base[index] = p->regval;
+ base[index] = lower_32_bits(p->regval);
else
p->regval = base[index];
@@ -1811,12 +1811,10 @@ static bool access_gic_hcr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
-
if (p->is_write)
- cpu_if->vgic_hcr = p->regval;
+ __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = lower_32_bits(p->regval);
else
- p->regval = cpu_if->vgic_hcr;
+ p->regval = __vcpu_sys_reg(vcpu, ICH_HCR_EL2);
return true;
}
@@ -1873,12 +1871,19 @@ static bool access_gic_vmcr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
-
if (p->is_write)
- cpu_if->vgic_vmcr = p->regval;
+ __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = (p->regval &
+ (ICH_VMCR_ENG0_MASK |
+ ICH_VMCR_ENG1_MASK |
+ ICH_VMCR_PMR_MASK |
+ ICH_VMCR_BPR0_MASK |
+ ICH_VMCR_BPR1_MASK |
+ ICH_VMCR_EOIM_MASK |
+ ICH_VMCR_CBPR_MASK |
+ ICH_VMCR_FIQ_EN_MASK |
+ ICH_VMCR_ACK_CTL_MASK));
else
- p->regval = cpu_if->vgic_vmcr;
+ p->regval = __vcpu_sys_reg(vcpu, ICH_VMCR_EL2);
return true;
}
@@ -1887,17 +1892,29 @@ static bool access_gic_lr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
u32 index;
+ u64 *base;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_LR0_EL2);
index = p->Op2;
if (p->CRm == 13)
index += 8;
- if (p->is_write)
- cpu_if->vgic_lr[index] = p->regval;
- else
- p->regval = cpu_if->vgic_lr[index];
+ if (p->is_write) {
+ u64 mask = (ICH_LR_VIRTUAL_ID_MASK |
+ ICH_LR_GROUP |
+ ICH_LR_HW |
+ ICH_LR_STATE);
+
+ if (p->regval & ICH_LR_HW)
+ mask |= ICH_LR_PHYS_ID_MASK;
+ else
+ mask |= ICH_LR_EOI;
+
+ base[index] = p->regval & mask;
+ } else {
+ p->regval = base[index];
+ }
return true;
}
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 94b1edb67011..51f7a521e829 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -16,11 +16,6 @@
#define CREATE_TRACE_POINTS
#include "vgic-nested-trace.h"
-static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
-{
- return &vcpu->arch.vgic_cpu.nested_vgic_v3;
-}
-
static inline struct vgic_v3_cpu_if *vcpu_shadow_if(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.vgic_cpu.shadow_vgic_v3;
@@ -33,12 +28,11 @@ static inline bool lr_triggers_eoi(u64 lr)
u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
u16 reg = 0;
int i;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- if (lr_triggers_eoi(cpu_if->vgic_lr[i]))
+ if (lr_triggers_eoi(__vcpu_sys_reg(vcpu, ICH_LRN(i))))
reg |= BIT(i);
}
@@ -47,12 +41,11 @@ u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
u16 reg = 0;
int i;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- if (!(cpu_if->vgic_lr[i] & ICH_LR_STATE))
+ if (!(__vcpu_sys_reg(vcpu, ICH_LRN(i)) & ICH_LR_STATE))
reg |= BIT(i);
}
@@ -61,14 +54,13 @@ u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
int nr_lr = kvm_vgic_global_state.nr_lr;
u64 reg = 0;
if (vgic_v3_get_eisr(vcpu))
reg |= ICH_MISR_EOI;
- if (cpu_if->vgic_hcr & ICH_HCR_UIE) {
+ if (__vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_UIE) {
int used_lrs;
used_lrs = nr_lr - hweight16(vgic_v3_get_elrsr(vcpu));
@@ -87,13 +79,12 @@ u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
*/
static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
struct vgic_irq *irq;
int i, used_lrs = 0;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- u64 lr = cpu_if->vgic_lr[i];
+ u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i));
int l1_irq;
if (!(lr & ICH_LR_HW))
@@ -123,36 +114,20 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
}
trace_vgic_create_shadow_lrs(vcpu, kvm_vgic_global_state.nr_lr,
- s_cpu_if->vgic_lr, cpu_if->vgic_lr);
+ s_cpu_if->vgic_lr,
+ __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_LR0_EL2));
s_cpu_if->used_lrs = used_lrs;
}
-/*
- * Change the shadow HWIRQ field back to the virtual value before copying over
- * the entire shadow struct to the nested state.
- */
-static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
-{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
- struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
- int lr;
-
- for (lr = 0; lr < kvm_vgic_global_state.nr_lr; lr++) {
- s_cpu_if->vgic_lr[lr] &= ~ICH_LR_PHYS_ID_MASK;
- s_cpu_if->vgic_lr[lr] |= cpu_if->vgic_lr[lr] & ICH_LR_PHYS_ID_MASK;
- }
-}
-
void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
struct vgic_irq *irq;
int i;
for (i = 0; i < s_cpu_if->used_lrs; i++) {
- u64 lr = cpu_if->vgic_lr[i];
+ u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i));
int l1_irq;
if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE))
@@ -178,14 +153,27 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
}
}
+void vgic_v3_create_shadow_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+ int i;
+
+ cpu_if->vgic_hcr = __vcpu_sys_reg(vcpu, ICH_HCR_EL2);
+ cpu_if->vgic_vmcr = __vcpu_sys_reg(vcpu, ICH_VMCR_EL2);
+
+ for (i = 0; i < 4; i++) {
+ cpu_if->vgic_ap0r[i] = __vcpu_sys_reg(vcpu, ICH_AP0RN(i));
+ cpu_if->vgic_ap1r[i] = __vcpu_sys_reg(vcpu, ICH_AP1RN(i));
+ }
+
+ vgic_v3_create_shadow_lr(vcpu);
+}
+
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_irq *irq;
unsigned long flags;
- vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
- vgic_v3_create_shadow_lr(vcpu);
__vgic_v3_restore_state(vcpu_shadow_if(vcpu));
irq = vgic_get_irq(vcpu->kvm, vcpu, vcpu->kvm->arch.vgic.maint_irq);
@@ -199,26 +187,40 @@ void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ int i;
- __vgic_v3_save_state(vcpu_shadow_if(vcpu));
+ __vgic_v3_save_state(s_cpu_if);
- trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr,
- vcpu_shadow_if(vcpu)->vgic_lr);
+ trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr, s_cpu_if->vgic_lr);
/*
* Translate the shadow state HW fields back to the virtual ones
* before copying the shadow struct back to the nested one.
*/
- vgic_v3_fixup_shadow_lr_state(vcpu);
- vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+ __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = s_cpu_if->vgic_hcr;
+ __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = s_cpu_if->vgic_vmcr;
+
+ for (i = 0; i < 4; i++) {
+ __vcpu_sys_reg(vcpu, ICH_AP0RN(i)) = s_cpu_if->vgic_ap0r[i];
+ __vcpu_sys_reg(vcpu, ICH_AP1RN(i)) = s_cpu_if->vgic_ap1r[i];
+ }
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ u64 val = __vcpu_sys_reg(vcpu, ICH_LRN(i));
+
+ val &= ~ICH_LR_STATE;
+ val |= s_cpu_if->vgic_lr[i] & ICH_LR_STATE;
+
+ __vcpu_sys_reg(vcpu, ICH_LRN(i)) = val;
+ }
+
irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
IRQCHIP_STATE_ACTIVE, false);
}
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
bool state;
/*
@@ -230,7 +232,7 @@ void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
if (!vgic_state_is_nested(vcpu))
return;
- state = cpu_if->vgic_hcr & ICH_HCR_EN;
+ state = __vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_EN;
state &= vgic_v3_get_misr(vcpu);
kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index 47ea2a0f3ae4..d3b36eea8665 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -292,10 +292,11 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
ICC_SRE_EL1_SRE);
/*
* If nesting is allowed, force GICv3 onto the nested
- * guests as well.
+ * guests as well by setting the shadow state to the
+ * same value.
*/
if (nested_virt_in_use(vcpu))
- vcpu->arch.vgic_cpu.nested_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
+ vcpu->arch.vgic_cpu.shadow_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
@@ -662,11 +663,15 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
/*
- * vgic_v3_load_nested only affects the LRs in the shadow
- * state, so it is fine to pass the nested state around.
+ * If the vgic is in nested state, populate the shadow state
+ * from the guest's nested state. As vgic_v3_load_nested()
+ * will only load LRs, let's deal with the rest of the state
+ * here as if it was a non-nested state. Cunning.
*/
- if (vgic_state_is_nested(vcpu))
- cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ if (vgic_state_is_nested(vcpu)) {
+ vgic_v3_create_shadow_state(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+ }
/*
* If dealing with a GICv2 emulation on GICv3, VMCR_EL2.VFIQen
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 64fcd7511110..8f513a122699 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -318,4 +318,14 @@ int vgic_v4_init(struct kvm *kvm);
void vgic_v4_teardown(struct kvm *kvm);
void vgic_v4_configure_vsgis(struct kvm *kvm);
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_create_shadow_state(struct kvm_vcpu *vcpu);
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
+
+#define ICH_LRN(n) (ICH_LR0_EL2 + (n))
+#define ICH_AP0RN(n) (ICH_AP0R0_EL2 + (n))
+#define ICH_AP1RN(n) (ICH_AP1R0_EL2 + (n))
+
#endif
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index e02bc45e9508..7c46a8fba2aa 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -309,9 +309,6 @@ struct vgic_cpu {
struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
- /* CPU vif control registers for the virtual GICH interface */
- struct vgic_v3_cpu_if nested_vgic_v3;
-
/*
* The shadow vif control register loaded to the hardware when
* running a nested L2 guest with the virtual IMO/FMO bit set.
@@ -374,10 +371,6 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
-void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu);
u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu);
u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 59/66] KVM: arm64: nv: Move nested vgic state into the sysreg file
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
The vgic nested state needs to be accessible from the VNCR page, and
thus needs to be part of the normal sysreg file. Let's move it there.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 9 +++
arch/arm64/kvm/sys_regs.c | 53 +++++++++++------
arch/arm64/kvm/vgic/vgic-v3-nested.c | 88 ++++++++++++++--------------
arch/arm64/kvm/vgic/vgic-v3.c | 17 ++++--
arch/arm64/kvm/vgic/vgic.h | 10 ++++
include/kvm/arm_vgic.h | 7 ---
6 files changed, 110 insertions(+), 74 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index a4832df71a52..78630bd5124d 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -298,6 +298,15 @@ enum vcpu_sysreg {
VNCR(CNTP_CVAL_EL0),
VNCR(CNTP_CTL_EL0),
+ VNCR(ICH_LR0_EL2),
+ ICH_LR15_EL2 = ICH_LR0_EL2 + 15,
+ VNCR(ICH_AP0R0_EL2),
+ ICH_AP0R3_EL2 = ICH_AP0R0_EL2 + 3,
+ VNCR(ICH_AP1R0_EL2),
+ ICH_AP1R3_EL2 = ICH_AP1R0_EL2 + 3,
+ VNCR(ICH_HCR_EL2),
+ VNCR(ICH_VMCR_EL2),
+
NR_SYS_REGS /* Nothing after this line! */
};
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index bd0a3e861ca5..c7b004982ea9 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1790,17 +1790,17 @@ static bool access_gic_apr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
- u32 index, *base;
+ u64 *base;
+ u8 index;
index = r->Op2;
if (r->CRm == 8)
- base = cpu_if->vgic_ap0r;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_AP0R0_EL2);
else
- base = cpu_if->vgic_ap1r;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_AP1R0_EL2);
if (p->is_write)
- base[index] = p->regval;
+ base[index] = lower_32_bits(p->regval);
else
p->regval = base[index];
@@ -1811,12 +1811,10 @@ static bool access_gic_hcr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
-
if (p->is_write)
- cpu_if->vgic_hcr = p->regval;
+ __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = lower_32_bits(p->regval);
else
- p->regval = cpu_if->vgic_hcr;
+ p->regval = __vcpu_sys_reg(vcpu, ICH_HCR_EL2);
return true;
}
@@ -1873,12 +1871,19 @@ static bool access_gic_vmcr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
-
if (p->is_write)
- cpu_if->vgic_vmcr = p->regval;
+ __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = (p->regval &
+ (ICH_VMCR_ENG0_MASK |
+ ICH_VMCR_ENG1_MASK |
+ ICH_VMCR_PMR_MASK |
+ ICH_VMCR_BPR0_MASK |
+ ICH_VMCR_BPR1_MASK |
+ ICH_VMCR_EOIM_MASK |
+ ICH_VMCR_CBPR_MASK |
+ ICH_VMCR_FIQ_EN_MASK |
+ ICH_VMCR_ACK_CTL_MASK));
else
- p->regval = cpu_if->vgic_vmcr;
+ p->regval = __vcpu_sys_reg(vcpu, ICH_VMCR_EL2);
return true;
}
@@ -1887,17 +1892,29 @@ static bool access_gic_lr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
u32 index;
+ u64 *base;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_LR0_EL2);
index = p->Op2;
if (p->CRm == 13)
index += 8;
- if (p->is_write)
- cpu_if->vgic_lr[index] = p->regval;
- else
- p->regval = cpu_if->vgic_lr[index];
+ if (p->is_write) {
+ u64 mask = (ICH_LR_VIRTUAL_ID_MASK |
+ ICH_LR_GROUP |
+ ICH_LR_HW |
+ ICH_LR_STATE);
+
+ if (p->regval & ICH_LR_HW)
+ mask |= ICH_LR_PHYS_ID_MASK;
+ else
+ mask |= ICH_LR_EOI;
+
+ base[index] = p->regval & mask;
+ } else {
+ p->regval = base[index];
+ }
return true;
}
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 94b1edb67011..51f7a521e829 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -16,11 +16,6 @@
#define CREATE_TRACE_POINTS
#include "vgic-nested-trace.h"
-static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
-{
- return &vcpu->arch.vgic_cpu.nested_vgic_v3;
-}
-
static inline struct vgic_v3_cpu_if *vcpu_shadow_if(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.vgic_cpu.shadow_vgic_v3;
@@ -33,12 +28,11 @@ static inline bool lr_triggers_eoi(u64 lr)
u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
u16 reg = 0;
int i;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- if (lr_triggers_eoi(cpu_if->vgic_lr[i]))
+ if (lr_triggers_eoi(__vcpu_sys_reg(vcpu, ICH_LRN(i))))
reg |= BIT(i);
}
@@ -47,12 +41,11 @@ u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
u16 reg = 0;
int i;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- if (!(cpu_if->vgic_lr[i] & ICH_LR_STATE))
+ if (!(__vcpu_sys_reg(vcpu, ICH_LRN(i)) & ICH_LR_STATE))
reg |= BIT(i);
}
@@ -61,14 +54,13 @@ u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
int nr_lr = kvm_vgic_global_state.nr_lr;
u64 reg = 0;
if (vgic_v3_get_eisr(vcpu))
reg |= ICH_MISR_EOI;
- if (cpu_if->vgic_hcr & ICH_HCR_UIE) {
+ if (__vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_UIE) {
int used_lrs;
used_lrs = nr_lr - hweight16(vgic_v3_get_elrsr(vcpu));
@@ -87,13 +79,12 @@ u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
*/
static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
struct vgic_irq *irq;
int i, used_lrs = 0;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- u64 lr = cpu_if->vgic_lr[i];
+ u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i));
int l1_irq;
if (!(lr & ICH_LR_HW))
@@ -123,36 +114,20 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
}
trace_vgic_create_shadow_lrs(vcpu, kvm_vgic_global_state.nr_lr,
- s_cpu_if->vgic_lr, cpu_if->vgic_lr);
+ s_cpu_if->vgic_lr,
+ __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_LR0_EL2));
s_cpu_if->used_lrs = used_lrs;
}
-/*
- * Change the shadow HWIRQ field back to the virtual value before copying over
- * the entire shadow struct to the nested state.
- */
-static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
-{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
- struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
- int lr;
-
- for (lr = 0; lr < kvm_vgic_global_state.nr_lr; lr++) {
- s_cpu_if->vgic_lr[lr] &= ~ICH_LR_PHYS_ID_MASK;
- s_cpu_if->vgic_lr[lr] |= cpu_if->vgic_lr[lr] & ICH_LR_PHYS_ID_MASK;
- }
-}
-
void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
struct vgic_irq *irq;
int i;
for (i = 0; i < s_cpu_if->used_lrs; i++) {
- u64 lr = cpu_if->vgic_lr[i];
+ u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i));
int l1_irq;
if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE))
@@ -178,14 +153,27 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
}
}
+void vgic_v3_create_shadow_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+ int i;
+
+ cpu_if->vgic_hcr = __vcpu_sys_reg(vcpu, ICH_HCR_EL2);
+ cpu_if->vgic_vmcr = __vcpu_sys_reg(vcpu, ICH_VMCR_EL2);
+
+ for (i = 0; i < 4; i++) {
+ cpu_if->vgic_ap0r[i] = __vcpu_sys_reg(vcpu, ICH_AP0RN(i));
+ cpu_if->vgic_ap1r[i] = __vcpu_sys_reg(vcpu, ICH_AP1RN(i));
+ }
+
+ vgic_v3_create_shadow_lr(vcpu);
+}
+
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_irq *irq;
unsigned long flags;
- vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
- vgic_v3_create_shadow_lr(vcpu);
__vgic_v3_restore_state(vcpu_shadow_if(vcpu));
irq = vgic_get_irq(vcpu->kvm, vcpu, vcpu->kvm->arch.vgic.maint_irq);
@@ -199,26 +187,40 @@ void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ int i;
- __vgic_v3_save_state(vcpu_shadow_if(vcpu));
+ __vgic_v3_save_state(s_cpu_if);
- trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr,
- vcpu_shadow_if(vcpu)->vgic_lr);
+ trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr, s_cpu_if->vgic_lr);
/*
* Translate the shadow state HW fields back to the virtual ones
* before copying the shadow struct back to the nested one.
*/
- vgic_v3_fixup_shadow_lr_state(vcpu);
- vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+ __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = s_cpu_if->vgic_hcr;
+ __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = s_cpu_if->vgic_vmcr;
+
+ for (i = 0; i < 4; i++) {
+ __vcpu_sys_reg(vcpu, ICH_AP0RN(i)) = s_cpu_if->vgic_ap0r[i];
+ __vcpu_sys_reg(vcpu, ICH_AP1RN(i)) = s_cpu_if->vgic_ap1r[i];
+ }
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ u64 val = __vcpu_sys_reg(vcpu, ICH_LRN(i));
+
+ val &= ~ICH_LR_STATE;
+ val |= s_cpu_if->vgic_lr[i] & ICH_LR_STATE;
+
+ __vcpu_sys_reg(vcpu, ICH_LRN(i)) = val;
+ }
+
irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
IRQCHIP_STATE_ACTIVE, false);
}
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
bool state;
/*
@@ -230,7 +232,7 @@ void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
if (!vgic_state_is_nested(vcpu))
return;
- state = cpu_if->vgic_hcr & ICH_HCR_EN;
+ state = __vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_EN;
state &= vgic_v3_get_misr(vcpu);
kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index 47ea2a0f3ae4..d3b36eea8665 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -292,10 +292,11 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
ICC_SRE_EL1_SRE);
/*
* If nesting is allowed, force GICv3 onto the nested
- * guests as well.
+ * guests as well by setting the shadow state to the
+ * same value.
*/
if (nested_virt_in_use(vcpu))
- vcpu->arch.vgic_cpu.nested_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
+ vcpu->arch.vgic_cpu.shadow_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
@@ -662,11 +663,15 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
/*
- * vgic_v3_load_nested only affects the LRs in the shadow
- * state, so it is fine to pass the nested state around.
+ * If the vgic is in nested state, populate the shadow state
+ * from the guest's nested state. As vgic_v3_load_nested()
+ * will only load LRs, let's deal with the rest of the state
+ * here as if it was a non-nested state. Cunning.
*/
- if (vgic_state_is_nested(vcpu))
- cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ if (vgic_state_is_nested(vcpu)) {
+ vgic_v3_create_shadow_state(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+ }
/*
* If dealing with a GICv2 emulation on GICv3, VMCR_EL2.VFIQen
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 64fcd7511110..8f513a122699 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -318,4 +318,14 @@ int vgic_v4_init(struct kvm *kvm);
void vgic_v4_teardown(struct kvm *kvm);
void vgic_v4_configure_vsgis(struct kvm *kvm);
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_create_shadow_state(struct kvm_vcpu *vcpu);
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
+
+#define ICH_LRN(n) (ICH_LR0_EL2 + (n))
+#define ICH_AP0RN(n) (ICH_AP0R0_EL2 + (n))
+#define ICH_AP1RN(n) (ICH_AP1R0_EL2 + (n))
+
#endif
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index e02bc45e9508..7c46a8fba2aa 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -309,9 +309,6 @@ struct vgic_cpu {
struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
- /* CPU vif control registers for the virtual GICH interface */
- struct vgic_v3_cpu_if nested_vgic_v3;
-
/*
* The shadow vif control register loaded to the hardware when
* running a nested L2 guest with the virtual IMO/FMO bit set.
@@ -374,10 +371,6 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
-void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu);
u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu);
u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 59/66] KVM: arm64: nv: Move nested vgic state into the sysreg file
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
The vgic nested state needs to be accessible from the VNCR page, and
thus needs to be part of the normal sysreg file. Let's move it there.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 9 +++
arch/arm64/kvm/sys_regs.c | 53 +++++++++++------
arch/arm64/kvm/vgic/vgic-v3-nested.c | 88 ++++++++++++++--------------
arch/arm64/kvm/vgic/vgic-v3.c | 17 ++++--
arch/arm64/kvm/vgic/vgic.h | 10 ++++
include/kvm/arm_vgic.h | 7 ---
6 files changed, 110 insertions(+), 74 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index a4832df71a52..78630bd5124d 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -298,6 +298,15 @@ enum vcpu_sysreg {
VNCR(CNTP_CVAL_EL0),
VNCR(CNTP_CTL_EL0),
+ VNCR(ICH_LR0_EL2),
+ ICH_LR15_EL2 = ICH_LR0_EL2 + 15,
+ VNCR(ICH_AP0R0_EL2),
+ ICH_AP0R3_EL2 = ICH_AP0R0_EL2 + 3,
+ VNCR(ICH_AP1R0_EL2),
+ ICH_AP1R3_EL2 = ICH_AP1R0_EL2 + 3,
+ VNCR(ICH_HCR_EL2),
+ VNCR(ICH_VMCR_EL2),
+
NR_SYS_REGS /* Nothing after this line! */
};
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index bd0a3e861ca5..c7b004982ea9 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1790,17 +1790,17 @@ static bool access_gic_apr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
- u32 index, *base;
+ u64 *base;
+ u8 index;
index = r->Op2;
if (r->CRm == 8)
- base = cpu_if->vgic_ap0r;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_AP0R0_EL2);
else
- base = cpu_if->vgic_ap1r;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_AP1R0_EL2);
if (p->is_write)
- base[index] = p->regval;
+ base[index] = lower_32_bits(p->regval);
else
p->regval = base[index];
@@ -1811,12 +1811,10 @@ static bool access_gic_hcr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
-
if (p->is_write)
- cpu_if->vgic_hcr = p->regval;
+ __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = lower_32_bits(p->regval);
else
- p->regval = cpu_if->vgic_hcr;
+ p->regval = __vcpu_sys_reg(vcpu, ICH_HCR_EL2);
return true;
}
@@ -1873,12 +1871,19 @@ static bool access_gic_vmcr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
-
if (p->is_write)
- cpu_if->vgic_vmcr = p->regval;
+ __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = (p->regval &
+ (ICH_VMCR_ENG0_MASK |
+ ICH_VMCR_ENG1_MASK |
+ ICH_VMCR_PMR_MASK |
+ ICH_VMCR_BPR0_MASK |
+ ICH_VMCR_BPR1_MASK |
+ ICH_VMCR_EOIM_MASK |
+ ICH_VMCR_CBPR_MASK |
+ ICH_VMCR_FIQ_EN_MASK |
+ ICH_VMCR_ACK_CTL_MASK));
else
- p->regval = cpu_if->vgic_vmcr;
+ p->regval = __vcpu_sys_reg(vcpu, ICH_VMCR_EL2);
return true;
}
@@ -1887,17 +1892,29 @@ static bool access_gic_lr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
u32 index;
+ u64 *base;
+ base = __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_LR0_EL2);
index = p->Op2;
if (p->CRm == 13)
index += 8;
- if (p->is_write)
- cpu_if->vgic_lr[index] = p->regval;
- else
- p->regval = cpu_if->vgic_lr[index];
+ if (p->is_write) {
+ u64 mask = (ICH_LR_VIRTUAL_ID_MASK |
+ ICH_LR_GROUP |
+ ICH_LR_HW |
+ ICH_LR_STATE);
+
+ if (p->regval & ICH_LR_HW)
+ mask |= ICH_LR_PHYS_ID_MASK;
+ else
+ mask |= ICH_LR_EOI;
+
+ base[index] = p->regval & mask;
+ } else {
+ p->regval = base[index];
+ }
return true;
}
diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c
index 94b1edb67011..51f7a521e829 100644
--- a/arch/arm64/kvm/vgic/vgic-v3-nested.c
+++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c
@@ -16,11 +16,6 @@
#define CREATE_TRACE_POINTS
#include "vgic-nested-trace.h"
-static inline struct vgic_v3_cpu_if *vcpu_nested_if(struct kvm_vcpu *vcpu)
-{
- return &vcpu->arch.vgic_cpu.nested_vgic_v3;
-}
-
static inline struct vgic_v3_cpu_if *vcpu_shadow_if(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.vgic_cpu.shadow_vgic_v3;
@@ -33,12 +28,11 @@ static inline bool lr_triggers_eoi(u64 lr)
u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
u16 reg = 0;
int i;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- if (lr_triggers_eoi(cpu_if->vgic_lr[i]))
+ if (lr_triggers_eoi(__vcpu_sys_reg(vcpu, ICH_LRN(i))))
reg |= BIT(i);
}
@@ -47,12 +41,11 @@ u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu)
u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
u16 reg = 0;
int i;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- if (!(cpu_if->vgic_lr[i] & ICH_LR_STATE))
+ if (!(__vcpu_sys_reg(vcpu, ICH_LRN(i)) & ICH_LR_STATE))
reg |= BIT(i);
}
@@ -61,14 +54,13 @@ u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu)
u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
int nr_lr = kvm_vgic_global_state.nr_lr;
u64 reg = 0;
if (vgic_v3_get_eisr(vcpu))
reg |= ICH_MISR_EOI;
- if (cpu_if->vgic_hcr & ICH_HCR_UIE) {
+ if (__vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_UIE) {
int used_lrs;
used_lrs = nr_lr - hweight16(vgic_v3_get_elrsr(vcpu));
@@ -87,13 +79,12 @@ u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu)
*/
static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
struct vgic_irq *irq;
int i, used_lrs = 0;
for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
- u64 lr = cpu_if->vgic_lr[i];
+ u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i));
int l1_irq;
if (!(lr & ICH_LR_HW))
@@ -123,36 +114,20 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu)
}
trace_vgic_create_shadow_lrs(vcpu, kvm_vgic_global_state.nr_lr,
- s_cpu_if->vgic_lr, cpu_if->vgic_lr);
+ s_cpu_if->vgic_lr,
+ __ctxt_sys_reg(&vcpu->arch.ctxt, ICH_LR0_EL2));
s_cpu_if->used_lrs = used_lrs;
}
-/*
- * Change the shadow HWIRQ field back to the virtual value before copying over
- * the entire shadow struct to the nested state.
- */
-static void vgic_v3_fixup_shadow_lr_state(struct kvm_vcpu *vcpu)
-{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
- struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
- int lr;
-
- for (lr = 0; lr < kvm_vgic_global_state.nr_lr; lr++) {
- s_cpu_if->vgic_lr[lr] &= ~ICH_LR_PHYS_ID_MASK;
- s_cpu_if->vgic_lr[lr] |= cpu_if->vgic_lr[lr] & ICH_LR_PHYS_ID_MASK;
- }
-}
-
void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
struct vgic_irq *irq;
int i;
for (i = 0; i < s_cpu_if->used_lrs; i++) {
- u64 lr = cpu_if->vgic_lr[i];
+ u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i));
int l1_irq;
if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE))
@@ -178,14 +153,27 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu)
}
}
+void vgic_v3_create_shadow_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+ int i;
+
+ cpu_if->vgic_hcr = __vcpu_sys_reg(vcpu, ICH_HCR_EL2);
+ cpu_if->vgic_vmcr = __vcpu_sys_reg(vcpu, ICH_VMCR_EL2);
+
+ for (i = 0; i < 4; i++) {
+ cpu_if->vgic_ap0r[i] = __vcpu_sys_reg(vcpu, ICH_AP0RN(i));
+ cpu_if->vgic_ap1r[i] = __vcpu_sys_reg(vcpu, ICH_AP1RN(i));
+ }
+
+ vgic_v3_create_shadow_lr(vcpu);
+}
+
void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_irq *irq;
unsigned long flags;
- vgic_cpu->shadow_vgic_v3 = vgic_cpu->nested_vgic_v3;
- vgic_v3_create_shadow_lr(vcpu);
__vgic_v3_restore_state(vcpu_shadow_if(vcpu));
irq = vgic_get_irq(vcpu->kvm, vcpu, vcpu->kvm->arch.vgic.maint_irq);
@@ -199,26 +187,40 @@ void vgic_v3_load_nested(struct kvm_vcpu *vcpu)
void vgic_v3_put_nested(struct kvm_vcpu *vcpu)
{
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_v3_cpu_if *s_cpu_if = vcpu_shadow_if(vcpu);
+ int i;
- __vgic_v3_save_state(vcpu_shadow_if(vcpu));
+ __vgic_v3_save_state(s_cpu_if);
- trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr,
- vcpu_shadow_if(vcpu)->vgic_lr);
+ trace_vgic_put_nested(vcpu, kvm_vgic_global_state.nr_lr, s_cpu_if->vgic_lr);
/*
* Translate the shadow state HW fields back to the virtual ones
* before copying the shadow struct back to the nested one.
*/
- vgic_v3_fixup_shadow_lr_state(vcpu);
- vgic_cpu->nested_vgic_v3 = vgic_cpu->shadow_vgic_v3;
+ __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = s_cpu_if->vgic_hcr;
+ __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = s_cpu_if->vgic_vmcr;
+
+ for (i = 0; i < 4; i++) {
+ __vcpu_sys_reg(vcpu, ICH_AP0RN(i)) = s_cpu_if->vgic_ap0r[i];
+ __vcpu_sys_reg(vcpu, ICH_AP1RN(i)) = s_cpu_if->vgic_ap1r[i];
+ }
+
+ for (i = 0; i < kvm_vgic_global_state.nr_lr; i++) {
+ u64 val = __vcpu_sys_reg(vcpu, ICH_LRN(i));
+
+ val &= ~ICH_LR_STATE;
+ val |= s_cpu_if->vgic_lr[i] & ICH_LR_STATE;
+
+ __vcpu_sys_reg(vcpu, ICH_LRN(i)) = val;
+ }
+
irq_set_irqchip_state(kvm_vgic_global_state.maint_irq,
IRQCHIP_STATE_ACTIVE, false);
}
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
{
- struct vgic_v3_cpu_if *cpu_if = vcpu_nested_if(vcpu);
bool state;
/*
@@ -230,7 +232,7 @@ void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu)
if (!vgic_state_is_nested(vcpu))
return;
- state = cpu_if->vgic_hcr & ICH_HCR_EN;
+ state = __vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_EN;
state &= vgic_v3_get_misr(vcpu);
kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
index 47ea2a0f3ae4..d3b36eea8665 100644
--- a/arch/arm64/kvm/vgic/vgic-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-v3.c
@@ -292,10 +292,11 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
ICC_SRE_EL1_SRE);
/*
* If nesting is allowed, force GICv3 onto the nested
- * guests as well.
+ * guests as well by setting the shadow state to the
+ * same value.
*/
if (nested_virt_in_use(vcpu))
- vcpu->arch.vgic_cpu.nested_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
+ vcpu->arch.vgic_cpu.shadow_vgic_v3.vgic_sre = vgic_v3->vgic_sre;
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
@@ -662,11 +663,15 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
/*
- * vgic_v3_load_nested only affects the LRs in the shadow
- * state, so it is fine to pass the nested state around.
+ * If the vgic is in nested state, populate the shadow state
+ * from the guest's nested state. As vgic_v3_load_nested()
+ * will only load LRs, let's deal with the rest of the state
+ * here as if it was a non-nested state. Cunning.
*/
- if (vgic_state_is_nested(vcpu))
- cpu_if = &vcpu->arch.vgic_cpu.nested_vgic_v3;
+ if (vgic_state_is_nested(vcpu)) {
+ vgic_v3_create_shadow_state(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.shadow_vgic_v3;
+ }
/*
* If dealing with a GICv2 emulation on GICv3, VMCR_EL2.VFIQen
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 64fcd7511110..8f513a122699 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -318,4 +318,14 @@ int vgic_v4_init(struct kvm *kvm);
void vgic_v4_teardown(struct kvm *kvm);
void vgic_v4_configure_vsgis(struct kvm *kvm);
+void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_create_shadow_state(struct kvm_vcpu *vcpu);
+void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
+void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
+
+#define ICH_LRN(n) (ICH_LR0_EL2 + (n))
+#define ICH_AP0RN(n) (ICH_AP0R0_EL2 + (n))
+#define ICH_AP1RN(n) (ICH_AP1R0_EL2 + (n))
+
#endif
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index e02bc45e9508..7c46a8fba2aa 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -309,9 +309,6 @@ struct vgic_cpu {
struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
- /* CPU vif control registers for the virtual GICH interface */
- struct vgic_v3_cpu_if nested_vgic_v3;
-
/*
* The shadow vif control register loaded to the hardware when
* running a nested L2 guest with the virtual IMO/FMO bit set.
@@ -374,10 +371,6 @@ void kvm_vgic_load(struct kvm_vcpu *vcpu);
void kvm_vgic_put(struct kvm_vcpu *vcpu);
void kvm_vgic_vmcr_sync(struct kvm_vcpu *vcpu);
-void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_put_nested(struct kvm_vcpu *vcpu);
-void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
u16 vgic_v3_get_eisr(struct kvm_vcpu *vcpu);
u16 vgic_v3_get_elrsr(struct kvm_vcpu *vcpu);
u64 vgic_v3_get_misr(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 60/66] KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Add the detection code for the ARMv8.4-NV feature.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/cpucaps.h | 1 +
arch/arm64/include/asm/kvm_nested.h | 6 ++++++
arch/arm64/kernel/cpufeature.c | 10 ++++++++++
3 files changed, 17 insertions(+)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 6c5b84692a01..31619c73e320 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -17,6 +17,7 @@
#define ARM64_WORKAROUND_834220 7
#define ARM64_HAS_NO_HW_PREFETCH 8
#define ARM64_HAS_NESTED_VIRT 9
+#define ARM64_HAS_ENHANCED_NESTED_VIRT 10
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 36f2cd2c6fdf..c3c57eaa493a 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -14,6 +14,12 @@ static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
}
+static inline bool enhanced_nested_virt_in_use(const struct kvm_vcpu *vcpu)
+{
+ return cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
+ nested_virt_in_use(vcpu);
+}
+
/* Translation helpers from non-VHE EL2 to EL1 */
static inline u64 tcr_el2_ips_to_tcr_el1_ps(u64 tcr_el2)
{
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index eaeea8a88292..855c37d2f941 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1815,6 +1815,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64MMFR2_NV_SHIFT,
.min_field_value = 1,
},
+ {
+ .desc = "Enhanced Nested Virtualization Support",
+ .capability = ARM64_HAS_ENHANCED_NESTED_VIRT,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_nested_virt_support,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_NV_SHIFT,
+ .min_field_value = 2,
+ },
#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 60/66] KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Add the detection code for the ARMv8.4-NV feature.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/cpucaps.h | 1 +
arch/arm64/include/asm/kvm_nested.h | 6 ++++++
arch/arm64/kernel/cpufeature.c | 10 ++++++++++
3 files changed, 17 insertions(+)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 6c5b84692a01..31619c73e320 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -17,6 +17,7 @@
#define ARM64_WORKAROUND_834220 7
#define ARM64_HAS_NO_HW_PREFETCH 8
#define ARM64_HAS_NESTED_VIRT 9
+#define ARM64_HAS_ENHANCED_NESTED_VIRT 10
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 36f2cd2c6fdf..c3c57eaa493a 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -14,6 +14,12 @@ static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
}
+static inline bool enhanced_nested_virt_in_use(const struct kvm_vcpu *vcpu)
+{
+ return cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
+ nested_virt_in_use(vcpu);
+}
+
/* Translation helpers from non-VHE EL2 to EL1 */
static inline u64 tcr_el2_ips_to_tcr_el1_ps(u64 tcr_el2)
{
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index eaeea8a88292..855c37d2f941 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1815,6 +1815,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64MMFR2_NV_SHIFT,
.min_field_value = 1,
},
+ {
+ .desc = "Enhanced Nested Virtualization Support",
+ .capability = ARM64_HAS_ENHANCED_NESTED_VIRT,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_nested_virt_support,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_NV_SHIFT,
+ .min_field_value = 2,
+ },
#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 60/66] KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Add the detection code for the ARMv8.4-NV feature.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/cpucaps.h | 1 +
arch/arm64/include/asm/kvm_nested.h | 6 ++++++
arch/arm64/kernel/cpufeature.c | 10 ++++++++++
3 files changed, 17 insertions(+)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 6c5b84692a01..31619c73e320 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -17,6 +17,7 @@
#define ARM64_WORKAROUND_834220 7
#define ARM64_HAS_NO_HW_PREFETCH 8
#define ARM64_HAS_NESTED_VIRT 9
+#define ARM64_HAS_ENHANCED_NESTED_VIRT 10
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 36f2cd2c6fdf..c3c57eaa493a 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -14,6 +14,12 @@ static inline bool nested_virt_in_use(const struct kvm_vcpu *vcpu)
test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features));
}
+static inline bool enhanced_nested_virt_in_use(const struct kvm_vcpu *vcpu)
+{
+ return cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
+ nested_virt_in_use(vcpu);
+}
+
/* Translation helpers from non-VHE EL2 to EL1 */
static inline u64 tcr_el2_ips_to_tcr_el1_ps(u64 tcr_el2)
{
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index eaeea8a88292..855c37d2f941 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1815,6 +1815,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64MMFR2_NV_SHIFT,
.min_field_value = 1,
},
+ {
+ .desc = "Enhanced Nested Virtualization Support",
+ .capability = ARM64_HAS_ENHANCED_NESTED_VIRT,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_nested_virt_support,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_NV_SHIFT,
+ .min_field_value = 2,
+ },
#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 61/66] KVM: arm64: nv: Synchronize PSTATE early on exit
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
The NV code relies on predicates such as is_hyp_ctxt() being
reliable. In turn, is_hyp_ctxt() relies on things like PSTATE
and the virtual HCR_EL2 being accurate.
But with ARMv8.4-NV removing trapping for a large part of the
EL2 system registers (among which HCR_EL2), we can't use such
trapping to synchronize the rest of the state.
Let's look at the following sequence for a VHE guest:
(1) enter guest in host EL0
(2) guest traps to guest vEL2 (no hypervisor intervention)
(3) guest clears virtual HCR_EL2.TGE (no trap either)
(4) host interrupt fires, exit
(5) is_hyp_ctxt() now says "guest" (PSTATE.M==EL1 and TGE==0)
It is obvious that such behaviour would be rather unfortunate,
and lead to interesting, difficult to catch bugs specially if
preemption kicks in (yes, I wasted a whole week chasing this one).
In order to preserve the invariant that a guest entered in host
context must exit in the same context, we must make sure that
is_hyp_ctxt() works correctly. Since we can always observe the
guest value of HCR_EL2.{E2H,TGE} in the VNCR_EL2 page, we solely
need to synchronize PSTATE as early as possible.
This basically amounts to moving from_hw_pstate() as close
as possible to the guest exit point, and fixup_guest_exit()
seems as good a place as any.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 16 ++++--
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 26 ++--------
arch/arm64/kvm/hyp/nvhe/switch.c | 8 ++-
arch/arm64/kvm/hyp/vhe/switch.c | 57 +++++++++++++++++++++-
4 files changed, 78 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e5e201314c87..3b56841eb328 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -407,11 +407,11 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)
}
/*
- * Return true when we were able to fixup the guest exit and should return to
- * the guest, false when we should restore the host state and return to the
- * main run loop.
+ * Prologue for the guest fixup, populating ESR_EL2 and fixing up PC
+ * if required.
*/
-static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+static inline void fixup_guest_exit_prologue(struct kvm_vcpu *vcpu,
+ u64 *exit_code)
{
if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ)
vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR);
@@ -430,7 +430,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
if (esr_ec == ESR_ELx_EC_HVC32 || esr_ec == ESR_ELx_EC_HVC64)
write_sysreg_el2(read_sysreg_el2(SYS_ELR) - 4, SYS_ELR);
}
+}
+/*
+ * Return true when we were able to fixup the guest exit and should return to
+ * the guest, false when we should restore the host state and return to the
+ * main run loop.
+ */
+static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
/*
* We're using the raw exception code in order to only process
* the trap if no SError is pending. We will come back to the
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index 92715fa01e88..1931c8667d52 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -51,32 +51,12 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt_sys_reg(ctxt, SPSR_EL1) = read_sysreg_el1(SYS_SPSR);
}
-static inline u64 from_hw_pstate(const struct kvm_cpu_context *ctxt)
-{
- u64 reg = read_sysreg_el2(SYS_SPSR);
-
- if (__is_hyp_ctxt(ctxt)) {
- u64 mode = reg & (PSR_MODE_MASK | PSR_MODE32_BIT);
-
- switch (mode) {
- case PSR_MODE_EL1t:
- mode = PSR_MODE_EL2t;
- break;
- case PSR_MODE_EL1h:
- mode = PSR_MODE_EL2h;
- break;
- }
-
- return (reg & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
- }
-
- return reg;
-}
-
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
{
+ /* On VHE, PSTATE is saved in fixup_guest_exit_vhe() */
+ if (!has_vhe())
+ ctxt->regs.pstate = read_sysreg_el2(SYS_SPSR);
ctxt->regs.pc = read_sysreg_el2(SYS_ELR);
- ctxt->regs.pstate = from_hw_pstate(ctxt);
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index 07bf5c03631b..fb49757d3446 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -166,6 +166,12 @@ static void __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt)
write_sysreg(pmu->events_host, pmcntenset_el0);
}
+static bool fixup_guest_exit_nvhe(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+ fixup_guest_exit_prologue(vcpu, exit_code);
+ return fixup_guest_exit(vcpu, exit_code);
+}
+
/* Switch to the guest for legacy non-VHE systems */
int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
@@ -219,7 +225,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
exit_code = __guest_enter(vcpu);
/* And we're baaack! */
- } while (fixup_guest_exit(vcpu, &exit_code));
+ } while (fixup_guest_exit_nvhe(vcpu, &exit_code));
__sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index d9dc470c7790..4d80596e32a5 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -156,12 +156,60 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
+ bool hyp_ctxt)
+{
+ u64 pstate = read_sysreg_el2(SYS_SPSR);
+
+ /*
+ * Sync pstate back as early as possible, so that is_hyp_ctxt()
+ * reflects the exact context. It is otherwise possible to get
+ * confused with a VHE guest and ARMv8.4-NV, such as:
+ *
+ * (1) enter guest in host EL0
+ * (2) guest traps to guest vEL2 (no hypervisor intervention)
+ * (3) guest clears virtual HCR_EL2.TGE (no trap either)
+ * (4) host interrupt fires, exit
+ * (5) is_hyp_ctxt() now says "guest" (pstate.M==EL1 and TGE==0)
+ *
+ * If host preemption occurs, vcpu_load/put() will be very confused.
+ *
+ * Consider this as the prologue before the fixup prologue...
+ */
+
+ if (unlikely(hyp_ctxt)) {
+ u64 mode = pstate & PSR_MODE_MASK;
+
+ switch (mode) {
+ case PSR_MODE_EL1t:
+ mode = PSR_MODE_EL2t;
+ break;
+ case PSR_MODE_EL1h:
+ mode = PSR_MODE_EL2h;
+ break;
+ }
+
+ pstate = (pstate & ~PSR_MODE_MASK) | mode;
+ }
+
+ *vcpu_cpsr(vcpu) = pstate;
+
+ fixup_guest_exit_prologue(vcpu, exit_code);
+
+ if (*exit_code == ARM_EXCEPTION_TRAP) {
+ /* more to come here */
+ }
+
+ return fixup_guest_exit(vcpu, exit_code);
+}
+
/* Switch to the guest for VHE systems running in EL2 */
static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
u64 exit_code;
+ bool hyp_ctxt;
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
host_ctxt->__hyp_running_vcpu = vcpu;
@@ -188,12 +236,19 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
+ /*
+ * Being in HYP context or not is an invariant here. If we enter in
+ * a given context, we exit in the same context. We can thus only
+ * sample the context once.
+ */
+ WRITE_ONCE(hyp_ctxt, is_hyp_ctxt(vcpu));
+
do {
/* Jump in the fire! */
exit_code = __guest_enter(vcpu);
/* And we're baaack! */
- } while (fixup_guest_exit(vcpu, &exit_code));
+ } while (fixup_guest_exit_vhe(vcpu, &exit_code, READ_ONCE(hyp_ctxt)));
sysreg_save_guest_state_vhe(guest_ctxt);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 61/66] KVM: arm64: nv: Synchronize PSTATE early on exit
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
The NV code relies on predicates such as is_hyp_ctxt() being
reliable. In turn, is_hyp_ctxt() relies on things like PSTATE
and the virtual HCR_EL2 being accurate.
But with ARMv8.4-NV removing trapping for a large part of the
EL2 system registers (among which HCR_EL2), we can't use such
trapping to synchronize the rest of the state.
Let's look at the following sequence for a VHE guest:
(1) enter guest in host EL0
(2) guest traps to guest vEL2 (no hypervisor intervention)
(3) guest clears virtual HCR_EL2.TGE (no trap either)
(4) host interrupt fires, exit
(5) is_hyp_ctxt() now says "guest" (PSTATE.M==EL1 and TGE==0)
It is obvious that such behaviour would be rather unfortunate,
and lead to interesting, difficult to catch bugs specially if
preemption kicks in (yes, I wasted a whole week chasing this one).
In order to preserve the invariant that a guest entered in host
context must exit in the same context, we must make sure that
is_hyp_ctxt() works correctly. Since we can always observe the
guest value of HCR_EL2.{E2H,TGE} in the VNCR_EL2 page, we solely
need to synchronize PSTATE as early as possible.
This basically amounts to moving from_hw_pstate() as close
as possible to the guest exit point, and fixup_guest_exit()
seems as good a place as any.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 16 ++++--
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 26 ++--------
arch/arm64/kvm/hyp/nvhe/switch.c | 8 ++-
arch/arm64/kvm/hyp/vhe/switch.c | 57 +++++++++++++++++++++-
4 files changed, 78 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e5e201314c87..3b56841eb328 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -407,11 +407,11 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)
}
/*
- * Return true when we were able to fixup the guest exit and should return to
- * the guest, false when we should restore the host state and return to the
- * main run loop.
+ * Prologue for the guest fixup, populating ESR_EL2 and fixing up PC
+ * if required.
*/
-static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+static inline void fixup_guest_exit_prologue(struct kvm_vcpu *vcpu,
+ u64 *exit_code)
{
if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ)
vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR);
@@ -430,7 +430,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
if (esr_ec == ESR_ELx_EC_HVC32 || esr_ec == ESR_ELx_EC_HVC64)
write_sysreg_el2(read_sysreg_el2(SYS_ELR) - 4, SYS_ELR);
}
+}
+/*
+ * Return true when we were able to fixup the guest exit and should return to
+ * the guest, false when we should restore the host state and return to the
+ * main run loop.
+ */
+static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
/*
* We're using the raw exception code in order to only process
* the trap if no SError is pending. We will come back to the
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index 92715fa01e88..1931c8667d52 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -51,32 +51,12 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt_sys_reg(ctxt, SPSR_EL1) = read_sysreg_el1(SYS_SPSR);
}
-static inline u64 from_hw_pstate(const struct kvm_cpu_context *ctxt)
-{
- u64 reg = read_sysreg_el2(SYS_SPSR);
-
- if (__is_hyp_ctxt(ctxt)) {
- u64 mode = reg & (PSR_MODE_MASK | PSR_MODE32_BIT);
-
- switch (mode) {
- case PSR_MODE_EL1t:
- mode = PSR_MODE_EL2t;
- break;
- case PSR_MODE_EL1h:
- mode = PSR_MODE_EL2h;
- break;
- }
-
- return (reg & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
- }
-
- return reg;
-}
-
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
{
+ /* On VHE, PSTATE is saved in fixup_guest_exit_vhe() */
+ if (!has_vhe())
+ ctxt->regs.pstate = read_sysreg_el2(SYS_SPSR);
ctxt->regs.pc = read_sysreg_el2(SYS_ELR);
- ctxt->regs.pstate = from_hw_pstate(ctxt);
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index 07bf5c03631b..fb49757d3446 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -166,6 +166,12 @@ static void __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt)
write_sysreg(pmu->events_host, pmcntenset_el0);
}
+static bool fixup_guest_exit_nvhe(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+ fixup_guest_exit_prologue(vcpu, exit_code);
+ return fixup_guest_exit(vcpu, exit_code);
+}
+
/* Switch to the guest for legacy non-VHE systems */
int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
@@ -219,7 +225,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
exit_code = __guest_enter(vcpu);
/* And we're baaack! */
- } while (fixup_guest_exit(vcpu, &exit_code));
+ } while (fixup_guest_exit_nvhe(vcpu, &exit_code));
__sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index d9dc470c7790..4d80596e32a5 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -156,12 +156,60 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
+ bool hyp_ctxt)
+{
+ u64 pstate = read_sysreg_el2(SYS_SPSR);
+
+ /*
+ * Sync pstate back as early as possible, so that is_hyp_ctxt()
+ * reflects the exact context. It is otherwise possible to get
+ * confused with a VHE guest and ARMv8.4-NV, such as:
+ *
+ * (1) enter guest in host EL0
+ * (2) guest traps to guest vEL2 (no hypervisor intervention)
+ * (3) guest clears virtual HCR_EL2.TGE (no trap either)
+ * (4) host interrupt fires, exit
+ * (5) is_hyp_ctxt() now says "guest" (pstate.M==EL1 and TGE==0)
+ *
+ * If host preemption occurs, vcpu_load/put() will be very confused.
+ *
+ * Consider this as the prologue before the fixup prologue...
+ */
+
+ if (unlikely(hyp_ctxt)) {
+ u64 mode = pstate & PSR_MODE_MASK;
+
+ switch (mode) {
+ case PSR_MODE_EL1t:
+ mode = PSR_MODE_EL2t;
+ break;
+ case PSR_MODE_EL1h:
+ mode = PSR_MODE_EL2h;
+ break;
+ }
+
+ pstate = (pstate & ~PSR_MODE_MASK) | mode;
+ }
+
+ *vcpu_cpsr(vcpu) = pstate;
+
+ fixup_guest_exit_prologue(vcpu, exit_code);
+
+ if (*exit_code == ARM_EXCEPTION_TRAP) {
+ /* more to come here */
+ }
+
+ return fixup_guest_exit(vcpu, exit_code);
+}
+
/* Switch to the guest for VHE systems running in EL2 */
static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
u64 exit_code;
+ bool hyp_ctxt;
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
host_ctxt->__hyp_running_vcpu = vcpu;
@@ -188,12 +236,19 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
+ /*
+ * Being in HYP context or not is an invariant here. If we enter in
+ * a given context, we exit in the same context. We can thus only
+ * sample the context once.
+ */
+ WRITE_ONCE(hyp_ctxt, is_hyp_ctxt(vcpu));
+
do {
/* Jump in the fire! */
exit_code = __guest_enter(vcpu);
/* And we're baaack! */
- } while (fixup_guest_exit(vcpu, &exit_code));
+ } while (fixup_guest_exit_vhe(vcpu, &exit_code, READ_ONCE(hyp_ctxt)));
sysreg_save_guest_state_vhe(guest_ctxt);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 61/66] KVM: arm64: nv: Synchronize PSTATE early on exit
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
The NV code relies on predicates such as is_hyp_ctxt() being
reliable. In turn, is_hyp_ctxt() relies on things like PSTATE
and the virtual HCR_EL2 being accurate.
But with ARMv8.4-NV removing trapping for a large part of the
EL2 system registers (among which HCR_EL2), we can't use such
trapping to synchronize the rest of the state.
Let's look at the following sequence for a VHE guest:
(1) enter guest in host EL0
(2) guest traps to guest vEL2 (no hypervisor intervention)
(3) guest clears virtual HCR_EL2.TGE (no trap either)
(4) host interrupt fires, exit
(5) is_hyp_ctxt() now says "guest" (PSTATE.M==EL1 and TGE==0)
It is obvious that such behaviour would be rather unfortunate,
and lead to interesting, difficult to catch bugs specially if
preemption kicks in (yes, I wasted a whole week chasing this one).
In order to preserve the invariant that a guest entered in host
context must exit in the same context, we must make sure that
is_hyp_ctxt() works correctly. Since we can always observe the
guest value of HCR_EL2.{E2H,TGE} in the VNCR_EL2 page, we solely
need to synchronize PSTATE as early as possible.
This basically amounts to moving from_hw_pstate() as close
as possible to the guest exit point, and fixup_guest_exit()
seems as good a place as any.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 16 ++++--
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 26 ++--------
arch/arm64/kvm/hyp/nvhe/switch.c | 8 ++-
arch/arm64/kvm/hyp/vhe/switch.c | 57 +++++++++++++++++++++-
4 files changed, 78 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e5e201314c87..3b56841eb328 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -407,11 +407,11 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)
}
/*
- * Return true when we were able to fixup the guest exit and should return to
- * the guest, false when we should restore the host state and return to the
- * main run loop.
+ * Prologue for the guest fixup, populating ESR_EL2 and fixing up PC
+ * if required.
*/
-static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+static inline void fixup_guest_exit_prologue(struct kvm_vcpu *vcpu,
+ u64 *exit_code)
{
if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ)
vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR);
@@ -430,7 +430,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
if (esr_ec == ESR_ELx_EC_HVC32 || esr_ec == ESR_ELx_EC_HVC64)
write_sysreg_el2(read_sysreg_el2(SYS_ELR) - 4, SYS_ELR);
}
+}
+/*
+ * Return true when we were able to fixup the guest exit and should return to
+ * the guest, false when we should restore the host state and return to the
+ * main run loop.
+ */
+static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
/*
* We're using the raw exception code in order to only process
* the trap if no SError is pending. We will come back to the
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index 92715fa01e88..1931c8667d52 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -51,32 +51,12 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt_sys_reg(ctxt, SPSR_EL1) = read_sysreg_el1(SYS_SPSR);
}
-static inline u64 from_hw_pstate(const struct kvm_cpu_context *ctxt)
-{
- u64 reg = read_sysreg_el2(SYS_SPSR);
-
- if (__is_hyp_ctxt(ctxt)) {
- u64 mode = reg & (PSR_MODE_MASK | PSR_MODE32_BIT);
-
- switch (mode) {
- case PSR_MODE_EL1t:
- mode = PSR_MODE_EL2t;
- break;
- case PSR_MODE_EL1h:
- mode = PSR_MODE_EL2h;
- break;
- }
-
- return (reg & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
- }
-
- return reg;
-}
-
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
{
+ /* On VHE, PSTATE is saved in fixup_guest_exit_vhe() */
+ if (!has_vhe())
+ ctxt->regs.pstate = read_sysreg_el2(SYS_SPSR);
ctxt->regs.pc = read_sysreg_el2(SYS_ELR);
- ctxt->regs.pstate = from_hw_pstate(ctxt);
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index 07bf5c03631b..fb49757d3446 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -166,6 +166,12 @@ static void __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt)
write_sysreg(pmu->events_host, pmcntenset_el0);
}
+static bool fixup_guest_exit_nvhe(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+ fixup_guest_exit_prologue(vcpu, exit_code);
+ return fixup_guest_exit(vcpu, exit_code);
+}
+
/* Switch to the guest for legacy non-VHE systems */
int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
@@ -219,7 +225,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
exit_code = __guest_enter(vcpu);
/* And we're baaack! */
- } while (fixup_guest_exit(vcpu, &exit_code));
+ } while (fixup_guest_exit_nvhe(vcpu, &exit_code));
__sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index d9dc470c7790..4d80596e32a5 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -156,12 +156,60 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
+ bool hyp_ctxt)
+{
+ u64 pstate = read_sysreg_el2(SYS_SPSR);
+
+ /*
+ * Sync pstate back as early as possible, so that is_hyp_ctxt()
+ * reflects the exact context. It is otherwise possible to get
+ * confused with a VHE guest and ARMv8.4-NV, such as:
+ *
+ * (1) enter guest in host EL0
+ * (2) guest traps to guest vEL2 (no hypervisor intervention)
+ * (3) guest clears virtual HCR_EL2.TGE (no trap either)
+ * (4) host interrupt fires, exit
+ * (5) is_hyp_ctxt() now says "guest" (pstate.M==EL1 and TGE==0)
+ *
+ * If host preemption occurs, vcpu_load/put() will be very confused.
+ *
+ * Consider this as the prologue before the fixup prologue...
+ */
+
+ if (unlikely(hyp_ctxt)) {
+ u64 mode = pstate & PSR_MODE_MASK;
+
+ switch (mode) {
+ case PSR_MODE_EL1t:
+ mode = PSR_MODE_EL2t;
+ break;
+ case PSR_MODE_EL1h:
+ mode = PSR_MODE_EL2h;
+ break;
+ }
+
+ pstate = (pstate & ~PSR_MODE_MASK) | mode;
+ }
+
+ *vcpu_cpsr(vcpu) = pstate;
+
+ fixup_guest_exit_prologue(vcpu, exit_code);
+
+ if (*exit_code == ARM_EXCEPTION_TRAP) {
+ /* more to come here */
+ }
+
+ return fixup_guest_exit(vcpu, exit_code);
+}
+
/* Switch to the guest for VHE systems running in EL2 */
static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
u64 exit_code;
+ bool hyp_ctxt;
host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt;
host_ctxt->__hyp_running_vcpu = vcpu;
@@ -188,12 +236,19 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
+ /*
+ * Being in HYP context or not is an invariant here. If we enter in
+ * a given context, we exit in the same context. We can thus only
+ * sample the context once.
+ */
+ WRITE_ONCE(hyp_ctxt, is_hyp_ctxt(vcpu));
+
do {
/* Jump in the fire! */
exit_code = __guest_enter(vcpu);
/* And we're baaack! */
- } while (fixup_guest_exit(vcpu, &exit_code));
+ } while (fixup_guest_exit_vhe(vcpu, &exit_code, READ_ONCE(hyp_ctxt)));
sysreg_save_guest_state_vhe(guest_ctxt);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 62/66] KVM: arm64: nv: Sync nested timer state with ARMv8.4
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
From: Christoffer Dall <christoffer.dall@arm.com>
Emulating the ARMv8.4-NV timers is a bit odd, as the timers can
be reconfigured behind our back without the hypervisor even
noticing. In the VHE case, that's an actual regression in the
architecture...
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arch_timer.c | 37 ++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/arm.c | 3 +++
include/kvm/arm_arch_timer.h | 1 +
3 files changed, 41 insertions(+)
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 1c2be6391951..5af09d9adfe0 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -778,6 +778,43 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
set_cntvoff(0);
}
+void kvm_timer_sync_nested(struct kvm_vcpu *vcpu)
+{
+ if (!is_hyp_ctxt(vcpu))
+ return;
+
+ /*
+ * Guest hypervisors using ARMv8.4 enhanced nested virt support have
+ * their EL1 timer register accesses redirected to the VNCR page.
+ */
+ if (!vcpu_el2_e2h_is_set(vcpu)) {
+ /*
+ * For a non-VHE guest hypervisor, we update the hardware
+ * timer registers with the latest value written by the guest
+ * to the VNCR page and let the hardware take care of the
+ * rest.
+ */
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTV_CTL_EL0), SYS_CNTV_CTL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTV_CVAL_EL0), SYS_CNTV_CVAL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTP_CTL_EL0), SYS_CNTP_CTL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTP_CVAL_EL0), SYS_CNTP_CVAL);
+ } else {
+ /*
+ * For a VHE guest hypervisor, the emulated state (which
+ * is stored in the VNCR page) could have been updated behind
+ * our back, and we must reset the emulation of the timers.
+ */
+
+ struct timer_map map;
+ get_timer_map(vcpu, &map);
+
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
+ soft_timer_cancel(&map.emul_ptimer->hrtimer);
+ timer_emulate(map.emul_vtimer);
+ timer_emulate(map.emul_ptimer);
+ }
+}
+
/*
* With a userspace irqchip we have to check if the guest de-asserted the
* timer and if so, unmask the timer irq signal on the host interrupt
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 818ec91801e2..e8f009ed096b 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -849,6 +849,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
if (static_branch_unlikely(&userspace_irqchip_in_use))
kvm_timer_sync_user(vcpu);
+ if (enhanced_nested_virt_in_use(vcpu))
+ kvm_timer_sync_nested(vcpu);
+
kvm_arch_vcpu_ctxsync_fp(vcpu);
/*
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 063f613fbc7e..2e20916e9025 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -68,6 +68,7 @@ int kvm_timer_hyp_init(bool);
int kvm_timer_enable(struct kvm_vcpu *vcpu);
int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu);
void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
+void kvm_timer_sync_nested(struct kvm_vcpu *vcpu);
void kvm_timer_sync_user(struct kvm_vcpu *vcpu);
bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu);
void kvm_timer_update_run(struct kvm_vcpu *vcpu);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 62/66] KVM: arm64: nv: Sync nested timer state with ARMv8.4
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
From: Christoffer Dall <christoffer.dall@arm.com>
Emulating the ARMv8.4-NV timers is a bit odd, as the timers can
be reconfigured behind our back without the hypervisor even
noticing. In the VHE case, that's an actual regression in the
architecture...
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arch_timer.c | 37 ++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/arm.c | 3 +++
include/kvm/arm_arch_timer.h | 1 +
3 files changed, 41 insertions(+)
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 1c2be6391951..5af09d9adfe0 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -778,6 +778,43 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
set_cntvoff(0);
}
+void kvm_timer_sync_nested(struct kvm_vcpu *vcpu)
+{
+ if (!is_hyp_ctxt(vcpu))
+ return;
+
+ /*
+ * Guest hypervisors using ARMv8.4 enhanced nested virt support have
+ * their EL1 timer register accesses redirected to the VNCR page.
+ */
+ if (!vcpu_el2_e2h_is_set(vcpu)) {
+ /*
+ * For a non-VHE guest hypervisor, we update the hardware
+ * timer registers with the latest value written by the guest
+ * to the VNCR page and let the hardware take care of the
+ * rest.
+ */
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTV_CTL_EL0), SYS_CNTV_CTL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTV_CVAL_EL0), SYS_CNTV_CVAL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTP_CTL_EL0), SYS_CNTP_CTL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTP_CVAL_EL0), SYS_CNTP_CVAL);
+ } else {
+ /*
+ * For a VHE guest hypervisor, the emulated state (which
+ * is stored in the VNCR page) could have been updated behind
+ * our back, and we must reset the emulation of the timers.
+ */
+
+ struct timer_map map;
+ get_timer_map(vcpu, &map);
+
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
+ soft_timer_cancel(&map.emul_ptimer->hrtimer);
+ timer_emulate(map.emul_vtimer);
+ timer_emulate(map.emul_ptimer);
+ }
+}
+
/*
* With a userspace irqchip we have to check if the guest de-asserted the
* timer and if so, unmask the timer irq signal on the host interrupt
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 818ec91801e2..e8f009ed096b 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -849,6 +849,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
if (static_branch_unlikely(&userspace_irqchip_in_use))
kvm_timer_sync_user(vcpu);
+ if (enhanced_nested_virt_in_use(vcpu))
+ kvm_timer_sync_nested(vcpu);
+
kvm_arch_vcpu_ctxsync_fp(vcpu);
/*
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 063f613fbc7e..2e20916e9025 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -68,6 +68,7 @@ int kvm_timer_hyp_init(bool);
int kvm_timer_enable(struct kvm_vcpu *vcpu);
int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu);
void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
+void kvm_timer_sync_nested(struct kvm_vcpu *vcpu);
void kvm_timer_sync_user(struct kvm_vcpu *vcpu);
bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu);
void kvm_timer_update_run(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 62/66] KVM: arm64: nv: Sync nested timer state with ARMv8.4
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
From: Christoffer Dall <christoffer.dall@arm.com>
Emulating the ARMv8.4-NV timers is a bit odd, as the timers can
be reconfigured behind our back without the hypervisor even
noticing. In the VHE case, that's an actual regression in the
architecture...
Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arch_timer.c | 37 ++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/arm.c | 3 +++
include/kvm/arm_arch_timer.h | 1 +
3 files changed, 41 insertions(+)
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 1c2be6391951..5af09d9adfe0 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -778,6 +778,43 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
set_cntvoff(0);
}
+void kvm_timer_sync_nested(struct kvm_vcpu *vcpu)
+{
+ if (!is_hyp_ctxt(vcpu))
+ return;
+
+ /*
+ * Guest hypervisors using ARMv8.4 enhanced nested virt support have
+ * their EL1 timer register accesses redirected to the VNCR page.
+ */
+ if (!vcpu_el2_e2h_is_set(vcpu)) {
+ /*
+ * For a non-VHE guest hypervisor, we update the hardware
+ * timer registers with the latest value written by the guest
+ * to the VNCR page and let the hardware take care of the
+ * rest.
+ */
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTV_CTL_EL0), SYS_CNTV_CTL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTV_CVAL_EL0), SYS_CNTV_CVAL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTP_CTL_EL0), SYS_CNTP_CTL);
+ write_sysreg_el0(__vcpu_sys_reg(vcpu, CNTP_CVAL_EL0), SYS_CNTP_CVAL);
+ } else {
+ /*
+ * For a VHE guest hypervisor, the emulated state (which
+ * is stored in the VNCR page) could have been updated behind
+ * our back, and we must reset the emulation of the timers.
+ */
+
+ struct timer_map map;
+ get_timer_map(vcpu, &map);
+
+ soft_timer_cancel(&map.emul_vtimer->hrtimer);
+ soft_timer_cancel(&map.emul_ptimer->hrtimer);
+ timer_emulate(map.emul_vtimer);
+ timer_emulate(map.emul_ptimer);
+ }
+}
+
/*
* With a userspace irqchip we have to check if the guest de-asserted the
* timer and if so, unmask the timer irq signal on the host interrupt
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 818ec91801e2..e8f009ed096b 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -849,6 +849,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
if (static_branch_unlikely(&userspace_irqchip_in_use))
kvm_timer_sync_user(vcpu);
+ if (enhanced_nested_virt_in_use(vcpu))
+ kvm_timer_sync_nested(vcpu);
+
kvm_arch_vcpu_ctxsync_fp(vcpu);
/*
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 063f613fbc7e..2e20916e9025 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -68,6 +68,7 @@ int kvm_timer_hyp_init(bool);
int kvm_timer_enable(struct kvm_vcpu *vcpu);
int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu);
void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
+void kvm_timer_sync_nested(struct kvm_vcpu *vcpu);
void kvm_timer_sync_user(struct kvm_vcpu *vcpu);
bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu);
void kvm_timer_update_run(struct kvm_vcpu *vcpu);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 15:59 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
If running a NV guest on an ARMv8.4-NV capable system, let's
allocate an additional page that will be used by the hypervisor
to fulfill system register accesses.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 3 ++-
arch/arm64/kvm/nested.c | 8 ++++++++
arch/arm64/kvm/reset.c | 1 +
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 78630bd5124d..dada0678c28e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
*/
static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
{
- if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
+ if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
+ r >= __VNCR_START__ && ctxt->vncr_array))
return &ctxt->vncr_array[r - __VNCR_START__];
return (u64 *)&ctxt->sys_regs[r];
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index eef8f9873814..88147ec99755 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
return -EINVAL;
+ if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
+ vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+ if (!vcpu->arch.ctxt.vncr_array)
+ return -ENOMEM;
+ }
+
mutex_lock(&kvm->lock);
/*
@@ -64,6 +70,8 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
+ free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
+ vcpu->arch.ctxt.vncr_array = NULL;
} else {
kvm->arch.nested_mmus_size = num_mmus;
ret = 0;
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 2d2c780e6c69..d281eb39036f 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -150,6 +150,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu)
void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
{
kfree(vcpu->arch.sve_state);
+ free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
}
static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
If running a NV guest on an ARMv8.4-NV capable system, let's
allocate an additional page that will be used by the hypervisor
to fulfill system register accesses.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 3 ++-
arch/arm64/kvm/nested.c | 8 ++++++++
arch/arm64/kvm/reset.c | 1 +
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 78630bd5124d..dada0678c28e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
*/
static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
{
- if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
+ if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
+ r >= __VNCR_START__ && ctxt->vncr_array))
return &ctxt->vncr_array[r - __VNCR_START__];
return (u64 *)&ctxt->sys_regs[r];
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index eef8f9873814..88147ec99755 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
return -EINVAL;
+ if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
+ vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+ if (!vcpu->arch.ctxt.vncr_array)
+ return -ENOMEM;
+ }
+
mutex_lock(&kvm->lock);
/*
@@ -64,6 +70,8 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
+ free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
+ vcpu->arch.ctxt.vncr_array = NULL;
} else {
kvm->arch.nested_mmus_size = num_mmus;
ret = 0;
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 2d2c780e6c69..d281eb39036f 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -150,6 +150,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu)
void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
{
kfree(vcpu->arch.sve_state);
+ free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
}
static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
@ 2020-12-10 15:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 15:59 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
If running a NV guest on an ARMv8.4-NV capable system, let's
allocate an additional page that will be used by the hypervisor
to fulfill system register accesses.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 3 ++-
arch/arm64/kvm/nested.c | 8 ++++++++
arch/arm64/kvm/reset.c | 1 +
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 78630bd5124d..dada0678c28e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
*/
static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
{
- if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
+ if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
+ r >= __VNCR_START__ && ctxt->vncr_array))
return &ctxt->vncr_array[r - __VNCR_START__];
return (u64 *)&ctxt->sys_regs[r];
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index eef8f9873814..88147ec99755 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
return -EINVAL;
+ if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
+ vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+ if (!vcpu->arch.ctxt.vncr_array)
+ return -ENOMEM;
+ }
+
mutex_lock(&kvm->lock);
/*
@@ -64,6 +70,8 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
+ free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
+ vcpu->arch.ctxt.vncr_array = NULL;
} else {
kvm->arch.nested_mmus_size = num_mmus;
ret = 0;
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 2d2c780e6c69..d281eb39036f 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -150,6 +150,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu)
void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
{
kfree(vcpu->arch.sve_state);
+ free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
}
static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 64/66] KVM: arm64: nv: Enable ARMv8.4-NV support
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 16:00 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
As all the VNCR-capable system registers are nicely separated
from the rest of the crowd, let's set HCR_EL2.NV2 on and let
the ball rolling.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_emulate.h | 23 +++++++++++++----------
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kvm/hyp/vhe/switch.c | 14 +++++++++++++-
4 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 0d88a7c51dec..37cd86aac727 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV2 (UL(1) << 45)
#define HCR_AT (UL(1) << 44)
#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 44b395854430..3afe937b81f1 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -242,21 +242,24 @@ static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
static inline u64 __fixup_spsr_el2_write(struct kvm_cpu_context *ctxt, u64 val)
{
- if (!__vcpu_el2_e2h_is_set(ctxt)) {
- /*
- * Clear the .M field when writing SPSR to the CPU, so that we
- * can detect when the CPU clobbered our SPSR copy during a
- * local exception.
- */
- val &= ~0xc;
- }
+ struct kvm_vcpu *vcpu = container_of(ctxt, struct kvm_vcpu, arch.ctxt);
+
+ if (enhanced_nested_virt_in_use(vcpu) || __vcpu_el2_e2h_is_set(ctxt))
+ return val;
- return val;
+ /*
+ * Clear the .M field when writing SPSR to the CPU, so that we
+ * can detect when the CPU clobbered our SPSR copy during a
+ * local exception.
+ */
+ return val &= ~0xc;
}
static inline u64 __fixup_spsr_el2_read(const struct kvm_cpu_context *ctxt, u64 val)
{
- if (__vcpu_el2_e2h_is_set(ctxt))
+ struct kvm_vcpu *vcpu = container_of(ctxt, struct kvm_vcpu, arch.ctxt);
+
+ if (enhanced_nested_virt_in_use(vcpu) || __vcpu_el2_e2h_is_set(ctxt))
return val;
/*
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index d047954400d4..4cb55cf3c1d9 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -487,6 +487,7 @@
#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
+#define SYS_VNCR_EL2 sys_reg(3, 4, 2, 2, 0)
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 4d80596e32a5..6d57d09b8503 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -48,7 +48,13 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
* the EL1 virtual memory control register accesses
* as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_TTLB | HCR_NV1;
+ if (enhanced_nested_virt_in_use(vcpu)) {
+ hcr &= ~HCR_TVM;
+ } else {
+ hcr |= HCR_TVM | HCR_TRVM | HCR_TTLB;
+ }
+
+ hcr |= HCR_AT | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -80,6 +86,12 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_tge_is_set(vcpu))
hcr |= HCR_AT | HCR_TTLB;
}
+
+ if (enhanced_nested_virt_in_use(vcpu)) {
+ hcr |= HCR_AT | HCR_TTLB | HCR_NV2;
+ write_sysreg_s(vcpu->arch.ctxt.vncr_array,
+ SYS_VNCR_EL2);
+ }
} else if (nested_virt_in_use(vcpu)) {
hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
}
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 64/66] KVM: arm64: nv: Enable ARMv8.4-NV support
@ 2020-12-10 16:00 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
As all the VNCR-capable system registers are nicely separated
from the rest of the crowd, let's set HCR_EL2.NV2 on and let
the ball rolling.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_emulate.h | 23 +++++++++++++----------
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kvm/hyp/vhe/switch.c | 14 +++++++++++++-
4 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 0d88a7c51dec..37cd86aac727 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV2 (UL(1) << 45)
#define HCR_AT (UL(1) << 44)
#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 44b395854430..3afe937b81f1 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -242,21 +242,24 @@ static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
static inline u64 __fixup_spsr_el2_write(struct kvm_cpu_context *ctxt, u64 val)
{
- if (!__vcpu_el2_e2h_is_set(ctxt)) {
- /*
- * Clear the .M field when writing SPSR to the CPU, so that we
- * can detect when the CPU clobbered our SPSR copy during a
- * local exception.
- */
- val &= ~0xc;
- }
+ struct kvm_vcpu *vcpu = container_of(ctxt, struct kvm_vcpu, arch.ctxt);
+
+ if (enhanced_nested_virt_in_use(vcpu) || __vcpu_el2_e2h_is_set(ctxt))
+ return val;
- return val;
+ /*
+ * Clear the .M field when writing SPSR to the CPU, so that we
+ * can detect when the CPU clobbered our SPSR copy during a
+ * local exception.
+ */
+ return val &= ~0xc;
}
static inline u64 __fixup_spsr_el2_read(const struct kvm_cpu_context *ctxt, u64 val)
{
- if (__vcpu_el2_e2h_is_set(ctxt))
+ struct kvm_vcpu *vcpu = container_of(ctxt, struct kvm_vcpu, arch.ctxt);
+
+ if (enhanced_nested_virt_in_use(vcpu) || __vcpu_el2_e2h_is_set(ctxt))
return val;
/*
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index d047954400d4..4cb55cf3c1d9 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -487,6 +487,7 @@
#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
+#define SYS_VNCR_EL2 sys_reg(3, 4, 2, 2, 0)
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 4d80596e32a5..6d57d09b8503 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -48,7 +48,13 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
* the EL1 virtual memory control register accesses
* as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_TTLB | HCR_NV1;
+ if (enhanced_nested_virt_in_use(vcpu)) {
+ hcr &= ~HCR_TVM;
+ } else {
+ hcr |= HCR_TVM | HCR_TRVM | HCR_TTLB;
+ }
+
+ hcr |= HCR_AT | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -80,6 +86,12 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_tge_is_set(vcpu))
hcr |= HCR_AT | HCR_TTLB;
}
+
+ if (enhanced_nested_virt_in_use(vcpu)) {
+ hcr |= HCR_AT | HCR_TTLB | HCR_NV2;
+ write_sysreg_s(vcpu->arch.ctxt.vncr_array,
+ SYS_VNCR_EL2);
+ }
} else if (nested_virt_in_use(vcpu)) {
hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
}
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 64/66] KVM: arm64: nv: Enable ARMv8.4-NV support
@ 2020-12-10 16:00 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
As all the VNCR-capable system registers are nicely separated
from the rest of the crowd, let's set HCR_EL2.NV2 on and let
the ball rolling.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_emulate.h | 23 +++++++++++++----------
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kvm/hyp/vhe/switch.c | 14 +++++++++++++-
4 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 0d88a7c51dec..37cd86aac727 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -14,6 +14,7 @@
/* Hyp Configuration Register (HCR) bits */
#define HCR_ATA (UL(1) << 56)
#define HCR_FWB (UL(1) << 46)
+#define HCR_NV2 (UL(1) << 45)
#define HCR_AT (UL(1) << 44)
#define HCR_NV1 (UL(1) << 43)
#define HCR_NV (UL(1) << 42)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 44b395854430..3afe937b81f1 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -242,21 +242,24 @@ static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu)
static inline u64 __fixup_spsr_el2_write(struct kvm_cpu_context *ctxt, u64 val)
{
- if (!__vcpu_el2_e2h_is_set(ctxt)) {
- /*
- * Clear the .M field when writing SPSR to the CPU, so that we
- * can detect when the CPU clobbered our SPSR copy during a
- * local exception.
- */
- val &= ~0xc;
- }
+ struct kvm_vcpu *vcpu = container_of(ctxt, struct kvm_vcpu, arch.ctxt);
+
+ if (enhanced_nested_virt_in_use(vcpu) || __vcpu_el2_e2h_is_set(ctxt))
+ return val;
- return val;
+ /*
+ * Clear the .M field when writing SPSR to the CPU, so that we
+ * can detect when the CPU clobbered our SPSR copy during a
+ * local exception.
+ */
+ return val &= ~0xc;
}
static inline u64 __fixup_spsr_el2_read(const struct kvm_cpu_context *ctxt, u64 val)
{
- if (__vcpu_el2_e2h_is_set(ctxt))
+ struct kvm_vcpu *vcpu = container_of(ctxt, struct kvm_vcpu, arch.ctxt);
+
+ if (enhanced_nested_virt_in_use(vcpu) || __vcpu_el2_e2h_is_set(ctxt))
return val;
/*
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index d047954400d4..4cb55cf3c1d9 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -487,6 +487,7 @@
#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
+#define SYS_VNCR_EL2 sys_reg(3, 4, 2, 2, 0)
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 4d80596e32a5..6d57d09b8503 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -48,7 +48,13 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
* the EL1 virtual memory control register accesses
* as well as the AT S1 operations.
*/
- hcr |= HCR_TVM | HCR_TRVM | HCR_AT | HCR_TTLB | HCR_NV1;
+ if (enhanced_nested_virt_in_use(vcpu)) {
+ hcr &= ~HCR_TVM;
+ } else {
+ hcr |= HCR_TVM | HCR_TRVM | HCR_TTLB;
+ }
+
+ hcr |= HCR_AT | HCR_NV1;
} else {
/*
* For a guest hypervisor on v8.1 (VHE), allow to
@@ -80,6 +86,12 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
if (!vcpu_el2_tge_is_set(vcpu))
hcr |= HCR_AT | HCR_TTLB;
}
+
+ if (enhanced_nested_virt_in_use(vcpu)) {
+ hcr |= HCR_AT | HCR_TTLB | HCR_NV2;
+ write_sysreg_s(vcpu->arch.ctxt.vncr_array,
+ SYS_VNCR_EL2);
+ }
} else if (nested_virt_in_use(vcpu)) {
hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
}
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 65/66] KVM: arm64: nv: Fast-track 'InHost' exception returns
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 16:00 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
A significant part of the ARMv8.3-NV extension is to trap ERET
instructions so that the hypervisor gets a chance to switch
from a vEL2 L1 guest to an EL1 L2 guest.
But this also has the unfortunate consequence of trapping ERET
in unsuspecting circumstances, such as staying at vEL2 (interrupt
handling while being in the guest hypervisor), or returning to host
userspace in the case of a VHE guest.
Although we already make some effort to handle these ERET quicker
by not doing the put/load dance, it is still way too far down the
line for it to be efficient enough.
For these cases, it would ideal to ERET directly, no question asked.
Of course, we can't do that. But the next best thing is to do it as
early as possible, in fixup_guest_exit(), much as we would handle
FPSIMD exceptions.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/emulate-nested.c | 26 ++--------------
arch/arm64/kvm/hyp/vhe/switch.c | 53 ++++++++++++++++++++++++++++++++-
2 files changed, 54 insertions(+), 25 deletions(-)
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index df4661515183..43197ff4a806 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -52,8 +52,7 @@ bool forward_nv_traps(struct kvm_vcpu *vcpu)
void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
{
- u64 spsr, elr, mode;
- bool direct_eret;
+ u64 spsr, elr;
/*
* Forward this trap to the virtual EL2 if the virtual
@@ -62,31 +61,10 @@ void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
if (forward_nv_traps(vcpu))
return;
- /*
- * Going through the whole put/load motions is a waste of time
- * if this is a VHE guest hypervisor returning to its own
- * userspace, or the hypervisor performing a local exception
- * return. No need to save/restore registers, no need to
- * switch S2 MMU. Just do the canonical ERET.
- */
- spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
- mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
-
- direct_eret = (mode == PSR_MODE_EL0t &&
- vcpu_el2_e2h_is_set(vcpu) &&
- vcpu_el2_tge_is_set(vcpu));
- direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
-
- if (direct_eret) {
- *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
- *vcpu_cpsr(vcpu) = spsr;
- trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
- return;
- }
-
preempt_disable();
kvm_arch_vcpu_put(vcpu);
+ spsr = __vcpu_sys_reg(vcpu, SPSR_EL2);
elr = __vcpu_sys_reg(vcpu, ELR_EL2);
trace_kvm_nested_eret(vcpu, elr, spsr);
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 6d57d09b8503..c90aed418f73 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -168,6 +168,56 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool __hyp_handle_eret(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ u64 spsr, mode;
+
+ /*
+ * Going through the whole put/load motions is a waste of time
+ * if this is a VHE guest hypervisor returning to its own
+ * userspace, or the hypervisor performing a local exception
+ * return. No need to save/restore registers, no need to
+ * switch S2 MMU. Just do the canonical ERET. Unless the trap
+ * has to be forwarded further down the line, of course...
+ */
+ if (kvm_vcpu_trap_get_class(vcpu) != ESR_ELx_EC_ERET)
+ return false;
+
+ /*
+ * Let the trap forwarding be handled by the normal exception
+ * handling code.
+ */
+ if (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV)
+ return false;
+
+ spsr = read_sysreg_el1(SYS_SPSR);
+ spsr = __fixup_spsr_el2_read(ctxt, spsr);
+ mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL0t:
+ if (!(vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)))
+ return false;
+ break;
+ case PSR_MODE_EL2t:
+ mode = PSR_MODE_EL1t;
+ break;
+ case PSR_MODE_EL2h:
+ mode = PSR_MODE_EL1h;
+ break;
+ default:
+ return false;
+ }
+
+ spsr = (spsr & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+
+ write_sysreg_el2(spsr, SYS_SPSR);
+ write_sysreg_el2(read_sysreg_el1(SYS_ELR), SYS_ELR);
+
+ return true;
+}
+
static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
bool hyp_ctxt)
{
@@ -209,7 +259,8 @@ static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
fixup_guest_exit_prologue(vcpu, exit_code);
if (*exit_code == ARM_EXCEPTION_TRAP) {
- /* more to come here */
+ if (__hyp_handle_eret(vcpu))
+ return true;
}
return fixup_guest_exit(vcpu, exit_code);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 65/66] KVM: arm64: nv: Fast-track 'InHost' exception returns
@ 2020-12-10 16:00 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
A significant part of the ARMv8.3-NV extension is to trap ERET
instructions so that the hypervisor gets a chance to switch
from a vEL2 L1 guest to an EL1 L2 guest.
But this also has the unfortunate consequence of trapping ERET
in unsuspecting circumstances, such as staying at vEL2 (interrupt
handling while being in the guest hypervisor), or returning to host
userspace in the case of a VHE guest.
Although we already make some effort to handle these ERET quicker
by not doing the put/load dance, it is still way too far down the
line for it to be efficient enough.
For these cases, it would ideal to ERET directly, no question asked.
Of course, we can't do that. But the next best thing is to do it as
early as possible, in fixup_guest_exit(), much as we would handle
FPSIMD exceptions.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/emulate-nested.c | 26 ++--------------
arch/arm64/kvm/hyp/vhe/switch.c | 53 ++++++++++++++++++++++++++++++++-
2 files changed, 54 insertions(+), 25 deletions(-)
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index df4661515183..43197ff4a806 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -52,8 +52,7 @@ bool forward_nv_traps(struct kvm_vcpu *vcpu)
void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
{
- u64 spsr, elr, mode;
- bool direct_eret;
+ u64 spsr, elr;
/*
* Forward this trap to the virtual EL2 if the virtual
@@ -62,31 +61,10 @@ void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
if (forward_nv_traps(vcpu))
return;
- /*
- * Going through the whole put/load motions is a waste of time
- * if this is a VHE guest hypervisor returning to its own
- * userspace, or the hypervisor performing a local exception
- * return. No need to save/restore registers, no need to
- * switch S2 MMU. Just do the canonical ERET.
- */
- spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
- mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
-
- direct_eret = (mode == PSR_MODE_EL0t &&
- vcpu_el2_e2h_is_set(vcpu) &&
- vcpu_el2_tge_is_set(vcpu));
- direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
-
- if (direct_eret) {
- *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
- *vcpu_cpsr(vcpu) = spsr;
- trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
- return;
- }
-
preempt_disable();
kvm_arch_vcpu_put(vcpu);
+ spsr = __vcpu_sys_reg(vcpu, SPSR_EL2);
elr = __vcpu_sys_reg(vcpu, ELR_EL2);
trace_kvm_nested_eret(vcpu, elr, spsr);
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 6d57d09b8503..c90aed418f73 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -168,6 +168,56 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool __hyp_handle_eret(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ u64 spsr, mode;
+
+ /*
+ * Going through the whole put/load motions is a waste of time
+ * if this is a VHE guest hypervisor returning to its own
+ * userspace, or the hypervisor performing a local exception
+ * return. No need to save/restore registers, no need to
+ * switch S2 MMU. Just do the canonical ERET. Unless the trap
+ * has to be forwarded further down the line, of course...
+ */
+ if (kvm_vcpu_trap_get_class(vcpu) != ESR_ELx_EC_ERET)
+ return false;
+
+ /*
+ * Let the trap forwarding be handled by the normal exception
+ * handling code.
+ */
+ if (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV)
+ return false;
+
+ spsr = read_sysreg_el1(SYS_SPSR);
+ spsr = __fixup_spsr_el2_read(ctxt, spsr);
+ mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL0t:
+ if (!(vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)))
+ return false;
+ break;
+ case PSR_MODE_EL2t:
+ mode = PSR_MODE_EL1t;
+ break;
+ case PSR_MODE_EL2h:
+ mode = PSR_MODE_EL1h;
+ break;
+ default:
+ return false;
+ }
+
+ spsr = (spsr & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+
+ write_sysreg_el2(spsr, SYS_SPSR);
+ write_sysreg_el2(read_sysreg_el1(SYS_ELR), SYS_ELR);
+
+ return true;
+}
+
static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
bool hyp_ctxt)
{
@@ -209,7 +259,8 @@ static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
fixup_guest_exit_prologue(vcpu, exit_code);
if (*exit_code == ARM_EXCEPTION_TRAP) {
- /* more to come here */
+ if (__hyp_handle_eret(vcpu))
+ return true;
}
return fixup_guest_exit(vcpu, exit_code);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 65/66] KVM: arm64: nv: Fast-track 'InHost' exception returns
@ 2020-12-10 16:00 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
A significant part of the ARMv8.3-NV extension is to trap ERET
instructions so that the hypervisor gets a chance to switch
from a vEL2 L1 guest to an EL1 L2 guest.
But this also has the unfortunate consequence of trapping ERET
in unsuspecting circumstances, such as staying at vEL2 (interrupt
handling while being in the guest hypervisor), or returning to host
userspace in the case of a VHE guest.
Although we already make some effort to handle these ERET quicker
by not doing the put/load dance, it is still way too far down the
line for it to be efficient enough.
For these cases, it would ideal to ERET directly, no question asked.
Of course, we can't do that. But the next best thing is to do it as
early as possible, in fixup_guest_exit(), much as we would handle
FPSIMD exceptions.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/emulate-nested.c | 26 ++--------------
arch/arm64/kvm/hyp/vhe/switch.c | 53 ++++++++++++++++++++++++++++++++-
2 files changed, 54 insertions(+), 25 deletions(-)
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index df4661515183..43197ff4a806 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -52,8 +52,7 @@ bool forward_nv_traps(struct kvm_vcpu *vcpu)
void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
{
- u64 spsr, elr, mode;
- bool direct_eret;
+ u64 spsr, elr;
/*
* Forward this trap to the virtual EL2 if the virtual
@@ -62,31 +61,10 @@ void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
if (forward_nv_traps(vcpu))
return;
- /*
- * Going through the whole put/load motions is a waste of time
- * if this is a VHE guest hypervisor returning to its own
- * userspace, or the hypervisor performing a local exception
- * return. No need to save/restore registers, no need to
- * switch S2 MMU. Just do the canonical ERET.
- */
- spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
- mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
-
- direct_eret = (mode == PSR_MODE_EL0t &&
- vcpu_el2_e2h_is_set(vcpu) &&
- vcpu_el2_tge_is_set(vcpu));
- direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
-
- if (direct_eret) {
- *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
- *vcpu_cpsr(vcpu) = spsr;
- trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
- return;
- }
-
preempt_disable();
kvm_arch_vcpu_put(vcpu);
+ spsr = __vcpu_sys_reg(vcpu, SPSR_EL2);
elr = __vcpu_sys_reg(vcpu, ELR_EL2);
trace_kvm_nested_eret(vcpu, elr, spsr);
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 6d57d09b8503..c90aed418f73 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -168,6 +168,56 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool __hyp_handle_eret(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+ u64 spsr, mode;
+
+ /*
+ * Going through the whole put/load motions is a waste of time
+ * if this is a VHE guest hypervisor returning to its own
+ * userspace, or the hypervisor performing a local exception
+ * return. No need to save/restore registers, no need to
+ * switch S2 MMU. Just do the canonical ERET. Unless the trap
+ * has to be forwarded further down the line, of course...
+ */
+ if (kvm_vcpu_trap_get_class(vcpu) != ESR_ELx_EC_ERET)
+ return false;
+
+ /*
+ * Let the trap forwarding be handled by the normal exception
+ * handling code.
+ */
+ if (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV)
+ return false;
+
+ spsr = read_sysreg_el1(SYS_SPSR);
+ spsr = __fixup_spsr_el2_read(ctxt, spsr);
+ mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
+
+ switch (mode) {
+ case PSR_MODE_EL0t:
+ if (!(vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)))
+ return false;
+ break;
+ case PSR_MODE_EL2t:
+ mode = PSR_MODE_EL1t;
+ break;
+ case PSR_MODE_EL2h:
+ mode = PSR_MODE_EL1h;
+ break;
+ default:
+ return false;
+ }
+
+ spsr = (spsr & ~(PSR_MODE_MASK | PSR_MODE32_BIT)) | mode;
+
+ write_sysreg_el2(spsr, SYS_SPSR);
+ write_sysreg_el2(read_sysreg_el1(SYS_ELR), SYS_ELR);
+
+ return true;
+}
+
static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
bool hyp_ctxt)
{
@@ -209,7 +259,8 @@ static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
fixup_guest_exit_prologue(vcpu, exit_code);
if (*exit_code == ARM_EXCEPTION_TRAP) {
- /* more to come here */
+ if (__hyp_handle_eret(vcpu))
+ return true;
}
return fixup_guest_exit(vcpu, exit_code);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 66/66] KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2020-12-10 16:00 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Andre Przywara, Christoffer Dall, Jintack Lim, Alexandru Elisei,
James Morse, Julien Thierry, Suzuki K Poulose, kernel-team
Due to the way ARMv8.4-NV suppresses traps when accessing EL2
system registers, we can't track when the guest changes its
HCR_EL2.TGE setting. This means we always trap EL1 TLBIs,
even if they don't affect any guest.
This obviously has a huge impact on performance, as we handle
TLBI traps as a normal exit, and a normal VHE host issues
thousands of TLBIs when booting (and quite a few when running
userspace).
A cheap way to reduce the overhead is to handle the limited
case of {E2H,TGE}=={1,1} as a guest fixup, as we already have
the right mmu configuration in place. Just execute the decoded
instruction right away and return to the guest.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 36 +++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/vhe/tlb.c | 6 ++++--
arch/arm64/kvm/sys_regs.c | 25 ++++++++---------------
3 files changed, 48 insertions(+), 19 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index c90aed418f73..0b9bc36340a5 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -168,6 +168,39 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool __hyp_handle_tlbi_el1(struct kvm_vcpu *vcpu)
+{
+ u32 instr;
+ u64 val;
+
+ /*
+ * Ideally, we would never trap on EL1 TLB invalidations when the
+ * guest's HCR_EL2.{E2H,TGE} == {1,1}. But "thanks" to ARMv8.4, we
+ * don't trap writes to HCR_EL2, meaning that we can't track
+ * changes to the virtual TGE bit. So we leave HCR_EL2.TTLB set on
+ * the host. Oopsie...
+ *
+ * In order to speed-up EL1 TLBIs from the vEL2 guest when TGE is
+ * set, try and handle these invalidation as quickly as possible,
+ * without fully exiting (unless this needs forwarding).
+ */
+ if (kvm_vcpu_trap_get_class(vcpu) != ESR_ELx_EC_SYS64 ||
+ !vcpu_mode_el2(vcpu) ||
+ (__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE))
+ return false;
+
+ instr = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+ if (sys_reg_Op0(instr) != TLBI_Op0 ||
+ sys_reg_Op1(instr) != TLBI_Op1_EL1)
+ return false;
+
+ val = vcpu_get_reg(vcpu, kvm_vcpu_sys_get_rt(vcpu));
+ __kvm_tlb_el1_instr(NULL, val, instr);
+ __kvm_skip_instr(vcpu);
+
+ return true;
+}
+
static bool __hyp_handle_eret(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
@@ -261,6 +294,9 @@ static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
if (*exit_code == ARM_EXCEPTION_TRAP) {
if (__hyp_handle_eret(vcpu))
return true;
+
+ if (__hyp_handle_tlbi_el1(vcpu))
+ return true;
}
return fixup_guest_exit(vcpu, exit_code);
diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c
index 52fda1d61308..6dd7f224e0f3 100644
--- a/arch/arm64/kvm/hyp/vhe/tlb.c
+++ b/arch/arm64/kvm/hyp/vhe/tlb.c
@@ -200,7 +200,8 @@ void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
dsb(ishst);
/* Switch to requested VMID */
- __tlb_switch_to_guest(mmu, &cxt);
+ if (mmu)
+ __tlb_switch_to_guest(mmu, &cxt);
/*
* Execute the same instruction as the guest hypervisor did,
@@ -239,5 +240,6 @@ void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
dsb(ish);
isb();
- __tlb_switch_to_host(&cxt);
+ if (mmu)
+ __tlb_switch_to_host(&cxt);
}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c7b004982ea9..cf03281f06a3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2684,6 +2684,8 @@ static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
u32 sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+ u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
/*
* If we're here, this is because we've trapped on a EL1 TLBI
@@ -2702,24 +2704,13 @@ static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
mutex_lock(&vcpu->kvm->lock);
- if ((__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
- u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
- struct kvm_s2_mmu *mmu;
-
- mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
- if (mmu)
- __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
- mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
- if (mmu)
- __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
- } else {
- /*
- * ARMv8.4-NV allows the guest to change TGE behind
- * our back, so we always trap EL1 TLBIs from vEL2...
- */
- __kvm_tlb_el1_instr(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
- }
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
mutex_unlock(&vcpu->kvm->lock);
--
2.29.2
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 66/66] KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
@ 2020-12-10 16:00 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team, Andre Przywara
Due to the way ARMv8.4-NV suppresses traps when accessing EL2
system registers, we can't track when the guest changes its
HCR_EL2.TGE setting. This means we always trap EL1 TLBIs,
even if they don't affect any guest.
This obviously has a huge impact on performance, as we handle
TLBI traps as a normal exit, and a normal VHE host issues
thousands of TLBIs when booting (and quite a few when running
userspace).
A cheap way to reduce the overhead is to handle the limited
case of {E2H,TGE}=={1,1} as a guest fixup, as we already have
the right mmu configuration in place. Just execute the decoded
instruction right away and return to the guest.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 36 +++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/vhe/tlb.c | 6 ++++--
arch/arm64/kvm/sys_regs.c | 25 ++++++++---------------
3 files changed, 48 insertions(+), 19 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index c90aed418f73..0b9bc36340a5 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -168,6 +168,39 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool __hyp_handle_tlbi_el1(struct kvm_vcpu *vcpu)
+{
+ u32 instr;
+ u64 val;
+
+ /*
+ * Ideally, we would never trap on EL1 TLB invalidations when the
+ * guest's HCR_EL2.{E2H,TGE} == {1,1}. But "thanks" to ARMv8.4, we
+ * don't trap writes to HCR_EL2, meaning that we can't track
+ * changes to the virtual TGE bit. So we leave HCR_EL2.TTLB set on
+ * the host. Oopsie...
+ *
+ * In order to speed-up EL1 TLBIs from the vEL2 guest when TGE is
+ * set, try and handle these invalidation as quickly as possible,
+ * without fully exiting (unless this needs forwarding).
+ */
+ if (kvm_vcpu_trap_get_class(vcpu) != ESR_ELx_EC_SYS64 ||
+ !vcpu_mode_el2(vcpu) ||
+ (__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE))
+ return false;
+
+ instr = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+ if (sys_reg_Op0(instr) != TLBI_Op0 ||
+ sys_reg_Op1(instr) != TLBI_Op1_EL1)
+ return false;
+
+ val = vcpu_get_reg(vcpu, kvm_vcpu_sys_get_rt(vcpu));
+ __kvm_tlb_el1_instr(NULL, val, instr);
+ __kvm_skip_instr(vcpu);
+
+ return true;
+}
+
static bool __hyp_handle_eret(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
@@ -261,6 +294,9 @@ static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
if (*exit_code == ARM_EXCEPTION_TRAP) {
if (__hyp_handle_eret(vcpu))
return true;
+
+ if (__hyp_handle_tlbi_el1(vcpu))
+ return true;
}
return fixup_guest_exit(vcpu, exit_code);
diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c
index 52fda1d61308..6dd7f224e0f3 100644
--- a/arch/arm64/kvm/hyp/vhe/tlb.c
+++ b/arch/arm64/kvm/hyp/vhe/tlb.c
@@ -200,7 +200,8 @@ void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
dsb(ishst);
/* Switch to requested VMID */
- __tlb_switch_to_guest(mmu, &cxt);
+ if (mmu)
+ __tlb_switch_to_guest(mmu, &cxt);
/*
* Execute the same instruction as the guest hypervisor did,
@@ -239,5 +240,6 @@ void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
dsb(ish);
isb();
- __tlb_switch_to_host(&cxt);
+ if (mmu)
+ __tlb_switch_to_host(&cxt);
}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c7b004982ea9..cf03281f06a3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2684,6 +2684,8 @@ static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
u32 sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+ u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
/*
* If we're here, this is because we've trapped on a EL1 TLBI
@@ -2702,24 +2704,13 @@ static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
mutex_lock(&vcpu->kvm->lock);
- if ((__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
- u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
- struct kvm_s2_mmu *mmu;
-
- mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
- if (mmu)
- __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
- mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
- if (mmu)
- __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
- } else {
- /*
- * ARMv8.4-NV allows the guest to change TGE behind
- * our back, so we always trap EL1 TLBIs from vEL2...
- */
- __kvm_tlb_el1_instr(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
- }
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
mutex_unlock(&vcpu->kvm->lock);
--
2.29.2
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* [PATCH v3 66/66] KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
@ 2020-12-10 16:00 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2020-12-10 16:00 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: kernel-team, Suzuki K Poulose, Andre Przywara, Christoffer Dall,
James Morse, Alexandru Elisei, Jintack Lim, Julien Thierry
Due to the way ARMv8.4-NV suppresses traps when accessing EL2
system registers, we can't track when the guest changes its
HCR_EL2.TGE setting. This means we always trap EL1 TLBIs,
even if they don't affect any guest.
This obviously has a huge impact on performance, as we handle
TLBI traps as a normal exit, and a normal VHE host issues
thousands of TLBIs when booting (and quite a few when running
userspace).
A cheap way to reduce the overhead is to handle the limited
case of {E2H,TGE}=={1,1} as a guest fixup, as we already have
the right mmu configuration in place. Just execute the decoded
instruction right away and return to the guest.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 36 +++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/vhe/tlb.c | 6 ++++--
arch/arm64/kvm/sys_regs.c | 25 ++++++++---------------
3 files changed, 48 insertions(+), 19 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index c90aed418f73..0b9bc36340a5 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -168,6 +168,39 @@ void deactivate_traps_vhe_put(void)
__deactivate_traps_common();
}
+static bool __hyp_handle_tlbi_el1(struct kvm_vcpu *vcpu)
+{
+ u32 instr;
+ u64 val;
+
+ /*
+ * Ideally, we would never trap on EL1 TLB invalidations when the
+ * guest's HCR_EL2.{E2H,TGE} == {1,1}. But "thanks" to ARMv8.4, we
+ * don't trap writes to HCR_EL2, meaning that we can't track
+ * changes to the virtual TGE bit. So we leave HCR_EL2.TTLB set on
+ * the host. Oopsie...
+ *
+ * In order to speed-up EL1 TLBIs from the vEL2 guest when TGE is
+ * set, try and handle these invalidation as quickly as possible,
+ * without fully exiting (unless this needs forwarding).
+ */
+ if (kvm_vcpu_trap_get_class(vcpu) != ESR_ELx_EC_SYS64 ||
+ !vcpu_mode_el2(vcpu) ||
+ (__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE))
+ return false;
+
+ instr = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+ if (sys_reg_Op0(instr) != TLBI_Op0 ||
+ sys_reg_Op1(instr) != TLBI_Op1_EL1)
+ return false;
+
+ val = vcpu_get_reg(vcpu, kvm_vcpu_sys_get_rt(vcpu));
+ __kvm_tlb_el1_instr(NULL, val, instr);
+ __kvm_skip_instr(vcpu);
+
+ return true;
+}
+
static bool __hyp_handle_eret(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
@@ -261,6 +294,9 @@ static bool fixup_guest_exit_vhe(struct kvm_vcpu *vcpu, u64 *exit_code,
if (*exit_code == ARM_EXCEPTION_TRAP) {
if (__hyp_handle_eret(vcpu))
return true;
+
+ if (__hyp_handle_tlbi_el1(vcpu))
+ return true;
}
return fixup_guest_exit(vcpu, exit_code);
diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c
index 52fda1d61308..6dd7f224e0f3 100644
--- a/arch/arm64/kvm/hyp/vhe/tlb.c
+++ b/arch/arm64/kvm/hyp/vhe/tlb.c
@@ -200,7 +200,8 @@ void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
dsb(ishst);
/* Switch to requested VMID */
- __tlb_switch_to_guest(mmu, &cxt);
+ if (mmu)
+ __tlb_switch_to_guest(mmu, &cxt);
/*
* Execute the same instruction as the guest hypervisor did,
@@ -239,5 +240,6 @@ void __kvm_tlb_el1_instr(struct kvm_s2_mmu *mmu, u64 val, u64 sys_encoding)
dsb(ish);
isb();
- __tlb_switch_to_host(&cxt);
+ if (mmu)
+ __tlb_switch_to_host(&cxt);
}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c7b004982ea9..cf03281f06a3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2684,6 +2684,8 @@ static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
u32 sys_encoding = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2);
+ u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
+ struct kvm_s2_mmu *mmu;
/*
* If we're here, this is because we've trapped on a EL1 TLBI
@@ -2702,24 +2704,13 @@ static bool handle_tlbi_el1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
mutex_lock(&vcpu->kvm->lock);
- if ((__vcpu_sys_reg(vcpu, HCR_EL2) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
- u64 virtual_vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
- struct kvm_s2_mmu *mmu;
-
- mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
- if (mmu)
- __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, HCR_VM);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
- mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
- if (mmu)
- __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
- } else {
- /*
- * ARMv8.4-NV allows the guest to change TGE behind
- * our back, so we always trap EL1 TLBIs from vEL2...
- */
- __kvm_tlb_el1_instr(&vcpu->kvm->arch.mmu, p->regval, sys_encoding);
- }
+ mmu = lookup_s2_mmu(vcpu->kvm, virtual_vttbr, 0);
+ if (mmu)
+ __kvm_tlb_el1_instr(mmu, p->regval, sys_encoding);
mutex_unlock(&vcpu->kvm->lock);
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 43/66] KVM: arm64: nv: arch_timer: Support hyp timer emulation
2020-12-10 15:59 ` Marc Zyngier
(?)
(?)
@ 2020-12-11 0:26 ` kernel test robot
-1 siblings, 0 replies; 249+ messages in thread
From: kernel test robot @ 2020-12-11 0:26 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]
Hi Marc,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvmarm/next]
[also build test WARNING on next-20201210]
[cannot apply to arm64/for-next/core soc/for-next arm/for-next xlnx/master linus/master kvm/linux-next v5.10-rc7]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Marc-Zyngier/KVM-arm64-ARMv8-3-8-4-Nested-Virtualization-support/20201211-004741
base: https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm.git next
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/af7a3cada1fa577612a86dd0a7c4016ffed68acc
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Marc-Zyngier/KVM-arm64-ARMv8-3-8-4-Nested-Virtualization-support/20201211-004741
git checkout af7a3cada1fa577612a86dd0a7c4016ffed68acc
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
arch/arm64/kvm/arch_timer.c: In function 'kvm_timer_vcpu_load_nested_switch':
>> arch/arm64/kvm/arch_timer.c:656:10: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
656 | int hw, ret;
| ^~~
vim +/ret +656 arch/arm64/kvm/arch_timer.c
652
653 static void kvm_timer_vcpu_load_nested_switch(struct kvm_vcpu *vcpu,
654 struct timer_map *map)
655 {
> 656 int hw, ret;
657
658 if (!irqchip_in_kernel(vcpu->kvm))
659 return;
660
661 /*
662 * We only ever unmap the vtimer irq on a VHE system that runs nested
663 * virtualization, in which case we have both a valid emul_vtimer,
664 * emul_ptimer, direct_vtimer, and direct_ptimer.
665 *
666 * Since this is called from kvm_timer_vcpu_load(), a change between
667 * vEL2 and vEL1/0 will have just happened, and the timer_map will
668 * represent this, and therefore we switch the emul/direct mappings
669 * below.
670 */
671 hw = kvm_vgic_get_map(vcpu, map->direct_vtimer->irq.irq);
672 if (hw < 0) {
673 kvm_vgic_unmap_phys_irq(vcpu, map->emul_vtimer->irq.irq);
674 kvm_vgic_unmap_phys_irq(vcpu, map->emul_ptimer->irq.irq);
675
676 ret = kvm_vgic_map_phys_irq(vcpu,
677 map->direct_vtimer->host_timer_irq,
678 map->direct_vtimer->irq.irq,
679 kvm_arch_timer_get_input_level);
680 ret = kvm_vgic_map_phys_irq(vcpu,
681 map->direct_ptimer->host_timer_irq,
682 map->direct_ptimer->irq.irq,
683 kvm_arch_timer_get_input_level);
684 }
685 }
686
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 52799 bytes --]
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2020-12-10 15:58 ` Marc Zyngier
` (67 preceding siblings ...)
(?)
@ 2021-01-11 7:20 ` Haibo Xu
2021-01-11 8:59 ` Marc Zyngier
-1 siblings, 1 reply; 249+ messages in thread
From: Haibo Xu @ 2021-01-11 7:20 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
[-- Attachment #1: Type: text/plain, Size: 14310 bytes --]
On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>
> This is a rework of the NV series that I posted 10 months ago[1], as a
> lot of the KVM code has changed since, and the series apply anymore
> (not that anybody really cares as the the HW is, as usual, made of
> unobtainium...).
>
> From the previous version:
>
> - Integration with the new page-table code
> - New exception injection code
> - No more messing with the nVHE code
> - No AArch32!!!!
> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>
> From a functionality perspective, you can expect a L2 guest to work,
> but don't even think of L3, as we only partially emulate the
> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> as well as anything that would require a Stage-1 PTW. What we want to
> achieve is that with NV disabled, there is no performance overhead and
> no regression.
>
> The series is roughly divided in 5 parts: exception handling, memory
> virtualization, interrupts and timers for ARMv8.3, followed by the
> ARMv8.4 support. There are of course some dependencies, but you'll
> hopefully get the gist of it.
>
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2. You can run the whole stack in the Foundation
> model. Don't be in a hurry ;-).
>
Hi Marc,
I got a kernel BUG message when booting the L2 guest kernel with the
kvmtool on a FVP setup.
Could you help have a look about the BUG message as well as my
environment configuration?
I think It probably caused by some local configurations of the FVP setup.
Thanks,
Haibo
------------------------------------------------------------------------------------------------------------------------------
1 # lkvm run -k ./Image -m 320 -c 2 --name guest-229
2 [ 77.714334] kernel BUG at arch/arm64/kernel/traps.c:407!
3 [ 77.715758] Internal error: Oops - BUG: 0 [#1] SMP
4 [ 77.716900] Modules linked in:
5 [ 77.717754] CPU: 0 PID: 229 Comm: lkvm Not tainted 5.11.0-rc1+ #2
6 [ 77.719193] Hardware name: linux,dummy-virt (DT)
7 [ 77.720300] pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--)
8 [ 77.722006] pc : do_undefinstr+0x4d0/0x5b0
9 [ 77.722900] lr : do_undefinstr+0x380/0x5b0
10 [ 77.723800] sp : ffff8000122ab8c0
11 [ 77.724674] x29: ffff8000122ab8c0 x28: ffff0000048c49c0
12 [ 77.725824] x27: 0000000000000000 x26: 0000000000000000
13 [ 77.727050] x25: 0000000000000000 x24: ffff000004b9d520
14 [ 77.728217] x23: 0000000080400009 x22: ffff8000100e42d0
15 [ 77.729700] x21: ffff8000122abaa0 x20: ffff0000048c49c0
16 [ 77.730856] x19: ffff8000122ab950 x18: 0000000000000000
17 [ 77.732047] x17: 0000000000000000 x16: 0000000000000000
18 [ 77.733276] x15: 0000000000000000 x14: 0000000000000000
19 [ 77.734423] x13: 0000000000000000 x12: 0000000000000000
20 [ 77.735629] x11: 0000000000000000 x10: 0000000000000000
21 [ 77.737200] x9 : ffff8000102fb0a0 x8 : ffff000004b9d730
22 [ 77.738369] x7 : 0000000000000000 x6 : 0000000080000000
23 [ 77.739533] x5 : 0000000000000000 x4 : 000000000000001f
24 [ 77.740700] x3 : 0000000000000000 x2 : ffff800011a05b80
25 [ 77.741963] x1 : ffff800011c09110 x0 : 0000000080400009
26 [ 77.743142] Call trace:
27 [ 77.743700] do_undefinstr+0x4d0/0x5b0
28 [ 77.744662] el1_undef+0x2c/0x48
29 [ 77.745832] el1_sync_handler+0x80/0xb0
30 [ 77.746736] el1_sync+0x74/0x100
31 [ 77.747612] reset_pmcr+0x8/0x88
32 [ 77.748471] kvm_reset_vcpu+0x128/0x290
33 [ 77.749425] kvm_arch_vcpu_ioctl+0x698/0x6c8
34 [ 77.750561] kvm_vcpu_ioctl+0x3c8/0x5f0
35 [ 77.751472] __arm64_sys_ioctl+0xa8/0xe8
36 [ 77.752375] el0_svc_common.constprop.0+0x78/0x188
37 [ 77.753920] do_el0_svc+0x28/0x88
38 [ 77.754821] el0_svc+0x1c/0x28
39 [ 77.755723] el0_sync_handler+0xa8/0xb0
40 [ 77.756678] el0_sync+0x160/0x180
41 [ 77.757582] Code: d2801400 17ffffa9 a9025bf5 f9001bf7 (d4210000)
42 [ 77.758970] ---[ end trace a8cdeac7ff43b5a5 ]---
43 [ 77.763012] ------------[ cut here ]------------
44 [ 77.769730] WARNING: CPU: 0 PID: 0 at kernel/rcu/tree.c:632
rcu_eqs_enter.isra.0+0x68/0x70
45 [ 77.771552] Modules linked in:
46 [ 77.772420] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G D
5.11.0-rc1+ #2
47 [ 77.774155] Hardware name: linux,dummy-virt (DT)
48 [ 77.775301] pstate: 204003c9 (nzCv DAIF +PAN -UAO -TCO BTYPE=--)
49 [ 77.776809] pc : rcu_eqs_enter.isra.0+0x68/0x70
50 [ 77.777999] lr : rcu_idle_enter+0x14/0x20
51 [ 77.778905] sp : ffff8000119f3e80
52 [ 77.779759] x29: ffff8000119f3e80 x28: ffff00001fefca00
53 [ 77.785541] x27: ffff800011a02ec0 x26: 0000000000000000
54 [ 77.786700] x25: 0000000000000000 x24: ffff8000119f9528
55 [ 77.787915] x23: ffff800011a02ec0 x22: ffff8000115837b8
56 [ 77.789166] x21: ffff8000119f9500 x20: 0000000000000000
57 [ 77.790345] x19: ffff80001156e000 x18: 0000000000000010
58 [ 77.791560] x17: 0000000000000000 x16: 0000000000000000
59 [ 77.792775] x15: 0000000000000000 x14: 0000000000000000
60 [ 77.793978] x13: 0000000000000002 x12: 0000000000000000
61 [ 77.795153] x11: 0000000000000001 x10: 0000000000000ae0
62 [ 77.805799] x9 : ffff800010db2d94 x8 : ffff800011a03a00
63 [ 77.813399] x7 : 0000000000000000 x6 : 000000166ce4e201
64 [ 77.821100] x5 : 00ffffffffffffff x4 : ffff80000e93e000
65 [ 77.828958] x3 : 4000000000000002 x2 : 4000000000000000
66 [ 77.836659] x1 : ffff800011585ac0 x0 : ffff00001fec3ac0
67 [ 77.844360] Call trace:
68 [ 77.847910] rcu_eqs_enter.isra.0+0x68/0x70
69 [ 77.851519] rcu_idle_enter+0x14/0x20
70 [ 77.854989] default_idle_call+0x3c/0x16c
71 [ 77.858460] do_idle+0x214/0x260
72 [ 77.862200] cpu_startup_entry+0x2c/0x90
73 [ 77.866046] rest_init+0xc4/0xd0
74 [ 77.869654] arch_call_rest_init+0x14/0x1c
75 [ 77.873334] start_kernel+0x80c/0x844
76 [ 77.877065] ---[ end trace a8cdeac7ff43b5a6 ]---
---------------------------------------------------------------------------------------------------------------
> And to be clear: although Jintack and Christoffer have written tons of
> the stuff originaly, I'm the one responsible for breaking it!
>
> [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
>
> Andre Przywara (1):
> KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
>
> Christoffer Dall (15):
> KVM: arm64: nv: Introduce nested virtualization VCPU feature
> KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> KVM: arm64: nv: Handle trapped ERET from virtual EL2
> KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> changes
> KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> KVM: arm64: nv: arch_timer: Support hyp timer emulation
> KVM: arm64: nv: vgic: Emulate the HW bit in software
> KVM: arm64: nv: Add nested GICv3 tracepoints
> KVM: arm64: nv: Sync nested timer state with ARMv8.4
>
> Jintack Lim (19):
> arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> KVM: arm64: nv: Support virtual EL2 exceptions
> KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> KVM: arm64: nv: Handle PSCI call via smc from the guest
> KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> KVM: arm64: nv: Set a handler for the system instruction traps
> KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> KVM: arm64: nv: Nested GICv3 Support
>
> Marc Zyngier (31):
> KVM: arm64: nv: Add EL2 system registers to vcpu context
> KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> KVM: arm64: nv: Handle virtual EL2 registers in
> vcpu_read/write_sys_reg()
> KVM: arm64: nv: Handle SPSR_EL2 specially
> KVM: arm64: nv: Handle HCR_EL2.E2H specially
> KVM: arm64: nv: Save/Restore vEL2 sysregs
> KVM: arm64: nv: Forward debug traps to the nested guest
> KVM: arm64: nv: Filter out unsupported features from ID regs
> KVM: arm64: nv: Hide RAS from nested guests
> KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> KVM: arm64: nv: Handle shadow stage 2 page faults
> KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> KVM: arm64: nv: Add handling of EL2-specific timer registers
> KVM: arm64: nv: Load timer before the GIC
> KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> guest
> KVM: arm64: nv: Implement maintenance interrupt forwarding
> KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> information
> KVM: arm64: Allow populating S2 SW bits
> KVM: arm64: nv: Tag shadow S2 entries with nested level
> KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> KVM: arm64: Map VNCR-capable registers to a separate page
> KVM: arm64: nv: Move nested vgic state into the sysreg file
> KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> KVM: arm64: nv: Synchronize PSTATE early on exit
> KVM: arm64: nv: Allocate VNCR page when required
> KVM: arm64: nv: Enable ARMv8.4-NV support
> KVM: arm64: nv: Fast-track 'InHost' exception returns
> KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
>
> .../admin-guide/kernel-parameters.txt | 4 +
> .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> arch/arm64/include/asm/cpucaps.h | 2 +
> arch/arm64/include/asm/esr.h | 6 +
> arch/arm64/include/asm/kvm_arm.h | 28 +-
> arch/arm64/include/asm/kvm_asm.h | 4 +
> arch/arm64/include/asm/kvm_emulate.h | 145 +-
> arch/arm64/include/asm/kvm_host.h | 175 ++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_mmu.h | 17 +-
> arch/arm64/include/asm/kvm_nested.h | 152 ++
> arch/arm64/include/asm/kvm_pgtable.h | 10 +
> arch/arm64/include/asm/sysreg.h | 104 +-
> arch/arm64/include/asm/vncr_mapping.h | 73 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kernel/cpufeature.c | 35 +
> arch/arm64/kvm/Makefile | 4 +-
> arch/arm64/kvm/arch_timer.c | 189 ++-
> arch/arm64/kvm/arm.c | 34 +-
> arch/arm64/kvm/at.c | 231 ++++
> arch/arm64/kvm/emulate-nested.c | 186 +++
> arch/arm64/kvm/guest.c | 6 +
> arch/arm64/kvm/handle_exit.c | 81 +-
> arch/arm64/kvm/hyp/exception.c | 44 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> arch/arm64/kvm/hyp/pgtable.c | 6 +
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> arch/arm64/kvm/inject_fault.c | 62 +-
> arch/arm64/kvm/mmu.c | 183 ++-
> arch/arm64/kvm/nested.c | 908 ++++++++++++
> arch/arm64/kvm/reset.c | 14 +-
> arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> arch/arm64/kvm/sys_regs.h | 6 +
> arch/arm64/kvm/trace_arm.h | 65 +-
> arch/arm64/kvm/vgic/vgic-init.c | 30 +
> arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> arch/arm64/kvm/vgic/vgic.c | 44 +
> arch/arm64/kvm/vgic/vgic.h | 10 +
> include/kvm/arm_arch_timer.h | 7 +
> include/kvm/arm_vgic.h | 16 +
> tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> 50 files changed, 4890 insertions(+), 160 deletions(-)
> create mode 100644 arch/arm64/include/asm/kvm_nested.h
> create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> create mode 100644 arch/arm64/kvm/at.c
> create mode 100644 arch/arm64/kvm/emulate-nested.c
> create mode 100644 arch/arm64/kvm/nested.c
> create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
>
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[-- Attachment #2: kernel_config --]
[-- Type: application/octet-stream, Size: 269068 bytes --]
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm64 5.11.0-rc1 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Ubuntu 9.3.0-11ubuntu0~18.04.1) 9.3.0"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=90300
CONFIG_LD_VERSION=230000000
CONFIG_CLANG_VERSION=0
CONFIG_LLD_VERSION=0
CONFIG_CC_CAN_LINK=y
CONFIG_CC_CAN_LINK_STATIC=y
CONFIG_CC_HAS_ASM_GOTO=y
CONFIG_CC_HAS_ASM_INLINE=y
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_TABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y
#
# General setup
#
CONFIG_INIT_ENV_ARG_LIMIT=32
# CONFIG_COMPILE_TEST is not set
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_BUILD_SALT=""
CONFIG_DEFAULT_INIT=""
CONFIG_DEFAULT_HOSTNAME="(none)"
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
# CONFIG_WATCH_QUEUE is not set
CONFIG_CROSS_MEMORY_ATTACH=y
CONFIG_USELIB=y
CONFIG_AUDIT=y
CONFIG_HAVE_ARCH_AUDITSYSCALL=y
CONFIG_AUDITSYSCALL=y
#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_IRQ_SHOW_LEVEL=y
CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK=y
CONFIG_GENERIC_IRQ_MIGRATION=y
CONFIG_HARDIRQS_SW_RESEND=y
CONFIG_GENERIC_IRQ_CHIP=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_DOMAIN_HIERARCHY=y
CONFIG_IRQ_FASTEOI_HIERARCHY_HANDLERS=y
CONFIG_GENERIC_IRQ_IPI=y
CONFIG_GENERIC_MSI_IRQ=y
CONFIG_GENERIC_MSI_IRQ_DOMAIN=y
CONFIG_IRQ_MSI_IOMMU=y
CONFIG_HANDLE_DOMAIN_IRQ=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
# CONFIG_GENERIC_IRQ_DEBUGFS is not set
# end of IRQ subsystem
CONFIG_GENERIC_IRQ_MULTI_HANDLER=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_ARCH_HAS_TICK_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
#
# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
# CONFIG_NO_HZ_FULL is not set
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
# end of Timers subsystem
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
#
# CPU/Task time and stats accounting
#
CONFIG_TICK_CPU_ACCOUNTING=y
# CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set
# CONFIG_IRQ_TIME_ACCOUNTING is not set
CONFIG_SCHED_THERMAL_PRESSURE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
# CONFIG_PSI is not set
# end of CPU/Task time and stats accounting
CONFIG_CPU_ISOLATION=y
#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
# CONFIG_RCU_EXPERT is not set
CONFIG_SRCU=y
CONFIG_TREE_SRCU=y
CONFIG_TASKS_RCU_GENERIC=y
CONFIG_TASKS_RUDE_RCU=y
CONFIG_TASKS_TRACE_RCU=y
CONFIG_RCU_STALL_COMMON=y
CONFIG_RCU_NEED_SEGCBLIST=y
# end of RCU Subsystem
CONFIG_BUILD_BIN2C=y
# CONFIG_IKCONFIG is not set
# CONFIG_IKHEADERS is not set
CONFIG_LOG_BUF_SHIFT=14
CONFIG_LOG_CPU_MAX_BUF_SHIFT=12
CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
CONFIG_GENERIC_SCHED_CLOCK=y
#
# Scheduler features
#
# CONFIG_UCLAMP_TASK is not set
# end of Scheduler features
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_CC_HAS_INT128=y
CONFIG_ARCH_SUPPORTS_INT128=y
CONFIG_NUMA_BALANCING=y
CONFIG_NUMA_BALANCING_DEFAULT_ENABLED=y
CONFIG_CGROUPS=y
CONFIG_PAGE_COUNTER=y
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_MEMCG_KMEM=y
CONFIG_BLK_CGROUP=y
CONFIG_CGROUP_WRITEBACK=y
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_CFS_BANDWIDTH=y
# CONFIG_RT_GROUP_SCHED is not set
CONFIG_CGROUP_PIDS=y
CONFIG_CGROUP_RDMA=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_HUGETLB=y
CONFIG_CPUSETS=y
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_BPF=y
# CONFIG_CGROUP_DEBUG is not set
CONFIG_SOCK_CGROUP_DATA=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_TIME_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_CHECKPOINT_RESTORE=y
CONFIG_SCHED_AUTOGROUP=y
# CONFIG_SYSFS_DEPRECATED is not set
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
CONFIG_BOOT_CONFIG=y
CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_LD_ORPHAN_WARN=y
CONFIG_SYSCTL=y
CONFIG_HAVE_UID16=y
CONFIG_SYSCTL_EXCEPTION_TRACE=y
CONFIG_BPF=y
CONFIG_EXPERT=y
CONFIG_UID16=y
CONFIG_MULTIUSER=y
CONFIG_SGETMASK_SYSCALL=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_FHANDLE=y
CONFIG_POSIX_TIMERS=y
CONFIG_PRINTK=y
CONFIG_PRINTK_NMI=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
CONFIG_HAVE_FUTEX_CMPXCHG=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_IO_URING=y
CONFIG_ADVISE_SYSCALLS=y
CONFIG_MEMBARRIER=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_BASE_RELATIVE=y
# CONFIG_BPF_LSM is not set
CONFIG_BPF_SYSCALL=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_BPF_JIT_DEFAULT_ON=y
# CONFIG_BPF_PRELOAD is not set
CONFIG_USERFAULTFD=y
CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE=y
CONFIG_RSEQ=y
# CONFIG_DEBUG_RSEQ is not set
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y
CONFIG_PC104=y
#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
# end of Kernel Performance Events And Counters
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_SLUB_DEBUG=y
CONFIG_SLUB_MEMCG_SYSFS_ON=y
# CONFIG_COMPAT_BRK is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
CONFIG_SLAB_MERGE_DEFAULT=y
CONFIG_SLAB_FREELIST_RANDOM=y
CONFIG_SLAB_FREELIST_HARDENED=y
CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
CONFIG_SLUB_CPU_PARTIAL=y
CONFIG_SYSTEM_DATA_VERIFICATION=y
CONFIG_PROFILING=y
CONFIG_TRACEPOINTS=y
# end of General setup
CONFIG_ARM64=y
CONFIG_64BIT=y
CONFIG_MMU=y
CONFIG_ARM64_PAGE_SHIFT=12
CONFIG_ARM64_CONT_PTE_SHIFT=4
CONFIG_ARM64_CONT_PMD_SHIFT=4
CONFIG_ARCH_MMAP_RND_BITS_MIN=18
CONFIG_ARCH_MMAP_RND_BITS_MAX=33
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=11
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_CSUM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_ZONE_DMA=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
CONFIG_SMP=y
CONFIG_KERNEL_MODE_NEON=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_PGTABLE_LEVELS=4
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_ARCH_PROC_KCORE_TEXT=y
#
# Platform selection
#
CONFIG_ARCH_ACTIONS=y
# CONFIG_ARCH_AGILEX is not set
CONFIG_ARCH_SUNXI=y
# CONFIG_ARCH_ALPINE is not set
CONFIG_ARCH_BCM2835=y
# CONFIG_ARCH_BCM4908 is not set
CONFIG_ARCH_BCM_IPROC=y
CONFIG_ARCH_BERLIN=y
# CONFIG_ARCH_BITMAIN is not set
CONFIG_ARCH_BRCMSTB=y
# CONFIG_ARCH_EXYNOS is not set
# CONFIG_ARCH_SPARX5 is not set
# CONFIG_ARCH_K3 is not set
CONFIG_ARCH_LAYERSCAPE=y
CONFIG_ARCH_LG1K=y
CONFIG_ARCH_HISI=y
# CONFIG_ARCH_KEEMBAY is not set
CONFIG_ARCH_MEDIATEK=y
# CONFIG_ARCH_MESON is not set
# CONFIG_ARCH_MVEBU is not set
# CONFIG_ARCH_MXC is not set
CONFIG_ARCH_QCOM=y
CONFIG_ARCH_REALTEK=y
CONFIG_ARCH_RENESAS=y
# CONFIG_ARCH_ROCKCHIP is not set
# CONFIG_ARCH_S32 is not set
CONFIG_ARCH_SEATTLE=y
CONFIG_ARCH_STRATIX10=y
CONFIG_ARCH_SYNQUACER=y
# CONFIG_ARCH_TEGRA is not set
CONFIG_ARCH_SPRD=y
CONFIG_ARCH_THUNDER=y
CONFIG_ARCH_THUNDER2=y
# CONFIG_ARCH_UNIPHIER is not set
CONFIG_ARCH_VEXPRESS=y
# CONFIG_ARCH_VISCONTI is not set
CONFIG_ARCH_XGENE=y
# CONFIG_ARCH_ZX is not set
CONFIG_ARCH_ZYNQMP=y
# end of Platform selection
#
# Kernel Features
#
#
# ARM errata workarounds via the alternatives framework
#
CONFIG_ARM64_WORKAROUND_CLEAN_CACHE=y
CONFIG_ARM64_ERRATUM_826319=y
CONFIG_ARM64_ERRATUM_827319=y
CONFIG_ARM64_ERRATUM_824069=y
CONFIG_ARM64_ERRATUM_819472=y
CONFIG_ARM64_ERRATUM_832075=y
CONFIG_ARM64_ERRATUM_834220=y
CONFIG_ARM64_ERRATUM_845719=y
CONFIG_ARM64_ERRATUM_843419=y
CONFIG_ARM64_ERRATUM_1024718=y
CONFIG_ARM64_ERRATUM_1418040=y
CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT=y
CONFIG_ARM64_ERRATUM_1165522=y
CONFIG_ARM64_ERRATUM_1319367=y
CONFIG_ARM64_ERRATUM_1530923=y
CONFIG_ARM64_WORKAROUND_REPEAT_TLBI=y
CONFIG_ARM64_ERRATUM_1286807=y
CONFIG_ARM64_ERRATUM_1463225=y
CONFIG_ARM64_ERRATUM_1542419=y
CONFIG_ARM64_ERRATUM_1508412=y
CONFIG_CAVIUM_ERRATUM_22375=y
CONFIG_CAVIUM_ERRATUM_23144=y
CONFIG_CAVIUM_ERRATUM_23154=y
CONFIG_CAVIUM_ERRATUM_27456=y
CONFIG_CAVIUM_ERRATUM_30115=y
CONFIG_CAVIUM_TX2_ERRATUM_219=y
CONFIG_FUJITSU_ERRATUM_010001=y
CONFIG_HISILICON_ERRATUM_161600802=y
CONFIG_QCOM_FALKOR_ERRATUM_1003=y
CONFIG_QCOM_FALKOR_ERRATUM_1009=y
CONFIG_QCOM_QDF2400_ERRATUM_0065=y
CONFIG_QCOM_FALKOR_ERRATUM_E1041=y
CONFIG_SOCIONEXT_SYNQUACER_PREITS=y
# end of ARM errata workarounds via the alternatives framework
CONFIG_ARM64_4K_PAGES=y
# CONFIG_ARM64_16K_PAGES is not set
# CONFIG_ARM64_64K_PAGES is not set
# CONFIG_ARM64_VA_BITS_39 is not set
CONFIG_ARM64_VA_BITS_48=y
CONFIG_ARM64_VA_BITS=48
CONFIG_ARM64_PA_BITS_48=y
CONFIG_ARM64_PA_BITS=48
# CONFIG_CPU_BIG_ENDIAN is not set
CONFIG_CPU_LITTLE_ENDIAN=y
CONFIG_SCHED_MC=y
CONFIG_SCHED_SMT=y
CONFIG_NR_CPUS=256
CONFIG_HOTPLUG_CPU=y
CONFIG_NUMA=y
CONFIG_NODES_SHIFT=6
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_HOLES_IN_ZONE=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_SCHED_HRTICK=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_HW_PERF_EVENTS=y
CONFIG_SYS_SUPPORTS_HUGETLBFS=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_TIME_ACCOUNTING is not set
CONFIG_KEXEC=y
# CONFIG_KEXEC_FILE is not set
CONFIG_CRASH_DUMP=y
CONFIG_XEN_DOM0=y
CONFIG_XEN=y
CONFIG_FORCE_MAX_ZONEORDER=11
CONFIG_UNMAP_KERNEL_AT_EL0=y
CONFIG_RODATA_FULL_DEFAULT_ENABLED=y
CONFIG_ARM64_SW_TTBR0_PAN=y
CONFIG_ARM64_TAGGED_ADDR_ABI=y
CONFIG_COMPAT=y
CONFIG_KUSER_HELPERS=y
CONFIG_ARMV8_DEPRECATED=y
CONFIG_SWP_EMULATION=y
CONFIG_CP15_BARRIER_EMULATION=y
CONFIG_SETEND_EMULATION=y
#
# ARMv8.1 architectural features
#
CONFIG_ARM64_HW_AFDBM=y
CONFIG_ARM64_PAN=y
CONFIG_AS_HAS_LDAPR=y
CONFIG_ARM64_LSE_ATOMICS=y
CONFIG_ARM64_USE_LSE_ATOMICS=y
CONFIG_ARM64_VHE=y
# end of ARMv8.1 architectural features
#
# ARMv8.2 architectural features
#
CONFIG_ARM64_PMEM=y
CONFIG_ARM64_RAS_EXTN=y
CONFIG_ARM64_CNP=y
# end of ARMv8.2 architectural features
#
# ARMv8.3 architectural features
#
CONFIG_CC_HAS_BRANCH_PROT_PAC_RET=y
CONFIG_CC_HAS_SIGN_RETURN_ADDRESS=y
CONFIG_AS_HAS_PAC=y
CONFIG_AS_HAS_CFI_NEGATE_RA_STATE=y
# end of ARMv8.3 architectural features
#
# ARMv8.4 architectural features
#
CONFIG_ARM64_AMU_EXTN=y
CONFIG_AS_HAS_ARMV8_4=y
CONFIG_ARM64_TLB_RANGE=y
# end of ARMv8.4 architectural features
#
# ARMv8.5 architectural features
#
CONFIG_AS_HAS_ARMV8_5=y
CONFIG_ARM64_BTI=y
CONFIG_CC_HAS_BRANCH_PROT_PAC_RET_BTI=y
CONFIG_ARM64_E0PD=y
CONFIG_ARCH_RANDOM=y
CONFIG_ARM64_AS_HAS_MTE=y
CONFIG_ARM64_MTE=y
# end of ARMv8.5 architectural features
CONFIG_ARM64_SVE=y
CONFIG_ARM64_MODULE_PLTS=y
# CONFIG_ARM64_PSEUDO_NMI is not set
CONFIG_RELOCATABLE=y
CONFIG_RANDOMIZE_BASE=y
CONFIG_RANDOMIZE_MODULE_REGION_FULL=y
CONFIG_CC_HAVE_STACKPROTECTOR_SYSREG=y
CONFIG_STACKPROTECTOR_PER_TASK=y
# end of Kernel Features
#
# Boot options
#
CONFIG_ARM64_ACPI_PARKING_PROTOCOL=y
CONFIG_CMDLINE="console=ttyAMA0"
CONFIG_CMDLINE_FROM_BOOTLOADER=y
# CONFIG_CMDLINE_EXTEND is not set
# CONFIG_CMDLINE_FORCE is not set
CONFIG_EFI_STUB=y
CONFIG_EFI=y
CONFIG_DMI=y
# end of Boot options
CONFIG_SYSVIPC_COMPAT=y
CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION=y
CONFIG_ARCH_ENABLE_THP_MIGRATION=y
#
# Power management options
#
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
# CONFIG_SUSPEND_SKIP_SYNC is not set
# CONFIG_HIBERNATION is not set
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
# CONFIG_PM_AUTOSLEEP is not set
CONFIG_PM_WAKELOCKS=y
CONFIG_PM_WAKELOCKS_LIMIT=100
CONFIG_PM_WAKELOCKS_GC=y
CONFIG_PM=y
CONFIG_PM_DEBUG=y
CONFIG_PM_ADVANCED_DEBUG=y
# CONFIG_PM_TEST_SUSPEND is not set
CONFIG_PM_SLEEP_DEBUG=y
# CONFIG_DPM_WATCHDOG is not set
CONFIG_PM_CLK=y
CONFIG_PM_GENERIC_DOMAINS=y
CONFIG_WQ_POWER_EFFICIENT_DEFAULT=y
CONFIG_PM_GENERIC_DOMAINS_SLEEP=y
CONFIG_PM_GENERIC_DOMAINS_OF=y
CONFIG_CPU_PM=y
# CONFIG_ENERGY_MODEL is not set
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
# end of Power management options
#
# CPU Power Management
#
#
# CPU Idle
#
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_MULTIPLE_DRIVERS=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
# CONFIG_CPU_IDLE_GOV_TEO is not set
CONFIG_DT_IDLE_STATES=y
#
# ARM CPU Idle Drivers
#
CONFIG_ARM_CPUIDLE=y
# CONFIG_ARM_PSCI_CPUIDLE is not set
# end of ARM CPU Idle Drivers
# end of CPU Idle
#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_ATTR_SET=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y
#
# CPU frequency scaling drivers
#
CONFIG_CPUFREQ_DT=y
CONFIG_CPUFREQ_DT_PLATDEV=y
CONFIG_ACPI_CPPC_CPUFREQ=m
# CONFIG_ARM_ALLWINNER_SUN50I_CPUFREQ_NVMEM is not set
CONFIG_ARM_SCPI_CPUFREQ=m
CONFIG_ARM_BRCMSTB_AVS_CPUFREQ=m
CONFIG_ARM_MEDIATEK_CPUFREQ=m
# CONFIG_ARM_QCOM_CPUFREQ_NVMEM is not set
# CONFIG_ARM_QCOM_CPUFREQ_HW is not set
CONFIG_QORIQ_CPUFREQ=m
# end of CPU Frequency scaling
# end of CPU Power Management
#
# Firmware Drivers
#
# CONFIG_ARM_SCMI_PROTOCOL is not set
CONFIG_ARM_SCPI_PROTOCOL=m
CONFIG_ARM_SCPI_POWER_DOMAIN=m
CONFIG_ARM_SDE_INTERFACE=y
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_DMIID=y
CONFIG_DMI_SYSFS=m
# CONFIG_ISCSI_IBFT is not set
CONFIG_RASPBERRYPI_FIRMWARE=y
CONFIG_FW_CFG_SYSFS=m
# CONFIG_FW_CFG_SYSFS_CMDLINE is not set
# CONFIG_INTEL_STRATIX10_SERVICE is not set
CONFIG_QCOM_SCM=y
# CONFIG_QCOM_SCM_DOWNLOAD_MODE_DEFAULT is not set
CONFIG_TEE_BNXT_FW=m
# CONFIG_GOOGLE_FIRMWARE is not set
#
# EFI (Extensible Firmware Interface) Support
#
CONFIG_EFI_ESRT=y
CONFIG_EFI_VARS_PSTORE=m
# CONFIG_EFI_VARS_PSTORE_DEFAULT_DISABLE is not set
CONFIG_EFI_PARAMS_FROM_FDT=y
CONFIG_EFI_RUNTIME_WRAPPERS=y
CONFIG_EFI_GENERIC_STUB=y
CONFIG_EFI_ARMSTUB_DTB_LOADER=y
# CONFIG_EFI_GENERIC_STUB_INITRD_CMDLINE_LOADER is not set
CONFIG_EFI_BOOTLOADER_CONTROL=m
CONFIG_EFI_CAPSULE_LOADER=m
CONFIG_EFI_TEST=m
CONFIG_RESET_ATTACK_MITIGATION=y
# CONFIG_EFI_DISABLE_PCI_DMA is not set
# end of EFI (Extensible Firmware Interface) Support
CONFIG_UEFI_CPER=y
CONFIG_UEFI_CPER_ARM=y
CONFIG_EFI_EARLYCON=y
CONFIG_EFI_CUSTOM_SSDT_OVERLAYS=y
CONFIG_ARM_PSCI_FW=y
# CONFIG_ARM_PSCI_CHECKER is not set
CONFIG_HAVE_ARM_SMCCC=y
CONFIG_HAVE_ARM_SMCCC_DISCOVERY=y
CONFIG_ARM_SMCCC_SOC_ID=y
#
# Tegra firmware driver
#
# end of Tegra firmware driver
#
# Zynq MPSoC Firmware Drivers
#
CONFIG_ZYNQMP_FIRMWARE=y
# CONFIG_ZYNQMP_FIRMWARE_DEBUG is not set
# end of Zynq MPSoC Firmware Drivers
# end of Firmware Drivers
CONFIG_ARCH_SUPPORTS_ACPI=y
CONFIG_ACPI=y
CONFIG_ACPI_GENERIC_GSI=y
CONFIG_ACPI_CCA_REQUIRED=y
# CONFIG_ACPI_DEBUGGER is not set
CONFIG_ACPI_SPCR_TABLE=y
CONFIG_ACPI_EC_DEBUGFS=m
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
# CONFIG_ACPI_TAD is not set
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR_IDLE=y
CONFIG_ACPI_MCFG=y
CONFIG_ACPI_CPPC_LIB=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_IPMI=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_THERMAL=y
CONFIG_ARCH_HAS_ACPI_TABLE_UPGRADE=y
CONFIG_ACPI_TABLE_UPGRADE=y
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_PCI_SLOT=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_HED=y
# CONFIG_ACPI_CUSTOM_METHOD is not set
CONFIG_ACPI_BGRT=y
CONFIG_ACPI_REDUCED_HARDWARE_ONLY=y
CONFIG_ACPI_NFIT=m
# CONFIG_NFIT_SECURITY_DEBUG is not set
CONFIG_ACPI_NUMA=y
# CONFIG_ACPI_HMAT is not set
CONFIG_HAVE_ACPI_APEI=y
CONFIG_ACPI_APEI=y
CONFIG_ACPI_APEI_GHES=y
CONFIG_ACPI_APEI_PCIEAER=y
CONFIG_ACPI_APEI_SEA=y
CONFIG_ACPI_APEI_MEMORY_FAILURE=y
CONFIG_ACPI_APEI_EINJ=m
# CONFIG_ACPI_APEI_ERST_DEBUG is not set
CONFIG_ACPI_WATCHDOG=y
CONFIG_ACPI_CONFIGFS=m
CONFIG_ACPI_IORT=y
CONFIG_ACPI_GTDT=y
CONFIG_ACPI_PPTT=y
# CONFIG_PMIC_OPREGION is not set
CONFIG_TPS68470_PMIC_OPREGION=y
CONFIG_IRQ_BYPASS_MANAGER=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE=y
CONFIG_KVM_ARM_PMU=y
CONFIG_ARM64_CRYPTO=y
CONFIG_CRYPTO_SHA256_ARM64=m
CONFIG_CRYPTO_SHA512_ARM64=m
CONFIG_CRYPTO_SHA1_ARM64_CE=m
CONFIG_CRYPTO_SHA2_ARM64_CE=m
# CONFIG_CRYPTO_SHA512_ARM64_CE is not set
# CONFIG_CRYPTO_SHA3_ARM64 is not set
# CONFIG_CRYPTO_SM3_ARM64_CE is not set
# CONFIG_CRYPTO_SM4_ARM64_CE is not set
CONFIG_CRYPTO_GHASH_ARM64_CE=m
CONFIG_CRYPTO_CRCT10DIF_ARM64_CE=m
CONFIG_CRYPTO_AES_ARM64=m
CONFIG_CRYPTO_AES_ARM64_CE=m
CONFIG_CRYPTO_AES_ARM64_CE_CCM=m
CONFIG_CRYPTO_AES_ARM64_CE_BLK=m
CONFIG_CRYPTO_AES_ARM64_NEON_BLK=m
CONFIG_CRYPTO_CHACHA20_NEON=m
# CONFIG_CRYPTO_POLY1305_NEON is not set
# CONFIG_CRYPTO_NHPOLY1305_NEON is not set
CONFIG_CRYPTO_AES_ARM64_BS=m
#
# General architecture-dependent options
#
CONFIG_CRASH_CORE=y
CONFIG_KEXEC_CORE=y
CONFIG_KPROBES=y
CONFIG_JUMP_LABEL=y
# CONFIG_STATIC_KEYS_SELFTEST is not set
CONFIG_UPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_FUNCTION_ERROR_INJECTION=y
CONFIG_HAVE_NMI=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_GENERIC_IDLE_POLL_SETUP=y
CONFIG_ARCH_HAS_FORTIFY_SOURCE=y
CONFIG_ARCH_HAS_KEEPINITRD=y
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_ARCH_HAS_SET_DIRECT_MAP=y
CONFIG_HAVE_ARCH_THREAD_STRUCT_WHITELIST=y
CONFIG_HAVE_ASM_MODVERSIONS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_RSEQ=y
CONFIG_HAVE_FUNCTION_ARG_ACCESS_API=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE=y
CONFIG_MMU_GATHER_TABLE_FREE=y
CONFIG_MMU_GATHER_RCU_TABLE_FREE=y
CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
CONFIG_HAVE_CMPXCHG_LOCAL=y
CONFIG_HAVE_CMPXCHG_DOUBLE=y
CONFIG_ARCH_WANT_COMPAT_IPC_PARSE_VERSION=y
CONFIG_HAVE_ARCH_SECCOMP=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
CONFIG_SECCOMP=y
CONFIG_SECCOMP_FILTER=y
# CONFIG_SECCOMP_CACHE_DEBUG is not set
CONFIG_HAVE_ARCH_STACKLEAK=y
CONFIG_HAVE_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR_STRONG=y
CONFIG_HAVE_CONTEXT_TRACKING=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_MOVE_PUD=y
CONFIG_HAVE_MOVE_PMD=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
CONFIG_ARCH_MMAP_RND_BITS=18
CONFIG_HAVE_ARCH_MMAP_RND_COMPAT_BITS=y
CONFIG_ARCH_MMAP_RND_COMPAT_BITS=11
CONFIG_ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT=y
CONFIG_CLONE_BACKWARDS=y
CONFIG_OLD_SIGSUSPEND3=y
CONFIG_COMPAT_OLD_SIGACTION=y
CONFIG_COMPAT_32BIT_TIME=y
CONFIG_HAVE_ARCH_VMAP_STACK=y
CONFIG_VMAP_STACK=y
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_STRICT_MODULE_RWX=y
CONFIG_HAVE_ARCH_COMPILER_H=y
CONFIG_HAVE_ARCH_PREL32_RELOCATIONS=y
CONFIG_ARCH_USE_MEMREMAP_PROT=y
# CONFIG_LOCK_EVENT_COUNTS is not set
CONFIG_ARCH_HAS_RELR=y
CONFIG_ARCH_WANT_LD_ORPHAN_WARN=y
CONFIG_HAVE_ARCH_PFN_VALID=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
# end of GCOV-based kernel profiling
CONFIG_HAVE_GCC_PLUGINS=y
# end of General architecture-dependent options
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODVERSIONS is not set
CONFIG_MODULE_SRCVERSION_ALL=y
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
CONFIG_MODULE_SIG_ALL=y
# CONFIG_MODULE_SIG_SHA1 is not set
# CONFIG_MODULE_SIG_SHA224 is not set
# CONFIG_MODULE_SIG_SHA256 is not set
# CONFIG_MODULE_SIG_SHA384 is not set
CONFIG_MODULE_SIG_SHA512=y
CONFIG_MODULE_SIG_HASH="sha512"
# CONFIG_MODULE_COMPRESS is not set
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
CONFIG_UNUSED_SYMBOLS=y
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_BLOCK=y
CONFIG_BLK_SCSI_REQUEST=y
CONFIG_BLK_CGROUP_RWSTAT=y
CONFIG_BLK_DEV_BSG=y
CONFIG_BLK_DEV_BSGLIB=y
CONFIG_BLK_DEV_INTEGRITY=y
CONFIG_BLK_DEV_INTEGRITY_T10=y
CONFIG_BLK_DEV_ZONED=y
CONFIG_BLK_DEV_THROTTLING=y
# CONFIG_BLK_DEV_THROTTLING_LOW is not set
CONFIG_BLK_CMDLINE_PARSER=y
CONFIG_BLK_WBT=y
# CONFIG_BLK_CGROUP_IOLATENCY is not set
# CONFIG_BLK_CGROUP_IOCOST is not set
CONFIG_BLK_WBT_MQ=y
CONFIG_BLK_DEBUG_FS=y
CONFIG_BLK_DEBUG_FS_ZONED=y
CONFIG_BLK_SED_OPAL=y
# CONFIG_BLK_INLINE_ENCRYPTION is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
CONFIG_AIX_PARTITION=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
CONFIG_SYSV68_PARTITION=y
CONFIG_CMDLINE_PARTITION=y
# end of Partition Types
CONFIG_BLOCK_COMPAT=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_MQ_RDMA=y
CONFIG_BLK_PM=y
#
# IO Schedulers
#
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=m
CONFIG_IOSCHED_BFQ=m
CONFIG_BFQ_GROUP_IOSCHED=y
# CONFIG_BFQ_CGROUP_DEBUG is not set
# end of IO Schedulers
CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_PADATA=y
CONFIG_ASN1=y
CONFIG_ARCH_INLINE_SPIN_TRYLOCK=y
CONFIG_ARCH_INLINE_SPIN_TRYLOCK_BH=y
CONFIG_ARCH_INLINE_SPIN_LOCK=y
CONFIG_ARCH_INLINE_SPIN_LOCK_BH=y
CONFIG_ARCH_INLINE_SPIN_LOCK_IRQ=y
CONFIG_ARCH_INLINE_SPIN_LOCK_IRQSAVE=y
CONFIG_ARCH_INLINE_SPIN_UNLOCK=y
CONFIG_ARCH_INLINE_SPIN_UNLOCK_BH=y
CONFIG_ARCH_INLINE_SPIN_UNLOCK_IRQ=y
CONFIG_ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE=y
CONFIG_ARCH_INLINE_READ_LOCK=y
CONFIG_ARCH_INLINE_READ_LOCK_BH=y
CONFIG_ARCH_INLINE_READ_LOCK_IRQ=y
CONFIG_ARCH_INLINE_READ_LOCK_IRQSAVE=y
CONFIG_ARCH_INLINE_READ_UNLOCK=y
CONFIG_ARCH_INLINE_READ_UNLOCK_BH=y
CONFIG_ARCH_INLINE_READ_UNLOCK_IRQ=y
CONFIG_ARCH_INLINE_READ_UNLOCK_IRQRESTORE=y
CONFIG_ARCH_INLINE_WRITE_LOCK=y
CONFIG_ARCH_INLINE_WRITE_LOCK_BH=y
CONFIG_ARCH_INLINE_WRITE_LOCK_IRQ=y
CONFIG_ARCH_INLINE_WRITE_LOCK_IRQSAVE=y
CONFIG_ARCH_INLINE_WRITE_UNLOCK=y
CONFIG_ARCH_INLINE_WRITE_UNLOCK_BH=y
CONFIG_ARCH_INLINE_WRITE_UNLOCK_IRQ=y
CONFIG_ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE=y
CONFIG_INLINE_SPIN_TRYLOCK=y
CONFIG_INLINE_SPIN_TRYLOCK_BH=y
CONFIG_INLINE_SPIN_LOCK=y
CONFIG_INLINE_SPIN_LOCK_BH=y
CONFIG_INLINE_SPIN_LOCK_IRQ=y
CONFIG_INLINE_SPIN_LOCK_IRQSAVE=y
CONFIG_INLINE_SPIN_UNLOCK_BH=y
CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE=y
CONFIG_INLINE_READ_LOCK=y
CONFIG_INLINE_READ_LOCK_BH=y
CONFIG_INLINE_READ_LOCK_IRQ=y
CONFIG_INLINE_READ_LOCK_IRQSAVE=y
CONFIG_INLINE_READ_UNLOCK=y
CONFIG_INLINE_READ_UNLOCK_BH=y
CONFIG_INLINE_READ_UNLOCK_IRQ=y
CONFIG_INLINE_READ_UNLOCK_IRQRESTORE=y
CONFIG_INLINE_WRITE_LOCK=y
CONFIG_INLINE_WRITE_LOCK_BH=y
CONFIG_INLINE_WRITE_LOCK_IRQ=y
CONFIG_INLINE_WRITE_LOCK_IRQSAVE=y
CONFIG_INLINE_WRITE_UNLOCK=y
CONFIG_INLINE_WRITE_UNLOCK_BH=y
CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_MUTEX_SPIN_ON_OWNER=y
CONFIG_RWSEM_SPIN_ON_OWNER=y
CONFIG_LOCK_SPIN_ON_OWNER=y
CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
CONFIG_QUEUED_SPINLOCKS=y
CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
CONFIG_QUEUED_RWLOCKS=y
CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE=y
CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y
CONFIG_FREEZER=y
#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_ARCH_BINFMT_ELF_STATE=y
CONFIG_ARCH_HAVE_ELF_PROT=y
CONFIG_ARCH_USE_GNU_PROPERTY=y
CONFIG_ELFCORE=y
CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BINFMT_MISC=m
CONFIG_COREDUMP=y
# end of Executable file formats
#
# Memory Management options
#
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_NEED_MULTIPLE_NODES=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_FAST_GUP=y
CONFIG_ARCH_KEEP_MEMBLOCK=y
CONFIG_MEMORY_ISOLATION=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_MEMORY_BALLOON=y
CONFIG_BALLOON_COMPACTION=y
CONFIG_COMPACTION=y
CONFIG_PAGE_REPORTING=y
CONFIG_MIGRATION=y
CONFIG_CONTIG_ALLOC=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_BOUNCE=y
CONFIG_MMU_NOTIFIER=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=32768
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
CONFIG_MEMORY_FAILURE=y
CONFIG_HWPOISON_INJECT=m
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
CONFIG_CLEANCACHE=y
CONFIG_FRONTSWAP=y
CONFIG_CMA=y
# CONFIG_CMA_DEBUG is not set
# CONFIG_CMA_DEBUGFS is not set
CONFIG_CMA_AREAS=7
CONFIG_ZSWAP=y
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_DEFLATE is not set
CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZO=y
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_842 is not set
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZ4 is not set
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZ4HC is not set
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_ZSTD is not set
CONFIG_ZSWAP_COMPRESSOR_DEFAULT="lzo"
CONFIG_ZSWAP_ZPOOL_DEFAULT_ZBUD=y
# CONFIG_ZSWAP_ZPOOL_DEFAULT_Z3FOLD is not set
# CONFIG_ZSWAP_ZPOOL_DEFAULT_ZSMALLOC is not set
CONFIG_ZSWAP_ZPOOL_DEFAULT="zbud"
# CONFIG_ZSWAP_DEFAULT_ON is not set
CONFIG_ZPOOL=y
CONFIG_ZBUD=y
CONFIG_Z3FOLD=m
CONFIG_ZSMALLOC=y
# CONFIG_ZSMALLOC_STAT is not set
CONFIG_GENERIC_EARLY_IOREMAP=y
# CONFIG_DEFERRED_STRUCT_PAGE_INIT is not set
CONFIG_IDLE_PAGE_TRACKING=y
CONFIG_ARCH_HAS_PTE_DEVMAP=y
CONFIG_HMM_MIRROR=y
CONFIG_FRAME_VECTOR=y
CONFIG_ARCH_USES_HIGH_VMA_FLAGS=y
# CONFIG_PERCPU_STATS is not set
# CONFIG_GUP_TEST is not set
# CONFIG_READ_ONLY_THP_FOR_FS is not set
CONFIG_ARCH_HAS_PTE_SPECIAL=y
# end of Memory Management options
CONFIG_NET=y
CONFIG_COMPAT_NETLINK_MESSAGES=y
CONFIG_NET_INGRESS=y
CONFIG_NET_EGRESS=y
CONFIG_NET_REDIRECT=y
CONFIG_SKB_EXTENSIONS=y
#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_DIAG=m
CONFIG_UNIX=y
CONFIG_UNIX_SCM=y
CONFIG_UNIX_DIAG=m
CONFIG_TLS=m
# CONFIG_TLS_DEVICE is not set
# CONFIG_TLS_TOE is not set
CONFIG_XFRM=y
CONFIG_XFRM_OFFLOAD=y
CONFIG_XFRM_ALGO=m
CONFIG_XFRM_USER=m
# CONFIG_XFRM_INTERFACE is not set
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_AH=m
CONFIG_XFRM_ESP=m
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_SMC=m
CONFIG_SMC_DIAG=m
# CONFIG_XDP_SOCKETS is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_FIB_TRIE_STATS=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_IP_ROUTE_CLASSID=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IP_TUNNEL=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE_COMMON=y
CONFIG_IP_MROUTE=y
# CONFIG_IP_MROUTE_MULTIPLE_TABLES is not set
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_NET_UDP_TUNNEL=m
CONFIG_NET_FOU=m
CONFIG_NET_FOU_IP_TUNNELS=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_ESP_OFFLOAD=m
# CONFIG_INET_ESPINTCP is not set
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
CONFIG_INET_UDP_DIAG=m
CONFIG_INET_RAW_DIAG=m
CONFIG_INET_DIAG_DESTROY=y
CONFIG_TCP_CONG_ADVANCED=y
CONFIG_TCP_CONG_BIC=m
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_NV=m
CONFIG_TCP_CONG_SCALABLE=m
CONFIG_TCP_CONG_LP=m
CONFIG_TCP_CONG_VENO=m
CONFIG_TCP_CONG_YEAH=m
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_CONG_DCTCP=m
CONFIG_TCP_CONG_CDG=m
CONFIG_TCP_CONG_BBR=m
CONFIG_DEFAULT_CUBIC=y
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_ESP_OFFLOAD=m
# CONFIG_INET6_ESPINTCP is not set
CONFIG_INET6_IPCOMP=m
CONFIG_IPV6_MIP6=m
CONFIG_IPV6_ILA=m
CONFIG_INET6_XFRM_TUNNEL=m
CONFIG_INET6_TUNNEL=m
CONFIG_IPV6_VTI=m
CONFIG_IPV6_SIT=m
CONFIG_IPV6_SIT_6RD=y
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
CONFIG_IPV6_GRE=m
CONFIG_IPV6_FOU=m
CONFIG_IPV6_FOU_TUNNEL=m
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_IPV6_SUBTREES=y
CONFIG_IPV6_MROUTE=y
CONFIG_IPV6_MROUTE_MULTIPLE_TABLES=y
CONFIG_IPV6_PIMSM_V2=y
CONFIG_IPV6_SEG6_LWTUNNEL=y
CONFIG_IPV6_SEG6_HMAC=y
CONFIG_IPV6_SEG6_BPF=y
# CONFIG_IPV6_RPL_LWTUNNEL is not set
CONFIG_NETLABEL=y
# CONFIG_MPTCP is not set
CONFIG_NETWORK_SECMARK=y
CONFIG_NET_PTP_CLASSIFY=y
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m
#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_INGRESS=y
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_FAMILY_BRIDGE=y
CONFIG_NETFILTER_FAMILY_ARP=y
CONFIG_NETFILTER_NETLINK_ACCT=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NETFILTER_NETLINK_OSF=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_LOG_COMMON=m
CONFIG_NF_LOG_NETDEV=m
CONFIG_NETFILTER_CONNCOUNT=m
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_CONNTRACK_SECMARK=y
CONFIG_NF_CONNTRACK_ZONES=y
# CONFIG_NF_CONNTRACK_PROCFS is not set
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CONNTRACK_TIMEOUT=y
CONFIG_NF_CONNTRACK_TIMESTAMP=y
CONFIG_NF_CONNTRACK_LABELS=y
CONFIG_NF_CT_PROTO_DCCP=y
CONFIG_NF_CT_PROTO_GRE=y
CONFIG_NF_CT_PROTO_SCTP=y
CONFIG_NF_CT_PROTO_UDPLITE=y
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NF_CONNTRACK_IRC=m
CONFIG_NF_CONNTRACK_BROADCAST=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NF_CONNTRACK_SNMP=m
CONFIG_NF_CONNTRACK_PPTP=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
CONFIG_NF_CT_NETLINK=m
CONFIG_NF_CT_NETLINK_TIMEOUT=m
CONFIG_NF_CT_NETLINK_HELPER=m
CONFIG_NETFILTER_NETLINK_GLUE_CT=y
CONFIG_NF_NAT=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_SIP=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_REDIRECT=y
CONFIG_NF_NAT_MASQUERADE=y
CONFIG_NETFILTER_SYNPROXY=m
CONFIG_NF_TABLES=m
# CONFIG_NF_TABLES_INET is not set
# CONFIG_NF_TABLES_NETDEV is not set
CONFIG_NFT_NUMGEN=m
CONFIG_NFT_CT=m
CONFIG_NFT_COUNTER=m
# CONFIG_NFT_CONNLIMIT is not set
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_MASQ=m
CONFIG_NFT_REDIR=m
# CONFIG_NFT_TUNNEL is not set
CONFIG_NFT_OBJREF=m
CONFIG_NFT_QUEUE=m
CONFIG_NFT_QUOTA=m
CONFIG_NFT_REJECT=m
CONFIG_NFT_COMPAT=m
CONFIG_NFT_HASH=m
# CONFIG_NFT_XFRM is not set
# CONFIG_NFT_SOCKET is not set
# CONFIG_NFT_OSF is not set
# CONFIG_NFT_TPROXY is not set
# CONFIG_NFT_SYNPROXY is not set
# CONFIG_NF_FLOW_TABLE is not set
CONFIG_NETFILTER_XTABLES=m
#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=m
CONFIG_NETFILTER_XT_CONNMARK=m
CONFIG_NETFILTER_XT_SET=m
#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
CONFIG_NETFILTER_XT_TARGET_CT=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_HMARK=m
CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
CONFIG_NETFILTER_XT_TARGET_LED=m
CONFIG_NETFILTER_XT_TARGET_LOG=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_NAT=m
CONFIG_NETFILTER_XT_TARGET_NETMAP=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
# CONFIG_NETFILTER_XT_TARGET_NOTRACK is not set
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
CONFIG_NETFILTER_XT_TARGET_REDIRECT=m
CONFIG_NETFILTER_XT_TARGET_MASQUERADE=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_NETFILTER_XT_MATCH_BPF=m
CONFIG_NETFILTER_XT_MATCH_CGROUP=m
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLABEL=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_CPU=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DEVGROUP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ECN=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
CONFIG_NETFILTER_XT_MATCH_IPCOMP=m
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_IPVS=m
CONFIG_NETFILTER_XT_MATCH_L2TP=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_NFACCT=m
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_TIME=m
CONFIG_NETFILTER_XT_MATCH_U32=m
# end of Core Netfilter Configuration
CONFIG_IP_SET=m
CONFIG_IP_SET_MAX=256
CONFIG_IP_SET_BITMAP_IP=m
CONFIG_IP_SET_BITMAP_IPMAC=m
CONFIG_IP_SET_BITMAP_PORT=m
CONFIG_IP_SET_HASH_IP=m
CONFIG_IP_SET_HASH_IPMARK=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
CONFIG_IP_SET_HASH_IPMAC=m
CONFIG_IP_SET_HASH_MAC=m
CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_IP_VS=m
CONFIG_IP_VS_IPV6=y
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12
#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
CONFIG_IP_VS_PROTO_SCTP=y
#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_FO=m
CONFIG_IP_VS_OVF=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
# CONFIG_IP_VS_MH is not set
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m
#
# IPVS SH scheduler
#
CONFIG_IP_VS_SH_TAB_BITS=8
#
# IPVS MH scheduler
#
CONFIG_IP_VS_MH_TAB_INDEX=12
#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PE_SIP=m
#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=m
CONFIG_NF_SOCKET_IPV4=m
CONFIG_NF_TPROXY_IPV4=m
# CONFIG_NF_TABLES_IPV4 is not set
# CONFIG_NF_TABLES_ARP is not set
CONFIG_NF_DUP_IPV4=m
CONFIG_NF_LOG_ARP=m
CONFIG_NF_LOG_IPV4=m
CONFIG_NF_REJECT_IPV4=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PPTP=m
CONFIG_NF_NAT_H323=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_RPFILTER=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_SECURITY=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
# end of IP: Netfilter Configuration
#
# IPv6: Netfilter Configuration
#
CONFIG_NF_SOCKET_IPV6=m
CONFIG_NF_TPROXY_IPV6=m
# CONFIG_NF_TABLES_IPV6 is not set
CONFIG_NF_DUP_IPV6=m
CONFIG_NF_REJECT_IPV6=m
CONFIG_NF_LOG_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_MH=m
CONFIG_IP6_NF_MATCH_RPFILTER=m
CONFIG_IP6_NF_MATCH_RT=m
# CONFIG_IP6_NF_MATCH_SRH is not set
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_IP6_NF_SECURITY=m
CONFIG_IP6_NF_NAT=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
# end of IPv6: Netfilter Configuration
CONFIG_NF_DEFRAG_IPV6=m
#
# DECnet: Netfilter Configuration
#
CONFIG_DECNET_NF_GRABULATOR=m
# end of DECnet: Netfilter Configuration
CONFIG_NF_TABLES_BRIDGE=m
CONFIG_NFT_BRIDGE_META=m
# CONFIG_NFT_BRIDGE_REJECT is not set
CONFIG_NF_LOG_BRIDGE=m
# CONFIG_NF_CONNTRACK_BRIDGE is not set
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
# CONFIG_BPFILTER is not set
CONFIG_IP_DCCP=m
CONFIG_INET_DCCP_DIAG=m
#
# DCCP CCIDs Configuration
#
# CONFIG_IP_DCCP_CCID2_DEBUG is not set
# CONFIG_IP_DCCP_CCID3 is not set
# end of DCCP CCIDs Configuration
#
# DCCP Kernel Hacking
#
# CONFIG_IP_DCCP_DEBUG is not set
# end of DCCP Kernel Hacking
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_DEFAULT_COOKIE_HMAC_MD5 is not set
CONFIG_SCTP_DEFAULT_COOKIE_HMAC_SHA1=y
# CONFIG_SCTP_DEFAULT_COOKIE_HMAC_NONE is not set
CONFIG_SCTP_COOKIE_HMAC_MD5=y
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_INET_SCTP_DIAG=m
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_TIPC=m
CONFIG_TIPC_MEDIA_IB=y
CONFIG_TIPC_MEDIA_UDP=y
CONFIG_TIPC_CRYPTO=y
CONFIG_TIPC_DIAG=m
CONFIG_ATM=m
CONFIG_ATM_CLIP=m
# CONFIG_ATM_CLIP_NO_ICMP is not set
CONFIG_ATM_LANE=m
CONFIG_ATM_MPOA=m
CONFIG_ATM_BR2684=m
# CONFIG_ATM_BR2684_IPFILTER is not set
CONFIG_L2TP=m
CONFIG_L2TP_DEBUGFS=m
CONFIG_L2TP_V3=y
CONFIG_L2TP_IP=m
CONFIG_L2TP_ETH=m
CONFIG_STP=m
CONFIG_GARP=m
CONFIG_MRP=m
CONFIG_BRIDGE=m
CONFIG_BRIDGE_IGMP_SNOOPING=y
CONFIG_BRIDGE_VLAN_FILTERING=y
# CONFIG_BRIDGE_MRP is not set
# CONFIG_BRIDGE_CFM is not set
CONFIG_HAVE_NET_DSA=y
CONFIG_NET_DSA=m
# CONFIG_NET_DSA_TAG_AR9331 is not set
CONFIG_NET_DSA_TAG_BRCM_COMMON=m
CONFIG_NET_DSA_TAG_BRCM=m
CONFIG_NET_DSA_TAG_BRCM_PREPEND=m
# CONFIG_NET_DSA_TAG_HELLCREEK is not set
# CONFIG_NET_DSA_TAG_GSWIP is not set
CONFIG_NET_DSA_TAG_DSA_COMMON=m
CONFIG_NET_DSA_TAG_DSA=m
CONFIG_NET_DSA_TAG_EDSA=m
CONFIG_NET_DSA_TAG_MTK=m
CONFIG_NET_DSA_TAG_KSZ=m
# CONFIG_NET_DSA_TAG_RTL4_A is not set
# CONFIG_NET_DSA_TAG_OCELOT is not set
CONFIG_NET_DSA_TAG_QCA=m
CONFIG_NET_DSA_TAG_LAN9303=m
# CONFIG_NET_DSA_TAG_SJA1105 is not set
CONFIG_NET_DSA_TAG_TRAILER=m
CONFIG_VLAN_8021Q=m
CONFIG_VLAN_8021Q_GVRP=y
CONFIG_VLAN_8021Q_MVRP=y
CONFIG_DECNET=m
# CONFIG_DECNET_ROUTER is not set
CONFIG_LLC=m
CONFIG_LLC2=m
CONFIG_ATALK=m
CONFIG_DEV_APPLETALK=m
# CONFIG_IPDDP is not set
CONFIG_X25=m
CONFIG_LAPB=m
CONFIG_PHONET=m
CONFIG_6LOWPAN=m
# CONFIG_6LOWPAN_DEBUGFS is not set
CONFIG_6LOWPAN_NHC=m
CONFIG_6LOWPAN_NHC_DEST=m
CONFIG_6LOWPAN_NHC_FRAGMENT=m
CONFIG_6LOWPAN_NHC_HOP=m
CONFIG_6LOWPAN_NHC_IPV6=m
CONFIG_6LOWPAN_NHC_MOBILITY=m
CONFIG_6LOWPAN_NHC_ROUTING=m
CONFIG_6LOWPAN_NHC_UDP=m
# CONFIG_6LOWPAN_GHC_EXT_HDR_HOP is not set
# CONFIG_6LOWPAN_GHC_UDP is not set
# CONFIG_6LOWPAN_GHC_ICMPV6 is not set
# CONFIG_6LOWPAN_GHC_EXT_HDR_DEST is not set
# CONFIG_6LOWPAN_GHC_EXT_HDR_FRAG is not set
# CONFIG_6LOWPAN_GHC_EXT_HDR_ROUTE is not set
CONFIG_IEEE802154=m
# CONFIG_IEEE802154_NL802154_EXPERIMENTAL is not set
CONFIG_IEEE802154_SOCKET=m
CONFIG_IEEE802154_6LOWPAN=m
CONFIG_MAC802154=m
CONFIG_NET_SCHED=y
#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_ATM=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_MULTIQ=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFB=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_CBS=m
# CONFIG_NET_SCH_ETF is not set
# CONFIG_NET_SCH_TAPRIO is not set
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_DRR=m
CONFIG_NET_SCH_MQPRIO=m
# CONFIG_NET_SCH_SKBPRIO is not set
CONFIG_NET_SCH_CHOKE=m
CONFIG_NET_SCH_QFQ=m
CONFIG_NET_SCH_CODEL=m
CONFIG_NET_SCH_FQ_CODEL=m
# CONFIG_NET_SCH_CAKE is not set
CONFIG_NET_SCH_FQ=m
CONFIG_NET_SCH_HHF=m
CONFIG_NET_SCH_PIE=m
# CONFIG_NET_SCH_FQ_PIE is not set
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_SCH_PLUG=m
# CONFIG_NET_SCH_ETS is not set
# CONFIG_NET_SCH_DEFAULT is not set
#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
# CONFIG_CLS_U32_PERF is not set
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_FLOW=m
CONFIG_NET_CLS_CGROUP=m
CONFIG_NET_CLS_BPF=m
CONFIG_NET_CLS_FLOWER=m
CONFIG_NET_CLS_MATCHALL=m
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
CONFIG_NET_EMATCH_CMP=m
CONFIG_NET_EMATCH_NBYTE=m
CONFIG_NET_EMATCH_U32=m
CONFIG_NET_EMATCH_META=m
CONFIG_NET_EMATCH_TEXT=m
CONFIG_NET_EMATCH_CANID=m
CONFIG_NET_EMATCH_IPSET=m
# CONFIG_NET_EMATCH_IPT is not set
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_SAMPLE=m
CONFIG_NET_ACT_IPT=m
CONFIG_NET_ACT_NAT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_NET_ACT_SIMP=m
CONFIG_NET_ACT_SKBEDIT=m
CONFIG_NET_ACT_CSUM=m
# CONFIG_NET_ACT_MPLS is not set
CONFIG_NET_ACT_VLAN=m
CONFIG_NET_ACT_BPF=m
CONFIG_NET_ACT_CONNMARK=m
# CONFIG_NET_ACT_CTINFO is not set
CONFIG_NET_ACT_SKBMOD=m
# CONFIG_NET_ACT_IFE is not set
CONFIG_NET_ACT_TUNNEL_KEY=m
# CONFIG_NET_ACT_GATE is not set
# CONFIG_NET_TC_SKB_EXT is not set
CONFIG_NET_SCH_FIFO=y
CONFIG_DCB=y
CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
# CONFIG_BATMAN_ADV_BATMAN_V is not set
CONFIG_BATMAN_ADV_BLA=y
CONFIG_BATMAN_ADV_DAT=y
CONFIG_BATMAN_ADV_NC=y
CONFIG_BATMAN_ADV_MCAST=y
# CONFIG_BATMAN_ADV_DEBUG is not set
# CONFIG_BATMAN_ADV_TRACING is not set
CONFIG_OPENVSWITCH=m
CONFIG_OPENVSWITCH_GRE=m
CONFIG_OPENVSWITCH_VXLAN=m
CONFIG_OPENVSWITCH_GENEVE=m
CONFIG_VSOCKETS=m
CONFIG_VSOCKETS_DIAG=m
CONFIG_VSOCKETS_LOOPBACK=m
CONFIG_VIRTIO_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS_COMMON=m
CONFIG_NETLINK_DIAG=m
CONFIG_MPLS=y
CONFIG_NET_MPLS_GSO=m
CONFIG_MPLS_ROUTING=m
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_HSR=m
CONFIG_NET_SWITCHDEV=y
CONFIG_NET_L3_MASTER_DEV=y
CONFIG_QRTR=m
CONFIG_QRTR_SMD=m
# CONFIG_QRTR_TUN is not set
CONFIG_NET_NCSI=y
# CONFIG_NCSI_OEM_CMD_GET_MAC is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_CGROUP_NET_CLASSID=y
CONFIG_NET_RX_BUSY_POLL=y
CONFIG_BQL=y
CONFIG_BPF_JIT=y
CONFIG_BPF_STREAM_PARSER=y
CONFIG_NET_FLOW_LIMIT=y
#
# Network testing
#
CONFIG_NET_PKTGEN=m
CONFIG_NET_DROP_MONITOR=y
# end of Network testing
# end of Networking options
CONFIG_HAMRADIO=y
#
# Packet Radio protocols
#
CONFIG_AX25=m
CONFIG_AX25_DAMA_SLAVE=y
CONFIG_NETROM=m
CONFIG_ROSE=m
#
# AX.25 network device drivers
#
CONFIG_MKISS=m
CONFIG_6PACK=m
CONFIG_BPQETHER=m
CONFIG_BAYCOM_SER_FDX=m
CONFIG_BAYCOM_SER_HDX=m
CONFIG_BAYCOM_PAR=m
CONFIG_YAM=m
# end of AX.25 network device drivers
CONFIG_CAN=m
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m
CONFIG_CAN_GW=m
# CONFIG_CAN_J1939 is not set
# CONFIG_CAN_ISOTP is not set
#
# CAN Device Drivers
#
CONFIG_CAN_VCAN=m
CONFIG_CAN_VXCAN=m
CONFIG_CAN_SLCAN=m
CONFIG_CAN_DEV=m
CONFIG_CAN_CALC_BITTIMING=y
# CONFIG_CAN_FLEXCAN is not set
CONFIG_CAN_GRCAN=m
CONFIG_CAN_JANZ_ICAN3=m
# CONFIG_CAN_KVASER_PCIEFD is not set
CONFIG_CAN_XILINXCAN=m
CONFIG_CAN_C_CAN=m
CONFIG_CAN_C_CAN_PLATFORM=m
CONFIG_CAN_C_CAN_PCI=m
CONFIG_CAN_CC770=m
CONFIG_CAN_CC770_ISA=m
CONFIG_CAN_CC770_PLATFORM=m
CONFIG_CAN_IFI_CANFD=m
CONFIG_CAN_M_CAN=m
# CONFIG_CAN_M_CAN_PCI is not set
# CONFIG_CAN_M_CAN_PLATFORM is not set
# CONFIG_CAN_M_CAN_TCAN4X5X is not set
CONFIG_CAN_PEAK_PCIEFD=m
CONFIG_CAN_RCAR=m
CONFIG_CAN_RCAR_CANFD=m
CONFIG_CAN_SJA1000=m
CONFIG_CAN_EMS_PCI=m
# CONFIG_CAN_F81601 is not set
CONFIG_CAN_KVASER_PCI=m
CONFIG_CAN_PEAK_PCI=m
CONFIG_CAN_PEAK_PCIEC=y
CONFIG_CAN_PLX_PCI=m
CONFIG_CAN_SJA1000_ISA=m
CONFIG_CAN_SJA1000_PLATFORM=m
CONFIG_CAN_SOFTING=m
#
# CAN SPI interfaces
#
CONFIG_CAN_HI311X=m
CONFIG_CAN_MCP251X=m
# CONFIG_CAN_MCP251XFD is not set
# end of CAN SPI interfaces
#
# CAN USB interfaces
#
CONFIG_CAN_8DEV_USB=m
CONFIG_CAN_EMS_USB=m
CONFIG_CAN_ESD_USB2=m
CONFIG_CAN_GS_USB=m
CONFIG_CAN_KVASER_USB=m
CONFIG_CAN_MCBA_USB=m
CONFIG_CAN_PEAK_USB=m
# CONFIG_CAN_UCAN is not set
# end of CAN USB interfaces
# CONFIG_CAN_DEBUG_DEVICES is not set
# end of CAN Device Drivers
CONFIG_BT=m
CONFIG_BT_BREDR=y
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_CMTP=m
CONFIG_BT_HIDP=m
CONFIG_BT_HS=y
CONFIG_BT_LE=y
CONFIG_BT_6LOWPAN=m
CONFIG_BT_LEDS=y
# CONFIG_BT_MSFTEXT is not set
CONFIG_BT_DEBUGFS=y
# CONFIG_BT_SELFTEST is not set
#
# Bluetooth device drivers
#
CONFIG_BT_INTEL=m
CONFIG_BT_BCM=m
CONFIG_BT_RTL=m
CONFIG_BT_QCA=m
CONFIG_BT_HCIBTUSB=m
# CONFIG_BT_HCIBTUSB_AUTOSUSPEND is not set
CONFIG_BT_HCIBTUSB_BCM=y
# CONFIG_BT_HCIBTUSB_MTK is not set
CONFIG_BT_HCIBTUSB_RTL=y
CONFIG_BT_HCIBTSDIO=m
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_SERDEV=y
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_NOKIA=m
CONFIG_BT_HCIUART_BCSP=y
CONFIG_BT_HCIUART_ATH3K=y
CONFIG_BT_HCIUART_LL=y
CONFIG_BT_HCIUART_3WIRE=y
CONFIG_BT_HCIUART_INTEL=y
CONFIG_BT_HCIUART_BCM=y
# CONFIG_BT_HCIUART_RTL is not set
CONFIG_BT_HCIUART_QCA=y
CONFIG_BT_HCIUART_AG6XX=y
CONFIG_BT_HCIUART_MRVL=y
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIVHCI=m
CONFIG_BT_MRVL=m
CONFIG_BT_MRVL_SDIO=m
CONFIG_BT_ATH3K=m
# CONFIG_BT_MTKSDIO is not set
# CONFIG_BT_MTKUART is not set
CONFIG_BT_QCOMSMD=m
CONFIG_BT_HCIRSI=m
# end of Bluetooth device drivers
CONFIG_AF_RXRPC=m
CONFIG_AF_RXRPC_IPV6=y
# CONFIG_AF_RXRPC_INJECT_LOSS is not set
# CONFIG_AF_RXRPC_DEBUG is not set
CONFIG_RXKAD=y
CONFIG_AF_KCM=m
CONFIG_STREAM_PARSER=y
CONFIG_FIB_RULES=y
CONFIG_WIRELESS=y
CONFIG_WIRELESS_EXT=y
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_WEXT_SPY=y
CONFIG_WEXT_PRIV=y
CONFIG_CFG80211=m
# CONFIG_NL80211_TESTMODE is not set
# CONFIG_CFG80211_DEVELOPER_WARNINGS is not set
# CONFIG_CFG80211_CERTIFICATION_ONUS is not set
CONFIG_CFG80211_REQUIRE_SIGNED_REGDB=y
CONFIG_CFG80211_USE_KERNEL_REGDB_KEYS=y
CONFIG_CFG80211_DEFAULT_PS=y
CONFIG_CFG80211_DEBUGFS=y
CONFIG_CFG80211_CRDA_SUPPORT=y
CONFIG_CFG80211_WEXT=y
CONFIG_CFG80211_WEXT_EXPORT=y
CONFIG_LIB80211=m
CONFIG_LIB80211_CRYPT_WEP=m
CONFIG_LIB80211_CRYPT_CCMP=m
CONFIG_LIB80211_CRYPT_TKIP=m
# CONFIG_LIB80211_DEBUG is not set
CONFIG_MAC80211=m
CONFIG_MAC80211_HAS_RC=y
CONFIG_MAC80211_RC_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT="minstrel_ht"
CONFIG_MAC80211_MESH=y
CONFIG_MAC80211_LEDS=y
CONFIG_MAC80211_DEBUGFS=y
CONFIG_MAC80211_MESSAGE_TRACING=y
# CONFIG_MAC80211_DEBUG_MENU is not set
CONFIG_MAC80211_STA_HASH_MAX_SIZE=0
CONFIG_RFKILL=y
CONFIG_RFKILL_LEDS=y
CONFIG_RFKILL_INPUT=y
CONFIG_RFKILL_GPIO=m
CONFIG_NET_9P=m
CONFIG_NET_9P_VIRTIO=m
CONFIG_NET_9P_XEN=m
CONFIG_NET_9P_RDMA=m
# CONFIG_NET_9P_DEBUG is not set
CONFIG_CAIF=m
# CONFIG_CAIF_DEBUG is not set
CONFIG_CAIF_NETDEV=m
CONFIG_CAIF_USB=m
CONFIG_CEPH_LIB=m
# CONFIG_CEPH_LIB_PRETTYDEBUG is not set
CONFIG_CEPH_LIB_USE_DNS_RESOLVER=y
CONFIG_NFC=m
CONFIG_NFC_DIGITAL=m
CONFIG_NFC_NCI=m
CONFIG_NFC_NCI_SPI=m
CONFIG_NFC_NCI_UART=m
CONFIG_NFC_HCI=m
CONFIG_NFC_SHDLC=y
#
# Near Field Communication (NFC) devices
#
CONFIG_NFC_TRF7970A=m
CONFIG_NFC_SIM=m
CONFIG_NFC_PORT100=m
CONFIG_NFC_FDP=m
CONFIG_NFC_FDP_I2C=m
CONFIG_NFC_PN544=m
CONFIG_NFC_PN544_I2C=m
CONFIG_NFC_PN533=m
CONFIG_NFC_PN533_USB=m
CONFIG_NFC_PN533_I2C=m
# CONFIG_NFC_PN532_UART is not set
CONFIG_NFC_MICROREAD=m
CONFIG_NFC_MICROREAD_I2C=m
CONFIG_NFC_MRVL=m
CONFIG_NFC_MRVL_USB=m
CONFIG_NFC_MRVL_UART=m
CONFIG_NFC_MRVL_I2C=m
CONFIG_NFC_MRVL_SPI=m
CONFIG_NFC_ST21NFCA=m
CONFIG_NFC_ST21NFCA_I2C=m
CONFIG_NFC_ST_NCI=m
CONFIG_NFC_ST_NCI_I2C=m
CONFIG_NFC_ST_NCI_SPI=m
CONFIG_NFC_NXP_NCI=m
CONFIG_NFC_NXP_NCI_I2C=m
CONFIG_NFC_S3FWRN5=m
CONFIG_NFC_S3FWRN5_I2C=m
# CONFIG_NFC_S3FWRN82_UART is not set
CONFIG_NFC_ST95HF=m
# end of Near Field Communication (NFC) devices
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_DST_CACHE=y
CONFIG_GRO_CELLS=y
CONFIG_NET_SOCK_MSG=y
CONFIG_NET_DEVLINK=y
CONFIG_PAGE_POOL=y
CONFIG_FAILOVER=y
CONFIG_ETHTOOL_NETLINK=y
CONFIG_HAVE_EBPF_JIT=y
#
# Device Drivers
#
CONFIG_ARM_AMBA=y
CONFIG_HAVE_PCI=y
CONFIG_PCI=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCI_DOMAINS_GENERIC=y
CONFIG_PCI_SYSCALL=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_PCIEAER=y
# CONFIG_PCIEAER_INJECT is not set
# CONFIG_PCIE_ECRC is not set
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEFAULT=y
# CONFIG_PCIEASPM_POWERSAVE is not set
# CONFIG_PCIEASPM_POWER_SUPERSAVE is not set
# CONFIG_PCIEASPM_PERFORMANCE is not set
CONFIG_PCIE_PME=y
CONFIG_PCIE_DPC=y
CONFIG_PCIE_PTM=y
# CONFIG_PCIE_BW is not set
# CONFIG_PCIE_EDR is not set
CONFIG_PCI_MSI=y
CONFIG_PCI_MSI_IRQ_DOMAIN=y
CONFIG_PCI_MSI_ARCH_FALLBACKS=y
CONFIG_PCI_QUIRKS=y
# CONFIG_PCI_DEBUG is not set
CONFIG_PCI_REALLOC_ENABLE_AUTO=y
CONFIG_PCI_STUB=m
# CONFIG_PCI_PF_STUB is not set
CONFIG_PCI_ATS=y
CONFIG_PCI_ECAM=y
CONFIG_PCI_IOV=y
CONFIG_PCI_PRI=y
CONFIG_PCI_PASID=y
CONFIG_PCI_LABEL=y
# CONFIG_PCIE_BUS_TUNE_OFF is not set
CONFIG_PCIE_BUS_DEFAULT=y
# CONFIG_PCIE_BUS_SAFE is not set
# CONFIG_PCIE_BUS_PERFORMANCE is not set
# CONFIG_PCIE_BUS_PEER2PEER is not set
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
CONFIG_HOTPLUG_PCI_CPCI=y
# CONFIG_HOTPLUG_PCI_SHPC is not set
#
# PCI controller drivers
#
CONFIG_PCIE_XILINX_NWL=y
# CONFIG_PCI_FTPCI100 is not set
CONFIG_PCIE_RCAR=y
CONFIG_PCIE_RCAR_HOST=y
# CONFIG_PCIE_RCAR_EP is not set
CONFIG_PCI_HOST_COMMON=y
CONFIG_PCI_HOST_GENERIC=y
# CONFIG_PCIE_XILINX is not set
# CONFIG_PCIE_XILINX_CPM is not set
CONFIG_PCI_XGENE=y
CONFIG_PCI_XGENE_MSI=y
CONFIG_PCIE_IPROC=m
CONFIG_PCIE_IPROC_PLATFORM=m
CONFIG_PCIE_IPROC_MSI=y
# CONFIG_PCIE_ALTERA is not set
CONFIG_PCI_HOST_THUNDER_PEM=y
CONFIG_PCI_HOST_THUNDER_ECAM=y
CONFIG_PCIE_MEDIATEK=y
# CONFIG_PCIE_BRCMSTB is not set
# CONFIG_PCIE_HISI_ERR is not set
#
# DesignWare PCI Core Support
#
CONFIG_PCIE_DW=y
CONFIG_PCIE_DW_HOST=y
# CONFIG_PCIE_DW_PLAT_HOST is not set
# CONFIG_PCIE_DW_PLAT_EP is not set
CONFIG_PCI_LAYERSCAPE=y
# CONFIG_PCI_LAYERSCAPE_EP is not set
CONFIG_PCI_HISI=y
CONFIG_PCIE_QCOM=y
CONFIG_PCIE_KIRIN=y
CONFIG_PCIE_HISI_STB=y
# CONFIG_PCI_MESON is not set
# CONFIG_PCIE_AL is not set
# end of DesignWare PCI Core Support
#
# Mobiveil PCIe Core Support
#
# CONFIG_PCIE_MOBIVEIL_PLAT is not set
# CONFIG_PCIE_LAYERSCAPE_GEN4 is not set
# end of Mobiveil PCIe Core Support
#
# Cadence PCIe controllers support
#
# CONFIG_PCIE_CADENCE_PLAT_HOST is not set
# CONFIG_PCIE_CADENCE_PLAT_EP is not set
# CONFIG_PCI_J721E_HOST is not set
# CONFIG_PCI_J721E_EP is not set
# end of Cadence PCIe controllers support
# end of PCI controller drivers
#
# PCI Endpoint
#
CONFIG_PCI_ENDPOINT=y
CONFIG_PCI_ENDPOINT_CONFIGFS=y
# CONFIG_PCI_EPF_TEST is not set
# end of PCI Endpoint
#
# PCI switch controller drivers
#
CONFIG_PCI_SW_SWITCHTEC=m
# end of PCI switch controller drivers
# CONFIG_PCCARD is not set
# CONFIG_RAPIDIO is not set
#
# Generic Driver Options
#
CONFIG_AUXILIARY_BUS=y
CONFIG_UEVENT_HELPER=y
CONFIG_UEVENT_HELPER_PATH=""
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
#
# Firmware loader
#
CONFIG_FW_LOADER=y
CONFIG_FW_LOADER_PAGED_BUF=y
CONFIG_EXTRA_FIRMWARE=""
CONFIG_FW_LOADER_USER_HELPER=y
# CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set
# CONFIG_FW_LOADER_COMPRESS is not set
CONFIG_FW_CACHE=y
# end of Firmware loader
CONFIG_WANT_DEV_COREDUMP=y
CONFIG_ALLOW_DEV_COREDUMP=y
CONFIG_DEV_COREDUMP=y
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
# CONFIG_TEST_ASYNC_DRIVER_PROBE is not set
CONFIG_SYS_HYPERVISOR=y
CONFIG_GENERIC_CPU_AUTOPROBE=y
CONFIG_GENERIC_CPU_VULNERABILITIES=y
CONFIG_SOC_BUS=y
CONFIG_REGMAP=y
CONFIG_REGMAP_I2C=y
CONFIG_REGMAP_SPI=y
CONFIG_REGMAP_SPMI=m
CONFIG_REGMAP_W1=m
CONFIG_REGMAP_MMIO=y
CONFIG_REGMAP_IRQ=y
CONFIG_DMA_SHARED_BUFFER=y
# CONFIG_DMA_FENCE_TRACE is not set
CONFIG_GENERIC_ARCH_TOPOLOGY=y
# end of Generic Driver Options
#
# Bus devices
#
CONFIG_ARM_CCI=y
CONFIG_ARM_CCI400_COMMON=y
CONFIG_BRCMSTB_GISB_ARB=y
# CONFIG_MOXTET is not set
CONFIG_HISILICON_LPC=y
CONFIG_QCOM_EBI2=y
CONFIG_SIMPLE_PM_BUS=y
CONFIG_SUN50I_DE2_BUS=y
CONFIG_SUNXI_RSB=m
CONFIG_VEXPRESS_CONFIG=y
CONFIG_FSL_MC_BUS=y
# CONFIG_MHI_BUS is not set
# end of Bus devices
CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
# CONFIG_GNSS is not set
CONFIG_MTD=m
# CONFIG_MTD_TESTS is not set
#
# Partition parsers
#
CONFIG_MTD_AR7_PARTS=m
CONFIG_MTD_CMDLINE_PARTS=m
CONFIG_MTD_OF_PARTS=m
CONFIG_MTD_AFS_PARTS=m
CONFIG_MTD_REDBOOT_PARTS=m
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-1
# CONFIG_MTD_REDBOOT_PARTS_UNALLOCATED is not set
# CONFIG_MTD_REDBOOT_PARTS_READONLY is not set
# end of Partition parsers
#
# User Modules And Translation Layers
#
CONFIG_MTD_BLKDEVS=m
CONFIG_MTD_BLOCK=m
CONFIG_MTD_BLOCK_RO=m
CONFIG_FTL=m
CONFIG_NFTL=m
CONFIG_NFTL_RW=y
CONFIG_INFTL=m
CONFIG_RFD_FTL=m
CONFIG_SSFDC=m
CONFIG_SM_FTL=m
CONFIG_MTD_OOPS=m
CONFIG_MTD_SWAP=m
# CONFIG_MTD_PARTITIONED_MASTER is not set
#
# RAM/ROM/Flash chip drivers
#
CONFIG_MTD_CFI=m
CONFIG_MTD_JEDECPROBE=m
CONFIG_MTD_GEN_PROBE=m
# CONFIG_MTD_CFI_ADV_OPTIONS is not set
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
CONFIG_MTD_CFI_INTELEXT=m
CONFIG_MTD_CFI_AMDSTD=m
CONFIG_MTD_CFI_STAA=m
CONFIG_MTD_CFI_UTIL=m
CONFIG_MTD_RAM=m
CONFIG_MTD_ROM=m
CONFIG_MTD_ABSENT=m
# end of RAM/ROM/Flash chip drivers
#
# Mapping drivers for chip access
#
CONFIG_MTD_COMPLEX_MAPPINGS=y
CONFIG_MTD_PHYSMAP=m
# CONFIG_MTD_PHYSMAP_COMPAT is not set
# CONFIG_MTD_PHYSMAP_OF is not set
# CONFIG_MTD_PHYSMAP_GPIO_ADDR is not set
CONFIG_MTD_PCI=m
CONFIG_MTD_INTEL_VR_NOR=m
CONFIG_MTD_PLATRAM=m
# end of Mapping drivers for chip access
#
# Self-contained MTD device drivers
#
CONFIG_MTD_PMC551=m
# CONFIG_MTD_PMC551_BUGFIX is not set
# CONFIG_MTD_PMC551_DEBUG is not set
CONFIG_MTD_DATAFLASH=m
# CONFIG_MTD_DATAFLASH_WRITE_VERIFY is not set
CONFIG_MTD_DATAFLASH_OTP=y
CONFIG_MTD_MCHP23K256=m
CONFIG_MTD_SST25L=m
CONFIG_MTD_SLRAM=m
CONFIG_MTD_PHRAM=m
CONFIG_MTD_MTDRAM=m
CONFIG_MTDRAM_TOTAL_SIZE=4096
CONFIG_MTDRAM_ERASE_SIZE=128
CONFIG_MTD_BLOCK2MTD=m
#
# Disk-On-Chip Device Drivers
#
CONFIG_MTD_DOCG3=m
CONFIG_BCH_CONST_M=14
CONFIG_BCH_CONST_T=4
# end of Self-contained MTD device drivers
#
# NAND
#
CONFIG_MTD_NAND_CORE=m
CONFIG_MTD_ONENAND=m
CONFIG_MTD_ONENAND_VERIFY_WRITE=y
CONFIG_MTD_ONENAND_GENERIC=m
# CONFIG_MTD_ONENAND_OTP is not set
CONFIG_MTD_ONENAND_2X_PROGRAM=y
# CONFIG_MTD_RAW_NAND is not set
# CONFIG_MTD_SPI_NAND is not set
#
# ECC engine support
#
CONFIG_MTD_NAND_ECC=y
CONFIG_MTD_NAND_ECC_SW_HAMMING=y
# CONFIG_MTD_NAND_ECC_SW_HAMMING_SMC is not set
# CONFIG_MTD_NAND_ECC_SW_BCH is not set
# end of ECC engine support
# end of NAND
#
# LPDDR & LPDDR2 PCM memory drivers
#
CONFIG_MTD_LPDDR=m
CONFIG_MTD_QINFO_PROBE=m
# end of LPDDR & LPDDR2 PCM memory drivers
CONFIG_MTD_SPI_NOR=m
CONFIG_MTD_SPI_NOR_USE_4K_SECTORS=y
# CONFIG_MTD_SPI_NOR_SWP_DISABLE is not set
CONFIG_MTD_SPI_NOR_SWP_DISABLE_ON_VOLATILE=y
# CONFIG_MTD_SPI_NOR_SWP_KEEP is not set
CONFIG_SPI_HISI_SFC=m
CONFIG_MTD_UBI=m
CONFIG_MTD_UBI_WL_THRESHOLD=4096
CONFIG_MTD_UBI_BEB_LIMIT=20
CONFIG_MTD_UBI_FASTMAP=y
CONFIG_MTD_UBI_GLUEBI=m
CONFIG_MTD_UBI_BLOCK=y
# CONFIG_MTD_HYPERBUS is not set
CONFIG_DTC=y
CONFIG_OF=y
# CONFIG_OF_UNITTEST is not set
CONFIG_OF_FLATTREE=y
CONFIG_OF_EARLY_FLATTREE=y
CONFIG_OF_KOBJ=y
CONFIG_OF_DYNAMIC=y
CONFIG_OF_ADDRESS=y
CONFIG_OF_IRQ=y
CONFIG_OF_NET=y
CONFIG_OF_RESERVED_MEM=y
CONFIG_OF_RESOLVE=y
CONFIG_OF_OVERLAY=y
CONFIG_OF_NUMA=y
CONFIG_PARPORT=m
CONFIG_PARPORT_AX88796=m
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
# CONFIG_PNP_DEBUG_MESSAGES is not set
#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_NULL_BLK=m
CONFIG_CDROM=y
CONFIG_BLK_DEV_PCIESSD_MTIP32XX=m
CONFIG_ZRAM=m
CONFIG_ZRAM_DEF_COMP_LZORLE=y
# CONFIG_ZRAM_DEF_COMP_ZSTD is not set
# CONFIG_ZRAM_DEF_COMP_LZ4 is not set
# CONFIG_ZRAM_DEF_COMP_LZO is not set
# CONFIG_ZRAM_DEF_COMP_LZ4HC is not set
# CONFIG_ZRAM_DEF_COMP_842 is not set
CONFIG_ZRAM_DEF_COMP="lzo-rle"
CONFIG_ZRAM_WRITEBACK=y
# CONFIG_ZRAM_MEMORY_TRACKING is not set
CONFIG_BLK_DEV_UMEM=m
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
# CONFIG_DRBD_FAULT_INJECTION is not set
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_SKD=m
CONFIG_BLK_DEV_SX8=m
CONFIG_BLK_DEV_RAM=m
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=65536
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
CONFIG_ATA_OVER_ETH=m
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_BACKEND=m
CONFIG_VIRTIO_BLK=y
CONFIG_BLK_DEV_RBD=m
CONFIG_BLK_DEV_RSXX=m
#
# NVME Support
#
CONFIG_NVME_CORE=m
CONFIG_BLK_DEV_NVME=m
CONFIG_NVME_MULTIPATH=y
# CONFIG_NVME_HWMON is not set
CONFIG_NVME_FABRICS=m
CONFIG_NVME_RDMA=m
CONFIG_NVME_FC=m
# CONFIG_NVME_TCP is not set
CONFIG_NVME_TARGET=m
# CONFIG_NVME_TARGET_PASSTHRU is not set
CONFIG_NVME_TARGET_LOOP=m
CONFIG_NVME_TARGET_RDMA=m
CONFIG_NVME_TARGET_FC=m
# CONFIG_NVME_TARGET_FCLOOP is not set
# CONFIG_NVME_TARGET_TCP is not set
# end of NVME Support
#
# Misc devices
#
CONFIG_SENSORS_LIS3LV02D=m
CONFIG_AD525X_DPOT=m
CONFIG_AD525X_DPOT_I2C=m
CONFIG_AD525X_DPOT_SPI=m
CONFIG_DUMMY_IRQ=m
CONFIG_PHANTOM=m
CONFIG_TIFM_CORE=m
CONFIG_TIFM_7XX1=m
CONFIG_ICS932S401=m
CONFIG_ENCLOSURE_SERVICES=m
CONFIG_HP_ILO=m
CONFIG_QCOM_COINCELL=m
# CONFIG_QCOM_FASTRPC is not set
CONFIG_APDS9802ALS=m
CONFIG_ISL29003=m
CONFIG_ISL29020=m
CONFIG_SENSORS_TSL2550=m
CONFIG_SENSORS_BH1770=m
CONFIG_SENSORS_APDS990X=m
CONFIG_HMC6352=m
CONFIG_DS1682=m
CONFIG_LATTICE_ECP3_CONFIG=m
CONFIG_SRAM=y
# CONFIG_PCI_ENDPOINT_TEST is not set
# CONFIG_XILINX_SDFEC is not set
CONFIG_MISC_RTSX=m
# CONFIG_PVPANIC is not set
# CONFIG_HISI_HIKEY_USB is not set
CONFIG_C2PORT=m
#
# EEPROM support
#
CONFIG_EEPROM_AT24=m
CONFIG_EEPROM_AT25=m
CONFIG_EEPROM_LEGACY=m
CONFIG_EEPROM_MAX6875=m
CONFIG_EEPROM_93CX6=m
CONFIG_EEPROM_93XX46=m
CONFIG_EEPROM_IDT_89HPESX=m
# CONFIG_EEPROM_EE1004 is not set
# end of EEPROM support
CONFIG_CB710_CORE=m
# CONFIG_CB710_DEBUG is not set
CONFIG_CB710_DEBUG_ASSUMPTIONS=y
#
# Texas Instruments shared transport line discipline
#
CONFIG_TI_ST=m
# end of Texas Instruments shared transport line discipline
CONFIG_SENSORS_LIS3_I2C=m
CONFIG_ALTERA_STAPL=m
CONFIG_GENWQE=m
CONFIG_GENWQE_PLATFORM_ERROR_RECOVERY=0
CONFIG_ECHO=m
# CONFIG_MISC_ALCOR_PCI is not set
CONFIG_MISC_RTSX_PCI=m
CONFIG_MISC_RTSX_USB=m
# CONFIG_HABANA_AI is not set
# CONFIG_UACCE is not set
# end of Misc devices
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=m
CONFIG_BLK_DEV_SR=y
CONFIG_CHR_DEV_SG=y
CONFIG_CHR_DEV_SCH=m
CONFIG_SCSI_ENCLOSURE=m
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y
#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
CONFIG_SCSI_ISCSI_ATTRS=m
CONFIG_SCSI_SAS_ATTRS=m
CONFIG_SCSI_SAS_LIBSAS=m
CONFIG_SCSI_SAS_ATA=y
CONFIG_SCSI_SAS_HOST_SMP=y
CONFIG_SCSI_SRP_ATTRS=m
# end of SCSI Transports
CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=m
CONFIG_ISCSI_BOOT_SYSFS=m
CONFIG_SCSI_CXGB3_ISCSI=m
CONFIG_SCSI_CXGB4_ISCSI=m
CONFIG_SCSI_BNX2_ISCSI=m
CONFIG_SCSI_BNX2X_FCOE=m
CONFIG_BE2ISCSI=m
CONFIG_BLK_DEV_3W_XXXX_RAID=m
CONFIG_SCSI_HPSA=m
CONFIG_SCSI_3W_9XXX=m
CONFIG_SCSI_3W_SAS=m
CONFIG_SCSI_ACARD=m
CONFIG_SCSI_AACRAID=m
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=8
CONFIG_AIC7XXX_RESET_DELAY_MS=5000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=5000
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC94XX=m
# CONFIG_AIC94XX_DEBUG is not set
CONFIG_SCSI_HISI_SAS=m
CONFIG_SCSI_HISI_SAS_PCI=m
CONFIG_SCSI_MVSAS=m
# CONFIG_SCSI_MVSAS_DEBUG is not set
# CONFIG_SCSI_MVSAS_TASKLET is not set
CONFIG_SCSI_MVUMI=m
CONFIG_SCSI_ADVANSYS=m
CONFIG_SCSI_ARCMSR=m
CONFIG_SCSI_ESAS2R=m
CONFIG_MEGARAID_NEWGEN=y
CONFIG_MEGARAID_MM=m
CONFIG_MEGARAID_MAILBOX=m
CONFIG_MEGARAID_LEGACY=m
CONFIG_MEGARAID_SAS=m
CONFIG_SCSI_MPT3SAS=m
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT3SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS=m
CONFIG_SCSI_SMARTPQI=m
CONFIG_SCSI_UFSHCD=m
CONFIG_SCSI_UFSHCD_PCI=m
CONFIG_SCSI_UFS_DWC_TC_PCI=m
CONFIG_SCSI_UFSHCD_PLATFORM=m
# CONFIG_SCSI_UFS_CDNS_PLATFORM is not set
CONFIG_SCSI_UFS_DWC_TC_PLATFORM=m
# CONFIG_SCSI_UFS_QCOM is not set
# CONFIG_SCSI_UFS_MEDIATEK is not set
# CONFIG_SCSI_UFS_HISI is not set
# CONFIG_SCSI_UFS_BSG is not set
CONFIG_SCSI_HPTIOP=m
# CONFIG_SCSI_MYRB is not set
# CONFIG_SCSI_MYRS is not set
CONFIG_XEN_SCSI_FRONTEND=m
CONFIG_LIBFC=m
CONFIG_LIBFCOE=m
CONFIG_FCOE=m
CONFIG_SCSI_SNIC=m
# CONFIG_SCSI_SNIC_DEBUG_FS is not set
CONFIG_SCSI_DMX3191D=m
# CONFIG_SCSI_FDOMAIN_PCI is not set
# CONFIG_SCSI_GDTH is not set
CONFIG_SCSI_IPS=m
CONFIG_SCSI_INITIO=m
CONFIG_SCSI_INIA100=m
CONFIG_SCSI_STEX=m
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y
CONFIG_SCSI_IPR=m
CONFIG_SCSI_IPR_TRACE=y
CONFIG_SCSI_IPR_DUMP=y
CONFIG_SCSI_QLOGIC_1280=m
CONFIG_SCSI_QLA_FC=m
CONFIG_TCM_QLA2XXX=m
# CONFIG_TCM_QLA2XXX_DEBUG is not set
CONFIG_SCSI_QLA_ISCSI=m
CONFIG_QEDI=m
CONFIG_QEDF=m
CONFIG_SCSI_LPFC=m
# CONFIG_SCSI_LPFC_DEBUG_FS is not set
CONFIG_SCSI_DC395x=m
CONFIG_SCSI_AM53C974=m
CONFIG_SCSI_WD719X=m
CONFIG_SCSI_DEBUG=m
CONFIG_SCSI_PMCRAID=m
CONFIG_SCSI_PM8001=m
CONFIG_SCSI_BFA_FC=m
CONFIG_SCSI_VIRTIO=m
CONFIG_SCSI_CHELSIO_FCOE=m
CONFIG_SCSI_DH=y
CONFIG_SCSI_DH_RDAC=m
CONFIG_SCSI_DH_HP_SW=m
CONFIG_SCSI_DH_EMC=m
CONFIG_SCSI_DH_ALUA=m
# end of SCSI device support
CONFIG_HAVE_PATA_PLATFORM=y
CONFIG_ATA=y
CONFIG_SATA_HOST=y
CONFIG_PATA_TIMINGS=y
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_FORCE=y
CONFIG_ATA_ACPI=y
CONFIG_SATA_ZPODD=y
CONFIG_SATA_PMP=y
#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=m
CONFIG_SATA_MOBILE_LPM_POLICY=0
CONFIG_SATA_AHCI_PLATFORM=m
CONFIG_AHCI_BRCM=m
CONFIG_AHCI_CEVA=m
CONFIG_AHCI_MTK=m
# CONFIG_AHCI_SUNXI is not set
CONFIG_AHCI_XGENE=m
CONFIG_AHCI_QORIQ=m
CONFIG_SATA_AHCI_SEATTLE=m
CONFIG_SATA_INIC162X=m
CONFIG_SATA_ACARD_AHCI=m
CONFIG_SATA_SIL24=m
CONFIG_ATA_SFF=y
#
# SFF controllers with custom DMA interface
#
CONFIG_PDC_ADMA=m
CONFIG_SATA_QSTOR=m
CONFIG_SATA_SX4=m
CONFIG_ATA_BMDMA=y
#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=m
CONFIG_SATA_DWC=m
CONFIG_SATA_DWC_OLD_DMA=y
# CONFIG_SATA_DWC_DEBUG is not set
CONFIG_SATA_MV=m
CONFIG_SATA_NV=m
CONFIG_SATA_PROMISE=m
CONFIG_SATA_RCAR=m
CONFIG_SATA_SIL=m
CONFIG_SATA_SIS=m
CONFIG_SATA_SVW=m
CONFIG_SATA_ULI=m
CONFIG_SATA_VIA=m
CONFIG_SATA_VITESSE=m
#
# PATA SFF controllers with BMDMA
#
CONFIG_PATA_ALI=m
CONFIG_PATA_AMD=m
CONFIG_PATA_ARTOP=m
CONFIG_PATA_ATIIXP=m
CONFIG_PATA_ATP867X=m
CONFIG_PATA_CMD64X=m
CONFIG_PATA_CYPRESS=m
CONFIG_PATA_EFAR=m
CONFIG_PATA_HPT366=m
CONFIG_PATA_HPT37X=m
CONFIG_PATA_HPT3X2N=m
CONFIG_PATA_HPT3X3=m
# CONFIG_PATA_HPT3X3_DMA is not set
CONFIG_PATA_IT8213=m
CONFIG_PATA_IT821X=m
CONFIG_PATA_JMICRON=m
CONFIG_PATA_MARVELL=m
CONFIG_PATA_NETCELL=m
CONFIG_PATA_NINJA32=m
CONFIG_PATA_NS87415=m
CONFIG_PATA_OLDPIIX=m
CONFIG_PATA_OPTIDMA=m
CONFIG_PATA_PDC2027X=m
CONFIG_PATA_PDC_OLD=m
CONFIG_PATA_RADISYS=m
CONFIG_PATA_RDC=m
CONFIG_PATA_SCH=m
CONFIG_PATA_SERVERWORKS=m
CONFIG_PATA_SIL680=m
CONFIG_PATA_SIS=m
CONFIG_PATA_TOSHIBA=m
CONFIG_PATA_TRIFLEX=m
CONFIG_PATA_VIA=m
CONFIG_PATA_WINBOND=m
#
# PIO-only SFF controllers
#
CONFIG_PATA_CMD640_PCI=m
CONFIG_PATA_MPIIX=m
CONFIG_PATA_NS87410=m
CONFIG_PATA_OPTI=m
CONFIG_PATA_PLATFORM=m
CONFIG_PATA_OF_PLATFORM=m
CONFIG_PATA_RZ1000=m
#
# Generic fallback / legacy drivers
#
CONFIG_PATA_ACPI=m
CONFIG_ATA_GENERIC=m
CONFIG_PATA_LEGACY=m
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=m
CONFIG_MD_MULTIPATH=m
CONFIG_MD_FAULTY=m
CONFIG_MD_CLUSTER=m
CONFIG_BCACHE=m
# CONFIG_BCACHE_DEBUG is not set
# CONFIG_BCACHE_CLOSURES_DEBUG is not set
# CONFIG_BCACHE_ASYNC_REGISTRATION is not set
CONFIG_BLK_DEV_DM_BUILTIN=y
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
CONFIG_DM_BUFIO=m
# CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING is not set
CONFIG_DM_BIO_PRISON=m
CONFIG_DM_PERSISTENT_DATA=m
# CONFIG_DM_UNSTRIPED is not set
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_THIN_PROVISIONING=m
CONFIG_DM_CACHE=m
CONFIG_DM_CACHE_SMQ=m
# CONFIG_DM_WRITECACHE is not set
# CONFIG_DM_EBS is not set
CONFIG_DM_ERA=m
# CONFIG_DM_CLONE is not set
CONFIG_DM_MIRROR=m
CONFIG_DM_LOG_USERSPACE=m
CONFIG_DM_RAID=m
CONFIG_DM_ZERO=m
CONFIG_DM_MULTIPATH=m
CONFIG_DM_MULTIPATH_QL=m
CONFIG_DM_MULTIPATH_ST=m
# CONFIG_DM_MULTIPATH_HST is not set
# CONFIG_DM_MULTIPATH_IOA is not set
CONFIG_DM_DELAY=m
# CONFIG_DM_DUST is not set
# CONFIG_DM_INIT is not set
CONFIG_DM_UEVENT=y
CONFIG_DM_FLAKEY=m
CONFIG_DM_VERITY=m
# CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG is not set
# CONFIG_DM_VERITY_FEC is not set
CONFIG_DM_SWITCH=m
CONFIG_DM_LOG_WRITES=m
CONFIG_DM_INTEGRITY=m
CONFIG_DM_ZONED=m
CONFIG_TARGET_CORE=m
CONFIG_TCM_IBLOCK=m
CONFIG_TCM_FILEIO=m
CONFIG_TCM_PSCSI=m
CONFIG_TCM_USER2=m
CONFIG_LOOPBACK_TARGET=m
CONFIG_TCM_FC=m
CONFIG_ISCSI_TARGET=m
CONFIG_ISCSI_TARGET_CXGB4=m
CONFIG_SBP_TARGET=m
CONFIG_FUSION=y
CONFIG_FUSION_SPI=m
CONFIG_FUSION_FC=m
CONFIG_FUSION_SAS=m
CONFIG_FUSION_MAX_SGE=128
CONFIG_FUSION_CTL=m
CONFIG_FUSION_LAN=m
CONFIG_FUSION_LOGGING=y
#
# IEEE 1394 (FireWire) support
#
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_SBP2=m
CONFIG_FIREWIRE_NET=m
CONFIG_FIREWIRE_NOSY=m
# end of IEEE 1394 (FireWire) support
CONFIG_NETDEVICES=y
CONFIG_MII=y
CONFIG_NET_CORE=y
CONFIG_BONDING=m
CONFIG_DUMMY=m
# CONFIG_WIREGUARD is not set
CONFIG_EQUALIZER=m
CONFIG_NET_FC=y
CONFIG_IFB=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
CONFIG_IPVLAN_L3S=y
CONFIG_IPVLAN=m
CONFIG_IPVTAP=m
CONFIG_VXLAN=m
CONFIG_GENEVE=m
# CONFIG_BAREUDP is not set
CONFIG_GTP=m
CONFIG_MACSEC=m
CONFIG_NETCONSOLE=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_NETPOLL=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_NTB_NETDEV=m
CONFIG_TUN=y
CONFIG_TAP=m
# CONFIG_TUN_VNET_CROSS_LE is not set
CONFIG_VETH=m
CONFIG_VIRTIO_NET=y
CONFIG_NLMON=m
CONFIG_NET_VRF=m
CONFIG_VSOCKMON=m
CONFIG_SUNGEM_PHY=m
CONFIG_ARCNET=m
CONFIG_ARCNET_1201=m
CONFIG_ARCNET_1051=m
CONFIG_ARCNET_RAW=m
CONFIG_ARCNET_CAP=m
CONFIG_ARCNET_COM90xx=m
CONFIG_ARCNET_COM90xxIO=m
CONFIG_ARCNET_RIM_I=m
CONFIG_ARCNET_COM20020=m
CONFIG_ARCNET_COM20020_PCI=m
CONFIG_ATM_DRIVERS=y
CONFIG_ATM_DUMMY=m
CONFIG_ATM_TCP=m
CONFIG_ATM_LANAI=m
CONFIG_ATM_ENI=m
# CONFIG_ATM_ENI_DEBUG is not set
# CONFIG_ATM_ENI_TUNE_BURST is not set
CONFIG_ATM_NICSTAR=m
# CONFIG_ATM_NICSTAR_USE_SUNI is not set
# CONFIG_ATM_NICSTAR_USE_IDT77105 is not set
CONFIG_ATM_IDT77252=m
# CONFIG_ATM_IDT77252_DEBUG is not set
# CONFIG_ATM_IDT77252_RCV_ALL is not set
CONFIG_ATM_IDT77252_USE_SUNI=y
CONFIG_ATM_IA=m
# CONFIG_ATM_IA_DEBUG is not set
CONFIG_ATM_FORE200E=m
# CONFIG_ATM_FORE200E_USE_TASKLET is not set
CONFIG_ATM_FORE200E_TX_RETRY=16
CONFIG_ATM_FORE200E_DEBUG=0
CONFIG_ATM_HE=m
CONFIG_ATM_HE_USE_SUNI=y
CONFIG_ATM_SOLOS=m
# CONFIG_CAIF_DRIVERS is not set
#
# Distributed Switch Architecture drivers
#
CONFIG_B53=m
CONFIG_B53_SPI_DRIVER=m
CONFIG_B53_MDIO_DRIVER=m
CONFIG_B53_MMAP_DRIVER=m
CONFIG_B53_SRAB_DRIVER=m
# CONFIG_B53_SERDES is not set
CONFIG_NET_DSA_BCM_SF2=m
# CONFIG_NET_DSA_LOOP is not set
# CONFIG_NET_DSA_HIRSCHMANN_HELLCREEK is not set
# CONFIG_NET_DSA_LANTIQ_GSWIP is not set
CONFIG_NET_DSA_MT7530=m
CONFIG_NET_DSA_MV88E6060=m
# CONFIG_NET_DSA_MICROCHIP_KSZ9477 is not set
# CONFIG_NET_DSA_MICROCHIP_KSZ8795 is not set
CONFIG_NET_DSA_MV88E6XXX=m
CONFIG_NET_DSA_MV88E6XXX_GLOBAL2=y
# CONFIG_NET_DSA_MV88E6XXX_PTP is not set
# CONFIG_NET_DSA_MSCC_FELIX is not set
# CONFIG_NET_DSA_MSCC_SEVILLE is not set
# CONFIG_NET_DSA_AR9331 is not set
# CONFIG_NET_DSA_SJA1105 is not set
CONFIG_NET_DSA_QCA8K=m
# CONFIG_NET_DSA_REALTEK_SMI is not set
CONFIG_NET_DSA_SMSC_LAN9303=m
CONFIG_NET_DSA_SMSC_LAN9303_I2C=m
CONFIG_NET_DSA_SMSC_LAN9303_MDIO=m
# CONFIG_NET_DSA_VITESSE_VSC73XX_SPI is not set
# CONFIG_NET_DSA_VITESSE_VSC73XX_PLATFORM is not set
# end of Distributed Switch Architecture drivers
CONFIG_ETHERNET=y
CONFIG_MDIO=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
CONFIG_NET_VENDOR_ADAPTEC=y
CONFIG_ADAPTEC_STARFIRE=m
CONFIG_NET_VENDOR_AGERE=y
CONFIG_ET131X=m
CONFIG_NET_VENDOR_ALACRITECH=y
CONFIG_SLICOSS=m
CONFIG_NET_VENDOR_ALLWINNER=y
# CONFIG_SUN4I_EMAC is not set
CONFIG_NET_VENDOR_ALTEON=y
CONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set
CONFIG_ALTERA_TSE=m
CONFIG_NET_VENDOR_AMAZON=y
# CONFIG_ENA_ETHERNET is not set
CONFIG_NET_VENDOR_AMD=y
CONFIG_AMD8111_ETH=m
CONFIG_PCNET32=m
CONFIG_AMD_XGBE=m
CONFIG_AMD_XGBE_DCB=y
CONFIG_NET_XGENE=m
CONFIG_NET_XGENE_V2=m
CONFIG_NET_VENDOR_AQUANTIA=y
# CONFIG_AQTION is not set
CONFIG_NET_VENDOR_ARC=y
CONFIG_NET_VENDOR_ATHEROS=y
CONFIG_ATL2=m
CONFIG_ATL1=m
CONFIG_ATL1E=m
CONFIG_ATL1C=m
CONFIG_ALX=m
CONFIG_NET_VENDOR_AURORA=y
CONFIG_AURORA_NB8800=m
CONFIG_NET_VENDOR_BROADCOM=y
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y
CONFIG_B44_PCICORE_AUTOSELECT=y
CONFIG_B44_PCI=y
CONFIG_BCMGENET=m
CONFIG_BNX2=m
CONFIG_CNIC=m
CONFIG_TIGON3=m
CONFIG_TIGON3_HWMON=y
CONFIG_BNX2X=m
CONFIG_BNX2X_SRIOV=y
CONFIG_BGMAC=y
CONFIG_BGMAC_PLATFORM=y
CONFIG_SYSTEMPORT=m
CONFIG_BNXT=m
CONFIG_BNXT_SRIOV=y
CONFIG_BNXT_FLOWER_OFFLOAD=y
CONFIG_BNXT_DCB=y
CONFIG_BNXT_HWMON=y
CONFIG_NET_VENDOR_BROCADE=y
CONFIG_BNA=m
CONFIG_NET_VENDOR_CADENCE=y
CONFIG_MACB=m
CONFIG_MACB_USE_HWSTAMP=y
CONFIG_MACB_PCI=m
CONFIG_NET_VENDOR_CAVIUM=y
CONFIG_THUNDER_NIC_PF=m
CONFIG_THUNDER_NIC_VF=m
CONFIG_THUNDER_NIC_BGX=m
CONFIG_THUNDER_NIC_RGX=m
CONFIG_CAVIUM_PTP=m
CONFIG_LIQUIDIO=m
CONFIG_LIQUIDIO_VF=m
CONFIG_NET_VENDOR_CHELSIO=y
CONFIG_CHELSIO_T1=m
CONFIG_CHELSIO_T1_1G=y
CONFIG_CHELSIO_T3=m
CONFIG_CHELSIO_T4=m
CONFIG_CHELSIO_T4_DCB=y
CONFIG_CHELSIO_T4_FCOE=y
CONFIG_CHELSIO_T4VF=m
CONFIG_CHELSIO_LIB=m
CONFIG_CHELSIO_INLINE_CRYPTO=y
# CONFIG_CHELSIO_IPSEC_INLINE is not set
CONFIG_NET_VENDOR_CISCO=y
CONFIG_ENIC=m
CONFIG_NET_VENDOR_CORTINA=y
# CONFIG_GEMINI_ETHERNET is not set
CONFIG_DNET=m
CONFIG_NET_VENDOR_DEC=y
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_DE2104X_DSL=0
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_ULI526X=m
CONFIG_NET_VENDOR_DLINK=y
CONFIG_DL2K=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_NET_VENDOR_EMULEX=y
CONFIG_BE2NET=m
CONFIG_BE2NET_HWMON=y
CONFIG_BE2NET_BE2=y
CONFIG_BE2NET_BE3=y
CONFIG_BE2NET_LANCER=y
CONFIG_BE2NET_SKYHAWK=y
CONFIG_NET_VENDOR_EZCHIP=y
CONFIG_EZCHIP_NPS_MANAGEMENT_ENET=m
CONFIG_NET_VENDOR_FREESCALE=y
CONFIG_FSL_FMAN=y
CONFIG_DPAA_ERRATUM_A050385=y
CONFIG_FSL_PQ_MDIO=m
CONFIG_FSL_XGMAC_MDIO=y
CONFIG_GIANFAR=m
CONFIG_FSL_DPAA_ETH=m
CONFIG_FSL_DPAA2_ETH=m
# CONFIG_FSL_DPAA2_ETH_DCB is not set
CONFIG_FSL_DPAA2_PTP_CLOCK=m
# CONFIG_FSL_ENETC is not set
# CONFIG_FSL_ENETC_VF is not set
# CONFIG_FSL_ENETC_MDIO is not set
CONFIG_NET_VENDOR_GOOGLE=y
# CONFIG_GVE is not set
CONFIG_NET_VENDOR_HISILICON=y
CONFIG_HIX5HD2_GMAC=m
CONFIG_HISI_FEMAC=m
CONFIG_HIP04_ETH=m
# CONFIG_HI13X1_GMAC is not set
CONFIG_HNS_MDIO=m
CONFIG_HNS=m
CONFIG_HNS_DSAF=m
CONFIG_HNS_ENET=m
CONFIG_HNS3=m
CONFIG_HNS3_HCLGE=m
CONFIG_HNS3_DCB=y
CONFIG_HNS3_HCLGEVF=m
CONFIG_HNS3_ENET=m
CONFIG_NET_VENDOR_HUAWEI=y
CONFIG_HINIC=m
CONFIG_NET_VENDOR_I825XX=y
CONFIG_NET_VENDOR_INTEL=y
CONFIG_E100=m
CONFIG_E1000=m
CONFIG_E1000E=m
CONFIG_IGB=m
CONFIG_IGB_HWMON=y
CONFIG_IGBVF=m
CONFIG_IXGB=m
CONFIG_IXGBE=m
CONFIG_IXGBE_HWMON=y
CONFIG_IXGBE_DCB=y
CONFIG_IXGBE_IPSEC=y
CONFIG_IXGBEVF=m
CONFIG_IXGBEVF_IPSEC=y
CONFIG_I40E=m
CONFIG_I40E_DCB=y
CONFIG_IAVF=m
CONFIG_I40EVF=m
# CONFIG_ICE is not set
CONFIG_FM10K=m
# CONFIG_IGC is not set
CONFIG_JME=m
CONFIG_NET_VENDOR_MARVELL=y
CONFIG_MVMDIO=m
CONFIG_PXA168_ETH=m
CONFIG_SKGE=m
# CONFIG_SKGE_DEBUG is not set
CONFIG_SKGE_GENESIS=y
CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set
# CONFIG_OCTEONTX2_AF is not set
# CONFIG_OCTEONTX2_PF is not set
# CONFIG_PRESTERA is not set
# CONFIG_NET_VENDOR_MEDIATEK is not set
CONFIG_NET_VENDOR_MELLANOX=y
CONFIG_MLX4_EN=m
CONFIG_MLX4_EN_DCB=y
CONFIG_MLX4_CORE=m
CONFIG_MLX4_DEBUG=y
CONFIG_MLX4_CORE_GEN2=y
CONFIG_MLX5_CORE=m
CONFIG_MLX5_ACCEL=y
CONFIG_MLX5_FPGA=y
CONFIG_MLX5_CORE_EN=y
CONFIG_MLX5_EN_ARFS=y
CONFIG_MLX5_EN_RXNFC=y
CONFIG_MLX5_MPFS=y
CONFIG_MLX5_ESWITCH=y
CONFIG_MLX5_CLS_ACT=y
CONFIG_MLX5_CORE_EN_DCB=y
CONFIG_MLX5_CORE_IPOIB=y
# CONFIG_MLX5_FPGA_IPSEC is not set
# CONFIG_MLX5_IPSEC is not set
CONFIG_MLX5_SW_STEERING=y
CONFIG_MLXSW_CORE=m
CONFIG_MLXSW_CORE_HWMON=y
CONFIG_MLXSW_CORE_THERMAL=y
CONFIG_MLXSW_PCI=m
CONFIG_MLXSW_I2C=m
CONFIG_MLXSW_SWITCHIB=m
CONFIG_MLXSW_SWITCHX2=m
CONFIG_MLXSW_SPECTRUM=m
CONFIG_MLXSW_SPECTRUM_DCB=y
CONFIG_MLXSW_MINIMAL=m
CONFIG_MLXFW=m
CONFIG_NET_VENDOR_MICREL=y
CONFIG_KS8842=m
CONFIG_KS8851=m
CONFIG_KS8851_MLL=m
CONFIG_KSZ884X_PCI=m
CONFIG_NET_VENDOR_MICROCHIP=y
CONFIG_ENC28J60=m
# CONFIG_ENC28J60_WRITEVERIFY is not set
CONFIG_ENCX24J600=m
# CONFIG_LAN743X is not set
CONFIG_NET_VENDOR_MICROSEMI=y
# CONFIG_MSCC_OCELOT_SWITCH is not set
CONFIG_NET_VENDOR_MYRI=y
CONFIG_MYRI10GE=m
CONFIG_FEALNX=m
CONFIG_NET_VENDOR_NATSEMI=y
CONFIG_NATSEMI=m
CONFIG_NS83820=m
CONFIG_NET_VENDOR_NETERION=y
CONFIG_S2IO=m
CONFIG_VXGE=m
# CONFIG_VXGE_DEBUG_TRACE_ALL is not set
CONFIG_NET_VENDOR_NETRONOME=y
CONFIG_NFP=m
CONFIG_NFP_APP_FLOWER=y
CONFIG_NFP_APP_ABM_NIC=y
# CONFIG_NFP_DEBUG is not set
CONFIG_NET_VENDOR_NI=y
# CONFIG_NI_XGE_MANAGEMENT_ENET is not set
CONFIG_NET_VENDOR_8390=y
CONFIG_NE2K_PCI=m
CONFIG_NET_VENDOR_NVIDIA=y
CONFIG_FORCEDETH=m
CONFIG_NET_VENDOR_OKI=y
CONFIG_ETHOC=m
CONFIG_NET_VENDOR_PACKET_ENGINES=y
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_NET_VENDOR_PENSANDO=y
# CONFIG_IONIC is not set
CONFIG_NET_VENDOR_QLOGIC=y
CONFIG_QLA3XXX=m
CONFIG_QLCNIC=m
CONFIG_QLCNIC_SRIOV=y
CONFIG_QLCNIC_DCB=y
CONFIG_QLCNIC_HWMON=y
CONFIG_NETXEN_NIC=m
CONFIG_QED=m
CONFIG_QED_LL2=y
CONFIG_QED_SRIOV=y
CONFIG_QEDE=m
CONFIG_QED_RDMA=y
CONFIG_QED_ISCSI=y
CONFIG_QED_FCOE=y
CONFIG_QED_OOO=y
CONFIG_NET_VENDOR_QUALCOMM=y
CONFIG_QCA7000=m
CONFIG_QCA7000_SPI=m
CONFIG_QCA7000_UART=m
CONFIG_QCOM_EMAC=m
CONFIG_RMNET=m
CONFIG_NET_VENDOR_RDC=y
CONFIG_R6040=m
CONFIG_NET_VENDOR_REALTEK=y
CONFIG_8139CP=m
CONFIG_8139TOO=m
CONFIG_8139TOO_PIO=y
# CONFIG_8139TOO_TUNE_TWISTER is not set
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_R8169=m
CONFIG_NET_VENDOR_RENESAS=y
CONFIG_SH_ETH=m
CONFIG_RAVB=m
CONFIG_NET_VENDOR_ROCKER=y
CONFIG_ROCKER=m
CONFIG_NET_VENDOR_SAMSUNG=y
CONFIG_SXGBE_ETH=m
CONFIG_NET_VENDOR_SEEQ=y
CONFIG_NET_VENDOR_SOLARFLARE=y
CONFIG_SFC=m
CONFIG_SFC_MTD=y
CONFIG_SFC_MCDI_MON=y
CONFIG_SFC_SRIOV=y
CONFIG_SFC_MCDI_LOGGING=y
CONFIG_SFC_FALCON=m
CONFIG_SFC_FALCON_MTD=y
CONFIG_NET_VENDOR_SILAN=y
CONFIG_SC92031=m
CONFIG_NET_VENDOR_SIS=y
CONFIG_SIS900=m
CONFIG_SIS190=m
CONFIG_NET_VENDOR_SMSC=y
CONFIG_SMC91X=y
CONFIG_EPIC100=m
CONFIG_SMSC911X=m
CONFIG_SMSC9420=m
CONFIG_NET_VENDOR_SOCIONEXT=y
CONFIG_SNI_NETSEC=m
CONFIG_NET_VENDOR_STMICRO=y
CONFIG_STMMAC_ETH=m
# CONFIG_STMMAC_SELFTESTS is not set
CONFIG_STMMAC_PLATFORM=m
CONFIG_DWMAC_DWC_QOS_ETH=m
CONFIG_DWMAC_GENERIC=m
CONFIG_DWMAC_IPQ806X=m
# CONFIG_DWMAC_MEDIATEK is not set
CONFIG_DWMAC_QCOM_ETHQOS=m
CONFIG_DWMAC_SOCFPGA=m
# CONFIG_DWMAC_SUNXI is not set
CONFIG_DWMAC_SUN8I=m
# CONFIG_DWMAC_INTEL_PLAT is not set
# CONFIG_STMMAC_PCI is not set
CONFIG_NET_VENDOR_SUN=y
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
CONFIG_NIU=m
CONFIG_NET_VENDOR_SYNOPSYS=y
CONFIG_DWC_XLGMAC=m
CONFIG_DWC_XLGMAC_PCI=m
CONFIG_NET_VENDOR_TEHUTI=y
CONFIG_TEHUTI=m
CONFIG_NET_VENDOR_TI=y
# CONFIG_TI_CPSW_PHY_SEL is not set
CONFIG_TLAN=m
CONFIG_NET_VENDOR_VIA=y
CONFIG_VIA_RHINE=m
CONFIG_VIA_RHINE_MMIO=y
CONFIG_VIA_VELOCITY=m
CONFIG_NET_VENDOR_WIZNET=y
CONFIG_WIZNET_W5100=m
CONFIG_WIZNET_W5300=m
# CONFIG_WIZNET_BUS_DIRECT is not set
# CONFIG_WIZNET_BUS_INDIRECT is not set
CONFIG_WIZNET_BUS_ANY=y
CONFIG_WIZNET_W5100_SPI=m
CONFIG_NET_VENDOR_XILINX=y
# CONFIG_XILINX_AXI_EMAC is not set
# CONFIG_XILINX_LL_TEMAC is not set
CONFIG_FDDI=y
CONFIG_DEFXX=m
# CONFIG_DEFXX_MMIO is not set
CONFIG_SKFP=m
# CONFIG_HIPPI is not set
CONFIG_NET_SB1000=m
CONFIG_PHYLINK=m
CONFIG_PHYLIB=y
CONFIG_SWPHY=y
CONFIG_LED_TRIGGER_PHY=y
CONFIG_FIXED_PHY=y
# CONFIG_SFP is not set
#
# MII PHY device drivers
#
CONFIG_AMD_PHY=m
# CONFIG_ADIN_PHY is not set
CONFIG_AQUANTIA_PHY=m
# CONFIG_AX88796B_PHY is not set
CONFIG_BROADCOM_PHY=m
# CONFIG_BCM54140_PHY is not set
CONFIG_BCM7XXX_PHY=m
# CONFIG_BCM84881_PHY is not set
CONFIG_BCM87XX_PHY=m
CONFIG_BCM_NET_PHYLIB=m
CONFIG_CICADA_PHY=m
CONFIG_CORTINA_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_LXT_PHY=m
CONFIG_INTEL_XWAY_PHY=m
CONFIG_LSI_ET1011C_PHY=m
CONFIG_MARVELL_PHY=m
CONFIG_MARVELL_10G_PHY=m
CONFIG_MICREL_PHY=m
CONFIG_MICROCHIP_PHY=m
# CONFIG_MICROCHIP_T1_PHY is not set
CONFIG_MICROSEMI_PHY=m
CONFIG_NATIONAL_PHY=m
# CONFIG_NXP_TJA11XX_PHY is not set
CONFIG_AT803X_PHY=m
CONFIG_QSEMI_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_RENESAS_PHY=m
CONFIG_ROCKCHIP_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_STE10XP=m
CONFIG_TERANETICS_PHY=m
CONFIG_DP83822_PHY=m
# CONFIG_DP83TC811_PHY is not set
CONFIG_DP83848_PHY=m
CONFIG_DP83867_PHY=m
# CONFIG_DP83869_PHY is not set
CONFIG_VITESSE_PHY=m
CONFIG_XILINX_GMII2RGMII=m
CONFIG_MICREL_KS8995MA=m
CONFIG_MDIO_DEVICE=y
CONFIG_MDIO_BUS=y
CONFIG_OF_MDIO=y
CONFIG_MDIO_DEVRES=y
# CONFIG_MDIO_SUN4I is not set
CONFIG_MDIO_XGENE=m
CONFIG_MDIO_BITBANG=m
# CONFIG_MDIO_BCM_IPROC is not set
CONFIG_MDIO_BCM_UNIMAC=m
CONFIG_MDIO_CAVIUM=m
CONFIG_MDIO_GPIO=m
CONFIG_MDIO_HISI_FEMAC=m
# CONFIG_MDIO_MVUSB is not set
# CONFIG_MDIO_MSCC_MIIM is not set
CONFIG_MDIO_OCTEON=m
# CONFIG_MDIO_IPQ4019 is not set
# CONFIG_MDIO_IPQ8064 is not set
CONFIG_MDIO_THUNDER=m
#
# MDIO Multiplexers
#
CONFIG_MDIO_BUS_MUX=y
CONFIG_MDIO_BUS_MUX_BCM_IPROC=y
CONFIG_MDIO_BUS_MUX_GPIO=m
# CONFIG_MDIO_BUS_MUX_MULTIPLEXER is not set
CONFIG_MDIO_BUS_MUX_MMIOREG=m
#
# PCS device drivers
#
CONFIG_PCS_XPCS=m
CONFIG_PCS_LYNX=m
# end of PCS device drivers
CONFIG_PLIP=m
CONFIG_PPP=y
CONFIG_PPP_BSDCOMP=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_FILTER=y
CONFIG_PPP_MPPE=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPPOATM=m
CONFIG_PPPOE=m
CONFIG_PPTP=m
CONFIG_PPPOL2TP=m
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_SLIP=m
CONFIG_SLHC=y
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLIP_SMART=y
CONFIG_SLIP_MODE_SLIP6=y
CONFIG_USB_NET_DRIVERS=m
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_RTL8152=m
CONFIG_USB_LAN78XX=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_AX88179_178A=m
CONFIG_USB_NET_CDCETHER=m
CONFIG_USB_NET_CDC_EEM=m
CONFIG_USB_NET_CDC_NCM=m
CONFIG_USB_NET_HUAWEI_CDC_NCM=m
CONFIG_USB_NET_CDC_MBIM=m
CONFIG_USB_NET_DM9601=m
CONFIG_USB_NET_SR9700=m
CONFIG_USB_NET_SR9800=m
CONFIG_USB_NET_SMSC75XX=m
CONFIG_USB_NET_SMSC95XX=m
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
CONFIG_USB_NET_MCS7830=m
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET_ENABLE=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_NET_CX82310_ETH=m
CONFIG_USB_NET_KALMIA=m
CONFIG_USB_NET_QMI_WWAN=m
CONFIG_USB_HSO=m
CONFIG_USB_NET_INT51X1=m
CONFIG_USB_CDC_PHONET=m
CONFIG_USB_IPHETH=m
CONFIG_USB_SIERRA_NET=m
CONFIG_USB_VL600=m
CONFIG_USB_NET_CH9200=m
# CONFIG_USB_NET_AQC111 is not set
CONFIG_USB_RTL8153_ECM=m
CONFIG_WLAN=y
CONFIG_WLAN_VENDOR_ADMTEK=y
CONFIG_ADM8211=m
CONFIG_ATH_COMMON=m
CONFIG_WLAN_VENDOR_ATH=y
# CONFIG_ATH_DEBUG is not set
CONFIG_ATH5K=m
# CONFIG_ATH5K_DEBUG is not set
# CONFIG_ATH5K_TRACER is not set
CONFIG_ATH5K_PCI=y
CONFIG_ATH9K_HW=m
CONFIG_ATH9K_COMMON=m
CONFIG_ATH9K_COMMON_DEBUG=y
CONFIG_ATH9K_BTCOEX_SUPPORT=y
CONFIG_ATH9K=m
CONFIG_ATH9K_PCI=y
CONFIG_ATH9K_AHB=y
CONFIG_ATH9K_DEBUGFS=y
CONFIG_ATH9K_STATION_STATISTICS=y
# CONFIG_ATH9K_DYNACK is not set
CONFIG_ATH9K_WOW=y
CONFIG_ATH9K_RFKILL=y
CONFIG_ATH9K_CHANNEL_CONTEXT=y
CONFIG_ATH9K_PCOEM=y
# CONFIG_ATH9K_PCI_NO_EEPROM is not set
CONFIG_ATH9K_HTC=m
CONFIG_ATH9K_HTC_DEBUGFS=y
CONFIG_ATH9K_HWRNG=y
# CONFIG_ATH9K_COMMON_SPECTRAL is not set
CONFIG_CARL9170=m
CONFIG_CARL9170_LEDS=y
# CONFIG_CARL9170_DEBUGFS is not set
CONFIG_CARL9170_WPC=y
CONFIG_CARL9170_HWRNG=y
CONFIG_ATH6KL=m
CONFIG_ATH6KL_SDIO=m
CONFIG_ATH6KL_USB=m
# CONFIG_ATH6KL_DEBUG is not set
# CONFIG_ATH6KL_TRACING is not set
CONFIG_AR5523=m
CONFIG_WIL6210=m
CONFIG_WIL6210_ISR_COR=y
CONFIG_WIL6210_TRACING=y
CONFIG_WIL6210_DEBUGFS=y
CONFIG_ATH10K=m
CONFIG_ATH10K_CE=y
CONFIG_ATH10K_PCI=m
CONFIG_ATH10K_AHB=y
CONFIG_ATH10K_SDIO=m
CONFIG_ATH10K_USB=m
# CONFIG_ATH10K_SNOC is not set
# CONFIG_ATH10K_DEBUG is not set
CONFIG_ATH10K_DEBUGFS=y
# CONFIG_ATH10K_SPECTRAL is not set
CONFIG_ATH10K_TRACING=y
CONFIG_WCN36XX=m
# CONFIG_WCN36XX_DEBUGFS is not set
# CONFIG_ATH11K is not set
CONFIG_WLAN_VENDOR_ATMEL=y
CONFIG_ATMEL=m
CONFIG_PCI_ATMEL=m
CONFIG_AT76C50X_USB=m
CONFIG_WLAN_VENDOR_BROADCOM=y
CONFIG_B43=m
CONFIG_B43_BCMA=y
CONFIG_B43_SSB=y
CONFIG_B43_BUSES_BCMA_AND_SSB=y
# CONFIG_B43_BUSES_BCMA is not set
# CONFIG_B43_BUSES_SSB is not set
CONFIG_B43_PCI_AUTOSELECT=y
CONFIG_B43_PCICORE_AUTOSELECT=y
# CONFIG_B43_SDIO is not set
CONFIG_B43_BCMA_PIO=y
CONFIG_B43_PIO=y
CONFIG_B43_PHY_G=y
CONFIG_B43_PHY_N=y
CONFIG_B43_PHY_LP=y
CONFIG_B43_PHY_HT=y
CONFIG_B43_LEDS=y
CONFIG_B43_HWRNG=y
# CONFIG_B43_DEBUG is not set
CONFIG_B43LEGACY=m
CONFIG_B43LEGACY_PCI_AUTOSELECT=y
CONFIG_B43LEGACY_PCICORE_AUTOSELECT=y
CONFIG_B43LEGACY_LEDS=y
CONFIG_B43LEGACY_HWRNG=y
# CONFIG_B43LEGACY_DEBUG is not set
CONFIG_B43LEGACY_DMA=y
CONFIG_B43LEGACY_PIO=y
CONFIG_B43LEGACY_DMA_AND_PIO_MODE=y
# CONFIG_B43LEGACY_DMA_MODE is not set
# CONFIG_B43LEGACY_PIO_MODE is not set
CONFIG_BRCMUTIL=m
CONFIG_BRCMSMAC=m
CONFIG_BRCMFMAC=m
CONFIG_BRCMFMAC_PROTO_BCDC=y
CONFIG_BRCMFMAC_PROTO_MSGBUF=y
CONFIG_BRCMFMAC_SDIO=y
CONFIG_BRCMFMAC_USB=y
CONFIG_BRCMFMAC_PCIE=y
CONFIG_BRCM_TRACING=y
# CONFIG_BRCMDBG is not set
CONFIG_WLAN_VENDOR_CISCO=y
CONFIG_WLAN_VENDOR_INTEL=y
CONFIG_IPW2100=m
CONFIG_IPW2100_MONITOR=y
# CONFIG_IPW2100_DEBUG is not set
CONFIG_IPW2200=m
CONFIG_IPW2200_MONITOR=y
CONFIG_IPW2200_RADIOTAP=y
CONFIG_IPW2200_PROMISCUOUS=y
CONFIG_IPW2200_QOS=y
# CONFIG_IPW2200_DEBUG is not set
CONFIG_LIBIPW=m
# CONFIG_LIBIPW_DEBUG is not set
CONFIG_IWLEGACY=m
CONFIG_IWL4965=m
CONFIG_IWL3945=m
#
# iwl3945 / iwl4965 Debugging Options
#
# CONFIG_IWLEGACY_DEBUG is not set
CONFIG_IWLEGACY_DEBUGFS=y
# end of iwl3945 / iwl4965 Debugging Options
CONFIG_IWLWIFI=m
CONFIG_IWLWIFI_LEDS=y
CONFIG_IWLDVM=m
CONFIG_IWLMVM=m
CONFIG_IWLWIFI_OPMODE_MODULAR=y
# CONFIG_IWLWIFI_BCAST_FILTERING is not set
#
# Debugging Options
#
# CONFIG_IWLWIFI_DEBUG is not set
CONFIG_IWLWIFI_DEBUGFS=y
CONFIG_IWLWIFI_DEVICE_TRACING=y
# end of Debugging Options
CONFIG_WLAN_VENDOR_INTERSIL=y
CONFIG_HOSTAP=m
CONFIG_HOSTAP_FIRMWARE=y
CONFIG_HOSTAP_FIRMWARE_NVRAM=y
CONFIG_HOSTAP_PLX=m
CONFIG_HOSTAP_PCI=m
CONFIG_HERMES=m
# CONFIG_HERMES_PRISM is not set
CONFIG_HERMES_CACHE_FW_ON_INIT=y
CONFIG_PLX_HERMES=m
CONFIG_TMD_HERMES=m
CONFIG_NORTEL_HERMES=m
CONFIG_ORINOCO_USB=m
CONFIG_P54_COMMON=m
CONFIG_P54_USB=m
CONFIG_P54_PCI=m
CONFIG_P54_SPI=m
# CONFIG_P54_SPI_DEFAULT_EEPROM is not set
CONFIG_P54_LEDS=y
# CONFIG_PRISM54 is not set
CONFIG_WLAN_VENDOR_MARVELL=y
CONFIG_LIBERTAS=m
CONFIG_LIBERTAS_USB=m
CONFIG_LIBERTAS_SDIO=m
CONFIG_LIBERTAS_SPI=m
# CONFIG_LIBERTAS_DEBUG is not set
CONFIG_LIBERTAS_MESH=y
CONFIG_LIBERTAS_THINFIRM=m
# CONFIG_LIBERTAS_THINFIRM_DEBUG is not set
CONFIG_LIBERTAS_THINFIRM_USB=m
CONFIG_MWIFIEX=m
CONFIG_MWIFIEX_SDIO=m
CONFIG_MWIFIEX_PCIE=m
CONFIG_MWIFIEX_USB=m
CONFIG_MWL8K=m
CONFIG_WLAN_VENDOR_MEDIATEK=y
CONFIG_MT7601U=m
# CONFIG_MT76x0U is not set
# CONFIG_MT76x0E is not set
# CONFIG_MT76x2E is not set
# CONFIG_MT76x2U is not set
# CONFIG_MT7603E is not set
# CONFIG_MT7615E is not set
# CONFIG_MT7663U is not set
# CONFIG_MT7663S is not set
# CONFIG_MT7915E is not set
CONFIG_WLAN_VENDOR_MICROCHIP=y
CONFIG_WILC1000=m
CONFIG_WILC1000_SDIO=m
CONFIG_WILC1000_SPI=m
CONFIG_WILC1000_HW_OOB_INTR=y
CONFIG_WLAN_VENDOR_RALINK=y
CONFIG_RT2X00=m
CONFIG_RT2400PCI=m
CONFIG_RT2500PCI=m
CONFIG_RT61PCI=m
CONFIG_RT2800PCI=m
CONFIG_RT2800PCI_RT33XX=y
CONFIG_RT2800PCI_RT35XX=y
CONFIG_RT2800PCI_RT53XX=y
CONFIG_RT2800PCI_RT3290=y
CONFIG_RT2500USB=m
CONFIG_RT73USB=m
CONFIG_RT2800USB=m
CONFIG_RT2800USB_RT33XX=y
CONFIG_RT2800USB_RT35XX=y
CONFIG_RT2800USB_RT3573=y
CONFIG_RT2800USB_RT53XX=y
CONFIG_RT2800USB_RT55XX=y
CONFIG_RT2800USB_UNKNOWN=y
CONFIG_RT2800_LIB=m
CONFIG_RT2800_LIB_MMIO=m
CONFIG_RT2X00_LIB_MMIO=m
CONFIG_RT2X00_LIB_PCI=m
CONFIG_RT2X00_LIB_USB=m
CONFIG_RT2X00_LIB=m
CONFIG_RT2X00_LIB_FIRMWARE=y
CONFIG_RT2X00_LIB_CRYPTO=y
CONFIG_RT2X00_LIB_LEDS=y
# CONFIG_RT2X00_LIB_DEBUGFS is not set
# CONFIG_RT2X00_DEBUG is not set
CONFIG_WLAN_VENDOR_REALTEK=y
CONFIG_RTL8180=m
CONFIG_RTL8187=m
CONFIG_RTL8187_LEDS=y
CONFIG_RTL_CARDS=m
CONFIG_RTL8192CE=m
CONFIG_RTL8192SE=m
CONFIG_RTL8192DE=m
CONFIG_RTL8723AE=m
CONFIG_RTL8723BE=m
CONFIG_RTL8188EE=m
CONFIG_RTL8192EE=m
CONFIG_RTL8821AE=m
CONFIG_RTL8192CU=m
CONFIG_RTLWIFI=m
CONFIG_RTLWIFI_PCI=m
CONFIG_RTLWIFI_USB=m
# CONFIG_RTLWIFI_DEBUG is not set
CONFIG_RTL8192C_COMMON=m
CONFIG_RTL8723_COMMON=m
CONFIG_RTLBTCOEXIST=m
CONFIG_RTL8XXXU=m
CONFIG_RTL8XXXU_UNTESTED=y
# CONFIG_RTW88 is not set
CONFIG_WLAN_VENDOR_RSI=y
CONFIG_RSI_91X=m
# CONFIG_RSI_DEBUGFS is not set
CONFIG_RSI_SDIO=m
CONFIG_RSI_USB=m
CONFIG_RSI_COEX=y
CONFIG_WLAN_VENDOR_ST=y
CONFIG_CW1200=m
CONFIG_CW1200_WLAN_SDIO=m
CONFIG_CW1200_WLAN_SPI=m
CONFIG_WLAN_VENDOR_TI=y
CONFIG_WL1251=m
CONFIG_WL1251_SPI=m
CONFIG_WL1251_SDIO=m
CONFIG_WL12XX=m
CONFIG_WL18XX=m
CONFIG_WLCORE=m
CONFIG_WLCORE_SPI=m
CONFIG_WLCORE_SDIO=m
CONFIG_WILINK_PLATFORM_DATA=y
CONFIG_WLAN_VENDOR_ZYDAS=y
CONFIG_USB_ZD1201=m
CONFIG_ZD1211RW=m
# CONFIG_ZD1211RW_DEBUG is not set
CONFIG_WLAN_VENDOR_QUANTENNA=y
# CONFIG_QTNFMAC_PCIE is not set
CONFIG_MAC80211_HWSIM=m
CONFIG_USB_NET_RNDIS_WLAN=m
# CONFIG_VIRT_WIFI is not set
CONFIG_WAN=y
CONFIG_HDLC=m
CONFIG_HDLC_RAW=m
CONFIG_HDLC_RAW_ETH=m
CONFIG_HDLC_CISCO=m
CONFIG_HDLC_FR=m
CONFIG_HDLC_PPP=m
CONFIG_HDLC_X25=m
CONFIG_PCI200SYN=m
CONFIG_WANXL=m
CONFIG_PC300TOO=m
CONFIG_FARSYNC=m
CONFIG_SLIC_DS26522=m
CONFIG_LAPBETHER=m
CONFIG_IEEE802154_DRIVERS=m
CONFIG_IEEE802154_FAKELB=m
CONFIG_IEEE802154_AT86RF230=m
CONFIG_IEEE802154_AT86RF230_DEBUGFS=y
CONFIG_IEEE802154_MRF24J40=m
CONFIG_IEEE802154_CC2520=m
CONFIG_IEEE802154_ATUSB=m
CONFIG_IEEE802154_ADF7242=m
CONFIG_IEEE802154_CA8210=m
CONFIG_IEEE802154_CA8210_DEBUGFS=y
# CONFIG_IEEE802154_MCR20A is not set
# CONFIG_IEEE802154_HWSIM is not set
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_NETDEV_BACKEND=m
CONFIG_VMXNET3=m
CONFIG_FUJITSU_ES=m
# CONFIG_NETDEVSIM is not set
CONFIG_NET_FAILOVER=y
CONFIG_ISDN=y
CONFIG_ISDN_CAPI=y
CONFIG_CAPI_TRACE=y
CONFIG_ISDN_CAPI_MIDDLEWARE=y
CONFIG_MISDN=m
CONFIG_MISDN_DSP=m
CONFIG_MISDN_L1OIP=m
#
# mISDN hardware drivers
#
CONFIG_MISDN_HFCPCI=m
CONFIG_MISDN_HFCMULTI=m
CONFIG_MISDN_HFCUSB=m
CONFIG_MISDN_AVMFRITZ=m
CONFIG_MISDN_SPEEDFAX=m
CONFIG_MISDN_INFINEON=m
CONFIG_MISDN_W6692=m
CONFIG_MISDN_NETJET=m
CONFIG_MISDN_HDLC=m
CONFIG_MISDN_IPAC=m
CONFIG_MISDN_ISAR=m
CONFIG_NVM=y
CONFIG_NVM_PBLK=m
# CONFIG_NVM_PBLK_DEBUG is not set
#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_LEDS=m
CONFIG_INPUT_FF_MEMLESS=m
CONFIG_INPUT_SPARSEKMAP=m
CONFIG_INPUT_MATRIXKMAP=m
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_EVDEV=y
CONFIG_INPUT_EVBUG=m
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ADC=m
CONFIG_KEYBOARD_ADP5520=m
CONFIG_KEYBOARD_ADP5588=m
CONFIG_KEYBOARD_ADP5589=m
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1050 is not set
CONFIG_KEYBOARD_QT1070=m
CONFIG_KEYBOARD_QT2160=m
CONFIG_KEYBOARD_DLINK_DIR685=m
CONFIG_KEYBOARD_LKKBD=m
CONFIG_KEYBOARD_GPIO=m
CONFIG_KEYBOARD_GPIO_POLLED=m
CONFIG_KEYBOARD_TCA6416=m
CONFIG_KEYBOARD_TCA8418=m
CONFIG_KEYBOARD_MATRIX=m
CONFIG_KEYBOARD_LM8323=m
CONFIG_KEYBOARD_LM8333=m
CONFIG_KEYBOARD_MAX7359=m
CONFIG_KEYBOARD_MCS=m
CONFIG_KEYBOARD_MPR121=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_KEYBOARD_OPENCORES=m
CONFIG_KEYBOARD_SAMSUNG=m
CONFIG_KEYBOARD_STOWAWAY=m
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_STMPE=m
# CONFIG_KEYBOARD_SUN4I_LRADC is not set
CONFIG_KEYBOARD_OMAP4=m
CONFIG_KEYBOARD_TC3589X=m
CONFIG_KEYBOARD_TM2_TOUCHKEY=m
CONFIG_KEYBOARD_TWL4030=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_KEYBOARD_CROS_EC=m
CONFIG_KEYBOARD_CAP11XX=m
CONFIG_KEYBOARD_BCM=m
# CONFIG_KEYBOARD_MTK_PMIC is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_BYD=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_SYNAPTICS_SMBUS=y
CONFIG_MOUSE_PS2_CYPRESS=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_MOUSE_PS2_ELANTECH_SMBUS=y
CONFIG_MOUSE_PS2_SENTELIC=y
CONFIG_MOUSE_PS2_TOUCHKIT=y
CONFIG_MOUSE_PS2_FOCALTECH=y
CONFIG_MOUSE_PS2_SMBUS=y
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_APPLETOUCH=m
CONFIG_MOUSE_BCM5974=m
CONFIG_MOUSE_CYAPA=m
CONFIG_MOUSE_ELAN_I2C=m
CONFIG_MOUSE_ELAN_I2C_I2C=y
CONFIG_MOUSE_ELAN_I2C_SMBUS=y
CONFIG_MOUSE_VSXXXAA=m
CONFIG_MOUSE_GPIO=m
CONFIG_MOUSE_SYNAPTICS_I2C=m
CONFIG_MOUSE_SYNAPTICS_USB=m
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
CONFIG_JOYSTICK_A3D=m
# CONFIG_JOYSTICK_ADC is not set
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_USB=m
CONFIG_JOYSTICK_IFORCE_232=m
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
CONFIG_JOYSTICK_TWIDJOY=m
CONFIG_JOYSTICK_ZHENHUA=m
CONFIG_JOYSTICK_DB9=m
CONFIG_JOYSTICK_GAMECON=m
CONFIG_JOYSTICK_TURBOGRAFX=m
CONFIG_JOYSTICK_AS5011=m
CONFIG_JOYSTICK_JOYDUMP=m
CONFIG_JOYSTICK_XPAD=m
CONFIG_JOYSTICK_XPAD_FF=y
CONFIG_JOYSTICK_XPAD_LEDS=y
CONFIG_JOYSTICK_WALKERA0701=m
CONFIG_JOYSTICK_PSXPAD_SPI=m
CONFIG_JOYSTICK_PSXPAD_SPI_FF=y
# CONFIG_JOYSTICK_PXRC is not set
# CONFIG_JOYSTICK_FSIA6B is not set
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=m
CONFIG_TABLET_USB_AIPTEK=m
CONFIG_TABLET_USB_HANWANG=m
CONFIG_TABLET_USB_KBTAB=m
CONFIG_TABLET_USB_PEGASUS=m
CONFIG_TABLET_SERIAL_WACOM4=m
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_PROPERTIES=y
CONFIG_TOUCHSCREEN_88PM860X=m
CONFIG_TOUCHSCREEN_ADS7846=m
CONFIG_TOUCHSCREEN_AD7877=m
CONFIG_TOUCHSCREEN_AD7879=m
CONFIG_TOUCHSCREEN_AD7879_I2C=m
CONFIG_TOUCHSCREEN_AD7879_SPI=m
# CONFIG_TOUCHSCREEN_ADC is not set
CONFIG_TOUCHSCREEN_AR1021_I2C=m
CONFIG_TOUCHSCREEN_ATMEL_MXT=m
CONFIG_TOUCHSCREEN_ATMEL_MXT_T37=y
CONFIG_TOUCHSCREEN_AUO_PIXCIR=m
CONFIG_TOUCHSCREEN_BU21013=m
# CONFIG_TOUCHSCREEN_BU21029 is not set
CONFIG_TOUCHSCREEN_CHIPONE_ICN8318=m
# CONFIG_TOUCHSCREEN_CHIPONE_ICN8505 is not set
# CONFIG_TOUCHSCREEN_CY8CTMA140 is not set
CONFIG_TOUCHSCREEN_CY8CTMG110=m
CONFIG_TOUCHSCREEN_CYTTSP_CORE=m
CONFIG_TOUCHSCREEN_CYTTSP_I2C=m
CONFIG_TOUCHSCREEN_CYTTSP_SPI=m
CONFIG_TOUCHSCREEN_CYTTSP4_CORE=m
CONFIG_TOUCHSCREEN_CYTTSP4_I2C=m
CONFIG_TOUCHSCREEN_CYTTSP4_SPI=m
CONFIG_TOUCHSCREEN_DA9034=m
CONFIG_TOUCHSCREEN_DA9052=m
CONFIG_TOUCHSCREEN_DYNAPRO=m
CONFIG_TOUCHSCREEN_HAMPSHIRE=m
CONFIG_TOUCHSCREEN_EETI=m
CONFIG_TOUCHSCREEN_EGALAX=m
CONFIG_TOUCHSCREEN_EGALAX_SERIAL=m
CONFIG_TOUCHSCREEN_EXC3000=m
CONFIG_TOUCHSCREEN_FUJITSU=m
CONFIG_TOUCHSCREEN_GOODIX=m
CONFIG_TOUCHSCREEN_HIDEEP=m
CONFIG_TOUCHSCREEN_ILI210X=m
CONFIG_TOUCHSCREEN_IPROC=m
CONFIG_TOUCHSCREEN_S6SY761=m
CONFIG_TOUCHSCREEN_GUNZE=m
CONFIG_TOUCHSCREEN_EKTF2127=m
CONFIG_TOUCHSCREEN_ELAN=m
CONFIG_TOUCHSCREEN_ELO=m
CONFIG_TOUCHSCREEN_WACOM_W8001=m
CONFIG_TOUCHSCREEN_WACOM_I2C=m
CONFIG_TOUCHSCREEN_MAX11801=m
CONFIG_TOUCHSCREEN_MCS5000=m
CONFIG_TOUCHSCREEN_MMS114=m
CONFIG_TOUCHSCREEN_MELFAS_MIP4=m
CONFIG_TOUCHSCREEN_MTOUCH=m
CONFIG_TOUCHSCREEN_IMX6UL_TSC=m
CONFIG_TOUCHSCREEN_INEXIO=m
CONFIG_TOUCHSCREEN_MK712=m
CONFIG_TOUCHSCREEN_PENMOUNT=m
CONFIG_TOUCHSCREEN_EDT_FT5X06=m
# CONFIG_TOUCHSCREEN_RASPBERRYPI_FW is not set
CONFIG_TOUCHSCREEN_TOUCHRIGHT=m
CONFIG_TOUCHSCREEN_TOUCHWIN=m
CONFIG_TOUCHSCREEN_TI_AM335X_TSC=m
CONFIG_TOUCHSCREEN_UCB1400=m
CONFIG_TOUCHSCREEN_PIXCIR=m
CONFIG_TOUCHSCREEN_WDT87XX_I2C=m
CONFIG_TOUCHSCREEN_WM831X=m
CONFIG_TOUCHSCREEN_WM97XX=m
CONFIG_TOUCHSCREEN_WM9705=y
CONFIG_TOUCHSCREEN_WM9712=y
CONFIG_TOUCHSCREEN_WM9713=y
CONFIG_TOUCHSCREEN_USB_COMPOSITE=m
CONFIG_TOUCHSCREEN_MC13783=m
CONFIG_TOUCHSCREEN_USB_EGALAX=y
CONFIG_TOUCHSCREEN_USB_PANJIT=y
CONFIG_TOUCHSCREEN_USB_3M=y
CONFIG_TOUCHSCREEN_USB_ITM=y
CONFIG_TOUCHSCREEN_USB_ETURBO=y
CONFIG_TOUCHSCREEN_USB_GUNZE=y
CONFIG_TOUCHSCREEN_USB_DMC_TSC10=y
CONFIG_TOUCHSCREEN_USB_IRTOUCH=y
CONFIG_TOUCHSCREEN_USB_IDEALTEK=y
CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y
CONFIG_TOUCHSCREEN_USB_GOTOP=y
CONFIG_TOUCHSCREEN_USB_JASTEC=y
CONFIG_TOUCHSCREEN_USB_ELO=y
CONFIG_TOUCHSCREEN_USB_E2I=y
CONFIG_TOUCHSCREEN_USB_ZYTRONIC=y
CONFIG_TOUCHSCREEN_USB_ETT_TC45USB=y
CONFIG_TOUCHSCREEN_USB_NEXIO=y
CONFIG_TOUCHSCREEN_USB_EASYTOUCH=y
CONFIG_TOUCHSCREEN_TOUCHIT213=m
CONFIG_TOUCHSCREEN_TSC_SERIO=m
CONFIG_TOUCHSCREEN_TSC200X_CORE=m
CONFIG_TOUCHSCREEN_TSC2004=m
CONFIG_TOUCHSCREEN_TSC2005=m
CONFIG_TOUCHSCREEN_TSC2007=m
CONFIG_TOUCHSCREEN_TSC2007_IIO=y
CONFIG_TOUCHSCREEN_PCAP=m
CONFIG_TOUCHSCREEN_RM_TS=m
CONFIG_TOUCHSCREEN_SILEAD=m
CONFIG_TOUCHSCREEN_SIS_I2C=m
CONFIG_TOUCHSCREEN_ST1232=m
CONFIG_TOUCHSCREEN_STMFTS=m
CONFIG_TOUCHSCREEN_STMPE=m
# CONFIG_TOUCHSCREEN_SUN4I is not set
CONFIG_TOUCHSCREEN_SUR40=m
CONFIG_TOUCHSCREEN_SURFACE3_SPI=m
CONFIG_TOUCHSCREEN_SX8654=m
CONFIG_TOUCHSCREEN_TPS6507X=m
CONFIG_TOUCHSCREEN_ZET6223=m
CONFIG_TOUCHSCREEN_ZFORCE=m
CONFIG_TOUCHSCREEN_COLIBRI_VF50=m
CONFIG_TOUCHSCREEN_ROHM_BU21023=m
# CONFIG_TOUCHSCREEN_IQS5XX is not set
# CONFIG_TOUCHSCREEN_ZINITIX is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_88PM860X_ONKEY=m
CONFIG_INPUT_88PM80X_ONKEY=m
CONFIG_INPUT_AD714X=m
CONFIG_INPUT_AD714X_I2C=m
CONFIG_INPUT_AD714X_SPI=m
CONFIG_INPUT_ARIZONA_HAPTICS=m
CONFIG_INPUT_ATMEL_CAPTOUCH=m
CONFIG_INPUT_BMA150=m
CONFIG_INPUT_E3X0_BUTTON=m
CONFIG_INPUT_PM8941_PWRKEY=m
CONFIG_INPUT_PM8XXX_VIBRATOR=m
CONFIG_INPUT_MAX77693_HAPTIC=m
CONFIG_INPUT_MAX8925_ONKEY=m
CONFIG_INPUT_MAX8997_HAPTIC=m
CONFIG_INPUT_MC13783_PWRBUTTON=m
CONFIG_INPUT_MMA8450=m
CONFIG_INPUT_GPIO_BEEPER=m
CONFIG_INPUT_GPIO_DECODER=m
# CONFIG_INPUT_GPIO_VIBRA is not set
CONFIG_INPUT_CPCAP_PWRBUTTON=m
CONFIG_INPUT_ATI_REMOTE2=m
CONFIG_INPUT_KEYSPAN_REMOTE=m
CONFIG_INPUT_KXTJ9=m
CONFIG_INPUT_POWERMATE=m
CONFIG_INPUT_YEALINK=m
CONFIG_INPUT_CM109=m
CONFIG_INPUT_REGULATOR_HAPTIC=m
CONFIG_INPUT_RETU_PWRBUTTON=m
CONFIG_INPUT_TPS65218_PWRBUTTON=m
CONFIG_INPUT_AXP20X_PEK=m
CONFIG_INPUT_TWL4030_PWRBUTTON=m
CONFIG_INPUT_TWL4030_VIBRA=m
CONFIG_INPUT_TWL6040_VIBRA=m
CONFIG_INPUT_UINPUT=y
CONFIG_INPUT_PALMAS_PWRBUTTON=m
CONFIG_INPUT_PCF50633_PMU=m
CONFIG_INPUT_PCF8574=m
CONFIG_INPUT_PWM_BEEPER=m
CONFIG_INPUT_PWM_VIBRA=m
CONFIG_INPUT_RK805_PWRKEY=m
CONFIG_INPUT_GPIO_ROTARY_ENCODER=m
# CONFIG_INPUT_DA7280_HAPTICS is not set
CONFIG_INPUT_DA9052_ONKEY=m
CONFIG_INPUT_DA9055_ONKEY=m
CONFIG_INPUT_DA9063_ONKEY=m
CONFIG_INPUT_WM831X_ON=m
CONFIG_INPUT_PCAP=m
CONFIG_INPUT_ADXL34X=m
CONFIG_INPUT_ADXL34X_I2C=m
CONFIG_INPUT_ADXL34X_SPI=m
CONFIG_INPUT_IMS_PCU=m
# CONFIG_INPUT_IQS269A is not set
CONFIG_INPUT_CMA3000=m
CONFIG_INPUT_CMA3000_I2C=m
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=m
CONFIG_INPUT_SOC_BUTTON_ARRAY=m
CONFIG_INPUT_DRV260X_HAPTICS=m
CONFIG_INPUT_DRV2665_HAPTICS=m
CONFIG_INPUT_DRV2667_HAPTICS=m
CONFIG_INPUT_HISI_POWERKEY=m
# CONFIG_INPUT_SC27XX_VIBRA is not set
CONFIG_RMI4_CORE=m
CONFIG_RMI4_I2C=m
CONFIG_RMI4_SPI=m
CONFIG_RMI4_SMB=m
CONFIG_RMI4_F03=y
CONFIG_RMI4_F03_SERIO=m
CONFIG_RMI4_2D_SENSOR=y
CONFIG_RMI4_F11=y
CONFIG_RMI4_F12=y
CONFIG_RMI4_F30=y
CONFIG_RMI4_F34=y
# CONFIG_RMI4_F3A is not set
CONFIG_RMI4_F54=y
CONFIG_RMI4_F55=y
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_PARKBD=m
CONFIG_SERIO_AMBAKMI=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
CONFIG_SERIO_ALTERA_PS2=m
CONFIG_SERIO_PS2MULT=m
CONFIG_SERIO_ARC_PS2=m
CONFIG_SERIO_APBPS2=m
# CONFIG_SERIO_SUN4I_PS2 is not set
CONFIG_SERIO_GPIO_PS2=m
CONFIG_USERIO=m
CONFIG_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_FM801=m
# end of Hardware I/O ports
# end of Input device support
#
# Character devices
#
CONFIG_TTY=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_VT_CONSOLE_SLEEP=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=0
CONFIG_LDISC_AUTOLOAD=y
#
# Serial drivers
#
CONFIG_SERIAL_EARLYCON=y
CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set
CONFIG_SERIAL_8250_PNP=y
# CONFIG_SERIAL_8250_16550A_VARIANTS is not set
CONFIG_SERIAL_8250_FINTEK=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_DMA=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_EXAR=m
CONFIG_SERIAL_8250_MEN_MCB=m
CONFIG_SERIAL_8250_NR_UARTS=48
CONFIG_SERIAL_8250_RUNTIME_UARTS=32
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_ASPEED_VUART=m
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_RSA=y
CONFIG_SERIAL_8250_DWLIB=y
# CONFIG_SERIAL_8250_BCM2835AUX is not set
CONFIG_SERIAL_8250_FSL=y
CONFIG_SERIAL_8250_DW=y
CONFIG_SERIAL_8250_RT288X=y
CONFIG_SERIAL_8250_MT6577=y
CONFIG_SERIAL_OF_PLATFORM=y
#
# Non-8250 serial port support
#
CONFIG_SERIAL_AMBA_PL010=m
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_SERIAL_EARLYCON_ARM_SEMIHOST=y
CONFIG_SERIAL_KGDB_NMI=y
CONFIG_SERIAL_MAX3100=m
CONFIG_SERIAL_MAX310X=y
CONFIG_SERIAL_UARTLITE=m
CONFIG_SERIAL_UARTLITE_NR_UARTS=1
CONFIG_SERIAL_SH_SCI=m
CONFIG_SERIAL_SH_SCI_NR_UARTS=2
CONFIG_SERIAL_SH_SCI_DMA=y
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_CONSOLE_POLL=y
CONFIG_SERIAL_JSM=m
CONFIG_SERIAL_MSM=y
CONFIG_SERIAL_MSM_CONSOLE=y
# CONFIG_SERIAL_SIFIVE is not set
CONFIG_SERIAL_SCCNXP=y
CONFIG_SERIAL_SCCNXP_CONSOLE=y
CONFIG_SERIAL_SC16IS7XX_CORE=m
CONFIG_SERIAL_SC16IS7XX=m
CONFIG_SERIAL_SC16IS7XX_I2C=y
CONFIG_SERIAL_SC16IS7XX_SPI=y
# CONFIG_SERIAL_BCM63XX is not set
CONFIG_SERIAL_ALTERA_JTAGUART=m
CONFIG_SERIAL_ALTERA_UART=m
CONFIG_SERIAL_ALTERA_UART_MAXPORTS=4
CONFIG_SERIAL_ALTERA_UART_BAUDRATE=115200
# CONFIG_SERIAL_IFX6X60 is not set
CONFIG_SERIAL_XILINX_PS_UART=m
CONFIG_SERIAL_ARC=m
CONFIG_SERIAL_ARC_NR_PORTS=1
CONFIG_SERIAL_RP2=m
CONFIG_SERIAL_RP2_NR_UARTS=32
CONFIG_SERIAL_FSL_LPUART=m
# CONFIG_SERIAL_FSL_LINFLEXUART is not set
CONFIG_SERIAL_CONEXANT_DIGICOLOR=m
CONFIG_SERIAL_MEN_Z135=m
CONFIG_SERIAL_SPRD=m
CONFIG_SERIAL_OWL=y
CONFIG_SERIAL_OWL_CONSOLE=y
# end of Serial drivers
CONFIG_SERIAL_MCTRL_GPIO=y
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_ROCKETPORT=m
CONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
CONFIG_MOXA_INTELLIO=m
CONFIG_MOXA_SMARTIO=m
CONFIG_SYNCLINK_GT=m
CONFIG_ISI=m
CONFIG_N_HDLC=m
CONFIG_N_GSM=m
CONFIG_NOZOMI=m
CONFIG_TRACE_ROUTER=m
CONFIG_TRACE_SINK=m
CONFIG_HVC_DRIVER=y
CONFIG_HVC_IRQ=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_HVC_DCC is not set
CONFIG_SERIAL_DEV_BUS=y
CONFIG_SERIAL_DEV_CTRL_TTYPORT=y
CONFIG_TTY_PRINTK=y
CONFIG_TTY_PRINTK_LEVEL=6
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
CONFIG_PPDEV=m
CONFIG_VIRTIO_CONSOLE=y
CONFIG_IPMI_HANDLER=m
CONFIG_IPMI_DMI_DECODE=y
CONFIG_IPMI_PLAT_DATA=y
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_SSIF=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
# CONFIG_IPMB_DEVICE_INTERFACE is not set
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TIMERIOMEM=m
# CONFIG_HW_RANDOM_BA431 is not set
CONFIG_HW_RANDOM_BCM2835=m
CONFIG_HW_RANDOM_IPROC_RNG200=m
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_HW_RANDOM_HISI=m
CONFIG_HW_RANDOM_XGENE=m
CONFIG_HW_RANDOM_CAVIUM=m
CONFIG_HW_RANDOM_MTK=m
CONFIG_HW_RANDOM_OPTEE=m
# CONFIG_HW_RANDOM_CCTRNG is not set
# CONFIG_HW_RANDOM_XIPHERA is not set
CONFIG_APPLICOM=m
CONFIG_DEVMEM=y
CONFIG_RAW_DRIVER=m
CONFIG_MAX_RAW_DEVS=256
CONFIG_DEVPORT=y
CONFIG_TCG_TPM=y
CONFIG_HW_RANDOM_TPM=y
CONFIG_TCG_TIS_CORE=y
CONFIG_TCG_TIS=y
CONFIG_TCG_TIS_SPI=m
# CONFIG_TCG_TIS_SPI_CR50 is not set
# CONFIG_TCG_TIS_SYNQUACER is not set
CONFIG_TCG_TIS_I2C_ATMEL=m
CONFIG_TCG_TIS_I2C_INFINEON=m
CONFIG_TCG_TIS_I2C_NUVOTON=m
CONFIG_TCG_ATMEL=m
CONFIG_TCG_INFINEON=m
CONFIG_TCG_XEN=m
CONFIG_TCG_CRB=y
CONFIG_TCG_VTPM_PROXY=m
# CONFIG_TCG_FTPM_TEE is not set
CONFIG_TCG_TIS_ST33ZP24=m
CONFIG_TCG_TIS_ST33ZP24_I2C=m
CONFIG_TCG_TIS_ST33ZP24_SPI=m
CONFIG_XILLYBUS=m
CONFIG_XILLYBUS_PCIE=m
CONFIG_XILLYBUS_OF=m
# end of Character devices
# CONFIG_RANDOM_TRUST_CPU is not set
# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
#
# I2C support
#
CONFIG_I2C=y
CONFIG_ACPI_I2C_OPREGION=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_MUX=m
#
# Multiplexer I2C Chip support
#
CONFIG_I2C_ARB_GPIO_CHALLENGE=m
CONFIG_I2C_MUX_GPIO=m
CONFIG_I2C_MUX_GPMUX=m
CONFIG_I2C_MUX_LTC4306=m
CONFIG_I2C_MUX_PCA9541=m
CONFIG_I2C_MUX_PCA954x=m
CONFIG_I2C_MUX_PINCTRL=m
CONFIG_I2C_MUX_REG=m
CONFIG_I2C_DEMUX_PINCTRL=m
CONFIG_I2C_MUX_MLXCPLD=m
# end of Multiplexer I2C Chip support
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCA=m
#
# I2C Hardware Bus support
#
#
# PC SMBus host controller drivers
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD8111=m
# CONFIG_I2C_AMD_MP2 is not set
CONFIG_I2C_HIX5HD2=m
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
# CONFIG_I2C_NVIDIA_GPU is not set
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
#
# ACPI drivers
#
CONFIG_I2C_SCMI=m
#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
CONFIG_I2C_BCM2835=m
CONFIG_I2C_BCM_IPROC=m
CONFIG_I2C_BRCMSTB=m
# CONFIG_I2C_CADENCE is not set
CONFIG_I2C_CBUS_GPIO=m
CONFIG_I2C_DESIGNWARE_CORE=y
# CONFIG_I2C_DESIGNWARE_SLAVE is not set
CONFIG_I2C_DESIGNWARE_PLATFORM=y
CONFIG_I2C_DESIGNWARE_PCI=m
# CONFIG_I2C_EMEV2 is not set
CONFIG_I2C_GPIO=m
# CONFIG_I2C_GPIO_FAULT_INJECTOR is not set
CONFIG_I2C_IMX=m
CONFIG_I2C_KEMPLD=m
CONFIG_I2C_MT65XX=m
CONFIG_I2C_MV64XXX=m
CONFIG_I2C_NOMADIK=m
CONFIG_I2C_OCORES=m
# CONFIG_I2C_OWL is not set
CONFIG_I2C_PCA_PLATFORM=m
# CONFIG_I2C_QCOM_CCI is not set
CONFIG_I2C_QUP=m
CONFIG_I2C_RIIC=m
CONFIG_I2C_RK3X=m
CONFIG_I2C_SH_MOBILE=m
CONFIG_I2C_SIMTEC=m
CONFIG_I2C_SPRD=y
# CONFIG_I2C_SYNQUACER is not set
CONFIG_I2C_VERSATILE=m
CONFIG_I2C_THUNDERX=m
CONFIG_I2C_XILINX=m
CONFIG_I2C_XLP9XX=m
CONFIG_I2C_RCAR=m
#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_DIOLAN_U2C=m
CONFIG_I2C_DLN2=m
CONFIG_I2C_PARPORT=m
CONFIG_I2C_ROBOTFUZZ_OSIF=m
CONFIG_I2C_TAOS_EVM=m
CONFIG_I2C_TINY_USB=m
CONFIG_I2C_VIPERBOARD=m
#
# Other I2C/SMBus bus drivers
#
CONFIG_I2C_CROS_EC_TUNNEL=m
CONFIG_I2C_XGENE_SLIMPRO=m
# CONFIG_I2C_FSI is not set
# end of I2C Hardware Bus support
CONFIG_I2C_STUB=m
CONFIG_I2C_SLAVE=y
CONFIG_I2C_SLAVE_EEPROM=m
# CONFIG_I2C_SLAVE_TESTUNIT is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# end of I2C support
# CONFIG_I3C is not set
CONFIG_SPI=y
# CONFIG_SPI_DEBUG is not set
CONFIG_SPI_MASTER=y
CONFIG_SPI_MEM=y
#
# SPI Master Controller Drivers
#
CONFIG_SPI_ALTERA=m
CONFIG_SPI_AXI_SPI_ENGINE=m
CONFIG_SPI_BCM2835=m
CONFIG_SPI_BCM2835AUX=m
CONFIG_SPI_BCM_QSPI=m
CONFIG_SPI_BITBANG=m
CONFIG_SPI_BUTTERFLY=m
CONFIG_SPI_CADENCE=m
CONFIG_SPI_CADENCE_QUADSPI=m
CONFIG_SPI_DESIGNWARE=m
# CONFIG_SPI_DW_DMA is not set
CONFIG_SPI_DW_PCI=m
CONFIG_SPI_DW_MMIO=m
CONFIG_SPI_DLN2=m
# CONFIG_SPI_FSI is not set
CONFIG_SPI_FSL_QUADSPI=m
# CONFIG_SPI_HISI_SFC_V3XX is not set
# CONFIG_SPI_NXP_FLEXSPI is not set
CONFIG_SPI_GPIO=m
CONFIG_SPI_LM70_LLP=m
CONFIG_SPI_FSL_LIB=y
CONFIG_SPI_FSL_SPI=y
CONFIG_SPI_FSL_DSPI=m
CONFIG_SPI_MT65XX=m
# CONFIG_SPI_MTK_NOR is not set
CONFIG_SPI_OC_TINY=m
CONFIG_SPI_PL022=m
CONFIG_SPI_PXA2XX=m
CONFIG_SPI_PXA2XX_PCI=m
CONFIG_SPI_ROCKCHIP=m
CONFIG_SPI_RSPI=m
# CONFIG_SPI_QCOM_QSPI is not set
CONFIG_SPI_QUP=m
CONFIG_SPI_SC18IS602=m
CONFIG_SPI_SH_MSIOF=m
CONFIG_SPI_SH_HSPI=m
# CONFIG_SPI_SIFIVE is not set
# CONFIG_SPI_SLAVE_MT27XX is not set
# CONFIG_SPI_SPRD is not set
CONFIG_SPI_SPRD_ADI=m
# CONFIG_SPI_SUN4I is not set
CONFIG_SPI_SUN6I=m
# CONFIG_SPI_SYNQUACER is not set
# CONFIG_SPI_MXIC is not set
CONFIG_SPI_THUNDERX=m
CONFIG_SPI_XCOMM=m
# CONFIG_SPI_XILINX is not set
CONFIG_SPI_XLP=m
CONFIG_SPI_ZYNQMP_GQSPI=m
# CONFIG_SPI_AMD is not set
#
# SPI Multiplexer support
#
# CONFIG_SPI_MUX is not set
#
# SPI Protocol Masters
#
CONFIG_SPI_SPIDEV=m
CONFIG_SPI_LOOPBACK_TEST=m
CONFIG_SPI_TLE62X0=m
CONFIG_SPI_SLAVE=y
CONFIG_SPI_SLAVE_TIME=m
CONFIG_SPI_SLAVE_SYSTEM_CONTROL=m
CONFIG_SPI_DYNAMIC=y
CONFIG_SPMI=m
CONFIG_SPMI_MSM_PMIC_ARB=m
CONFIG_HSI=m
CONFIG_HSI_BOARDINFO=y
#
# HSI controllers
#
#
# HSI clients
#
CONFIG_HSI_CHAR=m
CONFIG_PPS=m
# CONFIG_PPS_DEBUG is not set
#
# PPS clients support
#
# CONFIG_PPS_CLIENT_KTIMER is not set
CONFIG_PPS_CLIENT_LDISC=m
CONFIG_PPS_CLIENT_PARPORT=m
CONFIG_PPS_CLIENT_GPIO=m
#
# PPS generators support
#
#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK=m
CONFIG_PTP_1588_CLOCK_DTE=m
CONFIG_PTP_1588_CLOCK_QORIQ=m
#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
# CONFIG_PTP_1588_CLOCK_IDT82P33 is not set
# CONFIG_PTP_1588_CLOCK_IDTCM is not set
# CONFIG_PTP_1588_CLOCK_OCP is not set
# end of PTP clock support
CONFIG_PINCTRL=y
CONFIG_GENERIC_PINCTRL_GROUPS=y
CONFIG_PINMUX=y
CONFIG_GENERIC_PINMUX_FUNCTIONS=y
CONFIG_PINCONF=y
CONFIG_GENERIC_PINCONF=y
# CONFIG_DEBUG_PINCTRL is not set
CONFIG_PINCTRL_AS3722=y
# CONFIG_PINCTRL_AXP209 is not set
CONFIG_PINCTRL_AMD=y
# CONFIG_PINCTRL_DA9062 is not set
CONFIG_PINCTRL_MCP23S08_I2C=m
CONFIG_PINCTRL_MCP23S08_SPI=m
CONFIG_PINCTRL_MCP23S08=m
CONFIG_PINCTRL_SINGLE=y
CONFIG_PINCTRL_SX150X=y
# CONFIG_PINCTRL_STMFX is not set
CONFIG_PINCTRL_MAX77620=m
CONFIG_PINCTRL_PALMAS=y
CONFIG_PINCTRL_RK805=m
# CONFIG_PINCTRL_OCELOT is not set
# CONFIG_PINCTRL_MICROCHIP_SGPIO is not set
# CONFIG_PINCTRL_OWL is not set
CONFIG_PINCTRL_BCM2835=y
CONFIG_PINCTRL_IPROC_GPIO=y
CONFIG_PINCTRL_NS2_MUX=y
CONFIG_PINCTRL_BERLIN=y
# CONFIG_PINCTRL_AS370 is not set
CONFIG_PINCTRL_BERLIN_BG4CT=y
CONFIG_PINCTRL_MSM=y
CONFIG_PINCTRL_APQ8064=m
CONFIG_PINCTRL_APQ8084=m
CONFIG_PINCTRL_IPQ4019=m
CONFIG_PINCTRL_IPQ8064=m
CONFIG_PINCTRL_IPQ8074=m
# CONFIG_PINCTRL_IPQ6018 is not set
# CONFIG_PINCTRL_MSM8226 is not set
CONFIG_PINCTRL_MSM8660=m
CONFIG_PINCTRL_MSM8960=m
CONFIG_PINCTRL_MDM9615=m
CONFIG_PINCTRL_MSM8X74=m
CONFIG_PINCTRL_MSM8916=m
# CONFIG_PINCTRL_MSM8953 is not set
# CONFIG_PINCTRL_MSM8976 is not set
CONFIG_PINCTRL_MSM8994=m
CONFIG_PINCTRL_MSM8996=m
# CONFIG_PINCTRL_MSM8998 is not set
# CONFIG_PINCTRL_QCS404 is not set
CONFIG_PINCTRL_QDF2XXX=m
CONFIG_PINCTRL_QCOM_SPMI_PMIC=m
CONFIG_PINCTRL_QCOM_SSBI_PMIC=m
# CONFIG_PINCTRL_SC7180 is not set
# CONFIG_PINCTRL_SC7280 is not set
# CONFIG_PINCTRL_SDM660 is not set
# CONFIG_PINCTRL_SDM845 is not set
# CONFIG_PINCTRL_SDX55 is not set
# CONFIG_PINCTRL_SM8150 is not set
# CONFIG_PINCTRL_SM8250 is not set
# CONFIG_PINCTRL_LPASS_LPI is not set
#
# Renesas pinctrl drivers
#
CONFIG_PINCTRL_RENESAS=y
CONFIG_PINCTRL_SH_PFC=y
CONFIG_PINCTRL_PFC_R8A77995=y
CONFIG_PINCTRL_PFC_R8A77950=y
CONFIG_PINCTRL_PFC_R8A77951=y
CONFIG_PINCTRL_PFC_R8A77970=y
# end of Renesas pinctrl drivers
CONFIG_PINCTRL_SPRD=y
CONFIG_PINCTRL_SPRD_SC9860=y
CONFIG_PINCTRL_SUNXI=y
# CONFIG_PINCTRL_SUN4I_A10 is not set
# CONFIG_PINCTRL_SUN5I is not set
# CONFIG_PINCTRL_SUN6I_A31 is not set
# CONFIG_PINCTRL_SUN6I_A31_R is not set
# CONFIG_PINCTRL_SUN8I_A23 is not set
# CONFIG_PINCTRL_SUN8I_A33 is not set
# CONFIG_PINCTRL_SUN8I_A83T is not set
# CONFIG_PINCTRL_SUN8I_A83T_R is not set
# CONFIG_PINCTRL_SUN8I_A23_R is not set
# CONFIG_PINCTRL_SUN8I_H3 is not set
CONFIG_PINCTRL_SUN8I_H3_R=y
# CONFIG_PINCTRL_SUN8I_V3S is not set
# CONFIG_PINCTRL_SUN9I_A80 is not set
# CONFIG_PINCTRL_SUN9I_A80_R is not set
CONFIG_PINCTRL_SUN50I_A64=y
CONFIG_PINCTRL_SUN50I_A64_R=y
CONFIG_PINCTRL_SUN50I_A100=y
CONFIG_PINCTRL_SUN50I_A100_R=y
CONFIG_PINCTRL_SUN50I_H5=y
CONFIG_PINCTRL_SUN50I_H6=y
CONFIG_PINCTRL_SUN50I_H6_R=y
#
# MediaTek pinctrl drivers
#
CONFIG_EINT_MTK=y
CONFIG_PINCTRL_MTK=y
CONFIG_PINCTRL_MTK_V2=y
CONFIG_PINCTRL_MTK_MOORE=y
CONFIG_PINCTRL_MTK_PARIS=y
CONFIG_PINCTRL_MT2712=y
CONFIG_PINCTRL_MT6765=y
CONFIG_PINCTRL_MT6779=y
CONFIG_PINCTRL_MT6797=y
CONFIG_PINCTRL_MT7622=y
CONFIG_PINCTRL_MT8167=y
CONFIG_PINCTRL_MT8173=y
CONFIG_PINCTRL_MT8183=y
CONFIG_PINCTRL_MT8192=y
CONFIG_PINCTRL_MT8516=y
CONFIG_PINCTRL_MT6397=y
# end of MediaTek pinctrl drivers
CONFIG_GPIOLIB=y
CONFIG_GPIOLIB_FASTPATH_LIMIT=512
CONFIG_OF_GPIO=y
CONFIG_GPIO_ACPI=y
CONFIG_GPIOLIB_IRQCHIP=y
# CONFIG_DEBUG_GPIO is not set
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_CDEV=y
CONFIG_GPIO_CDEV_V1=y
CONFIG_GPIO_GENERIC=y
CONFIG_GPIO_MAX730X=m
#
# Memory mapped GPIO drivers
#
CONFIG_GPIO_74XX_MMIO=m
CONFIG_GPIO_ALTERA=m
CONFIG_GPIO_AMDPT=m
CONFIG_GPIO_RASPBERRYPI_EXP=y
CONFIG_GPIO_BCM_XGS_IPROC=y
CONFIG_GPIO_BRCMSTB=m
# CONFIG_GPIO_CADENCE is not set
CONFIG_GPIO_DWAPB=m
# CONFIG_GPIO_EIC_SPRD is not set
CONFIG_GPIO_EXAR=m
CONFIG_GPIO_FTGPIO010=y
CONFIG_GPIO_GENERIC_PLATFORM=y
CONFIG_GPIO_GRGPIO=m
# CONFIG_GPIO_HISI is not set
# CONFIG_GPIO_HLWD is not set
# CONFIG_GPIO_LOGICVC is not set
CONFIG_GPIO_MB86S7X=m
CONFIG_GPIO_MENZ127=m
CONFIG_GPIO_MPC8XXX=y
CONFIG_GPIO_PL061=y
# CONFIG_GPIO_PMIC_EIC_SPRD is not set
CONFIG_GPIO_RCAR=m
# CONFIG_GPIO_SAMA5D2_PIOBU is not set
# CONFIG_GPIO_SIFIVE is not set
# CONFIG_GPIO_SPRD is not set
CONFIG_GPIO_SYSCON=m
CONFIG_GPIO_THUNDERX=m
CONFIG_GPIO_XGENE=y
CONFIG_GPIO_XGENE_SB=m
CONFIG_GPIO_XILINX=y
CONFIG_GPIO_XLP=m
CONFIG_GPIO_ZYNQ=m
# CONFIG_GPIO_AMD_FCH is not set
# end of Memory mapped GPIO drivers
#
# I2C GPIO expanders
#
CONFIG_GPIO_ADP5588=m
CONFIG_GPIO_ADNP=m
# CONFIG_GPIO_GW_PLD is not set
CONFIG_GPIO_MAX7300=m
CONFIG_GPIO_MAX732X=m
CONFIG_GPIO_PCA953X=m
# CONFIG_GPIO_PCA953X_IRQ is not set
# CONFIG_GPIO_PCA9570 is not set
CONFIG_GPIO_PCF857X=m
CONFIG_GPIO_TPIC2810=m
# end of I2C GPIO expanders
#
# MFD GPIO expanders
#
CONFIG_GPIO_ADP5520=m
CONFIG_GPIO_ARIZONA=m
CONFIG_GPIO_BD9571MWV=m
CONFIG_GPIO_DA9052=m
CONFIG_GPIO_DA9055=m
CONFIG_GPIO_DLN2=m
CONFIG_GPIO_JANZ_TTL=m
CONFIG_GPIO_KEMPLD=m
CONFIG_GPIO_LP3943=m
CONFIG_GPIO_LP873X=m
CONFIG_GPIO_LP87565=m
CONFIG_GPIO_MAX77620=m
CONFIG_GPIO_PALMAS=y
CONFIG_GPIO_RC5T583=y
CONFIG_GPIO_STMPE=y
CONFIG_GPIO_TC3589X=y
CONFIG_GPIO_TPS65086=m
CONFIG_GPIO_TPS65218=m
CONFIG_GPIO_TPS6586X=y
CONFIG_GPIO_TPS65910=y
CONFIG_GPIO_TPS65912=m
CONFIG_GPIO_TPS68470=y
CONFIG_GPIO_TWL4030=m
CONFIG_GPIO_TWL6040=m
CONFIG_GPIO_UCB1400=m
CONFIG_GPIO_WM831X=m
CONFIG_GPIO_WM8350=m
CONFIG_GPIO_WM8994=m
# end of MFD GPIO expanders
#
# PCI GPIO expanders
#
CONFIG_GPIO_PCI_IDIO_16=m
# CONFIG_GPIO_PCIE_IDIO_24 is not set
CONFIG_GPIO_RDC321X=m
# end of PCI GPIO expanders
#
# SPI GPIO expanders
#
CONFIG_GPIO_74X164=m
CONFIG_GPIO_MAX3191X=m
CONFIG_GPIO_MAX7301=m
CONFIG_GPIO_MC33880=m
CONFIG_GPIO_PISOSR=m
CONFIG_GPIO_XRA1403=m
# end of SPI GPIO expanders
#
# USB GPIO expanders
#
CONFIG_GPIO_VIPERBOARD=m
# end of USB GPIO expanders
#
# Virtual GPIO drivers
#
# CONFIG_GPIO_AGGREGATOR is not set
# CONFIG_GPIO_MOCKUP is not set
# end of Virtual GPIO drivers
CONFIG_W1=m
CONFIG_W1_CON=y
#
# 1-wire Bus Masters
#
CONFIG_W1_MASTER_MATROX=m
CONFIG_W1_MASTER_DS2490=m
CONFIG_W1_MASTER_DS2482=m
CONFIG_W1_MASTER_DS1WM=m
CONFIG_W1_MASTER_GPIO=m
# CONFIG_W1_MASTER_SGI is not set
# end of 1-wire Bus Masters
#
# 1-wire Slaves
#
CONFIG_W1_SLAVE_THERM=m
CONFIG_W1_SLAVE_SMEM=m
CONFIG_W1_SLAVE_DS2405=m
CONFIG_W1_SLAVE_DS2408=m
CONFIG_W1_SLAVE_DS2408_READBACK=y
CONFIG_W1_SLAVE_DS2413=m
CONFIG_W1_SLAVE_DS2406=m
CONFIG_W1_SLAVE_DS2423=m
CONFIG_W1_SLAVE_DS2805=m
# CONFIG_W1_SLAVE_DS2430 is not set
CONFIG_W1_SLAVE_DS2431=m
CONFIG_W1_SLAVE_DS2433=m
# CONFIG_W1_SLAVE_DS2433_CRC is not set
CONFIG_W1_SLAVE_DS2438=m
# CONFIG_W1_SLAVE_DS250X is not set
CONFIG_W1_SLAVE_DS2780=m
CONFIG_W1_SLAVE_DS2781=m
CONFIG_W1_SLAVE_DS28E04=m
CONFIG_W1_SLAVE_DS28E17=m
# end of 1-wire Slaves
CONFIG_POWER_RESET=y
CONFIG_POWER_RESET_AS3722=y
# CONFIG_POWER_RESET_BRCMSTB is not set
CONFIG_POWER_RESET_GPIO=y
CONFIG_POWER_RESET_GPIO_RESTART=y
CONFIG_POWER_RESET_HISI=y
CONFIG_POWER_RESET_MSM=y
# CONFIG_POWER_RESET_QCOM_PON is not set
CONFIG_POWER_RESET_LTC2952=y
# CONFIG_POWER_RESET_MT6323 is not set
# CONFIG_POWER_RESET_REGULATOR is not set
CONFIG_POWER_RESET_RESTART=y
CONFIG_POWER_RESET_VEXPRESS=y
# CONFIG_POWER_RESET_XGENE is not set
CONFIG_POWER_RESET_SYSCON=y
CONFIG_POWER_RESET_SYSCON_POWEROFF=y
CONFIG_REBOOT_MODE=m
CONFIG_SYSCON_REBOOT_MODE=m
# CONFIG_POWER_RESET_SC27XX is not set
# CONFIG_NVMEM_REBOOT_MODE is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
CONFIG_POWER_SUPPLY_HWMON=y
CONFIG_PDA_POWER=m
CONFIG_GENERIC_ADC_BATTERY=m
CONFIG_MAX8925_POWER=m
CONFIG_WM831X_BACKUP=m
CONFIG_WM831X_POWER=m
CONFIG_WM8350_POWER=m
CONFIG_TEST_POWER=m
CONFIG_BATTERY_88PM860X=m
# CONFIG_CHARGER_ADP5061 is not set
CONFIG_BATTERY_ACT8945A=m
CONFIG_BATTERY_CPCAP=m
# CONFIG_BATTERY_CW2015 is not set
CONFIG_BATTERY_DS2760=m
CONFIG_BATTERY_DS2780=m
CONFIG_BATTERY_DS2781=m
CONFIG_BATTERY_DS2782=m
CONFIG_BATTERY_SBS=m
CONFIG_CHARGER_SBS=m
CONFIG_MANAGER_SBS=m
CONFIG_BATTERY_BQ27XXX=m
CONFIG_BATTERY_BQ27XXX_I2C=m
CONFIG_BATTERY_BQ27XXX_HDQ=m
# CONFIG_BATTERY_BQ27XXX_DT_UPDATES_NVM is not set
CONFIG_BATTERY_DA9030=m
CONFIG_BATTERY_DA9052=m
CONFIG_CHARGER_DA9150=m
CONFIG_BATTERY_DA9150=m
CONFIG_CHARGER_AXP20X=m
CONFIG_BATTERY_AXP20X=m
CONFIG_AXP20X_POWER=m
CONFIG_AXP288_FUEL_GAUGE=m
CONFIG_BATTERY_MAX17040=m
CONFIG_BATTERY_MAX17042=m
CONFIG_BATTERY_MAX1721X=m
CONFIG_BATTERY_TWL4030_MADC=m
CONFIG_CHARGER_88PM860X=m
CONFIG_CHARGER_PCF50633=m
CONFIG_BATTERY_RX51=m
CONFIG_CHARGER_ISP1704=m
CONFIG_CHARGER_MAX8903=m
CONFIG_CHARGER_TWL4030=m
CONFIG_CHARGER_LP8727=m
CONFIG_CHARGER_LP8788=m
CONFIG_CHARGER_GPIO=m
CONFIG_CHARGER_MANAGER=y
# CONFIG_CHARGER_LT3651 is not set
CONFIG_CHARGER_MAX14577=m
CONFIG_CHARGER_DETECTOR_MAX14656=m
CONFIG_CHARGER_MAX77693=m
CONFIG_CHARGER_MAX8997=m
CONFIG_CHARGER_MAX8998=m
CONFIG_CHARGER_QCOM_SMBB=m
CONFIG_CHARGER_BQ2415X=m
CONFIG_CHARGER_BQ24190=m
CONFIG_CHARGER_BQ24257=m
CONFIG_CHARGER_BQ24735=m
# CONFIG_CHARGER_BQ2515X is not set
CONFIG_CHARGER_BQ25890=m
# CONFIG_CHARGER_BQ25980 is not set
CONFIG_CHARGER_SMB347=m
CONFIG_CHARGER_TPS65090=m
CONFIG_CHARGER_TPS65217=m
CONFIG_BATTERY_GAUGE_LTC2941=m
CONFIG_BATTERY_RT5033=m
CONFIG_CHARGER_RT9455=m
# CONFIG_CHARGER_CROS_USBPD is not set
# CONFIG_CHARGER_SC2731 is not set
# CONFIG_FUEL_GAUGE_SC27XX is not set
# CONFIG_CHARGER_UCS1002 is not set
# CONFIG_CHARGER_BD99954 is not set
# CONFIG_RN5T618_POWER is not set
CONFIG_HWMON=y
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set
#
# Native drivers
#
CONFIG_SENSORS_AD7314=m
CONFIG_SENSORS_AD7414=m
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
# CONFIG_SENSORS_ADM1177 is not set
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7X10=m
CONFIG_SENSORS_ADT7310=m
CONFIG_SENSORS_ADT7410=m
CONFIG_SENSORS_ADT7411=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7475=m
# CONFIG_SENSORS_AS370 is not set
CONFIG_SENSORS_ASC7621=m
# CONFIG_SENSORS_AXI_FAN_CONTROL is not set
CONFIG_SENSORS_ARM_SCPI=m
CONFIG_SENSORS_ASPEED=m
CONFIG_SENSORS_ATXP1=m
# CONFIG_SENSORS_CORSAIR_CPRO is not set
# CONFIG_SENSORS_CORSAIR_PSU is not set
# CONFIG_SENSORS_DRIVETEMP is not set
CONFIG_SENSORS_DS620=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_DA9052_ADC=m
CONFIG_SENSORS_DA9055=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_MC13783_ADC=m
CONFIG_SENSORS_FTSTEUTATES=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_G760A=m
CONFIG_SENSORS_G762=m
CONFIG_SENSORS_GPIO_FAN=m
CONFIG_SENSORS_HIH6130=m
CONFIG_SENSORS_IBMAEM=m
CONFIG_SENSORS_IBMPEX=m
CONFIG_SENSORS_IIO_HWMON=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_JC42=m
CONFIG_SENSORS_POWR1220=m
CONFIG_SENSORS_LINEAGE=m
CONFIG_SENSORS_LTC2945=m
# CONFIG_SENSORS_LTC2947_I2C is not set
# CONFIG_SENSORS_LTC2947_SPI is not set
CONFIG_SENSORS_LTC2990=m
# CONFIG_SENSORS_LTC2992 is not set
CONFIG_SENSORS_LTC4151=m
CONFIG_SENSORS_LTC4215=m
CONFIG_SENSORS_LTC4222=m
CONFIG_SENSORS_LTC4245=m
CONFIG_SENSORS_LTC4260=m
CONFIG_SENSORS_LTC4261=m
CONFIG_SENSORS_MAX1111=m
# CONFIG_SENSORS_MAX127 is not set
CONFIG_SENSORS_MAX16065=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX1668=m
CONFIG_SENSORS_MAX197=m
CONFIG_SENSORS_MAX31722=m
# CONFIG_SENSORS_MAX31730 is not set
CONFIG_SENSORS_MAX6621=m
CONFIG_SENSORS_MAX6639=m
CONFIG_SENSORS_MAX6642=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_MAX6697=m
CONFIG_SENSORS_MAX31790=m
CONFIG_SENSORS_MCP3021=m
CONFIG_SENSORS_TC654=m
CONFIG_SENSORS_MENF21BMC_HWMON=m
# CONFIG_SENSORS_MR75203 is not set
CONFIG_SENSORS_ADCXX=m
CONFIG_SENSORS_LM63=m
CONFIG_SENSORS_LM70=m
CONFIG_SENSORS_LM73=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_LM95234=m
CONFIG_SENSORS_LM95241=m
CONFIG_SENSORS_LM95245=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_NTC_THERMISTOR=m
CONFIG_SENSORS_NCT6683=m
CONFIG_SENSORS_NCT6775=m
CONFIG_SENSORS_NCT7802=m
CONFIG_SENSORS_NCT7904=m
# CONFIG_SENSORS_NPCM7XX is not set
# CONFIG_SENSORS_OCC_P8_I2C is not set
CONFIG_SENSORS_PCF8591=m
CONFIG_PMBUS=m
CONFIG_SENSORS_PMBUS=m
# CONFIG_SENSORS_ADM1266 is not set
CONFIG_SENSORS_ADM1275=m
# CONFIG_SENSORS_BEL_PFE is not set
CONFIG_SENSORS_IBM_CFFPS=m
# CONFIG_SENSORS_INSPUR_IPSPS is not set
CONFIG_SENSORS_IR35221=m
# CONFIG_SENSORS_IR38064 is not set
# CONFIG_SENSORS_IRPS5401 is not set
# CONFIG_SENSORS_ISL68137 is not set
CONFIG_SENSORS_LM25066=m
CONFIG_SENSORS_LTC2978=m
CONFIG_SENSORS_LTC2978_REGULATOR=y
CONFIG_SENSORS_LTC3815=m
CONFIG_SENSORS_MAX16064=m
# CONFIG_SENSORS_MAX16601 is not set
# CONFIG_SENSORS_MAX20730 is not set
CONFIG_SENSORS_MAX20751=m
CONFIG_SENSORS_MAX31785=m
CONFIG_SENSORS_MAX34440=m
CONFIG_SENSORS_MAX8688=m
# CONFIG_SENSORS_MP2975 is not set
# CONFIG_SENSORS_PM6764TR is not set
# CONFIG_SENSORS_PXE1610 is not set
# CONFIG_SENSORS_Q54SJ108A2 is not set
CONFIG_SENSORS_TPS40422=m
CONFIG_SENSORS_TPS53679=m
CONFIG_SENSORS_UCD9000=m
CONFIG_SENSORS_UCD9200=m
# CONFIG_SENSORS_XDPE122 is not set
CONFIG_SENSORS_ZL6100=m
CONFIG_SENSORS_PWM_FAN=m
# CONFIG_SENSORS_RASPBERRYPI_HWMON is not set
# CONFIG_SENSORS_SBTSI is not set
CONFIG_SENSORS_SHT15=m
CONFIG_SENSORS_SHT21=m
CONFIG_SENSORS_SHT3x=m
CONFIG_SENSORS_SHTC1=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_EMC1403=m
CONFIG_SENSORS_EMC2103=m
CONFIG_SENSORS_EMC6W201=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_SCH56XX_COMMON=m
CONFIG_SENSORS_SCH5627=m
CONFIG_SENSORS_SCH5636=m
CONFIG_SENSORS_STTS751=m
CONFIG_SENSORS_SMM665=m
CONFIG_SENSORS_ADC128D818=m
CONFIG_SENSORS_ADS7828=m
CONFIG_SENSORS_ADS7871=m
CONFIG_SENSORS_AMC6821=m
CONFIG_SENSORS_INA209=m
CONFIG_SENSORS_INA2XX=m
CONFIG_SENSORS_INA3221=m
CONFIG_SENSORS_TC74=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP102=m
CONFIG_SENSORS_TMP103=m
CONFIG_SENSORS_TMP108=m
CONFIG_SENSORS_TMP401=m
CONFIG_SENSORS_TMP421=m
# CONFIG_SENSORS_TMP513 is not set
CONFIG_SENSORS_VEXPRESS=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
# CONFIG_SENSORS_W83773G is not set
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83795=m
# CONFIG_SENSORS_W83795_FANCTRL is not set
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
CONFIG_SENSORS_WM831X=m
CONFIG_SENSORS_WM8350=m
CONFIG_SENSORS_XGENE=m
#
# ACPI drivers
#
CONFIG_SENSORS_ACPI_POWER=m
CONFIG_THERMAL=y
# CONFIG_THERMAL_NETLINK is not set
# CONFIG_THERMAL_STATISTICS is not set
CONFIG_THERMAL_EMERGENCY_POWEROFF_DELAY_MS=0
CONFIG_THERMAL_HWMON=y
CONFIG_THERMAL_OF=y
CONFIG_THERMAL_WRITABLE_TRIPS=y
CONFIG_THERMAL_DEFAULT_GOV_STEP_WISE=y
# CONFIG_THERMAL_DEFAULT_GOV_FAIR_SHARE is not set
# CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE is not set
CONFIG_THERMAL_GOV_FAIR_SHARE=y
CONFIG_THERMAL_GOV_STEP_WISE=y
CONFIG_THERMAL_GOV_BANG_BANG=y
CONFIG_THERMAL_GOV_USER_SPACE=y
CONFIG_CPU_THERMAL=y
CONFIG_CPU_FREQ_THERMAL=y
CONFIG_DEVFREQ_THERMAL=y
CONFIG_THERMAL_EMULATION=y
# CONFIG_THERMAL_MMIO is not set
CONFIG_HISI_THERMAL=m
CONFIG_MAX77620_THERMAL=m
CONFIG_QORIQ_THERMAL=m
# CONFIG_SUN8I_THERMAL is not set
CONFIG_RCAR_THERMAL=m
CONFIG_RCAR_GEN3_THERMAL=m
CONFIG_DA9062_THERMAL=m
CONFIG_MTK_THERMAL=m
#
# Broadcom thermal drivers
#
# CONFIG_BCM2711_THERMAL is not set
CONFIG_BCM2835_THERMAL=m
CONFIG_BRCMSTB_THERMAL=m
CONFIG_BCM_NS_THERMAL=m
CONFIG_BCM_SR_THERMAL=y
# end of Broadcom thermal drivers
# CONFIG_TI_SOC_THERMAL is not set
CONFIG_GENERIC_ADC_THERMAL=m
#
# Qualcomm thermal drivers
#
CONFIG_QCOM_TSENS=m
CONFIG_QCOM_SPMI_TEMP_ALARM=m
# end of Qualcomm thermal drivers
# CONFIG_SPRD_THERMAL is not set
CONFIG_WATCHDOG=y
CONFIG_WATCHDOG_CORE=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED=y
CONFIG_WATCHDOG_OPEN_TIMEOUT=0
CONFIG_WATCHDOG_SYSFS=y
#
# Watchdog Pretimeout Governors
#
CONFIG_WATCHDOG_PRETIMEOUT_GOV=y
CONFIG_WATCHDOG_PRETIMEOUT_GOV_SEL=m
CONFIG_WATCHDOG_PRETIMEOUT_GOV_NOOP=y
CONFIG_WATCHDOG_PRETIMEOUT_GOV_PANIC=m
CONFIG_WATCHDOG_PRETIMEOUT_DEFAULT_GOV_NOOP=y
# CONFIG_WATCHDOG_PRETIMEOUT_DEFAULT_GOV_PANIC is not set
#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
CONFIG_SOFT_WATCHDOG_PRETIMEOUT=y
CONFIG_DA9052_WATCHDOG=m
CONFIG_DA9055_WATCHDOG=m
CONFIG_DA9063_WATCHDOG=m
CONFIG_DA9062_WATCHDOG=m
CONFIG_GPIO_WATCHDOG=m
CONFIG_MENF21BMC_WATCHDOG=m
# CONFIG_MENZ069_WATCHDOG is not set
CONFIG_WDAT_WDT=m
CONFIG_WM831X_WATCHDOG=m
CONFIG_WM8350_WATCHDOG=m
CONFIG_XILINX_WATCHDOG=m
CONFIG_ZIIRAVE_WATCHDOG=m
CONFIG_ARM_SP805_WATCHDOG=m
CONFIG_ARM_SBSA_WATCHDOG=m
CONFIG_CADENCE_WATCHDOG=m
CONFIG_DW_WATCHDOG=m
CONFIG_RN5T618_WATCHDOG=m
CONFIG_SUNXI_WATCHDOG=m
CONFIG_TWL4030_WATCHDOG=m
CONFIG_MAX63XX_WATCHDOG=m
CONFIG_MAX77620_WATCHDOG=m
CONFIG_IMX2_WDT=m
CONFIG_RETU_WATCHDOG=m
CONFIG_QCOM_WDT=m
CONFIG_MEDIATEK_WATCHDOG=m
# CONFIG_ARM_SMC_WATCHDOG is not set
CONFIG_RENESAS_WDT=m
CONFIG_RENESAS_RZAWDT=m
CONFIG_RTD119X_WATCHDOG=y
# CONFIG_SPRD_WATCHDOG is not set
# CONFIG_PM8916_WATCHDOG is not set
CONFIG_ALIM7101_WDT=m
CONFIG_I6300ESB_WDT=m
CONFIG_KEMPLD_WDT=m
CONFIG_BCM2835_WDT=m
CONFIG_BCM7038_WDT=m
CONFIG_MEN_A21_WDT=m
CONFIG_XEN_WDT=m
#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m
#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_SSB_POSSIBLE=y
CONFIG_SSB=m
CONFIG_SSB_SPROM=y
CONFIG_SSB_BLOCKIO=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
CONFIG_SSB_B43_PCI_BRIDGE=y
CONFIG_SSB_SDIOHOST_POSSIBLE=y
CONFIG_SSB_SDIOHOST=y
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y
CONFIG_SSB_DRIVER_GPIO=y
CONFIG_BCMA_POSSIBLE=y
CONFIG_BCMA=m
CONFIG_BCMA_BLOCKIO=y
CONFIG_BCMA_HOST_PCI_POSSIBLE=y
CONFIG_BCMA_HOST_PCI=y
CONFIG_BCMA_HOST_SOC=y
CONFIG_BCMA_DRIVER_PCI=y
CONFIG_BCMA_SFLASH=y
CONFIG_BCMA_DRIVER_GMAC_CMN=y
CONFIG_BCMA_DRIVER_GPIO=y
# CONFIG_BCMA_DEBUG is not set
#
# Multifunction device drivers
#
CONFIG_MFD_CORE=y
# CONFIG_MFD_ALTERA_SYSMGR is not set
CONFIG_MFD_ACT8945A=m
CONFIG_MFD_SUN4I_GPADC=m
CONFIG_MFD_AS3711=y
CONFIG_MFD_AS3722=y
CONFIG_PMIC_ADP5520=y
CONFIG_MFD_AAT2870_CORE=y
CONFIG_MFD_ATMEL_FLEXCOM=m
CONFIG_MFD_ATMEL_HLCDC=m
CONFIG_MFD_BCM590XX=m
CONFIG_MFD_BD9571MWV=m
# CONFIG_MFD_AC100 is not set
CONFIG_MFD_AXP20X=m
CONFIG_MFD_AXP20X_I2C=m
CONFIG_MFD_AXP20X_RSB=m
CONFIG_MFD_CROS_EC_DEV=m
# CONFIG_MFD_MADERA is not set
CONFIG_PMIC_DA903X=y
CONFIG_PMIC_DA9052=y
CONFIG_MFD_DA9052_SPI=y
CONFIG_MFD_DA9052_I2C=y
CONFIG_MFD_DA9055=y
CONFIG_MFD_DA9062=m
CONFIG_MFD_DA9063=y
CONFIG_MFD_DA9150=m
CONFIG_MFD_DLN2=m
# CONFIG_MFD_GATEWORKS_GSC is not set
CONFIG_MFD_MC13XXX=m
CONFIG_MFD_MC13XXX_SPI=m
CONFIG_MFD_MC13XXX_I2C=m
# CONFIG_MFD_MP2629 is not set
CONFIG_MFD_HI6421_PMIC=m
CONFIG_MFD_HI655X_PMIC=m
CONFIG_HTC_PASIC3=m
CONFIG_HTC_I2CPLD=y
CONFIG_LPC_ICH=m
CONFIG_LPC_SCH=m
# CONFIG_MFD_INTEL_PMT is not set
# CONFIG_MFD_IQS62X is not set
CONFIG_MFD_JANZ_CMODIO=m
CONFIG_MFD_KEMPLD=m
CONFIG_MFD_88PM800=m
CONFIG_MFD_88PM805=m
CONFIG_MFD_88PM860X=y
CONFIG_MFD_MAX14577=y
CONFIG_MFD_MAX77620=y
# CONFIG_MFD_MAX77650 is not set
CONFIG_MFD_MAX77686=y
CONFIG_MFD_MAX77693=y
CONFIG_MFD_MAX77843=y
CONFIG_MFD_MAX8907=m
CONFIG_MFD_MAX8925=y
CONFIG_MFD_MAX8997=y
CONFIG_MFD_MAX8998=y
# CONFIG_MFD_MT6360 is not set
CONFIG_MFD_MT6397=m
CONFIG_MFD_MENF21BMC=m
CONFIG_EZX_PCAP=y
CONFIG_MFD_CPCAP=m
CONFIG_MFD_VIPERBOARD=m
CONFIG_MFD_RETU=m
CONFIG_MFD_PCF50633=m
CONFIG_PCF50633_ADC=m
CONFIG_PCF50633_GPIO=m
CONFIG_UCB1400_CORE=m
CONFIG_MFD_QCOM_RPM=m
CONFIG_MFD_SPMI_PMIC=m
CONFIG_MFD_RDC321X=m
CONFIG_MFD_RT5033=m
CONFIG_MFD_RC5T583=y
CONFIG_MFD_RK808=m
CONFIG_MFD_RN5T618=m
CONFIG_MFD_SEC_CORE=y
CONFIG_MFD_SI476X_CORE=m
# CONFIG_MFD_SL28CPLD is not set
CONFIG_MFD_SM501=m
CONFIG_MFD_SM501_GPIO=y
CONFIG_MFD_SKY81452=m
CONFIG_MFD_SC27XX_PMIC=m
CONFIG_ABX500_CORE=y
CONFIG_AB3100_CORE=y
CONFIG_AB3100_OTP=m
CONFIG_MFD_STMPE=y
#
# STMicroelectronics STMPE Interface Drivers
#
CONFIG_STMPE_I2C=y
CONFIG_STMPE_SPI=y
# end of STMicroelectronics STMPE Interface Drivers
CONFIG_MFD_SUN6I_PRCM=y
CONFIG_MFD_SYSCON=y
CONFIG_MFD_TI_AM335X_TSCADC=m
CONFIG_MFD_LP3943=m
CONFIG_MFD_LP8788=y
CONFIG_MFD_TI_LMU=m
CONFIG_MFD_PALMAS=y
CONFIG_TPS6105X=m
CONFIG_TPS65010=m
CONFIG_TPS6507X=m
CONFIG_MFD_TPS65086=m
CONFIG_MFD_TPS65090=y
CONFIG_MFD_TPS65217=m
CONFIG_MFD_TPS68470=y
CONFIG_MFD_TI_LP873X=m
CONFIG_MFD_TI_LP87565=m
CONFIG_MFD_TPS65218=m
CONFIG_MFD_TPS6586X=y
CONFIG_MFD_TPS65910=y
CONFIG_MFD_TPS65912=y
CONFIG_MFD_TPS65912_I2C=y
CONFIG_MFD_TPS65912_SPI=y
CONFIG_MFD_TPS80031=y
CONFIG_TWL4030_CORE=y
CONFIG_MFD_TWL4030_AUDIO=y
CONFIG_TWL6040_CORE=y
CONFIG_MFD_WL1273_CORE=m
CONFIG_MFD_LM3533=m
CONFIG_MFD_TC3589X=y
# CONFIG_MFD_TQMX86 is not set
CONFIG_MFD_VX855=m
# CONFIG_MFD_LOCHNAGAR is not set
CONFIG_MFD_ARIZONA=y
CONFIG_MFD_ARIZONA_I2C=m
CONFIG_MFD_ARIZONA_SPI=m
CONFIG_MFD_CS47L24=y
CONFIG_MFD_WM5102=y
CONFIG_MFD_WM5110=y
CONFIG_MFD_WM8997=y
CONFIG_MFD_WM8998=y
CONFIG_MFD_WM8400=y
CONFIG_MFD_WM831X=y
CONFIG_MFD_WM831X_I2C=y
CONFIG_MFD_WM831X_SPI=y
CONFIG_MFD_WM8350=y
CONFIG_MFD_WM8350_I2C=y
CONFIG_MFD_WM8994=m
# CONFIG_MFD_ROHM_BD718XX is not set
# CONFIG_MFD_ROHM_BD70528 is not set
# CONFIG_MFD_ROHM_BD71828 is not set
# CONFIG_MFD_STPMIC1 is not set
# CONFIG_MFD_STMFX is not set
CONFIG_MFD_VEXPRESS_SYSREG=y
# CONFIG_RAVE_SP_CORE is not set
# CONFIG_MFD_INTEL_M10_BMC is not set
# end of Multifunction device drivers
CONFIG_REGULATOR=y
# CONFIG_REGULATOR_DEBUG is not set
CONFIG_REGULATOR_FIXED_VOLTAGE=m
CONFIG_REGULATOR_VIRTUAL_CONSUMER=m
CONFIG_REGULATOR_USERSPACE_CONSUMER=m
# CONFIG_REGULATOR_88PG86X is not set
CONFIG_REGULATOR_88PM800=m
CONFIG_REGULATOR_88PM8607=m
CONFIG_REGULATOR_ACT8865=m
CONFIG_REGULATOR_ACT8945A=m
CONFIG_REGULATOR_AD5398=m
CONFIG_REGULATOR_AAT2870=m
CONFIG_REGULATOR_AB3100=m
CONFIG_REGULATOR_ARIZONA_LDO1=m
CONFIG_REGULATOR_ARIZONA_MICSUPP=m
CONFIG_REGULATOR_AS3711=m
CONFIG_REGULATOR_AS3722=m
CONFIG_REGULATOR_AXP20X=m
CONFIG_REGULATOR_BCM590XX=m
CONFIG_REGULATOR_BD9571MWV=m
CONFIG_REGULATOR_CPCAP=m
# CONFIG_REGULATOR_CROS_EC is not set
CONFIG_REGULATOR_DA903X=m
CONFIG_REGULATOR_DA9052=m
CONFIG_REGULATOR_DA9055=m
CONFIG_REGULATOR_DA9062=m
CONFIG_REGULATOR_DA9063=m
# CONFIG_REGULATOR_DA9121 is not set
CONFIG_REGULATOR_DA9210=m
CONFIG_REGULATOR_DA9211=m
CONFIG_REGULATOR_FAN53555=m
# CONFIG_REGULATOR_FAN53880 is not set
CONFIG_REGULATOR_GPIO=m
CONFIG_REGULATOR_HI6421=m
CONFIG_REGULATOR_HI6421V530=m
CONFIG_REGULATOR_HI655X=m
CONFIG_REGULATOR_ISL9305=m
CONFIG_REGULATOR_ISL6271A=m
CONFIG_REGULATOR_LM363X=m
CONFIG_REGULATOR_LP3971=m
CONFIG_REGULATOR_LP3972=m
CONFIG_REGULATOR_LP872X=m
CONFIG_REGULATOR_LP873X=m
CONFIG_REGULATOR_LP8755=m
CONFIG_REGULATOR_LP87565=m
CONFIG_REGULATOR_LP8788=m
CONFIG_REGULATOR_LTC3589=m
CONFIG_REGULATOR_LTC3676=m
CONFIG_REGULATOR_MAX14577=m
CONFIG_REGULATOR_MAX1586=m
CONFIG_REGULATOR_MAX77620=m
CONFIG_REGULATOR_MAX8649=m
CONFIG_REGULATOR_MAX8660=m
CONFIG_REGULATOR_MAX8907=m
CONFIG_REGULATOR_MAX8925=m
CONFIG_REGULATOR_MAX8952=m
CONFIG_REGULATOR_MAX8973=m
CONFIG_REGULATOR_MAX8997=m
CONFIG_REGULATOR_MAX8998=m
CONFIG_REGULATOR_MAX77686=m
CONFIG_REGULATOR_MAX77693=m
CONFIG_REGULATOR_MAX77802=m
# CONFIG_REGULATOR_MAX77826 is not set
CONFIG_REGULATOR_MC13XXX_CORE=m
CONFIG_REGULATOR_MC13783=m
CONFIG_REGULATOR_MC13892=m
# CONFIG_REGULATOR_MCP16502 is not set
# CONFIG_REGULATOR_MP5416 is not set
# CONFIG_REGULATOR_MP8859 is not set
# CONFIG_REGULATOR_MP886X is not set
# CONFIG_REGULATOR_MPQ7920 is not set
CONFIG_REGULATOR_MT6311=m
CONFIG_REGULATOR_MT6323=m
# CONFIG_REGULATOR_MT6358 is not set
CONFIG_REGULATOR_MT6380=m
CONFIG_REGULATOR_MT6397=m
CONFIG_REGULATOR_PALMAS=m
# CONFIG_REGULATOR_PCA9450 is not set
CONFIG_REGULATOR_PCAP=m
CONFIG_REGULATOR_PCF50633=m
# CONFIG_REGULATOR_PF8X00 is not set
CONFIG_REGULATOR_PFUZE100=m
CONFIG_REGULATOR_PV88060=m
CONFIG_REGULATOR_PV88080=m
CONFIG_REGULATOR_PV88090=m
CONFIG_REGULATOR_PWM=m
CONFIG_REGULATOR_QCOM_RPM=m
CONFIG_REGULATOR_QCOM_SMD_RPM=m
CONFIG_REGULATOR_QCOM_SPMI=m
# CONFIG_REGULATOR_QCOM_USB_VBUS is not set
# CONFIG_REGULATOR_RASPBERRYPI_TOUCHSCREEN_ATTINY is not set
CONFIG_REGULATOR_RC5T583=m
CONFIG_REGULATOR_RK808=m
CONFIG_REGULATOR_RN5T618=m
# CONFIG_REGULATOR_RT4801 is not set
CONFIG_REGULATOR_RT5033=m
# CONFIG_REGULATOR_RTMV20 is not set
CONFIG_REGULATOR_S2MPA01=m
CONFIG_REGULATOR_S2MPS11=m
CONFIG_REGULATOR_S5M8767=m
# CONFIG_REGULATOR_SC2731 is not set
CONFIG_REGULATOR_SKY81452=m
# CONFIG_REGULATOR_SLG51000 is not set
# CONFIG_REGULATOR_SY8106A is not set
# CONFIG_REGULATOR_SY8824X is not set
# CONFIG_REGULATOR_SY8827N is not set
CONFIG_REGULATOR_TPS51632=m
CONFIG_REGULATOR_TPS6105X=m
CONFIG_REGULATOR_TPS62360=m
CONFIG_REGULATOR_TPS65023=m
CONFIG_REGULATOR_TPS6507X=m
CONFIG_REGULATOR_TPS65086=m
CONFIG_REGULATOR_TPS65090=m
CONFIG_REGULATOR_TPS65132=m
CONFIG_REGULATOR_TPS65217=m
CONFIG_REGULATOR_TPS65218=m
CONFIG_REGULATOR_TPS6524X=m
CONFIG_REGULATOR_TPS6586X=m
CONFIG_REGULATOR_TPS65910=m
CONFIG_REGULATOR_TPS65912=m
CONFIG_REGULATOR_TPS80031=m
CONFIG_REGULATOR_TWL4030=m
CONFIG_REGULATOR_VCTRL=m
CONFIG_REGULATOR_VEXPRESS=m
# CONFIG_REGULATOR_VQMMC_IPQ4019 is not set
CONFIG_REGULATOR_WM831X=m
CONFIG_REGULATOR_WM8350=m
CONFIG_REGULATOR_WM8400=m
CONFIG_REGULATOR_WM8994=m
# CONFIG_REGULATOR_QCOM_LABIBB is not set
CONFIG_RC_CORE=m
CONFIG_RC_MAP=m
# CONFIG_LIRC is not set
CONFIG_RC_DECODERS=y
CONFIG_IR_NEC_DECODER=m
CONFIG_IR_RC5_DECODER=m
CONFIG_IR_RC6_DECODER=m
CONFIG_IR_JVC_DECODER=m
CONFIG_IR_SONY_DECODER=m
CONFIG_IR_SANYO_DECODER=m
CONFIG_IR_SHARP_DECODER=m
CONFIG_IR_MCE_KBD_DECODER=m
CONFIG_IR_XMP_DECODER=m
# CONFIG_IR_IMON_DECODER is not set
# CONFIG_IR_RCMM_DECODER is not set
CONFIG_RC_DEVICES=y
CONFIG_RC_ATI_REMOTE=m
CONFIG_IR_ENE=m
CONFIG_IR_HIX5HD2=m
CONFIG_IR_IMON=m
# CONFIG_IR_IMON_RAW is not set
CONFIG_IR_MCEUSB=m
CONFIG_IR_ITE_CIR=m
CONFIG_IR_FINTEK=m
CONFIG_IR_MTK=m
CONFIG_IR_NUVOTON=m
CONFIG_IR_REDRAT3=m
CONFIG_IR_STREAMZAP=m
CONFIG_IR_IGORPLUGUSB=m
CONFIG_IR_IGUANA=m
CONFIG_IR_TTUSBIR=m
CONFIG_RC_LOOPBACK=m
CONFIG_IR_GPIO_CIR=m
CONFIG_IR_SUNXI=m
CONFIG_IR_SERIAL=m
CONFIG_IR_SERIAL_TRANSMITTER=y
CONFIG_IR_SIR=m
# CONFIG_RC_XBOX_DVD is not set
# CONFIG_IR_TOY is not set
CONFIG_CEC_CORE=m
CONFIG_CEC_NOTIFIER=y
CONFIG_MEDIA_CEC_RC=y
CONFIG_MEDIA_CEC_SUPPORT=y
# CONFIG_CEC_CH7322 is not set
# CONFIG_CEC_CROS_EC is not set
CONFIG_USB_PULSE8_CEC=m
CONFIG_USB_RAINSHADOW_CEC=m
CONFIG_MEDIA_SUPPORT=m
# CONFIG_MEDIA_SUPPORT_FILTER is not set
CONFIG_MEDIA_SUBDRV_AUTOSELECT=y
#
# Media device types
#
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_ANALOG_TV_SUPPORT=y
CONFIG_MEDIA_DIGITAL_TV_SUPPORT=y
CONFIG_MEDIA_RADIO_SUPPORT=y
CONFIG_MEDIA_SDR_SUPPORT=y
CONFIG_MEDIA_PLATFORM_SUPPORT=y
CONFIG_MEDIA_TEST_SUPPORT=y
# end of Media device types
#
# Media core support
#
CONFIG_VIDEO_DEV=m
CONFIG_MEDIA_CONTROLLER=y
CONFIG_DVB_CORE=m
# end of Media core support
#
# Video4Linux options
#
CONFIG_VIDEO_V4L2=m
CONFIG_VIDEO_V4L2_I2C=y
CONFIG_VIDEO_V4L2_SUBDEV_API=y
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
CONFIG_VIDEO_TUNER=m
CONFIG_V4L2_MEM2MEM_DEV=m
CONFIG_V4L2_FLASH_LED_CLASS=m
CONFIG_V4L2_FWNODE=m
CONFIG_VIDEOBUF_GEN=m
CONFIG_VIDEOBUF_DMA_SG=m
CONFIG_VIDEOBUF_VMALLOC=m
# end of Video4Linux options
#
# Media controller options
#
CONFIG_MEDIA_CONTROLLER_DVB=y
CONFIG_MEDIA_CONTROLLER_REQUEST_API=y
#
# Please notice that the enabled Media controller Request API is EXPERIMENTAL
#
# end of Media controller options
#
# Digital TV options
#
# CONFIG_DVB_MMAP is not set
CONFIG_DVB_NET=y
CONFIG_DVB_MAX_ADAPTERS=8
CONFIG_DVB_DYNAMIC_MINORS=y
# CONFIG_DVB_DEMUX_SECTION_LOSS_LOG is not set
# CONFIG_DVB_ULE_DEBUG is not set
# end of Digital TV options
#
# Media drivers
#
CONFIG_TTPCI_EEPROM=m
CONFIG_MEDIA_USB_SUPPORT=y
#
# Webcam devices
#
CONFIG_USB_VIDEO_CLASS=m
CONFIG_USB_VIDEO_CLASS_INPUT_EVDEV=y
CONFIG_USB_GSPCA=m
CONFIG_USB_M5602=m
CONFIG_USB_STV06XX=m
CONFIG_USB_GL860=m
CONFIG_USB_GSPCA_BENQ=m
CONFIG_USB_GSPCA_CONEX=m
CONFIG_USB_GSPCA_CPIA1=m
CONFIG_USB_GSPCA_DTCS033=m
CONFIG_USB_GSPCA_ETOMS=m
CONFIG_USB_GSPCA_FINEPIX=m
CONFIG_USB_GSPCA_JEILINJ=m
CONFIG_USB_GSPCA_JL2005BCD=m
CONFIG_USB_GSPCA_KINECT=m
CONFIG_USB_GSPCA_KONICA=m
CONFIG_USB_GSPCA_MARS=m
CONFIG_USB_GSPCA_MR97310A=m
CONFIG_USB_GSPCA_NW80X=m
CONFIG_USB_GSPCA_OV519=m
CONFIG_USB_GSPCA_OV534=m
CONFIG_USB_GSPCA_OV534_9=m
CONFIG_USB_GSPCA_PAC207=m
CONFIG_USB_GSPCA_PAC7302=m
CONFIG_USB_GSPCA_PAC7311=m
CONFIG_USB_GSPCA_SE401=m
CONFIG_USB_GSPCA_SN9C2028=m
CONFIG_USB_GSPCA_SN9C20X=m
CONFIG_USB_GSPCA_SONIXB=m
CONFIG_USB_GSPCA_SONIXJ=m
CONFIG_USB_GSPCA_SPCA500=m
CONFIG_USB_GSPCA_SPCA501=m
CONFIG_USB_GSPCA_SPCA505=m
CONFIG_USB_GSPCA_SPCA506=m
CONFIG_USB_GSPCA_SPCA508=m
CONFIG_USB_GSPCA_SPCA561=m
CONFIG_USB_GSPCA_SPCA1528=m
CONFIG_USB_GSPCA_SQ905=m
CONFIG_USB_GSPCA_SQ905C=m
CONFIG_USB_GSPCA_SQ930X=m
CONFIG_USB_GSPCA_STK014=m
CONFIG_USB_GSPCA_STK1135=m
CONFIG_USB_GSPCA_STV0680=m
CONFIG_USB_GSPCA_SUNPLUS=m
CONFIG_USB_GSPCA_T613=m
CONFIG_USB_GSPCA_TOPRO=m
CONFIG_USB_GSPCA_TOUPTEK=m
CONFIG_USB_GSPCA_TV8532=m
CONFIG_USB_GSPCA_VC032X=m
CONFIG_USB_GSPCA_VICAM=m
CONFIG_USB_GSPCA_XIRLINK_CIT=m
CONFIG_USB_GSPCA_ZC3XX=m
CONFIG_USB_PWC=m
# CONFIG_USB_PWC_DEBUG is not set
CONFIG_USB_PWC_INPUT_EVDEV=y
CONFIG_VIDEO_CPIA2=m
CONFIG_USB_ZR364XX=m
CONFIG_USB_STKWEBCAM=m
CONFIG_USB_S2255=m
CONFIG_VIDEO_USBTV=m
#
# Analog TV USB devices
#
CONFIG_VIDEO_PVRUSB2=m
CONFIG_VIDEO_PVRUSB2_SYSFS=y
CONFIG_VIDEO_PVRUSB2_DVB=y
# CONFIG_VIDEO_PVRUSB2_DEBUGIFC is not set
CONFIG_VIDEO_HDPVR=m
CONFIG_VIDEO_STK1160_COMMON=m
CONFIG_VIDEO_STK1160=m
CONFIG_VIDEO_GO7007=m
CONFIG_VIDEO_GO7007_USB=m
CONFIG_VIDEO_GO7007_LOADER=m
CONFIG_VIDEO_GO7007_USB_S2250_BOARD=m
#
# Analog/digital TV USB devices
#
CONFIG_VIDEO_AU0828=m
CONFIG_VIDEO_AU0828_V4L2=y
CONFIG_VIDEO_AU0828_RC=y
CONFIG_VIDEO_CX231XX=m
CONFIG_VIDEO_CX231XX_RC=y
CONFIG_VIDEO_CX231XX_ALSA=m
CONFIG_VIDEO_CX231XX_DVB=m
CONFIG_VIDEO_TM6000=m
CONFIG_VIDEO_TM6000_ALSA=m
CONFIG_VIDEO_TM6000_DVB=m
#
# Digital TV USB devices
#
CONFIG_DVB_USB=m
# CONFIG_DVB_USB_DEBUG is not set
CONFIG_DVB_USB_DIB3000MC=m
CONFIG_DVB_USB_A800=m
CONFIG_DVB_USB_DIBUSB_MB=m
# CONFIG_DVB_USB_DIBUSB_MB_FAULTY is not set
CONFIG_DVB_USB_DIBUSB_MC=m
CONFIG_DVB_USB_DIB0700=m
CONFIG_DVB_USB_UMT_010=m
CONFIG_DVB_USB_CXUSB=m
# CONFIG_DVB_USB_CXUSB_ANALOG is not set
CONFIG_DVB_USB_M920X=m
CONFIG_DVB_USB_DIGITV=m
CONFIG_DVB_USB_VP7045=m
CONFIG_DVB_USB_VP702X=m
CONFIG_DVB_USB_GP8PSK=m
CONFIG_DVB_USB_NOVA_T_USB2=m
CONFIG_DVB_USB_TTUSB2=m
CONFIG_DVB_USB_DTT200U=m
CONFIG_DVB_USB_OPERA1=m
CONFIG_DVB_USB_AF9005=m
CONFIG_DVB_USB_AF9005_REMOTE=m
CONFIG_DVB_USB_PCTV452E=m
CONFIG_DVB_USB_DW2102=m
CONFIG_DVB_USB_CINERGY_T2=m
CONFIG_DVB_USB_DTV5100=m
CONFIG_DVB_USB_AZ6027=m
CONFIG_DVB_USB_TECHNISAT_USB2=m
CONFIG_DVB_USB_V2=m
CONFIG_DVB_USB_AF9015=m
CONFIG_DVB_USB_AF9035=m
CONFIG_DVB_USB_ANYSEE=m
CONFIG_DVB_USB_AU6610=m
CONFIG_DVB_USB_AZ6007=m
CONFIG_DVB_USB_CE6230=m
CONFIG_DVB_USB_EC168=m
CONFIG_DVB_USB_GL861=m
CONFIG_DVB_USB_LME2510=m
CONFIG_DVB_USB_MXL111SF=m
CONFIG_DVB_USB_RTL28XXU=m
CONFIG_DVB_USB_DVBSKY=m
CONFIG_DVB_USB_ZD1301=m
CONFIG_DVB_TTUSB_BUDGET=m
CONFIG_DVB_TTUSB_DEC=m
CONFIG_SMS_USB_DRV=m
CONFIG_DVB_B2C2_FLEXCOP_USB=m
# CONFIG_DVB_B2C2_FLEXCOP_USB_DEBUG is not set
CONFIG_DVB_AS102=m
#
# Webcam, TV (analog/digital) USB devices
#
CONFIG_VIDEO_EM28XX=m
CONFIG_VIDEO_EM28XX_V4L2=m
CONFIG_VIDEO_EM28XX_ALSA=m
CONFIG_VIDEO_EM28XX_DVB=m
CONFIG_VIDEO_EM28XX_RC=m
#
# Software defined radio USB devices
#
CONFIG_USB_AIRSPY=m
CONFIG_USB_HACKRF=m
CONFIG_USB_MSI2500=m
CONFIG_MEDIA_PCI_SUPPORT=y
#
# Media capture support
#
CONFIG_VIDEO_SOLO6X10=m
CONFIG_VIDEO_TW5864=m
CONFIG_VIDEO_TW68=m
CONFIG_VIDEO_TW686X=m
#
# Media capture/analog TV support
#
CONFIG_VIDEO_IVTV=m
# CONFIG_VIDEO_IVTV_DEPRECATED_IOCTLS is not set
CONFIG_VIDEO_IVTV_ALSA=m
CONFIG_VIDEO_FB_IVTV=m
CONFIG_VIDEO_HEXIUM_GEMINI=m
CONFIG_VIDEO_HEXIUM_ORION=m
CONFIG_VIDEO_MXB=m
CONFIG_VIDEO_DT3155=m
#
# Media capture/analog/hybrid TV support
#
CONFIG_VIDEO_CX18=m
CONFIG_VIDEO_CX18_ALSA=m
CONFIG_VIDEO_CX23885=m
CONFIG_MEDIA_ALTERA_CI=m
CONFIG_VIDEO_CX25821=m
CONFIG_VIDEO_CX25821_ALSA=m
CONFIG_VIDEO_CX88=m
CONFIG_VIDEO_CX88_ALSA=m
CONFIG_VIDEO_CX88_BLACKBIRD=m
CONFIG_VIDEO_CX88_DVB=m
CONFIG_VIDEO_CX88_ENABLE_VP3054=y
CONFIG_VIDEO_CX88_VP3054=m
CONFIG_VIDEO_CX88_MPEG=m
CONFIG_VIDEO_BT848=m
CONFIG_DVB_BT8XX=m
CONFIG_VIDEO_SAA7134=m
CONFIG_VIDEO_SAA7134_ALSA=m
CONFIG_VIDEO_SAA7134_RC=y
CONFIG_VIDEO_SAA7134_DVB=m
CONFIG_VIDEO_SAA7134_GO7007=m
CONFIG_VIDEO_SAA7164=m
#
# Media digital TV PCI Adapters
#
CONFIG_DVB_AV7110_IR=y
CONFIG_DVB_AV7110=m
CONFIG_DVB_AV7110_OSD=y
CONFIG_DVB_BUDGET_CORE=m
CONFIG_DVB_BUDGET=m
CONFIG_DVB_BUDGET_CI=m
CONFIG_DVB_BUDGET_AV=m
CONFIG_DVB_BUDGET_PATCH=m
CONFIG_DVB_B2C2_FLEXCOP_PCI=m
# CONFIG_DVB_B2C2_FLEXCOP_PCI_DEBUG is not set
CONFIG_DVB_PLUTO2=m
CONFIG_DVB_DM1105=m
CONFIG_DVB_PT1=m
CONFIG_DVB_PT3=m
CONFIG_MANTIS_CORE=m
CONFIG_DVB_MANTIS=m
CONFIG_DVB_HOPPER=m
CONFIG_DVB_NGENE=m
CONFIG_DVB_DDBRIDGE=m
# CONFIG_DVB_DDBRIDGE_MSIENABLE is not set
CONFIG_DVB_SMIPCIE=m
CONFIG_DVB_NETUP_UNIDVB=m
CONFIG_RADIO_ADAPTERS=y
CONFIG_RADIO_TEA575X=m
CONFIG_RADIO_SI470X=m
CONFIG_USB_SI470X=m
CONFIG_I2C_SI470X=m
CONFIG_RADIO_SI4713=m
CONFIG_USB_SI4713=m
CONFIG_PLATFORM_SI4713=m
CONFIG_I2C_SI4713=m
CONFIG_RADIO_SI476X=m
CONFIG_USB_MR800=m
CONFIG_USB_DSBR=m
CONFIG_RADIO_MAXIRADIO=m
CONFIG_RADIO_SHARK=m
CONFIG_RADIO_SHARK2=m
CONFIG_USB_KEENE=m
CONFIG_USB_RAREMONO=m
CONFIG_USB_MA901=m
CONFIG_RADIO_TEA5764=m
CONFIG_RADIO_SAA7706H=m
CONFIG_RADIO_TEF6862=m
CONFIG_RADIO_WL1273=m
CONFIG_RADIO_WL128X=m
CONFIG_MEDIA_COMMON_OPTIONS=y
#
# common driver options
#
CONFIG_VIDEO_CX2341X=m
CONFIG_VIDEO_TVEEPROM=m
CONFIG_CYPRESS_FIRMWARE=m
CONFIG_VIDEOBUF2_CORE=m
CONFIG_VIDEOBUF2_V4L2=m
CONFIG_VIDEOBUF2_MEMOPS=m
CONFIG_VIDEOBUF2_DMA_CONTIG=m
CONFIG_VIDEOBUF2_VMALLOC=m
CONFIG_VIDEOBUF2_DMA_SG=m
CONFIG_VIDEOBUF2_DVB=m
CONFIG_DVB_B2C2_FLEXCOP=m
CONFIG_VIDEO_SAA7146=m
CONFIG_VIDEO_SAA7146_VV=m
CONFIG_SMS_SIANO_MDTV=m
CONFIG_SMS_SIANO_RC=y
CONFIG_SMS_SIANO_DEBUGFS=y
CONFIG_VIDEO_V4L2_TPG=m
CONFIG_V4L_PLATFORM_DRIVERS=y
CONFIG_VIDEO_CAFE_CCIC=m
# CONFIG_VIDEO_CADENCE is not set
# CONFIG_VIDEO_ASPEED is not set
CONFIG_VIDEO_MUX=m
CONFIG_VIDEO_QCOM_CAMSS=m
CONFIG_VIDEO_XILINX=m
# CONFIG_VIDEO_XILINX_CSI2RXSS is not set
CONFIG_VIDEO_XILINX_TPG=m
CONFIG_VIDEO_XILINX_VTC=m
# CONFIG_VIDEO_RCAR_CSI2 is not set
CONFIG_VIDEO_RCAR_VIN=m
# CONFIG_VIDEO_SUN4I_CSI is not set
# CONFIG_VIDEO_SUN6I_CSI is not set
CONFIG_V4L_MEM2MEM_DRIVERS=y
CONFIG_VIDEO_MEDIATEK_VPU=m
CONFIG_VIDEO_MEM2MEM_DEINTERLACE=m
CONFIG_VIDEO_RENESAS_FDP1=m
CONFIG_VIDEO_RENESAS_JPU=m
CONFIG_VIDEO_RENESAS_FCP=m
CONFIG_VIDEO_RENESAS_VSP1=m
CONFIG_VIDEO_QCOM_VENUS=m
# CONFIG_VIDEO_SUN8I_DEINTERLACE is not set
# CONFIG_VIDEO_SUN8I_ROTATE is not set
CONFIG_DVB_PLATFORM_DRIVERS=y
CONFIG_SDR_PLATFORM_DRIVERS=y
CONFIG_VIDEO_RCAR_DRIF=m
#
# MMC/SDIO DVB adapters
#
CONFIG_SMS_SDIO_DRV=m
CONFIG_V4L_TEST_DRIVERS=y
# CONFIG_VIDEO_VIMC is not set
CONFIG_VIDEO_VIVID=m
CONFIG_VIDEO_VIVID_CEC=y
CONFIG_VIDEO_VIVID_MAX_DEVS=64
CONFIG_VIDEO_VIM2M=m
# CONFIG_VIDEO_VICODEC is not set
# CONFIG_DVB_TEST_DRIVERS is not set
#
# FireWire (IEEE 1394) Adapters
#
CONFIG_DVB_FIREDTV=m
CONFIG_DVB_FIREDTV_INPUT=y
# end of Media drivers
#
# Media ancillary drivers
#
CONFIG_MEDIA_ATTACH=y
#
# IR I2C driver auto-selected by 'Autoselect ancillary drivers'
#
CONFIG_VIDEO_IR_I2C=m
#
# Audio decoders, processors and mixers
#
CONFIG_VIDEO_TVAUDIO=m
CONFIG_VIDEO_TDA7432=m
CONFIG_VIDEO_TDA9840=m
# CONFIG_VIDEO_TDA1997X is not set
CONFIG_VIDEO_TEA6415C=m
CONFIG_VIDEO_TEA6420=m
CONFIG_VIDEO_MSP3400=m
CONFIG_VIDEO_CS3308=m
CONFIG_VIDEO_CS5345=m
CONFIG_VIDEO_CS53L32A=m
# CONFIG_VIDEO_TLV320AIC23B is not set
CONFIG_VIDEO_UDA1342=m
CONFIG_VIDEO_WM8775=m
CONFIG_VIDEO_WM8739=m
CONFIG_VIDEO_VP27SMPX=m
CONFIG_VIDEO_SONY_BTF_MPX=m
# end of Audio decoders, processors and mixers
#
# RDS decoders
#
CONFIG_VIDEO_SAA6588=m
# end of RDS decoders
#
# Video decoders
#
# CONFIG_VIDEO_ADV7180 is not set
# CONFIG_VIDEO_ADV7183 is not set
# CONFIG_VIDEO_ADV748X is not set
CONFIG_VIDEO_ADV7604=m
# CONFIG_VIDEO_ADV7604_CEC is not set
CONFIG_VIDEO_ADV7842=m
# CONFIG_VIDEO_ADV7842_CEC is not set
# CONFIG_VIDEO_BT819 is not set
# CONFIG_VIDEO_BT856 is not set
# CONFIG_VIDEO_BT866 is not set
# CONFIG_VIDEO_KS0127 is not set
# CONFIG_VIDEO_ML86V7667 is not set
# CONFIG_VIDEO_SAA7110 is not set
CONFIG_VIDEO_SAA711X=m
# CONFIG_VIDEO_TC358743 is not set
# CONFIG_VIDEO_TVP514X is not set
CONFIG_VIDEO_TVP5150=m
# CONFIG_VIDEO_TVP7002 is not set
CONFIG_VIDEO_TW2804=m
CONFIG_VIDEO_TW9903=m
CONFIG_VIDEO_TW9906=m
# CONFIG_VIDEO_TW9910 is not set
# CONFIG_VIDEO_VPX3220 is not set
# CONFIG_VIDEO_MAX9286 is not set
#
# Video and audio decoders
#
CONFIG_VIDEO_SAA717X=m
CONFIG_VIDEO_CX25840=m
# end of Video decoders
#
# Video encoders
#
CONFIG_VIDEO_SAA7127=m
# CONFIG_VIDEO_SAA7185 is not set
# CONFIG_VIDEO_ADV7170 is not set
# CONFIG_VIDEO_ADV7175 is not set
# CONFIG_VIDEO_ADV7343 is not set
# CONFIG_VIDEO_ADV7393 is not set
# CONFIG_VIDEO_AD9389B is not set
# CONFIG_VIDEO_AK881X is not set
# CONFIG_VIDEO_THS8200 is not set
# end of Video encoders
#
# Video improvement chips
#
CONFIG_VIDEO_UPD64031A=m
CONFIG_VIDEO_UPD64083=m
# end of Video improvement chips
#
# Audio/Video compression chips
#
CONFIG_VIDEO_SAA6752HS=m
# end of Audio/Video compression chips
#
# SDR tuner chips
#
# CONFIG_SDR_MAX2175 is not set
# end of SDR tuner chips
#
# Miscellaneous helper chips
#
# CONFIG_VIDEO_THS7303 is not set
CONFIG_VIDEO_M52790=m
# CONFIG_VIDEO_I2C is not set
# CONFIG_VIDEO_ST_MIPID02 is not set
# end of Miscellaneous helper chips
#
# Camera sensor devices
#
# CONFIG_VIDEO_HI556 is not set
# CONFIG_VIDEO_IMX214 is not set
# CONFIG_VIDEO_IMX219 is not set
# CONFIG_VIDEO_IMX258 is not set
# CONFIG_VIDEO_IMX274 is not set
# CONFIG_VIDEO_IMX290 is not set
# CONFIG_VIDEO_IMX319 is not set
# CONFIG_VIDEO_IMX355 is not set
# CONFIG_VIDEO_OV02A10 is not set
CONFIG_VIDEO_OV2640=m
# CONFIG_VIDEO_OV2659 is not set
# CONFIG_VIDEO_OV2680 is not set
# CONFIG_VIDEO_OV2685 is not set
# CONFIG_VIDEO_OV2740 is not set
# CONFIG_VIDEO_OV5640 is not set
# CONFIG_VIDEO_OV5645 is not set
# CONFIG_VIDEO_OV5647 is not set
# CONFIG_VIDEO_OV6650 is not set
# CONFIG_VIDEO_OV5670 is not set
# CONFIG_VIDEO_OV5675 is not set
# CONFIG_VIDEO_OV5695 is not set
# CONFIG_VIDEO_OV7251 is not set
# CONFIG_VIDEO_OV772X is not set
CONFIG_VIDEO_OV7640=m
CONFIG_VIDEO_OV7670=m
# CONFIG_VIDEO_OV7740 is not set
# CONFIG_VIDEO_OV8856 is not set
# CONFIG_VIDEO_OV9640 is not set
# CONFIG_VIDEO_OV9650 is not set
# CONFIG_VIDEO_OV9734 is not set
# CONFIG_VIDEO_OV13858 is not set
# CONFIG_VIDEO_VS6624 is not set
# CONFIG_VIDEO_MT9M001 is not set
# CONFIG_VIDEO_MT9M032 is not set
CONFIG_VIDEO_MT9M111=m
# CONFIG_VIDEO_MT9P031 is not set
# CONFIG_VIDEO_MT9T001 is not set
# CONFIG_VIDEO_MT9T112 is not set
CONFIG_VIDEO_MT9V011=m
# CONFIG_VIDEO_MT9V032 is not set
# CONFIG_VIDEO_MT9V111 is not set
# CONFIG_VIDEO_SR030PC30 is not set
# CONFIG_VIDEO_NOON010PC30 is not set
# CONFIG_VIDEO_M5MOLS is not set
# CONFIG_VIDEO_RDACM20 is not set
# CONFIG_VIDEO_RJ54N1 is not set
# CONFIG_VIDEO_S5K6AA is not set
# CONFIG_VIDEO_S5K6A3 is not set
# CONFIG_VIDEO_S5K4ECGX is not set
# CONFIG_VIDEO_S5K5BAF is not set
# CONFIG_VIDEO_CCS is not set
# CONFIG_VIDEO_ET8EK8 is not set
# CONFIG_VIDEO_S5C73M3 is not set
# end of Camera sensor devices
#
# Lens drivers
#
# CONFIG_VIDEO_AD5820 is not set
# CONFIG_VIDEO_AK7375 is not set
# CONFIG_VIDEO_DW9714 is not set
# CONFIG_VIDEO_DW9768 is not set
# CONFIG_VIDEO_DW9807_VCM is not set
# end of Lens drivers
#
# Flash devices
#
# CONFIG_VIDEO_ADP1653 is not set
# CONFIG_VIDEO_LM3560 is not set
# CONFIG_VIDEO_LM3646 is not set
# end of Flash devices
#
# SPI helper chips
#
# CONFIG_VIDEO_GS1662 is not set
# end of SPI helper chips
#
# Media SPI Adapters
#
# CONFIG_CXD2880_SPI_DRV is not set
# end of Media SPI Adapters
CONFIG_MEDIA_TUNER=m
#
# Customize TV tuners
#
CONFIG_MEDIA_TUNER_SIMPLE=m
CONFIG_MEDIA_TUNER_TDA18250=m
CONFIG_MEDIA_TUNER_TDA8290=m
CONFIG_MEDIA_TUNER_TDA827X=m
CONFIG_MEDIA_TUNER_TDA18271=m
CONFIG_MEDIA_TUNER_TDA9887=m
CONFIG_MEDIA_TUNER_TEA5761=m
CONFIG_MEDIA_TUNER_TEA5767=m
CONFIG_MEDIA_TUNER_MSI001=m
CONFIG_MEDIA_TUNER_MT20XX=m
CONFIG_MEDIA_TUNER_MT2060=m
CONFIG_MEDIA_TUNER_MT2063=m
CONFIG_MEDIA_TUNER_MT2266=m
CONFIG_MEDIA_TUNER_MT2131=m
CONFIG_MEDIA_TUNER_QT1010=m
CONFIG_MEDIA_TUNER_XC2028=m
CONFIG_MEDIA_TUNER_XC5000=m
CONFIG_MEDIA_TUNER_XC4000=m
CONFIG_MEDIA_TUNER_MXL5005S=m
CONFIG_MEDIA_TUNER_MXL5007T=m
CONFIG_MEDIA_TUNER_MC44S803=m
CONFIG_MEDIA_TUNER_MAX2165=m
CONFIG_MEDIA_TUNER_TDA18218=m
CONFIG_MEDIA_TUNER_FC0011=m
CONFIG_MEDIA_TUNER_FC0012=m
CONFIG_MEDIA_TUNER_FC0013=m
CONFIG_MEDIA_TUNER_TDA18212=m
CONFIG_MEDIA_TUNER_E4000=m
CONFIG_MEDIA_TUNER_FC2580=m
CONFIG_MEDIA_TUNER_M88RS6000T=m
CONFIG_MEDIA_TUNER_TUA9001=m
CONFIG_MEDIA_TUNER_SI2157=m
CONFIG_MEDIA_TUNER_IT913X=m
CONFIG_MEDIA_TUNER_R820T=m
CONFIG_MEDIA_TUNER_MXL301RF=m
CONFIG_MEDIA_TUNER_QM1D1C0042=m
CONFIG_MEDIA_TUNER_QM1D1B0004=m
# end of Customize TV tuners
#
# Customise DVB Frontends
#
#
# Multistandard (satellite) frontends
#
CONFIG_DVB_STB0899=m
CONFIG_DVB_STB6100=m
CONFIG_DVB_STV090x=m
CONFIG_DVB_STV0910=m
CONFIG_DVB_STV6110x=m
CONFIG_DVB_STV6111=m
CONFIG_DVB_MXL5XX=m
CONFIG_DVB_M88DS3103=m
#
# Multistandard (cable + terrestrial) frontends
#
CONFIG_DVB_DRXK=m
CONFIG_DVB_TDA18271C2DD=m
CONFIG_DVB_SI2165=m
CONFIG_DVB_MN88472=m
CONFIG_DVB_MN88473=m
#
# DVB-S (satellite) frontends
#
CONFIG_DVB_CX24110=m
CONFIG_DVB_CX24123=m
CONFIG_DVB_MT312=m
CONFIG_DVB_ZL10036=m
CONFIG_DVB_ZL10039=m
CONFIG_DVB_S5H1420=m
CONFIG_DVB_STV0288=m
CONFIG_DVB_STB6000=m
CONFIG_DVB_STV0299=m
CONFIG_DVB_STV6110=m
CONFIG_DVB_STV0900=m
CONFIG_DVB_TDA8083=m
CONFIG_DVB_TDA10086=m
CONFIG_DVB_TDA8261=m
CONFIG_DVB_VES1X93=m
CONFIG_DVB_TUNER_ITD1000=m
CONFIG_DVB_TUNER_CX24113=m
CONFIG_DVB_TDA826X=m
CONFIG_DVB_TUA6100=m
CONFIG_DVB_CX24116=m
CONFIG_DVB_CX24117=m
CONFIG_DVB_CX24120=m
CONFIG_DVB_SI21XX=m
CONFIG_DVB_TS2020=m
CONFIG_DVB_DS3000=m
CONFIG_DVB_MB86A16=m
CONFIG_DVB_TDA10071=m
#
# DVB-T (terrestrial) frontends
#
CONFIG_DVB_SP8870=m
CONFIG_DVB_SP887X=m
CONFIG_DVB_CX22700=m
CONFIG_DVB_CX22702=m
# CONFIG_DVB_S5H1432 is not set
CONFIG_DVB_DRXD=m
CONFIG_DVB_L64781=m
CONFIG_DVB_TDA1004X=m
CONFIG_DVB_NXT6000=m
CONFIG_DVB_MT352=m
CONFIG_DVB_ZL10353=m
CONFIG_DVB_DIB3000MB=m
CONFIG_DVB_DIB3000MC=m
CONFIG_DVB_DIB7000M=m
CONFIG_DVB_DIB7000P=m
# CONFIG_DVB_DIB9000 is not set
CONFIG_DVB_TDA10048=m
CONFIG_DVB_AF9013=m
CONFIG_DVB_EC100=m
CONFIG_DVB_STV0367=m
CONFIG_DVB_CXD2820R=m
CONFIG_DVB_CXD2841ER=m
CONFIG_DVB_RTL2830=m
CONFIG_DVB_RTL2832=m
CONFIG_DVB_RTL2832_SDR=m
CONFIG_DVB_SI2168=m
CONFIG_DVB_AS102_FE=m
CONFIG_DVB_ZD1301_DEMOD=m
CONFIG_DVB_GP8PSK_FE=m
# CONFIG_DVB_CXD2880 is not set
#
# DVB-C (cable) frontends
#
CONFIG_DVB_VES1820=m
CONFIG_DVB_TDA10021=m
CONFIG_DVB_TDA10023=m
CONFIG_DVB_STV0297=m
#
# ATSC (North American/Korean Terrestrial/Cable DTV) frontends
#
CONFIG_DVB_NXT200X=m
CONFIG_DVB_OR51211=m
CONFIG_DVB_OR51132=m
CONFIG_DVB_BCM3510=m
CONFIG_DVB_LGDT330X=m
CONFIG_DVB_LGDT3305=m
CONFIG_DVB_LGDT3306A=m
CONFIG_DVB_LG2160=m
CONFIG_DVB_S5H1409=m
CONFIG_DVB_AU8522=m
CONFIG_DVB_AU8522_DTV=m
CONFIG_DVB_AU8522_V4L=m
CONFIG_DVB_S5H1411=m
#
# ISDB-T (terrestrial) frontends
#
CONFIG_DVB_S921=m
CONFIG_DVB_DIB8000=m
CONFIG_DVB_MB86A20S=m
#
# ISDB-S (satellite) & ISDB-T (terrestrial) frontends
#
CONFIG_DVB_TC90522=m
# CONFIG_DVB_MN88443X is not set
#
# Digital terrestrial only tuners/PLL
#
CONFIG_DVB_PLL=m
CONFIG_DVB_TUNER_DIB0070=m
CONFIG_DVB_TUNER_DIB0090=m
#
# SEC control devices for DVB-S
#
CONFIG_DVB_DRX39XYJ=m
CONFIG_DVB_LNBH25=m
# CONFIG_DVB_LNBH29 is not set
CONFIG_DVB_LNBP21=m
CONFIG_DVB_LNBP22=m
CONFIG_DVB_ISL6405=m
CONFIG_DVB_ISL6421=m
CONFIG_DVB_ISL6423=m
CONFIG_DVB_A8293=m
# CONFIG_DVB_LGS8GL5 is not set
CONFIG_DVB_LGS8GXX=m
CONFIG_DVB_ATBM8830=m
CONFIG_DVB_TDA665x=m
CONFIG_DVB_IX2505V=m
CONFIG_DVB_M88RS2000=m
CONFIG_DVB_AF9033=m
CONFIG_DVB_HORUS3A=m
CONFIG_DVB_ASCOT2E=m
CONFIG_DVB_HELENE=m
#
# Common Interface (EN50221) controller drivers
#
CONFIG_DVB_CXD2099=m
CONFIG_DVB_SP2=m
# end of Customise DVB Frontends
#
# Tools to develop new frontends
#
CONFIG_DVB_DUMMY_FE=m
# end of Media ancillary drivers
#
# Graphics support
#
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=16
CONFIG_DRM=m
CONFIG_DRM_MIPI_DBI=m
CONFIG_DRM_MIPI_DSI=y
CONFIG_DRM_DP_AUX_CHARDEV=y
# CONFIG_DRM_DEBUG_SELFTEST is not set
CONFIG_DRM_KMS_HELPER=m
CONFIG_DRM_KMS_FB_HELPER=y
# CONFIG_DRM_DEBUG_DP_MST_TOPOLOGY_REFS is not set
CONFIG_DRM_FBDEV_EMULATION=y
CONFIG_DRM_FBDEV_OVERALLOC=100
# CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM is not set
CONFIG_DRM_LOAD_EDID_FIRMWARE=y
# CONFIG_DRM_DP_CEC is not set
CONFIG_DRM_TTM=m
CONFIG_DRM_VRAM_HELPER=m
CONFIG_DRM_TTM_HELPER=m
CONFIG_DRM_GEM_CMA_HELPER=y
CONFIG_DRM_KMS_CMA_HELPER=y
CONFIG_DRM_GEM_SHMEM_HELPER=y
CONFIG_DRM_VM=y
CONFIG_DRM_SCHED=m
#
# I2C encoder or helper chips
#
CONFIG_DRM_I2C_CH7006=m
CONFIG_DRM_I2C_SIL164=m
CONFIG_DRM_I2C_NXP_TDA998X=m
# CONFIG_DRM_I2C_NXP_TDA9950 is not set
# end of I2C encoder or helper chips
#
# ARM devices
#
CONFIG_DRM_HDLCD=m
# CONFIG_DRM_HDLCD_SHOW_UNDERRUN is not set
CONFIG_DRM_MALI_DISPLAY=m
# CONFIG_DRM_KOMEDA is not set
# end of ARM devices
CONFIG_DRM_RADEON=m
# CONFIG_DRM_RADEON_USERPTR is not set
CONFIG_DRM_AMDGPU=m
CONFIG_DRM_AMDGPU_SI=y
CONFIG_DRM_AMDGPU_CIK=y
CONFIG_DRM_AMDGPU_USERPTR=y
# CONFIG_DRM_AMDGPU_GART_DEBUGFS is not set
#
# ACP (Audio CoProcessor) Configuration
#
CONFIG_DRM_AMD_ACP=y
# end of ACP (Audio CoProcessor) Configuration
#
# Display Engine Configuration
#
CONFIG_DRM_AMD_DC=y
CONFIG_DRM_AMD_DC_DCN=y
# CONFIG_DRM_AMD_DC_HDCP is not set
# CONFIG_DRM_AMD_DC_SI is not set
# CONFIG_DEBUG_KERNEL_DC is not set
# end of Display Engine Configuration
# CONFIG_HSA_AMD is not set
CONFIG_DRM_NOUVEAU=m
CONFIG_NOUVEAU_LEGACY_CTX_SUPPORT=y
CONFIG_NOUVEAU_DEBUG=5
CONFIG_NOUVEAU_DEBUG_DEFAULT=3
# CONFIG_NOUVEAU_DEBUG_MMU is not set
# CONFIG_NOUVEAU_DEBUG_PUSH is not set
CONFIG_DRM_NOUVEAU_BACKLIGHT=y
CONFIG_DRM_VGEM=m
# CONFIG_DRM_VKMS is not set
CONFIG_DRM_UDL=m
CONFIG_DRM_AST=m
CONFIG_DRM_MGAG200=m
CONFIG_DRM_RCAR_DU=m
CONFIG_DRM_RCAR_CMM=m
CONFIG_DRM_RCAR_DW_HDMI=m
CONFIG_DRM_RCAR_LVDS=m
CONFIG_DRM_RCAR_VSP=y
CONFIG_DRM_RCAR_WRITEBACK=y
# CONFIG_DRM_SUN4I is not set
CONFIG_DRM_QXL=m
# CONFIG_DRM_BOCHS is not set
CONFIG_DRM_VIRTIO_GPU=m
CONFIG_DRM_MSM=m
CONFIG_DRM_MSM_GPU_STATE=y
# CONFIG_DRM_MSM_REGISTER_LOGGING is not set
# CONFIG_DRM_MSM_GPU_SUDO is not set
CONFIG_DRM_MSM_HDMI_HDCP=y
CONFIG_DRM_MSM_DP=y
CONFIG_DRM_MSM_DSI=y
CONFIG_DRM_MSM_DSI_PLL=y
CONFIG_DRM_MSM_DSI_28NM_PHY=y
CONFIG_DRM_MSM_DSI_20NM_PHY=y
CONFIG_DRM_MSM_DSI_28NM_8960_PHY=y
CONFIG_DRM_MSM_DSI_14NM_PHY=y
CONFIG_DRM_MSM_DSI_10NM_PHY=y
CONFIG_DRM_MSM_DSI_7NM_PHY=y
CONFIG_DRM_PANEL=y
#
# Display Panels
#
# CONFIG_DRM_PANEL_ABT_Y030XX067A is not set
# CONFIG_DRM_PANEL_ARM_VERSATILE is not set
# CONFIG_DRM_PANEL_ASUS_Z00T_TM5P5_NT35596 is not set
# CONFIG_DRM_PANEL_BOE_HIMAX8279D is not set
# CONFIG_DRM_PANEL_BOE_TV101WUM_NL6 is not set
CONFIG_DRM_PANEL_LVDS=m
CONFIG_DRM_PANEL_SIMPLE=m
# CONFIG_DRM_PANEL_ELIDA_KD35T133 is not set
# CONFIG_DRM_PANEL_FEIXIN_K101_IM2BA02 is not set
# CONFIG_DRM_PANEL_FEIYANG_FY07024DI26A30D is not set
# CONFIG_DRM_PANEL_ILITEK_IL9322 is not set
# CONFIG_DRM_PANEL_ILITEK_ILI9881C is not set
CONFIG_DRM_PANEL_INNOLUX_P079ZCA=m
CONFIG_DRM_PANEL_JDI_LT070ME05000=m
# CONFIG_DRM_PANEL_KINGDISPLAY_KD097D04 is not set
# CONFIG_DRM_PANEL_LEADTEK_LTK050H3146W is not set
# CONFIG_DRM_PANEL_LEADTEK_LTK500HD1829 is not set
CONFIG_DRM_PANEL_SAMSUNG_LD9040=m
# CONFIG_DRM_PANEL_LG_LB035Q02 is not set
CONFIG_DRM_PANEL_LG_LG4573=m
# CONFIG_DRM_PANEL_NEC_NL8048HL11 is not set
# CONFIG_DRM_PANEL_NOVATEK_NT35510 is not set
# CONFIG_DRM_PANEL_NOVATEK_NT36672A is not set
# CONFIG_DRM_PANEL_NOVATEK_NT39016 is not set
# CONFIG_DRM_PANEL_MANTIX_MLAF057WE51 is not set
# CONFIG_DRM_PANEL_OLIMEX_LCD_OLINUXINO is not set
CONFIG_DRM_PANEL_ORISETECH_OTM8009A=m
# CONFIG_DRM_PANEL_OSD_OSD101T2587_53TS is not set
CONFIG_DRM_PANEL_PANASONIC_VVX10F034N00=m
CONFIG_DRM_PANEL_RASPBERRYPI_TOUCHSCREEN=m
# CONFIG_DRM_PANEL_RAYDIUM_RM67191 is not set
# CONFIG_DRM_PANEL_RAYDIUM_RM68200 is not set
# CONFIG_DRM_PANEL_RONBO_RB070D30 is not set
# CONFIG_DRM_PANEL_SAMSUNG_S6D16D0 is not set
CONFIG_DRM_PANEL_SAMSUNG_S6E3HA2=m
CONFIG_DRM_PANEL_SAMSUNG_S6E63J0X03=m
# CONFIG_DRM_PANEL_SAMSUNG_S6E63M0 is not set
# CONFIG_DRM_PANEL_SAMSUNG_S6E88A0_AMS452EF01 is not set
CONFIG_DRM_PANEL_SAMSUNG_S6E8AA0=m
# CONFIG_DRM_PANEL_SAMSUNG_SOFEF00 is not set
CONFIG_DRM_PANEL_SEIKO_43WVF1G=m
CONFIG_DRM_PANEL_SHARP_LQ101R1SX01=m
# CONFIG_DRM_PANEL_SHARP_LS037V7DW01 is not set
CONFIG_DRM_PANEL_SHARP_LS043T1LE01=m
# CONFIG_DRM_PANEL_SITRONIX_ST7701 is not set
# CONFIG_DRM_PANEL_SITRONIX_ST7703 is not set
CONFIG_DRM_PANEL_SITRONIX_ST7789V=m
# CONFIG_DRM_PANEL_SONY_ACX424AKP is not set
# CONFIG_DRM_PANEL_SONY_ACX565AKM is not set
# CONFIG_DRM_PANEL_TDO_TL070WSH30 is not set
# CONFIG_DRM_PANEL_TPO_TD028TTEC1 is not set
# CONFIG_DRM_PANEL_TPO_TD043MTEA1 is not set
# CONFIG_DRM_PANEL_TPO_TPG110 is not set
# CONFIG_DRM_PANEL_TRULY_NT35597_WQXGA is not set
# CONFIG_DRM_PANEL_VISIONOX_RM69299 is not set
# CONFIG_DRM_PANEL_XINPENG_XPP055C272 is not set
# end of Display Panels
CONFIG_DRM_BRIDGE=y
CONFIG_DRM_PANEL_BRIDGE=y
#
# Display Interface Bridges
#
# CONFIG_DRM_CDNS_DSI is not set
# CONFIG_DRM_CHRONTEL_CH7033 is not set
# CONFIG_DRM_DISPLAY_CONNECTOR is not set
# CONFIG_DRM_LONTIUM_LT9611 is not set
# CONFIG_DRM_LONTIUM_LT9611UXC is not set
# CONFIG_DRM_LVDS_CODEC is not set
CONFIG_DRM_MEGACHIPS_STDPXXXX_GE_B850V3_FW=m
# CONFIG_DRM_NWL_MIPI_DSI is not set
CONFIG_DRM_NXP_PTN3460=m
CONFIG_DRM_PARADE_PS8622=m
# CONFIG_DRM_PARADE_PS8640 is not set
CONFIG_DRM_SIL_SII8620=m
CONFIG_DRM_SII902X=m
CONFIG_DRM_SII9234=m
# CONFIG_DRM_SIMPLE_BRIDGE is not set
# CONFIG_DRM_THINE_THC63LVD1024 is not set
# CONFIG_DRM_TOSHIBA_TC358762 is not set
# CONFIG_DRM_TOSHIBA_TC358764 is not set
CONFIG_DRM_TOSHIBA_TC358767=m
# CONFIG_DRM_TOSHIBA_TC358768 is not set
# CONFIG_DRM_TOSHIBA_TC358775 is not set
CONFIG_DRM_TI_TFP410=m
# CONFIG_DRM_TI_SN65DSI86 is not set
# CONFIG_DRM_TI_TPD12S015 is not set
# CONFIG_DRM_ANALOGIX_ANX6345 is not set
CONFIG_DRM_ANALOGIX_ANX78XX=m
CONFIG_DRM_ANALOGIX_DP=m
# CONFIG_DRM_ANALOGIX_ANX7625 is not set
CONFIG_DRM_I2C_ADV7511=m
CONFIG_DRM_I2C_ADV7511_AUDIO=y
CONFIG_DRM_I2C_ADV7511_CEC=y
# CONFIG_DRM_CDNS_MHDP8546 is not set
CONFIG_DRM_DW_HDMI=m
CONFIG_DRM_DW_HDMI_AHB_AUDIO=m
CONFIG_DRM_DW_HDMI_I2S_AUDIO=m
CONFIG_DRM_DW_HDMI_CEC=m
# end of Display Interface Bridges
CONFIG_DRM_VC4=m
CONFIG_DRM_VC4_HDMI_CEC=y
# CONFIG_DRM_ETNAVIV is not set
CONFIG_DRM_ARCPGU=m
CONFIG_DRM_HISI_HIBMC=m
CONFIG_DRM_HISI_KIRIN=m
CONFIG_DRM_MEDIATEK=m
CONFIG_DRM_MEDIATEK_HDMI=m
CONFIG_DRM_MXS=y
CONFIG_DRM_MXSFB=m
CONFIG_DRM_CIRRUS_QEMU=m
# CONFIG_DRM_GM12U320 is not set
# CONFIG_TINYDRM_HX8357D is not set
# CONFIG_TINYDRM_ILI9225 is not set
# CONFIG_TINYDRM_ILI9341 is not set
# CONFIG_TINYDRM_ILI9486 is not set
CONFIG_TINYDRM_MI0283QT=m
CONFIG_TINYDRM_REPAPER=m
CONFIG_TINYDRM_ST7586=m
# CONFIG_TINYDRM_ST7735R is not set
CONFIG_DRM_PL111=m
# CONFIG_DRM_XEN is not set
# CONFIG_DRM_LIMA is not set
# CONFIG_DRM_PANFROST is not set
# CONFIG_DRM_TIDSS is not set
# CONFIG_DRM_ZYNQMP_DPSUB is not set
CONFIG_DRM_LEGACY=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
CONFIG_DRM_PANEL_ORIENTATION_QUIRKS=y
#
# Frame buffer Devices
#
CONFIG_FB_CMDLINE=y
CONFIG_FB_NOTIFY=y
CONFIG_FB=y
CONFIG_FIRMWARE_EDID=y
CONFIG_FB_DDC=m
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
CONFIG_FB_SYS_FILLRECT=m
CONFIG_FB_SYS_COPYAREA=m
CONFIG_FB_SYS_IMAGEBLIT=m
# CONFIG_FB_FOREIGN_ENDIAN is not set
CONFIG_FB_SYS_FOPS=m
CONFIG_FB_DEFERRED_IO=y
CONFIG_FB_SVGALIB=m
CONFIG_FB_BACKLIGHT=m
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y
#
# Frame buffer hardware drivers
#
CONFIG_FB_CIRRUS=m
CONFIG_FB_PM2=m
CONFIG_FB_PM2_FIFO_DISCONNECT=y
CONFIG_FB_ARMCLCD=y
CONFIG_FB_CYBER2000=m
CONFIG_FB_CYBER2000_DDC=y
CONFIG_FB_ASILIANT=y
CONFIG_FB_IMSTT=y
CONFIG_FB_UVESA=m
CONFIG_FB_EFI=y
CONFIG_FB_OPENCORES=m
CONFIG_FB_S1D13XXX=m
CONFIG_FB_NVIDIA=m
CONFIG_FB_NVIDIA_I2C=y
# CONFIG_FB_NVIDIA_DEBUG is not set
CONFIG_FB_NVIDIA_BACKLIGHT=y
CONFIG_FB_RIVA=m
CONFIG_FB_RIVA_I2C=y
# CONFIG_FB_RIVA_DEBUG is not set
CONFIG_FB_RIVA_BACKLIGHT=y
CONFIG_FB_I740=m
CONFIG_FB_MATROX=m
CONFIG_FB_MATROX_MILLENIUM=y
CONFIG_FB_MATROX_MYSTIQUE=y
CONFIG_FB_MATROX_G=y
CONFIG_FB_MATROX_I2C=m
CONFIG_FB_MATROX_MAVEN=m
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
CONFIG_FB_RADEON_BACKLIGHT=y
# CONFIG_FB_RADEON_DEBUG is not set
CONFIG_FB_ATY128=m
CONFIG_FB_ATY128_BACKLIGHT=y
CONFIG_FB_ATY=m
CONFIG_FB_ATY_CT=y
# CONFIG_FB_ATY_GENERIC_LCD is not set
CONFIG_FB_ATY_GX=y
CONFIG_FB_ATY_BACKLIGHT=y
CONFIG_FB_S3=m
CONFIG_FB_S3_DDC=y
CONFIG_FB_SAVAGE=m
CONFIG_FB_SAVAGE_I2C=y
# CONFIG_FB_SAVAGE_ACCEL is not set
CONFIG_FB_SIS=m
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
CONFIG_FB_NEOMAGIC=m
CONFIG_FB_KYRO=m
CONFIG_FB_3DFX=m
# CONFIG_FB_3DFX_ACCEL is not set
# CONFIG_FB_3DFX_I2C is not set
CONFIG_FB_VOODOO1=m
CONFIG_FB_VT8623=m
CONFIG_FB_TRIDENT=m
CONFIG_FB_ARK=m
CONFIG_FB_PM3=m
CONFIG_FB_CARMINE=m
CONFIG_FB_CARMINE_DRAM_EVAL=y
# CONFIG_CARMINE_DRAM_CUSTOM is not set
CONFIG_FB_SH_MOBILE_LCDC=m
CONFIG_FB_SM501=m
CONFIG_FB_SMSCUFX=m
CONFIG_FB_UDL=m
# CONFIG_FB_IBM_GXT4500 is not set
CONFIG_FB_XILINX=m
# CONFIG_FB_VIRTUAL is not set
CONFIG_XEN_FBDEV_FRONTEND=m
CONFIG_FB_METRONOME=m
CONFIG_FB_MB862XX=m
CONFIG_FB_MB862XX_PCI_GDC=y
CONFIG_FB_MB862XX_I2C=y
CONFIG_FB_SIMPLE=y
CONFIG_FB_SSD1307=m
CONFIG_FB_SM712=m
# end of Frame buffer Devices
#
# Backlight & LCD device support
#
CONFIG_LCD_CLASS_DEVICE=m
CONFIG_LCD_L4F00242T03=m
CONFIG_LCD_LMS283GF05=m
CONFIG_LCD_LTV350QV=m
CONFIG_LCD_ILI922X=m
CONFIG_LCD_ILI9320=m
CONFIG_LCD_TDO24M=m
CONFIG_LCD_VGG2432A4=m
CONFIG_LCD_PLATFORM=m
CONFIG_LCD_AMS369FG06=m
CONFIG_LCD_LMS501KF03=m
CONFIG_LCD_HX8357=m
# CONFIG_LCD_OTM3225A is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
# CONFIG_BACKLIGHT_KTD253 is not set
CONFIG_BACKLIGHT_LM3533=m
CONFIG_BACKLIGHT_PWM=m
CONFIG_BACKLIGHT_DA903X=m
CONFIG_BACKLIGHT_DA9052=m
CONFIG_BACKLIGHT_MAX8925=m
# CONFIG_BACKLIGHT_QCOM_WLED is not set
CONFIG_BACKLIGHT_WM831X=m
CONFIG_BACKLIGHT_ADP5520=m
CONFIG_BACKLIGHT_ADP8860=m
CONFIG_BACKLIGHT_ADP8870=m
CONFIG_BACKLIGHT_88PM860X=m
CONFIG_BACKLIGHT_PCF50633=m
CONFIG_BACKLIGHT_AAT2870=m
CONFIG_BACKLIGHT_LM3630A=m
CONFIG_BACKLIGHT_LM3639=m
CONFIG_BACKLIGHT_LP855X=m
CONFIG_BACKLIGHT_LP8788=m
CONFIG_BACKLIGHT_PANDORA=m
CONFIG_BACKLIGHT_SKY81452=m
CONFIG_BACKLIGHT_TPS65217=m
CONFIG_BACKLIGHT_AS3711=m
CONFIG_BACKLIGHT_GPIO=m
CONFIG_BACKLIGHT_LV5207LP=m
CONFIG_BACKLIGHT_BD6107=m
CONFIG_BACKLIGHT_ARCXCNN=m
# CONFIG_BACKLIGHT_LED is not set
# end of Backlight & LCD device support
CONFIG_VGASTATE=m
CONFIG_VIDEOMODE_HELPERS=y
CONFIG_HDMI=y
#
# Console display driver support
#
CONFIG_DUMMY_CONSOLE=y
CONFIG_DUMMY_CONSOLE_COLUMNS=80
CONFIG_DUMMY_CONSOLE_ROWS=25
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FRAMEBUFFER_CONSOLE_DEFERRED_TAKEOVER is not set
# end of Console display driver support
# CONFIG_LOGO is not set
# end of Graphics support
CONFIG_SOUND=m
CONFIG_SOUND_OSS_CORE=y
# CONFIG_SOUND_OSS_CORE_PRECLAIM is not set
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_PCM_ELD=y
CONFIG_SND_PCM_IEC958=y
CONFIG_SND_DMAENGINE_PCM=m
CONFIG_SND_HWDEP=m
CONFIG_SND_SEQ_DEVICE=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_JACK=y
CONFIG_SND_JACK_INPUT_DEV=y
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
# CONFIG_SND_PCM_OSS is not set
CONFIG_SND_PCM_TIMER=y
CONFIG_SND_HRTIMER=m
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_MAX_CARDS=32
CONFIG_SND_SUPPORT_OLD_API=y
CONFIG_SND_PROC_FS=y
CONFIG_SND_VERBOSE_PROCFS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_VMASTER=y
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
# CONFIG_SND_SEQUENCER_OSS is not set
CONFIG_SND_SEQ_HRTIMER_DEFAULT=y
CONFIG_SND_SEQ_MIDI_EVENT=m
CONFIG_SND_SEQ_MIDI=m
CONFIG_SND_SEQ_MIDI_EMUL=m
CONFIG_SND_SEQ_VIRMIDI=m
CONFIG_SND_MPU401_UART=m
CONFIG_SND_OPL3_LIB=m
CONFIG_SND_OPL3_LIB_SEQ=m
CONFIG_SND_VX_LIB=m
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_DRIVERS=y
CONFIG_SND_DUMMY=m
CONFIG_SND_ALOOP=m
CONFIG_SND_VIRMIDI=m
CONFIG_SND_MTPAV=m
CONFIG_SND_MTS64=m
CONFIG_SND_SERIAL_U16550=m
CONFIG_SND_MPU401=m
CONFIG_SND_PORTMAN2X4=m
CONFIG_SND_AC97_POWER_SAVE=y
CONFIG_SND_AC97_POWER_SAVE_DEFAULT=0
CONFIG_SND_PCI=y
CONFIG_SND_AD1889=m
CONFIG_SND_ALS300=m
CONFIG_SND_ALI5451=m
CONFIG_SND_ATIIXP=m
CONFIG_SND_ATIIXP_MODEM=m
CONFIG_SND_AU8810=m
CONFIG_SND_AU8820=m
CONFIG_SND_AU8830=m
CONFIG_SND_AW2=m
CONFIG_SND_AZT3328=m
CONFIG_SND_BT87X=m
# CONFIG_SND_BT87X_OVERCLOCK is not set
CONFIG_SND_CA0106=m
CONFIG_SND_CMIPCI=m
CONFIG_SND_OXYGEN_LIB=m
CONFIG_SND_OXYGEN=m
CONFIG_SND_CS4281=m
CONFIG_SND_CS46XX=m
CONFIG_SND_CS46XX_NEW_DSP=y
CONFIG_SND_CTXFI=m
CONFIG_SND_DARLA20=m
CONFIG_SND_GINA20=m
CONFIG_SND_LAYLA20=m
CONFIG_SND_DARLA24=m
CONFIG_SND_GINA24=m
CONFIG_SND_LAYLA24=m
CONFIG_SND_MONA=m
CONFIG_SND_MIA=m
CONFIG_SND_ECHO3G=m
CONFIG_SND_INDIGO=m
CONFIG_SND_INDIGOIO=m
CONFIG_SND_INDIGODJ=m
CONFIG_SND_INDIGOIOX=m
CONFIG_SND_INDIGODJX=m
CONFIG_SND_EMU10K1=m
CONFIG_SND_EMU10K1_SEQ=m
CONFIG_SND_EMU10K1X=m
CONFIG_SND_ENS1370=m
CONFIG_SND_ENS1371=m
CONFIG_SND_ES1938=m
CONFIG_SND_ES1968=m
CONFIG_SND_ES1968_INPUT=y
CONFIG_SND_ES1968_RADIO=y
CONFIG_SND_FM801=m
CONFIG_SND_FM801_TEA575X_BOOL=y
CONFIG_SND_HDSP=m
CONFIG_SND_HDSPM=m
CONFIG_SND_ICE1712=m
CONFIG_SND_ICE1724=m
CONFIG_SND_INTEL8X0=m
CONFIG_SND_INTEL8X0M=m
CONFIG_SND_KORG1212=m
CONFIG_SND_LOLA=m
CONFIG_SND_LX6464ES=m
CONFIG_SND_MAESTRO3=m
CONFIG_SND_MAESTRO3_INPUT=y
CONFIG_SND_MIXART=m
CONFIG_SND_NM256=m
CONFIG_SND_PCXHR=m
CONFIG_SND_RIPTIDE=m
CONFIG_SND_RME32=m
CONFIG_SND_RME96=m
CONFIG_SND_RME9652=m
CONFIG_SND_SONICVIBES=m
CONFIG_SND_TRIDENT=m
CONFIG_SND_VIA82XX=m
CONFIG_SND_VIA82XX_MODEM=m
CONFIG_SND_VIRTUOSO=m
CONFIG_SND_VX222=m
CONFIG_SND_YMFPCI=m
#
# HD-Audio
#
CONFIG_SND_HDA=m
CONFIG_SND_HDA_GENERIC_LEDS=y
CONFIG_SND_HDA_INTEL=m
CONFIG_SND_HDA_HWDEP=y
CONFIG_SND_HDA_RECONFIG=y
CONFIG_SND_HDA_INPUT_BEEP=y
CONFIG_SND_HDA_INPUT_BEEP_MODE=0
CONFIG_SND_HDA_PATCH_LOADER=y
CONFIG_SND_HDA_CODEC_REALTEK=m
CONFIG_SND_HDA_CODEC_ANALOG=m
CONFIG_SND_HDA_CODEC_SIGMATEL=m
CONFIG_SND_HDA_CODEC_VIA=m
CONFIG_SND_HDA_CODEC_HDMI=m
CONFIG_SND_HDA_CODEC_CIRRUS=m
CONFIG_SND_HDA_CODEC_CONEXANT=m
CONFIG_SND_HDA_CODEC_CA0110=m
CONFIG_SND_HDA_CODEC_CA0132=m
CONFIG_SND_HDA_CODEC_CA0132_DSP=y
CONFIG_SND_HDA_CODEC_CMEDIA=m
CONFIG_SND_HDA_CODEC_SI3054=m
CONFIG_SND_HDA_GENERIC=m
CONFIG_SND_HDA_POWER_SAVE_DEFAULT=0
# CONFIG_SND_HDA_INTEL_HDMI_SILENT_STREAM is not set
# end of HD-Audio
CONFIG_SND_HDA_CORE=m
CONFIG_SND_HDA_DSP_LOADER=y
CONFIG_SND_HDA_COMPONENT=y
CONFIG_SND_HDA_PREALLOC_SIZE=64
CONFIG_SND_INTEL_NHLT=y
CONFIG_SND_INTEL_DSP_CONFIG=m
CONFIG_SND_SPI=y
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=m
CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER=y
CONFIG_SND_USB_UA101=m
CONFIG_SND_USB_CAIAQ=m
CONFIG_SND_USB_CAIAQ_INPUT=y
CONFIG_SND_USB_6FIRE=m
CONFIG_SND_USB_HIFACE=m
CONFIG_SND_BCD2000=m
CONFIG_SND_USB_LINE6=m
CONFIG_SND_USB_POD=m
CONFIG_SND_USB_PODHD=m
CONFIG_SND_USB_TONEPORT=m
CONFIG_SND_USB_VARIAX=m
CONFIG_SND_FIREWIRE=y
CONFIG_SND_FIREWIRE_LIB=m
CONFIG_SND_DICE=m
CONFIG_SND_OXFW=m
CONFIG_SND_ISIGHT=m
CONFIG_SND_FIREWORKS=m
CONFIG_SND_BEBOB=m
CONFIG_SND_FIREWIRE_DIGI00X=m
CONFIG_SND_FIREWIRE_TASCAM=m
CONFIG_SND_FIREWIRE_MOTU=m
CONFIG_SND_FIREFACE=m
CONFIG_SND_SOC=m
CONFIG_SND_SOC_AC97_BUS=y
CONFIG_SND_SOC_GENERIC_DMAENGINE_PCM=y
# CONFIG_SND_SOC_ADI is not set
CONFIG_SND_SOC_AMD_ACP=m
# CONFIG_SND_SOC_AMD_CZ_DA7219MX98357_MACH is not set
CONFIG_SND_SOC_AMD_CZ_RT5645_MACH=m
CONFIG_SND_ATMEL_SOC=m
# CONFIG_SND_SOC_MIKROE_PROTO is not set
CONFIG_SND_BCM2835_SOC_I2S=m
# CONFIG_SND_BCM63XX_I2S_WHISTLER is not set
CONFIG_SND_DESIGNWARE_I2S=m
CONFIG_SND_DESIGNWARE_PCM=y
#
# SoC Audio for Freescale CPUs
#
#
# Common SoC Audio options for Freescale CPUs:
#
CONFIG_SND_SOC_FSL_ASRC=m
CONFIG_SND_SOC_FSL_SAI=m
# CONFIG_SND_SOC_FSL_MQS is not set
# CONFIG_SND_SOC_FSL_AUDMIX is not set
CONFIG_SND_SOC_FSL_SSI=m
CONFIG_SND_SOC_FSL_SPDIF=m
CONFIG_SND_SOC_FSL_ESAI=m
# CONFIG_SND_SOC_FSL_MICFIL is not set
# CONFIG_SND_SOC_FSL_EASRC is not set
# CONFIG_SND_SOC_FSL_XCVR is not set
CONFIG_SND_SOC_IMX_AUDMUX=m
# end of SoC Audio for Freescale CPUs
CONFIG_SND_I2S_HI6210_I2S=m
CONFIG_SND_SOC_IMG=y
CONFIG_SND_SOC_IMG_I2S_IN=m
CONFIG_SND_SOC_IMG_I2S_OUT=m
CONFIG_SND_SOC_IMG_PARALLEL_OUT=m
CONFIG_SND_SOC_IMG_SPDIF_IN=m
CONFIG_SND_SOC_IMG_SPDIF_OUT=m
CONFIG_SND_SOC_IMG_PISTACHIO_INTERNAL_DAC=m
# CONFIG_SND_SOC_MT2701 is not set
# CONFIG_SND_SOC_MT6797 is not set
# CONFIG_SND_SOC_MT8173 is not set
# CONFIG_SND_SOC_MT8183 is not set
# CONFIG_SND_SOC_MTK_BTCVSD is not set
# CONFIG_SND_SOC_MT8192 is not set
CONFIG_SND_SOC_QCOM=m
CONFIG_SND_SOC_LPASS_CPU=m
CONFIG_SND_SOC_LPASS_PLATFORM=m
CONFIG_SND_SOC_LPASS_IPQ806X=m
CONFIG_SND_SOC_LPASS_APQ8016=m
CONFIG_SND_SOC_STORM=m
CONFIG_SND_SOC_APQ8016_SBC=m
CONFIG_SND_SOC_QCOM_COMMON=m
# CONFIG_SND_SOC_SC7180 is not set
#
# SoC Audio support for Renesas SoCs
#
CONFIG_SND_SOC_SH4_FSI=m
CONFIG_SND_SOC_RCAR=m
# end of SoC Audio support for Renesas SoCs
# CONFIG_SND_SOC_SOF_TOPLEVEL is not set
# CONFIG_SND_SOC_SPRD is not set
#
# STMicroelectronics STM32 SOC audio support
#
# end of STMicroelectronics STM32 SOC audio support
#
# Allwinner SoC Audio support
#
# CONFIG_SND_SUN4I_CODEC is not set
# CONFIG_SND_SUN8I_CODEC is not set
CONFIG_SND_SUN8I_CODEC_ANALOG=m
# CONFIG_SND_SUN50I_CODEC_ANALOG is not set
# CONFIG_SND_SUN4I_I2S is not set
# CONFIG_SND_SUN4I_SPDIF is not set
CONFIG_SND_SUN8I_ADDA_PR_REGMAP=m
# end of Allwinner SoC Audio support
# CONFIG_SND_SOC_XILINX_I2S is not set
# CONFIG_SND_SOC_XILINX_AUDIO_FORMATTER is not set
# CONFIG_SND_SOC_XILINX_SPDIF is not set
CONFIG_SND_SOC_XTFPGA_I2S=m
CONFIG_ZX_TDM=m
CONFIG_SND_SOC_I2C_AND_SPI=m
#
# CODEC drivers
#
CONFIG_SND_SOC_AC97_CODEC=m
CONFIG_SND_SOC_ADAU_UTILS=m
# CONFIG_SND_SOC_ADAU1372_I2C is not set
# CONFIG_SND_SOC_ADAU1372_SPI is not set
CONFIG_SND_SOC_ADAU1701=m
CONFIG_SND_SOC_ADAU17X1=m
CONFIG_SND_SOC_ADAU1761=m
CONFIG_SND_SOC_ADAU1761_I2C=m
CONFIG_SND_SOC_ADAU1761_SPI=m
CONFIG_SND_SOC_ADAU7002=m
# CONFIG_SND_SOC_ADAU7118_HW is not set
# CONFIG_SND_SOC_ADAU7118_I2C is not set
CONFIG_SND_SOC_AK4104=m
# CONFIG_SND_SOC_AK4118 is not set
# CONFIG_SND_SOC_AK4458 is not set
CONFIG_SND_SOC_AK4554=m
CONFIG_SND_SOC_AK4613=m
CONFIG_SND_SOC_AK4642=m
CONFIG_SND_SOC_AK5386=m
# CONFIG_SND_SOC_AK5558 is not set
CONFIG_SND_SOC_ALC5623=m
# CONFIG_SND_SOC_BD28623 is not set
CONFIG_SND_SOC_BT_SCO=m
# CONFIG_SND_SOC_CPCAP is not set
# CONFIG_SND_SOC_CROS_EC_CODEC is not set
CONFIG_SND_SOC_CS35L32=m
CONFIG_SND_SOC_CS35L33=m
CONFIG_SND_SOC_CS35L34=m
CONFIG_SND_SOC_CS35L35=m
# CONFIG_SND_SOC_CS35L36 is not set
CONFIG_SND_SOC_CS42L42=m
CONFIG_SND_SOC_CS42L51=m
CONFIG_SND_SOC_CS42L51_I2C=m
CONFIG_SND_SOC_CS42L52=m
CONFIG_SND_SOC_CS42L56=m
CONFIG_SND_SOC_CS42L73=m
# CONFIG_SND_SOC_CS4234 is not set
CONFIG_SND_SOC_CS4265=m
CONFIG_SND_SOC_CS4270=m
CONFIG_SND_SOC_CS4271=m
CONFIG_SND_SOC_CS4271_I2C=m
CONFIG_SND_SOC_CS4271_SPI=m
CONFIG_SND_SOC_CS42XX8=m
CONFIG_SND_SOC_CS42XX8_I2C=m
CONFIG_SND_SOC_CS43130=m
# CONFIG_SND_SOC_CS4341 is not set
CONFIG_SND_SOC_CS4349=m
CONFIG_SND_SOC_CS53L30=m
# CONFIG_SND_SOC_CX2072X is not set
# CONFIG_SND_SOC_DA7213 is not set
# CONFIG_SND_SOC_DMIC is not set
CONFIG_SND_SOC_HDMI_CODEC=m
CONFIG_SND_SOC_ES7134=m
# CONFIG_SND_SOC_ES7241 is not set
CONFIG_SND_SOC_ES8316=m
CONFIG_SND_SOC_ES8328=m
CONFIG_SND_SOC_ES8328_I2C=m
CONFIG_SND_SOC_ES8328_SPI=m
CONFIG_SND_SOC_GTM601=m
CONFIG_SND_SOC_INNO_RK3036=m
# CONFIG_SND_SOC_MAX98088 is not set
CONFIG_SND_SOC_MAX98357A=m
CONFIG_SND_SOC_MAX98504=m
# CONFIG_SND_SOC_MAX9867 is not set
CONFIG_SND_SOC_MAX98927=m
# CONFIG_SND_SOC_MAX98373_I2C is not set
# CONFIG_SND_SOC_MAX98390 is not set
CONFIG_SND_SOC_MAX9860=m
CONFIG_SND_SOC_MSM8916_WCD_ANALOG=m
CONFIG_SND_SOC_MSM8916_WCD_DIGITAL=m
CONFIG_SND_SOC_PCM1681=m
# CONFIG_SND_SOC_PCM1789_I2C is not set
CONFIG_SND_SOC_PCM179X=m
CONFIG_SND_SOC_PCM179X_I2C=m
CONFIG_SND_SOC_PCM179X_SPI=m
# CONFIG_SND_SOC_PCM186X_I2C is not set
# CONFIG_SND_SOC_PCM186X_SPI is not set
# CONFIG_SND_SOC_PCM3060_I2C is not set
# CONFIG_SND_SOC_PCM3060_SPI is not set
CONFIG_SND_SOC_PCM3168A=m
CONFIG_SND_SOC_PCM3168A_I2C=m
CONFIG_SND_SOC_PCM3168A_SPI=m
# CONFIG_SND_SOC_PCM5102A is not set
CONFIG_SND_SOC_PCM512x=m
CONFIG_SND_SOC_PCM512x_I2C=m
CONFIG_SND_SOC_PCM512x_SPI=m
# CONFIG_SND_SOC_RK3328 is not set
CONFIG_SND_SOC_RL6231=m
CONFIG_SND_SOC_RT5616=m
CONFIG_SND_SOC_RT5631=m
CONFIG_SND_SOC_RT5645=m
CONFIG_SND_SOC_SGTL5000=m
CONFIG_SND_SOC_SI476X=m
CONFIG_SND_SOC_SIGMADSP=m
CONFIG_SND_SOC_SIGMADSP_I2C=m
CONFIG_SND_SOC_SIGMADSP_REGMAP=m
# CONFIG_SND_SOC_SIMPLE_AMPLIFIER is not set
# CONFIG_SND_SOC_SIMPLE_MUX is not set
CONFIG_SND_SOC_SIRF_AUDIO_CODEC=m
CONFIG_SND_SOC_SPDIF=m
# CONFIG_SND_SOC_SSM2305 is not set
CONFIG_SND_SOC_SSM2602=m
CONFIG_SND_SOC_SSM2602_SPI=m
CONFIG_SND_SOC_SSM2602_I2C=m
CONFIG_SND_SOC_SSM4567=m
CONFIG_SND_SOC_STA32X=m
CONFIG_SND_SOC_STA350=m
CONFIG_SND_SOC_STI_SAS=m
CONFIG_SND_SOC_TAS2552=m
# CONFIG_SND_SOC_TAS2562 is not set
# CONFIG_SND_SOC_TAS2764 is not set
# CONFIG_SND_SOC_TAS2770 is not set
CONFIG_SND_SOC_TAS5086=m
CONFIG_SND_SOC_TAS571X=m
CONFIG_SND_SOC_TAS5720=m
# CONFIG_SND_SOC_TAS6424 is not set
# CONFIG_SND_SOC_TDA7419 is not set
CONFIG_SND_SOC_TFA9879=m
CONFIG_SND_SOC_TLV320AIC23=m
CONFIG_SND_SOC_TLV320AIC23_I2C=m
CONFIG_SND_SOC_TLV320AIC23_SPI=m
CONFIG_SND_SOC_TLV320AIC31XX=m
# CONFIG_SND_SOC_TLV320AIC32X4_I2C is not set
# CONFIG_SND_SOC_TLV320AIC32X4_SPI is not set
CONFIG_SND_SOC_TLV320AIC3X=m
# CONFIG_SND_SOC_TLV320ADCX140 is not set
CONFIG_SND_SOC_TS3A227E=m
# CONFIG_SND_SOC_TSCS42XX is not set
# CONFIG_SND_SOC_TSCS454 is not set
# CONFIG_SND_SOC_UDA1334 is not set
CONFIG_SND_SOC_WM8510=m
CONFIG_SND_SOC_WM8523=m
CONFIG_SND_SOC_WM8524=m
CONFIG_SND_SOC_WM8580=m
CONFIG_SND_SOC_WM8711=m
CONFIG_SND_SOC_WM8728=m
CONFIG_SND_SOC_WM8731=m
CONFIG_SND_SOC_WM8737=m
CONFIG_SND_SOC_WM8741=m
CONFIG_SND_SOC_WM8750=m
CONFIG_SND_SOC_WM8753=m
CONFIG_SND_SOC_WM8770=m
CONFIG_SND_SOC_WM8776=m
# CONFIG_SND_SOC_WM8782 is not set
CONFIG_SND_SOC_WM8804=m
CONFIG_SND_SOC_WM8804_I2C=m
CONFIG_SND_SOC_WM8804_SPI=m
CONFIG_SND_SOC_WM8903=m
# CONFIG_SND_SOC_WM8904 is not set
CONFIG_SND_SOC_WM8960=m
CONFIG_SND_SOC_WM8962=m
CONFIG_SND_SOC_WM8974=m
CONFIG_SND_SOC_WM8978=m
CONFIG_SND_SOC_WM8985=m
# CONFIG_SND_SOC_ZL38060 is not set
CONFIG_SND_SOC_ZX_AUD96P22=m
# CONFIG_SND_SOC_MAX9759 is not set
# CONFIG_SND_SOC_MT6351 is not set
# CONFIG_SND_SOC_MT6358 is not set
# CONFIG_SND_SOC_MT6359 is not set
# CONFIG_SND_SOC_MT6660 is not set
# CONFIG_SND_SOC_NAU8315 is not set
CONFIG_SND_SOC_NAU8540=m
CONFIG_SND_SOC_NAU8810=m
# CONFIG_SND_SOC_NAU8822 is not set
CONFIG_SND_SOC_NAU8824=m
CONFIG_SND_SOC_TPA6130A2=m
# CONFIG_SND_SOC_LPASS_WSA_MACRO is not set
# CONFIG_SND_SOC_LPASS_VA_MACRO is not set
# end of CODEC drivers
CONFIG_SND_SIMPLE_CARD_UTILS=m
CONFIG_SND_SIMPLE_CARD=m
CONFIG_SND_AUDIO_GRAPH_CARD=m
CONFIG_SND_SYNTH_EMUX=m
# CONFIG_SND_XEN_FRONTEND is not set
CONFIG_AC97_BUS=m
#
# HID support
#
CONFIG_HID=m
CONFIG_HID_BATTERY_STRENGTH=y
CONFIG_HIDRAW=y
CONFIG_UHID=m
CONFIG_HID_GENERIC=m
#
# Special HID drivers
#
CONFIG_HID_A4TECH=m
CONFIG_HID_ACCUTOUCH=m
CONFIG_HID_ACRUX=m
CONFIG_HID_ACRUX_FF=y
CONFIG_HID_APPLE=m
CONFIG_HID_APPLEIR=m
CONFIG_HID_ASUS=m
CONFIG_HID_AUREAL=m
CONFIG_HID_BELKIN=m
CONFIG_HID_BETOP_FF=m
# CONFIG_HID_BIGBEN_FF is not set
CONFIG_HID_CHERRY=m
CONFIG_HID_CHICONY=m
CONFIG_HID_CORSAIR=m
# CONFIG_HID_COUGAR is not set
# CONFIG_HID_MACALLY is not set
CONFIG_HID_PRODIKEYS=m
CONFIG_HID_CMEDIA=m
CONFIG_HID_CP2112=m
# CONFIG_HID_CREATIVE_SB0540 is not set
CONFIG_HID_CYPRESS=m
CONFIG_HID_DRAGONRISE=m
CONFIG_DRAGONRISE_FF=y
CONFIG_HID_EMS_FF=m
# CONFIG_HID_ELAN is not set
CONFIG_HID_ELECOM=m
CONFIG_HID_ELO=m
CONFIG_HID_EZKEY=m
CONFIG_HID_GEMBIRD=m
CONFIG_HID_GFRM=m
# CONFIG_HID_GLORIOUS is not set
CONFIG_HID_HOLTEK=m
CONFIG_HOLTEK_FF=y
# CONFIG_HID_GOOGLE_HAMMER is not set
# CONFIG_HID_VIVALDI is not set
CONFIG_HID_GT683R=m
CONFIG_HID_KEYTOUCH=m
CONFIG_HID_KYE=m
CONFIG_HID_UCLOGIC=m
CONFIG_HID_WALTOP=m
# CONFIG_HID_VIEWSONIC is not set
CONFIG_HID_GYRATION=m
CONFIG_HID_ICADE=m
CONFIG_HID_ITE=m
# CONFIG_HID_JABRA is not set
CONFIG_HID_TWINHAN=m
CONFIG_HID_KENSINGTON=m
CONFIG_HID_LCPOWER=m
CONFIG_HID_LED=m
CONFIG_HID_LENOVO=m
CONFIG_HID_LOGITECH=m
CONFIG_HID_LOGITECH_DJ=m
CONFIG_HID_LOGITECH_HIDPP=m
CONFIG_LOGITECH_FF=y
CONFIG_LOGIRUMBLEPAD2_FF=y
CONFIG_LOGIG940_FF=y
CONFIG_LOGIWHEELS_FF=y
CONFIG_HID_MAGICMOUSE=m
# CONFIG_HID_MALTRON is not set
CONFIG_HID_MAYFLASH=m
# CONFIG_HID_REDRAGON is not set
CONFIG_HID_MICROSOFT=m
CONFIG_HID_MONTEREY=m
CONFIG_HID_MULTITOUCH=m
CONFIG_HID_NTI=m
CONFIG_HID_NTRIG=m
CONFIG_HID_ORTEK=m
CONFIG_HID_PANTHERLORD=m
CONFIG_PANTHERLORD_FF=y
CONFIG_HID_PENMOUNT=m
CONFIG_HID_PETALYNX=m
CONFIG_HID_PICOLCD=m
CONFIG_HID_PICOLCD_FB=y
CONFIG_HID_PICOLCD_BACKLIGHT=y
CONFIG_HID_PICOLCD_LCD=y
CONFIG_HID_PICOLCD_LEDS=y
CONFIG_HID_PICOLCD_CIR=y
CONFIG_HID_PLANTRONICS=m
CONFIG_HID_PRIMAX=m
CONFIG_HID_RETRODE=m
CONFIG_HID_ROCCAT=m
CONFIG_HID_SAITEK=m
CONFIG_HID_SAMSUNG=m
CONFIG_HID_SONY=m
CONFIG_SONY_FF=y
CONFIG_HID_SPEEDLINK=m
# CONFIG_HID_STEAM is not set
CONFIG_HID_STEELSERIES=m
CONFIG_HID_SUNPLUS=m
CONFIG_HID_RMI=m
CONFIG_HID_GREENASIA=m
CONFIG_GREENASIA_FF=y
CONFIG_HID_SMARTJOYPLUS=m
CONFIG_SMARTJOYPLUS_FF=y
CONFIG_HID_TIVO=m
CONFIG_HID_TOPSEED=m
CONFIG_HID_THINGM=m
CONFIG_HID_THRUSTMASTER=m
CONFIG_THRUSTMASTER_FF=y
CONFIG_HID_UDRAW_PS3=m
# CONFIG_HID_U2FZERO is not set
CONFIG_HID_WACOM=m
CONFIG_HID_WIIMOTE=m
CONFIG_HID_XINMO=m
CONFIG_HID_ZEROPLUS=m
CONFIG_ZEROPLUS_FF=y
CONFIG_HID_ZYDACRON=m
CONFIG_HID_SENSOR_HUB=m
CONFIG_HID_SENSOR_CUSTOM_SENSOR=m
CONFIG_HID_ALPS=m
# CONFIG_HID_MCP2221 is not set
# end of Special HID drivers
#
# USB HID support
#
CONFIG_USB_HID=m
CONFIG_HID_PID=y
CONFIG_USB_HIDDEV=y
#
# USB HID Boot Protocol drivers
#
CONFIG_USB_KBD=m
CONFIG_USB_MOUSE=m
# end of USB HID Boot Protocol drivers
# end of USB HID support
#
# I2C HID support
#
CONFIG_I2C_HID=m
# end of I2C HID support
# end of HID support
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_LED_TRIG=y
CONFIG_USB_ULPI_BUS=m
# CONFIG_USB_CONN_GPIO is not set
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_PCI=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
#
# Miscellaneous USB options
#
CONFIG_USB_DEFAULT_PERSIST=y
# CONFIG_USB_FEW_INIT_RETRIES is not set
CONFIG_USB_DYNAMIC_MINORS=y
# CONFIG_USB_OTG is not set
# CONFIG_USB_OTG_PRODUCTLIST is not set
# CONFIG_USB_OTG_DISABLE_EXTERNAL_HUB is not set
CONFIG_USB_LEDS_TRIGGER_USBPORT=m
CONFIG_USB_AUTOSUSPEND_DELAY=2
CONFIG_USB_MON=m
#
# USB Host Controller Drivers
#
CONFIG_USB_C67X00_HCD=m
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_DBGCAP=y
CONFIG_USB_XHCI_PCI=y
# CONFIG_USB_XHCI_PCI_RENESAS is not set
CONFIG_USB_XHCI_PLATFORM=m
# CONFIG_USB_XHCI_HISTB is not set
CONFIG_USB_XHCI_MTK=m
CONFIG_USB_XHCI_RCAR=m
# CONFIG_USB_BRCMSTB is not set
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_EHCI_PCI=y
# CONFIG_USB_EHCI_FSL is not set
CONFIG_USB_EHCI_HCD_PLATFORM=m
CONFIG_USB_OXU210HP_HCD=m
CONFIG_USB_ISP116X_HCD=m
CONFIG_USB_FOTG210_HCD=m
CONFIG_USB_MAX3421_HCD=m
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_OHCI_HCD_PCI=y
CONFIG_USB_OHCI_HCD_PLATFORM=m
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_U132_HCD=m
CONFIG_USB_SL811_HCD=m
CONFIG_USB_SL811_HCD_ISO=y
CONFIG_USB_R8A66597_HCD=m
CONFIG_USB_RENESAS_USBHS_HCD=m
CONFIG_USB_HCD_BCMA=m
CONFIG_USB_HCD_SSB=m
# CONFIG_USB_HCD_TEST_MODE is not set
CONFIG_USB_RENESAS_USBHS=m
#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
CONFIG_USB_WDM=m
CONFIG_USB_TMC=m
#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#
#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_REALTEK=m
CONFIG_REALTEK_AUTOPM=y
CONFIG_USB_STORAGE_DATAFAB=m
CONFIG_USB_STORAGE_FREECOM=m
CONFIG_USB_STORAGE_ISD200=m
CONFIG_USB_STORAGE_USBAT=m
CONFIG_USB_STORAGE_SDDR09=m
CONFIG_USB_STORAGE_SDDR55=m
CONFIG_USB_STORAGE_JUMPSHOT=m
CONFIG_USB_STORAGE_ALAUDA=m
CONFIG_USB_STORAGE_ONETOUCH=m
CONFIG_USB_STORAGE_KARMA=m
CONFIG_USB_STORAGE_CYPRESS_ATACB=m
CONFIG_USB_STORAGE_ENE_UB6250=m
CONFIG_USB_UAS=m
#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
CONFIG_USBIP_CORE=m
CONFIG_USBIP_VHCI_HCD=m
CONFIG_USBIP_VHCI_HC_PORTS=8
CONFIG_USBIP_VHCI_NR_HCS=1
CONFIG_USBIP_HOST=m
CONFIG_USBIP_VUDC=m
# CONFIG_USBIP_DEBUG is not set
# CONFIG_USB_CDNS3 is not set
CONFIG_USB_MTU3=m
# CONFIG_USB_MTU3_HOST is not set
# CONFIG_USB_MTU3_GADGET is not set
CONFIG_USB_MTU3_DUAL_ROLE=y
# CONFIG_USB_MTU3_DEBUG is not set
CONFIG_USB_MUSB_HDRC=m
# CONFIG_USB_MUSB_HOST is not set
# CONFIG_USB_MUSB_GADGET is not set
CONFIG_USB_MUSB_DUAL_ROLE=y
#
# Platform Glue Layer
#
CONFIG_USB_MUSB_SUNXI=m
# CONFIG_USB_MUSB_MEDIATEK is not set
#
# MUSB DMA mode
#
CONFIG_MUSB_PIO_ONLY=y
CONFIG_USB_DWC3=m
CONFIG_USB_DWC3_ULPI=y
# CONFIG_USB_DWC3_HOST is not set
# CONFIG_USB_DWC3_GADGET is not set
CONFIG_USB_DWC3_DUAL_ROLE=y
#
# Platform Glue Driver Support
#
CONFIG_USB_DWC3_PCI=m
CONFIG_USB_DWC3_HAPS=m
CONFIG_USB_DWC3_OF_SIMPLE=m
CONFIG_USB_DWC3_QCOM=m
CONFIG_USB_DWC2=y
CONFIG_USB_DWC2_HOST=y
#
# Gadget/Dual-role mode requires USB Gadget support to be enabled
#
CONFIG_USB_DWC2_PCI=m
# CONFIG_USB_DWC2_DEBUG is not set
# CONFIG_USB_DWC2_TRACK_MISSED_SOFS is not set
CONFIG_USB_CHIPIDEA=m
CONFIG_USB_CHIPIDEA_UDC=y
CONFIG_USB_CHIPIDEA_HOST=y
CONFIG_USB_CHIPIDEA_PCI=m
CONFIG_USB_CHIPIDEA_MSM=m
CONFIG_USB_CHIPIDEA_IMX=m
CONFIG_USB_CHIPIDEA_GENERIC=m
CONFIG_USB_CHIPIDEA_TEGRA=m
CONFIG_USB_ISP1760=m
CONFIG_USB_ISP1760_HCD=y
CONFIG_USB_ISP1761_UDC=y
# CONFIG_USB_ISP1760_HOST_ROLE is not set
# CONFIG_USB_ISP1760_GADGET_ROLE is not set
CONFIG_USB_ISP1760_DUAL_ROLE=y
#
# USB port drivers
#
CONFIG_USB_USS720=m
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_SIMPLE=m
CONFIG_USB_SERIAL_AIRCABLE=m
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_CH341=m
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_CP210X=m
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_F81232=m
CONFIG_USB_SERIAL_F8153X=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
CONFIG_USB_SERIAL_IUU=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_METRO=m
CONFIG_USB_SERIAL_MOS7720=m
CONFIG_USB_SERIAL_MOS7715_PARPORT=y
CONFIG_USB_SERIAL_MOS7840=m
CONFIG_USB_SERIAL_MXUPORT=m
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_OTI6858=m
CONFIG_USB_SERIAL_QCAUX=m
CONFIG_USB_SERIAL_QUALCOMM=m
CONFIG_USB_SERIAL_SPCP8X5=m
CONFIG_USB_SERIAL_SAFE=m
# CONFIG_USB_SERIAL_SAFE_PADDED is not set
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
CONFIG_USB_SERIAL_SYMBOL=m
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_WWAN=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_SERIAL_OPTICON=m
CONFIG_USB_SERIAL_XSENS_MT=m
CONFIG_USB_SERIAL_WISHBONE=m
CONFIG_USB_SERIAL_SSU100=m
CONFIG_USB_SERIAL_QT2=m
CONFIG_USB_SERIAL_UPD78F0730=m
CONFIG_USB_SERIAL_DEBUG=m
#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_ADUTUX=m
CONFIG_USB_SEVSEG=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
CONFIG_USB_CYPRESS_CY7C63=m
CONFIG_USB_CYTHERM=m
CONFIG_USB_IDMOUSE=m
CONFIG_USB_FTDI_ELAN=m
CONFIG_USB_APPLEDISPLAY=m
# CONFIG_APPLE_MFI_FASTCHARGE is not set
CONFIG_USB_SISUSBVGA=m
CONFIG_USB_LD=m
CONFIG_USB_TRANCEVIBRATOR=m
CONFIG_USB_IOWARRIOR=m
CONFIG_USB_TEST=m
CONFIG_USB_EHSET_TEST_FIXTURE=m
CONFIG_USB_ISIGHTFW=m
CONFIG_USB_YUREX=m
CONFIG_USB_EZUSB_FX2=m
CONFIG_USB_HUB_USB251XB=m
CONFIG_USB_HSIC_USB3503=m
CONFIG_USB_HSIC_USB4604=m
CONFIG_USB_LINK_LAYER_TEST=m
CONFIG_USB_CHAOSKEY=m
CONFIG_BRCM_USB_PINMAP=m
CONFIG_USB_ATM=m
CONFIG_USB_SPEEDTOUCH=m
CONFIG_USB_CXACRU=m
CONFIG_USB_UEAGLEATM=m
CONFIG_USB_XUSBATM=m
#
# USB Physical Layer drivers
#
CONFIG_USB_PHY=y
CONFIG_NOP_USB_XCEIV=m
CONFIG_USB_GPIO_VBUS=m
CONFIG_TAHVO_USB=m
CONFIG_TAHVO_USB_HOST_BY_DEFAULT=y
CONFIG_USB_ISP1301=m
CONFIG_USB_ULPI=y
CONFIG_USB_ULPI_VIEWPORT=y
# end of USB Physical Layer drivers
CONFIG_USB_GADGET=m
# CONFIG_USB_GADGET_DEBUG is not set
# CONFIG_USB_GADGET_DEBUG_FILES is not set
# CONFIG_USB_GADGET_DEBUG_FS is not set
CONFIG_USB_GADGET_VBUS_DRAW=2
CONFIG_USB_GADGET_STORAGE_NUM_BUFFERS=2
CONFIG_U_SERIAL_CONSOLE=y
#
# USB Peripheral Controller
#
CONFIG_USB_FOTG210_UDC=m
CONFIG_USB_GR_UDC=m
CONFIG_USB_R8A66597=m
CONFIG_USB_RENESAS_USBHS_UDC=m
CONFIG_USB_RENESAS_USB3=m
CONFIG_USB_PXA27X=m
CONFIG_USB_MV_UDC=m
CONFIG_USB_MV_U3D=m
CONFIG_USB_SNP_CORE=m
CONFIG_USB_SNP_UDC_PLAT=m
# CONFIG_USB_M66592 is not set
CONFIG_USB_BDC_UDC=m
#
# Platform Support
#
CONFIG_USB_BDC_PCI=m
CONFIG_USB_AMD5536UDC=m
CONFIG_USB_NET2272=m
CONFIG_USB_NET2272_DMA=y
CONFIG_USB_NET2280=m
CONFIG_USB_GOKU=m
CONFIG_USB_EG20T=m
CONFIG_USB_GADGET_XILINX=m
# CONFIG_USB_MAX3420_UDC is not set
# CONFIG_USB_DUMMY_HCD is not set
# end of USB Peripheral Controller
CONFIG_USB_LIBCOMPOSITE=m
CONFIG_USB_F_ACM=m
CONFIG_USB_F_SS_LB=m
CONFIG_USB_U_SERIAL=m
CONFIG_USB_U_ETHER=m
CONFIG_USB_U_AUDIO=m
CONFIG_USB_F_SERIAL=m
CONFIG_USB_F_OBEX=m
CONFIG_USB_F_NCM=m
CONFIG_USB_F_ECM=m
CONFIG_USB_F_PHONET=m
CONFIG_USB_F_EEM=m
CONFIG_USB_F_SUBSET=m
CONFIG_USB_F_RNDIS=m
CONFIG_USB_F_MASS_STORAGE=m
CONFIG_USB_F_FS=m
CONFIG_USB_F_UAC1=m
CONFIG_USB_F_UAC1_LEGACY=m
CONFIG_USB_F_UAC2=m
CONFIG_USB_F_UVC=m
CONFIG_USB_F_MIDI=m
CONFIG_USB_F_HID=m
CONFIG_USB_F_PRINTER=m
CONFIG_USB_F_TCM=m
CONFIG_USB_CONFIGFS=m
CONFIG_USB_CONFIGFS_SERIAL=y
CONFIG_USB_CONFIGFS_ACM=y
CONFIG_USB_CONFIGFS_OBEX=y
CONFIG_USB_CONFIGFS_NCM=y
CONFIG_USB_CONFIGFS_ECM=y
CONFIG_USB_CONFIGFS_ECM_SUBSET=y
CONFIG_USB_CONFIGFS_RNDIS=y
CONFIG_USB_CONFIGFS_EEM=y
CONFIG_USB_CONFIGFS_PHONET=y
CONFIG_USB_CONFIGFS_MASS_STORAGE=y
CONFIG_USB_CONFIGFS_F_LB_SS=y
CONFIG_USB_CONFIGFS_F_FS=y
CONFIG_USB_CONFIGFS_F_UAC1=y
CONFIG_USB_CONFIGFS_F_UAC1_LEGACY=y
CONFIG_USB_CONFIGFS_F_UAC2=y
CONFIG_USB_CONFIGFS_F_MIDI=y
CONFIG_USB_CONFIGFS_F_HID=y
CONFIG_USB_CONFIGFS_F_UVC=y
CONFIG_USB_CONFIGFS_F_PRINTER=y
CONFIG_USB_CONFIGFS_F_TCM=y
#
# USB Gadget precomposed configurations
#
CONFIG_USB_ZERO=m
CONFIG_USB_AUDIO=m
CONFIG_GADGET_UAC1=y
# CONFIG_GADGET_UAC1_LEGACY is not set
CONFIG_USB_ETH=m
CONFIG_USB_ETH_RNDIS=y
CONFIG_USB_ETH_EEM=y
CONFIG_USB_G_NCM=m
CONFIG_USB_GADGETFS=m
CONFIG_USB_FUNCTIONFS=m
CONFIG_USB_FUNCTIONFS_ETH=y
CONFIG_USB_FUNCTIONFS_RNDIS=y
CONFIG_USB_FUNCTIONFS_GENERIC=y
CONFIG_USB_MASS_STORAGE=m
CONFIG_USB_GADGET_TARGET=m
CONFIG_USB_G_SERIAL=m
CONFIG_USB_MIDI_GADGET=m
CONFIG_USB_G_PRINTER=m
CONFIG_USB_CDC_COMPOSITE=m
CONFIG_USB_G_NOKIA=m
CONFIG_USB_G_ACM_MS=m
# CONFIG_USB_G_MULTI is not set
CONFIG_USB_G_HID=m
CONFIG_USB_G_DBGP=m
# CONFIG_USB_G_DBGP_PRINTK is not set
CONFIG_USB_G_DBGP_SERIAL=y
CONFIG_USB_G_WEBCAM=m
# CONFIG_USB_RAW_GADGET is not set
# end of USB Gadget precomposed configurations
CONFIG_TYPEC=m
CONFIG_TYPEC_TCPM=m
CONFIG_TYPEC_TCPCI=m
# CONFIG_TYPEC_RT1711H is not set
# CONFIG_TYPEC_TCPCI_MAXIM is not set
CONFIG_TYPEC_FUSB302=m
CONFIG_TYPEC_UCSI=m
# CONFIG_UCSI_CCG is not set
CONFIG_UCSI_ACPI=m
# CONFIG_TYPEC_HD3SS3220 is not set
CONFIG_TYPEC_TPS6598X=m
# CONFIG_TYPEC_STUSB160X is not set
# CONFIG_TYPEC_QCOM_PMIC is not set
#
# USB Type-C Multiplexer/DeMultiplexer Switch support
#
# CONFIG_TYPEC_MUX_PI3USB30532 is not set
# end of USB Type-C Multiplexer/DeMultiplexer Switch support
#
# USB Type-C Alternate Mode drivers
#
# CONFIG_TYPEC_DP_ALTMODE is not set
# end of USB Type-C Alternate Mode drivers
CONFIG_USB_ROLE_SWITCH=y
CONFIG_MMC=y
CONFIG_PWRSEQ_EMMC=m
CONFIG_PWRSEQ_SD8787=m
CONFIG_PWRSEQ_SIMPLE=m
CONFIG_MMC_BLOCK=y
CONFIG_MMC_BLOCK_MINORS=8
CONFIG_SDIO_UART=m
# CONFIG_MMC_TEST is not set
#
# MMC/SD/SDIO Host Controller Drivers
#
# CONFIG_MMC_DEBUG is not set
CONFIG_MMC_ARMMMCI=y
CONFIG_MMC_QCOM_DML=y
CONFIG_MMC_STM32_SDMMC=y
CONFIG_MMC_SDHCI=m
CONFIG_MMC_SDHCI_IO_ACCESSORS=y
CONFIG_MMC_SDHCI_PCI=m
CONFIG_MMC_RICOH_MMC=y
CONFIG_MMC_SDHCI_ACPI=m
CONFIG_MMC_SDHCI_PLTFM=m
CONFIG_MMC_SDHCI_OF_ARASAN=m
# CONFIG_MMC_SDHCI_OF_ASPEED is not set
CONFIG_MMC_SDHCI_OF_AT91=m
CONFIG_MMC_SDHCI_OF_ESDHC=m
# CONFIG_MMC_SDHCI_OF_DWCMSHC is not set
CONFIG_MMC_SDHCI_CADENCE=m
CONFIG_MMC_SDHCI_PXAV3=m
CONFIG_MMC_SDHCI_F_SDH30=m
# CONFIG_MMC_SDHCI_MILBEAUT is not set
CONFIG_MMC_SDHCI_IPROC=m
CONFIG_MMC_SDHCI_MSM=m
CONFIG_MMC_TIFM_SD=m
CONFIG_MMC_SPI=m
# CONFIG_MMC_SDHCI_SPRD is not set
CONFIG_MMC_TMIO_CORE=m
CONFIG_MMC_SDHI=m
CONFIG_MMC_SDHI_SYS_DMAC=m
CONFIG_MMC_SDHI_INTERNAL_DMAC=m
CONFIG_MMC_CB710=m
CONFIG_MMC_VIA_SDMMC=m
CONFIG_MMC_CAVIUM_THUNDERX=m
CONFIG_MMC_DW=m
CONFIG_MMC_DW_PLTFM=m
# CONFIG_MMC_DW_BLUEFIELD is not set
CONFIG_MMC_DW_EXYNOS=m
# CONFIG_MMC_DW_HI3798CV200 is not set
CONFIG_MMC_DW_K3=m
CONFIG_MMC_DW_PCI=m
CONFIG_MMC_SH_MMCIF=m
CONFIG_MMC_VUB300=m
CONFIG_MMC_USHC=m
CONFIG_MMC_USDHI6ROL0=m
CONFIG_MMC_REALTEK_PCI=m
CONFIG_MMC_REALTEK_USB=m
CONFIG_MMC_SUNXI=m
CONFIG_MMC_CQHCI=m
# CONFIG_MMC_HSQ is not set
CONFIG_MMC_TOSHIBA_PCI=m
CONFIG_MMC_BCM2835=m
CONFIG_MMC_MTK=m
CONFIG_MMC_SDHCI_BRCMSTB=m
CONFIG_MMC_SDHCI_XENON=m
CONFIG_MMC_SDHCI_OMAP=m
# CONFIG_MMC_SDHCI_AM654 is not set
# CONFIG_MMC_OWL is not set
CONFIG_MMC_SDHCI_EXTERNAL_DMA=y
CONFIG_MEMSTICK=m
# CONFIG_MEMSTICK_DEBUG is not set
#
# MemoryStick drivers
#
# CONFIG_MEMSTICK_UNSAFE_RESUME is not set
CONFIG_MSPRO_BLOCK=m
CONFIG_MS_BLOCK=m
#
# MemoryStick Host Controller Drivers
#
CONFIG_MEMSTICK_TIFM_MS=m
CONFIG_MEMSTICK_JMICRON_38X=m
CONFIG_MEMSTICK_R592=m
CONFIG_MEMSTICK_REALTEK_PCI=m
CONFIG_MEMSTICK_REALTEK_USB=m
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
CONFIG_LEDS_CLASS_FLASH=m
# CONFIG_LEDS_CLASS_MULTICOLOR is not set
CONFIG_LEDS_BRIGHTNESS_HW_CHANGED=y
#
# LED drivers
#
CONFIG_LEDS_88PM860X=m
CONFIG_LEDS_AAT1290=m
# CONFIG_LEDS_AN30259A is not set
CONFIG_LEDS_AS3645A=m
# CONFIG_LEDS_AW2013 is not set
CONFIG_LEDS_BCM6328=m
CONFIG_LEDS_BCM6358=m
CONFIG_LEDS_CPCAP=m
# CONFIG_LEDS_CR0014114 is not set
# CONFIG_LEDS_EL15203000 is not set
CONFIG_LEDS_LM3530=m
# CONFIG_LEDS_LM3532 is not set
CONFIG_LEDS_LM3533=m
CONFIG_LEDS_LM3642=m
# CONFIG_LEDS_LM3692X is not set
# CONFIG_LEDS_LM3601X is not set
CONFIG_LEDS_MT6323=m
CONFIG_LEDS_PCA9532=m
CONFIG_LEDS_PCA9532_GPIO=y
CONFIG_LEDS_GPIO=m
CONFIG_LEDS_LP3944=m
CONFIG_LEDS_LP3952=m
# CONFIG_LEDS_LP50XX is not set
CONFIG_LEDS_LP55XX_COMMON=m
CONFIG_LEDS_LP5521=m
CONFIG_LEDS_LP5523=m
CONFIG_LEDS_LP5562=m
CONFIG_LEDS_LP8501=m
CONFIG_LEDS_LP8788=m
CONFIG_LEDS_LP8860=m
CONFIG_LEDS_PCA955X=m
CONFIG_LEDS_PCA955X_GPIO=y
CONFIG_LEDS_PCA963X=m
CONFIG_LEDS_WM831X_STATUS=m
CONFIG_LEDS_WM8350=m
CONFIG_LEDS_DA903X=m
CONFIG_LEDS_DA9052=m
CONFIG_LEDS_DAC124S085=m
CONFIG_LEDS_PWM=m
CONFIG_LEDS_REGULATOR=m
CONFIG_LEDS_BD2802=m
CONFIG_LEDS_LT3593=m
CONFIG_LEDS_ADP5520=m
CONFIG_LEDS_MC13783=m
CONFIG_LEDS_TCA6507=m
CONFIG_LEDS_TLC591XX=m
CONFIG_LEDS_MAX77693=m
CONFIG_LEDS_MAX8997=m
CONFIG_LEDS_LM355x=m
CONFIG_LEDS_MENF21BMC=m
CONFIG_LEDS_KTD2692=m
CONFIG_LEDS_IS31FL319X=m
CONFIG_LEDS_IS31FL32XX=m
# CONFIG_LEDS_SC27XX_BLTC is not set
#
# LED driver for blink(1) USB RGB LED is under Special HID drivers (HID_THINGM)
#
CONFIG_LEDS_BLINKM=m
CONFIG_LEDS_SYSCON=y
# CONFIG_LEDS_MLXREG is not set
CONFIG_LEDS_USER=m
# CONFIG_LEDS_SPI_BYTE is not set
# CONFIG_LEDS_TI_LMU_COMMON is not set
CONFIG_LEDS_TPS6105X=m
# CONFIG_LEDS_SGM3140 is not set
#
# LED Triggers
#
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_ONESHOT=m
CONFIG_LEDS_TRIGGER_DISK=y
CONFIG_LEDS_TRIGGER_MTD=y
CONFIG_LEDS_TRIGGER_HEARTBEAT=m
CONFIG_LEDS_TRIGGER_BACKLIGHT=m
CONFIG_LEDS_TRIGGER_CPU=y
CONFIG_LEDS_TRIGGER_ACTIVITY=m
CONFIG_LEDS_TRIGGER_GPIO=m
CONFIG_LEDS_TRIGGER_DEFAULT_ON=m
#
# iptables trigger is under Netfilter config (LED target)
#
CONFIG_LEDS_TRIGGER_TRANSIENT=m
CONFIG_LEDS_TRIGGER_CAMERA=m
CONFIG_LEDS_TRIGGER_PANIC=y
# CONFIG_LEDS_TRIGGER_NETDEV is not set
# CONFIG_LEDS_TRIGGER_PATTERN is not set
CONFIG_LEDS_TRIGGER_AUDIO=m
# CONFIG_ACCESSIBILITY is not set
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_INFINIBAND_USER_ACCESS=m
CONFIG_INFINIBAND_USER_MEM=y
CONFIG_INFINIBAND_ON_DEMAND_PAGING=y
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_ADDR_TRANS_CONFIGFS=y
CONFIG_INFINIBAND_VIRT_DMA=y
CONFIG_INFINIBAND_MTHCA=m
# CONFIG_INFINIBAND_MTHCA_DEBUG is not set
CONFIG_INFINIBAND_CXGB4=m
# CONFIG_INFINIBAND_EFA is not set
CONFIG_INFINIBAND_I40IW=m
CONFIG_MLX4_INFINIBAND=m
CONFIG_MLX5_INFINIBAND=m
CONFIG_INFINIBAND_OCRDMA=m
CONFIG_INFINIBAND_VMWARE_PVRDMA=m
CONFIG_INFINIBAND_HNS=m
# CONFIG_INFINIBAND_HNS_HIP06 is not set
# CONFIG_INFINIBAND_HNS_HIP08 is not set
CONFIG_INFINIBAND_BNXT_RE=m
CONFIG_INFINIBAND_QEDR=m
CONFIG_RDMA_RXE=m
# CONFIG_RDMA_SIW is not set
CONFIG_INFINIBAND_IPOIB=m
CONFIG_INFINIBAND_IPOIB_CM=y
# CONFIG_INFINIBAND_IPOIB_DEBUG is not set
CONFIG_INFINIBAND_SRP=m
CONFIG_INFINIBAND_SRPT=m
CONFIG_INFINIBAND_ISER=m
CONFIG_INFINIBAND_ISERT=m
# CONFIG_INFINIBAND_RTRS_CLIENT is not set
# CONFIG_INFINIBAND_RTRS_SERVER is not set
CONFIG_EDAC_SUPPORT=y
CONFIG_EDAC=y
# CONFIG_EDAC_LEGACY_SYSFS is not set
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_GHES=y
CONFIG_EDAC_LAYERSCAPE=m
CONFIG_EDAC_THUNDERX=m
# CONFIG_EDAC_ALTERA is not set
# CONFIG_EDAC_SYNOPSYS is not set
CONFIG_EDAC_XGENE=m
# CONFIG_EDAC_DMC520 is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
CONFIG_RTC_SYSTOHC=y
CONFIG_RTC_SYSTOHC_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set
CONFIG_RTC_NVMEM=y
#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set
#
# I2C RTC drivers
#
CONFIG_RTC_DRV_88PM860X=m
CONFIG_RTC_DRV_88PM80X=m
CONFIG_RTC_DRV_ABB5ZES3=m
# CONFIG_RTC_DRV_ABEOZ9 is not set
CONFIG_RTC_DRV_ABX80X=m
CONFIG_RTC_DRV_BRCMSTB=m
CONFIG_RTC_DRV_AS3722=m
CONFIG_RTC_DRV_DS1307=m
CONFIG_RTC_DRV_DS1307_CENTURY=y
CONFIG_RTC_DRV_DS1374=m
CONFIG_RTC_DRV_DS1374_WDT=y
CONFIG_RTC_DRV_DS1672=m
CONFIG_RTC_DRV_HYM8563=m
CONFIG_RTC_DRV_LP8788=m
CONFIG_RTC_DRV_MAX6900=m
CONFIG_RTC_DRV_MAX8907=m
CONFIG_RTC_DRV_MAX8925=m
CONFIG_RTC_DRV_MAX8998=m
CONFIG_RTC_DRV_MAX8997=m
CONFIG_RTC_DRV_MAX77686=m
CONFIG_RTC_DRV_RK808=m
CONFIG_RTC_DRV_RS5C372=m
CONFIG_RTC_DRV_ISL1208=m
CONFIG_RTC_DRV_ISL12022=m
# CONFIG_RTC_DRV_ISL12026 is not set
CONFIG_RTC_DRV_X1205=m
CONFIG_RTC_DRV_PCF8523=m
CONFIG_RTC_DRV_PCF85063=m
CONFIG_RTC_DRV_PCF85363=m
CONFIG_RTC_DRV_PCF8563=m
CONFIG_RTC_DRV_PCF8583=m
CONFIG_RTC_DRV_M41T80=m
CONFIG_RTC_DRV_M41T80_WDT=y
CONFIG_RTC_DRV_BQ32K=m
CONFIG_RTC_DRV_TWL4030=m
CONFIG_RTC_DRV_PALMAS=m
CONFIG_RTC_DRV_TPS6586X=m
CONFIG_RTC_DRV_TPS65910=m
CONFIG_RTC_DRV_TPS80031=m
CONFIG_RTC_DRV_RC5T583=m
# CONFIG_RTC_DRV_RC5T619 is not set
CONFIG_RTC_DRV_S35390A=m
CONFIG_RTC_DRV_FM3130=m
CONFIG_RTC_DRV_RX8010=m
CONFIG_RTC_DRV_RX8581=m
CONFIG_RTC_DRV_RX8025=m
CONFIG_RTC_DRV_EM3027=m
# CONFIG_RTC_DRV_RV3028 is not set
# CONFIG_RTC_DRV_RV3032 is not set
CONFIG_RTC_DRV_RV8803=m
CONFIG_RTC_DRV_S5M=m
# CONFIG_RTC_DRV_SD3078 is not set
#
# SPI RTC drivers
#
CONFIG_RTC_DRV_M41T93=m
CONFIG_RTC_DRV_M41T94=m
CONFIG_RTC_DRV_DS1302=m
CONFIG_RTC_DRV_DS1305=m
CONFIG_RTC_DRV_DS1343=m
CONFIG_RTC_DRV_DS1347=m
CONFIG_RTC_DRV_DS1390=m
CONFIG_RTC_DRV_MAX6916=m
CONFIG_RTC_DRV_R9701=m
CONFIG_RTC_DRV_RX4581=m
CONFIG_RTC_DRV_RS5C348=m
CONFIG_RTC_DRV_MAX6902=m
CONFIG_RTC_DRV_PCF2123=m
CONFIG_RTC_DRV_MCP795=m
CONFIG_RTC_I2C_AND_SPI=y
#
# SPI and I2C RTC drivers
#
CONFIG_RTC_DRV_DS3232=m
CONFIG_RTC_DRV_DS3232_HWMON=y
CONFIG_RTC_DRV_PCF2127=m
CONFIG_RTC_DRV_RV3029C2=m
CONFIG_RTC_DRV_RV3029_HWMON=y
CONFIG_RTC_DRV_RX6110=m
#
# Platform RTC drivers
#
CONFIG_RTC_DRV_DS1286=m
CONFIG_RTC_DRV_DS1511=m
CONFIG_RTC_DRV_DS1553=m
CONFIG_RTC_DRV_DS1685_FAMILY=m
CONFIG_RTC_DRV_DS1685=y
# CONFIG_RTC_DRV_DS1689 is not set
# CONFIG_RTC_DRV_DS17285 is not set
# CONFIG_RTC_DRV_DS17485 is not set
# CONFIG_RTC_DRV_DS17885 is not set
CONFIG_RTC_DRV_DS1742=m
CONFIG_RTC_DRV_DS2404=m
CONFIG_RTC_DRV_DA9052=m
CONFIG_RTC_DRV_DA9055=m
CONFIG_RTC_DRV_DA9063=m
CONFIG_RTC_DRV_EFI=y
CONFIG_RTC_DRV_STK17TA8=m
CONFIG_RTC_DRV_M48T86=m
CONFIG_RTC_DRV_M48T35=m
CONFIG_RTC_DRV_M48T59=m
CONFIG_RTC_DRV_MSM6242=m
CONFIG_RTC_DRV_BQ4802=m
CONFIG_RTC_DRV_RP5C01=m
CONFIG_RTC_DRV_V3020=m
CONFIG_RTC_DRV_WM831X=m
CONFIG_RTC_DRV_WM8350=m
CONFIG_RTC_DRV_SC27XX=m
CONFIG_RTC_DRV_PCF50633=m
CONFIG_RTC_DRV_AB3100=m
CONFIG_RTC_DRV_ZYNQMP=m
# CONFIG_RTC_DRV_CROS_EC is not set
#
# on-CPU RTC drivers
#
# CONFIG_RTC_DRV_FSL_FTM_ALARM is not set
CONFIG_RTC_DRV_SH=m
CONFIG_RTC_DRV_PL030=m
CONFIG_RTC_DRV_PL031=m
CONFIG_RTC_DRV_SUN6I=y
# CONFIG_RTC_DRV_CADENCE is not set
CONFIG_RTC_DRV_FTRTC010=m
CONFIG_RTC_DRV_PCAP=m
CONFIG_RTC_DRV_MC13XXX=m
CONFIG_RTC_DRV_PM8XXX=m
# CONFIG_RTC_DRV_MT2712 is not set
CONFIG_RTC_DRV_MT6397=m
CONFIG_RTC_DRV_MT7622=m
CONFIG_RTC_DRV_XGENE=y
CONFIG_RTC_DRV_R7301=m
CONFIG_RTC_DRV_CPCAP=m
CONFIG_RTC_DRV_RTD119X=y
#
# HID Sensor RTC drivers
#
CONFIG_RTC_DRV_HID_SENSOR_TIME=m
# CONFIG_RTC_DRV_GOLDFISH is not set
CONFIG_DMADEVICES=y
# CONFIG_DMADEVICES_DEBUG is not set
#
# DMA Devices
#
CONFIG_ASYNC_TX_ENABLE_CHANNEL_SWITCH=y
CONFIG_DMA_ENGINE=y
CONFIG_DMA_VIRTUAL_CHANNELS=y
CONFIG_DMA_ACPI=y
CONFIG_DMA_OF=y
CONFIG_ALTERA_MSGDMA=m
CONFIG_AMBA_PL08X=y
# CONFIG_AXI_DMAC is not set
CONFIG_BCM_SBA_RAID=m
CONFIG_DMA_BCM2835=y
CONFIG_DMA_SUN6I=m
# CONFIG_DW_AXI_DMAC is not set
CONFIG_FSL_EDMA=m
# CONFIG_FSL_QDMA is not set
# CONFIG_HISI_DMA is not set
CONFIG_INTEL_IDMA64=m
CONFIG_K3_DMA=m
CONFIG_MV_XOR_V2=y
# CONFIG_OWL_DMA is not set
CONFIG_PL330_DMA=m
# CONFIG_PLX_DMA is not set
CONFIG_SPRD_DMA=m
CONFIG_XGENE_DMA=m
CONFIG_XILINX_DMA=m
CONFIG_XILINX_ZYNQMP_DMA=m
# CONFIG_XILINX_ZYNQMP_DPDMA is not set
# CONFIG_MTK_HSDMA is not set
# CONFIG_MTK_CQDMA is not set
# CONFIG_MTK_UART_APDMA is not set
CONFIG_QCOM_BAM_DMA=m
# CONFIG_QCOM_GPI_DMA is not set
CONFIG_QCOM_HIDMA_MGMT=m
CONFIG_QCOM_HIDMA=m
CONFIG_DW_DMAC_CORE=m
CONFIG_DW_DMAC=m
CONFIG_DW_DMAC_PCI=m
# CONFIG_DW_EDMA is not set
# CONFIG_DW_EDMA_PCIE is not set
# CONFIG_SF_PDMA is not set
CONFIG_RENESAS_DMA=y
CONFIG_RCAR_DMAC=m
CONFIG_RENESAS_USB_DMAC=m
# CONFIG_FSL_DPAA2_QDMA is not set
#
# DMA Clients
#
CONFIG_ASYNC_TX_DMA=y
# CONFIG_DMATEST is not set
CONFIG_DMA_ENGINE_RAID=y
#
# DMABUF options
#
CONFIG_SYNC_FILE=y
CONFIG_SW_SYNC=y
# CONFIG_UDMABUF is not set
# CONFIG_DMABUF_MOVE_NOTIFY is not set
# CONFIG_DMABUF_SELFTESTS is not set
# CONFIG_DMABUF_HEAPS is not set
# end of DMABUF options
CONFIG_AUXDISPLAY=y
CONFIG_CHARLCD=m
CONFIG_HD44780_COMMON=m
CONFIG_HD44780=m
CONFIG_IMG_ASCII_LCD=m
CONFIG_HT16K33=m
# CONFIG_LCD2S is not set
CONFIG_PARPORT_PANEL=m
CONFIG_PANEL_PARPORT=0
CONFIG_PANEL_PROFILE=5
# CONFIG_PANEL_CHANGE_MESSAGE is not set
# CONFIG_CHARLCD_BL_OFF is not set
# CONFIG_CHARLCD_BL_ON is not set
CONFIG_CHARLCD_BL_FLASH=y
CONFIG_PANEL=m
CONFIG_UIO=m
CONFIG_UIO_CIF=m
CONFIG_UIO_PDRV_GENIRQ=m
CONFIG_UIO_DMEM_GENIRQ=m
CONFIG_UIO_AEC=m
CONFIG_UIO_SERCOS3=m
CONFIG_UIO_PCI_GENERIC=m
CONFIG_UIO_NETX=m
CONFIG_UIO_PRUSS=m
CONFIG_UIO_MF624=m
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO_VIRQFD=m
CONFIG_VFIO=m
CONFIG_VFIO_NOIOMMU=y
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PLATFORM=m
CONFIG_VFIO_AMBA=m
CONFIG_VFIO_PLATFORM_CALXEDAXGMAC_RESET=m
CONFIG_VFIO_PLATFORM_AMDXGBE_RESET=m
CONFIG_VFIO_PLATFORM_BCMFLEXRM_RESET=m
CONFIG_VFIO_MDEV=m
CONFIG_VFIO_MDEV_DEVICE=m
# CONFIG_VFIO_FSL_MC is not set
CONFIG_VIRT_DRIVERS=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_MENU=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_PCI_LEGACY=y
# CONFIG_VIRTIO_PMEM is not set
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_INPUT=m
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
CONFIG_VIRTIO_DMA_SHARED_BUFFER=m
# CONFIG_VDPA is not set
CONFIG_VHOST_IOTLB=m
CONFIG_VHOST=m
CONFIG_VHOST_MENU=y
CONFIG_VHOST_NET=m
CONFIG_VHOST_SCSI=m
CONFIG_VHOST_VSOCK=m
# CONFIG_VHOST_CROSS_ENDIAN_LEGACY is not set
#
# Microsoft Hyper-V guest support
#
# end of Microsoft Hyper-V guest support
#
# Xen driver support
#
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES_DEFAULT=y
CONFIG_XEN_DEV_EVTCHN=m
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=m
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=m
CONFIG_XEN_GRANT_DEV_ALLOC=m
# CONFIG_XEN_GRANT_DMA_ALLOC is not set
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_PVCALLS_FRONTEND=m
# CONFIG_XEN_PVCALLS_BACKEND is not set
CONFIG_XEN_SCSI_BACKEND=m
CONFIG_XEN_PRIVCMD=m
CONFIG_XEN_EFI=y
CONFIG_XEN_AUTO_XLATE=y
# end of Xen driver support
CONFIG_GREYBUS=m
CONFIG_GREYBUS_ES2=m
CONFIG_STAGING=y
CONFIG_PRISM2_USB=m
CONFIG_COMEDI=m
# CONFIG_COMEDI_DEBUG is not set
CONFIG_COMEDI_DEFAULT_BUF_SIZE_KB=2048
CONFIG_COMEDI_DEFAULT_BUF_MAXSIZE_KB=20480
CONFIG_COMEDI_MISC_DRIVERS=y
CONFIG_COMEDI_BOND=m
CONFIG_COMEDI_TEST=m
CONFIG_COMEDI_PARPORT=m
CONFIG_COMEDI_ISA_DRIVERS=y
CONFIG_COMEDI_PCL711=m
CONFIG_COMEDI_PCL724=m
CONFIG_COMEDI_PCL726=m
CONFIG_COMEDI_PCL730=m
CONFIG_COMEDI_PCL812=m
CONFIG_COMEDI_PCL816=m
CONFIG_COMEDI_PCL818=m
CONFIG_COMEDI_PCM3724=m
CONFIG_COMEDI_AMPLC_DIO200_ISA=m
CONFIG_COMEDI_AMPLC_PC236_ISA=m
CONFIG_COMEDI_AMPLC_PC263_ISA=m
CONFIG_COMEDI_RTI800=m
CONFIG_COMEDI_RTI802=m
CONFIG_COMEDI_DAC02=m
CONFIG_COMEDI_DAS16M1=m
CONFIG_COMEDI_DAS08_ISA=m
CONFIG_COMEDI_DAS16=m
CONFIG_COMEDI_DAS800=m
CONFIG_COMEDI_DAS1800=m
CONFIG_COMEDI_DAS6402=m
CONFIG_COMEDI_DT2801=m
CONFIG_COMEDI_DT2811=m
CONFIG_COMEDI_DT2814=m
CONFIG_COMEDI_DT2815=m
CONFIG_COMEDI_DT2817=m
CONFIG_COMEDI_DT282X=m
CONFIG_COMEDI_DMM32AT=m
CONFIG_COMEDI_FL512=m
CONFIG_COMEDI_AIO_AIO12_8=m
CONFIG_COMEDI_AIO_IIRO_16=m
CONFIG_COMEDI_II_PCI20KC=m
CONFIG_COMEDI_C6XDIGIO=m
CONFIG_COMEDI_MPC624=m
CONFIG_COMEDI_ADQ12B=m
CONFIG_COMEDI_NI_AT_A2150=m
CONFIG_COMEDI_NI_AT_AO=m
CONFIG_COMEDI_NI_ATMIO=m
CONFIG_COMEDI_NI_ATMIO16D=m
CONFIG_COMEDI_NI_LABPC_ISA=m
CONFIG_COMEDI_PCMAD=m
CONFIG_COMEDI_PCMDA12=m
CONFIG_COMEDI_PCMMIO=m
CONFIG_COMEDI_PCMUIO=m
CONFIG_COMEDI_MULTIQ3=m
CONFIG_COMEDI_S526=m
CONFIG_COMEDI_PCI_DRIVERS=m
CONFIG_COMEDI_8255_PCI=m
CONFIG_COMEDI_ADDI_WATCHDOG=m
CONFIG_COMEDI_ADDI_APCI_1032=m
CONFIG_COMEDI_ADDI_APCI_1500=m
CONFIG_COMEDI_ADDI_APCI_1516=m
CONFIG_COMEDI_ADDI_APCI_1564=m
CONFIG_COMEDI_ADDI_APCI_16XX=m
CONFIG_COMEDI_ADDI_APCI_2032=m
CONFIG_COMEDI_ADDI_APCI_2200=m
CONFIG_COMEDI_ADDI_APCI_3120=m
CONFIG_COMEDI_ADDI_APCI_3501=m
CONFIG_COMEDI_ADDI_APCI_3XXX=m
CONFIG_COMEDI_ADL_PCI6208=m
CONFIG_COMEDI_ADL_PCI7X3X=m
CONFIG_COMEDI_ADL_PCI8164=m
CONFIG_COMEDI_ADL_PCI9111=m
CONFIG_COMEDI_ADL_PCI9118=m
CONFIG_COMEDI_ADV_PCI1710=m
CONFIG_COMEDI_ADV_PCI1720=m
CONFIG_COMEDI_ADV_PCI1723=m
CONFIG_COMEDI_ADV_PCI1724=m
CONFIG_COMEDI_ADV_PCI1760=m
CONFIG_COMEDI_ADV_PCI_DIO=m
CONFIG_COMEDI_AMPLC_DIO200_PCI=m
CONFIG_COMEDI_AMPLC_PC236_PCI=m
CONFIG_COMEDI_AMPLC_PC263_PCI=m
CONFIG_COMEDI_AMPLC_PCI224=m
CONFIG_COMEDI_AMPLC_PCI230=m
CONFIG_COMEDI_CONTEC_PCI_DIO=m
CONFIG_COMEDI_DAS08_PCI=m
CONFIG_COMEDI_DT3000=m
CONFIG_COMEDI_DYNA_PCI10XX=m
CONFIG_COMEDI_GSC_HPDI=m
CONFIG_COMEDI_MF6X4=m
CONFIG_COMEDI_ICP_MULTI=m
CONFIG_COMEDI_DAQBOARD2000=m
CONFIG_COMEDI_JR3_PCI=m
CONFIG_COMEDI_KE_COUNTER=m
CONFIG_COMEDI_CB_PCIDAS64=m
CONFIG_COMEDI_CB_PCIDAS=m
CONFIG_COMEDI_CB_PCIDDA=m
CONFIG_COMEDI_CB_PCIMDAS=m
CONFIG_COMEDI_CB_PCIMDDA=m
CONFIG_COMEDI_ME4000=m
CONFIG_COMEDI_ME_DAQ=m
CONFIG_COMEDI_NI_6527=m
CONFIG_COMEDI_NI_65XX=m
CONFIG_COMEDI_NI_660X=m
CONFIG_COMEDI_NI_670X=m
CONFIG_COMEDI_NI_LABPC_PCI=m
CONFIG_COMEDI_NI_PCIDIO=m
CONFIG_COMEDI_NI_PCIMIO=m
CONFIG_COMEDI_RTD520=m
CONFIG_COMEDI_S626=m
CONFIG_COMEDI_MITE=m
CONFIG_COMEDI_NI_TIOCMD=m
CONFIG_COMEDI_USB_DRIVERS=m
CONFIG_COMEDI_DT9812=m
CONFIG_COMEDI_NI_USB6501=m
CONFIG_COMEDI_USBDUX=m
CONFIG_COMEDI_USBDUXFAST=m
CONFIG_COMEDI_USBDUXSIGMA=m
CONFIG_COMEDI_VMK80XX=m
CONFIG_COMEDI_8254=m
CONFIG_COMEDI_8255=m
CONFIG_COMEDI_8255_SA=m
CONFIG_COMEDI_KCOMEDILIB=m
CONFIG_COMEDI_AMPLC_DIO200=m
CONFIG_COMEDI_AMPLC_PC236=m
CONFIG_COMEDI_DAS08=m
CONFIG_COMEDI_NI_LABPC=m
CONFIG_COMEDI_NI_TIO=m
CONFIG_COMEDI_NI_ROUTING=m
CONFIG_RTL8192U=m
CONFIG_RTLLIB=m
CONFIG_RTLLIB_CRYPTO_CCMP=m
CONFIG_RTLLIB_CRYPTO_TKIP=m
CONFIG_RTLLIB_CRYPTO_WEP=m
CONFIG_RTL8192E=m
CONFIG_RTL8723BS=m
CONFIG_R8712U=m
CONFIG_R8188EU=m
CONFIG_88EU_AP_MODE=y
CONFIG_RTS5208=m
CONFIG_VT6655=m
CONFIG_VT6656=m
#
# IIO staging drivers
#
#
# Accelerometers
#
CONFIG_ADIS16203=m
CONFIG_ADIS16240=m
# end of Accelerometers
#
# Analog to digital converters
#
CONFIG_AD7816=m
CONFIG_AD7280=m
# end of Analog to digital converters
#
# Analog digital bi-direction converters
#
CONFIG_ADT7316=m
CONFIG_ADT7316_SPI=m
CONFIG_ADT7316_I2C=m
# end of Analog digital bi-direction converters
#
# Capacitance to digital converters
#
CONFIG_AD7150=m
CONFIG_AD7746=m
# end of Capacitance to digital converters
#
# Direct Digital Synthesis
#
CONFIG_AD9832=m
CONFIG_AD9834=m
# end of Direct Digital Synthesis
#
# Network Analyzer, Impedance Converters
#
CONFIG_AD5933=m
# end of Network Analyzer, Impedance Converters
#
# Active energy metering IC
#
CONFIG_ADE7854=m
CONFIG_ADE7854_I2C=m
CONFIG_ADE7854_SPI=m
# end of Active energy metering IC
#
# Resolver to digital converters
#
CONFIG_AD2S1210=m
# end of Resolver to digital converters
# end of IIO staging drivers
CONFIG_FB_SM750=m
CONFIG_USB_EMXX=m
CONFIG_STAGING_MEDIA=y
# CONFIG_VIDEO_ALLEGRO_DVT is not set
# CONFIG_VIDEO_SUNXI is not set
# CONFIG_VIDEO_ZORAN is not set
#
# Android
#
# end of Android
# CONFIG_STAGING_BOARD is not set
CONFIG_LTE_GDM724X=m
CONFIG_FIREWIRE_SERIAL=m
CONFIG_FWTTY_MAX_TOTAL_PORTS=64
CONFIG_FWTTY_MAX_CARD_PORTS=32
CONFIG_GS_FPGABOOT=m
# CONFIG_UNISYSSPAR is not set
CONFIG_FB_TFT=m
CONFIG_FB_TFT_AGM1264K_FL=m
CONFIG_FB_TFT_BD663474=m
CONFIG_FB_TFT_HX8340BN=m
CONFIG_FB_TFT_HX8347D=m
CONFIG_FB_TFT_HX8353D=m
CONFIG_FB_TFT_HX8357D=m
CONFIG_FB_TFT_ILI9163=m
CONFIG_FB_TFT_ILI9320=m
CONFIG_FB_TFT_ILI9325=m
CONFIG_FB_TFT_ILI9340=m
CONFIG_FB_TFT_ILI9341=m
CONFIG_FB_TFT_ILI9481=m
CONFIG_FB_TFT_ILI9486=m
CONFIG_FB_TFT_PCD8544=m
CONFIG_FB_TFT_RA8875=m
CONFIG_FB_TFT_S6D02A1=m
CONFIG_FB_TFT_S6D1121=m
# CONFIG_FB_TFT_SEPS525 is not set
CONFIG_FB_TFT_SH1106=m
CONFIG_FB_TFT_SSD1289=m
CONFIG_FB_TFT_SSD1305=m
CONFIG_FB_TFT_SSD1306=m
CONFIG_FB_TFT_SSD1331=m
CONFIG_FB_TFT_SSD1351=m
CONFIG_FB_TFT_ST7735R=m
CONFIG_FB_TFT_ST7789V=m
CONFIG_FB_TFT_TINYLCD=m
CONFIG_FB_TFT_TLS8204=m
CONFIG_FB_TFT_UC1611=m
CONFIG_FB_TFT_UC1701=m
CONFIG_FB_TFT_UPD161704=m
CONFIG_FB_TFT_WATTEROTT=m
CONFIG_FSL_DPAA2=y
# CONFIG_FSL_DPAA2_ETHSW is not set
# CONFIG_MOST_COMPONENTS is not set
CONFIG_KS7010=m
CONFIG_GREYBUS_AUDIO=m
# CONFIG_GREYBUS_AUDIO_APB_CODEC is not set
CONFIG_GREYBUS_BOOTROM=m
CONFIG_GREYBUS_FIRMWARE=m
CONFIG_GREYBUS_HID=m
CONFIG_GREYBUS_LIGHT=m
CONFIG_GREYBUS_LOG=m
CONFIG_GREYBUS_LOOPBACK=m
CONFIG_GREYBUS_POWER=m
CONFIG_GREYBUS_RAW=m
CONFIG_GREYBUS_VIBRATOR=m
CONFIG_GREYBUS_BRIDGED_PHY=m
CONFIG_GREYBUS_GPIO=m
CONFIG_GREYBUS_I2C=m
CONFIG_GREYBUS_PWM=m
CONFIG_GREYBUS_SDIO=m
CONFIG_GREYBUS_SPI=m
CONFIG_GREYBUS_UART=m
CONFIG_GREYBUS_USB=m
CONFIG_BCM_VIDEOCORE=m
CONFIG_BCM2835_VCHIQ=m
CONFIG_SND_BCM2835=m
CONFIG_VIDEO_BCM2835=m
CONFIG_BCM2835_VCHIQ_MMAL=m
CONFIG_PI433=m
#
# Gasket devices
#
# CONFIG_STAGING_GASKET_FRAMEWORK is not set
# end of Gasket devices
# CONFIG_XIL_AXIS_FIFO is not set
# CONFIG_FIELDBUS_DEV is not set
# CONFIG_KPC2000 is not set
CONFIG_QLGE=m
CONFIG_WIMAX=m
CONFIG_WIMAX_DEBUG_LEVEL=8
CONFIG_WIMAX_I2400M=m
CONFIG_WIMAX_I2400M_USB=m
CONFIG_WIMAX_I2400M_DEBUG_LEVEL=8
# CONFIG_WFX is not set
# CONFIG_PHY_HI3670_USB is not set
# CONFIG_SPMI_HISI3670 is not set
# CONFIG_MFD_HI6421_SPMI is not set
# CONFIG_GOLDFISH is not set
CONFIG_CHROME_PLATFORMS=y
# CONFIG_CHROMEOS_TBMC is not set
CONFIG_CROS_EC=m
# CONFIG_CROS_EC_I2C is not set
# CONFIG_CROS_EC_RPMSG is not set
# CONFIG_CROS_EC_SPI is not set
CONFIG_CROS_EC_PROTO=y
CONFIG_CROS_KBD_LED_BACKLIGHT=m
CONFIG_CROS_EC_CHARDEV=m
CONFIG_CROS_EC_LIGHTBAR=m
CONFIG_CROS_EC_VBC=m
CONFIG_CROS_EC_DEBUGFS=m
CONFIG_CROS_EC_SENSORHUB=m
CONFIG_CROS_EC_SYSFS=m
CONFIG_CROS_EC_TYPEC=m
CONFIG_CROS_USBPD_NOTIFY=m
# CONFIG_MELLANOX_PLATFORM is not set
CONFIG_SURFACE_PLATFORMS=y
# CONFIG_SURFACE_3_BUTTON is not set
# CONFIG_SURFACE_3_POWER_OPREGION is not set
# CONFIG_SURFACE_GPE is not set
# CONFIG_SURFACE_PRO3_BUTTON is not set
CONFIG_HAVE_CLK=y
CONFIG_CLKDEV_LOOKUP=y
CONFIG_HAVE_CLK_PREPARE=y
CONFIG_COMMON_CLK=y
CONFIG_COMMON_CLK_WM831X=m
#
# Clock driver for ARM Reference designs
#
# CONFIG_ICST is not set
CONFIG_CLK_SP810=y
CONFIG_CLK_VEXPRESS_OSC=y
# end of Clock driver for ARM Reference designs
CONFIG_COMMON_CLK_MAX77686=m
# CONFIG_COMMON_CLK_MAX9485 is not set
CONFIG_COMMON_CLK_RK808=m
CONFIG_COMMON_CLK_HI655X=m
CONFIG_COMMON_CLK_SCPI=m
# CONFIG_COMMON_CLK_SI5341 is not set
CONFIG_COMMON_CLK_SI5351=m
CONFIG_COMMON_CLK_SI514=m
# CONFIG_COMMON_CLK_SI544 is not set
CONFIG_COMMON_CLK_SI570=m
CONFIG_COMMON_CLK_CDCE706=m
CONFIG_COMMON_CLK_CDCE925=m
CONFIG_COMMON_CLK_CS2000_CP=m
# CONFIG_COMMON_CLK_FSL_FLEXSPI is not set
# CONFIG_COMMON_CLK_FSL_SAI is not set
CONFIG_COMMON_CLK_S2MPS11=m
CONFIG_CLK_TWL6040=m
CONFIG_CLK_QORIQ=y
CONFIG_CLK_LS1028A_PLLDIG=y
CONFIG_COMMON_CLK_XGENE=y
CONFIG_COMMON_CLK_PALMAS=m
CONFIG_COMMON_CLK_PWM=m
CONFIG_COMMON_CLK_VC5=m
# CONFIG_COMMON_CLK_FIXED_MMIO is not set
CONFIG_CLK_ACTIONS=y
CONFIG_CLK_OWL_S500=y
CONFIG_CLK_OWL_S700=y
CONFIG_CLK_OWL_S900=y
CONFIG_CLK_BCM2711_DVP=y
CONFIG_CLK_BCM2835=y
CONFIG_COMMON_CLK_IPROC=y
CONFIG_CLK_BCM_NS2=y
CONFIG_CLK_BCM_SR=y
# CONFIG_CLK_RASPBERRYPI is not set
CONFIG_COMMON_CLK_HI3516CV300=m
CONFIG_COMMON_CLK_HI3519=m
CONFIG_COMMON_CLK_HI3660=y
CONFIG_COMMON_CLK_HI3670=y
CONFIG_COMMON_CLK_HI3798CV200=m
CONFIG_COMMON_CLK_HI6220=y
CONFIG_RESET_HISI=y
CONFIG_STUB_CLK_HI6220=y
CONFIG_STUB_CLK_HI3660=y
#
# Clock driver for MediaTek SoC
#
CONFIG_COMMON_CLK_MEDIATEK=y
CONFIG_COMMON_CLK_MT2712=y
CONFIG_COMMON_CLK_MT2712_BDPSYS=y
CONFIG_COMMON_CLK_MT2712_IMGSYS=y
CONFIG_COMMON_CLK_MT2712_JPGDECSYS=y
CONFIG_COMMON_CLK_MT2712_MFGCFG=y
CONFIG_COMMON_CLK_MT2712_MMSYS=y
CONFIG_COMMON_CLK_MT2712_VDECSYS=y
CONFIG_COMMON_CLK_MT2712_VENCSYS=y
CONFIG_COMMON_CLK_MT6765=y
# CONFIG_COMMON_CLK_MT6765_AUDIOSYS is not set
# CONFIG_COMMON_CLK_MT6765_CAMSYS is not set
# CONFIG_COMMON_CLK_MT6765_GCESYS is not set
# CONFIG_COMMON_CLK_MT6765_MMSYS is not set
# CONFIG_COMMON_CLK_MT6765_IMGSYS is not set
# CONFIG_COMMON_CLK_MT6765_VCODECSYS is not set
# CONFIG_COMMON_CLK_MT6765_MFGSYS is not set
# CONFIG_COMMON_CLK_MT6765_MIPI0ASYS is not set
# CONFIG_COMMON_CLK_MT6765_MIPI0BSYS is not set
# CONFIG_COMMON_CLK_MT6765_MIPI1ASYS is not set
# CONFIG_COMMON_CLK_MT6765_MIPI1BSYS is not set
# CONFIG_COMMON_CLK_MT6765_MIPI2ASYS is not set
# CONFIG_COMMON_CLK_MT6765_MIPI2BSYS is not set
CONFIG_COMMON_CLK_MT6779=y
# CONFIG_COMMON_CLK_MT6779_MMSYS is not set
# CONFIG_COMMON_CLK_MT6779_IMGSYS is not set
# CONFIG_COMMON_CLK_MT6779_IPESYS is not set
# CONFIG_COMMON_CLK_MT6779_CAMSYS is not set
# CONFIG_COMMON_CLK_MT6779_VDECSYS is not set
# CONFIG_COMMON_CLK_MT6779_VENCSYS is not set
# CONFIG_COMMON_CLK_MT6779_MFGCFG is not set
# CONFIG_COMMON_CLK_MT6779_AUDSYS is not set
CONFIG_COMMON_CLK_MT6797=y
CONFIG_COMMON_CLK_MT6797_MMSYS=y
CONFIG_COMMON_CLK_MT6797_IMGSYS=y
CONFIG_COMMON_CLK_MT6797_VDECSYS=y
CONFIG_COMMON_CLK_MT6797_VENCSYS=y
CONFIG_COMMON_CLK_MT7622=y
CONFIG_COMMON_CLK_MT7622_ETHSYS=y
CONFIG_COMMON_CLK_MT7622_HIFSYS=y
CONFIG_COMMON_CLK_MT7622_AUDSYS=y
CONFIG_COMMON_CLK_MT8167=y
CONFIG_COMMON_CLK_MT8167_AUDSYS=y
CONFIG_COMMON_CLK_MT8167_IMGSYS=y
CONFIG_COMMON_CLK_MT8167_MFGCFG=y
CONFIG_COMMON_CLK_MT8167_MMSYS=y
CONFIG_COMMON_CLK_MT8167_VDECSYS=y
CONFIG_COMMON_CLK_MT8173=y
CONFIG_COMMON_CLK_MT8173_MMSYS=y
CONFIG_COMMON_CLK_MT8183=y
# CONFIG_COMMON_CLK_MT8183_AUDIOSYS is not set
# CONFIG_COMMON_CLK_MT8183_CAMSYS is not set
# CONFIG_COMMON_CLK_MT8183_IMGSYS is not set
# CONFIG_COMMON_CLK_MT8183_IPU_CORE0 is not set
# CONFIG_COMMON_CLK_MT8183_IPU_CORE1 is not set
# CONFIG_COMMON_CLK_MT8183_IPU_ADL is not set
# CONFIG_COMMON_CLK_MT8183_IPU_CONN is not set
# CONFIG_COMMON_CLK_MT8183_MFGCFG is not set
# CONFIG_COMMON_CLK_MT8183_MMSYS is not set
# CONFIG_COMMON_CLK_MT8183_VDECSYS is not set
# CONFIG_COMMON_CLK_MT8183_VENCSYS is not set
CONFIG_COMMON_CLK_MT8516=y
# CONFIG_COMMON_CLK_MT8516_AUDSYS is not set
# end of Clock driver for MediaTek SoC
CONFIG_QCOM_GDSC=y
CONFIG_QCOM_RPMCC=y
CONFIG_COMMON_CLK_QCOM=m
CONFIG_QCOM_A53PLL=m
CONFIG_QCOM_CLK_APCS_MSM8916=m
# CONFIG_QCOM_CLK_APCC_MSM8996 is not set
CONFIG_QCOM_CLK_RPM=m
CONFIG_QCOM_CLK_SMD_RPM=m
CONFIG_APQ_GCC_8084=m
CONFIG_APQ_MMCC_8084=m
# CONFIG_IPQ_APSS_PLL is not set
# CONFIG_IPQ_APSS_6018 is not set
CONFIG_IPQ_GCC_4019=m
# CONFIG_IPQ_GCC_6018 is not set
CONFIG_IPQ_GCC_806X=m
CONFIG_IPQ_LCC_806X=m
CONFIG_IPQ_GCC_8074=m
CONFIG_MSM_GCC_8660=m
CONFIG_MSM_GCC_8916=m
# CONFIG_MSM_GCC_8939 is not set
CONFIG_MSM_GCC_8960=m
CONFIG_MSM_LCC_8960=m
CONFIG_MDM_GCC_9615=m
CONFIG_MDM_LCC_9615=m
CONFIG_MSM_MMCC_8960=m
CONFIG_MSM_GCC_8974=m
CONFIG_MSM_MMCC_8974=m
CONFIG_MSM_GCC_8994=m
CONFIG_MSM_GCC_8996=m
CONFIG_MSM_MMCC_8996=m
# CONFIG_MSM_GCC_8998 is not set
# CONFIG_MSM_GPUCC_8998 is not set
# CONFIG_MSM_MMCC_8998 is not set
# CONFIG_QCS_GCC_404 is not set
# CONFIG_SC_CAMCC_7180 is not set
# CONFIG_SC_DISPCC_7180 is not set
# CONFIG_SC_GCC_7180 is not set
# CONFIG_SC_LPASS_CORECC_7180 is not set
# CONFIG_SC_GPUCC_7180 is not set
# CONFIG_SC_MSS_7180 is not set
# CONFIG_SC_VIDEOCC_7180 is not set
# CONFIG_SDM_CAMCC_845 is not set
# CONFIG_SDM_GCC_660 is not set
# CONFIG_QCS_TURING_404 is not set
# CONFIG_QCS_Q6SSTOP_404 is not set
# CONFIG_SDM_GCC_845 is not set
# CONFIG_SDM_GPUCC_845 is not set
# CONFIG_SDM_VIDEOCC_845 is not set
# CONFIG_SDM_DISPCC_845 is not set
# CONFIG_SDM_LPASSCC_845 is not set
# CONFIG_SDX_GCC_55 is not set
# CONFIG_SM_GCC_8150 is not set
# CONFIG_SM_GCC_8250 is not set
# CONFIG_SM_GPUCC_8150 is not set
# CONFIG_SM_GPUCC_8250 is not set
# CONFIG_SM_VIDEOCC_8150 is not set
# CONFIG_SM_VIDEOCC_8250 is not set
# CONFIG_SPMI_PMIC_CLKDIV is not set
# CONFIG_QCOM_HFPLL is not set
# CONFIG_KPSS_XCC is not set
# CONFIG_CLK_GFM_LPASS_SM8250 is not set
CONFIG_CLK_RENESAS=y
CONFIG_CLK_R8A7795=y
CONFIG_CLK_R8A77970=y
CONFIG_CLK_R8A77995=y
# CONFIG_CLK_R9A06G032 is not set
CONFIG_CLK_RCAR_GEN3_CPG=y
CONFIG_CLK_RCAR_USB2_CLOCK_SEL=y
CONFIG_CLK_RENESAS_CPG_MSSR=y
CONFIG_CLK_RENESAS_DIV6=y
CONFIG_SPRD_COMMON_CLK=y
CONFIG_SPRD_SC9860_CLK=y
CONFIG_SPRD_SC9863A_CLK=y
CONFIG_CLK_SUNXI=y
CONFIG_CLK_SUNXI_CLOCKS=y
CONFIG_CLK_SUNXI_PRCM_SUN6I=y
CONFIG_CLK_SUNXI_PRCM_SUN8I=y
CONFIG_CLK_SUNXI_PRCM_SUN9I=y
CONFIG_SUNXI_CCU=y
CONFIG_SUN50I_A64_CCU=y
CONFIG_SUN50I_A100_CCU=y
CONFIG_SUN50I_A100_R_CCU=y
CONFIG_SUN50I_H6_CCU=y
CONFIG_SUN50I_H6_R_CCU=y
CONFIG_SUN8I_A83T_CCU=y
CONFIG_SUN8I_H3_CCU=y
CONFIG_SUN8I_DE2_CCU=y
CONFIG_SUN8I_R_CCU=y
# CONFIG_COMMON_CLK_ZYNQMP is not set
CONFIG_HWSPINLOCK=y
CONFIG_HWSPINLOCK_QCOM=m
CONFIG_HWSPINLOCK_SPRD=m
#
# Clock Source drivers
#
CONFIG_TIMER_OF=y
CONFIG_TIMER_ACPI=y
CONFIG_TIMER_PROBE=y
CONFIG_CLKSRC_MMIO=y
CONFIG_DW_APB_TIMER=y
CONFIG_DW_APB_TIMER_OF=y
CONFIG_OWL_TIMER=y
CONFIG_ARM_ARCH_TIMER=y
CONFIG_ARM_ARCH_TIMER_EVTSTREAM=y
CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND=y
CONFIG_FSL_ERRATUM_A008585=y
CONFIG_HISILICON_ERRATUM_161010101=y
CONFIG_ARM64_ERRATUM_858921=y
CONFIG_SUN50I_ERRATUM_UNKNOWN1=y
CONFIG_ARM_TIMER_SP804=y
CONFIG_SYS_SUPPORTS_SH_CMT=y
CONFIG_MTK_TIMER=y
CONFIG_SPRD_TIMER=y
CONFIG_SYS_SUPPORTS_SH_TMU=y
CONFIG_SH_TIMER_CMT=y
CONFIG_SH_TIMER_TMU=y
# CONFIG_MICROCHIP_PIT64B is not set
# end of Clock Source drivers
CONFIG_MAILBOX=y
CONFIG_ARM_MHU=m
# CONFIG_ARM_MHU_V2 is not set
CONFIG_PLATFORM_MHU=m
CONFIG_PL320_MBOX=y
CONFIG_PCC=y
CONFIG_ALTERA_MBOX=m
CONFIG_BCM2835_MBOX=y
CONFIG_HI3660_MBOX=y
CONFIG_HI6220_MBOX=m
CONFIG_MAILBOX_TEST=m
CONFIG_QCOM_APCS_IPC=m
CONFIG_XGENE_SLIMPRO_MBOX=m
CONFIG_BCM_PDC_MBOX=m
CONFIG_BCM_FLEXRM_MBOX=m
# CONFIG_MTK_CMDQ_MBOX is not set
CONFIG_ZYNQMP_IPI_MBOX=y
CONFIG_SUN6I_MSGBOX=y
# CONFIG_SPRD_MBOX is not set
# CONFIG_QCOM_IPCC is not set
CONFIG_IOMMU_IOVA=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
#
# Generic IOMMU Pagetable Support
#
CONFIG_IOMMU_IO_PGTABLE=y
CONFIG_IOMMU_IO_PGTABLE_LPAE=y
# CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST is not set
# CONFIG_IOMMU_IO_PGTABLE_ARMV7S is not set
# end of Generic IOMMU Pagetable Support
# CONFIG_IOMMU_DEBUGFS is not set
# CONFIG_IOMMU_DEFAULT_PASSTHROUGH is not set
CONFIG_OF_IOMMU=y
CONFIG_IOMMU_DMA=y
# CONFIG_SUN50I_IOMMU is not set
# CONFIG_IPMMU_VMSA is not set
CONFIG_ARM_SMMU=y
# CONFIG_ARM_SMMU_LEGACY_DT_BINDINGS is not set
CONFIG_ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT=y
CONFIG_ARM_SMMU_V3=y
# CONFIG_ARM_SMMU_V3_SVA is not set
# CONFIG_MTK_IOMMU is not set
CONFIG_QCOM_IOMMU=y
# CONFIG_VIRTIO_IOMMU is not set
#
# Remoteproc drivers
#
# CONFIG_REMOTEPROC is not set
# end of Remoteproc drivers
#
# Rpmsg drivers
#
CONFIG_RPMSG=m
CONFIG_RPMSG_CHAR=m
CONFIG_RPMSG_NS=m
CONFIG_RPMSG_QCOM_GLINK=m
CONFIG_RPMSG_QCOM_GLINK_RPM=m
CONFIG_RPMSG_QCOM_GLINK_SMEM=m
CONFIG_RPMSG_QCOM_SMD=m
CONFIG_RPMSG_VIRTIO=m
# end of Rpmsg drivers
# CONFIG_SOUNDWIRE is not set
#
# SOC (System On Chip) specific Drivers
#
CONFIG_OWL_PM_DOMAINS_HELPER=y
CONFIG_OWL_PM_DOMAINS=y
#
# Amlogic SoC drivers
#
# end of Amlogic SoC drivers
#
# Broadcom SoC drivers
#
CONFIG_BCM2835_POWER=y
CONFIG_RASPBERRYPI_POWER=y
CONFIG_SOC_BRCMSTB=y
CONFIG_BRCMSTB_PM=y
# end of Broadcom SoC drivers
#
# NXP/Freescale QorIQ SoC drivers
#
CONFIG_FSL_DPAA=y
CONFIG_FSL_DPAA_CHECKING=y
CONFIG_FSL_BMAN_TEST=m
# CONFIG_FSL_BMAN_TEST_API is not set
# CONFIG_FSL_QMAN_TEST is not set
# CONFIG_QUICC_ENGINE is not set
CONFIG_FSL_GUTS=y
CONFIG_FSL_MC_DPIO=m
CONFIG_DPAA2_CONSOLE=y
# CONFIG_FSL_RCPM is not set
# end of NXP/Freescale QorIQ SoC drivers
#
# i.MX SoC drivers
#
# end of i.MX SoC drivers
#
# Enable LiteX SoC Builder specific drivers
#
# CONFIG_LITEX_SOC_CONTROLLER is not set
# end of Enable LiteX SoC Builder specific drivers
#
# MediaTek SoC drivers
#
# CONFIG_MTK_CMDQ is not set
# CONFIG_MTK_DEVAPC is not set
CONFIG_MTK_INFRACFG=y
CONFIG_MTK_PMIC_WRAP=m
CONFIG_MTK_SCPSYS=y
CONFIG_MTK_SCPSYS_PM_DOMAINS=y
CONFIG_MTK_MMSYS=y
# end of MediaTek SoC drivers
#
# Qualcomm SoC drivers
#
# CONFIG_QCOM_AOSS_QMP is not set
CONFIG_QCOM_COMMAND_DB=y
# CONFIG_QCOM_CPR is not set
# CONFIG_QCOM_GENI_SE is not set
CONFIG_QCOM_GSBI=m
# CONFIG_QCOM_LLCC is not set
CONFIG_QCOM_KRYO_L2_ACCESSORS=y
CONFIG_QCOM_MDT_LOADER=m
# CONFIG_QCOM_OCMEM is not set
CONFIG_QCOM_RMTFS_MEM=m
# CONFIG_QCOM_RPMH is not set
# CONFIG_QCOM_RPMPD is not set
CONFIG_QCOM_SMEM=m
CONFIG_QCOM_SMD_RPM=m
CONFIG_QCOM_SMEM_STATE=y
CONFIG_QCOM_SMP2P=m
CONFIG_QCOM_SMSM=m
# CONFIG_QCOM_SOCINFO is not set
CONFIG_QCOM_WCNSS_CTRL=m
# CONFIG_QCOM_APR is not set
# end of Qualcomm SoC drivers
CONFIG_SOC_RENESAS=y
CONFIG_ARCH_RCAR_GEN3=y
CONFIG_ARCH_R8A77995=y
# CONFIG_ARCH_R8A77990 is not set
CONFIG_ARCH_R8A77950=y
CONFIG_ARCH_R8A77951=y
# CONFIG_ARCH_R8A77965 is not set
# CONFIG_ARCH_R8A77960 is not set
# CONFIG_ARCH_R8A77961 is not set
# CONFIG_ARCH_R8A77980 is not set
CONFIG_ARCH_R8A77970=y
# CONFIG_ARCH_R8A779A0 is not set
# CONFIG_ARCH_R8A774C0 is not set
# CONFIG_ARCH_R8A774E1 is not set
# CONFIG_ARCH_R8A774A1 is not set
# CONFIG_ARCH_R8A774B1 is not set
CONFIG_RST_RCAR=y
CONFIG_SYSC_RCAR=y
CONFIG_SYSC_R8A77995=y
CONFIG_SYSC_R8A7795=y
CONFIG_SYSC_R8A77970=y
CONFIG_SUNXI_MBUS=y
CONFIG_SUNXI_SRAM=y
CONFIG_SOC_TI=y
#
# Xilinx SoC drivers
#
# CONFIG_XILINX_VCU is not set
CONFIG_ZYNQMP_POWER=y
CONFIG_ZYNQMP_PM_DOMAINS=y
# end of Xilinx SoC drivers
# end of SOC (System On Chip) specific Drivers
CONFIG_PM_DEVFREQ=y
#
# DEVFREQ Governors
#
CONFIG_DEVFREQ_GOV_SIMPLE_ONDEMAND=y
CONFIG_DEVFREQ_GOV_PERFORMANCE=y
CONFIG_DEVFREQ_GOV_POWERSAVE=y
CONFIG_DEVFREQ_GOV_USERSPACE=y
CONFIG_DEVFREQ_GOV_PASSIVE=y
#
# DEVFREQ Drivers
#
CONFIG_PM_DEVFREQ_EVENT=y
CONFIG_EXTCON=y
#
# Extcon Device Drivers
#
CONFIG_EXTCON_ADC_JACK=m
CONFIG_EXTCON_ARIZONA=m
# CONFIG_EXTCON_FSA9480 is not set
CONFIG_EXTCON_GPIO=m
CONFIG_EXTCON_MAX14577=m
CONFIG_EXTCON_MAX3355=m
CONFIG_EXTCON_MAX77693=m
CONFIG_EXTCON_MAX77843=m
CONFIG_EXTCON_MAX8997=m
CONFIG_EXTCON_PALMAS=m
# CONFIG_EXTCON_PTN5150 is not set
CONFIG_EXTCON_QCOM_SPMI_MISC=m
CONFIG_EXTCON_RT8973A=m
CONFIG_EXTCON_SM5502=m
CONFIG_EXTCON_USB_GPIO=m
CONFIG_EXTCON_USBC_CROS_EC=m
# CONFIG_EXTCON_USBC_TUSB320 is not set
CONFIG_MEMORY=y
CONFIG_ARM_PL172_MPMC=m
CONFIG_BRCMSTB_DPFE=y
CONFIG_MTK_SMI=y
# CONFIG_RENESAS_RPCIF is not set
CONFIG_IIO=m
CONFIG_IIO_BUFFER=y
CONFIG_IIO_BUFFER_CB=m
# CONFIG_IIO_BUFFER_DMA is not set
# CONFIG_IIO_BUFFER_DMAENGINE is not set
# CONFIG_IIO_BUFFER_HW_CONSUMER is not set
CONFIG_IIO_KFIFO_BUF=m
CONFIG_IIO_TRIGGERED_BUFFER=m
CONFIG_IIO_CONFIGFS=m
CONFIG_IIO_TRIGGER=y
CONFIG_IIO_CONSUMERS_PER_TRIGGER=2
CONFIG_IIO_SW_DEVICE=m
CONFIG_IIO_SW_TRIGGER=m
CONFIG_IIO_TRIGGERED_EVENT=m
#
# Accelerometers
#
CONFIG_ADIS16201=m
CONFIG_ADIS16209=m
# CONFIG_ADXL372_SPI is not set
# CONFIG_ADXL372_I2C is not set
CONFIG_BMA220=m
# CONFIG_BMA400 is not set
CONFIG_BMC150_ACCEL=m
CONFIG_BMC150_ACCEL_I2C=m
CONFIG_BMC150_ACCEL_SPI=m
CONFIG_DA280=m
CONFIG_DA311=m
CONFIG_DMARD06=m
CONFIG_DMARD09=m
CONFIG_DMARD10=m
CONFIG_HID_SENSOR_ACCEL_3D=m
CONFIG_IIO_CROS_EC_ACCEL_LEGACY=m
CONFIG_IIO_ST_ACCEL_3AXIS=m
CONFIG_IIO_ST_ACCEL_I2C_3AXIS=m
CONFIG_IIO_ST_ACCEL_SPI_3AXIS=m
CONFIG_KXSD9=m
CONFIG_KXSD9_SPI=m
CONFIG_KXSD9_I2C=m
CONFIG_KXCJK1013=m
CONFIG_MC3230=m
CONFIG_MMA7455=m
CONFIG_MMA7455_I2C=m
CONFIG_MMA7455_SPI=m
CONFIG_MMA7660=m
CONFIG_MMA8452=m
CONFIG_MMA9551_CORE=m
CONFIG_MMA9551=m
CONFIG_MMA9553=m
CONFIG_MXC4005=m
CONFIG_MXC6255=m
CONFIG_SCA3000=m
CONFIG_STK8312=m
CONFIG_STK8BA50=m
# end of Accelerometers
#
# Analog to digital converters
#
CONFIG_AD_SIGMA_DELTA=m
# CONFIG_AD7091R5 is not set
# CONFIG_AD7124 is not set
CONFIG_AD7192=m
CONFIG_AD7266=m
CONFIG_AD7291=m
# CONFIG_AD7292 is not set
CONFIG_AD7298=m
CONFIG_AD7476=m
CONFIG_AD7606=m
CONFIG_AD7606_IFACE_PARALLEL=m
CONFIG_AD7606_IFACE_SPI=m
CONFIG_AD7766=m
# CONFIG_AD7768_1 is not set
CONFIG_AD7780=m
CONFIG_AD7791=m
CONFIG_AD7793=m
CONFIG_AD7887=m
CONFIG_AD7923=m
# CONFIG_AD7949 is not set
CONFIG_AD799X=m
# CONFIG_AD9467 is not set
# CONFIG_ADI_AXI_ADC is not set
CONFIG_AXP20X_ADC=m
CONFIG_AXP288_ADC=m
CONFIG_BCM_IPROC_ADC=m
CONFIG_BERLIN2_ADC=m
CONFIG_CC10001_ADC=m
CONFIG_CPCAP_ADC=m
CONFIG_DA9150_GPADC=m
CONFIG_DLN2_ADC=m
CONFIG_ENVELOPE_DETECTOR=m
CONFIG_HI8435=m
CONFIG_HX711=m
CONFIG_INA2XX_ADC=m
CONFIG_LP8788_ADC=m
CONFIG_LTC2471=m
CONFIG_LTC2485=m
# CONFIG_LTC2496 is not set
CONFIG_LTC2497=m
CONFIG_MAX1027=m
CONFIG_MAX11100=m
CONFIG_MAX1118=m
# CONFIG_MAX1241 is not set
CONFIG_MAX1363=m
CONFIG_MAX9611=m
CONFIG_MCP320X=m
CONFIG_MCP3422=m
# CONFIG_MCP3911 is not set
CONFIG_MEDIATEK_MT6577_AUXADC=m
CONFIG_MEN_Z188_ADC=m
CONFIG_NAU7802=m
CONFIG_PALMAS_GPADC=m
CONFIG_QCOM_VADC_COMMON=m
CONFIG_QCOM_SPMI_IADC=m
CONFIG_QCOM_SPMI_VADC=m
# CONFIG_QCOM_SPMI_ADC5 is not set
# CONFIG_RN5T618_ADC is not set
# CONFIG_SC27XX_ADC is not set
# CONFIG_SD_ADC_MODULATOR is not set
# CONFIG_STMPE_ADC is not set
# CONFIG_SUN4I_GPADC is not set
CONFIG_TI_ADC081C=m
CONFIG_TI_ADC0832=m
CONFIG_TI_ADC084S021=m
CONFIG_TI_ADC12138=m
CONFIG_TI_ADC108S102=m
CONFIG_TI_ADC128S052=m
CONFIG_TI_ADC161S626=m
CONFIG_TI_ADS1015=m
CONFIG_TI_ADS7950=m
# CONFIG_TI_ADS8344 is not set
CONFIG_TI_ADS8688=m
# CONFIG_TI_ADS124S08 is not set
CONFIG_TI_AM335X_ADC=m
CONFIG_TI_TLC4541=m
CONFIG_TWL4030_MADC=m
CONFIG_TWL6030_GPADC=m
CONFIG_VF610_ADC=m
CONFIG_VIPERBOARD_ADC=m
# CONFIG_XILINX_XADC is not set
# end of Analog to digital converters
#
# Analog Front Ends
#
# CONFIG_IIO_RESCALE is not set
# end of Analog Front Ends
#
# Amplifiers
#
CONFIG_AD8366=m
# CONFIG_HMC425 is not set
# end of Amplifiers
#
# Chemical Sensors
#
CONFIG_ATLAS_PH_SENSOR=m
# CONFIG_ATLAS_EZO_SENSOR is not set
# CONFIG_BME680 is not set
CONFIG_CCS811=m
CONFIG_IAQCORE=m
# CONFIG_PMS7003 is not set
# CONFIG_SCD30_CORE is not set
# CONFIG_SENSIRION_SGP30 is not set
# CONFIG_SPS30 is not set
CONFIG_VZ89X=m
# end of Chemical Sensors
CONFIG_IIO_CROS_EC_SENSORS_CORE=m
CONFIG_IIO_CROS_EC_SENSORS=m
# CONFIG_IIO_CROS_EC_SENSORS_LID_ANGLE is not set
#
# Hid Sensor IIO Common
#
CONFIG_HID_SENSOR_IIO_COMMON=m
CONFIG_HID_SENSOR_IIO_TRIGGER=m
# end of Hid Sensor IIO Common
CONFIG_IIO_MS_SENSORS_I2C=m
#
# SSP Sensor Common
#
CONFIG_IIO_SSP_SENSORS_COMMONS=m
CONFIG_IIO_SSP_SENSORHUB=m
# end of SSP Sensor Common
CONFIG_IIO_ST_SENSORS_I2C=m
CONFIG_IIO_ST_SENSORS_SPI=m
CONFIG_IIO_ST_SENSORS_CORE=m
#
# Digital to analog converters
#
CONFIG_AD5064=m
CONFIG_AD5360=m
CONFIG_AD5380=m
CONFIG_AD5421=m
CONFIG_AD5446=m
CONFIG_AD5449=m
CONFIG_AD5592R_BASE=m
CONFIG_AD5592R=m
CONFIG_AD5593R=m
CONFIG_AD5504=m
CONFIG_AD5624R_SPI=m
# CONFIG_AD5686_SPI is not set
# CONFIG_AD5696_I2C is not set
CONFIG_AD5755=m
# CONFIG_AD5758 is not set
CONFIG_AD5761=m
CONFIG_AD5764=m
# CONFIG_AD5770R is not set
CONFIG_AD5791=m
CONFIG_AD7303=m
CONFIG_AD8801=m
CONFIG_DPOT_DAC=m
CONFIG_DS4424=m
# CONFIG_LTC1660 is not set
CONFIG_LTC2632=m
CONFIG_M62332=m
CONFIG_MAX517=m
CONFIG_MAX5821=m
CONFIG_MCP4725=m
CONFIG_MCP4922=m
CONFIG_TI_DAC082S085=m
# CONFIG_TI_DAC5571 is not set
# CONFIG_TI_DAC7311 is not set
# CONFIG_TI_DAC7612 is not set
CONFIG_VF610_DAC=m
# end of Digital to analog converters
#
# IIO dummy driver
#
CONFIG_IIO_SIMPLE_DUMMY=m
# CONFIG_IIO_SIMPLE_DUMMY_EVENTS is not set
# CONFIG_IIO_SIMPLE_DUMMY_BUFFER is not set
# end of IIO dummy driver
#
# Frequency Synthesizers DDS/PLL
#
#
# Clock Generator/Distribution
#
CONFIG_AD9523=m
# end of Clock Generator/Distribution
#
# Phase-Locked Loop (PLL) frequency synthesizers
#
CONFIG_ADF4350=m
# CONFIG_ADF4371 is not set
# end of Phase-Locked Loop (PLL) frequency synthesizers
# end of Frequency Synthesizers DDS/PLL
#
# Digital gyroscope sensors
#
CONFIG_ADIS16080=m
CONFIG_ADIS16130=m
CONFIG_ADIS16136=m
CONFIG_ADIS16260=m
# CONFIG_ADXRS290 is not set
CONFIG_ADXRS450=m
CONFIG_BMG160=m
CONFIG_BMG160_I2C=m
CONFIG_BMG160_SPI=m
# CONFIG_FXAS21002C is not set
CONFIG_HID_SENSOR_GYRO_3D=m
CONFIG_MPU3050=m
CONFIG_MPU3050_I2C=m
CONFIG_IIO_ST_GYRO_3AXIS=m
CONFIG_IIO_ST_GYRO_I2C_3AXIS=m
CONFIG_IIO_ST_GYRO_SPI_3AXIS=m
CONFIG_ITG3200=m
# end of Digital gyroscope sensors
#
# Health Sensors
#
#
# Heart Rate Monitors
#
CONFIG_AFE4403=m
CONFIG_AFE4404=m
CONFIG_MAX30100=m
CONFIG_MAX30102=m
# end of Heart Rate Monitors
# end of Health Sensors
#
# Humidity sensors
#
CONFIG_AM2315=m
CONFIG_DHT11=m
CONFIG_HDC100X=m
# CONFIG_HDC2010 is not set
CONFIG_HID_SENSOR_HUMIDITY=m
CONFIG_HTS221=m
CONFIG_HTS221_I2C=m
CONFIG_HTS221_SPI=m
CONFIG_HTU21=m
CONFIG_SI7005=m
CONFIG_SI7020=m
# end of Humidity sensors
#
# Inertial measurement units
#
CONFIG_ADIS16400=m
# CONFIG_ADIS16460 is not set
# CONFIG_ADIS16475 is not set
CONFIG_ADIS16480=m
CONFIG_BMI160=m
CONFIG_BMI160_I2C=m
CONFIG_BMI160_SPI=m
# CONFIG_FXOS8700_I2C is not set
# CONFIG_FXOS8700_SPI is not set
CONFIG_KMX61=m
# CONFIG_INV_ICM42600_I2C is not set
# CONFIG_INV_ICM42600_SPI is not set
CONFIG_INV_MPU6050_IIO=m
CONFIG_INV_MPU6050_I2C=m
CONFIG_INV_MPU6050_SPI=m
CONFIG_IIO_ST_LSM6DSX=m
CONFIG_IIO_ST_LSM6DSX_I2C=m
CONFIG_IIO_ST_LSM6DSX_SPI=m
# end of Inertial measurement units
CONFIG_IIO_ADIS_LIB=m
CONFIG_IIO_ADIS_LIB_BUFFER=y
#
# Light sensors
#
CONFIG_ACPI_ALS=m
CONFIG_ADJD_S311=m
# CONFIG_ADUX1020 is not set
# CONFIG_AL3010 is not set
CONFIG_AL3320A=m
CONFIG_APDS9300=m
CONFIG_APDS9960=m
# CONFIG_AS73211 is not set
CONFIG_BH1750=m
CONFIG_BH1780=m
CONFIG_CM32181=m
CONFIG_CM3232=m
CONFIG_CM3323=m
CONFIG_CM3605=m
CONFIG_CM36651=m
CONFIG_IIO_CROS_EC_LIGHT_PROX=m
# CONFIG_GP2AP002 is not set
CONFIG_GP2AP020A00F=m
CONFIG_SENSORS_ISL29018=m
CONFIG_SENSORS_ISL29028=m
CONFIG_ISL29125=m
CONFIG_HID_SENSOR_ALS=m
CONFIG_HID_SENSOR_PROX=m
CONFIG_JSA1212=m
CONFIG_RPR0521=m
CONFIG_SENSORS_LM3533=m
CONFIG_LTR501=m
# CONFIG_LV0104CS is not set
CONFIG_MAX44000=m
# CONFIG_MAX44009 is not set
# CONFIG_NOA1305 is not set
CONFIG_OPT3001=m
CONFIG_PA12203001=m
# CONFIG_SI1133 is not set
CONFIG_SI1145=m
CONFIG_STK3310=m
# CONFIG_ST_UVIS25 is not set
CONFIG_TCS3414=m
CONFIG_TCS3472=m
CONFIG_SENSORS_TSL2563=m
CONFIG_TSL2583=m
# CONFIG_TSL2772 is not set
CONFIG_TSL4531=m
CONFIG_US5182D=m
CONFIG_VCNL4000=m
# CONFIG_VCNL4035 is not set
# CONFIG_VEML6030 is not set
CONFIG_VEML6070=m
CONFIG_VL6180=m
# CONFIG_ZOPT2201 is not set
# end of Light sensors
#
# Magnetometer sensors
#
CONFIG_AK8974=m
CONFIG_AK8975=m
CONFIG_AK09911=m
CONFIG_BMC150_MAGN=m
CONFIG_BMC150_MAGN_I2C=m
CONFIG_BMC150_MAGN_SPI=m
CONFIG_MAG3110=m
CONFIG_HID_SENSOR_MAGNETOMETER_3D=m
CONFIG_MMC35240=m
CONFIG_IIO_ST_MAGN_3AXIS=m
CONFIG_IIO_ST_MAGN_I2C_3AXIS=m
CONFIG_IIO_ST_MAGN_SPI_3AXIS=m
CONFIG_SENSORS_HMC5843=m
CONFIG_SENSORS_HMC5843_I2C=m
CONFIG_SENSORS_HMC5843_SPI=m
# CONFIG_SENSORS_RM3100_I2C is not set
# CONFIG_SENSORS_RM3100_SPI is not set
# end of Magnetometer sensors
#
# Multiplexers
#
CONFIG_IIO_MUX=m
# end of Multiplexers
#
# Inclinometer sensors
#
CONFIG_HID_SENSOR_INCLINOMETER_3D=m
CONFIG_HID_SENSOR_DEVICE_ROTATION=m
# end of Inclinometer sensors
#
# Triggers - standalone
#
CONFIG_IIO_HRTIMER_TRIGGER=m
CONFIG_IIO_INTERRUPT_TRIGGER=m
CONFIG_IIO_TIGHTLOOP_TRIGGER=m
CONFIG_IIO_SYSFS_TRIGGER=m
# end of Triggers - standalone
#
# Linear and angular position sensors
#
# end of Linear and angular position sensors
#
# Digital potentiometers
#
# CONFIG_AD5272 is not set
CONFIG_DS1803=m
# CONFIG_MAX5432 is not set
CONFIG_MAX5481=m
CONFIG_MAX5487=m
# CONFIG_MCP4018 is not set
CONFIG_MCP4131=m
CONFIG_MCP4531=m
# CONFIG_MCP41010 is not set
CONFIG_TPL0102=m
# end of Digital potentiometers
#
# Digital potentiostats
#
CONFIG_LMP91000=m
# end of Digital potentiostats
#
# Pressure sensors
#
CONFIG_ABP060MG=m
CONFIG_BMP280=m
CONFIG_BMP280_I2C=m
CONFIG_BMP280_SPI=m
CONFIG_IIO_CROS_EC_BARO=m
# CONFIG_DLHL60D is not set
# CONFIG_DPS310 is not set
CONFIG_HID_SENSOR_PRESS=m
CONFIG_HP03=m
# CONFIG_ICP10100 is not set
CONFIG_MPL115=m
CONFIG_MPL115_I2C=m
CONFIG_MPL115_SPI=m
CONFIG_MPL3115=m
CONFIG_MS5611=m
CONFIG_MS5611_I2C=m
CONFIG_MS5611_SPI=m
CONFIG_MS5637=m
CONFIG_IIO_ST_PRESS=m
CONFIG_IIO_ST_PRESS_I2C=m
CONFIG_IIO_ST_PRESS_SPI=m
CONFIG_T5403=m
CONFIG_HP206C=m
CONFIG_ZPA2326=m
CONFIG_ZPA2326_I2C=m
CONFIG_ZPA2326_SPI=m
# end of Pressure sensors
#
# Lightning sensors
#
CONFIG_AS3935=m
# end of Lightning sensors
#
# Proximity and distance sensors
#
# CONFIG_ISL29501 is not set
CONFIG_LIDAR_LITE_V2=m
# CONFIG_MB1232 is not set
# CONFIG_PING is not set
CONFIG_RFD77402=m
CONFIG_SRF04=m
# CONFIG_SX9310 is not set
CONFIG_SX9500=m
CONFIG_SRF08=m
# CONFIG_VCNL3020 is not set
# CONFIG_VL53L0X_I2C is not set
# end of Proximity and distance sensors
#
# Resolver to digital converters
#
CONFIG_AD2S90=m
CONFIG_AD2S1200=m
# end of Resolver to digital converters
#
# Temperature sensors
#
# CONFIG_LTC2983 is not set
CONFIG_MAXIM_THERMOCOUPLE=m
CONFIG_HID_SENSOR_TEMP=m
CONFIG_MLX90614=m
# CONFIG_MLX90632 is not set
CONFIG_TMP006=m
CONFIG_TMP007=m
CONFIG_TSYS01=m
CONFIG_TSYS02D=m
# CONFIG_MAX31856 is not set
# end of Temperature sensors
CONFIG_NTB=m
# CONFIG_NTB_MSI is not set
CONFIG_NTB_IDT=m
CONFIG_NTB_SWITCHTEC=m
CONFIG_NTB_PINGPONG=m
CONFIG_NTB_TOOL=m
CONFIG_NTB_PERF=m
CONFIG_NTB_TRANSPORT=m
CONFIG_VME_BUS=y
#
# VME Bridge Drivers
#
CONFIG_VME_TSI148=m
CONFIG_VME_FAKE=m
#
# VME Board Drivers
#
CONFIG_VMIVME_7805=m
#
# VME Device Drivers
#
CONFIG_VME_USER=m
CONFIG_PWM=y
CONFIG_PWM_SYSFS=y
# CONFIG_PWM_DEBUG is not set
CONFIG_PWM_ATMEL_HLCDC_PWM=m
# CONFIG_PWM_ATMEL_TCB is not set
CONFIG_PWM_BCM_IPROC=m
CONFIG_PWM_BCM2835=m
CONFIG_PWM_BERLIN=m
CONFIG_PWM_BRCMSTB=m
CONFIG_PWM_CROS_EC=m
# CONFIG_PWM_DWC is not set
CONFIG_PWM_FSL_FTM=m
CONFIG_PWM_HIBVT=m
CONFIG_PWM_LP3943=m
CONFIG_PWM_MTK_DISP=m
CONFIG_PWM_MEDIATEK=m
CONFIG_PWM_PCA9685=m
CONFIG_PWM_RCAR=m
CONFIG_PWM_RENESAS_TPU=m
# CONFIG_PWM_SPRD is not set
CONFIG_PWM_STMPE=y
CONFIG_PWM_SUN4I=m
CONFIG_PWM_TWL=m
CONFIG_PWM_TWL_LED=m
#
# IRQ chip support
#
CONFIG_IRQCHIP=y
CONFIG_ARM_GIC=y
CONFIG_ARM_GIC_MAX_NR=1
CONFIG_ARM_GIC_V2M=y
CONFIG_ARM_GIC_V3=y
CONFIG_ARM_GIC_V3_ITS=y
CONFIG_ARM_GIC_V3_ITS_PCI=y
CONFIG_ARM_GIC_V3_ITS_FSL_MC=y
# CONFIG_AL_FIC is not set
CONFIG_BCM7038_L1_IRQ=y
CONFIG_BRCMSTB_L2_IRQ=y
CONFIG_DW_APB_ICTL=y
CONFIG_HISILICON_IRQ_MBIGEN=y
CONFIG_RENESAS_IRQC=y
CONFIG_LS_EXTIRQ=y
CONFIG_LS_SCFG_MSI=y
CONFIG_PARTITION_PERCPU=y
CONFIG_QCOM_IRQ_COMBINER=y
# CONFIG_QCOM_PDC is not set
CONFIG_MST_IRQ=y
# end of IRQ chip support
CONFIG_IPACK_BUS=m
CONFIG_BOARD_TPCI200=m
CONFIG_SERIAL_IPOCTAL=m
CONFIG_ARCH_HAS_RESET_CONTROLLER=y
CONFIG_RESET_CONTROLLER=y
CONFIG_RESET_BERLIN=y
CONFIG_RESET_BRCMSTB=y
CONFIG_RESET_BRCMSTB_RESCAL=y
# CONFIG_RESET_INTEL_GW is not set
# CONFIG_RESET_QCOM_AOSS is not set
# CONFIG_RESET_QCOM_PDC is not set
CONFIG_RESET_RASPBERRYPI=y
CONFIG_RESET_SIMPLE=y
CONFIG_RESET_SUNXI=y
CONFIG_RESET_TI_SYSCON=m
CONFIG_COMMON_RESET_HI3660=m
CONFIG_COMMON_RESET_HI6220=m
#
# PHY Subsystem
#
CONFIG_GENERIC_PHY=y
CONFIG_PHY_XGENE=y
# CONFIG_USB_LGM_PHY is not set
CONFIG_PHY_SUN4I_USB=m
# CONFIG_PHY_SUN6I_MIPI_DPHY is not set
# CONFIG_PHY_SUN9I_USB is not set
# CONFIG_PHY_SUN50I_USB3 is not set
CONFIG_PHY_BCM_SR_USB=y
CONFIG_BCM_KONA_USB2_PHY=m
CONFIG_PHY_BCM_NS_USB2=m
CONFIG_PHY_BCM_NS_USB3=m
CONFIG_PHY_NS2_PCIE=y
CONFIG_PHY_NS2_USB_DRD=m
CONFIG_PHY_BRCM_SATA=y
CONFIG_PHY_BRCM_USB=m
CONFIG_PHY_BCM_SR_PCIE=y
# CONFIG_PHY_CADENCE_TORRENT is not set
# CONFIG_PHY_CADENCE_DPHY is not set
# CONFIG_PHY_CADENCE_SIERRA is not set
# CONFIG_PHY_CADENCE_SALVO is not set
# CONFIG_PHY_FSL_IMX8MQ_USB is not set
# CONFIG_PHY_MIXEL_MIPI_DPHY is not set
CONFIG_PHY_HI6220_USB=m
# CONFIG_PHY_HI3660_USB is not set
# CONFIG_PHY_HISTB_COMBPHY is not set
# CONFIG_PHY_HISI_INNO_USB2 is not set
CONFIG_PHY_BERLIN_SATA=m
CONFIG_PHY_BERLIN_USB=m
CONFIG_PHY_PXA_28NM_HSIC=m
CONFIG_PHY_PXA_28NM_USB2=m
CONFIG_PHY_MTK_TPHY=m
# CONFIG_PHY_MTK_UFS is not set
# CONFIG_PHY_MTK_XSPHY is not set
CONFIG_PHY_MTK_HDMI=m
CONFIG_PHY_MTK_MIPI_DSI=m
CONFIG_PHY_CPCAP_USB=m
# CONFIG_PHY_MAPPHONE_MDM6600 is not set
# CONFIG_PHY_OCELOT_SERDES is not set
CONFIG_PHY_QCOM_APQ8064_SATA=m
# CONFIG_PHY_QCOM_IPQ4019_USB is not set
CONFIG_PHY_QCOM_IPQ806X_SATA=m
# CONFIG_PHY_QCOM_PCIE2 is not set
CONFIG_PHY_QCOM_QMP=m
CONFIG_PHY_QCOM_QUSB2=m
CONFIG_PHY_QCOM_USB_HS=m
# CONFIG_PHY_QCOM_USB_SNPS_FEMTO_V2 is not set
CONFIG_PHY_QCOM_USB_HSIC=m
# CONFIG_PHY_QCOM_USB_HS_28NM is not set
# CONFIG_PHY_QCOM_USB_SS is not set
# CONFIG_PHY_QCOM_IPQ806X_USB is not set
CONFIG_PHY_RCAR_GEN2=m
# CONFIG_PHY_RCAR_GEN3_PCIE is not set
CONFIG_PHY_RCAR_GEN3_USB2=m
CONFIG_PHY_RCAR_GEN3_USB3=m
CONFIG_PHY_SAMSUNG_USB2=m
CONFIG_PHY_TUSB1210=m
# CONFIG_PHY_XILINX_ZYNQMP is not set
# end of PHY Subsystem
CONFIG_POWERCAP=y
# CONFIG_IDLE_INJECT is not set
CONFIG_MCB=m
CONFIG_MCB_PCI=m
CONFIG_MCB_LPC=m
#
# Performance monitor support
#
CONFIG_ARM_CCI_PMU=y
CONFIG_ARM_CCI400_PMU=y
CONFIG_ARM_CCI5xx_PMU=y
CONFIG_ARM_CCN=y
# CONFIG_ARM_CMN is not set
CONFIG_ARM_PMU=y
CONFIG_ARM_PMU_ACPI=y
# CONFIG_ARM_SMMU_V3_PMU is not set
# CONFIG_ARM_DSU_PMU is not set
CONFIG_QCOM_L2_PMU=y
CONFIG_QCOM_L3_PMU=y
CONFIG_THUNDERX2_PMU=m
CONFIG_XGENE_PMU=y
CONFIG_ARM_SPE_PMU=m
# CONFIG_ARM_DMC620_PMU is not set
CONFIG_HISI_PMU=y
# end of Performance monitor support
CONFIG_RAS=y
# CONFIG_USB4 is not set
#
# Android
#
# CONFIG_ANDROID is not set
# end of Android
CONFIG_LIBNVDIMM=y
CONFIG_BLK_DEV_PMEM=m
CONFIG_ND_BLK=m
CONFIG_ND_CLAIM=y
CONFIG_ND_BTT=m
CONFIG_BTT=y
CONFIG_OF_PMEM=y
CONFIG_NVDIMM_KEYS=y
CONFIG_DAX_DRIVER=y
CONFIG_DAX=y
CONFIG_DEV_DAX=m
CONFIG_NVMEM=y
CONFIG_NVMEM_SYSFS=y
# CONFIG_MTK_EFUSE is not set
CONFIG_QCOM_QFPROM=m
# CONFIG_NVMEM_SPMI_SDAM is not set
CONFIG_NVMEM_BCM_OCOTP=m
CONFIG_NVMEM_SUNXI_SID=m
# CONFIG_SC27XX_EFUSE is not set
# CONFIG_NVMEM_ZYNQMP is not set
# CONFIG_SPRD_EFUSE is not set
#
# HW tracing support
#
CONFIG_STM=m
# CONFIG_STM_PROTO_BASIC is not set
# CONFIG_STM_PROTO_SYS_T is not set
CONFIG_STM_DUMMY=m
CONFIG_STM_SOURCE_CONSOLE=m
CONFIG_STM_SOURCE_HEARTBEAT=m
CONFIG_STM_SOURCE_FTRACE=m
CONFIG_INTEL_TH=m
CONFIG_INTEL_TH_PCI=m
# CONFIG_INTEL_TH_ACPI is not set
CONFIG_INTEL_TH_GTH=m
CONFIG_INTEL_TH_STH=m
CONFIG_INTEL_TH_MSU=m
CONFIG_INTEL_TH_PTI=m
# CONFIG_INTEL_TH_DEBUG is not set
# end of HW tracing support
CONFIG_FPGA=m
CONFIG_ALTERA_PR_IP_CORE=m
CONFIG_ALTERA_PR_IP_CORE_PLAT=m
CONFIG_FPGA_MGR_ALTERA_PS_SPI=m
CONFIG_FPGA_MGR_ALTERA_CVP=m
CONFIG_FPGA_MGR_XILINX_SPI=m
CONFIG_FPGA_MGR_ICE40_SPI=m
# CONFIG_FPGA_MGR_MACHXO2_SPI is not set
CONFIG_FPGA_BRIDGE=m
# CONFIG_ALTERA_FREEZE_BRIDGE is not set
CONFIG_XILINX_PR_DECOUPLER=m
CONFIG_FPGA_REGION=m
# CONFIG_OF_FPGA_REGION is not set
# CONFIG_FPGA_DFL is not set
# CONFIG_FPGA_MGR_ZYNQMP_FPGA is not set
CONFIG_FSI=m
# CONFIG_FSI_NEW_DEV_NODE is not set
CONFIG_FSI_MASTER_GPIO=m
CONFIG_FSI_MASTER_HUB=m
# CONFIG_FSI_MASTER_ASPEED is not set
CONFIG_FSI_SCOM=m
# CONFIG_FSI_SBEFIFO is not set
CONFIG_TEE=m
#
# TEE drivers
#
CONFIG_OPTEE=m
CONFIG_OPTEE_SHM_NUM_PRIV_PAGES=1
# end of TEE drivers
CONFIG_MULTIPLEXER=m
#
# Multiplexer drivers
#
CONFIG_MUX_ADG792A=m
# CONFIG_MUX_ADGS1408 is not set
CONFIG_MUX_GPIO=m
CONFIG_MUX_MMIO=m
# end of Multiplexer drivers
CONFIG_PM_OPP=y
# CONFIG_SIOX is not set
# CONFIG_SLIMBUS is not set
# CONFIG_INTERCONNECT is not set
# CONFIG_COUNTER is not set
CONFIG_MOST=m
# CONFIG_MOST_USB_HDM is not set
# CONFIG_MOST_CDEV is not set
# end of Device Drivers
#
# File systems
#
CONFIG_DCACHE_WORD_ACCESS=y
# CONFIG_VALIDATE_FS_PARSER is not set
CONFIG_FS_IOMAP=y
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT2=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_JBD2=y
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
CONFIG_REISERFS_FS_SECURITY=y
CONFIG_JFS_FS=m
CONFIG_JFS_POSIX_ACL=y
CONFIG_JFS_SECURITY=y
# CONFIG_JFS_DEBUG is not set
CONFIG_JFS_STATISTICS=y
CONFIG_XFS_FS=m
CONFIG_XFS_SUPPORT_V4=y
CONFIG_XFS_QUOTA=y
CONFIG_XFS_POSIX_ACL=y
CONFIG_XFS_RT=y
# CONFIG_XFS_ONLINE_SCRUB is not set
# CONFIG_XFS_WARN is not set
# CONFIG_XFS_DEBUG is not set
CONFIG_GFS2_FS=m
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_OCFS2_FS=m
CONFIG_OCFS2_FS_O2CB=m
CONFIG_OCFS2_FS_USERSPACE_CLUSTER=m
CONFIG_OCFS2_FS_STATS=y
CONFIG_OCFS2_DEBUG_MASKLOG=y
# CONFIG_OCFS2_DEBUG_FS is not set
CONFIG_BTRFS_FS=m
CONFIG_BTRFS_FS_POSIX_ACL=y
# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
# CONFIG_BTRFS_DEBUG is not set
# CONFIG_BTRFS_ASSERT is not set
# CONFIG_BTRFS_FS_REF_VERIFY is not set
CONFIG_NILFS2_FS=m
CONFIG_F2FS_FS=m
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
CONFIG_F2FS_FS_SECURITY=y
# CONFIG_F2FS_CHECK_FS is not set
# CONFIG_F2FS_IO_TRACE is not set
# CONFIG_F2FS_FAULT_INJECTION is not set
# CONFIG_F2FS_FS_COMPRESSION is not set
# CONFIG_ZONEFS_FS is not set
CONFIG_FS_DAX=y
CONFIG_FS_POSIX_ACL=y
CONFIG_EXPORTFS=y
CONFIG_EXPORTFS_BLOCK_OPS=y
CONFIG_FILE_LOCKING=y
CONFIG_MANDATORY_FILE_LOCKING=y
CONFIG_FS_ENCRYPTION=y
CONFIG_FS_ENCRYPTION_ALGS=y
# CONFIG_FS_VERITY is not set
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_FANOTIFY=y
CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not set
# CONFIG_QUOTA_DEBUG is not set
CONFIG_QUOTA_TREE=m
CONFIG_QFMT_V1=m
CONFIG_QFMT_V2=m
CONFIG_QUOTACTL=y
CONFIG_AUTOFS4_FS=m
CONFIG_AUTOFS_FS=m
CONFIG_FUSE_FS=y
CONFIG_CUSE=m
# CONFIG_VIRTIO_FS is not set
CONFIG_OVERLAY_FS=m
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
# CONFIG_OVERLAY_FS_INDEX is not set
# CONFIG_OVERLAY_FS_XINO_AUTO is not set
# CONFIG_OVERLAY_FS_METACOPY is not set
#
# Caches
#
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
# CONFIG_CACHEFILES_DEBUG is not set
# CONFIG_CACHEFILES_HISTOGRAM is not set
# end of Caches
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=m
# end of CD-ROM/DVD Filesystems
#
# DOS/FAT/EXFAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_FAT_DEFAULT_UTF8 is not set
# CONFIG_EXFAT_FS is not set
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set
# end of DOS/FAT/EXFAT/NT Filesystems
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_VMCORE=y
# CONFIG_PROC_VMCORE_DEVICE_DUMP is not set
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_PROC_CHILDREN=y
CONFIG_KERNFS=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y
# CONFIG_TMPFS_INODE64 is not set
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_MEMFD_CREATE=y
CONFIG_ARCH_HAS_GIGANTIC_PAGE=y
CONFIG_CONFIGFS_FS=y
CONFIG_EFIVAR_FS=y
# end of Pseudo filesystems
CONFIG_MISC_FILESYSTEMS=y
CONFIG_ORANGEFS_FS=m
CONFIG_ADFS_FS=m
# CONFIG_ADFS_FS_RW is not set
CONFIG_AFFS_FS=m
CONFIG_ECRYPT_FS=y
CONFIG_ECRYPT_FS_MESSAGING=y
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
CONFIG_BEFS_FS=m
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=m
CONFIG_EFS_FS=m
CONFIG_JFFS2_FS=m
CONFIG_JFFS2_FS_DEBUG=0
CONFIG_JFFS2_FS_WRITEBUFFER=y
# CONFIG_JFFS2_FS_WBUF_VERIFY is not set
# CONFIG_JFFS2_SUMMARY is not set
CONFIG_JFFS2_FS_XATTR=y
CONFIG_JFFS2_FS_POSIX_ACL=y
CONFIG_JFFS2_FS_SECURITY=y
CONFIG_JFFS2_COMPRESSION_OPTIONS=y
CONFIG_JFFS2_ZLIB=y
CONFIG_JFFS2_LZO=y
CONFIG_JFFS2_RTIME=y
# CONFIG_JFFS2_RUBIN is not set
# CONFIG_JFFS2_CMODE_NONE is not set
# CONFIG_JFFS2_CMODE_PRIORITY is not set
# CONFIG_JFFS2_CMODE_SIZE is not set
CONFIG_JFFS2_CMODE_FAVOURLZO=y
CONFIG_UBIFS_FS=m
# CONFIG_UBIFS_FS_ADVANCED_COMPR is not set
CONFIG_UBIFS_FS_LZO=y
CONFIG_UBIFS_FS_ZLIB=y
CONFIG_UBIFS_FS_ZSTD=y
# CONFIG_UBIFS_ATIME_SUPPORT is not set
CONFIG_UBIFS_FS_XATTR=y
CONFIG_UBIFS_FS_SECURITY=y
# CONFIG_UBIFS_FS_AUTHENTICATION is not set
CONFIG_CRAMFS=m
CONFIG_CRAMFS_BLOCKDEV=y
CONFIG_CRAMFS_MTD=y
CONFIG_SQUASHFS=y
# CONFIG_SQUASHFS_FILE_CACHE is not set
CONFIG_SQUASHFS_FILE_DIRECT=y
CONFIG_SQUASHFS_DECOMP_SINGLE=y
# CONFIG_SQUASHFS_DECOMP_MULTI is not set
# CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU is not set
CONFIG_SQUASHFS_XATTR=y
CONFIG_SQUASHFS_ZLIB=y
CONFIG_SQUASHFS_LZ4=y
CONFIG_SQUASHFS_LZO=y
CONFIG_SQUASHFS_XZ=y
CONFIG_SQUASHFS_ZSTD=y
# CONFIG_SQUASHFS_4K_DEVBLK_SIZE is not set
# CONFIG_SQUASHFS_EMBEDDED is not set
CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3
CONFIG_VXFS_FS=m
CONFIG_MINIX_FS=m
CONFIG_OMFS_FS=m
CONFIG_HPFS_FS=m
CONFIG_QNX4FS_FS=m
CONFIG_QNX6FS_FS=m
# CONFIG_QNX6FS_DEBUG is not set
CONFIG_ROMFS_FS=m
CONFIG_ROMFS_BACKED_BY_BLOCK=y
# CONFIG_ROMFS_BACKED_BY_MTD is not set
# CONFIG_ROMFS_BACKED_BY_BOTH is not set
CONFIG_ROMFS_ON_BLOCK=y
CONFIG_PSTORE=y
CONFIG_PSTORE_DEFAULT_KMSG_BYTES=10240
CONFIG_PSTORE_DEFLATE_COMPRESS=y
# CONFIG_PSTORE_LZO_COMPRESS is not set
# CONFIG_PSTORE_LZ4_COMPRESS is not set
# CONFIG_PSTORE_LZ4HC_COMPRESS is not set
# CONFIG_PSTORE_842_COMPRESS is not set
# CONFIG_PSTORE_ZSTD_COMPRESS is not set
CONFIG_PSTORE_COMPRESS=y
CONFIG_PSTORE_DEFLATE_COMPRESS_DEFAULT=y
CONFIG_PSTORE_COMPRESS_DEFAULT="deflate"
# CONFIG_PSTORE_CONSOLE is not set
# CONFIG_PSTORE_PMSG is not set
# CONFIG_PSTORE_FTRACE is not set
CONFIG_PSTORE_RAM=m
# CONFIG_PSTORE_BLK is not set
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_UFS_DEBUG is not set
# CONFIG_EROFS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=m
CONFIG_NFS_V2=m
CONFIG_NFS_V3=m
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_NFS_V4_1=y
CONFIG_NFS_V4_2=y
CONFIG_PNFS_FILE_LAYOUT=m
CONFIG_PNFS_BLOCK=m
CONFIG_PNFS_FLEXFILE_LAYOUT=m
CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org"
CONFIG_NFS_V4_1_MIGRATION=y
CONFIG_NFS_V4_SECURITY_LABEL=y
CONFIG_NFS_FSCACHE=y
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
CONFIG_NFS_DEBUG=y
CONFIG_NFS_DISABLE_UDP_SUPPORT=y
# CONFIG_NFS_V4_2_READ_PLUS is not set
CONFIG_NFSD=m
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_PNFS=y
CONFIG_NFSD_BLOCKLAYOUT=y
CONFIG_NFSD_SCSILAYOUT=y
CONFIG_NFSD_FLEXFILELAYOUT=y
# CONFIG_NFSD_V4_2_INTER_SSC is not set
CONFIG_NFSD_V4_SECURITY_LABEL=y
CONFIG_GRACE_PERIOD=m
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_NFS_ACL_SUPPORT=m
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_SUNRPC_BACKCHANNEL=y
CONFIG_SUNRPC_SWAP=y
CONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_SUNRPC_DISABLE_INSECURE_ENCTYPES is not set
CONFIG_SUNRPC_DEBUG=y
CONFIG_SUNRPC_XPRT_RDMA=m
CONFIG_CEPH_FS=m
CONFIG_CEPH_FSCACHE=y
CONFIG_CEPH_FS_POSIX_ACL=y
# CONFIG_CEPH_FS_SECURITY_LABEL is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS2 is not set
CONFIG_CIFS_ALLOW_INSECURE_LEGACY=y
CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_UPCALL=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_CIFS_DEBUG=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DEBUG_DUMP_KEYS is not set
CONFIG_CIFS_DFS_UPCALL=y
# CONFIG_CIFS_SWN_UPCALL is not set
# CONFIG_CIFS_SMB_DIRECT is not set
CONFIG_CIFS_FSCACHE=y
CONFIG_CODA_FS=m
CONFIG_AFS_FS=m
# CONFIG_AFS_DEBUG is not set
CONFIG_AFS_FSCACHE=y
# CONFIG_AFS_DEBUG_CURSOR is not set
CONFIG_9P_FS=m
CONFIG_9P_FSCACHE=y
CONFIG_9P_FS_POSIX_ACL=y
CONFIG_9P_FS_SECURITY=y
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_MAC_ROMAN=m
CONFIG_NLS_MAC_CELTIC=m
CONFIG_NLS_MAC_CENTEURO=m
CONFIG_NLS_MAC_CROATIAN=m
CONFIG_NLS_MAC_CYRILLIC=m
CONFIG_NLS_MAC_GAELIC=m
CONFIG_NLS_MAC_GREEK=m
CONFIG_NLS_MAC_ICELAND=m
CONFIG_NLS_MAC_INUIT=m
CONFIG_NLS_MAC_ROMANIAN=m
CONFIG_NLS_MAC_TURKISH=m
CONFIG_NLS_UTF8=m
CONFIG_DLM=m
# CONFIG_DLM_DEBUG is not set
# CONFIG_UNICODE is not set
CONFIG_IO_WQ=y
# end of File systems
#
# Security options
#
CONFIG_KEYS=y
# CONFIG_KEYS_REQUEST_CACHE is not set
CONFIG_PERSISTENT_KEYRINGS=y
CONFIG_TRUSTED_KEYS=y
CONFIG_ENCRYPTED_KEYS=y
CONFIG_KEY_DH_OPERATIONS=y
# CONFIG_SECURITY_DMESG_RESTRICT is not set
CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_INFINIBAND=y
CONFIG_SECURITY_NETWORK_XFRM=y
CONFIG_SECURITY_PATH=y
CONFIG_LSM_MMAP_MIN_ADDR=0
CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
CONFIG_HARDENED_USERCOPY=y
CONFIG_HARDENED_USERCOPY_FALLBACK=y
# CONFIG_HARDENED_USERCOPY_PAGESPAN is not set
CONFIG_FORTIFY_SOURCE=y
# CONFIG_STATIC_USERMODEHELPER is not set
CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
# CONFIG_SECURITY_SELINUX_DISABLE is not set
CONFIG_SECURITY_SELINUX_DEVELOP=y
CONFIG_SECURITY_SELINUX_AVC_STATS=y
CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
CONFIG_SECURITY_SELINUX_SIDTAB_HASH_BITS=9
CONFIG_SECURITY_SELINUX_SID2STR_CACHE_SIZE=256
CONFIG_SECURITY_SMACK=y
# CONFIG_SECURITY_SMACK_BRINGUP is not set
CONFIG_SECURITY_SMACK_NETFILTER=y
CONFIG_SECURITY_SMACK_APPEND_SIGNALS=y
CONFIG_SECURITY_TOMOYO=y
CONFIG_SECURITY_TOMOYO_MAX_ACCEPT_ENTRY=2048
CONFIG_SECURITY_TOMOYO_MAX_AUDIT_LOG=1024
# CONFIG_SECURITY_TOMOYO_OMIT_USERSPACE_LOADER is not set
CONFIG_SECURITY_TOMOYO_POLICY_LOADER="/sbin/tomoyo-init"
CONFIG_SECURITY_TOMOYO_ACTIVATION_TRIGGER="/sbin/init"
# CONFIG_SECURITY_TOMOYO_INSECURE_BUILTIN_SETTING is not set
CONFIG_SECURITY_APPARMOR=y
CONFIG_SECURITY_APPARMOR_HASH=y
CONFIG_SECURITY_APPARMOR_HASH_DEFAULT=y
# CONFIG_SECURITY_APPARMOR_DEBUG is not set
# CONFIG_SECURITY_LOADPIN is not set
CONFIG_SECURITY_YAMA=y
# CONFIG_SECURITY_SAFESETID is not set
# CONFIG_SECURITY_LOCKDOWN_LSM is not set
CONFIG_INTEGRITY=y
CONFIG_INTEGRITY_SIGNATURE=y
CONFIG_INTEGRITY_ASYMMETRIC_KEYS=y
CONFIG_INTEGRITY_TRUSTED_KEYRING=y
# CONFIG_INTEGRITY_PLATFORM_KEYRING is not set
CONFIG_INTEGRITY_AUDIT=y
CONFIG_IMA=y
CONFIG_IMA_MEASURE_PCR_IDX=10
CONFIG_IMA_LSM_RULES=y
# CONFIG_IMA_TEMPLATE is not set
CONFIG_IMA_NG_TEMPLATE=y
# CONFIG_IMA_SIG_TEMPLATE is not set
CONFIG_IMA_DEFAULT_TEMPLATE="ima-ng"
CONFIG_IMA_DEFAULT_HASH_SHA1=y
# CONFIG_IMA_DEFAULT_HASH_SHA256 is not set
# CONFIG_IMA_DEFAULT_HASH_SHA512 is not set
CONFIG_IMA_DEFAULT_HASH="sha1"
# CONFIG_IMA_WRITE_POLICY is not set
# CONFIG_IMA_READ_POLICY is not set
CONFIG_IMA_APPRAISE=y
# CONFIG_IMA_ARCH_POLICY is not set
# CONFIG_IMA_APPRAISE_BUILD_POLICY is not set
CONFIG_IMA_APPRAISE_BOOTPARAM=y
# CONFIG_IMA_APPRAISE_MODSIG is not set
CONFIG_IMA_TRUSTED_KEYRING=y
# CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY is not set
# CONFIG_IMA_BLACKLIST_KEYRING is not set
# CONFIG_IMA_LOAD_X509 is not set
CONFIG_IMA_MEASURE_ASYMMETRIC_KEYS=y
CONFIG_IMA_QUEUE_EARLY_BOOT_KEYS=y
# CONFIG_IMA_SECURE_AND_OR_TRUSTED_BOOT is not set
CONFIG_EVM=y
CONFIG_EVM_ATTR_FSUUID=y
CONFIG_EVM_EXTRA_SMACK_XATTRS=y
# CONFIG_EVM_ADD_XATTRS is not set
# CONFIG_EVM_LOAD_X509 is not set
# CONFIG_DEFAULT_SECURITY_SELINUX is not set
# CONFIG_DEFAULT_SECURITY_SMACK is not set
# CONFIG_DEFAULT_SECURITY_TOMOYO is not set
CONFIG_DEFAULT_SECURITY_APPARMOR=y
# CONFIG_DEFAULT_SECURITY_DAC is not set
CONFIG_LSM="lockdown,yama,loadpin,safesetid,integrity,apparmor,selinux,smack,tomoyo"
#
# Kernel hardening options
#
#
# Memory initialization
#
CONFIG_INIT_STACK_NONE=y
# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
# end of Memory initialization
# end of Kernel hardening options
# end of Security options
CONFIG_XOR_BLOCKS=m
CONFIG_ASYNC_CORE=m
CONFIG_ASYNC_MEMCPY=m
CONFIG_ASYNC_XOR=m
CONFIG_ASYNC_PQ=m
CONFIG_ASYNC_RAID6_RECOV=m
CONFIG_ASYNC_TX_DISABLE_PQ_VAL_DMA=y
CONFIG_ASYNC_TX_DISABLE_XOR_VAL_DMA=y
CONFIG_CRYPTO=y
#
# Crypto core or helper
#
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_SKCIPHER=y
CONFIG_CRYPTO_SKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=y
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_RNG_DEFAULT=y
CONFIG_CRYPTO_AKCIPHER2=y
CONFIG_CRYPTO_AKCIPHER=y
CONFIG_CRYPTO_KPP2=y
CONFIG_CRYPTO_KPP=y
CONFIG_CRYPTO_ACOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
CONFIG_CRYPTO_USER=m
CONFIG_CRYPTO_MANAGER_DISABLE_TESTS=y
CONFIG_CRYPTO_GF128MUL=y
CONFIG_CRYPTO_NULL=y
CONFIG_CRYPTO_NULL2=y
CONFIG_CRYPTO_PCRYPT=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_AUTHENC=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_SIMD=m
CONFIG_CRYPTO_ENGINE=m
#
# Public-key cryptography
#
CONFIG_CRYPTO_RSA=y
CONFIG_CRYPTO_DH=y
CONFIG_CRYPTO_ECC=m
CONFIG_CRYPTO_ECDH=m
# CONFIG_CRYPTO_ECRDSA is not set
# CONFIG_CRYPTO_SM2 is not set
# CONFIG_CRYPTO_CURVE25519 is not set
#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=y
CONFIG_CRYPTO_CHACHA20POLY1305=m
# CONFIG_CRYPTO_AEGIS128 is not set
CONFIG_CRYPTO_SEQIV=y
CONFIG_CRYPTO_ECHAINIV=m
#
# Block modes
#
CONFIG_CRYPTO_CBC=y
# CONFIG_CRYPTO_CFB is not set
CONFIG_CRYPTO_CTR=y
CONFIG_CRYPTO_CTS=y
CONFIG_CRYPTO_ECB=y
CONFIG_CRYPTO_LRW=m
# CONFIG_CRYPTO_OFB is not set
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_XTS=y
CONFIG_CRYPTO_KEYWRAP=m
# CONFIG_CRYPTO_ADIANTUM is not set
CONFIG_CRYPTO_ESSIV=m
#
# Hash modes
#
CONFIG_CRYPTO_CMAC=m
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=m
#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_XXHASH=m
CONFIG_CRYPTO_BLAKE2B=m
# CONFIG_CRYPTO_BLAKE2S is not set
CONFIG_CRYPTO_CRCT10DIF=y
CONFIG_CRYPTO_GHASH=y
CONFIG_CRYPTO_POLY1305=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD128=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_RMD256=m
CONFIG_CRYPTO_RMD320=m
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
CONFIG_CRYPTO_SHA3=m
CONFIG_CRYPTO_SM3=m
# CONFIG_CRYPTO_STREEBOG is not set
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m
#
# Ciphers
#
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_TI=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_BLOWFISH_COMMON=m
CONFIG_CRYPTO_CAMELLIA=m
CONFIG_CRYPTO_CAST_COMMON=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_SALSA20=m
CONFIG_CRYPTO_CHACHA20=m
CONFIG_CRYPTO_SEED=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_SM4=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_TWOFISH_COMMON=m
#
# Compression
#
CONFIG_CRYPTO_DEFLATE=y
CONFIG_CRYPTO_LZO=y
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m
CONFIG_CRYPTO_ZSTD=m
#
# Random Number Generation
#
CONFIG_CRYPTO_ANSI_CPRNG=m
CONFIG_CRYPTO_DRBG_MENU=y
CONFIG_CRYPTO_DRBG_HMAC=y
CONFIG_CRYPTO_DRBG_HASH=y
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_DRBG=y
CONFIG_CRYPTO_JITTERENTROPY=y
CONFIG_CRYPTO_USER_API=m
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_USER_API_RNG_CAVP is not set
CONFIG_CRYPTO_USER_API_AEAD=m
CONFIG_CRYPTO_USER_API_ENABLE_OBSOLETE=y
# CONFIG_CRYPTO_STATS is not set
CONFIG_CRYPTO_HASH_INFO=y
#
# Crypto library routines
#
CONFIG_CRYPTO_LIB_AES=y
CONFIG_CRYPTO_LIB_ARC4=m
# CONFIG_CRYPTO_LIB_BLAKE2S is not set
CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA=m
CONFIG_CRYPTO_LIB_CHACHA_GENERIC=m
# CONFIG_CRYPTO_LIB_CHACHA is not set
# CONFIG_CRYPTO_LIB_CURVE25519 is not set
CONFIG_CRYPTO_LIB_DES=m
CONFIG_CRYPTO_LIB_POLY1305_RSIZE=9
CONFIG_CRYPTO_LIB_POLY1305_GENERIC=m
# CONFIG_CRYPTO_LIB_POLY1305 is not set
# CONFIG_CRYPTO_LIB_CHACHA20POLY1305 is not set
CONFIG_CRYPTO_LIB_SHA256=y
CONFIG_CRYPTO_HW=y
CONFIG_CRYPTO_DEV_ALLWINNER=y
# CONFIG_CRYPTO_DEV_SUN4I_SS is not set
# CONFIG_CRYPTO_DEV_SUN8I_CE is not set
# CONFIG_CRYPTO_DEV_SUN8I_SS is not set
CONFIG_CRYPTO_DEV_FSL_CAAM_COMMON=m
CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_DESC=m
CONFIG_CRYPTO_DEV_FSL_CAAM_AHASH_API_DESC=m
CONFIG_CRYPTO_DEV_FSL_CAAM=m
# CONFIG_CRYPTO_DEV_FSL_CAAM_DEBUG is not set
CONFIG_CRYPTO_DEV_FSL_CAAM_JR=m
CONFIG_CRYPTO_DEV_FSL_CAAM_RINGSIZE=9
CONFIG_CRYPTO_DEV_FSL_CAAM_INTC=y
CONFIG_CRYPTO_DEV_FSL_CAAM_INTC_COUNT_THLD=255
CONFIG_CRYPTO_DEV_FSL_CAAM_INTC_TIME_THLD=2048
CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API=y
CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI=y
CONFIG_CRYPTO_DEV_FSL_CAAM_AHASH_API=y
CONFIG_CRYPTO_DEV_FSL_CAAM_PKC_API=y
CONFIG_CRYPTO_DEV_FSL_CAAM_RNG_API=y
# CONFIG_CRYPTO_DEV_FSL_DPAA2_CAAM is not set
# CONFIG_CRYPTO_DEV_ATMEL_ECC is not set
# CONFIG_CRYPTO_DEV_ATMEL_SHA204A is not set
CONFIG_CRYPTO_DEV_CCP=y
CONFIG_CRYPTO_DEV_CCP_DD=m
CONFIG_CRYPTO_DEV_SP_CCP=y
CONFIG_CRYPTO_DEV_CCP_CRYPTO=m
# CONFIG_CRYPTO_DEV_CCP_DEBUGFS is not set
CONFIG_CRYPTO_DEV_CPT=m
CONFIG_CAVIUM_CPT=m
CONFIG_CRYPTO_DEV_NITROX=m
CONFIG_CRYPTO_DEV_NITROX_CNN55XX=m
# CONFIG_CRYPTO_DEV_OCTEONTX_CPT is not set
CONFIG_CRYPTO_DEV_CAVIUM_ZIP=m
CONFIG_CRYPTO_DEV_QCE=m
CONFIG_CRYPTO_DEV_QCE_SKCIPHER=y
CONFIG_CRYPTO_DEV_QCE_SHA=y
CONFIG_CRYPTO_DEV_QCE_ENABLE_ALL=y
# CONFIG_CRYPTO_DEV_QCE_ENABLE_SKCIPHER is not set
# CONFIG_CRYPTO_DEV_QCE_ENABLE_SHA is not set
CONFIG_CRYPTO_DEV_QCE_SW_MAX_LEN=512
# CONFIG_CRYPTO_DEV_QCOM_RNG is not set
# CONFIG_CRYPTO_DEV_ZYNQMP_AES is not set
CONFIG_CRYPTO_DEV_CHELSIO=m
CONFIG_CRYPTO_DEV_VIRTIO=m
CONFIG_CRYPTO_DEV_BCM_SPU=m
# CONFIG_CRYPTO_DEV_SAFEXCEL is not set
CONFIG_CRYPTO_DEV_CCREE=m
# CONFIG_CRYPTO_DEV_HISI_SEC is not set
# CONFIG_CRYPTO_DEV_HISI_SEC2 is not set
# CONFIG_CRYPTO_DEV_HISI_ZIP is not set
# CONFIG_CRYPTO_DEV_HISI_HPRE is not set
# CONFIG_CRYPTO_DEV_HISI_TRNG is not set
# CONFIG_CRYPTO_DEV_AMLOGIC_GXL is not set
CONFIG_ASYMMETRIC_KEY_TYPE=y
CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y
# CONFIG_ASYMMETRIC_TPM_KEY_SUBTYPE is not set
CONFIG_X509_CERTIFICATE_PARSER=y
# CONFIG_PKCS8_PRIVATE_KEY_PARSER is not set
CONFIG_PKCS7_MESSAGE_PARSER=y
CONFIG_PKCS7_TEST_KEY=m
CONFIG_SIGNED_PE_FILE_VERIFICATION=y
#
# Certificates for signature checking
#
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS=""
CONFIG_SYSTEM_EXTRA_CERTIFICATE=y
CONFIG_SYSTEM_EXTRA_CERTIFICATE_SIZE=4096
CONFIG_SECONDARY_TRUSTED_KEYRING=y
CONFIG_SYSTEM_BLACKLIST_KEYRING=y
CONFIG_SYSTEM_BLACKLIST_HASH_LIST=""
# end of Certificates for signature checking
CONFIG_BINARY_PRINTF=y
#
# Library routines
#
CONFIG_RAID6_PQ=m
CONFIG_RAID6_PQ_BENCHMARK=y
CONFIG_LINEAR_RANGES=y
# CONFIG_PACKING is not set
CONFIG_BITREVERSE=y
CONFIG_HAVE_ARCH_BITREVERSE=y
CONFIG_GENERIC_STRNCPY_FROM_USER=y
CONFIG_GENERIC_STRNLEN_USER=y
CONFIG_GENERIC_NET_UTILS=y
CONFIG_CORDIC=m
# CONFIG_PRIME_NUMBERS is not set
CONFIG_RATIONAL=y
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
CONFIG_ARCH_HAS_FAST_MULTIPLIER=y
CONFIG_ARCH_USE_SYM_ANNOTATIONS=y
CONFIG_INDIRECT_PIO=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=y
# CONFIG_CRC32_SELFTEST is not set
CONFIG_CRC32_SLICEBY8=y
# CONFIG_CRC32_SLICEBY4 is not set
# CONFIG_CRC32_SARWATE is not set
# CONFIG_CRC32_BIT is not set
CONFIG_CRC64=m
CONFIG_CRC4=m
CONFIG_CRC7=m
CONFIG_LIBCRC32C=m
CONFIG_CRC8=y
CONFIG_XXHASH=y
CONFIG_AUDIT_GENERIC=y
CONFIG_AUDIT_ARCH_COMPAT_GENERIC=y
CONFIG_AUDIT_COMPAT_GENERIC=y
# CONFIG_RANDOM32_SELFTEST is not set
CONFIG_842_COMPRESS=m
CONFIG_842_DECOMPRESS=m
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_LZ4_COMPRESS=m
CONFIG_LZ4HC_COMPRESS=m
CONFIG_LZ4_DECOMPRESS=y
CONFIG_ZSTD_COMPRESS=m
CONFIG_ZSTD_DECOMPRESS=y
CONFIG_XZ_DEC=y
CONFIG_XZ_DEC_X86=y
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
CONFIG_XZ_DEC_ARM=y
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
CONFIG_XZ_DEC_TEST=m
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_DECOMPRESS_LZO=y
CONFIG_DECOMPRESS_LZ4=y
CONFIG_DECOMPRESS_ZSTD=y
CONFIG_GENERIC_ALLOCATOR=y
CONFIG_REED_SOLOMON=m
CONFIG_REED_SOLOMON_ENC8=y
CONFIG_REED_SOLOMON_DEC8=y
CONFIG_BCH=m
CONFIG_BCH_CONST_PARAMS=y
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=m
CONFIG_TEXTSEARCH_BM=m
CONFIG_TEXTSEARCH_FSM=m
CONFIG_BTREE=y
CONFIG_INTERVAL_TREE=y
CONFIG_XARRAY_MULTI=y
CONFIG_ASSOCIATIVE_ARRAY=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT_MAP=y
CONFIG_HAS_DMA=y
CONFIG_DMA_OPS=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_DMA_DECLARE_COHERENT=y
CONFIG_ARCH_HAS_SETUP_DMA_OPS=y
CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS=y
CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE=y
CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU=y
CONFIG_ARCH_HAS_DMA_PREP_COHERENT=y
CONFIG_SWIOTLB=y
CONFIG_DMA_NONCOHERENT_MMAP=y
CONFIG_DMA_COHERENT_POOL=y
CONFIG_DMA_REMAP=y
CONFIG_DMA_DIRECT_REMAP=y
# CONFIG_DMA_CMA is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_DMA_MAP_BENCHMARK is not set
CONFIG_SGL_ALLOC=y
CONFIG_CHECK_SIGNATURE=y
CONFIG_CPU_RMAP=y
CONFIG_DQL=y
CONFIG_GLOB=y
# CONFIG_GLOB_SELFTEST is not set
CONFIG_NLATTR=y
CONFIG_LRU_CACHE=m
CONFIG_CLZ_TAB=y
CONFIG_IRQ_POLL=y
CONFIG_MPILIB=y
CONFIG_SIGNATURE=y
CONFIG_DIMLIB=y
CONFIG_LIBFDT=y
CONFIG_OID_REGISTRY=y
CONFIG_UCS2_STRING=y
CONFIG_HAVE_GENERIC_VDSO=y
CONFIG_GENERIC_GETTIMEOFDAY=y
CONFIG_GENERIC_VDSO_TIME_NS=y
CONFIG_FONT_SUPPORT=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_SG_POOL=y
CONFIG_ARCH_HAS_PMEM_API=y
CONFIG_MEMREGION=y
CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE=y
CONFIG_ARCH_STACKWALK=y
CONFIG_SBITMAP=y
CONFIG_PARMAN=m
CONFIG_OBJAGG=m
# CONFIG_STRING_SELFTEST is not set
# end of Library routines
CONFIG_GENERIC_LIB_DEVMEM_IS_ALLOWED=y
#
# Kernel hacking
#
#
# printk and dmesg options
#
CONFIG_PRINTK_TIME=y
# CONFIG_PRINTK_CALLER is not set
CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
CONFIG_CONSOLE_LOGLEVEL_QUIET=4
CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
CONFIG_BOOT_PRINTK_DELAY=y
CONFIG_DYNAMIC_DEBUG=y
CONFIG_DYNAMIC_DEBUG_CORE=y
CONFIG_SYMBOLIC_ERRNAME=y
CONFIG_DEBUG_BUGVERBOSE=y
# end of printk and dmesg options
#
# Compile-time checks and compiler options
#
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_COMPRESSED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
CONFIG_DEBUG_INFO_DWARF4=y
# CONFIG_DEBUG_INFO_BTF is not set
CONFIG_GDB_SCRIPTS=y
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_FRAME_WARN=1024
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_READABLE_ASM is not set
# CONFIG_HEADERS_INSTALL is not set
# CONFIG_DEBUG_SECTION_MISMATCH is not set
CONFIG_SECTION_MISMATCH_WARN_ONLY=y
# CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_32B is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
# end of Compile-time checks and compiler options
#
# Generic Kernel Debugging Instruments
#
CONFIG_MAGIC_SYSRQ=y
CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x01b6
CONFIG_MAGIC_SYSRQ_SERIAL=y
CONFIG_MAGIC_SYSRQ_SERIAL_SEQUENCE=""
CONFIG_DEBUG_FS=y
CONFIG_DEBUG_FS_ALLOW_ALL=y
# CONFIG_DEBUG_FS_DISALLOW_MOUNT is not set
# CONFIG_DEBUG_FS_ALLOW_NONE is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_KGDB=y
CONFIG_KGDB_HONOUR_BLOCKLIST=y
CONFIG_KGDB_SERIAL_CONSOLE=y
# CONFIG_KGDB_TESTS is not set
CONFIG_KGDB_KDB=y
CONFIG_KDB_DEFAULT_ENABLE=0x1
CONFIG_KDB_KEYBOARD=y
CONFIG_KDB_CONTINUE_CATASTROPHIC=0
CONFIG_ARCH_HAS_UBSAN_SANITIZE_ALL=y
# CONFIG_UBSAN is not set
# end of Generic Kernel Debugging Instruments
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_MISC=y
#
# Memory Debugging
#
# CONFIG_PAGE_EXTENSION is not set
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_PAGE_OWNER is not set
# CONFIG_PAGE_POISONING is not set
# CONFIG_DEBUG_PAGE_REF is not set
# CONFIG_DEBUG_RODATA_TEST is not set
CONFIG_ARCH_HAS_DEBUG_WX=y
CONFIG_DEBUG_WX=y
CONFIG_GENERIC_PTDUMP=y
CONFIG_PTDUMP_CORE=y
# CONFIG_PTDUMP_DEBUGFS is not set
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_SLUB_STATS is not set
CONFIG_HAVE_DEBUG_KMEMLEAK=y
# CONFIG_DEBUG_KMEMLEAK is not set
# CONFIG_DEBUG_STACK_USAGE is not set
CONFIG_SCHED_STACK_END_CHECK=y
CONFIG_ARCH_HAS_DEBUG_VM_PGTABLE=y
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VM_PGTABLE is not set
CONFIG_ARCH_HAS_DEBUG_VIRTUAL=y
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_MEMORY_INIT is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_HAVE_ARCH_KASAN_SW_TAGS=y
CONFIG_HAVE_ARCH_KASAN_HW_TAGS=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_CC_HAS_WORKING_NOSANITIZE_ADDRESS=y
# CONFIG_KASAN is not set
# end of Memory Debugging
# CONFIG_DEBUG_SHIRQ is not set
#
# Debug Oops, Lockups and Hangs
#
# CONFIG_PANIC_ON_OOPS is not set
CONFIG_PANIC_ON_OOPS_VALUE=0
CONFIG_PANIC_TIMEOUT=0
CONFIG_LOCKUP_DETECTOR=y
CONFIG_SOFTLOCKUP_DETECTOR=y
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
# CONFIG_WQ_WATCHDOG is not set
# CONFIG_TEST_LOCKUP is not set
# end of Debug Oops, Lockups and Hangs
#
# Scheduler Debugging
#
CONFIG_SCHED_DEBUG=y
CONFIG_SCHED_INFO=y
CONFIG_SCHEDSTATS=y
# end of Scheduler Debugging
# CONFIG_DEBUG_TIMEKEEPING is not set
#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_WW_MUTEX_SLOWPATH is not set
# CONFIG_DEBUG_RWSEMS is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_DEBUG_ATOMIC_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_LOCK_TORTURE_TEST is not set
# CONFIG_WW_MUTEX_SELFTEST is not set
# CONFIG_SCF_TORTURE_TEST is not set
# CONFIG_CSD_LOCK_WAIT_DEBUG is not set
# end of Lock Debugging (spinlocks, mutexes, etc...)
CONFIG_STACKTRACE=y
# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_HAVE_DEBUG_BUGVERBOSE=y
#
# Debug kernel data structures
#
# CONFIG_DEBUG_LIST is not set
# CONFIG_DEBUG_PLIST is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_BUG_ON_DATA_CORRUPTION is not set
# end of Debug kernel data structures
# CONFIG_DEBUG_CREDENTIALS is not set
#
# RCU Debugging
#
# CONFIG_RCU_SCALE_TEST is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_REF_SCALE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=60
# CONFIG_RCU_TRACE is not set
# CONFIG_RCU_EQS_DEBUG is not set
# end of RCU Debugging
# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_CPU_HOTPLUG_STATE_CONTROL is not set
# CONFIG_LATENCYTOP is not set
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_TRACE_CLOCK=y
CONFIG_RING_BUFFER=y
CONFIG_EVENT_TRACING=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_TRACING=y
CONFIG_GENERIC_TRACER=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
CONFIG_BOOTTIME_TRACING=y
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_FUNCTION_PROFILER=y
CONFIG_STACK_TRACER=y
# CONFIG_IRQSOFF_TRACER is not set
CONFIG_SCHED_TRACER=y
CONFIG_HWLAT_TRACER=y
CONFIG_FTRACE_SYSCALLS=y
CONFIG_TRACER_SNAPSHOT=y
# CONFIG_TRACER_SNAPSHOT_PER_CPU_SWAP is not set
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_KPROBE_EVENTS=y
CONFIG_UPROBE_EVENTS=y
CONFIG_BPF_EVENTS=y
CONFIG_DYNAMIC_EVENTS=y
CONFIG_PROBE_EVENTS=y
# CONFIG_BPF_KPROBE_OVERRIDE is not set
CONFIG_FTRACE_MCOUNT_RECORD=y
CONFIG_TRACING_MAP=y
CONFIG_SYNTH_EVENTS=y
CONFIG_HIST_TRIGGERS=y
# CONFIG_TRACE_EVENT_INJECT is not set
# CONFIG_TRACEPOINT_BENCHMARK is not set
# CONFIG_RING_BUFFER_BENCHMARK is not set
# CONFIG_TRACE_EVAL_MAP_FILE is not set
# CONFIG_FTRACE_RECORD_RECURSION is not set
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_RING_BUFFER_STARTUP_TEST is not set
# CONFIG_RING_BUFFER_VALIDATE_TIME_DELTAS is not set
# CONFIG_PREEMPTIRQ_DELAY_TEST is not set
# CONFIG_SYNTH_EVENT_GEN_TEST is not set
# CONFIG_KPROBE_EVENT_GEN_TEST is not set
# CONFIG_HIST_TRIGGERS_DEBUG is not set
# CONFIG_SAMPLES is not set
CONFIG_STRICT_DEVMEM=y
# CONFIG_IO_STRICT_DEVMEM is not set
#
# arm64 Debugging
#
# CONFIG_PID_IN_CONTEXTIDR is not set
# CONFIG_DEBUG_EFI is not set
# CONFIG_ARM64_RELOC_TEST is not set
# CONFIG_CORESIGHT is not set
# end of arm64 Debugging
#
# Kernel Testing and Coverage
#
# CONFIG_KUNIT is not set
CONFIG_NOTIFIER_ERROR_INJECTION=m
CONFIG_PM_NOTIFIER_ERROR_INJECT=m
# CONFIG_OF_RECONFIG_NOTIFIER_ERROR_INJECT is not set
# CONFIG_NETDEV_NOTIFIER_ERROR_INJECT is not set
CONFIG_FUNCTION_ERROR_INJECTION=y
# CONFIG_FAULT_INJECTION is not set
CONFIG_ARCH_HAS_KCOV=y
CONFIG_CC_HAS_SANCOV_TRACE_PC=y
# CONFIG_KCOV is not set
CONFIG_RUNTIME_TESTING_MENU=y
# CONFIG_LKDTM is not set
# CONFIG_TEST_LIST_SORT is not set
# CONFIG_TEST_MIN_HEAP is not set
# CONFIG_TEST_SORT is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_RBTREE_TEST is not set
# CONFIG_REED_SOLOMON_TEST is not set
# CONFIG_INTERVAL_TREE_TEST is not set
# CONFIG_PERCPU_TEST is not set
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_ASYNC_RAID6_TEST is not set
# CONFIG_TEST_HEXDUMP is not set
# CONFIG_TEST_STRING_HELPERS is not set
# CONFIG_TEST_STRSCPY is not set
# CONFIG_TEST_KSTRTOX is not set
# CONFIG_TEST_PRINTF is not set
# CONFIG_TEST_BITMAP is not set
# CONFIG_TEST_UUID is not set
# CONFIG_TEST_XARRAY is not set
# CONFIG_TEST_OVERFLOW is not set
# CONFIG_TEST_RHASHTABLE is not set
# CONFIG_TEST_HASH is not set
# CONFIG_TEST_IDA is not set
# CONFIG_TEST_PARMAN is not set
CONFIG_TEST_LKM=m
# CONFIG_TEST_BITOPS is not set
# CONFIG_TEST_VMALLOC is not set
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
# CONFIG_TEST_BLACKHOLE_DEV is not set
# CONFIG_FIND_BIT_BENCHMARK is not set
CONFIG_TEST_FIRMWARE=m
# CONFIG_TEST_SYSCTL is not set
CONFIG_TEST_UDELAY=m
CONFIG_TEST_STATIC_KEYS=m
# CONFIG_TEST_KMOD is not set
# CONFIG_TEST_MEMCAT_P is not set
# CONFIG_TEST_OBJAGG is not set
# CONFIG_TEST_STACKINIT is not set
# CONFIG_TEST_MEMINIT is not set
# CONFIG_TEST_FREE_PAGES is not set
CONFIG_MEMTEST=y
# end of Kernel Testing and Coverage
# end of Kernel hacking
[-- Attachment #3: fvp_run.sh --]
[-- Type: text/x-sh, Size: 2083 bytes --]
1 #!/bin/bash
2
3 PLATFORM=Base_RevC-2xAEMv8A
4 #PLATFORM=Base_AEMv8A
5 #FVP_PATH=/opt/arm/developmentstudio_platinum-2020.a/bin/
6 FVP_PATH=/home/haibo/Downloads/FVP_6196/
7 FVP_GCCV=Linux64_GCC-6.4
8 FW_PATH=/home/haibo/nv_test
9
10 MODEL=${FVP_PATH}/FVP_${PLATFORM}
11 DTB_IMG=${FW_PATH}/fvp-base-aemv8a-aemv8a.dtb
12 BL1_IMG=${FW_PATH}/bl1.bin
13 FIP_IMG=${FW_PATH}/fip.bin
14 KERNEL=${FW_PATH}/Image-5.11
15 INITRD=${FW_PATH}/ramdisk.img
16 DISK_IMG0=${FW_PATH}/rootfs.img
17
18 CLUSTER0_NUM_CORES=${CLUSTER0_NUM_CORES:-1}
19 CLUSTER1_NUM_CORES=${CLUSTER1_NUM_CORES:-1}
20 SECURE_MEMORY=${SECURE_MEMORY:-0}
21 CACHE_STATE_MODELLED=${CACHE_STATE_MODELLED:=0}
22
23 KERN_ADDR=0x80080000
24 DTB_ADDR=0x82000000
25 INITRD_ADDR=0x84000000
26
27 ${MODEL} \
28 -C pctl.startup=0.0.0.0 \
29 -C bp.secure_memory=$SECURE_MEMORY \
30 -C cache_state_modelled=$CACHE_STATE_MODELLED \
31 -C bp.pl011_uart0.untimed_fifos=1 \
32 -C bp.pl011_uart0.unbuffered_output=1 \
33 -C cluster0.NUM_CORES=$CLUSTER0_NUM_CORES \
34 -C cluster0.has_nested_virtualization=1 \
35 -C cluster0.has_arm_v8-1=1 \
36 -C cluster0.has_arm_v8-2=1 \
37 -C cluster0.has_arm_v8-3=1 \
38 -C cluster0.has_arm_v8-4=1 \
39 -C cluster1.NUM_CORES=$CLUSTER1_NUM_CORES \
40 -C cluster1.has_nested_virtualization=1 \
41 -C cluster1.has_arm_v8-1=1 \
42 -C cluster1.has_arm_v8-2=1 \
43 -C cluster1.has_arm_v8-3=1 \
44 -C cluster1.has_arm_v8-4=1 \
45 -C bp.secureflashloader.fname=${BL1_IMG} \
46 -C bp.flashloader0.fname=${FIP_IMG} \
47 --data cluster0.cpu0=${KERNEL}@${KERN_ADDR} \
48 --data cluster0.cpu0=${INITRD}@${INITRD_ADDR} \
49 --data cluster0.cpu0=${DTB_IMG}@${DTB_ADDR} \
50 -C bp.virtioblockdevice.image_path=${DISK_IMG0} \
51 -C bp.smsc_91c111.enabled=1 \
52 -C bp.hostbridge.interfaceName="arm_tap0" \
[-- Attachment #4: Type: text/plain, Size: 151 bytes --]
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2021-01-11 7:20 ` [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Haibo Xu
2021-01-11 8:59 ` Marc Zyngier
@ 2021-01-11 8:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-01-11 8:59 UTC (permalink / raw)
To: Haibo Xu; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
Hi Haibo,
On 2021-01-11 07:20, Haibo Xu wrote:
> On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>>
>> This is a rework of the NV series that I posted 10 months ago[1], as a
>> lot of the KVM code has changed since, and the series apply anymore
>> (not that anybody really cares as the the HW is, as usual, made of
>> unobtainium...).
>>
>> From the previous version:
>>
>> - Integration with the new page-table code
>> - New exception injection code
>> - No more messing with the nVHE code
>> - No AArch32!!!!
>> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>>
>> From a functionality perspective, you can expect a L2 guest to work,
>> but don't even think of L3, as we only partially emulate the
>> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
>> as well as anything that would require a Stage-1 PTW. What we want to
>> achieve is that with NV disabled, there is no performance overhead and
>> no regression.
>>
>> The series is roughly divided in 5 parts: exception handling, memory
>> virtualization, interrupts and timers for ARMv8.3, followed by the
>> ARMv8.4 support. There are of course some dependencies, but you'll
>> hopefully get the gist of it.
>>
>> For the most courageous of you, I've put out a branch[2]. Of course,
>> you'll need some userspace. Andre maintains a hacked version of
>> kvmtool[3] that takes a --nested option, allowing the guest to be
>> started at EL2. You can run the whole stack in the Foundation
>> model. Don't be in a hurry ;-).
>>
>
> Hi Marc,
>
> I got a kernel BUG message when booting the L2 guest kernel with the
> kvmtool on a FVP setup.
> Could you help have a look about the BUG message as well as my
> environment configuration?
> I think It probably caused by some local configurations of the FVP
> setup.
No, this is likely a bug in your L1 guest, which was fixed in -rc3:
2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is
available")
and was found in the exact same circumstances. Alternatively, and if
you don't want to change your L1 guest, you can just pass the --pmu
option to kvmtool when starting the L1 guest.
Hope this helps,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-11 8:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-01-11 8:59 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
Hi Haibo,
On 2021-01-11 07:20, Haibo Xu wrote:
> On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>>
>> This is a rework of the NV series that I posted 10 months ago[1], as a
>> lot of the KVM code has changed since, and the series apply anymore
>> (not that anybody really cares as the the HW is, as usual, made of
>> unobtainium...).
>>
>> From the previous version:
>>
>> - Integration with the new page-table code
>> - New exception injection code
>> - No more messing with the nVHE code
>> - No AArch32!!!!
>> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>>
>> From a functionality perspective, you can expect a L2 guest to work,
>> but don't even think of L3, as we only partially emulate the
>> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
>> as well as anything that would require a Stage-1 PTW. What we want to
>> achieve is that with NV disabled, there is no performance overhead and
>> no regression.
>>
>> The series is roughly divided in 5 parts: exception handling, memory
>> virtualization, interrupts and timers for ARMv8.3, followed by the
>> ARMv8.4 support. There are of course some dependencies, but you'll
>> hopefully get the gist of it.
>>
>> For the most courageous of you, I've put out a branch[2]. Of course,
>> you'll need some userspace. Andre maintains a hacked version of
>> kvmtool[3] that takes a --nested option, allowing the guest to be
>> started at EL2. You can run the whole stack in the Foundation
>> model. Don't be in a hurry ;-).
>>
>
> Hi Marc,
>
> I got a kernel BUG message when booting the L2 guest kernel with the
> kvmtool on a FVP setup.
> Could you help have a look about the BUG message as well as my
> environment configuration?
> I think It probably caused by some local configurations of the FVP
> setup.
No, this is likely a bug in your L1 guest, which was fixed in -rc3:
2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is
available")
and was found in the exact same circumstances. Alternatively, and if
you don't want to change your L1 guest, you can just pass the --pmu
option to kvmtool when starting the L1 guest.
Hope this helps,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-11 8:59 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-01-11 8:59 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
Hi Haibo,
On 2021-01-11 07:20, Haibo Xu wrote:
> On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>>
>> This is a rework of the NV series that I posted 10 months ago[1], as a
>> lot of the KVM code has changed since, and the series apply anymore
>> (not that anybody really cares as the the HW is, as usual, made of
>> unobtainium...).
>>
>> From the previous version:
>>
>> - Integration with the new page-table code
>> - New exception injection code
>> - No more messing with the nVHE code
>> - No AArch32!!!!
>> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>>
>> From a functionality perspective, you can expect a L2 guest to work,
>> but don't even think of L3, as we only partially emulate the
>> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
>> as well as anything that would require a Stage-1 PTW. What we want to
>> achieve is that with NV disabled, there is no performance overhead and
>> no regression.
>>
>> The series is roughly divided in 5 parts: exception handling, memory
>> virtualization, interrupts and timers for ARMv8.3, followed by the
>> ARMv8.4 support. There are of course some dependencies, but you'll
>> hopefully get the gist of it.
>>
>> For the most courageous of you, I've put out a branch[2]. Of course,
>> you'll need some userspace. Andre maintains a hacked version of
>> kvmtool[3] that takes a --nested option, allowing the guest to be
>> started at EL2. You can run the whole stack in the Foundation
>> model. Don't be in a hurry ;-).
>>
>
> Hi Marc,
>
> I got a kernel BUG message when booting the L2 guest kernel with the
> kvmtool on a FVP setup.
> Could you help have a look about the BUG message as well as my
> environment configuration?
> I think It probably caused by some local configurations of the FVP
> setup.
No, this is likely a bug in your L1 guest, which was fixed in -rc3:
2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is
available")
and was found in the exact same circumstances. Alternatively, and if
you don't want to change your L1 guest, you can just pass the --pmu
option to kvmtool when starting the L1 guest.
Hope this helps,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2021-01-11 8:59 ` Marc Zyngier
(?)
@ 2021-01-12 8:02 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-12 8:02 UTC (permalink / raw)
To: Marc Zyngier; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Mon, 11 Jan 2021 at 16:59, Marc Zyngier <maz@kernel.org> wrote:
>
> Hi Haibo,
>
> On 2021-01-11 07:20, Haibo Xu wrote:
> > On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> This is a rework of the NV series that I posted 10 months ago[1], as a
> >> lot of the KVM code has changed since, and the series apply anymore
> >> (not that anybody really cares as the the HW is, as usual, made of
> >> unobtainium...).
> >>
> >> From the previous version:
> >>
> >> - Integration with the new page-table code
> >> - New exception injection code
> >> - No more messing with the nVHE code
> >> - No AArch32!!!!
> >> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
> >>
> >> From a functionality perspective, you can expect a L2 guest to work,
> >> but don't even think of L3, as we only partially emulate the
> >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> >> as well as anything that would require a Stage-1 PTW. What we want to
> >> achieve is that with NV disabled, there is no performance overhead and
> >> no regression.
> >>
> >> The series is roughly divided in 5 parts: exception handling, memory
> >> virtualization, interrupts and timers for ARMv8.3, followed by the
> >> ARMv8.4 support. There are of course some dependencies, but you'll
> >> hopefully get the gist of it.
> >>
> >> For the most courageous of you, I've put out a branch[2]. Of course,
> >> you'll need some userspace. Andre maintains a hacked version of
> >> kvmtool[3] that takes a --nested option, allowing the guest to be
> >> started at EL2. You can run the whole stack in the Foundation
> >> model. Don't be in a hurry ;-).
> >>
> >
> > Hi Marc,
> >
> > I got a kernel BUG message when booting the L2 guest kernel with the
> > kvmtool on a FVP setup.
> > Could you help have a look about the BUG message as well as my
> > environment configuration?
> > I think It probably caused by some local configurations of the FVP
> > setup.
>
> No, this is likely a bug in your L1 guest, which was fixed in -rc3:
>
> 2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is
> available")
>
> and was found in the exact same circumstances. Alternatively, and if
> you don't want to change your L1 guest, you can just pass the --pmu
> option to kvmtool when starting the L1 guest.
After passing --pmu when starting a L1 guest, I can successfully run a
L2 guest now!
Thanks so much for the help!
Haibo
>
> Hope this helps,
>
> M.
> --
> Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-12 8:02 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-12 8:02 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Mon, 11 Jan 2021 at 16:59, Marc Zyngier <maz@kernel.org> wrote:
>
> Hi Haibo,
>
> On 2021-01-11 07:20, Haibo Xu wrote:
> > On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> This is a rework of the NV series that I posted 10 months ago[1], as a
> >> lot of the KVM code has changed since, and the series apply anymore
> >> (not that anybody really cares as the the HW is, as usual, made of
> >> unobtainium...).
> >>
> >> From the previous version:
> >>
> >> - Integration with the new page-table code
> >> - New exception injection code
> >> - No more messing with the nVHE code
> >> - No AArch32!!!!
> >> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
> >>
> >> From a functionality perspective, you can expect a L2 guest to work,
> >> but don't even think of L3, as we only partially emulate the
> >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> >> as well as anything that would require a Stage-1 PTW. What we want to
> >> achieve is that with NV disabled, there is no performance overhead and
> >> no regression.
> >>
> >> The series is roughly divided in 5 parts: exception handling, memory
> >> virtualization, interrupts and timers for ARMv8.3, followed by the
> >> ARMv8.4 support. There are of course some dependencies, but you'll
> >> hopefully get the gist of it.
> >>
> >> For the most courageous of you, I've put out a branch[2]. Of course,
> >> you'll need some userspace. Andre maintains a hacked version of
> >> kvmtool[3] that takes a --nested option, allowing the guest to be
> >> started at EL2. You can run the whole stack in the Foundation
> >> model. Don't be in a hurry ;-).
> >>
> >
> > Hi Marc,
> >
> > I got a kernel BUG message when booting the L2 guest kernel with the
> > kvmtool on a FVP setup.
> > Could you help have a look about the BUG message as well as my
> > environment configuration?
> > I think It probably caused by some local configurations of the FVP
> > setup.
>
> No, this is likely a bug in your L1 guest, which was fixed in -rc3:
>
> 2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is
> available")
>
> and was found in the exact same circumstances. Alternatively, and if
> you don't want to change your L1 guest, you can just pass the --pmu
> option to kvmtool when starting the L1 guest.
After passing --pmu when starting a L1 guest, I can successfully run a
L2 guest now!
Thanks so much for the help!
Haibo
>
> Hope this helps,
>
> M.
> --
> Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-12 8:02 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-12 8:02 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Mon, 11 Jan 2021 at 16:59, Marc Zyngier <maz@kernel.org> wrote:
>
> Hi Haibo,
>
> On 2021-01-11 07:20, Haibo Xu wrote:
> > On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> This is a rework of the NV series that I posted 10 months ago[1], as a
> >> lot of the KVM code has changed since, and the series apply anymore
> >> (not that anybody really cares as the the HW is, as usual, made of
> >> unobtainium...).
> >>
> >> From the previous version:
> >>
> >> - Integration with the new page-table code
> >> - New exception injection code
> >> - No more messing with the nVHE code
> >> - No AArch32!!!!
> >> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
> >>
> >> From a functionality perspective, you can expect a L2 guest to work,
> >> but don't even think of L3, as we only partially emulate the
> >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> >> as well as anything that would require a Stage-1 PTW. What we want to
> >> achieve is that with NV disabled, there is no performance overhead and
> >> no regression.
> >>
> >> The series is roughly divided in 5 parts: exception handling, memory
> >> virtualization, interrupts and timers for ARMv8.3, followed by the
> >> ARMv8.4 support. There are of course some dependencies, but you'll
> >> hopefully get the gist of it.
> >>
> >> For the most courageous of you, I've put out a branch[2]. Of course,
> >> you'll need some userspace. Andre maintains a hacked version of
> >> kvmtool[3] that takes a --nested option, allowing the guest to be
> >> started at EL2. You can run the whole stack in the Foundation
> >> model. Don't be in a hurry ;-).
> >>
> >
> > Hi Marc,
> >
> > I got a kernel BUG message when booting the L2 guest kernel with the
> > kvmtool on a FVP setup.
> > Could you help have a look about the BUG message as well as my
> > environment configuration?
> > I think It probably caused by some local configurations of the FVP
> > setup.
>
> No, this is likely a bug in your L1 guest, which was fixed in -rc3:
>
> 2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is
> available")
>
> and was found in the exact same circumstances. Alternatively, and if
> you don't want to change your L1 guest, you can just pass the --pmu
> option to kvmtool when starting the L1 guest.
After passing --pmu when starting a L1 guest, I can successfully run a
L2 guest now!
Thanks so much for the help!
Haibo
>
> Hope this helps,
>
> M.
> --
> Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2021-01-18 17:38 ` André Przywara
-1 siblings, 0 replies; 249+ messages in thread
From: André Przywara @ 2021-01-18 17:38 UTC (permalink / raw)
To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm
Cc: Christoffer Dall, Jintack Lim, Alexandru Elisei, James Morse,
Julien Thierry, Suzuki K Poulose, kernel-team, Haibo Xu
On 10/12/2020 15:58, Marc Zyngier wrote:
Hi,
> This is a rework of the NV series that I posted 10 months ago[1], as a
....
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2.
I have updated the branch, as the headers changed as well:
https://gitlab.arm.com/linux-arm/kvmtool/-/commits/nv-5.11-WIP/
Cheers,
Andre
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-18 17:38 ` André Przywara
0 siblings, 0 replies; 249+ messages in thread
From: André Przywara @ 2021-01-18 17:38 UTC (permalink / raw)
To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm; +Cc: kernel-team
On 10/12/2020 15:58, Marc Zyngier wrote:
Hi,
> This is a rework of the NV series that I posted 10 months ago[1], as a
....
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2.
I have updated the branch, as the headers changed as well:
https://gitlab.arm.com/linux-arm/kvmtool/-/commits/nv-5.11-WIP/
Cheers,
Andre
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-18 17:38 ` André Przywara
0 siblings, 0 replies; 249+ messages in thread
From: André Przywara @ 2021-01-18 17:38 UTC (permalink / raw)
To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm
Cc: Suzuki K Poulose, kernel-team, Christoffer Dall, James Morse,
Haibo Xu, Alexandru Elisei, Jintack Lim, Julien Thierry
On 10/12/2020 15:58, Marc Zyngier wrote:
Hi,
> This is a rework of the NV series that I posted 10 months ago[1], as a
....
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2.
I have updated the branch, as the headers changed as well:
https://gitlab.arm.com/linux-arm/kvmtool/-/commits/nv-5.11-WIP/
Cheers,
Andre
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2020-12-10 15:58 ` Marc Zyngier
` (69 preceding siblings ...)
(?)
@ 2021-01-21 2:35 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:35 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
[-- Attachment #1.1: Type: text/plain, Size: 2569 bytes --]
On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>
> This is a rework of the NV series that I posted 10 months ago[1], as a
> lot of the KVM code has changed since, and the series apply anymore
> (not that anybody really cares as the the HW is, as usual, made of
> unobtainium...).
>
> From the previous version:
>
> - Integration with the new page-table code
> - New exception injection code
> - No more messing with the nVHE code
> - No AArch32!!!!
> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>
> From a functionality perspective, you can expect a L2 guest to work,
> but don't even think of L3, as we only partially emulate the
> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> as well as anything that would require a Stage-1 PTW. What we want to
> achieve is that with NV disabled, there is no performance overhead and
> no regression.
>
> The series is roughly divided in 5 parts: exception handling, memory
> virtualization, interrupts and timers for ARMv8.3, followed by the
> ARMv8.4 support. There are of course some dependencies, but you'll
> hopefully get the gist of it.
>
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2. You can run the whole stack in the Foundation
> model. Don't be in a hurry ;-).
>
> And to be clear: although Jintack and Christoffer have written tons of
> the stuff originaly, I'm the one responsible for breaking it!
>
> [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
kvm-arm64/nv-5.11.-WIP
> [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
Hi Marc,
I have tried to enable the NV support in Qemu, and now I can successfully
boot a L2 guest
in Qemu KVM mode.
This patch series looks good from the Qemu side except for two minor
requirements:
(1) Qemu will check whether a feature was supported by the KVM cap when the
user tries
to enable it in the command line, so a new capability was prefered for
the NV(KVM_CAP_ARM_NV?).
(2) According to the Documentation/virt/kvm/api.rst
<https://github.com/torvalds/linux/blob/master/Documentation/virt/kvm/api.rst#482-kvm_arm_vcpu_init>,
userspace can call KVM_ARM_VCPU_INIT
multiple times for a given vcpu, but the kvm_vcpu_init_nested() do
have some issue when
called multiple times(please refer to the detailed comments in patch
63)
Regards,
Haibo
[-- Attachment #1.2: Type: text/html, Size: 3211 bytes --]
[-- Attachment #2: Type: text/plain, Size: 151 bytes --]
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
2020-12-10 15:59 ` Marc Zyngier
(?)
@ 2021-01-21 2:47 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:47 UTC (permalink / raw)
To: Marc Zyngier; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
>
> If running a NV guest on an ARMv8.4-NV capable system, let's
> allocate an additional page that will be used by the hypervisor
> to fulfill system register accesses.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 3 ++-
> arch/arm64/kvm/nested.c | 8 ++++++++
> arch/arm64/kvm/reset.c | 1 +
> 3 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 78630bd5124d..dada0678c28e 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
> */
> static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
> {
> - if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
> + if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
> + r >= __VNCR_START__ && ctxt->vncr_array))
> return &ctxt->vncr_array[r - __VNCR_START__];
>
> return (u64 *)&ctxt->sys_regs[r];
> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index eef8f9873814..88147ec99755 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> return -EINVAL;
>
> + if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
> + vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
> + if (!vcpu->arch.ctxt.vncr_array)
> + return -ENOMEM;
> + }
> +
If KVM_ARM_VCPU_INIT was called multiple times, the above codes would
try to allocate a new page
without free-ing the previous one. Besides that, the following
kvm_free_stage2_pgd() call would fail in the
second call with the error message "kvm_arch already initialized?".
I think a possible fix is to add a new flag to indicate whether the NV
related meta data have been initialized,
and only initialize them for the first call.
> mutex_lock(&kvm->lock);
>
> /*
> @@ -64,6 +70,8 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
> kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
> kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
> + free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> + vcpu->arch.ctxt.vncr_array = NULL;
> } else {
> kvm->arch.nested_mmus_size = num_mmus;
> ret = 0;
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index 2d2c780e6c69..d281eb39036f 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -150,6 +150,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu)
> void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
> {
> kfree(vcpu->arch.sve_state);
> + free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> }
>
> static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
@ 2021-01-21 2:47 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:47 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
>
> If running a NV guest on an ARMv8.4-NV capable system, let's
> allocate an additional page that will be used by the hypervisor
> to fulfill system register accesses.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 3 ++-
> arch/arm64/kvm/nested.c | 8 ++++++++
> arch/arm64/kvm/reset.c | 1 +
> 3 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 78630bd5124d..dada0678c28e 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
> */
> static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
> {
> - if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
> + if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
> + r >= __VNCR_START__ && ctxt->vncr_array))
> return &ctxt->vncr_array[r - __VNCR_START__];
>
> return (u64 *)&ctxt->sys_regs[r];
> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index eef8f9873814..88147ec99755 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> return -EINVAL;
>
> + if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
> + vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
> + if (!vcpu->arch.ctxt.vncr_array)
> + return -ENOMEM;
> + }
> +
If KVM_ARM_VCPU_INIT was called multiple times, the above codes would
try to allocate a new page
without free-ing the previous one. Besides that, the following
kvm_free_stage2_pgd() call would fail in the
second call with the error message "kvm_arch already initialized?".
I think a possible fix is to add a new flag to indicate whether the NV
related meta data have been initialized,
and only initialize them for the first call.
> mutex_lock(&kvm->lock);
>
> /*
> @@ -64,6 +70,8 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
> kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
> kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
> + free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> + vcpu->arch.ctxt.vncr_array = NULL;
> } else {
> kvm->arch.nested_mmus_size = num_mmus;
> ret = 0;
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index 2d2c780e6c69..d281eb39036f 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -150,6 +150,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu)
> void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
> {
> kfree(vcpu->arch.sve_state);
> + free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> }
>
> static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
@ 2021-01-21 2:47 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:47 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
>
> If running a NV guest on an ARMv8.4-NV capable system, let's
> allocate an additional page that will be used by the hypervisor
> to fulfill system register accesses.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 3 ++-
> arch/arm64/kvm/nested.c | 8 ++++++++
> arch/arm64/kvm/reset.c | 1 +
> 3 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 78630bd5124d..dada0678c28e 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
> */
> static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
> {
> - if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
> + if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
> + r >= __VNCR_START__ && ctxt->vncr_array))
> return &ctxt->vncr_array[r - __VNCR_START__];
>
> return (u64 *)&ctxt->sys_regs[r];
> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index eef8f9873814..88147ec99755 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> return -EINVAL;
>
> + if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
> + vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
> + if (!vcpu->arch.ctxt.vncr_array)
> + return -ENOMEM;
> + }
> +
If KVM_ARM_VCPU_INIT was called multiple times, the above codes would
try to allocate a new page
without free-ing the previous one. Besides that, the following
kvm_free_stage2_pgd() call would fail in the
second call with the error message "kvm_arch already initialized?".
I think a possible fix is to add a new flag to indicate whether the NV
related meta data have been initialized,
and only initialize them for the first call.
> mutex_lock(&kvm->lock);
>
> /*
> @@ -64,6 +70,8 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
> kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
> kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
> + free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> + vcpu->arch.ctxt.vncr_array = NULL;
> } else {
> kvm->arch.nested_mmus_size = num_mmus;
> ret = 0;
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index 2d2c780e6c69..d281eb39036f 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -150,6 +150,7 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu)
> void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
> {
> kfree(vcpu->arch.sve_state);
> + free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> }
>
> static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
2020-12-10 15:59 ` Marc Zyngier
(?)
@ 2021-01-21 2:59 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:59 UTC (permalink / raw)
To: Marc Zyngier; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
>
> Add Stage-2 mmu data structures for virtual EL2 and for nested guests.
> We don't yet populate shadow Stage-2 page tables, but we now have a
> framework for getting to a shadow Stage-2 pgd.
>
> We allocate twice the number of vcpus as Stage-2 mmu structures because
> that's sufficient for each vcpu running two translation regimes without
> having to flush the Stage-2 page tables.
>
> Co-developed-by: Christoffer Dall <christoffer.dall@arm.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 29 +++++
> arch/arm64/include/asm/kvm_mmu.h | 8 ++
> arch/arm64/include/asm/kvm_nested.h | 7 ++
> arch/arm64/kvm/arm.c | 16 ++-
> arch/arm64/kvm/mmu.c | 18 ++-
> arch/arm64/kvm/nested.c | 183 ++++++++++++++++++++++++++++
> 6 files changed, 250 insertions(+), 11 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d731cf7a56cb..d99e51e7cbee 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -95,14 +95,43 @@ struct kvm_s2_mmu {
> int __percpu *last_vcpu_ran;
>
> struct kvm *kvm;
> +
> + /*
> + * For a shadow stage-2 MMU, the virtual vttbr programmed by the guest
> + * hypervisor. Unused for kvm_arch->mmu. Set to 1 when the structure
> + * contains no valid information.
> + */
> + u64 vttbr;
> +
> + /* true when this represents a nested context where virtual HCR_EL2.VM == 1 */
> + bool nested_stage2_enabled;
> +
> + /*
> + * 0: Nobody is currently using this, check vttbr for validity
> + * >0: Somebody is actively using this.
> + */
> + atomic_t refcnt;
> };
>
> +static inline bool kvm_s2_mmu_valid(struct kvm_s2_mmu *mmu)
> +{
> + return !(mmu->vttbr & 1);
> +}
> +
> struct kvm_arch_memory_slot {
> };
>
> struct kvm_arch {
> struct kvm_s2_mmu mmu;
>
> + /*
> + * Stage 2 paging stage for VMs with nested virtual using a virtual
> + * VMID.
> + */
> + struct kvm_s2_mmu *nested_mmus;
> + size_t nested_mmus_size;
> + int nested_mmus_next;
> +
> /* VTCR_EL2 value for this VM */
> u64 vtcr;
>
> diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
> index 76a8a0ca45b8..ec39015bb2a6 100644
> --- a/arch/arm64/include/asm/kvm_mmu.h
> +++ b/arch/arm64/include/asm/kvm_mmu.h
> @@ -126,6 +126,7 @@ alternative_cb_end
> #include <asm/cacheflush.h>
> #include <asm/mmu_context.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_nested.h>
>
> void kvm_update_va_mask(struct alt_instr *alt,
> __le32 *origptr, __le32 *updptr, int nr_inst);
> @@ -184,6 +185,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
> void **haddr);
> void free_hyp_pgds(void);
>
> +void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
> void stage2_unmap_vm(struct kvm *kvm);
> int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
> void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
> @@ -306,5 +308,11 @@ static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu)
> asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
> }
>
> +static inline u64 get_vmid(u64 vttbr)
> +{
> + return (vttbr & VTTBR_VMID_MASK(kvm_get_vmid_bits())) >>
> + VTTBR_VMID_SHIFT;
> +}
> +
> #endif /* __ASSEMBLY__ */
> #endif /* __ARM64_KVM_MMU_H__ */
> diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
> index 026ddaad972c..473ecd1d60d0 100644
> --- a/arch/arm64/include/asm/kvm_nested.h
> +++ b/arch/arm64/include/asm/kvm_nested.h
> @@ -61,6 +61,13 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
> (cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
> }
>
> +extern void kvm_init_nested(struct kvm *kvm);
> +extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
> +extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
> +extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
> +extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
> +extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
> +
> int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
> extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
> u64 control_bit);
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 6e637d2b4cfb..1656dd80bbc4 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -35,6 +35,7 @@
> #include <asm/kvm_arm.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmu.h>
> +#include <asm/kvm_nested.h>
> #include <asm/kvm_emulate.h>
> #include <asm/sections.h>
>
> @@ -142,6 +143,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> if (ret)
> return ret;
>
> + kvm_init_nested(kvm);
> +
> ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
> if (ret)
> goto out_free_stage2_pgd;
> @@ -385,6 +388,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> struct kvm_s2_mmu *mmu;
> int *last_ran;
>
> + if (nested_virt_in_use(vcpu))
> + kvm_vcpu_load_hw_mmu(vcpu);
> +
> mmu = vcpu->arch.hw_mmu;
> last_ran = this_cpu_ptr(mmu->last_vcpu_ran);
>
> @@ -426,6 +432,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> kvm_vgic_put(vcpu);
> kvm_vcpu_pmu_restore_host(vcpu);
>
> + if (nested_virt_in_use(vcpu))
> + kvm_vcpu_put_hw_mmu(vcpu);
> +
> vcpu->cpu = -1;
> }
>
> @@ -1026,8 +1035,13 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
>
> vcpu->arch.target = phys_target;
>
> + /* Prepare for nested if required */
> + ret = kvm_vcpu_init_nested(vcpu);
> +
> /* Now we know what it is, we can reset it. */
> - ret = kvm_reset_vcpu(vcpu);
> + if (!ret)
> + ret = kvm_reset_vcpu(vcpu);
> +
> if (ret) {
> vcpu->arch.target = -1;
> bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES);
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 1f41173e6149..2f0302211af3 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -113,7 +113,7 @@ static bool kvm_is_device_pfn(unsigned long pfn)
> * does.
> */
> /**
> - * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
> + * kvm_unmap_stage2_range -- Clear stage2 page table entries to unmap a range
> * @mmu: The KVM stage-2 MMU pointer
> * @start: The intermediate physical base address of the range to unmap
> * @size: The size of the area to unmap
> @@ -136,7 +136,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
> may_block));
> }
>
> -static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
> +void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
> {
> __unmap_stage2_range(mmu, start, size, true);
> }
> @@ -391,6 +391,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
> mmu->pgt = pgt;
> mmu->pgd_phys = __pa(pgt->pgd);
> mmu->vmid.vmid_gen = 0;
> +
> + kvm_init_nested_s2_mmu(mmu);
> +
> return 0;
>
> out_destroy_pgtable:
> @@ -435,7 +438,7 @@ static void stage2_unmap_memslot(struct kvm *kvm,
>
> if (!(vma->vm_flags & VM_PFNMAP)) {
> gpa_t gpa = addr + (vm_start - memslot->userspace_addr);
> - unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
> }
> hva = vm_end;
> } while (hva < reg_end);
> @@ -1360,7 +1363,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
>
> spin_lock(&kvm->mmu_lock);
> if (ret)
> - unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
> else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
> stage2_flush_memslot(kvm, memslot);
> spin_unlock(&kvm->mmu_lock);
> @@ -1377,11 +1380,6 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
> {
> }
>
> -void kvm_arch_flush_shadow_all(struct kvm *kvm)
> -{
> - kvm_free_stage2_pgd(&kvm->arch.mmu);
> -}
> -
> void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> struct kvm_memory_slot *slot)
> {
> @@ -1389,7 +1387,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> phys_addr_t size = slot->npages << PAGE_SHIFT;
>
> spin_lock(&kvm->mmu_lock);
> - unmap_stage2_range(&kvm->arch.mmu, gpa, size);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
> spin_unlock(&kvm->mmu_lock);
> }
>
> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index 9fb44bc7db3f..8e85d2ef24d9 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -19,12 +19,177 @@
> #include <linux/kvm.h>
> #include <linux/kvm_host.h>
>
> +#include <asm/kvm_arm.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_mmu.h>
> #include <asm/kvm_nested.h>
> #include <asm/sysreg.h>
>
> #include "sys_regs.h"
>
> +void kvm_init_nested(struct kvm *kvm)
> +{
> + kvm->arch.nested_mmus = NULL;
> + kvm->arch.nested_mmus_size = 0;
> +}
> +
> +int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct kvm_s2_mmu *tmp;
> + int num_mmus;
> + int ret = -ENOMEM;
> +
> + if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
> + return 0;
> +
> + if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> + return -EINVAL;
nit: returning a "not supported" kind of errno?
> +
> + mutex_lock(&kvm->lock);
> +
> + /*
> + * Let's treat memory allocation failures as benign: If we fail to
> + * allocate anything, return an error and keep the allocated array
> + * alive. Userspace may try to recover by intializing the vcpu
> + * again, and there is no reason to affect the whole VM for this.
> + */
> + num_mmus = atomic_read(&kvm->online_vcpus) * 2;
> + tmp = krealloc(kvm->arch.nested_mmus,
> + num_mmus * sizeof(*kvm->arch.nested_mmus),
> + GFP_KERNEL | __GFP_ZERO);
> + if (tmp) {
> + if (kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 1]) ||
> + kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
> + kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
> + kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
> + } else {
> + kvm->arch.nested_mmus_size = num_mmus;
> + ret = 0;
> + }
> +
> + kvm->arch.nested_mmus = tmp;
> + }
> +
> + mutex_unlock(&kvm->lock);
> + return ret;
> +}
> +
> +/* Must be called with kvm->lock held */
> +struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
> +{
> + bool nested_stage2_enabled = hcr & HCR_VM;
> + int i;
> +
> + /* Don't consider the CnP bit for the vttbr match */
> + vttbr = vttbr & ~VTTBR_CNP_BIT;
> +
> + /*
> + * Two possibilities when looking up a S2 MMU context:
> + *
> + * - either S2 is enabled in the guest, and we need a context that
> + * is S2-enabled and matches the full VTTBR (VMID+BADDR), which
> + * makes it safe from a TLB conflict perspective (a broken guest
> + * won't be able to generate them),
> + *
> + * - or S2 is disabled, and we need a context that is S2-disabled
> + * and matches the VMID only, as all TLBs are tagged by VMID even
> + * if S2 translation is enabled.
> + */
> + for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
> + struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
> +
> + if (!kvm_s2_mmu_valid(mmu))
> + continue;
> +
> + if (nested_stage2_enabled &&
> + mmu->nested_stage2_enabled &&
> + vttbr == mmu->vttbr)
> + return mmu;
> +
> + if (!nested_stage2_enabled &&
> + !mmu->nested_stage2_enabled &&
> + get_vmid(vttbr) == get_vmid(mmu->vttbr))
> + return mmu;
> + }
> + return NULL;
> +}
> +
> +static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
> + u64 hcr= vcpu_read_sys_reg(vcpu, HCR_EL2);
> + struct kvm_s2_mmu *s2_mmu;
> + int i;
> +
> + s2_mmu = lookup_s2_mmu(kvm, vttbr, hcr);
> + if (s2_mmu)
> + goto out;
> +
> + /*
> + * Make sure we don't always search from the same point, or we
> + * will always reuse a potentially active context, leaving
> + * free contexts unused.
> + */
> + for (i = kvm->arch.nested_mmus_next;
> + i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
> + i++) {
> + s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
> +
> + if (atomic_read(&s2_mmu->refcnt) == 0)
> + break;
> + }
> + BUG_ON(atomic_read(&s2_mmu->refcnt)); /* We have struct MMUs to spare */
> +
> + /* Set the scene for the next search */
> + kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
> +
> + if (kvm_s2_mmu_valid(s2_mmu)) {
> + /* Clear the old state */
> + kvm_unmap_stage2_range(s2_mmu, 0, kvm_phys_size(kvm));
> + if (s2_mmu->vmid.vmid_gen)
> + kvm_call_hyp(__kvm_tlb_flush_vmid, s2_mmu);
> + }
> +
> + /*
> + * The virtual VMID (modulo CnP) will be used as a key when matching
> + * an existing kvm_s2_mmu.
> + */
> + s2_mmu->vttbr = vttbr & ~VTTBR_CNP_BIT;
> + s2_mmu->nested_stage2_enabled = hcr & HCR_VM;
> +
> +out:
> + atomic_inc(&s2_mmu->refcnt);
> + return s2_mmu;
> +}
> +
> +void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)
> +{
> + mmu->vttbr = 1;
> + mmu->nested_stage2_enabled = false;
> + atomic_set(&mmu->refcnt, 0);
> +}
> +
> +void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
> +{
> + if (is_hyp_ctxt(vcpu)) {
> + vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
> + } else {
> + spin_lock(&vcpu->kvm->mmu_lock);
> + vcpu->arch.hw_mmu = get_s2_mmu_nested(vcpu);
> + spin_unlock(&vcpu->kvm->mmu_lock);
> + }
> +}
> +
> +void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu) {
> + atomic_dec(&vcpu->arch.hw_mmu->refcnt);
> + vcpu->arch.hw_mmu = NULL;
> + }
> +}
> +
> /*
> * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
> * the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
> @@ -43,6 +208,24 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
> return -EINVAL;
> }
>
> +void kvm_arch_flush_shadow_all(struct kvm *kvm)
> +{
> + int i;
> +
> + for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
> + struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
> +
> + WARN_ON(atomic_read(&mmu->refcnt));
> +
> + if (!atomic_read(&mmu->refcnt))
> + kvm_free_stage2_pgd(mmu);
> + }
> + kfree(kvm->arch.nested_mmus);
> + kvm->arch.nested_mmus = NULL;
> + kvm->arch.nested_mmus_size = 0;
> + kvm_free_stage2_pgd(&kvm->arch.mmu);
> +}
> +
> #define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
>
> /*
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
@ 2021-01-21 2:59 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:59 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
>
> Add Stage-2 mmu data structures for virtual EL2 and for nested guests.
> We don't yet populate shadow Stage-2 page tables, but we now have a
> framework for getting to a shadow Stage-2 pgd.
>
> We allocate twice the number of vcpus as Stage-2 mmu structures because
> that's sufficient for each vcpu running two translation regimes without
> having to flush the Stage-2 page tables.
>
> Co-developed-by: Christoffer Dall <christoffer.dall@arm.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 29 +++++
> arch/arm64/include/asm/kvm_mmu.h | 8 ++
> arch/arm64/include/asm/kvm_nested.h | 7 ++
> arch/arm64/kvm/arm.c | 16 ++-
> arch/arm64/kvm/mmu.c | 18 ++-
> arch/arm64/kvm/nested.c | 183 ++++++++++++++++++++++++++++
> 6 files changed, 250 insertions(+), 11 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d731cf7a56cb..d99e51e7cbee 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -95,14 +95,43 @@ struct kvm_s2_mmu {
> int __percpu *last_vcpu_ran;
>
> struct kvm *kvm;
> +
> + /*
> + * For a shadow stage-2 MMU, the virtual vttbr programmed by the guest
> + * hypervisor. Unused for kvm_arch->mmu. Set to 1 when the structure
> + * contains no valid information.
> + */
> + u64 vttbr;
> +
> + /* true when this represents a nested context where virtual HCR_EL2.VM == 1 */
> + bool nested_stage2_enabled;
> +
> + /*
> + * 0: Nobody is currently using this, check vttbr for validity
> + * >0: Somebody is actively using this.
> + */
> + atomic_t refcnt;
> };
>
> +static inline bool kvm_s2_mmu_valid(struct kvm_s2_mmu *mmu)
> +{
> + return !(mmu->vttbr & 1);
> +}
> +
> struct kvm_arch_memory_slot {
> };
>
> struct kvm_arch {
> struct kvm_s2_mmu mmu;
>
> + /*
> + * Stage 2 paging stage for VMs with nested virtual using a virtual
> + * VMID.
> + */
> + struct kvm_s2_mmu *nested_mmus;
> + size_t nested_mmus_size;
> + int nested_mmus_next;
> +
> /* VTCR_EL2 value for this VM */
> u64 vtcr;
>
> diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
> index 76a8a0ca45b8..ec39015bb2a6 100644
> --- a/arch/arm64/include/asm/kvm_mmu.h
> +++ b/arch/arm64/include/asm/kvm_mmu.h
> @@ -126,6 +126,7 @@ alternative_cb_end
> #include <asm/cacheflush.h>
> #include <asm/mmu_context.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_nested.h>
>
> void kvm_update_va_mask(struct alt_instr *alt,
> __le32 *origptr, __le32 *updptr, int nr_inst);
> @@ -184,6 +185,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
> void **haddr);
> void free_hyp_pgds(void);
>
> +void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
> void stage2_unmap_vm(struct kvm *kvm);
> int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
> void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
> @@ -306,5 +308,11 @@ static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu)
> asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
> }
>
> +static inline u64 get_vmid(u64 vttbr)
> +{
> + return (vttbr & VTTBR_VMID_MASK(kvm_get_vmid_bits())) >>
> + VTTBR_VMID_SHIFT;
> +}
> +
> #endif /* __ASSEMBLY__ */
> #endif /* __ARM64_KVM_MMU_H__ */
> diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
> index 026ddaad972c..473ecd1d60d0 100644
> --- a/arch/arm64/include/asm/kvm_nested.h
> +++ b/arch/arm64/include/asm/kvm_nested.h
> @@ -61,6 +61,13 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
> (cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
> }
>
> +extern void kvm_init_nested(struct kvm *kvm);
> +extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
> +extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
> +extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
> +extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
> +extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
> +
> int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
> extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
> u64 control_bit);
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 6e637d2b4cfb..1656dd80bbc4 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -35,6 +35,7 @@
> #include <asm/kvm_arm.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmu.h>
> +#include <asm/kvm_nested.h>
> #include <asm/kvm_emulate.h>
> #include <asm/sections.h>
>
> @@ -142,6 +143,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> if (ret)
> return ret;
>
> + kvm_init_nested(kvm);
> +
> ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
> if (ret)
> goto out_free_stage2_pgd;
> @@ -385,6 +388,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> struct kvm_s2_mmu *mmu;
> int *last_ran;
>
> + if (nested_virt_in_use(vcpu))
> + kvm_vcpu_load_hw_mmu(vcpu);
> +
> mmu = vcpu->arch.hw_mmu;
> last_ran = this_cpu_ptr(mmu->last_vcpu_ran);
>
> @@ -426,6 +432,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> kvm_vgic_put(vcpu);
> kvm_vcpu_pmu_restore_host(vcpu);
>
> + if (nested_virt_in_use(vcpu))
> + kvm_vcpu_put_hw_mmu(vcpu);
> +
> vcpu->cpu = -1;
> }
>
> @@ -1026,8 +1035,13 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
>
> vcpu->arch.target = phys_target;
>
> + /* Prepare for nested if required */
> + ret = kvm_vcpu_init_nested(vcpu);
> +
> /* Now we know what it is, we can reset it. */
> - ret = kvm_reset_vcpu(vcpu);
> + if (!ret)
> + ret = kvm_reset_vcpu(vcpu);
> +
> if (ret) {
> vcpu->arch.target = -1;
> bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES);
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 1f41173e6149..2f0302211af3 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -113,7 +113,7 @@ static bool kvm_is_device_pfn(unsigned long pfn)
> * does.
> */
> /**
> - * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
> + * kvm_unmap_stage2_range -- Clear stage2 page table entries to unmap a range
> * @mmu: The KVM stage-2 MMU pointer
> * @start: The intermediate physical base address of the range to unmap
> * @size: The size of the area to unmap
> @@ -136,7 +136,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
> may_block));
> }
>
> -static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
> +void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
> {
> __unmap_stage2_range(mmu, start, size, true);
> }
> @@ -391,6 +391,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
> mmu->pgt = pgt;
> mmu->pgd_phys = __pa(pgt->pgd);
> mmu->vmid.vmid_gen = 0;
> +
> + kvm_init_nested_s2_mmu(mmu);
> +
> return 0;
>
> out_destroy_pgtable:
> @@ -435,7 +438,7 @@ static void stage2_unmap_memslot(struct kvm *kvm,
>
> if (!(vma->vm_flags & VM_PFNMAP)) {
> gpa_t gpa = addr + (vm_start - memslot->userspace_addr);
> - unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
> }
> hva = vm_end;
> } while (hva < reg_end);
> @@ -1360,7 +1363,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
>
> spin_lock(&kvm->mmu_lock);
> if (ret)
> - unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
> else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
> stage2_flush_memslot(kvm, memslot);
> spin_unlock(&kvm->mmu_lock);
> @@ -1377,11 +1380,6 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
> {
> }
>
> -void kvm_arch_flush_shadow_all(struct kvm *kvm)
> -{
> - kvm_free_stage2_pgd(&kvm->arch.mmu);
> -}
> -
> void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> struct kvm_memory_slot *slot)
> {
> @@ -1389,7 +1387,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> phys_addr_t size = slot->npages << PAGE_SHIFT;
>
> spin_lock(&kvm->mmu_lock);
> - unmap_stage2_range(&kvm->arch.mmu, gpa, size);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
> spin_unlock(&kvm->mmu_lock);
> }
>
> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index 9fb44bc7db3f..8e85d2ef24d9 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -19,12 +19,177 @@
> #include <linux/kvm.h>
> #include <linux/kvm_host.h>
>
> +#include <asm/kvm_arm.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_mmu.h>
> #include <asm/kvm_nested.h>
> #include <asm/sysreg.h>
>
> #include "sys_regs.h"
>
> +void kvm_init_nested(struct kvm *kvm)
> +{
> + kvm->arch.nested_mmus = NULL;
> + kvm->arch.nested_mmus_size = 0;
> +}
> +
> +int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct kvm_s2_mmu *tmp;
> + int num_mmus;
> + int ret = -ENOMEM;
> +
> + if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
> + return 0;
> +
> + if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> + return -EINVAL;
nit: returning a "not supported" kind of errno?
> +
> + mutex_lock(&kvm->lock);
> +
> + /*
> + * Let's treat memory allocation failures as benign: If we fail to
> + * allocate anything, return an error and keep the allocated array
> + * alive. Userspace may try to recover by intializing the vcpu
> + * again, and there is no reason to affect the whole VM for this.
> + */
> + num_mmus = atomic_read(&kvm->online_vcpus) * 2;
> + tmp = krealloc(kvm->arch.nested_mmus,
> + num_mmus * sizeof(*kvm->arch.nested_mmus),
> + GFP_KERNEL | __GFP_ZERO);
> + if (tmp) {
> + if (kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 1]) ||
> + kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
> + kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
> + kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
> + } else {
> + kvm->arch.nested_mmus_size = num_mmus;
> + ret = 0;
> + }
> +
> + kvm->arch.nested_mmus = tmp;
> + }
> +
> + mutex_unlock(&kvm->lock);
> + return ret;
> +}
> +
> +/* Must be called with kvm->lock held */
> +struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
> +{
> + bool nested_stage2_enabled = hcr & HCR_VM;
> + int i;
> +
> + /* Don't consider the CnP bit for the vttbr match */
> + vttbr = vttbr & ~VTTBR_CNP_BIT;
> +
> + /*
> + * Two possibilities when looking up a S2 MMU context:
> + *
> + * - either S2 is enabled in the guest, and we need a context that
> + * is S2-enabled and matches the full VTTBR (VMID+BADDR), which
> + * makes it safe from a TLB conflict perspective (a broken guest
> + * won't be able to generate them),
> + *
> + * - or S2 is disabled, and we need a context that is S2-disabled
> + * and matches the VMID only, as all TLBs are tagged by VMID even
> + * if S2 translation is enabled.
> + */
> + for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
> + struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
> +
> + if (!kvm_s2_mmu_valid(mmu))
> + continue;
> +
> + if (nested_stage2_enabled &&
> + mmu->nested_stage2_enabled &&
> + vttbr == mmu->vttbr)
> + return mmu;
> +
> + if (!nested_stage2_enabled &&
> + !mmu->nested_stage2_enabled &&
> + get_vmid(vttbr) == get_vmid(mmu->vttbr))
> + return mmu;
> + }
> + return NULL;
> +}
> +
> +static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
> + u64 hcr= vcpu_read_sys_reg(vcpu, HCR_EL2);
> + struct kvm_s2_mmu *s2_mmu;
> + int i;
> +
> + s2_mmu = lookup_s2_mmu(kvm, vttbr, hcr);
> + if (s2_mmu)
> + goto out;
> +
> + /*
> + * Make sure we don't always search from the same point, or we
> + * will always reuse a potentially active context, leaving
> + * free contexts unused.
> + */
> + for (i = kvm->arch.nested_mmus_next;
> + i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
> + i++) {
> + s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
> +
> + if (atomic_read(&s2_mmu->refcnt) == 0)
> + break;
> + }
> + BUG_ON(atomic_read(&s2_mmu->refcnt)); /* We have struct MMUs to spare */
> +
> + /* Set the scene for the next search */
> + kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
> +
> + if (kvm_s2_mmu_valid(s2_mmu)) {
> + /* Clear the old state */
> + kvm_unmap_stage2_range(s2_mmu, 0, kvm_phys_size(kvm));
> + if (s2_mmu->vmid.vmid_gen)
> + kvm_call_hyp(__kvm_tlb_flush_vmid, s2_mmu);
> + }
> +
> + /*
> + * The virtual VMID (modulo CnP) will be used as a key when matching
> + * an existing kvm_s2_mmu.
> + */
> + s2_mmu->vttbr = vttbr & ~VTTBR_CNP_BIT;
> + s2_mmu->nested_stage2_enabled = hcr & HCR_VM;
> +
> +out:
> + atomic_inc(&s2_mmu->refcnt);
> + return s2_mmu;
> +}
> +
> +void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)
> +{
> + mmu->vttbr = 1;
> + mmu->nested_stage2_enabled = false;
> + atomic_set(&mmu->refcnt, 0);
> +}
> +
> +void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
> +{
> + if (is_hyp_ctxt(vcpu)) {
> + vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
> + } else {
> + spin_lock(&vcpu->kvm->mmu_lock);
> + vcpu->arch.hw_mmu = get_s2_mmu_nested(vcpu);
> + spin_unlock(&vcpu->kvm->mmu_lock);
> + }
> +}
> +
> +void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu) {
> + atomic_dec(&vcpu->arch.hw_mmu->refcnt);
> + vcpu->arch.hw_mmu = NULL;
> + }
> +}
> +
> /*
> * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
> * the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
> @@ -43,6 +208,24 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
> return -EINVAL;
> }
>
> +void kvm_arch_flush_shadow_all(struct kvm *kvm)
> +{
> + int i;
> +
> + for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
> + struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
> +
> + WARN_ON(atomic_read(&mmu->refcnt));
> +
> + if (!atomic_read(&mmu->refcnt))
> + kvm_free_stage2_pgd(mmu);
> + }
> + kfree(kvm->arch.nested_mmus);
> + kvm->arch.nested_mmus = NULL;
> + kvm->arch.nested_mmus_size = 0;
> + kvm_free_stage2_pgd(&kvm->arch.mmu);
> +}
> +
> #define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
>
> /*
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
@ 2021-01-21 2:59 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 2:59 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
>
> Add Stage-2 mmu data structures for virtual EL2 and for nested guests.
> We don't yet populate shadow Stage-2 page tables, but we now have a
> framework for getting to a shadow Stage-2 pgd.
>
> We allocate twice the number of vcpus as Stage-2 mmu structures because
> that's sufficient for each vcpu running two translation regimes without
> having to flush the Stage-2 page tables.
>
> Co-developed-by: Christoffer Dall <christoffer.dall@arm.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 29 +++++
> arch/arm64/include/asm/kvm_mmu.h | 8 ++
> arch/arm64/include/asm/kvm_nested.h | 7 ++
> arch/arm64/kvm/arm.c | 16 ++-
> arch/arm64/kvm/mmu.c | 18 ++-
> arch/arm64/kvm/nested.c | 183 ++++++++++++++++++++++++++++
> 6 files changed, 250 insertions(+), 11 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d731cf7a56cb..d99e51e7cbee 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -95,14 +95,43 @@ struct kvm_s2_mmu {
> int __percpu *last_vcpu_ran;
>
> struct kvm *kvm;
> +
> + /*
> + * For a shadow stage-2 MMU, the virtual vttbr programmed by the guest
> + * hypervisor. Unused for kvm_arch->mmu. Set to 1 when the structure
> + * contains no valid information.
> + */
> + u64 vttbr;
> +
> + /* true when this represents a nested context where virtual HCR_EL2.VM == 1 */
> + bool nested_stage2_enabled;
> +
> + /*
> + * 0: Nobody is currently using this, check vttbr for validity
> + * >0: Somebody is actively using this.
> + */
> + atomic_t refcnt;
> };
>
> +static inline bool kvm_s2_mmu_valid(struct kvm_s2_mmu *mmu)
> +{
> + return !(mmu->vttbr & 1);
> +}
> +
> struct kvm_arch_memory_slot {
> };
>
> struct kvm_arch {
> struct kvm_s2_mmu mmu;
>
> + /*
> + * Stage 2 paging stage for VMs with nested virtual using a virtual
> + * VMID.
> + */
> + struct kvm_s2_mmu *nested_mmus;
> + size_t nested_mmus_size;
> + int nested_mmus_next;
> +
> /* VTCR_EL2 value for this VM */
> u64 vtcr;
>
> diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
> index 76a8a0ca45b8..ec39015bb2a6 100644
> --- a/arch/arm64/include/asm/kvm_mmu.h
> +++ b/arch/arm64/include/asm/kvm_mmu.h
> @@ -126,6 +126,7 @@ alternative_cb_end
> #include <asm/cacheflush.h>
> #include <asm/mmu_context.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_nested.h>
>
> void kvm_update_va_mask(struct alt_instr *alt,
> __le32 *origptr, __le32 *updptr, int nr_inst);
> @@ -184,6 +185,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
> void **haddr);
> void free_hyp_pgds(void);
>
> +void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size);
> void stage2_unmap_vm(struct kvm *kvm);
> int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu);
> void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu);
> @@ -306,5 +308,11 @@ static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu)
> asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
> }
>
> +static inline u64 get_vmid(u64 vttbr)
> +{
> + return (vttbr & VTTBR_VMID_MASK(kvm_get_vmid_bits())) >>
> + VTTBR_VMID_SHIFT;
> +}
> +
> #endif /* __ASSEMBLY__ */
> #endif /* __ARM64_KVM_MMU_H__ */
> diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
> index 026ddaad972c..473ecd1d60d0 100644
> --- a/arch/arm64/include/asm/kvm_nested.h
> +++ b/arch/arm64/include/asm/kvm_nested.h
> @@ -61,6 +61,13 @@ static inline u64 translate_cnthctl_el2_to_cntkctl_el1(u64 cnthctl)
> (cnthctl & (CNTHCTL_EVNTI | CNTHCTL_EVNTDIR | CNTHCTL_EVNTEN)));
> }
>
> +extern void kvm_init_nested(struct kvm *kvm);
> +extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
> +extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
> +extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr);
> +extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);
> +extern void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu);
> +
> int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe);
> extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg,
> u64 control_bit);
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 6e637d2b4cfb..1656dd80bbc4 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -35,6 +35,7 @@
> #include <asm/kvm_arm.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmu.h>
> +#include <asm/kvm_nested.h>
> #include <asm/kvm_emulate.h>
> #include <asm/sections.h>
>
> @@ -142,6 +143,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> if (ret)
> return ret;
>
> + kvm_init_nested(kvm);
> +
> ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
> if (ret)
> goto out_free_stage2_pgd;
> @@ -385,6 +388,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> struct kvm_s2_mmu *mmu;
> int *last_ran;
>
> + if (nested_virt_in_use(vcpu))
> + kvm_vcpu_load_hw_mmu(vcpu);
> +
> mmu = vcpu->arch.hw_mmu;
> last_ran = this_cpu_ptr(mmu->last_vcpu_ran);
>
> @@ -426,6 +432,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> kvm_vgic_put(vcpu);
> kvm_vcpu_pmu_restore_host(vcpu);
>
> + if (nested_virt_in_use(vcpu))
> + kvm_vcpu_put_hw_mmu(vcpu);
> +
> vcpu->cpu = -1;
> }
>
> @@ -1026,8 +1035,13 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
>
> vcpu->arch.target = phys_target;
>
> + /* Prepare for nested if required */
> + ret = kvm_vcpu_init_nested(vcpu);
> +
> /* Now we know what it is, we can reset it. */
> - ret = kvm_reset_vcpu(vcpu);
> + if (!ret)
> + ret = kvm_reset_vcpu(vcpu);
> +
> if (ret) {
> vcpu->arch.target = -1;
> bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES);
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 1f41173e6149..2f0302211af3 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -113,7 +113,7 @@ static bool kvm_is_device_pfn(unsigned long pfn)
> * does.
> */
> /**
> - * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
> + * kvm_unmap_stage2_range -- Clear stage2 page table entries to unmap a range
> * @mmu: The KVM stage-2 MMU pointer
> * @start: The intermediate physical base address of the range to unmap
> * @size: The size of the area to unmap
> @@ -136,7 +136,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
> may_block));
> }
>
> -static void unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
> +void kvm_unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)
> {
> __unmap_stage2_range(mmu, start, size, true);
> }
> @@ -391,6 +391,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
> mmu->pgt = pgt;
> mmu->pgd_phys = __pa(pgt->pgd);
> mmu->vmid.vmid_gen = 0;
> +
> + kvm_init_nested_s2_mmu(mmu);
> +
> return 0;
>
> out_destroy_pgtable:
> @@ -435,7 +438,7 @@ static void stage2_unmap_memslot(struct kvm *kvm,
>
> if (!(vma->vm_flags & VM_PFNMAP)) {
> gpa_t gpa = addr + (vm_start - memslot->userspace_addr);
> - unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, vm_end - vm_start);
> }
> hva = vm_end;
> } while (hva < reg_end);
> @@ -1360,7 +1363,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
>
> spin_lock(&kvm->mmu_lock);
> if (ret)
> - unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size);
> else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
> stage2_flush_memslot(kvm, memslot);
> spin_unlock(&kvm->mmu_lock);
> @@ -1377,11 +1380,6 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
> {
> }
>
> -void kvm_arch_flush_shadow_all(struct kvm *kvm)
> -{
> - kvm_free_stage2_pgd(&kvm->arch.mmu);
> -}
> -
> void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> struct kvm_memory_slot *slot)
> {
> @@ -1389,7 +1387,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> phys_addr_t size = slot->npages << PAGE_SHIFT;
>
> spin_lock(&kvm->mmu_lock);
> - unmap_stage2_range(&kvm->arch.mmu, gpa, size);
> + kvm_unmap_stage2_range(&kvm->arch.mmu, gpa, size);
> spin_unlock(&kvm->mmu_lock);
> }
>
> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index 9fb44bc7db3f..8e85d2ef24d9 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -19,12 +19,177 @@
> #include <linux/kvm.h>
> #include <linux/kvm_host.h>
>
> +#include <asm/kvm_arm.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_mmu.h>
> #include <asm/kvm_nested.h>
> #include <asm/sysreg.h>
>
> #include "sys_regs.h"
>
> +void kvm_init_nested(struct kvm *kvm)
> +{
> + kvm->arch.nested_mmus = NULL;
> + kvm->arch.nested_mmus_size = 0;
> +}
> +
> +int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct kvm_s2_mmu *tmp;
> + int num_mmus;
> + int ret = -ENOMEM;
> +
> + if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
> + return 0;
> +
> + if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> + return -EINVAL;
nit: returning a "not supported" kind of errno?
> +
> + mutex_lock(&kvm->lock);
> +
> + /*
> + * Let's treat memory allocation failures as benign: If we fail to
> + * allocate anything, return an error and keep the allocated array
> + * alive. Userspace may try to recover by intializing the vcpu
> + * again, and there is no reason to affect the whole VM for this.
> + */
> + num_mmus = atomic_read(&kvm->online_vcpus) * 2;
> + tmp = krealloc(kvm->arch.nested_mmus,
> + num_mmus * sizeof(*kvm->arch.nested_mmus),
> + GFP_KERNEL | __GFP_ZERO);
> + if (tmp) {
> + if (kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 1]) ||
> + kvm_init_stage2_mmu(kvm, &tmp[num_mmus - 2])) {
> + kvm_free_stage2_pgd(&tmp[num_mmus - 1]);
> + kvm_free_stage2_pgd(&tmp[num_mmus - 2]);
> + } else {
> + kvm->arch.nested_mmus_size = num_mmus;
> + ret = 0;
> + }
> +
> + kvm->arch.nested_mmus = tmp;
> + }
> +
> + mutex_unlock(&kvm->lock);
> + return ret;
> +}
> +
> +/* Must be called with kvm->lock held */
> +struct kvm_s2_mmu *lookup_s2_mmu(struct kvm *kvm, u64 vttbr, u64 hcr)
> +{
> + bool nested_stage2_enabled = hcr & HCR_VM;
> + int i;
> +
> + /* Don't consider the CnP bit for the vttbr match */
> + vttbr = vttbr & ~VTTBR_CNP_BIT;
> +
> + /*
> + * Two possibilities when looking up a S2 MMU context:
> + *
> + * - either S2 is enabled in the guest, and we need a context that
> + * is S2-enabled and matches the full VTTBR (VMID+BADDR), which
> + * makes it safe from a TLB conflict perspective (a broken guest
> + * won't be able to generate them),
> + *
> + * - or S2 is disabled, and we need a context that is S2-disabled
> + * and matches the VMID only, as all TLBs are tagged by VMID even
> + * if S2 translation is enabled.
> + */
> + for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
> + struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
> +
> + if (!kvm_s2_mmu_valid(mmu))
> + continue;
> +
> + if (nested_stage2_enabled &&
> + mmu->nested_stage2_enabled &&
> + vttbr == mmu->vttbr)
> + return mmu;
> +
> + if (!nested_stage2_enabled &&
> + !mmu->nested_stage2_enabled &&
> + get_vmid(vttbr) == get_vmid(mmu->vttbr))
> + return mmu;
> + }
> + return NULL;
> +}
> +
> +static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + u64 vttbr = vcpu_read_sys_reg(vcpu, VTTBR_EL2);
> + u64 hcr= vcpu_read_sys_reg(vcpu, HCR_EL2);
> + struct kvm_s2_mmu *s2_mmu;
> + int i;
> +
> + s2_mmu = lookup_s2_mmu(kvm, vttbr, hcr);
> + if (s2_mmu)
> + goto out;
> +
> + /*
> + * Make sure we don't always search from the same point, or we
> + * will always reuse a potentially active context, leaving
> + * free contexts unused.
> + */
> + for (i = kvm->arch.nested_mmus_next;
> + i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
> + i++) {
> + s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
> +
> + if (atomic_read(&s2_mmu->refcnt) == 0)
> + break;
> + }
> + BUG_ON(atomic_read(&s2_mmu->refcnt)); /* We have struct MMUs to spare */
> +
> + /* Set the scene for the next search */
> + kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
> +
> + if (kvm_s2_mmu_valid(s2_mmu)) {
> + /* Clear the old state */
> + kvm_unmap_stage2_range(s2_mmu, 0, kvm_phys_size(kvm));
> + if (s2_mmu->vmid.vmid_gen)
> + kvm_call_hyp(__kvm_tlb_flush_vmid, s2_mmu);
> + }
> +
> + /*
> + * The virtual VMID (modulo CnP) will be used as a key when matching
> + * an existing kvm_s2_mmu.
> + */
> + s2_mmu->vttbr = vttbr & ~VTTBR_CNP_BIT;
> + s2_mmu->nested_stage2_enabled = hcr & HCR_VM;
> +
> +out:
> + atomic_inc(&s2_mmu->refcnt);
> + return s2_mmu;
> +}
> +
> +void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)
> +{
> + mmu->vttbr = 1;
> + mmu->nested_stage2_enabled = false;
> + atomic_set(&mmu->refcnt, 0);
> +}
> +
> +void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
> +{
> + if (is_hyp_ctxt(vcpu)) {
> + vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
> + } else {
> + spin_lock(&vcpu->kvm->mmu_lock);
> + vcpu->arch.hw_mmu = get_s2_mmu_nested(vcpu);
> + spin_unlock(&vcpu->kvm->mmu_lock);
> + }
> +}
> +
> +void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.hw_mmu != &vcpu->kvm->arch.mmu) {
> + atomic_dec(&vcpu->arch.hw_mmu->refcnt);
> + vcpu->arch.hw_mmu = NULL;
> + }
> +}
> +
> /*
> * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and
> * the virtual HCR_EL2.TWX is set. Otherwise, let the host hypervisor
> @@ -43,6 +208,24 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe)
> return -EINVAL;
> }
>
> +void kvm_arch_flush_shadow_all(struct kvm *kvm)
> +{
> + int i;
> +
> + for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
> + struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
> +
> + WARN_ON(atomic_read(&mmu->refcnt));
> +
> + if (!atomic_read(&mmu->refcnt))
> + kvm_free_stage2_pgd(mmu);
> + }
> + kfree(kvm->arch.nested_mmus);
> + kvm->arch.nested_mmus = NULL;
> + kvm->arch.nested_mmus_size = 0;
> + kvm_free_stage2_pgd(&kvm->arch.mmu);
> +}
> +
> #define FEATURE(x) (GENMASK_ULL(x##_SHIFT + 3, x##_SHIFT))
>
> /*
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2020-12-10 15:58 ` Marc Zyngier
(?)
@ 2021-01-21 3:03 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 3:03 UTC (permalink / raw)
To: Marc Zyngier; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
Re-send in case the previous email was blocked for the inlined hyper-link.
Hi Marc,
I have tried to enable the NV support in Qemu, and now I can
successfully boot a L2 guest
in Qemu KVM mode.
This patch series looks good from the Qemu side except for two minor
requirements:
(1) Qemu will check whether a feature was supported by the KVM cap
when the user tries
to enable it in the command line, so a new capability was
prefered for the NV(KVM_CAP_ARM_NV?).
(2) According to the Documentation/virt/kvm/api.rst, userspace can
call KVM_ARM_VCPU_INIT
multiple times for a given vcpu, but the kvm_vcpu_init_nested()
do have some issue when
called multiple times(please refer to the detailed comments in patch 63)
Regards,
Haibo
On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>
> This is a rework of the NV series that I posted 10 months ago[1], as a
> lot of the KVM code has changed since, and the series apply anymore
> (not that anybody really cares as the the HW is, as usual, made of
> unobtainium...).
>
> From the previous version:
>
> - Integration with the new page-table code
> - New exception injection code
> - No more messing with the nVHE code
> - No AArch32!!!!
> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>
> From a functionality perspective, you can expect a L2 guest to work,
> but don't even think of L3, as we only partially emulate the
> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> as well as anything that would require a Stage-1 PTW. What we want to
> achieve is that with NV disabled, there is no performance overhead and
> no regression.
>
> The series is roughly divided in 5 parts: exception handling, memory
> virtualization, interrupts and timers for ARMv8.3, followed by the
> ARMv8.4 support. There are of course some dependencies, but you'll
> hopefully get the gist of it.
>
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2. You can run the whole stack in the Foundation
> model. Don't be in a hurry ;-).
>
> And to be clear: although Jintack and Christoffer have written tons of
> the stuff originaly, I'm the one responsible for breaking it!
>
> [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
>
> Andre Przywara (1):
> KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
>
> Christoffer Dall (15):
> KVM: arm64: nv: Introduce nested virtualization VCPU feature
> KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> KVM: arm64: nv: Handle trapped ERET from virtual EL2
> KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> changes
> KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> KVM: arm64: nv: arch_timer: Support hyp timer emulation
> KVM: arm64: nv: vgic: Emulate the HW bit in software
> KVM: arm64: nv: Add nested GICv3 tracepoints
> KVM: arm64: nv: Sync nested timer state with ARMv8.4
>
> Jintack Lim (19):
> arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> KVM: arm64: nv: Support virtual EL2 exceptions
> KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> KVM: arm64: nv: Handle PSCI call via smc from the guest
> KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> KVM: arm64: nv: Set a handler for the system instruction traps
> KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> KVM: arm64: nv: Nested GICv3 Support
>
> Marc Zyngier (31):
> KVM: arm64: nv: Add EL2 system registers to vcpu context
> KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> KVM: arm64: nv: Handle virtual EL2 registers in
> vcpu_read/write_sys_reg()
> KVM: arm64: nv: Handle SPSR_EL2 specially
> KVM: arm64: nv: Handle HCR_EL2.E2H specially
> KVM: arm64: nv: Save/Restore vEL2 sysregs
> KVM: arm64: nv: Forward debug traps to the nested guest
> KVM: arm64: nv: Filter out unsupported features from ID regs
> KVM: arm64: nv: Hide RAS from nested guests
> KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> KVM: arm64: nv: Handle shadow stage 2 page faults
> KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> KVM: arm64: nv: Add handling of EL2-specific timer registers
> KVM: arm64: nv: Load timer before the GIC
> KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> guest
> KVM: arm64: nv: Implement maintenance interrupt forwarding
> KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> information
> KVM: arm64: Allow populating S2 SW bits
> KVM: arm64: nv: Tag shadow S2 entries with nested level
> KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> KVM: arm64: Map VNCR-capable registers to a separate page
> KVM: arm64: nv: Move nested vgic state into the sysreg file
> KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> KVM: arm64: nv: Synchronize PSTATE early on exit
> KVM: arm64: nv: Allocate VNCR page when required
> KVM: arm64: nv: Enable ARMv8.4-NV support
> KVM: arm64: nv: Fast-track 'InHost' exception returns
> KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
>
> .../admin-guide/kernel-parameters.txt | 4 +
> .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> arch/arm64/include/asm/cpucaps.h | 2 +
> arch/arm64/include/asm/esr.h | 6 +
> arch/arm64/include/asm/kvm_arm.h | 28 +-
> arch/arm64/include/asm/kvm_asm.h | 4 +
> arch/arm64/include/asm/kvm_emulate.h | 145 +-
> arch/arm64/include/asm/kvm_host.h | 175 ++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_mmu.h | 17 +-
> arch/arm64/include/asm/kvm_nested.h | 152 ++
> arch/arm64/include/asm/kvm_pgtable.h | 10 +
> arch/arm64/include/asm/sysreg.h | 104 +-
> arch/arm64/include/asm/vncr_mapping.h | 73 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kernel/cpufeature.c | 35 +
> arch/arm64/kvm/Makefile | 4 +-
> arch/arm64/kvm/arch_timer.c | 189 ++-
> arch/arm64/kvm/arm.c | 34 +-
> arch/arm64/kvm/at.c | 231 ++++
> arch/arm64/kvm/emulate-nested.c | 186 +++
> arch/arm64/kvm/guest.c | 6 +
> arch/arm64/kvm/handle_exit.c | 81 +-
> arch/arm64/kvm/hyp/exception.c | 44 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> arch/arm64/kvm/hyp/pgtable.c | 6 +
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> arch/arm64/kvm/inject_fault.c | 62 +-
> arch/arm64/kvm/mmu.c | 183 ++-
> arch/arm64/kvm/nested.c | 908 ++++++++++++
> arch/arm64/kvm/reset.c | 14 +-
> arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> arch/arm64/kvm/sys_regs.h | 6 +
> arch/arm64/kvm/trace_arm.h | 65 +-
> arch/arm64/kvm/vgic/vgic-init.c | 30 +
> arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> arch/arm64/kvm/vgic/vgic.c | 44 +
> arch/arm64/kvm/vgic/vgic.h | 10 +
> include/kvm/arm_arch_timer.h | 7 +
> include/kvm/arm_vgic.h | 16 +
> tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> 50 files changed, 4890 insertions(+), 160 deletions(-)
> create mode 100644 arch/arm64/include/asm/kvm_nested.h
> create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> create mode 100644 arch/arm64/kvm/at.c
> create mode 100644 arch/arm64/kvm/emulate-nested.c
> create mode 100644 arch/arm64/kvm/nested.c
> create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
>
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-21 3:03 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 3:03 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
Re-send in case the previous email was blocked for the inlined hyper-link.
Hi Marc,
I have tried to enable the NV support in Qemu, and now I can
successfully boot a L2 guest
in Qemu KVM mode.
This patch series looks good from the Qemu side except for two minor
requirements:
(1) Qemu will check whether a feature was supported by the KVM cap
when the user tries
to enable it in the command line, so a new capability was
prefered for the NV(KVM_CAP_ARM_NV?).
(2) According to the Documentation/virt/kvm/api.rst, userspace can
call KVM_ARM_VCPU_INIT
multiple times for a given vcpu, but the kvm_vcpu_init_nested()
do have some issue when
called multiple times(please refer to the detailed comments in patch 63)
Regards,
Haibo
On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>
> This is a rework of the NV series that I posted 10 months ago[1], as a
> lot of the KVM code has changed since, and the series apply anymore
> (not that anybody really cares as the the HW is, as usual, made of
> unobtainium...).
>
> From the previous version:
>
> - Integration with the new page-table code
> - New exception injection code
> - No more messing with the nVHE code
> - No AArch32!!!!
> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>
> From a functionality perspective, you can expect a L2 guest to work,
> but don't even think of L3, as we only partially emulate the
> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> as well as anything that would require a Stage-1 PTW. What we want to
> achieve is that with NV disabled, there is no performance overhead and
> no regression.
>
> The series is roughly divided in 5 parts: exception handling, memory
> virtualization, interrupts and timers for ARMv8.3, followed by the
> ARMv8.4 support. There are of course some dependencies, but you'll
> hopefully get the gist of it.
>
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2. You can run the whole stack in the Foundation
> model. Don't be in a hurry ;-).
>
> And to be clear: although Jintack and Christoffer have written tons of
> the stuff originaly, I'm the one responsible for breaking it!
>
> [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
>
> Andre Przywara (1):
> KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
>
> Christoffer Dall (15):
> KVM: arm64: nv: Introduce nested virtualization VCPU feature
> KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> KVM: arm64: nv: Handle trapped ERET from virtual EL2
> KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> changes
> KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> KVM: arm64: nv: arch_timer: Support hyp timer emulation
> KVM: arm64: nv: vgic: Emulate the HW bit in software
> KVM: arm64: nv: Add nested GICv3 tracepoints
> KVM: arm64: nv: Sync nested timer state with ARMv8.4
>
> Jintack Lim (19):
> arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> KVM: arm64: nv: Support virtual EL2 exceptions
> KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> KVM: arm64: nv: Handle PSCI call via smc from the guest
> KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> KVM: arm64: nv: Set a handler for the system instruction traps
> KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> KVM: arm64: nv: Nested GICv3 Support
>
> Marc Zyngier (31):
> KVM: arm64: nv: Add EL2 system registers to vcpu context
> KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> KVM: arm64: nv: Handle virtual EL2 registers in
> vcpu_read/write_sys_reg()
> KVM: arm64: nv: Handle SPSR_EL2 specially
> KVM: arm64: nv: Handle HCR_EL2.E2H specially
> KVM: arm64: nv: Save/Restore vEL2 sysregs
> KVM: arm64: nv: Forward debug traps to the nested guest
> KVM: arm64: nv: Filter out unsupported features from ID regs
> KVM: arm64: nv: Hide RAS from nested guests
> KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> KVM: arm64: nv: Handle shadow stage 2 page faults
> KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> KVM: arm64: nv: Add handling of EL2-specific timer registers
> KVM: arm64: nv: Load timer before the GIC
> KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> guest
> KVM: arm64: nv: Implement maintenance interrupt forwarding
> KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> information
> KVM: arm64: Allow populating S2 SW bits
> KVM: arm64: nv: Tag shadow S2 entries with nested level
> KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> KVM: arm64: Map VNCR-capable registers to a separate page
> KVM: arm64: nv: Move nested vgic state into the sysreg file
> KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> KVM: arm64: nv: Synchronize PSTATE early on exit
> KVM: arm64: nv: Allocate VNCR page when required
> KVM: arm64: nv: Enable ARMv8.4-NV support
> KVM: arm64: nv: Fast-track 'InHost' exception returns
> KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
>
> .../admin-guide/kernel-parameters.txt | 4 +
> .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> arch/arm64/include/asm/cpucaps.h | 2 +
> arch/arm64/include/asm/esr.h | 6 +
> arch/arm64/include/asm/kvm_arm.h | 28 +-
> arch/arm64/include/asm/kvm_asm.h | 4 +
> arch/arm64/include/asm/kvm_emulate.h | 145 +-
> arch/arm64/include/asm/kvm_host.h | 175 ++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_mmu.h | 17 +-
> arch/arm64/include/asm/kvm_nested.h | 152 ++
> arch/arm64/include/asm/kvm_pgtable.h | 10 +
> arch/arm64/include/asm/sysreg.h | 104 +-
> arch/arm64/include/asm/vncr_mapping.h | 73 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kernel/cpufeature.c | 35 +
> arch/arm64/kvm/Makefile | 4 +-
> arch/arm64/kvm/arch_timer.c | 189 ++-
> arch/arm64/kvm/arm.c | 34 +-
> arch/arm64/kvm/at.c | 231 ++++
> arch/arm64/kvm/emulate-nested.c | 186 +++
> arch/arm64/kvm/guest.c | 6 +
> arch/arm64/kvm/handle_exit.c | 81 +-
> arch/arm64/kvm/hyp/exception.c | 44 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> arch/arm64/kvm/hyp/pgtable.c | 6 +
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> arch/arm64/kvm/inject_fault.c | 62 +-
> arch/arm64/kvm/mmu.c | 183 ++-
> arch/arm64/kvm/nested.c | 908 ++++++++++++
> arch/arm64/kvm/reset.c | 14 +-
> arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> arch/arm64/kvm/sys_regs.h | 6 +
> arch/arm64/kvm/trace_arm.h | 65 +-
> arch/arm64/kvm/vgic/vgic-init.c | 30 +
> arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> arch/arm64/kvm/vgic/vgic.c | 44 +
> arch/arm64/kvm/vgic/vgic.h | 10 +
> include/kvm/arm_arch_timer.h | 7 +
> include/kvm/arm_vgic.h | 16 +
> tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> 50 files changed, 4890 insertions(+), 160 deletions(-)
> create mode 100644 arch/arm64/include/asm/kvm_nested.h
> create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> create mode 100644 arch/arm64/kvm/at.c
> create mode 100644 arch/arm64/kvm/emulate-nested.c
> create mode 100644 arch/arm64/kvm/nested.c
> create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
>
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-01-21 3:03 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-01-21 3:03 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
Re-send in case the previous email was blocked for the inlined hyper-link.
Hi Marc,
I have tried to enable the NV support in Qemu, and now I can
successfully boot a L2 guest
in Qemu KVM mode.
This patch series looks good from the Qemu side except for two minor
requirements:
(1) Qemu will check whether a feature was supported by the KVM cap
when the user tries
to enable it in the command line, so a new capability was
prefered for the NV(KVM_CAP_ARM_NV?).
(2) According to the Documentation/virt/kvm/api.rst, userspace can
call KVM_ARM_VCPU_INIT
multiple times for a given vcpu, but the kvm_vcpu_init_nested()
do have some issue when
called multiple times(please refer to the detailed comments in patch 63)
Regards,
Haibo
On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
>
> This is a rework of the NV series that I posted 10 months ago[1], as a
> lot of the KVM code has changed since, and the series apply anymore
> (not that anybody really cares as the the HW is, as usual, made of
> unobtainium...).
>
> From the previous version:
>
> - Integration with the new page-table code
> - New exception injection code
> - No more messing with the nVHE code
> - No AArch32!!!!
> - Rebased on v5.10-rc4 + kvmarm/next for 5.11
>
> From a functionality perspective, you can expect a L2 guest to work,
> but don't even think of L3, as we only partially emulate the
> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> as well as anything that would require a Stage-1 PTW. What we want to
> achieve is that with NV disabled, there is no performance overhead and
> no regression.
>
> The series is roughly divided in 5 parts: exception handling, memory
> virtualization, interrupts and timers for ARMv8.3, followed by the
> ARMv8.4 support. There are of course some dependencies, but you'll
> hopefully get the gist of it.
>
> For the most courageous of you, I've put out a branch[2]. Of course,
> you'll need some userspace. Andre maintains a hacked version of
> kvmtool[3] that takes a --nested option, allowing the guest to be
> started at EL2. You can run the whole stack in the Foundation
> model. Don't be in a hurry ;-).
>
> And to be clear: although Jintack and Christoffer have written tons of
> the stuff originaly, I'm the one responsible for breaking it!
>
> [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
>
> Andre Przywara (1):
> KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
>
> Christoffer Dall (15):
> KVM: arm64: nv: Introduce nested virtualization VCPU feature
> KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> KVM: arm64: nv: Handle trapped ERET from virtual EL2
> KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> changes
> KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> KVM: arm64: nv: arch_timer: Support hyp timer emulation
> KVM: arm64: nv: vgic: Emulate the HW bit in software
> KVM: arm64: nv: Add nested GICv3 tracepoints
> KVM: arm64: nv: Sync nested timer state with ARMv8.4
>
> Jintack Lim (19):
> arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> KVM: arm64: nv: Support virtual EL2 exceptions
> KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> KVM: arm64: nv: Handle PSCI call via smc from the guest
> KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> KVM: arm64: nv: Set a handler for the system instruction traps
> KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> KVM: arm64: nv: Nested GICv3 Support
>
> Marc Zyngier (31):
> KVM: arm64: nv: Add EL2 system registers to vcpu context
> KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> KVM: arm64: nv: Handle virtual EL2 registers in
> vcpu_read/write_sys_reg()
> KVM: arm64: nv: Handle SPSR_EL2 specially
> KVM: arm64: nv: Handle HCR_EL2.E2H specially
> KVM: arm64: nv: Save/Restore vEL2 sysregs
> KVM: arm64: nv: Forward debug traps to the nested guest
> KVM: arm64: nv: Filter out unsupported features from ID regs
> KVM: arm64: nv: Hide RAS from nested guests
> KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> KVM: arm64: nv: Handle shadow stage 2 page faults
> KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> KVM: arm64: nv: Add handling of EL2-specific timer registers
> KVM: arm64: nv: Load timer before the GIC
> KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> guest
> KVM: arm64: nv: Implement maintenance interrupt forwarding
> KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> information
> KVM: arm64: Allow populating S2 SW bits
> KVM: arm64: nv: Tag shadow S2 entries with nested level
> KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> KVM: arm64: Map VNCR-capable registers to a separate page
> KVM: arm64: nv: Move nested vgic state into the sysreg file
> KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> KVM: arm64: nv: Synchronize PSTATE early on exit
> KVM: arm64: nv: Allocate VNCR page when required
> KVM: arm64: nv: Enable ARMv8.4-NV support
> KVM: arm64: nv: Fast-track 'InHost' exception returns
> KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
>
> .../admin-guide/kernel-parameters.txt | 4 +
> .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> arch/arm64/include/asm/cpucaps.h | 2 +
> arch/arm64/include/asm/esr.h | 6 +
> arch/arm64/include/asm/kvm_arm.h | 28 +-
> arch/arm64/include/asm/kvm_asm.h | 4 +
> arch/arm64/include/asm/kvm_emulate.h | 145 +-
> arch/arm64/include/asm/kvm_host.h | 175 ++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_mmu.h | 17 +-
> arch/arm64/include/asm/kvm_nested.h | 152 ++
> arch/arm64/include/asm/kvm_pgtable.h | 10 +
> arch/arm64/include/asm/sysreg.h | 104 +-
> arch/arm64/include/asm/vncr_mapping.h | 73 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kernel/cpufeature.c | 35 +
> arch/arm64/kvm/Makefile | 4 +-
> arch/arm64/kvm/arch_timer.c | 189 ++-
> arch/arm64/kvm/arm.c | 34 +-
> arch/arm64/kvm/at.c | 231 ++++
> arch/arm64/kvm/emulate-nested.c | 186 +++
> arch/arm64/kvm/guest.c | 6 +
> arch/arm64/kvm/handle_exit.c | 81 +-
> arch/arm64/kvm/hyp/exception.c | 44 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> arch/arm64/kvm/hyp/pgtable.c | 6 +
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> arch/arm64/kvm/inject_fault.c | 62 +-
> arch/arm64/kvm/mmu.c | 183 ++-
> arch/arm64/kvm/nested.c | 908 ++++++++++++
> arch/arm64/kvm/reset.c | 14 +-
> arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> arch/arm64/kvm/sys_regs.h | 6 +
> arch/arm64/kvm/trace_arm.h | 65 +-
> arch/arm64/kvm/vgic/vgic-init.c | 30 +
> arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> arch/arm64/kvm/vgic/vgic.c | 44 +
> arch/arm64/kvm/vgic/vgic.h | 10 +
> include/kvm/arm_arch_timer.h | 7 +
> include/kvm/arm_vgic.h | 16 +
> tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> 50 files changed, 4890 insertions(+), 160 deletions(-)
> create mode 100644 arch/arm64/include/asm/kvm_nested.h
> create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> create mode 100644 arch/arm64/kvm/at.c
> create mode 100644 arch/arm64/kvm/emulate-nested.c
> create mode 100644 arch/arm64/kvm/nested.c
> create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
>
> --
> 2.29.2
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context
2020-12-10 15:59 ` Marc Zyngier
(?)
@ 2021-01-29 18:04 ` Andre Przywara
-1 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:04 UTC (permalink / raw)
To: Marc Zyngier
Cc: linux-arm-kernel, kvmarm, kvm, Christoffer Dall, Jintack Lim,
Alexandru Elisei, James Morse, Julien Thierry, Suzuki K Poulose,
kernel-team
On Thu, 10 Dec 2020 15:59:01 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> Add the minimal set of EL2 system registers to the vcpu context.
> Nothing uses them just yet.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Checked against the ARM ARM that this list contains the _EL2 registers
available in ARMv8.1, minus timer and GIC registers.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Cheers,
Andre
> ---
> arch/arm64/include/asm/kvm_host.h | 34 ++++++++++++++++++++++++++++++-
> 1 file changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 11beda85ee7e..d731cf7a56cb 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -206,12 +206,44 @@ enum vcpu_sysreg {
> CNTP_CVAL_EL0,
> CNTP_CTL_EL0,
>
> - /* 32bit specific registers. Keep them at the end of the range */
> + /* 32bit specific registers. */
> DACR32_EL2, /* Domain Access Control Register */
> IFSR32_EL2, /* Instruction Fault Status Register */
> FPEXC32_EL2, /* Floating-Point Exception Control Register */
> DBGVCR32_EL2, /* Debug Vector Catch Register */
>
> + /* EL2 registers */
> + VPIDR_EL2, /* Virtualization Processor ID Register */
> + VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
> + SCTLR_EL2, /* System Control Register (EL2) */
> + ACTLR_EL2, /* Auxiliary Control Register (EL2) */
> + HCR_EL2, /* Hypervisor Configuration Register */
> + MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
> + CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
> + HSTR_EL2, /* Hypervisor System Trap Register */
> + HACR_EL2, /* Hypervisor Auxiliary Control Register */
> + TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
> + TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
> + TCR_EL2, /* Translation Control Register (EL2) */
> + VTTBR_EL2, /* Virtualization Translation Table Base Register */
> + VTCR_EL2, /* Virtualization Translation Control Register */
> + SPSR_EL2, /* EL2 saved program status register */
> + ELR_EL2, /* EL2 exception link register */
> + AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
> + AFSR1_EL2, /* Auxiliary Fault Status Register 1 (EL2) */
> + ESR_EL2, /* Exception Syndrome Register (EL2) */
> + FAR_EL2, /* Hypervisor IPA Fault Address Register */
> + HPFAR_EL2, /* Hypervisor IPA Fault Address Register */
> + MAIR_EL2, /* Memory Attribute Indirection Register (EL2) */
> + AMAIR_EL2, /* Auxiliary Memory Attribute Indirection Register (EL2) */
> + VBAR_EL2, /* Vector Base Address Register (EL2) */
> + RVBAR_EL2, /* Reset Vector Base Address Register */
> + RMR_EL2, /* Reset Management Register */
> + CONTEXTIDR_EL2, /* Context ID Register (EL2) */
> + TPIDR_EL2, /* EL2 Software Thread ID Register */
> + CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
> + SP_EL2, /* EL2 Stack Pointer */
> +
> NR_SYS_REGS /* Nothing after this line! */
> };
>
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context
@ 2021-01-29 18:04 ` Andre Przywara
0 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:04 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvm, kernel-team, kvmarm, linux-arm-kernel
On Thu, 10 Dec 2020 15:59:01 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> Add the minimal set of EL2 system registers to the vcpu context.
> Nothing uses them just yet.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Checked against the ARM ARM that this list contains the _EL2 registers
available in ARMv8.1, minus timer and GIC registers.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Cheers,
Andre
> ---
> arch/arm64/include/asm/kvm_host.h | 34 ++++++++++++++++++++++++++++++-
> 1 file changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 11beda85ee7e..d731cf7a56cb 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -206,12 +206,44 @@ enum vcpu_sysreg {
> CNTP_CVAL_EL0,
> CNTP_CTL_EL0,
>
> - /* 32bit specific registers. Keep them at the end of the range */
> + /* 32bit specific registers. */
> DACR32_EL2, /* Domain Access Control Register */
> IFSR32_EL2, /* Instruction Fault Status Register */
> FPEXC32_EL2, /* Floating-Point Exception Control Register */
> DBGVCR32_EL2, /* Debug Vector Catch Register */
>
> + /* EL2 registers */
> + VPIDR_EL2, /* Virtualization Processor ID Register */
> + VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
> + SCTLR_EL2, /* System Control Register (EL2) */
> + ACTLR_EL2, /* Auxiliary Control Register (EL2) */
> + HCR_EL2, /* Hypervisor Configuration Register */
> + MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
> + CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
> + HSTR_EL2, /* Hypervisor System Trap Register */
> + HACR_EL2, /* Hypervisor Auxiliary Control Register */
> + TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
> + TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
> + TCR_EL2, /* Translation Control Register (EL2) */
> + VTTBR_EL2, /* Virtualization Translation Table Base Register */
> + VTCR_EL2, /* Virtualization Translation Control Register */
> + SPSR_EL2, /* EL2 saved program status register */
> + ELR_EL2, /* EL2 exception link register */
> + AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
> + AFSR1_EL2, /* Auxiliary Fault Status Register 1 (EL2) */
> + ESR_EL2, /* Exception Syndrome Register (EL2) */
> + FAR_EL2, /* Hypervisor IPA Fault Address Register */
> + HPFAR_EL2, /* Hypervisor IPA Fault Address Register */
> + MAIR_EL2, /* Memory Attribute Indirection Register (EL2) */
> + AMAIR_EL2, /* Auxiliary Memory Attribute Indirection Register (EL2) */
> + VBAR_EL2, /* Vector Base Address Register (EL2) */
> + RVBAR_EL2, /* Reset Vector Base Address Register */
> + RMR_EL2, /* Reset Management Register */
> + CONTEXTIDR_EL2, /* Context ID Register (EL2) */
> + TPIDR_EL2, /* EL2 Software Thread ID Register */
> + CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
> + SP_EL2, /* EL2 Stack Pointer */
> +
> NR_SYS_REGS /* Nothing after this line! */
> };
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context
@ 2021-01-29 18:04 ` Andre Przywara
0 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:04 UTC (permalink / raw)
To: Marc Zyngier
Cc: kvm, Suzuki K Poulose, kernel-team, Christoffer Dall, kvmarm,
James Morse, Julien Thierry, Alexandru Elisei, Jintack Lim,
linux-arm-kernel
On Thu, 10 Dec 2020 15:59:01 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> Add the minimal set of EL2 system registers to the vcpu context.
> Nothing uses them just yet.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Checked against the ARM ARM that this list contains the _EL2 registers
available in ARMv8.1, minus timer and GIC registers.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Cheers,
Andre
> ---
> arch/arm64/include/asm/kvm_host.h | 34 ++++++++++++++++++++++++++++++-
> 1 file changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 11beda85ee7e..d731cf7a56cb 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -206,12 +206,44 @@ enum vcpu_sysreg {
> CNTP_CVAL_EL0,
> CNTP_CTL_EL0,
>
> - /* 32bit specific registers. Keep them at the end of the range */
> + /* 32bit specific registers. */
> DACR32_EL2, /* Domain Access Control Register */
> IFSR32_EL2, /* Instruction Fault Status Register */
> FPEXC32_EL2, /* Floating-Point Exception Control Register */
> DBGVCR32_EL2, /* Debug Vector Catch Register */
>
> + /* EL2 registers */
> + VPIDR_EL2, /* Virtualization Processor ID Register */
> + VMPIDR_EL2, /* Virtualization Multiprocessor ID Register */
> + SCTLR_EL2, /* System Control Register (EL2) */
> + ACTLR_EL2, /* Auxiliary Control Register (EL2) */
> + HCR_EL2, /* Hypervisor Configuration Register */
> + MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
> + CPTR_EL2, /* Architectural Feature Trap Register (EL2) */
> + HSTR_EL2, /* Hypervisor System Trap Register */
> + HACR_EL2, /* Hypervisor Auxiliary Control Register */
> + TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */
> + TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */
> + TCR_EL2, /* Translation Control Register (EL2) */
> + VTTBR_EL2, /* Virtualization Translation Table Base Register */
> + VTCR_EL2, /* Virtualization Translation Control Register */
> + SPSR_EL2, /* EL2 saved program status register */
> + ELR_EL2, /* EL2 exception link register */
> + AFSR0_EL2, /* Auxiliary Fault Status Register 0 (EL2) */
> + AFSR1_EL2, /* Auxiliary Fault Status Register 1 (EL2) */
> + ESR_EL2, /* Exception Syndrome Register (EL2) */
> + FAR_EL2, /* Hypervisor IPA Fault Address Register */
> + HPFAR_EL2, /* Hypervisor IPA Fault Address Register */
> + MAIR_EL2, /* Memory Attribute Indirection Register (EL2) */
> + AMAIR_EL2, /* Auxiliary Memory Attribute Indirection Register (EL2) */
> + VBAR_EL2, /* Vector Base Address Register (EL2) */
> + RVBAR_EL2, /* Reset Vector Base Address Register */
> + RMR_EL2, /* Reset Management Register */
> + CONTEXTIDR_EL2, /* Context ID Register (EL2) */
> + TPIDR_EL2, /* EL2 Software Thread ID Register */
> + CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
> + SP_EL2, /* EL2 Stack Pointer */
> +
> NR_SYS_REGS /* Nothing after this line! */
> };
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps
2020-12-10 15:59 ` Marc Zyngier
(?)
@ 2021-01-29 18:05 ` Andre Przywara
-1 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:05 UTC (permalink / raw)
To: Marc Zyngier
Cc: linux-arm-kernel, kvmarm, kvm, Christoffer Dall, Jintack Lim,
Alexandru Elisei, James Morse, Julien Thierry, Suzuki K Poulose,
kernel-team, Jintack Lim
On Thu, 10 Dec 2020 15:59:03 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> From: Jintack Lim <jintack.lim@linaro.org>
>
> ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When
> this bit is set, accessing EL2 registers in EL1 traps to EL2. In
> addition, executing the following instructions in EL1 will trap to EL2:
> tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the
> instructions that trap to EL2 with the NV bit were undef at EL1 prior to
> ARM v8.3. The only instruction that was not undef is eret.
>
> This patch sets up a handler for EL2 registers and SP_EL1 register
> accesses at EL1. The host hypervisor keeps those register values in
> memory, and will emulate their behavior.
>
> This patch doesn't set the NV bit yet. It will be set in a later patch
> once nested virtualization support is completed.
>
> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
> [maz: added SCTLR_EL2 RES0/RES1 handling]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
I compared the system register encodings in the first hunk against the
ARMv8 ARM, they are all fine.
I also checked the last hunk for (copy&paste) typos, all good as well.
Some comments below:
> ---
> arch/arm64/include/asm/sysreg.h | 44 ++++++++++++-
> arch/arm64/kvm/sys_regs.c | 112 ++++++++++++++++++++++++++++++--
> 2 files changed, 150 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 82521cdbfc1c..05b49eafbb49 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -469,19 +469,50 @@
>
> #define SYS_PMCCFILTR_EL0 sys_reg(3, 3, 14, 15, 7)
>
> +#define SYS_VPIDR_EL2 sys_reg(3, 4, 0, 0, 0)
> +#define SYS_VMPIDR_EL2 sys_reg(3, 4, 0, 0, 5)
> +
> #define SYS_SCTLR_EL2 sys_reg(3, 4, 1, 0, 0)
> +#define SYS_ACTLR_EL2 sys_reg(3, 4, 1, 0, 1)
> +#define SYS_HCR_EL2 sys_reg(3, 4, 1, 1, 0)
> +#define SYS_MDCR_EL2 sys_reg(3, 4, 1, 1, 1)
> +#define SYS_CPTR_EL2 sys_reg(3, 4, 1, 1, 2)
> +#define SYS_HSTR_EL2 sys_reg(3, 4, 1, 1, 3)
> +#define SYS_HACR_EL2 sys_reg(3, 4, 1, 1, 7)
> +
> #define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
> +
> +#define SYS_TTBR0_EL2 sys_reg(3, 4, 2, 0, 0)
> +#define SYS_TTBR1_EL2 sys_reg(3, 4, 2, 0, 1)
> +#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
> +#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
> +#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
> +
> #define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
> +
> #define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0)
> #define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1)
> +#define SYS_SP_EL1 sys_reg(3, 4, 4, 1, 0)
> +
> #define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
> +#define SYS_AFSR0_EL2 sys_reg(3, 4, 5, 1, 0)
> +#define SYS_AFSR1_EL2 sys_reg(3, 4, 5, 1, 1)
> #define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0)
> #define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3)
> #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
> #define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0)
> #define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
>
> -#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
> +#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
> +#define SYS_HPFAR_EL2 sys_reg(3, 4, 6, 0, 4)
> +
> +#define SYS_MAIR_EL2 sys_reg(3, 4, 10, 2, 0)
> +#define SYS_AMAIR_EL2 sys_reg(3, 4, 10, 3, 0)
> +
> +#define SYS_VBAR_EL2 sys_reg(3, 4, 12, 0, 0)
> +#define SYS_RVBAR_EL2 sys_reg(3, 4, 12, 0, 1)
> +#define SYS_RMR_EL2 sys_reg(3, 4, 12, 0, 2)
> +#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
> #define __SYS__AP0Rx_EL2(x) sys_reg(3, 4, 12, 8, x)
> #define SYS_ICH_AP0R0_EL2 __SYS__AP0Rx_EL2(0)
> #define SYS_ICH_AP0R1_EL2 __SYS__AP0Rx_EL2(1)
> @@ -523,15 +554,24 @@
> #define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
> #define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
>
> +#define SYS_CONTEXTIDR_EL2 sys_reg(3, 4, 13, 0, 1)
> +#define SYS_TPIDR_EL2 sys_reg(3, 4, 13, 0, 2)
> +
> +#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
> +#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
> +
> /* VHE encodings for architectural EL0/1 system registers */
> #define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
> #define SYS_CPACR_EL12 sys_reg(3, 5, 1, 0, 2)
> #define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
> +
> #define SYS_TTBR0_EL12 sys_reg(3, 5, 2, 0, 0)
> #define SYS_TTBR1_EL12 sys_reg(3, 5, 2, 0, 1)
> #define SYS_TCR_EL12 sys_reg(3, 5, 2, 0, 2)
> +
> #define SYS_SPSR_EL12 sys_reg(3, 5, 4, 0, 0)
> #define SYS_ELR_EL12 sys_reg(3, 5, 4, 0, 1)
> +
> #define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0)
> #define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1)
> #define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0)
> @@ -549,6 +589,8 @@
> #define SYS_CNTV_CTL_EL02 sys_reg(3, 5, 14, 3, 1)
> #define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
>
> +#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
> +
> /* Common SCTLR_ELx flags. */
> #define SCTLR_ELx_DSSBS (BIT(44))
> #define SCTLR_ELx_ATA (BIT(43))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 3313dedfa505..c049867a39bc 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -104,6 +104,57 @@ static u32 get_ccsidr(u32 csselr)
> return ccsidr;
> }
>
> +static bool access_rw(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + vcpu_write_sys_reg(vcpu, p->regval, r->reg);
> + else
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> +
> + return true;
> +}
> +
> +static bool access_wi(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + return ignore_write(vcpu, p);
> +
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> + return true;
> +}
> +
> +static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write) {
> + u64 val = p->regval;
> +
> + if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) {
> + val &= ~(GENMASK_ULL(63,45) | GENMASK_ULL(34, 32) |
In ARMv8 ARM F.c bits 49-45 are used for the TWE delay value. The
manual says it's RES0 when FEAT_TWED is not implemented, but this is
true for a lot of other feature bits we don't implement? So shall we
let a guest set them, reducing the mask to (63,50)?
> + BIT_ULL(17));
The ARMv8 ARM F.c lists bit 9 as RES0 as well for (E2H,TGE)=(1,1).
> + val |= SCTLR_EL1_RES1;
> + } else {
> + val &= ~(GENMASK_ULL(63,45) | BIT_ULL(42) |
> + GENMASK_ULL(39, 38) | GENMASK_ULL(35, 32) |
> + BIT_ULL(26) | BIT_ULL(24) | BIT_ULL(20) |
> + BIT_ULL(17) | GENMASK_ULL(15, 14) |
> + GENMASK(10, 7));
> + val |= SCTLR_EL2_RES1;
> + }
> +
> + vcpu_write_sys_reg(vcpu, val, r->reg);
> + } else {
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> + }
> +
> + return true;
> +}
> +
> /*
> * See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
> */
> @@ -342,12 +393,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
> struct sys_reg_params *p,
> const struct sys_reg_desc *r)
> {
> - if (p->is_write) {
> - vcpu_write_sys_reg(vcpu, p->regval, r->reg);
> + access_rw(vcpu, p, r);
> + if (p->is_write)
> vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
> - } else {
> - p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> - }
>
> trace_trap_reg(__func__, r->reg, p->is_write, p->regval);
>
> @@ -1314,6 +1362,18 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> .set_user = set_raz_id_reg, \
> }
>
> +static bool access_sp_el1(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + __vcpu_sys_reg(vcpu, SP_EL1) = p->regval;
> + else
> + p->regval = __vcpu_sys_reg(vcpu, SP_EL1);
> +
> + return true;
> +}
> +
> /*
> * Architected system registers.
> * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
> @@ -1692,9 +1752,51 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> */
> { SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
>
> + { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
> + { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
> +
> + { SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
> + { SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
> + { SYS_DESC(SYS_HCR_EL2), access_rw, reset_val, HCR_EL2, 0 },
> + { SYS_DESC(SYS_MDCR_EL2), access_rw, reset_val, MDCR_EL2, 0 },
> + { SYS_DESC(SYS_CPTR_EL2), access_rw, reset_val, CPTR_EL2, CPTR_EL2_RES1 },
> + { SYS_DESC(SYS_HSTR_EL2), access_rw, reset_val, HSTR_EL2, 0 },
> + { SYS_DESC(SYS_HACR_EL2), access_rw, reset_val, HACR_EL2, 0 },
> +
> + { SYS_DESC(SYS_TTBR0_EL2), access_rw, reset_val, TTBR0_EL2, 0 },
> + { SYS_DESC(SYS_TTBR1_EL2), access_rw, reset_val, TTBR1_EL2, 0 },
> + { SYS_DESC(SYS_TCR_EL2), access_rw, reset_val, TCR_EL2, TCR_EL2_RES1 },
> + { SYS_DESC(SYS_VTTBR_EL2), access_rw, reset_val, VTTBR_EL2, 0 },
> + { SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
> +
> { SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
> + { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
> + { SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
> + { SYS_DESC(SYS_SP_EL1), access_sp_el1},
> +
> { SYS_DESC(SYS_IFSR32_EL2), NULL, reset_unknown, IFSR32_EL2 },
> + { SYS_DESC(SYS_AFSR0_EL2), access_rw, reset_val, AFSR0_EL2, 0 },
> + { SYS_DESC(SYS_AFSR1_EL2), access_rw, reset_val, AFSR1_EL2, 0 },
> + { SYS_DESC(SYS_ESR_EL2), access_rw, reset_val, ESR_EL2, 0 },
> { SYS_DESC(SYS_FPEXC32_EL2), NULL, reset_val, FPEXC32_EL2, 0x700 },
> +
> + { SYS_DESC(SYS_FAR_EL2), access_rw, reset_val, FAR_EL2, 0 },
> + { SYS_DESC(SYS_HPFAR_EL2), access_rw, reset_val, HPFAR_EL2, 0 },
> +
> + { SYS_DESC(SYS_MAIR_EL2), access_rw, reset_val, MAIR_EL2, 0 },
> + { SYS_DESC(SYS_AMAIR_EL2), access_rw, reset_val, AMAIR_EL2, 0 },
> +
> + { SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
> + { SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
> + { SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
Why is this WI, exactly? Isn't EL2 the highest implemented EL from a
guest's point of view, and thus a write to RMR_EL2 with bit 1 set should
trigger a reset (to AArch64)?
I see that we don't NEED to implement this register (as we don't
support AArch32), but that should look differently here then?
The rest looks alright to me.
Cheers,
Andre
> +
> + { SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
> + { SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
> +
> + { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
> + { SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
> +
> + { SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
> };
>
> static bool trap_dbgidr(struct kvm_vcpu *vcpu,
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps
@ 2021-01-29 18:05 ` Andre Przywara
0 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:05 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvm, kernel-team, kvmarm, Jintack Lim, linux-arm-kernel
On Thu, 10 Dec 2020 15:59:03 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> From: Jintack Lim <jintack.lim@linaro.org>
>
> ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When
> this bit is set, accessing EL2 registers in EL1 traps to EL2. In
> addition, executing the following instructions in EL1 will trap to EL2:
> tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the
> instructions that trap to EL2 with the NV bit were undef at EL1 prior to
> ARM v8.3. The only instruction that was not undef is eret.
>
> This patch sets up a handler for EL2 registers and SP_EL1 register
> accesses at EL1. The host hypervisor keeps those register values in
> memory, and will emulate their behavior.
>
> This patch doesn't set the NV bit yet. It will be set in a later patch
> once nested virtualization support is completed.
>
> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
> [maz: added SCTLR_EL2 RES0/RES1 handling]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
I compared the system register encodings in the first hunk against the
ARMv8 ARM, they are all fine.
I also checked the last hunk for (copy&paste) typos, all good as well.
Some comments below:
> ---
> arch/arm64/include/asm/sysreg.h | 44 ++++++++++++-
> arch/arm64/kvm/sys_regs.c | 112 ++++++++++++++++++++++++++++++--
> 2 files changed, 150 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 82521cdbfc1c..05b49eafbb49 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -469,19 +469,50 @@
>
> #define SYS_PMCCFILTR_EL0 sys_reg(3, 3, 14, 15, 7)
>
> +#define SYS_VPIDR_EL2 sys_reg(3, 4, 0, 0, 0)
> +#define SYS_VMPIDR_EL2 sys_reg(3, 4, 0, 0, 5)
> +
> #define SYS_SCTLR_EL2 sys_reg(3, 4, 1, 0, 0)
> +#define SYS_ACTLR_EL2 sys_reg(3, 4, 1, 0, 1)
> +#define SYS_HCR_EL2 sys_reg(3, 4, 1, 1, 0)
> +#define SYS_MDCR_EL2 sys_reg(3, 4, 1, 1, 1)
> +#define SYS_CPTR_EL2 sys_reg(3, 4, 1, 1, 2)
> +#define SYS_HSTR_EL2 sys_reg(3, 4, 1, 1, 3)
> +#define SYS_HACR_EL2 sys_reg(3, 4, 1, 1, 7)
> +
> #define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
> +
> +#define SYS_TTBR0_EL2 sys_reg(3, 4, 2, 0, 0)
> +#define SYS_TTBR1_EL2 sys_reg(3, 4, 2, 0, 1)
> +#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
> +#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
> +#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
> +
> #define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
> +
> #define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0)
> #define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1)
> +#define SYS_SP_EL1 sys_reg(3, 4, 4, 1, 0)
> +
> #define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
> +#define SYS_AFSR0_EL2 sys_reg(3, 4, 5, 1, 0)
> +#define SYS_AFSR1_EL2 sys_reg(3, 4, 5, 1, 1)
> #define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0)
> #define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3)
> #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
> #define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0)
> #define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
>
> -#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
> +#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
> +#define SYS_HPFAR_EL2 sys_reg(3, 4, 6, 0, 4)
> +
> +#define SYS_MAIR_EL2 sys_reg(3, 4, 10, 2, 0)
> +#define SYS_AMAIR_EL2 sys_reg(3, 4, 10, 3, 0)
> +
> +#define SYS_VBAR_EL2 sys_reg(3, 4, 12, 0, 0)
> +#define SYS_RVBAR_EL2 sys_reg(3, 4, 12, 0, 1)
> +#define SYS_RMR_EL2 sys_reg(3, 4, 12, 0, 2)
> +#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
> #define __SYS__AP0Rx_EL2(x) sys_reg(3, 4, 12, 8, x)
> #define SYS_ICH_AP0R0_EL2 __SYS__AP0Rx_EL2(0)
> #define SYS_ICH_AP0R1_EL2 __SYS__AP0Rx_EL2(1)
> @@ -523,15 +554,24 @@
> #define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
> #define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
>
> +#define SYS_CONTEXTIDR_EL2 sys_reg(3, 4, 13, 0, 1)
> +#define SYS_TPIDR_EL2 sys_reg(3, 4, 13, 0, 2)
> +
> +#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
> +#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
> +
> /* VHE encodings for architectural EL0/1 system registers */
> #define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
> #define SYS_CPACR_EL12 sys_reg(3, 5, 1, 0, 2)
> #define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
> +
> #define SYS_TTBR0_EL12 sys_reg(3, 5, 2, 0, 0)
> #define SYS_TTBR1_EL12 sys_reg(3, 5, 2, 0, 1)
> #define SYS_TCR_EL12 sys_reg(3, 5, 2, 0, 2)
> +
> #define SYS_SPSR_EL12 sys_reg(3, 5, 4, 0, 0)
> #define SYS_ELR_EL12 sys_reg(3, 5, 4, 0, 1)
> +
> #define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0)
> #define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1)
> #define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0)
> @@ -549,6 +589,8 @@
> #define SYS_CNTV_CTL_EL02 sys_reg(3, 5, 14, 3, 1)
> #define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
>
> +#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
> +
> /* Common SCTLR_ELx flags. */
> #define SCTLR_ELx_DSSBS (BIT(44))
> #define SCTLR_ELx_ATA (BIT(43))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 3313dedfa505..c049867a39bc 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -104,6 +104,57 @@ static u32 get_ccsidr(u32 csselr)
> return ccsidr;
> }
>
> +static bool access_rw(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + vcpu_write_sys_reg(vcpu, p->regval, r->reg);
> + else
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> +
> + return true;
> +}
> +
> +static bool access_wi(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + return ignore_write(vcpu, p);
> +
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> + return true;
> +}
> +
> +static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write) {
> + u64 val = p->regval;
> +
> + if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) {
> + val &= ~(GENMASK_ULL(63,45) | GENMASK_ULL(34, 32) |
In ARMv8 ARM F.c bits 49-45 are used for the TWE delay value. The
manual says it's RES0 when FEAT_TWED is not implemented, but this is
true for a lot of other feature bits we don't implement? So shall we
let a guest set them, reducing the mask to (63,50)?
> + BIT_ULL(17));
The ARMv8 ARM F.c lists bit 9 as RES0 as well for (E2H,TGE)=(1,1).
> + val |= SCTLR_EL1_RES1;
> + } else {
> + val &= ~(GENMASK_ULL(63,45) | BIT_ULL(42) |
> + GENMASK_ULL(39, 38) | GENMASK_ULL(35, 32) |
> + BIT_ULL(26) | BIT_ULL(24) | BIT_ULL(20) |
> + BIT_ULL(17) | GENMASK_ULL(15, 14) |
> + GENMASK(10, 7));
> + val |= SCTLR_EL2_RES1;
> + }
> +
> + vcpu_write_sys_reg(vcpu, val, r->reg);
> + } else {
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> + }
> +
> + return true;
> +}
> +
> /*
> * See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
> */
> @@ -342,12 +393,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
> struct sys_reg_params *p,
> const struct sys_reg_desc *r)
> {
> - if (p->is_write) {
> - vcpu_write_sys_reg(vcpu, p->regval, r->reg);
> + access_rw(vcpu, p, r);
> + if (p->is_write)
> vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
> - } else {
> - p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> - }
>
> trace_trap_reg(__func__, r->reg, p->is_write, p->regval);
>
> @@ -1314,6 +1362,18 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> .set_user = set_raz_id_reg, \
> }
>
> +static bool access_sp_el1(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + __vcpu_sys_reg(vcpu, SP_EL1) = p->regval;
> + else
> + p->regval = __vcpu_sys_reg(vcpu, SP_EL1);
> +
> + return true;
> +}
> +
> /*
> * Architected system registers.
> * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
> @@ -1692,9 +1752,51 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> */
> { SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
>
> + { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
> + { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
> +
> + { SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
> + { SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
> + { SYS_DESC(SYS_HCR_EL2), access_rw, reset_val, HCR_EL2, 0 },
> + { SYS_DESC(SYS_MDCR_EL2), access_rw, reset_val, MDCR_EL2, 0 },
> + { SYS_DESC(SYS_CPTR_EL2), access_rw, reset_val, CPTR_EL2, CPTR_EL2_RES1 },
> + { SYS_DESC(SYS_HSTR_EL2), access_rw, reset_val, HSTR_EL2, 0 },
> + { SYS_DESC(SYS_HACR_EL2), access_rw, reset_val, HACR_EL2, 0 },
> +
> + { SYS_DESC(SYS_TTBR0_EL2), access_rw, reset_val, TTBR0_EL2, 0 },
> + { SYS_DESC(SYS_TTBR1_EL2), access_rw, reset_val, TTBR1_EL2, 0 },
> + { SYS_DESC(SYS_TCR_EL2), access_rw, reset_val, TCR_EL2, TCR_EL2_RES1 },
> + { SYS_DESC(SYS_VTTBR_EL2), access_rw, reset_val, VTTBR_EL2, 0 },
> + { SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
> +
> { SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
> + { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
> + { SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
> + { SYS_DESC(SYS_SP_EL1), access_sp_el1},
> +
> { SYS_DESC(SYS_IFSR32_EL2), NULL, reset_unknown, IFSR32_EL2 },
> + { SYS_DESC(SYS_AFSR0_EL2), access_rw, reset_val, AFSR0_EL2, 0 },
> + { SYS_DESC(SYS_AFSR1_EL2), access_rw, reset_val, AFSR1_EL2, 0 },
> + { SYS_DESC(SYS_ESR_EL2), access_rw, reset_val, ESR_EL2, 0 },
> { SYS_DESC(SYS_FPEXC32_EL2), NULL, reset_val, FPEXC32_EL2, 0x700 },
> +
> + { SYS_DESC(SYS_FAR_EL2), access_rw, reset_val, FAR_EL2, 0 },
> + { SYS_DESC(SYS_HPFAR_EL2), access_rw, reset_val, HPFAR_EL2, 0 },
> +
> + { SYS_DESC(SYS_MAIR_EL2), access_rw, reset_val, MAIR_EL2, 0 },
> + { SYS_DESC(SYS_AMAIR_EL2), access_rw, reset_val, AMAIR_EL2, 0 },
> +
> + { SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
> + { SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
> + { SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
Why is this WI, exactly? Isn't EL2 the highest implemented EL from a
guest's point of view, and thus a write to RMR_EL2 with bit 1 set should
trigger a reset (to AArch64)?
I see that we don't NEED to implement this register (as we don't
support AArch32), but that should look differently here then?
The rest looks alright to me.
Cheers,
Andre
> +
> + { SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
> + { SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
> +
> + { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
> + { SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
> +
> + { SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
> };
>
> static bool trap_dbgidr(struct kvm_vcpu *vcpu,
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps
@ 2021-01-29 18:05 ` Andre Przywara
0 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:05 UTC (permalink / raw)
To: Marc Zyngier
Cc: kvm, Suzuki K Poulose, kernel-team, Christoffer Dall, kvmarm,
James Morse, Julien Thierry, Jintack Lim, Alexandru Elisei,
Jintack Lim, linux-arm-kernel
On Thu, 10 Dec 2020 15:59:03 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> From: Jintack Lim <jintack.lim@linaro.org>
>
> ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When
> this bit is set, accessing EL2 registers in EL1 traps to EL2. In
> addition, executing the following instructions in EL1 will trap to EL2:
> tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the
> instructions that trap to EL2 with the NV bit were undef at EL1 prior to
> ARM v8.3. The only instruction that was not undef is eret.
>
> This patch sets up a handler for EL2 registers and SP_EL1 register
> accesses at EL1. The host hypervisor keeps those register values in
> memory, and will emulate their behavior.
>
> This patch doesn't set the NV bit yet. It will be set in a later patch
> once nested virtualization support is completed.
>
> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
> [maz: added SCTLR_EL2 RES0/RES1 handling]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
I compared the system register encodings in the first hunk against the
ARMv8 ARM, they are all fine.
I also checked the last hunk for (copy&paste) typos, all good as well.
Some comments below:
> ---
> arch/arm64/include/asm/sysreg.h | 44 ++++++++++++-
> arch/arm64/kvm/sys_regs.c | 112 ++++++++++++++++++++++++++++++--
> 2 files changed, 150 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 82521cdbfc1c..05b49eafbb49 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -469,19 +469,50 @@
>
> #define SYS_PMCCFILTR_EL0 sys_reg(3, 3, 14, 15, 7)
>
> +#define SYS_VPIDR_EL2 sys_reg(3, 4, 0, 0, 0)
> +#define SYS_VMPIDR_EL2 sys_reg(3, 4, 0, 0, 5)
> +
> #define SYS_SCTLR_EL2 sys_reg(3, 4, 1, 0, 0)
> +#define SYS_ACTLR_EL2 sys_reg(3, 4, 1, 0, 1)
> +#define SYS_HCR_EL2 sys_reg(3, 4, 1, 1, 0)
> +#define SYS_MDCR_EL2 sys_reg(3, 4, 1, 1, 1)
> +#define SYS_CPTR_EL2 sys_reg(3, 4, 1, 1, 2)
> +#define SYS_HSTR_EL2 sys_reg(3, 4, 1, 1, 3)
> +#define SYS_HACR_EL2 sys_reg(3, 4, 1, 1, 7)
> +
> #define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
> +
> +#define SYS_TTBR0_EL2 sys_reg(3, 4, 2, 0, 0)
> +#define SYS_TTBR1_EL2 sys_reg(3, 4, 2, 0, 1)
> +#define SYS_TCR_EL2 sys_reg(3, 4, 2, 0, 2)
> +#define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0)
> +#define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
> +
> #define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
> +
> #define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0)
> #define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1)
> +#define SYS_SP_EL1 sys_reg(3, 4, 4, 1, 0)
> +
> #define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
> +#define SYS_AFSR0_EL2 sys_reg(3, 4, 5, 1, 0)
> +#define SYS_AFSR1_EL2 sys_reg(3, 4, 5, 1, 1)
> #define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0)
> #define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3)
> #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
> #define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0)
> #define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
>
> -#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
> +#define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0)
> +#define SYS_HPFAR_EL2 sys_reg(3, 4, 6, 0, 4)
> +
> +#define SYS_MAIR_EL2 sys_reg(3, 4, 10, 2, 0)
> +#define SYS_AMAIR_EL2 sys_reg(3, 4, 10, 3, 0)
> +
> +#define SYS_VBAR_EL2 sys_reg(3, 4, 12, 0, 0)
> +#define SYS_RVBAR_EL2 sys_reg(3, 4, 12, 0, 1)
> +#define SYS_RMR_EL2 sys_reg(3, 4, 12, 0, 2)
> +#define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1)
> #define __SYS__AP0Rx_EL2(x) sys_reg(3, 4, 12, 8, x)
> #define SYS_ICH_AP0R0_EL2 __SYS__AP0Rx_EL2(0)
> #define SYS_ICH_AP0R1_EL2 __SYS__AP0Rx_EL2(1)
> @@ -523,15 +554,24 @@
> #define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
> #define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
>
> +#define SYS_CONTEXTIDR_EL2 sys_reg(3, 4, 13, 0, 1)
> +#define SYS_TPIDR_EL2 sys_reg(3, 4, 13, 0, 2)
> +
> +#define SYS_CNTVOFF_EL2 sys_reg(3, 4, 14, 0, 3)
> +#define SYS_CNTHCTL_EL2 sys_reg(3, 4, 14, 1, 0)
> +
> /* VHE encodings for architectural EL0/1 system registers */
> #define SYS_SCTLR_EL12 sys_reg(3, 5, 1, 0, 0)
> #define SYS_CPACR_EL12 sys_reg(3, 5, 1, 0, 2)
> #define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
> +
> #define SYS_TTBR0_EL12 sys_reg(3, 5, 2, 0, 0)
> #define SYS_TTBR1_EL12 sys_reg(3, 5, 2, 0, 1)
> #define SYS_TCR_EL12 sys_reg(3, 5, 2, 0, 2)
> +
> #define SYS_SPSR_EL12 sys_reg(3, 5, 4, 0, 0)
> #define SYS_ELR_EL12 sys_reg(3, 5, 4, 0, 1)
> +
> #define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0)
> #define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1)
> #define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0)
> @@ -549,6 +589,8 @@
> #define SYS_CNTV_CTL_EL02 sys_reg(3, 5, 14, 3, 1)
> #define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
>
> +#define SYS_SP_EL2 sys_reg(3, 6, 4, 1, 0)
> +
> /* Common SCTLR_ELx flags. */
> #define SCTLR_ELx_DSSBS (BIT(44))
> #define SCTLR_ELx_ATA (BIT(43))
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 3313dedfa505..c049867a39bc 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -104,6 +104,57 @@ static u32 get_ccsidr(u32 csselr)
> return ccsidr;
> }
>
> +static bool access_rw(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + vcpu_write_sys_reg(vcpu, p->regval, r->reg);
> + else
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> +
> + return true;
> +}
> +
> +static bool access_wi(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + return ignore_write(vcpu, p);
> +
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> + return true;
> +}
> +
> +static bool access_sctlr_el2(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write) {
> + u64 val = p->regval;
> +
> + if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) {
> + val &= ~(GENMASK_ULL(63,45) | GENMASK_ULL(34, 32) |
In ARMv8 ARM F.c bits 49-45 are used for the TWE delay value. The
manual says it's RES0 when FEAT_TWED is not implemented, but this is
true for a lot of other feature bits we don't implement? So shall we
let a guest set them, reducing the mask to (63,50)?
> + BIT_ULL(17));
The ARMv8 ARM F.c lists bit 9 as RES0 as well for (E2H,TGE)=(1,1).
> + val |= SCTLR_EL1_RES1;
> + } else {
> + val &= ~(GENMASK_ULL(63,45) | BIT_ULL(42) |
> + GENMASK_ULL(39, 38) | GENMASK_ULL(35, 32) |
> + BIT_ULL(26) | BIT_ULL(24) | BIT_ULL(20) |
> + BIT_ULL(17) | GENMASK_ULL(15, 14) |
> + GENMASK(10, 7));
> + val |= SCTLR_EL2_RES1;
> + }
> +
> + vcpu_write_sys_reg(vcpu, val, r->reg);
> + } else {
> + p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> + }
> +
> + return true;
> +}
> +
> /*
> * See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
> */
> @@ -342,12 +393,9 @@ static bool trap_debug_regs(struct kvm_vcpu *vcpu,
> struct sys_reg_params *p,
> const struct sys_reg_desc *r)
> {
> - if (p->is_write) {
> - vcpu_write_sys_reg(vcpu, p->regval, r->reg);
> + access_rw(vcpu, p, r);
> + if (p->is_write)
> vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
> - } else {
> - p->regval = vcpu_read_sys_reg(vcpu, r->reg);
> - }
>
> trace_trap_reg(__func__, r->reg, p->is_write, p->regval);
>
> @@ -1314,6 +1362,18 @@ static bool access_ccsidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> .set_user = set_raz_id_reg, \
> }
>
> +static bool access_sp_el1(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + if (p->is_write)
> + __vcpu_sys_reg(vcpu, SP_EL1) = p->regval;
> + else
> + p->regval = __vcpu_sys_reg(vcpu, SP_EL1);
> +
> + return true;
> +}
> +
> /*
> * Architected system registers.
> * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
> @@ -1692,9 +1752,51 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> */
> { SYS_DESC(SYS_PMCCFILTR_EL0), access_pmu_evtyper, reset_val, PMCCFILTR_EL0, 0 },
>
> + { SYS_DESC(SYS_VPIDR_EL2), access_rw, reset_val, VPIDR_EL2, 0 },
> + { SYS_DESC(SYS_VMPIDR_EL2), access_rw, reset_val, VMPIDR_EL2, 0 },
> +
> + { SYS_DESC(SYS_SCTLR_EL2), access_sctlr_el2, reset_val, SCTLR_EL2, SCTLR_EL2_RES1 },
> + { SYS_DESC(SYS_ACTLR_EL2), access_rw, reset_val, ACTLR_EL2, 0 },
> + { SYS_DESC(SYS_HCR_EL2), access_rw, reset_val, HCR_EL2, 0 },
> + { SYS_DESC(SYS_MDCR_EL2), access_rw, reset_val, MDCR_EL2, 0 },
> + { SYS_DESC(SYS_CPTR_EL2), access_rw, reset_val, CPTR_EL2, CPTR_EL2_RES1 },
> + { SYS_DESC(SYS_HSTR_EL2), access_rw, reset_val, HSTR_EL2, 0 },
> + { SYS_DESC(SYS_HACR_EL2), access_rw, reset_val, HACR_EL2, 0 },
> +
> + { SYS_DESC(SYS_TTBR0_EL2), access_rw, reset_val, TTBR0_EL2, 0 },
> + { SYS_DESC(SYS_TTBR1_EL2), access_rw, reset_val, TTBR1_EL2, 0 },
> + { SYS_DESC(SYS_TCR_EL2), access_rw, reset_val, TCR_EL2, TCR_EL2_RES1 },
> + { SYS_DESC(SYS_VTTBR_EL2), access_rw, reset_val, VTTBR_EL2, 0 },
> + { SYS_DESC(SYS_VTCR_EL2), access_rw, reset_val, VTCR_EL2, 0 },
> +
> { SYS_DESC(SYS_DACR32_EL2), NULL, reset_unknown, DACR32_EL2 },
> + { SYS_DESC(SYS_SPSR_EL2), access_rw, reset_val, SPSR_EL2, 0 },
> + { SYS_DESC(SYS_ELR_EL2), access_rw, reset_val, ELR_EL2, 0 },
> + { SYS_DESC(SYS_SP_EL1), access_sp_el1},
> +
> { SYS_DESC(SYS_IFSR32_EL2), NULL, reset_unknown, IFSR32_EL2 },
> + { SYS_DESC(SYS_AFSR0_EL2), access_rw, reset_val, AFSR0_EL2, 0 },
> + { SYS_DESC(SYS_AFSR1_EL2), access_rw, reset_val, AFSR1_EL2, 0 },
> + { SYS_DESC(SYS_ESR_EL2), access_rw, reset_val, ESR_EL2, 0 },
> { SYS_DESC(SYS_FPEXC32_EL2), NULL, reset_val, FPEXC32_EL2, 0x700 },
> +
> + { SYS_DESC(SYS_FAR_EL2), access_rw, reset_val, FAR_EL2, 0 },
> + { SYS_DESC(SYS_HPFAR_EL2), access_rw, reset_val, HPFAR_EL2, 0 },
> +
> + { SYS_DESC(SYS_MAIR_EL2), access_rw, reset_val, MAIR_EL2, 0 },
> + { SYS_DESC(SYS_AMAIR_EL2), access_rw, reset_val, AMAIR_EL2, 0 },
> +
> + { SYS_DESC(SYS_VBAR_EL2), access_rw, reset_val, VBAR_EL2, 0 },
> + { SYS_DESC(SYS_RVBAR_EL2), access_rw, reset_val, RVBAR_EL2, 0 },
> + { SYS_DESC(SYS_RMR_EL2), access_wi, reset_val, RMR_EL2, 1 },
Why is this WI, exactly? Isn't EL2 the highest implemented EL from a
guest's point of view, and thus a write to RMR_EL2 with bit 1 set should
trigger a reset (to AArch64)?
I see that we don't NEED to implement this register (as we don't
support AArch32), but that should look differently here then?
The rest looks alright to me.
Cheers,
Andre
> +
> + { SYS_DESC(SYS_CONTEXTIDR_EL2), access_rw, reset_val, CONTEXTIDR_EL2, 0 },
> + { SYS_DESC(SYS_TPIDR_EL2), access_rw, reset_val, TPIDR_EL2, 0 },
> +
> + { SYS_DESC(SYS_CNTVOFF_EL2), access_rw, reset_val, CNTVOFF_EL2, 0 },
> + { SYS_DESC(SYS_CNTHCTL_EL2), access_rw, reset_val, CNTHCTL_EL2, 0 },
> +
> + { SYS_DESC(SYS_SP_EL2), NULL, reset_unknown, SP_EL2 },
> };
>
> static bool trap_dbgidr(struct kvm_vcpu *vcpu,
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
2020-12-10 15:59 ` Marc Zyngier
(?)
@ 2021-01-29 18:05 ` Andre Przywara
-1 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:05 UTC (permalink / raw)
To: Marc Zyngier
Cc: linux-arm-kernel, kvmarm, kvm, Christoffer Dall, Jintack Lim,
Alexandru Elisei, James Morse, Julien Thierry, Suzuki K Poulose,
kernel-team, Jintack Lim
On Thu, 10 Dec 2020 15:59:05 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> From: Jintack Lim <jintack.lim@linaro.org>
>
> Support injecting exceptions and performing exception returns to and
> from virtual EL2. This must be done entirely in software except when
> taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE}
> == {1,1} (a VHE guest hypervisor).
>
> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
> [maz: switch to common exception injection framework]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_arm.h | 17 +++
> arch/arm64/include/asm/kvm_emulate.h | 10 ++
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/emulate-nested.c | 176 +++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/exception.c | 44 +++++--
> arch/arm64/kvm/inject_fault.c | 62 ++++++++--
> arch/arm64/kvm/trace_arm.h | 59 +++++++++
> 7 files changed, 352 insertions(+), 18 deletions(-)
> create mode 100644 arch/arm64/kvm/emulate-nested.c
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 4e90c2debf70..e9e10e498785 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -332,4 +332,21 @@
> #define CPACR_EL1_TTA (1 << 28)
> #define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
>
> +#define kvm_mode_names \
> + { PSR_MODE_EL0t, "EL0t" }, \
> + { PSR_MODE_EL1t, "EL1t" }, \
> + { PSR_MODE_EL1h, "EL1h" }, \
> + { PSR_MODE_EL2t, "EL2t" }, \
> + { PSR_MODE_EL2h, "EL2h" }, \
> + { PSR_MODE_EL3t, "EL3t" }, \
> + { PSR_MODE_EL3h, "EL3h" }, \
> + { PSR_AA32_MODE_USR, "32-bit USR" }, \
> + { PSR_AA32_MODE_FIQ, "32-bit FIQ" }, \
> + { PSR_AA32_MODE_IRQ, "32-bit IRQ" }, \
> + { PSR_AA32_MODE_SVC, "32-bit SVC" }, \
> + { PSR_AA32_MODE_ABT, "32-bit ABT" }, \
> + { PSR_AA32_MODE_HYP, "32-bit HYP" }, \
> + { PSR_AA32_MODE_UND, "32-bit UND" }, \
> + { PSR_AA32_MODE_SYS, "32-bit SYS" }
> +
> #endif /* __ARM64_KVM_ARM_H__ */
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e3e22d364dc9..deffd3cc4443 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -33,6 +33,12 @@ enum exception_type {
> except_type_serror = 0x180,
> };
>
> +#define kvm_exception_type_names \
> + { except_type_sync, "SYNC" }, \
> + { except_type_irq, "IRQ" }, \
> + { except_type_fiq, "FIQ" }, \
> + { except_type_serror, "SERROR" }
> +
> bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);
> void kvm_skip_instr32(struct kvm_vcpu *vcpu);
>
> @@ -41,6 +47,10 @@ void kvm_inject_vabt(struct kvm_vcpu *vcpu);
> void kvm_inject_dabt(struct kvm_vcpu *vcpu, unsigned long addr);
> void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
>
> +void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu);
> +int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2);
> +int kvm_inject_nested_irq(struct kvm_vcpu *vcpu);
> +
> static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> {
> return !(vcpu->arch.hcr_el2 & HCR_RW);
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 60fd181df624..672acdf1e993 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -17,11 +17,11 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
> guest.o debug.o reset.o sys_regs.o \
> vgic-sys-reg-v3.o fpsimd.o pmu.o \
> arch_timer.o \
> + emulate-nested.o \
> vgic/vgic.o vgic/vgic-init.o \
> vgic/vgic-irqfd.o vgic/vgic-v2.o \
> vgic/vgic-v3.o vgic/vgic-v4.o \
> vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
> vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
> vgic/vgic-its.o vgic/vgic-debug.o
> -
> kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
> diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
> new file mode 100644
> index 000000000000..ee91bcd925d8
> --- /dev/null
> +++ b/arch/arm64/kvm/emulate-nested.c
> @@ -0,0 +1,176 @@
> +/*
> + * Copyright (C) 2016 - Linaro and Columbia University
> + * Author: Jintack Lim <jintack.lim@linaro.org>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_nested.h>
> +
> +#include "hyp/include/hyp/adjust_pc.h"
> +
> +#include "trace.h"
> +
> +void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
> +{
> + u64 spsr, elr, mode;
> + bool direct_eret;
> +
> + /*
> + * Going through the whole put/load motions is a waste of time
> + * if this is a VHE guest hypervisor returning to its own
> + * userspace, or the hypervisor performing a local exception
> + * return. No need to save/restore registers, no need to
> + * switch S2 MMU. Just do the canonical ERET.
> + */
> + spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
> + mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
> +
> + direct_eret = (mode == PSR_MODE_EL0t &&
> + vcpu_el2_e2h_is_set(vcpu) &&
> + vcpu_el2_tge_is_set(vcpu));
> + direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
> +
> + if (direct_eret) {
> + *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
> + *vcpu_cpsr(vcpu) = spsr;
> + trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
> + return;
> + }
> +
> + preempt_disable();
> + kvm_arch_vcpu_put(vcpu);
> +
> + elr = __vcpu_sys_reg(vcpu, ELR_EL2);
> +
> + trace_kvm_nested_eret(vcpu, elr, spsr);
> +
> + /*
> + * Note that the current exception level is always the virtual EL2,
> + * since we set HCR_EL2.NV bit only when entering the virtual EL2.
> + */
> + *vcpu_pc(vcpu) = elr;
> + *vcpu_cpsr(vcpu) = spsr;
> +
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + preempt_enable();
> +}
> +
> +static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,
> + enum exception_type type)
> +{
> + trace_kvm_inject_nested_exception(vcpu, esr_el2, type);
> +
> + switch (type) {
> + case except_type_sync:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_SYNC;
> + break;
> + case except_type_irq:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_IRQ;
> + break;
> + default:
> + WARN_ONCE(1, "Unsupported EL2 exception injection %d\n", type);
> + }
> +
> + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL2 |
> + KVM_ARM64_PENDING_EXCEPTION);
> +
> + vcpu_write_sys_reg(vcpu, esr_el2, ESR_EL2);
> +}
> +
> +/*
> + * Emulate taking an exception to EL2.
> + * See ARM ARM J8.1.2 AArch64.TakeException()
> + */
> +static int kvm_inject_nested(struct kvm_vcpu *vcpu, u64 esr_el2,
> + enum exception_type type)
> +{
> + u64 pstate, mode;
> + bool direct_inject;
> +
> + if (!nested_virt_in_use(vcpu)) {
> + kvm_err("Unexpected call to %s for the non-nesting configuration\n",
> + __func__);
> + return -EINVAL;
> + }
> +
> + /*
> + * As for ERET, we can avoid doing too much on the injection path by
> + * checking that we either took the exception from a VHE host
> + * userspace or from vEL2. In these cases, there is no change in
> + * translation regime (or anything else), so let's do as little as
> + * possible.
> + */
> + pstate = *vcpu_cpsr(vcpu);
> + mode = pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
> +
> + direct_inject = (mode == PSR_MODE_EL0t &&
> + vcpu_el2_e2h_is_set(vcpu) &&
> + vcpu_el2_tge_is_set(vcpu));
> + direct_inject |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
> +
> + if (direct_inject) {
> + kvm_inject_el2_exception(vcpu, esr_el2, type);
> + return 1;
> + }
> +
> + preempt_disable();
> + kvm_arch_vcpu_put(vcpu);
> +
> + kvm_inject_el2_exception(vcpu, esr_el2, type);
> +
> + /*
> + * A hard requirement is that a switch between EL1 and EL2
> + * contexts has to happen between a put/load, so that we can
> + * pick the correct timer and interrupt configuration, among
> + * other things.
> + *
> + * Make sure the exception actually took place before we load
> + * the new context.
> + */
> + __adjust_pc(vcpu);
> +
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + preempt_enable();
> +
> + return 1;
> +}
> +
> +int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2)
> +{
> + return kvm_inject_nested(vcpu, esr_el2, except_type_sync);
> +}
> +
> +int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)
> +{
> + /*
> + * Do not inject an irq if the:
> + * - Current exception level is EL2, and
> + * - virtual HCR_EL2.TGE == 0
> + * - virtual HCR_EL2.IMO == 0
> + *
> + * See Table D1-17 "Physical interrupt target and masking when EL3 is
> + * not implemented and EL2 is implemented" in ARM DDI 0487C.a.
> + */
> +
> + if (vcpu_mode_el2(vcpu) && !vcpu_el2_tge_is_set(vcpu) &&
> + !(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO))
> + return 1;
> +
> + /* esr_el2 value doesn't matter for exits due to irqs. */
> + return kvm_inject_nested(vcpu, 0, except_type_irq);
> +}
> diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
> index 73629094f903..7c8b3681dafd 100644
> --- a/arch/arm64/kvm/hyp/exception.c
> +++ b/arch/arm64/kvm/hyp/
> @@ -22,7 +22,9 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
> {
> u64 val;
>
> - if (__vcpu_read_sys_reg_from_cpu(reg, &val))
> + if (unlikely(nested_virt_in_use(vcpu)))
> + return vcpu_read_sys_reg(vcpu, reg);
> + else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
> return val;
>
> return __vcpu_sys_reg(vcpu, reg);
> @@ -30,14 +32,26 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
>
> static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
> {
> - if (__vcpu_write_sys_reg_to_cpu(val, reg))
> + if (unlikely(nested_virt_in_use(vcpu)))
> + vcpu_write_sys_reg(vcpu, val, reg);
> + else if (__vcpu_write_sys_reg_to_cpu(val, reg))
> return;
>
> __vcpu_sys_reg(vcpu, reg) = val;
> }
>
> -static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, u64 val)
> +static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode,
> + u64 val)
> {
> + if (unlikely(nested_virt_in_use(vcpu))) {
> + if (target_mode == PSR_MODE_EL1h)
> + vcpu_write_sys_reg(vcpu, val, SPSR_EL1);
> + else
> + vcpu_write_sys_reg(vcpu, val, SPSR_EL2);
> +
> + return;
> + }
> +
> write_sysreg_el1(val, SYS_SPSR);
> }
>
> @@ -97,6 +111,11 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
> sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL1);
> __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL1);
> break;
> + case PSR_MODE_EL2h:
> + vbar = __vcpu_read_sys_reg(vcpu, VBAR_EL2);
> + sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL2);
> + __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL2);
> + break;
> default:
> /* Don't do that */
> BUG();
> @@ -148,7 +167,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
> new |= target_mode;
>
> *vcpu_cpsr(vcpu) = new;
> - __vcpu_write_spsr(vcpu, old);
> + __vcpu_write_spsr(vcpu, target_mode, old);
> }
>
> /*
> @@ -319,11 +338,22 @@ void kvm_inject_exception(struct kvm_vcpu *vcpu)
> KVM_ARM64_EXCEPT_AA64_EL1):
> enter_exception64(vcpu, PSR_MODE_EL1h, except_type_sync);
> break;
> +
> + case (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> + KVM_ARM64_EXCEPT_AA64_EL2):
> + enter_exception64(vcpu, PSR_MODE_EL2h, except_type_sync);
> + break;
> +
> + case (KVM_ARM64_EXCEPT_AA64_ELx_IRQ |
> + KVM_ARM64_EXCEPT_AA64_EL2):
> + enter_exception64(vcpu, PSR_MODE_EL2h, except_type_irq);
> + break;
> +
> default:
> /*
> - * Only EL1_SYNC makes sense so far, EL2_{SYNC,IRQ}
> - * will be implemented at some point. Everything
> - * else gets silently ignored.
> + * Only EL1_SYNC and EL2_{SYNC,IRQ} makes
> + * sense so far. Everything else gets silently
> + * ignored.
> */
> break;
> }
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index b47df73e98d7..530562eb09fd 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -14,17 +14,50 @@
> #include <asm/kvm_emulate.h>
> #include <asm/esr.h>
>
> +static void pend_sync_exception(struct kvm_vcpu *vcpu)
> +{
> + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> + KVM_ARM64_PENDING_EXCEPTION);
> +
> + /* If not nesting, EL1 is the only possible exception target */
> + if (likely(!nested_virt_in_use(vcpu))) {
This breaks compilation for the next 14 patches, because
nested_virt_in_use() is unknown here. Patch 23/66 fixes this by adding
kvm_nested.h to kvm_emulate.h.
Shall we do this already in this patch here, or pull in kvm_nested.h in
every C file we use nested_virt_in_use(), like exception.c above?
Cheers,
Andre
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + return;
> + }
> +
> + /*
> + * With NV, we need to pick between EL1 and EL2. Note that we
> + * never deal with a nesting exception here, hence never
> + * changing context, and the exception itself can be delayed
> + * until the next entry.
> + */
> + switch(*vcpu_cpsr(vcpu) & PSR_MODE_MASK) {
> + case PSR_MODE_EL2h:
> + case PSR_MODE_EL2t:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
> + break;
> + case PSR_MODE_EL1h:
> + case PSR_MODE_EL1t:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + break;
> + case PSR_MODE_EL0t:
> + if (vcpu_el2_tge_is_set(vcpu) & HCR_TGE)
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
> + else
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + break;
> + default:
> + BUG();
> + }
> +}
> +
> static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> {
> unsigned long cpsr = *vcpu_cpsr(vcpu);
> bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> u32 esr = 0;
>
> - vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
> - KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> - KVM_ARM64_PENDING_EXCEPTION);
> -
> - vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
> + pend_sync_exception(vcpu);
>
> /*
> * Build an {i,d}abort, depending on the level and the
> @@ -45,16 +78,22 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> if (!is_iabt)
> esr |= ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT;
>
> - vcpu_write_sys_reg(vcpu, esr | ESR_ELx_FSC_EXTABT, ESR_EL1);
> + esr |= ESR_ELx_FSC_EXTABT;
> +
> + if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1) {
> + vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + } else {
> + vcpu_write_sys_reg(vcpu, addr, FAR_EL2);
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
> + }
> }
>
> static void inject_undef64(struct kvm_vcpu *vcpu)
> {
> u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
>
> - vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
> - KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> - KVM_ARM64_PENDING_EXCEPTION);
> + pend_sync_exception(vcpu);
>
> /*
> * Build an unknown exception, depending on the instruction
> @@ -63,7 +102,10 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> if (kvm_vcpu_trap_il_is32bit(vcpu))
> esr |= ESR_ELx_IL;
>
> - vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1)
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + else
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
> }
>
> #define DFSR_FSC_EXTABT_LPAE 0x10
> diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
> index ff0444352bba..5707011c4f47 100644
> --- a/arch/arm64/kvm/trace_arm.h
> +++ b/arch/arm64/kvm/trace_arm.h
> @@ -2,6 +2,7 @@
> #if !defined(_TRACE_ARM_ARM64_KVM_H) || defined(TRACE_HEADER_MULTI_READ)
> #define _TRACE_ARM_ARM64_KVM_H
>
> +#include <asm/kvm_emulate.h>
> #include <kvm/arm_arch_timer.h>
> #include <linux/tracepoint.h>
>
> @@ -367,6 +368,64 @@ TRACE_EVENT(kvm_timer_emulate,
> __entry->timer_idx, __entry->should_fire)
> );
>
> +TRACE_EVENT(kvm_nested_eret,
> + TP_PROTO(struct kvm_vcpu *vcpu, unsigned long elr_el2,
> + unsigned long spsr_el2),
> + TP_ARGS(vcpu, elr_el2, spsr_el2),
> +
> + TP_STRUCT__entry(
> + __field(struct kvm_vcpu *, vcpu)
> + __field(unsigned long, elr_el2)
> + __field(unsigned long, spsr_el2)
> + __field(unsigned long, target_mode)
> + __field(unsigned long, hcr_el2)
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu = vcpu;
> + __entry->elr_el2 = elr_el2;
> + __entry->spsr_el2 = spsr_el2;
> + __entry->target_mode = spsr_el2 & (PSR_MODE_MASK | PSR_MODE32_BIT);
> + __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> + ),
> +
> + TP_printk("elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
> + __entry->elr_el2, __entry->spsr_el2,
> + __print_symbolic(__entry->target_mode, kvm_mode_names),
> + __entry->hcr_el2)
> +);
> +
> +TRACE_EVENT(kvm_inject_nested_exception,
> + TP_PROTO(struct kvm_vcpu *vcpu, u64 esr_el2, int type),
> + TP_ARGS(vcpu, esr_el2, type),
> +
> + TP_STRUCT__entry(
> + __field(struct kvm_vcpu *, vcpu)
> + __field(unsigned long, esr_el2)
> + __field(int, type)
> + __field(unsigned long, spsr_el2)
> + __field(unsigned long, pc)
> + __field(unsigned long, source_mode)
> + __field(unsigned long, hcr_el2)
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu = vcpu;
> + __entry->esr_el2 = esr_el2;
> + __entry->type = type;
> + __entry->spsr_el2 = *vcpu_cpsr(vcpu);
> + __entry->pc = *vcpu_pc(vcpu);
> + __entry->source_mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
> + __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> + ),
> +
> + TP_printk("%s: esr_el2 0x%lx elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
> + __print_symbolic(__entry->type, kvm_exception_type_names),
> + __entry->esr_el2, __entry->pc, __entry->spsr_el2,
> + __print_symbolic(__entry->source_mode, kvm_mode_names),
> + __entry->hcr_el2)
> +);
> +
> #endif /* _TRACE_ARM_ARM64_KVM_H */
>
> #undef TRACE_INCLUDE_PATH
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
@ 2021-01-29 18:05 ` Andre Przywara
0 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:05 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvm, kernel-team, kvmarm, Jintack Lim, linux-arm-kernel
On Thu, 10 Dec 2020 15:59:05 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> From: Jintack Lim <jintack.lim@linaro.org>
>
> Support injecting exceptions and performing exception returns to and
> from virtual EL2. This must be done entirely in software except when
> taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE}
> == {1,1} (a VHE guest hypervisor).
>
> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
> [maz: switch to common exception injection framework]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_arm.h | 17 +++
> arch/arm64/include/asm/kvm_emulate.h | 10 ++
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/emulate-nested.c | 176 +++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/exception.c | 44 +++++--
> arch/arm64/kvm/inject_fault.c | 62 ++++++++--
> arch/arm64/kvm/trace_arm.h | 59 +++++++++
> 7 files changed, 352 insertions(+), 18 deletions(-)
> create mode 100644 arch/arm64/kvm/emulate-nested.c
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 4e90c2debf70..e9e10e498785 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -332,4 +332,21 @@
> #define CPACR_EL1_TTA (1 << 28)
> #define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
>
> +#define kvm_mode_names \
> + { PSR_MODE_EL0t, "EL0t" }, \
> + { PSR_MODE_EL1t, "EL1t" }, \
> + { PSR_MODE_EL1h, "EL1h" }, \
> + { PSR_MODE_EL2t, "EL2t" }, \
> + { PSR_MODE_EL2h, "EL2h" }, \
> + { PSR_MODE_EL3t, "EL3t" }, \
> + { PSR_MODE_EL3h, "EL3h" }, \
> + { PSR_AA32_MODE_USR, "32-bit USR" }, \
> + { PSR_AA32_MODE_FIQ, "32-bit FIQ" }, \
> + { PSR_AA32_MODE_IRQ, "32-bit IRQ" }, \
> + { PSR_AA32_MODE_SVC, "32-bit SVC" }, \
> + { PSR_AA32_MODE_ABT, "32-bit ABT" }, \
> + { PSR_AA32_MODE_HYP, "32-bit HYP" }, \
> + { PSR_AA32_MODE_UND, "32-bit UND" }, \
> + { PSR_AA32_MODE_SYS, "32-bit SYS" }
> +
> #endif /* __ARM64_KVM_ARM_H__ */
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e3e22d364dc9..deffd3cc4443 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -33,6 +33,12 @@ enum exception_type {
> except_type_serror = 0x180,
> };
>
> +#define kvm_exception_type_names \
> + { except_type_sync, "SYNC" }, \
> + { except_type_irq, "IRQ" }, \
> + { except_type_fiq, "FIQ" }, \
> + { except_type_serror, "SERROR" }
> +
> bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);
> void kvm_skip_instr32(struct kvm_vcpu *vcpu);
>
> @@ -41,6 +47,10 @@ void kvm_inject_vabt(struct kvm_vcpu *vcpu);
> void kvm_inject_dabt(struct kvm_vcpu *vcpu, unsigned long addr);
> void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
>
> +void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu);
> +int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2);
> +int kvm_inject_nested_irq(struct kvm_vcpu *vcpu);
> +
> static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> {
> return !(vcpu->arch.hcr_el2 & HCR_RW);
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 60fd181df624..672acdf1e993 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -17,11 +17,11 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
> guest.o debug.o reset.o sys_regs.o \
> vgic-sys-reg-v3.o fpsimd.o pmu.o \
> arch_timer.o \
> + emulate-nested.o \
> vgic/vgic.o vgic/vgic-init.o \
> vgic/vgic-irqfd.o vgic/vgic-v2.o \
> vgic/vgic-v3.o vgic/vgic-v4.o \
> vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
> vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
> vgic/vgic-its.o vgic/vgic-debug.o
> -
> kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
> diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
> new file mode 100644
> index 000000000000..ee91bcd925d8
> --- /dev/null
> +++ b/arch/arm64/kvm/emulate-nested.c
> @@ -0,0 +1,176 @@
> +/*
> + * Copyright (C) 2016 - Linaro and Columbia University
> + * Author: Jintack Lim <jintack.lim@linaro.org>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_nested.h>
> +
> +#include "hyp/include/hyp/adjust_pc.h"
> +
> +#include "trace.h"
> +
> +void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
> +{
> + u64 spsr, elr, mode;
> + bool direct_eret;
> +
> + /*
> + * Going through the whole put/load motions is a waste of time
> + * if this is a VHE guest hypervisor returning to its own
> + * userspace, or the hypervisor performing a local exception
> + * return. No need to save/restore registers, no need to
> + * switch S2 MMU. Just do the canonical ERET.
> + */
> + spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
> + mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
> +
> + direct_eret = (mode == PSR_MODE_EL0t &&
> + vcpu_el2_e2h_is_set(vcpu) &&
> + vcpu_el2_tge_is_set(vcpu));
> + direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
> +
> + if (direct_eret) {
> + *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
> + *vcpu_cpsr(vcpu) = spsr;
> + trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
> + return;
> + }
> +
> + preempt_disable();
> + kvm_arch_vcpu_put(vcpu);
> +
> + elr = __vcpu_sys_reg(vcpu, ELR_EL2);
> +
> + trace_kvm_nested_eret(vcpu, elr, spsr);
> +
> + /*
> + * Note that the current exception level is always the virtual EL2,
> + * since we set HCR_EL2.NV bit only when entering the virtual EL2.
> + */
> + *vcpu_pc(vcpu) = elr;
> + *vcpu_cpsr(vcpu) = spsr;
> +
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + preempt_enable();
> +}
> +
> +static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,
> + enum exception_type type)
> +{
> + trace_kvm_inject_nested_exception(vcpu, esr_el2, type);
> +
> + switch (type) {
> + case except_type_sync:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_SYNC;
> + break;
> + case except_type_irq:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_IRQ;
> + break;
> + default:
> + WARN_ONCE(1, "Unsupported EL2 exception injection %d\n", type);
> + }
> +
> + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL2 |
> + KVM_ARM64_PENDING_EXCEPTION);
> +
> + vcpu_write_sys_reg(vcpu, esr_el2, ESR_EL2);
> +}
> +
> +/*
> + * Emulate taking an exception to EL2.
> + * See ARM ARM J8.1.2 AArch64.TakeException()
> + */
> +static int kvm_inject_nested(struct kvm_vcpu *vcpu, u64 esr_el2,
> + enum exception_type type)
> +{
> + u64 pstate, mode;
> + bool direct_inject;
> +
> + if (!nested_virt_in_use(vcpu)) {
> + kvm_err("Unexpected call to %s for the non-nesting configuration\n",
> + __func__);
> + return -EINVAL;
> + }
> +
> + /*
> + * As for ERET, we can avoid doing too much on the injection path by
> + * checking that we either took the exception from a VHE host
> + * userspace or from vEL2. In these cases, there is no change in
> + * translation regime (or anything else), so let's do as little as
> + * possible.
> + */
> + pstate = *vcpu_cpsr(vcpu);
> + mode = pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
> +
> + direct_inject = (mode == PSR_MODE_EL0t &&
> + vcpu_el2_e2h_is_set(vcpu) &&
> + vcpu_el2_tge_is_set(vcpu));
> + direct_inject |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
> +
> + if (direct_inject) {
> + kvm_inject_el2_exception(vcpu, esr_el2, type);
> + return 1;
> + }
> +
> + preempt_disable();
> + kvm_arch_vcpu_put(vcpu);
> +
> + kvm_inject_el2_exception(vcpu, esr_el2, type);
> +
> + /*
> + * A hard requirement is that a switch between EL1 and EL2
> + * contexts has to happen between a put/load, so that we can
> + * pick the correct timer and interrupt configuration, among
> + * other things.
> + *
> + * Make sure the exception actually took place before we load
> + * the new context.
> + */
> + __adjust_pc(vcpu);
> +
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + preempt_enable();
> +
> + return 1;
> +}
> +
> +int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2)
> +{
> + return kvm_inject_nested(vcpu, esr_el2, except_type_sync);
> +}
> +
> +int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)
> +{
> + /*
> + * Do not inject an irq if the:
> + * - Current exception level is EL2, and
> + * - virtual HCR_EL2.TGE == 0
> + * - virtual HCR_EL2.IMO == 0
> + *
> + * See Table D1-17 "Physical interrupt target and masking when EL3 is
> + * not implemented and EL2 is implemented" in ARM DDI 0487C.a.
> + */
> +
> + if (vcpu_mode_el2(vcpu) && !vcpu_el2_tge_is_set(vcpu) &&
> + !(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO))
> + return 1;
> +
> + /* esr_el2 value doesn't matter for exits due to irqs. */
> + return kvm_inject_nested(vcpu, 0, except_type_irq);
> +}
> diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
> index 73629094f903..7c8b3681dafd 100644
> --- a/arch/arm64/kvm/hyp/exception.c
> +++ b/arch/arm64/kvm/hyp/
> @@ -22,7 +22,9 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
> {
> u64 val;
>
> - if (__vcpu_read_sys_reg_from_cpu(reg, &val))
> + if (unlikely(nested_virt_in_use(vcpu)))
> + return vcpu_read_sys_reg(vcpu, reg);
> + else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
> return val;
>
> return __vcpu_sys_reg(vcpu, reg);
> @@ -30,14 +32,26 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
>
> static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
> {
> - if (__vcpu_write_sys_reg_to_cpu(val, reg))
> + if (unlikely(nested_virt_in_use(vcpu)))
> + vcpu_write_sys_reg(vcpu, val, reg);
> + else if (__vcpu_write_sys_reg_to_cpu(val, reg))
> return;
>
> __vcpu_sys_reg(vcpu, reg) = val;
> }
>
> -static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, u64 val)
> +static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode,
> + u64 val)
> {
> + if (unlikely(nested_virt_in_use(vcpu))) {
> + if (target_mode == PSR_MODE_EL1h)
> + vcpu_write_sys_reg(vcpu, val, SPSR_EL1);
> + else
> + vcpu_write_sys_reg(vcpu, val, SPSR_EL2);
> +
> + return;
> + }
> +
> write_sysreg_el1(val, SYS_SPSR);
> }
>
> @@ -97,6 +111,11 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
> sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL1);
> __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL1);
> break;
> + case PSR_MODE_EL2h:
> + vbar = __vcpu_read_sys_reg(vcpu, VBAR_EL2);
> + sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL2);
> + __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL2);
> + break;
> default:
> /* Don't do that */
> BUG();
> @@ -148,7 +167,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
> new |= target_mode;
>
> *vcpu_cpsr(vcpu) = new;
> - __vcpu_write_spsr(vcpu, old);
> + __vcpu_write_spsr(vcpu, target_mode, old);
> }
>
> /*
> @@ -319,11 +338,22 @@ void kvm_inject_exception(struct kvm_vcpu *vcpu)
> KVM_ARM64_EXCEPT_AA64_EL1):
> enter_exception64(vcpu, PSR_MODE_EL1h, except_type_sync);
> break;
> +
> + case (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> + KVM_ARM64_EXCEPT_AA64_EL2):
> + enter_exception64(vcpu, PSR_MODE_EL2h, except_type_sync);
> + break;
> +
> + case (KVM_ARM64_EXCEPT_AA64_ELx_IRQ |
> + KVM_ARM64_EXCEPT_AA64_EL2):
> + enter_exception64(vcpu, PSR_MODE_EL2h, except_type_irq);
> + break;
> +
> default:
> /*
> - * Only EL1_SYNC makes sense so far, EL2_{SYNC,IRQ}
> - * will be implemented at some point. Everything
> - * else gets silently ignored.
> + * Only EL1_SYNC and EL2_{SYNC,IRQ} makes
> + * sense so far. Everything else gets silently
> + * ignored.
> */
> break;
> }
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index b47df73e98d7..530562eb09fd 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -14,17 +14,50 @@
> #include <asm/kvm_emulate.h>
> #include <asm/esr.h>
>
> +static void pend_sync_exception(struct kvm_vcpu *vcpu)
> +{
> + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> + KVM_ARM64_PENDING_EXCEPTION);
> +
> + /* If not nesting, EL1 is the only possible exception target */
> + if (likely(!nested_virt_in_use(vcpu))) {
This breaks compilation for the next 14 patches, because
nested_virt_in_use() is unknown here. Patch 23/66 fixes this by adding
kvm_nested.h to kvm_emulate.h.
Shall we do this already in this patch here, or pull in kvm_nested.h in
every C file we use nested_virt_in_use(), like exception.c above?
Cheers,
Andre
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + return;
> + }
> +
> + /*
> + * With NV, we need to pick between EL1 and EL2. Note that we
> + * never deal with a nesting exception here, hence never
> + * changing context, and the exception itself can be delayed
> + * until the next entry.
> + */
> + switch(*vcpu_cpsr(vcpu) & PSR_MODE_MASK) {
> + case PSR_MODE_EL2h:
> + case PSR_MODE_EL2t:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
> + break;
> + case PSR_MODE_EL1h:
> + case PSR_MODE_EL1t:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + break;
> + case PSR_MODE_EL0t:
> + if (vcpu_el2_tge_is_set(vcpu) & HCR_TGE)
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
> + else
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + break;
> + default:
> + BUG();
> + }
> +}
> +
> static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> {
> unsigned long cpsr = *vcpu_cpsr(vcpu);
> bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> u32 esr = 0;
>
> - vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
> - KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> - KVM_ARM64_PENDING_EXCEPTION);
> -
> - vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
> + pend_sync_exception(vcpu);
>
> /*
> * Build an {i,d}abort, depending on the level and the
> @@ -45,16 +78,22 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> if (!is_iabt)
> esr |= ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT;
>
> - vcpu_write_sys_reg(vcpu, esr | ESR_ELx_FSC_EXTABT, ESR_EL1);
> + esr |= ESR_ELx_FSC_EXTABT;
> +
> + if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1) {
> + vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + } else {
> + vcpu_write_sys_reg(vcpu, addr, FAR_EL2);
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
> + }
> }
>
> static void inject_undef64(struct kvm_vcpu *vcpu)
> {
> u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
>
> - vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
> - KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> - KVM_ARM64_PENDING_EXCEPTION);
> + pend_sync_exception(vcpu);
>
> /*
> * Build an unknown exception, depending on the instruction
> @@ -63,7 +102,10 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> if (kvm_vcpu_trap_il_is32bit(vcpu))
> esr |= ESR_ELx_IL;
>
> - vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1)
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + else
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
> }
>
> #define DFSR_FSC_EXTABT_LPAE 0x10
> diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
> index ff0444352bba..5707011c4f47 100644
> --- a/arch/arm64/kvm/trace_arm.h
> +++ b/arch/arm64/kvm/trace_arm.h
> @@ -2,6 +2,7 @@
> #if !defined(_TRACE_ARM_ARM64_KVM_H) || defined(TRACE_HEADER_MULTI_READ)
> #define _TRACE_ARM_ARM64_KVM_H
>
> +#include <asm/kvm_emulate.h>
> #include <kvm/arm_arch_timer.h>
> #include <linux/tracepoint.h>
>
> @@ -367,6 +368,64 @@ TRACE_EVENT(kvm_timer_emulate,
> __entry->timer_idx, __entry->should_fire)
> );
>
> +TRACE_EVENT(kvm_nested_eret,
> + TP_PROTO(struct kvm_vcpu *vcpu, unsigned long elr_el2,
> + unsigned long spsr_el2),
> + TP_ARGS(vcpu, elr_el2, spsr_el2),
> +
> + TP_STRUCT__entry(
> + __field(struct kvm_vcpu *, vcpu)
> + __field(unsigned long, elr_el2)
> + __field(unsigned long, spsr_el2)
> + __field(unsigned long, target_mode)
> + __field(unsigned long, hcr_el2)
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu = vcpu;
> + __entry->elr_el2 = elr_el2;
> + __entry->spsr_el2 = spsr_el2;
> + __entry->target_mode = spsr_el2 & (PSR_MODE_MASK | PSR_MODE32_BIT);
> + __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> + ),
> +
> + TP_printk("elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
> + __entry->elr_el2, __entry->spsr_el2,
> + __print_symbolic(__entry->target_mode, kvm_mode_names),
> + __entry->hcr_el2)
> +);
> +
> +TRACE_EVENT(kvm_inject_nested_exception,
> + TP_PROTO(struct kvm_vcpu *vcpu, u64 esr_el2, int type),
> + TP_ARGS(vcpu, esr_el2, type),
> +
> + TP_STRUCT__entry(
> + __field(struct kvm_vcpu *, vcpu)
> + __field(unsigned long, esr_el2)
> + __field(int, type)
> + __field(unsigned long, spsr_el2)
> + __field(unsigned long, pc)
> + __field(unsigned long, source_mode)
> + __field(unsigned long, hcr_el2)
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu = vcpu;
> + __entry->esr_el2 = esr_el2;
> + __entry->type = type;
> + __entry->spsr_el2 = *vcpu_cpsr(vcpu);
> + __entry->pc = *vcpu_pc(vcpu);
> + __entry->source_mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
> + __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> + ),
> +
> + TP_printk("%s: esr_el2 0x%lx elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
> + __print_symbolic(__entry->type, kvm_exception_type_names),
> + __entry->esr_el2, __entry->pc, __entry->spsr_el2,
> + __print_symbolic(__entry->source_mode, kvm_mode_names),
> + __entry->hcr_el2)
> +);
> +
> #endif /* _TRACE_ARM_ARM64_KVM_H */
>
> #undef TRACE_INCLUDE_PATH
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
@ 2021-01-29 18:05 ` Andre Przywara
0 siblings, 0 replies; 249+ messages in thread
From: Andre Przywara @ 2021-01-29 18:05 UTC (permalink / raw)
To: Marc Zyngier
Cc: kvm, Suzuki K Poulose, kernel-team, Christoffer Dall, kvmarm,
James Morse, Julien Thierry, Jintack Lim, Alexandru Elisei,
Jintack Lim, linux-arm-kernel
On Thu, 10 Dec 2020 15:59:05 +0000
Marc Zyngier <maz@kernel.org> wrote:
Hi,
> From: Jintack Lim <jintack.lim@linaro.org>
>
> Support injecting exceptions and performing exception returns to and
> from virtual EL2. This must be done entirely in software except when
> taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE}
> == {1,1} (a VHE guest hypervisor).
>
> Signed-off-by: Jintack Lim <jintack.lim@linaro.org>
> Signed-off-by: Christoffer Dall <christoffer.dall@arm.com>
> [maz: switch to common exception injection framework]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/include/asm/kvm_arm.h | 17 +++
> arch/arm64/include/asm/kvm_emulate.h | 10 ++
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/emulate-nested.c | 176 +++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/exception.c | 44 +++++--
> arch/arm64/kvm/inject_fault.c | 62 ++++++++--
> arch/arm64/kvm/trace_arm.h | 59 +++++++++
> 7 files changed, 352 insertions(+), 18 deletions(-)
> create mode 100644 arch/arm64/kvm/emulate-nested.c
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 4e90c2debf70..e9e10e498785 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -332,4 +332,21 @@
> #define CPACR_EL1_TTA (1 << 28)
> #define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
>
> +#define kvm_mode_names \
> + { PSR_MODE_EL0t, "EL0t" }, \
> + { PSR_MODE_EL1t, "EL1t" }, \
> + { PSR_MODE_EL1h, "EL1h" }, \
> + { PSR_MODE_EL2t, "EL2t" }, \
> + { PSR_MODE_EL2h, "EL2h" }, \
> + { PSR_MODE_EL3t, "EL3t" }, \
> + { PSR_MODE_EL3h, "EL3h" }, \
> + { PSR_AA32_MODE_USR, "32-bit USR" }, \
> + { PSR_AA32_MODE_FIQ, "32-bit FIQ" }, \
> + { PSR_AA32_MODE_IRQ, "32-bit IRQ" }, \
> + { PSR_AA32_MODE_SVC, "32-bit SVC" }, \
> + { PSR_AA32_MODE_ABT, "32-bit ABT" }, \
> + { PSR_AA32_MODE_HYP, "32-bit HYP" }, \
> + { PSR_AA32_MODE_UND, "32-bit UND" }, \
> + { PSR_AA32_MODE_SYS, "32-bit SYS" }
> +
> #endif /* __ARM64_KVM_ARM_H__ */
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e3e22d364dc9..deffd3cc4443 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -33,6 +33,12 @@ enum exception_type {
> except_type_serror = 0x180,
> };
>
> +#define kvm_exception_type_names \
> + { except_type_sync, "SYNC" }, \
> + { except_type_irq, "IRQ" }, \
> + { except_type_fiq, "FIQ" }, \
> + { except_type_serror, "SERROR" }
> +
> bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);
> void kvm_skip_instr32(struct kvm_vcpu *vcpu);
>
> @@ -41,6 +47,10 @@ void kvm_inject_vabt(struct kvm_vcpu *vcpu);
> void kvm_inject_dabt(struct kvm_vcpu *vcpu, unsigned long addr);
> void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
>
> +void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu);
> +int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2);
> +int kvm_inject_nested_irq(struct kvm_vcpu *vcpu);
> +
> static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> {
> return !(vcpu->arch.hcr_el2 & HCR_RW);
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 60fd181df624..672acdf1e993 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -17,11 +17,11 @@ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
> guest.o debug.o reset.o sys_regs.o \
> vgic-sys-reg-v3.o fpsimd.o pmu.o \
> arch_timer.o \
> + emulate-nested.o \
> vgic/vgic.o vgic/vgic-init.o \
> vgic/vgic-irqfd.o vgic/vgic-v2.o \
> vgic/vgic-v3.o vgic/vgic-v4.o \
> vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
> vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
> vgic/vgic-its.o vgic/vgic-debug.o
> -
> kvm-$(CONFIG_KVM_ARM_PMU) += pmu-emul.o
> diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
> new file mode 100644
> index 000000000000..ee91bcd925d8
> --- /dev/null
> +++ b/arch/arm64/kvm/emulate-nested.c
> @@ -0,0 +1,176 @@
> +/*
> + * Copyright (C) 2016 - Linaro and Columbia University
> + * Author: Jintack Lim <jintack.lim@linaro.org>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_nested.h>
> +
> +#include "hyp/include/hyp/adjust_pc.h"
> +
> +#include "trace.h"
> +
> +void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu)
> +{
> + u64 spsr, elr, mode;
> + bool direct_eret;
> +
> + /*
> + * Going through the whole put/load motions is a waste of time
> + * if this is a VHE guest hypervisor returning to its own
> + * userspace, or the hypervisor performing a local exception
> + * return. No need to save/restore registers, no need to
> + * switch S2 MMU. Just do the canonical ERET.
> + */
> + spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2);
> + mode = spsr & (PSR_MODE_MASK | PSR_MODE32_BIT);
> +
> + direct_eret = (mode == PSR_MODE_EL0t &&
> + vcpu_el2_e2h_is_set(vcpu) &&
> + vcpu_el2_tge_is_set(vcpu));
> + direct_eret |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
> +
> + if (direct_eret) {
> + *vcpu_pc(vcpu) = vcpu_read_sys_reg(vcpu, ELR_EL2);
> + *vcpu_cpsr(vcpu) = spsr;
> + trace_kvm_nested_eret(vcpu, *vcpu_pc(vcpu), spsr);
> + return;
> + }
> +
> + preempt_disable();
> + kvm_arch_vcpu_put(vcpu);
> +
> + elr = __vcpu_sys_reg(vcpu, ELR_EL2);
> +
> + trace_kvm_nested_eret(vcpu, elr, spsr);
> +
> + /*
> + * Note that the current exception level is always the virtual EL2,
> + * since we set HCR_EL2.NV bit only when entering the virtual EL2.
> + */
> + *vcpu_pc(vcpu) = elr;
> + *vcpu_cpsr(vcpu) = spsr;
> +
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + preempt_enable();
> +}
> +
> +static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,
> + enum exception_type type)
> +{
> + trace_kvm_inject_nested_exception(vcpu, esr_el2, type);
> +
> + switch (type) {
> + case except_type_sync:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_SYNC;
> + break;
> + case except_type_irq:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_ELx_IRQ;
> + break;
> + default:
> + WARN_ONCE(1, "Unsupported EL2 exception injection %d\n", type);
> + }
> +
> + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL2 |
> + KVM_ARM64_PENDING_EXCEPTION);
> +
> + vcpu_write_sys_reg(vcpu, esr_el2, ESR_EL2);
> +}
> +
> +/*
> + * Emulate taking an exception to EL2.
> + * See ARM ARM J8.1.2 AArch64.TakeException()
> + */
> +static int kvm_inject_nested(struct kvm_vcpu *vcpu, u64 esr_el2,
> + enum exception_type type)
> +{
> + u64 pstate, mode;
> + bool direct_inject;
> +
> + if (!nested_virt_in_use(vcpu)) {
> + kvm_err("Unexpected call to %s for the non-nesting configuration\n",
> + __func__);
> + return -EINVAL;
> + }
> +
> + /*
> + * As for ERET, we can avoid doing too much on the injection path by
> + * checking that we either took the exception from a VHE host
> + * userspace or from vEL2. In these cases, there is no change in
> + * translation regime (or anything else), so let's do as little as
> + * possible.
> + */
> + pstate = *vcpu_cpsr(vcpu);
> + mode = pstate & (PSR_MODE_MASK | PSR_MODE32_BIT);
> +
> + direct_inject = (mode == PSR_MODE_EL0t &&
> + vcpu_el2_e2h_is_set(vcpu) &&
> + vcpu_el2_tge_is_set(vcpu));
> + direct_inject |= (mode == PSR_MODE_EL2h || mode == PSR_MODE_EL2t);
> +
> + if (direct_inject) {
> + kvm_inject_el2_exception(vcpu, esr_el2, type);
> + return 1;
> + }
> +
> + preempt_disable();
> + kvm_arch_vcpu_put(vcpu);
> +
> + kvm_inject_el2_exception(vcpu, esr_el2, type);
> +
> + /*
> + * A hard requirement is that a switch between EL1 and EL2
> + * contexts has to happen between a put/load, so that we can
> + * pick the correct timer and interrupt configuration, among
> + * other things.
> + *
> + * Make sure the exception actually took place before we load
> + * the new context.
> + */
> + __adjust_pc(vcpu);
> +
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + preempt_enable();
> +
> + return 1;
> +}
> +
> +int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2)
> +{
> + return kvm_inject_nested(vcpu, esr_el2, except_type_sync);
> +}
> +
> +int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)
> +{
> + /*
> + * Do not inject an irq if the:
> + * - Current exception level is EL2, and
> + * - virtual HCR_EL2.TGE == 0
> + * - virtual HCR_EL2.IMO == 0
> + *
> + * See Table D1-17 "Physical interrupt target and masking when EL3 is
> + * not implemented and EL2 is implemented" in ARM DDI 0487C.a.
> + */
> +
> + if (vcpu_mode_el2(vcpu) && !vcpu_el2_tge_is_set(vcpu) &&
> + !(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_IMO))
> + return 1;
> +
> + /* esr_el2 value doesn't matter for exits due to irqs. */
> + return kvm_inject_nested(vcpu, 0, except_type_irq);
> +}
> diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
> index 73629094f903..7c8b3681dafd 100644
> --- a/arch/arm64/kvm/hyp/exception.c
> +++ b/arch/arm64/kvm/hyp/
> @@ -22,7 +22,9 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
> {
> u64 val;
>
> - if (__vcpu_read_sys_reg_from_cpu(reg, &val))
> + if (unlikely(nested_virt_in_use(vcpu)))
> + return vcpu_read_sys_reg(vcpu, reg);
> + else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
> return val;
>
> return __vcpu_sys_reg(vcpu, reg);
> @@ -30,14 +32,26 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
>
> static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
> {
> - if (__vcpu_write_sys_reg_to_cpu(val, reg))
> + if (unlikely(nested_virt_in_use(vcpu)))
> + vcpu_write_sys_reg(vcpu, val, reg);
> + else if (__vcpu_write_sys_reg_to_cpu(val, reg))
> return;
>
> __vcpu_sys_reg(vcpu, reg) = val;
> }
>
> -static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, u64 val)
> +static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode,
> + u64 val)
> {
> + if (unlikely(nested_virt_in_use(vcpu))) {
> + if (target_mode == PSR_MODE_EL1h)
> + vcpu_write_sys_reg(vcpu, val, SPSR_EL1);
> + else
> + vcpu_write_sys_reg(vcpu, val, SPSR_EL2);
> +
> + return;
> + }
> +
> write_sysreg_el1(val, SYS_SPSR);
> }
>
> @@ -97,6 +111,11 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
> sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL1);
> __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL1);
> break;
> + case PSR_MODE_EL2h:
> + vbar = __vcpu_read_sys_reg(vcpu, VBAR_EL2);
> + sctlr = __vcpu_read_sys_reg(vcpu, SCTLR_EL2);
> + __vcpu_write_sys_reg(vcpu, *vcpu_pc(vcpu), ELR_EL2);
> + break;
> default:
> /* Don't do that */
> BUG();
> @@ -148,7 +167,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
> new |= target_mode;
>
> *vcpu_cpsr(vcpu) = new;
> - __vcpu_write_spsr(vcpu, old);
> + __vcpu_write_spsr(vcpu, target_mode, old);
> }
>
> /*
> @@ -319,11 +338,22 @@ void kvm_inject_exception(struct kvm_vcpu *vcpu)
> KVM_ARM64_EXCEPT_AA64_EL1):
> enter_exception64(vcpu, PSR_MODE_EL1h, except_type_sync);
> break;
> +
> + case (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> + KVM_ARM64_EXCEPT_AA64_EL2):
> + enter_exception64(vcpu, PSR_MODE_EL2h, except_type_sync);
> + break;
> +
> + case (KVM_ARM64_EXCEPT_AA64_ELx_IRQ |
> + KVM_ARM64_EXCEPT_AA64_EL2):
> + enter_exception64(vcpu, PSR_MODE_EL2h, except_type_irq);
> + break;
> +
> default:
> /*
> - * Only EL1_SYNC makes sense so far, EL2_{SYNC,IRQ}
> - * will be implemented at some point. Everything
> - * else gets silently ignored.
> + * Only EL1_SYNC and EL2_{SYNC,IRQ} makes
> + * sense so far. Everything else gets silently
> + * ignored.
> */
> break;
> }
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index b47df73e98d7..530562eb09fd 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -14,17 +14,50 @@
> #include <asm/kvm_emulate.h>
> #include <asm/esr.h>
>
> +static void pend_sync_exception(struct kvm_vcpu *vcpu)
> +{
> + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> + KVM_ARM64_PENDING_EXCEPTION);
> +
> + /* If not nesting, EL1 is the only possible exception target */
> + if (likely(!nested_virt_in_use(vcpu))) {
This breaks compilation for the next 14 patches, because
nested_virt_in_use() is unknown here. Patch 23/66 fixes this by adding
kvm_nested.h to kvm_emulate.h.
Shall we do this already in this patch here, or pull in kvm_nested.h in
every C file we use nested_virt_in_use(), like exception.c above?
Cheers,
Andre
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + return;
> + }
> +
> + /*
> + * With NV, we need to pick between EL1 and EL2. Note that we
> + * never deal with a nesting exception here, hence never
> + * changing context, and the exception itself can be delayed
> + * until the next entry.
> + */
> + switch(*vcpu_cpsr(vcpu) & PSR_MODE_MASK) {
> + case PSR_MODE_EL2h:
> + case PSR_MODE_EL2t:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
> + break;
> + case PSR_MODE_EL1h:
> + case PSR_MODE_EL1t:
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + break;
> + case PSR_MODE_EL0t:
> + if (vcpu_el2_tge_is_set(vcpu) & HCR_TGE)
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL2;
> + else
> + vcpu->arch.flags |= KVM_ARM64_EXCEPT_AA64_EL1;
> + break;
> + default:
> + BUG();
> + }
> +}
> +
> static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> {
> unsigned long cpsr = *vcpu_cpsr(vcpu);
> bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> u32 esr = 0;
>
> - vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
> - KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> - KVM_ARM64_PENDING_EXCEPTION);
> -
> - vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
> + pend_sync_exception(vcpu);
>
> /*
> * Build an {i,d}abort, depending on the level and the
> @@ -45,16 +78,22 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> if (!is_iabt)
> esr |= ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT;
>
> - vcpu_write_sys_reg(vcpu, esr | ESR_ELx_FSC_EXTABT, ESR_EL1);
> + esr |= ESR_ELx_FSC_EXTABT;
> +
> + if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1) {
> + vcpu_write_sys_reg(vcpu, addr, FAR_EL1);
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + } else {
> + vcpu_write_sys_reg(vcpu, addr, FAR_EL2);
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
> + }
> }
>
> static void inject_undef64(struct kvm_vcpu *vcpu)
> {
> u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
>
> - vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_EL1 |
> - KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> - KVM_ARM64_PENDING_EXCEPTION);
> + pend_sync_exception(vcpu);
>
> /*
> * Build an unknown exception, depending on the instruction
> @@ -63,7 +102,10 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> if (kvm_vcpu_trap_il_is32bit(vcpu))
> esr |= ESR_ELx_IL;
>
> - vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + if (vcpu->arch.flags & KVM_ARM64_EXCEPT_AA64_EL1)
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL1);
> + else
> + vcpu_write_sys_reg(vcpu, esr, ESR_EL2);
> }
>
> #define DFSR_FSC_EXTABT_LPAE 0x10
> diff --git a/arch/arm64/kvm/trace_arm.h b/arch/arm64/kvm/trace_arm.h
> index ff0444352bba..5707011c4f47 100644
> --- a/arch/arm64/kvm/trace_arm.h
> +++ b/arch/arm64/kvm/trace_arm.h
> @@ -2,6 +2,7 @@
> #if !defined(_TRACE_ARM_ARM64_KVM_H) || defined(TRACE_HEADER_MULTI_READ)
> #define _TRACE_ARM_ARM64_KVM_H
>
> +#include <asm/kvm_emulate.h>
> #include <kvm/arm_arch_timer.h>
> #include <linux/tracepoint.h>
>
> @@ -367,6 +368,64 @@ TRACE_EVENT(kvm_timer_emulate,
> __entry->timer_idx, __entry->should_fire)
> );
>
> +TRACE_EVENT(kvm_nested_eret,
> + TP_PROTO(struct kvm_vcpu *vcpu, unsigned long elr_el2,
> + unsigned long spsr_el2),
> + TP_ARGS(vcpu, elr_el2, spsr_el2),
> +
> + TP_STRUCT__entry(
> + __field(struct kvm_vcpu *, vcpu)
> + __field(unsigned long, elr_el2)
> + __field(unsigned long, spsr_el2)
> + __field(unsigned long, target_mode)
> + __field(unsigned long, hcr_el2)
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu = vcpu;
> + __entry->elr_el2 = elr_el2;
> + __entry->spsr_el2 = spsr_el2;
> + __entry->target_mode = spsr_el2 & (PSR_MODE_MASK | PSR_MODE32_BIT);
> + __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> + ),
> +
> + TP_printk("elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
> + __entry->elr_el2, __entry->spsr_el2,
> + __print_symbolic(__entry->target_mode, kvm_mode_names),
> + __entry->hcr_el2)
> +);
> +
> +TRACE_EVENT(kvm_inject_nested_exception,
> + TP_PROTO(struct kvm_vcpu *vcpu, u64 esr_el2, int type),
> + TP_ARGS(vcpu, esr_el2, type),
> +
> + TP_STRUCT__entry(
> + __field(struct kvm_vcpu *, vcpu)
> + __field(unsigned long, esr_el2)
> + __field(int, type)
> + __field(unsigned long, spsr_el2)
> + __field(unsigned long, pc)
> + __field(unsigned long, source_mode)
> + __field(unsigned long, hcr_el2)
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu = vcpu;
> + __entry->esr_el2 = esr_el2;
> + __entry->type = type;
> + __entry->spsr_el2 = *vcpu_cpsr(vcpu);
> + __entry->pc = *vcpu_pc(vcpu);
> + __entry->source_mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
> + __entry->hcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> + ),
> +
> + TP_printk("%s: esr_el2 0x%lx elr_el2: 0x%lx spsr_el2: 0x%08lx (M: %s) hcr_el2: %lx",
> + __print_symbolic(__entry->type, kvm_exception_type_names),
> + __entry->esr_el2, __entry->pc, __entry->spsr_el2,
> + __print_symbolic(__entry->source_mode, kvm_mode_names),
> + __entry->hcr_el2)
> +);
> +
> #endif /* _TRACE_ARM_ARM64_KVM_H */
>
> #undef TRACE_INCLUDE_PATH
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2021-01-21 3:03 ` Haibo Xu
(?)
@ 2021-02-04 7:51 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-02-04 7:51 UTC (permalink / raw)
To: Marc Zyngier; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
Kindly ping!
On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
>
> Re-send in case the previous email was blocked for the inlined hyper-link.
>
> Hi Marc,
>
> I have tried to enable the NV support in Qemu, and now I can
> successfully boot a L2 guest
> in Qemu KVM mode.
>
> This patch series looks good from the Qemu side except for two minor
> requirements:
> (1) Qemu will check whether a feature was supported by the KVM cap
> when the user tries
> to enable it in the command line, so a new capability was
> prefered for the NV(KVM_CAP_ARM_NV?).
> (2) According to the Documentation/virt/kvm/api.rst, userspace can
> call KVM_ARM_VCPU_INIT
> multiple times for a given vcpu, but the kvm_vcpu_init_nested()
> do have some issue when
> called multiple times(please refer to the detailed comments in patch 63)
>
> Regards,
> Haibo
>
> On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
> >
> > This is a rework of the NV series that I posted 10 months ago[1], as a
> > lot of the KVM code has changed since, and the series apply anymore
> > (not that anybody really cares as the the HW is, as usual, made of
> > unobtainium...).
> >
> > From the previous version:
> >
> > - Integration with the new page-table code
> > - New exception injection code
> > - No more messing with the nVHE code
> > - No AArch32!!!!
> > - Rebased on v5.10-rc4 + kvmarm/next for 5.11
> >
> > From a functionality perspective, you can expect a L2 guest to work,
> > but don't even think of L3, as we only partially emulate the
> > ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> > as well as anything that would require a Stage-1 PTW. What we want to
> > achieve is that with NV disabled, there is no performance overhead and
> > no regression.
> >
> > The series is roughly divided in 5 parts: exception handling, memory
> > virtualization, interrupts and timers for ARMv8.3, followed by the
> > ARMv8.4 support. There are of course some dependencies, but you'll
> > hopefully get the gist of it.
> >
> > For the most courageous of you, I've put out a branch[2]. Of course,
> > you'll need some userspace. Andre maintains a hacked version of
> > kvmtool[3] that takes a --nested option, allowing the guest to be
> > started at EL2. You can run the whole stack in the Foundation
> > model. Don't be in a hurry ;-).
> >
> > And to be clear: although Jintack and Christoffer have written tons of
> > the stuff originaly, I'm the one responsible for breaking it!
> >
> > [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> > [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> > [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
> >
> > Andre Przywara (1):
> > KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
> >
> > Christoffer Dall (15):
> > KVM: arm64: nv: Introduce nested virtualization VCPU feature
> > KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> > KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> > KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> > KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> > KVM: arm64: nv: Handle trapped ERET from virtual EL2
> > KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> > KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> > KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> > changes
> > KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> > KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> > KVM: arm64: nv: arch_timer: Support hyp timer emulation
> > KVM: arm64: nv: vgic: Emulate the HW bit in software
> > KVM: arm64: nv: Add nested GICv3 tracepoints
> > KVM: arm64: nv: Sync nested timer state with ARMv8.4
> >
> > Jintack Lim (19):
> > arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> > KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> > KVM: arm64: nv: Support virtual EL2 exceptions
> > KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> > KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> > KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> > KVM: arm64: nv: Handle PSCI call via smc from the guest
> > KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> > KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> > KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> > KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> > KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> > KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> > KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> > KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> > KVM: arm64: nv: Set a handler for the system instruction traps
> > KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> > KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> > KVM: arm64: nv: Nested GICv3 Support
> >
> > Marc Zyngier (31):
> > KVM: arm64: nv: Add EL2 system registers to vcpu context
> > KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> > KVM: arm64: nv: Handle virtual EL2 registers in
> > vcpu_read/write_sys_reg()
> > KVM: arm64: nv: Handle SPSR_EL2 specially
> > KVM: arm64: nv: Handle HCR_EL2.E2H specially
> > KVM: arm64: nv: Save/Restore vEL2 sysregs
> > KVM: arm64: nv: Forward debug traps to the nested guest
> > KVM: arm64: nv: Filter out unsupported features from ID regs
> > KVM: arm64: nv: Hide RAS from nested guests
> > KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> > KVM: arm64: nv: Handle shadow stage 2 page faults
> > KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> > KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> > KVM: arm64: nv: Add handling of EL2-specific timer registers
> > KVM: arm64: nv: Load timer before the GIC
> > KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> > guest
> > KVM: arm64: nv: Implement maintenance interrupt forwarding
> > KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> > KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> > KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> > information
> > KVM: arm64: Allow populating S2 SW bits
> > KVM: arm64: nv: Tag shadow S2 entries with nested level
> > KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> > KVM: arm64: Map VNCR-capable registers to a separate page
> > KVM: arm64: nv: Move nested vgic state into the sysreg file
> > KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> > KVM: arm64: nv: Synchronize PSTATE early on exit
> > KVM: arm64: nv: Allocate VNCR page when required
> > KVM: arm64: nv: Enable ARMv8.4-NV support
> > KVM: arm64: nv: Fast-track 'InHost' exception returns
> > KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
> >
> > .../admin-guide/kernel-parameters.txt | 4 +
> > .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> > arch/arm64/include/asm/cpucaps.h | 2 +
> > arch/arm64/include/asm/esr.h | 6 +
> > arch/arm64/include/asm/kvm_arm.h | 28 +-
> > arch/arm64/include/asm/kvm_asm.h | 4 +
> > arch/arm64/include/asm/kvm_emulate.h | 145 +-
> > arch/arm64/include/asm/kvm_host.h | 175 ++-
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/include/asm/kvm_mmu.h | 17 +-
> > arch/arm64/include/asm/kvm_nested.h | 152 ++
> > arch/arm64/include/asm/kvm_pgtable.h | 10 +
> > arch/arm64/include/asm/sysreg.h | 104 +-
> > arch/arm64/include/asm/vncr_mapping.h | 73 +
> > arch/arm64/include/uapi/asm/kvm.h | 2 +
> > arch/arm64/kernel/cpufeature.c | 35 +
> > arch/arm64/kvm/Makefile | 4 +-
> > arch/arm64/kvm/arch_timer.c | 189 ++-
> > arch/arm64/kvm/arm.c | 34 +-
> > arch/arm64/kvm/at.c | 231 ++++
> > arch/arm64/kvm/emulate-nested.c | 186 +++
> > arch/arm64/kvm/guest.c | 6 +
> > arch/arm64/kvm/handle_exit.c | 81 +-
> > arch/arm64/kvm/hyp/exception.c | 44 +-
> > arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> > arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> > arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> > arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> > arch/arm64/kvm/hyp/pgtable.c | 6 +
> > arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> > arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> > arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> > arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> > arch/arm64/kvm/inject_fault.c | 62 +-
> > arch/arm64/kvm/mmu.c | 183 ++-
> > arch/arm64/kvm/nested.c | 908 ++++++++++++
> > arch/arm64/kvm/reset.c | 14 +-
> > arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> > arch/arm64/kvm/sys_regs.h | 6 +
> > arch/arm64/kvm/trace_arm.h | 65 +-
> > arch/arm64/kvm/vgic/vgic-init.c | 30 +
> > arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> > arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> > arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> > arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> > arch/arm64/kvm/vgic/vgic.c | 44 +
> > arch/arm64/kvm/vgic/vgic.h | 10 +
> > include/kvm/arm_arch_timer.h | 7 +
> > include/kvm/arm_vgic.h | 16 +
> > tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> > 50 files changed, 4890 insertions(+), 160 deletions(-)
> > create mode 100644 arch/arm64/include/asm/kvm_nested.h
> > create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> > create mode 100644 arch/arm64/kvm/at.c
> > create mode 100644 arch/arm64/kvm/emulate-nested.c
> > create mode 100644 arch/arm64/kvm/nested.c
> > create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> > create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
> >
> > --
> > 2.29.2
> >
> > _______________________________________________
> > kvmarm mailing list
> > kvmarm@lists.cs.columbia.edu
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-02-04 7:51 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-02-04 7:51 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
Kindly ping!
On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
>
> Re-send in case the previous email was blocked for the inlined hyper-link.
>
> Hi Marc,
>
> I have tried to enable the NV support in Qemu, and now I can
> successfully boot a L2 guest
> in Qemu KVM mode.
>
> This patch series looks good from the Qemu side except for two minor
> requirements:
> (1) Qemu will check whether a feature was supported by the KVM cap
> when the user tries
> to enable it in the command line, so a new capability was
> prefered for the NV(KVM_CAP_ARM_NV?).
> (2) According to the Documentation/virt/kvm/api.rst, userspace can
> call KVM_ARM_VCPU_INIT
> multiple times for a given vcpu, but the kvm_vcpu_init_nested()
> do have some issue when
> called multiple times(please refer to the detailed comments in patch 63)
>
> Regards,
> Haibo
>
> On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
> >
> > This is a rework of the NV series that I posted 10 months ago[1], as a
> > lot of the KVM code has changed since, and the series apply anymore
> > (not that anybody really cares as the the HW is, as usual, made of
> > unobtainium...).
> >
> > From the previous version:
> >
> > - Integration with the new page-table code
> > - New exception injection code
> > - No more messing with the nVHE code
> > - No AArch32!!!!
> > - Rebased on v5.10-rc4 + kvmarm/next for 5.11
> >
> > From a functionality perspective, you can expect a L2 guest to work,
> > but don't even think of L3, as we only partially emulate the
> > ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> > as well as anything that would require a Stage-1 PTW. What we want to
> > achieve is that with NV disabled, there is no performance overhead and
> > no regression.
> >
> > The series is roughly divided in 5 parts: exception handling, memory
> > virtualization, interrupts and timers for ARMv8.3, followed by the
> > ARMv8.4 support. There are of course some dependencies, but you'll
> > hopefully get the gist of it.
> >
> > For the most courageous of you, I've put out a branch[2]. Of course,
> > you'll need some userspace. Andre maintains a hacked version of
> > kvmtool[3] that takes a --nested option, allowing the guest to be
> > started at EL2. You can run the whole stack in the Foundation
> > model. Don't be in a hurry ;-).
> >
> > And to be clear: although Jintack and Christoffer have written tons of
> > the stuff originaly, I'm the one responsible for breaking it!
> >
> > [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> > [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> > [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
> >
> > Andre Przywara (1):
> > KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
> >
> > Christoffer Dall (15):
> > KVM: arm64: nv: Introduce nested virtualization VCPU feature
> > KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> > KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> > KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> > KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> > KVM: arm64: nv: Handle trapped ERET from virtual EL2
> > KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> > KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> > KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> > changes
> > KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> > KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> > KVM: arm64: nv: arch_timer: Support hyp timer emulation
> > KVM: arm64: nv: vgic: Emulate the HW bit in software
> > KVM: arm64: nv: Add nested GICv3 tracepoints
> > KVM: arm64: nv: Sync nested timer state with ARMv8.4
> >
> > Jintack Lim (19):
> > arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> > KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> > KVM: arm64: nv: Support virtual EL2 exceptions
> > KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> > KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> > KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> > KVM: arm64: nv: Handle PSCI call via smc from the guest
> > KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> > KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> > KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> > KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> > KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> > KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> > KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> > KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> > KVM: arm64: nv: Set a handler for the system instruction traps
> > KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> > KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> > KVM: arm64: nv: Nested GICv3 Support
> >
> > Marc Zyngier (31):
> > KVM: arm64: nv: Add EL2 system registers to vcpu context
> > KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> > KVM: arm64: nv: Handle virtual EL2 registers in
> > vcpu_read/write_sys_reg()
> > KVM: arm64: nv: Handle SPSR_EL2 specially
> > KVM: arm64: nv: Handle HCR_EL2.E2H specially
> > KVM: arm64: nv: Save/Restore vEL2 sysregs
> > KVM: arm64: nv: Forward debug traps to the nested guest
> > KVM: arm64: nv: Filter out unsupported features from ID regs
> > KVM: arm64: nv: Hide RAS from nested guests
> > KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> > KVM: arm64: nv: Handle shadow stage 2 page faults
> > KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> > KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> > KVM: arm64: nv: Add handling of EL2-specific timer registers
> > KVM: arm64: nv: Load timer before the GIC
> > KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> > guest
> > KVM: arm64: nv: Implement maintenance interrupt forwarding
> > KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> > KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> > KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> > information
> > KVM: arm64: Allow populating S2 SW bits
> > KVM: arm64: nv: Tag shadow S2 entries with nested level
> > KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> > KVM: arm64: Map VNCR-capable registers to a separate page
> > KVM: arm64: nv: Move nested vgic state into the sysreg file
> > KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> > KVM: arm64: nv: Synchronize PSTATE early on exit
> > KVM: arm64: nv: Allocate VNCR page when required
> > KVM: arm64: nv: Enable ARMv8.4-NV support
> > KVM: arm64: nv: Fast-track 'InHost' exception returns
> > KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
> >
> > .../admin-guide/kernel-parameters.txt | 4 +
> > .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> > arch/arm64/include/asm/cpucaps.h | 2 +
> > arch/arm64/include/asm/esr.h | 6 +
> > arch/arm64/include/asm/kvm_arm.h | 28 +-
> > arch/arm64/include/asm/kvm_asm.h | 4 +
> > arch/arm64/include/asm/kvm_emulate.h | 145 +-
> > arch/arm64/include/asm/kvm_host.h | 175 ++-
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/include/asm/kvm_mmu.h | 17 +-
> > arch/arm64/include/asm/kvm_nested.h | 152 ++
> > arch/arm64/include/asm/kvm_pgtable.h | 10 +
> > arch/arm64/include/asm/sysreg.h | 104 +-
> > arch/arm64/include/asm/vncr_mapping.h | 73 +
> > arch/arm64/include/uapi/asm/kvm.h | 2 +
> > arch/arm64/kernel/cpufeature.c | 35 +
> > arch/arm64/kvm/Makefile | 4 +-
> > arch/arm64/kvm/arch_timer.c | 189 ++-
> > arch/arm64/kvm/arm.c | 34 +-
> > arch/arm64/kvm/at.c | 231 ++++
> > arch/arm64/kvm/emulate-nested.c | 186 +++
> > arch/arm64/kvm/guest.c | 6 +
> > arch/arm64/kvm/handle_exit.c | 81 +-
> > arch/arm64/kvm/hyp/exception.c | 44 +-
> > arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> > arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> > arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> > arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> > arch/arm64/kvm/hyp/pgtable.c | 6 +
> > arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> > arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> > arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> > arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> > arch/arm64/kvm/inject_fault.c | 62 +-
> > arch/arm64/kvm/mmu.c | 183 ++-
> > arch/arm64/kvm/nested.c | 908 ++++++++++++
> > arch/arm64/kvm/reset.c | 14 +-
> > arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> > arch/arm64/kvm/sys_regs.h | 6 +
> > arch/arm64/kvm/trace_arm.h | 65 +-
> > arch/arm64/kvm/vgic/vgic-init.c | 30 +
> > arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> > arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> > arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> > arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> > arch/arm64/kvm/vgic/vgic.c | 44 +
> > arch/arm64/kvm/vgic/vgic.h | 10 +
> > include/kvm/arm_arch_timer.h | 7 +
> > include/kvm/arm_vgic.h | 16 +
> > tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> > 50 files changed, 4890 insertions(+), 160 deletions(-)
> > create mode 100644 arch/arm64/include/asm/kvm_nested.h
> > create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> > create mode 100644 arch/arm64/kvm/at.c
> > create mode 100644 arch/arm64/kvm/emulate-nested.c
> > create mode 100644 arch/arm64/kvm/nested.c
> > create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> > create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
> >
> > --
> > 2.29.2
> >
> > _______________________________________________
> > kvmarm mailing list
> > kvmarm@lists.cs.columbia.edu
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-02-04 7:51 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-02-04 7:51 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
Kindly ping!
On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
>
> Re-send in case the previous email was blocked for the inlined hyper-link.
>
> Hi Marc,
>
> I have tried to enable the NV support in Qemu, and now I can
> successfully boot a L2 guest
> in Qemu KVM mode.
>
> This patch series looks good from the Qemu side except for two minor
> requirements:
> (1) Qemu will check whether a feature was supported by the KVM cap
> when the user tries
> to enable it in the command line, so a new capability was
> prefered for the NV(KVM_CAP_ARM_NV?).
> (2) According to the Documentation/virt/kvm/api.rst, userspace can
> call KVM_ARM_VCPU_INIT
> multiple times for a given vcpu, but the kvm_vcpu_init_nested()
> do have some issue when
> called multiple times(please refer to the detailed comments in patch 63)
>
> Regards,
> Haibo
>
> On Fri, 11 Dec 2020 at 00:00, Marc Zyngier <maz@kernel.org> wrote:
> >
> > This is a rework of the NV series that I posted 10 months ago[1], as a
> > lot of the KVM code has changed since, and the series apply anymore
> > (not that anybody really cares as the the HW is, as usual, made of
> > unobtainium...).
> >
> > From the previous version:
> >
> > - Integration with the new page-table code
> > - New exception injection code
> > - No more messing with the nVHE code
> > - No AArch32!!!!
> > - Rebased on v5.10-rc4 + kvmarm/next for 5.11
> >
> > From a functionality perspective, you can expect a L2 guest to work,
> > but don't even think of L3, as we only partially emulate the
> > ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU,
> > as well as anything that would require a Stage-1 PTW. What we want to
> > achieve is that with NV disabled, there is no performance overhead and
> > no regression.
> >
> > The series is roughly divided in 5 parts: exception handling, memory
> > virtualization, interrupts and timers for ARMv8.3, followed by the
> > ARMv8.4 support. There are of course some dependencies, but you'll
> > hopefully get the gist of it.
> >
> > For the most courageous of you, I've put out a branch[2]. Of course,
> > you'll need some userspace. Andre maintains a hacked version of
> > kvmtool[3] that takes a --nested option, allowing the guest to be
> > started at EL2. You can run the whole stack in the Foundation
> > model. Don't be in a hurry ;-).
> >
> > And to be clear: although Jintack and Christoffer have written tons of
> > the stuff originaly, I'm the one responsible for breaking it!
> >
> > [1] https://lore.kernel.org/r/20200211174938.27809-1-maz@kernel.org
> > [2] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-5.11.-WIP
> > [3] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
> >
> > Andre Przywara (1):
> > KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
> >
> > Christoffer Dall (15):
> > KVM: arm64: nv: Introduce nested virtualization VCPU feature
> > KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> > KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> > KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> > KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> > KVM: arm64: nv: Handle trapped ERET from virtual EL2
> > KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> > KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> > KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> > changes
> > KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> > KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> > KVM: arm64: nv: arch_timer: Support hyp timer emulation
> > KVM: arm64: nv: vgic: Emulate the HW bit in software
> > KVM: arm64: nv: Add nested GICv3 tracepoints
> > KVM: arm64: nv: Sync nested timer state with ARMv8.4
> >
> > Jintack Lim (19):
> > arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> > KVM: arm64: nv: Handle HCR_EL2.NV system register traps
> > KVM: arm64: nv: Support virtual EL2 exceptions
> > KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> > KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> > KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> > KVM: arm64: nv: Handle PSCI call via smc from the guest
> > KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> > KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> > KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> > KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> > KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> > KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> > KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> > KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> > KVM: arm64: nv: Set a handler for the system instruction traps
> > KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> > KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> > KVM: arm64: nv: Nested GICv3 Support
> >
> > Marc Zyngier (31):
> > KVM: arm64: nv: Add EL2 system registers to vcpu context
> > KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> > KVM: arm64: nv: Handle virtual EL2 registers in
> > vcpu_read/write_sys_reg()
> > KVM: arm64: nv: Handle SPSR_EL2 specially
> > KVM: arm64: nv: Handle HCR_EL2.E2H specially
> > KVM: arm64: nv: Save/Restore vEL2 sysregs
> > KVM: arm64: nv: Forward debug traps to the nested guest
> > KVM: arm64: nv: Filter out unsupported features from ID regs
> > KVM: arm64: nv: Hide RAS from nested guests
> > KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> > KVM: arm64: nv: Handle shadow stage 2 page faults
> > KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> > KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> > KVM: arm64: nv: Add handling of EL2-specific timer registers
> > KVM: arm64: nv: Load timer before the GIC
> > KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> > guest
> > KVM: arm64: nv: Implement maintenance interrupt forwarding
> > KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> > KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
> > KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> > information
> > KVM: arm64: Allow populating S2 SW bits
> > KVM: arm64: nv: Tag shadow S2 entries with nested level
> > KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> > KVM: arm64: Map VNCR-capable registers to a separate page
> > KVM: arm64: nv: Move nested vgic state into the sysreg file
> > KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
> > KVM: arm64: nv: Synchronize PSTATE early on exit
> > KVM: arm64: nv: Allocate VNCR page when required
> > KVM: arm64: nv: Enable ARMv8.4-NV support
> > KVM: arm64: nv: Fast-track 'InHost' exception returns
> > KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
> >
> > .../admin-guide/kernel-parameters.txt | 4 +
> > .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> > arch/arm64/include/asm/cpucaps.h | 2 +
> > arch/arm64/include/asm/esr.h | 6 +
> > arch/arm64/include/asm/kvm_arm.h | 28 +-
> > arch/arm64/include/asm/kvm_asm.h | 4 +
> > arch/arm64/include/asm/kvm_emulate.h | 145 +-
> > arch/arm64/include/asm/kvm_host.h | 175 ++-
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/include/asm/kvm_mmu.h | 17 +-
> > arch/arm64/include/asm/kvm_nested.h | 152 ++
> > arch/arm64/include/asm/kvm_pgtable.h | 10 +
> > arch/arm64/include/asm/sysreg.h | 104 +-
> > arch/arm64/include/asm/vncr_mapping.h | 73 +
> > arch/arm64/include/uapi/asm/kvm.h | 2 +
> > arch/arm64/kernel/cpufeature.c | 35 +
> > arch/arm64/kvm/Makefile | 4 +-
> > arch/arm64/kvm/arch_timer.c | 189 ++-
> > arch/arm64/kvm/arm.c | 34 +-
> > arch/arm64/kvm/at.c | 231 ++++
> > arch/arm64/kvm/emulate-nested.c | 186 +++
> > arch/arm64/kvm/guest.c | 6 +
> > arch/arm64/kvm/handle_exit.c | 81 +-
> > arch/arm64/kvm/hyp/exception.c | 44 +-
> > arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +-
> > arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +-
> > arch/arm64/kvm/hyp/nvhe/switch.c | 10 +-
> > arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> > arch/arm64/kvm/hyp/pgtable.c | 6 +
> > arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
> > arch/arm64/kvm/hyp/vhe/switch.c | 207 ++-
> > arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
> > arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> > arch/arm64/kvm/inject_fault.c | 62 +-
> > arch/arm64/kvm/mmu.c | 183 ++-
> > arch/arm64/kvm/nested.c | 908 ++++++++++++
> > arch/arm64/kvm/reset.c | 14 +-
> > arch/arm64/kvm/sys_regs.c | 1226 ++++++++++++++++-
> > arch/arm64/kvm/sys_regs.h | 6 +
> > arch/arm64/kvm/trace_arm.h | 65 +-
> > arch/arm64/kvm/vgic/vgic-init.c | 30 +
> > arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
> > arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
> > arch/arm64/kvm/vgic/vgic-v3-nested.c | 240 ++++
> > arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
> > arch/arm64/kvm/vgic/vgic.c | 44 +
> > arch/arm64/kvm/vgic/vgic.h | 10 +
> > include/kvm/arm_arch_timer.h | 7 +
> > include/kvm/arm_vgic.h | 16 +
> > tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> > 50 files changed, 4890 insertions(+), 160 deletions(-)
> > create mode 100644 arch/arm64/include/asm/kvm_nested.h
> > create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> > create mode 100644 arch/arm64/kvm/at.c
> > create mode 100644 arch/arm64/kvm/emulate-nested.c
> > create mode 100644 arch/arm64/kvm/nested.c
> > create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
> > create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
> >
> > --
> > 2.29.2
> >
> > _______________________________________________
> > kvmarm mailing list
> > kvmarm@lists.cs.columbia.edu
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
2021-01-21 2:59 ` Haibo Xu
(?)
@ 2021-02-16 18:34 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 18:34 UTC (permalink / raw)
To: Haibo Xu; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Thu, 21 Jan 2021 02:59:13 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
[...]
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index 9fb44bc7db3f..8e85d2ef24d9 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -19,12 +19,177 @@
> > #include <linux/kvm.h>
> > #include <linux/kvm_host.h>
> >
> > +#include <asm/kvm_arm.h>
> > #include <asm/kvm_emulate.h>
> > +#include <asm/kvm_mmu.h>
> > #include <asm/kvm_nested.h>
> > #include <asm/sysreg.h>
> >
> > #include "sys_regs.h"
> >
> > +void kvm_init_nested(struct kvm *kvm)
> > +{
> > + kvm->arch.nested_mmus = NULL;
> > + kvm->arch.nested_mmus_size = 0;
> > +}
> > +
> > +int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct kvm_s2_mmu *tmp;
> > + int num_mmus;
> > + int ret = -ENOMEM;
> > +
> > + if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
> > + return 0;
> > +
> > + if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> > + return -EINVAL;
>
> nit: returning a "not supported" kind of errno?
I think this is consistent with what we otherwise return when there is
a mismatch between requested features from userspace and host
capabilities (such as asking for a 32bit guest on a 64bit only CPU).
What seems to be missing though is a way to *advertise* the feature to
userspace as such:
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 97d377265d8e..0225b81d3434 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -243,6 +243,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_ARM_EL1_32BIT:
r = cpus_have_const_cap(ARM64_HAS_32BIT_EL1);
break;
+ case KVM_CAP_ARM_EL2:
+ r = cpus_have_const_cap(ARM64_HAS_NESTED_VIRT);
+ break;
case KVM_CAP_GUEST_DEBUG_HW_BPS:
r = get_num_brps();
break;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 374c67875cdb..555a60ac00d8 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1058,6 +1058,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190
#define KVM_CAP_SYS_HYPERV_CPUID 191
#define KVM_CAP_DIRTY_LOG_RING 192
+#define KVM_CAP_ARM_EL2 193
#ifdef KVM_CAP_IRQ_ROUTING
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
@ 2021-02-16 18:34 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 18:34 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 21 Jan 2021 02:59:13 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
[...]
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index 9fb44bc7db3f..8e85d2ef24d9 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -19,12 +19,177 @@
> > #include <linux/kvm.h>
> > #include <linux/kvm_host.h>
> >
> > +#include <asm/kvm_arm.h>
> > #include <asm/kvm_emulate.h>
> > +#include <asm/kvm_mmu.h>
> > #include <asm/kvm_nested.h>
> > #include <asm/sysreg.h>
> >
> > #include "sys_regs.h"
> >
> > +void kvm_init_nested(struct kvm *kvm)
> > +{
> > + kvm->arch.nested_mmus = NULL;
> > + kvm->arch.nested_mmus_size = 0;
> > +}
> > +
> > +int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct kvm_s2_mmu *tmp;
> > + int num_mmus;
> > + int ret = -ENOMEM;
> > +
> > + if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
> > + return 0;
> > +
> > + if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> > + return -EINVAL;
>
> nit: returning a "not supported" kind of errno?
I think this is consistent with what we otherwise return when there is
a mismatch between requested features from userspace and host
capabilities (such as asking for a 32bit guest on a 64bit only CPU).
What seems to be missing though is a way to *advertise* the feature to
userspace as such:
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 97d377265d8e..0225b81d3434 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -243,6 +243,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_ARM_EL1_32BIT:
r = cpus_have_const_cap(ARM64_HAS_32BIT_EL1);
break;
+ case KVM_CAP_ARM_EL2:
+ r = cpus_have_const_cap(ARM64_HAS_NESTED_VIRT);
+ break;
case KVM_CAP_GUEST_DEBUG_HW_BPS:
r = get_num_brps();
break;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 374c67875cdb..555a60ac00d8 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1058,6 +1058,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190
#define KVM_CAP_SYS_HYPERV_CPUID 191
#define KVM_CAP_DIRTY_LOG_RING 192
+#define KVM_CAP_ARM_EL2 193
#ifdef KVM_CAP_IRQ_ROUTING
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
@ 2021-02-16 18:34 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 18:34 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 21 Jan 2021 02:59:13 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
[...]
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index 9fb44bc7db3f..8e85d2ef24d9 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -19,12 +19,177 @@
> > #include <linux/kvm.h>
> > #include <linux/kvm_host.h>
> >
> > +#include <asm/kvm_arm.h>
> > #include <asm/kvm_emulate.h>
> > +#include <asm/kvm_mmu.h>
> > #include <asm/kvm_nested.h>
> > #include <asm/sysreg.h>
> >
> > #include "sys_regs.h"
> >
> > +void kvm_init_nested(struct kvm *kvm)
> > +{
> > + kvm->arch.nested_mmus = NULL;
> > + kvm->arch.nested_mmus_size = 0;
> > +}
> > +
> > +int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct kvm_s2_mmu *tmp;
> > + int num_mmus;
> > + int ret = -ENOMEM;
> > +
> > + if (!test_bit(KVM_ARM_VCPU_HAS_EL2, vcpu->arch.features))
> > + return 0;
> > +
> > + if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> > + return -EINVAL;
>
> nit: returning a "not supported" kind of errno?
I think this is consistent with what we otherwise return when there is
a mismatch between requested features from userspace and host
capabilities (such as asking for a 32bit guest on a 64bit only CPU).
What seems to be missing though is a way to *advertise* the feature to
userspace as such:
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 97d377265d8e..0225b81d3434 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -243,6 +243,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_ARM_EL1_32BIT:
r = cpus_have_const_cap(ARM64_HAS_32BIT_EL1);
break;
+ case KVM_CAP_ARM_EL2:
+ r = cpus_have_const_cap(ARM64_HAS_NESTED_VIRT);
+ break;
case KVM_CAP_GUEST_DEBUG_HW_BPS:
r = get_num_brps();
break;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 374c67875cdb..555a60ac00d8 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1058,6 +1058,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190
#define KVM_CAP_SYS_HYPERV_CPUID 191
#define KVM_CAP_DIRTY_LOG_RING 192
+#define KVM_CAP_ARM_EL2 193
#ifdef KVM_CAP_IRQ_ROUTING
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
2021-01-21 2:47 ` Haibo Xu
(?)
@ 2021-02-16 19:03 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 19:03 UTC (permalink / raw)
To: Haibo Xu; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Thu, 21 Jan 2021 02:47:45 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
> >
> > If running a NV guest on an ARMv8.4-NV capable system, let's
> > allocate an additional page that will be used by the hypervisor
> > to fulfill system register accesses.
> >
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > ---
> > arch/arm64/include/asm/kvm_host.h | 3 ++-
> > arch/arm64/kvm/nested.c | 8 ++++++++
> > arch/arm64/kvm/reset.c | 1 +
> > 3 files changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 78630bd5124d..dada0678c28e 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
> > */
> > static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
> > {
> > - if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
> > + if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
> > + r >= __VNCR_START__ && ctxt->vncr_array))
> > return &ctxt->vncr_array[r - __VNCR_START__];
> >
> > return (u64 *)&ctxt->sys_regs[r];
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index eef8f9873814..88147ec99755 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> > if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> > return -EINVAL;
> >
> > + if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
> > + vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
> > + if (!vcpu->arch.ctxt.vncr_array)
> > + return -ENOMEM;
> > + }
> > +
>
> If KVM_ARM_VCPU_INIT was called multiple times, the above codes
> would try to allocate a new page without free-ing the previous
> one. Besides that, the following kvm_free_stage2_pgd() call would
I assume you mean kvm_init_stage2_mmu() here.
> fail in the second call with the error message "kvm_arch already
> initialized?". I think a possible fix is to add a new flag to
> indicate whether the NV related meta data have been initialized, and
> only initialize them for the first call.
Good catch. But I think we have all the data we need at this stage to
avoid this issue:
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index abb0669bdd4c..baff7373863f 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -390,7 +390,20 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
int cpu, err;
struct kvm_pgtable *pgt;
+ /*
+ * If we already have our page tables in place, and that the
+ * MMU context is the canonical one, we have a bug somewhere,
+ * as this is only supposed to ever happen once per VM.
+ *
+ * Otherwise, we're building nested page tables, and that's
+ * probably because userspace called KVM_ARM_VCPU_INIT more
+ * than once on the same vcpu. Since that's actually legal,
+ * don't kick a fuss and leave gracefully.
+ */
if (mmu->pgt != NULL) {
+ if (&kvm->arch.mmu != mmu)
+ return 0;
+
kvm_err("kvm_arch already initialized?\n");
return -EINVAL;
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 88147ec99755..3b21ea57fbce 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -48,7 +48,9 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return -EINVAL;
if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
- vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+ if (!vcpu->arch.ctxt.vncr_array)
+ vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+
if (!vcpu->arch.ctxt.vncr_array)
return -ENOMEM;
}
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
@ 2021-02-16 19:03 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 19:03 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 21 Jan 2021 02:47:45 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
> >
> > If running a NV guest on an ARMv8.4-NV capable system, let's
> > allocate an additional page that will be used by the hypervisor
> > to fulfill system register accesses.
> >
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > ---
> > arch/arm64/include/asm/kvm_host.h | 3 ++-
> > arch/arm64/kvm/nested.c | 8 ++++++++
> > arch/arm64/kvm/reset.c | 1 +
> > 3 files changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 78630bd5124d..dada0678c28e 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
> > */
> > static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
> > {
> > - if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
> > + if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
> > + r >= __VNCR_START__ && ctxt->vncr_array))
> > return &ctxt->vncr_array[r - __VNCR_START__];
> >
> > return (u64 *)&ctxt->sys_regs[r];
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index eef8f9873814..88147ec99755 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> > if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> > return -EINVAL;
> >
> > + if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
> > + vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
> > + if (!vcpu->arch.ctxt.vncr_array)
> > + return -ENOMEM;
> > + }
> > +
>
> If KVM_ARM_VCPU_INIT was called multiple times, the above codes
> would try to allocate a new page without free-ing the previous
> one. Besides that, the following kvm_free_stage2_pgd() call would
I assume you mean kvm_init_stage2_mmu() here.
> fail in the second call with the error message "kvm_arch already
> initialized?". I think a possible fix is to add a new flag to
> indicate whether the NV related meta data have been initialized, and
> only initialize them for the first call.
Good catch. But I think we have all the data we need at this stage to
avoid this issue:
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index abb0669bdd4c..baff7373863f 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -390,7 +390,20 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
int cpu, err;
struct kvm_pgtable *pgt;
+ /*
+ * If we already have our page tables in place, and that the
+ * MMU context is the canonical one, we have a bug somewhere,
+ * as this is only supposed to ever happen once per VM.
+ *
+ * Otherwise, we're building nested page tables, and that's
+ * probably because userspace called KVM_ARM_VCPU_INIT more
+ * than once on the same vcpu. Since that's actually legal,
+ * don't kick a fuss and leave gracefully.
+ */
if (mmu->pgt != NULL) {
+ if (&kvm->arch.mmu != mmu)
+ return 0;
+
kvm_err("kvm_arch already initialized?\n");
return -EINVAL;
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 88147ec99755..3b21ea57fbce 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -48,7 +48,9 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return -EINVAL;
if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
- vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+ if (!vcpu->arch.ctxt.vncr_array)
+ vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+
if (!vcpu->arch.ctxt.vncr_array)
return -ENOMEM;
}
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required
@ 2021-02-16 19:03 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 19:03 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 21 Jan 2021 02:47:45 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> On Fri, 11 Dec 2020 at 00:04, Marc Zyngier <maz@kernel.org> wrote:
> >
> > If running a NV guest on an ARMv8.4-NV capable system, let's
> > allocate an additional page that will be used by the hypervisor
> > to fulfill system register accesses.
> >
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > ---
> > arch/arm64/include/asm/kvm_host.h | 3 ++-
> > arch/arm64/kvm/nested.c | 8 ++++++++
> > arch/arm64/kvm/reset.c | 1 +
> > 3 files changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 78630bd5124d..dada0678c28e 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -523,7 +523,8 @@ struct kvm_vcpu_arch {
> > */
> > static inline u64 *__ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r)
> > {
> > - if (unlikely(r >= __VNCR_START__ && ctxt->vncr_array))
> > + if (unlikely(cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT) &&
> > + r >= __VNCR_START__ && ctxt->vncr_array))
> > return &ctxt->vncr_array[r - __VNCR_START__];
> >
> > return (u64 *)&ctxt->sys_regs[r];
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index eef8f9873814..88147ec99755 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -47,6 +47,12 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
> > if (!cpus_have_final_cap(ARM64_HAS_NESTED_VIRT))
> > return -EINVAL;
> >
> > + if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
> > + vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
> > + if (!vcpu->arch.ctxt.vncr_array)
> > + return -ENOMEM;
> > + }
> > +
>
> If KVM_ARM_VCPU_INIT was called multiple times, the above codes
> would try to allocate a new page without free-ing the previous
> one. Besides that, the following kvm_free_stage2_pgd() call would
I assume you mean kvm_init_stage2_mmu() here.
> fail in the second call with the error message "kvm_arch already
> initialized?". I think a possible fix is to add a new flag to
> indicate whether the NV related meta data have been initialized, and
> only initialize them for the first call.
Good catch. But I think we have all the data we need at this stage to
avoid this issue:
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index abb0669bdd4c..baff7373863f 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -390,7 +390,20 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
int cpu, err;
struct kvm_pgtable *pgt;
+ /*
+ * If we already have our page tables in place, and that the
+ * MMU context is the canonical one, we have a bug somewhere,
+ * as this is only supposed to ever happen once per VM.
+ *
+ * Otherwise, we're building nested page tables, and that's
+ * probably because userspace called KVM_ARM_VCPU_INIT more
+ * than once on the same vcpu. Since that's actually legal,
+ * don't kick a fuss and leave gracefully.
+ */
if (mmu->pgt != NULL) {
+ if (&kvm->arch.mmu != mmu)
+ return 0;
+
kvm_err("kvm_arch already initialized?\n");
return -EINVAL;
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 88147ec99755..3b21ea57fbce 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -48,7 +48,9 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return -EINVAL;
if (cpus_have_final_cap(ARM64_HAS_ENHANCED_NESTED_VIRT)) {
- vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+ if (!vcpu->arch.ctxt.vncr_array)
+ vcpu->arch.ctxt.vncr_array = (u64 *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+
if (!vcpu->arch.ctxt.vncr_array)
return -ENOMEM;
}
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 249+ messages in thread
* Re: [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
2021-01-29 18:05 ` Andre Przywara
(?)
@ 2021-02-16 22:05 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 22:05 UTC (permalink / raw)
To: Andre Przywara
Cc: linux-arm-kernel, kvmarm, kvm, Christoffer Dall, Jintack Lim,
Alexandru Elisei, James Morse, Julien Thierry, Suzuki K Poulose,
kernel-team, Jintack Lim
On Fri, 29 Jan 2021 18:05:39 +0000,
Andre Przywara <andre.przywara@arm.com> wrote:
>
> On Thu, 10 Dec 2020 15:59:05 +0000
> Marc Zyngier <maz@kernel.org> wrote:
[...]
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index b47df73e98d7..530562eb09fd 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -14,17 +14,50 @@
> > #include <asm/kvm_emulate.h>
> > #include <asm/esr.h>
> >
> > +static void pend_sync_exception(struct kvm_vcpu *vcpu)
> > +{
> > + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> > + KVM_ARM64_PENDING_EXCEPTION);
> > +
> > + /* If not nesting, EL1 is the only possible exception target */
> > + if (likely(!nested_virt_in_use(vcpu))) {
>
> This breaks compilation for the next 14 patches, because
> nested_virt_in_use() is unknown here. Patch 23/66 fixes this by adding
> kvm_nested.h to kvm_emulate.h.
Indeed, well caught.
> Shall we do this already in this patch here, or pull in kvm_nested.h in
> every C file we use nested_virt_in_use(), like exception.c above?
I've just added kvm_nested.h in the two places that were required at
this stage (and verified that it all bisects correctly now).
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
@ 2021-02-16 22:05 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 22:05 UTC (permalink / raw)
To: Andre Przywara; +Cc: kvm, kernel-team, kvmarm, Jintack Lim, linux-arm-kernel
On Fri, 29 Jan 2021 18:05:39 +0000,
Andre Przywara <andre.przywara@arm.com> wrote:
>
> On Thu, 10 Dec 2020 15:59:05 +0000
> Marc Zyngier <maz@kernel.org> wrote:
[...]
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index b47df73e98d7..530562eb09fd 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -14,17 +14,50 @@
> > #include <asm/kvm_emulate.h>
> > #include <asm/esr.h>
> >
> > +static void pend_sync_exception(struct kvm_vcpu *vcpu)
> > +{
> > + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> > + KVM_ARM64_PENDING_EXCEPTION);
> > +
> > + /* If not nesting, EL1 is the only possible exception target */
> > + if (likely(!nested_virt_in_use(vcpu))) {
>
> This breaks compilation for the next 14 patches, because
> nested_virt_in_use() is unknown here. Patch 23/66 fixes this by adding
> kvm_nested.h to kvm_emulate.h.
Indeed, well caught.
> Shall we do this already in this patch here, or pull in kvm_nested.h in
> every C file we use nested_virt_in_use(), like exception.c above?
I've just added kvm_nested.h in the two places that were required at
this stage (and verified that it all bisects correctly now).
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions
@ 2021-02-16 22:05 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-16 22:05 UTC (permalink / raw)
To: Andre Przywara
Cc: kvm, Suzuki K Poulose, kernel-team, Christoffer Dall, kvmarm,
James Morse, Julien Thierry, Jintack Lim, Alexandru Elisei,
Jintack Lim, linux-arm-kernel
On Fri, 29 Jan 2021 18:05:39 +0000,
Andre Przywara <andre.przywara@arm.com> wrote:
>
> On Thu, 10 Dec 2020 15:59:05 +0000
> Marc Zyngier <maz@kernel.org> wrote:
[...]
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index b47df73e98d7..530562eb09fd 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -14,17 +14,50 @@
> > #include <asm/kvm_emulate.h>
> > #include <asm/esr.h>
> >
> > +static void pend_sync_exception(struct kvm_vcpu *vcpu)
> > +{
> > + vcpu->arch.flags |= (KVM_ARM64_EXCEPT_AA64_ELx_SYNC |
> > + KVM_ARM64_PENDING_EXCEPTION);
> > +
> > + /* If not nesting, EL1 is the only possible exception target */
> > + if (likely(!nested_virt_in_use(vcpu))) {
>
> This breaks compilation for the next 14 patches, because
> nested_virt_in_use() is unknown here. Patch 23/66 fixes this by adding
> kvm_nested.h to kvm_emulate.h.
Indeed, well caught.
> Shall we do this already in this patch here, or pull in kvm_nested.h in
> every C file we use nested_virt_in_use(), like exception.c above?
I've just added kvm_nested.h in the two places that were required at
this stage (and verified that it all bisects correctly now).
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2021-02-04 7:51 ` Haibo Xu
(?)
@ 2021-02-17 22:10 ` Marc Zyngier
-1 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-17 22:10 UTC (permalink / raw)
To: Haibo Xu; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Thu, 04 Feb 2021 07:51:37 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> Kindly ping!
>
> On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
> >
> > Re-send in case the previous email was blocked for the inlined hyper-link.
> >
> > Hi Marc,
> >
> > I have tried to enable the NV support in Qemu, and now I can
> > successfully boot a L2 guest
> > in Qemu KVM mode.
> >
> > This patch series looks good from the Qemu side except for two minor
> > requirements:
> > (1) Qemu will check whether a feature was supported by the KVM cap
> > when the user tries to enable it in the command line, so a new
> > capability was prefered for the NV(KVM_CAP_ARM_NV?).
I have added KVM_CAP_ARM_EL2 (rather than NV) to that effect.
> > (2) According to the Documentation/virt/kvm/api.rst, userspace can
> > call KVM_ARM_VCPU_INIT multiple times for a given vcpu, but the
> > kvm_vcpu_init_nested() do have some issue when called multiple
> > times(please refer to the detailed comments in patch 63)
This is now fixed, I believe.
I have pushed out a branch [1] that addresses all the reported
issues, though it currently lack some testing. Please let me know if
it works for you.
Thanks,
M.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv-5.12-WIP
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-02-17 22:10 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-17 22:10 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 04 Feb 2021 07:51:37 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> Kindly ping!
>
> On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
> >
> > Re-send in case the previous email was blocked for the inlined hyper-link.
> >
> > Hi Marc,
> >
> > I have tried to enable the NV support in Qemu, and now I can
> > successfully boot a L2 guest
> > in Qemu KVM mode.
> >
> > This patch series looks good from the Qemu side except for two minor
> > requirements:
> > (1) Qemu will check whether a feature was supported by the KVM cap
> > when the user tries to enable it in the command line, so a new
> > capability was prefered for the NV(KVM_CAP_ARM_NV?).
I have added KVM_CAP_ARM_EL2 (rather than NV) to that effect.
> > (2) According to the Documentation/virt/kvm/api.rst, userspace can
> > call KVM_ARM_VCPU_INIT multiple times for a given vcpu, but the
> > kvm_vcpu_init_nested() do have some issue when called multiple
> > times(please refer to the detailed comments in patch 63)
This is now fixed, I believe.
I have pushed out a branch [1] that addresses all the reported
issues, though it currently lack some testing. Please let me know if
it works for you.
Thanks,
M.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv-5.12-WIP
--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-02-17 22:10 ` Marc Zyngier
0 siblings, 0 replies; 249+ messages in thread
From: Marc Zyngier @ 2021-02-17 22:10 UTC (permalink / raw)
To: Haibo Xu; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 04 Feb 2021 07:51:37 +0000,
Haibo Xu <haibo.xu@linaro.org> wrote:
>
> Kindly ping!
>
> On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
> >
> > Re-send in case the previous email was blocked for the inlined hyper-link.
> >
> > Hi Marc,
> >
> > I have tried to enable the NV support in Qemu, and now I can
> > successfully boot a L2 guest
> > in Qemu KVM mode.
> >
> > This patch series looks good from the Qemu side except for two minor
> > requirements:
> > (1) Qemu will check whether a feature was supported by the KVM cap
> > when the user tries to enable it in the command line, so a new
> > capability was prefered for the NV(KVM_CAP_ARM_NV?).
I have added KVM_CAP_ARM_EL2 (rather than NV) to that effect.
> > (2) According to the Documentation/virt/kvm/api.rst, userspace can
> > call KVM_ARM_VCPU_INIT multiple times for a given vcpu, but the
> > kvm_vcpu_init_nested() do have some issue when called multiple
> > times(please refer to the detailed comments in patch 63)
This is now fixed, I believe.
I have pushed out a branch [1] that addresses all the reported
issues, though it currently lack some testing. Please let me know if
it works for you.
Thanks,
M.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv-5.12-WIP
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
2021-02-17 22:10 ` Marc Zyngier
(?)
@ 2021-02-23 9:43 ` Haibo Xu
-1 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-02-23 9:43 UTC (permalink / raw)
To: Marc Zyngier; +Cc: arm-mail-list, kvmarm, kvm, kernel-team, Andre Przywara
On Thu, 18 Feb 2021 at 06:10, Marc Zyngier <maz@kernel.org> wrote:
>
> On Thu, 04 Feb 2021 07:51:37 +0000,
> Haibo Xu <haibo.xu@linaro.org> wrote:
> >
> > Kindly ping!
> >
> > On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
> > >
> > > Re-send in case the previous email was blocked for the inlined hyper-link.
> > >
> > > Hi Marc,
> > >
> > > I have tried to enable the NV support in Qemu, and now I can
> > > successfully boot a L2 guest
> > > in Qemu KVM mode.
> > >
> > > This patch series looks good from the Qemu side except for two minor
> > > requirements:
> > > (1) Qemu will check whether a feature was supported by the KVM cap
> > > when the user tries to enable it in the command line, so a new
> > > capability was prefered for the NV(KVM_CAP_ARM_NV?).
>
> I have added KVM_CAP_ARM_EL2 (rather than NV) to that effect.
>
> > > (2) According to the Documentation/virt/kvm/api.rst, userspace can
> > > call KVM_ARM_VCPU_INIT multiple times for a given vcpu, but the
> > > kvm_vcpu_init_nested() do have some issue when called multiple
> > > times(please refer to the detailed comments in patch 63)
>
> This is now fixed, I believe.
>
> I have pushed out a branch [1] that addresses all the reported
> issues, though it currently lack some testing. Please let me know if
> it works for you.
>
Hi Marc,
I have verified the fix, and it works well with Qemu.
thanks,
Haibo
> Thanks,
>
> M.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv-5.12-WIP
>
> --
> Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-02-23 9:43 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-02-23 9:43 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 18 Feb 2021 at 06:10, Marc Zyngier <maz@kernel.org> wrote:
>
> On Thu, 04 Feb 2021 07:51:37 +0000,
> Haibo Xu <haibo.xu@linaro.org> wrote:
> >
> > Kindly ping!
> >
> > On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
> > >
> > > Re-send in case the previous email was blocked for the inlined hyper-link.
> > >
> > > Hi Marc,
> > >
> > > I have tried to enable the NV support in Qemu, and now I can
> > > successfully boot a L2 guest
> > > in Qemu KVM mode.
> > >
> > > This patch series looks good from the Qemu side except for two minor
> > > requirements:
> > > (1) Qemu will check whether a feature was supported by the KVM cap
> > > when the user tries to enable it in the command line, so a new
> > > capability was prefered for the NV(KVM_CAP_ARM_NV?).
>
> I have added KVM_CAP_ARM_EL2 (rather than NV) to that effect.
>
> > > (2) According to the Documentation/virt/kvm/api.rst, userspace can
> > > call KVM_ARM_VCPU_INIT multiple times for a given vcpu, but the
> > > kvm_vcpu_init_nested() do have some issue when called multiple
> > > times(please refer to the detailed comments in patch 63)
>
> This is now fixed, I believe.
>
> I have pushed out a branch [1] that addresses all the reported
> issues, though it currently lack some testing. Please let me know if
> it works for you.
>
Hi Marc,
I have verified the fix, and it works well with Qemu.
thanks,
Haibo
> Thanks,
>
> M.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv-5.12-WIP
>
> --
> Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 249+ messages in thread
* Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
@ 2021-02-23 9:43 ` Haibo Xu
0 siblings, 0 replies; 249+ messages in thread
From: Haibo Xu @ 2021-02-23 9:43 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Andre Przywara, kernel-team, kvmarm, arm-mail-list, kvm
On Thu, 18 Feb 2021 at 06:10, Marc Zyngier <maz@kernel.org> wrote:
>
> On Thu, 04 Feb 2021 07:51:37 +0000,
> Haibo Xu <haibo.xu@linaro.org> wrote:
> >
> > Kindly ping!
> >
> > On Thu, 21 Jan 2021 at 11:03, Haibo Xu <haibo.xu@linaro.org> wrote:
> > >
> > > Re-send in case the previous email was blocked for the inlined hyper-link.
> > >
> > > Hi Marc,
> > >
> > > I have tried to enable the NV support in Qemu, and now I can
> > > successfully boot a L2 guest
> > > in Qemu KVM mode.
> > >
> > > This patch series looks good from the Qemu side except for two minor
> > > requirements:
> > > (1) Qemu will check whether a feature was supported by the KVM cap
> > > when the user tries to enable it in the command line, so a new
> > > capability was prefered for the NV(KVM_CAP_ARM_NV?).
>
> I have added KVM_CAP_ARM_EL2 (rather than NV) to that effect.
>
> > > (2) According to the Documentation/virt/kvm/api.rst, userspace can
> > > call KVM_ARM_VCPU_INIT multiple times for a given vcpu, but the
> > > kvm_vcpu_init_nested() do have some issue when called multiple
> > > times(please refer to the detailed comments in patch 63)
>
> This is now fixed, I believe.
>
> I have pushed out a branch [1] that addresses all the reported
> issues, though it currently lack some testing. Please let me know if
> it works for you.
>
Hi Marc,
I have verified the fix, and it works well with Qemu.
thanks,
Haibo
> Thanks,
>
> M.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv-5.12-WIP
>
> --
> Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 249+ messages in thread
end of thread, other threads:[~2021-02-23 9:45 UTC | newest]
Thread overview: 249+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-10 15:58 [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` [PATCH v3 01/66] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` [PATCH v3 02/66] KVM: arm64: nv: Introduce nested virtualization VCPU feature Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` [PATCH v3 03/66] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:58 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 04/66] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2021-01-29 18:04 ` Andre Przywara
2021-01-29 18:04 ` Andre Przywara
2021-01-29 18:04 ` Andre Przywara
2020-12-10 15:59 ` [PATCH v3 06/66] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2021-01-29 18:05 ` Andre Przywara
2021-01-29 18:05 ` Andre Przywara
2021-01-29 18:05 ` Andre Przywara
2020-12-10 15:59 ` [PATCH v3 08/66] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 09/66] KVM: arm64: nv: Support virtual EL2 exceptions Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2021-01-29 18:05 ` Andre Przywara
2021-01-29 18:05 ` Andre Przywara
2021-01-29 18:05 ` Andre Przywara
2021-02-16 22:05 ` Marc Zyngier
2021-02-16 22:05 ` Marc Zyngier
2021-02-16 22:05 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 10/66] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2 Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 11/66] KVM: arm64: nv: Handle trapped ERET from " Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 12/66] KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 13/66] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg() Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 14/66] KVM: arm64: nv: Handle SPSR_EL2 specially Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 15/66] KVM: arm64: nv: Handle HCR_EL2.E2H specially Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 16/66] KVM: arm64: nv: Save/Restore vEL2 sysregs Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 17/66] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 18/66] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2 Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 19/66] KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from " Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 20/66] KVM: arm64: nv: Trap CPACR_EL1 access in " Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 21/66] KVM: arm64: nv: Handle PSCI call via smc from the guest Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 22/66] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 23/66] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 23/66] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP, FPEN} settings Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 24/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 25/66] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 26/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 27/66] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2 Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 28/66] KVM: arm64: nv: Forward debug traps to the nested guest Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 29/66] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 30/66] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 31/66] KVM: arm64: nv: Filter out unsupported features from ID regs Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 32/66] KVM: arm64: nv: Hide RAS from nested guests Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2021-01-21 2:59 ` Haibo Xu
2021-01-21 2:59 ` Haibo Xu
2021-01-21 2:59 ` Haibo Xu
2021-02-16 18:34 ` Marc Zyngier
2021-02-16 18:34 ` Marc Zyngier
2021-02-16 18:34 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 34/66] KVM: arm64: nv: Implement nested Stage-2 page table walk logic Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 35/66] KVM: arm64: nv: Handle shadow stage 2 page faults Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 36/66] KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 37/66] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 38/66] KVM: arm64: nv: Introduce sys_reg_desc.forward_trap Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 39/66] KVM: arm64: nv: Set a handler for the system instruction traps Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 40/66] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2 Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 41/66] KVM: arm64: nv: Trap and emulate TLBI " Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 42/66] KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 43/66] KVM: arm64: nv: arch_timer: Support hyp timer emulation Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-11 0:26 ` kernel test robot
2020-12-10 15:59 ` [PATCH v3 44/66] KVM: arm64: nv: Add handling of EL2-specific timer registers Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 45/66] KVM: arm64: nv: Load timer before the GIC Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 46/66] KVM: arm64: nv: Nested GICv3 Support Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 47/66] KVM: arm64: nv: Don't load the GICv4 context on entering a nested guest Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 48/66] KVM: arm64: nv: vgic: Emulate the HW bit in software Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 49/66] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 50/66] KVM: arm64: nv: Implement maintenance interrupt forwarding Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 51/66] KVM: arm64: nv: Add nested GICv3 tracepoints Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 52/66] KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 53/66] KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 54/66] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 55/66] KVM: arm64: Allow populating S2 SW bits Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 56/66] KVM: arm64: nv: Tag shadow S2 entries with nested level Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 57/66] KVM: arm64: nv: Add include containing the VNCR_EL2 offsets Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 58/66] KVM: arm64: Map VNCR-capable registers to a separate page Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 59/66] KVM: arm64: nv: Move nested vgic state into the sysreg file Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 60/66] KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 61/66] KVM: arm64: nv: Synchronize PSTATE early on exit Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 62/66] KVM: arm64: nv: Sync nested timer state with ARMv8.4 Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` [PATCH v3 63/66] KVM: arm64: nv: Allocate VNCR page when required Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2020-12-10 15:59 ` Marc Zyngier
2021-01-21 2:47 ` Haibo Xu
2021-01-21 2:47 ` Haibo Xu
2021-01-21 2:47 ` Haibo Xu
2021-02-16 19:03 ` Marc Zyngier
2021-02-16 19:03 ` Marc Zyngier
2021-02-16 19:03 ` Marc Zyngier
2020-12-10 16:00 ` [PATCH v3 64/66] KVM: arm64: nv: Enable ARMv8.4-NV support Marc Zyngier
2020-12-10 16:00 ` Marc Zyngier
2020-12-10 16:00 ` Marc Zyngier
2020-12-10 16:00 ` [PATCH v3 65/66] KVM: arm64: nv: Fast-track 'InHost' exception returns Marc Zyngier
2020-12-10 16:00 ` Marc Zyngier
2020-12-10 16:00 ` Marc Zyngier
2020-12-10 16:00 ` [PATCH v3 66/66] KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests Marc Zyngier
2020-12-10 16:00 ` Marc Zyngier
2020-12-10 16:00 ` Marc Zyngier
2021-01-11 7:20 ` [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Haibo Xu
2021-01-11 8:59 ` Marc Zyngier
2021-01-11 8:59 ` Marc Zyngier
2021-01-11 8:59 ` Marc Zyngier
2021-01-12 8:02 ` Haibo Xu
2021-01-12 8:02 ` Haibo Xu
2021-01-12 8:02 ` Haibo Xu
2021-01-18 17:38 ` André Przywara
2021-01-18 17:38 ` André Przywara
2021-01-18 17:38 ` André Przywara
2021-01-21 2:35 ` Haibo Xu
2021-01-21 3:03 ` Haibo Xu
2021-01-21 3:03 ` Haibo Xu
2021-01-21 3:03 ` Haibo Xu
2021-02-04 7:51 ` Haibo Xu
2021-02-04 7:51 ` Haibo Xu
2021-02-04 7:51 ` Haibo Xu
2021-02-17 22:10 ` Marc Zyngier
2021-02-17 22:10 ` Marc Zyngier
2021-02-17 22:10 ` Marc Zyngier
2021-02-23 9:43 ` Haibo Xu
2021-02-23 9:43 ` Haibo Xu
2021-02-23 9:43 ` Haibo Xu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.