* [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
KVM's current means of saving/restoring system counters is plagued with
temporal issues. At least on ARM64 and x86, we migrate the guest's
system counter by-value through the respective guest system register
values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
brittle as the state is not idempotent: the host system counter is still
oscillating between the attempted save and restore. Furthermore, VMMs
may wish to transparently live migrate guest VMs, meaning that they
include the elapsed time due to live migration blackout in the guest
system counter view. The VMM thread could be preempted for any number of
reasons (scheduler, L0 hypervisor under nested) between the time that
it calculates the desired guest counter value and when KVM actually sets
this counter state.
Despite the value-based interface that we present to userspace, KVM
actually has idempotent guest controls by way of system counter offsets.
We can avoid all of the issues associated with a value-based interface
by abstracting these offset controls in new ioctls. This series
introduces new vCPU device attributes to provide userspace access to the
vCPU's system counter offset.
Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
ioctls to provide userspace with a (host_tsc, realtime) instant. This is
essential for a VMM to perform precise migration of the guest's system
counters.
Patches 2-3 add support for x86 by shoehorning the new controls into the
pre-existing synchronization heuristics.
Patches 4-5 implement a test for the new additions to
KVM_{GET,SET}_CLOCK.
Patches 6-7 implement at test for the tsc offset attribute introduced in
patch 3.
Patch 8 adds a device attribute for the arm64 virtual counter-timer
offset.
Patch 9 extends the test from patch 7 to cover the arm64 virtual
counter-timer offset.
Patch 10 adds a device attribute for the arm64 physical counter-timer
offset. Currently, this is implemented as a synthetic register, forcing
the guest to trap to the host and emulating the offset in the fast exit
path. Later down the line we will have hardware with FEAT_ECV, which
allows the hypervisor to perform physical counter-timer offsetting in
hardware (CNTPOFF_EL2).
Patch 11 extends the test from patch 7 to cover the arm64 physical
counter-timer offset.
Patch 12 introduces a benchmark to measure the overhead of emulation in
patch 10.
Physical counter benchmark
--------------------------
The following data was collected by running 10000 iterations of the
benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
machine with 2 80-core Ampere Altra SoCs. Measurements were collected
for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
parameter.
nVHE
----
+--------------------+--------+---------+
| Metric | Native | Trapped |
+--------------------+--------+---------+
| Average | 54ns | 148ns |
| Standard Deviation | 124ns | 122ns |
| 95th Percentile | 258ns | 348ns |
+--------------------+--------+---------+
VHE
---
+--------------------+--------+---------+
| Metric | Native | Trapped |
+--------------------+--------+---------+
| Average | 53ns | 152ns |
| Standard Deviation | 92ns | 94ns |
| 95th Percentile | 204ns | 307ns |
+--------------------+--------+---------+
This series applies cleanly to the following commit:
1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
v1 -> v2:
- Reimplemented as vCPU device attributes instead of a distinct ioctl.
- Added the (realtime, host_tsc) instant support to
KVM_{GET,SET}_CLOCK
- Changed the arm64 implementation to broadcast counter offset values
to all vCPUs in a guest. This upholds the architectural expectations
of a consistent counter-timer across CPUs.
- Fixed a bug with traps in VHE mode. We now configure traps on every
transition into a guest to handle differing VMs (trapped, emulated).
Oliver Upton (12):
KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
KVM: x86: Refactor tsc synchronization code
KVM: x86: Expose TSC offset controls to userspace
tools: arch: x86: pull in pvclock headers
selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
selftests: KVM: Add helpers for vCPU device attributes
selftests: KVM: Introduce system counter offset test
KVM: arm64: Allow userspace to configure a vCPU's virtual offset
selftests: KVM: Add support for aarch64 to system_counter_offset_test
KVM: arm64: Provide userspace access to the physical counter offset
selftests: KVM: Test physical counter offsetting
selftests: KVM: Add counter emulation benchmark
Documentation/virt/kvm/api.rst | 42 +-
Documentation/virt/kvm/locking.rst | 11 +
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kvm/arch_timer.c | 118 ++++-
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +
arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++
arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +-
arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++
arch/x86/include/asm/kvm_host.h | 4 +
arch/x86/include/uapi/asm/kvm.h | 4 +
arch/x86/kvm/x86.c | 421 ++++++++++++++----
include/kvm/arm_arch_timer.h | 2 -
include/uapi/linux/kvm.h | 7 +-
tools/arch/x86/include/asm/pvclock-abi.h | 48 ++
tools/arch/x86/include/asm/pvclock.h | 103 +++++
tools/testing/selftests/kvm/.gitignore | 3 +
tools/testing/selftests/kvm/Makefile | 4 +
.../kvm/aarch64/counter_emulation_benchmark.c | 215 +++++++++
.../selftests/kvm/include/aarch64/processor.h | 24 +
.../testing/selftests/kvm/include/kvm_util.h | 11 +
tools/testing/selftests/kvm/lib/kvm_util.c | 38 ++
.../kvm/system_counter_offset_test.c | 206 +++++++++
.../selftests/kvm/x86_64/kvm_clock_test.c | 210 +++++++++
28 files changed, 1447 insertions(+), 133 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
create mode 100644 tools/arch/x86/include/asm/pvclock.h
create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
KVM's current means of saving/restoring system counters is plagued with
temporal issues. At least on ARM64 and x86, we migrate the guest's
system counter by-value through the respective guest system register
values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
brittle as the state is not idempotent: the host system counter is still
oscillating between the attempted save and restore. Furthermore, VMMs
may wish to transparently live migrate guest VMs, meaning that they
include the elapsed time due to live migration blackout in the guest
system counter view. The VMM thread could be preempted for any number of
reasons (scheduler, L0 hypervisor under nested) between the time that
it calculates the desired guest counter value and when KVM actually sets
this counter state.
Despite the value-based interface that we present to userspace, KVM
actually has idempotent guest controls by way of system counter offsets.
We can avoid all of the issues associated with a value-based interface
by abstracting these offset controls in new ioctls. This series
introduces new vCPU device attributes to provide userspace access to the
vCPU's system counter offset.
Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
ioctls to provide userspace with a (host_tsc, realtime) instant. This is
essential for a VMM to perform precise migration of the guest's system
counters.
Patches 2-3 add support for x86 by shoehorning the new controls into the
pre-existing synchronization heuristics.
Patches 4-5 implement a test for the new additions to
KVM_{GET,SET}_CLOCK.
Patches 6-7 implement at test for the tsc offset attribute introduced in
patch 3.
Patch 8 adds a device attribute for the arm64 virtual counter-timer
offset.
Patch 9 extends the test from patch 7 to cover the arm64 virtual
counter-timer offset.
Patch 10 adds a device attribute for the arm64 physical counter-timer
offset. Currently, this is implemented as a synthetic register, forcing
the guest to trap to the host and emulating the offset in the fast exit
path. Later down the line we will have hardware with FEAT_ECV, which
allows the hypervisor to perform physical counter-timer offsetting in
hardware (CNTPOFF_EL2).
Patch 11 extends the test from patch 7 to cover the arm64 physical
counter-timer offset.
Patch 12 introduces a benchmark to measure the overhead of emulation in
patch 10.
Physical counter benchmark
--------------------------
The following data was collected by running 10000 iterations of the
benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
machine with 2 80-core Ampere Altra SoCs. Measurements were collected
for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
parameter.
nVHE
----
+--------------------+--------+---------+
| Metric | Native | Trapped |
+--------------------+--------+---------+
| Average | 54ns | 148ns |
| Standard Deviation | 124ns | 122ns |
| 95th Percentile | 258ns | 348ns |
+--------------------+--------+---------+
VHE
---
+--------------------+--------+---------+
| Metric | Native | Trapped |
+--------------------+--------+---------+
| Average | 53ns | 152ns |
| Standard Deviation | 92ns | 94ns |
| 95th Percentile | 204ns | 307ns |
+--------------------+--------+---------+
This series applies cleanly to the following commit:
1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
v1 -> v2:
- Reimplemented as vCPU device attributes instead of a distinct ioctl.
- Added the (realtime, host_tsc) instant support to
KVM_{GET,SET}_CLOCK
- Changed the arm64 implementation to broadcast counter offset values
to all vCPUs in a guest. This upholds the architectural expectations
of a consistent counter-timer across CPUs.
- Fixed a bug with traps in VHE mode. We now configure traps on every
transition into a guest to handle differing VMs (trapped, emulated).
Oliver Upton (12):
KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
KVM: x86: Refactor tsc synchronization code
KVM: x86: Expose TSC offset controls to userspace
tools: arch: x86: pull in pvclock headers
selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
selftests: KVM: Add helpers for vCPU device attributes
selftests: KVM: Introduce system counter offset test
KVM: arm64: Allow userspace to configure a vCPU's virtual offset
selftests: KVM: Add support for aarch64 to system_counter_offset_test
KVM: arm64: Provide userspace access to the physical counter offset
selftests: KVM: Test physical counter offsetting
selftests: KVM: Add counter emulation benchmark
Documentation/virt/kvm/api.rst | 42 +-
Documentation/virt/kvm/locking.rst | 11 +
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kvm/arch_timer.c | 118 ++++-
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +
arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++
arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +-
arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++
arch/x86/include/asm/kvm_host.h | 4 +
arch/x86/include/uapi/asm/kvm.h | 4 +
arch/x86/kvm/x86.c | 421 ++++++++++++++----
include/kvm/arm_arch_timer.h | 2 -
include/uapi/linux/kvm.h | 7 +-
tools/arch/x86/include/asm/pvclock-abi.h | 48 ++
tools/arch/x86/include/asm/pvclock.h | 103 +++++
tools/testing/selftests/kvm/.gitignore | 3 +
tools/testing/selftests/kvm/Makefile | 4 +
.../kvm/aarch64/counter_emulation_benchmark.c | 215 +++++++++
.../selftests/kvm/include/aarch64/processor.h | 24 +
.../testing/selftests/kvm/include/kvm_util.h | 11 +
tools/testing/selftests/kvm/lib/kvm_util.c | 38 ++
.../kvm/system_counter_offset_test.c | 206 +++++++++
.../selftests/kvm/x86_64/kvm_clock_test.c | 210 +++++++++
28 files changed, 1447 insertions(+), 133 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
create mode 100644 tools/arch/x86/include/asm/pvclock.h
create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread* [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Handling the migration of TSCs correctly is difficult, in part because
Linux does not provide userspace with the ability to retrieve a (TSC,
realtime) clock pair for a single instant in time. In lieu of a more
convenient facility, KVM can report similar information in the kvm_clock
structure.
Provide userspace with a host TSC & realtime pair iff the realtime clock
is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid
realtime value, advance the KVM clock by the amount of elapsed time. Do
not step the KVM clock backwards, though, as it is a monotonic
oscillator.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
Documentation/virt/kvm/api.rst | 42 +++++++--
arch/x86/include/asm/kvm_host.h | 3 +
arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------
include/uapi/linux/kvm.h | 7 +-
4 files changed, 137 insertions(+), 64 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index b9ddce5638f5..26bb01a6e82e 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -993,20 +993,34 @@ such as migration.
When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the
set of bits that KVM can return in struct kvm_clock_data's flag member.
-The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned
-value is the exact kvmclock value seen by all VCPUs at the instant
-when KVM_GET_CLOCK was called. If clear, the returned value is simply
-CLOCK_MONOTONIC plus a constant offset; the offset can be modified
-with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock,
-but the exact value read by each VCPU could differ, because the host
-TSC is not stable.
+FLAGS:
+
+KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock
+value seen by all VCPUs at the instant when KVM_GET_CLOCK was called.
+If clear, the returned value is simply CLOCK_MONOTONIC plus a constant
+offset; the offset can be modified with KVM_SET_CLOCK. KVM will try
+to make all VCPUs follow this clock, but the exact value read by each
+VCPU could differ, because the host TSC is not stable.
+
+KVM_CLOCK_REAL_TIME. If set, the `realtime` field in the kvm_clock_data
+structure is populated with the value of the host's real time
+clocksource at the instant when KVM_GET_CLOCK was called. If clear,
+the `realtime` field does not contain a value.
+
+KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data
+structure is populated with the value of the host's timestamp counter (TSC)
+at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field
+does not contain a value.
::
struct kvm_clock_data {
__u64 clock; /* kvmclock current value */
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
@@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter.
In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios
such as migration.
+FLAGS:
+
+KVM_CLOCK_REAL_TIME. If set, KVM will compare the value of the `realtime` field
+with the value of the host's real time clocksource at the instant when
+KVM_SET_CLOCK was called. The difference in elapsed time is added to the final
+kvmclock value that will be provided to guests.
+
::
struct kvm_clock_data {
__u64 clock; /* kvmclock current value */
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 974cbfb1eefe..e527d7259415 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1936,4 +1936,7 @@ int kvm_cpu_dirty_log_size(void);
int alloc_all_memslots_rmaps(struct kvm *kvm);
+#define KVM_CLOCK_VALID_FLAGS \
+ (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
+
#endif /* _ASM_X86_KVM_HOST_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d715ae9f9108..4e803632cdca 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2780,17 +2780,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)
#endif
}
-u64 get_kvmclock_ns(struct kvm *kvm)
+/**
+ * Returns true if realtime and TSC values were written back to the caller.
+ * Returns false if a clock triplet cannot be obtained, such as if the host's
+ * realtime clock is not based on the TSC.
+ */
+static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
+ u64 *realtime_ns, u64 *tsc)
{
struct kvm_arch *ka = &kvm->arch;
struct pvclock_vcpu_time_info hv_clock;
unsigned long flags;
- u64 ret;
+ bool ret = false;
spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
if (!ka->use_master_clock) {
spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
- return get_kvmclock_base_ns() + ka->kvmclock_offset;
+ *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
+ return false;
}
hv_clock.tsc_timestamp = ka->master_cycle_now;
@@ -2801,18 +2808,36 @@ u64 get_kvmclock_ns(struct kvm *kvm)
get_cpu();
if (__this_cpu_read(cpu_tsc_khz)) {
+ struct timespec64 ts;
+ u64 tsc_val;
+
kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
&hv_clock.tsc_shift,
&hv_clock.tsc_to_system_mul);
- ret = __pvclock_read_cycles(&hv_clock, rdtsc());
+
+ if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
+ *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
+ *tsc = tsc_val;
+ ret = true;
+ }
+
+ *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
} else
- ret = get_kvmclock_base_ns() + ka->kvmclock_offset;
+ *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
put_cpu();
return ret;
}
+u64 get_kvmclock_ns(struct kvm *kvm)
+{
+ u64 kvmclock_ns, realtime_ns, tsc;
+
+ get_kvmclock_and_realtime(kvm, &kvmclock_ns, &realtime_ns, &tsc);
+ return kvmclock_ns;
+}
+
static void kvm_setup_pvclock_page(struct kvm_vcpu *v,
struct gfn_to_hva_cache *cache,
unsigned int offset)
@@ -4031,7 +4056,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
r = KVM_SYNC_X86_VALID_FIELDS;
break;
case KVM_CAP_ADJUST_CLOCK:
- r = KVM_CLOCK_TSC_STABLE;
+ r = KVM_CLOCK_VALID_FLAGS;
break;
case KVM_CAP_X86_DISABLE_EXITS:
r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE |
@@ -5806,6 +5831,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state)
}
#endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */
+static int kvm_vm_ioctl_get_clock(struct kvm *kvm,
+ void __user *argp)
+{
+ struct kvm_clock_data data;
+
+ memset(&data, 0, sizeof(data));
+
+ if (get_kvmclock_and_realtime(kvm, &data.clock, &data.realtime,
+ &data.host_tsc))
+ data.flags |= KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC;
+
+ if (kvm->arch.use_master_clock)
+ data.flags |= KVM_CLOCK_TSC_STABLE;
+
+ if (copy_to_user(argp, &data, sizeof(data)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int kvm_vm_ioctl_set_clock(struct kvm *kvm,
+ void __user *argp)
+{
+ struct kvm_arch *ka = &kvm->arch;
+ struct kvm_clock_data data;
+ u64 now_raw_ns;
+
+ if (copy_from_user(&data, argp, sizeof(data)))
+ return -EFAULT;
+
+ if (data.flags & ~KVM_CLOCK_REAL_TIME)
+ return -EINVAL;
+
+ /*
+ * TODO: userspace has to take care of races with VCPU_RUN, so
+ * kvm_gen_update_masterclock() can be cut down to locked
+ * pvclock_update_vm_gtod_copy().
+ */
+ kvm_gen_update_masterclock(kvm);
+
+ spin_lock_irq(&ka->pvclock_gtod_sync_lock);
+ if (data.flags & KVM_CLOCK_REAL_TIME) {
+ u64 now_real_ns = ktime_get_real_ns();
+
+ /*
+ * Avoid stepping the kvmclock backwards.
+ */
+ if (now_real_ns > data.realtime)
+ data.clock += now_real_ns - data.realtime;
+ }
+
+ if (ka->use_master_clock)
+ now_raw_ns = ka->master_kernel_ns;
+ else
+ now_raw_ns = get_kvmclock_base_ns();
+ ka->kvmclock_offset = data.clock - now_raw_ns;
+ spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
+
+ kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
+ return 0;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -6050,57 +6137,11 @@ long kvm_arch_vm_ioctl(struct file *filp,
}
#endif
case KVM_SET_CLOCK: {
- struct kvm_arch *ka = &kvm->arch;
- struct kvm_clock_data user_ns;
- u64 now_ns;
-
- r = -EFAULT;
- if (copy_from_user(&user_ns, argp, sizeof(user_ns)))
- goto out;
-
- r = -EINVAL;
- if (user_ns.flags)
- goto out;
-
- r = 0;
- /*
- * TODO: userspace has to take care of races with VCPU_RUN, so
- * kvm_gen_update_masterclock() can be cut down to locked
- * pvclock_update_vm_gtod_copy().
- */
- kvm_gen_update_masterclock(kvm);
-
- /*
- * This pairs with kvm_guest_time_update(): when masterclock is
- * in use, we use master_kernel_ns + kvmclock_offset to set
- * unsigned 'system_time' so if we use get_kvmclock_ns() (which
- * is slightly ahead) here we risk going negative on unsigned
- * 'system_time' when 'user_ns.clock' is very small.
- */
- spin_lock_irq(&ka->pvclock_gtod_sync_lock);
- if (kvm->arch.use_master_clock)
- now_ns = ka->master_kernel_ns;
- else
- now_ns = get_kvmclock_base_ns();
- ka->kvmclock_offset = user_ns.clock - now_ns;
- spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
-
- kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
+ r = kvm_vm_ioctl_set_clock(kvm, argp);
break;
}
case KVM_GET_CLOCK: {
- struct kvm_clock_data user_ns;
- u64 now_ns;
-
- now_ns = get_kvmclock_ns(kvm);
- user_ns.clock = now_ns;
- user_ns.flags = kvm->arch.use_master_clock ? KVM_CLOCK_TSC_STABLE : 0;
- memset(&user_ns.pad, 0, sizeof(user_ns.pad));
-
- r = -EFAULT;
- if (copy_to_user(argp, &user_ns, sizeof(user_ns)))
- goto out;
- r = 0;
+ r = kvm_vm_ioctl_get_clock(kvm, argp);
break;
}
case KVM_MEMORY_ENCRYPT_OP: {
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index d9e4aabcb31a..f2da6760b606 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1223,11 +1223,16 @@ struct kvm_irqfd {
/* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */
#define KVM_CLOCK_TSC_STABLE 2
+#define KVM_CLOCK_REAL_TIME (1 << 2)
+#define KVM_CLOCK_HOST_TSC (1 << 3)
struct kvm_clock_data {
__u64 clock;
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
/* For KVM_CAP_SW_TLB */
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Handling the migration of TSCs correctly is difficult, in part because
Linux does not provide userspace with the ability to retrieve a (TSC,
realtime) clock pair for a single instant in time. In lieu of a more
convenient facility, KVM can report similar information in the kvm_clock
structure.
Provide userspace with a host TSC & realtime pair iff the realtime clock
is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid
realtime value, advance the KVM clock by the amount of elapsed time. Do
not step the KVM clock backwards, though, as it is a monotonic
oscillator.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
Documentation/virt/kvm/api.rst | 42 +++++++--
arch/x86/include/asm/kvm_host.h | 3 +
arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------
include/uapi/linux/kvm.h | 7 +-
4 files changed, 137 insertions(+), 64 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index b9ddce5638f5..26bb01a6e82e 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -993,20 +993,34 @@ such as migration.
When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the
set of bits that KVM can return in struct kvm_clock_data's flag member.
-The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned
-value is the exact kvmclock value seen by all VCPUs at the instant
-when KVM_GET_CLOCK was called. If clear, the returned value is simply
-CLOCK_MONOTONIC plus a constant offset; the offset can be modified
-with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock,
-but the exact value read by each VCPU could differ, because the host
-TSC is not stable.
+FLAGS:
+
+KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock
+value seen by all VCPUs at the instant when KVM_GET_CLOCK was called.
+If clear, the returned value is simply CLOCK_MONOTONIC plus a constant
+offset; the offset can be modified with KVM_SET_CLOCK. KVM will try
+to make all VCPUs follow this clock, but the exact value read by each
+VCPU could differ, because the host TSC is not stable.
+
+KVM_CLOCK_REAL_TIME. If set, the `realtime` field in the kvm_clock_data
+structure is populated with the value of the host's real time
+clocksource at the instant when KVM_GET_CLOCK was called. If clear,
+the `realtime` field does not contain a value.
+
+KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data
+structure is populated with the value of the host's timestamp counter (TSC)
+at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field
+does not contain a value.
::
struct kvm_clock_data {
__u64 clock; /* kvmclock current value */
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
@@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter.
In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios
such as migration.
+FLAGS:
+
+KVM_CLOCK_REAL_TIME. If set, KVM will compare the value of the `realtime` field
+with the value of the host's real time clocksource at the instant when
+KVM_SET_CLOCK was called. The difference in elapsed time is added to the final
+kvmclock value that will be provided to guests.
+
::
struct kvm_clock_data {
__u64 clock; /* kvmclock current value */
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 974cbfb1eefe..e527d7259415 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1936,4 +1936,7 @@ int kvm_cpu_dirty_log_size(void);
int alloc_all_memslots_rmaps(struct kvm *kvm);
+#define KVM_CLOCK_VALID_FLAGS \
+ (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
+
#endif /* _ASM_X86_KVM_HOST_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d715ae9f9108..4e803632cdca 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2780,17 +2780,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)
#endif
}
-u64 get_kvmclock_ns(struct kvm *kvm)
+/**
+ * Returns true if realtime and TSC values were written back to the caller.
+ * Returns false if a clock triplet cannot be obtained, such as if the host's
+ * realtime clock is not based on the TSC.
+ */
+static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
+ u64 *realtime_ns, u64 *tsc)
{
struct kvm_arch *ka = &kvm->arch;
struct pvclock_vcpu_time_info hv_clock;
unsigned long flags;
- u64 ret;
+ bool ret = false;
spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
if (!ka->use_master_clock) {
spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
- return get_kvmclock_base_ns() + ka->kvmclock_offset;
+ *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
+ return false;
}
hv_clock.tsc_timestamp = ka->master_cycle_now;
@@ -2801,18 +2808,36 @@ u64 get_kvmclock_ns(struct kvm *kvm)
get_cpu();
if (__this_cpu_read(cpu_tsc_khz)) {
+ struct timespec64 ts;
+ u64 tsc_val;
+
kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
&hv_clock.tsc_shift,
&hv_clock.tsc_to_system_mul);
- ret = __pvclock_read_cycles(&hv_clock, rdtsc());
+
+ if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
+ *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
+ *tsc = tsc_val;
+ ret = true;
+ }
+
+ *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
} else
- ret = get_kvmclock_base_ns() + ka->kvmclock_offset;
+ *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
put_cpu();
return ret;
}
+u64 get_kvmclock_ns(struct kvm *kvm)
+{
+ u64 kvmclock_ns, realtime_ns, tsc;
+
+ get_kvmclock_and_realtime(kvm, &kvmclock_ns, &realtime_ns, &tsc);
+ return kvmclock_ns;
+}
+
static void kvm_setup_pvclock_page(struct kvm_vcpu *v,
struct gfn_to_hva_cache *cache,
unsigned int offset)
@@ -4031,7 +4056,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
r = KVM_SYNC_X86_VALID_FIELDS;
break;
case KVM_CAP_ADJUST_CLOCK:
- r = KVM_CLOCK_TSC_STABLE;
+ r = KVM_CLOCK_VALID_FLAGS;
break;
case KVM_CAP_X86_DISABLE_EXITS:
r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE |
@@ -5806,6 +5831,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state)
}
#endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */
+static int kvm_vm_ioctl_get_clock(struct kvm *kvm,
+ void __user *argp)
+{
+ struct kvm_clock_data data;
+
+ memset(&data, 0, sizeof(data));
+
+ if (get_kvmclock_and_realtime(kvm, &data.clock, &data.realtime,
+ &data.host_tsc))
+ data.flags |= KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC;
+
+ if (kvm->arch.use_master_clock)
+ data.flags |= KVM_CLOCK_TSC_STABLE;
+
+ if (copy_to_user(argp, &data, sizeof(data)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int kvm_vm_ioctl_set_clock(struct kvm *kvm,
+ void __user *argp)
+{
+ struct kvm_arch *ka = &kvm->arch;
+ struct kvm_clock_data data;
+ u64 now_raw_ns;
+
+ if (copy_from_user(&data, argp, sizeof(data)))
+ return -EFAULT;
+
+ if (data.flags & ~KVM_CLOCK_REAL_TIME)
+ return -EINVAL;
+
+ /*
+ * TODO: userspace has to take care of races with VCPU_RUN, so
+ * kvm_gen_update_masterclock() can be cut down to locked
+ * pvclock_update_vm_gtod_copy().
+ */
+ kvm_gen_update_masterclock(kvm);
+
+ spin_lock_irq(&ka->pvclock_gtod_sync_lock);
+ if (data.flags & KVM_CLOCK_REAL_TIME) {
+ u64 now_real_ns = ktime_get_real_ns();
+
+ /*
+ * Avoid stepping the kvmclock backwards.
+ */
+ if (now_real_ns > data.realtime)
+ data.clock += now_real_ns - data.realtime;
+ }
+
+ if (ka->use_master_clock)
+ now_raw_ns = ka->master_kernel_ns;
+ else
+ now_raw_ns = get_kvmclock_base_ns();
+ ka->kvmclock_offset = data.clock - now_raw_ns;
+ spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
+
+ kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
+ return 0;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -6050,57 +6137,11 @@ long kvm_arch_vm_ioctl(struct file *filp,
}
#endif
case KVM_SET_CLOCK: {
- struct kvm_arch *ka = &kvm->arch;
- struct kvm_clock_data user_ns;
- u64 now_ns;
-
- r = -EFAULT;
- if (copy_from_user(&user_ns, argp, sizeof(user_ns)))
- goto out;
-
- r = -EINVAL;
- if (user_ns.flags)
- goto out;
-
- r = 0;
- /*
- * TODO: userspace has to take care of races with VCPU_RUN, so
- * kvm_gen_update_masterclock() can be cut down to locked
- * pvclock_update_vm_gtod_copy().
- */
- kvm_gen_update_masterclock(kvm);
-
- /*
- * This pairs with kvm_guest_time_update(): when masterclock is
- * in use, we use master_kernel_ns + kvmclock_offset to set
- * unsigned 'system_time' so if we use get_kvmclock_ns() (which
- * is slightly ahead) here we risk going negative on unsigned
- * 'system_time' when 'user_ns.clock' is very small.
- */
- spin_lock_irq(&ka->pvclock_gtod_sync_lock);
- if (kvm->arch.use_master_clock)
- now_ns = ka->master_kernel_ns;
- else
- now_ns = get_kvmclock_base_ns();
- ka->kvmclock_offset = user_ns.clock - now_ns;
- spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
-
- kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
+ r = kvm_vm_ioctl_set_clock(kvm, argp);
break;
}
case KVM_GET_CLOCK: {
- struct kvm_clock_data user_ns;
- u64 now_ns;
-
- now_ns = get_kvmclock_ns(kvm);
- user_ns.clock = now_ns;
- user_ns.flags = kvm->arch.use_master_clock ? KVM_CLOCK_TSC_STABLE : 0;
- memset(&user_ns.pad, 0, sizeof(user_ns.pad));
-
- r = -EFAULT;
- if (copy_to_user(argp, &user_ns, sizeof(user_ns)))
- goto out;
- r = 0;
+ r = kvm_vm_ioctl_get_clock(kvm, argp);
break;
}
case KVM_MEMORY_ENCRYPT_OP: {
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index d9e4aabcb31a..f2da6760b606 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1223,11 +1223,16 @@ struct kvm_irqfd {
/* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */
#define KVM_CLOCK_TSC_STABLE 2
+#define KVM_CLOCK_REAL_TIME (1 << 2)
+#define KVM_CLOCK_HOST_TSC (1 << 3)
struct kvm_clock_data {
__u64 clock;
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
/* For KVM_CAP_SW_TLB */
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Handling the migration of TSCs correctly is difficult, in part because
Linux does not provide userspace with the ability to retrieve a (TSC,
realtime) clock pair for a single instant in time. In lieu of a more
convenient facility, KVM can report similar information in the kvm_clock
structure.
Provide userspace with a host TSC & realtime pair iff the realtime clock
is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid
realtime value, advance the KVM clock by the amount of elapsed time. Do
not step the KVM clock backwards, though, as it is a monotonic
oscillator.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
Documentation/virt/kvm/api.rst | 42 +++++++--
arch/x86/include/asm/kvm_host.h | 3 +
arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------
include/uapi/linux/kvm.h | 7 +-
4 files changed, 137 insertions(+), 64 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index b9ddce5638f5..26bb01a6e82e 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -993,20 +993,34 @@ such as migration.
When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the
set of bits that KVM can return in struct kvm_clock_data's flag member.
-The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned
-value is the exact kvmclock value seen by all VCPUs at the instant
-when KVM_GET_CLOCK was called. If clear, the returned value is simply
-CLOCK_MONOTONIC plus a constant offset; the offset can be modified
-with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock,
-but the exact value read by each VCPU could differ, because the host
-TSC is not stable.
+FLAGS:
+
+KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock
+value seen by all VCPUs at the instant when KVM_GET_CLOCK was called.
+If clear, the returned value is simply CLOCK_MONOTONIC plus a constant
+offset; the offset can be modified with KVM_SET_CLOCK. KVM will try
+to make all VCPUs follow this clock, but the exact value read by each
+VCPU could differ, because the host TSC is not stable.
+
+KVM_CLOCK_REAL_TIME. If set, the `realtime` field in the kvm_clock_data
+structure is populated with the value of the host's real time
+clocksource at the instant when KVM_GET_CLOCK was called. If clear,
+the `realtime` field does not contain a value.
+
+KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data
+structure is populated with the value of the host's timestamp counter (TSC)
+at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field
+does not contain a value.
::
struct kvm_clock_data {
__u64 clock; /* kvmclock current value */
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
@@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter.
In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios
such as migration.
+FLAGS:
+
+KVM_CLOCK_REAL_TIME. If set, KVM will compare the value of the `realtime` field
+with the value of the host's real time clocksource at the instant when
+KVM_SET_CLOCK was called. The difference in elapsed time is added to the final
+kvmclock value that will be provided to guests.
+
::
struct kvm_clock_data {
__u64 clock; /* kvmclock current value */
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 974cbfb1eefe..e527d7259415 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1936,4 +1936,7 @@ int kvm_cpu_dirty_log_size(void);
int alloc_all_memslots_rmaps(struct kvm *kvm);
+#define KVM_CLOCK_VALID_FLAGS \
+ (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
+
#endif /* _ASM_X86_KVM_HOST_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d715ae9f9108..4e803632cdca 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2780,17 +2780,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)
#endif
}
-u64 get_kvmclock_ns(struct kvm *kvm)
+/**
+ * Returns true if realtime and TSC values were written back to the caller.
+ * Returns false if a clock triplet cannot be obtained, such as if the host's
+ * realtime clock is not based on the TSC.
+ */
+static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
+ u64 *realtime_ns, u64 *tsc)
{
struct kvm_arch *ka = &kvm->arch;
struct pvclock_vcpu_time_info hv_clock;
unsigned long flags;
- u64 ret;
+ bool ret = false;
spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
if (!ka->use_master_clock) {
spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
- return get_kvmclock_base_ns() + ka->kvmclock_offset;
+ *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
+ return false;
}
hv_clock.tsc_timestamp = ka->master_cycle_now;
@@ -2801,18 +2808,36 @@ u64 get_kvmclock_ns(struct kvm *kvm)
get_cpu();
if (__this_cpu_read(cpu_tsc_khz)) {
+ struct timespec64 ts;
+ u64 tsc_val;
+
kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
&hv_clock.tsc_shift,
&hv_clock.tsc_to_system_mul);
- ret = __pvclock_read_cycles(&hv_clock, rdtsc());
+
+ if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
+ *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
+ *tsc = tsc_val;
+ ret = true;
+ }
+
+ *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
} else
- ret = get_kvmclock_base_ns() + ka->kvmclock_offset;
+ *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
put_cpu();
return ret;
}
+u64 get_kvmclock_ns(struct kvm *kvm)
+{
+ u64 kvmclock_ns, realtime_ns, tsc;
+
+ get_kvmclock_and_realtime(kvm, &kvmclock_ns, &realtime_ns, &tsc);
+ return kvmclock_ns;
+}
+
static void kvm_setup_pvclock_page(struct kvm_vcpu *v,
struct gfn_to_hva_cache *cache,
unsigned int offset)
@@ -4031,7 +4056,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
r = KVM_SYNC_X86_VALID_FIELDS;
break;
case KVM_CAP_ADJUST_CLOCK:
- r = KVM_CLOCK_TSC_STABLE;
+ r = KVM_CLOCK_VALID_FLAGS;
break;
case KVM_CAP_X86_DISABLE_EXITS:
r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE |
@@ -5806,6 +5831,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state)
}
#endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */
+static int kvm_vm_ioctl_get_clock(struct kvm *kvm,
+ void __user *argp)
+{
+ struct kvm_clock_data data;
+
+ memset(&data, 0, sizeof(data));
+
+ if (get_kvmclock_and_realtime(kvm, &data.clock, &data.realtime,
+ &data.host_tsc))
+ data.flags |= KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC;
+
+ if (kvm->arch.use_master_clock)
+ data.flags |= KVM_CLOCK_TSC_STABLE;
+
+ if (copy_to_user(argp, &data, sizeof(data)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int kvm_vm_ioctl_set_clock(struct kvm *kvm,
+ void __user *argp)
+{
+ struct kvm_arch *ka = &kvm->arch;
+ struct kvm_clock_data data;
+ u64 now_raw_ns;
+
+ if (copy_from_user(&data, argp, sizeof(data)))
+ return -EFAULT;
+
+ if (data.flags & ~KVM_CLOCK_REAL_TIME)
+ return -EINVAL;
+
+ /*
+ * TODO: userspace has to take care of races with VCPU_RUN, so
+ * kvm_gen_update_masterclock() can be cut down to locked
+ * pvclock_update_vm_gtod_copy().
+ */
+ kvm_gen_update_masterclock(kvm);
+
+ spin_lock_irq(&ka->pvclock_gtod_sync_lock);
+ if (data.flags & KVM_CLOCK_REAL_TIME) {
+ u64 now_real_ns = ktime_get_real_ns();
+
+ /*
+ * Avoid stepping the kvmclock backwards.
+ */
+ if (now_real_ns > data.realtime)
+ data.clock += now_real_ns - data.realtime;
+ }
+
+ if (ka->use_master_clock)
+ now_raw_ns = ka->master_kernel_ns;
+ else
+ now_raw_ns = get_kvmclock_base_ns();
+ ka->kvmclock_offset = data.clock - now_raw_ns;
+ spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
+
+ kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
+ return 0;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -6050,57 +6137,11 @@ long kvm_arch_vm_ioctl(struct file *filp,
}
#endif
case KVM_SET_CLOCK: {
- struct kvm_arch *ka = &kvm->arch;
- struct kvm_clock_data user_ns;
- u64 now_ns;
-
- r = -EFAULT;
- if (copy_from_user(&user_ns, argp, sizeof(user_ns)))
- goto out;
-
- r = -EINVAL;
- if (user_ns.flags)
- goto out;
-
- r = 0;
- /*
- * TODO: userspace has to take care of races with VCPU_RUN, so
- * kvm_gen_update_masterclock() can be cut down to locked
- * pvclock_update_vm_gtod_copy().
- */
- kvm_gen_update_masterclock(kvm);
-
- /*
- * This pairs with kvm_guest_time_update(): when masterclock is
- * in use, we use master_kernel_ns + kvmclock_offset to set
- * unsigned 'system_time' so if we use get_kvmclock_ns() (which
- * is slightly ahead) here we risk going negative on unsigned
- * 'system_time' when 'user_ns.clock' is very small.
- */
- spin_lock_irq(&ka->pvclock_gtod_sync_lock);
- if (kvm->arch.use_master_clock)
- now_ns = ka->master_kernel_ns;
- else
- now_ns = get_kvmclock_base_ns();
- ka->kvmclock_offset = user_ns.clock - now_ns;
- spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
-
- kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
+ r = kvm_vm_ioctl_set_clock(kvm, argp);
break;
}
case KVM_GET_CLOCK: {
- struct kvm_clock_data user_ns;
- u64 now_ns;
-
- now_ns = get_kvmclock_ns(kvm);
- user_ns.clock = now_ns;
- user_ns.flags = kvm->arch.use_master_clock ? KVM_CLOCK_TSC_STABLE : 0;
- memset(&user_ns.pad, 0, sizeof(user_ns.pad));
-
- r = -EFAULT;
- if (copy_to_user(argp, &user_ns, sizeof(user_ns)))
- goto out;
- r = 0;
+ r = kvm_vm_ioctl_get_clock(kvm, argp);
break;
}
case KVM_MEMORY_ENCRYPT_OP: {
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index d9e4aabcb31a..f2da6760b606 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1223,11 +1223,16 @@ struct kvm_irqfd {
/* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */
#define KVM_CLOCK_TSC_STABLE 2
+#define KVM_CLOCK_REAL_TIME (1 << 2)
+#define KVM_CLOCK_HOST_TSC (1 << 3)
struct kvm_clock_data {
__u64 clock;
__u32 flags;
- __u32 pad[9];
+ __u32 pad0;
+ __u64 realtime;
+ __u64 host_tsc;
+ __u32 pad[4];
};
/* For KVM_CAP_SW_TLB */
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-18 20:02 ` kernel test robot
-1 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 20:02 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: clang-built-linux, kbuild-all, Paolo Bonzini,
Sean Christopherson, Marc Zyngier, Peter Shier, Jim Mattson,
David Matlack, Ricardo Koller, Jing Zhang
[-- Attachment #1: Type: text/plain, Size: 3884 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: x86_64-randconfig-a014-20210718 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 5d5b08761f944d5b9822d582378333cc4b36a0a7)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/x86/kvm/x86.c:2784: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Returns true if realtime and TSC values were written back to the caller.
vim +2784 arch/x86/kvm/x86.c
2782
2783 /**
> 2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 35504 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-18 20:02 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 20:02 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3979 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: x86_64-randconfig-a014-20210718 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 5d5b08761f944d5b9822d582378333cc4b36a0a7)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/x86/kvm/x86.c:2784: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Returns true if realtime and TSC values were written back to the caller.
vim +2784 arch/x86/kvm/x86.c
2782
2783 /**
> 2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 35504 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-18 20:02 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 20:02 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Sean Christopherson, Peter Shier, clang-built-linux,
Marc Zyngier, David Matlack, Paolo Bonzini, Jim Mattson
[-- Attachment #1: Type: text/plain, Size: 3884 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: x86_64-randconfig-a014-20210718 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 5d5b08761f944d5b9822d582378333cc4b36a0a7)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/x86/kvm/x86.c:2784: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Returns true if realtime and TSC values were written back to the caller.
vim +2784 arch/x86/kvm/x86.c
2782
2783 /**
> 2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 35504 bytes --]
[-- Attachment #3: Type: text/plain, Size: 151 bytes --]
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-18 22:30 ` kernel test robot
-1 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 22:30 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Paolo Bonzini, Sean Christopherson, Marc Zyngier,
Peter Shier, Jim Mattson, David Matlack, Ricardo Koller,
Jing Zhang
[-- Attachment #1: Type: text/plain, Size: 3631 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: x86_64-rhel-8.3-kselftests (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce:
# apt-get install sparse
# sparse version: v0.6.3-341-g8af24329-dirty
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/x86/kvm/x86.c:2784: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Returns true if realtime and TSC values were written back to the caller.
vim +2784 arch/x86/kvm/x86.c
2782
2783 /**
> 2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41845 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-18 22:30 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 22:30 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3724 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: x86_64-rhel-8.3-kselftests (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce:
# apt-get install sparse
# sparse version: v0.6.3-341-g8af24329-dirty
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/x86/kvm/x86.c:2784: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Returns true if realtime and TSC values were written back to the caller.
vim +2784 arch/x86/kvm/x86.c
2782
2783 /**
> 2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 41845 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-18 22:30 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 22:30 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Marc Zyngier, Peter Shier, Sean Christopherson,
David Matlack, Paolo Bonzini, Jim Mattson
[-- Attachment #1: Type: text/plain, Size: 3631 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: x86_64-rhel-8.3-kselftests (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce:
# apt-get install sparse
# sparse version: v0.6.3-341-g8af24329-dirty
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/x86/kvm/x86.c:2784: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Returns true if realtime and TSC values were written back to the caller.
vim +2784 arch/x86/kvm/x86.c
2782
2783 /**
> 2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41845 bytes --]
[-- Attachment #3: Type: text/plain, Size: 151 bytes --]
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-19 0:48 ` kernel test robot
-1 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-19 0:48 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Paolo Bonzini, Sean Christopherson, Marc Zyngier,
Peter Shier, Jim Mattson, David Matlack, Ricardo Koller,
Jing Zhang
[-- Attachment #1: Type: text/plain, Size: 3685 bytes --]
Hi Oliver,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on kvm/queue]
[also build test ERROR on vhost/linux-next v5.14-rc2 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: i386-randconfig-a013-20210718 (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
arch/x86/kvm/x86.c: In function 'get_kvmclock_and_realtime':
>> arch/x86/kvm/x86.c:2818:7: error: implicit declaration of function 'kvm_get_walltime_and_clockread' [-Werror=implicit-function-declaration]
2818 | if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/kvm_get_walltime_and_clockread +2818 arch/x86/kvm/x86.c
2782
2783 /**
2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
> 2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36661 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-19 0:48 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-19 0:48 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3779 bytes --]
Hi Oliver,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on kvm/queue]
[also build test ERROR on vhost/linux-next v5.14-rc2 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: i386-randconfig-a013-20210718 (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
arch/x86/kvm/x86.c: In function 'get_kvmclock_and_realtime':
>> arch/x86/kvm/x86.c:2818:7: error: implicit declaration of function 'kvm_get_walltime_and_clockread' [-Werror=implicit-function-declaration]
2818 | if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/kvm_get_walltime_and_clockread +2818 arch/x86/kvm/x86.c
2782
2783 /**
2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
> 2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36661 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-19 0:48 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-19 0:48 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Marc Zyngier, Peter Shier, Sean Christopherson,
David Matlack, Paolo Bonzini, Jim Mattson
[-- Attachment #1: Type: text/plain, Size: 3685 bytes --]
Hi Oliver,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on kvm/queue]
[also build test ERROR on vhost/linux-next v5.14-rc2 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: i386-randconfig-a013-20210718 (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/72b6d584e6ac692038a7d70e8782f0dfa179e5fb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 72b6d584e6ac692038a7d70e8782f0dfa179e5fb
# save the attached .config to linux build tree
make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
arch/x86/kvm/x86.c: In function 'get_kvmclock_and_realtime':
>> arch/x86/kvm/x86.c:2818:7: error: implicit declaration of function 'kvm_get_walltime_and_clockread' [-Werror=implicit-function-declaration]
2818 | if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/kvm_get_walltime_and_clockread +2818 arch/x86/kvm/x86.c
2782
2783 /**
2784 * Returns true if realtime and TSC values were written back to the caller.
2785 * Returns false if a clock triplet cannot be obtained, such as if the host's
2786 * realtime clock is not based on the TSC.
2787 */
2788 static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
2789 u64 *realtime_ns, u64 *tsc)
2790 {
2791 struct kvm_arch *ka = &kvm->arch;
2792 struct pvclock_vcpu_time_info hv_clock;
2793 unsigned long flags;
2794 bool ret = false;
2795
2796 spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
2797 if (!ka->use_master_clock) {
2798 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2799 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2800 return false;
2801 }
2802
2803 hv_clock.tsc_timestamp = ka->master_cycle_now;
2804 hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
2805 spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
2806
2807 /* both __this_cpu_read() and rdtsc() should be on the same cpu */
2808 get_cpu();
2809
2810 if (__this_cpu_read(cpu_tsc_khz)) {
2811 struct timespec64 ts;
2812 u64 tsc_val;
2813
2814 kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
2815 &hv_clock.tsc_shift,
2816 &hv_clock.tsc_to_system_mul);
2817
> 2818 if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
2819 *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
2820 *tsc = tsc_val;
2821 ret = true;
2822 }
2823
2824 *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val);
2825 } else
2826 *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset;
2827
2828 put_cpu();
2829
2830 return ret;
2831 }
2832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36661 bytes --]
[-- Attachment #3: Type: text/plain, Size: 151 bytes --]
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 02/12] KVM: x86: Refactor tsc synchronization code
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Refactor kvm_synchronize_tsc to make a new function that allows callers
to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly
for the sake of participating in TSC synchronization.
This changes the locking semantics around TSC writes. Writes to the TSC
will now take the pvclock gtod lock while holding the tsc write lock,
whereas before these locks were disjoint.
Reviewed-by: David Matlack <dmatlack@google.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
Documentation/virt/kvm/locking.rst | 11 +++
arch/x86/kvm/x86.c | 106 +++++++++++++++++------------
2 files changed, 74 insertions(+), 43 deletions(-)
diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst
index 35eca377543d..ac62e1c76694 100644
--- a/Documentation/virt/kvm/locking.rst
+++ b/Documentation/virt/kvm/locking.rst
@@ -30,6 +30,9 @@ On x86:
holding kvm->arch.mmu_lock (typically with ``read_lock``, otherwise
there's no need to take kvm->arch.tdp_mmu_pages_lock at all).
+- kvm->arch.tsc_write_lock is taken outside
+ kvm->arch.pvclock_gtod_sync_lock
+
Everything else is a leaf: no other lock is taken inside the critical
sections.
@@ -216,6 +219,14 @@ time it will be set using the Dirty tracking mechanism described above.
:Comment: 'raw' because hardware enabling/disabling must be atomic /wrt
migration.
+:Name: kvm_arch::pvclock_gtod_sync_lock
+:Type: raw_spinlock_t
+:Arch: x86
+:Protects: kvm_arch::{cur_tsc_generation,cur_tsc_nsec,cur_tsc_write,
+ cur_tsc_offset,nr_vcpus_matched_tsc}
+:Comment: 'raw' because updating the kvm master clock must not be
+ preempted.
+
:Name: kvm_arch::tsc_write_lock
:Type: raw_spinlock
:Arch: x86
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 4e803632cdca..e1b7c8b67428 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2441,13 +2441,73 @@ static inline bool kvm_check_tsc_unstable(void)
return check_tsc_unstable();
}
+/*
+ * Infers attempts to synchronize the guest's tsc from host writes. Sets the
+ * offset for the vcpu and tracks the TSC matching generation that the vcpu
+ * participates in.
+ *
+ * Must hold kvm->arch.tsc_write_lock to call this function.
+ */
+static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
+ u64 ns, bool matched)
+{
+ struct kvm *kvm = vcpu->kvm;
+ bool already_matched;
+ unsigned long flags;
+
+ lockdep_assert_held(&kvm->arch.tsc_write_lock);
+
+ already_matched =
+ (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation);
+
+ /*
+ * We track the most recent recorded KHZ, write and time to
+ * allow the matching interval to be extended at each write.
+ */
+ kvm->arch.last_tsc_nsec = ns;
+ kvm->arch.last_tsc_write = tsc;
+ kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
+
+ vcpu->arch.last_guest_tsc = tsc;
+
+ /* Keep track of which generation this VCPU has synchronized to */
+ vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation;
+ vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec;
+ vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write;
+
+ kvm_vcpu_write_tsc_offset(vcpu, offset);
+
+ spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
+ if (!matched) {
+ /*
+ * We split periods of matched TSC writes into generations.
+ * For each generation, we track the original measured
+ * nanosecond time, offset, and write, so if TSCs are in
+ * sync, we can match exact offset, and if not, we can match
+ * exact software computation in compute_guest_tsc()
+ *
+ * These values are tracked in kvm->arch.cur_xxx variables.
+ */
+ kvm->arch.nr_vcpus_matched_tsc = 0;
+ kvm->arch.cur_tsc_generation++;
+ kvm->arch.cur_tsc_nsec = ns;
+ kvm->arch.cur_tsc_write = tsc;
+ kvm->arch.cur_tsc_offset = offset;
+ matched = false;
+ } else if (!already_matched) {
+ kvm->arch.nr_vcpus_matched_tsc++;
+ }
+
+ kvm_track_tsc_matching(vcpu);
+ spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
+}
+
static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
{
struct kvm *kvm = vcpu->kvm;
u64 offset, ns, elapsed;
unsigned long flags;
- bool matched;
- bool already_matched;
+ bool matched = false;
bool synchronizing = false;
raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
@@ -2493,51 +2553,11 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
offset = kvm_compute_l1_tsc_offset(vcpu, data);
}
matched = true;
- already_matched = (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation);
- } else {
- /*
- * We split periods of matched TSC writes into generations.
- * For each generation, we track the original measured
- * nanosecond time, offset, and write, so if TSCs are in
- * sync, we can match exact offset, and if not, we can match
- * exact software computation in compute_guest_tsc()
- *
- * These values are tracked in kvm->arch.cur_xxx variables.
- */
- kvm->arch.cur_tsc_generation++;
- kvm->arch.cur_tsc_nsec = ns;
- kvm->arch.cur_tsc_write = data;
- kvm->arch.cur_tsc_offset = offset;
- matched = false;
}
- /*
- * We also track th most recent recorded KHZ, write and time to
- * allow the matching interval to be extended at each write.
- */
- kvm->arch.last_tsc_nsec = ns;
- kvm->arch.last_tsc_write = data;
- kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
-
- vcpu->arch.last_guest_tsc = data;
+ __kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
- /* Keep track of which generation this VCPU has synchronized to */
- vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation;
- vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec;
- vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write;
-
- kvm_vcpu_write_tsc_offset(vcpu, offset);
raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
-
- spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
- if (!matched) {
- kvm->arch.nr_vcpus_matched_tsc = 0;
- } else if (!already_matched) {
- kvm->arch.nr_vcpus_matched_tsc++;
- }
-
- kvm_track_tsc_matching(vcpu);
- spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
}
static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu,
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 02/12] KVM: x86: Refactor tsc synchronization code
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Refactor kvm_synchronize_tsc to make a new function that allows callers
to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly
for the sake of participating in TSC synchronization.
This changes the locking semantics around TSC writes. Writes to the TSC
will now take the pvclock gtod lock while holding the tsc write lock,
whereas before these locks were disjoint.
Reviewed-by: David Matlack <dmatlack@google.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
Documentation/virt/kvm/locking.rst | 11 +++
arch/x86/kvm/x86.c | 106 +++++++++++++++++------------
2 files changed, 74 insertions(+), 43 deletions(-)
diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst
index 35eca377543d..ac62e1c76694 100644
--- a/Documentation/virt/kvm/locking.rst
+++ b/Documentation/virt/kvm/locking.rst
@@ -30,6 +30,9 @@ On x86:
holding kvm->arch.mmu_lock (typically with ``read_lock``, otherwise
there's no need to take kvm->arch.tdp_mmu_pages_lock at all).
+- kvm->arch.tsc_write_lock is taken outside
+ kvm->arch.pvclock_gtod_sync_lock
+
Everything else is a leaf: no other lock is taken inside the critical
sections.
@@ -216,6 +219,14 @@ time it will be set using the Dirty tracking mechanism described above.
:Comment: 'raw' because hardware enabling/disabling must be atomic /wrt
migration.
+:Name: kvm_arch::pvclock_gtod_sync_lock
+:Type: raw_spinlock_t
+:Arch: x86
+:Protects: kvm_arch::{cur_tsc_generation,cur_tsc_nsec,cur_tsc_write,
+ cur_tsc_offset,nr_vcpus_matched_tsc}
+:Comment: 'raw' because updating the kvm master clock must not be
+ preempted.
+
:Name: kvm_arch::tsc_write_lock
:Type: raw_spinlock
:Arch: x86
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 4e803632cdca..e1b7c8b67428 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2441,13 +2441,73 @@ static inline bool kvm_check_tsc_unstable(void)
return check_tsc_unstable();
}
+/*
+ * Infers attempts to synchronize the guest's tsc from host writes. Sets the
+ * offset for the vcpu and tracks the TSC matching generation that the vcpu
+ * participates in.
+ *
+ * Must hold kvm->arch.tsc_write_lock to call this function.
+ */
+static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
+ u64 ns, bool matched)
+{
+ struct kvm *kvm = vcpu->kvm;
+ bool already_matched;
+ unsigned long flags;
+
+ lockdep_assert_held(&kvm->arch.tsc_write_lock);
+
+ already_matched =
+ (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation);
+
+ /*
+ * We track the most recent recorded KHZ, write and time to
+ * allow the matching interval to be extended at each write.
+ */
+ kvm->arch.last_tsc_nsec = ns;
+ kvm->arch.last_tsc_write = tsc;
+ kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
+
+ vcpu->arch.last_guest_tsc = tsc;
+
+ /* Keep track of which generation this VCPU has synchronized to */
+ vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation;
+ vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec;
+ vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write;
+
+ kvm_vcpu_write_tsc_offset(vcpu, offset);
+
+ spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
+ if (!matched) {
+ /*
+ * We split periods of matched TSC writes into generations.
+ * For each generation, we track the original measured
+ * nanosecond time, offset, and write, so if TSCs are in
+ * sync, we can match exact offset, and if not, we can match
+ * exact software computation in compute_guest_tsc()
+ *
+ * These values are tracked in kvm->arch.cur_xxx variables.
+ */
+ kvm->arch.nr_vcpus_matched_tsc = 0;
+ kvm->arch.cur_tsc_generation++;
+ kvm->arch.cur_tsc_nsec = ns;
+ kvm->arch.cur_tsc_write = tsc;
+ kvm->arch.cur_tsc_offset = offset;
+ matched = false;
+ } else if (!already_matched) {
+ kvm->arch.nr_vcpus_matched_tsc++;
+ }
+
+ kvm_track_tsc_matching(vcpu);
+ spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
+}
+
static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
{
struct kvm *kvm = vcpu->kvm;
u64 offset, ns, elapsed;
unsigned long flags;
- bool matched;
- bool already_matched;
+ bool matched = false;
bool synchronizing = false;
raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
@@ -2493,51 +2553,11 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
offset = kvm_compute_l1_tsc_offset(vcpu, data);
}
matched = true;
- already_matched = (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation);
- } else {
- /*
- * We split periods of matched TSC writes into generations.
- * For each generation, we track the original measured
- * nanosecond time, offset, and write, so if TSCs are in
- * sync, we can match exact offset, and if not, we can match
- * exact software computation in compute_guest_tsc()
- *
- * These values are tracked in kvm->arch.cur_xxx variables.
- */
- kvm->arch.cur_tsc_generation++;
- kvm->arch.cur_tsc_nsec = ns;
- kvm->arch.cur_tsc_write = data;
- kvm->arch.cur_tsc_offset = offset;
- matched = false;
}
- /*
- * We also track th most recent recorded KHZ, write and time to
- * allow the matching interval to be extended at each write.
- */
- kvm->arch.last_tsc_nsec = ns;
- kvm->arch.last_tsc_write = data;
- kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
-
- vcpu->arch.last_guest_tsc = data;
+ __kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
- /* Keep track of which generation this VCPU has synchronized to */
- vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation;
- vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec;
- vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write;
-
- kvm_vcpu_write_tsc_offset(vcpu, offset);
raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
-
- spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
- if (!matched) {
- kvm->arch.nr_vcpus_matched_tsc = 0;
- } else if (!already_matched) {
- kvm->arch.nr_vcpus_matched_tsc++;
- }
-
- kvm_track_tsc_matching(vcpu);
- spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
}
static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu,
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 02/12] KVM: x86: Refactor tsc synchronization code
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Refactor kvm_synchronize_tsc to make a new function that allows callers
to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly
for the sake of participating in TSC synchronization.
This changes the locking semantics around TSC writes. Writes to the TSC
will now take the pvclock gtod lock while holding the tsc write lock,
whereas before these locks were disjoint.
Reviewed-by: David Matlack <dmatlack@google.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
Documentation/virt/kvm/locking.rst | 11 +++
arch/x86/kvm/x86.c | 106 +++++++++++++++++------------
2 files changed, 74 insertions(+), 43 deletions(-)
diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst
index 35eca377543d..ac62e1c76694 100644
--- a/Documentation/virt/kvm/locking.rst
+++ b/Documentation/virt/kvm/locking.rst
@@ -30,6 +30,9 @@ On x86:
holding kvm->arch.mmu_lock (typically with ``read_lock``, otherwise
there's no need to take kvm->arch.tdp_mmu_pages_lock at all).
+- kvm->arch.tsc_write_lock is taken outside
+ kvm->arch.pvclock_gtod_sync_lock
+
Everything else is a leaf: no other lock is taken inside the critical
sections.
@@ -216,6 +219,14 @@ time it will be set using the Dirty tracking mechanism described above.
:Comment: 'raw' because hardware enabling/disabling must be atomic /wrt
migration.
+:Name: kvm_arch::pvclock_gtod_sync_lock
+:Type: raw_spinlock_t
+:Arch: x86
+:Protects: kvm_arch::{cur_tsc_generation,cur_tsc_nsec,cur_tsc_write,
+ cur_tsc_offset,nr_vcpus_matched_tsc}
+:Comment: 'raw' because updating the kvm master clock must not be
+ preempted.
+
:Name: kvm_arch::tsc_write_lock
:Type: raw_spinlock
:Arch: x86
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 4e803632cdca..e1b7c8b67428 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2441,13 +2441,73 @@ static inline bool kvm_check_tsc_unstable(void)
return check_tsc_unstable();
}
+/*
+ * Infers attempts to synchronize the guest's tsc from host writes. Sets the
+ * offset for the vcpu and tracks the TSC matching generation that the vcpu
+ * participates in.
+ *
+ * Must hold kvm->arch.tsc_write_lock to call this function.
+ */
+static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
+ u64 ns, bool matched)
+{
+ struct kvm *kvm = vcpu->kvm;
+ bool already_matched;
+ unsigned long flags;
+
+ lockdep_assert_held(&kvm->arch.tsc_write_lock);
+
+ already_matched =
+ (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation);
+
+ /*
+ * We track the most recent recorded KHZ, write and time to
+ * allow the matching interval to be extended at each write.
+ */
+ kvm->arch.last_tsc_nsec = ns;
+ kvm->arch.last_tsc_write = tsc;
+ kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
+
+ vcpu->arch.last_guest_tsc = tsc;
+
+ /* Keep track of which generation this VCPU has synchronized to */
+ vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation;
+ vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec;
+ vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write;
+
+ kvm_vcpu_write_tsc_offset(vcpu, offset);
+
+ spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
+ if (!matched) {
+ /*
+ * We split periods of matched TSC writes into generations.
+ * For each generation, we track the original measured
+ * nanosecond time, offset, and write, so if TSCs are in
+ * sync, we can match exact offset, and if not, we can match
+ * exact software computation in compute_guest_tsc()
+ *
+ * These values are tracked in kvm->arch.cur_xxx variables.
+ */
+ kvm->arch.nr_vcpus_matched_tsc = 0;
+ kvm->arch.cur_tsc_generation++;
+ kvm->arch.cur_tsc_nsec = ns;
+ kvm->arch.cur_tsc_write = tsc;
+ kvm->arch.cur_tsc_offset = offset;
+ matched = false;
+ } else if (!already_matched) {
+ kvm->arch.nr_vcpus_matched_tsc++;
+ }
+
+ kvm_track_tsc_matching(vcpu);
+ spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
+}
+
static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
{
struct kvm *kvm = vcpu->kvm;
u64 offset, ns, elapsed;
unsigned long flags;
- bool matched;
- bool already_matched;
+ bool matched = false;
bool synchronizing = false;
raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
@@ -2493,51 +2553,11 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
offset = kvm_compute_l1_tsc_offset(vcpu, data);
}
matched = true;
- already_matched = (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation);
- } else {
- /*
- * We split periods of matched TSC writes into generations.
- * For each generation, we track the original measured
- * nanosecond time, offset, and write, so if TSCs are in
- * sync, we can match exact offset, and if not, we can match
- * exact software computation in compute_guest_tsc()
- *
- * These values are tracked in kvm->arch.cur_xxx variables.
- */
- kvm->arch.cur_tsc_generation++;
- kvm->arch.cur_tsc_nsec = ns;
- kvm->arch.cur_tsc_write = data;
- kvm->arch.cur_tsc_offset = offset;
- matched = false;
}
- /*
- * We also track th most recent recorded KHZ, write and time to
- * allow the matching interval to be extended at each write.
- */
- kvm->arch.last_tsc_nsec = ns;
- kvm->arch.last_tsc_write = data;
- kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
-
- vcpu->arch.last_guest_tsc = data;
+ __kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
- /* Keep track of which generation this VCPU has synchronized to */
- vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation;
- vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec;
- vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write;
-
- kvm_vcpu_write_tsc_offset(vcpu, offset);
raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
-
- spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
- if (!matched) {
- kvm->arch.nr_vcpus_matched_tsc = 0;
- } else if (!already_matched) {
- kvm->arch.nr_vcpus_matched_tsc++;
- }
-
- kvm_track_tsc_matching(vcpu);
- spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
}
static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu,
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton, Oliver Upton
To date, VMM-directed TSC synchronization and migration has been a bit
messy. KVM has some baked-in heuristics around TSC writes to infer if
the VMM is attempting to synchronize. This is problematic, as it depends
on host userspace writing to the guest's TSC within 1 second of the last
write.
A much cleaner approach to configuring the guest's views of the TSC is to
simply migrate the TSC offset for every vCPU. Offsets are idempotent,
and thus not subject to change depending on when the VMM actually
reads/writes values from/to KVM. The VMM can then read the TSC once with
KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when
the guest is paused.
Cc: David Matlack <dmatlack@google.com>
Signed-off-by: Oliver Upton <oupton@gooogle.com>
---
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/include/uapi/asm/kvm.h | 4 +
arch/x86/kvm/x86.c | 166 ++++++++++++++++++++++++++++++++
3 files changed, 171 insertions(+)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index e527d7259415..45134b7b14d6 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1070,6 +1070,7 @@ struct kvm_arch {
u64 last_tsc_nsec;
u64 last_tsc_write;
u32 last_tsc_khz;
+ u64 last_tsc_offset;
u64 cur_tsc_nsec;
u64 cur_tsc_write;
u64 cur_tsc_offset;
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index a6c327f8ad9e..0b22e1e84e78 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {
#define KVM_PMU_EVENT_ALLOW 0
#define KVM_PMU_EVENT_DENY 1
+/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
+#define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
+#define KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
+
#endif /* _ASM_X86_KVM_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e1b7c8b67428..d22de0a1988a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2411,6 +2411,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)
static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset);
}
+static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.l1_tsc_offset;
+}
+
static void kvm_vcpu_write_tsc_multiplier(struct kvm_vcpu *vcpu, u64 l1_multiplier)
{
vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;
@@ -2467,6 +2472,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
kvm->arch.last_tsc_nsec = ns;
kvm->arch.last_tsc_write = tsc;
kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
+ kvm->arch.last_tsc_offset = offset;
vcpu->arch.last_guest_tsc = tsc;
@@ -4914,6 +4920,136 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
return 0;
}
+static int kvm_arch_tsc_has_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET:
+ r = 0;
+ break;
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ void __user *uaddr = (void __user *)attr->addr;
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET: {
+ u64 offset;
+
+ offset = kvm_vcpu_read_tsc_offset(vcpu);
+ r = -EFAULT;
+ if (copy_to_user(uaddr, &offset, sizeof(offset)))
+ break;
+
+ r = 0;
+ }
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_arch_tsc_set_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ void __user *uaddr = (void __user *)attr->addr;
+ struct kvm *kvm = vcpu->kvm;
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET: {
+ u64 offset, tsc, ns;
+ unsigned long flags;
+ bool matched;
+
+ r = -EFAULT;
+ if (copy_from_user(&offset, uaddr, sizeof(offset)))
+ break;
+
+ raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
+
+ matched = (vcpu->arch.virtual_tsc_khz &&
+ kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz &&
+ kvm->arch.last_tsc_offset == offset);
+
+ tsc = kvm_scale_tsc(vcpu, rdtsc(), vcpu->arch.l1_tsc_scaling_ratio) + offset;
+ ns = get_kvmclock_base_ns();
+
+ __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched);
+ raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
+
+ r = 0;
+ break;
+ }
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_has_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_has_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_get_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_get_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_set_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_set_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
struct kvm_enable_cap *cap)
{
@@ -5368,6 +5504,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = __set_sregs2(vcpu, u.sregs2);
break;
}
+ case KVM_HAS_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_has_device_attr(vcpu, &attr);
+ break;
+ }
+ case KVM_GET_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_get_device_attr(vcpu, &attr);
+ break;
+ }
+ case KVM_SET_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_set_device_attr(vcpu, &attr);
+ break;
+ }
default:
r = -EINVAL;
}
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton, Oliver Upton
To date, VMM-directed TSC synchronization and migration has been a bit
messy. KVM has some baked-in heuristics around TSC writes to infer if
the VMM is attempting to synchronize. This is problematic, as it depends
on host userspace writing to the guest's TSC within 1 second of the last
write.
A much cleaner approach to configuring the guest's views of the TSC is to
simply migrate the TSC offset for every vCPU. Offsets are idempotent,
and thus not subject to change depending on when the VMM actually
reads/writes values from/to KVM. The VMM can then read the TSC once with
KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when
the guest is paused.
Cc: David Matlack <dmatlack@google.com>
Signed-off-by: Oliver Upton <oupton@gooogle.com>
---
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/include/uapi/asm/kvm.h | 4 +
arch/x86/kvm/x86.c | 166 ++++++++++++++++++++++++++++++++
3 files changed, 171 insertions(+)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index e527d7259415..45134b7b14d6 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1070,6 +1070,7 @@ struct kvm_arch {
u64 last_tsc_nsec;
u64 last_tsc_write;
u32 last_tsc_khz;
+ u64 last_tsc_offset;
u64 cur_tsc_nsec;
u64 cur_tsc_write;
u64 cur_tsc_offset;
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index a6c327f8ad9e..0b22e1e84e78 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {
#define KVM_PMU_EVENT_ALLOW 0
#define KVM_PMU_EVENT_DENY 1
+/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
+#define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
+#define KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
+
#endif /* _ASM_X86_KVM_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e1b7c8b67428..d22de0a1988a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2411,6 +2411,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)
static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset);
}
+static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.l1_tsc_offset;
+}
+
static void kvm_vcpu_write_tsc_multiplier(struct kvm_vcpu *vcpu, u64 l1_multiplier)
{
vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;
@@ -2467,6 +2472,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
kvm->arch.last_tsc_nsec = ns;
kvm->arch.last_tsc_write = tsc;
kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
+ kvm->arch.last_tsc_offset = offset;
vcpu->arch.last_guest_tsc = tsc;
@@ -4914,6 +4920,136 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
return 0;
}
+static int kvm_arch_tsc_has_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET:
+ r = 0;
+ break;
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ void __user *uaddr = (void __user *)attr->addr;
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET: {
+ u64 offset;
+
+ offset = kvm_vcpu_read_tsc_offset(vcpu);
+ r = -EFAULT;
+ if (copy_to_user(uaddr, &offset, sizeof(offset)))
+ break;
+
+ r = 0;
+ }
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_arch_tsc_set_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ void __user *uaddr = (void __user *)attr->addr;
+ struct kvm *kvm = vcpu->kvm;
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET: {
+ u64 offset, tsc, ns;
+ unsigned long flags;
+ bool matched;
+
+ r = -EFAULT;
+ if (copy_from_user(&offset, uaddr, sizeof(offset)))
+ break;
+
+ raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
+
+ matched = (vcpu->arch.virtual_tsc_khz &&
+ kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz &&
+ kvm->arch.last_tsc_offset == offset);
+
+ tsc = kvm_scale_tsc(vcpu, rdtsc(), vcpu->arch.l1_tsc_scaling_ratio) + offset;
+ ns = get_kvmclock_base_ns();
+
+ __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched);
+ raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
+
+ r = 0;
+ break;
+ }
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_has_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_has_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_get_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_get_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_set_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_set_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
struct kvm_enable_cap *cap)
{
@@ -5368,6 +5504,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = __set_sregs2(vcpu, u.sregs2);
break;
}
+ case KVM_HAS_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_has_device_attr(vcpu, &attr);
+ break;
+ }
+ case KVM_GET_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_get_device_attr(vcpu, &attr);
+ break;
+ }
+ case KVM_SET_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_set_device_attr(vcpu, &attr);
+ break;
+ }
default:
r = -EINVAL;
}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini, Oliver Upton,
linux-arm-kernel, Jim Mattson
To date, VMM-directed TSC synchronization and migration has been a bit
messy. KVM has some baked-in heuristics around TSC writes to infer if
the VMM is attempting to synchronize. This is problematic, as it depends
on host userspace writing to the guest's TSC within 1 second of the last
write.
A much cleaner approach to configuring the guest's views of the TSC is to
simply migrate the TSC offset for every vCPU. Offsets are idempotent,
and thus not subject to change depending on when the VMM actually
reads/writes values from/to KVM. The VMM can then read the TSC once with
KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when
the guest is paused.
Cc: David Matlack <dmatlack@google.com>
Signed-off-by: Oliver Upton <oupton@gooogle.com>
---
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/include/uapi/asm/kvm.h | 4 +
arch/x86/kvm/x86.c | 166 ++++++++++++++++++++++++++++++++
3 files changed, 171 insertions(+)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index e527d7259415..45134b7b14d6 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1070,6 +1070,7 @@ struct kvm_arch {
u64 last_tsc_nsec;
u64 last_tsc_write;
u32 last_tsc_khz;
+ u64 last_tsc_offset;
u64 cur_tsc_nsec;
u64 cur_tsc_write;
u64 cur_tsc_offset;
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index a6c327f8ad9e..0b22e1e84e78 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {
#define KVM_PMU_EVENT_ALLOW 0
#define KVM_PMU_EVENT_DENY 1
+/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
+#define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
+#define KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
+
#endif /* _ASM_X86_KVM_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e1b7c8b67428..d22de0a1988a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2411,6 +2411,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)
static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset);
}
+static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.l1_tsc_offset;
+}
+
static void kvm_vcpu_write_tsc_multiplier(struct kvm_vcpu *vcpu, u64 l1_multiplier)
{
vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;
@@ -2467,6 +2472,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
kvm->arch.last_tsc_nsec = ns;
kvm->arch.last_tsc_write = tsc;
kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
+ kvm->arch.last_tsc_offset = offset;
vcpu->arch.last_guest_tsc = tsc;
@@ -4914,6 +4920,136 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
return 0;
}
+static int kvm_arch_tsc_has_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET:
+ r = 0;
+ break;
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ void __user *uaddr = (void __user *)attr->addr;
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET: {
+ u64 offset;
+
+ offset = kvm_vcpu_read_tsc_offset(vcpu);
+ r = -EFAULT;
+ if (copy_to_user(uaddr, &offset, sizeof(offset)))
+ break;
+
+ r = 0;
+ }
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_arch_tsc_set_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ void __user *uaddr = (void __user *)attr->addr;
+ struct kvm *kvm = vcpu->kvm;
+ int r;
+
+ switch (attr->attr) {
+ case KVM_VCPU_TSC_OFFSET: {
+ u64 offset, tsc, ns;
+ unsigned long flags;
+ bool matched;
+
+ r = -EFAULT;
+ if (copy_from_user(&offset, uaddr, sizeof(offset)))
+ break;
+
+ raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
+
+ matched = (vcpu->arch.virtual_tsc_khz &&
+ kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz &&
+ kvm->arch.last_tsc_offset == offset);
+
+ tsc = kvm_scale_tsc(vcpu, rdtsc(), vcpu->arch.l1_tsc_scaling_ratio) + offset;
+ ns = get_kvmclock_base_ns();
+
+ __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched);
+ raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
+
+ r = 0;
+ break;
+ }
+ default:
+ r = -ENXIO;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_has_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_has_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_get_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_get_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_vcpu_ioctl_set_device_attr(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr)
+{
+ int r;
+
+ switch (attr->group) {
+ case KVM_VCPU_TSC_CTRL:
+ r = kvm_arch_tsc_set_attr(vcpu, attr);
+ break;
+ default:
+ r = -ENXIO;
+ break;
+ }
+
+ return r;
+}
+
static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
struct kvm_enable_cap *cap)
{
@@ -5368,6 +5504,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = __set_sregs2(vcpu, u.sregs2);
break;
}
+ case KVM_HAS_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_has_device_attr(vcpu, &attr);
+ break;
+ }
+ case KVM_GET_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_get_device_attr(vcpu, &attr);
+ break;
+ }
+ case KVM_SET_DEVICE_ATTR: {
+ struct kvm_device_attr attr;
+
+ r = -EFAULT;
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ goto out;
+
+ r = kvm_vcpu_ioctl_set_device_attr(vcpu, &attr);
+ break;
+ }
default:
r = -EINVAL;
}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread* Re: [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-18 20:42 ` Paolo Bonzini
-1 siblings, 0 replies; 73+ messages in thread
From: Paolo Bonzini @ 2021-07-18 20:42 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: Sean Christopherson, Marc Zyngier, Peter Shier, Jim Mattson,
David Matlack, Ricardo Koller, Jing Zhang, Raghavendra Rao Anata,
James Morse, Alexandru Elisei, Suzuki K Poulose,
linux-arm-kernel, Oliver Upton
On 16/07/21 23:26, Oliver Upton wrote:
> To date, VMM-directed TSC synchronization and migration has been a bit
> messy. KVM has some baked-in heuristics around TSC writes to infer if
> the VMM is attempting to synchronize. This is problematic, as it depends
> on host userspace writing to the guest's TSC within 1 second of the last
> write.
>
> A much cleaner approach to configuring the guest's views of the TSC is to
> simply migrate the TSC offset for every vCPU. Offsets are idempotent,
> and thus not subject to change depending on when the VMM actually
> reads/writes values from/to KVM. The VMM can then read the TSC once with
> KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when
> the guest is paused.
>
> Cc: David Matlack <dmatlack@google.com>
> Signed-off-by: Oliver Upton <oupton@gooogle.com>
> ---
> arch/x86/include/asm/kvm_host.h | 1 +
> arch/x86/include/uapi/asm/kvm.h | 4 +
> arch/x86/kvm/x86.c | 166 ++++++++++++++++++++++++++++++++
> 3 files changed, 171 insertions(+)
This is missing documentation. The documentation should also include
the algorithm in https://www.spinics.net/lists/kvm-arm/msg47383.html
(modulo the fact that KVM_GET/SET_CLOCK return or pass realtime_ns
rather than kvmclock_ns - relatime_ns; which is fine of course).
Paolo
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index e527d7259415..45134b7b14d6 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1070,6 +1070,7 @@ struct kvm_arch {
> u64 last_tsc_nsec;
> u64 last_tsc_write;
> u32 last_tsc_khz;
> + u64 last_tsc_offset;
> u64 cur_tsc_nsec;
> u64 cur_tsc_write;
> u64 cur_tsc_offset;
> diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
> index a6c327f8ad9e..0b22e1e84e78 100644
> --- a/arch/x86/include/uapi/asm/kvm.h
> +++ b/arch/x86/include/uapi/asm/kvm.h
> @@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {
> #define KVM_PMU_EVENT_ALLOW 0
> #define KVM_PMU_EVENT_DENY 1
>
> +/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
> +#define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
> +#define KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
> +
> #endif /* _ASM_X86_KVM_H */
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index e1b7c8b67428..d22de0a1988a 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -2411,6 +2411,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)
> static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset);
> }
>
> +static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu)
> +{
> + return vcpu->arch.l1_tsc_offset;
> +}
> +
> static void kvm_vcpu_write_tsc_multiplier(struct kvm_vcpu *vcpu, u64 l1_multiplier)
> {
> vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;
> @@ -2467,6 +2472,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
> kvm->arch.last_tsc_nsec = ns;
> kvm->arch.last_tsc_write = tsc;
> kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
> + kvm->arch.last_tsc_offset = offset;
>
> vcpu->arch.last_guest_tsc = tsc;
>
> @@ -4914,6 +4920,136 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
> return 0;
> }
>
> +static int kvm_arch_tsc_has_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET:
> + r = 0;
> + break;
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + void __user *uaddr = (void __user *)attr->addr;
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET: {
> + u64 offset;
> +
> + offset = kvm_vcpu_read_tsc_offset(vcpu);
> + r = -EFAULT;
> + if (copy_to_user(uaddr, &offset, sizeof(offset)))
> + break;
> +
> + r = 0;
> + }
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_arch_tsc_set_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + void __user *uaddr = (void __user *)attr->addr;
> + struct kvm *kvm = vcpu->kvm;
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET: {
> + u64 offset, tsc, ns;
> + unsigned long flags;
> + bool matched;
> +
> + r = -EFAULT;
> + if (copy_from_user(&offset, uaddr, sizeof(offset)))
> + break;
> +
> + raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
> +
> + matched = (vcpu->arch.virtual_tsc_khz &&
> + kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz &&
> + kvm->arch.last_tsc_offset == offset);
> +
> + tsc = kvm_scale_tsc(vcpu, rdtsc(), vcpu->arch.l1_tsc_scaling_ratio) + offset;
> + ns = get_kvmclock_base_ns();
> +
> + __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched);
> + raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
> +
> + r = 0;
> + break;
> + }
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_has_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_has_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_get_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_get_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_set_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_set_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
> struct kvm_enable_cap *cap)
> {
> @@ -5368,6 +5504,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
> r = __set_sregs2(vcpu, u.sregs2);
> break;
> }
> + case KVM_HAS_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_has_device_attr(vcpu, &attr);
> + break;
> + }
> + case KVM_GET_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_get_device_attr(vcpu, &attr);
> + break;
> + }
> + case KVM_SET_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_set_device_attr(vcpu, &attr);
> + break;
> + }
> default:
> r = -EINVAL;
> }
>
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
@ 2021-07-18 20:42 ` Paolo Bonzini
0 siblings, 0 replies; 73+ messages in thread
From: Paolo Bonzini @ 2021-07-18 20:42 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: Sean Christopherson, Marc Zyngier, Peter Shier, Jim Mattson,
David Matlack, Ricardo Koller, Jing Zhang, Raghavendra Rao Anata,
James Morse, Alexandru Elisei, Suzuki K Poulose,
linux-arm-kernel, Oliver Upton
On 16/07/21 23:26, Oliver Upton wrote:
> To date, VMM-directed TSC synchronization and migration has been a bit
> messy. KVM has some baked-in heuristics around TSC writes to infer if
> the VMM is attempting to synchronize. This is problematic, as it depends
> on host userspace writing to the guest's TSC within 1 second of the last
> write.
>
> A much cleaner approach to configuring the guest's views of the TSC is to
> simply migrate the TSC offset for every vCPU. Offsets are idempotent,
> and thus not subject to change depending on when the VMM actually
> reads/writes values from/to KVM. The VMM can then read the TSC once with
> KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when
> the guest is paused.
>
> Cc: David Matlack <dmatlack@google.com>
> Signed-off-by: Oliver Upton <oupton@gooogle.com>
> ---
> arch/x86/include/asm/kvm_host.h | 1 +
> arch/x86/include/uapi/asm/kvm.h | 4 +
> arch/x86/kvm/x86.c | 166 ++++++++++++++++++++++++++++++++
> 3 files changed, 171 insertions(+)
This is missing documentation. The documentation should also include
the algorithm in https://www.spinics.net/lists/kvm-arm/msg47383.html
(modulo the fact that KVM_GET/SET_CLOCK return or pass realtime_ns
rather than kvmclock_ns - relatime_ns; which is fine of course).
Paolo
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index e527d7259415..45134b7b14d6 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1070,6 +1070,7 @@ struct kvm_arch {
> u64 last_tsc_nsec;
> u64 last_tsc_write;
> u32 last_tsc_khz;
> + u64 last_tsc_offset;
> u64 cur_tsc_nsec;
> u64 cur_tsc_write;
> u64 cur_tsc_offset;
> diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
> index a6c327f8ad9e..0b22e1e84e78 100644
> --- a/arch/x86/include/uapi/asm/kvm.h
> +++ b/arch/x86/include/uapi/asm/kvm.h
> @@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {
> #define KVM_PMU_EVENT_ALLOW 0
> #define KVM_PMU_EVENT_DENY 1
>
> +/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
> +#define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
> +#define KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
> +
> #endif /* _ASM_X86_KVM_H */
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index e1b7c8b67428..d22de0a1988a 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -2411,6 +2411,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)
> static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset);
> }
>
> +static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu)
> +{
> + return vcpu->arch.l1_tsc_offset;
> +}
> +
> static void kvm_vcpu_write_tsc_multiplier(struct kvm_vcpu *vcpu, u64 l1_multiplier)
> {
> vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;
> @@ -2467,6 +2472,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
> kvm->arch.last_tsc_nsec = ns;
> kvm->arch.last_tsc_write = tsc;
> kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
> + kvm->arch.last_tsc_offset = offset;
>
> vcpu->arch.last_guest_tsc = tsc;
>
> @@ -4914,6 +4920,136 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
> return 0;
> }
>
> +static int kvm_arch_tsc_has_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET:
> + r = 0;
> + break;
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + void __user *uaddr = (void __user *)attr->addr;
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET: {
> + u64 offset;
> +
> + offset = kvm_vcpu_read_tsc_offset(vcpu);
> + r = -EFAULT;
> + if (copy_to_user(uaddr, &offset, sizeof(offset)))
> + break;
> +
> + r = 0;
> + }
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_arch_tsc_set_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + void __user *uaddr = (void __user *)attr->addr;
> + struct kvm *kvm = vcpu->kvm;
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET: {
> + u64 offset, tsc, ns;
> + unsigned long flags;
> + bool matched;
> +
> + r = -EFAULT;
> + if (copy_from_user(&offset, uaddr, sizeof(offset)))
> + break;
> +
> + raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
> +
> + matched = (vcpu->arch.virtual_tsc_khz &&
> + kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz &&
> + kvm->arch.last_tsc_offset == offset);
> +
> + tsc = kvm_scale_tsc(vcpu, rdtsc(), vcpu->arch.l1_tsc_scaling_ratio) + offset;
> + ns = get_kvmclock_base_ns();
> +
> + __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched);
> + raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
> +
> + r = 0;
> + break;
> + }
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_has_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_has_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_get_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_get_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_set_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_set_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
> struct kvm_enable_cap *cap)
> {
> @@ -5368,6 +5504,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
> r = __set_sregs2(vcpu, u.sregs2);
> break;
> }
> + case KVM_HAS_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_has_device_attr(vcpu, &attr);
> + break;
> + }
> + case KVM_GET_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_get_device_attr(vcpu, &attr);
> + break;
> + }
> + case KVM_SET_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_set_device_attr(vcpu, &attr);
> + break;
> + }
> default:
> r = -EINVAL;
> }
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
@ 2021-07-18 20:42 ` Paolo Bonzini
0 siblings, 0 replies; 73+ messages in thread
From: Paolo Bonzini @ 2021-07-18 20:42 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: Sean Christopherson, Raghavendra Rao Anata, Peter Shier,
Marc Zyngier, David Matlack, Oliver Upton, linux-arm-kernel,
Jim Mattson
On 16/07/21 23:26, Oliver Upton wrote:
> To date, VMM-directed TSC synchronization and migration has been a bit
> messy. KVM has some baked-in heuristics around TSC writes to infer if
> the VMM is attempting to synchronize. This is problematic, as it depends
> on host userspace writing to the guest's TSC within 1 second of the last
> write.
>
> A much cleaner approach to configuring the guest's views of the TSC is to
> simply migrate the TSC offset for every vCPU. Offsets are idempotent,
> and thus not subject to change depending on when the VMM actually
> reads/writes values from/to KVM. The VMM can then read the TSC once with
> KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when
> the guest is paused.
>
> Cc: David Matlack <dmatlack@google.com>
> Signed-off-by: Oliver Upton <oupton@gooogle.com>
> ---
> arch/x86/include/asm/kvm_host.h | 1 +
> arch/x86/include/uapi/asm/kvm.h | 4 +
> arch/x86/kvm/x86.c | 166 ++++++++++++++++++++++++++++++++
> 3 files changed, 171 insertions(+)
This is missing documentation. The documentation should also include
the algorithm in https://www.spinics.net/lists/kvm-arm/msg47383.html
(modulo the fact that KVM_GET/SET_CLOCK return or pass realtime_ns
rather than kvmclock_ns - relatime_ns; which is fine of course).
Paolo
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index e527d7259415..45134b7b14d6 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1070,6 +1070,7 @@ struct kvm_arch {
> u64 last_tsc_nsec;
> u64 last_tsc_write;
> u32 last_tsc_khz;
> + u64 last_tsc_offset;
> u64 cur_tsc_nsec;
> u64 cur_tsc_write;
> u64 cur_tsc_offset;
> diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
> index a6c327f8ad9e..0b22e1e84e78 100644
> --- a/arch/x86/include/uapi/asm/kvm.h
> +++ b/arch/x86/include/uapi/asm/kvm.h
> @@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {
> #define KVM_PMU_EVENT_ALLOW 0
> #define KVM_PMU_EVENT_DENY 1
>
> +/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
> +#define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
> +#define KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
> +
> #endif /* _ASM_X86_KVM_H */
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index e1b7c8b67428..d22de0a1988a 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -2411,6 +2411,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)
> static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset);
> }
>
> +static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu)
> +{
> + return vcpu->arch.l1_tsc_offset;
> +}
> +
> static void kvm_vcpu_write_tsc_multiplier(struct kvm_vcpu *vcpu, u64 l1_multiplier)
> {
> vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;
> @@ -2467,6 +2472,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
> kvm->arch.last_tsc_nsec = ns;
> kvm->arch.last_tsc_write = tsc;
> kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz;
> + kvm->arch.last_tsc_offset = offset;
>
> vcpu->arch.last_guest_tsc = tsc;
>
> @@ -4914,6 +4920,136 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
> return 0;
> }
>
> +static int kvm_arch_tsc_has_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET:
> + r = 0;
> + break;
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + void __user *uaddr = (void __user *)attr->addr;
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET: {
> + u64 offset;
> +
> + offset = kvm_vcpu_read_tsc_offset(vcpu);
> + r = -EFAULT;
> + if (copy_to_user(uaddr, &offset, sizeof(offset)))
> + break;
> +
> + r = 0;
> + }
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_arch_tsc_set_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + void __user *uaddr = (void __user *)attr->addr;
> + struct kvm *kvm = vcpu->kvm;
> + int r;
> +
> + switch (attr->attr) {
> + case KVM_VCPU_TSC_OFFSET: {
> + u64 offset, tsc, ns;
> + unsigned long flags;
> + bool matched;
> +
> + r = -EFAULT;
> + if (copy_from_user(&offset, uaddr, sizeof(offset)))
> + break;
> +
> + raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
> +
> + matched = (vcpu->arch.virtual_tsc_khz &&
> + kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz &&
> + kvm->arch.last_tsc_offset == offset);
> +
> + tsc = kvm_scale_tsc(vcpu, rdtsc(), vcpu->arch.l1_tsc_scaling_ratio) + offset;
> + ns = get_kvmclock_base_ns();
> +
> + __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched);
> + raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
> +
> + r = 0;
> + break;
> + }
> + default:
> + r = -ENXIO;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_has_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_has_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_get_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_get_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_vcpu_ioctl_set_device_attr(struct kvm_vcpu *vcpu,
> + struct kvm_device_attr *attr)
> +{
> + int r;
> +
> + switch (attr->group) {
> + case KVM_VCPU_TSC_CTRL:
> + r = kvm_arch_tsc_set_attr(vcpu, attr);
> + break;
> + default:
> + r = -ENXIO;
> + break;
> + }
> +
> + return r;
> +}
> +
> static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
> struct kvm_enable_cap *cap)
> {
> @@ -5368,6 +5504,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
> r = __set_sregs2(vcpu, u.sregs2);
> break;
> }
> + case KVM_HAS_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_has_device_attr(vcpu, &attr);
> + break;
> + }
> + case KVM_GET_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_get_device_attr(vcpu, &attr);
> + break;
> + }
> + case KVM_SET_DEVICE_ATTR: {
> + struct kvm_device_attr attr;
> +
> + r = -EFAULT;
> + if (copy_from_user(&attr, argp, sizeof(attr)))
> + goto out;
> +
> + r = kvm_vcpu_ioctl_set_device_attr(vcpu, &attr);
> + break;
> + }
> default:
> r = -EINVAL;
> }
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-18 20:50 ` kernel test robot
-1 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 20:50 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Paolo Bonzini, Sean Christopherson, Marc Zyngier,
Peter Shier, Jim Mattson, David Matlack, Ricardo Koller,
Jing Zhang
[-- Attachment #1: Type: text/plain, Size: 3270 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: i386-randconfig-a013-20210718 (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/1e9a9a8e956c70659f72ad66a7d8176ff4756061
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 1e9a9a8e956c70659f72ad66a7d8176ff4756061
# save the attached .config to linux build tree
make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
arch/x86/kvm/x86.c: In function 'get_kvmclock_and_realtime':
arch/x86/kvm/x86.c:2844:7: error: implicit declaration of function 'kvm_get_walltime_and_clockread' [-Werror=implicit-function-declaration]
2844 | if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_get_attr':
>> arch/x86/kvm/x86.c:4945:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
4945 | void __user *uaddr = (void __user *)attr->addr;
| ^
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_set_attr':
arch/x86/kvm/x86.c:4969:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
4969 | void __user *uaddr = (void __user *)attr->addr;
| ^
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_get_attr':
arch/x86/kvm/x86.c:4949:28: warning: this statement may fall through [-Wimplicit-fallthrough=]
4949 | case KVM_VCPU_TSC_OFFSET: {
| ^
arch/x86/kvm/x86.c:4959:2: note: here
4959 | default:
| ^~~~~~~
cc1: some warnings being treated as errors
vim +4945 arch/x86/kvm/x86.c
4941
4942 static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
4943 struct kvm_device_attr *attr)
4944 {
> 4945 void __user *uaddr = (void __user *)attr->addr;
4946 int r;
4947
4948 switch (attr->attr) {
4949 case KVM_VCPU_TSC_OFFSET: {
4950 u64 offset;
4951
4952 offset = kvm_vcpu_read_tsc_offset(vcpu);
4953 r = -EFAULT;
4954 if (copy_to_user(uaddr, &offset, sizeof(offset)))
4955 break;
4956
4957 r = 0;
4958 }
4959 default:
4960 r = -ENXIO;
4961 }
4962
4963 return r;
4964 }
4965
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36661 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
@ 2021-07-18 20:50 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 20:50 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3353 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: i386-randconfig-a013-20210718 (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/1e9a9a8e956c70659f72ad66a7d8176ff4756061
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 1e9a9a8e956c70659f72ad66a7d8176ff4756061
# save the attached .config to linux build tree
make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
arch/x86/kvm/x86.c: In function 'get_kvmclock_and_realtime':
arch/x86/kvm/x86.c:2844:7: error: implicit declaration of function 'kvm_get_walltime_and_clockread' [-Werror=implicit-function-declaration]
2844 | if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_get_attr':
>> arch/x86/kvm/x86.c:4945:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
4945 | void __user *uaddr = (void __user *)attr->addr;
| ^
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_set_attr':
arch/x86/kvm/x86.c:4969:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
4969 | void __user *uaddr = (void __user *)attr->addr;
| ^
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_get_attr':
arch/x86/kvm/x86.c:4949:28: warning: this statement may fall through [-Wimplicit-fallthrough=]
4949 | case KVM_VCPU_TSC_OFFSET: {
| ^
arch/x86/kvm/x86.c:4959:2: note: here
4959 | default:
| ^~~~~~~
cc1: some warnings being treated as errors
vim +4945 arch/x86/kvm/x86.c
4941
4942 static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
4943 struct kvm_device_attr *attr)
4944 {
> 4945 void __user *uaddr = (void __user *)attr->addr;
4946 int r;
4947
4948 switch (attr->attr) {
4949 case KVM_VCPU_TSC_OFFSET: {
4950 u64 offset;
4951
4952 offset = kvm_vcpu_read_tsc_offset(vcpu);
4953 r = -EFAULT;
4954 if (copy_to_user(uaddr, &offset, sizeof(offset)))
4955 break;
4956
4957 r = 0;
4958 }
4959 default:
4960 r = -ENXIO;
4961 }
4962
4963 return r;
4964 }
4965
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36661 bytes --]
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 03/12] KVM: x86: Expose TSC offset controls to userspace
@ 2021-07-18 20:50 ` kernel test robot
0 siblings, 0 replies; 73+ messages in thread
From: kernel test robot @ 2021-07-18 20:50 UTC (permalink / raw)
To: Oliver Upton, kvm, kvmarm
Cc: kbuild-all, Marc Zyngier, Peter Shier, Sean Christopherson,
David Matlack, Paolo Bonzini, Jim Mattson
[-- Attachment #1: Type: text/plain, Size: 3270 bytes --]
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc1 next-20210716]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
config: i386-randconfig-a013-20210718 (attached as .config)
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/1e9a9a8e956c70659f72ad66a7d8176ff4756061
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210718-103407
git checkout 1e9a9a8e956c70659f72ad66a7d8176ff4756061
# save the attached .config to linux build tree
make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
arch/x86/kvm/x86.c: In function 'get_kvmclock_and_realtime':
arch/x86/kvm/x86.c:2844:7: error: implicit declaration of function 'kvm_get_walltime_and_clockread' [-Werror=implicit-function-declaration]
2844 | if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_get_attr':
>> arch/x86/kvm/x86.c:4945:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
4945 | void __user *uaddr = (void __user *)attr->addr;
| ^
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_set_attr':
arch/x86/kvm/x86.c:4969:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
4969 | void __user *uaddr = (void __user *)attr->addr;
| ^
arch/x86/kvm/x86.c: In function 'kvm_arch_tsc_get_attr':
arch/x86/kvm/x86.c:4949:28: warning: this statement may fall through [-Wimplicit-fallthrough=]
4949 | case KVM_VCPU_TSC_OFFSET: {
| ^
arch/x86/kvm/x86.c:4959:2: note: here
4959 | default:
| ^~~~~~~
cc1: some warnings being treated as errors
vim +4945 arch/x86/kvm/x86.c
4941
4942 static int kvm_arch_tsc_get_attr(struct kvm_vcpu *vcpu,
4943 struct kvm_device_attr *attr)
4944 {
> 4945 void __user *uaddr = (void __user *)attr->addr;
4946 int r;
4947
4948 switch (attr->attr) {
4949 case KVM_VCPU_TSC_OFFSET: {
4950 u64 offset;
4951
4952 offset = kvm_vcpu_read_tsc_offset(vcpu);
4953 r = -EFAULT;
4954 if (copy_to_user(uaddr, &offset, sizeof(offset)))
4955 break;
4956
4957 r = 0;
4958 }
4959 default:
4960 r = -ENXIO;
4961 }
4962
4963 return r;
4964 }
4965
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36661 bytes --]
[-- Attachment #3: Type: text/plain, Size: 151 bytes --]
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 04/12] tools: arch: x86: pull in pvclock headers
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Copy over approximately clean versions of the pvclock headers into
tools. Reconcile headers/symbols missing in tools that are unneeded.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/arch/x86/include/asm/pvclock-abi.h | 48 +++++++++++
tools/arch/x86/include/asm/pvclock.h | 103 +++++++++++++++++++++++
2 files changed, 151 insertions(+)
create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
create mode 100644 tools/arch/x86/include/asm/pvclock.h
diff --git a/tools/arch/x86/include/asm/pvclock-abi.h b/tools/arch/x86/include/asm/pvclock-abi.h
new file mode 100644
index 000000000000..1436226efe3e
--- /dev/null
+++ b/tools/arch/x86/include/asm/pvclock-abi.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_PVCLOCK_ABI_H
+#define _ASM_X86_PVCLOCK_ABI_H
+#ifndef __ASSEMBLY__
+
+/*
+ * These structs MUST NOT be changed.
+ * They are the ABI between hypervisor and guest OS.
+ * Both Xen and KVM are using this.
+ *
+ * pvclock_vcpu_time_info holds the system time and the tsc timestamp
+ * of the last update. So the guest can use the tsc delta to get a
+ * more precise system time. There is one per virtual cpu.
+ *
+ * pvclock_wall_clock references the point in time when the system
+ * time was zero (usually boot time), thus the guest calculates the
+ * current wall clock by adding the system time.
+ *
+ * Protocol for the "version" fields is: hypervisor raises it (making
+ * it uneven) before it starts updating the fields and raises it again
+ * (making it even) when it is done. Thus the guest can make sure the
+ * time values it got are consistent by checking the version before
+ * and after reading them.
+ */
+
+struct pvclock_vcpu_time_info {
+ u32 version;
+ u32 pad0;
+ u64 tsc_timestamp;
+ u64 system_time;
+ u32 tsc_to_system_mul;
+ s8 tsc_shift;
+ u8 flags;
+ u8 pad[2];
+} __attribute__((__packed__)); /* 32 bytes */
+
+struct pvclock_wall_clock {
+ u32 version;
+ u32 sec;
+ u32 nsec;
+} __attribute__((__packed__));
+
+#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
+/* PVCLOCK_COUNTS_FROM_ZERO broke ABI and can't be used anymore. */
+#define PVCLOCK_COUNTS_FROM_ZERO (1 << 2)
+#endif /* __ASSEMBLY__ */
+#endif /* _ASM_X86_PVCLOCK_ABI_H */
diff --git a/tools/arch/x86/include/asm/pvclock.h b/tools/arch/x86/include/asm/pvclock.h
new file mode 100644
index 000000000000..2628f9a6330b
--- /dev/null
+++ b/tools/arch/x86/include/asm/pvclock.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_PVCLOCK_H
+#define _ASM_X86_PVCLOCK_H
+
+#include <asm/barrier.h>
+#include <asm/pvclock-abi.h>
+
+/* some helper functions for xen and kvm pv clock sources */
+u64 pvclock_clocksource_read(struct pvclock_vcpu_time_info *src);
+u8 pvclock_read_flags(struct pvclock_vcpu_time_info *src);
+void pvclock_set_flags(u8 flags);
+unsigned long pvclock_tsc_khz(struct pvclock_vcpu_time_info *src);
+void pvclock_resume(void);
+
+void pvclock_touch_watchdogs(void);
+
+static __always_inline
+unsigned pvclock_read_begin(const struct pvclock_vcpu_time_info *src)
+{
+ unsigned version = src->version & ~1;
+ /* Make sure that the version is read before the data. */
+ rmb();
+ return version;
+}
+
+static __always_inline
+bool pvclock_read_retry(const struct pvclock_vcpu_time_info *src,
+ unsigned version)
+{
+ /* Make sure that the version is re-read after the data. */
+ rmb();
+ return version != src->version;
+}
+
+/*
+ * Scale a 64-bit delta by scaling and multiplying by a 32-bit fraction,
+ * yielding a 64-bit result.
+ */
+static inline u64 pvclock_scale_delta(u64 delta, u32 mul_frac, int shift)
+{
+ u64 product;
+#ifdef __i386__
+ u32 tmp1, tmp2;
+#else
+ unsigned long tmp;
+#endif
+
+ if (shift < 0)
+ delta >>= -shift;
+ else
+ delta <<= shift;
+
+#ifdef __i386__
+ __asm__ (
+ "mul %5 ; "
+ "mov %4,%%eax ; "
+ "mov %%edx,%4 ; "
+ "mul %5 ; "
+ "xor %5,%5 ; "
+ "add %4,%%eax ; "
+ "adc %5,%%edx ; "
+ : "=A" (product), "=r" (tmp1), "=r" (tmp2)
+ : "a" ((u32)delta), "1" ((u32)(delta >> 32)), "2" (mul_frac) );
+#elif defined(__x86_64__)
+ __asm__ (
+ "mulq %[mul_frac] ; shrd $32, %[hi], %[lo]"
+ : [lo]"=a"(product),
+ [hi]"=d"(tmp)
+ : "0"(delta),
+ [mul_frac]"rm"((u64)mul_frac));
+#else
+#error implement me!
+#endif
+
+ return product;
+}
+
+static __always_inline
+u64 __pvclock_read_cycles(const struct pvclock_vcpu_time_info *src, u64 tsc)
+{
+ u64 delta = tsc - src->tsc_timestamp;
+ u64 offset = pvclock_scale_delta(delta, src->tsc_to_system_mul,
+ src->tsc_shift);
+ return src->system_time + offset;
+}
+
+struct pvclock_vsyscall_time_info {
+ struct pvclock_vcpu_time_info pvti;
+} __attribute__((__aligned__(64)));
+
+#define PVTI_SIZE sizeof(struct pvclock_vsyscall_time_info)
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+void pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti);
+struct pvclock_vsyscall_time_info *pvclock_get_pvti_cpu0_va(void);
+#else
+static inline struct pvclock_vsyscall_time_info *pvclock_get_pvti_cpu0_va(void)
+{
+ return NULL;
+}
+#endif
+
+#endif /* _ASM_X86_PVCLOCK_H */
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 04/12] tools: arch: x86: pull in pvclock headers
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Copy over approximately clean versions of the pvclock headers into
tools. Reconcile headers/symbols missing in tools that are unneeded.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/arch/x86/include/asm/pvclock-abi.h | 48 +++++++++++
tools/arch/x86/include/asm/pvclock.h | 103 +++++++++++++++++++++++
2 files changed, 151 insertions(+)
create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
create mode 100644 tools/arch/x86/include/asm/pvclock.h
diff --git a/tools/arch/x86/include/asm/pvclock-abi.h b/tools/arch/x86/include/asm/pvclock-abi.h
new file mode 100644
index 000000000000..1436226efe3e
--- /dev/null
+++ b/tools/arch/x86/include/asm/pvclock-abi.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_PVCLOCK_ABI_H
+#define _ASM_X86_PVCLOCK_ABI_H
+#ifndef __ASSEMBLY__
+
+/*
+ * These structs MUST NOT be changed.
+ * They are the ABI between hypervisor and guest OS.
+ * Both Xen and KVM are using this.
+ *
+ * pvclock_vcpu_time_info holds the system time and the tsc timestamp
+ * of the last update. So the guest can use the tsc delta to get a
+ * more precise system time. There is one per virtual cpu.
+ *
+ * pvclock_wall_clock references the point in time when the system
+ * time was zero (usually boot time), thus the guest calculates the
+ * current wall clock by adding the system time.
+ *
+ * Protocol for the "version" fields is: hypervisor raises it (making
+ * it uneven) before it starts updating the fields and raises it again
+ * (making it even) when it is done. Thus the guest can make sure the
+ * time values it got are consistent by checking the version before
+ * and after reading them.
+ */
+
+struct pvclock_vcpu_time_info {
+ u32 version;
+ u32 pad0;
+ u64 tsc_timestamp;
+ u64 system_time;
+ u32 tsc_to_system_mul;
+ s8 tsc_shift;
+ u8 flags;
+ u8 pad[2];
+} __attribute__((__packed__)); /* 32 bytes */
+
+struct pvclock_wall_clock {
+ u32 version;
+ u32 sec;
+ u32 nsec;
+} __attribute__((__packed__));
+
+#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
+/* PVCLOCK_COUNTS_FROM_ZERO broke ABI and can't be used anymore. */
+#define PVCLOCK_COUNTS_FROM_ZERO (1 << 2)
+#endif /* __ASSEMBLY__ */
+#endif /* _ASM_X86_PVCLOCK_ABI_H */
diff --git a/tools/arch/x86/include/asm/pvclock.h b/tools/arch/x86/include/asm/pvclock.h
new file mode 100644
index 000000000000..2628f9a6330b
--- /dev/null
+++ b/tools/arch/x86/include/asm/pvclock.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_PVCLOCK_H
+#define _ASM_X86_PVCLOCK_H
+
+#include <asm/barrier.h>
+#include <asm/pvclock-abi.h>
+
+/* some helper functions for xen and kvm pv clock sources */
+u64 pvclock_clocksource_read(struct pvclock_vcpu_time_info *src);
+u8 pvclock_read_flags(struct pvclock_vcpu_time_info *src);
+void pvclock_set_flags(u8 flags);
+unsigned long pvclock_tsc_khz(struct pvclock_vcpu_time_info *src);
+void pvclock_resume(void);
+
+void pvclock_touch_watchdogs(void);
+
+static __always_inline
+unsigned pvclock_read_begin(const struct pvclock_vcpu_time_info *src)
+{
+ unsigned version = src->version & ~1;
+ /* Make sure that the version is read before the data. */
+ rmb();
+ return version;
+}
+
+static __always_inline
+bool pvclock_read_retry(const struct pvclock_vcpu_time_info *src,
+ unsigned version)
+{
+ /* Make sure that the version is re-read after the data. */
+ rmb();
+ return version != src->version;
+}
+
+/*
+ * Scale a 64-bit delta by scaling and multiplying by a 32-bit fraction,
+ * yielding a 64-bit result.
+ */
+static inline u64 pvclock_scale_delta(u64 delta, u32 mul_frac, int shift)
+{
+ u64 product;
+#ifdef __i386__
+ u32 tmp1, tmp2;
+#else
+ unsigned long tmp;
+#endif
+
+ if (shift < 0)
+ delta >>= -shift;
+ else
+ delta <<= shift;
+
+#ifdef __i386__
+ __asm__ (
+ "mul %5 ; "
+ "mov %4,%%eax ; "
+ "mov %%edx,%4 ; "
+ "mul %5 ; "
+ "xor %5,%5 ; "
+ "add %4,%%eax ; "
+ "adc %5,%%edx ; "
+ : "=A" (product), "=r" (tmp1), "=r" (tmp2)
+ : "a" ((u32)delta), "1" ((u32)(delta >> 32)), "2" (mul_frac) );
+#elif defined(__x86_64__)
+ __asm__ (
+ "mulq %[mul_frac] ; shrd $32, %[hi], %[lo]"
+ : [lo]"=a"(product),
+ [hi]"=d"(tmp)
+ : "0"(delta),
+ [mul_frac]"rm"((u64)mul_frac));
+#else
+#error implement me!
+#endif
+
+ return product;
+}
+
+static __always_inline
+u64 __pvclock_read_cycles(const struct pvclock_vcpu_time_info *src, u64 tsc)
+{
+ u64 delta = tsc - src->tsc_timestamp;
+ u64 offset = pvclock_scale_delta(delta, src->tsc_to_system_mul,
+ src->tsc_shift);
+ return src->system_time + offset;
+}
+
+struct pvclock_vsyscall_time_info {
+ struct pvclock_vcpu_time_info pvti;
+} __attribute__((__aligned__(64)));
+
+#define PVTI_SIZE sizeof(struct pvclock_vsyscall_time_info)
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+void pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti);
+struct pvclock_vsyscall_time_info *pvclock_get_pvti_cpu0_va(void);
+#else
+static inline struct pvclock_vsyscall_time_info *pvclock_get_pvti_cpu0_va(void)
+{
+ return NULL;
+}
+#endif
+
+#endif /* _ASM_X86_PVCLOCK_H */
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 04/12] tools: arch: x86: pull in pvclock headers
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Copy over approximately clean versions of the pvclock headers into
tools. Reconcile headers/symbols missing in tools that are unneeded.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/arch/x86/include/asm/pvclock-abi.h | 48 +++++++++++
tools/arch/x86/include/asm/pvclock.h | 103 +++++++++++++++++++++++
2 files changed, 151 insertions(+)
create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
create mode 100644 tools/arch/x86/include/asm/pvclock.h
diff --git a/tools/arch/x86/include/asm/pvclock-abi.h b/tools/arch/x86/include/asm/pvclock-abi.h
new file mode 100644
index 000000000000..1436226efe3e
--- /dev/null
+++ b/tools/arch/x86/include/asm/pvclock-abi.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_PVCLOCK_ABI_H
+#define _ASM_X86_PVCLOCK_ABI_H
+#ifndef __ASSEMBLY__
+
+/*
+ * These structs MUST NOT be changed.
+ * They are the ABI between hypervisor and guest OS.
+ * Both Xen and KVM are using this.
+ *
+ * pvclock_vcpu_time_info holds the system time and the tsc timestamp
+ * of the last update. So the guest can use the tsc delta to get a
+ * more precise system time. There is one per virtual cpu.
+ *
+ * pvclock_wall_clock references the point in time when the system
+ * time was zero (usually boot time), thus the guest calculates the
+ * current wall clock by adding the system time.
+ *
+ * Protocol for the "version" fields is: hypervisor raises it (making
+ * it uneven) before it starts updating the fields and raises it again
+ * (making it even) when it is done. Thus the guest can make sure the
+ * time values it got are consistent by checking the version before
+ * and after reading them.
+ */
+
+struct pvclock_vcpu_time_info {
+ u32 version;
+ u32 pad0;
+ u64 tsc_timestamp;
+ u64 system_time;
+ u32 tsc_to_system_mul;
+ s8 tsc_shift;
+ u8 flags;
+ u8 pad[2];
+} __attribute__((__packed__)); /* 32 bytes */
+
+struct pvclock_wall_clock {
+ u32 version;
+ u32 sec;
+ u32 nsec;
+} __attribute__((__packed__));
+
+#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
+/* PVCLOCK_COUNTS_FROM_ZERO broke ABI and can't be used anymore. */
+#define PVCLOCK_COUNTS_FROM_ZERO (1 << 2)
+#endif /* __ASSEMBLY__ */
+#endif /* _ASM_X86_PVCLOCK_ABI_H */
diff --git a/tools/arch/x86/include/asm/pvclock.h b/tools/arch/x86/include/asm/pvclock.h
new file mode 100644
index 000000000000..2628f9a6330b
--- /dev/null
+++ b/tools/arch/x86/include/asm/pvclock.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_PVCLOCK_H
+#define _ASM_X86_PVCLOCK_H
+
+#include <asm/barrier.h>
+#include <asm/pvclock-abi.h>
+
+/* some helper functions for xen and kvm pv clock sources */
+u64 pvclock_clocksource_read(struct pvclock_vcpu_time_info *src);
+u8 pvclock_read_flags(struct pvclock_vcpu_time_info *src);
+void pvclock_set_flags(u8 flags);
+unsigned long pvclock_tsc_khz(struct pvclock_vcpu_time_info *src);
+void pvclock_resume(void);
+
+void pvclock_touch_watchdogs(void);
+
+static __always_inline
+unsigned pvclock_read_begin(const struct pvclock_vcpu_time_info *src)
+{
+ unsigned version = src->version & ~1;
+ /* Make sure that the version is read before the data. */
+ rmb();
+ return version;
+}
+
+static __always_inline
+bool pvclock_read_retry(const struct pvclock_vcpu_time_info *src,
+ unsigned version)
+{
+ /* Make sure that the version is re-read after the data. */
+ rmb();
+ return version != src->version;
+}
+
+/*
+ * Scale a 64-bit delta by scaling and multiplying by a 32-bit fraction,
+ * yielding a 64-bit result.
+ */
+static inline u64 pvclock_scale_delta(u64 delta, u32 mul_frac, int shift)
+{
+ u64 product;
+#ifdef __i386__
+ u32 tmp1, tmp2;
+#else
+ unsigned long tmp;
+#endif
+
+ if (shift < 0)
+ delta >>= -shift;
+ else
+ delta <<= shift;
+
+#ifdef __i386__
+ __asm__ (
+ "mul %5 ; "
+ "mov %4,%%eax ; "
+ "mov %%edx,%4 ; "
+ "mul %5 ; "
+ "xor %5,%5 ; "
+ "add %4,%%eax ; "
+ "adc %5,%%edx ; "
+ : "=A" (product), "=r" (tmp1), "=r" (tmp2)
+ : "a" ((u32)delta), "1" ((u32)(delta >> 32)), "2" (mul_frac) );
+#elif defined(__x86_64__)
+ __asm__ (
+ "mulq %[mul_frac] ; shrd $32, %[hi], %[lo]"
+ : [lo]"=a"(product),
+ [hi]"=d"(tmp)
+ : "0"(delta),
+ [mul_frac]"rm"((u64)mul_frac));
+#else
+#error implement me!
+#endif
+
+ return product;
+}
+
+static __always_inline
+u64 __pvclock_read_cycles(const struct pvclock_vcpu_time_info *src, u64 tsc)
+{
+ u64 delta = tsc - src->tsc_timestamp;
+ u64 offset = pvclock_scale_delta(delta, src->tsc_to_system_mul,
+ src->tsc_shift);
+ return src->system_time + offset;
+}
+
+struct pvclock_vsyscall_time_info {
+ struct pvclock_vcpu_time_info pvti;
+} __attribute__((__aligned__(64)));
+
+#define PVTI_SIZE sizeof(struct pvclock_vsyscall_time_info)
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+void pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti);
+struct pvclock_vsyscall_time_info *pvclock_get_pvti_cpu0_va(void);
+#else
+static inline struct pvclock_vsyscall_time_info *pvclock_get_pvti_cpu0_va(void)
+{
+ return NULL;
+}
+#endif
+
+#endif /* _ASM_X86_PVCLOCK_H */
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 05/12] selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Add a selftest for the new KVM clock UAPI that was introduced. Ensure
that the KVM clock is consistent between userspace and the guest, and
that the difference in realtime will only ever cause the KVM clock to
advance forward.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../testing/selftests/kvm/include/kvm_util.h | 2 +
.../selftests/kvm/x86_64/kvm_clock_test.c | 210 ++++++++++++++++++
4 files changed, 214 insertions(+)
create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 06a351b4f93b..d0877d01e771 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -11,6 +11,7 @@
/x86_64/emulator_error_test
/x86_64/get_cpuid_test
/x86_64/get_msr_index_features
+/x86_64/kvm_clock_test
/x86_64/kvm_pv_test
/x86_64/hyperv_clock
/x86_64/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index b853be2ae3c6..f7e24f334c6e 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
+TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index 010b59b13917..a8ac5d52e17b 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -19,6 +19,8 @@
#define KVM_DEV_PATH "/dev/kvm"
#define KVM_MAX_VCPUS 512
+#define NSEC_PER_SEC 1000000000L
+
/*
* Callers of kvm_util only have an incomplete/opaque description of the
* structure kvm_util is using to maintain the state of a VM.
diff --git a/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
new file mode 100644
index 000000000000..34c48f2dde54
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Google LLC.
+ *
+ * Tests for adjusting the KVM clock from userspace
+ */
+#include <asm/kvm_para.h>
+#include <asm/pvclock.h>
+#include <asm/pvclock-abi.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define VCPU_ID 0
+
+struct test_case {
+ uint64_t kvmclock_base;
+ int64_t realtime_offset;
+};
+
+static struct test_case test_cases[] = {
+ { .kvmclock_base = 0 },
+ { .kvmclock_base = 180 * NSEC_PER_SEC },
+ { .kvmclock_base = 0, .realtime_offset = -180 * NSEC_PER_SEC },
+ { .kvmclock_base = 0, .realtime_offset = 180 * NSEC_PER_SEC },
+};
+
+#define GUEST_SYNC_CLOCK(__stage, __val) \
+ GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
+
+static void guest_main(vm_paddr_t pvti_pa, struct pvclock_vcpu_time_info *pvti)
+{
+ int i;
+
+ wrmsr(MSR_KVM_SYSTEM_TIME_NEW, pvti_pa | KVM_MSR_ENABLED);
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ GUEST_SYNC_CLOCK(i, __pvclock_read_cycles(pvti, rdtsc()));
+ }
+
+ GUEST_DONE();
+}
+
+#define EXPECTED_FLAGS (KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
+
+static inline void assert_flags(struct kvm_clock_data *data)
+{
+ TEST_ASSERT((data->flags & EXPECTED_FLAGS) == EXPECTED_FLAGS,
+ "unexpected clock data flags: %x (want set: %x)",
+ data->flags, EXPECTED_FLAGS);
+}
+
+static void handle_sync(struct ucall *uc, struct kvm_clock_data *start,
+ struct kvm_clock_data *end)
+{
+ uint64_t obs, exp_lo, exp_hi;
+
+ obs = uc->args[2];
+ exp_lo = start->clock;
+ exp_hi = end->clock;
+
+ assert_flags(start);
+ assert_flags(end);
+
+ TEST_ASSERT(exp_lo <= obs && obs <= exp_hi,
+ "unexpected kvm-clock value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
+ obs, exp_lo, exp_hi);
+
+ pr_info("kvm-clock value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
+ obs, exp_lo, exp_hi);
+}
+
+static void handle_abort(struct ucall *uc)
+{
+ TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
+ __FILE__, uc->args[1]);
+}
+
+static void setup_clock(struct kvm_vm *vm, struct test_case *test_case)
+{
+ struct kvm_clock_data data;
+
+ memset(&data, 0, sizeof(data));
+
+ data.clock = test_case->kvmclock_base;
+ if (test_case->realtime_offset) {
+ struct timespec ts;
+ int r;
+
+ data.flags |= KVM_CLOCK_REAL_TIME;
+ do {
+ r = clock_gettime(CLOCK_REALTIME, &ts);
+ if (!r)
+ break;
+ } while (errno == EINTR);
+
+ TEST_ASSERT(!r, "clock_gettime() failed: %d\n", r);
+
+ data.realtime = ts.tv_sec * NSEC_PER_SEC;
+ data.realtime += ts.tv_nsec;
+ data.realtime += test_case->realtime_offset;
+ }
+
+ vm_ioctl(vm, KVM_SET_CLOCK, &data);
+}
+
+static void enter_guest(struct kvm_vm *vm)
+{
+ struct kvm_clock_data start, end;
+ struct kvm_run *run;
+ struct ucall uc;
+ int i, r;
+
+ run = vcpu_state(vm, VCPU_ID);
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ setup_clock(vm, &test_cases[i]);
+
+ vm_ioctl(vm, KVM_GET_CLOCK, &start);
+
+ r = _vcpu_run(vm, VCPU_ID);
+ vm_ioctl(vm, KVM_GET_CLOCK, &end);
+
+ TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
+ TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
+ "unexpected exit reason: %u (%s)",
+ run->exit_reason, exit_reason_str(run->exit_reason));
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_SYNC:
+ handle_sync(&uc, &start, &end);
+ break;
+ case UCALL_ABORT:
+ handle_abort(&uc);
+ return;
+ case UCALL_DONE:
+ return;
+ }
+ }
+}
+
+#define CLOCKSOURCE_PATH "/sys/devices/system/clocksource/clocksource0/current_clocksource"
+
+static void check_clocksource(void)
+{
+ char *clk_name;
+ struct stat st;
+ FILE *fp;
+
+ fp = fopen(CLOCKSOURCE_PATH, "r");
+ if (!fp) {
+ pr_info("failed to open clocksource file: %d; assuming TSC.\n",
+ errno);
+ return;
+ }
+
+ if (fstat(fileno(fp), &st)) {
+ pr_info("failed to stat clocksource file: %d; assuming TSC.\n",
+ errno);
+ goto out;
+ }
+
+ clk_name = malloc(st.st_size);
+ TEST_ASSERT(clk_name, "failed to allocate buffer to read file\n");
+
+ if (!fgets(clk_name, st.st_size, fp)) {
+ pr_info("failed to read clocksource file: %d; assuming TSC.\n",
+ ferror(fp));
+ goto out;
+ }
+
+ TEST_ASSERT(!strncmp(clk_name, "tsc\n", st.st_size),
+ "clocksource not supported: %s", clk_name);
+out:
+ fclose(fp);
+}
+
+int main(void)
+{
+ struct kvm_cpuid2 *best;
+ vm_vaddr_t pvti_gva;
+ vm_paddr_t pvti_gpa;
+ struct kvm_vm *vm;
+ int flags;
+
+ flags = kvm_check_cap(KVM_CAP_ADJUST_CLOCK);
+ if (!(flags & KVM_CLOCK_REAL_TIME)) {
+ print_skip("KVM_CLOCK_REAL_TIME not supported; flags: %x",
+ flags);
+ exit(KSFT_SKIP);
+ }
+
+ check_clocksource();
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+
+ best = kvm_get_supported_cpuid();
+ vcpu_set_cpuid(vm, VCPU_ID, best);
+
+ pvti_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000);
+ pvti_gpa = addr_gva2gpa(vm, pvti_gva);
+ vcpu_args_set(vm, VCPU_ID, 2, pvti_gpa, pvti_gva);
+
+ enter_guest(vm);
+ kvm_vm_free(vm);
+}
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 05/12] selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Add a selftest for the new KVM clock UAPI that was introduced. Ensure
that the KVM clock is consistent between userspace and the guest, and
that the difference in realtime will only ever cause the KVM clock to
advance forward.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../testing/selftests/kvm/include/kvm_util.h | 2 +
.../selftests/kvm/x86_64/kvm_clock_test.c | 210 ++++++++++++++++++
4 files changed, 214 insertions(+)
create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 06a351b4f93b..d0877d01e771 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -11,6 +11,7 @@
/x86_64/emulator_error_test
/x86_64/get_cpuid_test
/x86_64/get_msr_index_features
+/x86_64/kvm_clock_test
/x86_64/kvm_pv_test
/x86_64/hyperv_clock
/x86_64/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index b853be2ae3c6..f7e24f334c6e 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
+TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index 010b59b13917..a8ac5d52e17b 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -19,6 +19,8 @@
#define KVM_DEV_PATH "/dev/kvm"
#define KVM_MAX_VCPUS 512
+#define NSEC_PER_SEC 1000000000L
+
/*
* Callers of kvm_util only have an incomplete/opaque description of the
* structure kvm_util is using to maintain the state of a VM.
diff --git a/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
new file mode 100644
index 000000000000..34c48f2dde54
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Google LLC.
+ *
+ * Tests for adjusting the KVM clock from userspace
+ */
+#include <asm/kvm_para.h>
+#include <asm/pvclock.h>
+#include <asm/pvclock-abi.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define VCPU_ID 0
+
+struct test_case {
+ uint64_t kvmclock_base;
+ int64_t realtime_offset;
+};
+
+static struct test_case test_cases[] = {
+ { .kvmclock_base = 0 },
+ { .kvmclock_base = 180 * NSEC_PER_SEC },
+ { .kvmclock_base = 0, .realtime_offset = -180 * NSEC_PER_SEC },
+ { .kvmclock_base = 0, .realtime_offset = 180 * NSEC_PER_SEC },
+};
+
+#define GUEST_SYNC_CLOCK(__stage, __val) \
+ GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
+
+static void guest_main(vm_paddr_t pvti_pa, struct pvclock_vcpu_time_info *pvti)
+{
+ int i;
+
+ wrmsr(MSR_KVM_SYSTEM_TIME_NEW, pvti_pa | KVM_MSR_ENABLED);
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ GUEST_SYNC_CLOCK(i, __pvclock_read_cycles(pvti, rdtsc()));
+ }
+
+ GUEST_DONE();
+}
+
+#define EXPECTED_FLAGS (KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
+
+static inline void assert_flags(struct kvm_clock_data *data)
+{
+ TEST_ASSERT((data->flags & EXPECTED_FLAGS) == EXPECTED_FLAGS,
+ "unexpected clock data flags: %x (want set: %x)",
+ data->flags, EXPECTED_FLAGS);
+}
+
+static void handle_sync(struct ucall *uc, struct kvm_clock_data *start,
+ struct kvm_clock_data *end)
+{
+ uint64_t obs, exp_lo, exp_hi;
+
+ obs = uc->args[2];
+ exp_lo = start->clock;
+ exp_hi = end->clock;
+
+ assert_flags(start);
+ assert_flags(end);
+
+ TEST_ASSERT(exp_lo <= obs && obs <= exp_hi,
+ "unexpected kvm-clock value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
+ obs, exp_lo, exp_hi);
+
+ pr_info("kvm-clock value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
+ obs, exp_lo, exp_hi);
+}
+
+static void handle_abort(struct ucall *uc)
+{
+ TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
+ __FILE__, uc->args[1]);
+}
+
+static void setup_clock(struct kvm_vm *vm, struct test_case *test_case)
+{
+ struct kvm_clock_data data;
+
+ memset(&data, 0, sizeof(data));
+
+ data.clock = test_case->kvmclock_base;
+ if (test_case->realtime_offset) {
+ struct timespec ts;
+ int r;
+
+ data.flags |= KVM_CLOCK_REAL_TIME;
+ do {
+ r = clock_gettime(CLOCK_REALTIME, &ts);
+ if (!r)
+ break;
+ } while (errno == EINTR);
+
+ TEST_ASSERT(!r, "clock_gettime() failed: %d\n", r);
+
+ data.realtime = ts.tv_sec * NSEC_PER_SEC;
+ data.realtime += ts.tv_nsec;
+ data.realtime += test_case->realtime_offset;
+ }
+
+ vm_ioctl(vm, KVM_SET_CLOCK, &data);
+}
+
+static void enter_guest(struct kvm_vm *vm)
+{
+ struct kvm_clock_data start, end;
+ struct kvm_run *run;
+ struct ucall uc;
+ int i, r;
+
+ run = vcpu_state(vm, VCPU_ID);
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ setup_clock(vm, &test_cases[i]);
+
+ vm_ioctl(vm, KVM_GET_CLOCK, &start);
+
+ r = _vcpu_run(vm, VCPU_ID);
+ vm_ioctl(vm, KVM_GET_CLOCK, &end);
+
+ TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
+ TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
+ "unexpected exit reason: %u (%s)",
+ run->exit_reason, exit_reason_str(run->exit_reason));
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_SYNC:
+ handle_sync(&uc, &start, &end);
+ break;
+ case UCALL_ABORT:
+ handle_abort(&uc);
+ return;
+ case UCALL_DONE:
+ return;
+ }
+ }
+}
+
+#define CLOCKSOURCE_PATH "/sys/devices/system/clocksource/clocksource0/current_clocksource"
+
+static void check_clocksource(void)
+{
+ char *clk_name;
+ struct stat st;
+ FILE *fp;
+
+ fp = fopen(CLOCKSOURCE_PATH, "r");
+ if (!fp) {
+ pr_info("failed to open clocksource file: %d; assuming TSC.\n",
+ errno);
+ return;
+ }
+
+ if (fstat(fileno(fp), &st)) {
+ pr_info("failed to stat clocksource file: %d; assuming TSC.\n",
+ errno);
+ goto out;
+ }
+
+ clk_name = malloc(st.st_size);
+ TEST_ASSERT(clk_name, "failed to allocate buffer to read file\n");
+
+ if (!fgets(clk_name, st.st_size, fp)) {
+ pr_info("failed to read clocksource file: %d; assuming TSC.\n",
+ ferror(fp));
+ goto out;
+ }
+
+ TEST_ASSERT(!strncmp(clk_name, "tsc\n", st.st_size),
+ "clocksource not supported: %s", clk_name);
+out:
+ fclose(fp);
+}
+
+int main(void)
+{
+ struct kvm_cpuid2 *best;
+ vm_vaddr_t pvti_gva;
+ vm_paddr_t pvti_gpa;
+ struct kvm_vm *vm;
+ int flags;
+
+ flags = kvm_check_cap(KVM_CAP_ADJUST_CLOCK);
+ if (!(flags & KVM_CLOCK_REAL_TIME)) {
+ print_skip("KVM_CLOCK_REAL_TIME not supported; flags: %x",
+ flags);
+ exit(KSFT_SKIP);
+ }
+
+ check_clocksource();
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+
+ best = kvm_get_supported_cpuid();
+ vcpu_set_cpuid(vm, VCPU_ID, best);
+
+ pvti_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000);
+ pvti_gpa = addr_gva2gpa(vm, pvti_gva);
+ vcpu_args_set(vm, VCPU_ID, 2, pvti_gpa, pvti_gva);
+
+ enter_guest(vm);
+ kvm_vm_free(vm);
+}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 05/12] selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Add a selftest for the new KVM clock UAPI that was introduced. Ensure
that the KVM clock is consistent between userspace and the guest, and
that the difference in realtime will only ever cause the KVM clock to
advance forward.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../testing/selftests/kvm/include/kvm_util.h | 2 +
.../selftests/kvm/x86_64/kvm_clock_test.c | 210 ++++++++++++++++++
4 files changed, 214 insertions(+)
create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 06a351b4f93b..d0877d01e771 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -11,6 +11,7 @@
/x86_64/emulator_error_test
/x86_64/get_cpuid_test
/x86_64/get_msr_index_features
+/x86_64/kvm_clock_test
/x86_64/kvm_pv_test
/x86_64/hyperv_clock
/x86_64/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index b853be2ae3c6..f7e24f334c6e 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
+TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index 010b59b13917..a8ac5d52e17b 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -19,6 +19,8 @@
#define KVM_DEV_PATH "/dev/kvm"
#define KVM_MAX_VCPUS 512
+#define NSEC_PER_SEC 1000000000L
+
/*
* Callers of kvm_util only have an incomplete/opaque description of the
* structure kvm_util is using to maintain the state of a VM.
diff --git a/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
new file mode 100644
index 000000000000..34c48f2dde54
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Google LLC.
+ *
+ * Tests for adjusting the KVM clock from userspace
+ */
+#include <asm/kvm_para.h>
+#include <asm/pvclock.h>
+#include <asm/pvclock-abi.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define VCPU_ID 0
+
+struct test_case {
+ uint64_t kvmclock_base;
+ int64_t realtime_offset;
+};
+
+static struct test_case test_cases[] = {
+ { .kvmclock_base = 0 },
+ { .kvmclock_base = 180 * NSEC_PER_SEC },
+ { .kvmclock_base = 0, .realtime_offset = -180 * NSEC_PER_SEC },
+ { .kvmclock_base = 0, .realtime_offset = 180 * NSEC_PER_SEC },
+};
+
+#define GUEST_SYNC_CLOCK(__stage, __val) \
+ GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
+
+static void guest_main(vm_paddr_t pvti_pa, struct pvclock_vcpu_time_info *pvti)
+{
+ int i;
+
+ wrmsr(MSR_KVM_SYSTEM_TIME_NEW, pvti_pa | KVM_MSR_ENABLED);
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ GUEST_SYNC_CLOCK(i, __pvclock_read_cycles(pvti, rdtsc()));
+ }
+
+ GUEST_DONE();
+}
+
+#define EXPECTED_FLAGS (KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
+
+static inline void assert_flags(struct kvm_clock_data *data)
+{
+ TEST_ASSERT((data->flags & EXPECTED_FLAGS) == EXPECTED_FLAGS,
+ "unexpected clock data flags: %x (want set: %x)",
+ data->flags, EXPECTED_FLAGS);
+}
+
+static void handle_sync(struct ucall *uc, struct kvm_clock_data *start,
+ struct kvm_clock_data *end)
+{
+ uint64_t obs, exp_lo, exp_hi;
+
+ obs = uc->args[2];
+ exp_lo = start->clock;
+ exp_hi = end->clock;
+
+ assert_flags(start);
+ assert_flags(end);
+
+ TEST_ASSERT(exp_lo <= obs && obs <= exp_hi,
+ "unexpected kvm-clock value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
+ obs, exp_lo, exp_hi);
+
+ pr_info("kvm-clock value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
+ obs, exp_lo, exp_hi);
+}
+
+static void handle_abort(struct ucall *uc)
+{
+ TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
+ __FILE__, uc->args[1]);
+}
+
+static void setup_clock(struct kvm_vm *vm, struct test_case *test_case)
+{
+ struct kvm_clock_data data;
+
+ memset(&data, 0, sizeof(data));
+
+ data.clock = test_case->kvmclock_base;
+ if (test_case->realtime_offset) {
+ struct timespec ts;
+ int r;
+
+ data.flags |= KVM_CLOCK_REAL_TIME;
+ do {
+ r = clock_gettime(CLOCK_REALTIME, &ts);
+ if (!r)
+ break;
+ } while (errno == EINTR);
+
+ TEST_ASSERT(!r, "clock_gettime() failed: %d\n", r);
+
+ data.realtime = ts.tv_sec * NSEC_PER_SEC;
+ data.realtime += ts.tv_nsec;
+ data.realtime += test_case->realtime_offset;
+ }
+
+ vm_ioctl(vm, KVM_SET_CLOCK, &data);
+}
+
+static void enter_guest(struct kvm_vm *vm)
+{
+ struct kvm_clock_data start, end;
+ struct kvm_run *run;
+ struct ucall uc;
+ int i, r;
+
+ run = vcpu_state(vm, VCPU_ID);
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ setup_clock(vm, &test_cases[i]);
+
+ vm_ioctl(vm, KVM_GET_CLOCK, &start);
+
+ r = _vcpu_run(vm, VCPU_ID);
+ vm_ioctl(vm, KVM_GET_CLOCK, &end);
+
+ TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
+ TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
+ "unexpected exit reason: %u (%s)",
+ run->exit_reason, exit_reason_str(run->exit_reason));
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_SYNC:
+ handle_sync(&uc, &start, &end);
+ break;
+ case UCALL_ABORT:
+ handle_abort(&uc);
+ return;
+ case UCALL_DONE:
+ return;
+ }
+ }
+}
+
+#define CLOCKSOURCE_PATH "/sys/devices/system/clocksource/clocksource0/current_clocksource"
+
+static void check_clocksource(void)
+{
+ char *clk_name;
+ struct stat st;
+ FILE *fp;
+
+ fp = fopen(CLOCKSOURCE_PATH, "r");
+ if (!fp) {
+ pr_info("failed to open clocksource file: %d; assuming TSC.\n",
+ errno);
+ return;
+ }
+
+ if (fstat(fileno(fp), &st)) {
+ pr_info("failed to stat clocksource file: %d; assuming TSC.\n",
+ errno);
+ goto out;
+ }
+
+ clk_name = malloc(st.st_size);
+ TEST_ASSERT(clk_name, "failed to allocate buffer to read file\n");
+
+ if (!fgets(clk_name, st.st_size, fp)) {
+ pr_info("failed to read clocksource file: %d; assuming TSC.\n",
+ ferror(fp));
+ goto out;
+ }
+
+ TEST_ASSERT(!strncmp(clk_name, "tsc\n", st.st_size),
+ "clocksource not supported: %s", clk_name);
+out:
+ fclose(fp);
+}
+
+int main(void)
+{
+ struct kvm_cpuid2 *best;
+ vm_vaddr_t pvti_gva;
+ vm_paddr_t pvti_gpa;
+ struct kvm_vm *vm;
+ int flags;
+
+ flags = kvm_check_cap(KVM_CAP_ADJUST_CLOCK);
+ if (!(flags & KVM_CLOCK_REAL_TIME)) {
+ print_skip("KVM_CLOCK_REAL_TIME not supported; flags: %x",
+ flags);
+ exit(KSFT_SKIP);
+ }
+
+ check_clocksource();
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+
+ best = kvm_get_supported_cpuid();
+ vcpu_set_cpuid(vm, VCPU_ID, best);
+
+ pvti_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000);
+ pvti_gpa = addr_gva2gpa(vm, pvti_gva);
+ vcpu_args_set(vm, VCPU_ID, 2, pvti_gpa, pvti_gva);
+
+ enter_guest(vm);
+ kvm_vm_free(vm);
+}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread* Re: [PATCH v2 05/12] selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-21 14:58 ` Andrew Jones
-1 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 14:58 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:22PM +0000, Oliver Upton wrote:
> Add a selftest for the new KVM clock UAPI that was introduced. Ensure
> that the KVM clock is consistent between userspace and the guest, and
> that the difference in realtime will only ever cause the KVM clock to
> advance forward.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 1 +
> tools/testing/selftests/kvm/Makefile | 1 +
> .../testing/selftests/kvm/include/kvm_util.h | 2 +
> .../selftests/kvm/x86_64/kvm_clock_test.c | 210 ++++++++++++++++++
> 4 files changed, 214 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 06a351b4f93b..d0877d01e771 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -11,6 +11,7 @@
> /x86_64/emulator_error_test
> /x86_64/get_cpuid_test
> /x86_64/get_msr_index_features
> +/x86_64/kvm_clock_test
> /x86_64/kvm_pv_test
> /x86_64/hyperv_clock
> /x86_64/hyperv_cpuid
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index b853be2ae3c6..f7e24f334c6e 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
> +TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
> TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
> TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
> TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index 010b59b13917..a8ac5d52e17b 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -19,6 +19,8 @@
> #define KVM_DEV_PATH "/dev/kvm"
> #define KVM_MAX_VCPUS 512
>
> +#define NSEC_PER_SEC 1000000000L
> +
> /*
> * Callers of kvm_util only have an incomplete/opaque description of the
> * structure kvm_util is using to maintain the state of a VM.
> diff --git a/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
> new file mode 100644
> index 000000000000..34c48f2dde54
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
> @@ -0,0 +1,210 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021, Google LLC.
> + *
> + * Tests for adjusting the KVM clock from userspace
> + */
> +#include <asm/kvm_para.h>
> +#include <asm/pvclock.h>
> +#include <asm/pvclock-abi.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/stat.h>
> +#include <time.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +
> +#define VCPU_ID 0
> +
> +struct test_case {
> + uint64_t kvmclock_base;
> + int64_t realtime_offset;
> +};
> +
> +static struct test_case test_cases[] = {
> + { .kvmclock_base = 0 },
> + { .kvmclock_base = 180 * NSEC_PER_SEC },
> + { .kvmclock_base = 0, .realtime_offset = -180 * NSEC_PER_SEC },
> + { .kvmclock_base = 0, .realtime_offset = 180 * NSEC_PER_SEC },
> +};
> +
> +#define GUEST_SYNC_CLOCK(__stage, __val) \
> + GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
> +
> +static void guest_main(vm_paddr_t pvti_pa, struct pvclock_vcpu_time_info *pvti)
> +{
> + int i;
> +
> + wrmsr(MSR_KVM_SYSTEM_TIME_NEW, pvti_pa | KVM_MSR_ENABLED);
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + GUEST_SYNC_CLOCK(i, __pvclock_read_cycles(pvti, rdtsc()));
> + }
> +
> + GUEST_DONE();
> +}
> +
> +#define EXPECTED_FLAGS (KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
> +
> +static inline void assert_flags(struct kvm_clock_data *data)
> +{
> + TEST_ASSERT((data->flags & EXPECTED_FLAGS) == EXPECTED_FLAGS,
> + "unexpected clock data flags: %x (want set: %x)",
> + data->flags, EXPECTED_FLAGS);
> +}
> +
> +static void handle_sync(struct ucall *uc, struct kvm_clock_data *start,
> + struct kvm_clock_data *end)
> +{
> + uint64_t obs, exp_lo, exp_hi;
> +
> + obs = uc->args[2];
> + exp_lo = start->clock;
> + exp_hi = end->clock;
> +
> + assert_flags(start);
> + assert_flags(end);
> +
> + TEST_ASSERT(exp_lo <= obs && obs <= exp_hi,
> + "unexpected kvm-clock value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
> + obs, exp_lo, exp_hi);
> +
> + pr_info("kvm-clock value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
> + obs, exp_lo, exp_hi);
> +}
> +
> +static void handle_abort(struct ucall *uc)
> +{
> + TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
> + __FILE__, uc->args[1]);
> +}
> +
> +static void setup_clock(struct kvm_vm *vm, struct test_case *test_case)
> +{
> + struct kvm_clock_data data;
> +
> + memset(&data, 0, sizeof(data));
> +
> + data.clock = test_case->kvmclock_base;
> + if (test_case->realtime_offset) {
> + struct timespec ts;
> + int r;
> +
> + data.flags |= KVM_CLOCK_REAL_TIME;
> + do {
> + r = clock_gettime(CLOCK_REALTIME, &ts);
> + if (!r)
> + break;
> + } while (errno == EINTR);
> +
> + TEST_ASSERT(!r, "clock_gettime() failed: %d\n", r);
> +
> + data.realtime = ts.tv_sec * NSEC_PER_SEC;
> + data.realtime += ts.tv_nsec;
> + data.realtime += test_case->realtime_offset;
> + }
> +
> + vm_ioctl(vm, KVM_SET_CLOCK, &data);
> +}
> +
> +static void enter_guest(struct kvm_vm *vm)
> +{
> + struct kvm_clock_data start, end;
> + struct kvm_run *run;
> + struct ucall uc;
> + int i, r;
> +
> + run = vcpu_state(vm, VCPU_ID);
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + setup_clock(vm, &test_cases[i]);
> +
> + vm_ioctl(vm, KVM_GET_CLOCK, &start);
> +
> + r = _vcpu_run(vm, VCPU_ID);
> + vm_ioctl(vm, KVM_GET_CLOCK, &end);
> +
> + TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
> + TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
> + "unexpected exit reason: %u (%s)",
> + run->exit_reason, exit_reason_str(run->exit_reason));
> +
> + switch (get_ucall(vm, VCPU_ID, &uc)) {
> + case UCALL_SYNC:
> + handle_sync(&uc, &start, &end);
> + break;
> + case UCALL_ABORT:
> + handle_abort(&uc);
> + return;
> + case UCALL_DONE:
> + return;
> + }
> + }
> +}
> +
> +#define CLOCKSOURCE_PATH "/sys/devices/system/clocksource/clocksource0/current_clocksource"
> +
> +static void check_clocksource(void)
> +{
> + char *clk_name;
> + struct stat st;
> + FILE *fp;
> +
> + fp = fopen(CLOCKSOURCE_PATH, "r");
> + if (!fp) {
> + pr_info("failed to open clocksource file: %d; assuming TSC.\n",
> + errno);
> + return;
> + }
> +
> + if (fstat(fileno(fp), &st)) {
> + pr_info("failed to stat clocksource file: %d; assuming TSC.\n",
> + errno);
> + goto out;
> + }
> +
> + clk_name = malloc(st.st_size);
> + TEST_ASSERT(clk_name, "failed to allocate buffer to read file\n");
> +
> + if (!fgets(clk_name, st.st_size, fp)) {
> + pr_info("failed to read clocksource file: %d; assuming TSC.\n",
> + ferror(fp));
> + goto out;
> + }
> +
> + TEST_ASSERT(!strncmp(clk_name, "tsc\n", st.st_size),
> + "clocksource not supported: %s", clk_name);
> +out:
> + fclose(fp);
> +}
> +
> +int main(void)
> +{
> + struct kvm_cpuid2 *best;
> + vm_vaddr_t pvti_gva;
> + vm_paddr_t pvti_gpa;
> + struct kvm_vm *vm;
> + int flags;
> +
> + flags = kvm_check_cap(KVM_CAP_ADJUST_CLOCK);
> + if (!(flags & KVM_CLOCK_REAL_TIME)) {
> + print_skip("KVM_CLOCK_REAL_TIME not supported; flags: %x",
> + flags);
> + exit(KSFT_SKIP);
> + }
> +
> + check_clocksource();
> +
> + vm = vm_create_default(VCPU_ID, 0, guest_main);
> +
> + best = kvm_get_supported_cpuid();
> + vcpu_set_cpuid(vm, VCPU_ID, best);
Default vm types (which use default vcpu types) will already set cpuids
like this for x86. See the bottom of
lib/x86_64/processor.c:vm_vcpu_add_default
Thanks,
drew
> +
> + pvti_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000);
> + pvti_gpa = addr_gva2gpa(vm, pvti_gva);
> + vcpu_args_set(vm, VCPU_ID, 2, pvti_gpa, pvti_gva);
> +
> + enter_guest(vm);
> + kvm_vm_free(vm);
> +}
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 05/12] selftests: KVM: Add test for KVM_{GET, SET}_CLOCK
@ 2021-07-21 14:58 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 14:58 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:22PM +0000, Oliver Upton wrote:
> Add a selftest for the new KVM clock UAPI that was introduced. Ensure
> that the KVM clock is consistent between userspace and the guest, and
> that the difference in realtime will only ever cause the KVM clock to
> advance forward.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 1 +
> tools/testing/selftests/kvm/Makefile | 1 +
> .../testing/selftests/kvm/include/kvm_util.h | 2 +
> .../selftests/kvm/x86_64/kvm_clock_test.c | 210 ++++++++++++++++++
> 4 files changed, 214 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 06a351b4f93b..d0877d01e771 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -11,6 +11,7 @@
> /x86_64/emulator_error_test
> /x86_64/get_cpuid_test
> /x86_64/get_msr_index_features
> +/x86_64/kvm_clock_test
> /x86_64/kvm_pv_test
> /x86_64/hyperv_clock
> /x86_64/hyperv_cpuid
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index b853be2ae3c6..f7e24f334c6e 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
> +TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
> TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
> TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
> TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index 010b59b13917..a8ac5d52e17b 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -19,6 +19,8 @@
> #define KVM_DEV_PATH "/dev/kvm"
> #define KVM_MAX_VCPUS 512
>
> +#define NSEC_PER_SEC 1000000000L
> +
> /*
> * Callers of kvm_util only have an incomplete/opaque description of the
> * structure kvm_util is using to maintain the state of a VM.
> diff --git a/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
> new file mode 100644
> index 000000000000..34c48f2dde54
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
> @@ -0,0 +1,210 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021, Google LLC.
> + *
> + * Tests for adjusting the KVM clock from userspace
> + */
> +#include <asm/kvm_para.h>
> +#include <asm/pvclock.h>
> +#include <asm/pvclock-abi.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/stat.h>
> +#include <time.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +
> +#define VCPU_ID 0
> +
> +struct test_case {
> + uint64_t kvmclock_base;
> + int64_t realtime_offset;
> +};
> +
> +static struct test_case test_cases[] = {
> + { .kvmclock_base = 0 },
> + { .kvmclock_base = 180 * NSEC_PER_SEC },
> + { .kvmclock_base = 0, .realtime_offset = -180 * NSEC_PER_SEC },
> + { .kvmclock_base = 0, .realtime_offset = 180 * NSEC_PER_SEC },
> +};
> +
> +#define GUEST_SYNC_CLOCK(__stage, __val) \
> + GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
> +
> +static void guest_main(vm_paddr_t pvti_pa, struct pvclock_vcpu_time_info *pvti)
> +{
> + int i;
> +
> + wrmsr(MSR_KVM_SYSTEM_TIME_NEW, pvti_pa | KVM_MSR_ENABLED);
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + GUEST_SYNC_CLOCK(i, __pvclock_read_cycles(pvti, rdtsc()));
> + }
> +
> + GUEST_DONE();
> +}
> +
> +#define EXPECTED_FLAGS (KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
> +
> +static inline void assert_flags(struct kvm_clock_data *data)
> +{
> + TEST_ASSERT((data->flags & EXPECTED_FLAGS) == EXPECTED_FLAGS,
> + "unexpected clock data flags: %x (want set: %x)",
> + data->flags, EXPECTED_FLAGS);
> +}
> +
> +static void handle_sync(struct ucall *uc, struct kvm_clock_data *start,
> + struct kvm_clock_data *end)
> +{
> + uint64_t obs, exp_lo, exp_hi;
> +
> + obs = uc->args[2];
> + exp_lo = start->clock;
> + exp_hi = end->clock;
> +
> + assert_flags(start);
> + assert_flags(end);
> +
> + TEST_ASSERT(exp_lo <= obs && obs <= exp_hi,
> + "unexpected kvm-clock value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
> + obs, exp_lo, exp_hi);
> +
> + pr_info("kvm-clock value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
> + obs, exp_lo, exp_hi);
> +}
> +
> +static void handle_abort(struct ucall *uc)
> +{
> + TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
> + __FILE__, uc->args[1]);
> +}
> +
> +static void setup_clock(struct kvm_vm *vm, struct test_case *test_case)
> +{
> + struct kvm_clock_data data;
> +
> + memset(&data, 0, sizeof(data));
> +
> + data.clock = test_case->kvmclock_base;
> + if (test_case->realtime_offset) {
> + struct timespec ts;
> + int r;
> +
> + data.flags |= KVM_CLOCK_REAL_TIME;
> + do {
> + r = clock_gettime(CLOCK_REALTIME, &ts);
> + if (!r)
> + break;
> + } while (errno == EINTR);
> +
> + TEST_ASSERT(!r, "clock_gettime() failed: %d\n", r);
> +
> + data.realtime = ts.tv_sec * NSEC_PER_SEC;
> + data.realtime += ts.tv_nsec;
> + data.realtime += test_case->realtime_offset;
> + }
> +
> + vm_ioctl(vm, KVM_SET_CLOCK, &data);
> +}
> +
> +static void enter_guest(struct kvm_vm *vm)
> +{
> + struct kvm_clock_data start, end;
> + struct kvm_run *run;
> + struct ucall uc;
> + int i, r;
> +
> + run = vcpu_state(vm, VCPU_ID);
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + setup_clock(vm, &test_cases[i]);
> +
> + vm_ioctl(vm, KVM_GET_CLOCK, &start);
> +
> + r = _vcpu_run(vm, VCPU_ID);
> + vm_ioctl(vm, KVM_GET_CLOCK, &end);
> +
> + TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
> + TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
> + "unexpected exit reason: %u (%s)",
> + run->exit_reason, exit_reason_str(run->exit_reason));
> +
> + switch (get_ucall(vm, VCPU_ID, &uc)) {
> + case UCALL_SYNC:
> + handle_sync(&uc, &start, &end);
> + break;
> + case UCALL_ABORT:
> + handle_abort(&uc);
> + return;
> + case UCALL_DONE:
> + return;
> + }
> + }
> +}
> +
> +#define CLOCKSOURCE_PATH "/sys/devices/system/clocksource/clocksource0/current_clocksource"
> +
> +static void check_clocksource(void)
> +{
> + char *clk_name;
> + struct stat st;
> + FILE *fp;
> +
> + fp = fopen(CLOCKSOURCE_PATH, "r");
> + if (!fp) {
> + pr_info("failed to open clocksource file: %d; assuming TSC.\n",
> + errno);
> + return;
> + }
> +
> + if (fstat(fileno(fp), &st)) {
> + pr_info("failed to stat clocksource file: %d; assuming TSC.\n",
> + errno);
> + goto out;
> + }
> +
> + clk_name = malloc(st.st_size);
> + TEST_ASSERT(clk_name, "failed to allocate buffer to read file\n");
> +
> + if (!fgets(clk_name, st.st_size, fp)) {
> + pr_info("failed to read clocksource file: %d; assuming TSC.\n",
> + ferror(fp));
> + goto out;
> + }
> +
> + TEST_ASSERT(!strncmp(clk_name, "tsc\n", st.st_size),
> + "clocksource not supported: %s", clk_name);
> +out:
> + fclose(fp);
> +}
> +
> +int main(void)
> +{
> + struct kvm_cpuid2 *best;
> + vm_vaddr_t pvti_gva;
> + vm_paddr_t pvti_gpa;
> + struct kvm_vm *vm;
> + int flags;
> +
> + flags = kvm_check_cap(KVM_CAP_ADJUST_CLOCK);
> + if (!(flags & KVM_CLOCK_REAL_TIME)) {
> + print_skip("KVM_CLOCK_REAL_TIME not supported; flags: %x",
> + flags);
> + exit(KSFT_SKIP);
> + }
> +
> + check_clocksource();
> +
> + vm = vm_create_default(VCPU_ID, 0, guest_main);
> +
> + best = kvm_get_supported_cpuid();
> + vcpu_set_cpuid(vm, VCPU_ID, best);
Default vm types (which use default vcpu types) will already set cpuids
like this for x86. See the bottom of
lib/x86_64/processor.c:vm_vcpu_add_default
Thanks,
drew
> +
> + pvti_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000);
> + pvti_gpa = addr_gva2gpa(vm, pvti_gva);
> + vcpu_args_set(vm, VCPU_ID, 2, pvti_gpa, pvti_gva);
> +
> + enter_guest(vm);
> + kvm_vm_free(vm);
> +}
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 05/12] selftests: KVM: Add test for KVM_{GET, SET}_CLOCK
@ 2021-07-21 14:58 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 14:58 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini, kvmarm,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:22PM +0000, Oliver Upton wrote:
> Add a selftest for the new KVM clock UAPI that was introduced. Ensure
> that the KVM clock is consistent between userspace and the guest, and
> that the difference in realtime will only ever cause the KVM clock to
> advance forward.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 1 +
> tools/testing/selftests/kvm/Makefile | 1 +
> .../testing/selftests/kvm/include/kvm_util.h | 2 +
> .../selftests/kvm/x86_64/kvm_clock_test.c | 210 ++++++++++++++++++
> 4 files changed, 214 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 06a351b4f93b..d0877d01e771 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -11,6 +11,7 @@
> /x86_64/emulator_error_test
> /x86_64/get_cpuid_test
> /x86_64/get_msr_index_features
> +/x86_64/kvm_clock_test
> /x86_64/kvm_pv_test
> /x86_64/hyperv_clock
> /x86_64/hyperv_cpuid
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index b853be2ae3c6..f7e24f334c6e 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
> TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
> +TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
> TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
> TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
> TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index 010b59b13917..a8ac5d52e17b 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -19,6 +19,8 @@
> #define KVM_DEV_PATH "/dev/kvm"
> #define KVM_MAX_VCPUS 512
>
> +#define NSEC_PER_SEC 1000000000L
> +
> /*
> * Callers of kvm_util only have an incomplete/opaque description of the
> * structure kvm_util is using to maintain the state of a VM.
> diff --git a/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
> new file mode 100644
> index 000000000000..34c48f2dde54
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
> @@ -0,0 +1,210 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021, Google LLC.
> + *
> + * Tests for adjusting the KVM clock from userspace
> + */
> +#include <asm/kvm_para.h>
> +#include <asm/pvclock.h>
> +#include <asm/pvclock-abi.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/stat.h>
> +#include <time.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +
> +#define VCPU_ID 0
> +
> +struct test_case {
> + uint64_t kvmclock_base;
> + int64_t realtime_offset;
> +};
> +
> +static struct test_case test_cases[] = {
> + { .kvmclock_base = 0 },
> + { .kvmclock_base = 180 * NSEC_PER_SEC },
> + { .kvmclock_base = 0, .realtime_offset = -180 * NSEC_PER_SEC },
> + { .kvmclock_base = 0, .realtime_offset = 180 * NSEC_PER_SEC },
> +};
> +
> +#define GUEST_SYNC_CLOCK(__stage, __val) \
> + GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
> +
> +static void guest_main(vm_paddr_t pvti_pa, struct pvclock_vcpu_time_info *pvti)
> +{
> + int i;
> +
> + wrmsr(MSR_KVM_SYSTEM_TIME_NEW, pvti_pa | KVM_MSR_ENABLED);
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + GUEST_SYNC_CLOCK(i, __pvclock_read_cycles(pvti, rdtsc()));
> + }
> +
> + GUEST_DONE();
> +}
> +
> +#define EXPECTED_FLAGS (KVM_CLOCK_REAL_TIME | KVM_CLOCK_HOST_TSC)
> +
> +static inline void assert_flags(struct kvm_clock_data *data)
> +{
> + TEST_ASSERT((data->flags & EXPECTED_FLAGS) == EXPECTED_FLAGS,
> + "unexpected clock data flags: %x (want set: %x)",
> + data->flags, EXPECTED_FLAGS);
> +}
> +
> +static void handle_sync(struct ucall *uc, struct kvm_clock_data *start,
> + struct kvm_clock_data *end)
> +{
> + uint64_t obs, exp_lo, exp_hi;
> +
> + obs = uc->args[2];
> + exp_lo = start->clock;
> + exp_hi = end->clock;
> +
> + assert_flags(start);
> + assert_flags(end);
> +
> + TEST_ASSERT(exp_lo <= obs && obs <= exp_hi,
> + "unexpected kvm-clock value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
> + obs, exp_lo, exp_hi);
> +
> + pr_info("kvm-clock value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
> + obs, exp_lo, exp_hi);
> +}
> +
> +static void handle_abort(struct ucall *uc)
> +{
> + TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
> + __FILE__, uc->args[1]);
> +}
> +
> +static void setup_clock(struct kvm_vm *vm, struct test_case *test_case)
> +{
> + struct kvm_clock_data data;
> +
> + memset(&data, 0, sizeof(data));
> +
> + data.clock = test_case->kvmclock_base;
> + if (test_case->realtime_offset) {
> + struct timespec ts;
> + int r;
> +
> + data.flags |= KVM_CLOCK_REAL_TIME;
> + do {
> + r = clock_gettime(CLOCK_REALTIME, &ts);
> + if (!r)
> + break;
> + } while (errno == EINTR);
> +
> + TEST_ASSERT(!r, "clock_gettime() failed: %d\n", r);
> +
> + data.realtime = ts.tv_sec * NSEC_PER_SEC;
> + data.realtime += ts.tv_nsec;
> + data.realtime += test_case->realtime_offset;
> + }
> +
> + vm_ioctl(vm, KVM_SET_CLOCK, &data);
> +}
> +
> +static void enter_guest(struct kvm_vm *vm)
> +{
> + struct kvm_clock_data start, end;
> + struct kvm_run *run;
> + struct ucall uc;
> + int i, r;
> +
> + run = vcpu_state(vm, VCPU_ID);
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + setup_clock(vm, &test_cases[i]);
> +
> + vm_ioctl(vm, KVM_GET_CLOCK, &start);
> +
> + r = _vcpu_run(vm, VCPU_ID);
> + vm_ioctl(vm, KVM_GET_CLOCK, &end);
> +
> + TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
> + TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
> + "unexpected exit reason: %u (%s)",
> + run->exit_reason, exit_reason_str(run->exit_reason));
> +
> + switch (get_ucall(vm, VCPU_ID, &uc)) {
> + case UCALL_SYNC:
> + handle_sync(&uc, &start, &end);
> + break;
> + case UCALL_ABORT:
> + handle_abort(&uc);
> + return;
> + case UCALL_DONE:
> + return;
> + }
> + }
> +}
> +
> +#define CLOCKSOURCE_PATH "/sys/devices/system/clocksource/clocksource0/current_clocksource"
> +
> +static void check_clocksource(void)
> +{
> + char *clk_name;
> + struct stat st;
> + FILE *fp;
> +
> + fp = fopen(CLOCKSOURCE_PATH, "r");
> + if (!fp) {
> + pr_info("failed to open clocksource file: %d; assuming TSC.\n",
> + errno);
> + return;
> + }
> +
> + if (fstat(fileno(fp), &st)) {
> + pr_info("failed to stat clocksource file: %d; assuming TSC.\n",
> + errno);
> + goto out;
> + }
> +
> + clk_name = malloc(st.st_size);
> + TEST_ASSERT(clk_name, "failed to allocate buffer to read file\n");
> +
> + if (!fgets(clk_name, st.st_size, fp)) {
> + pr_info("failed to read clocksource file: %d; assuming TSC.\n",
> + ferror(fp));
> + goto out;
> + }
> +
> + TEST_ASSERT(!strncmp(clk_name, "tsc\n", st.st_size),
> + "clocksource not supported: %s", clk_name);
> +out:
> + fclose(fp);
> +}
> +
> +int main(void)
> +{
> + struct kvm_cpuid2 *best;
> + vm_vaddr_t pvti_gva;
> + vm_paddr_t pvti_gpa;
> + struct kvm_vm *vm;
> + int flags;
> +
> + flags = kvm_check_cap(KVM_CAP_ADJUST_CLOCK);
> + if (!(flags & KVM_CLOCK_REAL_TIME)) {
> + print_skip("KVM_CLOCK_REAL_TIME not supported; flags: %x",
> + flags);
> + exit(KSFT_SKIP);
> + }
> +
> + check_clocksource();
> +
> + vm = vm_create_default(VCPU_ID, 0, guest_main);
> +
> + best = kvm_get_supported_cpuid();
> + vcpu_set_cpuid(vm, VCPU_ID, best);
Default vm types (which use default vcpu types) will already set cpuids
like this for x86. See the bottom of
lib/x86_64/processor.c:vm_vcpu_add_default
Thanks,
drew
> +
> + pvti_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000);
> + pvti_gpa = addr_gva2gpa(vm, pvti_gva);
> + vcpu_args_set(vm, VCPU_ID, 2, pvti_gpa, pvti_gva);
> +
> + enter_guest(vm);
> + kvm_vm_free(vm);
> +}
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 06/12] selftests: KVM: Add helpers for vCPU device attributes
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
vCPU file descriptors are abstracted away from test code in KVM
selftests, meaning that tests cannot directly access a vCPU's device
attributes. Add helpers that tests can use to get at vCPU device
attributes.
Signed-off-by: Oliver Upton <oupton@google.com>
---
.../testing/selftests/kvm/include/kvm_util.h | 9 +++++
tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index a8ac5d52e17b..1b3ef5757819 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
void *val, bool write);
+int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr);
+int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr);
+int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write);
+int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write);
+
const char *exit_reason_str(unsigned int exit_reason);
void virt_pgd_alloc(struct kvm_vm *vm);
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 10a8ed691c66..b595e7dc3fc5 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
return ret;
}
+int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr)
+{
+ struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+
+ TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
+
+ return _kvm_device_check_attr(vcpu->fd, group, attr);
+}
+
+int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr)
+{
+ int ret = _vcpu_has_device_attr(vm, vcpuid, group, attr);
+
+ TEST_ASSERT(!ret, "KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
+ return ret;
+}
+
+int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write)
+{
+ struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+
+ TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
+
+ return _kvm_device_access(vcpu->fd, group, attr, val, write);
+}
+
+int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write)
+{
+ int ret = _vcpu_access_device_attr(vm, vcpuid, group, attr, val, write);
+
+ TEST_ASSERT(!ret, "KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
+ return ret;
+}
+
/*
* VM Dump
*
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 06/12] selftests: KVM: Add helpers for vCPU device attributes
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
vCPU file descriptors are abstracted away from test code in KVM
selftests, meaning that tests cannot directly access a vCPU's device
attributes. Add helpers that tests can use to get at vCPU device
attributes.
Signed-off-by: Oliver Upton <oupton@google.com>
---
.../testing/selftests/kvm/include/kvm_util.h | 9 +++++
tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index a8ac5d52e17b..1b3ef5757819 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
void *val, bool write);
+int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr);
+int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr);
+int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write);
+int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write);
+
const char *exit_reason_str(unsigned int exit_reason);
void virt_pgd_alloc(struct kvm_vm *vm);
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 10a8ed691c66..b595e7dc3fc5 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
return ret;
}
+int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr)
+{
+ struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+
+ TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
+
+ return _kvm_device_check_attr(vcpu->fd, group, attr);
+}
+
+int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr)
+{
+ int ret = _vcpu_has_device_attr(vm, vcpuid, group, attr);
+
+ TEST_ASSERT(!ret, "KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
+ return ret;
+}
+
+int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write)
+{
+ struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+
+ TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
+
+ return _kvm_device_access(vcpu->fd, group, attr, val, write);
+}
+
+int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write)
+{
+ int ret = _vcpu_access_device_attr(vm, vcpuid, group, attr, val, write);
+
+ TEST_ASSERT(!ret, "KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
+ return ret;
+}
+
/*
* VM Dump
*
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 06/12] selftests: KVM: Add helpers for vCPU device attributes
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
vCPU file descriptors are abstracted away from test code in KVM
selftests, meaning that tests cannot directly access a vCPU's device
attributes. Add helpers that tests can use to get at vCPU device
attributes.
Signed-off-by: Oliver Upton <oupton@google.com>
---
.../testing/selftests/kvm/include/kvm_util.h | 9 +++++
tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index a8ac5d52e17b..1b3ef5757819 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
void *val, bool write);
+int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr);
+int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr);
+int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write);
+int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write);
+
const char *exit_reason_str(unsigned int exit_reason);
void virt_pgd_alloc(struct kvm_vm *vm);
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 10a8ed691c66..b595e7dc3fc5 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
return ret;
}
+int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr)
+{
+ struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+
+ TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
+
+ return _kvm_device_check_attr(vcpu->fd, group, attr);
+}
+
+int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr)
+{
+ int ret = _vcpu_has_device_attr(vm, vcpuid, group, attr);
+
+ TEST_ASSERT(!ret, "KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
+ return ret;
+}
+
+int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write)
+{
+ struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+
+ TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
+
+ return _kvm_device_access(vcpu->fd, group, attr, val, write);
+}
+
+int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
+ uint64_t attr, void *val, bool write)
+{
+ int ret = _vcpu_access_device_attr(vm, vcpuid, group, attr, val, write);
+
+ TEST_ASSERT(!ret, "KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
+ return ret;
+}
+
/*
* VM Dump
*
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread* Re: [PATCH v2 06/12] selftests: KVM: Add helpers for vCPU device attributes
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-21 15:14 ` Andrew Jones
-1 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:14 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:23PM +0000, Oliver Upton wrote:
> vCPU file descriptors are abstracted away from test code in KVM
> selftests, meaning that tests cannot directly access a vCPU's device
> attributes. Add helpers that tests can use to get at vCPU device
> attributes.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> .../testing/selftests/kvm/include/kvm_util.h | 9 +++++
> tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index a8ac5d52e17b..1b3ef5757819 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> void *val, bool write);
>
> +int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr);
> +int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr);
> +int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write);
> +int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write);
> +
> const char *exit_reason_str(unsigned int exit_reason);
>
> void virt_pgd_alloc(struct kvm_vm *vm);
> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index 10a8ed691c66..b595e7dc3fc5 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> return ret;
> }
>
> +int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr)
> +{
> + struct vcpu *vcpu = vcpu_find(vm, vcpuid);
> +
> + TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
> +
> + return _kvm_device_check_attr(vcpu->fd, group, attr);
> +}
> +
> +int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr)
> +{
> + int ret = _vcpu_has_device_attr(vm, vcpuid, group, attr);
> +
> + TEST_ASSERT(!ret, "KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
> + return ret;
> +}
> +
> +int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write)
> +{
> + struct vcpu *vcpu = vcpu_find(vm, vcpuid);
> +
> + TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
> +
> + return _kvm_device_access(vcpu->fd, group, attr, val, write);
> +}
> +
> +int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write)
> +{
> + int ret = _vcpu_access_device_attr(vm, vcpuid, group, attr, val, write);
> +
> + TEST_ASSERT(!ret, "KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
> + return ret;
> +}
Reviewed-by: Andrew Jones <drjones@redhat.com>
The 'assert !ret's are correct here. I see they are not correct in
kvm_device_check_attr
kvm_create_device
kvm_device_access
though, as they are 'assert ret >= 0', but the documentation says 0 on
success. It'd be nice to get that fixed before we build more API on top
of it.
Thanks,
drew
> +
> /*
> * VM Dump
> *
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 06/12] selftests: KVM: Add helpers for vCPU device attributes
@ 2021-07-21 15:14 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:14 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:23PM +0000, Oliver Upton wrote:
> vCPU file descriptors are abstracted away from test code in KVM
> selftests, meaning that tests cannot directly access a vCPU's device
> attributes. Add helpers that tests can use to get at vCPU device
> attributes.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> .../testing/selftests/kvm/include/kvm_util.h | 9 +++++
> tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index a8ac5d52e17b..1b3ef5757819 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> void *val, bool write);
>
> +int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr);
> +int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr);
> +int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write);
> +int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write);
> +
> const char *exit_reason_str(unsigned int exit_reason);
>
> void virt_pgd_alloc(struct kvm_vm *vm);
> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index 10a8ed691c66..b595e7dc3fc5 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> return ret;
> }
>
> +int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr)
> +{
> + struct vcpu *vcpu = vcpu_find(vm, vcpuid);
> +
> + TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
> +
> + return _kvm_device_check_attr(vcpu->fd, group, attr);
> +}
> +
> +int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr)
> +{
> + int ret = _vcpu_has_device_attr(vm, vcpuid, group, attr);
> +
> + TEST_ASSERT(!ret, "KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
> + return ret;
> +}
> +
> +int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write)
> +{
> + struct vcpu *vcpu = vcpu_find(vm, vcpuid);
> +
> + TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
> +
> + return _kvm_device_access(vcpu->fd, group, attr, val, write);
> +}
> +
> +int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write)
> +{
> + int ret = _vcpu_access_device_attr(vm, vcpuid, group, attr, val, write);
> +
> + TEST_ASSERT(!ret, "KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
> + return ret;
> +}
Reviewed-by: Andrew Jones <drjones@redhat.com>
The 'assert !ret's are correct here. I see they are not correct in
kvm_device_check_attr
kvm_create_device
kvm_device_access
though, as they are 'assert ret >= 0', but the documentation says 0 on
success. It'd be nice to get that fixed before we build more API on top
of it.
Thanks,
drew
> +
> /*
> * VM Dump
> *
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 06/12] selftests: KVM: Add helpers for vCPU device attributes
@ 2021-07-21 15:14 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:14 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini, kvmarm,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:23PM +0000, Oliver Upton wrote:
> vCPU file descriptors are abstracted away from test code in KVM
> selftests, meaning that tests cannot directly access a vCPU's device
> attributes. Add helpers that tests can use to get at vCPU device
> attributes.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> .../testing/selftests/kvm/include/kvm_util.h | 9 +++++
> tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index a8ac5d52e17b..1b3ef5757819 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> void *val, bool write);
>
> +int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr);
> +int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr);
> +int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write);
> +int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write);
> +
> const char *exit_reason_str(unsigned int exit_reason);
>
> void virt_pgd_alloc(struct kvm_vm *vm);
> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index 10a8ed691c66..b595e7dc3fc5 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
> return ret;
> }
>
> +int _vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr)
> +{
> + struct vcpu *vcpu = vcpu_find(vm, vcpuid);
> +
> + TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
> +
> + return _kvm_device_check_attr(vcpu->fd, group, attr);
> +}
> +
> +int vcpu_has_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr)
> +{
> + int ret = _vcpu_has_device_attr(vm, vcpuid, group, attr);
> +
> + TEST_ASSERT(!ret, "KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
> + return ret;
> +}
> +
> +int _vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write)
> +{
> + struct vcpu *vcpu = vcpu_find(vm, vcpuid);
> +
> + TEST_ASSERT(vcpu, "nonexistent vcpu id: %d", vcpuid);
> +
> + return _kvm_device_access(vcpu->fd, group, attr, val, write);
> +}
> +
> +int vcpu_access_device_attr(struct kvm_vm *vm, uint32_t vcpuid, uint32_t group,
> + uint64_t attr, void *val, bool write)
> +{
> + int ret = _vcpu_access_device_attr(vm, vcpuid, group, attr, val, write);
> +
> + TEST_ASSERT(!ret, "KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i", ret, errno);
> + return ret;
> +}
Reviewed-by: Andrew Jones <drjones@redhat.com>
The 'assert !ret's are correct here. I see they are not correct in
kvm_device_check_attr
kvm_create_device
kvm_device_access
though, as they are 'assert ret >= 0', but the documentation says 0 on
success. It'd be nice to get that fixed before we build more API on top
of it.
Thanks,
drew
> +
> /*
> * VM Dump
> *
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 07/12] selftests: KVM: Introduce system counter offset test
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Introduce a KVM selftest to verify that userspace manipulation of the
TSC (via the new vCPU attribute) results in the correct behavior within
the guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../kvm/system_counter_offset_test.c | 133 ++++++++++++++++++
3 files changed, 135 insertions(+)
create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index d0877d01e771..2752813d5090 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -50,3 +50,4 @@
/set_memory_region_test
/steal_time
/kvm_binary_stats_test
+/system_counter_offset_test
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index f7e24f334c6e..7bf2e5fb1d5a 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test
TEST_GEN_PROGS_x86_64 += set_memory_region_test
TEST_GEN_PROGS_x86_64 += steal_time
TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
+TEST_GEN_PROGS_x86_64 += system_counter_offset_test
TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
new file mode 100644
index 000000000000..7e9015770759
--- /dev/null
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Google LLC.
+ *
+ * Tests for adjusting the system counter from userspace
+ */
+#include <asm/kvm_para.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define VCPU_ID 0
+
+#ifdef __x86_64__
+
+struct test_case {
+ uint64_t tsc_offset;
+};
+
+static struct test_case test_cases[] = {
+ { 0 },
+ { 180 * NSEC_PER_SEC },
+ { -180 * NSEC_PER_SEC },
+};
+
+static void check_preconditions(struct kvm_vm *vm)
+{
+ if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL, KVM_VCPU_TSC_OFFSET))
+ return;
+
+ print_skip("KVM_VCPU_TSC_OFFSET not supported; skipping test");
+ exit(KSFT_SKIP);
+}
+
+static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL,
+ KVM_VCPU_TSC_OFFSET, &test->tsc_offset, true);
+}
+
+static uint64_t guest_read_system_counter(struct test_case *test)
+{
+ return rdtsc();
+}
+
+static uint64_t host_read_guest_system_counter(struct test_case *test)
+{
+ return rdtsc() + test->tsc_offset;
+}
+
+#else /* __x86_64__ */
+
+#error test not implemented for this architecture!
+
+#endif
+
+#define GUEST_SYNC_CLOCK(__stage, __val) \
+ GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
+
+static void guest_main(void)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ struct test_case *test = &test_cases[i];
+
+ GUEST_SYNC_CLOCK(i, guest_read_system_counter(test));
+ }
+
+ GUEST_DONE();
+}
+
+static void handle_sync(struct ucall *uc, uint64_t start, uint64_t end)
+{
+ uint64_t obs = uc->args[2];
+
+ TEST_ASSERT(start <= obs && obs <= end,
+ "unexpected system counter value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
+ obs, start, end);
+
+ pr_info("system counter value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
+ obs, start, end);
+}
+
+static void handle_abort(struct ucall *uc)
+{
+ TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
+ __FILE__, uc->args[1]);
+}
+
+static void enter_guest(struct kvm_vm *vm)
+{
+ uint64_t start, end;
+ struct ucall uc;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ struct test_case *test = &test_cases[i];
+
+ setup_system_counter(vm, test);
+ start = host_read_guest_system_counter(test);
+ vcpu_run(vm, VCPU_ID);
+ end = host_read_guest_system_counter(test);
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_SYNC:
+ handle_sync(&uc, start, end);
+ break;
+ case UCALL_ABORT:
+ handle_abort(&uc);
+ return;
+ case UCALL_DONE:
+ return;
+ }
+ }
+}
+
+int main(void)
+{
+ struct kvm_vm *vm;
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ check_preconditions(vm);
+ ucall_init(vm, NULL);
+
+ enter_guest(vm);
+ kvm_vm_free(vm);
+}
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 07/12] selftests: KVM: Introduce system counter offset test
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Introduce a KVM selftest to verify that userspace manipulation of the
TSC (via the new vCPU attribute) results in the correct behavior within
the guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../kvm/system_counter_offset_test.c | 133 ++++++++++++++++++
3 files changed, 135 insertions(+)
create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index d0877d01e771..2752813d5090 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -50,3 +50,4 @@
/set_memory_region_test
/steal_time
/kvm_binary_stats_test
+/system_counter_offset_test
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index f7e24f334c6e..7bf2e5fb1d5a 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test
TEST_GEN_PROGS_x86_64 += set_memory_region_test
TEST_GEN_PROGS_x86_64 += steal_time
TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
+TEST_GEN_PROGS_x86_64 += system_counter_offset_test
TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
new file mode 100644
index 000000000000..7e9015770759
--- /dev/null
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Google LLC.
+ *
+ * Tests for adjusting the system counter from userspace
+ */
+#include <asm/kvm_para.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define VCPU_ID 0
+
+#ifdef __x86_64__
+
+struct test_case {
+ uint64_t tsc_offset;
+};
+
+static struct test_case test_cases[] = {
+ { 0 },
+ { 180 * NSEC_PER_SEC },
+ { -180 * NSEC_PER_SEC },
+};
+
+static void check_preconditions(struct kvm_vm *vm)
+{
+ if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL, KVM_VCPU_TSC_OFFSET))
+ return;
+
+ print_skip("KVM_VCPU_TSC_OFFSET not supported; skipping test");
+ exit(KSFT_SKIP);
+}
+
+static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL,
+ KVM_VCPU_TSC_OFFSET, &test->tsc_offset, true);
+}
+
+static uint64_t guest_read_system_counter(struct test_case *test)
+{
+ return rdtsc();
+}
+
+static uint64_t host_read_guest_system_counter(struct test_case *test)
+{
+ return rdtsc() + test->tsc_offset;
+}
+
+#else /* __x86_64__ */
+
+#error test not implemented for this architecture!
+
+#endif
+
+#define GUEST_SYNC_CLOCK(__stage, __val) \
+ GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
+
+static void guest_main(void)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ struct test_case *test = &test_cases[i];
+
+ GUEST_SYNC_CLOCK(i, guest_read_system_counter(test));
+ }
+
+ GUEST_DONE();
+}
+
+static void handle_sync(struct ucall *uc, uint64_t start, uint64_t end)
+{
+ uint64_t obs = uc->args[2];
+
+ TEST_ASSERT(start <= obs && obs <= end,
+ "unexpected system counter value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
+ obs, start, end);
+
+ pr_info("system counter value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
+ obs, start, end);
+}
+
+static void handle_abort(struct ucall *uc)
+{
+ TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
+ __FILE__, uc->args[1]);
+}
+
+static void enter_guest(struct kvm_vm *vm)
+{
+ uint64_t start, end;
+ struct ucall uc;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ struct test_case *test = &test_cases[i];
+
+ setup_system_counter(vm, test);
+ start = host_read_guest_system_counter(test);
+ vcpu_run(vm, VCPU_ID);
+ end = host_read_guest_system_counter(test);
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_SYNC:
+ handle_sync(&uc, start, end);
+ break;
+ case UCALL_ABORT:
+ handle_abort(&uc);
+ return;
+ case UCALL_DONE:
+ return;
+ }
+ }
+}
+
+int main(void)
+{
+ struct kvm_vm *vm;
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ check_preconditions(vm);
+ ucall_init(vm, NULL);
+
+ enter_guest(vm);
+ kvm_vm_free(vm);
+}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 07/12] selftests: KVM: Introduce system counter offset test
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Introduce a KVM selftest to verify that userspace manipulation of the
TSC (via the new vCPU attribute) results in the correct behavior within
the guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../kvm/system_counter_offset_test.c | 133 ++++++++++++++++++
3 files changed, 135 insertions(+)
create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index d0877d01e771..2752813d5090 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -50,3 +50,4 @@
/set_memory_region_test
/steal_time
/kvm_binary_stats_test
+/system_counter_offset_test
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index f7e24f334c6e..7bf2e5fb1d5a 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test
TEST_GEN_PROGS_x86_64 += set_memory_region_test
TEST_GEN_PROGS_x86_64 += steal_time
TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
+TEST_GEN_PROGS_x86_64 += system_counter_offset_test
TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
new file mode 100644
index 000000000000..7e9015770759
--- /dev/null
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Google LLC.
+ *
+ * Tests for adjusting the system counter from userspace
+ */
+#include <asm/kvm_para.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define VCPU_ID 0
+
+#ifdef __x86_64__
+
+struct test_case {
+ uint64_t tsc_offset;
+};
+
+static struct test_case test_cases[] = {
+ { 0 },
+ { 180 * NSEC_PER_SEC },
+ { -180 * NSEC_PER_SEC },
+};
+
+static void check_preconditions(struct kvm_vm *vm)
+{
+ if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL, KVM_VCPU_TSC_OFFSET))
+ return;
+
+ print_skip("KVM_VCPU_TSC_OFFSET not supported; skipping test");
+ exit(KSFT_SKIP);
+}
+
+static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL,
+ KVM_VCPU_TSC_OFFSET, &test->tsc_offset, true);
+}
+
+static uint64_t guest_read_system_counter(struct test_case *test)
+{
+ return rdtsc();
+}
+
+static uint64_t host_read_guest_system_counter(struct test_case *test)
+{
+ return rdtsc() + test->tsc_offset;
+}
+
+#else /* __x86_64__ */
+
+#error test not implemented for this architecture!
+
+#endif
+
+#define GUEST_SYNC_CLOCK(__stage, __val) \
+ GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
+
+static void guest_main(void)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ struct test_case *test = &test_cases[i];
+
+ GUEST_SYNC_CLOCK(i, guest_read_system_counter(test));
+ }
+
+ GUEST_DONE();
+}
+
+static void handle_sync(struct ucall *uc, uint64_t start, uint64_t end)
+{
+ uint64_t obs = uc->args[2];
+
+ TEST_ASSERT(start <= obs && obs <= end,
+ "unexpected system counter value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
+ obs, start, end);
+
+ pr_info("system counter value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
+ obs, start, end);
+}
+
+static void handle_abort(struct ucall *uc)
+{
+ TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
+ __FILE__, uc->args[1]);
+}
+
+static void enter_guest(struct kvm_vm *vm)
+{
+ uint64_t start, end;
+ struct ucall uc;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
+ struct test_case *test = &test_cases[i];
+
+ setup_system_counter(vm, test);
+ start = host_read_guest_system_counter(test);
+ vcpu_run(vm, VCPU_ID);
+ end = host_read_guest_system_counter(test);
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_SYNC:
+ handle_sync(&uc, start, end);
+ break;
+ case UCALL_ABORT:
+ handle_abort(&uc);
+ return;
+ case UCALL_DONE:
+ return;
+ }
+ }
+}
+
+int main(void)
+{
+ struct kvm_vm *vm;
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ check_preconditions(vm);
+ ucall_init(vm, NULL);
+
+ enter_guest(vm);
+ kvm_vm_free(vm);
+}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread* Re: [PATCH v2 07/12] selftests: KVM: Introduce system counter offset test
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-21 15:17 ` Andrew Jones
-1 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:17 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:24PM +0000, Oliver Upton wrote:
> Introduce a KVM selftest to verify that userspace manipulation of the
> TSC (via the new vCPU attribute) results in the correct behavior within
> the guest.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 1 +
> tools/testing/selftests/kvm/Makefile | 1 +
> .../kvm/system_counter_offset_test.c | 133 ++++++++++++++++++
> 3 files changed, 135 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index d0877d01e771..2752813d5090 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -50,3 +50,4 @@
> /set_memory_region_test
> /steal_time
> /kvm_binary_stats_test
> +/system_counter_offset_test
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index f7e24f334c6e..7bf2e5fb1d5a 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test
> TEST_GEN_PROGS_x86_64 += set_memory_region_test
> TEST_GEN_PROGS_x86_64 += steal_time
> TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
> +TEST_GEN_PROGS_x86_64 += system_counter_offset_test
>
> TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
> TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
> diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
> new file mode 100644
> index 000000000000..7e9015770759
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021, Google LLC.
> + *
> + * Tests for adjusting the system counter from userspace
> + */
> +#include <asm/kvm_para.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/stat.h>
> +#include <time.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +
> +#define VCPU_ID 0
> +
> +#ifdef __x86_64__
> +
> +struct test_case {
> + uint64_t tsc_offset;
> +};
> +
> +static struct test_case test_cases[] = {
> + { 0 },
> + { 180 * NSEC_PER_SEC },
> + { -180 * NSEC_PER_SEC },
> +};
> +
> +static void check_preconditions(struct kvm_vm *vm)
> +{
> + if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL, KVM_VCPU_TSC_OFFSET))
> + return;
> +
> + print_skip("KVM_VCPU_TSC_OFFSET not supported; skipping test");
> + exit(KSFT_SKIP);
> +}
> +
> +static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
> +{
> + vcpu_access_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL,
> + KVM_VCPU_TSC_OFFSET, &test->tsc_offset, true);
> +}
> +
> +static uint64_t guest_read_system_counter(struct test_case *test)
> +{
> + return rdtsc();
> +}
> +
> +static uint64_t host_read_guest_system_counter(struct test_case *test)
> +{
> + return rdtsc() + test->tsc_offset;
> +}
> +
> +#else /* __x86_64__ */
> +
> +#error test not implemented for this architecture!
> +
> +#endif
> +
> +#define GUEST_SYNC_CLOCK(__stage, __val) \
> + GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
> +
> +static void guest_main(void)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + struct test_case *test = &test_cases[i];
> +
> + GUEST_SYNC_CLOCK(i, guest_read_system_counter(test));
> + }
> +
> + GUEST_DONE();
> +}
> +
> +static void handle_sync(struct ucall *uc, uint64_t start, uint64_t end)
> +{
> + uint64_t obs = uc->args[2];
> +
> + TEST_ASSERT(start <= obs && obs <= end,
> + "unexpected system counter value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
> + obs, start, end);
> +
> + pr_info("system counter value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
> + obs, start, end);
> +}
> +
> +static void handle_abort(struct ucall *uc)
> +{
> + TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
> + __FILE__, uc->args[1]);
> +}
> +
> +static void enter_guest(struct kvm_vm *vm)
> +{
> + uint64_t start, end;
> + struct ucall uc;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + struct test_case *test = &test_cases[i];
> +
> + setup_system_counter(vm, test);
> + start = host_read_guest_system_counter(test);
> + vcpu_run(vm, VCPU_ID);
> + end = host_read_guest_system_counter(test);
> +
> + switch (get_ucall(vm, VCPU_ID, &uc)) {
> + case UCALL_SYNC:
> + handle_sync(&uc, start, end);
> + break;
> + case UCALL_ABORT:
> + handle_abort(&uc);
> + return;
> + case UCALL_DONE:
> + return;
> + }
> + }
> +}
> +
> +int main(void)
> +{
> + struct kvm_vm *vm;
> +
> + vm = vm_create_default(VCPU_ID, 0, guest_main);
> + check_preconditions(vm);
> + ucall_init(vm, NULL);
> +
> + enter_guest(vm);
> + kvm_vm_free(vm);
> +}
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 07/12] selftests: KVM: Introduce system counter offset test
@ 2021-07-21 15:17 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:17 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:24PM +0000, Oliver Upton wrote:
> Introduce a KVM selftest to verify that userspace manipulation of the
> TSC (via the new vCPU attribute) results in the correct behavior within
> the guest.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 1 +
> tools/testing/selftests/kvm/Makefile | 1 +
> .../kvm/system_counter_offset_test.c | 133 ++++++++++++++++++
> 3 files changed, 135 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index d0877d01e771..2752813d5090 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -50,3 +50,4 @@
> /set_memory_region_test
> /steal_time
> /kvm_binary_stats_test
> +/system_counter_offset_test
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index f7e24f334c6e..7bf2e5fb1d5a 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test
> TEST_GEN_PROGS_x86_64 += set_memory_region_test
> TEST_GEN_PROGS_x86_64 += steal_time
> TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
> +TEST_GEN_PROGS_x86_64 += system_counter_offset_test
>
> TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
> TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
> diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
> new file mode 100644
> index 000000000000..7e9015770759
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021, Google LLC.
> + *
> + * Tests for adjusting the system counter from userspace
> + */
> +#include <asm/kvm_para.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/stat.h>
> +#include <time.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +
> +#define VCPU_ID 0
> +
> +#ifdef __x86_64__
> +
> +struct test_case {
> + uint64_t tsc_offset;
> +};
> +
> +static struct test_case test_cases[] = {
> + { 0 },
> + { 180 * NSEC_PER_SEC },
> + { -180 * NSEC_PER_SEC },
> +};
> +
> +static void check_preconditions(struct kvm_vm *vm)
> +{
> + if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL, KVM_VCPU_TSC_OFFSET))
> + return;
> +
> + print_skip("KVM_VCPU_TSC_OFFSET not supported; skipping test");
> + exit(KSFT_SKIP);
> +}
> +
> +static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
> +{
> + vcpu_access_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL,
> + KVM_VCPU_TSC_OFFSET, &test->tsc_offset, true);
> +}
> +
> +static uint64_t guest_read_system_counter(struct test_case *test)
> +{
> + return rdtsc();
> +}
> +
> +static uint64_t host_read_guest_system_counter(struct test_case *test)
> +{
> + return rdtsc() + test->tsc_offset;
> +}
> +
> +#else /* __x86_64__ */
> +
> +#error test not implemented for this architecture!
> +
> +#endif
> +
> +#define GUEST_SYNC_CLOCK(__stage, __val) \
> + GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
> +
> +static void guest_main(void)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + struct test_case *test = &test_cases[i];
> +
> + GUEST_SYNC_CLOCK(i, guest_read_system_counter(test));
> + }
> +
> + GUEST_DONE();
> +}
> +
> +static void handle_sync(struct ucall *uc, uint64_t start, uint64_t end)
> +{
> + uint64_t obs = uc->args[2];
> +
> + TEST_ASSERT(start <= obs && obs <= end,
> + "unexpected system counter value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
> + obs, start, end);
> +
> + pr_info("system counter value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
> + obs, start, end);
> +}
> +
> +static void handle_abort(struct ucall *uc)
> +{
> + TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
> + __FILE__, uc->args[1]);
> +}
> +
> +static void enter_guest(struct kvm_vm *vm)
> +{
> + uint64_t start, end;
> + struct ucall uc;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + struct test_case *test = &test_cases[i];
> +
> + setup_system_counter(vm, test);
> + start = host_read_guest_system_counter(test);
> + vcpu_run(vm, VCPU_ID);
> + end = host_read_guest_system_counter(test);
> +
> + switch (get_ucall(vm, VCPU_ID, &uc)) {
> + case UCALL_SYNC:
> + handle_sync(&uc, start, end);
> + break;
> + case UCALL_ABORT:
> + handle_abort(&uc);
> + return;
> + case UCALL_DONE:
> + return;
> + }
> + }
> +}
> +
> +int main(void)
> +{
> + struct kvm_vm *vm;
> +
> + vm = vm_create_default(VCPU_ID, 0, guest_main);
> + check_preconditions(vm);
> + ucall_init(vm, NULL);
> +
> + enter_guest(vm);
> + kvm_vm_free(vm);
> +}
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 07/12] selftests: KVM: Introduce system counter offset test
@ 2021-07-21 15:17 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:17 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini, kvmarm,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:24PM +0000, Oliver Upton wrote:
> Introduce a KVM selftest to verify that userspace manipulation of the
> TSC (via the new vCPU attribute) results in the correct behavior within
> the guest.
>
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 1 +
> tools/testing/selftests/kvm/Makefile | 1 +
> .../kvm/system_counter_offset_test.c | 133 ++++++++++++++++++
> 3 files changed, 135 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index d0877d01e771..2752813d5090 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -50,3 +50,4 @@
> /set_memory_region_test
> /steal_time
> /kvm_binary_stats_test
> +/system_counter_offset_test
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index f7e24f334c6e..7bf2e5fb1d5a 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test
> TEST_GEN_PROGS_x86_64 += set_memory_region_test
> TEST_GEN_PROGS_x86_64 += steal_time
> TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
> +TEST_GEN_PROGS_x86_64 += system_counter_offset_test
>
> TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
> TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
> diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
> new file mode 100644
> index 000000000000..7e9015770759
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021, Google LLC.
> + *
> + * Tests for adjusting the system counter from userspace
> + */
> +#include <asm/kvm_para.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/stat.h>
> +#include <time.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +
> +#define VCPU_ID 0
> +
> +#ifdef __x86_64__
> +
> +struct test_case {
> + uint64_t tsc_offset;
> +};
> +
> +static struct test_case test_cases[] = {
> + { 0 },
> + { 180 * NSEC_PER_SEC },
> + { -180 * NSEC_PER_SEC },
> +};
> +
> +static void check_preconditions(struct kvm_vm *vm)
> +{
> + if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL, KVM_VCPU_TSC_OFFSET))
> + return;
> +
> + print_skip("KVM_VCPU_TSC_OFFSET not supported; skipping test");
> + exit(KSFT_SKIP);
> +}
> +
> +static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
> +{
> + vcpu_access_device_attr(vm, VCPU_ID, KVM_VCPU_TSC_CTRL,
> + KVM_VCPU_TSC_OFFSET, &test->tsc_offset, true);
> +}
> +
> +static uint64_t guest_read_system_counter(struct test_case *test)
> +{
> + return rdtsc();
> +}
> +
> +static uint64_t host_read_guest_system_counter(struct test_case *test)
> +{
> + return rdtsc() + test->tsc_offset;
> +}
> +
> +#else /* __x86_64__ */
> +
> +#error test not implemented for this architecture!
> +
> +#endif
> +
> +#define GUEST_SYNC_CLOCK(__stage, __val) \
> + GUEST_SYNC_ARGS(__stage, __val, 0, 0, 0)
> +
> +static void guest_main(void)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + struct test_case *test = &test_cases[i];
> +
> + GUEST_SYNC_CLOCK(i, guest_read_system_counter(test));
> + }
> +
> + GUEST_DONE();
> +}
> +
> +static void handle_sync(struct ucall *uc, uint64_t start, uint64_t end)
> +{
> + uint64_t obs = uc->args[2];
> +
> + TEST_ASSERT(start <= obs && obs <= end,
> + "unexpected system counter value: %"PRIu64" expected range: [%"PRIu64", %"PRIu64"]",
> + obs, start, end);
> +
> + pr_info("system counter value: %"PRIu64" expected range [%"PRIu64", %"PRIu64"]\n",
> + obs, start, end);
> +}
> +
> +static void handle_abort(struct ucall *uc)
> +{
> + TEST_FAIL("%s at %s:%ld", (const char *)uc->args[0],
> + __FILE__, uc->args[1]);
> +}
> +
> +static void enter_guest(struct kvm_vm *vm)
> +{
> + uint64_t start, end;
> + struct ucall uc;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(test_cases); i++) {
> + struct test_case *test = &test_cases[i];
> +
> + setup_system_counter(vm, test);
> + start = host_read_guest_system_counter(test);
> + vcpu_run(vm, VCPU_ID);
> + end = host_read_guest_system_counter(test);
> +
> + switch (get_ucall(vm, VCPU_ID, &uc)) {
> + case UCALL_SYNC:
> + handle_sync(&uc, start, end);
> + break;
> + case UCALL_ABORT:
> + handle_abort(&uc);
> + return;
> + case UCALL_DONE:
> + return;
> + }
> + }
> +}
> +
> +int main(void)
> +{
> + struct kvm_vm *vm;
> +
> + vm = vm_create_default(VCPU_ID, 0, guest_main);
> + check_preconditions(vm);
> + ucall_init(vm, NULL);
> +
> + enter_guest(vm);
> + kvm_vm_free(vm);
> +}
> --
> 2.32.0.402.g57bb445576-goog
>
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 08/12] KVM: arm64: Allow userspace to configure a vCPU's virtual offset
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Add a new vCPU attribute that allows userspace to directly manipulate
the virtual counter-timer offset. Exposing such an interface allows for
the precise migration of guest virtual counter-timers, as it is an
indepotent interface.
Uphold the existing behavior of writes to CNTVOFF_EL2 for this new
interface, wherein a write to a single vCPU is broadcasted to all vCPUs
within a VM.
Signed-off-by: Oliver Upton <oupton@google.com>
---
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/arch_timer.c | 68 ++++++++++++++++++++++++++++++-
2 files changed, 67 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index b3edde68bc3e..008d0518d2b1 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -365,6 +365,7 @@ struct kvm_arm_copy_mte_tags {
#define KVM_ARM_VCPU_TIMER_CTRL 1
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
+#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2
#define KVM_ARM_VCPU_PVTIME_CTRL 2
#define KVM_ARM_VCPU_PVTIME_IPA 0
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 3df67c127489..d2b1b13af658 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -1305,7 +1305,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq)
}
}
-int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+int kvm_arm_timer_set_attr_irq(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
int __user *uaddr = (int __user *)(long)attr->addr;
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
@@ -1338,7 +1338,39 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return 0;
}
-int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 offset;
+
+ if (get_user(offset, uaddr))
+ return -EFAULT;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ update_vtimer_cntvoff(vcpu, offset);
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
+int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
+ case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ return kvm_arm_timer_set_attr_irq(vcpu, attr);
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ return kvm_arm_timer_set_attr_offset(vcpu, attr);
+ }
+
+ return -ENXIO;
+}
+
+int kvm_arm_timer_get_attr_irq(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
int __user *uaddr = (int __user *)(long)attr->addr;
struct arch_timer_context *timer;
@@ -1359,11 +1391,43 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return put_user(irq, uaddr);
}
+int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ struct arch_timer_context *timer;
+ u64 offset;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ timer = vcpu_vtimer(vcpu);
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ offset = timer_get_offset(timer);
+ return put_user(offset, uaddr);
+}
+
+int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
+ case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ return kvm_arm_timer_get_attr_irq(vcpu, attr);
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ return kvm_arm_timer_get_attr_offset(vcpu, attr);
+ }
+
+ return -ENXIO;
+}
+
int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
return 0;
}
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 08/12] KVM: arm64: Allow userspace to configure a vCPU's virtual offset
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Add a new vCPU attribute that allows userspace to directly manipulate
the virtual counter-timer offset. Exposing such an interface allows for
the precise migration of guest virtual counter-timers, as it is an
indepotent interface.
Uphold the existing behavior of writes to CNTVOFF_EL2 for this new
interface, wherein a write to a single vCPU is broadcasted to all vCPUs
within a VM.
Signed-off-by: Oliver Upton <oupton@google.com>
---
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/arch_timer.c | 68 ++++++++++++++++++++++++++++++-
2 files changed, 67 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index b3edde68bc3e..008d0518d2b1 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -365,6 +365,7 @@ struct kvm_arm_copy_mte_tags {
#define KVM_ARM_VCPU_TIMER_CTRL 1
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
+#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2
#define KVM_ARM_VCPU_PVTIME_CTRL 2
#define KVM_ARM_VCPU_PVTIME_IPA 0
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 3df67c127489..d2b1b13af658 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -1305,7 +1305,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq)
}
}
-int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+int kvm_arm_timer_set_attr_irq(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
int __user *uaddr = (int __user *)(long)attr->addr;
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
@@ -1338,7 +1338,39 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return 0;
}
-int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 offset;
+
+ if (get_user(offset, uaddr))
+ return -EFAULT;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ update_vtimer_cntvoff(vcpu, offset);
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
+int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
+ case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ return kvm_arm_timer_set_attr_irq(vcpu, attr);
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ return kvm_arm_timer_set_attr_offset(vcpu, attr);
+ }
+
+ return -ENXIO;
+}
+
+int kvm_arm_timer_get_attr_irq(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
int __user *uaddr = (int __user *)(long)attr->addr;
struct arch_timer_context *timer;
@@ -1359,11 +1391,43 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return put_user(irq, uaddr);
}
+int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ struct arch_timer_context *timer;
+ u64 offset;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ timer = vcpu_vtimer(vcpu);
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ offset = timer_get_offset(timer);
+ return put_user(offset, uaddr);
+}
+
+int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
+ case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ return kvm_arm_timer_get_attr_irq(vcpu, attr);
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ return kvm_arm_timer_get_attr_offset(vcpu, attr);
+ }
+
+ return -ENXIO;
+}
+
int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
return 0;
}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 08/12] KVM: arm64: Allow userspace to configure a vCPU's virtual offset
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Add a new vCPU attribute that allows userspace to directly manipulate
the virtual counter-timer offset. Exposing such an interface allows for
the precise migration of guest virtual counter-timers, as it is an
indepotent interface.
Uphold the existing behavior of writes to CNTVOFF_EL2 for this new
interface, wherein a write to a single vCPU is broadcasted to all vCPUs
within a VM.
Signed-off-by: Oliver Upton <oupton@google.com>
---
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/arch_timer.c | 68 ++++++++++++++++++++++++++++++-
2 files changed, 67 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index b3edde68bc3e..008d0518d2b1 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -365,6 +365,7 @@ struct kvm_arm_copy_mte_tags {
#define KVM_ARM_VCPU_TIMER_CTRL 1
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
+#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2
#define KVM_ARM_VCPU_PVTIME_CTRL 2
#define KVM_ARM_VCPU_PVTIME_IPA 0
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 3df67c127489..d2b1b13af658 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -1305,7 +1305,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq)
}
}
-int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+int kvm_arm_timer_set_attr_irq(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
int __user *uaddr = (int __user *)(long)attr->addr;
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
@@ -1338,7 +1338,39 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return 0;
}
-int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 offset;
+
+ if (get_user(offset, uaddr))
+ return -EFAULT;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ update_vtimer_cntvoff(vcpu, offset);
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
+int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
+ case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ return kvm_arm_timer_set_attr_irq(vcpu, attr);
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ return kvm_arm_timer_set_attr_offset(vcpu, attr);
+ }
+
+ return -ENXIO;
+}
+
+int kvm_arm_timer_get_attr_irq(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
int __user *uaddr = (int __user *)(long)attr->addr;
struct arch_timer_context *timer;
@@ -1359,11 +1391,43 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return put_user(irq, uaddr);
}
+int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ struct arch_timer_context *timer;
+ u64 offset;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ timer = vcpu_vtimer(vcpu);
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ offset = timer_get_offset(timer);
+ return put_user(offset, uaddr);
+}
+
+int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
+ case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ return kvm_arm_timer_get_attr_irq(vcpu, attr);
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ return kvm_arm_timer_get_attr_offset(vcpu, attr);
+ }
+
+ return -ENXIO;
+}
+
int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
{
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
return 0;
}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 09/12] selftests: KVM: Add support for aarch64 to system_counter_offset_test
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
KVM/arm64 now allows userspace to adjust the guest virtual counter-timer
via a vCPU device attribute. Test that changes to the virtual
counter-timer offset result in the correct view being presented to the
guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/Makefile | 1 +
.../selftests/kvm/include/aarch64/processor.h | 12 +++++
.../kvm/system_counter_offset_test.c | 54 ++++++++++++++++++-
3 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 7bf2e5fb1d5a..d89908108c97 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -96,6 +96,7 @@ TEST_GEN_PROGS_aarch64 += kvm_page_table_test
TEST_GEN_PROGS_aarch64 += set_memory_region_test
TEST_GEN_PROGS_aarch64 += steal_time
TEST_GEN_PROGS_aarch64 += kvm_binary_stats_test
+TEST_GEN_PROGS_aarch64 += system_counter_offset_test
TEST_GEN_PROGS_s390x = s390x/memop
TEST_GEN_PROGS_s390x += s390x/resets
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 27dc5c2e56b9..3168cdbae6ee 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -129,4 +129,16 @@ void vm_install_sync_handler(struct kvm_vm *vm,
#define isb() asm volatile("isb" : : : "memory")
+static inline uint64_t read_cntvct_ordered(void)
+{
+ uint64_t r;
+
+ __asm__ __volatile__("isb\n\t"
+ "mrs %0, cntvct_el0\n\t"
+ "isb\n\t"
+ : "=r"(r));
+
+ return r;
+}
+
#endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
index 7e9015770759..88ad997f5b69 100644
--- a/tools/testing/selftests/kvm/system_counter_offset_test.c
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -53,7 +53,59 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)
return rdtsc() + test->tsc_offset;
}
-#else /* __x86_64__ */
+#elif __aarch64__ /* __x86_64__ */
+
+enum arch_counter {
+ VIRTUAL,
+};
+
+struct test_case {
+ enum arch_counter counter;
+ uint64_t offset;
+};
+
+static struct test_case test_cases[] = {
+ { .counter = VIRTUAL, .offset = 0 },
+ { .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC },
+ { .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC },
+};
+
+static void check_preconditions(struct kvm_vm *vm)
+{
+ if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER))
+ return;
+
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test");
+ exit(KSFT_SKIP);
+}
+
+static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER, &test->offset,
+ true);
+}
+
+static uint64_t guest_read_system_counter(struct test_case *test)
+{
+ switch (test->counter) {
+ case VIRTUAL:
+ return read_cntvct_ordered();
+ default:
+ GUEST_ASSERT(0);
+ }
+
+ /* unreachable */
+ return 0;
+}
+
+static uint64_t host_read_guest_system_counter(struct test_case *test)
+{
+ return read_cntvct_ordered() - test->offset;
+}
+
+#else /* __aarch64__ */
#error test not implemented for this architecture!
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 09/12] selftests: KVM: Add support for aarch64 to system_counter_offset_test
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
KVM/arm64 now allows userspace to adjust the guest virtual counter-timer
via a vCPU device attribute. Test that changes to the virtual
counter-timer offset result in the correct view being presented to the
guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/Makefile | 1 +
.../selftests/kvm/include/aarch64/processor.h | 12 +++++
.../kvm/system_counter_offset_test.c | 54 ++++++++++++++++++-
3 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 7bf2e5fb1d5a..d89908108c97 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -96,6 +96,7 @@ TEST_GEN_PROGS_aarch64 += kvm_page_table_test
TEST_GEN_PROGS_aarch64 += set_memory_region_test
TEST_GEN_PROGS_aarch64 += steal_time
TEST_GEN_PROGS_aarch64 += kvm_binary_stats_test
+TEST_GEN_PROGS_aarch64 += system_counter_offset_test
TEST_GEN_PROGS_s390x = s390x/memop
TEST_GEN_PROGS_s390x += s390x/resets
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 27dc5c2e56b9..3168cdbae6ee 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -129,4 +129,16 @@ void vm_install_sync_handler(struct kvm_vm *vm,
#define isb() asm volatile("isb" : : : "memory")
+static inline uint64_t read_cntvct_ordered(void)
+{
+ uint64_t r;
+
+ __asm__ __volatile__("isb\n\t"
+ "mrs %0, cntvct_el0\n\t"
+ "isb\n\t"
+ : "=r"(r));
+
+ return r;
+}
+
#endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
index 7e9015770759..88ad997f5b69 100644
--- a/tools/testing/selftests/kvm/system_counter_offset_test.c
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -53,7 +53,59 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)
return rdtsc() + test->tsc_offset;
}
-#else /* __x86_64__ */
+#elif __aarch64__ /* __x86_64__ */
+
+enum arch_counter {
+ VIRTUAL,
+};
+
+struct test_case {
+ enum arch_counter counter;
+ uint64_t offset;
+};
+
+static struct test_case test_cases[] = {
+ { .counter = VIRTUAL, .offset = 0 },
+ { .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC },
+ { .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC },
+};
+
+static void check_preconditions(struct kvm_vm *vm)
+{
+ if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER))
+ return;
+
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test");
+ exit(KSFT_SKIP);
+}
+
+static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER, &test->offset,
+ true);
+}
+
+static uint64_t guest_read_system_counter(struct test_case *test)
+{
+ switch (test->counter) {
+ case VIRTUAL:
+ return read_cntvct_ordered();
+ default:
+ GUEST_ASSERT(0);
+ }
+
+ /* unreachable */
+ return 0;
+}
+
+static uint64_t host_read_guest_system_counter(struct test_case *test)
+{
+ return read_cntvct_ordered() - test->offset;
+}
+
+#else /* __aarch64__ */
#error test not implemented for this architecture!
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 09/12] selftests: KVM: Add support for aarch64 to system_counter_offset_test
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
KVM/arm64 now allows userspace to adjust the guest virtual counter-timer
via a vCPU device attribute. Test that changes to the virtual
counter-timer offset result in the correct view being presented to the
guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/Makefile | 1 +
.../selftests/kvm/include/aarch64/processor.h | 12 +++++
.../kvm/system_counter_offset_test.c | 54 ++++++++++++++++++-
3 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 7bf2e5fb1d5a..d89908108c97 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -96,6 +96,7 @@ TEST_GEN_PROGS_aarch64 += kvm_page_table_test
TEST_GEN_PROGS_aarch64 += set_memory_region_test
TEST_GEN_PROGS_aarch64 += steal_time
TEST_GEN_PROGS_aarch64 += kvm_binary_stats_test
+TEST_GEN_PROGS_aarch64 += system_counter_offset_test
TEST_GEN_PROGS_s390x = s390x/memop
TEST_GEN_PROGS_s390x += s390x/resets
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 27dc5c2e56b9..3168cdbae6ee 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -129,4 +129,16 @@ void vm_install_sync_handler(struct kvm_vm *vm,
#define isb() asm volatile("isb" : : : "memory")
+static inline uint64_t read_cntvct_ordered(void)
+{
+ uint64_t r;
+
+ __asm__ __volatile__("isb\n\t"
+ "mrs %0, cntvct_el0\n\t"
+ "isb\n\t"
+ : "=r"(r));
+
+ return r;
+}
+
#endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
index 7e9015770759..88ad997f5b69 100644
--- a/tools/testing/selftests/kvm/system_counter_offset_test.c
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -53,7 +53,59 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)
return rdtsc() + test->tsc_offset;
}
-#else /* __x86_64__ */
+#elif __aarch64__ /* __x86_64__ */
+
+enum arch_counter {
+ VIRTUAL,
+};
+
+struct test_case {
+ enum arch_counter counter;
+ uint64_t offset;
+};
+
+static struct test_case test_cases[] = {
+ { .counter = VIRTUAL, .offset = 0 },
+ { .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC },
+ { .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC },
+};
+
+static void check_preconditions(struct kvm_vm *vm)
+{
+ if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER))
+ return;
+
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test");
+ exit(KSFT_SKIP);
+}
+
+static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER, &test->offset,
+ true);
+}
+
+static uint64_t guest_read_system_counter(struct test_case *test)
+{
+ switch (test->counter) {
+ case VIRTUAL:
+ return read_cntvct_ordered();
+ default:
+ GUEST_ASSERT(0);
+ }
+
+ /* unreachable */
+ return 0;
+}
+
+static uint64_t host_read_guest_system_counter(struct test_case *test)
+{
+ return read_cntvct_ordered() - test->offset;
+}
+
+#else /* __aarch64__ */
#error test not implemented for this architecture!
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 10/12] KVM: arm64: Provide userspace access to the physical counter offset
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Presently, KVM provides no facilities for correctly migrating a guest
that depends on the physical counter-timer. While most guests (barring
NV, of course) should not depend on the physical counter-timer, an
operator may still wish to provide a consistent view of the physical
counter-timer across migrations.
Provide userspace with a new vCPU attribute to modify the guest physical
counter-timer offset. Since the base architecture doesn't provide a
physical counter-timer offset register, emulate the correct behavior by
trapping accesses to the physical counter-timer whenever the offset
value is non-zero.
Uphold the same behavior as CNTVOFF_EL2 and broadcast the physical
offset to all vCPUs whenever written. This guarantees that the
counter-timer we provide the guest remains architectural, wherein all
views of the counter-timer are consistent across vCPUs. Reconfigure
timer traps for VHE on every guest entry, as different VMs will now have
different traps enabled. Enable physical counter traps for nVHE whenever
the offset is nonzero (we already trap physical timer registers in
nVHE).
FEAT_ECV provides a guest physical counter-timer offset register
(CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of
writing so support for it was elided for the sake of the author :)
Signed-off-by: Oliver Upton <oupton@google.com>
---
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/arch_timer.c | 50 ++++++++++++++++++++---
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++
arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++++++++++++
arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +++++-----
arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++++++++++++
include/kvm/arm_arch_timer.h | 2 -
12 files changed, 136 insertions(+), 24 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 41911585ae0c..de92fa678924 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -204,6 +204,7 @@ enum vcpu_sysreg {
SP_EL1,
SPSR_EL1,
+ CNTPOFF_EL2,
CNTVOFF_EL2,
CNTV_CVAL_EL0,
CNTV_CTL_EL0,
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 9d60b3006efc..01eb3864e50f 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -65,10 +65,8 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_restore_aprs(struct vgic_v3_cpu_if *cpu_if);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
-#ifdef __KVM_NVHE_HYPERVISOR__
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-#endif
#ifdef __KVM_NVHE_HYPERVISOR__
void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 347ccac2341e..243e36c088e7 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -505,6 +505,7 @@
#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)
#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0)
+#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)
#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)
#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 008d0518d2b1..3e42c72d4c68 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -366,6 +366,7 @@ struct kvm_arm_copy_mte_tags {
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2
+#define KVM_ARM_VCPU_TIMER_OFFSET_PTIMER 3
#define KVM_ARM_VCPU_PVTIME_CTRL 2
#define KVM_ARM_VCPU_PVTIME_IPA 0
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index d2b1b13af658..05ec385e26b5 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -89,7 +89,10 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt)
switch(arch_timer_ctx_index(ctxt)) {
case TIMER_VTIMER:
return __vcpu_sys_reg(vcpu, CNTVOFF_EL2);
+ case TIMER_PTIMER:
+ return __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
default:
+ WARN_ONCE(1, "unrecognized timer index %ld", arch_timer_ctx_index(ctxt));
return 0;
}
}
@@ -134,6 +137,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)
case TIMER_VTIMER:
__vcpu_sys_reg(vcpu, CNTVOFF_EL2) = offset;
break;
+ case TIMER_PTIMER:
+ __vcpu_sys_reg(vcpu, CNTPOFF_EL2) = offset;
+ break;
default:
WARN(offset, "timer %ld\n", arch_timer_ctx_index(ctxt));
}
@@ -144,9 +150,24 @@ u64 kvm_phys_timer_read(void)
return timecounter->cc->read(timecounter->cc);
}
+static bool timer_emulation_required(struct arch_timer_context *ctx)
+{
+ enum kvm_arch_timers timer = arch_timer_ctx_index(ctx);
+
+ switch (timer) {
+ case TIMER_VTIMER:
+ return false;
+ case TIMER_PTIMER:
+ return timer_get_offset(ctx);
+ default:
+ WARN_ONCE(1, "unrecognized timer index %ld\n", timer);
+ return false;
+ }
+}
+
static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map)
{
- if (has_vhe()) {
+ if (has_vhe() && !timer_emulation_required(vcpu_ptimer(vcpu))) {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = vcpu_ptimer(vcpu);
map->emul_ptimer = NULL;
@@ -747,8 +768,9 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
return 0;
}
-/* Make the updates of cntvoff for all vtimer contexts atomic */
-static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
+/* Make the updates of offset for all timer contexts atomic */
+static void update_timer_offset(struct kvm_vcpu *vcpu,
+ enum kvm_arch_timers timer, u64 offset)
{
int i;
struct kvm *kvm = vcpu->kvm;
@@ -756,16 +778,26 @@ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
mutex_lock(&kvm->lock);
kvm_for_each_vcpu(i, tmp, kvm)
- timer_set_offset(vcpu_vtimer(tmp), cntvoff);
+ timer_set_offset(vcpu_get_timer(tmp, timer), offset);
/*
* When called from the vcpu create path, the CPU being created is not
* included in the loop above, so we just set it here as well.
*/
- timer_set_offset(vcpu_vtimer(vcpu), cntvoff);
+ timer_set_offset(vcpu_get_timer(vcpu, timer), offset);
mutex_unlock(&kvm->lock);
}
+static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
+{
+ update_timer_offset(vcpu, TIMER_VTIMER, cntvoff);
+}
+
+static void update_ptimer_cntpoff(struct kvm_vcpu *vcpu, u64 cntpoff)
+{
+ update_timer_offset(vcpu, TIMER_PTIMER, cntpoff);
+}
+
void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
{
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
@@ -1350,6 +1382,9 @@ int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
update_vtimer_cntvoff(vcpu, offset);
break;
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
+ update_ptimer_cntpoff(vcpu, offset);
+ break;
default:
return -ENXIO;
}
@@ -1364,6 +1399,7 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
return kvm_arm_timer_set_attr_irq(vcpu, attr);
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return kvm_arm_timer_set_attr_offset(vcpu, attr);
}
@@ -1400,6 +1436,8 @@ int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
timer = vcpu_vtimer(vcpu);
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
+ timer = vcpu_ptimer(vcpu);
break;
default:
return -ENXIO;
@@ -1416,6 +1454,7 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
return kvm_arm_timer_get_attr_irq(vcpu, attr);
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return kvm_arm_timer_get_attr_offset(vcpu, attr);
}
@@ -1428,6 +1467,7 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return 0;
}
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index e0b81870ff2a..217f4c6038e8 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1549,9 +1549,7 @@ static void cpu_hyp_reinit(void)
cpu_hyp_reset();
- if (is_kernel_in_hyp_mode())
- kvm_timer_init_vhe();
- else
+ if (!is_kernel_in_hyp_mode())
cpu_init_hyp_mode();
cpu_set_hyp_vector();
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e4a2f295a394..c3ae1e0614a2 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -8,6 +8,7 @@
#define __ARM64_KVM_HYP_SWITCH_H__
#include <hyp/adjust_pc.h>
+#include <hyp/timer-sr.h>
#include <linux/arm-smccc.h>
#include <linux/kvm_host.h>
@@ -113,6 +114,8 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu)
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE))
write_sysreg_s(vcpu->arch.vsesr_el2, SYS_VSESR_EL2);
+
+ __timer_enable_traps(vcpu);
}
static inline void ___deactivate_traps(struct kvm_vcpu *vcpu)
@@ -127,6 +130,8 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 &= ~HCR_VSE;
vcpu->arch.hcr_el2 |= read_sysreg(hcr_el2) & HCR_VSE;
}
+
+ __timer_disable_traps(vcpu);
}
static inline bool __translate_far_to_hpfar(u64 far, u64 *hpfar)
@@ -405,6 +410,21 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)
return true;
}
+static inline bool __hyp_handle_counter(struct kvm_vcpu *vcpu)
+{
+ u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+ int rt = kvm_vcpu_sys_get_rt(vcpu);
+ u64 rv;
+
+ if (sysreg != SYS_CNTPCT_EL0)
+ return false;
+
+ rv = __timer_read_cntpct(vcpu);
+ vcpu_set_reg(vcpu, rt, rv);
+ __kvm_skip_instr(vcpu);
+ return true;
+}
+
/*
* Return true when we were able to fixup the guest exit and should return to
* the guest, false when we should restore the host state and return to the
@@ -439,6 +459,9 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
if (*exit_code != ARM_EXCEPTION_TRAP)
goto exit;
+ if (__hyp_handle_counter(vcpu))
+ goto guest;
+
if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) &&
kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 &&
handle_tx2_tvm(vcpu))
diff --git a/arch/arm64/kvm/hyp/include/hyp/timer-sr.h b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h
new file mode 100644
index 000000000000..c91fbf9e5170
--- /dev/null
+++ b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2021 Google LLC
+ * Author: Oliver Upton <oupton@google.com>
+ */
+
+#ifndef __ARM64_KVM_HYP_TIMER_SR_H__
+#define __ARM64_KVM_HYP_TIMER_SR_H__
+
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_asm.h>
+#include <asm/kvm_hyp.h>
+
+static inline bool __timer_physical_emulation_required(struct kvm_vcpu *vcpu)
+{
+ return __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
+}
+
+static inline u64 __timer_read_cntpct(struct kvm_vcpu *vcpu)
+{
+ return read_sysreg(cntpct_el0) - __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
+}
+
+#endif /* __ARM64_KVM_HYP_TIMER_SR_H__ */
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f7af9688c1f7..4a190c932f8c 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -217,7 +217,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__activate_traps(vcpu);
__hyp_vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
__debug_switch_to_guest(vcpu);
@@ -230,7 +229,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
- __timer_disable_traps(vcpu);
__hyp_vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
index 9072e71693ba..ebc3f0d0908d 100644
--- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
@@ -9,16 +9,13 @@
#include <linux/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <hyp/timer-sr.h>
void __kvm_timer_set_cntvoff(u64 cntvoff)
{
write_sysreg(cntvoff, cntvoff_el2);
}
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe().
- */
void __timer_disable_traps(struct kvm_vcpu *vcpu)
{
u64 val;
@@ -29,20 +26,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)
write_sysreg(val, cnthctl_el2);
}
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe().
- */
void __timer_enable_traps(struct kvm_vcpu *vcpu)
{
u64 val;
/*
* Disallow physical timer access for the guest
- * Physical counter access is allowed
*/
val = read_sysreg(cnthctl_el2);
val &= ~CNTHCTL_EL1PCEN;
- val |= CNTHCTL_EL1PCTEN;
+
+ /*
+ * Disallow physical counter access for the guest if offsetting is
+ * requested.
+ */
+ if (__timer_physical_emulation_required(vcpu))
+ val &= ~CNTHCTL_EL1PCTEN;
+ else
+ val |= CNTHCTL_EL1PCTEN;
+
write_sysreg(val, cnthctl_el2);
}
diff --git a/arch/arm64/kvm/hyp/vhe/timer-sr.c b/arch/arm64/kvm/hyp/vhe/timer-sr.c
index 4cda674a8be6..10506f3ce8a1 100644
--- a/arch/arm64/kvm/hyp/vhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/timer-sr.c
@@ -4,9 +4,36 @@
* Author: Marc Zyngier <marc.zyngier@arm.com>
*/
+#include <asm/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <hyp/timer-sr.h>
void __kvm_timer_set_cntvoff(u64 cntvoff)
{
write_sysreg(cntvoff, cntvoff_el2);
}
+
+void __timer_enable_traps(struct kvm_vcpu *vcpu)
+{
+ /* When HCR_EL2.E2H == 1, EL1PCEN nad EL1PCTEN are shifted by 10 */
+ u32 cnthctl_shift = 10;
+ u64 val, mask;
+
+ mask = CNTHCTL_EL1PCEN << cnthctl_shift;
+ mask |= CNTHCTL_EL1PCTEN << cnthctl_shift;
+
+ val = read_sysreg(cnthctl_el2);
+
+ /*
+ * VHE systems allow the guest direct access to the EL1 physical
+ * timer/counter if offsetting isn't requested.
+ */
+ if (__timer_physical_emulation_required(vcpu))
+ val &= ~mask;
+ else
+ val |= mask;
+
+ write_sysreg(val, cnthctl_el2);
+}
+
+void __timer_disable_traps(struct kvm_vcpu *vcpu) {}
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 51c19381108c..f24fc435c401 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -83,8 +83,6 @@ u64 kvm_phys_timer_read(void);
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu);
void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
-void kvm_timer_init_vhe(void);
-
bool kvm_arch_timer_get_input_level(int vintid);
#define vcpu_timer(v) (&(v)->arch.timer_cpu)
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 10/12] KVM: arm64: Provide userspace access to the physical counter offset
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Presently, KVM provides no facilities for correctly migrating a guest
that depends on the physical counter-timer. While most guests (barring
NV, of course) should not depend on the physical counter-timer, an
operator may still wish to provide a consistent view of the physical
counter-timer across migrations.
Provide userspace with a new vCPU attribute to modify the guest physical
counter-timer offset. Since the base architecture doesn't provide a
physical counter-timer offset register, emulate the correct behavior by
trapping accesses to the physical counter-timer whenever the offset
value is non-zero.
Uphold the same behavior as CNTVOFF_EL2 and broadcast the physical
offset to all vCPUs whenever written. This guarantees that the
counter-timer we provide the guest remains architectural, wherein all
views of the counter-timer are consistent across vCPUs. Reconfigure
timer traps for VHE on every guest entry, as different VMs will now have
different traps enabled. Enable physical counter traps for nVHE whenever
the offset is nonzero (we already trap physical timer registers in
nVHE).
FEAT_ECV provides a guest physical counter-timer offset register
(CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of
writing so support for it was elided for the sake of the author :)
Signed-off-by: Oliver Upton <oupton@google.com>
---
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/arch_timer.c | 50 ++++++++++++++++++++---
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++
arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++++++++++++
arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +++++-----
arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++++++++++++
include/kvm/arm_arch_timer.h | 2 -
12 files changed, 136 insertions(+), 24 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 41911585ae0c..de92fa678924 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -204,6 +204,7 @@ enum vcpu_sysreg {
SP_EL1,
SPSR_EL1,
+ CNTPOFF_EL2,
CNTVOFF_EL2,
CNTV_CVAL_EL0,
CNTV_CTL_EL0,
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 9d60b3006efc..01eb3864e50f 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -65,10 +65,8 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_restore_aprs(struct vgic_v3_cpu_if *cpu_if);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
-#ifdef __KVM_NVHE_HYPERVISOR__
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-#endif
#ifdef __KVM_NVHE_HYPERVISOR__
void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 347ccac2341e..243e36c088e7 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -505,6 +505,7 @@
#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)
#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0)
+#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)
#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)
#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 008d0518d2b1..3e42c72d4c68 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -366,6 +366,7 @@ struct kvm_arm_copy_mte_tags {
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2
+#define KVM_ARM_VCPU_TIMER_OFFSET_PTIMER 3
#define KVM_ARM_VCPU_PVTIME_CTRL 2
#define KVM_ARM_VCPU_PVTIME_IPA 0
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index d2b1b13af658..05ec385e26b5 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -89,7 +89,10 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt)
switch(arch_timer_ctx_index(ctxt)) {
case TIMER_VTIMER:
return __vcpu_sys_reg(vcpu, CNTVOFF_EL2);
+ case TIMER_PTIMER:
+ return __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
default:
+ WARN_ONCE(1, "unrecognized timer index %ld", arch_timer_ctx_index(ctxt));
return 0;
}
}
@@ -134,6 +137,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)
case TIMER_VTIMER:
__vcpu_sys_reg(vcpu, CNTVOFF_EL2) = offset;
break;
+ case TIMER_PTIMER:
+ __vcpu_sys_reg(vcpu, CNTPOFF_EL2) = offset;
+ break;
default:
WARN(offset, "timer %ld\n", arch_timer_ctx_index(ctxt));
}
@@ -144,9 +150,24 @@ u64 kvm_phys_timer_read(void)
return timecounter->cc->read(timecounter->cc);
}
+static bool timer_emulation_required(struct arch_timer_context *ctx)
+{
+ enum kvm_arch_timers timer = arch_timer_ctx_index(ctx);
+
+ switch (timer) {
+ case TIMER_VTIMER:
+ return false;
+ case TIMER_PTIMER:
+ return timer_get_offset(ctx);
+ default:
+ WARN_ONCE(1, "unrecognized timer index %ld\n", timer);
+ return false;
+ }
+}
+
static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map)
{
- if (has_vhe()) {
+ if (has_vhe() && !timer_emulation_required(vcpu_ptimer(vcpu))) {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = vcpu_ptimer(vcpu);
map->emul_ptimer = NULL;
@@ -747,8 +768,9 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
return 0;
}
-/* Make the updates of cntvoff for all vtimer contexts atomic */
-static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
+/* Make the updates of offset for all timer contexts atomic */
+static void update_timer_offset(struct kvm_vcpu *vcpu,
+ enum kvm_arch_timers timer, u64 offset)
{
int i;
struct kvm *kvm = vcpu->kvm;
@@ -756,16 +778,26 @@ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
mutex_lock(&kvm->lock);
kvm_for_each_vcpu(i, tmp, kvm)
- timer_set_offset(vcpu_vtimer(tmp), cntvoff);
+ timer_set_offset(vcpu_get_timer(tmp, timer), offset);
/*
* When called from the vcpu create path, the CPU being created is not
* included in the loop above, so we just set it here as well.
*/
- timer_set_offset(vcpu_vtimer(vcpu), cntvoff);
+ timer_set_offset(vcpu_get_timer(vcpu, timer), offset);
mutex_unlock(&kvm->lock);
}
+static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
+{
+ update_timer_offset(vcpu, TIMER_VTIMER, cntvoff);
+}
+
+static void update_ptimer_cntpoff(struct kvm_vcpu *vcpu, u64 cntpoff)
+{
+ update_timer_offset(vcpu, TIMER_PTIMER, cntpoff);
+}
+
void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
{
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
@@ -1350,6 +1382,9 @@ int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
update_vtimer_cntvoff(vcpu, offset);
break;
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
+ update_ptimer_cntpoff(vcpu, offset);
+ break;
default:
return -ENXIO;
}
@@ -1364,6 +1399,7 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
return kvm_arm_timer_set_attr_irq(vcpu, attr);
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return kvm_arm_timer_set_attr_offset(vcpu, attr);
}
@@ -1400,6 +1436,8 @@ int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
timer = vcpu_vtimer(vcpu);
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
+ timer = vcpu_ptimer(vcpu);
break;
default:
return -ENXIO;
@@ -1416,6 +1454,7 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
return kvm_arm_timer_get_attr_irq(vcpu, attr);
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return kvm_arm_timer_get_attr_offset(vcpu, attr);
}
@@ -1428,6 +1467,7 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return 0;
}
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index e0b81870ff2a..217f4c6038e8 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1549,9 +1549,7 @@ static void cpu_hyp_reinit(void)
cpu_hyp_reset();
- if (is_kernel_in_hyp_mode())
- kvm_timer_init_vhe();
- else
+ if (!is_kernel_in_hyp_mode())
cpu_init_hyp_mode();
cpu_set_hyp_vector();
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e4a2f295a394..c3ae1e0614a2 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -8,6 +8,7 @@
#define __ARM64_KVM_HYP_SWITCH_H__
#include <hyp/adjust_pc.h>
+#include <hyp/timer-sr.h>
#include <linux/arm-smccc.h>
#include <linux/kvm_host.h>
@@ -113,6 +114,8 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu)
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE))
write_sysreg_s(vcpu->arch.vsesr_el2, SYS_VSESR_EL2);
+
+ __timer_enable_traps(vcpu);
}
static inline void ___deactivate_traps(struct kvm_vcpu *vcpu)
@@ -127,6 +130,8 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 &= ~HCR_VSE;
vcpu->arch.hcr_el2 |= read_sysreg(hcr_el2) & HCR_VSE;
}
+
+ __timer_disable_traps(vcpu);
}
static inline bool __translate_far_to_hpfar(u64 far, u64 *hpfar)
@@ -405,6 +410,21 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)
return true;
}
+static inline bool __hyp_handle_counter(struct kvm_vcpu *vcpu)
+{
+ u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+ int rt = kvm_vcpu_sys_get_rt(vcpu);
+ u64 rv;
+
+ if (sysreg != SYS_CNTPCT_EL0)
+ return false;
+
+ rv = __timer_read_cntpct(vcpu);
+ vcpu_set_reg(vcpu, rt, rv);
+ __kvm_skip_instr(vcpu);
+ return true;
+}
+
/*
* Return true when we were able to fixup the guest exit and should return to
* the guest, false when we should restore the host state and return to the
@@ -439,6 +459,9 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
if (*exit_code != ARM_EXCEPTION_TRAP)
goto exit;
+ if (__hyp_handle_counter(vcpu))
+ goto guest;
+
if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) &&
kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 &&
handle_tx2_tvm(vcpu))
diff --git a/arch/arm64/kvm/hyp/include/hyp/timer-sr.h b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h
new file mode 100644
index 000000000000..c91fbf9e5170
--- /dev/null
+++ b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2021 Google LLC
+ * Author: Oliver Upton <oupton@google.com>
+ */
+
+#ifndef __ARM64_KVM_HYP_TIMER_SR_H__
+#define __ARM64_KVM_HYP_TIMER_SR_H__
+
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_asm.h>
+#include <asm/kvm_hyp.h>
+
+static inline bool __timer_physical_emulation_required(struct kvm_vcpu *vcpu)
+{
+ return __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
+}
+
+static inline u64 __timer_read_cntpct(struct kvm_vcpu *vcpu)
+{
+ return read_sysreg(cntpct_el0) - __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
+}
+
+#endif /* __ARM64_KVM_HYP_TIMER_SR_H__ */
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f7af9688c1f7..4a190c932f8c 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -217,7 +217,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__activate_traps(vcpu);
__hyp_vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
__debug_switch_to_guest(vcpu);
@@ -230,7 +229,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
- __timer_disable_traps(vcpu);
__hyp_vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
index 9072e71693ba..ebc3f0d0908d 100644
--- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
@@ -9,16 +9,13 @@
#include <linux/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <hyp/timer-sr.h>
void __kvm_timer_set_cntvoff(u64 cntvoff)
{
write_sysreg(cntvoff, cntvoff_el2);
}
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe().
- */
void __timer_disable_traps(struct kvm_vcpu *vcpu)
{
u64 val;
@@ -29,20 +26,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)
write_sysreg(val, cnthctl_el2);
}
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe().
- */
void __timer_enable_traps(struct kvm_vcpu *vcpu)
{
u64 val;
/*
* Disallow physical timer access for the guest
- * Physical counter access is allowed
*/
val = read_sysreg(cnthctl_el2);
val &= ~CNTHCTL_EL1PCEN;
- val |= CNTHCTL_EL1PCTEN;
+
+ /*
+ * Disallow physical counter access for the guest if offsetting is
+ * requested.
+ */
+ if (__timer_physical_emulation_required(vcpu))
+ val &= ~CNTHCTL_EL1PCTEN;
+ else
+ val |= CNTHCTL_EL1PCTEN;
+
write_sysreg(val, cnthctl_el2);
}
diff --git a/arch/arm64/kvm/hyp/vhe/timer-sr.c b/arch/arm64/kvm/hyp/vhe/timer-sr.c
index 4cda674a8be6..10506f3ce8a1 100644
--- a/arch/arm64/kvm/hyp/vhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/timer-sr.c
@@ -4,9 +4,36 @@
* Author: Marc Zyngier <marc.zyngier@arm.com>
*/
+#include <asm/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <hyp/timer-sr.h>
void __kvm_timer_set_cntvoff(u64 cntvoff)
{
write_sysreg(cntvoff, cntvoff_el2);
}
+
+void __timer_enable_traps(struct kvm_vcpu *vcpu)
+{
+ /* When HCR_EL2.E2H == 1, EL1PCEN nad EL1PCTEN are shifted by 10 */
+ u32 cnthctl_shift = 10;
+ u64 val, mask;
+
+ mask = CNTHCTL_EL1PCEN << cnthctl_shift;
+ mask |= CNTHCTL_EL1PCTEN << cnthctl_shift;
+
+ val = read_sysreg(cnthctl_el2);
+
+ /*
+ * VHE systems allow the guest direct access to the EL1 physical
+ * timer/counter if offsetting isn't requested.
+ */
+ if (__timer_physical_emulation_required(vcpu))
+ val &= ~mask;
+ else
+ val |= mask;
+
+ write_sysreg(val, cnthctl_el2);
+}
+
+void __timer_disable_traps(struct kvm_vcpu *vcpu) {}
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 51c19381108c..f24fc435c401 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -83,8 +83,6 @@ u64 kvm_phys_timer_read(void);
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu);
void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
-void kvm_timer_init_vhe(void);
-
bool kvm_arch_timer_get_input_level(int vintid);
#define vcpu_timer(v) (&(v)->arch.timer_cpu)
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 10/12] KVM: arm64: Provide userspace access to the physical counter offset
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Presently, KVM provides no facilities for correctly migrating a guest
that depends on the physical counter-timer. While most guests (barring
NV, of course) should not depend on the physical counter-timer, an
operator may still wish to provide a consistent view of the physical
counter-timer across migrations.
Provide userspace with a new vCPU attribute to modify the guest physical
counter-timer offset. Since the base architecture doesn't provide a
physical counter-timer offset register, emulate the correct behavior by
trapping accesses to the physical counter-timer whenever the offset
value is non-zero.
Uphold the same behavior as CNTVOFF_EL2 and broadcast the physical
offset to all vCPUs whenever written. This guarantees that the
counter-timer we provide the guest remains architectural, wherein all
views of the counter-timer are consistent across vCPUs. Reconfigure
timer traps for VHE on every guest entry, as different VMs will now have
different traps enabled. Enable physical counter traps for nVHE whenever
the offset is nonzero (we already trap physical timer registers in
nVHE).
FEAT_ECV provides a guest physical counter-timer offset register
(CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of
writing so support for it was elided for the sake of the author :)
Signed-off-by: Oliver Upton <oupton@google.com>
---
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/arch_timer.c | 50 ++++++++++++++++++++---
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++
arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++++++++++++
arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +++++-----
arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++++++++++++
include/kvm/arm_arch_timer.h | 2 -
12 files changed, 136 insertions(+), 24 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 41911585ae0c..de92fa678924 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -204,6 +204,7 @@ enum vcpu_sysreg {
SP_EL1,
SPSR_EL1,
+ CNTPOFF_EL2,
CNTVOFF_EL2,
CNTV_CVAL_EL0,
CNTV_CTL_EL0,
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 9d60b3006efc..01eb3864e50f 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -65,10 +65,8 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if);
void __vgic_v3_restore_aprs(struct vgic_v3_cpu_if *cpu_if);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
-#ifdef __KVM_NVHE_HYPERVISOR__
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-#endif
#ifdef __KVM_NVHE_HYPERVISOR__
void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 347ccac2341e..243e36c088e7 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -505,6 +505,7 @@
#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)
#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0)
+#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)
#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)
#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 008d0518d2b1..3e42c72d4c68 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -366,6 +366,7 @@ struct kvm_arm_copy_mte_tags {
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2
+#define KVM_ARM_VCPU_TIMER_OFFSET_PTIMER 3
#define KVM_ARM_VCPU_PVTIME_CTRL 2
#define KVM_ARM_VCPU_PVTIME_IPA 0
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index d2b1b13af658..05ec385e26b5 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -89,7 +89,10 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt)
switch(arch_timer_ctx_index(ctxt)) {
case TIMER_VTIMER:
return __vcpu_sys_reg(vcpu, CNTVOFF_EL2);
+ case TIMER_PTIMER:
+ return __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
default:
+ WARN_ONCE(1, "unrecognized timer index %ld", arch_timer_ctx_index(ctxt));
return 0;
}
}
@@ -134,6 +137,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)
case TIMER_VTIMER:
__vcpu_sys_reg(vcpu, CNTVOFF_EL2) = offset;
break;
+ case TIMER_PTIMER:
+ __vcpu_sys_reg(vcpu, CNTPOFF_EL2) = offset;
+ break;
default:
WARN(offset, "timer %ld\n", arch_timer_ctx_index(ctxt));
}
@@ -144,9 +150,24 @@ u64 kvm_phys_timer_read(void)
return timecounter->cc->read(timecounter->cc);
}
+static bool timer_emulation_required(struct arch_timer_context *ctx)
+{
+ enum kvm_arch_timers timer = arch_timer_ctx_index(ctx);
+
+ switch (timer) {
+ case TIMER_VTIMER:
+ return false;
+ case TIMER_PTIMER:
+ return timer_get_offset(ctx);
+ default:
+ WARN_ONCE(1, "unrecognized timer index %ld\n", timer);
+ return false;
+ }
+}
+
static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map)
{
- if (has_vhe()) {
+ if (has_vhe() && !timer_emulation_required(vcpu_ptimer(vcpu))) {
map->direct_vtimer = vcpu_vtimer(vcpu);
map->direct_ptimer = vcpu_ptimer(vcpu);
map->emul_ptimer = NULL;
@@ -747,8 +768,9 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
return 0;
}
-/* Make the updates of cntvoff for all vtimer contexts atomic */
-static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
+/* Make the updates of offset for all timer contexts atomic */
+static void update_timer_offset(struct kvm_vcpu *vcpu,
+ enum kvm_arch_timers timer, u64 offset)
{
int i;
struct kvm *kvm = vcpu->kvm;
@@ -756,16 +778,26 @@ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
mutex_lock(&kvm->lock);
kvm_for_each_vcpu(i, tmp, kvm)
- timer_set_offset(vcpu_vtimer(tmp), cntvoff);
+ timer_set_offset(vcpu_get_timer(tmp, timer), offset);
/*
* When called from the vcpu create path, the CPU being created is not
* included in the loop above, so we just set it here as well.
*/
- timer_set_offset(vcpu_vtimer(vcpu), cntvoff);
+ timer_set_offset(vcpu_get_timer(vcpu, timer), offset);
mutex_unlock(&kvm->lock);
}
+static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
+{
+ update_timer_offset(vcpu, TIMER_VTIMER, cntvoff);
+}
+
+static void update_ptimer_cntpoff(struct kvm_vcpu *vcpu, u64 cntpoff)
+{
+ update_timer_offset(vcpu, TIMER_PTIMER, cntpoff);
+}
+
void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
{
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
@@ -1350,6 +1382,9 @@ int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
update_vtimer_cntvoff(vcpu, offset);
break;
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
+ update_ptimer_cntpoff(vcpu, offset);
+ break;
default:
return -ENXIO;
}
@@ -1364,6 +1399,7 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
return kvm_arm_timer_set_attr_irq(vcpu, attr);
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return kvm_arm_timer_set_attr_offset(vcpu, attr);
}
@@ -1400,6 +1436,8 @@ int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr
switch (attr->attr) {
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
timer = vcpu_vtimer(vcpu);
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
+ timer = vcpu_ptimer(vcpu);
break;
default:
return -ENXIO;
@@ -1416,6 +1454,7 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
return kvm_arm_timer_get_attr_irq(vcpu, attr);
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return kvm_arm_timer_get_attr_offset(vcpu, attr);
}
@@ -1428,6 +1467,7 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_TIMER_IRQ_VTIMER:
case KVM_ARM_VCPU_TIMER_IRQ_PTIMER:
case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
+ case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER:
return 0;
}
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index e0b81870ff2a..217f4c6038e8 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1549,9 +1549,7 @@ static void cpu_hyp_reinit(void)
cpu_hyp_reset();
- if (is_kernel_in_hyp_mode())
- kvm_timer_init_vhe();
- else
+ if (!is_kernel_in_hyp_mode())
cpu_init_hyp_mode();
cpu_set_hyp_vector();
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e4a2f295a394..c3ae1e0614a2 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -8,6 +8,7 @@
#define __ARM64_KVM_HYP_SWITCH_H__
#include <hyp/adjust_pc.h>
+#include <hyp/timer-sr.h>
#include <linux/arm-smccc.h>
#include <linux/kvm_host.h>
@@ -113,6 +114,8 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu)
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE))
write_sysreg_s(vcpu->arch.vsesr_el2, SYS_VSESR_EL2);
+
+ __timer_enable_traps(vcpu);
}
static inline void ___deactivate_traps(struct kvm_vcpu *vcpu)
@@ -127,6 +130,8 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 &= ~HCR_VSE;
vcpu->arch.hcr_el2 |= read_sysreg(hcr_el2) & HCR_VSE;
}
+
+ __timer_disable_traps(vcpu);
}
static inline bool __translate_far_to_hpfar(u64 far, u64 *hpfar)
@@ -405,6 +410,21 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)
return true;
}
+static inline bool __hyp_handle_counter(struct kvm_vcpu *vcpu)
+{
+ u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+ int rt = kvm_vcpu_sys_get_rt(vcpu);
+ u64 rv;
+
+ if (sysreg != SYS_CNTPCT_EL0)
+ return false;
+
+ rv = __timer_read_cntpct(vcpu);
+ vcpu_set_reg(vcpu, rt, rv);
+ __kvm_skip_instr(vcpu);
+ return true;
+}
+
/*
* Return true when we were able to fixup the guest exit and should return to
* the guest, false when we should restore the host state and return to the
@@ -439,6 +459,9 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
if (*exit_code != ARM_EXCEPTION_TRAP)
goto exit;
+ if (__hyp_handle_counter(vcpu))
+ goto guest;
+
if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) &&
kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 &&
handle_tx2_tvm(vcpu))
diff --git a/arch/arm64/kvm/hyp/include/hyp/timer-sr.h b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h
new file mode 100644
index 000000000000..c91fbf9e5170
--- /dev/null
+++ b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2021 Google LLC
+ * Author: Oliver Upton <oupton@google.com>
+ */
+
+#ifndef __ARM64_KVM_HYP_TIMER_SR_H__
+#define __ARM64_KVM_HYP_TIMER_SR_H__
+
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_asm.h>
+#include <asm/kvm_hyp.h>
+
+static inline bool __timer_physical_emulation_required(struct kvm_vcpu *vcpu)
+{
+ return __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
+}
+
+static inline u64 __timer_read_cntpct(struct kvm_vcpu *vcpu)
+{
+ return read_sysreg(cntpct_el0) - __vcpu_sys_reg(vcpu, CNTPOFF_EL2);
+}
+
+#endif /* __ARM64_KVM_HYP_TIMER_SR_H__ */
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f7af9688c1f7..4a190c932f8c 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -217,7 +217,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__activate_traps(vcpu);
__hyp_vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
__debug_switch_to_guest(vcpu);
@@ -230,7 +229,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
- __timer_disable_traps(vcpu);
__hyp_vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
index 9072e71693ba..ebc3f0d0908d 100644
--- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
@@ -9,16 +9,13 @@
#include <linux/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <hyp/timer-sr.h>
void __kvm_timer_set_cntvoff(u64 cntvoff)
{
write_sysreg(cntvoff, cntvoff_el2);
}
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe().
- */
void __timer_disable_traps(struct kvm_vcpu *vcpu)
{
u64 val;
@@ -29,20 +26,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)
write_sysreg(val, cnthctl_el2);
}
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe().
- */
void __timer_enable_traps(struct kvm_vcpu *vcpu)
{
u64 val;
/*
* Disallow physical timer access for the guest
- * Physical counter access is allowed
*/
val = read_sysreg(cnthctl_el2);
val &= ~CNTHCTL_EL1PCEN;
- val |= CNTHCTL_EL1PCTEN;
+
+ /*
+ * Disallow physical counter access for the guest if offsetting is
+ * requested.
+ */
+ if (__timer_physical_emulation_required(vcpu))
+ val &= ~CNTHCTL_EL1PCTEN;
+ else
+ val |= CNTHCTL_EL1PCTEN;
+
write_sysreg(val, cnthctl_el2);
}
diff --git a/arch/arm64/kvm/hyp/vhe/timer-sr.c b/arch/arm64/kvm/hyp/vhe/timer-sr.c
index 4cda674a8be6..10506f3ce8a1 100644
--- a/arch/arm64/kvm/hyp/vhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/timer-sr.c
@@ -4,9 +4,36 @@
* Author: Marc Zyngier <marc.zyngier@arm.com>
*/
+#include <asm/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <hyp/timer-sr.h>
void __kvm_timer_set_cntvoff(u64 cntvoff)
{
write_sysreg(cntvoff, cntvoff_el2);
}
+
+void __timer_enable_traps(struct kvm_vcpu *vcpu)
+{
+ /* When HCR_EL2.E2H == 1, EL1PCEN nad EL1PCTEN are shifted by 10 */
+ u32 cnthctl_shift = 10;
+ u64 val, mask;
+
+ mask = CNTHCTL_EL1PCEN << cnthctl_shift;
+ mask |= CNTHCTL_EL1PCTEN << cnthctl_shift;
+
+ val = read_sysreg(cnthctl_el2);
+
+ /*
+ * VHE systems allow the guest direct access to the EL1 physical
+ * timer/counter if offsetting isn't requested.
+ */
+ if (__timer_physical_emulation_required(vcpu))
+ val &= ~mask;
+ else
+ val |= mask;
+
+ write_sysreg(val, cnthctl_el2);
+}
+
+void __timer_disable_traps(struct kvm_vcpu *vcpu) {}
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 51c19381108c..f24fc435c401 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -83,8 +83,6 @@ u64 kvm_phys_timer_read(void);
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu);
void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
-void kvm_timer_init_vhe(void);
-
bool kvm_arch_timer_get_input_level(int vintid);
#define vcpu_timer(v) (&(v)->arch.timer_cpu)
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 11/12] selftests: KVM: Test physical counter offsetting
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Test that userpace adjustment of the guest physical counter-timer
results in the correct view of within the guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
.../selftests/kvm/include/aarch64/processor.h | 12 ++++++++
.../kvm/system_counter_offset_test.c | 29 ++++++++++++++++---
2 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 3168cdbae6ee..7f53d90e9512 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -141,4 +141,16 @@ static inline uint64_t read_cntvct_ordered(void)
return r;
}
+static inline uint64_t read_cntpct_ordered(void)
+{
+ uint64_t r;
+
+ __asm__ __volatile__("isb\n\t"
+ "mrs %0, cntpct_el0\n\t"
+ "isb\n\t"
+ : "=r"(r));
+
+ return r;
+}
+
#endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
index 88ad997f5b69..3eed9dcb7693 100644
--- a/tools/testing/selftests/kvm/system_counter_offset_test.c
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -57,6 +57,7 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)
enum arch_counter {
VIRTUAL,
+ PHYSICAL,
};
struct test_case {
@@ -68,23 +69,41 @@ static struct test_case test_cases[] = {
{ .counter = VIRTUAL, .offset = 0 },
{ .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC },
{ .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC },
+ { .counter = PHYSICAL, .offset = 0 },
+ { .counter = PHYSICAL, .offset = 180 * NSEC_PER_SEC },
+ { .counter = PHYSICAL, .offset = -180 * NSEC_PER_SEC },
};
static void check_preconditions(struct kvm_vm *vm)
{
if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
- KVM_ARM_VCPU_TIMER_OFFSET_VTIMER))
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER) &&
+ !_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER))
return;
- print_skip("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test");
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_{VTIMER,PTIMER} not supported; skipping test");
exit(KSFT_SKIP);
}
static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
{
+ u64 attr = 0;
+
+ switch (test->counter) {
+ case VIRTUAL:
+ attr = KVM_ARM_VCPU_TIMER_OFFSET_VTIMER;
+ break;
+ case PHYSICAL:
+ attr = KVM_ARM_VCPU_TIMER_OFFSET_PTIMER;
+ break;
+ default:
+ TEST_ASSERT(false, "unrecognized counter index %u",
+ test->counter);
+ }
+
vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
- KVM_ARM_VCPU_TIMER_OFFSET_VTIMER, &test->offset,
- true);
+ attr, &test->offset, true);
}
static uint64_t guest_read_system_counter(struct test_case *test)
@@ -92,6 +111,8 @@ static uint64_t guest_read_system_counter(struct test_case *test)
switch (test->counter) {
case VIRTUAL:
return read_cntvct_ordered();
+ case PHYSICAL:
+ return read_cntpct_ordered();
default:
GUEST_ASSERT(0);
}
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 11/12] selftests: KVM: Test physical counter offsetting
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Test that userpace adjustment of the guest physical counter-timer
results in the correct view of within the guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
.../selftests/kvm/include/aarch64/processor.h | 12 ++++++++
.../kvm/system_counter_offset_test.c | 29 ++++++++++++++++---
2 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 3168cdbae6ee..7f53d90e9512 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -141,4 +141,16 @@ static inline uint64_t read_cntvct_ordered(void)
return r;
}
+static inline uint64_t read_cntpct_ordered(void)
+{
+ uint64_t r;
+
+ __asm__ __volatile__("isb\n\t"
+ "mrs %0, cntpct_el0\n\t"
+ "isb\n\t"
+ : "=r"(r));
+
+ return r;
+}
+
#endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
index 88ad997f5b69..3eed9dcb7693 100644
--- a/tools/testing/selftests/kvm/system_counter_offset_test.c
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -57,6 +57,7 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)
enum arch_counter {
VIRTUAL,
+ PHYSICAL,
};
struct test_case {
@@ -68,23 +69,41 @@ static struct test_case test_cases[] = {
{ .counter = VIRTUAL, .offset = 0 },
{ .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC },
{ .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC },
+ { .counter = PHYSICAL, .offset = 0 },
+ { .counter = PHYSICAL, .offset = 180 * NSEC_PER_SEC },
+ { .counter = PHYSICAL, .offset = -180 * NSEC_PER_SEC },
};
static void check_preconditions(struct kvm_vm *vm)
{
if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
- KVM_ARM_VCPU_TIMER_OFFSET_VTIMER))
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER) &&
+ !_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER))
return;
- print_skip("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test");
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_{VTIMER,PTIMER} not supported; skipping test");
exit(KSFT_SKIP);
}
static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
{
+ u64 attr = 0;
+
+ switch (test->counter) {
+ case VIRTUAL:
+ attr = KVM_ARM_VCPU_TIMER_OFFSET_VTIMER;
+ break;
+ case PHYSICAL:
+ attr = KVM_ARM_VCPU_TIMER_OFFSET_PTIMER;
+ break;
+ default:
+ TEST_ASSERT(false, "unrecognized counter index %u",
+ test->counter);
+ }
+
vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
- KVM_ARM_VCPU_TIMER_OFFSET_VTIMER, &test->offset,
- true);
+ attr, &test->offset, true);
}
static uint64_t guest_read_system_counter(struct test_case *test)
@@ -92,6 +111,8 @@ static uint64_t guest_read_system_counter(struct test_case *test)
switch (test->counter) {
case VIRTUAL:
return read_cntvct_ordered();
+ case PHYSICAL:
+ return read_cntpct_ordered();
default:
GUEST_ASSERT(0);
}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 11/12] selftests: KVM: Test physical counter offsetting
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Test that userpace adjustment of the guest physical counter-timer
results in the correct view of within the guest.
Signed-off-by: Oliver Upton <oupton@google.com>
---
.../selftests/kvm/include/aarch64/processor.h | 12 ++++++++
.../kvm/system_counter_offset_test.c | 29 ++++++++++++++++---
2 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 3168cdbae6ee..7f53d90e9512 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -141,4 +141,16 @@ static inline uint64_t read_cntvct_ordered(void)
return r;
}
+static inline uint64_t read_cntpct_ordered(void)
+{
+ uint64_t r;
+
+ __asm__ __volatile__("isb\n\t"
+ "mrs %0, cntpct_el0\n\t"
+ "isb\n\t"
+ : "=r"(r));
+
+ return r;
+}
+
#endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/system_counter_offset_test.c b/tools/testing/selftests/kvm/system_counter_offset_test.c
index 88ad997f5b69..3eed9dcb7693 100644
--- a/tools/testing/selftests/kvm/system_counter_offset_test.c
+++ b/tools/testing/selftests/kvm/system_counter_offset_test.c
@@ -57,6 +57,7 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)
enum arch_counter {
VIRTUAL,
+ PHYSICAL,
};
struct test_case {
@@ -68,23 +69,41 @@ static struct test_case test_cases[] = {
{ .counter = VIRTUAL, .offset = 0 },
{ .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC },
{ .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC },
+ { .counter = PHYSICAL, .offset = 0 },
+ { .counter = PHYSICAL, .offset = 180 * NSEC_PER_SEC },
+ { .counter = PHYSICAL, .offset = -180 * NSEC_PER_SEC },
};
static void check_preconditions(struct kvm_vm *vm)
{
if (!_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
- KVM_ARM_VCPU_TIMER_OFFSET_VTIMER))
+ KVM_ARM_VCPU_TIMER_OFFSET_VTIMER) &&
+ !_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER))
return;
- print_skip("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test");
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_{VTIMER,PTIMER} not supported; skipping test");
exit(KSFT_SKIP);
}
static void setup_system_counter(struct kvm_vm *vm, struct test_case *test)
{
+ u64 attr = 0;
+
+ switch (test->counter) {
+ case VIRTUAL:
+ attr = KVM_ARM_VCPU_TIMER_OFFSET_VTIMER;
+ break;
+ case PHYSICAL:
+ attr = KVM_ARM_VCPU_TIMER_OFFSET_PTIMER;
+ break;
+ default:
+ TEST_ASSERT(false, "unrecognized counter index %u",
+ test->counter);
+ }
+
vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
- KVM_ARM_VCPU_TIMER_OFFSET_VTIMER, &test->offset,
- true);
+ attr, &test->offset, true);
}
static uint64_t guest_read_system_counter(struct test_case *test)
@@ -92,6 +111,8 @@ static uint64_t guest_read_system_counter(struct test_case *test)
switch (test->counter) {
case VIRTUAL:
return read_cntvct_ordered();
+ case PHYSICAL:
+ return read_cntpct_ordered();
default:
GUEST_ASSERT(0);
}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 12/12] selftests: KVM: Add counter emulation benchmark
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:26 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Add a test case for counter emulation on arm64. A side effect of how KVM
handles physical counter offsetting on non-ECV systems is that the
virtual counter will always hit hardware and the physical could be
emulated. Force emulation by writing a nonzero offset to the physical
counter and compare the elapsed cycles to a direct read of the hardware
register.
Reviewed-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../kvm/aarch64/counter_emulation_benchmark.c | 215 ++++++++++++++++++
3 files changed, 217 insertions(+)
create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 2752813d5090..1d811c6a769b 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
/aarch64/debug-exceptions
+/aarch64/counter_emulation_benchmark
/aarch64/get-reg-list
/aarch64/vgic_init
/s390x/memop
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index d89908108c97..e560a3e74bc2 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -86,6 +86,7 @@ TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
TEST_GEN_PROGS_x86_64 += system_counter_offset_test
TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
+TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark
TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
TEST_GEN_PROGS_aarch64 += demand_paging_test
diff --git a/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
new file mode 100644
index 000000000000..73aeb6cdebfe
--- /dev/null
+++ b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
@@ -0,0 +1,215 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * counter_emulation_benchmark.c -- test to measure the effects of counter
+ * emulation on guest reads of the physical counter.
+ *
+ * Copyright (c) 2021, Google LLC.
+ */
+
+#define _GNU_SOURCE
+#include <asm/kvm.h>
+#include <linux/kvm.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define VCPU_ID 0
+
+static struct counter_values {
+ uint64_t cntvct_start;
+ uint64_t cntpct;
+ uint64_t cntvct_end;
+} counter_values;
+
+static uint64_t nr_iterations = 1000;
+
+static void do_test(void)
+{
+ /*
+ * Open-coded approach instead of using helper methods to keep a tight
+ * interval around the physical counter read.
+ */
+ asm volatile("isb\n\t"
+ "mrs %[cntvct_start], cntvct_el0\n\t"
+ "isb\n\t"
+ "mrs %[cntpct], cntpct_el0\n\t"
+ "isb\n\t"
+ "mrs %[cntvct_end], cntvct_el0\n\t"
+ "isb\n\t"
+ : [cntvct_start] "=r"(counter_values.cntvct_start),
+ [cntpct] "=r"(counter_values.cntpct),
+ [cntvct_end] "=r"(counter_values.cntvct_end));
+}
+
+static void guest_main(void)
+{
+ int i;
+
+ for (i = 0; i < nr_iterations; i++) {
+ do_test();
+ GUEST_SYNC(i);
+ }
+
+ for (i = 0; i < nr_iterations; i++) {
+ do_test();
+ GUEST_SYNC(i);
+ }
+
+ GUEST_DONE();
+}
+
+static bool enter_guest(struct kvm_vm *vm)
+{
+ struct ucall uc;
+
+ vcpu_ioctl(vm, VCPU_ID, KVM_RUN, NULL);
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_DONE:
+ return true;
+ case UCALL_SYNC:
+ break;
+ case UCALL_ABORT:
+ TEST_ASSERT(false, "%s at %s:%ld", (const char *)uc.args[0],
+ __FILE__, uc.args[1]);
+ break;
+ default:
+ TEST_ASSERT(false, "unexpected exit: %s",
+ exit_reason_str(vcpu_state(vm, VCPU_ID)->exit_reason));
+ break;
+ }
+
+ /* more work to do in the guest */
+ return false;
+}
+
+static double counter_frequency(void)
+{
+ uint32_t freq;
+
+ asm volatile("mrs %0, cntfrq_el0"
+ : "=r" (freq));
+
+ return freq / 1000000.0;
+}
+
+static void log_csv(FILE *csv, bool trapped)
+{
+ double freq = counter_frequency();
+
+ fprintf(csv, "%s,%.02f,%lu,%lu,%lu\n",
+ trapped ? "true" : "false", freq,
+ counter_values.cntvct_start,
+ counter_values.cntpct,
+ counter_values.cntvct_end);
+}
+
+static double run_loop(struct kvm_vm *vm, FILE *csv, bool trapped)
+{
+ double avg = 0;
+ int i;
+
+ for (i = 0; i < nr_iterations; i++) {
+ uint64_t delta;
+
+ TEST_ASSERT(!enter_guest(vm), "guest exited unexpectedly");
+ sync_global_from_guest(vm, counter_values);
+
+ if (csv)
+ log_csv(csv, trapped);
+
+ delta = counter_values.cntvct_end - counter_values.cntvct_start;
+ avg = ((avg * i) + delta) / (i + 1);
+ }
+
+ return avg;
+}
+
+static void setup_counter(struct kvm_vm *vm, uint64_t offset)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER, &offset,
+ true);
+}
+
+static void run_tests(struct kvm_vm *vm, FILE *csv)
+{
+ double avg_trapped, avg_native, freq;
+
+ freq = counter_frequency();
+
+ if (csv)
+ fputs("trapped,freq_mhz,cntvct_start,cntpct,cntvct_end\n", csv);
+
+ /* no physical offsetting; kvm allows reads of cntpct_el0 */
+ setup_counter(vm, 0);
+ avg_native = run_loop(vm, csv, false);
+
+ /* force emulation of the physical counter */
+ setup_counter(vm, 1);
+ avg_trapped = run_loop(vm, csv, true);
+
+ TEST_ASSERT(enter_guest(vm), "guest didn't run to completion");
+ pr_info("%lu iterations: average cycles (@%.02fMHz) native: %.02f, trapped: %.02f\n",
+ nr_iterations, freq, avg_native, avg_trapped);
+}
+
+static void usage(const char *program_name)
+{
+ fprintf(stderr,
+ "Usage: %s [-h] [-o csv_file] [-n iterations]\n"
+ " -h prints this message\n"
+ " -n number of test iterations (default: %lu)\n"
+ " -o csv file to write data\n",
+ program_name, nr_iterations);
+}
+
+int main(int argc, char **argv)
+{
+ struct kvm_vm *vm;
+ FILE *csv = NULL;
+ int opt;
+
+ while ((opt = getopt(argc, argv, "hn:o:")) != -1) {
+ switch (opt) {
+ case 'o':
+ csv = fopen(optarg, "w");
+ if (!csv) {
+ fprintf(stderr, "failed to open file '%s': %d\n",
+ optarg, errno);
+ exit(1);
+ }
+ break;
+ case 'n':
+ nr_iterations = strtoul(optarg, NULL, 0);
+ break;
+ default:
+ fprintf(stderr, "unrecognized option: '-%c'\n", opt);
+ /* fallthrough */
+ case 'h':
+ usage(argv[0]);
+ exit(1);
+ }
+ }
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ sync_global_to_guest(vm, nr_iterations);
+ ucall_init(vm, NULL);
+
+ if (_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER)) {
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_PTIMER not supported.");
+ exit(KSFT_SKIP);
+ }
+
+ run_tests(vm, csv);
+ kvm_vm_free(vm);
+
+ if (csv)
+ fclose(csv);
+}
--
2.32.0.402.g57bb445576-goog
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 12/12] selftests: KVM: Add counter emulation benchmark
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel, Oliver Upton
Add a test case for counter emulation on arm64. A side effect of how KVM
handles physical counter offsetting on non-ECV systems is that the
virtual counter will always hit hardware and the physical could be
emulated. Force emulation by writing a nonzero offset to the physical
counter and compare the elapsed cycles to a direct read of the hardware
register.
Reviewed-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../kvm/aarch64/counter_emulation_benchmark.c | 215 ++++++++++++++++++
3 files changed, 217 insertions(+)
create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 2752813d5090..1d811c6a769b 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
/aarch64/debug-exceptions
+/aarch64/counter_emulation_benchmark
/aarch64/get-reg-list
/aarch64/vgic_init
/s390x/memop
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index d89908108c97..e560a3e74bc2 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -86,6 +86,7 @@ TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
TEST_GEN_PROGS_x86_64 += system_counter_offset_test
TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
+TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark
TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
TEST_GEN_PROGS_aarch64 += demand_paging_test
diff --git a/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
new file mode 100644
index 000000000000..73aeb6cdebfe
--- /dev/null
+++ b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
@@ -0,0 +1,215 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * counter_emulation_benchmark.c -- test to measure the effects of counter
+ * emulation on guest reads of the physical counter.
+ *
+ * Copyright (c) 2021, Google LLC.
+ */
+
+#define _GNU_SOURCE
+#include <asm/kvm.h>
+#include <linux/kvm.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define VCPU_ID 0
+
+static struct counter_values {
+ uint64_t cntvct_start;
+ uint64_t cntpct;
+ uint64_t cntvct_end;
+} counter_values;
+
+static uint64_t nr_iterations = 1000;
+
+static void do_test(void)
+{
+ /*
+ * Open-coded approach instead of using helper methods to keep a tight
+ * interval around the physical counter read.
+ */
+ asm volatile("isb\n\t"
+ "mrs %[cntvct_start], cntvct_el0\n\t"
+ "isb\n\t"
+ "mrs %[cntpct], cntpct_el0\n\t"
+ "isb\n\t"
+ "mrs %[cntvct_end], cntvct_el0\n\t"
+ "isb\n\t"
+ : [cntvct_start] "=r"(counter_values.cntvct_start),
+ [cntpct] "=r"(counter_values.cntpct),
+ [cntvct_end] "=r"(counter_values.cntvct_end));
+}
+
+static void guest_main(void)
+{
+ int i;
+
+ for (i = 0; i < nr_iterations; i++) {
+ do_test();
+ GUEST_SYNC(i);
+ }
+
+ for (i = 0; i < nr_iterations; i++) {
+ do_test();
+ GUEST_SYNC(i);
+ }
+
+ GUEST_DONE();
+}
+
+static bool enter_guest(struct kvm_vm *vm)
+{
+ struct ucall uc;
+
+ vcpu_ioctl(vm, VCPU_ID, KVM_RUN, NULL);
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_DONE:
+ return true;
+ case UCALL_SYNC:
+ break;
+ case UCALL_ABORT:
+ TEST_ASSERT(false, "%s at %s:%ld", (const char *)uc.args[0],
+ __FILE__, uc.args[1]);
+ break;
+ default:
+ TEST_ASSERT(false, "unexpected exit: %s",
+ exit_reason_str(vcpu_state(vm, VCPU_ID)->exit_reason));
+ break;
+ }
+
+ /* more work to do in the guest */
+ return false;
+}
+
+static double counter_frequency(void)
+{
+ uint32_t freq;
+
+ asm volatile("mrs %0, cntfrq_el0"
+ : "=r" (freq));
+
+ return freq / 1000000.0;
+}
+
+static void log_csv(FILE *csv, bool trapped)
+{
+ double freq = counter_frequency();
+
+ fprintf(csv, "%s,%.02f,%lu,%lu,%lu\n",
+ trapped ? "true" : "false", freq,
+ counter_values.cntvct_start,
+ counter_values.cntpct,
+ counter_values.cntvct_end);
+}
+
+static double run_loop(struct kvm_vm *vm, FILE *csv, bool trapped)
+{
+ double avg = 0;
+ int i;
+
+ for (i = 0; i < nr_iterations; i++) {
+ uint64_t delta;
+
+ TEST_ASSERT(!enter_guest(vm), "guest exited unexpectedly");
+ sync_global_from_guest(vm, counter_values);
+
+ if (csv)
+ log_csv(csv, trapped);
+
+ delta = counter_values.cntvct_end - counter_values.cntvct_start;
+ avg = ((avg * i) + delta) / (i + 1);
+ }
+
+ return avg;
+}
+
+static void setup_counter(struct kvm_vm *vm, uint64_t offset)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER, &offset,
+ true);
+}
+
+static void run_tests(struct kvm_vm *vm, FILE *csv)
+{
+ double avg_trapped, avg_native, freq;
+
+ freq = counter_frequency();
+
+ if (csv)
+ fputs("trapped,freq_mhz,cntvct_start,cntpct,cntvct_end\n", csv);
+
+ /* no physical offsetting; kvm allows reads of cntpct_el0 */
+ setup_counter(vm, 0);
+ avg_native = run_loop(vm, csv, false);
+
+ /* force emulation of the physical counter */
+ setup_counter(vm, 1);
+ avg_trapped = run_loop(vm, csv, true);
+
+ TEST_ASSERT(enter_guest(vm), "guest didn't run to completion");
+ pr_info("%lu iterations: average cycles (@%.02fMHz) native: %.02f, trapped: %.02f\n",
+ nr_iterations, freq, avg_native, avg_trapped);
+}
+
+static void usage(const char *program_name)
+{
+ fprintf(stderr,
+ "Usage: %s [-h] [-o csv_file] [-n iterations]\n"
+ " -h prints this message\n"
+ " -n number of test iterations (default: %lu)\n"
+ " -o csv file to write data\n",
+ program_name, nr_iterations);
+}
+
+int main(int argc, char **argv)
+{
+ struct kvm_vm *vm;
+ FILE *csv = NULL;
+ int opt;
+
+ while ((opt = getopt(argc, argv, "hn:o:")) != -1) {
+ switch (opt) {
+ case 'o':
+ csv = fopen(optarg, "w");
+ if (!csv) {
+ fprintf(stderr, "failed to open file '%s': %d\n",
+ optarg, errno);
+ exit(1);
+ }
+ break;
+ case 'n':
+ nr_iterations = strtoul(optarg, NULL, 0);
+ break;
+ default:
+ fprintf(stderr, "unrecognized option: '-%c'\n", opt);
+ /* fallthrough */
+ case 'h':
+ usage(argv[0]);
+ exit(1);
+ }
+ }
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ sync_global_to_guest(vm, nr_iterations);
+ ucall_init(vm, NULL);
+
+ if (_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER)) {
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_PTIMER not supported.");
+ exit(KSFT_SKIP);
+ }
+
+ run_tests(vm, csv);
+ kvm_vm_free(vm);
+
+ if (csv)
+ fclose(csv);
+}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 73+ messages in thread* [PATCH v2 12/12] selftests: KVM: Add counter emulation benchmark
@ 2021-07-16 21:26 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:26 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
Add a test case for counter emulation on arm64. A side effect of how KVM
handles physical counter offsetting on non-ECV systems is that the
virtual counter will always hit hardware and the physical could be
emulated. Force emulation by writing a nonzero offset to the physical
counter and compare the elapsed cycles to a direct read of the hardware
register.
Reviewed-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Oliver Upton <oupton@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../kvm/aarch64/counter_emulation_benchmark.c | 215 ++++++++++++++++++
3 files changed, 217 insertions(+)
create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 2752813d5090..1d811c6a769b 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
/aarch64/debug-exceptions
+/aarch64/counter_emulation_benchmark
/aarch64/get-reg-list
/aarch64/vgic_init
/s390x/memop
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index d89908108c97..e560a3e74bc2 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -86,6 +86,7 @@ TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
TEST_GEN_PROGS_x86_64 += system_counter_offset_test
TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
+TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark
TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
TEST_GEN_PROGS_aarch64 += demand_paging_test
diff --git a/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
new file mode 100644
index 000000000000..73aeb6cdebfe
--- /dev/null
+++ b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
@@ -0,0 +1,215 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * counter_emulation_benchmark.c -- test to measure the effects of counter
+ * emulation on guest reads of the physical counter.
+ *
+ * Copyright (c) 2021, Google LLC.
+ */
+
+#define _GNU_SOURCE
+#include <asm/kvm.h>
+#include <linux/kvm.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define VCPU_ID 0
+
+static struct counter_values {
+ uint64_t cntvct_start;
+ uint64_t cntpct;
+ uint64_t cntvct_end;
+} counter_values;
+
+static uint64_t nr_iterations = 1000;
+
+static void do_test(void)
+{
+ /*
+ * Open-coded approach instead of using helper methods to keep a tight
+ * interval around the physical counter read.
+ */
+ asm volatile("isb\n\t"
+ "mrs %[cntvct_start], cntvct_el0\n\t"
+ "isb\n\t"
+ "mrs %[cntpct], cntpct_el0\n\t"
+ "isb\n\t"
+ "mrs %[cntvct_end], cntvct_el0\n\t"
+ "isb\n\t"
+ : [cntvct_start] "=r"(counter_values.cntvct_start),
+ [cntpct] "=r"(counter_values.cntpct),
+ [cntvct_end] "=r"(counter_values.cntvct_end));
+}
+
+static void guest_main(void)
+{
+ int i;
+
+ for (i = 0; i < nr_iterations; i++) {
+ do_test();
+ GUEST_SYNC(i);
+ }
+
+ for (i = 0; i < nr_iterations; i++) {
+ do_test();
+ GUEST_SYNC(i);
+ }
+
+ GUEST_DONE();
+}
+
+static bool enter_guest(struct kvm_vm *vm)
+{
+ struct ucall uc;
+
+ vcpu_ioctl(vm, VCPU_ID, KVM_RUN, NULL);
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_DONE:
+ return true;
+ case UCALL_SYNC:
+ break;
+ case UCALL_ABORT:
+ TEST_ASSERT(false, "%s at %s:%ld", (const char *)uc.args[0],
+ __FILE__, uc.args[1]);
+ break;
+ default:
+ TEST_ASSERT(false, "unexpected exit: %s",
+ exit_reason_str(vcpu_state(vm, VCPU_ID)->exit_reason));
+ break;
+ }
+
+ /* more work to do in the guest */
+ return false;
+}
+
+static double counter_frequency(void)
+{
+ uint32_t freq;
+
+ asm volatile("mrs %0, cntfrq_el0"
+ : "=r" (freq));
+
+ return freq / 1000000.0;
+}
+
+static void log_csv(FILE *csv, bool trapped)
+{
+ double freq = counter_frequency();
+
+ fprintf(csv, "%s,%.02f,%lu,%lu,%lu\n",
+ trapped ? "true" : "false", freq,
+ counter_values.cntvct_start,
+ counter_values.cntpct,
+ counter_values.cntvct_end);
+}
+
+static double run_loop(struct kvm_vm *vm, FILE *csv, bool trapped)
+{
+ double avg = 0;
+ int i;
+
+ for (i = 0; i < nr_iterations; i++) {
+ uint64_t delta;
+
+ TEST_ASSERT(!enter_guest(vm), "guest exited unexpectedly");
+ sync_global_from_guest(vm, counter_values);
+
+ if (csv)
+ log_csv(csv, trapped);
+
+ delta = counter_values.cntvct_end - counter_values.cntvct_start;
+ avg = ((avg * i) + delta) / (i + 1);
+ }
+
+ return avg;
+}
+
+static void setup_counter(struct kvm_vm *vm, uint64_t offset)
+{
+ vcpu_access_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER, &offset,
+ true);
+}
+
+static void run_tests(struct kvm_vm *vm, FILE *csv)
+{
+ double avg_trapped, avg_native, freq;
+
+ freq = counter_frequency();
+
+ if (csv)
+ fputs("trapped,freq_mhz,cntvct_start,cntpct,cntvct_end\n", csv);
+
+ /* no physical offsetting; kvm allows reads of cntpct_el0 */
+ setup_counter(vm, 0);
+ avg_native = run_loop(vm, csv, false);
+
+ /* force emulation of the physical counter */
+ setup_counter(vm, 1);
+ avg_trapped = run_loop(vm, csv, true);
+
+ TEST_ASSERT(enter_guest(vm), "guest didn't run to completion");
+ pr_info("%lu iterations: average cycles (@%.02fMHz) native: %.02f, trapped: %.02f\n",
+ nr_iterations, freq, avg_native, avg_trapped);
+}
+
+static void usage(const char *program_name)
+{
+ fprintf(stderr,
+ "Usage: %s [-h] [-o csv_file] [-n iterations]\n"
+ " -h prints this message\n"
+ " -n number of test iterations (default: %lu)\n"
+ " -o csv file to write data\n",
+ program_name, nr_iterations);
+}
+
+int main(int argc, char **argv)
+{
+ struct kvm_vm *vm;
+ FILE *csv = NULL;
+ int opt;
+
+ while ((opt = getopt(argc, argv, "hn:o:")) != -1) {
+ switch (opt) {
+ case 'o':
+ csv = fopen(optarg, "w");
+ if (!csv) {
+ fprintf(stderr, "failed to open file '%s': %d\n",
+ optarg, errno);
+ exit(1);
+ }
+ break;
+ case 'n':
+ nr_iterations = strtoul(optarg, NULL, 0);
+ break;
+ default:
+ fprintf(stderr, "unrecognized option: '-%c'\n", opt);
+ /* fallthrough */
+ case 'h':
+ usage(argv[0]);
+ exit(1);
+ }
+ }
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ sync_global_to_guest(vm, nr_iterations);
+ ucall_init(vm, NULL);
+
+ if (_vcpu_has_device_attr(vm, VCPU_ID, KVM_ARM_VCPU_TIMER_CTRL,
+ KVM_ARM_VCPU_TIMER_OFFSET_PTIMER)) {
+ print_skip("KVM_ARM_VCPU_TIMER_OFFSET_PTIMER not supported.");
+ exit(KSFT_SKIP);
+ }
+
+ run_tests(vm, csv);
+ kvm_vm_free(vm);
+
+ if (csv)
+ fclose(csv);
+}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-16 21:29 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:29 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel
On Fri, Jul 16, 2021 at 2:26 PM Oliver Upton <oupton@google.com> wrote:
>
> KVM's current means of saving/restoring system counters is plagued with
> temporal issues. At least on ARM64 and x86, we migrate the guest's
> system counter by-value through the respective guest system register
> values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> brittle as the state is not idempotent: the host system counter is still
> oscillating between the attempted save and restore. Furthermore, VMMs
> may wish to transparently live migrate guest VMs, meaning that they
> include the elapsed time due to live migration blackout in the guest
> system counter view. The VMM thread could be preempted for any number of
> reasons (scheduler, L0 hypervisor under nested) between the time that
> it calculates the desired guest counter value and when KVM actually sets
> this counter state.
>
> Despite the value-based interface that we present to userspace, KVM
> actually has idempotent guest controls by way of system counter offsets.
> We can avoid all of the issues associated with a value-based interface
> by abstracting these offset controls in new ioctls. This series
> introduces new vCPU device attributes to provide userspace access to the
> vCPU's system counter offset.
>
> Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> essential for a VMM to perform precise migration of the guest's system
> counters.
>
> Patches 2-3 add support for x86 by shoehorning the new controls into the
> pre-existing synchronization heuristics.
>
> Patches 4-5 implement a test for the new additions to
> KVM_{GET,SET}_CLOCK.
>
> Patches 6-7 implement at test for the tsc offset attribute introduced in
> patch 3.
>
> Patch 8 adds a device attribute for the arm64 virtual counter-timer
> offset.
>
> Patch 9 extends the test from patch 7 to cover the arm64 virtual
> counter-timer offset.
>
> Patch 10 adds a device attribute for the arm64 physical counter-timer
> offset. Currently, this is implemented as a synthetic register, forcing
> the guest to trap to the host and emulating the offset in the fast exit
> path. Later down the line we will have hardware with FEAT_ECV, which
> allows the hypervisor to perform physical counter-timer offsetting in
> hardware (CNTPOFF_EL2).
>
> Patch 11 extends the test from patch 7 to cover the arm64 physical
> counter-timer offset.
>
> Patch 12 introduces a benchmark to measure the overhead of emulation in
> patch 10.
>
> Physical counter benchmark
> --------------------------
>
> The following data was collected by running 10000 iterations of the
> benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> parameter.
>
> nVHE
> ----
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 54ns | 148ns |
> | Standard Deviation | 124ns | 122ns |
> | 95th Percentile | 258ns | 348ns |
> +--------------------+--------+---------+
>
> VHE
> ---
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 53ns | 152ns |
> | Standard Deviation | 92ns | 94ns |
> | 95th Percentile | 204ns | 307ns |
> +--------------------+--------+---------+
>
> This series applies cleanly to the following commit:
>
> 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
v1: https://lore.kernel.org/kvm/20210608214742.1897483-1-oupton@google.com/
> v1 -> v2:
> - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> - Added the (realtime, host_tsc) instant support to
> KVM_{GET,SET}_CLOCK
> - Changed the arm64 implementation to broadcast counter offset values
> to all vCPUs in a guest. This upholds the architectural expectations
> of a consistent counter-timer across CPUs.
> - Fixed a bug with traps in VHE mode. We now configure traps on every
> transition into a guest to handle differing VMs (trapped, emulated).
>
> Oliver Upton (12):
> KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
> KVM: x86: Refactor tsc synchronization code
> KVM: x86: Expose TSC offset controls to userspace
> tools: arch: x86: pull in pvclock headers
> selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
> selftests: KVM: Add helpers for vCPU device attributes
> selftests: KVM: Introduce system counter offset test
> KVM: arm64: Allow userspace to configure a vCPU's virtual offset
> selftests: KVM: Add support for aarch64 to system_counter_offset_test
> KVM: arm64: Provide userspace access to the physical counter offset
> selftests: KVM: Test physical counter offsetting
> selftests: KVM: Add counter emulation benchmark
>
> Documentation/virt/kvm/api.rst | 42 +-
> Documentation/virt/kvm/locking.rst | 11 +
> arch/arm64/include/asm/kvm_host.h | 1 +
> arch/arm64/include/asm/kvm_hyp.h | 2 -
> arch/arm64/include/asm/sysreg.h | 1 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kvm/arch_timer.c | 118 ++++-
> arch/arm64/kvm/arm.c | 4 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +
> arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++
> arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
> arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +-
> arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++
> arch/x86/include/asm/kvm_host.h | 4 +
> arch/x86/include/uapi/asm/kvm.h | 4 +
> arch/x86/kvm/x86.c | 421 ++++++++++++++----
> include/kvm/arm_arch_timer.h | 2 -
> include/uapi/linux/kvm.h | 7 +-
> tools/arch/x86/include/asm/pvclock-abi.h | 48 ++
> tools/arch/x86/include/asm/pvclock.h | 103 +++++
> tools/testing/selftests/kvm/.gitignore | 3 +
> tools/testing/selftests/kvm/Makefile | 4 +
> .../kvm/aarch64/counter_emulation_benchmark.c | 215 +++++++++
> .../selftests/kvm/include/aarch64/processor.h | 24 +
> .../testing/selftests/kvm/include/kvm_util.h | 11 +
> tools/testing/selftests/kvm/lib/kvm_util.c | 38 ++
> .../kvm/system_counter_offset_test.c | 206 +++++++++
> .../selftests/kvm/x86_64/kvm_clock_test.c | 210 +++++++++
> 28 files changed, 1447 insertions(+), 133 deletions(-)
> create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
> create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
> create mode 100644 tools/arch/x86/include/asm/pvclock.h
> create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
> create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
> create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
>
> --
> 2.32.0.402.g57bb445576-goog
>
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-16 21:29 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:29 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier,
Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang,
Raghavendra Rao Anata, James Morse, Alexandru Elisei,
Suzuki K Poulose, linux-arm-kernel
On Fri, Jul 16, 2021 at 2:26 PM Oliver Upton <oupton@google.com> wrote:
>
> KVM's current means of saving/restoring system counters is plagued with
> temporal issues. At least on ARM64 and x86, we migrate the guest's
> system counter by-value through the respective guest system register
> values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> brittle as the state is not idempotent: the host system counter is still
> oscillating between the attempted save and restore. Furthermore, VMMs
> may wish to transparently live migrate guest VMs, meaning that they
> include the elapsed time due to live migration blackout in the guest
> system counter view. The VMM thread could be preempted for any number of
> reasons (scheduler, L0 hypervisor under nested) between the time that
> it calculates the desired guest counter value and when KVM actually sets
> this counter state.
>
> Despite the value-based interface that we present to userspace, KVM
> actually has idempotent guest controls by way of system counter offsets.
> We can avoid all of the issues associated with a value-based interface
> by abstracting these offset controls in new ioctls. This series
> introduces new vCPU device attributes to provide userspace access to the
> vCPU's system counter offset.
>
> Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> essential for a VMM to perform precise migration of the guest's system
> counters.
>
> Patches 2-3 add support for x86 by shoehorning the new controls into the
> pre-existing synchronization heuristics.
>
> Patches 4-5 implement a test for the new additions to
> KVM_{GET,SET}_CLOCK.
>
> Patches 6-7 implement at test for the tsc offset attribute introduced in
> patch 3.
>
> Patch 8 adds a device attribute for the arm64 virtual counter-timer
> offset.
>
> Patch 9 extends the test from patch 7 to cover the arm64 virtual
> counter-timer offset.
>
> Patch 10 adds a device attribute for the arm64 physical counter-timer
> offset. Currently, this is implemented as a synthetic register, forcing
> the guest to trap to the host and emulating the offset in the fast exit
> path. Later down the line we will have hardware with FEAT_ECV, which
> allows the hypervisor to perform physical counter-timer offsetting in
> hardware (CNTPOFF_EL2).
>
> Patch 11 extends the test from patch 7 to cover the arm64 physical
> counter-timer offset.
>
> Patch 12 introduces a benchmark to measure the overhead of emulation in
> patch 10.
>
> Physical counter benchmark
> --------------------------
>
> The following data was collected by running 10000 iterations of the
> benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> parameter.
>
> nVHE
> ----
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 54ns | 148ns |
> | Standard Deviation | 124ns | 122ns |
> | 95th Percentile | 258ns | 348ns |
> +--------------------+--------+---------+
>
> VHE
> ---
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 53ns | 152ns |
> | Standard Deviation | 92ns | 94ns |
> | 95th Percentile | 204ns | 307ns |
> +--------------------+--------+---------+
>
> This series applies cleanly to the following commit:
>
> 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
v1: https://lore.kernel.org/kvm/20210608214742.1897483-1-oupton@google.com/
> v1 -> v2:
> - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> - Added the (realtime, host_tsc) instant support to
> KVM_{GET,SET}_CLOCK
> - Changed the arm64 implementation to broadcast counter offset values
> to all vCPUs in a guest. This upholds the architectural expectations
> of a consistent counter-timer across CPUs.
> - Fixed a bug with traps in VHE mode. We now configure traps on every
> transition into a guest to handle differing VMs (trapped, emulated).
>
> Oliver Upton (12):
> KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
> KVM: x86: Refactor tsc synchronization code
> KVM: x86: Expose TSC offset controls to userspace
> tools: arch: x86: pull in pvclock headers
> selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
> selftests: KVM: Add helpers for vCPU device attributes
> selftests: KVM: Introduce system counter offset test
> KVM: arm64: Allow userspace to configure a vCPU's virtual offset
> selftests: KVM: Add support for aarch64 to system_counter_offset_test
> KVM: arm64: Provide userspace access to the physical counter offset
> selftests: KVM: Test physical counter offsetting
> selftests: KVM: Add counter emulation benchmark
>
> Documentation/virt/kvm/api.rst | 42 +-
> Documentation/virt/kvm/locking.rst | 11 +
> arch/arm64/include/asm/kvm_host.h | 1 +
> arch/arm64/include/asm/kvm_hyp.h | 2 -
> arch/arm64/include/asm/sysreg.h | 1 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kvm/arch_timer.c | 118 ++++-
> arch/arm64/kvm/arm.c | 4 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +
> arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++
> arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
> arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +-
> arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++
> arch/x86/include/asm/kvm_host.h | 4 +
> arch/x86/include/uapi/asm/kvm.h | 4 +
> arch/x86/kvm/x86.c | 421 ++++++++++++++----
> include/kvm/arm_arch_timer.h | 2 -
> include/uapi/linux/kvm.h | 7 +-
> tools/arch/x86/include/asm/pvclock-abi.h | 48 ++
> tools/arch/x86/include/asm/pvclock.h | 103 +++++
> tools/testing/selftests/kvm/.gitignore | 3 +
> tools/testing/selftests/kvm/Makefile | 4 +
> .../kvm/aarch64/counter_emulation_benchmark.c | 215 +++++++++
> .../selftests/kvm/include/aarch64/processor.h | 24 +
> .../testing/selftests/kvm/include/kvm_util.h | 11 +
> tools/testing/selftests/kvm/lib/kvm_util.c | 38 ++
> .../kvm/system_counter_offset_test.c | 206 +++++++++
> .../selftests/kvm/x86_64/kvm_clock_test.c | 210 +++++++++
> 28 files changed, 1447 insertions(+), 133 deletions(-)
> create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
> create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
> create mode 100644 tools/arch/x86/include/asm/pvclock.h
> create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
> create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
> create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
>
> --
> 2.32.0.402.g57bb445576-goog
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-16 21:29 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-16 21:29 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 2:26 PM Oliver Upton <oupton@google.com> wrote:
>
> KVM's current means of saving/restoring system counters is plagued with
> temporal issues. At least on ARM64 and x86, we migrate the guest's
> system counter by-value through the respective guest system register
> values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> brittle as the state is not idempotent: the host system counter is still
> oscillating between the attempted save and restore. Furthermore, VMMs
> may wish to transparently live migrate guest VMs, meaning that they
> include the elapsed time due to live migration blackout in the guest
> system counter view. The VMM thread could be preempted for any number of
> reasons (scheduler, L0 hypervisor under nested) between the time that
> it calculates the desired guest counter value and when KVM actually sets
> this counter state.
>
> Despite the value-based interface that we present to userspace, KVM
> actually has idempotent guest controls by way of system counter offsets.
> We can avoid all of the issues associated with a value-based interface
> by abstracting these offset controls in new ioctls. This series
> introduces new vCPU device attributes to provide userspace access to the
> vCPU's system counter offset.
>
> Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> essential for a VMM to perform precise migration of the guest's system
> counters.
>
> Patches 2-3 add support for x86 by shoehorning the new controls into the
> pre-existing synchronization heuristics.
>
> Patches 4-5 implement a test for the new additions to
> KVM_{GET,SET}_CLOCK.
>
> Patches 6-7 implement at test for the tsc offset attribute introduced in
> patch 3.
>
> Patch 8 adds a device attribute for the arm64 virtual counter-timer
> offset.
>
> Patch 9 extends the test from patch 7 to cover the arm64 virtual
> counter-timer offset.
>
> Patch 10 adds a device attribute for the arm64 physical counter-timer
> offset. Currently, this is implemented as a synthetic register, forcing
> the guest to trap to the host and emulating the offset in the fast exit
> path. Later down the line we will have hardware with FEAT_ECV, which
> allows the hypervisor to perform physical counter-timer offsetting in
> hardware (CNTPOFF_EL2).
>
> Patch 11 extends the test from patch 7 to cover the arm64 physical
> counter-timer offset.
>
> Patch 12 introduces a benchmark to measure the overhead of emulation in
> patch 10.
>
> Physical counter benchmark
> --------------------------
>
> The following data was collected by running 10000 iterations of the
> benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> parameter.
>
> nVHE
> ----
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 54ns | 148ns |
> | Standard Deviation | 124ns | 122ns |
> | 95th Percentile | 258ns | 348ns |
> +--------------------+--------+---------+
>
> VHE
> ---
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 53ns | 152ns |
> | Standard Deviation | 92ns | 94ns |
> | 95th Percentile | 204ns | 307ns |
> +--------------------+--------+---------+
>
> This series applies cleanly to the following commit:
>
> 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
v1: https://lore.kernel.org/kvm/20210608214742.1897483-1-oupton@google.com/
> v1 -> v2:
> - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> - Added the (realtime, host_tsc) instant support to
> KVM_{GET,SET}_CLOCK
> - Changed the arm64 implementation to broadcast counter offset values
> to all vCPUs in a guest. This upholds the architectural expectations
> of a consistent counter-timer across CPUs.
> - Fixed a bug with traps in VHE mode. We now configure traps on every
> transition into a guest to handle differing VMs (trapped, emulated).
>
> Oliver Upton (12):
> KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
> KVM: x86: Refactor tsc synchronization code
> KVM: x86: Expose TSC offset controls to userspace
> tools: arch: x86: pull in pvclock headers
> selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
> selftests: KVM: Add helpers for vCPU device attributes
> selftests: KVM: Introduce system counter offset test
> KVM: arm64: Allow userspace to configure a vCPU's virtual offset
> selftests: KVM: Add support for aarch64 to system_counter_offset_test
> KVM: arm64: Provide userspace access to the physical counter offset
> selftests: KVM: Test physical counter offsetting
> selftests: KVM: Add counter emulation benchmark
>
> Documentation/virt/kvm/api.rst | 42 +-
> Documentation/virt/kvm/locking.rst | 11 +
> arch/arm64/include/asm/kvm_host.h | 1 +
> arch/arm64/include/asm/kvm_hyp.h | 2 -
> arch/arm64/include/asm/sysreg.h | 1 +
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kvm/arch_timer.c | 118 ++++-
> arch/arm64/kvm/arm.c | 4 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +
> arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++
> arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
> arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +-
> arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++
> arch/x86/include/asm/kvm_host.h | 4 +
> arch/x86/include/uapi/asm/kvm.h | 4 +
> arch/x86/kvm/x86.c | 421 ++++++++++++++----
> include/kvm/arm_arch_timer.h | 2 -
> include/uapi/linux/kvm.h | 7 +-
> tools/arch/x86/include/asm/pvclock-abi.h | 48 ++
> tools/arch/x86/include/asm/pvclock.h | 103 +++++
> tools/testing/selftests/kvm/.gitignore | 3 +
> tools/testing/selftests/kvm/Makefile | 4 +
> .../kvm/aarch64/counter_emulation_benchmark.c | 215 +++++++++
> .../selftests/kvm/include/aarch64/processor.h | 24 +
> .../testing/selftests/kvm/include/kvm_util.h | 11 +
> tools/testing/selftests/kvm/lib/kvm_util.c | 38 ++
> .../kvm/system_counter_offset_test.c | 206 +++++++++
> .../selftests/kvm/x86_64/kvm_clock_test.c | 210 +++++++++
> 28 files changed, 1447 insertions(+), 133 deletions(-)
> create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
> create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
> create mode 100644 tools/arch/x86/include/asm/pvclock.h
> create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
> create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
> create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
>
> --
> 2.32.0.402.g57bb445576-goog
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
2021-07-16 21:26 ` Oliver Upton
(?)
@ 2021-07-21 15:28 ` Andrew Jones
-1 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:28 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:17PM +0000, Oliver Upton wrote:
> KVM's current means of saving/restoring system counters is plagued with
> temporal issues. At least on ARM64 and x86, we migrate the guest's
> system counter by-value through the respective guest system register
> values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> brittle as the state is not idempotent: the host system counter is still
> oscillating between the attempted save and restore. Furthermore, VMMs
> may wish to transparently live migrate guest VMs, meaning that they
> include the elapsed time due to live migration blackout in the guest
> system counter view. The VMM thread could be preempted for any number of
> reasons (scheduler, L0 hypervisor under nested) between the time that
> it calculates the desired guest counter value and when KVM actually sets
> this counter state.
>
> Despite the value-based interface that we present to userspace, KVM
> actually has idempotent guest controls by way of system counter offsets.
> We can avoid all of the issues associated with a value-based interface
> by abstracting these offset controls in new ioctls. This series
> introduces new vCPU device attributes to provide userspace access to the
> vCPU's system counter offset.
>
> Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> essential for a VMM to perform precise migration of the guest's system
> counters.
>
> Patches 2-3 add support for x86 by shoehorning the new controls into the
> pre-existing synchronization heuristics.
>
> Patches 4-5 implement a test for the new additions to
> KVM_{GET,SET}_CLOCK.
>
> Patches 6-7 implement at test for the tsc offset attribute introduced in
> patch 3.
>
> Patch 8 adds a device attribute for the arm64 virtual counter-timer
> offset.
>
> Patch 9 extends the test from patch 7 to cover the arm64 virtual
> counter-timer offset.
>
> Patch 10 adds a device attribute for the arm64 physical counter-timer
> offset. Currently, this is implemented as a synthetic register, forcing
> the guest to trap to the host and emulating the offset in the fast exit
> path. Later down the line we will have hardware with FEAT_ECV, which
> allows the hypervisor to perform physical counter-timer offsetting in
> hardware (CNTPOFF_EL2).
>
> Patch 11 extends the test from patch 7 to cover the arm64 physical
> counter-timer offset.
>
> Patch 12 introduces a benchmark to measure the overhead of emulation in
> patch 10.
>
> Physical counter benchmark
> --------------------------
>
> The following data was collected by running 10000 iterations of the
> benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> parameter.
>
> nVHE
> ----
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 54ns | 148ns |
> | Standard Deviation | 124ns | 122ns |
> | 95th Percentile | 258ns | 348ns |
> +--------------------+--------+---------+
>
> VHE
> ---
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 53ns | 152ns |
> | Standard Deviation | 92ns | 94ns |
> | 95th Percentile | 204ns | 307ns |
> +--------------------+--------+---------+
>
> This series applies cleanly to the following commit:
>
> 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
>
> v1 -> v2:
> - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> - Added the (realtime, host_tsc) instant support to
> KVM_{GET,SET}_CLOCK
> - Changed the arm64 implementation to broadcast counter offset values
> to all vCPUs in a guest. This upholds the architectural expectations
> of a consistent counter-timer across CPUs.
> - Fixed a bug with traps in VHE mode. We now configure traps on every
> transition into a guest to handle differing VMs (trapped, emulated).
>
Oops, I see there's a v3 of this series. I'll switch to reviewing that. I
think my comments / r-b's apply to that version as well though.
Thanks,
drew
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-21 15:28 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:28 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:17PM +0000, Oliver Upton wrote:
> KVM's current means of saving/restoring system counters is plagued with
> temporal issues. At least on ARM64 and x86, we migrate the guest's
> system counter by-value through the respective guest system register
> values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> brittle as the state is not idempotent: the host system counter is still
> oscillating between the attempted save and restore. Furthermore, VMMs
> may wish to transparently live migrate guest VMs, meaning that they
> include the elapsed time due to live migration blackout in the guest
> system counter view. The VMM thread could be preempted for any number of
> reasons (scheduler, L0 hypervisor under nested) between the time that
> it calculates the desired guest counter value and when KVM actually sets
> this counter state.
>
> Despite the value-based interface that we present to userspace, KVM
> actually has idempotent guest controls by way of system counter offsets.
> We can avoid all of the issues associated with a value-based interface
> by abstracting these offset controls in new ioctls. This series
> introduces new vCPU device attributes to provide userspace access to the
> vCPU's system counter offset.
>
> Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> essential for a VMM to perform precise migration of the guest's system
> counters.
>
> Patches 2-3 add support for x86 by shoehorning the new controls into the
> pre-existing synchronization heuristics.
>
> Patches 4-5 implement a test for the new additions to
> KVM_{GET,SET}_CLOCK.
>
> Patches 6-7 implement at test for the tsc offset attribute introduced in
> patch 3.
>
> Patch 8 adds a device attribute for the arm64 virtual counter-timer
> offset.
>
> Patch 9 extends the test from patch 7 to cover the arm64 virtual
> counter-timer offset.
>
> Patch 10 adds a device attribute for the arm64 physical counter-timer
> offset. Currently, this is implemented as a synthetic register, forcing
> the guest to trap to the host and emulating the offset in the fast exit
> path. Later down the line we will have hardware with FEAT_ECV, which
> allows the hypervisor to perform physical counter-timer offsetting in
> hardware (CNTPOFF_EL2).
>
> Patch 11 extends the test from patch 7 to cover the arm64 physical
> counter-timer offset.
>
> Patch 12 introduces a benchmark to measure the overhead of emulation in
> patch 10.
>
> Physical counter benchmark
> --------------------------
>
> The following data was collected by running 10000 iterations of the
> benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> parameter.
>
> nVHE
> ----
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 54ns | 148ns |
> | Standard Deviation | 124ns | 122ns |
> | 95th Percentile | 258ns | 348ns |
> +--------------------+--------+---------+
>
> VHE
> ---
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 53ns | 152ns |
> | Standard Deviation | 92ns | 94ns |
> | 95th Percentile | 204ns | 307ns |
> +--------------------+--------+---------+
>
> This series applies cleanly to the following commit:
>
> 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
>
> v1 -> v2:
> - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> - Added the (realtime, host_tsc) instant support to
> KVM_{GET,SET}_CLOCK
> - Changed the arm64 implementation to broadcast counter offset values
> to all vCPUs in a guest. This upholds the architectural expectations
> of a consistent counter-timer across CPUs.
> - Fixed a bug with traps in VHE mode. We now configure traps on every
> transition into a guest to handle differing VMs (trapped, emulated).
>
Oops, I see there's a v3 of this series. I'll switch to reviewing that. I
think my comments / r-b's apply to that version as well though.
Thanks,
drew
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-21 15:28 ` Andrew Jones
0 siblings, 0 replies; 73+ messages in thread
From: Andrew Jones @ 2021-07-21 15:28 UTC (permalink / raw)
To: Oliver Upton
Cc: kvm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini, kvmarm,
linux-arm-kernel, Jim Mattson
On Fri, Jul 16, 2021 at 09:26:17PM +0000, Oliver Upton wrote:
> KVM's current means of saving/restoring system counters is plagued with
> temporal issues. At least on ARM64 and x86, we migrate the guest's
> system counter by-value through the respective guest system register
> values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> brittle as the state is not idempotent: the host system counter is still
> oscillating between the attempted save and restore. Furthermore, VMMs
> may wish to transparently live migrate guest VMs, meaning that they
> include the elapsed time due to live migration blackout in the guest
> system counter view. The VMM thread could be preempted for any number of
> reasons (scheduler, L0 hypervisor under nested) between the time that
> it calculates the desired guest counter value and when KVM actually sets
> this counter state.
>
> Despite the value-based interface that we present to userspace, KVM
> actually has idempotent guest controls by way of system counter offsets.
> We can avoid all of the issues associated with a value-based interface
> by abstracting these offset controls in new ioctls. This series
> introduces new vCPU device attributes to provide userspace access to the
> vCPU's system counter offset.
>
> Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> essential for a VMM to perform precise migration of the guest's system
> counters.
>
> Patches 2-3 add support for x86 by shoehorning the new controls into the
> pre-existing synchronization heuristics.
>
> Patches 4-5 implement a test for the new additions to
> KVM_{GET,SET}_CLOCK.
>
> Patches 6-7 implement at test for the tsc offset attribute introduced in
> patch 3.
>
> Patch 8 adds a device attribute for the arm64 virtual counter-timer
> offset.
>
> Patch 9 extends the test from patch 7 to cover the arm64 virtual
> counter-timer offset.
>
> Patch 10 adds a device attribute for the arm64 physical counter-timer
> offset. Currently, this is implemented as a synthetic register, forcing
> the guest to trap to the host and emulating the offset in the fast exit
> path. Later down the line we will have hardware with FEAT_ECV, which
> allows the hypervisor to perform physical counter-timer offsetting in
> hardware (CNTPOFF_EL2).
>
> Patch 11 extends the test from patch 7 to cover the arm64 physical
> counter-timer offset.
>
> Patch 12 introduces a benchmark to measure the overhead of emulation in
> patch 10.
>
> Physical counter benchmark
> --------------------------
>
> The following data was collected by running 10000 iterations of the
> benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> parameter.
>
> nVHE
> ----
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 54ns | 148ns |
> | Standard Deviation | 124ns | 122ns |
> | 95th Percentile | 258ns | 348ns |
> +--------------------+--------+---------+
>
> VHE
> ---
>
> +--------------------+--------+---------+
> | Metric | Native | Trapped |
> +--------------------+--------+---------+
> | Average | 53ns | 152ns |
> | Standard Deviation | 92ns | 94ns |
> | 95th Percentile | 204ns | 307ns |
> +--------------------+--------+---------+
>
> This series applies cleanly to the following commit:
>
> 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
>
> v1 -> v2:
> - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> - Added the (realtime, host_tsc) instant support to
> KVM_{GET,SET}_CLOCK
> - Changed the arm64 implementation to broadcast counter offset values
> to all vCPUs in a guest. This upholds the architectural expectations
> of a consistent counter-timer across CPUs.
> - Fixed a bug with traps in VHE mode. We now configure traps on every
> transition into a guest to handle differing VMs (trapped, emulated).
>
Oops, I see there's a v3 of this series. I'll switch to reviewing that. I
think my comments / r-b's apply to that version as well though.
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
2021-07-21 15:28 ` Andrew Jones
(?)
@ 2021-07-22 15:42 ` Oliver Upton
-1 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-22 15:42 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Wed, Jul 21, 2021 at 8:28 AM Andrew Jones <drjones@redhat.com> wrote:
>
> On Fri, Jul 16, 2021 at 09:26:17PM +0000, Oliver Upton wrote:
> > KVM's current means of saving/restoring system counters is plagued with
> > temporal issues. At least on ARM64 and x86, we migrate the guest's
> > system counter by-value through the respective guest system register
> > values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> > brittle as the state is not idempotent: the host system counter is still
> > oscillating between the attempted save and restore. Furthermore, VMMs
> > may wish to transparently live migrate guest VMs, meaning that they
> > include the elapsed time due to live migration blackout in the guest
> > system counter view. The VMM thread could be preempted for any number of
> > reasons (scheduler, L0 hypervisor under nested) between the time that
> > it calculates the desired guest counter value and when KVM actually sets
> > this counter state.
> >
> > Despite the value-based interface that we present to userspace, KVM
> > actually has idempotent guest controls by way of system counter offsets.
> > We can avoid all of the issues associated with a value-based interface
> > by abstracting these offset controls in new ioctls. This series
> > introduces new vCPU device attributes to provide userspace access to the
> > vCPU's system counter offset.
> >
> > Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> > ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> > essential for a VMM to perform precise migration of the guest's system
> > counters.
> >
> > Patches 2-3 add support for x86 by shoehorning the new controls into the
> > pre-existing synchronization heuristics.
> >
> > Patches 4-5 implement a test for the new additions to
> > KVM_{GET,SET}_CLOCK.
> >
> > Patches 6-7 implement at test for the tsc offset attribute introduced in
> > patch 3.
> >
> > Patch 8 adds a device attribute for the arm64 virtual counter-timer
> > offset.
> >
> > Patch 9 extends the test from patch 7 to cover the arm64 virtual
> > counter-timer offset.
> >
> > Patch 10 adds a device attribute for the arm64 physical counter-timer
> > offset. Currently, this is implemented as a synthetic register, forcing
> > the guest to trap to the host and emulating the offset in the fast exit
> > path. Later down the line we will have hardware with FEAT_ECV, which
> > allows the hypervisor to perform physical counter-timer offsetting in
> > hardware (CNTPOFF_EL2).
> >
> > Patch 11 extends the test from patch 7 to cover the arm64 physical
> > counter-timer offset.
> >
> > Patch 12 introduces a benchmark to measure the overhead of emulation in
> > patch 10.
> >
> > Physical counter benchmark
> > --------------------------
> >
> > The following data was collected by running 10000 iterations of the
> > benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> > machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> > for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> > parameter.
> >
> > nVHE
> > ----
> >
> > +--------------------+--------+---------+
> > | Metric | Native | Trapped |
> > +--------------------+--------+---------+
> > | Average | 54ns | 148ns |
> > | Standard Deviation | 124ns | 122ns |
> > | 95th Percentile | 258ns | 348ns |
> > +--------------------+--------+---------+
> >
> > VHE
> > ---
> >
> > +--------------------+--------+---------+
> > | Metric | Native | Trapped |
> > +--------------------+--------+---------+
> > | Average | 53ns | 152ns |
> > | Standard Deviation | 92ns | 94ns |
> > | 95th Percentile | 204ns | 307ns |
> > +--------------------+--------+---------+
> >
> > This series applies cleanly to the following commit:
> >
> > 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
> >
> > v1 -> v2:
> > - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> > - Added the (realtime, host_tsc) instant support to
> > KVM_{GET,SET}_CLOCK
> > - Changed the arm64 implementation to broadcast counter offset values
> > to all vCPUs in a guest. This upholds the architectural expectations
> > of a consistent counter-timer across CPUs.
> > - Fixed a bug with traps in VHE mode. We now configure traps on every
> > transition into a guest to handle differing VMs (trapped, emulated).
> >
>
> Oops, I see there's a v3 of this series. I'll switch to reviewing that. I
> think my comments / r-b's apply to that version as well though.
Hey Drew,
Thanks for the review. I'll address your comments from both v2 and v3
in the next series.
--
Thanks,
Oliver
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-22 15:42 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-22 15:42 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, kvmarm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
On Wed, Jul 21, 2021 at 8:28 AM Andrew Jones <drjones@redhat.com> wrote:
>
> On Fri, Jul 16, 2021 at 09:26:17PM +0000, Oliver Upton wrote:
> > KVM's current means of saving/restoring system counters is plagued with
> > temporal issues. At least on ARM64 and x86, we migrate the guest's
> > system counter by-value through the respective guest system register
> > values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> > brittle as the state is not idempotent: the host system counter is still
> > oscillating between the attempted save and restore. Furthermore, VMMs
> > may wish to transparently live migrate guest VMs, meaning that they
> > include the elapsed time due to live migration blackout in the guest
> > system counter view. The VMM thread could be preempted for any number of
> > reasons (scheduler, L0 hypervisor under nested) between the time that
> > it calculates the desired guest counter value and when KVM actually sets
> > this counter state.
> >
> > Despite the value-based interface that we present to userspace, KVM
> > actually has idempotent guest controls by way of system counter offsets.
> > We can avoid all of the issues associated with a value-based interface
> > by abstracting these offset controls in new ioctls. This series
> > introduces new vCPU device attributes to provide userspace access to the
> > vCPU's system counter offset.
> >
> > Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> > ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> > essential for a VMM to perform precise migration of the guest's system
> > counters.
> >
> > Patches 2-3 add support for x86 by shoehorning the new controls into the
> > pre-existing synchronization heuristics.
> >
> > Patches 4-5 implement a test for the new additions to
> > KVM_{GET,SET}_CLOCK.
> >
> > Patches 6-7 implement at test for the tsc offset attribute introduced in
> > patch 3.
> >
> > Patch 8 adds a device attribute for the arm64 virtual counter-timer
> > offset.
> >
> > Patch 9 extends the test from patch 7 to cover the arm64 virtual
> > counter-timer offset.
> >
> > Patch 10 adds a device attribute for the arm64 physical counter-timer
> > offset. Currently, this is implemented as a synthetic register, forcing
> > the guest to trap to the host and emulating the offset in the fast exit
> > path. Later down the line we will have hardware with FEAT_ECV, which
> > allows the hypervisor to perform physical counter-timer offsetting in
> > hardware (CNTPOFF_EL2).
> >
> > Patch 11 extends the test from patch 7 to cover the arm64 physical
> > counter-timer offset.
> >
> > Patch 12 introduces a benchmark to measure the overhead of emulation in
> > patch 10.
> >
> > Physical counter benchmark
> > --------------------------
> >
> > The following data was collected by running 10000 iterations of the
> > benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> > machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> > for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> > parameter.
> >
> > nVHE
> > ----
> >
> > +--------------------+--------+---------+
> > | Metric | Native | Trapped |
> > +--------------------+--------+---------+
> > | Average | 54ns | 148ns |
> > | Standard Deviation | 124ns | 122ns |
> > | 95th Percentile | 258ns | 348ns |
> > +--------------------+--------+---------+
> >
> > VHE
> > ---
> >
> > +--------------------+--------+---------+
> > | Metric | Native | Trapped |
> > +--------------------+--------+---------+
> > | Average | 53ns | 152ns |
> > | Standard Deviation | 92ns | 94ns |
> > | 95th Percentile | 204ns | 307ns |
> > +--------------------+--------+---------+
> >
> > This series applies cleanly to the following commit:
> >
> > 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
> >
> > v1 -> v2:
> > - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> > - Added the (realtime, host_tsc) instant support to
> > KVM_{GET,SET}_CLOCK
> > - Changed the arm64 implementation to broadcast counter offset values
> > to all vCPUs in a guest. This upholds the architectural expectations
> > of a consistent counter-timer across CPUs.
> > - Fixed a bug with traps in VHE mode. We now configure traps on every
> > transition into a guest to handle differing VMs (trapped, emulated).
> >
>
> Oops, I see there's a v3 of this series. I'll switch to reviewing that. I
> think my comments / r-b's apply to that version as well though.
Hey Drew,
Thanks for the review. I'll address your comments from both v2 and v3
in the next series.
--
Thanks,
Oliver
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [PATCH v2 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-22 15:42 ` Oliver Upton
0 siblings, 0 replies; 73+ messages in thread
From: Oliver Upton @ 2021-07-22 15:42 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini, kvmarm,
linux-arm-kernel, Jim Mattson
On Wed, Jul 21, 2021 at 8:28 AM Andrew Jones <drjones@redhat.com> wrote:
>
> On Fri, Jul 16, 2021 at 09:26:17PM +0000, Oliver Upton wrote:
> > KVM's current means of saving/restoring system counters is plagued with
> > temporal issues. At least on ARM64 and x86, we migrate the guest's
> > system counter by-value through the respective guest system register
> > values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> > brittle as the state is not idempotent: the host system counter is still
> > oscillating between the attempted save and restore. Furthermore, VMMs
> > may wish to transparently live migrate guest VMs, meaning that they
> > include the elapsed time due to live migration blackout in the guest
> > system counter view. The VMM thread could be preempted for any number of
> > reasons (scheduler, L0 hypervisor under nested) between the time that
> > it calculates the desired guest counter value and when KVM actually sets
> > this counter state.
> >
> > Despite the value-based interface that we present to userspace, KVM
> > actually has idempotent guest controls by way of system counter offsets.
> > We can avoid all of the issues associated with a value-based interface
> > by abstracting these offset controls in new ioctls. This series
> > introduces new vCPU device attributes to provide userspace access to the
> > vCPU's system counter offset.
> >
> > Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> > ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> > essential for a VMM to perform precise migration of the guest's system
> > counters.
> >
> > Patches 2-3 add support for x86 by shoehorning the new controls into the
> > pre-existing synchronization heuristics.
> >
> > Patches 4-5 implement a test for the new additions to
> > KVM_{GET,SET}_CLOCK.
> >
> > Patches 6-7 implement at test for the tsc offset attribute introduced in
> > patch 3.
> >
> > Patch 8 adds a device attribute for the arm64 virtual counter-timer
> > offset.
> >
> > Patch 9 extends the test from patch 7 to cover the arm64 virtual
> > counter-timer offset.
> >
> > Patch 10 adds a device attribute for the arm64 physical counter-timer
> > offset. Currently, this is implemented as a synthetic register, forcing
> > the guest to trap to the host and emulating the offset in the fast exit
> > path. Later down the line we will have hardware with FEAT_ECV, which
> > allows the hypervisor to perform physical counter-timer offsetting in
> > hardware (CNTPOFF_EL2).
> >
> > Patch 11 extends the test from patch 7 to cover the arm64 physical
> > counter-timer offset.
> >
> > Patch 12 introduces a benchmark to measure the overhead of emulation in
> > patch 10.
> >
> > Physical counter benchmark
> > --------------------------
> >
> > The following data was collected by running 10000 iterations of the
> > benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
> > machine with 2 80-core Ampere Altra SoCs. Measurements were collected
> > for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
> > parameter.
> >
> > nVHE
> > ----
> >
> > +--------------------+--------+---------+
> > | Metric | Native | Trapped |
> > +--------------------+--------+---------+
> > | Average | 54ns | 148ns |
> > | Standard Deviation | 124ns | 122ns |
> > | 95th Percentile | 258ns | 348ns |
> > +--------------------+--------+---------+
> >
> > VHE
> > ---
> >
> > +--------------------+--------+---------+
> > | Metric | Native | Trapped |
> > +--------------------+--------+---------+
> > | Average | 53ns | 152ns |
> > | Standard Deviation | 92ns | 94ns |
> > | 95th Percentile | 204ns | 307ns |
> > +--------------------+--------+---------+
> >
> > This series applies cleanly to the following commit:
> >
> > 1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
> >
> > v1 -> v2:
> > - Reimplemented as vCPU device attributes instead of a distinct ioctl.
> > - Added the (realtime, host_tsc) instant support to
> > KVM_{GET,SET}_CLOCK
> > - Changed the arm64 implementation to broadcast counter offset values
> > to all vCPUs in a guest. This upholds the architectural expectations
> > of a consistent counter-timer across CPUs.
> > - Fixed a bug with traps in VHE mode. We now configure traps on every
> > transition into a guest to handle differing VMs (trapped, emulated).
> >
>
> Oops, I see there's a v3 of this series. I'll switch to reviewing that. I
> think my comments / r-b's apply to that version as well though.
Hey Drew,
Thanks for the review. I'll address your comments from both v2 and v3
in the next series.
--
Thanks,
Oliver
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 73+ messages in thread