kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v16 0/9] Enable ptp_kvm for arm/arm64
@ 2020-12-09  6:09 Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor Jianyong Wu
                   ` (10 more replies)
  0 siblings, 11 replies; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

Currently, we offen use ntp (sync time with remote network clock)
to sync time in VM. But the precision of ntp is subject to network delay
so it's difficult to sync time in a high precision.

kvm virtual ptp clock (ptp_kvm) offers another way to sync time in VM,
as the remote clock locates in the host instead of remote network clock.
It targets to sync time between guest and host in virtualization
environment and in this way, we can keep the time of all the VMs running
in the same host in sync. In general, the delay of communication between
host and guest is quiet small, so ptp_kvm can offer time sync precision
up to in order of nanosecond. Please keep in mind that ptp_kvm just
limits itself to be a channel which transmit the remote clock from
host to guest and leaves the time sync jobs to an application, eg. chrony,
in usersapce in VM.

How ptp_kvm works:
After ptp_kvm initialized, there will be a new device node under
/dev called ptp%d. A guest userspace service, like chrony, can use this
device to get host walltime, sometimes also counter cycle, which depends
on the service it calls. Then this guest userspace service can use those
data to do the time sync for guest.
here is a rough sketch to show how kvm ptp clock works.

|----------------------------|              |--------------------------|
|       guest userspace      |              |          host            |
|ioctl -> /dev/ptp%d         |              |                          |
|       ^   |                |              |                          |
|----------------------------|              |                          |
|       |   | guest kernel   |              |                          |
|       |   V      (get host walltime/counter cycle)                   |
|      ptp_kvm -> hypercall - - - - - - - - - - ->hypercall service    |
|                         <- - - - - - - - - - - -                     |
|----------------------------|              |--------------------------|

1. time sync service in guest userspace call ptp device through /dev/ptp%d.
2. ptp_kvm module in guest receives this request then invoke hypercall to route
into host kernel to request host walltime/counter cycle.
3. ptp_kvm hypercall service in host response to the request and send data back.
4. ptp (not ptp_kvm) in guest copy the data to userspace.

This ptp_kvm implementation focuses itself to step 2 and 3 and step 2 works
in guest comparing step 3 works in host kernel.

change log:

from v15 to v16:
        (1) remove ARM_PTP_NONE_COUNTER suggested by Marc.
        (2) add more detail for ptp_kvm doc.
        (3) fix ci issues reported by test robot.

from v14 to v15:
        (1) enable ptp_kvm on arm32 guest, also ptp_kvm has been tested
on both arm64 and arm32 guest running on arm64 kvm host.
        (2) move arch-agnostic part of ptp_kvm.rst into timekeeping.rst.
        (3) rename KVM_CAP_ARM_PTP_KVM to KVM_CAP_PTP_KVM as it should be
arch agnostic.
        (4) add description for KVM_CAP_PTP_KVM in Documentation/virt/kvm/api.rst.
        (5) adjust dependency in Kconfig for ptp_kvm.
        (6) refine multi-arch process in driver/ptp/Makefile.
        (7) fix make pdfdocs htmldocs issue for ptp_kvm doc.
        (8) address other issues from comments in v14.
        (9) fold hypercall service of ptp_kvm as a function.
        (10) rebase to 5.10-rc3.

from v13 to v14
        (1) rebase code on 5.9-rc3.
        (2) add a document to introduce implementation of PTP_KVM on
arm64.
        (3) fix comments issue in hypercall.c.
        (4) export arm_smccc_1_1_get_conduit using EXPORT_SYMBOL_GPL.
        (5) fix make issue on x86 reported by kernel test robot.

from v12 to v13:
        (1) rebase code on 5.8-rc1.
        (2) this patch set base on 2 patches of 1/8 and 2/8 from Will Decon.
        (3) remove the change to ptp device code of extend getcrosststamp.
        (4) remove the mechanism of letting user choose the counter type in
ptp_kvm for arm64.
        (5) add virtual counter option in ptp_kvm service to let user choose
the specific counter explicitly.

from v11 to v12:
        (1) rebase code on 5.7-rc6 and rebase 2 patches from Will Decon
including 1/11 and 2/11. as these patches introduce discover mechanism of
vendor smccc service.
        (2) rebase ptp_kvm hypercall service from standard smccc to vendor
smccc and add ptp_kvm to vendor smccc service discover mechanism.
        (3) add detail of why we need ptp_kvm and how ptp_kvm works in cover
letter.

from v10 to v11:
        (1) rebase code on 5.7-rc2.
        (2) remove support for arm32, as kvm support for arm32 will be
removed [1]
        (3) add error report in ptp_kvm initialization.

from v9 to v10:
        (1) change code base to v5.5.
        (2) enable ptp_kvm both for arm32 and arm64.
        (3) let user choose which of virtual counter or physical counter
should return when using crosstimestamp mode of ptp_kvm for arm/arm64.
        (4) extend input argument for getcrosstimestamp API.

from v8 to v9:
        (1) move ptp_kvm.h to driver/ptp/
        (2) replace license declaration of ptp_kvm.h the same with other
header files in the same directory.

from v7 to v8:
        (1) separate adding clocksource id for arm_arch_counter as a
single patch.
        (2) update commit message for patch 4/8.
        (3) refine patch 7/8 and patch 8/8 to make them more independent.

from v5 to v6:
        (1) apply Mark's patch[4] to get SMCCC conduit.
        (2) add mechanism to recognize current clocksource by add
clocksouce_id value into struct clocksource instead of method in patch-v5.
        (3) rename kvm_arch_ptp_get_clock_fn into
kvm_arch_ptp_get_crosststamp.

from v4 to v5:
        (1) remove hvc delay compensasion as it should leave to userspace.
        (2) check current clocksource in hvc call service.
        (3) expose current clocksource by adding it to
system_time_snapshot.
        (4) add helper to check if clocksource is arm_arch_counter.
        (5) rename kvm_ptp.c to ptp_kvm_common.c

from v3 to v4:
        (1) fix clocksource of ptp_kvm to arch_sys_counter.
        (2) move kvm_arch_ptp_get_clock_fn into arm_arch_timer.c
        (3) subtract cntvoff before return cycles from host.
        (4) use ktime_get_snapshot instead of getnstimeofday and
get_current_counterval to return time and counter value.
        (5) split ktime and counter into two 32-bit block respectively
to avoid Y2038-safe issue.
        (6) set time compensation to device time as half of the delay of
hvc call.
        (7) add ARM_ARCH_TIMER as dependency of ptp_kvm for
arm64.

from v2 to v3:
        (1) fix some issues in commit log.
        (2) add some receivers in send list.

from v1 to v2:
        (1) move arch-specific code from arch/ to driver/ptp/
        (2) offer mechanism to inform userspace if ptp_kvm service is
available.
        (3) separate ptp_kvm code for arm64 into hypervisor part and
guest part.
        (4) add API to expose monotonic clock and counter value.
        (5) refine code: remove no necessary part and reconsitution.

[1] https://patchwork.kernel.org/cover/11373351/


Jianyong Wu (6):
  ptp: Reorganize ptp_kvm module to make it arch-independent.
  clocksource: Add clocksource id for arm arch counter
  arm64/kvm: Add hypercall service for kvm ptp.
  ptp: arm/arm64: Enable ptp_kvm for arm/arm64
  doc: add ptp_kvm introduction for arm64 support
  arm64: Add kvm capability check extension for ptp_kvm

Thomas Gleixner (1):
  time: Add mechanism to recognize clocksource in time_get_snapshot

Will Deacon (2):
  arm64: Probe for the presence of KVM hypervisor
  arm/arm64: KVM: Advertise KVM UID to guests via SMCCC

 Documentation/virt/kvm/api.rst              |  9 ++
 Documentation/virt/kvm/arm/index.rst        |  1 +
 Documentation/virt/kvm/arm/ptp_kvm.rst      | 31 +++++++
 Documentation/virt/kvm/timekeeping.rst      | 35 ++++++++
 arch/arm/kernel/setup.c                     |  5 ++
 arch/arm64/kernel/setup.c                   |  1 +
 arch/arm64/kvm/arm.c                        |  1 +
 arch/arm64/kvm/hypercalls.c                 | 86 ++++++++++++++++--
 drivers/clocksource/arm_arch_timer.c        | 31 +++++++
 drivers/firmware/smccc/smccc.c              | 37 ++++++++
 drivers/ptp/Kconfig                         |  2 +-
 drivers/ptp/Makefile                        |  2 +
 drivers/ptp/ptp_kvm_arm.c                   | 45 ++++++++++
 drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 84 +++++-------------
 drivers/ptp/ptp_kvm_x86.c                   | 96 +++++++++++++++++++++
 include/linux/arm-smccc.h                   | 59 +++++++++++++
 include/linux/clocksource.h                 |  6 ++
 include/linux/clocksource_ids.h             | 12 +++
 include/linux/ptp_kvm.h                     | 16 ++++
 include/linux/timekeeping.h                 | 12 +--
 include/uapi/linux/kvm.h                    |  1 +
 kernel/time/clocksource.c                   |  2 +
 kernel/time/timekeeping.c                   |  1 +
 23 files changed, 498 insertions(+), 77 deletions(-)
 create mode 100644 Documentation/virt/kvm/arm/ptp_kvm.rst
 create mode 100644 drivers/ptp/ptp_kvm_arm.c
 rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%)
 create mode 100644 drivers/ptp/ptp_kvm_x86.c
 create mode 100644 include/linux/clocksource_ids.h
 create mode 100644 include/linux/ptp_kvm.h

-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2021-02-02 13:18   ` Marc Zyngier
  2020-12-09  6:09 ` [PATCH v16 2/9] arm/arm64: KVM: Advertise KVM UID to guests via SMCCC Jianyong Wu
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

From: Will Deacon <will@kernel.org>

Although the SMCCC specification provides some limited functionality for
describing the presence of hypervisor and firmware services, this is
generally applicable only to functions designated as "Arm Architecture
Service Functions" and no portable discovery mechanism is provided for
standard hypervisor services, despite having a designated range of
function identifiers reserved by the specification.

In an attempt to avoid the need for additional firmware changes every
time a new function is added, introduce a UID to identify the service
provider as being compatible with KVM. Once this has been established,
additional services can be discovered via a feature bitmap.

Change from Jianyong Wu:
mv kvm_arm_hyp_service_available to common place to let both arm/arm64 touch it.
add kvm_init_hyp_services also under arm arch to let arm kvm guest use this service.

Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 arch/arm/kernel/setup.c        |  5 ++++
 arch/arm64/kernel/setup.c      |  1 +
 drivers/firmware/smccc/smccc.c | 37 +++++++++++++++++++++++++++++
 include/linux/arm-smccc.h      | 43 ++++++++++++++++++++++++++++++++++
 4 files changed, 86 insertions(+)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 1a5edf562e85..adcefa9c8fab 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1156,6 +1156,11 @@ void __init setup_arch(char **cmdline_p)
 
 	arm_dt_init_cpu_maps();
 	psci_dt_init();
+
+#ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY
+	kvm_init_hyp_services();
+#endif
+
 #ifdef CONFIG_SMP
 	if (is_smp()) {
 		if (!mdesc->smp_init || !mdesc->smp_init()) {
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index a950d5bc1ba5..97037b15c6ea 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -353,6 +353,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
 	else
 		psci_acpi_init();
 
+	kvm_init_hyp_services();
 	init_bootcpu_ops();
 	smp_init_cpus();
 	smp_build_mpidr_hash();
diff --git a/drivers/firmware/smccc/smccc.c b/drivers/firmware/smccc/smccc.c
index 00c88b809c0c..e153c71ece99 100644
--- a/drivers/firmware/smccc/smccc.c
+++ b/drivers/firmware/smccc/smccc.c
@@ -7,10 +7,47 @@
 
 #include <linux/init.h>
 #include <linux/arm-smccc.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
 
 static u32 smccc_version = ARM_SMCCC_VERSION_1_0;
 static enum arm_smccc_conduit smccc_conduit = SMCCC_CONDUIT_NONE;
 
+DECLARE_BITMAP(__kvm_arm_hyp_services, ARM_SMCCC_KVM_NUM_FUNCS) = { };
+EXPORT_SYMBOL_GPL(__kvm_arm_hyp_services);
+
+void __init kvm_init_hyp_services(void)
+{
+	int i;
+	struct arm_smccc_res res;
+
+	if (arm_smccc_get_version() == ARM_SMCCC_VERSION_1_0)
+		return;
+
+	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+	if (res.a0 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0 ||
+	    res.a1 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1 ||
+	    res.a2 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2 ||
+	    res.a3 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3)
+		return;
+
+	memset(&res, 0, sizeof(res));
+	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID, &res);
+	for (i = 0; i < 32; ++i) {
+		if (res.a0 & (i))
+			set_bit(i + (32 * 0), __kvm_arm_hyp_services);
+		if (res.a1 & (i))
+			set_bit(i + (32 * 1), __kvm_arm_hyp_services);
+		if (res.a2 & (i))
+			set_bit(i + (32 * 2), __kvm_arm_hyp_services);
+		if (res.a3 & (i))
+			set_bit(i + (32 * 3), __kvm_arm_hyp_services);
+	}
+
+	pr_info("KVM hypervisor services detected (0x%08lx 0x%08lx 0x%08lx 0x%08lx)\n",
+		 res.a3, res.a2, res.a1, res.a0);
+}
+
 void __init arm_smccc_version_init(u32 version, enum arm_smccc_conduit conduit)
 {
 	smccc_version = version;
diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
index f860645f6512..d75408141137 100644
--- a/include/linux/arm-smccc.h
+++ b/include/linux/arm-smccc.h
@@ -55,6 +55,8 @@
 #define ARM_SMCCC_OWNER_TRUSTED_OS	50
 #define ARM_SMCCC_OWNER_TRUSTED_OS_END	63
 
+#define ARM_SMCCC_FUNC_QUERY_CALL_UID  0xff01
+
 #define ARM_SMCCC_QUIRK_NONE		0
 #define ARM_SMCCC_QUIRK_QCOM_A6		1 /* Save/restore register a6 */
 
@@ -87,6 +89,29 @@
 			   ARM_SMCCC_SMC_32,				\
 			   0, 0x7fff)
 
+#define ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID				\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
+			   ARM_SMCCC_SMC_32,				\
+			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
+			   ARM_SMCCC_FUNC_QUERY_CALL_UID)
+
+/* KVM UID value: 28b46fb6-2ec5-11e9-a9ca-4b564d003a74 */
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0	0xb66fb428U
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1	0xe911c52eU
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2	0x564bcaa9U
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3	0x743a004dU
+
+/* KVM "vendor specific" services */
+#define ARM_SMCCC_KVM_FUNC_FEATURES		0
+#define ARM_SMCCC_KVM_FUNC_FEATURES_2		127
+#define ARM_SMCCC_KVM_NUM_FUNCS			128
+
+#define ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID			\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
+			   ARM_SMCCC_SMC_32,				\
+			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
+			   ARM_SMCCC_KVM_FUNC_FEATURES)
+
 #define SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED	1
 
 /* Paravirtualised time calls (defined by ARM DEN0057A) */
@@ -391,5 +416,23 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
 		method;							\
 	})
 
+void __init kvm_init_hyp_services(void);
+
+/*
+ * This helper will be called in guest. We put it here then both arm and arm64
+ * guest can touch it.
+ */
+#include <linux/kernel.h>
+#include <linux/err.h>
+static inline bool kvm_arm_hyp_service_available(u32 func_id)
+{
+	extern DECLARE_BITMAP(__kvm_arm_hyp_services, ARM_SMCCC_KVM_NUM_FUNCS);
+
+	if (func_id >= ARM_SMCCC_KVM_NUM_FUNCS)
+		return -EINVAL;
+
+	return test_bit(func_id, __kvm_arm_hyp_services);
+}
+
 #endif /*__ASSEMBLY__*/
 #endif /*__LINUX_ARM_SMCCC_H*/
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 2/9] arm/arm64: KVM: Advertise KVM UID to guests via SMCCC
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 3/9] ptp: Reorganize ptp_kvm module to make it arch-independent Jianyong Wu
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

From: Will Deacon <will@kernel.org>

We can advertise ourselves to guests as KVM and provide a basic features
bitmap for discoverability of future hypervisor services.

Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 arch/arm64/kvm/hypercalls.c | 27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c
index 25ea4ecb6449..b9d8607083eb 100644
--- a/arch/arm64/kvm/hypercalls.c
+++ b/arch/arm64/kvm/hypercalls.c
@@ -12,13 +12,13 @@
 int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 {
 	u32 func_id = smccc_get_function(vcpu);
-	long val = SMCCC_RET_NOT_SUPPORTED;
+	u64 val[4] = {SMCCC_RET_NOT_SUPPORTED};
 	u32 feature;
 	gpa_t gpa;
 
 	switch (func_id) {
 	case ARM_SMCCC_VERSION_FUNC_ID:
-		val = ARM_SMCCC_VERSION_1_1;
+		val[0] = ARM_SMCCC_VERSION_1_1;
 		break;
 	case ARM_SMCCC_ARCH_FEATURES_FUNC_ID:
 		feature = smccc_get_arg1(vcpu);
@@ -28,10 +28,10 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 			case SPECTRE_VULNERABLE:
 				break;
 			case SPECTRE_MITIGATED:
-				val = SMCCC_RET_SUCCESS;
+				val[0] = SMCCC_RET_SUCCESS;
 				break;
 			case SPECTRE_UNAFFECTED:
-				val = SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED;
+				val[0] = SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED;
 				break;
 			}
 			break;
@@ -54,27 +54,36 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 					break;
 				fallthrough;
 			case SPECTRE_UNAFFECTED:
-				val = SMCCC_RET_NOT_REQUIRED;
+				val[0] = SMCCC_RET_NOT_REQUIRED;
 				break;
 			}
 			break;
 		case ARM_SMCCC_HV_PV_TIME_FEATURES:
-			val = SMCCC_RET_SUCCESS;
+			val[0] = SMCCC_RET_SUCCESS;
 			break;
 		}
 		break;
 	case ARM_SMCCC_HV_PV_TIME_FEATURES:
-		val = kvm_hypercall_pv_features(vcpu);
+		val[0] = kvm_hypercall_pv_features(vcpu);
 		break;
 	case ARM_SMCCC_HV_PV_TIME_ST:
 		gpa = kvm_init_stolen_time(vcpu);
 		if (gpa != GPA_INVALID)
-			val = gpa;
+			val[0] = gpa;
+		break;
+	case ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID:
+		val[0] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0;
+		val[1] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1;
+		val[2] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2;
+		val[3] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3;
+		break;
+	case ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID:
+		val[0] = BIT(ARM_SMCCC_KVM_FUNC_FEATURES);
 		break;
 	default:
 		return kvm_psci_call(vcpu);
 	}
 
-	smccc_set_retval(vcpu, val, 0, 0, 0);
+	smccc_set_retval(vcpu, val[0], val[1], val[2], val[3]);
 	return 1;
 }
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 3/9] ptp: Reorganize ptp_kvm module to make it arch-independent.
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 2/9] arm/arm64: KVM: Advertise KVM UID to guests via SMCCC Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2021-02-02 13:23   ` Marc Zyngier
  2020-12-09  6:09 ` [PATCH v16 4/9] time: Add mechanism to recognize clocksource in time_get_snapshot Jianyong Wu
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

Currently, ptp_kvm modules implementation is only for x86 which includes
large part of arch-specific code.  This patch moves all of this code
into a new arch related file in the same directory.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 drivers/ptp/Makefile                        |  1 +
 drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 84 +++++-------------
 drivers/ptp/ptp_kvm_x86.c                   | 96 +++++++++++++++++++++
 include/linux/ptp_kvm.h                     | 16 ++++
 4 files changed, 135 insertions(+), 62 deletions(-)
 rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%)
 create mode 100644 drivers/ptp/ptp_kvm_x86.c
 create mode 100644 include/linux/ptp_kvm.h

diff --git a/drivers/ptp/Makefile b/drivers/ptp/Makefile
index 7aff75f745dc..699a4e4d19c2 100644
--- a/drivers/ptp/Makefile
+++ b/drivers/ptp/Makefile
@@ -4,6 +4,7 @@
 #
 
 ptp-y					:= ptp_clock.o ptp_chardev.o ptp_sysfs.o
+ptp_kvm-$(CONFIG_X86)			:= ptp_kvm_x86.o ptp_kvm_common.o
 obj-$(CONFIG_PTP_1588_CLOCK)		+= ptp.o
 obj-$(CONFIG_PTP_1588_CLOCK_DTE)	+= ptp_dte.o
 obj-$(CONFIG_PTP_1588_CLOCK_INES)	+= ptp_ines.o
diff --git a/drivers/ptp/ptp_kvm.c b/drivers/ptp/ptp_kvm_common.c
similarity index 60%
rename from drivers/ptp/ptp_kvm.c
rename to drivers/ptp/ptp_kvm_common.c
index 658d33fc3195..721ddcede5e1 100644
--- a/drivers/ptp/ptp_kvm.c
+++ b/drivers/ptp/ptp_kvm_common.c
@@ -8,11 +8,11 @@
 #include <linux/err.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
+#include <linux/slab.h>
 #include <linux/module.h>
+#include <linux/ptp_kvm.h>
 #include <uapi/linux/kvm_para.h>
 #include <asm/kvm_para.h>
-#include <asm/pvclock.h>
-#include <asm/kvmclock.h>
 #include <uapi/asm/kvm_para.h>
 
 #include <linux/ptp_clock_kernel.h>
@@ -24,56 +24,29 @@ struct kvm_ptp_clock {
 
 static DEFINE_SPINLOCK(kvm_ptp_lock);
 
-static struct pvclock_vsyscall_time_info *hv_clock;
-
-static struct kvm_clock_pairing clock_pair;
-static phys_addr_t clock_pair_gpa;
-
 static int ptp_kvm_get_time_fn(ktime_t *device_time,
 			       struct system_counterval_t *system_counter,
 			       void *ctx)
 {
-	unsigned long ret;
+	long ret;
+	u64 cycle;
 	struct timespec64 tspec;
-	unsigned version;
-	int cpu;
-	struct pvclock_vcpu_time_info *src;
+	struct clocksource *cs;
 
 	spin_lock(&kvm_ptp_lock);
 
 	preempt_disable_notrace();
-	cpu = smp_processor_id();
-	src = &hv_clock[cpu].pvti;
-
-	do {
-		/*
-		 * We are using a TSC value read in the hosts
-		 * kvm_hc_clock_pairing handling.
-		 * So any changes to tsc_to_system_mul
-		 * and tsc_shift or any other pvclock
-		 * data invalidate that measurement.
-		 */
-		version = pvclock_read_begin(src);
-
-		ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING,
-				     clock_pair_gpa,
-				     KVM_CLOCK_PAIRING_WALLCLOCK);
-		if (ret != 0) {
-			pr_err_ratelimited("clock pairing hypercall ret %lu\n", ret);
-			spin_unlock(&kvm_ptp_lock);
-			preempt_enable_notrace();
-			return -EOPNOTSUPP;
-		}
-
-		tspec.tv_sec = clock_pair.sec;
-		tspec.tv_nsec = clock_pair.nsec;
-		ret = __pvclock_read_cycles(src, clock_pair.tsc);
-	} while (pvclock_read_retry(src, version));
+	ret = kvm_arch_ptp_get_crosststamp(&cycle, &tspec, &cs);
+	if (ret) {
+		spin_unlock(&kvm_ptp_lock);
+		preempt_enable_notrace();
+		return ret;
+	}
 
 	preempt_enable_notrace();
 
-	system_counter->cycles = ret;
-	system_counter->cs = &kvm_clock;
+	system_counter->cycles = cycle;
+	system_counter->cs = cs;
 
 	*device_time = timespec64_to_ktime(tspec);
 
@@ -111,22 +84,17 @@ static int ptp_kvm_settime(struct ptp_clock_info *ptp,
 
 static int ptp_kvm_gettime(struct ptp_clock_info *ptp, struct timespec64 *ts)
 {
-	unsigned long ret;
+	long ret;
 	struct timespec64 tspec;
 
 	spin_lock(&kvm_ptp_lock);
 
-	ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING,
-			     clock_pair_gpa,
-			     KVM_CLOCK_PAIRING_WALLCLOCK);
-	if (ret != 0) {
-		pr_err_ratelimited("clock offset hypercall ret %lu\n", ret);
+	ret = kvm_arch_ptp_get_clock(&tspec);
+	if (ret) {
 		spin_unlock(&kvm_ptp_lock);
-		return -EOPNOTSUPP;
+		return ret;
 	}
 
-	tspec.tv_sec = clock_pair.sec;
-	tspec.tv_nsec = clock_pair.nsec;
 	spin_unlock(&kvm_ptp_lock);
 
 	memcpy(ts, &tspec, sizeof(struct timespec64));
@@ -168,19 +136,11 @@ static int __init ptp_kvm_init(void)
 {
 	long ret;
 
-	if (!kvm_para_available())
-		return -ENODEV;
-
-	clock_pair_gpa = slow_virt_to_phys(&clock_pair);
-	hv_clock = pvclock_get_pvti_cpu0_va();
-
-	if (!hv_clock)
-		return -ENODEV;
-
-	ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, clock_pair_gpa,
-			KVM_CLOCK_PAIRING_WALLCLOCK);
-	if (ret == -KVM_ENOSYS || ret == -KVM_EOPNOTSUPP)
-		return -ENODEV;
+	ret = kvm_arch_ptp_init();
+	if (ret) {
+		pr_err("fail to initialize ptp_kvm");
+		return ret;
+	}
 
 	kvm_ptp_clock.caps = ptp_kvm_caps;
 
diff --git a/drivers/ptp/ptp_kvm_x86.c b/drivers/ptp/ptp_kvm_x86.c
new file mode 100644
index 000000000000..e011d691e549
--- /dev/null
+++ b/drivers/ptp/ptp_kvm_x86.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Virtual PTP 1588 clock for use with KVM guests
+ *
+ * Copyright (C) 2017 Red Hat Inc.
+ */
+
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <asm/pvclock.h>
+#include <asm/kvmclock.h>
+#include <linux/module.h>
+#include <uapi/asm/kvm_para.h>
+#include <uapi/linux/kvm_para.h>
+#include <linux/ptp_clock_kernel.h>
+#include <linux/ptp_kvm.h>
+
+phys_addr_t clock_pair_gpa;
+struct kvm_clock_pairing clock_pair;
+struct pvclock_vsyscall_time_info *hv_clock;
+
+int kvm_arch_ptp_init(void)
+{
+	long ret;
+
+	if (!kvm_para_available())
+		return -ENODEV;
+
+	clock_pair_gpa = slow_virt_to_phys(&clock_pair);
+	hv_clock = pvclock_get_pvti_cpu0_va();
+	if (!hv_clock)
+		return -ENODEV;
+
+	ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, clock_pair_gpa,
+			     KVM_CLOCK_PAIRING_WALLCLOCK);
+	if (ret == -KVM_ENOSYS || ret == -KVM_EOPNOTSUPP)
+		return -ENODEV;
+
+	return 0;
+}
+
+int kvm_arch_ptp_get_clock(struct timespec64 *ts)
+{
+	unsigned long ret;
+
+	ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING,
+			     clock_pair_gpa,
+			     KVM_CLOCK_PAIRING_WALLCLOCK);
+	if (ret != 0) {
+		pr_err_ratelimited("clock offset hypercall ret %lu\n", ret);
+		return -EOPNOTSUPP;
+	}
+
+	ts->tv_sec = clock_pair.sec;
+	ts->tv_nsec = clock_pair.nsec;
+
+	return 0;
+}
+
+int kvm_arch_ptp_get_crosststamp(u64 *cycle, struct timespec64 *tspec,
+			      struct clocksource **cs)
+{
+	unsigned long ret;
+	unsigned int version;
+	int cpu;
+	struct pvclock_vcpu_time_info *src;
+
+	cpu = smp_processor_id();
+	src = &hv_clock[cpu].pvti;
+
+	do {
+		/*
+		 * We are using a TSC value read in the hosts
+		 * kvm_hc_clock_pairing handling.
+		 * So any changes to tsc_to_system_mul
+		 * and tsc_shift or any other pvclock
+		 * data invalidate that measurement.
+		 */
+		version = pvclock_read_begin(src);
+
+		ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING,
+				     clock_pair_gpa,
+				     KVM_CLOCK_PAIRING_WALLCLOCK);
+		if (ret != 0) {
+			pr_err_ratelimited("clock pairing hypercall ret %lu\n", ret);
+			return -EOPNOTSUPP;
+		}
+		tspec->tv_sec = clock_pair.sec;
+		tspec->tv_nsec = clock_pair.nsec;
+		*cycle = __pvclock_read_cycles(src, clock_pair.tsc);
+	} while (pvclock_read_retry(src, version));
+
+	*cs = &kvm_clock;
+
+	return 0;
+}
diff --git a/include/linux/ptp_kvm.h b/include/linux/ptp_kvm.h
new file mode 100644
index 000000000000..6f104b1967bb
--- /dev/null
+++ b/include/linux/ptp_kvm.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Virtual PTP 1588 clock for use with KVM guests
+ *
+ * Copyright (C) 2017 Red Hat Inc.
+ */
+
+#ifndef _PTP_KVM_H_
+#define _PTP_KVM_H_
+
+int kvm_arch_ptp_init(void);
+int kvm_arch_ptp_get_clock(struct timespec64 *ts);
+int kvm_arch_ptp_get_crosststamp(u64 *cycle,
+		struct timespec64 *tspec, struct clocksource **cs);
+
+#endif /* _PTP_KVM_H_ */
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 4/9] time: Add mechanism to recognize clocksource in time_get_snapshot
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (2 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 3/9] ptp: Reorganize ptp_kvm module to make it arch-independent Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 5/9] clocksource: Add clocksource id for arm arch counter Jianyong Wu
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

From: Thomas Gleixner <tglx@linutronix.de>

System time snapshots are not conveying information about the current
clocksource which was used, but callers like the PTP KVM guest
implementation have the requirement to evaluate the clocksource type to
select the appropriate mechanism.

Introduce a clocksource id field in struct clocksource which is by default
set to CSID_GENERIC (0). Clocksource implementations can set that field to
a value which allows to identify the clocksource.

Store the clocksource id of the current clocksource in the
system_time_snapshot so callers can evaluate which clocksource was used to
take the snapshot and act accordingly.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 include/linux/clocksource.h     |  6 ++++++
 include/linux/clocksource_ids.h | 11 +++++++++++
 include/linux/timekeeping.h     | 12 +++++++-----
 kernel/time/clocksource.c       |  2 ++
 kernel/time/timekeeping.c       |  1 +
 5 files changed, 27 insertions(+), 5 deletions(-)
 create mode 100644 include/linux/clocksource_ids.h

diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 86d143db6523..1290d0dce840 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -17,6 +17,7 @@
 #include <linux/timer.h>
 #include <linux/init.h>
 #include <linux/of.h>
+#include <linux/clocksource_ids.h>
 #include <asm/div64.h>
 #include <asm/io.h>
 
@@ -62,6 +63,10 @@ struct module;
  *			400-499: Perfect
  *				The ideal clocksource. A must-use where
  *				available.
+ * @id:			Defaults to CSID_GENERIC. The id value is captured
+ *			in certain snapshot functions to allow callers to
+ *			validate the clocksource from which the snapshot was
+ *			taken.
  * @flags:		Flags describing special properties
  * @enable:		Optional function to enable the clocksource
  * @disable:		Optional function to disable the clocksource
@@ -100,6 +105,7 @@ struct clocksource {
 	const char		*name;
 	struct list_head	list;
 	int			rating;
+	enum clocksource_ids	id;
 	enum vdso_clock_mode	vdso_clock_mode;
 	unsigned long		flags;
 
diff --git a/include/linux/clocksource_ids.h b/include/linux/clocksource_ids.h
new file mode 100644
index 000000000000..4d8e19e05328
--- /dev/null
+++ b/include/linux/clocksource_ids.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_CLOCKSOURCE_IDS_H
+#define _LINUX_CLOCKSOURCE_IDS_H
+
+/* Enum to give clocksources a unique identifier */
+enum clocksource_ids {
+	CSID_GENERIC		= 0,
+	CSID_MAX,
+};
+
+#endif
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index d47009611109..688ec2e1a3bf 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -3,6 +3,7 @@
 #define _LINUX_TIMEKEEPING_H
 
 #include <linux/errno.h>
+#include <linux/clocksource_ids.h>
 
 /* Included from linux/ktime.h */
 
@@ -243,11 +244,12 @@ struct ktime_timestamps {
  * @cs_was_changed_seq:	The sequence number of clocksource change events
  */
 struct system_time_snapshot {
-	u64		cycles;
-	ktime_t		real;
-	ktime_t		raw;
-	unsigned int	clock_was_set_seq;
-	u8		cs_was_changed_seq;
+	u64			cycles;
+	ktime_t			real;
+	ktime_t			raw;
+	enum clocksource_ids	cs_id;
+	unsigned int		clock_was_set_seq;
+	u8			cs_was_changed_seq;
 };
 
 /**
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index cce484a2cc7c..4fe1df894ee5 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -920,6 +920,8 @@ int __clocksource_register_scale(struct clocksource *cs, u32 scale, u32 freq)
 
 	clocksource_arch_init(cs);
 
+	if (WARN_ON_ONCE((unsigned int)cs->id >= CSID_MAX))
+		cs->id = CSID_GENERIC;
 	if (cs->vdso_clock_mode < 0 ||
 	    cs->vdso_clock_mode >= VDSO_CLOCKMODE_MAX) {
 		pr_warn("clocksource %s registered with invalid VDSO mode %d. Disabling VDSO support.\n",
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index a45cedda93a7..50f08632165c 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1049,6 +1049,7 @@ void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot)
 	do {
 		seq = read_seqcount_begin(&tk_core.seq);
 		now = tk_clock_read(&tk->tkr_mono);
+		systime_snapshot->cs_id = tk->tkr_mono.clock->id;
 		systime_snapshot->cs_was_changed_seq = tk->cs_was_changed_seq;
 		systime_snapshot->clock_was_set_seq = tk->clock_was_set_seq;
 		base_real = ktime_add(tk->tkr_mono.base,
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 5/9] clocksource: Add clocksource id for arm arch counter
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (3 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 4/9] time: Add mechanism to recognize clocksource in time_get_snapshot Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2020-12-09  6:09 ` [PATCH v16 6/9] arm64/kvm: Add hypercall service for kvm ptp Jianyong Wu
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

Add clocksource id for arm arch counter to let it be identified easily and
elegantly in ptp_kvm implementation for arm.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 2 ++
 include/linux/clocksource_ids.h      | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 6c3e84180146..d55acffb0b90 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -16,6 +16,7 @@
 #include <linux/cpu_pm.h>
 #include <linux/clockchips.h>
 #include <linux/clocksource.h>
+#include <linux/clocksource_ids.h>
 #include <linux/interrupt.h>
 #include <linux/of_irq.h>
 #include <linux/of_address.h>
@@ -191,6 +192,7 @@ static u64 arch_counter_read_cc(const struct cyclecounter *cc)
 
 static struct clocksource clocksource_counter = {
 	.name	= "arch_sys_counter",
+	.id	= CSID_ARM_ARCH_COUNTER,
 	.rating	= 400,
 	.read	= arch_counter_read,
 	.mask	= CLOCKSOURCE_MASK(56),
diff --git a/include/linux/clocksource_ids.h b/include/linux/clocksource_ids.h
index 4d8e19e05328..16775d7d8f8d 100644
--- a/include/linux/clocksource_ids.h
+++ b/include/linux/clocksource_ids.h
@@ -5,6 +5,7 @@
 /* Enum to give clocksources a unique identifier */
 enum clocksource_ids {
 	CSID_GENERIC		= 0,
+	CSID_ARM_ARCH_COUNTER,
 	CSID_MAX,
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 6/9] arm64/kvm: Add hypercall service for kvm ptp.
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (4 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 5/9] clocksource: Add clocksource id for arm arch counter Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2021-02-02 13:32   ` Marc Zyngier
  2020-12-09  6:09 ` [PATCH v16 7/9] ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (4 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

ptp_kvm will get this service through SMCC call.
The service offers wall time and cycle count of host to guest.
The caller must specify whether they want the host cycle count
or the difference between host cycle count and cntvoff.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 arch/arm64/kvm/hypercalls.c | 59 +++++++++++++++++++++++++++++++++++++
 include/linux/arm-smccc.h   | 16 ++++++++++
 2 files changed, 75 insertions(+)

diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c
index b9d8607083eb..9a4834502388 100644
--- a/arch/arm64/kvm/hypercalls.c
+++ b/arch/arm64/kvm/hypercalls.c
@@ -9,6 +9,49 @@
 #include <kvm/arm_hypercalls.h>
 #include <kvm/arm_psci.h>
 
+static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val)
+{
+	struct system_time_snapshot systime_snapshot;
+	u64 cycles = ~0UL;
+	u32 feature;
+
+	/*
+	 * system time and counter value must captured in the same
+	 * time to keep consistency and precision.
+	 */
+	ktime_get_snapshot(&systime_snapshot);
+
+	// binding ptp_kvm clocksource to arm_arch_counter
+	if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
+		return;
+
+	val[0] = upper_32_bits(systime_snapshot.real);
+	val[1] = lower_32_bits(systime_snapshot.real);
+
+	/*
+	 * which of virtual counter or physical counter being
+	 * asked for is decided by the r1 value of SMCCC
+	 * call. If no invalid r1 value offered, default cycle
+	 * value(-1) will be returned.
+	 * Note: keep in mind that feature is u32 and smccc_get_arg1
+	 * will return u64, so need auto cast here.
+	 */
+	feature = smccc_get_arg1(vcpu);
+	switch (feature) {
+	case ARM_PTP_VIRT_COUNTER:
+		cycles = systime_snapshot.cycles - vcpu_read_sys_reg(vcpu, CNTVOFF_EL2);
+		break;
+	case ARM_PTP_PHY_COUNTER:
+		cycles = systime_snapshot.cycles;
+		break;
+	default:
+		val[0] = SMCCC_RET_NOT_SUPPORTED;
+		break;
+	}
+	val[2] = upper_32_bits(cycles);
+	val[3] = lower_32_bits(cycles);
+}
+
 int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 {
 	u32 func_id = smccc_get_function(vcpu);
@@ -79,6 +122,22 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 		break;
 	case ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID:
 		val[0] = BIT(ARM_SMCCC_KVM_FUNC_FEATURES);
+		val[0] |= BIT(ARM_SMCCC_KVM_FUNC_PTP);
+		break;
+	/*
+	 * This serves virtual kvm_ptp.
+	 * Four values will be passed back.
+	 * reg0 stores high 32-bits of host ktime;
+	 * reg1 stores low 32-bits of host ktime;
+	 * For ARM_PTP_VIRT_COUNTER:
+	 * reg2 stores high 32-bits of difference of host cycles and cntvoff;
+	 * reg3 stores low 32-bits of difference of host cycles and cntvoff.
+	 * For ARM_PTP_PHY_COUNTER:
+	 * reg2 stores the high 32-bits of host cycles;
+	 * reg3 stores the low 32-bits of host cycles.
+	 */
+	case ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID:
+		kvm_ptp_get_time(vcpu, val);
 		break;
 	default:
 		return kvm_psci_call(vcpu);
diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
index d75408141137..7924069f8f0a 100644
--- a/include/linux/arm-smccc.h
+++ b/include/linux/arm-smccc.h
@@ -103,6 +103,7 @@
 
 /* KVM "vendor specific" services */
 #define ARM_SMCCC_KVM_FUNC_FEATURES		0
+#define ARM_SMCCC_KVM_FUNC_PTP			1
 #define ARM_SMCCC_KVM_FUNC_FEATURES_2		127
 #define ARM_SMCCC_KVM_NUM_FUNCS			128
 
@@ -114,6 +115,21 @@
 
 #define SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED	1
 
+/*
+ * ptp_kvm is a feature used for time sync between vm and host.
+ * ptp_kvm module in guest kernel will get service from host using
+ * this hypercall ID.
+ */
+#define ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID				\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
+			   ARM_SMCCC_SMC_32,				\
+			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
+			   ARM_SMCCC_KVM_FUNC_PTP)
+
+/* ptp_kvm counter type ID */
+#define ARM_PTP_VIRT_COUNTER			0
+#define ARM_PTP_PHY_COUNTER			1
+
 /* Paravirtualised time calls (defined by ARM DEN0057A) */
 #define ARM_SMCCC_HV_PV_TIME_FEATURES				\
 	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,			\
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 7/9] ptp: arm/arm64: Enable ptp_kvm for arm/arm64
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (5 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 6/9] arm64/kvm: Add hypercall service for kvm ptp Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2021-02-02 13:37   ` Marc Zyngier
  2020-12-09  6:09 ` [PATCH v16 8/9] doc: add ptp_kvm introduction for arm64 support Jianyong Wu
                   ` (3 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

Currently, there is no mechanism to keep time sync between guest and host
in arm/arm64 virtualization environment. Time in guest will drift compared
with host after boot up as they may both use third party time sources
to correct their time respectively. The time deviation will be in order
of milliseconds. But in some scenarios,like in cloud environment, we ask
for higher time precision.

kvm ptp clock, which chooses the host clock source as a reference
clock to sync time between guest and host, has been adopted by x86
which takes the time sync order from milliseconds to nanoseconds.

This patch enables kvm ptp clock for arm/arm64 and improves clock sync precision
significantly.

Test result comparisons between with kvm ptp clock and without it in arm/arm64
are as follows. This test derived from the result of command 'chronyc
sources'. we should take more care of the last sample column which shows
the offset between the local clock and the source at the last measurement.

no kvm ptp in guest:
MS Name/IP address   Stratum Poll Reach LastRx Last sample
========================================================================
^* dns1.synet.edu.cn      2   6   377    13  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    21  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    29  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    37  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    45  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    53  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    61  +1040us[+1581us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377     4   -130us[ +796us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    12   -130us[ +796us] +/-   21ms
^* dns1.synet.edu.cn      2   6   377    20   -130us[ +796us] +/-   21ms

in host:
MS Name/IP address   Stratum Poll Reach LastRx Last sample
========================================================================
^* 120.25.115.20          2   7   377    72   -470us[ -603us] +/-   18ms
^* 120.25.115.20          2   7   377    92   -470us[ -603us] +/-   18ms
^* 120.25.115.20          2   7   377   112   -470us[ -603us] +/-   18ms
^* 120.25.115.20          2   7   377     2   +872ns[-6808ns] +/-   17ms
^* 120.25.115.20          2   7   377    22   +872ns[-6808ns] +/-   17ms
^* 120.25.115.20          2   7   377    43   +872ns[-6808ns] +/-   17ms
^* 120.25.115.20          2   7   377    63   +872ns[-6808ns] +/-   17ms
^* 120.25.115.20          2   7   377    83   +872ns[-6808ns] +/-   17ms
^* 120.25.115.20          2   7   377   103   +872ns[-6808ns] +/-   17ms
^* 120.25.115.20          2   7   377   123   +872ns[-6808ns] +/-   17ms

The dns1.synet.edu.cn is the network reference clock for guest and
120.25.115.20 is the network reference clock for host. we can't get the
clock error between guest and host directly, but a roughly estimated value
will be in order of hundreds of us to ms.

with kvm ptp in guest:
chrony has been disabled in host to remove the disturb by network clock.

MS Name/IP address         Stratum Poll Reach LastRx Last sample
========================================================================
* PHC0                    0   3   377     8     -7ns[   +1ns] +/-    3ns
* PHC0                    0   3   377     8     +1ns[  +16ns] +/-    3ns
* PHC0                    0   3   377     6     -4ns[   -0ns] +/-    6ns
* PHC0                    0   3   377     6     -8ns[  -12ns] +/-    5ns
* PHC0                    0   3   377     5     +2ns[   +4ns] +/-    4ns
* PHC0                    0   3   377    13     +2ns[   +4ns] +/-    4ns
* PHC0                    0   3   377    12     -4ns[   -6ns] +/-    4ns
* PHC0                    0   3   377    11     -8ns[  -11ns] +/-    6ns
* PHC0                    0   3   377    10    -14ns[  -20ns] +/-    4ns
* PHC0                    0   3   377     8     +4ns[   +5ns] +/-    4ns

The PHC0 is the ptp clock which choose the host clock as its source
clock. So we can see that the clock difference between host and guest
is in order of ns.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 29 ++++++++++++++++++
 drivers/ptp/Kconfig                  |  2 +-
 drivers/ptp/Makefile                 |  1 +
 drivers/ptp/ptp_kvm_arm.c            | 45 ++++++++++++++++++++++++++++
 4 files changed, 76 insertions(+), 1 deletion(-)
 create mode 100644 drivers/ptp/ptp_kvm_arm.c

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index d55acffb0b90..16cd0a663587 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -25,6 +25,8 @@
 #include <linux/sched/clock.h>
 #include <linux/sched_clock.h>
 #include <linux/acpi.h>
+#include <linux/arm-smccc.h>
+#include <linux/ptp_kvm.h>
 
 #include <asm/arch_timer.h>
 #include <asm/virt.h>
@@ -1650,3 +1652,30 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
 }
 TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init);
 #endif
+
+int kvm_arch_ptp_get_crosststamp(u64 *cycle, struct timespec64 *ts,
+			      struct clocksource **cs)
+{
+	struct arm_smccc_res hvc_res;
+	ktime_t ktime;
+	u32 ptp_counter;
+
+	if (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)
+		ptp_counter = ARM_PTP_VIRT_COUNTER;
+	else
+		ptp_counter = ARM_PTP_PHY_COUNTER;
+
+	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
+			     ptp_counter, &hvc_res);
+
+	if ((int)(hvc_res.a0) < 0)
+		return -EOPNOTSUPP;
+
+	ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
+	*ts = ktime_to_timespec64(ktime);
+	*cycle = (u64)hvc_res.a2 << 32 | hvc_res.a3;
+	*cs = &clocksource_counter;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_arch_ptp_get_crosststamp);
diff --git a/drivers/ptp/Kconfig b/drivers/ptp/Kconfig
index 942f72d8151d..677c7f696b70 100644
--- a/drivers/ptp/Kconfig
+++ b/drivers/ptp/Kconfig
@@ -106,7 +106,7 @@ config PTP_1588_CLOCK_PCH
 config PTP_1588_CLOCK_KVM
 	tristate "KVM virtual PTP clock"
 	depends on PTP_1588_CLOCK
-	depends on KVM_GUEST && X86
+	depends on KVM_GUEST && X86 || (HAVE_ARM_SMCCC_DISCOVERY && ARM_ARCH_TIMER)
 	default y
 	help
 	  This driver adds support for using kvm infrastructure as a PTP
diff --git a/drivers/ptp/Makefile b/drivers/ptp/Makefile
index 699a4e4d19c2..9fa5ede44b2b 100644
--- a/drivers/ptp/Makefile
+++ b/drivers/ptp/Makefile
@@ -5,6 +5,7 @@
 
 ptp-y					:= ptp_clock.o ptp_chardev.o ptp_sysfs.o
 ptp_kvm-$(CONFIG_X86)			:= ptp_kvm_x86.o ptp_kvm_common.o
+ptp_kvm-$(CONFIG_HAVE_ARM_SMCCC)	:= ptp_kvm_arm.o ptp_kvm_common.o
 obj-$(CONFIG_PTP_1588_CLOCK)		+= ptp.o
 obj-$(CONFIG_PTP_1588_CLOCK_DTE)	+= ptp_dte.o
 obj-$(CONFIG_PTP_1588_CLOCK_INES)	+= ptp_ines.o
diff --git a/drivers/ptp/ptp_kvm_arm.c b/drivers/ptp/ptp_kvm_arm.c
new file mode 100644
index 000000000000..ecb5ef273be5
--- /dev/null
+++ b/drivers/ptp/ptp_kvm_arm.c
@@ -0,0 +1,45 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  Virtual PTP 1588 clock for use with KVM guests
+ *  Copyright (C) 2019 ARM Ltd.
+ *  All Rights Reserved
+ */
+
+#include <linux/kernel.h>
+#include <linux/err.h>
+#include <asm/hypervisor.h>
+#include <linux/module.h>
+#include <linux/psci.h>
+#include <linux/arm-smccc.h>
+#include <linux/timecounter.h>
+#include <linux/ptp_kvm.h>
+#include <linux/sched/clock.h>
+#include <asm/arch_timer.h>
+#include <asm/hypervisor.h>
+
+int kvm_arch_ptp_init(void)
+{
+	int ret;
+
+	ret = kvm_arm_hyp_service_available(ARM_SMCCC_KVM_FUNC_PTP);
+	if (ret <= 0)
+		return -EOPNOTSUPP;
+
+	return 0;
+}
+
+int kvm_arch_ptp_get_clock(struct timespec64 *ts)
+{
+	ktime_t ktime;
+	struct arm_smccc_res hvc_res;
+
+	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
+			     ARM_PTP_VIRT_COUNTER, &hvc_res);
+	if ((int)(hvc_res.a0) < 0)
+		return -EOPNOTSUPP;
+
+	ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
+	*ts = ktime_to_timespec64(ktime);
+
+	return 0;
+}
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 8/9] doc: add ptp_kvm introduction for arm64 support
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (6 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 7/9] ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2021-02-02 13:43   ` Marc Zyngier
  2020-12-09  6:09 ` [PATCH v16 9/9] arm64: Add kvm capability check extension for ptp_kvm Jianyong Wu
                   ` (2 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

PTP_KVM implementation depends on hypercall using SMCCC. So we
introduce a new SMCCC service ID. This doc explains how does the
ID define and how does PTP_KVM works on arm/arm64.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 Documentation/virt/kvm/api.rst         |  9 +++++++
 Documentation/virt/kvm/arm/index.rst   |  1 +
 Documentation/virt/kvm/arm/ptp_kvm.rst | 31 +++++++++++++++++++++++
 Documentation/virt/kvm/timekeeping.rst | 35 ++++++++++++++++++++++++++
 4 files changed, 76 insertions(+)
 create mode 100644 Documentation/virt/kvm/arm/ptp_kvm.rst

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index e00a66d72372..3769cc2f7d9c 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6390,3 +6390,12 @@ When enabled, KVM will disable paravirtual features provided to the
 guest according to the bits in the KVM_CPUID_FEATURES CPUID leaf
 (0x40000001). Otherwise, a guest may use the paravirtual features
 regardless of what has actually been exposed through the CPUID leaf.
+
+8.27 KVM_CAP_PTP_KVM
+--------------------
+
+:Architectures: arm64
+
+This capability indicates that KVM virtual PTP service is supported in host.
+It must company with the implementation of KVM virtual PTP service in host
+so VMM can probe if there is the service in host by checking this capability.
diff --git a/Documentation/virt/kvm/arm/index.rst b/Documentation/virt/kvm/arm/index.rst
index 3e2b2aba90fc..78a9b670aafe 100644
--- a/Documentation/virt/kvm/arm/index.rst
+++ b/Documentation/virt/kvm/arm/index.rst
@@ -10,3 +10,4 @@ ARM
    hyp-abi
    psci
    pvtime
+   ptp_kvm
diff --git a/Documentation/virt/kvm/arm/ptp_kvm.rst b/Documentation/virt/kvm/arm/ptp_kvm.rst
new file mode 100644
index 000000000000..d729c1388a5c
--- /dev/null
+++ b/Documentation/virt/kvm/arm/ptp_kvm.rst
@@ -0,0 +1,31 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+PTP_KVM support for arm/arm64
+=============================
+
+PTP_KVM is used for time sync between guest and host in a high precision.
+It needs to get the wall time and counter value from the host and transfer these
+to guest via hypercall service. So one more hypercall service has been added.
+
+This new SMCCC hypercall is defined as:
+
+* ARM_SMCCC_HYP_KVM_PTP_FUNC_ID: 0x86000001
+
+As both 32 and 64-bits ptp_kvm client should be supported, we choose SMC32/HVC32
+calling convention.
+
+ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
+
+    =============    ==========    ==========
+    Function ID:     (uint32)      0x86000001
+    Arguments:       (uint32)      ARM_PTP_PHY_COUNTER(1) or ARM_PTP_VIRT_COUNTER(0)
+                                   which indicate acquiring physical counter or
+                                   virtual counter respectively.
+    Return Value:    val0(uint32)  NOT_SUPPORTED(-1) or upper 32 bits of wall clock time(64-bits).
+                     val1(uint32)  Lower 32 bits of wall clock time.
+                     val2(uint32)  Upper 32 bits of counter cycle(64-bits).
+                     val3(uint32)  Lower 32 bits of counter cycle.
+    Endianness:                    No Restrictions.
+    =============    ==========    ==========
+
+More info see section 5 in Documentation/virt/kvm/timekeeping.rst.
diff --git a/Documentation/virt/kvm/timekeeping.rst b/Documentation/virt/kvm/timekeeping.rst
index 21ae7efa29ba..c81383e38372 100644
--- a/Documentation/virt/kvm/timekeeping.rst
+++ b/Documentation/virt/kvm/timekeeping.rst
@@ -13,6 +13,7 @@ Timekeeping Virtualization for X86-Based Architectures
    2) Timing Devices
    3) TSC Hardware
    4) Virtualization Problems
+   5) KVM virtual PTP clock
 
 1. Overview
 ===========
@@ -643,3 +644,37 @@ by using CPU utilization itself as a signalling channel.  Preventing such
 problems would require completely isolated virtual time which may not track
 real time any longer.  This may be useful in certain security or QA contexts,
 but in general isn't recommended for real-world deployment scenarios.
+
+5. KVM virtual PTP clock
+========================
+
+NTP (Network Time Protocol) is often used to sync time in a VM. Unfortunately,
+the precision of NTP is limited due to unknown delays in the network.
+
+KVM virtual PTP clock (PTP_KVM) offers another way to sync time in VM; use the
+host's clock rather than one from a remote machine. Having a synchronization
+mechanism for the virtualization environment allows us to keep all the guests
+running on the same host in sync.
+In general, the delay of communication between host and guest is quite
+small, so ptp_kvm can offer time sync precision up to in order of nanoseconds.
+Please keep in mind that ptp_kvm just limits itself to be a channel which
+transmits the remote clock from host to guest. An application, eg. chrony, is
+needed in usersapce of VM in order to set the guest time.
+
+After ptp_kvm is initialized, there will be a new device node under /dev called
+ptp%d. A guest userspace service, like chrony, can use this device to get host
+walltime, sometimes also counter cycle, which depends on the service it calls.
+Then this guest userspace service can use those data to do the time sync for
+the guest.
+The following is the work flow of ptp_kvm:
+
+a) time sync service in guest userspace call ioctl on ptp device /dev/ptp%d.
+b) ptp_kvm module in guest receives this request then invokes hypercall to
+   route into host kernel to request host's walltime/counter cycle.
+c) ptp_kvm hypercall service on the host responds to the request and sends data
+   back.
+d) ptp in guest copies the data to userspace.
+
+ptp_kvm consists of components running on the guest and host. Step 2 consists of
+a guest driver making a hypercall whilst step 3 involves the hypervisor responding
+with information.
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v16 9/9] arm64: Add kvm capability check extension for ptp_kvm
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (7 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 8/9] doc: add ptp_kvm introduction for arm64 support Jianyong Wu
@ 2020-12-09  6:09 ` Jianyong Wu
  2021-02-02 13:44   ` Marc Zyngier
  2021-01-06  8:29 ` [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
  2021-02-02 14:15 ` Marc Zyngier
  10 siblings, 1 reply; 19+ messages in thread
From: Jianyong Wu @ 2020-12-09  6:09 UTC (permalink / raw)
  To: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, jianyong.wu, nd

Let userspace check if there is kvm ptp service in host.
Before VMs migrate to another host, VMM may check if this
cap is available to determine the next behavior.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Suggested-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/arm.c     | 1 +
 include/uapi/linux/kvm.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index f60f4a5e1a22..1bb1f64f9bb5 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -199,6 +199,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_INJECT_EXT_DABT:
 	case KVM_CAP_SET_GUEST_DEBUG:
 	case KVM_CAP_VCPU_ATTRIBUTES:
+	case KVM_CAP_PTP_KVM:
 		r = 1;
 		break;
 	case KVM_CAP_ARM_SET_DEVICE_ADDR:
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index ca41220b40b8..797c40bbc31f 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1053,6 +1053,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_X86_USER_SPACE_MSR 188
 #define KVM_CAP_X86_MSR_FILTER 189
 #define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190
+#define KVM_CAP_PTP_KVM 191
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* RE: [PATCH v16 0/9] Enable ptp_kvm for arm/arm64
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (8 preceding siblings ...)
  2020-12-09  6:09 ` [PATCH v16 9/9] arm64: Add kvm capability check extension for ptp_kvm Jianyong Wu
@ 2021-01-06  8:29 ` Jianyong Wu
  2021-02-02 14:15 ` Marc Zyngier
  10 siblings, 0 replies; 19+ messages in thread
From: Jianyong Wu @ 2021-01-06  8:29 UTC (permalink / raw)
  To: Jianyong Wu, netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, maz, richardcochran, Mark Rutland, will,
	Suzuki Poulose, Andre Przywara, Steven Price
  Cc: linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve Capper, Justin He, nd

Ping...
Any comments? Thanks!

> -----Original Message-----
> From: Jianyong Wu <jianyong.wu@arm.com>
> Sent: Wednesday, December 9, 2020 2:09 PM
> To: netdev@vger.kernel.org; yangbo.lu@nxp.com; john.stultz@linaro.org;
> tglx@linutronix.de; pbonzini@redhat.com; sean.j.christopherson@intel.com;
> maz@kernel.org; richardcochran@gmail.com; Mark Rutland
> <Mark.Rutland@arm.com>; will@kernel.org; Suzuki Poulose
> <Suzuki.Poulose@arm.com>; Andre Przywara <Andre.Przywara@arm.com>;
> Steven Price <Steven.Price@arm.com>
> Cc: linux-kernel@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; Steve Capper
> <Steve.Capper@arm.com>; Justin He <Justin.He@arm.com>; Jianyong Wu
> <Jianyong.Wu@arm.com>; nd <nd@arm.com>
> Subject: [PATCH v16 0/9] Enable ptp_kvm for arm/arm64
> 
> Currently, we offen use ntp (sync time with remote network clock) to sync
> time in VM. But the precision of ntp is subject to network delay so it's difficult
> to sync time in a high precision.
> 
> kvm virtual ptp clock (ptp_kvm) offers another way to sync time in VM, as
> the remote clock locates in the host instead of remote network clock.
> It targets to sync time between guest and host in virtualization environment
> and in this way, we can keep the time of all the VMs running in the same host
> in sync. In general, the delay of communication between host and guest is
> quiet small, so ptp_kvm can offer time sync precision up to in order of
> nanosecond. Please keep in mind that ptp_kvm just limits itself to be a
> channel which transmit the remote clock from host to guest and leaves the
> time sync jobs to an application, eg. chrony, in usersapce in VM.
> 
> How ptp_kvm works:
> After ptp_kvm initialized, there will be a new device node under /dev called
> ptp%d. A guest userspace service, like chrony, can use this device to get host
> walltime, sometimes also counter cycle, which depends on the service it calls.
> Then this guest userspace service can use those data to do the time sync for
> guest.
> here is a rough sketch to show how kvm ptp clock works.
> 
> |----------------------------|              |--------------------------|
> |       guest userspace      |              |          host            |
> |ioctl -> /dev/ptp%d         |              |                          |
> |       ^   |                |              |                          |
> |----------------------------|              |                          |
> |       |   | guest kernel   |              |                          |
> |       |   V      (get host walltime/counter cycle)                   |
> |      ptp_kvm -> hypercall - - - - - - - - - - ->hypercall service    |
> |                         <- - - - - - - - - - - -                     |
> |----------------------------|              |--------------------------|
> 
> 1. time sync service in guest userspace call ptp device through /dev/ptp%d.
> 2. ptp_kvm module in guest receives this request then invoke hypercall to
> route into host kernel to request host walltime/counter cycle.
> 3. ptp_kvm hypercall service in host response to the request and send data
> back.
> 4. ptp (not ptp_kvm) in guest copy the data to userspace.
> 
> This ptp_kvm implementation focuses itself to step 2 and 3 and step 2 works
> in guest comparing step 3 works in host kernel.
> 
> change log:
> 
> from v15 to v16:
>         (1) remove ARM_PTP_NONE_COUNTER suggested by Marc.
>         (2) add more detail for ptp_kvm doc.
>         (3) fix ci issues reported by test robot.
> 
> from v14 to v15:
>         (1) enable ptp_kvm on arm32 guest, also ptp_kvm has been tested on
> both arm64 and arm32 guest running on arm64 kvm host.
>         (2) move arch-agnostic part of ptp_kvm.rst into timekeeping.rst.
>         (3) rename KVM_CAP_ARM_PTP_KVM to KVM_CAP_PTP_KVM as it
> should be arch agnostic.
>         (4) add description for KVM_CAP_PTP_KVM in
> Documentation/virt/kvm/api.rst.
>         (5) adjust dependency in Kconfig for ptp_kvm.
>         (6) refine multi-arch process in driver/ptp/Makefile.
>         (7) fix make pdfdocs htmldocs issue for ptp_kvm doc.
>         (8) address other issues from comments in v14.
>         (9) fold hypercall service of ptp_kvm as a function.
>         (10) rebase to 5.10-rc3.
> 
> from v13 to v14
>         (1) rebase code on 5.9-rc3.
>         (2) add a document to introduce implementation of PTP_KVM on arm64.
>         (3) fix comments issue in hypercall.c.
>         (4) export arm_smccc_1_1_get_conduit using EXPORT_SYMBOL_GPL.
>         (5) fix make issue on x86 reported by kernel test robot.
> 
> from v12 to v13:
>         (1) rebase code on 5.8-rc1.
>         (2) this patch set base on 2 patches of 1/8 and 2/8 from Will Decon.
>         (3) remove the change to ptp device code of extend getcrosststamp.
>         (4) remove the mechanism of letting user choose the counter type in
> ptp_kvm for arm64.
>         (5) add virtual counter option in ptp_kvm service to let user choose the
> specific counter explicitly.
> 
> from v11 to v12:
>         (1) rebase code on 5.7-rc6 and rebase 2 patches from Will Decon
> including 1/11 and 2/11. as these patches introduce discover mechanism of
> vendor smccc service.
>         (2) rebase ptp_kvm hypercall service from standard smccc to vendor
> smccc and add ptp_kvm to vendor smccc service discover mechanism.
>         (3) add detail of why we need ptp_kvm and how ptp_kvm works in cover
> letter.
> 
> from v10 to v11:
>         (1) rebase code on 5.7-rc2.
>         (2) remove support for arm32, as kvm support for arm32 will be removed
> [1]
>         (3) add error report in ptp_kvm initialization.
> 
> from v9 to v10:
>         (1) change code base to v5.5.
>         (2) enable ptp_kvm both for arm32 and arm64.
>         (3) let user choose which of virtual counter or physical counter should
> return when using crosstimestamp mode of ptp_kvm for arm/arm64.
>         (4) extend input argument for getcrosstimestamp API.
> 
> from v8 to v9:
>         (1) move ptp_kvm.h to driver/ptp/
>         (2) replace license declaration of ptp_kvm.h the same with other header
> files in the same directory.
> 
> from v7 to v8:
>         (1) separate adding clocksource id for arm_arch_counter as a single patch.
>         (2) update commit message for patch 4/8.
>         (3) refine patch 7/8 and patch 8/8 to make them more independent.
> 
> from v5 to v6:
>         (1) apply Mark's patch[4] to get SMCCC conduit.
>         (2) add mechanism to recognize current clocksource by add
> clocksouce_id value into struct clocksource instead of method in patch-v5.
>         (3) rename kvm_arch_ptp_get_clock_fn into
> kvm_arch_ptp_get_crosststamp.
> 
> from v4 to v5:
>         (1) remove hvc delay compensasion as it should leave to userspace.
>         (2) check current clocksource in hvc call service.
>         (3) expose current clocksource by adding it to system_time_snapshot.
>         (4) add helper to check if clocksource is arm_arch_counter.
>         (5) rename kvm_ptp.c to ptp_kvm_common.c
> 
> from v3 to v4:
>         (1) fix clocksource of ptp_kvm to arch_sys_counter.
>         (2) move kvm_arch_ptp_get_clock_fn into arm_arch_timer.c
>         (3) subtract cntvoff before return cycles from host.
>         (4) use ktime_get_snapshot instead of getnstimeofday and
> get_current_counterval to return time and counter value.
>         (5) split ktime and counter into two 32-bit block respectively to avoid
> Y2038-safe issue.
>         (6) set time compensation to device time as half of the delay of hvc call.
>         (7) add ARM_ARCH_TIMER as dependency of ptp_kvm for arm64.
> 
> from v2 to v3:
>         (1) fix some issues in commit log.
>         (2) add some receivers in send list.
> 
> from v1 to v2:
>         (1) move arch-specific code from arch/ to driver/ptp/
>         (2) offer mechanism to inform userspace if ptp_kvm service is available.
>         (3) separate ptp_kvm code for arm64 into hypervisor part and guest part.
>         (4) add API to expose monotonic clock and counter value.
>         (5) refine code: remove no necessary part and reconsitution.
> 
> [1] https://patchwork.kernel.org/cover/11373351/
> 
> 
> Jianyong Wu (6):
>   ptp: Reorganize ptp_kvm module to make it arch-independent.
>   clocksource: Add clocksource id for arm arch counter
>   arm64/kvm: Add hypercall service for kvm ptp.
>   ptp: arm/arm64: Enable ptp_kvm for arm/arm64
>   doc: add ptp_kvm introduction for arm64 support
>   arm64: Add kvm capability check extension for ptp_kvm
> 
> Thomas Gleixner (1):
>   time: Add mechanism to recognize clocksource in time_get_snapshot
> 
> Will Deacon (2):
>   arm64: Probe for the presence of KVM hypervisor
>   arm/arm64: KVM: Advertise KVM UID to guests via SMCCC
> 
>  Documentation/virt/kvm/api.rst              |  9 ++
>  Documentation/virt/kvm/arm/index.rst        |  1 +
>  Documentation/virt/kvm/arm/ptp_kvm.rst      | 31 +++++++
>  Documentation/virt/kvm/timekeeping.rst      | 35 ++++++++
>  arch/arm/kernel/setup.c                     |  5 ++
>  arch/arm64/kernel/setup.c                   |  1 +
>  arch/arm64/kvm/arm.c                        |  1 +
>  arch/arm64/kvm/hypercalls.c                 | 86 ++++++++++++++++--
>  drivers/clocksource/arm_arch_timer.c        | 31 +++++++
>  drivers/firmware/smccc/smccc.c              | 37 ++++++++
>  drivers/ptp/Kconfig                         |  2 +-
>  drivers/ptp/Makefile                        |  2 +
>  drivers/ptp/ptp_kvm_arm.c                   | 45 ++++++++++
>  drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 84 +++++-------------
>  drivers/ptp/ptp_kvm_x86.c                   | 96 +++++++++++++++++++++
>  include/linux/arm-smccc.h                   | 59 +++++++++++++
>  include/linux/clocksource.h                 |  6 ++
>  include/linux/clocksource_ids.h             | 12 +++
>  include/linux/ptp_kvm.h                     | 16 ++++
>  include/linux/timekeeping.h                 | 12 +--
>  include/uapi/linux/kvm.h                    |  1 +
>  kernel/time/clocksource.c                   |  2 +
>  kernel/time/timekeeping.c                   |  1 +
>  23 files changed, 498 insertions(+), 77 deletions(-)  create mode 100644
> Documentation/virt/kvm/arm/ptp_kvm.rst
>  create mode 100644 drivers/ptp/ptp_kvm_arm.c  rename
> drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%)  create mode 100644
> drivers/ptp/ptp_kvm_x86.c  create mode 100644
> include/linux/clocksource_ids.h  create mode 100644
> include/linux/ptp_kvm.h
> 
> --
> 2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor
  2020-12-09  6:09 ` [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor Jianyong Wu
@ 2021-02-02 13:18   ` Marc Zyngier
  0 siblings, 0 replies; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 13:18 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price, linux-kernel,
	linux-arm-kernel, kvmarm, kvm, Steve.Capper, justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> From: Will Deacon <will@kernel.org>
> 
> Although the SMCCC specification provides some limited functionality 
> for
> describing the presence of hypervisor and firmware services, this is
> generally applicable only to functions designated as "Arm Architecture
> Service Functions" and no portable discovery mechanism is provided for
> standard hypervisor services, despite having a designated range of
> function identifiers reserved by the specification.
> 
> In an attempt to avoid the need for additional firmware changes every
> time a new function is added, introduce a UID to identify the service
> provider as being compatible with KVM. Once this has been established,
> additional services can be discovered via a feature bitmap.
> 
> Change from Jianyong Wu:
> mv kvm_arm_hyp_service_available to common place to let both arm/arm64 
> touch it.
> add kvm_init_hyp_services also under arm arch to let arm kvm guest use
> this service.
> 
> Cc: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Will Deacon <will@kernel.org>
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> ---
>  arch/arm/kernel/setup.c        |  5 ++++
>  arch/arm64/kernel/setup.c      |  1 +
>  drivers/firmware/smccc/smccc.c | 37 +++++++++++++++++++++++++++++
>  include/linux/arm-smccc.h      | 43 ++++++++++++++++++++++++++++++++++
>  4 files changed, 86 insertions(+)
> 
> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
> index 1a5edf562e85..adcefa9c8fab 100644
> --- a/arch/arm/kernel/setup.c
> +++ b/arch/arm/kernel/setup.c
> @@ -1156,6 +1156,11 @@ void __init setup_arch(char **cmdline_p)
> 
>  	arm_dt_init_cpu_maps();
>  	psci_dt_init();
> +
> +#ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY
> +	kvm_init_hyp_services();
> +#endif
> +
>  #ifdef CONFIG_SMP
>  	if (is_smp()) {
>  		if (!mdesc->smp_init || !mdesc->smp_init()) {
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index a950d5bc1ba5..97037b15c6ea 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -353,6 +353,7 @@ void __init __no_sanitize_address setup_arch(char
> **cmdline_p)
>  	else
>  		psci_acpi_init();
> 
> +	kvm_init_hyp_services();

Given that there is a dependency between the KVM discovery and PSCI,
it may make more sense to directly call this from the PSCI init,
and leave the arch-specific code alone.

>  	init_bootcpu_ops();
>  	smp_init_cpus();
>  	smp_build_mpidr_hash();
> diff --git a/drivers/firmware/smccc/smccc.c 
> b/drivers/firmware/smccc/smccc.c
> index 00c88b809c0c..e153c71ece99 100644
> --- a/drivers/firmware/smccc/smccc.c
> +++ b/drivers/firmware/smccc/smccc.c
> @@ -7,10 +7,47 @@
> 
>  #include <linux/init.h>
>  #include <linux/arm-smccc.h>
> +#include <linux/kernel.h>
> +#include <linux/string.h>
> 
>  static u32 smccc_version = ARM_SMCCC_VERSION_1_0;
>  static enum arm_smccc_conduit smccc_conduit = SMCCC_CONDUIT_NONE;
> 
> +DECLARE_BITMAP(__kvm_arm_hyp_services, ARM_SMCCC_KVM_NUM_FUNCS) = { };
> +EXPORT_SYMBOL_GPL(__kvm_arm_hyp_services);
> +
> +void __init kvm_init_hyp_services(void)
> +{
> +	int i;
> +	struct arm_smccc_res res;
> +
> +	if (arm_smccc_get_version() == ARM_SMCCC_VERSION_1_0)
> +		return;
> +
> +	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);

I think we may end-up calling into EL3 when running this on bare
metal if the systems implement PSCI 1,1. Robust firmware should
handle it, but there is plenty of broken ones around...

Checking on the SMCCC conduit would be safer.

> +	if (res.a0 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0 ||
> +	    res.a1 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1 ||
> +	    res.a2 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2 ||
> +	    res.a3 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3)
> +		return;
> +
> +	memset(&res, 0, sizeof(res));
> +	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID, 
> &res);
> +	for (i = 0; i < 32; ++i) {
> +		if (res.a0 & (i))
> +			set_bit(i + (32 * 0), __kvm_arm_hyp_services);
> +		if (res.a1 & (i))
> +			set_bit(i + (32 * 1), __kvm_arm_hyp_services);
> +		if (res.a2 & (i))
> +			set_bit(i + (32 * 2), __kvm_arm_hyp_services);
> +		if (res.a3 & (i))
> +			set_bit(i + (32 * 3), __kvm_arm_hyp_services);
> +	}
> +
> +	pr_info("KVM hypervisor services detected (0x%08lx 0x%08lx 0x%08lx
> 0x%08lx)\n",
> +		 res.a3, res.a2, res.a1, res.a0);
> +}
> +

Overall, this code should be in its own file, much like we already
do for the SOC_ID stuff.

>  void __init arm_smccc_version_init(u32 version, enum arm_smccc_conduit 
> conduit)
>  {
>  	smccc_version = version;
> diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
> index f860645f6512..d75408141137 100644
> --- a/include/linux/arm-smccc.h
> +++ b/include/linux/arm-smccc.h
> @@ -55,6 +55,8 @@
>  #define ARM_SMCCC_OWNER_TRUSTED_OS	50
>  #define ARM_SMCCC_OWNER_TRUSTED_OS_END	63
> 
> +#define ARM_SMCCC_FUNC_QUERY_CALL_UID  0xff01
> +
>  #define ARM_SMCCC_QUIRK_NONE		0
>  #define ARM_SMCCC_QUIRK_QCOM_A6		1 /* Save/restore register a6 */
> 
> @@ -87,6 +89,29 @@
>  			   ARM_SMCCC_SMC_32,				\
>  			   0, 0x7fff)
> 
> +#define ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID				\
> +	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
> +			   ARM_SMCCC_SMC_32,				\
> +			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
> +			   ARM_SMCCC_FUNC_QUERY_CALL_UID)
> +
> +/* KVM UID value: 28b46fb6-2ec5-11e9-a9ca-4b564d003a74 */
> +#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0	0xb66fb428U
> +#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1	0xe911c52eU
> +#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2	0x564bcaa9U
> +#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3	0x743a004dU
> +
> +/* KVM "vendor specific" services */
> +#define ARM_SMCCC_KVM_FUNC_FEATURES		0
> +#define ARM_SMCCC_KVM_FUNC_FEATURES_2		127
> +#define ARM_SMCCC_KVM_NUM_FUNCS			128
> +
> +#define ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID			\
> +	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
> +			   ARM_SMCCC_SMC_32,				\
> +			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
> +			   ARM_SMCCC_KVM_FUNC_FEATURES)
> +
>  #define SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED	1
> 
>  /* Paravirtualised time calls (defined by ARM DEN0057A) */
> @@ -391,5 +416,23 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0,
> unsigned long a1,
>  		method;							\
>  	})
> 
> +void __init kvm_init_hyp_services(void);
> +
> +/*
> + * This helper will be called in guest. We put it here then both arm 
> and arm64
> + * guest can touch it.
> + */
> +#include <linux/kernel.h>
> +#include <linux/err.h>

I don't see the need for linux/err.h.

> +static inline bool kvm_arm_hyp_service_available(u32 func_id)
> +{
> +	extern DECLARE_BITMAP(__kvm_arm_hyp_services, 
> ARM_SMCCC_KVM_NUM_FUNCS);
> +
> +	if (func_id >= ARM_SMCCC_KVM_NUM_FUNCS)
> +		return -EINVAL;
> +
> +	return test_bit(func_id, __kvm_arm_hyp_services);
> +}
> +
>  #endif /*__ASSEMBLY__*/
>  #endif /*__LINUX_ARM_SMCCC_H*/

If we move this as part of a file holding the whole of the
KVM discovery, we can make it non-inline and export it
instead of the bitmap.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 3/9] ptp: Reorganize ptp_kvm module to make it arch-independent.
  2020-12-09  6:09 ` [PATCH v16 3/9] ptp: Reorganize ptp_kvm module to make it arch-independent Jianyong Wu
@ 2021-02-02 13:23   ` Marc Zyngier
  0 siblings, 0 replies; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 13:23 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price, linux-kernel,
	linux-arm-kernel, kvmarm, kvm, Steve.Capper, justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> Currently, ptp_kvm modules implementation is only for x86 which 
> includes
> large part of arch-specific code.  This patch moves all of this code
> into a new arch related file in the same directory.
> 
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> ---
>  drivers/ptp/Makefile                        |  1 +
>  drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 84 +++++-------------
>  drivers/ptp/ptp_kvm_x86.c                   | 96 +++++++++++++++++++++
>  include/linux/ptp_kvm.h                     | 16 ++++
>  4 files changed, 135 insertions(+), 62 deletions(-)
>  rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%)
>  create mode 100644 drivers/ptp/ptp_kvm_x86.c
>  create mode 100644 include/linux/ptp_kvm.h
> 

[...]

> diff --git a/include/linux/ptp_kvm.h b/include/linux/ptp_kvm.h
> new file mode 100644
> index 000000000000..6f104b1967bb
> --- /dev/null
> +++ b/include/linux/ptp_kvm.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Virtual PTP 1588 clock for use with KVM guests
> + *
> + * Copyright (C) 2017 Red Hat Inc.
> + */
> +
> +#ifndef _PTP_KVM_H_
> +#define _PTP_KVM_H_
> +
> +int kvm_arch_ptp_init(void);
> +int kvm_arch_ptp_get_clock(struct timespec64 *ts);
> +int kvm_arch_ptp_get_crosststamp(u64 *cycle,
> +		struct timespec64 *tspec, struct clocksource **cs);

You probably want some forward declarations for timespec64 and
clocksource, so that this include file is standalone.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 6/9] arm64/kvm: Add hypercall service for kvm ptp.
  2020-12-09  6:09 ` [PATCH v16 6/9] arm64/kvm: Add hypercall service for kvm ptp Jianyong Wu
@ 2021-02-02 13:32   ` Marc Zyngier
  0 siblings, 0 replies; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 13:32 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price, linux-kernel,
	linux-arm-kernel, kvmarm, kvm, Steve.Capper, justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> ptp_kvm will get this service through SMCC call.
> The service offers wall time and cycle count of host to guest.
> The caller must specify whether they want the host cycle count
> or the difference between host cycle count and cntvoff.
> 
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> ---
>  arch/arm64/kvm/hypercalls.c | 59 +++++++++++++++++++++++++++++++++++++
>  include/linux/arm-smccc.h   | 16 ++++++++++
>  2 files changed, 75 insertions(+)
> 
> diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c
> index b9d8607083eb..9a4834502388 100644
> --- a/arch/arm64/kvm/hypercalls.c
> +++ b/arch/arm64/kvm/hypercalls.c
> @@ -9,6 +9,49 @@
>  #include <kvm/arm_hypercalls.h>
>  #include <kvm/arm_psci.h>
> 
> +static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val)
> +{
> +	struct system_time_snapshot systime_snapshot;
> +	u64 cycles = ~0UL;
> +	u32 feature;
> +
> +	/*
> +	 * system time and counter value must captured in the same
> +	 * time to keep consistency and precision.
> +	 */
> +	ktime_get_snapshot(&systime_snapshot);
> +
> +	// binding ptp_kvm clocksource to arm_arch_counter
> +	if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
> +		return;
> +
> +	val[0] = upper_32_bits(systime_snapshot.real);
> +	val[1] = lower_32_bits(systime_snapshot.real);
> +
> +	/*
> +	 * which of virtual counter or physical counter being
> +	 * asked for is decided by the r1 value of SMCCC
> +	 * call. If no invalid r1 value offered, default cycle
> +	 * value(-1) will be returned.
> +	 * Note: keep in mind that feature is u32 and smccc_get_arg1
> +	 * will return u64, so need auto cast here.

This feels like a useless comment. On the other hand, you don't
document why we don't need to worry about -1 being used as the
top 32bits of the real time.

> +	 */
> +	feature = smccc_get_arg1(vcpu);
> +	switch (feature) {
> +	case ARM_PTP_VIRT_COUNTER:

This #define should at least contain "KVM", because this isn't
something that is architectural.

> +		cycles = systime_snapshot.cycles - vcpu_read_sys_reg(vcpu, 
> CNTVOFF_EL2);
> +		break;
> +	case ARM_PTP_PHY_COUNTER:

s/PHY/PHYS/

> +		cycles = systime_snapshot.cycles;
> +		break;
> +	default:
> +		val[0] = SMCCC_RET_NOT_SUPPORTED;
> +		break;
> +	}
> +	val[2] = upper_32_bits(cycles);
> +	val[3] = lower_32_bits(cycles);

The control flow is really odd. You start by writing values to
the return array, and then have to overwrite it if you fail.

I came up with the following instead (including comments):

@@ -16,38 +16,44 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, 
u64 *val)
  	u32 feature;

  	/*
-	 * system time and counter value must captured in the same
+	 * system time and counter value must captured at the same
  	 * time to keep consistency and precision.
  	 */
  	ktime_get_snapshot(&systime_snapshot);

-	// binding ptp_kvm clocksource to arm_arch_counter
+	/*
+	 * This is only valid if the current clocksource is the
+	 * architected counter, as this is the only one the guest
+	 * can see.
+	 */
  	if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
  		return;

-	val[0] = upper_32_bits(systime_snapshot.real);
-	val[1] = lower_32_bits(systime_snapshot.real);
-
  	/*
-	 * which of virtual counter or physical counter being
-	 * asked for is decided by the r1 value of SMCCC
-	 * call. If no invalid r1 value offered, default cycle
-	 * value(-1) will be returned.
-	 * Note: keep in mind that feature is u32 and smccc_get_arg1
-	 * will return u64, so need auto cast here.
+	 * The guest selects one of the two reference counters
+	 * (virtual or physical) with the first argument of the SMCCC
+	 * call. In case the identifier is not supported, error out.
  	 */
  	feature = smccc_get_arg1(vcpu);
  	switch (feature) {
-	case ARM_PTP_VIRT_COUNTER:
+	case KVM_PTP_VIRT_COUNTER:
  		cycles = systime_snapshot.cycles - vcpu_read_sys_reg(vcpu, 
CNTVOFF_EL2);
  		break;
-	case ARM_PTP_PHY_COUNTER:
+	case KVM_PTP_PHYS_COUNTER:
  		cycles = systime_snapshot.cycles;
  		break;
  	default:
-		val[0] = SMCCC_RET_NOT_SUPPORTED;
-		break;
+		return;
  	}
+
+	/*
+	 * This relies on the top bit of val[0] never being set for
+	 * valid values of system time, because that is *really* far
+	 * in the future (about 292 years from 1970, and at that stage
+	 * nobody will give a damn about it).
+	 */
+	val[0] = upper_32_bits(systime_snapshot.real);
+	val[1] = lower_32_bits(systime_snapshot.real);
  	val[2] = upper_32_bits(cycles);
  	val[3] = lower_32_bits(cycles);
  }


> +}
> +
>  int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
>  {
>  	u32 func_id = smccc_get_function(vcpu);
> @@ -79,6 +122,22 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
>  		break;
>  	case ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID:
>  		val[0] = BIT(ARM_SMCCC_KVM_FUNC_FEATURES);
> +		val[0] |= BIT(ARM_SMCCC_KVM_FUNC_PTP);
> +		break;
> +	/*
> +	 * This serves virtual kvm_ptp.
> +	 * Four values will be passed back.
> +	 * reg0 stores high 32-bits of host ktime;
> +	 * reg1 stores low 32-bits of host ktime;
> +	 * For ARM_PTP_VIRT_COUNTER:
> +	 * reg2 stores high 32-bits of difference of host cycles and cntvoff;
> +	 * reg3 stores low 32-bits of difference of host cycles and cntvoff.
> +	 * For ARM_PTP_PHY_COUNTER:
> +	 * reg2 stores the high 32-bits of host cycles;
> +	 * reg3 stores the low 32-bits of host cycles.
> +	 */

You have a patch with documentation in this series, and I don't think
this adds much.

> +	case ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID:
> +		kvm_ptp_get_time(vcpu, val);
>  		break;
>  	default:
>  		return kvm_psci_call(vcpu);
> diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
> index d75408141137..7924069f8f0a 100644
> --- a/include/linux/arm-smccc.h
> +++ b/include/linux/arm-smccc.h
> @@ -103,6 +103,7 @@
> 
>  /* KVM "vendor specific" services */
>  #define ARM_SMCCC_KVM_FUNC_FEATURES		0
> +#define ARM_SMCCC_KVM_FUNC_PTP			1
>  #define ARM_SMCCC_KVM_FUNC_FEATURES_2		127
>  #define ARM_SMCCC_KVM_NUM_FUNCS			128
> 
> @@ -114,6 +115,21 @@
> 
>  #define SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED	1
> 
> +/*
> + * ptp_kvm is a feature used for time sync between vm and host.
> + * ptp_kvm module in guest kernel will get service from host using
> + * this hypercall ID.
> + */
> +#define ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID				\
> +	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
> +			   ARM_SMCCC_SMC_32,				\
> +			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
> +			   ARM_SMCCC_KVM_FUNC_PTP)
> +
> +/* ptp_kvm counter type ID */
> +#define ARM_PTP_VIRT_COUNTER			0
> +#define ARM_PTP_PHY_COUNTER			1

+#define KVM_PTP_VIRT_COUNTER			0
+#define KVM_PTP_PHYS_COUNTER			1

> +
>  /* Paravirtualised time calls (defined by ARM DEN0057A) */
>  #define ARM_SMCCC_HV_PV_TIME_FEATURES				\
>  	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,			\

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 7/9] ptp: arm/arm64: Enable ptp_kvm for arm/arm64
  2020-12-09  6:09 ` [PATCH v16 7/9] ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jianyong Wu
@ 2021-02-02 13:37   ` Marc Zyngier
  0 siblings, 0 replies; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 13:37 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price, linux-kernel,
	linux-arm-kernel, kvmarm, kvm, Steve.Capper, justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> Currently, there is no mechanism to keep time sync between guest and 
> host
> in arm/arm64 virtualization environment. Time in guest will drift 
> compared
> with host after boot up as they may both use third party time sources
> to correct their time respectively. The time deviation will be in order
> of milliseconds. But in some scenarios,like in cloud environment, we 
> ask
> for higher time precision.
> 
> kvm ptp clock, which chooses the host clock source as a reference
> clock to sync time between guest and host, has been adopted by x86
> which takes the time sync order from milliseconds to nanoseconds.
> 
> This patch enables kvm ptp clock for arm/arm64 and improves clock sync 
> precision
> significantly.
> 
> Test result comparisons between with kvm ptp clock and without it in 
> arm/arm64
> are as follows. This test derived from the result of command 'chronyc
> sources'. we should take more care of the last sample column which 
> shows
> the offset between the local clock and the source at the last 
> measurement.
> 
> no kvm ptp in guest:
> MS Name/IP address   Stratum Poll Reach LastRx Last sample
> ========================================================================
> ^* dns1.synet.edu.cn      2   6   377    13  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    21  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    29  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    37  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    45  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    53  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    61  +1040us[+1581us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377     4   -130us[ +796us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    12   -130us[ +796us] +/-   
> 21ms
> ^* dns1.synet.edu.cn      2   6   377    20   -130us[ +796us] +/-   
> 21ms
> 
> in host:
> MS Name/IP address   Stratum Poll Reach LastRx Last sample
> ========================================================================
> ^* 120.25.115.20          2   7   377    72   -470us[ -603us] +/-   
> 18ms
> ^* 120.25.115.20          2   7   377    92   -470us[ -603us] +/-   
> 18ms
> ^* 120.25.115.20          2   7   377   112   -470us[ -603us] +/-   
> 18ms
> ^* 120.25.115.20          2   7   377     2   +872ns[-6808ns] +/-   
> 17ms
> ^* 120.25.115.20          2   7   377    22   +872ns[-6808ns] +/-   
> 17ms
> ^* 120.25.115.20          2   7   377    43   +872ns[-6808ns] +/-   
> 17ms
> ^* 120.25.115.20          2   7   377    63   +872ns[-6808ns] +/-   
> 17ms
> ^* 120.25.115.20          2   7   377    83   +872ns[-6808ns] +/-   
> 17ms
> ^* 120.25.115.20          2   7   377   103   +872ns[-6808ns] +/-   
> 17ms
> ^* 120.25.115.20          2   7   377   123   +872ns[-6808ns] +/-   
> 17ms
> 
> The dns1.synet.edu.cn is the network reference clock for guest and
> 120.25.115.20 is the network reference clock for host. we can't get the
> clock error between guest and host directly, but a roughly estimated 
> value
> will be in order of hundreds of us to ms.
> 
> with kvm ptp in guest:
> chrony has been disabled in host to remove the disturb by network 
> clock.
> 
> MS Name/IP address         Stratum Poll Reach LastRx Last sample
> ========================================================================
> * PHC0                    0   3   377     8     -7ns[   +1ns] +/-    
> 3ns
> * PHC0                    0   3   377     8     +1ns[  +16ns] +/-    
> 3ns
> * PHC0                    0   3   377     6     -4ns[   -0ns] +/-    
> 6ns
> * PHC0                    0   3   377     6     -8ns[  -12ns] +/-    
> 5ns
> * PHC0                    0   3   377     5     +2ns[   +4ns] +/-    
> 4ns
> * PHC0                    0   3   377    13     +2ns[   +4ns] +/-    
> 4ns
> * PHC0                    0   3   377    12     -4ns[   -6ns] +/-    
> 4ns
> * PHC0                    0   3   377    11     -8ns[  -11ns] +/-    
> 6ns
> * PHC0                    0   3   377    10    -14ns[  -20ns] +/-    
> 4ns
> * PHC0                    0   3   377     8     +4ns[   +5ns] +/-    
> 4ns
> 
> The PHC0 is the ptp clock which choose the host clock as its source
> clock. So we can see that the clock difference between host and guest
> is in order of ns.
> 
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> ---
>  drivers/clocksource/arm_arch_timer.c | 29 ++++++++++++++++++
>  drivers/ptp/Kconfig                  |  2 +-
>  drivers/ptp/Makefile                 |  1 +
>  drivers/ptp/ptp_kvm_arm.c            | 45 ++++++++++++++++++++++++++++
>  4 files changed, 76 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/ptp/ptp_kvm_arm.c
> 
> diff --git a/drivers/clocksource/arm_arch_timer.c
> b/drivers/clocksource/arm_arch_timer.c
> index d55acffb0b90..16cd0a663587 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -25,6 +25,8 @@
>  #include <linux/sched/clock.h>
>  #include <linux/sched_clock.h>
>  #include <linux/acpi.h>
> +#include <linux/arm-smccc.h>
> +#include <linux/ptp_kvm.h>
> 
>  #include <asm/arch_timer.h>
>  #include <asm/virt.h>
> @@ -1650,3 +1652,30 @@ static int __init arch_timer_acpi_init(struct
> acpi_table_header *table)
>  }
>  TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init);
>  #endif
> +
> +int kvm_arch_ptp_get_crosststamp(u64 *cycle, struct timespec64 *ts,
> +			      struct clocksource **cs)
> +{
> +	struct arm_smccc_res hvc_res;
> +	ktime_t ktime;
> +	u32 ptp_counter;
> +
> +	if (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)
> +		ptp_counter = ARM_PTP_VIRT_COUNTER;
> +	else
> +		ptp_counter = ARM_PTP_PHY_COUNTER;
> +
> +	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
> +			     ptp_counter, &hvc_res);
> +
> +	if ((int)(hvc_res.a0) < 0)
> +		return -EOPNOTSUPP;
> +
> +	ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
> +	*ts = ktime_to_timespec64(ktime);
> +	*cycle = (u64)hvc_res.a2 << 32 | hvc_res.a3;
> +	*cs = &clocksource_counter;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(kvm_arch_ptp_get_crosststamp);
> diff --git a/drivers/ptp/Kconfig b/drivers/ptp/Kconfig
> index 942f72d8151d..677c7f696b70 100644
> --- a/drivers/ptp/Kconfig
> +++ b/drivers/ptp/Kconfig
> @@ -106,7 +106,7 @@ config PTP_1588_CLOCK_PCH
>  config PTP_1588_CLOCK_KVM
>  	tristate "KVM virtual PTP clock"
>  	depends on PTP_1588_CLOCK
> -	depends on KVM_GUEST && X86
> +	depends on KVM_GUEST && X86 || (HAVE_ARM_SMCCC_DISCOVERY && 
> ARM_ARCH_TIMER)
>  	default y
>  	help
>  	  This driver adds support for using kvm infrastructure as a PTP
> diff --git a/drivers/ptp/Makefile b/drivers/ptp/Makefile
> index 699a4e4d19c2..9fa5ede44b2b 100644
> --- a/drivers/ptp/Makefile
> +++ b/drivers/ptp/Makefile
> @@ -5,6 +5,7 @@
> 
>  ptp-y					:= ptp_clock.o ptp_chardev.o ptp_sysfs.o
>  ptp_kvm-$(CONFIG_X86)			:= ptp_kvm_x86.o ptp_kvm_common.o
> +ptp_kvm-$(CONFIG_HAVE_ARM_SMCCC)	:= ptp_kvm_arm.o ptp_kvm_common.o
>  obj-$(CONFIG_PTP_1588_CLOCK)		+= ptp.o
>  obj-$(CONFIG_PTP_1588_CLOCK_DTE)	+= ptp_dte.o
>  obj-$(CONFIG_PTP_1588_CLOCK_INES)	+= ptp_ines.o
> diff --git a/drivers/ptp/ptp_kvm_arm.c b/drivers/ptp/ptp_kvm_arm.c
> new file mode 100644
> index 000000000000..ecb5ef273be5
> --- /dev/null
> +++ b/drivers/ptp/ptp_kvm_arm.c
> @@ -0,0 +1,45 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + *  Virtual PTP 1588 clock for use with KVM guests
> + *  Copyright (C) 2019 ARM Ltd.
> + *  All Rights Reserved
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/err.h>
> +#include <asm/hypervisor.h>
> +#include <linux/module.h>
> +#include <linux/psci.h>
> +#include <linux/arm-smccc.h>
> +#include <linux/timecounter.h>
> +#include <linux/ptp_kvm.h>
> +#include <linux/sched/clock.h>
> +#include <asm/arch_timer.h>
> +#include <asm/hypervisor.h>
> +
> +int kvm_arch_ptp_init(void)
> +{
> +	int ret;
> +
> +	ret = kvm_arm_hyp_service_available(ARM_SMCCC_KVM_FUNC_PTP);
> +	if (ret <= 0)
> +		return -EOPNOTSUPP;
> +
> +	return 0;
> +}
> +
> +int kvm_arch_ptp_get_clock(struct timespec64 *ts)
> +{
> +	ktime_t ktime;
> +	struct arm_smccc_res hvc_res;
> +
> +	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
> +			     ARM_PTP_VIRT_COUNTER, &hvc_res);
> +	if ((int)(hvc_res.a0) < 0)
> +		return -EOPNOTSUPP;
> +
> +	ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
> +	*ts = ktime_to_timespec64(ktime);

This whole function can be trivially replaced with a call to
kvm_arch_ptp_get_crosststamp with minimal changes to that function:

diff --git a/drivers/clocksource/arm_arch_timer.c 
b/drivers/clocksource/arm_arch_timer.c
index e3a5ebbfb3d0..e7b07166e771 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -1663,16 +1663,16 @@ TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, 
arch_timer_acpi_init);
  #endif

  int kvm_arch_ptp_get_crosststamp(u64 *cycle, struct timespec64 *ts,
-			      struct clocksource **cs)
+				 struct clocksource **cs)
  {
  	struct arm_smccc_res hvc_res;
-	ktime_t ktime;
  	u32 ptp_counter;
+	ktime_t ktime;

  	if (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)
-		ptp_counter = ARM_PTP_VIRT_COUNTER;
+		ptp_counter = KVM_PTP_VIRT_COUNTER;
  	else
-		ptp_counter = ARM_PTP_PHY_COUNTER;
+		ptp_counter = KVM_PTP_PHYS_COUNTER;

  	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
  			     ptp_counter, &hvc_res);
@@ -1682,8 +1682,10 @@ int kvm_arch_ptp_get_crosststamp(u64 *cycle, 
struct timespec64 *ts,

  	ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
  	*ts = ktime_to_timespec64(ktime);
-	*cycle = (u64)hvc_res.a2 << 32 | hvc_res.a3;
-	*cs = &clocksource_counter;
+	if (cycle)
+		*cycle = (u64)hvc_res.a2 << 32 | hvc_res.a3;
+	if (cs)
+		*cs = &clocksource_counter;

  	return 0;
  }
diff --git a/drivers/ptp/ptp_kvm_arm.c b/drivers/ptp/ptp_kvm_arm.c
index ecb5ef273be5..b7d28c8dfb84 100644
--- a/drivers/ptp/ptp_kvm_arm.c
+++ b/drivers/ptp/ptp_kvm_arm.c
@@ -5,15 +5,9 @@
   *  All Rights Reserved
   */

-#include <linux/kernel.h>
-#include <linux/err.h>
-#include <asm/hypervisor.h>
-#include <linux/module.h>
-#include <linux/psci.h>
  #include <linux/arm-smccc.h>
-#include <linux/timecounter.h>
  #include <linux/ptp_kvm.h>
-#include <linux/sched/clock.h>
+
  #include <asm/arch_timer.h>
  #include <asm/hypervisor.h>

@@ -30,16 +24,5 @@ int kvm_arch_ptp_init(void)

  int kvm_arch_ptp_get_clock(struct timespec64 *ts)
  {
-	ktime_t ktime;
-	struct arm_smccc_res hvc_res;
-
-	arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
-			     ARM_PTP_VIRT_COUNTER, &hvc_res);
-	if ((int)(hvc_res.a0) < 0)
-		return -EOPNOTSUPP;
-
-	ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
-	*ts = ktime_to_timespec64(ktime);
-
-	return 0;
+	return kvm_arch_ptp_get_crosststamp(NULL, ts, NULL);
  }

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 8/9] doc: add ptp_kvm introduction for arm64 support
  2020-12-09  6:09 ` [PATCH v16 8/9] doc: add ptp_kvm introduction for arm64 support Jianyong Wu
@ 2021-02-02 13:43   ` Marc Zyngier
  0 siblings, 0 replies; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 13:43 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini, richardcochran,
	Mark.Rutland, will, suzuki.poulose, Andre.Przywara, steven.price,
	linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> PTP_KVM implementation depends on hypercall using SMCCC. So we
> introduce a new SMCCC service ID. This doc explains how does the
> ID define and how does PTP_KVM works on arm/arm64.
> 
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> ---
>  Documentation/virt/kvm/api.rst         |  9 +++++++
>  Documentation/virt/kvm/arm/index.rst   |  1 +
>  Documentation/virt/kvm/arm/ptp_kvm.rst | 31 +++++++++++++++++++++++
>  Documentation/virt/kvm/timekeeping.rst | 35 ++++++++++++++++++++++++++
>  4 files changed, 76 insertions(+)
>  create mode 100644 Documentation/virt/kvm/arm/ptp_kvm.rst
> 
> diff --git a/Documentation/virt/kvm/api.rst 
> b/Documentation/virt/kvm/api.rst
> index e00a66d72372..3769cc2f7d9c 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -6390,3 +6390,12 @@ When enabled, KVM will disable paravirtual
> features provided to the
>  guest according to the bits in the KVM_CPUID_FEATURES CPUID leaf
>  (0x40000001). Otherwise, a guest may use the paravirtual features
>  regardless of what has actually been exposed through the CPUID leaf.
> +
> +8.27 KVM_CAP_PTP_KVM
> +--------------------
> +
> +:Architectures: arm64
> +
> +This capability indicates that KVM virtual PTP service is supported in 
> host.
> +It must company with the implementation of KVM virtual PTP service in 
> host
> +so VMM can probe if there is the service in host by checking this 
> capability.

This reads a bit odd. I came up with the following:

+This capability indicates that the KVM virtual PTP service is
+supported in the host. A VMM can check whether the service is
+available to the guest on migration.


> diff --git a/Documentation/virt/kvm/arm/index.rst
> b/Documentation/virt/kvm/arm/index.rst
> index 3e2b2aba90fc..78a9b670aafe 100644
> --- a/Documentation/virt/kvm/arm/index.rst
> +++ b/Documentation/virt/kvm/arm/index.rst
> @@ -10,3 +10,4 @@ ARM
>     hyp-abi
>     psci
>     pvtime
> +   ptp_kvm
> diff --git a/Documentation/virt/kvm/arm/ptp_kvm.rst
> b/Documentation/virt/kvm/arm/ptp_kvm.rst
> new file mode 100644
> index 000000000000..d729c1388a5c
> --- /dev/null
> +++ b/Documentation/virt/kvm/arm/ptp_kvm.rst
> @@ -0,0 +1,31 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +PTP_KVM support for arm/arm64
> +=============================
> +
> +PTP_KVM is used for time sync between guest and host in a high 
> precision.
> +It needs to get the wall time and counter value from the host and
> transfer these
> +to guest via hypercall service. So one more hypercall service has been 
> added.
> +
> +This new SMCCC hypercall is defined as:

It won't be new anymore the minute this is merged.

> +
> +* ARM_SMCCC_HYP_KVM_PTP_FUNC_ID: 0x86000001
> +
> +As both 32 and 64-bits ptp_kvm client should be supported, we choose
> SMC32/HVC32
> +calling convention.
> +
> +ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
> +
> +    =============    ==========    ==========
> +    Function ID:     (uint32)      0x86000001
> +    Arguments:       (uint32)      ARM_PTP_PHY_COUNTER(1) or
> ARM_PTP_VIRT_COUNTER(0)
> +                                   which indicate acquiring physical 
> counter or
> +                                   virtual counter respectively.
> +    Return Value:    val0(uint32)  NOT_SUPPORTED(-1) or upper 32 bits
> of wall clock time(64-bits).
> +                     val1(uint32)  Lower 32 bits of wall clock time.
> +                     val2(uint32)  Upper 32 bits of counter 
> cycle(64-bits).
> +                     val3(uint32)  Lower 32 bits of counter cycle.
> +    Endianness:                    No Restrictions.
> +    =============    ==========    ==========
> +
> +More info see section 5 in Documentation/virt/kvm/timekeeping.rst.

I've tidied this up like this:

diff --git a/Documentation/virt/kvm/arm/ptp_kvm.rst 
b/Documentation/virt/kvm/arm/ptp_kvm.rst
index d729c1388a5c..68cffb50d8bf 100644
--- a/Documentation/virt/kvm/arm/ptp_kvm.rst
+++ b/Documentation/virt/kvm/arm/ptp_kvm.rst
@@ -3,29 +3,23 @@
  PTP_KVM support for arm/arm64
  =============================

-PTP_KVM is used for time sync between guest and host in a high 
precision.
-It needs to get the wall time and counter value from the host and 
transfer these
-to guest via hypercall service. So one more hypercall service has been 
added.
-
-This new SMCCC hypercall is defined as:
+PTP_KVM is used for high precision time sync between host and guests.
+It relies on transferring the wall clock and counter value from the
+host to the guest using a KVM-specific hypercall.

  * ARM_SMCCC_HYP_KVM_PTP_FUNC_ID: 0x86000001

-As both 32 and 64-bits ptp_kvm client should be supported, we choose 
SMC32/HVC32
-calling convention.
-
-ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
+This hypercall uses the SMC32/HVC32 calling convention:

+ARM_SMCCC_HYP_KVM_PTP_FUNC_ID
      =============    ==========    ==========
      Function ID:     (uint32)      0x86000001
-    Arguments:       (uint32)      ARM_PTP_PHY_COUNTER(1) or 
ARM_PTP_VIRT_COUNTER(0)
-                                   which indicate acquiring physical 
counter or
-                                   virtual counter respectively.
-    Return Value:    val0(uint32)  NOT_SUPPORTED(-1) or upper 32 bits 
of wall clock time(64-bits).
-                     val1(uint32)  Lower 32 bits of wall clock time.
-                     val2(uint32)  Upper 32 bits of counter 
cycle(64-bits).
-                     val3(uint32)  Lower 32 bits of counter cycle.
+    Arguments:       (uint32)      KVM_PTP_VIRT_COUNTER(0)
+                                   KVM_PTP_PHYS_COUNTER(1)
+    Return Values:   (int32)       NOT_SUPPORTED(-1) on error, or
+                     (uint32)      Upper 32 bits of wall clock time 
(r0)
+                     (uint32)      Lower 32 bits of wall clock time 
(r1)
+                     (uint32)      Upper 32 bits of counter (r2)
+                     (uint32)      Lower 32 bits of counter (r3)
      Endianness:                    No Restrictions.
      =============    ==========    ==========
-
-More info see section 5 in Documentation/virt/kvm/timekeeping.rst.

> diff --git a/Documentation/virt/kvm/timekeeping.rst
> b/Documentation/virt/kvm/timekeeping.rst
> index 21ae7efa29ba..c81383e38372 100644
> --- a/Documentation/virt/kvm/timekeeping.rst
> +++ b/Documentation/virt/kvm/timekeeping.rst
> @@ -13,6 +13,7 @@ Timekeeping Virtualization for X86-Based 
> Architectures
>     2) Timing Devices
>     3) TSC Hardware
>     4) Virtualization Problems
> +   5) KVM virtual PTP clock
> 
>  1. Overview
>  ===========
> @@ -643,3 +644,37 @@ by using CPU utilization itself as a signalling
> channel.  Preventing such
>  problems would require completely isolated virtual time which may not 
> track
>  real time any longer.  This may be useful in certain security or QA 
> contexts,
>  but in general isn't recommended for real-world deployment scenarios.
> +
> +5. KVM virtual PTP clock
> +========================
> +
> +NTP (Network Time Protocol) is often used to sync time in a VM. 
> Unfortunately,
> +the precision of NTP is limited due to unknown delays in the network.
> +
> +KVM virtual PTP clock (PTP_KVM) offers another way to sync time in VM; 
> use the
> +host's clock rather than one from a remote machine. Having a 
> synchronization
> +mechanism for the virtualization environment allows us to keep all the 
> guests
> +running on the same host in sync.
> +In general, the delay of communication between host and guest is quite
> +small, so ptp_kvm can offer time sync precision up to in order of 
> nanoseconds.
> +Please keep in mind that ptp_kvm just limits itself to be a channel 
> which
> +transmits the remote clock from host to guest. An application, eg. 
> chrony, is
> +needed in usersapce of VM in order to set the guest time.
> +
> +After ptp_kvm is initialized, there will be a new device node under 
> /dev called
> +ptp%d. A guest userspace service, like chrony, can use this device to 
> get host
> +walltime, sometimes also counter cycle, which depends on the service 
> it calls.
> +Then this guest userspace service can use those data to do the time 
> sync for
> +the guest.
> +The following is the work flow of ptp_kvm:
> +
> +a) time sync service in guest userspace call ioctl on ptp device 
> /dev/ptp%d.
> +b) ptp_kvm module in guest receives this request then invokes 
> hypercall to
> +   route into host kernel to request host's walltime/counter cycle.
> +c) ptp_kvm hypercall service on the host responds to the request and 
> sends data
> +   back.
> +d) ptp in guest copies the data to userspace.
> +
> +ptp_kvm consists of components running on the guest and host. Step 2
> consists of
> +a guest driver making a hypercall whilst step 3 involves the
> hypervisor responding
> +with information.

I don't think we need any of this here, as the whole file
focuses on x86-specific issues for timekeeping. If we want
to document KVM PTP, this should probably be a separate document.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 9/9] arm64: Add kvm capability check extension for ptp_kvm
  2020-12-09  6:09 ` [PATCH v16 9/9] arm64: Add kvm capability check extension for ptp_kvm Jianyong Wu
@ 2021-02-02 13:44   ` Marc Zyngier
  0 siblings, 0 replies; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 13:44 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini,
	sean.j.christopherson, richardcochran, Mark.Rutland, will,
	suzuki.poulose, Andre.Przywara, steven.price, linux-kernel,
	linux-arm-kernel, kvmarm, kvm, Steve.Capper, justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> Let userspace check if there is kvm ptp service in host.
> Before VMs migrate to another host, VMM may check if this
> cap is available to determine the next behavior.
> 
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> Suggested-by: Marc Zyngier <maz@kernel.org>
> ---
>  arch/arm64/kvm/arm.c     | 1 +
>  include/uapi/linux/kvm.h | 1 +
>  2 files changed, 2 insertions(+)
> 
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index f60f4a5e1a22..1bb1f64f9bb5 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -199,6 +199,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, 
> long ext)
>  	case KVM_CAP_ARM_INJECT_EXT_DABT:
>  	case KVM_CAP_SET_GUEST_DEBUG:
>  	case KVM_CAP_VCPU_ATTRIBUTES:
> +	case KVM_CAP_PTP_KVM:
>  		r = 1;
>  		break;
>  	case KVM_CAP_ARM_SET_DEVICE_ADDR:
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index ca41220b40b8..797c40bbc31f 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -1053,6 +1053,7 @@ struct kvm_ppc_resize_hpt {
>  #define KVM_CAP_X86_USER_SPACE_MSR 188
>  #define KVM_CAP_X86_MSR_FILTER 189
>  #define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190
> +#define KVM_CAP_PTP_KVM 191
> 
>  #ifdef KVM_CAP_IRQ_ROUTING

This really should be squashed with the patch implementing
the hypercall.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v16 0/9] Enable ptp_kvm for arm/arm64
  2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
                   ` (9 preceding siblings ...)
  2021-01-06  8:29 ` [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
@ 2021-02-02 14:15 ` Marc Zyngier
  2021-02-03  2:40   ` Jianyong Wu
  10 siblings, 1 reply; 19+ messages in thread
From: Marc Zyngier @ 2021-02-02 14:15 UTC (permalink / raw)
  To: Jianyong Wu
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini, richardcochran,
	Mark.Rutland, will, suzuki.poulose, Andre.Przywara, steven.price,
	linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve.Capper,
	justin.he, nd

On 2020-12-09 06:09, Jianyong Wu wrote:
> Currently, we offen use ntp (sync time with remote network clock)
> to sync time in VM. But the precision of ntp is subject to network 
> delay
> so it's difficult to sync time in a high precision.
> 
> kvm virtual ptp clock (ptp_kvm) offers another way to sync time in VM,
> as the remote clock locates in the host instead of remote network 
> clock.
> It targets to sync time between guest and host in virtualization
> environment and in this way, we can keep the time of all the VMs 
> running
> in the same host in sync. In general, the delay of communication 
> between
> host and guest is quiet small, so ptp_kvm can offer time sync precision
> up to in order of nanosecond. Please keep in mind that ptp_kvm just
> limits itself to be a channel which transmit the remote clock from
> host to guest and leaves the time sync jobs to an application, eg. 
> chrony,
> in usersapce in VM.
> 
> How ptp_kvm works:
> After ptp_kvm initialized, there will be a new device node under
> /dev called ptp%d. A guest userspace service, like chrony, can use this
> device to get host walltime, sometimes also counter cycle, which 
> depends
> on the service it calls. Then this guest userspace service can use 
> those
> data to do the time sync for guest.
> here is a rough sketch to show how kvm ptp clock works.
> 
> |----------------------------|              
> |--------------------------|
> |       guest userspace      |              |          host            
> |
> |ioctl -> /dev/ptp%d         |              |                          
> |
> |       ^   |                |              |                          
> |
> |----------------------------|              |                          
> |
> |       |   | guest kernel   |              |                          
> |
> |       |   V      (get host walltime/counter cycle)                   
> |
> |      ptp_kvm -> hypercall - - - - - - - - - - ->hypercall service    
> |
> |                         <- - - - - - - - - - - -                     
> |
> |----------------------------|              
> |--------------------------|
> 
> 1. time sync service in guest userspace call ptp device through 
> /dev/ptp%d.
> 2. ptp_kvm module in guest receives this request then invoke hypercall 
> to route
> into host kernel to request host walltime/counter cycle.
> 3. ptp_kvm hypercall service in host response to the request and send 
> data back.
> 4. ptp (not ptp_kvm) in guest copy the data to userspace.
> 
> This ptp_kvm implementation focuses itself to step 2 and 3 and step 2 
> works
> in guest comparing step 3 works in host kernel.

FWIW, and in order to speed up the review, I've posted a reworked
version[0] of this series with changes that address the comments
I had for on v16.

Thanks,

         M.

[0] https://lore.kernel.org/r/20210202141204.3134855-1-maz@kernel.org
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 19+ messages in thread

* RE: [PATCH v16 0/9] Enable ptp_kvm for arm/arm64
  2021-02-02 14:15 ` Marc Zyngier
@ 2021-02-03  2:40   ` Jianyong Wu
  0 siblings, 0 replies; 19+ messages in thread
From: Jianyong Wu @ 2021-02-03  2:40 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: netdev, yangbo.lu, john.stultz, tglx, pbonzini, richardcochran,
	Mark Rutland, will, Suzuki Poulose, Andre Przywara, Steven Price,
	linux-kernel, linux-arm-kernel, kvmarm, kvm, Steve Capper,
	Justin He, nd

Hi Marc,

> -----Original Message-----
> From: Marc Zyngier <maz@kernel.org>
> Sent: Tuesday, February 2, 2021 10:15 PM
> To: Jianyong Wu <Jianyong.Wu@arm.com>
> Cc: netdev@vger.kernel.org; yangbo.lu@nxp.com; john.stultz@linaro.org;
> tglx@linutronix.de; pbonzini@redhat.com; richardcochran@gmail.com; Mark
> Rutland <Mark.Rutland@arm.com>; will@kernel.org; Suzuki Poulose
> <Suzuki.Poulose@arm.com>; Andre Przywara <Andre.Przywara@arm.com>;
> Steven Price <Steven.Price@arm.com>; linux-kernel@vger.kernel.org; linux-
> arm-kernel@lists.infradead.org; kvmarm@lists.cs.columbia.edu;
> kvm@vger.kernel.org; Steve Capper <Steve.Capper@arm.com>; Justin He
> <Justin.He@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v16 0/9] Enable ptp_kvm for arm/arm64
> 
> On 2020-12-09 06:09, Jianyong Wu wrote:
> > Currently, we offen use ntp (sync time with remote network clock) to
> > sync time in VM. But the precision of ntp is subject to network delay
> > so it's difficult to sync time in a high precision.
> >
> > kvm virtual ptp clock (ptp_kvm) offers another way to sync time in VM,
> > as the remote clock locates in the host instead of remote network
> > clock.
> > It targets to sync time between guest and host in virtualization
> > environment and in this way, we can keep the time of all the VMs
> > running in the same host in sync. In general, the delay of
> > communication between host and guest is quiet small, so ptp_kvm can
> > offer time sync precision up to in order of nanosecond. Please keep in
> > mind that ptp_kvm just limits itself to be a channel which transmit
> > the remote clock from host to guest and leaves the time sync jobs to
> > an application, eg.
> > chrony,
> > in usersapce in VM.
> >
> > How ptp_kvm works:
> > After ptp_kvm initialized, there will be a new device node under /dev
> > called ptp%d. A guest userspace service, like chrony, can use this
> > device to get host walltime, sometimes also counter cycle, which
> > depends on the service it calls. Then this guest userspace service can
> > use those data to do the time sync for guest.
> > here is a rough sketch to show how kvm ptp clock works.
> >
> > |----------------------------|
> > |--------------------------|
> > |       guest userspace      |              |          host
> > |
> > |ioctl -> /dev/ptp%d         |              |
> > |
> > |       ^   |                |              |
> > |
> > |----------------------------|              |
> > |
> > |       |   | guest kernel   |              |
> > |
> > |       |   V      (get host walltime/counter cycle)
> > |
> > |      ptp_kvm -> hypercall - - - - - - - - - - ->hypercall service
> > |
> > |                         <- - - - - - - - - - - -
> > |
> > |----------------------------|
> > |--------------------------|
> >
> > 1. time sync service in guest userspace call ptp device through
> > /dev/ptp%d.
> > 2. ptp_kvm module in guest receives this request then invoke hypercall
> > to route into host kernel to request host walltime/counter cycle.
> > 3. ptp_kvm hypercall service in host response to the request and send
> > data back.
> > 4. ptp (not ptp_kvm) in guest copy the data to userspace.
> >
> > This ptp_kvm implementation focuses itself to step 2 and 3 and step 2
> > works in guest comparing step 3 works in host kernel.
> 
> FWIW, and in order to speed up the review, I've posted a reworked
> version[0] of this series with changes that address the comments I had for on
> v16.
> 

Great!!!
Good news for me, thanks Marc.

Thanks
Jianyong

> Thanks,
> 
>          M.
> 
> [0] https://lore.kernel.org/r/20210202141204.3134855-1-maz@kernel.org
> --
> Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2021-02-03  2:41 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-09  6:09 [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
2020-12-09  6:09 ` [PATCH v16 1/9] arm64: Probe for the presence of KVM hypervisor Jianyong Wu
2021-02-02 13:18   ` Marc Zyngier
2020-12-09  6:09 ` [PATCH v16 2/9] arm/arm64: KVM: Advertise KVM UID to guests via SMCCC Jianyong Wu
2020-12-09  6:09 ` [PATCH v16 3/9] ptp: Reorganize ptp_kvm module to make it arch-independent Jianyong Wu
2021-02-02 13:23   ` Marc Zyngier
2020-12-09  6:09 ` [PATCH v16 4/9] time: Add mechanism to recognize clocksource in time_get_snapshot Jianyong Wu
2020-12-09  6:09 ` [PATCH v16 5/9] clocksource: Add clocksource id for arm arch counter Jianyong Wu
2020-12-09  6:09 ` [PATCH v16 6/9] arm64/kvm: Add hypercall service for kvm ptp Jianyong Wu
2021-02-02 13:32   ` Marc Zyngier
2020-12-09  6:09 ` [PATCH v16 7/9] ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jianyong Wu
2021-02-02 13:37   ` Marc Zyngier
2020-12-09  6:09 ` [PATCH v16 8/9] doc: add ptp_kvm introduction for arm64 support Jianyong Wu
2021-02-02 13:43   ` Marc Zyngier
2020-12-09  6:09 ` [PATCH v16 9/9] arm64: Add kvm capability check extension for ptp_kvm Jianyong Wu
2021-02-02 13:44   ` Marc Zyngier
2021-01-06  8:29 ` [PATCH v16 0/9] Enable ptp_kvm for arm/arm64 Jianyong Wu
2021-02-02 14:15 ` Marc Zyngier
2021-02-03  2:40   ` Jianyong Wu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).