[Xen-devel] [PATCH for-next 0/7] Implement Hyper-V reference TSC based clock source

[Xen-devel] [PATCH for-next 0/7] Implement Hyper-V reference TSC based clock source

[Wei Liu]

Hi all

This series adds a clock source based on Hyper-V's reference TSC. The meat
is in the last patch.

With this series, Xen on Hyper-V no longer runs on emulated PIT.

(XEN) Platform timer is 2294.686MHz HYPER-V REFERENCE TSC

This series depends on [0].

Wei.

0: https://lists.xen.org/archives/html/xen-devel/2019-10/msg01420.html

Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>
Cc: Roger Pau Monné <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>

Wei Liu (7):
  x86: import hyperv-tlfs.h from Linux
  x86: fix up hyperv-tlfs.h
  x86/hyperv: extract more information from Hyper-V
  x86: add a comment regarding the location of hypervisor_probe
  x86: use running_on_hypervisor to gate hypervisor_setup
  x86/hyperv: provide hyperv_guest variable
  x86: implement Hyper-V clock source

 xen/arch/x86/guest/hyperv/hyperv.c      |  17 +
 xen/arch/x86/setup.c                    |   6 +-
 xen/arch/x86/time.c                     |  87 +++
 xen/include/asm-x86/guest/hyperv-tlfs.h | 907 ++++++++++++++++++++++++
 xen/include/asm-x86/guest/hyperv.h      |  14 +
 5 files changed, 1030 insertions(+), 1 deletion(-)
 create mode 100644 xen/include/asm-x86/guest/hyperv-tlfs.h

-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Wei Liu]

Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.

This is a pristine copy from Linux. It is not used yet and probably
doesn't compile. Changes to make it work will come later.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
 xen/include/asm-x86/guest/hyperv-tlfs.h | 906 ++++++++++++++++++++++++
 1 file changed, 906 insertions(+)
 create mode 100644 xen/include/asm-x86/guest/hyperv-tlfs.h

diff --git a/xen/include/asm-x86/guest/hyperv-tlfs.h b/xen/include/asm-x86/guest/hyperv-tlfs.h
new file mode 100644
index 0000000000..7741e211f7
--- /dev/null
+++ b/xen/include/asm-x86/guest/hyperv-tlfs.h
@@ -0,0 +1,906 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * This file contains definitions from Hyper-V Hypervisor Top-Level Functional
+ * Specification (TLFS):
+ * https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs
+ */
+
+#ifndef _ASM_X86_HYPERV_TLFS_H
+#define _ASM_X86_HYPERV_TLFS_H
+
+#include <linux/types.h>
+#include <asm/page.h>
+
+/*
+ * While not explicitly listed in the TLFS, Hyper-V always runs with a page size
+ * of 4096. These definitions are used when communicating with Hyper-V using
+ * guest physical pages and guest physical page addresses, since the guest page
+ * size may not be 4096 on all architectures.
+ */
+#define HV_HYP_PAGE_SHIFT      12
+#define HV_HYP_PAGE_SIZE       BIT(HV_HYP_PAGE_SHIFT)
+#define HV_HYP_PAGE_MASK       (~(HV_HYP_PAGE_SIZE - 1))
+
+/*
+ * The below CPUID leaves are present if VersionAndFeatures.HypervisorPresent
+ * is set by CPUID(HvCpuIdFunctionVersionAndFeatures).
+ */
+#define HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS	0x40000000
+#define HYPERV_CPUID_INTERFACE			0x40000001
+#define HYPERV_CPUID_VERSION			0x40000002
+#define HYPERV_CPUID_FEATURES			0x40000003
+#define HYPERV_CPUID_ENLIGHTMENT_INFO		0x40000004
+#define HYPERV_CPUID_IMPLEMENT_LIMITS		0x40000005
+#define HYPERV_CPUID_NESTED_FEATURES		0x4000000A
+
+#define HYPERV_HYPERVISOR_PRESENT_BIT		0x80000000
+#define HYPERV_CPUID_MIN			0x40000005
+#define HYPERV_CPUID_MAX			0x4000ffff
+
+/*
+ * Feature identification. EAX indicates which features are available
+ * to the partition based upon the current partition privileges.
+ * These are HYPERV_CPUID_FEATURES.EAX bits.
+ */
+
+/* VP Runtime (HV_X64_MSR_VP_RUNTIME) available */
+#define HV_X64_MSR_VP_RUNTIME_AVAILABLE		BIT(0)
+/* Partition Reference Counter (HV_X64_MSR_TIME_REF_COUNT) available*/
+#define HV_MSR_TIME_REF_COUNT_AVAILABLE		BIT(1)
+/*
+ * Basic SynIC MSRs (HV_X64_MSR_SCONTROL through HV_X64_MSR_EOM
+ * and HV_X64_MSR_SINT0 through HV_X64_MSR_SINT15) available
+ */
+#define HV_X64_MSR_SYNIC_AVAILABLE		BIT(2)
+/*
+ * Synthetic Timer MSRs (HV_X64_MSR_STIMER0_CONFIG through
+ * HV_X64_MSR_STIMER3_COUNT) available
+ */
+#define HV_MSR_SYNTIMER_AVAILABLE		BIT(3)
+/*
+ * APIC access MSRs (HV_X64_MSR_EOI, HV_X64_MSR_ICR and HV_X64_MSR_TPR)
+ * are available
+ */
+#define HV_X64_MSR_APIC_ACCESS_AVAILABLE	BIT(4)
+/* Hypercall MSRs (HV_X64_MSR_GUEST_OS_ID and HV_X64_MSR_HYPERCALL) available*/
+#define HV_X64_MSR_HYPERCALL_AVAILABLE		BIT(5)
+/* Access virtual processor index MSR (HV_X64_MSR_VP_INDEX) available*/
+#define HV_X64_MSR_VP_INDEX_AVAILABLE		BIT(6)
+/* Virtual system reset MSR (HV_X64_MSR_RESET) is available*/
+#define HV_X64_MSR_RESET_AVAILABLE		BIT(7)
+/*
+ * Access statistics pages MSRs (HV_X64_MSR_STATS_PARTITION_RETAIL_PAGE,
+ * HV_X64_MSR_STATS_PARTITION_INTERNAL_PAGE, HV_X64_MSR_STATS_VP_RETAIL_PAGE,
+ * HV_X64_MSR_STATS_VP_INTERNAL_PAGE) available
+ */
+#define HV_X64_MSR_STAT_PAGES_AVAILABLE		BIT(8)
+/* Partition reference TSC MSR is available */
+#define HV_MSR_REFERENCE_TSC_AVAILABLE		BIT(9)
+/* Partition Guest IDLE MSR is available */
+#define HV_X64_MSR_GUEST_IDLE_AVAILABLE		BIT(10)
+/*
+ * There is a single feature flag that signifies if the partition has access
+ * to MSRs with local APIC and TSC frequencies.
+ */
+#define HV_X64_ACCESS_FREQUENCY_MSRS		BIT(11)
+/* AccessReenlightenmentControls privilege */
+#define HV_X64_ACCESS_REENLIGHTENMENT		BIT(13)
+
+/*
+ * Feature identification: indicates which flags were specified at partition
+ * creation. The format is the same as the partition creation flag structure
+ * defined in section Partition Creation Flags.
+ * These are HYPERV_CPUID_FEATURES.EBX bits.
+ */
+#define HV_X64_CREATE_PARTITIONS		BIT(0)
+#define HV_X64_ACCESS_PARTITION_ID		BIT(1)
+#define HV_X64_ACCESS_MEMORY_POOL		BIT(2)
+#define HV_X64_ADJUST_MESSAGE_BUFFERS		BIT(3)
+#define HV_X64_POST_MESSAGES			BIT(4)
+#define HV_X64_SIGNAL_EVENTS			BIT(5)
+#define HV_X64_CREATE_PORT			BIT(6)
+#define HV_X64_CONNECT_PORT			BIT(7)
+#define HV_X64_ACCESS_STATS			BIT(8)
+#define HV_X64_DEBUGGING			BIT(11)
+#define HV_X64_CPU_POWER_MANAGEMENT		BIT(12)
+
+/*
+ * Feature identification. EDX indicates which miscellaneous features
+ * are available to the partition.
+ * These are HYPERV_CPUID_FEATURES.EDX bits.
+ */
+/* The MWAIT instruction is available (per section MONITOR / MWAIT) */
+#define HV_X64_MWAIT_AVAILABLE				BIT(0)
+/* Guest debugging support is available */
+#define HV_X64_GUEST_DEBUGGING_AVAILABLE		BIT(1)
+/* Performance Monitor support is available*/
+#define HV_X64_PERF_MONITOR_AVAILABLE			BIT(2)
+/* Support for physical CPU dynamic partitioning events is available*/
+#define HV_X64_CPU_DYNAMIC_PARTITIONING_AVAILABLE	BIT(3)
+/*
+ * Support for passing hypercall input parameter block via XMM
+ * registers is available
+ */
+#define HV_X64_HYPERCALL_PARAMS_XMM_AVAILABLE		BIT(4)
+/* Support for a virtual guest idle state is available */
+#define HV_X64_GUEST_IDLE_STATE_AVAILABLE		BIT(5)
+/* Frequency MSRs available */
+#define HV_FEATURE_FREQUENCY_MSRS_AVAILABLE		BIT(8)
+/* Crash MSR available */
+#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE		BIT(10)
+/* stimer Direct Mode is available */
+#define HV_STIMER_DIRECT_MODE_AVAILABLE			BIT(19)
+
+/*
+ * Implementation recommendations. Indicates which behaviors the hypervisor
+ * recommends the OS implement for optimal performance.
+ * These are HYPERV_CPUID_ENLIGHTMENT_INFO.EAX bits.
+ */
+/*
+ * Recommend using hypercall for address space switches rather
+ * than MOV to CR3 instruction
+ */
+#define HV_X64_AS_SWITCH_RECOMMENDED			BIT(0)
+/* Recommend using hypercall for local TLB flushes rather
+ * than INVLPG or MOV to CR3 instructions */
+#define HV_X64_LOCAL_TLB_FLUSH_RECOMMENDED		BIT(1)
+/*
+ * Recommend using hypercall for remote TLB flushes rather
+ * than inter-processor interrupts
+ */
+#define HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED		BIT(2)
+/*
+ * Recommend using MSRs for accessing APIC registers
+ * EOI, ICR and TPR rather than their memory-mapped counterparts
+ */
+#define HV_X64_APIC_ACCESS_RECOMMENDED			BIT(3)
+/* Recommend using the hypervisor-provided MSR to initiate a system RESET */
+#define HV_X64_SYSTEM_RESET_RECOMMENDED			BIT(4)
+/*
+ * Recommend using relaxed timing for this partition. If used,
+ * the VM should disable any watchdog timeouts that rely on the
+ * timely delivery of external interrupts
+ */
+#define HV_X64_RELAXED_TIMING_RECOMMENDED		BIT(5)
+
+/*
+ * Recommend not using Auto End-Of-Interrupt feature
+ */
+#define HV_DEPRECATING_AEOI_RECOMMENDED			BIT(9)
+
+/*
+ * Recommend using cluster IPI hypercalls.
+ */
+#define HV_X64_CLUSTER_IPI_RECOMMENDED			BIT(10)
+
+/* Recommend using the newer ExProcessorMasks interface */
+#define HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED		BIT(11)
+
+/* Recommend using enlightened VMCS */
+#define HV_X64_ENLIGHTENED_VMCS_RECOMMENDED		BIT(14)
+
+/*
+ * Virtual processor will never share a physical core with another virtual
+ * processor, except for virtual processors that are reported as sibling SMT
+ * threads.
+ */
+#define HV_X64_NO_NONARCH_CORESHARING                  BIT(18)
+
+/* Nested features. These are HYPERV_CPUID_NESTED_FEATURES.EAX bits. */
+#define HV_X64_NESTED_DIRECT_FLUSH			BIT(17)
+#define HV_X64_NESTED_GUEST_MAPPING_FLUSH		BIT(18)
+#define HV_X64_NESTED_MSR_BITMAP			BIT(19)
+
+/* Hyper-V specific model specific registers (MSRs) */
+
+/* MSR used to identify the guest OS. */
+#define HV_X64_MSR_GUEST_OS_ID			0x40000000
+
+/* MSR used to setup pages used to communicate with the hypervisor. */
+#define HV_X64_MSR_HYPERCALL			0x40000001
+
+/* MSR used to provide vcpu index */
+#define HV_X64_MSR_VP_INDEX			0x40000002
+
+/* MSR used to reset the guest OS. */
+#define HV_X64_MSR_RESET			0x40000003
+
+/* MSR used to provide vcpu runtime in 100ns units */
+#define HV_X64_MSR_VP_RUNTIME			0x40000010
+
+/* MSR used to read the per-partition time reference counter */
+#define HV_X64_MSR_TIME_REF_COUNT		0x40000020
+
+/* A partition's reference time stamp counter (TSC) page */
+#define HV_X64_MSR_REFERENCE_TSC		0x40000021
+
+/* MSR used to retrieve the TSC frequency */
+#define HV_X64_MSR_TSC_FREQUENCY		0x40000022
+
+/* MSR used to retrieve the local APIC timer frequency */
+#define HV_X64_MSR_APIC_FREQUENCY		0x40000023
+
+/* Define the virtual APIC registers */
+#define HV_X64_MSR_EOI				0x40000070
+#define HV_X64_MSR_ICR				0x40000071
+#define HV_X64_MSR_TPR				0x40000072
+#define HV_X64_MSR_VP_ASSIST_PAGE		0x40000073
+
+/* Define synthetic interrupt controller model specific registers. */
+#define HV_X64_MSR_SCONTROL			0x40000080
+#define HV_X64_MSR_SVERSION			0x40000081
+#define HV_X64_MSR_SIEFP			0x40000082
+#define HV_X64_MSR_SIMP				0x40000083
+#define HV_X64_MSR_EOM				0x40000084
+#define HV_X64_MSR_SINT0			0x40000090
+#define HV_X64_MSR_SINT1			0x40000091
+#define HV_X64_MSR_SINT2			0x40000092
+#define HV_X64_MSR_SINT3			0x40000093
+#define HV_X64_MSR_SINT4			0x40000094
+#define HV_X64_MSR_SINT5			0x40000095
+#define HV_X64_MSR_SINT6			0x40000096
+#define HV_X64_MSR_SINT7			0x40000097
+#define HV_X64_MSR_SINT8			0x40000098
+#define HV_X64_MSR_SINT9			0x40000099
+#define HV_X64_MSR_SINT10			0x4000009A
+#define HV_X64_MSR_SINT11			0x4000009B
+#define HV_X64_MSR_SINT12			0x4000009C
+#define HV_X64_MSR_SINT13			0x4000009D
+#define HV_X64_MSR_SINT14			0x4000009E
+#define HV_X64_MSR_SINT15			0x4000009F
+
+/*
+ * Synthetic Timer MSRs. Four timers per vcpu.
+ */
+#define HV_X64_MSR_STIMER0_CONFIG		0x400000B0
+#define HV_X64_MSR_STIMER0_COUNT		0x400000B1
+#define HV_X64_MSR_STIMER1_CONFIG		0x400000B2
+#define HV_X64_MSR_STIMER1_COUNT		0x400000B3
+#define HV_X64_MSR_STIMER2_CONFIG		0x400000B4
+#define HV_X64_MSR_STIMER2_COUNT		0x400000B5
+#define HV_X64_MSR_STIMER3_CONFIG		0x400000B6
+#define HV_X64_MSR_STIMER3_COUNT		0x400000B7
+
+/* Hyper-V guest idle MSR */
+#define HV_X64_MSR_GUEST_IDLE			0x400000F0
+
+/* Hyper-V guest crash notification MSR's */
+#define HV_X64_MSR_CRASH_P0			0x40000100
+#define HV_X64_MSR_CRASH_P1			0x40000101
+#define HV_X64_MSR_CRASH_P2			0x40000102
+#define HV_X64_MSR_CRASH_P3			0x40000103
+#define HV_X64_MSR_CRASH_P4			0x40000104
+#define HV_X64_MSR_CRASH_CTL			0x40000105
+
+/* TSC emulation after migration */
+#define HV_X64_MSR_REENLIGHTENMENT_CONTROL	0x40000106
+#define HV_X64_MSR_TSC_EMULATION_CONTROL	0x40000107
+#define HV_X64_MSR_TSC_EMULATION_STATUS		0x40000108
+
+/*
+ * Declare the MSR used to setup pages used to communicate with the hypervisor.
+ */
+union hv_x64_msr_hypercall_contents {
+	u64 as_uint64;
+	struct {
+		u64 enable:1;
+		u64 reserved:11;
+		u64 guest_physical_address:52;
+	} __packed;
+};
+
+/*
+ * TSC page layout.
+ */
+struct ms_hyperv_tsc_page {
+	volatile u32 tsc_sequence;
+	u32 reserved1;
+	volatile u64 tsc_scale;
+	volatile s64 tsc_offset;
+	u64 reserved2[509];
+}  __packed;
+
+/*
+ * The guest OS needs to register the guest ID with the hypervisor.
+ * The guest ID is a 64 bit entity and the structure of this ID is
+ * specified in the Hyper-V specification:
+ *
+ * msdn.microsoft.com/en-us/library/windows/hardware/ff542653%28v=vs.85%29.aspx
+ *
+ * While the current guideline does not specify how Linux guest ID(s)
+ * need to be generated, our plan is to publish the guidelines for
+ * Linux and other guest operating systems that currently are hosted
+ * on Hyper-V. The implementation here conforms to this yet
+ * unpublished guidelines.
+ *
+ *
+ * Bit(s)
+ * 63 - Indicates if the OS is Open Source or not; 1 is Open Source
+ * 62:56 - Os Type; Linux is 0x100
+ * 55:48 - Distro specific identification
+ * 47:16 - Linux kernel version number
+ * 15:0  - Distro specific identification
+ *
+ *
+ */
+
+#define HV_LINUX_VENDOR_ID              0x8100
+
+struct hv_reenlightenment_control {
+	__u64 vector:8;
+	__u64 reserved1:8;
+	__u64 enabled:1;
+	__u64 reserved2:15;
+	__u64 target_vp:32;
+}  __packed;
+
+struct hv_tsc_emulation_control {
+	__u64 enabled:1;
+	__u64 reserved:63;
+} __packed;
+
+struct hv_tsc_emulation_status {
+	__u64 inprogress:1;
+	__u64 reserved:63;
+} __packed;
+
+#define HV_X64_MSR_HYPERCALL_ENABLE		0x00000001
+#define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT	12
+#define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK	\
+		(~((1ull << HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
+
+/*
+ * Crash notification (HV_X64_MSR_CRASH_CTL) flags.
+ */
+#define HV_CRASH_CTL_CRASH_NOTIFY_MSG		BIT_ULL(62)
+#define HV_CRASH_CTL_CRASH_NOTIFY		BIT_ULL(63)
+#define HV_X64_MSR_CRASH_PARAMS		\
+		(1 + (HV_X64_MSR_CRASH_P4 - HV_X64_MSR_CRASH_P0))
+
+#define HV_IPI_LOW_VECTOR	0x10
+#define HV_IPI_HIGH_VECTOR	0xff
+
+/* Declare the various hypercall operations. */
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
+#define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
+#define HVCALL_SEND_IPI				0x000b
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX  0x0013
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX   0x0014
+#define HVCALL_SEND_IPI_EX			0x0015
+#define HVCALL_POST_MESSAGE			0x005c
+#define HVCALL_SIGNAL_EVENT			0x005d
+#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
+#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
+
+#define HV_X64_MSR_VP_ASSIST_PAGE_ENABLE	0x00000001
+#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT	12
+#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK	\
+		(~((1ull << HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT) - 1))
+
+/* Hyper-V Enlightened VMCS version mask in nested features CPUID */
+#define HV_X64_ENLIGHTENED_VMCS_VERSION		0xff
+
+#define HV_X64_MSR_TSC_REFERENCE_ENABLE		0x00000001
+#define HV_X64_MSR_TSC_REFERENCE_ADDRESS_SHIFT	12
+
+#define HV_PROCESSOR_POWER_STATE_C0		0
+#define HV_PROCESSOR_POWER_STATE_C1		1
+#define HV_PROCESSOR_POWER_STATE_C2		2
+#define HV_PROCESSOR_POWER_STATE_C3		3
+
+#define HV_FLUSH_ALL_PROCESSORS			BIT(0)
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	BIT(1)
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	BIT(2)
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	BIT(3)
+
+enum HV_GENERIC_SET_FORMAT {
+	HV_GENERIC_SET_SPARSE_4K,
+	HV_GENERIC_SET_ALL,
+};
+
+#define HV_HYPERCALL_RESULT_MASK	GENMASK_ULL(15, 0)
+#define HV_HYPERCALL_FAST_BIT		BIT(16)
+#define HV_HYPERCALL_VARHEAD_OFFSET	17
+#define HV_HYPERCALL_REP_COMP_OFFSET	32
+#define HV_HYPERCALL_REP_COMP_MASK	GENMASK_ULL(43, 32)
+#define HV_HYPERCALL_REP_START_OFFSET	48
+#define HV_HYPERCALL_REP_START_MASK	GENMASK_ULL(59, 48)
+
+/* hypercall status code */
+#define HV_STATUS_SUCCESS			0
+#define HV_STATUS_INVALID_HYPERCALL_CODE	2
+#define HV_STATUS_INVALID_HYPERCALL_INPUT	3
+#define HV_STATUS_INVALID_ALIGNMENT		4
+#define HV_STATUS_INVALID_PARAMETER		5
+#define HV_STATUS_INSUFFICIENT_MEMORY		11
+#define HV_STATUS_INVALID_PORT_ID		17
+#define HV_STATUS_INVALID_CONNECTION_ID		18
+#define HV_STATUS_INSUFFICIENT_BUFFERS		19
+
+/*
+ * The Hyper-V TimeRefCount register and the TSC
+ * page provide a guest VM clock with 100ns tick rate
+ */
+#define HV_CLOCK_HZ (NSEC_PER_SEC/100)
+
+typedef struct _HV_REFERENCE_TSC_PAGE {
+	__u32 tsc_sequence;
+	__u32 res1;
+	__u64 tsc_scale;
+	__s64 tsc_offset;
+}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
+
+/* Define the number of synthetic interrupt sources. */
+#define HV_SYNIC_SINT_COUNT		(16)
+/* Define the expected SynIC version. */
+#define HV_SYNIC_VERSION_1		(0x1)
+/* Valid SynIC vectors are 16-255. */
+#define HV_SYNIC_FIRST_VALID_VECTOR	(16)
+
+#define HV_SYNIC_CONTROL_ENABLE		(1ULL << 0)
+#define HV_SYNIC_SIMP_ENABLE		(1ULL << 0)
+#define HV_SYNIC_SIEFP_ENABLE		(1ULL << 0)
+#define HV_SYNIC_SINT_MASKED		(1ULL << 16)
+#define HV_SYNIC_SINT_AUTO_EOI		(1ULL << 17)
+#define HV_SYNIC_SINT_VECTOR_MASK	(0xFF)
+
+#define HV_SYNIC_STIMER_COUNT		(4)
+
+/* Define synthetic interrupt controller message constants. */
+#define HV_MESSAGE_SIZE			(256)
+#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
+#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
+
+/* Define hypervisor message types. */
+enum hv_message_type {
+	HVMSG_NONE			= 0x00000000,
+
+	/* Memory access messages. */
+	HVMSG_UNMAPPED_GPA		= 0x80000000,
+	HVMSG_GPA_INTERCEPT		= 0x80000001,
+
+	/* Timer notification messages. */
+	HVMSG_TIMER_EXPIRED			= 0x80000010,
+
+	/* Error messages. */
+	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
+	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
+	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
+
+	/* Trace buffer complete messages. */
+	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
+
+	/* Platform-specific processor intercept messages. */
+	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
+	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
+	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
+	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
+	HVMSG_X64_APIC_EOI			= 0x80010004,
+	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
+};
+
+/* Define synthetic interrupt controller message flags. */
+union hv_message_flags {
+	__u8 asu8;
+	struct {
+		__u8 msg_pending:1;
+		__u8 reserved:7;
+	} __packed;
+};
+
+/* Define port identifier type. */
+union hv_port_id {
+	__u32 asu32;
+	struct {
+		__u32 id:24;
+		__u32 reserved:8;
+	} __packed u;
+};
+
+/* Define synthetic interrupt controller message header. */
+struct hv_message_header {
+	__u32 message_type;
+	__u8 payload_size;
+	union hv_message_flags message_flags;
+	__u8 reserved[2];
+	union {
+		__u64 sender;
+		union hv_port_id port;
+	};
+} __packed;
+
+/* Define synthetic interrupt controller message format. */
+struct hv_message {
+	struct hv_message_header header;
+	union {
+		__u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
+	} u;
+} __packed;
+
+/* Define the synthetic interrupt message page layout. */
+struct hv_message_page {
+	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
+} __packed;
+
+/* Define timer message payload structure. */
+struct hv_timer_message_payload {
+	__u32 timer_index;
+	__u32 reserved;
+	__u64 expiration_time;	/* When the timer expired */
+	__u64 delivery_time;	/* When the message was delivered */
+} __packed;
+
+struct hv_nested_enlightenments_control {
+	struct {
+		__u32 directhypercall:1;
+		__u32 reserved:31;
+	} features;
+	struct {
+		__u32 reserved;
+	} hypercallControls;
+} __packed;
+
+/* Define virtual processor assist page structure. */
+struct hv_vp_assist_page {
+	__u32 apic_assist;
+	__u32 reserved1;
+	__u64 vtl_control[3];
+	struct hv_nested_enlightenments_control nested_control;
+	__u8 enlighten_vmentry;
+	__u8 reserved2[7];
+	__u64 current_nested_vmcs;
+} __packed;
+
+struct hv_enlightened_vmcs {
+	u32 revision_id;
+	u32 abort;
+
+	u16 host_es_selector;
+	u16 host_cs_selector;
+	u16 host_ss_selector;
+	u16 host_ds_selector;
+	u16 host_fs_selector;
+	u16 host_gs_selector;
+	u16 host_tr_selector;
+
+	u16 padding16_1;
+
+	u64 host_ia32_pat;
+	u64 host_ia32_efer;
+
+	u64 host_cr0;
+	u64 host_cr3;
+	u64 host_cr4;
+
+	u64 host_ia32_sysenter_esp;
+	u64 host_ia32_sysenter_eip;
+	u64 host_rip;
+	u32 host_ia32_sysenter_cs;
+
+	u32 pin_based_vm_exec_control;
+	u32 vm_exit_controls;
+	u32 secondary_vm_exec_control;
+
+	u64 io_bitmap_a;
+	u64 io_bitmap_b;
+	u64 msr_bitmap;
+
+	u16 guest_es_selector;
+	u16 guest_cs_selector;
+	u16 guest_ss_selector;
+	u16 guest_ds_selector;
+	u16 guest_fs_selector;
+	u16 guest_gs_selector;
+	u16 guest_ldtr_selector;
+	u16 guest_tr_selector;
+
+	u32 guest_es_limit;
+	u32 guest_cs_limit;
+	u32 guest_ss_limit;
+	u32 guest_ds_limit;
+	u32 guest_fs_limit;
+	u32 guest_gs_limit;
+	u32 guest_ldtr_limit;
+	u32 guest_tr_limit;
+	u32 guest_gdtr_limit;
+	u32 guest_idtr_limit;
+
+	u32 guest_es_ar_bytes;
+	u32 guest_cs_ar_bytes;
+	u32 guest_ss_ar_bytes;
+	u32 guest_ds_ar_bytes;
+	u32 guest_fs_ar_bytes;
+	u32 guest_gs_ar_bytes;
+	u32 guest_ldtr_ar_bytes;
+	u32 guest_tr_ar_bytes;
+
+	u64 guest_es_base;
+	u64 guest_cs_base;
+	u64 guest_ss_base;
+	u64 guest_ds_base;
+	u64 guest_fs_base;
+	u64 guest_gs_base;
+	u64 guest_ldtr_base;
+	u64 guest_tr_base;
+	u64 guest_gdtr_base;
+	u64 guest_idtr_base;
+
+	u64 padding64_1[3];
+
+	u64 vm_exit_msr_store_addr;
+	u64 vm_exit_msr_load_addr;
+	u64 vm_entry_msr_load_addr;
+
+	u64 cr3_target_value0;
+	u64 cr3_target_value1;
+	u64 cr3_target_value2;
+	u64 cr3_target_value3;
+
+	u32 page_fault_error_code_mask;
+	u32 page_fault_error_code_match;
+
+	u32 cr3_target_count;
+	u32 vm_exit_msr_store_count;
+	u32 vm_exit_msr_load_count;
+	u32 vm_entry_msr_load_count;
+
+	u64 tsc_offset;
+	u64 virtual_apic_page_addr;
+	u64 vmcs_link_pointer;
+
+	u64 guest_ia32_debugctl;
+	u64 guest_ia32_pat;
+	u64 guest_ia32_efer;
+
+	u64 guest_pdptr0;
+	u64 guest_pdptr1;
+	u64 guest_pdptr2;
+	u64 guest_pdptr3;
+
+	u64 guest_pending_dbg_exceptions;
+	u64 guest_sysenter_esp;
+	u64 guest_sysenter_eip;
+
+	u32 guest_activity_state;
+	u32 guest_sysenter_cs;
+
+	u64 cr0_guest_host_mask;
+	u64 cr4_guest_host_mask;
+	u64 cr0_read_shadow;
+	u64 cr4_read_shadow;
+	u64 guest_cr0;
+	u64 guest_cr3;
+	u64 guest_cr4;
+	u64 guest_dr7;
+
+	u64 host_fs_base;
+	u64 host_gs_base;
+	u64 host_tr_base;
+	u64 host_gdtr_base;
+	u64 host_idtr_base;
+	u64 host_rsp;
+
+	u64 ept_pointer;
+
+	u16 virtual_processor_id;
+	u16 padding16_2[3];
+
+	u64 padding64_2[5];
+	u64 guest_physical_address;
+
+	u32 vm_instruction_error;
+	u32 vm_exit_reason;
+	u32 vm_exit_intr_info;
+	u32 vm_exit_intr_error_code;
+	u32 idt_vectoring_info_field;
+	u32 idt_vectoring_error_code;
+	u32 vm_exit_instruction_len;
+	u32 vmx_instruction_info;
+
+	u64 exit_qualification;
+	u64 exit_io_instruction_ecx;
+	u64 exit_io_instruction_esi;
+	u64 exit_io_instruction_edi;
+	u64 exit_io_instruction_eip;
+
+	u64 guest_linear_address;
+	u64 guest_rsp;
+	u64 guest_rflags;
+
+	u32 guest_interruptibility_info;
+	u32 cpu_based_vm_exec_control;
+	u32 exception_bitmap;
+	u32 vm_entry_controls;
+	u32 vm_entry_intr_info_field;
+	u32 vm_entry_exception_error_code;
+	u32 vm_entry_instruction_len;
+	u32 tpr_threshold;
+
+	u64 guest_rip;
+
+	u32 hv_clean_fields;
+	u32 hv_padding_32;
+	u32 hv_synthetic_controls;
+	struct {
+		u32 nested_flush_hypercall:1;
+		u32 msr_bitmap:1;
+		u32 reserved:30;
+	}  __packed hv_enlightenments_control;
+	u32 hv_vp_id;
+
+	u64 hv_vm_id;
+	u64 partition_assist_page;
+	u64 padding64_4[4];
+	u64 guest_bndcfgs;
+	u64 padding64_5[7];
+	u64 xss_exit_bitmap;
+	u64 padding64_6[7];
+} __packed;
+
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE			0
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP		BIT(0)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP		BIT(1)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2		BIT(2)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1		BIT(3)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC		BIT(4)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EVENT		BIT(5)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_ENTRY		BIT(6)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EXCPN		BIT(7)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CRDR			BIT(8)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_XLAT		BIT(9)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC		BIT(10)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP1		BIT(11)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP2		BIT(12)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER		BIT(13)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_GRP1		BIT(14)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL	BIT(15)
+
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL			0xFFFF
+
+/* Define synthetic interrupt controller flag constants. */
+#define HV_EVENT_FLAGS_COUNT		(256 * 8)
+#define HV_EVENT_FLAGS_LONG_COUNT	(256 / sizeof(unsigned long))
+
+/*
+ * Synthetic timer configuration.
+ */
+union hv_stimer_config {
+	u64 as_uint64;
+	struct {
+		u64 enable:1;
+		u64 periodic:1;
+		u64 lazy:1;
+		u64 auto_enable:1;
+		u64 apic_vector:8;
+		u64 direct_mode:1;
+		u64 reserved_z0:3;
+		u64 sintx:4;
+		u64 reserved_z1:44;
+	} __packed;
+};
+
+
+/* Define the synthetic interrupt controller event flags format. */
+union hv_synic_event_flags {
+	unsigned long flags[HV_EVENT_FLAGS_LONG_COUNT];
+};
+
+/* Define SynIC control register. */
+union hv_synic_scontrol {
+	u64 as_uint64;
+	struct {
+		u64 enable:1;
+		u64 reserved:63;
+	} __packed;
+};
+
+/* Define synthetic interrupt source. */
+union hv_synic_sint {
+	u64 as_uint64;
+	struct {
+		u64 vector:8;
+		u64 reserved1:8;
+		u64 masked:1;
+		u64 auto_eoi:1;
+		u64 reserved2:46;
+	} __packed;
+};
+
+/* Define the format of the SIMP register */
+union hv_synic_simp {
+	u64 as_uint64;
+	struct {
+		u64 simp_enabled:1;
+		u64 preserved:11;
+		u64 base_simp_gpa:52;
+	} __packed;
+};
+
+/* Define the format of the SIEFP register */
+union hv_synic_siefp {
+	u64 as_uint64;
+	struct {
+		u64 siefp_enabled:1;
+		u64 preserved:11;
+		u64 base_siefp_gpa:52;
+	} __packed;
+};
+
+struct hv_vpset {
+	u64 format;
+	u64 valid_bank_mask;
+	u64 bank_contents[];
+} __packed;
+
+/* HvCallSendSyntheticClusterIpi hypercall */
+struct hv_send_ipi {
+	u32 vector;
+	u32 reserved;
+	u64 cpu_mask;
+} __packed;
+
+/* HvCallSendSyntheticClusterIpiEx hypercall */
+struct hv_send_ipi_ex {
+	u32 vector;
+	u32 reserved;
+	struct hv_vpset vp_set;
+} __packed;
+
+/* HvFlushGuestPhysicalAddressSpace hypercalls */
+struct hv_guest_mapping_flush {
+	u64 address_space;
+	u64 flags;
+} __packed;
+
+/*
+ *  HV_MAX_FLUSH_PAGES = "additional_pages" + 1. It's limited
+ *  by the bitwidth of "additional_pages" in union hv_gpa_page_range.
+ */
+#define HV_MAX_FLUSH_PAGES (2048)
+
+/* HvFlushGuestPhysicalAddressList hypercall */
+union hv_gpa_page_range {
+	u64 address_space;
+	struct {
+		u64 additional_pages:11;
+		u64 largepage:1;
+		u64 basepfn:52;
+	} page;
+};
+
+/*
+ * All input flush parameters should be in single page. The max flush
+ * count is equal with how many entries of union hv_gpa_page_range can
+ * be populated into the input parameter page.
+ */
+#define HV_MAX_FLUSH_REP_COUNT ((HV_HYP_PAGE_SIZE - 2 * sizeof(u64)) /	\
+				sizeof(union hv_gpa_page_range))
+
+struct hv_guest_mapping_flush_list {
+	u64 address_space;
+	u64 flags;
+	union hv_gpa_page_range gpa_list[HV_MAX_FLUSH_REP_COUNT];
+};
+
+/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */
+struct hv_tlb_flush {
+	u64 address_space;
+	u64 flags;
+	u64 processor_mask;
+	u64 gva_list[];
+} __packed;
+
+/* HvFlushVirtualAddressSpaceEx, HvFlushVirtualAddressListEx hypercalls */
+struct hv_tlb_flush_ex {
+	u64 address_space;
+	u64 flags;
+	struct hv_vpset hv_vp_set;
+	u64 gva_list[];
+} __packed;
+
+struct hv_partition_assist_pg {
+	u32 tlb_lock_count;
+};
+#endif
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 2/7] x86: fix up hyperv-tlfs.h

[Wei Liu]

Do the following:
1. include xen/types.h and xen/bitops.h
2. fix up invocations of BIT macro

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
This can be squashed into previous patch if preferred.
---
 xen/include/asm-x86/guest/hyperv-tlfs.h | 141 ++++++++++++------------
 1 file changed, 71 insertions(+), 70 deletions(-)

diff --git a/xen/include/asm-x86/guest/hyperv-tlfs.h b/xen/include/asm-x86/guest/hyperv-tlfs.h
index 7741e211f7..ccd9850b27 100644
--- a/xen/include/asm-x86/guest/hyperv-tlfs.h
+++ b/xen/include/asm-x86/guest/hyperv-tlfs.h
@@ -9,7 +9,8 @@
 #ifndef _ASM_X86_HYPERV_TLFS_H
 #define _ASM_X86_HYPERV_TLFS_H
 
-#include <linux/types.h>
+#include <xen/bitops.h>
+#include <xen/types.h>
 #include <asm/page.h>
 
 /*
@@ -19,7 +20,7 @@
  * size may not be 4096 on all architectures.
  */
 #define HV_HYP_PAGE_SHIFT      12
-#define HV_HYP_PAGE_SIZE       BIT(HV_HYP_PAGE_SHIFT)
+#define HV_HYP_PAGE_SIZE       BIT(HV_HYP_PAGE_SHIFT, UL)
 #define HV_HYP_PAGE_MASK       (~(HV_HYP_PAGE_SIZE - 1))
 
 /*
@@ -45,47 +46,47 @@
  */
 
 /* VP Runtime (HV_X64_MSR_VP_RUNTIME) available */
-#define HV_X64_MSR_VP_RUNTIME_AVAILABLE		BIT(0)
+#define HV_X64_MSR_VP_RUNTIME_AVAILABLE		BIT(0, UL)
 /* Partition Reference Counter (HV_X64_MSR_TIME_REF_COUNT) available*/
-#define HV_MSR_TIME_REF_COUNT_AVAILABLE		BIT(1)
+#define HV_MSR_TIME_REF_COUNT_AVAILABLE		BIT(1, UL)
 /*
  * Basic SynIC MSRs (HV_X64_MSR_SCONTROL through HV_X64_MSR_EOM
  * and HV_X64_MSR_SINT0 through HV_X64_MSR_SINT15) available
  */
-#define HV_X64_MSR_SYNIC_AVAILABLE		BIT(2)
+#define HV_X64_MSR_SYNIC_AVAILABLE		BIT(2, UL)
 /*
  * Synthetic Timer MSRs (HV_X64_MSR_STIMER0_CONFIG through
  * HV_X64_MSR_STIMER3_COUNT) available
  */
-#define HV_MSR_SYNTIMER_AVAILABLE		BIT(3)
+#define HV_MSR_SYNTIMER_AVAILABLE		BIT(3, UL)
 /*
  * APIC access MSRs (HV_X64_MSR_EOI, HV_X64_MSR_ICR and HV_X64_MSR_TPR)
  * are available
  */
-#define HV_X64_MSR_APIC_ACCESS_AVAILABLE	BIT(4)
+#define HV_X64_MSR_APIC_ACCESS_AVAILABLE	BIT(4, UL)
 /* Hypercall MSRs (HV_X64_MSR_GUEST_OS_ID and HV_X64_MSR_HYPERCALL) available*/
-#define HV_X64_MSR_HYPERCALL_AVAILABLE		BIT(5)
+#define HV_X64_MSR_HYPERCALL_AVAILABLE		BIT(5, UL)
 /* Access virtual processor index MSR (HV_X64_MSR_VP_INDEX) available*/
-#define HV_X64_MSR_VP_INDEX_AVAILABLE		BIT(6)
+#define HV_X64_MSR_VP_INDEX_AVAILABLE		BIT(6, UL)
 /* Virtual system reset MSR (HV_X64_MSR_RESET) is available*/
-#define HV_X64_MSR_RESET_AVAILABLE		BIT(7)
+#define HV_X64_MSR_RESET_AVAILABLE		BIT(7, UL)
 /*
  * Access statistics pages MSRs (HV_X64_MSR_STATS_PARTITION_RETAIL_PAGE,
  * HV_X64_MSR_STATS_PARTITION_INTERNAL_PAGE, HV_X64_MSR_STATS_VP_RETAIL_PAGE,
  * HV_X64_MSR_STATS_VP_INTERNAL_PAGE) available
  */
-#define HV_X64_MSR_STAT_PAGES_AVAILABLE		BIT(8)
+#define HV_X64_MSR_STAT_PAGES_AVAILABLE		BIT(8, UL)
 /* Partition reference TSC MSR is available */
-#define HV_MSR_REFERENCE_TSC_AVAILABLE		BIT(9)
+#define HV_MSR_REFERENCE_TSC_AVAILABLE		BIT(9, UL)
 /* Partition Guest IDLE MSR is available */
-#define HV_X64_MSR_GUEST_IDLE_AVAILABLE		BIT(10)
+#define HV_X64_MSR_GUEST_IDLE_AVAILABLE		BIT(10, UL)
 /*
  * There is a single feature flag that signifies if the partition has access
  * to MSRs with local APIC and TSC frequencies.
  */
-#define HV_X64_ACCESS_FREQUENCY_MSRS		BIT(11)
+#define HV_X64_ACCESS_FREQUENCY_MSRS		BIT(11, UL)
 /* AccessReenlightenmentControls privilege */
-#define HV_X64_ACCESS_REENLIGHTENMENT		BIT(13)
+#define HV_X64_ACCESS_REENLIGHTENMENT		BIT(13, UL)
 
 /*
  * Feature identification: indicates which flags were specified at partition
@@ -93,17 +94,17 @@
  * defined in section Partition Creation Flags.
  * These are HYPERV_CPUID_FEATURES.EBX bits.
  */
-#define HV_X64_CREATE_PARTITIONS		BIT(0)
-#define HV_X64_ACCESS_PARTITION_ID		BIT(1)
-#define HV_X64_ACCESS_MEMORY_POOL		BIT(2)
-#define HV_X64_ADJUST_MESSAGE_BUFFERS		BIT(3)
-#define HV_X64_POST_MESSAGES			BIT(4)
-#define HV_X64_SIGNAL_EVENTS			BIT(5)
-#define HV_X64_CREATE_PORT			BIT(6)
-#define HV_X64_CONNECT_PORT			BIT(7)
-#define HV_X64_ACCESS_STATS			BIT(8)
-#define HV_X64_DEBUGGING			BIT(11)
-#define HV_X64_CPU_POWER_MANAGEMENT		BIT(12)
+#define HV_X64_CREATE_PARTITIONS		BIT(0, UL)
+#define HV_X64_ACCESS_PARTITION_ID		BIT(1, UL)
+#define HV_X64_ACCESS_MEMORY_POOL		BIT(2, UL)
+#define HV_X64_ADJUST_MESSAGE_BUFFERS		BIT(3, UL)
+#define HV_X64_POST_MESSAGES			BIT(4, UL)
+#define HV_X64_SIGNAL_EVENTS			BIT(5, UL)
+#define HV_X64_CREATE_PORT			BIT(6, UL)
+#define HV_X64_CONNECT_PORT			BIT(7, UL)
+#define HV_X64_ACCESS_STATS			BIT(8, UL)
+#define HV_X64_DEBUGGING			BIT(11, UL)
+#define HV_X64_CPU_POWER_MANAGEMENT		BIT(12, UL)
 
 /*
  * Feature identification. EDX indicates which miscellaneous features
@@ -111,26 +112,26 @@
  * These are HYPERV_CPUID_FEATURES.EDX bits.
  */
 /* The MWAIT instruction is available (per section MONITOR / MWAIT) */
-#define HV_X64_MWAIT_AVAILABLE				BIT(0)
+#define HV_X64_MWAIT_AVAILABLE				BIT(0, UL)
 /* Guest debugging support is available */
-#define HV_X64_GUEST_DEBUGGING_AVAILABLE		BIT(1)
+#define HV_X64_GUEST_DEBUGGING_AVAILABLE		BIT(1, UL)
 /* Performance Monitor support is available*/
-#define HV_X64_PERF_MONITOR_AVAILABLE			BIT(2)
+#define HV_X64_PERF_MONITOR_AVAILABLE			BIT(2, UL)
 /* Support for physical CPU dynamic partitioning events is available*/
-#define HV_X64_CPU_DYNAMIC_PARTITIONING_AVAILABLE	BIT(3)
+#define HV_X64_CPU_DYNAMIC_PARTITIONING_AVAILABLE	BIT(3, UL)
 /*
  * Support for passing hypercall input parameter block via XMM
  * registers is available
  */
-#define HV_X64_HYPERCALL_PARAMS_XMM_AVAILABLE		BIT(4)
+#define HV_X64_HYPERCALL_PARAMS_XMM_AVAILABLE		BIT(4, UL)
 /* Support for a virtual guest idle state is available */
-#define HV_X64_GUEST_IDLE_STATE_AVAILABLE		BIT(5)
+#define HV_X64_GUEST_IDLE_STATE_AVAILABLE		BIT(5, UL)
 /* Frequency MSRs available */
-#define HV_FEATURE_FREQUENCY_MSRS_AVAILABLE		BIT(8)
+#define HV_FEATURE_FREQUENCY_MSRS_AVAILABLE		BIT(8, UL)
 /* Crash MSR available */
-#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE		BIT(10)
+#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE		BIT(10, UL)
 /* stimer Direct Mode is available */
-#define HV_STIMER_DIRECT_MODE_AVAILABLE			BIT(19)
+#define HV_STIMER_DIRECT_MODE_AVAILABLE			BIT(19, UL)
 
 /*
  * Implementation recommendations. Indicates which behaviors the hypervisor
@@ -141,56 +142,56 @@
  * Recommend using hypercall for address space switches rather
  * than MOV to CR3 instruction
  */
-#define HV_X64_AS_SWITCH_RECOMMENDED			BIT(0)
+#define HV_X64_AS_SWITCH_RECOMMENDED			BIT(0, UL)
 /* Recommend using hypercall for local TLB flushes rather
  * than INVLPG or MOV to CR3 instructions */
-#define HV_X64_LOCAL_TLB_FLUSH_RECOMMENDED		BIT(1)
+#define HV_X64_LOCAL_TLB_FLUSH_RECOMMENDED		BIT(1, UL)
 /*
  * Recommend using hypercall for remote TLB flushes rather
  * than inter-processor interrupts
  */
-#define HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED		BIT(2)
+#define HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED		BIT(2, UL)
 /*
  * Recommend using MSRs for accessing APIC registers
  * EOI, ICR and TPR rather than their memory-mapped counterparts
  */
-#define HV_X64_APIC_ACCESS_RECOMMENDED			BIT(3)
+#define HV_X64_APIC_ACCESS_RECOMMENDED			BIT(3, UL)
 /* Recommend using the hypervisor-provided MSR to initiate a system RESET */
-#define HV_X64_SYSTEM_RESET_RECOMMENDED			BIT(4)
+#define HV_X64_SYSTEM_RESET_RECOMMENDED			BIT(4, UL)
 /*
  * Recommend using relaxed timing for this partition. If used,
  * the VM should disable any watchdog timeouts that rely on the
  * timely delivery of external interrupts
  */
-#define HV_X64_RELAXED_TIMING_RECOMMENDED		BIT(5)
+#define HV_X64_RELAXED_TIMING_RECOMMENDED		BIT(5, UL)
 
 /*
  * Recommend not using Auto End-Of-Interrupt feature
  */
-#define HV_DEPRECATING_AEOI_RECOMMENDED			BIT(9)
+#define HV_DEPRECATING_AEOI_RECOMMENDED			BIT(9, UL)
 
 /*
  * Recommend using cluster IPI hypercalls.
  */
-#define HV_X64_CLUSTER_IPI_RECOMMENDED			BIT(10)
+#define HV_X64_CLUSTER_IPI_RECOMMENDED			BIT(10, UL)
 
 /* Recommend using the newer ExProcessorMasks interface */
-#define HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED		BIT(11)
+#define HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED		BIT(11, UL)
 
 /* Recommend using enlightened VMCS */
-#define HV_X64_ENLIGHTENED_VMCS_RECOMMENDED		BIT(14)
+#define HV_X64_ENLIGHTENED_VMCS_RECOMMENDED		BIT(14, UL)
 
 /*
  * Virtual processor will never share a physical core with another virtual
  * processor, except for virtual processors that are reported as sibling SMT
  * threads.
  */
-#define HV_X64_NO_NONARCH_CORESHARING                  BIT(18)
+#define HV_X64_NO_NONARCH_CORESHARING                  BIT(18, UL)
 
 /* Nested features. These are HYPERV_CPUID_NESTED_FEATURES.EAX bits. */
-#define HV_X64_NESTED_DIRECT_FLUSH			BIT(17)
-#define HV_X64_NESTED_GUEST_MAPPING_FLUSH		BIT(18)
-#define HV_X64_NESTED_MSR_BITMAP			BIT(19)
+#define HV_X64_NESTED_DIRECT_FLUSH			BIT(17, UL)
+#define HV_X64_NESTED_GUEST_MAPPING_FLUSH		BIT(18, UL)
+#define HV_X64_NESTED_MSR_BITMAP			BIT(19, UL)
 
 /* Hyper-V specific model specific registers (MSRs) */
 
@@ -390,10 +391,10 @@ struct hv_tsc_emulation_status {
 #define HV_PROCESSOR_POWER_STATE_C2		2
 #define HV_PROCESSOR_POWER_STATE_C3		3
 
-#define HV_FLUSH_ALL_PROCESSORS			BIT(0)
-#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	BIT(1)
-#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	BIT(2)
-#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	BIT(3)
+#define HV_FLUSH_ALL_PROCESSORS			BIT(0, UL)
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	BIT(1, UL)
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	BIT(2, UL)
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	BIT(3, UL)
 
 enum HV_GENERIC_SET_FORMAT {
 	HV_GENERIC_SET_SPARSE_4K,
@@ -401,7 +402,7 @@ enum HV_GENERIC_SET_FORMAT {
 };
 
 #define HV_HYPERCALL_RESULT_MASK	GENMASK_ULL(15, 0)
-#define HV_HYPERCALL_FAST_BIT		BIT(16)
+#define HV_HYPERCALL_FAST_BIT		BIT(16, UL)
 #define HV_HYPERCALL_VARHEAD_OFFSET	17
 #define HV_HYPERCALL_REP_COMP_OFFSET	32
 #define HV_HYPERCALL_REP_COMP_MASK	GENMASK_ULL(43, 32)
@@ -740,22 +741,22 @@ struct hv_enlightened_vmcs {
 } __packed;
 
 #define HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE			0
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP		BIT(0)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP		BIT(1)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2		BIT(2)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1		BIT(3)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC		BIT(4)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EVENT		BIT(5)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_ENTRY		BIT(6)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EXCPN		BIT(7)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CRDR			BIT(8)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_XLAT		BIT(9)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC		BIT(10)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP1		BIT(11)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP2		BIT(12)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER		BIT(13)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_GRP1		BIT(14)
-#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL	BIT(15)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP		BIT(0, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP		BIT(1, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2		BIT(2, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1		BIT(3, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC		BIT(4, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EVENT		BIT(5, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_ENTRY		BIT(6, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EXCPN		BIT(7, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CRDR			BIT(8, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_XLAT		BIT(9, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC		BIT(10, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP1		BIT(11, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP2		BIT(12, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER		BIT(13, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_GRP1		BIT(14, UL)
+#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL	BIT(15, UL)
 
 #define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL			0xFFFF
 
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 3/7] x86/hyperv: extract more information from Hyper-V

[Wei Liu]

Provide a structure to store that information. The structure will be
accessed from other places later so make it public.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
 xen/arch/x86/guest/hyperv/hyperv.c | 14 ++++++++++++++
 xen/include/asm-x86/guest/hyperv.h | 12 ++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 7ab4b127f3..041166f344 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -21,6 +21,9 @@
 #include <xen/init.h>
 
 #include <asm/guest.h>
+#include <asm/guest/hyperv-tlfs.h>
+
+struct ms_hyperv_info ms_hyperv;
 
 bool __init hyperv_probe(void)
 {
@@ -36,6 +39,17 @@ bool __init hyperv_probe(void)
     if ( eax != 0x31237648 )    /* Hv#1 */
         return false;
 
+    /* Extract more information from Hyper-V */
+    ms_hyperv.features = cpuid_eax(HYPERV_CPUID_FEATURES);
+    ms_hyperv.misc_features = cpuid_edx(HYPERV_CPUID_FEATURES);
+    ms_hyperv.hints = cpuid_eax(HYPERV_CPUID_ENLIGHTMENT_INFO);
+
+    if ( ms_hyperv.hints & HV_X64_ENLIGHTENED_VMCS_RECOMMENDED )
+        ms_hyperv.nested_features = cpuid_eax(HYPERV_CPUID_NESTED_FEATURES);
+
+    ms_hyperv.max_vp_index = cpuid_eax(HYPERV_CPUID_IMPLEMENT_LIMITS);
+    ms_hyperv.max_lp_index = cpuid_ebx(HYPERV_CPUID_IMPLEMENT_LIMITS);
+
     return true;
 }
 
diff --git a/xen/include/asm-x86/guest/hyperv.h b/xen/include/asm-x86/guest/hyperv.h
index 4b9cc5a836..0f8800040a 100644
--- a/xen/include/asm-x86/guest/hyperv.h
+++ b/xen/include/asm-x86/guest/hyperv.h
@@ -21,8 +21,20 @@
 
 #ifdef CONFIG_HYPERV_GUEST
 
+#include <xen/types.h>
+
 #include <asm/guest/hypervisor.h>
 
+struct ms_hyperv_info {
+    uint32_t features;
+    uint32_t misc_features;
+    uint32_t hints;
+    uint32_t nested_features;
+    uint32_t max_vp_index;
+    uint32_t max_lp_index;
+};
+extern struct ms_hyperv_info ms_hyperv;
+
 extern struct hypervisor_ops hyperv_ops;
 
 bool hyperv_probe(void);
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 4/7] x86: add a comment regarding the location of hypervisor_probe

[Wei Liu]

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
 xen/arch/x86/setup.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index cf5a7b8e1e..4aa0af5a12 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -764,6 +764,10 @@ void __init noreturn __start_xen(unsigned long mbi_p)
      * allocing any xenheap structures wanted in lower memory. */
     kexec_early_calculations();
 
+    /*
+     * The probing has to be done _before_ initialising console,
+     * otherwise we couldn't set up Xen's PV console correctly.
+     */
     running_on_hypervisor = hypervisor_probe();
 
     parse_video_info();
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 5/7] x86: use running_on_hypervisor to gate hypervisor_setup

[Wei Liu]

The hypervisor_setup method is not unique to Xen guest.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
 xen/arch/x86/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 4aa0af5a12..044c45be36 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1577,7 +1577,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
             max_cpus = nr_cpu_ids;
     }
 
-    if ( xen_guest )
+    if ( running_on_hypervisor )
         hypervisor_setup();
 
     /* Low mappings were only needed for some BIOS table parsing. */
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 6/7] x86/hyperv: provide hyperv_guest variable

[Wei Liu]

It will be used to gate Hyper-V related code outside of the guest
directory.

No functional change.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
 xen/arch/x86/guest/hyperv/hyperv.c | 3 +++
 xen/include/asm-x86/guest/hyperv.h | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 041166f344..ee649426ce 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -24,6 +24,7 @@
 #include <asm/guest/hyperv-tlfs.h>
 
 struct ms_hyperv_info ms_hyperv;
+bool hyperv_guest;
 
 bool __init hyperv_probe(void)
 {
@@ -50,6 +51,8 @@ bool __init hyperv_probe(void)
     ms_hyperv.max_vp_index = cpuid_eax(HYPERV_CPUID_IMPLEMENT_LIMITS);
     ms_hyperv.max_lp_index = cpuid_ebx(HYPERV_CPUID_IMPLEMENT_LIMITS);
 
+    hyperv_guest = true;
+
     return true;
 }
 
diff --git a/xen/include/asm-x86/guest/hyperv.h b/xen/include/asm-x86/guest/hyperv.h
index 0f8800040a..86f5c24ec6 100644
--- a/xen/include/asm-x86/guest/hyperv.h
+++ b/xen/include/asm-x86/guest/hyperv.h
@@ -35,6 +35,8 @@ struct ms_hyperv_info {
 };
 extern struct ms_hyperv_info ms_hyperv;
 
+extern bool hyperv_guest;
+
 extern struct hypervisor_ops hyperv_ops;
 
 bool hyperv_probe(void);
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Wei Liu]

Implement a clock source using Hyper-V's reference TSC page.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
Relevant spec:

https://github.com/MicrosoftDocs/Virtualization-Documentation/raw/live/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v5.0C.pdf

Section 12.6.
---
 xen/arch/x86/time.c | 87 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index d8242295ef..f7e93b8a1f 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -30,6 +30,7 @@
 #include <asm/processor.h>
 #include <asm/fixmap.h>
 #include <asm/guest.h>
+#include <asm/guest/hyperv-tlfs.h>
 #include <asm/mc146818rtc.h>
 #include <asm/div64.h>
 #include <asm/acpi.h>
@@ -614,6 +615,89 @@ static struct platform_timesource __initdata plt_xen_timer =
 };
 #endif
 
+#ifdef CONFIG_HYPERV_GUEST
+/************************************************************
+ * PLATFORM TIMER 6: HYPER-V REFERENCE TSC
+ */
+
+static struct ms_hyperv_tsc_page hyperv_tsc_page __aligned(PAGE_SIZE);
+
+static int64_t __init init_hyperv_timer(struct platform_timesource *pts)
+{
+    unsigned long maddr;
+    uint64_t tsc_msr, freq;
+
+    if ( !hyperv_guest ||
+         !(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE) )
+        return 0;
+
+    maddr = virt_to_maddr(&hyperv_tsc_page);
+
+    /*
+     * Per Hyper-V TLFS:
+     *   1. Read existing MSR value
+     *   2. Preserve bits [11:1]
+     *   3. Set bits [63:12] to be guest physical address of tsc page
+     *   4. Set enabled bit (0)
+     *   5. Write back new MSR value
+     */
+    rdmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr);
+    tsc_msr &= GENMASK_ULL(11, 1);
+    tsc_msr = tsc_msr | (uint64_t)maddr | 1 /* enabled */;
+    wrmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr);
+
+    /* Get TSC frequency from Hyper-V */
+    rdmsrl(HV_X64_MSR_TSC_FREQUENCY, freq);
+    pts->frequency = freq;
+
+    return freq;
+}
+
+static inline uint64_t read_hyperv_timer(void)
+{
+    uint64_t scale, offset, ret, tsc;
+    uint32_t seq;
+    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;
+
+    do {
+        seq = tsc_page->tsc_sequence;
+
+        /* Seq 0 is special. It means the TSC enlightenment is not
+         * available at the moment. The reference time can only be
+         * obtained from the Reference Counter MSR.
+         */
+        if ( seq == 0 )
+        {
+            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
+            return ret;
+        }
+
+        smp_rmb();
+
+        tsc = rdtsc_ordered();
+        scale = tsc_page->tsc_scale;
+        offset = tsc_page->tsc_offset;
+
+        smp_rmb();
+
+    } while (tsc_page->tsc_sequence != seq);
+
+    /* x86 has ARCH_SUPPORTS_INT128 */
+    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;
+
+    return ret;
+}
+
+static struct platform_timesource __initdata plt_hyperv_timer =
+{
+    .id = "hyperv",
+    .name = "HYPER-V REFERENCE TSC",
+    .read_counter = read_hyperv_timer,
+    .init = init_hyperv_timer,
+    .counter_bits = 63,
+};
+#endif
+
 /************************************************************
  * GENERIC PLATFORM TIMER INFRASTRUCTURE
  */
@@ -763,6 +847,9 @@ static u64 __init init_platform_timer(void)
     static struct platform_timesource * __initdata plt_timers[] = {
 #ifdef CONFIG_XEN_GUEST
         &plt_xen_timer,
+#endif
+#ifdef CONFIG_HYPERV_GUEST
+        &plt_hyperv_timer,
 #endif
         &plt_hpet, &plt_pmtimer, &plt_pit
     };
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.
> 
> This is a pristine copy from Linux. It is not used yet and probably
> doesn't compile. Changes to make it work will come later.
> 
> Signed-off-by: Wei Liu <liuwe@microsoft.com>

This coming from Linux and assuming at least a fair part of it is
going to be used, in principle
Acked-by: Jan Beulich <jbeulich@suse.com>

However, there are many seemingly unnecessary uses of __packed
here, which I'd rather not see go in at all (i.e. not be dropped
later on, and then potentially missing some). I find ...

> +typedef struct _HV_REFERENCE_TSC_PAGE {
> +	__u32 tsc_sequence;
> +	__u32 res1;
> +	__u64 tsc_scale;
> +	__s64 tsc_offset;
> +}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;

.. this one particularly suspicious: I don't think it is well
defined for __packed to also apply to the type
PHV_REFERENCE_TSC_PAGE points to (and I suspect it doesn't).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 2/7] x86: fix up hyperv-tlfs.h

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> Do the following:
> 1. include xen/types.h and xen/bitops.h
> 2. fix up invocations of BIT macro

Is it truly BIT(..., UL) in _all_ cases, and not BIT(..., U) in some?

> Signed-off-by: Wei Liu <liuwe@microsoft.com>
> ---
> This can be squashed into previous patch if preferred.

Afaic - yes please.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Durrant, Paul]

> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Jan
> Beulich
> Sent: 10 December 2019 15:34
> To: Wei Liu <wl@xen.org>
> Cc: Wei Liu <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>; Andrew
> Cooper <andrew.cooper3@citrix.com>; Michael Kelley
> <mikelley@microsoft.com>; Xen Development List <xen-
> devel@lists.xenproject.org>; Roger Pau Monné <roger.pau@citrix.com>
> Subject: Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h
> from Linux
> 
> On 25.10.2019 11:16, Wei Liu wrote:
> > Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.
> >
> > This is a pristine copy from Linux. It is not used yet and probably
> > doesn't compile. Changes to make it work will come later.
> >
> > Signed-off-by: Wei Liu <liuwe@microsoft.com>
> 
> This coming from Linux and assuming at least a fair part of it is
> going to be used, in principle
> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> However, there are many seemingly unnecessary uses of __packed
> here, which I'd rather not see go in at all (i.e. not be dropped
> later on, and then potentially missing some). I find ...
> 
> > +typedef struct _HV_REFERENCE_TSC_PAGE {
> > +	__u32 tsc_sequence;
> > +	__u32 res1;
> > +	__u64 tsc_scale;
> > +	__s64 tsc_offset;
> > +}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
>

You realise there's a definition of this in the viridian code already, right?

  Paul
 
> .. this one particularly suspicious: I don't think it is well
> defined for __packed to also apply to the type
> PHV_REFERENCE_TSC_PAGE points to (and I suspect it doesn't).
> 
> Jan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Jan Beulich]

On 10.12.2019 16:37, Durrant, Paul wrote:
>> -----Original Message-----
>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Jan
>> Beulich
>> Sent: 10 December 2019 15:34
>> To: Wei Liu <wl@xen.org>
>> Cc: Wei Liu <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>; Andrew
>> Cooper <andrew.cooper3@citrix.com>; Michael Kelley
>> <mikelley@microsoft.com>; Xen Development List <xen-
>> devel@lists.xenproject.org>; Roger Pau Monné <roger.pau@citrix.com>
>> Subject: Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h
>> from Linux
>>
>> On 25.10.2019 11:16, Wei Liu wrote:
>>> Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.
>>>
>>> This is a pristine copy from Linux. It is not used yet and probably
>>> doesn't compile. Changes to make it work will come later.
>>>
>>> Signed-off-by: Wei Liu <liuwe@microsoft.com>
>>
>> This coming from Linux and assuming at least a fair part of it is
>> going to be used, in principle
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>
>> However, there are many seemingly unnecessary uses of __packed
>> here, which I'd rather not see go in at all (i.e. not be dropped
>> later on, and then potentially missing some). I find ...
>>
>>> +typedef struct _HV_REFERENCE_TSC_PAGE {
>>> +	__u32 tsc_sequence;
>>> +	__u32 res1;
>>> +	__u64 tsc_scale;
>>> +	__s64 tsc_offset;
>>> +}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
>>
> 
> You realise there's a definition of this in the viridian code already, right?

It looked familiar, but it didn't occur to me to point this out.
Yes, there looks to be room for deduplication...

Actually, Wei, one more thing I was curious about - what is "tlfs"
an acronym of?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 3/7] x86/hyperv: extract more information from Hyper-V

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> --- a/xen/arch/x86/guest/hyperv/hyperv.c
> +++ b/xen/arch/x86/guest/hyperv/hyperv.c
> @@ -21,6 +21,9 @@
>  #include <xen/init.h>
>  
>  #include <asm/guest.h>
> +#include <asm/guest/hyperv-tlfs.h>
> +
> +struct ms_hyperv_info ms_hyperv;
>  
>  bool __init hyperv_probe(void)
>  {
> @@ -36,6 +39,17 @@ bool __init hyperv_probe(void)
>      if ( eax != 0x31237648 )    /* Hv#1 */
>          return false;
>  
> +    /* Extract more information from Hyper-V */
> +    ms_hyperv.features = cpuid_eax(HYPERV_CPUID_FEATURES);
> +    ms_hyperv.misc_features = cpuid_edx(HYPERV_CPUID_FEATURES);

It's __init code, so it doesn't matter all that much, but perhaps
obtain these two and ...

> +    ms_hyperv.hints = cpuid_eax(HYPERV_CPUID_ENLIGHTMENT_INFO);
> +
> +    if ( ms_hyperv.hints & HV_X64_ENLIGHTENED_VMCS_RECOMMENDED )
> +        ms_hyperv.nested_features = cpuid_eax(HYPERV_CPUID_NESTED_FEATURES);
> +
> +    ms_hyperv.max_vp_index = cpuid_eax(HYPERV_CPUID_IMPLEMENT_LIMITS);
> +    ms_hyperv.max_lp_index = cpuid_ebx(HYPERV_CPUID_IMPLEMENT_LIMITS);

... these two with just one CPUID invocation each? Preferably
with this adjustment
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 4/7] x86: add a comment regarding the location of hypervisor_probe

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> Signed-off-by: Wei Liu <liuwe@microsoft.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 5/7] x86: use running_on_hypervisor to gate hypervisor_setup

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1577,7 +1577,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>              max_cpus = nr_cpu_ids;
>      }
>  
> -    if ( xen_guest )
> +    if ( running_on_hypervisor )
>          hypervisor_setup();

This code is using hypervisor_name already, so I guess the patch
has become unnecessary?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 6/7] x86/hyperv: provide hyperv_guest variable

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> --- a/xen/arch/x86/guest/hyperv/hyperv.c
> +++ b/xen/arch/x86/guest/hyperv/hyperv.c
> @@ -24,6 +24,7 @@
>  #include <asm/guest/hyperv-tlfs.h>
>  
>  struct ms_hyperv_info ms_hyperv;
> +bool hyperv_guest;

With __read_mostly added here (perhaps also in the earlier
patch for the adjacent one)
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Jan Beulich]

On 25.10.2019 11:16, Wei Liu wrote:
> @@ -614,6 +615,89 @@ static struct platform_timesource __initdata plt_xen_timer =
>  };
>  #endif
>  
> +#ifdef CONFIG_HYPERV_GUEST
> +/************************************************************
> + * PLATFORM TIMER 6: HYPER-V REFERENCE TSC

I don't think numbering is very helpful for optionally built code.
(I realize though that this same anomaly exists for the Xen guest
timer already.)

> + */
> +
> +static struct ms_hyperv_tsc_page hyperv_tsc_page __aligned(PAGE_SIZE);

Does this need to be a statically allocated page?

> +static int64_t __init init_hyperv_timer(struct platform_timesource *pts)
> +{
> +    unsigned long maddr;

paddr_t ?

> +    uint64_t tsc_msr, freq;
> +
> +    if ( !hyperv_guest ||
> +         !(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE) )

Is the hyperv_guest check really needed? The feature bit won't be
set without that variable being true anyway, will it?

> +        return 0;
> +
> +    maddr = virt_to_maddr(&hyperv_tsc_page);
> +
> +    /*
> +     * Per Hyper-V TLFS:
> +     *   1. Read existing MSR value
> +     *   2. Preserve bits [11:1]
> +     *   3. Set bits [63:12] to be guest physical address of tsc page
> +     *   4. Set enabled bit (0)
> +     *   5. Write back new MSR value
> +     */
> +    rdmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr);
> +    tsc_msr &= GENMASK_ULL(11, 1);

A discussion not so long ago has resulted in, iirc, Andrew and me
agreeing that in its current shape we don't want to see any uses
of this macro outside of Arm-specific code.

> +    tsc_msr = tsc_msr | (uint64_t)maddr | 1 /* enabled */;

Why the cast? And maybe easier as "tsc_msr |= "?

> +    wrmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr);
> +
> +    /* Get TSC frequency from Hyper-V */
> +    rdmsrl(HV_X64_MSR_TSC_FREQUENCY, freq);
> +    pts->frequency = freq;
> +
> +    return freq;
> +}
> +
> +static inline uint64_t read_hyperv_timer(void)
> +{
> +    uint64_t scale, offset, ret, tsc;
> +    uint32_t seq;
> +    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;

const?

> +    do {
> +        seq = tsc_page->tsc_sequence;
> +
> +        /* Seq 0 is special. It means the TSC enlightenment is not
> +         * available at the moment. The reference time can only be
> +         * obtained from the Reference Counter MSR.
> +         */
> +        if ( seq == 0 )
> +        {
> +            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
> +            return ret;
> +        }
> +
> +        smp_rmb();
> +
> +        tsc = rdtsc_ordered();

This already includes at least a read fence.

> +        scale = tsc_page->tsc_scale;
> +        offset = tsc_page->tsc_offset;
> +
> +        smp_rmb();
> +
> +    } while (tsc_page->tsc_sequence != seq);
> +
> +    /* x86 has ARCH_SUPPORTS_INT128 */
> +    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;

The final cast isn't really needed, is it? As to the multiplication
- are you sure all compilers in all cases will avoid falling back
to a library call here? In other similar places I think we use
inline assembly instead.

> +    return ret;
> +}
> +
> +static struct platform_timesource __initdata plt_hyperv_timer =
> +{
> +    .id = "hyperv",
> +    .name = "HYPER-V REFERENCE TSC",
> +    .read_counter = read_hyperv_timer,
> +    .init = init_hyperv_timer,
> +    .counter_bits = 63,

Why 63? The calculation above is a uint64_t one. If there are
wrapping concerns like for the TSC source, please add a
respective comment (which may be as brief as a reference to
the other one, if that's appropriate).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Wei Liu]

On Tue, Dec 10, 2019 at 04:43:30PM +0100, Jan Beulich wrote:
> On 10.12.2019 16:37, Durrant, Paul wrote:
> >> -----Original Message-----
> >> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Jan
> >> Beulich
> >> Sent: 10 December 2019 15:34
> >> To: Wei Liu <wl@xen.org>
> >> Cc: Wei Liu <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>; Andrew
> >> Cooper <andrew.cooper3@citrix.com>; Michael Kelley
> >> <mikelley@microsoft.com>; Xen Development List <xen-
> >> devel@lists.xenproject.org>; Roger Pau Monné <roger.pau@citrix.com>
> >> Subject: Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h
> >> from Linux
> >>
> >> On 25.10.2019 11:16, Wei Liu wrote:
> >>> Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.
> >>>
> >>> This is a pristine copy from Linux. It is not used yet and probably
> >>> doesn't compile. Changes to make it work will come later.
> >>>
> >>> Signed-off-by: Wei Liu <liuwe@microsoft.com>
> >>
> >> This coming from Linux and assuming at least a fair part of it is
> >> going to be used, in principle
> >> Acked-by: Jan Beulich <jbeulich@suse.com>
> >>
> >> However, there are many seemingly unnecessary uses of __packed
> >> here, which I'd rather not see go in at all (i.e. not be dropped
> >> later on, and then potentially missing some). I find ...
> >>
> >>> +typedef struct _HV_REFERENCE_TSC_PAGE {
> >>> +	__u32 tsc_sequence;
> >>> +	__u32 res1;
> >>> +	__u64 tsc_scale;
> >>> +	__s64 tsc_offset;
> >>> +}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
> >>
> > 
> > You realise there's a definition of this in the viridian code already, right?
> 
> It looked familiar, but it didn't occur to me to point this out.
> Yes, there looks to be room for deduplication...
> 

I had a plan to make viridian code use this copy directly.


> Actually, Wei, one more thing I was curious about - what is "tlfs"
> an acronym of?

It means "Top-Level Function Specification".

(I wish Xen had something similar)

Wei.

> 
> Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Durrant, Paul]

> -----Original Message-----
> From: Wei Liu <wl@xen.org>
> Sent: 11 December 2019 11:15
> To: Jan Beulich <jbeulich@suse.com>
> Cc: Durrant, Paul <pdurrant@amazon.com>; Wei Liu <wl@xen.org>; Wei Liu
> <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>; Andrew Cooper
> <andrew.cooper3@citrix.com>; Michael Kelley <mikelley@microsoft.com>; Xen
> Development List <xen-devel@lists.xenproject.org>; Roger Pau Monné
> <roger.pau@citrix.com>
> Subject: Re: [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux
> 
> On Tue, Dec 10, 2019 at 04:43:30PM +0100, Jan Beulich wrote:
> > On 10.12.2019 16:37, Durrant, Paul wrote:
> > >> -----Original Message-----
> > >> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> Jan
> > >> Beulich
> > >> Sent: 10 December 2019 15:34
> > >> To: Wei Liu <wl@xen.org>
> > >> Cc: Wei Liu <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>;
> Andrew
> > >> Cooper <andrew.cooper3@citrix.com>; Michael Kelley
> > >> <mikelley@microsoft.com>; Xen Development List <xen-
> > >> devel@lists.xenproject.org>; Roger Pau Monné <roger.pau@citrix.com>
> > >> Subject: Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-
> tlfs.h
> > >> from Linux
> > >>
> > >> On 25.10.2019 11:16, Wei Liu wrote:
> > >>> Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.
> > >>>
> > >>> This is a pristine copy from Linux. It is not used yet and probably
> > >>> doesn't compile. Changes to make it work will come later.
> > >>>
> > >>> Signed-off-by: Wei Liu <liuwe@microsoft.com>
> > >>
> > >> This coming from Linux and assuming at least a fair part of it is
> > >> going to be used, in principle
> > >> Acked-by: Jan Beulich <jbeulich@suse.com>
> > >>
> > >> However, there are many seemingly unnecessary uses of __packed
> > >> here, which I'd rather not see go in at all (i.e. not be dropped
> > >> later on, and then potentially missing some). I find ...
> > >>
> > >>> +typedef struct _HV_REFERENCE_TSC_PAGE {
> > >>> +	__u32 tsc_sequence;
> > >>> +	__u32 res1;
> > >>> +	__u64 tsc_scale;
> > >>> +	__s64 tsc_offset;
> > >>> +}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
> > >>
> > >
> > > You realise there's a definition of this in the viridian code already,
> right?
> >
> > It looked familiar, but it didn't occur to me to point this out.
> > Yes, there looks to be room for deduplication...
> >
> 
> I had a plan to make viridian code use this copy directly.
> 

I have no objection to that, but I think it ought to be done as part of this series so that we don't end up with long-term duplication.

  Paul

> 
> > Actually, Wei, one more thing I was curious about - what is "tlfs"
> > an acronym of?
> 
> It means "Top-Level Function Specification".
> 
> (I wish Xen had something similar)
> 
> Wei.
> 
> >
> > Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux

[Wei Liu]

On Wed, Dec 11, 2019 at 11:22:49AM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Wei Liu <wl@xen.org>
> > Sent: 11 December 2019 11:15
> > To: Jan Beulich <jbeulich@suse.com>
> > Cc: Durrant, Paul <pdurrant@amazon.com>; Wei Liu <wl@xen.org>; Wei Liu
> > <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>; Andrew Cooper
> > <andrew.cooper3@citrix.com>; Michael Kelley <mikelley@microsoft.com>; Xen
> > Development List <xen-devel@lists.xenproject.org>; Roger Pau Monné
> > <roger.pau@citrix.com>
> > Subject: Re: [PATCH for-next 1/7] x86: import hyperv-tlfs.h from Linux
> > 
> > On Tue, Dec 10, 2019 at 04:43:30PM +0100, Jan Beulich wrote:
> > > On 10.12.2019 16:37, Durrant, Paul wrote:
> > > >> -----Original Message-----
> > > >> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> > Jan
> > > >> Beulich
> > > >> Sent: 10 December 2019 15:34
> > > >> To: Wei Liu <wl@xen.org>
> > > >> Cc: Wei Liu <liuwe@microsoft.com>; Paul Durrant <paul@xen.org>;
> > Andrew
> > > >> Cooper <andrew.cooper3@citrix.com>; Michael Kelley
> > > >> <mikelley@microsoft.com>; Xen Development List <xen-
> > > >> devel@lists.xenproject.org>; Roger Pau Monné <roger.pau@citrix.com>
> > > >> Subject: Re: [Xen-devel] [PATCH for-next 1/7] x86: import hyperv-
> > tlfs.h
> > > >> from Linux
> > > >>
> > > >> On 25.10.2019 11:16, Wei Liu wrote:
> > > >>> Taken from Linux commit b2d8b167e15bb5ec2691d1119c025630a247f649.
> > > >>>
> > > >>> This is a pristine copy from Linux. It is not used yet and probably
> > > >>> doesn't compile. Changes to make it work will come later.
> > > >>>
> > > >>> Signed-off-by: Wei Liu <liuwe@microsoft.com>
> > > >>
> > > >> This coming from Linux and assuming at least a fair part of it is
> > > >> going to be used, in principle
> > > >> Acked-by: Jan Beulich <jbeulich@suse.com>
> > > >>
> > > >> However, there are many seemingly unnecessary uses of __packed
> > > >> here, which I'd rather not see go in at all (i.e. not be dropped
> > > >> later on, and then potentially missing some). I find ...
> > > >>
> > > >>> +typedef struct _HV_REFERENCE_TSC_PAGE {
> > > >>> +	__u32 tsc_sequence;
> > > >>> +	__u32 res1;
> > > >>> +	__u64 tsc_scale;
> > > >>> +	__s64 tsc_offset;
> > > >>> +}  __packed HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
> > > >>
> > > >
> > > > You realise there's a definition of this in the viridian code already,
> > right?
> > >
> > > It looked familiar, but it didn't occur to me to point this out.
> > > Yes, there looks to be room for deduplication...
> > >
> > 
> > I had a plan to make viridian code use this copy directly.
> > 
> 
> I have no objection to that, but I think it ought to be done as part of this series so that we don't end up with long-term duplication.

Sure. That shouldn't be too hard.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 5/7] x86: use running_on_hypervisor to gate hypervisor_setup

[Wei Liu]

On Tue, Dec 10, 2019 at 05:17:28PM +0100, Jan Beulich wrote:
> On 25.10.2019 11:16, Wei Liu wrote:
> > --- a/xen/arch/x86/setup.c
> > +++ b/xen/arch/x86/setup.c
> > @@ -1577,7 +1577,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
> >              max_cpus = nr_cpu_ids;
> >      }
> >  
> > -    if ( xen_guest )
> > +    if ( running_on_hypervisor )
> >          hypervisor_setup();
> 
> This code is using hypervisor_name already, so I guess the patch
> has become unnecessary?

Yes. I basically squashed this patch into my previous series while
reworking that.

This patch here is not needed anymore.

Wei.

> 
> Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 2/7] x86: fix up hyperv-tlfs.h

[Wei Liu]

On Tue, Dec 10, 2019 at 04:35:41PM +0100, Jan Beulich wrote:
> On 25.10.2019 11:16, Wei Liu wrote:
> > Do the following:
> > 1. include xen/types.h and xen/bitops.h
> > 2. fix up invocations of BIT macro
> 
> Is it truly BIT(..., UL) in _all_ cases, and not BIT(..., U) in some?

In Linux BIT is 

   #define BIT(nr) (UL(1) << nr)

so yes Linux developers did mean UL everywhere.

I haven't checked them one by one per TLFS's definitions though.

Wei.

> 
> > Signed-off-by: Wei Liu <liuwe@microsoft.com>
> > ---
> > This can be squashed into previous patch if preferred.
> 
> Afaic - yes please.
> 
> Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Wei Liu]

On Tue, Dec 10, 2019 at 05:59:04PM +0100, Jan Beulich wrote:
> On 25.10.2019 11:16, Wei Liu wrote:
> > @@ -614,6 +615,89 @@ static struct platform_timesource __initdata plt_xen_timer =
> >  };
> >  #endif
> >  
> > +#ifdef CONFIG_HYPERV_GUEST
> > +/************************************************************
> > + * PLATFORM TIMER 6: HYPER-V REFERENCE TSC
> 
> I don't think numbering is very helpful for optionally built code.
> (I realize though that this same anomaly exists for the Xen guest
> timer already.)

I will delete the numbering bit.

> 
> > + */
> > +
> > +static struct ms_hyperv_tsc_page hyperv_tsc_page __aligned(PAGE_SIZE);
> 
> Does this need to be a statically allocated page?
> 

At first I thought early_time_init was called before allocator has been
setup because arch_init_memory is called right after it.

Upon closer inspection I think that assumption was wrong. So yes I
should be able to just allocate a page from domheap here.

> > +static int64_t __init init_hyperv_timer(struct platform_timesource *pts)
> > +{
> > +    unsigned long maddr;
> 
> paddr_t ?
> 

Ack.

> > +    uint64_t tsc_msr, freq;
> > +
> > +    if ( !hyperv_guest ||
> > +         !(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE) )
> 
> Is the hyperv_guest check really needed? The feature bit won't be
> set without that variable being true anyway, will it?
> 

Yes you're right.

> > +        return 0;
> > +
> > +    maddr = virt_to_maddr(&hyperv_tsc_page);
> > +
> > +    /*
> > +     * Per Hyper-V TLFS:
> > +     *   1. Read existing MSR value
> > +     *   2. Preserve bits [11:1]
> > +     *   3. Set bits [63:12] to be guest physical address of tsc page
> > +     *   4. Set enabled bit (0)
> > +     *   5. Write back new MSR value
> > +     */
> > +    rdmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr);
> > +    tsc_msr &= GENMASK_ULL(11, 1);
> 
> A discussion not so long ago has resulted in, iirc, Andrew and me
> agreeing that in its current shape we don't want to see any uses
> of this macro outside of Arm-specific code.
> 

Fair enough. I will use 0xFFEul instead.

> > +    tsc_msr = tsc_msr | (uint64_t)maddr | 1 /* enabled */;
> 
> Why the cast? And maybe easier as "tsc_msr |= "?
> 
> > +    wrmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr);
> > +
> > +    /* Get TSC frequency from Hyper-V */
> > +    rdmsrl(HV_X64_MSR_TSC_FREQUENCY, freq);
> > +    pts->frequency = freq;
> > +
> > +    return freq;
> > +}
> > +
> > +static inline uint64_t read_hyperv_timer(void)
> > +{
> > +    uint64_t scale, offset, ret, tsc;
> > +    uint32_t seq;
> > +    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;
> 
> const?

Ack.

> 
> > +    do {
> > +        seq = tsc_page->tsc_sequence;
> > +
> > +        /* Seq 0 is special. It means the TSC enlightenment is not
> > +         * available at the moment. The reference time can only be
> > +         * obtained from the Reference Counter MSR.
> > +         */
> > +        if ( seq == 0 )
> > +        {
> > +            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
> > +            return ret;
> > +        }
> > +
> > +        smp_rmb();
> > +
> > +        tsc = rdtsc_ordered();
> 
> This already includes at least a read fence.

OK. rdtsc() should be enough here.

> 
> > +        scale = tsc_page->tsc_scale;
> > +        offset = tsc_page->tsc_offset;
> > +
> > +        smp_rmb();
> > +
> > +    } while (tsc_page->tsc_sequence != seq);
> > +
> > +    /* x86 has ARCH_SUPPORTS_INT128 */
> > +    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;
> 
> The final cast isn't really needed, is it? As to the multiplication
> - are you sure all compilers in all cases will avoid falling back
> to a library call here? In other similar places I think we use
> inline assembly instead.

What library call? A function in libgcc (or clang's equivalence)?
ISTR libgcc is always linked, but I could be wrong here.

I'm happy to change it to inline assembly though.

> 
> > +    return ret;
> > +}
> > +
> > +static struct platform_timesource __initdata plt_hyperv_timer =
> > +{
> > +    .id = "hyperv",
> > +    .name = "HYPER-V REFERENCE TSC",
> > +    .read_counter = read_hyperv_timer,
> > +    .init = init_hyperv_timer,
> > +    .counter_bits = 63,
> 
> Why 63? The calculation above is a uint64_t one. If there are
> wrapping concerns like for the TSC source, please add a
> respective comment (which may be as brief as a reference to
> the other one, if that's appropriate).

OK. I will add a comment to reference the previous comment.

Wei.

> 
> Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Jan Beulich]

On 18.12.2019 13:38, Wei Liu wrote:
> On Tue, Dec 10, 2019 at 05:59:04PM +0100, Jan Beulich wrote:
>> On 25.10.2019 11:16, Wei Liu wrote:
>>> +static inline uint64_t read_hyperv_timer(void)
>>> +{
>>> +    uint64_t scale, offset, ret, tsc;
>>> +    uint32_t seq;
>>> +    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;
>>> +
>>> +    do {
>>> +        seq = tsc_page->tsc_sequence;
>>> +
>>> +        /* Seq 0 is special. It means the TSC enlightenment is not
>>> +         * available at the moment. The reference time can only be
>>> +         * obtained from the Reference Counter MSR.
>>> +         */
>>> +        if ( seq == 0 )
>>> +        {
>>> +            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
>>> +            return ret;
>>> +        }
>>> +
>>> +        smp_rmb();
>>> +
>>> +        tsc = rdtsc_ordered();
>>
>> This already includes at least a read fence.
> 
> OK. rdtsc() should be enough here.

Are you sure? My comment was rather towards the dropping of smp_rmb()
(maybe replacing by a comment).

>>> +        scale = tsc_page->tsc_scale;
>>> +        offset = tsc_page->tsc_offset;
>>> +
>>> +        smp_rmb();
>>> +
>>> +    } while (tsc_page->tsc_sequence != seq);
>>> +
>>> +    /* x86 has ARCH_SUPPORTS_INT128 */
>>> +    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;
>>
>> The final cast isn't really needed, is it? As to the multiplication
>> - are you sure all compilers in all cases will avoid falling back
>> to a library call here? In other similar places I think we use
>> inline assembly instead.
> 
> What library call? A function in libgcc (or clang's equivalence)?
> ISTR libgcc is always linked, but I could be wrong here.

No, the hypervisor (at least the x86 one) doesn't link libgcc afaik.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Andrew Cooper]

On 18/12/2019 12:51, Jan Beulich wrote:
>>>> +        scale = tsc_page->tsc_scale;
>>>> +        offset = tsc_page->tsc_offset;
>>>> +
>>>> +        smp_rmb();
>>>> +
>>>> +    } while (tsc_page->tsc_sequence != seq);
>>>> +
>>>> +    /* x86 has ARCH_SUPPORTS_INT128 */
>>>> +    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;
>>> The final cast isn't really needed, is it? As to the multiplication
>>> - are you sure all compilers in all cases will avoid falling back
>>> to a library call here? In other similar places I think we use
>>> inline assembly instead.
>> What library call? A function in libgcc (or clang's equivalence)?
>> ISTR libgcc is always linked, but I could be wrong here.
> No, the hypervisor (at least the x86 one) doesn't link libgcc afaik.

x86 can multiply uint128_t by uint64_t without a library call.  If this
compiles, then it should be fine.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Wei Liu]

On Wed, Dec 18, 2019 at 12:56:41PM +0000, Andrew Cooper wrote:
> On 18/12/2019 12:51, Jan Beulich wrote:
> >>>> +        scale = tsc_page->tsc_scale;
> >>>> +        offset = tsc_page->tsc_offset;
> >>>> +
> >>>> +        smp_rmb();
> >>>> +
> >>>> +    } while (tsc_page->tsc_sequence != seq);
> >>>> +
> >>>> +    /* x86 has ARCH_SUPPORTS_INT128 */
> >>>> +    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;
> >>> The final cast isn't really needed, is it? As to the multiplication
> >>> - are you sure all compilers in all cases will avoid falling back
> >>> to a library call here? In other similar places I think we use
> >>> inline assembly instead.
> >> What library call? A function in libgcc (or clang's equivalence)?
> >> ISTR libgcc is always linked, but I could be wrong here.
> > No, the hypervisor (at least the x86 one) doesn't link libgcc afaik.
> 
> x86 can multiply uint128_t by uint64_t without a library call.  If this
> compiles, then it should be fine.

This obviously compiles fine for me. But Jan's comment was more about
making sure all possible x86 compilers can deal with it. I don't have a
definitive answer to that.

Wei.

> 
> ~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Wei Liu]

On Wed, Dec 18, 2019 at 01:51:54PM +0100, Jan Beulich wrote:
> On 18.12.2019 13:38, Wei Liu wrote:
> > On Tue, Dec 10, 2019 at 05:59:04PM +0100, Jan Beulich wrote:
> >> On 25.10.2019 11:16, Wei Liu wrote:
> >>> +static inline uint64_t read_hyperv_timer(void)
> >>> +{
> >>> +    uint64_t scale, offset, ret, tsc;
> >>> +    uint32_t seq;
> >>> +    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;
> >>> +
> >>> +    do {
> >>> +        seq = tsc_page->tsc_sequence;
> >>> +
> >>> +        /* Seq 0 is special. It means the TSC enlightenment is not
> >>> +         * available at the moment. The reference time can only be
> >>> +         * obtained from the Reference Counter MSR.
> >>> +         */
> >>> +        if ( seq == 0 )
> >>> +        {
> >>> +            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
> >>> +            return ret;
> >>> +        }
> >>> +
> >>> +        smp_rmb();
> >>> +
> >>> +        tsc = rdtsc_ordered();
> >>
> >> This already includes at least a read fence.
> > 
> > OK. rdtsc() should be enough here.
> 
> Are you sure? My comment was rather towards the dropping of smp_rmb()
> (maybe replacing by a comment).

I do mean to keep smp_rmb() before it. Is that not enough?

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Jan Beulich]

On 18.12.2019 14:18, Wei Liu wrote:
> On Wed, Dec 18, 2019 at 01:51:54PM +0100, Jan Beulich wrote:
>> On 18.12.2019 13:38, Wei Liu wrote:
>>> On Tue, Dec 10, 2019 at 05:59:04PM +0100, Jan Beulich wrote:
>>>> On 25.10.2019 11:16, Wei Liu wrote:
>>>>> +static inline uint64_t read_hyperv_timer(void)
>>>>> +{
>>>>> +    uint64_t scale, offset, ret, tsc;
>>>>> +    uint32_t seq;
>>>>> +    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;
>>>>> +
>>>>> +    do {
>>>>> +        seq = tsc_page->tsc_sequence;
>>>>> +
>>>>> +        /* Seq 0 is special. It means the TSC enlightenment is not
>>>>> +         * available at the moment. The reference time can only be
>>>>> +         * obtained from the Reference Counter MSR.
>>>>> +         */
>>>>> +        if ( seq == 0 )
>>>>> +        {
>>>>> +            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
>>>>> +            return ret;
>>>>> +        }
>>>>> +
>>>>> +        smp_rmb();
>>>>> +
>>>>> +        tsc = rdtsc_ordered();
>>>>
>>>> This already includes at least a read fence.
>>>
>>> OK. rdtsc() should be enough here.
>>
>> Are you sure? My comment was rather towards the dropping of smp_rmb()
>> (maybe replacing by a comment).
> 
> I do mean to keep smp_rmb() before it. Is that not enough?

With

#define smp_rmb()       barrier()

it isn't - it's merely a compiler barrier, but for the ordering
you want you need a fence.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Jan Beulich]

On 18.12.2019 13:56, Andrew Cooper wrote:
> On 18/12/2019 12:51, Jan Beulich wrote:
>>>>> +        scale = tsc_page->tsc_scale;
>>>>> +        offset = tsc_page->tsc_offset;
>>>>> +
>>>>> +        smp_rmb();
>>>>> +
>>>>> +    } while (tsc_page->tsc_sequence != seq);
>>>>> +
>>>>> +    /* x86 has ARCH_SUPPORTS_INT128 */
>>>>> +    ret = (uint64_t)(((__uint128_t)tsc * scale) >> 64) + offset;
>>>> The final cast isn't really needed, is it? As to the multiplication
>>>> - are you sure all compilers in all cases will avoid falling back
>>>> to a library call here? In other similar places I think we use
>>>> inline assembly instead.
>>> What library call? A function in libgcc (or clang's equivalence)?
>>> ISTR libgcc is always linked, but I could be wrong here.
>> No, the hypervisor (at least the x86 one) doesn't link libgcc afaik.
> 
> x86 can multiply uint128_t by uint64_t without a library call.  If this
> compiles, then it should be fine.

Hmm, a quick test proves what you say, but I'm uncertain relying
on it is a good idea. Especially with -Os the compiler _should_
really emit a library call.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next 7/7] x86: implement Hyper-V clock source

[Wei Liu]

On Wed, Dec 18, 2019 at 02:24:33PM +0100, Jan Beulich wrote:
> On 18.12.2019 14:18, Wei Liu wrote:
> > On Wed, Dec 18, 2019 at 01:51:54PM +0100, Jan Beulich wrote:
> >> On 18.12.2019 13:38, Wei Liu wrote:
> >>> On Tue, Dec 10, 2019 at 05:59:04PM +0100, Jan Beulich wrote:
> >>>> On 25.10.2019 11:16, Wei Liu wrote:
> >>>>> +static inline uint64_t read_hyperv_timer(void)
> >>>>> +{
> >>>>> +    uint64_t scale, offset, ret, tsc;
> >>>>> +    uint32_t seq;
> >>>>> +    struct ms_hyperv_tsc_page *tsc_page = &hyperv_tsc_page;
> >>>>> +
> >>>>> +    do {
> >>>>> +        seq = tsc_page->tsc_sequence;
> >>>>> +
> >>>>> +        /* Seq 0 is special. It means the TSC enlightenment is not
> >>>>> +         * available at the moment. The reference time can only be
> >>>>> +         * obtained from the Reference Counter MSR.
> >>>>> +         */
> >>>>> +        if ( seq == 0 )
> >>>>> +        {
> >>>>> +            rdmsrl(HV_X64_MSR_TIME_REF_COUNT, ret);
> >>>>> +            return ret;
> >>>>> +        }
> >>>>> +
> >>>>> +        smp_rmb();
> >>>>> +
> >>>>> +        tsc = rdtsc_ordered();
> >>>>
> >>>> This already includes at least a read fence.
> >>>
> >>> OK. rdtsc() should be enough here.
> >>
> >> Are you sure? My comment was rather towards the dropping of smp_rmb()
> >> (maybe replacing by a comment).
> > 
> > I do mean to keep smp_rmb() before it. Is that not enough?
> 
> With
> 
> #define smp_rmb()       barrier()
> 
> it isn't - it's merely a compiler barrier, but for the ordering
> you want you need a fence.

Ah, I see. Thank you.

Wei.

> 
> Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel