* [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC
@ 2019-10-03 15:52 Andrea Parri
2019-10-03 16:03 ` Andrea Parri
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Andrea Parri @ 2019-10-03 15:52 UTC (permalink / raw)
To: linux-kernel, linux-hyperv, x86
Cc: K . Y . Srinivasan, Haiyang Zhang, Stephen Hemminger,
Sasha Levin, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H . Peter Anvin, Michael Kelley, Andrea Parri
If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
configuration version. Bit 15 corresponds to the
AccessTscInvariantControls privilege. If this privilege bit is set,
guests can access the HvSyntheticInvariantTscControl MSR: guests can
set bit 0 of this synthetic MSR to enable the InvariantTSC feature.
After setting the synthetic MSR, CPUID will enumerate support for
InvariantTSC.
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
arch/x86/include/asm/hyperv-tlfs.h | 5 +++++
arch/x86/kernel/cpu/mshyperv.c | 7 ++++++-
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index 7741e211f7f51..5f10f7f2098db 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -86,6 +86,8 @@
#define HV_X64_ACCESS_FREQUENCY_MSRS BIT(11)
/* AccessReenlightenmentControls privilege */
#define HV_X64_ACCESS_REENLIGHTENMENT BIT(13)
+/* AccessTscInvariantControls privilege */
+#define HV_X64_ACCESS_TSC_INVARIANT BIT(15)
/*
* Feature identification: indicates which flags were specified at partition
@@ -278,6 +280,9 @@
#define HV_X64_MSR_TSC_EMULATION_CONTROL 0x40000107
#define HV_X64_MSR_TSC_EMULATION_STATUS 0x40000108
+/* TSC invariant control */
+#define HV_X64_MSR_TSC_INVARIANT_CONTROL 0x40000118
+
/*
* Declare the MSR used to setup pages used to communicate with the hypervisor.
*/
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 267daad8c0360..105844d542e5c 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -286,7 +286,12 @@ static void __init ms_hyperv_init_platform(void)
machine_ops.shutdown = hv_machine_shutdown;
machine_ops.crash_shutdown = hv_machine_crash_shutdown;
#endif
- mark_tsc_unstable("running on Hyper-V");
+ if (ms_hyperv.features & HV_X64_ACCESS_TSC_INVARIANT) {
+ wrmsrl(HV_X64_MSR_TSC_INVARIANT_CONTROL, 0x1);
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+ } else {
+ mark_tsc_unstable("running on Hyper-V");
+ }
/*
* Generation 2 instances don't support reading the NMI status from
--
2.23.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC
2019-10-03 15:52 [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC Andrea Parri
@ 2019-10-03 16:03 ` Andrea Parri
2019-10-03 23:15 ` Michael Kelley
2019-10-04 16:57 ` Vitaly Kuznetsov
2 siblings, 0 replies; 6+ messages in thread
From: Andrea Parri @ 2019-10-03 16:03 UTC (permalink / raw)
To: linux-kernel, linux-hyperv, x86
Cc: K . Y . Srinivasan, Haiyang Zhang, Stephen Hemminger,
Sasha Levin, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H . Peter Anvin, Michael Kelley
On Thu, Oct 03, 2019 at 05:52:00PM +0200, Andrea Parri wrote:
> If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> configuration version. Bit 15 corresponds to the
> AccessTscInvariantControls privilege. If this privilege bit is set,
> guests can access the HvSyntheticInvariantTscControl MSR: guests can
> set bit 0 of this synthetic MSR to enable the InvariantTSC feature.
> After setting the synthetic MSR, CPUID will enumerate support for
> InvariantTSC.
>
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Subject should have been "[PATCH] ...", i.e., there is no 2/2 planned
(not for this patchset at least). Please let me know if I should re-
submit with the subject fixed.
Thanks,
Andrea
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC
2019-10-03 15:52 [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC Andrea Parri
2019-10-03 16:03 ` Andrea Parri
@ 2019-10-03 23:15 ` Michael Kelley
2019-10-04 16:57 ` Vitaly Kuznetsov
2 siblings, 0 replies; 6+ messages in thread
From: Michael Kelley @ 2019-10-03 23:15 UTC (permalink / raw)
To: Andrea Parri, linux-kernel, linux-hyperv, x86
Cc: KY Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H . Peter Anvin
From: Andrea Parri <parri.andrea@gmail.com> Sent: Thursday, October 3, 2019 8:52 AM
>
> If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> configuration version. Bit 15 corresponds to the
> AccessTscInvariantControls privilege. If this privilege bit is set,
> guests can access the HvSyntheticInvariantTscControl MSR: guests can
> set bit 0 of this synthetic MSR to enable the InvariantTSC feature.
> After setting the synthetic MSR, CPUID will enumerate support for
> InvariantTSC.
>
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
> arch/x86/include/asm/hyperv-tlfs.h | 5 +++++
> arch/x86/kernel/cpu/mshyperv.c | 7 ++++++-
> 2 files changed, 11 insertions(+), 1 deletion(-)
>
As noted in a separate email, this patch is standalone, not 1 of 2 as
indicated in the subject line. Modulo that,
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC
2019-10-03 15:52 [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC Andrea Parri
2019-10-03 16:03 ` Andrea Parri
2019-10-03 23:15 ` Michael Kelley
@ 2019-10-04 16:57 ` Vitaly Kuznetsov
2019-10-04 22:05 ` Michael Kelley
2 siblings, 1 reply; 6+ messages in thread
From: Vitaly Kuznetsov @ 2019-10-04 16:57 UTC (permalink / raw)
To: Andrea Parri, linux-kernel, linux-hyperv, x86
Cc: K . Y . Srinivasan, Haiyang Zhang, Stephen Hemminger,
Sasha Levin, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H . Peter Anvin, Michael Kelley, Andrea Parri
Andrea Parri <parri.andrea@gmail.com> writes:
> If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> configuration version. Bit 15 corresponds to the
> AccessTscInvariantControls privilege. If this privilege bit is set,
> guests can access the HvSyntheticInvariantTscControl MSR: guests can
> set bit 0 of this synthetic MSR to enable the InvariantTSC feature.
> After setting the synthetic MSR, CPUID will enumerate support for
> InvariantTSC.
I tried getting more information from TLFS but as of 5.0C this feature
is not described there. I'm really interested in why this additional
interface is needed, e.g. why can't Hyper-V just set InvariantTSC
unconditionally when TSC scaling is supported?
>
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
> arch/x86/include/asm/hyperv-tlfs.h | 5 +++++
> arch/x86/kernel/cpu/mshyperv.c | 7 ++++++-
> 2 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
> index 7741e211f7f51..5f10f7f2098db 100644
> --- a/arch/x86/include/asm/hyperv-tlfs.h
> +++ b/arch/x86/include/asm/hyperv-tlfs.h
> @@ -86,6 +86,8 @@
> #define HV_X64_ACCESS_FREQUENCY_MSRS BIT(11)
> /* AccessReenlightenmentControls privilege */
> #define HV_X64_ACCESS_REENLIGHTENMENT BIT(13)
> +/* AccessTscInvariantControls privilege */
> +#define HV_X64_ACCESS_TSC_INVARIANT BIT(15)
>
> /*
> * Feature identification: indicates which flags were specified at partition
> @@ -278,6 +280,9 @@
> #define HV_X64_MSR_TSC_EMULATION_CONTROL 0x40000107
> #define HV_X64_MSR_TSC_EMULATION_STATUS 0x40000108
>
> +/* TSC invariant control */
> +#define HV_X64_MSR_TSC_INVARIANT_CONTROL 0x40000118
> +
> /*
> * Declare the MSR used to setup pages used to communicate with the hypervisor.
> */
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 267daad8c0360..105844d542e5c 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -286,7 +286,12 @@ static void __init ms_hyperv_init_platform(void)
> machine_ops.shutdown = hv_machine_shutdown;
> machine_ops.crash_shutdown = hv_machine_crash_shutdown;
> #endif
> - mark_tsc_unstable("running on Hyper-V");
> + if (ms_hyperv.features & HV_X64_ACCESS_TSC_INVARIANT) {
> + wrmsrl(HV_X64_MSR_TSC_INVARIANT_CONTROL, 0x1);
> + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
> + } else {
> + mark_tsc_unstable("running on Hyper-V");
> + }
>
> /*
> * Generation 2 instances don't support reading the NMI status from
--
Vitaly
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC
2019-10-04 16:57 ` Vitaly Kuznetsov
@ 2019-10-04 22:05 ` Michael Kelley
2019-10-08 15:12 ` Vitaly Kuznetsov
0 siblings, 1 reply; 6+ messages in thread
From: Michael Kelley @ 2019-10-04 22:05 UTC (permalink / raw)
To: vkuznets, Andrea Parri, linux-kernel, linux-hyperv, x86
Cc: KY Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H . Peter Anvin,
Andrea Parri
From: Vitaly Kuznetsov <vkuznets@redhat.com> Sent: Friday, October 4, 2019 9:57 AM
>
> Andrea Parri <parri.andrea@gmail.com> writes:
>
> > If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> > HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> > configuration version. Bit 15 corresponds to the
> > AccessTscInvariantControls privilege. If this privilege bit is set,
> > guests can access the HvSyntheticInvariantTscControl MSR: guests can
> > set bit 0 of this synthetic MSR to enable the InvariantTSC feature.
> > After setting the synthetic MSR, CPUID will enumerate support for
> > InvariantTSC.
>
> I tried getting more information from TLFS but as of 5.0C this feature
> is not described there. I'm really interested in why this additional
> interface is needed, e.g. why can't Hyper-V just set InvariantTSC
> unconditionally when TSC scaling is supported?
>
Yes, this is very new functionality that is not yet available in a released
version of Hyper-V. And as you know, the Hyper-V TLFS has gotten
woefully out-of-date. :-(
Your question is the same question I asked. The reason given by
Hyper-V is to take the more cautious approach of not "automatically"
giving VMs an InvariantTSC due to updating the underlying Hyper-V
version. Instead, guest VMs must have been explicitly coded to take
advantage of the new InvariantTSC feature. It's not clear to me how
much of this caution is driven by Windows guests vs. Linux or FreeBSD
guests, but it is what it is.
Having to explicitly enable the InvariantTSC does give the Linux code
the opportunity to be a bit cleaner by doing things like not marking
the TSC as unstable when the InvariantTSC feature is present, and to
mark the TSC as reliable so we don't try to do TSC synchronization
(which Hyper-V does not want guests to try to do).
Michael
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC
2019-10-04 22:05 ` Michael Kelley
@ 2019-10-08 15:12 ` Vitaly Kuznetsov
0 siblings, 0 replies; 6+ messages in thread
From: Vitaly Kuznetsov @ 2019-10-08 15:12 UTC (permalink / raw)
To: Michael Kelley, Andrea Parri, linux-kernel, linux-hyperv, x86
Cc: KY Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H . Peter Anvin,
Andrea Parri
Michael Kelley <mikelley@microsoft.com> writes:
> From: Vitaly Kuznetsov <vkuznets@redhat.com> Sent: Friday, October 4, 2019 9:57 AM
>>
>> Andrea Parri <parri.andrea@gmail.com> writes:
>>
>> > If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
>> > HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
>> > configuration version. Bit 15 corresponds to the
>> > AccessTscInvariantControls privilege. If this privilege bit is set,
>> > guests can access the HvSyntheticInvariantTscControl MSR: guests can
>> > set bit 0 of this synthetic MSR to enable the InvariantTSC feature.
>> > After setting the synthetic MSR, CPUID will enumerate support for
>> > InvariantTSC.
>>
>> I tried getting more information from TLFS but as of 5.0C this feature
>> is not described there. I'm really interested in why this additional
>> interface is needed, e.g. why can't Hyper-V just set InvariantTSC
>> unconditionally when TSC scaling is supported?
>>
>
> Yes, this is very new functionality that is not yet available in a released
> version of Hyper-V. And as you know, the Hyper-V TLFS has gotten
> woefully out-of-date. :-(
>
> Your question is the same question I asked. The reason given by
> Hyper-V is to take the more cautious approach of not "automatically"
> giving VMs an InvariantTSC due to updating the underlying Hyper-V
> version. Instead, guest VMs must have been explicitly coded to take
> advantage of the new InvariantTSC feature. It's not clear to me how
> much of this caution is driven by Windows guests vs. Linux or FreeBSD
> guests, but it is what it is.
>
> Having to explicitly enable the InvariantTSC does give the Linux code
> the opportunity to be a bit cleaner by doing things like not marking
> the TSC as unstable when the InvariantTSC feature is present, and to
> mark the TSC as reliable so we don't try to do TSC synchronization
> (which Hyper-V does not want guests to try to do).
Thank you for these additional details Michael,
we'll probably have to add support for this bit to KVM and I'd like to
know the background. From Linux perspective, no matter what's the
interface we'd like to get InvariantTSC.
Feel free to add
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
--
Vitaly
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-10-08 15:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-03 15:52 [PATCH 1/2] x86/hyperv: Allow guests to enable InvariantTSC Andrea Parri
2019-10-03 16:03 ` Andrea Parri
2019-10-03 23:15 ` Michael Kelley
2019-10-04 16:57 ` Vitaly Kuznetsov
2019-10-04 22:05 ` Michael Kelley
2019-10-08 15:12 ` Vitaly Kuznetsov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).