Re: [PATCH 09/10] clocksource/drivers/hyper-v: Set clocksource rating based on Hyper-V feature

From: Boqun Feng <boqun.feng@gmail.com>
To: Michael Kelley <mikelley@microsoft.com>
Cc: sthemmin@microsoft.com, kys@microsoft.com, wei.liu@kernel.org,
	tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
	hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de,
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
	x86@kernel.org, linux-arch@vger.kernel.org
Subject: Re: [PATCH 09/10] clocksource/drivers/hyper-v: Set clocksource rating based on Hyper-V feature
Date: Tue, 23 Feb 2021 00:01:11 +0800	[thread overview]
Message-ID: <YDPVRy+QnZzoM+eF@boqun-archlinux> (raw)
In-Reply-To: <1611779025-21503-10-git-send-email-mikelley@microsoft.com>

On Wed, Jan 27, 2021 at 12:23:44PM -0800, Michael Kelley wrote:
> On x86/x64, the TSC clocksource is available in a Hyper-V VM only if
> Hyper-V provides the TSC_INVARIANT flag. The rating on the Hyper-V
> Reference TSC page clocksource is currently set so that it will not
> override the TSC clocksource in this case.  Alternatively, if the TSC
> clocksource is not available, then the Hyper-V clocksource is used.
> 
> But on ARM64, the Hyper-V Reference TSC page clocksource should
> override the ARM arch counter, since the Hyper-V clocksource provides
> scaling and offsetting during live migrations that is not provided
> for the ARM arch counter.
> 
> To get the needed behavior for both x86/x64 and ARM64, tweak the
> logic by defaulting the Hyper-V Reference TSC page clocksource
> rating to a large value that will always override.  If the Hyper-V
> TSC_INVARIANT flag is set, then reduce the rating so that it will not
> override the TSC.
> 
> While the logic for getting there is slightly different, the net
> result in the normal cases is no functional change.
> 

One question here, please see below:

> Signed-off-by: Michael Kelley <mikelley@microsoft.com>
> ---
>  drivers/clocksource/hyperv_timer.c | 23 +++++++++++++----------
>  1 file changed, 13 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
> index a2bee50..edf2d43 100644
> --- a/drivers/clocksource/hyperv_timer.c
> +++ b/drivers/clocksource/hyperv_timer.c
> @@ -302,14 +302,6 @@ void hv_stimer_global_cleanup(void)
>   * the other that uses the TSC reference page feature as defined in the
>   * TLFS.  The MSR version is for compatibility with old versions of
>   * Hyper-V and 32-bit x86.  The TSC reference page version is preferred.
> - *
> - * The Hyper-V clocksource ratings of 250 are chosen to be below the
> - * TSC clocksource rating of 300.  In configurations where Hyper-V offers
> - * an InvariantTSC, the TSC is not marked "unstable", so the TSC clocksource
> - * is available and preferred.  With the higher rating, it will be the
> - * default.  On older hardware and Hyper-V versions, the TSC is marked
> - * "unstable", so no TSC clocksource is created and the selected Hyper-V
> - * clocksource will be the default.
>   */
>  
>  u64 (*hv_read_reference_counter)(void);
> @@ -380,7 +372,7 @@ static int hv_cs_enable(struct clocksource *cs)
>  
>  static struct clocksource hyperv_cs_tsc = {
>  	.name	= "hyperv_clocksource_tsc_page",
> -	.rating	= 250,
> +	.rating	= 500,
>  	.read	= read_hv_clock_tsc_cs,
>  	.mask	= CLOCKSOURCE_MASK(64),
>  	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
> @@ -417,7 +409,7 @@ static u64 notrace read_hv_sched_clock_msr(void)
>  
>  static struct clocksource hyperv_cs_msr = {
>  	.name	= "hyperv_clocksource_msr",
> -	.rating	= 250,
> +	.rating	= 500,

Before this patch, since the ".rating" of hyper_cs_msr is 250 which is
smaller than the TSC clocksource rating, the TSC clocksource is better.
After this patch, in the case where HV_MSR_REFERENCE_TSC_AVAILABLE bit
is 0, we make hyperv_cs_msr better than the TSC clocksource (and we
don't lower the rating of hyperv_cs_msr if TSC_INVARIANT is not
offered), right?  Could you explain why we need the change? Or maybe I'm
missing something?

Regards,
Boqun

>  	.read	= read_hv_clock_msr_cs,
>  	.mask	= CLOCKSOURCE_MASK(64),
>  	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
> @@ -452,6 +444,17 @@ static bool __init hv_init_tsc_clocksource(void)
>  	if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE))
>  		return false;
>  
> +	/*
> +	 * If Hyper-V offers TSC_INVARIANT, then the virtualized TSC correctly
> +	 * handles frequency and offset changes due to live migration,
> +	 * pause/resume, and other VM management operations.  So lower the
> +	 * Hyper-V Reference TSC rating, causing the generic TSC to be used.
> +	 * TSC_INVARIANT is not offered on ARM64, so the Hyper-V Reference
> +	 * TSC will be preferred over the virtualized ARM64 arch counter.
> +	 */
> +	if (ms_hyperv.features & HV_ACCESS_TSC_INVARIANT)
> +		hyperv_cs_tsc.rating = 250;
> +
>  	hv_read_reference_counter = read_hv_clock_tsc;
>  	phys_addr = virt_to_phys(hv_get_tsc_page());
>  
> -- 
> 1.8.3.1
> 