From: Michael Kelley <mikelley@microsoft.com>
To: Boqun Feng <boqun.feng@gmail.com>
Cc: Stephen Hemminger <sthemmin@microsoft.com>,
KY Srinivasan <kys@microsoft.com>,
"wei.liu@kernel.org" <wei.liu@kernel.org>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"mingo@redhat.com" <mingo@redhat.com>,
"bp@alien8.de" <bp@alien8.de>, "hpa@zytor.com" <hpa@zytor.com>,
"daniel.lezcano@linaro.org" <daniel.lezcano@linaro.org>,
"arnd@arndb.de" <arnd@arndb.de>,
"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"x86@kernel.org" <x86@kernel.org>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>
Subject: RE: [PATCH 09/10] clocksource/drivers/hyper-v: Set clocksource rating based on Hyper-V feature
Date: Mon, 22 Feb 2021 22:48:46 +0000 [thread overview]
Message-ID: <MWHPR21MB159313CF48D9124805E4A694D7819@MWHPR21MB1593.namprd21.prod.outlook.com> (raw)
In-Reply-To: <YDPVRy+QnZzoM+eF@boqun-archlinux>
From: Boqun Feng <boqun.feng@gmail.com> Sent: Monday, February 22, 2021 8:01 AM
>
> On Wed, Jan 27, 2021 at 12:23:44PM -0800, Michael Kelley wrote:
> > On x86/x64, the TSC clocksource is available in a Hyper-V VM only if
> > Hyper-V provides the TSC_INVARIANT flag. The rating on the Hyper-V
> > Reference TSC page clocksource is currently set so that it will not
> > override the TSC clocksource in this case. Alternatively, if the TSC
> > clocksource is not available, then the Hyper-V clocksource is used.
> >
> > But on ARM64, the Hyper-V Reference TSC page clocksource should
> > override the ARM arch counter, since the Hyper-V clocksource provides
> > scaling and offsetting during live migrations that is not provided
> > for the ARM arch counter.
> >
> > To get the needed behavior for both x86/x64 and ARM64, tweak the
> > logic by defaulting the Hyper-V Reference TSC page clocksource
> > rating to a large value that will always override. If the Hyper-V
> > TSC_INVARIANT flag is set, then reduce the rating so that it will not
> > override the TSC.
> >
> > While the logic for getting there is slightly different, the net
> > result in the normal cases is no functional change.
> >
>
> One question here, please see below:
>
> > Signed-off-by: Michael Kelley <mikelley@microsoft.com>
> > ---
> > drivers/clocksource/hyperv_timer.c | 23 +++++++++++++----------
> > 1 file changed, 13 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
> > index a2bee50..edf2d43 100644
> > --- a/drivers/clocksource/hyperv_timer.c
> > +++ b/drivers/clocksource/hyperv_timer.c
> > @@ -302,14 +302,6 @@ void hv_stimer_global_cleanup(void)
> > * the other that uses the TSC reference page feature as defined in the
> > * TLFS. The MSR version is for compatibility with old versions of
> > * Hyper-V and 32-bit x86. The TSC reference page version is preferred.
> > - *
> > - * The Hyper-V clocksource ratings of 250 are chosen to be below the
> > - * TSC clocksource rating of 300. In configurations where Hyper-V offers
> > - * an InvariantTSC, the TSC is not marked "unstable", so the TSC clocksource
> > - * is available and preferred. With the higher rating, it will be the
> > - * default. On older hardware and Hyper-V versions, the TSC is marked
> > - * "unstable", so no TSC clocksource is created and the selected Hyper-V
> > - * clocksource will be the default.
> > */
> >
> > u64 (*hv_read_reference_counter)(void);
> > @@ -380,7 +372,7 @@ static int hv_cs_enable(struct clocksource *cs)
> >
> > static struct clocksource hyperv_cs_tsc = {
> > .name = "hyperv_clocksource_tsc_page",
> > - .rating = 250,
> > + .rating = 500,
> > .read = read_hv_clock_tsc_cs,
> > .mask = CLOCKSOURCE_MASK(64),
> > .flags = CLOCK_SOURCE_IS_CONTINUOUS,
> > @@ -417,7 +409,7 @@ static u64 notrace read_hv_sched_clock_msr(void)
> >
> > static struct clocksource hyperv_cs_msr = {
> > .name = "hyperv_clocksource_msr",
> > - .rating = 250,
> > + .rating = 500,
>
> Before this patch, since the ".rating" of hyper_cs_msr is 250 which is
> smaller than the TSC clocksource rating, the TSC clocksource is better.
> After this patch, in the case where HV_MSR_REFERENCE_TSC_AVAILABLE bit
> is 0, we make hyperv_cs_msr better than the TSC clocksource (and we
> don't lower the rating of hyperv_cs_msr if TSC_INVARIANT is not
> offered), right? Could you explain why we need the change? Or maybe I'm
> missing something?
>
You make a good point. The code path that sets hyperv_cs_tsc.rating
to 250 should also be setting hyperv_cs_msr.rating to 250. The reality
is that the hyperv_cs_msr clock is a backup that is never used under
normal circumstances, so I didn't pay careful attention to that case.
I'll fix it.
Michael
>
> Regards,
> Boqun
>
> > .read = read_hv_clock_msr_cs,
> > .mask = CLOCKSOURCE_MASK(64),
> > .flags = CLOCK_SOURCE_IS_CONTINUOUS,
> > @@ -452,6 +444,17 @@ static bool __init hv_init_tsc_clocksource(void)
> > if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE))
> > return false;
> >
> > + /*
> > + * If Hyper-V offers TSC_INVARIANT, then the virtualized TSC correctly
> > + * handles frequency and offset changes due to live migration,
> > + * pause/resume, and other VM management operations. So lower the
> > + * Hyper-V Reference TSC rating, causing the generic TSC to be used.
> > + * TSC_INVARIANT is not offered on ARM64, so the Hyper-V Reference
> > + * TSC will be preferred over the virtualized ARM64 arch counter.
> > + */
> > + if (ms_hyperv.features & HV_ACCESS_TSC_INVARIANT)
> > + hyperv_cs_tsc.rating = 250;
> > +
> > hv_read_reference_counter = read_hv_clock_tsc;
> > phys_addr = virt_to_phys(hv_get_tsc_page());
> >
> > --
> > 1.8.3.1
> >
next prev parent reply other threads:[~2021-02-22 22:50 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-27 20:23 [PATCH 00/10] Refactor arch specific Hyper-V code Michael Kelley
2021-01-27 20:23 ` [PATCH 01/10] Drivers: hv: vmbus: Move Hyper-V page allocator to arch neutral code Michael Kelley
2021-02-22 3:04 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 02/10] x86/hyper-v: Move hv_message_type to architecture neutral module Michael Kelley
2021-02-22 3:19 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 03/10] Drivers: hv: Redo Hyper-V synthetic MSR get/set functions Michael Kelley
2021-02-22 3:25 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 04/10] Drivers: hv: vmbus: Move hyperv_report_panic_msg to arch neutral code Michael Kelley
2021-02-22 3:27 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 05/10] Drivers: hv: vmbus: Handle auto EOI quirk inline Michael Kelley
2021-02-22 3:30 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 06/10] Drivers: hv: vmbus: Move handling of VMbus interrupts Michael Kelley
2021-02-22 3:54 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 07/10] clocksource/drivers/hyper-v: Handle vDSO differences inline Michael Kelley
2021-02-22 4:07 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 08/10] clocksource/drivers/hyper-v: Handle sched_clock " Michael Kelley
2021-02-01 18:55 ` Wei Liu
2021-02-04 16:28 ` Michael Kelley
2021-02-04 16:31 ` Wei Liu
2021-02-22 15:17 ` Boqun Feng
2021-01-27 20:23 ` [PATCH 09/10] clocksource/drivers/hyper-v: Set clocksource rating based on Hyper-V feature Michael Kelley
2021-02-22 16:01 ` Boqun Feng
2021-02-22 22:48 ` Michael Kelley [this message]
2021-01-27 20:23 ` [PATCH 10/10] clocksource/drivers/hyper-v: Move handling of STIMER0 interrupts Michael Kelley
2021-02-01 19:53 ` Wei Liu
2021-02-04 16:30 ` Michael Kelley
2021-02-23 6:47 ` Boqun Feng
2021-02-25 18:56 ` Michael Kelley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=MWHPR21MB159313CF48D9124805E4A694D7819@MWHPR21MB1593.namprd21.prod.outlook.com \
--to=mikelley@microsoft.com \
--cc=arnd@arndb.de \
--cc=boqun.feng@gmail.com \
--cc=bp@alien8.de \
--cc=daniel.lezcano@linaro.org \
--cc=hpa@zytor.com \
--cc=kys@microsoft.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=sthemmin@microsoft.com \
--cc=tglx@linutronix.de \
--cc=wei.liu@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).