xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Joao Martins <joao.m.martins@oracle.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Keir Fraser <keir@xen.org>,
	xen-devel@lists.xen.org
Subject: Re: [PATCH 2/5] x86/time: implement tsc as clocksource
Date: Tue, 22 Mar 2016 15:51:19 +0000	[thread overview]
Message-ID: <56F169F7.3080700@oracle.com> (raw)
In-Reply-To: <56F14CA202000078000DF28B@prv-mh.provo.novell.com>



On 03/22/2016 12:46 PM, Jan Beulich wrote:
>>>> On 22.03.16 at 13:41, <joao.m.martins@oracle.com> wrote:
> 
>>
>> On 03/18/2016 08:21 PM, Andrew Cooper wrote:
>>> On 17/03/16 16:12, Joao Martins wrote:
>>>> Introduce support for using TSC as platform time which is the highest
>>>> resolution time and most performant to get (~20 nsecs).  Though there
>>>> are also several problems associated with its usage, and there isn't a
>>>> complete (and architecturally defined) guarantee that all machines
>>>> will provide reliable and monotonic TSC across all CPUs, on different
>>>> sockets and different P/C states.  I believe Intel to be the only that
>>>> can guarantee that. For this reason it's set with less priority when
>>>> compared to HPET unless adminstrator changes "clocksource" boot option
>>>> to "tsc". Upon initializing it, we also check for time warps and
>>>> appropriate CPU features i.e.  TSC_RELIABLE, CONSTANT_TSC and
>>>> NONSTOP_TSC. And in case none of these conditions are met, we fail in
>>>> order to fallback to the next available clocksource.
>>>>
>>>> It is also worth noting that with clocksource=tsc there isn't any
>>>> need to synchronise with another clocksource, and I could verify that
>>>> great portion the time skew was eliminated and seeing much less time
>>>> warps happening. With HPET I observed ~500 warps in the period
>>>> of 1h of around 27 us, and with TSC down to 50 warps in the same
>>>> period having each warp < 100 ns. The warps still exist though but
>>>> are only related to cross CPU calibration (being how much it takes to
>>>> rendezvous with master), in which a later patch in this series aims to
>>>> solve.
>>>>
>>>> Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
>>>> ---
>>>> Cc: Keir Fraser <keir@xen.org>
>>>> Cc: Jan Beulich <jbeulich@suse.com>
>>>> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
>>>
>>> Some style corrections, but no functional problems.
>>>
>>> Reviewed-by Andrew Cooper <andrew.cooper3@citrix.com>
>>>
>> I found out one issue in the tsc clocksource initialization path with respect to
>> the reliability check. This check is running when initializing platform timer,
>> but not all CPUS are up at that point (init_xen_time()) which means that the
>> check will always succeed. So for clocksource=tsc I need to defer initialization
>> to a later point (on verify_tsc_reliability()) and if successful switch to that
>> clocksource. Unless you disagree, v2 would have this and just requires one
>> additional preparatory change prior to this patch.
> 
> Hmm, that's suspicious when thinking about CPU hotplug: What
> do you intend to do when a CPU comes online later, failing the
> check?
Good point, but I am not sure whether that would happen. The initcall
verify_tsc_reliability seems to be called only for the boot processor. Or are
you saying that it's case that initcalls are called too when hotplugging cpus
later on? If that's the case then my suggestion wouldn't work as you point out -
or rather without having runtime switching support as you point out below.

> So far we don't do runtime switching of the clock source,
> and I'm not sure that's a good idea to do when there are running
> guests.
Totally agree, but I would be proposing to be at initialization phase where
there wouldn't be guests running. We would start with HPET, then only switch to
TSC if that check has passed (and would happen once).

Simpler alternative would be to call init_xen_time after all CPUs are brought up
(and would also keep this patch as is), but I am not sure about the
repercussions of that.

Joao

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

  reply	other threads:[~2016-03-22 15:51 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-17 16:12 [PATCH 0/5] x86/time: PVCLOCK_TSC_STABLE_BIT support Joao Martins
2016-03-17 16:12 ` [PATCH 1/5] public/xen.h: add flags field to vcpu_time_info Joao Martins
2016-03-18 20:12   ` Andrew Cooper
2016-03-21 11:42     ` Joao Martins
2016-03-21 11:43       ` Andrew Cooper
2016-03-21 11:51         ` Joao Martins
2016-03-21 15:10   ` Jan Beulich
2016-03-21 15:27     ` Andrew Cooper
2016-03-21 15:40       ` Joao Martins
2016-03-17 16:12 ` [PATCH 2/5] x86/time: implement tsc as clocksource Joao Martins
2016-03-18 20:21   ` Andrew Cooper
2016-03-21 11:43     ` Joao Martins
2016-03-22 12:41     ` Joao Martins
2016-03-22 12:46       ` Jan Beulich
2016-03-22 15:51         ` Joao Martins [this message]
2016-03-22 16:02           ` Jan Beulich
2016-03-22 20:40             ` Joao Martins
2016-03-23  7:28               ` Jan Beulich
2016-03-23 12:05                 ` Joao Martins
2016-03-23 14:05                   ` Jan Beulich
2016-03-17 16:12 ` [PATCH 3/5] x86/time: streamline platform time init on plt_init() Joao Martins
2016-03-18 20:32   ` Andrew Cooper
2016-03-21 11:45     ` Joao Martins
2016-03-17 16:12 ` [PATCH 4/5] x86/time: refactor read_platform_stime() Joao Martins
2016-03-18 20:34   ` Andrew Cooper
2016-03-21 11:45     ` Joao Martins
2016-03-21 13:08       ` Andrew Cooper
2016-03-17 16:12 ` [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT Joao Martins
2016-03-18 20:58   ` Andrew Cooper
2016-03-21 11:50     ` Joao Martins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=56F169F7.3080700@oracle.com \
    --to=joao.m.martins@oracle.com \
    --cc=JBeulich@suse.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=keir@xen.org \
    --cc=xen-devel@lists.xen.org \
    --subject='Re: [PATCH 2/5] x86/time: implement tsc as clocksource' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).