From: Joao Martins <joao.m.martins@oracle.com> To: xen-devel@lists.xen.org Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Joao Martins <joao.m.martins@oracle.com>, Keir Fraser <keir@xen.org>, Jan Beulich <jbeulich@suse.com> Subject: [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT Date: Thu, 17 Mar 2016 09:12:16 -0700 (PDT) [thread overview] Message-ID: <1458231136-13457-6-git-send-email-joao.m.martins@oracle.com> (raw) In-Reply-To: <1458231136-13457-1-git-send-email-joao.m.martins@oracle.com> When using TSC as clocksource we will solely rely on TSC for updating vcpu time infos (pvti). Right now, each vCPU takes the tsc_timestamp at different instants meaning every EPOCH + delta. This delta is variable depending on the time the CPU calibrates with CPU 0 (master), and will likely be different and variable across vCPUS. This means that each VCPU pvti won't account to its calibration error which could lead to time going backwards, and allowing a situation where time read on VCPU B immediately after A being smaller. While this doesn't happen a lot, I was able to observe (for clocksource=tsc) around 50 times in an hour having warps of < 100 ns. This patch proposes relying on host TSC synchronization and passthrough of the master tsc to the guest, when running on a TSC-safe platform. On the rendezvous function we will retrieve the platform time in ns and the last count read by the clocksource that was used to compute system time. master will write both master_tsc_stamp and master_stime, and the other vCPUS (slave) will use it to update their correspondent time infos. This way we can guarantee that on a platform with a constant and reliable TSC, that the time read on vcpu B right after A is bigger independently of the VCPU calibration error. Since pvclock time infos are monotonic as seen by any vCPU set PVCLOCK_TSC_STABLE_BIT, which then enables usage of VDSO on Linux. IIUC, this is similar to how it's implemented on KVM. Signed-off-by: Joao Martins <joao.m.martins@oracle.com> --- Cc: Keir Fraser <keir@xen.org> Cc: Jan Beulich <jbeulich@suse.com> Cc: Andrew Cooper <andrew.cooper3@citrix.com> --- xen/arch/x86/time.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c index 89c35d0..a17529c 100644 --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -917,6 +917,8 @@ static void __update_vcpu_system_time(struct vcpu *v, int force) _u.tsc_timestamp = tsc_stamp; _u.system_time = t->stime_local_stamp; + if ( clocksource_is_tsc ) + _u.flags |= PVCLOCK_TSC_STABLE_BIT; if ( is_hvm_domain(d) ) _u.tsc_timestamp += v->arch.hvm_vcpu.cache_tsc_offset; @@ -1377,9 +1379,12 @@ static void time_calibration_std_rendezvous(void *_r) if ( smp_processor_id() == 0 ) { + u64 last_counter; while ( atomic_read(&r->semaphore) != (total_cpus - 1) ) cpu_relax(); - r->master_stime = read_platform_stime(); + r->master_stime = read_platform_stime(&last_counter); + if ( clocksource_is_tsc ) + r->master_tsc_stamp = last_counter; mb(); /* write r->master_stime /then/ signal */ atomic_inc(&r->semaphore); } @@ -1391,7 +1396,10 @@ static void time_calibration_std_rendezvous(void *_r) mb(); /* receive signal /then/ read r->master_stime */ } - c->local_tsc_stamp = rdtsc(); + if ( clocksource_is_tsc ) + c->local_tsc_stamp = r->master_tsc_stamp; + else + c->local_tsc_stamp = rdtsc(); c->stime_local_stamp = get_s_time(); c->stime_master_stamp = r->master_stime; -- 2.1.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-03-17 16:12 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-03-17 16:12 [PATCH 0/5] x86/time: PVCLOCK_TSC_STABLE_BIT support Joao Martins 2016-03-17 16:12 ` [PATCH 1/5] public/xen.h: add flags field to vcpu_time_info Joao Martins 2016-03-18 20:12 ` Andrew Cooper 2016-03-21 11:42 ` Joao Martins 2016-03-21 11:43 ` Andrew Cooper 2016-03-21 11:51 ` Joao Martins 2016-03-21 15:10 ` Jan Beulich 2016-03-21 15:27 ` Andrew Cooper 2016-03-21 15:40 ` Joao Martins 2016-03-17 16:12 ` [PATCH 2/5] x86/time: implement tsc as clocksource Joao Martins 2016-03-18 20:21 ` Andrew Cooper 2016-03-21 11:43 ` Joao Martins 2016-03-22 12:41 ` Joao Martins 2016-03-22 12:46 ` Jan Beulich 2016-03-22 15:51 ` Joao Martins 2016-03-22 16:02 ` Jan Beulich 2016-03-22 20:40 ` Joao Martins 2016-03-23 7:28 ` Jan Beulich 2016-03-23 12:05 ` Joao Martins 2016-03-23 14:05 ` Jan Beulich 2016-03-17 16:12 ` [PATCH 3/5] x86/time: streamline platform time init on plt_init() Joao Martins 2016-03-18 20:32 ` Andrew Cooper 2016-03-21 11:45 ` Joao Martins 2016-03-17 16:12 ` [PATCH 4/5] x86/time: refactor read_platform_stime() Joao Martins 2016-03-18 20:34 ` Andrew Cooper 2016-03-21 11:45 ` Joao Martins 2016-03-21 13:08 ` Andrew Cooper 2016-03-17 16:12 ` Joao Martins [this message] 2016-03-18 20:58 ` [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT Andrew Cooper 2016-03-21 11:50 ` Joao Martins
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1458231136-13457-6-git-send-email-joao.m.martins@oracle.com \ --to=joao.m.martins@oracle.com \ --cc=andrew.cooper3@citrix.com \ --cc=jbeulich@suse.com \ --cc=keir@xen.org \ --cc=xen-devel@lists.xen.org \ --subject='Re: [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).