From: Joao Martins <joao.m.martins@oracle.com> To: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Keir Fraser <keir@xen.org>, Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xen.org Subject: Re: [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT Date: Mon, 21 Mar 2016 11:50:05 +0000 [thread overview] Message-ID: <56EFDFED.3010809@oracle.com> (raw) In-Reply-To: <56EC6C04.5060608@citrix.com> On 03/18/2016 08:58 PM, Andrew Cooper wrote: > On 17/03/16 16:12, Joao Martins wrote: >> When using TSC as clocksource we will solely rely on TSC for updating >> vcpu time infos (pvti). Right now, each vCPU takes the tsc_timestamp at >> different instants meaning every EPOCH + delta. This delta is variable >> depending on the time the CPU calibrates with CPU 0 (master), and will >> likely be different and variable across vCPUS. This means that each VCPU >> pvti won't account to its calibration error which could lead to time >> going backwards, and allowing a situation where time read on VCPU B >> immediately after A being smaller. While this doesn't happen a lot, I >> was able to observe (for clocksource=tsc) around 50 times in an hour >> having warps of < 100 ns. >> >> This patch proposes relying on host TSC synchronization and passthrough >> of the master tsc to the guest, when running on a TSC-safe platform. On >> the rendezvous function we will retrieve the platform time in ns and the >> last count read by the clocksource that was used to compute system time. >> master will write both master_tsc_stamp and master_stime, and the other >> vCPUS (slave) will use it to update their correspondent time infos. >> This way we can guarantee that on a platform with a constant and >> reliable TSC, that the time read on vcpu B right after A is bigger >> independently of the VCPU calibration error. Since pvclock time infos >> are monotonic as seen by any vCPU set PVCLOCK_TSC_STABLE_BIT, which then >> enables usage of VDSO on Linux. IIUC, this is similar to how it's >> implemented on KVM. >> >> Signed-off-by: Joao Martins <joao.m.martins@oracle.com> >> --- >> Cc: Keir Fraser <keir@xen.org> >> Cc: Jan Beulich <jbeulich@suse.com> >> Cc: Andrew Cooper <andrew.cooper3@citrix.com> >> --- >> xen/arch/x86/time.c | 12 ++++++++++-- >> 1 file changed, 10 insertions(+), 2 deletions(-) >> >> diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c >> index 89c35d0..a17529c 100644 >> --- a/xen/arch/x86/time.c >> +++ b/xen/arch/x86/time.c >> @@ -917,6 +917,8 @@ static void __update_vcpu_system_time(struct vcpu *v, int force) >> >> _u.tsc_timestamp = tsc_stamp; >> _u.system_time = t->stime_local_stamp; >> + if ( clocksource_is_tsc ) >> + _u.flags |= PVCLOCK_TSC_STABLE_BIT; >> >> if ( is_hvm_domain(d) ) >> _u.tsc_timestamp += v->arch.hvm_vcpu.cache_tsc_offset; >> @@ -1377,9 +1379,12 @@ static void time_calibration_std_rendezvous(void *_r) >> >> if ( smp_processor_id() == 0 ) >> { >> + u64 last_counter; > > Blank line here please. > >> while ( atomic_read(&r->semaphore) != (total_cpus - 1) ) >> cpu_relax(); >> - r->master_stime = read_platform_stime(); >> + r->master_stime = read_platform_stime(&last_counter); >> + if ( clocksource_is_tsc ) >> + r->master_tsc_stamp = last_counter; >> mb(); /* write r->master_stime /then/ signal */ >> atomic_inc(&r->semaphore); >> } >> @@ -1391,7 +1396,10 @@ static void time_calibration_std_rendezvous(void *_r) >> mb(); /* receive signal /then/ read r->master_stime */ >> } >> >> - c->local_tsc_stamp = rdtsc(); >> + if ( clocksource_is_tsc ) >> + c->local_tsc_stamp = r->master_tsc_stamp; >> + else >> + c->local_tsc_stamp = rdtsc(); >> c->stime_local_stamp = get_s_time(); >> c->stime_master_stamp = r->master_stime; >> > > The point of the rendezvous is to run rdtsc() at a the time on each cpu > at the same time. With this logic, it seems that you don't need the > rendezvous at all. > > Avoiding the time_calibration_std_rendezvous() entirely in this > situation would be the better, surely? Indeed, and would look cleaner too. I've changed the approach in this patch for v2 following your guideline, along with some retesting. > > ~Andrew > _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
prev parent reply other threads:[~2016-03-21 11:50 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-03-17 16:12 [PATCH 0/5] x86/time: PVCLOCK_TSC_STABLE_BIT support Joao Martins 2016-03-17 16:12 ` [PATCH 1/5] public/xen.h: add flags field to vcpu_time_info Joao Martins 2016-03-18 20:12 ` Andrew Cooper 2016-03-21 11:42 ` Joao Martins 2016-03-21 11:43 ` Andrew Cooper 2016-03-21 11:51 ` Joao Martins 2016-03-21 15:10 ` Jan Beulich 2016-03-21 15:27 ` Andrew Cooper 2016-03-21 15:40 ` Joao Martins 2016-03-17 16:12 ` [PATCH 2/5] x86/time: implement tsc as clocksource Joao Martins 2016-03-18 20:21 ` Andrew Cooper 2016-03-21 11:43 ` Joao Martins 2016-03-22 12:41 ` Joao Martins 2016-03-22 12:46 ` Jan Beulich 2016-03-22 15:51 ` Joao Martins 2016-03-22 16:02 ` Jan Beulich 2016-03-22 20:40 ` Joao Martins 2016-03-23 7:28 ` Jan Beulich 2016-03-23 12:05 ` Joao Martins 2016-03-23 14:05 ` Jan Beulich 2016-03-17 16:12 ` [PATCH 3/5] x86/time: streamline platform time init on plt_init() Joao Martins 2016-03-18 20:32 ` Andrew Cooper 2016-03-21 11:45 ` Joao Martins 2016-03-17 16:12 ` [PATCH 4/5] x86/time: refactor read_platform_stime() Joao Martins 2016-03-18 20:34 ` Andrew Cooper 2016-03-21 11:45 ` Joao Martins 2016-03-21 13:08 ` Andrew Cooper 2016-03-17 16:12 ` [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT Joao Martins 2016-03-18 20:58 ` Andrew Cooper 2016-03-21 11:50 ` Joao Martins [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=56EFDFED.3010809@oracle.com \ --to=joao.m.martins@oracle.com \ --cc=andrew.cooper3@citrix.com \ --cc=jbeulich@suse.com \ --cc=keir@xen.org \ --cc=xen-devel@lists.xen.org \ --subject='Re: [PATCH 5/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).