linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
       [not found] <1411451081-13423-1-git-send-email-huishao@microsoft.com>
@ 2014-09-23  7:56 ` Olaf Hering
  2014-09-23  9:47   ` Thomas Shao
  2014-09-25  6:07   ` Sitsofe Wheeler
  0 siblings, 2 replies; 14+ messages in thread
From: Olaf Hering @ 2014-09-23  7:56 UTC (permalink / raw)
  To: Thomas Shao
  Cc: gregkh, linux-kernel, driverdev-devel, apw, jasowang, kys, haiyangz

On Tue, Sep 23, Thomas Shao wrote:

> In current hyper-v time sync service,it only gets the initial clock time
> from the host. It didn't process the following time samples. This change
> introduced a module parameter called host_time_sync. If it is set to true,
> the guest will periodically sychronize it's time with the host clock using
> host time sample. By default it is disabled, because we still recommend
> user to configure NTP for time synchronization.

Is that new knob a temporary thing to aid testing and development?
Because once it sneaks in it becomes some sort of ABIT.
So if the outcome of all that is that 'sync with host every once in a
while' will be the default, the new knob is not required.

Olaf

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-23  7:56 ` [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample Olaf Hering
@ 2014-09-23  9:47   ` Thomas Shao
  2014-09-25  6:07   ` Sitsofe Wheeler
  1 sibling, 0 replies; 14+ messages in thread
From: Thomas Shao @ 2014-09-23  9:47 UTC (permalink / raw)
  To: Olaf Hering
  Cc: gregkh, linux-kernel, driverdev-devel, apw, jasowang,
	KY Srinivasan, Haiyang Zhang

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1647 bytes --]


> -----Original Message-----
> From: Olaf Hering [mailto:olaf@aepfle.de]
> Sent: Tuesday, September 23, 2014 3:56 PM
> To: Thomas Shao
> Cc: gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> driverdev-devel@linuxdriverproject.org; apw@canonical.com;
> jasowang@redhat.com; KY Srinivasan; Haiyang Zhang
> Subject: Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using
> host time sample
> 
> On Tue, Sep 23, Thomas Shao wrote:
> 
> > In current hyper-v time sync service,it only gets the initial clock
> > time from the host. It didn't process the following time samples. This
> > change introduced a module parameter called host_time_sync. If it is
> > set to true, the guest will periodically sychronize it's time with the
> > host clock using host time sample. By default it is disabled, because
> > we still recommend user to configure NTP for time synchronization.
> 
> Is that new knob a temporary thing to aid testing and development?
> Because once it sneaks in it becomes some sort of ABIT.
> So if the outcome of all that is that 'sync with host every once in a while' will be
> the default, the new knob is not required.
> 

Hi Olaf, 
If the guest is configured with other time sync source, like NTP, we don't want the 
guest to periodically sync with host. This feature is useful when the guest didn't have
Internet connection to setup NTP. So the new knob is to allow users to only turn on the
host-guest sync when they need.

Regards
Thomas

> Olaf
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-23  7:56 ` [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample Olaf Hering
  2014-09-23  9:47   ` Thomas Shao
@ 2014-09-25  6:07   ` Sitsofe Wheeler
  2014-09-25  9:40     ` Thomas Shao
  1 sibling, 1 reply; 14+ messages in thread
From: Sitsofe Wheeler @ 2014-09-25  6:07 UTC (permalink / raw)
  To: Olaf Hering
  Cc: Thomas Shao, gregkh, linux-kernel, driverdev-devel, apw,
	jasowang, kys, haiyangz

On Tue, Sep 23, 2014 at 09:56:10AM +0200, Olaf Hering wrote:
> On Tue, Sep 23, Thomas Shao wrote:
> 
> > In current hyper-v time sync service,it only gets the initial clock time
> > from the host. It didn't process the following time samples. This change
> > introduced a module parameter called host_time_sync. If it is set to true,
> > the guest will periodically sychronize it's time with the host clock using
> > host time sample. By default it is disabled, because we still recommend
> > user to configure NTP for time synchronization.

You [Microsoft?] do? Can you link to public sources where is this stated
please? I don't see any mention of doing this on
http://technet.microsoft.com/en-us/library/dn720239.aspx .. The only
official Microsoft information with respect to Hyper-V guest time
synchronisation I've seen has been for Windows guests but perhaps I've
looked in the wrong places.

The reason I ask is because regular ntpd is not enough to discipline a
Linux Hyper-V guest's clock. So much drift can occur under load that
ntpd can't bring the clock under sync. For now, I've been using Chrony
which has a higher tolerance for correcting drifting clocks. I'm not the
only one seeing this either (see http://serverfault.com/a/488528/203726
and
http://serverfault.com/questions/523389/linux-clock-loses-10-minutes-every-week
).

It would be good to something official about this issue as it is painful
when happens.

-- 
Sitsofe | http://sucs.org/~sits/

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-25  6:07   ` Sitsofe Wheeler
@ 2014-09-25  9:40     ` Thomas Shao
  2014-09-25 13:28       ` Time keeps on slipping... on Hyper-V Sitsofe Wheeler
  0 siblings, 1 reply; 14+ messages in thread
From: Thomas Shao @ 2014-09-25  9:40 UTC (permalink / raw)
  To: Sitsofe Wheeler, Olaf Hering
  Cc: gregkh, linux-kernel, driverdev-devel, apw, jasowang,
	KY Srinivasan, Haiyang Zhang



> -----Original Message-----
> From: Sitsofe Wheeler [mailto:sitsofe@gmail.com]
> Sent: Thursday, September 25, 2014 2:08 PM
> To: Olaf Hering
> Cc: Thomas Shao; gregkh@linuxfoundation.org; linux-
> kernel@vger.kernel.org; driverdev-devel@linuxdriverproject.org;
> apw@canonical.com; jasowang@redhat.com; KY Srinivasan; Haiyang Zhang
> Subject: Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using
> host time sample
> 
> On Tue, Sep 23, 2014 at 09:56:10AM +0200, Olaf Hering wrote:
> > On Tue, Sep 23, Thomas Shao wrote:
> >
> > > In current hyper-v time sync service,it only gets the initial clock
> > > time from the host. It didn't process the following time samples.
> > > This change introduced a module parameter called host_time_sync. If
> > > it is set to true, the guest will periodically sychronize it's time
> > > with the host clock using host time sample. By default it is
> > > disabled, because we still recommend user to configure NTP for time
> synchronization.
> 
> You [Microsoft?] do? Can you link to public sources where is this stated
> please? I don't see any mention of doing this on
> http://technet.microsoft.com/en-us/library/dn720239.aspx .. The only
> official Microsoft information with respect to Hyper-V guest time
> synchronisation I've seen has been for Windows guests but perhaps I've
> looked in the wrong places.
> 
> The reason I ask is because regular ntpd is not enough to discipline a Linux
> Hyper-V guest's clock. So much drift can occur under load that ntpd can't
> bring the clock under sync. For now, I've been using Chrony which has a
> higher tolerance for correcting drifting clocks. I'm not the only one seeing this
> either (see http://serverfault.com/a/488528/203726
> and
> http://serverfault.com/questions/523389/linux-clock-loses-10-minutes-
> every-week
> ).
> 
> It would be good to something official about this issue as it is painful when
> happens.
> 

Hi Sitsofe,

As far as I know, currently the document about time sync in Hyper-V is only for Windows. Below is a doc from MSDN:
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/11/19/time-synchronization-in-hyper-v.aspx 
We are working on the document about Time Sync on Hyper-V to cover both Linux and Windows.

> --
> Sitsofe | http://sucs.org/~sits/

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Time keeps on slipping... on Hyper-V
  2014-09-25  9:40     ` Thomas Shao
@ 2014-09-25 13:28       ` Sitsofe Wheeler
  2014-09-26  6:57         ` Thomas Shao
  0 siblings, 1 reply; 14+ messages in thread
From: Sitsofe Wheeler @ 2014-09-25 13:28 UTC (permalink / raw)
  To: Thomas Shao
  Cc: Olaf Hering, gregkh, linux-kernel, driverdev-devel, apw,
	jasowang, KY Srinivasan, Haiyang Zhang

On Thu, Sep 25, 2014 at 09:40:42AM +0000, Thomas Shao wrote:
> > > On Tue, Sep 23, Thomas Shao wrote:
> > >
> > > with the host clock using host time sample. By default it is
> > > disabled, because we still recommend user to configure NTP for time
> >
> > -----Original Message-----
> > From: Sitsofe Wheeler [mailto:sitsofe@gmail.com]
> > Sent: Thursday, September 25, 2014 2:08 PM
> >
> > You [Microsoft?] do? Can you link to public sources where is this stated
> 
> As far as I know, currently the document about time sync in Hyper-V is
> only for Windows. Below is a doc from MSDN:
> http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/11/19/time-synchronization-in-hyper-v.aspx

That post is problematic and not quite enough because it doesn't say "we
recommend you always configure an external time source for a guest" but
rather gives a more complicated statement saying "because guest time
drifts we have a time synchronisation service that will sort things out.
If you happen to be running a domain controller we recommend you
configure regular synchronisation to an external time source in
addition". This leads to questions like: if my (Linux) guest's sole
purpose is to run a web server am I in the domain controller situation
(domain controllers have to re-serve time so perhaps they are special)?
It also doesn't cover the "clock drift is so bad ntpd can't correct it"
issue.

Have you seen this document on TechNet -
http://blogs.technet.com/b/enterprise_admin/archive/2010/04/04/linux-on-hyper-v-timesync.aspx
"With the new Integration Services, Linux virtual machines can! 

TimeSync means that you don’t have to hassle with configur[ing]
network-based time sync for your Linux VMs on Hyper-V…they can get the
correct time from the host!"

This document gives simpler advice looks quite compelling at first
glance.

> We are working on the document about Time Sync on Hyper-V to cover
> both Linux and Windows.

The sooner you can get such an official document out the better. All
sources of guest time synchronisation information (including Microsoft
blogs and non-Microsoft Q&A sites) need updating so they say the same
(correct!) thing in a clear way.

For the meantime can you state what you believe the recommend practice
is for Linux Hyper-V guests with regards to time synchronisation?

-- 
Sitsofe | http://sucs.org/~sits/

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: Time keeps on slipping... on Hyper-V
  2014-09-25 13:28       ` Time keeps on slipping... on Hyper-V Sitsofe Wheeler
@ 2014-09-26  6:57         ` Thomas Shao
  2014-09-26  9:24           ` Mike Surcouf
  0 siblings, 1 reply; 14+ messages in thread
From: Thomas Shao @ 2014-09-26  6:57 UTC (permalink / raw)
  To: Sitsofe Wheeler
  Cc: Olaf Hering, gregkh, linux-kernel, driverdev-devel, apw,
	jasowang, KY Srinivasan, Haiyang Zhang

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3614 bytes --]



> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org
> [mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Sitsofe Wheeler
> Sent: Thursday, September 25, 2014 9:28 PM
> To: Thomas Shao
> Cc: Olaf Hering; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> driverdev-devel@linuxdriverproject.org; apw@canonical.com;
> jasowang@redhat.com; KY Srinivasan; Haiyang Zhang
> Subject: Re: Time keeps on slipping... on Hyper-V
> 
> On Thu, Sep 25, 2014 at 09:40:42AM +0000, Thomas Shao wrote:
> > > > On Tue, Sep 23, Thomas Shao wrote:
> > > >
> > > > with the host clock using host time sample. By default it is
> > > > disabled, because we still recommend user to configure NTP for
> > > > time
> > >
> > > -----Original Message-----
> > > From: Sitsofe Wheeler [mailto:sitsofe@gmail.com]
> > > Sent: Thursday, September 25, 2014 2:08 PM
> > >
> > > You [Microsoft?] do? Can you link to public sources where is this
> > > stated
> >
> > As far as I know, currently the document about time sync in Hyper-V is
> > only for Windows. Below is a doc from MSDN:
> > http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/11/19/time-synchro
> > nization-in-hyper-v.aspx
> 
> That post is problematic and not quite enough because it doesn't say "we
> recommend you always configure an external time source for a guest" but
> rather gives a more complicated statement saying "because guest time drifts
> we have a time synchronisation service that will sort things out.
> If you happen to be running a domain controller we recommend you configure
> regular synchronisation to an external time source in addition". This leads to
> questions like: if my (Linux) guest's sole purpose is to run a web server am I in
> the domain controller situation (domain controllers have to re-serve time so
> perhaps they are special)?
> It also doesn't cover the "clock drift is so bad ntpd can't correct it"
> issue.
> 
> Have you seen this document on TechNet -
> http://blogs.technet.com/b/enterprise_admin/archive/2010/04/04/linux-on-hy
> per-v-timesync.aspx
> "With the new Integration Services, Linux virtual machines can!
> 
> TimeSync means that you don’t have to hassle with configur[ing]
> network-based time sync for your Linux VMs on Hyper-V…they can get the
> correct time from the host!"
> 
> This document gives simpler advice looks quite compelling at first glance.
> 
> > We are working on the document about Time Sync on Hyper-V to cover
> > both Linux and Windows.
> 
> The sooner you can get such an official document out the better. All sources of
> guest time synchronisation information (including Microsoft blogs and
> non-Microsoft Q&A sites) need updating so they say the same
> (correct!) thing in a clear way.
> 
> For the meantime can you state what you believe the recommend practice is
> for Linux Hyper-V guests with regards to time synchronisation?
> 

We still recommend user to configure NTP in the guest VM. With the new time sync feature in this patch, 
you could have one more option to enable the guest-host sync, if the NTP didn't work in the environment.
For example the guest VM didn't have network connection.

> --
> Sitsofe | http://sucs.org/~sits/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Time keeps on slipping... on Hyper-V
  2014-09-26  6:57         ` Thomas Shao
@ 2014-09-26  9:24           ` Mike Surcouf
  2014-09-26  9:46             ` Thomas Shao
  0 siblings, 1 reply; 14+ messages in thread
From: Mike Surcouf @ 2014-09-26  9:24 UTC (permalink / raw)
  To: Thomas Shao
  Cc: Sitsofe Wheeler, Olaf Hering, gregkh, linux-kernel,
	driverdev-devel, apw, jasowang, KY Srinivasan, Haiyang Zhang

> We still recommend user to configure NTP in the guest VM. With the new time sync feature in this patch,
> you could have one more option to enable the guest-host sync, if the NTP didn't work in the environment.
> For example the guest VM didn't have network connection.

Microsoft used to use host time-samples in their older drivers but
this was dropped (when I don't know).
Now we must use NTP to correct hyperv_clocksource which suffers from
the usual problems associated with virtual environment  cpu loading.

Host time-samples in conjunction with an effective clock stability
algorithm with slews rather than steps should be the default.
NTP is a workaround and should not be the primary solution.

I have seen a lots of posts for RHEL CENTOS  Linux that have a
FAST.hyper_clocksource confirmed by myself on CENTOS 6.5 and 7.0
(also confirmed by Olaf on SLES).

If this patch steps the clock once in a while without any form of
slewing then it has the potential to break things as files and logs
travel BACK in time.

Apologies Thomas could you send me the patch via email as lkml.org
does not have it in the archives.

Regards Mike

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: Time keeps on slipping... on Hyper-V
  2014-09-26  9:24           ` Mike Surcouf
@ 2014-09-26  9:46             ` Thomas Shao
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Shao @ 2014-09-26  9:46 UTC (permalink / raw)
  To: Mike Surcouf
  Cc: Sitsofe Wheeler, Olaf Hering, gregkh, linux-kernel,
	driverdev-devel, apw, jasowang, KY Srinivasan, Haiyang Zhang

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1985 bytes --]



> -----Original Message-----
> From: Mike Surcouf [mailto:mps.surcouf.lkml@gmail.com]
> Sent: Friday, September 26, 2014 5:24 PM
> To: Thomas Shao
> Cc: Sitsofe Wheeler; Olaf Hering; gregkh@linuxfoundation.org; linux-
> kernel@vger.kernel.org; driverdev-devel@linuxdriverproject.org;
> apw@canonical.com; jasowang@redhat.com; KY Srinivasan; Haiyang Zhang
> Subject: Re: Time keeps on slipping... on Hyper-V
> 
> > We still recommend user to configure NTP in the guest VM. With the new
> > time sync feature in this patch, you could have one more option to enable
> the guest-host sync, if the NTP didn't work in the environment.
> > For example the guest VM didn't have network connection.
> 
> Microsoft used to use host time-samples in their older drivers but this was
> dropped (when I don't know).
> Now we must use NTP to correct hyperv_clocksource which suffers from the
> usual problems associated with virtual environment  cpu loading.
> 
> Host time-samples in conjunction with an effective clock stability algorithm
> with slews rather than steps should be the default.
> NTP is a workaround and should not be the primary solution.
> 
> I have seen a lots of posts for RHEL CENTOS  Linux that have a
> FAST.hyper_clocksource confirmed by myself on CENTOS 6.5 and 7.0 (also
> confirmed by Olaf on SLES).
> 
> If this patch steps the clock once in a while without any form of slewing then
> it has the potential to break things as files and logs travel BACK in time.
> 
> Apologies Thomas could you send me the patch via email as lkml.org does
> not have it in the archives.
> 
Sure. I'll send the package again. It's strange that this patch didn't in the archives. And in my patch, it will slew the time, if the time drift is less than 1 s. It only steps the clock if the drift is larger than 1 s.

> Regards Mike
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-26 13:52       ` Mike Surcouf
@ 2014-09-26 15:13         ` Thomas Shao
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Shao @ 2014-09-26 15:13 UTC (permalink / raw)
  To: Mike Surcouf
  Cc: gregkh, linux-kernel, driverdev-devel, Olaf Hering, Robo Bot,
	jasowang, KY Srinivasan, Haiyang Zhang

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1051 bytes --]



> -----Original Message-----
> From: Mike Surcouf [mailto:mps.surcouf.lkml@gmail.com]
> Sent: Friday, September 26, 2014 9:52 PM
> To: Thomas Shao
> Cc: gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> driverdev-devel@linuxdriverproject.org; Olaf Hering; Robo Bot;
> jasowang@redhat.com; KY Srinivasan; Haiyang Zhang
> Subject: Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using
> host time sample
> 
> In CENTOS and RHEL 6 and upwards the official way would be to use
> 
> /usr/sbin/tickadj
> 
> which is provided by ntp/chrony package
> 
> This dependency on user space packages that may or may not be installed is
> going to cause a lot of confusion in an already confused space.
> Are we sure theres no other way?

Thanks Mike.
I think another way is to expose the do_adjtimex to kernel module. I'm not sure if there is any concerns. I'll investigate that.
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-26 13:07     ` Mike Surcouf
@ 2014-09-26 13:52       ` Mike Surcouf
  2014-09-26 15:13         ` Thomas Shao
  0 siblings, 1 reply; 14+ messages in thread
From: Mike Surcouf @ 2014-09-26 13:52 UTC (permalink / raw)
  To: Thomas Shao
  Cc: gregkh, linux-kernel, driverdev-devel, Olaf Hering, Robo Bot,
	jasowang, KY Srinivasan, Haiyang Zhang

In CENTOS and RHEL 6 and upwards the official way would be to use

/usr/sbin/tickadj

which is provided by ntp/chrony package

This dependency on user space packages that may or may not be
installed is going to cause a lot of confusion in an already confused
space.
Are we sure theres no other way?

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-26 12:30   ` Thomas Shao
@ 2014-09-26 13:07     ` Mike Surcouf
  2014-09-26 13:52       ` Mike Surcouf
  0 siblings, 1 reply; 14+ messages in thread
From: Mike Surcouf @ 2014-09-26 13:07 UTC (permalink / raw)
  To: Thomas Shao
  Cc: gregkh, linux-kernel, driverdev-devel, Olaf Hering, Robo Bot,
	jasowang, KY Srinivasan, Haiyang Zhang

>> What happens when adjtimex is not present?
>> Is there no kernel space function for that?
>> Does this patch affect "not setting correct time on restore" or "not setting
>> correct time on  live migration" bug?
>>
>
> If adjtimex is not present, then the slew time part didn't take effect.
> There is a kernel space do_adjtmex, but it's only for sys call and not exposed to module. So I didn't use it in my patch.
> With this patch, it will step the clock if the time drift is larger than 1 seconds, so it can solve
> the issue your mentioned. And on the other hand, "setting correct time on restore" is already fixed
> in other patch, which is already in upstream.
>

AFAIK  CENTOS and RHEL stopped providing adjtimex for a while now (since V5?).
Even went as far as removing references from there man page for hwclock. in V6

On CENTOS 6.5

yum provides */adjtimex

Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
 * base: centosc6.centos.org
 * extras: centosb5.centos.org
 * updates: centosb6.centos.org
No Matches found

So the end user will have to go searching the internet rpmfind or
something to install an unofficial package on an enterprise product.
So by default you will be stepping time on CENTOS/RHEL 6 and above:
with the result of an unstable clock.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-26 11:53 ` Mike Surcouf
@ 2014-09-26 12:30   ` Thomas Shao
  2014-09-26 13:07     ` Mike Surcouf
  0 siblings, 1 reply; 14+ messages in thread
From: Thomas Shao @ 2014-09-26 12:30 UTC (permalink / raw)
  To: Mike Surcouf
  Cc: gregkh, linux-kernel, driverdev-devel, Olaf Hering, Robo Bot,
	jasowang, KY Srinivasan, Haiyang Zhang

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1863 bytes --]


> -----Original Message-----
> From: Mike Surcouf [mailto:mps.surcouf.lkml@gmail.com]
> Sent: Friday, September 26, 2014 7:53 PM
> To: Thomas Shao
> Cc: gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> driverdev-devel@linuxdriverproject.org; Olaf Hering; Robo Bot;
> jasowang@redhat.com; KY Srinivasan; Haiyang Zhang
> Subject: Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using
> host time sample
> 
> > +/* helper function to call adjtimex command in user mode */ static
> > +void run_adjtimex_cmd(s64 tickvalue) {
> > +       char *argv[4], *envp[3];
> > +       char str_tickvalue[20];
> > +
> > +       sprintf(str_tickvalue, "%lld", tickvalue);
> > +
> > +       argv[0] = "/sbin/adjtimex";
> > +       argv[1] = "-t";
> > +       argv[2] = str_tickvalue;
> > +       argv[3] = NULL;
> > +
> > +       envp[0] = "HOME=/";
> > +       envp[1] = "PATH=/sbin:/bin:/usr/sbin:/usr/bin";
> > +       envp[2] = NULL;
> > +
> > +       call_usermodehelper(argv[0], argv, envp, UMH_WAIT_EXEC); }
> 
> 
> What happens when adjtimex is not present?
> Is there no kernel space function for that?
> Does this patch affect "not setting correct time on restore" or "not setting
> correct time on  live migration" bug?
> 

If adjtimex is not present, then the slew time part didn't take effect.
There is a kernel space do_adjtmex, but it's only for sys call and not exposed to module. So I didn't use it in my patch.
With this patch, it will step the clock if the time drift is larger than 1 seconds, so it can solve
the issue your mentioned. And on the other hand, "setting correct time on restore" is already fixed
in other patch, which is already in upstream.


> Thanks
> 
> Mike
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
  2014-09-26 10:02 [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample Thomas Shao
@ 2014-09-26 11:53 ` Mike Surcouf
  2014-09-26 12:30   ` Thomas Shao
  0 siblings, 1 reply; 14+ messages in thread
From: Mike Surcouf @ 2014-09-26 11:53 UTC (permalink / raw)
  To: Thomas Shao
  Cc: gregkh, linux-kernel, driverdev-devel, Olaf Hering, Robo Bot,
	jasowang, KY Srinivasan, Haiyang Zhang

> +/* helper function to call adjtimex command in user mode */
> +static void run_adjtimex_cmd(s64 tickvalue)
> +{
> +       char *argv[4], *envp[3];
> +       char str_tickvalue[20];
> +
> +       sprintf(str_tickvalue, "%lld", tickvalue);
> +
> +       argv[0] = "/sbin/adjtimex";
> +       argv[1] = "-t";
> +       argv[2] = str_tickvalue;
> +       argv[3] = NULL;
> +
> +       envp[0] = "HOME=/";
> +       envp[1] = "PATH=/sbin:/bin:/usr/sbin:/usr/bin";
> +       envp[2] = NULL;
> +
> +       call_usermodehelper(argv[0], argv, envp, UMH_WAIT_EXEC);
> +}


What happens when adjtimex is not present?
Is there no kernel space function for that?
Does this patch affect "not setting correct time on restore" or "not
setting correct time on  live migration" bug?

Thanks

Mike

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample
@ 2014-09-26 10:02 Thomas Shao
  2014-09-26 11:53 ` Mike Surcouf
  0 siblings, 1 reply; 14+ messages in thread
From: Thomas Shao @ 2014-09-26 10:02 UTC (permalink / raw)
  To: gregkh, linux-kernel, driverdev-devel, olaf, apw, jasowang,
	mps.surcouf.lkml
  Cc: kys, haiyangz, Thomas Shao

In current hyper-v time sync service,it only gets the initial clock time
from the host. It didn't process the following time samples. This change
introduced a module parameter called host_time_sync. If it is set to true,
the guest will periodically sychronize it's time with the host clock using
host time sample. By default it is disabled, because we still recommend
user to configure NTP for time synchronization.

Signed-off-by: Thomas Shao <huishao@microsoft.com>
Reviewed-by: K. Y. Srinivasan <kys@microsoft.com>
---
 drivers/hv/hv_util.c |  133 +++++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 125 insertions(+), 8 deletions(-)

diff --git a/drivers/hv/hv_util.c b/drivers/hv/hv_util.c
index 3b9c9ef..9b167c6 100644
--- a/drivers/hv/hv_util.c
+++ b/drivers/hv/hv_util.c
@@ -51,11 +51,30 @@
 #define HB_WS2008_MAJOR	1
 #define HB_WS2008_VERSION	(HB_WS2008_MAJOR << 16 | HB_MINOR)
 
+#define  TIMESAMPLE_INTERVAL 5000000000L  /* 5s in nanosecond */
+
+/*host sends time sample for every 5s.So the max polling interval
+ *is 128*5 = 640s.
+*/
+#define  TIME_ADJ_MAX_INTERVAL 128 /*Max polling interval */
+
 static int sd_srv_version;
 static int ts_srv_version;
 static int hb_srv_version;
 static int util_fw_version;
 
+/*host sends time sample for every 5s.So the initial polling interval
+ *is 5s.
+*/
+static s32 adj_interval = 1;
+
+/*The host_time_sync module parameter is used to control the time
+  sync between host and guest.
+*/
+static bool host_time_sync;
+module_param(host_time_sync, bool, (S_IRUGO | S_IWUSR));
+MODULE_PARM_DESC(host_time_sync, "If the guest sync time with host");
+
 static void shutdown_onchannelcallback(void *context);
 static struct hv_util_service util_shutdown = {
 	.util_cb = shutdown_onchannelcallback,
@@ -160,18 +179,83 @@ static void shutdown_onchannelcallback(void *context)
 		schedule_work(&shutdown_work);
 }
 
+/* helper function to call adjtimex command in user mode */
+static void run_adjtimex_cmd(s64 tickvalue)
+{
+	char *argv[4], *envp[3];
+	char str_tickvalue[20];
+
+	sprintf(str_tickvalue, "%lld", tickvalue);
+
+	argv[0] = "/sbin/adjtimex";
+	argv[1] = "-t";
+	argv[2] = str_tickvalue;
+	argv[3] = NULL;
+
+	envp[0] = "HOME=/";
+	envp[1] = "PATH=/sbin:/bin:/usr/sbin:/usr/bin";
+	envp[2] = NULL;
+
+	call_usermodehelper(argv[0], argv, envp, UMH_WAIT_EXEC);
+}
+
 /*
  * Set guest time to host UTC time.
  */
-static inline void do_adj_guesttime(u64 hosttime)
+static inline void do_adj_guesttime(u64 hosttime, bool forceSync)
 {
-	s64 host_tns;
-	struct timespec host_ts;
+	s64 host_tns, guest_tns, diff;
+	struct timespec host_ts, guest_ts;
+	s64 tickchg, tickval;
+	int diff_sign;
 
 	host_tns = (hosttime - WLTIMEDELTA) * 100;
 	host_ts = ns_to_timespec(host_tns);
 
-	do_settimeofday(&host_ts);
+	if (forceSync) {
+		do_settimeofday(&host_ts);
+	} else {
+		/* for the first time, reset the tick value */
+		if (adj_interval == 1) {
+			run_adjtimex_cmd(TICK_USEC);
+			adj_interval = adj_interval * 2;
+			do_settimeofday(&host_ts);
+			return;
+		}
+
+		guest_ts = CURRENT_TIME;
+		guest_tns = timespec_to_ns(&guest_ts);
+		diff = host_tns - guest_tns;
+		if (diff >= 0) {
+			diff_sign = 1;
+		} else {
+			diff_sign = -1;
+			diff = -diff;
+		}
+
+		/*1s in nanosecond */
+		if (diff > 1000000000 || diff < -1000000000) {
+			do_settimeofday(&host_ts);
+			return;
+		}
+
+		/*1ms in nanosecond */
+		if (diff > 1000000 || diff < -1000000) {
+			tickchg = diff * TICK_USEC /
+					(TIMESAMPLE_INTERVAL * adj_interval);
+
+			if (tickchg > TICK_USEC/10)
+				tickchg = TICK_USEC/10;
+
+			tickval = TICK_USEC + diff_sign * tickchg;
+			run_adjtimex_cmd(tickval);
+
+		} else {
+			/* double the polling interval*/
+			if (adj_interval < TIME_ADJ_MAX_INTERVAL)
+				adj_interval = adj_interval * 2;
+		}
+	}
 }
 
 /*
@@ -179,8 +263,9 @@ static inline void do_adj_guesttime(u64 hosttime)
  */
 
 struct adj_time_work {
-	struct work_struct work;
+	struct	work_struct work;
 	u64	host_time;
+	bool	forceSync;
 };
 
 static void hv_set_host_time(struct work_struct *work)
@@ -188,7 +273,7 @@ static void hv_set_host_time(struct work_struct *work)
 	struct adj_time_work	*wrk;
 
 	wrk = container_of(work, struct adj_time_work, work);
-	do_adj_guesttime(wrk->host_time);
+	do_adj_guesttime(wrk->host_time, wrk->forceSync);
 	kfree(wrk);
 }
 
@@ -202,11 +287,14 @@ static void hv_set_host_time(struct work_struct *work)
  * thing is, systime is automatically set to emulated hardware clock which may
  * not be UTC time or in the same time zone. So, to override these effects, we
  * use the first 50 time samples for initial system time setting.
+ * If the host_time_sync module parameter is set, we will use the host time
+ * samples to adjust guest time after the first 50 samples.
  */
 static inline void adj_guesttime(u64 hosttime, u8 flags)
 {
 	struct adj_time_work    *wrk;
 	static s32 scnt = 50;
+	static s32 sample_count;
 
 	wrk = kmalloc(sizeof(struct adj_time_work), GFP_ATOMIC);
 	if (wrk == NULL)
@@ -214,6 +302,7 @@ static inline void adj_guesttime(u64 hosttime, u8 flags)
 
 	wrk->host_time = hosttime;
 	if ((flags & ICTIMESYNCFLAG_SYNC) != 0) {
+		wrk->forceSync = true;
 		INIT_WORK(&wrk->work, hv_set_host_time);
 		schedule_work(&wrk->work);
 		return;
@@ -221,10 +310,38 @@ static inline void adj_guesttime(u64 hosttime, u8 flags)
 
 	if ((flags & ICTIMESYNCFLAG_SAMPLE) != 0 && scnt > 0) {
 		scnt--;
+		wrk->forceSync = true;
 		INIT_WORK(&wrk->work, hv_set_host_time);
 		schedule_work(&wrk->work);
-	} else
-		kfree(wrk);
+		return;
+	}
+
+	if (host_time_sync) {
+		/*
+		* Use the Hyper-V time sample to adjust the guest time. The
+		* algorithm is: If the sample offsets exceeds 1 second, we
+		* directly set the clock to the server time. If the offset is
+		* less than 1ms, we ignore the time sample. Otherwise we adjust
+		* the clock.
+		*/
+
+		if ((flags & ICTIMESYNCFLAG_SAMPLE) != 0) {
+			if (sample_count < adj_interval) {
+				sample_count++;
+				goto cleanup;
+			}
+			/* reset the polling interval */
+			sample_count = 0;
+			wrk->forceSync = false;
+			INIT_WORK(&wrk->work, hv_set_host_time);
+			schedule_work(&wrk->work);
+			return;
+		}
+	}
+
+cleanup:
+	kfree(wrk);
+
 }
 
 /*
-- 
1.7.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2014-09-26 15:14 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <1411451081-13423-1-git-send-email-huishao@microsoft.com>
2014-09-23  7:56 ` [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample Olaf Hering
2014-09-23  9:47   ` Thomas Shao
2014-09-25  6:07   ` Sitsofe Wheeler
2014-09-25  9:40     ` Thomas Shao
2014-09-25 13:28       ` Time keeps on slipping... on Hyper-V Sitsofe Wheeler
2014-09-26  6:57         ` Thomas Shao
2014-09-26  9:24           ` Mike Surcouf
2014-09-26  9:46             ` Thomas Shao
2014-09-26 10:02 [PATCH] Drivers: hv: util: Implement Time Synchronization using host time sample Thomas Shao
2014-09-26 11:53 ` Mike Surcouf
2014-09-26 12:30   ` Thomas Shao
2014-09-26 13:07     ` Mike Surcouf
2014-09-26 13:52       ` Mike Surcouf
2014-09-26 15:13         ` Thomas Shao

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).