All of lore.kernel.org
 help / color / mirror / Atom feed
* RE: kvm-clock again
@ 2016-04-05 15:07 Avi Cohen
  2016-04-05 20:25 ` Radim Krčmář
  0 siblings, 1 reply; 10+ messages in thread
From: Avi Cohen @ 2016-04-05 15:07 UTC (permalink / raw)
  To: kvm

What do you think about this simple solution:
 Currently  the KVM updates the system-time in the structure pvclock_vcpu_time_info  in the guest memory  with the MONOTONIC time 
struct pvclock_vcpu_time_info {
        u32   version;
        u32   pad0;
        u64   tsc_timestamp;
        u64   system_time;
        u32   tsc_to_system_mul;
        s8    tsc_shift;
        u8    flags;
        u8    pad[2];
} __attribute__((__packed__));

This is the default kvm-clcok.
I think to add another clock source named - kvm-clock-realtime, and to update system_time with host's REAL_TIME clock.
What is your opinion ?


I want to enter the KVM  code and try to make a REAL_TIME and accurate synchronization between the host and  guest  clocks .
Currently I don’t see that KVM  updates the guest’s clock when I manually set the host’s clock. I don’t see any impact on the guest’s clock Also after booting the guest I see a  diff-time of ~1ms between the 2 clocks. 
Questions:
- What can you tell about the feasibility of this task ?
- Can you give me a reference/guides  for this task’s  design ? where to start ? potential problems ? pvclock ? when to update , when reentering the VM ? etc..

(I’m working with kvm-clock as a source clock, my machine is a very modern x86 with full virtualization support, constant TSC etc..) Thanks  in adavance Avi



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: kvm-clock again
  2016-04-05 15:07 kvm-clock again Avi Cohen
@ 2016-04-05 20:25 ` Radim Krčmář
  2016-04-06 13:51   ` Avi Cohen
  2016-04-07  7:09   ` Avi Cohen
  0 siblings, 2 replies; 10+ messages in thread
From: Radim Krčmář @ 2016-04-05 20:25 UTC (permalink / raw)
  To: Avi Cohen; +Cc: kvm

2016-04-05 15:07+0000, Avi Cohen:
> What do you think about this simple solution:
>  Currently  the KVM updates the system-time in the structure pvclock_vcpu_time_info  in the guest memory  with the MONOTONIC time 
> struct pvclock_vcpu_time_info {
>         u32   version;
>         u32   pad0;
>         u64   tsc_timestamp;
>         u64   system_time;
>         u32   tsc_to_system_mul;
>         s8    tsc_shift;
>         u8    flags;
>         u8    pad[2];
> } __attribute__((__packed__));
> 
> This is the default kvm-clcok.
> I think to add another clock source named - kvm-clock-realtime, and to update system_time with host's REAL_TIME clock.
> What is your opinion ?

Would you be ok with a wall_time entry in pvclock_vcpu_time_info that
would tell the host real time at system_time?
struct pvclock_wall_clock should say the host real time when system_time
was 0, so it's the same.  wallclock = CLOCK_REALTIME, most likely.

Creating a new clock would allow us to get rid of many other problems,
but your goal might be achievable with existing interfaces.

> I want to enter the KVM  code and try to make a REAL_TIME and accurate synchronization between the host and  guest  clocks .
> Currently I don’t see that KVM  updates the guest’s clock when I manually set the host’s clock. I don’t see any impact on the guest’s clock Also after booting the guest I see a  diff-time of ~1ms between the 2 clocks. 
> Questions:
> - What can you tell about the feasibility of this task ?

Doable.

> - Can you give me a reference/guides  for this task’s  design ? where to start ? potential problems ? pvclock ? when to update , when reentering the VM ? etc..

Start by using kvm_get_wallclock to pass the host CLOCK_REALTIME into
the guest and turn wallclock into guest CLOCK_REALTIME.

A reasonable solution would be to create a thread that periodically
synchronizes CLOCK_REALTIME with wallclock.  The wallclock thread would
behave like PTP/NTP, so it should be easy to do.
(A notification from the host that the wallclock has changed would
 be better, but needs new interface.)

I think that using wallclock directly as CLOCK_REALTIME would cause more
complications:
- many guest interfaces expect that they can touch CLOCK_REALTIME, but
  that shouldn't be allowed
- changes in wallclock should trigger guest notifiers as if the change
  was done by the guest
- wallclock is updated only when the guest writes to the MSR, which
  would be wasteful for frequent reads and not possible in userspace

We'll have enough experience to decide on followup if no solution using
the existing framework is acceptable.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: kvm-clock again
  2016-04-05 20:25 ` Radim Krčmář
@ 2016-04-06 13:51   ` Avi Cohen
  2016-04-06 14:17     ` Radim Krčmář
  2016-04-07  7:09   ` Avi Cohen
  1 sibling, 1 reply; 10+ messages in thread
From: Avi Cohen @ 2016-04-06 13:51 UTC (permalink / raw)
  To: Radim Krčmář; +Cc: kvm


> 2016-04-05 15:07+0000, Avi Cohen:
> > What do you think about this simple solution:
> >  Currently  the KVM updates the system-time in the structure
> > pvclock_vcpu_time_info  in the guest memory  with the MONOTONIC time
> struct pvclock_vcpu_time_info {
> >         u32   version;
> >         u32   pad0;
> >         u64   tsc_timestamp;
> >         u64   system_time;
> >         u32   tsc_to_system_mul;
> >         s8    tsc_shift;
> >         u8    flags;
> >         u8    pad[2];
> > } __attribute__((__packed__));
> >
> > This is the default kvm-clcok.
> > I think to add another clock source named - kvm-clock-realtime, and to update
> system_time with host's REAL_TIME clock.
> > What is your opinion ?
> 
> Would you be ok with a wall_time entry in pvclock_vcpu_time_info that would
> tell the host real time at system_time?
> struct pvclock_wall_clock should say the host real time when system_time was
> 0, so it's the same.  wallclock = CLOCK_REALTIME, most likely.
> 
> Creating a new clock would allow us to get rid of many other problems, but your
> goal might be achievable with existing interfaces.
> 

Yes - it will be ok to update the pvclock_wall_clock with the host CLOCK_REALTIME.
But I think  I also need the guest's TSC at that time of update, 
struct pvclock_wall_clock {
 37         u32   version;
 38         u32   sec;
 39         u32   nsec;
 New field  tsc_timestamp;
New field  tsc_to_system_mul;
 40 } __attribute__((__packed__));

Now the  guest - when need to read the current time  - can do something like this :
Current_time   = sec * sec_to_nano + nsec +  (rdtdc () - tsc_timestamp) *  tsc_to_system_mul

> > I want to enter the KVM  code and try to make a REAL_TIME and accurate
> synchronization between the host and  guest  clocks .
> > Currently I don’t see that KVM  updates the guest’s clock when I manually set
> the host’s clock. I don’t see any impact on the guest’s clock Also after booting
> the guest I see a  diff-time of ~1ms between the 2 clocks.
> > Questions:
> > - What can you tell about the feasibility of this task ?
> 
> Doable.
> 
> > - Can you give me a reference/guides  for this task’s  design ? where to start ?
> potential problems ? pvclock ? when to update , when reentering the VM ? etc..
> 
> Start by using kvm_get_wallclock to pass the host CLOCK_REALTIME into the
> guest and turn wallclock into guest CLOCK_REALTIME.
> 
> A reasonable solution would be to create a thread that periodically synchronizes
> CLOCK_REALTIME with wallclock.  The wallclock thread would behave like
> PTP/NTP, so it should be easy to do.
> (A notification from the host that the wallclock has changed would  be better,
> but needs new interface.)
> 
> I think that using wallclock directly as CLOCK_REALTIME would cause more
> complications:
> - many guest interfaces expect that they can touch CLOCK_REALTIME, but
>   that shouldn't be allowed
> - changes in wallclock should trigger guest notifiers as if the change
>   was done by the guest
> - wallclock is updated only when the guest writes to the MSR, which
>   would be wasteful for frequent reads and not possible in userspace
> 
> We'll have enough experience to decide on followup if no solution using the
> existing framework is acceptable.

I will do some  design work with existing framework and send it soon


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: kvm-clock again
  2016-04-06 13:51   ` Avi Cohen
@ 2016-04-06 14:17     ` Radim Krčmář
  0 siblings, 0 replies; 10+ messages in thread
From: Radim Krčmář @ 2016-04-06 14:17 UTC (permalink / raw)
  To: Avi Cohen; +Cc: kvm

2016-04-06 13:51+0000, Avi Cohen:
>> 2016-04-05 15:07+0000, Avi Cohen:
>> > What do you think about this simple solution:
>> >  Currently  the KVM updates the system-time in the structure
>> > pvclock_vcpu_time_info  in the guest memory  with the MONOTONIC time
>> struct pvclock_vcpu_time_info {
>> >         u32   version;
>> >         u32   pad0;
>> >         u64   tsc_timestamp;
>> >         u64   system_time;
>> >         u32   tsc_to_system_mul;
>> >         s8    tsc_shift;
>> >         u8    flags;
>> >         u8    pad[2];
>> > } __attribute__((__packed__));
>> >
>> > This is the default kvm-clcok.
>> > I think to add another clock source named - kvm-clock-realtime, and to update
>> system_time with host's REAL_TIME clock.
>> > What is your opinion ?
>> 
>> Would you be ok with a wall_time entry in pvclock_vcpu_time_info that would
>> tell the host real time at system_time?
>> struct pvclock_wall_clock should say the host real time when system_time was
>> 0, so it's the same.  wallclock = CLOCK_REALTIME, most likely.
>> 
>> Creating a new clock would allow us to get rid of many other problems, but your
>> goal might be achievable with existing interfaces.
>> 
> 
> Yes - it will be ok to update the pvclock_wall_clock with the host CLOCK_REALTIME.
> But I think  I also need the guest's TSC at that time of update, 
> struct pvclock_wall_clock {
>  37         u32   version;
>  38         u32   sec;
>  39         u32   nsec;
>  New field  tsc_timestamp;
> New field  tsc_to_system_mul;
>  40 } __attribute__((__packed__));
> 
> Now the  guest - when need to read the current time  - can do something like this :
> Current_time   = sec * sec_to_nano + nsec +  (rdtdc () - tsc_timestamp) *  tsc_to_system_mul

No need.  pvclock_wall_clock.{sec,nsec} is synchronized with system_time
and system_time is synchronized with TSC.  To get the current real time,
you just add times from pvclock_vcpu_time_info, pvclock_wall_clock and
TSC, like kvm_get_wallclock() does.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: kvm-clock again
  2016-04-05 20:25 ` Radim Krčmář
  2016-04-06 13:51   ` Avi Cohen
@ 2016-04-07  7:09   ` Avi Cohen
  2016-04-07 14:20     ` Radim Krčmář
  1 sibling, 1 reply; 10+ messages in thread
From: Avi Cohen @ 2016-04-07  7:09 UTC (permalink / raw)
  To: Radim Krčmář; +Cc: kvm


> From: Radim Krčmář [mailto:rkrcmar@redhat.com]
> Sent: Tuesday, 05 April, 2016 11:25 PM
> To: Avi Cohen
> Cc: kvm@vger.kernel.org
> Subject: Re: kvm-clock again


> Would you be ok with a wall_time entry in pvclock_vcpu_time_info that would
> tell the host real time at system_time?
> struct pvclock_wall_clock should say the host real time when system_time was
> 0, so it's the same.  wallclock = CLOCK_REALTIME, most likely.
> 
> Creating a new clock would allow us to get rid of many other problems, but your
> goal might be achievable with existing interfaces.
> 
> > I want to enter the KVM  code and try to make a REAL_TIME and accurate
> synchronization between the host and  guest  clocks .
> > Currently I don’t see that KVM  updates the guest’s clock when I manually set
> the host’s clock. I don’t see any impact on the guest’s clock Also after booting
> the guest I see a  diff-time of ~1ms between the 2 clocks.
> > Questions:
> > - What can you tell about the feasibility of this task ?
> 
> Doable.
> 
> > - Can you give me a reference/guides  for this task’s  design ? where to start ?
> potential problems ? pvclock ? when to update , when reentering the VM ? etc..
> 
> Start by using kvm_get_wallclock to pass the host CLOCK_REALTIME into the
> guest and turn wallclock into guest CLOCK_REALTIME.
> 
> A reasonable solution would be to create a thread that periodically synchronizes
> CLOCK_REALTIME with wallclock.  The wallclock thread would behave like
> PTP/NTP, so it should be easy to do.
> (A notification from the host that the wallclock has changed would  be better,
> but needs new interface.)
> 
> I think that using wallclock directly as CLOCK_REALTIME would cause more
> complications:
> - many guest interfaces expect that they can touch CLOCK_REALTIME, but
>   that shouldn't be allowed
Do you mean simultaneously guests write to MSR ? 
Guests running on different physical CPU's can simultaneously write to the MSR and this should trigger kvm update of wall-clock - correct ?
I understand that guest writing to MSR does not necessarily  trigger VM-exit   - correct ?

> - changes in wallclock should trigger guest notifiers as if the change
>   was done by the guest
I don’t understand what do u mean here. 
if for example the host's clock was changed because of PTP time-set, then after some time a guest is writing to MSR to get host real-time clock ,
I expect that the new wall clock update will include this time-set. 
> - wallclock is updated only when the guest writes to the MSR, which
>   would be wasteful for frequent reads and not possible in userspace
> 
As you suggest - this will be performed by a new kernel thread which should only write to the MSR and exit. i.e. it will not look into the wall-clock structure.
Only when user application is requesting the current time (clock_gettime) -  this call (clock_gettime or a another new call) will check the content of the 
pvclock_wall_clock  and calculate the current time.

> We'll have enough experience to decide on followup if no solution using the
> existing framework is acceptable.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: kvm-clock again
  2016-04-07  7:09   ` Avi Cohen
@ 2016-04-07 14:20     ` Radim Krčmář
  2016-04-08 15:00       ` Avi Cohen
  2016-04-13 12:46       ` Avi Cohen
  0 siblings, 2 replies; 10+ messages in thread
From: Radim Krčmář @ 2016-04-07 14:20 UTC (permalink / raw)
  To: Avi Cohen; +Cc: kvm

2016-04-07 07:09+0000, Avi Cohen:
>> From: Radim Krčmář [mailto:rkrcmar@redhat.com]
>> > - Can you give me a reference/guides  for this task’s  design ? where to start ?
>> potential problems ? pvclock ? when to update , when reentering the VM ? etc..
>> 
>> Start by using kvm_get_wallclock to pass the host CLOCK_REALTIME into the
>> guest and turn wallclock into guest CLOCK_REALTIME.
>> 
>> A reasonable solution would be to create a thread that periodically synchronizes
>> CLOCK_REALTIME with wallclock.  The wallclock thread would behave like
>> PTP/NTP, so it should be easy to do.
>> (A notification from the host that the wallclock has changed would  be better,
>> but needs new interface.)
>> 
>> I think that using wallclock directly as CLOCK_REALTIME would cause more
>> complications:
>> - many guest interfaces expect that they can touch CLOCK_REALTIME, but
>>   that shouldn't be allowed
> Do you mean simultaneously guests write to MSR ? 

I meant interface like clock_settime() -- we probably never denied write
access before, so some buggy userspace applications could misbehave.
Even if we silently ignore the write, then people would definitely come
complaining:  "`date -s 0` doesn't work!".  Wasted time all around.

> Guests running on different physical CPU's can simultaneously write to the MSR and this should trigger kvm update of wall-clock - correct ?

Yes, wall-clock is per-guest.

(Note that simultaneous writes to this MSR from one guest are buggy, but
 the guest doesn't have a reason to access it simultaneously, because the
 MSR is actually global.)

> I understand that guest writing to MSR does not necessarily  trigger VM-exit   - correct ?

No, it always exits.  The CPU doesn't know how to emulate that MSR.

>> - changes in wallclock should trigger guest notifiers as if the change
>>   was done by the guest
> I don’t understand what do u mean here. 

Let's say the guest wants to do something at 12:00:00 and it's 11:59:00
now.  The guest creates a timer with 60 second timeout.  Then the host
advances its CLOCK_REALTIME by 10 seconds.  The timeout would expire at
12:00:10 after that, so the guest has to notice that CLOCK_REALTIME has
changed and modify its timers.

> if for example the host's clock was changed because of PTP time-set, then after some time a guest is writing to MSR to get host real-time clock ,
> I expect that the new wall clock update will include this time-set. 

It will, but the guest must know that the time has changed, because of
decisions that were made with the old time.

(Any behavior can be excused when time is considered, but humans tend
 not to be that benevolent.)

>> - wallclock is updated only when the guest writes to the MSR, which
>>   would be wasteful for frequent reads and not possible in userspace
>> 
> As you suggest - this will be performed by a new kernel thread which should only write to the MSR and exit. i.e. it will not look into the wall-clock structure.
> Only when user application is requesting the current time (clock_gettime) -  this call (clock_gettime or a another new call) will check the content of the 
> pvclock_wall_clock  and calculate the current time.

No, this is the problematic solution, just avoiding the MSR overhead.
wall-clock should be used in the thread to set the time.  Rough idea:

  while sleep(magic): do_settimeofday64(kvm_get_wallclock())

The polished solution should change the time like PTP/NTP does.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: kvm-clock again
  2016-04-07 14:20     ` Radim Krčmář
@ 2016-04-08 15:00       ` Avi Cohen
  2016-04-13 12:46       ` Avi Cohen
  1 sibling, 0 replies; 10+ messages in thread
From: Avi Cohen @ 2016-04-08 15:00 UTC (permalink / raw)
  To: Radim Krčmář; +Cc: kvm

Thanks Radim
I agree with you that the 'PTP like' clock/frequency adjustment will be very nice here 
Regards
Avi
> -----Original Message-----
> From: Radim Krčmář [mailto:rkrcmar@redhat.com]
> Sent: Thursday, 07 April, 2016 5:21 PM
> To: Avi Cohen
> Cc: kvm@vger.kernel.org
> Subject: Re: kvm-clock again
> 
> 2016-04-07 07:09+0000, Avi Cohen:
> >> From: Radim Krčmář [mailto:rkrcmar@redhat.com]
> >> > - Can you give me a reference/guides  for this task’s  design ? where to start
> ?
> >> potential problems ? pvclock ? when to update , when reentering the VM ?
> etc..
> >>
> >> Start by using kvm_get_wallclock to pass the host CLOCK_REALTIME into
> >> the guest and turn wallclock into guest CLOCK_REALTIME.
> >>
> >> A reasonable solution would be to create a thread that periodically
> >> synchronizes CLOCK_REALTIME with wallclock.  The wallclock thread
> >> would behave like PTP/NTP, so it should be easy to do.
> >> (A notification from the host that the wallclock has changed would
> >> be better, but needs new interface.)
> >>
> >> I think that using wallclock directly as CLOCK_REALTIME would cause
> >> more
> >> complications:
> >> - many guest interfaces expect that they can touch CLOCK_REALTIME, but
> >>   that shouldn't be allowed
> > Do you mean simultaneously guests write to MSR ?
> 
> I meant interface like clock_settime() -- we probably never denied write access
> before, so some buggy userspace applications could misbehave.
> Even if we silently ignore the write, then people would definitely come
> complaining:  "`date -s 0` doesn't work!".  Wasted time all around.
> 
> > Guests running on different physical CPU's can simultaneously write to the
> MSR and this should trigger kvm update of wall-clock - correct ?
> 
> Yes, wall-clock is per-guest.
> 
> (Note that simultaneous writes to this MSR from one guest are buggy, but  the
> guest doesn't have a reason to access it simultaneously, because the  MSR is
> actually global.)
> 
> > I understand that guest writing to MSR does not necessarily  trigger VM-exit   -
> correct ?
> 
> No, it always exits.  The CPU doesn't know how to emulate that MSR.
> 
> >> - changes in wallclock should trigger guest notifiers as if the change
> >>   was done by the guest
> > I don’t understand what do u mean here.
> 
> Let's say the guest wants to do something at 12:00:00 and it's 11:59:00 now.
> The guest creates a timer with 60 second timeout.  Then the host advances its
> CLOCK_REALTIME by 10 seconds.  The timeout would expire at
> 12:00:10 after that, so the guest has to notice that CLOCK_REALTIME has
> changed and modify its timers.
> 
> > if for example the host's clock was changed because of PTP time-set,
> > then after some time a guest is writing to MSR to get host real-time clock , I
> expect that the new wall clock update will include this time-set.
> 
> It will, but the guest must know that the time has changed, because of decisions
> that were made with the old time.
> 
> (Any behavior can be excused when time is considered, but humans tend  not to
> be that benevolent.)
> 
> >> - wallclock is updated only when the guest writes to the MSR, which
> >>   would be wasteful for frequent reads and not possible in userspace
> >>
> > As you suggest - this will be performed by a new kernel thread which should
> only write to the MSR and exit. i.e. it will not look into the wall-clock structure.
> > Only when user application is requesting the current time
> > (clock_gettime) -  this call (clock_gettime or a another new call) will check the
> content of the pvclock_wall_clock  and calculate the current time.
> 
> No, this is the problematic solution, just avoiding the MSR overhead.
> wall-clock should be used in the thread to set the time.  Rough idea:
> 
>   while sleep(magic): do_settimeofday64(kvm_get_wallclock())
> 
> The polished solution should change the time like PTP/NTP does.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: kvm-clock again
  2016-04-07 14:20     ` Radim Krčmář
  2016-04-08 15:00       ` Avi Cohen
@ 2016-04-13 12:46       ` Avi Cohen
  2016-04-14 18:23         ` Radim Krčmář
  1 sibling, 1 reply; 10+ messages in thread
From: Avi Cohen @ 2016-04-13 12:46 UTC (permalink / raw)
  To: Radim Krčmář; +Cc: kvm

Hi Radim,
1. What do you think is the reason for the real-time clock difference (about 800ms) between guest and host , measured just a short time after boot ?
The host set the guest's wall-clock with its real-time clock @ boot , I  see in the code that the guest tries to fetch this time in a loop  - till it verifies that the version field (in the wall-clock structure) was changed - which means that the host wrote it's clock to the wall-clock page. 
What can be the reason for this big delta/error ?
2. when the host updates the system/monotonic -clock page (pvclock_system_time), whenever it re-enters the guest. It updates also the guest's TSC(**)  at this current time.
I understand that this TSC is per guest/VCPU - correct ? how accurate is this TSC ? is it virtualized ? 
When guest reads its TSC by RDTSC -  this does not trigger VM-exit .  how this TSC (read by guest) is related to the prev. TSC(**) which is updated by host  ? 
3. suppose the host is running a PTP and I'm going to implement a periodic kernel thread to query the host's real-time clock every 1 second. Why do you this that implementing another 'PTP like' clock/frequency  adjustment in the guest is required ?
Thanks and best regards,
Avi 


> -----Original Message-----
> From: Radim Krčmář [mailto:rkrcmar@redhat.com]
> Sent: Thursday, 07 April, 2016 5:21 PM
> To: Avi Cohen
> Cc: kvm@vger.kernel.org
> Subject: Re: kvm-clock again
> 
> 2016-04-07 07:09+0000, Avi Cohen:
> >> From: Radim Krčmář [mailto:rkrcmar@redhat.com]
> >> > - Can you give me a reference/guides  for this task’s  design ? where to start
> ?
> >> potential problems ? pvclock ? when to update , when reentering the VM ?
> etc..
> >>
> >> Start by using kvm_get_wallclock to pass the host CLOCK_REALTIME into
> >> the guest and turn wallclock into guest CLOCK_REALTIME.
> >>
> >> A reasonable solution would be to create a thread that periodically
> >> synchronizes CLOCK_REALTIME with wallclock.  The wallclock thread
> >> would behave like PTP/NTP, so it should be easy to do.
> >> (A notification from the host that the wallclock has changed would
> >> be better, but needs new interface.)
> >>
> >> I think that using wallclock directly as CLOCK_REALTIME would cause
> >> more
> >> complications:
> >> - many guest interfaces expect that they can touch CLOCK_REALTIME, but
> >>   that shouldn't be allowed
> > Do you mean simultaneously guests write to MSR ?
> 
> I meant interface like clock_settime() -- we probably never denied write access
> before, so some buggy userspace applications could misbehave.
> Even if we silently ignore the write, then people would definitely come
> complaining:  "`date -s 0` doesn't work!".  Wasted time all around.
> 
> > Guests running on different physical CPU's can simultaneously write to the
> MSR and this should trigger kvm update of wall-clock - correct ?
> 
> Yes, wall-clock is per-guest.
> 
> (Note that simultaneous writes to this MSR from one guest are buggy, but  the
> guest doesn't have a reason to access it simultaneously, because the  MSR is
> actually global.)
> 
> > I understand that guest writing to MSR does not necessarily  trigger VM-exit   -
> correct ?
> 
> No, it always exits.  The CPU doesn't know how to emulate that MSR.
> 
> >> - changes in wallclock should trigger guest notifiers as if the change
> >>   was done by the guest
> > I don’t understand what do u mean here.
> 
> Let's say the guest wants to do something at 12:00:00 and it's 11:59:00 now.
> The guest creates a timer with 60 second timeout.  Then the host advances its
> CLOCK_REALTIME by 10 seconds.  The timeout would expire at
> 12:00:10 after that, so the guest has to notice that CLOCK_REALTIME has
> changed and modify its timers.
> 
> > if for example the host's clock was changed because of PTP time-set,
> > then after some time a guest is writing to MSR to get host real-time clock , I
> expect that the new wall clock update will include this time-set.
> 
> It will, but the guest must know that the time has changed, because of decisions
> that were made with the old time.
> 
> (Any behavior can be excused when time is considered, but humans tend  not to
> be that benevolent.)
> 
> >> - wallclock is updated only when the guest writes to the MSR, which
> >>   would be wasteful for frequent reads and not possible in userspace
> >>
> > As you suggest - this will be performed by a new kernel thread which should
> only write to the MSR and exit. i.e. it will not look into the wall-clock structure.
> > Only when user application is requesting the current time
> > (clock_gettime) -  this call (clock_gettime or a another new call) will check the
> content of the pvclock_wall_clock  and calculate the current time.
> 
> No, this is the problematic solution, just avoiding the MSR overhead.
> wall-clock should be used in the thread to set the time.  Rough idea:
> 
>   while sleep(magic): do_settimeofday64(kvm_get_wallclock())
> 
> The polished solution should change the time like PTP/NTP does.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: kvm-clock again
  2016-04-13 12:46       ` Avi Cohen
@ 2016-04-14 18:23         ` Radim Krčmář
  0 siblings, 0 replies; 10+ messages in thread
From: Radim Krčmář @ 2016-04-14 18:23 UTC (permalink / raw)
  To: Avi Cohen; +Cc: kvm

2016-04-13 12:46+0000, Avi Cohen:
> Hi Radim,
> 1. What do you think is the reason for the real-time clock difference (about 800ms) between guest and host , measured just a short time after boot ?
> The host set the guest's wall-clock with its real-time clock @ boot , I  see in the code that the guest tries to fetch this time in a loop  - till it verifies that the version field (in the wall-clock structure) was changed - which means that the host wrote it's clock to the wall-clock page. 
> What can be the reason for this big delta/error ?

The guest then overwrites wall-clock time with time from emulated RTC,
which could explain some delay and then the measurement will add more,
but 800 ms is ridiculously large ... most likely a bug somewhere, but
it's hard to guess without knowing your whole setup.

> 2. when the host updates the system/monotonic -clock page (pvclock_system_time), whenever it re-enters the guest. It updates also the guest's TSC(**)  at this current time.

KVM doesn't update pvclock_system_time on every entry when you have
master_clock (which you should).

> I understand that this TSC is per guest/VCPU - correct ?

tsc_timestamp is per VCPU, it is TSC value at the same time when
system_time was taken.  If you have master_clock, then tsc_timestamp has
to be identical on all VCPUs.

> how accurate is this TSC ?

Perfectly, host and guest use the same TSC, the guest just have some
offset and scaling that it doesn't know about.

> is it virtualized ? 

I'm not sure what you're asking for.

> When guest reads its TSC by RDTSC -  this does not trigger VM-exit .  how this TSC (read by guest) is related to the prev. TSC(**) which is updated by host  ? 

RDTSC in the guest does not exit, CPU just reads the TSC value, applies
offset and scaling, and gives it to the guest.

> how this TSC (read by guest) is related to the prev. TSC(**) which is updated by host  ? 

If the guest could RDTSC at the time when the host updated TSC(**), then
the guest would have read that value.

> 3. suppose the host is running a PTP and I'm going to implement a periodic kernel thread to query the host's real-time clock every 1 second. Why do you this that implementing another 'PTP like' clock/frequency  adjustment in the guest is required ?

You need to synchronize the time and PTP and NTP deal with time
synchronization, so they are likely using all necessary interfaces to
set the time -- you'll just plug wall-clock, because wall-clock is a
simple Time Protocol.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* kvm-clock again
@ 2016-04-05 11:12 Avi Cohen
  0 siblings, 0 replies; 10+ messages in thread
From: Avi Cohen @ 2016-04-05 11:12 UTC (permalink / raw)
  To: kvm

Hi,
I’m here again with the kvm-clock issue.
I want to enter the KVM  code and try to make a REAL_TIME and accurate synchronization between the host and  guest  clocks .
Currently I don’t see that KVM  updates the guest’s clock when I manually set the host’s clock. I don’t see any impact on the guest’s clock
Also after booting the guest I see a  diff-time of ~1ms between the 2 clocks. 
Questions:
- What can you tell about the feasibility of this task ?
- Can you give me a reference/guides  for this task’s  design ? where to start ? potential problems ? pvclock ? when to update , when reentering the VM ? etc..

(I’m working with kvm-clock as a source clock, my machine is a very modern x86 with full virtualization support, constant TSC etc..)
Thanks  in adavance                     
Avi



^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2016-04-14 18:23 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-05 15:07 kvm-clock again Avi Cohen
2016-04-05 20:25 ` Radim Krčmář
2016-04-06 13:51   ` Avi Cohen
2016-04-06 14:17     ` Radim Krčmář
2016-04-07  7:09   ` Avi Cohen
2016-04-07 14:20     ` Radim Krčmář
2016-04-08 15:00       ` Avi Cohen
2016-04-13 12:46       ` Avi Cohen
2016-04-14 18:23         ` Radim Krčmář
  -- strict thread matches above, loose matches on Subject: below --
2016-04-05 11:12 Avi Cohen

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.