* Re: Could you please answer some questions regarding Solaris PVHVM pvclock support. [not found] <52D6566C.5050302@oracle.com> @ 2014-01-16 12:17 ` Stefano Stabellini 2014-01-24 0:21 ` Mukesh Rathor 2014-02-26 11:52 ` Qin Li 0 siblings, 2 replies; 6+ messages in thread From: Stefano Stabellini @ 2014-01-16 12:17 UTC (permalink / raw) To: Qin Li; +Cc: Roger Pau Monne, xen-devel, Stefano Stabellini CC'ing xen-devel and Roger. On Wed, 15 Jan 2014, Qin Li wrote: > Hi Stefano, > > How do you do? > > Currently, Solaris only works as PV-on-HVM guest on Xen, but recently, Actually now you have a better way of running Solaris on Xen: PVH. http://wiki.xen.org/wiki/Xen_Overview#PV_in_an_HVM_Container_.28PVH.29_-_New_in_Xen_4.4 Roger already ported FreeBSD to Xen as a PVH guest: http://marc.info/?l=freebsd-current&m=138971161228874&w=2 > we decide to changes this situation by work out below 2 features for > Solaris guest OS: > > /* x86: Does this Xen host support the HVM callback vector type? */ > #define XENFEAT_hvm_callback_vector 8 > > /* x86: pvclock algorithm is safe to use on HVM */ > #define XENFEAT_hvm_safe_pvclock FYI both these features are available and used by PVH guests. > For XENFEAT_hvm_callback_vector, it's straightforward that the original > apic interrupt handler needs to be registered onto each vCPU's IDT. > But for XENFEAT_hvm_safe_pvclock, I have some confusions as following: > . Why the pvclock implementation within guest OS, has to depend on > XENFEAT_hvm_callback_vector? Because you need to be able to receive timer interrupts on multiple vcpus and without XENFEAT_hvm_callback_vector you would only receive interrupts from the Xen Platform PCI device. > . For a PV-on-HVM guest OS, the "shared_info->vcpu_info->vcpu_time_info" > is already visiable. Does guest OS still need any action to ask > hypervisor to update this piece of memory periodically? I don't think you need to ask the hypervisor to update vcpu_time_info periodically, what gave you that idea? ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Could you please answer some questions regarding Solaris PVHVM pvclock support. 2014-01-16 12:17 ` Could you please answer some questions regarding Solaris PVHVM pvclock support Stefano Stabellini @ 2014-01-24 0:21 ` Mukesh Rathor 2014-02-26 11:52 ` Qin Li 1 sibling, 0 replies; 6+ messages in thread From: Mukesh Rathor @ 2014-01-24 0:21 UTC (permalink / raw) To: Stefano Stabellini; +Cc: Roger Pau Monne, xen-devel, Qin Li On Thu, 16 Jan 2014 12:17:59 +0000 Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote: > CC'ing xen-devel and Roger. > > On Wed, 15 Jan 2014, Qin Li wrote: > > Hi Stefano, > > > > How do you do? > > > > Currently, Solaris only works as PV-on-HVM guest on Xen, but > > recently, > > Actually now you have a better way of running Solaris on Xen: PVH. > > http://wiki.xen.org/wiki/Xen_Overview#PV_in_an_HVM_Container_.28PVH.29_-_New_in_Xen_4.4 > > Roger already ported FreeBSD to Xen as a PVH guest: > > http://marc.info/?l=freebsd-current&m=138971161228874&w=2 Qin, I will be giving a talk internally on PVH in the next couple weeks to the PMs and others, so if you are interested, ping me offline. thanks Mukesh ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Could you please answer some questions regarding Solaris PVHVM pvclock support. 2014-01-16 12:17 ` Could you please answer some questions regarding Solaris PVHVM pvclock support Stefano Stabellini 2014-01-24 0:21 ` Mukesh Rathor @ 2014-02-26 11:52 ` Qin Li 2014-02-27 13:03 ` Stefano Stabellini 1 sibling, 1 reply; 6+ messages in thread From: Qin Li @ 2014-02-26 11:52 UTC (permalink / raw) To: Stefano Stabellini; +Cc: xen-devel, Roger Pau Monne [-- Attachment #1.1: Type: text/plain, Size: 2364 bytes --] ? 2014/1/16 20:17, Stefano Stabellini ??: >> . For a PV-on-HVM guest OS, the "shared_info->vcpu_info->vcpu_time_info" >> >is already visiable. Does guest OS still need any action to ask >> >hypervisor to update this piece of memory periodically? > I don't think you need to ask the hypervisor to update vcpu_time_info > periodically, what gave you that idea? Hi Stefano, Now, I see it's the hypervisor that will update vcpu_time_info, but another thing confuse me: HVM guest has time drift issue because TSC on different vCPU could be out-of-sync, especially after domain suspend/resume. But how pvclock actually fix this issue? Let's see how FreeBSD port calculate the system time: ================== static uint64_t get_nsec_offset(struct vcpu_time_info *tinfo) { return (scale_delta*(rdtsc() -* tinfo->tsc_timestamp, tinfo->tsc_to_system_mul, tinfo->tsc_shift)); } /** * \brief Get the current time, in nanoseconds, since the hypervisor booted. * * \note This function returns the current CPU's idea of this value, unless * it happens to be less than another CPU's previously determined value. */ static uint64_t xen_fetch_vcpu_time(void) { struct vcpu_time_info dst; struct vcpu_time_info *src; uint32_t pre_version; uint64_t now; volatile uint64_t last; struct vcpu_info *vcpu = DPCPU_GET(vcpu_info); src = &vcpu->time; critical_enter(); do { pre_version = xen_fetch_vcpu_tinfo(&dst, src); barrier(); now = dst.system_time + *get_nsec_offset(&dst);* barrier(); } while (pre_version != src->version); /* * Enforce a monotonically increasing clock time across all * VCPUs. If our time is too old, use the last time and return. * Otherwise, try to update the last time. */ do { last = last_time; if (last > now) { now = last; break; } } while (!atomic_cmpset_64(&last_time, last, now)); critical_exit(); return (now); } ================================== I guest linux guest will do the same thing, rdtsc() fetch current timestamp from current running vCPU, TSC out-of-sync issue is still there. It seems to me pvclock finally fix the time drift issue just because the workaround enforced as above, right? Thanks, Michael [-- Attachment #1.2: Type: text/html, Size: 4304 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Could you please answer some questions regarding Solaris PVHVM pvclock support. 2014-02-26 11:52 ` Qin Li @ 2014-02-27 13:03 ` Stefano Stabellini 2014-02-28 3:27 ` Qin Li 0 siblings, 1 reply; 6+ messages in thread From: Stefano Stabellini @ 2014-02-27 13:03 UTC (permalink / raw) To: Qin Li; +Cc: Roger Pau Monne, xen-devel, Stefano Stabellini [-- Attachment #1: Type: text/plain, Size: 2947 bytes --] On Wed, 26 Feb 2014, Qin Li wrote: > > 于 2014/1/16 20:17, Stefano Stabellini 写道: > > . For a PV-on-HVM guest OS, the "shared_info->vcpu_info->vcpu_time_info" > > is already visiable. Does guest OS still need any action to ask > > hypervisor to update this piece of memory periodically? > > I don't think you need to ask the hypervisor to update vcpu_time_info > periodically, what gave you that idea? > > Hi Stefano, > > Now, I see it's the hypervisor that will update vcpu_time_info, but another thing confuse me: > HVM guest has time drift issue because TSC on different vCPU could be out-of-sync, especially after domain suspend/resume. > But how pvclock actually fix this issue? Let's see how FreeBSD port calculate the system time: > > ================== > static uint64_t > get_nsec_offset(struct vcpu_time_info *tinfo) > { > > return (scale_delta(rdtsc() - tinfo->tsc_timestamp, > tinfo->tsc_to_system_mul, tinfo->tsc_shift)); > } > > /** > * \brief Get the current time, in nanoseconds, since the hypervisor booted. > * > * \note This function returns the current CPU's idea of this value, unless > * it happens to be less than another CPU's previously determined value. > */ > static uint64_t > xen_fetch_vcpu_time(void) > { > struct vcpu_time_info dst; > struct vcpu_time_info *src; > uint32_t pre_version; > uint64_t now; > volatile uint64_t last; > struct vcpu_info *vcpu = DPCPU_GET(vcpu_info); > > src = &vcpu->time; > > critical_enter(); > do { > pre_version = xen_fetch_vcpu_tinfo(&dst, src); > barrier(); > now = dst.system_time + get_nsec_offset(&dst); > barrier(); > } while (pre_version != src->version); > > /* > * Enforce a monotonically increasing clock time across all > * VCPUs. If our time is too old, use the last time and return. > * Otherwise, try to update the last time. > */ > do { > last = last_time; > if (last > now) { > now = last; > break; > } > } while (!atomic_cmpset_64(&last_time, last, now)); > > critical_exit(); > > return (now); > } > ================================== > > I guest linux guest will do the same thing, rdtsc() fetch current timestamp from current running vCPU, TSC out-of-sync issue is still there. > It seems to me pvclock finally fix the time drift issue just because the workaround enforced as above, right? First you should know that TSC is not always guaranteed to be synchronized across multiple processors, especially on older systems. On "TSC-safe" systems, Xen would export a consistent TSC to guests, by setting the vtsc offset and scale appropriately. [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Could you please answer some questions regarding Solaris PVHVM pvclock support. 2014-02-27 13:03 ` Stefano Stabellini @ 2014-02-28 3:27 ` Qin Li 2014-02-28 15:04 ` Stefano Stabellini 0 siblings, 1 reply; 6+ messages in thread From: Qin Li @ 2014-02-28 3:27 UTC (permalink / raw) To: Stefano Stabellini; +Cc: xen-devel, Roger Pau Monne 于 2014/2/27 21:03, Stefano Stabellini 写道: >> >I guest linux guest will do the same thing, rdtsc() fetch current timestamp from current running vCPU, TSC out-of-sync issue is still there. >> >It seems to me pvclock finally fix the time drift issue just because the workaround enforced as above, right? > First you should know that TSC is not always guaranteed to be > synchronized across multiple processors, especially on older systems. > On "TSC-safe" systems, Xen would export a consistent TSC to guests, by > setting the vtsc offset and scale appropriately. Stefano, Is there a easy way to check whether the system is "TSC-safe"? Do you mean "X86_FEATURE_TSC_RELIABLE" bit in "boot_cpu_data.x86_capability"? Thanks, Michael _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Could you please answer some questions regarding Solaris PVHVM pvclock support. 2014-02-28 3:27 ` Qin Li @ 2014-02-28 15:04 ` Stefano Stabellini 0 siblings, 0 replies; 6+ messages in thread From: Stefano Stabellini @ 2014-02-28 15:04 UTC (permalink / raw) To: Qin Li; +Cc: Roger Pau Monne, xen-devel, Stefano Stabellini [-- Attachment #1: Type: text/plain, Size: 875 bytes --] On Fri, 28 Feb 2014, Qin Li wrote: > 于 2014/2/27 21:03, Stefano Stabellini 写道: > > > >I guest linux guest will do the same thing, rdtsc() fetch current > > > timestamp from current running vCPU, TSC out-of-sync issue is still there. > > > >It seems to me pvclock finally fix the time drift issue just because the > > > workaround enforced as above, right? > > First you should know that TSC is not always guaranteed to be > > synchronized across multiple processors, especially on older systems. > > On "TSC-safe" systems, Xen would export a consistent TSC to guests, by > > setting the vtsc offset and scale appropriately. > Stefano, > > Is there a easy way to check whether the system is "TSC-safe"? Do you mean > "X86_FEATURE_TSC_RELIABLE" bit in "boot_cpu_data.x86_capability"? Yes, I think that's it. Maybe you want to read docs/misc/tscmode.txt. [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-02-28 15:04 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <52D6566C.5050302@oracle.com> 2014-01-16 12:17 ` Could you please answer some questions regarding Solaris PVHVM pvclock support Stefano Stabellini 2014-01-24 0:21 ` Mukesh Rathor 2014-02-26 11:52 ` Qin Li 2014-02-27 13:03 ` Stefano Stabellini 2014-02-28 3:27 ` Qin Li 2014-02-28 15:04 ` Stefano Stabellini
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.