Re: [PATCH v2 06/14] x86/time.c: Scale host TSC in pvclock properly

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 12/07/2015 08:52 PM, Haozhong Zhang wrote:
> On 12/07/15 13:14, Boris Ostrovsky wrote:
>> On 12/06/2015 03:58 PM, Haozhong Zhang wrote:
>>> This patch makes the pvclock return the scaled host TSC and
>>> corresponding scaling parameters to HVM domains if guest TSC is not
>>> emulated and TSC scaling is enabled.
>>>
>>> Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
>> +Joao who has been staring at this code.
>>
>> Joao, can you run this series through your test with non-native frequency?
>> (http://lists.xenproject.org/archives/html/xen-devel/2015-12/msg00683.html
>> provides an interface to set it in config file).
>>
>>
>>> ---
>>>   xen/arch/x86/time.c | 16 ++++++++++++----
>>>   1 file changed, 12 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
>>> index 95df4f1..732d1e9 100644
>>> --- a/xen/arch/x86/time.c
>>> +++ b/xen/arch/x86/time.c
>>> @@ -815,10 +815,18 @@ static void __update_vcpu_system_time(struct vcpu *v, int force)
>>>       }
>>>       else
>>>       {
>>> -        tsc_stamp = t->local_tsc_stamp;
>>> -
>>> -        _u.tsc_to_system_mul = t->tsc_scale.mul_frac;
>>> -        _u.tsc_shift         = (s8)t->tsc_scale.shift;
>>> +        if ( is_hvm_domain(d) && cpu_has_tsc_ratio )
>>> +        {
>>> +            tsc_stamp            = hvm_funcs.scale_tsc(v, t->local_tsc_stamp);
>>> +            _u.tsc_to_system_mul = d->arch.vtsc_to_ns.mul_frac;
>>> +            _u.tsc_shift         = d->arch.vtsc_to_ns.shift;
>> I am not sure this is correct (which is why I asked Joao to look at this and
>> test). The scaler below is calculated as result of TSC calibration across
>> physical CPUs and what you use above (vtsc_to_ns) is an uncalibrated value.
>>
> Because guest TSC is synchronized among all vcpus of a domain, I think
> it's safe to use d->arch.vtsc_to_ns here.

This is only guaranteed if we have constant/reliable TSC. So perhaps you 
should set tsc_scaling_supported only when either (or both?) of these 
TSC properties are true. Which is probably the case anyway but may be 
worth explicitly checking in start_svm/vmx? (and use 
tsc_scaling_supported instead of cpu_has_tsc_ratio in the 'if' statement)

And just like I asked in the previous email --- should we then use the 
same scaler (which would be vtsc_to_ns) in both cases? At least for 
guests in HVM containers (it may work for PV guests as well, but it 
would need to be confirmed).

Also, I noticed that this routine uses is_hvm_domain(). I think it 
should be has_hvm_container_domain().

-boris