Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

From: Andy Lutomirski <luto@amacapital.net>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Maxim Levitsky <mlevitsk@redhat.com>,
	kvm@vger.kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Jim Mattson <jmattson@google.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	Sean Christopherson <sean.j.christopherson@intel.com>,
	open list <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@redhat.com>,
	"maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)"
	<x86@kernel.org>, Joerg Roedel <joro@8bytes.org>,
	Borislav Petkov <bp@alien8.de>, Shuah Khan <shuah@kernel.org>,
	Andrew Jones <drjones@redhat.com>,
	Oliver Upton <oupton@google.com>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>
Subject: Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE
Date: Mon, 7 Dec 2020 08:53:59 -0800	[thread overview]
Message-ID: <905DFDCE-71A5-4711-A31B-9FCFEA2CFC52@amacapital.net> (raw)
In-Reply-To: <87a6up606r.fsf@nanos.tec.linutronix.de>

> On Dec 7, 2020, at 8:38 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> On Mon, Dec 07 2020 at 14:16, Maxim Levitsky wrote:
>>> On Sun, 2020-12-06 at 17:19 +0100, Thomas Gleixner wrote:
>>> From a timekeeping POV and the guests expectation of TSC this is
>>> fundamentally wrong:
>>> 
>>>      tscguest = scaled(hosttsc) + offset
>>> 
>>> The TSC has to be viewed systemwide and not per CPU. It's systemwide
>>> used for timekeeping and for that to work it has to be synchronized. 
>>> 
>>> Why would this be different on virt? Just because it's virt or what? 
>>> 
>>> Migration is a guest wide thing and you're not migrating single vCPUs.
>>> 
>>> This hackery just papers over he underlying design fail that KVM looks
>>> at the TSC per vCPU which is the root cause and that needs to be fixed.
>> 
>> I don't disagree with you.
>> As far as I know the main reasons that kvm tracks TSC per guest are
>> 
>> 1. cases when host tsc is not stable 
>> (hopefully rare now, and I don't mind making
>> the new API just refuse to work when this is detected, and revert to old way
>> of doing things).
> 
> That's a trainwreck to begin with and I really would just not support it
> for anything new which aims to be more precise and correct.  TSC has
> become pretty reliable over the years.
> 
>> 2. (theoretical) ability of the guest to introduce per core tsc offfset
>> by either using TSC_ADJUST (for which I got recently an idea to stop
>> advertising this feature to the guest), or writing TSC directly which
>> is allowed by Intel's PRM:
> 
> For anything halfways modern the write to TSC is reflected in TSC_ADJUST
> which means you get the precise offset.
> 
> The general principle still applies from a system POV.
> 
>     TSC base (systemwide view) - The sane case
> 
>     TSC CPU  = TSC base + TSC_ADJUST
> 
> The guest TSC base is a per guest constant offset to the host TSC.
> 
>     TSC guest base = TSC host base + guest base offset
> 
> If the guest want's this different per vCPU by writing to the MSR or to
> TSC_ADJUST then you still can have a per vCPU offset in TSC_ADJUST which
> is the offset to the TSC base of the guest.

How about, if the guest wants to write TSC_ADJUST, it can turn off all paravirt features and keep both pieces?