Re: [RFC PATCH 0/6] x86/kvm: Virtual suspend time injection support

From: Maxim Levitsky <mlevitsk@redhat.com>
To: Hikaru Nishida <hikalium@chromium.org>, kvm@vger.kernel.org
Cc: suleiman@google.com, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	John Stultz <john.stultz@linaro.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Stephen Boyd <sboyd@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	x86@kernel.org
Subject: Re: [RFC PATCH 0/6] x86/kvm: Virtual suspend time injection support
Date: Mon, 26 Apr 2021 16:15:52 +0300	[thread overview]
Message-ID: <0cfc93405443bed335981ffd5cc07272ffb0ce3a.camel@redhat.com> (raw)
In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org>

On Mon, 2021-04-26 at 18:06 +0900, Hikaru Nishida wrote:
> Hi,
> 
> This patch series adds virtual suspend time injection support to KVM.
> 
> Before this change, if the host goes into suspended state while the
> guest is running, the guest will experience a time jump after the host's
> resume. This can confuse some services in the guest since they can't
> detect if the system went into suspend or not by comparing
> CLOCK_BOOTTIME and CLOCK_MONOTONIC.
> 
> To solve this problem, we wanted to add a way to adjust the guest clocks
> without actually suspending the guests. However, there was no way to
> modify a gap between CLOCK_BOOTTIME and CLOCK_MONOTONIC without actually
> suspending the guests. Therefore, this series introduce a new struct
> called kvm_host_suspend_time to share the suspend time between host and
> guest and a mechanism to inject a suspend time to the guest while
> keeping
> monotonicity of the clocks.
> 
> Could you take a look and let me know how we can improve the patches if
> they are doing something wrong?
> 
> Thanks,
> 
> Hikaru Nishida
> 

I haven't yet looked at that, but in my experience when I suspend the host
with VMs running, after resume all my VMs complain something about TSC watchdog
and stop using it. The TSC is stable/synchornized, but after resume it does
reset to 0 on all CPUs.

I use INVTSC flag for all my VMs.
I haven't investigated this futher yet.

Just my 0.2 cents.

Best regards,
	Maxim Levitsky

> 
> 
> Hikaru Nishida (6):
>   x86/kvm: Reserve KVM_FEATURE_HOST_SUSPEND_TIME and
>     MSR_KVM_HOST_SUSPEND_TIME
>   x86/kvm: Add a struct and constants for virtual suspend time injection
>   x86/kvm: Add CONFIG_KVM_VIRT_SUSPEND_TIMING
>   x86/kvm: Add a host side support for virtual suspend time injection
>   x86/kvm: Add CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST
>   x86/kvm: Add a guest side support for virtual suspend time injection
> 
>  Documentation/virt/kvm/cpuid.rst     |  3 +
>  Documentation/virt/kvm/msr.rst       | 29 +++++++++
>  arch/x86/Kconfig                     | 13 ++++
>  arch/x86/include/asm/kvm_host.h      |  5 ++
>  arch/x86/include/asm/kvm_para.h      |  9 +++
>  arch/x86/include/uapi/asm/kvm_para.h |  6 ++
>  arch/x86/kernel/kvmclock.c           | 25 ++++++++
>  arch/x86/kvm/Kconfig                 | 13 ++++
>  arch/x86/kvm/cpuid.c                 |  4 ++
>  arch/x86/kvm/x86.c                   | 89 +++++++++++++++++++++++++++-
>  include/linux/kvm_host.h             |  7 +++
>  include/linux/timekeeper_internal.h  |  4 ++
>  kernel/time/timekeeping.c            | 31 ++++++++++
>  13 files changed, 237 insertions(+), 1 deletion(-)
> 