From: Oliver Upton <oupton@google.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
Sean Christopherson <seanjc@google.com>,
Marc Zyngier <maz@kernel.org>, Peter Shier <pshier@google.com>,
Jim Mattson <jmattson@google.com>,
David Matlack <dmatlack@google.com>,
Ricardo Koller <ricarkol@google.com>,
Jing Zhang <jingzhangos@google.com>,
Raghavendra Rao Anata <rananta@google.com>,
James Morse <james.morse@arm.com>,
Alexandru Elisei <Alexandru.Elisei@arm.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
linux-arm-kernel@lists.infradead.org,
Andrew Jones <drjones@redhat.com>, Will Deacon <will@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>
Subject: Re: [PATCH v6 00/21] KVM: Add idempotent controls for migrating system counter state
Date: Wed, 11 Aug 2021 11:56:22 -0700 [thread overview]
Message-ID: <CAOQ_Qsjby+z_kU49_s0PDKo5c3V-UD+FRg-2PcPycNq-p2gPVg@mail.gmail.com> (raw)
In-Reply-To: <927240ff-a4f4-fcc6-ae1b-92cefeda9e59@redhat.com>
On Wed, Aug 11, 2021 at 6:05 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 04/08/21 10:57, Oliver Upton wrote:
> > KVM's current means of saving/restoring system counters is plagued with
> > temporal issues. At least on ARM64 and x86, we migrate the guest's
> > system counter by-value through the respective guest system register
> > values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
> > brittle as the state is not idempotent: the host system counter is still
> > oscillating between the attempted save and restore. Furthermore, VMMs
> > may wish to transparently live migrate guest VMs, meaning that they
> > include the elapsed time due to live migration blackout in the guest
> > system counter view. The VMM thread could be preempted for any number of
> > reasons (scheduler, L0 hypervisor under nested) between the time that
> > it calculates the desired guest counter value and when KVM actually sets
> > this counter state.
> >
> > Despite the value-based interface that we present to userspace, KVM
> > actually has idempotent guest controls by way of system counter offsets.
> > We can avoid all of the issues associated with a value-based interface
> > by abstracting these offset controls in new ioctls. This series
> > introduces new vCPU device attributes to provide userspace access to the
> > vCPU's system counter offset.
> >
> > Patch 1 addresses a possible race in KVM_GET_CLOCK where
> > use_master_clock is read outside of the pvclock_gtod_sync_lock.
> >
> > Patch 2 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
> > ioctls to provide userspace with a (host_tsc, realtime) instant. This is
> > essential for a VMM to perform precise migration of the guest's system
> > counters.
> >
> > Patches 3-4 are some preparatory changes for exposing the TSC offset to
> > userspace. Patch 5 provides a vCPU attribute to provide userspace access
> > to the TSC offset.
> >
> > Patches 6-7 implement a test for the new additions to
> > KVM_{GET,SET}_CLOCK.
> >
> > Patch 8 fixes some assertions in the kvm device attribute helpers.
> >
> > Patches 9-10 implement at test for the tsc offset attribute introduced in
> > patch 5.
>
> The x86 parts look good, except that patch 3 is a bit redundant with my
> idea of altogether getting rid of the pvclock_gtod_sync_lock. That said
> I agree that patches 1 and 2 (and extracting kvm_vm_ioctl_get_clock and
> kvm_vm_ioctl_set_clock) should be done before whatever locking changes
> have to be done.
Following up on patch 3.
> Time is ticking for 5.15 due to my vacation, I'll see if I have some
> time to look at it further next week.
>
> I agree that arm64 can be done separately from x86.
Marc, just a disclaimer:
I'm going to separate these two series, although there will still
exist dependencies in the selftests changes. Otherwise, kernel changes
are disjoint.
--
Thanks,
Oliver
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-08-11 18:58 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-04 8:57 [PATCH v6 00/21] KVM: Add idempotent controls for migrating system counter state Oliver Upton
2021-08-04 8:57 ` [PATCH v6 01/21] KVM: x86: Fix potential race in KVM_GET_CLOCK Oliver Upton
2021-08-11 12:23 ` Paolo Bonzini
2021-08-13 10:39 ` Oliver Upton
2021-08-13 10:44 ` Paolo Bonzini
2021-08-13 17:46 ` Oliver Upton
2021-08-04 8:58 ` [PATCH v6 02/21] KVM: x86: Report host tsc and realtime values " Oliver Upton
2021-08-04 8:58 ` [PATCH v6 03/21] KVM: x86: Take the pvclock sync lock behind the tsc_write_lock Oliver Upton
2021-08-04 8:58 ` [PATCH v6 04/21] KVM: x86: Refactor tsc synchronization code Oliver Upton
2021-08-04 8:58 ` [PATCH v6 05/21] KVM: x86: Expose TSC offset controls to userspace Oliver Upton
2021-08-04 8:58 ` [PATCH v6 06/21] tools: arch: x86: pull in pvclock headers Oliver Upton
2021-08-04 8:58 ` [PATCH v6 07/21] selftests: KVM: Add test for KVM_{GET,SET}_CLOCK Oliver Upton
2021-08-04 8:58 ` [PATCH v6 08/21] selftests: KVM: Fix kvm device helper ioctl assertions Oliver Upton
2021-08-04 8:58 ` [PATCH v6 09/21] selftests: KVM: Add helpers for vCPU device attributes Oliver Upton
2021-08-04 8:58 ` [PATCH v6 10/21] selftests: KVM: Introduce system counter offset test Oliver Upton
2021-08-04 8:58 ` [PATCH v6 11/21] KVM: arm64: Refactor update_vtimer_cntvoff() Oliver Upton
2021-08-04 9:23 ` Andrew Jones
2021-08-04 8:58 ` [PATCH v6 12/21] KVM: arm64: Separate guest/host counter offset values Oliver Upton
2021-08-04 10:19 ` Andrew Jones
2021-08-04 8:58 ` [PATCH v6 13/21] KVM: arm64: Allow userspace to configure a vCPU's virtual offset Oliver Upton
2021-08-04 10:20 ` Andrew Jones
2021-08-10 9:35 ` Marc Zyngier
2021-08-10 9:44 ` Oliver Upton
2021-08-11 15:22 ` Marc Zyngier
2021-08-04 8:58 ` [PATCH v6 14/21] selftests: KVM: Add helper to check for register presence Oliver Upton
2021-08-04 9:14 ` Andrew Jones
2021-08-04 8:58 ` [PATCH v6 15/21] selftests: KVM: Add support for aarch64 to system_counter_offset_test Oliver Upton
2021-08-04 8:58 ` [PATCH v6 16/21] arm64: cpufeature: Enumerate support for Enhanced Counter Virtualization Oliver Upton
2021-08-10 9:38 ` Marc Zyngier
2021-08-04 8:58 ` [PATCH v6 17/21] KVM: arm64: Allow userspace to configure a guest's counter-timer offset Oliver Upton
2021-08-04 10:17 ` Andrew Jones
2021-08-04 10:22 ` Oliver Upton
2021-08-10 10:56 ` Marc Zyngier
2021-08-10 17:55 ` Oliver Upton
2021-08-11 9:01 ` Marc Zyngier
2021-08-04 8:58 ` [PATCH v6 18/21] KVM: arm64: Configure timer traps in vcpu_load() for VHE Oliver Upton
2021-08-04 10:25 ` Andrew Jones
2021-08-04 8:58 ` [PATCH v6 19/21] KVM: arm64: Emulate physical counter offsetting on non-ECV systems Oliver Upton
2021-08-04 11:05 ` Andrew Jones
2021-08-05 6:27 ` Oliver Upton
2021-08-10 11:27 ` Marc Zyngier
2021-08-04 8:58 ` [PATCH v6 20/21] selftests: KVM: Test physical counter offsetting Oliver Upton
2021-08-04 11:03 ` Andrew Jones
2021-08-04 8:58 ` [PATCH v6 21/21] selftests: KVM: Add counter emulation benchmark Oliver Upton
2021-08-04 11:05 ` [PATCH v6 00/21] KVM: Add idempotent controls for migrating system counter state Oliver Upton
2021-08-04 22:03 ` Oliver Upton
2021-08-10 0:04 ` Oliver Upton
2021-08-10 12:30 ` Marc Zyngier
2021-08-11 13:05 ` Paolo Bonzini
2021-08-11 18:56 ` Oliver Upton [this message]
2021-08-11 19:01 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOQ_Qsjby+z_kU49_s0PDKo5c3V-UD+FRg-2PcPycNq-p2gPVg@mail.gmail.com \
--to=oupton@google.com \
--cc=Alexandru.Elisei@arm.com \
--cc=catalin.marinas@arm.com \
--cc=dmatlack@google.com \
--cc=drjones@redhat.com \
--cc=james.morse@arm.com \
--cc=jingzhangos@google.com \
--cc=jmattson@google.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=maz@kernel.org \
--cc=pbonzini@redhat.com \
--cc=pshier@google.com \
--cc=rananta@google.com \
--cc=ricarkol@google.com \
--cc=seanjc@google.com \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).