KVM Archive on lore.kernel.org
 help / color / Atom feed
From: Peter Xu <peterx@redhat.com>
To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	Christophe de Dinechin <dinechin@redhat.com>,
	peterx@redhat.com,
	Sean Christopherson <sean.j.christopherson@intel.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>
Subject: [PATCH RESEND v2 00/17] KVM: Dirty ring interface
Date: Fri, 20 Dec 2019 20:49:21 -0500
Message-ID: <20191221014938.58831-1-peterx@redhat.com> (raw)

Branch is here: https://github.com/xzpeter/linux/tree/kvm-dirty-ring
(based on 5.4.0)

This is v2 of the dirty ring series, and also the first non-RFC
version of it.  I didn't put a changelog from v1-rfc because I feel
like it would be easier to go into the patchset comparing to read that
lengthy and probably helpless changelog.  However I do like to do a
summary here on what has majorly changed, and also some conclusions on
the previous v1 discussions.

======================

* Per-vm ring is dropped

For x86 (which is still the major focus for now), we found that kvmgt
is probably the only one that still writes to the guest without a vcpu
context.  It would be a complete pity if we keep the per-vm ring only
for kvmgt (who shouldn't write directly to guest via kvm api after
all...), so remove it.  Work should be ongoing in parallel to refactor
kvmgt to not use kvm apis like kvm_write_guest().

However I don't want to break kvmgt before it's fixed.  So this series
uses an interim way to solve this by fallback no-vcpu-context writes
to vcpu0 if there is.  So we will keep the interface clean (per-vcpu
only), while we don't break the code base.  After kvmgt is fixed, we
can probably even drop this special fallback and kvm->dirty_ring_lock.

* Waitqueue is still kept (for now)

We did plan to drop the waitqueue, however again if with kvmgt we
still have chance to ful-fill a ring (and I feel like it'll definitely
happen if we migrate a kvmgt guest).  This series will only trigger
the waitqueue mechanism if it's the special case (no-vcpu-context) and
actually it naturally avoids another mmu lock deadlock issue I've
encountered, which is good.

For vcpu context writes, now the series is even more strict that we'll
directly fail the KVM_RUN if the dirty ring is soft full, until the
userspace collects the dirty rings first.  That'll guarantee the ring
will never be full.  With that, I dropped KVM_REQ_DIRTY_RING_FULL
together because then it's not needed.

Potentially this could still also be used by ARM when there're code
paths that dump the ARM device information to the guests
(e.g. KVM_DEV_ARM_ITS_SAVE_TABLES).  We'll see.  No matter what, even
if the code is there, x86 (as long as without kvmgt) should never
trigger waitqueue.

Although the waitqueue is kept, I dropped the complete waitqueue test,
simply because now I can never trigger it without kvmgt...

* Why not virtio?

There's already some discussion during v1 patchset on whether it's
good to use virtio for the data path of delivering dirty pages [1].
I'd confess the only thing that we might consider to use is the vring
layout (because virtqueue is tightly bound to devices, while we don't
have a device contet here), however it's a pity that even we only use
the most low-level vring api it'll be at least iov based which is
already an overkill for dirty ring (which is literally an array of
addresses).  So I just kept things easy.

======================

About the patchset:

Patch 1-5:    Mostly cleanups
Patch 6,7:    Prepare for the dirty ring interface
Patch 8-10:   Dirty ring implementation (majorly patch 8)
Patch 11-17:  Test cases update

Please have a look, thanks.

[1] V1 is here: https://lore.kernel.org/kvm/20191129213505.18472-1-peterx@redhat.com

Paolo Bonzini (1):
  KVM: Move running VCPU from ARM to common code

Peter Xu (16):
  KVM: Remove kvm_read_guest_atomic()
  KVM: X86: Change parameter for fast_page_fault tracepoint
  KVM: X86: Don't track dirty for KVM_SET_[TSS_ADDR|IDENTITY_MAP_ADDR]
  KVM: Cache as_id in kvm_memory_slot
  KVM: Add build-time error check on kvm_run size
  KVM: Pass in kvm pointer into mark_page_dirty_in_slot()
  KVM: X86: Implement ring-based dirty memory tracking
  KVM: Make dirty ring exclusive to dirty bitmap log
  KVM: Don't allocate dirty bitmap if dirty ring is enabled
  KVM: selftests: Always clear dirty bitmap after iteration
  KVM: selftests: Sync uapi/linux/kvm.h to tools/
  KVM: selftests: Use a single binary for dirty/clear log test
  KVM: selftests: Introduce after_vcpu_run hook for dirty log test
  KVM: selftests: Add dirty ring buffer test
  KVM: selftests: Let dirty_log_test async for dirty ring test
  KVM: selftests: Add "-c" parameter to dirty log test

 Documentation/virt/kvm/api.txt                |  96 ++++
 arch/arm/include/asm/kvm_host.h               |   2 -
 arch/arm64/include/asm/kvm_host.h             |   2 -
 arch/x86/include/asm/kvm_host.h               |   3 +
 arch/x86/include/uapi/asm/kvm.h               |   1 +
 arch/x86/kvm/Makefile                         |   3 +-
 arch/x86/kvm/mmu.c                            |   6 +
 arch/x86/kvm/mmutrace.h                       |   9 +-
 arch/x86/kvm/vmx/vmx.c                        |  25 +-
 arch/x86/kvm/x86.c                            |   9 +
 include/linux/kvm_dirty_ring.h                |  57 +++
 include/linux/kvm_host.h                      |  44 +-
 include/trace/events/kvm.h                    |  78 ++++
 include/uapi/linux/kvm.h                      |  31 ++
 tools/include/uapi/linux/kvm.h                |  36 ++
 tools/testing/selftests/kvm/Makefile          |   2 -
 .../selftests/kvm/clear_dirty_log_test.c      |   2 -
 tools/testing/selftests/kvm/dirty_log_test.c  | 420 ++++++++++++++++--
 .../testing/selftests/kvm/include/kvm_util.h  |   4 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  64 +++
 .../selftests/kvm/lib/kvm_util_internal.h     |   3 +
 virt/kvm/arm/arch_timer.c                     |   2 +-
 virt/kvm/arm/arm.c                            |  29 --
 virt/kvm/arm/perf.c                           |   6 +-
 virt/kvm/arm/vgic/vgic-mmio.c                 |  15 +-
 virt/kvm/dirty_ring.c                         | 201 +++++++++
 virt/kvm/kvm_main.c                           | 269 +++++++++--
 27 files changed, 1274 insertions(+), 145 deletions(-)
 create mode 100644 include/linux/kvm_dirty_ring.h
 delete mode 100644 tools/testing/selftests/kvm/clear_dirty_log_test.c
 create mode 100644 virt/kvm/dirty_ring.c

-- 
2.24.1


             reply index

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-21  1:49 Peter Xu [this message]
2019-12-21  1:49 ` [PATCH RESEND v2 01/17] KVM: Remove kvm_read_guest_atomic() Peter Xu
2020-01-08 17:45   ` Paolo Bonzini
2019-12-21  1:49 ` [PATCH RESEND v2 02/17] KVM: X86: Change parameter for fast_page_fault tracepoint Peter Xu
2020-01-08 17:46   ` Paolo Bonzini
2019-12-21  1:49 ` [PATCH RESEND v2 03/17] KVM: X86: Don't track dirty for KVM_SET_[TSS_ADDR|IDENTITY_MAP_ADDR] Peter Xu
2019-12-21 13:51   ` Paolo Bonzini
2019-12-23 17:27     ` Peter Xu
2019-12-23 17:59       ` Paolo Bonzini
2019-12-23 20:10         ` Peter Xu
2020-01-08 17:46           ` Paolo Bonzini
2020-01-08 19:15             ` Peter Xu
2020-01-08 19:44               ` Paolo Bonzini
2020-01-08 21:02                 ` Peter Xu
2019-12-21  1:49 ` [PATCH RESEND v2 04/17] KVM: Cache as_id in kvm_memory_slot Peter Xu
2020-01-08 17:47   ` Paolo Bonzini
2019-12-21  1:49 ` [PATCH RESEND v2 05/17] KVM: Add build-time error check on kvm_run size Peter Xu
2019-12-21  1:49 ` [PATCH RESEND v2 06/17] KVM: Pass in kvm pointer into mark_page_dirty_in_slot() Peter Xu
2020-01-08 17:47   ` Paolo Bonzini
2019-12-21  1:49 ` [PATCH RESEND v2 07/17] KVM: Move running VCPU from ARM to common code Peter Xu
2020-01-08 17:47   ` Paolo Bonzini
2019-12-21  1:49 ` [PATCH RESEND v2 08/17] KVM: X86: Implement ring-based dirty memory tracking Peter Xu
2019-12-24  6:16   ` Jason Wang
2019-12-24 15:08     ` Peter Xu
2019-12-25  3:23       ` Jason Wang
2020-01-08 15:52   ` Peter Xu
2020-01-08 17:41     ` Paolo Bonzini
2020-01-08 19:06       ` Peter Xu
2020-01-08 19:44         ` Paolo Bonzini
2020-01-08 19:59           ` Peter Xu
2020-01-08 20:06             ` Paolo Bonzini
2019-12-21  1:49 ` [PATCH RESEND v2 09/17] KVM: Make dirty ring exclusive to dirty bitmap log Peter Xu
2019-12-21  1:58 ` [PATCH RESEND v2 10/17] KVM: Don't allocate dirty bitmap if dirty ring is enabled Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 11/17] KVM: selftests: Always clear dirty bitmap after iteration Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 12/17] KVM: selftests: Sync uapi/linux/kvm.h to tools/ Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 13/17] KVM: selftests: Use a single binary for dirty/clear log test Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 14/17] KVM: selftests: Introduce after_vcpu_run hook for dirty " Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 15/17] KVM: selftests: Add dirty ring buffer test Peter Xu
2019-12-24  6:18   ` Jason Wang
2019-12-24 15:22     ` Peter Xu
2019-12-24  6:50   ` Jason Wang
2019-12-24 15:24     ` Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 16/17] KVM: selftests: Let dirty_log_test async for dirty ring test Peter Xu
2019-12-21  2:04 ` [PATCH RESEND v2 17/17] KVM: selftests: Add "-c" parameter to dirty log test Peter Xu
2019-12-24  6:34 ` [PATCH RESEND v2 00/17] KVM: Dirty ring interface Jason Wang

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191221014938.58831-1-peterx@redhat.com \
    --to=peterx@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=dinechin@redhat.com \
    --cc=jasowang@redhat.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mst@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=sean.j.christopherson@intel.com \
    --cc=vkuznets@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
		kvm@vger.kernel.org
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.kvm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git