Linux-MIPS Archive on lore.kernel.org
 help / color / Atom feed
From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Paul Mackerras" <paulus@ozlabs.org>,
	"Christian Borntraeger" <borntraeger@de.ibm.com>,
	"Janosch Frank" <frankja@linux.ibm.com>,
	"David Hildenbrand" <david@redhat.com>,
	"Cornelia Huck" <cohuck@redhat.com>,
	"Sean Christopherson" <sean.j.christopherson@intel.com>,
	"Vitaly Kuznetsov" <vkuznets@redhat.com>,
	"Wanpeng Li" <wanpengli@tencent.com>,
	"Jim Mattson" <jmattson@google.com>,
	"Joerg Roedel" <joro@8bytes.org>, "Marc Zyngier" <maz@kernel.org>,
	"James Morse" <james.morse@arm.com>,
	"Julien Thierry" <julien.thierry.kdev@gmail.com>,
	"Suzuki K Poulose" <suzuki.poulose@arm.com>,
	linux-mips@vger.kernel.org, kvm@vger.kernel.org,
	kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org,
	"Christoffer Dall" <christoffer.dall@arm.com>,
	"Peter Xu" <peterx@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>
Subject: [PATCH v5 00/19] KVM: Dynamically size memslot arrays
Date: Tue, 21 Jan 2020 14:31:38 -0800
Message-ID: <20200121223157.15263-1-sean.j.christopherson@intel.com> (raw)

I'd love to get this into 5.6 if possible.  The rebase to kvm/queue only
had a single superficial conflict with 668effb63de8 ("KVM: Fix some wrong
function names in comment"), and s390 (Christian) and arm64 (Marc) both
got smoke tested in v4.

Thanks!

v5:
  - Make the selftest x86-only. [Christian].
  - Collect tags. [Peter]
  - Rebase to kvm/queue, fb0c5f8fb698 ("KVM: x86: inline memslot_...").

v4:
  - Add patch 01 to fix an x86 rmap/lpage bug, and patches 10 and 11 to
    resolve hidden conflicts with the bug fix.
  - Collect tags [Christian, Marc, Philippe].
  - Rebase to kvm/queue, commit e41a90be9659 ("KVM: x86/mmu: WARN if
    root_hpa is invalid when handling a page fault").
v3:
  - Fix build errors on PPC and MIPS due to missed params during
    refactoring [kbuild test robot].
  - Rename the helpers for update_memslots() and add comments describing
    the new algorithm and how it interacts with searching [Paolo].
  - Remove the unnecessary and obnoxious warning regarding memslots being
    a flexible array [Paolo].
  - Fix typos in the changelog of patch 09/15 [Christoffer].
  - Collect tags [Christoffer].

v2:
  - Split "Drop kvm_arch_create_memslot()" into three patches to move
    minor functional changes to standalone patches [Janosch].
  - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
  - Collect an Acked-by and a Reviewed-by

*** v1 cover letter ***

The end goal of this series is to dynamically size the memslot array so
that KVM allocates memory based on the number of memslots in use, as
opposed to unconditionally allocating memory for the maximum number of
memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
the memory footprint from 90k to ~2.6k bytes.

The changes required to support dynamic sizing are relatively small,
e.g. are essentially contained in patches 17/19 and 18/19.

Patches 2-16 clean up the memslot code, which has gotten quite crusty,
especially __kvm_set_memory_region().  The clean up is likely not strictly
necessary to switch to dynamic sizing, but I didn't have a remotely
reasonable level of confidence in the correctness of the dynamic sizing
without first doing the clean up.

The only functional change in v4 is the addition of an x86-specific bug
fix in x86's handling of KVM_MR_MOVE.  The bug fix is not directly related
to dynamically allocating memslots, but it has subtle and hidden conflicts
with the cleanup patches, and the fix is higher priority than anything
else in the series, i.e. should be merged first.

On non-x86 architectures, v3 and v4 should be functionally equivalent,
the only non-x86 change in v4 is the dropping of a "const" in
kvm_arch_commit_memory_region().

Sean Christopherson (19):
  KVM: x86: Allocate new rmap and large page tracking when moving
    memslot
  KVM: Reinstall old memslots if arch preparation fails
  KVM: Don't free new memslot if allocation of said memslot fails
  KVM: PPC: Move memslot memory allocation into prepare_memory_region()
  KVM: x86: Allocate memslot resources during prepare_memory_region()
  KVM: Drop kvm_arch_create_memslot()
  KVM: Explicitly free allocated-but-unused dirty bitmap
  KVM: Refactor error handling for setting memory region
  KVM: Move setting of memslot into helper routine
  KVM: Drop "const" attribute from old memslot in commit_memory_region()
  KVM: x86: Free arrays for old memslot when moving memslot's base gfn
  KVM: Move memslot deletion to helper function
  KVM: Simplify kvm_free_memslot() and all its descendents
  KVM: Clean up local variable usage in __kvm_set_memory_region()
  KVM: Provide common implementation for generic dirty log functions
  KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
  KVM: Terminate memslot walks via used_slots
  KVM: Dynamically size memslot array based on number of used slots
  KVM: selftests: Add test for KVM_SET_USER_MEMORY_REGION

 arch/mips/include/asm/kvm_host.h              |   2 +-
 arch/mips/kvm/mips.c                          |  71 +-
 arch/powerpc/include/asm/kvm_ppc.h            |  17 +-
 arch/powerpc/kvm/book3s.c                     |  22 +-
 arch/powerpc/kvm/book3s_hv.c                  |  36 +-
 arch/powerpc/kvm/book3s_pr.c                  |  20 +-
 arch/powerpc/kvm/booke.c                      |  17 +-
 arch/powerpc/kvm/powerpc.c                    |  15 +-
 arch/s390/include/asm/kvm_host.h              |   2 +-
 arch/s390/kvm/kvm-s390.c                      |  23 +-
 arch/x86/include/asm/kvm_page_track.h         |   3 +-
 arch/x86/kvm/mmu/page_track.c                 |  15 +-
 arch/x86/kvm/x86.c                            | 114 +---
 include/linux/kvm_host.h                      |  48 +-
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/include/kvm_util.h  |   1 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  30 +
 .../kvm/x86_64/set_memory_region_test.c       | 142 ++++
 virt/kvm/arm/arm.c                            |  48 +-
 virt/kvm/arm/mmu.c                            |  20 +-
 virt/kvm/kvm_main.c                           | 621 ++++++++++++------
 22 files changed, 734 insertions(+), 535 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/set_memory_region_test.c

-- 
2.24.1


             reply index

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-21 22:31 Sean Christopherson [this message]
2020-01-21 22:31 ` [PATCH v5 01/19] KVM: x86: Allocate new rmap and large page tracking when moving memslot Sean Christopherson
2020-02-05 21:49   ` Peter Xu
2020-02-05 23:55     ` Sean Christopherson
2020-02-06  2:00       ` Peter Xu
2020-02-06  2:17         ` Sean Christopherson
2020-02-06  2:58           ` Peter Xu
2020-02-06  5:05             ` Sean Christopherson
2020-01-21 22:31 ` [PATCH v5 02/19] KVM: Reinstall old memslots if arch preparation fails Sean Christopherson
2020-02-05 22:08   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 03/19] KVM: Don't free new memslot if allocation of said memslot fails Sean Christopherson
2020-02-05 22:28   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 04/19] KVM: PPC: Move memslot memory allocation into prepare_memory_region() Sean Christopherson
2020-02-05 22:41   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 05/19] KVM: x86: Allocate memslot resources during prepare_memory_region() Sean Christopherson
2020-01-21 22:31 ` [PATCH v5 06/19] KVM: Drop kvm_arch_create_memslot() Sean Christopherson
2020-02-05 22:45   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 07/19] KVM: Explicitly free allocated-but-unused dirty bitmap Sean Christopherson
2020-01-21 22:31 ` [PATCH v5 08/19] KVM: Refactor error handling for setting memory region Sean Christopherson
2020-02-05 22:48   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 09/19] KVM: Move setting of memslot into helper routine Sean Christopherson
2020-02-06 16:26   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 10/19] KVM: Drop "const" attribute from old memslot in commit_memory_region() Sean Christopherson
2020-02-06 16:26   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 11/19] KVM: x86: Free arrays for old memslot when moving memslot's base gfn Sean Christopherson
2020-01-21 22:31 ` [PATCH v5 12/19] KVM: Move memslot deletion to helper function Sean Christopherson
2020-02-06 16:14   ` Peter Xu
2020-02-06 16:28     ` Sean Christopherson
2020-02-06 16:51       ` Peter Xu
2020-02-07 17:59         ` Sean Christopherson
2020-02-07 18:07           ` Sean Christopherson
2020-02-07 18:17           ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 13/19] KVM: Simplify kvm_free_memslot() and all its descendents Sean Christopherson
2020-02-06 16:29   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 14/19] KVM: Clean up local variable usage in __kvm_set_memory_region() Sean Christopherson
2020-02-06 19:06   ` Peter Xu
2020-02-06 19:22     ` Sean Christopherson
2020-02-06 19:36       ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 15/19] KVM: Provide common implementation for generic dirty log functions Sean Christopherson
2020-02-06 20:02   ` Peter Xu
2020-02-06 21:21     ` Sean Christopherson
2020-02-06 21:41       ` Peter Xu
2020-02-07 19:45         ` Sean Christopherson
2020-02-08  0:18           ` Peter Xu
2020-02-08  0:42             ` Sean Christopherson
2020-02-08  0:53               ` Peter Xu
2020-02-08  1:29                 ` Sean Christopherson
2020-02-17 15:39                   ` Vitaly Kuznetsov
2020-02-18 17:10                     ` Sean Christopherson
2020-02-17 15:35           ` Vitaly Kuznetsov
2020-02-06 21:24   ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 16/19] KVM: Ensure validity of memslot with respect to kvm_get_dirty_log() Sean Christopherson
2020-01-21 22:31 ` [PATCH v5 17/19] KVM: Terminate memslot walks via used_slots Sean Christopherson
2020-02-06 21:09   ` Peter Xu
2020-02-07 18:33     ` Sean Christopherson
2020-02-07 20:39       ` Peter Xu
2020-02-07 21:10         ` Sean Christopherson
2020-02-07 21:46           ` Peter Xu
2020-02-07 22:03             ` Sean Christopherson
2020-02-07 22:24               ` Peter Xu
2020-01-21 22:31 ` [PATCH v5 18/19] KVM: Dynamically size memslot array based on number of used slots Sean Christopherson
2020-02-06 22:12   ` Peter Xu
2020-02-07 15:38     ` Sean Christopherson
2020-02-07 16:05       ` Peter Xu
2020-02-07 16:15         ` Sean Christopherson
2020-02-07 16:37           ` Peter Xu
2020-02-07 16:47             ` Sean Christopherson
2020-01-21 22:31 ` [PATCH v5 19/19] KVM: selftests: Add test for KVM_SET_USER_MEMORY_REGION Sean Christopherson
2020-02-06 22:30   ` Peter Xu
2020-02-06 23:11     ` Sean Christopherson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200121223157.15263-1-sean.j.christopherson@intel.com \
    --to=sean.j.christopherson@intel.com \
    --cc=borntraeger@de.ibm.com \
    --cc=christoffer.dall@arm.com \
    --cc=cohuck@redhat.com \
    --cc=david@redhat.com \
    --cc=f4bug@amsat.org \
    --cc=frankja@linux.ibm.com \
    --cc=james.morse@arm.com \
    --cc=jmattson@google.com \
    --cc=joro@8bytes.org \
    --cc=julien.thierry.kdev@gmail.com \
    --cc=kvm-ppc@vger.kernel.org \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mips@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=paulus@ozlabs.org \
    --cc=pbonzini@redhat.com \
    --cc=peterx@redhat.com \
    --cc=suzuki.poulose@arm.com \
    --cc=vkuznets@redhat.com \
    --cc=wanpengli@tencent.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-MIPS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mips/0 linux-mips/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mips linux-mips/ https://lore.kernel.org/linux-mips \
		linux-mips@vger.kernel.org
	public-inbox-index linux-mips

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-mips


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git