KVM Archive on lore.kernel.org
 help / color / Atom feed
From: David Woodhouse <dwmw@amazon.co.uk>
To: tglx@linutronix.de, karahmed@amazon.de, sironi@amazon.de,
	x86@kernel.org, kvm@vger.kernel.org,
	torvalds@linux-foundation.org, pbonzini@redhat.com,
	linux-kernel@vger.kernel.org, bp@alien8.de, peterz@infradead.org,
	jmattson@google.com, rkrcmar@redhat.com,
	arjan.van.de.ven@intel.com, dave.hansen@intel.com
Subject: [PATCH v2 3/6] KVM: x86: Reduce retpoline performance impact in slot_handle_level_range()
Date: Sat, 10 Feb 2018 23:39:24 +0000
Message-ID: <1518305967-31356-4-git-send-email-dwmw@amazon.co.uk> (raw)
In-Reply-To: <1518305967-31356-1-git-send-email-dwmw@amazon.co.uk>

With retpoline, tight loops of "call this function for every XXX" are
very much pessimised by taking a prediction miss *every* time. This one
is by far the biggest contributor to the guest launch time with retpoline.

By marking the iterator slot_handle_…() functions always_inline, we can
ensure that the indirect function call can be optimised away into a
direct call and it actually generates slightly smaller code because
some of the other conditionals can get optimised away too.

Performance is now pretty close to what we see with nospectre_v2 on
the command line.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Filippo Sironi <sironi@amazon.de>
Tested-by: Filippo Sironi <sironi@amazon.de>
 arch/x86/kvm/mmu.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 2b8eb4d..cc83bdc 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -5058,7 +5058,7 @@ void kvm_mmu_uninit_vm(struct kvm *kvm)
 typedef bool (*slot_level_handler) (struct kvm *kvm, struct kvm_rmap_head *rmap_head);
 /* The caller should hold mmu-lock before calling this function. */
-static bool
+static __always_inline bool
 slot_handle_level_range(struct kvm *kvm, struct kvm_memory_slot *memslot,
 			slot_level_handler fn, int start_level, int end_level,
 			gfn_t start_gfn, gfn_t end_gfn, bool lock_flush_tlb)
@@ -5088,7 +5088,7 @@ slot_handle_level_range(struct kvm *kvm, struct kvm_memory_slot *memslot,
 	return flush;
-static bool
+static __always_inline bool
 slot_handle_level(struct kvm *kvm, struct kvm_memory_slot *memslot,
 		  slot_level_handler fn, int start_level, int end_level,
 		  bool lock_flush_tlb)
@@ -5099,7 +5099,7 @@ slot_handle_level(struct kvm *kvm, struct kvm_memory_slot *memslot,
-static bool
+static __always_inline bool
 slot_handle_all_level(struct kvm *kvm, struct kvm_memory_slot *memslot,
 		      slot_level_handler fn, bool lock_flush_tlb)
@@ -5107,7 +5107,7 @@ slot_handle_all_level(struct kvm *kvm, struct kvm_memory_slot *memslot,
 				 PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb);
-static bool
+static __always_inline bool
 slot_handle_large_level(struct kvm *kvm, struct kvm_memory_slot *memslot,
 			slot_level_handler fn, bool lock_flush_tlb)
@@ -5115,7 +5115,7 @@ slot_handle_large_level(struct kvm *kvm, struct kvm_memory_slot *memslot,
 				 PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb);
-static bool
+static __always_inline bool
 slot_handle_leaf(struct kvm *kvm, struct kvm_memory_slot *memslot,
 		 slot_level_handler fn, bool lock_flush_tlb)

  parent reply index

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-10 23:39 [PATCH v2 0/6] Spectre v2 updates David Woodhouse
2018-02-10 23:39 ` [PATCH v2 1/6] x86/speculation: Update Speculation Control microcode blacklist David Woodhouse
2018-02-12  9:50   ` Darren Kenny
2018-02-12 14:16   ` David Woodhouse
2018-02-12 14:32     ` Thomas Gleixner
2018-02-10 23:39 ` [PATCH v2 2/6] Revert "x86/speculation: Simplify indirect_branch_prediction_barrier()" David Woodhouse
2018-02-10 23:39 ` David Woodhouse [this message]
2018-02-10 23:39 ` [PATCH v2 4/6] X86/nVMX: Properly set spec_ctrl and pred_cmd before merging MSRs David Woodhouse
2018-02-10 23:39 ` [PATCH v2 5/6] KVM/nVMX: Set the CPU_BASED_USE_MSR_BITMAPS if we have a valid L02 MSR bitmap David Woodhouse
2018-02-11 10:19   ` Ingo Molnar
     [not found]     ` <1518345844.3677.365.camel@amazon.co.uk>
2018-02-11 10:55       ` Ingo Molnar
2018-02-10 23:39 ` [PATCH v2 6/6] x86/speculation: Use IBRS if available before calling into firmware David Woodhouse
2018-02-11 11:46   ` Ingo Molnar
2018-02-11 10:41 ` [PATCH v2 0/6] Spectre v2 updates Ingo Molnar
     [not found] ` <1518362359-1005-1-git-send-email-dwmw@amazon.co.uk>
2018-02-11 18:50   ` [PATCH] x86/speculation: Clean up various Spectre related details Ingo Molnar
2018-02-11 19:25     ` David Woodhouse
2018-02-11 19:43       ` Ingo Molnar
2018-02-12 15:30         ` David Woodhouse
2018-02-13  8:04           ` Ingo Molnar
2018-02-12  8:27 ` [PATCH v2 0/6] Spectre v2 updates Paolo Bonzini
2018-02-13  7:59   ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1518305967-31356-4-git-send-email-dwmw@amazon.co.uk \
    --to=dwmw@amazon.co.uk \
    --cc=arjan.van.de.ven@intel.com \
    --cc=bp@alien8.de \
    --cc=dave.hansen@intel.com \
    --cc=jmattson@google.com \
    --cc=karahmed@amazon.de \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rkrcmar@redhat.com \
    --cc=sironi@amazon.de \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=x86@kernel.org \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git