[RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Aneesh Kumar K.V]

Hi All,

This patch series support enabling HV and PR KVM together in the same kernel. We
extend machine property with new property "kvm_type". A value of 1 will force HV
KVM and 2 PR KVM. The default value is 0 which will select the fastest KVM mode.
ie, HV if that is supported otherwise PR.

With Qemu command line having

 -machine pseries,accel=kvm,kvm_type=1

[root@llmp24l02 qemu]# bash ../qemu
failed to initialize KVM: Invalid argument
[root@llmp24l02 qemu]# modprobe kvm-pr
[root@llmp24l02 qemu]# bash ../qemu
failed to initialize KVM: Invalid argument
[root@llmp24l02 qemu]# modprobe  kvm-hv
[root@llmp24l02 qemu]# bash ../qemu

now with

 -machine pseries,accel=kvm,kvm_type=2

[root@llmp24l02 qemu]# rmmod kvm-pr
[root@llmp24l02 qemu]# bash ../qemu
failed to initialize KVM: Invalid argument
[root@llmp24l02 qemu]#
[root@llmp24l02 qemu]# modprobe kvm-pr
[root@llmp24l02 qemu]# bash ../qemu

if don't specify kvm_type machine property, it will take a default value 0,
which means fastest supported.

-aneesh

[RFC PATCH 01/11] kvm: powerpc: book3s hv: Fix vcore leak

[Aneesh Kumar K.V]

From: Paul Mackerras <paulus@samba.org>

add kvmppc_free_vcores() to free the kvmppc_vcore structures
that we allocate for a guest, which are currently being leaked.

Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/kvm/book3s_hv.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 62a2b5a..edc7f9f 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -1931,10 +1931,20 @@ int kvmppc_core_init_vm(struct kvm *kvm)
 	return 0;
 }
 
+static void kvmppc_free_vcores(struct kvm *kvm)
+{
+	long int i;
+
+	for (i = 0; i < KVM_MAX_VCORES; ++i)
+		kfree(kvm->arch.vcores[i]);
+	kvm->arch.online_vcores = 0;
+}
+
 void kvmppc_core_destroy_vm(struct kvm *kvm)
 {
 	uninhibit_secondary_onlining();
 
+	kvmppc_free_vcores(kvm);
 	if (kvm->arch.rma) {
 		kvm_release_rma(kvm->arch.rma);
 		kvm->arch.rma = NULL;
-- 
1.8.1.2

[RFC PATCH 02/11] kvm: powerpc: book3s: remove kvmppc_handler_highmem label

[Aneesh Kumar K.V]

From: Paul Mackerras <paulus@samba.org>

This label is not used now.

Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/kvm/book3s_hv_interrupts.S | 3 ---
 arch/powerpc/kvm/book3s_interrupts.S    | 3 ---
 2 files changed, 6 deletions(-)

diff --git a/arch/powerpc/kvm/book3s_hv_interrupts.S b/arch/powerpc/kvm/book3s_hv_interrupts.S
index 37f1cc4..928142c 100644
--- a/arch/powerpc/kvm/book3s_hv_interrupts.S
+++ b/arch/powerpc/kvm/book3s_hv_interrupts.S
@@ -158,9 +158,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_201)
  * Interrupts are enabled again at this point.
  */
 
-.global kvmppc_handler_highmem
-kvmppc_handler_highmem:
-
 	/*
 	 * Register usage at this point:
 	 *
diff --git a/arch/powerpc/kvm/book3s_interrupts.S b/arch/powerpc/kvm/book3s_interrupts.S
index bf8f1ab..279ee3f 100644
--- a/arch/powerpc/kvm/book3s_interrupts.S
+++ b/arch/powerpc/kvm/book3s_interrupts.S
@@ -121,9 +121,6 @@ kvm_start_lightweight:
  *
  */
 
-.global kvmppc_handler_highmem
-kvmppc_handler_highmem:
-
 	/*
 	 * Register usage at this point:
 	 *
-- 
1.8.1.2

[RFC PATCH 03/11] kvm: powerpc: book3s: move book3s_64_vio_hv.c into the main kernel binary

[Aneesh Kumar K.V]

From: Paul Mackerras <paulus@samba.org>

Since the code in book3s_64_vio_hv.c is called from real mode with HV
KVM, and therefore has to be built into the main kernel binary, this
makes it always built-in rather than part of the KVM module.  It gets
called from the KVM module by PR KVM, so this adds an EXPORT_SYMBOL_GPL().

Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/kvm/Makefile           | 12 ++++++++----
 arch/powerpc/kvm/book3s_64_vio_hv.c |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 6646c95..104e8dc 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -53,32 +53,36 @@ kvm-e500mc-objs := \
 	e500_emulate.o
 kvm-objs-$(CONFIG_KVM_E500MC) := $(kvm-e500mc-objs)
 
+kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) := \
+	book3s_64_vio_hv.o
+
 kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_PR) := \
 	$(KVM)/coalesced_mmio.o \
 	fpu.o \
 	book3s_paired_singles.o \
 	book3s_pr.o \
 	book3s_pr_papr.o \
-	book3s_64_vio_hv.o \
 	book3s_emulate.o \
 	book3s_interrupts.o \
 	book3s_mmu_hpte.o \
 	book3s_64_mmu_host.o \
 	book3s_64_mmu.o \
 	book3s_32_mmu.o
-kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_PR) := \
+
+kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_PR) += \
 	book3s_rmhandlers.o
 
 kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
 	book3s_hv.o \
 	book3s_hv_interrupts.o \
 	book3s_64_mmu_hv.o
+
 kvm-book3s_64-builtin-xics-objs-$(CONFIG_KVM_XICS) := \
 	book3s_hv_rm_xics.o
-kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
+
+kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HV) += \
 	book3s_hv_rmhandlers.o \
 	book3s_hv_rm_mmu.o \
-	book3s_64_vio_hv.o \
 	book3s_hv_ras.o \
 	book3s_hv_builtin.o \
 	book3s_hv_cma.o \
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 30c2f3b..70d6594 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -74,3 +74,5 @@ long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
 	/* Didn't find the liobn, punt it to userspace */
 	return H_TOO_HARD;
 }
+EXPORT_SYMBOL_GPL(kvmppc_h_put_tce);
+
-- 
1.8.1.2

[RFC PATCH 04/11] kvm: powerpc: book3s: Add a new config variable CONFIG_KVM_BOOK3S_HV

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

This help ups to select the relevant code in the kernel code
when we later move HV and PR bits as seperate modules.

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_book3s_64.h  |  6 +++---
 arch/powerpc/include/asm/kvm_book3s_asm.h |  2 +-
 arch/powerpc/include/asm/kvm_host.h       | 10 +++++-----
 arch/powerpc/include/asm/kvm_ppc.h        |  2 +-
 arch/powerpc/kernel/asm-offsets.c         |  8 ++++----
 arch/powerpc/kernel/exceptions-64s.S      |  2 +-
 arch/powerpc/kernel/idle_power7.S         |  2 +-
 arch/powerpc/kvm/Kconfig                  |  4 ++++
 arch/powerpc/kvm/book3s_exports.c         |  5 +++--
 arch/powerpc/kvm/book3s_xics.c            |  4 ++--
 10 files changed, 25 insertions(+), 20 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h
index 86d638a..37af4ae 100644
--- a/arch/powerpc/include/asm/kvm_book3s_64.h
+++ b/arch/powerpc/include/asm/kvm_book3s_64.h
@@ -35,7 +35,7 @@ static inline void svcpu_put(struct kvmppc_book3s_shadow_vcpu *svcpu)
 
 #define SPAPR_TCE_SHIFT		12
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 #define KVM_DEFAULT_HPT_ORDER	24	/* 16MB HPT by default */
 extern unsigned long kvm_rma_pages;
 #endif
@@ -278,7 +278,7 @@ static inline int is_vrma_hpte(unsigned long hpte_v)
 		(HPTE_V_1TB_SEG | (VRMA_VSID << (40 - 16)));
 }
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 /*
  * Note modification of an HPTE; set the HPTE modified bit
  * if anyone is interested.
@@ -289,6 +289,6 @@ static inline void note_hpte_modification(struct kvm *kvm,
 	if (atomic_read(&kvm->arch.hpte_mod_interest))
 		rev->guest_rpte |= HPTE_GR_MODIFIED;
 }
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
+#endif /* CONFIG_KVM_BOOK3S_HV */
 
 #endif /* __ASM_KVM_BOOK3S_64_H__ */
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 360742a..1272178 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -84,7 +84,7 @@ struct kvmppc_host_state {
 	u8 restore_hid5;
 	u8 napping;
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	u8 hwthread_req;
 	u8 hwthread_state;
 	u8 host_ipi;
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 78627c2..283e52e 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -229,15 +229,15 @@ struct revmap_entry {
 #define KVMPPC_GOT_PAGE		0x80
 
 struct kvm_arch_memory_slot {
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	unsigned long *rmap;
 	unsigned long *slot_phys;
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
+#endif /* CONFIG_KVM_BOOK3S_HV */
 };
 
 struct kvm_arch {
 	unsigned int lpid;
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	unsigned long hpt_virt;
 	struct revmap_entry *revmap;
 	unsigned int host_lpid;
@@ -261,7 +261,7 @@ struct kvm_arch {
 	cpumask_t need_tlb_flush;
 	struct kvmppc_vcore *vcores[KVM_MAX_VCORES];
 	int hpt_cma_alloc;
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
+#endif /* CONFIG_KVM_BOOK3S_HV */
 #ifdef CONFIG_KVM_BOOK3S_PR
 	struct mutex hpt_mutex;
 	bool relon_disabled;
@@ -598,7 +598,7 @@ struct kvm_vcpu_arch {
 	struct kvmppc_icp *icp; /* XICS presentation controller */
 #endif
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	struct kvm_vcpu_arch_shared shregs;
 
 	unsigned long pgfault_addr;
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index b15554a..9161bd1 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -260,7 +260,7 @@ void kvmppc_set_pid(struct kvm_vcpu *vcpu, u32 pid);
 
 struct openpic;
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 extern void kvm_cma_reserve(void) __init;
 static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr)
 {
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 67b5d56..dd7ad4a 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -445,7 +445,7 @@ int main(void)
 	DEFINE(VCPU_LR, offsetof(struct kvm_vcpu, arch.lr));
 	DEFINE(VCPU_CR, offsetof(struct kvm_vcpu, arch.cr));
 	DEFINE(VCPU_PC, offsetof(struct kvm_vcpu, arch.pc));
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	DEFINE(VCPU_MSR, offsetof(struct kvm_vcpu, arch.shregs.msr));
 	DEFINE(VCPU_SRR0, offsetof(struct kvm_vcpu, arch.shregs.srr0));
 	DEFINE(VCPU_SRR1, offsetof(struct kvm_vcpu, arch.shregs.srr1));
@@ -476,7 +476,7 @@ int main(void)
 	DEFINE(KVM_LPID, offsetof(struct kvm, arch.lpid));
 
 	/* book3s */
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	DEFINE(KVM_SDR1, offsetof(struct kvm, arch.sdr1));
 	DEFINE(KVM_HOST_LPID, offsetof(struct kvm, arch.host_lpid));
 	DEFINE(KVM_HOST_LPCR, offsetof(struct kvm, arch.host_lpcr));
@@ -581,7 +581,7 @@ int main(void)
 	HSTATE_FIELD(HSTATE_RESTORE_HID5, restore_hid5);
 	HSTATE_FIELD(HSTATE_NAPPING, napping);
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	HSTATE_FIELD(HSTATE_HWTHREAD_REQ, hwthread_req);
 	HSTATE_FIELD(HSTATE_HWTHREAD_STATE, hwthread_state);
 	HSTATE_FIELD(HSTATE_KVM_VCPU, kvm_vcpu);
@@ -597,7 +597,7 @@ int main(void)
 	HSTATE_FIELD(HSTATE_DABR, dabr);
 	HSTATE_FIELD(HSTATE_DECEXP, dec_expires);
 	DEFINE(IPI_PRIORITY, IPI_PRIORITY);
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
+#endif /* CONFIG_KVM_BOOK3S_HV */
 
 #ifdef CONFIG_PPC_BOOK3S_64
 	HSTATE_FIELD(HSTATE_CFAR, cfar);
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 2a273be..9f2f352 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -126,7 +126,7 @@ BEGIN_FTR_SECTION
 	bgt	cr1,.
 	GET_PACA(r13)
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	li	r0,KVM_HWTHREAD_IN_KERNEL
 	stb	r0,HSTATE_HWTHREAD_STATE(r13)
 	/* Order setting hwthread_state vs. testing hwthread_req */
diff --git a/arch/powerpc/kernel/idle_power7.S b/arch/powerpc/kernel/idle_power7.S
index e11863f..6aa4477 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -84,7 +84,7 @@ _GLOBAL(power7_nap)
 	std	r9,_MSR(r1)
 	std	r1,PACAR1(r13)
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	/* Tell KVM we're napping */
 	li	r4,KVM_HWTHREAD_IN_NAP
 	stb	r4,HSTATE_HWTHREAD_STATE(r13)
diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index ffaef2c..719c946 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -39,6 +39,9 @@ config KVM_BOOK3S_PR
 	select KVM_MMIO
 	select MMU_NOTIFIER
 
+config KVM_BOOK3S_HV
+	bool
+
 config KVM_BOOK3S_32
 	tristate "KVM support for PowerPC book3s_32 processors"
 	depends on PPC_BOOK3S_32 && !SMP && !PTE_64BIT
@@ -71,6 +74,7 @@ config KVM_BOOK3S_64
 config KVM_BOOK3S_64_HV
 	bool "KVM support for POWER7 and PPC970 using hypervisor mode in host"
 	depends on KVM_BOOK3S_64
+	select KVM_BOOK3S_HV
 	select MMU_NOTIFIER
 	select CMA
 	---help---
diff --git a/arch/powerpc/kvm/book3s_exports.c b/arch/powerpc/kvm/book3s_exports.c
index 7057a02..99504db 100644
--- a/arch/powerpc/kvm/book3s_exports.c
+++ b/arch/powerpc/kvm/book3s_exports.c
@@ -20,9 +20,10 @@
 #include <linux/export.h>
 #include <asm/kvm_book3s.h>
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 EXPORT_SYMBOL_GPL(kvmppc_hv_entry_trampoline);
-#else
+#endif
+#ifdef CONFIG_KVM_BOOK3S_PR
 EXPORT_SYMBOL_GPL(kvmppc_entry_trampoline);
 EXPORT_SYMBOL_GPL(kvmppc_load_up_fpu);
 #ifdef CONFIG_ALTIVEC
diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c
index a3a5cb8..f0c732e 100644
--- a/arch/powerpc/kvm/book3s_xics.c
+++ b/arch/powerpc/kvm/book3s_xics.c
@@ -1250,13 +1250,13 @@ static int kvmppc_xics_create(struct kvm_device *dev, u32 type)
 
 	xics_debugfs_init(xics);
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	if (cpu_has_feature(CPU_FTR_ARCH_206)) {
 		/* Enable real mode support */
 		xics->real_mode = ENABLE_REALMODE;
 		xics->real_mode_dbg = DEBUG_REALMODE;
 	}
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
+#endif /* CONFIG_KVM_BOOK3S_HV */
 
 	return 0;
 }
-- 
1.8.1.2

[RFC PATCH 05/11] kvm: powerpc: book3s: Add kvmppc_ops callback for HV and PR specific operations

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

This moves HV and PR specific functions to kvmppc_ops callback.
This is needed so that we can enable HV and PR together in the
same kernel. Actual changes to enable both come in the later
patch.This also renames almost all of the symbols that exist in both PR and HV
KVM for clarity. Symbols in the PR KVM implementation get "_pr"
appended, and those in the HV KVM implementation get "_hv".  Then,
in book3s.c, we add a function with the name without the suffix and
arrange for it to call the appropriate kvmppc_ops callback depending on
which kvm type we selected during VM creation.

NOTE: we still don't enable selecting both the HV and PR together
in this commit that will be done by a later commit.

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_book3s.h |   5 +-
 arch/powerpc/include/asm/kvm_ppc.h    |  63 ++++++++--
 arch/powerpc/kvm/Kconfig              |  15 ++-
 arch/powerpc/kvm/Makefile             |   9 +-
 arch/powerpc/kvm/book3s.c             | 145 +++++++++++++++++++++-
 arch/powerpc/kvm/book3s_32_mmu_host.c |   2 +-
 arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
 arch/powerpc/kvm/book3s_64_mmu_hv.c   |  17 ++-
 arch/powerpc/kvm/book3s_emulate.c     |   8 +-
 arch/powerpc/kvm/book3s_hv.c          | 226 +++++++++++++++++++++++++---------
 arch/powerpc/kvm/book3s_interrupts.S  |   2 +-
 arch/powerpc/kvm/book3s_pr.c          | 196 ++++++++++++++++++-----------
 arch/powerpc/kvm/emulate.c            |   6 +-
 arch/powerpc/kvm/powerpc.c            |  58 +++------
 14 files changed, 539 insertions(+), 215 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h
index 1b32f6c..3efba3c 100644
--- a/arch/powerpc/include/asm/kvm_book3s.h
+++ b/arch/powerpc/include/asm/kvm_book3s.h
@@ -24,6 +24,8 @@
 #include <linux/kvm_host.h>
 #include <asm/kvm_book3s_asm.h>
 
+union kvmppc_one_reg;
+
 struct kvmppc_bat {
 	u64 raw;
 	u32 bepi;
@@ -124,7 +126,6 @@ extern void kvmppc_mmu_pte_flush(struct kvm_vcpu *vcpu, ulong ea, ulong ea_mask)
 extern void kvmppc_mmu_pte_vflush(struct kvm_vcpu *vcpu, u64 vp, u64 vp_mask);
 extern void kvmppc_mmu_pte_pflush(struct kvm_vcpu *vcpu, ulong pa_start, ulong pa_end);
 extern void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 new_msr);
-extern void kvmppc_set_pvr(struct kvm_vcpu *vcpu, u32 pvr);
 extern void kvmppc_mmu_book3s_64_init(struct kvm_vcpu *vcpu);
 extern void kvmppc_mmu_book3s_32_init(struct kvm_vcpu *vcpu);
 extern void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu);
@@ -193,8 +194,6 @@ static inline struct kvmppc_vcpu_book3s *to_book3s(struct kvm_vcpu *vcpu)
 	return vcpu->arch.book3s;
 }
 
-extern void kvm_return_point(void);
-
 /* Also add subarch specific defines */
 
 #ifdef CONFIG_KVM_BOOK3S_32_HANDLER
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 9161bd1..4d9641c 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -106,13 +106,6 @@ extern void kvmppc_core_queue_external(struct kvm_vcpu *vcpu,
                                        struct kvm_interrupt *irq);
 extern void kvmppc_core_dequeue_external(struct kvm_vcpu *vcpu);
 extern void kvmppc_core_flush_tlb(struct kvm_vcpu *vcpu);
-
-extern int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
-                                  unsigned int op, int *advance);
-extern int kvmppc_core_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn,
-				     ulong val);
-extern int kvmppc_core_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn,
-				     ulong *val);
 extern int kvmppc_core_check_requests(struct kvm_vcpu *vcpu);
 
 extern int kvmppc_booke_init(void);
@@ -135,8 +128,6 @@ extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
 				struct kvm_create_spapr_tce *args);
 extern long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
 			     unsigned long ioba, unsigned long tce);
-extern long kvm_vm_ioctl_allocate_rma(struct kvm *kvm,
-				struct kvm_allocate_rma *rma);
 extern struct kvm_rma_info *kvm_alloc_rma(void);
 extern void kvm_release_rma(struct kvm_rma_info *ri);
 extern struct page *kvm_alloc_hpt(unsigned long nr_pages);
@@ -177,6 +168,55 @@ extern int kvmppc_xics_get_xive(struct kvm *kvm, u32 irq, u32 *server,
 extern int kvmppc_xics_int_on(struct kvm *kvm, u32 irq);
 extern int kvmppc_xics_int_off(struct kvm *kvm, u32 irq);
 
+struct kvmppc_ops {
+	int (*get_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
+	int (*set_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
+	int (*get_one_reg)(struct kvm_vcpu *vcpu, u64 id,
+			   union kvmppc_one_reg *val);
+	int (*set_one_reg)(struct kvm_vcpu *vcpu, u64 id,
+			   union kvmppc_one_reg *val);
+	void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
+	void (*vcpu_put)(struct kvm_vcpu *vcpu);
+	void (*set_msr)(struct kvm_vcpu *vcpu, u64 msr);
+	int (*vcpu_run)(struct kvm_run *run, struct kvm_vcpu *vcpu);
+	struct kvm_vcpu *(*vcpu_create)(struct kvm *kvm, unsigned int id);
+	void (*vcpu_free)(struct kvm_vcpu *vcpu);
+	int (*check_requests)(struct kvm_vcpu *vcpu);
+	int (*get_dirty_log)(struct kvm *kvm, struct kvm_dirty_log *log);
+	void (*flush_memslot)(struct kvm *kvm, struct kvm_memory_slot *memslot);
+	int (*prepare_memory_region)(struct kvm *kvm,
+				     struct kvm_memory_slot *memslot,
+				     struct kvm_userspace_memory_region *mem);
+	void (*commit_memory_region)(struct kvm *kvm,
+				     struct kvm_userspace_memory_region *mem,
+				     const struct kvm_memory_slot *old);
+	int (*unmap_hva)(struct kvm *kvm, unsigned long hva);
+	int (*unmap_hva_range)(struct kvm *kvm, unsigned long start,
+			   unsigned long end);
+	int (*age_hva)(struct kvm *kvm, unsigned long hva);
+	int (*test_age_hva)(struct kvm *kvm, unsigned long hva);
+	void (*set_spte_hva)(struct kvm *kvm, unsigned long hva, pte_t pte);
+	void (*mmu_destroy)(struct kvm_vcpu *vcpu);
+	void (*free_memslot)(struct kvm_memory_slot *free,
+			     struct kvm_memory_slot *dont);
+	int (*create_memslot)(struct kvm_memory_slot *slot,
+			      unsigned long npages);
+	int (*init_vm)(struct kvm *kvm);
+	void (*destroy_vm)(struct kvm *kvm);
+	int (*check_processor_compat)(void);
+	int (*get_smmu_info)(struct kvm *kvm, struct kvm_ppc_smmu_info *info);
+	int (*emulate_op)(struct kvm_run *run, struct kvm_vcpu *vcpu,
+			  unsigned int inst, int *advance);
+	int (*emulate_mtspr)(struct kvm_vcpu *vcpu, int sprn, ulong spr_val);
+	int (*emulate_mfspr)(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val);
+	void (*fast_vcpu_kick)(struct kvm_vcpu *vcpu);
+	long (*arch_vm_ioctl)(struct file *filp, unsigned int ioctl,
+			      unsigned long arg);
+
+};
+
+extern struct kvmppc_ops *kvmppc_ops;
+
 /*
  * Cuts out inst bits with ordering according to spec.
  * That means the leftmost bit is zero. All given bits are included.
@@ -281,7 +321,10 @@ static inline void kvmppc_set_host_ipi(int cpu, u8 host_ipi)
 	paca[cpu].kvm_hstate.host_ipi = host_ipi;
 }
 
-extern void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu);
+static inline void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu)
+{
+	kvmppc_ops->fast_vcpu_kick(vcpu);
+}
 
 #else
 static inline void __init kvm_cma_reserve(void)
diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 719c946..5c62459 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -62,6 +62,7 @@ config KVM_BOOK3S_64
 	depends on PPC_BOOK3S_64
 	select KVM_BOOK3S_64_HANDLER
 	select KVM
+	select KVM_BOOK3S_PR if !KVM_BOOK3S_HV
 	---help---
 	  Support running unmodified book3s_64 and book3s_32 guest kernels
 	  in virtual machines on book3s_64 host processors.
@@ -93,9 +94,21 @@ config KVM_BOOK3S_64_HV
 	  If unsure, say N.
 
 config KVM_BOOK3S_64_PR
-	def_bool y
+	bool "KVM support without using hypervisor mode in host"
 	depends on KVM_BOOK3S_64 && !KVM_BOOK3S_64_HV
 	select KVM_BOOK3S_PR
+	---help---
+	  Support running guest kernels in virtual machines on processors
+	  without using hypervisor mode in the host, by running the
+	  guest in user mode (problem state) and emulating all
+	  privileged instructions and registers.
+
+	  This is not as fast as using hypervisor mode, but works on
+	  machines where hypervisor mode is not available or not usable,
+	  and can emulate processors that are different from the host
+	  processor, including emulating 32-bit processors on a 64-bit
+	  host.
+
 
 config KVM_BOOKE_HV
 	bool
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 104e8dc..c343793 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -69,8 +69,13 @@ kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_PR) := \
 	book3s_64_mmu.o \
 	book3s_32_mmu.o
 
-kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_PR) += \
+ifdef CONFIG_KVM_BOOK3S_PR
+kvm-book3s_64-module-objs := \
+	$(KVM)/coalesced_mmio.o
+
+kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
 	book3s_rmhandlers.o
+endif
 
 kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
 	book3s_hv.o \
@@ -91,7 +96,7 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HV) += \
 kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
 	book3s_xics.o
 
-kvm-book3s_64-module-objs := \
+kvm-book3s_64-module-objs += \
 	$(KVM)/kvm_main.o \
 	$(KVM)/eventfd.o \
 	powerpc.o \
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index 700df6f..bdc3f95 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -419,6 +419,18 @@ void kvmppc_subarch_vcpu_uninit(struct kvm_vcpu *vcpu)
 {
 }
 
+int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
+				  struct kvm_sregs *sregs)
+{
+	return kvmppc_ops->get_sregs(vcpu, sregs);
+}
+
+int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
+				  struct kvm_sregs *sregs)
+{
+	return kvmppc_ops->set_sregs(vcpu, sregs);
+}
+
 int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 {
 	int i;
@@ -495,8 +507,7 @@ int kvm_vcpu_ioctl_get_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 	if (size > sizeof(val))
 		return -EINVAL;
 
-	r = kvmppc_get_one_reg(vcpu, reg->id, &val);
-
+	r = kvmppc_ops->get_one_reg(vcpu, reg->id, &val);
 	if (r == -EINVAL) {
 		r = 0;
 		switch (reg->id) {
@@ -572,8 +583,7 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 	if (copy_from_user(&val, (char __user *)(unsigned long)reg->addr, size))
 		return -EFAULT;
 
-	r = kvmppc_set_one_reg(vcpu, reg->id, &val);
-
+	r = kvmppc_ops->set_one_reg(vcpu, reg->id, &val);
 	if (r == -EINVAL) {
 		r = 0;
 		switch (reg->id) {
@@ -625,6 +635,26 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 	return r;
 }
 
+void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	kvmppc_ops->vcpu_load(vcpu, cpu);
+}
+
+void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	kvmppc_ops->vcpu_put(vcpu);
+}
+
+void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)
+{
+	kvmppc_ops->set_msr(vcpu, msr);
+}
+
+int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
+{
+	return kvmppc_ops->vcpu_run(kvm_run, vcpu);
+}
+
 int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu,
                                   struct kvm_translation *tr)
 {
@@ -644,3 +674,110 @@ void kvmppc_decrementer_func(unsigned long data)
 	kvmppc_core_queue_dec(vcpu);
 	kvm_vcpu_kick(vcpu);
 }
+
+struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
+{
+	return kvmppc_ops->vcpu_create(kvm, id);
+}
+
+void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	kvmppc_ops->vcpu_free(vcpu);
+}
+
+int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
+{
+	return kvmppc_ops->check_requests(vcpu);
+}
+
+int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
+{
+	return kvmppc_ops->get_dirty_log(kvm, log);
+}
+
+void kvmppc_core_free_memslot(struct kvm_memory_slot *free,
+			      struct kvm_memory_slot *dont)
+{
+	kvmppc_ops->free_memslot(free, dont);
+}
+
+int kvmppc_core_create_memslot(struct kvm_memory_slot *slot,
+			       unsigned long npages)
+{
+	return kvmppc_ops->create_memslot(slot, npages);
+}
+
+void kvmppc_core_flush_memslot(struct kvm *kvm, struct kvm_memory_slot *memslot)
+{
+	kvmppc_ops->flush_memslot(kvm, memslot);
+}
+
+int kvmppc_core_prepare_memory_region(struct kvm *kvm,
+				struct kvm_memory_slot *memslot,
+				struct kvm_userspace_memory_region *mem)
+{
+	return kvmppc_ops->prepare_memory_region(kvm, memslot, mem);
+}
+
+void kvmppc_core_commit_memory_region(struct kvm *kvm,
+				struct kvm_userspace_memory_region *mem,
+				const struct kvm_memory_slot *old)
+{
+	kvmppc_ops->commit_memory_region(kvm, mem, old);
+}
+
+int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
+{
+	return kvmppc_ops->unmap_hva(kvm, hva);
+}
+
+int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)
+{
+	return kvmppc_ops->unmap_hva_range(kvm, start, end);
+}
+
+int kvm_age_hva(struct kvm *kvm, unsigned long hva)
+{
+	return kvmppc_ops->age_hva(kvm, hva);
+}
+
+int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
+{
+	return kvmppc_ops->test_age_hva(kvm, hva);
+}
+
+void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte)
+{
+	kvmppc_ops->set_spte_hva(kvm, hva, pte);
+}
+
+void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
+{
+	kvmppc_ops->mmu_destroy(vcpu);
+}
+
+int kvmppc_core_init_vm(struct kvm *kvm)
+{
+
+#ifdef CONFIG_PPC64
+	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
+	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
+#endif
+
+	return kvmppc_ops->init_vm(kvm);
+}
+
+void kvmppc_core_destroy_vm(struct kvm *kvm)
+{
+	kvmppc_ops->destroy_vm(kvm);
+
+#ifdef CONFIG_PPC64
+	kvmppc_rtas_tokens_free(kvm);
+	WARN_ON(!list_empty(&kvm->arch.spapr_tce_tables));
+#endif
+}
+
+int kvmppc_core_check_processor_compat(void)
+{
+	return kvmppc_ops->check_processor_compat();
+}
diff --git a/arch/powerpc/kvm/book3s_32_mmu_host.c b/arch/powerpc/kvm/book3s_32_mmu_host.c
index 00e619b..c4361ef 100644
--- a/arch/powerpc/kvm/book3s_32_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_32_mmu_host.c
@@ -341,7 +341,7 @@ void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu)
 	svcpu_put(svcpu);
 }
 
-void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
+void kvmppc_mmu_destroy_pr(struct kvm_vcpu *vcpu)
 {
 	int i;
 
diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index 6bda504..329a978 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -341,7 +341,7 @@ void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu)
 	svcpu_put(svcpu);
 }
 
-void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
+void kvmppc_mmu_destroy_pr(struct kvm_vcpu *vcpu)
 {
 	kvmppc_mmu_hpte_destroy(vcpu);
 	__destroy_context(to_book3s(vcpu)->context_id[0]);
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 043eec8..6f0485f 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -260,10 +260,6 @@ int kvmppc_mmu_hv_init(void)
 	return 0;
 }
 
-void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
-{
-}
-
 static void kvmppc_mmu_book3s_64_hv_reset_msr(struct kvm_vcpu *vcpu)
 {
 	kvmppc_set_msr(vcpu, MSR_SF | MSR_ME);
@@ -906,21 +902,22 @@ static int kvm_unmap_rmapp(struct kvm *kvm, unsigned long *rmapp,
 	return 0;
 }
 
-int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
+int kvm_unmap_hva_hv(struct kvm *kvm, unsigned long hva)
 {
 	if (kvm->arch.using_mmu_notifiers)
 		kvm_handle_hva(kvm, hva, kvm_unmap_rmapp);
 	return 0;
 }
 
-int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)
+int kvm_unmap_hva_range_hv(struct kvm *kvm, unsigned long start, unsigned long end)
 {
 	if (kvm->arch.using_mmu_notifiers)
 		kvm_handle_hva_range(kvm, start, end, kvm_unmap_rmapp);
 	return 0;
 }
 
-void kvmppc_core_flush_memslot(struct kvm *kvm, struct kvm_memory_slot *memslot)
+void kvmppc_core_flush_memslot_hv(struct kvm *kvm,
+				  struct kvm_memory_slot *memslot)
 {
 	unsigned long *rmapp;
 	unsigned long gfn;
@@ -994,7 +991,7 @@ static int kvm_age_rmapp(struct kvm *kvm, unsigned long *rmapp,
 	return ret;
 }
 
-int kvm_age_hva(struct kvm *kvm, unsigned long hva)
+int kvm_age_hva_hv(struct kvm *kvm, unsigned long hva)
 {
 	if (!kvm->arch.using_mmu_notifiers)
 		return 0;
@@ -1032,14 +1029,14 @@ static int kvm_test_age_rmapp(struct kvm *kvm, unsigned long *rmapp,
 	return ret;
 }
 
-int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
+int kvm_test_age_hva_hv(struct kvm *kvm, unsigned long hva)
 {
 	if (!kvm->arch.using_mmu_notifiers)
 		return 0;
 	return kvm_handle_hva(kvm, hva, kvm_test_age_rmapp);
 }
 
-void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte)
+void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, pte_t pte)
 {
 	if (!kvm->arch.using_mmu_notifiers)
 		return;
diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c
index 34044b1..b9841ad 100644
--- a/arch/powerpc/kvm/book3s_emulate.c
+++ b/arch/powerpc/kvm/book3s_emulate.c
@@ -86,8 +86,8 @@ static bool spr_allowed(struct kvm_vcpu *vcpu, enum priv_level level)
 	return true;
 }
 
-int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
-                           unsigned int inst, int *advance)
+int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
+			      unsigned int inst, int *advance)
 {
 	int emulated = EMULATE_DONE;
 	int rt = get_rt(inst);
@@ -345,7 +345,7 @@ static struct kvmppc_bat *kvmppc_find_bat(struct kvm_vcpu *vcpu, int sprn)
 	return bat;
 }
 
-int kvmppc_core_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)
+int kvmppc_core_emulate_mtspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)
 {
 	int emulated = EMULATE_DONE;
 
@@ -468,7 +468,7 @@ unprivileged:
 	return emulated;
 }
 
-int kvmppc_core_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val)
+int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val)
 {
 	int emulated = EMULATE_DONE;
 
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index edc7f9f..36d7920 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -66,7 +66,7 @@
 static void kvmppc_end_cede(struct kvm_vcpu *vcpu);
 static int kvmppc_hv_setup_htab_rma(struct kvm_vcpu *vcpu);
 
-void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu)
+static void kvmppc_fast_vcpu_kick_hv(struct kvm_vcpu *vcpu)
 {
 	int me;
 	int cpu = vcpu->cpu;
@@ -125,7 +125,7 @@ void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu)
  * purely defensive; they should never fail.)
  */
 
-void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+static void kvmppc_core_vcpu_load_hv(struct kvm_vcpu *vcpu, int cpu)
 {
 	struct kvmppc_vcore *vc = vcpu->arch.vcore;
 
@@ -143,7 +143,7 @@ void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	spin_unlock(&vcpu->arch.tbacct_lock);
 }
 
-void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
+static void kvmppc_core_vcpu_put_hv(struct kvm_vcpu *vcpu)
 {
 	struct kvmppc_vcore *vc = vcpu->arch.vcore;
 
@@ -155,13 +155,13 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
 	spin_unlock(&vcpu->arch.tbacct_lock);
 }
 
-void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)
+static void kvmppc_set_msr_hv(struct kvm_vcpu *vcpu, u64 msr)
 {
 	vcpu->arch.shregs.msr = msr;
 	kvmppc_end_cede(vcpu);
 }
 
-void kvmppc_set_pvr(struct kvm_vcpu *vcpu, u32 pvr)
+void kvmppc_set_pvr_hv(struct kvm_vcpu *vcpu, u32 pvr)
 {
 	vcpu->arch.pvr = pvr;
 }
@@ -576,8 +576,8 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)
 	return RESUME_GUEST;
 }
 
-static int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
-			      struct task_struct *tsk)
+static int kvmppc_handle_exit_hv(struct kvm_run *run, struct kvm_vcpu *vcpu,
+				 struct task_struct *tsk)
 {
 	int r = RESUME_HOST;
 
@@ -679,8 +679,8 @@ static int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
 	return r;
 }
 
-int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
-				  struct kvm_sregs *sregs)
+static int kvm_arch_vcpu_ioctl_get_sregs_hv(struct kvm_vcpu *vcpu,
+					    struct kvm_sregs *sregs)
 {
 	int i;
 
@@ -694,12 +694,12 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
-				  struct kvm_sregs *sregs)
+static int kvm_arch_vcpu_ioctl_set_sregs_hv(struct kvm_vcpu *vcpu,
+					    struct kvm_sregs *sregs)
 {
 	int i, j;
 
-	kvmppc_set_pvr(vcpu, sregs->pvr);
+	kvmppc_set_pvr_hv(vcpu, sregs->pvr);
 
 	j = 0;
 	for (i = 0; i < vcpu->arch.slb_nr; i++) {
@@ -714,7 +714,8 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-int kvmppc_get_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
+static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
+				 union kvmppc_one_reg *val)
 {
 	int r = 0;
 	long int i;
@@ -795,7 +796,8 @@ int kvmppc_get_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	return r;
 }
 
-int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
+static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
+				 union kvmppc_one_reg *val)
 {
 	int r = 0;
 	long int i;
@@ -888,14 +890,8 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	return r;
 }
 
-int kvmppc_core_check_processor_compat(void)
-{
-	if (cpu_has_feature(CPU_FTR_HVMODE))
-		return 0;
-	return -EIO;
-}
-
-struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
+static struct kvm_vcpu *kvmppc_core_vcpu_create_hv(struct kvm *kvm,
+						   unsigned int id)
 {
 	struct kvm_vcpu *vcpu;
 	int err = -EINVAL;
@@ -920,7 +916,6 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
 	vcpu->arch.ctrl = CTRL_RUNLATCH;
 	/* default to host PVR, since we can't spoof it */
 	vcpu->arch.pvr = mfspr(SPRN_PVR);
-	kvmppc_set_pvr(vcpu, vcpu->arch.pvr);
 	spin_lock_init(&vcpu->arch.vpa_update_lock);
 	spin_lock_init(&vcpu->arch.tbacct_lock);
 	vcpu->arch.busy_preempt = TB_NIL;
@@ -972,7 +967,7 @@ static void unpin_vpa(struct kvm *kvm, struct kvmppc_vpa *vpa)
 					vpa->dirty);
 }
 
-void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
+static void kvmppc_core_vcpu_free_hv(struct kvm_vcpu *vcpu)
 {
 	spin_lock(&vcpu->arch.vpa_update_lock);
 	unpin_vpa(vcpu->kvm, &vcpu->arch.dtl);
@@ -983,6 +978,12 @@ void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
 	kmem_cache_free(kvm_vcpu_cache, vcpu);
 }
 
+static int kvmppc_core_check_requests_hv(struct kvm_vcpu *vcpu)
+{
+	/* Indicate we want to get back into the guest */
+	return 1;
+}
+
 static void kvmppc_set_timer(struct kvm_vcpu *vcpu)
 {
 	unsigned long dec_nsec, now;
@@ -1264,8 +1265,8 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc)
 
 		ret = RESUME_GUEST;
 		if (vcpu->arch.trap)
-			ret = kvmppc_handle_exit(vcpu->arch.kvm_run, vcpu,
-						 vcpu->arch.run_task);
+			ret = kvmppc_handle_exit_hv(vcpu->arch.kvm_run, vcpu,
+						    vcpu->arch.run_task);
 
 		vcpu->arch.ret = ret;
 		vcpu->arch.trap = 0;
@@ -1424,7 +1425,7 @@ static int kvmppc_run_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 	return vcpu->arch.ret;
 }
 
-int kvmppc_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu)
+static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu)
 {
 	int r;
 	int srcu_idx;
@@ -1546,7 +1547,8 @@ static const struct file_operations kvm_rma_fops = {
 	.release	= kvm_rma_release,
 };
 
-long kvm_vm_ioctl_allocate_rma(struct kvm *kvm, struct kvm_allocate_rma *ret)
+static long kvm_vm_ioctl_allocate_rma(struct kvm *kvm,
+				      struct kvm_allocate_rma *ret)
 {
 	long fd;
 	struct kvm_rma_info *ri;
@@ -1592,7 +1594,8 @@ static void kvmppc_add_seg_page_size(struct kvm_ppc_one_seg_page_size **sps,
 	(*sps)++;
 }
 
-int kvm_vm_ioctl_get_smmu_info(struct kvm *kvm, struct kvm_ppc_smmu_info *info)
+static int kvm_vm_ioctl_get_smmu_info_hv(struct kvm *kvm,
+					 struct kvm_ppc_smmu_info *info)
 {
 	struct kvm_ppc_one_seg_page_size *sps;
 
@@ -1613,7 +1616,8 @@ int kvm_vm_ioctl_get_smmu_info(struct kvm *kvm, struct kvm_ppc_smmu_info *info)
 /*
  * Get (and clear) the dirty memory log for a memory slot.
  */
-int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
+static int kvm_vm_ioctl_get_dirty_log_hv(struct kvm *kvm,
+					 struct kvm_dirty_log *log)
 {
 	struct kvm_memory_slot *memslot;
 	int r;
@@ -1667,8 +1671,8 @@ static void unpin_slot(struct kvm_memory_slot *memslot)
 	}
 }
 
-void kvmppc_core_free_memslot(struct kvm_memory_slot *free,
-			      struct kvm_memory_slot *dont)
+static void kvmppc_core_free_memslot_hv(struct kvm_memory_slot *free,
+					struct kvm_memory_slot *dont)
 {
 	if (!dont || free->arch.rmap != dont->arch.rmap) {
 		vfree(free->arch.rmap);
@@ -1681,8 +1685,8 @@ void kvmppc_core_free_memslot(struct kvm_memory_slot *free,
 	}
 }
 
-int kvmppc_core_create_memslot(struct kvm_memory_slot *slot,
-			       unsigned long npages)
+static int kvmppc_core_create_memslot_hv(struct kvm_memory_slot *slot,
+					 unsigned long npages)
 {
 	slot->arch.rmap = vzalloc(npages * sizeof(*slot->arch.rmap));
 	if (!slot->arch.rmap)
@@ -1692,9 +1696,9 @@ int kvmppc_core_create_memslot(struct kvm_memory_slot *slot,
 	return 0;
 }
 
-int kvmppc_core_prepare_memory_region(struct kvm *kvm,
-				      struct kvm_memory_slot *memslot,
-				      struct kvm_userspace_memory_region *mem)
+static int kvmppc_core_prepare_memory_region_hv(struct kvm *kvm,
+					struct kvm_memory_slot *memslot,
+					struct kvm_userspace_memory_region *mem)
 {
 	unsigned long *phys;
 
@@ -1710,9 +1714,9 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm,
 	return 0;
 }
 
-void kvmppc_core_commit_memory_region(struct kvm *kvm,
-				      struct kvm_userspace_memory_region *mem,
-				      const struct kvm_memory_slot *old)
+static void kvmppc_core_commit_memory_region_hv(struct kvm *kvm,
+				struct kvm_userspace_memory_region *mem,
+				const struct kvm_memory_slot *old)
 {
 	unsigned long npages = mem->memory_size >> PAGE_SHIFT;
 	struct kvm_memory_slot *memslot;
@@ -1729,6 +1733,11 @@ void kvmppc_core_commit_memory_region(struct kvm *kvm,
 	}
 }
 
+static void kvmppc_mmu_destroy_hv(struct kvm_vcpu *vcpu)
+{
+	return;
+}
+
 static int kvmppc_hv_setup_htab_rma(struct kvm_vcpu *vcpu)
 {
 	int err = 0;
@@ -1875,7 +1884,7 @@ static int kvmppc_hv_setup_htab_rma(struct kvm_vcpu *vcpu)
 	goto out_srcu;
 }
 
-int kvmppc_core_init_vm(struct kvm *kvm)
+static int kvmppc_core_init_vm_hv(struct kvm *kvm)
 {
 	unsigned long lpcr, lpid;
 
@@ -1893,9 +1902,6 @@ int kvmppc_core_init_vm(struct kvm *kvm)
 	 */
 	cpumask_setall(&kvm->arch.need_tlb_flush);
 
-	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
-	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
-
 	kvm->arch.rma = NULL;
 
 	kvm->arch.host_sdr1 = mfspr(SPRN_SDR1);
@@ -1940,7 +1946,7 @@ static void kvmppc_free_vcores(struct kvm *kvm)
 	kvm->arch.online_vcores = 0;
 }
 
-void kvmppc_core_destroy_vm(struct kvm *kvm)
+static void kvmppc_core_destroy_vm_hv(struct kvm *kvm)
 {
 	uninhibit_secondary_onlining();
 
@@ -1950,39 +1956,137 @@ void kvmppc_core_destroy_vm(struct kvm *kvm)
 		kvm->arch.rma = NULL;
 	}
 
-	kvmppc_rtas_tokens_free(kvm);
-
 	kvmppc_free_hpt(kvm);
-	WARN_ON(!list_empty(&kvm->arch.spapr_tce_tables));
 }
 
-/* These are stubs for now */
-void kvmppc_mmu_pte_pflush(struct kvm_vcpu *vcpu, ulong pa_start, ulong pa_end)
+/* We don't need to emulate any privileged instructions or dcbz */
+static int kvmppc_core_emulate_op_hv(struct kvm_run *run, struct kvm_vcpu *vcpu,
+				     unsigned int inst, int *advance)
 {
+	return EMULATE_FAIL;
 }
 
-/* We don't need to emulate any privileged instructions or dcbz */
-int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
-                           unsigned int inst, int *advance)
+static int kvmppc_core_emulate_mtspr_hv(struct kvm_vcpu *vcpu, int sprn,
+					ulong spr_val)
 {
 	return EMULATE_FAIL;
 }
 
-int kvmppc_core_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)
+static int kvmppc_core_emulate_mfspr_hv(struct kvm_vcpu *vcpu, int sprn,
+					ulong *spr_val)
 {
 	return EMULATE_FAIL;
 }
 
-int kvmppc_core_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val)
+static int kvmppc_core_check_processor_compat_hv(void)
 {
-	return EMULATE_FAIL;
+	if (!cpu_has_feature(CPU_FTR_HVMODE))
+		return -EIO;
+	return 0;
+}
+
+static long kvm_arch_vm_ioctl_hv(struct file *filp,
+				 unsigned int ioctl, unsigned long arg)
+{
+	struct kvm *kvm __maybe_unused = filp->private_data;
+	void __user *argp = (void __user *)arg;
+	long r;
+
+	switch (ioctl) {
+
+	case KVM_ALLOCATE_RMA: {
+		struct kvm_allocate_rma rma;
+		struct kvm *kvm = filp->private_data;
+
+		r = kvm_vm_ioctl_allocate_rma(kvm, &rma);
+		if (r >= 0 && copy_to_user(argp, &rma, sizeof(rma)))
+			r = -EFAULT;
+		break;
+	}
+
+	case KVM_PPC_ALLOCATE_HTAB: {
+		u32 htab_order;
+
+		r = -EFAULT;
+		if (get_user(htab_order, (u32 __user *)argp))
+			break;
+		r = kvmppc_alloc_reset_hpt(kvm, &htab_order);
+		if (r)
+			break;
+		r = -EFAULT;
+		if (put_user(htab_order, (u32 __user *)argp))
+			break;
+		r = 0;
+		break;
+	}
+
+	case KVM_PPC_GET_HTAB_FD: {
+		struct kvm_get_htab_fd ghf;
+
+		r = -EFAULT;
+		if (copy_from_user(&ghf, argp, sizeof(ghf)))
+			break;
+		r = kvm_vm_ioctl_get_htab_fd(kvm, &ghf);
+		break;
+	}
+
+	default:
+		r = -ENOTTY;
+	}
+
+	return r;
 }
 
-static int kvmppc_book3s_hv_init(void)
+/* FIXME!! move to header */
+extern void kvmppc_core_flush_memslot_hv(struct kvm *kvm,
+					 struct kvm_memory_slot *memslot);
+extern int kvm_unmap_hva_hv(struct kvm *kvm, unsigned long hva);
+extern int kvm_unmap_hva_range_hv(struct kvm *kvm, unsigned long start,
+				  unsigned long end);
+extern int kvm_age_hva_hv(struct kvm *kvm, unsigned long hva);
+extern int kvm_test_age_hva_hv(struct kvm *kvm, unsigned long hva);
+extern void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, pte_t pte);
+
+static struct kvmppc_ops kvmppc_hv_ops = {
+	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv,
+	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
+	.get_one_reg = kvmppc_get_one_reg_hv,
+	.set_one_reg = kvmppc_set_one_reg_hv,
+	.vcpu_load   = kvmppc_core_vcpu_load_hv,
+	.vcpu_put    = kvmppc_core_vcpu_put_hv,
+	.set_msr     = kvmppc_set_msr_hv,
+	.vcpu_run    = kvmppc_vcpu_run_hv,
+	.vcpu_create = kvmppc_core_vcpu_create_hv,
+	.vcpu_free   = kvmppc_core_vcpu_free_hv,
+	.check_requests = kvmppc_core_check_requests_hv,
+	.get_dirty_log  = kvm_vm_ioctl_get_dirty_log_hv,
+	.flush_memslot  = kvmppc_core_flush_memslot_hv,
+	.prepare_memory_region = kvmppc_core_prepare_memory_region_hv,
+	.commit_memory_region  = kvmppc_core_commit_memory_region_hv,
+	.unmap_hva = kvm_unmap_hva_hv,
+	.unmap_hva_range = kvm_unmap_hva_range_hv,
+	.age_hva  = kvm_age_hva_hv,
+	.test_age_hva = kvm_test_age_hva_hv,
+	.set_spte_hva = kvm_set_spte_hva_hv,
+	.mmu_destroy  = kvmppc_mmu_destroy_hv,
+	.free_memslot = kvmppc_core_free_memslot_hv,
+	.create_memslot = kvmppc_core_create_memslot_hv,
+	.init_vm =  kvmppc_core_init_vm_hv,
+	.destroy_vm = kvmppc_core_destroy_vm_hv,
+	.check_processor_compat = kvmppc_core_check_processor_compat_hv,
+	.get_smmu_info = kvm_vm_ioctl_get_smmu_info_hv,
+	.emulate_op = kvmppc_core_emulate_op_hv,
+	.emulate_mtspr = kvmppc_core_emulate_mtspr_hv,
+	.emulate_mfspr = kvmppc_core_emulate_mfspr_hv,
+	.fast_vcpu_kick = kvmppc_fast_vcpu_kick_hv,
+	.arch_vm_ioctl  = kvm_arch_vm_ioctl_hv,
+};
+
+static int kvmppc_book3s_init_hv(void)
 {
 	int r;
 
-	r = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+	r = kvm_init(&kvmppc_hv_ops, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
 
 	if (r)
 		return r;
@@ -1992,10 +2096,10 @@ static int kvmppc_book3s_hv_init(void)
 	return r;
 }
 
-static void kvmppc_book3s_hv_exit(void)
+static void kvmppc_book3s_exit_hv(void)
 {
 	kvm_exit();
 }
 
-module_init(kvmppc_book3s_hv_init);
-module_exit(kvmppc_book3s_hv_exit);
+module_init(kvmppc_book3s_init_hv);
+module_exit(kvmppc_book3s_exit_hv);
diff --git a/arch/powerpc/kvm/book3s_interrupts.S b/arch/powerpc/kvm/book3s_interrupts.S
index 279ee3f..491d4b4 100644
--- a/arch/powerpc/kvm/book3s_interrupts.S
+++ b/arch/powerpc/kvm/book3s_interrupts.S
@@ -181,7 +181,7 @@ kvm_start_lightweight:
 
 	/* Restore r3 (kvm_run) and r4 (vcpu) */
 	REST_2GPRS(3, r1)
-	bl	FUNC(kvmppc_handle_exit)
+	bl	FUNC(kvmppc_handle_exit_pr)
 
 	/* If RESUME_GUEST, get back in the loop */
 	cmpwi	r3, RESUME_GUEST
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 17f7bfb..ff5bd24 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -56,7 +56,7 @@ static int kvmppc_handle_ext(struct kvm_vcpu *vcpu, unsigned int exit_nr,
 #define HW_PAGE_SIZE PAGE_SIZE
 #endif
 
-void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+static void kvmppc_core_vcpu_load_pr(struct kvm_vcpu *vcpu, int cpu)
 {
 #ifdef CONFIG_PPC_BOOK3S_64
 	struct kvmppc_book3s_shadow_vcpu *svcpu = svcpu_get(vcpu);
@@ -70,7 +70,7 @@ void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 #endif
 }
 
-void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
+static void kvmppc_core_vcpu_put_pr(struct kvm_vcpu *vcpu)
 {
 #ifdef CONFIG_PPC_BOOK3S_64
 	struct kvmppc_book3s_shadow_vcpu *svcpu = svcpu_get(vcpu);
@@ -137,7 +137,7 @@ void kvmppc_copy_from_svcpu(struct kvm_vcpu *vcpu,
 	vcpu->arch.last_inst   = svcpu->last_inst;
 }
 
-int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
+static int kvmppc_core_check_requests_pr(struct kvm_vcpu *vcpu)
 {
 	int r = 1; /* Indicate we want to get back into the guest */
 
@@ -151,7 +151,7 @@ int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
 
 /************* MMU Notifiers *************/
 
-int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
+static int kvm_unmap_hva_pr(struct kvm *kvm, unsigned long hva)
 {
 	trace_kvm_unmap_hva(hva);
 
@@ -164,7 +164,8 @@ int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
 	return 0;
 }
 
-int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)
+static int kvm_unmap_hva_range_pr(struct kvm *kvm, unsigned long start,
+			   unsigned long end)
 {
 	/* kvm_unmap_hva flushes everything anyways */
 	kvm_unmap_hva(kvm, start);
@@ -172,19 +173,19 @@ int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)
 	return 0;
 }
 
-int kvm_age_hva(struct kvm *kvm, unsigned long hva)
+static int kvm_age_hva_pr(struct kvm *kvm, unsigned long hva)
 {
 	/* XXX could be more clever ;) */
 	return 0;
 }
 
-int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
+static int kvm_test_age_hva_pr(struct kvm *kvm, unsigned long hva)
 {
 	/* XXX could be more clever ;) */
 	return 0;
 }
 
-void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte)
+static void kvm_set_spte_hva_pr(struct kvm *kvm, unsigned long hva, pte_t pte)
 {
 	/* The page will get remapped properly on its next fault */
 	kvm_unmap_hva(kvm, hva);
@@ -209,7 +210,7 @@ static void kvmppc_recalc_shadow_msr(struct kvm_vcpu *vcpu)
 	vcpu->arch.shadow_msr = smsr;
 }
 
-void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)
+static void kvmppc_set_msr_pr(struct kvm_vcpu *vcpu, u64 msr)
 {
 	ulong old_msr = vcpu->arch.shared->msr;
 
@@ -269,7 +270,7 @@ void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)
 		kvmppc_handle_ext(vcpu, BOOK3S_INTERRUPT_FP_UNAVAIL, MSR_FP);
 }
 
-void kvmppc_set_pvr(struct kvm_vcpu *vcpu, u32 pvr)
+void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, u32 pvr)
 {
 	u32 host_pvr;
 
@@ -688,8 +689,8 @@ static void kvmppc_handle_lost_ext(struct kvm_vcpu *vcpu)
 	current->thread.regs->msr |= lost_ext;
 }
 
-int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
-                       unsigned int exit_nr)
+int kvmppc_handle_exit_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
+			  unsigned int exit_nr)
 {
 	int r = RESUME_HOST;
 	int s;
@@ -989,8 +990,8 @@ program_interrupt:
 	return r;
 }
 
-int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
-                                  struct kvm_sregs *sregs)
+static int kvm_arch_vcpu_ioctl_get_sregs_pr(struct kvm_vcpu *vcpu,
+					    struct kvm_sregs *sregs)
 {
 	struct kvmppc_vcpu_book3s *vcpu3s = to_book3s(vcpu);
 	int i;
@@ -1016,13 +1017,13 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
-                                  struct kvm_sregs *sregs)
+static int kvm_arch_vcpu_ioctl_set_sregs_pr(struct kvm_vcpu *vcpu,
+					    struct kvm_sregs *sregs)
 {
 	struct kvmppc_vcpu_book3s *vcpu3s = to_book3s(vcpu);
 	int i;
 
-	kvmppc_set_pvr(vcpu, sregs->pvr);
+	kvmppc_set_pvr_pr(vcpu, sregs->pvr);
 
 	vcpu3s->sdr1 = sregs->u.s.sdr1;
 	if (vcpu->arch.hflags & BOOK3S_HFLAG_SLB) {
@@ -1052,7 +1053,8 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-int kvmppc_get_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
+static int kvmppc_get_one_reg_pr(struct kvm_vcpu *vcpu, u64 id,
+				 union kvmppc_one_reg *val)
 {
 	int r = 0;
 
@@ -1081,7 +1083,8 @@ int kvmppc_get_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	return r;
 }
 
-int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
+static int kvmppc_set_one_reg_pr(struct kvm_vcpu *vcpu, u64 id,
+				 union kvmppc_one_reg *val)
 {
 	int r = 0;
 
@@ -1111,12 +1114,8 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	return r;
 }
 
-int kvmppc_core_check_processor_compat(void)
-{
-	return 0;
-}
-
-struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
+static struct kvm_vcpu *kvmppc_core_vcpu_create_pr(struct kvm *kvm,
+						   unsigned int id)
 {
 	struct kvmppc_vcpu_book3s *vcpu_book3s;
 	struct kvm_vcpu *vcpu;
@@ -1163,7 +1162,7 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
 	/* default to book3s_32 (750) */
 	vcpu->arch.pvr = 0x84202;
 #endif
-	kvmppc_set_pvr(vcpu, vcpu->arch.pvr);
+	kvmppc_set_pvr_pr(vcpu, vcpu->arch.pvr);
 	vcpu->arch.slb_nr = 64;
 
 	vcpu->arch.shadow_msr = MSR_USER64;
@@ -1188,7 +1187,7 @@ out:
 	return ERR_PTR(err);
 }
 
-void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
+static void kvmppc_core_vcpu_free_pr(struct kvm_vcpu *vcpu)
 {
 	struct kvmppc_vcpu_book3s *vcpu_book3s = to_book3s(vcpu);
 
@@ -1242,7 +1241,7 @@ static void enable_relon_interrupts(struct kvm *kvm)
 	kvm->arch.relon_disabled = false;
 }
 
-int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
+static int kvmppc_vcpu_run_pr(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 {
 	int ret;
 	double fpr[32][TS_FPRWIDTH];
@@ -1354,8 +1353,8 @@ out:
 /*
  * Get (and clear) the dirty memory log for a memory slot.
  */
-int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,
-				      struct kvm_dirty_log *log)
+static int kvm_vm_ioctl_get_dirty_log_pr(struct kvm *kvm,
+					 struct kvm_dirty_log *log)
 {
 	struct kvm_memory_slot *memslot;
 	struct kvm_vcpu *vcpu;
@@ -1390,8 +1389,42 @@ out:
 	return r;
 }
 
+static void kvmppc_core_flush_memslot_pr(struct kvm *kvm,
+					 struct kvm_memory_slot *memslot)
+{
+	return;
+}
+
+static int kvmppc_core_prepare_memory_region_pr(struct kvm *kvm,
+					struct kvm_memory_slot *memslot,
+					struct kvm_userspace_memory_region *mem)
+{
+	return 0;
+}
+
+static void kvmppc_core_commit_memory_region_pr(struct kvm *kvm,
+				struct kvm_userspace_memory_region *mem,
+				const struct kvm_memory_slot *old)
+{
+	return;
+}
+
+static void kvmppc_core_free_memslot_pr(struct kvm_memory_slot *free,
+					struct kvm_memory_slot *dont)
+{
+	return;
+}
+
+static int kvmppc_core_create_memslot_pr(struct kvm_memory_slot *slot,
+					 unsigned long npages)
+{
+	return 0;
+}
+
+
 #ifdef CONFIG_PPC64
-int kvm_vm_ioctl_get_smmu_info(struct kvm *kvm, struct kvm_ppc_smmu_info *info)
+static int kvm_vm_ioctl_get_smmu_info_pr(struct kvm *kvm,
+					 struct kvm_ppc_smmu_info *info)
 {
 	long int i;
 	struct kvm_vcpu *vcpu;
@@ -1437,40 +1470,8 @@ int kvm_vm_ioctl_get_smmu_info(struct kvm *kvm, struct kvm_ppc_smmu_info *info)
 }
 #endif /* CONFIG_PPC64 */
 
-void kvmppc_core_free_memslot(struct kvm_memory_slot *free,
-			      struct kvm_memory_slot *dont)
-{
-}
-
-int kvmppc_core_create_memslot(struct kvm_memory_slot *slot,
-			       unsigned long npages)
-{
-	return 0;
-}
-
-int kvmppc_core_prepare_memory_region(struct kvm *kvm,
-				      struct kvm_memory_slot *memslot,
-				      struct kvm_userspace_memory_region *mem)
-{
-	return 0;
-}
-
-void kvmppc_core_commit_memory_region(struct kvm *kvm,
-				struct kvm_userspace_memory_region *mem,
-				const struct kvm_memory_slot *old)
-{
-}
-
-void kvmppc_core_flush_memslot(struct kvm *kvm, struct kvm_memory_slot *memslot)
-{
-}
-
-int kvmppc_core_init_vm(struct kvm *kvm)
+static int kvmppc_core_init_vm_pr(struct kvm *kvm)
 {
-#ifdef CONFIG_PPC64
-	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
-	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
-#endif
 	mutex_init(&kvm->arch.hpt_mutex);
 
 	/*
@@ -1482,16 +1483,71 @@ int kvmppc_core_init_vm(struct kvm *kvm)
 	return 0;
 }
 
-void kvmppc_core_destroy_vm(struct kvm *kvm)
+static void kvmppc_core_destroy_vm_pr(struct kvm *kvm)
 {
 	enable_relon_interrupts(kvm);
 }
 
-static int kvmppc_book3s_init(void)
+static int kvmppc_core_check_processor_compat_pr(void)
+{
+	/* we are always compatible */
+	return 0;
+}
+
+static long kvm_arch_vm_ioctl_pr(struct file *filp,
+				 unsigned int ioctl, unsigned long arg)
+{
+	return -ENOTTY;
+}
+
+extern void kvmppc_mmu_destroy_pr(struct kvm_vcpu *vcpu);
+extern int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
+				     unsigned int inst, int *advance);
+extern int kvmppc_core_emulate_mtspr_pr(struct kvm_vcpu *vcpu,
+					int sprn, ulong spr_val);
+extern int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu,
+					int sprn, ulong *spr_val);
+
+static struct kvmppc_ops kvmppc_pr_ops = {
+	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_pr,
+	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_pr,
+	.get_one_reg = kvmppc_get_one_reg_pr,
+	.set_one_reg = kvmppc_set_one_reg_pr,
+	.vcpu_load   = kvmppc_core_vcpu_load_pr,
+	.vcpu_put    = kvmppc_core_vcpu_put_pr,
+	.set_msr     = kvmppc_set_msr_pr,
+	.vcpu_run    = kvmppc_vcpu_run_pr,
+	.vcpu_create = kvmppc_core_vcpu_create_pr,
+	.vcpu_free   = kvmppc_core_vcpu_free_pr,
+	.check_requests = kvmppc_core_check_requests_pr,
+	.get_dirty_log = kvm_vm_ioctl_get_dirty_log_pr,
+	.flush_memslot = kvmppc_core_flush_memslot_pr,
+	.prepare_memory_region = kvmppc_core_prepare_memory_region_pr,
+	.commit_memory_region = kvmppc_core_commit_memory_region_pr,
+	.unmap_hva = kvm_unmap_hva_pr,
+	.unmap_hva_range = kvm_unmap_hva_range_pr,
+	.age_hva  = kvm_age_hva_pr,
+	.test_age_hva = kvm_test_age_hva_pr,
+	.set_spte_hva = kvm_set_spte_hva_pr,
+	.mmu_destroy  = kvmppc_mmu_destroy_pr,
+	.free_memslot = kvmppc_core_free_memslot_pr,
+	.create_memslot = kvmppc_core_create_memslot_pr,
+	.init_vm = kvmppc_core_init_vm_pr,
+	.destroy_vm = kvmppc_core_destroy_vm_pr,
+	.check_processor_compat = kvmppc_core_check_processor_compat_pr,
+	.get_smmu_info = kvm_vm_ioctl_get_smmu_info_pr,
+	.emulate_op = kvmppc_core_emulate_op_pr,
+	.emulate_mtspr = kvmppc_core_emulate_mtspr_pr,
+	.emulate_mfspr = kvmppc_core_emulate_mfspr_pr,
+	.fast_vcpu_kick = kvm_vcpu_kick,
+	.arch_vm_ioctl  = kvm_arch_vm_ioctl_pr,
+};
+
+static int kvmppc_book3s_init_pr(void)
 {
 	int r;
 
-	r = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+	r = kvm_init(&kvmppc_pr_ops, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
 
 	if (r)
 		return r;
@@ -1501,11 +1557,11 @@ static int kvmppc_book3s_init(void)
 	return r;
 }
 
-static void kvmppc_book3s_exit(void)
+static void kvmppc_book3s_exit_pr(void)
 {
 	kvmppc_mmu_hpte_sysexit();
 	kvm_exit();
 }
 
-module_init(kvmppc_book3s_init);
-module_exit(kvmppc_book3s_exit);
+module_init(kvmppc_book3s_init_pr);
+module_exit(kvmppc_book3s_exit_pr);
diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
index 751cd45..81762eb 100644
--- a/arch/powerpc/kvm/emulate.c
+++ b/arch/powerpc/kvm/emulate.c
@@ -130,7 +130,7 @@ static int kvmppc_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs)
 	case SPRN_PIR: break;
 
 	default:
-		emulated = kvmppc_core_emulate_mtspr(vcpu, sprn,
+		emulated = kvmppc_ops->emulate_mtspr(vcpu, sprn,
 						     spr_val);
 		if (emulated == EMULATE_FAIL)
 			printk(KERN_INFO "mtspr: unknown spr "
@@ -191,7 +191,7 @@ static int kvmppc_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, int rt)
 		spr_val = kvmppc_get_dec(vcpu, get_tb());
 		break;
 	default:
-		emulated = kvmppc_core_emulate_mfspr(vcpu, sprn,
+		emulated = kvmppc_ops->emulate_mfspr(vcpu, sprn,
 						     &spr_val);
 		if (unlikely(emulated == EMULATE_FAIL)) {
 			printk(KERN_INFO "mfspr: unknown spr "
@@ -464,7 +464,7 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
 	}
 
 	if (emulated == EMULATE_FAIL) {
-		emulated = kvmppc_core_emulate_op(run, vcpu, inst, &advance);
+		emulated = kvmppc_ops->emulate_op(run, vcpu, inst, &advance);
 		if (emulated == EMULATE_AGAIN) {
 			advance = 0;
 		} else if (emulated == EMULATE_FAIL) {
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 07c0106..69b9305 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -39,6 +39,8 @@
 #define CREATE_TRACE_POINTS
 #include "trace.h"
 
+struct kvmppc_ops *kvmppc_ops;
+
 int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
 {
 	return !!(v->arch.pending_exceptions) ||
@@ -1024,52 +1026,11 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		r = kvm_vm_ioctl_create_spapr_tce(kvm, &create_tce);
 		goto out;
 	}
-#endif /* CONFIG_PPC_BOOK3S_64 */
-
-#ifdef CONFIG_KVM_BOOK3S_64_HV
-	case KVM_ALLOCATE_RMA: {
-		struct kvm_allocate_rma rma;
-		struct kvm *kvm = filp->private_data;
-
-		r = kvm_vm_ioctl_allocate_rma(kvm, &rma);
-		if (r >= 0 && copy_to_user(argp, &rma, sizeof(rma)))
-			r = -EFAULT;
-		break;
-	}
-
-	case KVM_PPC_ALLOCATE_HTAB: {
-		u32 htab_order;
-
-		r = -EFAULT;
-		if (get_user(htab_order, (u32 __user *)argp))
-			break;
-		r = kvmppc_alloc_reset_hpt(kvm, &htab_order);
-		if (r)
-			break;
-		r = -EFAULT;
-		if (put_user(htab_order, (u32 __user *)argp))
-			break;
-		r = 0;
-		break;
-	}
-
-	case KVM_PPC_GET_HTAB_FD: {
-		struct kvm_get_htab_fd ghf;
-
-		r = -EFAULT;
-		if (copy_from_user(&ghf, argp, sizeof(ghf)))
-			break;
-		r = kvm_vm_ioctl_get_htab_fd(kvm, &ghf);
-		break;
-	}
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
-
-#ifdef CONFIG_PPC_BOOK3S_64
 	case KVM_PPC_GET_SMMU_INFO: {
 		struct kvm_ppc_smmu_info info;
 
 		memset(&info, 0, sizeof(info));
-		r = kvm_vm_ioctl_get_smmu_info(kvm, &info);
+		r = kvmppc_ops->get_smmu_info(kvm, &info);
 		if (r >= 0 && copy_to_user(argp, &info, sizeof(info)))
 			r = -EFAULT;
 		break;
@@ -1080,11 +1041,14 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		r = kvm_vm_ioctl_rtas_define_token(kvm, argp);
 		break;
 	}
-#endif /* CONFIG_PPC_BOOK3S_64 */
+	default:
+		r = kvmppc_ops->arch_vm_ioctl(filp, ioctl, arg);
+
+#else /* CONFIG_PPC_BOOK3S_64 */
 	default:
 		r = -ENOTTY;
+#endif
 	}
-
 out:
 	return r;
 }
@@ -1125,9 +1089,15 @@ void kvmppc_init_lpid(unsigned long nr_lpids_param)
 
 int kvm_arch_init(void *opaque)
 {
+	if (kvmppc_ops) {
+		printk(KERN_ERR "kvm: already loaded the other module\n");
+		return -EEXIST;
+	}
+	kvmppc_ops = (struct kvmppc_ops *)opaque;
 	return 0;
 }
 
 void kvm_arch_exit(void)
 {
+	kvmppc_ops = NULL;
 }
-- 
1.8.1.2

[RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

This help us to identify whether we are running with hypervisor mode KVM
enabled. The change is needed so that we can have both HV and PR kvm
enabled in the same kernel.

If both HV and PR KVM are included, interrupts come in to the HV version
of the kvmppc_interrupt code, which then jumps to the PR handler,
renamed to kvmppc_interrupt_pr, if the guest is a PR guest.

Allowing both PR and HV in the same kernel required some changes to
kvm_dev_ioctl_check_extension(), since the values returned now can't
be selected with #ifdefs as much as previously. We look at is_hv_enabled
to return the right value when checking for capabilities.For capabilities that
are only provided by HV KVM, we return the HV value only if
is_hv_enabled is true. For capabilities provided by PR KVM but not HV,
we return the PR value only if is_hv_enabled is false.

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_book3s.h   | 53 --------------------------------
 arch/powerpc/include/asm/kvm_ppc.h      |  5 +--
 arch/powerpc/kvm/Makefile               |  2 +-
 arch/powerpc/kvm/book3s.c               | 44 +++++++++++++++++++++++++++
 arch/powerpc/kvm/book3s_hv.c            |  1 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 +++
 arch/powerpc/kvm/book3s_pr.c            |  1 +
 arch/powerpc/kvm/book3s_segment.S       |  7 ++++-
 arch/powerpc/kvm/book3s_xics.c          |  2 +-
 arch/powerpc/kvm/powerpc.c              | 54 ++++++++++++++++++---------------
 10 files changed, 90 insertions(+), 83 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h
index 3efba3c..ba33c49 100644
--- a/arch/powerpc/include/asm/kvm_book3s.h
+++ b/arch/powerpc/include/asm/kvm_book3s.h
@@ -297,59 +297,6 @@ static inline ulong kvmppc_get_fault_dar(struct kvm_vcpu *vcpu)
 	return vcpu->arch.fault_dar;
 }
 
-#ifdef CONFIG_KVM_BOOK3S_PR
-
-static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
-{
-	return to_book3s(vcpu)->hior;
-}
-
-static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
-			unsigned long pending_now, unsigned long old_pending)
-{
-	if (pending_now)
-		vcpu->arch.shared->int_pending = 1;
-	else if (old_pending)
-		vcpu->arch.shared->int_pending = 0;
-}
-
-static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
-{
-	ulong crit_raw = vcpu->arch.shared->critical;
-	ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
-	bool crit;
-
-	/* Truncate crit indicators in 32 bit mode */
-	if (!(vcpu->arch.shared->msr & MSR_SF)) {
-		crit_raw &= 0xffffffff;
-		crit_r1 &= 0xffffffff;
-	}
-
-	/* Critical section when crit == r1 */
-	crit = (crit_raw == crit_r1);
-	/* ... and we're in supervisor mode */
-	crit = crit && !(vcpu->arch.shared->msr & MSR_PR);
-
-	return crit;
-}
-#else /* CONFIG_KVM_BOOK3S_PR */
-
-static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
-{
-	return 0;
-}
-
-static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
-			unsigned long pending_now, unsigned long old_pending)
-{
-}
-
-static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
-{
-	return false;
-}
-#endif
-
 /* Magic register values loaded into r3 and r4 before the 'sc' assembly
  * instruction for the OSI hypercalls */
 #define OSI_SC_MAGIC_R3			0x113724FA
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 4d9641c..58e732f 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -169,6 +169,7 @@ extern int kvmppc_xics_int_on(struct kvm *kvm, u32 irq);
 extern int kvmppc_xics_int_off(struct kvm *kvm, u32 irq);
 
 struct kvmppc_ops {
+	bool is_hv_enabled;
 	int (*get_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
 	int (*set_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
 	int (*get_one_reg)(struct kvm_vcpu *vcpu, u64 id,
@@ -309,10 +310,10 @@ static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr)
 
 static inline u32 kvmppc_get_xics_latch(void)
 {
-	u32 xirr = get_paca()->kvm_hstate.saved_xirr;
+	u32 xirr;
 
+	xirr = get_paca()->kvm_hstate.saved_xirr;
 	get_paca()->kvm_hstate.saved_xirr = 0;
-
 	return xirr;
 }
 
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index c343793..a514ecd 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -77,7 +77,7 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
 	book3s_rmhandlers.o
 endif
 
-kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
+kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  += \
 	book3s_hv.o \
 	book3s_hv_interrupts.o \
 	book3s_64_mmu_hv.o
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index bdc3f95..12f94bf 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -69,6 +69,50 @@ void kvmppc_core_load_guest_debugstate(struct kvm_vcpu *vcpu)
 {
 }
 
+static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
+{
+	if (!kvmppc_ops->is_hv_enabled)
+		return to_book3s(vcpu)->hior;
+	return 0;
+}
+
+static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
+			unsigned long pending_now, unsigned long old_pending)
+{
+	if (kvmppc_ops->is_hv_enabled)
+		return;
+	if (pending_now)
+		vcpu->arch.shared->int_pending = 1;
+	else if (old_pending)
+		vcpu->arch.shared->int_pending = 0;
+}
+
+static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
+{
+	ulong crit_raw;
+	ulong crit_r1;
+	bool crit;
+
+	if (kvmppc_ops->is_hv_enabled)
+		return false;
+
+	crit_raw = vcpu->arch.shared->critical;
+	crit_r1 = kvmppc_get_gpr(vcpu, 1);
+
+	/* Truncate crit indicators in 32 bit mode */
+	if (!(vcpu->arch.shared->msr & MSR_SF)) {
+		crit_raw &= 0xffffffff;
+		crit_r1 &= 0xffffffff;
+	}
+
+	/* Critical section when crit == r1 */
+	crit = (crit_raw == crit_r1);
+	/* ... and we're in supervisor mode */
+	crit = crit && !(vcpu->arch.shared->msr & MSR_PR);
+
+	return crit;
+}
+
 void kvmppc_inject_interrupt(struct kvm_vcpu *vcpu, int vec, u64 flags)
 {
 	vcpu->arch.shared->srr0 = kvmppc_get_pc(vcpu);
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 36d7920..eec353d 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2048,6 +2048,7 @@ extern int kvm_test_age_hva_hv(struct kvm *kvm, unsigned long hva);
 extern void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, pte_t pte);
 
 static struct kvmppc_ops kvmppc_hv_ops = {
+	.is_hv_enabled = true,
 	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv,
 	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
 	.get_one_reg = kvmppc_get_one_reg_hv,
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 5ede7fc..4838fdb 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -577,6 +577,10 @@ kvmppc_interrupt:
 	lbz	r9, HSTATE_IN_GUEST(r13)
 	cmpwi	r9, KVM_GUEST_MODE_HOST_HV
 	beq	kvmppc_bad_host_intr
+#ifdef CONFIG_KVM_BOOK3S_PR
+	cmpwi	r9, KVM_GUEST_MODE_GUEST
+	beq	kvmppc_interrupt_pr
+#endif
 	/* We're now back in the host but in guest MMU context */
 	li	r9, KVM_GUEST_MODE_HOST_HV
 	stb	r9, HSTATE_IN_GUEST(r13)
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index ff5bd24..2a97279 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -1509,6 +1509,7 @@ extern int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu,
 					int sprn, ulong *spr_val);
 
 static struct kvmppc_ops kvmppc_pr_ops = {
+	.is_hv_enabled = false,
 	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_pr,
 	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_pr,
 	.get_one_reg = kvmppc_get_one_reg_pr,
diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
index 1abe478..e0229dd 100644
--- a/arch/powerpc/kvm/book3s_segment.S
+++ b/arch/powerpc/kvm/book3s_segment.S
@@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
 .global kvmppc_handler_trampoline_exit
 kvmppc_handler_trampoline_exit:
 
+#if defined(CONFIG_KVM_BOOK3S_HV)
+.global kvmppc_interrupt_pr
+kvmppc_interrupt_pr:
+	ld	r9, HSTATE_SCRATCH2(r13)
+#else
 .global kvmppc_interrupt
 kvmppc_interrupt:
-
+#endif
 	/* Register usage at this point:
 	 *
 	 * SPRG_SCRATCH0  = guest R13
diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c
index f0c732e..fa7625d 100644
--- a/arch/powerpc/kvm/book3s_xics.c
+++ b/arch/powerpc/kvm/book3s_xics.c
@@ -818,7 +818,7 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req)
 	}
 
 	/* Check for real mode returning too hard */
-	if (xics->real_mode)
+	if (xics->real_mode && kvmppc_ops->is_hv_enabled)
 		return kvmppc_xics_rm_complete(vcpu, req);
 
 	switch (req) {
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 69b9305..00a96fc 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -52,7 +52,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
-#ifndef CONFIG_KVM_BOOK3S_64_HV
 /*
  * Common checks before entering the guest world.  Call with interrupts
  * disabled.
@@ -127,7 +126,6 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 
 	return r;
 }
-#endif /* CONFIG_KVM_BOOK3S_64_HV */
 
 int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
 {
@@ -194,11 +192,9 @@ int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
 	if ((vcpu->arch.cpu_type != KVM_CPU_3S_64) && vcpu->arch.papr_enabled)
 		goto out;
 
-#ifdef CONFIG_KVM_BOOK3S_64_HV
 	/* HV KVM can only do PAPR mode for now */
-	if (!vcpu->arch.papr_enabled)
+	if (!vcpu->arch.papr_enabled && kvmppc_ops->is_hv_enabled)
 		goto out;
-#endif
 
 #ifdef CONFIG_KVM_BOOKE_HV
 	if (!cpu_has_feature(CPU_FTR_EMB_HV))
@@ -322,22 +318,26 @@ int kvm_dev_ioctl_check_extension(long ext)
 	case KVM_CAP_DEVICE_CTRL:
 		r = 1;
 		break;
-#ifndef CONFIG_KVM_BOOK3S_64_HV
 	case KVM_CAP_PPC_PAIRED_SINGLES:
 	case KVM_CAP_PPC_OSI:
 	case KVM_CAP_PPC_GET_PVINFO:
 #if defined(CONFIG_KVM_E500V2) || defined(CONFIG_KVM_E500MC)
 	case KVM_CAP_SW_TLB:
 #endif
-#ifdef CONFIG_KVM_MPIC
-	case KVM_CAP_IRQ_MPIC:
-#endif
-		r = 1;
+		/* We support this only for PR */
+		r = !kvmppc_ops->is_hv_enabled;
 		break;
+#ifdef CONFIG_KVM_MMIO
 	case KVM_CAP_COALESCED_MMIO:
 		r = KVM_COALESCED_MMIO_PAGE_OFFSET;
 		break;
 #endif
+#ifdef CONFIG_KVM_MPIC
+	case KVM_CAP_IRQ_MPIC:
+		r = 1;
+		break;
+#endif
+
 #ifdef CONFIG_PPC_BOOK3S_64
 	case KVM_CAP_SPAPR_TCE:
 	case KVM_CAP_PPC_ALLOC_HTAB:
@@ -348,32 +348,37 @@ int kvm_dev_ioctl_check_extension(long ext)
 		r = 1;
 		break;
 #endif /* CONFIG_PPC_BOOK3S_64 */
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_HV
 	case KVM_CAP_PPC_SMT:
-		r = threads_per_core;
+		if (kvmppc_ops->is_hv_enabled)
+			r = threads_per_core;
+		else
+			r = 0;
 		break;
 	case KVM_CAP_PPC_RMA:
-		r = 1;
+		r = kvmppc_ops->is_hv_enabled;
 		/* PPC970 requires an RMA */
-		if (cpu_has_feature(CPU_FTR_ARCH_201))
+		if (r && cpu_has_feature(CPU_FTR_ARCH_201))
 			r = 2;
 		break;
 #endif
 	case KVM_CAP_SYNC_MMU:
-#ifdef CONFIG_KVM_BOOK3S_64_HV
-		r = cpu_has_feature(CPU_FTR_ARCH_206) ? 1 : 0;
+#ifdef CONFIG_KVM_BOOK3S_HV
+		if (kvmppc_ops->is_hv_enabled)
+			r = cpu_has_feature(CPU_FTR_ARCH_206) ? 1 : 0;
+		else
+			r = 0;
 #elif defined(KVM_ARCH_WANT_MMU_NOTIFIER)
 		r = 1;
 #else
 		r = 0;
-		break;
 #endif
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+		break;
+#ifdef CONFIG_KVM_BOOK3S_HV
 	case KVM_CAP_PPC_HTAB_FD:
-		r = 1;
+		r = kvmppc_ops->is_hv_enabled;
 		break;
 #endif
-		break;
 	case KVM_CAP_NR_VCPUS:
 		/*
 		 * Recommending a number of CPUs is somewhat arbitrary; we
@@ -381,11 +386,10 @@ int kvm_dev_ioctl_check_extension(long ext)
 		 * will have secondary threads "offline"), and for other KVM
 		 * implementations just count online CPUs.
 		 */
-#ifdef CONFIG_KVM_BOOK3S_64_HV
-		r = num_present_cpus();
-#else
-		r = num_online_cpus();
-#endif
+		if (kvmppc_ops->is_hv_enabled)
+			r = num_present_cpus();
+		else
+			r = num_online_cpus();
 		break;
 	case KVM_CAP_MAX_VCPUS:
 		r = KVM_MAX_VCPUS;
-- 
1.8.1.2

[RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

This patch moves PR related tracepoints to a separate header. This
enables in converting PR to a kernel module which will be done in
later patches

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
 arch/powerpc/kvm/book3s_mmu_hpte.c    |   2 +-
 arch/powerpc/kvm/book3s_pr.c          |   3 +-
 arch/powerpc/kvm/trace.h              | 234 +--------------------------
 arch/powerpc/kvm/trace_pr.h           | 297 ++++++++++++++++++++++++++++++++++
 5 files changed, 308 insertions(+), 230 deletions(-)
 create mode 100644 arch/powerpc/kvm/trace_pr.h

diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index 329a978..fd5b393 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -27,7 +27,7 @@
 #include <asm/machdep.h>
 #include <asm/mmu_context.h>
 #include <asm/hw_irq.h>
-#include "trace.h"
+#include "trace_pr.h"
 
 #define PTE_SIZE 12
 
diff --git a/arch/powerpc/kvm/book3s_mmu_hpte.c b/arch/powerpc/kvm/book3s_mmu_hpte.c
index d2d280b..4556168 100644
--- a/arch/powerpc/kvm/book3s_mmu_hpte.c
+++ b/arch/powerpc/kvm/book3s_mmu_hpte.c
@@ -28,7 +28,7 @@
 #include <asm/mmu_context.h>
 #include <asm/hw_irq.h>
 
-#include "trace.h"
+#include "trace_pr.h"
 
 #define PTE_SIZE	12
 
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 2a97279..99d0839 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -41,7 +41,8 @@
 #include <linux/vmalloc.h>
 #include <linux/highmem.h>
 
-#include "trace.h"
+#define CREATE_TRACE_POINTS
+#include "trace_pr.h"
 
 /* #define EXIT_DEBUG */
 /* #define DEBUG_EXT */
diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
index a088e9a..7d5a136 100644
--- a/arch/powerpc/kvm/trace.h
+++ b/arch/powerpc/kvm/trace.h
@@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
 	{41, "HV_PRIV"}
 #endif
 
+#ifndef CONFIG_KVM_BOOK3S_PR
+/*
+ * For pr we define this in trace_pr.h since it pr can be built as
+ * a module
+ */
+
 TRACE_EVENT(kvm_exit,
 	TP_PROTO(unsigned int exit_nr, struct kvm_vcpu *vcpu),
 	TP_ARGS(exit_nr, vcpu),
@@ -94,9 +100,6 @@ TRACE_EVENT(kvm_exit,
 		__field(	unsigned long,	pc		)
 		__field(	unsigned long,	msr		)
 		__field(	unsigned long,	dar		)
-#ifdef CONFIG_KVM_BOOK3S_PR
-		__field(	unsigned long,	srr1		)
-#endif
 		__field(	unsigned long,	last_inst	)
 	),
 
@@ -105,9 +108,6 @@ TRACE_EVENT(kvm_exit,
 		__entry->pc		= kvmppc_get_pc(vcpu);
 		__entry->dar		= kvmppc_get_fault_dar(vcpu);
 		__entry->msr		= vcpu->arch.shared->msr;
-#ifdef CONFIG_KVM_BOOK3S_PR
-		__entry->srr1		= vcpu->arch.shadow_srr1;
-#endif
 		__entry->last_inst	= vcpu->arch.last_inst;
 	),
 
@@ -115,18 +115,12 @@ TRACE_EVENT(kvm_exit,
 		" | pc=0x%lx"
 		" | msr=0x%lx"
 		" | dar=0x%lx"
-#ifdef CONFIG_KVM_BOOK3S_PR
-		" | srr1=0x%lx"
-#endif
 		" | last_inst=0x%lx"
 		,
 		__print_symbolic(__entry->exit_nr, kvm_trace_symbol_exit),
 		__entry->pc,
 		__entry->msr,
 		__entry->dar,
-#ifdef CONFIG_KVM_BOOK3S_PR
-		__entry->srr1,
-#endif
 		__entry->last_inst
 		)
 );
@@ -145,6 +139,7 @@ TRACE_EVENT(kvm_unmap_hva,
 
 	TP_printk("unmap hva 0x%lx\n", __entry->hva)
 );
+#endif
 
 TRACE_EVENT(kvm_stlb_inval,
 	TP_PROTO(unsigned int stlb_index),
@@ -231,221 +226,6 @@ TRACE_EVENT(kvm_check_requests,
 		__entry->cpu_nr, __entry->requests)
 );
 
-
-/*************************************************************************
- *                         Book3S trace points                           *
- *************************************************************************/
-
-#ifdef CONFIG_KVM_BOOK3S_PR
-
-TRACE_EVENT(kvm_book3s_reenter,
-	TP_PROTO(int r, struct kvm_vcpu *vcpu),
-	TP_ARGS(r, vcpu),
-
-	TP_STRUCT__entry(
-		__field(	unsigned int,	r		)
-		__field(	unsigned long,	pc		)
-	),
-
-	TP_fast_assign(
-		__entry->r		= r;
-		__entry->pc		= kvmppc_get_pc(vcpu);
-	),
-
-	TP_printk("reentry r=%d | pc=0x%lx", __entry->r, __entry->pc)
-);
-
-#ifdef CONFIG_PPC_BOOK3S_64
-
-TRACE_EVENT(kvm_book3s_64_mmu_map,
-	TP_PROTO(int rflags, ulong hpteg, ulong va, pfn_t hpaddr,
-		 struct kvmppc_pte *orig_pte),
-	TP_ARGS(rflags, hpteg, va, hpaddr, orig_pte),
-
-	TP_STRUCT__entry(
-		__field(	unsigned char,		flag_w		)
-		__field(	unsigned char,		flag_x		)
-		__field(	unsigned long,		eaddr		)
-		__field(	unsigned long,		hpteg		)
-		__field(	unsigned long,		va		)
-		__field(	unsigned long long,	vpage		)
-		__field(	unsigned long,		hpaddr		)
-	),
-
-	TP_fast_assign(
-		__entry->flag_w	= ((rflags & HPTE_R_PP) == 3) ? '-' : 'w';
-		__entry->flag_x	= (rflags & HPTE_R_N) ? '-' : 'x';
-		__entry->eaddr	= orig_pte->eaddr;
-		__entry->hpteg	= hpteg;
-		__entry->va	= va;
-		__entry->vpage	= orig_pte->vpage;
-		__entry->hpaddr	= hpaddr;
-	),
-
-	TP_printk("KVM: %c%c Map 0x%lx: [%lx] 0x%lx (0x%llx) -> %lx",
-		  __entry->flag_w, __entry->flag_x, __entry->eaddr,
-		  __entry->hpteg, __entry->va, __entry->vpage, __entry->hpaddr)
-);
-
-#endif /* CONFIG_PPC_BOOK3S_64 */
-
-TRACE_EVENT(kvm_book3s_mmu_map,
-	TP_PROTO(struct hpte_cache *pte),
-	TP_ARGS(pte),
-
-	TP_STRUCT__entry(
-		__field(	u64,		host_vpn	)
-		__field(	u64,		pfn		)
-		__field(	ulong,		eaddr		)
-		__field(	u64,		vpage		)
-		__field(	ulong,		raddr		)
-		__field(	int,		flags		)
-	),
-
-	TP_fast_assign(
-		__entry->host_vpn	= pte->host_vpn;
-		__entry->pfn		= pte->pfn;
-		__entry->eaddr		= pte->pte.eaddr;
-		__entry->vpage		= pte->pte.vpage;
-		__entry->raddr		= pte->pte.raddr;
-		__entry->flags		= (pte->pte.may_read ? 0x4 : 0) |
-					  (pte->pte.may_write ? 0x2 : 0) |
-					  (pte->pte.may_execute ? 0x1 : 0);
-	),
-
-	TP_printk("Map: hvpn=%llx pfn=%llx ea=%lx vp=%llx ra=%lx [%x]",
-		  __entry->host_vpn, __entry->pfn, __entry->eaddr,
-		  __entry->vpage, __entry->raddr, __entry->flags)
-);
-
-TRACE_EVENT(kvm_book3s_mmu_invalidate,
-	TP_PROTO(struct hpte_cache *pte),
-	TP_ARGS(pte),
-
-	TP_STRUCT__entry(
-		__field(	u64,		host_vpn	)
-		__field(	u64,		pfn		)
-		__field(	ulong,		eaddr		)
-		__field(	u64,		vpage		)
-		__field(	ulong,		raddr		)
-		__field(	int,		flags		)
-	),
-
-	TP_fast_assign(
-		__entry->host_vpn	= pte->host_vpn;
-		__entry->pfn		= pte->pfn;
-		__entry->eaddr		= pte->pte.eaddr;
-		__entry->vpage		= pte->pte.vpage;
-		__entry->raddr		= pte->pte.raddr;
-		__entry->flags		= (pte->pte.may_read ? 0x4 : 0) |
-					  (pte->pte.may_write ? 0x2 : 0) |
-					  (pte->pte.may_execute ? 0x1 : 0);
-	),
-
-	TP_printk("Flush: hva=%llx pfn=%llx ea=%lx vp=%llx ra=%lx [%x]",
-		  __entry->host_vpn, __entry->pfn, __entry->eaddr,
-		  __entry->vpage, __entry->raddr, __entry->flags)
-);
-
-TRACE_EVENT(kvm_book3s_mmu_flush,
-	TP_PROTO(const char *type, struct kvm_vcpu *vcpu, unsigned long long p1,
-		 unsigned long long p2),
-	TP_ARGS(type, vcpu, p1, p2),
-
-	TP_STRUCT__entry(
-		__field(	int,			count		)
-		__field(	unsigned long long,	p1		)
-		__field(	unsigned long long,	p2		)
-		__field(	const char *,		type		)
-	),
-
-	TP_fast_assign(
-		__entry->count		= to_book3s(vcpu)->hpte_cache_count;
-		__entry->p1		= p1;
-		__entry->p2		= p2;
-		__entry->type		= type;
-	),
-
-	TP_printk("Flush %d %sPTEs: %llx - %llx",
-		  __entry->count, __entry->type, __entry->p1, __entry->p2)
-);
-
-TRACE_EVENT(kvm_book3s_slb_found,
-	TP_PROTO(unsigned long long gvsid, unsigned long long hvsid),
-	TP_ARGS(gvsid, hvsid),
-
-	TP_STRUCT__entry(
-		__field(	unsigned long long,	gvsid		)
-		__field(	unsigned long long,	hvsid		)
-	),
-
-	TP_fast_assign(
-		__entry->gvsid		= gvsid;
-		__entry->hvsid		= hvsid;
-	),
-
-	TP_printk("%llx -> %llx", __entry->gvsid, __entry->hvsid)
-);
-
-TRACE_EVENT(kvm_book3s_slb_fail,
-	TP_PROTO(u16 sid_map_mask, unsigned long long gvsid),
-	TP_ARGS(sid_map_mask, gvsid),
-
-	TP_STRUCT__entry(
-		__field(	unsigned short,		sid_map_mask	)
-		__field(	unsigned long long,	gvsid		)
-	),
-
-	TP_fast_assign(
-		__entry->sid_map_mask	= sid_map_mask;
-		__entry->gvsid		= gvsid;
-	),
-
-	TP_printk("%x/%x: %llx", __entry->sid_map_mask,
-		  SID_MAP_MASK - __entry->sid_map_mask, __entry->gvsid)
-);
-
-TRACE_EVENT(kvm_book3s_slb_map,
-	TP_PROTO(u16 sid_map_mask, unsigned long long gvsid,
-		 unsigned long long hvsid),
-	TP_ARGS(sid_map_mask, gvsid, hvsid),
-
-	TP_STRUCT__entry(
-		__field(	unsigned short,		sid_map_mask	)
-		__field(	unsigned long long,	guest_vsid	)
-		__field(	unsigned long long,	host_vsid	)
-	),
-
-	TP_fast_assign(
-		__entry->sid_map_mask	= sid_map_mask;
-		__entry->guest_vsid	= gvsid;
-		__entry->host_vsid	= hvsid;
-	),
-
-	TP_printk("%x: %llx -> %llx", __entry->sid_map_mask,
-		  __entry->guest_vsid, __entry->host_vsid)
-);
-
-TRACE_EVENT(kvm_book3s_slbmte,
-	TP_PROTO(u64 slb_vsid, u64 slb_esid),
-	TP_ARGS(slb_vsid, slb_esid),
-
-	TP_STRUCT__entry(
-		__field(	u64,	slb_vsid	)
-		__field(	u64,	slb_esid	)
-	),
-
-	TP_fast_assign(
-		__entry->slb_vsid	= slb_vsid;
-		__entry->slb_esid	= slb_esid;
-	),
-
-	TP_printk("%llx, %llx", __entry->slb_vsid, __entry->slb_esid)
-);
-
-#endif /* CONFIG_PPC_BOOK3S */
-
-
 /*************************************************************************
  *                         Book3E trace points                           *
  *************************************************************************/
diff --git a/arch/powerpc/kvm/trace_pr.h b/arch/powerpc/kvm/trace_pr.h
new file mode 100644
index 0000000..8b22e47
--- /dev/null
+++ b/arch/powerpc/kvm/trace_pr.h
@@ -0,0 +1,297 @@
+
+#if !defined(_TRACE_KVM_PR_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_KVM_PR_H
+
+#include <linux/tracepoint.h>
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM kvm_pr
+#define TRACE_INCLUDE_PATH .
+#define TRACE_INCLUDE_FILE trace_pr
+
+#define kvm_trace_symbol_exit \
+	{0x100, "SYSTEM_RESET"}, \
+	{0x200, "MACHINE_CHECK"}, \
+	{0x300, "DATA_STORAGE"}, \
+	{0x380, "DATA_SEGMENT"}, \
+	{0x400, "INST_STORAGE"}, \
+	{0x480, "INST_SEGMENT"}, \
+	{0x500, "EXTERNAL"}, \
+	{0x501, "EXTERNAL_LEVEL"}, \
+	{0x502, "EXTERNAL_HV"}, \
+	{0x600, "ALIGNMENT"}, \
+	{0x700, "PROGRAM"}, \
+	{0x800, "FP_UNAVAIL"}, \
+	{0x900, "DECREMENTER"}, \
+	{0x980, "HV_DECREMENTER"}, \
+	{0xc00, "SYSCALL"}, \
+	{0xd00, "TRACE"}, \
+	{0xe00, "H_DATA_STORAGE"}, \
+	{0xe20, "H_INST_STORAGE"}, \
+	{0xe40, "H_EMUL_ASSIST"}, \
+	{0xf00, "PERFMON"}, \
+	{0xf20, "ALTIVEC"}, \
+	{0xf40, "VSX"}
+
+TRACE_EVENT(kvm_book3s_reenter,
+	TP_PROTO(int r, struct kvm_vcpu *vcpu),
+	TP_ARGS(r, vcpu),
+
+	TP_STRUCT__entry(
+		__field(	unsigned int,	r		)
+		__field(	unsigned long,	pc		)
+	),
+
+	TP_fast_assign(
+		__entry->r		= r;
+		__entry->pc		= kvmppc_get_pc(vcpu);
+	),
+
+	TP_printk("reentry r=%d | pc=0x%lx", __entry->r, __entry->pc)
+);
+
+#ifdef CONFIG_PPC_BOOK3S_64
+
+TRACE_EVENT(kvm_book3s_64_mmu_map,
+	TP_PROTO(int rflags, ulong hpteg, ulong va, pfn_t hpaddr,
+		 struct kvmppc_pte *orig_pte),
+	TP_ARGS(rflags, hpteg, va, hpaddr, orig_pte),
+
+	TP_STRUCT__entry(
+		__field(	unsigned char,		flag_w		)
+		__field(	unsigned char,		flag_x		)
+		__field(	unsigned long,		eaddr		)
+		__field(	unsigned long,		hpteg		)
+		__field(	unsigned long,		va		)
+		__field(	unsigned long long,	vpage		)
+		__field(	unsigned long,		hpaddr		)
+	),
+
+	TP_fast_assign(
+		__entry->flag_w	= ((rflags & HPTE_R_PP) == 3) ? '-' : 'w';
+		__entry->flag_x	= (rflags & HPTE_R_N) ? '-' : 'x';
+		__entry->eaddr	= orig_pte->eaddr;
+		__entry->hpteg	= hpteg;
+		__entry->va	= va;
+		__entry->vpage	= orig_pte->vpage;
+		__entry->hpaddr	= hpaddr;
+	),
+
+	TP_printk("KVM: %c%c Map 0x%lx: [%lx] 0x%lx (0x%llx) -> %lx",
+		  __entry->flag_w, __entry->flag_x, __entry->eaddr,
+		  __entry->hpteg, __entry->va, __entry->vpage, __entry->hpaddr)
+);
+
+#endif /* CONFIG_PPC_BOOK3S_64 */
+
+TRACE_EVENT(kvm_book3s_mmu_map,
+	TP_PROTO(struct hpte_cache *pte),
+	TP_ARGS(pte),
+
+	TP_STRUCT__entry(
+		__field(	u64,		host_vpn	)
+		__field(	u64,		pfn		)
+		__field(	ulong,		eaddr		)
+		__field(	u64,		vpage		)
+		__field(	ulong,		raddr		)
+		__field(	int,		flags		)
+	),
+
+	TP_fast_assign(
+		__entry->host_vpn	= pte->host_vpn;
+		__entry->pfn		= pte->pfn;
+		__entry->eaddr		= pte->pte.eaddr;
+		__entry->vpage		= pte->pte.vpage;
+		__entry->raddr		= pte->pte.raddr;
+		__entry->flags		= (pte->pte.may_read ? 0x4 : 0) |
+					  (pte->pte.may_write ? 0x2 : 0) |
+					  (pte->pte.may_execute ? 0x1 : 0);
+	),
+
+	TP_printk("Map: hvpn=%llx pfn=%llx ea=%lx vp=%llx ra=%lx [%x]",
+		  __entry->host_vpn, __entry->pfn, __entry->eaddr,
+		  __entry->vpage, __entry->raddr, __entry->flags)
+);
+
+TRACE_EVENT(kvm_book3s_mmu_invalidate,
+	TP_PROTO(struct hpte_cache *pte),
+	TP_ARGS(pte),
+
+	TP_STRUCT__entry(
+		__field(	u64,		host_vpn	)
+		__field(	u64,		pfn		)
+		__field(	ulong,		eaddr		)
+		__field(	u64,		vpage		)
+		__field(	ulong,		raddr		)
+		__field(	int,		flags		)
+	),
+
+	TP_fast_assign(
+		__entry->host_vpn	= pte->host_vpn;
+		__entry->pfn		= pte->pfn;
+		__entry->eaddr		= pte->pte.eaddr;
+		__entry->vpage		= pte->pte.vpage;
+		__entry->raddr		= pte->pte.raddr;
+		__entry->flags		= (pte->pte.may_read ? 0x4 : 0) |
+					  (pte->pte.may_write ? 0x2 : 0) |
+					  (pte->pte.may_execute ? 0x1 : 0);
+	),
+
+	TP_printk("Flush: hva=%llx pfn=%llx ea=%lx vp=%llx ra=%lx [%x]",
+		  __entry->host_vpn, __entry->pfn, __entry->eaddr,
+		  __entry->vpage, __entry->raddr, __entry->flags)
+);
+
+TRACE_EVENT(kvm_book3s_mmu_flush,
+	TP_PROTO(const char *type, struct kvm_vcpu *vcpu, unsigned long long p1,
+		 unsigned long long p2),
+	TP_ARGS(type, vcpu, p1, p2),
+
+	TP_STRUCT__entry(
+		__field(	int,			count		)
+		__field(	unsigned long long,	p1		)
+		__field(	unsigned long long,	p2		)
+		__field(	const char *,		type		)
+	),
+
+	TP_fast_assign(
+		__entry->count		= to_book3s(vcpu)->hpte_cache_count;
+		__entry->p1		= p1;
+		__entry->p2		= p2;
+		__entry->type		= type;
+	),
+
+	TP_printk("Flush %d %sPTEs: %llx - %llx",
+		  __entry->count, __entry->type, __entry->p1, __entry->p2)
+);
+
+TRACE_EVENT(kvm_book3s_slb_found,
+	TP_PROTO(unsigned long long gvsid, unsigned long long hvsid),
+	TP_ARGS(gvsid, hvsid),
+
+	TP_STRUCT__entry(
+		__field(	unsigned long long,	gvsid		)
+		__field(	unsigned long long,	hvsid		)
+	),
+
+	TP_fast_assign(
+		__entry->gvsid		= gvsid;
+		__entry->hvsid		= hvsid;
+	),
+
+	TP_printk("%llx -> %llx", __entry->gvsid, __entry->hvsid)
+);
+
+TRACE_EVENT(kvm_book3s_slb_fail,
+	TP_PROTO(u16 sid_map_mask, unsigned long long gvsid),
+	TP_ARGS(sid_map_mask, gvsid),
+
+	TP_STRUCT__entry(
+		__field(	unsigned short,		sid_map_mask	)
+		__field(	unsigned long long,	gvsid		)
+	),
+
+	TP_fast_assign(
+		__entry->sid_map_mask	= sid_map_mask;
+		__entry->gvsid		= gvsid;
+	),
+
+	TP_printk("%x/%x: %llx", __entry->sid_map_mask,
+		  SID_MAP_MASK - __entry->sid_map_mask, __entry->gvsid)
+);
+
+TRACE_EVENT(kvm_book3s_slb_map,
+	TP_PROTO(u16 sid_map_mask, unsigned long long gvsid,
+		 unsigned long long hvsid),
+	TP_ARGS(sid_map_mask, gvsid, hvsid),
+
+	TP_STRUCT__entry(
+		__field(	unsigned short,		sid_map_mask	)
+		__field(	unsigned long long,	guest_vsid	)
+		__field(	unsigned long long,	host_vsid	)
+	),
+
+	TP_fast_assign(
+		__entry->sid_map_mask	= sid_map_mask;
+		__entry->guest_vsid	= gvsid;
+		__entry->host_vsid	= hvsid;
+	),
+
+	TP_printk("%x: %llx -> %llx", __entry->sid_map_mask,
+		  __entry->guest_vsid, __entry->host_vsid)
+);
+
+TRACE_EVENT(kvm_book3s_slbmte,
+	TP_PROTO(u64 slb_vsid, u64 slb_esid),
+	TP_ARGS(slb_vsid, slb_esid),
+
+	TP_STRUCT__entry(
+		__field(	u64,	slb_vsid	)
+		__field(	u64,	slb_esid	)
+	),
+
+	TP_fast_assign(
+		__entry->slb_vsid	= slb_vsid;
+		__entry->slb_esid	= slb_esid;
+	),
+
+	TP_printk("%llx, %llx", __entry->slb_vsid, __entry->slb_esid)
+);
+
+TRACE_EVENT(kvm_exit,
+	TP_PROTO(unsigned int exit_nr, struct kvm_vcpu *vcpu),
+	TP_ARGS(exit_nr, vcpu),
+
+	TP_STRUCT__entry(
+		__field(	unsigned int,	exit_nr		)
+		__field(	unsigned long,	pc		)
+		__field(	unsigned long,	msr		)
+		__field(	unsigned long,	dar		)
+		__field(	unsigned long,	srr1		)
+		__field(	unsigned long,	last_inst	)
+	),
+
+	TP_fast_assign(
+		__entry->exit_nr	= exit_nr;
+		__entry->pc		= kvmppc_get_pc(vcpu);
+		__entry->dar		= kvmppc_get_fault_dar(vcpu);
+		__entry->msr		= vcpu->arch.shared->msr;
+		__entry->srr1		= vcpu->arch.shadow_srr1;
+		__entry->last_inst	= vcpu->arch.last_inst;
+	),
+
+	TP_printk("exit=%s"
+		" | pc=0x%lx"
+		" | msr=0x%lx"
+		" | dar=0x%lx"
+		" | srr1=0x%lx"
+		" | last_inst=0x%lx"
+		,
+		__print_symbolic(__entry->exit_nr, kvm_trace_symbol_exit),
+		__entry->pc,
+		__entry->msr,
+		__entry->dar,
+		__entry->srr1,
+		__entry->last_inst
+		)
+);
+
+TRACE_EVENT(kvm_unmap_hva,
+	TP_PROTO(unsigned long hva),
+	TP_ARGS(hva),
+
+	TP_STRUCT__entry(
+		__field(	unsigned long,	hva		)
+	),
+
+	TP_fast_assign(
+		__entry->hva		= hva;
+	),
+
+	TP_printk("unmap hva 0x%lx\n", __entry->hva)
+);
+
+#endif /* _TRACE_KVM_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
-- 
1.8.1.2

[RFC PATCH 08/11] kvm: powerpc: book3s: Support building HV and PR KVM as module

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/kvm/Kconfig              |  6 +++---
 arch/powerpc/kvm/Makefile             | 12 ++++++++----
 arch/powerpc/kvm/book3s.c             | 19 ++++++++++++++++++-
 arch/powerpc/kvm/book3s_64_mmu_host.c |  1 +
 arch/powerpc/kvm/book3s_emulate.c     |  2 +-
 arch/powerpc/kvm/book3s_hv.c          |  4 ++++
 arch/powerpc/kvm/book3s_pr.c          |  5 ++++-
 arch/powerpc/kvm/book3s_rtas.c        |  1 +
 arch/powerpc/kvm/emulate.c            |  1 +
 arch/powerpc/kvm/powerpc.c            |  6 ++++++
 virt/kvm/kvm_main.c                   |  3 +++
 11 files changed, 50 insertions(+), 10 deletions(-)

diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 5c62459..f63fd8f 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -73,7 +73,7 @@ config KVM_BOOK3S_64
 	  If unsure, say N.
 
 config KVM_BOOK3S_64_HV
-	bool "KVM support for POWER7 and PPC970 using hypervisor mode in host"
+	tristate "KVM support for POWER7 and PPC970 using hypervisor mode in host"
 	depends on KVM_BOOK3S_64
 	select KVM_BOOK3S_HV
 	select MMU_NOTIFIER
@@ -94,8 +94,8 @@ config KVM_BOOK3S_64_HV
 	  If unsure, say N.
 
 config KVM_BOOK3S_64_PR
-	bool "KVM support without using hypervisor mode in host"
-	depends on KVM_BOOK3S_64 && !KVM_BOOK3S_64_HV
+	tristate "KVM support without using hypervisor mode in host"
+	depends on KVM_BOOK3S_64
 	select KVM_BOOK3S_PR
 	---help---
 	  Support running guest kernels in virtual machines on processors
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index a514ecd..861b8da 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -56,8 +56,7 @@ kvm-objs-$(CONFIG_KVM_E500MC) := $(kvm-e500mc-objs)
 kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) := \
 	book3s_64_vio_hv.o
 
-kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_PR) := \
-	$(KVM)/coalesced_mmio.o \
+kvm-pr-y := \
 	fpu.o \
 	book3s_paired_singles.o \
 	book3s_pr.o \
@@ -77,7 +76,7 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
 	book3s_rmhandlers.o
 endif
 
-kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  += \
+kvm-hv-y += \
 	book3s_hv.o \
 	book3s_hv_interrupts.o \
 	book3s_64_mmu_hv.o
@@ -85,13 +84,15 @@ kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  += \
 kvm-book3s_64-builtin-xics-objs-$(CONFIG_KVM_XICS) := \
 	book3s_hv_rm_xics.o
 
-kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HV) += \
+ifdef CONFIG_KVM_BOOK3S_HV
+kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
 	book3s_hv_rmhandlers.o \
 	book3s_hv_rm_mmu.o \
 	book3s_hv_ras.o \
 	book3s_hv_builtin.o \
 	book3s_hv_cma.o \
 	$(kvm-book3s_64-builtin-xics-objs-y)
+endif
 
 kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
 	book3s_xics.o
@@ -132,4 +133,7 @@ obj-$(CONFIG_KVM_E500MC) += kvm.o
 obj-$(CONFIG_KVM_BOOK3S_64) += kvm.o
 obj-$(CONFIG_KVM_BOOK3S_32) += kvm.o
 
+obj-$(CONFIG_KVM_BOOK3S_64_PR) += kvm-pr.o
+obj-$(CONFIG_KVM_BOOK3S_64_HV) += kvm-hv.o
+
 obj-y += $(kvm-book3s_64-builtin-objs-y)
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index 12f94bf..ca617e1 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -170,13 +170,14 @@ void kvmppc_book3s_queue_irqprio(struct kvm_vcpu *vcpu, unsigned int vec)
 	printk(KERN_INFO "Queueing interrupt %x\n", vec);
 #endif
 }
-
+EXPORT_SYMBOL_GPL(kvmppc_book3s_queue_irqprio);
 
 void kvmppc_core_queue_program(struct kvm_vcpu *vcpu, ulong flags)
 {
 	/* might as well deliver this straight away */
 	kvmppc_inject_interrupt(vcpu, BOOK3S_INTERRUPT_PROGRAM, flags);
 }
+EXPORT_SYMBOL_GPL(kvmppc_core_queue_program);
 
 void kvmppc_core_queue_dec(struct kvm_vcpu *vcpu)
 {
@@ -351,6 +352,7 @@ pfn_t kvmppc_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn)
 
 	return gfn_to_pfn(vcpu->kvm, gfn);
 }
+EXPORT_SYMBOL_GPL(kvmppc_gfn_to_pfn);
 
 static int kvmppc_xlate(struct kvm_vcpu *vcpu, ulong eaddr, bool data,
 			 struct kvmppc_pte *pte)
@@ -418,6 +420,7 @@ int kvmppc_st(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr,
 
 	return EMULATE_DONE;
 }
+EXPORT_SYMBOL_GPL(kvmppc_st);
 
 int kvmppc_ld(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr,
 		      bool data)
@@ -448,6 +451,7 @@ nopte:
 mmio:
 	return EMULATE_DO_MMIO;
 }
+EXPORT_SYMBOL_GPL(kvmppc_ld);
 
 int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu)
 {
@@ -693,6 +697,7 @@ void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)
 {
 	kvmppc_ops->set_msr(vcpu, msr);
 }
+EXPORT_SYMBOL_GPL(kvmppc_set_msr);
 
 int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 {
@@ -774,6 +779,7 @@ int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
 {
 	return kvmppc_ops->unmap_hva(kvm, hva);
 }
+EXPORT_SYMBOL_GPL(kvm_unmap_hva);
 
 int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)
 {
@@ -825,3 +831,14 @@ int kvmppc_core_check_processor_compat(void)
 {
 	return kvmppc_ops->check_processor_compat();
 }
+
+EXPORT_SYMBOL_GPL(kvm_get_dirty_log);
+EXPORT_SYMBOL_GPL(kvmppc_core_pending_dec);
+EXPORT_SYMBOL_GPL(kvmppc_init_lpid);
+EXPORT_SYMBOL_GPL(kvmppc_core_dequeue_dec);
+EXPORT_SYMBOL_GPL(kvmppc_claim_lpid);
+EXPORT_SYMBOL_GPL(kvmppc_alloc_lpid);
+EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter);
+EXPORT_SYMBOL_GPL(kvmppc_core_queue_dec);
+EXPORT_SYMBOL_GPL(kvmppc_free_lpid);
+
diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index fd5b393..775d368 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -27,6 +27,7 @@
 #include <asm/machdep.h>
 #include <asm/mmu_context.h>
 #include <asm/hw_irq.h>
+
 #include "trace_pr.h"
 
 #define PTE_SIZE 12
diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c
index b9841ad..20d03c2 100644
--- a/arch/powerpc/kvm/book3s_emulate.c
+++ b/arch/powerpc/kvm/book3s_emulate.c
@@ -172,7 +172,7 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
 			vcpu->arch.mmu.tlbie(vcpu, addr, large);
 			break;
 		}
-#ifdef CONFIG_KVM_BOOK3S_64_PR
+#ifdef CONFIG_KVM_BOOK3S_PR
 		case OP_31_XOP_FAKE_SC1:
 		{
 			/* SC 1 papr hypercalls */
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index eec353d..ff57be8 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -53,6 +53,8 @@
 #include <linux/highmem.h>
 #include <linux/hugetlb.h>
 
+#include <linux/module.h>
+
 /* #define EXIT_DEBUG */
 /* #define EXIT_DEBUG_SIMPLE */
 /* #define EXIT_DEBUG_INT */
@@ -2104,3 +2106,5 @@ static void kvmppc_book3s_exit_hv(void)
 
 module_init(kvmppc_book3s_init_hv);
 module_exit(kvmppc_book3s_exit_hv);
+
+MODULE_LICENSE("GPL");
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 99d0839..df48d89 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -40,6 +40,7 @@
 #include <linux/sched.h>
 #include <linux/vmalloc.h>
 #include <linux/highmem.h>
+#include <linux/module.h>
 
 #define CREATE_TRACE_POINTS
 #include "trace_pr.h"
@@ -871,7 +872,7 @@ program_interrupt:
 			ulong cmd = kvmppc_get_gpr(vcpu, 3);
 			int i;
 
-#ifdef CONFIG_KVM_BOOK3S_64_PR
+#ifdef CONFIG_PPC_BOOK3S_64
 			if (kvmppc_h_pr(vcpu, cmd) == EMULATE_DONE) {
 				r = RESUME_GUEST;
 				break;
@@ -1567,3 +1568,5 @@ static void kvmppc_book3s_exit_pr(void)
 
 module_init(kvmppc_book3s_init_pr);
 module_exit(kvmppc_book3s_exit_pr);
+
+MODULE_LICENSE("GPL");
diff --git a/arch/powerpc/kvm/book3s_rtas.c b/arch/powerpc/kvm/book3s_rtas.c
index 3219ba8..cf95cde 100644
--- a/arch/powerpc/kvm/book3s_rtas.c
+++ b/arch/powerpc/kvm/book3s_rtas.c
@@ -260,6 +260,7 @@ fail:
 	 */
 	return rc;
 }
+EXPORT_SYMBOL_GPL(kvmppc_rtas_hcall);
 
 void kvmppc_rtas_tokens_free(struct kvm *kvm)
 {
diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
index 81762eb..de9a340 100644
--- a/arch/powerpc/kvm/emulate.c
+++ b/arch/powerpc/kvm/emulate.c
@@ -483,3 +483,4 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
 
 	return emulated;
 }
+EXPORT_SYMBOL_GPL(kvmppc_emulate_instruction);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 00a96fc..3019edc 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -126,6 +126,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvmppc_prepare_to_enter);
 
 int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
 {
@@ -179,6 +180,7 @@ int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
 
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvmppc_kvm_pv);
 
 int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
 {
@@ -207,6 +209,7 @@ out:
 	vcpu->arch.sane = r;
 	return r ? 0 : -EINVAL;
 }
+EXPORT_SYMBOL_GPL(kvmppc_sanity_check);
 
 int kvmppc_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu)
 {
@@ -241,6 +244,7 @@ int kvmppc_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu)
 
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvmppc_emulate_mmio);
 
 int kvm_arch_hardware_enable(void *garbage)
 {
@@ -665,6 +669,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
 
 	return EMULATE_DO_MMIO;
 }
+EXPORT_SYMBOL_GPL(kvmppc_handle_load);
 
 /* Same as above, but sign extends */
 int kvmppc_handle_loads(struct kvm_run *run, struct kvm_vcpu *vcpu,
@@ -726,6 +731,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct kvm_vcpu *vcpu,
 
 	return EMULATE_DO_MMIO;
 }
+EXPORT_SYMBOL_GPL(kvmppc_handle_store);
 
 int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 {
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index bf040c4..66df1d2 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -186,6 +186,7 @@ void kvm_flush_remote_tlbs(struct kvm *kvm)
 		++kvm->stat.remote_tlb_flush;
 	cmpxchg(&kvm->tlbs_dirty, dirty_count, 0);
 }
+EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs);
 
 void kvm_reload_remote_mmus(struct kvm *kvm)
 {
@@ -1648,6 +1649,7 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn)
 	memslot = gfn_to_memslot(kvm, gfn);
 	mark_page_dirty_in_slot(kvm, memslot, gfn);
 }
+EXPORT_SYMBOL_GPL(mark_page_dirty);
 
 /*
  * The vCPU has executed a HLT instruction with in-kernel mode enabled.
@@ -1673,6 +1675,7 @@ void kvm_vcpu_block(struct kvm_vcpu *vcpu)
 
 	finish_wait(&vcpu->wq, &wait);
 }
+EXPORT_SYMBOL_GPL(kvm_vcpu_block);
 
 #ifndef CONFIG_S390
 /*
-- 
1.8.1.2

[RFC PATCH 09/11] kvm: simplify processor compat check

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/arm/kvm/arm.c                 |  4 ++--
 arch/ia64/kvm/kvm-ia64.c           |  4 ++--
 arch/mips/kvm/kvm_mips.c           |  6 ++----
 arch/powerpc/include/asm/kvm_ppc.h |  2 +-
 arch/powerpc/kvm/44x.c             |  2 +-
 arch/powerpc/kvm/book3s.c          | 15 ++++++++++++---
 arch/powerpc/kvm/book3s_hv.c       |  9 ++++++---
 arch/powerpc/kvm/book3s_pr.c       |  5 +++--
 arch/powerpc/kvm/e500.c            |  2 +-
 arch/powerpc/kvm/e500mc.c          |  2 +-
 arch/powerpc/kvm/powerpc.c         |  5 -----
 arch/s390/kvm/kvm-s390.c           |  3 ++-
 arch/x86/kvm/x86.c                 | 13 +++++++++++--
 include/linux/kvm_host.h           |  2 +-
 virt/kvm/kvm_main.c                | 14 +++++---------
 15 files changed, 50 insertions(+), 38 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 9c697db..cccb121 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -109,9 +109,9 @@ void kvm_arch_hardware_unsetup(void)
 {
 }
 
-void kvm_arch_check_processor_compat(void *rtn)
+int kvm_arch_check_processor_compat(void *opaque)
 {
-	*(int *)rtn = 0;
+	return 0;
 }
 
 void kvm_arch_sync_events(struct kvm *kvm)
diff --git a/arch/ia64/kvm/kvm-ia64.c b/arch/ia64/kvm/kvm-ia64.c
index bdfd878..065942c 100644
--- a/arch/ia64/kvm/kvm-ia64.c
+++ b/arch/ia64/kvm/kvm-ia64.c
@@ -185,9 +185,9 @@ void kvm_arch_hardware_disable(void *garbage)
 	ia64_ptr_entry(0x3, slot);
 }
 
-void kvm_arch_check_processor_compat(void *rtn)
+int kvm_arch_check_processor_compat(void *opaque)
 {
-	*(int *)rtn = 0;
+	return 0;
 }
 
 int kvm_dev_ioctl_check_extension(long ext)
diff --git a/arch/mips/kvm/kvm_mips.c b/arch/mips/kvm/kvm_mips.c
index a7b0445..4512739 100644
--- a/arch/mips/kvm/kvm_mips.c
+++ b/arch/mips/kvm/kvm_mips.c
@@ -97,11 +97,9 @@ void kvm_arch_hardware_unsetup(void)
 {
 }
 
-void kvm_arch_check_processor_compat(void *rtn)
+int kvm_arch_check_processor_compat(void *opaque)
 {
-	int *r = (int *)rtn;
-	*r = 0;
-	return;
+	return 0;
 }
 
 static void kvm_mips_init_tlbs(struct kvm *kvm)
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 58e732f..592501b 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -204,7 +204,7 @@ struct kvmppc_ops {
 			      unsigned long npages);
 	int (*init_vm)(struct kvm *kvm);
 	void (*destroy_vm)(struct kvm *kvm);
-	int (*check_processor_compat)(void);
+	void (*check_processor_compat)(void *r);
 	int (*get_smmu_info)(struct kvm *kvm, struct kvm_ppc_smmu_info *info);
 	int (*emulate_op)(struct kvm_run *run, struct kvm_vcpu *vcpu,
 			  unsigned int inst, int *advance);
diff --git a/arch/powerpc/kvm/44x.c b/arch/powerpc/kvm/44x.c
index 2f5c6b6..a1f4e60 100644
--- a/arch/powerpc/kvm/44x.c
+++ b/arch/powerpc/kvm/44x.c
@@ -43,7 +43,7 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
 	kvmppc_booke_vcpu_put(vcpu);
 }
 
-int kvmppc_core_check_processor_compat(void)
+int kvm_arch_check_processor_compat(void *opaque)
 {
 	int r;
 
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index ca617e1..485a6ff 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -827,9 +827,18 @@ void kvmppc_core_destroy_vm(struct kvm *kvm)
 #endif
 }
 
-int kvmppc_core_check_processor_compat(void)
-{
-	return kvmppc_ops->check_processor_compat();
+int kvm_arch_check_processor_compat(void *opaque)
+{
+	int r,cpu;
+	struct kvmppc_ops *kvm_ops = (struct kvmppc_ops *)opaque;
+	for_each_online_cpu(cpu) {
+		smp_call_function_single(cpu,
+					 kvm_ops->check_processor_compat,
+					 &r, 1);
+		if (r < 0)
+			break;
+	}
+	return r;
 }
 
 EXPORT_SYMBOL_GPL(kvm_get_dirty_log);
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index ff57be8..4322db4 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -1980,11 +1980,14 @@ static int kvmppc_core_emulate_mfspr_hv(struct kvm_vcpu *vcpu, int sprn,
 	return EMULATE_FAIL;
 }
 
-static int kvmppc_core_check_processor_compat_hv(void)
+
+static void kvmppc_core_check_processor_compat_hv(void *r)
 {
 	if (!cpu_has_feature(CPU_FTR_HVMODE))
-		return -EIO;
-	return 0;
+		*(int *)r = -EIO;
+	else
+		*(int *)r = 0;
+	return;
 }
 
 static long kvm_arch_vm_ioctl_hv(struct file *filp,
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index df48d89..127b961 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -1490,10 +1490,11 @@ static void kvmppc_core_destroy_vm_pr(struct kvm *kvm)
 	enable_relon_interrupts(kvm);
 }
 
-static int kvmppc_core_check_processor_compat_pr(void)
+static void kvmppc_core_check_processor_compat_pr(void *r)
 {
 	/* we are always compatible */
-	return 0;
+	*(int *)r = 0;
+	return;
 }
 
 static long kvm_arch_vm_ioctl_pr(struct file *filp,
diff --git a/arch/powerpc/kvm/e500.c b/arch/powerpc/kvm/e500.c
index ce6b73c..0681cb1 100644
--- a/arch/powerpc/kvm/e500.c
+++ b/arch/powerpc/kvm/e500.c
@@ -323,7 +323,7 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
 	kvmppc_booke_vcpu_put(vcpu);
 }
 
-int kvmppc_core_check_processor_compat(void)
+int kvm_arch_check_processor_compat(void *opaque)
 {
 	int r;
 
diff --git a/arch/powerpc/kvm/e500mc.c b/arch/powerpc/kvm/e500mc.c
index 19c8379..48b3ba5 100644
--- a/arch/powerpc/kvm/e500mc.c
+++ b/arch/powerpc/kvm/e500mc.c
@@ -169,7 +169,7 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
 	kvmppc_booke_vcpu_put(vcpu);
 }
 
-int kvmppc_core_check_processor_compat(void)
+int kvm_arch_check_processor_compat(void *opaque)
 {
 	int r;
 
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 3019edc..1404f4d 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -264,11 +264,6 @@ void kvm_arch_hardware_unsetup(void)
 {
 }
 
-void kvm_arch_check_processor_compat(void *rtn)
-{
-	*(int *)rtn = kvmppc_core_check_processor_compat();
-}
-
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
 	if (type)
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 776dafe..01d4000 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -119,8 +119,9 @@ void kvm_arch_hardware_unsetup(void)
 	gmap_unregister_ipte_notifier(&gmap_notifier);
 }
 
-void kvm_arch_check_processor_compat(void *rtn)
+int kvm_arch_check_processor_compat(void *opaque)
 {
+	return 0;
 }
 
 int kvm_arch_init(void *opaque)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e5ca72a..53c8308 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6873,9 +6873,18 @@ void kvm_arch_hardware_unsetup(void)
 	kvm_x86_ops->hardware_unsetup();
 }
 
-void kvm_arch_check_processor_compat(void *rtn)
+int kvm_arch_check_processor_compat(void *opaque)
 {
-	kvm_x86_ops->check_processor_compatibility(rtn);
+	int r,cpu;
+
+	for_each_online_cpu(cpu) {
+		smp_call_function_single(cpu,
+			 kvm_x86_ops->check_processor_compatibility,
+			 &r, 1);
+		if (r < 0)
+			break;
+	}
+	return r;
 }
 
 bool kvm_vcpu_compatible(struct kvm_vcpu *vcpu)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index ca645a0..459b359 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -649,7 +649,7 @@ int kvm_arch_hardware_enable(void *garbage);
 void kvm_arch_hardware_disable(void *garbage);
 int kvm_arch_hardware_setup(void);
 void kvm_arch_hardware_unsetup(void);
-void kvm_arch_check_processor_compat(void *rtn);
+int kvm_arch_check_processor_compat(void *opaque);
 int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 66df1d2..0594b22 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -3166,10 +3166,9 @@ static void kvm_sched_out(struct preempt_notifier *pn,
 }
 
 int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
-		  struct module *module)
+	     struct module *module)
 {
 	int r;
-	int cpu;
 
 	r = kvm_arch_init(opaque);
 	if (r)
@@ -3195,13 +3194,10 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 	if (r < 0)
 		goto out_free_0a;
 
-	for_each_online_cpu(cpu) {
-		smp_call_function_single(cpu,
-				kvm_arch_check_processor_compat,
-				&r, 1);
-		if (r < 0)
-			goto out_free_1;
-	}
+
+	r = kvm_arch_check_processor_compat(opaque);
+	if (r < 0)
+		goto out_free_1;
 
 	r = register_cpu_notifier(&kvm_cpu_notifier);
 	if (r)
-- 
1.8.1.2

[RFC PATCH 10/11] kvm: powerpc: book3s: Allow the HV and PR selection per virtual machine

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

This moves the kvmppc_ops callbacks to be a per VM entity. This
enables us to select HV and PR mode when creating a VM

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_host.h |  3 ++
 arch/powerpc/include/asm/kvm_ppc.h  | 11 ++++--
 arch/powerpc/kvm/book3s.c           | 60 +++++++++++++++---------------
 arch/powerpc/kvm/book3s_hv.c        | 18 +++++----
 arch/powerpc/kvm/book3s_pr.c        | 18 ++++++---
 arch/powerpc/kvm/emulate.c          | 11 +++---
 arch/powerpc/kvm/powerpc.c          | 74 ++++++++++++++++++++++++-------------
 include/linux/kvm_host.h            |  7 ++--
 include/uapi/linux/kvm.h            |  4 ++
 virt/kvm/kvm_main.c                 | 18 ++++-----
 10 files changed, 135 insertions(+), 89 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 283e52e..61a297fc 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -276,6 +276,9 @@ struct kvm_arch {
 #ifdef CONFIG_KVM_XICS
 	struct kvmppc_xics *xics;
 #endif
+#ifdef CONFIG_PPC_BOOK3S_64
+	struct kvmppc_ops *kvm_ops;
+#endif
 };
 
 /*
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 592501b..a4a5893 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -134,9 +134,11 @@ extern struct page *kvm_alloc_hpt(unsigned long nr_pages);
 extern void kvm_release_hpt(struct page *page, unsigned long nr_pages);
 extern int kvmppc_core_init_vm(struct kvm *kvm);
 extern void kvmppc_core_destroy_vm(struct kvm *kvm);
-extern void kvmppc_core_free_memslot(struct kvm_memory_slot *free,
+extern void kvmppc_core_free_memslot(struct kvm *kvm,
+				     struct kvm_memory_slot *free,
 				     struct kvm_memory_slot *dont);
-extern int kvmppc_core_create_memslot(struct kvm_memory_slot *slot,
+extern int kvmppc_core_create_memslot(struct kvm *kvm,
+				      struct kvm_memory_slot *slot,
 				      unsigned long npages);
 extern int kvmppc_core_prepare_memory_region(struct kvm *kvm,
 				struct kvm_memory_slot *memslot,
@@ -216,7 +218,8 @@ struct kvmppc_ops {
 
 };
 
-extern struct kvmppc_ops *kvmppc_ops;
+extern struct kvmppc_ops *kvmppc_hv_ops;
+extern struct kvmppc_ops *kvmppc_pr_ops;
 
 /*
  * Cuts out inst bits with ordering according to spec.
@@ -324,7 +327,7 @@ static inline void kvmppc_set_host_ipi(int cpu, u8 host_ipi)
 
 static inline void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu)
 {
-	kvmppc_ops->fast_vcpu_kick(vcpu);
+	vcpu->kvm->arch.kvm_ops->fast_vcpu_kick(vcpu);
 }
 
 #else
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index 485a6ff..34e189c 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -71,7 +71,7 @@ void kvmppc_core_load_guest_debugstate(struct kvm_vcpu *vcpu)
 
 static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
 {
-	if (!kvmppc_ops->is_hv_enabled)
+	if (!vcpu->kvm->arch.kvm_ops->is_hv_enabled)
 		return to_book3s(vcpu)->hior;
 	return 0;
 }
@@ -79,7 +79,7 @@ static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
 static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
 			unsigned long pending_now, unsigned long old_pending)
 {
-	if (kvmppc_ops->is_hv_enabled)
+	if (vcpu->kvm->arch.kvm_ops->is_hv_enabled)
 		return;
 	if (pending_now)
 		vcpu->arch.shared->int_pending = 1;
@@ -93,7 +93,7 @@ static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
 	ulong crit_r1;
 	bool crit;
 
-	if (kvmppc_ops->is_hv_enabled)
+	if (vcpu->kvm->arch.kvm_ops->is_hv_enabled)
 		return false;
 
 	crit_raw = vcpu->arch.shared->critical;
@@ -470,13 +470,13 @@ void kvmppc_subarch_vcpu_uninit(struct kvm_vcpu *vcpu)
 int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
 				  struct kvm_sregs *sregs)
 {
-	return kvmppc_ops->get_sregs(vcpu, sregs);
+	return vcpu->kvm->arch.kvm_ops->get_sregs(vcpu, sregs);
 }
 
 int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 				  struct kvm_sregs *sregs)
 {
-	return kvmppc_ops->set_sregs(vcpu, sregs);
+	return vcpu->kvm->arch.kvm_ops->set_sregs(vcpu, sregs);
 }
 
 int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
@@ -555,7 +555,7 @@ int kvm_vcpu_ioctl_get_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 	if (size > sizeof(val))
 		return -EINVAL;
 
-	r = kvmppc_ops->get_one_reg(vcpu, reg->id, &val);
+	r = vcpu->kvm->arch.kvm_ops->get_one_reg(vcpu, reg->id, &val);
 	if (r == -EINVAL) {
 		r = 0;
 		switch (reg->id) {
@@ -631,7 +631,7 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 	if (copy_from_user(&val, (char __user *)(unsigned long)reg->addr, size))
 		return -EFAULT;
 
-	r = kvmppc_ops->set_one_reg(vcpu, reg->id, &val);
+	r = vcpu->kvm->arch.kvm_ops->set_one_reg(vcpu, reg->id, &val);
 	if (r == -EINVAL) {
 		r = 0;
 		switch (reg->id) {
@@ -685,23 +685,23 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 
 void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 {
-	kvmppc_ops->vcpu_load(vcpu, cpu);
+	vcpu->kvm->arch.kvm_ops->vcpu_load(vcpu, cpu);
 }
 
 void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
 {
-	kvmppc_ops->vcpu_put(vcpu);
+	vcpu->kvm->arch.kvm_ops->vcpu_put(vcpu);
 }
 
 void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)
 {
-	kvmppc_ops->set_msr(vcpu, msr);
+	vcpu->kvm->arch.kvm_ops->set_msr(vcpu, msr);
 }
 EXPORT_SYMBOL_GPL(kvmppc_set_msr);
 
 int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 {
-	return kvmppc_ops->vcpu_run(kvm_run, vcpu);
+	return vcpu->kvm->arch.kvm_ops->vcpu_run(kvm_run, vcpu);
 }
 
 int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu,
@@ -726,84 +726,84 @@ void kvmppc_decrementer_func(unsigned long data)
 
 struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
 {
-	return kvmppc_ops->vcpu_create(kvm, id);
+	return kvm->arch.kvm_ops->vcpu_create(kvm, id);
 }
 
 void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
 {
-	kvmppc_ops->vcpu_free(vcpu);
+	vcpu->kvm->arch.kvm_ops->vcpu_free(vcpu);
 }
 
 int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
 {
-	return kvmppc_ops->check_requests(vcpu);
+	return vcpu->kvm->arch.kvm_ops->check_requests(vcpu);
 }
 
 int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
 {
-	return kvmppc_ops->get_dirty_log(kvm, log);
+	return kvm->arch.kvm_ops->get_dirty_log(kvm, log);
 }
 
-void kvmppc_core_free_memslot(struct kvm_memory_slot *free,
+void kvmppc_core_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 			      struct kvm_memory_slot *dont)
 {
-	kvmppc_ops->free_memslot(free, dont);
+	kvm->arch.kvm_ops->free_memslot(free, dont);
 }
 
-int kvmppc_core_create_memslot(struct kvm_memory_slot *slot,
+int kvmppc_core_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 			       unsigned long npages)
 {
-	return kvmppc_ops->create_memslot(slot, npages);
+	return kvm->arch.kvm_ops->create_memslot(slot, npages);
 }
 
 void kvmppc_core_flush_memslot(struct kvm *kvm, struct kvm_memory_slot *memslot)
 {
-	kvmppc_ops->flush_memslot(kvm, memslot);
+	kvm->arch.kvm_ops->flush_memslot(kvm, memslot);
 }
 
 int kvmppc_core_prepare_memory_region(struct kvm *kvm,
 				struct kvm_memory_slot *memslot,
 				struct kvm_userspace_memory_region *mem)
 {
-	return kvmppc_ops->prepare_memory_region(kvm, memslot, mem);
+	return kvm->arch.kvm_ops->prepare_memory_region(kvm, memslot, mem);
 }
 
 void kvmppc_core_commit_memory_region(struct kvm *kvm,
 				struct kvm_userspace_memory_region *mem,
 				const struct kvm_memory_slot *old)
 {
-	kvmppc_ops->commit_memory_region(kvm, mem, old);
+	kvm->arch.kvm_ops->commit_memory_region(kvm, mem, old);
 }
 
 int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
 {
-	return kvmppc_ops->unmap_hva(kvm, hva);
+	return kvm->arch.kvm_ops->unmap_hva(kvm, hva);
 }
 EXPORT_SYMBOL_GPL(kvm_unmap_hva);
 
 int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)
 {
-	return kvmppc_ops->unmap_hva_range(kvm, start, end);
+	return kvm->arch.kvm_ops->unmap_hva_range(kvm, start, end);
 }
 
 int kvm_age_hva(struct kvm *kvm, unsigned long hva)
 {
-	return kvmppc_ops->age_hva(kvm, hva);
+	return kvm->arch.kvm_ops->age_hva(kvm, hva);
 }
 
 int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
 {
-	return kvmppc_ops->test_age_hva(kvm, hva);
+	return kvm->arch.kvm_ops->test_age_hva(kvm, hva);
 }
 
 void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte)
 {
-	kvmppc_ops->set_spte_hva(kvm, hva, pte);
+	kvm->arch.kvm_ops->set_spte_hva(kvm, hva, pte);
 }
 
 void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
 {
-	kvmppc_ops->mmu_destroy(vcpu);
+	vcpu->kvm->arch.kvm_ops->mmu_destroy(vcpu);
 }
 
 int kvmppc_core_init_vm(struct kvm *kvm)
@@ -814,12 +814,12 @@ int kvmppc_core_init_vm(struct kvm *kvm)
 	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
 #endif
 
-	return kvmppc_ops->init_vm(kvm);
+	return kvm->arch.kvm_ops->init_vm(kvm);
 }
 
 void kvmppc_core_destroy_vm(struct kvm *kvm)
 {
-	kvmppc_ops->destroy_vm(kvm);
+	kvm->arch.kvm_ops->destroy_vm(kvm);
 
 #ifdef CONFIG_PPC64
 	kvmppc_rtas_tokens_free(kvm);
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 4322db4..0a684a7 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2052,7 +2052,7 @@ extern int kvm_age_hva_hv(struct kvm *kvm, unsigned long hva);
 extern int kvm_test_age_hva_hv(struct kvm *kvm, unsigned long hva);
 extern void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, pte_t pte);
 
-static struct kvmppc_ops kvmppc_hv_ops = {
+static struct kvmppc_ops kvm_ops_hv = {
 	.is_hv_enabled = true,
 	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv,
 	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
@@ -2092,11 +2092,13 @@ static int kvmppc_book3s_init_hv(void)
 {
 	int r;
 
-	r = kvm_init(&kvmppc_hv_ops, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
-
-	if (r)
-		return r;
-
+	if (!kvmppc_pr_ops) {
+		r = kvm_init(&kvm_ops_hv, sizeof(struct kvm_vcpu),
+			     0, THIS_MODULE);
+		if (r)
+			return r;
+	}
+	kvmppc_hv_ops = &kvm_ops_hv;
 	r = kvmppc_mmu_hv_init();
 
 	return r;
@@ -2104,7 +2106,9 @@ static int kvmppc_book3s_init_hv(void)
 
 static void kvmppc_book3s_exit_hv(void)
 {
-	kvm_exit();
+	kvmppc_hv_ops = NULL;
+	if (!kvmppc_pr_ops)
+		kvm_exit();
 }
 
 module_init(kvmppc_book3s_init_hv);
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 127b961..e49e4b0 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -1511,7 +1511,7 @@ extern int kvmppc_core_emulate_mtspr_pr(struct kvm_vcpu *vcpu,
 extern int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu,
 					int sprn, ulong *spr_val);
 
-static struct kvmppc_ops kvmppc_pr_ops = {
+static struct kvmppc_ops kvm_ops_pr = {
 	.is_hv_enabled = false,
 	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_pr,
 	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_pr,
@@ -1551,10 +1551,14 @@ static int kvmppc_book3s_init_pr(void)
 {
 	int r;
 
-	r = kvm_init(&kvmppc_pr_ops, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
-
-	if (r)
-		return r;
+	if (!kvmppc_hv_ops) {
+		r = kvm_init(&kvm_ops_pr, sizeof(struct kvm_vcpu),
+			     0, THIS_MODULE);
+		if (r)
+			return r;
+	}
+	/* Assign the global value */
+	kvmppc_pr_ops = &kvm_ops_pr;
 
 	r = kvmppc_mmu_hpte_sysinit();
 
@@ -1563,8 +1567,10 @@ static int kvmppc_book3s_init_pr(void)
 
 static void kvmppc_book3s_exit_pr(void)
 {
+	kvmppc_pr_ops = NULL;
 	kvmppc_mmu_hpte_sysexit();
-	kvm_exit();
+	if (!kvmppc_hv_ops)
+		kvm_exit();
 }
 
 module_init(kvmppc_book3s_init_pr);
diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
index de9a340..2f9a087 100644
--- a/arch/powerpc/kvm/emulate.c
+++ b/arch/powerpc/kvm/emulate.c
@@ -130,8 +130,8 @@ static int kvmppc_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs)
 	case SPRN_PIR: break;
 
 	default:
-		emulated = kvmppc_ops->emulate_mtspr(vcpu, sprn,
-						     spr_val);
+		emulated = vcpu->kvm->arch.kvm_ops->emulate_mtspr(vcpu, sprn,
+								  spr_val);
 		if (emulated == EMULATE_FAIL)
 			printk(KERN_INFO "mtspr: unknown spr "
 				"0x%x\n", sprn);
@@ -191,8 +191,8 @@ static int kvmppc_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, int rt)
 		spr_val = kvmppc_get_dec(vcpu, get_tb());
 		break;
 	default:
-		emulated = kvmppc_ops->emulate_mfspr(vcpu, sprn,
-						     &spr_val);
+		emulated = vcpu->kvm->arch.kvm_ops->emulate_mfspr(vcpu, sprn,
+								  &spr_val);
 		if (unlikely(emulated == EMULATE_FAIL)) {
 			printk(KERN_INFO "mfspr: unknown spr "
 				"0x%x\n", sprn);
@@ -464,7 +464,8 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
 	}
 
 	if (emulated == EMULATE_FAIL) {
-		emulated = kvmppc_ops->emulate_op(run, vcpu, inst, &advance);
+		emulated = vcpu->kvm->arch.kvm_ops->emulate_op(run, vcpu, inst,
+							       &advance);
 		if (emulated == EMULATE_AGAIN) {
 			advance = 0;
 		} else if (emulated == EMULATE_FAIL) {
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 1404f4d..1209229 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -39,7 +39,11 @@
 #define CREATE_TRACE_POINTS
 #include "trace.h"
 
-struct kvmppc_ops *kvmppc_ops;
+struct kvmppc_ops *kvmppc_hv_ops;
+struct kvmppc_ops *kvmppc_pr_ops;
+EXPORT_SYMBOL_GPL(kvmppc_hv_ops);
+EXPORT_SYMBOL_GPL(kvmppc_pr_ops);
+
 
 int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
 {
@@ -195,7 +199,7 @@ int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
 		goto out;
 
 	/* HV KVM can only do PAPR mode for now */
-	if (!vcpu->arch.papr_enabled && kvmppc_ops->is_hv_enabled)
+	if (!vcpu->arch.papr_enabled && vcpu->kvm->arch.kvm_ops->is_hv_enabled)
 		goto out;
 
 #ifdef CONFIG_KVM_BOOKE_HV
@@ -266,10 +270,28 @@ void kvm_arch_hardware_unsetup(void)
 
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
-	if (type)
-		return -EINVAL;
+	/*
+	 * if we have both HV and PR enabled, default is HV
+	 */
+	if (type == 0) {
+		if (kvmppc_hv_ops)
+			kvm->arch.kvm_ops = kvmppc_hv_ops;
+		else
+			kvm->arch.kvm_ops = kvmppc_pr_ops;
+	} else	if (type == KVM_VM_PPC_HV) {
+		if (!kvmppc_hv_ops)
+			goto err_out;
+		kvm->arch.kvm_ops = kvmppc_hv_ops;
+	} else if (type == KVM_VM_PPC_PR) {
+		if (!kvmppc_pr_ops)
+			goto err_out;
+		kvm->arch.kvm_ops = kvmppc_pr_ops;
+	} else
+		goto err_out;
 
 	return kvmppc_core_init_vm(kvm);
+err_out:
+	return -EINVAL;
 }
 
 void kvm_arch_destroy_vm(struct kvm *kvm)
@@ -295,9 +317,13 @@ void kvm_arch_sync_events(struct kvm *kvm)
 {
 }
 
-int kvm_dev_ioctl_check_extension(long ext)
+int kvm_dev_ioctl_check_extension(struct file *filp, long ext)
 {
 	int r;
+	/* FIXME!!
+	 * Should some of this be vm ioctl ? is it possible now ?
+	 */
+	int hv_enabled = kvmppc_hv_ops ? 1: 0;
 
 	switch (ext) {
 #ifdef CONFIG_BOOKE
@@ -324,7 +350,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 	case KVM_CAP_SW_TLB:
 #endif
 		/* We support this only for PR */
-		r = !kvmppc_ops->is_hv_enabled;
+		r = !hv_enabled;
 		break;
 #ifdef CONFIG_KVM_MMIO
 	case KVM_CAP_COALESCED_MMIO:
@@ -349,13 +375,13 @@ int kvm_dev_ioctl_check_extension(long ext)
 #endif /* CONFIG_PPC_BOOK3S_64 */
 #ifdef CONFIG_KVM_BOOK3S_HV
 	case KVM_CAP_PPC_SMT:
-		if (kvmppc_ops->is_hv_enabled)
+		if (hv_enabled)
 			r = threads_per_core;
 		else
 			r = 0;
 		break;
 	case KVM_CAP_PPC_RMA:
-		r = kvmppc_ops->is_hv_enabled;
+		r = hv_enabled;
 		/* PPC970 requires an RMA */
 		if (r && cpu_has_feature(CPU_FTR_ARCH_201))
 			r = 2;
@@ -363,7 +389,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 #endif
 	case KVM_CAP_SYNC_MMU:
 #ifdef CONFIG_KVM_BOOK3S_HV
-		if (kvmppc_ops->is_hv_enabled)
+		if (hv_enabled)
 			r = cpu_has_feature(CPU_FTR_ARCH_206) ? 1 : 0;
 		else
 			r = 0;
@@ -375,7 +401,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 		break;
 #ifdef CONFIG_KVM_BOOK3S_HV
 	case KVM_CAP_PPC_HTAB_FD:
-		r = kvmppc_ops->is_hv_enabled;
+		r = hv_enabled;
 		break;
 #endif
 	case KVM_CAP_NR_VCPUS:
@@ -385,7 +411,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 		 * will have secondary threads "offline"), and for other KVM
 		 * implementations just count online CPUs.
 		 */
-		if (kvmppc_ops->is_hv_enabled)
+		if (hv_enabled)
 			r = num_present_cpus();
 		else
 			r = num_online_cpus();
@@ -412,15 +438,16 @@ long kvm_arch_dev_ioctl(struct file *filp,
 	return -EINVAL;
 }
 
-void kvm_arch_free_memslot(struct kvm_memory_slot *free,
+void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 			   struct kvm_memory_slot *dont)
 {
-	kvmppc_core_free_memslot(free, dont);
+	kvmppc_core_free_memslot(kvm, free, dont);
 }
 
-int kvm_arch_create_memslot(struct kvm_memory_slot *slot, unsigned long npages)
+int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
+			    unsigned long npages)
 {
-	return kvmppc_core_create_memslot(slot, npages);
+	return kvmppc_core_create_memslot(kvm, slot, npages);
 }
 
 void kvm_arch_memslots_updated(struct kvm *kvm)
@@ -1033,9 +1060,10 @@ long kvm_arch_vm_ioctl(struct file *filp,
 	}
 	case KVM_PPC_GET_SMMU_INFO: {
 		struct kvm_ppc_smmu_info info;
+		struct kvm *kvm = filp->private_data;
 
 		memset(&info, 0, sizeof(info));
-		r = kvmppc_ops->get_smmu_info(kvm, &info);
+		r = kvm->arch.kvm_ops->get_smmu_info(kvm, &info);
 		if (r >= 0 && copy_to_user(argp, &info, sizeof(info)))
 			r = -EFAULT;
 		break;
@@ -1046,9 +1074,10 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		r = kvm_vm_ioctl_rtas_define_token(kvm, argp);
 		break;
 	}
-	default:
-		r = kvmppc_ops->arch_vm_ioctl(filp, ioctl, arg);
-
+	default: {
+		struct kvm *kvm = filp->private_data;
+		r = kvm->arch.kvm_ops->arch_vm_ioctl(filp, ioctl, arg);
+	}
 #else /* CONFIG_PPC_BOOK3S_64 */
 	default:
 		r = -ENOTTY;
@@ -1094,15 +1123,10 @@ void kvmppc_init_lpid(unsigned long nr_lpids_param)
 
 int kvm_arch_init(void *opaque)
 {
-	if (kvmppc_ops) {
-		printk(KERN_ERR "kvm: already loaded the other module\n");
-		return -EEXIST;
-	}
-	kvmppc_ops = (struct kvmppc_ops *)opaque;
 	return 0;
 }
 
 void kvm_arch_exit(void)
 {
-	kvmppc_ops = NULL;
+
 }
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 459b359..2938636 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -508,9 +508,10 @@ int kvm_set_memory_region(struct kvm *kvm,
 			  struct kvm_userspace_memory_region *mem);
 int __kvm_set_memory_region(struct kvm *kvm,
 			    struct kvm_userspace_memory_region *mem);
-void kvm_arch_free_memslot(struct kvm_memory_slot *free,
+void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 			   struct kvm_memory_slot *dont);
-int kvm_arch_create_memslot(struct kvm_memory_slot *slot, unsigned long npages);
+int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
+			    unsigned long npages);
 void kvm_arch_memslots_updated(struct kvm *kvm);
 int kvm_arch_prepare_memory_region(struct kvm *kvm,
 				struct kvm_memory_slot *memslot,
@@ -597,7 +598,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 			 unsigned int ioctl, unsigned long arg);
 int kvm_arch_vcpu_fault(struct kvm_vcpu *vcpu, struct vm_fault *vmf);
 
-int kvm_dev_ioctl_check_extension(long ext);
+int kvm_dev_ioctl_check_extension(struct file *filp, long ext);
 
 int kvm_get_dirty_log(struct kvm *kvm,
 			struct kvm_dirty_log *log, int *is_dirty);
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 99c2533..aebfc11 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -518,6 +518,10 @@ struct kvm_ppc_smmu_info {
 /* machine type bits, to be used as argument to KVM_CREATE_VM */
 #define KVM_VM_S390_UCONTROL	1
 
+/* on ppc, 0 indicate default, 1 should force HV and 2 PR */
+#define KVM_VM_PPC_HV 1
+#define KVM_VM_PPC_PR 2
+
 #define KVM_S390_SIE_PAGE_OFFSET 1
 
 /*
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 0594b22..2d43702 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -541,13 +541,13 @@ static void kvm_destroy_dirty_bitmap(struct kvm_memory_slot *memslot)
 /*
  * Free any memory in @free but not in @dont.
  */
-static void kvm_free_physmem_slot(struct kvm_memory_slot *free,
+static void kvm_free_physmem_slot(struct kvm *kvm, struct kvm_memory_slot *free,
 				  struct kvm_memory_slot *dont)
 {
 	if (!dont || free->dirty_bitmap != dont->dirty_bitmap)
 		kvm_destroy_dirty_bitmap(free);
 
-	kvm_arch_free_memslot(free, dont);
+	kvm_arch_free_memslot(kvm, free, dont);
 
 	free->npages = 0;
 }
@@ -558,7 +558,7 @@ void kvm_free_physmem(struct kvm *kvm)
 	struct kvm_memory_slot *memslot;
 
 	kvm_for_each_memslot(memslot, slots)
-		kvm_free_physmem_slot(memslot, NULL);
+		kvm_free_physmem_slot(kvm, memslot, NULL);
 
 	kfree(kvm->memslots);
 }
@@ -822,7 +822,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
 	if (change == KVM_MR_CREATE) {
 		new.userspace_addr = mem->userspace_addr;
 
-		if (kvm_arch_create_memslot(&new, npages))
+		if (kvm_arch_create_memslot(kvm, &new, npages))
 			goto out_free;
 	}
 
@@ -898,7 +898,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
 
 	kvm_arch_commit_memory_region(kvm, mem, &old, change);
 
-	kvm_free_physmem_slot(&old, &new);
+	kvm_free_physmem_slot(kvm, &old, &new);
 	kfree(old_memslots);
 
 	return 0;
@@ -906,7 +906,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
 out_slots:
 	kfree(slots);
 out_free:
-	kvm_free_physmem_slot(&new, &old);
+	kvm_free_physmem_slot(kvm, &new, &old);
 out:
 	return r;
 }
@@ -2579,7 +2579,7 @@ static int kvm_dev_ioctl_create_vm(unsigned long type)
 	return r;
 }
 
-static long kvm_dev_ioctl_check_extension_generic(long arg)
+static long kvm_dev_ioctl_check_extension_generic(struct file *filp, long arg)
 {
 	switch (arg) {
 	case KVM_CAP_USER_MEMORY:
@@ -2603,7 +2603,7 @@ static long kvm_dev_ioctl_check_extension_generic(long arg)
 	default:
 		break;
 	}
-	return kvm_dev_ioctl_check_extension(arg);
+	return kvm_dev_ioctl_check_extension(filp, arg);
 }
 
 static long kvm_dev_ioctl(struct file *filp,
@@ -2622,7 +2622,7 @@ static long kvm_dev_ioctl(struct file *filp,
 		r = kvm_dev_ioctl_create_vm(arg);
 		break;
 	case KVM_CHECK_EXTENSION:
-		r = kvm_dev_ioctl_check_extension_generic(arg);
+		r = kvm_dev_ioctl_check_extension_generic(filp, arg);
 		break;
 	case KVM_GET_VCPU_MMAP_SIZE:
 		r = -EINVAL;
-- 
1.8.1.2

[RFC PATCH 11/11] kvm: powerpc: book3s: Fix module ownership

[Aneesh Kumar K.V]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

This moves /dev/kvm ownership to kvm.ko module. Depending on
which KVM mode we select during VM creation we take a reference
count on respective module

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_ppc.h |  1 +
 arch/powerpc/kvm/book3s.c          | 21 +++++++++++++++++++++
 arch/powerpc/kvm/book3s_hv.c       | 15 ++++++---------
 arch/powerpc/kvm/book3s_pr.c       | 15 +++++----------
 arch/powerpc/kvm/powerpc.c         | 19 +++++++++++++++----
 5 files changed, 48 insertions(+), 23 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index a4a5893..2022720 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -171,6 +171,7 @@ extern int kvmppc_xics_int_on(struct kvm *kvm, u32 irq);
 extern int kvmppc_xics_int_off(struct kvm *kvm, u32 irq);
 
 struct kvmppc_ops {
+	struct module *owner;
 	bool is_hv_enabled;
 	int (*get_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
 	int (*set_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index 34e189c..363df6a 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -831,6 +831,10 @@ int kvm_arch_check_processor_compat(void *opaque)
 {
 	int r,cpu;
 	struct kvmppc_ops *kvm_ops = (struct kvmppc_ops *)opaque;
+
+	if (!kvm_ops)
+		return 0;
+
 	for_each_online_cpu(cpu) {
 		smp_call_function_single(cpu,
 					 kvm_ops->check_processor_compat,
@@ -840,6 +844,7 @@ int kvm_arch_check_processor_compat(void *opaque)
 	}
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvm_arch_check_processor_compat);
 
 EXPORT_SYMBOL_GPL(kvm_get_dirty_log);
 EXPORT_SYMBOL_GPL(kvmppc_core_pending_dec);
@@ -851,3 +856,19 @@ EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter);
 EXPORT_SYMBOL_GPL(kvmppc_core_queue_dec);
 EXPORT_SYMBOL_GPL(kvmppc_free_lpid);
 
+static int kvmppc_book3s_init(void)
+{
+	int r;
+
+	r = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+	return r;
+}
+
+static void kvmppc_book3s_exit(void)
+{
+	kvm_exit();
+}
+
+module_init(kvmppc_book3s_init);
+module_exit(kvmppc_book3s_exit);
+
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 0a684a7..7bdc780 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2092,23 +2092,20 @@ static int kvmppc_book3s_init_hv(void)
 {
 	int r;
 
-	if (!kvmppc_pr_ops) {
-		r = kvm_init(&kvm_ops_hv, sizeof(struct kvm_vcpu),
-			     0, THIS_MODULE);
-		if (r)
-			return r;
-	}
+	r = kvm_arch_check_processor_compat(&kvm_ops_hv);
+	if (r < 0)
+		return r;
+
+	kvm_ops_hv.owner = THIS_MODULE;
 	kvmppc_hv_ops = &kvm_ops_hv;
-	r = kvmppc_mmu_hv_init();
 
+	r = kvmppc_mmu_hv_init();
 	return r;
 }
 
 static void kvmppc_book3s_exit_hv(void)
 {
 	kvmppc_hv_ops = NULL;
-	if (!kvmppc_pr_ops)
-		kvm_exit();
 }
 
 module_init(kvmppc_book3s_init_hv);
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index e49e4b0..c79fada 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -1551,17 +1551,14 @@ static int kvmppc_book3s_init_pr(void)
 {
 	int r;
 
-	if (!kvmppc_hv_ops) {
-		r = kvm_init(&kvm_ops_pr, sizeof(struct kvm_vcpu),
-			     0, THIS_MODULE);
-		if (r)
-			return r;
-	}
-	/* Assign the global value */
+	r = kvm_arch_check_processor_compat(&kvm_ops_pr);
+	if (r < 0)
+		return r;
+
+	kvm_ops_pr.owner = THIS_MODULE;
 	kvmppc_pr_ops = &kvm_ops_pr;
 
 	r = kvmppc_mmu_hpte_sysinit();
-
 	return r;
 }
 
@@ -1569,8 +1566,6 @@ static void kvmppc_book3s_exit_pr(void)
 {
 	kvmppc_pr_ops = NULL;
 	kvmppc_mmu_hpte_sysexit();
-	if (!kvmppc_hv_ops)
-		kvm_exit();
 }
 
 module_init(kvmppc_book3s_init_pr);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 1209229..677fa7e 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -26,6 +26,7 @@
 #include <linux/fs.h>
 #include <linux/slab.h>
 #include <linux/file.h>
+#include <linux/module.h>
 #include <asm/cputable.h>
 #include <asm/uaccess.h>
 #include <asm/kvm_ppc.h>
@@ -270,25 +271,32 @@ void kvm_arch_hardware_unsetup(void)
 
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
+	struct kvmppc_ops *kvm_ops = NULL;
 	/*
 	 * if we have both HV and PR enabled, default is HV
 	 */
 	if (type == 0) {
 		if (kvmppc_hv_ops)
-			kvm->arch.kvm_ops = kvmppc_hv_ops;
+			kvm_ops = kvmppc_hv_ops;
 		else
-			kvm->arch.kvm_ops = kvmppc_pr_ops;
+			kvm_ops = kvmppc_pr_ops;
+		if (!kvm_ops)
+			goto err_out;
 	} else	if (type == KVM_VM_PPC_HV) {
 		if (!kvmppc_hv_ops)
 			goto err_out;
-		kvm->arch.kvm_ops = kvmppc_hv_ops;
+		kvm_ops = kvmppc_hv_ops;
 	} else if (type == KVM_VM_PPC_PR) {
 		if (!kvmppc_pr_ops)
 			goto err_out;
-		kvm->arch.kvm_ops = kvmppc_pr_ops;
+		kvm_ops = kvmppc_pr_ops;
 	} else
 		goto err_out;
 
+	if (kvm_ops->owner && !try_module_get(kvm_ops->owner))
+		return -ENOENT;
+
+	kvm->arch.kvm_ops = kvm_ops;
 	return kvmppc_core_init_vm(kvm);
 err_out:
 	return -EINVAL;
@@ -311,6 +319,9 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvmppc_core_destroy_vm(kvm);
 
 	mutex_unlock(&kvm->lock);
+
+	/* drop the module reference */
+	module_put(kvm->arch.kvm_ops->owner);
 }
 
 void kvm_arch_sync_events(struct kvm *kvm)
-- 
1.8.1.2

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Aneesh Kumar K.V]

"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> writes:

> Hi All,
>
> This patch series support enabling HV and PR KVM together in the same kernel. We
> extend machine property with new property "kvm_type". A value of 1 will force HV
> KVM and 2 PR KVM. The default value is 0 which will select the fastest KVM mode.
> ie, HV if that is supported otherwise PR.
>
> With Qemu command line having
>
>  -machine pseries,accel=kvm,kvm_type=1
>
> [root@llmp24l02 qemu]# bash ../qemu
> failed to initialize KVM: Invalid argument
> [root@llmp24l02 qemu]# modprobe kvm-pr
> [root@llmp24l02 qemu]# bash ../qemu
> failed to initialize KVM: Invalid argument
> [root@llmp24l02 qemu]# modprobe  kvm-hv
> [root@llmp24l02 qemu]# bash ../qemu
>
> now with
>
>  -machine pseries,accel=kvm,kvm_type=2
>
> [root@llmp24l02 qemu]# rmmod kvm-pr
> [root@llmp24l02 qemu]# bash ../qemu
> failed to initialize KVM: Invalid argument
> [root@llmp24l02 qemu]#
> [root@llmp24l02 qemu]# modprobe kvm-pr
> [root@llmp24l02 qemu]# bash ../qemu
>
> if don't specify kvm_type machine property, it will take a default value 0,
> which means fastest supported.

Related qemu patch

commit 8d139053177d48a70cb710b211ea4c2843eccdfb
Author: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Date:   Mon Sep 23 12:28:37 2013 +0530

    kvm: Add a new machine property kvm_type
    
    Targets like ppc64 support different type of KVM, one which use
    hypervisor mode and the other which doesn't. Add a new machine
    property kvm_type that helps in selecting the respective ones
    
    Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>

diff --git a/kvm-all.c b/kvm-all.c
index b87215c..a061eda 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1350,7 +1350,7 @@ int kvm_init(void)
     KVMState *s;
     const KVMCapabilityInfo *missing_cap;
     int ret;
-    int i;
+    int i, kvm_type;
     int max_vcpus;
 
     s = g_malloc0(sizeof(KVMState));
@@ -1407,7 +1407,8 @@ int kvm_init(void)
         goto err;
     }
 
-    s->vmfd = kvm_ioctl(s, KVM_CREATE_VM, 0);
+    kvm_type = qemu_opt_get_number(qemu_get_machine_opts(), "kvm_type", 0);
+    s->vmfd = kvm_ioctl(s, KVM_CREATE_VM, kvm_type);
     if (s->vmfd < 0) {
 #ifdef TARGET_S390X
         fprintf(stderr, "Please add the 'switch_amode' kernel parameter to "
diff --git a/vl.c b/vl.c
index 4e709d5..4374b17 100644
--- a/vl.c
+++ b/vl.c
@@ -427,7 +427,12 @@ static QemuOptsList qemu_machine_opts = {
             .name = "usb",
             .type = QEMU_OPT_BOOL,
             .help = "Set on/off to enable/disable usb",
+        },{
+            .name = "kvm_type",
+            .type = QEMU_OPT_NUMBER,
+            .help = "Set to kvm type to be used in create vm ioctl",
         },
+
         { /* End of list */ }
     },
 };

Re: [RFC PATCH 01/11] kvm: powerpc: book3s hv: Fix vcore leak

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: Paul Mackerras <paulus@samba.org>
> 
> add kvmppc_free_vcores() to free the kvmppc_vcore structures
> that we allocate for a guest, which are currently being leaked.
> 
> Signed-off-by: Paul Mackerras <paulus@samba.org>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>

This one doesn't look like an RFC to me. Applied to kvm-ppc-queue :).


Alex

> ---
> arch/powerpc/kvm/book3s_hv.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
> 
> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> index 62a2b5a..edc7f9f 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -1931,10 +1931,20 @@ int kvmppc_core_init_vm(struct kvm *kvm)
> 	return 0;
> }
> 
> +static void kvmppc_free_vcores(struct kvm *kvm)
> +{
> +	long int i;
> +
> +	for (i = 0; i < KVM_MAX_VCORES; ++i)
> +		kfree(kvm->arch.vcores[i]);
> +	kvm->arch.online_vcores = 0;
> +}
> +
> void kvmppc_core_destroy_vm(struct kvm *kvm)
> {
> 	uninhibit_secondary_onlining();
> 
> +	kvmppc_free_vcores(kvm);
> 	if (kvm->arch.rma) {
> 		kvm_release_rma(kvm->arch.rma);
> 		kvm->arch.rma = NULL;
> -- 
> 1.8.1.2
> 

Re: [RFC PATCH 04/11] kvm: powerpc: book3s: Add a new config variable CONFIG_KVM_BOOK3S_HV

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>=20
> This help ups to select the relevant code in the kernel code
> when we later move HV and PR bits as seperate modules.

I don't think I grasp what semantically the difference between =
CONFIG_KVM_BOOK3S_HV and CONFIG_KVM_BOOK3S_64_HV is :).


Alex

Re: [RFC PATCH 05/11] kvm: powerpc: book3s: Add kvmppc_ops callback for HV and PR specific operations

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>=20
> This moves HV and PR specific functions to kvmppc_ops callback.
> This is needed so that we can enable HV and PR together in the
> same kernel. Actual changes to enable both come in the later
> patch.This also renames almost all of the symbols that exist in both =
PR and HV
> KVM for clarity. Symbols in the PR KVM implementation get "_pr"
> appended, and those in the HV KVM implementation get "_hv".  Then,
> in book3s.c, we add a function with the name without the suffix and
> arrange for it to call the appropriate kvmppc_ops callback depending =
on
> which kvm type we selected during VM creation.
>=20
> NOTE: we still don't enable selecting both the HV and PR together
> in this commit that will be done by a later commit.
>=20
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/include/asm/kvm_book3s.h |   5 +-
> arch/powerpc/include/asm/kvm_ppc.h    |  63 ++++++++--
> arch/powerpc/kvm/Kconfig              |  15 ++-
> arch/powerpc/kvm/Makefile             |   9 +-
> arch/powerpc/kvm/book3s.c             | 145 +++++++++++++++++++++-
> arch/powerpc/kvm/book3s_32_mmu_host.c |   2 +-
> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
> arch/powerpc/kvm/book3s_64_mmu_hv.c   |  17 ++-
> arch/powerpc/kvm/book3s_emulate.c     |   8 +-
> arch/powerpc/kvm/book3s_hv.c          | 226 =
+++++++++++++++++++++++++---------
> arch/powerpc/kvm/book3s_interrupts.S  |   2 +-
> arch/powerpc/kvm/book3s_pr.c          | 196 =
++++++++++++++++++-----------
> arch/powerpc/kvm/emulate.c            |   6 +-
> arch/powerpc/kvm/powerpc.c            |  58 +++------
> 14 files changed, 539 insertions(+), 215 deletions(-)
>=20
>=20

[...]

> @@ -888,14 +890,8 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 =
id, union kvmppc_one_reg *val)
> 	return r;
> }
>=20
> -int kvmppc_core_check_processor_compat(void)
> -{
> -	if (cpu_has_feature(CPU_FTR_HVMODE))
> -		return 0;
> -	return -EIO;
> -}
> -
> -struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned =
int id)
> +static struct kvm_vcpu *kvmppc_core_vcpu_create_hv(struct kvm *kvm,
> +						   unsigned int id)
> {
> 	struct kvm_vcpu *vcpu;
> 	int err =3D -EINVAL;
> @@ -920,7 +916,6 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct =
kvm *kvm, unsigned int id)
> 	vcpu->arch.ctrl =3D CTRL_RUNLATCH;
> 	/* default to host PVR, since we can't spoof it */
> 	vcpu->arch.pvr =3D mfspr(SPRN_PVR);
> -	kvmppc_set_pvr(vcpu, vcpu->arch.pvr);

Where is this one going?

> 	spin_lock_init(&vcpu->arch.vpa_update_lock);
> 	spin_lock_init(&vcpu->arch.tbacct_lock);
> 	vcpu->arch.busy_preempt =3D TB_NIL;
> @@ -972,7 +967,7 @@ static void unpin_vpa(struct kvm *kvm, struct =
kvmppc_vpa *vpa)
> 					vpa->dirty);
> }
>=20
> -void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
> +static void kvmppc_core_vcpu_free_hv(struct kvm_vcpu *vcpu)
> {
> 	spin_lock(&vcpu->arch.vpa_update_lock);
> 	unpin_vpa(vcpu->kvm, &vcpu->arch.dtl);
> @@ -983,6 +978,12 @@ void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
> 	kmem_cache_free(kvm_vcpu_cache, vcpu);
> }
>=20
> +static int kvmppc_core_check_requests_hv(struct kvm_vcpu *vcpu)
> +{
> +	/* Indicate we want to get back into the guest */
> +	return 1;
> +}
> +
>=20

[...]

> +	case KVM_PPC_GET_HTAB_FD: {
> +		struct kvm_get_htab_fd ghf;
> +
> +		r =3D -EFAULT;
> +		if (copy_from_user(&ghf, argp, sizeof(ghf)))
> +			break;
> +		r =3D kvm_vm_ioctl_get_htab_fd(kvm, &ghf);
> +		break;
> +	}
> +
> +	default:
> +		r =3D -ENOTTY;
> +	}
> +
> +	return r;
> }
>=20
> -static int kvmppc_book3s_hv_init(void)
> +/* FIXME!! move to header */

Hrm :)

> +extern void kvmppc_core_flush_memslot_hv(struct kvm *kvm,
> +					 struct kvm_memory_slot =
*memslot);
> +extern int kvm_unmap_hva_hv(struct kvm *kvm, unsigned long hva);
> +extern int kvm_unmap_hva_range_hv(struct kvm *kvm, unsigned long =
start,
> +				  unsigned long end);
> +extern int kvm_age_hva_hv(struct kvm *kvm, unsigned long hva);
> +extern int kvm_test_age_hva_hv(struct kvm *kvm, unsigned long hva);
> +extern void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, =
pte_t pte);
> +
> +static struct kvmppc_ops kvmppc_hv_ops =3D {
> +	.get_sregs =3D kvm_arch_vcpu_ioctl_get_sregs_hv,
> +	.set_sregs =3D kvm_arch_vcpu_ioctl_set_sregs_hv,
> +	.get_one_reg =3D kvmppc_get_one_reg_hv,
> +	.set_one_reg =3D kvmppc_set_one_reg_hv,
> +	.vcpu_load   =3D kvmppc_core_vcpu_load_hv,
> +	.vcpu_put    =3D kvmppc_core_vcpu_put_hv,
> +	.set_msr     =3D kvmppc_set_msr_hv,
> +	.vcpu_run    =3D kvmppc_vcpu_run_hv,
> +	.vcpu_create =3D kvmppc_core_vcpu_create_hv,
> +	.vcpu_free   =3D kvmppc_core_vcpu_free_hv,
> +	.check_requests =3D kvmppc_core_check_requests_hv,
> +	.get_dirty_log  =3D kvm_vm_ioctl_get_dirty_log_hv,
> +	.flush_memslot  =3D kvmppc_core_flush_memslot_hv,
> +	.prepare_memory_region =3D kvmppc_core_prepare_memory_region_hv,
> +	.commit_memory_region  =3D kvmppc_core_commit_memory_region_hv,
> +	.unmap_hva =3D kvm_unmap_hva_hv,
> +	.unmap_hva_range =3D kvm_unmap_hva_range_hv,
> +	.age_hva  =3D kvm_age_hva_hv,
> +	.test_age_hva =3D kvm_test_age_hva_hv,
> +	.set_spte_hva =3D kvm_set_spte_hva_hv,
> +	.mmu_destroy  =3D kvmppc_mmu_destroy_hv,
> +	.free_memslot =3D kvmppc_core_free_memslot_hv,
> +	.create_memslot =3D kvmppc_core_create_memslot_hv,
> +	.init_vm =3D  kvmppc_core_init_vm_hv,
> +	.destroy_vm =3D kvmppc_core_destroy_vm_hv,
> +	.check_processor_compat =3D =
kvmppc_core_check_processor_compat_hv,
> +	.get_smmu_info =3D kvm_vm_ioctl_get_smmu_info_hv,
> +	.emulate_op =3D kvmppc_core_emulate_op_hv,
> +	.emulate_mtspr =3D kvmppc_core_emulate_mtspr_hv,
> +	.emulate_mfspr =3D kvmppc_core_emulate_mfspr_hv,
> +	.fast_vcpu_kick =3D kvmppc_fast_vcpu_kick_hv,
> +	.arch_vm_ioctl  =3D kvm_arch_vm_ioctl_hv,
> +};
> +
>=20

[...]

> @@ -1390,8 +1389,42 @@ out:
> 	return r;
> }
>=20
> +static void kvmppc_core_flush_memslot_pr(struct kvm *kvm,
> +					 struct kvm_memory_slot =
*memslot)
> +{
> +	return;
> +}
> +
> +static int kvmppc_core_prepare_memory_region_pr(struct kvm *kvm,
> +					struct kvm_memory_slot *memslot,
> +					struct =
kvm_userspace_memory_region *mem)
> +{
> +	return 0;
> +}
> +
> +static void kvmppc_core_commit_memory_region_pr(struct kvm *kvm,
> +				struct kvm_userspace_memory_region *mem,
> +				const struct kvm_memory_slot *old)
> +{
> +	return;
> +}
> +
> +static void kvmppc_core_free_memslot_pr(struct kvm_memory_slot *free,
> +					struct kvm_memory_slot *dont)
> +{
> +	return;
> +}
> +
> +static int kvmppc_core_create_memslot_pr(struct kvm_memory_slot =
*slot,
> +					 unsigned long npages)
> +{
> +	return 0;
> +}
> +
> +
> #ifdef CONFIG_PPC64
> -int kvm_vm_ioctl_get_smmu_info(struct kvm *kvm, struct =
kvm_ppc_smmu_info *info)
> +static int kvm_vm_ioctl_get_smmu_info_pr(struct kvm *kvm,
> +					 struct kvm_ppc_smmu_info *info)

You're dereferencing this function unconditionally now, probably =
breaking book3s_32 along the way :).

I'm not really happy with the naming scheme either, but I can't really =
think of anything better right now. In an ideal world all functions =
would still have the same names and we merely make them static and refer =
to them through structs :).


Alex

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>=20
> This help us to identify whether we are running with hypervisor mode =
KVM
> enabled. The change is needed so that we can have both HV and PR kvm
> enabled in the same kernel.
>=20
> If both HV and PR KVM are included, interrupts come in to the HV =
version
> of the kvmppc_interrupt code, which then jumps to the PR handler,
> renamed to kvmppc_interrupt_pr, if the guest is a PR guest.
>=20
> Allowing both PR and HV in the same kernel required some changes to
> kvm_dev_ioctl_check_extension(), since the values returned now can't
> be selected with #ifdefs as much as previously. We look at =
is_hv_enabled
> to return the right value when checking for capabilities.For =
capabilities that
> are only provided by HV KVM, we return the HV value only if
> is_hv_enabled is true. For capabilities provided by PR KVM but not HV,
> we return the PR value only if is_hv_enabled is false.
>=20
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/include/asm/kvm_book3s.h   | 53 =
--------------------------------
> arch/powerpc/include/asm/kvm_ppc.h      |  5 +--
> arch/powerpc/kvm/Makefile               |  2 +-
> arch/powerpc/kvm/book3s.c               | 44 =
+++++++++++++++++++++++++++
> arch/powerpc/kvm/book3s_hv.c            |  1 +
> arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 +++
> arch/powerpc/kvm/book3s_pr.c            |  1 +
> arch/powerpc/kvm/book3s_segment.S       |  7 ++++-
> arch/powerpc/kvm/book3s_xics.c          |  2 +-
> arch/powerpc/kvm/powerpc.c              | 54 =
++++++++++++++++++---------------
> 10 files changed, 90 insertions(+), 83 deletions(-)
>=20
> diff --git a/arch/powerpc/include/asm/kvm_book3s.h =
b/arch/powerpc/include/asm/kvm_book3s.h
> index 3efba3c..ba33c49 100644
> --- a/arch/powerpc/include/asm/kvm_book3s.h
> +++ b/arch/powerpc/include/asm/kvm_book3s.h
> @@ -297,59 +297,6 @@ static inline ulong kvmppc_get_fault_dar(struct =
kvm_vcpu *vcpu)
> 	return vcpu->arch.fault_dar;
> }
>=20
> -#ifdef CONFIG_KVM_BOOK3S_PR
> -
> -static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu =
*vcpu)
> -{
> -	return to_book3s(vcpu)->hior;
> -}
> -
> -static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
> -			unsigned long pending_now, unsigned long =
old_pending)
> -{
> -	if (pending_now)
> -		vcpu->arch.shared->int_pending =3D 1;
> -	else if (old_pending)
> -		vcpu->arch.shared->int_pending =3D 0;
> -}
> -
> -static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
> -{
> -	ulong crit_raw =3D vcpu->arch.shared->critical;
> -	ulong crit_r1 =3D kvmppc_get_gpr(vcpu, 1);
> -	bool crit;
> -
> -	/* Truncate crit indicators in 32 bit mode */
> -	if (!(vcpu->arch.shared->msr & MSR_SF)) {
> -		crit_raw &=3D 0xffffffff;
> -		crit_r1 &=3D 0xffffffff;
> -	}
> -
> -	/* Critical section when crit =3D=3D r1 */
> -	crit =3D (crit_raw =3D=3D crit_r1);
> -	/* ... and we're in supervisor mode */
> -	crit =3D crit && !(vcpu->arch.shared->msr & MSR_PR);
> -
> -	return crit;
> -}
> -#else /* CONFIG_KVM_BOOK3S_PR */
> -
> -static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu =
*vcpu)
> -{
> -	return 0;
> -}
> -
> -static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
> -			unsigned long pending_now, unsigned long =
old_pending)
> -{
> -}
> -
> -static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
> -{
> -	return false;
> -}
> -#endif
> -
> /* Magic register values loaded into r3 and r4 before the 'sc' =
assembly
>  * instruction for the OSI hypercalls */
> #define OSI_SC_MAGIC_R3			0x113724FA
> diff --git a/arch/powerpc/include/asm/kvm_ppc.h =
b/arch/powerpc/include/asm/kvm_ppc.h
> index 4d9641c..58e732f 100644
> --- a/arch/powerpc/include/asm/kvm_ppc.h
> +++ b/arch/powerpc/include/asm/kvm_ppc.h
> @@ -169,6 +169,7 @@ extern int kvmppc_xics_int_on(struct kvm *kvm, u32 =
irq);
> extern int kvmppc_xics_int_off(struct kvm *kvm, u32 irq);
>=20
> struct kvmppc_ops {
> +	bool is_hv_enabled;

This doesn't really belong into an ops struct. Either you compare

  if (kvmppc_ops =3D=3D kvmppc_ops_pr)

against a global known good ops struct or you put the hint somewhere =
into the kvm struct.

> 	int (*get_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs =
*sregs);
> 	int (*set_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs =
*sregs);
> 	int (*get_one_reg)(struct kvm_vcpu *vcpu, u64 id,
> @@ -309,10 +310,10 @@ static inline void kvmppc_set_xics_phys(int cpu, =
unsigned long addr)
>=20
> static inline u32 kvmppc_get_xics_latch(void)
> {
> -	u32 xirr =3D get_paca()->kvm_hstate.saved_xirr;
> +	u32 xirr;
>=20
> +	xirr =3D get_paca()->kvm_hstate.saved_xirr;
> 	get_paca()->kvm_hstate.saved_xirr =3D 0;
> -

I don't see any functionality change here?

> 	return xirr;
> }
>=20
> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> index c343793..a514ecd 100644
> --- a/arch/powerpc/kvm/Makefile
> +++ b/arch/powerpc/kvm/Makefile
> @@ -77,7 +77,7 @@ =
kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) +=3D \
> 	book3s_rmhandlers.o
> endif
>=20
> -kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) :=3D \
> +kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  +=3D \

This change looks unrelated?

> 	book3s_hv.o \
> 	book3s_hv_interrupts.o \
> 	book3s_64_mmu_hv.o
> diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
> index bdc3f95..12f94bf 100644
> --- a/arch/powerpc/kvm/book3s.c
> +++ b/arch/powerpc/kvm/book3s.c
> @@ -69,6 +69,50 @@ void kvmppc_core_load_guest_debugstate(struct =
kvm_vcpu *vcpu)
> {
> }
>=20
> +static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu =
*vcpu)

Please drop the "inline" keyword on functions in .c files :). That way =
the compiler tells us about unused functions.

> +{
> +	if (!kvmppc_ops->is_hv_enabled)
> +		return to_book3s(vcpu)->hior;
> +	return 0;
> +}
> +
> +static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
> +			unsigned long pending_now, unsigned long =
old_pending)
> +{
> +	if (kvmppc_ops->is_hv_enabled)
> +		return;
> +	if (pending_now)
> +		vcpu->arch.shared->int_pending =3D 1;
> +	else if (old_pending)
> +		vcpu->arch.shared->int_pending =3D 0;
> +}
> +
> +static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
> +{
> +	ulong crit_raw;
> +	ulong crit_r1;
> +	bool crit;
> +
> +	if (kvmppc_ops->is_hv_enabled)
> +		return false;
> +
> +	crit_raw =3D vcpu->arch.shared->critical;
> +	crit_r1 =3D kvmppc_get_gpr(vcpu, 1);
> +
> +	/* Truncate crit indicators in 32 bit mode */
> +	if (!(vcpu->arch.shared->msr & MSR_SF)) {
> +		crit_raw &=3D 0xffffffff;
> +		crit_r1 &=3D 0xffffffff;
> +	}
> +
> +	/* Critical section when crit =3D=3D r1 */
> +	crit =3D (crit_raw =3D=3D crit_r1);
> +	/* ... and we're in supervisor mode */
> +	crit =3D crit && !(vcpu->arch.shared->msr & MSR_PR);
> +
> +	return crit;
> +}
> +
> void kvmppc_inject_interrupt(struct kvm_vcpu *vcpu, int vec, u64 =
flags)
> {
> 	vcpu->arch.shared->srr0 =3D kvmppc_get_pc(vcpu);
> diff --git a/arch/powerpc/kvm/book3s_hv.c =
b/arch/powerpc/kvm/book3s_hv.c
> index 36d7920..eec353d 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -2048,6 +2048,7 @@ extern int kvm_test_age_hva_hv(struct kvm *kvm, =
unsigned long hva);
> extern void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, =
pte_t pte);
>=20
> static struct kvmppc_ops kvmppc_hv_ops =3D {
> +	.is_hv_enabled =3D true,
> 	.get_sregs =3D kvm_arch_vcpu_ioctl_get_sregs_hv,
> 	.set_sregs =3D kvm_arch_vcpu_ioctl_set_sregs_hv,
> 	.get_one_reg =3D kvmppc_get_one_reg_hv,
> diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S =
b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
> index 5ede7fc..4838fdb 100644
> --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
> +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
> @@ -577,6 +577,10 @@ kvmppc_interrupt:
> 	lbz	r9, HSTATE_IN_GUEST(r13)
> 	cmpwi	r9, KVM_GUEST_MODE_HOST_HV
> 	beq	kvmppc_bad_host_intr
> +#ifdef CONFIG_KVM_BOOK3S_PR
> +	cmpwi	r9, KVM_GUEST_MODE_GUEST
> +	beq	kvmppc_interrupt_pr
> +#endif
> 	/* We're now back in the host but in guest MMU context */
> 	li	r9, KVM_GUEST_MODE_HOST_HV
> 	stb	r9, HSTATE_IN_GUEST(r13)
> diff --git a/arch/powerpc/kvm/book3s_pr.c =
b/arch/powerpc/kvm/book3s_pr.c
> index ff5bd24..2a97279 100644
> --- a/arch/powerpc/kvm/book3s_pr.c
> +++ b/arch/powerpc/kvm/book3s_pr.c
> @@ -1509,6 +1509,7 @@ extern int kvmppc_core_emulate_mfspr_pr(struct =
kvm_vcpu *vcpu,
> 					int sprn, ulong *spr_val);
>=20
> static struct kvmppc_ops kvmppc_pr_ops =3D {
> +	.is_hv_enabled =3D false,
> 	.get_sregs =3D kvm_arch_vcpu_ioctl_get_sregs_pr,
> 	.set_sregs =3D kvm_arch_vcpu_ioctl_set_sregs_pr,
> 	.get_one_reg =3D kvmppc_get_one_reg_pr,
> diff --git a/arch/powerpc/kvm/book3s_segment.S =
b/arch/powerpc/kvm/book3s_segment.S
> index 1abe478..e0229dd 100644
> --- a/arch/powerpc/kvm/book3s_segment.S
> +++ b/arch/powerpc/kvm/book3s_segment.S
> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
> .global kvmppc_handler_trampoline_exit
> kvmppc_handler_trampoline_exit:
>=20
> +#if defined(CONFIG_KVM_BOOK3S_HV)
> +.global kvmppc_interrupt_pr
> +kvmppc_interrupt_pr:
> +	ld	r9, HSTATE_SCRATCH2(r13)
> +#else
> .global kvmppc_interrupt
> kvmppc_interrupt:

Just always call it kvmppc_interrupt_pr and thus share at least that =
part of the code :).

> -
> +#endif
> 	/* Register usage at this point:
> 	 *
> 	 * SPRG_SCRATCH0  =3D guest R13
> diff --git a/arch/powerpc/kvm/book3s_xics.c =
b/arch/powerpc/kvm/book3s_xics.c
> index f0c732e..fa7625d 100644
> --- a/arch/powerpc/kvm/book3s_xics.c
> +++ b/arch/powerpc/kvm/book3s_xics.c
> @@ -818,7 +818,7 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 =
req)
> 	}
>=20
> 	/* Check for real mode returning too hard */
> -	if (xics->real_mode)
> +	if (xics->real_mode && kvmppc_ops->is_hv_enabled)
> 		return kvmppc_xics_rm_complete(vcpu, req);
>=20
> 	switch (req) {
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 69b9305..00a96fc 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -52,7 +52,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
> 	return 1;
> }
>=20
> -#ifndef CONFIG_KVM_BOOK3S_64_HV
> /*
>  * Common checks before entering the guest world.  Call with =
interrupts
>  * disabled.
> @@ -127,7 +126,6 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>=20
> 	return r;
> }
> -#endif /* CONFIG_KVM_BOOK3S_64_HV */
>=20
> int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
> {
> @@ -194,11 +192,9 @@ int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
> 	if ((vcpu->arch.cpu_type !=3D KVM_CPU_3S_64) && =
vcpu->arch.papr_enabled)
> 		goto out;
>=20
> -#ifdef CONFIG_KVM_BOOK3S_64_HV
> 	/* HV KVM can only do PAPR mode for now */
> -	if (!vcpu->arch.papr_enabled)
> +	if (!vcpu->arch.papr_enabled && kvmppc_ops->is_hv_enabled)
> 		goto out;
> -#endif
>=20
> #ifdef CONFIG_KVM_BOOKE_HV
> 	if (!cpu_has_feature(CPU_FTR_EMB_HV))
> @@ -322,22 +318,26 @@ int kvm_dev_ioctl_check_extension(long ext)
> 	case KVM_CAP_DEVICE_CTRL:
> 		r =3D 1;
> 		break;
> -#ifndef CONFIG_KVM_BOOK3S_64_HV
> 	case KVM_CAP_PPC_PAIRED_SINGLES:
> 	case KVM_CAP_PPC_OSI:
> 	case KVM_CAP_PPC_GET_PVINFO:
> #if defined(CONFIG_KVM_E500V2) || defined(CONFIG_KVM_E500MC)
> 	case KVM_CAP_SW_TLB:
> #endif
> -#ifdef CONFIG_KVM_MPIC
> -	case KVM_CAP_IRQ_MPIC:
> -#endif
> -		r =3D 1;
> +		/* We support this only for PR */
> +		r =3D !kvmppc_ops->is_hv_enabled;

Reading this, have you test compiled your code against e500 configs?

> 		break;
> +#ifdef CONFIG_KVM_MMIO
> 	case KVM_CAP_COALESCED_MMIO:
> 		r =3D KVM_COALESCED_MMIO_PAGE_OFFSET;
> 		break;
> #endif
> +#ifdef CONFIG_KVM_MPIC
> +	case KVM_CAP_IRQ_MPIC:
> +		r =3D 1;
> +		break;
> +#endif
> +
> #ifdef CONFIG_PPC_BOOK3S_64
> 	case KVM_CAP_SPAPR_TCE:
> 	case KVM_CAP_PPC_ALLOC_HTAB:
> @@ -348,32 +348,37 @@ int kvm_dev_ioctl_check_extension(long ext)
> 		r =3D 1;
> 		break;
> #endif /* CONFIG_PPC_BOOK3S_64 */
> -#ifdef CONFIG_KVM_BOOK3S_64_HV
> +#ifdef CONFIG_KVM_BOOK3S_HV
> 	case KVM_CAP_PPC_SMT:
> -		r =3D threads_per_core;
> +		if (kvmppc_ops->is_hv_enabled)
> +			r =3D threads_per_core;
> +		else
> +			r =3D 0;

0? Or 1?


Alex

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>=20
> This patch moves PR related tracepoints to a separate header. This
> enables in converting PR to a kernel module which will be done in
> later patches
>=20
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
> arch/powerpc/kvm/book3s_mmu_hpte.c    |   2 +-
> arch/powerpc/kvm/book3s_pr.c          |   3 +-
> arch/powerpc/kvm/trace.h              | 234 =
+--------------------------
> arch/powerpc/kvm/trace_pr.h           | 297 =
++++++++++++++++++++++++++++++++++
> 5 files changed, 308 insertions(+), 230 deletions(-)
> create mode 100644 arch/powerpc/kvm/trace_pr.h
>=20
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c =
b/arch/powerpc/kvm/book3s_64_mmu_host.c
> index 329a978..fd5b393 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
> @@ -27,7 +27,7 @@
> #include <asm/machdep.h>
> #include <asm/mmu_context.h>
> #include <asm/hw_irq.h>
> -#include "trace.h"
> +#include "trace_pr.h"
>=20
> #define PTE_SIZE 12
>=20
> diff --git a/arch/powerpc/kvm/book3s_mmu_hpte.c =
b/arch/powerpc/kvm/book3s_mmu_hpte.c
> index d2d280b..4556168 100644
> --- a/arch/powerpc/kvm/book3s_mmu_hpte.c
> +++ b/arch/powerpc/kvm/book3s_mmu_hpte.c
> @@ -28,7 +28,7 @@
> #include <asm/mmu_context.h>
> #include <asm/hw_irq.h>
>=20
> -#include "trace.h"
> +#include "trace_pr.h"
>=20
> #define PTE_SIZE	12
>=20
> diff --git a/arch/powerpc/kvm/book3s_pr.c =
b/arch/powerpc/kvm/book3s_pr.c
> index 2a97279..99d0839 100644
> --- a/arch/powerpc/kvm/book3s_pr.c
> +++ b/arch/powerpc/kvm/book3s_pr.c
> @@ -41,7 +41,8 @@
> #include <linux/vmalloc.h>
> #include <linux/highmem.h>
>=20
> -#include "trace.h"
> +#define CREATE_TRACE_POINTS
> +#include "trace_pr.h"
>=20
> /* #define EXIT_DEBUG */
> /* #define DEBUG_EXT */
> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
> index a088e9a..7d5a136 100644
> --- a/arch/powerpc/kvm/trace.h
> +++ b/arch/powerpc/kvm/trace.h
> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
> 	{41, "HV_PRIV"}
> #endif
>=20
> +#ifndef CONFIG_KVM_BOOK3S_PR
> +/*
> + * For pr we define this in trace_pr.h since it pr can be built as
> + * a module

Not sure I understand the need. If the config option is available, so =
should the struct field. Worst case that happens with HV is that we get =
empty shadow_srr1 values in our trace, no?

If your goal is to make it more obvious whether we are tracing in PR or =
HV land (which is a reasonable goal), then you should also split off all =
non-common trace points into a special hv trace header so that it's =
obvious whether we are looking at HV or PR.


Alex

Re: [RFC PATCH 08/11] kvm: powerpc: book3s: Support building HV and PR KVM as module

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>=20
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/kvm/Kconfig              |  6 +++---
> arch/powerpc/kvm/Makefile             | 12 ++++++++----
> arch/powerpc/kvm/book3s.c             | 19 ++++++++++++++++++-
> arch/powerpc/kvm/book3s_64_mmu_host.c |  1 +
> arch/powerpc/kvm/book3s_emulate.c     |  2 +-
> arch/powerpc/kvm/book3s_hv.c          |  4 ++++
> arch/powerpc/kvm/book3s_pr.c          |  5 ++++-
> arch/powerpc/kvm/book3s_rtas.c        |  1 +
> arch/powerpc/kvm/emulate.c            |  1 +
> arch/powerpc/kvm/powerpc.c            |  6 ++++++
> virt/kvm/kvm_main.c                   |  3 +++
> 11 files changed, 50 insertions(+), 10 deletions(-)
>=20
> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
> index 5c62459..f63fd8f 100644
> --- a/arch/powerpc/kvm/Kconfig
> +++ b/arch/powerpc/kvm/Kconfig
> @@ -73,7 +73,7 @@ config KVM_BOOK3S_64
> 	  If unsure, say N.
>=20
> config KVM_BOOK3S_64_HV
> -	bool "KVM support for POWER7 and PPC970 using hypervisor mode in =
host"
> +	tristate "KVM support for POWER7 and PPC970 using hypervisor =
mode in host"
> 	depends on KVM_BOOK3S_64
> 	select KVM_BOOK3S_HV
> 	select MMU_NOTIFIER
> @@ -94,8 +94,8 @@ config KVM_BOOK3S_64_HV
> 	  If unsure, say N.
>=20
> config KVM_BOOK3S_64_PR
> -	bool "KVM support without using hypervisor mode in host"
> -	depends on KVM_BOOK3S_64 && !KVM_BOOK3S_64_HV
> +	tristate "KVM support without using hypervisor mode in host"
> +	depends on KVM_BOOK3S_64
> 	select KVM_BOOK3S_PR
> 	---help---
> 	  Support running guest kernels in virtual machines on =
processors
> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> index a514ecd..861b8da 100644
> --- a/arch/powerpc/kvm/Makefile
> +++ b/arch/powerpc/kvm/Makefile
> @@ -56,8 +56,7 @@ kvm-objs-$(CONFIG_KVM_E500MC) :=3D =
$(kvm-e500mc-objs)
> kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) :=3D \
> 	book3s_64_vio_hv.o
>=20
> -kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_PR) :=3D \
> -	$(KVM)/coalesced_mmio.o \
> +kvm-pr-y :=3D \
> 	fpu.o \
> 	book3s_paired_singles.o \
> 	book3s_pr.o \
> @@ -77,7 +76,7 @@ =
kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) +=3D \
> 	book3s_rmhandlers.o
> endif
>=20
> -kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  +=3D \
> +kvm-hv-y +=3D \
> 	book3s_hv.o \
> 	book3s_hv_interrupts.o \
> 	book3s_64_mmu_hv.o
> @@ -85,13 +84,15 @@ kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  +=3D =
\
> kvm-book3s_64-builtin-xics-objs-$(CONFIG_KVM_XICS) :=3D \
> 	book3s_hv_rm_xics.o
>=20
> -kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HV) +=3D \
> +ifdef CONFIG_KVM_BOOK3S_HV
> +kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) +=3D \
> 	book3s_hv_rmhandlers.o \
> 	book3s_hv_rm_mmu.o \
> 	book3s_hv_ras.o \
> 	book3s_hv_builtin.o \
> 	book3s_hv_cma.o \
> 	$(kvm-book3s_64-builtin-xics-objs-y)
> +endif
>=20
> kvm-book3s_64-objs-$(CONFIG_KVM_XICS) +=3D \
> 	book3s_xics.o
> @@ -132,4 +133,7 @@ obj-$(CONFIG_KVM_E500MC) +=3D kvm.o
> obj-$(CONFIG_KVM_BOOK3S_64) +=3D kvm.o
> obj-$(CONFIG_KVM_BOOK3S_32) +=3D kvm.o
>=20
> +obj-$(CONFIG_KVM_BOOK3S_64_PR) +=3D kvm-pr.o
> +obj-$(CONFIG_KVM_BOOK3S_64_HV) +=3D kvm-hv.o
> +
> obj-y +=3D $(kvm-book3s_64-builtin-objs-y)
> diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
> index 12f94bf..ca617e1 100644
> --- a/arch/powerpc/kvm/book3s.c
> +++ b/arch/powerpc/kvm/book3s.c
> @@ -170,13 +170,14 @@ void kvmppc_book3s_queue_irqprio(struct kvm_vcpu =
*vcpu, unsigned int vec)
> 	printk(KERN_INFO "Queueing interrupt %x\n", vec);
> #endif
> }
> -
> +EXPORT_SYMBOL_GPL(kvmppc_book3s_queue_irqprio);
>=20
> void kvmppc_core_queue_program(struct kvm_vcpu *vcpu, ulong flags)
> {
> 	/* might as well deliver this straight away */
> 	kvmppc_inject_interrupt(vcpu, BOOK3S_INTERRUPT_PROGRAM, flags);
> }
> +EXPORT_SYMBOL_GPL(kvmppc_core_queue_program);
>=20
> void kvmppc_core_queue_dec(struct kvm_vcpu *vcpu)
> {
> @@ -351,6 +352,7 @@ pfn_t kvmppc_gfn_to_pfn(struct kvm_vcpu *vcpu, =
gfn_t gfn)
>=20
> 	return gfn_to_pfn(vcpu->kvm, gfn);
> }
> +EXPORT_SYMBOL_GPL(kvmppc_gfn_to_pfn);
>=20
> static int kvmppc_xlate(struct kvm_vcpu *vcpu, ulong eaddr, bool data,
> 			 struct kvmppc_pte *pte)
> @@ -418,6 +420,7 @@ int kvmppc_st(struct kvm_vcpu *vcpu, ulong *eaddr, =
int size, void *ptr,
>=20
> 	return EMULATE_DONE;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_st);
>=20
> int kvmppc_ld(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void =
*ptr,
> 		      bool data)
> @@ -448,6 +451,7 @@ nopte:
> mmio:
> 	return EMULATE_DO_MMIO;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_ld);
>=20
> int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu)
> {
> @@ -693,6 +697,7 @@ void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 =
msr)
> {
> 	kvmppc_ops->set_msr(vcpu, msr);
> }
> +EXPORT_SYMBOL_GPL(kvmppc_set_msr);
>=20
> int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
> {
> @@ -774,6 +779,7 @@ int kvm_unmap_hva(struct kvm *kvm, unsigned long =
hva)
> {
> 	return kvmppc_ops->unmap_hva(kvm, hva);
> }
> +EXPORT_SYMBOL_GPL(kvm_unmap_hva);
>=20
> int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned =
long end)
> {
> @@ -825,3 +831,14 @@ int kvmppc_core_check_processor_compat(void)
> {
> 	return kvmppc_ops->check_processor_compat();
> }
> +
> +EXPORT_SYMBOL_GPL(kvm_get_dirty_log);
> +EXPORT_SYMBOL_GPL(kvmppc_core_pending_dec);
> +EXPORT_SYMBOL_GPL(kvmppc_init_lpid);
> +EXPORT_SYMBOL_GPL(kvmppc_core_dequeue_dec);
> +EXPORT_SYMBOL_GPL(kvmppc_claim_lpid);
> +EXPORT_SYMBOL_GPL(kvmppc_alloc_lpid);
> +EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter);
> +EXPORT_SYMBOL_GPL(kvmppc_core_queue_dec);
> +EXPORT_SYMBOL_GPL(kvmppc_free_lpid);
> +
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c =
b/arch/powerpc/kvm/book3s_64_mmu_host.c
> index fd5b393..775d368 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
> @@ -27,6 +27,7 @@
> #include <asm/machdep.h>
> #include <asm/mmu_context.h>
> #include <asm/hw_irq.h>
> +

Stray whitespace change

> #include "trace_pr.h"
>=20
> #define PTE_SIZE 12
> diff --git a/arch/powerpc/kvm/book3s_emulate.c =
b/arch/powerpc/kvm/book3s_emulate.c
> index b9841ad..20d03c2 100644
> --- a/arch/powerpc/kvm/book3s_emulate.c
> +++ b/arch/powerpc/kvm/book3s_emulate.c
> @@ -172,7 +172,7 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, =
struct kvm_vcpu *vcpu,
> 			vcpu->arch.mmu.tlbie(vcpu, addr, large);
> 			break;
> 		}
> -#ifdef CONFIG_KVM_BOOK3S_64_PR
> +#ifdef CONFIG_KVM_BOOK3S_PR

Why?

> 		case OP_31_XOP_FAKE_SC1:
> 		{
> 			/* SC 1 papr hypercalls */
> diff --git a/arch/powerpc/kvm/book3s_hv.c =
b/arch/powerpc/kvm/book3s_hv.c
> index eec353d..ff57be8 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -53,6 +53,8 @@
> #include <linux/highmem.h>
> #include <linux/hugetlb.h>
>=20
> +#include <linux/module.h>
> +
> /* #define EXIT_DEBUG */
> /* #define EXIT_DEBUG_SIMPLE */
> /* #define EXIT_DEBUG_INT */
> @@ -2104,3 +2106,5 @@ static void kvmppc_book3s_exit_hv(void)
>=20
> module_init(kvmppc_book3s_init_hv);
> module_exit(kvmppc_book3s_exit_hv);
> +
> +MODULE_LICENSE("GPL");
> diff --git a/arch/powerpc/kvm/book3s_pr.c =
b/arch/powerpc/kvm/book3s_pr.c
> index 99d0839..df48d89 100644
> --- a/arch/powerpc/kvm/book3s_pr.c
> +++ b/arch/powerpc/kvm/book3s_pr.c
> @@ -40,6 +40,7 @@
> #include <linux/sched.h>
> #include <linux/vmalloc.h>
> #include <linux/highmem.h>
> +#include <linux/module.h>
>=20
> #define CREATE_TRACE_POINTS
> #include "trace_pr.h"
> @@ -871,7 +872,7 @@ program_interrupt:
> 			ulong cmd =3D kvmppc_get_gpr(vcpu, 3);
> 			int i;
>=20
> -#ifdef CONFIG_KVM_BOOK3S_64_PR
> +#ifdef CONFIG_PPC_BOOK3S_64

Why?


Alex

> 			if (kvmppc_h_pr(vcpu, cmd) =3D=3D EMULATE_DONE) =
{
> 				r =3D RESUME_GUEST;
> 				break;
> @@ -1567,3 +1568,5 @@ static void kvmppc_book3s_exit_pr(void)
>=20
> module_init(kvmppc_book3s_init_pr);
> module_exit(kvmppc_book3s_exit_pr);
> +
> +MODULE_LICENSE("GPL");
> diff --git a/arch/powerpc/kvm/book3s_rtas.c =
b/arch/powerpc/kvm/book3s_rtas.c
> index 3219ba8..cf95cde 100644
> --- a/arch/powerpc/kvm/book3s_rtas.c
> +++ b/arch/powerpc/kvm/book3s_rtas.c
> @@ -260,6 +260,7 @@ fail:
> 	 */
> 	return rc;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_rtas_hcall);
>=20
> void kvmppc_rtas_tokens_free(struct kvm *kvm)
> {
> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
> index 81762eb..de9a340 100644
> --- a/arch/powerpc/kvm/emulate.c
> +++ b/arch/powerpc/kvm/emulate.c
> @@ -483,3 +483,4 @@ int kvmppc_emulate_instruction(struct kvm_run =
*run, struct kvm_vcpu *vcpu)
>=20
> 	return emulated;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_emulate_instruction);
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 00a96fc..3019edc 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -126,6 +126,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>=20
> 	return r;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_prepare_to_enter);
>=20
> int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
> {
> @@ -179,6 +180,7 @@ int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
>=20
> 	return r;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_kvm_pv);
>=20
> int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
> {
> @@ -207,6 +209,7 @@ out:
> 	vcpu->arch.sane =3D r;
> 	return r ? 0 : -EINVAL;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_sanity_check);
>=20
> int kvmppc_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu)
> {
> @@ -241,6 +244,7 @@ int kvmppc_emulate_mmio(struct kvm_run *run, =
struct kvm_vcpu *vcpu)
>=20
> 	return r;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_emulate_mmio);
>=20
> int kvm_arch_hardware_enable(void *garbage)
> {
> @@ -665,6 +669,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct =
kvm_vcpu *vcpu,
>=20
> 	return EMULATE_DO_MMIO;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_handle_load);
>=20
> /* Same as above, but sign extends */
> int kvmppc_handle_loads(struct kvm_run *run, struct kvm_vcpu *vcpu,
> @@ -726,6 +731,7 @@ int kvmppc_handle_store(struct kvm_run *run, =
struct kvm_vcpu *vcpu,
>=20
> 	return EMULATE_DO_MMIO;
> }
> +EXPORT_SYMBOL_GPL(kvmppc_handle_store);
>=20
> int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run =
*run)
> {
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index bf040c4..66df1d2 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -186,6 +186,7 @@ void kvm_flush_remote_tlbs(struct kvm *kvm)
> 		++kvm->stat.remote_tlb_flush;
> 	cmpxchg(&kvm->tlbs_dirty, dirty_count, 0);
> }
> +EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs);
>=20
> void kvm_reload_remote_mmus(struct kvm *kvm)
> {
> @@ -1648,6 +1649,7 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn)
> 	memslot =3D gfn_to_memslot(kvm, gfn);
> 	mark_page_dirty_in_slot(kvm, memslot, gfn);
> }
> +EXPORT_SYMBOL_GPL(mark_page_dirty);
>=20
> /*
>  * The vCPU has executed a HLT instruction with in-kernel mode =
enabled.
> @@ -1673,6 +1675,7 @@ void kvm_vcpu_block(struct kvm_vcpu *vcpu)
>=20
> 	finish_wait(&vcpu->wq, &wait);
> }
> +EXPORT_SYMBOL_GPL(kvm_vcpu_block);
>=20
> #ifndef CONFIG_S390
> /*
> --=20
> 1.8.1.2
>=20
> --
> To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Alexander Graf]

On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

Missing patch description.

> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>

I fail to see how this really simplifies things, but at the end of the =
day it's Gleb's and Paolo's call.

Which brings me to the next issue: You forgot to CC kvm@vger on your =
patch set. Gleb and Paolo don't read kvm-ppc@vger. And they shouldn't =
have to. Every kvm patch that you want review on or that should get =
applied needs to be sent to kvm@vger. If you want to tag it as PPC =
specific patch, do so by CC'ing kvm-ppc@vger.


Alex

> ---
> arch/arm/kvm/arm.c                 |  4 ++--
> arch/ia64/kvm/kvm-ia64.c           |  4 ++--
> arch/mips/kvm/kvm_mips.c           |  6 ++----
> arch/powerpc/include/asm/kvm_ppc.h |  2 +-
> arch/powerpc/kvm/44x.c             |  2 +-
> arch/powerpc/kvm/book3s.c          | 15 ++++++++++++---
> arch/powerpc/kvm/book3s_hv.c       |  9 ++++++---
> arch/powerpc/kvm/book3s_pr.c       |  5 +++--
> arch/powerpc/kvm/e500.c            |  2 +-
> arch/powerpc/kvm/e500mc.c          |  2 +-
> arch/powerpc/kvm/powerpc.c         |  5 -----
> arch/s390/kvm/kvm-s390.c           |  3 ++-
> arch/x86/kvm/x86.c                 | 13 +++++++++++--
> include/linux/kvm_host.h           |  2 +-
> virt/kvm/kvm_main.c                | 14 +++++---------
> 15 files changed, 50 insertions(+), 38 deletions(-)
>=20
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index 9c697db..cccb121 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -109,9 +109,9 @@ void kvm_arch_hardware_unsetup(void)
> {
> }
>=20
> -void kvm_arch_check_processor_compat(void *rtn)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> -	*(int *)rtn =3D 0;
> +	return 0;
> }
>=20
> void kvm_arch_sync_events(struct kvm *kvm)
> diff --git a/arch/ia64/kvm/kvm-ia64.c b/arch/ia64/kvm/kvm-ia64.c
> index bdfd878..065942c 100644
> --- a/arch/ia64/kvm/kvm-ia64.c
> +++ b/arch/ia64/kvm/kvm-ia64.c
> @@ -185,9 +185,9 @@ void kvm_arch_hardware_disable(void *garbage)
> 	ia64_ptr_entry(0x3, slot);
> }
>=20
> -void kvm_arch_check_processor_compat(void *rtn)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> -	*(int *)rtn =3D 0;
> +	return 0;
> }
>=20
> int kvm_dev_ioctl_check_extension(long ext)
> diff --git a/arch/mips/kvm/kvm_mips.c b/arch/mips/kvm/kvm_mips.c
> index a7b0445..4512739 100644
> --- a/arch/mips/kvm/kvm_mips.c
> +++ b/arch/mips/kvm/kvm_mips.c
> @@ -97,11 +97,9 @@ void kvm_arch_hardware_unsetup(void)
> {
> }
>=20
> -void kvm_arch_check_processor_compat(void *rtn)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> -	int *r =3D (int *)rtn;
> -	*r =3D 0;
> -	return;
> +	return 0;
> }
>=20
> static void kvm_mips_init_tlbs(struct kvm *kvm)
> diff --git a/arch/powerpc/include/asm/kvm_ppc.h =
b/arch/powerpc/include/asm/kvm_ppc.h
> index 58e732f..592501b 100644
> --- a/arch/powerpc/include/asm/kvm_ppc.h
> +++ b/arch/powerpc/include/asm/kvm_ppc.h
> @@ -204,7 +204,7 @@ struct kvmppc_ops {
> 			      unsigned long npages);
> 	int (*init_vm)(struct kvm *kvm);
> 	void (*destroy_vm)(struct kvm *kvm);
> -	int (*check_processor_compat)(void);
> +	void (*check_processor_compat)(void *r);
> 	int (*get_smmu_info)(struct kvm *kvm, struct kvm_ppc_smmu_info =
*info);
> 	int (*emulate_op)(struct kvm_run *run, struct kvm_vcpu *vcpu,
> 			  unsigned int inst, int *advance);
> diff --git a/arch/powerpc/kvm/44x.c b/arch/powerpc/kvm/44x.c
> index 2f5c6b6..a1f4e60 100644
> --- a/arch/powerpc/kvm/44x.c
> +++ b/arch/powerpc/kvm/44x.c
> @@ -43,7 +43,7 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
> 	kvmppc_booke_vcpu_put(vcpu);
> }
>=20
> -int kvmppc_core_check_processor_compat(void)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> 	int r;
>=20
> diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
> index ca617e1..485a6ff 100644
> --- a/arch/powerpc/kvm/book3s.c
> +++ b/arch/powerpc/kvm/book3s.c
> @@ -827,9 +827,18 @@ void kvmppc_core_destroy_vm(struct kvm *kvm)
> #endif
> }
>=20
> -int kvmppc_core_check_processor_compat(void)
> -{
> -	return kvmppc_ops->check_processor_compat();
> +int kvm_arch_check_processor_compat(void *opaque)
> +{
> +	int r,cpu;
> +	struct kvmppc_ops *kvm_ops =3D (struct kvmppc_ops *)opaque;
> +	for_each_online_cpu(cpu) {
> +		smp_call_function_single(cpu,
> +					 =
kvm_ops->check_processor_compat,
> +					 &r, 1);
> +		if (r < 0)
> +			break;
> +	}
> +	return r;
> }
>=20
> EXPORT_SYMBOL_GPL(kvm_get_dirty_log);
> diff --git a/arch/powerpc/kvm/book3s_hv.c =
b/arch/powerpc/kvm/book3s_hv.c
> index ff57be8..4322db4 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -1980,11 +1980,14 @@ static int kvmppc_core_emulate_mfspr_hv(struct =
kvm_vcpu *vcpu, int sprn,
> 	return EMULATE_FAIL;
> }
>=20
> -static int kvmppc_core_check_processor_compat_hv(void)
> +
> +static void kvmppc_core_check_processor_compat_hv(void *r)
> {
> 	if (!cpu_has_feature(CPU_FTR_HVMODE))
> -		return -EIO;
> -	return 0;
> +		*(int *)r =3D -EIO;
> +	else
> +		*(int *)r =3D 0;
> +	return;
> }
>=20
> static long kvm_arch_vm_ioctl_hv(struct file *filp,
> diff --git a/arch/powerpc/kvm/book3s_pr.c =
b/arch/powerpc/kvm/book3s_pr.c
> index df48d89..127b961 100644
> --- a/arch/powerpc/kvm/book3s_pr.c
> +++ b/arch/powerpc/kvm/book3s_pr.c
> @@ -1490,10 +1490,11 @@ static void kvmppc_core_destroy_vm_pr(struct =
kvm *kvm)
> 	enable_relon_interrupts(kvm);
> }
>=20
> -static int kvmppc_core_check_processor_compat_pr(void)
> +static void kvmppc_core_check_processor_compat_pr(void *r)
> {
> 	/* we are always compatible */
> -	return 0;
> +	*(int *)r =3D 0;
> +	return;
> }
>=20
> static long kvm_arch_vm_ioctl_pr(struct file *filp,
> diff --git a/arch/powerpc/kvm/e500.c b/arch/powerpc/kvm/e500.c
> index ce6b73c..0681cb1 100644
> --- a/arch/powerpc/kvm/e500.c
> +++ b/arch/powerpc/kvm/e500.c
> @@ -323,7 +323,7 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
> 	kvmppc_booke_vcpu_put(vcpu);
> }
>=20
> -int kvmppc_core_check_processor_compat(void)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> 	int r;
>=20
> diff --git a/arch/powerpc/kvm/e500mc.c b/arch/powerpc/kvm/e500mc.c
> index 19c8379..48b3ba5 100644
> --- a/arch/powerpc/kvm/e500mc.c
> +++ b/arch/powerpc/kvm/e500mc.c
> @@ -169,7 +169,7 @@ void kvmppc_core_vcpu_put(struct kvm_vcpu *vcpu)
> 	kvmppc_booke_vcpu_put(vcpu);
> }
>=20
> -int kvmppc_core_check_processor_compat(void)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> 	int r;
>=20
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 3019edc..1404f4d 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -264,11 +264,6 @@ void kvm_arch_hardware_unsetup(void)
> {
> }
>=20
> -void kvm_arch_check_processor_compat(void *rtn)
> -{
> -	*(int *)rtn =3D kvmppc_core_check_processor_compat();
> -}
> -
> int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> {
> 	if (type)
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 776dafe..01d4000 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -119,8 +119,9 @@ void kvm_arch_hardware_unsetup(void)
> 	gmap_unregister_ipte_notifier(&gmap_notifier);
> }
>=20
> -void kvm_arch_check_processor_compat(void *rtn)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> +	return 0;
> }
>=20
> int kvm_arch_init(void *opaque)
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index e5ca72a..53c8308 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -6873,9 +6873,18 @@ void kvm_arch_hardware_unsetup(void)
> 	kvm_x86_ops->hardware_unsetup();
> }
>=20
> -void kvm_arch_check_processor_compat(void *rtn)
> +int kvm_arch_check_processor_compat(void *opaque)
> {
> -	kvm_x86_ops->check_processor_compatibility(rtn);
> +	int r,cpu;
> +
> +	for_each_online_cpu(cpu) {
> +		smp_call_function_single(cpu,
> +			 kvm_x86_ops->check_processor_compatibility,
> +			 &r, 1);
> +		if (r < 0)
> +			break;
> +	}
> +	return r;
> }
>=20
> bool kvm_vcpu_compatible(struct kvm_vcpu *vcpu)
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index ca645a0..459b359 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -649,7 +649,7 @@ int kvm_arch_hardware_enable(void *garbage);
> void kvm_arch_hardware_disable(void *garbage);
> int kvm_arch_hardware_setup(void);
> void kvm_arch_hardware_unsetup(void);
> -void kvm_arch_check_processor_compat(void *rtn);
> +int kvm_arch_check_processor_compat(void *opaque);
> int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
> int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
>=20
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 66df1d2..0594b22 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -3166,10 +3166,9 @@ static void kvm_sched_out(struct =
preempt_notifier *pn,
> }
>=20
> int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
> -		  struct module *module)
> +	     struct module *module)
> {
> 	int r;
> -	int cpu;
>=20
> 	r =3D kvm_arch_init(opaque);
> 	if (r)
> @@ -3195,13 +3194,10 @@ int kvm_init(void *opaque, unsigned vcpu_size, =
unsigned vcpu_align,
> 	if (r < 0)
> 		goto out_free_0a;
>=20
> -	for_each_online_cpu(cpu) {
> -		smp_call_function_single(cpu,
> -				kvm_arch_check_processor_compat,
> -				&r, 1);
> -		if (r < 0)
> -			goto out_free_1;
> -	}
> +
> +	r =3D kvm_arch_check_processor_compat(opaque);
> +	if (r < 0)
> +		goto out_free_1;
>=20
> 	r =3D register_cpu_notifier(&kvm_cpu_notifier);
> 	if (r)
> --=20
> 1.8.1.2
>=20

Re: [RFC PATCH 04/11] kvm: powerpc: book3s: Add a new config variable CONFIG_KVM_BOOK3S_HV

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>
>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>> 
>> This help ups to select the relevant code in the kernel code
>> when we later move HV and PR bits as seperate modules.
>
> I don't think I grasp what semantically the difference between
> CONFIG_KVM_BOOK3S_HV and CONFIG_KVM_BOOK3S_64_HV is :).

When we later enable kvm-hv CONFIG_KVM_BOOK3S_64_HV becomes 'm' So we
can't do #ifdef CONFIG_KVM_BOOK3S_64_HV

-aneesh

Re: [RFC PATCH 05/11] kvm: powerpc: book3s: Add kvmppc_ops callback for HV and PR specific operations

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>
>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>> 
>> This moves HV and PR specific functions to kvmppc_ops callback.
>> This is needed so that we can enable HV and PR together in the
>> same kernel. Actual changes to enable both come in the later
>> patch.This also renames almost all of the symbols that exist in both PR and HV
>> KVM for clarity. Symbols in the PR KVM implementation get "_pr"
>> appended, and those in the HV KVM implementation get "_hv".  Then,
>> in book3s.c, we add a function with the name without the suffix and
>> arrange for it to call the appropriate kvmppc_ops callback depending on
>> which kvm type we selected during VM creation.
>> 
>> NOTE: we still don't enable selecting both the HV and PR together
>> in this commit that will be done by a later commit.
>> 
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> ---
>> arch/powerpc/include/asm/kvm_book3s.h |   5 +-
>> arch/powerpc/include/asm/kvm_ppc.h    |  63 ++++++++--
>> arch/powerpc/kvm/Kconfig              |  15 ++-
>> arch/powerpc/kvm/Makefile             |   9 +-
>> arch/powerpc/kvm/book3s.c             | 145 +++++++++++++++++++++-
>> arch/powerpc/kvm/book3s_32_mmu_host.c |   2 +-
>> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
>> arch/powerpc/kvm/book3s_64_mmu_hv.c   |  17 ++-
>> arch/powerpc/kvm/book3s_emulate.c     |   8 +-
>> arch/powerpc/kvm/book3s_hv.c          | 226 +++++++++++++++++++++++++---------
>> arch/powerpc/kvm/book3s_interrupts.S  |   2 +-
>> arch/powerpc/kvm/book3s_pr.c          | 196 ++++++++++++++++++-----------
>> arch/powerpc/kvm/emulate.c            |   6 +-
>> arch/powerpc/kvm/powerpc.c            |  58 +++------
>> 14 files changed, 539 insertions(+), 215 deletions(-)
>> 
>> 
>
> [...]
>
>> @@ -888,14 +890,8 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
>> 	return r;
>> }
>> 
>> -int kvmppc_core_check_processor_compat(void)
>> -{
>> -	if (cpu_has_feature(CPU_FTR_HVMODE))
>> -		return 0;
>> -	return -EIO;
>> -}
>> -
>> -struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
>> +static struct kvm_vcpu *kvmppc_core_vcpu_create_hv(struct kvm *kvm,
>> +						   unsigned int id)
>> {
>> 	struct kvm_vcpu *vcpu;
>> 	int err = -EINVAL;
>> @@ -920,7 +916,6 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
>> 	vcpu->arch.ctrl = CTRL_RUNLATCH;
>> 	/* default to host PVR, since we can't spoof it */
>> 	vcpu->arch.pvr = mfspr(SPRN_PVR);
>> -	kvmppc_set_pvr(vcpu, vcpu->arch.pvr);
>
> Where is this one going?

That is same as the line above. 

void kvmppc_set_pvr_hv(struct kvm_vcpu *vcpu, u32 pvr)
{
	vcpu->arch.pvr = pvr;
}


>
>> 	spin_lock_init(&vcpu->arch.vpa_update_lock);
>> 	spin_lock_init(&vcpu->arch.tbacct_lock);
>> 	vcpu->arch.busy_preempt = TB_NIL;
>> @@ -972,7 +967,7 @@ static void unpin_vpa(struct kvm *kvm, struct kvmppc_vpa *vpa)
>> 					vpa->dirty);
>> }
>> 
>> -void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
>> +static void kvmppc_core_vcpu_free_hv(struct kvm_vcpu *vcpu)
>> {
>> 	spin_lock(&vcpu->arch.vpa_update_lock);
>> 	unpin_vpa(vcpu->kvm, &vcpu->arch.dtl);
>> @@ -983,6 +978,12 @@ void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
>> 	kmem_cache_free(kvm_vcpu_cache, vcpu);
>> }
>> 
>> +static int kvmppc_core_check_requests_hv(struct kvm_vcpu *vcpu)
>> +{
>> +	/* Indicate we want to get back into the guest */
>> +	return 1;
>> +}
>> +
>> 
>
> [...]
>
>> +	case KVM_PPC_GET_HTAB_FD: {
>> +		struct kvm_get_htab_fd ghf;
>> +
>> +		r = -EFAULT;
>> +		if (copy_from_user(&ghf, argp, sizeof(ghf)))
>> +			break;
>> +		r = kvm_vm_ioctl_get_htab_fd(kvm, &ghf);
>> +		break;
>> +	}
>> +
>> +	default:
>> +		r = -ENOTTY;
>> +	}
>> +
>> +	return r;
>> }
>> 
>> -static int kvmppc_book3s_hv_init(void)
>> +/* FIXME!! move to header */
>
> Hrm :)

yes, want to get something out for review. Will fix if we agree on the
approach.

>
>> +extern void kvmppc_core_flush_memslot_hv(struct kvm *kvm,
>> +					 struct kvm_memory_slot *memslot);
>> +extern int kvm_unmap_hva_hv(struct kvm *kvm, unsigned long hva);
>> +extern int kvm_unmap_hva_range_hv(struct kvm *kvm, unsigned long start,
>> +				  unsigned long end);
>> +extern int kvm_age_hva_hv(struct kvm *kvm, unsigned long hva);
>> +extern int kvm_test_age_hva_hv(struct kvm *kvm, unsigned long hva);
>> +extern void kvm_set_spte_hva_hv(struct kvm *kvm, unsigned long hva, pte_t pte);
>> +
>> +static struct kvmppc_ops kvmppc_hv_ops = {
>> +	.get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv,
>> +	.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
>> +	.get_one_reg = kvmppc_get_one_reg_hv,
>> +	.set_one_reg = kvmppc_set_one_reg_hv,
>> +	.vcpu_load   = kvmppc_core_vcpu_load_hv,
>> +	.vcpu_put    = kvmppc_core_vcpu_put_hv,
>> +	.set_msr     = kvmppc_set_msr_hv,
>> +	.vcpu_run    = kvmppc_vcpu_run_hv,
>> +	.vcpu_create = kvmppc_core_vcpu_create_hv,
>> +	.vcpu_free   = kvmppc_core_vcpu_free_hv,
>> +	.check_requests = kvmppc_core_check_requests_hv,
>> +	.get_dirty_log  = kvm_vm_ioctl_get_dirty_log_hv,
>> +	.flush_memslot  = kvmppc_core_flush_memslot_hv,
>> +	.prepare_memory_region = kvmppc_core_prepare_memory_region_hv,
>> +	.commit_memory_region  = kvmppc_core_commit_memory_region_hv,
>> +	.unmap_hva = kvm_unmap_hva_hv,
>> +	.unmap_hva_range = kvm_unmap_hva_range_hv,
>> +	.age_hva  = kvm_age_hva_hv,
>> +	.test_age_hva = kvm_test_age_hva_hv,
>> +	.set_spte_hva = kvm_set_spte_hva_hv,
>> +	.mmu_destroy  = kvmppc_mmu_destroy_hv,
>> +	.free_memslot = kvmppc_core_free_memslot_hv,
>> +	.create_memslot = kvmppc_core_create_memslot_hv,
>> +	.init_vm =  kvmppc_core_init_vm_hv,
>> +	.destroy_vm = kvmppc_core_destroy_vm_hv,
>> +	.check_processor_compat = kvmppc_core_check_processor_compat_hv,
>> +	.get_smmu_info = kvm_vm_ioctl_get_smmu_info_hv,
>> +	.emulate_op = kvmppc_core_emulate_op_hv,
>> +	.emulate_mtspr = kvmppc_core_emulate_mtspr_hv,
>> +	.emulate_mfspr = kvmppc_core_emulate_mfspr_hv,
>> +	.fast_vcpu_kick = kvmppc_fast_vcpu_kick_hv,
>> +	.arch_vm_ioctl  = kvm_arch_vm_ioctl_hv,
>> +};
>> +
>> 
>
> [...]
>
>> @@ -1390,8 +1389,42 @@ out:
>> 	return r;
>> }
>> 
>> +static void kvmppc_core_flush_memslot_pr(struct kvm *kvm,
>> +					 struct kvm_memory_slot *memslot)
>> +{
>> +	return;
>> +}
>> +
>> +static int kvmppc_core_prepare_memory_region_pr(struct kvm *kvm,
>> +					struct kvm_memory_slot *memslot,
>> +					struct kvm_userspace_memory_region *mem)
>> +{
>> +	return 0;
>> +}
>> +
>> +static void kvmppc_core_commit_memory_region_pr(struct kvm *kvm,
>> +				struct kvm_userspace_memory_region *mem,
>> +				const struct kvm_memory_slot *old)
>> +{
>> +	return;
>> +}
>> +
>> +static void kvmppc_core_free_memslot_pr(struct kvm_memory_slot *free,
>> +					struct kvm_memory_slot *dont)
>> +{
>> +	return;
>> +}
>> +
>> +static int kvmppc_core_create_memslot_pr(struct kvm_memory_slot *slot,
>> +					 unsigned long npages)
>> +{
>> +	return 0;
>> +}
>> +
>> +
>> #ifdef CONFIG_PPC64
>> -int kvm_vm_ioctl_get_smmu_info(struct kvm *kvm, struct kvm_ppc_smmu_info *info)
>> +static int kvm_vm_ioctl_get_smmu_info_pr(struct kvm *kvm,
>> +					 struct kvm_ppc_smmu_info *info)
>
> You're dereferencing this function unconditionally now, probably
> breaking book3s_32 along the way :).


will double check that.

>
> I'm not really happy with the naming scheme either, but I can't really
> think of anything better right now. In an ideal world all functions
> would still have the same names and we merely make them static and
> refer to them through structs :).

I was following rest of the kernel source there. For ex: struct
file_operations function pointers get pointed to by different fs
specific callback, they all have fs details in their name. I also found
that having _hv and _pr in the name allowed for easy grep and clarity
in what different files should contain.

-aneesh

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>
>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>> 
>> This help us to identify whether we are running with hypervisor mode KVM
>> enabled. The change is needed so that we can have both HV and PR kvm
>> enabled in the same kernel.
>> 
>> If both HV and PR KVM are included, interrupts come in to the HV version
>> of the kvmppc_interrupt code, which then jumps to the PR handler,
>> renamed to kvmppc_interrupt_pr, if the guest is a PR guest.
>> 
>> Allowing both PR and HV in the same kernel required some changes to
>> kvm_dev_ioctl_check_extension(), since the values returned now can't
>> be selected with #ifdefs as much as previously. We look at is_hv_enabled
>> to return the right value when checking for capabilities.For capabilities that
>> are only provided by HV KVM, we return the HV value only if
>> is_hv_enabled is true. For capabilities provided by PR KVM but not HV,
>> we return the PR value only if is_hv_enabled is false.
>> 
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> ---
>> arch/powerpc/include/asm/kvm_book3s.h   | 53 --------------------------------
>> arch/powerpc/include/asm/kvm_ppc.h      |  5 +--
>> arch/powerpc/kvm/Makefile               |  2 +-
>> arch/powerpc/kvm/book3s.c               | 44 +++++++++++++++++++++++++++
>> arch/powerpc/kvm/book3s_hv.c            |  1 +
>> arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 +++
>> arch/powerpc/kvm/book3s_pr.c            |  1 +
>> arch/powerpc/kvm/book3s_segment.S       |  7 ++++-
>> arch/powerpc/kvm/book3s_xics.c          |  2 +-
>> arch/powerpc/kvm/powerpc.c              | 54 ++++++++++++++++++---------------
>> 10 files changed, 90 insertions(+), 83 deletions(-)
>> 
>> diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h
>> index 3efba3c..ba33c49 100644
>> --- a/arch/powerpc/include/asm/kvm_book3s.h
>> +++ b/arch/powerpc/include/asm/kvm_book3s.h
>> @@ -297,59 +297,6 @@ static inline ulong kvmppc_get_fault_dar(struct kvm_vcpu *vcpu)
>> 	return vcpu->arch.fault_dar;
>> }
>> 
>> -#ifdef CONFIG_KVM_BOOK3S_PR
>> -
>> -static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
>> -{
>> -	return to_book3s(vcpu)->hior;
>> -}
>> -
>> -static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
>> -			unsigned long pending_now, unsigned long old_pending)
>> -{
>> -	if (pending_now)
>> -		vcpu->arch.shared->int_pending = 1;
>> -	else if (old_pending)
>> -		vcpu->arch.shared->int_pending = 0;
>> -}
>> -
>> -static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
>> -{
>> -	ulong crit_raw = vcpu->arch.shared->critical;
>> -	ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
>> -	bool crit;
>> -
>> -	/* Truncate crit indicators in 32 bit mode */
>> -	if (!(vcpu->arch.shared->msr & MSR_SF)) {
>> -		crit_raw &= 0xffffffff;
>> -		crit_r1 &= 0xffffffff;
>> -	}
>> -
>> -	/* Critical section when crit == r1 */
>> -	crit = (crit_raw == crit_r1);
>> -	/* ... and we're in supervisor mode */
>> -	crit = crit && !(vcpu->arch.shared->msr & MSR_PR);
>> -
>> -	return crit;
>> -}
>> -#else /* CONFIG_KVM_BOOK3S_PR */
>> -
>> -static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
>> -{
>> -	return 0;
>> -}
>> -
>> -static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
>> -			unsigned long pending_now, unsigned long old_pending)
>> -{
>> -}
>> -
>> -static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
>> -{
>> -	return false;
>> -}
>> -#endif
>> -
>> /* Magic register values loaded into r3 and r4 before the 'sc' assembly
>>  * instruction for the OSI hypercalls */
>> #define OSI_SC_MAGIC_R3			0x113724FA
>> diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
>> index 4d9641c..58e732f 100644
>> --- a/arch/powerpc/include/asm/kvm_ppc.h
>> +++ b/arch/powerpc/include/asm/kvm_ppc.h
>> @@ -169,6 +169,7 @@ extern int kvmppc_xics_int_on(struct kvm *kvm, u32 irq);
>> extern int kvmppc_xics_int_off(struct kvm *kvm, u32 irq);
>> 
>> struct kvmppc_ops {
>> +	bool is_hv_enabled;
>
> This doesn't really belong into an ops struct. Either you compare
>
>   if (kvmppc_ops == kvmppc_ops_pr)

will do that in the next update. 

>
> against a global known good ops struct or you put the hint somewhere into the kvm struct.
>
>> 	int (*get_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
>> 	int (*set_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);
>> 	int (*get_one_reg)(struct kvm_vcpu *vcpu, u64 id,
>> @@ -309,10 +310,10 @@ static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr)
>> 
>> static inline u32 kvmppc_get_xics_latch(void)
>> {
>> -	u32 xirr = get_paca()->kvm_hstate.saved_xirr;
>> +	u32 xirr;
>> 
>> +	xirr = get_paca()->kvm_hstate.saved_xirr;
>> 	get_paca()->kvm_hstate.saved_xirr = 0;
>> -
>
> I don't see any functionality change here?
>
>> 	return xirr;

Mistake on my side, I had a if condition in there before, which i later
removed. But forgot to move the assignment back. Will fix. 

>> }
>> 
>> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
>> index c343793..a514ecd 100644
>> --- a/arch/powerpc/kvm/Makefile
>> +++ b/arch/powerpc/kvm/Makefile
>> @@ -77,7 +77,7 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
>> 	book3s_rmhandlers.o
>> endif
>> 
>> -kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
>> +kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  += \
>
> This change looks unrelated?
>

yes. Will move it to the correct patch


>> 	book3s_hv.o \
>> 	book3s_hv_interrupts.o \
>> 	book3s_64_mmu_hv.o
>> diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
>> index bdc3f95..12f94bf 100644
>> --- a/arch/powerpc/kvm/book3s.c
>> +++ b/arch/powerpc/kvm/book3s.c
>> @@ -69,6 +69,50 @@ void kvmppc_core_load_guest_debugstate(struct kvm_vcpu *vcpu)
>> {
>> }
>> 
>> +static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
>
> Please drop the "inline" keyword on functions in .c files :). That way
> the compiler tells us about unused functions.

ok .

>
>> +{
>> +	if (!kvmppc_ops->is_hv_enabled)
>> +		return to_book3s(vcpu)->hior;
>> +	return 0;
>> +}
>> +
>> +static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
>> +			unsigned long pending_now, unsigned long old_pending)
>> +{
>> +	if (kvmppc_ops->is_hv_enabled)
>> +		return;
>> +	if (pending_now)
>> +		vcpu->arch.shared->int_pending = 1;
>> +	else if (old_pending)
>> +		vcpu->arch.shared->int_pending = 0;
>> +}
>> +

....

>> diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
>> index 1abe478..e0229dd 100644
>> --- a/arch/powerpc/kvm/book3s_segment.S
>> +++ b/arch/powerpc/kvm/book3s_segment.S
>> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
>> .global kvmppc_handler_trampoline_exit
>> kvmppc_handler_trampoline_exit:
>> 
>> +#if defined(CONFIG_KVM_BOOK3S_HV)
>> +.global kvmppc_interrupt_pr
>> +kvmppc_interrupt_pr:
>> +	ld	r9, HSTATE_SCRATCH2(r13)
>> +#else
>> .global kvmppc_interrupt
>> kvmppc_interrupt:
>
> Just always call it kvmppc_interrupt_pr and thus share at least that
> part of the code :).

But if i don't have HV enabled, we don't compile book3s_hv_rmhandlers.S
Hence don't have the kvmppc_interrupt symbol defined.

>
>> -
>> +#endif
>> 	/* Register usage at this point:
>> 	 *
>> 	 * SPRG_SCRATCH0  = guest R13
>> diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c
>> index f0c732e..fa7625d 100644
>> --- a/arch/powerpc/kvm/book3s_xics.c
>> +++ b/arch/powerpc/kvm/book3s_xics.c
>> @@ -818,7 +818,7 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req)
>> 	}
>> 
>> 	/* Check for real mode returning too hard */
>> -	if (xics->real_mode)
>> +	if (xics->real_mode && kvmppc_ops->is_hv_enabled)
>> 		return kvmppc_xics_rm_complete(vcpu, req);
>> 
>> 	switch (req) {
>> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
>> index 69b9305..00a96fc 100644
>> --- a/arch/powerpc/kvm/powerpc.c
>> +++ b/arch/powerpc/kvm/powerpc.c
>> @@ -52,7 +52,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>> 	return 1;
>> }
>> 
>> -#ifndef CONFIG_KVM_BOOK3S_64_HV
>> /*
>>  * Common checks before entering the guest world.  Call with interrupts
>>  * disabled.
>> @@ -127,7 +126,6 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>> 
>> 	return r;
>> }
>> -#endif /* CONFIG_KVM_BOOK3S_64_HV */
>> 
>> int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
>> {
>> @@ -194,11 +192,9 @@ int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
>> 	if ((vcpu->arch.cpu_type != KVM_CPU_3S_64) && vcpu->arch.papr_enabled)
>> 		goto out;
>> 
>> -#ifdef CONFIG_KVM_BOOK3S_64_HV
>> 	/* HV KVM can only do PAPR mode for now */
>> -	if (!vcpu->arch.papr_enabled)
>> +	if (!vcpu->arch.papr_enabled && kvmppc_ops->is_hv_enabled)
>> 		goto out;
>> -#endif
>> 
>> #ifdef CONFIG_KVM_BOOKE_HV
>> 	if (!cpu_has_feature(CPU_FTR_EMB_HV))
>> @@ -322,22 +318,26 @@ int kvm_dev_ioctl_check_extension(long ext)
>> 	case KVM_CAP_DEVICE_CTRL:
>> 		r = 1;
>> 		break;
>> -#ifndef CONFIG_KVM_BOOK3S_64_HV
>> 	case KVM_CAP_PPC_PAIRED_SINGLES:
>> 	case KVM_CAP_PPC_OSI:
>> 	case KVM_CAP_PPC_GET_PVINFO:
>> #if defined(CONFIG_KVM_E500V2) || defined(CONFIG_KVM_E500MC)
>> 	case KVM_CAP_SW_TLB:
>> #endif
>> -#ifdef CONFIG_KVM_MPIC
>> -	case KVM_CAP_IRQ_MPIC:
>> -#endif
>> -		r = 1;
>> +		/* We support this only for PR */
>> +		r = !kvmppc_ops->is_hv_enabled;
>
> Reading this, have you test compiled your code against e500 configs?


Not yet.


>
>> 		break;
>> +#ifdef CONFIG_KVM_MMIO
>> 	case KVM_CAP_COALESCED_MMIO:
>> 		r = KVM_COALESCED_MMIO_PAGE_OFFSET;
>> 		break;
>> #endif
>> +#ifdef CONFIG_KVM_MPIC
>> +	case KVM_CAP_IRQ_MPIC:
>> +		r = 1;
>> +		break;
>> +#endif
>> +
>> #ifdef CONFIG_PPC_BOOK3S_64
>> 	case KVM_CAP_SPAPR_TCE:
>> 	case KVM_CAP_PPC_ALLOC_HTAB:
>> @@ -348,32 +348,37 @@ int kvm_dev_ioctl_check_extension(long ext)
>> 		r = 1;
>> 		break;
>> #endif /* CONFIG_PPC_BOOK3S_64 */
>> -#ifdef CONFIG_KVM_BOOK3S_64_HV
>> +#ifdef CONFIG_KVM_BOOK3S_HV
>> 	case KVM_CAP_PPC_SMT:
>> -		r = threads_per_core;
>> +		if (kvmppc_ops->is_hv_enabled)
>> +			r = threads_per_core;
>> +		else
>> +			r = 0;
>
> 0? Or 1?
>

That check extension was not supported before on PR. So not sure what
the value should be. May be 1 is better indicating we have one thread.
Will change.

-aneesh

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>
>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>> 
>> This patch moves PR related tracepoints to a separate header. This
>> enables in converting PR to a kernel module which will be done in
>> later patches
>> 
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> ---
>> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
>> arch/powerpc/kvm/book3s_mmu_hpte.c    |   2 +-
>> arch/powerpc/kvm/book3s_pr.c          |   3 +-
>> arch/powerpc/kvm/trace.h              | 234 +--------------------------
>> arch/powerpc/kvm/trace_pr.h           | 297 ++++++++++++++++++++++++++++++++++
>> 5 files changed, 308 insertions(+), 230 deletions(-)
>> create mode 100644 arch/powerpc/kvm/trace_pr.h
>> 
>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
>> index 329a978..fd5b393 100644
>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>> @@ -27,7 +27,7 @@
>> #include <asm/machdep.h>
>> #include <asm/mmu_context.h>
>> #include <asm/hw_irq.h>
>> -#include "trace.h"
>> +#include "trace_pr.h"
>> 
>> #define PTE_SIZE 12
>> 
>> diff --git a/arch/powerpc/kvm/book3s_mmu_hpte.c b/arch/powerpc/kvm/book3s_mmu_hpte.c
>> index d2d280b..4556168 100644
>> --- a/arch/powerpc/kvm/book3s_mmu_hpte.c
>> +++ b/arch/powerpc/kvm/book3s_mmu_hpte.c
>> @@ -28,7 +28,7 @@
>> #include <asm/mmu_context.h>
>> #include <asm/hw_irq.h>
>> 
>> -#include "trace.h"
>> +#include "trace_pr.h"
>> 
>> #define PTE_SIZE	12
>> 
>> diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
>> index 2a97279..99d0839 100644
>> --- a/arch/powerpc/kvm/book3s_pr.c
>> +++ b/arch/powerpc/kvm/book3s_pr.c
>> @@ -41,7 +41,8 @@
>> #include <linux/vmalloc.h>
>> #include <linux/highmem.h>
>> 
>> -#include "trace.h"
>> +#define CREATE_TRACE_POINTS
>> +#include "trace_pr.h"
>> 
>> /* #define EXIT_DEBUG */
>> /* #define DEBUG_EXT */
>> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
>> index a088e9a..7d5a136 100644
>> --- a/arch/powerpc/kvm/trace.h
>> +++ b/arch/powerpc/kvm/trace.h
>> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
>> 	{41, "HV_PRIV"}
>> #endif
>> 
>> +#ifndef CONFIG_KVM_BOOK3S_PR
>> +/*
>> + * For pr we define this in trace_pr.h since it pr can be built as
>> + * a module
>
> Not sure I understand the need. If the config option is available, so
> should the struct field. Worst case that happens with HV is that we
> get empty shadow_srr1 values in our trace, no?

That is not the real reason. trace.h get built as part of kvm.ko or as
part of kernel. These trace functions actually get called from
kvm-pr.ko. To make they build i would either need EXPORT_SYMBOL or move
the definition of them to kvm-pr.ko. I did the later and moved only pr
related traces to kvm-pr.ko

>
> If your goal is to make it more obvious whether we are tracing in PR
> or HV land (which is a reasonable goal), then you should also split
> off all non-common trace points into a special hv trace header so that
> it's obvious whether we are looking at HV or PR.

-aneesh

Re: [RFC PATCH 08/11] kvm: powerpc: book3s: Support building HV and PR KVM as module

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>
>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
>> index fd5b393..775d368 100644
>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>> @@ -27,6 +27,7 @@
>> #include <asm/machdep.h>
>> #include <asm/mmu_context.h>
>> #include <asm/hw_irq.h>
>> +
>
> Stray whitespace change
>

will fix

>> #include "trace_pr.h"
>> 
>> #define PTE_SIZE 12
>> diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c
>> index b9841ad..20d03c2 100644
>> --- a/arch/powerpc/kvm/book3s_emulate.c
>> +++ b/arch/powerpc/kvm/book3s_emulate.c
>> @@ -172,7 +172,7 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
>> 			vcpu->arch.mmu.tlbie(vcpu, addr, large);
>> 			break;
>> 		}
>> -#ifdef CONFIG_KVM_BOOK3S_64_PR
>> +#ifdef CONFIG_KVM_BOOK3S_PR
>
> Why?

If i have CONFIG_KVM_BOOK3S_64_PR=m  #ifdef CONFIG_KVM_BOOK3S_64_PR will
not work. There is a runtime check I can use IS_ENABLED(). But didn't
want to do those. Hence moved to the symbol which will be set as
CONFIG_KVM_BOOK3S_PR = y

-aneesh

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>
>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> Missing patch description.
>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>
> I fail to see how this really simplifies things, but at the end of the
> day it's Gleb's and Paolo's call.

will do. It avoid calling 

	for_each_online_cpu(cpu) {
		smp_call_function_single() 

on multiple architecture.

We also want to make the smp call function a callback of opaque. Hence
this should be made arch specific. 

int kvm_arch_check_processor_compat(void *opaque)
{
	int r,cpu;
	struct kvmppc_ops *kvm_ops = (struct kvmppc_ops *)opaque;
	for_each_online_cpu(cpu) {
		smp_call_function_single(cpu,
					 kvm_ops->check_processor_compat,
					 &r, 1);
		if (r < 0)
			break;
	}
	return r;
}

against

-	for_each_online_cpu(cpu) {
-		smp_call_function_single(cpu,
-				kvm_arch_check_processor_compat,
-				&r, 1);
-		if (r < 0)
-			goto out_free_1;
-	}
+
+	r = kvm_arch_check_processor_compat(opaque);
+	if (r < 0)
+		goto out_free_1;



>
> Which brings me to the next issue: You forgot to CC kvm@vger on your
> patch set. Gleb and Paolo don't read kvm-ppc@vger. And they shouldn't
> have to. Every kvm patch that you want review on or that should get
> applied needs to be sent to kvm@vger. If you want to tag it as PPC
> specific patch, do so by CC'ing kvm-ppc@vger.

Will do in the next update

-aneesh

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Paolo Bonzini]

Il 27/09/2013 15:13, Aneesh Kumar K.V ha scritto:
> Alexander Graf <agraf@suse.de> writes:
> 
>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>
>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>>
>> Missing patch description.
>>
>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>>
>> I fail to see how this really simplifies things, but at the end of the
>> day it's Gleb's and Paolo's call.
> 
> will do. It avoid calling 
> 
> 	for_each_online_cpu(cpu) {
> 		smp_call_function_single() 
> 
> on multiple architecture.

I agree with Alex.

The current code is not specially awesome; having
kvm_arch_check_processor_compat take an int* disguised as a void* is a
bit ugly indeed.

However, the API makes sense and tells you that it is being passed as a
callback (to smp_call_function_single in this case).

You are making the API more complicated to use on the arch layer
(because arch maintainers now have to think "do I need to check this on
all online CPUs?") and making the "leaf" POWER code less legible because
it still has the weird void()(void *) calling convention.

If anything, you could change kvm_arch_check_processor_compat to return
an int and accept no argument, and introduce a wrapper that kvm_init
passes to smp_call_function_single.

Paolo

> We also want to make the smp call function a callback of opaque. Hence
> this should be made arch specific. 
> 
> int kvm_arch_check_processor_compat(void *opaque)
> {
> 	int r,cpu;
> 	struct kvmppc_ops *kvm_ops = (struct kvmppc_ops *)opaque;
> 	for_each_online_cpu(cpu) {
> 		smp_call_function_single(cpu,
> 					 kvm_ops->check_processor_compat,
> 					 &r, 1);
> 		if (r < 0)
> 			break;
> 	}
> 	return r;
> }
> 
> against
> 
> -	for_each_online_cpu(cpu) {
> -		smp_call_function_single(cpu,
> -				kvm_arch_check_processor_compat,
> -				&r, 1);
> -		if (r < 0)
> -			goto out_free_1;
> -	}
> +
> +	r = kvm_arch_check_processor_compat(opaque);
> +	if (r < 0)
> +		goto out_free_1;
> 
> 
> 
>>
>> Which brings me to the next issue: You forgot to CC kvm@vger on your
>> patch set. Gleb and Paolo don't read kvm-ppc@vger. And they shouldn't
>> have to. Every kvm patch that you want review on or that should get
>> applied needs to be sent to kvm@vger. If you want to tag it as PPC
>> specific patch, do so by CC'ing kvm-ppc@vger.
> 
> Will do in the next update
> 
> -aneesh
> 

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Aneesh Kumar K.V]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 27/09/2013 15:13, Aneesh Kumar K.V ha scritto:
>> Alexander Graf <agraf@suse.de> writes:
>> 
>>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>>
>>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>>>
>>> Missing patch description.
>>>
>>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>>>
>>> I fail to see how this really simplifies things, but at the end of the
>>> day it's Gleb's and Paolo's call.
>> 
>> will do. It avoid calling 
>> 
>> 	for_each_online_cpu(cpu) {
>> 		smp_call_function_single() 
>> 
>> on multiple architecture.
>
> I agree with Alex.
>
> The current code is not specially awesome; having
> kvm_arch_check_processor_compat take an int* disguised as a void* is a
> bit ugly indeed.
>
> However, the API makes sense and tells you that it is being passed as a
> callback (to smp_call_function_single in this case).

But whether to check on all cpus or not is arch dependent right?.
IIUC only x86 and ppc64 need to do that. Also on ppc64 it really
depends on whether HV or PR. We need to check on all cpus only if it is
HV. 

>
> You are making the API more complicated to use on the arch layer
> (because arch maintainers now have to think "do I need to check this on
> all online CPUs?") and making the "leaf" POWER code less legible because
> it still has the weird void()(void *) calling convention.
>

IIUC what we wanted to check is to find out whether kvm can run on this
system. That is really an arch specific check. So for core kvm the call
should be a simple 

if (kvm_arch_check_process_compat() < 0)
        error;

Now how each arch figure out whether kvm can run on this system should
be arch specific. For x86 we do need to check all the cpus. On ppc64 for
HV we need to. For other archs we always allow kvm. 


> If anything, you could change kvm_arch_check_processor_compat to return
> an int and accept no argument, and introduce a wrapper that kvm_init
> passes to smp_call_function_single.

What i am suggesting in the patch is to avoid calling
smp_call_function_single from kvm_init and let arch decide whether to
check on all cpus or not.

-aneesh

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Gleb Natapov]

On Sat, Sep 28, 2013 at 09:06:47PM +0530, Aneesh Kumar K.V wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
> > Il 27/09/2013 15:13, Aneesh Kumar K.V ha scritto:
> >> Alexander Graf <agraf@suse.de> writes:
> >> 
> >>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
> >>>
> >>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
> >>>
> >>> Missing patch description.
> >>>
> >>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> >>>
> >>> I fail to see how this really simplifies things, but at the end of the
> >>> day it's Gleb's and Paolo's call.
> >> 
> >> will do. It avoid calling 
> >> 
> >> 	for_each_online_cpu(cpu) {
> >> 		smp_call_function_single() 
> >> 
> >> on multiple architecture.
> >
> > I agree with Alex.
> >
> > The current code is not specially awesome; having
> > kvm_arch_check_processor_compat take an int* disguised as a void* is a
> > bit ugly indeed.
> >
> > However, the API makes sense and tells you that it is being passed as a
> > callback (to smp_call_function_single in this case).
> 
> But whether to check on all cpus or not is arch dependent right?.
> IIUC only x86 and ppc64 need to do that. Also on ppc64 it really
> depends on whether HV or PR. We need to check on all cpus only if it is
> HV. 
> 
> >
> > You are making the API more complicated to use on the arch layer
> > (because arch maintainers now have to think "do I need to check this on
> > all online CPUs?") and making the "leaf" POWER code less legible because
> > it still has the weird void()(void *) calling convention.
> >
> 
> IIUC what we wanted to check is to find out whether kvm can run on this
> system. That is really an arch specific check. So for core kvm the call
> should be a simple 
> 
> if (kvm_arch_check_process_compat() < 0)
>         error;
We have that already, just return error from kvm_arch_hardware_setup. This
is specific processor compatibility check and you are arguing that the
processor check should be part of kvm_arch_hardware_setup().

> 
> Now how each arch figure out whether kvm can run on this system should
> be arch specific. For x86 we do need to check all the cpus. On ppc64 for
> HV we need to. For other archs we always allow kvm. 
> 
This is really a sanity check. Theoretically on x86 we also should
not need to check all cpus since SMP configuration with different cpu
models is not supported by the architecture (AFAIK), but bugs happen
(BIOS bugs may cause difference in capabilities for instance). So some
arches opted out from this sanity check for now and this is their choice,
but the code makes it explicit what are we checking here.

> 
> > If anything, you could change kvm_arch_check_processor_compat to return
> > an int and accept no argument, and introduce a wrapper that kvm_init
> > passes to smp_call_function_single.
> 
> What i am suggesting in the patch is to avoid calling
> smp_call_function_single from kvm_init and let arch decide whether to
> check on all cpus or not.
> 
> -aneesh

--
			Gleb.

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Aneesh Kumar K.V]

Gleb Natapov <gleb@redhat.com> writes:

> On Sat, Sep 28, 2013 at 09:06:47PM +0530, Aneesh Kumar K.V wrote:
>> Paolo Bonzini <pbonzini@redhat.com> writes:
>> 
>> > Il 27/09/2013 15:13, Aneesh Kumar K.V ha scritto:
>> >> Alexander Graf <agraf@suse.de> writes:
>> >> 
>> >>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>> >>>
>> >>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>> >>>
>> >>> Missing patch description.
>> >>>
>> >>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> >>>
>> >>> I fail to see how this really simplifies things, but at the end of the
>> >>> day it's Gleb's and Paolo's call.
>> >> 
>> >> will do. It avoid calling 
>> >> 
>> >> 	for_each_online_cpu(cpu) {
>> >> 		smp_call_function_single() 
>> >> 
>> >> on multiple architecture.
>> >
>> > I agree with Alex.
>> >
>> > The current code is not specially awesome; having
>> > kvm_arch_check_processor_compat take an int* disguised as a void* is a
>> > bit ugly indeed.
>> >
>> > However, the API makes sense and tells you that it is being passed as a
>> > callback (to smp_call_function_single in this case).
>> 
>> But whether to check on all cpus or not is arch dependent right?.
>> IIUC only x86 and ppc64 need to do that. Also on ppc64 it really
>> depends on whether HV or PR. We need to check on all cpus only if it is
>> HV. 
>> 
>> >
>> > You are making the API more complicated to use on the arch layer
>> > (because arch maintainers now have to think "do I need to check this on
>> > all online CPUs?") and making the "leaf" POWER code less legible because
>> > it still has the weird void()(void *) calling convention.
>> >
>> 
>> IIUC what we wanted to check is to find out whether kvm can run on this
>> system. That is really an arch specific check. So for core kvm the call
>> should be a simple 
>> 
>> if (kvm_arch_check_process_compat() < 0)
>>         error;
> We have that already, just return error from kvm_arch_hardware_setup. This
> is specific processor compatibility check and you are arguing that the
> processor check should be part of kvm_arch_hardware_setup().


What about the success case ?. ie, on arch like arm we do

void kvm_arch_check_processor_compat(void *rtn)
{
	*(int *)rtn = 0;
}

for_each_online_cpu(cpu) {
	smp_call_function_single(cpu,
			kvm_arch_check_processor_compat,
			&r, 1);
	if (r < 0)
		goto out_free_1;
}

There is no need to do that for loop for arm. 

The only reason I wanted this patch in the series is to make
kvm_arch_check_processor_compat take additional argument opaque. 
I am dropping that requirement in the last patch. Considering
that we have objection to this one, I will drop this patch in
the next posting by rearranging the patches.

-aneesh

Re: [RFC PATCH 09/11] kvm: simplify processor compat check

[Gleb Natapov]

On Sun, Sep 29, 2013 at 08:35:16PM +0530, Aneesh Kumar K.V wrote:
> Gleb Natapov <gleb@redhat.com> writes:
> 
> > On Sat, Sep 28, 2013 at 09:06:47PM +0530, Aneesh Kumar K.V wrote:
> >> Paolo Bonzini <pbonzini@redhat.com> writes:
> >> 
> >> > Il 27/09/2013 15:13, Aneesh Kumar K.V ha scritto:
> >> >> Alexander Graf <agraf@suse.de> writes:
> >> >> 
> >> >>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
> >> >>>
> >> >>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
> >> >>>
> >> >>> Missing patch description.
> >> >>>
> >> >>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> >> >>>
> >> >>> I fail to see how this really simplifies things, but at the end of the
> >> >>> day it's Gleb's and Paolo's call.
> >> >> 
> >> >> will do. It avoid calling 
> >> >> 
> >> >> 	for_each_online_cpu(cpu) {
> >> >> 		smp_call_function_single() 
> >> >> 
> >> >> on multiple architecture.
> >> >
> >> > I agree with Alex.
> >> >
> >> > The current code is not specially awesome; having
> >> > kvm_arch_check_processor_compat take an int* disguised as a void* is a
> >> > bit ugly indeed.
> >> >
> >> > However, the API makes sense and tells you that it is being passed as a
> >> > callback (to smp_call_function_single in this case).
> >> 
> >> But whether to check on all cpus or not is arch dependent right?.
> >> IIUC only x86 and ppc64 need to do that. Also on ppc64 it really
> >> depends on whether HV or PR. We need to check on all cpus only if it is
> >> HV. 
> >> 
> >> >
> >> > You are making the API more complicated to use on the arch layer
> >> > (because arch maintainers now have to think "do I need to check this on
> >> > all online CPUs?") and making the "leaf" POWER code less legible because
> >> > it still has the weird void()(void *) calling convention.
> >> >
> >> 
> >> IIUC what we wanted to check is to find out whether kvm can run on this
> >> system. That is really an arch specific check. So for core kvm the call
> >> should be a simple 
> >> 
> >> if (kvm_arch_check_process_compat() < 0)
> >>         error;
> > We have that already, just return error from kvm_arch_hardware_setup. This
> > is specific processor compatibility check and you are arguing that the
> > processor check should be part of kvm_arch_hardware_setup().
> 
> 
> What about the success case ?. ie, on arch like arm we do
> 
> void kvm_arch_check_processor_compat(void *rtn)
> {
> 	*(int *)rtn = 0;
> }
> 
> for_each_online_cpu(cpu) {
As I said they opted out from doing the check. They may reconsider after
first bad HW will be discovered.

> 	smp_call_function_single(cpu,
> 			kvm_arch_check_processor_compat,
> 			&r, 1);
> 	if (r < 0)
> 		goto out_free_1;
> }
> 
> There is no need to do that for loop for arm. 
It's done once during module initialisation. Why is this a big deal?

--
			Gleb.

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Alexander Graf]

On 27.09.2013, at 15:06, Aneesh Kumar K.V wrote:

> Alexander Graf <agraf@suse.de> writes:
>=20
>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>=20
>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>>>=20
>>> This patch moves PR related tracepoints to a separate header. This
>>> enables in converting PR to a kernel module which will be done in
>>> later patches
>>>=20
>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>>> ---
>>> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
>>> arch/powerpc/kvm/book3s_mmu_hpte.c    |   2 +-
>>> arch/powerpc/kvm/book3s_pr.c          |   3 +-
>>> arch/powerpc/kvm/trace.h              | 234 =
+--------------------------
>>> arch/powerpc/kvm/trace_pr.h           | 297 =
++++++++++++++++++++++++++++++++++
>>> 5 files changed, 308 insertions(+), 230 deletions(-)
>>> create mode 100644 arch/powerpc/kvm/trace_pr.h
>>>=20
>>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c =
b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>> index 329a978..fd5b393 100644
>>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>> @@ -27,7 +27,7 @@
>>> #include <asm/machdep.h>
>>> #include <asm/mmu_context.h>
>>> #include <asm/hw_irq.h>
>>> -#include "trace.h"
>>> +#include "trace_pr.h"
>>>=20
>>> #define PTE_SIZE 12
>>>=20
>>> diff --git a/arch/powerpc/kvm/book3s_mmu_hpte.c =
b/arch/powerpc/kvm/book3s_mmu_hpte.c
>>> index d2d280b..4556168 100644
>>> --- a/arch/powerpc/kvm/book3s_mmu_hpte.c
>>> +++ b/arch/powerpc/kvm/book3s_mmu_hpte.c
>>> @@ -28,7 +28,7 @@
>>> #include <asm/mmu_context.h>
>>> #include <asm/hw_irq.h>
>>>=20
>>> -#include "trace.h"
>>> +#include "trace_pr.h"
>>>=20
>>> #define PTE_SIZE	12
>>>=20
>>> diff --git a/arch/powerpc/kvm/book3s_pr.c =
b/arch/powerpc/kvm/book3s_pr.c
>>> index 2a97279..99d0839 100644
>>> --- a/arch/powerpc/kvm/book3s_pr.c
>>> +++ b/arch/powerpc/kvm/book3s_pr.c
>>> @@ -41,7 +41,8 @@
>>> #include <linux/vmalloc.h>
>>> #include <linux/highmem.h>
>>>=20
>>> -#include "trace.h"
>>> +#define CREATE_TRACE_POINTS
>>> +#include "trace_pr.h"
>>>=20
>>> /* #define EXIT_DEBUG */
>>> /* #define DEBUG_EXT */
>>> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
>>> index a088e9a..7d5a136 100644
>>> --- a/arch/powerpc/kvm/trace.h
>>> +++ b/arch/powerpc/kvm/trace.h
>>> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
>>> 	{41, "HV_PRIV"}
>>> #endif
>>>=20
>>> +#ifndef CONFIG_KVM_BOOK3S_PR
>>> +/*
>>> + * For pr we define this in trace_pr.h since it pr can be built as
>>> + * a module
>>=20
>> Not sure I understand the need. If the config option is available, so
>> should the struct field. Worst case that happens with HV is that we
>> get empty shadow_srr1 values in our trace, no?
>=20
> That is not the real reason. trace.h get built as part of kvm.ko or as
> part of kernel. These trace functions actually get called from
> kvm-pr.ko. To make they build i would either need EXPORT_SYMBOL or =
move
> the definition of them to kvm-pr.ko. I did the later and moved only pr
> related traces to kvm-pr.ko

I fail to see why we wouldn't have a trace_hv.h file then, as that can =
also be built as a module, no? And at that point I don't see why we =
would need any conditionals at all in trace.h anymore, as it would only =
cover generic code.


Alex

Re: [RFC PATCH 08/11] kvm: powerpc: book3s: Support building HV and PR KVM as module

[Alexander Graf]

On 27.09.2013, at 15:08, Aneesh Kumar K.V wrote:

> Alexander Graf <agraf@suse.de> writes:
>=20
>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>=20
>>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c =
b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>> index fd5b393..775d368 100644
>>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>> @@ -27,6 +27,7 @@
>>> #include <asm/machdep.h>
>>> #include <asm/mmu_context.h>
>>> #include <asm/hw_irq.h>
>>> +
>>=20
>> Stray whitespace change
>>=20
>=20
> will fix
>=20
>>> #include "trace_pr.h"
>>>=20
>>> #define PTE_SIZE 12
>>> diff --git a/arch/powerpc/kvm/book3s_emulate.c =
b/arch/powerpc/kvm/book3s_emulate.c
>>> index b9841ad..20d03c2 100644
>>> --- a/arch/powerpc/kvm/book3s_emulate.c
>>> +++ b/arch/powerpc/kvm/book3s_emulate.c
>>> @@ -172,7 +172,7 @@ int kvmppc_core_emulate_op_pr(struct kvm_run =
*run, struct kvm_vcpu *vcpu,
>>> 			vcpu->arch.mmu.tlbie(vcpu, addr, large);
>>> 			break;
>>> 		}
>>> -#ifdef CONFIG_KVM_BOOK3S_64_PR
>>> +#ifdef CONFIG_KVM_BOOK3S_PR
>>=20
>> Why?
>=20
> If i have CONFIG_KVM_BOOK3S_64_PR=3Dm  #ifdef CONFIG_KVM_BOOK3S_64_PR =
will
> not work. There is a runtime check I can use IS_ENABLED(). But didn't
> want to do those. Hence moved to the symbol which will be set as
> CONFIG_KVM_BOOK3S_PR =3D y

Ah, I see. The naming starts to get really confusing. Could you please =
give this a slightly better name, such as CONFIG_KVM_BOOK3S_PR_POSSIBLE =
or such, so that it's obvious that this #ifdef doesn't mean "option is =
enabled", but instead means "option can potentially be enabled through a =
module".


Alex

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Alexander Graf]

On 27.09.2013, at 15:03, Aneesh Kumar K.V wrote:

> Alexander Graf <agraf@suse.de> writes:
>=20
>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>=20
>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>>>=20
>>> This help us to identify whether we are running with hypervisor mode =
KVM
>>> enabled. The change is needed so that we can have both HV and PR kvm
>>> enabled in the same kernel.
>>>=20
>>> If both HV and PR KVM are included, interrupts come in to the HV =
version
>>> of the kvmppc_interrupt code, which then jumps to the PR handler,
>>> renamed to kvmppc_interrupt_pr, if the guest is a PR guest.
>>>=20
>>> Allowing both PR and HV in the same kernel required some changes to
>>> kvm_dev_ioctl_check_extension(), since the values returned now can't
>>> be selected with #ifdefs as much as previously. We look at =
is_hv_enabled
>>> to return the right value when checking for capabilities.For =
capabilities that
>>> are only provided by HV KVM, we return the HV value only if
>>> is_hv_enabled is true. For capabilities provided by PR KVM but not =
HV,
>>> we return the PR value only if is_hv_enabled is false.
>>>=20
>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>>> ---
>>> arch/powerpc/include/asm/kvm_book3s.h   | 53 =
--------------------------------
>>> arch/powerpc/include/asm/kvm_ppc.h      |  5 +--
>>> arch/powerpc/kvm/Makefile               |  2 +-
>>> arch/powerpc/kvm/book3s.c               | 44 =
+++++++++++++++++++++++++++
>>> arch/powerpc/kvm/book3s_hv.c            |  1 +
>>> arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 +++
>>> arch/powerpc/kvm/book3s_pr.c            |  1 +
>>> arch/powerpc/kvm/book3s_segment.S       |  7 ++++-
>>> arch/powerpc/kvm/book3s_xics.c          |  2 +-
>>> arch/powerpc/kvm/powerpc.c              | 54 =
++++++++++++++++++---------------
>>> 10 files changed, 90 insertions(+), 83 deletions(-)
>>>=20
>>> diff --git a/arch/powerpc/include/asm/kvm_book3s.h =
b/arch/powerpc/include/asm/kvm_book3s.h
>>> index 3efba3c..ba33c49 100644
>>> --- a/arch/powerpc/include/asm/kvm_book3s.h
>>> +++ b/arch/powerpc/include/asm/kvm_book3s.h
>>> @@ -297,59 +297,6 @@ static inline ulong kvmppc_get_fault_dar(struct =
kvm_vcpu *vcpu)
>>> 	return vcpu->arch.fault_dar;
>>> }
>>>=20
>>> -#ifdef CONFIG_KVM_BOOK3S_PR
>>> -
>>> -static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu =
*vcpu)
>>> -{
>>> -	return to_book3s(vcpu)->hior;
>>> -}
>>> -
>>> -static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
>>> -			unsigned long pending_now, unsigned long =
old_pending)
>>> -{
>>> -	if (pending_now)
>>> -		vcpu->arch.shared->int_pending =3D 1;
>>> -	else if (old_pending)
>>> -		vcpu->arch.shared->int_pending =3D 0;
>>> -}
>>> -
>>> -static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
>>> -{
>>> -	ulong crit_raw =3D vcpu->arch.shared->critical;
>>> -	ulong crit_r1 =3D kvmppc_get_gpr(vcpu, 1);
>>> -	bool crit;
>>> -
>>> -	/* Truncate crit indicators in 32 bit mode */
>>> -	if (!(vcpu->arch.shared->msr & MSR_SF)) {
>>> -		crit_raw &=3D 0xffffffff;
>>> -		crit_r1 &=3D 0xffffffff;
>>> -	}
>>> -
>>> -	/* Critical section when crit =3D=3D r1 */
>>> -	crit =3D (crit_raw =3D=3D crit_r1);
>>> -	/* ... and we're in supervisor mode */
>>> -	crit =3D crit && !(vcpu->arch.shared->msr & MSR_PR);
>>> -
>>> -	return crit;
>>> -}
>>> -#else /* CONFIG_KVM_BOOK3S_PR */
>>> -
>>> -static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu =
*vcpu)
>>> -{
>>> -	return 0;
>>> -}
>>> -
>>> -static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
>>> -			unsigned long pending_now, unsigned long =
old_pending)
>>> -{
>>> -}
>>> -
>>> -static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
>>> -{
>>> -	return false;
>>> -}
>>> -#endif
>>> -
>>> /* Magic register values loaded into r3 and r4 before the 'sc' =
assembly
>>> * instruction for the OSI hypercalls */
>>> #define OSI_SC_MAGIC_R3			0x113724FA
>>> diff --git a/arch/powerpc/include/asm/kvm_ppc.h =
b/arch/powerpc/include/asm/kvm_ppc.h
>>> index 4d9641c..58e732f 100644
>>> --- a/arch/powerpc/include/asm/kvm_ppc.h
>>> +++ b/arch/powerpc/include/asm/kvm_ppc.h
>>> @@ -169,6 +169,7 @@ extern int kvmppc_xics_int_on(struct kvm *kvm, =
u32 irq);
>>> extern int kvmppc_xics_int_off(struct kvm *kvm, u32 irq);
>>>=20
>>> struct kvmppc_ops {
>>> +	bool is_hv_enabled;
>>=20
>> This doesn't really belong into an ops struct. Either you compare
>>=20
>>  if (kvmppc_ops =3D=3D kvmppc_ops_pr)
>=20
> will do that in the next update.=20
>=20
>>=20
>> against a global known good ops struct or you put the hint somewhere =
into the kvm struct.
>>=20
>>> 	int (*get_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs =
*sregs);
>>> 	int (*set_sregs)(struct kvm_vcpu *vcpu, struct kvm_sregs =
*sregs);
>>> 	int (*get_one_reg)(struct kvm_vcpu *vcpu, u64 id,
>>> @@ -309,10 +310,10 @@ static inline void kvmppc_set_xics_phys(int =
cpu, unsigned long addr)
>>>=20
>>> static inline u32 kvmppc_get_xics_latch(void)
>>> {
>>> -	u32 xirr =3D get_paca()->kvm_hstate.saved_xirr;
>>> +	u32 xirr;
>>>=20
>>> +	xirr =3D get_paca()->kvm_hstate.saved_xirr;
>>> 	get_paca()->kvm_hstate.saved_xirr =3D 0;
>>> -
>>=20
>> I don't see any functionality change here?
>>=20
>>> 	return xirr;
>=20
> Mistake on my side, I had a if condition in there before, which i =
later
> removed. But forgot to move the assignment back. Will fix.=20
>=20
>>> }
>>>=20
>>> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
>>> index c343793..a514ecd 100644
>>> --- a/arch/powerpc/kvm/Makefile
>>> +++ b/arch/powerpc/kvm/Makefile
>>> @@ -77,7 +77,7 @@ =
kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) +=3D \
>>> 	book3s_rmhandlers.o
>>> endif
>>>=20
>>> -kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) :=3D \
>>> +kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV)  +=3D \
>>=20
>> This change looks unrelated?
>>=20
>=20
> yes. Will move it to the correct patch
>=20
>=20
>>> 	book3s_hv.o \
>>> 	book3s_hv_interrupts.o \
>>> 	book3s_64_mmu_hv.o
>>> diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
>>> index bdc3f95..12f94bf 100644
>>> --- a/arch/powerpc/kvm/book3s.c
>>> +++ b/arch/powerpc/kvm/book3s.c
>>> @@ -69,6 +69,50 @@ void kvmppc_core_load_guest_debugstate(struct =
kvm_vcpu *vcpu)
>>> {
>>> }
>>>=20
>>> +static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu =
*vcpu)
>>=20
>> Please drop the "inline" keyword on functions in .c files :). That =
way
>> the compiler tells us about unused functions.
>=20
> ok .
>=20
>>=20
>>> +{
>>> +	if (!kvmppc_ops->is_hv_enabled)
>>> +		return to_book3s(vcpu)->hior;
>>> +	return 0;
>>> +}
>>> +
>>> +static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
>>> +			unsigned long pending_now, unsigned long =
old_pending)
>>> +{
>>> +	if (kvmppc_ops->is_hv_enabled)
>>> +		return;
>>> +	if (pending_now)
>>> +		vcpu->arch.shared->int_pending =3D 1;
>>> +	else if (old_pending)
>>> +		vcpu->arch.shared->int_pending =3D 0;
>>> +}
>>> +
>=20
> ....
>=20
>>> diff --git a/arch/powerpc/kvm/book3s_segment.S =
b/arch/powerpc/kvm/book3s_segment.S
>>> index 1abe478..e0229dd 100644
>>> --- a/arch/powerpc/kvm/book3s_segment.S
>>> +++ b/arch/powerpc/kvm/book3s_segment.S
>>> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
>>> .global kvmppc_handler_trampoline_exit
>>> kvmppc_handler_trampoline_exit:
>>>=20
>>> +#if defined(CONFIG_KVM_BOOK3S_HV)
>>> +.global kvmppc_interrupt_pr
>>> +kvmppc_interrupt_pr:
>>> +	ld	r9, HSTATE_SCRATCH2(r13)
>>> +#else
>>> .global kvmppc_interrupt
>>> kvmppc_interrupt:
>>=20
>> Just always call it kvmppc_interrupt_pr and thus share at least that
>> part of the code :).
>=20
> But if i don't have HV enabled, we don't compile =
book3s_hv_rmhandlers.S
> Hence don't have the kvmppc_interrupt symbol defined.

Ah, because we're always jumping to kvmppc_interrupt. Can we make this =
slightly less magical? How about we always call kvmppc_interrupt_hv when =
CONFIG_KVM_BOOK3S_HV_POSSIBLE and always kvmppc_interrupt_pr when =
CONFIG_KVM_BOOK3S_PR_POSSIBLE and then branch to kvmppc_interrupt_pr =
from kvmppc_interrupt_hv?

IMHO that would make the code flow more obvious.

>=20
>>=20
>>> -
>>> +#endif
>>> 	/* Register usage at this point:
>>> 	 *
>>> 	 * SPRG_SCRATCH0  =3D guest R13
>>> diff --git a/arch/powerpc/kvm/book3s_xics.c =
b/arch/powerpc/kvm/book3s_xics.c
>>> index f0c732e..fa7625d 100644
>>> --- a/arch/powerpc/kvm/book3s_xics.c
>>> +++ b/arch/powerpc/kvm/book3s_xics.c
>>> @@ -818,7 +818,7 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 =
req)
>>> 	}
>>>=20
>>> 	/* Check for real mode returning too hard */
>>> -	if (xics->real_mode)
>>> +	if (xics->real_mode && kvmppc_ops->is_hv_enabled)
>>> 		return kvmppc_xics_rm_complete(vcpu, req);
>>>=20
>>> 	switch (req) {
>>> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
>>> index 69b9305..00a96fc 100644
>>> --- a/arch/powerpc/kvm/powerpc.c
>>> +++ b/arch/powerpc/kvm/powerpc.c
>>> @@ -52,7 +52,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu =
*vcpu)
>>> 	return 1;
>>> }
>>>=20
>>> -#ifndef CONFIG_KVM_BOOK3S_64_HV
>>> /*
>>> * Common checks before entering the guest world.  Call with =
interrupts
>>> * disabled.
>>> @@ -127,7 +126,6 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu =
*vcpu)
>>>=20
>>> 	return r;
>>> }
>>> -#endif /* CONFIG_KVM_BOOK3S_64_HV */
>>>=20
>>> int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
>>> {
>>> @@ -194,11 +192,9 @@ int kvmppc_sanity_check(struct kvm_vcpu *vcpu)
>>> 	if ((vcpu->arch.cpu_type !=3D KVM_CPU_3S_64) && =
vcpu->arch.papr_enabled)
>>> 		goto out;
>>>=20
>>> -#ifdef CONFIG_KVM_BOOK3S_64_HV
>>> 	/* HV KVM can only do PAPR mode for now */
>>> -	if (!vcpu->arch.papr_enabled)
>>> +	if (!vcpu->arch.papr_enabled && kvmppc_ops->is_hv_enabled)
>>> 		goto out;
>>> -#endif
>>>=20
>>> #ifdef CONFIG_KVM_BOOKE_HV
>>> 	if (!cpu_has_feature(CPU_FTR_EMB_HV))
>>> @@ -322,22 +318,26 @@ int kvm_dev_ioctl_check_extension(long ext)
>>> 	case KVM_CAP_DEVICE_CTRL:
>>> 		r =3D 1;
>>> 		break;
>>> -#ifndef CONFIG_KVM_BOOK3S_64_HV
>>> 	case KVM_CAP_PPC_PAIRED_SINGLES:
>>> 	case KVM_CAP_PPC_OSI:
>>> 	case KVM_CAP_PPC_GET_PVINFO:
>>> #if defined(CONFIG_KVM_E500V2) || defined(CONFIG_KVM_E500MC)
>>> 	case KVM_CAP_SW_TLB:
>>> #endif
>>> -#ifdef CONFIG_KVM_MPIC
>>> -	case KVM_CAP_IRQ_MPIC:
>>> -#endif
>>> -		r =3D 1;
>>> +		/* We support this only for PR */
>>> +		r =3D !kvmppc_ops->is_hv_enabled;
>>=20
>> Reading this, have you test compiled your code against e500 configs?
>=20
>=20
> Not yet.

Please do so - for every patch in your series. If you like I can give =
you my example configs I usually use to test compile this.

>=20
>=20
>>=20
>>> 		break;
>>> +#ifdef CONFIG_KVM_MMIO
>>> 	case KVM_CAP_COALESCED_MMIO:
>>> 		r =3D KVM_COALESCED_MMIO_PAGE_OFFSET;
>>> 		break;
>>> #endif
>>> +#ifdef CONFIG_KVM_MPIC
>>> +	case KVM_CAP_IRQ_MPIC:
>>> +		r =3D 1;
>>> +		break;
>>> +#endif
>>> +
>>> #ifdef CONFIG_PPC_BOOK3S_64
>>> 	case KVM_CAP_SPAPR_TCE:
>>> 	case KVM_CAP_PPC_ALLOC_HTAB:
>>> @@ -348,32 +348,37 @@ int kvm_dev_ioctl_check_extension(long ext)
>>> 		r =3D 1;
>>> 		break;
>>> #endif /* CONFIG_PPC_BOOK3S_64 */
>>> -#ifdef CONFIG_KVM_BOOK3S_64_HV
>>> +#ifdef CONFIG_KVM_BOOK3S_HV
>>> 	case KVM_CAP_PPC_SMT:
>>> -		r =3D threads_per_core;
>>> +		if (kvmppc_ops->is_hv_enabled)
>>> +			r =3D threads_per_core;
>>> +		else
>>> +			r =3D 0;
>>=20
>> 0? Or 1?
>>=20
>=20
> That check extension was not supported before on PR. So not sure what
> the value should be. May be 1 is better indicating we have one thread.
> Will change.

Ah, the default case (cap unknown) returns 0, so this is fine.


Alex

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Alexander Graf]

On 27.09.2013, at 12:52, Aneesh Kumar K.V wrote:

> "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> writes:
>=20
>> Hi All,
>>=20
>> This patch series support enabling HV and PR KVM together in the same =
kernel. We
>> extend machine property with new property "kvm_type". A value of 1 =
will force HV
>> KVM and 2 PR KVM. The default value is 0 which will select the =
fastest KVM mode.
>> ie, HV if that is supported otherwise PR.
>>=20
>> With Qemu command line having
>>=20
>> -machine pseries,accel=3Dkvm,kvm_type=3D1
>>=20
>> [root@llmp24l02 qemu]# bash ../qemu
>> failed to initialize KVM: Invalid argument
>> [root@llmp24l02 qemu]# modprobe kvm-pr
>> [root@llmp24l02 qemu]# bash ../qemu
>> failed to initialize KVM: Invalid argument
>> [root@llmp24l02 qemu]# modprobe  kvm-hv
>> [root@llmp24l02 qemu]# bash ../qemu
>>=20
>> now with
>>=20
>> -machine pseries,accel=3Dkvm,kvm_type=3D2
>>=20
>> [root@llmp24l02 qemu]# rmmod kvm-pr
>> [root@llmp24l02 qemu]# bash ../qemu
>> failed to initialize KVM: Invalid argument
>> [root@llmp24l02 qemu]#
>> [root@llmp24l02 qemu]# modprobe kvm-pr
>> [root@llmp24l02 qemu]# bash ../qemu
>>=20
>> if don't specify kvm_type machine property, it will take a default =
value 0,
>> which means fastest supported.
>=20
> Related qemu patch
>=20
> commit 8d139053177d48a70cb710b211ea4c2843eccdfb
> Author: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Date:   Mon Sep 23 12:28:37 2013 +0530
>=20
>    kvm: Add a new machine property kvm_type
>=20
>    Targets like ppc64 support different type of KVM, one which use
>    hypervisor mode and the other which doesn't. Add a new machine
>    property kvm_type that helps in selecting the respective ones
>=20
>    Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>

This really is too early, as we can't possibly run in HV mode for =
non-pseries machines, so the interpretation (or at least sanity =
checking) of what values are reasonable should occur in the machine. =
That's why it's a variable in the "machine opts".

Also, users don't want to say type=3D0. They want to say type=3DPR or =
type=3DHV or type=3DHV,PR. In fact, can't you make this a property of =
-accel? Then it's truly accel specific and everything should be well.


Alex

>=20
> diff --git a/kvm-all.c b/kvm-all.c
> index b87215c..a061eda 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -1350,7 +1350,7 @@ int kvm_init(void)
>     KVMState *s;
>     const KVMCapabilityInfo *missing_cap;
>     int ret;
> -    int i;
> +    int i, kvm_type;
>     int max_vcpus;
>=20
>     s =3D g_malloc0(sizeof(KVMState));
> @@ -1407,7 +1407,8 @@ int kvm_init(void)
>         goto err;
>     }
>=20
> -    s->vmfd =3D kvm_ioctl(s, KVM_CREATE_VM, 0);
> +    kvm_type =3D qemu_opt_get_number(qemu_get_machine_opts(), =
"kvm_type", 0);
> +    s->vmfd =3D kvm_ioctl(s, KVM_CREATE_VM, kvm_type);
>     if (s->vmfd < 0) {
> #ifdef TARGET_S390X
>         fprintf(stderr, "Please add the 'switch_amode' kernel =
parameter to "
> diff --git a/vl.c b/vl.c
> index 4e709d5..4374b17 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -427,7 +427,12 @@ static QemuOptsList qemu_machine_opts =3D {
>             .name =3D "usb",
>             .type =3D QEMU_OPT_BOOL,
>             .help =3D "Set on/off to enable/disable usb",
> +        },{
> +            .name =3D "kvm_type",
> +            .type =3D QEMU_OPT_NUMBER,
> +            .help =3D "Set to kvm type to be used in create vm =
ioctl",
>         },
> +
>         { /* End of list */ }
>     },
> };
>=20
> --
> To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 15:03, Aneesh Kumar K.V wrote:
>
>> Alexander Graf <agraf@suse.de> writes:
>> 
>> 
>>>> diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
>>>> index 1abe478..e0229dd 100644
>>>> --- a/arch/powerpc/kvm/book3s_segment.S
>>>> +++ b/arch/powerpc/kvm/book3s_segment.S
>>>> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
>>>> .global kvmppc_handler_trampoline_exit
>>>> kvmppc_handler_trampoline_exit:
>>>> 
>>>> +#if defined(CONFIG_KVM_BOOK3S_HV)
>>>> +.global kvmppc_interrupt_pr
>>>> +kvmppc_interrupt_pr:
>>>> +	ld	r9, HSTATE_SCRATCH2(r13)
>>>> +#else
>>>> .global kvmppc_interrupt
>>>> kvmppc_interrupt:
>>> 
>>> Just always call it kvmppc_interrupt_pr and thus share at least that
>>> part of the code :).
>> 
>> But if i don't have HV enabled, we don't compile book3s_hv_rmhandlers.S
>> Hence don't have the kvmppc_interrupt symbol defined.
>
> Ah, because we're always jumping to kvmppc_interrupt. Can we make this
> slightly less magical? How about we always call kvmppc_interrupt_hv
> when CONFIG_KVM_BOOK3S_HV_POSSIBLE and always kvmppc_interrupt_pr when
> CONFIG_KVM_BOOK3S_PR_POSSIBLE and then branch to kvmppc_interrupt_pr
> from kvmppc_interrupt_hv?
>
> IMHO that would make the code flow more obvious.


To make sure I understand you correctly, what you are suggesting is
to update __KVM_HANDLER to call kvmppc_interupt_pr when HV is not
enabled ?

-aneesh

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 15:06, Aneesh Kumar K.V wrote:
>
>> Alexander Graf <agraf@suse.de> writes:
>> 
>>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>> 
>>>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>>>> 
>>>> This patch moves PR related tracepoints to a separate header. This
>>>> enables in converting PR to a kernel module which will be done in
>>>> later patches
>>>> 
>>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>>>> ---
>>>> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
>>>> arch/powerpc/kvm/book3s_mmu_hpte.c    |   2 +-
>>>> arch/powerpc/kvm/book3s_pr.c          |   3 +-
>>>> arch/powerpc/kvm/trace.h              | 234 +--------------------------
>>>> arch/powerpc/kvm/trace_pr.h           | 297 ++++++++++++++++++++++++++++++++++
>>>> 5 files changed, 308 insertions(+), 230 deletions(-)
>>>> create mode 100644 arch/powerpc/kvm/trace_pr.h
>>>> 
>>>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>> index 329a978..fd5b393 100644
>>>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>> @@ -27,7 +27,7 @@
>>>> #include <asm/machdep.h>
>>>> #include <asm/mmu_context.h>
>>>> #include <asm/hw_irq.h>
>>>> -#include "trace.h"
>>>> +#include "trace_pr.h"
>>>> 
>>>> #define PTE_SIZE 12
>>>> 
>>>> diff --git a/arch/powerpc/kvm/book3s_mmu_hpte.c b/arch/powerpc/kvm/book3s_mmu_hpte.c
>>>> index d2d280b..4556168 100644
>>>> --- a/arch/powerpc/kvm/book3s_mmu_hpte.c
>>>> +++ b/arch/powerpc/kvm/book3s_mmu_hpte.c
>>>> @@ -28,7 +28,7 @@
>>>> #include <asm/mmu_context.h>
>>>> #include <asm/hw_irq.h>
>>>> 
>>>> -#include "trace.h"
>>>> +#include "trace_pr.h"
>>>> 
>>>> #define PTE_SIZE	12
>>>> 
>>>> diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
>>>> index 2a97279..99d0839 100644
>>>> --- a/arch/powerpc/kvm/book3s_pr.c
>>>> +++ b/arch/powerpc/kvm/book3s_pr.c
>>>> @@ -41,7 +41,8 @@
>>>> #include <linux/vmalloc.h>
>>>> #include <linux/highmem.h>
>>>> 
>>>> -#include "trace.h"
>>>> +#define CREATE_TRACE_POINTS
>>>> +#include "trace_pr.h"
>>>> 
>>>> /* #define EXIT_DEBUG */
>>>> /* #define DEBUG_EXT */
>>>> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
>>>> index a088e9a..7d5a136 100644
>>>> --- a/arch/powerpc/kvm/trace.h
>>>> +++ b/arch/powerpc/kvm/trace.h
>>>> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
>>>> 	{41, "HV_PRIV"}
>>>> #endif
>>>> 
>>>> +#ifndef CONFIG_KVM_BOOK3S_PR
>>>> +/*
>>>> + * For pr we define this in trace_pr.h since it pr can be built as
>>>> + * a module
>>> 
>>> Not sure I understand the need. If the config option is available, so
>>> should the struct field. Worst case that happens with HV is that we
>>> get empty shadow_srr1 values in our trace, no?
>> 
>> That is not the real reason. trace.h get built as part of kvm.ko or as
>> part of kernel. These trace functions actually get called from
>> kvm-pr.ko. To make they build i would either need EXPORT_SYMBOL or move
>> the definition of them to kvm-pr.ko. I did the later and moved only pr
>> related traces to kvm-pr.ko
>
> I fail to see why we wouldn't have a trace_hv.h file then, as that can
> also be built as a module, no? And at that point I don't see why we
> would need any conditionals at all in trace.h anymore, as it would
> only cover generic code.

Currently HV module is not using any tracepoints. Once it start using
tracepoints we would have trace_hv.h

-aneesh

Re: [RFC PATCH 08/11] kvm: powerpc: book3s: Support building HV and PR KVM as module

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 15:08, Aneesh Kumar K.V wrote:
>
>> Alexander Graf <agraf@suse.de> writes:
>> 
>>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>> 
>>>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>> index fd5b393..775d368 100644
>>>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>> @@ -27,6 +27,7 @@
>>>> #include <asm/machdep.h>
>>>> #include <asm/mmu_context.h>
>>>> #include <asm/hw_irq.h>
>>>> +
>>> 
>>> Stray whitespace change
>>> 
>> 
>> will fix
>> 
>>>> #include "trace_pr.h"
>>>> 
>>>> #define PTE_SIZE 12
>>>> diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c
>>>> index b9841ad..20d03c2 100644
>>>> --- a/arch/powerpc/kvm/book3s_emulate.c
>>>> +++ b/arch/powerpc/kvm/book3s_emulate.c
>>>> @@ -172,7 +172,7 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
>>>> 			vcpu->arch.mmu.tlbie(vcpu, addr, large);
>>>> 			break;
>>>> 		}
>>>> -#ifdef CONFIG_KVM_BOOK3S_64_PR
>>>> +#ifdef CONFIG_KVM_BOOK3S_PR
>>> 
>>> Why?
>> 
>> If i have CONFIG_KVM_BOOK3S_64_PR=m  #ifdef CONFIG_KVM_BOOK3S_64_PR will
>> not work. There is a runtime check I can use IS_ENABLED(). But didn't
>> want to do those. Hence moved to the symbol which will be set as
>> CONFIG_KVM_BOOK3S_PR = y
>
> Ah, I see. The naming starts to get really confusing. Could you please
> give this a slightly better name, such as
> CONFIG_KVM_BOOK3S_PR_POSSIBLE or such, so that it's obvious that this
> #ifdef doesn't mean "option is enabled", but instead means "option can
> potentially be enabled through a module".

Will do

-aneesh

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 27.09.2013, at 12:52, Aneesh Kumar K.V wrote:
>
>> "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> writes:
>> 
>>> Hi All,
>>> 
>>> This patch series support enabling HV and PR KVM together in the same kernel. We
>>> extend machine property with new property "kvm_type". A value of 1 will force HV
>>> KVM and 2 PR KVM. The default value is 0 which will select the fastest KVM mode.
>>> ie, HV if that is supported otherwise PR.
>>> 
>>> With Qemu command line having
>>> 
>>> -machine pseries,accel=kvm,kvm_type=1
>>> 
>>> [root@llmp24l02 qemu]# bash ../qemu
>>> failed to initialize KVM: Invalid argument
>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>> [root@llmp24l02 qemu]# bash ../qemu
>>> failed to initialize KVM: Invalid argument
>>> [root@llmp24l02 qemu]# modprobe  kvm-hv
>>> [root@llmp24l02 qemu]# bash ../qemu
>>> 
>>> now with
>>> 
>>> -machine pseries,accel=kvm,kvm_type=2
>>> 
>>> [root@llmp24l02 qemu]# rmmod kvm-pr
>>> [root@llmp24l02 qemu]# bash ../qemu
>>> failed to initialize KVM: Invalid argument
>>> [root@llmp24l02 qemu]#
>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>> [root@llmp24l02 qemu]# bash ../qemu
>>> 
>>> if don't specify kvm_type machine property, it will take a default value 0,
>>> which means fastest supported.
>> 
>> Related qemu patch
>> 
>> commit 8d139053177d48a70cb710b211ea4c2843eccdfb
>> Author: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> Date:   Mon Sep 23 12:28:37 2013 +0530
>> 
>>    kvm: Add a new machine property kvm_type
>> 
>>    Targets like ppc64 support different type of KVM, one which use
>>    hypervisor mode and the other which doesn't. Add a new machine
>>    property kvm_type that helps in selecting the respective ones
>> 
>>    Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>
> This really is too early, as we can't possibly run in HV mode for
> non-pseries machines, so the interpretation (or at least sanity
> checking) of what values are reasonable should occur in the
> machine. That's why it's a variable in the "machine opts".

With the current code CREATE_VM will fail, because we won't have
kvm-hv.ko loaded and trying to create a vm with type 1 will fail. 
Now the challenge related to moving that to machine_init or later is, we
depend on HV or PR callback early in CREATE_VM. With the changes we have

int kvmppc_core_init_vm(struct kvm *kvm)
{

#ifdef CONFIG_PPC64
	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
#endif

	return kvm->arch.kvm_ops->init_vm(kvm);
}

Also the mmu notifier callback do end up calling kvm_unmap_hva etc which
are all HV/PR dependent. 



>
> Also, users don't want to say type=0. They want to say type=PR or
> type=HV or type=HV,PR. In fact, can't you make this a property of
> -accel? Then it's truly accel specific and everything should be well.

If we are doing this as machine property, we can't specify string,
because "HV"/"PR" are all powerpc dependent, so parsing that is not
possible in kvm_init in qemu. But, yes ideally it would be nice to be
able to speicy the type using string. I thought accel is a machine
property, hence was not sure whether I can have additional properties
against that. I was using it as below.

 -machine pseries,accel=kvm,kvm_type=1

will look into more details to check whether this can be accel property.

-aneesh

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Alexander Graf]

On 09/30/2013 02:56 PM, Aneesh Kumar K.V wrote:
> Alexander Graf<agraf@suse.de>  writes:
>
>> On 27.09.2013, at 15:03, Aneesh Kumar K.V wrote:
>>
>>> Alexander Graf<agraf@suse.de>  writes:
>>>
>>>
>>>>> diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
>>>>> index 1abe478..e0229dd 100644
>>>>> --- a/arch/powerpc/kvm/book3s_segment.S
>>>>> +++ b/arch/powerpc/kvm/book3s_segment.S
>>>>> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
>>>>> .global kvmppc_handler_trampoline_exit
>>>>> kvmppc_handler_trampoline_exit:
>>>>>
>>>>> +#if defined(CONFIG_KVM_BOOK3S_HV)
>>>>> +.global kvmppc_interrupt_pr
>>>>> +kvmppc_interrupt_pr:
>>>>> +	ld	r9, HSTATE_SCRATCH2(r13)
>>>>> +#else
>>>>> .global kvmppc_interrupt
>>>>> kvmppc_interrupt:
>>>> Just always call it kvmppc_interrupt_pr and thus share at least that
>>>> part of the code :).
>>> But if i don't have HV enabled, we don't compile book3s_hv_rmhandlers.S
>>> Hence don't have the kvmppc_interrupt symbol defined.
>> Ah, because we're always jumping to kvmppc_interrupt. Can we make this
>> slightly less magical? How about we always call kvmppc_interrupt_hv
>> when CONFIG_KVM_BOOK3S_HV_POSSIBLE and always kvmppc_interrupt_pr when
>> CONFIG_KVM_BOOK3S_PR_POSSIBLE and then branch to kvmppc_interrupt_pr
>> from kvmppc_interrupt_hv?
>>
>> IMHO that would make the code flow more obvious.
>
> To make sure I understand you correctly, what you are suggesting is
> to update __KVM_HANDLER to call kvmppc_interupt_pr when HV is not
> enabled ?

Yes, I think that makes the code flow more obvious. Every function 
always has the same name regardless of config options then.


Alex

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Alexander Graf]

On 09/30/2013 02:57 PM, Aneesh Kumar K.V wrote:
> Alexander Graf<agraf@suse.de>  writes:
>
>> On 27.09.2013, at 15:06, Aneesh Kumar K.V wrote:
>>
>>> Alexander Graf<agraf@suse.de>  writes:
>>>
>>>> On 27.09.2013, at 12:03, Aneesh Kumar K.V wrote:
>>>>
>>>>> From: "Aneesh Kumar K.V"<aneesh.kumar@linux.vnet.ibm.com>
>>>>>
>>>>> This patch moves PR related tracepoints to a separate header. This
>>>>> enables in converting PR to a kernel module which will be done in
>>>>> later patches
>>>>>
>>>>> Signed-off-by: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>>>> ---
>>>>> arch/powerpc/kvm/book3s_64_mmu_host.c |   2 +-
>>>>> arch/powerpc/kvm/book3s_mmu_hpte.c    |   2 +-
>>>>> arch/powerpc/kvm/book3s_pr.c          |   3 +-
>>>>> arch/powerpc/kvm/trace.h              | 234 +--------------------------
>>>>> arch/powerpc/kvm/trace_pr.h           | 297 ++++++++++++++++++++++++++++++++++
>>>>> 5 files changed, 308 insertions(+), 230 deletions(-)
>>>>> create mode 100644 arch/powerpc/kvm/trace_pr.h
>>>>>
>>>>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>>> index 329a978..fd5b393 100644
>>>>> --- a/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>>> +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
>>>>> @@ -27,7 +27,7 @@
>>>>> #include<asm/machdep.h>
>>>>> #include<asm/mmu_context.h>
>>>>> #include<asm/hw_irq.h>
>>>>> -#include "trace.h"
>>>>> +#include "trace_pr.h"
>>>>>
>>>>> #define PTE_SIZE 12
>>>>>
>>>>> diff --git a/arch/powerpc/kvm/book3s_mmu_hpte.c b/arch/powerpc/kvm/book3s_mmu_hpte.c
>>>>> index d2d280b..4556168 100644
>>>>> --- a/arch/powerpc/kvm/book3s_mmu_hpte.c
>>>>> +++ b/arch/powerpc/kvm/book3s_mmu_hpte.c
>>>>> @@ -28,7 +28,7 @@
>>>>> #include<asm/mmu_context.h>
>>>>> #include<asm/hw_irq.h>
>>>>>
>>>>> -#include "trace.h"
>>>>> +#include "trace_pr.h"
>>>>>
>>>>> #define PTE_SIZE	12
>>>>>
>>>>> diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
>>>>> index 2a97279..99d0839 100644
>>>>> --- a/arch/powerpc/kvm/book3s_pr.c
>>>>> +++ b/arch/powerpc/kvm/book3s_pr.c
>>>>> @@ -41,7 +41,8 @@
>>>>> #include<linux/vmalloc.h>
>>>>> #include<linux/highmem.h>
>>>>>
>>>>> -#include "trace.h"
>>>>> +#define CREATE_TRACE_POINTS
>>>>> +#include "trace_pr.h"
>>>>>
>>>>> /* #define EXIT_DEBUG */
>>>>> /* #define DEBUG_EXT */
>>>>> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
>>>>> index a088e9a..7d5a136 100644
>>>>> --- a/arch/powerpc/kvm/trace.h
>>>>> +++ b/arch/powerpc/kvm/trace.h
>>>>> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
>>>>> 	{41, "HV_PRIV"}
>>>>> #endif
>>>>>
>>>>> +#ifndef CONFIG_KVM_BOOK3S_PR
>>>>> +/*
>>>>> + * For pr we define this in trace_pr.h since it pr can be built as
>>>>> + * a module
>>>> Not sure I understand the need. If the config option is available, so
>>>> should the struct field. Worst case that happens with HV is that we
>>>> get empty shadow_srr1 values in our trace, no?
>>> That is not the real reason. trace.h get built as part of kvm.ko or as
>>> part of kernel. These trace functions actually get called from
>>> kvm-pr.ko. To make they build i would either need EXPORT_SYMBOL or move
>>> the definition of them to kvm-pr.ko. I did the later and moved only pr
>>> related traces to kvm-pr.ko
>> I fail to see why we wouldn't have a trace_hv.h file then, as that can
>> also be built as a module, no? And at that point I don't see why we
>> would need any conditionals at all in trace.h anymore, as it would
>> only cover generic code.
> Currently HV module is not using any tracepoints. Once it start using
> tracepoints we would have trace_hv.h

So why would there be an #ifndef in trace.h?


Alex

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Alexander Graf]

On 09/30/2013 03:09 PM, Aneesh Kumar K.V wrote:
> Alexander Graf<agraf@suse.de>  writes:
>
>> On 27.09.2013, at 12:52, Aneesh Kumar K.V wrote:
>>
>>> "Aneesh Kumar K.V"<aneesh.kumar@linux.vnet.ibm.com>  writes:
>>>
>>>> Hi All,
>>>>
>>>> This patch series support enabling HV and PR KVM together in the same kernel. We
>>>> extend machine property with new property "kvm_type". A value of 1 will force HV
>>>> KVM and 2 PR KVM. The default value is 0 which will select the fastest KVM mode.
>>>> ie, HV if that is supported otherwise PR.
>>>>
>>>> With Qemu command line having
>>>>
>>>> -machine pseries,accel=kvm,kvm_type=1
>>>>
>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>> failed to initialize KVM: Invalid argument
>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>> failed to initialize KVM: Invalid argument
>>>> [root@llmp24l02 qemu]# modprobe  kvm-hv
>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>
>>>> now with
>>>>
>>>> -machine pseries,accel=kvm,kvm_type=2
>>>>
>>>> [root@llmp24l02 qemu]# rmmod kvm-pr
>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>> failed to initialize KVM: Invalid argument
>>>> [root@llmp24l02 qemu]#
>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>
>>>> if don't specify kvm_type machine property, it will take a default value 0,
>>>> which means fastest supported.
>>> Related qemu patch
>>>
>>> commit 8d139053177d48a70cb710b211ea4c2843eccdfb
>>> Author: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>> Date:   Mon Sep 23 12:28:37 2013 +0530
>>>
>>>     kvm: Add a new machine property kvm_type
>>>
>>>     Targets like ppc64 support different type of KVM, one which use
>>>     hypervisor mode and the other which doesn't. Add a new machine
>>>     property kvm_type that helps in selecting the respective ones
>>>
>>>     Signed-off-by: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>> This really is too early, as we can't possibly run in HV mode for
>> non-pseries machines, so the interpretation (or at least sanity
>> checking) of what values are reasonable should occur in the
>> machine. That's why it's a variable in the "machine opts".
> With the current code CREATE_VM will fail, because we won't have
> kvm-hv.ko loaded and trying to create a vm with type 1 will fail.
> Now the challenge related to moving that to machine_init or later is, we
> depend on HV or PR callback early in CREATE_VM. With the changes we have
>
> int kvmppc_core_init_vm(struct kvm *kvm)
> {
>
> #ifdef CONFIG_PPC64
> 	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
> 	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
> #endif
>
> 	return kvm->arch.kvm_ops->init_vm(kvm);
> }
>
> Also the mmu notifier callback do end up calling kvm_unmap_hva etc which
> are all HV/PR dependent.

Yes, so we should verify in the machine models that we're runnable with 
the currently selected type at least, to give the user a sensible error 
message.

>
>
>
>> Also, users don't want to say type=0. They want to say type=PR or
>> type=HV or type=HV,PR. In fact, can't you make this a property of
>> -accel? Then it's truly accel specific and everything should be well.
> If we are doing this as machine property, we can't specify string,
> because "HV"/"PR" are all powerpc dependent, so parsing that is not
> possible in kvm_init in qemu. But, yes ideally it would be nice to be

Well, we could do the "name to integer" conversion in an arch specific 
function, no?

> able to speicy the type using string. I thought accel is a machine
> property, hence was not sure whether I can have additional properties
> against that. I was using it as below.
>
>   -machine pseries,accel=kvm,kvm_type=1
>
> will look into more details to check whether this can be accel property.

Great :).


Alex

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 09/30/2013 02:57 PM, Aneesh Kumar K.V wrote:
>> Alexander Graf<agraf@suse.de>  writes:
>>>>>> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
>>>>>> index a088e9a..7d5a136 100644
>>>>>> --- a/arch/powerpc/kvm/trace.h
>>>>>> +++ b/arch/powerpc/kvm/trace.h
>>>>>> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
>>>>>> 	{41, "HV_PRIV"}
>>>>>> #endif
>>>>>>
>>>>>> +#ifndef CONFIG_KVM_BOOK3S_PR
>>>>>> +/*
>>>>>> + * For pr we define this in trace_pr.h since it pr can be built as
>>>>>> + * a module
>>>>> Not sure I understand the need. If the config option is available, so
>>>>> should the struct field. Worst case that happens with HV is that we
>>>>> get empty shadow_srr1 values in our trace, no?
>>>> That is not the real reason. trace.h get built as part of kvm.ko or as
>>>> part of kernel. These trace functions actually get called from
>>>> kvm-pr.ko. To make they build i would either need EXPORT_SYMBOL or move
>>>> the definition of them to kvm-pr.ko. I did the later and moved only pr
>>>> related traces to kvm-pr.ko
>>> I fail to see why we wouldn't have a trace_hv.h file then, as that can
>>> also be built as a module, no? And at that point I don't see why we
>>> would need any conditionals at all in trace.h anymore, as it would
>>> only cover generic code.
>> Currently HV module is not using any tracepoints. Once it start using
>> tracepoints we would have trace_hv.h
>
> So why would there be an #ifndef in trace.h?
>

to handle things like trace_kvm_exit in booke.c. For that we still don't
have kernel module, and booke.c include trace.h.

-aneesh

Re: [RFC PATCH 07/11] kvm: powerpc: book3s: pr: move PR related tracepoints to a separate header

[Alexander Graf]

On 09/30/2013 05:53 PM, Aneesh Kumar K.V wrote:
> Alexander Graf<agraf@suse.de>  writes:
>
>> On 09/30/2013 02:57 PM, Aneesh Kumar K.V wrote:
>>> Alexander Graf<agraf@suse.de>   writes:
>>>>>>> diff --git a/arch/powerpc/kvm/trace.h b/arch/powerpc/kvm/trace.h
>>>>>>> index a088e9a..7d5a136 100644
>>>>>>> --- a/arch/powerpc/kvm/trace.h
>>>>>>> +++ b/arch/powerpc/kvm/trace.h
>>>>>>> @@ -85,6 +85,12 @@ TRACE_EVENT(kvm_ppc_instr,
>>>>>>> 	{41, "HV_PRIV"}
>>>>>>> #endif
>>>>>>>
>>>>>>> +#ifndef CONFIG_KVM_BOOK3S_PR
>>>>>>> +/*
>>>>>>> + * For pr we define this in trace_pr.h since it pr can be built as
>>>>>>> + * a module
>>>>>> Not sure I understand the need. If the config option is available, so
>>>>>> should the struct field. Worst case that happens with HV is that we
>>>>>> get empty shadow_srr1 values in our trace, no?
>>>>> That is not the real reason. trace.h get built as part of kvm.ko or as
>>>>> part of kernel. These trace functions actually get called from
>>>>> kvm-pr.ko. To make they build i would either need EXPORT_SYMBOL or move
>>>>> the definition of them to kvm-pr.ko. I did the later and moved only pr
>>>>> related traces to kvm-pr.ko
>>>> I fail to see why we wouldn't have a trace_hv.h file then, as that can
>>>> also be built as a module, no? And at that point I don't see why we
>>>> would need any conditionals at all in trace.h anymore, as it would
>>>> only cover generic code.
>>> Currently HV module is not using any tracepoints. Once it start using
>>> tracepoints we would have trace_hv.h
>> So why would there be an #ifndef in trace.h?
>>
> to handle things like trace_kvm_exit in booke.c. For that we still don't
> have kernel module, and booke.c include trace.h.

Ah, so it's for booke! :)

Just move them to trace_booke.h then to clarify things.


Alex

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 09/30/2013 02:56 PM, Aneesh Kumar K.V wrote:
>> Alexander Graf<agraf@suse.de>  writes:
>>
>>> On 27.09.2013, at 15:03, Aneesh Kumar K.V wrote:
>>>
>>>> Alexander Graf<agraf@suse.de>  writes:
>>>>
>>>>
>>>>>> diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
>>>>>> index 1abe478..e0229dd 100644
>>>>>> --- a/arch/powerpc/kvm/book3s_segment.S
>>>>>> +++ b/arch/powerpc/kvm/book3s_segment.S
>>>>>> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
>>>>>> .global kvmppc_handler_trampoline_exit
>>>>>> kvmppc_handler_trampoline_exit:
>>>>>>
>>>>>> +#if defined(CONFIG_KVM_BOOK3S_HV)
>>>>>> +.global kvmppc_interrupt_pr
>>>>>> +kvmppc_interrupt_pr:
>>>>>> +	ld	r9, HSTATE_SCRATCH2(r13)
>>>>>> +#else
>>>>>> .global kvmppc_interrupt
>>>>>> kvmppc_interrupt:
>>>>> Just always call it kvmppc_interrupt_pr and thus share at least that
>>>>> part of the code :).
>>>> But if i don't have HV enabled, we don't compile book3s_hv_rmhandlers.S
>>>> Hence don't have the kvmppc_interrupt symbol defined.
>>> Ah, because we're always jumping to kvmppc_interrupt. Can we make this
>>> slightly less magical? How about we always call kvmppc_interrupt_hv
>>> when CONFIG_KVM_BOOK3S_HV_POSSIBLE and always kvmppc_interrupt_pr when
>>> CONFIG_KVM_BOOK3S_PR_POSSIBLE and then branch to kvmppc_interrupt_pr
>>> from kvmppc_interrupt_hv?
>>>
>>> IMHO that would make the code flow more obvious.
>>
>> To make sure I understand you correctly, what you are suggesting is
>> to update __KVM_HANDLER to call kvmppc_interupt_pr when HV is not
>> enabled ?
>
> Yes, I think that makes the code flow more obvious. Every function 
> always has the same name regardless of config options then.
>

Something like this ( btw non tested )

diff --git a/arch/powerpc/include/asm/exception-64s.h b/arch/powerpc/include/asm/exception-64s.h
index cca12f0..0b798d4 100644
--- a/arch/powerpc/include/asm/exception-64s.h
+++ b/arch/powerpc/include/asm/exception-64s.h
@@ -198,6 +198,17 @@ END_FTR_SECTION_NESTED(ftr,ftr,943)
 	cmpwi	r10,0;							\
 	bne	do_kvm_##n
 
+#ifdef CONFIG_KVM_BOOK3S_HV
+/*
+ * If hv is possible, interrupts come into to the hv version
+ * of the kvmppc_interrupt code, which then jumps to the PR handler,
+ * kvmppc_interrupt_pr, if the guest is a PR guest.
+ */
+#define kvmppc_interrupt kvmppc_interrupt_hv
+#else
+#define kvmppc_interrupt kvmppc_interrupt_pr
+#endif
+
 #define __KVM_HANDLER(area, h, n)					\
 do_kvm_##n:								\
 	BEGIN_FTR_SECTION_NESTED(947)					\
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 5ede7fc..2eb6622 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -563,8 +563,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
 /*
  * We come here from the first-level interrupt handlers.
  */
-	.globl	kvmppc_interrupt
-kvmppc_interrupt:
+	.globl	kvmppc_interrupt_hv
+kvmppc_interrupt_hv:
 	/*
 	 * Register contents:
 	 * R12		= interrupt vector
@@ -577,6 +577,11 @@ kvmppc_interrupt:
 	lbz	r9, HSTATE_IN_GUEST(r13)
 	cmpwi	r9, KVM_GUEST_MODE_HOST_HV
 	beq	kvmppc_bad_host_intr
+#ifdef CONFIG_KVM_BOOK3S_PR
+	cmpwi	r9, KVM_GUEST_MODE_GUEST
+	ld	r9, HSTATE_SCRATCH2(r13)
+	beq	kvmppc_interrupt_pr
+#endif
 	/* We're now back in the host but in guest MMU context */
 	li	r9, KVM_GUEST_MODE_HOST_HV
 	stb	r9, HSTATE_IN_GUEST(r13)
diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
index 1abe478..bc50c97 100644
--- a/arch/powerpc/kvm/book3s_segment.S
+++ b/arch/powerpc/kvm/book3s_segment.S
@@ -161,8 +161,8 @@ kvmppc_handler_trampoline_enter_end:
 .global kvmppc_handler_trampoline_exit
 kvmppc_handler_trampoline_exit:
 
-.global kvmppc_interrupt
-kvmppc_interrupt:
+.global kvmppc_interrupt_pr
+kvmppc_interrupt_pr:
 
 	/* Register usage at this point:
 	 *

Re: [RFC PATCH 06/11] kvm: powerpc: book3s: Add is_hv_enabled to kvmppc_ops

[Alexander Graf]

On 09/30/2013 06:20 PM, Aneesh Kumar K.V wrote:
> Alexander Graf<agraf@suse.de>  writes:
>
>> On 09/30/2013 02:56 PM, Aneesh Kumar K.V wrote:
>>> Alexander Graf<agraf@suse.de>   writes:
>>>
>>>> On 27.09.2013, at 15:03, Aneesh Kumar K.V wrote:
>>>>
>>>>> Alexander Graf<agraf@suse.de>   writes:
>>>>>
>>>>>
>>>>>>> diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S
>>>>>>> index 1abe478..e0229dd 100644
>>>>>>> --- a/arch/powerpc/kvm/book3s_segment.S
>>>>>>> +++ b/arch/powerpc/kvm/book3s_segment.S
>>>>>>> @@ -161,9 +161,14 @@ kvmppc_handler_trampoline_enter_end:
>>>>>>> .global kvmppc_handler_trampoline_exit
>>>>>>> kvmppc_handler_trampoline_exit:
>>>>>>>
>>>>>>> +#if defined(CONFIG_KVM_BOOK3S_HV)
>>>>>>> +.global kvmppc_interrupt_pr
>>>>>>> +kvmppc_interrupt_pr:
>>>>>>> +	ld	r9, HSTATE_SCRATCH2(r13)
>>>>>>> +#else
>>>>>>> .global kvmppc_interrupt
>>>>>>> kvmppc_interrupt:
>>>>>> Just always call it kvmppc_interrupt_pr and thus share at least that
>>>>>> part of the code :).
>>>>> But if i don't have HV enabled, we don't compile book3s_hv_rmhandlers.S
>>>>> Hence don't have the kvmppc_interrupt symbol defined.
>>>> Ah, because we're always jumping to kvmppc_interrupt. Can we make this
>>>> slightly less magical? How about we always call kvmppc_interrupt_hv
>>>> when CONFIG_KVM_BOOK3S_HV_POSSIBLE and always kvmppc_interrupt_pr when
>>>> CONFIG_KVM_BOOK3S_PR_POSSIBLE and then branch to kvmppc_interrupt_pr
>>>> from kvmppc_interrupt_hv?
>>>>
>>>> IMHO that would make the code flow more obvious.
>>> To make sure I understand you correctly, what you are suggesting is
>>> to update __KVM_HANDLER to call kvmppc_interupt_pr when HV is not
>>> enabled ?
>> Yes, I think that makes the code flow more obvious. Every function
>> always has the same name regardless of config options then.
>>
> Something like this ( btw non tested )

Yes :).


Alex

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Aneesh Kumar K.V]

Alexander Graf <agraf@suse.de> writes:

> On 09/30/2013 03:09 PM, Aneesh Kumar K.V wrote:
>> Alexander Graf<agraf@suse.de>  writes:
>>
>>> On 27.09.2013, at 12:52, Aneesh Kumar K.V wrote:
>>>
>>>> "Aneesh Kumar K.V"<aneesh.kumar@linux.vnet.ibm.com>  writes:
>>>>
>>>>> Hi All,
>>>>>
>>>>> This patch series support enabling HV and PR KVM together in the same kernel. We
>>>>> extend machine property with new property "kvm_type". A value of 1 will force HV
>>>>> KVM and 2 PR KVM. The default value is 0 which will select the fastest KVM mode.
>>>>> ie, HV if that is supported otherwise PR.
>>>>>
>>>>> With Qemu command line having
>>>>>
>>>>> -machine pseries,accel=kvm,kvm_type=1
>>>>>
>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>> failed to initialize KVM: Invalid argument
>>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>> failed to initialize KVM: Invalid argument
>>>>> [root@llmp24l02 qemu]# modprobe  kvm-hv
>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>
>>>>> now with
>>>>>
>>>>> -machine pseries,accel=kvm,kvm_type=2
>>>>>
>>>>> [root@llmp24l02 qemu]# rmmod kvm-pr
>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>> failed to initialize KVM: Invalid argument
>>>>> [root@llmp24l02 qemu]#
>>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>
>>>>> if don't specify kvm_type machine property, it will take a default value 0,
>>>>> which means fastest supported.
>>>> Related qemu patch
>>>>
>>>> commit 8d139053177d48a70cb710b211ea4c2843eccdfb
>>>> Author: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>>> Date:   Mon Sep 23 12:28:37 2013 +0530
>>>>
>>>>     kvm: Add a new machine property kvm_type
>>>>
>>>>     Targets like ppc64 support different type of KVM, one which use
>>>>     hypervisor mode and the other which doesn't. Add a new machine
>>>>     property kvm_type that helps in selecting the respective ones
>>>>
>>>>     Signed-off-by: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>> This really is too early, as we can't possibly run in HV mode for
>>> non-pseries machines, so the interpretation (or at least sanity
>>> checking) of what values are reasonable should occur in the
>>> machine. That's why it's a variable in the "machine opts".
>> With the current code CREATE_VM will fail, because we won't have
>> kvm-hv.ko loaded and trying to create a vm with type 1 will fail.
>> Now the challenge related to moving that to machine_init or later is, we
>> depend on HV or PR callback early in CREATE_VM. With the changes we have
>>
>> int kvmppc_core_init_vm(struct kvm *kvm)
>> {
>>
>> #ifdef CONFIG_PPC64
>> 	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
>> 	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
>> #endif
>>
>> 	return kvm->arch.kvm_ops->init_vm(kvm);
>> }
>>
>> Also the mmu notifier callback do end up calling kvm_unmap_hva etc which
>> are all HV/PR dependent.
>
> Yes, so we should verify in the machine models that we're runnable with 
> the currently selected type at least, to give the user a sensible error 
> message.

Something like the below

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 004184d..7d59ac1 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1337,6 +1337,21 @@ static void ppc_spapr_init(QEMUMachineInitArgs *args)
     assert(spapr->fdt_skel != NULL);
 }
 
+static int spapr_get_vm_type(const char *vm_type)
+{
+    if (!vm_type)
+        return 0;
+
+    if (!strcmp(vm_type, "HV"))
+        return 1;
+
+    if (!strcmp(vm_type, "PR"))
+        return 2;
+
+    hw_error("qemu: unknown kvm_type specified '%s'", vm_type);
+    exit(1);
+}
+
 static QEMUMachine spapr_machine = {
     .name = "pseries",
     .desc = "pSeries Logical Partition (PAPR compliant)",
@@ -1347,6 +1362,7 @@ static QEMUMachine spapr_machine = {
     .max_cpus = MAX_CPUS,
     .no_parallel = 1,
     .default_boot_order = NULL,
+    .get_vm_type = spapr_get_vm_type,
 };
 
 static void spapr_machine_init(void)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 5a7ae9f..2130488 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -21,6 +21,8 @@ typedef void QEMUMachineResetFunc(void);
 
 typedef void QEMUMachineHotAddCPUFunc(const int64_t id, Error **errp);
 
+typedef int QEMUMachineGetVmTypeFunc(const char *arg);
+
 typedef struct QEMUMachine {
     const char *name;
     const char *alias;
@@ -28,6 +30,7 @@ typedef struct QEMUMachine {
     QEMUMachineInitFunc *init;
     QEMUMachineResetFunc *reset;
     QEMUMachineHotAddCPUFunc *hot_add_cpu;
+    QEMUMachineGetVmTypeFunc *get_vm_type;
     BlockInterfaceType block_default_type;
     int max_cpus;
     unsigned int no_serial:1,
diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
index e1f88bf..acc3d74 100644
--- a/include/hw/xen/xen.h
+++ b/include/hw/xen/xen.h
@@ -36,7 +36,8 @@ void xen_cmos_set_s3_resume(void *opaque, int irq, int level);
 
 qemu_irq *xen_interrupt_controller_init(void);
 
-int xen_init(void);
+typedef struct QEMUMachine QEMUMachine;
+int xen_init(QEMUMachine *machine);
 int xen_hvm_init(MemoryRegion **ram_memory);
 void xenstore_store_pv_console_info(int i, struct CharDriverState *chr);
 
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 9bbe3db..f25caec 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -142,8 +142,8 @@ typedef struct KVMState KVMState;
 extern KVMState *kvm_state;
 
 /* external API */
-
-int kvm_init(void);
+typedef struct QEMUMachine QEMUMachine;
+int kvm_init(QEMUMachine *machine);
 
 int kvm_has_sync_mmu(void);
 int kvm_has_vcpu_events(void);
diff --git a/include/sysemu/qtest.h b/include/sysemu/qtest.h
index 9a0c6b3..d71343d 100644
--- a/include/sysemu/qtest.h
+++ b/include/sysemu/qtest.h
@@ -31,7 +31,8 @@ static inline int qtest_available(void)
     return 1;
 }
 
-int qtest_init(void);
+typedef struct QEMUMachine QEMUMachine;
+int qtest_init(QEMUMachine *machine);
 #else
 static inline bool qtest_enabled(void)
 {
@@ -43,7 +44,7 @@ static inline int qtest_available(void)
     return 0;
 }
 
-static inline int qtest_init(void)
+static inline int qtest_init(QEMUMachine *machine)
 {
     return 0;
 }
diff --git a/kvm-all.c b/kvm-all.c
index b87215c..3863abd 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -35,6 +35,8 @@
 #include "qemu/event_notifier.h"
 #include "trace.h"
 
+#include "hw/boards.h"
+
 /* This check must be after config-host.h is included */
 #ifdef CONFIG_EVENTFD
 #include <sys/eventfd.h>
@@ -1342,7 +1344,7 @@ static int kvm_max_vcpus(KVMState *s)
     return 4;
 }
 
-int kvm_init(void)
+int kvm_init(QEMUMachine *machine)
 {
     static const char upgrade_note[] =
         "Please upgrade to at least kernel 2.6.29 or recent kvm-kmod\n"
@@ -1350,7 +1352,7 @@ int kvm_init(void)
     KVMState *s;
     const KVMCapabilityInfo *missing_cap;
     int ret;
-    int i;
+    int i, kvm_type = 0;
     int max_vcpus;
 
     s = g_malloc0(sizeof(KVMState));
@@ -1407,7 +1409,11 @@ int kvm_init(void)
         goto err;
     }
 
-    s->vmfd = kvm_ioctl(s, KVM_CREATE_VM, 0);
+    if (machine->get_vm_type) {
+        kvm_type = machine->get_vm_type(qemu_opt_get(qemu_get_machine_opts(),
+                                                     "kvm_type"));
+    }
+    s->vmfd = kvm_ioctl(s, KVM_CREATE_VM, kvm_type);
     if (s->vmfd < 0) {
 #ifdef TARGET_S390X
         fprintf(stderr, "Please add the 'switch_amode' kernel parameter to "
diff --git a/kvm-stub.c b/kvm-stub.c
index 548f471..ccb7b8c 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -19,6 +19,8 @@
 #include "hw/pci/msi.h"
 #endif
 
+#include "hw/boards.h"
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -33,7 +35,7 @@ int kvm_init_vcpu(CPUState *cpu)
     return -ENOSYS;
 }
 
-int kvm_init(void)
+int kvm_init(QEMUMachine *machine)
 {
     return -ENOSYS;
 }
diff --git a/qtest.c b/qtest.c
index 584c707..ef3c473 100644
--- a/qtest.c
+++ b/qtest.c
@@ -502,7 +502,7 @@ static void qtest_event(void *opaque, int event)
     }
 }
 
-int qtest_init(void)
+int qtest_init(QEMUMachine *machine)
 {
     CharDriverState *chr;
 
diff --git a/vl.c b/vl.c
index 4e709d5..7ecc581 100644
--- a/vl.c
+++ b/vl.c
@@ -427,7 +427,12 @@ static QemuOptsList qemu_machine_opts = {
             .name = "usb",
             .type = QEMU_OPT_BOOL,
             .help = "Set on/off to enable/disable usb",
+        },{
+            .name = "kvm_type",
+            .type = QEMU_OPT_STRING,
+            .help = "Set to kvm type to be used in create vm ioctl",
         },
+
         { /* End of list */ }
     },
 };
@@ -2608,7 +2613,7 @@ static QEMUMachine *machine_parse(const char *name)
     exit(!name || !is_help_option(name));
 }
 
-static int tcg_init(void)
+static int tcg_init(QEMUMachine *machine)
 {
     tcg_exec_init(tcg_tb_size * 1024 * 1024);
     return 0;
@@ -2618,7 +2623,7 @@ static struct {
     const char *opt_name;
     const char *name;
     int (*available)(void);
-    int (*init)(void);
+    int (*init)(QEMUMachine *);
     bool *allowed;
 } accel_list[] = {
     { "tcg", "tcg", tcg_available, tcg_init, &tcg_allowed },
@@ -2627,7 +2632,7 @@ static struct {
     { "qtest", "QTest", qtest_available, qtest_init, &qtest_allowed },
 };
 
-static int configure_accelerator(void)
+static int configure_accelerator(QEMUMachine *machine)
 {
     const char *p;
     char buf[10];
@@ -2654,7 +2659,7 @@ static int configure_accelerator(void)
                     continue;
                 }
                 *(accel_list[i].allowed) = true;
-                ret = accel_list[i].init();
+                ret = accel_list[i].init(machine);
                 if (ret < 0) {
                     init_failed = true;
                     fprintf(stderr, "failed to initialize %s: %s\n",
@@ -4037,10 +4042,10 @@ int main(int argc, char **argv, char **envp)
         exit(0);
     }
 
-    configure_accelerator();
+    configure_accelerator(machine);
 
     if (!qtest_enabled() && qtest_chrdev) {
-        qtest_init();
+        qtest_init(machine);
     }
 
     machine_opts = qemu_get_machine_opts();
diff --git a/xen-all.c b/xen-all.c
index 839f14f..ac3654b 100644
--- a/xen-all.c
+++ b/xen-all.c
@@ -1000,7 +1000,7 @@ static void xen_exit_notifier(Notifier *n, void *data)
     xs_daemon_close(state->xenstore);
 }
 
-int xen_init(void)
+int xen_init(QEMUMachine *machine)
 {
     xen_xc = xen_xc_interface_open(0, 0, 0);
     if (xen_xc == XC_HANDLER_INITIAL_VALUE) {
diff --git a/xen-stub.c b/xen-stub.c
index ad189a6..59927cb 100644
--- a/xen-stub.c
+++ b/xen-stub.c
@@ -47,7 +47,7 @@ qemu_irq *xen_interrupt_controller_init(void)
     return NULL;
 }
 
-int xen_init(void)
+int xen_init(QEMUMachine *machine)
 {
     return -ENOSYS;
 }

>
>>
>>
>>
>>> Also, users don't want to say type=0. They want to say type=PR or
>>> type=HV or type=HV,PR. In fact, can't you make this a property of
>>> -accel? Then it's truly accel specific and everything should be well.
>> If we are doing this as machine property, we can't specify string,
>> because "HV"/"PR" are all powerpc dependent, so parsing that is not
>> possible in kvm_init in qemu. But, yes ideally it would be nice to be
>
> Well, we could do the "name to integer" conversion in an arch specific 
> function, no?
>
>> able to speicy the type using string. I thought accel is a machine
>> property, hence was not sure whether I can have additional properties
>> against that. I was using it as below.
>>
>>   -machine pseries,accel=kvm,kvm_type=1

Can we really specific -accel ? I check and I am finding that as machine
property. 

-aneesh

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Alexander Graf]

On 10/01/2013 01:26 PM, Aneesh Kumar K.V wrote:
> Alexander Graf<agraf@suse.de>  writes:
>
>> On 09/30/2013 03:09 PM, Aneesh Kumar K.V wrote:
>>> Alexander Graf<agraf@suse.de>   writes:
>>>
>>>> On 27.09.2013, at 12:52, Aneesh Kumar K.V wrote:
>>>>
>>>>> "Aneesh Kumar K.V"<aneesh.kumar@linux.vnet.ibm.com>   writes:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> This patch series support enabling HV and PR KVM together in the same kernel. We
>>>>>> extend machine property with new property "kvm_type". A value of 1 will force HV
>>>>>> KVM and 2 PR KVM. The default value is 0 which will select the fastest KVM mode.
>>>>>> ie, HV if that is supported otherwise PR.
>>>>>>
>>>>>> With Qemu command line having
>>>>>>
>>>>>> -machine pseries,accel=kvm,kvm_type=1
>>>>>>
>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>> failed to initialize KVM: Invalid argument
>>>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>> failed to initialize KVM: Invalid argument
>>>>>> [root@llmp24l02 qemu]# modprobe  kvm-hv
>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>
>>>>>> now with
>>>>>>
>>>>>> -machine pseries,accel=kvm,kvm_type=2
>>>>>>
>>>>>> [root@llmp24l02 qemu]# rmmod kvm-pr
>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>> failed to initialize KVM: Invalid argument
>>>>>> [root@llmp24l02 qemu]#
>>>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>
>>>>>> if don't specify kvm_type machine property, it will take a default value 0,
>>>>>> which means fastest supported.
>>>>> Related qemu patch
>>>>>
>>>>> commit 8d139053177d48a70cb710b211ea4c2843eccdfb
>>>>> Author: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>>>> Date:   Mon Sep 23 12:28:37 2013 +0530
>>>>>
>>>>>      kvm: Add a new machine property kvm_type
>>>>>
>>>>>      Targets like ppc64 support different type of KVM, one which use
>>>>>      hypervisor mode and the other which doesn't. Add a new machine
>>>>>      property kvm_type that helps in selecting the respective ones
>>>>>
>>>>>      Signed-off-by: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>>> This really is too early, as we can't possibly run in HV mode for
>>>> non-pseries machines, so the interpretation (or at least sanity
>>>> checking) of what values are reasonable should occur in the
>>>> machine. That's why it's a variable in the "machine opts".
>>> With the current code CREATE_VM will fail, because we won't have
>>> kvm-hv.ko loaded and trying to create a vm with type 1 will fail.
>>> Now the challenge related to moving that to machine_init or later is, we
>>> depend on HV or PR callback early in CREATE_VM. With the changes we have
>>>
>>> int kvmppc_core_init_vm(struct kvm *kvm)
>>> {
>>>
>>> #ifdef CONFIG_PPC64
>>> 	INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
>>> 	INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
>>> #endif
>>>
>>> 	return kvm->arch.kvm_ops->init_vm(kvm);
>>> }
>>>
>>> Also the mmu notifier callback do end up calling kvm_unmap_hva etc which
>>> are all HV/PR dependent.
>> Yes, so we should verify in the machine models that we're runnable with
>> the currently selected type at least, to give the user a sensible error
>> message.
> Something like the below

I like that one a lot. Andreas, Paolo, what do you think?


Alex

>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 004184d..7d59ac1 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1337,6 +1337,21 @@ static void ppc_spapr_init(QEMUMachineInitArgs *args)
>       assert(spapr->fdt_skel != NULL);
>   }
>
> +static int spapr_get_vm_type(const char *vm_type)
> +{
> +    if (!vm_type)
> +        return 0;
> +
> +    if (!strcmp(vm_type, "HV"))
> +        return 1;
> +
> +    if (!strcmp(vm_type, "PR"))
> +        return 2;
> +
> +    hw_error("qemu: unknown kvm_type specified '%s'", vm_type);
> +    exit(1);
> +}
> +
>   static QEMUMachine spapr_machine = {
>       .name = "pseries",
>       .desc = "pSeries Logical Partition (PAPR compliant)",
> @@ -1347,6 +1362,7 @@ static QEMUMachine spapr_machine = {
>       .max_cpus = MAX_CPUS,
>       .no_parallel = 1,
>       .default_boot_order = NULL,
> +    .get_vm_type = spapr_get_vm_type,
>   };
>
>   static void spapr_machine_init(void)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 5a7ae9f..2130488 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -21,6 +21,8 @@ typedef void QEMUMachineResetFunc(void);
>
>   typedef void QEMUMachineHotAddCPUFunc(const int64_t id, Error **errp);
>
> +typedef int QEMUMachineGetVmTypeFunc(const char *arg);
> +
>   typedef struct QEMUMachine {
>       const char *name;
>       const char *alias;
> @@ -28,6 +30,7 @@ typedef struct QEMUMachine {
>       QEMUMachineInitFunc *init;
>       QEMUMachineResetFunc *reset;
>       QEMUMachineHotAddCPUFunc *hot_add_cpu;
> +    QEMUMachineGetVmTypeFunc *get_vm_type;
>       BlockInterfaceType block_default_type;
>       int max_cpus;
>       unsigned int no_serial:1,
> diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
> index e1f88bf..acc3d74 100644
> --- a/include/hw/xen/xen.h
> +++ b/include/hw/xen/xen.h
> @@ -36,7 +36,8 @@ void xen_cmos_set_s3_resume(void *opaque, int irq, int level);
>
>   qemu_irq *xen_interrupt_controller_init(void);
>
> -int xen_init(void);
> +typedef struct QEMUMachine QEMUMachine;
> +int xen_init(QEMUMachine *machine);
>   int xen_hvm_init(MemoryRegion **ram_memory);
>   void xenstore_store_pv_console_info(int i, struct CharDriverState *chr);
>
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index 9bbe3db..f25caec 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -142,8 +142,8 @@ typedef struct KVMState KVMState;
>   extern KVMState *kvm_state;
>
>   /* external API */
> -
> -int kvm_init(void);
> +typedef struct QEMUMachine QEMUMachine;
> +int kvm_init(QEMUMachine *machine);
>
>   int kvm_has_sync_mmu(void);
>   int kvm_has_vcpu_events(void);
> diff --git a/include/sysemu/qtest.h b/include/sysemu/qtest.h
> index 9a0c6b3..d71343d 100644
> --- a/include/sysemu/qtest.h
> +++ b/include/sysemu/qtest.h
> @@ -31,7 +31,8 @@ static inline int qtest_available(void)
>       return 1;
>   }
>
> -int qtest_init(void);
> +typedef struct QEMUMachine QEMUMachine;
> +int qtest_init(QEMUMachine *machine);
>   #else
>   static inline bool qtest_enabled(void)
>   {
> @@ -43,7 +44,7 @@ static inline int qtest_available(void)
>       return 0;
>   }
>
> -static inline int qtest_init(void)
> +static inline int qtest_init(QEMUMachine *machine)
>   {
>       return 0;
>   }
> diff --git a/kvm-all.c b/kvm-all.c
> index b87215c..3863abd 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -35,6 +35,8 @@
>   #include "qemu/event_notifier.h"
>   #include "trace.h"
>
> +#include "hw/boards.h"
> +
>   /* This check must be after config-host.h is included */
>   #ifdef CONFIG_EVENTFD
>   #include<sys/eventfd.h>
> @@ -1342,7 +1344,7 @@ static int kvm_max_vcpus(KVMState *s)
>       return 4;
>   }
>
> -int kvm_init(void)
> +int kvm_init(QEMUMachine *machine)
>   {
>       static const char upgrade_note[] =
>           "Please upgrade to at least kernel 2.6.29 or recent kvm-kmod\n"
> @@ -1350,7 +1352,7 @@ int kvm_init(void)
>       KVMState *s;
>       const KVMCapabilityInfo *missing_cap;
>       int ret;
> -    int i;
> +    int i, kvm_type = 0;
>       int max_vcpus;
>
>       s = g_malloc0(sizeof(KVMState));
> @@ -1407,7 +1409,11 @@ int kvm_init(void)
>           goto err;
>       }
>
> -    s->vmfd = kvm_ioctl(s, KVM_CREATE_VM, 0);
> +    if (machine->get_vm_type) {
> +        kvm_type = machine->get_vm_type(qemu_opt_get(qemu_get_machine_opts(),
> +                                                     "kvm_type"));
> +    }
> +    s->vmfd = kvm_ioctl(s, KVM_CREATE_VM, kvm_type);
>       if (s->vmfd<  0) {
>   #ifdef TARGET_S390X
>           fprintf(stderr, "Please add the 'switch_amode' kernel parameter to "
> diff --git a/kvm-stub.c b/kvm-stub.c
> index 548f471..ccb7b8c 100644
> --- a/kvm-stub.c
> +++ b/kvm-stub.c
> @@ -19,6 +19,8 @@
>   #include "hw/pci/msi.h"
>   #endif
>
> +#include "hw/boards.h"
> +
>   KVMState *kvm_state;
>   bool kvm_kernel_irqchip;
>   bool kvm_async_interrupts_allowed;
> @@ -33,7 +35,7 @@ int kvm_init_vcpu(CPUState *cpu)
>       return -ENOSYS;
>   }
>
> -int kvm_init(void)
> +int kvm_init(QEMUMachine *machine)
>   {
>       return -ENOSYS;
>   }
> diff --git a/qtest.c b/qtest.c
> index 584c707..ef3c473 100644
> --- a/qtest.c
> +++ b/qtest.c
> @@ -502,7 +502,7 @@ static void qtest_event(void *opaque, int event)
>       }
>   }
>
> -int qtest_init(void)
> +int qtest_init(QEMUMachine *machine)
>   {
>       CharDriverState *chr;
>
> diff --git a/vl.c b/vl.c
> index 4e709d5..7ecc581 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -427,7 +427,12 @@ static QemuOptsList qemu_machine_opts = {
>               .name = "usb",
>               .type = QEMU_OPT_BOOL,
>               .help = "Set on/off to enable/disable usb",
> +        },{
> +            .name = "kvm_type",
> +            .type = QEMU_OPT_STRING,
> +            .help = "Set to kvm type to be used in create vm ioctl",
>           },
> +
>           { /* End of list */ }
>       },
>   };
> @@ -2608,7 +2613,7 @@ static QEMUMachine *machine_parse(const char *name)
>       exit(!name || !is_help_option(name));
>   }
>
> -static int tcg_init(void)
> +static int tcg_init(QEMUMachine *machine)
>   {
>       tcg_exec_init(tcg_tb_size * 1024 * 1024);
>       return 0;
> @@ -2618,7 +2623,7 @@ static struct {
>       const char *opt_name;
>       const char *name;
>       int (*available)(void);
> -    int (*init)(void);
> +    int (*init)(QEMUMachine *);
>       bool *allowed;
>   } accel_list[] = {
>       { "tcg", "tcg", tcg_available, tcg_init,&tcg_allowed },
> @@ -2627,7 +2632,7 @@ static struct {
>       { "qtest", "QTest", qtest_available, qtest_init,&qtest_allowed },
>   };
>
> -static int configure_accelerator(void)
> +static int configure_accelerator(QEMUMachine *machine)
>   {
>       const char *p;
>       char buf[10];
> @@ -2654,7 +2659,7 @@ static int configure_accelerator(void)
>                       continue;
>                   }
>                   *(accel_list[i].allowed) = true;
> -                ret = accel_list[i].init();
> +                ret = accel_list[i].init(machine);
>                   if (ret<  0) {
>                       init_failed = true;
>                       fprintf(stderr, "failed to initialize %s: %s\n",
> @@ -4037,10 +4042,10 @@ int main(int argc, char **argv, char **envp)
>           exit(0);
>       }
>
> -    configure_accelerator();
> +    configure_accelerator(machine);
>
>       if (!qtest_enabled()&&  qtest_chrdev) {
> -        qtest_init();
> +        qtest_init(machine);
>       }
>
>       machine_opts = qemu_get_machine_opts();
> diff --git a/xen-all.c b/xen-all.c
> index 839f14f..ac3654b 100644
> --- a/xen-all.c
> +++ b/xen-all.c
> @@ -1000,7 +1000,7 @@ static void xen_exit_notifier(Notifier *n, void *data)
>       xs_daemon_close(state->xenstore);
>   }
>
> -int xen_init(void)
> +int xen_init(QEMUMachine *machine)
>   {
>       xen_xc = xen_xc_interface_open(0, 0, 0);
>       if (xen_xc == XC_HANDLER_INITIAL_VALUE) {
> diff --git a/xen-stub.c b/xen-stub.c
> index ad189a6..59927cb 100644
> --- a/xen-stub.c
> +++ b/xen-stub.c
> @@ -47,7 +47,7 @@ qemu_irq *xen_interrupt_controller_init(void)
>       return NULL;
>   }
>
> -int xen_init(void)
> +int xen_init(QEMUMachine *machine)
>   {
>       return -ENOSYS;
>   }
>
>>>
>>>
>>>> Also, users don't want to say type=0. They want to say type=PR or
>>>> type=HV or type=HV,PR. In fact, can't you make this a property of
>>>> -accel? Then it's truly accel specific and everything should be well.
>>> If we are doing this as machine property, we can't specify string,
>>> because "HV"/"PR" are all powerpc dependent, so parsing that is not
>>> possible in kvm_init in qemu. But, yes ideally it would be nice to be
>> Well, we could do the "name to integer" conversion in an arch specific
>> function, no?
>>
>>> able to speicy the type using string. I thought accel is a machine
>>> property, hence was not sure whether I can have additional properties
>>> against that. I was using it as below.
>>>
>>>    -machine pseries,accel=kvm,kvm_type=1
> Can we really specific -accel ? I check and I am finding that as machine
> property.
>
> -aneesh
>

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Paolo Bonzini]

Il 01/10/2013 13:36, Alexander Graf ha scritto:
>>>>
>>> Yes, so we should verify in the machine models that we're runnable with
>>> the currently selected type at least, to give the user a sensible error
>>> message.
>> Something like the below
> 
> I like that one a lot. Andreas, Paolo, what do you think?

Yes, it's fine.

Paolo

Re: [RFC PATCH 00/11 Allow PR and HV KVM to coexist in one kernel

[Alexander Graf]

On 10/01/2013 01:36 PM, Alexander Graf wrote:
> On 10/01/2013 01:26 PM, Aneesh Kumar K.V wrote:
>> Alexander Graf<agraf@suse.de>  writes:
>>
>>> On 09/30/2013 03:09 PM, Aneesh Kumar K.V wrote:
>>>> Alexander Graf<agraf@suse.de>   writes:
>>>>
>>>>> On 27.09.2013, at 12:52, Aneesh Kumar K.V wrote:
>>>>>
>>>>>> "Aneesh Kumar K.V"<aneesh.kumar@linux.vnet.ibm.com>   writes:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> This patch series support enabling HV and PR KVM together in the 
>>>>>>> same kernel. We
>>>>>>> extend machine property with new property "kvm_type". A value of 
>>>>>>> 1 will force HV
>>>>>>> KVM and 2 PR KVM. The default value is 0 which will select the 
>>>>>>> fastest KVM mode.
>>>>>>> ie, HV if that is supported otherwise PR.
>>>>>>>
>>>>>>> With Qemu command line having
>>>>>>>
>>>>>>> -machine pseries,accel=kvm,kvm_type=1
>>>>>>>
>>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>> failed to initialize KVM: Invalid argument
>>>>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>> failed to initialize KVM: Invalid argument
>>>>>>> [root@llmp24l02 qemu]# modprobe  kvm-hv
>>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>>
>>>>>>> now with
>>>>>>>
>>>>>>> -machine pseries,accel=kvm,kvm_type=2
>>>>>>>
>>>>>>> [root@llmp24l02 qemu]# rmmod kvm-pr
>>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>> failed to initialize KVM: Invalid argument
>>>>>>> [root@llmp24l02 qemu]#
>>>>>>> [root@llmp24l02 qemu]# modprobe kvm-pr
>>>>>>> [root@llmp24l02 qemu]# bash ../qemu
>>>>>>>
>>>>>>> if don't specify kvm_type machine property, it will take a 
>>>>>>> default value 0,
>>>>>>> which means fastest supported.
>>>>>> Related qemu patch
>>>>>>
>>>>>> commit 8d139053177d48a70cb710b211ea4c2843eccdfb
>>>>>> Author: Aneesh Kumar K.V<aneesh.kumar@linux.vnet.ibm.com>
>>>>>> Date:   Mon Sep 23 12:28:37 2013 +0530
>>>>>>
>>>>>>      kvm: Add a new machine property kvm_type
>>>>>>
>>>>>>      Targets like ppc64 support different type of KVM, one which use
>>>>>>      hypervisor mode and the other which doesn't. Add a new machine
>>>>>>      property kvm_type that helps in selecting the respective ones
>>>>>>
>>>>>>      Signed-off-by: Aneesh Kumar 
>>>>>> K.V<aneesh.kumar@linux.vnet.ibm.com>
>>>>> This really is too early, as we can't possibly run in HV mode for
>>>>> non-pseries machines, so the interpretation (or at least sanity
>>>>> checking) of what values are reasonable should occur in the
>>>>> machine. That's why it's a variable in the "machine opts".
>>>> With the current code CREATE_VM will fail, because we won't have
>>>> kvm-hv.ko loaded and trying to create a vm with type 1 will fail.
>>>> Now the challenge related to moving that to machine_init or later 
>>>> is, we
>>>> depend on HV or PR callback early in CREATE_VM. With the changes we 
>>>> have
>>>>
>>>> int kvmppc_core_init_vm(struct kvm *kvm)
>>>> {
>>>>
>>>> #ifdef CONFIG_PPC64
>>>>     INIT_LIST_HEAD(&kvm->arch.spapr_tce_tables);
>>>>     INIT_LIST_HEAD(&kvm->arch.rtas_tokens);
>>>> #endif
>>>>
>>>>     return kvm->arch.kvm_ops->init_vm(kvm);
>>>> }
>>>>
>>>> Also the mmu notifier callback do end up calling kvm_unmap_hva etc 
>>>> which
>>>> are all HV/PR dependent.
>>> Yes, so we should verify in the machine models that we're runnable with
>>> the currently selected type at least, to give the user a sensible error
>>> message.
>> Something like the below
>
> I like that one a lot. Andreas, Paolo, what do you think?

To clarify this a bit more, this patch is missing a few critical pieces:

   1) The default implementation for the kvm_type callback needs to 
error out when kvm_type is set to anything
   2) All non-papr-non-e500-PPC machines should force "PR only" mode and 
bail out on anything else
   3) We're missing sensible error messages for the user still when the 
VM could not get created

but I like the overall concept :).


Alex