[PATCH v7 0/3] KVM: PPC: Add FWNMI support for KVM guests on POWER

[PATCH v7 0/3] KVM: PPC: Add FWNMI support for KVM guests on POWER

[Mahesh J Salgaonkar]

From: Aravinda Prasad <aravinda@linux.vnet.ibm.com>

This series of patches add FWNMI support for KVM guests
on POWER.

Memory errors such as bit flips that cannot be corrected
by hardware is passed on to the kernel for handling
by raising machine check exception (an NMI). Upon such
machine check exceptions, if the address in error belongs
to the guest, the error is passed on to the guest
kernel for handling. However, for guest kernels that
have issued "ibm,nmi-register" call, QEMU should build
an error log and pass on the error log to the guest-
kernel registered machine check handler routine.

This patch series adds the functionality to pass on the
machine check exception to the guest kernel by
giving control to QEMU. QEMU builds the error log
and invokes the guest-kernel registered handler.

QEMU part can be found at:
http://lists.nongnu.org/archive/html/qemu-ppc/2015-12/msg00199.html

Change Log v7:
  - Move the FWNMI capability stuff inside #ifdef CONFIG_PPC_BOOK3S_64 block.
  - Split 2nd patch into two, one that adds mce hook and other that touches
    kvm code.

Change Log v6:
  - Deliver all MCE errors (handled/unhandled) for FWNMI capable guest.
  - Use kvm_run->flags to pass NMI disposition status.

Change Log v5:
  - Added capability documentation. No functionality/code change.

Change Log v4:
  - Allow host-side handling of the machine check exception before
    passing on the exception to the guest.

Change Log v3:
  - Split the patch into 2. First patch introduces the
    new capability while the second one enhances KVM to
    redirect MCE.
  - Fix access width bug

Change Log v2:
  - Added KVM capability
---

Aravinda Prasad (2):
      KVM: PPC: Add new capability to control MCE behaviour
      KVM: PPC: Exit guest upon MCE when FWNMI capability is enabled

Mahesh Salgaonkar (1):
      powerpc/powernv: Introduce a machine check hook for Guest MCEs.


 Documentation/virtual/kvm/api.txt       |   11 +++++++
 arch/powerpc/include/asm/kvm_host.h     |    3 ++
 arch/powerpc/include/asm/machdep.h      |    7 ++++
 arch/powerpc/include/asm/opal.h         |    4 ++
 arch/powerpc/include/uapi/asm/kvm.h     |    6 ++++
 arch/powerpc/kernel/asm-offsets.c       |    1 +
 arch/powerpc/kvm/book3s_hv.c            |   24 ++++++++++----
 arch/powerpc/kvm/book3s_hv_ras.c        |   18 ++++++++++-
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |   52 ++++++++++++++++++-------------
 arch/powerpc/kvm/powerpc.c              |    9 +++++
 arch/powerpc/platforms/powernv/opal.c   |   26 ++++++++++++++++
 arch/powerpc/platforms/powernv/setup.c  |    3 ++
 include/uapi/linux/kvm.h                |    1 +
 13 files changed, 135 insertions(+), 30 deletions(-)

--
Signature

[PATCH v7 1/3] KVM: PPC: Add new capability to control MCE behaviour

[Mahesh J Salgaonkar]

From: Aravinda Prasad <aravinda@linux.vnet.ibm.com>

This patch introduces a new KVM capability to control
how KVM behaves on machine check exception (MCE).
Without this capability, KVM redirects machine check
exceptions to guest's 0x200 vector, if the address in
error belongs to the guest. With this capability KVM
causes a guest exit with NMI exit reason.

The new capability is required to avoid problems if
a new kernel/KVM is used with an old QEMU for guests
that don't issue "ibm,nmi-register". As old QEMU does
not understand the NMI exit type, it treats it as a
fatal error. However, the guest could have handled
the machine check error if the exception was delivered
to guest's 0x200 interrupt vector instead of NMI exit
in case of old QEMU.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 Documentation/virtual/kvm/api.txt   |   11 +++++++++++
 arch/powerpc/include/asm/kvm_host.h |    1 +
 arch/powerpc/kernel/asm-offsets.c   |    1 +
 arch/powerpc/kvm/powerpc.c          |    9 +++++++++
 include/uapi/linux/kvm.h            |    1 +
 5 files changed, 23 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index fd10689..1c9d281 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -4164,6 +4164,17 @@ to take care of that.
 This capability can be enabled dynamically even if VCPUs were already
 created and are running.
 
+7.9 KVM_CAP_PPC_FWNMI
+
+Architectures: ppc
+Parameters: none
+
+With this capability a machine check exception in the guest address
+space will cause KVM to exit the guest with NMI exit reason. This
+enables QEMU to build error log and branch to guest kernel registered
+machine check handling routine. Without this capability KVM will
+branch to guests' 0x200 interrupt vector.
+
 8. Other capabilities.
 ----------------------
 
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 7bba8f4..4d02cbc 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -279,6 +279,7 @@ struct kvm_arch {
 	struct dentry *debugfs_dir;
 	struct dentry *htab_dentry;
 	struct kvm_resize_hpt *resize_hpt; /* protected by kvm->lock */
+	u8 fwnmi_enabled;
 #endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */
 #ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
 	struct mutex hpt_mutex;
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 4367e7d..0daa47b 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -481,6 +481,7 @@ int main(void)
 	OFFSET(KVM_ENABLED_HCALLS, kvm, arch.enabled_hcalls);
 	OFFSET(KVM_VRMA_SLB_V, kvm, arch.vrma_slb_v);
 	OFFSET(KVM_RADIX, kvm, arch.radix);
+	OFFSET(KVM_FWNMI, kvm, arch.fwnmi_enabled);
 	OFFSET(VCPU_DSISR, kvm_vcpu, arch.shregs.dsisr);
 	OFFSET(VCPU_DAR, kvm_vcpu, arch.shregs.dar);
 	OFFSET(VCPU_VPA, kvm_vcpu, arch.vpa.pinned_addr);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 95c91a9..b8e8cd4 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -618,6 +618,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		/* Disable this on POWER9 until code handles new HPTE format */
 		r = !!hv_enabled && !cpu_has_feature(CPU_FTR_ARCH_300);
 		break;
+	case KVM_CAP_PPC_FWNMI:
+		r = 1;
+		break;
 #endif
 	case KVM_CAP_PPC_HTM:
 		r = cpu_has_feature(CPU_FTR_TM_COMP) &&
@@ -1226,6 +1229,12 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 		break;
 	}
 #endif /* CONFIG_KVM_XICS */
+#ifdef CONFIG_PPC_BOOK3S_64
+	case KVM_CAP_PPC_FWNMI:
+		r = 0;
+		vcpu->kvm->arch.fwnmi_enabled = true;
+		break;
+#endif /* CONFIG_PPC_BOOK3S_64 */
 	default:
 		r = -EINVAL;
 		break;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index f51d508..d5428a7 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -883,6 +883,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_PPC_MMU_RADIX 134
 #define KVM_CAP_PPC_MMU_HASH_V3 135
 #define KVM_CAP_IMMEDIATE_EXIT 136
+#define KVM_CAP_PPC_FWNMI 137
 
 #ifdef KVM_CAP_IRQ_ROUTING
 

[PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[Mahesh J Salgaonkar]

From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>

This patch introduces a mce hook which is invoked at the time of guest
exit to facilitate the host-side handling of machine check exception
before the exception is passed on to the guest. This hook will be invoked
from host virtual mode from KVM (before exiting the guest with
KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.

Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/machdep.h     |    7 +++++++
 arch/powerpc/include/asm/opal.h        |    4 ++++
 arch/powerpc/platforms/powernv/opal.c  |   26 ++++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/setup.c |    3 +++
 4 files changed, 40 insertions(+)

diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index 5011b69..9d74e7a 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -15,6 +15,7 @@
 #include <linux/export.h>
 
 #include <asm/setup.h>
+#include <asm/mce.h>
 
 /* We export this macro for external modules like Alsa to know if
  * ppc_md.feature_call is implemented or not
@@ -112,6 +113,12 @@ struct machdep_calls {
 	/* Called during machine check exception to retrive fixup address. */
 	bool		(*mce_check_early_recovery)(struct pt_regs *regs);
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+	/* Called after KVM interrupt handler finishes handling MCE for guest */
+	int		(*machine_check_exception_guest)
+					(struct machine_check_event *evt);
+#endif
+
 	/* Motherboard/chipset features. This is a kind of general purpose
 	 * hook used to control some machine specific features (like reset
 	 * lines, chip power control, etc...).
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 1ff03a6..9b1fcbf 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -17,6 +17,7 @@
 #ifndef __ASSEMBLY__
 
 #include <linux/notifier.h>
+#include <asm/mce.h>
 
 /* We calculate number of sg entries based on PAGE_SIZE */
 #define SG_ENTRIES_PER_NODE ((PAGE_SIZE - 16) / sizeof(struct opal_sg_entry))
@@ -273,6 +274,9 @@ extern int opal_hmi_handler_init(void);
 extern int opal_event_init(void);
 
 extern int opal_machine_check(struct pt_regs *regs);
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+extern int opal_machine_check_guest(struct machine_check_event *evt);
+#endif
 extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
 extern int opal_hmi_exception_early(struct pt_regs *regs);
 extern int opal_handle_hmi_exception(struct pt_regs *regs);
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index e0f856b..5e633a4 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
 	return 0;
 }
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+/*
+ * opal_machine_check_guest() is a hook which is invoked at the time
+ * of guest exit to facilitate the host-side handling of machine check
+ * exception before the exception is passed on to the guest. This hook
+ * is invoked from host virtual mode from KVM (before exiting the guest
+ * with KVM_EXIT_NMI reason) for machine check exception that occurs in
+ * the guest.
+ *
+ * Currently no action is performed in the host other than printing the
+ * event information. The machine check exception is passed on to the
+ * guest kernel and the guest kernel will attempt for recovery.
+ */
+int opal_machine_check_guest(struct machine_check_event *evt)
+{
+	/* Print things out */
+	if (evt->version != MCE_V1) {
+		pr_err("Machine Check Exception, Unknown event version %d !\n",
+		       evt->version);
+		return 0;
+	}
+	machine_check_print_event_info(evt);
+	return 0;
+}
+#endif
+
 /* Early hmi handler called in real mode. */
 int opal_hmi_exception_early(struct pt_regs *regs)
 {
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
index d50c7d9..333ee09 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -264,6 +264,9 @@ static void __init pnv_setup_machdep_opal(void)
 	ppc_md.mce_check_early_recovery = opal_mce_check_early_recovery;
 	ppc_md.hmi_exception_early = opal_hmi_exception_early;
 	ppc_md.handle_hmi_exception = opal_handle_hmi_exception;
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+	ppc_md.machine_check_exception_guest = opal_machine_check_guest;
+#endif
 }
 
 static int __init pnv_probe(void)

[PATCH v7 3/3] KVM: PPC: Exit guest upon MCE when FWNMI capability is enabled

[Mahesh J Salgaonkar]

From: Aravinda Prasad <aravinda@linux.vnet.ibm.com>

Enhance KVM to cause a guest exit with KVM_EXIT_NMI
exit reason upon a machine check exception (MCE) in
the guest address space if the KVM_CAP_PPC_FWNMI
capability is enabled (instead of delivering a 0x200
interrupt to guest). This enables QEMU to build error
log and deliver machine check exception to guest via
guest registered machine check handler.

This approach simplifies the delivery of machine
check exception to guest OS compared to the earlier
approach of KVM directly invoking 0x200 guest interrupt
vector.

This design/approach is based on the feedback for the
QEMU patches to handle machine check exception. Details
of earlier approach of handling machine check exception
in QEMU and related discussions can be found at:

https://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg00813.html

Note:

This patch invokes an mce hook at the time of guest exit to
facilitate the host-side handling of machine check exception
before the exception is passed on to the guest. Hence, the
host-side handling which was performed earlier via machine_check_fwnmi
is removed.

The reasons for this approach is (i) it is not possible
to distinguish whether the exception occurred in the
guest or the host from the pt_regs passed on the
machine_check_exception(). Hence machine_check_exception()
calls panic, instead of passing on the exception to
the guest, if the machine check exception is not
recoverable. (ii) the approach introduced in this
patch gives opportunity to the host kernel to perform
actions in virtual mode before passing on the exception
to the guest. This approach does not require complex
tweaks to machine_check_fwnmi and friends.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_host.h     |    2 +
 arch/powerpc/include/uapi/asm/kvm.h     |    6 ++++
 arch/powerpc/kvm/book3s_hv.c            |   24 ++++++++++----
 arch/powerpc/kvm/book3s_hv_ras.c        |   18 ++++++++++-
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |   52 ++++++++++++++++++-------------
 5 files changed, 72 insertions(+), 30 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 4d02cbc..0627a5a 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -35,6 +35,7 @@
 #include <asm/page.h>
 #include <asm/cacheflush.h>
 #include <asm/hvcall.h>
+#include <asm/mce.h>
 
 #define KVM_MAX_VCPUS		NR_CPUS
 #define KVM_MAX_VCORES		NR_CPUS
@@ -675,6 +676,7 @@ struct kvm_vcpu_arch {
 	int prev_cpu;
 	bool timer_running;
 	wait_queue_head_t cpu_run;
+	struct machine_check_event mce_evt; /* Valid if trap == 0x200 */
 
 	struct kvm_vcpu_arch_shared *shared;
 #if defined(CONFIG_PPC_BOOK3S_64) && defined(CONFIG_KVM_BOOK3S_PR_POSSIBLE)
diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
index 4edbe4b..1fdf64c 100644
--- a/arch/powerpc/include/uapi/asm/kvm.h
+++ b/arch/powerpc/include/uapi/asm/kvm.h
@@ -57,6 +57,12 @@ struct kvm_regs {
 
 #define KVM_SREGS_E_FSL_PIDn	(1 << 0) /* PID1/PID2 */
 
+/* flags for kvm_run.flags */
+#define KVM_RUN_PPC_NMI_DISP_MASK		(3 << 0)
+#define   KVM_RUN_PPC_NMI_DISP_FULLY_RECOV	(1 << 0)
+#define   KVM_RUN_PPC_NMI_DISP_LIMITED_RECOV	(2 << 0)
+#define   KVM_RUN_PPC_NMI_DISP_NOT_RECOV	(3 << 0)
+
 /*
  * Feature bits indicate which sections of the sregs struct are valid,
  * both in KVM_GET_SREGS and KVM_SET_SREGS.  On KVM_SET_SREGS, registers
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 1ec86d9..4d488b1 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -966,15 +966,25 @@ static int kvmppc_handle_exit_hv(struct kvm_run *run, struct kvm_vcpu *vcpu,
 		r = RESUME_GUEST;
 		break;
 	case BOOK3S_INTERRUPT_MACHINE_CHECK:
+		/* Exit to guest with KVM_EXIT_NMI as exit reason */
+		run->exit_reason = KVM_EXIT_NMI;
+		run->hw.hardware_exit_reason = vcpu->arch.trap;
+		/* Clear out the old NMI status from run->flags */
+		run->flags &= ~KVM_RUN_PPC_NMI_DISP_MASK;
+		/* Now set the NMI status */
+		if (vcpu->arch.mce_evt.disposition == MCE_DISPOSITION_RECOVERED)
+			run->flags |= KVM_RUN_PPC_NMI_DISP_FULLY_RECOV;
+		else
+			run->flags |= KVM_RUN_PPC_NMI_DISP_NOT_RECOV;
+
+		r = RESUME_HOST;
 		/*
-		 * Deliver a machine check interrupt to the guest.
-		 * We have to do this, even if the host has handled the
-		 * machine check, because machine checks use SRR0/1 and
-		 * the interrupt might have trashed guest state in them.
+		 * Invoke host-kernel handler to perform any host-side
+		 * handling before exiting the guest.
 		 */
-		kvmppc_book3s_queue_irqprio(vcpu,
-					    BOOK3S_INTERRUPT_MACHINE_CHECK);
-		r = RESUME_GUEST;
+		if (ppc_md.machine_check_exception_guest)
+			ppc_md.machine_check_exception_guest(
+							&vcpu->arch.mce_evt);
 		break;
 	case BOOK3S_INTERRUPT_PROGRAM:
 	{
diff --git a/arch/powerpc/kvm/book3s_hv_ras.c b/arch/powerpc/kvm/book3s_hv_ras.c
index 7ef0993..c356f9a 100644
--- a/arch/powerpc/kvm/book3s_hv_ras.c
+++ b/arch/powerpc/kvm/book3s_hv_ras.c
@@ -130,12 +130,28 @@ static long kvmppc_realmode_mc_power7(struct kvm_vcpu *vcpu)
 
 out:
 	/*
+	 * For guest that supports FWNMI capability, hook the MCE event into
+	 * vcpu structure. We are going to exit the guest with KVM_EXIT_NMI
+	 * exit reason. On our way to exit we will pull this event from vcpu
+	 * structure and print it from thread 0 of the core/subcore.
+	 *
+	 * For guest that does not support FWNMI capability (old QEMU):
 	 * We are now going enter guest either through machine check
 	 * interrupt (for unhandled errors) or will continue from
 	 * current HSRR0 (for handled errors) in guest. Hence
 	 * queue up the event so that we can log it from host console later.
 	 */
-	machine_check_queue_event();
+	if (vcpu->kvm->arch.fwnmi_enabled) {
+		/*
+		 * Hook up the mce event on to vcpu structure.
+		 * First clear the old event.
+		 */
+		memset(&vcpu->arch.mce_evt, 0, sizeof(vcpu->arch.mce_evt));
+		if (get_mce_event(&mce_evt, MCE_EVENT_RELEASE)) {
+			vcpu->arch.mce_evt = mce_evt;
+		}
+	} else
+		machine_check_queue_event();
 
 	return handled;
 }
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 7c6477d..b6fe741 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -134,15 +134,13 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	stb	r0, HSTATE_HWTHREAD_REQ(r13)
 
 	/*
-	 * For external and machine check interrupts, we need
-	 * to call the Linux handler to process the interrupt.
-	 * We do that by jumping to absolute address 0x500 for
-	 * external interrupts, or the machine_check_fwnmi label
-	 * for machine checks (since firmware might have patched
-	 * the vector area at 0x200).  The [h]rfid at the end of the
-	 * handler will return to the book3s_hv_interrupts.S code.
-	 * For other interrupts we do the rfid to get back
-	 * to the book3s_hv_interrupts.S code here.
+	 * For external interrupts we need to call the Linux
+	 * handler to process the interrupt. We do that by jumping
+	 * to absolute address 0x500 for external interrupts.
+	 * The [h]rfid at the end of the handler will return to
+	 * the book3s_hv_interrupts.S code. For other interrupts
+	 * we do the rfid to get back to the book3s_hv_interrupts.S
+	 * code here.
 	 */
 	ld	r8, 112+PPC_LR_STKOFF(r1)
 	addi	r1, r1, 112
@@ -157,7 +155,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	andi.	r0, r0, MSR_IR		/* in real mode? */
 	bne	.Lvirt_return
 
-	cmpwi	cr1, r12, BOOK3S_INTERRUPT_MACHINE_CHECK
 	cmpwi	r12, BOOK3S_INTERRUPT_EXTERNAL
 	beq	11f
 	cmpwi	r12, BOOK3S_INTERRUPT_H_DOORBELL
@@ -172,7 +169,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	mtmsrd	r6, 1			/* Clear RI in MSR */
 	mtsrr0	r8
 	mtsrr1	r7
-	beq	cr1, 13f		/* machine check */
+	/*
+	 * BOOK3S_INTERRUPT_MACHINE_CHECK is handled at the
+	 * time of guest exit
+	 */
 	RFI
 
 	/* On POWER7, we have external interrupts set to use HSRR0/1 */
@@ -180,8 +180,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	mtspr	SPRN_HSRR1, r7
 	ba	0x500
 
-13:	b	machine_check_fwnmi
-
 14:	mtspr	SPRN_HSRR0, r8
 	mtspr	SPRN_HSRR1, r7
 	b	hmi_exception_after_realmode
@@ -2494,22 +2492,32 @@ machine_check_realmode:
 	ld	r9, HSTATE_KVM_VCPU(r13)
 	li	r12, BOOK3S_INTERRUPT_MACHINE_CHECK
 	/*
-	 * Deliver unhandled/fatal (e.g. UE) MCE errors to guest through
-	 * machine check interrupt (set HSRR0 to 0x200). And for handled
-	 * errors (no-fatal), just go back to guest execution with current
-	 * HSRR0 instead of exiting guest. This new approach will inject
-	 * machine check to guest for fatal error causing guest to crash.
-	 *
-	 * The old code used to return to host for unhandled errors which
-	 * was causing guest to hang with soft lockups inside guest and
-	 * makes it difficult to recover guest instance.
+	 * For the guest that is FWNMI capable, deliver all the MCE errors
+	 * (handled/unhandled) by exiting the guest with KVM_EXIT_NMI exit
+	 * reason. This new approach injects machine check errors in guest
+	 * address space to guest with additional information in the form
+	 * of RTAS event, thus enabling guest kernel to suitably handle
+	 * such errors.
 	 *
+	 * For the guest that is not FWNMI capable (old QEMU) fallback
+	 * to old behaviour for backward compatibility:
+	 * Deliver unhandled/fatal (e.g. UE) MCE errors to guest either
+	 * through machine check interrupt (set HSRR0 to 0x200).
+	 * For handled errors (no-fatal), just go back to guest execution
+	 * with current HSRR0.
 	 * if we receive machine check with MSR(RI=0) then deliver it to
 	 * guest as machine check causing guest to crash.
 	 */
 	ld	r11, VCPU_MSR(r9)
 	rldicl.	r0, r11, 64-MSR_HV_LG, 63 /* check if it happened in HV mode */
 	bne	mc_cont			/* if so, exit to host */
+	/* Check if guest is capable of handling NMI exit */
+	ld	r10, VCPU_KVM(r9)
+	lbz	r10, KVM_FWNMI(r10)
+	cmpdi	r10, 1			/* FWNMI capable? */
+	beq	mc_cont			/* if so, exit with KVM_EXIT_NMI. */
+
+	/* if not, fall through for backward compatibility. */
 	andi.	r10, r11, MSR_RI	/* check for unrecoverable exception */
 	beq	1f			/* Deliver a machine check to guest */
 	ld	r10, VCPU_PC(r9)

[PATCH v7 0/3] KVM: PPC: Add FWNMI support for KVM guests on POWER

[Mahesh J Salgaonkar]

From: Aravinda Prasad <aravinda@linux.vnet.ibm.com>

This series of patches add FWNMI support for KVM guests
on POWER.

Memory errors such as bit flips that cannot be corrected
by hardware is passed on to the kernel for handling
by raising machine check exception (an NMI). Upon such
machine check exceptions, if the address in error belongs
to the guest, the error is passed on to the guest
kernel for handling. However, for guest kernels that
have issued "ibm,nmi-register" call, QEMU should build
an error log and pass on the error log to the guest-
kernel registered machine check handler routine.

This patch series adds the functionality to pass on the
machine check exception to the guest kernel by
giving control to QEMU. QEMU builds the error log
and invokes the guest-kernel registered handler.

QEMU part can be found at:
http://lists.nongnu.org/archive/html/qemu-ppc/2015-12/msg00199.html

Change Log v7:
  - Move the FWNMI capability stuff inside #ifdef CONFIG_PPC_BOOK3S_64 block.
  - Split 2nd patch into two, one that adds mce hook and other that touches
    kvm code.

Change Log v6:
  - Deliver all MCE errors (handled/unhandled) for FWNMI capable guest.
  - Use kvm_run->flags to pass NMI disposition status.

Change Log v5:
  - Added capability documentation. No functionality/code change.

Change Log v4:
  - Allow host-side handling of the machine check exception before
    passing on the exception to the guest.

Change Log v3:
  - Split the patch into 2. First patch introduces the
    new capability while the second one enhances KVM to
    redirect MCE.
  - Fix access width bug

Change Log v2:
  - Added KVM capability
---

Aravinda Prasad (2):
      KVM: PPC: Add new capability to control MCE behaviour
      KVM: PPC: Exit guest upon MCE when FWNMI capability is enabled

Mahesh Salgaonkar (1):
      powerpc/powernv: Introduce a machine check hook for Guest MCEs.


 Documentation/virtual/kvm/api.txt       |   11 +++++++
 arch/powerpc/include/asm/kvm_host.h     |    3 ++
 arch/powerpc/include/asm/machdep.h      |    7 ++++
 arch/powerpc/include/asm/opal.h         |    4 ++
 arch/powerpc/include/uapi/asm/kvm.h     |    6 ++++
 arch/powerpc/kernel/asm-offsets.c       |    1 +
 arch/powerpc/kvm/book3s_hv.c            |   24 ++++++++++----
 arch/powerpc/kvm/book3s_hv_ras.c        |   18 ++++++++++-
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |   52 ++++++++++++++++++-------------
 arch/powerpc/kvm/powerpc.c              |    9 +++++
 arch/powerpc/platforms/powernv/opal.c   |   26 ++++++++++++++++
 arch/powerpc/platforms/powernv/setup.c  |    3 ++
 include/uapi/linux/kvm.h                |    1 +
 13 files changed, 135 insertions(+), 30 deletions(-)

--
Signature

[PATCH v7 1/3] KVM: PPC: Add new capability to control MCE behaviour

[Mahesh J Salgaonkar]

From: Aravinda Prasad <aravinda@linux.vnet.ibm.com>

This patch introduces a new KVM capability to control
how KVM behaves on machine check exception (MCE).
Without this capability, KVM redirects machine check
exceptions to guest's 0x200 vector, if the address in
error belongs to the guest. With this capability KVM
causes a guest exit with NMI exit reason.

The new capability is required to avoid problems if
a new kernel/KVM is used with an old QEMU for guests
that don't issue "ibm,nmi-register". As old QEMU does
not understand the NMI exit type, it treats it as a
fatal error. However, the guest could have handled
the machine check error if the exception was delivered
to guest's 0x200 interrupt vector instead of NMI exit
in case of old QEMU.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 Documentation/virtual/kvm/api.txt   |   11 +++++++++++
 arch/powerpc/include/asm/kvm_host.h |    1 +
 arch/powerpc/kernel/asm-offsets.c   |    1 +
 arch/powerpc/kvm/powerpc.c          |    9 +++++++++
 include/uapi/linux/kvm.h            |    1 +
 5 files changed, 23 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index fd10689..1c9d281 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -4164,6 +4164,17 @@ to take care of that.
 This capability can be enabled dynamically even if VCPUs were already
 created and are running.
 
+7.9 KVM_CAP_PPC_FWNMI
+
+Architectures: ppc
+Parameters: none
+
+With this capability a machine check exception in the guest address
+space will cause KVM to exit the guest with NMI exit reason. This
+enables QEMU to build error log and branch to guest kernel registered
+machine check handling routine. Without this capability KVM will
+branch to guests' 0x200 interrupt vector.
+
 8. Other capabilities.
 ----------------------
 
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 7bba8f4..4d02cbc 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -279,6 +279,7 @@ struct kvm_arch {
 	struct dentry *debugfs_dir;
 	struct dentry *htab_dentry;
 	struct kvm_resize_hpt *resize_hpt; /* protected by kvm->lock */
+	u8 fwnmi_enabled;
 #endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */
 #ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
 	struct mutex hpt_mutex;
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 4367e7d..0daa47b 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -481,6 +481,7 @@ int main(void)
 	OFFSET(KVM_ENABLED_HCALLS, kvm, arch.enabled_hcalls);
 	OFFSET(KVM_VRMA_SLB_V, kvm, arch.vrma_slb_v);
 	OFFSET(KVM_RADIX, kvm, arch.radix);
+	OFFSET(KVM_FWNMI, kvm, arch.fwnmi_enabled);
 	OFFSET(VCPU_DSISR, kvm_vcpu, arch.shregs.dsisr);
 	OFFSET(VCPU_DAR, kvm_vcpu, arch.shregs.dar);
 	OFFSET(VCPU_VPA, kvm_vcpu, arch.vpa.pinned_addr);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 95c91a9..b8e8cd4 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -618,6 +618,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		/* Disable this on POWER9 until code handles new HPTE format */
 		r = !!hv_enabled && !cpu_has_feature(CPU_FTR_ARCH_300);
 		break;
+	case KVM_CAP_PPC_FWNMI:
+		r = 1;
+		break;
 #endif
 	case KVM_CAP_PPC_HTM:
 		r = cpu_has_feature(CPU_FTR_TM_COMP) &&
@@ -1226,6 +1229,12 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 		break;
 	}
 #endif /* CONFIG_KVM_XICS */
+#ifdef CONFIG_PPC_BOOK3S_64
+	case KVM_CAP_PPC_FWNMI:
+		r = 0;
+		vcpu->kvm->arch.fwnmi_enabled = true;
+		break;
+#endif /* CONFIG_PPC_BOOK3S_64 */
 	default:
 		r = -EINVAL;
 		break;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index f51d508..d5428a7 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -883,6 +883,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_PPC_MMU_RADIX 134
 #define KVM_CAP_PPC_MMU_HASH_V3 135
 #define KVM_CAP_IMMEDIATE_EXIT 136
+#define KVM_CAP_PPC_FWNMI 137
 
 #ifdef KVM_CAP_IRQ_ROUTING
 

[PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[Mahesh J Salgaonkar]

From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>

This patch introduces a mce hook which is invoked at the time of guest
exit to facilitate the host-side handling of machine check exception
before the exception is passed on to the guest. This hook will be invoked
from host virtual mode from KVM (before exiting the guest with
KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.

Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/machdep.h     |    7 +++++++
 arch/powerpc/include/asm/opal.h        |    4 ++++
 arch/powerpc/platforms/powernv/opal.c  |   26 ++++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/setup.c |    3 +++
 4 files changed, 40 insertions(+)

diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index 5011b69..9d74e7a 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -15,6 +15,7 @@
 #include <linux/export.h>
 
 #include <asm/setup.h>
+#include <asm/mce.h>
 
 /* We export this macro for external modules like Alsa to know if
  * ppc_md.feature_call is implemented or not
@@ -112,6 +113,12 @@ struct machdep_calls {
 	/* Called during machine check exception to retrive fixup address. */
 	bool		(*mce_check_early_recovery)(struct pt_regs *regs);
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+	/* Called after KVM interrupt handler finishes handling MCE for guest */
+	int		(*machine_check_exception_guest)
+					(struct machine_check_event *evt);
+#endif
+
 	/* Motherboard/chipset features. This is a kind of general purpose
 	 * hook used to control some machine specific features (like reset
 	 * lines, chip power control, etc...).
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 1ff03a6..9b1fcbf 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -17,6 +17,7 @@
 #ifndef __ASSEMBLY__
 
 #include <linux/notifier.h>
+#include <asm/mce.h>
 
 /* We calculate number of sg entries based on PAGE_SIZE */
 #define SG_ENTRIES_PER_NODE ((PAGE_SIZE - 16) / sizeof(struct opal_sg_entry))
@@ -273,6 +274,9 @@ extern int opal_hmi_handler_init(void);
 extern int opal_event_init(void);
 
 extern int opal_machine_check(struct pt_regs *regs);
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+extern int opal_machine_check_guest(struct machine_check_event *evt);
+#endif
 extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
 extern int opal_hmi_exception_early(struct pt_regs *regs);
 extern int opal_handle_hmi_exception(struct pt_regs *regs);
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index e0f856b..5e633a4 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
 	return 0;
 }
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+/*
+ * opal_machine_check_guest() is a hook which is invoked at the time
+ * of guest exit to facilitate the host-side handling of machine check
+ * exception before the exception is passed on to the guest. This hook
+ * is invoked from host virtual mode from KVM (before exiting the guest
+ * with KVM_EXIT_NMI reason) for machine check exception that occurs in
+ * the guest.
+ *
+ * Currently no action is performed in the host other than printing the
+ * event information. The machine check exception is passed on to the
+ * guest kernel and the guest kernel will attempt for recovery.
+ */
+int opal_machine_check_guest(struct machine_check_event *evt)
+{
+	/* Print things out */
+	if (evt->version != MCE_V1) {
+		pr_err("Machine Check Exception, Unknown event version %d !\n",
+		       evt->version);
+		return 0;
+	}
+	machine_check_print_event_info(evt);
+	return 0;
+}
+#endif
+
 /* Early hmi handler called in real mode. */
 int opal_hmi_exception_early(struct pt_regs *regs)
 {
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
index d50c7d9..333ee09 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -264,6 +264,9 @@ static void __init pnv_setup_machdep_opal(void)
 	ppc_md.mce_check_early_recovery = opal_mce_check_early_recovery;
 	ppc_md.hmi_exception_early = opal_hmi_exception_early;
 	ppc_md.handle_hmi_exception = opal_handle_hmi_exception;
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+	ppc_md.machine_check_exception_guest = opal_machine_check_guest;
+#endif
 }
 
 static int __init pnv_probe(void)

[PATCH v7 3/3] KVM: PPC: Exit guest upon MCE when FWNMI capability is enabled

[Mahesh J Salgaonkar]

From: Aravinda Prasad <aravinda@linux.vnet.ibm.com>

Enhance KVM to cause a guest exit with KVM_EXIT_NMI
exit reason upon a machine check exception (MCE) in
the guest address space if the KVM_CAP_PPC_FWNMI
capability is enabled (instead of delivering a 0x200
interrupt to guest). This enables QEMU to build error
log and deliver machine check exception to guest via
guest registered machine check handler.

This approach simplifies the delivery of machine
check exception to guest OS compared to the earlier
approach of KVM directly invoking 0x200 guest interrupt
vector.

This design/approach is based on the feedback for the
QEMU patches to handle machine check exception. Details
of earlier approach of handling machine check exception
in QEMU and related discussions can be found at:

https://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg00813.html

Note:

This patch invokes an mce hook at the time of guest exit to
facilitate the host-side handling of machine check exception
before the exception is passed on to the guest. Hence, the
host-side handling which was performed earlier via machine_check_fwnmi
is removed.

The reasons for this approach is (i) it is not possible
to distinguish whether the exception occurred in the
guest or the host from the pt_regs passed on the
machine_check_exception(). Hence machine_check_exception()
calls panic, instead of passing on the exception to
the guest, if the machine check exception is not
recoverable. (ii) the approach introduced in this
patch gives opportunity to the host kernel to perform
actions in virtual mode before passing on the exception
to the guest. This approach does not require complex
tweaks to machine_check_fwnmi and friends.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/kvm_host.h     |    2 +
 arch/powerpc/include/uapi/asm/kvm.h     |    6 ++++
 arch/powerpc/kvm/book3s_hv.c            |   24 ++++++++++----
 arch/powerpc/kvm/book3s_hv_ras.c        |   18 ++++++++++-
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |   52 ++++++++++++++++++-------------
 5 files changed, 72 insertions(+), 30 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 4d02cbc..0627a5a 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -35,6 +35,7 @@
 #include <asm/page.h>
 #include <asm/cacheflush.h>
 #include <asm/hvcall.h>
+#include <asm/mce.h>
 
 #define KVM_MAX_VCPUS		NR_CPUS
 #define KVM_MAX_VCORES		NR_CPUS
@@ -675,6 +676,7 @@ struct kvm_vcpu_arch {
 	int prev_cpu;
 	bool timer_running;
 	wait_queue_head_t cpu_run;
+	struct machine_check_event mce_evt; /* Valid if trap = 0x200 */
 
 	struct kvm_vcpu_arch_shared *shared;
 #if defined(CONFIG_PPC_BOOK3S_64) && defined(CONFIG_KVM_BOOK3S_PR_POSSIBLE)
diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
index 4edbe4b..1fdf64c 100644
--- a/arch/powerpc/include/uapi/asm/kvm.h
+++ b/arch/powerpc/include/uapi/asm/kvm.h
@@ -57,6 +57,12 @@ struct kvm_regs {
 
 #define KVM_SREGS_E_FSL_PIDn	(1 << 0) /* PID1/PID2 */
 
+/* flags for kvm_run.flags */
+#define KVM_RUN_PPC_NMI_DISP_MASK		(3 << 0)
+#define   KVM_RUN_PPC_NMI_DISP_FULLY_RECOV	(1 << 0)
+#define   KVM_RUN_PPC_NMI_DISP_LIMITED_RECOV	(2 << 0)
+#define   KVM_RUN_PPC_NMI_DISP_NOT_RECOV	(3 << 0)
+
 /*
  * Feature bits indicate which sections of the sregs struct are valid,
  * both in KVM_GET_SREGS and KVM_SET_SREGS.  On KVM_SET_SREGS, registers
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 1ec86d9..4d488b1 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -966,15 +966,25 @@ static int kvmppc_handle_exit_hv(struct kvm_run *run, struct kvm_vcpu *vcpu,
 		r = RESUME_GUEST;
 		break;
 	case BOOK3S_INTERRUPT_MACHINE_CHECK:
+		/* Exit to guest with KVM_EXIT_NMI as exit reason */
+		run->exit_reason = KVM_EXIT_NMI;
+		run->hw.hardware_exit_reason = vcpu->arch.trap;
+		/* Clear out the old NMI status from run->flags */
+		run->flags &= ~KVM_RUN_PPC_NMI_DISP_MASK;
+		/* Now set the NMI status */
+		if (vcpu->arch.mce_evt.disposition = MCE_DISPOSITION_RECOVERED)
+			run->flags |= KVM_RUN_PPC_NMI_DISP_FULLY_RECOV;
+		else
+			run->flags |= KVM_RUN_PPC_NMI_DISP_NOT_RECOV;
+
+		r = RESUME_HOST;
 		/*
-		 * Deliver a machine check interrupt to the guest.
-		 * We have to do this, even if the host has handled the
-		 * machine check, because machine checks use SRR0/1 and
-		 * the interrupt might have trashed guest state in them.
+		 * Invoke host-kernel handler to perform any host-side
+		 * handling before exiting the guest.
 		 */
-		kvmppc_book3s_queue_irqprio(vcpu,
-					    BOOK3S_INTERRUPT_MACHINE_CHECK);
-		r = RESUME_GUEST;
+		if (ppc_md.machine_check_exception_guest)
+			ppc_md.machine_check_exception_guest(
+							&vcpu->arch.mce_evt);
 		break;
 	case BOOK3S_INTERRUPT_PROGRAM:
 	{
diff --git a/arch/powerpc/kvm/book3s_hv_ras.c b/arch/powerpc/kvm/book3s_hv_ras.c
index 7ef0993..c356f9a 100644
--- a/arch/powerpc/kvm/book3s_hv_ras.c
+++ b/arch/powerpc/kvm/book3s_hv_ras.c
@@ -130,12 +130,28 @@ static long kvmppc_realmode_mc_power7(struct kvm_vcpu *vcpu)
 
 out:
 	/*
+	 * For guest that supports FWNMI capability, hook the MCE event into
+	 * vcpu structure. We are going to exit the guest with KVM_EXIT_NMI
+	 * exit reason. On our way to exit we will pull this event from vcpu
+	 * structure and print it from thread 0 of the core/subcore.
+	 *
+	 * For guest that does not support FWNMI capability (old QEMU):
 	 * We are now going enter guest either through machine check
 	 * interrupt (for unhandled errors) or will continue from
 	 * current HSRR0 (for handled errors) in guest. Hence
 	 * queue up the event so that we can log it from host console later.
 	 */
-	machine_check_queue_event();
+	if (vcpu->kvm->arch.fwnmi_enabled) {
+		/*
+		 * Hook up the mce event on to vcpu structure.
+		 * First clear the old event.
+		 */
+		memset(&vcpu->arch.mce_evt, 0, sizeof(vcpu->arch.mce_evt));
+		if (get_mce_event(&mce_evt, MCE_EVENT_RELEASE)) {
+			vcpu->arch.mce_evt = mce_evt;
+		}
+	} else
+		machine_check_queue_event();
 
 	return handled;
 }
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 7c6477d..b6fe741 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -134,15 +134,13 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	stb	r0, HSTATE_HWTHREAD_REQ(r13)
 
 	/*
-	 * For external and machine check interrupts, we need
-	 * to call the Linux handler to process the interrupt.
-	 * We do that by jumping to absolute address 0x500 for
-	 * external interrupts, or the machine_check_fwnmi label
-	 * for machine checks (since firmware might have patched
-	 * the vector area at 0x200).  The [h]rfid at the end of the
-	 * handler will return to the book3s_hv_interrupts.S code.
-	 * For other interrupts we do the rfid to get back
-	 * to the book3s_hv_interrupts.S code here.
+	 * For external interrupts we need to call the Linux
+	 * handler to process the interrupt. We do that by jumping
+	 * to absolute address 0x500 for external interrupts.
+	 * The [h]rfid at the end of the handler will return to
+	 * the book3s_hv_interrupts.S code. For other interrupts
+	 * we do the rfid to get back to the book3s_hv_interrupts.S
+	 * code here.
 	 */
 	ld	r8, 112+PPC_LR_STKOFF(r1)
 	addi	r1, r1, 112
@@ -157,7 +155,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	andi.	r0, r0, MSR_IR		/* in real mode? */
 	bne	.Lvirt_return
 
-	cmpwi	cr1, r12, BOOK3S_INTERRUPT_MACHINE_CHECK
 	cmpwi	r12, BOOK3S_INTERRUPT_EXTERNAL
 	beq	11f
 	cmpwi	r12, BOOK3S_INTERRUPT_H_DOORBELL
@@ -172,7 +169,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	mtmsrd	r6, 1			/* Clear RI in MSR */
 	mtsrr0	r8
 	mtsrr1	r7
-	beq	cr1, 13f		/* machine check */
+	/*
+	 * BOOK3S_INTERRUPT_MACHINE_CHECK is handled at the
+	 * time of guest exit
+	 */
 	RFI
 
 	/* On POWER7, we have external interrupts set to use HSRR0/1 */
@@ -180,8 +180,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 	mtspr	SPRN_HSRR1, r7
 	ba	0x500
 
-13:	b	machine_check_fwnmi
-
 14:	mtspr	SPRN_HSRR0, r8
 	mtspr	SPRN_HSRR1, r7
 	b	hmi_exception_after_realmode
@@ -2494,22 +2492,32 @@ machine_check_realmode:
 	ld	r9, HSTATE_KVM_VCPU(r13)
 	li	r12, BOOK3S_INTERRUPT_MACHINE_CHECK
 	/*
-	 * Deliver unhandled/fatal (e.g. UE) MCE errors to guest through
-	 * machine check interrupt (set HSRR0 to 0x200). And for handled
-	 * errors (no-fatal), just go back to guest execution with current
-	 * HSRR0 instead of exiting guest. This new approach will inject
-	 * machine check to guest for fatal error causing guest to crash.
-	 *
-	 * The old code used to return to host for unhandled errors which
-	 * was causing guest to hang with soft lockups inside guest and
-	 * makes it difficult to recover guest instance.
+	 * For the guest that is FWNMI capable, deliver all the MCE errors
+	 * (handled/unhandled) by exiting the guest with KVM_EXIT_NMI exit
+	 * reason. This new approach injects machine check errors in guest
+	 * address space to guest with additional information in the form
+	 * of RTAS event, thus enabling guest kernel to suitably handle
+	 * such errors.
 	 *
+	 * For the guest that is not FWNMI capable (old QEMU) fallback
+	 * to old behaviour for backward compatibility:
+	 * Deliver unhandled/fatal (e.g. UE) MCE errors to guest either
+	 * through machine check interrupt (set HSRR0 to 0x200).
+	 * For handled errors (no-fatal), just go back to guest execution
+	 * with current HSRR0.
 	 * if we receive machine check with MSR(RI=0) then deliver it to
 	 * guest as machine check causing guest to crash.
 	 */
 	ld	r11, VCPU_MSR(r9)
 	rldicl.	r0, r11, 64-MSR_HV_LG, 63 /* check if it happened in HV mode */
 	bne	mc_cont			/* if so, exit to host */
+	/* Check if guest is capable of handling NMI exit */
+	ld	r10, VCPU_KVM(r9)
+	lbz	r10, KVM_FWNMI(r10)
+	cmpdi	r10, 1			/* FWNMI capable? */
+	beq	mc_cont			/* if so, exit with KVM_EXIT_NMI. */
+
+	/* if not, fall through for backward compatibility. */
 	andi.	r10, r11, MSR_RI	/* check for unrecoverable exception */
 	beq	1f			/* Deliver a machine check to guest */
 	ld	r10, VCPU_PC(r9)

Re: [PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 5299 bytes --]

On Thu, Apr 06, 2017 at 02:17:22AM +0530, Mahesh J Salgaonkar wrote:
> From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
> 
> This patch introduces a mce hook which is invoked at the time of guest
> exit to facilitate the host-side handling of machine check exception
> before the exception is passed on to the guest. This hook will be invoked
> from host virtual mode from KVM (before exiting the guest with
> KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.
> 
> Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>

Um.. this introduces the hook, and puts in an implementation of it,
but AFAICT, nothing calls it, either here or in the next patch.  That
seems a bit pointless.

> ---
>  arch/powerpc/include/asm/machdep.h     |    7 +++++++
>  arch/powerpc/include/asm/opal.h        |    4 ++++
>  arch/powerpc/platforms/powernv/opal.c  |   26 ++++++++++++++++++++++++++
>  arch/powerpc/platforms/powernv/setup.c |    3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
> index 5011b69..9d74e7a 100644
> --- a/arch/powerpc/include/asm/machdep.h
> +++ b/arch/powerpc/include/asm/machdep.h
> @@ -15,6 +15,7 @@
>  #include <linux/export.h>
>  
>  #include <asm/setup.h>
> +#include <asm/mce.h>
>  
>  /* We export this macro for external modules like Alsa to know if
>   * ppc_md.feature_call is implemented or not
> @@ -112,6 +113,12 @@ struct machdep_calls {
>  	/* Called during machine check exception to retrive fixup address. */
>  	bool		(*mce_check_early_recovery)(struct pt_regs *regs);
>  
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +	/* Called after KVM interrupt handler finishes handling MCE for guest */
> +	int		(*machine_check_exception_guest)
> +					(struct machine_check_event *evt);
> +#endif
> +
>  	/* Motherboard/chipset features. This is a kind of general purpose
>  	 * hook used to control some machine specific features (like reset
>  	 * lines, chip power control, etc...).
> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
> index 1ff03a6..9b1fcbf 100644
> --- a/arch/powerpc/include/asm/opal.h
> +++ b/arch/powerpc/include/asm/opal.h
> @@ -17,6 +17,7 @@
>  #ifndef __ASSEMBLY__
>  
>  #include <linux/notifier.h>
> +#include <asm/mce.h>
>  
>  /* We calculate number of sg entries based on PAGE_SIZE */
>  #define SG_ENTRIES_PER_NODE ((PAGE_SIZE - 16) / sizeof(struct opal_sg_entry))
> @@ -273,6 +274,9 @@ extern int opal_hmi_handler_init(void);
>  extern int opal_event_init(void);
>  
>  extern int opal_machine_check(struct pt_regs *regs);
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +extern int opal_machine_check_guest(struct machine_check_event *evt);
> +#endif
>  extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
>  extern int opal_hmi_exception_early(struct pt_regs *regs);
>  extern int opal_handle_hmi_exception(struct pt_regs *regs);
> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
> index e0f856b..5e633a4 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
>  	return 0;
>  }
>  
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +/*
> + * opal_machine_check_guest() is a hook which is invoked at the time
> + * of guest exit to facilitate the host-side handling of machine check
> + * exception before the exception is passed on to the guest. This hook
> + * is invoked from host virtual mode from KVM (before exiting the guest
> + * with KVM_EXIT_NMI reason) for machine check exception that occurs in
> + * the guest.
> + *
> + * Currently no action is performed in the host other than printing the
> + * event information. The machine check exception is passed on to the
> + * guest kernel and the guest kernel will attempt for recovery.
> + */
> +int opal_machine_check_guest(struct machine_check_event *evt)
> +{
> +	/* Print things out */
> +	if (evt->version != MCE_V1) {
> +		pr_err("Machine Check Exception, Unknown event version %d !\n",
> +		       evt->version);
> +		return 0;
> +	}
> +	machine_check_print_event_info(evt);
> +	return 0;
> +}
> +#endif
> +
>  /* Early hmi handler called in real mode. */
>  int opal_hmi_exception_early(struct pt_regs *regs)
>  {
> diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
> index d50c7d9..333ee09 100644
> --- a/arch/powerpc/platforms/powernv/setup.c
> +++ b/arch/powerpc/platforms/powernv/setup.c
> @@ -264,6 +264,9 @@ static void __init pnv_setup_machdep_opal(void)
>  	ppc_md.mce_check_early_recovery = opal_mce_check_early_recovery;
>  	ppc_md.hmi_exception_early = opal_hmi_exception_early;
>  	ppc_md.handle_hmi_exception = opal_handle_hmi_exception;
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +	ppc_md.machine_check_exception_guest = opal_machine_check_guest;
> +#endif
>  }
>  
>  static int __init pnv_probe(void)
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 5299 bytes --]

On Thu, Apr 06, 2017 at 02:17:22AM +0530, Mahesh J Salgaonkar wrote:
> From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
> 
> This patch introduces a mce hook which is invoked at the time of guest
> exit to facilitate the host-side handling of machine check exception
> before the exception is passed on to the guest. This hook will be invoked
> from host virtual mode from KVM (before exiting the guest with
> KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.
> 
> Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>

Um.. this introduces the hook, and puts in an implementation of it,
but AFAICT, nothing calls it, either here or in the next patch.  That
seems a bit pointless.

> ---
>  arch/powerpc/include/asm/machdep.h     |    7 +++++++
>  arch/powerpc/include/asm/opal.h        |    4 ++++
>  arch/powerpc/platforms/powernv/opal.c  |   26 ++++++++++++++++++++++++++
>  arch/powerpc/platforms/powernv/setup.c |    3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
> index 5011b69..9d74e7a 100644
> --- a/arch/powerpc/include/asm/machdep.h
> +++ b/arch/powerpc/include/asm/machdep.h
> @@ -15,6 +15,7 @@
>  #include <linux/export.h>
>  
>  #include <asm/setup.h>
> +#include <asm/mce.h>
>  
>  /* We export this macro for external modules like Alsa to know if
>   * ppc_md.feature_call is implemented or not
> @@ -112,6 +113,12 @@ struct machdep_calls {
>  	/* Called during machine check exception to retrive fixup address. */
>  	bool		(*mce_check_early_recovery)(struct pt_regs *regs);
>  
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +	/* Called after KVM interrupt handler finishes handling MCE for guest */
> +	int		(*machine_check_exception_guest)
> +					(struct machine_check_event *evt);
> +#endif
> +
>  	/* Motherboard/chipset features. This is a kind of general purpose
>  	 * hook used to control some machine specific features (like reset
>  	 * lines, chip power control, etc...).
> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
> index 1ff03a6..9b1fcbf 100644
> --- a/arch/powerpc/include/asm/opal.h
> +++ b/arch/powerpc/include/asm/opal.h
> @@ -17,6 +17,7 @@
>  #ifndef __ASSEMBLY__
>  
>  #include <linux/notifier.h>
> +#include <asm/mce.h>
>  
>  /* We calculate number of sg entries based on PAGE_SIZE */
>  #define SG_ENTRIES_PER_NODE ((PAGE_SIZE - 16) / sizeof(struct opal_sg_entry))
> @@ -273,6 +274,9 @@ extern int opal_hmi_handler_init(void);
>  extern int opal_event_init(void);
>  
>  extern int opal_machine_check(struct pt_regs *regs);
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +extern int opal_machine_check_guest(struct machine_check_event *evt);
> +#endif
>  extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
>  extern int opal_hmi_exception_early(struct pt_regs *regs);
>  extern int opal_handle_hmi_exception(struct pt_regs *regs);
> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
> index e0f856b..5e633a4 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
>  	return 0;
>  }
>  
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +/*
> + * opal_machine_check_guest() is a hook which is invoked at the time
> + * of guest exit to facilitate the host-side handling of machine check
> + * exception before the exception is passed on to the guest. This hook
> + * is invoked from host virtual mode from KVM (before exiting the guest
> + * with KVM_EXIT_NMI reason) for machine check exception that occurs in
> + * the guest.
> + *
> + * Currently no action is performed in the host other than printing the
> + * event information. The machine check exception is passed on to the
> + * guest kernel and the guest kernel will attempt for recovery.
> + */
> +int opal_machine_check_guest(struct machine_check_event *evt)
> +{
> +	/* Print things out */
> +	if (evt->version != MCE_V1) {
> +		pr_err("Machine Check Exception, Unknown event version %d !\n",
> +		       evt->version);
> +		return 0;
> +	}
> +	machine_check_print_event_info(evt);
> +	return 0;
> +}
> +#endif
> +
>  /* Early hmi handler called in real mode. */
>  int opal_hmi_exception_early(struct pt_regs *regs)
>  {
> diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
> index d50c7d9..333ee09 100644
> --- a/arch/powerpc/platforms/powernv/setup.c
> +++ b/arch/powerpc/platforms/powernv/setup.c
> @@ -264,6 +264,9 @@ static void __init pnv_setup_machdep_opal(void)
>  	ppc_md.mce_check_early_recovery = opal_mce_check_early_recovery;
>  	ppc_md.hmi_exception_early = opal_hmi_exception_early;
>  	ppc_md.handle_hmi_exception = opal_handle_hmi_exception;
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +	ppc_md.machine_check_exception_guest = opal_machine_check_guest;
> +#endif
>  }
>  
>  static int __init pnv_probe(void)
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[Michael Ellerman]

Mahesh J Salgaonkar <mahesh@linux.vnet.ibm.com> writes:

> From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
>
> This patch introduces a mce hook which is invoked at the time of guest
> exit to facilitate the host-side handling of machine check exception
> before the exception is passed on to the guest. This hook will be invoked
> from host virtual mode from KVM (before exiting the guest with
> KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.

Hi Mahesh,

This patch doesn't meaningfully abstract anything.

opal_machine_check_guest() doesn't call opal, or do anything opal
specific. It just passes the evt struct back to code in mce.c, which is
where the struct came from in the first place.

> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
>  	return 0;
>  }
>  
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +/*
> + * opal_machine_check_guest() is a hook which is invoked at the time
> + * of guest exit to facilitate the host-side handling of machine check
> + * exception before the exception is passed on to the guest. This hook
> + * is invoked from host virtual mode from KVM (before exiting the guest
> + * with KVM_EXIT_NMI reason) for machine check exception that occurs in
> + * the guest.
> + *
> + * Currently no action is performed in the host other than printing the
> + * event information. The machine check exception is passed on to the
> + * guest kernel and the guest kernel will attempt for recovery.
> + */
> +int opal_machine_check_guest(struct machine_check_event *evt)
> +{
> +	/* Print things out */
> +	if (evt->version != MCE_V1) {
> +		pr_err("Machine Check Exception, Unknown event version %d !\n",
> +		       evt->version);
> +		return 0;
> +	}
> +	machine_check_print_event_info(evt);
> +	return 0;
> +}
> +#endif

Called from kvmppc_handle_exit_hv():

+		if (ppc_md.machine_check_exception_guest)
+			ppc_md.machine_check_exception_guest(
+							&vcpu->arch.mce_evt);
 		break;

Just call machine_check_print_event_info() directly from
kvmppc_handle_exit_hv().

You'll need to export symbol it, but that's all AFAICS, I don't think it
even needs any #ifdefs.

If we ever want to do anything different we can change the code then.

cheers

Re: [PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[Michael Ellerman]

Mahesh J Salgaonkar <mahesh@linux.vnet.ibm.com> writes:

> From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
>
> This patch introduces a mce hook which is invoked at the time of guest
> exit to facilitate the host-side handling of machine check exception
> before the exception is passed on to the guest. This hook will be invoked
> from host virtual mode from KVM (before exiting the guest with
> KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.

Hi Mahesh,

This patch doesn't meaningfully abstract anything.

opal_machine_check_guest() doesn't call opal, or do anything opal
specific. It just passes the evt struct back to code in mce.c, which is
where the struct came from in the first place.

> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
>  	return 0;
>  }
>  
> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> +/*
> + * opal_machine_check_guest() is a hook which is invoked at the time
> + * of guest exit to facilitate the host-side handling of machine check
> + * exception before the exception is passed on to the guest. This hook
> + * is invoked from host virtual mode from KVM (before exiting the guest
> + * with KVM_EXIT_NMI reason) for machine check exception that occurs in
> + * the guest.
> + *
> + * Currently no action is performed in the host other than printing the
> + * event information. The machine check exception is passed on to the
> + * guest kernel and the guest kernel will attempt for recovery.
> + */
> +int opal_machine_check_guest(struct machine_check_event *evt)
> +{
> +	/* Print things out */
> +	if (evt->version != MCE_V1) {
> +		pr_err("Machine Check Exception, Unknown event version %d !\n",
> +		       evt->version);
> +		return 0;
> +	}
> +	machine_check_print_event_info(evt);
> +	return 0;
> +}
> +#endif

Called from kvmppc_handle_exit_hv():

+		if (ppc_md.machine_check_exception_guest)
+			ppc_md.machine_check_exception_guest(
+							&vcpu->arch.mce_evt);
 		break;

Just call machine_check_print_event_info() directly from
kvmppc_handle_exit_hv().

You'll need to export symbol it, but that's all AFAICS, I don't think it
even needs any #ifdefs.

If we ever want to do anything different we can change the code then.

cheers

Re: [PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[Mahesh Jagannath Salgaonkar]

On 04/06/2017 10:52 AM, David Gibson wrote:
> On Thu, Apr 06, 2017 at 02:17:22AM +0530, Mahesh J Salgaonkar wrote:
>> From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
>>
>> This patch introduces a mce hook which is invoked at the time of guest
>> exit to facilitate the host-side handling of machine check exception
>> before the exception is passed on to the guest. This hook will be invoked
>> from host virtual mode from KVM (before exiting the guest with
>> KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.
>>
>> Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
> 
> Um.. this introduces the hook, and puts in an implementation of it,
> but AFAICT, nothing calls it, either here or in the next patch.  That
> seems a bit pointless.

It gets called in the next patch [3/3] through
ppc_md.machine_check_exception_guest(). See the hunk for file
arch/powerpc/kvm/book3s_hv.c in next patch.

> 
>> ---
>>  arch/powerpc/include/asm/machdep.h     |    7 +++++++
>>  arch/powerpc/include/asm/opal.h        |    4 ++++
>>  arch/powerpc/platforms/powernv/opal.c  |   26 ++++++++++++++++++++++++++
>>  arch/powerpc/platforms/powernv/setup.c |    3 +++
>>  4 files changed, 40 insertions(+)
>>
>> diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
>> index 5011b69..9d74e7a 100644
>> --- a/arch/powerpc/include/asm/machdep.h
>> +++ b/arch/powerpc/include/asm/machdep.h
>> @@ -15,6 +15,7 @@
>>  #include <linux/export.h>
>>  
>>  #include <asm/setup.h>
>> +#include <asm/mce.h>
>>  
>>  /* We export this macro for external modules like Alsa to know if
>>   * ppc_md.feature_call is implemented or not
>> @@ -112,6 +113,12 @@ struct machdep_calls {
>>  	/* Called during machine check exception to retrive fixup address. */
>>  	bool		(*mce_check_early_recovery)(struct pt_regs *regs);
>>  
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +	/* Called after KVM interrupt handler finishes handling MCE for guest */
>> +	int		(*machine_check_exception_guest)
>> +					(struct machine_check_event *evt);
>> +#endif
>> +
>>  	/* Motherboard/chipset features. This is a kind of general purpose
>>  	 * hook used to control some machine specific features (like reset
>>  	 * lines, chip power control, etc...).
>> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
>> index 1ff03a6..9b1fcbf 100644
>> --- a/arch/powerpc/include/asm/opal.h
>> +++ b/arch/powerpc/include/asm/opal.h
>> @@ -17,6 +17,7 @@
>>  #ifndef __ASSEMBLY__
>>  
>>  #include <linux/notifier.h>
>> +#include <asm/mce.h>
>>  
>>  /* We calculate number of sg entries based on PAGE_SIZE */
>>  #define SG_ENTRIES_PER_NODE ((PAGE_SIZE - 16) / sizeof(struct opal_sg_entry))
>> @@ -273,6 +274,9 @@ extern int opal_hmi_handler_init(void);
>>  extern int opal_event_init(void);
>>  
>>  extern int opal_machine_check(struct pt_regs *regs);
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +extern int opal_machine_check_guest(struct machine_check_event *evt);
>> +#endif
>>  extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
>>  extern int opal_hmi_exception_early(struct pt_regs *regs);
>>  extern int opal_handle_hmi_exception(struct pt_regs *regs);
>> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
>> index e0f856b..5e633a4 100644
>> --- a/arch/powerpc/platforms/powernv/opal.c
>> +++ b/arch/powerpc/platforms/powernv/opal.c
>> @@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
>>  	return 0;
>>  }
>>  
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +/*
>> + * opal_machine_check_guest() is a hook which is invoked at the time
>> + * of guest exit to facilitate the host-side handling of machine check
>> + * exception before the exception is passed on to the guest. This hook
>> + * is invoked from host virtual mode from KVM (before exiting the guest
>> + * with KVM_EXIT_NMI reason) for machine check exception that occurs in
>> + * the guest.
>> + *
>> + * Currently no action is performed in the host other than printing the
>> + * event information. The machine check exception is passed on to the
>> + * guest kernel and the guest kernel will attempt for recovery.
>> + */
>> +int opal_machine_check_guest(struct machine_check_event *evt)
>> +{
>> +	/* Print things out */
>> +	if (evt->version != MCE_V1) {
>> +		pr_err("Machine Check Exception, Unknown event version %d !\n",
>> +		       evt->version);
>> +		return 0;
>> +	}
>> +	machine_check_print_event_info(evt);
>> +	return 0;
>> +}
>> +#endif
>> +
>>  /* Early hmi handler called in real mode. */
>>  int opal_hmi_exception_early(struct pt_regs *regs)
>>  {
>> diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
>> index d50c7d9..333ee09 100644
>> --- a/arch/powerpc/platforms/powernv/setup.c
>> +++ b/arch/powerpc/platforms/powernv/setup.c
>> @@ -264,6 +264,9 @@ static void __init pnv_setup_machdep_opal(void)
>>  	ppc_md.mce_check_early_recovery = opal_mce_check_early_recovery;
>>  	ppc_md.hmi_exception_early = opal_hmi_exception_early;
>>  	ppc_md.handle_hmi_exception = opal_handle_hmi_exception;
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +	ppc_md.machine_check_exception_guest = opal_machine_check_guest;
>> +#endif
>>  }
>>  
>>  static int __init pnv_probe(void)
>>
> 

Re: [PATCH v7 2/3] powerpc/powernv: Introduce a machine check hook for Guest MCEs.

[Mahesh Jagannath Salgaonkar]

On 04/06/2017 10:52 AM, David Gibson wrote:
> On Thu, Apr 06, 2017 at 02:17:22AM +0530, Mahesh J Salgaonkar wrote:
>> From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
>>
>> This patch introduces a mce hook which is invoked at the time of guest
>> exit to facilitate the host-side handling of machine check exception
>> before the exception is passed on to the guest. This hook will be invoked
>> from host virtual mode from KVM (before exiting the guest with
>> KVM_EXIT_NMI reason) for machine check exception that occurs in the guest.
>>
>> Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
> 
> Um.. this introduces the hook, and puts in an implementation of it,
> but AFAICT, nothing calls it, either here or in the next patch.  That
> seems a bit pointless.

It gets called in the next patch [3/3] through
ppc_md.machine_check_exception_guest(). See the hunk for file
arch/powerpc/kvm/book3s_hv.c in next patch.

> 
>> ---
>>  arch/powerpc/include/asm/machdep.h     |    7 +++++++
>>  arch/powerpc/include/asm/opal.h        |    4 ++++
>>  arch/powerpc/platforms/powernv/opal.c  |   26 ++++++++++++++++++++++++++
>>  arch/powerpc/platforms/powernv/setup.c |    3 +++
>>  4 files changed, 40 insertions(+)
>>
>> diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
>> index 5011b69..9d74e7a 100644
>> --- a/arch/powerpc/include/asm/machdep.h
>> +++ b/arch/powerpc/include/asm/machdep.h
>> @@ -15,6 +15,7 @@
>>  #include <linux/export.h>
>>  
>>  #include <asm/setup.h>
>> +#include <asm/mce.h>
>>  
>>  /* We export this macro for external modules like Alsa to know if
>>   * ppc_md.feature_call is implemented or not
>> @@ -112,6 +113,12 @@ struct machdep_calls {
>>  	/* Called during machine check exception to retrive fixup address. */
>>  	bool		(*mce_check_early_recovery)(struct pt_regs *regs);
>>  
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +	/* Called after KVM interrupt handler finishes handling MCE for guest */
>> +	int		(*machine_check_exception_guest)
>> +					(struct machine_check_event *evt);
>> +#endif
>> +
>>  	/* Motherboard/chipset features. This is a kind of general purpose
>>  	 * hook used to control some machine specific features (like reset
>>  	 * lines, chip power control, etc...).
>> diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
>> index 1ff03a6..9b1fcbf 100644
>> --- a/arch/powerpc/include/asm/opal.h
>> +++ b/arch/powerpc/include/asm/opal.h
>> @@ -17,6 +17,7 @@
>>  #ifndef __ASSEMBLY__
>>  
>>  #include <linux/notifier.h>
>> +#include <asm/mce.h>
>>  
>>  /* We calculate number of sg entries based on PAGE_SIZE */
>>  #define SG_ENTRIES_PER_NODE ((PAGE_SIZE - 16) / sizeof(struct opal_sg_entry))
>> @@ -273,6 +274,9 @@ extern int opal_hmi_handler_init(void);
>>  extern int opal_event_init(void);
>>  
>>  extern int opal_machine_check(struct pt_regs *regs);
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +extern int opal_machine_check_guest(struct machine_check_event *evt);
>> +#endif
>>  extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
>>  extern int opal_hmi_exception_early(struct pt_regs *regs);
>>  extern int opal_handle_hmi_exception(struct pt_regs *regs);
>> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
>> index e0f856b..5e633a4 100644
>> --- a/arch/powerpc/platforms/powernv/opal.c
>> +++ b/arch/powerpc/platforms/powernv/opal.c
>> @@ -479,6 +479,32 @@ int opal_machine_check(struct pt_regs *regs)
>>  	return 0;
>>  }
>>  
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +/*
>> + * opal_machine_check_guest() is a hook which is invoked at the time
>> + * of guest exit to facilitate the host-side handling of machine check
>> + * exception before the exception is passed on to the guest. This hook
>> + * is invoked from host virtual mode from KVM (before exiting the guest
>> + * with KVM_EXIT_NMI reason) for machine check exception that occurs in
>> + * the guest.
>> + *
>> + * Currently no action is performed in the host other than printing the
>> + * event information. The machine check exception is passed on to the
>> + * guest kernel and the guest kernel will attempt for recovery.
>> + */
>> +int opal_machine_check_guest(struct machine_check_event *evt)
>> +{
>> +	/* Print things out */
>> +	if (evt->version != MCE_V1) {
>> +		pr_err("Machine Check Exception, Unknown event version %d !\n",
>> +		       evt->version);
>> +		return 0;
>> +	}
>> +	machine_check_print_event_info(evt);
>> +	return 0;
>> +}
>> +#endif
>> +
>>  /* Early hmi handler called in real mode. */
>>  int opal_hmi_exception_early(struct pt_regs *regs)
>>  {
>> diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
>> index d50c7d9..333ee09 100644
>> --- a/arch/powerpc/platforms/powernv/setup.c
>> +++ b/arch/powerpc/platforms/powernv/setup.c
>> @@ -264,6 +264,9 @@ static void __init pnv_setup_machdep_opal(void)
>>  	ppc_md.mce_check_early_recovery = opal_mce_check_early_recovery;
>>  	ppc_md.hmi_exception_early = opal_hmi_exception_early;
>>  	ppc_md.handle_hmi_exception = opal_handle_hmi_exception;
>> +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
>> +	ppc_md.machine_check_exception_guest = opal_machine_check_guest;
>> +#endif
>>  }
>>  
>>  static int __init pnv_probe(void)
>>
>