[Qemu-devel] [PATCH v5 0/6] target-ppc/spapr: Add FWNMI support in QEMU for PowerKVM guests

[Qemu-devel] [PATCH v5 0/6] target-ppc/spapr: Add FWNMI support in QEMU for PowerKVM guests

[Aravinda Prasad]

This series of patches adds support for FWNMI in PowerKVM guests.

Memory error such as bit flips that cannot be corrected
by hardware is passed on to the kernel for handling
by raising machine check exception (an NMI). Upon such
machine check exception, if the address in error belongs
to guest then KVM causes a guest exit with KVM_EXIT_NMI
exit reason.

This patch series adds functionality to pass on such
machine check exception to the guest kernel by suitably
handling KVM_EXIT_NMI exit and building the error log.

The KVM changes are now part of the upstream kernel
(commit e20bbd3d). This series contain QEMU changes.

Change Log v5:
    - Handled VM migrations by including rtas_addr in VMSTATE.
    - Migration is blocked while a machine check is in progress.

Change Log v4:
    - Included the RTAS_DISP_FULLY_RECOVERED scenario in
      patch 4/5.

Change Log v3:
    - Rebased to v2.10.0-rc0 (git://github.com/dgibson/qemu)

Change Log v2:
    - Added KVM capability
    - Serialized multiple NMIs with a conditional wait

---

Aravinda Prasad (6):
      ppc: spapr: Register and handle HCALL to receive updated RTAS region
      ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls
      Wrapper function to wait on condition for the main loop mutex
      target/ppc: Handle NMI guest exit
      ppc: spapr: Enable FWNMI capability
      migration: Block migration while handling machine check


 cpus.c                   |    5 +++
 hw/ppc/spapr.c           |   23 ++++++++++++
 hw/ppc/spapr_events.c    |   62 +++++++++++++++++++++++++++++++
 hw/ppc/spapr_hcall.c     |    8 ++++
 hw/ppc/spapr_rtas.c      |   53 +++++++++++++++++++++++++++
 include/hw/ppc/spapr.h   |   23 +++++++++++-
 include/qemu/main-loop.h |    8 ++++
 target/ppc/kvm.c         |   92 ++++++++++++++++++++++++++++++++++++++++++++++
 target/ppc/kvm_ppc.h     |   20 ++++++++++
 9 files changed, 292 insertions(+), 2 deletions(-)

--
Aravinda Prasad

[Qemu-devel] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[Aravinda Prasad]

Receive updates from SLOF about the updated rtas-base.
A separate patch for SLOF [1] (commit f9a60de3) adds
functionality to invoke a private HCALL whenever OS
issues instantiate-rtas with a new rtas-base.

This is required as QEMU needs to know the updated rtas-base
as it allocates error reporting structure in RTAS space upon
a machine check exception.

[1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr.c         |   11 +++++++++++
 hw/ppc/spapr_hcall.c   |    8 ++++++++
 include/hw/ppc/spapr.h |    4 +++-
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index ff87f15..5deae30 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1675,6 +1675,16 @@ static const VMStateDescription vmstate_spapr_patb_entry = {
     },
 };
 
+static const VMStateDescription vmstate_spapr_rtas_addr = {
+    .name = "spapr_rtas_addr",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(rtas_addr, sPAPRMachineState),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
 static const VMStateDescription vmstate_spapr = {
     .name = "spapr",
     .version_id = 3,
@@ -1694,6 +1704,7 @@ static const VMStateDescription vmstate_spapr = {
         &vmstate_spapr_ov5_cas,
         &vmstate_spapr_patb_entry,
         &vmstate_spapr_pending_events,
+        &vmstate_spapr_rtas_addr,
         NULL
     }
 };
diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 8d72bb7..c15a93c 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -1088,6 +1088,13 @@ static target_ulong h_rtas(PowerPCCPU *cpu, sPAPRMachineState *spapr,
                            nret, rtas_r3 + 12 + 4*nargs);
 }
 
+static target_ulong h_rtas_update(PowerPCCPU *cpu, sPAPRMachineState *spapr,
+                                  target_ulong opcode, target_ulong *args)
+{
+    spapr->rtas_addr = args[0];
+    return 0;
+}
+
 static target_ulong h_logical_load(PowerPCCPU *cpu, sPAPRMachineState *spapr,
                                    target_ulong opcode, target_ulong *args)
 {
@@ -1750,6 +1757,7 @@ static void hypercall_register_types(void)
 
     /* qemu/KVM-PPC specific hcalls */
     spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
+    spapr_register_hypercall(KVMPPC_H_RTAS_UPDATE, h_rtas_update);
 
     /* ibm,client-architecture-support support */
     spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index c1b365f..b395aa7 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -90,6 +90,7 @@ struct sPAPRMachineState {
 
     hwaddr rma_size;
     int vrma_adjust;
+    hwaddr rtas_addr;
     ssize_t rtas_size;
     void *rtas_blob;
     long kernel_size;
@@ -400,7 +401,8 @@ struct sPAPRMachineState {
 #define KVMPPC_H_LOGICAL_MEMOP  (KVMPPC_HCALL_BASE + 0x1)
 /* Client Architecture support */
 #define KVMPPC_H_CAS            (KVMPPC_HCALL_BASE + 0x2)
-#define KVMPPC_HCALL_MAX        KVMPPC_H_CAS
+#define KVMPPC_H_RTAS_UPDATE    (KVMPPC_HCALL_BASE + 0x3)
+#define KVMPPC_HCALL_MAX        KVMPPC_H_RTAS_UPDATE
 
 typedef struct sPAPRDeviceTreeUpdateHeader {
     uint32_t version_id;

[Qemu-devel] [PATCH v5 2/6] ppc: spapr: Handle "ibm, nmi-register" and "ibm, nmi-interlock" RTAS calls

[Aravinda Prasad]

This patch adds support in QEMU to handle "ibm,nmi-register"
and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the
OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors
experience machine check at or about the same time by
handling "ibm,nmi-interlock" call. In such cases, as per
PAPR, subsequent processors serialize waiting for the first
processor to issue the "ibm,nmi-interlock" call. The second
processor waits till the first processor, which also
received a machine check error, is done reading the error
log. The first processor issues "ibm,nmi-interlock" call
when the error log is consumed. This patch implements the
releasing part of the error-log while subsequent patch
(which builds error log) handles the locking part.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
---
 hw/ppc/spapr.c         |    8 ++++++++
 hw/ppc/spapr_rtas.c    |   35 +++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |   11 ++++++++++-
 3 files changed, 53 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 5deae30..d568ea6 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1477,6 +1477,11 @@ static void ppc_spapr_reset(void)
     first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
 
     spapr->cas_reboot = false;
+
+    spapr->mc_status = -1;
+    spapr->guest_machine_check_addr = 0;
+    qemu_cond_destroy(&spapr->mc_delivery_cond);
+    qemu_cond_init(&spapr->mc_delivery_cond);
 }
 
 static void spapr_create_nvram(sPAPRMachineState *spapr)
@@ -2598,6 +2603,9 @@ static void ppc_spapr_init(MachineState *machine)
 
         kvmppc_spapr_enable_inkernel_multitce();
     }
+
+    spapr->mc_status = -1;
+    qemu_cond_init(&spapr->mc_delivery_cond);
 }
 
 static int spapr_kvm_type(const char *vm_type)
diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index cdf0b60..08e9a5e 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -348,6 +348,37 @@ static void rtas_get_power_level(PowerPCCPU *cpu, sPAPRMachineState *spapr,
     rtas_st(rets, 1, 100);
 }
 
+static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
+                                  sPAPRMachineState *spapr,
+                                  uint32_t token, uint32_t nargs,
+                                  target_ulong args,
+                                  uint32_t nret, target_ulong rets)
+{
+    spapr->guest_machine_check_addr = rtas_ld(args, 1);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
+static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
+                                   sPAPRMachineState *spapr,
+                                   uint32_t token, uint32_t nargs,
+                                   target_ulong args,
+                                   uint32_t nret, target_ulong rets)
+{
+    if (!spapr->guest_machine_check_addr) {
+        /* NMI register not called */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+    } else {
+        /*
+         * VCPU issuing "ibm,nmi-interlock" is done with NMI handling,
+         * hence unset mc_status.
+         */
+        spapr->mc_status = -1;
+        qemu_cond_signal(&spapr->mc_delivery_cond);
+        rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+    }
+}
+
+
 static struct rtas_call {
     const char *name;
     spapr_rtas_fn fn;
@@ -489,6 +520,10 @@ static void core_rtas_register_types(void)
                         rtas_set_power_level);
     spapr_rtas_register(RTAS_GET_POWER_LEVEL, "get-power-level",
                         rtas_get_power_level);
+    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
+                        rtas_ibm_nmi_register);
+    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
+                        rtas_ibm_nmi_interlock);
 }
 
 type_init(core_rtas_register_types)
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index b395aa7..28b6e2e 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -124,6 +124,13 @@ struct sPAPRMachineState {
      * occurs during the unplug process. */
     QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
 
+    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
+    target_ulong guest_machine_check_addr;
+    /* mc_status is set to -1 if mc is not in progress, else is set to the CPU
+     * handling the mc. */
+    int mc_status;
+    QemuCond mc_delivery_cond;
+
     /*< public >*/
     char *kvm_type;
     MemoryHotplugState hotplug_memory;
@@ -520,8 +527,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
 #define RTAS_IBM_CREATE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x27)
 #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
 #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2B)
 
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2C)
 
 /* RTAS ibm,get-system-parameter token values */
 #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20

[Qemu-devel] [PATCH v5 3/6] Wrapper function to wait on condition for the main loop mutex

[Aravinda Prasad]

Introduce a wrapper function to wait on condition for
the main loop mutex. This function atomically releases
the main loop mutex and causes the calling thread to
block on the condition.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
---
 cpus.c                   |    5 +++++
 include/qemu/main-loop.h |    8 ++++++++
 2 files changed, 13 insertions(+)

diff --git a/cpus.c b/cpus.c
index c9a6240..60be7f2 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1589,6 +1589,11 @@ void qemu_mutex_unlock_iothread(void)
     qemu_mutex_unlock(&qemu_global_mutex);
 }
 
+void qemu_cond_wait_iothread(QemuCond *cond)
+{
+    qemu_cond_wait(cond, &qemu_global_mutex);
+}
+
 static bool all_vcpus_paused(void)
 {
     CPUState *cpu;
diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index 6b4b60b..618cb2f 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -279,6 +279,14 @@ void qemu_mutex_lock_iothread(void);
  */
 void qemu_mutex_unlock_iothread(void);
 
+/**
+ * qemu_cond_wait_iothread: Wait on condition for the main loop mutex
+ *
+ * This function atomically releases the main loop mutex and causes
+ * the calling thread to block on the condition.
+ */
+void qemu_cond_wait_iothread(QemuCond *cond);
+
 /* internal interfaces */
 
 void qemu_fd_register(int fd);

[Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit

[Aravinda Prasad]

Memory error such as bit flips that cannot be corrected
by hardware are passed on to the kernel for handling.
If the memory address in error belongs to guest then
the guest kernel is responsible for taking suitable action.
Patch [1] enhances KVM to exit guest with exit reason
set to KVM_EXIT_NMI in such cases.

This patch handles KVM_EXIT_NMI exit. If the guest OS
has registered the machine check handling routine by
calling "ibm,nmi-register", then the handler builds
the error log and invokes the registered handler else
invokes the handler at 0x200.

Note that FWNMI handles synchronous machine check exceptions
triggered by the hardware and hence we do not extend
such support to the "nmi" command available in the QEMU
monitor. Hence, "nmi" command from the monitor will
always go through 0x200 vector.

[1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
	(e20bbd3d and related commits)

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
 hw/ppc/spapr.c         |    4 +++
 hw/ppc/spapr_events.c  |   62 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |    6 +++++
 target/ppc/kvm.c       |   62 ++++++++++++++++++++++++++++++++++++++++++++++++
 target/ppc/kvm_ppc.h   |   14 +++++++++++
 5 files changed, 148 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index d568ea6..7780434 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2453,6 +2453,10 @@ static void ppc_spapr_init(MachineState *machine)
         error_report("Could not get size of LPAR rtas '%s'", filename);
         exit(1);
     }
+
+    /* Resize blob to accommodate error log. */
+    spapr->rtas_size = spapr_get_rtas_size();
+
     spapr->rtas_blob = g_malloc(spapr->rtas_size);
     if (load_image_size(filename, spapr->rtas_blob, spapr->rtas_size) < 0) {
         error_report("Could not load LPAR rtas '%s'", filename);
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index e377fc7..ac93a7b 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -41,6 +41,7 @@
 #include "qemu/bcd.h"
 #include "hw/ppc/spapr_ovec.h"
 #include <libfdt.h>
+#include <linux/kvm.h>
 
 #define RTAS_LOG_VERSION_MASK                   0xff000000
 #define   RTAS_LOG_VERSION_6                    0x06000000
@@ -174,6 +175,22 @@ struct epow_extended_log {
     struct rtas_event_log_v6_epow epow;
 } QEMU_PACKED;
 
+/*
+ * Data format in RTAS Blob
+ *
+ * This structure contains error information related to Machine
+ * Check exception. This is filled up and copied to rtas blob
+ * upon machine check exception. The address of rtas blob is
+ * passed on to OS registered machine check notification
+ * routines upon machine check exception.
+ */
+struct rtas_event_log_mce {
+    target_ulong r3;
+    struct rtas_error_log rtas_error_log;
+    unsigned char   buffer[1];      /* Start of extended log */
+} QEMU_PACKED;
+
+
 union drc_identifier {
     uint32_t index;
     uint32_t count;
@@ -623,6 +640,51 @@ void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,
                             RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
 }
 
+ssize_t spapr_get_rtas_size(void)
+{
+    return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);
+}
+
+target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,
+                                 uint16_t flags, bool err_type, bool le)
+{
+    struct rtas_event_log_mce mc_log;
+    uint32_t summary;
+
+    /* Set error log fields */
+    mc_log.r3 = r3;
+
+    summary = RTAS_LOG_SEVERITY_ERROR_SYNC;
+
+    if (flags & KVM_RUN_PPC_NMI_DISP_FULLY_RECOV) {
+        summary |= RTAS_LOG_DISPOSITION_FULLY_RECOVERED;
+    } else {
+        summary |= RTAS_LOG_DISPOSITION_NOT_RECOVERED;
+    }
+
+    summary |= (RTAS_LOG_INITIATOR_MEMORY | RTAS_LOG_TARGET_MEMORY);
+
+    if (err_type) {
+        summary |= RTAS_LOG_TYPE_ECC_UNCORR;
+    } else {
+        summary |= RTAS_LOG_TYPE_ECC_CORR;
+    }
+
+    mc_log.rtas_error_log.summary = cpu_to_be32(summary);
+
+    /* Handle all Host/Guest LE/BE combinations */
+    if (le) {
+        mc_log.r3 = cpu_to_le64(mc_log.r3);
+    } else {
+        mc_log.r3 = cpu_to_be64(mc_log.r3);
+    }
+
+    cpu_physical_memory_write(rtas_addr + RTAS_ERRLOG_OFFSET,
+                              &mc_log, sizeof(mc_log));
+
+    return rtas_addr + RTAS_ERRLOG_OFFSET;
+}
+
 static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,
                             uint32_t token, uint32_t nargs,
                             target_ulong args,
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 28b6e2e..a75e9cf 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
 #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2
 #define DIAGNOSTICS_RUN_MODE_PERIODIC  3
 
+/* Offset from rtas-base where error log is placed */
+#define RTAS_ERRLOG_OFFSET       0x200
+
 static inline uint64_t ppc64_phys_to_real(uint64_t addr)
 {
     return addr & ~0xF000000000000000ULL;
@@ -675,6 +678,9 @@ int spapr_hpt_shift_for_ramsize(uint64_t ramsize);
 void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
                           Error **errp);
 void spapr_clear_pending_events(sPAPRMachineState *spapr);
+ssize_t spapr_get_rtas_size(void);
+target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,
+                                 uint16_t flags, bool err_type, bool le);
 
 /* CPU and LMB DRC release callbacks. */
 void spapr_core_release(DeviceState *dev);
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 171d3d8..7e4ce02 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -1798,6 +1798,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
         ret = 0;
         break;
 
+    case KVM_EXIT_NMI:
+        DPRINTF("handle NMI exception\n");
+        ret = kvm_handle_nmi(cpu, run);
+        break;
+
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
@@ -2746,6 +2751,63 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
     return data & 0xffff;
 }
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
+{
+    CPUPPCState *env = &cpu->env;
+    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
+    target_ulong msr = 0;
+    bool type, le;
+
+    cpu_synchronize_state(CPU(cpu));
+
+    /*
+     * Properly set bits in MSR before we invoke the handler.
+     * SRR0/1, DAR and DSISR are properly set by KVM
+     */
+    if (!(*pcc->interrupts_big_endian)(cpu)) {
+        msr |= (1ULL << MSR_LE);
+    }
+
+    if (env->msr && (1ULL << MSR_SF)) {
+        msr |= (1ULL << MSR_SF);
+    }
+
+    msr |= (1ULL << MSR_ME);
+    env->msr = msr;
+
+    if (!spapr->guest_machine_check_addr) {
+        /*
+         * If OS has not registered with "ibm,nmi-register"
+         * jump to 0x200
+         */
+        env->nip = 0x200;
+        return 0;
+    }
+
+    while (spapr->mc_status != -1) {
+        /*
+         * Check whether the same CPU got machine check error
+         * while still handling the mc error (i.e., before
+         * that CPU called "ibm,nmi-interlock"
+         */
+        if (spapr->mc_status == cpu->vcpu_id) {
+            qemu_system_guest_panicked(NULL);
+        }
+        qemu_cond_wait_iothread(&spapr->mc_delivery_cond);
+    }
+
+    spapr->mc_status = cpu->vcpu_id;
+
+    type = !!(env->spr[SPR_DSISR] & P7_DSISR_MC_UE);
+    le = !!(env->msr & (1ULL << MSR_LE));
+    env->gpr[3] = spapr_mce_req_event(env->gpr[3], spapr->rtas_addr,
+                                      run->flags, type, le);
+    env->nip = spapr->guest_machine_check_addr;
+
+    return 0;
+}
+
 int kvmppc_enable_hwrng(void)
 {
     if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {
diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
index d6be38e..0139dae 100644
--- a/target/ppc/kvm_ppc.h
+++ b/target/ppc/kvm_ppc.h
@@ -71,6 +71,20 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu);
 
 bool kvmppc_is_mem_backend_page_size_ok(const char *obj_path);
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);
+
+/*
+ * Currently KVM only passes on the uncorrected machine
+ * check memory error to guest. Other machine check errors
+ * such as SLB multi-hit and TLB multi-hit are recovered
+ * in KVM and are not passed on to guest.
+ *
+ * DSISR Bit for uncorrected machine check error. Based
+ * on arch/powerpc/include/asm/mce.h
+ */
+#define PPC_BIT(bit)                (0x8000000000000000ULL >> bit)
+#define P7_DSISR_MC_UE              (PPC_BIT(48))  /* P8 too */
+
 #else
 
 static inline uint32_t kvmppc_get_tbfreq(void)

[Qemu-devel] [PATCH v5 5/6] ppc: spapr: Enable FWNMI capability

[Aravinda Prasad]

Enable the KVM capability KVM_CAP_PPC_FWNMI so that
the KVM causes guest exit with NMI as exit reason
when it encounters a machine check exception on the
address belonging to a guest. Without this capability
enabled, KVM redirects machine check exceptions to
guest's 0x200 vector.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
---
 hw/ppc/spapr_rtas.c  |   15 +++++++++++++++
 target/ppc/kvm.c     |   13 +++++++++++++
 target/ppc/kvm_ppc.h |    6 ++++++
 3 files changed, 34 insertions(+)

diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index 08e9a5e..d017a67 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -46,6 +46,7 @@
 #include "qemu/cutils.h"
 #include "trace.h"
 #include "hw/ppc/fdt.h"
+#include "kvm_ppc.h"
 
 static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
                                    uint32_t token, uint32_t nargs,
@@ -354,6 +355,20 @@ static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
                                   target_ulong args,
                                   uint32_t nret, target_ulong rets)
 {
+    int ret;
+
+    ret = kvmppc_fwnmi_enable(cpu);
+
+    if (ret == 1) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    if (ret < 0) {
+        rtas_st(rets, 0, RTAS_OUT_HW_ERROR);
+        return;
+    }
+
     spapr->guest_machine_check_addr = rtas_ld(args, 1);
     rtas_st(rets, 0, RTAS_OUT_SUCCESS);
 }
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 7e4ce02..59b3322 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -92,6 +92,7 @@ static int cap_mmu_radix;
 static int cap_mmu_hash_v3;
 static int cap_resize_hpt;
 static int cap_ppc_pvr_compat;
+static int cap_fwnmi;
 
 static uint32_t debug_inst_opcode;
 
@@ -150,6 +151,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
     cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
     cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
+    cap_fwnmi = kvm_check_extension(s, KVM_CAP_PPC_FWNMI);
     /*
      * Note: setting it to false because there is not such capability
      * in KVM at this moment.
@@ -2142,6 +2144,17 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
     }
 }
 
+int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
+{
+    CPUState *cs = CPU(cpu);
+
+    if (!cap_fwnmi) {
+        return 1;
+    }
+
+    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);
+}
+
 int kvmppc_smt_threads(void)
 {
     return cap_ppc_smt ? cap_ppc_smt : 1;
diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
index 0139dae..55b6df2 100644
--- a/target/ppc/kvm_ppc.h
+++ b/target/ppc/kvm_ppc.h
@@ -28,6 +28,7 @@ void kvmppc_enable_clear_ref_mod_hcalls(void);
 void kvmppc_set_papr(PowerPCCPU *cpu);
 int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
 void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
+int kvmppc_fwnmi_enable(PowerPCCPU *cpu);
 int kvmppc_smt_threads(void);
 void kvmppc_hint_smt_possible(Error **errp);
 int kvmppc_set_smt_threads(int smt);
@@ -157,6 +158,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
 {
 }
 
+int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
+{
+    return 1;
+}
+
 static inline int kvmppc_smt_threads(void)
 {
     return 1;

[Qemu-devel] [PATCH v5 6/6] migration: Block migration while handling machine check

[Aravinda Prasad]

Block VM migration requests until the machine check
error handling is complete as (i) these errors are
specific to the source hardware and is irrelevant on
the target hardware, (ii) these errors cause data
corruption and should be handled before migration.

Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
---
 hw/ppc/spapr_rtas.c    |    3 +++
 include/hw/ppc/spapr.h |    2 ++
 target/ppc/kvm.c       |   17 +++++++++++++++++
 3 files changed, 22 insertions(+)

diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index d017a67..17f6567 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -47,6 +47,7 @@
 #include "trace.h"
 #include "hw/ppc/fdt.h"
 #include "kvm_ppc.h"
+#include "migration/blocker.h"
 
 static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
                                    uint32_t token, uint32_t nargs,
@@ -390,6 +391,8 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
         spapr->mc_status = -1;
         qemu_cond_signal(&spapr->mc_delivery_cond);
         rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+        migrate_del_blocker(spapr->migration_blocker);
+        error_free(spapr->migration_blocker);
     }
 }
 
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index a75e9cf..0890a44 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -7,6 +7,7 @@
 #include "hw/ppc/spapr_drc.h"
 #include "hw/mem/pc-dimm.h"
 #include "hw/ppc/spapr_ovec.h"
+#include "qapi/error.h"
 
 struct VIOsPAPRBus;
 struct sPAPRPHBState;
@@ -136,6 +137,7 @@ struct sPAPRMachineState {
     MemoryHotplugState hotplug_memory;
 
     const char *icp_type;
+    Error *migration_blocker;
 };
 
 #define H_SUCCESS         0
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 59b3322..58de7ea 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -52,6 +52,7 @@
 #endif
 #include "elf.h"
 #include "sysemu/kvm_int.h"
+#include "migration/blocker.h"
 
 //#define DEBUG_KVM
 
@@ -2770,10 +2771,26 @@ int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
     sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
     PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
     target_ulong msr = 0;
+    Error *local_err = NULL;
+    int ret;
     bool type, le;
 
     cpu_synchronize_state(CPU(cpu));
 
+    error_setg(&spapr->migration_blocker,
+            "Live migration not supported during machine check error handling");
+    ret = migrate_add_blocker(spapr->migration_blocker, &local_err);
+    if (ret < 0) {
+        /*
+         * We don't want to abort and let the migration to continue. In a
+         * rare case, the machine check handler will run on the target
+         * hardware. Though this is not preferable, it is better than aborting
+         * the migration or killing the VM.
+         */
+        error_free(spapr->migration_blocker);
+        fprintf(stderr, "Warning: Machine check during VM migration\n");
+    }
+
     /*
      * Properly set bits in MSR before we invoke the handler.
      * SRR0/1, DAR and DSISR are properly set by KVM

Re: [Qemu-devel] [PATCH v5 0/6] target-ppc/spapr: Add FWNMI support in QEMU for PowerKVM guests

[no-reply]

Hi,

This series failed build test on s390x host. Please find the details below.

Type: series
Subject: [Qemu-devel] [PATCH v5 0/6] target-ppc/spapr: Add FWNMI support in QEMU for PowerKVM guests
Message-id: 150659494872.25889.2069124544245723984.stgit@aravinda

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
set -e
echo "=== ENV ==="
env
echo "=== PACKAGES ==="
rpm -qa
echo "=== TEST BEGIN ==="
CC=$HOME/bin/cc
INSTALL=$PWD/install
BUILD=$PWD/build
echo -n "Using CC: "
realpath $CC
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL
make -j4
# XXX: we need reliable clean up
# make check -j4 V=1
make install
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/1506592361-26168-1-git-send-email-thuth@redhat.com -> patchew/1506592361-26168-1-git-send-email-thuth@redhat.com
 * [new tag]         patchew/150659494872.25889.2069124544245723984.stgit@aravinda -> patchew/150659494872.25889.2069124544245723984.stgit@aravinda
Switched to a new branch 'test'
8868f85 migration: Block migration while handling machine check
853ba58 ppc: spapr: Enable FWNMI capability
febbd34 target/ppc: Handle NMI guest exit
f6f4820 Wrapper function to wait on condition for the main loop mutex
2cb68e5 ppc: spapr: Handle "ibm, nmi-register" and "ibm, nmi-interlock" RTAS calls
79a66b4 ppc: spapr: Register and handle HCALL to receive updated RTAS region

=== OUTPUT BEGIN ===
=== ENV ===
XDG_SESSION_ID=22623
SHELL=/bin/sh
USER=fam
PATCHEW=/home/fam/patchew/patchew-cli -s http://patchew.org --nodebug
PATH=/usr/bin:/bin
PWD=/var/tmp/patchew-tester-tmp-bbu6uy4t/src
LANG=en_US.UTF-8
HOME=/home/fam
SHLVL=2
LOGNAME=fam
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1012/bus
XDG_RUNTIME_DIR=/run/user/1012
_=/usr/bin/env
=== PACKAGES ===
gpg-pubkey-873529b8-54e386ff
xz-libs-5.2.2-2.fc24.s390x
libxshmfence-1.2-3.fc24.s390x
giflib-4.1.6-15.fc24.s390x
trousers-lib-0.3.13-6.fc24.s390x
ncurses-base-6.0-6.20160709.fc25.noarch
gmp-6.1.1-1.fc25.s390x
libidn-1.33-1.fc25.s390x
slang-2.3.0-7.fc25.s390x
pkgconfig-0.29.1-1.fc25.s390x
alsa-lib-1.1.1-2.fc25.s390x
yum-metadata-parser-1.1.4-17.fc25.s390x
python3-slip-dbus-0.6.4-4.fc25.noarch
python2-cssselect-0.9.2-1.fc25.noarch
createrepo_c-libs-0.10.0-6.fc25.s390x
initscripts-9.69-1.fc25.s390x
parted-3.2-21.fc25.s390x
flex-2.6.0-3.fc25.s390x
colord-libs-1.3.4-1.fc25.s390x
python-osbs-client-0.33-3.fc25.noarch
perl-Pod-Simple-3.35-1.fc25.noarch
python2-simplejson-3.10.0-1.fc25.s390x
brltty-5.4-2.fc25.s390x
librados2-10.2.4-2.fc25.s390x
tcp_wrappers-7.6-83.fc25.s390x
libcephfs_jni1-10.2.4-2.fc25.s390x
nettle-devel-3.3-1.fc25.s390x
bzip2-devel-1.0.6-21.fc25.s390x
libuuid-2.28.2-2.fc25.s390x
python3-dnf-1.1.10-6.fc25.noarch
texlive-kpathsea-doc-svn41139-33.fc25.1.noarch
openssh-7.4p1-4.fc25.s390x
texlive-kpathsea-bin-svn40473-33.20160520.fc25.1.s390x
texlive-graphics-svn41015-33.fc25.1.noarch
texlive-dvipdfmx-def-svn40328-33.fc25.1.noarch
texlive-mfware-svn40768-33.fc25.1.noarch
texlive-texlive-scripts-svn41433-33.fc25.1.noarch
texlive-euro-svn22191.1.1-33.fc25.1.noarch
texlive-etex-svn37057.0-33.fc25.1.noarch
texlive-iftex-svn29654.0.2-33.fc25.1.noarch
texlive-palatino-svn31835.0-33.fc25.1.noarch
texlive-texlive-docindex-svn41430-33.fc25.1.noarch
texlive-xunicode-svn30466.0.981-33.fc25.1.noarch
texlive-koma-script-svn41508-33.fc25.1.noarch
texlive-pst-grad-svn15878.1.06-33.fc25.1.noarch
texlive-pst-blur-svn15878.2.0-33.fc25.1.noarch
texlive-jknapltx-svn19440.0-33.fc25.1.noarch
texinfo-6.1-4.fc25.s390x
openssl-devel-1.0.2k-1.fc25.s390x
jansson-2.10-2.fc25.s390x
fedora-repos-25-4.noarch
perl-Errno-1.25-387.fc25.s390x
acl-2.2.52-13.fc25.s390x
systemd-pam-231-17.fc25.s390x
NetworkManager-libnm-1.4.4-5.fc25.s390x
poppler-0.45.0-5.fc25.s390x
ccache-3.3.4-1.fc25.s390x
valgrind-3.12.0-9.fc25.s390x
perl-open-1.10-387.fc25.noarch
libgcc-6.4.1-1.fc25.s390x
libsoup-2.56.1-1.fc25.s390x
libstdc++-devel-6.4.1-1.fc25.s390x
libobjc-6.4.1-1.fc25.s390x
python2-rpm-4.13.0.1-2.fc25.s390x
python2-gluster-3.10.5-1.fc25.s390x
rpm-build-4.13.0.1-2.fc25.s390x
glibc-static-2.24-10.fc25.s390x
lz4-1.8.0-1.fc25.s390x
xapian-core-libs-1.2.24-1.fc25.s390x
elfutils-libelf-devel-0.169-1.fc25.s390x
nss-softokn-3.32.0-1.2.fc25.s390x
pango-1.40.9-1.fc25.s390x
glibc-debuginfo-common-2.24-10.fc25.s390x
libaio-0.3.110-6.fc24.s390x
libfontenc-1.1.3-3.fc24.s390x
lzo-2.08-8.fc24.s390x
isl-0.14-5.fc24.s390x
libXau-1.0.8-6.fc24.s390x
linux-atm-libs-2.5.1-14.fc24.s390x
libXext-1.3.3-4.fc24.s390x
libXxf86vm-1.1.4-3.fc24.s390x
bison-3.0.4-4.fc24.s390x
perl-srpm-macros-1-20.fc25.noarch
gawk-4.1.3-8.fc25.s390x
libwayland-client-1.12.0-1.fc25.s390x
perl-Exporter-5.72-366.fc25.noarch
perl-version-0.99.17-1.fc25.s390x
fftw-libs-double-3.3.5-3.fc25.s390x
libssh2-1.8.0-1.fc25.s390x
ModemManager-glib-1.6.4-1.fc25.s390x
newt-python3-0.52.19-2.fc25.s390x
python-munch-2.0.4-3.fc25.noarch
python-bugzilla-1.2.2-4.fc25.noarch
libedit-3.1-16.20160618cvs.fc25.s390x
createrepo_c-0.10.0-6.fc25.s390x
device-mapper-multipath-libs-0.4.9-83.fc25.s390x
yum-3.4.3-510.fc25.noarch
mozjs17-17.0.0-16.fc25.s390x
libselinux-2.5-13.fc25.s390x
python2-pyparsing-2.1.10-1.fc25.noarch
cairo-gobject-1.14.8-1.fc25.s390x
xorg-x11-proto-devel-7.7-20.fc25.noarch
brlapi-0.6.5-2.fc25.s390x
librados-devel-10.2.4-2.fc25.s390x
libXinerama-devel-1.1.3-6.fc24.s390x
lua-posix-33.3.1-3.fc25.s390x
usbredir-devel-0.7.1-2.fc24.s390x
python3-dnf-plugins-core-0.1.21-5.fc25.noarch
texlive-pdftex-doc-svn41149-33.fc25.1.noarch
openssh-clients-7.4p1-4.fc25.s390x
iptables-1.6.0-3.fc25.s390x
texlive-texlive.infra-svn41280-33.fc25.1.noarch
texlive-graphics-cfg-svn40269-33.fc25.1.noarch
texlive-bibtex-svn40768-33.fc25.1.noarch
texlive-mfware-bin-svn40473-33.20160520.fc25.1.s390x
texlive-texlive-scripts-bin-svn29741.0-33.20160520.fc25.1.noarch
texlive-sauerj-svn15878.0-33.fc25.1.noarch
texlive-enctex-svn34957.0-33.fc25.1.noarch
texlive-ifetex-svn24853.1.2-33.fc25.1.noarch
texlive-ntgclass-svn15878.2.1a-33.fc25.1.noarch
texlive-tex-gyre-math-svn41264-33.fc25.1.noarch
texlive-bera-svn20031.0-33.fc25.1.noarch
texlive-ms-svn29849.0-33.fc25.1.noarch
texlive-pst-fill-svn15878.1.01-33.fc25.1.noarch
texlive-ctable-svn38672-33.fc25.1.noarch
texlive-extsizes-svn17263.1.4a-33.fc25.1.noarch
texlive-collection-latexrecommended-svn35765.0-33.20160520.fc25.1.noarch
perl-Filter-1.57-1.fc25.s390x
python2-rpm-macros-3-12.fc25.noarch
gdbm-1.13-1.fc25.s390x
libtasn1-4.12-1.fc25.s390x
fedora-release-25-2.noarch
gdb-headless-7.12.1-48.fc25.s390x
perl-macros-5.24.2-387.fc25.s390x
pcre-devel-8.41-1.fc25.s390x
libX11-1.6.5-1.fc25.s390x
coreutils-8.25-17.fc25.s390x
python2-openidc-client-0-3.20170523git77cb3ee.fc25.noarch
systemtap-client-3.1-5.fc25.s390x
firewalld-0.4.4.5-1.fc25.noarch
glibc-2.24-10.fc25.s390x
libglvnd-egl-0.2.999-24.20170818git8d4d03f.fc25.s390x
libkadm5-1.14.4-8.fc25.s390x
glusterfs-fuse-3.10.5-1.fc25.s390x
libgfortran-6.4.1-1.fc25.s390x
python3-rpm-4.13.0.1-2.fc25.s390x
glusterfs-extra-xlators-3.10.5-1.fc25.s390x
dracut-config-rescue-046-2.git20170811.fc25.s390x
python2-sphinx-theme-alabaster-0.7.9-1.fc25.noarch
elfutils-devel-0.169-1.fc25.s390x
nss-3.32.0-1.1.fc25.s390x
pango-devel-1.40.9-1.fc25.s390x
glibc-debuginfo-2.24-10.fc25.s390x
gpg-pubkey-efe550f5-5220ba41
gpg-pubkey-81b46521-55b3ca9a
filesystem-3.2-37.fc24.s390x
libffi-3.1-9.fc24.s390x
keyutils-libs-1.5.9-8.fc24.s390x
libnfnetlink-1.0.1-8.fc24.s390x
libtheora-1.1.1-14.fc24.s390x
xml-common-0.6.3-44.fc24.noarch
autoconf-2.69-22.fc24.noarch
libXt-1.1.5-3.fc24.s390x
kbd-legacy-2.0.3-3.fc24.noarch
ghostscript-fonts-5.50-35.fc24.noarch
libXevie-1.0.3-11.fc24.s390x
libcap-2.25-2.fc25.s390x
mpfr-3.1.5-1.fc25.s390x
perl-Carp-1.40-365.fc25.noarch
libmnl-1.0.4-1.fc25.s390x
perl-Unicode-EastAsianWidth-1.33-8.fc25.noarch
libwayland-cursor-1.12.0-1.fc25.s390x
python-krbV-1.0.90-12.fc25.s390x
python2-urllib3-1.15.1-3.fc25.noarch
fipscheck-1.4.1-11.fc25.s390x
libndp-1.6-1.fc25.s390x
gnupg2-2.1.13-2.fc25.s390x
libXfixes-5.0.3-1.fc25.s390x
adwaita-icon-theme-3.22.0-1.fc25.noarch
dconf-0.26.0-1.fc25.s390x
ncurses-devel-6.0-6.20160709.fc25.s390x
dejagnu-1.6-1.fc25.noarch
device-mapper-1.02.136-3.fc25.s390x
libevent-2.0.22-1.fc25.s390x
atk-devel-2.22.0-1.fc25.s390x
libev-4.24-1.fc25.s390x
xorg-x11-fonts-Type1-7.5-16.fc24.noarch
brlapi-devel-0.6.5-2.fc25.s390x
pulseaudio-libs-10.0-2.fc25.s390x
glib2-2.50.3-1.fc25.s390x
dnf-1.1.10-6.fc25.noarch
texlive-metafont-bin-svn40987-33.20160520.fc25.1.s390x
texlive-xkeyval-svn35741.2.7a-33.fc25.1.noarch
texlive-euler-svn17261.2.5-33.fc25.1.noarch
texlive-mptopdf-svn41282-33.fc25.1.noarch
texlive-wasy-svn35831.0-33.fc25.1.noarch
texlive-avantgar-svn31835.0-33.fc25.1.noarch
texlive-eurosym-svn17265.1.4_subrfix-33.fc25.1.noarch
texlive-knuth-lib-svn35820.0-33.fc25.1.noarch
texlive-parallel-svn15878.0-33.fc25.1.noarch
texlive-texlive-msg-translations-svn41431-33.fc25.1.noarch
texlive-latex-svn40218-33.fc25.1.noarch
texlive-lualatex-math-svn40621-33.fc25.1.noarch
texlive-auto-pst-pdf-svn23723.0.6-33.fc25.1.noarch
texlive-powerdot-svn38984-33.fc25.1.noarch
texlive-wasysym-svn15878.2.0-33.fc25.1.noarch
ImageMagick-libs-6.9.3.0-6.fc25.s390x
geoclue2-2.4.5-1.fc25.s390x
perl-IO-Socket-IP-0.39-1.fc25.noarch
python2-pyasn1-0.2.3-1.fc25.noarch
at-spi2-core-devel-2.22.1-1.fc25.s390x
libacl-2.2.52-13.fc25.s390x
perl-libs-5.24.2-387.fc25.s390x
mesa-libglapi-17.0.5-3.fc25.s390x
python3-rpmconf-1.0.19-1.fc25.noarch
pcre-utf32-8.41-1.fc25.s390x
libX11-common-1.6.5-1.fc25.noarch
coreutils-common-8.25-17.fc25.s390x
mesa-libEGL-17.0.5-3.fc25.s390x
systemtap-runtime-3.1-5.fc25.s390x
NetworkManager-glib-1.4.4-5.fc25.s390x
audit-2.7.7-1.fc25.s390x
glibc-all-langpacks-2.24-10.fc25.s390x
libglvnd-glx-0.2.999-24.20170818git8d4d03f.fc25.s390x
glusterfs-api-3.10.5-1.fc25.s390x
libepoxy-devel-1.4.3-1.fc25.1.s390x
gtk3-3.22.17-2.fc25.s390x
rpm-4.13.0.1-2.fc25.s390x
net-snmp-agent-libs-5.7.3-15.fc25.s390x
pcre2-devel-10.23-9.fc25.s390x
subversion-1.9.7-1.fc25.s390x
libsndfile-1.0.28-6.fc25.s390x
lasi-1.1.2-6.fc24.s390x
python2-snowballstemmer-1.2.1-1.fc25.noarch
nss-util-devel-3.32.0-1.0.fc25.s390x
vim-common-8.0.1030-1.fc25.s390x
gd-2.2.5-1.fc25.s390x
gpg-pubkey-34ec9cba-54e38751
gpg-pubkey-030d5aed-55b577f0
basesystem-11-2.fc24.noarch
libmpc-1.0.2-5.fc24.s390x
libunistring-0.9.4-3.fc24.s390x
libmodman-2.0.1-12.fc24.s390x
lsscsi-0.28-3.fc24.s390x
kbd-misc-2.0.3-3.fc24.noarch
kmod-23-1.fc25.s390x
newt-0.52.19-2.fc25.s390x
perl-Text-Unidecode-1.27-3.fc25.noarch
plymouth-core-libs-0.9.3-0.6.20160620git0e65b86c.fc25.s390x
which-2.21-1.fc25.s390x
python3-slip-0.6.4-4.fc25.noarch
python3-systemd-232-1.fc25.s390x
python-lockfile-0.11.0-4.fc25.noarch
python2-requests-2.10.0-4.fc25.noarch
libnghttp2-1.13.0-2.fc25.s390x
python-urlgrabber-3.10.1-9.fc25.noarch
iputils-20161105-1.fc25.s390x
rest-0.8.0-1.fc25.s390x
adwaita-cursor-theme-3.22.0-1.fc25.noarch
authconfig-6.2.10-14.fc25.s390x
automake-1.15-7.fc25.noarch
shared-mime-info-1.8-1.fc25.s390x
pigz-2.3.4-1.fc25.s390x
device-mapper-libs-1.02.136-3.fc25.s390x
dnsmasq-2.76-2.fc25.s390x
fedora-packager-0.6.0.1-1.fc25.noarch
libwebp-0.5.2-1.fc25.s390x
boost-system-1.60.0-10.fc25.s390x
libasyncns-0.8-10.fc24.s390x
libXau-devel-1.0.8-6.fc24.s390x
libverto-libev-0.2.6-6.fc24.s390x
python3-html5lib-0.999-9.fc25.noarch
ttmkfdir-3.0.9-48.fc24.s390x
pulseaudio-libs-glib2-10.0-2.fc25.s390x
texlive-lib-2016-33.20160520.fc25.s390x
libXi-devel-1.7.9-1.fc25.s390x
python3-distro-1.0.3-1.fc25.noarch
texlive-texlive-common-doc-svn40682-33.fc25.1.noarch
packagedb-cli-2.14.1-1.fc25.noarch
texlive-metafont-svn40793-33.fc25.1.noarch
texlive-tools-svn40934-33.fc25.1.noarch
texlive-enumitem-svn24146.3.5.2-33.fc25.1.noarch
texlive-mptopdf-bin-svn18674.0-33.20160520.fc25.1.noarch
texlive-underscore-svn18261.0-33.fc25.1.noarch
texlive-anysize-svn15878.0-33.fc25.1.noarch
texlive-euenc-svn19795.0.1h-33.fc25.1.noarch
texlive-kastrup-svn15878.0-33.fc25.1.noarch
texlive-paralist-svn39247-33.fc25.1.noarch
texlive-texlive-en-svn41185-33.fc25.1.noarch
texlive-tipa-svn29349.1.3-33.fc25.1.noarch
texlive-currfile-svn40725-33.fc25.1.noarch
texlive-pst-node-svn40743-33.fc25.1.noarch
texlive-pst-slpe-svn24391.1.31-33.fc25.1.noarch
texlive-typehtml-svn17134.0-33.fc25.1.noarch
SDL2-devel-2.0.5-3.fc25.s390x
libcroco-0.6.11-3.fc25.s390x
bluez-libs-devel-5.44-1.fc25.s390x
firewalld-filesystem-0.4.4.5-1.fc25.noarch
pcre-cpp-8.41-1.fc25.s390x
python3-firewall-0.4.4.5-1.fc25.noarch
freetype-devel-2.6.5-9.fc25.s390x
pcre-utf16-8.41-1.fc25.s390x
linux-firmware-20170605-74.git37857004.fc25.noarch
kernel-modules-4.11.10-200.fc25.s390x
systemtap-devel-3.1-5.fc25.s390x
polkit-0.113-8.fc25.s390x
perl-SelfLoader-1.23-387.fc25.noarch
libdb-utils-5.3.28-24.fc25.s390x
glibc-common-2.24-10.fc25.s390x
libglvnd-0.2.999-24.20170818git8d4d03f.fc25.s390x
webkitgtk4-2.16.6-1.fc25.s390x
rpm-build-libs-4.13.0.1-2.fc25.s390x
libglvnd-core-devel-0.2.999-24.20170818git8d4d03f.fc25.s390x
rpm-devel-4.13.0.1-2.fc25.s390x
kernel-4.12.9-200.fc25.s390x
libtool-ltdl-2.4.6-14.fc25.s390x
gts-0.7.6-29.20121130.fc24.s390x
python2-imagesize-0.7.1-2.fc25.noarch
nss-softokn-freebl-3.32.0-1.2.fc25.s390x
selinux-policy-3.13.1-225.22.fc25.noarch
kernel-devel-4.12.11-200.fc25.s390x
fontpackages-filesystem-1.44-17.fc24.noarch
groff-base-1.22.3-8.fc24.s390x
ilmbase-2.2.0-5.fc24.s390x
OpenEXR-libs-2.2.0-5.fc24.s390x
hesiod-3.2.1-6.fc24.s390x
sysfsutils-2.1.0-19.fc24.s390x
ocaml-srpm-macros-2-4.fc24.noarch
mailx-12.5-19.fc24.s390x
ncurses-libs-6.0-6.20160709.fc25.s390x
ipset-libs-6.29-1.fc25.s390x
gmp-devel-6.1.1-1.fc25.s390x
python-pip-8.1.2-2.fc25.noarch
harfbuzz-1.3.2-1.fc25.s390x
python2-iniparse-0.4-20.fc25.noarch
python3-iniparse-0.4-20.fc25.noarch
python3-kickstart-2.32-1.fc25.noarch
perl-Net-SSLeay-1.78-1.fc25.s390x
drpm-0.3.0-3.fc25.s390x
glib-networking-2.50.0-1.fc25.s390x
webkitgtk3-2.4.11-3.fc25.s390x
libXaw-1.0.13-4.fc25.s390x
xorg-x11-font-utils-7.5-32.fc25.s390x
hardlink-1.1-1.fc25.s390x
libcom_err-1.43.3-1.fc25.s390x
python2-dateutil-2.6.0-1.fc25.noarch
libXpm-3.5.12-1.fc25.s390x
poppler-data-0.4.7-6.fc25.noarch
librbd1-10.2.4-2.fc25.s390x
perl-Digest-MD5-2.55-2.fc25.s390x
wayland-protocols-devel-1.7-1.fc25.noarch
texi2html-5.0-4.fc24.noarch
libxkbcommon-0.7.1-1.fc25.s390x
libuuid-devel-2.28.2-2.fc25.s390x
libcacard-2.5.3-1.fc25.s390x
libwmf-lite-0.2.8.4-50.fc25.s390x
texlive-tetex-svn41059-33.fc25.1.noarch
texlive-thumbpdf-svn34621.3.16-33.fc25.1.noarch
texlive-carlisle-svn18258.0-33.fc25.1.noarch
texlive-makeindex-bin-svn40473-33.20160520.fc25.1.s390x
texlive-pdftex-svn41149-33.fc25.1.noarch
texlive-csquotes-svn39538-33.fc25.1.noarch
texlive-courier-svn35058.0-33.fc25.1.noarch
texlive-helvetic-svn31835.0-33.fc25.1.noarch
texlive-mfnfss-svn19410.0-33.fc25.1.noarch
texlive-sepnum-svn20186.2.0-33.fc25.1.noarch
texlive-utopia-svn15878.0-33.fc25.1.noarch
texlive-luatexbase-svn38550-33.fc25.1.noarch
texlive-pst-3d-svn17257.1.10-33.fc25.1.noarch
texlive-latex-bin-bin-svn14050.0-33.20160520.fc25.1.noarch
texlive-l3experimental-svn41163-33.fc25.1.noarch
net-tools-2.0-0.40.20160329git.fc25.s390x
perl-Pod-Perldoc-3.28-1.fc25.noarch
openssl-1.0.2k-1.fc25.s390x
man-pages-4.06-4.fc25.noarch
libxml2-2.9.4-2.fc25.s390x
python3-dateutil-2.6.0-1.fc25.noarch
perl-threads-shared-1.57-1.fc25.s390x
libnotify-0.7.7-1.fc25.s390x
unzip-6.0-32.fc25.s390x
python-beautifulsoup4-4.6.0-1.fc25.noarch
dhcp-client-4.3.5-3.fc25.s390x
python2-fedora-0.9.0-6.fc25.noarch
gdb-7.12.1-48.fc25.s390x
sqlite-libs-3.14.2-3.fc25.s390x
webkitgtk4-jsc-2.16.6-1.fc25.s390x
libgomp-6.4.1-1.fc25.s390x
p11-kit-trust-0.23.8-1.fc25.s390x
gdk-pixbuf2-devel-2.36.9-1.fc25.s390x
rpm-plugin-selinux-4.13.0.1-2.fc25.s390x
mariadb-common-10.1.25-1.fc25.s390x
dbus-devel-1.11.16-1.fc25.s390x
lz4-libs-1.8.0-1.fc25.s390x
python2-jinja2-2.8.1-1.fc25.noarch
system-python-libs-3.5.4-1.fc25.s390x
python2-rpkg-1.50-2.fc25.noarch
libsolv-0.6.29-1.fc25.s390x
gpg-pubkey-95a43f54-5284415a
dejavu-fonts-common-2.35-3.fc24.noarch
libSM-1.2.2-4.fc24.s390x
diffutils-3.3-13.fc24.s390x
libogg-1.3.2-5.fc24.s390x
hunspell-en-US-0.20140811.1-5.fc24.noarch
libdaemon-0.14-10.fc24.s390x
patch-2.7.5-3.fc24.s390x
libsysfs-2.1.0-19.fc24.s390x
procmail-3.22-39.fc24.s390x
libXdamage-1.1.4-8.fc24.s390x
libotf-0.9.13-7.fc24.s390x
urw-fonts-2.4-22.fc24.noarch
crontabs-1.11-12.20150630git.fc24.noarch
ppp-2.4.7-9.fc24.s390x
cyrus-sasl-2.1.26-26.2.fc24.s390x
zlib-devel-1.2.8-10.fc24.s390x
time-1.7-49.fc24.s390x
gpg-pubkey-fdb19c98-56fd6333
libcap-ng-0.7.8-1.fc25.s390x
binutils-2.26.1-1.fc25.s390x
lcms2-2.8-2.fc25.s390x
libcomps-0.1.7-5.fc25.s390x
perl-constant-1.33-367.fc25.noarch
perl-Data-Dumper-2.161-1.fc25.s390x
ipcalc-0.1.8-1.fc25.s390x
gmp-c++-6.1.1-1.fc25.s390x
fontconfig-2.12.1-1.fc25.s390x
enchant-1.6.0-14.fc25.s390x
pyliblzma-0.5.3-16.fc25.s390x
libsepol-devel-2.5-10.fc25.s390x
python3-ordered-set-2.0.0-4.fc25.noarch
python-ipaddress-1.0.16-3.fc25.noarch
python2-kerberos-1.2.5-1.fc25.s390x
python2-pysocks-1.5.6-5.fc25.noarch
fipscheck-lib-1.4.1-11.fc25.s390x
libatomic_ops-7.4.4-1.fc25.s390x
python2-pygpgme-0.3-18.fc25.s390x
orc-0.4.26-1.fc25.s390x
yum-utils-1.1.31-511.fc25.noarch
libXrender-0.9.10-1.fc25.s390x
libXrandr-1.5.1-1.fc25.s390x
go-srpm-macros-2-7.fc25.noarch
gnupg2-smime-2.1.13-2.fc25.s390x
guile-devel-2.0.13-1.fc25.s390x
uboot-tools-2016.09.01-2.fc25.s390x
pykickstart-2.32-1.fc25.noarch
python-bunch-1.0.1-9.fc25.noarch
perl-generators-1.10-1.fc25.noarch
perl-Mozilla-CA-20160104-3.fc25.noarch
bzip2-libs-1.0.6-21.fc25.s390x
libpng-1.6.27-1.fc25.s390x
desktop-file-utils-0.23-2.fc25.s390x
python2-cccolutils-1.4-1.fc25.s390x
python2-lxml-3.7.2-1.fc25.s390x
redhat-rpm-config-45-1.fc25.noarch
device-mapper-event-libs-1.02.136-3.fc25.s390x
lvm2-libs-2.02.167-3.fc25.s390x
libselinux-python-2.5-13.fc25.s390x
boost-thread-1.60.0-10.fc25.s390x
librbd-devel-10.2.4-2.fc25.s390x
libXcursor-devel-1.1.14-6.fc24.s390x
latex2html-2012-7.fc24.noarch
lksctp-tools-1.0.16-5.fc24.s390x
libfdt-1.4.2-1.fc25.s390x
libXft-devel-2.3.2-4.fc24.s390x
libattr-devel-2.4.47-16.fc24.s390x
libiscsi-devel-1.15.0-2.fc24.s390x
gettext-0.19.8.1-3.fc25.s390x
libjpeg-turbo-devel-1.5.1-0.fc25.s390x
pulseaudio-libs-devel-10.0-2.fc25.s390x
libmount-2.28.2-2.fc25.s390x
python3-decorator-4.0.11-1.fc25.noarch
tzdata-java-2017b-1.fc25.noarch
python-srpm-macros-3-12.fc25.noarch
libsmartcols-2.28.2-2.fc25.s390x
texlive-kpathsea-svn41139-33.fc25.1.noarch
texlive-amsmath-svn41561-33.fc25.1.noarch
texlive-thumbpdf-bin-svn6898.0-33.20160520.fc25.1.noarch
texlive-psnfss-svn33946.9.2a-33.fc25.1.noarch
texlive-subfig-svn15878.1.3-33.fc25.1.noarch
texlive-fancybox-svn18304.1.4-33.fc25.1.noarch
texlive-lua-alt-getopt-svn29349.0.7.0-33.fc25.1.noarch
texlive-natbib-svn20668.8.31b-33.fc25.1.noarch
texlive-pdftex-bin-svn40987-33.20160520.fc25.1.s390x
texlive-xdvi-svn40768-33.fc25.1.noarch
texlive-crop-svn15878.1.5-33.fc25.1.noarch
texlive-babel-english-svn30264.3.3p-33.fc25.1.noarch
texlive-cmextra-svn32831.0-33.fc25.1.noarch
texlive-fancyhdr-svn15878.3.1-33.fc25.1.noarch
texlive-luatex-svn40963-33.fc25.1.noarch
texlive-knuth-local-svn38627-33.fc25.1.noarch
texlive-mflogo-font-svn36898.1.002-33.fc25.1.noarch
texlive-parskip-svn19963.2.0-33.fc25.1.noarch
texlive-section-svn20180.0-33.fc25.1.noarch
texlive-textcase-svn15878.0-33.fc25.1.noarch
texlive-updmap-map-svn41159-33.fc25.1.noarch
texlive-attachfile-svn38830-33.fc25.1.noarch
libtiff-4.0.8-1.fc25.s390x
libdb-5.3.28-24.fc25.s390x
bind-license-9.10.5-2.P2.fc25.noarch
mesa-libGLES-17.0.5-3.fc25.s390x
python3-requests-kerberos-0.10.0-2.fc25.noarch
python3-pyOpenSSL-16.2.0-1.fc25.noarch
perl-threads-2.16-1.fc25.s390x
cryptsetup-libs-1.7.5-1.fc25.s390x
netpbm-10.79.00-1.fc25.s390x
qrencode-libs-3.4.4-1.fc25.s390x
gstreamer1-plugins-base-1.10.5-1.fc25.s390x
elfutils-default-yama-scope-0.169-1.fc25.noarch
systemd-udev-231-17.fc25.s390x
python2-koji-1.13.0-2.fc25.noarch
unbound-libs-1.6.3-1.fc25.s390x
openldap-2.4.44-11.fc25.s390x
koji-1.13.0-2.fc25.noarch
bind99-libs-9.9.10-2.P3.fc25.s390x
mesa-libGL-devel-17.0.5-3.fc25.s390x
graphite2-devel-1.3.10-1.fc25.s390x
systemtap-sdt-devel-3.1-5.fc25.s390x
iproute-tc-4.11.0-1.fc25.s390x
libarchive-3.2.2-2.fc25.s390x
publicsuffix-list-dafsa-20170424-1.fc25.noarch
expat-2.2.3-1.fc25.s390x
p11-kit-0.23.8-1.fc25.s390x
kernel-core-4.12.9-200.fc25.s390x
emacs-filesystem-25.2-3.fc25.noarch
ca-certificates-2017.2.16-1.0.fc25.noarch
librsvg2-2.40.18-1.fc25.s390x
gtk-update-icon-cache-3.22.17-2.fc25.s390x
libidn2-2.0.4-1.fc25.s390x
rpm-libs-4.13.0.1-2.fc25.s390x
mariadb-libs-10.1.25-1.fc25.s390x
java-1.8.0-openjdk-headless-1.8.0.144-5.b01.fc25.s390x
gcc-objc-6.4.1-1.fc25.s390x
p11-kit-devel-0.23.8-1.fc25.s390x
ethtool-4.11-1.fc25.s390x
python2-sssdconfig-1.15.3-1.fc25.noarch
xorg-x11-fonts-ISO8859-1-100dpi-7.5-16.fc24.noarch
lato-fonts-2.015-2.fc24.noarch
python-sphinx-locale-1.5.2-2.fc25.noarch
dpkg-1.17.27-1.fc25.s390x
gnutls-3.5.15-1.fc25.s390x
nss-softokn-freebl-devel-3.32.0-1.2.fc25.s390x
vim-filesystem-8.0.1030-1.fc25.s390x
gnutls-devel-3.5.15-1.fc25.s390x
kernel-headers-4.12.11-200.fc25.s390x
texlive-luaotfload-svn40902-33.fc25.1.noarch
texlive-unicode-math-svn38462-33.fc25.1.noarch
texlive-fancyvrb-svn18492.2.8-33.fc25.1.noarch
texlive-pst-pdf-bin-svn7838.0-33.20160520.fc25.1.noarch
texlive-amscls-svn36804.0-33.fc25.1.noarch
texlive-ltxmisc-svn21927.0-33.fc25.1.noarch
texlive-breqn-svn38099.0.98d-33.fc25.1.noarch
texlive-xetex-def-svn40327-33.fc25.1.noarch
openssh-server-7.4p1-4.fc25.s390x
sendmail-8.15.2-8.fc25.s390x
tzdata-2017b-1.fc25.noarch
hunspell-1.4.1-2.fc25.s390x
gpg-pubkey-8e1431d5-53bcbac7
zlib-1.2.8-10.fc24.s390x
sed-4.2.2-15.fc24.s390x
psmisc-22.21-8.fc24.s390x
gpm-libs-1.20.7-9.fc24.s390x
zip-3.0-16.fc24.s390x
libyubikey-1.13-2.fc24.s390x
sg3_utils-libs-1.41-3.fc24.s390x
polkit-pkla-compat-0.1-7.fc24.s390x
passwd-0.79-8.fc24.s390x
trousers-0.3.13-6.fc24.s390x
grubby-8.40-3.fc24.s390x
rootfiles-8.1-19.fc24.noarch
nettle-3.3-1.fc25.s390x
libksba-1.3.5-1.fc25.s390x
perl-Text-ParseWords-3.30-365.fc25.noarch
perl-PathTools-3.63-366.fc25.s390x
perl-File-Temp-0.23.04-365.fc25.noarch
fuse-libs-2.9.7-1.fc25.s390x
perl-Pod-Escapes-1.07-365.fc25.noarch
perl-Term-ANSIColor-4.05-2.fc25.noarch
perl-URI-1.71-5.fc25.noarch
libXfont-1.5.2-1.fc25.s390x
python-six-1.10.0-3.fc25.noarch
dbus-glib-0.108-1.fc25.s390x
gobject-introspection-1.50.0-1.fc25.s390x
libpwquality-1.3.0-6.fc25.s390x
python-gobject-base-3.22.0-1.fc25.s390x
python-html5lib-0.999-9.fc25.noarch
python3-dbus-1.2.4-2.fc25.s390x
python3-chardet-2.3.0-1.fc25.noarch
python3-urllib3-1.15.1-3.fc25.noarch
python-offtrac-0.1.0-7.fc25.noarch
python2-cryptography-1.5.3-3.fc25.s390x
python2-requests-kerberos-0.10.0-2.fc25.noarch
libserf-1.3.9-1.fc25.s390x
libdatrie-0.2.9-3.fc25.s390x
s390utils-base-1.36.0-1.fc25.s390x
kpartx-0.4.9-83.fc25.s390x
s390utils-cpuplugd-1.36.0-1.fc25.s390x
s390utils-osasnmpd-1.36.0-1.fc25.s390x
python-dnf-plugins-extras-common-0.0.12-4.fc25.noarch
fpc-srpm-macros-1.0-1.fc25.noarch
libuser-0.62-4.fc25.s390x
man-db-2.7.5-3.fc25.s390x
python-systemd-doc-232-1.fc25.s390x
bodhi-client-0.9.12.2-6.fc25.noarch
cairo-1.14.8-1.fc25.s390x
cracklib-dicts-2.9.6-4.fc25.s390x
libselinux-python3-2.5-13.fc25.s390x
python2-enchant-1.6.8-1.fc25.noarch
boost-iostreams-1.60.0-10.fc25.s390x
userspace-rcu-0.9.2-2.fc25.s390x
libXext-devel-1.3.3-4.fc24.s390x
libXrandr-devel-1.5.1-1.fc25.s390x
python3-lxml-3.7.2-1.fc25.s390x
libiscsi-1.15.0-2.fc24.s390x
fontconfig-devel-2.12.1-1.fc25.s390x
libfdt-devel-1.4.2-1.fc25.s390x
ceph-devel-compat-10.2.4-2.fc25.s390x
zlib-static-1.2.8-10.fc24.s390x
chrpath-0.16-3.fc24.s390x
info-6.1-4.fc25.s390x
iptables-libs-1.6.0-3.fc25.s390x
libfdisk-2.28.2-2.fc25.s390x
dnf-plugins-core-0.1.21-5.fc25.noarch
perl-Storable-2.56-368.fc25.s390x
python2-decorator-4.0.11-1.fc25.noarch
libnetfilter_conntrack-1.0.6-2.fc25.s390x
texlive-texlive.infra-bin-svn40312-33.20160520.fc25.1.s390x
texlive-ifluatex-svn41346-33.fc25.1.noarch
texlive-fp-svn15878.0-33.fc25.1.noarch
texlive-latex-fonts-svn28888.0-33.fc25.1.noarch
texlive-bibtex-bin-svn40473-33.20160520.fc25.1.s390x
texlive-glyphlist-svn28576.0-33.fc25.1.noarch
texlive-marvosym-svn29349.2.2a-33.fc25.1.noarch
texlive-tex-bin-svn40987-33.20160520.fc25.1.s390x
texlive-texconfig-svn40768-33.fc25.1.noarch
texlive-wasy2-ps-svn35830.0-33.fc25.1.noarch
texlive-psfrag-svn15878.3.04-33.fc25.1.noarch
texlive-charter-svn15878.0-33.fc25.1.noarch
texlive-ec-svn25033.1.0-33.fc25.1.noarch
texlive-lineno-svn21442.4.41-33.fc25.1.noarch
texlive-hyphen-base-svn41138-33.fc25.1.noarch
texlive-manfnt-font-svn35799.0-33.fc25.1.noarch
texlive-ncntrsbk-svn31835.0-33.fc25.1.noarch
texlive-pst-math-svn34786.0.63-33.fc25.1.noarch
texlive-symbol-svn31835.0-33.fc25.1.noarch
texlive-environ-svn33821.0.3-33.fc25.1.noarch
texlive-algorithms-svn38085.0.1-33.fc25.1.noarch
python3-hawkey-0.6.4-3.fc25.s390x
freetype-2.6.5-9.fc25.s390x
mesa-libwayland-egl-17.0.5-3.fc25.s390x
libicu-57.1-5.fc25.s390x
libnl3-cli-3.2.29-3.fc25.s390x
cups-libs-2.2.0-9.fc25.s390x
bind-libs-lite-9.10.5-2.P2.fc25.s390x
python3-kerberos-1.2.5-1.fc25.s390x
python3-cryptography-1.5.3-3.fc25.s390x
perl-IO-1.36-387.fc25.s390x
dhcp-libs-4.3.5-3.fc25.s390x
rsync-3.1.2-4.fc25.s390x
make-4.1-6.fc25.s390x
quota-4.03-8.fc25.s390x
libX11-devel-1.6.5-1.fc25.s390x
ghostscript-9.20-9.fc25.s390x
rpcbind-0.2.4-6.rc2.fc25.s390x
pyOpenSSL-16.2.0-1.fc25.noarch
python3-pycurl-7.43.0-6.fc25.s390x
bind99-license-9.9.10-2.P3.fc25.noarch
python-firewall-0.4.4.5-1.fc25.noarch
netpbm-progs-10.79.00-1.fc25.s390x
wget-1.18-3.fc25.s390x
libsemanage-2.5-9.fc25.s390x
telnet-0.17-68.fc25.s390x
gdk-pixbuf2-2.36.9-1.fc25.s390x
dbus-libs-1.11.16-1.fc25.s390x
glusterfs-client-xlators-3.10.5-1.fc25.s390x
libepoxy-1.4.3-1.fc25.1.s390x
dracut-046-2.git20170811.fc25.s390x
net-snmp-libs-5.7.3-15.fc25.s390x
libgo-devel-6.4.1-1.fc25.s390x
libglvnd-opengl-0.2.999-24.20170818git8d4d03f.fc25.s390x
sqlite-devel-3.14.2-3.fc25.s390x
cpp-6.4.1-1.fc25.s390x
git-2.9.5-1.fc25.s390x
pcre2-10.23-9.fc25.s390x
python2-GitPython-2.1.5-1.fc25.noarch
glusterfs-devel-3.10.5-1.fc25.s390x
net-snmp-5.7.3-15.fc25.s390x
rpm-plugin-systemd-inhibit-4.13.0.1-2.fc25.s390x
emacs-25.2-3.fc25.s390x
libstdc++-static-6.4.1-1.fc25.s390x
expat-devel-2.2.3-1.fc25.s390x
perl-Time-HiRes-1.9744-1.fc25.s390x
fontawesome-fonts-4.7.0-1.fc25.noarch
python-markupsafe-0.23-10.fc25.s390x
pytz-2016.6.1-1.fc25.noarch
python2-sphinx-1.5.2-2.fc25.noarch
nss-util-3.32.0-1.0.fc25.s390x
nss-sysinit-3.32.0-1.1.fc25.s390x
python3-3.5.4-1.fc25.s390x
selinux-policy-targeted-3.13.1-225.22.fc25.noarch
vim-minimal-8.0.1030-1.fc25.s390x
texlive-ifplatform-svn21156.0.4-33.fc25.1.noarch
texlive-eso-pic-svn37925.2.0g-33.fc25.1.noarch
texlive-xcolor-svn41044-33.fc25.1.noarch
texlive-pst-eps-svn15878.1.0-33.fc25.1.noarch
texlive-pst-text-svn15878.1.00-33.fc25.1.noarch
texlive-rotating-svn16832.2.16b-33.fc25.1.noarch
texlive-pdfpages-svn40638-33.fc25.1.noarch
texlive-cm-super-svn15878.0-33.fc25.1.noarch
texlive-xetex-svn41438-33.fc25.1.noarch
dnf-yum-1.1.10-6.fc25.noarch
libseccomp-devel-2.3.2-1.fc25.s390x
gpgme-1.8.0-10.fc25.s390x
apr-util-1.5.4-3.fc24.s390x
jbigkit-libs-2.1-5.fc24.s390x
pixman-0.34.0-2.fc24.s390x
dwz-0.12-2.fc24.s390x
expect-5.45-22.fc24.s390x
libsigsegv-2.10-10.fc24.s390x
fakeroot-libs-1.20.2-4.fc24.s390x
m17n-lib-1.7.0-5.fc24.s390x
libverto-0.2.6-6.fc24.s390x
libXmu-1.1.2-4.fc24.s390x
libXcursor-1.1.14-6.fc24.s390x
python-kitchen-1.2.4-2.fc24.noarch
fakeroot-1.20.2-4.fc24.s390x
blktrace-1.1.0-3.fc24.s390x
usermode-1.111-8.fc24.s390x
kbd-2.0.3-3.fc24.s390x
libaio-devel-0.3.110-6.fc24.s390x
web-assets-filesystem-5-4.fc24.noarch
libgpg-error-1.24-1.fc25.s390x
findutils-4.6.0-8.fc25.s390x
libassuan-2.4.3-1.fc25.s390x
libusbx-1.0.21-1.fc25.s390x
libxslt-1.1.28-13.fc25.s390x
libmetalink-0.1.3-1.fc25.s390x
perl-MIME-Base64-3.15-365.fc25.s390x
ncurses-6.0-6.20160709.fc25.s390x
libwayland-server-1.12.0-1.fc25.s390x
perl-Fedora-VSP-0.001-4.fc25.noarch
perl-libintl-perl-1.26-1.fc25.s390x
shadow-utils-4.2.1-11.fc25.s390x
atk-2.22.0-1.fc25.s390x
pam-1.3.0-1.fc25.s390x
harfbuzz-icu-1.3.2-1.fc25.s390x
libsecret-0.18.5-2.fc25.s390x
s390utils-iucvterm-1.36.0-1.fc25.s390x
python3-requests-2.10.0-4.fc25.noarch
pyusb-1.0.0-2.fc25.noarch
python-enum34-1.0.4-6.fc25.noarch
pyxattr-0.5.3-8.fc25.s390x
libbabeltrace-1.4.0-3.fc25.s390x
libthai-0.1.25-1.fc25.s390x
deltarpm-3.6-17.fc25.s390x
s390utils-mon_statd-1.36.0-1.fc25.s390x
device-mapper-multipath-0.4.9-83.fc25.s390x
python3-pygpgme-0.3-18.fc25.s390x
libreport-filesystem-2.8.0-1.fc25.s390x
ghc-srpm-macros-1.4.2-4.fc25.noarch
rpmdevtools-8.9-1.fc25.noarch
python-dnf-plugins-extras-migrate-0.0.12-4.fc25.noarch
perl-IO-Socket-SSL-2.038-1.fc25.noarch
perl-File-ShareDir-1.102-7.fc25.noarch
tcl-8.6.6-1.fc25.s390x
bzip2-1.0.6-21.fc25.s390x
libss-1.43.3-1.fc25.s390x
libselinux-utils-2.5-13.fc25.s390x
python3-enchant-1.6.8-1.fc25.noarch
python2-dockerfile-parse-0.0.5-7.fc25.noarch
systemd-bootchart-231-2.fc25.s390x
e2fsprogs-1.43.3-1.fc25.s390x
libpng-devel-1.6.27-1.fc25.s390x
perl-XML-Parser-2.44-5.fc25.s390x
lttng-ust-2.8.1-2.fc25.s390x
libXfixes-devel-5.0.3-1.fc25.s390x
libXcomposite-devel-0.4.4-8.fc24.s390x
python3-javapackages-4.7.0-6.1.fc25.noarch
libcephfs_jni-devel-10.2.4-2.fc25.s390x
keyutils-libs-devel-1.5.9-8.fc24.s390x
harfbuzz-devel-1.3.2-1.fc25.s390x
libidn-devel-1.33-1.fc25.s390x
libnfs-1.9.8-2.fc24.s390x
libssh2-devel-1.8.0-1.fc25.s390x
qemu-sanity-check-nodeps-1.1.5-5.fc24.s390x
alsa-lib-devel-1.1.1-2.fc25.s390x
libpsl-0.17.0-1.fc25.s390x
libseccomp-2.3.2-1.fc25.s390x
json-glib-1.2.6-1.fc25.s390x
python2-dnf-1.1.10-6.fc25.noarch
texlive-tetex-bin-svn36770.0-33.20160520.fc25.1.noarch
texlive-amsfonts-svn29208.3.04-33.fc25.1.noarch
texlive-babel-svn40706-33.fc25.1.noarch
texlive-colortbl-svn29803.v1.0a-33.fc25.1.noarch
texlive-babelbib-svn25245.1.31-33.fc25.1.noarch
texlive-footmisc-svn23330.5.5b-33.fc25.1.noarch
texlive-makeindex-svn40768-33.fc25.1.noarch
texlive-plain-svn40274-33.fc25.1.noarch
texlive-texconfig-bin-svn29741.0-33.20160520.fc25.1.noarch
texlive-zapfding-svn31835.0-33.fc25.1.noarch
texlive-microtype-svn41127-33.fc25.1.noarch
texlive-bookman-svn31835.0-33.fc25.1.noarch
texlive-dvisvgm-def-svn41011-33.fc25.1.noarch
texlive-finstrut-svn21719.0.5-33.fc25.1.noarch
texlive-hyph-utf8-svn41189-33.fc25.1.noarch
texlive-lualibs-svn40370-33.fc25.1.noarch
python2-hawkey-0.6.4-3.fc25.s390x
elfutils-libelf-0.169-1.fc25.s390x
libnl3-3.2.29-3.fc25.s390x
gstreamer1-1.10.5-1.fc25.s390x
polkit-libs-0.113-8.fc25.s390x
libtirpc-1.0.2-0.fc25.s390x
libteam-1.27-1.fc25.s390x
python3-pyasn1-0.2.3-1.fc25.noarch
perl-File-Path-2.12-366.fc25.noarch
mesa-libwayland-egl-devel-17.0.5-3.fc25.s390x
libacl-devel-2.2.52-13.fc25.s390x
lua-libs-5.3.4-3.fc25.s390x
quota-nls-4.03-8.fc25.noarch
ghostscript-x11-9.20-9.fc25.s390x
systemd-231-17.fc25.s390x
dhcp-common-4.3.5-3.fc25.noarch
vte291-devel-0.46.2-1.fc25.s390x
python-devel-2.7.13-2.fc25.s390x
elfutils-0.169-1.fc25.s390x
lua-5.3.4-3.fc25.s390x
python3-beautifulsoup4-4.6.0-1.fc25.noarch
libmicrohttpd-0.9.55-1.fc25.s390x
screen-4.6.1-1.fc25.s390x
strace-4.18-1.fc25.s390x
libstdc++-6.4.1-1.fc25.s390x
glusterfs-3.10.5-1.fc25.s390x
file-5.29-9.fc25.s390x
libgo-6.4.1-1.fc25.s390x
tar-1.29-4.fc25.s390x
subversion-libs-1.9.7-1.fc25.s390x
libglvnd-gles-0.2.999-24.20170818git8d4d03f.fc25.s390x
gdk-pixbuf2-modules-2.36.9-1.fc25.s390x
gcc-6.4.1-1.fc25.s390x
curl-7.51.0-9.fc25.s390x
pcre2-utf16-10.23-9.fc25.s390x
mariadb-config-10.1.25-1.fc25.s390x
distribution-gpg-keys-1.14-1.fc25.noarch
libcurl-devel-7.51.0-9.fc25.s390x
gtk3-devel-3.22.17-2.fc25.s390x
krb5-devel-1.14.4-8.fc25.s390x
wpa_supplicant-2.6-3.fc25.s390x
fontawesome-fonts-web-4.7.0-1.fc25.noarch
python2-pygments-2.2.0-7.fc25.noarch
python2-babel-2.3.4-2.fc25.noarch
doxygen-1.8.13-9.fc25.s390x
nspr-devel-4.16.0-1.fc25.s390x
kernel-core-4.12.11-200.fc25.s390x
rpmlint-1.10-3.fc25.noarch
vim-enhanced-8.0.1030-1.fc25.s390x
openjpeg2-2.2.0-3.fc25.s390x
texlive-mparhack-svn15878.1.4-33.fc25.1.noarch
texlive-pspicture-svn15878.0-33.fc25.1.noarch
texlive-soul-svn15878.2.4-33.fc25.1.noarch
texlive-trimspaces-svn15878.1.1-33.fc25.1.noarch
texlive-varwidth-svn24104.0.92-33.fc25.1.noarch
texlive-geometry-svn19716.5.6-33.fc25.1.noarch
texlive-memoir-svn41203-33.fc25.1.noarch
texlive-pgf-svn40966-33.fc25.1.noarch
texlive-pst-coil-svn37377.1.07-33.fc25.1.noarch
texlive-pst-plot-svn41242-33.fc25.1.noarch
texlive-latex-bin-svn41438-33.fc25.1.noarch
texlive-ucs-svn35853.2.2-33.fc25.1.noarch
texlive-ae-svn15878.1.4-33.fc25.1.noarch
texlive-xetex-bin-svn41091-33.20160520.fc25.1.s390x
fedora-upgrade-26.1-1.fc25.noarch
perl-Thread-Queue-3.12-1.fc25.noarch
cdparanoia-libs-10.2-21.fc24.s390x
ustr-1.0.4-21.fc24.s390x
libusb-0.1.5-7.fc24.s390x
readline-devel-6.3-8.fc24.s390x
chkconfig-1.8-1.fc25.s390x
avahi-libs-0.6.32-4.fc25.s390x
perl-Unicode-Normalize-1.25-365.fc25.s390x
perl-libnet-3.10-1.fc25.noarch
perl-podlators-4.09-1.fc25.noarch
dbus-python-1.2.4-2.fc25.s390x
libgnome-keyring-3.12.0-7.fc25.s390x
python-backports-1.0-8.fc25.s390x
python-pycparser-2.14-7.fc25.noarch
plymouth-scripts-0.9.3-0.6.20160620git0e65b86c.fc25.s390x
cronie-1.5.1-2.fc25.s390x
python2-librepo-1.7.18-3.fc25.s390x
libXv-1.0.11-1.fc25.s390x
python2-ndg_httpsclient-0.4.0-4.fc25.noarch
btrfs-progs-4.6.1-1.fc25.s390x
perl-Encode-2.88-5.fc25.s390x
cracklib-2.9.6-4.fc25.s390x
python3-dnf-plugin-system-upgrade-0.7.1-4.fc25.noarch
boost-random-1.60.0-10.fc25.s390x
libref_array-0.1.5-29.fc25.s390x
libXrender-devel-0.9.10-1.fc25.s390x
javapackages-tools-4.7.0-6.1.fc25.noarch
keyutils-1.5.9-8.fc24.s390x
libcom_err-devel-1.43.3-1.fc25.s390x
lzo-minilzo-2.08-8.fc24.s390x
libusbx-devel-1.0.21-1.fc25.s390x
virglrenderer-devel-0.5.0-1.20160411git61846f92f.fc25.s390x
acpica-tools-20160831-1.fc25.s390x
grep-2.27-2.fc25.s390x
dnf-conf-1.1.10-6.fc25.noarch
crypto-policies-20160921-4.gitf3018dd.fc25.noarch
libnfsidmap-0.27-1.fc25.s390x
SDL2-2.0.5-3.fc25.s390x
texlive-etex-pkg-svn39355-33.fc25.1.noarch
texlive-multido-svn18302.1.42-33.fc25.1.noarch
texlive-gsftopk-svn40768-33.fc25.1.noarch
texlive-pst-ovl-svn40873-33.fc25.1.noarch
texlive-ltabptch-svn17533.1.74d-33.fc25.1.noarch
texlive-cite-svn36428.5.5-33.fc25.1.noarch
texlive-fpl-svn15878.1.002-33.fc25.1.noarch
texlive-mathpazo-svn15878.1.003-33.fc25.1.noarch
texlive-rcs-svn15878.0-33.fc25.1.noarch
texlive-type1cm-svn21820.0-33.fc25.1.noarch
texlive-l3kernel-svn41246-33.fc25.1.noarch
texlive-hyperref-svn41396-33.fc25.1.noarch
texlive-pst-tree-svn24142.1.12-33.fc25.1.noarch
texlive-sansmathaccent-svn30187.0-33.fc25.1.noarch
texlive-dvipdfmx-bin-svn40273-33.20160520.fc25.1.s390x
texlive-zapfchan-svn31835.0-33.fc25.1.noarch
glib2-static-2.50.3-1.fc25.s390x
bash-completion-2.5-1.fc25.noarch
hyphen-2.8.8-4.fc24.s390x
python3-idna-2.5-1.fc25.noarch
less-481-7.fc25.s390x
rpmconf-base-1.0.19-1.fc25.noarch
gtk2-2.24.31-2.fc25.s390x
mesa-libgbm-17.0.5-3.fc25.s390x
nfs-utils-2.1.1-5.rc4.fc25.s390x
mc-4.8.19-5.fc25.s390x
pcre-static-8.41-1.fc25.s390x
bind-libs-9.10.5-2.P2.fc25.s390x
libproxy-0.4.15-2.fc25.s390x
file-libs-5.29-9.fc25.s390x
glibc-devel-2.24-10.fc25.s390x
glusterfs-server-3.10.5-1.fc25.s390x
git-core-doc-2.9.5-1.fc25.s390x
python2-smmap-2.0.3-1.fc25.noarch
glusterfs-api-devel-3.10.5-1.fc25.s390x
gcc-gdb-plugin-6.4.1-1.fc25.s390x
python3-magic-5.29-9.fc25.noarch
GeoIP-GeoLite-data-2017.07-1.fc25.noarch
python2-funcsigs-1.0.2-2.fc25.noarch
dos2unix-7.3.4-1.fc25.s390x
gnutls-c++-3.5.15-1.fc25.s390x
nss-tools-3.32.0-1.1.fc25.s390x
gpg-pubkey-a29cb19c-53bcbba6
m4-1.4.17-9.fc24.s390x
liblockfile-1.09-4.fc24.s390x
sg3_utils-1.41-3.fc24.s390x
libXinerama-1.1.3-6.fc24.s390x
libXft-2.3.2-4.fc24.s390x
tcp_wrappers-libs-7.6-83.fc25.s390x
perl-Text-Tabs+Wrap-2013.0523-365.fc25.noarch
perl-Error-0.17024-7.fc25.noarch
perl-Term-Cap-1.17-365.fc25.noarch
perl-Pod-Usage-1.69-1.fc25.noarch
device-mapper-persistent-data-0.6.3-1.fc25.s390x
python3-six-1.10.0-3.fc25.noarch
python3-pysocks-1.5.6-5.fc25.noarch
python-chardet-2.3.0-1.fc25.noarch
python2-cffi-1.7.0-2.fc25.s390x
gc-devel-7.4.4-1.fc25.s390x
plymouth-0.9.3-0.6.20160620git0e65b86c.fc25.s390x
ebtables-2.0.10-21.fc25.s390x
python3-librepo-1.7.18-3.fc25.s390x
at-spi2-atk-2.22.0-1.fc25.s390x
avahi-autoipd-0.6.32-4.fc25.s390x
pyparsing-2.1.10-1.fc25.noarch
python3-pyparsing-2.1.10-1.fc25.noarch
libcollection-0.7.0-29.fc25.s390x
libcephfs-devel-10.2.4-2.fc25.s390x
libXdamage-devel-1.1.4-8.fc24.s390x
libverto-devel-0.2.6-6.fc24.s390x
snappy-1.1.3-2.fc24.s390x
cairo-gobject-devel-1.14.8-1.fc25.s390x
cyrus-sasl-devel-2.1.26-26.2.fc24.s390x
libXi-1.7.9-1.fc25.s390x
texlive-base-2016-33.20160520.fc25.noarch
texlive-booktabs-svn40846-33.fc25.1.noarch
texlive-lm-svn28119.2.004-33.fc25.1.noarch
texlive-gsftopk-bin-svn40473-33.20160520.fc25.1.s390x
texlive-tex-svn40793-33.fc25.1.noarch
texlive-fancyref-svn15878.0.9c-33.fc25.1.noarch
texlive-chngcntr-svn17157.1.0a-33.fc25.1.noarch
texlive-fix2col-svn38770-33.fc25.1.noarch
texlive-marginnote-svn41382-33.fc25.1.noarch
texlive-pxfonts-svn15878.0-33.fc25.1.noarch
texlive-txfonts-svn15878.0-33.fc25.1.noarch
texlive-l3packages-svn41246-33.fc25.1.noarch
texlive-oberdiek-svn41346-33.fc25.1.noarch
texlive-pst-tools-svn34067.0.05-33.fc25.1.noarch
texlive-tex-gyre-svn18651.2.004-33.fc25.1.noarch
texlive-dvipdfmx-svn41149-33.fc25.1.noarch
texlive-collection-fontsrecommended-svn35830.0-33.20160520.fc25.1.noarch
libcacard-devel-2.5.3-1.fc25.s390x
ykpers-1.18.0-2.fc25.s390x
python2-idna-2.5-1.fc25.noarch
policycoreutils-2.5-20.fc25.s390x
libgcrypt-1.7.8-1.fc25.s390x
pcre-8.41-1.fc25.s390x
GeoIP-1.6.11-1.fc25.s390x
ghostscript-core-9.20-9.fc25.s390x
python3-cffi-1.7.0-2.fc25.s390x
json-c-0.12.1-2.fc25.s390x
vte291-0.46.2-1.fc25.s390x
gssproxy-0.7.0-9.fc25.s390x
systemtap-3.1-5.fc25.s390x
mesa-libgbm-devel-17.0.5-3.fc25.s390x
libgusb-0.2.10-1.fc25.s390x
kernel-modules-4.12.9-200.fc25.s390x
sqlite-3.14.2-3.fc25.s390x
perl-Git-2.9.5-1.fc25.noarch
python2-gitdb-2.0.2-1.fc25.noarch
libglvnd-devel-0.2.999-24.20170818git8d4d03f.fc25.s390x
gcc-c++-6.4.1-1.fc25.s390x
python-magic-5.29-9.fc25.noarch
kernel-devel-4.12.9-200.fc25.s390x
python2-mock-2.0.0-2.fc25.noarch
nspr-4.16.0-1.fc25.s390x
python3-libs-3.5.4-1.fc25.s390x
system-python-3.5.4-1.fc25.s390x
python-async-0.6.1-9.fc22.s390x
dejavu-sans-mono-fonts-2.35-3.fc24.noarch
popt-1.16-7.fc24.s390x
cyrus-sasl-lib-2.1.26-26.2.fc24.s390x
xz-5.2.2-2.fc24.s390x
libpipeline-1.4.1-2.fc24.s390x
pinentry-0.9.7-2.fc24.s390x
pth-2.0.7-27.fc24.s390x
libsepol-2.5-10.fc25.s390x
libxcb-1.12-1.fc25.s390x
perl-Getopt-Long-2.49.1-1.fc25.noarch
avahi-glib-0.6.32-4.fc25.s390x
python3-pip-8.1.2-2.fc25.noarch
python3-libcomps-0.1.7-5.fc25.s390x
python-slip-0.6.4-4.fc25.noarch
python2-libcomps-0.1.7-5.fc25.s390x
gc-7.4.4-1.fc25.s390x
s390utils-cmsfs-1.36.0-1.fc25.s390x
newt-python-0.52.19-2.fc25.s390x
qt5-srpm-macros-5.7.1-1.fc25.noarch
device-mapper-event-1.02.136-3.fc25.s390x
perl-Class-Inspector-1.31-2.fc25.noarch
libbasicobjects-0.1.1-29.fc25.s390x
libradosstriper1-10.2.4-2.fc25.s390x
libXxf86vm-devel-1.1.4-3.fc24.s390x
zziplib-0.13.62-7.fc24.s390x
libpaper-1.1.24-12.fc24.s390x
libini_config-1.3.0-29.fc25.s390x
snappy-devel-1.1.3-2.fc24.s390x
libcap-ng-devel-0.7.8-1.fc25.s390x
libxkbcommon-devel-0.7.1-1.fc25.s390x
openssl-libs-1.0.2k-1.fc25.s390x
util-linux-2.28.2-2.fc25.s390x
texlive-etoolbox-svn38031.2.2a-33.fc25.1.noarch
texlive-dvips-svn41149-33.fc25.1.noarch
texlive-latexconfig-svn40274-33.fc25.1.noarch
texlive-tex-ini-files-svn40533-33.fc25.1.noarch
texlive-qstest-svn15878.0-33.fc25.1.noarch
texlive-cmap-svn41168-33.fc25.1.noarch
texlive-luatex-bin-svn41091-33.20160520.fc25.1.s390x
texlive-mflogo-svn38628-33.fc25.1.noarch
texlive-sansmath-svn17997.1.1-33.fc25.1.noarch
texlive-unicode-data-svn39808-33.fc25.1.noarch
texlive-luaotfload-bin-svn34647.0-33.20160520.fc25.1.noarch
texlive-listings-svn37534.1.6-33.fc25.1.noarch
texlive-pstricks-svn41321-33.fc25.1.noarch
texlive-metalogo-svn18611.0.12-33.fc25.1.noarch
texlive-collection-latex-svn41011-33.20160520.fc25.1.noarch
python2-dnf-plugins-core-0.1.21-5.fc25.noarch
xkeyboard-config-2.20-2.fc25.noarch
perl-Test-Harness-3.39-1.fc25.noarch
systemd-libs-231-17.fc25.s390x
python3-pycparser-2.14-7.fc25.noarch
kernel-devel-4.11.10-200.fc25.s390x
gsm-1.0.17-1.fc25.s390x
python-2.7.13-2.fc25.s390x
kernel-4.11.10-200.fc25.s390x
rpmconf-1.0.19-1.fc25.noarch
teamd-1.27-1.fc25.s390x
jasper-libs-1.900.13-4.fc25.s390x
glusterfs-libs-3.10.5-1.fc25.s390x
libcrypt-nss-2.24-10.fc25.s390x
emacs-common-25.2-3.fc25.s390x
libcurl-7.51.0-9.fc25.s390x
java-1.8.0-openjdk-1.8.0.144-5.b01.fc25.s390x
gcc-go-6.4.1-1.fc25.s390x
perl-XML-XPath-1.39-2.fc25.noarch
python2-sphinx_rtd_theme-0.1.9-2.fc24.noarch
libxml2-devel-2.9.4-2.fc25.s390x
nss-softokn-devel-3.32.0-1.2.fc25.s390x
nss-devel-3.32.0-1.1.fc25.s390x
libattr-2.4.47-16.fc24.s390x
libvisual-0.4.0-20.fc24.s390x
libpcap-1.7.4-2.fc24.s390x
libutempter-1.1.6-8.fc24.s390x
libgudev-230-3.fc24.s390x
popt-devel-1.16-7.fc24.s390x
hicolor-icon-theme-0.15-3.fc24.noarch
setup-2.10.4-1.fc25.noarch
bash-4.3.43-4.fc25.s390x
libjpeg-turbo-1.5.1-0.fc25.s390x
perl-Socket-2.024-1.fc25.s390x
perl-HTTP-Tiny-0.070-1.fc25.noarch
ipset-6.29-1.fc25.s390x
python2-setuptools-25.1.1-1.fc25.noarch
gsettings-desktop-schemas-3.22.0-1.fc25.s390x
python3-setuptools-25.1.1-1.fc25.noarch
python-slip-dbus-0.6.4-4.fc25.noarch
python2-ply-3.8-2.fc25.noarch
dtc-1.4.2-1.fc25.s390x
guile-2.0.13-1.fc25.s390x
cronie-anacron-1.5.1-2.fc25.s390x
libXtst-1.2.3-1.fc25.s390x
iso-codes-3.70-1.fc25.noarch
s390utils-1.36.0-1.fc25.s390x
python-backports-ssl_match_hostname-3.5.0.1-3.fc25.noarch
fedora-cert-0.6.0.1-1.fc25.noarch
dnf-plugin-system-upgrade-0.7.1-4.fc25.noarch
lvm2-2.02.167-3.fc25.s390x
libselinux-devel-2.5-13.fc25.s390x
perl-Time-Local-1.250-1.fc25.noarch
libradosstriper-devel-10.2.4-2.fc25.s390x
flac-libs-1.3.2-1.fc25.s390x
perl-Digest-1.17-366.fc25.noarch
teckit-2.5.1-15.fc24.s390x
libpath_utils-0.2.1-29.fc25.s390x
attr-2.4.47-16.fc24.s390x
usbredir-0.7.1-2.fc24.s390x
cairo-devel-1.14.8-1.fc25.s390x
lzo-devel-2.08-8.fc24.s390x
libcap-devel-2.25-2.fc25.s390x
libbsd-0.8.3-1.fc25.s390x
texlive-url-svn32528.3.4-33.fc25.1.noarch
texlive-dvips-bin-svn40987-33.20160520.fc25.1.s390x
texlive-index-svn24099.4.1beta-33.fc25.1.noarch
texlive-setspace-svn24881.6.7a-33.fc25.1.noarch
texlive-mathtools-svn38833-33.fc25.1.noarch
texlive-cm-svn32865.0-33.fc25.1.noarch
texlive-graphics-def-svn41879-33.fc25.1.noarch
texlive-mdwtools-svn15878.1.05.4-33.fc25.1.noarch
texlive-rsfs-svn15878.0-33.fc25.1.noarch
texlive-ucharcat-svn38907-33.fc25.1.noarch
texlive-fontspec-svn41262-33.fc25.1.noarch
texlive-showexpl-svn32737.v0.3l-33.fc25.1.noarch
texlive-pstricks-add-svn40744-33.fc25.1.noarch
texlive-beamer-svn36461.3.36-33.fc25.1.noarch
texlive-collection-basic-svn41149-33.20160520.fc25.1.noarch
xemacs-filesystem-21.5.34-20.20170124hgf412e9f093d4.fc25.noarch
hawkey-0.6.4-3.fc25.s390x
bluez-libs-5.44-1.fc25.s390x
audit-libs-2.7.7-1.fc25.s390x
iproute-4.11.0-1.fc25.s390x
libICE-1.0.9-9.fc25.s390x
python3-ply-3.8-2.fc25.noarch
perl-5.24.2-387.fc25.s390x
graphite2-1.3.10-1.fc25.s390x
vte-profile-0.46.2-1.fc25.s390x
python-libs-2.7.13-2.fc25.s390x
mesa-libGL-17.0.5-3.fc25.s390x
python2-pycurl-7.43.0-6.fc25.s390x
NetworkManager-1.4.4-5.fc25.s390x
mesa-libEGL-devel-17.0.5-3.fc25.s390x
mesa-libGLES-devel-17.0.5-3.fc25.s390x
hostname-3.15-8.fc25.s390x
glibc-headers-2.24-10.fc25.s390x
glusterfs-cli-3.10.5-1.fc25.s390x
git-core-2.9.5-1.fc25.s390x
mock-1.4.3-1.fc25.noarch
gcc-gfortran-6.4.1-1.fc25.s390x
webkitgtk4-plugin-process-gtk2-2.16.6-1.fc25.s390x
perl-Module-CoreList-5.20170821-1.fc25.noarch
python2-pbr-1.10.0-1.fc25.noarch
libtool-2.4.6-14.fc25.s390x
gnutls-dane-3.5.15-1.fc25.s390x
kernel-4.12.11-200.fc25.s390x
gpg-pubkey-a0a7badb-52844296
readline-6.3-8.fc24.s390x
cpio-2.12-3.fc24.s390x
libXcomposite-0.4.4-8.fc24.s390x
procps-ng-3.3.10-11.fc24.s390x
GConf2-3.2.6-16.fc24.s390x
xz-devel-5.2.2-2.fc24.s390x
fedora-logos-22.0.0-3.fc24.s390x
gpg-pubkey-e372e838-56fd7943
kmod-libs-23-1.fc25.s390x
perl-parent-0.236-1.fc25.noarch
perl-TermReadKey-2.37-1.fc25.s390x
ncurses-c++-libs-6.0-6.20160709.fc25.s390x
gzip-1.8-1.fc25.s390x
python3-gobject-base-3.22.0-1.fc25.s390x
python2-yubico-1.3.2-3.fc25.noarch
s390utils-ziomon-1.36.0-1.fc25.s390x
librepo-1.7.18-3.fc25.s390x
gnat-srpm-macros-4-1.fc25.noarch
python-decoratortools-1.8-12.fc25.noarch
m17n-db-1.7.0-7.fc25.noarch
e2fsprogs-libs-1.43.3-1.fc25.s390x
libvorbis-1.3.5-1.fc25.s390x
npth-1.3-1.fc25.s390x
libcephfs1-10.2.4-2.fc25.s390x
wayland-devel-1.12.0-1.fc25.s390x
libxcb-devel-1.12-1.fc25.s390x
perl-encoding-2.19-5.fc25.s390x
python3-cssselect-0.9.2-1.fc25.noarch
gettext-libs-0.19.8.1-3.fc25.s390x
at-spi2-atk-devel-2.22.0-1.fc25.s390x
virglrenderer-0.5.0-1.20160411git61846f92f.fc25.s390x
pixman-devel-0.34.0-2.fc24.s390x
libnfs-devel-1.9.8-2.fc24.s390x
libblkid-2.28.2-2.fc25.s390x
glib2-devel-2.50.3-1.fc25.s390x
texlive-ifxetex-svn19685.0.5-33.fc25.1.noarch
texlive-caption-svn41409-33.fc25.1.noarch
texlive-float-svn15878.1.3d-33.fc25.1.noarch
texlive-pdftex-def-svn22653.0.06d-33.fc25.1.noarch
texlive-xdvi-bin-svn40750-33.20160520.fc25.1.s390x
texlive-beton-svn15878.0-33.fc25.1.noarch
texlive-filecontents-svn24250.1.3-33.fc25.1.noarch
texlive-lm-math-svn36915.1.959-33.fc25.1.noarch
texlive-pslatex-svn16416.0-33.fc25.1.noarch
texlive-times-svn35058.0-33.fc25.1.noarch
texlive-breakurl-svn29901.1.40-33.fc25.1.noarch
texlive-filehook-svn24280.0.5d-33.fc25.1.noarch
texlive-pst-pdf-svn31660.1.1v-33.fc25.1.noarch
texlive-seminar-svn34011.1.62-33.fc25.1.noarch
texlive-xetexconfig-svn41133-33.fc25.1.noarch
python-rpm-macros-3-12.fc25.noarch
nss-pem-1.0.3-3.fc25.s390x
at-spi2-core-2.22.1-1.fc25.s390x
perl-Scalar-List-Utils-1.48-1.fc25.s390x
libtasn1-devel-4.12-1.fc25.s390x
python3-koji-1.13.0-2.fc25.noarch
opus-1.1.5-1.fc25.s390x
elfutils-libs-0.169-1.fc25.s390x
kernel-core-4.11.10-200.fc25.s390x
systemd-container-231-17.fc25.s390x
sudo-1.8.20p2-1.fc25.s390x
libicu-devel-57.1-5.fc25.s390x
js-jquery-2.2.4-3.fc25.noarch
krb5-libs-1.14.4-8.fc25.s390x
apr-1.6.2-1.fc25.s390x
dbus-1.11.16-1.fc25.s390x
libdrm-2.4.82-1.fc25.s390x
pcre2-utf32-10.23-9.fc25.s390x
copy-jdk-configs-2.3-1.fc25.noarch
libdrm-devel-2.4.82-1.fc25.s390x
krb5-workstation-1.14.4-8.fc25.s390x
python3-sssdconfig-1.15.3-1.fc25.noarch
python2-docutils-0.13.1-3.fc25.noarch
graphviz-2.38.0-39.fc25.s390x
kernel-modules-4.12.11-200.fc25.s390x
fedpkg-1.29-3.fc25.noarch
=== TEST BEGIN ===
Using CC: /home/fam/bin/cc
Install prefix    /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install
BIOS directory    /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/share/qemu
firmware path     /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/share/qemu-firmware
binary directory  /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/bin
library directory /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/lib
module directory  /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/lib/qemu
libexec directory /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/libexec
include directory /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/include
config directory  /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/etc
local state directory   /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/var
Manual directory  /var/tmp/patchew-tester-tmp-bbu6uy4t/src/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /var/tmp/patchew-tester-tmp-bbu6uy4t/src
C compiler        /home/fam/bin/cc
Host C compiler   cc
C++ compiler      c++
Objective-C compiler /home/fam/bin/cc
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS       -I/usr/include/pixman-1  -Werror -DHAS_LIBSSH2_SFTP_FSYNC -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -DNCURSES_WIDECHAR -D_GNU_SOURCE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/include/p11-kit-1    -I/usr/include/libpng16 -I/usr/include/libdrm 
LDFLAGS           -Wl,--warn-common -m64 -g 
make              make
install           install
python            python -B
smbd              /usr/sbin/smbd
module support    no
host CPU          s390x
host big endian   yes
target list       aarch64-softmmu alpha-softmmu arm-softmmu cris-softmmu i386-softmmu lm32-softmmu m68k-softmmu microblazeel-softmmu microblaze-softmmu mips64el-softmmu mips64-softmmu mipsel-softmmu mips-softmmu moxie-softmmu nios2-softmmu or1k-softmmu ppc64-softmmu ppcemb-softmmu ppc-softmmu s390x-softmmu sh4eb-softmmu sh4-softmmu sparc64-softmmu sparc-softmmu tricore-softmmu unicore32-softmmu x86_64-softmmu xtensaeb-softmmu xtensa-softmmu aarch64-linux-user alpha-linux-user armeb-linux-user arm-linux-user cris-linux-user hppa-linux-user i386-linux-user m68k-linux-user microblazeel-linux-user microblaze-linux-user mips64el-linux-user mips64-linux-user mipsel-linux-user mips-linux-user mipsn32el-linux-user mipsn32-linux-user nios2-linux-user or1k-linux-user ppc64abi32-linux-user ppc64le-linux-user ppc64-linux-user ppc-linux-user s390x-linux-user sh4eb-linux-user sh4-linux-user sparc32plus-linux-user sparc64-linux-user sparc-linux-user tilegx-linux-user x86_64-linux-user
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
SDL support       yes (2.0.5)
GTK support       yes (3.22.17)
GTK GL support    yes
VTE support       yes (0.46.2)
TLS priority      NORMAL
GNUTLS support    yes
GNUTLS rnd        yes
libgcrypt         no
libgcrypt kdf     no
nettle            yes (3.3)
nettle kdf        yes
libtasn1          yes
curses support    yes
virgl support     yes
curl support      yes
mingw32 support   no
Audio drivers     oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    yes
Multipath support no
VNC support       yes
VNC SASL support  yes
VNC JPEG support  yes
VNC PNG support   yes
xen support       no
brlapi support    yes
bluez  support    yes
Documentation     yes
PIE               no
vde support       no
netmap support    no
Linux AIO support yes
ATTR/XATTR support yes
Install blobs     yes
KVM support       yes
HAX support       no
TCG support       yes
TCG debug enabled no
TCG interpreter   no
RDMA support      no
fdt support       yes
preadv support    yes
fdatasync         yes
madvise           yes
posix_madvise     yes
libcap-ng support yes
vhost-net support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends    log
spice support     no 
rbd support       yes
xfsctl support    no
smartcard support yes
libusb            yes
usb net redir     yes
OpenGL support    yes
OpenGL dmabufs    yes
libiscsi support  yes
libnfs support    yes
build guest agent yes
QGA VSS support   no
QGA w32 disk info no
QGA MSI support   no
seccomp support   yes
coroutine backend ucontext
coroutine pool    yes
debug stack usage no
crypto afalg      no
GlusterFS support yes
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   yes
TPM passthrough   no
QOM debugging     yes
Live block migration yes
lzo support       yes
snappy support    yes
bzip2 support     yes
NUMA host support no
tcmalloc support  no
jemalloc support  no
avx2 optimization no
replication support yes
VxHS block device no
  GEN     aarch64-softmmu/config-devices.mak.tmp
  GEN     alpha-softmmu/config-devices.mak.tmp
  GEN     arm-softmmu/config-devices.mak.tmp
  GEN     cris-softmmu/config-devices.mak.tmp
  GEN     cris-softmmu/config-devices.mak
  GEN     arm-softmmu/config-devices.mak
  GEN     i386-softmmu/config-devices.mak.tmp
  GEN     alpha-softmmu/config-devices.mak
  GEN     lm32-softmmu/config-devices.mak.tmp
  GEN     aarch64-softmmu/config-devices.mak
  GEN     m68k-softmmu/config-devices.mak.tmp
  GEN     microblazeel-softmmu/config-devices.mak.tmp
  GEN     lm32-softmmu/config-devices.mak
  GEN     m68k-softmmu/config-devices.mak
  GEN     microblaze-softmmu/config-devices.mak.tmp
  GEN     i386-softmmu/config-devices.mak
  GEN     mips64el-softmmu/config-devices.mak.tmp
  GEN     mips64-softmmu/config-devices.mak.tmp
  GEN     microblazeel-softmmu/config-devices.mak
  GEN     microblaze-softmmu/config-devices.mak
  GEN     mipsel-softmmu/config-devices.mak.tmp
  GEN     mips-softmmu/config-devices.mak.tmp
  GEN     mips64el-softmmu/config-devices.mak
  GEN     moxie-softmmu/config-devices.mak.tmp
  GEN     mips64-softmmu/config-devices.mak
  GEN     nios2-softmmu/config-devices.mak.tmp
  GEN     moxie-softmmu/config-devices.mak
  GEN     mipsel-softmmu/config-devices.mak
  GEN     or1k-softmmu/config-devices.mak.tmp
  GEN     mips-softmmu/config-devices.mak
  GEN     ppc64-softmmu/config-devices.mak.tmp
  GEN     nios2-softmmu/config-devices.mak
  GEN     ppcemb-softmmu/config-devices.mak.tmp
  GEN     ppc-softmmu/config-devices.mak.tmp
  GEN     or1k-softmmu/config-devices.mak
  GEN     s390x-softmmu/config-devices.mak.tmp
  GEN     ppc64-softmmu/config-devices.mak
  GEN     s390x-softmmu/config-devices.mak
  GEN     ppcemb-softmmu/config-devices.mak
  GEN     sh4eb-softmmu/config-devices.mak.tmp
  GEN     sh4-softmmu/config-devices.mak.tmp
  GEN     ppc-softmmu/config-devices.mak
  GEN     sparc64-softmmu/config-devices.mak.tmp
  GEN     sparc-softmmu/config-devices.mak.tmp
  GEN     sparc-softmmu/config-devices.mak
  GEN     sh4eb-softmmu/config-devices.mak
  GEN     sh4-softmmu/config-devices.mak
  GEN     tricore-softmmu/config-devices.mak.tmp
  GEN     sparc64-softmmu/config-devices.mak
  GEN     unicore32-softmmu/config-devices.mak.tmp
  GEN     x86_64-softmmu/config-devices.mak.tmp
  GEN     xtensaeb-softmmu/config-devices.mak.tmp
  GEN     tricore-softmmu/config-devices.mak
  GEN     unicore32-softmmu/config-devices.mak
  GEN     xtensa-softmmu/config-devices.mak.tmp
  GEN     aarch64-linux-user/config-devices.mak.tmp
  GEN     xtensaeb-softmmu/config-devices.mak
  GEN     x86_64-softmmu/config-devices.mak
  GEN     alpha-linux-user/config-devices.mak.tmp
  GEN     xtensa-softmmu/config-devices.mak
  GEN     armeb-linux-user/config-devices.mak.tmp
  GEN     aarch64-linux-user/config-devices.mak
  GEN     cris-linux-user/config-devices.mak.tmp
  GEN     arm-linux-user/config-devices.mak.tmp
  GEN     armeb-linux-user/config-devices.mak
  GEN     alpha-linux-user/config-devices.mak
  GEN     arm-linux-user/config-devices.mak
  GEN     cris-linux-user/config-devices.mak
  GEN     hppa-linux-user/config-devices.mak.tmp
  GEN     i386-linux-user/config-devices.mak.tmp
  GEN     m68k-linux-user/config-devices.mak.tmp
  GEN     microblazeel-linux-user/config-devices.mak.tmp
  GEN     hppa-linux-user/config-devices.mak
  GEN     microblaze-linux-user/config-devices.mak.tmp
  GEN     m68k-linux-user/config-devices.mak
  GEN     i386-linux-user/config-devices.mak
  GEN     microblazeel-linux-user/config-devices.mak
  GEN     mips64el-linux-user/config-devices.mak.tmp
  GEN     mips64-linux-user/config-devices.mak.tmp
  GEN     mipsel-linux-user/config-devices.mak.tmp
  GEN     microblaze-linux-user/config-devices.mak
  GEN     mips64-linux-user/config-devices.mak
  GEN     mips-linux-user/config-devices.mak.tmp
  GEN     mips64el-linux-user/config-devices.mak
  GEN     mipsel-linux-user/config-devices.mak
  GEN     mipsn32el-linux-user/config-devices.mak.tmp
  GEN     mipsn32-linux-user/config-devices.mak.tmp
  GEN     nios2-linux-user/config-devices.mak.tmp
  GEN     mips-linux-user/config-devices.mak
  GEN     or1k-linux-user/config-devices.mak.tmp
  GEN     mipsn32-linux-user/config-devices.mak
  GEN     mipsn32el-linux-user/config-devices.mak
  GEN     nios2-linux-user/config-devices.mak
  GEN     ppc64abi32-linux-user/config-devices.mak.tmp
  GEN     ppc64le-linux-user/config-devices.mak.tmp
  GEN     ppc64-linux-user/config-devices.mak.tmp
  GEN     or1k-linux-user/config-devices.mak
  GEN     ppc-linux-user/config-devices.mak.tmp
  GEN     ppc64abi32-linux-user/config-devices.mak
  GEN     ppc64-linux-user/config-devices.mak
  GEN     ppc64le-linux-user/config-devices.mak
  GEN     sh4eb-linux-user/config-devices.mak.tmp
  GEN     s390x-linux-user/config-devices.mak.tmp
  GEN     sh4-linux-user/config-devices.mak.tmp
  GEN     ppc-linux-user/config-devices.mak
  GEN     sparc32plus-linux-user/config-devices.mak.tmp
  GEN     sh4eb-linux-user/config-devices.mak
  GEN     s390x-linux-user/config-devices.mak
  GEN     sh4-linux-user/config-devices.mak
  GEN     sparc64-linux-user/config-devices.mak.tmp
  GEN     sparc-linux-user/config-devices.mak.tmp
  GEN     tilegx-linux-user/config-devices.mak.tmp
  GEN     sparc32plus-linux-user/config-devices.mak
  GEN     x86_64-linux-user/config-devices.mak.tmp
  GEN     sparc64-linux-user/config-devices.mak
  GEN     sparc-linux-user/config-devices.mak
  GEN     tilegx-linux-user/config-devices.mak
  GEN     config-host.h
  GEN     qemu-options.def
  GEN     x86_64-linux-user/config-devices.mak
  GEN     qmp-commands.h
  GEN     qapi-types.h
  GEN     qapi-visit.h
  GEN     qapi-event.h
  GEN     qmp-marshal.c
  GEN     qapi-types.c
  GEN     qapi-visit.c
  GEN     qapi-event.c
  GEN     qmp-introspect.h
  GEN     qmp-introspect.c
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-helpers.h
  GEN     trace/generated-helpers.c
  GEN     module_block.h
  GEN     tests/test-qapi-types.h
  GEN     tests/test-qapi-visit.h
  GEN     tests/test-qmp-commands.h
  GEN     tests/test-qapi-event.h
  GEN     tests/test-qmp-introspect.h
  GEN     trace-root.h
  GEN     crypto/trace.h
  GEN     util/trace.h
  GEN     io/trace.h
  GEN     migration/trace.h
  GEN     block/trace.h
  GEN     chardev/trace.h
  GEN     hw/block/trace.h
  GEN     hw/block/dataplane/trace.h
  GEN     hw/char/trace.h
  GEN     hw/intc/trace.h
  GEN     hw/net/trace.h
  GEN     hw/virtio/trace.h
  GEN     hw/audio/trace.h
  GEN     hw/misc/trace.h
  GEN     hw/usb/trace.h
  GEN     hw/scsi/trace.h
  GEN     hw/nvram/trace.h
  GEN     hw/display/trace.h
  GEN     hw/timer/trace.h
  GEN     hw/input/trace.h
  GEN     hw/dma/trace.h
  GEN     hw/sparc/trace.h
  GEN     hw/sd/trace.h
  GEN     hw/isa/trace.h
  GEN     hw/mem/trace.h
  GEN     hw/i386/trace.h
  GEN     hw/i386/xen/trace.h
  GEN     hw/9pfs/trace.h
  GEN     hw/ppc/trace.h
  GEN     hw/s390x/trace.h
  GEN     hw/pci/trace.h
  GEN     hw/vfio/trace.h
  GEN     hw/acpi/trace.h
  GEN     hw/arm/trace.h
  GEN     hw/alpha/trace.h
  GEN     hw/xen/trace.h
  GEN     hw/ide/trace.h
  GEN     ui/trace.h
  GEN     audio/trace.h
  GEN     net/trace.h
  GEN     target/arm/trace.h
  GEN     target/i386/trace.h
  GEN     target/mips/trace.h
  GEN     target/sparc/trace.h
  GEN     target/s390x/trace.h
  GEN     target/ppc/trace.h
  GEN     qom/trace.h
  GEN     linux-user/trace.h
  GEN     qapi/trace.h
  GEN     accel/tcg/trace.h
  GEN     accel/kvm/trace.h
  GEN     nbd/trace.h
  GEN     scsi/trace.h
  GEN     trace-root.c
  GEN     util/trace.c
  GEN     crypto/trace.c
  GEN     io/trace.c
  GEN     migration/trace.c
  GEN     block/trace.c
  GEN     chardev/trace.c
  GEN     hw/block/trace.c
  GEN     hw/block/dataplane/trace.c
  GEN     hw/char/trace.c
  GEN     hw/intc/trace.c
  GEN     hw/net/trace.c
  GEN     hw/virtio/trace.c
  GEN     hw/audio/trace.c
  GEN     hw/misc/trace.c
  GEN     hw/scsi/trace.c
  GEN     hw/nvram/trace.c
  GEN     hw/usb/trace.c
  GEN     hw/display/trace.c
  GEN     hw/input/trace.c
  GEN     hw/dma/trace.c
  GEN     hw/timer/trace.c
  GEN     hw/sparc/trace.c
  GEN     hw/sd/trace.c
  GEN     hw/mem/trace.c
  GEN     hw/isa/trace.c
  GEN     hw/i386/trace.c
  GEN     hw/i386/xen/trace.c
  GEN     hw/9pfs/trace.c
  GEN     hw/ppc/trace.c
  GEN     hw/pci/trace.c
  GEN     hw/s390x/trace.c
  GEN     hw/vfio/trace.c
  GEN     hw/acpi/trace.c
  GEN     hw/arm/trace.c
  GEN     hw/alpha/trace.c
  GEN     hw/xen/trace.c
  GEN     hw/ide/trace.c
  GEN     ui/trace.c
  GEN     audio/trace.c
  GEN     net/trace.c
  GEN     target/arm/trace.c
  GEN     target/i386/trace.c
  GEN     target/mips/trace.c
  GEN     target/sparc/trace.c
  GEN     target/s390x/trace.c
  GEN     target/ppc/trace.c
  GEN     qom/trace.c
  GEN     linux-user/trace.c
  GEN     qapi/trace.c
  GEN     accel/tcg/trace.c
  GEN     accel/kvm/trace.c
  GEN     nbd/trace.c
  GEN     scsi/trace.c
  GEN     config-all-devices.mak
  GEN     docs/version.texi
  GEN     qemu-options.texi
  CC      tests/qemu-iotests/socket_scm_helper.o
  GEN     qemu-img-cmds.texi
  GEN     qemu-monitor.texi
  GEN     qemu-monitor-info.texi
  GEN     qemu-img.1
  GEN     qemu-nbd.8
  GEN     qemu-ga.8
  GEN     docs/interop/qemu-qmp-qapi.texi
  GEN     docs/interop/qemu-ga-qapi.texi
  GEN     docs/qemu-block-drivers.7
  GEN     fsdev/virtfs-proxy-helper.1
  GEN     qga/qapi-generated/qga-qapi-types.h
  GEN     qga/qapi-generated/qga-qapi-visit.h
  GEN     qga/qapi-generated/qga-qmp-commands.h
  GEN     qga/qapi-generated/qga-qapi-visit.c
  GEN     qga/qapi-generated/qga-qapi-types.c
  GEN     qga/qapi-generated/qga-qmp-marshal.c
  CC      qmp-introspect.o
  CC      qapi-types.o
  CC      qapi-visit.o
  CC      qapi-event.o
  CC      qapi/qapi-visit-core.o
  CC      qapi/qapi-dealloc-visitor.o
  CC      qapi/qobject-input-visitor.o
  CC      qapi/qobject-output-visitor.o
  CC      qapi/qmp-registry.o
  CC      qapi/qmp-dispatch.o
  CC      qapi/string-output-visitor.o
  CC      qapi/string-input-visitor.o
  CC      qapi/opts-visitor.o
  CC      qapi/qapi-clone-visitor.o
  CC      qapi/qmp-event.o
  CC      qobject/qnull.o
  CC      qapi/qapi-util.o
  CC      qobject/qnum.o
  CC      qobject/qstring.o
  CC      qobject/qdict.o
  CC      qobject/qlist.o
  CC      qobject/qbool.o
  CC      qobject/qlit.o
  CC      qobject/qjson.o
  CC      qobject/qobject.o
  CC      qobject/json-lexer.o
  CC      qobject/json-streamer.o
  CC      qobject/json-parser.o
  CC      trace/control.o
  CC      trace/qmp.o
  CC      util/osdep.o
  CC      util/cutils.o
  CC      util/unicode.o
  CC      util/qemu-timer-common.o
  CC      util/lockcnt.o
  CC      util/bufferiszero.o
  CC      util/aiocb.o
  CC      util/async.o
  CC      util/thread-pool.o
  CC      util/qemu-timer.o
  CC      util/main-loop.o
  CC      util/iohandler.o
  CC      util/compatfd.o
  CC      util/aio-posix.o
  CC      util/event_notifier-posix.o
  CC      util/mmap-alloc.o
  CC      util/oslib-posix.o
  CC      util/qemu-openpty.o
  CC      util/qemu-thread-posix.o
  CC      util/memfd.o
  CC      util/path.o
  CC      util/envlist.o
  CC      util/module.o
  CC      util/host-utils.o
  CC      util/bitmap.o
  CC      util/bitops.o
  CC      util/hbitmap.o
  CC      util/fifo8.o
  CC      util/acl.o
  CC      util/cacheinfo.o
  CC      util/error.o
  CC      util/qemu-error.o
  CC      util/id.o
  CC      util/iov.o
  CC      util/qemu-config.o
  CC      util/qemu-sockets.o
  CC      util/uri.o
  CC      util/notify.o
  CC      util/qemu-option.o
  CC      util/qemu-progress.o
  CC      util/keyval.o
  CC      util/hexdump.o
  CC      util/crc32c.o
  CC      util/uuid.o
  CC      util/throttle.o
  CC      util/getauxval.o
  CC      util/readline.o
  CC      util/qemu-coroutine.o
  CC      util/rcu.o
  CC      util/qemu-coroutine-lock.o
  CC      util/qemu-coroutine-io.o
  CC      util/qemu-coroutine-sleep.o
  CC      util/coroutine-ucontext.o
  CC      util/timed-average.o
  CC      util/buffer.o
  CC      util/base64.o
  CC      util/log.o
  CC      util/qdist.o
  CC      util/qht.o
  CC      util/range.o
  CC      util/stats64.o
  CC      trace-root.o
  CC      util/systemd.o
  CC      util/trace.o
  CC      crypto/trace.o
  CC      io/trace.o
  CC      migration/trace.o
  CC      block/trace.o
  CC      chardev/trace.o
  CC      hw/block/trace.o
  CC      hw/block/dataplane/trace.o
  CC      hw/char/trace.o
  CC      hw/intc/trace.o
  CC      hw/net/trace.o
  CC      hw/virtio/trace.o
  CC      hw/audio/trace.o
  CC      hw/misc/trace.o
  CC      hw/usb/trace.o
  CC      hw/scsi/trace.o
  CC      hw/nvram/trace.o
  CC      hw/display/trace.o
  CC      hw/input/trace.o
  CC      hw/timer/trace.o
  CC      hw/dma/trace.o
  CC      hw/sparc/trace.o
  CC      hw/sd/trace.o
  CC      hw/isa/trace.o
  CC      hw/mem/trace.o
  CC      hw/i386/trace.o
  CC      hw/i386/xen/trace.o
  CC      hw/9pfs/trace.o
  CC      hw/ppc/trace.o
  CC      hw/pci/trace.o
  CC      hw/s390x/trace.o
  CC      hw/vfio/trace.o
  CC      hw/acpi/trace.o
  CC      hw/arm/trace.o
  CC      hw/alpha/trace.o
  CC      hw/xen/trace.o
  CC      ui/trace.o
  CC      hw/ide/trace.o
  CC      audio/trace.o
  CC      net/trace.o
  CC      target/arm/trace.o
  CC      target/i386/trace.o
  CC      target/mips/trace.o
  CC      target/sparc/trace.o
  CC      target/s390x/trace.o
  CC      target/ppc/trace.o
  CC      qom/trace.o
  CC      linux-user/trace.o
  CC      qapi/trace.o
  CC      accel/tcg/trace.o
  CC      accel/kvm/trace.o
  CC      nbd/trace.o
  CC      scsi/trace.o
  CC      crypto/pbkdf-stub.o
  CC      stubs/arch-query-cpu-def.o
  CC      stubs/arch-query-cpu-model-expansion.o
  CC      stubs/arch-query-cpu-model-comparison.o
  CC      stubs/arch-query-cpu-model-baseline.o
  CC      stubs/bdrv-next-monitor-owned.o
  CC      stubs/blk-commit-all.o
  CC      stubs/blockdev-close-all-bdrv-states.o
  CC      stubs/clock-warp.o
  CC      stubs/cpu-get-clock.o
  CC      stubs/cpu-get-icount.o
  CC      stubs/dump.o
  CC      stubs/error-printf.o
  CC      stubs/fdset.o
  CC      stubs/gdbstub.o
  CC      stubs/get-vm-name.o
  CC      stubs/iothread.o
  CC      stubs/iothread-lock.o
  CC      stubs/is-daemonized.o
  CC      stubs/linux-aio.o
  CC      stubs/machine-init-done.o
  CC      stubs/migr-blocker.o
  CC      stubs/change-state-handler.o
  CC      stubs/monitor.o
  CC      stubs/notify-event.o
  CC      stubs/qtest.o
  CC      stubs/runstate-check.o
  CC      stubs/replay.o
  CC      stubs/set-fd-handler.o
  CC      stubs/slirp.o
  CC      stubs/sysbus.o
  CC      stubs/trace-control.o
  CC      stubs/uuid.o
  CC      stubs/vm-stop.o
  CC      stubs/vmstate.o
  CC      stubs/qmp_pc_dimm.o
  CC      stubs/target-get-monitor-def.o
  CC      stubs/target-monitor-defs.o
  CC      stubs/pc_madt_cpu_entry.o
  CC      stubs/vmgenid.o
  CC      stubs/xen-common.o
  CC      stubs/xen-hvm.o
  CC      stubs/pci-host-piix.o
  CC      contrib/ivshmem-client/ivshmem-client.o
  CC      contrib/ivshmem-client/main.o
  CC      contrib/ivshmem-server/ivshmem-server.o
  CC      contrib/ivshmem-server/main.o
  CC      qemu-nbd.o
  CC      block.o
  CC      blockjob.o
  CC      qemu-io-cmds.o
  CC      replication.o
  CC      block/raw-format.o
  CC      block/qcow.o
  CC      block/vdi.o
  CC      block/vmdk.o
  CC      block/cloop.o
  CC      block/bochs.o
  CC      block/vpc.o
  CC      block/vvfat.o
  CC      block/dmg.o
  CC      block/qcow2.o
  CC      block/qcow2-refcount.o
  CC      block/qcow2-cluster.o
  CC      block/qcow2-snapshot.o
  CC      block/qcow2-cache.o
  CC      block/qcow2-bitmap.o
  CC      block/qed.o
  CC      block/qed-l2-cache.o
  CC      block/qed-table.o
  CC      block/qed-cluster.o
  CC      block/qed-check.o
  CC      block/vhdx.o
  CC      block/vhdx-endian.o
  CC      block/vhdx-log.o
  CC      block/quorum.o
  CC      block/parallels.o
  CC      block/blkdebug.o
  CC      block/blkverify.o
  CC      block/blkreplay.o
  CC      block/block-backend.o
  CC      block/snapshot.o
  CC      block/qapi.o
  CC      block/file-posix.o
  CC      block/linux-aio.o
  CC      block/null.o
  CC      block/mirror.o
  CC      block/commit.o
  CC      block/io.o
  CC      block/throttle-groups.o
  CC      block/nbd.o
  CC      block/nbd-client.o
  CC      block/sheepdog.o
  CC      block/iscsi-opts.o
  CC      block/accounting.o
  CC      block/dirty-bitmap.o
  CC      block/write-threshold.o
  CC      block/backup.o
  CC      block/replication.o
  CC      block/throttle.o
  CC      block/crypto.o
  CC      nbd/server.o
  CC      nbd/client.o
  CC      nbd/common.o
  CC      scsi/utils.o
  CC      scsi/pr-manager.o
  CC      scsi/pr-manager-helper.o
  CC      block/iscsi.o
  CC      block/nfs.o
  CC      block/curl.o
  CC      block/rbd.o
  CC      block/gluster.o
  CC      block/ssh.o
  CC      block/dmg-bz2.o
  CC      crypto/init.o
  CC      crypto/hash.o
  CC      crypto/hash-nettle.o
  CC      crypto/hmac.o
  CC      crypto/hmac-nettle.o
  CC      crypto/aes.o
  CC      crypto/desrfb.o
  CC      crypto/cipher.o
  CC      crypto/tlscreds.o
  CC      crypto/tlscredsanon.o
  CC      crypto/tlscredsx509.o
  CC      crypto/tlssession.o
  CC      crypto/secret.o
  CC      crypto/random-gnutls.o
  CC      crypto/pbkdf.o
  CC      crypto/pbkdf-nettle.o
  CC      crypto/ivgen.o
  CC      crypto/ivgen-essiv.o
  CC      crypto/ivgen-plain.o
  CC      crypto/ivgen-plain64.o
  CC      crypto/afsplit.o
  CC      crypto/xts.o
  CC      crypto/block.o
  CC      crypto/block-qcow.o
  CC      crypto/block-luks.o
  CC      io/channel.o
  CC      io/channel-buffer.o
  CC      io/channel-command.o
  CC      io/channel-file.o
  CC      io/channel-socket.o
  CC      io/channel-tls.o
  CC      io/channel-watch.o
  CC      io/channel-websock.o
  CC      io/channel-util.o
  CC      io/dns-resolver.o
  CC      io/task.o
  CC      qom/object.o
  CC      qom/container.o
  CC      qom/qom-qobject.o
  CC      qom/object_interfaces.o
  CC      qemu-io.o
  GEN     qemu-img-cmds.h
  CC      fsdev/9p-marshal.o
  CC      fsdev/virtfs-proxy-helper.o
  CC      scsi/qemu-pr-helper.o
  CC      fsdev/9p-iov-marshal.o
  CC      qemu-bridge-helper.o
  CC      blockdev.o
  CC      blockdev-nbd.o
  CC      bootdevice.o
  CC      iothread.o
  CC      qdev-monitor.o
  CC      device-hotplug.o
  CC      os-posix.o
  CC      bt-host.o
  CC      bt-vhci.o
  CC      dma-helpers.o
  CC      vl.o
  CC      tpm.o
  CC      qemu-seccomp.o
  CC      device_tree.o
  CC      qmp-marshal.o
  CC      qmp.o
  CC      hmp.o
  CC      cpus-common.o
  CC      audio/audio.o
  CC      audio/noaudio.o
  CC      audio/wavaudio.o
  CC      audio/mixeng.o
  CC      audio/sdlaudio.o
  CC      audio/ossaudio.o
  CC      audio/wavcapture.o
  CC      backends/rng.o
  CC      backends/rng-egd.o
  CC      backends/rng-random.o
  CC      backends/tpm.o
  CC      backends/hostmem.o
  CC      backends/hostmem-ram.o
  CC      backends/hostmem-file.o
  CC      backends/cryptodev.o
  CC      backends/cryptodev-builtin.o
  CC      block/stream.o
  CC      chardev/msmouse.o
  CC      chardev/wctablet.o
  CC      chardev/testdev.o
  CC      chardev/baum.o
  CC      disas/alpha.o
  CC      disas/arm.o
  CXX     disas/arm-a64.o
  CC      disas/cris.o
  CC      disas/hppa.o
  CC      disas/m68k.o
  CC      disas/microblaze.o
  CC      disas/i386.o
  CC      disas/mips.o
  CC      disas/nios2.o
  CC      disas/moxie.o
  CC      disas/ppc.o
  CC      disas/s390.o
  CC      disas/sh4.o
  CC      disas/sparc.o
  CC      disas/lm32.o
  CXX     disas/libvixl/vixl/utils.o
  CXX     disas/libvixl/vixl/compiler-intrinsics.o
  CXX     disas/libvixl/vixl/a64/instructions-a64.o
  CXX     disas/libvixl/vixl/a64/decoder-a64.o
  CXX     disas/libvixl/vixl/a64/disasm-a64.o
  CC      fsdev/qemu-fsdev.o
  CC      fsdev/qemu-fsdev-opts.o
  CC      fsdev/qemu-fsdev-throttle.o
  CC      fsdev/qemu-fsdev-dummy.o
  CC      hw/9pfs/9p.o
  CC      hw/9pfs/9p-util.o
  CC      hw/9pfs/9p-local.o
  CC      hw/9pfs/9p-xattr.o
  CC      hw/9pfs/9p-xattr-user.o
  CC      hw/9pfs/9p-posix-acl.o
  CC      hw/9pfs/coth.o
  CC      hw/9pfs/cofs.o
  CC      hw/9pfs/cofile.o
  CC      hw/9pfs/codir.o
  CC      hw/9pfs/coxattr.o
  CC      hw/9pfs/9p-synth.o
  CC      hw/9pfs/9p-handle.o
  CC      hw/9pfs/9p-proxy.o
  CC      hw/acpi/core.o
  CC      hw/acpi/piix4.o
  CC      hw/acpi/pcihp.o
  CC      hw/acpi/ich9.o
  CC      hw/acpi/tco.o
  CC      hw/acpi/cpu_hotplug.o
  CC      hw/acpi/memory_hotplug.o
  CC      hw/acpi/cpu.o
  CC      hw/acpi/vmgenid.o
  CC      hw/acpi/nvdimm.o
  CC      hw/acpi/acpi_interface.o
  CC      hw/acpi/bios-linker-loader.o
  CC      hw/acpi/aml-build.o
  CC      hw/acpi/ipmi.o
  CC      hw/acpi/acpi-stub.o
  CC      hw/acpi/ipmi-stub.o
  CC      hw/audio/sb16.o
  CC      hw/audio/es1370.o
  CC      hw/audio/ac97.o
  CC      hw/audio/fmopl.o
  CC      hw/audio/adlib.o
  CC      hw/audio/gus.o
  CC      hw/audio/gusemu_hal.o
  CC      hw/audio/gusemu_mixer.o
  CC      hw/audio/cs4231a.o
  CC      hw/audio/hda-codec.o
  CC      hw/audio/intel-hda.o
  CC      hw/audio/pcspk.o
  CC      hw/audio/wm8750.o
  CC      hw/audio/pl041.o
  CC      hw/audio/lm4549.o
  CC      hw/audio/cs4231.o
  CC      hw/audio/marvell_88w8618.o
  CC      hw/audio/milkymist-ac97.o
  CC      hw/audio/soundhw.o
  CC      hw/block/block.o
  CC      hw/block/cdrom.o
  CC      hw/block/hd-geometry.o
  CC      hw/block/fdc.o
  CC      hw/block/m25p80.o
  CC      hw/block/nand.o
  CC      hw/block/pflash_cfi01.o
  CC      hw/block/pflash_cfi02.o
  CC      hw/block/ecc.o
  CC      hw/block/onenand.o
  CC      hw/block/nvme.o
  CC      hw/bt/core.o
  CC      hw/bt/l2cap.o
  CC      hw/bt/sdp.o
  CC      hw/bt/hci.o
  CC      hw/bt/hid.o
  CC      hw/bt/hci-csr.o
  CC      hw/char/ipoctal232.o
  CC      hw/char/escc.o
  CC      hw/char/parallel.o
  CC      hw/char/pl011.o
  CC      hw/char/serial.o
  CC      hw/char/serial-isa.o
  CC      hw/char/serial-pci.o
  CC      hw/char/virtio-console.o
  CC      hw/char/xilinx_uartlite.o
  CC      hw/char/cadence_uart.o
  CC      hw/char/cmsdk-apb-uart.o
  CC      hw/char/etraxfs_ser.o
  CC      hw/char/debugcon.o
  CC      hw/char/grlib_apbuart.o
  CC      hw/char/imx_serial.o
  CC      hw/char/lm32_juart.o
  CC      hw/char/lm32_uart.o
  CC      hw/char/milkymist-uart.o
  CC      hw/char/sclpconsole.o
  CC      hw/char/sclpconsole-lm.o
  CC      hw/core/qdev.o
  CC      hw/core/qdev-properties.o
  CC      hw/core/bus.o
  CC      hw/core/reset.o
  CC      hw/core/fw-path-provider.o
  CC      hw/core/irq.o
  CC      hw/core/hotplug.o
  CC      hw/core/nmi.o
  CC      hw/core/empty_slot.o
  CC      hw/core/stream.o
  CC      hw/core/ptimer.o
  CC      hw/core/sysbus.o
  CC      hw/core/machine.o
  CC      hw/core/loader.o
  CC      hw/core/loader-fit.o
  CC      hw/core/qdev-properties-system.o
  CC      hw/core/register.o
  CC      hw/core/or-irq.o
  CC      hw/core/platform-bus.o
  CC      hw/cpu/core.o
  CC      hw/display/ads7846.o
  CC      hw/display/cirrus_vga.o
  CC      hw/display/g364fb.o
  CC      hw/display/jazz_led.o
  CC      hw/display/pl110.o
  CC      hw/display/ssd0303.o
  CC      hw/display/ssd0323.o
  CC      hw/display/vga-pci.o
  CC      hw/display/vga-isa.o
  CC      hw/display/vga-isa-mm.o
  CC      hw/display/vmware_vga.o
  CC      hw/display/blizzard.o
  CC      hw/display/exynos4210_fimd.o
  CC      hw/display/framebuffer.o
  CC      hw/display/milkymist-vgafb.o
  CC      hw/display/tc6393xb.o
  CC      hw/display/milkymist-tmu2.o
  CC      hw/dma/puv3_dma.o
  CC      hw/dma/rc4030.o
  CC      hw/dma/pl080.o
  CC      hw/dma/pl330.o
  CC      hw/dma/i8257.o
  CC      hw/dma/i82374.o
  CC      hw/dma/xilinx_axidma.o
  CC      hw/dma/xlnx-zynq-devcfg.o
  CC      hw/dma/etraxfs_dma.o
  CC      hw/dma/sparc32_dma.o
  CC      hw/dma/sun4m_iommu.o
  CC      hw/gpio/max7310.o
  CC      hw/gpio/pl061.o
  CC      hw/gpio/puv3_gpio.o
  CC      hw/gpio/zaurus.o
  CC      hw/gpio/mpc8xxx.o
  CC      hw/gpio/gpio_key.o
  CC      hw/i2c/core.o
  CC      hw/i2c/smbus.o
  CC      hw/i2c/smbus_eeprom.o
  CC      hw/i2c/i2c-ddc.o
  CC      hw/i2c/versatile_i2c.o
  CC      hw/i2c/smbus_ich9.o
  CC      hw/i2c/pm_smbus.o
  CC      hw/i2c/bitbang_i2c.o
  CC      hw/i2c/exynos4210_i2c.o
  CC      hw/i2c/imx_i2c.o
  CC      hw/i2c/aspeed_i2c.o
  CC      hw/ide/core.o
  CC      hw/ide/atapi.o
  CC      hw/ide/qdev.o
  CC      hw/ide/pci.o
  CC      hw/ide/isa.o
  CC      hw/ide/piix.o
  CC      hw/ide/cmd646.o
  CC      hw/ide/macio.o
  CC      hw/ide/mmio.o
  CC      hw/ide/microdrive.o
  CC      hw/ide/via.o
  CC      hw/ide/ahci.o
  CC      hw/ide/ich.o
  CC      hw/input/adb.o
  CC      hw/input/hid.o
  CC      hw/input/lm832x.o
  CC      hw/input/pckbd.o
  CC      hw/input/pl050.o
  CC      hw/input/ps2.o
  CC      hw/input/stellaris_input.o
  CC      hw/input/tsc2005.o
  CC      hw/input/vmmouse.o
  CC      hw/input/virtio-input.o
  CC      hw/input/virtio-input-hid.o
  CC      hw/input/virtio-input-host.o
  CC      hw/intc/heathrow_pic.o
  CC      hw/intc/i8259_common.o
  CC      hw/intc/i8259.o
  CC      hw/intc/pl190.o
  CC      hw/intc/puv3_intc.o
  CC      hw/intc/xilinx_intc.o
  CC      hw/intc/etraxfs_pic.o
  CC      hw/intc/imx_avic.o
  CC      hw/intc/lm32_pic.o
  CC      hw/intc/realview_gic.o
  CC      hw/intc/ioapic_common.o
  CC      hw/intc/slavio_intctl.o
  CC      hw/intc/arm_gic_common.o
  CC      hw/intc/arm_gic.o
  CC      hw/intc/arm_gicv2m.o
  CC      hw/intc/arm_gicv3_common.o
  CC      hw/intc/arm_gicv3.o
  CC      hw/intc/arm_gicv3_dist.o
  CC      hw/intc/arm_gicv3_redist.o
  CC      hw/intc/arm_gicv3_its_common.o
  CC      hw/intc/openpic.o
  CC      hw/intc/intc.o
  CC      hw/ipack/ipack.o
  CC      hw/ipack/tpci200.o
  CC      hw/ipmi/ipmi.o
  CC      hw/ipmi/ipmi_bmc_sim.o
  CC      hw/ipmi/ipmi_bmc_extern.o
  CC      hw/ipmi/isa_ipmi_kcs.o
  CC      hw/ipmi/isa_ipmi_bt.o
  CC      hw/isa/isa-bus.o
  CC      hw/isa/apm.o
  CC      hw/isa/i82378.o
  CC      hw/isa/pc87312.o
  CC      hw/isa/piix4.o
  CC      hw/isa/vt82c686.o
  CC      hw/mem/pc-dimm.o
  CC      hw/mem/nvdimm.o
  CC      hw/misc/applesmc.o
  CC      hw/misc/max111x.o
  CC      hw/misc/tmp105.o
  CC      hw/misc/tmp421.o
  CC      hw/misc/debugexit.o
  CC      hw/misc/sga.o
  CC      hw/misc/pc-testdev.o
  CC      hw/misc/pci-testdev.o
  CC      hw/misc/edu.o
  CC      hw/misc/unimp.o
  CC      hw/misc/arm_l2x0.o
  CC      hw/misc/arm_integrator_debug.o
  CC      hw/misc/a9scu.o
  CC      hw/misc/arm11scu.o
  CC      hw/misc/puv3_pm.o
  CC      hw/misc/macio/macio.o
  CC      hw/misc/macio/cuda.o
  CC      hw/misc/macio/mac_dbdma.o
  CC      hw/net/dp8393x.o
  CC      hw/net/ne2000.o
  CC      hw/net/eepro100.o
  CC      hw/net/pcnet-pci.o
  CC      hw/net/pcnet.o
  CC      hw/net/e1000.o
  CC      hw/net/e1000x_common.o
  CC      hw/net/net_tx_pkt.o
  CC      hw/net/net_rx_pkt.o
  CC      hw/net/e1000e_core.o
  CC      hw/net/e1000e.o
  CC      hw/net/vmxnet3.o
  CC      hw/net/rtl8139.o
  CC      hw/net/smc91c111.o
  CC      hw/net/lan9118.o
  CC      hw/net/ne2000-isa.o
  CC      hw/net/opencores_eth.o
  CC      hw/net/xgmac.o
  CC      hw/net/mipsnet.o
  CC      hw/net/xilinx_axienet.o
  CC      hw/net/allwinner_emac.o
  CC      hw/net/imx_fec.o
  CC      hw/net/cadence_gem.o
  CC      hw/net/stellaris_enet.o
  CC      hw/net/lance.o
  CC      hw/net/sunhme.o
  CC      hw/net/ftgmac100.o
  CC      hw/net/sungem.o
  CC      hw/net/rocker/rocker.o
  CC      hw/net/rocker/rocker_desc.o
  CC      hw/net/rocker/rocker_fp.o
  CC      hw/net/rocker/rocker_world.o
  CC      hw/net/rocker/rocker_of_dpa.o
  CC      hw/nvram/ds1225y.o
  CC      hw/nvram/eeprom93xx.o
  CC      hw/nvram/fw_cfg.o
  CC      hw/nvram/chrp_nvram.o
  CC      hw/nvram/mac_nvram.o
  CC      hw/pci-bridge/pci_bridge_dev.o
  CC      hw/pci-bridge/pcie_pci_bridge.o
  CC      hw/pci-bridge/pcie_root_port.o
  CC      hw/pci-bridge/gen_pcie_root_port.o
  CC      hw/pci-bridge/pci_expander_bridge.o
  CC      hw/pci-bridge/xio3130_upstream.o
  CC      hw/pci-bridge/xio3130_downstream.o
  CC      hw/pci-bridge/ioh3420.o
  CC      hw/pci-bridge/i82801b11.o
  CC      hw/pci-bridge/dec.o
  CC      hw/pci-host/pam.o
  CC      hw/pci-host/prep.o
  CC      hw/pci-host/grackle.o
  CC      hw/pci-host/uninorth.o
  CC      hw/pci-host/ppce500.o
  CC      hw/pci-host/versatile.o
  CC      hw/pci-host/apb.o
  CC      hw/pci-host/bonito.o
  CC      hw/pci-host/piix.o
  CC      hw/pci-host/q35.o
  CC      hw/pci-host/gpex.o
  CC      hw/pci-host/xilinx-pcie.o
  CC      hw/pci/pci_bridge.o
  CC      hw/pci/pci.o
  CC      hw/pci/msix.o
  CC      hw/pci/msi.o
  CC      hw/pci/shpc.o
  CC      hw/pci/slotid_cap.o
  CC      hw/pci/pci_host.o
  CC      hw/pci/pcie_host.o
  CC      hw/pci/pcie.o
  CC      hw/pci/pcie_aer.o
  CC      hw/pci/pcie_port.o
  CC      hw/pci/pci-stub.o
  CC      hw/pcmcia/pcmcia.o
  CC      hw/scsi/scsi-disk.o
  CC      hw/scsi/scsi-generic.o
  CC      hw/scsi/scsi-bus.o
  CC      hw/scsi/lsi53c895a.o
  CC      hw/scsi/mptsas.o
  CC      hw/scsi/mptconfig.o
  CC      hw/scsi/mptendian.o
  CC      hw/scsi/megasas.o
  CC      hw/scsi/vmw_pvscsi.o
  CC      hw/scsi/esp.o
  CC      hw/scsi/esp-pci.o
  CC      hw/sd/pl181.o
  CC      hw/sd/ssi-sd.o
  CC      hw/sd/sd.o
  CC      hw/sd/core.o
  CC      hw/sd/sdhci.o
  CC      hw/smbios/smbios.o
  CC      hw/smbios/smbios_type_38.o
  CC      hw/smbios/smbios-stub.o
  CC      hw/smbios/smbios_type_38-stub.o
  CC      hw/ssi/pl022.o
  CC      hw/ssi/ssi.o
  CC      hw/ssi/xilinx_spi.o
  CC      hw/ssi/xilinx_spips.o
  CC      hw/ssi/aspeed_smc.o
  CC      hw/ssi/stm32f2xx_spi.o
  CC      hw/ssi/mss-spi.o
  CC      hw/timer/arm_timer.o
  CC      hw/timer/arm_mptimer.o
  CC      hw/timer/armv7m_systick.o
  CC      hw/timer/a9gtimer.o
  CC      hw/timer/cadence_ttc.o
  CC      hw/timer/ds1338.o
  CC      hw/timer/hpet.o
  CC      hw/timer/i8254_common.o
  CC      hw/timer/i8254.o
  CC      hw/timer/m48t59.o
  CC      hw/timer/m48t59-isa.o
  CC      hw/timer/pl031.o
  CC      hw/timer/puv3_ost.o
  CC      hw/timer/twl92230.o
  CC      hw/timer/xilinx_timer.o
  CC      hw/timer/slavio_timer.o
  CC      hw/timer/etraxfs_timer.o
  CC      hw/timer/grlib_gptimer.o
  CC      hw/timer/imx_epit.o
  CC      hw/timer/imx_gpt.o
  CC      hw/timer/lm32_timer.o
  CC      hw/timer/milkymist-sysctl.o
  CC      hw/timer/stm32f2xx_timer.o
  CC      hw/timer/sun4v-rtc.o
  CC      hw/timer/aspeed_timer.o
  CC      hw/timer/cmsdk-apb-timer.o
  CC      hw/timer/mss-timer.o
  CC      hw/tpm/tpm_tis.o
  CC      hw/usb/core.o
  CC      hw/usb/combined-packet.o
  CC      hw/usb/bus.o
  CC      hw/usb/libhw.o
  CC      hw/usb/desc-msos.o
  CC      hw/usb/desc.o
  CC      hw/usb/hcd-uhci.o
  CC      hw/usb/hcd-ohci.o
  CC      hw/usb/hcd-ehci.o
  CC      hw/usb/hcd-ehci-pci.o
  CC      hw/usb/hcd-ehci-sysbus.o
  CC      hw/usb/hcd-xhci.o
  CC      hw/usb/hcd-xhci-nec.o
  CC      hw/usb/hcd-musb.o
  CC      hw/usb/dev-hub.o
  CC      hw/usb/dev-hid.o
  CC      hw/usb/dev-wacom.o
  CC      hw/usb/dev-storage.o
  CC      hw/usb/dev-uas.o
  CC      hw/usb/dev-audio.o
  CC      hw/usb/dev-serial.o
  CC      hw/usb/dev-network.o
  CC      hw/usb/dev-smartcard-reader.o
  CC      hw/usb/dev-bluetooth.o
  CC      hw/usb/ccid-card-passthru.o
  CC      hw/usb/ccid-card-emulated.o
  CC      hw/usb/dev-mtp.o
  CC      hw/usb/redirect.o
  CC      hw/usb/quirks.o
  CC      hw/usb/host-stub.o
  CC      hw/virtio/virtio-rng.o
  CC      hw/virtio/virtio-pci.o
  CC      hw/virtio/virtio-mmio.o
  CC      hw/virtio/vhost-stub.o
  CC      hw/virtio/virtio-bus.o
  CC      hw/watchdog/watchdog.o
  CC      hw/watchdog/wdt_i6300esb.o
  CC      hw/watchdog/wdt_ib700.o
  CC      hw/watchdog/wdt_diag288.o
  CC      hw/watchdog/wdt_aspeed.o
  CC      migration/migration.o
  CC      migration/socket.o
  CC      migration/fd.o
  CC      migration/exec.o
  CC      migration/tls.o
  CC      migration/channel.o
  CC      migration/savevm.o
  CC      migration/colo-comm.o
  CC      migration/colo.o
  CC      migration/colo-failover.o
  CC      migration/vmstate.o
  CC      migration/vmstate-types.o
  CC      migration/page_cache.o
  CC      migration/qemu-file.o
  CC      migration/global_state.o
  CC      migration/qemu-file-channel.o
  CC      migration/xbzrle.o
  CC      migration/postcopy-ram.o
  CC      migration/qjson.o
  CC      migration/block.o
  CC      net/net.o
  CC      net/queue.o
  CC      net/checksum.o
  CC      net/util.o
  CC      net/socket.o
  CC      net/hub.o
  CC      net/eth.o
  CC      net/dump.o
  CC      net/vhost-user.o
  CC      net/l2tpv3.o
  CC      net/slirp.o
  CC      net/filter.o
  CC      net/filter-buffer.o
  CC      net/filter-mirror.o
  CC      net/colo-compare.o
  CC      net/colo.o
  CC      net/filter-replay.o
  CC      net/filter-rewriter.o
  CC      net/tap-linux.o
  CC      net/tap.o
  CC      qom/cpu.o
  CC      replay/replay.o
  CC      replay/replay-internal.o
  CC      replay/replay-events.o
  CC      replay/replay-time.o
  CC      replay/replay-input.o
  CC      replay/replay-char.o
  CC      replay/replay-snapshot.o
  CC      replay/replay-net.o
  CC      replay/replay-audio.o
  CC      slirp/if.o
  CC      slirp/cksum.o
  CC      slirp/ip_icmp.o
  CC      slirp/ip6_icmp.o
  CC      slirp/ip6_input.o
  CC      slirp/ip6_output.o
  CC      slirp/ip_input.o
  CC      slirp/ip_output.o
  CC      slirp/dnssearch.o
  CC      slirp/slirp.o
  CC      slirp/mbuf.o
  CC      slirp/dhcpv6.o
  CC      slirp/sbuf.o
  CC      slirp/misc.o
  CC      slirp/socket.o
  CC      slirp/tcp_input.o
  CC      slirp/tcp_output.o
  CC      slirp/tcp_subr.o
  CC      slirp/udp.o
  CC      slirp/tcp_timer.o
  CC      slirp/udp6.o
  CC      slirp/bootp.o
  CC      slirp/tftp.o
  CC      slirp/arp_table.o
  CC      slirp/ndp_table.o
  CC      slirp/ncsi.o
  CC      ui/keymaps.o
  CC      ui/console.o
  CC      ui/cursor.o
  CC      ui/qemu-pixman.o
  CC      ui/input.o
  CC      ui/input-keymap.o
  CC      ui/input-legacy.o
  CC      ui/input-linux.o
  CC      ui/sdl2.o
  CC      ui/sdl2-input.o
  CC      ui/sdl2-2d.o
  CC      ui/sdl2-gl.o
  CC      ui/x_keymap.o
  CC      ui/curses.o
  CC      ui/vnc.o
  CC      ui/vnc-enc-zlib.o
  CC      ui/vnc-enc-hextile.o
  CC      ui/vnc-enc-tight.o
  CC      ui/vnc-palette.o
  CC      ui/vnc-enc-zrle.o
  CC      ui/vnc-auth-vencrypt.o
  CC      ui/vnc-auth-sasl.o
  CC      ui/vnc-ws.o
  CC      ui/vnc-jobs.o
  CC      ui/gtk.o
  CC      ui/shader.o
  VERT    ui/shader/texture-blit-vert.h
  FRAG    ui/shader/texture-blit-frag.h
  CC      ui/egl-helpers.o
  CC      ui/egl-context.o
  CC      ui/egl-headless.o
  CC      ui/gtk-gl-area.o
  CC      chardev/char.o
  CC      chardev/char-fd.o
  CC      chardev/char-fe.o
  CC      chardev/char-file.o
  CC      chardev/char-io.o
  CC      chardev/char-mux.o
  CC      chardev/char-null.o
  CC      chardev/char-parallel.o
  CC      chardev/char-pipe.o
  CC      chardev/char-pty.o
  CC      chardev/char-ringbuf.o
  CC      chardev/char-serial.o
  CC      chardev/char-socket.o
  CC      chardev/char-stdio.o
  CC      chardev/char-udp.o
  CCAS    s390-ccw/start.o
  CC      s390-ccw/main.o
  CC      s390-ccw/bootmap.o
  LINK    tests/qemu-iotests/socket_scm_helper
  CC      s390-ccw/sclp.o
  GEN     qemu-doc.html
  CC      s390-ccw/virtio.o
  CC      s390-ccw/virtio-scsi.o
  CC      s390-ccw/virtio-blkdev.o
s390-netboot.img not built since roms/SLOF/ is not available.
  GEN     qemu-doc.txt
  GEN     qemu.1
  BUILD   s390-ccw/s390-ccw.elf
  STRIP   s390-ccw/s390-ccw.img
  GEN     docs/interop/qemu-qmp-ref.html
  GEN     docs/interop/qemu-qmp-ref.txt
  GEN     docs/interop/qemu-qmp-ref.7
  GEN     docs/interop/qemu-ga-ref.html
  GEN     docs/interop/qemu-ga-ref.txt
  GEN     docs/interop/qemu-ga-ref.7
  CC      qga/commands.o
  CC      qga/guest-agent-command-state.o
  CC      qga/main.o
  CC      qga/commands-posix.o
  CC      qga/channel-posix.o
  CC      qga/qapi-generated/qga-qapi-types.o
  CC      qga/qapi-generated/qga-qapi-visit.o
  CC      qga/qapi-generated/qga-qmp-marshal.o
  AR      libqemuutil.a
  CC      qemu-img.o
  LINK    qemu-io
  LINK    fsdev/virtfs-proxy-helper
  LINK    scsi/qemu-pr-helper
  LINK    qemu-bridge-helper
  CC      ui/console-gl.o
  LINK    qemu-ga
  LINK    ivshmem-client
  LINK    ivshmem-server
  LINK    qemu-nbd
  GEN     alpha-softmmu/hmp-commands.h
  GEN     aarch64-softmmu/hmp-commands.h
  GEN     alpha-softmmu/hmp-commands-info.h
  GEN     alpha-softmmu/config-target.h
  GEN     arm-softmmu/hmp-commands.h
  GEN     aarch64-softmmu/hmp-commands-info.h
  CC      alpha-softmmu/exec.o
  GEN     aarch64-softmmu/config-target.h
  GEN     arm-softmmu/hmp-commands-info.h
  GEN     arm-softmmu/config-target.h
  CC      aarch64-softmmu/exec.o
  CC      arm-softmmu/exec.o
  GEN     cris-softmmu/hmp-commands.h
  GEN     cris-softmmu/hmp-commands-info.h
  GEN     cris-softmmu/config-target.h
  CC      cris-softmmu/exec.o
  CC      alpha-softmmu/tcg/tcg.o
  CC      arm-softmmu/tcg/tcg.o
  CC      aarch64-softmmu/tcg/tcg.o
  CC      cris-softmmu/tcg/tcg.o
  CC      alpha-softmmu/tcg/tcg-op.o
  CC      arm-softmmu/tcg/tcg-op.o
  CC      aarch64-softmmu/tcg/tcg-op.o
  CC      cris-softmmu/tcg/tcg-op.o
  CC      aarch64-softmmu/tcg/optimize.o
  CC      alpha-softmmu/tcg/optimize.o
  CC      arm-softmmu/tcg/optimize.o
  CC      alpha-softmmu/tcg/tcg-common.o
  CC      cris-softmmu/tcg/optimize.o
  CC      arm-softmmu/tcg/tcg-common.o
  CC      alpha-softmmu/fpu/softfloat.o
  CC      aarch64-softmmu/tcg/tcg-common.o
  CC      arm-softmmu/fpu/softfloat.o
  CC      aarch64-softmmu/fpu/softfloat.o
  CC      cris-softmmu/tcg/tcg-common.o
  CC      cris-softmmu/fpu/softfloat.o
  CC      alpha-softmmu/disas.o
  CC      aarch64-softmmu/disas.o
  CC      alpha-softmmu/arch_init.o
  CC      arm-softmmu/disas.o
  CC      alpha-softmmu/cpus.o
  GEN     aarch64-softmmu/gdbstub-xml.c
  GEN     arm-softmmu/gdbstub-xml.c
  CC      arm-softmmu/arch_init.o
  CC      aarch64-softmmu/arch_init.o
  CC      arm-softmmu/cpus.o
  CC      alpha-softmmu/monitor.o
  CC      aarch64-softmmu/cpus.o
  CC      arm-softmmu/monitor.o
  CC      cris-softmmu/disas.o
  CC      cris-softmmu/arch_init.o
  CC      aarch64-softmmu/monitor.o
  CC      cris-softmmu/cpus.o
  CC      alpha-softmmu/gdbstub.o
  CC      cris-softmmu/monitor.o
  CC      arm-softmmu/gdbstub.o
  CC      alpha-softmmu/balloon.o
  CC      aarch64-softmmu/gdbstub.o
  CC      arm-softmmu/balloon.o
  CC      alpha-softmmu/ioport.o
  CC      arm-softmmu/ioport.o
  CC      aarch64-softmmu/balloon.o
  CC      alpha-softmmu/numa.o
  CC      aarch64-softmmu/ioport.o
  CC      cris-softmmu/gdbstub.o
  CC      arm-softmmu/numa.o
  CC      alpha-softmmu/qtest.o
  CC      aarch64-softmmu/numa.o
  CC      arm-softmmu/qtest.o
  CC      aarch64-softmmu/qtest.o
  CC      cris-softmmu/balloon.o
  CC      alpha-softmmu/memory.o
  CC      cris-softmmu/ioport.o
  CC      arm-softmmu/memory.o
  CC      aarch64-softmmu/memory.o
  CC      cris-softmmu/numa.o
  CC      alpha-softmmu/memory_mapping.o
  CC      cris-softmmu/qtest.o
  CC      arm-softmmu/memory_mapping.o
  CC      alpha-softmmu/dump.o
  CC      cris-softmmu/memory.o
  CC      aarch64-softmmu/memory_mapping.o
  CC      arm-softmmu/dump.o
  CC      aarch64-softmmu/dump.o
  CC      alpha-softmmu/migration/ram.o
  CC      arm-softmmu/migration/ram.o
  CC      cris-softmmu/memory_mapping.o
  CC      aarch64-softmmu/migration/ram.o
  CC      alpha-softmmu/accel/accel.o
  CC      cris-softmmu/dump.o
  CC      alpha-softmmu/accel/stubs/hax-stub.o
  CC      arm-softmmu/accel/accel.o
  CC      aarch64-softmmu/accel/accel.o
  CC      alpha-softmmu/accel/stubs/kvm-stub.o
  CC      cris-softmmu/migration/ram.o
  CC      arm-softmmu/accel/stubs/hax-stub.o
  CC      aarch64-softmmu/accel/stubs/hax-stub.o
  CC      aarch64-softmmu/accel/stubs/kvm-stub.o
  CC      arm-softmmu/accel/stubs/kvm-stub.o
  CC      alpha-softmmu/accel/tcg/tcg-all.o
  CC      arm-softmmu/accel/tcg/tcg-all.o
  CC      aarch64-softmmu/accel/tcg/tcg-all.o
  CC      alpha-softmmu/accel/tcg/cputlb.o
  CC      aarch64-softmmu/accel/tcg/cputlb.o
  CC      arm-softmmu/accel/tcg/cputlb.o
  CC      cris-softmmu/accel/accel.o
  CC      cris-softmmu/accel/stubs/hax-stub.o
  CC      alpha-softmmu/accel/tcg/tcg-runtime.o
  CC      alpha-softmmu/accel/tcg/cpu-exec.o
  CC      cris-softmmu/accel/stubs/kvm-stub.o
  CC      arm-softmmu/accel/tcg/tcg-runtime.o
  CC      cris-softmmu/accel/tcg/tcg-all.o
  CC      alpha-softmmu/accel/tcg/cpu-exec-common.o
  CC      aarch64-softmmu/accel/tcg/tcg-runtime.o
  CC      alpha-softmmu/accel/tcg/translate-all.o
  CC      arm-softmmu/accel/tcg/cpu-exec.o
  CC      cris-softmmu/accel/tcg/cputlb.o
  CC      aarch64-softmmu/accel/tcg/cpu-exec.o
  CC      alpha-softmmu/accel/tcg/translator.o
  CC      arm-softmmu/accel/tcg/cpu-exec-common.o
  CC      alpha-softmmu/hw/9pfs/virtio-9p-device.o
  CC      arm-softmmu/accel/tcg/translate-all.o
  CC      alpha-softmmu/hw/block/virtio-blk.o
  CC      aarch64-softmmu/accel/tcg/cpu-exec-common.o
  CC      cris-softmmu/accel/tcg/tcg-runtime.o
  CC      aarch64-softmmu/accel/tcg/translate-all.o
  CC      alpha-softmmu/hw/block/dataplane/virtio-blk.o
  CC      cris-softmmu/accel/tcg/cpu-exec.o
  CC      arm-softmmu/accel/tcg/translator.o
  CC      alpha-softmmu/hw/char/virtio-serial-bus.o
  CC      cris-softmmu/accel/tcg/cpu-exec-common.o
  CC      aarch64-softmmu/accel/tcg/translator.o
  CC      arm-softmmu/hw/9pfs/virtio-9p-device.o
  CC      cris-softmmu/accel/tcg/translate-all.o
  CC      arm-softmmu/hw/adc/stm32f2xx_adc.o
  CC      alpha-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/hw/9pfs/virtio-9p-device.o
  CC      arm-softmmu/hw/block/virtio-blk.o
  CC      aarch64-softmmu/hw/adc/stm32f2xx_adc.o
  CC      cris-softmmu/accel/tcg/translator.o
  CC      alpha-softmmu/hw/core/null-machine.o
  CC      aarch64-softmmu/hw/block/virtio-blk.o
  CC      cris-softmmu/hw/core/generic-loader.o
  CC      alpha-softmmu/hw/display/vga.o
  CC      arm-softmmu/hw/block/dataplane/virtio-blk.o
  CC      cris-softmmu/hw/core/null-machine.o
  CC      aarch64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      arm-softmmu/hw/char/exynos4210_uart.o
  CC      cris-softmmu/hw/misc/mmio_interface.o
  CC      aarch64-softmmu/hw/char/exynos4210_uart.o
  CC      arm-softmmu/hw/char/omap_uart.o
  CC      alpha-softmmu/hw/display/virtio-gpu.o
  CC      cris-softmmu/hw/net/etraxfs_eth.o
  CC      aarch64-softmmu/hw/char/omap_uart.o
  CC      arm-softmmu/hw/char/digic-uart.o
  CC      cris-softmmu/hw/net/vhost_net.o
  CC      aarch64-softmmu/hw/char/digic-uart.o
  CC      arm-softmmu/hw/char/stm32f2xx_usart.o
  CC      cris-softmmu/hw/net/rocker/qmp-norocker.o
  CC      aarch64-softmmu/hw/char/stm32f2xx_usart.o
  CC      alpha-softmmu/hw/display/virtio-gpu-3d.o
  CC      cris-softmmu/hw/vfio/common.o
  CC      arm-softmmu/hw/char/bcm2835_aux.o
  CC      aarch64-softmmu/hw/char/bcm2835_aux.o
  CC      arm-softmmu/hw/char/virtio-serial-bus.o
  CC      alpha-softmmu/hw/display/virtio-gpu-pci.o
  CC      aarch64-softmmu/hw/char/virtio-serial-bus.o
  CC      cris-softmmu/hw/vfio/platform.o
  CC      arm-softmmu/hw/core/generic-loader.o
  CC      alpha-softmmu/hw/misc/ivshmem.o
  CC      aarch64-softmmu/hw/core/generic-loader.o
  CC      cris-softmmu/hw/vfio/spapr.o
  CC      arm-softmmu/hw/core/null-machine.o
  CC      alpha-softmmu/hw/misc/mmio_interface.o
  CC      arm-softmmu/hw/cpu/arm11mpcore.o
  CC      cris-softmmu/hw/cris/boot.o
  CC      aarch64-softmmu/hw/core/null-machine.o
  CC      alpha-softmmu/hw/net/virtio-net.o
  CC      cris-softmmu/hw/cris/axis_dev88.o
  CC      arm-softmmu/hw/cpu/realview_mpcore.o
  CC      aarch64-softmmu/hw/cpu/arm11mpcore.o
  CC      arm-softmmu/hw/cpu/a9mpcore.o
  CC      cris-softmmu/target/cris/translate.o
  CC      aarch64-softmmu/hw/cpu/realview_mpcore.o
  CC      arm-softmmu/hw/cpu/a15mpcore.o
  CC      alpha-softmmu/hw/net/vhost_net.o
  CC      arm-softmmu/hw/display/omap_dss.o
  CC      aarch64-softmmu/hw/cpu/a9mpcore.o
  CC      alpha-softmmu/hw/scsi/virtio-scsi.o
  CC      aarch64-softmmu/hw/cpu/a15mpcore.o
  CC      arm-softmmu/hw/display/omap_lcdc.o
  CC      alpha-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      aarch64-softmmu/hw/display/omap_dss.o
  CC      arm-softmmu/hw/display/pxa2xx_lcd.o
  CC      alpha-softmmu/hw/scsi/vhost-scsi-common.o
  CC      cris-softmmu/target/cris/op_helper.o
  CC      aarch64-softmmu/hw/display/omap_lcdc.o
  CC      alpha-softmmu/hw/scsi/vhost-scsi.o
  CC      cris-softmmu/target/cris/helper.o
  CC      alpha-softmmu/hw/scsi/vhost-user-scsi.o
  CC      aarch64-softmmu/hw/display/pxa2xx_lcd.o
  CC      cris-softmmu/target/cris/cpu.o
  CC      alpha-softmmu/hw/timer/mc146818rtc.o
  CC      arm-softmmu/hw/display/bcm2835_fb.o
  CC      cris-softmmu/target/cris/gdbstub.o
  CC      alpha-softmmu/hw/vfio/common.o
  CC      cris-softmmu/target/cris/mmu.o
  CC      arm-softmmu/hw/display/vga.o
  CC      cris-softmmu/target/cris/machine.o
  CC      alpha-softmmu/hw/vfio/pci.o
  CC      aarch64-softmmu/hw/display/bcm2835_fb.o
  GEN     trace/generated-helpers.c
  CC      cris-softmmu/trace/control-target.o
  CC      cris-softmmu/trace/generated-helpers.o
  CC      aarch64-softmmu/hw/display/vga.o
  CC      arm-softmmu/hw/display/virtio-gpu.o
  LINK    cris-softmmu/qemu-system-cris
  CC      alpha-softmmu/hw/vfio/pci-quirks.o
  CC      arm-softmmu/hw/display/virtio-gpu-3d.o
  CC      aarch64-softmmu/hw/display/virtio-gpu.o
  CC      alpha-softmmu/hw/vfio/platform.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-3d.o
  CC      arm-softmmu/hw/display/virtio-gpu-pci.o
  CC      alpha-softmmu/hw/vfio/spapr.o
  CC      alpha-softmmu/hw/virtio/virtio.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-pci.o
  CC      arm-softmmu/hw/dma/omap_dma.o
  CC      aarch64-softmmu/hw/display/dpcd.o
  CC      arm-softmmu/hw/dma/soc_dma.o
  CC      aarch64-softmmu/hw/display/xlnx_dp.o
  CC      arm-softmmu/hw/dma/pxa2xx_dma.o
  CC      alpha-softmmu/hw/virtio/virtio-balloon.o
  GEN     i386-softmmu/hmp-commands.h
  GEN     i386-softmmu/hmp-commands-info.h
  GEN     i386-softmmu/config-target.h
  CC      i386-softmmu/exec.o
  CC      aarch64-softmmu/hw/dma/xlnx_dpdma.o
  CC      arm-softmmu/hw/dma/bcm2835_dma.o
  CC      alpha-softmmu/hw/virtio/vhost.o
  CC      arm-softmmu/hw/gpio/omap_gpio.o
  CC      aarch64-softmmu/hw/dma/omap_dma.o
  CC      alpha-softmmu/hw/virtio/vhost-backend.o
  CC      arm-softmmu/hw/gpio/imx_gpio.o
  CC      alpha-softmmu/hw/virtio/vhost-user.o
  CC      aarch64-softmmu/hw/dma/soc_dma.o
  CC      arm-softmmu/hw/gpio/bcm2835_gpio.o
  CC      alpha-softmmu/hw/virtio/vhost-vsock.o
  CC      aarch64-softmmu/hw/dma/pxa2xx_dma.o
  CC      i386-softmmu/tcg/tcg.o
  CC      alpha-softmmu/hw/virtio/virtio-crypto.o
  CC      arm-softmmu/hw/i2c/omap_i2c.o
  CC      aarch64-softmmu/hw/dma/bcm2835_dma.o
  CC      alpha-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      arm-softmmu/hw/input/pxa2xx_keypad.o
  CC      aarch64-softmmu/hw/gpio/omap_gpio.o
  CC      alpha-softmmu/hw/alpha/dp264.o
  CC      arm-softmmu/hw/input/tsc210x.o
  CC      aarch64-softmmu/hw/gpio/imx_gpio.o
  CC      alpha-softmmu/hw/alpha/pci.o
  CC      aarch64-softmmu/hw/gpio/bcm2835_gpio.o
  CC      arm-softmmu/hw/intc/armv7m_nvic.o
  CC      alpha-softmmu/hw/alpha/typhoon.o
  CC      aarch64-softmmu/hw/i2c/omap_i2c.o
  CC      i386-softmmu/tcg/tcg-op.o
  CC      alpha-softmmu/target/alpha/machine.o
  CC      arm-softmmu/hw/intc/exynos4210_gic.o
  CC      aarch64-softmmu/hw/input/pxa2xx_keypad.o
  CC      alpha-softmmu/target/alpha/translate.o
  CC      arm-softmmu/hw/intc/exynos4210_combiner.o
  CC      arm-softmmu/hw/intc/omap_intc.o
  CC      aarch64-softmmu/hw/input/tsc210x.o
  CC      alpha-softmmu/target/alpha/helper.o
  CC      arm-softmmu/hw/intc/bcm2835_ic.o
  CC      aarch64-softmmu/hw/intc/armv7m_nvic.o
  CC      arm-softmmu/hw/intc/bcm2836_control.o
  CC      alpha-softmmu/target/alpha/cpu.o
  CC      alpha-softmmu/target/alpha/int_helper.o
  CC      arm-softmmu/hw/intc/allwinner-a10-pic.o
  CC      i386-softmmu/tcg/optimize.o
  CC      arm-softmmu/hw/intc/aspeed_vic.o
  CC      aarch64-softmmu/hw/intc/exynos4210_gic.o
  CC      alpha-softmmu/target/alpha/fpu_helper.o
  CC      alpha-softmmu/target/alpha/vax_helper.o
  CC      arm-softmmu/hw/intc/arm_gicv3_cpuif.o
  CC      aarch64-softmmu/hw/intc/exynos4210_combiner.o
  CC      alpha-softmmu/target/alpha/sys_helper.o
  CC      alpha-softmmu/target/alpha/mem_helper.o
  CC      i386-softmmu/tcg/tcg-common.o
  CC      aarch64-softmmu/hw/intc/omap_intc.o
  CC      alpha-softmmu/target/alpha/gdbstub.o
  CC      i386-softmmu/fpu/softfloat.o
  CC      aarch64-softmmu/hw/intc/bcm2835_ic.o
  GEN     trace/generated-helpers.c
  CC      arm-softmmu/hw/misc/ivshmem.o
  CC      alpha-softmmu/trace/control-target.o
  CC      aarch64-softmmu/hw/intc/bcm2836_control.o
  CC      alpha-softmmu/trace/generated-helpers.o
  CC      arm-softmmu/hw/misc/arm_sysctl.o
  CC      aarch64-softmmu/hw/intc/allwinner-a10-pic.o
  LINK    alpha-softmmu/qemu-system-alpha
  CC      arm-softmmu/hw/misc/cbus.o
  CC      aarch64-softmmu/hw/intc/aspeed_vic.o
  CC      arm-softmmu/hw/misc/exynos4210_pmu.o
  CC      aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
  CC      arm-softmmu/hw/misc/exynos4210_clk.o
  CC      arm-softmmu/hw/misc/exynos4210_rng.o
  CC      arm-softmmu/hw/misc/imx_ccm.o
  CC      i386-softmmu/disas.o
  GEN     lm32-softmmu/hmp-commands.h
  CC      arm-softmmu/hw/misc/imx31_ccm.o
  CC      aarch64-softmmu/hw/misc/ivshmem.o
  GEN     lm32-softmmu/hmp-commands-info.h
  GEN     lm32-softmmu/config-target.h
  CC      lm32-softmmu/exec.o
  GEN     i386-softmmu/gdbstub-xml.c
  CC      arm-softmmu/hw/misc/imx25_ccm.o
  CC      arm-softmmu/hw/misc/imx6_ccm.o
  CC      aarch64-softmmu/hw/misc/arm_sysctl.o
  CC      i386-softmmu/arch_init.o
  CC      arm-softmmu/hw/misc/imx6_src.o
  CC      aarch64-softmmu/hw/misc/cbus.o
  CC      arm-softmmu/hw/misc/mst_fpga.o
  CC      i386-softmmu/cpus.o
  CC      aarch64-softmmu/hw/misc/exynos4210_pmu.o
  CC      arm-softmmu/hw/misc/omap_clk.o
  CC      aarch64-softmmu/hw/misc/exynos4210_clk.o
  CC      arm-softmmu/hw/misc/omap_gpmc.o
  CC      i386-softmmu/monitor.o
  CC      aarch64-softmmu/hw/misc/exynos4210_rng.o
  CC      lm32-softmmu/tcg/tcg.o
  CC      arm-softmmu/hw/misc/omap_l4.o
  CC      aarch64-softmmu/hw/misc/imx_ccm.o
  CC      arm-softmmu/hw/misc/omap_sdrc.o
  CC      aarch64-softmmu/hw/misc/imx31_ccm.o
  CC      i386-softmmu/gdbstub.o
  CC      arm-softmmu/hw/misc/omap_tap.o
  CC      aarch64-softmmu/hw/misc/imx25_ccm.o
  CC      arm-softmmu/hw/misc/bcm2835_mbox.o
  CC      aarch64-softmmu/hw/misc/imx6_ccm.o
  CC      i386-softmmu/balloon.o
  CC      arm-softmmu/hw/misc/bcm2835_property.o
  CC      aarch64-softmmu/hw/misc/imx6_src.o
  CC      i386-softmmu/ioport.o
  CC      lm32-softmmu/tcg/tcg-op.o
  CC      aarch64-softmmu/hw/misc/mst_fpga.o
  CC      arm-softmmu/hw/misc/bcm2835_rng.o
  CC      i386-softmmu/numa.o
  CC      aarch64-softmmu/hw/misc/omap_clk.o
  CC      arm-softmmu/hw/misc/zynq_slcr.o
  CC      aarch64-softmmu/hw/misc/omap_gpmc.o
  CC      arm-softmmu/hw/misc/zynq-xadc.o
  CC      i386-softmmu/qtest.o
  CC      arm-softmmu/hw/misc/stm32f2xx_syscfg.o
  CC      aarch64-softmmu/hw/misc/omap_l4.o
  CC      i386-softmmu/memory.o
  CC      arm-softmmu/hw/misc/mps2-scc.o
  CC      aarch64-softmmu/hw/misc/omap_sdrc.o
  CC      arm-softmmu/hw/misc/aspeed_scu.o
  CC      aarch64-softmmu/hw/misc/omap_tap.o
  CC      lm32-softmmu/tcg/optimize.o
  CC      arm-softmmu/hw/misc/aspeed_sdmc.o
  CC      aarch64-softmmu/hw/misc/bcm2835_mbox.o
  CC      arm-softmmu/hw/misc/mmio_interface.o
  CC      aarch64-softmmu/hw/misc/bcm2835_property.o
  CC      i386-softmmu/memory_mapping.o
  CC      arm-softmmu/hw/misc/msf2-sysreg.o
  CC      lm32-softmmu/tcg/tcg-common.o
  CC      i386-softmmu/dump.o
  CC      arm-softmmu/hw/net/virtio-net.o
  CC      aarch64-softmmu/hw/misc/bcm2835_rng.o
  CC      aarch64-softmmu/hw/misc/zynq_slcr.o
  CC      lm32-softmmu/fpu/softfloat.o
  CC      aarch64-softmmu/hw/misc/zynq-xadc.o
  CC      arm-softmmu/hw/net/vhost_net.o
  CC      i386-softmmu/migration/ram.o
  CC      aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
  CC      arm-softmmu/hw/pcmcia/pxa2xx.o
  CC      aarch64-softmmu/hw/misc/mps2-scc.o
  CC      i386-softmmu/accel/accel.o
  CC      arm-softmmu/hw/scsi/virtio-scsi.o
  CC      aarch64-softmmu/hw/misc/auxbus.o
  CC      i386-softmmu/accel/stubs/hax-stub.o
  CC      i386-softmmu/accel/stubs/kvm-stub.o
  CC      arm-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      aarch64-softmmu/hw/misc/aspeed_scu.o
  CC      i386-softmmu/accel/tcg/tcg-all.o
  CC      arm-softmmu/hw/scsi/vhost-scsi-common.o
  CC      aarch64-softmmu/hw/misc/aspeed_sdmc.o
  CC      i386-softmmu/accel/tcg/cputlb.o
  CC      arm-softmmu/hw/scsi/vhost-scsi.o
  CC      aarch64-softmmu/hw/misc/mmio_interface.o
  CC      lm32-softmmu/disas.o
  CC      arm-softmmu/hw/scsi/vhost-user-scsi.o
  CC      aarch64-softmmu/hw/misc/msf2-sysreg.o
  CC      lm32-softmmu/arch_init.o
  CC      arm-softmmu/hw/sd/omap_mmc.o
  CC      aarch64-softmmu/hw/net/virtio-net.o
  CC      arm-softmmu/hw/sd/pxa2xx_mmci.o
  CC      lm32-softmmu/cpus.o
  CC      arm-softmmu/hw/sd/bcm2835_sdhost.o
  CC      i386-softmmu/accel/tcg/tcg-runtime.o
  CC      aarch64-softmmu/hw/net/vhost_net.o
  CC      arm-softmmu/hw/ssi/omap_spi.o
  CC      lm32-softmmu/monitor.o
  CC      aarch64-softmmu/hw/pcmcia/pxa2xx.o
  CC      arm-softmmu/hw/ssi/imx_spi.o
  CC      i386-softmmu/accel/tcg/cpu-exec.o
  CC      arm-softmmu/hw/timer/exynos4210_mct.o
  CC      aarch64-softmmu/hw/scsi/virtio-scsi.o
  CC      i386-softmmu/accel/tcg/cpu-exec-common.o
  CC      arm-softmmu/hw/timer/exynos4210_pwm.o
  CC      i386-softmmu/accel/tcg/translate-all.o
  CC      aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      arm-softmmu/hw/timer/exynos4210_rtc.o
  CC      lm32-softmmu/gdbstub.o
  CC      aarch64-softmmu/hw/scsi/vhost-scsi-common.o
  CC      arm-softmmu/hw/timer/omap_gptimer.o
  CC      i386-softmmu/accel/tcg/translator.o
  CC      aarch64-softmmu/hw/scsi/vhost-scsi.o
  CC      lm32-softmmu/balloon.o
  CC      arm-softmmu/hw/timer/omap_synctimer.o
  CC      aarch64-softmmu/hw/scsi/vhost-user-scsi.o
  CC      i386-softmmu/hw/9pfs/virtio-9p-device.o
  CC      lm32-softmmu/ioport.o
  CC      arm-softmmu/hw/timer/pxa2xx_timer.o
  CC      aarch64-softmmu/hw/sd/omap_mmc.o
  CC      i386-softmmu/hw/block/virtio-blk.o
  CC      arm-softmmu/hw/timer/digic-timer.o
  CC      lm32-softmmu/numa.o
  CC      aarch64-softmmu/hw/sd/pxa2xx_mmci.o
  CC      arm-softmmu/hw/timer/allwinner-a10-pit.o
  CC      i386-softmmu/hw/block/dataplane/virtio-blk.o
  CC      aarch64-softmmu/hw/sd/bcm2835_sdhost.o
  CC      arm-softmmu/hw/usb/tusb6010.o
  CC      lm32-softmmu/qtest.o
  CC      aarch64-softmmu/hw/ssi/omap_spi.o
  CC      i386-softmmu/hw/char/virtio-serial-bus.o
  CC      aarch64-softmmu/hw/ssi/imx_spi.o
  CC      arm-softmmu/hw/vfio/common.o
  CC      aarch64-softmmu/hw/timer/exynos4210_mct.o
  CC      lm32-softmmu/memory.o
  CC      aarch64-softmmu/hw/timer/exynos4210_pwm.o
  CC      arm-softmmu/hw/vfio/pci.o
  CC      i386-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/hw/timer/exynos4210_rtc.o
  CC      i386-softmmu/hw/core/null-machine.o
  CC      aarch64-softmmu/hw/timer/omap_gptimer.o
  CC      i386-softmmu/hw/display/vga.o
  CC      arm-softmmu/hw/vfio/pci-quirks.o
  CC      lm32-softmmu/memory_mapping.o
  CC      aarch64-softmmu/hw/timer/omap_synctimer.o
  CC      lm32-softmmu/dump.o
  CC      aarch64-softmmu/hw/timer/pxa2xx_timer.o
  CC      arm-softmmu/hw/vfio/platform.o
  CC      arm-softmmu/hw/vfio/calxeda-xgmac.o
  CC      i386-softmmu/hw/display/virtio-gpu.o
  CC      aarch64-softmmu/hw/timer/digic-timer.o
  CC      lm32-softmmu/migration/ram.o
  CC      arm-softmmu/hw/vfio/amd-xgbe.o
  CC      aarch64-softmmu/hw/timer/allwinner-a10-pit.o
  CC      arm-softmmu/hw/vfio/spapr.o
  CC      arm-softmmu/hw/virtio/virtio.o
  CC      aarch64-softmmu/hw/usb/tusb6010.o
  CC      i386-softmmu/hw/display/virtio-gpu-3d.o
  CC      lm32-softmmu/accel/accel.o
  CC      aarch64-softmmu/hw/vfio/common.o
  CC      lm32-softmmu/accel/stubs/hax-stub.o
  CC      i386-softmmu/hw/display/virtio-gpu-pci.o
  CC      arm-softmmu/hw/virtio/virtio-balloon.o
  CC      lm32-softmmu/accel/stubs/kvm-stub.o
  CC      i386-softmmu/hw/display/virtio-vga.o
  CC      lm32-softmmu/accel/tcg/tcg-all.o
  CC      aarch64-softmmu/hw/vfio/pci.o
  CC      arm-softmmu/hw/virtio/vhost.o
  CC      lm32-softmmu/accel/tcg/cputlb.o
  CC      i386-softmmu/hw/intc/apic.o
  CC      arm-softmmu/hw/virtio/vhost-backend.o
  CC      aarch64-softmmu/hw/vfio/pci-quirks.o
  CC      i386-softmmu/hw/intc/apic_common.o
  CC      arm-softmmu/hw/virtio/vhost-user.o
  CC      aarch64-softmmu/hw/vfio/platform.o
  CC      i386-softmmu/hw/intc/ioapic.o
  CC      lm32-softmmu/accel/tcg/tcg-runtime.o
  CC      arm-softmmu/hw/virtio/vhost-vsock.o
  CC      aarch64-softmmu/hw/vfio/calxeda-xgmac.o
  CC      lm32-softmmu/accel/tcg/cpu-exec.o
  CC      arm-softmmu/hw/virtio/virtio-crypto.o
  CC      i386-softmmu/hw/isa/lpc_ich9.o
  CC      aarch64-softmmu/hw/vfio/amd-xgbe.o
  CC      lm32-softmmu/accel/tcg/cpu-exec-common.o
  CC      arm-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      aarch64-softmmu/hw/vfio/spapr.o
  CC      i386-softmmu/hw/misc/vmport.o
  CC      lm32-softmmu/accel/tcg/translate-all.o
  CC      arm-softmmu/hw/arm/boot.o
  CC      aarch64-softmmu/hw/virtio/virtio.o
  CC      i386-softmmu/hw/misc/ivshmem.o
  CC      lm32-softmmu/accel/tcg/translator.o
  CC      arm-softmmu/hw/arm/collie.o
  CC      i386-softmmu/hw/misc/pvpanic.o
  CC      lm32-softmmu/hw/core/generic-loader.o
  CC      i386-softmmu/hw/misc/mmio_interface.o
  CC      arm-softmmu/hw/arm/exynos4_boards.o
  CC      i386-softmmu/hw/net/virtio-net.o
  CC      aarch64-softmmu/hw/virtio/virtio-balloon.o
  CC      lm32-softmmu/hw/core/null-machine.o
  CC      arm-softmmu/hw/arm/gumstix.o
  CC      lm32-softmmu/hw/input/milkymist-softusb.o
  CC      aarch64-softmmu/hw/virtio/vhost.o
  CC      i386-softmmu/hw/net/vhost_net.o
  CC      lm32-softmmu/hw/misc/milkymist-hpdmc.o
  CC      arm-softmmu/hw/arm/highbank.o
  CC      lm32-softmmu/hw/misc/milkymist-pfpu.o
  CC      aarch64-softmmu/hw/virtio/vhost-backend.o
  CC      i386-softmmu/hw/scsi/virtio-scsi.o
  CC      arm-softmmu/hw/arm/digic_boards.o
  CC      aarch64-softmmu/hw/virtio/vhost-user.o
  CC      lm32-softmmu/hw/misc/mmio_interface.o
  CC      arm-softmmu/hw/arm/integratorcp.o
  CC      i386-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      lm32-softmmu/hw/net/milkymist-minimac2.o
  CC      aarch64-softmmu/hw/virtio/vhost-vsock.o
  CC      i386-softmmu/hw/scsi/vhost-scsi-common.o
  CC      arm-softmmu/hw/arm/mainstone.o
  CC      aarch64-softmmu/hw/virtio/virtio-crypto.o
  CC      lm32-softmmu/hw/net/vhost_net.o
  CC      i386-softmmu/hw/scsi/vhost-scsi.o
  CC      arm-softmmu/hw/arm/musicpal.o
  CC      lm32-softmmu/hw/net/rocker/qmp-norocker.o
  CC      i386-softmmu/hw/scsi/vhost-user-scsi.o
  CC      aarch64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      lm32-softmmu/hw/sd/milkymist-memcard.o
  CC      i386-softmmu/hw/timer/mc146818rtc.o
  CC      aarch64-softmmu/hw/arm/boot.o
  CC      arm-softmmu/hw/arm/nseries.o
  CC      lm32-softmmu/hw/vfio/common.o
  CC      i386-softmmu/hw/vfio/common.o
  CC      aarch64-softmmu/hw/arm/collie.o
  CC      aarch64-softmmu/hw/arm/exynos4_boards.o
  CC      arm-softmmu/hw/arm/omap_sx1.o
  CC      aarch64-softmmu/hw/arm/gumstix.o
  CC      lm32-softmmu/hw/vfio/platform.o
  CC      i386-softmmu/hw/vfio/pci.o
  CC      arm-softmmu/hw/arm/palm.o
  CC      aarch64-softmmu/hw/arm/highbank.o
  CC      lm32-softmmu/hw/vfio/spapr.o
  CC      aarch64-softmmu/hw/arm/digic_boards.o
  CC      lm32-softmmu/hw/lm32/lm32_boards.o
  CC      arm-softmmu/hw/arm/realview.o
  CC      lm32-softmmu/hw/lm32/milkymist.o
  CC      aarch64-softmmu/hw/arm/integratorcp.o
  CC      i386-softmmu/hw/vfio/pci-quirks.o
  CC      lm32-softmmu/target/lm32/translate.o
  CC      arm-softmmu/hw/arm/spitz.o
  CC      aarch64-softmmu/hw/arm/mainstone.o
  CC      aarch64-softmmu/hw/arm/musicpal.o
  CC      i386-softmmu/hw/vfio/platform.o
  CC      arm-softmmu/hw/arm/stellaris.o
  CC      lm32-softmmu/target/lm32/op_helper.o
  CC      i386-softmmu/hw/vfio/spapr.o
  CC      aarch64-softmmu/hw/arm/nseries.o
  CC      arm-softmmu/hw/arm/tosa.o
  CC      i386-softmmu/hw/virtio/virtio.o
  CC      lm32-softmmu/target/lm32/helper.o
  CC      arm-softmmu/hw/arm/versatilepb.o
  CC      lm32-softmmu/target/lm32/cpu.o
  CC      aarch64-softmmu/hw/arm/omap_sx1.o
  CC      arm-softmmu/hw/arm/vexpress.o
  CC      lm32-softmmu/target/lm32/gdbstub.o
  CC      aarch64-softmmu/hw/arm/palm.o
  CC      i386-softmmu/hw/virtio/virtio-balloon.o
  CC      lm32-softmmu/target/lm32/lm32-semi.o
  CC      arm-softmmu/hw/arm/virt.o
  CC      lm32-softmmu/target/lm32/machine.o
  CC      i386-softmmu/hw/virtio/vhost.o
  CC      aarch64-softmmu/hw/arm/realview.o
  GEN     trace/generated-helpers.c
  CC      lm32-softmmu/trace/control-target.o
  CC      aarch64-softmmu/hw/arm/spitz.o
  CC      i386-softmmu/hw/virtio/vhost-backend.o
  CC      arm-softmmu/hw/arm/xilinx_zynq.o
  CC      lm32-softmmu/trace/generated-helpers.o
  CC      i386-softmmu/hw/virtio/vhost-user.o
  LINK    lm32-softmmu/qemu-system-lm32
  CC      aarch64-softmmu/hw/arm/stellaris.o
  CC      arm-softmmu/hw/arm/z2.o
  CC      i386-softmmu/hw/virtio/vhost-vsock.o
  CC      arm-softmmu/hw/arm/virt-acpi-build.o
  CC      aarch64-softmmu/hw/arm/tosa.o
  CC      i386-softmmu/hw/virtio/virtio-crypto.o
  CC      aarch64-softmmu/hw/arm/versatilepb.o
  CC      arm-softmmu/hw/arm/netduino2.o
  CC      i386-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      i386-softmmu/hw/i386/multiboot.o
  CC      aarch64-softmmu/hw/arm/vexpress.o
  CC      arm-softmmu/hw/arm/sysbus-fdt.o
  CC      aarch64-softmmu/hw/arm/virt.o
  CC      arm-softmmu/hw/arm/armv7m.o
  CC      i386-softmmu/hw/i386/pc.o
  CC      arm-softmmu/hw/arm/exynos4210.o
  CC      aarch64-softmmu/hw/arm/xilinx_zynq.o
  CC      arm-softmmu/hw/arm/pxa2xx.o
  CC      aarch64-softmmu/hw/arm/z2.o
  GEN     m68k-softmmu/hmp-commands.h
  GEN     m68k-softmmu/hmp-commands-info.h
  GEN     m68k-softmmu/config-target.h
  CC      i386-softmmu/hw/i386/pc_piix.o
  CC      m68k-softmmu/exec.o
  CC      arm-softmmu/hw/arm/pxa2xx_gpio.o
  CC      aarch64-softmmu/hw/arm/virt-acpi-build.o
  CC      arm-softmmu/hw/arm/pxa2xx_pic.o
  CC      i386-softmmu/hw/i386/pc_q35.o
  CC      aarch64-softmmu/hw/arm/netduino2.o
  CC      arm-softmmu/hw/arm/digic.o
  CC      arm-softmmu/hw/arm/omap1.o
  CC      aarch64-softmmu/hw/arm/sysbus-fdt.o
  CC      i386-softmmu/hw/i386/pc_sysfw.o
  CC      aarch64-softmmu/hw/arm/armv7m.o
  CC      i386-softmmu/hw/i386/x86-iommu.o
  CC      aarch64-softmmu/hw/arm/exynos4210.o
  CC      i386-softmmu/hw/i386/intel_iommu.o
  CC      m68k-softmmu/tcg/tcg.o
  CC      aarch64-softmmu/hw/arm/pxa2xx.o
  CC      arm-softmmu/hw/arm/omap2.o
  CC      arm-softmmu/hw/arm/strongarm.o
  CC      i386-softmmu/hw/i386/amd_iommu.o
  CC      aarch64-softmmu/hw/arm/pxa2xx_gpio.o
  CC      arm-softmmu/hw/arm/allwinner-a10.o
  CC      aarch64-softmmu/hw/arm/pxa2xx_pic.o
  CC      i386-softmmu/hw/i386/kvmvapic.o
  CC      arm-softmmu/hw/arm/cubieboard.o
  CC      aarch64-softmmu/hw/arm/digic.o
  CC      m68k-softmmu/tcg/tcg-op.o
  CC      i386-softmmu/hw/i386/acpi-build.o
  CC      arm-softmmu/hw/arm/bcm2835_peripherals.o
  CC      aarch64-softmmu/hw/arm/omap1.o
  CC      arm-softmmu/hw/arm/bcm2836.o
  CC      i386-softmmu/hw/i386/pci-assign-load-rom.o
  CC      arm-softmmu/hw/arm/raspi.o
  CC      i386-softmmu/target/i386/helper.o
  CC      aarch64-softmmu/hw/arm/omap2.o
  CC      arm-softmmu/hw/arm/stm32f205_soc.o
  CC      aarch64-softmmu/hw/arm/strongarm.o
  CC      i386-softmmu/target/i386/cpu.o
  CC      arm-softmmu/hw/arm/fsl-imx25.o
  CC      m68k-softmmu/tcg/optimize.o
  CC      aarch64-softmmu/hw/arm/allwinner-a10.o
  CC      arm-softmmu/hw/arm/imx25_pdk.o
  CC      arm-softmmu/hw/arm/fsl-imx31.o
  CC      i386-softmmu/target/i386/gdbstub.o
  CC      aarch64-softmmu/hw/arm/cubieboard.o
  CC      m68k-softmmu/tcg/tcg-common.o
  CC      arm-softmmu/hw/arm/kzm.o
  CC      i386-softmmu/target/i386/xsave_helper.o
  CC      aarch64-softmmu/hw/arm/bcm2835_peripherals.o
  CC      arm-softmmu/hw/arm/fsl-imx6.o
  CC      m68k-softmmu/fpu/softfloat.o
  CC      i386-softmmu/target/i386/translate.o
  CC      aarch64-softmmu/hw/arm/bcm2836.o
  CC      arm-softmmu/hw/arm/sabrelite.o
  CC      arm-softmmu/hw/arm/aspeed_soc.o
  CC      aarch64-softmmu/hw/arm/raspi.o
  CC      aarch64-softmmu/hw/arm/stm32f205_soc.o
  CC      arm-softmmu/hw/arm/aspeed.o
  CC      aarch64-softmmu/hw/arm/xlnx-zynqmp.o
  CC      arm-softmmu/hw/arm/mps2.o
  CC      arm-softmmu/hw/arm/msf2-soc.o
  CC      aarch64-softmmu/hw/arm/xlnx-zcu102.o
  CC      m68k-softmmu/disas.o
  GEN     m68k-softmmu/gdbstub-xml.c
  CC      arm-softmmu/hw/arm/msf2-som.o
  CC      arm-softmmu/target/arm/arm-semi.o
  CC      m68k-softmmu/arch_init.o
  CC      aarch64-softmmu/hw/arm/fsl-imx25.o
  CC      m68k-softmmu/cpus.o
  CC      aarch64-softmmu/hw/arm/imx25_pdk.o
  CC      aarch64-softmmu/hw/arm/fsl-imx31.o
  CC      arm-softmmu/target/arm/machine.o
  CC      aarch64-softmmu/hw/arm/kzm.o
  CC      m68k-softmmu/monitor.o
  CC      arm-softmmu/target/arm/psci.o
  CC      aarch64-softmmu/hw/arm/fsl-imx6.o
  CC      arm-softmmu/target/arm/arch_dump.o
  CC      aarch64-softmmu/hw/arm/sabrelite.o
  CC      aarch64-softmmu/hw/arm/aspeed_soc.o
  CC      aarch64-softmmu/hw/arm/aspeed.o
  CC      arm-softmmu/target/arm/monitor.o
  CC      i386-softmmu/target/i386/bpt_helper.o
  CC      m68k-softmmu/gdbstub.o
  CC      aarch64-softmmu/hw/arm/mps2.o
  CC      arm-softmmu/target/arm/kvm-stub.o
  CC      i386-softmmu/target/i386/cc_helper.o
  CC      aarch64-softmmu/hw/arm/msf2-soc.o
  CC      arm-softmmu/target/arm/translate.o
  CC      m68k-softmmu/balloon.o
  CC      i386-softmmu/target/i386/excp_helper.o
  CC      aarch64-softmmu/hw/arm/msf2-som.o
  CC      m68k-softmmu/ioport.o
  CC      i386-softmmu/target/i386/fpu_helper.o
  CC      aarch64-softmmu/target/arm/arm-semi.o
  CC      m68k-softmmu/numa.o
  CC      aarch64-softmmu/target/arm/machine.o
  CC      m68k-softmmu/qtest.o
  CC      aarch64-softmmu/target/arm/psci.o
  CC      m68k-softmmu/memory.o
  CC      aarch64-softmmu/target/arm/arch_dump.o
  CC      aarch64-softmmu/target/arm/monitor.o
  CC      i386-softmmu/target/i386/int_helper.o
  CC      aarch64-softmmu/target/arm/kvm-stub.o
  CC      aarch64-softmmu/target/arm/translate.o
  CC      m68k-softmmu/memory_mapping.o
  CC      m68k-softmmu/dump.o
  CC      i386-softmmu/target/i386/mem_helper.o
  CC      i386-softmmu/target/i386/misc_helper.o
  CC      m68k-softmmu/migration/ram.o
  CC      i386-softmmu/target/i386/mpx_helper.o
  CC      arm-softmmu/target/arm/op_helper.o
  CC      i386-softmmu/target/i386/seg_helper.o
  CC      m68k-softmmu/accel/accel.o
  CC      arm-softmmu/target/arm/helper.o
  CC      m68k-softmmu/accel/stubs/hax-stub.o
  CC      m68k-softmmu/accel/stubs/kvm-stub.o
  CC      i386-softmmu/target/i386/smm_helper.o
  CC      m68k-softmmu/accel/tcg/tcg-all.o
  CC      i386-softmmu/target/i386/svm_helper.o
  CC      m68k-softmmu/accel/tcg/cputlb.o
  CC      aarch64-softmmu/target/arm/op_helper.o
  CC      i386-softmmu/target/i386/machine.o
  CC      i386-softmmu/target/i386/arch_memory_mapping.o
  CC      arm-softmmu/target/arm/cpu.o
  CC      i386-softmmu/target/i386/arch_dump.o
  CC      aarch64-softmmu/target/arm/helper.o
  CC      m68k-softmmu/accel/tcg/tcg-runtime.o
  CC      i386-softmmu/target/i386/monitor.o
  CC      arm-softmmu/target/arm/neon_helper.o
  CC      m68k-softmmu/accel/tcg/cpu-exec.o
  CC      i386-softmmu/target/i386/kvm-stub.o
  CC      m68k-softmmu/accel/tcg/cpu-exec-common.o
  GEN     trace/generated-helpers.c
  CC      i386-softmmu/trace/control-target.o
  CC      m68k-softmmu/accel/tcg/translate-all.o
  CC      arm-softmmu/target/arm/iwmmxt_helper.o
  CC      i386-softmmu/gdbstub-xml.o
  CC      m68k-softmmu/accel/tcg/translator.o
  CC      i386-softmmu/trace/generated-helpers.o
  CC      m68k-softmmu/hw/char/mcf_uart.o
  CC      arm-softmmu/target/arm/gdbstub.o
  LINK    i386-softmmu/qemu-system-i386
  CC      arm-softmmu/target/arm/crypto_helper.o
  CC      m68k-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/target/arm/cpu.o
  CC      arm-softmmu/target/arm/arm-powerctl.o
  CC      m68k-softmmu/hw/core/null-machine.o
  GEN     trace/generated-helpers.c
  CC      m68k-softmmu/hw/misc/mmio_interface.o
  CC      arm-softmmu/trace/control-target.o
  CC      aarch64-softmmu/target/arm/neon_helper.o
  CC      m68k-softmmu/hw/net/mcf_fec.o
  CC      arm-softmmu/gdbstub-xml.o
  CC      m68k-softmmu/hw/net/vhost_net.o
  CC      arm-softmmu/trace/generated-helpers.o
  CC      m68k-softmmu/hw/net/rocker/qmp-norocker.o
  GEN     microblazeel-softmmu/hmp-commands.h
  LINK    arm-softmmu/qemu-system-arm
  GEN     microblazeel-softmmu/hmp-commands-info.h
  CC      m68k-softmmu/hw/vfio/common.o
  GEN     microblazeel-softmmu/config-target.h
  CC      aarch64-softmmu/target/arm/iwmmxt_helper.o
  CC      microblazeel-softmmu/exec.o
  CC      m68k-softmmu/hw/vfio/platform.o
  CC      aarch64-softmmu/target/arm/gdbstub.o
  CC      aarch64-softmmu/target/arm/cpu64.o
  CC      m68k-softmmu/hw/vfio/spapr.o
  CC      aarch64-softmmu/target/arm/translate-a64.o
  CC      microblazeel-softmmu/tcg/tcg.o
  CC      m68k-softmmu/hw/m68k/an5206.o
  CC      aarch64-softmmu/target/arm/helper-a64.o
  CC      m68k-softmmu/hw/m68k/mcf5208.o
  CC      m68k-softmmu/hw/m68k/mcf5206.o
  CC      aarch64-softmmu/target/arm/gdbstub64.o
  CC      m68k-softmmu/hw/m68k/mcf_intc.o
  CC      microblazeel-softmmu/tcg/tcg-op.o
  CC      m68k-softmmu/target/m68k/m68k-semi.o
  GEN     microblaze-softmmu/hmp-commands.h
  GEN     microblaze-softmmu/hmp-commands-info.h
  GEN     microblaze-softmmu/config-target.h
  CC      m68k-softmmu/target/m68k/translate.o
  CC      microblaze-softmmu/exec.o
  CC      aarch64-softmmu/target/arm/crypto_helper.o
  CC      aarch64-softmmu/target/arm/arm-powerctl.o
  CC      microblazeel-softmmu/tcg/optimize.o
  GEN     trace/generated-helpers.c
  CC      aarch64-softmmu/trace/control-target.o
  CC      microblaze-softmmu/tcg/tcg.o
  CC      aarch64-softmmu/gdbstub-xml.o
  CC      m68k-softmmu/target/m68k/op_helper.o
  CC      microblazeel-softmmu/tcg/tcg-common.o
  CC      m68k-softmmu/target/m68k/helper.o
  CC      aarch64-softmmu/trace/generated-helpers.o
  CC      microblazeel-softmmu/fpu/softfloat.o
  CC      m68k-softmmu/target/m68k/cpu.o
  LINK    aarch64-softmmu/qemu-system-aarch64
  CC      m68k-softmmu/target/m68k/fpu_helper.o
  CC      m68k-softmmu/target/m68k/gdbstub.o
  CC      microblaze-softmmu/tcg/tcg-op.o
  GEN     trace/generated-helpers.c
  CC      m68k-softmmu/trace/control-target.o
  CC      m68k-softmmu/gdbstub-xml.o
  CC      microblaze-softmmu/tcg/optimize.o
  CC      m68k-softmmu/trace/generated-helpers.o
  CC      microblazeel-softmmu/disas.o
  LINK    m68k-softmmu/qemu-system-m68k
  CC      microblazeel-softmmu/arch_init.o
  CC      microblazeel-softmmu/cpus.o
  CC      microblaze-softmmu/tcg/tcg-common.o
  CC      microblaze-softmmu/fpu/softfloat.o
  CC      microblazeel-softmmu/monitor.o
  CC      microblaze-softmmu/disas.o
  CC      microblaze-softmmu/arch_init.o
  CC      microblazeel-softmmu/gdbstub.o
  GEN     mips64el-softmmu/hmp-commands.h
  GEN     mips64el-softmmu/hmp-commands-info.h
  GEN     mips64el-softmmu/config-target.h
  CC      mips64el-softmmu/exec.o
  GEN     mips64-softmmu/hmp-commands.h
  CC      microblazeel-softmmu/balloon.o
  GEN     mips64-softmmu/hmp-commands-info.h
  GEN     mips64-softmmu/config-target.h
  CC      mips64-softmmu/exec.o
  CC      microblazeel-softmmu/ioport.o
  CC      microblaze-softmmu/cpus.o
  CC      microblazeel-softmmu/numa.o
  CC      microblaze-softmmu/monitor.o
  CC      mips64el-softmmu/tcg/tcg.o
  CC      microblazeel-softmmu/qtest.o
  CC      mips64-softmmu/tcg/tcg.o
  CC      microblazeel-softmmu/memory.o
  CC      microblaze-softmmu/gdbstub.o
  CC      mips64el-softmmu/tcg/tcg-op.o
  CC      microblaze-softmmu/balloon.o
  CC      mips64-softmmu/tcg/tcg-op.o
  CC      microblaze-softmmu/ioport.o
  CC      microblazeel-softmmu/memory_mapping.o
  CC      microblazeel-softmmu/dump.o
  CC      microblaze-softmmu/numa.o
  CC      microblazeel-softmmu/migration/ram.o
  CC      microblaze-softmmu/qtest.o
  CC      mips64el-softmmu/tcg/optimize.o
  CC      microblazeel-softmmu/accel/accel.o
  CC      microblaze-softmmu/memory.o
  CC      mips64-softmmu/tcg/optimize.o
  CC      microblazeel-softmmu/accel/stubs/hax-stub.o
  CC      microblazeel-softmmu/accel/stubs/kvm-stub.o
  CC      mips64el-softmmu/tcg/tcg-common.o
  CC      mips64el-softmmu/fpu/softfloat.o
  CC      mips64-softmmu/tcg/tcg-common.o
  CC      microblaze-softmmu/memory_mapping.o
  CC      microblazeel-softmmu/accel/tcg/tcg-all.o
  CC      microblaze-softmmu/dump.o
  CC      microblazeel-softmmu/accel/tcg/cputlb.o
  CC      mips64-softmmu/fpu/softfloat.o
  CC      microblaze-softmmu/migration/ram.o
  CC      microblazeel-softmmu/accel/tcg/tcg-runtime.o
  CC      microblaze-softmmu/accel/accel.o
  CC      microblazeel-softmmu/accel/tcg/cpu-exec.o
  CC      microblaze-softmmu/accel/stubs/hax-stub.o
  CC      mips64el-softmmu/disas.o
  CC      microblazeel-softmmu/accel/tcg/cpu-exec-common.o
  CC      microblaze-softmmu/accel/stubs/kvm-stub.o
  CC      mips64el-softmmu/arch_init.o
  CC      mips64-softmmu/disas.o
  CC      microblazeel-softmmu/accel/tcg/translate-all.o
  CC      microblaze-softmmu/accel/tcg/tcg-all.o
  CC      mips64el-softmmu/cpus.o
  CC      mips64-softmmu/arch_init.o
  CC      microblaze-softmmu/accel/tcg/cputlb.o
  CC      mips64-softmmu/cpus.o
  CC      microblazeel-softmmu/accel/tcg/translator.o
  CC      mips64el-softmmu/monitor.o
  CC      microblazeel-softmmu/hw/core/generic-loader.o
  CC      mips64-softmmu/monitor.o
  CC      microblazeel-softmmu/hw/core/null-machine.o
  CC      microblazeel-softmmu/hw/misc/mmio_interface.o
  CC      microblaze-softmmu/accel/tcg/tcg-runtime.o
  CC      microblazeel-softmmu/hw/net/xilinx_ethlite.o
  CC      mips64el-softmmu/gdbstub.o
  CC      microblaze-softmmu/accel/tcg/cpu-exec.o
  CC      microblazeel-softmmu/hw/net/vhost_net.o
  CC      microblaze-softmmu/accel/tcg/cpu-exec-common.o
  CC      mips64el-softmmu/balloon.o
  CC      microblazeel-softmmu/hw/net/rocker/qmp-norocker.o
  CC      mips64-softmmu/gdbstub.o
  CC      microblazeel-softmmu/hw/vfio/common.o
  CC      mips64el-softmmu/ioport.o
  CC      microblaze-softmmu/accel/tcg/translate-all.o
  CC      mips64el-softmmu/numa.o
  CC      mips64-softmmu/balloon.o
  CC      microblaze-softmmu/accel/tcg/translator.o
  CC      microblazeel-softmmu/hw/vfio/platform.o
  CC      mips64el-softmmu/qtest.o
  CC      mips64-softmmu/ioport.o
  CC      mips64-softmmu/numa.o
  CC      microblaze-softmmu/hw/core/generic-loader.o
  CC      microblazeel-softmmu/hw/vfio/spapr.o
  CC      mips64el-softmmu/memory.o
  CC      microblaze-softmmu/hw/core/null-machine.o
  CC      microblazeel-softmmu/hw/microblaze/petalogix_s3adsp1800_mmu.o
  CC      mips64-softmmu/qtest.o
  CC      microblaze-softmmu/hw/misc/mmio_interface.o
  CC      microblazeel-softmmu/hw/microblaze/petalogix_ml605_mmu.o
  CC      microblaze-softmmu/hw/net/xilinx_ethlite.o
  CC      microblazeel-softmmu/hw/microblaze/boot.o
  CC      mips64-softmmu/memory.o
  CC      microblaze-softmmu/hw/net/vhost_net.o
  CC      microblazeel-softmmu/target/microblaze/translate.o
  CC      mips64el-softmmu/memory_mapping.o
  CC      microblaze-softmmu/hw/net/rocker/qmp-norocker.o
  CC      mips64el-softmmu/dump.o
  CC      microblaze-softmmu/hw/vfio/common.o
  CC      mips64-softmmu/memory_mapping.o
  CC      microblaze-softmmu/hw/vfio/platform.o
  CC      microblazeel-softmmu/target/microblaze/op_helper.o
  CC      mips64-softmmu/dump.o
  CC      mips64el-softmmu/migration/ram.o
  CC      microblaze-softmmu/hw/vfio/spapr.o
  CC      microblazeel-softmmu/target/microblaze/helper.o
  CC      mips64-softmmu/migration/ram.o
  CC      microblaze-softmmu/hw/microblaze/petalogix_s3adsp1800_mmu.o
  CC      microblazeel-softmmu/target/microblaze/cpu.o
  CC      mips64el-softmmu/accel/accel.o
  CC      microblazeel-softmmu/target/microblaze/gdbstub.o
  CC      microblaze-softmmu/hw/microblaze/petalogix_ml605_mmu.o
  CC      mips64el-softmmu/accel/stubs/hax-stub.o
  CC      microblaze-softmmu/hw/microblaze/boot.o
  CC      microblazeel-softmmu/target/microblaze/mmu.o
  CC      mips64-softmmu/accel/accel.o
  CC      mips64el-softmmu/accel/stubs/kvm-stub.o
  CC      microblaze-softmmu/target/microblaze/translate.o
  GEN     trace/generated-helpers.c
  CC      microblazeel-softmmu/trace/control-target.o
  CC      mips64-softmmu/accel/stubs/hax-stub.o
  CC      mips64el-softmmu/accel/tcg/tcg-all.o
  CC      microblazeel-softmmu/trace/generated-helpers.o
  CC      mips64-softmmu/accel/stubs/kvm-stub.o
  CC      mips64-softmmu/accel/tcg/tcg-all.o
  CC      mips64el-softmmu/accel/tcg/cputlb.o
  CC      mips64-softmmu/accel/tcg/cputlb.o
  LINK    microblazeel-softmmu/qemu-system-microblazeel
  CC      microblaze-softmmu/target/microblaze/op_helper.o
  CC      mips64-softmmu/accel/tcg/tcg-runtime.o
  CC      microblaze-softmmu/target/microblaze/helper.o
  CC      mips64-softmmu/accel/tcg/cpu-exec.o
  CC      mips64el-softmmu/accel/tcg/tcg-runtime.o
  CC      microblaze-softmmu/target/microblaze/cpu.o
  CC      mips64-softmmu/accel/tcg/cpu-exec-common.o
  CC      microblaze-softmmu/target/microblaze/gdbstub.o
  CC      mips64el-softmmu/accel/tcg/cpu-exec.o
  CC      mips64-softmmu/accel/tcg/translate-all.o
  GEN     mipsel-softmmu/hmp-commands.h
  GEN     mipsel-softmmu/hmp-commands-info.h
  GEN     mipsel-softmmu/config-target.h
  CC      microblaze-softmmu/target/microblaze/mmu.o
  CC      mipsel-softmmu/exec.o
  GEN     trace/generated-helpers.c
  CC      microblaze-softmmu/trace/control-target.o
  CC      mips64-softmmu/accel/tcg/translator.o
  CC      mips64el-softmmu/accel/tcg/cpu-exec-common.o
  CC      microblaze-softmmu/trace/generated-helpers.o
  CC      mips64el-softmmu/accel/tcg/translate-all.o
  LINK    microblaze-softmmu/qemu-system-microblaze
  CC      mips64-softmmu/hw/9pfs/virtio-9p-device.o
  CC      mips64el-softmmu/accel/tcg/translator.o
  CC      mips64-softmmu/hw/block/virtio-blk.o
  CC      mipsel-softmmu/tcg/tcg.o
  CC      mips64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      mips64el-softmmu/hw/9pfs/virtio-9p-device.o
  CC      mips64-softmmu/hw/char/virtio-serial-bus.o
  CC      mips64el-softmmu/hw/block/virtio-blk.o
  CC      mips64-softmmu/hw/core/generic-loader.o
  CC      mips64-softmmu/hw/core/null-machine.o
  CC      mips64-softmmu/hw/display/vga.o
  CC      mips64el-softmmu/hw/block/dataplane/virtio-blk.o
  CC      mips64el-softmmu/hw/char/virtio-serial-bus.o
  GEN     mips-softmmu/hmp-commands.h
  GEN     mips-softmmu/hmp-commands-info.h
  GEN     mips-softmmu/config-target.h
  CC      mips64el-softmmu/hw/core/generic-loader.o
  CC      mipsel-softmmu/tcg/tcg-op.o
  CC      mips-softmmu/exec.o
  CC      mips64-softmmu/hw/display/virtio-gpu.o
  CC      mips64el-softmmu/hw/core/null-machine.o
  CC      mips64el-softmmu/hw/display/vga.o
  CC      mips64-softmmu/hw/display/virtio-gpu-3d.o
  CC      mips64el-softmmu/hw/display/virtio-gpu.o
  CC      mips64-softmmu/hw/display/virtio-gpu-pci.o
  CC      mips-softmmu/tcg/tcg.o
  CC      mips64-softmmu/hw/intc/mips_gic.o
  CC      mips64el-softmmu/hw/display/virtio-gpu-3d.o
  CC      mipsel-softmmu/tcg/optimize.o
  CC      mips64-softmmu/hw/misc/ivshmem.o
  CC      mips64el-softmmu/hw/display/virtio-gpu-pci.o
  CC      mips64-softmmu/hw/misc/mips_cmgcr.o
  CC      mips64el-softmmu/hw/intc/mips_gic.o
  CC      mips64-softmmu/hw/misc/mips_cpc.o
  CC      mips64-softmmu/hw/misc/mips_itu.o
  CC      mipsel-softmmu/tcg/tcg-common.o
  CC      mips64el-softmmu/hw/misc/ivshmem.o
  CC      mips-softmmu/tcg/tcg-op.o
  CC      mipsel-softmmu/fpu/softfloat.o
  CC      mips64-softmmu/hw/misc/mmio_interface.o
  CC      mips64-softmmu/hw/net/virtio-net.o
  CC      mips64el-softmmu/hw/misc/mips_cmgcr.o
  CC      mips64el-softmmu/hw/misc/mips_cpc.o
  CC      mips64-softmmu/hw/net/vhost_net.o
  CC      mips64el-softmmu/hw/misc/mips_itu.o
  CC      mips64el-softmmu/hw/misc/mmio_interface.o
  CC      mips64-softmmu/hw/scsi/virtio-scsi.o
  CC      mips64el-softmmu/hw/net/virtio-net.o
  CC      mips64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      mips64el-softmmu/hw/net/vhost_net.o
  CC      mips64-softmmu/hw/scsi/vhost-scsi-common.o
  CC      mips-softmmu/tcg/optimize.o
  CC      mipsel-softmmu/disas.o
  CC      mips64el-softmmu/hw/scsi/virtio-scsi.o
  CC      mips64-softmmu/hw/scsi/vhost-scsi.o
  CC      mipsel-softmmu/arch_init.o
  CC      mipsel-softmmu/cpus.o
  CC      mips64el-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      mips64-softmmu/hw/scsi/vhost-user-scsi.o
  CC      mips64-softmmu/hw/timer/mips_gictimer.o
  CC      mips-softmmu/tcg/tcg-common.o
  CC      mips64el-softmmu/hw/scsi/vhost-scsi-common.o
  CC      mips-softmmu/fpu/softfloat.o
  CC      mipsel-softmmu/monitor.o
  CC      mips64el-softmmu/hw/scsi/vhost-scsi.o
  CC      mips64-softmmu/hw/timer/mc146818rtc.o
  CC      mips64el-softmmu/hw/scsi/vhost-user-scsi.o
  CC      mips64-softmmu/hw/vfio/common.o
  CC      mips64el-softmmu/hw/timer/mips_gictimer.o
  CC      mips64el-softmmu/hw/timer/mc146818rtc.o
  CC      mips64-softmmu/hw/vfio/pci.o
  CC      mipsel-softmmu/gdbstub.o
  CC      mips64el-softmmu/hw/vfio/common.o
  CC      mipsel-softmmu/balloon.o
  CC      mipsel-softmmu/ioport.o
  CC      mips64el-softmmu/hw/vfio/pci.o
  CC      mips64-softmmu/hw/vfio/pci-quirks.o
  CC      mipsel-softmmu/numa.o
  CC      mips-softmmu/disas.o
  CC      mips-softmmu/arch_init.o
  CC      mipsel-softmmu/qtest.o
  CC      mips-softmmu/cpus.o
  CC      mips64-softmmu/hw/vfio/platform.o
  CC      mipsel-softmmu/memory.o
  CC      mips64-softmmu/hw/vfio/spapr.o
  CC      mips64el-softmmu/hw/vfio/pci-quirks.o
  CC      mips-softmmu/monitor.o
  CC      mips64-softmmu/hw/virtio/virtio.o
  CC      mips64el-softmmu/hw/vfio/platform.o
  CC      mipsel-softmmu/memory_mapping.o
  CC      mips-softmmu/gdbstub.o
  CC      mips64el-softmmu/hw/vfio/spapr.o
  CC      mipsel-softmmu/dump.o
  CC      mips64-softmmu/hw/virtio/virtio-balloon.o
  CC      mips64el-softmmu/hw/virtio/virtio.o
  CC      mips-softmmu/balloon.o
  CC      mipsel-softmmu/migration/ram.o
  CC      mips64-softmmu/hw/virtio/vhost.o
  CC      mips-softmmu/ioport.o
  CC      mips64el-softmmu/hw/virtio/virtio-balloon.o
  CC      mips64-softmmu/hw/virtio/vhost-backend.o
  CC      mips-softmmu/numa.o
  CC      mipsel-softmmu/accel/accel.o
  CC      mips64-softmmu/hw/virtio/vhost-user.o
  CC      mips64el-softmmu/hw/virtio/vhost.o
  CC      mips-softmmu/qtest.o
  CC      mipsel-softmmu/accel/stubs/hax-stub.o
  CC      mips64-softmmu/hw/virtio/vhost-vsock.o
  CC      mips-softmmu/memory.o
  CC      mips64el-softmmu/hw/virtio/vhost-backend.o
  CC      mipsel-softmmu/accel/stubs/kvm-stub.o
  CC      mips64-softmmu/hw/virtio/virtio-crypto.o
  CC      mips64el-softmmu/hw/virtio/vhost-user.o
  CC      mipsel-softmmu/accel/tcg/tcg-all.o
  CC      mips64el-softmmu/hw/virtio/vhost-vsock.o
  CC      mipsel-softmmu/accel/tcg/cputlb.o
  CC      mips64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      mips64el-softmmu/hw/virtio/virtio-crypto.o
  CC      mips64el-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      mips64-softmmu/hw/mips/mips_r4k.o
  CC      mips-softmmu/memory_mapping.o
  CC      mips64el-softmmu/hw/mips/mips_r4k.o
  CC      mips64-softmmu/hw/mips/mips_malta.o
  CC      mips-softmmu/dump.o
  CC      mipsel-softmmu/accel/tcg/tcg-runtime.o
  CC      mips64el-softmmu/hw/mips/mips_malta.o
  CC      mips64-softmmu/hw/mips/mips_mipssim.o
  CC      mips-softmmu/migration/ram.o
  CC      mipsel-softmmu/accel/tcg/cpu-exec.o
  CC      mips64el-softmmu/hw/mips/mips_mipssim.o
  CC      mips64-softmmu/hw/mips/addr.o
  CC      mipsel-softmmu/accel/tcg/cpu-exec-common.o
  CC      mips64-softmmu/hw/mips/mips_int.o
  CC      mips64el-softmmu/hw/mips/addr.o
  CC      mipsel-softmmu/accel/tcg/translate-all.o
  CC      mips64-softmmu/hw/mips/mips_jazz.o
  CC      mips64el-softmmu/hw/mips/mips_int.o
  CC      mips-softmmu/accel/accel.o
  CC      mipsel-softmmu/accel/tcg/translator.o
  CC      mips64el-softmmu/hw/mips/mips_jazz.o
  CC      mips-softmmu/accel/stubs/hax-stub.o
  CC      mips64-softmmu/hw/mips/gt64xxx_pci.o
  CC      mips-softmmu/accel/stubs/kvm-stub.o
  CC      mips64el-softmmu/hw/mips/mips_fulong2e.o
  CC      mips64-softmmu/hw/mips/cps.o
  CC      mipsel-softmmu/hw/9pfs/virtio-9p-device.o
  CC      mips64el-softmmu/hw/mips/gt64xxx_pci.o
  CC      mips-softmmu/accel/tcg/tcg-all.o
  CC      mips64-softmmu/target/mips/translate.o
  CC      mipsel-softmmu/hw/block/virtio-blk.o
  CC      mips-softmmu/accel/tcg/cputlb.o
  CC      mips64el-softmmu/hw/mips/cps.o
  CC      mips64el-softmmu/hw/mips/boston.o
  CC      mipsel-softmmu/hw/block/dataplane/virtio-blk.o
  CC      mips64el-softmmu/target/mips/translate.o
  CC      mipsel-softmmu/hw/char/virtio-serial-bus.o
  CC      mips-softmmu/accel/tcg/tcg-runtime.o
  CC      mipsel-softmmu/hw/core/generic-loader.o
  CC      mips-softmmu/accel/tcg/cpu-exec.o
  CC      mipsel-softmmu/hw/core/null-machine.o
  CC      mips-softmmu/accel/tcg/cpu-exec-common.o
  CC      mipsel-softmmu/hw/display/vga.o
  CC      mips-softmmu/accel/tcg/translate-all.o
  CC      mips-softmmu/accel/tcg/translator.o
  CC      mipsel-softmmu/hw/display/virtio-gpu.o
  CC      mips-softmmu/hw/9pfs/virtio-9p-device.o
  CC      mips-softmmu/hw/block/virtio-blk.o
  CC      mipsel-softmmu/hw/display/virtio-gpu-3d.o
  CC      mips-softmmu/hw/block/dataplane/virtio-blk.o
  CC      mips64-softmmu/target/mips/dsp_helper.o
  CC      mipsel-softmmu/hw/display/virtio-gpu-pci.o
  CC      mips-softmmu/hw/char/virtio-serial-bus.o
  CC      mipsel-softmmu/hw/intc/mips_gic.o
  CC      mips-softmmu/hw/core/generic-loader.o
  CC      mipsel-softmmu/hw/misc/ivshmem.o
  CC      mips-softmmu/hw/core/null-machine.o
  CC      mips-softmmu/hw/display/vga.o
  CC      mips64-softmmu/target/mips/op_helper.o
  CC      mips64el-softmmu/target/mips/dsp_helper.o
  CC      mipsel-softmmu/hw/misc/mips_cmgcr.o
  CC      mipsel-softmmu/hw/misc/mips_cpc.o
  CC      mipsel-softmmu/hw/misc/mips_itu.o
  CC      mips-softmmu/hw/display/virtio-gpu.o
  CC      mipsel-softmmu/hw/misc/mmio_interface.o
  CC      mips64el-softmmu/target/mips/op_helper.o
  CC      mipsel-softmmu/hw/net/virtio-net.o
  CC      mips-softmmu/hw/display/virtio-gpu-3d.o
  CC      mipsel-softmmu/hw/net/vhost_net.o
  CC      mips64-softmmu/target/mips/lmi_helper.o
  CC      mipsel-softmmu/hw/scsi/virtio-scsi.o
  CC      mips-softmmu/hw/display/virtio-gpu-pci.o
  CC      mips-softmmu/hw/intc/mips_gic.o
  CC      mips64-softmmu/target/mips/helper.o
  CC      mipsel-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      mips-softmmu/hw/misc/ivshmem.o
  CC      mips64-softmmu/target/mips/cpu.o
  CC      mipsel-softmmu/hw/scsi/vhost-scsi-common.o
  CC      mips64el-softmmu/target/mips/lmi_helper.o
  CC      mips64-softmmu/target/mips/gdbstub.o
  CC      mipsel-softmmu/hw/scsi/vhost-scsi.o
  CC      mips-softmmu/hw/misc/mips_cmgcr.o
  CC      mipsel-softmmu/hw/scsi/vhost-user-scsi.o
  CC      mips64-softmmu/target/mips/msa_helper.o
  CC      mips-softmmu/hw/misc/mips_cpc.o
  CC      mips64el-softmmu/target/mips/helper.o
  CC      mips-softmmu/hw/misc/mips_itu.o
  CC      mipsel-softmmu/hw/timer/mips_gictimer.o
  CC      mips-softmmu/hw/misc/mmio_interface.o
  CC      mipsel-softmmu/hw/timer/mc146818rtc.o
  CC      mips64el-softmmu/target/mips/cpu.o
  CC      mips-softmmu/hw/net/virtio-net.o
  CC      mips64el-softmmu/target/mips/gdbstub.o
  CC      mipsel-softmmu/hw/vfio/common.o
  CC      mips64el-softmmu/target/mips/msa_helper.o
  CC      mips-softmmu/hw/net/vhost_net.o
  CC      mips-softmmu/hw/scsi/virtio-scsi.o
  CC      mipsel-softmmu/hw/vfio/pci.o
  CC      mips-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      mips-softmmu/hw/scsi/vhost-scsi-common.o
  CC      mipsel-softmmu/hw/vfio/pci-quirks.o
  CC      mips64-softmmu/target/mips/mips-semi.o
  CC      mips-softmmu/hw/scsi/vhost-scsi.o
  CC      mipsel-softmmu/hw/vfio/platform.o
  CC      mips-softmmu/hw/scsi/vhost-user-scsi.o
  CC      mips64-softmmu/target/mips/machine.o
  CC      mips-softmmu/hw/timer/mips_gictimer.o
  CC      mips64-softmmu/target/mips/cp0_timer.o
  CC      mipsel-softmmu/hw/vfio/spapr.o
  CC      mips-softmmu/hw/timer/mc146818rtc.o
  GEN     trace/generated-helpers.c
  CC      mips64-softmmu/trace/control-target.o
  CC      mipsel-softmmu/hw/virtio/virtio.o
  CC      mips-softmmu/hw/vfio/common.o
  CC      mips64el-softmmu/target/mips/mips-semi.o
  CC      mips64-softmmu/trace/generated-helpers.o
  LINK    mips64-softmmu/qemu-system-mips64
  CC      mips64el-softmmu/target/mips/machine.o
  CC      mips-softmmu/hw/vfio/pci.o
  CC      mipsel-softmmu/hw/virtio/virtio-balloon.o
  CC      mips64el-softmmu/target/mips/cp0_timer.o
  GEN     trace/generated-helpers.c
  CC      mipsel-softmmu/hw/virtio/vhost.o
  CC      mips64el-softmmu/trace/control-target.o
  CC      mips-softmmu/hw/vfio/pci-quirks.o
  CC      mips64el-softmmu/trace/generated-helpers.o
  CC      mipsel-softmmu/hw/virtio/vhost-backend.o
  LINK    mips64el-softmmu/qemu-system-mips64el
  GEN     moxie-softmmu/hmp-commands.h
  GEN     moxie-softmmu/hmp-commands-info.h
  CC      mips-softmmu/hw/vfio/platform.o
  GEN     moxie-softmmu/config-target.h
  CC      mipsel-softmmu/hw/virtio/vhost-user.o
  CC      moxie-softmmu/exec.o
  CC      mips-softmmu/hw/vfio/spapr.o
  CC      mipsel-softmmu/hw/virtio/vhost-vsock.o
  CC      mips-softmmu/hw/virtio/virtio.o
  CC      mipsel-softmmu/hw/virtio/virtio-crypto.o
  CC      moxie-softmmu/tcg/tcg.o
  CC      mipsel-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      mips-softmmu/hw/virtio/virtio-balloon.o
  CC      moxie-softmmu/tcg/tcg-op.o
  CC      mipsel-softmmu/hw/mips/mips_r4k.o
  CC      mips-softmmu/hw/virtio/vhost.o
  CC      mipsel-softmmu/hw/mips/mips_malta.o
  CC      mips-softmmu/hw/virtio/vhost-backend.o
  CC      moxie-softmmu/tcg/optimize.o
  CC      mips-softmmu/hw/virtio/vhost-user.o
  CC      mipsel-softmmu/hw/mips/mips_mipssim.o
  CC      moxie-softmmu/tcg/tcg-common.o
  CC      mips-softmmu/hw/virtio/vhost-vsock.o
  CC      mipsel-softmmu/hw/mips/addr.o
  CC      moxie-softmmu/fpu/softfloat.o
  CC      mips-softmmu/hw/virtio/virtio-crypto.o
  CC      mipsel-softmmu/hw/mips/mips_int.o
  CC      mipsel-softmmu/hw/mips/gt64xxx_pci.o
  GEN     nios2-softmmu/hmp-commands.h
  GEN     nios2-softmmu/hmp-commands-info.h
  GEN     nios2-softmmu/config-target.h
  CC      nios2-softmmu/exec.o
  CC      mips-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      mipsel-softmmu/hw/mips/cps.o
  CC      mips-softmmu/hw/mips/mips_r4k.o
  CC      mipsel-softmmu/target/mips/translate.o
  CC      mips-softmmu/hw/mips/mips_malta.o
  CC      mips-softmmu/hw/mips/mips_mipssim.o
  CC      mips-softmmu/hw/mips/addr.o
  CC      nios2-softmmu/tcg/tcg.o
  CC      moxie-softmmu/disas.o
  CC      mips-softmmu/hw/mips/mips_int.o
  CC      moxie-softmmu/arch_init.o
  CC      mips-softmmu/hw/mips/gt64xxx_pci.o
  CC      mips-softmmu/hw/mips/cps.o
  CC      moxie-softmmu/cpus.o
  CC      mips-softmmu/target/mips/translate.o
  CC      nios2-softmmu/tcg/tcg-op.o
  CC      moxie-softmmu/monitor.o
  CC      mipsel-softmmu/target/mips/dsp_helper.o
  CC      moxie-softmmu/gdbstub.o
  CC      mipsel-softmmu/target/mips/op_helper.o
  CC      nios2-softmmu/tcg/optimize.o
  CC      moxie-softmmu/balloon.o
  CC      moxie-softmmu/ioport.o
  CC      nios2-softmmu/tcg/tcg-common.o
  CC      moxie-softmmu/numa.o
  CC      nios2-softmmu/fpu/softfloat.o
  CC      moxie-softmmu/qtest.o
  CC      moxie-softmmu/memory.o
  CC      mips-softmmu/target/mips/dsp_helper.o
  CC      mipsel-softmmu/target/mips/lmi_helper.o
  CC      mipsel-softmmu/target/mips/helper.o
  CC      moxie-softmmu/memory_mapping.o
  CC      mips-softmmu/target/mips/op_helper.o
  CC      nios2-softmmu/disas.o
  CC      moxie-softmmu/dump.o
  CC      mipsel-softmmu/target/mips/cpu.o
  CC      nios2-softmmu/arch_init.o
  CC      mipsel-softmmu/target/mips/gdbstub.o
  CC      moxie-softmmu/migration/ram.o
  CC      nios2-softmmu/cpus.o
  CC      mipsel-softmmu/target/mips/msa_helper.o
  CC      nios2-softmmu/monitor.o
  CC      moxie-softmmu/accel/accel.o
  CC      mips-softmmu/target/mips/lmi_helper.o
  CC      moxie-softmmu/accel/stubs/hax-stub.o
  CC      moxie-softmmu/accel/stubs/kvm-stub.o
  CC      mips-softmmu/target/mips/helper.o
  CC      moxie-softmmu/accel/tcg/tcg-all.o
  CC      nios2-softmmu/gdbstub.o
  CC      mips-softmmu/target/mips/cpu.o
  CC      moxie-softmmu/accel/tcg/cputlb.o
  CC      mips-softmmu/target/mips/gdbstub.o
  CC      nios2-softmmu/balloon.o
  CC      mips-softmmu/target/mips/msa_helper.o
  CC      moxie-softmmu/accel/tcg/tcg-runtime.o
  CC      nios2-softmmu/ioport.o
  CC      moxie-softmmu/accel/tcg/cpu-exec.o
  CC      nios2-softmmu/numa.o
  CC      mipsel-softmmu/target/mips/mips-semi.o
  CC      moxie-softmmu/accel/tcg/cpu-exec-common.o
  CC      nios2-softmmu/qtest.o
  CC      moxie-softmmu/accel/tcg/translate-all.o
  CC      mipsel-softmmu/target/mips/machine.o
  CC      moxie-softmmu/accel/tcg/translator.o
  CC      mipsel-softmmu/target/mips/cp0_timer.o
  CC      nios2-softmmu/memory.o
  GEN     trace/generated-helpers.c
  CC      mipsel-softmmu/trace/control-target.o
  CC      moxie-softmmu/hw/core/generic-loader.o
  CC      moxie-softmmu/hw/core/null-machine.o
  CC      mipsel-softmmu/trace/generated-helpers.o
  CC      moxie-softmmu/hw/display/vga.o
  CC      nios2-softmmu/memory_mapping.o
  CC      mips-softmmu/target/mips/mips-semi.o
  LINK    mipsel-softmmu/qemu-system-mipsel
  CC      nios2-softmmu/dump.o
  CC      mips-softmmu/target/mips/machine.o
  CC      mips-softmmu/target/mips/cp0_timer.o
  CC      moxie-softmmu/hw/misc/mmio_interface.o
  CC      nios2-softmmu/migration/ram.o
  GEN     trace/generated-helpers.c
  CC      mips-softmmu/trace/control-target.o
  CC      moxie-softmmu/hw/net/vhost_net.o
  CC      moxie-softmmu/hw/net/rocker/qmp-norocker.o
  CC      nios2-softmmu/accel/accel.o
  CC      mips-softmmu/trace/generated-helpers.o
  CC      moxie-softmmu/hw/timer/mc146818rtc.o
  LINK    mips-softmmu/qemu-system-mips
  CC      nios2-softmmu/accel/stubs/hax-stub.o
  GEN     or1k-softmmu/hmp-commands.h
  CC      moxie-softmmu/hw/vfio/common.o
  GEN     or1k-softmmu/hmp-commands-info.h
  CC      nios2-softmmu/accel/stubs/kvm-stub.o
  GEN     or1k-softmmu/config-target.h
  CC      or1k-softmmu/exec.o
  CC      nios2-softmmu/accel/tcg/tcg-all.o
  CC      moxie-softmmu/hw/vfio/platform.o
  CC      nios2-softmmu/accel/tcg/cputlb.o
  CC      moxie-softmmu/hw/vfio/spapr.o
  CC      moxie-softmmu/hw/moxie/moxiesim.o
  CC      moxie-softmmu/target/moxie/translate.o
  GEN     ppc64-softmmu/hmp-commands.h
  GEN     ppc64-softmmu/hmp-commands-info.h
  GEN     ppc64-softmmu/config-target.h
  CC      ppc64-softmmu/exec.o
  CC      nios2-softmmu/accel/tcg/tcg-runtime.o
  CC      moxie-softmmu/target/moxie/helper.o
  CC      or1k-softmmu/tcg/tcg.o
  CC      nios2-softmmu/accel/tcg/cpu-exec.o
  CC      moxie-softmmu/target/moxie/machine.o
  CC      moxie-softmmu/target/moxie/cpu.o
  CC      nios2-softmmu/accel/tcg/cpu-exec-common.o
  CC      nios2-softmmu/accel/tcg/translate-all.o
  CC      moxie-softmmu/target/moxie/mmu.o
  GEN     trace/generated-helpers.c
  CC      moxie-softmmu/trace/control-target.o
  CC      nios2-softmmu/accel/tcg/translator.o
  CC      ppc64-softmmu/tcg/tcg.o
  CC      nios2-softmmu/hw/core/generic-loader.o
  CC      moxie-softmmu/trace/generated-helpers.o
  CC      or1k-softmmu/tcg/tcg-op.o
  LINK    moxie-softmmu/qemu-system-moxie
  CC      nios2-softmmu/hw/core/null-machine.o
  CC      nios2-softmmu/hw/intc/nios2_iic.o
  CC      nios2-softmmu/hw/misc/mmio_interface.o
  CC      nios2-softmmu/hw/net/vhost_net.o
  CC      nios2-softmmu/hw/net/rocker/qmp-norocker.o
  CC      ppc64-softmmu/tcg/tcg-op.o
  CC      or1k-softmmu/tcg/optimize.o
  CC      nios2-softmmu/hw/timer/altera_timer.o
  CC      nios2-softmmu/hw/vfio/common.o
  CC      or1k-softmmu/tcg/tcg-common.o
  CC      or1k-softmmu/fpu/softfloat.o
  CC      or1k-softmmu/disas.o
  CC      nios2-softmmu/hw/vfio/platform.o
  CC      nios2-softmmu/hw/vfio/spapr.o
  CC      nios2-softmmu/hw/nios2/boot.o
  CC      nios2-softmmu/hw/nios2/cpu_pic.o
  GEN     ppcemb-softmmu/hmp-commands.h
  GEN     ppcemb-softmmu/hmp-commands-info.h
  CC      nios2-softmmu/hw/nios2/10m50_devboard.o
  GEN     ppcemb-softmmu/config-target.h
  CC      ppc64-softmmu/tcg/optimize.o
  CC      ppcemb-softmmu/exec.o
  CC      nios2-softmmu/target/nios2/translate.o
  CC      nios2-softmmu/target/nios2/op_helper.o
  CC      ppc64-softmmu/tcg/tcg-common.o
  CC      nios2-softmmu/target/nios2/helper.o
  CC      or1k-softmmu/arch_init.o
  CC      ppcemb-softmmu/tcg/tcg.o
  CC      ppc64-softmmu/fpu/softfloat.o
  CC      nios2-softmmu/target/nios2/cpu.o
  CC      or1k-softmmu/cpus.o
  CC      nios2-softmmu/target/nios2/mmu.o
  CC      nios2-softmmu/target/nios2/monitor.o
  CC      or1k-softmmu/monitor.o
  GEN     trace/generated-helpers.c
  CC      nios2-softmmu/trace/control-target.o
  CC      nios2-softmmu/trace/generated-helpers.o
  CC      ppc64-softmmu/disas.o
  LINK    nios2-softmmu/qemu-system-nios2
  CC      ppcemb-softmmu/tcg/tcg-op.o
  CC      or1k-softmmu/gdbstub.o
  GEN     ppc64-softmmu/gdbstub-xml.c
  CC      or1k-softmmu/balloon.o
  CC      or1k-softmmu/ioport.o
  GEN     ppc-softmmu/hmp-commands.h
  CC      ppc64-softmmu/libdecnumber/decContext.o
  GEN     ppc-softmmu/hmp-commands-info.h
  GEN     ppc-softmmu/config-target.h
  CC      ppc-softmmu/exec.o
  CC      or1k-softmmu/numa.o
  CC      ppc64-softmmu/libdecnumber/decNumber.o
  CC      or1k-softmmu/qtest.o
  CC      or1k-softmmu/memory.o
  CC      ppcemb-softmmu/tcg/optimize.o
  CC      ppc-softmmu/tcg/tcg.o
  CC      ppc64-softmmu/libdecnumber/dpd/decimal32.o
  CC      or1k-softmmu/memory_mapping.o
  CC      ppc64-softmmu/libdecnumber/dpd/decimal64.o
  CC      ppcemb-softmmu/tcg/tcg-common.o
  CC      ppc64-softmmu/libdecnumber/dpd/decimal128.o
  CC      ppcemb-softmmu/fpu/softfloat.o
  CC      or1k-softmmu/dump.o
  CC      ppc64-softmmu/arch_init.o
  CC      or1k-softmmu/migration/ram.o
  CC      ppc64-softmmu/cpus.o
  CC      ppc-softmmu/tcg/tcg-op.o
  CC      or1k-softmmu/accel/accel.o
  CC      ppc64-softmmu/monitor.o
  CC      or1k-softmmu/accel/stubs/hax-stub.o
  CC      or1k-softmmu/accel/stubs/kvm-stub.o
  CC      or1k-softmmu/accel/tcg/tcg-all.o
  CC      or1k-softmmu/accel/tcg/cputlb.o
  CC      ppcemb-softmmu/disas.o
  CC      ppc64-softmmu/gdbstub.o
  GEN     ppcemb-softmmu/gdbstub-xml.c
  CC      or1k-softmmu/accel/tcg/tcg-runtime.o
  CC      ppc64-softmmu/balloon.o
  CC      ppc-softmmu/tcg/optimize.o
  CC      ppcemb-softmmu/libdecnumber/decContext.o
  CC      or1k-softmmu/accel/tcg/cpu-exec.o
  CC      ppc64-softmmu/ioport.o
  CC      ppcemb-softmmu/libdecnumber/decNumber.o
  CC      or1k-softmmu/accel/tcg/cpu-exec-common.o
  CC      ppc64-softmmu/numa.o
  CC      or1k-softmmu/accel/tcg/translate-all.o
  CC      ppc-softmmu/tcg/tcg-common.o
  CC      ppc-softmmu/fpu/softfloat.o
  CC      ppc64-softmmu/qtest.o
  CC      or1k-softmmu/accel/tcg/translator.o
  CC      ppcemb-softmmu/libdecnumber/dpd/decimal32.o
  CC      ppc64-softmmu/memory.o
  CC      ppcemb-softmmu/libdecnumber/dpd/decimal64.o
  CC      or1k-softmmu/hw/core/generic-loader.o
  CC      or1k-softmmu/hw/core/null-machine.o
  CC      ppcemb-softmmu/libdecnumber/dpd/decimal128.o
  CC      or1k-softmmu/hw/misc/mmio_interface.o
  CC      ppcemb-softmmu/arch_init.o
  CC      or1k-softmmu/hw/net/vhost_net.o
  CC      ppc64-softmmu/memory_mapping.o
  CC      or1k-softmmu/hw/net/rocker/qmp-norocker.o
  CC      ppcemb-softmmu/cpus.o
  CC      ppc64-softmmu/dump.o
  CC      ppc-softmmu/disas.o
  CC      or1k-softmmu/hw/vfio/common.o
  GEN     ppc-softmmu/gdbstub-xml.c
  CC      or1k-softmmu/hw/vfio/platform.o
  CC      ppcemb-softmmu/monitor.o
  CC      ppc64-softmmu/migration/ram.o
  CC      or1k-softmmu/hw/vfio/spapr.o
  CC      ppc-softmmu/libdecnumber/decContext.o
  CC      or1k-softmmu/hw/openrisc/pic_cpu.o
  CC      ppc-softmmu/libdecnumber/decNumber.o
  CC      or1k-softmmu/hw/openrisc/cputimer.o
  CC      ppc64-softmmu/accel/accel.o
  CC      or1k-softmmu/hw/openrisc/openrisc_sim.o
  CC      ppcemb-softmmu/gdbstub.o
  CC      ppc64-softmmu/accel/stubs/hax-stub.o
  CC      or1k-softmmu/target/openrisc/machine.o
  CC      ppc64-softmmu/accel/stubs/kvm-stub.o
  CC      or1k-softmmu/target/openrisc/cpu.o
  CC      ppc64-softmmu/accel/tcg/tcg-all.o
  CC      ppcemb-softmmu/balloon.o
  CC      ppc64-softmmu/accel/tcg/cputlb.o
  CC      or1k-softmmu/target/openrisc/exception.o
  CC      or1k-softmmu/target/openrisc/interrupt.o
  CC      ppcemb-softmmu/ioport.o
  CC      or1k-softmmu/target/openrisc/mmu.o
  CC      ppc-softmmu/libdecnumber/dpd/decimal32.o
  CC      ppcemb-softmmu/numa.o
  CC      or1k-softmmu/target/openrisc/translate.o
  CC      ppc-softmmu/libdecnumber/dpd/decimal64.o
  CC      ppc64-softmmu/accel/tcg/tcg-runtime.o
  CC      ppcemb-softmmu/qtest.o
  CC      ppc-softmmu/libdecnumber/dpd/decimal128.o
  CC      ppc64-softmmu/accel/tcg/cpu-exec.o
  CC      ppc-softmmu/arch_init.o
  CC      or1k-softmmu/target/openrisc/exception_helper.o
  CC      ppcemb-softmmu/memory.o
  CC      ppc64-softmmu/accel/tcg/cpu-exec-common.o
  CC      ppc-softmmu/cpus.o
  CC      or1k-softmmu/target/openrisc/fpu_helper.o
  CC      ppc64-softmmu/accel/tcg/translate-all.o
  CC      or1k-softmmu/target/openrisc/interrupt_helper.o
  CC      ppc-softmmu/monitor.o
  CC      or1k-softmmu/target/openrisc/mmu_helper.o
  CC      ppcemb-softmmu/memory_mapping.o
  CC      ppc64-softmmu/accel/tcg/translator.o
  CC      ppcemb-softmmu/dump.o
  CC      or1k-softmmu/target/openrisc/sys_helper.o
  CC      or1k-softmmu/target/openrisc/gdbstub.o
  CC      ppc64-softmmu/hw/9pfs/virtio-9p-device.o
  CC      ppc-softmmu/gdbstub.o
  GEN     trace/generated-helpers.c
  CC      or1k-softmmu/trace/control-target.o
  CC      ppcemb-softmmu/migration/ram.o
  CC      ppc64-softmmu/hw/block/virtio-blk.o
  CC      or1k-softmmu/trace/generated-helpers.o
  LINK    or1k-softmmu/qemu-system-or1k
  CC      ppc64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      ppc-softmmu/balloon.o
  CC      ppcemb-softmmu/accel/accel.o
  CC      ppc64-softmmu/hw/char/spapr_vty.o
  CC      ppc-softmmu/ioport.o
  CC      ppcemb-softmmu/accel/stubs/hax-stub.o
  CC      ppc64-softmmu/hw/char/virtio-serial-bus.o
  CC      ppcemb-softmmu/accel/stubs/kvm-stub.o
  CC      ppc-softmmu/numa.o
  CC      ppc64-softmmu/hw/core/generic-loader.o
  CC      ppcemb-softmmu/accel/tcg/tcg-all.o
  CC      ppc-softmmu/qtest.o
  GEN     s390x-softmmu/hmp-commands.h
  CC      ppc64-softmmu/hw/core/null-machine.o
  CC      ppcemb-softmmu/accel/tcg/cputlb.o
  GEN     s390x-softmmu/hmp-commands-info.h
  CC      s390x-softmmu/gen-features
  CC      ppc64-softmmu/hw/display/sm501.o
  CC      ppc-softmmu/memory.o
  GEN     s390x-softmmu/config-target.h
  GEN     s390x-softmmu/gen-features.h
  CC      s390x-softmmu/exec.o
  CC      ppcemb-softmmu/accel/tcg/tcg-runtime.o
  CC      ppc64-softmmu/hw/display/vga.o
  CC      ppc-softmmu/memory_mapping.o
  CC      ppcemb-softmmu/accel/tcg/cpu-exec.o
  CC      ppc-softmmu/dump.o
  CC      ppcemb-softmmu/accel/tcg/cpu-exec-common.o
  CC      ppc64-softmmu/hw/display/virtio-gpu.o
  CC      s390x-softmmu/tcg/tcg.o
  CC      ppc-softmmu/migration/ram.o
  CC      ppcemb-softmmu/accel/tcg/translate-all.o
  CC      ppcemb-softmmu/accel/tcg/translator.o
  CC      ppc64-softmmu/hw/display/virtio-gpu-3d.o
  CC      ppc-softmmu/accel/accel.o
  CC      ppc-softmmu/accel/stubs/hax-stub.o
  CC      ppc64-softmmu/hw/display/virtio-gpu-pci.o
  CC      ppcemb-softmmu/hw/9pfs/virtio-9p-device.o
  CC      ppc-softmmu/accel/stubs/kvm-stub.o
  CC      ppcemb-softmmu/hw/block/virtio-blk.o
  CC      s390x-softmmu/tcg/tcg-op.o
  CC      ppc64-softmmu/hw/display/virtio-vga.o
  CC      ppc-softmmu/accel/tcg/tcg-all.o
  CC      ppcemb-softmmu/hw/block/dataplane/virtio-blk.o
  CC      ppc-softmmu/accel/tcg/cputlb.o
  CC      ppc64-softmmu/hw/i2c/ppc4xx_i2c.o
  CC      ppcemb-softmmu/hw/char/virtio-serial-bus.o
  CC      ppc64-softmmu/hw/intc/xics.o
  CC      ppcemb-softmmu/hw/core/generic-loader.o
  CC      ppc-softmmu/accel/tcg/tcg-runtime.o
  CC      ppc64-softmmu/hw/intc/xics_spapr.o
  CC      ppcemb-softmmu/hw/core/null-machine.o
  CC      ppc-softmmu/accel/tcg/cpu-exec.o
  CC      s390x-softmmu/tcg/optimize.o
  CC      ppc64-softmmu/hw/intc/xics_pnv.o
  CC      ppcemb-softmmu/hw/display/sm501.o
  CC      ppc-softmmu/accel/tcg/cpu-exec-common.o
  CC      ppc64-softmmu/hw/misc/ivshmem.o
  CC      ppc-softmmu/accel/tcg/translate-all.o
  CC      s390x-softmmu/tcg/tcg-common.o
  CC      ppcemb-softmmu/hw/display/vga.o
  CC      ppc64-softmmu/hw/misc/mmio_interface.o
  CC      s390x-softmmu/fpu/softfloat.o
  CC      ppc-softmmu/accel/tcg/translator.o
  CC      ppc64-softmmu/hw/net/spapr_llan.o
  CC      ppc-softmmu/hw/9pfs/virtio-9p-device.o
  CC      ppcemb-softmmu/hw/display/virtio-gpu.o
  CC      ppc64-softmmu/hw/net/xilinx_ethlite.o
  CC      ppc-softmmu/hw/block/virtio-blk.o
  CC      ppc64-softmmu/hw/net/virtio-net.o
  CC      ppcemb-softmmu/hw/display/virtio-gpu-3d.o
  CC      ppc-softmmu/hw/block/dataplane/virtio-blk.o
  CC      ppc64-softmmu/hw/net/vhost_net.o
  CC      ppc-softmmu/hw/char/virtio-serial-bus.o
  CC      ppc64-softmmu/hw/net/fsl_etsec/etsec.o
  CC      s390x-softmmu/disas.o
  CC      ppc64-softmmu/hw/net/fsl_etsec/registers.o
  CC      ppcemb-softmmu/hw/display/virtio-gpu-pci.o
  CC      ppc-softmmu/hw/core/generic-loader.o
  GEN     s390x-softmmu/gdbstub-xml.c
  CC      ppc64-softmmu/hw/net/fsl_etsec/rings.o
  CC      ppcemb-softmmu/hw/i2c/ppc4xx_i2c.o
  CC      ppcemb-softmmu/hw/misc/ivshmem.o
  CC      ppc-softmmu/hw/core/null-machine.o
  CC      ppc64-softmmu/hw/net/fsl_etsec/miim.o
  CC      ppc-softmmu/hw/display/sm501.o
  CC      ppcemb-softmmu/hw/misc/mmio_interface.o
  CC      ppc64-softmmu/hw/nvram/spapr_nvram.o
  CC      s390x-softmmu/arch_init.o
  CC      ppcemb-softmmu/hw/net/xilinx_ethlite.o
  CC      ppc64-softmmu/hw/scsi/spapr_vscsi.o
  CC      ppc-softmmu/hw/display/vga.o
  CC      s390x-softmmu/cpus.o
  CC      ppcemb-softmmu/hw/net/virtio-net.o
  CC      ppc64-softmmu/hw/scsi/virtio-scsi.o
  CC      s390x-softmmu/monitor.o
  CC      ppcemb-softmmu/hw/net/vhost_net.o
  CC      ppc-softmmu/hw/display/virtio-gpu.o
  CC      ppcemb-softmmu/hw/scsi/virtio-scsi.o
  CC      ppc64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      ppcemb-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      ppc64-softmmu/hw/scsi/vhost-scsi-common.o
  CC      ppc-softmmu/hw/display/virtio-gpu-3d.o
  CC      ppcemb-softmmu/hw/scsi/vhost-scsi-common.o
  CC      ppc64-softmmu/hw/scsi/vhost-scsi.o
  CC      s390x-softmmu/gdbstub.o
  CC      ppcemb-softmmu/hw/scsi/vhost-scsi.o
  CC      ppc-softmmu/hw/display/virtio-gpu-pci.o
  CC      ppc64-softmmu/hw/scsi/vhost-user-scsi.o
  CC      ppcemb-softmmu/hw/scsi/vhost-user-scsi.o
  CC      s390x-softmmu/balloon.o
  CC      ppc-softmmu/hw/i2c/ppc4xx_i2c.o
  CC      ppcemb-softmmu/hw/vfio/common.o
  CC      ppc64-softmmu/hw/timer/mc146818rtc.o
  CC      s390x-softmmu/ioport.o
  CC      ppc-softmmu/hw/misc/ivshmem.o
  CC      ppc64-softmmu/hw/vfio/common.o
  CC      s390x-softmmu/numa.o
  CC      ppcemb-softmmu/hw/vfio/pci.o
  CC      ppc-softmmu/hw/misc/mmio_interface.o
  CC      s390x-softmmu/qtest.o
  CC      ppc64-softmmu/hw/vfio/pci.o
  CC      ppc-softmmu/hw/net/xilinx_ethlite.o
  CC      s390x-softmmu/memory.o
  CC      ppc-softmmu/hw/net/virtio-net.o
  CC      ppcemb-softmmu/hw/vfio/pci-quirks.o
  CC      ppc64-softmmu/hw/vfio/pci-quirks.o
  CC      ppc-softmmu/hw/net/vhost_net.o
  CC      ppc-softmmu/hw/net/fsl_etsec/etsec.o
  CC      ppcemb-softmmu/hw/vfio/platform.o
  CC      ppc64-softmmu/hw/vfio/platform.o
  CC      s390x-softmmu/memory_mapping.o
  CC      ppc-softmmu/hw/net/fsl_etsec/registers.o
  CC      s390x-softmmu/dump.o
  CC      ppcemb-softmmu/hw/vfio/spapr.o
  CC      ppc64-softmmu/hw/vfio/spapr.o
  CC      ppc-softmmu/hw/net/fsl_etsec/rings.o
  CC      ppcemb-softmmu/hw/virtio/virtio.o
  CC      ppc64-softmmu/hw/virtio/virtio.o
  CC      ppc-softmmu/hw/net/fsl_etsec/miim.o
  CC      s390x-softmmu/migration/ram.o
  CC      ppc-softmmu/hw/scsi/virtio-scsi.o
  CC      ppcemb-softmmu/hw/virtio/virtio-balloon.o
  CC      ppc64-softmmu/hw/virtio/virtio-balloon.o
  CC      ppc-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      s390x-softmmu/accel/accel.o
  CC      ppcemb-softmmu/hw/virtio/vhost.o
  CC      ppc64-softmmu/hw/virtio/vhost.o
  CC      ppc-softmmu/hw/scsi/vhost-scsi-common.o
  CC      s390x-softmmu/accel/kvm/kvm-all.o
  CC      ppcemb-softmmu/hw/virtio/vhost-backend.o
  CC      ppc-softmmu/hw/scsi/vhost-scsi.o
  CC      ppc64-softmmu/hw/virtio/vhost-backend.o
  CC      ppcemb-softmmu/hw/virtio/vhost-user.o
  CC      ppc-softmmu/hw/scsi/vhost-user-scsi.o
  CC      s390x-softmmu/accel/stubs/hax-stub.o
  CC      ppc64-softmmu/hw/virtio/vhost-user.o
  CC      s390x-softmmu/accel/tcg/tcg-all.o
  CC      ppc-softmmu/hw/timer/mc146818rtc.o
  CC      ppcemb-softmmu/hw/virtio/vhost-vsock.o
  CC      ppc64-softmmu/hw/virtio/vhost-vsock.o
  CC      s390x-softmmu/accel/tcg/cputlb.o
  CC      ppcemb-softmmu/hw/virtio/virtio-crypto.o
  CC      ppc64-softmmu/hw/virtio/virtio-crypto.o
  CC      ppc-softmmu/hw/vfio/common.o
  CC      ppcemb-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      ppc64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      ppc-softmmu/hw/vfio/pci.o
  CC      ppcemb-softmmu/hw/ppc/ppc.o
  CC      ppc64-softmmu/hw/ppc/ppc.o
  CC      s390x-softmmu/accel/tcg/tcg-runtime.o
In file included from /var/tmp/patchew-tester-tmp-bbu6uy4t/src/hw/ppc/ppc.c:39:0:
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/target/ppc/kvm_ppc.h:161:5: error: no previous prototype for ‘kvmppc_fwnmi_enable’ [-Werror=missing-prototypes]
 int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
     ^~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/rules.mak:66: recipe for target 'hw/ppc/ppc.o' failed
make[1]: *** [hw/ppc/ppc.o] Error 1
Makefile:327: recipe for target 'subdir-ppcemb-softmmu' failed
make: *** [subdir-ppcemb-softmmu] Error 2
make: *** Waiting for unfinished jobs....
In file included from /var/tmp/patchew-tester-tmp-bbu6uy4t/src/hw/ppc/ppc.c:39:0:
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/target/ppc/kvm_ppc.h:161:5: error: no previous prototype for ‘kvmppc_fwnmi_enable’ [-Werror=missing-prototypes]
 int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
     ^~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/rules.mak:66: recipe for target 'hw/ppc/ppc.o' failed
make[1]: *** [hw/ppc/ppc.o] Error 1
make[1]: *** Waiting for unfinished jobs....
  CC      ppc64-softmmu/hw/ppc/ppc_booke.o
  CC      s390x-softmmu/accel/tcg/cpu-exec.o
In file included from /var/tmp/patchew-tester-tmp-bbu6uy4t/src/hw/ppc/ppc_booke.c:34:0:
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/target/ppc/kvm_ppc.h:161:5: error: no previous prototype for ‘kvmppc_fwnmi_enable’ [-Werror=missing-prototypes]
 int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
     ^~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/rules.mak:66: recipe for target 'hw/ppc/ppc_booke.o' failed
make[1]: *** [hw/ppc/ppc_booke.o] Error 1
Makefile:327: recipe for target 'subdir-ppc64-softmmu' failed
make: *** [subdir-ppc64-softmmu] Error 2
  CC      s390x-softmmu/accel/tcg/cpu-exec-common.o
  CC      ppc-softmmu/hw/vfio/pci-quirks.o
  CC      s390x-softmmu/accel/tcg/translate-all.o
  CC      ppc-softmmu/hw/vfio/platform.o
  CC      ppc-softmmu/hw/vfio/spapr.o
  CC      s390x-softmmu/accel/tcg/translator.o
  CC      s390x-softmmu/hw/9pfs/virtio-9p-device.o
  CC      ppc-softmmu/hw/virtio/virtio-balloon.o
  CC      ppc-softmmu/hw/virtio/virtio.o
  CC      s390x-softmmu/hw/block/virtio-blk.o
  CC      ppc-softmmu/hw/virtio/vhost.o
  CC      s390x-softmmu/hw/block/dataplane/virtio-blk.o
  CC      s390x-softmmu/hw/char/virtio-serial-bus.o
  CC      s390x-softmmu/hw/char/terminal3270.o
  CC      ppc-softmmu/hw/virtio/vhost-backend.o
  CC      ppc-softmmu/hw/virtio/vhost-user.o
  CC      s390x-softmmu/hw/core/generic-loader.o
  CC      s390x-softmmu/hw/core/null-machine.o
  CC      ppc-softmmu/hw/virtio/vhost-vsock.o
  CC      s390x-softmmu/hw/display/virtio-gpu.o
  CC      s390x-softmmu/hw/display/virtio-gpu-3d.o
  CC      ppc-softmmu/hw/virtio/virtio-crypto.o
  CC      ppc-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      ppc-softmmu/hw/ppc/ppc.o
  CC      ppc-softmmu/hw/ppc/ppc_booke.o
  CC      s390x-softmmu/hw/display/virtio-gpu-pci.o
  CC      s390x-softmmu/hw/intc/s390_flic.o
In file included from /var/tmp/patchew-tester-tmp-bbu6uy4t/src/hw/ppc/ppc_booke.c:34:0:
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/target/ppc/kvm_ppc.h:161:5: error: no previous prototype for ‘kvmppc_fwnmi_enable’ [-Werror=missing-prototypes]
 int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
     ^~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/rules.mak:66: recipe for target 'hw/ppc/ppc_booke.o' failed
make[1]: *** [hw/ppc/ppc_booke.o] Error 1
make[1]: *** Waiting for unfinished jobs....
  CC      s390x-softmmu/hw/intc/s390_flic_kvm.o
In file included from /var/tmp/patchew-tester-tmp-bbu6uy4t/src/hw/ppc/ppc.c:39:0:
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/target/ppc/kvm_ppc.h:161:5: error: no previous prototype for ‘kvmppc_fwnmi_enable’ [-Werror=missing-prototypes]
 int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
     ^~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/var/tmp/patchew-tester-tmp-bbu6uy4t/src/rules.mak:66: recipe for target 'hw/ppc/ppc.o' failed
make[1]: *** [hw/ppc/ppc.o] Error 1
Makefile:327: recipe for target 'subdir-ppc-softmmu' failed
make: *** [subdir-ppc-softmmu] Error 2
  CC      s390x-softmmu/hw/misc/mmio_interface.o
  CC      s390x-softmmu/hw/net/virtio-net.o
  CC      s390x-softmmu/hw/net/vhost_net.o
  CC      s390x-softmmu/hw/net/rocker/qmp-norocker.o
  CC      s390x-softmmu/hw/scsi/virtio-scsi.o
  CC      s390x-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      s390x-softmmu/hw/scsi/vhost-scsi-common.o
  CC      s390x-softmmu/hw/scsi/vhost-scsi.o
  CC      s390x-softmmu/hw/scsi/vhost-user-scsi.o
  CC      s390x-softmmu/hw/vfio/common.o
  CC      s390x-softmmu/hw/vfio/pci.o
  CC      s390x-softmmu/hw/vfio/pci-quirks.o
  CC      s390x-softmmu/hw/vfio/ccw.o
  CC      s390x-softmmu/hw/vfio/platform.o
  CC      s390x-softmmu/hw/vfio/spapr.o
  CC      s390x-softmmu/hw/virtio/virtio.o
  CC      s390x-softmmu/hw/virtio/virtio-balloon.o
  CC      s390x-softmmu/hw/virtio/vhost.o
  CC      s390x-softmmu/hw/virtio/vhost-backend.o
  CC      s390x-softmmu/hw/virtio/vhost-user.o
  CC      s390x-softmmu/hw/virtio/vhost-vsock.o
  CC      s390x-softmmu/hw/virtio/virtio-crypto.o
  CC      s390x-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      s390x-softmmu/hw/s390x/s390-virtio-hcall.o
  CC      s390x-softmmu/hw/s390x/sclp.o
  CC      s390x-softmmu/hw/s390x/event-facility.o
  CC      s390x-softmmu/hw/s390x/sclpquiesce.o
  CC      s390x-softmmu/hw/s390x/sclpcpu.o
  CC      s390x-softmmu/hw/s390x/ipl.o
  CC      s390x-softmmu/hw/s390x/css.o
  CC      s390x-softmmu/hw/s390x/s390-virtio-ccw.o
  CC      s390x-softmmu/hw/s390x/3270-ccw.o
  CC      s390x-softmmu/hw/s390x/virtio-ccw.o
  CC      s390x-softmmu/hw/s390x/css-bridge.o
  CC      s390x-softmmu/hw/s390x/ccw-device.o
  CC      s390x-softmmu/hw/s390x/s390-pci-bus.o
  CC      s390x-softmmu/hw/s390x/s390-pci-inst.o
  CC      s390x-softmmu/hw/s390x/s390-skeys.o
  CC      s390x-softmmu/hw/s390x/s390-stattrib.o
  CC      s390x-softmmu/hw/s390x/s390-skeys-kvm.o
  CC      s390x-softmmu/hw/s390x/s390-stattrib-kvm.o
  CC      s390x-softmmu/hw/s390x/s390-ccw.o
  CC      s390x-softmmu/target/s390x/cpu.o
  CC      s390x-softmmu/target/s390x/cpu_models.o
  CC      s390x-softmmu/target/s390x/cpu_features.o
  CC      s390x-softmmu/target/s390x/gdbstub.o
  CC      s390x-softmmu/target/s390x/interrupt.o
  CC      s390x-softmmu/target/s390x/helper.o
  CC      s390x-softmmu/target/s390x/translate.o
  CC      s390x-softmmu/target/s390x/excp_helper.o
  CC      s390x-softmmu/target/s390x/cc_helper.o
  CC      s390x-softmmu/target/s390x/fpu_helper.o
  CC      s390x-softmmu/target/s390x/int_helper.o
  CC      s390x-softmmu/target/s390x/mem_helper.o
  CC      s390x-softmmu/target/s390x/misc_helper.o
  CC      s390x-softmmu/target/s390x/machine.o
  CC      s390x-softmmu/target/s390x/ioinst.o
  CC      s390x-softmmu/target/s390x/arch_dump.o
  CC      s390x-softmmu/target/s390x/mmu_helper.o
  CC      s390x-softmmu/target/s390x/diag.o
  CC      s390x-softmmu/target/s390x/kvm.o
  GEN     trace/generated-helpers.c
  CC      s390x-softmmu/trace/control-target.o
  CC      s390x-softmmu/gdbstub-xml.o
  CC      s390x-softmmu/trace/generated-helpers.o
  LINK    s390x-softmmu/qemu-system-s390x
=== OUTPUT END ===

Test command exited with code: 2


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org

Re: [Qemu-devel] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4370 bytes --]

On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
> Receive updates from SLOF about the updated rtas-base.
> A separate patch for SLOF [1] (commit f9a60de3) adds
> functionality to invoke a private HCALL whenever OS
> issues instantiate-rtas with a new rtas-base.
> 
> This is required as QEMU needs to know the updated rtas-base
> as it allocates error reporting structure in RTAS space upon
> a machine check exception.
> 
> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
> 
> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

Ao I acked this earlier, but I've now realized there might be some
connection between this and discussions taking place elsewhere about
qemu not knowing what SLOF does with the device tree.

At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
the time of instantiate-rtas, is that right?

Does SLOF put the RTAS blob address in its internal device tree, or
does it only pass it to the guest via the return parameters from
instantiate-rtas?

> ---
>  hw/ppc/spapr.c         |   11 +++++++++++
>  hw/ppc/spapr_hcall.c   |    8 ++++++++
>  include/hw/ppc/spapr.h |    4 +++-
>  3 files changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index ff87f15..5deae30 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1675,6 +1675,16 @@ static const VMStateDescription vmstate_spapr_patb_entry = {
>      },
>  };
>  
> +static const VMStateDescription vmstate_spapr_rtas_addr = {
> +    .name = "spapr_rtas_addr",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT64(rtas_addr, sPAPRMachineState),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
>  static const VMStateDescription vmstate_spapr = {
>      .name = "spapr",
>      .version_id = 3,
> @@ -1694,6 +1704,7 @@ static const VMStateDescription vmstate_spapr = {
>          &vmstate_spapr_ov5_cas,
>          &vmstate_spapr_patb_entry,
>          &vmstate_spapr_pending_events,
> +        &vmstate_spapr_rtas_addr,
>          NULL
>      }
>  };
> diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
> index 8d72bb7..c15a93c 100644
> --- a/hw/ppc/spapr_hcall.c
> +++ b/hw/ppc/spapr_hcall.c
> @@ -1088,6 +1088,13 @@ static target_ulong h_rtas(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>                             nret, rtas_r3 + 12 + 4*nargs);
>  }
>  
> +static target_ulong h_rtas_update(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> +                                  target_ulong opcode, target_ulong *args)
> +{
> +    spapr->rtas_addr = args[0];
> +    return 0;
> +}
> +
>  static target_ulong h_logical_load(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>                                     target_ulong opcode, target_ulong *args)
>  {
> @@ -1750,6 +1757,7 @@ static void hypercall_register_types(void)
>  
>      /* qemu/KVM-PPC specific hcalls */
>      spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
> +    spapr_register_hypercall(KVMPPC_H_RTAS_UPDATE, h_rtas_update);
>  
>      /* ibm,client-architecture-support support */
>      spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index c1b365f..b395aa7 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -90,6 +90,7 @@ struct sPAPRMachineState {
>  
>      hwaddr rma_size;
>      int vrma_adjust;
> +    hwaddr rtas_addr;
>      ssize_t rtas_size;
>      void *rtas_blob;
>      long kernel_size;
> @@ -400,7 +401,8 @@ struct sPAPRMachineState {
>  #define KVMPPC_H_LOGICAL_MEMOP  (KVMPPC_HCALL_BASE + 0x1)
>  /* Client Architecture support */
>  #define KVMPPC_H_CAS            (KVMPPC_HCALL_BASE + 0x2)
> -#define KVMPPC_HCALL_MAX        KVMPPC_H_CAS
> +#define KVMPPC_H_RTAS_UPDATE    (KVMPPC_HCALL_BASE + 0x3)
> +#define KVMPPC_HCALL_MAX        KVMPPC_H_RTAS_UPDATE
>  
>  typedef struct sPAPRDeviceTreeUpdateHeader {
>      uint32_t version_id;
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v5 2/6] ppc: spapr: Handle "ibm, nmi-register" and "ibm, nmi-interlock" RTAS calls

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 6699 bytes --]

I don't suppose there's a way to stop your mailer from inserting
spaces after the commas in the subject line,


On Thu, Sep 28, 2017 at 04:07:48PM +0530, Aravinda Prasad wrote:
> This patch adds support in QEMU to handle "ibm,nmi-register"
> and "ibm,nmi-interlock" RTAS calls.
> 
> The machine check notification address is saved when the
> OS issues "ibm,nmi-register" RTAS call.
> 
> This patch also handles the case when multiple processors
> experience machine check at or about the same time by
> handling "ibm,nmi-interlock" call. In such cases, as per
> PAPR, subsequent processors serialize waiting for the first
> processor to issue the "ibm,nmi-interlock" call. The second
> processor waits till the first processor, which also
> received a machine check error, is done reading the error
> log. The first processor issues "ibm,nmi-interlock" call
> when the error log is consumed. This patch implements the
> releasing part of the error-log while subsequent patch
> (which builds error log) handles the locking part.
> 
> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> ---
>  hw/ppc/spapr.c         |    8 ++++++++
>  hw/ppc/spapr_rtas.c    |   35 +++++++++++++++++++++++++++++++++++
>  include/hw/ppc/spapr.h |   11 ++++++++++-
>  3 files changed, 53 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 5deae30..d568ea6 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1477,6 +1477,11 @@ static void ppc_spapr_reset(void)
>      first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
>  
>      spapr->cas_reboot = false;
> +
> +    spapr->mc_status = -1;
> +    spapr->guest_machine_check_addr = 0;

You should probably use -1 as the default for this - strictly speaking
the guest could legitimately choose to have it's machine check
processing code at 0, though it would be a strange choice.

> +    qemu_cond_destroy(&spapr->mc_delivery_cond);
> +    qemu_cond_init(&spapr->mc_delivery_cond);

What will this do if one of the vcpus is waiting on the condition
variable right now?  I suspect you're going to need to first wake up
the queue, and marshal any blocked vcpus back into dead mode instead
of this.

>  }
>  
>  static void spapr_create_nvram(sPAPRMachineState *spapr)
> @@ -2598,6 +2603,9 @@ static void ppc_spapr_init(MachineState *machine)
>  
>          kvmppc_spapr_enable_inkernel_multitce();
>      }
> +
> +    spapr->mc_status = -1;
> +    qemu_cond_init(&spapr->mc_delivery_cond);
>  }
>  
>  static int spapr_kvm_type(const char *vm_type)
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index cdf0b60..08e9a5e 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -348,6 +348,37 @@ static void rtas_get_power_level(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>      rtas_st(rets, 1, 100);
>  }
>  
> +static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
> +                                  sPAPRMachineState *spapr,
> +                                  uint32_t token, uint32_t nargs,
> +                                  target_ulong args,
> +                                  uint32_t nret, target_ulong rets)
> +{
> +    spapr->guest_machine_check_addr = rtas_ld(args, 1);
> +    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
> +}
> +
> +static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
> +                                   sPAPRMachineState *spapr,
> +                                   uint32_t token, uint32_t nargs,
> +                                   target_ulong args,
> +                                   uint32_t nret, target_ulong rets)
> +{
> +    if (!spapr->guest_machine_check_addr) {
> +        /* NMI register not called */
> +        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> +    } else {
> +        /*
> +         * VCPU issuing "ibm,nmi-interlock" is done with NMI handling,
> +         * hence unset mc_status.
> +         */
> +        spapr->mc_status = -1;
> +        qemu_cond_signal(&spapr->mc_delivery_cond);
> +        rtas_st(rets, 0, RTAS_OUT_SUCCESS);
> +    }
> +}
> +
> +
>  static struct rtas_call {
>      const char *name;
>      spapr_rtas_fn fn;
> @@ -489,6 +520,10 @@ static void core_rtas_register_types(void)
>                          rtas_set_power_level);
>      spapr_rtas_register(RTAS_GET_POWER_LEVEL, "get-power-level",
>                          rtas_get_power_level);
> +    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
> +                        rtas_ibm_nmi_register);
> +    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
> +                        rtas_ibm_nmi_interlock);
>  }
>  
>  type_init(core_rtas_register_types)
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index b395aa7..28b6e2e 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -124,6 +124,13 @@ struct sPAPRMachineState {
>       * occurs during the unplug process. */
>      QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
>  
> +    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
> +    target_ulong guest_machine_check_addr;

You need to migrate this.  This is separate and simpler from the
problems of migrating during machine check handling - at the moment if
the guest calls nmi-register, then you migrate, then a machine check
is triggered you will have lost the information on how to handle it
properly.


> +    /* mc_status is set to -1 if mc is not in progress, else is set to the CPU
> +     * handling the mc. */
> +    int mc_status;
> +    QemuCond mc_delivery_cond;
> +
>      /*< public >*/
>      char *kvm_type;
>      MemoryHotplugState hotplug_memory;
> @@ -520,8 +527,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
>  #define RTAS_IBM_CREATE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x27)
>  #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
>  #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
> +#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2A)
> +#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2B)
>  
> -#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2A)
> +#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2C)
>  
>  /* RTAS ibm,get-system-parameter token values */
>  #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[Nikunj A Dadhania]

David Gibson <david@gibson.dropbear.id.au> writes:

> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
>> Receive updates from SLOF about the updated rtas-base.
>> A separate patch for SLOF [1] (commit f9a60de3) adds
>> functionality to invoke a private HCALL whenever OS
>> issues instantiate-rtas with a new rtas-base.
>> 
>> This is required as QEMU needs to know the updated rtas-base
>> as it allocates error reporting structure in RTAS space upon
>> a machine check exception.
>> 
>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
>> 
>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
>
> Ao I acked this earlier, but I've now realized there might be some
> connection between this and discussions taking place elsewhere about
> qemu not knowing what SLOF does with the device tree.
>
> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
> the time of instantiate-rtas, is that right?

The call happens from
arch/powerpc/kernel/prom_init.c:prom_instantiate_rtas() and after that
linux kernel makes two entries in the DT

....
       if (call_prom_ret("call-method", 3, 2, &entry,
                          ADDR("instantiate-rtas"),
                          rtas_inst, base) != 0
            || entry == 0) {
                prom_printf(" failed\n");
                return;
        }
        prom_printf(" done\n");

        reserve_mem(base, size);

        val = cpu_to_be32(base);
        prom_setprop(rtas_node, "/rtas", "linux,rtas-base",
                     &val, sizeof(val));
        val = cpu_to_be32(entry);
        prom_setprop(rtas_node, "/rtas", "linux,rtas-entry",
                     &val, sizeof(val));
....

Quiesce is called after this. 

> Does SLOF put the RTAS blob address in its internal device tree, or
> does it only pass it to the guest via the return parameters from
> instantiate-rtas?

Entry was made to the DT by linux kernel prom_init code, will this be
visible to QEMU?

Regards,
Nikunj

Re: [Qemu-devel] [Qemu-ppc] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[Alexey Kardashevskiy]

On 29/09/17 21:52, Nikunj A Dadhania wrote:
> David Gibson <david@gibson.dropbear.id.au> writes:
> 
>> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
>>> Receive updates from SLOF about the updated rtas-base.
>>> A separate patch for SLOF [1] (commit f9a60de3) adds
>>> functionality to invoke a private HCALL whenever OS
>>> issues instantiate-rtas with a new rtas-base.
>>>
>>> This is required as QEMU needs to know the updated rtas-base
>>> as it allocates error reporting structure in RTAS space upon
>>> a machine check exception.
>>>
>>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
>>>
>>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
>>
>> Ao I acked this earlier, but I've now realized there might be some
>> connection between this and discussions taking place elsewhere about
>> qemu not knowing what SLOF does with the device tree.
>>
>> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
>> the time of instantiate-rtas, is that right?
> 
> The call happens from
> arch/powerpc/kernel/prom_init.c:prom_instantiate_rtas() and after that
> linux kernel makes two entries in the DT
> 
> ....
>        if (call_prom_ret("call-method", 3, 2, &entry,
>                           ADDR("instantiate-rtas"),
>                           rtas_inst, base) != 0
>             || entry == 0) {
>                 prom_printf(" failed\n");
>                 return;
>         }
>         prom_printf(" done\n");
> 
>         reserve_mem(base, size);
> 
>         val = cpu_to_be32(base);
>         prom_setprop(rtas_node, "/rtas", "linux,rtas-base",
>                      &val, sizeof(val));
>         val = cpu_to_be32(entry);
>         prom_setprop(rtas_node, "/rtas", "linux,rtas-entry",
>                      &val, sizeof(val));
> ....
> 
> Quiesce is called after this. 
> 
>> Does SLOF put the RTAS blob address in its internal device tree, or
>> does it only pass it to the guest via the return parameters from
>> instantiate-rtas?
> 
> Entry was made to the DT by linux kernel prom_init code, will this be
> visible to QEMU?

With my recent SLOF FDT patch - yes:

aik@fstn1-p1:~$ grep rtas dbg.dts
	rtas {
		linux,rtas-entry = <0x2fff0000>;
		linux,rtas-base = <0x2fff0000>;
[...]


-- 
Alexey

Re: [Qemu-devel] [PATCH v5 2/6] ppc: spapr: Handle "ibm, nmi-register" and "ibm, nmi-interlock" RTAS calls

[Aravinda Prasad]

On Friday 29 September 2017 12:19 PM, David Gibson wrote:
> I don't suppose there's a way to stop your mailer from inserting
> spaces after the commas in the subject line,

Not sure how to stop the mailer from doing this as I don't have any
spaces in the subject line of my actual patch.

> 
> 
> On Thu, Sep 28, 2017 at 04:07:48PM +0530, Aravinda Prasad wrote:
>> This patch adds support in QEMU to handle "ibm,nmi-register"
>> and "ibm,nmi-interlock" RTAS calls.
>>
>> The machine check notification address is saved when the
>> OS issues "ibm,nmi-register" RTAS call.
>>
>> This patch also handles the case when multiple processors
>> experience machine check at or about the same time by
>> handling "ibm,nmi-interlock" call. In such cases, as per
>> PAPR, subsequent processors serialize waiting for the first
>> processor to issue the "ibm,nmi-interlock" call. The second
>> processor waits till the first processor, which also
>> received a machine check error, is done reading the error
>> log. The first processor issues "ibm,nmi-interlock" call
>> when the error log is consumed. This patch implements the
>> releasing part of the error-log while subsequent patch
>> (which builds error log) handles the locking part.
>>
>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>> ---
>>  hw/ppc/spapr.c         |    8 ++++++++
>>  hw/ppc/spapr_rtas.c    |   35 +++++++++++++++++++++++++++++++++++
>>  include/hw/ppc/spapr.h |   11 ++++++++++-
>>  3 files changed, 53 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
>> index 5deae30..d568ea6 100644
>> --- a/hw/ppc/spapr.c
>> +++ b/hw/ppc/spapr.c
>> @@ -1477,6 +1477,11 @@ static void ppc_spapr_reset(void)
>>      first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
>>  
>>      spapr->cas_reboot = false;
>> +
>> +    spapr->mc_status = -1;
>> +    spapr->guest_machine_check_addr = 0;
> 
> You should probably use -1 as the default for this - strictly speaking
> the guest could legitimately choose to have it's machine check
> processing code at 0, though it would be a strange choice.

Sure.

> 
>> +    qemu_cond_destroy(&spapr->mc_delivery_cond);
>> +    qemu_cond_init(&spapr->mc_delivery_cond);
> 
> What will this do if one of the vcpus is waiting on the condition
> variable right now?  I suspect you're going to need to first wake up
> the queue, and marshal any blocked vcpus back into dead mode instead
> of this.

Do we need to wake-up the queue as the system, anyway, is being reset?

> 
>>  }
>>  
>>  static void spapr_create_nvram(sPAPRMachineState *spapr)
>> @@ -2598,6 +2603,9 @@ static void ppc_spapr_init(MachineState *machine)
>>  
>>          kvmppc_spapr_enable_inkernel_multitce();
>>      }
>> +
>> +    spapr->mc_status = -1;
>> +    qemu_cond_init(&spapr->mc_delivery_cond);
>>  }
>>  
>>  static int spapr_kvm_type(const char *vm_type)
>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>> index cdf0b60..08e9a5e 100644
>> --- a/hw/ppc/spapr_rtas.c
>> +++ b/hw/ppc/spapr_rtas.c
>> @@ -348,6 +348,37 @@ static void rtas_get_power_level(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>>      rtas_st(rets, 1, 100);
>>  }
>>  
>> +static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
>> +                                  sPAPRMachineState *spapr,
>> +                                  uint32_t token, uint32_t nargs,
>> +                                  target_ulong args,
>> +                                  uint32_t nret, target_ulong rets)
>> +{
>> +    spapr->guest_machine_check_addr = rtas_ld(args, 1);
>> +    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>> +}
>> +
>> +static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
>> +                                   sPAPRMachineState *spapr,
>> +                                   uint32_t token, uint32_t nargs,
>> +                                   target_ulong args,
>> +                                   uint32_t nret, target_ulong rets)
>> +{
>> +    if (!spapr->guest_machine_check_addr) {
>> +        /* NMI register not called */
>> +        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
>> +    } else {
>> +        /*
>> +         * VCPU issuing "ibm,nmi-interlock" is done with NMI handling,
>> +         * hence unset mc_status.
>> +         */
>> +        spapr->mc_status = -1;
>> +        qemu_cond_signal(&spapr->mc_delivery_cond);
>> +        rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>> +    }
>> +}
>> +
>> +
>>  static struct rtas_call {
>>      const char *name;
>>      spapr_rtas_fn fn;
>> @@ -489,6 +520,10 @@ static void core_rtas_register_types(void)
>>                          rtas_set_power_level);
>>      spapr_rtas_register(RTAS_GET_POWER_LEVEL, "get-power-level",
>>                          rtas_get_power_level);
>> +    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
>> +                        rtas_ibm_nmi_register);
>> +    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
>> +                        rtas_ibm_nmi_interlock);
>>  }
>>  
>>  type_init(core_rtas_register_types)
>> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
>> index b395aa7..28b6e2e 100644
>> --- a/include/hw/ppc/spapr.h
>> +++ b/include/hw/ppc/spapr.h
>> @@ -124,6 +124,13 @@ struct sPAPRMachineState {
>>       * occurs during the unplug process. */
>>      QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
>>  
>> +    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
>> +    target_ulong guest_machine_check_addr;
> 
> You need to migrate this.  This is separate and simpler from the
> problems of migrating during machine check handling - at the moment if
> the guest calls nmi-register, then you migrate, then a machine check
> is triggered you will have lost the information on how to handle it
> properly.

I missed it. Will include.

Regards,
Aravinda

> 
> 
>> +    /* mc_status is set to -1 if mc is not in progress, else is set to the CPU
>> +     * handling the mc. */
>> +    int mc_status;
>> +    QemuCond mc_delivery_cond;
>> +
>>      /*< public >*/
>>      char *kvm_type;
>>      MemoryHotplugState hotplug_memory;
>> @@ -520,8 +527,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
>>  #define RTAS_IBM_CREATE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x27)
>>  #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
>>  #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
>> +#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2A)
>> +#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2B)
>>  
>> -#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2A)
>> +#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2C)
>>  
>>  /* RTAS ibm,get-system-parameter token values */
>>  #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
>>
> 

-- 
Regards,
Aravinda

Re: [Qemu-devel] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[Aravinda Prasad]

On Friday 29 September 2017 11:47 AM, David Gibson wrote:
> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
>> Receive updates from SLOF about the updated rtas-base.
>> A separate patch for SLOF [1] (commit f9a60de3) adds
>> functionality to invoke a private HCALL whenever OS
>> issues instantiate-rtas with a new rtas-base.
>>
>> This is required as QEMU needs to know the updated rtas-base
>> as it allocates error reporting structure in RTAS space upon
>> a machine check exception.
>>
>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
>>
>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> 
> Ao I acked this earlier, but I've now realized there might be some
> connection between this and discussions taking place elsewhere about
> qemu not knowing what SLOF does with the device tree.
> 
> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
> the time of instantiate-rtas, is that right?
> 
> Does SLOF put the RTAS blob address in its internal device tree, or
> does it only pass it to the guest via the return parameters from
> instantiate-rtas?

The new set of patches posted by Alexey exports RTAS address via device
tree to QEMU. Once this is in, I will remove this patch and rebase the
rest of the patches.

Regards,
Aravinda

> 
>> ---
>>  hw/ppc/spapr.c         |   11 +++++++++++
>>  hw/ppc/spapr_hcall.c   |    8 ++++++++
>>  include/hw/ppc/spapr.h |    4 +++-
>>  3 files changed, 22 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
>> index ff87f15..5deae30 100644
>> --- a/hw/ppc/spapr.c
>> +++ b/hw/ppc/spapr.c
>> @@ -1675,6 +1675,16 @@ static const VMStateDescription vmstate_spapr_patb_entry = {
>>      },
>>  };
>>  
>> +static const VMStateDescription vmstate_spapr_rtas_addr = {
>> +    .name = "spapr_rtas_addr",
>> +    .version_id = 1,
>> +    .minimum_version_id = 1,
>> +    .fields = (VMStateField[]) {
>> +        VMSTATE_UINT64(rtas_addr, sPAPRMachineState),
>> +        VMSTATE_END_OF_LIST()
>> +    },
>> +};
>> +
>>  static const VMStateDescription vmstate_spapr = {
>>      .name = "spapr",
>>      .version_id = 3,
>> @@ -1694,6 +1704,7 @@ static const VMStateDescription vmstate_spapr = {
>>          &vmstate_spapr_ov5_cas,
>>          &vmstate_spapr_patb_entry,
>>          &vmstate_spapr_pending_events,
>> +        &vmstate_spapr_rtas_addr,
>>          NULL
>>      }
>>  };
>> diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
>> index 8d72bb7..c15a93c 100644
>> --- a/hw/ppc/spapr_hcall.c
>> +++ b/hw/ppc/spapr_hcall.c
>> @@ -1088,6 +1088,13 @@ static target_ulong h_rtas(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>>                             nret, rtas_r3 + 12 + 4*nargs);
>>  }
>>  
>> +static target_ulong h_rtas_update(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>> +                                  target_ulong opcode, target_ulong *args)
>> +{
>> +    spapr->rtas_addr = args[0];
>> +    return 0;
>> +}
>> +
>>  static target_ulong h_logical_load(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>>                                     target_ulong opcode, target_ulong *args)
>>  {
>> @@ -1750,6 +1757,7 @@ static void hypercall_register_types(void)
>>  
>>      /* qemu/KVM-PPC specific hcalls */
>>      spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
>> +    spapr_register_hypercall(KVMPPC_H_RTAS_UPDATE, h_rtas_update);
>>  
>>      /* ibm,client-architecture-support support */
>>      spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
>> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
>> index c1b365f..b395aa7 100644
>> --- a/include/hw/ppc/spapr.h
>> +++ b/include/hw/ppc/spapr.h
>> @@ -90,6 +90,7 @@ struct sPAPRMachineState {
>>  
>>      hwaddr rma_size;
>>      int vrma_adjust;
>> +    hwaddr rtas_addr;
>>      ssize_t rtas_size;
>>      void *rtas_blob;
>>      long kernel_size;
>> @@ -400,7 +401,8 @@ struct sPAPRMachineState {
>>  #define KVMPPC_H_LOGICAL_MEMOP  (KVMPPC_HCALL_BASE + 0x1)
>>  /* Client Architecture support */
>>  #define KVMPPC_H_CAS            (KVMPPC_HCALL_BASE + 0x2)
>> -#define KVMPPC_HCALL_MAX        KVMPPC_H_CAS
>> +#define KVMPPC_H_RTAS_UPDATE    (KVMPPC_HCALL_BASE + 0x3)
>> +#define KVMPPC_HCALL_MAX        KVMPPC_H_RTAS_UPDATE
>>  
>>  typedef struct sPAPRDeviceTreeUpdateHeader {
>>      uint32_t version_id;
>>
> 

-- 
Regards,
Aravinda

Re: [Qemu-devel] [Qemu-ppc] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3343 bytes --]

On Mon, Oct 02, 2017 at 02:02:19PM +1100, Alexey Kardashevskiy wrote:
> On 29/09/17 21:52, Nikunj A Dadhania wrote:
> > David Gibson <david@gibson.dropbear.id.au> writes:
> > 
> >> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
> >>> Receive updates from SLOF about the updated rtas-base.
> >>> A separate patch for SLOF [1] (commit f9a60de3) adds
> >>> functionality to invoke a private HCALL whenever OS
> >>> issues instantiate-rtas with a new rtas-base.
> >>>
> >>> This is required as QEMU needs to know the updated rtas-base
> >>> as it allocates error reporting structure in RTAS space upon
> >>> a machine check exception.
> >>>
> >>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
> >>>
> >>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> >>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> >>
> >> Ao I acked this earlier, but I've now realized there might be some
> >> connection between this and discussions taking place elsewhere about
> >> qemu not knowing what SLOF does with the device tree.
> >>
> >> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
> >> the time of instantiate-rtas, is that right?
> > 
> > The call happens from
> > arch/powerpc/kernel/prom_init.c:prom_instantiate_rtas() and after that
> > linux kernel makes two entries in the DT
> > 
> > ....
> >        if (call_prom_ret("call-method", 3, 2, &entry,
> >                           ADDR("instantiate-rtas"),
> >                           rtas_inst, base) != 0
> >             || entry == 0) {
> >                 prom_printf(" failed\n");
> >                 return;
> >         }
> >         prom_printf(" done\n");
> > 
> >         reserve_mem(base, size);
> > 
> >         val = cpu_to_be32(base);
> >         prom_setprop(rtas_node, "/rtas", "linux,rtas-base",
> >                      &val, sizeof(val));
> >         val = cpu_to_be32(entry);
> >         prom_setprop(rtas_node, "/rtas", "linux,rtas-entry",
> >                      &val, sizeof(val));
> > ....
> > 
> > Quiesce is called after this. 
> > 
> >> Does SLOF put the RTAS blob address in its internal device tree, or
> >> does it only pass it to the guest via the return parameters from
> >> instantiate-rtas?
> > 
> > Entry was made to the DT by linux kernel prom_init code, will this be
> > visible to QEMU?
> 
> With my recent SLOF FDT patch - yes:
> 
> aik@fstn1-p1:~$ grep rtas dbg.dts
> 	rtas {
> 		linux,rtas-entry = <0x2fff0000>;
> 		linux,rtas-base = <0x2fff0000>;
> [...]

Ah.. except.. isn't that relying on the kernel putting the RTAS
address into the device tree before it calls quiesce and kills SLOF?

The SLOF image is bundled in with qemu, so it's ok for us to rely on
its behaviour up to a point.  It's not really ok for us to rely on the
kernel's behaviour here, unless that behaviour is mandated by PAPR,
which this isn't.

So, I think we either need to have *SLOF* update the device tree with
that address at instantiate-rtas time, or we'll need to resurrect
Aravinda's original UPDATE_RTAS hcall.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v5 2/6] ppc: spapr: Handle "ibm, nmi-register" and "ibm, nmi-interlock" RTAS calls

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 7858 bytes --]

On Tue, Oct 03, 2017 at 11:21:27AM +0530, Aravinda Prasad wrote:
> 
> 
> On Friday 29 September 2017 12:19 PM, David Gibson wrote:
> > I don't suppose there's a way to stop your mailer from inserting
> > spaces after the commas in the subject line,
> 
> Not sure how to stop the mailer from doing this as I don't have any
> spaces in the subject line of my actual patch.
> 
> > 
> > 
> > On Thu, Sep 28, 2017 at 04:07:48PM +0530, Aravinda Prasad wrote:
> >> This patch adds support in QEMU to handle "ibm,nmi-register"
> >> and "ibm,nmi-interlock" RTAS calls.
> >>
> >> The machine check notification address is saved when the
> >> OS issues "ibm,nmi-register" RTAS call.
> >>
> >> This patch also handles the case when multiple processors
> >> experience machine check at or about the same time by
> >> handling "ibm,nmi-interlock" call. In such cases, as per
> >> PAPR, subsequent processors serialize waiting for the first
> >> processor to issue the "ibm,nmi-interlock" call. The second
> >> processor waits till the first processor, which also
> >> received a machine check error, is done reading the error
> >> log. The first processor issues "ibm,nmi-interlock" call
> >> when the error log is consumed. This patch implements the
> >> releasing part of the error-log while subsequent patch
> >> (which builds error log) handles the locking part.
> >>
> >> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> >> ---
> >>  hw/ppc/spapr.c         |    8 ++++++++
> >>  hw/ppc/spapr_rtas.c    |   35 +++++++++++++++++++++++++++++++++++
> >>  include/hw/ppc/spapr.h |   11 ++++++++++-
> >>  3 files changed, 53 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> >> index 5deae30..d568ea6 100644
> >> --- a/hw/ppc/spapr.c
> >> +++ b/hw/ppc/spapr.c
> >> @@ -1477,6 +1477,11 @@ static void ppc_spapr_reset(void)
> >>      first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
> >>  
> >>      spapr->cas_reboot = false;
> >> +
> >> +    spapr->mc_status = -1;
> >> +    spapr->guest_machine_check_addr = 0;
> > 
> > You should probably use -1 as the default for this - strictly speaking
> > the guest could legitimately choose to have it's machine check
> > processing code at 0, though it would be a strange choice.
> 
> Sure.
> 
> > 
> >> +    qemu_cond_destroy(&spapr->mc_delivery_cond);
> >> +    qemu_cond_init(&spapr->mc_delivery_cond);
> > 
> > What will this do if one of the vcpus is waiting on the condition
> > variable right now?  I suspect you're going to need to first wake up
> > the queue, and marshal any blocked vcpus back into dead mode instead
> > of this.
> 
> Do we need to wake-up the queue as the system, anyway, is being
> reset?

The _virtual_ system is being reset.  These wait queues are in the
qemu process and exist outside the virtual system.  They won't be
reset with the virtual system unless we write the code to make them be
reset.


> 
> > 
> >>  }
> >>  
> >>  static void spapr_create_nvram(sPAPRMachineState *spapr)
> >> @@ -2598,6 +2603,9 @@ static void ppc_spapr_init(MachineState *machine)
> >>  
> >>          kvmppc_spapr_enable_inkernel_multitce();
> >>      }
> >> +
> >> +    spapr->mc_status = -1;
> >> +    qemu_cond_init(&spapr->mc_delivery_cond);
> >>  }
> >>  
> >>  static int spapr_kvm_type(const char *vm_type)
> >> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> >> index cdf0b60..08e9a5e 100644
> >> --- a/hw/ppc/spapr_rtas.c
> >> +++ b/hw/ppc/spapr_rtas.c
> >> @@ -348,6 +348,37 @@ static void rtas_get_power_level(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> >>      rtas_st(rets, 1, 100);
> >>  }
> >>  
> >> +static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
> >> +                                  sPAPRMachineState *spapr,
> >> +                                  uint32_t token, uint32_t nargs,
> >> +                                  target_ulong args,
> >> +                                  uint32_t nret, target_ulong rets)
> >> +{
> >> +    spapr->guest_machine_check_addr = rtas_ld(args, 1);
> >> +    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
> >> +}
> >> +
> >> +static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
> >> +                                   sPAPRMachineState *spapr,
> >> +                                   uint32_t token, uint32_t nargs,
> >> +                                   target_ulong args,
> >> +                                   uint32_t nret, target_ulong rets)
> >> +{
> >> +    if (!spapr->guest_machine_check_addr) {
> >> +        /* NMI register not called */
> >> +        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> >> +    } else {
> >> +        /*
> >> +         * VCPU issuing "ibm,nmi-interlock" is done with NMI handling,
> >> +         * hence unset mc_status.
> >> +         */
> >> +        spapr->mc_status = -1;
> >> +        qemu_cond_signal(&spapr->mc_delivery_cond);
> >> +        rtas_st(rets, 0, RTAS_OUT_SUCCESS);
> >> +    }
> >> +}
> >> +
> >> +
> >>  static struct rtas_call {
> >>      const char *name;
> >>      spapr_rtas_fn fn;
> >> @@ -489,6 +520,10 @@ static void core_rtas_register_types(void)
> >>                          rtas_set_power_level);
> >>      spapr_rtas_register(RTAS_GET_POWER_LEVEL, "get-power-level",
> >>                          rtas_get_power_level);
> >> +    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
> >> +                        rtas_ibm_nmi_register);
> >> +    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
> >> +                        rtas_ibm_nmi_interlock);
> >>  }
> >>  
> >>  type_init(core_rtas_register_types)
> >> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> >> index b395aa7..28b6e2e 100644
> >> --- a/include/hw/ppc/spapr.h
> >> +++ b/include/hw/ppc/spapr.h
> >> @@ -124,6 +124,13 @@ struct sPAPRMachineState {
> >>       * occurs during the unplug process. */
> >>      QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
> >>  
> >> +    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
> >> +    target_ulong guest_machine_check_addr;
> > 
> > You need to migrate this.  This is separate and simpler from the
> > problems of migrating during machine check handling - at the moment if
> > the guest calls nmi-register, then you migrate, then a machine check
> > is triggered you will have lost the information on how to handle it
> > properly.
> 
> I missed it. Will include.
> 
> Regards,
> Aravinda
> 
> > 
> > 
> >> +    /* mc_status is set to -1 if mc is not in progress, else is set to the CPU
> >> +     * handling the mc. */
> >> +    int mc_status;
> >> +    QemuCond mc_delivery_cond;
> >> +
> >>      /*< public >*/
> >>      char *kvm_type;
> >>      MemoryHotplugState hotplug_memory;
> >> @@ -520,8 +527,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
> >>  #define RTAS_IBM_CREATE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x27)
> >>  #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
> >>  #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
> >> +#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2A)
> >> +#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2B)
> >>  
> >> -#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2A)
> >> +#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2C)
> >>  
> >>  /* RTAS ibm,get-system-parameter token values */
> >>  #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
> >>
> > 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[Alexey Kardashevskiy]

[-- Attachment #1: Type: text/plain, Size: 3294 bytes --]

On 03/10/17 17:07, David Gibson wrote:
> On Mon, Oct 02, 2017 at 02:02:19PM +1100, Alexey Kardashevskiy wrote:
>> On 29/09/17 21:52, Nikunj A Dadhania wrote:
>>> David Gibson <david@gibson.dropbear.id.au> writes:
>>>
>>>> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
>>>>> Receive updates from SLOF about the updated rtas-base.
>>>>> A separate patch for SLOF [1] (commit f9a60de3) adds
>>>>> functionality to invoke a private HCALL whenever OS
>>>>> issues instantiate-rtas with a new rtas-base.
>>>>>
>>>>> This is required as QEMU needs to know the updated rtas-base
>>>>> as it allocates error reporting structure in RTAS space upon
>>>>> a machine check exception.
>>>>>
>>>>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
>>>>>
>>>>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>>>>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
>>>>
>>>> Ao I acked this earlier, but I've now realized there might be some
>>>> connection between this and discussions taking place elsewhere about
>>>> qemu not knowing what SLOF does with the device tree.
>>>>
>>>> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
>>>> the time of instantiate-rtas, is that right?
>>>
>>> The call happens from
>>> arch/powerpc/kernel/prom_init.c:prom_instantiate_rtas() and after that
>>> linux kernel makes two entries in the DT
>>>
>>> ....
>>>        if (call_prom_ret("call-method", 3, 2, &entry,
>>>                           ADDR("instantiate-rtas"),
>>>                           rtas_inst, base) != 0
>>>             || entry == 0) {
>>>                 prom_printf(" failed\n");
>>>                 return;
>>>         }
>>>         prom_printf(" done\n");
>>>
>>>         reserve_mem(base, size);
>>>
>>>         val = cpu_to_be32(base);
>>>         prom_setprop(rtas_node, "/rtas", "linux,rtas-base",
>>>                      &val, sizeof(val));
>>>         val = cpu_to_be32(entry);
>>>         prom_setprop(rtas_node, "/rtas", "linux,rtas-entry",
>>>                      &val, sizeof(val));
>>> ....
>>>
>>> Quiesce is called after this. 
>>>
>>>> Does SLOF put the RTAS blob address in its internal device tree, or
>>>> does it only pass it to the guest via the return parameters from
>>>> instantiate-rtas?
>>>
>>> Entry was made to the DT by linux kernel prom_init code, will this be
>>> visible to QEMU?
>>
>> With my recent SLOF FDT patch - yes:
>>
>> aik@fstn1-p1:~$ grep rtas dbg.dts
>> 	rtas {
>> 		linux,rtas-entry = <0x2fff0000>;
>> 		linux,rtas-base = <0x2fff0000>;
>> [...]
> 
> Ah.. except.. isn't that relying on the kernel putting the RTAS
> address into the device tree before it calls quiesce and kills SLOF?
> 
> The SLOF image is bundled in with qemu, so it's ok for us to rely on
> its behaviour up to a point.  It's not really ok for us to rely on the
> kernel's behaviour here, unless that behaviour is mandated by PAPR,
> which this isn't.

Fair point.

> So, I think we either need to have *SLOF* update the device tree with
> that address at instantiate-rtas time,

I can do that, in a separate patch.

> or we'll need to resurrect
> Aravinda's original UPDATE_RTAS hcall.





-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]

Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 11722 bytes --]

On Thu, Sep 28, 2017 at 04:08:10PM +0530, Aravinda Prasad wrote:
> Memory error such as bit flips that cannot be corrected
> by hardware are passed on to the kernel for handling.
> If the memory address in error belongs to guest then
> the guest kernel is responsible for taking suitable action.
> Patch [1] enhances KVM to exit guest with exit reason
> set to KVM_EXIT_NMI in such cases.
> 
> This patch handles KVM_EXIT_NMI exit. If the guest OS
> has registered the machine check handling routine by
> calling "ibm,nmi-register", then the handler builds
> the error log and invokes the registered handler else
> invokes the handler at 0x200.
> 
> Note that FWNMI handles synchronous machine check exceptions
> triggered by the hardware and hence we do not extend
> such support to the "nmi" command available in the QEMU
> monitor. Hence, "nmi" command from the monitor will
> always go through 0x200 vector.
> 
> [1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
> 	(e20bbd3d and related commits)

What does happen on KVM if an asynchronous machine check exception
occurs while in the guest?  Or under PowerVM for that matter.

> 
> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
> ---
>  hw/ppc/spapr.c         |    4 +++
>  hw/ppc/spapr_events.c  |   62 ++++++++++++++++++++++++++++++++++++++++++++++++
>  include/hw/ppc/spapr.h |    6 +++++
>  target/ppc/kvm.c       |   62 ++++++++++++++++++++++++++++++++++++++++++++++++
>  target/ppc/kvm_ppc.h   |   14 +++++++++++
>  5 files changed, 148 insertions(+)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index d568ea6..7780434 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2453,6 +2453,10 @@ static void ppc_spapr_init(MachineState *machine)
>          error_report("Could not get size of LPAR rtas '%s'", filename);
>          exit(1);
>      }
> +
> +    /* Resize blob to accommodate error log. */
> +    spapr->rtas_size = spapr_get_rtas_size();
> +
>      spapr->rtas_blob = g_malloc(spapr->rtas_size);
>      if (load_image_size(filename, spapr->rtas_blob, spapr->rtas_size) < 0) {
>          error_report("Could not load LPAR rtas '%s'", filename);
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index e377fc7..ac93a7b 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -41,6 +41,7 @@
>  #include "qemu/bcd.h"
>  #include "hw/ppc/spapr_ovec.h"
>  #include <libfdt.h>
> +#include <linux/kvm.h>
>  
>  #define RTAS_LOG_VERSION_MASK                   0xff000000
>  #define   RTAS_LOG_VERSION_6                    0x06000000
> @@ -174,6 +175,22 @@ struct epow_extended_log {
>      struct rtas_event_log_v6_epow epow;
>  } QEMU_PACKED;
>  
> +/*
> + * Data format in RTAS Blob
> + *
> + * This structure contains error information related to Machine
> + * Check exception. This is filled up and copied to rtas blob
> + * upon machine check exception. The address of rtas blob is
> + * passed on to OS registered machine check notification
> + * routines upon machine check exception.
> + */
> +struct rtas_event_log_mce {
> +    target_ulong r3;
> +    struct rtas_error_log rtas_error_log;
> +    unsigned char   buffer[1];      /* Start of extended log */

I believe we allow C99 extensions in qemu, so you can use buffer[], a
C99 flexible array member, rather than the length 1 hack.

> +} QEMU_PACKED;
> +
> +
>  union drc_identifier {
>      uint32_t index;
>      uint32_t count;
> @@ -623,6 +640,51 @@ void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,
>                              RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
>  }
>  
> +ssize_t spapr_get_rtas_size(void)
> +{
> +    return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);

Erm.. because of the definition of rtas_event_log_mce, this only
allows for 1 byte of extended log buffer.  That doesn't seem right.

> +}
> +
> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,
> +                                 uint16_t flags, bool err_type, bool le)

err_tpe isn't a very informative name for a boolean.  'uncorrectable'
would be better.  Although, didn't you say only uncorrectable errors
are directed to the guest, so does this have any purpose anyway?

> +{
> +    struct rtas_event_log_mce mc_log;
> +    uint32_t summary;
> +
> +    /* Set error log fields */
> +    mc_log.r3 = r3;
> +
> +    summary = RTAS_LOG_SEVERITY_ERROR_SYNC;
> +
> +    if (flags & KVM_RUN_PPC_NMI_DISP_FULLY_RECOV) {

KVM specific flags shouldn't be here, this translation should happen
in the caller.

> +        summary |= RTAS_LOG_DISPOSITION_FULLY_RECOVERED;
> +    } else {
> +        summary |= RTAS_LOG_DISPOSITION_NOT_RECOVERED;
> +    }
> +
> +    summary |= (RTAS_LOG_INITIATOR_MEMORY | RTAS_LOG_TARGET_MEMORY);
> +
> +    if (err_type) {
> +        summary |= RTAS_LOG_TYPE_ECC_UNCORR;
> +    } else {
> +        summary |= RTAS_LOG_TYPE_ECC_CORR;
> +    }
> +
> +    mc_log.rtas_error_log.summary = cpu_to_be32(summary);
> +
> +    /* Handle all Host/Guest LE/BE combinations */

I'd prefer you do this logic immediate as you store mc_log.r3.  I
really dislike storing values into a structure in the wrong
endianness, even temporarily - it makes it harder for someone reading
the code to discern what endianness the field is supposed to be in.

> +    if (le) {
> +        mc_log.r3 = cpu_to_le64(mc_log.r3);
> +    } else {
> +        mc_log.r3 = cpu_to_be64(mc_log.r3);
> +    }
> +
> +    cpu_physical_memory_write(rtas_addr + RTAS_ERRLOG_OFFSET,
> +                              &mc_log, sizeof(mc_log));
> +
> +    return rtas_addr + RTAS_ERRLOG_OFFSET;
> +}
> +
>  static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>                              uint32_t token, uint32_t nargs,
>                              target_ulong args,
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 28b6e2e..a75e9cf 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
>  #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2
>  #define DIAGNOSTICS_RUN_MODE_PERIODIC  3
>  
> +/* Offset from rtas-base where error log is placed */
> +#define RTAS_ERRLOG_OFFSET       0x200

Is there any particular rationale for this offset?  Our actual RTAS
code is 20 bytes, much smaller than this.

> +
>  static inline uint64_t ppc64_phys_to_real(uint64_t addr)
>  {
>      return addr & ~0xF000000000000000ULL;
> @@ -675,6 +678,9 @@ int spapr_hpt_shift_for_ramsize(uint64_t ramsize);
>  void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
>                            Error **errp);
>  void spapr_clear_pending_events(sPAPRMachineState *spapr);
> +ssize_t spapr_get_rtas_size(void);
> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,
> +                                 uint16_t flags, bool err_type, bool le);
>  
>  /* CPU and LMB DRC release callbacks. */
>  void spapr_core_release(DeviceState *dev);
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 171d3d8..7e4ce02 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -1798,6 +1798,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
>          ret = 0;
>          break;
>  
> +    case KVM_EXIT_NMI:
> +        DPRINTF("handle NMI exception\n");
> +        ret = kvm_handle_nmi(cpu, run);
> +        break;
> +
>      default:
>          fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
>          ret = -1;
> @@ -2746,6 +2751,63 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
>      return data & 0xffff;
>  }
>  
> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)

Most of the logic here - everything except actually parsing the
relevant fields from kvm_run - should move to spapr_events.  We may
not have any way of generating synchronous MCEs in TCG at the moment,
but we shouldn't exclude the possibility (being able to inject
uncorrectable memory errors in TCG sounds like it could be quite a
useful debugging tool).

> +{
> +    CPUPPCState *env = &cpu->env;
> +    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> +    PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
> +    target_ulong msr = 0;
> +    bool type, le;
> +
> +    cpu_synchronize_state(CPU(cpu));
> +
> +    /*
> +     * Properly set bits in MSR before we invoke the handler.
> +     * SRR0/1, DAR and DSISR are properly set by KVM
> +     */
> +    if (!(*pcc->interrupts_big_endian)(cpu)) {
> +        msr |= (1ULL << MSR_LE);
> +    }
> +
> +    if (env->msr && (1ULL << MSR_SF)) {
> +        msr |= (1ULL << MSR_SF);
> +    }
> +
> +    msr |= (1ULL << MSR_ME);
> +    env->msr = msr;
> +
> +    if (!spapr->guest_machine_check_addr) {
> +        /*
> +         * If OS has not registered with "ibm,nmi-register"
> +         * jump to 0x200
> +         */
> +        env->nip = 0x200;

Sure you don't need to set some diagnostic registers in this case?

> +        return 0;
> +    }
> +
> +    while (spapr->mc_status != -1) {
> +        /*
> +         * Check whether the same CPU got machine check error
> +         * while still handling the mc error (i.e., before
> +         * that CPU called "ibm,nmi-interlock"
> +         */
> +        if (spapr->mc_status == cpu->vcpu_id) {
> +            qemu_system_guest_panicked(NULL);
> +        }

I think you need a check to break out of here if the system has been
reset.  Otherwise if you get:

   1. MC generated on CPUs 0 & 1
   2. MC delivered CPU0, CPU1 blocked here
   3. system reset

I think the CPU1 thread will still be stuck here, waiting to get back
to the main loop that would check for the reset.

> +        qemu_cond_wait_iothread(&spapr->mc_delivery_cond);
> +    }
> +
> +    spapr->mc_status = cpu->vcpu_id;
> +
> +    type = !!(env->spr[SPR_DSISR] & P7_DSISR_MC_UE);
> +    le = !!(env->msr & (1ULL << MSR_LE));
> +    env->gpr[3] = spapr_mce_req_event(env->gpr[3], spapr->rtas_addr,
> +                                      run->flags, type, le);
> +    env->nip = spapr->guest_machine_check_addr;
> +
> +    return 0;
> +}
> +
>  int kvmppc_enable_hwrng(void)
>  {
>      if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {
> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
> index d6be38e..0139dae 100644
> --- a/target/ppc/kvm_ppc.h
> +++ b/target/ppc/kvm_ppc.h
> @@ -71,6 +71,20 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu);
>  
>  bool kvmppc_is_mem_backend_page_size_ok(const char *obj_path);
>  
> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);
> +
> +/*
> + * Currently KVM only passes on the uncorrected machine
> + * check memory error to guest. Other machine check errors
> + * such as SLB multi-hit and TLB multi-hit are recovered
> + * in KVM and are not passed on to guest.
> + *
> + * DSISR Bit for uncorrected machine check error. Based
> + * on arch/powerpc/include/asm/mce.h
> + */
> +#define PPC_BIT(bit)                (0x8000000000000000ULL >> bit)
> +#define P7_DSISR_MC_UE              (PPC_BIT(48))  /* P8 too */
> +
>  #else
>  
>  static inline uint32_t kvmppc_get_tbfreq(void)
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v5 5/6] ppc: spapr: Enable FWNMI capability

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4676 bytes --]

On Thu, Sep 28, 2017 at 04:08:21PM +0530, Aravinda Prasad wrote:
> Enable the KVM capability KVM_CAP_PPC_FWNMI so that
> the KVM causes guest exit with NMI as exit reason
> when it encounters a machine check exception on the
> address belonging to a guest. Without this capability
> enabled, KVM redirects machine check exceptions to
> guest's 0x200 vector.
> 
> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> ---
>  hw/ppc/spapr_rtas.c  |   15 +++++++++++++++
>  target/ppc/kvm.c     |   13 +++++++++++++
>  target/ppc/kvm_ppc.h |    6 ++++++
>  3 files changed, 34 insertions(+)
> 
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index 08e9a5e..d017a67 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -46,6 +46,7 @@
>  #include "qemu/cutils.h"
>  #include "trace.h"
>  #include "hw/ppc/fdt.h"
> +#include "kvm_ppc.h"
>  
>  static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>                                     uint32_t token, uint32_t nargs,
> @@ -354,6 +355,20 @@ static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
>                                    target_ulong args,
>                                    uint32_t nret, target_ulong rets)
>  {
> +    int ret;
> +
> +    ret = kvmppc_fwnmi_enable(cpu);

If you're enabling it here, doesn't that mean you need to disable it
on reset?

> +    if (ret == 1) {
> +        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
> +        return;
> +    }
> +
> +    if (ret < 0) {
> +        rtas_st(rets, 0, RTAS_OUT_HW_ERROR);
> +        return;
> +    }
> +
>      spapr->guest_machine_check_addr = rtas_ld(args, 1);
>      rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>  }
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 7e4ce02..59b3322 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -92,6 +92,7 @@ static int cap_mmu_radix;
>  static int cap_mmu_hash_v3;
>  static int cap_resize_hpt;
>  static int cap_ppc_pvr_compat;
> +static int cap_fwnmi;
>  
>  static uint32_t debug_inst_opcode;
>  
> @@ -150,6 +151,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>      cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
>      cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
>      cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
> +    cap_fwnmi = kvm_check_extension(s, KVM_CAP_PPC_FWNMI);
>      /*
>       * Note: setting it to false because there is not such capability
>       * in KVM at this moment.
> @@ -2142,6 +2144,17 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
>      }
>  }
>  
> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
> +{
> +    CPUState *cs = CPU(cpu);
> +
> +    if (!cap_fwnmi) {
> +        return 1;
> +    }
> +
> +    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);

Yeah, this is no good.  It means migration from a host that's fwnmi
capable to one that isn't will be subtly broken.  Instead you need to
make fwnmi capability a machine property.  If the property is
requested and the host kernel doesn't support it, you need to outright
fail, rather than try to fall back.

> +}
> +
>  int kvmppc_smt_threads(void)
>  {
>      return cap_ppc_smt ? cap_ppc_smt : 1;
> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
> index 0139dae..55b6df2 100644
> --- a/target/ppc/kvm_ppc.h
> +++ b/target/ppc/kvm_ppc.h
> @@ -28,6 +28,7 @@ void kvmppc_enable_clear_ref_mod_hcalls(void);
>  void kvmppc_set_papr(PowerPCCPU *cpu);
>  int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
>  void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu);
>  int kvmppc_smt_threads(void);
>  void kvmppc_hint_smt_possible(Error **errp);
>  int kvmppc_set_smt_threads(int smt);
> @@ -157,6 +158,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
>  {
>  }
>  
> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
> +{
> +    return 1;

Likewise, this should be available, not banned, on TCG.  I think there
are existing problems with TCG<->KVM migration, but there's no
inherent reason they shouldn't work, so we don't want to introduce
extra reasons they don't.

Even if TCG will never generate fwnmis (for now), it should allow the
guest to register for them.

> +}
> +
>  static inline int kvmppc_smt_threads(void)
>  {
>      return 1;
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v5 6/6] migration: Block migration while handling machine check

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4157 bytes --]

On Thu, Sep 28, 2017 at 04:08:31PM +0530, Aravinda Prasad wrote:
> Block VM migration requests until the machine check
> error handling is complete as (i) these errors are
> specific to the source hardware and is irrelevant on
> the target hardware, (ii) these errors cause data
> corruption and should be handled before migration.
> 
> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> ---
>  hw/ppc/spapr_rtas.c    |    3 +++
>  include/hw/ppc/spapr.h |    2 ++
>  target/ppc/kvm.c       |   17 +++++++++++++++++
>  3 files changed, 22 insertions(+)
> 
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index d017a67..17f6567 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -47,6 +47,7 @@
>  #include "trace.h"
>  #include "hw/ppc/fdt.h"
>  #include "kvm_ppc.h"
> +#include "migration/blocker.h"
>  
>  static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>                                     uint32_t token, uint32_t nargs,
> @@ -390,6 +391,8 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
>          spapr->mc_status = -1;
>          qemu_cond_signal(&spapr->mc_delivery_cond);
>          rtas_st(rets, 0, RTAS_OUT_SUCCESS);
> +        migrate_del_blocker(spapr->migration_blocker);
> +        error_free(spapr->migration_blocker);
>      }
>  }
>  
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index a75e9cf..0890a44 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -7,6 +7,7 @@
>  #include "hw/ppc/spapr_drc.h"
>  #include "hw/mem/pc-dimm.h"
>  #include "hw/ppc/spapr_ovec.h"
> +#include "qapi/error.h"
>  
>  struct VIOsPAPRBus;
>  struct sPAPRPHBState;
> @@ -136,6 +137,7 @@ struct sPAPRMachineState {
>      MemoryHotplugState hotplug_memory;
>  
>      const char *icp_type;
> +    Error *migration_blocker;

This isn't a good name, because it's _specifically_ the fwnmi as a
migration blocker - trying to put another migration blocker in here
would break horribly, because nmi-interlock would clear it regardless.

>  };
>  
>  #define H_SUCCESS         0
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 59b3322..58de7ea 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -52,6 +52,7 @@
>  #endif
>  #include "elf.h"
>  #include "sysemu/kvm_int.h"
> +#include "migration/blocker.h"
>  
>  //#define DEBUG_KVM
>  
> @@ -2770,10 +2771,26 @@ int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
>      sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
>      PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
>      target_ulong msr = 0;
> +    Error *local_err = NULL;
> +    int ret;
>      bool type, le;
>  
>      cpu_synchronize_state(CPU(cpu));
>  
> +    error_setg(&spapr->migration_blocker,
> +            "Live migration not supported during machine check error handling");

In fact, there's no real reason to generate the error here.  The
error's always the same so you could just create it at startup as a
global and just add/remove it to the block list.

> +    ret = migrate_add_blocker(spapr->migration_blocker, &local_err);
> +    if (ret < 0) {
> +        /*
> +         * We don't want to abort and let the migration to continue. In a
> +         * rare case, the machine check handler will run on the target
> +         * hardware. Though this is not preferable, it is better than aborting

Why is it not preferable?  I mean it's an edge case, but AFAICT it's
still the correct behaviour.

> +         * the migration or killing the VM.
> +         */
> +        error_free(spapr->migration_blocker);
> +        fprintf(stderr, "Warning: Machine check during VM migration\n");

Use error_report(), not fprintf().

> +    }
> +
>      /*
>       * Properly set bits in MSR before we invoke the handler.
>       * SRR0/1, DAR and DSISR are properly set by KVM
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[Alexey Kardashevskiy]

[-- Attachment #1: Type: text/plain, Size: 3841 bytes --]

On 03/10/17 20:12, Alexey Kardashevskiy wrote:
> On 03/10/17 17:07, David Gibson wrote:
>> On Mon, Oct 02, 2017 at 02:02:19PM +1100, Alexey Kardashevskiy wrote:
>>> On 29/09/17 21:52, Nikunj A Dadhania wrote:
>>>> David Gibson <david@gibson.dropbear.id.au> writes:
>>>>
>>>>> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
>>>>>> Receive updates from SLOF about the updated rtas-base.
>>>>>> A separate patch for SLOF [1] (commit f9a60de3) adds
>>>>>> functionality to invoke a private HCALL whenever OS
>>>>>> issues instantiate-rtas with a new rtas-base.
>>>>>>
>>>>>> This is required as QEMU needs to know the updated rtas-base
>>>>>> as it allocates error reporting structure in RTAS space upon
>>>>>> a machine check exception.
>>>>>>
>>>>>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
>>>>>>
>>>>>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>>>>>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
>>>>>
>>>>> Ao I acked this earlier, but I've now realized there might be some
>>>>> connection between this and discussions taking place elsewhere about
>>>>> qemu not knowing what SLOF does with the device tree.
>>>>>
>>>>> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
>>>>> the time of instantiate-rtas, is that right?
>>>>
>>>> The call happens from
>>>> arch/powerpc/kernel/prom_init.c:prom_instantiate_rtas() and after that
>>>> linux kernel makes two entries in the DT
>>>>
>>>> ....
>>>>        if (call_prom_ret("call-method", 3, 2, &entry,
>>>>                           ADDR("instantiate-rtas"),
>>>>                           rtas_inst, base) != 0
>>>>             || entry == 0) {
>>>>                 prom_printf(" failed\n");
>>>>                 return;
>>>>         }
>>>>         prom_printf(" done\n");
>>>>
>>>>         reserve_mem(base, size);
>>>>
>>>>         val = cpu_to_be32(base);
>>>>         prom_setprop(rtas_node, "/rtas", "linux,rtas-base",
>>>>                      &val, sizeof(val));
>>>>         val = cpu_to_be32(entry);
>>>>         prom_setprop(rtas_node, "/rtas", "linux,rtas-entry",
>>>>                      &val, sizeof(val));
>>>> ....
>>>>
>>>> Quiesce is called after this. 
>>>>
>>>>> Does SLOF put the RTAS blob address in its internal device tree, or
>>>>> does it only pass it to the guest via the return parameters from
>>>>> instantiate-rtas?
>>>>
>>>> Entry was made to the DT by linux kernel prom_init code, will this be
>>>> visible to QEMU?
>>>
>>> With my recent SLOF FDT patch - yes:
>>>
>>> aik@fstn1-p1:~$ grep rtas dbg.dts
>>> 	rtas {
>>> 		linux,rtas-entry = <0x2fff0000>;
>>> 		linux,rtas-base = <0x2fff0000>;
>>> [...]
>>
>> Ah.. except.. isn't that relying on the kernel putting the RTAS
>> address into the device tree before it calls quiesce and kills SLOF?
>>
>> The SLOF image is bundled in with qemu, so it's ok for us to rely on
>> its behaviour up to a point.  It's not really ok for us to rely on the
>> kernel's behaviour here, unless that behaviour is mandated by PAPR,
>> which this isn't.
> 
> Fair point.
> 
>> So, I think we either need to have *SLOF* update the device tree with
>> that address at instantiate-rtas time,
> 
> I can do that, in a separate patch.


One comment though - if I create the properties in SLOF, I have to name
them different, like rtas-entry/rtas-base or slof,rtas-entry/slof,rtas-base
to avoid colliding with the ones create by the guest kernel.

So what do I name them? And do we need 2 copies of the same thing, do we
ever expect rtas-entry!=rtas-base? The guest can potentially get them
different (under powervm) but not with SLOF.


> 
>> or we'll need to resurrect
>> Aravinda's original UPDATE_RTAS hcall.



-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH v5 1/6] ppc: spapr: Register and handle HCALL to receive updated RTAS region

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4648 bytes --]

On Wed, Oct 04, 2017 at 02:32:11PM +1100, Alexey Kardashevskiy wrote:
> On 03/10/17 20:12, Alexey Kardashevskiy wrote:
> > On 03/10/17 17:07, David Gibson wrote:
> >> On Mon, Oct 02, 2017 at 02:02:19PM +1100, Alexey Kardashevskiy wrote:
> >>> On 29/09/17 21:52, Nikunj A Dadhania wrote:
> >>>> David Gibson <david@gibson.dropbear.id.au> writes:
> >>>>
> >>>>> On Thu, Sep 28, 2017 at 04:07:38PM +0530, Aravinda Prasad wrote:
> >>>>>> Receive updates from SLOF about the updated rtas-base.
> >>>>>> A separate patch for SLOF [1] (commit f9a60de3) adds
> >>>>>> functionality to invoke a private HCALL whenever OS
> >>>>>> issues instantiate-rtas with a new rtas-base.
> >>>>>>
> >>>>>> This is required as QEMU needs to know the updated rtas-base
> >>>>>> as it allocates error reporting structure in RTAS space upon
> >>>>>> a machine check exception.
> >>>>>>
> >>>>>> [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-August/120386.html
> >>>>>>
> >>>>>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> >>>>>> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> >>>>>
> >>>>> Ao I acked this earlier, but I've now realized there might be some
> >>>>> connection between this and discussions taking place elsewhere about
> >>>>> qemu not knowing what SLOF does with the device tree.
> >>>>>
> >>>>> At what point will SLOF call the UPDATE_RTAS hcall?  I'm guessing at
> >>>>> the time of instantiate-rtas, is that right?
> >>>>
> >>>> The call happens from
> >>>> arch/powerpc/kernel/prom_init.c:prom_instantiate_rtas() and after that
> >>>> linux kernel makes two entries in the DT
> >>>>
> >>>> ....
> >>>>        if (call_prom_ret("call-method", 3, 2, &entry,
> >>>>                           ADDR("instantiate-rtas"),
> >>>>                           rtas_inst, base) != 0
> >>>>             || entry == 0) {
> >>>>                 prom_printf(" failed\n");
> >>>>                 return;
> >>>>         }
> >>>>         prom_printf(" done\n");
> >>>>
> >>>>         reserve_mem(base, size);
> >>>>
> >>>>         val = cpu_to_be32(base);
> >>>>         prom_setprop(rtas_node, "/rtas", "linux,rtas-base",
> >>>>                      &val, sizeof(val));
> >>>>         val = cpu_to_be32(entry);
> >>>>         prom_setprop(rtas_node, "/rtas", "linux,rtas-entry",
> >>>>                      &val, sizeof(val));
> >>>> ....
> >>>>
> >>>> Quiesce is called after this. 
> >>>>
> >>>>> Does SLOF put the RTAS blob address in its internal device tree, or
> >>>>> does it only pass it to the guest via the return parameters from
> >>>>> instantiate-rtas?
> >>>>
> >>>> Entry was made to the DT by linux kernel prom_init code, will this be
> >>>> visible to QEMU?
> >>>
> >>> With my recent SLOF FDT patch - yes:
> >>>
> >>> aik@fstn1-p1:~$ grep rtas dbg.dts
> >>> 	rtas {
> >>> 		linux,rtas-entry = <0x2fff0000>;
> >>> 		linux,rtas-base = <0x2fff0000>;
> >>> [...]
> >>
> >> Ah.. except.. isn't that relying on the kernel putting the RTAS
> >> address into the device tree before it calls quiesce and kills SLOF?
> >>
> >> The SLOF image is bundled in with qemu, so it's ok for us to rely on
> >> its behaviour up to a point.  It's not really ok for us to rely on the
> >> kernel's behaviour here, unless that behaviour is mandated by PAPR,
> >> which this isn't.
> > 
> > Fair point.
> > 
> >> So, I think we either need to have *SLOF* update the device tree with
> >> that address at instantiate-rtas time,
> > 
> > I can do that, in a separate patch.
> 
> 
> One comment though - if I create the properties in SLOF, I have to name
> them different, like rtas-entry/rtas-base or slof,rtas-entry/slof,rtas-base
> to avoid colliding with the ones create by the guest kernel.

That's fine.  I don't know if the kernel will error if the properties
are already there or just overwrite them, but using new names might be
safer.

> So what do I name them? And do we need 2 copies of the same thing,

Need, no, but if having 2 copies is the simpler approach that's fine.

> do we
> ever expect rtas-entry!=rtas-base? The guest can potentially get them
> different (under powervm) but not with SLOF.

More to the point, qemu doesn't actually need to know the entry point
for the fwnmi stuff, only the base address.

> 
> 
> > 
> >> or we'll need to resurrect
> >> Aravinda's original UPDATE_RTAS hcall.
> 
> 
> 




-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v5 6/6] migration: Block migration while handling machine check

[Aravinda Prasad]

On Wednesday 04 October 2017 07:09 AM, David Gibson wrote:
> On Thu, Sep 28, 2017 at 04:08:31PM +0530, Aravinda Prasad wrote:
>> Block VM migration requests until the machine check
>> error handling is complete as (i) these errors are
>> specific to the source hardware and is irrelevant on
>> the target hardware, (ii) these errors cause data
>> corruption and should be handled before migration.
>>
>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>> ---
>>  hw/ppc/spapr_rtas.c    |    3 +++
>>  include/hw/ppc/spapr.h |    2 ++
>>  target/ppc/kvm.c       |   17 +++++++++++++++++
>>  3 files changed, 22 insertions(+)
>>
>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>> index d017a67..17f6567 100644
>> --- a/hw/ppc/spapr_rtas.c
>> +++ b/hw/ppc/spapr_rtas.c
>> @@ -47,6 +47,7 @@
>>  #include "trace.h"
>>  #include "hw/ppc/fdt.h"
>>  #include "kvm_ppc.h"
>> +#include "migration/blocker.h"
>>  
>>  static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>>                                     uint32_t token, uint32_t nargs,
>> @@ -390,6 +391,8 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
>>          spapr->mc_status = -1;
>>          qemu_cond_signal(&spapr->mc_delivery_cond);
>>          rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>> +        migrate_del_blocker(spapr->migration_blocker);
>> +        error_free(spapr->migration_blocker);
>>      }
>>  }
>>  
>> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
>> index a75e9cf..0890a44 100644
>> --- a/include/hw/ppc/spapr.h
>> +++ b/include/hw/ppc/spapr.h
>> @@ -7,6 +7,7 @@
>>  #include "hw/ppc/spapr_drc.h"
>>  #include "hw/mem/pc-dimm.h"
>>  #include "hw/ppc/spapr_ovec.h"
>> +#include "qapi/error.h"
>>  
>>  struct VIOsPAPRBus;
>>  struct sPAPRPHBState;
>> @@ -136,6 +137,7 @@ struct sPAPRMachineState {
>>      MemoryHotplugState hotplug_memory;
>>  
>>      const char *icp_type;
>> +    Error *migration_blocker;
> 
> This isn't a good name, because it's _specifically_ the fwnmi as a
> migration blocker - trying to put another migration blocker in here
> would break horribly, because nmi-interlock would clear it regardless.

Will rename it.

> 
>>  };
>>  
>>  #define H_SUCCESS         0
>> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
>> index 59b3322..58de7ea 100644
>> --- a/target/ppc/kvm.c
>> +++ b/target/ppc/kvm.c
>> @@ -52,6 +52,7 @@
>>  #endif
>>  #include "elf.h"
>>  #include "sysemu/kvm_int.h"
>> +#include "migration/blocker.h"
>>  
>>  //#define DEBUG_KVM
>>  
>> @@ -2770,10 +2771,26 @@ int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
>>      sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
>>      PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
>>      target_ulong msr = 0;
>> +    Error *local_err = NULL;
>> +    int ret;
>>      bool type, le;
>>  
>>      cpu_synchronize_state(CPU(cpu));
>>  
>> +    error_setg(&spapr->migration_blocker,
>> +            "Live migration not supported during machine check error handling");
> 
> In fact, there's no real reason to generate the error here.  The
> error's always the same so you could just create it at startup as a
> global and just add/remove it to the block list.

ok.

> 
>> +    ret = migrate_add_blocker(spapr->migration_blocker, &local_err);
>> +    if (ret < 0) {
>> +        /*
>> +         * We don't want to abort and let the migration to continue. In a
>> +         * rare case, the machine check handler will run on the target
>> +         * hardware. Though this is not preferable, it is better than aborting
> 
> Why is it not preferable?  I mean it's an edge case, but AFAICT it's
> still the correct behaviour.

Will remove that line in the comment.

> 
>> +         * the migration or killing the VM.
>> +         */
>> +        error_free(spapr->migration_blocker);
>> +        fprintf(stderr, "Warning: Machine check during VM migration\n");
> 
> Use error_report(), not fprintf().

ok.

Regards,
Aravinda

> 
>> +    }
>> +
>>      /*
>>       * Properly set bits in MSR before we invoke the handler.
>>       * SRR0/1, DAR and DSISR are properly set by KVM
>>
> 

-- 
Regards,
Aravinda

Re: [Qemu-devel] [PATCH v5 5/6] ppc: spapr: Enable FWNMI capability

[Aravinda Prasad]

On Wednesday 04 October 2017 07:04 AM, David Gibson wrote:
> On Thu, Sep 28, 2017 at 04:08:21PM +0530, Aravinda Prasad wrote:
>> Enable the KVM capability KVM_CAP_PPC_FWNMI so that
>> the KVM causes guest exit with NMI as exit reason
>> when it encounters a machine check exception on the
>> address belonging to a guest. Without this capability
>> enabled, KVM redirects machine check exceptions to
>> guest's 0x200 vector.
>>
>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>> ---
>>  hw/ppc/spapr_rtas.c  |   15 +++++++++++++++
>>  target/ppc/kvm.c     |   13 +++++++++++++
>>  target/ppc/kvm_ppc.h |    6 ++++++
>>  3 files changed, 34 insertions(+)
>>
>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>> index 08e9a5e..d017a67 100644
>> --- a/hw/ppc/spapr_rtas.c
>> +++ b/hw/ppc/spapr_rtas.c
>> @@ -46,6 +46,7 @@
>>  #include "qemu/cutils.h"
>>  #include "trace.h"
>>  #include "hw/ppc/fdt.h"
>> +#include "kvm_ppc.h"
>>  
>>  static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>>                                     uint32_t token, uint32_t nargs,
>> @@ -354,6 +355,20 @@ static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
>>                                    target_ulong args,
>>                                    uint32_t nret, target_ulong rets)
>>  {
>> +    int ret;
>> +
>> +    ret = kvmppc_fwnmi_enable(cpu);
> 
> If you're enabling it here, doesn't that mean you need to disable it
> on reset?

We don't have a way in KVM to disable it once enabled.

> 
>> +    if (ret == 1) {
>> +        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
>> +        return;
>> +    }
>> +
>> +    if (ret < 0) {
>> +        rtas_st(rets, 0, RTAS_OUT_HW_ERROR);
>> +        return;
>> +    }
>> +
>>      spapr->guest_machine_check_addr = rtas_ld(args, 1);
>>      rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>>  }
>> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
>> index 7e4ce02..59b3322 100644
>> --- a/target/ppc/kvm.c
>> +++ b/target/ppc/kvm.c
>> @@ -92,6 +92,7 @@ static int cap_mmu_radix;
>>  static int cap_mmu_hash_v3;
>>  static int cap_resize_hpt;
>>  static int cap_ppc_pvr_compat;
>> +static int cap_fwnmi;
>>  
>>  static uint32_t debug_inst_opcode;
>>  
>> @@ -150,6 +151,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>>      cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
>>      cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
>>      cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
>> +    cap_fwnmi = kvm_check_extension(s, KVM_CAP_PPC_FWNMI);
>>      /*
>>       * Note: setting it to false because there is not such capability
>>       * in KVM at this moment.
>> @@ -2142,6 +2144,17 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
>>      }
>>  }
>>  
>> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
>> +{
>> +    CPUState *cs = CPU(cpu);
>> +
>> +    if (!cap_fwnmi) {
>> +        return 1;
>> +    }
>> +
>> +    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);
> 
> Yeah, this is no good.  It means migration from a host that's fwnmi
> capable to one that isn't will be subtly broken.  Instead you need to
> make fwnmi capability a machine property.  If the property is
> requested and the host kernel doesn't support it, you need to outright
> fail, rather than try to fall back.

Sure.

> 
>> +}
>> +
>>  int kvmppc_smt_threads(void)
>>  {
>>      return cap_ppc_smt ? cap_ppc_smt : 1;
>> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
>> index 0139dae..55b6df2 100644
>> --- a/target/ppc/kvm_ppc.h
>> +++ b/target/ppc/kvm_ppc.h
>> @@ -28,6 +28,7 @@ void kvmppc_enable_clear_ref_mod_hcalls(void);
>>  void kvmppc_set_papr(PowerPCCPU *cpu);
>>  int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
>>  void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
>> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu);
>>  int kvmppc_smt_threads(void);
>>  void kvmppc_hint_smt_possible(Error **errp);
>>  int kvmppc_set_smt_threads(int smt);
>> @@ -157,6 +158,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
>>  {
>>  }
>>  
>> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
>> +{
>> +    return 1;
> 
> Likewise, this should be available, not banned, on TCG.  I think there
> are existing problems with TCG<->KVM migration, but there's no
> inherent reason they shouldn't work, so we don't want to introduce
> extra reasons they don't.
> 
> Even if TCG will never generate fwnmis (for now), it should allow the
> guest to register for them.

Should this be then changed later only when TCG generates fwnmis?
Because we don't want to set spapr->guest_machine_check_addr for TCG in
rtas_ibm_nmi_register(). If set then all machine checks are redirected
to this address which is not desirable for TCG as we still want machine
checks to be passed via 0x200.

Regards,
Aravinda

> 
>> +}
>> +
>>  static inline int kvmppc_smt_threads(void)
>>  {
>>      return 1;
>>
> 

-- 
Regards,
Aravinda

Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit

[Aravinda Prasad]

On Wednesday 04 October 2017 06:59 AM, David Gibson wrote:
> On Thu, Sep 28, 2017 at 04:08:10PM +0530, Aravinda Prasad wrote:
>> Memory error such as bit flips that cannot be corrected
>> by hardware are passed on to the kernel for handling.
>> If the memory address in error belongs to guest then
>> the guest kernel is responsible for taking suitable action.
>> Patch [1] enhances KVM to exit guest with exit reason
>> set to KVM_EXIT_NMI in such cases.
>>
>> This patch handles KVM_EXIT_NMI exit. If the guest OS
>> has registered the machine check handling routine by
>> calling "ibm,nmi-register", then the handler builds
>> the error log and invokes the registered handler else
>> invokes the handler at 0x200.
>>
>> Note that FWNMI handles synchronous machine check exceptions
>> triggered by the hardware and hence we do not extend
>> such support to the "nmi" command available in the QEMU
>> monitor. Hence, "nmi" command from the monitor will
>> always go through 0x200 vector.
>>
>> [1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
>> 	(e20bbd3d and related commits)
> 
> What does happen on KVM if an asynchronous machine check exception
> occurs while in the guest?  Or under PowerVM for that matter.

AFAIK asynchronous errors take a different path in KVM as it can happen
in a different process context.

> 
>>
>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>> Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
>> ---
>>  hw/ppc/spapr.c         |    4 +++
>>  hw/ppc/spapr_events.c  |   62 ++++++++++++++++++++++++++++++++++++++++++++++++
>>  include/hw/ppc/spapr.h |    6 +++++
>>  target/ppc/kvm.c       |   62 ++++++++++++++++++++++++++++++++++++++++++++++++
>>  target/ppc/kvm_ppc.h   |   14 +++++++++++
>>  5 files changed, 148 insertions(+)
>>
>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
>> index d568ea6..7780434 100644
>> --- a/hw/ppc/spapr.c
>> +++ b/hw/ppc/spapr.c
>> @@ -2453,6 +2453,10 @@ static void ppc_spapr_init(MachineState *machine)
>>          error_report("Could not get size of LPAR rtas '%s'", filename);
>>          exit(1);
>>      }
>> +
>> +    /* Resize blob to accommodate error log. */
>> +    spapr->rtas_size = spapr_get_rtas_size();
>> +
>>      spapr->rtas_blob = g_malloc(spapr->rtas_size);
>>      if (load_image_size(filename, spapr->rtas_blob, spapr->rtas_size) < 0) {
>>          error_report("Could not load LPAR rtas '%s'", filename);
>> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
>> index e377fc7..ac93a7b 100644
>> --- a/hw/ppc/spapr_events.c
>> +++ b/hw/ppc/spapr_events.c
>> @@ -41,6 +41,7 @@
>>  #include "qemu/bcd.h"
>>  #include "hw/ppc/spapr_ovec.h"
>>  #include <libfdt.h>
>> +#include <linux/kvm.h>
>>  
>>  #define RTAS_LOG_VERSION_MASK                   0xff000000
>>  #define   RTAS_LOG_VERSION_6                    0x06000000
>> @@ -174,6 +175,22 @@ struct epow_extended_log {
>>      struct rtas_event_log_v6_epow epow;
>>  } QEMU_PACKED;
>>  
>> +/*
>> + * Data format in RTAS Blob
>> + *
>> + * This structure contains error information related to Machine
>> + * Check exception. This is filled up and copied to rtas blob
>> + * upon machine check exception. The address of rtas blob is
>> + * passed on to OS registered machine check notification
>> + * routines upon machine check exception.
>> + */
>> +struct rtas_event_log_mce {
>> +    target_ulong r3;
>> +    struct rtas_error_log rtas_error_log;
>> +    unsigned char   buffer[1];      /* Start of extended log */
> 
> I believe we allow C99 extensions in qemu, so you can use buffer[], a
> C99 flexible array member, rather than the length 1 hack.

ok.

> 
>> +} QEMU_PACKED;
>> +
>> +
>>  union drc_identifier {
>>      uint32_t index;
>>      uint32_t count;
>> @@ -623,6 +640,51 @@ void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,
>>                              RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
>>  }
>>  
>> +ssize_t spapr_get_rtas_size(void)
>> +{
>> +    return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);
> 
> Erm.. because of the definition of rtas_event_log_mce, this only
> allows for 1 byte of extended log buffer.  That doesn't seem right.

This is directly taken from the kernel's RTAS log (struct rtas_error_log
in arch/powerpc/include/asm/rtas.h). I am not sure why they use 1 byte
extended log buffer.

> 
>> +}
>> +
>> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,
>> +                                 uint16_t flags, bool err_type, bool le)
> 
> err_tpe isn't a very informative name for a boolean.  'uncorrectable'
> would be better.  Although, didn't you say only uncorrectable errors
> are directed to the guest, so does this have any purpose anyway?

Right now only uncorrectable errors, but can be enhanced later to pass
on correctable errors to inform the guest.

> 
>> +{
>> +    struct rtas_event_log_mce mc_log;
>> +    uint32_t summary;
>> +
>> +    /* Set error log fields */
>> +    mc_log.r3 = r3;
>> +
>> +    summary = RTAS_LOG_SEVERITY_ERROR_SYNC;
>> +
>> +    if (flags & KVM_RUN_PPC_NMI_DISP_FULLY_RECOV) {
> 
> KVM specific flags shouldn't be here, this translation should happen
> in the caller.

ok.

> 
>> +        summary |= RTAS_LOG_DISPOSITION_FULLY_RECOVERED;
>> +    } else {
>> +        summary |= RTAS_LOG_DISPOSITION_NOT_RECOVERED;
>> +    }
>> +
>> +    summary |= (RTAS_LOG_INITIATOR_MEMORY | RTAS_LOG_TARGET_MEMORY);
>> +
>> +    if (err_type) {
>> +        summary |= RTAS_LOG_TYPE_ECC_UNCORR;
>> +    } else {
>> +        summary |= RTAS_LOG_TYPE_ECC_CORR;
>> +    }
>> +
>> +    mc_log.rtas_error_log.summary = cpu_to_be32(summary);
>> +
>> +    /* Handle all Host/Guest LE/BE combinations */
> 
> I'd prefer you do this logic immediate as you store mc_log.r3.  I
> really dislike storing values into a structure in the wrong
> endianness, even temporarily - it makes it harder for someone reading
> the code to discern what endianness the field is supposed to be in.

ok.

> 
>> +    if (le) {
>> +        mc_log.r3 = cpu_to_le64(mc_log.r3);
>> +    } else {
>> +        mc_log.r3 = cpu_to_be64(mc_log.r3);
>> +    }
>> +
>> +    cpu_physical_memory_write(rtas_addr + RTAS_ERRLOG_OFFSET,
>> +                              &mc_log, sizeof(mc_log));
>> +
>> +    return rtas_addr + RTAS_ERRLOG_OFFSET;
>> +}
>> +
>>  static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>>                              uint32_t token, uint32_t nargs,
>>                              target_ulong args,
>> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
>> index 28b6e2e..a75e9cf 100644
>> --- a/include/hw/ppc/spapr.h
>> +++ b/include/hw/ppc/spapr.h
>> @@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
>>  #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2
>>  #define DIAGNOSTICS_RUN_MODE_PERIODIC  3
>>  
>> +/* Offset from rtas-base where error log is placed */
>> +#define RTAS_ERRLOG_OFFSET       0x200
> 
> Is there any particular rationale for this offset?  Our actual RTAS
> code is 20 bytes, much smaller than this.

Just to ensure some space if in case RTAS code needs to be extended in
future.

> 
>> +
>>  static inline uint64_t ppc64_phys_to_real(uint64_t addr)
>>  {
>>      return addr & ~0xF000000000000000ULL;
>> @@ -675,6 +678,9 @@ int spapr_hpt_shift_for_ramsize(uint64_t ramsize);
>>  void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
>>                            Error **errp);
>>  void spapr_clear_pending_events(sPAPRMachineState *spapr);
>> +ssize_t spapr_get_rtas_size(void);
>> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,
>> +                                 uint16_t flags, bool err_type, bool le);
>>  
>>  /* CPU and LMB DRC release callbacks. */
>>  void spapr_core_release(DeviceState *dev);
>> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
>> index 171d3d8..7e4ce02 100644
>> --- a/target/ppc/kvm.c
>> +++ b/target/ppc/kvm.c
>> @@ -1798,6 +1798,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
>>          ret = 0;
>>          break;
>>  
>> +    case KVM_EXIT_NMI:
>> +        DPRINTF("handle NMI exception\n");
>> +        ret = kvm_handle_nmi(cpu, run);
>> +        break;
>> +
>>      default:
>>          fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
>>          ret = -1;
>> @@ -2746,6 +2751,63 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
>>      return data & 0xffff;
>>  }
>>  
>> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
> 
> Most of the logic here - everything except actually parsing the
> relevant fields from kvm_run - should move to spapr_events.  We may
> not have any way of generating synchronous MCEs in TCG at the moment,
> but we shouldn't exclude the possibility (being able to inject
> uncorrectable memory errors in TCG sounds like it could be quite a
> useful debugging tool).

ok.

> 
>> +{
>> +    CPUPPCState *env = &cpu->env;
>> +    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
>> +    PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
>> +    target_ulong msr = 0;
>> +    bool type, le;
>> +
>> +    cpu_synchronize_state(CPU(cpu));
>> +
>> +    /*
>> +     * Properly set bits in MSR before we invoke the handler.
>> +     * SRR0/1, DAR and DSISR are properly set by KVM
>> +     */
>> +    if (!(*pcc->interrupts_big_endian)(cpu)) {
>> +        msr |= (1ULL << MSR_LE);
>> +    }
>> +
>> +    if (env->msr && (1ULL << MSR_SF)) {
>> +        msr |= (1ULL << MSR_SF);
>> +    }
>> +
>> +    msr |= (1ULL << MSR_ME);
>> +    env->msr = msr;
>> +
>> +    if (!spapr->guest_machine_check_addr) {
>> +        /*
>> +         * If OS has not registered with "ibm,nmi-register"
>> +         * jump to 0x200
>> +         */
>> +        env->nip = 0x200;
> 
> Sure you don't need to set some diagnostic registers in this case?

Not sure. Let me check.

> 
>> +        return 0;
>> +    }
>> +
>> +    while (spapr->mc_status != -1) {
>> +        /*
>> +         * Check whether the same CPU got machine check error
>> +         * while still handling the mc error (i.e., before
>> +         * that CPU called "ibm,nmi-interlock"
>> +         */
>> +        if (spapr->mc_status == cpu->vcpu_id) {
>> +            qemu_system_guest_panicked(NULL);
>> +        }
> 
> I think you need a check to break out of here if the system has been
> reset.  Otherwise if you get:
> 
>    1. MC generated on CPUs 0 & 1
>    2. MC delivered CPU0, CPU1 blocked here
>    3. system reset
> 
> I think the CPU1 thread will still be stuck here, waiting to get back
> to the main loop that would check for the reset.

sure.

Regards,
Aravinda
> 
>> +        qemu_cond_wait_iothread(&spapr->mc_delivery_cond);
>> +    }
>> +
>> +    spapr->mc_status = cpu->vcpu_id;
>> +
>> +    type = !!(env->spr[SPR_DSISR] & P7_DSISR_MC_UE);
>> +    le = !!(env->msr & (1ULL << MSR_LE));
>> +    env->gpr[3] = spapr_mce_req_event(env->gpr[3], spapr->rtas_addr,
>> +                                      run->flags, type, le);
>> +    env->nip = spapr->guest_machine_check_addr;
>> +
>> +    return 0;
>> +}
>> +
>>  int kvmppc_enable_hwrng(void)
>>  {
>>      if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {
>> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
>> index d6be38e..0139dae 100644
>> --- a/target/ppc/kvm_ppc.h
>> +++ b/target/ppc/kvm_ppc.h
>> @@ -71,6 +71,20 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu);
>>  
>>  bool kvmppc_is_mem_backend_page_size_ok(const char *obj_path);
>>  
>> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);
>> +
>> +/*
>> + * Currently KVM only passes on the uncorrected machine
>> + * check memory error to guest. Other machine check errors
>> + * such as SLB multi-hit and TLB multi-hit are recovered
>> + * in KVM and are not passed on to guest.
>> + *
>> + * DSISR Bit for uncorrected machine check error. Based
>> + * on arch/powerpc/include/asm/mce.h
>> + */
>> +#define PPC_BIT(bit)                (0x8000000000000000ULL >> bit)
>> +#define P7_DSISR_MC_UE              (PPC_BIT(48))  /* P8 too */
>> +
>>  #else
>>  
>>  static inline uint32_t kvmppc_get_tbfreq(void)
>>
> 

-- 
Regards,
Aravinda

Re: [Qemu-devel] [PATCH v5 5/6] ppc: spapr: Enable FWNMI capability

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 6315 bytes --]

On Sun, Oct 08, 2017 at 01:56:06PM +0530, Aravinda Prasad wrote:
> 
> 
> On Wednesday 04 October 2017 07:04 AM, David Gibson wrote:
> > On Thu, Sep 28, 2017 at 04:08:21PM +0530, Aravinda Prasad wrote:
> >> Enable the KVM capability KVM_CAP_PPC_FWNMI so that
> >> the KVM causes guest exit with NMI as exit reason
> >> when it encounters a machine check exception on the
> >> address belonging to a guest. Without this capability
> >> enabled, KVM redirects machine check exceptions to
> >> guest's 0x200 vector.
> >>
> >> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
> >> ---
> >>  hw/ppc/spapr_rtas.c  |   15 +++++++++++++++
> >>  target/ppc/kvm.c     |   13 +++++++++++++
> >>  target/ppc/kvm_ppc.h |    6 ++++++
> >>  3 files changed, 34 insertions(+)
> >>
> >> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> >> index 08e9a5e..d017a67 100644
> >> --- a/hw/ppc/spapr_rtas.c
> >> +++ b/hw/ppc/spapr_rtas.c
> >> @@ -46,6 +46,7 @@
> >>  #include "qemu/cutils.h"
> >>  #include "trace.h"
> >>  #include "hw/ppc/fdt.h"
> >> +#include "kvm_ppc.h"
> >>  
> >>  static void rtas_display_character(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> >>                                     uint32_t token, uint32_t nargs,
> >> @@ -354,6 +355,20 @@ static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
> >>                                    target_ulong args,
> >>                                    uint32_t nret, target_ulong rets)
> >>  {
> >> +    int ret;
> >> +
> >> +    ret = kvmppc_fwnmi_enable(cpu);
> > 
> > If you're enabling it here, doesn't that mean you need to disable it
> > on reset?
> 
> We don't have a way in KVM to disable it once enabled.

Hm, ok.  Is it possible to simulate "old style" handling in qemu, even
if KVM is in the new mode?  I guess you can catch the NMI exit from
KVM, but redirect it back to the 0x200 vector in the guest, yes?

If that's so, you might as well try to enable this in the kernel at
machine init time, rather than waiting until nmi-register is called.
That way the user gets an error early if the kernel can't handle it
on a VM that's supposed to support it.

> 
> > 
> >> +    if (ret == 1) {
> >> +        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
> >> +        return;
> >> +    }
> >> +
> >> +    if (ret < 0) {
> >> +        rtas_st(rets, 0, RTAS_OUT_HW_ERROR);
> >> +        return;
> >> +    }
> >> +
> >>      spapr->guest_machine_check_addr = rtas_ld(args, 1);
> >>      rtas_st(rets, 0, RTAS_OUT_SUCCESS);
> >>  }
> >> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> >> index 7e4ce02..59b3322 100644
> >> --- a/target/ppc/kvm.c
> >> +++ b/target/ppc/kvm.c
> >> @@ -92,6 +92,7 @@ static int cap_mmu_radix;
> >>  static int cap_mmu_hash_v3;
> >>  static int cap_resize_hpt;
> >>  static int cap_ppc_pvr_compat;
> >> +static int cap_fwnmi;
> >>  
> >>  static uint32_t debug_inst_opcode;
> >>  
> >> @@ -150,6 +151,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> >>      cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
> >>      cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
> >>      cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
> >> +    cap_fwnmi = kvm_check_extension(s, KVM_CAP_PPC_FWNMI);
> >>      /*
> >>       * Note: setting it to false because there is not such capability
> >>       * in KVM at this moment.
> >> @@ -2142,6 +2144,17 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
> >>      }
> >>  }
> >>  
> >> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
> >> +{
> >> +    CPUState *cs = CPU(cpu);
> >> +
> >> +    if (!cap_fwnmi) {
> >> +        return 1;
> >> +    }
> >> +
> >> +    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);
> > 
> > Yeah, this is no good.  It means migration from a host that's fwnmi
> > capable to one that isn't will be subtly broken.  Instead you need to
> > make fwnmi capability a machine property.  If the property is
> > requested and the host kernel doesn't support it, you need to outright
> > fail, rather than try to fall back.
> 
> Sure.
> 
> > 
> >> +}
> >> +
> >>  int kvmppc_smt_threads(void)
> >>  {
> >>      return cap_ppc_smt ? cap_ppc_smt : 1;
> >> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
> >> index 0139dae..55b6df2 100644
> >> --- a/target/ppc/kvm_ppc.h
> >> +++ b/target/ppc/kvm_ppc.h
> >> @@ -28,6 +28,7 @@ void kvmppc_enable_clear_ref_mod_hcalls(void);
> >>  void kvmppc_set_papr(PowerPCCPU *cpu);
> >>  int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
> >>  void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
> >> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu);
> >>  int kvmppc_smt_threads(void);
> >>  void kvmppc_hint_smt_possible(Error **errp);
> >>  int kvmppc_set_smt_threads(int smt);
> >> @@ -157,6 +158,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
> >>  {
> >>  }
> >>  
> >> +int kvmppc_fwnmi_enable(PowerPCCPU *cpu)
> >> +{
> >> +    return 1;
> > 
> > Likewise, this should be available, not banned, on TCG.  I think there
> > are existing problems with TCG<->KVM migration, but there's no
> > inherent reason they shouldn't work, so we don't want to introduce
> > extra reasons they don't.
> > 
> > Even if TCG will never generate fwnmis (for now), it should allow the
> > guest to register for them.
> 
> Should this be then changed later only when TCG generates fwnmis?
> Because we don't want to set spapr->guest_machine_check_addr for TCG in
> rtas_ibm_nmi_register(). If set then all machine checks are redirected
> to this address which is not desirable for TCG as we still want machine
> checks to be passed via 0x200.

Um.. why?  The guest has asked for (synchronous) machine checks to go
to its specified address.  That's what it should get, regardless of
the accelerator it's running on.

> 
> Regards,
> Aravinda
> 
> > 
> >> +}
> >> +
> >>  static inline int kvmppc_smt_threads(void)
> >>  {
> >>      return 1;
> >>
> > 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3301 bytes --]

On Sun, Oct 08, 2017 at 02:29:26PM +0530, Aravinda Prasad wrote:
> 
> 
> On Wednesday 04 October 2017 06:59 AM, David Gibson wrote:
> > On Thu, Sep 28, 2017 at 04:08:10PM +0530, Aravinda Prasad wrote:
> >> Memory error such as bit flips that cannot be corrected
> >> by hardware are passed on to the kernel for handling.
> >> If the memory address in error belongs to guest then
> >> the guest kernel is responsible for taking suitable action.
> >> Patch [1] enhances KVM to exit guest with exit reason
> >> set to KVM_EXIT_NMI in such cases.
> >>
> >> This patch handles KVM_EXIT_NMI exit. If the guest OS
> >> has registered the machine check handling routine by
> >> calling "ibm,nmi-register", then the handler builds
> >> the error log and invokes the registered handler else
> >> invokes the handler at 0x200.
> >>
> >> Note that FWNMI handles synchronous machine check exceptions
> >> triggered by the hardware and hence we do not extend
> >> such support to the "nmi" command available in the QEMU
> >> monitor. Hence, "nmi" command from the monitor will
> >> always go through 0x200 vector.
> >>
> >> [1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
> >> 	(e20bbd3d and related commits)
> > 
> > What does happen on KVM if an asynchronous machine check exception
> > occurs while in the guest?  Or under PowerVM for that matter.
> 
> AFAIK asynchronous errors take a different path in KVM as it can happen
> in a different process context.

Well, obviously, I'm wondering what impact it will have on the guest,
one way or another.

[snip]
> >> +ssize_t spapr_get_rtas_size(void)
> >> +{
> >> +    return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);
> > 
> > Erm.. because of the definition of rtas_event_log_mce, this only
> > allows for 1 byte of extended log buffer.  That doesn't seem right.
> 
> This is directly taken from the kernel's RTAS log (struct rtas_error_log
> in arch/powerpc/include/asm/rtas.h). I am not sure why they use 1 byte
> extended log buffer.

I think you'd better find out, then.

[snip]
> >> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> >> index 28b6e2e..a75e9cf 100644
> >> --- a/include/hw/ppc/spapr.h
> >> +++ b/include/hw/ppc/spapr.h
> >> @@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
> >>  #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2
> >>  #define DIAGNOSTICS_RUN_MODE_PERIODIC  3
> >>  
> >> +/* Offset from rtas-base where error log is placed */
> >> +#define RTAS_ERRLOG_OFFSET       0x200
> > 
> > Is there any particular rationale for this offset?  Our actual RTAS
> > code is 20 bytes, much smaller than this.
> 
> Just to ensure some space if in case RTAS code needs to be extended in
> future.

Hm, but IIUC, we control both sides here.  qemu puts the log into the
RTAS buffer at a particular offset, and qemu tells the guest where to
find it at a particular offset within the RTAS buffer.

So, if we need to extend the RTAS code (unlikely) we can increase our
offset, and the guest will be none the wiser.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]