[PATCH 00/19] qemu-kvm: Cleanup and switch to upstream - Part I

[PATCH 00/19] qemu-kvm: Cleanup and switch to upstream - Part I

[Jan Kiszka]

All required bits for this cleanup of qemu-kvm are now upstream and
merged back - it's time to start the show. There are now 65 patches in
my queue, and I'm planning for at least 4 rounds.

This first part primarily aims at using upstream kvm_arch_init. Further
highlights are:
 - switch to upstream CPU register get/put
 - switch to upstream kvmclock
 - sustainable breakage of IA64

Please review/merge.

Jan Kiszka (19):
  qemu-kvm: Switch to upstream mp_state functions
  qemu-kvm: x86: Use upstream kvmclock device
  Revert "introduce VMSTATE_U64"
  qemu-kvm: x86: Drop MSR reset
  qemu-kvm: Use upstream VCPU reset services
  qemu-kvm: Use upstream vcpu initialization
  qemu-kvm: Start using qemu-thread services
  qemu-kvm: Use upstream kvm_arch_get/put_registers
  qemu-kvm: Use upstream state synchronization services
  qemu-kvm: Drop remaining libkvm fragments
  qemu-kvm: Drop some more unused code
  qemu-kvm: Drop some obsolete/unused fields from kvm_context
  qemu-kvm: Refactor in-kernel irqchip and pit control
  qemu-kvm: Fold kvm_create into kvm_create_context
  qemu-kvm: Fold kvm_arch_qemu_create_context into kvm_arch_create
  qemu-kvm: Push PIT reinjection control into x86 code
  qemu-kvm: Replace kvm_show_regs/code with cpu_dump_state
  qemu-kvm: Fold kvm_init_coalesced_mmio into kvm_create_context
  qemu-kvm: x86: Use kvm_arch_init

 configure            |    1 +
 cpu-defs.h           |    1 -
 hw/hw.h              |   13 --
 hw/pc_piix.c         |    2 -
 kvm-all.c            |   18 +-
 qemu-kvm-ia64.c      |    6 +-
 qemu-kvm-x86.c       |  503 +------------------------------------------------
 qemu-kvm.c           |  381 +++++++-------------------------------
 qemu-kvm.h           |  265 +--------------------------
 savevm.c             |   23 ---
 target-i386/kvm.c    |   37 +++-
 target-i386/libkvm.h |   28 ---
 target-ppc/libkvm.c  |  102 ----------
 target-ppc/libkvm.h  |   36 ----
 14 files changed, 116 insertions(+), 1300 deletions(-)
 delete mode 100644 target-i386/libkvm.h
 delete mode 100644 target-ppc/libkvm.c
 delete mode 100644 target-ppc/libkvm.h

[PATCH 01/19] qemu-kvm: Switch to upstream mp_state functions

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

They are now equivalent to qemu-kvm's version (minus the obsolete
feature check).

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c    |   40 ++--------------------------------------
 target-i386/kvm.c |    2 --
 2 files changed, 2 insertions(+), 40 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index d37f217..26ac852 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -546,42 +546,6 @@ int kvm_arch_qemu_create_context(void)
     return 0;
 }
 
-static void kvm_arch_save_mpstate(CPUState *env)
-{
-#ifdef KVM_CAP_MP_STATE
-    int r;
-    struct kvm_mp_state mp_state;
-
-    r = kvm_get_mpstate(env, &mp_state);
-    if (r < 0) {
-        env->mp_state = -1;
-    } else {
-        env->mp_state = mp_state.mp_state;
-        if (kvm_irqchip_in_kernel()) {
-            env->halted = (env->mp_state == KVM_MP_STATE_HALTED);
-        }
-    }
-#else
-    env->mp_state = -1;
-#endif
-}
-
-static void kvm_arch_load_mpstate(CPUState *env)
-{
-#ifdef KVM_CAP_MP_STATE
-    struct kvm_mp_state mp_state;
-
-    /*
-     * -1 indicates that the host did not support GET_MP_STATE ioctl,
-     *  so don't touch it.
-     */
-    if (env->mp_state != -1) {
-        mp_state.mp_state = env->mp_state;
-        kvm_set_mpstate(env, &mp_state);
-    }
-#endif
-}
-
 #define XSAVE_CWD_RIP     2
 #define XSAVE_CWD_RDP     4
 #define XSAVE_MXCSR       6
@@ -609,7 +573,7 @@ void kvm_arch_load_regs(CPUState *env, int level)
     }
 
     if (level >= KVM_PUT_RESET_STATE) {
-        kvm_arch_load_mpstate(env);
+        kvm_put_mp_state(env);
         kvm_load_lapic(env);
     }
     if (level == KVM_PUT_FULL_STATE) {
@@ -643,7 +607,7 @@ void kvm_arch_save_regs(CPUState *env)
         perror("kvm_get_msrs FAILED");
     }
 
-    kvm_arch_save_mpstate(env);
+    kvm_get_mp_state(env);
     kvm_save_lapic(env);
     kvm_get_vcpu_events(env);
     kvm_get_debugregs(env);
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 48d4f97..7974e62 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -1280,7 +1280,6 @@ static int kvm_get_msrs(CPUState *env)
     return 0;
 }
 
-#ifdef OBSOLETE_KVM_IMPL
 static int kvm_put_mp_state(CPUState *env)
 {
     struct kvm_mp_state mp_state = { .mp_state = env->mp_state };
@@ -1303,7 +1302,6 @@ static int kvm_get_mp_state(CPUState *env)
     }
     return 0;
 }
-#endif
 
 static int kvm_put_vcpu_events(CPUState *env, int level)
 {
-- 
1.7.1

[PATCH 02/19] qemu-kvm: x86: Use upstream kvmclock device

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Upstream kvmclock device is actually more advanced these days while
using a compatible vmstate. Time to switch over.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 hw/pc_piix.c   |    2 --
 qemu-kvm-x86.c |   37 -------------------------------------
 2 files changed, 0 insertions(+), 39 deletions(-)

diff --git a/hw/pc_piix.c b/hw/pc_piix.c
index 8ee7bed..7b7c4de 100644
--- a/hw/pc_piix.c
+++ b/hw/pc_piix.c
@@ -94,11 +94,9 @@ static void pc_init1(ram_addr_t ram_size,
 
     pc_cpus_init(cpu_model);
 
-#ifdef OBSOLETE_KVM_IMPL
     if (kvmclock_enabled) {
         kvmclock_create();
     }
-#endif
 
     /* allocate ram and load rom/bios */
     pc_memory_init(ram_size, kernel_filename, kernel_cmdline, initrd_filename,
diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 26ac852..844d345 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -478,37 +478,6 @@ static int kvm_enable_tpr_access_reporting(CPUState *env)
 }
 #endif
 
-#ifdef KVM_CAP_ADJUST_CLOCK
-static struct kvm_clock_data kvmclock_data;
-
-static void kvmclock_pre_save(void *opaque)
-{
-    struct kvm_clock_data *cl = opaque;
-
-    kvm_vm_ioctl(kvm_state, KVM_GET_CLOCK, cl);
-}
-
-static int kvmclock_post_load(void *opaque, int version_id)
-{
-    struct kvm_clock_data *cl = opaque;
-
-    return kvm_vm_ioctl(kvm_state, KVM_SET_CLOCK, cl);
-}
-
-static const VMStateDescription vmstate_kvmclock= {
-    .name = "kvmclock",
-    .version_id = 1,
-    .minimum_version_id = 1,
-    .minimum_version_id_old = 1,
-    .pre_save = kvmclock_pre_save,
-    .post_load = kvmclock_post_load,
-    .fields      = (VMStateField []) {
-        VMSTATE_U64(clock, struct kvm_clock_data),
-        VMSTATE_END_OF_LIST()
-    }
-};
-#endif
-
 int kvm_arch_qemu_create_context(void)
 {
     int r;
@@ -532,12 +501,6 @@ int kvm_arch_qemu_create_context(void)
         return -1;
     }
 
-#ifdef KVM_CAP_ADJUST_CLOCK
-    if (kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK)) {
-        vmstate_register(NULL, 0, &vmstate_kvmclock, &kvmclock_data);
-    }
-#endif
-
     r = kvm_set_boot_cpu_id(0);
     if (r < 0 && r != -ENOSYS) {
         return r;
-- 
1.7.1

[PATCH 03/19] Revert "introduce VMSTATE_U64"

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This reverts commit e4d6d49061b45e7134437cdd5bae2d443eedc3bc.

It's no longer used.

Conflicts:

	savevm.c

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 hw/hw.h  |   13 -------------
 savevm.c |   23 -----------------------
 2 files changed, 0 insertions(+), 36 deletions(-)

diff --git a/hw/hw.h b/hw/hw.h
index 2ca601d..56447a7 100644
--- a/hw/hw.h
+++ b/hw/hw.h
@@ -353,10 +353,6 @@ extern const VMStateInfo vmstate_info_uint16;
 extern const VMStateInfo vmstate_info_uint32;
 extern const VMStateInfo vmstate_info_uint64;
 
-#ifdef __linux__
-extern const VMStateInfo vmstate_info_u64;
-#endif
-
 extern const VMStateInfo vmstate_info_timer;
 extern const VMStateInfo vmstate_info_ptimer;
 extern const VMStateInfo vmstate_info_buffer;
@@ -766,15 +762,6 @@ extern const VMStateDescription vmstate_ptimer;
 #define VMSTATE_UINT64(_f, _s)                                        \
     VMSTATE_UINT64_V(_f, _s, 0)
 
-/* This is needed because on linux __u64 is unsigned long long
-   and on glibc uint64_t is unsigned long on 64 bits */
-#ifdef __linux__
-#define VMSTATE_U64_V(_f, _s, _v)                                     \
-    VMSTATE_SINGLE(_f, _s, _v, vmstate_info_u64, __u64)
-#define VMSTATE_U64(_f, _s)                                           \
-    VMSTATE_U64_V(_f, _s, 0)
-#endif
-
 #define VMSTATE_UINT8_EQUAL(_f, _s)                                   \
     VMSTATE_SINGLE(_f, _s, 0, vmstate_info_uint8_equal, uint8_t)
 
diff --git a/savevm.c b/savevm.c
index e26bf6e..f4ff1a1 100644
--- a/savevm.c
+++ b/savevm.c
@@ -925,29 +925,6 @@ const VMStateInfo vmstate_info_uint64 = {
     .put  = put_uint64,
 };
 
-/* 64 bit linux kernel unsigned int */
-
-#ifdef __linux__
-static int get_u64(QEMUFile *f, void *pv, size_t size)
-{
-    uint64_t *v = pv;
-    qemu_get_be64s(f, v);
-    return 0;
-}
-
-static void put_u64(QEMUFile *f, void *pv, size_t size)
-{
-    uint64_t *v = pv;
-    qemu_put_be64s(f, v);
-}
-
-const VMStateInfo vmstate_info_u64 = {
-    .name = "__u64",
-    .get  = get_u64,
-    .put  = put_u64,
-};
-#endif /* __linux__ */
-
 /* 8 bit int. See that the received value is the same than the one
    in the field */
 
-- 
1.7.1

[PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Paravirtual MSRs are properly cleared on reset now, and blindly clearing
the rest is questionable anyway (better address those one by one,
re-initializing their backing CPU state fields).

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |   69 --------------------------------------------------------
 1 files changed, 0 insertions(+), 69 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 844d345..eb8faf2 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -26,7 +26,6 @@
 
 #define MSR_IA32_TSC            0x10
 
-static struct kvm_msr_list *kvm_msr_list;
 extern unsigned int kvm_shadow_memory;
 
 int kvm_set_tss_addr(kvm_context_t kvm, unsigned long addr)
@@ -338,35 +337,6 @@ void kvm_show_code(CPUState *env)
     fprintf(stderr, "code:%s\n", code_str);
 }
 
-
-/*
- * Returns available msr list.  User must free.
- */
-static struct kvm_msr_list *kvm_get_msr_list(void)
-{
-    struct kvm_msr_list sizer, *msrs;
-    int r;
-
-    sizer.nmsrs = 0;
-    r = kvm_ioctl(kvm_state, KVM_GET_MSR_INDEX_LIST, &sizer);
-    if (r < 0 && r != -E2BIG) {
-        return NULL;
-    }
-    /* Old kernel modules had a bug and could write beyond the provided
-       memory. Allocate at least a safe amount of 1K. */
-    msrs = qemu_malloc(MAX(1024, sizeof(*msrs) +
-                           sizer.nmsrs * sizeof(*msrs->indices)));
-
-    msrs->nmsrs = sizer.nmsrs;
-    r = kvm_ioctl(kvm_state, KVM_GET_MSR_INDEX_LIST, msrs);
-    if (r < 0) {
-        free(msrs);
-        errno = r;
-        return NULL;
-    }
-    return msrs;
-}
-
 static void print_seg(FILE *file, const char *name, struct kvm_segment *seg)
 {
     fprintf(stderr,
@@ -496,11 +466,6 @@ int kvm_arch_qemu_create_context(void)
         return r;
     }
 
-    kvm_msr_list = kvm_get_msr_list();
-    if (!kvm_msr_list) {
-        return -1;
-    }
-
     r = kvm_set_boot_cpu_id(0);
     if (r < 0 && r != -ENOSYS) {
         return r;
@@ -653,42 +618,8 @@ void kvm_arch_push_nmi(void *opaque)
 }
 #endif /* KVM_CAP_USER_NMI */
 
-static int kvm_reset_msrs(CPUState *env)
-{
-    struct {
-        struct kvm_msrs info;
-        struct kvm_msr_entry entries[100];
-    } msr_data;
-    int n;
-    struct kvm_msr_entry *msrs = msr_data.entries;
-    uint32_t index;
-    uint64_t data;
-
-    if (!kvm_msr_list) {
-        return -1;
-    }
-
-    for (n = 0; n < kvm_msr_list->nmsrs; n++) {
-        index = kvm_msr_list->indices[n];
-        switch (index) {
-        case MSR_PAT:
-            data = 0x0007040600070406ULL;
-            break;
-        default:
-            data = 0;
-        }
-        kvm_msr_entry_set(&msrs[n], kvm_msr_list->indices[n], data);
-    }
-
-    msr_data.info.nmsrs = n;
-
-    return kvm_vcpu_ioctl(env, KVM_SET_MSRS, &msr_data);
-}
-
-
 void kvm_arch_cpu_reset(CPUState *env)
 {
-    kvm_reset_msrs(env);
     kvm_arch_reset_vcpu(env);
 }
 
-- 
1.7.1

[PATCH 05/19] qemu-kvm: Use upstream VCPU reset services

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 kvm-all.c       |    2 --
 qemu-kvm-ia64.c |    2 +-
 qemu-kvm-x86.c  |    5 -----
 qemu-kvm.c      |    7 -------
 qemu-kvm.h      |    1 -
 5 files changed, 1 insertions(+), 16 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 693bb4a..a871987 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -183,14 +183,12 @@ static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot)
     return kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
 }
 
-#ifdef OBSOLETE_KVM_IMPL
 static void kvm_reset_vcpu(void *opaque)
 {
     CPUState *env = opaque;
 
     kvm_arch_reset_vcpu(env);
 }
-#endif
 
 int kvm_irqchip_in_kernel(void)
 {
diff --git a/qemu-kvm-ia64.c b/qemu-kvm-ia64.c
index 39bcbeb..c3c7918 100644
--- a/qemu-kvm-ia64.c
+++ b/qemu-kvm-ia64.c
@@ -120,7 +120,7 @@ void kvm_arch_load_mpstate(CPUState *env)
 #endif
 }
 
-void kvm_arch_cpu_reset(CPUState *env)
+void kvm_arch_reset_vcpu(CPUState *env)
 {
     if (kvm_irqchip_in_kernel(kvm_context)) {
 #ifdef KVM_CAP_MP_STATE
diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index eb8faf2..d180630 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -618,11 +618,6 @@ void kvm_arch_push_nmi(void *opaque)
 }
 #endif /* KVM_CAP_USER_NMI */
 
-void kvm_arch_cpu_reset(CPUState *env)
-{
-    kvm_arch_reset_vcpu(env);
-}
-
 #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT
 void kvm_arch_do_ioperm(void *_data)
 {
diff --git a/qemu-kvm.c b/qemu-kvm.c
index b951146..ed72d1c 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -224,13 +224,6 @@ void kvm_disable_pit_creation(kvm_context_t kvm)
     kvm->no_pit_creation = 1;
 }
 
-static void kvm_reset_vcpu(void *opaque)
-{
-    CPUState *env = opaque;
-
-    kvm_arch_cpu_reset(env);
-}
-
 static void kvm_create_vcpu(CPUState *env, int id)
 {
     long mmap_size;
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 6440d2c..79eb90d 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -663,7 +663,6 @@ int kvm_arch_has_work(CPUState *env);
 void kvm_arch_process_irqchip_events(CPUState *env);
 int kvm_arch_try_push_interrupts(void *opaque);
 void kvm_arch_push_nmi(void *opaque);
-void kvm_arch_cpu_reset(CPUState *env);
 int kvm_set_boot_cpu_id(uint32_t id);
 
 void qemu_kvm_aio_wait_start(void);
-- 
1.7.1

[PATCH 06/19] qemu-kvm: Use upstream vcpu initialization

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This is a bit ugly, but as we overload kvm_init_vcpu with a wrapper, we
need to rename upstream's core implementation for now. Better than
keeping two identical versions of this non-trivial function around.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 kvm-all.c  |    4 +---
 qemu-kvm.c |   53 +++--------------------------------------------------
 2 files changed, 4 insertions(+), 53 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index a871987..2e7de45 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -200,8 +200,7 @@ int kvm_pit_in_kernel(void)
     return kvm_state->pit_in_kernel;
 }
 
-#ifdef OBSOLETE_KVM_IMPL
-int kvm_init_vcpu(CPUState *env)
+static int kvm_create_vcpu(CPUState *env)
 {
     KVMState *s = kvm_state;
     long mmap_size;
@@ -247,7 +246,6 @@ int kvm_init_vcpu(CPUState *env)
 err:
     return ret;
 }
-#endif
 
 /*
  * dirty pages logging control
diff --git a/qemu-kvm.c b/qemu-kvm.c
index ed72d1c..e066582 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -224,55 +224,6 @@ void kvm_disable_pit_creation(kvm_context_t kvm)
     kvm->no_pit_creation = 1;
 }
 
-static void kvm_create_vcpu(CPUState *env, int id)
-{
-    long mmap_size;
-    int r;
-    KVMState *s = kvm_state;
-
-    r = kvm_vm_ioctl(kvm_state, KVM_CREATE_VCPU, id);
-    if (r < 0) {
-        fprintf(stderr, "kvm_create_vcpu: %m\n");
-        fprintf(stderr, "Failed to create vCPU. Check the -smp parameter.\n");
-        goto err;
-    }
-
-    env->kvm_fd = r;
-    env->kvm_state = kvm_state;
-    env->kvm_vcpu_dirty = 1;
-
-    mmap_size = kvm_ioctl(kvm_state, KVM_GET_VCPU_MMAP_SIZE, 0);
-    if (mmap_size < 0) {
-        fprintf(stderr, "get vcpu mmap size: %m\n");
-        goto err_fd;
-    }
-    env->kvm_run =
-        mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_SHARED, env->kvm_fd,
-             0);
-    if (env->kvm_run == MAP_FAILED) {
-        fprintf(stderr, "mmap vcpu area: %m\n");
-        goto err_fd;
-    }
-
-#ifdef KVM_CAP_COALESCED_MMIO
-    if (s->coalesced_mmio && !s->coalesced_mmio_ring)
-        s->coalesced_mmio_ring = (void *) env->kvm_run +
-               s->coalesced_mmio * PAGE_SIZE;
-#endif
-
-    r = kvm_arch_init_vcpu(env);
-    if (r == 0) {
-        qemu_register_reset(kvm_reset_vcpu, env);
-    }
-
-    return;
-  err_fd:
-    close(env->kvm_fd);
-  err:
-    /* We're no good with semi-broken states. */
-    abort();
-}
-
 static int kvm_set_boot_vcpu_id(kvm_context_t kvm, uint32_t id)
 {
 #ifdef KVM_CAP_SET_BOOT_CPU_ID
@@ -1436,7 +1387,9 @@ static void *ap_main_loop(void *_env)
     pthread_mutex_lock(&qemu_mutex);
     cpu_single_env = env;
 
-    kvm_create_vcpu(env, env->cpu_index);
+    if (kvm_create_vcpu(env) < 0) {
+        abort();
+    }
     setup_kernel_sigmask(env);
 
     /* signal VCPU creation */
-- 
1.7.1

[PATCH 07/19] qemu-kvm: Start using qemu-thread services

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Convert VCPU thread creation to qemu_thread_create. This allows to drop
the thread field from KVMCPUState. It requires us to rename qemu-kvm's
qemu_cond_wait as a temporary solution until we find a common version.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 configure      |    1 +
 cpu-defs.h     |    1 -
 qemu-kvm-x86.c |    4 ++--
 qemu-kvm.c     |   29 ++++++++++++++---------------
 4 files changed, 17 insertions(+), 18 deletions(-)

diff --git a/configure b/configure
index 75071ad..2aa9075 100755
--- a/configure
+++ b/configure
@@ -3382,6 +3382,7 @@ case "$target_arch2" in
       \( "$target_arch2" = "x86_64" -a "$cpu" = "i386"   \) -o \
       \( "$target_arch2" = "i386"   -a "$cpu" = "x86_64" \) \) ; then
       echo "CONFIG_KVM=y" >> $config_target_mak
+      echo "CONFIG_THREAD=y" >> $config_host_mak
       echo "KVM_CFLAGS=$kvm_cflags" >> $config_target_mak
       if test "$kvm_para" = "yes"; then
         echo "CONFIG_KVM_PARA=y" >> $config_target_mak
diff --git a/cpu-defs.h b/cpu-defs.h
index e3d3546..624fd1a 100644
--- a/cpu-defs.h
+++ b/cpu-defs.h
@@ -158,7 +158,6 @@ typedef struct CPUWatchpoint {
 struct qemu_work_item;
 
 struct KVMCPUState {
-    pthread_t thread;
     int signalled;
     struct qemu_work_item *queued_work_first, *queued_work_last;
 };
diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index d180630..9d546d0 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -486,7 +486,7 @@ void kvm_arch_load_regs(CPUState *env, int level)
 {
     int rc;
 
-    assert(kvm_cpu_is_stopped(env) || env->thread_id == kvm_get_thread_id());
+    assert(kvm_cpu_is_stopped(env) || qemu_cpu_is_self(env));
 
     kvm_getput_regs(env, 1);
 
@@ -521,7 +521,7 @@ void kvm_arch_save_regs(CPUState *env)
 {
     int rc;
 
-    assert(kvm_cpu_is_stopped(env) || env->thread_id == kvm_get_thread_id());
+    assert(kvm_cpu_is_stopped(env) || qemu_cpu_is_self(env));
 
     kvm_getput_regs(env, 0);
 
diff --git a/qemu-kvm.c b/qemu-kvm.c
index e066582..d86fdcc 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -19,6 +19,7 @@
 #include "gdbstub.h"
 #include "monitor.h"
 #include "cpus.h"
+#include "qemu-thread.h"
 
 #include "qemu-kvm.h"
 #include "libkvm.h"
@@ -498,7 +499,7 @@ int kvm_run(CPUState *env)
     }
     if (env->exit_request) {
         env->exit_request = 0;
-        pthread_kill(env->kvm_cpu_state.thread, SIG_IPI);
+        pthread_kill(env->thread->thread, SIG_IPI);
     }
     r = ioctl(fd, KVM_RUN, 0);
 
@@ -1034,7 +1035,7 @@ unsigned long kvm_get_thread_id(void)
     return syscall(SYS_gettid);
 }
 
-static void qemu_cond_wait(pthread_cond_t *cond)
+static void kvm_cond_wait(pthread_cond_t *cond)
 {
     CPUState *env = cpu_single_env;
 
@@ -1090,9 +1091,9 @@ void on_vcpu(CPUState *env, void (*func)(void *data), void *data)
     wi.next = NULL;
     wi.done = false;
 
-    pthread_kill(env->kvm_cpu_state.thread, SIG_IPI);
+    pthread_kill(env->thread->thread, SIG_IPI);
     while (!wi.done) {
-        qemu_cond_wait(&qemu_work_cond);
+        kvm_cond_wait(&qemu_work_cond);
     }
 }
 
@@ -1153,8 +1154,8 @@ void kvm_update_interrupt_request(CPUState *env)
 
         if (signal) {
             env->kvm_cpu_state.signalled = 1;
-            if (env->kvm_cpu_state.thread) {
-                pthread_kill(env->kvm_cpu_state.thread, SIG_IPI);
+            if (env->thread) {
+                pthread_kill(env->thread->thread, SIG_IPI);
             }
         }
     }
@@ -1272,7 +1273,7 @@ static void pause_all_threads(void)
     while (penv) {
         if (penv != cpu_single_env) {
             penv->stop = 1;
-            pthread_kill(penv->kvm_cpu_state.thread, SIG_IPI);
+            pthread_kill(penv->thread->thread, SIG_IPI);
         } else {
             penv->stop = 0;
             penv->stopped = 1;
@@ -1282,7 +1283,7 @@ static void pause_all_threads(void)
     }
 
     while (!all_threads_paused()) {
-        qemu_cond_wait(&qemu_pause_cond);
+        kvm_cond_wait(&qemu_pause_cond);
     }
 }
 
@@ -1295,7 +1296,7 @@ static void resume_all_threads(void)
     while (penv) {
         penv->stop = 0;
         penv->stopped = 0;
-        pthread_kill(penv->kvm_cpu_state.thread, SIG_IPI);
+        pthread_kill(penv->thread->thread, SIG_IPI);
         penv = (CPUState *) penv->next_cpu;
     }
 }
@@ -1368,15 +1369,12 @@ static int kvm_main_loop_cpu(CPUState *env)
 static void *ap_main_loop(void *_env)
 {
     CPUState *env = _env;
-    sigset_t signals;
 #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT
     struct ioperm_data *data = NULL;
 #endif
 
     current_env = env;
     env->thread_id = kvm_get_thread_id();
-    sigfillset(&signals);
-    sigprocmask(SIG_BLOCK, &signals, NULL);
 
 #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT
     /* do ioperm for io ports of assigned devices */
@@ -1398,7 +1396,7 @@ static void *ap_main_loop(void *_env)
 
     /* and wait for machine initialization */
     while (!qemu_system_ready) {
-        qemu_cond_wait(&qemu_system_cond);
+        kvm_cond_wait(&qemu_system_cond);
     }
 
     /* re-initialize cpu_single_env after re-acquiring qemu_mutex */
@@ -1410,10 +1408,11 @@ static void *ap_main_loop(void *_env)
 
 int kvm_init_vcpu(CPUState *env)
 {
-    pthread_create(&env->kvm_cpu_state.thread, NULL, ap_main_loop, env);
+    env->thread = qemu_mallocz(sizeof(QemuThread));
+    qemu_thread_create(env->thread, ap_main_loop, env);
 
     while (env->created == 0) {
-        qemu_cond_wait(&qemu_vcpu_cond);
+        kvm_cond_wait(&qemu_vcpu_cond);
     }
 
     return 0;
-- 
1.7.1

[PATCH 08/19] qemu-kvm: Use upstream kvm_arch_get/put_registers

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Having two implementations of these services used to be a source of
constant pain in the past. Now they are almost equivalent, and we can
switch to upstream's versions. We just need to import the APIC bits into
the upstream code until they are merge as well.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-ia64.c   |    4 +-
 qemu-kvm-x86.c    |   67 -----------------------------------------------------
 qemu-kvm.c        |    8 +++---
 qemu-kvm.h        |    2 -
 target-i386/kvm.c |   12 +++++++--
 5 files changed, 15 insertions(+), 78 deletions(-)

diff --git a/qemu-kvm-ia64.c b/qemu-kvm-ia64.c
index c3c7918..bda57b6 100644
--- a/qemu-kvm-ia64.c
+++ b/qemu-kvm-ia64.c
@@ -16,12 +16,12 @@ int kvm_arch_qemu_create_context(void)
     return 0;
 }
 
-void kvm_arch_load_regs(CPUState *env, int level)
+void kvm_arch_put_registers(CPUState *env, int level)
 {
 }
 
 
-void kvm_arch_save_regs(CPUState *env)
+void kvm_arch_get_registers(CPUState *env)
 {
 }
 
diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 9d546d0..f7cd30d 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -474,73 +474,6 @@ int kvm_arch_qemu_create_context(void)
     return 0;
 }
 
-#define XSAVE_CWD_RIP     2
-#define XSAVE_CWD_RDP     4
-#define XSAVE_MXCSR       6
-#define XSAVE_ST_SPACE    8
-#define XSAVE_XMM_SPACE   40
-#define XSAVE_XSTATE_BV   128
-#define XSAVE_YMMH_SPACE  144
-
-void kvm_arch_load_regs(CPUState *env, int level)
-{
-    int rc;
-
-    assert(kvm_cpu_is_stopped(env) || qemu_cpu_is_self(env));
-
-    kvm_getput_regs(env, 1);
-
-    kvm_put_xsave(env);
-    kvm_put_xcrs(env);
-
-    kvm_put_sregs(env);
-
-    rc = kvm_put_msrs(env, level);
-    if (rc < 0) {
-        perror("kvm__msrs FAILED");
-    }
-
-    if (level >= KVM_PUT_RESET_STATE) {
-        kvm_put_mp_state(env);
-        kvm_load_lapic(env);
-    }
-    if (level == KVM_PUT_FULL_STATE) {
-        if (env->kvm_vcpu_update_vapic) {
-            kvm_tpr_enable_vapic(env);
-        }
-    }
-
-    kvm_put_vcpu_events(env, level);
-    kvm_put_debugregs(env);
-
-    /* must be last */
-    kvm_guest_debug_workarounds(env);
-}
-
-void kvm_arch_save_regs(CPUState *env)
-{
-    int rc;
-
-    assert(kvm_cpu_is_stopped(env) || qemu_cpu_is_self(env));
-
-    kvm_getput_regs(env, 0);
-
-    kvm_get_xsave(env);
-    kvm_get_xcrs(env);
-
-    kvm_get_sregs(env);
-
-    rc = kvm_get_msrs(env);
-    if (rc < 0) {
-        perror("kvm_get_msrs FAILED");
-    }
-
-    kvm_get_mp_state(env);
-    kvm_save_lapic(env);
-    kvm_get_vcpu_events(env);
-    kvm_get_debugregs(env);
-}
-
 static int _kvm_arch_init_vcpu(CPUState *env)
 {
     kvm_arch_reset_vcpu(env);
diff --git a/qemu-kvm.c b/qemu-kvm.c
index d86fdcc..508580e 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -483,7 +483,7 @@ int kvm_run(CPUState *env)
 
   again:
     if (env->kvm_vcpu_dirty) {
-        kvm_arch_load_regs(env, KVM_PUT_RUNTIME_STATE);
+        kvm_arch_put_registers(env, KVM_PUT_RUNTIME_STATE);
         env->kvm_vcpu_dirty = 0;
     }
     push_nmi(kvm);
@@ -1102,7 +1102,7 @@ static void do_kvm_cpu_synchronize_state(void *_env)
     CPUState *env = _env;
 
     if (!env->kvm_vcpu_dirty) {
-        kvm_arch_save_regs(env);
+        kvm_arch_get_registers(env);
         env->kvm_vcpu_dirty = 1;
     }
 }
@@ -1116,13 +1116,13 @@ void kvm_cpu_synchronize_state(CPUState *env)
 
 void kvm_cpu_synchronize_post_reset(CPUState *env)
 {
-    kvm_arch_load_regs(env, KVM_PUT_RESET_STATE);
+    kvm_arch_put_registers(env, KVM_PUT_RESET_STATE);
     env->kvm_vcpu_dirty = 0;
 }
 
 void kvm_cpu_synchronize_post_init(CPUState *env)
 {
-    kvm_arch_load_regs(env, KVM_PUT_FULL_STATE);
+    kvm_arch_put_registers(env, KVM_PUT_FULL_STATE);
     env->kvm_vcpu_dirty = 0;
 }
 
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 79eb90d..654ea2e 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -657,8 +657,6 @@ int kvm_qemu_destroy_memory_alias(uint64_t phys_start);
 
 int kvm_arch_qemu_create_context(void);
 
-void kvm_arch_save_regs(CPUState *env);
-void kvm_arch_load_regs(CPUState *env, int level);
 int kvm_arch_has_work(CPUState *env);
 void kvm_arch_process_irqchip_events(CPUState *env);
 int kvm_arch_try_push_interrupts(void *opaque);
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 7974e62..bbc324b 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -331,7 +331,6 @@ int kvm_arch_on_sigbus(int code, void *addr)
     return 0;
 }
 
-#ifdef OBSOLETE_KVM_IMPL
 static int kvm_inject_mce_oldstyle(CPUState *env)
 {
 #ifdef KVM_CAP_MCE
@@ -363,7 +362,6 @@ static int kvm_inject_mce_oldstyle(CPUState *env)
 #endif /* KVM_CAP_MCE */
     return 0;
 }
-#endif
 
 static void cpu_update_state(void *opaque, int running, int reason)
 {
@@ -1454,7 +1452,6 @@ static int kvm_get_debugregs(CPUState *env)
     return 0;
 }
 
-#ifdef OBSOLETE_KVM_IMPL
 int kvm_arch_put_registers(CPUState *env, int level)
 {
     int ret;
@@ -1491,6 +1488,8 @@ int kvm_arch_put_registers(CPUState *env, int level)
         if (ret < 0) {
             return ret;
         }
+
+        kvm_load_lapic(env);
     }
     ret = kvm_put_vcpu_events(env, level);
     if (ret < 0) {
@@ -1500,6 +1499,11 @@ int kvm_arch_put_registers(CPUState *env, int level)
     if (ret < 0) {
         return ret;
     }
+    if (level == KVM_PUT_FULL_STATE) {
+        if (env->kvm_vcpu_update_vapic) {
+            kvm_tpr_enable_vapic(env);
+        }
+    }
     /* must be last */
     ret = kvm_guest_debug_workarounds(env);
     if (ret < 0) {
@@ -1538,6 +1542,7 @@ int kvm_arch_get_registers(CPUState *env)
     if (ret < 0) {
         return ret;
     }
+    kvm_save_lapic(env);
     ret = kvm_get_vcpu_events(env);
     if (ret < 0) {
         return ret;
@@ -1549,6 +1554,7 @@ int kvm_arch_get_registers(CPUState *env)
     return 0;
 }
 
+#ifdef OBSOLETE_KVM_IMPL
 void kvm_arch_pre_run(CPUState *env, struct kvm_run *run)
 {
     int ret;
-- 
1.7.1

[PATCH 09/19] qemu-kvm: Use upstream state synchronization services

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Both implementations are identical if we map run_on_cpu on qemu-kvm's
on_vcpu.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 kvm-all.c  |   12 ++++++------
 qemu-kvm.c |   29 -----------------------------
 2 files changed, 6 insertions(+), 35 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 2e7de45..c1be056 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -33,6 +33,10 @@
 #include <sys/eventfd.h>
 #endif
 
+#ifndef OBSOLETE_KVM_IMPL
+#define run_on_cpu on_vcpu
+#endif /* !OBSOLETE_KVM_IMPL */
+
 /* KVM uses PAGE_SIZE in it's definition of COALESCED_MMIO_MAX */
 #define PAGE_SIZE TARGET_PAGE_SIZE
 
@@ -896,8 +900,6 @@ void kvm_flush_coalesced_mmio_buffer(void)
     }
 }
 
-#ifdef OBSOLETE_KVM_IMPL
-
 static void do_kvm_cpu_synchronize_state(void *_env)
 {
     CPUState *env = _env;
@@ -927,6 +929,8 @@ void kvm_cpu_synchronize_post_init(CPUState *env)
     env->kvm_vcpu_dirty = 0;
 }
 
+#ifdef OBSOLETE_KVM_IMPL
+
 int kvm_cpu_exec(CPUState *env)
 {
     struct kvm_run *run = env->kvm_run;
@@ -1137,10 +1141,6 @@ void kvm_setup_guest_memory(void *start, size_t size)
 }
 
 #ifdef KVM_CAP_SET_GUEST_DEBUG
-#ifndef OBSOLETE_KVM_IMPL
-#define run_on_cpu on_vcpu
-#endif /* !OBSOLETE_KVM_IMPL */
-
 struct kvm_sw_breakpoint *kvm_find_sw_breakpoint(CPUState *env,
                                                  target_ulong pc)
 {
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 508580e..d65ff67 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -1097,35 +1097,6 @@ void on_vcpu(CPUState *env, void (*func)(void *data), void *data)
     }
 }
 
-static void do_kvm_cpu_synchronize_state(void *_env)
-{
-    CPUState *env = _env;
-
-    if (!env->kvm_vcpu_dirty) {
-        kvm_arch_get_registers(env);
-        env->kvm_vcpu_dirty = 1;
-    }
-}
-
-void kvm_cpu_synchronize_state(CPUState *env)
-{
-    if (!env->kvm_vcpu_dirty) {
-        on_vcpu(env, do_kvm_cpu_synchronize_state, env);
-    }
-}
-
-void kvm_cpu_synchronize_post_reset(CPUState *env)
-{
-    kvm_arch_put_registers(env, KVM_PUT_RESET_STATE);
-    env->kvm_vcpu_dirty = 0;
-}
-
-void kvm_cpu_synchronize_post_init(CPUState *env)
-{
-    kvm_arch_put_registers(env, KVM_PUT_FULL_STATE);
-    env->kvm_vcpu_dirty = 0;
-}
-
 static void inject_interrupt(void *data)
 {
     cpu_interrupt(current_env, (long) data);
-- 
1.7.1

[PATCH 10/19] qemu-kvm: Drop remaining libkvm fragments

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

The bits under kvm/ are kept for now, not sure if they still serve any
purpose. Same for anything in target-ia64. That arch is now definitely
broken, but I bet it was already.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c       |    3 +-
 qemu-kvm.c           |    6 ---
 qemu-kvm.h           |   11 -----
 target-i386/libkvm.h |   28 --------------
 target-ppc/libkvm.c  |  102 --------------------------------------------------
 target-ppc/libkvm.h  |   36 -----------------
 6 files changed, 1 insertions(+), 185 deletions(-)
 delete mode 100644 target-i386/libkvm.h
 delete mode 100644 target-ppc/libkvm.c
 delete mode 100644 target-ppc/libkvm.h

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index f7cd30d..5207e29 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -15,7 +15,6 @@
 #include <sys/io.h>
 
 #include "qemu-kvm.h"
-#include "libkvm.h"
 #include <pthread.h>
 #include <sys/utsname.h>
 #include <linux/kvm_para.h>
@@ -28,7 +27,7 @@
 
 extern unsigned int kvm_shadow_memory;
 
-int kvm_set_tss_addr(kvm_context_t kvm, unsigned long addr)
+static int kvm_set_tss_addr(kvm_context_t kvm, unsigned long addr)
 {
     int r;
 
diff --git a/qemu-kvm.c b/qemu-kvm.c
index d65ff67..816e877 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -22,7 +22,6 @@
 #include "qemu-thread.h"
 
 #include "qemu-kvm.h"
-#include "libkvm.h"
 
 #include <pthread.h>
 #include <sys/utsname.h>
@@ -84,9 +83,6 @@ static QLIST_HEAD(, ioperm_data) ioperm_head;
 
 #define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1))
 
-int kvm_abi = EXPECTED_KVM_API_VERSION;
-int kvm_page_size;
-
 static int handle_unhandled(uint64_t reason)
 {
     fprintf(stderr, "kvm: unhandled exit %" PRIx64 "\n", reason);
@@ -162,8 +158,6 @@ int kvm_init(void)
         fprintf(stderr, "kvm userspace version too old\n");
         goto out_close;
     }
-    kvm_abi = r;
-    kvm_page_size = getpagesize();
     kvm_state = qemu_mallocz(sizeof(*kvm_state));
     kvm_context = &kvm_state->kvm_context;
 
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 654ea2e..4daadc7 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -43,9 +43,6 @@
 #define MAX_VCPUS 16
 #endif
 
-/* kvm abi verison variable */
-extern int kvm_abi;
-
 /**
  * \brief The KVM context
  *
@@ -457,14 +454,6 @@ int kvm_enable_vapic(CPUState *env, uint64_t vapic);
 
 #endif
 
-#if defined(__s390__)
-int kvm_s390_initial_reset(kvm_context_t kvm, int slot);
-int kvm_s390_interrupt(kvm_context_t kvm, int slot,
-                       struct kvm_s390_interrupt *kvmint);
-int kvm_s390_set_initial_psw(kvm_context_t kvm, int slot, psw_t psw);
-int kvm_s390_store_status(kvm_context_t kvm, int slot, unsigned long addr);
-#endif
-
 #ifdef KVM_CAP_DEVICE_ASSIGNMENT
 /*!
  * \brief Notifies host kernel about a PCI device to be assigned to a guest
diff --git a/target-i386/libkvm.h b/target-i386/libkvm.h
deleted file mode 100644
index d85b6a1..0000000
--- a/target-i386/libkvm.h
+++ /dev/null
@@ -1,28 +0,0 @@
-/*
- * This header is for functions & variables that will ONLY be
- * used inside libkvm for x86.
- * THESE ARE NOT EXPOSED TO THE USER AND ARE ONLY FOR USE
- * WITHIN LIBKVM.
- *
- * derived from libkvm.c
- *
- * Copyright (C) 2006 Qumranet, Inc.
- *
- * Authors:
- *	Avi Kivity   <avi@qumranet.com>
- *	Yaniv Kamay  <yaniv@qumranet.com>
- *
- * This work is licensed under the GNU LGPL license, version 2.
- */
-
-#ifndef KVM_X86_H
-#define KVM_X86_H
-
-#define PAGE_SIZE 4096ul
-#define PAGE_MASK (~(PAGE_SIZE - 1))
-
-int kvm_set_tss_addr(kvm_context_t kvm, unsigned long addr);
-
-#define smp_wmb()   asm volatile("" ::: "memory")
-
-#endif
diff --git a/target-ppc/libkvm.c b/target-ppc/libkvm.c
deleted file mode 100644
index da93026..0000000
--- a/target-ppc/libkvm.c
+++ /dev/null
@@ -1,102 +0,0 @@
-/*
- * This file contains the powerpc specific implementation for the
- * architecture dependent functions defined in kvm-common.h and
- * libkvm.h
- *
- * Copyright (C) 2006 Qumranet, Inc.
- *
- * Authors:
- *      Avi Kivity   <avi@qumranet.com>
- *      Yaniv Kamay  <yaniv@qumranet.com>
- *
- * Copyright IBM Corp. 2007,2008
- * Authors:
- * 	Jerone Young <jyoung5@us.ibm.com>
- * 	Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
- *
- * This work is licensed under the GNU LGPL license, version 2.
- */
-
-#include "libkvm-all.h"
-#include "libkvm.h"
-#include <errno.h>
-#include <stdio.h>
-#include <inttypes.h>
-
-int handle_dcr(kvm_vcpu_context_t vcpu)
-{
-	int ret = 0;
-	struct kvm_run *run = vcpu->run;
-	kvm_context_t kvm = vcpu->kvm;
-
-	if (run->dcr.is_write)
-		ret = kvm->callbacks->powerpc_dcr_write(vcpu,
-							run->dcr.dcrn,
-							run->dcr.data);
-	else
-		ret = kvm->callbacks->powerpc_dcr_read(vcpu,
-							run->dcr.dcrn,
-							&(run->dcr.data));
-
-	return ret;
-}
-
-void kvm_show_code(kvm_vcpu_context_t vcpu)
-{
-	fprintf(stderr, "%s: Operation not supported\n", __FUNCTION__);
-}
-
-void kvm_show_regs(kvm_vcpu_context_t vcpu)
-{
-	struct kvm_regs regs;
-	int i;
-
-	if (kvm_get_regs(vcpu, &regs))
-		return;
-
-	fprintf(stderr,"guest vcpu #%d\n", vcpu);
-	fprintf(stderr,"pc:   %016"PRIx64" msr:  %016"PRIx64"\n",
-	        regs.pc, regs.msr);
-	fprintf(stderr,"lr:   %016"PRIx64" ctr:  %016"PRIx64"\n",
-	        regs.lr, regs.ctr);
-	fprintf(stderr,"srr0: %016"PRIx64" srr1: %016"PRIx64"\n",
-	        regs.srr0, regs.srr1);
-	for (i=0; i<32; i+=4)
-	{
-		fprintf(stderr, "gpr%02d: %016"PRIx64" %016"PRIx64" %016"PRIx64
-		        " %016"PRIx64"\n", i,
-			regs.gpr[i],
-			regs.gpr[i+1],
-			regs.gpr[i+2],
-			regs.gpr[i+3]);
-	}
-
-	fflush(stdout);
-}
-
-int kvm_arch_create(kvm_context_t kvm, unsigned long phys_mem_bytes,
-			 void **vm_mem)
-{
-	int r;
-
-	r = kvm_init_coalesced_mmio(kvm);
-	if (r < 0)
-		return r;
-
-	return 0;
-}
-
-int kvm_arch_run(kvm_vcpu_context_t vcpu)
-{
-	int ret = 0;
-
-	switch (vcpu->run->exit_reason){
-	case KVM_EXIT_DCR:
-		ret = handle_dcr(vcpu);
-		break;
-	default:
-		ret = 1;
-		break;
-	}
-	return ret;
-}
diff --git a/target-ppc/libkvm.h b/target-ppc/libkvm.h
deleted file mode 100644
index 80b6b06..0000000
--- a/target-ppc/libkvm.h
+++ /dev/null
@@ -1,36 +0,0 @@
-/*
- * This header is for functions & variables that will ONLY be
- * used inside libkvm for powerpc.
- * THESE ARE NOT EXPOSED TO THE USER AND ARE ONLY FOR USE
- * WITHIN LIBKVM.
- *
- * Copyright (C) 2006 Qumranet, Inc.
- *
- * Authors:
- *	Avi Kivity   <avi@qumranet.com>
- *	Yaniv Kamay  <yaniv@qumranet.com>
- *
- * Copyright 2007 IBM Corporation.
- * Added by: Jerone Young <jyoung5@us.ibm.com>
- *
- * This work is licensed under the GNU LGPL license, version 2.
- */
-
-#ifndef KVM_POWERPC_H
-#define KVM_POWERPC_H
-
-#include "libkvm-all.h"
-
-extern int kvm_page_size;
-
-#define PAGE_SIZE kvm_page_size
-#define PAGE_MASK (~(PAGE_SIZE - 1))
-
-static inline void eieio(void)
-{
-	asm volatile("eieio" : : : "memory");
-}
-
-#define smp_wmb()	eieio()
-
-#endif
-- 
1.7.1

[PATCH 11/19] qemu-kvm: Drop some more unused code

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

No more users (except broken ia64), so remove this dead code.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |   15 -----
 qemu-kvm.c     |   92 --------------------------------
 qemu-kvm.h     |  161 --------------------------------------------------------
 3 files changed, 0 insertions(+), 268 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 5207e29..fd32d92 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -418,21 +418,6 @@ int kvm_set_shadow_pages(kvm_context_t kvm, unsigned int nrshadow_pages)
     return -1;
 }
 
-int kvm_get_shadow_pages(kvm_context_t kvm, unsigned int *nrshadow_pages)
-{
-#ifdef KVM_CAP_MMU_SHADOW_CACHE_CONTROL
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION,
-                  KVM_CAP_MMU_SHADOW_CACHE_CONTROL);
-    if (r > 0) {
-        *nrshadow_pages = kvm_vm_ioctl(kvm_state, KVM_GET_NR_MMU_PAGES);
-        return 0;
-    }
-#endif
-    return -1;
-}
-
 #ifdef KVM_CAP_VAPIC
 static int kvm_enable_tpr_access_reporting(CPUState *env)
 {
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 816e877..85f9d1c 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -31,9 +31,6 @@
 #include "compatfd.h"
 #include <sys/prctl.h>
 
-#define false 0
-#define true 1
-
 #ifndef PR_MCE_KILL
 #define PR_MCE_KILL 33
 #endif
@@ -71,9 +68,6 @@ static int qemu_system_ready;
 
 #define SIG_IPI (SIGRTMIN+4)
 
-pthread_t io_thread;
-static int io_thread_sigfd = -1;
-
 CPUState *kvm_debug_cpu_requested;
 
 #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT
@@ -378,40 +372,6 @@ int kvm_set_irqchip(kvm_context_t kvm, struct kvm_irqchip *chip)
 
 #endif
 
-int kvm_get_regs(CPUState *env, struct kvm_regs *regs)
-{
-    return kvm_vcpu_ioctl(env, KVM_GET_REGS, regs);
-}
-
-int kvm_set_regs(CPUState *env, struct kvm_regs *regs)
-{
-    return kvm_vcpu_ioctl(env, KVM_SET_REGS, regs);
-}
-
-#ifdef KVM_CAP_MP_STATE
-int kvm_get_mpstate(CPUState *env, struct kvm_mp_state *mp_state)
-{
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_MP_STATE);
-    if (r > 0) {
-        return kvm_vcpu_ioctl(env, KVM_GET_MP_STATE, mp_state);
-    }
-    return -ENOSYS;
-}
-
-int kvm_set_mpstate(CPUState *env, struct kvm_mp_state *mp_state)
-{
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_MP_STATE);
-    if (r > 0) {
-        return kvm_vcpu_ioctl(env, KVM_SET_MP_STATE, mp_state);
-    }
-    return -ENOSYS;
-}
-#endif
-
 static int handle_mmio(CPUState *env)
 {
     unsigned long addr = env->kvm_run->mmio.phys_addr;
@@ -979,51 +939,6 @@ int kvm_assign_set_msix_entry(kvm_context_t kvm,
 }
 #endif
 
-#if defined(KVM_CAP_IRQFD) && defined(CONFIG_EVENTFD)
-
-#include <sys/eventfd.h>
-
-static int _kvm_irqfd(kvm_context_t kvm, int fd, int gsi, int flags)
-{
-    struct kvm_irqfd data = {
-        .fd = fd,
-        .gsi = gsi,
-        .flags = flags,
-    };
-
-    return kvm_vm_ioctl(kvm_state, KVM_IRQFD, &data);
-}
-
-int kvm_irqfd(kvm_context_t kvm, int gsi, int flags)
-{
-    int r;
-    int fd;
-
-    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQFD))
-        return -ENOENT;
-
-    fd = eventfd(0, 0);
-    if (fd < 0) {
-        return -errno;
-    }
-
-    r = _kvm_irqfd(kvm, fd, gsi, 0);
-    if (r < 0) {
-        close(fd);
-        return -errno;
-    }
-
-    return fd;
-}
-
-#else                           /* KVM_CAP_IRQFD */
-
-int kvm_irqfd(kvm_context_t kvm, int gsi, int flags)
-{
-    return -ENOSYS;
-}
-
-#endif                          /* KVM_CAP_IRQFD */
 unsigned long kvm_get_thread_id(void)
 {
     return syscall(SYS_gettid);
@@ -1383,11 +1298,6 @@ int kvm_init_vcpu(CPUState *env)
     return 0;
 }
 
-int kvm_vcpu_inited(CPUState *env)
-{
-    return env->created;
-}
-
 #ifdef TARGET_I386
 void kvm_hpet_disable_kpit(void)
 {
@@ -1465,7 +1375,6 @@ int kvm_main_loop(void)
     sigset_t mask;
     int sigfd;
 
-    io_thread = pthread_self();
     qemu_system_ready = 1;
 
     sigemptyset(&mask);
@@ -1487,7 +1396,6 @@ int kvm_main_loop(void)
 
     pthread_cond_broadcast(&qemu_system_cond);
 
-    io_thread_sigfd = sigfd;
     cpu_single_env = NULL;
 
     while (1) {
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 4daadc7..d62f38d 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -72,10 +72,6 @@ struct kvm_context {
 typedef struct kvm_context *kvm_context_t;
 
 #include "kvm.h"
-int kvm_alloc_kernel_memory(kvm_context_t kvm, unsigned long memory,
-                            void **vm_mem);
-int kvm_alloc_userspace_memory(kvm_context_t kvm, unsigned long memory,
-                               void **vm_mem);
 
 int kvm_arch_create(kvm_context_t kvm, unsigned long phys_mem_bytes,
                     void **vm_mem);
@@ -93,10 +89,6 @@ int pre_kvm_run(kvm_context_t kvm, CPUState *env);
 int handle_io_window(kvm_context_t kvm);
 int try_push_interrupts(kvm_context_t kvm);
 
-#if defined(__x86_64__) || defined(__i386__)
-struct kvm_x86_mce;
-#endif
-
 /*!
  * \brief Disable the in-kernel IRQCHIP creation
  *
@@ -170,52 +162,6 @@ int kvm_run(CPUState *env);
  */
 int kvm_is_ready_for_interrupt_injection(CPUState *env);
 
-/*!
- * \brief Read VCPU registers
- *
- * This gets the GP registers from the VCPU and outputs them
- * into a kvm_regs structure
- *
- * \note This function returns a \b copy of the VCPUs registers.\n
- * If you wish to modify the VCPUs GP registers, you should call kvm_set_regs()
- *
- * \param kvm Pointer to the current kvm_context
- * \param vcpu Which virtual CPU should get dumped
- * \param regs Pointer to a kvm_regs which will be populated with the VCPUs
- * registers values
- * \return 0 on success
- */
-int kvm_get_regs(CPUState *env, struct kvm_regs *regs);
-
-/*!
- * \brief Write VCPU registers
- *
- * This sets the GP registers on the VCPU from a kvm_regs structure
- *
- * \note When this function returns, the regs pointer and the data it points to
- * can be discarded
- * \param kvm Pointer to the current kvm_context
- * \param vcpu Which virtual CPU should get dumped
- * \param regs Pointer to a kvm_regs which will be populated with the VCPUs
- * registers values
- * \return 0 on success
- */
-int kvm_set_regs(CPUState *env, struct kvm_regs *regs);
-
-#ifdef KVM_CAP_MP_STATE
-/*!
- *  * \brief Read VCPU MP state
- *
- */
-int kvm_get_mpstate(CPUState *env, struct kvm_mp_state *mp_state);
-
-/*!
- *  * \brief Write VCPU MP state
- *
- */
-int kvm_set_mpstate(CPUState *env, struct kvm_mp_state *mp_state);
-#endif
-
 #if defined(__i386__) || defined(__x86_64__)
 /*!
  * \brief Simulate an external vectored interrupt
@@ -238,14 +184,6 @@ int kvm_inject_irq(CPUState *env, unsigned irq);
  */
 int kvm_set_shadow_pages(kvm_context_t kvm, unsigned int nrshadow_pages);
 
-/*!
- * \brief Getting the number of shadow pages that are allocated to the vm
- *
- * \param kvm pointer to kvm_context
- * \param nrshadow_pages number of pages to be allocated
- */
-int kvm_get_shadow_pages(kvm_context_t kvm, unsigned int *nrshadow_pages);
-
 #endif
 
 /*!
@@ -262,62 +200,8 @@ int kvm_get_shadow_pages(kvm_context_t kvm, unsigned int *nrshadow_pages);
  */
 void kvm_show_regs(CPUState *env);
 
-
-void *kvm_create_phys_mem(kvm_context_t, unsigned long phys_start,
-                          unsigned long len, int log, int writable);
-void kvm_destroy_phys_mem(kvm_context_t, unsigned long phys_start,
-                          unsigned long len);
-
-int kvm_is_containing_region(kvm_context_t kvm, unsigned long phys_start,
-                             unsigned long size);
-int kvm_register_phys_mem(kvm_context_t kvm, unsigned long phys_start,
-                          void *userspace_addr, unsigned long len, int log);
-int kvm_get_dirty_pages_range(kvm_context_t kvm, unsigned long phys_addr,
-                              unsigned long end_addr, void *opaque,
-                              int (*cb)(unsigned long start,
-                                        unsigned long len, void *bitmap,
-                                        void *opaque));
-int kvm_register_coalesced_mmio(kvm_context_t kvm, uint64_t addr,
-                                uint32_t size);
-int kvm_unregister_coalesced_mmio(kvm_context_t kvm, uint64_t addr,
-                                  uint32_t size);
-
-/*!
- * \brief Get a bitmap of guest ram pages which are allocated to the guest.
- *
- * \param kvm Pointer to the current kvm_context
- * \param phys_addr Memory slot phys addr
- * \param bitmap Long aligned address of a big enough bitmap (one bit per page)
- */
-int kvm_get_mem_map(kvm_context_t kvm, unsigned long phys_addr, void *bitmap);
-int kvm_get_mem_map_range(kvm_context_t kvm, unsigned long phys_addr,
-                          unsigned long len, void *buf, void *opaque,
-                          int (*cb)(unsigned long start,
-                                    unsigned long len, void *bitmap,
-                                    void *opaque));
 int kvm_set_irq_level(kvm_context_t kvm, int irq, int level, int *status);
 
-int kvm_dirty_pages_log_enable_slot(kvm_context_t kvm, uint64_t phys_start,
-                                    uint64_t len);
-int kvm_dirty_pages_log_disable_slot(kvm_context_t kvm, uint64_t phys_start,
-                                     uint64_t len);
-/*!
- * \brief Enable dirty-pages-logging for all memory regions
- *
- * \param kvm Pointer to the current kvm_context
- */
-int kvm_dirty_pages_log_enable_all(kvm_context_t kvm);
-
-/*!
- * \brief Disable dirty-page-logging for some memory regions
- *
- * Disable dirty-pages-logging for those memory regions that were
- * created with dirty-page-logging disabled.
- *
- * \param kvm Pointer to the current kvm_context
- */
-int kvm_dirty_pages_log_reset(kvm_context_t kvm);
-
 #ifdef KVM_CAP_IRQCHIP
 /*!
  * \brief Dump in kernel IRQCHIP contents
@@ -570,20 +454,6 @@ int kvm_update_routing_entry(struct kvm_irq_routing_entry *entry,
                              struct kvm_irq_routing_entry *newentry);
 
 
-/*!
- * \brief Create a file descriptor for injecting interrupts
- *
- * Creates an eventfd based file-descriptor that maps to a specific GSI
- * in the guest.  eventfd compliant signaling (write() from userspace, or
- * eventfd_signal() from kernelspace) will cause the GSI to inject
- * itself into the guest at the next available window.
- *
- * \param kvm Pointer to the current kvm_context
- * \param gsi GSI to assign to this fd
- * \param flags reserved, must be zero
- */
-int kvm_irqfd(kvm_context_t kvm, int gsi, int flags);
-
 #ifdef KVM_CAP_DEVICE_MSIX
 int kvm_assign_set_msix_nr(kvm_context_t kvm,
                            struct kvm_assigned_msix_nr *msix_nr);
@@ -594,7 +464,6 @@ int kvm_assign_set_msix_entry(kvm_context_t kvm,
 #else                           /* !CONFIG_KVM */
 
 typedef struct kvm_context *kvm_context_t;
-typedef struct kvm_vcpu_context *kvm_vcpu_context_t;
 
 struct kvm_pit_state {
 };
@@ -616,33 +485,15 @@ int kvm_init(void);
 
 int kvm_main_loop(void);
 int kvm_init_ap(void);
-int kvm_vcpu_inited(CPUState *env);
 void kvm_save_lapic(CPUState *env);
 void kvm_load_lapic(CPUState *env);
 
 void kvm_hpet_enable_kpit(void);
 void kvm_hpet_disable_kpit(void);
 
-int kvm_physical_memory_set_dirty_tracking(int enable);
-
 void on_vcpu(CPUState *env, void (*func)(void *data), void *data);
-void qemu_kvm_call_with_env(void (*func)(void *), void *data, CPUState *env);
-void qemu_kvm_cpuid_on_env(CPUState *env);
 void kvm_inject_interrupt(CPUState *env, int mask);
-void kvm_update_after_sipi(CPUState *env);
 void kvm_update_interrupt_request(CPUState *env);
-#ifndef CONFIG_USER_ONLY
-void *kvm_cpu_create_phys_mem(target_phys_addr_t start_addr, unsigned long size,
-                              int log, int writable);
-
-void kvm_cpu_destroy_phys_mem(target_phys_addr_t start_addr,
-                              unsigned long size);
-void kvm_qemu_log_memory(target_phys_addr_t start, target_phys_addr_t size,
-                         int log);
-#endif
-int kvm_qemu_create_memory_alias(uint64_t phys_start, uint64_t len,
-                                 uint64_t target_phys);
-int kvm_qemu_destroy_memory_alias(uint64_t phys_start);
 
 int kvm_arch_qemu_create_context(void);
 
@@ -652,17 +503,10 @@ int kvm_arch_try_push_interrupts(void *opaque);
 void kvm_arch_push_nmi(void *opaque);
 int kvm_set_boot_cpu_id(uint32_t id);
 
-void qemu_kvm_aio_wait_start(void);
-void qemu_kvm_aio_wait(void);
-void qemu_kvm_aio_wait_end(void);
-
 void kvm_tpr_access_report(CPUState *env, uint64_t rip, int is_write);
 
 int kvm_arch_init_irq_routing(void);
 
-int kvm_mmio_read(void *opaque, uint64_t addr, uint8_t * data, int len);
-int kvm_mmio_write(void *opaque, uint64_t addr, uint8_t * data, int len);
-
 #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT
 struct ioperm_data;
 
@@ -672,9 +516,6 @@ void kvm_remove_ioperm_data(unsigned long start_port, unsigned long num);
 void kvm_arch_do_ioperm(void *_data);
 #endif
 
-#define ALIGN(x, y)  (((x)+(y)-1) & ~((y)-1))
-#define BITMAP_SIZE(m) (ALIGN(((m)>>TARGET_PAGE_BITS), HOST_LONG_BITS) / 8)
-
 #ifdef CONFIG_KVM
 #include "qemu-queue.h"
 
@@ -691,7 +532,6 @@ struct ioperm_data {
     QLIST_ENTRY(ioperm_data) entries;
 };
 
-void qemu_kvm_cpu_stop(CPUState *env);
 int kvm_arch_halt(CPUState *env);
 int handle_tpr_access(void *opaque, CPUState *env, uint64_t rip,
                       int is_write);
@@ -706,7 +546,6 @@ int handle_tpr_access(void *opaque, CPUState *env, uint64_t rip,
 #ifdef TARGET_I386
 #define qemu_kvm_has_pit_state2() (0)
 #endif
-#define qemu_kvm_cpu_stop(env) do {} while(0)
 #endif
 
 #ifdef CONFIG_KVM
-- 
1.7.1

[PATCH 12/19] qemu-kvm: Drop some obsolete/unused fields from kvm_context

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

irqchip_in_kernel cam be migrated to the field in KVMState, and opaque
as well as dirty_pages_log_all were write-only.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |    2 +-
 qemu-kvm.c     |   18 ++++++------------
 qemu-kvm.h     |    7 +------
 3 files changed, 8 insertions(+), 19 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index fd32d92..6d3a243 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -518,7 +518,7 @@ int kvm_arch_try_push_interrupts(void *opaque)
 }
 
 #ifdef KVM_CAP_USER_NMI
-void kvm_arch_push_nmi(void *opaque)
+void kvm_arch_push_nmi(void)
 {
     CPUState *env = cpu_single_env;
     int r;
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 85f9d1c..e846f1c 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -157,10 +157,6 @@ int kvm_init(void)
 
     kvm_state->fd = fd;
     kvm_state->vmfd = -1;
-    kvm_context->opaque = cpu_single_env;
-    kvm_context->dirty_pages_log_all = 0;
-    kvm_context->no_irqchip_creation = 0;
-    kvm_context->no_pit_creation = 0;
 
 #ifdef KVM_CAP_SET_GUEST_DEBUG
     QTAILQ_INIT(&kvm_state->kvm_sw_breakpoints);
@@ -263,7 +259,6 @@ void kvm_create_irqchip(kvm_context_t kvm)
 {
     int r;
 
-    kvm->irqchip_in_kernel = 0;
 #ifdef KVM_CAP_IRQCHIP
     if (!kvm->no_irqchip_creation) {
         r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_IRQCHIP);
@@ -278,13 +273,12 @@ void kvm_create_irqchip(kvm_context_t kvm)
                     kvm->irqchip_inject_ioctl = KVM_IRQ_LINE_STATUS;
                 }
 #endif
-                kvm->irqchip_in_kernel = 1;
+                kvm_state->irqchip_in_kernel = 1;
             } else
                 fprintf(stderr, "Create kernel PIC irqchip failed\n");
         }
     }
 #endif
-    kvm_state->irqchip_in_kernel = kvm->irqchip_in_kernel;
 }
 
 int kvm_create(kvm_context_t kvm, unsigned long phys_mem_bytes, void **vm_mem)
@@ -320,7 +314,7 @@ int kvm_set_irq_level(kvm_context_t kvm, int irq, int level, int *status)
     struct kvm_irq_level event;
     int r;
 
-    if (!kvm->irqchip_in_kernel) {
+    if (!kvm_state->irqchip_in_kernel) {
         return 0;
     }
     event.level = level;
@@ -346,7 +340,7 @@ int kvm_get_irqchip(kvm_context_t kvm, struct kvm_irqchip *chip)
 {
     int r;
 
-    if (!kvm->irqchip_in_kernel) {
+    if (!kvm_state->irqchip_in_kernel) {
         return 0;
     }
     r = kvm_vm_ioctl(kvm_state, KVM_GET_IRQCHIP, chip);
@@ -360,7 +354,7 @@ int kvm_set_irqchip(kvm_context_t kvm, struct kvm_irqchip *chip)
 {
     int r;
 
-    if (!kvm->irqchip_in_kernel) {
+    if (!kvm_state->irqchip_in_kernel) {
         return 0;
     }
     r = kvm_vm_ioctl(kvm_state, KVM_SET_IRQCHIP, chip);
@@ -404,7 +398,7 @@ int handle_shutdown(kvm_context_t kvm, CPUState *env)
 static inline void push_nmi(kvm_context_t kvm)
 {
 #ifdef KVM_CAP_USER_NMI
-    kvm_arch_push_nmi(kvm->opaque);
+    kvm_arch_push_nmi();
 #endif                          /* KVM_CAP_USER_NMI */
 }
 
@@ -442,7 +436,7 @@ int kvm_run(CPUState *env)
     }
     push_nmi(kvm);
 #if !defined(__s390__)
-    if (!kvm->irqchip_in_kernel) {
+    if (!kvm_state->irqchip_in_kernel) {
         run->request_interrupt_window = kvm_arch_try_push_interrupts(env);
     }
 #endif
diff --git a/qemu-kvm.h b/qemu-kvm.h
index d62f38d..3db0484 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -50,13 +50,8 @@
  */
 
 struct kvm_context {
-    void *opaque;
-    /// is dirty pages logging enabled for all regions or not
-    int dirty_pages_log_all;
     /// do not create in-kernel irqchip if set
     int no_irqchip_creation;
-    /// in-kernel irqchip status
-    int irqchip_in_kernel;
     /// ioctl to use to inject interrupts
     int irqchip_inject_ioctl;
     /// do not create in-kernel pit if set
@@ -500,7 +495,7 @@ int kvm_arch_qemu_create_context(void);
 int kvm_arch_has_work(CPUState *env);
 void kvm_arch_process_irqchip_events(CPUState *env);
 int kvm_arch_try_push_interrupts(void *opaque);
-void kvm_arch_push_nmi(void *opaque);
+void kvm_arch_push_nmi(void);
 int kvm_set_boot_cpu_id(uint32_t id);
 
 void kvm_tpr_access_report(CPUState *env, uint64_t rip, int is_write);
-- 
1.7.1

[PATCH 13/19] qemu-kvm: Refactor in-kernel irqchip and pit control

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Simplify the logic for detecting and querying if in-kernel irqchip and
pit should be created. This also allows to drop two further fields from
kvm_context.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |   16 +++++-----------
 qemu-kvm.c     |   20 +++-----------------
 qemu-kvm.h     |   24 ------------------------
 3 files changed, 8 insertions(+), 52 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 6d3a243..a88f8be 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -105,17 +105,11 @@ static int kvm_create_pit(kvm_context_t kvm)
 #ifdef KVM_CAP_PIT
     int r;
 
-    kvm_state->pit_in_kernel = 0;
-    if (!kvm->no_pit_creation) {
-        r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_PIT);
-        if (r > 0) {
-            r = kvm_vm_ioctl(kvm_state, KVM_CREATE_PIT);
-            if (r >= 0) {
-                kvm_state->pit_in_kernel = 1;
-            } else {
-                fprintf(stderr, "Create kernel PIC irqchip failed\n");
-                return r;
-            }
+    if (kvm_pit_in_kernel()) {
+        r = kvm_vm_ioctl(kvm_state, KVM_CREATE_PIT);
+        if (r < 0) {
+            fprintf(stderr, "Create kernel PIC irqchip failed\n");
+            return r;
         }
     }
 #endif
diff --git a/qemu-kvm.c b/qemu-kvm.c
index e846f1c..e03b3d5 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -199,16 +199,6 @@ static void kvm_finalize(KVMState *s)
     free(s);
 }
 
-void kvm_disable_irqchip_creation(kvm_context_t kvm)
-{
-    kvm->no_irqchip_creation = 1;
-}
-
-void kvm_disable_pit_creation(kvm_context_t kvm)
-{
-    kvm->no_pit_creation = 1;
-}
-
 static int kvm_set_boot_vcpu_id(kvm_context_t kvm, uint32_t id)
 {
 #ifdef KVM_CAP_SET_BOOT_CPU_ID
@@ -260,7 +250,7 @@ void kvm_create_irqchip(kvm_context_t kvm)
     int r;
 
 #ifdef KVM_CAP_IRQCHIP
-    if (!kvm->no_irqchip_creation) {
+    if (kvm_irqchip) {
         r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_IRQCHIP);
         if (r > 0) {            /* kernel irqchip supported */
             r = kvm_vm_ioctl(kvm_state, KVM_CREATE_IRQCHIP);
@@ -1437,12 +1427,8 @@ static int kvm_create_context(void)
 
     int r;
 
-    if (!kvm_irqchip) {
-        kvm_disable_irqchip_creation(kvm_context);
-    }
-    if (!kvm_pit) {
-        kvm_disable_pit_creation(kvm_context);
-    }
+    kvm_state->pit_in_kernel = kvm_pit;
+
     if (kvm_create(kvm_context, 0, NULL) < 0) {
         kvm_finalize(kvm_state);
         return -1;
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 3db0484..409809e 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -50,12 +50,8 @@
  */
 
 struct kvm_context {
-    /// do not create in-kernel irqchip if set
-    int no_irqchip_creation;
     /// ioctl to use to inject interrupts
     int irqchip_inject_ioctl;
-    /// do not create in-kernel pit if set
-    int no_pit_creation;
 #ifdef KVM_CAP_IRQ_ROUTING
     struct kvm_irq_routing *irq_routes;
     int nr_allocated_irq_routes;
@@ -85,26 +81,6 @@ int handle_io_window(kvm_context_t kvm);
 int try_push_interrupts(kvm_context_t kvm);
 
 /*!
- * \brief Disable the in-kernel IRQCHIP creation
- *
- * In-kernel irqchip is enabled by default. If userspace irqchip is to be used,
- * this should be called prior to kvm_create().
- *
- * \param kvm Pointer to the kvm_context
- */
-void kvm_disable_irqchip_creation(kvm_context_t kvm);
-
-/*!
- * \brief Disable the in-kernel PIT creation
- *
- * In-kernel pit is enabled by default. If userspace pit is to be used,
- * this should be called prior to kvm_create().
- *
- *  \param kvm Pointer to the kvm_context
- */
-void kvm_disable_pit_creation(kvm_context_t kvm);
-
-/*!
  * \brief Create new virtual machine
  *
  * This creates a new virtual machine, maps physical RAM to it, and creates a
-- 
1.7.1

[PATCH 14/19] qemu-kvm: Fold kvm_create into kvm_create_context

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

A small step towards switching to upstream code: reduce the init
function maze.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm.c |   94 +++++++++++++++++++++---------------------------------------
 qemu-kvm.h |   17 -----------
 2 files changed, 33 insertions(+), 78 deletions(-)

diff --git a/qemu-kvm.c b/qemu-kvm.c
index e03b3d5..359bec7 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -127,7 +127,7 @@ static int kvm_create_context(void);
 int kvm_init(void)
 {
     int fd;
-    int r, gsi_count;
+    int r, gsi_count, i;
 
 
     fd = open("/dev/kvm", O_RDWR);
@@ -162,6 +162,21 @@ int kvm_init(void)
     QTAILQ_INIT(&kvm_state->kvm_sw_breakpoints);
 #endif
 
+    for (i = 0; i < ARRAY_SIZE(kvm_state->slots); i++) {
+        kvm_state->slots[i].slot = i;
+    }
+
+#ifdef KVM_CAP_USER_MEMORY
+    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_USER_MEMORY);
+    if (r <= 0) {
+        fprintf(stderr,
+                "Hypervisor too old: KVM_CAP_USER_MEMORY extension not supported\n");
+        goto out_close;
+    }
+#else
+#error Hypervisor too old: KVM_CAP_USER_MEMORY extension not supported
+#endif
+
     gsi_count = kvm_get_gsi_count(kvm_context);
     if (gsi_count > 0) {
         int gsi_bits, i;
@@ -212,39 +227,6 @@ static int kvm_set_boot_vcpu_id(kvm_context_t kvm, uint32_t id)
 #endif
 }
 
-int kvm_create_vm(kvm_context_t kvm)
-{
-    int fd;
-#ifdef KVM_CAP_IRQ_ROUTING
-    kvm->irq_routes = qemu_mallocz(sizeof(*kvm->irq_routes));
-    kvm->nr_allocated_irq_routes = 0;
-#endif
-
-    fd = kvm_ioctl(kvm_state, KVM_CREATE_VM, 0);
-    if (fd < 0) {
-        fprintf(stderr, "kvm_create_vm: %m\n");
-        return -1;
-    }
-    kvm_state->vmfd = fd;
-    return 0;
-}
-
-static int kvm_create_default_phys_mem(kvm_context_t kvm,
-                                       unsigned long phys_mem_bytes,
-                                       void **vm_mem)
-{
-#ifdef KVM_CAP_USER_MEMORY
-    int r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_USER_MEMORY);
-    if (r > 0)
-        return 0;
-    fprintf(stderr,
-            "Hypervisor too old: KVM_CAP_USER_MEMORY extension not supported\n");
-#else
-#error Hypervisor too old: KVM_CAP_USER_MEMORY extension not supported
-#endif
-    return -1;
-}
-
 void kvm_create_irqchip(kvm_context_t kvm)
 {
     int r;
@@ -271,32 +253,6 @@ void kvm_create_irqchip(kvm_context_t kvm)
 #endif
 }
 
-int kvm_create(kvm_context_t kvm, unsigned long phys_mem_bytes, void **vm_mem)
-{
-    int r, i;
-
-    r = kvm_create_vm(kvm);
-    if (r < 0) {
-        return r;
-    }
-    r = kvm_arch_create(kvm, phys_mem_bytes, vm_mem);
-    if (r < 0) {
-        return r;
-    }
-    for (i = 0; i < ARRAY_SIZE(kvm_state->slots); i++) {
-        kvm_state->slots[i].slot = i;
-    }
-
-    r = kvm_create_default_phys_mem(kvm, phys_mem_bytes, vm_mem);
-    if (r < 0) {
-        return r;
-    }
-
-    kvm_create_irqchip(kvm);
-
-    return 0;
-}
-
 #ifdef KVM_CAP_IRQCHIP
 
 int kvm_set_irq_level(kvm_context_t kvm, int irq, int level, int *status)
@@ -1429,10 +1385,26 @@ static int kvm_create_context(void)
 
     kvm_state->pit_in_kernel = kvm_pit;
 
-    if (kvm_create(kvm_context, 0, NULL) < 0) {
+#ifdef KVM_CAP_IRQ_ROUTING
+    kvm_context->irq_routes = qemu_mallocz(sizeof(*kvm_context->irq_routes));
+    kvm_context->nr_allocated_irq_routes = 0;
+#endif
+
+    kvm_state->vmfd = kvm_ioctl(kvm_state, KVM_CREATE_VM, 0);
+    if (kvm_state->vmfd < 0) {
+        fprintf(stderr, "kvm_create_vm: %m\n");
         kvm_finalize(kvm_state);
         return -1;
     }
+
+    r = kvm_arch_create(kvm_context, 0, NULL);
+    if (r < 0) {
+        kvm_finalize(kvm_state);
+        return r;
+    }
+
+    kvm_create_irqchip(kvm_context);
+
     r = kvm_arch_qemu_create_context();
     if (r < 0) {
         kvm_finalize(kvm_state);
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 409809e..a4f1816 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -80,23 +80,6 @@ int pre_kvm_run(kvm_context_t kvm, CPUState *env);
 int handle_io_window(kvm_context_t kvm);
 int try_push_interrupts(kvm_context_t kvm);
 
-/*!
- * \brief Create new virtual machine
- *
- * This creates a new virtual machine, maps physical RAM to it, and creates a
- * virtual CPU for it.\n
- * \n
- * Memory gets mapped for addresses 0->0xA0000, 0xC0000->phys_mem_bytes
- *
- * \param kvm Pointer to the current kvm_context
- * \param phys_mem_bytes The amount of physical ram you want the VM to have
- * \param phys_mem This pointer will be set to point to the memory that
- * kvm_create allocates for physical RAM
- * \return 0 on success
- */
-int kvm_create(kvm_context_t kvm, unsigned long phys_mem_bytes,
-               void **phys_mem);
-int kvm_create_vm(kvm_context_t kvm);
 void kvm_create_irqchip(kvm_context_t kvm);
 
 /*!
-- 
1.7.1

[PATCH 15/19] qemu-kvm: Fold kvm_arch_qemu_create_context into kvm_arch_create

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This reorders kvm_create_irqchip and the content of
kvm_arch_qemu_create_context, but this should not cause any problems.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |   48 ++++++++++++++++++++----------------------------
 qemu-kvm.c     |    7 +------
 qemu-kvm.h     |    5 +----
 3 files changed, 22 insertions(+), 38 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index a88f8be..6b816bc 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -116,9 +116,9 @@ static int kvm_create_pit(kvm_context_t kvm)
     return 0;
 }
 
-int kvm_arch_create(kvm_context_t kvm, unsigned long phys_mem_bytes,
-                        void **vm_mem)
+int kvm_arch_create(kvm_context_t kvm)
 {
+    struct utsname utsname;
     int r = 0;
 
     r = kvm_init_tss(kvm);
@@ -149,6 +149,24 @@ int kvm_arch_create(kvm_context_t kvm, unsigned long phys_mem_bytes,
         return r;
     }
 
+    uname(&utsname);
+    lm_capable_kernel = strcmp(utsname.machine, "x86_64") == 0;
+
+    if (kvm_shadow_memory) {
+        kvm_set_shadow_pages(kvm_context, kvm_shadow_memory);
+    }
+
+    /* initialize has_msr_star/has_msr_hsave_pa */
+    r = kvm_get_supported_msrs(kvm_state);
+    if (r < 0) {
+        return r;
+    }
+
+    r = kvm_set_boot_cpu_id(0);
+    if (r < 0 && r != -ENOSYS) {
+        return r;
+    }
+
     return 0;
 }
 
@@ -426,32 +444,6 @@ static int kvm_enable_tpr_access_reporting(CPUState *env)
 }
 #endif
 
-int kvm_arch_qemu_create_context(void)
-{
-    int r;
-    struct utsname utsname;
-
-    uname(&utsname);
-    lm_capable_kernel = strcmp(utsname.machine, "x86_64") == 0;
-
-    if (kvm_shadow_memory) {
-        kvm_set_shadow_pages(kvm_context, kvm_shadow_memory);
-    }
-
-    /* initialize has_msr_star/has_msr_hsave_pa */
-    r = kvm_get_supported_msrs(kvm_state);
-    if (r < 0) {
-        return r;
-    }
-
-    r = kvm_set_boot_cpu_id(0);
-    if (r < 0 && r != -ENOSYS) {
-        return r;
-    }
-
-    return 0;
-}
-
 static int _kvm_arch_init_vcpu(CPUState *env)
 {
     kvm_arch_reset_vcpu(env);
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 359bec7..ecee3dc 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -1397,7 +1397,7 @@ static int kvm_create_context(void)
         return -1;
     }
 
-    r = kvm_arch_create(kvm_context, 0, NULL);
+    r = kvm_arch_create(kvm_context);
     if (r < 0) {
         kvm_finalize(kvm_state);
         return r;
@@ -1405,11 +1405,6 @@ static int kvm_create_context(void)
 
     kvm_create_irqchip(kvm_context);
 
-    r = kvm_arch_qemu_create_context();
-    if (r < 0) {
-        kvm_finalize(kvm_state);
-        return -1;
-    }
     if (kvm_pit && !kvm_pit_reinject) {
         if (kvm_reinject_control(kvm_context, 0)) {
             fprintf(stderr, "failure to disable in-kernel PIT reinjection\n");
diff --git a/qemu-kvm.h b/qemu-kvm.h
index a4f1816..0814883 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -64,8 +64,7 @@ typedef struct kvm_context *kvm_context_t;
 
 #include "kvm.h"
 
-int kvm_arch_create(kvm_context_t kvm, unsigned long phys_mem_bytes,
-                    void **vm_mem);
+int kvm_arch_create(kvm_context_t kvm);
 
 int kvm_arch_run(CPUState *env);
 
@@ -449,8 +448,6 @@ void on_vcpu(CPUState *env, void (*func)(void *data), void *data);
 void kvm_inject_interrupt(CPUState *env, int mask);
 void kvm_update_interrupt_request(CPUState *env);
 
-int kvm_arch_qemu_create_context(void);
-
 int kvm_arch_has_work(CPUState *env);
 void kvm_arch_process_irqchip_events(CPUState *env);
 int kvm_arch_try_push_interrupts(void *opaque);
-- 
1.7.1

[PATCH 16/19] qemu-kvm: Push PIT reinjection control into x86 code

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This belongs where the in-kernel PIT is created.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |    8 ++++++++
 qemu-kvm.c     |    7 -------
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 6b816bc..c0061a1 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -111,6 +111,14 @@ static int kvm_create_pit(kvm_context_t kvm)
             fprintf(stderr, "Create kernel PIC irqchip failed\n");
             return r;
         }
+        if (!kvm_pit_reinject) {
+            r = kvm_reinject_control(kvm_context, 0);
+            if (r < 0) {
+                fprintf(stderr,
+                        "failure to disable in-kernel PIT reinjection\n");
+                return r;
+            }
+        }
     }
 #endif
     return 0;
diff --git a/qemu-kvm.c b/qemu-kvm.c
index ecee3dc..bd595b1 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -1405,13 +1405,6 @@ static int kvm_create_context(void)
 
     kvm_create_irqchip(kvm_context);
 
-    if (kvm_pit && !kvm_pit_reinject) {
-        if (kvm_reinject_control(kvm_context, 0)) {
-            fprintf(stderr, "failure to disable in-kernel PIT reinjection\n");
-            return -1;
-        }
-    }
-
     /* There was a nasty bug in < kvm-80 that prevents memory slots from being
      * destroyed properly.  Since we rely on this capability, refuse to work
      * with any kernel without this capability. */
-- 
1.7.1

[PATCH 17/19] qemu-kvm: Replace kvm_show_regs/code with cpu_dump_state

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

The latter provides even more information.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |   95 --------------------------------------------------------
 qemu-kvm.c     |    7 ++--
 qemu-kvm.h     |   17 ----------
 3 files changed, 3 insertions(+), 116 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index c0061a1..df4f0b4 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -319,101 +319,6 @@ int kvm_has_pit_state2(kvm_context_t kvm)
     return r;
 }
 
-void kvm_show_code(CPUState *env)
-{
-#define SHOW_CODE_LEN 50
-    struct kvm_regs regs;
-    struct kvm_sregs sregs;
-    int r, n;
-    int back_offset;
-    unsigned char code;
-    char code_str[SHOW_CODE_LEN * 3 + 1];
-    unsigned long rip;
-
-    r = kvm_vcpu_ioctl(env, KVM_GET_SREGS, &sregs);
-    if (r < 0 ) {
-        perror("KVM_GET_SREGS");
-        return;
-    }
-    r = kvm_vcpu_ioctl(env, KVM_GET_REGS, &regs);
-    if (r < 0) {
-        perror("KVM_GET_REGS");
-        return;
-    }
-    rip = sregs.cs.base + regs.rip;
-    back_offset = regs.rip;
-    if (back_offset > 20) {
-        back_offset = 20;
-    }
-    *code_str = 0;
-    for (n = -back_offset; n < SHOW_CODE_LEN-back_offset; ++n) {
-        if (n == 0) {
-            strcat(code_str, " -->");
-        }
-        cpu_physical_memory_rw(rip + n, &code, 1, 1);
-        sprintf(code_str + strlen(code_str), " %02x", code);
-    }
-    fprintf(stderr, "code:%s\n", code_str);
-}
-
-static void print_seg(FILE *file, const char *name, struct kvm_segment *seg)
-{
-    fprintf(stderr,
-            "%s %04x (%08llx/%08x p %d dpl %d db %d s %d type %x l %d"
-            " g %d avl %d)\n",
-            name, seg->selector, seg->base, seg->limit, seg->present,
-            seg->dpl, seg->db, seg->s, seg->type, seg->l, seg->g,
-            seg->avl);
-}
-
-static void print_dt(FILE *file, const char *name, struct kvm_dtable *dt)
-{
-    fprintf(stderr, "%s %llx/%x\n", name, dt->base, dt->limit);
-}
-
-void kvm_show_regs(CPUState *env)
-{
-    struct kvm_regs regs;
-    struct kvm_sregs sregs;
-    int r;
-
-    r = kvm_vcpu_ioctl(env, KVM_GET_REGS, &regs);
-    if (r < 0) {
-        perror("KVM_GET_REGS");
-        return;
-    }
-    fprintf(stderr,
-            "rax %016llx rbx %016llx rcx %016llx rdx %016llx\n"
-            "rsi %016llx rdi %016llx rsp %016llx rbp %016llx\n"
-            "r8  %016llx r9  %016llx r10 %016llx r11 %016llx\n"
-            "r12 %016llx r13 %016llx r14 %016llx r15 %016llx\n"
-            "rip %016llx rflags %08llx\n",
-            regs.rax, regs.rbx, regs.rcx, regs.rdx,
-            regs.rsi, regs.rdi, regs.rsp, regs.rbp,
-            regs.r8,  regs.r9,  regs.r10, regs.r11,
-            regs.r12, regs.r13, regs.r14, regs.r15,
-            regs.rip, regs.rflags);
-    r = kvm_vcpu_ioctl(env, KVM_GET_SREGS, &sregs);
-    if (r < 0) {
-        perror("KVM_GET_SREGS");
-        return;
-    }
-    print_seg(stderr, "cs", &sregs.cs);
-    print_seg(stderr, "ds", &sregs.ds);
-    print_seg(stderr, "es", &sregs.es);
-    print_seg(stderr, "ss", &sregs.ss);
-    print_seg(stderr, "fs", &sregs.fs);
-    print_seg(stderr, "gs", &sregs.gs);
-    print_seg(stderr, "tr", &sregs.tr);
-    print_seg(stderr, "ldt", &sregs.ldt);
-    print_dt(stderr, "gdt", &sregs.gdt);
-    print_dt(stderr, "idt", &sregs.idt);
-    fprintf(stderr, "cr0 %llx cr2 %llx cr3 %llx cr4 %llx cr8 %llx"
-            " efer %llx\n",
-            sregs.cr0, sregs.cr2, sregs.cr3, sregs.cr4, sregs.cr8,
-            sregs.efer);
-}
-
 static void kvm_set_cr8(CPUState *env, uint64_t cr8)
 {
     env->kvm_run->cr8 = cr8;
diff --git a/qemu-kvm.c b/qemu-kvm.c
index bd595b1..55de971 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -425,8 +425,7 @@ int kvm_run(CPUState *env)
         case KVM_EXIT_EXCEPTION:
             fprintf(stderr, "exception %d (%x)\n", run->ex.exception,
                     run->ex.error_code);
-            kvm_show_regs(env);
-            kvm_show_code(env);
+            cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
             abort();
             break;
         case KVM_EXIT_IO:
@@ -463,7 +462,7 @@ int kvm_run(CPUState *env)
             r = kvm_arch_run(env);
             if (r < 0) {
                 fprintf(stderr, "unhandled vm exit: 0x%x\n", run->exit_reason);
-                kvm_show_regs(env);
+                cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
                 abort();
             }
             if (r > 0) {
@@ -988,7 +987,7 @@ int kvm_cpu_exec(CPUState *env)
     r = kvm_run(env);
     if (r < 0) {
         printf("kvm_run returned %d\n", r);
-        kvm_show_regs(env);
+        cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
         vm_stop(VMSTOP_PANIC);
     }
 
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 0814883..752aebd 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -68,9 +68,6 @@ int kvm_arch_create(kvm_context_t kvm);
 
 int kvm_arch_run(CPUState *env);
 
-
-void kvm_show_code(CPUState *env);
-
 int handle_halt(CPUState *env);
 
 int handle_shutdown(kvm_context_t kvm, CPUState *env);
@@ -139,20 +136,6 @@ int kvm_set_shadow_pages(kvm_context_t kvm, unsigned int nrshadow_pages);
 
 #endif
 
-/*!
- * \brief Dump VCPU registers
- *
- * This dumps some of the information that KVM has about a virtual CPU, namely:
- * - GP Registers
- *
- * A much more verbose version of this is available as kvm_dump_vcpu()
- *
- * \param kvm Pointer to the current kvm_context
- * \param vcpu Which virtual CPU should get dumped
- * \return 0 on success
- */
-void kvm_show_regs(CPUState *env);
-
 int kvm_set_irq_level(kvm_context_t kvm, int irq, int level, int *status);
 
 #ifdef KVM_CAP_IRQCHIP
-- 
1.7.1

[PATCH 18/19] qemu-kvm: Fold kvm_init_coalesced_mmio into kvm_create_context

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Aligns the initialization to upstream.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c |    5 -----
 qemu-kvm.c     |   20 ++++++--------------
 qemu-kvm.h     |    9 ---------
 3 files changed, 6 insertions(+), 28 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index df4f0b4..ece5a8e 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -152,11 +152,6 @@ int kvm_arch_create(kvm_context_t kvm)
         return r;
     }
 
-    r = kvm_init_coalesced_mmio(kvm);
-    if (r < 0) {
-        return r;
-    }
-
     uname(&utsname);
     lm_capable_kernel = strcmp(utsname.machine, "x86_64") == 0;
 
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 55de971..3001392 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -495,20 +495,6 @@ int kvm_inject_nmi(CPUState *env)
 #endif
 }
 
-int kvm_init_coalesced_mmio(kvm_context_t kvm)
-{
-    int r = 0;
-    kvm_state->coalesced_mmio = 0;
-#ifdef KVM_CAP_COALESCED_MMIO
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_COALESCED_MMIO);
-    if (r > 0) {
-        kvm_state->coalesced_mmio = r;
-        return 0;
-    }
-#endif
-    return r;
-}
-
 #ifdef KVM_CAP_DEVICE_ASSIGNMENT
 int kvm_assign_pci_device(kvm_context_t kvm,
                           struct kvm_assigned_pci_dev *assigned_dev)
@@ -1419,6 +1405,12 @@ static int kvm_create_context(void)
         return r;
     }
 
+    kvm_state->coalesced_mmio = 0;
+#ifdef KVM_CAP_COALESCED_MMIO
+    kvm_state->coalesced_mmio =
+        kvm_check_extension(kvm_state, KVM_CAP_COALESCED_MMIO);
+#endif
+
     kvm_state->vcpu_events = 0;
 #ifdef KVM_CAP_VCPU_EVENTS
     kvm_state->vcpu_events = kvm_check_extension(kvm_state, KVM_CAP_VCPU_EVENTS);
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 752aebd..1644582 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -200,15 +200,6 @@ int kvm_inject_nmi(CPUState *env);
 
 #endif
 
-/*!
- * \brief Initialize coalesced MMIO
- *
- * Check for coalesced MMIO capability and store in context
- *
- * \param kvm Pointer to the current kvm_context
- */
-int kvm_init_coalesced_mmio(kvm_context_t kvm);
-
 #ifdef KVM_CAP_PIT
 
 #if defined(__i386__) || defined(__x86_64__)
-- 
1.7.1

[PATCH 19/19] qemu-kvm: x86: Use kvm_arch_init

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Fold the qemu-kvm specific bits of kvm_arch_create into upstream's
kvm_arch_init and call that service directly. This nicely obsolete quite
a few more functions.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 qemu-kvm-x86.c    |  145 -----------------------------------------------------
 qemu-kvm.c        |    2 +-
 qemu-kvm.h        |   13 +----
 target-i386/kvm.c |   23 +++++++--
 4 files changed, 20 insertions(+), 163 deletions(-)

diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index ece5a8e..6c3a0a9 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -23,83 +23,6 @@
 #include "kvm.h"
 #include "hw/apic.h"
 
-#define MSR_IA32_TSC            0x10
-
-extern unsigned int kvm_shadow_memory;
-
-static int kvm_set_tss_addr(kvm_context_t kvm, unsigned long addr)
-{
-    int r;
-
-    r = kvm_vm_ioctl(kvm_state, KVM_SET_TSS_ADDR, addr);
-    if (r < 0) {
-        fprintf(stderr, "kvm_set_tss_addr: %m\n");
-        return r;
-    }
-    return 0;
-}
-
-static int kvm_init_tss(kvm_context_t kvm)
-{
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_SET_TSS_ADDR);
-    if (r > 0) {
-        /*
-         * this address is 3 pages before the bios, and the bios should present
-         * as unavaible memory
-         */
-        r = kvm_set_tss_addr(kvm, 0xfeffd000);
-        if (r < 0) {
-            fprintf(stderr, "kvm_init_tss: unable to set tss addr\n");
-            return r;
-        }
-    } else {
-        fprintf(stderr, "kvm does not support KVM_CAP_SET_TSS_ADDR\n");
-    }
-    return 0;
-}
-
-static int kvm_set_identity_map_addr(kvm_context_t kvm, uint64_t addr)
-{
-#ifdef KVM_CAP_SET_IDENTITY_MAP_ADDR
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_SET_IDENTITY_MAP_ADDR);
-    if (r > 0) {
-        r = kvm_vm_ioctl(kvm_state, KVM_SET_IDENTITY_MAP_ADDR, &addr);
-        if (r == -1) {
-            fprintf(stderr, "kvm_set_identity_map_addr: %m\n");
-            return -errno;
-        }
-        return 0;
-    }
-#endif
-    return -ENOSYS;
-}
-
-static int kvm_init_identity_map_page(kvm_context_t kvm)
-{
-#ifdef KVM_CAP_SET_IDENTITY_MAP_ADDR
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_SET_IDENTITY_MAP_ADDR);
-    if (r > 0) {
-        /*
-         * this address is 4 pages before the bios, and the bios should present
-         * as unavaible memory
-         */
-        r = kvm_set_identity_map_addr(kvm, 0xfeffc000);
-        if (r < 0) {
-            fprintf(stderr, "kvm_init_identity_map_page: "
-                    "unable to set identity mapping addr\n");
-            return r;
-        }
-    }
-#endif
-    return 0;
-}
-
 static int kvm_create_pit(kvm_context_t kvm)
 {
 #ifdef KVM_CAP_PIT
@@ -124,55 +47,6 @@ static int kvm_create_pit(kvm_context_t kvm)
     return 0;
 }
 
-int kvm_arch_create(kvm_context_t kvm)
-{
-    struct utsname utsname;
-    int r = 0;
-
-    r = kvm_init_tss(kvm);
-    if (r < 0) {
-        return r;
-    }
-
-    r = kvm_init_identity_map_page(kvm);
-    if (r < 0) {
-        return r;
-    }
-
-    /*
-     * Tell fw_cfg to notify the BIOS to reserve the range.
-     */
-    if (e820_add_entry(0xfeffc000, 0x4000, E820_RESERVED) < 0) {
-        perror("e820_add_entry() table is full");
-        exit(1);
-    }
-
-    r = kvm_create_pit(kvm);
-    if (r < 0) {
-        return r;
-    }
-
-    uname(&utsname);
-    lm_capable_kernel = strcmp(utsname.machine, "x86_64") == 0;
-
-    if (kvm_shadow_memory) {
-        kvm_set_shadow_pages(kvm_context, kvm_shadow_memory);
-    }
-
-    /* initialize has_msr_star/has_msr_hsave_pa */
-    r = kvm_get_supported_msrs(kvm_state);
-    if (r < 0) {
-        return r;
-    }
-
-    r = kvm_set_boot_cpu_id(0);
-    if (r < 0 && r != -ENOSYS) {
-        return r;
-    }
-
-    return 0;
-}
-
 #ifdef KVM_EXIT_TPR_ACCESS
 
 static int kvm_handle_tpr_access(CPUState *env)
@@ -319,25 +193,6 @@ static void kvm_set_cr8(CPUState *env, uint64_t cr8)
     env->kvm_run->cr8 = cr8;
 }
 
-int kvm_set_shadow_pages(kvm_context_t kvm, unsigned int nrshadow_pages)
-{
-#ifdef KVM_CAP_MMU_SHADOW_CACHE_CONTROL
-    int r;
-
-    r = kvm_ioctl(kvm_state, KVM_CHECK_EXTENSION,
-                  KVM_CAP_MMU_SHADOW_CACHE_CONTROL);
-    if (r > 0) {
-        r = kvm_vm_ioctl(kvm_state, KVM_SET_NR_MMU_PAGES, nrshadow_pages);
-        if (r < 0) {
-            fprintf(stderr, "kvm_set_shadow_pages: %m\n");
-            return r;
-        }
-        return 0;
-    }
-#endif
-    return -1;
-}
-
 #ifdef KVM_CAP_VAPIC
 static int kvm_enable_tpr_access_reporting(CPUState *env)
 {
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 3001392..cf02ceb 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -1382,7 +1382,7 @@ static int kvm_create_context(void)
         return -1;
     }
 
-    r = kvm_arch_create(kvm_context);
+    r = kvm_arch_init(kvm_state);
     if (r < 0) {
         kvm_finalize(kvm_state);
         return r;
diff --git a/qemu-kvm.h b/qemu-kvm.h
index 1644582..8b3ce9c 100644
--- a/qemu-kvm.h
+++ b/qemu-kvm.h
@@ -64,8 +64,6 @@ typedef struct kvm_context *kvm_context_t;
 
 #include "kvm.h"
 
-int kvm_arch_create(kvm_context_t kvm);
-
 int kvm_arch_run(CPUState *env);
 
 int handle_halt(CPUState *env);
@@ -124,16 +122,6 @@ int kvm_is_ready_for_interrupt_injection(CPUState *env);
  * \return 0 on success
  */
 int kvm_inject_irq(CPUState *env, unsigned irq);
-
-
-/*!
- * \brief Setting the number of shadow pages to be allocated to the vm
- *
- * \param kvm pointer to kvm_context
- * \param nrshadow_pages number of pages to be allocated
- */
-int kvm_set_shadow_pages(kvm_context_t kvm, unsigned int nrshadow_pages);
-
 #endif
 
 int kvm_set_irq_level(kvm_context_t kvm, int irq, int level, int *status);
@@ -449,6 +437,7 @@ extern int kvm_pit;
 extern int kvm_pit_reinject;
 extern int kvm_nested;
 extern kvm_context_t kvm_context;
+extern unsigned int kvm_shadow_memory;
 
 struct ioperm_data {
     unsigned long start_port;
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index bbc324b..a8b5c6d 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -197,7 +197,6 @@ typedef struct HWPoisonPage {
 static QLIST_HEAD(, HWPoisonPage) hwpoison_page_list =
     QLIST_HEAD_INITIALIZER(hwpoison_page_list);
 
-#ifdef OBSOLETE_KVM_IMPL
 static void kvm_unpoison_all(void *param)
 {
     HWPoisonPage *page, *next_page;
@@ -208,7 +207,6 @@ static void kvm_unpoison_all(void *param)
         qemu_free(page);
     }
 }
-#endif
 
 #ifdef KVM_CAP_MCE
 static void kvm_hwpoison_page_add(ram_addr_t ram_addr)
@@ -603,7 +601,7 @@ static int kvm_get_supported_msrs(KVMState *s)
     return ret;
 }
 
-#ifdef OBSOLETE_KVM_IMPL
+static int kvm_create_pit(kvm_context_t kvm);
 
 int kvm_arch_init(KVMState *s)
 {
@@ -655,11 +653,26 @@ int kvm_arch_init(KVMState *s)
     }
     qemu_register_reset(kvm_unpoison_all, NULL);
 
+    ret = kvm_create_pit(&s->kvm_context);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if (kvm_shadow_memory) {
+        ret = kvm_vm_ioctl(s, KVM_SET_NR_MMU_PAGES, kvm_shadow_memory);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
+    ret = kvm_set_boot_cpu_id(0);
+    if (ret < 0 && ret != -ENOSYS) {
+        return ret;
+    }
+
     return 0;
 }
 
-#endif
-
 static void set_v8086_seg(struct kvm_segment *lhs, const SegmentCache *rhs)
 {
     lhs->selector = rhs->selector;
-- 
1.7.1

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/04/2011 10:43 PM, Jan Kiszka wrote:
> From: Jan Kiszka<jan.kiszka@siemens.com>
>
> Paravirtual MSRs are properly cleared on reset now, and blindly clearing
> the rest is questionable anyway (better address those one by one,
> re-initializing their backing CPU state fields).
>

This can introduce a regression when new paravirtual MSRs are added.  So 
we either need to port this, or query the reset state from the kernel 
immediately after creating the vcpu and saving it.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

On 2011-05-05 10:08, Avi Kivity wrote:
> On 05/04/2011 10:43 PM, Jan Kiszka wrote:
>> From: Jan Kiszka<jan.kiszka@siemens.com>
>>
>> Paravirtual MSRs are properly cleared on reset now, and blindly clearing
>> the rest is questionable anyway (better address those one by one,
>> re-initializing their backing CPU state fields).
>>
> 
> This can introduce a regression when new paravirtual MSRs are added.

You mean MSRs already included or future ones?

>  So
> we either need to port this, or query the reset state from the kernel
> immediately after creating the vcpu and saving it.

Can't completely follow what you mean.

My general point remains: Every MSR requires individual care, not blind
overwriting like qemu-kvm does. So the person contributing a new MSR,
real or pv, has to tackle this aspect, and we need to review the code in
this regard.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/05/2011 11:11 AM, Jan Kiszka wrote:
> On 2011-05-05 10:08, Avi Kivity wrote:
> >  On 05/04/2011 10:43 PM, Jan Kiszka wrote:
> >>  From: Jan Kiszka<jan.kiszka@siemens.com>
> >>
> >>  Paravirtual MSRs are properly cleared on reset now, and blindly clearing
> >>  the rest is questionable anyway (better address those one by one,
> >>  re-initializing their backing CPU state fields).
> >>
> >
> >  This can introduce a regression when new paravirtual MSRs are added.
>
> You mean MSRs already included or future ones?

Future ones.

> >   So
> >  we either need to port this, or query the reset state from the kernel
> >  immediately after creating the vcpu and saving it.
>
> Can't completely follow what you mean.
>
> My general point remains: Every MSR requires individual care, not blind
> overwriting like qemu-kvm does. So the person contributing a new MSR,
> real or pv, has to tackle this aspect, and we need to review the code in
> this regard.

It's a trick to avoid needing individual care.

1. Call KVM_CREATE_VCPU.  This causes all MSRs to be initialized to 
their power-on reset values.
2. Issue KVM_GET_MSR_LIST, and then KVM_GET_MSRS to read all MSRs.  
Stash them all in safe places - the ones known to qemu but also the 
unknown ones.  Qemu may use its own values for the MSRs it knows about 
(for example if different cpu models have different power-on values)
3. On reset, issue KVM_SET_MSRS with the MSR values obtained in step 2.

The result is forward and backwards compatibility without lockstepping 
qemu and kvm.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 00/19] qemu-kvm: Cleanup and switch to upstream - Part I

[Avi Kivity]

On 05/04/2011 10:43 PM, Jan Kiszka wrote:
> All required bits for this cleanup of qemu-kvm are now upstream and
> merged back - it's time to start the show. There are now 65 patches in
> my queue, and I'm planning for at least 4 rounds.
>
> This first part primarily aims at using upstream kvm_arch_init. Further
> highlights are:
>   - switch to upstream CPU register get/put
>   - switch to upstream kvmclock
>   - sustainable breakage of IA64
>
> Please review/merge.
>

I didn't do a line-by-line review, just a general look around, and it 
all looks good except for the MSR behavioural change.

How did you test it?  I suggest testing at least vmport in addition to 
an autotest run (if you don't have a setup ready, we can do it, but it 
would be really helpful to test pre-merge).  I guess we can expect a few 
regressions but if we're too careful this mess will continue forever.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

On 2011-05-05 10:16, Avi Kivity wrote:
> On 05/05/2011 11:11 AM, Jan Kiszka wrote:
>> On 2011-05-05 10:08, Avi Kivity wrote:
>>>  On 05/04/2011 10:43 PM, Jan Kiszka wrote:
>>>>  From: Jan Kiszka<jan.kiszka@siemens.com>
>>>>
>>>>  Paravirtual MSRs are properly cleared on reset now, and blindly clearing
>>>>  the rest is questionable anyway (better address those one by one,
>>>>  re-initializing their backing CPU state fields).
>>>>
>>>
>>>  This can introduce a regression when new paravirtual MSRs are added.
>>
>> You mean MSRs already included or future ones?
> 
> Future ones.
> 
>>>   So
>>>  we either need to port this, or query the reset state from the kernel
>>>  immediately after creating the vcpu and saving it.
>>
>> Can't completely follow what you mean.
>>
>> My general point remains: Every MSR requires individual care, not blind
>> overwriting like qemu-kvm does. So the person contributing a new MSR,
>> real or pv, has to tackle this aspect, and we need to review the code in
>> this regard.
> 
> It's a trick to avoid needing individual care.
> 
> 1. Call KVM_CREATE_VCPU.  This causes all MSRs to be initialized to 
> their power-on reset values.

Almost all: Which ones are CPU specific like the APIC_BASE?

> 2. Issue KVM_GET_MSR_LIST, and then KVM_GET_MSRS to read all MSRs.  
> Stash them all in safe places - the ones known to qemu but also the 
> unknown ones.  Qemu may use its own values for the MSRs it knows about 
> (for example if different cpu models have different power-on values)
> 3. On reset, issue KVM_SET_MSRS with the MSR values obtained in step 2.
> 
> The result is forward and backwards compatibility without lockstepping 
> qemu and kvm.

OK, sounds good. I will work out a patch for uq/master.

Nevertheless, the qemu-kvm code is already unneeded today and can safely
be removed IMHO.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 00/19] qemu-kvm: Cleanup and switch to upstream - Part I

[Jan Kiszka]

On 2011-05-05 10:22, Avi Kivity wrote:
> On 05/04/2011 10:43 PM, Jan Kiszka wrote:
>> All required bits for this cleanup of qemu-kvm are now upstream and
>> merged back - it's time to start the show. There are now 65 patches in
>> my queue, and I'm planning for at least 4 rounds.
>>
>> This first part primarily aims at using upstream kvm_arch_init. Further
>> highlights are:
>>   - switch to upstream CPU register get/put
>>   - switch to upstream kvmclock
>>   - sustainable breakage of IA64
>>
>> Please review/merge.
>>
> 
> I didn't do a line-by-line review, just a general look around, and it
> all looks good except for the MSR behavioural change.
> 
> How did you test it?

Running various VMs, mostly Linux (including vmmouse support) but also
some XP guests. I'm using the whole series for internal purposes for a
few months now (frequently rebased). So I have a quite good feeling
about it, except for the unknown corner cases. :)

>  I suggest testing at least vmport in addition to
> an autotest run (if you don't have a setup ready, we can do it, but it
> would be really helpful to test pre-merge).  I guess we can expect a few
> regressions but if we're too careful this mess will continue forever.

I would be happy to off-load the full autotest run to $someone_else.
Will jump in again if it fails.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/05/2011 11:27 AM, Jan Kiszka wrote:
> >
> >  1. Call KVM_CREATE_VCPU.  This causes all MSRs to be initialized to
> >  their power-on reset values.
>
> Almost all: Which ones are CPU specific like the APIC_BASE?

Do you mean cpu specific as in smp or cpu specific as in cpu model?

If the former, we simply do the reset operation per-cpu.  It's the 
natural thing anyway.

> >  2. Issue KVM_GET_MSR_LIST, and then KVM_GET_MSRS to read all MSRs.
> >  Stash them all in safe places - the ones known to qemu but also the
> >  unknown ones.  Qemu may use its own values for the MSRs it knows about
> >  (for example if different cpu models have different power-on values)
> >  3. On reset, issue KVM_SET_MSRS with the MSR values obtained in step 2.
> >
> >  The result is forward and backwards compatibility without lockstepping
> >  qemu and kvm.
>
> OK, sounds good. I will work out a patch for uq/master.

Great, thanks.

> Nevertheless, the qemu-kvm code is already unneeded today and can safely
> be removed IMHO.

I don't follow?  Won't it cause a regression?

Of course, if we get a patch soon no one will ever see the regression so 
we can apply the series.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

On 2011-05-05 10:33, Avi Kivity wrote:
> On 05/05/2011 11:27 AM, Jan Kiszka wrote:
>>>
>>>  1. Call KVM_CREATE_VCPU.  This causes all MSRs to be initialized to
>>>  their power-on reset values.
>>
>> Almost all: Which ones are CPU specific like the APIC_BASE?
> 
> Do you mean cpu specific as in smp or cpu specific as in cpu model?

Yep.

> 
> If the former, we simply do the reset operation per-cpu.  It's the 
> natural thing anyway.

Quite wasteful /wrt to memory given that the majority will be identical.

> 
>>>  2. Issue KVM_GET_MSR_LIST, and then KVM_GET_MSRS to read all MSRs.
>>>  Stash them all in safe places - the ones known to qemu but also the
>>>  unknown ones.  Qemu may use its own values for the MSRs it knows about
>>>  (for example if different cpu models have different power-on values)
>>>  3. On reset, issue KVM_SET_MSRS with the MSR values obtained in step 2.
>>>
>>>  The result is forward and backwards compatibility without lockstepping
>>>  qemu and kvm.
>>
>> OK, sounds good. I will work out a patch for uq/master.
> 
> Great, thanks.
> 
>> Nevertheless, the qemu-kvm code is already unneeded today and can safely
>> be removed IMHO.
> 
> I don't follow?  Won't it cause a regression?

Not at all. We use the "individual care" pattern upstream now,
specifically for those MSRs (kvmclock) for which the qemu-kvm code was
introduced.

> 
> Of course, if we get a patch soon no one will ever see the regression so 
> we can apply the series.

I will still require the usual testing and merging round via upstream
and back. Not sure when I'll be able to work on it, probably not the
next days.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/05/2011 11:44 AM, Jan Kiszka wrote:
> On 2011-05-05 10:33, Avi Kivity wrote:
> >  On 05/05/2011 11:27 AM, Jan Kiszka wrote:
> >>>
> >>>   1. Call KVM_CREATE_VCPU.  This causes all MSRs to be initialized to
> >>>   their power-on reset values.
> >>
> >>  Almost all: Which ones are CPU specific like the APIC_BASE?
> >
> >  Do you mean cpu specific as in smp or cpu specific as in cpu model?
>
> Yep.

Doh.

> >
> >  If the former, we simply do the reset operation per-cpu.  It's the
> >  natural thing anyway.
>
> Quite wasteful /wrt to memory given that the majority will be identical.

We're talking a few hundred bytes per cpu.  If you want to save memory, 
look at the PhysPageDesc array, it takes up 0.4% of guest memory, so 4MB 
for a 1GB guest.

> >>  Nevertheless, the qemu-kvm code is already unneeded today and can safely
> >>  be removed IMHO.
> >
> >  I don't follow?  Won't it cause a regression?
>
> Not at all. We use the "individual care" pattern upstream now,
> specifically for those MSRs (kvmclock) for which the qemu-kvm code was
> introduced.

I mean a future regression with current+patch qemu and a new kernel.

> >
> >  Of course, if we get a patch soon no one will ever see the regression so
> >  we can apply the series.
>
> I will still require the usual testing and merging round via upstream
> and back. Not sure when I'll be able to work on it, probably not the
> next days.

If you can do it within a couple of weeks or so that should be fine.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

On 2011-05-05 10:53, Avi Kivity wrote:
> On 05/05/2011 11:44 AM, Jan Kiszka wrote:
>> On 2011-05-05 10:33, Avi Kivity wrote:
>> >  On 05/05/2011 11:27 AM, Jan Kiszka wrote:
>> >>>
>> >>>   1. Call KVM_CREATE_VCPU.  This causes all MSRs to be initialized to
>> >>>   their power-on reset values.
>> >>
>> >>  Almost all: Which ones are CPU specific like the APIC_BASE?
>> >
>> >  Do you mean cpu specific as in smp or cpu specific as in cpu model?
>>
>> Yep.
> 
> Doh.
> 
>> >
>> >  If the former, we simply do the reset operation per-cpu.  It's the
>> >  natural thing anyway.
>>
>> Quite wasteful /wrt to memory given that the majority will be identical.
> 
> We're talking a few hundred bytes per cpu.  If you want to save memory,
> look at the PhysPageDesc array, it takes up 0.4% of guest memory, so 4MB
> for a 1GB guest.

I know (that's fixable, BTW). But that should not excuse needless memory
wasting elsewhere.

> 
>> >>  Nevertheless, the qemu-kvm code is already unneeded today and can
>> safely
>> >>  be removed IMHO.
>> >
>> >  I don't follow?  Won't it cause a regression?
>>
>> Not at all. We use the "individual care" pattern upstream now,
>> specifically for those MSRs (kvmclock) for which the qemu-kvm code was
>> introduced.
> 
> I mean a future regression with current+patch qemu and a new kernel.

For sane scenarios, such a combination should never expose new (ie.
unknown from qemu's POV) MSRs to the guest. Thus not clearing them
cannot cause any harm.

BTW, you also do not know if 0 will be the right reset value for these
to-be-invented MSRs. That could cause regression as well.

> 
>> >
>> >  Of course, if we get a patch soon no one will ever see the
>> regression so
>> >  we can apply the series.
>>
>> I will still require the usual testing and merging round via upstream
>> and back. Not sure when I'll be able to work on it, probably not the
>> next days.
> 
> If you can do it within a couple of weeks or so that should be fine.
> 

We'll see, but I still do not share your concern regarding future
regressions when removing the fragile reset code.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/05/2011 12:32 PM, Jan Kiszka wrote:
> >>  >
> >>  >   If the former, we simply do the reset operation per-cpu.  It's the
> >>  >   natural thing anyway.
> >>
> >>  Quite wasteful /wrt to memory given that the majority will be identical.
> >
> >  We're talking a few hundred bytes per cpu.  If you want to save memory,
> >  look at the PhysPageDesc array, it takes up 0.4% of guest memory, so 4MB
> >  for a 1GB guest.
>
> I know (that's fixable, BTW). But that should not excuse needless memory
> wasting elsewhere.

IMO a few hundred bytes is worth the correctness here.

> >
> >>  >>   Nevertheless, the qemu-kvm code is already unneeded today and can
> >>  safely
> >>  >>   be removed IMHO.
> >>  >
> >>  >   I don't follow?  Won't it cause a regression?
> >>
> >>  Not at all. We use the "individual care" pattern upstream now,
> >>  specifically for those MSRs (kvmclock) for which the qemu-kvm code was
> >>  introduced.
> >
> >  I mean a future regression with current+patch qemu and a new kernel.
>
> For sane scenarios, such a combination should never expose new (ie.
> unknown from qemu's POV) MSRs to the guest. Thus not clearing them
> cannot cause any harm.

The problem is with hardware MSRs (PV MSRs are protected by cpuid, and 
always disable themselves when zeroed).

> BTW, you also do not know if 0 will be the right reset value for these
> to-be-invented MSRs. That could cause regression as well.

What I suggested wasn't zeroing them, but writing the value we read just 
after vcpu creation.

We had a regression when we started supporting PAT.  Zeroing it causes 
the cache to be disabled, making everything ridiculously slow.  We now 
special case it; my proposed solution would have taken care of it.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

On 2011-05-05 12:22, Avi Kivity wrote:
> On 05/05/2011 12:32 PM, Jan Kiszka wrote:
>> >>  >
>> >>  >   If the former, we simply do the reset operation per-cpu.  It's
>> the
>> >>  >   natural thing anyway.
>> >>
>> >>  Quite wasteful /wrt to memory given that the majority will be
>> identical.
>> >
>> >  We're talking a few hundred bytes per cpu.  If you want to save
>> memory,
>> >  look at the PhysPageDesc array, it takes up 0.4% of guest memory,
>> so 4MB
>> >  for a 1GB guest.
>>
>> I know (that's fixable, BTW). But that should not excuse needless memory
>> wasting elsewhere.
> 
> IMO a few hundred bytes is worth the correctness here.
> 
>> >
>> >>  >>   Nevertheless, the qemu-kvm code is already unneeded today and
>> can
>> >>  safely
>> >>  >>   be removed IMHO.
>> >>  >
>> >>  >   I don't follow?  Won't it cause a regression?
>> >>
>> >>  Not at all. We use the "individual care" pattern upstream now,
>> >>  specifically for those MSRs (kvmclock) for which the qemu-kvm code
>> was
>> >>  introduced.
>> >
>> >  I mean a future regression with current+patch qemu and a new kernel.
>>
>> For sane scenarios, such a combination should never expose new (ie.
>> unknown from qemu's POV) MSRs to the guest. Thus not clearing them
>> cannot cause any harm.
> 
> The problem is with hardware MSRs (PV MSRs are protected by cpuid, and
> always disable themselves when zeroed).

Well, this doesn't change the problem of the existing code.

> 
>> BTW, you also do not know if 0 will be the right reset value for these
>> to-be-invented MSRs. That could cause regression as well.
> 
> What I suggested wasn't zeroing them, but writing the value we read just
> after vcpu creation.
> 
> We had a regression when we started supporting PAT.  Zeroing it causes
> the cache to be disabled, making everything ridiculously slow.  We now
> special case it; my proposed solution would have taken care of it.

I'm talking about the current code, not the proper way to do it in the
future. PAT demonstrates why regressions can happen as long as we zero
and it's better to stop this now even without the new code in place.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Gleb Natapov]

On Thu, May 05, 2011 at 11:32:26AM +0200, Jan Kiszka wrote:
> >> >
> >> >  Of course, if we get a patch soon no one will ever see the
> >> regression so
> >> >  we can apply the series.
> >>
> >> I will still require the usual testing and merging round via upstream
> >> and back. Not sure when I'll be able to work on it, probably not the
> >> next days.
> > 
> > If you can do it within a couple of weeks or so that should be fine.
> > 
> 
> We'll see, but I still do not share your concern regarding future
> regressions when removing the fragile reset code.
> 
Why do we rely on userspace to properly reset kernel component anyway?
We should introduce cpu/lapic/ioapic/pit/pic resets ASAP.

--
			Gleb.

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/05/2011 01:36 PM, Jan Kiszka wrote:
> >
> >  The problem is with hardware MSRs (PV MSRs are protected by cpuid, and
> >  always disable themselves when zeroed).
>
> Well, this doesn't change the problem of the existing code.

Right.

> >
> >  What I suggested wasn't zeroing them, but writing the value we read just
> >  after vcpu creation.
> >
> >  We had a regression when we started supporting PAT.  Zeroing it causes
> >  the cache to be disabled, making everything ridiculously slow.  We now
> >  special case it; my proposed solution would have taken care of it.
>
> I'm talking about the current code, not the proper way to do it in the
> future. PAT demonstrates why regressions can happen as long as we zero
> and it's better to stop this now even without the new code in place.

I'm not sure, but if we do adopt the new reset mechanism, it doesn't matter.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Avi Kivity]

On 05/05/2011 02:22 PM, Gleb Natapov wrote:
> >
> >  We'll see, but I still do not share your concern regarding future
> >  regressions when removing the fragile reset code.
> >
> Why do we rely on userspace to properly reset kernel component anyway?
> We should introduce cpu/lapic/ioapic/pit/pic resets ASAP.

We should, but we'll always have to deal with kernels that don't have 
reset ioctls.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Gleb Natapov]

On Thu, May 05, 2011 at 02:58:27PM +0300, Avi Kivity wrote:
> On 05/05/2011 02:22 PM, Gleb Natapov wrote:
> >>
> >>  We'll see, but I still do not share your concern regarding future
> >>  regressions when removing the fragile reset code.
> >>
> >Why do we rely on userspace to properly reset kernel component anyway?
> >We should introduce cpu/lapic/ioapic/pit/pic resets ASAP.
> 
> We should, but we'll always have to deal with kernels that don't
> have reset ioctls.
> 
s/always/for quite a while/. Unfortunately yes. Unless we put qemu in the kernel
tree and will release them in lock steps of course :)

--
			Gleb.

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Jan Kiszka]

On 2011-05-05 14:23, Gleb Natapov wrote:
> On Thu, May 05, 2011 at 02:58:27PM +0300, Avi Kivity wrote:
>> On 05/05/2011 02:22 PM, Gleb Natapov wrote:
>>>>
>>>>  We'll see, but I still do not share your concern regarding future
>>>>  regressions when removing the fragile reset code.
>>>>
>>> Why do we rely on userspace to properly reset kernel component anyway?
>>> We should introduce cpu/lapic/ioapic/pit/pic resets ASAP.
>>
>> We should, but we'll always have to deal with kernels that don't
>> have reset ioctls.
>>
> s/always/for quite a while/. Unfortunately yes. Unless we put qemu in the kernel
> tree and will release them in lock steps of course :)

Seriously, I do not see much added-value of a reset service. The pattern
we will use for MSRs could just be applied to other in-kernel resources
as well - unless they are already architecturally defined in a way that
leaves no questions regarding the proper future reset state. Except for
the CPU, all other in-kernel devices are not extensible in their current
form.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Marcelo Tosatti]

On Thu, May 05, 2011 at 02:22:57PM +0300, Gleb Natapov wrote:
> On Thu, May 05, 2011 at 11:32:26AM +0200, Jan Kiszka wrote:
> > >> >
> > >> >  Of course, if we get a patch soon no one will ever see the
> > >> regression so
> > >> >  we can apply the series.
> > >>
> > >> I will still require the usual testing and merging round via upstream
> > >> and back. Not sure when I'll be able to work on it, probably not the
> > >> next days.
> > > 
> > > If you can do it within a couple of weeks or so that should be fine.
> > > 
> > 
> > We'll see, but I still do not share your concern regarding future
> > regressions when removing the fragile reset code.
> > 
> Why do we rely on userspace to properly reset kernel component anyway?

In general its required that userspace properly sets (some of) the
parameters of emulated hardware, including on reset.

> We should introduce cpu/lapic/ioapic/pit/pic resets ASAP.

Thats done through userspace via ioctls, don't get your point?

Re: [PATCH 04/19] qemu-kvm: x86: Drop MSR reset

[Gleb Natapov]

On Thu, May 05, 2011 at 10:33:01AM -0300, Marcelo Tosatti wrote:
> On Thu, May 05, 2011 at 02:22:57PM +0300, Gleb Natapov wrote:
> > On Thu, May 05, 2011 at 11:32:26AM +0200, Jan Kiszka wrote:
> > > >> >
> > > >> >  Of course, if we get a patch soon no one will ever see the
> > > >> regression so
> > > >> >  we can apply the series.
> > > >>
> > > >> I will still require the usual testing and merging round via upstream
> > > >> and back. Not sure when I'll be able to work on it, probably not the
> > > >> next days.
> > > > 
> > > > If you can do it within a couple of weeks or so that should be fine.
> > > > 
> > > 
> > > We'll see, but I still do not share your concern regarding future
> > > regressions when removing the fragile reset code.
> > > 
> > Why do we rely on userspace to properly reset kernel component anyway?
> 
> In general its required that userspace properly sets (some of) the
> parameters of emulated hardware, including on reset.
That's just how things work now. It doesn't have to be this way.

> 
> > We should introduce cpu/lapic/ioapic/pit/pic resets ASAP.
> 
> Thats done through userspace via ioctls, don't get your point?

Why userspace shouldn't know such low level details about in kernel
device? We had same devices emulated in qemu so it was easy to reuse
that for reset, but think about writing device model from the start.
With current interfaces you either need to have very low level knowledge
about in kernel device in userspace just to reset it properly, or you
need to save device state just after creation and reload the saved state
on reset (kind of what Avi proposed for MSR). Both solutions are not
optional IMO.

--
			Gleb.

Re: [PATCH 00/19] qemu-kvm: Cleanup and switch to upstream - Part I

[Marcelo Tosatti]

On Wed, May 04, 2011 at 09:43:34PM +0200, Jan Kiszka wrote:
> All required bits for this cleanup of qemu-kvm are now upstream and
> merged back - it's time to start the show. There are now 65 patches in
> my queue, and I'm planning for at least 4 rounds.
> 
> This first part primarily aims at using upstream kvm_arch_init. Further
> highlights are:
>  - switch to upstream CPU register get/put
>  - switch to upstream kvmclock
>  - sustainable breakage of IA64
> 
> Please review/merge.

Applied, thanks.