From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42226) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WILHW-0003Lu-QY for qemu-devel@nongnu.org; Tue, 25 Feb 2014 11:55:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WILHN-0002RF-Nq for qemu-devel@nongnu.org; Tue, 25 Feb 2014 11:54:58 -0500 Received: from mail-we0-f181.google.com ([74.125.82.181]:48197) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WILHN-0002R9-Eb for qemu-devel@nongnu.org; Tue, 25 Feb 2014 11:54:49 -0500 Received: by mail-we0-f181.google.com with SMTP id w61so651378wes.12 for ; Tue, 25 Feb 2014 08:54:48 -0800 (PST) From: Alvise Rigo Date: Tue, 25 Feb 2014 17:52:46 +0100 Message-Id: <1393347170-28502-1-git-send-email-a.rigo@virtualopensystems.com> Subject: [Qemu-devel] [RFC 0/4] target-arm: KVM to TCG migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: tech@virtualopensystems.com, Alvise Rigo QEMU at the moment doesn't allow the migration between KVM and TCG, in both directions. This limitation is due first of all to the different set of coprocessor registers supported by KVM and TCG, but also in the way the coprocessors values are copied from the incoming data to the local QEMU structures. This set of patches is focused on the KVM to TCG migration using the CortexA15 processor in both sides and tries to enhance the migration of cp registers in order to prevent QEMU from failing migration in some particular situations. To illustrate the changes proposed with this patch series, we will call "generator" a register that generates other registers in the TCG coprocessor hash table, such a register can have one or more wildcarded fields in its definition, or can have an CP_ARM_STATE_BOTH state (in this case the generator is the AARCH64 view of the register). With the term "child" a register generated by a generator will be indicated. It turns out that KVM tries to migrate some child registers (that in TCG are marked as CP_ARM_NO_MIGRATE) leading to a fault during the copy of cp registers. These patch series modify the coprocessor migration with the following steps: - during the copy of the incoming cp registers inside cpu_post_load a list is kept containing the registers that haven't matched any of the registers inside the cpreg_list. - only if the previous step has generated a not empty list, we try to handle the copy of these registers "by hand", verifying for every register in the list if it is: - already present in the hash table - flagged with CP_ARM_NO_MIGRATE In this case if the register has a generator which is migratable we raw write its value and then we remove it from the list, otherwise we simply remove it from the list (because still a not migratable register for TCG). When we raw_write the value of the register in the hash table we take care of marking the generator register as already "updated" (modifying the flag skip_cpreglist to true) because it has the same value of its children. This will prevent the method write_list_to_cpustate from overwriting the value of a child register with the default value of its parent: this is what happens when KVM migrates a child register and not its generator. Finally, it is also necessary to not update those registers with constant values, because KVM and TCG in some cases don't agree with their default values; this is done setting the skip_cpreglist flag to true for every ARM_CP_CONST registers. This is the case of constant registers like ID_PFR0 (the Jazelle bit for instance is set in TCG and not set in KVM) and AUXCR (KVM read its value directly form the processor while in TCG is not set); here a much cleaner solution is needed. All this "handling" is confined inside the method handle_cpreg_kvm2tcg_migration. This method will be called only if is detected that we are migrating from KVM to TCG (a new flag inside ARMCPU was added to address this purpose). If we are performing a normal migration, be it TCG<->TCG or KVM<->KVM, nothing will change and the usual copy of cp registers and aborting criteria will hold. The other way of migration is not yet covered by this patch set. KVM lacks of several coprocessor registers that in TCG are instead supported; solving this open issue will be the main challenge of the TCG to KVM migration. This set of patches has been tested by booting a guest running the Linux 3.13 kernel in a host using KVM. Two types of hosts have been used: ARM FastModels and the ARM Chromebook. Then, in the middle of the kernel boot, the guest was migrated to a x86 host running TCG. The machine model used is vexpress-a15 with cortex-a15 as CPU. Alvise Rigo (4): Fix issue affecting get_int32_le() in vmstate.c Added flag in ARMCPU to track last execution mode Add l2ctlr cp register to CPUARMState Relevant changes to enable KVM to TCG migration target-arm/cpu-qom.h | 3 + target-arm/cpu.c | 22 +++++-- target-arm/cpu.h | 43 +++++++++++++ target-arm/helper.c | 167 +++++++++++++++++++++++++++++++++++++++++++++++++-- target-arm/machine.c | 46 ++++++++++---- vmstate.c | 8 +-- 6 files changed, 263 insertions(+), 26 deletions(-) -- 1.8.3.2