* [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21
@ 2019-06-21 11:29 Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 01/25] kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del Paolo Bonzini
` (26 more replies)
0 siblings, 27 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:29 UTC (permalink / raw)
To: qemu-devel
The following changes since commit 33d609990621dea6c7d056c86f707b8811320ac1:
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-06-18 17:00:52 +0100)
are available in the git repository at:
git://github.com/bonzini/qemu.git tags/for-upstream
for you to fetch changes up to 8e8cbed09ad9d577955691b4c061b61b602406d1:
hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1 (2019-06-21 13:25:29 +0200)
----------------------------------------------------------------
* Nuke hw_compat_4_0_1 and pc_compat_4_0_1 (Greg)
* Static analysis fixes (Igor, Lidong)
* X86 Hyper-V CPUID improvements (Vitaly)
* X86 nested virt migration (Liran)
* New MSR-based features (Xiaoyao)
----------------------------------------------------------------
Colin Xu (1):
hax: Honor CPUState::halted
Greg Kurz (1):
hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1
Lidong Chen (2):
sd: Fix out-of-bounds assertions
util/main-loop: Fix incorrect assertion
Liran Alon (10):
target/i386: kvm: Delete VMX migration blocker on vCPU init failure
KVM: Introduce kvm_arch_destroy_vcpu()
target/i386: kvm: Use symbolic constant for #DB/#BP exception constants
target/i386: kvm: Re-inject #DB to guest with updated DR6
target/i386: kvm: Block migration for vCPUs exposed with nested virtualization
linux-headers: sync with latest KVM headers from Linux 5.2
vmstate: Add support for kernel integer types
target/i386: kvm: Add support for save and restore nested state
target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD
target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
Vitaly Kuznetsov (9):
i386/kvm: convert hyperv enlightenments properties from bools to bits
i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID
i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties()
i386/kvm: document existing Hyper-V enlightenments
i386/kvm: implement 'hv-passthrough' mode
i386/kvm: hv-stimer requires hv-time and hv-synic
i386/kvm: hv-tlbflush/ipi require hv-vpindex
i386/kvm: hv-evmcs requires hv-vapic
i386/kvm: add support for Direct Mode for Hyper-V synthetic timers
Xiaoyao Li (1):
target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY
Yury Kotov (1):
kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del
accel/kvm/kvm-all.c | 25 +-
cpus.c | 1 -
docs/hyperv.txt | 201 +++++++++
hw/core/machine.c | 5 +-
hw/i386/pc.c | 6 +-
hw/i386/pc_q35.c | 12 +-
hw/sd/sd.c | 4 +-
include/hw/boards.h | 3 -
include/hw/i386/pc.h | 3 -
include/migration/vmstate.h | 26 ++
include/sysemu/kvm.h | 2 +
linux-headers/asm-x86/kvm.h | 33 +-
target/arm/kvm32.c | 5 +
target/arm/kvm64.c | 5 +
target/i386/cpu.c | 81 +++-
target/i386/cpu.h | 65 ++-
target/i386/hax-all.c | 36 +-
target/i386/hvf/hvf.c | 10 +-
target/i386/hvf/x86hvf.c | 4 +-
target/i386/hyperv-proto.h | 1 +
target/i386/hyperv.c | 2 +-
target/i386/kvm.c | 999 +++++++++++++++++++++++++++++++++-----------
target/i386/machine.c | 284 ++++++++++++-
target/mips/kvm.c | 5 +
target/ppc/kvm.c | 5 +
target/s390x/kvm.c | 10 +
util/main-loop.c | 2 +-
27 files changed, 1506 insertions(+), 329 deletions(-)
create mode 100644 docs/hyperv.txt
--
1.8.3.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 01/25] kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
@ 2019-06-21 11:29 ` Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 02/25] hax: Honor CPUState::halted Paolo Bonzini
` (25 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Yury Kotov
From: Yury Kotov <yury-kotov@yandex-team.ru>
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190607090830.18807-1-yury-kotov@yandex-team.ru>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index b0c4bed..d2f481a 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -863,8 +863,8 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,
data, true, int128_get64(section->size),
match_data);
if (r < 0) {
- fprintf(stderr, "%s: error adding ioeventfd: %s\n",
- __func__, strerror(-r));
+ fprintf(stderr, "%s: error adding ioeventfd: %s (%d)\n",
+ __func__, strerror(-r), -r);
abort();
}
}
@@ -881,6 +881,8 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,
data, false, int128_get64(section->size),
match_data);
if (r < 0) {
+ fprintf(stderr, "%s: error deleting ioeventfd: %s (%d)\n",
+ __func__, strerror(-r), -r);
abort();
}
}
@@ -897,8 +899,8 @@ static void kvm_io_ioeventfd_add(MemoryListener *listener,
data, true, int128_get64(section->size),
match_data);
if (r < 0) {
- fprintf(stderr, "%s: error adding ioeventfd: %s\n",
- __func__, strerror(-r));
+ fprintf(stderr, "%s: error adding ioeventfd: %s (%d)\n",
+ __func__, strerror(-r), -r);
abort();
}
}
@@ -916,6 +918,8 @@ static void kvm_io_ioeventfd_del(MemoryListener *listener,
data, false, int128_get64(section->size),
match_data);
if (r < 0) {
+ fprintf(stderr, "%s: error deleting ioeventfd: %s (%d)\n",
+ __func__, strerror(-r), -r);
abort();
}
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 02/25] hax: Honor CPUState::halted
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 01/25] kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del Paolo Bonzini
@ 2019-06-21 11:29 ` Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits Paolo Bonzini
` (24 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Colin Xu, Yu Ning, Chuanxiao Dong
From: Colin Xu <colin.xu@intel.com>
QEMU tracks whether a vcpu is halted using CPUState::halted. E.g.,
after initialization or reset, halted is 0 for the BSP (vcpu 0)
and 1 for the APs (vcpu 1, 2, ...). A halted vcpu should not be
handed to the hypervisor to run (e.g. hax_vcpu_run()).
Under HAXM, Android Emulator sometimes boots into a "vcpu shutdown
request" error while executing in SeaBIOS, with the HAXM driver
logging a guest triple fault in vcpu 1, 2, ... at RIP 0x3. That is
ultimately because the HAX accelerator asks HAXM to run those APs
when they are still in the halted state.
Normally, the vcpu thread for an AP will start by looping in
qemu_wait_io_event(), until the BSP kicks it via a pair of IPIs
(INIT followed by SIPI). But because the HAX accelerator does not
honor cpu->halted, it allows the AP vcpu thread to proceed to
hax_vcpu_run() as soon as it receives any kick, even if the kick
does not come from the BSP. It turns out that emulator has a
worker thread which periodically kicks every vcpu thread (possibly
to collect CPU usage data), and if one of these kicks comes before
those by the BSP, the AP will start execution from the wrong RIP,
resulting in the aforementioned SMP boot failure.
The solution is inspired by the KVM accelerator (credit to
Chuanxiao Dong <chuanxiao.dong@intel.com> for the pointer):
1. Get rid of questionable logic that unconditionally resets
cpu->halted before hax_vcpu_run(). Instead, only reset it at the
right moments (there are only a few "unhalt" events).
2. Add a check for cpu->halted before hax_vcpu_run().
Note that although the non-Unrestricted Guest (!ug_platform) code
path also forcibly resets cpu->halted, it is left untouched,
because only the UG code path supports SMP guests.
The patch is first merged to android emulator with Change-Id:
I9c5752cc737fd305d7eace1768ea12a07309d716
Cc: Yu Ning <yu.ning@intel.com>
Cc: Chuanxiao Dong <chuanxiao.dong@intel.com>
Signed-off-by: Colin Xu <colin.xu@intel.com>
Message-Id: <20190610021939.13669-1-colin.xu@intel.com>
---
cpus.c | 1 -
target/i386/hax-all.c | 36 ++++++++++++++++++++++++++++++++++--
2 files changed, 34 insertions(+), 3 deletions(-)
diff --git a/cpus.c b/cpus.c
index dde3b7b..1af51b7 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1594,7 +1594,6 @@ static void *qemu_hax_cpu_thread_fn(void *arg)
cpu->thread_id = qemu_get_thread_id();
cpu->created = true;
- cpu->halted = 0;
current_cpu = cpu;
hax_init_vcpu(cpu);
diff --git a/target/i386/hax-all.c b/target/i386/hax-all.c
index 64fd51a..9e7b779 100644
--- a/target/i386/hax-all.c
+++ b/target/i386/hax-all.c
@@ -471,13 +471,35 @@ static int hax_vcpu_hax_exec(CPUArchState *env)
return 0;
}
- cpu->halted = 0;
-
if (cpu->interrupt_request & CPU_INTERRUPT_POLL) {
cpu->interrupt_request &= ~CPU_INTERRUPT_POLL;
apic_poll_irq(x86_cpu->apic_state);
}
+ /* After a vcpu is halted (either because it is an AP and has just been
+ * reset, or because it has executed the HLT instruction), it will not be
+ * run (hax_vcpu_run()) until it is unhalted. The next few if blocks check
+ * for events that may change the halted state of this vcpu:
+ * a) Maskable interrupt, when RFLAGS.IF is 1;
+ * Note: env->eflags may not reflect the current RFLAGS state, because
+ * it is not updated after each hax_vcpu_run(). We cannot afford
+ * to fail to recognize any unhalt-by-maskable-interrupt event
+ * (in which case the vcpu will halt forever), and yet we cannot
+ * afford the overhead of hax_vcpu_sync_state(). The current
+ * solution is to err on the side of caution and have the HLT
+ * handler (see case HAX_EXIT_HLT below) unconditionally set the
+ * IF_MASK bit in env->eflags, which, in effect, disables the
+ * RFLAGS.IF check.
+ * b) NMI;
+ * c) INIT signal;
+ * d) SIPI signal.
+ */
+ if (((cpu->interrupt_request & CPU_INTERRUPT_HARD) &&
+ (env->eflags & IF_MASK)) ||
+ (cpu->interrupt_request & CPU_INTERRUPT_NMI)) {
+ cpu->halted = 0;
+ }
+
if (cpu->interrupt_request & CPU_INTERRUPT_INIT) {
DPRINTF("\nhax_vcpu_hax_exec: handling INIT for %d\n",
cpu->cpu_index);
@@ -493,6 +515,16 @@ static int hax_vcpu_hax_exec(CPUArchState *env)
hax_vcpu_sync_state(env, 1);
}
+ if (cpu->halted) {
+ /* If this vcpu is halted, we must not ask HAXM to run it. Instead, we
+ * break out of hax_smp_cpu_exec() as if this vcpu had executed HLT.
+ * That way, this vcpu thread will be trapped in qemu_wait_io_event(),
+ * until the vcpu is unhalted.
+ */
+ cpu->exception_index = EXCP_HLT;
+ return 0;
+ }
+
do {
int hax_ret;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 01/25] kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 02/25] hax: Honor CPUState::halted Paolo Bonzini
@ 2019-06-21 11:29 ` Paolo Bonzini
2019-06-24 11:04 ` Alex Bennée
2019-06-21 11:29 ` [Qemu-devel] [PULL 04/25] i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID Paolo Bonzini
` (23 subsequent siblings)
26 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Representing Hyper-V properties as bits will allow us to check features
and dependencies between them in a natural way.
Suggested-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-2-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/i386/pc.c | 3 +-
target/i386/cpu.c | 44 ++++++++++++++++++---------
target/i386/cpu.h | 37 ++++++++++++++---------
target/i386/hyperv.c | 2 +-
target/i386/kvm.c | 83 +++++++++++++++++++++------------------------------
target/i386/machine.c | 2 +-
6 files changed, 91 insertions(+), 80 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2c5446b..e41192b 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2386,7 +2386,8 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
}
cpu->thread_id = topo.smt_id;
- if (cpu->hyperv_vpindex && !kvm_hv_vpindex_settable()) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) &&
+ !kvm_hv_vpindex_settable()) {
error_setg(errp, "kernel doesn't allow setting HyperV VP_INDEX");
return;
}
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index fbed2eb..e90c1ac 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5853,21 +5853,37 @@ static Property x86_cpu_properties[] = {
#endif
DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID),
DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
+
{ .name = "hv-spinlocks", .info = &qdev_prop_spinlocks },
- DEFINE_PROP_BOOL("hv-relaxed", X86CPU, hyperv_relaxed_timing, false),
- DEFINE_PROP_BOOL("hv-vapic", X86CPU, hyperv_vapic, false),
- DEFINE_PROP_BOOL("hv-time", X86CPU, hyperv_time, false),
- DEFINE_PROP_BOOL("hv-crash", X86CPU, hyperv_crash, false),
- DEFINE_PROP_BOOL("hv-reset", X86CPU, hyperv_reset, false),
- DEFINE_PROP_BOOL("hv-vpindex", X86CPU, hyperv_vpindex, false),
- DEFINE_PROP_BOOL("hv-runtime", X86CPU, hyperv_runtime, false),
- DEFINE_PROP_BOOL("hv-synic", X86CPU, hyperv_synic, false),
- DEFINE_PROP_BOOL("hv-stimer", X86CPU, hyperv_stimer, false),
- DEFINE_PROP_BOOL("hv-frequencies", X86CPU, hyperv_frequencies, false),
- DEFINE_PROP_BOOL("hv-reenlightenment", X86CPU, hyperv_reenlightenment, false),
- DEFINE_PROP_BOOL("hv-tlbflush", X86CPU, hyperv_tlbflush, false),
- DEFINE_PROP_BOOL("hv-evmcs", X86CPU, hyperv_evmcs, false),
- DEFINE_PROP_BOOL("hv-ipi", X86CPU, hyperv_ipi, false),
+ DEFINE_PROP_BIT64("hv-relaxed", X86CPU, hyperv_features,
+ HYPERV_FEAT_RELAXED, 0),
+ DEFINE_PROP_BIT64("hv-vapic", X86CPU, hyperv_features,
+ HYPERV_FEAT_VAPIC, 0),
+ DEFINE_PROP_BIT64("hv-time", X86CPU, hyperv_features,
+ HYPERV_FEAT_TIME, 0),
+ DEFINE_PROP_BIT64("hv-crash", X86CPU, hyperv_features,
+ HYPERV_FEAT_CRASH, 0),
+ DEFINE_PROP_BIT64("hv-reset", X86CPU, hyperv_features,
+ HYPERV_FEAT_RESET, 0),
+ DEFINE_PROP_BIT64("hv-vpindex", X86CPU, hyperv_features,
+ HYPERV_FEAT_VPINDEX, 0),
+ DEFINE_PROP_BIT64("hv-runtime", X86CPU, hyperv_features,
+ HYPERV_FEAT_RUNTIME, 0),
+ DEFINE_PROP_BIT64("hv-synic", X86CPU, hyperv_features,
+ HYPERV_FEAT_SYNIC, 0),
+ DEFINE_PROP_BIT64("hv-stimer", X86CPU, hyperv_features,
+ HYPERV_FEAT_STIMER, 0),
+ DEFINE_PROP_BIT64("hv-frequencies", X86CPU, hyperv_features,
+ HYPERV_FEAT_FREQUENCIES, 0),
+ DEFINE_PROP_BIT64("hv-reenlightenment", X86CPU, hyperv_features,
+ HYPERV_FEAT_REENLIGHTENMENT, 0),
+ DEFINE_PROP_BIT64("hv-tlbflush", X86CPU, hyperv_features,
+ HYPERV_FEAT_TLBFLUSH, 0),
+ DEFINE_PROP_BIT64("hv-evmcs", X86CPU, hyperv_features,
+ HYPERV_FEAT_EVMCS, 0),
+ DEFINE_PROP_BIT64("hv-ipi", X86CPU, hyperv_features,
+ HYPERV_FEAT_IPI, 0),
+
DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true),
DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false),
DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0732e05..30cd1a0 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -734,6 +734,22 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
#define MSR_ARCH_CAP_SKIP_L1DFL_VMENTRY (1U << 3)
#define MSR_ARCH_CAP_SSB_NO (1U << 4)
+/* Supported Hyper-V Enlightenments */
+#define HYPERV_FEAT_RELAXED 0
+#define HYPERV_FEAT_VAPIC 1
+#define HYPERV_FEAT_TIME 2
+#define HYPERV_FEAT_CRASH 3
+#define HYPERV_FEAT_RESET 4
+#define HYPERV_FEAT_VPINDEX 5
+#define HYPERV_FEAT_RUNTIME 6
+#define HYPERV_FEAT_SYNIC 7
+#define HYPERV_FEAT_STIMER 8
+#define HYPERV_FEAT_FREQUENCIES 9
+#define HYPERV_FEAT_REENLIGHTENMENT 10
+#define HYPERV_FEAT_TLBFLUSH 11
+#define HYPERV_FEAT_EVMCS 12
+#define HYPERV_FEAT_IPI 13
+
#ifndef HYPERV_SPINLOCK_NEVER_RETRY
#define HYPERV_SPINLOCK_NEVER_RETRY 0xFFFFFFFF
#endif
@@ -1370,23 +1386,11 @@ struct X86CPU {
CPUNegativeOffsetState neg;
CPUX86State env;
- bool hyperv_vapic;
- bool hyperv_relaxed_timing;
int hyperv_spinlock_attempts;
char *hyperv_vendor_id;
- bool hyperv_time;
- bool hyperv_crash;
- bool hyperv_reset;
- bool hyperv_vpindex;
- bool hyperv_runtime;
- bool hyperv_synic;
bool hyperv_synic_kvm_only;
- bool hyperv_stimer;
- bool hyperv_frequencies;
- bool hyperv_reenlightenment;
- bool hyperv_tlbflush;
- bool hyperv_evmcs;
- bool hyperv_ipi;
+ uint64_t hyperv_features;
+
bool check_cpuid;
bool enforce_cpuid;
bool expose_kvm;
@@ -1906,4 +1910,9 @@ void x86_cpu_xrstor_all_areas(X86CPU *cpu, const X86XSaveArea *buf);
void x86_cpu_xsave_all_areas(X86CPU *cpu, X86XSaveArea *buf);
void x86_update_hflags(CPUX86State* env);
+static inline bool hyperv_feat_enabled(X86CPU *cpu, int feat)
+{
+ return !!(cpu->hyperv_features & BIT(feat));
+}
+
#endif /* I386_CPU_H */
diff --git a/target/i386/hyperv.c b/target/i386/hyperv.c
index b264a28..26efc1e 100644
--- a/target/i386/hyperv.c
+++ b/target/i386/hyperv.c
@@ -52,7 +52,7 @@ int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit)
switch (exit->type) {
case KVM_EXIT_HYPERV_SYNIC:
- if (!cpu->hyperv_synic) {
+ if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
return -1;
}
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 6899061..b34eb81 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -634,28 +634,12 @@ unsigned long kvm_arch_vcpu_id(CPUState *cs)
#define KVM_CPUID_SIGNATURE_NEXT 0x40000100
#endif
-static bool hyperv_hypercall_available(X86CPU *cpu)
-{
- return cpu->hyperv_vapic ||
- (cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY);
-}
-
static bool hyperv_enabled(X86CPU *cpu)
{
CPUState *cs = CPU(cpu);
return kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0 &&
- (hyperv_hypercall_available(cpu) ||
- cpu->hyperv_time ||
- cpu->hyperv_relaxed_timing ||
- cpu->hyperv_crash ||
- cpu->hyperv_reset ||
- cpu->hyperv_vpindex ||
- cpu->hyperv_runtime ||
- cpu->hyperv_synic ||
- cpu->hyperv_stimer ||
- cpu->hyperv_reenlightenment ||
- cpu->hyperv_tlbflush ||
- cpu->hyperv_ipi);
+ ((cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY) ||
+ cpu->hyperv_features);
}
static int kvm_arch_set_tsc_khz(CPUState *cs)
@@ -704,14 +688,14 @@ static int hyperv_handle_properties(CPUState *cs)
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
- if (cpu->hyperv_relaxed_timing) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RELAXED)) {
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_APIC_ACCESS_AVAILABLE;
}
- if (cpu->hyperv_time) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) <= 0) {
fprintf(stderr, "Hyper-V clocksources "
"(requested by 'hv-time' cpu flag) "
@@ -722,7 +706,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_TIME_REF_COUNT_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_REFERENCE_TSC_AVAILABLE;
}
- if (cpu->hyperv_frequencies) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_FREQUENCIES)) {
if (!has_msr_hv_frequencies) {
fprintf(stderr, "Hyper-V frequency MSRs "
"(requested by 'hv-frequencies' cpu flag) "
@@ -732,7 +716,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_FREQUENCY_MSRS;
env->features[FEAT_HYPERV_EDX] |= HV_FREQUENCY_MSRS_AVAILABLE;
}
- if (cpu->hyperv_crash) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_CRASH)) {
if (!has_msr_hv_crash) {
fprintf(stderr, "Hyper-V crash MSRs "
"(requested by 'hv-crash' cpu flag) "
@@ -741,7 +725,7 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EDX] |= HV_GUEST_CRASH_MSR_AVAILABLE;
}
- if (cpu->hyperv_reenlightenment) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
if (!has_msr_hv_reenlightenment) {
fprintf(stderr,
"Hyper-V Reenlightenment MSRs "
@@ -752,7 +736,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_REENLIGHTENMENTS_CONTROL;
}
env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
- if (cpu->hyperv_reset) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RESET)) {
if (!has_msr_hv_reset) {
fprintf(stderr, "Hyper-V reset MSR "
"(requested by 'hv-reset' cpu flag) "
@@ -761,7 +745,7 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EAX] |= HV_RESET_AVAILABLE;
}
- if (cpu->hyperv_vpindex) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
if (!has_msr_hv_vpindex) {
fprintf(stderr, "Hyper-V VP_INDEX MSR "
"(requested by 'hv-vpindex' cpu flag) "
@@ -770,7 +754,7 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EAX] |= HV_VP_INDEX_AVAILABLE;
}
- if (cpu->hyperv_runtime) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RUNTIME)) {
if (!has_msr_hv_runtime) {
fprintf(stderr, "Hyper-V VP_RUNTIME MSR "
"(requested by 'hv-runtime' cpu flag) "
@@ -779,10 +763,10 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EAX] |= HV_VP_RUNTIME_AVAILABLE;
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
unsigned int cap = KVM_CAP_HYPERV_SYNIC;
if (!cpu->hyperv_synic_kvm_only) {
- if (!cpu->hyperv_vpindex) {
+ if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
fprintf(stderr, "Hyper-V SynIC "
"(requested by 'hv-synic' cpu flag) "
"requires Hyper-V VP_INDEX ('hv-vpindex')\n");
@@ -799,20 +783,20 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_SYNIC_AVAILABLE;
}
- if (cpu->hyperv_stimer) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_STIMER)) {
if (!has_msr_hv_stimer) {
fprintf(stderr, "Hyper-V timers aren't supported by kernel\n");
return -ENOSYS;
}
env->features[FEAT_HYPERV_EAX] |= HV_SYNTIMERS_AVAILABLE;
}
- if (cpu->hyperv_relaxed_timing) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RELAXED)) {
env->features[FEAT_HV_RECOMM_EAX] |= HV_RELAXED_TIMING_RECOMMENDED;
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
env->features[FEAT_HV_RECOMM_EAX] |= HV_APIC_ACCESS_RECOMMENDED;
}
- if (cpu->hyperv_tlbflush) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TLBFLUSH)) {
if (kvm_check_extension(cs->kvm_state,
KVM_CAP_HYPERV_TLBFLUSH) <= 0) {
fprintf(stderr, "Hyper-V TLB flush support "
@@ -823,7 +807,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HV_RECOMM_EAX] |= HV_REMOTE_TLB_FLUSH_RECOMMENDED;
env->features[FEAT_HV_RECOMM_EAX] |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
}
- if (cpu->hyperv_ipi) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_IPI)) {
if (kvm_check_extension(cs->kvm_state,
KVM_CAP_HYPERV_SEND_IPI) <= 0) {
fprintf(stderr, "Hyper-V IPI send support "
@@ -834,7 +818,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HV_RECOMM_EAX] |= HV_CLUSTER_IPI_RECOMMENDED;
env->features[FEAT_HV_RECOMM_EAX] |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
}
- if (cpu->hyperv_evmcs) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
uint16_t evmcs_version;
if (kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
@@ -856,7 +840,7 @@ static int hyperv_init_vcpu(X86CPU *cpu)
CPUState *cs = CPU(cpu);
int ret;
- if (cpu->hyperv_vpindex && !hv_vpindex_settable) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) && !hv_vpindex_settable) {
/*
* the kernel doesn't support setting vp_index; assert that its value
* is in sync
@@ -881,7 +865,7 @@ static int hyperv_init_vcpu(X86CPU *cpu)
}
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
uint32_t synic_cap = cpu->hyperv_synic_kvm_only ?
KVM_CAP_HYPERV_SYNIC : KVM_CAP_HYPERV_SYNIC2;
ret = kvm_vcpu_enable_cap(cs, synic_cap, 0);
@@ -972,7 +956,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
memset(signature, 0, 12);
memcpy(signature, cpu->hyperv_vendor_id, len);
}
- c->eax = cpu->hyperv_evmcs ?
+ c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
c->ebx = signature[0];
c->ecx = signature[1];
@@ -1016,7 +1000,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
kvm_base = KVM_CPUID_SIGNATURE_NEXT;
has_msr_hv_hypercall = true;
- if (cpu->hyperv_evmcs) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
__u32 function;
/* Create zeroed 0x40000006..0x40000009 leaves */
@@ -1360,7 +1344,7 @@ void kvm_arch_reset_vcpu(X86CPU *cpu)
env->mp_state = KVM_MP_STATE_RUNNABLE;
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
int i;
for (i = 0; i < ARRAY_SIZE(env->msr_hv_synic_sint); i++) {
env->msr_hv_synic_sint[i] = HV_SINT_MASKED;
@@ -2100,11 +2084,11 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
kvm_msr_entry_add(cpu, HV_X64_MSR_HYPERCALL,
env->msr_hv_hypercall);
}
- if (cpu->hyperv_time) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC,
env->msr_hv_tsc);
}
- if (cpu->hyperv_reenlightenment) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL,
env->msr_hv_reenlightenment_control);
kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL,
@@ -2113,7 +2097,7 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
env->msr_hv_tsc_emulation_status);
}
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_APIC_ASSIST_PAGE,
env->msr_hv_vapic);
}
@@ -2129,11 +2113,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
if (has_msr_hv_runtime) {
kvm_msr_entry_add(cpu, HV_X64_MSR_VP_RUNTIME, env->msr_hv_runtime);
}
- if (cpu->hyperv_vpindex && hv_vpindex_settable) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)
+ && hv_vpindex_settable) {
kvm_msr_entry_add(cpu, HV_X64_MSR_VP_INDEX,
hyperv_vp_index(CPU(cpu)));
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
int j;
kvm_msr_entry_add(cpu, HV_X64_MSR_SVERSION, HV_SYNIC_VERSION);
@@ -2473,13 +2458,13 @@ static int kvm_get_msrs(X86CPU *cpu)
kvm_msr_entry_add(cpu, HV_X64_MSR_HYPERCALL, 0);
kvm_msr_entry_add(cpu, HV_X64_MSR_GUEST_OS_ID, 0);
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_APIC_ASSIST_PAGE, 0);
}
- if (cpu->hyperv_time) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC, 0);
}
- if (cpu->hyperv_reenlightenment) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL, 0);
kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL, 0);
kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_STATUS, 0);
@@ -2494,7 +2479,7 @@ static int kvm_get_msrs(X86CPU *cpu)
if (has_msr_hv_runtime) {
kvm_msr_entry_add(cpu, HV_X64_MSR_VP_RUNTIME, 0);
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
uint32_t msr;
kvm_msr_entry_add(cpu, HV_X64_MSR_SCONTROL, 0);
diff --git a/target/i386/machine.c b/target/i386/machine.c
index 4aff1a7..a39ce7f 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -634,7 +634,7 @@ static bool hyperv_runtime_enable_needed(void *opaque)
X86CPU *cpu = opaque;
CPUX86State *env = &cpu->env;
- if (!cpu->hyperv_runtime) {
+ if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_RUNTIME)) {
return false;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 04/25] i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (2 preceding siblings ...)
2019-06-21 11:29 ` [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits Paolo Bonzini
@ 2019-06-21 11:29 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 05/25] i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties() Paolo Bonzini
` (22 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
KVM now supports reporting supported Hyper-V features through CPUID
(KVM_GET_SUPPORTED_HV_CPUID ioctl). Going forward, this is going to be
the only way to announce new functionality and this has already happened
with Direct Mode stimers.
While we could just support KVM_GET_SUPPORTED_HV_CPUID for new features,
it seems to be beneficial to use it for all Hyper-V enlightenments when
possible. This way we can implement 'hv-all' pass-through mode giving the
guest all supported Hyper-V features even when QEMU knows nothing about
them.
Implementation-wise we create a new kvm_hyperv_properties structure
defining Hyper-V features, get_supported_hv_cpuid()/
get_supported_hv_cpuid_legacy() returning the supported CPUID set and
a bit over-engineered hv_cpuid_check_and_set() which we will also be
used to set cpu->hyperv_* properties for 'hv-all' mode.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-3-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 474 ++++++++++++++++++++++++++++++++++++++++--------------
1 file changed, 356 insertions(+), 118 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b34eb81..cd492d4 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -683,156 +683,394 @@ static bool tsc_is_stable_and_known(CPUX86State *env)
|| env->user_tsc_khz;
}
-static int hyperv_handle_properties(CPUState *cs)
+static struct {
+ const char *desc;
+ struct {
+ uint32_t fw;
+ uint32_t bits;
+ } flags[2];
+} kvm_hyperv_properties[] = {
+ [HYPERV_FEAT_RELAXED] = {
+ .desc = "relaxed timing (hv-relaxed)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_HYPERCALL_AVAILABLE},
+ {.fw = FEAT_HV_RECOMM_EAX,
+ .bits = HV_RELAXED_TIMING_RECOMMENDED}
+ }
+ },
+ [HYPERV_FEAT_VAPIC] = {
+ .desc = "virtual APIC (hv-vapic)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_HYPERCALL_AVAILABLE | HV_APIC_ACCESS_AVAILABLE},
+ {.fw = FEAT_HV_RECOMM_EAX,
+ .bits = HV_APIC_ACCESS_RECOMMENDED}
+ }
+ },
+ [HYPERV_FEAT_TIME] = {
+ .desc = "clocksources (hv-time)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_HYPERCALL_AVAILABLE | HV_TIME_REF_COUNT_AVAILABLE |
+ HV_REFERENCE_TSC_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_CRASH] = {
+ .desc = "crash MSRs (hv-crash)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EDX,
+ .bits = HV_GUEST_CRASH_MSR_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_RESET] = {
+ .desc = "reset MSR (hv-reset)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_RESET_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_VPINDEX] = {
+ .desc = "VP_INDEX MSR (hv-vpindex)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_VP_INDEX_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_RUNTIME] = {
+ .desc = "VP_RUNTIME MSR (hv-runtime)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_VP_RUNTIME_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_SYNIC] = {
+ .desc = "synthetic interrupt controller (hv-synic)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_SYNIC_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_STIMER] = {
+ .desc = "synthetic timers (hv-stimer)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_SYNTIMERS_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_FREQUENCIES] = {
+ .desc = "frequency MSRs (hv-frequencies)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_ACCESS_FREQUENCY_MSRS},
+ {.fw = FEAT_HYPERV_EDX,
+ .bits = HV_FREQUENCY_MSRS_AVAILABLE}
+ }
+ },
+ [HYPERV_FEAT_REENLIGHTENMENT] = {
+ .desc = "reenlightenment MSRs (hv-reenlightenment)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EAX,
+ .bits = HV_ACCESS_REENLIGHTENMENTS_CONTROL}
+ }
+ },
+ [HYPERV_FEAT_TLBFLUSH] = {
+ .desc = "paravirtualized TLB flush (hv-tlbflush)",
+ .flags = {
+ {.fw = FEAT_HV_RECOMM_EAX,
+ .bits = HV_REMOTE_TLB_FLUSH_RECOMMENDED |
+ HV_EX_PROCESSOR_MASKS_RECOMMENDED}
+ }
+ },
+ [HYPERV_FEAT_EVMCS] = {
+ .desc = "enlightened VMCS (hv-evmcs)",
+ .flags = {
+ {.fw = FEAT_HV_RECOMM_EAX,
+ .bits = HV_ENLIGHTENED_VMCS_RECOMMENDED}
+ }
+ },
+ [HYPERV_FEAT_IPI] = {
+ .desc = "paravirtualized IPI (hv-ipi)",
+ .flags = {
+ {.fw = FEAT_HV_RECOMM_EAX,
+ .bits = HV_CLUSTER_IPI_RECOMMENDED |
+ HV_EX_PROCESSOR_MASKS_RECOMMENDED}
+ }
+ },
+};
+
+static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max)
+{
+ struct kvm_cpuid2 *cpuid;
+ int r, size;
+
+ size = sizeof(*cpuid) + max * sizeof(*cpuid->entries);
+ cpuid = g_malloc0(size);
+ cpuid->nent = max;
+
+ r = kvm_vcpu_ioctl(cs, KVM_GET_SUPPORTED_HV_CPUID, cpuid);
+ if (r == 0 && cpuid->nent >= max) {
+ r = -E2BIG;
+ }
+ if (r < 0) {
+ if (r == -E2BIG) {
+ g_free(cpuid);
+ return NULL;
+ } else {
+ fprintf(stderr, "KVM_GET_SUPPORTED_HV_CPUID failed: %s\n",
+ strerror(-r));
+ exit(1);
+ }
+ }
+ return cpuid;
+}
+
+/*
+ * Run KVM_GET_SUPPORTED_HV_CPUID ioctl(), allocating a buffer large enough
+ * for all entries.
+ */
+static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
+{
+ struct kvm_cpuid2 *cpuid;
+ int max = 7; /* 0x40000000..0x40000005, 0x4000000A */
+
+ /*
+ * When the buffer is too small, KVM_GET_SUPPORTED_HV_CPUID fails with
+ * -E2BIG, however, it doesn't report back the right size. Keep increasing
+ * it and re-trying until we succeed.
+ */
+ while ((cpuid = try_get_hv_cpuid(cs, max)) == NULL) {
+ max++;
+ }
+ return cpuid;
+}
+
+/*
+ * When KVM_GET_SUPPORTED_HV_CPUID is not supported we fill CPUID feature
+ * leaves from KVM_CAP_HYPERV* and present MSRs data.
+ */
+static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)
{
X86CPU *cpu = X86_CPU(cs);
- CPUX86State *env = &cpu->env;
+ struct kvm_cpuid2 *cpuid;
+ struct kvm_cpuid_entry2 *entry_feat, *entry_recomm;
+
+ /* HV_CPUID_FEATURES, HV_CPUID_ENLIGHTMENT_INFO */
+ cpuid = g_malloc0(sizeof(*cpuid) + 2 * sizeof(*cpuid->entries));
+ cpuid->nent = 2;
+
+ /* HV_CPUID_VENDOR_AND_MAX_FUNCTIONS */
+ entry_feat = &cpuid->entries[0];
+ entry_feat->function = HV_CPUID_FEATURES;
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RELAXED)) {
- env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
+ entry_recomm = &cpuid->entries[1];
+ entry_recomm->function = HV_CPUID_ENLIGHTMENT_INFO;
+ entry_recomm->ebx = cpu->hyperv_spinlock_attempts;
+
+ if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0) {
+ entry_feat->eax |= HV_HYPERCALL_AVAILABLE;
+ entry_feat->eax |= HV_APIC_ACCESS_AVAILABLE;
+ entry_feat->edx |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
+ entry_recomm->eax |= HV_RELAXED_TIMING_RECOMMENDED;
+ entry_recomm->eax |= HV_APIC_ACCESS_RECOMMENDED;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
- env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
- env->features[FEAT_HYPERV_EAX] |= HV_APIC_ACCESS_AVAILABLE;
+
+ if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) > 0) {
+ entry_feat->eax |= HV_TIME_REF_COUNT_AVAILABLE;
+ entry_feat->eax |= HV_REFERENCE_TSC_AVAILABLE;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
- if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) <= 0) {
- fprintf(stderr, "Hyper-V clocksources "
- "(requested by 'hv-time' cpu flag) "
- "are not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
- env->features[FEAT_HYPERV_EAX] |= HV_TIME_REF_COUNT_AVAILABLE;
- env->features[FEAT_HYPERV_EAX] |= HV_REFERENCE_TSC_AVAILABLE;
+
+ if (has_msr_hv_frequencies) {
+ entry_feat->eax |= HV_ACCESS_FREQUENCY_MSRS;
+ entry_feat->edx |= HV_FREQUENCY_MSRS_AVAILABLE;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_FREQUENCIES)) {
- if (!has_msr_hv_frequencies) {
- fprintf(stderr, "Hyper-V frequency MSRs "
- "(requested by 'hv-frequencies' cpu flag) "
- "are not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_FREQUENCY_MSRS;
- env->features[FEAT_HYPERV_EDX] |= HV_FREQUENCY_MSRS_AVAILABLE;
+
+ if (has_msr_hv_crash) {
+ entry_feat->edx |= HV_GUEST_CRASH_MSR_AVAILABLE;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_CRASH)) {
- if (!has_msr_hv_crash) {
- fprintf(stderr, "Hyper-V crash MSRs "
- "(requested by 'hv-crash' cpu flag) "
- "are not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EDX] |= HV_GUEST_CRASH_MSR_AVAILABLE;
+
+ if (has_msr_hv_reenlightenment) {
+ entry_feat->eax |= HV_ACCESS_REENLIGHTENMENTS_CONTROL;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
- if (!has_msr_hv_reenlightenment) {
- fprintf(stderr,
- "Hyper-V Reenlightenment MSRs "
- "(requested by 'hv-reenlightenment' cpu flag) "
- "are not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_REENLIGHTENMENTS_CONTROL;
+
+ if (has_msr_hv_reset) {
+ entry_feat->eax |= HV_RESET_AVAILABLE;
}
- env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RESET)) {
- if (!has_msr_hv_reset) {
- fprintf(stderr, "Hyper-V reset MSR "
- "(requested by 'hv-reset' cpu flag) "
- "is not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EAX] |= HV_RESET_AVAILABLE;
+
+ if (has_msr_hv_vpindex) {
+ entry_feat->eax |= HV_VP_INDEX_AVAILABLE;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
- if (!has_msr_hv_vpindex) {
- fprintf(stderr, "Hyper-V VP_INDEX MSR "
- "(requested by 'hv-vpindex' cpu flag) "
- "is not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EAX] |= HV_VP_INDEX_AVAILABLE;
+
+ if (has_msr_hv_runtime) {
+ entry_feat->eax |= HV_VP_RUNTIME_AVAILABLE;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RUNTIME)) {
- if (!has_msr_hv_runtime) {
- fprintf(stderr, "Hyper-V VP_RUNTIME MSR "
- "(requested by 'hv-runtime' cpu flag) "
- "is not supported by kernel\n");
- return -ENOSYS;
+
+ if (has_msr_hv_synic) {
+ unsigned int cap = cpu->hyperv_synic_kvm_only ?
+ KVM_CAP_HYPERV_SYNIC : KVM_CAP_HYPERV_SYNIC2;
+
+ if (kvm_check_extension(cs->kvm_state, cap) > 0) {
+ entry_feat->eax |= HV_SYNIC_AVAILABLE;
}
- env->features[FEAT_HYPERV_EAX] |= HV_VP_RUNTIME_AVAILABLE;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
- unsigned int cap = KVM_CAP_HYPERV_SYNIC;
- if (!cpu->hyperv_synic_kvm_only) {
- if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
- fprintf(stderr, "Hyper-V SynIC "
- "(requested by 'hv-synic' cpu flag) "
- "requires Hyper-V VP_INDEX ('hv-vpindex')\n");
- return -ENOSYS;
- }
- cap = KVM_CAP_HYPERV_SYNIC2;
- }
- if (!has_msr_hv_synic || !kvm_check_extension(cs->kvm_state, cap)) {
- fprintf(stderr, "Hyper-V SynIC (requested by 'hv-synic' cpu flag) "
- "is not supported by kernel\n");
- return -ENOSYS;
- }
+ if (has_msr_hv_stimer) {
+ entry_feat->eax |= HV_SYNTIMERS_AVAILABLE;
+ }
- env->features[FEAT_HYPERV_EAX] |= HV_SYNIC_AVAILABLE;
+ if (kvm_check_extension(cs->kvm_state,
+ KVM_CAP_HYPERV_TLBFLUSH) > 0) {
+ entry_recomm->eax |= HV_REMOTE_TLB_FLUSH_RECOMMENDED;
+ entry_recomm->eax |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_STIMER)) {
- if (!has_msr_hv_stimer) {
- fprintf(stderr, "Hyper-V timers aren't supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HYPERV_EAX] |= HV_SYNTIMERS_AVAILABLE;
+
+ if (kvm_check_extension(cs->kvm_state,
+ KVM_CAP_HYPERV_ENLIGHTENED_VMCS) > 0) {
+ entry_recomm->eax |= HV_ENLIGHTENED_VMCS_RECOMMENDED;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RELAXED)) {
- env->features[FEAT_HV_RECOMM_EAX] |= HV_RELAXED_TIMING_RECOMMENDED;
+
+ if (kvm_check_extension(cs->kvm_state,
+ KVM_CAP_HYPERV_SEND_IPI) > 0) {
+ entry_recomm->eax |= HV_CLUSTER_IPI_RECOMMENDED;
+ entry_recomm->eax |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
- env->features[FEAT_HV_RECOMM_EAX] |= HV_APIC_ACCESS_RECOMMENDED;
- }
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TLBFLUSH)) {
- if (kvm_check_extension(cs->kvm_state,
- KVM_CAP_HYPERV_TLBFLUSH) <= 0) {
- fprintf(stderr, "Hyper-V TLB flush support "
- "(requested by 'hv-tlbflush' cpu flag) "
- " is not supported by kernel\n");
- return -ENOSYS;
- }
- env->features[FEAT_HV_RECOMM_EAX] |= HV_REMOTE_TLB_FLUSH_RECOMMENDED;
- env->features[FEAT_HV_RECOMM_EAX] |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
+
+ return cpuid;
+}
+
+static int hv_cpuid_get_fw(struct kvm_cpuid2 *cpuid, int fw, uint32_t *r)
+{
+ struct kvm_cpuid_entry2 *entry;
+ uint32_t func;
+ int reg;
+
+ switch (fw) {
+ case FEAT_HYPERV_EAX:
+ reg = R_EAX;
+ func = HV_CPUID_FEATURES;
+ break;
+ case FEAT_HYPERV_EDX:
+ reg = R_EDX;
+ func = HV_CPUID_FEATURES;
+ break;
+ case FEAT_HV_RECOMM_EAX:
+ reg = R_EAX;
+ func = HV_CPUID_ENLIGHTMENT_INFO;
+ break;
+ default:
+ return -EINVAL;
}
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_IPI)) {
- if (kvm_check_extension(cs->kvm_state,
- KVM_CAP_HYPERV_SEND_IPI) <= 0) {
- fprintf(stderr, "Hyper-V IPI send support "
- "(requested by 'hv-ipi' cpu flag) "
- " is not supported by kernel\n");
- return -ENOSYS;
+
+ entry = cpuid_find_entry(cpuid, func, 0);
+ if (!entry) {
+ return -ENOENT;
+ }
+
+ switch (reg) {
+ case R_EAX:
+ *r = entry->eax;
+ break;
+ case R_EDX:
+ *r = entry->edx;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
+ int feature)
+{
+ X86CPU *cpu = X86_CPU(cs);
+ CPUX86State *env = &cpu->env;
+ uint32_t r, fw, bits;;
+ int i;
+
+ if (!hyperv_feat_enabled(cpu, feature)) {
+ return 0;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(kvm_hyperv_properties[feature].flags); i++) {
+ fw = kvm_hyperv_properties[feature].flags[i].fw;
+ bits = kvm_hyperv_properties[feature].flags[i].bits;
+
+ if (!fw) {
+ continue;
}
- env->features[FEAT_HV_RECOMM_EAX] |= HV_CLUSTER_IPI_RECOMMENDED;
- env->features[FEAT_HV_RECOMM_EAX] |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
+
+ if (hv_cpuid_get_fw(cpuid, fw, &r) || (r & bits) != bits) {
+ fprintf(stderr,
+ "Hyper-V %s is not supported by kernel\n",
+ kvm_hyperv_properties[feature].desc);
+ return 1;
+ }
+
+ env->features[fw] |= bits;
}
+
+ return 0;
+}
+
+static int hyperv_handle_properties(CPUState *cs)
+{
+ X86CPU *cpu = X86_CPU(cs);
+ CPUX86State *env = &cpu->env;
+ struct kvm_cpuid2 *cpuid;
+ int r = 0;
+
if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
uint16_t evmcs_version;
if (kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
(uintptr_t)&evmcs_version)) {
- fprintf(stderr, "Hyper-V Enlightened VMCS "
- "(requested by 'hv-evmcs' cpu flag) "
- "is not supported by kernel\n");
+ fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
+ kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
return -ENOSYS;
}
env->features[FEAT_HV_RECOMM_EAX] |= HV_ENLIGHTENED_VMCS_RECOMMENDED;
env->features[FEAT_HV_NESTED_EAX] = evmcs_version;
}
- return 0;
+ if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
+ cpuid = get_supported_hv_cpuid(cs);
+ } else {
+ cpuid = get_supported_hv_cpuid_legacy(cs);
+ }
+
+ /* Features */
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RELAXED);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VAPIC);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TIME);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_CRASH);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RESET);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VPINDEX);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RUNTIME);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_SYNIC);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_STIMER);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_FREQUENCIES);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_REENLIGHTENMENT);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TLBFLUSH);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_EVMCS);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_IPI);
+
+ /* Dependencies */
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC) &&
+ !cpu->hyperv_synic_kvm_only &&
+ !hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
+ fprintf(stderr, "Hyper-V %s requires %s\n",
+ kvm_hyperv_properties[HYPERV_FEAT_SYNIC].desc,
+ kvm_hyperv_properties[HYPERV_FEAT_VPINDEX].desc);
+ r |= 1;
+ }
+
+ /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
+ env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
+
+ g_free(cpuid);
+
+ return r ? -ENOSYS : 0;
}
static int hyperv_init_vcpu(X86CPU *cpu)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 05/25] i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties()
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (3 preceding siblings ...)
2019-06-21 11:29 ` [Qemu-devel] [PULL 04/25] i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 06/25] i386/kvm: document existing Hyper-V enlightenments Paolo Bonzini
` (21 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Let's consolidate Hyper-V features handling in hyperv_handle_properties().
The change is necessary to support 'hv-passthrough' mode as we'll be just
copying CPUIDs from KVM instead of filling them in.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-4-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 163 ++++++++++++++++++++++++++++++------------------------
1 file changed, 90 insertions(+), 73 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index cd492d4..a041b4d 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1013,13 +1013,25 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
return 0;
}
-static int hyperv_handle_properties(CPUState *cs)
+/*
+ * Fill in Hyper-V CPUIDs. Returns the number of entries filled in cpuid_ent in
+ * case of success, errno < 0 in case of failure and 0 when no Hyper-V
+ * extentions are enabled.
+ */
+static int hyperv_handle_properties(CPUState *cs,
+ struct kvm_cpuid_entry2 *cpuid_ent)
{
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
struct kvm_cpuid2 *cpuid;
+ struct kvm_cpuid_entry2 *c;
+ uint32_t signature[3];
+ uint32_t cpuid_i = 0;
int r = 0;
+ if (!hyperv_enabled(cpu))
+ return 0;
+
if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
uint16_t evmcs_version;
@@ -1068,9 +1080,80 @@ static int hyperv_handle_properties(CPUState *cs)
/* Not exposed by KVM but needed to make CPU hotplug in Windows work */
env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
+ if (r) {
+ r = -ENOSYS;
+ goto free;
+ }
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
+ if (!cpu->hyperv_vendor_id) {
+ memcpy(signature, "Microsoft Hv", 12);
+ } else {
+ size_t len = strlen(cpu->hyperv_vendor_id);
+
+ if (len > 12) {
+ error_report("hv-vendor-id truncated to 12 characters");
+ len = 12;
+ }
+ memset(signature, 0, 12);
+ memcpy(signature, cpu->hyperv_vendor_id, len);
+ }
+ c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
+ HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
+ c->ebx = signature[0];
+ c->ecx = signature[1];
+ c->edx = signature[2];
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_INTERFACE;
+ memcpy(signature, "Hv#1\0\0\0\0\0\0\0\0", 12);
+ c->eax = signature[0];
+ c->ebx = 0;
+ c->ecx = 0;
+ c->edx = 0;
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_VERSION;
+ c->eax = 0x00001bbc;
+ c->ebx = 0x00060001;
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_FEATURES;
+ c->eax = env->features[FEAT_HYPERV_EAX];
+ c->ebx = env->features[FEAT_HYPERV_EBX];
+ c->edx = env->features[FEAT_HYPERV_EDX];
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_ENLIGHTMENT_INFO;
+ c->eax = env->features[FEAT_HV_RECOMM_EAX];
+ c->ebx = cpu->hyperv_spinlock_attempts;
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_IMPLEMENT_LIMITS;
+ c->eax = cpu->hv_max_vps;
+ c->ebx = 0x40;
+
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
+ __u32 function;
+
+ /* Create zeroed 0x40000006..0x40000009 leaves */
+ for (function = HV_CPUID_IMPLEMENT_LIMITS + 1;
+ function < HV_CPUID_NESTED_FEATURES; function++) {
+ c = &cpuid_ent[cpuid_i++];
+ c->function = function;
+ }
+
+ c = &cpuid_ent[cpuid_i++];
+ c->function = HV_CPUID_NESTED_FEATURES;
+ c->eax = env->features[FEAT_HV_NESTED_EAX];
+ }
+ r = cpuid_i;
+
+free:
g_free(cpuid);
- return r ? -ENOSYS : 0;
+ return r;
}
static int hyperv_init_vcpu(X86CPU *cpu)
@@ -1179,79 +1262,13 @@ int kvm_arch_init_vcpu(CPUState *cs)
}
/* Paravirtualization CPUIDs */
- if (hyperv_enabled(cpu)) {
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
- if (!cpu->hyperv_vendor_id) {
- memcpy(signature, "Microsoft Hv", 12);
- } else {
- size_t len = strlen(cpu->hyperv_vendor_id);
-
- if (len > 12) {
- error_report("hv-vendor-id truncated to 12 characters");
- len = 12;
- }
- memset(signature, 0, 12);
- memcpy(signature, cpu->hyperv_vendor_id, len);
- }
- c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
- HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
- c->ebx = signature[0];
- c->ecx = signature[1];
- c->edx = signature[2];
-
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_INTERFACE;
- memcpy(signature, "Hv#1\0\0\0\0\0\0\0\0", 12);
- c->eax = signature[0];
- c->ebx = 0;
- c->ecx = 0;
- c->edx = 0;
-
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_VERSION;
- c->eax = 0x00001bbc;
- c->ebx = 0x00060001;
-
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_FEATURES;
- r = hyperv_handle_properties(cs);
- if (r) {
- return r;
- }
- c->eax = env->features[FEAT_HYPERV_EAX];
- c->ebx = env->features[FEAT_HYPERV_EBX];
- c->edx = env->features[FEAT_HYPERV_EDX];
-
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_ENLIGHTMENT_INFO;
-
- c->eax = env->features[FEAT_HV_RECOMM_EAX];
- c->ebx = cpu->hyperv_spinlock_attempts;
-
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_IMPLEMENT_LIMITS;
-
- c->eax = cpu->hv_max_vps;
- c->ebx = 0x40;
-
+ r = hyperv_handle_properties(cs, cpuid_data.entries);
+ if (r < 0) {
+ return r;
+ } else if (r > 0) {
+ cpuid_i = r;
kvm_base = KVM_CPUID_SIGNATURE_NEXT;
has_msr_hv_hypercall = true;
-
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
- __u32 function;
-
- /* Create zeroed 0x40000006..0x40000009 leaves */
- for (function = HV_CPUID_IMPLEMENT_LIMITS + 1;
- function < HV_CPUID_NESTED_FEATURES; function++) {
- c = &cpuid_data.entries[cpuid_i++];
- c->function = function;
- }
-
- c = &cpuid_data.entries[cpuid_i++];
- c->function = HV_CPUID_NESTED_FEATURES;
- c->eax = env->features[FEAT_HV_NESTED_EAX];
- }
}
if (cpu->expose_kvm) {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 06/25] i386/kvm: document existing Hyper-V enlightenments
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (4 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 05/25] i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties() Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 07/25] i386/kvm: implement 'hv-passthrough' mode Paolo Bonzini
` (20 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Currently, there is no doc describing hv-* CPU flags, people are
encouraged to get the information from Microsoft Hyper-V Top Level
Functional specification (TLFS). There is, however, a bit of QEMU
specifics.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-5-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
docs/hyperv.txt | 181 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 181 insertions(+)
create mode 100644 docs/hyperv.txt
diff --git a/docs/hyperv.txt b/docs/hyperv.txt
new file mode 100644
index 0000000..c423e0f
--- /dev/null
+++ b/docs/hyperv.txt
@@ -0,0 +1,181 @@
+Hyper-V Enlightenments
+======================
+
+
+1. Description
+===============
+In some cases when implementing a hardware interface in software is slow, KVM
+implements its own paravirtualized interfaces. This works well for Linux as
+guest support for such features is added simultaneously with the feature itself.
+It may, however, be hard-to-impossible to add support for these interfaces to
+proprietary OSes, namely, Microsoft Windows.
+
+KVM on x86 implements Hyper-V Enlightenments for Windows guests. These features
+make Windows and Hyper-V guests think they're running on top of a Hyper-V
+compatible hypervisor and use Hyper-V specific features.
+
+
+2. Setup
+=========
+No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In
+QEMU, individual enlightenments can be enabled through CPU flags, e.g:
+
+ qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ...
+
+Sometimes there are dependencies between enlightenments, QEMU is supposed to
+check that the supplied configuration is sane.
+
+When any set of the Hyper-V enlightenments is enabled, QEMU changes hypervisor
+identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identification
+and features are kept in leaves 0x40000100..0x40000101.
+
+
+3. Existing enlightenments
+===========================
+
+3.1. hv-relaxed
+================
+This feature tells guest OS to disable watchdog timeouts as it is running on a
+hypervisor. It is known that some Windows versions will do this even when they
+see 'hypervisor' CPU flag.
+
+3.2. hv-vapic
+==============
+Provides so-called VP Assist page MSR to guest allowing it to work with APIC
+more efficiently. In particular, this enlightenment allows paravirtualized
+(exit-less) EOI processing.
+
+3.3. hv-spinlocks=xxx
+======================
+Enables paravirtualized spinlocks. The parameter indicates how many times
+spinlock acquisition should be attempted before indicating the situation to the
+hypervisor. A special value 0xffffffff indicates "never to retry".
+
+3.4. hv-vpindex
+================
+Provides HV_X64_MSR_VP_INDEX (0x40000002) MSR to the guest which has Virtual
+processor index information. This enlightenment makes sense in conjunction with
+hv-synic, hv-stimer and other enlightenments which require the guest to know its
+Virtual Processor indices (e.g. when VP index needs to be passed in a
+hypercall).
+
+3.5. hv-runtime
+================
+Provides HV_X64_MSR_VP_RUNTIME (0x40000010) MSR to the guest. The MSR keeps the
+virtual processor run time in 100ns units. This gives guest operating system an
+idea of how much time was 'stolen' from it (when the virtual CPU was preempted
+to perform some other work).
+
+3.6. hv-crash
+==============
+Provides HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 (0x40000100..0x40000105) and
+HV_X64_MSR_CRASH_CTL (0x40000105) MSRs to the guest. These MSRs are written to
+by the guest when it crashes, HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 MSRs
+contain additional crash information. This information is outputted in QEMU log
+and through QAPI.
+Note: unlike under genuine Hyper-V, write to HV_X64_MSR_CRASH_CTL causes guest
+to shutdown. This effectively blocks crash dump generation by Windows.
+
+3.7. hv-time
+=============
+Enables two Hyper-V-specific clocksources available to the guest: MSR-based
+Hyper-V clocksource (HV_X64_MSR_TIME_REF_COUNT, 0x40000020) and Reference TSC
+page (enabled via MSR HV_X64_MSR_REFERENCE_TSC, 0x40000021). Both clocksources
+are per-guest, Reference TSC page clocksource allows for exit-less time stamp
+readings. Using this enlightenment leads to significant speedup of all timestamp
+related operations.
+
+3.8. hv-synic
+==============
+Enables Hyper-V Synthetic interrupt controller - an extension of a local APIC.
+When enabled, this enlightenment provides additional communication facilities
+to the guest: SynIC messages and Events. This is a pre-requisite for
+implementing VMBus devices (not yet in QEMU). Additionally, this enlightenment
+is needed to enable Hyper-V synthetic timers. SynIC is controlled through MSRs
+HV_X64_MSR_SCONTROL..HV_X64_MSR_EOM (0x40000080..0x40000084) and
+HV_X64_MSR_SINT0..HV_X64_MSR_SINT15 (0x40000090..0x4000009F)
+
+Requires: hv-vpindex
+
+3.9. hv-stimer
+===============
+Enables Hyper-V synthetic timers. There are four synthetic timers per virtual
+CPU controlled through HV_X64_MSR_STIMER0_CONFIG..HV_X64_MSR_STIMER3_COUNT
+(0x400000B0..0x400000B7) MSRs. These timers can work either in single-shot or
+periodic mode. It is known that certain Windows versions revert to using HPET
+(or even RTC when HPET is unavailable) extensively when this enlightenment is
+not provided; this can lead to significant CPU consumption, even when virtual
+CPU is idle.
+
+Requires: hv-vpindex, hv-synic, hv-time
+
+3.10. hv-tlbflush
+==================
+Enables paravirtualized TLB shoot-down mechanism. On x86 architecture, remote
+TLB flush procedure requires sending IPIs and waiting for other CPUs to perform
+local TLB flush. In virtualized environment some virtual CPUs may not even be
+scheduled at the time of the call and may not require flushing (or, flushing
+may be postponed until the virtual CPU is scheduled). hv-tlbflush enlightenment
+implements TLB shoot-down through hypervisor enabling the optimization.
+
+Requires: hv-vpindex
+
+3.11. hv-ipi
+=============
+Enables paravirtualized IPI send mechanism. HvCallSendSyntheticClusterIpi
+hypercall may target more than 64 virtual CPUs simultaneously, doing the same
+through APIC requires more than one access (and thus exit to the hypervisor).
+
+Requires: hv-vpindex
+
+3.12. hv-vendor-id=xxx
+=======================
+This changes Hyper-V identification in CPUID 0x40000000.EBX-EDX from the default
+"Microsoft Hv". The parameter should be no longer than 12 characters. According
+to the specification, guests shouldn't use this information and it is unknown
+if there is a Windows version which acts differently.
+Note: hv-vendor-id is not an enlightenment and thus doesn't enable Hyper-V
+identification when specified without some other enlightenment.
+
+3.13. hv-reset
+===============
+Provides HV_X64_MSR_RESET (0x40000003) MSR to the guest allowing it to reset
+itself by writing to it. Even when this MSR is enabled, it is not a recommended
+way for Windows to perform system reboot and thus it may not be used.
+
+3.14. hv-frequencies
+============================================
+Provides HV_X64_MSR_TSC_FREQUENCY (0x40000022) and HV_X64_MSR_APIC_FREQUENCY
+(0x40000023) allowing the guest to get its TSC/APIC frequencies without doing
+measurements.
+
+3.15 hv-reenlightenment
+========================
+The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+enabled, it provides HV_X64_MSR_REENLIGHTENMENT_CONTROL (0x40000106),
+HV_X64_MSR_TSC_EMULATION_CONTROL (0x40000107)and HV_X64_MSR_TSC_EMULATION_STATUS
+(0x40000108) MSRs allowing the guest to get notified when TSC frequency changes
+(only happens on migration) and keep using old frequency (through emulation in
+the hypervisor) until it is ready to switch to the new one. This, in conjunction
+with hv-frequencies, allows Hyper-V on KVM to pass stable clocksource (Reference
+TSC page) to its own guests.
+
+Recommended: hv-frequencies
+
+3.16. hv-evmcs
+===============
+The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+enabled, it provides Enlightened VMCS feature to the guest. The feature
+implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V)
+hypervisors making L2 exits to the hypervisor faster. The feature is Intel-only.
+Note: some virtualization features (e.g. Posted Interrupts) are disabled when
+hv-evmcs is enabled. It may make sense to measure your nested workload with and
+without the feature to find out if enabling it is beneficial.
+
+Requires: hv-vapic
+
+
+4. Useful links
+================
+Hyper-V Top Level Functional specification and other information:
+https://github.com/MicrosoftDocs/Virtualization-Documentation
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 07/25] i386/kvm: implement 'hv-passthrough' mode
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (5 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 06/25] i386/kvm: document existing Hyper-V enlightenments Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 08/25] i386/kvm: hv-stimer requires hv-time and hv-synic Paolo Bonzini
` (19 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
In many case we just want to give Windows guests all currently supported
Hyper-V enlightenments and that's where this new mode may come handy. We
pass through what was returned by KVM_GET_SUPPORTED_HV_CPUID.
hv_cpuid_check_and_set() is modified to also set cpu->hyperv_* flags as
we may want to check them later (and we actually do for hv_runtime,
hv_synic,...).
'hv-passthrough' is a development only feature, a migration blocker is
added to prevent issues while migrating between hosts with different
feature sets.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-6-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
docs/hyperv.txt | 10 +++++++
target/i386/cpu.c | 1 +
target/i386/cpu.h | 1 +
target/i386/kvm.c | 89 ++++++++++++++++++++++++++++++++++++++++++++++---------
4 files changed, 87 insertions(+), 14 deletions(-)
diff --git a/docs/hyperv.txt b/docs/hyperv.txt
index c423e0f..beadb2d 100644
--- a/docs/hyperv.txt
+++ b/docs/hyperv.txt
@@ -175,6 +175,16 @@ without the feature to find out if enabling it is beneficial.
Requires: hv-vapic
+4. Development features
+========================
+In some cases (e.g. during development) it may make sense to use QEMU in
+'pass-through' mode and give Windows guests all enlightenments currently
+supported by KVM. This pass-through mode is enabled by "hv-passthrough" CPU
+flag.
+Note: enabling this flag effectively prevents migration as supported features
+may differ between target and destination.
+
+
4. Useful links
================
Hyper-V Top Level Functional specification and other information:
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index e90c1ac..e07996c 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5883,6 +5883,7 @@ static Property x86_cpu_properties[] = {
HYPERV_FEAT_EVMCS, 0),
DEFINE_PROP_BIT64("hv-ipi", X86CPU, hyperv_features,
HYPERV_FEAT_IPI, 0),
+ DEFINE_PROP_BOOL("hv-passthrough", X86CPU, hyperv_passthrough, false),
DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true),
DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 30cd1a0..86edbf5 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1390,6 +1390,7 @@ struct X86CPU {
char *hyperv_vendor_id;
bool hyperv_synic_kvm_only;
uint64_t hyperv_features;
+ bool hyperv_passthrough;
bool check_cpuid;
bool enforce_cpuid;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index a041b4d..93ac6ba 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -639,7 +639,7 @@ static bool hyperv_enabled(X86CPU *cpu)
CPUState *cs = CPU(cpu);
return kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0 &&
((cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY) ||
- cpu->hyperv_features);
+ cpu->hyperv_features || cpu->hyperv_passthrough);
}
static int kvm_arch_set_tsc_khz(CPUState *cs)
@@ -985,10 +985,10 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
{
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
- uint32_t r, fw, bits;;
+ uint32_t r, fw, bits;
int i;
- if (!hyperv_feat_enabled(cpu, feature)) {
+ if (!hyperv_feat_enabled(cpu, feature) && !cpu->hyperv_passthrough) {
return 0;
}
@@ -1001,15 +1001,23 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
}
if (hv_cpuid_get_fw(cpuid, fw, &r) || (r & bits) != bits) {
- fprintf(stderr,
- "Hyper-V %s is not supported by kernel\n",
- kvm_hyperv_properties[feature].desc);
- return 1;
+ if (hyperv_feat_enabled(cpu, feature)) {
+ fprintf(stderr,
+ "Hyper-V %s is not supported by kernel\n",
+ kvm_hyperv_properties[feature].desc);
+ return 1;
+ } else {
+ return 0;
+ }
}
env->features[fw] |= bits;
}
+ if (cpu->hyperv_passthrough) {
+ cpu->hyperv_features |= BIT(feature);
+ }
+
return 0;
}
@@ -1027,22 +1035,29 @@ static int hyperv_handle_properties(CPUState *cs,
struct kvm_cpuid_entry2 *c;
uint32_t signature[3];
uint32_t cpuid_i = 0;
- int r = 0;
+ int r;
if (!hyperv_enabled(cpu))
return 0;
- if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ||
+ cpu->hyperv_passthrough) {
uint16_t evmcs_version;
- if (kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
- (uintptr_t)&evmcs_version)) {
+ r = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
+ (uintptr_t)&evmcs_version);
+
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) && r) {
fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
return -ENOSYS;
}
- env->features[FEAT_HV_RECOMM_EAX] |= HV_ENLIGHTENED_VMCS_RECOMMENDED;
- env->features[FEAT_HV_NESTED_EAX] = evmcs_version;
+
+ if (!r) {
+ env->features[FEAT_HV_RECOMM_EAX] |=
+ HV_ENLIGHTENED_VMCS_RECOMMENDED;
+ env->features[FEAT_HV_NESTED_EAX] = evmcs_version;
+ }
}
if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
@@ -1051,8 +1066,33 @@ static int hyperv_handle_properties(CPUState *cs,
cpuid = get_supported_hv_cpuid_legacy(cs);
}
+ if (cpu->hyperv_passthrough) {
+ memcpy(cpuid_ent, &cpuid->entries[0],
+ cpuid->nent * sizeof(cpuid->entries[0]));
+
+ c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
+ if (c) {
+ env->features[FEAT_HYPERV_EAX] = c->eax;
+ env->features[FEAT_HYPERV_EBX] = c->ebx;
+ env->features[FEAT_HYPERV_EDX] = c->eax;
+ }
+ c = cpuid_find_entry(cpuid, HV_CPUID_ENLIGHTMENT_INFO, 0);
+ if (c) {
+ env->features[FEAT_HV_RECOMM_EAX] = c->eax;
+
+ /* hv-spinlocks may have been overriden */
+ if (cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY) {
+ c->ebx = cpu->hyperv_spinlock_attempts;
+ }
+ }
+ c = cpuid_find_entry(cpuid, HV_CPUID_NESTED_FEATURES, 0);
+ if (c) {
+ env->features[FEAT_HV_NESTED_EAX] = c->eax;
+ }
+ }
+
/* Features */
- r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RELAXED);
+ r = hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RELAXED);
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VAPIC);
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TIME);
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_CRASH);
@@ -1085,6 +1125,12 @@ static int hyperv_handle_properties(CPUState *cs,
goto free;
}
+ if (cpu->hyperv_passthrough) {
+ /* We already copied all feature words from KVM as is */
+ r = cpuid->nent;
+ goto free;
+ }
+
c = &cpuid_ent[cpuid_i++];
c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
if (!cpu->hyperv_vendor_id) {
@@ -1156,11 +1202,26 @@ free:
return r;
}
+static Error *hv_passthrough_mig_blocker;
+
static int hyperv_init_vcpu(X86CPU *cpu)
{
CPUState *cs = CPU(cpu);
+ Error *local_err = NULL;
int ret;
+ if (cpu->hyperv_passthrough && hv_passthrough_mig_blocker == NULL) {
+ error_setg(&hv_passthrough_mig_blocker,
+ "'hv-passthrough' CPU flag prevents migration, use explicit"
+ " set of hv-* flags instead");
+ ret = migrate_add_blocker(hv_passthrough_mig_blocker, &local_err);
+ if (local_err) {
+ error_report_err(local_err);
+ error_free(hv_passthrough_mig_blocker);
+ return ret;
+ }
+ }
+
if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) && !hv_vpindex_settable) {
/*
* the kernel doesn't support setting vp_index; assert that its value
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 08/25] i386/kvm: hv-stimer requires hv-time and hv-synic
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (6 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 07/25] i386/kvm: implement 'hv-passthrough' mode Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 09/25] i386/kvm: hv-tlbflush/ipi require hv-vpindex Paolo Bonzini
` (18 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Synthetic timers operate in hv-time time and Windows won't use these
without SynIC.
Add .dependencies field to kvm_hyperv_properties[] and a generic mechanism
to check dependencies between features.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-7-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 23 +++++++++++++++++++----
1 file changed, 19 insertions(+), 4 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 93ac6ba..58afa31 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -689,6 +689,7 @@ static struct {
uint32_t fw;
uint32_t bits;
} flags[2];
+ uint64_t dependencies;
} kvm_hyperv_properties[] = {
[HYPERV_FEAT_RELAXED] = {
.desc = "relaxed timing (hv-relaxed)",
@@ -756,7 +757,8 @@ static struct {
.flags = {
{.fw = FEAT_HYPERV_EAX,
.bits = HV_SYNTIMERS_AVAILABLE}
- }
+ },
+ .dependencies = BIT(HYPERV_FEAT_SYNIC) | BIT(HYPERV_FEAT_TIME)
},
[HYPERV_FEAT_FREQUENCIES] = {
.desc = "frequency MSRs (hv-frequencies)",
@@ -986,12 +988,25 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
uint32_t r, fw, bits;
- int i;
+ uint64_t deps;
+ int i, dep_feat = 0;
if (!hyperv_feat_enabled(cpu, feature) && !cpu->hyperv_passthrough) {
return 0;
}
+ deps = kvm_hyperv_properties[feature].dependencies;
+ while ((dep_feat = find_next_bit(&deps, 64, dep_feat)) < 64) {
+ if (!(hyperv_feat_enabled(cpu, dep_feat))) {
+ fprintf(stderr,
+ "Hyper-V %s requires Hyper-V %s\n",
+ kvm_hyperv_properties[feature].desc,
+ kvm_hyperv_properties[dep_feat].desc);
+ return 1;
+ }
+ dep_feat++;
+ }
+
for (i = 0; i < ARRAY_SIZE(kvm_hyperv_properties[feature].flags); i++) {
fw = kvm_hyperv_properties[feature].flags[i].fw;
bits = kvm_hyperv_properties[feature].flags[i].bits;
@@ -1107,11 +1122,11 @@ static int hyperv_handle_properties(CPUState *cs,
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_EVMCS);
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_IPI);
- /* Dependencies */
+ /* Additional dependencies not covered by kvm_hyperv_properties[] */
if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC) &&
!cpu->hyperv_synic_kvm_only &&
!hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
- fprintf(stderr, "Hyper-V %s requires %s\n",
+ fprintf(stderr, "Hyper-V %s requires Hyper-V %s\n",
kvm_hyperv_properties[HYPERV_FEAT_SYNIC].desc,
kvm_hyperv_properties[HYPERV_FEAT_VPINDEX].desc);
r |= 1;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 09/25] i386/kvm: hv-tlbflush/ipi require hv-vpindex
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (7 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 08/25] i386/kvm: hv-stimer requires hv-time and hv-synic Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 10/25] i386/kvm: hv-evmcs requires hv-vapic Paolo Bonzini
` (17 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
The corresponding hypercalls require using VP indexes.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-8-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 58afa31..1dfa282 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -782,7 +782,8 @@ static struct {
{.fw = FEAT_HV_RECOMM_EAX,
.bits = HV_REMOTE_TLB_FLUSH_RECOMMENDED |
HV_EX_PROCESSOR_MASKS_RECOMMENDED}
- }
+ },
+ .dependencies = BIT(HYPERV_FEAT_VPINDEX)
},
[HYPERV_FEAT_EVMCS] = {
.desc = "enlightened VMCS (hv-evmcs)",
@@ -797,7 +798,8 @@ static struct {
{.fw = FEAT_HV_RECOMM_EAX,
.bits = HV_CLUSTER_IPI_RECOMMENDED |
HV_EX_PROCESSOR_MASKS_RECOMMENDED}
- }
+ },
+ .dependencies = BIT(HYPERV_FEAT_VPINDEX)
},
};
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 10/25] i386/kvm: hv-evmcs requires hv-vapic
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (8 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 09/25] i386/kvm: hv-tlbflush/ipi require hv-vpindex Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 11/25] i386/kvm: add support for Direct Mode for Hyper-V synthetic timers Paolo Bonzini
` (16 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Enlightened VMCS is enabled by writing to a field in VP assist page and
these require virtual APIC.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-9-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 1dfa282..86de510 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -790,7 +790,8 @@ static struct {
.flags = {
{.fw = FEAT_HV_RECOMM_EAX,
.bits = HV_ENLIGHTENED_VMCS_RECOMMENDED}
- }
+ },
+ .dependencies = BIT(HYPERV_FEAT_VAPIC)
},
[HYPERV_FEAT_IPI] = {
.desc = "paravirtualized IPI (hv-ipi)",
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 11/25] i386/kvm: add support for Direct Mode for Hyper-V synthetic timers
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (9 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 10/25] i386/kvm: hv-evmcs requires hv-vapic Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 12/25] target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY Paolo Bonzini
` (15 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Hyper-V on KVM can only use Synthetic timers with Direct Mode (opting for
an interrupt instead of VMBus message). This new capability is only
announced in KVM_GET_SUPPORTED_HV_CPUID.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-10-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
docs/hyperv.txt | 10 ++++++++++
target/i386/cpu.c | 2 ++
target/i386/cpu.h | 1 +
target/i386/hyperv-proto.h | 1 +
target/i386/kvm.c | 9 +++++++++
5 files changed, 23 insertions(+)
diff --git a/docs/hyperv.txt b/docs/hyperv.txt
index beadb2d..8fdf25c 100644
--- a/docs/hyperv.txt
+++ b/docs/hyperv.txt
@@ -174,6 +174,16 @@ without the feature to find out if enabling it is beneficial.
Requires: hv-vapic
+3.17. hv-stimer-direct
+=======================
+Hyper-V specification allows synthetic timer operation in two modes: "classic",
+when expiration event is delivered as SynIC message and "direct", when the event
+is delivered via normal interrupt. It is known that nested Hyper-V can only
+use synthetic timers in direct mode and thus 'hv-stimer-direct' needs to be
+enabled.
+
+Requires: hv-vpindex, hv-synic, hv-time, hv-stimer
+
4. Development features
========================
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index e07996c..7beb8ab 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5883,6 +5883,8 @@ static Property x86_cpu_properties[] = {
HYPERV_FEAT_EVMCS, 0),
DEFINE_PROP_BIT64("hv-ipi", X86CPU, hyperv_features,
HYPERV_FEAT_IPI, 0),
+ DEFINE_PROP_BIT64("hv-stimer-direct", X86CPU, hyperv_features,
+ HYPERV_FEAT_STIMER_DIRECT, 0),
DEFINE_PROP_BOOL("hv-passthrough", X86CPU, hyperv_passthrough, false),
DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 86edbf5..7470acf 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -749,6 +749,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
#define HYPERV_FEAT_TLBFLUSH 11
#define HYPERV_FEAT_EVMCS 12
#define HYPERV_FEAT_IPI 13
+#define HYPERV_FEAT_STIMER_DIRECT 14
#ifndef HYPERV_SPINLOCK_NEVER_RETRY
#define HYPERV_SPINLOCK_NEVER_RETRY 0xFFFFFFFF
diff --git a/target/i386/hyperv-proto.h b/target/i386/hyperv-proto.h
index c0272b3..cffac10 100644
--- a/target/i386/hyperv-proto.h
+++ b/target/i386/hyperv-proto.h
@@ -49,6 +49,7 @@
#define HV_GUEST_IDLE_STATE_AVAILABLE (1u << 5)
#define HV_FREQUENCY_MSRS_AVAILABLE (1u << 8)
#define HV_GUEST_CRASH_MSR_AVAILABLE (1u << 10)
+#define HV_STIMER_DIRECT_MODE_AVAILABLE (1u << 19)
/*
* HV_CPUID_ENLIGHTMENT_INFO.EAX bits
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 86de510..a323b1f 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -802,6 +802,14 @@ static struct {
},
.dependencies = BIT(HYPERV_FEAT_VPINDEX)
},
+ [HYPERV_FEAT_STIMER_DIRECT] = {
+ .desc = "direct mode synthetic timers (hv-stimer-direct)",
+ .flags = {
+ {.fw = FEAT_HYPERV_EDX,
+ .bits = HV_STIMER_DIRECT_MODE_AVAILABLE}
+ },
+ .dependencies = BIT(HYPERV_FEAT_STIMER)
+ },
};
static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max)
@@ -1124,6 +1132,7 @@ static int hyperv_handle_properties(CPUState *cs,
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TLBFLUSH);
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_EVMCS);
r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_IPI);
+ r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_STIMER_DIRECT);
/* Additional dependencies not covered by kvm_hyperv_properties[] */
if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC) &&
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 12/25] target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (10 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 11/25] i386/kvm: add support for Direct Mode for Hyper-V synthetic timers Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 13/25] target/i386: kvm: Delete VMX migration blocker on vCPU init failure Paolo Bonzini
` (14 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Xiaoyao Li
From: Xiaoyao Li <xiaoyao.li@linux.intel.com>
MSR IA32_CORE_CAPABILITY is a feature-enumerating MSR, which only
enumerates the feature split lock detection (via bit 5) by now.
The existence of MSR IA32_CORE_CAPABILITY is enumerated by CPUID.7_0:EDX[30].
The latest kernel patches about them can be found here:
https://lkml.org/lkml/2019/4/24/1909
Signed-off-by: Xiaoyao Li <xiaoyao.li@linux.intel.com>
Message-Id: <20190617153654.916-1-xiaoyao.li@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/cpu.c | 22 +++++++++++++++++++++-
target/i386/cpu.h | 5 +++++
target/i386/kvm.c | 9 +++++++++
3 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 7beb8ab..c330fd9 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1085,7 +1085,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
NULL, NULL, NULL, NULL,
NULL, NULL, NULL, NULL,
NULL, NULL, "spec-ctrl", "stibp",
- NULL, "arch-capabilities", NULL, "ssbd",
+ NULL, "arch-capabilities", "core-capability", "ssbd",
},
.cpuid = {
.eax = 7,
@@ -1203,6 +1203,26 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
}
},
},
+ [FEAT_CORE_CAPABILITY] = {
+ .type = MSR_FEATURE_WORD,
+ .feat_names = {
+ NULL, NULL, NULL, NULL,
+ NULL, "split-lock-detect", NULL, NULL,
+ NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
+ },
+ .msr = {
+ .index = MSR_IA32_CORE_CAPABILITY,
+ .cpuid_dep = {
+ FEAT_7_0_EDX,
+ CPUID_7_0_EDX_CORE_CAPABILITY,
+ },
+ },
+ },
};
typedef struct X86RegisterInfo32 {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 7470acf..7f48136 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -345,6 +345,7 @@ typedef enum X86Seg {
#define MSR_IA32_SPEC_CTRL 0x48
#define MSR_VIRT_SSBD 0xc001011f
#define MSR_IA32_PRED_CMD 0x49
+#define MSR_IA32_CORE_CAPABILITY 0xcf
#define MSR_IA32_ARCH_CAPABILITIES 0x10a
#define MSR_IA32_TSCDEADLINE 0x6e0
@@ -496,6 +497,7 @@ typedef enum FeatureWord {
FEAT_XSAVE_COMP_LO, /* CPUID[EAX=0xd,ECX=0].EAX */
FEAT_XSAVE_COMP_HI, /* CPUID[EAX=0xd,ECX=0].EDX */
FEAT_ARCH_CAPABILITIES,
+ FEAT_CORE_CAPABILITY,
FEATURE_WORDS,
} FeatureWord;
@@ -687,6 +689,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
#define CPUID_7_0_EDX_AVX512_4FMAPS (1U << 3) /* AVX512 Multiply Accumulation Single Precision */
#define CPUID_7_0_EDX_SPEC_CTRL (1U << 26) /* Speculation Control */
#define CPUID_7_0_EDX_ARCH_CAPABILITIES (1U << 29) /*Arch Capabilities*/
+#define CPUID_7_0_EDX_CORE_CAPABILITY (1U << 30) /*Core Capability*/
#define CPUID_7_0_EDX_SPEC_CTRL_SSBD (1U << 31) /* Speculative Store Bypass Disable */
#define CPUID_8000_0008_EBX_WBNOINVD (1U << 9) /* Write back and
@@ -734,6 +737,8 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
#define MSR_ARCH_CAP_SKIP_L1DFL_VMENTRY (1U << 3)
#define MSR_ARCH_CAP_SSB_NO (1U << 4)
+#define MSR_CORE_CAP_SPLIT_LOCK_DETECT (1U << 5)
+
/* Supported Hyper-V Enlightenments */
#define HYPERV_FEAT_RELAXED 0
#define HYPERV_FEAT_VAPIC 1
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index a323b1f..279f99a 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -95,6 +95,7 @@ static bool has_msr_spec_ctrl;
static bool has_msr_virt_ssbd;
static bool has_msr_smi_count;
static bool has_msr_arch_capabs;
+static bool has_msr_core_capabs;
static uint32_t has_architectural_pmu_version;
static uint32_t num_architectural_pmu_gp_counters;
@@ -1842,6 +1843,9 @@ static int kvm_get_supported_msrs(KVMState *s)
case MSR_IA32_ARCH_CAPABILITIES:
has_msr_arch_capabs = true;
break;
+ case MSR_IA32_CORE_CAPABILITY:
+ has_msr_core_capabs = true;
+ break;
}
}
}
@@ -2368,6 +2372,11 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
env->features[FEAT_ARCH_CAPABILITIES]);
}
+ if (has_msr_core_capabs) {
+ kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
+ env->features[FEAT_CORE_CAPABILITY]);
+ }
+
/*
* The following MSRs have side effects on the guest or are too heavy
* for normal writeback. Limit them to reset or full state updates.
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 13/25] target/i386: kvm: Delete VMX migration blocker on vCPU init failure
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (11 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 12/25] target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 14/25] KVM: Introduce kvm_arch_destroy_vcpu() Paolo Bonzini
` (13 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker")
added migration blocker for vCPU exposed with Intel VMX because QEMU
doesn't yet contain code to support migration of nested virtualization
workloads.
However, that commit missed adding deletion of the migration blocker in
case init of vCPU failed. Similar to invtsc_mig_blocker. This commit fix
that issue.
Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker")
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Message-Id: <20190619162140.133674-2-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 279f99a..c5cbead 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1333,7 +1333,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
r = kvm_arch_set_tsc_khz(cs);
if (r < 0) {
- goto fail;
+ return r;
}
/* vcpu's TSC frequency is either specified by user, or following
@@ -1622,7 +1622,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
if (local_err) {
error_report_err(local_err);
error_free(invtsc_mig_blocker);
- return r;
+ goto fail2;
}
}
}
@@ -1673,6 +1673,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
fail:
migrate_del_blocker(invtsc_mig_blocker);
+ fail2:
+ migrate_del_blocker(vmx_mig_blocker);
+
return r;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 14/25] KVM: Introduce kvm_arch_destroy_vcpu()
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (12 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 13/25] target/i386: kvm: Delete VMX migration blocker on vCPU init failure Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 15/25] target/i386: kvm: Use symbolic constant for #DB/#BP exception constants Paolo Bonzini
` (12 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Simiar to how kvm_init_vcpu() calls kvm_arch_init_vcpu() to perform
arch-dependent initialisation, introduce kvm_arch_destroy_vcpu()
to be called from kvm_destroy_vcpu() to perform arch-dependent
destruction.
This was added because some architectures (Such as i386)
currently do not free memory that it have allocated in
kvm_arch_init_vcpu().
Suggested-by: Maran Wilson <maran.wilson@oracle.com>
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20190619162140.133674-3-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 5 +++++
include/sysemu/kvm.h | 1 +
target/arm/kvm32.c | 5 +++++
target/arm/kvm64.c | 5 +++++
target/i386/kvm.c | 12 ++++++++++++
target/mips/kvm.c | 5 +++++
target/ppc/kvm.c | 5 +++++
target/s390x/kvm.c | 10 ++++++++++
8 files changed, 48 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index d2f481a..f0f5ab8 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -291,6 +291,11 @@ int kvm_destroy_vcpu(CPUState *cpu)
DPRINTF("kvm_destroy_vcpu\n");
+ ret = kvm_arch_destroy_vcpu(cpu);
+ if (ret < 0) {
+ goto err;
+ }
+
mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
if (mmap_size < 0) {
ret = mmap_size;
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index a6d1cd1..64f55e5 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -371,6 +371,7 @@ int kvm_arch_put_registers(CPUState *cpu, int level);
int kvm_arch_init(MachineState *ms, KVMState *s);
int kvm_arch_init_vcpu(CPUState *cpu);
+int kvm_arch_destroy_vcpu(CPUState *cpu);
bool kvm_vcpu_id_is_valid(int vcpu_id);
diff --git a/target/arm/kvm32.c b/target/arm/kvm32.c
index 4e54e37..51f78f7 100644
--- a/target/arm/kvm32.c
+++ b/target/arm/kvm32.c
@@ -240,6 +240,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
return kvm_arm_init_cpreg_list(cpu);
}
+int kvm_arch_destroy_vcpu(CPUState *cs)
+{
+ return 0;
+}
+
typedef struct Reg {
uint64_t id;
int offset;
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index 998d21f..22d19c9 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -654,6 +654,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
return kvm_arm_init_cpreg_list(cpu);
}
+int kvm_arch_destroy_vcpu(CPUState *cs)
+{
+ return 0;
+}
+
bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t regidx)
{
/* Return true if the regidx is a register we should synchronize
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index c5cbead..26d8c61 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1679,6 +1679,18 @@ int kvm_arch_init_vcpu(CPUState *cs)
return r;
}
+int kvm_arch_destroy_vcpu(CPUState *cs)
+{
+ X86CPU *cpu = X86_CPU(cs);
+
+ if (cpu->kvm_msr_buf) {
+ g_free(cpu->kvm_msr_buf);
+ cpu->kvm_msr_buf = NULL;
+ }
+
+ return 0;
+}
+
void kvm_arch_reset_vcpu(X86CPU *cpu)
{
CPUX86State *env = &cpu->env;
diff --git a/target/mips/kvm.c b/target/mips/kvm.c
index 8e72850..938f8f1 100644
--- a/target/mips/kvm.c
+++ b/target/mips/kvm.c
@@ -91,6 +91,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
return ret;
}
+int kvm_arch_destroy_vcpu(CPUState *cs)
+{
+ return 0;
+}
+
void kvm_mips_reset_vcpu(MIPSCPU *cpu)
{
CPUMIPSState *env = &cpu->env;
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index d4107dd..4b4989c 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -521,6 +521,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
return ret;
}
+int kvm_arch_destroy_vcpu(CPUState *cs)
+{
+ return 0;
+}
+
static void kvm_sw_tlb_put(PowerPCCPU *cpu)
{
CPUPPCState *env = &cpu->env;
diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index bcec979..0267c6c 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -368,6 +368,16 @@ int kvm_arch_init_vcpu(CPUState *cs)
return 0;
}
+int kvm_arch_destroy_vcpu(CPUState *cs)
+{
+ S390CPU *cpu = S390_CPU(cs);
+
+ g_free(cpu->irqstate);
+ cpu->irqstate = NULL;
+
+ return 0;
+}
+
void kvm_s390_reset_vcpu(S390CPU *cpu)
{
CPUState *cs = CPU(cpu);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 15/25] target/i386: kvm: Use symbolic constant for #DB/#BP exception constants
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (13 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 14/25] KVM: Introduce kvm_arch_destroy_vcpu() Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 16/25] target/i386: kvm: Re-inject #DB to guest with updated DR6 Paolo Bonzini
` (11 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Krish Sadhukhan <krish.sadhukhan@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20190619162140.133674-4-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 26d8c61..c763643 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -3345,9 +3345,9 @@ static int kvm_guest_debug_workarounds(X86CPU *cpu)
unsigned long reinject_trap = 0;
if (!kvm_has_vcpu_events()) {
- if (env->exception_injected == 1) {
+ if (env->exception_injected == EXCP01_DB) {
reinject_trap = KVM_GUESTDBG_INJECT_DB;
- } else if (env->exception_injected == 3) {
+ } else if (env->exception_injected == EXCP03_INT3) {
reinject_trap = KVM_GUESTDBG_INJECT_BP;
}
env->exception_injected = -1;
@@ -3859,8 +3859,8 @@ static int kvm_handle_debug(X86CPU *cpu,
int ret = 0;
int n;
- if (arch_info->exception == 1) {
- if (arch_info->dr6 & (1 << 14)) {
+ if (arch_info->exception == EXCP01_DB) {
+ if (arch_info->dr6 & DR6_BS) {
if (cs->singlestep_enabled) {
ret = EXCP_DEBUG;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 16/25] target/i386: kvm: Re-inject #DB to guest with updated DR6
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (14 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 15/25] target/i386: kvm: Use symbolic constant for #DB/#BP exception constants Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization Paolo Bonzini
` (10 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
If userspace (QEMU) debug guest, when #DB is raised in guest and
intercepted by KVM, KVM forwards information on #DB to userspace
instead of injecting #DB to guest.
While doing so, KVM don't update vCPU DR6 but instead report the #DB DR6
value to userspace for further handling.
See KVM's handle_exception() DB_VECTOR handler.
QEMU handler for this case is kvm_handle_debug(). This handler basically
checks if #DB is related to one of user set hardware breakpoints and if
not, it re-inject #DB into guest.
The re-injection is done by setting env->exception_injected to #DB which
will later be passed as events.exception.nr to KVM_SET_VCPU_EVENTS ioctl
by kvm_put_vcpu_events().
However, in case userspace re-injects #DB, KVM expects userspace to set
vCPU DR6 as reported to userspace when #DB was intercepted! Otherwise,
KVM_REQ_EVENT handler will inject #DB with wrong DR6 to guest.
Fix this issue by updating vCPU DR6 appropriately when re-inject #DB to
guest.
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Krish Sadhukhan <krish.sadhukhan@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20190619162140.133674-5-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index c763643..9864aa0 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -3897,6 +3897,9 @@ static int kvm_handle_debug(X86CPU *cpu,
/* pass to guest */
env->exception_injected = arch_info->exception;
env->has_error_code = 0;
+ if (arch_info->exception == EXCP01_DB) {
+ env->dr[6] = arch_info->dr6;
+ }
}
return ret;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (15 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 16/25] target/i386: kvm: Re-inject #DB to guest with updated DR6 Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-07-08 18:21 ` Jan Kiszka
2019-06-21 11:30 ` [Qemu-devel] [PULL 18/25] linux-headers: sync with latest KVM headers from Linux 5.2 Paolo Bonzini
` (9 subsequent siblings)
26 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker")
added a migration blocker for vCPU exposed with Intel VMX.
However, migration should also be blocked for vCPU exposed with
AMD SVM.
Both cases should be blocked because QEMU should extract additional
vCPU state from KVM that should be migrated as part of vCPU VMState.
E.g. Whether vCPU is running in guest-mode or host-mode.
Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker")
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20190619162140.133674-6-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/cpu.c | 6 ------
target/i386/cpu.h | 12 ++++++++++++
target/i386/kvm.c | 14 +++++++-------
3 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index c330fd9..61e44cb 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5215,12 +5215,6 @@ static int x86_cpu_filter_features(X86CPU *cpu)
return rv;
}
-#define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \
- (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \
- (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3)
-#define IS_AMD_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_AMD_1 && \
- (env)->cpuid_vendor2 == CPUID_VENDOR_AMD_2 && \
- (env)->cpuid_vendor3 == CPUID_VENDOR_AMD_3)
static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
{
CPUState *cs = CPU(dev);
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 7f48136..bf0c9c2 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -722,6 +722,13 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
#define CPUID_VENDOR_HYGON "HygonGenuine"
+#define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \
+ (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \
+ (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3)
+#define IS_AMD_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_AMD_1 && \
+ (env)->cpuid_vendor2 == CPUID_VENDOR_AMD_2 && \
+ (env)->cpuid_vendor3 == CPUID_VENDOR_AMD_3)
+
#define CPUID_MWAIT_IBE (1U << 1) /* Interrupts can exit capability */
#define CPUID_MWAIT_EMX (1U << 0) /* enumeration supported */
@@ -1848,6 +1855,11 @@ static inline int32_t x86_get_a20_mask(CPUX86State *env)
}
}
+static inline bool cpu_has_vmx(CPUX86State *env)
+{
+ return env->features[FEAT_1_ECX] & CPUID_EXT_VMX;
+}
+
/* fpu_helper.c */
void update_fp_status(CPUX86State *env);
void update_mxcsr_status(CPUX86State *env);
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 9864aa0..f9872f1 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1299,7 +1299,7 @@ static int hyperv_init_vcpu(X86CPU *cpu)
}
static Error *invtsc_mig_blocker;
-static Error *vmx_mig_blocker;
+static Error *nested_virt_mig_blocker;
#define KVM_MAX_CPUID_ENTRIES 100
@@ -1597,13 +1597,13 @@ int kvm_arch_init_vcpu(CPUState *cs)
!!(c->ecx & CPUID_EXT_SMX);
}
- if ((env->features[FEAT_1_ECX] & CPUID_EXT_VMX) && !vmx_mig_blocker) {
- error_setg(&vmx_mig_blocker,
- "Nested VMX virtualization does not support live migration yet");
- r = migrate_add_blocker(vmx_mig_blocker, &local_err);
+ if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
+ error_setg(&nested_virt_mig_blocker,
+ "Nested virtualization does not support live migration yet");
+ r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
if (local_err) {
error_report_err(local_err);
- error_free(vmx_mig_blocker);
+ error_free(nested_virt_mig_blocker);
return r;
}
}
@@ -1674,7 +1674,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
fail:
migrate_del_blocker(invtsc_mig_blocker);
fail2:
- migrate_del_blocker(vmx_mig_blocker);
+ migrate_del_blocker(nested_virt_mig_blocker);
return r;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 18/25] linux-headers: sync with latest KVM headers from Linux 5.2
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (16 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 19/25] vmstate: Add support for kernel integer types Paolo Bonzini
` (8 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Improve the KVM_{GET,SET}_NESTED_STATE structs by detailing the format
of VMX nested state data in a struct.
In order to avoid changing the ioctl values of
KVM_{GET,SET}_NESTED_STATE, there is a need to preserve
sizeof(struct kvm_nested_state). This is done by defining the data
struct as "data.vmx[0]". It was the most elegant way I found to
preserve struct size while still keeping struct readable and easy to
maintain. It does have a misfortunate side-effect that now it has to be
accessed as "data.vmx[0]" rather than just "data.vmx".
Because we are already modifying these structs, I also modified the
following:
* Define the "format" field values as macros.
* Rename vmcs_pa to vmcs12_pa for better readability.
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Message-Id: <20190619162140.133674-7-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
| 33 ++++++++++++++++++++++-----------
1 file changed, 22 insertions(+), 11 deletions(-)
--git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index 7a0e64c..6e7dd79 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -383,16 +383,26 @@ struct kvm_sync_regs {
#define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2)
#define KVM_X86_QUIRK_OUT_7E_INC_RIP (1 << 3)
+#define KVM_STATE_NESTED_FORMAT_VMX 0
+#define KVM_STATE_NESTED_FORMAT_SVM 1
+
#define KVM_STATE_NESTED_GUEST_MODE 0x00000001
#define KVM_STATE_NESTED_RUN_PENDING 0x00000002
#define KVM_STATE_NESTED_EVMCS 0x00000004
+#define KVM_STATE_NESTED_VMX_VMCS_SIZE 0x1000
+
#define KVM_STATE_NESTED_SMM_GUEST_MODE 0x00000001
#define KVM_STATE_NESTED_SMM_VMXON 0x00000002
-struct kvm_vmx_nested_state {
+struct kvm_vmx_nested_state_data {
+ __u8 vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE];
+ __u8 shadow_vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE];
+};
+
+struct kvm_vmx_nested_state_hdr {
__u64 vmxon_pa;
- __u64 vmcs_pa;
+ __u64 vmcs12_pa;
struct {
__u16 flags;
@@ -401,24 +411,25 @@ struct kvm_vmx_nested_state {
/* for KVM_CAP_NESTED_STATE */
struct kvm_nested_state {
- /* KVM_STATE_* flags */
__u16 flags;
-
- /* 0 for VMX, 1 for SVM. */
__u16 format;
-
- /* 128 for SVM, 128 + VMCS size for VMX. */
__u32 size;
union {
- /* VMXON, VMCS */
- struct kvm_vmx_nested_state vmx;
+ struct kvm_vmx_nested_state_hdr vmx;
/* Pad the header to 128 bytes. */
__u8 pad[120];
- };
+ } hdr;
- __u8 data[0];
+ /*
+ * Define data region as 0 bytes to preserve backwards-compatability
+ * to old definition of kvm_nested_state in order to avoid changing
+ * KVM_{GET,PUT}_NESTED_STATE ioctl values.
+ */
+ union {
+ struct kvm_vmx_nested_state_data vmx[0];
+ } data;
};
#endif /* _ASM_X86_KVM_H */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 19/25] vmstate: Add support for kernel integer types
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (17 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 18/25] linux-headers: sync with latest KVM headers from Linux 5.2 Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state Paolo Bonzini
` (7 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190619162140.133674-8-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
include/migration/vmstate.h | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 9224370..ca68584 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -797,6 +797,19 @@ extern const VMStateInfo vmstate_info_qtailq;
#define VMSTATE_UINT64_V(_f, _s, _v) \
VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint64, uint64_t)
+#ifdef CONFIG_LINUX
+
+#define VMSTATE_U8_V(_f, _s, _v) \
+ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint8, __u8)
+#define VMSTATE_U16_V(_f, _s, _v) \
+ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint16, __u16)
+#define VMSTATE_U32_V(_f, _s, _v) \
+ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint32, __u32)
+#define VMSTATE_U64_V(_f, _s, _v) \
+ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint64, __u64)
+
+#endif
+
#define VMSTATE_BOOL(_f, _s) \
VMSTATE_BOOL_V(_f, _s, 0)
@@ -818,6 +831,19 @@ extern const VMStateInfo vmstate_info_qtailq;
#define VMSTATE_UINT64(_f, _s) \
VMSTATE_UINT64_V(_f, _s, 0)
+#ifdef CONFIG_LINUX
+
+#define VMSTATE_U8(_f, _s) \
+ VMSTATE_U8_V(_f, _s, 0)
+#define VMSTATE_U16(_f, _s) \
+ VMSTATE_U16_V(_f, _s, 0)
+#define VMSTATE_U32(_f, _s) \
+ VMSTATE_U32_V(_f, _s, 0)
+#define VMSTATE_U64(_f, _s) \
+ VMSTATE_U64_V(_f, _s, 0)
+
+#endif
+
#define VMSTATE_UINT8_EQUAL(_f, _s, _err_hint) \
VMSTATE_SINGLE_FULL(_f, _s, 0, 0, \
vmstate_info_uint8_equal, uint8_t, _err_hint)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (18 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 19/25] vmstate: Add support for kernel integer types Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 12:29 ` Liran Alon
2019-06-21 11:30 ` [Qemu-devel] [PULL 21/25] target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD Paolo Bonzini
` (6 subsequent siblings)
26 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Kernel commit 8fcc4b5923af ("kvm: nVMX: Introduce KVM_CAP_NESTED_STATE")
introduced new IOCTLs to extract and restore vCPU state related to
Intel VMX & AMD SVM.
Utilize these IOCTLs to add support for migration of VMs which are
running nested hypervisors.
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Tested-by: Maran Wilson <maran.wilson@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20190619162140.133674-9-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 8 ++
include/sysemu/kvm.h | 1 +
target/i386/cpu.h | 3 +
target/i386/kvm.c | 80 ++++++++++++++++++++
target/i386/machine.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 290 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index f0f5ab8..e3cf728 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -87,6 +87,7 @@ struct KVMState
#ifdef KVM_CAP_SET_GUEST_DEBUG
QTAILQ_HEAD(, kvm_sw_breakpoint) kvm_sw_breakpoints;
#endif
+ int max_nested_state_len;
int many_ioeventfds;
int intx_set_mask;
bool sync_mmu;
@@ -1681,6 +1682,8 @@ static int kvm_init(MachineState *ms)
s->debugregs = kvm_check_extension(s, KVM_CAP_DEBUGREGS);
#endif
+ s->max_nested_state_len = kvm_check_extension(s, KVM_CAP_NESTED_STATE);
+
#ifdef KVM_CAP_IRQ_ROUTING
kvm_direct_msi_allowed = (kvm_check_extension(s, KVM_CAP_SIGNAL_MSI) > 0);
#endif
@@ -2248,6 +2251,11 @@ int kvm_has_debugregs(void)
return kvm_state->debugregs;
}
+int kvm_max_nested_state_length(void)
+{
+ return kvm_state->max_nested_state_len;
+}
+
int kvm_has_many_ioeventfds(void)
{
if (!kvm_enabled()) {
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 64f55e5..acd90ae 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -210,6 +210,7 @@ bool kvm_has_sync_mmu(void);
int kvm_has_vcpu_events(void);
int kvm_has_robust_singlestep(void);
int kvm_has_debugregs(void);
+int kvm_max_nested_state_length(void);
int kvm_has_pit_state2(void);
int kvm_has_many_ioeventfds(void);
int kvm_has_gsi_routing(void);
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index bf0c9c2..17116ef 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1360,6 +1360,9 @@ typedef struct CPUX86State {
#if defined(CONFIG_KVM) || defined(CONFIG_HVF)
void *xsave_buf;
#endif
+#if defined(CONFIG_KVM)
+ struct kvm_nested_state *nested_state;
+#endif
#if defined(CONFIG_HVF)
HVFX86EmulatorState *hvf_emul;
#endif
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index f9872f1..e924663 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1324,6 +1324,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
struct kvm_cpuid_entry2 *c;
uint32_t signature[3];
int kvm_base = KVM_CPUID_SIGNATURE;
+ int max_nested_state_len;
int r;
Error *local_err = NULL;
@@ -1658,6 +1659,24 @@ int kvm_arch_init_vcpu(CPUState *cs)
if (has_xsave) {
env->xsave_buf = qemu_memalign(4096, sizeof(struct kvm_xsave));
}
+
+ max_nested_state_len = kvm_max_nested_state_length();
+ if (max_nested_state_len > 0) {
+ assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data));
+ env->nested_state = g_malloc0(max_nested_state_len);
+
+ env->nested_state->size = max_nested_state_len;
+
+ if (IS_INTEL_CPU(env)) {
+ struct kvm_vmx_nested_state_hdr *vmx_hdr =
+ &env->nested_state->hdr.vmx;
+
+ env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
+ vmx_hdr->vmxon_pa = -1ull;
+ vmx_hdr->vmcs12_pa = -1ull;
+ }
+ }
+
cpu->kvm_msr_buf = g_malloc0(MSR_BUF_SIZE);
if (!(env->features[FEAT_8000_0001_EDX] & CPUID_EXT2_RDTSCP)) {
@@ -1682,12 +1701,18 @@ int kvm_arch_init_vcpu(CPUState *cs)
int kvm_arch_destroy_vcpu(CPUState *cs)
{
X86CPU *cpu = X86_CPU(cs);
+ CPUX86State *env = &cpu->env;
if (cpu->kvm_msr_buf) {
g_free(cpu->kvm_msr_buf);
cpu->kvm_msr_buf = NULL;
}
+ if (env->nested_state) {
+ g_free(env->nested_state);
+ env->nested_state = NULL;
+ }
+
return 0;
}
@@ -3411,6 +3436,52 @@ static int kvm_get_debugregs(X86CPU *cpu)
return 0;
}
+static int kvm_put_nested_state(X86CPU *cpu)
+{
+ CPUX86State *env = &cpu->env;
+ int max_nested_state_len = kvm_max_nested_state_length();
+
+ if (max_nested_state_len <= 0) {
+ return 0;
+ }
+
+ assert(env->nested_state->size <= max_nested_state_len);
+ return kvm_vcpu_ioctl(CPU(cpu), KVM_SET_NESTED_STATE, env->nested_state);
+}
+
+static int kvm_get_nested_state(X86CPU *cpu)
+{
+ CPUX86State *env = &cpu->env;
+ int max_nested_state_len = kvm_max_nested_state_length();
+ int ret;
+
+ if (max_nested_state_len <= 0) {
+ return 0;
+ }
+
+ /*
+ * It is possible that migration restored a smaller size into
+ * nested_state->hdr.size than what our kernel support.
+ * We preserve migration origin nested_state->hdr.size for
+ * call to KVM_SET_NESTED_STATE but wish that our next call
+ * to KVM_GET_NESTED_STATE will use max size our kernel support.
+ */
+ env->nested_state->size = max_nested_state_len;
+
+ ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_NESTED_STATE, env->nested_state);
+ if (ret < 0) {
+ return ret;
+ }
+
+ if (env->nested_state->flags & KVM_STATE_NESTED_GUEST_MODE) {
+ env->hflags |= HF_GUEST_MASK;
+ } else {
+ env->hflags &= ~HF_GUEST_MASK;
+ }
+
+ return ret;
+}
+
int kvm_arch_put_registers(CPUState *cpu, int level)
{
X86CPU *x86_cpu = X86_CPU(cpu);
@@ -3418,6 +3489,11 @@ int kvm_arch_put_registers(CPUState *cpu, int level)
assert(cpu_is_stopped(cpu) || qemu_cpu_is_self(cpu));
+ ret = kvm_put_nested_state(x86_cpu);
+ if (ret < 0) {
+ return ret;
+ }
+
if (level >= KVM_PUT_RESET_STATE) {
ret = kvm_put_msr_feature_control(x86_cpu);
if (ret < 0) {
@@ -3533,6 +3609,10 @@ int kvm_arch_get_registers(CPUState *cs)
if (ret < 0) {
goto out;
}
+ ret = kvm_get_nested_state(cpu);
+ if (ret < 0) {
+ goto out;
+ }
ret = 0;
out:
cpu_sync_bndcs_hflags(&cpu->env);
diff --git a/target/i386/machine.c b/target/i386/machine.c
index a39ce7f..a6afdf8 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -231,6 +231,15 @@ static int cpu_pre_save(void *opaque)
env->segs[R_SS].flags &= ~(env->segs[R_SS].flags & DESC_DPL_MASK);
}
+#ifdef CONFIG_KVM
+ /* Verify we have nested virtualization state from kernel if required */
+ if (cpu_has_nested_virt(env) && !env->nested_state) {
+ error_report("Guest enabled nested virtualization but kernel "
+ "does not support saving of nested state");
+ return -EINVAL;
+ }
+#endif
+
return 0;
}
@@ -278,6 +287,16 @@ static int cpu_post_load(void *opaque, int version_id)
env->hflags &= ~HF_CPL_MASK;
env->hflags |= (env->segs[R_SS].flags >> DESC_DPL_SHIFT) & HF_CPL_MASK;
+#ifdef CONFIG_KVM
+ if ((env->hflags & HF_GUEST_MASK) &&
+ (!env->nested_state ||
+ !(env->nested_state->flags & KVM_STATE_NESTED_GUEST_MODE))) {
+ error_report("vCPU set in guest-mode inconsistent with "
+ "migrated kernel nested state");
+ return -EINVAL;
+ }
+#endif
+
env->fpstt = (env->fpus_vmstate >> 11) & 7;
env->fpus = env->fpus_vmstate & ~0x3800;
env->fptag_vmstate ^= 0xff;
@@ -851,6 +870,182 @@ static const VMStateDescription vmstate_tsc_khz = {
}
};
+#ifdef CONFIG_KVM
+
+static bool vmx_vmcs12_needed(void *opaque)
+{
+ struct kvm_nested_state *nested_state = opaque;
+ return (nested_state->size >
+ offsetof(struct kvm_nested_state, data.vmx[0].vmcs12));
+}
+
+static const VMStateDescription vmstate_vmx_vmcs12 = {
+ .name = "cpu/kvm_nested_state/vmx/vmcs12",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = vmx_vmcs12_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_UINT8_ARRAY(data.vmx[0].vmcs12,
+ struct kvm_nested_state,
+ KVM_STATE_NESTED_VMX_VMCS_SIZE),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
+static bool vmx_shadow_vmcs12_needed(void *opaque)
+{
+ struct kvm_nested_state *nested_state = opaque;
+ return (nested_state->size >
+ offsetof(struct kvm_nested_state, data.vmx[0].shadow_vmcs12));
+}
+
+static const VMStateDescription vmstate_vmx_shadow_vmcs12 = {
+ .name = "cpu/kvm_nested_state/vmx/shadow_vmcs12",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = vmx_shadow_vmcs12_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_UINT8_ARRAY(data.vmx[0].shadow_vmcs12,
+ struct kvm_nested_state,
+ KVM_STATE_NESTED_VMX_VMCS_SIZE),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
+static bool vmx_nested_state_needed(void *opaque)
+{
+ struct kvm_nested_state *nested_state = opaque;
+
+ return ((nested_state->format == KVM_STATE_NESTED_FORMAT_VMX) &&
+ ((nested_state->hdr.vmx.vmxon_pa != -1ull) ||
+ (nested_state->hdr.vmx.smm.flags & KVM_STATE_NESTED_SMM_VMXON)));
+}
+
+static const VMStateDescription vmstate_vmx_nested_state = {
+ .name = "cpu/kvm_nested_state/vmx",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = vmx_nested_state_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_U64(hdr.vmx.vmxon_pa, struct kvm_nested_state),
+ VMSTATE_U64(hdr.vmx.vmcs12_pa, struct kvm_nested_state),
+ VMSTATE_U16(hdr.vmx.smm.flags, struct kvm_nested_state),
+ VMSTATE_END_OF_LIST()
+ },
+ .subsections = (const VMStateDescription*[]) {
+ &vmstate_vmx_vmcs12,
+ &vmstate_vmx_shadow_vmcs12,
+ NULL,
+ }
+};
+
+static bool svm_nested_state_needed(void *opaque)
+{
+ struct kvm_nested_state *nested_state = opaque;
+
+ return (nested_state->format == KVM_STATE_NESTED_FORMAT_SVM);
+}
+
+static const VMStateDescription vmstate_svm_nested_state = {
+ .name = "cpu/kvm_nested_state/svm",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = svm_nested_state_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_END_OF_LIST()
+ }
+};
+
+static bool nested_state_needed(void *opaque)
+{
+ X86CPU *cpu = opaque;
+ CPUX86State *env = &cpu->env;
+
+ return (env->nested_state &&
+ (vmx_nested_state_needed(env->nested_state) ||
+ svm_nested_state_needed(env->nested_state)));
+}
+
+static int nested_state_post_load(void *opaque, int version_id)
+{
+ X86CPU *cpu = opaque;
+ CPUX86State *env = &cpu->env;
+ struct kvm_nested_state *nested_state = env->nested_state;
+ int min_nested_state_len = offsetof(struct kvm_nested_state, data);
+ int max_nested_state_len = kvm_max_nested_state_length();
+
+ /*
+ * If our kernel don't support setting nested state
+ * and we have received nested state from migration stream,
+ * we need to fail migration
+ */
+ if (max_nested_state_len <= 0) {
+ error_report("Received nested state when kernel cannot restore it");
+ return -EINVAL;
+ }
+
+ /*
+ * Verify that the size of received nested_state struct
+ * at least cover required header and is not larger
+ * than the max size that our kernel support
+ */
+ if (nested_state->size < min_nested_state_len) {
+ error_report("Received nested state size less than min: "
+ "len=%d, min=%d",
+ nested_state->size, min_nested_state_len);
+ return -EINVAL;
+ }
+ if (nested_state->size > max_nested_state_len) {
+ error_report("Recieved unsupported nested state size: "
+ "nested_state->size=%d, max=%d",
+ nested_state->size, max_nested_state_len);
+ return -EINVAL;
+ }
+
+ /* Verify format is valid */
+ if ((nested_state->format != KVM_STATE_NESTED_FORMAT_VMX) &&
+ (nested_state->format != KVM_STATE_NESTED_FORMAT_SVM)) {
+ error_report("Received invalid nested state format: %d",
+ nested_state->format);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const VMStateDescription vmstate_kvm_nested_state = {
+ .name = "cpu/kvm_nested_state",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .fields = (VMStateField[]) {
+ VMSTATE_U16(flags, struct kvm_nested_state),
+ VMSTATE_U16(format, struct kvm_nested_state),
+ VMSTATE_U32(size, struct kvm_nested_state),
+ VMSTATE_END_OF_LIST()
+ },
+ .subsections = (const VMStateDescription*[]) {
+ &vmstate_vmx_nested_state,
+ &vmstate_svm_nested_state,
+ NULL
+ }
+};
+
+static const VMStateDescription vmstate_nested_state = {
+ .name = "cpu/nested_state",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = nested_state_needed,
+ .post_load = nested_state_post_load,
+ .fields = (VMStateField[]) {
+ VMSTATE_STRUCT_POINTER(env.nested_state, X86CPU,
+ vmstate_kvm_nested_state,
+ struct kvm_nested_state),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
+#endif
+
static bool mcg_ext_ctl_needed(void *opaque)
{
X86CPU *cpu = opaque;
@@ -1113,6 +1308,9 @@ VMStateDescription vmstate_x86_cpu = {
#ifndef TARGET_X86_64
&vmstate_efer32,
#endif
+#ifdef CONFIG_KVM
+ &vmstate_nested_state,
+#endif
NULL
}
};
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 21/25] target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (19 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities Paolo Bonzini
` (5 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Kernel commit c4f55198c7c2 ("kvm: x86: Introduce KVM_CAP_EXCEPTION_PAYLOAD")
introduced a new KVM capability which allows userspace to correctly
distinguish between pending and injected exceptions.
This distinguish is important in case of nested virtualization scenarios
because a L2 pending exception can still be intercepted by the L1 hypervisor
while a L2 injected exception cannot.
Furthermore, when an exception is attempted to be injected by QEMU,
QEMU should specify the exception payload (CR2 in case of #PF or
DR6 in case of #DB) instead of having the payload already delivered in
the respective vCPU register. Because in case exception is injected to
L2 guest and is intercepted by L1 hypervisor, then payload needs to be
reported to L1 intercept (VMExit handler) while still preserving
respective vCPU register unchanged.
This commit adds support for QEMU to properly utilise this new KVM
capability (KVM_CAP_EXCEPTION_PAYLOAD).
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20190619162140.133674-10-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/cpu.c | 6 ++-
target/i386/cpu.h | 6 ++-
target/i386/hvf/hvf.c | 10 +++--
target/i386/hvf/x86hvf.c | 4 +-
target/i386/kvm.c | 101 +++++++++++++++++++++++++++++++++++++++--------
target/i386/machine.c | 84 ++++++++++++++++++++++++++++++++++++++-
6 files changed, 187 insertions(+), 24 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 61e44cb..da6eb67 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4819,7 +4819,11 @@ static void x86_cpu_reset(CPUState *s)
memset(env->mtrr_fixed, 0, sizeof(env->mtrr_fixed));
env->interrupt_injected = -1;
- env->exception_injected = -1;
+ env->exception_nr = -1;
+ env->exception_pending = 0;
+ env->exception_injected = 0;
+ env->exception_has_payload = false;
+ env->exception_payload = 0;
env->nmi_injected = false;
#if !defined(CONFIG_USER_ONLY)
/* We hard-wire the BSP to the first CPU. */
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 17116ef..9334579 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1348,10 +1348,14 @@ typedef struct CPUX86State {
/* For KVM */
uint32_t mp_state;
- int32_t exception_injected;
+ int32_t exception_nr;
int32_t interrupt_injected;
uint8_t soft_interrupt;
+ uint8_t exception_pending;
+ uint8_t exception_injected;
uint8_t has_error_code;
+ uint8_t exception_has_payload;
+ uint64_t exception_payload;
uint32_t ins_len;
uint32_t sipi_vector;
bool tsc_valid;
diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index 2751c81..dc4bb63 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -605,7 +605,9 @@ static void hvf_store_events(CPUState *cpu, uint32_t ins_len, uint64_t idtvec_in
X86CPU *x86_cpu = X86_CPU(cpu);
CPUX86State *env = &x86_cpu->env;
- env->exception_injected = -1;
+ env->exception_nr = -1;
+ env->exception_pending = 0;
+ env->exception_injected = 0;
env->interrupt_injected = -1;
env->nmi_injected = false;
if (idtvec_info & VMCS_IDT_VEC_VALID) {
@@ -619,7 +621,8 @@ static void hvf_store_events(CPUState *cpu, uint32_t ins_len, uint64_t idtvec_in
break;
case VMCS_IDT_VEC_HWEXCEPTION:
case VMCS_IDT_VEC_SWEXCEPTION:
- env->exception_injected = idtvec_info & VMCS_IDT_VEC_VECNUM;
+ env->exception_nr = idtvec_info & VMCS_IDT_VEC_VECNUM;
+ env->exception_injected = 1;
break;
case VMCS_IDT_VEC_PRIV_SWEXCEPTION:
default:
@@ -912,7 +915,8 @@ int hvf_vcpu_exec(CPUState *cpu)
macvm_set_rip(cpu, rip + ins_len);
break;
case VMX_REASON_VMCALL:
- env->exception_injected = EXCP0D_GPF;
+ env->exception_nr = EXCP0D_GPF;
+ env->exception_injected = 1;
env->has_error_code = true;
env->error_code = 0;
break;
diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c
index df8e946..e0ea02d 100644
--- a/target/i386/hvf/x86hvf.c
+++ b/target/i386/hvf/x86hvf.c
@@ -362,8 +362,8 @@ bool hvf_inject_interrupts(CPUState *cpu_state)
if (env->interrupt_injected != -1) {
vector = env->interrupt_injected;
intr_type = VMCS_INTR_T_SWINTR;
- } else if (env->exception_injected != -1) {
- vector = env->exception_injected;
+ } else if (env->exception_nr != -1) {
+ vector = env->exception_nr;
if (vector == EXCP03_INT3 || vector == EXCP04_INTO) {
intr_type = VMCS_INTR_T_SWEXCEPTION;
} else {
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index e924663..c931e9d 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -104,6 +104,7 @@ static uint32_t num_architectural_pmu_fixed_counters;
static int has_xsave;
static int has_xcrs;
static int has_pit_state2;
+static int has_exception_payload;
static bool has_msr_mcg_ext_ctl;
@@ -584,15 +585,56 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
/* Hope we are lucky for AO MCE */
}
+static void kvm_reset_exception(CPUX86State *env)
+{
+ env->exception_nr = -1;
+ env->exception_pending = 0;
+ env->exception_injected = 0;
+ env->exception_has_payload = false;
+ env->exception_payload = 0;
+}
+
+static void kvm_queue_exception(CPUX86State *env,
+ int32_t exception_nr,
+ uint8_t exception_has_payload,
+ uint64_t exception_payload)
+{
+ assert(env->exception_nr == -1);
+ assert(!env->exception_pending);
+ assert(!env->exception_injected);
+ assert(!env->exception_has_payload);
+
+ env->exception_nr = exception_nr;
+
+ if (has_exception_payload) {
+ env->exception_pending = 1;
+
+ env->exception_has_payload = exception_has_payload;
+ env->exception_payload = exception_payload;
+ } else {
+ env->exception_injected = 1;
+
+ if (exception_nr == EXCP01_DB) {
+ assert(exception_has_payload);
+ env->dr[6] = exception_payload;
+ } else if (exception_nr == EXCP0E_PAGE) {
+ assert(exception_has_payload);
+ env->cr[2] = exception_payload;
+ } else {
+ assert(!exception_has_payload);
+ }
+ }
+}
+
static int kvm_inject_mce_oldstyle(X86CPU *cpu)
{
CPUX86State *env = &cpu->env;
- if (!kvm_has_vcpu_events() && env->exception_injected == EXCP12_MCHK) {
+ if (!kvm_has_vcpu_events() && env->exception_nr == EXCP12_MCHK) {
unsigned int bank, bank_num = env->mcg_cap & 0xff;
struct kvm_x86_mce mce;
- env->exception_injected = -1;
+ kvm_reset_exception(env);
/*
* There must be at least one bank in use if an MCE is pending.
@@ -1943,6 +1985,16 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
hv_vpindex_settable = kvm_check_extension(s, KVM_CAP_HYPERV_VP_INDEX);
+ has_exception_payload = kvm_check_extension(s, KVM_CAP_EXCEPTION_PAYLOAD);
+ if (has_exception_payload) {
+ ret = kvm_vm_enable_cap(s, KVM_CAP_EXCEPTION_PAYLOAD, 0, true);
+ if (ret < 0) {
+ error_report("kvm: Failed to enable exception payload cap: %s",
+ strerror(-ret));
+ return ret;
+ }
+ }
+
ret = kvm_get_supported_msrs(s);
if (ret < 0) {
return ret;
@@ -3253,8 +3305,16 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level)
return 0;
}
- events.exception.injected = (env->exception_injected >= 0);
- events.exception.nr = env->exception_injected;
+ events.flags = 0;
+
+ if (has_exception_payload) {
+ events.flags |= KVM_VCPUEVENT_VALID_PAYLOAD;
+ events.exception.pending = env->exception_pending;
+ events.exception_has_payload = env->exception_has_payload;
+ events.exception_payload = env->exception_payload;
+ }
+ events.exception.nr = env->exception_nr;
+ events.exception.injected = env->exception_injected;
events.exception.has_error_code = env->has_error_code;
events.exception.error_code = env->error_code;
@@ -3267,7 +3327,6 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level)
events.nmi.masked = !!(env->hflags2 & HF2_NMI_MASK);
events.sipi_vector = env->sipi_vector;
- events.flags = 0;
if (has_msr_smbase) {
events.smi.smm = !!(env->hflags & HF_SMM_MASK);
@@ -3317,8 +3376,19 @@ static int kvm_get_vcpu_events(X86CPU *cpu)
if (ret < 0) {
return ret;
}
- env->exception_injected =
- events.exception.injected ? events.exception.nr : -1;
+
+ if (events.flags & KVM_VCPUEVENT_VALID_PAYLOAD) {
+ env->exception_pending = events.exception.pending;
+ env->exception_has_payload = events.exception_has_payload;
+ env->exception_payload = events.exception_payload;
+ } else {
+ env->exception_pending = 0;
+ env->exception_has_payload = false;
+ }
+ env->exception_injected = events.exception.injected;
+ env->exception_nr =
+ (env->exception_pending || env->exception_injected) ?
+ events.exception.nr : -1;
env->has_error_code = events.exception.has_error_code;
env->error_code = events.exception.error_code;
@@ -3370,12 +3440,12 @@ static int kvm_guest_debug_workarounds(X86CPU *cpu)
unsigned long reinject_trap = 0;
if (!kvm_has_vcpu_events()) {
- if (env->exception_injected == EXCP01_DB) {
+ if (env->exception_nr == EXCP01_DB) {
reinject_trap = KVM_GUESTDBG_INJECT_DB;
} else if (env->exception_injected == EXCP03_INT3) {
reinject_trap = KVM_GUESTDBG_INJECT_BP;
}
- env->exception_injected = -1;
+ kvm_reset_exception(env);
}
/*
@@ -3751,13 +3821,13 @@ int kvm_arch_process_async_events(CPUState *cs)
kvm_cpu_synchronize_state(cs);
- if (env->exception_injected == EXCP08_DBLE) {
+ if (env->exception_nr == EXCP08_DBLE) {
/* this means triple fault */
qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET);
cs->exit_request = 1;
return 0;
}
- env->exception_injected = EXCP12_MCHK;
+ kvm_queue_exception(env, EXCP12_MCHK, 0, 0);
env->has_error_code = 0;
cs->halted = 0;
@@ -3972,14 +4042,13 @@ static int kvm_handle_debug(X86CPU *cpu,
}
if (ret == 0) {
cpu_synchronize_state(cs);
- assert(env->exception_injected == -1);
+ assert(env->exception_nr == -1);
/* pass to guest */
- env->exception_injected = arch_info->exception;
+ kvm_queue_exception(env, arch_info->exception,
+ arch_info->exception == EXCP01_DB,
+ arch_info->dr6);
env->has_error_code = 0;
- if (arch_info->exception == EXCP01_DB) {
- env->dr[6] = arch_info->dr6;
- }
}
return ret;
diff --git a/target/i386/machine.c b/target/i386/machine.c
index a6afdf8..fc49e5a 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -240,6 +240,41 @@ static int cpu_pre_save(void *opaque)
}
#endif
+ /*
+ * When vCPU is running L2 and exception is still pending,
+ * it can potentially be intercepted by L1 hypervisor.
+ * In contrast to an injected exception which cannot be
+ * intercepted anymore.
+ *
+ * Furthermore, when a L2 exception is intercepted by L1
+ * hypervisor, it's exception payload (CR2/DR6 on #PF/#DB)
+ * should not be set yet in the respective vCPU register.
+ * Thus, in case an exception is pending, it is
+ * important to save the exception payload seperately.
+ *
+ * Therefore, if an exception is not in a pending state
+ * or vCPU is not in guest-mode, it is not important to
+ * distinguish between a pending and injected exception
+ * and we don't need to store seperately the exception payload.
+ *
+ * In order to preserve better backwards-compatabile migration,
+ * convert a pending exception to an injected exception in
+ * case it is not important to distingiush between them
+ * as described above.
+ */
+ if (env->exception_pending && !(env->hflags & HF_GUEST_MASK)) {
+ env->exception_pending = 0;
+ env->exception_injected = 1;
+
+ if (env->exception_has_payload) {
+ if (env->exception_nr == EXCP01_DB) {
+ env->dr[6] = env->exception_payload;
+ } else if (env->exception_nr == EXCP0E_PAGE) {
+ env->cr[2] = env->exception_payload;
+ }
+ }
+ }
+
return 0;
}
@@ -297,6 +332,23 @@ static int cpu_post_load(void *opaque, int version_id)
}
#endif
+ /*
+ * There are cases that we can get valid exception_nr with both
+ * exception_pending and exception_injected being cleared.
+ * This can happen in one of the following scenarios:
+ * 1) Source is older QEMU without KVM_CAP_EXCEPTION_PAYLOAD support.
+ * 2) Source is running on kernel without KVM_CAP_EXCEPTION_PAYLOAD support.
+ * 3) "cpu/exception_info" subsection not sent because there is no exception
+ * pending or guest wasn't running L2 (See comment in cpu_pre_save()).
+ *
+ * In those cases, we can just deduce that a valid exception_nr means
+ * we can treat the exception as already injected.
+ */
+ if ((env->exception_nr != -1) &&
+ !env->exception_pending && !env->exception_injected) {
+ env->exception_injected = 1;
+ }
+
env->fpstt = (env->fpus_vmstate >> 11) & 7;
env->fpus = env->fpus_vmstate & ~0x3800;
env->fptag_vmstate ^= 0xff;
@@ -342,6 +394,35 @@ static bool steal_time_msr_needed(void *opaque)
return cpu->env.steal_time_msr != 0;
}
+static bool exception_info_needed(void *opaque)
+{
+ X86CPU *cpu = opaque;
+ CPUX86State *env = &cpu->env;
+
+ /*
+ * It is important to save exception-info only in case
+ * we need to distingiush between a pending and injected
+ * exception. Which is only required in case there is a
+ * pending exception and vCPU is running L2.
+ * For more info, refer to comment in cpu_pre_save().
+ */
+ return env->exception_pending && (env->hflags & HF_GUEST_MASK);
+}
+
+static const VMStateDescription vmstate_exception_info = {
+ .name = "cpu/exception_info",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = exception_info_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_UINT8(env.exception_pending, X86CPU),
+ VMSTATE_UINT8(env.exception_injected, X86CPU),
+ VMSTATE_UINT8(env.exception_has_payload, X86CPU),
+ VMSTATE_UINT64(env.exception_payload, X86CPU),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
static const VMStateDescription vmstate_steal_time_msr = {
.name = "cpu/steal_time_msr",
.version_id = 1,
@@ -1251,7 +1332,7 @@ VMStateDescription vmstate_x86_cpu = {
VMSTATE_INT32(env.interrupt_injected, X86CPU),
VMSTATE_UINT32(env.mp_state, X86CPU),
VMSTATE_UINT64(env.tsc, X86CPU),
- VMSTATE_INT32(env.exception_injected, X86CPU),
+ VMSTATE_INT32(env.exception_nr, X86CPU),
VMSTATE_UINT8(env.soft_interrupt, X86CPU),
VMSTATE_UINT8(env.nmi_injected, X86CPU),
VMSTATE_UINT8(env.nmi_pending, X86CPU),
@@ -1275,6 +1356,7 @@ VMStateDescription vmstate_x86_cpu = {
/* The above list is not sorted /wrt version numbers, watch out! */
},
.subsections = (const VMStateDescription*[]) {
+ &vmstate_exception_info,
&vmstate_async_pf_msr,
&vmstate_pv_eoi_msr,
&vmstate_steal_time_msr,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (20 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 21/25] target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 12:39 ` Liran Alon
2019-07-08 18:31 ` Jan Kiszka
2019-06-21 11:30 ` [Qemu-devel] [PULL 23/25] sd: Fix out-of-bounds assertions Paolo Bonzini
` (4 subsequent siblings)
26 siblings, 2 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Alon
From: Liran Alon <liran.alon@oracle.com>
Previous commits have added support for migration of nested virtualization
workloads. This was done by utilising two new KVM capabilities:
KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
required in order to correctly migrate such workloads.
Therefore, change code to add a migration blocker for vCPUs exposed with
Intel VMX or AMD SVM in case one of these kernel capabilities is
missing.
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
Message-Id: <20190619162140.133674-11-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/kvm.c | 9 +++++++--
target/i386/machine.c | 2 +-
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index c931e9d..e4b4f57 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1640,9 +1640,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
!!(c->ecx & CPUID_EXT_SMX);
}
- if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
+ if (cpu_has_vmx(env) && !nested_virt_mig_blocker &&
+ ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
error_setg(&nested_virt_mig_blocker,
- "Nested virtualization does not support live migration yet");
+ "Kernel do not provide required capabilities for "
+ "nested virtualization migration. "
+ "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
+ kvm_max_nested_state_length() > 0,
+ has_exception_payload);
r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
if (local_err) {
error_report_err(local_err);
diff --git a/target/i386/machine.c b/target/i386/machine.c
index fc49e5a..851b249 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -233,7 +233,7 @@ static int cpu_pre_save(void *opaque)
#ifdef CONFIG_KVM
/* Verify we have nested virtualization state from kernel if required */
- if (cpu_has_nested_virt(env) && !env->nested_state) {
+ if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
error_report("Guest enabled nested virtualization but kernel "
"does not support saving of nested state");
return -EINVAL;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 23/25] sd: Fix out-of-bounds assertions
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (21 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 24/25] util/main-loop: Fix incorrect assertion Paolo Bonzini
` (3 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Lidong Chen
From: Lidong Chen <lidong.chen@oracle.com>
Due to an off-by-one error, the assert statements allow an
out-of-bound array access. This doesn't happen in practice,
but the static analyzer notices.
Signed-off-by: Lidong Chen <lidong.chen@oracle.com>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Li Qiang <liq3ea@gmail.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <6b19cb7359a10a6bedc3ea0fce22fed3ef93c102.1560806687.git.lidong.chen@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/sd/sd.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/hw/sd/sd.c b/hw/sd/sd.c
index 60500ec..917195a6 100644
--- a/hw/sd/sd.c
+++ b/hw/sd/sd.c
@@ -145,7 +145,7 @@ static const char *sd_state_name(enum SDCardStates state)
if (state == sd_inactive_state) {
return "inactive";
}
- assert(state <= ARRAY_SIZE(state_name));
+ assert(state < ARRAY_SIZE(state_name));
return state_name[state];
}
@@ -166,7 +166,7 @@ static const char *sd_response_name(sd_rsp_type_t rsp)
if (rsp == sd_r1b) {
rsp = sd_r1;
}
- assert(rsp <= ARRAY_SIZE(response_name));
+ assert(rsp < ARRAY_SIZE(response_name));
return response_name[rsp];
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 24/25] util/main-loop: Fix incorrect assertion
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (22 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 23/25] sd: Fix out-of-bounds assertions Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 25/25] hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1 Paolo Bonzini
` (2 subsequent siblings)
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Lidong Chen
From: Lidong Chen <lidong.chen@oracle.com>
The check for poll_fds in g_assert() was incorrect. The correct assertion
should check "n_poll_fds + w->num <= ARRAY_SIZE(poll_fds)" because the
subsequent for-loop is doing access to poll_fds[n_poll_fds + i] where i
is in [0, w->num). This could happen with a very high number of file
descriptors and/or wait objects.
Signed-off-by: Lidong Chen <lidong.chen@oracle.com>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Suggested-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Li Qiang <liq3ea@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <ded30967982811617ce7f0222d11228130c198b7.1560806687.git.lidong.chen@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
util/main-loop.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/util/main-loop.c b/util/main-loop.c
index e1e349c..a9f4e8d 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -422,7 +422,7 @@ static int os_host_main_loop_wait(int64_t timeout)
g_main_context_prepare(context, &max_priority);
n_poll_fds = g_main_context_query(context, max_priority, &poll_timeout,
poll_fds, ARRAY_SIZE(poll_fds));
- g_assert(n_poll_fds <= ARRAY_SIZE(poll_fds));
+ g_assert(n_poll_fds + w->num <= ARRAY_SIZE(poll_fds));
for (i = 0; i < w->num; i++) {
poll_fds[n_poll_fds + i].fd = (DWORD_PTR)w->events[i];
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 25/25] hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (23 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 24/25] util/main-loop: Fix incorrect assertion Paolo Bonzini
@ 2019-06-21 11:30 ` Paolo Bonzini
2019-06-21 13:53 ` [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 no-reply
2019-06-21 14:40 ` Peter Maydell
26 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 11:30 UTC (permalink / raw)
To: qemu-devel; +Cc: Greg Kurz
From: Greg Kurz <groug@kaod.org>
Commit c87759ce876a fixed a regression affecting pc-q35 machines by
introducing a new pc-q35-4.0.1 machine version to be used instead
of pc-q35-4.0. The only purpose was to revert the default behaviour
of not using split irqchip, but the change also introduced the usual
hw_compat and pc_compat bits, and wired them for pc-q35 only.
This raises questions when it comes to add new compat properties for
4.0* machine versions of any architecture. Where to add them ? In
4.0, 4.0.1 or both ? Error prone. Another possibility would be to teach
all other architectures about 4.0.1. This solution isn't satisfying,
especially since this is a pc-q35 specific issue.
It turns out that the split irqchip default is handled in the machine
option function and doesn't involve compat lists at all.
Drop all the 4.0.1 compat lists and use the 4.0 ones instead in the 4.0.1
machine option function.
Move the compat props that were added to the 4.0.1 since c87759ce876a to
4.0.
Even if only hw_compat_4_0_1 had an impact on other architectures,
drop pc_compat_4_0_1 as well for consistency.
Fixes: c87759ce876a "q35: Revert to kernel irqchip"
Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <156051774276.244890.8660277280145466396.stgit@bahia.lan>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/core/machine.c | 5 +----
hw/i386/pc.c | 3 ---
hw/i386/pc_q35.c | 12 ++++++++----
include/hw/boards.h | 3 ---
include/hw/i386/pc.h | 3 ---
5 files changed, 9 insertions(+), 17 deletions(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 84ebb8d..ea5a01a 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -24,16 +24,13 @@
#include "hw/pci/pci.h"
#include "hw/mem/nvdimm.h"
-GlobalProperty hw_compat_4_0_1[] = {
+GlobalProperty hw_compat_4_0[] = {
{ "VGA", "edid", "false" },
{ "secondary-vga", "edid", "false" },
{ "bochs-display", "edid", "false" },
{ "virtio-vga", "edid", "false" },
{ "virtio-gpu-pci", "edid", "false" },
};
-const size_t hw_compat_4_0_1_len = G_N_ELEMENTS(hw_compat_4_0_1);
-
-GlobalProperty hw_compat_4_0[] = {};
const size_t hw_compat_4_0_len = G_N_ELEMENTS(hw_compat_4_0);
GlobalProperty hw_compat_3_1[] = {
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index e41192b..e96360b 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -111,9 +111,6 @@ struct hpet_fw_config hpet_cfg = {.count = UINT8_MAX};
/* Physical Address of PVH entry point read from kernel ELF NOTE */
static size_t pvh_start_addr;
-GlobalProperty pc_compat_4_0_1[] = {};
-const size_t pc_compat_4_0_1_len = G_N_ELEMENTS(pc_compat_4_0_1);
-
GlobalProperty pc_compat_4_0[] = {};
const size_t pc_compat_4_0_len = G_N_ELEMENTS(pc_compat_4_0);
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index dcddc64..57232ae 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -378,8 +378,13 @@ static void pc_q35_4_0_1_machine_options(MachineClass *m)
{
pc_q35_4_1_machine_options(m);
m->alias = NULL;
- compat_props_add(m->compat_props, hw_compat_4_0_1, hw_compat_4_0_1_len);
- compat_props_add(m->compat_props, pc_compat_4_0_1, pc_compat_4_0_1_len);
+ /*
+ * This is the default machine for the 4.0-stable branch. It is basically
+ * a 4.0 that doesn't use split irqchip by default. It MUST hence apply the
+ * 4.0 compat props.
+ */
+ compat_props_add(m->compat_props, hw_compat_4_0, hw_compat_4_0_len);
+ compat_props_add(m->compat_props, pc_compat_4_0, pc_compat_4_0_len);
}
DEFINE_Q35_MACHINE(v4_0_1, "pc-q35-4.0.1", NULL,
@@ -390,8 +395,7 @@ static void pc_q35_4_0_machine_options(MachineClass *m)
pc_q35_4_0_1_machine_options(m);
m->default_kernel_irqchip_split = true;
m->alias = NULL;
- compat_props_add(m->compat_props, hw_compat_4_0, hw_compat_4_0_len);
- compat_props_add(m->compat_props, pc_compat_4_0, pc_compat_4_0_len);
+ /* Compat props are applied by the 4.0.1 machine */
}
DEFINE_Q35_MACHINE(v4_0, "pc-q35-4.0", NULL,
diff --git a/include/hw/boards.h b/include/hw/boards.h
index b7362af..eaa050a 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -293,9 +293,6 @@ struct MachineState {
} \
type_init(machine_initfn##_register_types)
-extern GlobalProperty hw_compat_4_0_1[];
-extern const size_t hw_compat_4_0_1_len;
-
extern GlobalProperty hw_compat_4_0[];
extern const size_t hw_compat_4_0_len;
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index a7d0b87..c54cc54 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -293,9 +293,6 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t);
int e820_get_num_entries(void);
bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *);
-extern GlobalProperty pc_compat_4_0_1[];
-extern const size_t pc_compat_4_0_1_len;
-
extern GlobalProperty pc_compat_4_0[];
extern const size_t pc_compat_4_0_len;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 11:30 ` [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state Paolo Bonzini
@ 2019-06-21 12:29 ` Liran Alon
2019-06-21 12:45 ` Paolo Bonzini
0 siblings, 1 reply; 54+ messages in thread
From: Liran Alon @ 2019-06-21 12:29 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 14:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> From: Liran Alon <liran.alon@oracle.com>
>
> Kernel commit 8fcc4b5923af ("kvm: nVMX: Introduce KVM_CAP_NESTED_STATE")
> introduced new IOCTLs to extract and restore vCPU state related to
> Intel VMX & AMD SVM.
>
> Utilize these IOCTLs to add support for migration of VMs which are
> running nested hypervisors.
>
> Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
> Tested-by: Maran Wilson <maran.wilson@oracle.com>
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Message-Id: <20190619162140.133674-9-liran.alon@oracle.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> accel/kvm/kvm-all.c | 8 ++
> include/sysemu/kvm.h | 1 +
> target/i386/cpu.h | 3 +
> target/i386/kvm.c | 80 ++++++++++++++++++++
> target/i386/machine.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 5 files changed, 290 insertions(+)
>
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index f0f5ab8..e3cf728 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -87,6 +87,7 @@ struct KVMState
> #ifdef KVM_CAP_SET_GUEST_DEBUG
> QTAILQ_HEAD(, kvm_sw_breakpoint) kvm_sw_breakpoints;
> #endif
> + int max_nested_state_len;
> int many_ioeventfds;
> int intx_set_mask;
> bool sync_mmu;
> @@ -1681,6 +1682,8 @@ static int kvm_init(MachineState *ms)
> s->debugregs = kvm_check_extension(s, KVM_CAP_DEBUGREGS);
> #endif
>
> + s->max_nested_state_len = kvm_check_extension(s, KVM_CAP_NESTED_STATE);
> +
> #ifdef KVM_CAP_IRQ_ROUTING
> kvm_direct_msi_allowed = (kvm_check_extension(s, KVM_CAP_SIGNAL_MSI) > 0);
> #endif
> @@ -2248,6 +2251,11 @@ int kvm_has_debugregs(void)
> return kvm_state->debugregs;
> }
>
> +int kvm_max_nested_state_length(void)
> +{
> + return kvm_state->max_nested_state_len;
> +}
> +
> int kvm_has_many_ioeventfds(void)
> {
> if (!kvm_enabled()) {
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index 64f55e5..acd90ae 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -210,6 +210,7 @@ bool kvm_has_sync_mmu(void);
> int kvm_has_vcpu_events(void);
> int kvm_has_robust_singlestep(void);
> int kvm_has_debugregs(void);
> +int kvm_max_nested_state_length(void);
> int kvm_has_pit_state2(void);
> int kvm_has_many_ioeventfds(void);
> int kvm_has_gsi_routing(void);
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index bf0c9c2..17116ef 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -1360,6 +1360,9 @@ typedef struct CPUX86State {
> #if defined(CONFIG_KVM) || defined(CONFIG_HVF)
> void *xsave_buf;
> #endif
> +#if defined(CONFIG_KVM)
> + struct kvm_nested_state *nested_state;
> +#endif
> #if defined(CONFIG_HVF)
> HVFX86EmulatorState *hvf_emul;
> #endif
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index f9872f1..e924663 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1324,6 +1324,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> struct kvm_cpuid_entry2 *c;
> uint32_t signature[3];
> int kvm_base = KVM_CPUID_SIGNATURE;
> + int max_nested_state_len;
> int r;
> Error *local_err = NULL;
>
> @@ -1658,6 +1659,24 @@ int kvm_arch_init_vcpu(CPUState *cs)
> if (has_xsave) {
> env->xsave_buf = qemu_memalign(4096, sizeof(struct kvm_xsave));
> }
> +
> + max_nested_state_len = kvm_max_nested_state_length();
> + if (max_nested_state_len > 0) {
> + assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data));
> + env->nested_state = g_malloc0(max_nested_state_len);
> +
> + env->nested_state->size = max_nested_state_len;
> +
> + if (IS_INTEL_CPU(env)) {
I think it’s better to change this to: “if (cpu_has_vmx(env))” {
> + struct kvm_vmx_nested_state_hdr *vmx_hdr =
> + &env->nested_state->hdr.vmx;
> +
> + env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
> + vmx_hdr->vmxon_pa = -1ull;
> + vmx_hdr->vmcs12_pa = -1ull;
> + }
> + }
I think we should add here:
} else if (cpu_has_svm(env)) {
env->nested_state->format = KVM_STATE_NESTED_FORMAT_SVM;
}
> +
> cpu->kvm_msr_buf = g_malloc0(MSR_BUF_SIZE);
>
> if (!(env->features[FEAT_8000_0001_EDX] & CPUID_EXT2_RDTSCP)) {
> @@ -1682,12 +1701,18 @@ int kvm_arch_init_vcpu(CPUState *cs)
> int kvm_arch_destroy_vcpu(CPUState *cs)
> {
> X86CPU *cpu = X86_CPU(cs);
> + CPUX86State *env = &cpu->env;
>
> if (cpu->kvm_msr_buf) {
> g_free(cpu->kvm_msr_buf);
> cpu->kvm_msr_buf = NULL;
> }
>
> + if (env->nested_state) {
> + g_free(env->nested_state);
> + env->nested_state = NULL;
> + }
> +
> return 0;
> }
>
> @@ -3411,6 +3436,52 @@ static int kvm_get_debugregs(X86CPU *cpu)
> return 0;
> }
>
> +static int kvm_put_nested_state(X86CPU *cpu)
> +{
> + CPUX86State *env = &cpu->env;
> + int max_nested_state_len = kvm_max_nested_state_length();
> +
> + if (max_nested_state_len <= 0) {
> + return 0;
> + }
> +
> + assert(env->nested_state->size <= max_nested_state_len);
> + return kvm_vcpu_ioctl(CPU(cpu), KVM_SET_NESTED_STATE, env->nested_state);
> +}
> +
> +static int kvm_get_nested_state(X86CPU *cpu)
> +{
> + CPUX86State *env = &cpu->env;
> + int max_nested_state_len = kvm_max_nested_state_length();
> + int ret;
> +
> + if (max_nested_state_len <= 0) {
> + return 0;
> + }
> +
> + /*
> + * It is possible that migration restored a smaller size into
> + * nested_state->hdr.size than what our kernel support.
> + * We preserve migration origin nested_state->hdr.size for
> + * call to KVM_SET_NESTED_STATE but wish that our next call
> + * to KVM_GET_NESTED_STATE will use max size our kernel support.
> + */
> + env->nested_state->size = max_nested_state_len;
> +
> + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_NESTED_STATE, env->nested_state);
> + if (ret < 0) {
> + return ret;
> + }
> +
> + if (env->nested_state->flags & KVM_STATE_NESTED_GUEST_MODE) {
> + env->hflags |= HF_GUEST_MASK;
> + } else {
> + env->hflags &= ~HF_GUEST_MASK;
> + }
> +
> + return ret;
> +}
> +
> int kvm_arch_put_registers(CPUState *cpu, int level)
> {
> X86CPU *x86_cpu = X86_CPU(cpu);
> @@ -3418,6 +3489,11 @@ int kvm_arch_put_registers(CPUState *cpu, int level)
>
> assert(cpu_is_stopped(cpu) || qemu_cpu_is_self(cpu));
>
> + ret = kvm_put_nested_state(x86_cpu);
> + if (ret < 0) {
> + return ret;
> + }
> +
> if (level >= KVM_PUT_RESET_STATE) {
> ret = kvm_put_msr_feature_control(x86_cpu);
> if (ret < 0) {
> @@ -3533,6 +3609,10 @@ int kvm_arch_get_registers(CPUState *cs)
> if (ret < 0) {
> goto out;
> }
> + ret = kvm_get_nested_state(cpu);
> + if (ret < 0) {
> + goto out;
> + }
> ret = 0;
> out:
> cpu_sync_bndcs_hflags(&cpu->env);
> diff --git a/target/i386/machine.c b/target/i386/machine.c
> index a39ce7f..a6afdf8 100644
> --- a/target/i386/machine.c
> +++ b/target/i386/machine.c
> @@ -231,6 +231,15 @@ static int cpu_pre_save(void *opaque)
> env->segs[R_SS].flags &= ~(env->segs[R_SS].flags & DESC_DPL_MASK);
> }
>
> +#ifdef CONFIG_KVM
> + /* Verify we have nested virtualization state from kernel if required */
> + if (cpu_has_nested_virt(env) && !env->nested_state) {
> + error_report("Guest enabled nested virtualization but kernel "
> + "does not support saving of nested state");
> + return -EINVAL;
> + }
> +#endif
Paolo, just note that this will effectively block migration for AMD vCPU exposed with SVM.
I think it’s the right thing to do but you were afraid of backwards compatibility regarding this no?
-Liran
> +
> return 0;
> }
>
> @@ -278,6 +287,16 @@ static int cpu_post_load(void *opaque, int version_id)
> env->hflags &= ~HF_CPL_MASK;
> env->hflags |= (env->segs[R_SS].flags >> DESC_DPL_SHIFT) & HF_CPL_MASK;
>
> +#ifdef CONFIG_KVM
> + if ((env->hflags & HF_GUEST_MASK) &&
> + (!env->nested_state ||
> + !(env->nested_state->flags & KVM_STATE_NESTED_GUEST_MODE))) {
> + error_report("vCPU set in guest-mode inconsistent with "
> + "migrated kernel nested state");
> + return -EINVAL;
> + }
> +#endif
> +
> env->fpstt = (env->fpus_vmstate >> 11) & 7;
> env->fpus = env->fpus_vmstate & ~0x3800;
> env->fptag_vmstate ^= 0xff;
> @@ -851,6 +870,182 @@ static const VMStateDescription vmstate_tsc_khz = {
> }
> };
>
> +#ifdef CONFIG_KVM
> +
> +static bool vmx_vmcs12_needed(void *opaque)
> +{
> + struct kvm_nested_state *nested_state = opaque;
> + return (nested_state->size >
> + offsetof(struct kvm_nested_state, data.vmx[0].vmcs12));
> +}
> +
> +static const VMStateDescription vmstate_vmx_vmcs12 = {
> + .name = "cpu/kvm_nested_state/vmx/vmcs12",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .needed = vmx_vmcs12_needed,
> + .fields = (VMStateField[]) {
> + VMSTATE_UINT8_ARRAY(data.vmx[0].vmcs12,
> + struct kvm_nested_state,
> + KVM_STATE_NESTED_VMX_VMCS_SIZE),
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> +static bool vmx_shadow_vmcs12_needed(void *opaque)
> +{
> + struct kvm_nested_state *nested_state = opaque;
> + return (nested_state->size >
> + offsetof(struct kvm_nested_state, data.vmx[0].shadow_vmcs12));
> +}
> +
> +static const VMStateDescription vmstate_vmx_shadow_vmcs12 = {
> + .name = "cpu/kvm_nested_state/vmx/shadow_vmcs12",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .needed = vmx_shadow_vmcs12_needed,
> + .fields = (VMStateField[]) {
> + VMSTATE_UINT8_ARRAY(data.vmx[0].shadow_vmcs12,
> + struct kvm_nested_state,
> + KVM_STATE_NESTED_VMX_VMCS_SIZE),
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> +static bool vmx_nested_state_needed(void *opaque)
> +{
> + struct kvm_nested_state *nested_state = opaque;
> +
> + return ((nested_state->format == KVM_STATE_NESTED_FORMAT_VMX) &&
> + ((nested_state->hdr.vmx.vmxon_pa != -1ull) ||
> + (nested_state->hdr.vmx.smm.flags & KVM_STATE_NESTED_SMM_VMXON)));
> +}
> +
> +static const VMStateDescription vmstate_vmx_nested_state = {
> + .name = "cpu/kvm_nested_state/vmx",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .needed = vmx_nested_state_needed,
> + .fields = (VMStateField[]) {
> + VMSTATE_U64(hdr.vmx.vmxon_pa, struct kvm_nested_state),
> + VMSTATE_U64(hdr.vmx.vmcs12_pa, struct kvm_nested_state),
> + VMSTATE_U16(hdr.vmx.smm.flags, struct kvm_nested_state),
> + VMSTATE_END_OF_LIST()
> + },
> + .subsections = (const VMStateDescription*[]) {
> + &vmstate_vmx_vmcs12,
> + &vmstate_vmx_shadow_vmcs12,
> + NULL,
> + }
> +};
> +
> +static bool svm_nested_state_needed(void *opaque)
> +{
> + struct kvm_nested_state *nested_state = opaque;
> +
> + return (nested_state->format == KVM_STATE_NESTED_FORMAT_SVM);
> +}
> +
> +static const VMStateDescription vmstate_svm_nested_state = {
> + .name = "cpu/kvm_nested_state/svm",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .needed = svm_nested_state_needed,
> + .fields = (VMStateField[]) {
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> +static bool nested_state_needed(void *opaque)
> +{
> + X86CPU *cpu = opaque;
> + CPUX86State *env = &cpu->env;
> +
> + return (env->nested_state &&
> + (vmx_nested_state_needed(env->nested_state) ||
> + svm_nested_state_needed(env->nested_state)));
> +}
> +
> +static int nested_state_post_load(void *opaque, int version_id)
> +{
> + X86CPU *cpu = opaque;
> + CPUX86State *env = &cpu->env;
> + struct kvm_nested_state *nested_state = env->nested_state;
> + int min_nested_state_len = offsetof(struct kvm_nested_state, data);
> + int max_nested_state_len = kvm_max_nested_state_length();
> +
> + /*
> + * If our kernel don't support setting nested state
> + * and we have received nested state from migration stream,
> + * we need to fail migration
> + */
> + if (max_nested_state_len <= 0) {
> + error_report("Received nested state when kernel cannot restore it");
> + return -EINVAL;
> + }
> +
> + /*
> + * Verify that the size of received nested_state struct
> + * at least cover required header and is not larger
> + * than the max size that our kernel support
> + */
> + if (nested_state->size < min_nested_state_len) {
> + error_report("Received nested state size less than min: "
> + "len=%d, min=%d",
> + nested_state->size, min_nested_state_len);
> + return -EINVAL;
> + }
> + if (nested_state->size > max_nested_state_len) {
> + error_report("Recieved unsupported nested state size: "
> + "nested_state->size=%d, max=%d",
> + nested_state->size, max_nested_state_len);
> + return -EINVAL;
> + }
> +
> + /* Verify format is valid */
> + if ((nested_state->format != KVM_STATE_NESTED_FORMAT_VMX) &&
> + (nested_state->format != KVM_STATE_NESTED_FORMAT_SVM)) {
> + error_report("Received invalid nested state format: %d",
> + nested_state->format);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static const VMStateDescription vmstate_kvm_nested_state = {
> + .name = "cpu/kvm_nested_state",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .fields = (VMStateField[]) {
> + VMSTATE_U16(flags, struct kvm_nested_state),
> + VMSTATE_U16(format, struct kvm_nested_state),
> + VMSTATE_U32(size, struct kvm_nested_state),
> + VMSTATE_END_OF_LIST()
> + },
> + .subsections = (const VMStateDescription*[]) {
> + &vmstate_vmx_nested_state,
> + &vmstate_svm_nested_state,
> + NULL
> + }
> +};
> +
> +static const VMStateDescription vmstate_nested_state = {
> + .name = "cpu/nested_state",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .needed = nested_state_needed,
> + .post_load = nested_state_post_load,
> + .fields = (VMStateField[]) {
> + VMSTATE_STRUCT_POINTER(env.nested_state, X86CPU,
> + vmstate_kvm_nested_state,
> + struct kvm_nested_state),
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> +#endif
> +
> static bool mcg_ext_ctl_needed(void *opaque)
> {
> X86CPU *cpu = opaque;
> @@ -1113,6 +1308,9 @@ VMStateDescription vmstate_x86_cpu = {
> #ifndef TARGET_X86_64
> &vmstate_efer32,
> #endif
> +#ifdef CONFIG_KVM
> + &vmstate_nested_state,
> +#endif
> NULL
> }
> };
> --
> 1.8.3.1
>
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 11:30 ` [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities Paolo Bonzini
@ 2019-06-21 12:39 ` Liran Alon
2019-06-21 15:02 ` Paolo Bonzini
2019-07-08 18:31 ` Jan Kiszka
1 sibling, 1 reply; 54+ messages in thread
From: Liran Alon @ 2019-06-21 12:39 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 14:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> From: Liran Alon <liran.alon@oracle.com>
>
> Previous commits have added support for migration of nested virtualization
> workloads. This was done by utilising two new KVM capabilities:
> KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
> required in order to correctly migrate such workloads.
>
> Therefore, change code to add a migration blocker for vCPUs exposed with
> Intel VMX or AMD SVM in case one of these kernel capabilities is
> missing.
>
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
> Message-Id: <20190619162140.133674-11-liran.alon@oracle.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> target/i386/kvm.c | 9 +++++++--
> target/i386/machine.c | 2 +-
> 2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index c931e9d..e4b4f57 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1640,9 +1640,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
> !!(c->ecx & CPUID_EXT_SMX);
> }
>
> - if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
> + if (cpu_has_vmx(env) && !nested_virt_mig_blocker &&
The change here from cpu_has_nested_virt(env) to cpu_has_vmx(env) is not clear.
We should explicitly explain that it’s because we still wish to preserve backwards-compatability
to migrating AMD vCPU exposed with SVM.
In addition, commit ("target/i386: kvm: Block migration for vCPUs exposed with nested virtualization")
doesn’t make sense in case we still want to allow migrating AMD vCPU exposed with SVM.
Since QEMU commit 75d373ef9729 ("target-i386: Disable SVM by default in KVM mode"),
machine-types since v2.2 don’t expose AMD SVM by default.
Therefore, my personal opinion on this is that it’s fine to block migration in this case.
> + ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
> error_setg(&nested_virt_mig_blocker,
> - "Nested virtualization does not support live migration yet");
> + "Kernel do not provide required capabilities for “
As Maran have noted, we should change this “do not” to “does not”.
Sorry for my bad English grammer. :)
> + "nested virtualization migration. "
> + "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
> + kvm_max_nested_state_length() > 0,
> + has_exception_payload);
> r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
> if (local_err) {
> error_report_err(local_err);
> diff --git a/target/i386/machine.c b/target/i386/machine.c
> index fc49e5a..851b249 100644
> --- a/target/i386/machine.c
> +++ b/target/i386/machine.c
> @@ -233,7 +233,7 @@ static int cpu_pre_save(void *opaque)
>
> #ifdef CONFIG_KVM
> /* Verify we have nested virtualization state from kernel if required */
> - if (cpu_has_nested_virt(env) && !env->nested_state) {
> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
Good catch regarding adding kvm_enabled() here.
But I think this should have been added to commit ("target/i386: kvm: Add support for save and restore nested state”).
-Liran
> error_report("Guest enabled nested virtualization but kernel "
> "does not support saving of nested state");
> return -EINVAL;
> --
> 1.8.3.1
>
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 12:29 ` Liran Alon
@ 2019-06-21 12:45 ` Paolo Bonzini
2019-06-21 12:48 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 12:45 UTC (permalink / raw)
To: Liran Alon; +Cc: qemu-devel
On 21/06/19 14:29, Liran Alon wrote:
>> + max_nested_state_len = kvm_max_nested_state_length();
>> + if (max_nested_state_len > 0) {
>> + assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data));
>> + env->nested_state = g_malloc0(max_nested_state_len);
>> +
>> + env->nested_state->size = max_nested_state_len;
>> +
>> + if (IS_INTEL_CPU(env)) {
> I think it’s better to change this to: “if (cpu_has_vmx(env))” {
>
>> + struct kvm_vmx_nested_state_hdr *vmx_hdr =
>> + &env->nested_state->hdr.vmx;
>> +
>> + env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
>> + vmx_hdr->vmxon_pa = -1ull;
>> + vmx_hdr->vmcs12_pa = -1ull;
>> + }
>> + }
> I think we should add here:
> } else if (cpu_has_svm(env)) {
> env->nested_state->format = KVM_STATE_NESTED_FORMAT_SVM;
> }
Or even force max_nested_state_len to 0 for AMD hosts, so that
kvm_get/put_nested_state are dropped completely.
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 12:45 ` Paolo Bonzini
@ 2019-06-21 12:48 ` Liran Alon
2019-06-21 14:55 ` Paolo Bonzini
0 siblings, 1 reply; 54+ messages in thread
From: Liran Alon @ 2019-06-21 12:48 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 15:45, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 21/06/19 14:29, Liran Alon wrote:
>>> + max_nested_state_len = kvm_max_nested_state_length();
>>> + if (max_nested_state_len > 0) {
>>> + assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data));
>>> + env->nested_state = g_malloc0(max_nested_state_len);
>>> +
>>> + env->nested_state->size = max_nested_state_len;
>>> +
>>> + if (IS_INTEL_CPU(env)) {
>> I think it’s better to change this to: “if (cpu_has_vmx(env))” {
>>
>>> + struct kvm_vmx_nested_state_hdr *vmx_hdr =
>>> + &env->nested_state->hdr.vmx;
>>> +
>>> + env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
>>> + vmx_hdr->vmxon_pa = -1ull;
>>> + vmx_hdr->vmcs12_pa = -1ull;
>>> + }
>>> + }
>> I think we should add here:
>> } else if (cpu_has_svm(env)) {
>> env->nested_state->format = KVM_STATE_NESTED_FORMAT_SVM;
>> }
>
> Or even force max_nested_state_len to 0 for AMD hosts, so that
> kvm_get/put_nested_state are dropped completely.
>
> Paolo
>
On AMD hosts, KVM returns 0 for KVM_CAP_NESTED_STATE because
Kvm-and.ko have kvm_x86_ops->get_nested_state set to NULL.
See kvm_vm_ioctl_check_extension().
I just thought it will be nicer to add what I proposed above as when kernel adds support
for nested state on AMD host, QEMU would maybe just work.
(Because maybe all state required for AMD nSVM is just flags in env->nested_state->flags).
-Liran
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (24 preceding siblings ...)
2019-06-21 11:30 ` [Qemu-devel] [PULL 25/25] hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1 Paolo Bonzini
@ 2019-06-21 13:53 ` no-reply
2019-06-21 14:40 ` Peter Maydell
26 siblings, 0 replies; 54+ messages in thread
From: no-reply @ 2019-06-21 13:53 UTC (permalink / raw)
To: pbonzini; +Cc: qemu-devel
Patchew URL: https://patchew.org/QEMU/1561116620-22245-1-git-send-email-pbonzini@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Subject: [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21
Message-id: 1561116620-22245-1-git-send-email-pbonzini@redhat.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/20190621132324.2165-1-mreitz@redhat.com -> patchew/20190621132324.2165-1-mreitz@redhat.com
Switched to a new branch 'test'
cf66e55 hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1
6c88e3a util/main-loop: Fix incorrect assertion
13371b1 sd: Fix out-of-bounds assertions
0b837d9 target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
1583048 target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD
4e4d392 target/i386: kvm: Add support for save and restore nested state
c952fd2 vmstate: Add support for kernel integer types
be0bc2a linux-headers: sync with latest KVM headers from Linux 5.2
fcc123c target/i386: kvm: Block migration for vCPUs exposed with nested virtualization
ff0e7df target/i386: kvm: Re-inject #DB to guest with updated DR6
b03c3ff target/i386: kvm: Use symbolic constant for #DB/#BP exception constants
67ea33b KVM: Introduce kvm_arch_destroy_vcpu()
9e26b32 target/i386: kvm: Delete VMX migration blocker on vCPU init failure
7020f89 target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY
3a0d495 i386/kvm: add support for Direct Mode for Hyper-V synthetic timers
6a7127a i386/kvm: hv-evmcs requires hv-vapic
c7514b4 i386/kvm: hv-tlbflush/ipi require hv-vpindex
51cdfac i386/kvm: hv-stimer requires hv-time and hv-synic
1f02e73 i386/kvm: implement 'hv-passthrough' mode
aeaf6f8 i386/kvm: document existing Hyper-V enlightenments
82d1c81 i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties()
f5e0932 i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID
b0b89da i386/kvm: convert hyperv enlightenments properties from bools to bits
d49efb1 hax: Honor CPUState::halted
113cfe4 kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del
=== OUTPUT BEGIN ===
1/25 Checking commit 113cfe429a07 (kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del)
2/25 Checking commit d49efb131de1 (hax: Honor CPUState::halted)
WARNING: Block comments use a leading /* on a separate line
#77: FILE: target/i386/hax-all.c:479:
+ /* After a vcpu is halted (either because it is an AP and has just been
WARNING: Block comments use a leading /* on a separate line
#109: FILE: target/i386/hax-all.c:519:
+ /* If this vcpu is halted, we must not ask HAXM to run it. Instead, we
total: 0 errors, 2 warnings, 60 lines checked
Patch 2/25 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/25 Checking commit b0b89daa3c59 (i386/kvm: convert hyperv enlightenments properties from bools to bits)
4/25 Checking commit f5e09320671b (i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID)
5/25 Checking commit 82d1c8147a54 (i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties())
6/25 Checking commit aeaf6f88598f (i386/kvm: document existing Hyper-V enlightenments)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#18:
new file mode 100644
total: 0 errors, 1 warnings, 181 lines checked
Patch 6/25 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/25 Checking commit 1f02e738d953 (i386/kvm: implement 'hv-passthrough' mode)
8/25 Checking commit 51cdfac4fca3 (i386/kvm: hv-stimer requires hv-time and hv-synic)
9/25 Checking commit c7514b44e2d0 (i386/kvm: hv-tlbflush/ipi require hv-vpindex)
10/25 Checking commit 6a7127a97552 (i386/kvm: hv-evmcs requires hv-vapic)
11/25 Checking commit 3a0d49524898 (i386/kvm: add support for Direct Mode for Hyper-V synthetic timers)
12/25 Checking commit 7020f89f2f08 (target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY)
13/25 Checking commit 9e26b329c385 (target/i386: kvm: Delete VMX migration blocker on vCPU init failure)
14/25 Checking commit 67ea33bcc49e (KVM: Introduce kvm_arch_destroy_vcpu())
ERROR: code indent should never use tabs
#61: FILE: target/arm/kvm32.c:245:
+^Ireturn 0;$
ERROR: g_free(NULL) is safe this check is probably not required
#96: FILE: target/i386/kvm.c:1687:
+ if (cpu->kvm_msr_buf) {
+ g_free(cpu->kvm_msr_buf);
total: 2 errors, 0 warnings, 96 lines checked
Patch 14/25 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
15/25 Checking commit b03c3ffbb26e (target/i386: kvm: Use symbolic constant for #DB/#BP exception constants)
16/25 Checking commit ff0e7dfa1207 (target/i386: kvm: Re-inject #DB to guest with updated DR6)
17/25 Checking commit fcc123ca5c8e (target/i386: kvm: Block migration for vCPUs exposed with nested virtualization)
18/25 Checking commit be0bc2a41ca8 (linux-headers: sync with latest KVM headers from Linux 5.2)
19/25 Checking commit c952fd2398ef (vmstate: Add support for kernel integer types)
20/25 Checking commit 4e4d392b5f30 (target/i386: kvm: Add support for save and restore nested state)
ERROR: g_free(NULL) is safe this check is probably not required
#130: FILE: target/i386/kvm.c:1712:
+ if (env->nested_state) {
+ g_free(env->nested_state);
ERROR: spaces required around that '*' (ctx:VxV)
#316: FILE: target/i386/machine.c:935:
+ .subsections = (const VMStateDescription*[]) {
^
ERROR: return is not a function, parentheses are not required
#345: FILE: target/i386/machine.c:964:
+ return (env->nested_state &&
ERROR: spaces required around that '*' (ctx:VxV)
#407: FILE: target/i386/machine.c:1026:
+ .subsections = (const VMStateDescription*[]) {
^
total: 4 errors, 0 warnings, 386 lines checked
Patch 20/25 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
21/25 Checking commit 158304885e51 (target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD)
22/25 Checking commit 0b837d9595d8 (target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities)
23/25 Checking commit 13371b1ad995 (sd: Fix out-of-bounds assertions)
24/25 Checking commit 6c88e3a62741 (util/main-loop: Fix incorrect assertion)
25/25 Checking commit cf66e55c4dbe (hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1)
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/1561116620-22245-1-git-send-email-pbonzini@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
` (25 preceding siblings ...)
2019-06-21 13:53 ` [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 no-reply
@ 2019-06-21 14:40 ` Peter Maydell
26 siblings, 0 replies; 54+ messages in thread
From: Peter Maydell @ 2019-06-21 14:40 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: QEMU Developers
On Fri, 21 Jun 2019 at 12:34, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following changes since commit 33d609990621dea6c7d056c86f707b8811320ac1:
>
> Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-06-18 17:00:52 +0100)
>
> are available in the git repository at:
>
>
> git://github.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 8e8cbed09ad9d577955691b4c061b61b602406d1:
>
> hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1 (2019-06-21 13:25:29 +0200)
>
> ----------------------------------------------------------------
> * Nuke hw_compat_4_0_1 and pc_compat_4_0_1 (Greg)
> * Static analysis fixes (Igor, Lidong)
> * X86 Hyper-V CPUID improvements (Vitaly)
> * X86 nested virt migration (Liran)
> * New MSR-based features (Xiaoyao)
>
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/4.1
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 12:48 ` Liran Alon
@ 2019-06-21 14:55 ` Paolo Bonzini
2019-06-21 15:00 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 14:55 UTC (permalink / raw)
To: Liran Alon; +Cc: qemu-devel
On 21/06/19 14:48, Liran Alon wrote:
>
>
>> On 21 Jun 2019, at 15:45, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> On 21/06/19 14:29, Liran Alon wrote:
>>>> + max_nested_state_len = kvm_max_nested_state_length();
>>>> + if (max_nested_state_len > 0) {
>>>> + assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data));
>>>> + env->nested_state = g_malloc0(max_nested_state_len);
>>>> +
>>>> + env->nested_state->size = max_nested_state_len;
>>>> +
>>>> + if (IS_INTEL_CPU(env)) {
>>> I think it’s better to change this to: “if (cpu_has_vmx(env))” {
>>>
>>>> + struct kvm_vmx_nested_state_hdr *vmx_hdr =
>>>> + &env->nested_state->hdr.vmx;
>>>> +
>>>> + env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
>>>> + vmx_hdr->vmxon_pa = -1ull;
>>>> + vmx_hdr->vmcs12_pa = -1ull;
>>>> + }
>>>> + }
>>> I think we should add here:
>>> } else if (cpu_has_svm(env)) {
>>> env->nested_state->format = KVM_STATE_NESTED_FORMAT_SVM;
>>> }
>>
>> Or even force max_nested_state_len to 0 for AMD hosts, so that
>> kvm_get/put_nested_state are dropped completely.
>>
>> Paolo
>>
>
> On AMD hosts, KVM returns 0 for KVM_CAP_NESTED_STATE because
> Kvm-and.ko have kvm_x86_ops->get_nested_state set to NULL.
> See kvm_vm_ioctl_check_extension().
Yes, now it does, my idea was to force that behavior in QEMU until we
know what SVM support actually looks like.
In principle I don't like the idea of crossing fingers, even though
there's an actual possibility that it could work. But it couldn't be
worse than what we have now, so maybe it *is* actually a good idea, just
with some comment that explains the rationale.
Paolo
> I just thought it will be nicer to add what I proposed above as when kernel adds support
> for nested state on AMD host, QEMU would maybe just work.
> (Because maybe all state required for AMD nSVM is just flags in env->nested_state->flags).
>
> -Liran
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 14:55 ` Paolo Bonzini
@ 2019-06-21 15:00 ` Liran Alon
2019-06-21 15:39 ` Paolo Bonzini
0 siblings, 1 reply; 54+ messages in thread
From: Liran Alon @ 2019-06-21 15:00 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 17:55, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 21/06/19 14:48, Liran Alon wrote:
>>
>>
>>> On 21 Jun 2019, at 15:45, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>
>>> On 21/06/19 14:29, Liran Alon wrote:
>>>>> + max_nested_state_len = kvm_max_nested_state_length();
>>>>> + if (max_nested_state_len > 0) {
>>>>> + assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data));
>>>>> + env->nested_state = g_malloc0(max_nested_state_len);
>>>>> +
>>>>> + env->nested_state->size = max_nested_state_len;
>>>>> +
>>>>> + if (IS_INTEL_CPU(env)) {
>>>> I think it’s better to change this to: “if (cpu_has_vmx(env))” {
>>>>
>>>>> + struct kvm_vmx_nested_state_hdr *vmx_hdr =
>>>>> + &env->nested_state->hdr.vmx;
>>>>> +
>>>>> + env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
>>>>> + vmx_hdr->vmxon_pa = -1ull;
>>>>> + vmx_hdr->vmcs12_pa = -1ull;
>>>>> + }
>>>>> + }
>>>> I think we should add here:
>>>> } else if (cpu_has_svm(env)) {
>>>> env->nested_state->format = KVM_STATE_NESTED_FORMAT_SVM;
>>>> }
>>>
>>> Or even force max_nested_state_len to 0 for AMD hosts, so that
>>> kvm_get/put_nested_state are dropped completely.
>>>
>>> Paolo
>>>
>>
>> On AMD hosts, KVM returns 0 for KVM_CAP_NESTED_STATE because
>> Kvm-and.ko have kvm_x86_ops->get_nested_state set to NULL.
>> See kvm_vm_ioctl_check_extension().
>
> Yes, now it does, my idea was to force that behavior in QEMU until we
> know what SVM support actually looks like.
>
> In principle I don't like the idea of crossing fingers, even though
> there's an actual possibility that it could work. But it couldn't be
> worse than what we have now, so maybe it *is* actually a good idea, just
> with some comment that explains the rationale.
>
> Paolo
Cool.
Are you planning to make those changes when applying/merging or
do you need me to submit a new patch-series version?
Also note my comment on the other patch regarding block migration on AMD vCPU which expose SVM.
-Liran
>
>
>> I just thought it will be nicer to add what I proposed above as when kernel adds support
>> for nested state on AMD host, QEMU would maybe just work.
>> (Because maybe all state required for AMD nSVM is just flags in env->nested_state->flags).
>>
>> -Liran
>>
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 12:39 ` Liran Alon
@ 2019-06-21 15:02 ` Paolo Bonzini
2019-06-21 15:07 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 15:02 UTC (permalink / raw)
To: Liran Alon; +Cc: qemu-devel
On 21/06/19 14:39, Liran Alon wrote:
>> On 21 Jun 2019, at 14:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> From: Liran Alon <liran.alon@oracle.com>
>>
>> Previous commits have added support for migration of nested virtualization
>> workloads. This was done by utilising two new KVM capabilities:
>> KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
>> required in order to correctly migrate such workloads.
>>
>> Therefore, change code to add a migration blocker for vCPUs exposed with
>> Intel VMX or AMD SVM in case one of these kernel capabilities is
>> missing.
>>
>> Signed-off-by: Liran Alon <liran.alon@oracle.com>
>> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
>> Message-Id: <20190619162140.133674-11-liran.alon@oracle.com>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>> target/i386/kvm.c | 9 +++++++--
>> target/i386/machine.c | 2 +-
>> 2 files changed, 8 insertions(+), 3 deletions(-)
>>
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index c931e9d..e4b4f57 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -1640,9 +1640,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
>> !!(c->ecx & CPUID_EXT_SMX);
>> }
>>
>> - if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
>> + if (cpu_has_vmx(env) && !nested_virt_mig_blocker &&
>
> The change here from cpu_has_nested_virt(env) to cpu_has_vmx(env) is not clear.
> We should explicitly explain that it’s because we still wish to preserve backwards-compatability
> to migrating AMD vCPU exposed with SVM.
>
> In addition, commit ("target/i386: kvm: Block migration for vCPUs exposed with nested virtualization")
> doesn’t make sense in case we still want to allow migrating AMD vCPU exposed with SVM.
>
> Since QEMU commit 75d373ef9729 ("target-i386: Disable SVM by default in KVM mode"),
> machine-types since v2.2 don’t expose AMD SVM by default.
> Therefore, my personal opinion on this is that it’s fine to block migration in this case.
I totally missed that commit. My bad.
Actually, now that I think about it SVM *will* have some state while
running in guest mode, namely:
- the NPT page table root
- the L1 CR4.PAE, EFER.LMA and EFER.NXE bits, which determine the format
of the NPT12 page tables
These are covered by the existing vmstate_svm_npt subsection.
On the other hand, the lack of something like VMXON/VMCS12 state means
that AMD already sorta works unless you're migrating while in guest
mode. For Intel, just execute VMXON before migration, and starting any
VM after migration is doomed.
So, overall I prefer not to block migration.
>> + ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
>> error_setg(&nested_virt_mig_blocker,
>> - "Nested virtualization does not support live migration yet");
>> + "Kernel do not provide required capabilities for “
>
> As Maran have noted, we should change this “do not” to “does not”.
> Sorry for my bad English grammer. :)
>
>> + "nested virtualization migration. "
>> + "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
>> + kvm_max_nested_state_length() > 0,
>> + has_exception_payload);
>> r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
>> if (local_err) {
>> error_report_err(local_err);
>> diff --git a/target/i386/machine.c b/target/i386/machine.c
>> index fc49e5a..851b249 100644
>> --- a/target/i386/machine.c
>> +++ b/target/i386/machine.c
>> @@ -233,7 +233,7 @@ static int cpu_pre_save(void *opaque)
>>
>> #ifdef CONFIG_KVM
>> /* Verify we have nested virtualization state from kernel if required */
>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>
> Good catch regarding adding kvm_enabled() here.
Caught by "make check", not by me!
> But I think this should have been added to commit ("target/i386: kvm: Add support for save and restore nested state”).
This commit is where bisection broke. :)
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 15:02 ` Paolo Bonzini
@ 2019-06-21 15:07 ` Liran Alon
2019-06-21 17:27 ` Paolo Bonzini
0 siblings, 1 reply; 54+ messages in thread
From: Liran Alon @ 2019-06-21 15:07 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 18:02, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 21/06/19 14:39, Liran Alon wrote:
>>> On 21 Jun 2019, at 14:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>
>>> From: Liran Alon <liran.alon@oracle.com>
>>>
>>> Previous commits have added support for migration of nested virtualization
>>> workloads. This was done by utilising two new KVM capabilities:
>>> KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
>>> required in order to correctly migrate such workloads.
>>>
>>> Therefore, change code to add a migration blocker for vCPUs exposed with
>>> Intel VMX or AMD SVM in case one of these kernel capabilities is
>>> missing.
>>>
>>> Signed-off-by: Liran Alon <liran.alon@oracle.com>
>>> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
>>> Message-Id: <20190619162140.133674-11-liran.alon@oracle.com>
>>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>>> ---
>>> target/i386/kvm.c | 9 +++++++--
>>> target/i386/machine.c | 2 +-
>>> 2 files changed, 8 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>>> index c931e9d..e4b4f57 100644
>>> --- a/target/i386/kvm.c
>>> +++ b/target/i386/kvm.c
>>> @@ -1640,9 +1640,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
>>> !!(c->ecx & CPUID_EXT_SMX);
>>> }
>>>
>>> - if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
>>> + if (cpu_has_vmx(env) && !nested_virt_mig_blocker &&
>>
>> The change here from cpu_has_nested_virt(env) to cpu_has_vmx(env) is not clear.
>> We should explicitly explain that it’s because we still wish to preserve backwards-compatability
>> to migrating AMD vCPU exposed with SVM.
>>
>> In addition, commit ("target/i386: kvm: Block migration for vCPUs exposed with nested virtualization")
>> doesn’t make sense in case we still want to allow migrating AMD vCPU exposed with SVM.
>>
>> Since QEMU commit 75d373ef9729 ("target-i386: Disable SVM by default in KVM mode"),
>> machine-types since v2.2 don’t expose AMD SVM by default.
>> Therefore, my personal opinion on this is that it’s fine to block migration in this case.
>
> I totally missed that commit. My bad.
>
> Actually, now that I think about it SVM *will* have some state while
> running in guest mode, namely:
>
> - the NPT page table root
>
> - the L1 CR4.PAE, EFER.LMA and EFER.NXE bits, which determine the format
> of the NPT12 page tables
>
> These are covered by the existing vmstate_svm_npt subsection.
>
> On the other hand, the lack of something like VMXON/VMCS12 state means
> that AMD already sorta works unless you're migrating while in guest
> mode. For Intel, just execute VMXON before migration, and starting any
> VM after migration is doomed.
True.
>
> So, overall I prefer not to block migration.
I’m not sure I agree.
It is quite likely that vCPU is currently in guest-mode while you are migrating…
A good hypervisor tries to maximise CPU time to be in guest-mode rather than host-mode. :)
I prefer to block migration and once we formally complete the implementation of SVM nested state,
we can modify QEMU code such that migration of vCPU exposed with SVM will work in case
nested-state states that guest is in host-mode.
>
>>> + ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
>>> error_setg(&nested_virt_mig_blocker,
>>> - "Nested virtualization does not support live migration yet");
>>> + "Kernel do not provide required capabilities for “
>>
>> As Maran have noted, we should change this “do not” to “does not”.
>> Sorry for my bad English grammer. :)
>>
>>> + "nested virtualization migration. "
>>> + "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
>>> + kvm_max_nested_state_length() > 0,
>>> + has_exception_payload);
>>> r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
>>> if (local_err) {
>>> error_report_err(local_err);
>>> diff --git a/target/i386/machine.c b/target/i386/machine.c
>>> index fc49e5a..851b249 100644
>>> --- a/target/i386/machine.c
>>> +++ b/target/i386/machine.c
>>> @@ -233,7 +233,7 @@ static int cpu_pre_save(void *opaque)
>>>
>>> #ifdef CONFIG_KVM
>>> /* Verify we have nested virtualization state from kernel if required */
>>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>>
>> Good catch regarding adding kvm_enabled() here.
>
> Caught by "make check", not by me!
Ah nice to know :) Thanks for the tip.
>
>> But I think this should have been added to commit ("target/i386: kvm: Add support for save and restore nested state”).
>
> This commit is where bisection broke. :)
>
> Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 15:00 ` Liran Alon
@ 2019-06-21 15:39 ` Paolo Bonzini
2019-06-21 15:44 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 15:39 UTC (permalink / raw)
To: Liran Alon; +Cc: qemu-devel
On 21/06/19 17:00, Liran Alon wrote:
> Cool.
> Are you planning to make those changes when applying/merging or
> do you need me to submit a new patch-series version?
> Also note my comment on the other patch regarding block migration on AMD vCPU which expose SVM.
It's already merged, but it's not a big deal since it's only AMD. We
can follow up before release.
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 15:39 ` Paolo Bonzini
@ 2019-06-21 15:44 ` Liran Alon
2019-06-21 16:01 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Liran Alon @ 2019-06-21 15:44 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 18:39, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 21/06/19 17:00, Liran Alon wrote:
>> Cool.
>> Are you planning to make those changes when applying/merging or
>> do you need me to submit a new patch-series version?
>> Also note my comment on the other patch regarding block migration on AMD vCPU which expose SVM.
>
> It's already merged, but it's not a big deal since it's only AMD. We
> can follow up before release.
>
> Paolo
Ok then. We at least now have nVMX migration working in QEMU! :)
I will just submit additional separate patches on top of QEMU master.
Thanks,
-Liran
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state
2019-06-21 15:44 ` Liran Alon
@ 2019-06-21 16:01 ` Liran Alon
0 siblings, 0 replies; 54+ messages in thread
From: Liran Alon @ 2019-06-21 16:01 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 18:44, Liran Alon <liran.alon@oracle.com> wrote:
>
>
>
>> On 21 Jun 2019, at 18:39, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> On 21/06/19 17:00, Liran Alon wrote:
>>> Cool.
>>> Are you planning to make those changes when applying/merging or
>>> do you need me to submit a new patch-series version?
>>> Also note my comment on the other patch regarding block migration on AMD vCPU which expose SVM.
>>
>> It's already merged, but it's not a big deal since it's only AMD. We
>> can follow up before release.
>>
>> Paolo
>
> Ok then. We at least now have nVMX migration working in QEMU! :)
> I will just submit additional separate patches on top of QEMU master.
>
> Thanks,
> -Liran
>
Oh the applied patch-series is not very nice actually…
It seems that some of the commits cannot even compile such as "target/i386: kvm: Block migration for vCPUs exposed with nested virtualization".
You have removed cpu_has_nested_virt(env) from that commit even though it is used…
But as the author of the commit I will be blamed for this broken bisection :(
LOL. Oh well… Mistakes happen. :)
-Liran
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 15:07 ` Liran Alon
@ 2019-06-21 17:27 ` Paolo Bonzini
2019-06-21 17:54 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 17:27 UTC (permalink / raw)
To: Liran Alon; +Cc: qemu-devel
On 21/06/19 17:07, Liran Alon wrote:
>> So, overall I prefer not to block migration.
> I’m not sure I agree.
> It is quite likely that vCPU is currently in guest-mode while you are migrating…
> A good hypervisor tries to maximise CPU time to be in guest-mode rather than host-mode. :)
True, but it is even more likely that they are not using KVM at all and
just happen to have the CPUID flag set. :)
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 17:27 ` Paolo Bonzini
@ 2019-06-21 17:54 ` Liran Alon
0 siblings, 0 replies; 54+ messages in thread
From: Liran Alon @ 2019-06-21 17:54 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
> On 21 Jun 2019, at 20:27, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 21/06/19 17:07, Liran Alon wrote:
>>> So, overall I prefer not to block migration.
>> I’m not sure I agree.
>> It is quite likely that vCPU is currently in guest-mode while you are migrating…
>> A good hypervisor tries to maximise CPU time to be in guest-mode rather than host-mode. :)
>
> True, but it is even more likely that they are not using KVM at all and
> just happen to have the CPUID flag set. :)
>
> Paolo
Since QEMU commit 75d373ef9729 ("target-i386: Disable SVM by default in KVM mode"),
An AMD vCPU that is virtualized by KVM don’t expose SVM by default. Even if you use "-cpu host".
Therefore, it is unlikely that vCPU expose SVM CPUID flag when user is not running any nSVM
workload inside guest. Unless I’m missing something obvious.
Otherwise, I would have agreed with you.
-Liran
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits
2019-06-21 11:29 ` [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits Paolo Bonzini
@ 2019-06-24 11:04 ` Alex Bennée
2019-06-24 12:26 ` Vitaly Kuznetsov
0 siblings, 1 reply; 54+ messages in thread
From: Alex Bennée @ 2019-06-24 11:04 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
Paolo Bonzini <pbonzini@redhat.com> writes:
> From: Vitaly Kuznetsov <vkuznets@redhat.com>
>
> Representing Hyper-V properties as bits will allow us to check features
> and dependencies between them in a natural way.
>
> Suggested-by: Roman Kagan <rkagan@virtuozzo.com>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> Message-Id: <20190517141924.19024-2-vkuznets@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> hw/i386/pc.c | 3 +-
> target/i386/cpu.c | 44 ++++++++++++++++++---------
> target/i386/cpu.h | 37 ++++++++++++++---------
> target/i386/hyperv.c | 2 +-
> target/i386/kvm.c | 83 +++++++++++++++++++++------------------------------
> target/i386/machine.c | 2 +-
> 6 files changed, 91 insertions(+), 80 deletions(-)
>
<snip>
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 0732e05..30cd1a0 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -734,6 +734,22 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
> #define MSR_ARCH_CAP_SKIP_L1DFL_VMENTRY (1U << 3)
> #define MSR_ARCH_CAP_SSB_NO (1U << 4)
>
> +/* Supported Hyper-V Enlightenments */
> +#define HYPERV_FEAT_RELAXED 0
> +#define HYPERV_FEAT_VAPIC 1
> +#define HYPERV_FEAT_TIME 2
> +#define HYPERV_FEAT_CRASH 3
> +#define HYPERV_FEAT_RESET 4
> +#define HYPERV_FEAT_VPINDEX 5
> +#define HYPERV_FEAT_RUNTIME 6
> +#define HYPERV_FEAT_SYNIC 7
> +#define HYPERV_FEAT_STIMER 8
> +#define HYPERV_FEAT_FREQUENCIES 9
> +#define HYPERV_FEAT_REENLIGHTENMENT 10
> +#define HYPERV_FEAT_TLBFLUSH 11
> +#define HYPERV_FEAT_EVMCS 12
> +#define HYPERV_FEAT_IPI 13
> +
> #ifndef HYPERV_SPINLOCK_NEVER_RETRY
> #define HYPERV_SPINLOCK_NEVER_RETRY 0xFFFFFFFF
> #endif
> @@ -1370,23 +1386,11 @@ struct X86CPU {
> CPUNegativeOffsetState neg;
> CPUX86State env;
>
> - bool hyperv_vapic;
> - bool hyperv_relaxed_timing;
> int hyperv_spinlock_attempts;
> char *hyperv_vendor_id;
> - bool hyperv_time;
> - bool hyperv_crash;
> - bool hyperv_reset;
> - bool hyperv_vpindex;
> - bool hyperv_runtime;
> - bool hyperv_synic;
> bool hyperv_synic_kvm_only;
This looks like it's broken the build:
configure' '--without-default-devices' '--disable-user'
with:
/home/alex/lsrc/qemu.git/target/i386/hyperv-stub.c: In function ‘kvm_hv_handle_exit’:
/home/alex/lsrc/qemu.git/target/i386/hyperv-stub.c:18:19: error: ‘X86CPU’ {aka ‘struct X86CPU’} has no member named ‘hyperv_synic’; did you mean ‘hyperv_vendor_id’?
if (!cpu->hyperv_synic) {
^~~~~~~~~~~~
Does the stub need updating for the new flag? I'm a little confused
about the relationship between HyperV and KVM. I thought they were
different hypervisors?
--
Alex Bennée
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits
2019-06-24 11:04 ` Alex Bennée
@ 2019-06-24 12:26 ` Vitaly Kuznetsov
2019-06-24 12:43 ` Alex Bennée
0 siblings, 1 reply; 54+ messages in thread
From: Vitaly Kuznetsov @ 2019-06-24 12:26 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel
Alex Bennée <alex.bennee@linaro.org> writes:
>
> This looks like it's broken the build:
>
> configure' '--without-default-devices' '--disable-user'
>
> with:
>
> /home/alex/lsrc/qemu.git/target/i386/hyperv-stub.c: In function ‘kvm_hv_handle_exit’:
> /home/alex/lsrc/qemu.git/target/i386/hyperv-stub.c:18:19: error: ‘X86CPU’ {aka ‘struct X86CPU’} has no member named ‘hyperv_synic’; did you mean ‘hyperv_vendor_id’?
> if (!cpu->hyperv_synic) {
> ^~~~~~~~~~~~
>
> Does the stub need updating for the new flag? I'm a little confused
> about the relationship between HyperV and KVM. I thought they were
> different hypervisors?
Yes, they are. But to run Windows guests we can pretend being Hyper-V
:-)
I'll send a patch to fix the stub if nobody beats me to it. Thanks!
--
Vitaly
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits
2019-06-24 12:26 ` Vitaly Kuznetsov
@ 2019-06-24 12:43 ` Alex Bennée
0 siblings, 0 replies; 54+ messages in thread
From: Alex Bennée @ 2019-06-24 12:43 UTC (permalink / raw)
To: Vitaly Kuznetsov; +Cc: qemu-devel
Vitaly Kuznetsov <vkuznets@redhat.com> writes:
> Alex Bennée <alex.bennee@linaro.org> writes:
>
>>
>> This looks like it's broken the build:
>>
>> configure' '--without-default-devices' '--disable-user'
>>
>> with:
>>
>> /home/alex/lsrc/qemu.git/target/i386/hyperv-stub.c: In function ‘kvm_hv_handle_exit’:
>> /home/alex/lsrc/qemu.git/target/i386/hyperv-stub.c:18:19: error: ‘X86CPU’ {aka ‘struct X86CPU’} has no member named ‘hyperv_synic’; did you mean ‘hyperv_vendor_id’?
>> if (!cpu->hyperv_synic) {
>> ^~~~~~~~~~~~
>>
>> Does the stub need updating for the new flag? I'm a little confused
>> about the relationship between HyperV and KVM. I thought they were
>> different hypervisors?
>
> Yes, they are. But to run Windows guests we can pretend being Hyper-V
> :-)
>
> I'll send a patch to fix the stub if nobody beats me to it. Thanks!
See:
Subject: [PATCH] target/i386: fix feature check in hyperv-stub.c
Date: Mon, 24 Jun 2019 13:38:35 +0100
Message-Id: <20190624123835.28869-1-alex.bennee@linaro.org>
And let me know if it is correct. I'll include it in testing/next as
it is a build fix.
--
Alex Bennée
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization
2019-06-21 11:30 ` [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization Paolo Bonzini
@ 2019-07-08 18:21 ` Jan Kiszka
2019-07-08 22:26 ` Liran Alon
0 siblings, 1 reply; 54+ messages in thread
From: Jan Kiszka @ 2019-07-08 18:21 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Liran Alon
On 21.06.19 13:30, Paolo Bonzini wrote:
> From: Liran Alon <liran.alon@oracle.com>
>
> Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker")
> added a migration blocker for vCPU exposed with Intel VMX.
> However, migration should also be blocked for vCPU exposed with
> AMD SVM.
>
> Both cases should be blocked because QEMU should extract additional
> vCPU state from KVM that should be migrated as part of vCPU VMState.
> E.g. Whether vCPU is running in guest-mode or host-mode.
>
> Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker")
> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Message-Id: <20190619162140.133674-6-liran.alon@oracle.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> target/i386/cpu.c | 6 ------
> target/i386/cpu.h | 12 ++++++++++++
> target/i386/kvm.c | 14 +++++++-------
> 3 files changed, 19 insertions(+), 13 deletions(-)
>
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index c330fd9..61e44cb 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -5215,12 +5215,6 @@ static int x86_cpu_filter_features(X86CPU *cpu)
> return rv;
> }
>
> -#define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \
> - (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \
> - (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3)
> -#define IS_AMD_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_AMD_1 && \
> - (env)->cpuid_vendor2 == CPUID_VENDOR_AMD_2 && \
> - (env)->cpuid_vendor3 == CPUID_VENDOR_AMD_3)
> static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
> {
> CPUState *cs = CPU(dev);
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 7f48136..bf0c9c2 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -722,6 +722,13 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
>
> #define CPUID_VENDOR_HYGON "HygonGenuine"
>
> +#define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \
> + (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \
> + (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3)
> +#define IS_AMD_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_AMD_1 && \
> + (env)->cpuid_vendor2 == CPUID_VENDOR_AMD_2 && \
> + (env)->cpuid_vendor3 == CPUID_VENDOR_AMD_3)
> +
> #define CPUID_MWAIT_IBE (1U << 1) /* Interrupts can exit capability */
> #define CPUID_MWAIT_EMX (1U << 0) /* enumeration supported */
>
> @@ -1848,6 +1855,11 @@ static inline int32_t x86_get_a20_mask(CPUX86State *env)
> }
> }
>
> +static inline bool cpu_has_vmx(CPUX86State *env)
> +{
> + return env->features[FEAT_1_ECX] & CPUID_EXT_VMX;
> +}
> +
> /* fpu_helper.c */
> void update_fp_status(CPUX86State *env);
> void update_mxcsr_status(CPUX86State *env);
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 9864aa0..f9872f1 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1299,7 +1299,7 @@ static int hyperv_init_vcpu(X86CPU *cpu)
> }
>
> static Error *invtsc_mig_blocker;
> -static Error *vmx_mig_blocker;
> +static Error *nested_virt_mig_blocker;
>
> #define KVM_MAX_CPUID_ENTRIES 100
>
> @@ -1597,13 +1597,13 @@ int kvm_arch_init_vcpu(CPUState *cs)
> !!(c->ecx & CPUID_EXT_SMX);
> }
>
> - if ((env->features[FEAT_1_ECX] & CPUID_EXT_VMX) && !vmx_mig_blocker) {
> - error_setg(&vmx_mig_blocker,
> - "Nested VMX virtualization does not support live migration yet");
> - r = migrate_add_blocker(vmx_mig_blocker, &local_err);
> + if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
This broke bisection (which I currently have to do because this pull manages to
lock up 5.1 host kernels): cpu_has_nested_virt will only come later in this series.
Jan
> + error_setg(&nested_virt_mig_blocker,
> + "Nested virtualization does not support live migration yet");
> + r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
> if (local_err) {
> error_report_err(local_err);
> - error_free(vmx_mig_blocker);
> + error_free(nested_virt_mig_blocker);
> return r;
> }
> }
> @@ -1674,7 +1674,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> fail:
> migrate_del_blocker(invtsc_mig_blocker);
> fail2:
> - migrate_del_blocker(vmx_mig_blocker);
> + migrate_del_blocker(nested_virt_mig_blocker);
>
> return r;
> }
>
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-06-21 11:30 ` [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities Paolo Bonzini
2019-06-21 12:39 ` Liran Alon
@ 2019-07-08 18:31 ` Jan Kiszka
2019-07-08 18:51 ` Jan Kiszka
2019-07-10 14:40 ` Paolo Bonzini
1 sibling, 2 replies; 54+ messages in thread
From: Jan Kiszka @ 2019-07-08 18:31 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Liran Alon
On 21.06.19 13:30, Paolo Bonzini wrote:
> From: Liran Alon <liran.alon@oracle.com>
>
> Previous commits have added support for migration of nested virtualization
> workloads. This was done by utilising two new KVM capabilities:
> KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
> required in order to correctly migrate such workloads.
>
> Therefore, change code to add a migration blocker for vCPUs exposed with
> Intel VMX or AMD SVM in case one of these kernel capabilities is
> missing.
>
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
> Message-Id: <20190619162140.133674-11-liran.alon@oracle.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> target/i386/kvm.c | 9 +++++++--
> target/i386/machine.c | 2 +-
> 2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index c931e9d..e4b4f57 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1640,9 +1640,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
> !!(c->ecx & CPUID_EXT_SMX);
> }
>
> - if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
> + if (cpu_has_vmx(env) && !nested_virt_mig_blocker &&
> + ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
> error_setg(&nested_virt_mig_blocker,
> - "Nested virtualization does not support live migration yet");
> + "Kernel do not provide required capabilities for "
> + "nested virtualization migration. "
> + "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
> + kvm_max_nested_state_length() > 0,
> + has_exception_payload);
> r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
> if (local_err) {
> error_report_err(local_err);
> diff --git a/target/i386/machine.c b/target/i386/machine.c
> index fc49e5a..851b249 100644
> --- a/target/i386/machine.c
> +++ b/target/i386/machine.c
> @@ -233,7 +233,7 @@ static int cpu_pre_save(void *opaque)
>
> #ifdef CONFIG_KVM
> /* Verify we have nested virtualization state from kernel if required */
> - if (cpu_has_nested_virt(env) && !env->nested_state) {
> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
> error_report("Guest enabled nested virtualization but kernel "
> "does not support saving of nested state");
> return -EINVAL;
>
Starting with this commit latest (bisection issue...), running Jailhouse in a
guest first stalls L1 (looks like we lose interrupts), and if I try to reset
that VM, I lose my host as well:
kvm: vmptrld (null)/6eb900000000 failed
kvm: vmclear fail: (null)/6eb900000000
and then things start to lock up because we seem to lose the CPUs the guest was
running on. Once I had this in the logs:
rcu: INFO: rcu_sched detected expedited stalls on CPUs/tasks: { 7-... } 15040
jiffies s: 4673 root: 0x80/.
rcu: blocking rcu_node structures:
Task dump for CPU 7:
qemu-system-x86 R running task 0 17413 17345 0x00000008
Call Trace:
? x86_virt_spec_ctrl+0x7/0xe0
? vmx_vcpu_run.part.0+0x2a4/0xfa0 [kvm_intel]
? vcpu_enter_guest+0x349/0xe80 [kvm]
? kvm_arch_vcpu_ioctl_run+0xff/0x550 [kvm]
? kvm_vcpu_ioctl+0x20d/0x590 [kvm]
? get_futex_key+0x35d/0x3b0
? do_vfs_ioctl+0x447/0x640
? do_futex+0x157/0x1d0
? ksys_ioctl+0x5e/0x90
? __x64_sys_ioctl+0x16/0x20
? do_syscall_64+0x60/0x120
? entry_SYSCALL_64_after_hwframe+0x49/0xbe
This was on a 5.1.16 distro kernel. Currently rebuilding 5.2 vanilla.
Looks like we have up to two critical bugs here...
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-07-08 18:31 ` Jan Kiszka
@ 2019-07-08 18:51 ` Jan Kiszka
2019-07-10 14:40 ` Paolo Bonzini
1 sibling, 0 replies; 54+ messages in thread
From: Jan Kiszka @ 2019-07-08 18:51 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Liran Alon
On 08.07.19 20:31, Jan Kiszka wrote:
>
> On 21.06.19 13:30, Paolo Bonzini wrote:
>> From: Liran Alon <liran.alon@oracle.com>
>>
>> Previous commits have added support for migration of nested virtualization
>> workloads. This was done by utilising two new KVM capabilities:
>> KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
>> required in order to correctly migrate such workloads.
>>
>> Therefore, change code to add a migration blocker for vCPUs exposed with
>> Intel VMX or AMD SVM in case one of these kernel capabilities is
>> missing.
>>
>> Signed-off-by: Liran Alon <liran.alon@oracle.com>
>> Reviewed-by: Maran Wilson <maran.wilson@oracle.com>
>> Message-Id: <20190619162140.133674-11-liran.alon@oracle.com>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>> target/i386/kvm.c | 9 +++++++--
>> target/i386/machine.c | 2 +-
>> 2 files changed, 8 insertions(+), 3 deletions(-)
>>
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index c931e9d..e4b4f57 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -1640,9 +1640,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
>> !!(c->ecx & CPUID_EXT_SMX);
>> }
>>
>> - if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {
>> + if (cpu_has_vmx(env) && !nested_virt_mig_blocker &&
>> + ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
>> error_setg(&nested_virt_mig_blocker,
>> - "Nested virtualization does not support live migration yet");
>> + "Kernel do not provide required capabilities for "
>> + "nested virtualization migration. "
>> + "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
>> + kvm_max_nested_state_length() > 0,
>> + has_exception_payload);
>> r = migrate_add_blocker(nested_virt_mig_blocker, &local_err);
>> if (local_err) {
>> error_report_err(local_err);
>> diff --git a/target/i386/machine.c b/target/i386/machine.c
>> index fc49e5a..851b249 100644
>> --- a/target/i386/machine.c
>> +++ b/target/i386/machine.c
>> @@ -233,7 +233,7 @@ static int cpu_pre_save(void *opaque)
>>
>> #ifdef CONFIG_KVM
>> /* Verify we have nested virtualization state from kernel if required */
>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>> error_report("Guest enabled nested virtualization but kernel "
>> "does not support saving of nested state");
>> return -EINVAL;
>>
>
> Starting with this commit latest (bisection issue...), running Jailhouse in a
> guest first stalls L1 (looks like we lose interrupts), and if I try to reset
> that VM, I lose my host as well:
>
> kvm: vmptrld (null)/6eb900000000 failed
> kvm: vmclear fail: (null)/6eb900000000
>
> and then things start to lock up because we seem to lose the CPUs the guest was
> running on. Once I had this in the logs:
>
> rcu: INFO: rcu_sched detected expedited stalls on CPUs/tasks: { 7-... } 15040
> jiffies s: 4673 root: 0x80/.
> rcu: blocking rcu_node structures:
> Task dump for CPU 7:
> qemu-system-x86 R running task 0 17413 17345 0x00000008
> Call Trace:
> ? x86_virt_spec_ctrl+0x7/0xe0
> ? vmx_vcpu_run.part.0+0x2a4/0xfa0 [kvm_intel]
> ? vcpu_enter_guest+0x349/0xe80 [kvm]
> ? kvm_arch_vcpu_ioctl_run+0xff/0x550 [kvm]
> ? kvm_vcpu_ioctl+0x20d/0x590 [kvm]
> ? get_futex_key+0x35d/0x3b0
> ? do_vfs_ioctl+0x447/0x640
> ? do_futex+0x157/0x1d0
> ? ksys_ioctl+0x5e/0x90
> ? __x64_sys_ioctl+0x16/0x20
> ? do_syscall_64+0x60/0x120
> ? entry_SYSCALL_64_after_hwframe+0x49/0xbe
>
> This was on a 5.1.16 distro kernel. Currently rebuilding 5.2 vanilla.
>
> Looks like we have up to two critical bugs here...
>
> Jan
>
It turns out it's actually patch 20 that introduces the problem. Maybe it is
only the kernel, but I rather suspect a combination of userspace not providing
the right state (specifically on reset) and the kernel accepting that.
Continuing the search on 5.2 now.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization
2019-07-08 18:21 ` Jan Kiszka
@ 2019-07-08 22:26 ` Liran Alon
0 siblings, 0 replies; 54+ messages in thread
From: Liran Alon @ 2019-07-08 22:26 UTC (permalink / raw)
To: Jan Kiszka; +Cc: Paolo Bonzini, qemu-devel
> On 8 Jul 2019, at 21:21, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>
> On 21.06.19 13:30, Paolo Bonzini wrote:
>> From: Liran Alon <liran.alon@oracle.com>
>
> This broke bisection (which I currently have to do because this pull manages to
> lock up 5.1 host kernels): cpu_has_nested_virt will only come later in this series.
>
> Jan
>
That’s correct Jan.
The original patch series I sent upstream didn’t had this problem. (See: https://patchwork.kernel.org/project/kvm/list/?series=135023)
But commit was changed during merge to upstream which mistakenly introduced compile errors to some of the commits.
As this is already merged, there is no way to fix it anymore. But this is just an honest innocent mistake.
-Liran
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-07-08 18:31 ` Jan Kiszka
2019-07-08 18:51 ` Jan Kiszka
@ 2019-07-10 14:40 ` Paolo Bonzini
2019-07-10 16:08 ` Jan Kiszka
1 sibling, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-07-10 14:40 UTC (permalink / raw)
To: Jan Kiszka, qemu-devel; +Cc: Liran Alon
On 08/07/19 20:31, Jan Kiszka wrote:
>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>> error_report("Guest enabled nested virtualization but kernel "
>> "does not support saving of nested state");
>> return -EINVAL;
>>
> Starting with this commit latest (bisection issue...), running Jailhouse in a
> guest first stalls L1 (looks like we lose interrupts), and if I try to reset
> that VM, I lose my host as well:
If 5.2 is still broken, can you gather a dump of KVM_SET_NESTED_STATE's
payload?
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-07-10 14:40 ` Paolo Bonzini
@ 2019-07-10 16:08 ` Jan Kiszka
2019-07-10 16:34 ` Paolo Bonzini
0 siblings, 1 reply; 54+ messages in thread
From: Jan Kiszka @ 2019-07-10 16:08 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Liran Alon
On 10.07.19 16:40, Paolo Bonzini wrote:
> On 08/07/19 20:31, Jan Kiszka wrote:
>>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>>> error_report("Guest enabled nested virtualization but kernel "
>>> "does not support saving of nested state");
>>> return -EINVAL;
>>>
>> Starting with this commit latest (bisection issue...), running Jailhouse in a
>> guest first stalls L1 (looks like we lose interrupts), and if I try to reset
>> that VM, I lose my host as well:
>
> If 5.2 is still broken, can you gather a dump of KVM_SET_NESTED_STATE's
> payload?
savevm or is there a dump function in QEMU or the kernel?
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-07-10 16:08 ` Jan Kiszka
@ 2019-07-10 16:34 ` Paolo Bonzini
2019-07-10 17:15 ` Jan Kiszka
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Bonzini @ 2019-07-10 16:34 UTC (permalink / raw)
To: Jan Kiszka, qemu-devel; +Cc: Liran Alon
On 10/07/19 18:08, Jan Kiszka wrote:
> On 10.07.19 16:40, Paolo Bonzini wrote:
>> On 08/07/19 20:31, Jan Kiszka wrote:
>>>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>>>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>>>> error_report("Guest enabled nested virtualization but kernel "
>>>> "does not support saving of nested state");
>>>> return -EINVAL;
>>>>
>>> Starting with this commit latest (bisection issue...), running Jailhouse in a
>>> guest first stalls L1 (looks like we lose interrupts), and if I try to reset
>>> that VM, I lose my host as well:
>>
>> If 5.2 is still broken, can you gather a dump of KVM_SET_NESTED_STATE's
>> payload?
>
> savevm or is there a dump function in QEMU or the kernel?
There is qemu_hexdump.
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-07-10 16:34 ` Paolo Bonzini
@ 2019-07-10 17:15 ` Jan Kiszka
2019-07-10 17:18 ` Jan Kiszka
0 siblings, 1 reply; 54+ messages in thread
From: Jan Kiszka @ 2019-07-10 17:15 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Liran Alon
On 10.07.19 18:34, Paolo Bonzini wrote:
> On 10/07/19 18:08, Jan Kiszka wrote:
>> On 10.07.19 16:40, Paolo Bonzini wrote:
>>> On 08/07/19 20:31, Jan Kiszka wrote:
>>>>> - if (cpu_has_nested_virt(env) && !env->nested_state) {
>>>>> + if (kvm_enabled() && cpu_has_vmx(env) && !env->nested_state) {
>>>>> error_report("Guest enabled nested virtualization but kernel "
>>>>> "does not support saving of nested state");
>>>>> return -EINVAL;
>>>>>
>>>> Starting with this commit latest (bisection issue...), running Jailhouse in a
>>>> guest first stalls L1 (looks like we lose interrupts), and if I try to reset
>>>> that VM, I lose my host as well:
>>>
>>> If 5.2 is still broken, can you gather a dump of KVM_SET_NESTED_STATE's
>>> payload?
>>
>> savevm or is there a dump function in QEMU or the kernel?
>
> There is qemu_hexdump.
>
Here are two states taken during the vmport loop:
2: nested_state: 0000: 01 00 00 00 80 10 00 00 00 00 24 3a 00 00 00 00 ..........$:....
2: nested_state: 0010: 00 10 24 3a 00 00 00 00 00 00 00 00 00 00 00 00 ..$:............
2: nested_state: 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0080: d0 7e e5 11 00 00 00 00 01 00 00 00 00 00 00 00 .~..............
2: nested_state: 0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00a0: 00 00 00 00 00 00 00 00 00 70 25 3a 00 00 00 00 .........p%:....
2: nested_state: 00b0: 00 80 25 3a 00 00 00 00 00 f0 00 3a 00 00 00 00 ..%:.......:....
2: nested_state: 00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00e0: 00 00 00 00 00 00 00 00 00 50 01 3a 00 00 00 00 .........P.:....
2: nested_state: 00f0: 00 00 00 00 00 00 00 00 1e e0 22 3a 00 00 00 00 ..........":....
2: nested_state: 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0120: 00 00 00 00 00 00 00 00 10 00 00 00 01 00 00 00 ................
2: nested_state: 0130: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 ................
2: nested_state: 0140: 06 01 07 00 06 01 07 00 01 0d 00 00 00 00 00 00 ................
2: nested_state: 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0180: 06 01 07 00 00 00 00 00 00 05 00 00 00 00 00 00 ................
2: nested_state: 0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01d0: 00 00 00 00 00 00 00 00 30 00 00 60 ff ff ff ff ........0..`....
2: nested_state: 01e0: 00 00 00 00 00 00 00 00 33 00 05 80 00 00 00 00 ........3.......
2: nested_state: 01f0: a0 26 04 00 00 00 00 00 00 00 00 00 00 00 00 00 .&..............
2: nested_state: 0200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0220: 00 00 00 00 00 00 00 00 33 00 05 80 00 00 00 00 ........3.......
2: nested_state: 0230: 00 20 6f 32 00 00 00 00 a0 26 04 00 00 00 00 00 . o2.....&......
2: nested_state: 0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0260: 00 97 c4 ec b4 7f 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0270: 00 00 00 00 00 00 00 00 00 90 06 00 00 fe ff ff ................
2: nested_state: 0280: 00 70 06 00 00 fe ff ff 00 00 00 00 00 fe ff ff .p..............
2: nested_state: 0290: 00 04 00 00 00 00 00 00 88 33 16 8a fd 7f 00 00 .........3......
2: nested_state: 02a0: e4 d4 70 e6 b4 7f 00 00 17 32 00 00 00 00 00 00 ..p......2......
2: nested_state: 02b0: 00 00 00 00 00 00 00 00 00 82 06 00 00 fe ff ff ................
2: nested_state: 02c0: 80 11 a0 81 ff ff ff ff 3b 00 01 80 00 00 00 00 ........;.......
2: nested_state: 02d0: 00 20 24 3a 00 00 00 00 20 20 04 00 00 00 00 00 . $:.... ......
2: nested_state: 02e0: 00 00 00 00 00 00 00 00 00 00 d0 3f 80 88 ff ff ...........?....
2: nested_state: 02f0: 00 00 00 00 00 00 00 00 80 02 01 f0 ff ff ff ff ................
2: nested_state: 0300: 80 90 01 f0 ff ff ff ff 00 00 00 00 00 00 00 00 ................
2: nested_state: 0310: 00 00 00 00 00 00 00 00 00 10 01 00 80 00 00 00 ................
2: nested_state: 0320: d4 32 00 f0 ff ff ff ff 00 00 00 00 00 00 00 00 .2..............
2: nested_state: 0330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0360: 00 00 00 00 00 00 00 00 1e 00 00 00 72 61 00 96 ............ra..
2: nested_state: 0370: 02 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0380: ff 6f 3f 00 00 00 00 00 00 00 00 00 ff d3 00 00 .o?.............
2: nested_state: 0390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03a0: 00 00 00 00 83 00 10 00 00 00 00 00 20 00 00 00 ............ ...
2: nested_state: 03b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03c0: 02 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ................
2: nested_state: 03d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ................
2: nested_state: 03e0: 00 00 00 00 6f 20 00 00 7f 00 00 00 ff 0f 00 00 ....o ..........
2: nested_state: 03f0: 00 c0 01 00 fb a0 00 00 f3 c0 00 00 00 c0 01 00 ................
2: nested_state: 0400: 00 c0 01 00 00 c0 01 00 00 00 01 00 8b 00 00 00 ................
2: nested_state: 0410: 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 00 ................
2: nested_state: 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0440: 00 00 00 00 00 00 33 00 2b 00 00 00 00 00 00 00 ......3.+.......
2: nested_state: 0450: 00 00 40 00 00 00 00 00 08 00 00 00 00 00 00 00 ..@.............
2: nested_state: 0460: 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0000: 01 00 00 00 80 10 00 00 00 00 24 3a 00 00 00 00 ..........$:....
2: nested_state: 0010: 00 10 24 3a 00 00 00 00 00 00 00 00 00 00 00 00 ..$:............
2: nested_state: 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0080: d0 7e e5 11 00 00 00 00 01 00 00 00 00 00 00 00 .~..............
2: nested_state: 0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00a0: 00 00 00 00 00 00 00 00 00 70 25 3a 00 00 00 00 .........p%:....
2: nested_state: 00b0: 00 80 25 3a 00 00 00 00 00 f0 00 3a 00 00 00 00 ..%:.......:....
2: nested_state: 00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00e0: 00 00 00 00 00 00 00 00 00 50 01 3a 00 00 00 00 .........P.:....
2: nested_state: 00f0: 00 00 00 00 00 00 00 00 1e e0 22 3a 00 00 00 00 ..........":....
2: nested_state: 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0120: 00 00 00 00 00 00 00 00 10 00 00 00 01 00 00 00 ................
2: nested_state: 0130: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 ................
2: nested_state: 0140: 06 01 07 00 06 01 07 00 01 0d 00 00 00 00 00 00 ................
2: nested_state: 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0170: 18 f0 fe f2 fc 3f 00 00 00 00 00 00 00 00 00 00 .....?..........
2: nested_state: 0180: 06 01 07 00 00 00 00 00 00 05 00 00 00 00 00 00 ................
2: nested_state: 0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01d0: 00 00 00 00 00 00 00 00 30 00 00 60 ff ff ff ff ........0..`....
2: nested_state: 01e0: 00 00 00 00 00 00 00 00 33 00 05 80 00 00 00 00 ........3.......
2: nested_state: 01f0: a0 26 04 00 00 00 00 00 00 00 00 00 00 00 00 00 .&..............
2: nested_state: 0200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0220: 00 00 00 00 00 00 00 00 33 00 05 80 00 00 00 00 ........3.......
2: nested_state: 0230: 00 20 6f 32 00 00 00 00 a0 26 04 00 00 00 00 00 . o2.....&......
2: nested_state: 0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0260: 00 97 c4 ec b4 7f 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0270: 00 00 00 00 00 00 00 00 00 90 06 00 00 fe ff ff ................
2: nested_state: 0280: 00 70 06 00 00 fe ff ff 00 00 00 00 00 fe ff ff .p..............
2: nested_state: 0290: 00 04 00 00 00 00 00 00 88 33 16 8a fd 7f 00 00 .........3......
2: nested_state: 02a0: e4 d4 70 e6 b4 7f 00 00 17 32 00 00 00 00 00 00 ..p......2......
2: nested_state: 02b0: 00 00 00 00 00 00 00 00 00 82 06 00 00 fe ff ff ................
2: nested_state: 02c0: 80 11 a0 81 ff ff ff ff 3b 00 01 80 00 00 00 00 ........;.......
2: nested_state: 02d0: 00 20 24 3a 00 00 00 00 20 20 04 00 00 00 00 00 . $:.... ......
2: nested_state: 02e0: 00 00 00 00 00 00 00 00 00 00 d0 3f 80 88 ff ff ...........?....
2: nested_state: 02f0: 00 00 00 00 00 00 00 00 80 02 01 f0 ff ff ff ff ................
2: nested_state: 0300: 80 90 01 f0 ff ff ff ff 00 00 00 00 00 00 00 00 ................
2: nested_state: 0310: 00 00 00 00 00 00 00 00 00 10 01 00 80 00 00 00 ................
2: nested_state: 0320: d4 32 00 f0 ff ff ff ff 00 00 00 00 00 00 00 00 .2..............
2: nested_state: 0330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0360: 00 00 00 00 00 00 00 00 1e 00 00 00 72 61 00 96 ............ra..
2: nested_state: 0370: 02 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0380: ff 6f 3f 00 00 00 00 00 00 00 00 00 ff d3 00 00 .o?.............
2: nested_state: 0390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03a0: 00 00 00 00 83 00 10 00 00 00 00 00 20 00 00 00 ............ ...
2: nested_state: 03b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03c0: 02 00 00 00 00 00 00 00 ff ff 0f 00 ff ff ff ff ................
2: nested_state: 03d0: ff ff ff ff ff ff 0f 00 ff ff 0f 00 ff ff 0f 00 ................
2: nested_state: 03e0: 00 00 00 00 6f 20 00 00 7f 00 00 00 ff 0f 00 00 ....o ..........
2: nested_state: 03f0: 00 00 01 00 fb a0 00 00 f3 c0 00 00 00 00 01 00 ................
2: nested_state: 0400: 00 00 01 00 00 00 01 00 00 00 01 00 8b 00 00 00 ................
2: nested_state: 0410: 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 00 ................
2: nested_state: 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0440: 00 00 00 00 00 00 33 00 2b 00 00 00 00 00 00 00 ......3.+.......
2: nested_state: 0450: 00 00 40 00 00 00 00 00 08 00 00 00 00 00 00 00 ..@.............
2: nested_state: 0460: 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
I will try to catch the reset thing as well.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities
2019-07-10 17:15 ` Jan Kiszka
@ 2019-07-10 17:18 ` Jan Kiszka
0 siblings, 0 replies; 54+ messages in thread
From: Jan Kiszka @ 2019-07-10 17:18 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Liran Alon
On 10.07.19 19:15, Jan Kiszka wrote:
> I will try to catch the reset thing as well.
>
And here are the state of the 4 cpus when issuing a reset while running an L2 guest (qemu/kvm).
(qemu) system_reset
0: nested_state: 0000: 00 00 00 00 80 00 00 00 00 e0 e9 2e 00 00 00 00 ................
0: nested_state: 0010: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 ................
0: nested_state: 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 02a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 02b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 02c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 02d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 02e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 02f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 03a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 03b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 03c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 03d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 03e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 03f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 04a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 04b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 04c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 04d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 04e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 04f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 05a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 05b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 05c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 05d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 05e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 05f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 06a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 06b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 06c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 06d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 06e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 06f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 07a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 07b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 07c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 07d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 07e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 07f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 08b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 08d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 08e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 08f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 09a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 09b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 09c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 09d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 09e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 09f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 0ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 10a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 10b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 10c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 10d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 10e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 10f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 11a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 11b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 11c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 11d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 11e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 11f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 12a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 12b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 12c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 12d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 12e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 12f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 13a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 13b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 13c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 13d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 13e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 13f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 14a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 14b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 14c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 14d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 14e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 14f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 15a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 15b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 15c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 15d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 15e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 15f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 16a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 16b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 16c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 16d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 16e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 16f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 17a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 17b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 17c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 17d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 17e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 17f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 18a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 18b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 18c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 18d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 18e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 18f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 19a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 19b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 19c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 19d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 19e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 19f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 1ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0: nested_state: 2070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0000: 00 00 00 00 80 10 00 00 00 d0 e9 2e 00 00 00 00 ................
1: nested_state: 0010: 00 90 c4 34 00 00 00 00 00 00 00 00 00 00 00 00 ...4............
1: nested_state: 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0080: d0 7e e5 11 00 00 00 00 01 00 00 00 00 00 00 00 .~..............
1: nested_state: 0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 00b0: 00 00 00 00 00 00 00 00 00 20 e4 35 00 00 00 00 ......... .5....
1: nested_state: 00c0: 00 00 00 00 00 00 00 00 b0 f9 3e 31 00 00 00 00 ..........>1....
1: nested_state: 00d0: 20 f9 3e 31 00 00 00 00 48 f5 31 1a f3 ff ff ff .>1....H.1.....
1: nested_state: 00e0: 00 f0 f2 13 00 00 00 00 00 f0 dc 27 00 00 00 00 ...........'....
1: nested_state: 00f0: 00 00 00 00 00 00 00 00 5e b0 cd 34 00 00 00 00 ........^..4....
1: nested_state: 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0120: 00 00 00 00 00 00 00 00 00 8f 0b 00 00 00 00 00 ................
1: nested_state: 0130: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 ................
1: nested_state: 0140: 06 04 07 00 06 04 07 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0150: 00 00 00 00 00 00 00 00 19 b0 00 00 ff 3f 00 00 .............?..
1: nested_state: 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0180: 06 01 07 00 06 01 07 00 01 0d 00 00 00 00 00 00 ................
1: nested_state: 0190: 00 00 00 00 00 00 00 00 00 10 22 34 00 00 00 00 .........."4....
1: nested_state: 01a0: 00 10 22 34 00 00 00 00 00 00 00 00 00 00 00 00 .."4............
1: nested_state: 01b0: 00 00 00 00 00 00 00 00 00 10 84 33 00 00 00 00 ...........3....
1: nested_state: 01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 01d0: 00 00 00 00 00 00 00 00 f7 ff ff ff ff ff ff ff ................
1: nested_state: 01e0: 71 e8 ff ff ff ff ff ff 10 00 00 00 00 00 00 00 q...............
1: nested_state: 01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0220: 00 00 00 00 00 00 00 00 30 00 00 00 00 00 00 00 ........0.......
1: nested_state: 0230: 00 00 00 00 00 00 00 00 00 20 00 00 00 00 00 00 ......... ......
1: nested_state: 0240: 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 ................
1: nested_state: 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0280: 80 6a 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 .j..............
1: nested_state: 0290: 00 04 00 00 00 00 00 00 a0 6f 00 00 00 00 00 00 .........o......
1: nested_state: 02a0: 7d bf 00 00 00 00 00 00 46 02 00 00 00 00 00 00 }.......F.......
1: nested_state: 02b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 02c0: 00 00 00 00 00 00 00 00 33 00 05 80 00 00 00 00 ........3.......
1: nested_state: 02d0: 00 70 0c 14 00 00 00 00 a0 26 04 00 00 00 00 00 .p.......&......
1: nested_state: 02e0: 00 47 ac bc 56 7f 00 00 00 00 c8 3f 80 88 ff ff .G..V......?....
1: nested_state: 02f0: 00 60 03 00 00 fe ff ff 00 40 03 00 00 fe ff ff .`.......@......
1: nested_state: 0300: 00 00 00 00 00 fe ff ff 00 52 03 00 00 fe ff ff .........R......
1: nested_state: 0310: 80 11 a0 81 ff ff ff ff a0 bc d9 00 00 c9 ff ff ................
1: nested_state: 0320: 10 62 28 a0 ff ff ff ff 00 00 00 00 00 00 00 00 .b(.............
1: nested_state: 0330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0360: 00 00 00 00 00 00 00 00 3f 00 00 00 fa 6d a0 b5 ........?....m..
1: nested_state: 0370: 42 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 B...............
1: nested_state: 0380: ff ef 2b 00 00 00 00 00 00 00 00 00 ff d1 00 00 ..+.............
1: nested_state: 0390: 00 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 03a0: 00 00 00 00 e3 20 02 00 00 00 00 00 0c 00 00 00 ..... ..........
1: nested_state: 03b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 03c0: 01 00 00 00 00 00 00 00 ff ff 00 00 ff ff 00 00 ................
1: nested_state: 03d0: ff ff 00 00 ff ff 00 00 ff ff 00 00 ff ff 00 00 ................
1: nested_state: 03e0: ff ff 00 00 ff ff 00 00 37 00 00 00 ff 03 00 00 ........7.......
1: nested_state: 03f0: 93 00 00 00 9b 00 00 00 93 00 00 00 93 00 00 00 ................
1: nested_state: 0400: 93 00 00 00 93 00 00 00 82 00 00 00 8b 00 00 00 ................
1: nested_state: 0410: 00 00 00 00 00 00 00 00 00 00 00 00 10 00 00 00 ................
1: nested_state: 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0440: 01 00 00 00 00 00 00 f0 00 00 00 00 00 00 00 00 ................
1: nested_state: 0450: 00 00 00 00 00 00 00 00 10 00 18 00 00 00 00 00 ................
1: nested_state: 0460: 00 00 40 00 ff 01 00 00 00 00 00 00 00 00 00 00 ..@.............
1: nested_state: 0470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 04a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 04b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 04c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 04d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 04e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 04f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 05a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 05b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 05c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 05d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 05e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 05f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 06a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 06b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 06c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 06d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 06e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 06f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 07a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 07b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 07c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 07d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 07e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 07f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 08b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 08d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 08e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 08f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 09a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 09b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 09c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 09d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 09e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 09f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 0ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 10a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 10b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 10c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 10d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 10e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 10f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 11a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 11b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 11c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 11d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 11e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 11f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 12a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 12b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 12c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 12d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 12e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 12f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 13a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 13b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 13c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 13d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 13e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 13f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 14a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 14b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 14c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 14d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 14e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 14f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 15a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 15b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 15c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 15d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 15e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 15f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 16a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 16b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 16c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 16d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 16e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 16f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 17a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 17b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 17c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 17d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 17e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 17f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 18a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 18b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 18c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 18d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 18e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 18f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 19a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 19b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 19c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 19d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 19e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 19f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 1ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
1: nested_state: 2070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0000: 00 00 00 00 80 00 00 00 00 c0 e9 2e 00 00 00 00 ................
2: nested_state: 0010: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 ................
2: nested_state: 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 02a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 02b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 02c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 02d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 02e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 02f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 03f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 04f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 05f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 06f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 07f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 08f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 09f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 0ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 10f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 11f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 12f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 13f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 14f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 15f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 16f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 17f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 18f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 19f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 1ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2: nested_state: 2070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0000: 00 00 00 00 80 00 00 00 00 b0 e9 2e 00 00 00 00 ................
3: nested_state: 0010: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 ................
3: nested_state: 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 02a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 02b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 02c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 02d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 02e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 02f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 03a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 03b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 03c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 03d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 03e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 03f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 04a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 04b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 04c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 04d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 04e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 04f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 05a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 05b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 05c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 05d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 05e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 05f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 06a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 06b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 06c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 06d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 06e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 06f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 07a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 07b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 07c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 07d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 07e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 07f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 08b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 08d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 08e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 08f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 09a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 09b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 09c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 09d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 09e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 09f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 0ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 10a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 10b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 10c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 10d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 10e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 10f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 11a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 11b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 11c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 11d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 11e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 11f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 12a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 12b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 12c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 12d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 12e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 12f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 13a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 13b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 13c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 13d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 13e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 13f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 14a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 14b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 14c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 14d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 14e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 14f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 15a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 15b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 15c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 15d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 15e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 15f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 16a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 16b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 16c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 16d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 16e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 16f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 17a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 17b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 17c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 17d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 17e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 17f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 18a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 18b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 18c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 18d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 18e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 18f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 19a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 19b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 19c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 19d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 19e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 19f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 1ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3: nested_state: 2070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
CPU 1 is likely interesting. I didn't confirm if this state would have crashed because I didn't submit the states, just left QEMU after the last CPU.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 54+ messages in thread
* [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits
2019-06-21 1:42 [Qemu-devel] [PULL " Paolo Bonzini
@ 2019-06-21 1:42 ` Paolo Bonzini
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Bonzini @ 2019-06-21 1:42 UTC (permalink / raw)
To: qemu-devel; +Cc: Vitaly Kuznetsov
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Representing Hyper-V properties as bits will allow us to check features
and dependencies between them in a natural way.
Suggested-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20190517141924.19024-2-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/i386/pc.c | 3 +-
target/i386/cpu.c | 44 ++++++++++++++++++---------
target/i386/cpu.h | 37 ++++++++++++++---------
target/i386/hyperv.c | 2 +-
target/i386/kvm.c | 83 +++++++++++++++++++++------------------------------
target/i386/machine.c | 2 +-
6 files changed, 91 insertions(+), 80 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2c5446b..e41192b 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2386,7 +2386,8 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
}
cpu->thread_id = topo.smt_id;
- if (cpu->hyperv_vpindex && !kvm_hv_vpindex_settable()) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) &&
+ !kvm_hv_vpindex_settable()) {
error_setg(errp, "kernel doesn't allow setting HyperV VP_INDEX");
return;
}
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index fbed2eb..e90c1ac 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5853,21 +5853,37 @@ static Property x86_cpu_properties[] = {
#endif
DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID),
DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
+
{ .name = "hv-spinlocks", .info = &qdev_prop_spinlocks },
- DEFINE_PROP_BOOL("hv-relaxed", X86CPU, hyperv_relaxed_timing, false),
- DEFINE_PROP_BOOL("hv-vapic", X86CPU, hyperv_vapic, false),
- DEFINE_PROP_BOOL("hv-time", X86CPU, hyperv_time, false),
- DEFINE_PROP_BOOL("hv-crash", X86CPU, hyperv_crash, false),
- DEFINE_PROP_BOOL("hv-reset", X86CPU, hyperv_reset, false),
- DEFINE_PROP_BOOL("hv-vpindex", X86CPU, hyperv_vpindex, false),
- DEFINE_PROP_BOOL("hv-runtime", X86CPU, hyperv_runtime, false),
- DEFINE_PROP_BOOL("hv-synic", X86CPU, hyperv_synic, false),
- DEFINE_PROP_BOOL("hv-stimer", X86CPU, hyperv_stimer, false),
- DEFINE_PROP_BOOL("hv-frequencies", X86CPU, hyperv_frequencies, false),
- DEFINE_PROP_BOOL("hv-reenlightenment", X86CPU, hyperv_reenlightenment, false),
- DEFINE_PROP_BOOL("hv-tlbflush", X86CPU, hyperv_tlbflush, false),
- DEFINE_PROP_BOOL("hv-evmcs", X86CPU, hyperv_evmcs, false),
- DEFINE_PROP_BOOL("hv-ipi", X86CPU, hyperv_ipi, false),
+ DEFINE_PROP_BIT64("hv-relaxed", X86CPU, hyperv_features,
+ HYPERV_FEAT_RELAXED, 0),
+ DEFINE_PROP_BIT64("hv-vapic", X86CPU, hyperv_features,
+ HYPERV_FEAT_VAPIC, 0),
+ DEFINE_PROP_BIT64("hv-time", X86CPU, hyperv_features,
+ HYPERV_FEAT_TIME, 0),
+ DEFINE_PROP_BIT64("hv-crash", X86CPU, hyperv_features,
+ HYPERV_FEAT_CRASH, 0),
+ DEFINE_PROP_BIT64("hv-reset", X86CPU, hyperv_features,
+ HYPERV_FEAT_RESET, 0),
+ DEFINE_PROP_BIT64("hv-vpindex", X86CPU, hyperv_features,
+ HYPERV_FEAT_VPINDEX, 0),
+ DEFINE_PROP_BIT64("hv-runtime", X86CPU, hyperv_features,
+ HYPERV_FEAT_RUNTIME, 0),
+ DEFINE_PROP_BIT64("hv-synic", X86CPU, hyperv_features,
+ HYPERV_FEAT_SYNIC, 0),
+ DEFINE_PROP_BIT64("hv-stimer", X86CPU, hyperv_features,
+ HYPERV_FEAT_STIMER, 0),
+ DEFINE_PROP_BIT64("hv-frequencies", X86CPU, hyperv_features,
+ HYPERV_FEAT_FREQUENCIES, 0),
+ DEFINE_PROP_BIT64("hv-reenlightenment", X86CPU, hyperv_features,
+ HYPERV_FEAT_REENLIGHTENMENT, 0),
+ DEFINE_PROP_BIT64("hv-tlbflush", X86CPU, hyperv_features,
+ HYPERV_FEAT_TLBFLUSH, 0),
+ DEFINE_PROP_BIT64("hv-evmcs", X86CPU, hyperv_features,
+ HYPERV_FEAT_EVMCS, 0),
+ DEFINE_PROP_BIT64("hv-ipi", X86CPU, hyperv_features,
+ HYPERV_FEAT_IPI, 0),
+
DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true),
DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false),
DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0732e05..30cd1a0 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -734,6 +734,22 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
#define MSR_ARCH_CAP_SKIP_L1DFL_VMENTRY (1U << 3)
#define MSR_ARCH_CAP_SSB_NO (1U << 4)
+/* Supported Hyper-V Enlightenments */
+#define HYPERV_FEAT_RELAXED 0
+#define HYPERV_FEAT_VAPIC 1
+#define HYPERV_FEAT_TIME 2
+#define HYPERV_FEAT_CRASH 3
+#define HYPERV_FEAT_RESET 4
+#define HYPERV_FEAT_VPINDEX 5
+#define HYPERV_FEAT_RUNTIME 6
+#define HYPERV_FEAT_SYNIC 7
+#define HYPERV_FEAT_STIMER 8
+#define HYPERV_FEAT_FREQUENCIES 9
+#define HYPERV_FEAT_REENLIGHTENMENT 10
+#define HYPERV_FEAT_TLBFLUSH 11
+#define HYPERV_FEAT_EVMCS 12
+#define HYPERV_FEAT_IPI 13
+
#ifndef HYPERV_SPINLOCK_NEVER_RETRY
#define HYPERV_SPINLOCK_NEVER_RETRY 0xFFFFFFFF
#endif
@@ -1370,23 +1386,11 @@ struct X86CPU {
CPUNegativeOffsetState neg;
CPUX86State env;
- bool hyperv_vapic;
- bool hyperv_relaxed_timing;
int hyperv_spinlock_attempts;
char *hyperv_vendor_id;
- bool hyperv_time;
- bool hyperv_crash;
- bool hyperv_reset;
- bool hyperv_vpindex;
- bool hyperv_runtime;
- bool hyperv_synic;
bool hyperv_synic_kvm_only;
- bool hyperv_stimer;
- bool hyperv_frequencies;
- bool hyperv_reenlightenment;
- bool hyperv_tlbflush;
- bool hyperv_evmcs;
- bool hyperv_ipi;
+ uint64_t hyperv_features;
+
bool check_cpuid;
bool enforce_cpuid;
bool expose_kvm;
@@ -1906,4 +1910,9 @@ void x86_cpu_xrstor_all_areas(X86CPU *cpu, const X86XSaveArea *buf);
void x86_cpu_xsave_all_areas(X86CPU *cpu, X86XSaveArea *buf);
void x86_update_hflags(CPUX86State* env);
+static inline bool hyperv_feat_enabled(X86CPU *cpu, int feat)
+{
+ return !!(cpu->hyperv_features & BIT(feat));
+}
+
#endif /* I386_CPU_H */
diff --git a/target/i386/hyperv.c b/target/i386/hyperv.c
index b264a28..26efc1e 100644
--- a/target/i386/hyperv.c
+++ b/target/i386/hyperv.c
@@ -52,7 +52,7 @@ int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit)
switch (exit->type) {
case KVM_EXIT_HYPERV_SYNIC:
- if (!cpu->hyperv_synic) {
+ if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
return -1;
}
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 6899061..b34eb81 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -634,28 +634,12 @@ unsigned long kvm_arch_vcpu_id(CPUState *cs)
#define KVM_CPUID_SIGNATURE_NEXT 0x40000100
#endif
-static bool hyperv_hypercall_available(X86CPU *cpu)
-{
- return cpu->hyperv_vapic ||
- (cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY);
-}
-
static bool hyperv_enabled(X86CPU *cpu)
{
CPUState *cs = CPU(cpu);
return kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0 &&
- (hyperv_hypercall_available(cpu) ||
- cpu->hyperv_time ||
- cpu->hyperv_relaxed_timing ||
- cpu->hyperv_crash ||
- cpu->hyperv_reset ||
- cpu->hyperv_vpindex ||
- cpu->hyperv_runtime ||
- cpu->hyperv_synic ||
- cpu->hyperv_stimer ||
- cpu->hyperv_reenlightenment ||
- cpu->hyperv_tlbflush ||
- cpu->hyperv_ipi);
+ ((cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY) ||
+ cpu->hyperv_features);
}
static int kvm_arch_set_tsc_khz(CPUState *cs)
@@ -704,14 +688,14 @@ static int hyperv_handle_properties(CPUState *cs)
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
- if (cpu->hyperv_relaxed_timing) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RELAXED)) {
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_APIC_ACCESS_AVAILABLE;
}
- if (cpu->hyperv_time) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) <= 0) {
fprintf(stderr, "Hyper-V clocksources "
"(requested by 'hv-time' cpu flag) "
@@ -722,7 +706,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_TIME_REF_COUNT_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_REFERENCE_TSC_AVAILABLE;
}
- if (cpu->hyperv_frequencies) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_FREQUENCIES)) {
if (!has_msr_hv_frequencies) {
fprintf(stderr, "Hyper-V frequency MSRs "
"(requested by 'hv-frequencies' cpu flag) "
@@ -732,7 +716,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_FREQUENCY_MSRS;
env->features[FEAT_HYPERV_EDX] |= HV_FREQUENCY_MSRS_AVAILABLE;
}
- if (cpu->hyperv_crash) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_CRASH)) {
if (!has_msr_hv_crash) {
fprintf(stderr, "Hyper-V crash MSRs "
"(requested by 'hv-crash' cpu flag) "
@@ -741,7 +725,7 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EDX] |= HV_GUEST_CRASH_MSR_AVAILABLE;
}
- if (cpu->hyperv_reenlightenment) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
if (!has_msr_hv_reenlightenment) {
fprintf(stderr,
"Hyper-V Reenlightenment MSRs "
@@ -752,7 +736,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_REENLIGHTENMENTS_CONTROL;
}
env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
- if (cpu->hyperv_reset) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RESET)) {
if (!has_msr_hv_reset) {
fprintf(stderr, "Hyper-V reset MSR "
"(requested by 'hv-reset' cpu flag) "
@@ -761,7 +745,7 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EAX] |= HV_RESET_AVAILABLE;
}
- if (cpu->hyperv_vpindex) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
if (!has_msr_hv_vpindex) {
fprintf(stderr, "Hyper-V VP_INDEX MSR "
"(requested by 'hv-vpindex' cpu flag) "
@@ -770,7 +754,7 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EAX] |= HV_VP_INDEX_AVAILABLE;
}
- if (cpu->hyperv_runtime) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RUNTIME)) {
if (!has_msr_hv_runtime) {
fprintf(stderr, "Hyper-V VP_RUNTIME MSR "
"(requested by 'hv-runtime' cpu flag) "
@@ -779,10 +763,10 @@ static int hyperv_handle_properties(CPUState *cs)
}
env->features[FEAT_HYPERV_EAX] |= HV_VP_RUNTIME_AVAILABLE;
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
unsigned int cap = KVM_CAP_HYPERV_SYNIC;
if (!cpu->hyperv_synic_kvm_only) {
- if (!cpu->hyperv_vpindex) {
+ if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
fprintf(stderr, "Hyper-V SynIC "
"(requested by 'hv-synic' cpu flag) "
"requires Hyper-V VP_INDEX ('hv-vpindex')\n");
@@ -799,20 +783,20 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_SYNIC_AVAILABLE;
}
- if (cpu->hyperv_stimer) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_STIMER)) {
if (!has_msr_hv_stimer) {
fprintf(stderr, "Hyper-V timers aren't supported by kernel\n");
return -ENOSYS;
}
env->features[FEAT_HYPERV_EAX] |= HV_SYNTIMERS_AVAILABLE;
}
- if (cpu->hyperv_relaxed_timing) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_RELAXED)) {
env->features[FEAT_HV_RECOMM_EAX] |= HV_RELAXED_TIMING_RECOMMENDED;
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
env->features[FEAT_HV_RECOMM_EAX] |= HV_APIC_ACCESS_RECOMMENDED;
}
- if (cpu->hyperv_tlbflush) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TLBFLUSH)) {
if (kvm_check_extension(cs->kvm_state,
KVM_CAP_HYPERV_TLBFLUSH) <= 0) {
fprintf(stderr, "Hyper-V TLB flush support "
@@ -823,7 +807,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HV_RECOMM_EAX] |= HV_REMOTE_TLB_FLUSH_RECOMMENDED;
env->features[FEAT_HV_RECOMM_EAX] |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
}
- if (cpu->hyperv_ipi) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_IPI)) {
if (kvm_check_extension(cs->kvm_state,
KVM_CAP_HYPERV_SEND_IPI) <= 0) {
fprintf(stderr, "Hyper-V IPI send support "
@@ -834,7 +818,7 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HV_RECOMM_EAX] |= HV_CLUSTER_IPI_RECOMMENDED;
env->features[FEAT_HV_RECOMM_EAX] |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
}
- if (cpu->hyperv_evmcs) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
uint16_t evmcs_version;
if (kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
@@ -856,7 +840,7 @@ static int hyperv_init_vcpu(X86CPU *cpu)
CPUState *cs = CPU(cpu);
int ret;
- if (cpu->hyperv_vpindex && !hv_vpindex_settable) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) && !hv_vpindex_settable) {
/*
* the kernel doesn't support setting vp_index; assert that its value
* is in sync
@@ -881,7 +865,7 @@ static int hyperv_init_vcpu(X86CPU *cpu)
}
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
uint32_t synic_cap = cpu->hyperv_synic_kvm_only ?
KVM_CAP_HYPERV_SYNIC : KVM_CAP_HYPERV_SYNIC2;
ret = kvm_vcpu_enable_cap(cs, synic_cap, 0);
@@ -972,7 +956,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
memset(signature, 0, 12);
memcpy(signature, cpu->hyperv_vendor_id, len);
}
- c->eax = cpu->hyperv_evmcs ?
+ c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
c->ebx = signature[0];
c->ecx = signature[1];
@@ -1016,7 +1000,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
kvm_base = KVM_CPUID_SIGNATURE_NEXT;
has_msr_hv_hypercall = true;
- if (cpu->hyperv_evmcs) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
__u32 function;
/* Create zeroed 0x40000006..0x40000009 leaves */
@@ -1360,7 +1344,7 @@ void kvm_arch_reset_vcpu(X86CPU *cpu)
env->mp_state = KVM_MP_STATE_RUNNABLE;
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
int i;
for (i = 0; i < ARRAY_SIZE(env->msr_hv_synic_sint); i++) {
env->msr_hv_synic_sint[i] = HV_SINT_MASKED;
@@ -2100,11 +2084,11 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
kvm_msr_entry_add(cpu, HV_X64_MSR_HYPERCALL,
env->msr_hv_hypercall);
}
- if (cpu->hyperv_time) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC,
env->msr_hv_tsc);
}
- if (cpu->hyperv_reenlightenment) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL,
env->msr_hv_reenlightenment_control);
kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL,
@@ -2113,7 +2097,7 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
env->msr_hv_tsc_emulation_status);
}
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_APIC_ASSIST_PAGE,
env->msr_hv_vapic);
}
@@ -2129,11 +2113,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
if (has_msr_hv_runtime) {
kvm_msr_entry_add(cpu, HV_X64_MSR_VP_RUNTIME, env->msr_hv_runtime);
}
- if (cpu->hyperv_vpindex && hv_vpindex_settable) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)
+ && hv_vpindex_settable) {
kvm_msr_entry_add(cpu, HV_X64_MSR_VP_INDEX,
hyperv_vp_index(CPU(cpu)));
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
int j;
kvm_msr_entry_add(cpu, HV_X64_MSR_SVERSION, HV_SYNIC_VERSION);
@@ -2473,13 +2458,13 @@ static int kvm_get_msrs(X86CPU *cpu)
kvm_msr_entry_add(cpu, HV_X64_MSR_HYPERCALL, 0);
kvm_msr_entry_add(cpu, HV_X64_MSR_GUEST_OS_ID, 0);
}
- if (cpu->hyperv_vapic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VAPIC)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_APIC_ASSIST_PAGE, 0);
}
- if (cpu->hyperv_time) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_TIME)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC, 0);
}
- if (cpu->hyperv_reenlightenment) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_REENLIGHTENMENT)) {
kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL, 0);
kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL, 0);
kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_STATUS, 0);
@@ -2494,7 +2479,7 @@ static int kvm_get_msrs(X86CPU *cpu)
if (has_msr_hv_runtime) {
kvm_msr_entry_add(cpu, HV_X64_MSR_VP_RUNTIME, 0);
}
- if (cpu->hyperv_synic) {
+ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
uint32_t msr;
kvm_msr_entry_add(cpu, HV_X64_MSR_SCONTROL, 0);
diff --git a/target/i386/machine.c b/target/i386/machine.c
index 4aff1a7..a39ce7f 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -634,7 +634,7 @@ static bool hyperv_runtime_enable_needed(void *opaque)
X86CPU *cpu = opaque;
CPUX86State *env = &cpu->env;
- if (!cpu->hyperv_runtime) {
+ if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_RUNTIME)) {
return false;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
end of thread, other threads:[~2019-07-10 17:25 UTC | newest]
Thread overview: 54+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-21 11:29 [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 01/25] kvm-all: Add/update fprintf's for kvm_*_ioeventfd_del Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 02/25] hax: Honor CPUState::halted Paolo Bonzini
2019-06-21 11:29 ` [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits Paolo Bonzini
2019-06-24 11:04 ` Alex Bennée
2019-06-24 12:26 ` Vitaly Kuznetsov
2019-06-24 12:43 ` Alex Bennée
2019-06-21 11:29 ` [Qemu-devel] [PULL 04/25] i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 05/25] i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties() Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 06/25] i386/kvm: document existing Hyper-V enlightenments Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 07/25] i386/kvm: implement 'hv-passthrough' mode Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 08/25] i386/kvm: hv-stimer requires hv-time and hv-synic Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 09/25] i386/kvm: hv-tlbflush/ipi require hv-vpindex Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 10/25] i386/kvm: hv-evmcs requires hv-vapic Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 11/25] i386/kvm: add support for Direct Mode for Hyper-V synthetic timers Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 12/25] target/i386: define a new MSR based feature word - FEAT_CORE_CAPABILITY Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 13/25] target/i386: kvm: Delete VMX migration blocker on vCPU init failure Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 14/25] KVM: Introduce kvm_arch_destroy_vcpu() Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 15/25] target/i386: kvm: Use symbolic constant for #DB/#BP exception constants Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 16/25] target/i386: kvm: Re-inject #DB to guest with updated DR6 Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 17/25] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization Paolo Bonzini
2019-07-08 18:21 ` Jan Kiszka
2019-07-08 22:26 ` Liran Alon
2019-06-21 11:30 ` [Qemu-devel] [PULL 18/25] linux-headers: sync with latest KVM headers from Linux 5.2 Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 19/25] vmstate: Add support for kernel integer types Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 20/25] target/i386: kvm: Add support for save and restore nested state Paolo Bonzini
2019-06-21 12:29 ` Liran Alon
2019-06-21 12:45 ` Paolo Bonzini
2019-06-21 12:48 ` Liran Alon
2019-06-21 14:55 ` Paolo Bonzini
2019-06-21 15:00 ` Liran Alon
2019-06-21 15:39 ` Paolo Bonzini
2019-06-21 15:44 ` Liran Alon
2019-06-21 16:01 ` Liran Alon
2019-06-21 11:30 ` [Qemu-devel] [PULL 21/25] target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 22/25] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities Paolo Bonzini
2019-06-21 12:39 ` Liran Alon
2019-06-21 15:02 ` Paolo Bonzini
2019-06-21 15:07 ` Liran Alon
2019-06-21 17:27 ` Paolo Bonzini
2019-06-21 17:54 ` Liran Alon
2019-07-08 18:31 ` Jan Kiszka
2019-07-08 18:51 ` Jan Kiszka
2019-07-10 14:40 ` Paolo Bonzini
2019-07-10 16:08 ` Jan Kiszka
2019-07-10 16:34 ` Paolo Bonzini
2019-07-10 17:15 ` Jan Kiszka
2019-07-10 17:18 ` Jan Kiszka
2019-06-21 11:30 ` [Qemu-devel] [PULL 23/25] sd: Fix out-of-bounds assertions Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 24/25] util/main-loop: Fix incorrect assertion Paolo Bonzini
2019-06-21 11:30 ` [Qemu-devel] [PULL 25/25] hw: Nuke hw_compat_4_0_1 and pc_compat_4_0_1 Paolo Bonzini
2019-06-21 13:53 ` [Qemu-devel] [PULL v2 00/25] Misc (mostly x86) patches for 2019-06-21 no-reply
2019-06-21 14:40 ` Peter Maydell
-- strict thread matches above, loose matches on Subject: below --
2019-06-21 1:42 [Qemu-devel] [PULL " Paolo Bonzini
2019-06-21 1:42 ` [Qemu-devel] [PULL 03/25] i386/kvm: convert hyperv enlightenments properties from bools to bits Paolo Bonzini
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).