Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option

From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Igor Mammedov <imammedo@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	qemu-devel@nongnu.org,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>
Subject: Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Date: Thu, 21 Jan 2021 17:51:18 +0100	[thread overview]
Message-ID: <87o8hi44jd.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <20210121144947.306512e1@redhat.com>

Igor Mammedov <imammedo@redhat.com> writes:

> On Thu, 21 Jan 2021 09:45:33 +0100
> Vitaly Kuznetsov <vkuznets@redhat.com> wrote:
>
>> >
>> > So far I read snippet above as a problem:
>> > 1:
>> >   host supports evmcs:
>> >   and exposes HYPERV_FEAT_EVMCS in CPUID  
>> 
>> Host with EVMCS is Intel
>> 
>> > 2: we migrate to host without evmcs  
>> 
>> Host without EVMCS is AMD, there are no other options. It is a pure
>> software feature available for KVM-intel. And if your KVM is so old that
>> it doesn't know anything about EVMCS, a bunch of other options from
>> 'hv-default' will not start as well.
>> > 2.1 start target QEMU, it happily creates vCPUs without
>> > HYPERV_FEAT_EVMCS in CPUID  
>> 
>> No, it doesn't as on host1 we had at least VMX CPU feature enabled (or a
>> CPU model implying it) to make this all work.
>> 
>> > 2.2 if I'm not mistaken CPUID is not part of migration stream,
>> >     nothing could check and fail migration
>> > 2.3 guest runs fine till it tries to use non existing feature, ..  
>> 
>> I'm also very sceptical about possibilities for migration
>> Windows/Hyper-V VMs from Intel to AMD. Hyper-V doesn't even boot if you
>> don't have fresh-enough CPU so the common denominator for Intel/AMD
>> would definitely not work. 
>
> Like you said host doesn't have to be AMD, just old enough kernel will
> do the job. What exactly will prevent migration 'successfully' completing?
>

First, you can't start a VM with 'hv-default' with an old-enough kernel
because it won't have many other 'hv-' enlightenments
implemented. 'hv-default' will only work for a 'recent enough' kernel
(>= 5.0 when hv-stimer-direct was implemented). 

You can probably try doing '-cpu xxx,hv_default,hv-stimer-direct=off' to
trigger the problem but then KVM should also support nested state
migration to actually migrate a VM using VMX and EVMCS support for it
also emerged in 5.0. I believe that trying to call KVM_SET_NESTED_STATE
(which only appeared in 4.19 btw) on something in between will fail.

> The way it's currently written migration stream won't prevent it.
>
> One way that might solve issue is to add subsection that's enabled when
> kvm_hv_evmcs_available() == true, and check on target that the feature
> is available or fail migration.

Yes, we can but I don't think there's a real issue worth fighting
for. Nested migration was so broken in upstream KVM untill recently that
I don't see why 'old kernel' can be a problem at all. And, again, Intel
to AMD migration is likely off question.

>
> Maybe Eduardo or David can add more how to deal with it if needed.
>

-- 
Vitaly