From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Igor Mammedov <imammedo@redhat.com>
Cc: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
Sean Christopherson <seanjc@google.com>,
Wanpeng Li <wanpengli@tencent.com>,
Jim Mattson <jmattson@google.com>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/4] KVM: x86: Partially allow KVM_SET_CPUID{,2} after KVM_RUN for CPU hotplug
Date: Tue, 18 Jan 2022 17:34:09 +0100 [thread overview]
Message-ID: <87ee55knpa.fsf@redhat.com> (raw)
In-Reply-To: <20220118153531.11e73048@redhat.com>
Igor Mammedov <imammedo@redhat.com> writes:
> On Mon, 17 Jan 2022 16:05:38 +0100
> Vitaly Kuznetsov <vkuznets@redhat.com> wrote:
>
>> Changes since v1:
>> - Drop the allowlist of items which were allowed to change and just allow
>> the exact same CPUID data [Sean, Paolo]. Adjust selftest accordingly.
>> - Drop PATCH1 as the exact same change got merged upstream.
>>
>> Recently, KVM made it illegal to change CPUID after KVM_RUN but
>> unfortunately this change is not fully compatible with existing VMMs.
>> In particular, QEMU reuses vCPU fds for CPU hotplug after unplug and it
>> calls KVM_SET_CPUID2. Relax the requirement by implementing an allowing
>> KVM_SET_CPUID{,2} with the exact same data.
>
>
> Can you check following scenario:
> * on host that has IA32_TSX_CTRL and TSX enabled (RTM/HLE cpuid bits present)
> * boot 2 vcpus VM with TSX enabled on VMM side but with tsx=off on kernel CLI
>
> that should cause kernel to set MSR_IA32_TSX_CTRL to 3H from initial 0H
> and clear RTM+HLE bits in CPUID, check that RTM/HLE cpuid it
> cleared
Forgive me my ignorance around (not only) TSX :-) I took a "Intel(R)
Xeon(R) CPU E3-1270 v5 @ 3.60GHz" host which seems to have rtm/hle and
booted a guest with 'cpu=host' and with (and without) 'tsx=off' on the
kernel command line. I decided to check what's is MSR_IA32_TSX_CTRL but
I see the following:
# rdmsr 0x122
rdmsr: CPU 0 cannot read MSR 0x00000122
I tried adding 'tsx_ctrl' to my QEMU command line but it complains with
qemu-system-x86_64: warning: host doesn't support requested feature: MSR(10AH).tsx-ctrl [bit 7]
so I think my host is not good enough :-(
Also, I've looked at tsx_clear_cpuid() but it actually writes to
MSR_TSX_FORCE_ABORT MSR (0x10F), not MSR_IA32_TSX_CTRL so I'm confused.
>
> * hotunplug a VCPU and then replug it again
> if IA32_TSX_CTRL is reset to initial state, that should re-enable
> RTM/HLE cpuid bits and KVM_SET_CPUID2 might fail due to difference
Could you please teach me this kung-fu, I mean hot to unplug a
cold-plugged CPU with QMP? Previoulsy, I only did un-plugging for what
I've hotplugged, something like:
(QEMU) device_add driver=host-x86_64-cpu socket-id=0 core-id=2 thread-id=0 id=cpu2
{"return": {}}
(QEMU) device_del id=cpu2
{"return": {}}
What's the ids of the cold-plugged CPUs?
>
> and as Sean pointed out there might be other non constant leafs,
> where exact match check could leave userspace broken.
Indeed, while testing your suggestion I've stumbled upon
CPUID.(EAX=0x12, ECX=1) (SGX) where we mangle ECX from
kvm_vcpu_after_set_cpuid():
best = kvm_find_cpuid_entry(vcpu, 0x12, 0x1);
if (best) {
best->ecx &= vcpu->arch.guest_supported_xcr0 & 0xffffffff;
best->edx &= vcpu->arch.guest_supported_xcr0 >> 32;
best->ecx |= XFEATURE_MASK_FPSSE;
}
In theory, we should just move this to __kvm_update_cpuid_runtime()...
I'll take a look tomorrow.
--
Vitaly
next prev parent reply other threads:[~2022-01-18 16:34 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-17 15:05 [PATCH v2 0/4] KVM: x86: Partially allow KVM_SET_CPUID{,2} after KVM_RUN for CPU hotplug Vitaly Kuznetsov
2022-01-17 15:05 ` [PATCH v2 1/4] KVM: x86: Do runtime CPUID update before updating vcpu->arch.cpuid_entries Vitaly Kuznetsov
2022-01-17 15:05 ` [PATCH v2 2/4] KVM: x86: Partially allow KVM_SET_CPUID{,2} after KVM_RUN Vitaly Kuznetsov
2022-01-17 17:30 ` Paolo Bonzini
2022-01-18 8:40 ` Vitaly Kuznetsov
2022-01-18 14:37 ` Vitaly Kuznetsov
2022-01-17 15:05 ` [PATCH v2 3/4] KVM: selftests: Rename 'get_cpuid_test' to 'cpuid_test' Vitaly Kuznetsov
2022-01-17 15:05 ` [PATCH v2 4/4] KVM: selftests: Test KVM_SET_CPUID2 after KVM_RUN Vitaly Kuznetsov
2022-01-18 14:35 ` [PATCH v2 0/4] KVM: x86: Partially allow KVM_SET_CPUID{,2} after KVM_RUN for CPU hotplug Igor Mammedov
2022-01-18 16:17 ` Paolo Bonzini
2022-01-18 16:53 ` Sean Christopherson
2022-01-18 17:12 ` Paolo Bonzini
2022-01-18 16:34 ` Vitaly Kuznetsov [this message]
2022-01-19 7:59 ` Igor Mammedov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ee55knpa.fsf@redhat.com \
--to=vkuznets@redhat.com \
--cc=imammedo@redhat.com \
--cc=jmattson@google.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).