* Nested virtualization not working with hyperv guest/windows 11
@ 2023-03-24 10:55 Michał Zegan
2023-03-24 14:40 ` Sean Christopherson
0 siblings, 1 reply; 5+ messages in thread
From: Michał Zegan @ 2023-03-24 10:55 UTC (permalink / raw)
To: kvm
Hi,
I've sent this some time ago, but was not subscribed here, so unsure if
I didn't get a reply or maybe missed it, so repeating:
I have a linux host with cpu intel core i7 12700h, kernel currently 6.2,
fedora37.
I have a kvm/qemu/libvirt virtual machine, cpu model set to host,
machine type q35, uefi with secureboot enabled, smm on.
The kvm_intel module has nested=y set in parameters so nested
virtualization is enabled on host.
The virtual machine has windows11 pro guest installed.
When I install hyperv/virtualization platform/other similar functions,
after reboot, the windows does not boot. Namely it reboots three times
and then goes to recovery.
The only way to fix the problem is disabling nested virtualization or
vmx cpu feature, or uninstalling hyperv. However I wanted nested
virtualization to be working.
Any clue what might be going on and how to fix it or even see the problem?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Nested virtualization not working with hyperv guest/windows 11
2023-03-24 10:55 Nested virtualization not working with hyperv guest/windows 11 Michał Zegan
@ 2023-03-24 14:40 ` Sean Christopherson
2023-03-24 14:59 ` Michał Zegan
[not found] ` <a054ea77-e53c-8207-1e25-1081e4ecbb50@outlook.com>
0 siblings, 2 replies; 5+ messages in thread
From: Sean Christopherson @ 2023-03-24 14:40 UTC (permalink / raw)
To: Michał Zegan; +Cc: kvm
On Fri, Mar 24, 2023, Michał Zegan wrote:
> Hi,
>
> I've sent this some time ago, but was not subscribed here, so unsure if I
> didn't get a reply or maybe missed it, so repeating:
>
> I have a linux host with cpu intel core i7 12700h, kernel currently 6.2,
> fedora37.
>
> I have a kvm/qemu/libvirt virtual machine, cpu model set to host, machine
> type q35, uefi with secureboot enabled, smm on.
>
> The kvm_intel module has nested=y set in parameters so nested virtualization
> is enabled on host.
>
> The virtual machine has windows11 pro guest installed.
>
> When I install hyperv/virtualization platform/other similar functions, after
> reboot, the windows does not boot. Namely it reboots three times and then
> goes to recovery.
This is going to be nearly impossible to debug without more information. Assuming
you can't extract more information from the guest, can you try enabling KVM
tracepoints? E.g. to see if KVM is injecting an exception or a nested VM-Entry
failure that leads to the reboot.
I.e. enable tracing
echo 1 > /sys/kernel/debug/tracing/tracing_on
and then to get the full blast from the trace firehose:
echo 1 > /sys/kernel/debug/tracing/events/kvm/enable
or to get slightly less noisy log:
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_entry/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_exit/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_inj_exception/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_intercepts/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_intr_vmexit/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter_failed/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit_inject/enable
echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter/enable
To capture something useful, you may need to (significantly) increase the size of
the buffer,
echo 131072 > /sys/kernel/debug/tracing/buffer_size_kb
The log itself can be found at
/sys/kernel/debug/tracing/trace
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Nested virtualization not working with hyperv guest/windows 11
2023-03-24 14:40 ` Sean Christopherson
@ 2023-03-24 14:59 ` Michał Zegan
[not found] ` <a054ea77-e53c-8207-1e25-1081e4ecbb50@outlook.com>
1 sibling, 0 replies; 5+ messages in thread
From: Michał Zegan @ 2023-03-24 14:59 UTC (permalink / raw)
To: Sean Christopherson; +Cc: kvm
Hello,
Thanks for the reply.
doesn't qemu actually panic on entry failure? note i don't have crashes
nor anything in logs, I've tried to enable ignore_msrs in kvm parameters
to see if maybe this will report something, but no luck.
Tracing data are here: https://transfer.sh/1lfnMc/kvmtrace.gz
Also unsure if I didn't lose some data in the process, although if this
was actually three reboots, not just instantly going to recovery, it
shouldn't matter.
I definitely cannot get anything useful from it.
W dniu 24.03.2023 o 15:40, Sean Christopherson pisze:
> On Fri, Mar 24, 2023, Michał Zegan wrote:
>> Hi,
>>
>> I've sent this some time ago, but was not subscribed here, so unsure if I
>> didn't get a reply or maybe missed it, so repeating:
>>
>> I have a linux host with cpu intel core i7 12700h, kernel currently 6.2,
>> fedora37.
>>
>> I have a kvm/qemu/libvirt virtual machine, cpu model set to host, machine
>> type q35, uefi with secureboot enabled, smm on.
>>
>> The kvm_intel module has nested=y set in parameters so nested virtualization
>> is enabled on host.
>>
>> The virtual machine has windows11 pro guest installed.
>>
>> When I install hyperv/virtualization platform/other similar functions, after
>> reboot, the windows does not boot. Namely it reboots three times and then
>> goes to recovery.
> This is going to be nearly impossible to debug without more information. Assuming
> you can't extract more information from the guest, can you try enabling KVM
> tracepoints? E.g. to see if KVM is injecting an exception or a nested VM-Entry
> failure that leads to the reboot.
>
> I.e. enable tracing
>
> echo 1 > /sys/kernel/debug/tracing/tracing_on
>
> and then to get the full blast from the trace firehose:
>
> echo 1 > /sys/kernel/debug/tracing/events/kvm/enable
>
> or to get slightly less noisy log:
>
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_entry/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_exit/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_inj_exception/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_intercepts/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_intr_vmexit/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter_failed/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit_inject/enable
> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter/enable
>
> To capture something useful, you may need to (significantly) increase the size of
> the buffer,
>
> echo 131072 > /sys/kernel/debug/tracing/buffer_size_kb
>
> The log itself can be found at
>
> /sys/kernel/debug/tracing/trace
^ permalink raw reply [flat|nested] 5+ messages in thread
* ping: Re: Nested virtualization not working with hyperv guest/windows 11
[not found] ` <a054ea77-e53c-8207-1e25-1081e4ecbb50@outlook.com>
@ 2023-03-30 16:46 ` Michał Zegan
[not found] ` <9e2ac05c-74f7-ecf6-b81f-873e24425795@outlook.com>
1 sibling, 0 replies; 5+ messages in thread
From: Michał Zegan @ 2023-03-30 16:46 UTC (permalink / raw)
To: kvm
W dniu 24.03.2023 o 15:59, Michał Zegan pisze:
> Hello,
>
> Thanks for the reply.
>
> doesn't qemu actually panic on entry failure? note i don't have
> crashes nor anything in logs, I've tried to enable ignore_msrs in kvm
> parameters to see if maybe this will report something, but no luck.
>
> Tracing data are here: https://transfer.sh/1lfnMc/kvmtrace.gz
>
> Also unsure if I didn't lose some data in the process, although if
> this was actually three reboots, not just instantly going to recovery,
> it shouldn't matter.
>
> I definitely cannot get anything useful from it.
>
> W dniu 24.03.2023 o 15:40, Sean Christopherson pisze:
>> On Fri, Mar 24, 2023, Michał Zegan wrote:
>>> Hi,
>>>
>>> I've sent this some time ago, but was not subscribed here, so unsure
>>> if I
>>> didn't get a reply or maybe missed it, so repeating:
>>>
>>> I have a linux host with cpu intel core i7 12700h, kernel currently
>>> 6.2,
>>> fedora37.
>>>
>>> I have a kvm/qemu/libvirt virtual machine, cpu model set to host,
>>> machine
>>> type q35, uefi with secureboot enabled, smm on.
>>>
>>> The kvm_intel module has nested=y set in parameters so nested
>>> virtualization
>>> is enabled on host.
>>>
>>> The virtual machine has windows11 pro guest installed.
>>>
>>> When I install hyperv/virtualization platform/other similar
>>> functions, after
>>> reboot, the windows does not boot. Namely it reboots three times and
>>> then
>>> goes to recovery.
>> This is going to be nearly impossible to debug without more
>> information. Assuming
>> you can't extract more information from the guest, can you try
>> enabling KVM
>> tracepoints? E.g. to see if KVM is injecting an exception or a
>> nested VM-Entry
>> failure that leads to the reboot.
>>
>> I.e. enable tracing
>>
>> echo 1 > /sys/kernel/debug/tracing/tracing_on
>>
>> and then to get the full blast from the trace firehose:
>>
>> echo 1 > /sys/kernel/debug/tracing/events/kvm/enable
>>
>> or to get slightly less noisy log:
>>
>> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_entry/enable
>> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_exit/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_inj_exception/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_intercepts/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_intr_vmexit/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter_failed/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit_inject/enable
>> echo 1 >
>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter/enable
>>
>> To capture something useful, you may need to (significantly) increase
>> the size of
>> the buffer,
>>
>> echo 131072 > /sys/kernel/debug/tracing/buffer_size_kb
>>
>> The log itself can be found at
>>
>> /sys/kernel/debug/tracing/trace
^ permalink raw reply [flat|nested] 5+ messages in thread
* ping: Re: Nested virtualization not working with hyperv guest/windows 11
[not found] ` <9e2ac05c-74f7-ecf6-b81f-873e24425795@outlook.com>
@ 2023-04-04 15:38 ` Michał Zegan
0 siblings, 0 replies; 5+ messages in thread
From: Michał Zegan @ 2023-04-04 15:38 UTC (permalink / raw)
To: kvm
W dniu 30.03.2023 o 18:46, Michał Zegan pisze:
>
> W dniu 24.03.2023 o 15:59, Michał Zegan pisze:
>> Hello,
>>
>> Thanks for the reply.
>>
>> doesn't qemu actually panic on entry failure? note i don't have
>> crashes nor anything in logs, I've tried to enable ignore_msrs in kvm
>> parameters to see if maybe this will report something, but no luck.
>>
>> Tracing data are here: https://transfer.sh/1lfnMc/kvmtrace.gz
>>
>> Also unsure if I didn't lose some data in the process, although if
>> this was actually three reboots, not just instantly going to
>> recovery, it shouldn't matter.
>>
>> I definitely cannot get anything useful from it.
>>
>> W dniu 24.03.2023 o 15:40, Sean Christopherson pisze:
>>> On Fri, Mar 24, 2023, Michał Zegan wrote:
>>>> Hi,
>>>>
>>>> I've sent this some time ago, but was not subscribed here, so
>>>> unsure if I
>>>> didn't get a reply or maybe missed it, so repeating:
>>>>
>>>> I have a linux host with cpu intel core i7 12700h, kernel currently
>>>> 6.2,
>>>> fedora37.
>>>>
>>>> I have a kvm/qemu/libvirt virtual machine, cpu model set to host,
>>>> machine
>>>> type q35, uefi with secureboot enabled, smm on.
>>>>
>>>> The kvm_intel module has nested=y set in parameters so nested
>>>> virtualization
>>>> is enabled on host.
>>>>
>>>> The virtual machine has windows11 pro guest installed.
>>>>
>>>> When I install hyperv/virtualization platform/other similar
>>>> functions, after
>>>> reboot, the windows does not boot. Namely it reboots three times
>>>> and then
>>>> goes to recovery.
>>> This is going to be nearly impossible to debug without more
>>> information. Assuming
>>> you can't extract more information from the guest, can you try
>>> enabling KVM
>>> tracepoints? E.g. to see if KVM is injecting an exception or a
>>> nested VM-Entry
>>> failure that leads to the reboot.
>>>
>>> I.e. enable tracing
>>>
>>> echo 1 > /sys/kernel/debug/tracing/tracing_on
>>>
>>> and then to get the full blast from the trace firehose:
>>>
>>> echo 1 > /sys/kernel/debug/tracing/events/kvm/enable
>>>
>>> or to get slightly less noisy log:
>>>
>>> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_entry/enable
>>> echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_exit/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_inj_exception/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_intercepts/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_intr_vmexit/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter_failed/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit_inject/enable
>>> echo 1 >
>>> /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter/enable
>>>
>>> To capture something useful, you may need to (significantly)
>>> increase the size of
>>> the buffer,
>>>
>>> echo 131072 > /sys/kernel/debug/tracing/buffer_size_kb
>>>
>>> The log itself can be found at
>>>
>>> /sys/kernel/debug/tracing/trace
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-04-04 15:39 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-24 10:55 Nested virtualization not working with hyperv guest/windows 11 Michał Zegan
2023-03-24 14:40 ` Sean Christopherson
2023-03-24 14:59 ` Michał Zegan
[not found] ` <a054ea77-e53c-8207-1e25-1081e4ecbb50@outlook.com>
2023-03-30 16:46 ` ping: " Michał Zegan
[not found] ` <9e2ac05c-74f7-ecf6-b81f-873e24425795@outlook.com>
2023-04-04 15:38 ` Michał Zegan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.