Fwd: Why qemu with kvm enabled can boot kernel even if identity page map is not set correctly?

Fwd: Why qemu with kvm enabled can boot kernel even if identity page map is not set correctly?

[丁飞]

---------- Forwarded message ----------
From: 丁飞 <danix800@gmail.com>
Date: Wed, Nov 22, 2017 at 12:58 AM
Subject: Why qemu with kvm enabled can boot kernel even if identity
page map is not set correctly?
To: kvm@vger.kernel.org


Hi, KVM developers. Firstly, sorry if it's the wrong place to ask such
a question!

In the early stages of boot process, kernel need identity mapped page setup
when switching gdt
[https://github.com/torvalds/linux/blob/ed30b147e1f6e396e70a52dbb6c7d66befedd786/arch/x86/kernel/head_64.S#L133-L137]
as code here [https://github.com/torvalds/linux/blob/ed30b147e1f6e396e70a52dbb6c7d66befedd786/arch/x86/kernel/head64.c#L98-L138]
implies. That's why the first few entries
of early_dynamic_pgts are set to map the kernel text range [_text, _end].
But as we discussed about the role of the first few entries of
early_dynamic_pgts,
we delete them [https://github.com/torvalds/linux/blob/ed30b147e1f6e396e70a52dbb6c7d66befedd786/arch/x86/kernel/head64.c#L98-L138]
and recompile the kernel, then test it on qemu.

Without '-enable-kvm' option the kernel won't boot as we expected, but with kvm
option on, the kernel can boot and everything runs well, really to our surprise.

So I guess there are something under the hood done by kvm, which doesn't obey
the rules of how a real physical machine behaves.

I've setup a debug environment that the page table mis-configed kernel
runs inside
qemu, which nested inside vmware workstation with EPT enabled, and gdb
on the host to debug the kernel kvm of vmware kernel.

But without any luck I've spent a whole day try to catch what is
happening inside kvm,
I still can't figure out the real magic point that jump through the
broken page table.
It seems that the code just jumps randomly.

Can anyone confirm what we've observed? Is it designed to be like that?
Any details or explanation would be really appreciated!

--
Best Regards

丁飞, Ding Fei
E-mail: danix800@gmail.com


-- 
Best Regards

丁飞, Ding Fei
E-mail: danix800@gmail.com

Re: Fwd: Why qemu with kvm enabled can boot kernel even if identity page map is not set correctly?

[Paolo Bonzini]

On 22/11/2017 03:58, 丁飞 wrote:
> ---------- Forwarded message ----------
> From: 丁飞 <danix800@gmail.com>
> Date: Wed, Nov 22, 2017 at 12:58 AM
> Subject: Why qemu with kvm enabled can boot kernel even if identity
> page map is not set correctly?
> To: kvm@vger.kernel.org
> 
> 
> Hi, KVM developers. Firstly, sorry if it's the wrong place to ask such
> a question!
> 
> In the early stages of boot process, kernel need identity mapped page setup
> when switching gdt
> [https://github.com/torvalds/linux/blob/ed30b147e1f6e396e70a52dbb6c7d66befedd786/arch/x86/kernel/head_64.S#L133-L137]
> as code here [https://github.com/torvalds/linux/blob/ed30b147e1f6e396e70a52dbb6c7d66befedd786/arch/x86/kernel/head64.c#L98-L138]
> implies. That's why the first few entries
> of early_dynamic_pgts are set to map the kernel text range [_text, _end].
> But as we discussed about the role of the first few entries of
> early_dynamic_pgts,
> we delete them [https://github.com/torvalds/linux/blob/ed30b147e1f6e396e70a52dbb6c7d66befedd786/arch/x86/kernel/head64.c#L98-L138]
> and recompile the kernel, then test it on qemu.
> 
> Without '-enable-kvm' option the kernel won't boot as we expected, but with kvm
> option on, the kernel can boot and everything runs well, really to our surprise.
> 
> So I guess there are something under the hood done by kvm, which doesn't obey
> the rules of how a real physical machine behaves.
> 
> I've setup a debug environment that the page table mis-configed kernel
> runs inside
> qemu, which nested inside vmware workstation with EPT enabled, and gdb
> on the host to debug the kernel kvm of vmware kernel.
> 
> But without any luck I've spent a whole day try to catch what is
> happening inside kvm,
> I still can't figure out the real magic point that jump through the
> broken page table.
> It seems that the code just jumps randomly.
> 
> Can anyone confirm what we've observed? Is it designed to be like that?
> Any details or explanation would be really appreciated!

I'm sorry, I don't know.  There are many differences in TLB behavior
between emulation and real hardware, those could be the culprit.

Paolo