Problem with virtual to physical memory translation when KVM is enabled.

Problem with virtual to physical memory translation when KVM is enabled.

[Wayne Li]

Dear KVM list members,

We developed a virtual machine using the QEMU code.  This virtual
machine emulates a certain custom-made computer that runs on a certain
military platform.  All I can tell you about this virtual machine is
that it emulates a computer that has an e5500 processor.  Currently I
am running this virtual machine on a T4240-RDB which has a PowerPC
e6500 processor.

Anyway, right now I’m trying to get this virtual machine working with
KVM enabled.  But the problem I’m having is the VM doesn’t do anything
after the KVM_RUN ioctl call is made (NIP doesn’t progress and no
registers change).  What seems to be the problem is the VM doesn’t run
the instruction that’s supposed to be retrieved from the virtual
address 0xFFFF_FFFC.   When KVM isn’t enabled and the VM is running
using TCG (tiny code generator), a branch instruction to 0xFFFF_F700
is retrieved from the virtual address 0xFFFF_FFFC and the VM kicks off
running from there.

So what could be causing this problem?  I’m guessing it has something
to do with the translation lookaside buffers (TLBs)?  But the
translation between virtual and physical memory clearly works when KVM
isn’t enabled.  So what could cause this to stop working when KVM is
enabled?  Or maybe I’m not understanding something right and missing
what the problem actually is?  Let me know your thoughts.

-Thanks, Wayne Li

Re: Problem with virtual to physical memory translation when KVM is enabled.

[Peter Maydell]

On Tue, 25 Feb 2020 at 16:10, Wayne Li <waynli329@gmail.com> wrote:
> So what could be causing this problem?  I’m guessing it has something
> to do with the translation lookaside buffers (TLBs)?  But the
> translation between virtual and physical memory clearly works when KVM
> isn’t enabled.  So what could cause this to stop working when KVM is
> enabled?

When you're not using KVM, virtual-to-physical lookups are
done using QEMU's emulation code that emulates the MMU.
When you are using KVM, virtual-to-physical lookups
are done entirely using the host CPU (except for corner
cases like when we come out of the kernel and the user
does things with the gdb debug stub). So all the page
tables and other guest setup of the MMU had better match
what the host CPU expects. (I don't know how big the
differences between e5500 and e6500 MMU are or whether
the PPC architecture/KVM supports emulating the one on
the other: some PPC expert will probably be able to tell you.)

thanks
-- PMM

Re: Problem with virtual to physical memory translation when KVM is enabled.

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1965 bytes --]

On Tue, Feb 25, 2020 at 04:16:43PM +0000, Peter Maydell wrote:
> On Tue, 25 Feb 2020 at 16:10, Wayne Li <waynli329@gmail.com> wrote:
> > So what could be causing this problem?  I’m guessing it has something
> > to do with the translation lookaside buffers (TLBs)?  But the
> > translation between virtual and physical memory clearly works when KVM
> > isn’t enabled.  So what could cause this to stop working when KVM is
> > enabled?
> 
> When you're not using KVM, virtual-to-physical lookups are
> done using QEMU's emulation code that emulates the MMU.
> When you are using KVM, virtual-to-physical lookups
> are done entirely using the host CPU (except for corner
> cases like when we come out of the kernel and the user
> does things with the gdb debug stub). So all the page
> tables and other guest setup of the MMU had better match
> what the host CPU expects. (I don't know how big the
> differences between e5500 and e6500 MMU are or whether
> the PPC architecture/KVM supports emulating the one on
> the other: some PPC expert will probably be able to tell you.)

Well, sort of.  Including things like KVM-PR, things get complicated.
But in any case, the resposibility for translation lies somewhere
between the cpu itself and the KVM code - qemu is not involved.

Depending on exactly what the host's MMU looks like and what it has in
the way of virtualization features, that might make it impossible to
run a guest expecting a substantially different cpu model from the
host's.

Unfortunately, I'm not really at all familiar with the Freescale
parts, and even less with the KVM implementation for them.  It doesn't
surprise me that there are substantial bugs there, but I wouldn't
realy now where to begin to fix them.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]