Re: [Qemu-devel] virsh dump (qemu guest memory dump?): KASLR enabled linux guest support

[Dave Anderson <anderson@redhat.com>]

----- Original Message -----
> On Wed, Nov 09, 2016 at 11:01:46AM +0800, Dave Young wrote:
> > Hi,
> > 
> > Latest linux kernel enabled kaslr to randomiz phys/virt memory
> > addresses, we had some effort to support kexec/kdump so that crash
> > utility can still works in case crashed kernel has kaslr enabled.
> > 
> > But according to Dave Anderson virsh dump does not work, quoted messages
> > from Dave below:
> > 
> > """
> > with virsh dump, there's no way of even knowing that KASLR
> > has randomized the kernel __START_KERNEL_map region, because there is no
> > virtual address information -- e.g., like "SYMBOL(_stext)" in the kdump
> > vmcoreinfo data to compare against the vmlinux file symbol value.
> > Unless virsh dump can export some basic virtual memory data, which
> > they say it can't, I don't see how KASLR can ever be supported.
> > """
> > 
> > I assume virsh dump is using qemu guest memory dump facility so it
> > should be first addressed in qemu. Thus post this query to qemu devel
> > list. If this is not correct please let me know.
> > 
> > Could you qemu dump people make it work? Or we can not support virt dump
> > as long as KASLR being enabled. Latest Fedora kernel has enabled it in
> > x86_64.
> >
> 
> When the -kernel command line option is used, then it may be possible
> to extract some information that could be used to supplement the memory
> dump that dump-guest-memory provides. However, that would be a specific
> use. In general, QEMU knows nothing about the guest kernel. It doesn't
> know where it is in the disk image, and it doesn't even know if it's
> Linux.
> 
> Is there anything a guest userspace application could probe from e.g.
> /proc that would work? If so, then the guest agent could gain a new
> feature providing that.
> 
> Thanks,
> drew

I'm not sure whether this "guest userspace agent" is still in play here,
but if there were such a thing, it could theoretically do the same
thing that crash currently does when running on a live system.

Two basic necessities are are needed, whether running live or against
a dumpfile:

(1) the CONFIG_RANDOMIZE_BASE relocation value that modifies the
    kernel virtual address range compiled into the vmlinux file, which
    starts at the hardwired __START_KERNEL_map address.

(2) the contents of the kernel's "phys_base" symbol.

Both of those are available or calculatable from the contents of
a kdump header.  However, on a live system, it's done like this:

- /proc/kallsyms is queried for the symbol value of "_text", which would
  be relocated if KASLR is in play.  That value is compared against the
  "_text" symbol value compiled into the vmlinux file to determine the
  relocation value generated by CONFIG_RANDOMIZE_BASE.

Given that relocation value, and before any kernel memory is accessed,
crash goes in a backdoor into its embedded gdb module, and modifies
the data structures of all kernel symbols, applying the relocation
value.

Once that's done, in order to read kernel symbols from the 
statically-mapped kernel region based at __START_KERNEL_map, it 
translates a (possibly relocated) kernel virtual address into a
physical address like this:

  physical-address = virtual-address - __START_KERNEL_map + phys_base

But it's a chicken-and-egg deal, because the contents of the "phys_base"
symbol are needed to calculate the physical address, but it can't
read the "phys_base" symbol contents without first knowing its contents.

So on a live system, the "phys_base" is calculated by reading
the "Kernel Code:" value from /proc/iomem, and then doing this:

  phys_base = [Kernel Code: value] - ["_text" symbol value] - __START_KERNEL_map

So theoretically, the guest agent could read /proc/iomem and /proc/kallsyms
for the information required.  (I think...)

Dave