[Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Goswin von Brederlow]

Hi,

starting your own toy kernel is a fun thing to do and there are many
tutorials out there on how to do it. Unfortunately when one wants to
write a kernel in 64bit it becomes much harder because one can't    
compile 64bit code as elf32 image and converting a elf64 image to
elf32 format is a major hassle and looses debug information and symbols.

So I looked into why kvm does not load an multiboot image in elf64 format
and what would be neccessary to support this. And the answere is that
all it needs is to remove the error message. Kvm alredy supports loading
a multiboot image in elf64 format perfectly. There is no need to force the
user to convert their toy kernel from elf64 to elf32 format at all.

MfG
	Goswin
--
Description: Allow 64bit elf binaries in multiboot format
 This patch allows 64bit elf files with multiboot header to be loaded.
 The entry point will still be called in 32bit mode and the kernel
 must switch to 64bit mode on its own. The image and all modules must
 also be located in the lower 2GB of ram. All the restrictions of a
 32bit image still apply.
Author: Goswin von Brederlow <goswin-v-b@web.de>
Last-Updated: 2011-04-08
---

--- qemu-kvm-0.14.0+dfsg.orig/hw/multiboot.c
+++ qemu-kvm-0.14.0+dfsg/hw/multiboot.c
@@ -173,8 +173,7 @@ int load_multiboot(void *fw_cfg,
         fclose(f);
 
         if (((struct elf64_hdr*)header)->e_machine == EM_X86_64) {
-            fprintf(stderr, "Cannot load x86-64 image, give a 32bit one.\n");
-            exit(1);
+	    mb_debug("qemu: 64bit elf, I hope you know what you are doing\n");
         }
 
         kernel_size = load_elf(kernel_filename, NULL, NULL, &elf_entry,

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Stefan Hajnoczi]

On Sat, Feb 4, 2012 at 5:38 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Description: Allow 64bit elf binaries in multiboot format
>  This patch allows 64bit elf files with multiboot header to be loaded.
>  The entry point will still be called in 32bit mode and the kernel
>  must switch to 64bit mode on its own. The image and all modules must
>  also be located in the lower 2GB of ram. All the restrictions of a
>  32bit image still apply.
> Author: Goswin von Brederlow <goswin-v-b@web.de>
> Last-Updated: 2011-04-08
> ---

The multiboot specification is 32-bit only.  This patch enables a
non-standard 64-bit version of multiboot.  Have you checked whether
GRUB or other multiboot loaders have equivalent functionality?  Have
you contacted the multiboot specification authors?

> --- qemu-kvm-0.14.0+dfsg.orig/hw/multiboot.c
> +++ qemu-kvm-0.14.0+dfsg/hw/multiboot.c
> @@ -173,8 +173,7 @@ int load_multiboot(void *fw_cfg,
>         fclose(f);
>
>         if (((struct elf64_hdr*)header)->e_machine == EM_X86_64) {
> -            fprintf(stderr, "Cannot load x86-64 image, give a 32bit one.\n");
> -            exit(1);
> +           mb_debug("qemu: 64bit elf, I hope you know what you are doing\n");

This is silent by default, but given the nature of 64-bit multiboot
support I think this warning should be on by default.  Anyone using
this really needs to know what they are doing and QEMU should not
silently do weird things.

Stefan

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Goswin von Brederlow]

Stefan Hajnoczi <stefanha@gmail.com> writes:

> On Sat, Feb 4, 2012 at 5:38 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> Description: Allow 64bit elf binaries in multiboot format
>>  This patch allows 64bit elf files with multiboot header to be loaded.
>>  The entry point will still be called in 32bit mode and the kernel
>>  must switch to 64bit mode on its own. The image and all modules must
>>  also be located in the lower 2GB of ram. All the restrictions of a
>>  32bit image still apply.
>> Author: Goswin von Brederlow <goswin-v-b@web.de>
>> Last-Updated: 2011-04-08
>> ---
>
> The multiboot specification is 32-bit only.  This patch enables a
> non-standard 64-bit version of multiboot.  Have you checked whether
> GRUB or other multiboot loaders have equivalent functionality?  Have
> you contacted the multiboot specification authors?

Not really. The multiboot specification allow for different executable
formats as long as the multiboot header is correct. For example you can
have an a.out multiboot image. The entry vector specified in the
multiboot header is still called in 32bit mode, as per specs. All that
changes is that the kvm allows another executable format for loading the
image.

Actually per mutliboot specs the elf64 image should already be loaded as
plain image (which means not neccessarily reloacted to the right
address) just like a.out would but kvm doesn't support that.

I don't think other loaders support elf64 (yet) unless they support
plain images. Frankly I never tried booting a toy kernel on real
hardware so there never was the need.

>> --- qemu-kvm-0.14.0+dfsg.orig/hw/multiboot.c
>> +++ qemu-kvm-0.14.0+dfsg/hw/multiboot.c
>> @@ -173,8 +173,7 @@ int load_multiboot(void *fw_cfg,
>>         fclose(f);
>>
>>         if (((struct elf64_hdr*)header)->e_machine == EM_X86_64) {
>> -            fprintf(stderr, "Cannot load x86-64 image, give a 32bit one.\n");
>> -            exit(1);
>> +           mb_debug("qemu: 64bit elf, I hope you know what you are doing\n");
>
> This is silent by default, but given the nature of 64-bit multiboot
> support I think this warning should be on by default.  Anyone using
> this really needs to know what they are doing and QEMU should not
> silently do weird things.
>
> Stefan

Fine by me to make this more verbose. I only care about removing the
exit(1).

MfG
        Goswin

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Kevin Wolf]

Am 08.02.2012 10:53, schrieb Goswin von Brederlow:
> Stefan Hajnoczi <stefanha@gmail.com> writes:
> 
>> On Sat, Feb 4, 2012 at 5:38 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>>> Description: Allow 64bit elf binaries in multiboot format
>>>  This patch allows 64bit elf files with multiboot header to be loaded.
>>>  The entry point will still be called in 32bit mode and the kernel
>>>  must switch to 64bit mode on its own. The image and all modules must
>>>  also be located in the lower 2GB of ram. All the restrictions of a
>>>  32bit image still apply.
>>> Author: Goswin von Brederlow <goswin-v-b@web.de>
>>> Last-Updated: 2011-04-08
>>> ---
>>
>> The multiboot specification is 32-bit only.  This patch enables a
>> non-standard 64-bit version of multiboot.  Have you checked whether
>> GRUB or other multiboot loaders have equivalent functionality?  Have
>> you contacted the multiboot specification authors?
> 
> Not really. The multiboot specification allow for different executable
> formats as long as the multiboot header is correct. For example you can
> have an a.out multiboot image. The entry vector specified in the
> multiboot header is still called in 32bit mode, as per specs. All that
> changes is that the kvm allows another executable format for loading the
> image.
> 
> Actually per mutliboot specs the elf64 image should already be loaded as
> plain image (which means not neccessarily reloacted to the right
> address) just like a.out would but kvm doesn't support that.
> 
> I don't think other loaders support elf64 (yet) unless they support
> plain images. Frankly I never tried booting a toy kernel on real
> hardware so there never was the need.

If you want to use it this way, you need to use the a.out kludge, i.e.
the multiboot header must have set bit 16 of the flag and the address
fields must be present in the multiboot header. In this case, the code
path that you're patching isn't even run.

Kevin

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Stefan Hajnoczi]

On Wed, Feb 8, 2012 at 9:53 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Stefan Hajnoczi <stefanha@gmail.com> writes:
>
>> On Sat, Feb 4, 2012 at 5:38 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>>> Description: Allow 64bit elf binaries in multiboot format
>>>  This patch allows 64bit elf files with multiboot header to be loaded.
>>>  The entry point will still be called in 32bit mode and the kernel
>>>  must switch to 64bit mode on its own. The image and all modules must
>>>  also be located in the lower 2GB of ram. All the restrictions of a
>>>  32bit image still apply.
>>> Author: Goswin von Brederlow <goswin-v-b@web.de>
>>> Last-Updated: 2011-04-08
>>> ---
>>
>> The multiboot specification is 32-bit only.  This patch enables a
>> non-standard 64-bit version of multiboot.  Have you checked whether
>> GRUB or other multiboot loaders have equivalent functionality?  Have
>> you contacted the multiboot specification authors?
>
> Not really. The multiboot specification allow for different executable
> formats as long as the multiboot header is correct. For example you can
> have an a.out multiboot image. The entry vector specified in the
> multiboot header is still called in 32bit mode, as per specs. All that
> changes is that the kvm allows another executable format for loading the
> image.
>
> Actually per mutliboot specs the elf64 image should already be loaded as
> plain image (which means not neccessarily reloacted to the right
> address) just like a.out would but kvm doesn't support that.
>
> I don't think other loaders support elf64 (yet) unless they support
> plain images. Frankly I never tried booting a toy kernel on real
> hardware so there never was the need.

Yes, this is why I asked about support in other software.  If each
bootloader implements a different custom method then it will be a pain
to run your binary on real hardware in the future.  It's worth at
least checking their source first - maybe there is already a similar
code path that we can be compatible with, hence making life easier for
developers who want to play with 64-bit payloads in multiboot
executables.

Stefan

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Paul Brook]

> > starting your own toy kernel is a fun thing to do and there are many
> > tutorials out there on how to do it. Unfortunately when one wants to
> > write a kernel in 64bit it becomes much harder because one can't
> > compile 64bit code as elf32 image and converting a elf64 image to
> > elf32 format is a major hassle and looses debug information and symbols.

So just have two versions of your image:
- The elf64 image that has debug info, symbols, etc.  Point gdb at this.
- An elf32 image that you give to the bootloader (in this case kvm)

Generating the latter from the former is a trivial objcopy invocation.
The bootloader variant only needs enough information to get the loadable 
sections into memory.  We don't care about non-resident clutter like debug 
info or symbols.  Anything that cares about those will be using the full elf64 
image.

This is all standard practice.  I don't think I've never actually used a 
system where the image loaded by the target is the same file as the one that 
comes out of the linker and is used by the debugger.

> Yes, this is why I asked about support in other software.  If each
> bootloader implements a different custom method then it will be a pain
> to run your binary on real hardware in the future.  It's worth at
> least checking their source first - maybe there is already a similar
> code path that we can be compatible with, hence making life easier for
> developers who want to play with 64-bit payloads in multiboot
> executables.

The whole idea of entering a 64-bit image in 32-bit mode seems distinctly 
sketchy.  Surely it'd make more sense to define a 64-bit multiboot variant and 
do the job properly.

Paul

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Goswin von Brederlow]

Paul Brook <paul@codesourcery.com> writes:

>> > starting your own toy kernel is a fun thing to do and there are many
>> > tutorials out there on how to do it. Unfortunately when one wants to
>> > write a kernel in 64bit it becomes much harder because one can't
>> > compile 64bit code as elf32 image and converting a elf64 image to
>> > elf32 format is a major hassle and looses debug information and symbols.
>
> So just have two versions of your image:
> - The elf64 image that has debug info, symbols, etc.  Point gdb at this.
> - An elf32 image that you give to the bootloader (in this case kvm)
>
> Generating the latter from the former is a trivial objcopy invocation.

Is it? I tried for a while and couldn't figure it out. I checked how
linux does it and it does quite a dance to achieve it.

> The bootloader variant only needs enough information to get the loadable 
> sections into memory.  We don't care about non-resident clutter like debug 
> info or symbols.  Anything that cares about those will be using the full elf64 
> image.
>
> This is all standard practice.  I don't think I've never actually used a 
> system where the image loaded by the target is the same file as the one that 
> comes out of the linker and is used by the debugger.
>
>> Yes, this is why I asked about support in other software.  If each
>> bootloader implements a different custom method then it will be a pain
>> to run your binary on real hardware in the future.  It's worth at
>> least checking their source first - maybe there is already a similar
>> code path that we can be compatible with, hence making life easier for
>> developers who want to play with 64-bit payloads in multiboot
>> executables.
>
> The whole idea of entering a 64-bit image in 32-bit mode seems distinctly 
> sketchy.  Surely it'd make more sense to define a 64-bit multiboot variant and 
> do the job properly.
>
> Paul

That will happen in multiboot2 format and is a much larger patch,
including a complete new boot rom actualy. I do have patches for that
too but there aren't complete yet. I only support the bare minimum of
the multiboot2 specs so far.

MfG
        Goswin

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Paul Brook]

> Paul Brook <paul@codesourcery.com> writes:
> >> > starting your own toy kernel is a fun thing to do and there are many
> >> > tutorials out there on how to do it. Unfortunately when one wants to
> >> > write a kernel in 64bit it becomes much harder because one can't
> >> > compile 64bit code as elf32 image and converting a elf64 image to
> >> > elf32 format is a major hassle and looses debug information and
> >> > symbols.
> > 
> > So just have two versions of your image:
> > - The elf64 image that has debug info, symbols, etc.  Point gdb at this.
> > - An elf32 image that you give to the bootloader (in this case kvm)
> > 
> > Generating the latter from the former is a trivial objcopy invocation.
> 
> Is it? I tried for a while and couldn't figure it out. I checked how
> linux does it and it does quite a dance to achieve it.

"objcopy -I elf64-x86-64 -O elf32-i386 64.elf 32.elf" worked for me.  
Relocations get a bit confused, but you shouldn't have relocations in your 
multiboot images to start with.

Linux is a bit special because it has its own boot protocol. AFAIK it can't be 
used as a regular multiboot image directly, you need to add a wrapper (i.e. a 
secondary bootloader).

Paul

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Goswin von Brederlow]

Paul Brook <paul@codesourcery.com> writes:

>> Paul Brook <paul@codesourcery.com> writes:
>> >> > starting your own toy kernel is a fun thing to do and there are many
>> >> > tutorials out there on how to do it. Unfortunately when one wants to
>> >> > write a kernel in 64bit it becomes much harder because one can't
>> >> > compile 64bit code as elf32 image and converting a elf64 image to
>> >> > elf32 format is a major hassle and looses debug information and
>> >> > symbols.
>> > 
>> > So just have two versions of your image:
>> > - The elf64 image that has debug info, symbols, etc.  Point gdb at this.
>> > - An elf32 image that you give to the bootloader (in this case kvm)
>> > 
>> > Generating the latter from the former is a trivial objcopy invocation.
>> 
>> Is it? I tried for a while and couldn't figure it out. I checked how
>> linux does it and it does quite a dance to achieve it.
>
> "objcopy -I elf64-x86-64 -O elf32-i386 64.elf 32.elf" worked for me.  
> Relocations get a bit confused, but you shouldn't have relocations in your 
> multiboot images to start with.

Why no relocations? Isn't exactly that the advantage of building an elf
image, that you can build a relocatable image?

I do remeber getting errors because x86_32_RELOC or something couldn't
be mapped to elf32-i386 output format. Can't reproduce it now
though. Your line above seems to work on my minimal hello-world kernel.

> Linux is a bit special because it has its own boot protocol. AFAIK it can't be 
> used as a regular multiboot image directly, you need to add a wrapper (i.e. a 
> secondary bootloader).
>
> Paul

I noticed. I quickly gave up using the linux kernel and build system as
reference for a simple toy kernel.

MfG
        Goswin

Re: [Qemu-devel] Support for multiboot images in elf64 (EM_X86_64) format

[Paul Brook]

> > "objcopy -I elf64-x86-64 -O elf32-i386 64.elf 32.elf" worked for me.
> > Relocations get a bit confused, but you shouldn't have relocations in
> > your multiboot images to start with.
> 
> Why no relocations? Isn't exactly that the advantage of building an elf
> image, that you can build a relocatable image?

In this context I'd say no.  The advantage of an ELF image is that it allows 
you to specify the entry point and load address. It also allows you to have 
multiple discontiguous data sections, which is often helpful.

If you want to relocate images you need a dynamic loader capable of doing that 
relocation.  The qemu loader is about the dumbest static loader possible.

Paul