[Qemu-devel] [PATCH v5 0/2] aarch64: Allow -kernel option to take a gzip-compressed kernel.

[Qemu-devel] [PATCH v5 0/2] aarch64: Allow -kernel option to take a gzip-compressed kernel.

[Richard W.M. Jones]

Changes since v4:

 - Split the patch into a generic loader part, and specific arm64
   support.

 - There is now a specific limit for the gzip loader, plus a comment
   to indicate that it's just there to stop an excessive malloc.  The
   limit is now decoupled (and larger) than the u-boot limit, since
   this function might be used for non-kernels in future.

 - Remove comment about aarch64 in generic code.

Rich.

[Qemu-devel] [PATCH v5 1/2] loader: Add load_image_gzipped function.

[Richard W.M. Jones]

As the name suggests this lets you load a ROM/disk image that is
gzipped.  It is uncompressed before storing it in guest memory.

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
 hw/core/loader.c    | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/hw/loader.h |  1 +
 2 files changed, 49 insertions(+)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 2bf6b8f..e773aab 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -577,6 +577,54 @@ int load_ramdisk(const char *filename, hwaddr addr, uint64_t max_sz)
     return load_uboot_image(filename, NULL, &addr, NULL, IH_TYPE_RAMDISK);
 }
 
+/* This simply prevents g_malloc in the function below from allocating
+ * a huge amount of memory, by placing a limit on the maximum
+ * uncompressed image size that load_image_gzipped will read.
+ */
+#define LOAD_IMAGE_MAX_GUNZIP_BYTES (256 << 20)
+
+/* Load a gzip-compressed kernel. */
+int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz)
+{
+    uint8_t *compressed_data = NULL;
+    uint8_t *data = NULL;
+    gsize len;
+    ssize_t bytes;
+    int ret = -1;
+
+    if (!g_file_get_contents(filename, (char **) &compressed_data, &len,
+                             NULL)) {
+        goto out;
+    }
+
+    /* Is it a gzip-compressed file? */
+    if (len < 2 ||
+        compressed_data[0] != '\x1f' ||
+        compressed_data[1] != '\x8b') {
+        goto out;
+    }
+
+    if (max_sz > LOAD_IMAGE_MAX_GUNZIP_BYTES) {
+        max_sz = LOAD_IMAGE_MAX_GUNZIP_BYTES;
+    }
+
+    data = g_malloc(max_sz);
+    bytes = gunzip(data, max_sz, compressed_data, len);
+    if (bytes < 0) {
+        fprintf(stderr, "%s: unable to decompress gzipped kernel file\n",
+                filename);
+        goto out;
+    }
+
+    rom_add_blob_fixed(filename, data, bytes, addr);
+    ret = bytes;
+
+ out:
+    g_free(compressed_data);
+    g_free(data);
+    return ret;
+}
+
 /*
  * Functions for reboot-persistent memory regions.
  *  - used for vga bios and option roms.
diff --git a/include/hw/loader.h b/include/hw/loader.h
index 796cbf9..00c9117 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -15,6 +15,7 @@ int get_image_size(const char *filename);
 int load_image(const char *filename, uint8_t *addr); /* deprecated */
 int load_image_targphys(const char *filename, hwaddr,
                         uint64_t max_sz);
+int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz);
 
 #define ELF_LOAD_FAILED       -1
 #define ELF_LOAD_NOT_ELF      -2
-- 
2.0.4

[Qemu-devel] [PATCH v5 2/2] aarch64: Allow -kernel option to take a gzip-compressed kernel.

[Richard W.M. Jones]

On aarch64 it is the bootloader's job to uncompress the kernel.  UEFI
and u-boot bootloaders do this automatically when the kernel is
gzip-compressed.

However the qemu -kernel option does not do this.  The following
command does not work:

  qemu-system-aarch64 [...] -kernel /boot/vmlinuz

because it tries to execute the gzip-compressed data.

This commit lets gzip-compressed kernels be uncompressed
transparently.

Currently this is only done when emulating aarch64.

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
 hw/arm/boot.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 3d1f4a2..1d541db 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -444,6 +444,7 @@ static void do_cpu_reset(void *opaque)
 void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
 {
     CPUState *cs = CPU(cpu);
+    int allow_compressed_kernels = 0;
     int kernel_size;
     int initrd_size;
     int is_linux = 0;
@@ -465,6 +466,7 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
         primary_loader = bootloader_aarch64;
         kernel_load_offset = KERNEL64_LOAD_ADDR;
         elf_machine = EM_AARCH64;
+        allow_compressed_kernels = 1;
     } else {
         primary_loader = bootloader;
         kernel_load_offset = KERNEL_LOAD_ADDR;
@@ -510,6 +512,13 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
         kernel_size = load_uimage(info->kernel_filename, &entry, NULL,
                                   &is_linux);
     }
+    /* On aarch64, it's the bootloader's job to uncompress the kernel. */
+    if (allow_compressed_kernels && kernel_size < 0) {
+        entry = info->loader_start + kernel_load_offset;
+        kernel_size = load_image_gzipped(info->kernel_filename, entry,
+                                         info->ram_size - kernel_load_offset);
+        is_linux = 1;
+    }
     if (kernel_size < 0) {
         entry = info->loader_start + kernel_load_offset;
         kernel_size = load_image_targphys(info->kernel_filename, entry,
-- 
2.0.4

Re: [Qemu-devel] [PATCH v5 1/2] loader: Add load_image_gzipped function.

[Alex Bennée]

Richard W.M. Jones writes:

> As the name suggests this lets you load a ROM/disk image that is
> gzipped.  It is uncompressed before storing it in guest memory.
>
<snip>
> +
> +    /* Is it a gzip-compressed file? */
> +    if (len < 2 ||
> +        compressed_data[0] != '\x1f' ||
> +        compressed_data[1] != '\x8b') {
> +        goto out;
> +    }

I was looking to see if the zlib library provided the magic numbers or a
verification routine here but I couldn't find it.

> +
> +    if (max_sz > LOAD_IMAGE_MAX_GUNZIP_BYTES) {
> +        max_sz = LOAD_IMAGE_MAX_GUNZIP_BYTES;
> +    }
> +
> +    data = g_malloc(max_sz);
> +    bytes = gunzip(data, max_sz, compressed_data, len);
> +    if (bytes < 0) {
> +        fprintf(stderr, "%s: unable to decompress gzipped kernel file\n",
> +                filename);
> +        goto out;
> +    }
> +
> +    rom_add_blob_fixed(filename, data, bytes, addr);
> +    ret = bytes;
> +
> + out:
> +    g_free(compressed_data);
> +    g_free(data);
> +    return ret;
> +}
> +
>  /*
>   * Functions for reboot-persistent memory regions.
>   *  - used for vga bios and option roms.
> diff --git a/include/hw/loader.h b/include/hw/loader.h
> index 796cbf9..00c9117 100644
> --- a/include/hw/loader.h
> +++ b/include/hw/loader.h
> @@ -15,6 +15,7 @@ int get_image_size(const char *filename);
>  int load_image(const char *filename, uint8_t *addr); /* deprecated */
>  int load_image_targphys(const char *filename, hwaddr,
>                          uint64_t max_sz);
> +int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz);
>  
>  #define ELF_LOAD_FAILED       -1
>  #define ELF_LOAD_NOT_ELF      -2

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [Qemu-devel] [PATCH v5 1/2] loader: Add load_image_gzipped function.

[Alex Bennée]

Richard W.M. Jones writes:

> As the name suggests this lets you load a ROM/disk image that is
> gzipped.  It is uncompressed before storing it in guest memory.
>
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
<snip>
> +    /* Is it a gzip-compressed file? */
> +    if (len < 2 ||
> +        compressed_data[0] != '\x1f' ||
> +        compressed_data[1] != '\x8b') {
> +        goto out;
> +    }
<snip>

Hmm serves me right for not compiling this first. I had to explicit
literals to get this to compile:

	Modified   hw/core/loader.c
diff --git a/hw/core/loader.c b/hw/core/loader.c
index e773aab..83136e8 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -599,8 +599,8 @@ int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz)
 
     /* Is it a gzip-compressed file? */
     if (len < 2 ||
-        compressed_data[0] != '\x1f' ||
-        compressed_data[1] != '\x8b') {
+        compressed_data[0] != 0x1f ||
+        compressed_data[1] != 0x8b ) {
         goto out;
     }

Otherwise I get:
hw/core/loader.c: In function ‘load_image_gzipped’:
hw/core/loader.c:603:9: error: comparison is always true due to limited range of data type [-Werror=type-limits]
         compressed_data[1] != '\x8b') {

-- 
Alex Bennée

Re: [Qemu-devel] [PATCH v5 2/2] aarch64: Allow -kernel option to take a gzip-compressed kernel.

[Alex Bennée]

Richard W.M. Jones writes:

> On aarch64 it is the bootloader's job to uncompress the kernel.  UEFI
> and u-boot bootloaders do this automatically when the kernel is
> gzip-compressed.
>
> However the qemu -kernel option does not do this.  The following
> command does not work:
>
>   qemu-system-aarch64 [...] -kernel /boot/vmlinuz
>
> because it tries to execute the gzip-compressed data.
>
> This commit lets gzip-compressed kernels be uncompressed
> transparently.
>
> Currently this is only done when emulating aarch64.
>
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> ---
>  hw/arm/boot.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/hw/arm/boot.c b/hw/arm/boot.c
> index 3d1f4a2..1d541db 100644
> --- a/hw/arm/boot.c
> +++ b/hw/arm/boot.c
> @@ -444,6 +444,7 @@ static void do_cpu_reset(void *opaque)
>  void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
>  {
>      CPUState *cs = CPU(cpu);
> +    int allow_compressed_kernels = 0;
>      int kernel_size;
>      int initrd_size;
>      int is_linux = 0;
> @@ -465,6 +466,7 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
>          primary_loader = bootloader_aarch64;
>          kernel_load_offset = KERNEL64_LOAD_ADDR;
>          elf_machine = EM_AARCH64;
> +        allow_compressed_kernels = 1;
>      } else {
>          primary_loader = bootloader;
>          kernel_load_offset = KERNEL_LOAD_ADDR;
> @@ -510,6 +512,13 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
>          kernel_size = load_uimage(info->kernel_filename, &entry, NULL,
>                                    &is_linux);
>      }
> +    /* On aarch64, it's the bootloader's job to uncompress the kernel. */
> +    if (allow_compressed_kernels && kernel_size < 0) {
> +        entry = info->loader_start + kernel_load_offset;
> +        kernel_size = load_image_gzipped(info->kernel_filename, entry,
> +                                         info->ram_size - kernel_load_offset);
> +        is_linux = 1;
> +    }
>      if (kernel_size < 0) {
>          entry = info->loader_start + kernel_load_offset;
>          kernel_size = load_image_targphys(info->kernel_filename, entry,

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [Qemu-devel] [PATCH v5 1/2] loader: Add load_image_gzipped function.

[Richard W.M. Jones]

On Tue, Aug 05, 2014 at 10:57:26AM +0100, Alex Bennée wrote:
> 
> Richard W.M. Jones writes:
> 
> > As the name suggests this lets you load a ROM/disk image that is
> > gzipped.  It is uncompressed before storing it in guest memory.
> >
> > Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> <snip>
> > +    /* Is it a gzip-compressed file? */
> > +    if (len < 2 ||
> > +        compressed_data[0] != '\x1f' ||
> > +        compressed_data[1] != '\x8b') {
> > +        goto out;
> > +    }
> <snip>
> 
> Hmm serves me right for not compiling this first. I had to explicit
> literals to get this to compile:
> 
> 	Modified   hw/core/loader.c
> diff --git a/hw/core/loader.c b/hw/core/loader.c
> index e773aab..83136e8 100644
> --- a/hw/core/loader.c
> +++ b/hw/core/loader.c
> @@ -599,8 +599,8 @@ int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz)
>  
>      /* Is it a gzip-compressed file? */
>      if (len < 2 ||
> -        compressed_data[0] != '\x1f' ||
> -        compressed_data[1] != '\x8b') {
> +        compressed_data[0] != 0x1f ||
> +        compressed_data[1] != 0x8b ) {
>          goto out;
>      }
> 
> Otherwise I get:
> hw/core/loader.c: In function ‘load_image_gzipped’:
> hw/core/loader.c:603:9: error: comparison is always true due to limited range of data type [-Werror=type-limits]
>          compressed_data[1] != '\x8b') {

This is probably because I only compiled and tested this on aarch64
where char == unsigned char (not signed char).

I'll fix this in v6, thanks.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top