/usr/shared/qemu binaries

/usr/shared/qemu binaries

[Liviu Ionescu]

I successfully built two QEMU 6.2 binary packages, one with Arm and one with RISC-V, each with standalone variants for Linux Intel, Linux Arm, Windows Intel, macOS Intel and macOS Arm.

Each package includes only the relevant executables (qemu-system-arm/aarch64 and respectively qemu-system-riscv32/riscv64).

However I noticed that the `make install` also creates a `.../shared/qemu` folder with more than 200 MB of binaries.


Given that my binary packages are intended to be installed as dependencies during tests, I think it would be a waste of bandwidth to include all those binaries in all distributions.


However I don't know if any of those binaries are internally referred by various configuration options, and I'm concerned that simply removing the binaries will result in failures to start the emulation. For bare-metal emulation I expect none, but for Linux emulation there might be some (unfortunately my experience with emulating Linux is poor, and I cannot tell).


If there are such cases, could you suggest which binaries are mandatory for inclusion in the QEMU Arm package and which in the QEMU RISC-V package, if any?

Also, is there a configuration option to disable the inclusion of those binaries?


Thank you,

Liviu


------------------------------------------------------------------

p.s. For completeness, I added the list of files generated by my build of qemu-system-arm/aarch64 6.2.0:


ilg@wks qemu-arm % du -s -m share/qemu  
221	share/qemu

ilg@wks qemu-arm % ls -l share/qemu
total 448560
-rw-r--r--   1 ilg  staff       850 Jan  7 14:50 QEMU,cgthree.bin
-rw-r--r--   1 ilg  staff      1402 Jan  7 14:50 QEMU,tcx.bin
-rw-r--r--   1 ilg  staff      3211 Jan  7 14:50 bamboo.dtb
-rw-r--r--   1 ilg  staff    262144 Jan  7 14:50 bios-256k.bin
-rw-r--r--   1 ilg  staff    131072 Jan  7 14:50 bios-microvm.bin
-rw-r--r--   1 ilg  staff    131072 Jan  7 14:50 bios.bin
-rw-r--r--   1 ilg  staff      9779 Jan  7 14:50 canyonlands.dtb
-rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-aarch64-code.fd
-rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-code.fd
-rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-vars.fd
-rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-i386-code.fd
-rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-i386-secure-code.fd
-rw-r--r--   1 ilg  staff    540672 Jan  7 14:52 edk2-i386-vars.fd
-rw-r--r--   1 ilg  staff     42903 Jan  7 14:50 edk2-licenses.txt
-rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-x86_64-code.fd
-rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-x86_64-secure-code.fd
-rw-r--r--   1 ilg  staff    159232 Jan  7 14:50 efi-e1000.rom
-rw-r--r--   1 ilg  staff    159232 Jan  7 14:50 efi-e1000e.rom
-rw-r--r--   1 ilg  staff    159232 Jan  7 14:50 efi-eepro100.rom
-rw-r--r--   1 ilg  staff    157696 Jan  7 14:50 efi-ne2k_pci.rom
-rw-r--r--   1 ilg  staff    157696 Jan  7 14:50 efi-pcnet.rom
-rw-r--r--   1 ilg  staff    160768 Jan  7 14:50 efi-rtl8139.rom
-rw-r--r--   1 ilg  staff    160768 Jan  7 14:50 efi-virtio.rom
-rw-r--r--   1 ilg  staff    156672 Jan  7 14:50 efi-vmxnet3.rom
drwxr-xr-x   8 ilg  staff       256 Jan  7 14:53 firmware
-rw-r--r--   1 ilg  staff    757144 Jan  7 14:50 hppa-firmware.img
drwxr-xr-x  36 ilg  staff      1152 Jan  7 14:53 keymaps
-rw-r--r--   1 ilg  staff      9216 Jan  7 14:50 kvmvapic.bin
-rw-r--r--   1 ilg  staff      1024 Jan  7 14:50 linuxboot.bin
-rw-r--r--   1 ilg  staff      1536 Jan  7 14:50 linuxboot_dma.bin
-rw-r--r--   1 ilg  staff      1024 Jan  7 14:50 multiboot.bin
-rw-r--r--   1 ilg  staff      1024 Jan  7 14:50 multiboot_dma.bin
-rw-r--r--   1 ilg  staff       768 Jan  7 14:50 npcm7xx_bootrom.bin
-rw-r--r--   1 ilg  staff    696912 Jan  7 14:50 openbios-ppc
-rw-r--r--   1 ilg  staff    382048 Jan  7 14:50 openbios-sparc32
-rw-r--r--   1 ilg  staff   1593408 Jan  7 14:50 openbios-sparc64
-rw-r--r--   1 ilg  staff     78680 Jan  7 14:50 opensbi-riscv32-generic-fw_dynamic.bin
-rw-r--r--   1 ilg  staff    727464 Jan  7 14:50 opensbi-riscv32-generic-fw_dynamic.elf
-rw-r--r--   1 ilg  staff     75096 Jan  7 14:50 opensbi-riscv64-generic-fw_dynamic.bin
-rw-r--r--   1 ilg  staff    781264 Jan  7 14:50 opensbi-riscv64-generic-fw_dynamic.elf
-rw-r--r--   1 ilg  staff    153728 Jan  7 14:50 palcode-clipper
-rw-r--r--   1 ilg  staff      9882 Jan  7 14:50 petalogix-ml605.dtb
-rw-r--r--   1 ilg  staff      8161 Jan  7 14:50 petalogix-s3adsp1800.dtb
-rw-r--r--   1 ilg  staff      1536 Jan  7 14:50 pvh.bin
-rw-r--r--   1 ilg  staff     67072 Jan  7 14:50 pxe-e1000.rom
-rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-eepro100.rom
-rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-ne2k_pci.rom
-rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-pcnet.rom
-rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-rtl8139.rom
-rw-r--r--   1 ilg  staff     60416 Jan  7 14:50 pxe-virtio.rom
-rw-r--r--   1 ilg  staff     65536 Jan  7 14:50 qboot.rom
-rw-r--r--   1 ilg  staff    154542 Jan  7 14:50 qemu-nsis.bmp
-rw-r--r--   1 ilg  staff     18752 Jan  7 14:50 qemu_vga.ndrv
-rw-r--r--   1 ilg  staff     50936 Jan  7 14:50 s390-ccw.img
-rw-r--r--   1 ilg  staff     79688 Jan  7 14:50 s390-netboot.img
-rw-r--r--   1 ilg  staff      4096 Jan  7 14:50 sgabios.bin
-rw-r--r--   1 ilg  staff   2528128 Jan  7 14:50 skiboot.lid
-rw-r--r--   1 ilg  staff    991744 Jan  7 14:50 slof.bin
-rw-r--r--   1 ilg  staff    401308 Jan  7 14:51 trace-events-all
-rw-r--r--   1 ilg  staff    524288 Jan  7 14:50 u-boot-sam460-20100605.bin
-rw-r--r--   1 ilg  staff    421720 Jan  7 14:50 u-boot.e500
-rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-ati.bin
-rw-r--r--   1 ilg  staff     28672 Jan  7 14:50 vgabios-bochs-display.bin
-rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-cirrus.bin
-rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-qxl.bin
-rw-r--r--   1 ilg  staff     28672 Jan  7 14:50 vgabios-ramfb.bin
-rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-stdvga.bin
-rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-virtio.bin
-rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-vmware.bin
-rw-r--r--   1 ilg  staff     38912 Jan  7 14:50 vgabios.bin

Re: /usr/shared/qemu binaries

[Alistair Francis]

On Mon, Jan 10, 2022 at 8:16 PM Liviu Ionescu <ilg@livius.net> wrote:
>
> I successfully built two QEMU 6.2 binary packages, one with Arm and one with RISC-V, each with standalone variants for Linux Intel, Linux Arm, Windows Intel, macOS Intel and macOS Arm.
>
> Each package includes only the relevant executables (qemu-system-arm/aarch64 and respectively qemu-system-riscv32/riscv64).
>
> However I noticed that the `make install` also creates a `.../shared/qemu` folder with more than 200 MB of binaries.
>
>
> Given that my binary packages are intended to be installed as dependencies during tests, I think it would be a waste of bandwidth to include all those binaries in all distributions.
>
>
> However I don't know if any of those binaries are internally referred by various configuration options, and I'm concerned that simply removing the binaries will result in failures to start the emulation. For bare-metal emulation I expect none, but for Linux emulation there might be some (unfortunately my experience with emulating Linux is poor, and I cannot tell).
>
>
> If there are such cases, could you suggest which binaries are mandatory for inclusion in the QEMU Arm package and which in the QEMU RISC-V package, if any?
>
> Also, is there a configuration option to disable the inclusion of those binaries?
>
>
> Thank you,
>
> Liviu
>
>
> ------------------------------------------------------------------
>
> p.s. For completeness, I added the list of files generated by my build of qemu-system-arm/aarch64 6.2.0:
>
>
> ilg@wks qemu-arm % du -s -m share/qemu
> 221     share/qemu
>
> ilg@wks qemu-arm % ls -l share/qemu
> total 448560
> -rw-r--r--   1 ilg  staff       850 Jan  7 14:50 QEMU,cgthree.bin
> -rw-r--r--   1 ilg  staff      1402 Jan  7 14:50 QEMU,tcx.bin
> -rw-r--r--   1 ilg  staff      3211 Jan  7 14:50 bamboo.dtb
> -rw-r--r--   1 ilg  staff    262144 Jan  7 14:50 bios-256k.bin
> -rw-r--r--   1 ilg  staff    131072 Jan  7 14:50 bios-microvm.bin
> -rw-r--r--   1 ilg  staff    131072 Jan  7 14:50 bios.bin
> -rw-r--r--   1 ilg  staff      9779 Jan  7 14:50 canyonlands.dtb
> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-aarch64-code.fd
> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-code.fd
> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-vars.fd
> -rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-i386-code.fd
> -rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-i386-secure-code.fd
> -rw-r--r--   1 ilg  staff    540672 Jan  7 14:52 edk2-i386-vars.fd
> -rw-r--r--   1 ilg  staff     42903 Jan  7 14:50 edk2-licenses.txt
> -rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-x86_64-code.fd
> -rw-r--r--   1 ilg  staff   3653632 Jan  7 14:52 edk2-x86_64-secure-code.fd
> -rw-r--r--   1 ilg  staff    159232 Jan  7 14:50 efi-e1000.rom
> -rw-r--r--   1 ilg  staff    159232 Jan  7 14:50 efi-e1000e.rom
> -rw-r--r--   1 ilg  staff    159232 Jan  7 14:50 efi-eepro100.rom
> -rw-r--r--   1 ilg  staff    157696 Jan  7 14:50 efi-ne2k_pci.rom
> -rw-r--r--   1 ilg  staff    157696 Jan  7 14:50 efi-pcnet.rom
> -rw-r--r--   1 ilg  staff    160768 Jan  7 14:50 efi-rtl8139.rom
> -rw-r--r--   1 ilg  staff    160768 Jan  7 14:50 efi-virtio.rom
> -rw-r--r--   1 ilg  staff    156672 Jan  7 14:50 efi-vmxnet3.rom
> drwxr-xr-x   8 ilg  staff       256 Jan  7 14:53 firmware
> -rw-r--r--   1 ilg  staff    757144 Jan  7 14:50 hppa-firmware.img
> drwxr-xr-x  36 ilg  staff      1152 Jan  7 14:53 keymaps
> -rw-r--r--   1 ilg  staff      9216 Jan  7 14:50 kvmvapic.bin
> -rw-r--r--   1 ilg  staff      1024 Jan  7 14:50 linuxboot.bin
> -rw-r--r--   1 ilg  staff      1536 Jan  7 14:50 linuxboot_dma.bin
> -rw-r--r--   1 ilg  staff      1024 Jan  7 14:50 multiboot.bin
> -rw-r--r--   1 ilg  staff      1024 Jan  7 14:50 multiboot_dma.bin
> -rw-r--r--   1 ilg  staff       768 Jan  7 14:50 npcm7xx_bootrom.bin
> -rw-r--r--   1 ilg  staff    696912 Jan  7 14:50 openbios-ppc
> -rw-r--r--   1 ilg  staff    382048 Jan  7 14:50 openbios-sparc32
> -rw-r--r--   1 ilg  staff   1593408 Jan  7 14:50 openbios-sparc64

Most of these seem to be for other architectures besides ARM/RISC-V.
My guess would be keep *arm*/*aarch64*, keymaps, npcm7xx_bootrom.bin,
efi-* and linuxboot*/multiboot*. That should ensure that everything
works for you, but I'm just guessing here.

> -rw-r--r--   1 ilg  staff     78680 Jan  7 14:50 opensbi-riscv32-generic-fw_dynamic.bin
> -rw-r--r--   1 ilg  staff    727464 Jan  7 14:50 opensbi-riscv32-generic-fw_dynamic.elf
> -rw-r--r--   1 ilg  staff     75096 Jan  7 14:50 opensbi-riscv64-generic-fw_dynamic.bin
> -rw-r--r--   1 ilg  staff    781264 Jan  7 14:50 opensbi-riscv64-generic-fw_dynamic.elf

If you want to boot Linux on RISC-V QEMU you will need OpenSBI. You
can either use these or build and supply your own binaries.

> -rw-r--r--   1 ilg  staff    153728 Jan  7 14:50 palcode-clipper
> -rw-r--r--   1 ilg  staff      9882 Jan  7 14:50 petalogix-ml605.dtb
> -rw-r--r--   1 ilg  staff      8161 Jan  7 14:50 petalogix-s3adsp1800.dtb
> -rw-r--r--   1 ilg  staff      1536 Jan  7 14:50 pvh.bin
> -rw-r--r--   1 ilg  staff     67072 Jan  7 14:50 pxe-e1000.rom
> -rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-eepro100.rom
> -rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-ne2k_pci.rom
> -rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-pcnet.rom
> -rw-r--r--   1 ilg  staff     61440 Jan  7 14:50 pxe-rtl8139.rom
> -rw-r--r--   1 ilg  staff     60416 Jan  7 14:50 pxe-virtio.rom
> -rw-r--r--   1 ilg  staff     65536 Jan  7 14:50 qboot.rom
> -rw-r--r--   1 ilg  staff    154542 Jan  7 14:50 qemu-nsis.bmp
> -rw-r--r--   1 ilg  staff     18752 Jan  7 14:50 qemu_vga.ndrv
> -rw-r--r--   1 ilg  staff     50936 Jan  7 14:50 s390-ccw.img
> -rw-r--r--   1 ilg  staff     79688 Jan  7 14:50 s390-netboot.img
> -rw-r--r--   1 ilg  staff      4096 Jan  7 14:50 sgabios.bin
> -rw-r--r--   1 ilg  staff   2528128 Jan  7 14:50 skiboot.lid
> -rw-r--r--   1 ilg  staff    991744 Jan  7 14:50 slof.bin
> -rw-r--r--   1 ilg  staff    401308 Jan  7 14:51 trace-events-all
> -rw-r--r--   1 ilg  staff    524288 Jan  7 14:50 u-boot-sam460-20100605.bin
> -rw-r--r--   1 ilg  staff    421720 Jan  7 14:50 u-boot.e500
> -rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-ati.bin
> -rw-r--r--   1 ilg  staff     28672 Jan  7 14:50 vgabios-bochs-display.bin
> -rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-cirrus.bin
> -rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-qxl.bin
> -rw-r--r--   1 ilg  staff     28672 Jan  7 14:50 vgabios-ramfb.bin
> -rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-stdvga.bin
> -rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-virtio.bin
> -rw-r--r--   1 ilg  staff     39424 Jan  7 14:50 vgabios-vmware.bin
> -rw-r--r--   1 ilg  staff     38912 Jan  7 14:50 vgabios.bin

Not sure about these, you probably don't need them though (again, just a guess)

Alistair

>
>
>

Re: /usr/shared/qemu binaries

[Liviu Ionescu]

> On 10 Jan 2022, at 14:10, Alistair Francis <alistair23@gmail.com> wrote:
> 
> My guess would be keep *arm*/*aarch64*, keymaps, npcm7xx_bootrom.bin,
> efi-* and linuxboot*/multiboot*. That should ensure that everything
> works for you, but I'm just guessing here.

Do you know if those files are referred internally by QEMU, or the user should provide them in various command options explicitly?

About the efi-*.rom files, are they usable on Arm machines too? I thought that they are x86 specific.

> If you want to boot Linux on RISC-V QEMU you will need OpenSBI. You
> can either use these or build and supply your own binaries.

I don't know what to say, my first thought was that if those files can be supplied by the user, I'd rather not include them in the binary distribution.

But if they are referred internally, and in certain configurations QEMU does not start without them, I have to reconsider.


Regards,

Liviu

Re: /usr/shared/qemu binaries

[Alistair Francis]

On Mon, Jan 10, 2022 at 11:02 PM Liviu Ionescu <ilg@livius.net> wrote:
>
>
>
> > On 10 Jan 2022, at 14:10, Alistair Francis <alistair23@gmail.com> wrote:
> >
> > My guess would be keep *arm*/*aarch64*, keymaps, npcm7xx_bootrom.bin,
> > efi-* and linuxboot*/multiboot*. That should ensure that everything
> > works for you, but I'm just guessing here.
>
> Do you know if those files are referred internally by QEMU, or the user should provide them in various command options explicitly?

I would expect them to just be referred to by QEMU internally.

>
> About the efi-*.rom files, are they usable on Arm machines too? I thought that they are x86 specific.

They probably are x86 specific, but ARM can use EFI so
I'm not sure how that works.

>
> > If you want to boot Linux on RISC-V QEMU you will need OpenSBI. You
> > can either use these or build and supply your own binaries.
>
> I don't know what to say, my first thought was that if those files can be supplied by the user, I'd rather not include them in the binary distribution.
>
> But if they are referred internally, and in certain configurations QEMU does not start without them, I have to reconsider.

By default they are referred to internally, but if a user specifies
`-bios` when starting QEMU then they will not be used.

Alistair

>
>
> Regards,
>
> Liviu
>

Re: /usr/shared/qemu binaries

[Liviu Ionescu]

> On 11 Jan 2022, at 00:55, Alistair Francis <alistair23@gmail.com> wrote:
> 
> they are referred to internally,

Ok, so things are a bit more complicated than I hoped.

I'll search for the names in the source code, and keep those referred internally.


Thank you,

Liviu

Re: /usr/shared/qemu binaries

[Liviu Ionescu]

> On 11 Jan 2022, at 00:55, Alistair Francis <alistair23@gmail.com> wrote:
> 
> I would expect them to just be referred to by QEMU internally.

I checked and there are indeed several files referred internally.

> My guess would be keep *arm*/*aarch64*, keymaps, npcm7xx_bootrom.bin,
> efi-* and linuxboot*/multiboot*.


To start an Ubuntu 18.04 only some efi-*.rom files were needed.

The linuxboot*/multiboot* seem to be specific to x86.

>> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-aarch64-code.fd
>> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-code.fd
>> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-vars.fd

These are the top 3 largest files. They are referred as "@DATADIR@/edk2-aarch64-code.fd" from the JSONs in the firmware folder.

Any idea when are these files used? 


Liviu

Re: /usr/shared/qemu binaries

[Peter Maydell]

On Wed, 12 Jan 2022 at 13:53, Liviu Ionescu <ilg@livius.net> wrote:
> >> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-aarch64-code.fd
> >> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-code.fd
> >> -rw-r--r--   1 ilg  staff  67108864 Jan  7 14:52 edk2-arm-vars.fd
>
> These are the top 3 largest files. They are referred as "@DATADIR@/edk2-aarch64-code.fd" from the JSONs in the firmware folder.
>
> Any idea when are these files used?

Those are UEFI firmware images which are suitable for using with
the arm/aarch64 "virt" board. They're only used if the user specifically
asks to use them on the command line (eg with
"-drive if=pflash,format=raw,file=pc-bios/edk2-aarch64-code.fd" or
similar).

-- PMM

Re: /usr/shared/qemu binaries

[Liviu Ionescu]

> On 12 Jan 2022, at 15:56, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> Those are UEFI firmware images which are suitable for using with
> the arm/aarch64 "virt" board. 

Then it would probably be useful to keep them.

For the xPack QEMU Arm package, I ended up with the following script:

```
    cd .../share/qemu
    find . -type f -maxdepth 2 \
      -not \( -path './efi-*.rom' -prune \) \
      -not \( -path './npcm7xx_bootrom.bin' -prune \) \
      -not \( -path './edk2-arm*.*' -prune \) \
      -not \( -path './edk2-aarch64*.*' -prune \) \
      -not \( -path './edk2-licenses.*' -prune \) \
      -not \( -path './firmware/*-edk2-arm*.*' -prune \) \
      -not \( -path './firmware/*-edk2-aarch64*.*' -prune \) \
      -not \( -path './keymaps/*' -prune \) \
      -exec rm -rf {} \;
```

I hope it covers most use cases.

Thank you,

Liviu

Re: /usr/shared/qemu binaries

[Paolo Bonzini]

On 1/12/22 14:56, Peter Maydell wrote:
> Those are UEFI firmware images which are suitable for using with
> the arm/aarch64 "virt" board. They're only used if the user specifically
> asks to use them on the command line (eg with
> "-drive if=pflash,format=raw,file=pc-bios/edk2-aarch64-code.fd" or
> similar).

There must be lots of zeros in there. Maybe we should tell QEMU to 
unpack firmware .gz or .lzo files?

Paolo

Re: /usr/shared/qemu binaries

[Peter Maydell]

On Thu, 13 Jan 2022 at 17:13, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 1/12/22 14:56, Peter Maydell wrote:
> > Those are UEFI firmware images which are suitable for using with
> > the arm/aarch64 "virt" board. They're only used if the user specifically
> > asks to use them on the command line (eg with
> > "-drive if=pflash,format=raw,file=pc-bios/edk2-aarch64-code.fd" or
> > similar).
>
> There must be lots of zeros in there. Maybe we should tell QEMU to
> unpack firmware .gz or .lzo files?

Not hugely keen on adding more "do what I mean" behaviour...

-- PMM

Re: /usr/shared/qemu binaries

[Liviu Ionescu]

> On 13 Jan 2022, at 19:13, Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> There must be lots of zeros in there.

I confirm, when compressed, those files get significantly smaller.

> Maybe we should tell QEMU to unpack firmware .gz or .lzo files?

That would be great. I think `.gz` files are easier to create.

Liviu

Re: /usr/shared/qemu binaries

[BALATON Zoltan]

On Thu, 13 Jan 2022, Paolo Bonzini wrote:
> On 1/12/22 14:56, Peter Maydell wrote:
>> Those are UEFI firmware images which are suitable for using with
>> the arm/aarch64 "virt" board. They're only used if the user specifically
>> asks to use them on the command line (eg with
>> "-drive if=pflash,format=raw,file=pc-bios/edk2-aarch64-code.fd" or
>> similar).
>
> There must be lots of zeros in there. Maybe we should tell QEMU to unpack 
> firmware .gz or .lzo files?

May be a crazy idea, but could the above command read format=qcov2 files 
that don't need to have the zeros or may be compressed without adding any 
more support to QEMU? (Or any other compressed format already supprted by 
-drive if there are any.)

Regards,
BALATON Zoltan

Re: /usr/shared/qemu binaries

[Paolo Bonzini]

On 1/13/22 18:23, Peter Maydell wrote:
> On Thu, 13 Jan 2022 at 17:13, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> On 1/12/22 14:56, Peter Maydell wrote:
>>> Those are UEFI firmware images which are suitable for using with
>>> the arm/aarch64 "virt" board. They're only used if the user specifically
>>> asks to use them on the command line (eg with
>>> "-drive if=pflash,format=raw,file=pc-bios/edk2-aarch64-code.fd" or
>>> similar).
>>
>> There must be lots of zeros in there. Maybe we should tell QEMU to
>> unpack firmware .gz or .lzo files?
> 
> Not hugely keen on adding more "do what I mean" behaviour...

Certainly no autodetection (with writable pflash there's the possibility 
of the guest causing real problems), but we already distribute firmware 
as compressed files so the zeroes _are_ causing problems for us as well. 
  We are just telling the users to deal with it.

Paolo

Re: /usr/shared/qemu binaries

[Daniel P. Berrangé]

On Tue, Jan 18, 2022 at 01:30:35PM +0100, Paolo Bonzini wrote:
> On 1/13/22 18:23, Peter Maydell wrote:
> > On Thu, 13 Jan 2022 at 17:13, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > > 
> > > On 1/12/22 14:56, Peter Maydell wrote:
> > > > Those are UEFI firmware images which are suitable for using with
> > > > the arm/aarch64 "virt" board. They're only used if the user specifically
> > > > asks to use them on the command line (eg with
> > > > "-drive if=pflash,format=raw,file=pc-bios/edk2-aarch64-code.fd" or
> > > > similar).
> > > 
> > > There must be lots of zeros in there. Maybe we should tell QEMU to
> > > unpack firmware .gz or .lzo files?
> > 
> > Not hugely keen on adding more "do what I mean" behaviour...
> 
> Certainly no autodetection (with writable pflash there's the possibility of
> the guest causing real problems), but we already distribute firmware as
> compressed files so the zeroes _are_ causing problems for us as well.  We
> are just telling the users to deal with it.

I don't think compression will make a meaningful difference here.
The firmware is split with both code and vars files. When booting
a guest the vars file is deep copied. The vars file can't be
compressed because it needs to be writable at runtime.

So for 100 guests, you have  1 x 64 MB of code.fd plus 100 x 64 MB
of vars.fd. If only the code.fd can be compressed, that isn't going
to dent the disk space consumed.

If we want the firmware to be smaller the file could be made sparse
on disk. If the firmware only needs to be used with pflash, then
we could distribute it in qcow2 format instead of raw, which would
get rid of the zeros too.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|