From: Christoffer Dall <christoffer.dall@linaro.org> To: cross-distro@lists.linaro.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, xen-devel@lists.xen.org Cc: Michael Casadevall <Michael.casadevall@linaro.org>, Rob Herring <rob.herring@linaro.org>, Peter Maydell <peter.maydell@linaro.org>, "marc.zyngier@arm.com" <marc.zyngier@arm.com>, Grant Likely <grant.likely@linaro.org>, Leif Lindholm <leif.lindholm@linaro.org>, Robie Basak <robie.basak@canonical.com>, Stefano Stabellini <stefano.stabellini@citrix.com>, Ian Campbell <ian.campbell@citrix.com> Subject: [RFC v2] ARM VM System Specification Date: Fri, 28 Mar 2014 11:45:17 -0700 [thread overview] Message-ID: <20140328184517.GA27219@cbox> (raw) ARM VM System Specification =========================== Goal ---- The goal of this spec is to allow suitably-built OS images to run on all ARM virtualization solutions, such as KVM or Xen. Recommendations in this spec are valid for aarch32 and aarch64 alike, and they aim to be hypervisor agnostic. Note that simply adhering to the SBSA [2] is not a valid approach, for example because the SBSA mandates EL2, which will not be available for VMs. Further, this spec also covers the aarch32 execution mode, not covered in the SBSA. Image format ------------ The image format, as presented to the VM, needs to be well-defined in order for prepared disk images to be bootable across various virtualization implementations. The raw disk format as presented to the VM must be partitioned with a GUID Partition Table (GPT). The bootable software must be placed in the EFI System Partition (ESP), using the UEFI removable media path, and must be an EFI application complying to the UEFI Specification 2.4 Revision A [6]. The ESP partition's GPT entry's partition type GUID must be C12A7328-F81F-11D2-BA4B-00A0C93EC93B and the file system must be formatted as FAT32/vfat as per Section 12.3.1.1 in [6]. The removable media path is \EFI\BOOT\BOOTARM.EFI for the aarch32 execution state and is \EFI\BOOT\BOOTAA64.EFI for the aarch64 execution state as specified in Section 3.3 (3.3 (Boot Option Variables Default Boot Behavior) and 3.4.1.1 (Removable Media Boot Behavior) in [6]. This ensures that tools for both Xen and KVM can load a binary UEFI firmware which can read and boot the EFI application in the disk image. A typical scenario will be GRUB2 packaged as an EFI application, which mounts the system boot partition and boots Linux. Virtual Firmware ---------------- The VM system must be UEFI compliant in order to be able to boot the EFI application in the ESP. It is recommended that this is achieved by loading a UEFI binary as the first software executed by the VM, which then executes the EFI application. The UEFI implementation should be compliant with UEFI Specification 2.4 Revision A [6] or later. This document strongly recommends that the VM implementation supports persistent environment storage for virtual firmware implementation in order to ensure probable use cases such as adding additional disk images to a VM or running installers to perform upgrades. This document strongly recommends that VM implementations implement persistent variable storage for their UEFI implementation. Persistent variable storage shall be a property of a VM instance, but shall not be stored as part of a portable disk image. Portable disk images shall conform to the UEFI removable disk requirements from the UEFI spec and cannot rely on on a pre-configured UEFI environment. The binary UEFI firmware implementation should not be distributed as part of the VM image, but is specific to the VM implementation. Hardware Description -------------------- The VM system must be UEFI compliant and therefore the UEFI system table will provide a means to access hardware description data. The VM implementation must provide through its UEFI implementation: a complete FDT which describes the entire VM system and will boot mainline kernels driven by device tree alone For more information about the arm and arm64 boot conventions, see Documentation/arm/Booting and Documentation/arm64/booting.txt in the Linux kernel source tree. For more information about UEFI booting, see [4] and [5]. VM Platform ----------- The specification does not mandate any specific memory map. The guest OS must be able to enumerate all processing elements, devices, and memory through HW description data (FDT) or a bus-specific mechanism such as PCI. If aarch64 physical CPUs implement support for the aarch32 execution state in EL1 and EL0 execution, it is recommended that the VM implementation supports booting the VM at EL1 in both aarch32 and aarch64 execution states. The virtual hardware platform must provide a number of mandatory peripherals: Serial console: The platform should provide a console, based on an emulated pl011, a virtio-console, or a Xen PV console. An ARM Generic Interrupt Controller v2 (GICv2) [3] or newer. GICv2 limits the the number of virtual CPUs to 8 cores, newer GIC versions removes this limitation. The ARM virtual timer and counter should be available to the VM as per the ARM Generic Timers specification in the ARM ARM [1]. It is strongly recommended that the VM implementation provides a hotpluggable bus to support hotplug of at least block and network devices. Suitable buses include a virtual PCIe bus and the Xen PV bus. For the VM image to be compliant with this spec, the following applies for the guest OS in the VM image: The guest OS must include support for pl011 UART, virtio-console, and the Xen PV console. The guest OS must include support for GICv2 and any available newer version of the GIC architecture to maintain compatibility with older VM implementations. It is strongly recommended to include support for all available (block, network, console, balloon) virtio-pci, virtio-mmio, and Xen PV drivers in the guest OS kernel or initial ramdisk. Other common peripherals for block devices, networking, and more can (and typically will) be provided, but OS software written and compiled to run on ARM VMs cannot make any assumptions about which variations of these should exist or which implementation they use (e.g. VirtIO or Xen PV). See "Hardware Description" above. Changes from previous versions of this RFC ------------------------------------------ Changes v1-v2: - Clearly specify that the guest must support the pl011, virtio-console, and Xen PV console. (Note that it was discussed to mandate a pl011 during Linaro Connect Asia 2014, but that was under the impression that the SBSA specification was an output-only no-irq serial port, which is not the case. The only two benefits from mandating a specific serial type was to handle "console=ttyAMA0" kernel command line parameters and earlyprintk; Grant Likely has submitted patches to avoid the need for "console=" parameters, and Rob Herring has submitted patches for paravirtualized earlyprintk consoles.) - Reference EFI specification for bootable paths. - Remove discussion on ACPI boot methods and do not suggest ACPI support in VMs. - Add specification about UEFI persistent variable storage and portability. References ---------- [1]: The ARM Architecture Reference Manual, ARMv8, Issue A.b http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0487a.b/index.html [2]: ARM Server Base System Architecture http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0029/index.html [3]: The ARM Generic Interrupt Controller Architecture Specifications v2.0 http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0487a.b/index.html [4]: http://www.secretlab.ca/archives/27 [5]: https://git.linaro.org/people/leif.lindholm/linux.git/blob/refs/heads/uefi-for-upstream:/Documentation/arm/uefi.txt [6]: UEFI Specification 2.4 Revision A http://www.uefi.org/sites/default/files/resources/2_4_Errata_A.pdf
WARNING: multiple messages have this Message-ID (diff)
From: christoffer.dall@linaro.org (Christoffer Dall) To: linux-arm-kernel@lists.infradead.org Subject: [RFC v2] ARM VM System Specification Date: Fri, 28 Mar 2014 11:45:17 -0700 [thread overview] Message-ID: <20140328184517.GA27219@cbox> (raw) ARM VM System Specification =========================== Goal ---- The goal of this spec is to allow suitably-built OS images to run on all ARM virtualization solutions, such as KVM or Xen. Recommendations in this spec are valid for aarch32 and aarch64 alike, and they aim to be hypervisor agnostic. Note that simply adhering to the SBSA [2] is not a valid approach, for example because the SBSA mandates EL2, which will not be available for VMs. Further, this spec also covers the aarch32 execution mode, not covered in the SBSA. Image format ------------ The image format, as presented to the VM, needs to be well-defined in order for prepared disk images to be bootable across various virtualization implementations. The raw disk format as presented to the VM must be partitioned with a GUID Partition Table (GPT). The bootable software must be placed in the EFI System Partition (ESP), using the UEFI removable media path, and must be an EFI application complying to the UEFI Specification 2.4 Revision A [6]. The ESP partition's GPT entry's partition type GUID must be C12A7328-F81F-11D2-BA4B-00A0C93EC93B and the file system must be formatted as FAT32/vfat as per Section 12.3.1.1 in [6]. The removable media path is \EFI\BOOT\BOOTARM.EFI for the aarch32 execution state and is \EFI\BOOT\BOOTAA64.EFI for the aarch64 execution state as specified in Section 3.3 (3.3 (Boot Option Variables Default Boot Behavior) and 3.4.1.1 (Removable Media Boot Behavior) in [6]. This ensures that tools for both Xen and KVM can load a binary UEFI firmware which can read and boot the EFI application in the disk image. A typical scenario will be GRUB2 packaged as an EFI application, which mounts the system boot partition and boots Linux. Virtual Firmware ---------------- The VM system must be UEFI compliant in order to be able to boot the EFI application in the ESP. It is recommended that this is achieved by loading a UEFI binary as the first software executed by the VM, which then executes the EFI application. The UEFI implementation should be compliant with UEFI Specification 2.4 Revision A [6] or later. This document strongly recommends that the VM implementation supports persistent environment storage for virtual firmware implementation in order to ensure probable use cases such as adding additional disk images to a VM or running installers to perform upgrades. This document strongly recommends that VM implementations implement persistent variable storage for their UEFI implementation. Persistent variable storage shall be a property of a VM instance, but shall not be stored as part of a portable disk image. Portable disk images shall conform to the UEFI removable disk requirements from the UEFI spec and cannot rely on on a pre-configured UEFI environment. The binary UEFI firmware implementation should not be distributed as part of the VM image, but is specific to the VM implementation. Hardware Description -------------------- The VM system must be UEFI compliant and therefore the UEFI system table will provide a means to access hardware description data. The VM implementation must provide through its UEFI implementation: a complete FDT which describes the entire VM system and will boot mainline kernels driven by device tree alone For more information about the arm and arm64 boot conventions, see Documentation/arm/Booting and Documentation/arm64/booting.txt in the Linux kernel source tree. For more information about UEFI booting, see [4] and [5]. VM Platform ----------- The specification does not mandate any specific memory map. The guest OS must be able to enumerate all processing elements, devices, and memory through HW description data (FDT) or a bus-specific mechanism such as PCI. If aarch64 physical CPUs implement support for the aarch32 execution state in EL1 and EL0 execution, it is recommended that the VM implementation supports booting the VM at EL1 in both aarch32 and aarch64 execution states. The virtual hardware platform must provide a number of mandatory peripherals: Serial console: The platform should provide a console, based on an emulated pl011, a virtio-console, or a Xen PV console. An ARM Generic Interrupt Controller v2 (GICv2) [3] or newer. GICv2 limits the the number of virtual CPUs to 8 cores, newer GIC versions removes this limitation. The ARM virtual timer and counter should be available to the VM as per the ARM Generic Timers specification in the ARM ARM [1]. It is strongly recommended that the VM implementation provides a hotpluggable bus to support hotplug of at least block and network devices. Suitable buses include a virtual PCIe bus and the Xen PV bus. For the VM image to be compliant with this spec, the following applies for the guest OS in the VM image: The guest OS must include support for pl011 UART, virtio-console, and the Xen PV console. The guest OS must include support for GICv2 and any available newer version of the GIC architecture to maintain compatibility with older VM implementations. It is strongly recommended to include support for all available (block, network, console, balloon) virtio-pci, virtio-mmio, and Xen PV drivers in the guest OS kernel or initial ramdisk. Other common peripherals for block devices, networking, and more can (and typically will) be provided, but OS software written and compiled to run on ARM VMs cannot make any assumptions about which variations of these should exist or which implementation they use (e.g. VirtIO or Xen PV). See "Hardware Description" above. Changes from previous versions of this RFC ------------------------------------------ Changes v1-v2: - Clearly specify that the guest must support the pl011, virtio-console, and Xen PV console. (Note that it was discussed to mandate a pl011 during Linaro Connect Asia 2014, but that was under the impression that the SBSA specification was an output-only no-irq serial port, which is not the case. The only two benefits from mandating a specific serial type was to handle "console=ttyAMA0" kernel command line parameters and earlyprintk; Grant Likely has submitted patches to avoid the need for "console=" parameters, and Rob Herring has submitted patches for paravirtualized earlyprintk consoles.) - Reference EFI specification for bootable paths. - Remove discussion on ACPI boot methods and do not suggest ACPI support in VMs. - Add specification about UEFI persistent variable storage and portability. References ---------- [1]: The ARM Architecture Reference Manual, ARMv8, Issue A.b http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0487a.b/index.html [2]: ARM Server Base System Architecture http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0029/index.html [3]: The ARM Generic Interrupt Controller Architecture Specifications v2.0 http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0487a.b/index.html [4]: http://www.secretlab.ca/archives/27 [5]: https://git.linaro.org/people/leif.lindholm/linux.git/blob/refs/heads/uefi-for-upstream:/Documentation/arm/uefi.txt [6]: UEFI Specification 2.4 Revision A http://www.uefi.org/sites/default/files/resources/2_4_Errata_A.pdf
next reply other threads:[~2014-03-28 18:45 UTC|newest] Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-03-28 18:45 Christoffer Dall [this message] 2014-03-28 18:45 ` [RFC v2] ARM VM System Specification Christoffer Dall 2014-03-30 22:10 ` Olof Johansson 2014-03-30 22:10 ` Olof Johansson 2014-03-30 22:10 ` Olof Johansson 2014-03-31 17:26 ` Christoffer Dall 2014-03-31 17:26 ` Christoffer Dall 2014-03-31 17:26 ` Christoffer Dall 2014-04-01 9:49 ` Ian Campbell 2014-04-01 9:49 ` Ian Campbell 2014-04-01 9:49 ` Ian Campbell 2014-04-01 9:57 ` Michael Casadevall 2014-04-01 9:57 ` Michael Casadevall 2014-04-01 9:57 ` Michael Casadevall 2014-04-01 10:16 ` Grant Likely 2014-04-01 10:16 ` Grant Likely 2014-04-01 10:16 ` Grant Likely 2014-04-29 14:42 ` Christoffer Dall 2014-04-29 14:42 ` Christoffer Dall 2014-04-29 14:42 ` Christoffer Dall 2014-04-30 8:14 ` Grant Likely 2014-04-30 8:14 ` Grant Likely 2014-04-30 8:14 ` Grant Likely 2014-06-10 14:42 ` Peter Maydell 2014-06-10 14:42 ` Peter Maydell 2014-06-10 14:42 ` Peter Maydell 2014-06-10 15:03 ` Ian Campbell 2014-06-10 15:03 ` Ian Campbell 2014-06-10 15:03 ` Ian Campbell 2014-06-10 17:00 ` Paolo Bonzini 2014-06-10 17:00 ` Paolo Bonzini 2014-06-10 17:00 ` Paolo Bonzini 2014-06-10 17:04 ` Christopher Covington 2014-06-10 17:04 ` Christopher Covington 2014-06-10 17:04 ` Christopher Covington 2014-06-10 18:08 ` Peter Maydell 2014-06-10 18:08 ` Peter Maydell 2014-06-10 18:08 ` Peter Maydell 2014-06-10 18:56 ` Paolo Bonzini 2014-06-10 18:56 ` Paolo Bonzini 2014-06-10 18:56 ` Paolo Bonzini 2014-06-10 19:18 ` Paolo Bonzini 2014-06-10 19:18 ` Paolo Bonzini 2014-06-10 19:18 ` Paolo Bonzini 2014-06-10 19:18 ` Paolo Bonzini 2014-06-10 19:18 ` Paolo Bonzini 2014-06-10 19:18 ` Paolo Bonzini 2014-06-11 6:54 ` Christoffer Dall 2014-06-11 6:54 ` Christoffer Dall 2014-06-11 6:54 ` Christoffer Dall 2014-06-11 8:16 ` Paolo Bonzini 2014-06-11 8:16 ` Paolo Bonzini 2014-06-11 8:16 ` Paolo Bonzini 2014-06-11 9:06 ` Arnd Bergmann 2014-06-11 9:06 ` Arnd Bergmann 2014-06-11 9:06 ` Arnd Bergmann 2014-06-30 16:19 ` Jon Masters 2014-06-30 16:19 ` Jon Masters 2014-06-30 16:19 ` Jon Masters 2014-06-30 20:46 ` Christoffer Dall 2014-06-30 20:46 ` Christoffer Dall 2014-06-30 21:14 ` Peter Maydell 2014-06-30 21:14 ` Peter Maydell 2014-06-30 21:14 ` Peter Maydell 2014-07-01 17:03 ` Stefano Stabellini 2014-07-01 17:03 ` Stefano Stabellini 2014-07-01 17:10 ` Peter Maydell 2014-07-01 17:10 ` Peter Maydell 2014-07-01 17:10 ` Peter Maydell 2014-07-02 10:13 ` Christoffer Dall 2014-07-02 10:13 ` Christoffer Dall 2014-07-02 10:13 ` Christoffer Dall 2014-07-01 17:03 ` Stefano Stabellini 2014-06-30 20:46 ` Christoffer Dall 2014-06-11 11:33 ` Grant Likely 2014-06-11 11:33 ` Grant Likely 2014-06-11 11:58 ` Arnd Bergmann 2014-06-11 11:58 ` Arnd Bergmann 2014-06-11 11:58 ` Arnd Bergmann 2014-06-11 12:02 ` Grant Likely 2014-06-11 12:02 ` Grant Likely 2014-06-11 12:02 ` Grant Likely 2014-06-11 14:14 ` Peter Maydell 2014-06-11 14:14 ` Peter Maydell 2014-06-11 14:14 ` Peter Maydell 2014-06-11 11:33 ` Grant Likely 2014-06-10 16:44 ` Claudio Fontana 2014-06-10 16:44 ` Claudio Fontana 2014-06-10 16:44 ` Claudio Fontana 2014-06-10 19:21 ` Arnd Bergmann 2014-06-10 19:21 ` Arnd Bergmann 2014-06-10 19:21 ` Arnd Bergmann 2014-06-11 9:50 ` Stefano Stabellini 2014-06-11 9:50 ` Stefano Stabellini 2014-06-11 9:50 ` Stefano Stabellini 2014-06-11 9:55 ` Christoffer Dall 2014-06-11 9:55 ` Christoffer Dall 2014-06-11 11:28 ` Grant Likely 2014-06-11 11:28 ` Grant Likely 2014-06-11 11:28 ` Grant Likely 2014-06-11 12:04 ` Christoffer Dall 2014-06-11 12:04 ` Christoffer Dall 2014-06-11 12:04 ` Christoffer Dall 2014-06-11 9:55 ` Christoffer Dall 2014-06-11 10:27 ` Arnd Bergmann 2014-06-11 10:27 ` Arnd Bergmann 2014-06-11 10:27 ` Arnd Bergmann 2014-06-11 11:22 ` Grant Likely 2014-06-11 11:22 ` Grant Likely 2014-06-11 11:22 ` Grant Likely 2014-03-28 18:45 Christoffer Dall
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20140328184517.GA27219@cbox \ --to=christoffer.dall@linaro.org \ --cc=Michael.casadevall@linaro.org \ --cc=cross-distro@lists.linaro.org \ --cc=grant.likely@linaro.org \ --cc=ian.campbell@citrix.com \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=leif.lindholm@linaro.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=marc.zyngier@arm.com \ --cc=peter.maydell@linaro.org \ --cc=rob.herring@linaro.org \ --cc=robie.basak@canonical.com \ --cc=stefano.stabellini@citrix.com \ --cc=xen-devel@lists.xen.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.