Xen-ARM DomUs

Xen-ARM DomUs

[Elliott Mitchell]

Finally getting to the truly productive stages of my project with Xen on
ARM.

How many of the OSes which function as x86 DomUs for Xen, function as
ARM DomUs?  Getting Linux operational was straightforward, but what of
others?

The available examples seem geared towards Linux DomUs.  I'm looking at a
FreeBSD installation image and it appears to expect an EFI firmware.
Beyond having a bunch of files appearing oriented towards booting on EFI
I can't say much about (booting) FreeBSD/ARM DomUs.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Re: Xen-ARM DomUs

[Roman Shaposhnik]

On Mon, Dec 14, 2020 at 6:16 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
>
> Finally getting to the truly productive stages of my project with Xen on
> ARM.
>
> How many of the OSes which function as x86 DomUs for Xen, function as
> ARM DomUs?  Getting Linux operational was straightforward, but what of
> others?

On EVE we have Windows running as a pretty much a customer-facing demo:
    https://wiki.lfedge.org/display/EVE/How+get+Windows+10+running+on+a+Raspberry+Pi

> The available examples seem geared towards Linux DomUs.  I'm looking at a
> FreeBSD installation image and it appears to expect an EFI firmware.
> Beyond having a bunch of files appearing oriented towards booting on EFI
> I can't say much about (booting) FreeBSD/ARM DomUs.

Personally I'm about to make Plan9 (well 9front really) run as well ;-)

Thanks,
Roman.

Re: Xen-ARM DomUs

[Elliott Mitchell]

On Mon, Dec 14, 2020 at 06:35:14PM -0800, Roman Shaposhnik wrote:
> On Mon, Dec 14, 2020 at 6:16 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> >
> > Finally getting to the truly productive stages of my project with Xen on
> > ARM.
> >
> > How many of the OSes which function as x86 DomUs for Xen, function as
> > ARM DomUs?  Getting Linux operational was straightforward, but what of
> > others?
> 
> On EVE we have Windows running as a pretty much a customer-facing demo:
>     https://wiki.lfedge.org/display/EVE/How+get+Windows+10+running+on+a+Raspberry+Pi
> 

Sorry to spoil the achievement, but Tianocore beat you to having
Windows on a RP4 by 4 months:
https://rpi4-uefi.dev/alternate-guide-running-windows-10-on-the-pi-4/

> > The available examples seem geared towards Linux DomUs.  I'm looking at a
> > FreeBSD installation image and it appears to expect an EFI firmware.
> > Beyond having a bunch of files appearing oriented towards booting on EFI
> > I can't say much about (booting) FreeBSD/ARM DomUs.
> 
> Personally I'm about to make Plan9 (well 9front really) run as well ;-)

Some people may like those types of instructions, but I really hate them.
I like Tianocore's better, since I can do my type of adjustment better.
(using different amount of storage or other virtual devices)

I've already got FreeBSD installation media, issue is setting up a xl.cfg
file and/or figuring out which bits I need to extract off their media
(ah, actual kernel is /boot/kernel/kernel; an ELF file using the
interpreter /red/herring).


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Re: Xen-ARM DomUs

[Roman Shaposhnik]

On Mon, Dec 14, 2020 at 6:59 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
>
> On Mon, Dec 14, 2020 at 06:35:14PM -0800, Roman Shaposhnik wrote:
> > On Mon, Dec 14, 2020 at 6:16 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > >
> > > Finally getting to the truly productive stages of my project with Xen on
> > > ARM.
> > >
> > > How many of the OSes which function as x86 DomUs for Xen, function as
> > > ARM DomUs?  Getting Linux operational was straightforward, but what of
> > > others?
> >
> > On EVE we have Windows running as a pretty much a customer-facing demo:
> >     https://wiki.lfedge.org/display/EVE/How+get+Windows+10+running+on+a+Raspberry+Pi
> >
>
> Sorry to spoil the achievement, but Tianocore beat you to having
> Windows on a RP4 by 4 months:
> https://rpi4-uefi.dev/alternate-guide-running-windows-10-on-the-pi-4/

Not to be pedantic, but Stefano and I beat them -- we made it possible
around August ;-)

> > > The available examples seem geared towards Linux DomUs.  I'm looking at a
> > > FreeBSD installation image and it appears to expect an EFI firmware.
> > > Beyond having a bunch of files appearing oriented towards booting on EFI
> > > I can't say much about (booting) FreeBSD/ARM DomUs.
> >
> > Personally I'm about to make Plan9 (well 9front really) run as well ;-)
>
> Some people may like those types of instructions, but I really hate them.
> I like Tianocore's better, since I can do my type of adjustment better.
> (using different amount of storage or other virtual devices)
>
> I've already got FreeBSD installation media, issue is setting up a xl.cfg
> file and/or figuring out which bits I need to extract off their media
> (ah, actual kernel is /boot/kernel/kernel; an ELF file using the
> interpreter /red/herring).

Well, Xen requires some kind of a management solution underneath, so until
Xen/RPi4 support shows up in Raspbian -- the choice is to either stick with
EVE or follow long lists of instructions.

Thanks,
Roman.

Re: Xen-ARM DomUs

[Stefano Stabellini]

On Mon, 14 Dec 2020, Elliott Mitchell wrote:
> Finally getting to the truly productive stages of my project with Xen on
> ARM.
> 
> How many of the OSes which function as x86 DomUs for Xen, function as
> ARM DomUs?  Getting Linux operational was straightforward, but what of
> others?

I know of FreeRTOS, Zephyr, VxWorks.


> The available examples seem geared towards Linux DomUs.  I'm looking at a
> FreeBSD installation image and it appears to expect an EFI firmware.
> Beyond having a bunch of files appearing oriented towards booting on EFI
> I can't say much about (booting) FreeBSD/ARM DomUs.

Running EFI firmware in a domU is possible with both Tianocore and
U-Boot. You should be able to build the firmware and pass it as a
kernel= binary in the xl file. Then the firmware will be able to load
the necessary binaries from the virtual disk.

I ran Tianocore this way years ago. Recently, u-boot has been ported to
be run in a domU by Oleksandr Andrushchenko (CCed).

Re: Xen-ARM DomUs

[Julien Grall]

Hi,

On 15/12/2020 02:16, Elliott Mitchell wrote:
> Finally getting to the truly productive stages of my project with Xen on
> ARM.
> 
> How many of the OSes which function as x86 DomUs for Xen, function as
> ARM DomUs?  Getting Linux operational was straightforward, but what of
> others?

If you are interested to port an new OS on Xen, I would suggest to read [1].

> 
> The available examples seem geared towards Linux DomUs.  I'm looking at a
> FreeBSD installation image and it appears to expect an EFI firmware.
> Beyond having a bunch of files appearing oriented towards booting on EFI
> I can't say much about (booting) FreeBSD/ARM DomUs.

I wrote PoC a few years ago to boot FreeBSD on Xen on Arm (see [2]). I 
haven't touched it for quite a while, so you may need to use a different 
branch in that tree.

Cheers,

[1] 
https://events.static.linuxfound.org/sites/events/files/slides/Porting%20FreeBSD%20on%20Xen%20on%20ARM%20.pdf
[2] 
https://xenbits.xen.org/gitweb/?p=people/julieng/freebsd.git;a=shortlog;h=refs/heads/dev-arm64

> 
> 

-- 
Julien Grall

Re: Xen-ARM DomUs

[Julien Grall]

Hi,

On 15/12/2020 02:35, Roman Shaposhnik wrote:
> On Mon, Dec 14, 2020 at 6:16 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
>>
>> Finally getting to the truly productive stages of my project with Xen on
>> ARM.
>>
>> How many of the OSes which function as x86 DomUs for Xen, function as
>> ARM DomUs?  Getting Linux operational was straightforward, but what of
>> others?
> 
> On EVE we have Windows running as a pretty much a customer-facing demo:
>      https://wiki.lfedge.org/display/EVE/How+get+Windows+10+running+on+a+Raspberry+Pi

Are you saying that Windows is booting on top of Xen on Arm?

Cheers,

-- 
Julien Grall

Re: Xen-ARM DomUs

[Elliott Mitchell]

On Tue, Dec 15, 2020 at 08:36:34AM -0800, Stefano Stabellini wrote:
> On Mon, 14 Dec 2020, Elliott Mitchell wrote:
> > The available examples seem geared towards Linux DomUs.  I'm looking at a
> > FreeBSD installation image and it appears to expect an EFI firmware.
> > Beyond having a bunch of files appearing oriented towards booting on EFI
> > I can't say much about (booting) FreeBSD/ARM DomUs.
> 
> Running EFI firmware in a domU is possible with both Tianocore and
> U-Boot. You should be able to build the firmware and pass it as a
> kernel= binary in the xl file. Then the firmware will be able to load
> the necessary binaries from the virtual disk.

Hmm, no mention of this on:
https://wiki.xenproject.org/wiki/OVMF

In fact that appears 100% x86.  Perhaps tools/firmware needs to be
adjusted to make it work on ARM?

Really the xlexample files in tools/examples need equivalents for ARM...

*This* reads like the approach I'm looking for, but building Tianocore
is an adventure even with a good guide.

> I ran Tianocore this way years ago. Recently, u-boot has been ported to
> be run in a domU by Oleksandr Andrushchenko (CCed).

The Xen wiki has no information on this.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Re: Xen-ARM DomUs

[Stefano Stabellini]

On Tue, 15 Dec 2020, Elliott Mitchell wrote:
> On Tue, Dec 15, 2020 at 08:36:34AM -0800, Stefano Stabellini wrote:
> > On Mon, 14 Dec 2020, Elliott Mitchell wrote:
> > > The available examples seem geared towards Linux DomUs.  I'm looking at a
> > > FreeBSD installation image and it appears to expect an EFI firmware.
> > > Beyond having a bunch of files appearing oriented towards booting on EFI
> > > I can't say much about (booting) FreeBSD/ARM DomUs.
> > 
> > Running EFI firmware in a domU is possible with both Tianocore and
> > U-Boot. You should be able to build the firmware and pass it as a
> > kernel= binary in the xl file. Then the firmware will be able to load
> > the necessary binaries from the virtual disk.
> 
> Hmm, no mention of this on:
> https://wiki.xenproject.org/wiki/OVMF
> 
> In fact that appears 100% x86.  Perhaps tools/firmware needs to be
> adjusted to make it work on ARM?
> 
> Really the xlexample files in tools/examples need equivalents for ARM...
> 
> *This* reads like the approach I'm looking for, but building Tianocore
> is an adventure even with a good guide.

Tianocore has been working for many years as domU kernel, but I haven't
tried it in a while. You should definitely be able to get it to boot.
Linaro offers pre-built binaries of it with Xen enabled:

http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/4123/XEN-AARCH64/RELEASE_GCC5/


> > I ran Tianocore this way years ago. Recently, u-boot has been ported to
> > be run in a domU by Oleksandr Andrushchenko (CCed).
> 
> The Xen wiki has no information on this.

This is relatively new. Maybe Oleksandr should add a page to the wiki
when he gets a chance :-)