[RFC v2] Add support for Xen ARM guest on FreeBSD

[RFC v2] Add support for Xen ARM guest on FreeBSD

[Julien Grall]

Hello all,

At the beginning of the year, I have sent a first RFC to add support for 
FreeBSD on Xen ARM [1].

The first version was very primitive: hardcoded DTB, only single 
user-mode support,...

Since then, I have improved the support and rebased everything on 
master. Thanks for the FreeBSD ARM team which did a great job and remove 
all most of my issues (Userspace hanging, Device Tree Bindings).

Major changes in this new version:
  	* Add Device Tree support via Linux Boot ABI
	* Add zImage support
	* Netfront support
	* Blkfront fixes
	* DOM0 support (separate branch see below)

The former item is very hackish. I was wondering if there is another way 
to do it? Or maybe we should support FreeBSD Bootloader in ARM guest?

The patch series is divided in X parts:
	* #1 - #14: Clean up and bug fixes for Xen. They can be applied without 
the rest of the series
         * #15 - #19: Update Xen interface to 4.4 and fix compilation. 
It's required for ARM.
         * #20 - #26: Update Xen code to support ARM
         * #27 - #33: Rework the event channel code for supporting ARM. 
I will work with Royger to come with a common interface with x86
         * #34 - #36: Add support for ARM in Xen code
         * #37 - #46: ARM bug fixes and new features. Some of thoses 
patches (#37 - #40) could be applied without the rest of the series
         * #47 - #48: Add Xen ARM platform

I don't really know how works patch review on Freebsd. Therefore, I 
provided the series in git format and file format. All based on Royger's 
pvh dom0 work v8:
   git://xenbits.xen.org/people/julieng/freebsd.git branch xen-arm-v2
   http://xenbits.xen.org/people/julieng/xen-arm-v2.1/

As said above, there is a separate branch for DOM0, the patches are not 
part of this series. The support has been done and demoed for the 
Arndale (though I've been tested since a while), but there is lots of 
work to clean up (device tree stuff and hack in the code):
   git://xenbits.xen.org/people/julieng/freebsd.git branch dom0-arm-v0
   http://xenbits.xen.org/people/julieng/dom0-arm-v0

TODO:
	* Add SMP/PSCI support in FreeBSD. Could be useful other platform too
	* Only FreeBSD to load anywhere. Currently there is a 2M alignment 
which require a patch in Xen.
         * ELF support in Xen ARM? Not sure it's useful.

Any help, comments, questions are welcomed.

Sincerely yours,

[1] http://lists.freebsd.org/pipermail/freebsd-xen/2014-January/001974.html

============= Instruction to test FreeBSD on Xen on ARM ===========

FreeBSD miss some support to fully boot on Xen ARM. This patch applied 
to Xen ARM help FreeBSD to boot correctly for the time being:
	* https://patches.linaro.org/32742/

To compile and boot Xen on your board, you can refer to the wiki page:
http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions

The instruction to compile FreeBSD for Xen on ARM:
$ truncate -s 512 xenvm.img
$ sudo mdconfig -f xenvm.img -u0
$ sudo newfs /dev/md0
$ sudo mount /dev/md0 /mnt

$ sudo make TARGET_ARCH=armv6 kernel-toolchain
$ sudo make TARGET_ARCH=armv6 KERNCONF=XENHVM buildkernel
$ sudo make TARGET_ARCH=armv6 buildworld
$ sudo make TARGET_ARCH=armv6 DESTDIR=/mnt installworld distribution

$ echo "/dev/xbd0       /       ufs     rw      1       1" > /mnt/etc/fstab
$ vi /mnt/etc/ttys (add the line 'xc0 "/usr/libexec/getty Pc" xterm on 
secure")

$ sudo umount /mnt
$ sudo mdconfig -d u0

Then you can copy the rootfs and the kernel to DOM 0 on your board.

To boot the a FreeBSD your will required the following configuration file
$ cat freebsd.xl
kernel="kernel"
memory=64
name="freebsd"
vcpus=1
autoballon="off"
disk=[ 'phy:/dev/loop0,xvda,w' ]
$ losetup /dev/loop0 xenvm.img
$ xl create freebsd.xl
$ xl console freebsd

-- 
Julien Grall

Re: [RFC v2] Add support for Xen ARM guest on FreeBSD

[Julien Grall]

On 02/12/2014 18:30, Warner Losh wrote:
> Hey Julien,

Hi Warner,

> Have you rebased your patch train after Andrew’s commits?

I just pushed a new branch rebased on the latest master:

git://xenbits.xen.org/people/julieng/freebsd.git branch xen-arm-v2.2

I can re-export the patch into files if necessary.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [RFC v2] Add support for Xen ARM guest on FreeBSD

[Warner Losh]

[-- Attachment #1.1: Type: text/plain, Size: 2406 bytes --]

Hey Julien,

Have you rebased your patch train after Andrew’s commits?

Warner

> On Dec 2, 2014, at 8:31 AM, Julien Grall <julien.grall@linaro.org> wrote:
> 
> Hello Andrew,
> 
> On 28/11/2014 13:57, Andrew Turner wrote:
>> On Sun, 23 Nov 2014 22:35:36 +0000
>> Julien Grall <julien.grall@linaro.org> wrote:
>>> Major changes in this new version:
>>>   	* Add Device Tree support via Linux Boot ABI
>>> 	* Add zImage support
>>> 	* Netfront support
>>> 	* Blkfront fixes
>>> 	* DOM0 support (separate branch see below)
>>> 
>>> The former item is very hackish. I was wondering if there is another
>>> way to do it? Or maybe we should support FreeBSD Bootloader in ARM
>>> guest?
>> 
>> I think using the loader is the correct way to handle booting in Xen. It
>> allows us to relocate the dtb as required. It look like a zImage then
>> use the Xen console to interact with the user.
> 
> Thanks, I will give a look to this solution.
> 
>>> 
>>> The patch series is divided in X parts:
>>> 	* #1 - #14: Clean up and bug fixes for Xen. They can be
>>> applied without the rest of the series
>>>          * #15 - #19: Update Xen interface to 4.4 and fix
>>> compilation. It's required for ARM.
>>>          * #20 - #26: Update Xen code to support ARM
>>>          * #27 - #33: Rework the event channel code for supporting
>>> ARM. I will work with Royger to come with a common interface with x86
>>>          * #34 - #36: Add support for ARM in Xen code
>>>          * #37 - #46: ARM bug fixes and new features. Some of thoses
>>> patches (#37 - #40) could be applied without the rest of the series
>>>          * #47 - #48: Add Xen ARM platform
>> 
>> I have committed patches 30 and 40 as they look good.
> 
> Thanks!
> 
>> I'm not familiar
>> with the code to review 37 or 38, however from my quick look at 38 I
>> appears _bus_dmamap_load_buffer does take in to account buflen and
>> dmat->maxsegsz when setting sgsize just not dmat->alignment.
> 
> Right, I guess I could just keep the roundup2.
> 
>> 
>> ...
>>> 
>>> TODO:
>>> 	* Add SMP/PSCI support in FreeBSD. Could be useful other
>>> platform too
>> 
>> Adding PSCI support is on my TODO lost for arm64, however I don't
>> expect to get on ti in until early next year.
> 
> BTW, what is the actual status of the ARM64 port? I plan to give a look
> for adding Xen support too.


[-- Attachment #1.2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [RFC v2] Add support for Xen ARM guest on FreeBSD

[Julien Grall]

Hello Andrew,

On 28/11/2014 13:57, Andrew Turner wrote:
> On Sun, 23 Nov 2014 22:35:36 +0000
> Julien Grall <julien.grall@linaro.org> wrote:
>> Major changes in this new version:
>>    	* Add Device Tree support via Linux Boot ABI
>> 	* Add zImage support
>> 	* Netfront support
>> 	* Blkfront fixes
>> 	* DOM0 support (separate branch see below)
>>
>> The former item is very hackish. I was wondering if there is another
>> way to do it? Or maybe we should support FreeBSD Bootloader in ARM
>> guest?
>
> I think using the loader is the correct way to handle booting in Xen. It
> allows us to relocate the dtb as required. It look like a zImage then
> use the Xen console to interact with the user.

Thanks, I will give a look to this solution.

>>
>> The patch series is divided in X parts:
>> 	* #1 - #14: Clean up and bug fixes for Xen. They can be
>> applied without the rest of the series
>>           * #15 - #19: Update Xen interface to 4.4 and fix
>> compilation. It's required for ARM.
>>           * #20 - #26: Update Xen code to support ARM
>>           * #27 - #33: Rework the event channel code for supporting
>> ARM. I will work with Royger to come with a common interface with x86
>>           * #34 - #36: Add support for ARM in Xen code
>>           * #37 - #46: ARM bug fixes and new features. Some of thoses
>> patches (#37 - #40) could be applied without the rest of the series
>>           * #47 - #48: Add Xen ARM platform
>
> I have committed patches 30 and 40 as they look good.

Thanks!

> I'm not familiar
> with the code to review 37 or 38, however from my quick look at 38 I
> appears _bus_dmamap_load_buffer does take in to account buflen and
> dmat->maxsegsz when setting sgsize just not dmat->alignment.

Right, I guess I could just keep the roundup2.

>
> ...
>>
>> TODO:
>> 	* Add SMP/PSCI support in FreeBSD. Could be useful other
>> platform too
>
> Adding PSCI support is on my TODO lost for arm64, however I don't
> expect to get on ti in until early next year.

BTW, what is the actual status of the ARM64 port? I plan to give a look
for adding Xen support too.

Regards,


-- 
Julien Grall

Re: [RFC v2] Add support for Xen ARM guest on FreeBSD

[Andrew Turner]

On Sun, 23 Nov 2014 22:35:36 +0000
Julien Grall <julien.grall@linaro.org> wrote:
> Hello all,
> 
> At the beginning of the year, I have sent a first RFC to add support
> for FreeBSD on Xen ARM [1].
...
> Major changes in this new version:
>   	* Add Device Tree support via Linux Boot ABI
> 	* Add zImage support
> 	* Netfront support
> 	* Blkfront fixes
> 	* DOM0 support (separate branch see below)
> 
> The former item is very hackish. I was wondering if there is another
> way to do it? Or maybe we should support FreeBSD Bootloader in ARM
> guest?

I think using the loader is the correct way to handle booting in Xen. It
allows us to relocate the dtb as required. It look like a zImage then
use the Xen console to interact with the user.

> 
> The patch series is divided in X parts:
> 	* #1 - #14: Clean up and bug fixes for Xen. They can be
> applied without the rest of the series
>          * #15 - #19: Update Xen interface to 4.4 and fix
> compilation. It's required for ARM.
>          * #20 - #26: Update Xen code to support ARM
>          * #27 - #33: Rework the event channel code for supporting
> ARM. I will work with Royger to come with a common interface with x86
>          * #34 - #36: Add support for ARM in Xen code
>          * #37 - #46: ARM bug fixes and new features. Some of thoses 
> patches (#37 - #40) could be applied without the rest of the series
>          * #47 - #48: Add Xen ARM platform

I have committed patches 30 and 40 as they look good. I'm not familiar
with the code to review 37 or 38, however from my quick look at 38 I
appears _bus_dmamap_load_buffer does take in to account buflen and
dmat->maxsegsz when setting sgsize just not dmat->alignment.

...
> 
> TODO:
> 	* Add SMP/PSCI support in FreeBSD. Could be useful other
> platform too

Adding PSCI support is on my TODO lost for arm64, however I don't
expect to get on ti in until early next year.

> 	* Only FreeBSD to load anywhere. Currently there is a 2M
> alignment which require a patch in Xen.

If you use the loader this will be fixed as the loader can load the
kernel at the correct alignment.