[GSoC] Xen on ARM: create multiple guests from device tree

[GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

[-- Attachment #1.1: Type: text/plain, Size: 1010 bytes --]

Hi,
I'm an undergraduate student from the Computer Science Bachelor's Degree
of the University of Modena and Reggio Emilia (Italy).

I'm very interested in the "Xen on ARM: create multiple guests from 
device tree"
project.
A year ago I started a project on the "Hikey" board with the Android 
operating
system in order to integrate BFQ - which is a proportional-share storage-I/O
scheduler - in it  improving the device performances. Now the project is 
still in
development and it is supervised by my professor which, also, told me 
about GSoC.
Since I studied the C language for a full year at university and I 
already had to deal
with kernel-stuffs, cross-compiling and so on, I chose this project.

Since I'm not very familiar with the "device tree" notion, I'm wondering 
if you could give me
some suggestions on where I could start from.

The first thing I will do is to install xen and start to familiarize 
with it.
Any advice regarding this project will be really appreciated.

Best regards,

Luca


[-- Attachment #1.2: Type: text/html, Size: 2234 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2462 bytes --]

On Tue, 21 Mar 2017, Luca Miccio wrote:
> Hi,
> I'm an undergraduate student from the Computer Science Bachelor's Degree
> of the University of Modena and Reggio Emilia (Italy).
> 
> I'm very interested in the "Xen on ARM: create multiple guests from device tree"
> project.
> A year ago I started a project on the "Hikey" board with the Android operating
> system in order to integrate BFQ - which is a proportional-share storage-I/O
> scheduler - in it  improving the device performances. Now the project is still in
> development and it is supervised by my professor which, also, told me about GSoC.
> Since I studied the C language for a full year at university and I already had to deal
> with kernel-stuffs, cross-compiling and so on, I chose this project.
> 
> Since I'm not very familiar with the "device tree" notion, I'm wondering if you could give me
> some suggestions on where I could start from.
> 
> The first thing I will do is to install xen and start to familiarize with it.
> Any advice regarding this project will be really appreciated.

Hello Luca,

Thanks for your interest in Xen on ARM! Like you wrote, I would start by
compiling Xen from source and installing it on your board. The HiKey is
a decent option, but it can be difficult to work with if you don't have
serial access. I know that uart connectors for the HiKey are sold
separetely:

http://www.96boards.org/product/uarts/

Otherwise, you could use the ARMv8 Foundation Platform, which is a
free (as in beer) software emulator: 

https://developer.arm.com/products/system-design/fixed-virtual-platforms
https://community.arm.com/processors/b/blog/posts/virtualization-on-arm-with-xen

Unfortunately all our guides on installing Xen on the Foundation Model
are outdated, but I know that Andre (CC'ed) is using it for development.
He should be able to help you if you run into any issues.


After you complete the setup of the building and testing environments,
the next thing to do is getting familiar with the code. The best way to
do that is by fixing any of the bugs or implementing any of the features
on the Xen Project Tracker:

https://xenproject.atlassian.net/projects/XEN/issues

Most of the items listed are large projects, but some of them are pretty
small, small enough to be a starting point. For example XEN-29 (Add
-fstack-protector support to the hypervisor), or XEN-10 (Clean target
makefile should remove all temporary for every architecture).

Cheers,

Stefano

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

[-- Attachment #1.1: Type: text/plain, Size: 5186 bytes --]

Hi Stefano,

2017-03-21 21:36 GMT+01:00 Stefano Stabellini <sstabellini@kernel.org>:

> On Tue, 21 Mar 2017, Luca Miccio wrote:
> > Hi,
> > I'm an undergraduate student from the Computer Science Bachelor's Degree
> > of the University of Modena and Reggio Emilia (Italy).
> >
> > I'm very interested in the "Xen on ARM: create multiple guests from
> device tree"
> > project.
> > A year ago I started a project on the "Hikey" board with the Android
> operating
> > system in order to integrate BFQ - which is a proportional-share
> storage-I/O
> > scheduler - in it  improving the device performances. Now the project is
> still in
> > development and it is supervised by my professor which, also, told me
> about GSoC.
> > Since I studied the C language for a full year at university and I
> already had to deal
> > with kernel-stuffs, cross-compiling and so on, I chose this project.
> >
> > Since I'm not very familiar with the "device tree" notion, I'm wondering
> if you could give me
> > some suggestions on where I could start from.
> >
> > The first thing I will do is to install xen and start to familiarize
> with it.
> > Any advice regarding this project will be really appreciated.
>
> Hello Luca,
>
> Thanks for your interest in Xen on ARM! Like you wrote, I would start by
> compiling Xen from source and installing it on your board. The HiKey is
> a decent option, but it can be difficult to work with if you don't have
> serial access. I know that uart connectors for the HiKey are sold
> separetely:
>
> http://www.96boards.org/product/uarts/
>
Otherwise, you could use the ARMv8 Foundation Platform, which is a
> free (as in beer) software emulator:
>
> https://developer.arm.com/products/system-design/fixed-virtual-platforms
> https://community.arm.com/processors/b/blog/posts/
> virtualization-on-arm-with-xen
>
>
I chose this option for now

Unfortunately all our guides on installing Xen on the Foundation Model
> are outdated, but I know that Andre (CC'ed) is using it for development.
> He should be able to help you if you run into any issues.
>

I tried to install and setting up xen with the blog post that you have
mentioned
above and i have some issues when compiling the boot wrapper with
the git repository provided in the post.

In particular i get this error when executing this command:
$ make CROSS_COMPILE=aarch64-linux-gnu- FDT_SRC=foundation-v8.dts
IMAGE=xen-system.axf

This is the error i get:

aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000 -DUART_BASE=0x1c090000
-DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
-DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
-DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
-DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
-DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000 -DUART_BASE=0x1c090000
-DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
-DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
-DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
-DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E -P
-C -o model.lds model.lds.S
aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
aarch64-linux-gnu-ld: section .xen LMA [0000000080a00000,0000000080ac814f]
overlaps section .kernel LMA [0000000080080000,0000000080f009ff]
Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
make: *** [xen-system.axf] Errore 1

Clearly there is a problem with the offset of xen section but i can't
figure out how to solve it.

I booted correctly a different kernel image with another .axf file only to
test the Foundation Model
and it worked with no problems.

It maybe be helpful my toolchain version:
 aarch64-linux-gnu-gcc (Debian 6.3.0-9) 6.3.0 20170316

I am using all the file provided by the post at the link ( in particular
the Foundation-v8.dts file):
https://community.arm.com/cfs-file/__key/telligent-evolution-components-attachments/01-2142-00-00-00-00-89-21/2972.zip

If you need any type of information, please ask.


However I have correctly compiled xen from source  on my laptop and i am
trying to become familiar with the hypervisor itself.

>
> After you complete the setup of the building and testing environments,
> the next thing to do is getting familiar with the code. The best way to
> do that is by fixing any of the bugs or implementing any of the features
> on the Xen Project Tracker:
>
> https://xenproject.atlassian.net/projects/XEN/issues
>
> Most of the items listed are large projects, but some of them are pretty
> small, small enough to be a starting point. For example XEN-29 (Add
> -fstack-protector support to the hypervisor), or XEN-10 (Clean target
> makefile should remove all temporary for every architecture).
>
> I took a look at the list and i have already fixed the XEN-10 task. If
it's not a problem  i
would submit the patch for this small fix, while i am figuring out how to
solve my compilation
issue described above.

Cheers,
>
> Stefano


Best regards,

Luca

[-- Attachment #1.2: Type: text/html, Size: 7639 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Julien Grall]

On 22/03/2017 19:45, Luca Miccio wrote:
> Hi Stefano,

Hello Luca,

> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
> -P -C -o model.lds model.lds.S
> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
> aarch64-linux-gnu-ld: section .xen LMA
> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
> [0000000080080000,0000000080f009ff]
> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
> make: *** [xen-system.axf] Errore 1\
> 
> Clearly there is a problem with the offset of xen section but i can't
> figure out how to solve it.

I would recommend to use the latest version of bootwrapper which is now
including support for Xen. I haven't yet updated the wiki with the runes
but here a quick summary of the step I am using:

* Building bootwrapper

42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
42sh> autoreconf -i
42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
--with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
--with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
--with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
42sh> make

Where:
  * <kernel-dir> is the path to the kernel repository
  * <dtb> is the path to the device-tree. I am using the one shipped with Linux
    in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
    It will be built if you did make all in Linux
  * <xen-bin> is the path to Xen binary (xen/xen in the repo)

* Command line for the foundation model
42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4

Where:
  * <image> is the path to xen-system.axf produced by bootwrapper
  * <rootfs> is the patch to your rootfs

Let me know if you need more details.

[...]

> I took a look at the list and i have already fixed the XEN-10 task. If
> it's not a problem  i 
> would submit the patch for this small fix, while i am figuring out how
> to solve my compilation
> issue described above.

I am fine if you send the patch now. XEN-10 is a build system modifications
that will not impact the Xen binary.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

Hi Luca,

please don't use HTML emails.


On Wed, 22 Mar 2017, Julien Grall wrote:
> On 22/03/2017 19:45, Luca Miccio wrote:
> > Hi Stefano,
> 
> Hello Luca,
> 
> > aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> > -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> > -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
> > aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> > -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
> > -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
> > -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
> > aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> > -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> > -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
> > aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> > -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
> > -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
> > -P -C -o model.lds model.lds.S
> > aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
> > aarch64-linux-gnu-ld: section .xen LMA
> > [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
> > [0000000080080000,0000000080f009ff]
> > Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
> > make: *** [xen-system.axf] Errore 1\
> > 
> > Clearly there is a problem with the offset of xen section but i can't
> > figure out how to solve it.
> 
> I would recommend to use the latest version of bootwrapper which is now
> including support for Xen. I haven't yet updated the wiki with the runes
> but here a quick summary of the step I am using:
> 
> * Building bootwrapper
> 
> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
> 42sh> autoreconf -i
> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
> 42sh> make
> 
> Where:
>   * <kernel-dir> is the path to the kernel repository
>   * <dtb> is the path to the device-tree. I am using the one shipped with Linux
>     in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
>     It will be built if you did make all in Linux
>   * <xen-bin> is the path to Xen binary (xen/xen in the repo)
> 
> * Command line for the foundation model
> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
> 
> Where:
>   * <image> is the path to xen-system.axf produced by bootwrapper
>   * <rootfs> is the patch to your rootfs
> 
> Let me know if you need more details.

Thanks Julien, I successfully booted Xen and Linux following these
steps. I quickly updated the wiki.

Luca, I took me a while to figure out that for root=/dev/vda to work
properly, Linux requires not just all the filesystem and virtio options
enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
you should be able to boot Xen, Linux and mount the root filesystem. If
you download one of the Ubuntu arm64 cloud images, the root device will
be /dev/vda1.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

Hi Stefano and Julien,

> Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
> Hi Luca,
> 
> please don't use HTML emails.
> 
> 
Sorry for that.

> On Wed, 22 Mar 2017, Julien Grall wrote:
>> On 22/03/2017 19:45, Luca Miccio wrote:
>>> Hi Stefano,
>> 
>> Hello Luca,
>> 
>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
>>> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
>>> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
>>> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
>>> -P -C -o model.lds model.lds.S
>>> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
>>> aarch64-linux-gnu-ld: section .xen LMA
>>> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
>>> [0000000080080000,0000000080f009ff]
>>> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
>>> make: *** [xen-system.axf] Errore 1\
>>> 
>>> Clearly there is a problem with the offset of xen section but i can't
>>> figure out how to solve it.
>> 
>> I would recommend to use the latest version of bootwrapper which is now
>> including support for Xen. I haven't yet updated the wiki with the runes
>> but here a quick summary of the step I am using:
>> 
>> * Building bootwrapper
>> 
>> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
>> 42sh> autoreconf -i
>> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
>> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
>> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
>> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
>> 42sh> make
>> 
>> Where:
>>  * <kernel-dir> is the path to the kernel repository
>>  * <dtb> is the path to the device-tree. I am using the one shipped with Linux
>>    in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
>>    It will be built if you did make all in Linux
>>  * <xen-bin> is the path to Xen binary (xen/xen in the repo)
>> 
>> * Command line for the foundation model
>> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
>> 
>> Where:
>>  * <image> is the path to xen-system.axf produced by bootwrapper
>>  * <rootfs> is the patch to your rootfs
>> 
>> Let me know if you need more details.
> 
> Thanks Julien, I successfully booted Xen and Linux following these
> steps. I quickly updated the wiki.
> 
> Luca, I took me a while to figure out that for root=/dev/vda to work
> properly, Linux requires not just all the filesystem and virtio options
> enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
> you should be able to boot Xen, Linux and mount the root filesystem. If
> you download one of the Ubuntu arm64 cloud images, the root device will
> be /dev/vda1.

First of all, thank you for your help.

I think that i keep on doing somenthing wrong because now the bootwrapper produces
with no errors the xen-system.axf  with the configuration that Julien provided. 
But when i launch the emulator it simply stops at:
(XEN) Bringing up CPU1

To help you understand, this is what i have done:
- First i compiled the xen source with the following command:
$ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
( I am using the staging branch)

- Get the linux source with git: 
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

-Checkout on the v4.4 branch 

- Follow the steps described in the blog and, as you said, i set in my .config file:
 CONFIG_VIRTIO=y
 CONFIG_VIRTIO_MMIO=y
 CONFIG_VIRTIO_BLK=y
I also checked all the filesystem and virtio options. 

- Compile the linux kernel with the toolchain:
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)

- Configured the bootwrapper with the command that Julian wrote and then launched the 
foundation model with the simple command:
$ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4

The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/

But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
properly to /dev/vda1.

Maybe my kernel configuration is wrong?

Thank you,
Luca


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

On Thu, 23 Mar 2017, Luca Miccio wrote:
> Hi Stefano and Julien,
> 
> > Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> > 
> > Hi Luca,
> > 
> > please don't use HTML emails.
> > 
> > 
> Sorry for that.
> 
> > On Wed, 22 Mar 2017, Julien Grall wrote:
> >> On 22/03/2017 19:45, Luca Miccio wrote:
> >>> Hi Stefano,
> >> 
> >> Hello Luca,
> >> 
> >>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> >>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> >>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
> >>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> >>> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
> >>> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
> >>> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
> >>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> >>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> >>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
> >>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> >>> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
> >>> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
> >>> -P -C -o model.lds model.lds.S
> >>> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
> >>> aarch64-linux-gnu-ld: section .xen LMA
> >>> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
> >>> [0000000080080000,0000000080f009ff]
> >>> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
> >>> make: *** [xen-system.axf] Errore 1\
> >>> 
> >>> Clearly there is a problem with the offset of xen section but i can't
> >>> figure out how to solve it.
> >> 
> >> I would recommend to use the latest version of bootwrapper which is now
> >> including support for Xen. I haven't yet updated the wiki with the runes
> >> but here a quick summary of the step I am using:
> >> 
> >> * Building bootwrapper
> >> 
> >> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
> >> 42sh> autoreconf -i
> >> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
> >> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
> >> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
> >> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
> >> 42sh> make
> >> 
> >> Where:
> >>  * <kernel-dir> is the path to the kernel repository
> >>  * <dtb> is the path to the device-tree. I am using the one shipped with Linux
> >>    in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
> >>    It will be built if you did make all in Linux
> >>  * <xen-bin> is the path to Xen binary (xen/xen in the repo)
> >> 
> >> * Command line for the foundation model
> >> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
> >> 
> >> Where:
> >>  * <image> is the path to xen-system.axf produced by bootwrapper
> >>  * <rootfs> is the patch to your rootfs
> >> 
> >> Let me know if you need more details.
> > 
> > Thanks Julien, I successfully booted Xen and Linux following these
> > steps. I quickly updated the wiki.
> > 
> > Luca, I took me a while to figure out that for root=/dev/vda to work
> > properly, Linux requires not just all the filesystem and virtio options
> > enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
> > you should be able to boot Xen, Linux and mount the root filesystem. If
> > you download one of the Ubuntu arm64 cloud images, the root device will
> > be /dev/vda1.
> 
> First of all, thank you for your help.
> 
> I think that i keep on doing somenthing wrong because now the bootwrapper produces
> with no errors the xen-system.axf  with the configuration that Julien provided. 
> But when i launch the emulator it simply stops at:
> (XEN) Bringing up CPU1
> 
> To help you understand, this is what i have done:
> - First i compiled the xen source with the following command:
> $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
> ( I am using the staging branch)
> 
> - Get the linux source with git: 
> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> 
> -Checkout on the v4.4 branch 
> 
> - Follow the steps described in the blog and, as you said, i set in my .config file:
>  CONFIG_VIRTIO=y
>  CONFIG_VIRTIO_MMIO=y
>  CONFIG_VIRTIO_BLK=y
> I also checked all the filesystem and virtio options. 
> 
> - Compile the linux kernel with the toolchain:
> $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)
> 
> - Configured the bootwrapper with the command that Julian wrote and then launched the 
> foundation model with the simple command:
> $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4

Try removing --cores=4, does it work?


> The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/
> 
> But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
> properly to /dev/vda1.
> 
> Maybe my kernel configuration is wrong?
> 
> Thank you,
> Luca
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

> Il giorno 23 mar 2017, alle ore 01:27, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
> On Thu, 23 Mar 2017, Luca Miccio wrote:
>> Hi Stefano and Julien,
>> 
>>> Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
>>> 
>>> Hi Luca,
>>> 
>>> please don't use HTML emails.
>>> 
>>> 
>> Sorry for that.
>> 
>>> On Wed, 22 Mar 2017, Julien Grall wrote:
>>>> On 22/03/2017 19:45, Luca Miccio wrote:
>>>>> Hi Stefano,
>>>> 
>>>> Hello Luca,
>>>> 
>>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
>>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>>>> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
>>>>> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
>>>>> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
>>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
>>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>>>> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
>>>>> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
>>>>> -P -C -o model.lds model.lds.S
>>>>> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
>>>>> aarch64-linux-gnu-ld: section .xen LMA
>>>>> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
>>>>> [0000000080080000,0000000080f009ff]
>>>>> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
>>>>> make: *** [xen-system.axf] Errore 1\
>>>>> 
>>>>> Clearly there is a problem with the offset of xen section but i can't
>>>>> figure out how to solve it.
>>>> 
>>>> I would recommend to use the latest version of bootwrapper which is now
>>>> including support for Xen. I haven't yet updated the wiki with the runes
>>>> but here a quick summary of the step I am using:
>>>> 
>>>> * Building bootwrapper
>>>> 
>>>> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
>>>> 42sh> autoreconf -i
>>>> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
>>>> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
>>>> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
>>>> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
>>>> 42sh> make
>>>> 
>>>> Where:
>>>> * <kernel-dir> is the path to the kernel repository
>>>> * <dtb> is the path to the device-tree. I am using the one shipped with Linux
>>>>   in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
>>>>   It will be built if you did make all in Linux
>>>> * <xen-bin> is the path to Xen binary (xen/xen in the repo)
>>>> 
>>>> * Command line for the foundation model
>>>> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
>>>> 
>>>> Where:
>>>> * <image> is the path to xen-system.axf produced by bootwrapper
>>>> * <rootfs> is the patch to your rootfs
>>>> 
>>>> Let me know if you need more details.
>>> 
>>> Thanks Julien, I successfully booted Xen and Linux following these
>>> steps. I quickly updated the wiki.
>>> 
>>> Luca, I took me a while to figure out that for root=/dev/vda to work
>>> properly, Linux requires not just all the filesystem and virtio options
>>> enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
>>> you should be able to boot Xen, Linux and mount the root filesystem. If
>>> you download one of the Ubuntu arm64 cloud images, the root device will
>>> be /dev/vda1.
>> 
>> First of all, thank you for your help.
>> 
>> I think that i keep on doing somenthing wrong because now the bootwrapper produces
>> with no errors the xen-system.axf  with the configuration that Julien provided. 
>> But when i launch the emulator it simply stops at:
>> (XEN) Bringing up CPU1
>> 
>> To help you understand, this is what i have done:
>> - First i compiled the xen source with the following command:
>> $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
>> ( I am using the staging branch)
>> 
>> - Get the linux source with git: 
>> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>> 
>> -Checkout on the v4.4 branch 
>> 
>> - Follow the steps described in the blog and, as you said, i set in my .config file:
>> CONFIG_VIRTIO=y
>> CONFIG_VIRTIO_MMIO=y
>> CONFIG_VIRTIO_BLK=y
>> I also checked all the filesystem and virtio options. 
>> 
>> - Compile the linux kernel with the toolchain:
>> $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)
>> 
>> - Configured the bootwrapper with the command that Julian wrote and then launched the 
>> foundation model with the simple command:
>> $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4
> 
> Try removing --cores=4, does it work?
> 
I have already tried this option with no results
>> The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/
>> 
>> But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
>> properly to /dev/vda1.
>> 
>> Maybe my kernel configuration is wrong?
>> 
>> Thank you,
>> Luca
>> 
Luca


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 6448 bytes --]

On Thu, 23 Mar 2017, Luca Miccio wrote:
> > Il giorno 23 mar 2017, alle ore 01:27, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> > 
> > On Thu, 23 Mar 2017, Luca Miccio wrote:
> >> Hi Stefano and Julien,
> >> 
> >>> Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> >>> 
> >>> Hi Luca,
> >>> 
> >>> please don't use HTML emails.
> >>> 
> >>> 
> >> Sorry for that.
> >> 
> >>> On Wed, 22 Mar 2017, Julien Grall wrote:
> >>>> On 22/03/2017 19:45, Luca Miccio wrote:
> >>>>> Hi Stefano,
> >>>> 
> >>>> Hello Luca,
> >>>> 
> >>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> >>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> >>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
> >>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> >>>>> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
> >>>>> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
> >>>>> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
> >>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
> >>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
> >>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
> >>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
> >>>>> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
> >>>>> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
> >>>>> -P -C -o model.lds model.lds.S
> >>>>> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
> >>>>> aarch64-linux-gnu-ld: section .xen LMA
> >>>>> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
> >>>>> [0000000080080000,0000000080f009ff]
> >>>>> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
> >>>>> make: *** [xen-system.axf] Errore 1\
> >>>>> 
> >>>>> Clearly there is a problem with the offset of xen section but i can't
> >>>>> figure out how to solve it.
> >>>> 
> >>>> I would recommend to use the latest version of bootwrapper which is now
> >>>> including support for Xen. I haven't yet updated the wiki with the runes
> >>>> but here a quick summary of the step I am using:
> >>>> 
> >>>> * Building bootwrapper
> >>>> 
> >>>> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
> >>>> 42sh> autoreconf -i
> >>>> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
> >>>> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
> >>>> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
> >>>> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
> >>>> 42sh> make
> >>>> 
> >>>> Where:
> >>>> * <kernel-dir> is the path to the kernel repository
> >>>> * <dtb> is the path to the device-tree. I am using the one shipped with Linux
> >>>>   in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
> >>>>   It will be built if you did make all in Linux
> >>>> * <xen-bin> is the path to Xen binary (xen/xen in the repo)
> >>>> 
> >>>> * Command line for the foundation model
> >>>> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
> >>>> 
> >>>> Where:
> >>>> * <image> is the path to xen-system.axf produced by bootwrapper
> >>>> * <rootfs> is the patch to your rootfs
> >>>> 
> >>>> Let me know if you need more details.
> >>> 
> >>> Thanks Julien, I successfully booted Xen and Linux following these
> >>> steps. I quickly updated the wiki.
> >>> 
> >>> Luca, I took me a while to figure out that for root=/dev/vda to work
> >>> properly, Linux requires not just all the filesystem and virtio options
> >>> enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
> >>> you should be able to boot Xen, Linux and mount the root filesystem. If
> >>> you download one of the Ubuntu arm64 cloud images, the root device will
> >>> be /dev/vda1.
> >> 
> >> First of all, thank you for your help.
> >> 
> >> I think that i keep on doing somenthing wrong because now the bootwrapper produces
> >> with no errors the xen-system.axf  with the configuration that Julien provided. 
> >> But when i launch the emulator it simply stops at:
> >> (XEN) Bringing up CPU1
> >> 
> >> To help you understand, this is what i have done:
> >> - First i compiled the xen source with the following command:
> >> $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
> >> ( I am using the staging branch)
> >> 
> >> - Get the linux source with git: 
> >> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> >> 
> >> -Checkout on the v4.4 branch 
> >> 
> >> - Follow the steps described in the blog and, as you said, i set in my .config file:
> >> CONFIG_VIRTIO=y
> >> CONFIG_VIRTIO_MMIO=y
> >> CONFIG_VIRTIO_BLK=y
> >> I also checked all the filesystem and virtio options. 
> >> 
> >> - Compile the linux kernel with the toolchain:
> >> $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)
> >> 
> >> - Configured the bootwrapper with the command that Julian wrote and then launched the 
> >> foundation model with the simple command:
> >> $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4
> > 
> > Try removing --cores=4, does it work?
> > 
> I have already tried this option with no results

The problem is that no matter the --cores=4 option, the device tree
(foundation-v8.dtb) has 4 cpu cores by default. It also says that the
"enable-method" is "spin-table". At the same time Julien wrote to pass
--enable-psci to the bootwrapper, which is a different way to start
secondary cpus. I am guessing that is the problem. I suggest to change
boot method to psci, to do that you need to modify the dts (device tree
source), see the attached patch for Linux. Then you can rebuild the dtb
(device tree binary) with:

make dtbs

Finally rebuild again the bootwrapper and run the model with --cores 4.

For me, that works. Great way to get familiar with device tree ;-)


> >> The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/
> >> 
> >> But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
> >> properly to /dev/vda1.
> >> 
> >> Maybe my kernel configuration is wrong?

[-- Attachment #2: Type: TEXT/PLAIN, Size: 1209 bytes --]

diff --git a/arch/arm64/boot/dts/arm/foundation-v8.dtsi b/arch/arm64/boot/dts/arm/foundation-v8.dtsi
index 7cfa8e4..daf2aab 100644
--- a/arch/arm64/boot/dts/arm/foundation-v8.dtsi
+++ b/arch/arm64/boot/dts/arm/foundation-v8.dtsi
@@ -32,7 +32,7 @@
 			device_type = "cpu";
 			compatible = "arm,armv8";
 			reg = <0x0 0x0>;
-			enable-method = "spin-table";
+			enable-method = "psci";
 			cpu-release-addr = <0x0 0x8000fff8>;
 			next-level-cache = <&L2_0>;
 		};
@@ -40,7 +40,7 @@
 			device_type = "cpu";
 			compatible = "arm,armv8";
 			reg = <0x0 0x1>;
-			enable-method = "spin-table";
+			enable-method = "psci";
 			cpu-release-addr = <0x0 0x8000fff8>;
 			next-level-cache = <&L2_0>;
 		};
@@ -48,7 +48,7 @@
 			device_type = "cpu";
 			compatible = "arm,armv8";
 			reg = <0x0 0x2>;
-			enable-method = "spin-table";
+			enable-method = "psci";
 			cpu-release-addr = <0x0 0x8000fff8>;
 			next-level-cache = <&L2_0>;
 		};
@@ -56,7 +56,7 @@
 			device_type = "cpu";
 			compatible = "arm,armv8";
 			reg = <0x0 0x3>;
-			enable-method = "spin-table";
+			enable-method = "psci";
 			cpu-release-addr = <0x0 0x8000fff8>;
 			next-level-cache = <&L2_0>;
 		};

[-- Attachment #3: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

[-- Attachment #1.1: Type: text/plain, Size: 7230 bytes --]


> Il giorno 23 mar 2017, alle ore 01:55, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
> On Thu, 23 Mar 2017, Luca Miccio wrote:
>>> Il giorno 23 mar 2017, alle ore 01:27, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
>>> 
>>> On Thu, 23 Mar 2017, Luca Miccio wrote:
>>>> Hi Stefano and Julien,
>>>> 
>>>>> Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
>>>>> 
>>>>> Hi Luca,
>>>>> 
>>>>> please don't use HTML emails.
>>>>> 
>>>>> 
>>>> Sorry for that.
>>>> 
>>>>> On Wed, 22 Mar 2017, Julien Grall wrote:
>>>>>> On 22/03/2017 19:45, Luca Miccio wrote:
>>>>>>> Hi Stefano,
>>>>>> 
>>>>>> Hello Luca,
>>>>>> 
>>>>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>>>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>>>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
>>>>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>>>>>> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
>>>>>>> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
>>>>>>> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
>>>>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>>>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>>>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
>>>>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>>>>>> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
>>>>>>> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
>>>>>>> -P -C -o model.lds model.lds.S
>>>>>>> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
>>>>>>> aarch64-linux-gnu-ld: section .xen LMA
>>>>>>> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
>>>>>>> [0000000080080000,0000000080f009ff]
>>>>>>> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
>>>>>>> make: *** [xen-system.axf] Errore 1\
>>>>>>> 
>>>>>>> Clearly there is a problem with the offset of xen section but i can't
>>>>>>> figure out how to solve it.
>>>>>> 
>>>>>> I would recommend to use the latest version of bootwrapper which is now
>>>>>> including support for Xen. I haven't yet updated the wiki with the runes
>>>>>> but here a quick summary of the step I am using:
>>>>>> 
>>>>>> * Building bootwrapper
>>>>>> 
>>>>>> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
>>>>>> 42sh> autoreconf -i
>>>>>> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
>>>>>> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
>>>>>> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
>>>>>> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
>>>>>> 42sh> make
>>>>>> 
>>>>>> Where:
>>>>>> * <kernel-dir> is the path to the kernel repository
>>>>>> * <dtb> is the path to the device-tree. I am using the one shipped with Linux
>>>>>>  in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
>>>>>>  It will be built if you did make all in Linux
>>>>>> * <xen-bin> is the path to Xen binary (xen/xen in the repo)
>>>>>> 
>>>>>> * Command line for the foundation model
>>>>>> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
>>>>>> 
>>>>>> Where:
>>>>>> * <image> is the path to xen-system.axf produced by bootwrapper
>>>>>> * <rootfs> is the patch to your rootfs
>>>>>> 
>>>>>> Let me know if you need more details.
>>>>> 
>>>>> Thanks Julien, I successfully booted Xen and Linux following these
>>>>> steps. I quickly updated the wiki.
>>>>> 
>>>>> Luca, I took me a while to figure out that for root=/dev/vda to work
>>>>> properly, Linux requires not just all the filesystem and virtio options
>>>>> enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
>>>>> you should be able to boot Xen, Linux and mount the root filesystem. If
>>>>> you download one of the Ubuntu arm64 cloud images, the root device will
>>>>> be /dev/vda1.
>>>> 
>>>> First of all, thank you for your help.
>>>> 
>>>> I think that i keep on doing somenthing wrong because now the bootwrapper produces
>>>> with no errors the xen-system.axf  with the configuration that Julien provided. 
>>>> But when i launch the emulator it simply stops at:
>>>> (XEN) Bringing up CPU1
>>>> 
>>>> To help you understand, this is what i have done:
>>>> - First i compiled the xen source with the following command:
>>>> $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
>>>> ( I am using the staging branch)
>>>> 
>>>> - Get the linux source with git: 
>>>> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>>>> 
>>>> -Checkout on the v4.4 branch 
>>>> 
>>>> - Follow the steps described in the blog and, as you said, i set in my .config file:
>>>> CONFIG_VIRTIO=y
>>>> CONFIG_VIRTIO_MMIO=y
>>>> CONFIG_VIRTIO_BLK=y
>>>> I also checked all the filesystem and virtio options. 
>>>> 
>>>> - Compile the linux kernel with the toolchain:
>>>> $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)
>>>> 
>>>> - Configured the bootwrapper with the command that Julian wrote and then launched the 
>>>> foundation model with the simple command:
>>>> $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4
>>> 
>>> Try removing --cores=4, does it work?
>>> 
>> I have already tried this option with no results
> 
> The problem is that no matter the --cores=4 option, the device tree
> (foundation-v8.dtb) has 4 cpu cores by default. It also says that the
> "enable-method" is "spin-table". At the same time Julien wrote to pass
> --enable-psci to the bootwrapper, which is a different way to start
> secondary cpus. I am guessing that is the problem. I suggest to change
> boot method to psci, to do that you need to modify the dts (device tree
> source), see the attached patch for Linux. Then you can rebuild the dtb
> (device tree binary) with:
> 
> make dtbs
> 
> Finally rebuild again the bootwrapper and run the model with --cores 4.
> 
> For me, that works. Great way to get familiar with device tree ;-)

I tried also this method but i had the same error. So i came up with the idea of removing the
psci method from the configure and also from the dtsi file that i have changed. 

With this configuration the model booted up correctly, even if the process took about 10 minutes.
Just for my personal knowledge: it is normal that the boot is very slow? Is the rootfs that i’m using?


Thank you for your response,
Luca


> 
>>>> The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/ <https://cloud-images.ubuntu.com/releases/16.04/release/>
>>>> 
>>>> But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
>>>> properly to /dev/vda1.
>>>> 
>>>> Maybe my kernel configuration is wrong?
> <1.txt>




[-- Attachment #1.2: Type: text/html, Size: 20119 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Julien Grall]

Hello Luca,

On 23/03/17 02:23, Luca Miccio wrote:
> I tried also this method but i had the same error. So i came up with the
> idea of removing the
> psci method from the configure and also from the dtsi file that i have
> changed.

I guess you are using only 1 core in this case. Am I right?

>
> With this configuration the model booted up correctly, even if the
> process took about 10 minutes.
> Just for my personal knowledge: it is normal that the boot is very slow?
> Is the rootfs that i’m using?

Booting a full distro on the model can be slow. Usually I write my own 
initscript to start the necessary daemon and skip systemd.

42sh> cat /root/init.sh

set -x
mount -t proc proc  /proc
mount -t devtmpfs dev /dev

export HOME=/root

rm -rf /var/run/xenstore
rm -rf /var/run/xen

/etc/init.d/xencommons start

exec /bin/bash

-------------------

And I pass init=/root/init.sh on the Linux kernel command line.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

Hello Julien
> Il giorno 23 mar 2017, alle ore 15:49, Julien Grall <julien.grall@arm.com> ha scritto:
> 
> Hello Luca,
> 
> On 23/03/17 02:23, Luca Miccio wrote:
>> I tried also this method but i had the same error. So i came up with the
>> idea of removing the
>> psci method from the configure and also from the dtsi file that i have
>> changed.
> 
> I guess you are using only 1 core in this case. Am I right?
> 
No, i am using 4 cores, with —cores option and i think that i am using actually 4 cores
because on the model “control panel” i have 4 green triangles which ones of them should
represent the number of cores and their status. 
It is only my opinion so, please, tell me if i am wrong.
>> 
>> With this configuration the model booted up correctly, even if the
>> process took about 10 minutes.
>> Just for my personal knowledge: it is normal that the boot is very slow?
>> Is the rootfs that i’m using?
> 
> Booting a full distro on the model can be slow. Usually I write my own initscript to start the necessary daemon and skip systemd.
> 
> 42sh> cat /root/init.sh
> 
> set -x
> mount -t proc proc  /proc
> mount -t devtmpfs dev /dev
> 
> export HOME=/root
> 
> rm -rf /var/run/xenstore
> rm -rf /var/run/xen
> 
> /etc/init.d/xencommons start
> 
> exec /bin/bash
> 
> -------------------
> 
> And I pass init=/root/init.sh on the Linux kernel command line.
> 
That helped a lot, thank you.
> Cheers,
> 
> -- 
> Julien Grall

Luca.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2543 bytes --]

Hello Luca,

did you manage to make any progress on the number of cores exposed by the model?

Thank you for the Makefile patch! If you have time, I encourage you to
fix one of the other small outstanding bugs, for example XEN-20 or
XEN-11 (just the ability to load a Dom0 kernel bigger than 128MB or just
the ability to parse the new image_size field introduced in Linux 3.17).

Finally, I would like to remind you of the upcoming deadline for
applications submissions, which is the 3rd of April for GSoC, see:

http://marc.info/?l=xen-devel&m=149071502330534

The GSoC application system is
https://summerofcode.withgoogle.com/get-started/.

Please give a look at the Xen Project application template here:

https://wiki.xenproject.org/wiki/GSoC_Student_Application_Template

It also includes an "Implementation Plan", where you have the chance to
explain the implementation plan for the project you would like to apply
for.

Thanks,

Stefano

On Thu, 23 Mar 2017, Luca Miccio wrote:
> Hello Julien
> > Il giorno 23 mar 2017, alle ore 15:49, Julien Grall <julien.grall@arm.com> ha scritto:
> > 
> > Hello Luca,
> > 
> > On 23/03/17 02:23, Luca Miccio wrote:
> >> I tried also this method but i had the same error. So i came up with the
> >> idea of removing the
> >> psci method from the configure and also from the dtsi file that i have
> >> changed.
> > 
> > I guess you are using only 1 core in this case. Am I right?
> > 
> No, i am using 4 cores, with —cores option and i think that i am using actually 4 cores
> because on the model “control panel” i have 4 green triangles which ones of them should
> represent the number of cores and their status. 
> It is only my opinion so, please, tell me if i am wrong.
> >> 
> >> With this configuration the model booted up correctly, even if the
> >> process took about 10 minutes.
> >> Just for my personal knowledge: it is normal that the boot is very slow?
> >> Is the rootfs that i’m using?
> > 
> > Booting a full distro on the model can be slow. Usually I write my own initscript to start the necessary daemon and skip systemd.
> > 
> > 42sh> cat /root/init.sh
> > 
> > set -x
> > mount -t proc proc  /proc
> > mount -t devtmpfs dev /dev
> > 
> > export HOME=/root
> > 
> > rm -rf /var/run/xenstore
> > rm -rf /var/run/xen
> > 
> > /etc/init.d/xencommons start
> > 
> > exec /bin/bash
> > 
> > -------------------
> > 
> > And I pass init=/root/init.sh on the Linux kernel command line.
> > 
> That helped a lot, thank you.
> > Cheers,
> > 
> > -- 
> > Julien Grall
> 
> Luca.
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2836 bytes --]

On Tue, 28 Mar 2017, Stefano Stabellini wrote:
> Hello Luca,
> 
> did you manage to make any progress on the number of cores exposed by the model?
> 
> Thank you for the Makefile patch! If you have time, I encourage you to
> fix one of the other small outstanding bugs, for example XEN-20 or
> XEN-11 (just the ability to load a Dom0 kernel bigger than 128MB or just
> the ability to parse the new image_size field introduced in Linux 3.17).
> 
> Finally, I would like to remind you of the upcoming deadline for
> applications submissions, which is the 3rd of April for GSoC, see:
> 
> http://marc.info/?l=xen-devel&m=149071502330534
> 
> The GSoC application system is
> https://summerofcode.withgoogle.com/get-started/.
> 
> Please give a look at the Xen Project application template here:
> 
> https://wiki.xenproject.org/wiki/GSoC_Student_Application_Template
> 
> It also includes an "Implementation Plan", where you have the chance to
> explain the implementation plan for the project you would like to apply
> for.

FYI I added some more information on this project here:
http://marc.info/?l=xen-devel&m=149074542307880



> 
> On Thu, 23 Mar 2017, Luca Miccio wrote:
> > Hello Julien
> > > Il giorno 23 mar 2017, alle ore 15:49, Julien Grall <julien.grall@arm.com> ha scritto:
> > > 
> > > Hello Luca,
> > > 
> > > On 23/03/17 02:23, Luca Miccio wrote:
> > >> I tried also this method but i had the same error. So i came up with the
> > >> idea of removing the
> > >> psci method from the configure and also from the dtsi file that i have
> > >> changed.
> > > 
> > > I guess you are using only 1 core in this case. Am I right?
> > > 
> > No, i am using 4 cores, with —cores option and i think that i am using actually 4 cores
> > because on the model “control panel” i have 4 green triangles which ones of them should
> > represent the number of cores and their status. 
> > It is only my opinion so, please, tell me if i am wrong.
> > >> 
> > >> With this configuration the model booted up correctly, even if the
> > >> process took about 10 minutes.
> > >> Just for my personal knowledge: it is normal that the boot is very slow?
> > >> Is the rootfs that i’m using?
> > > 
> > > Booting a full distro on the model can be slow. Usually I write my own initscript to start the necessary daemon and skip systemd.
> > > 
> > > 42sh> cat /root/init.sh
> > > 
> > > set -x
> > > mount -t proc proc  /proc
> > > mount -t devtmpfs dev /dev
> > > 
> > > export HOME=/root
> > > 
> > > rm -rf /var/run/xenstore
> > > rm -rf /var/run/xen
> > > 
> > > /etc/init.d/xencommons start
> > > 
> > > exec /bin/bash
> > > 
> > > -------------------
> > > 
> > > And I pass init=/root/init.sh on the Linux kernel command line.
> > > 
> > That helped a lot, thank you.
> > > Cheers,
> > > 
> > > -- 
> > > Julien Grall
> > 
> > Luca.
> > 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio]

[-- Attachment #1.1: Type: text/plain, Size: 3872 bytes --]

Hi Stefano,
sorry if i am so late but my schedule was full this week. I am currently preparing a presentation for an event here
in Italy and i had no time to check even my emails.

> Il giorno 29 mar 2017, alle ore 02:13, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
> On Tue, 28 Mar 2017, Stefano Stabellini wrote:
>> Hello Luca,
>> 
>> did you manage to make any progress on the number of cores exposed by the model?
>> 
I am still not using the psci and the XEN console says that he is bringing up all the 4 CPUs.
>> Thank you for the Makefile patch! If you have time, I encourage you to
>> fix one of the other small outstanding bugs, for example XEN-20 or
>> XEN-11 (just the ability to load a Dom0 kernel bigger than 128MB or just
>> the ability to parse the new image_size field introduced in Linux 3.17).
>> 
I will try to give it a look this weekend but i am not sure to have the time to
make e fix soon. 
>> Finally, I would like to remind you of the upcoming deadline for
>> applications submissions, which is the 3rd of April for GSoC, see:
>> 
>> http://marc.info/?l=xen-devel&m=149071502330534
>> 
>> The GSoC application system is
>> https://summerofcode.withgoogle.com/get-started/.
>> 
>> Please give a look at the Xen Project application template here:
>> 
>> https://wiki.xenproject.org/wiki/GSoC_Student_Application_Template
>> 
>> It also includes an "Implementation Plan", where you have the chance to
>> explain the implementation plan for the project you would like to apply
>> for.
> 
> FYI I added some more information on this project here:
> http://marc.info/?l=xen-devel&m=149074542307880 <http://marc.info/?l=xen-devel&m=149074542307880>
> 

Thank you for all the information. Unfortunately I have not considered all of my personal deadlines
before attempting to approach this project. It was my mistake. Anyway I will try with all my strength
to fill up the application this weekend and try to submit it anyway. I like the Xen project, and even if
I will not be chosen for this task, surely I will continue to support and follow it diligently.

> 
>> 
>> On Thu, 23 Mar 2017, Luca Miccio wrote:
>>> Hello Julien
>>>> Il giorno 23 mar 2017, alle ore 15:49, Julien Grall <julien.grall@arm.com> ha scritto:
>>>> 
>>>> Hello Luca,
>>>> 
>>>> On 23/03/17 02:23, Luca Miccio wrote:
>>>>> I tried also this method but i had the same error. So i came up with the
>>>>> idea of removing the
>>>>> psci method from the configure and also from the dtsi file that i have
>>>>> changed.
>>>> 
>>>> I guess you are using only 1 core in this case. Am I right?
>>>> 
>>> No, i am using 4 cores, with —cores option and i think that i am using actually 4 cores
>>> because on the model “control panel” i have 4 green triangles which ones of them should
>>> represent the number of cores and their status. 
>>> It is only my opinion so, please, tell me if i am wrong.
>>>>> 
>>>>> With this configuration the model booted up correctly, even if the
>>>>> process took about 10 minutes.
>>>>> Just for my personal knowledge: it is normal that the boot is very slow?
>>>>> Is the rootfs that i’m using?
>>>> 
>>>> Booting a full distro on the model can be slow. Usually I write my own initscript to start the necessary daemon and skip systemd.
>>>> 
>>>> 42sh> cat /root/init.sh
>>>> 
>>>> set -x
>>>> mount -t proc proc  /proc
>>>> mount -t devtmpfs dev /dev
>>>> 
>>>> export HOME=/root
>>>> 
>>>> rm -rf /var/run/xenstore
>>>> rm -rf /var/run/xen
>>>> 
>>>> /etc/init.d/xencommons start
>>>> 
>>>> exec /bin/bash
>>>> 
>>>> -------------------
>>>> 
>>>> And I pass init=/root/init.sh on the Linux kernel command line.
>>>> 
>>> That helped a lot, thank you.
>>>> Cheers,
>>>> 
>>>> -- 
>>>> Julien Grall
>>> 
>>> Luca.
Thank you,
Luca


[-- Attachment #1.2: Type: text/html, Size: 10141 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Julien Grall]

Hi Luca,

On 23/03/2017 18:03, Luca Miccio wrote:
> Hello Julien
>> Il giorno 23 mar 2017, alle ore 15:49, Julien Grall <julien.grall@arm.com> ha scritto:
>>
>> Hello Luca,
>>
>> On 23/03/17 02:23, Luca Miccio wrote:
>>> I tried also this method but i had the same error. So i came up with the
>>> idea of removing the
>>> psci method from the configure and also from the dtsi file that i have
>>> changed.
>>
>> I guess you are using only 1 core in this case. Am I right?
>>
> No, i am using 4 cores, with —cores option and i think that i am using actually 4 cores
> because on the model “control panel” i have 4 green triangles which ones of them should
> represent the number of cores and their status.
> It is only my opinion so, please, tell me if i am wrong.

Sorry by "I guess you are using only 1 core" I meant Xen will only bring 
up one care if you drop the PSCI method. You can find in the log the 
number of CPUs brought by Xen (see "Brought up N CPUs".

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 9090 bytes --]

You are still using HTML emails

On Thu, 23 Mar 2017, Luca Miccio wrote:
> 
>       Il giorno 23 mar 2017, alle ore 01:55, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
> On Thu, 23 Mar 2017, Luca Miccio wrote:
>             Il giorno 23 mar 2017, alle ore 01:27, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
>             On Thu, 23 Mar 2017, Luca Miccio wrote:
>                   Hi Stefano and Julien,
> 
>                         Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
>                         Hi Luca,
> 
>                         please don't use HTML emails.
> 
> 
>                   Sorry for that.
> 
>                         On Wed, 22 Mar 2017, Julien Grall wrote:
>                               On 22/03/2017 19:45, Luca Miccio wrote:
>                                     Hi Stefano,
> 
> 
>                               Hello Luca,
> 
>                                     aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>                                     -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>                                     -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
>                                     aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>                                     -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
>                                     -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
>                                     -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
>                                     aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>                                     -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>                                     -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
>                                     aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>                                     -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
>                                     -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
>                                     -P -C -o model.lds model.lds.S
>                                     aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
>                                     aarch64-linux-gnu-ld: section .xen LMA
>                                     [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
>                                     [0000000080080000,0000000080f009ff]
>                                     Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
>                                     make: *** [xen-system.axf] Errore 1\
> 
>                                     Clearly there is a problem with the offset of xen section but i can't
>                                     figure out how to solve it.
> 
> 
>                               I would recommend to use the latest version of bootwrapper which is now
>                               including support for Xen. I haven't yet updated the wiki with the runes
>                               but here a quick summary of the step I am using:
> 
>                               * Building bootwrapper
> 
>                               42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
>                               42sh> autoreconf -i
>                               42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
>                               --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
>                               --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
>                               --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
>                               42sh> make
> 
>                               Where:
>                               * <kernel-dir> is the path to the kernel repository
>                               * <dtb> is the path to the device-tree. I am using the one shipped with Linux
>                                in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
>                                It will be built if you did make all in Linux
>                               * <xen-bin> is the path to Xen binary (xen/xen in the repo)
> 
>                               * Command line for the foundation model
>                               42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
> 
>                               Where:
>                               * <image> is the path to xen-system.axf produced by bootwrapper
>                               * <rootfs> is the patch to your rootfs
> 
>                               Let me know if you need more details.
> 
> 
>                         Thanks Julien, I successfully booted Xen and Linux following these
>                         steps. I quickly updated the wiki.
> 
>                         Luca, I took me a while to figure out that for root=/dev/vda to work
>                         properly, Linux requires not just all the filesystem and virtio options
>                         enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
>                         you should be able to boot Xen, Linux and mount the root filesystem. If
>                         you download one of the Ubuntu arm64 cloud images, the root device will
>                         be /dev/vda1.
> 
> 
>                   First of all, thank you for your help.
> 
>                   I think that i keep on doing somenthing wrong because now the bootwrapper produces
>                   with no errors the xen-system.axf  with the configuration that Julien provided. 
>                   But when i launch the emulator it simply stops at:
>                   (XEN) Bringing up CPU1
> 
>                   To help you understand, this is what i have done:
>                   - First i compiled the xen source with the following command:
>                   $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
>                   ( I am using the staging branch)
> 
>                   - Get the linux source with git: 
>                   $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> 
>                   -Checkout on the v4.4 branch 
> 
>                   - Follow the steps described in the blog and, as you said, i set in my .config file:
>                   CONFIG_VIRTIO=y
>                   CONFIG_VIRTIO_MMIO=y
>                   CONFIG_VIRTIO_BLK=y
>                   I also checked all the filesystem and virtio options. 
> 
>                   - Compile the linux kernel with the toolchain:
>                   $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)
> 
>                   - Configured the bootwrapper with the command that Julian wrote and then launched the 
>                   foundation model with the simple command:
>                   $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4
> 
> 
>             Try removing --cores=4, does it work?
> 
>       I have already tried this option with no results
> 
> 
> The problem is that no matter the --cores=4 option, the device tree
> (foundation-v8.dtb) has 4 cpu cores by default. It also says that the
> "enable-method" is "spin-table". At the same time Julien wrote to pass
> --enable-psci to the bootwrapper, which is a different way to start
> secondary cpus. I am guessing that is the problem. I suggest to change
> boot method to psci, to do that you need to modify the dts (device tree
> source), see the attached patch for Linux. Then you can rebuild the dtb
> (device tree binary) with:
> 
> make dtbs
> 
> Finally rebuild again the bootwrapper and run the model with --cores 4.
> 
> For me, that works. Great way to get familiar with device tree ;-)
> 
> 
> I tried also this method but i had the same error. So i came up with the idea of removing the
> psci method from the configure and also from the dtsi file that i have changed. 
> 
> With this configuration the model booted up correctly, even if the process took about 10 minutes.
> Just for my personal knowledge: it is normal that the boot is very slow? Is the rootfs that i’m using?
> 
> 
> Thank you for your response,
> Luca
> 
> 
> 
>                         The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/
> 
>                         But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
>                         properly to /dev/vda1.
> 
>                         Maybe my kernel configuration is wrong?
> 
>       <1.txt>
> 
> 
> 
> 
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Julien Grall]

Hi Stefano,

On 23/03/17 00:55, Stefano Stabellini wrote:
> The problem is that no matter the --cores=4 option, the device tree
> (foundation-v8.dtb) has 4 cpu cores by default. It also says that the
> "enable-method" is "spin-table". At the same time Julien wrote to pass
> --enable-psci to the bootwrapper, which is a different way to start
> secondary cpus. I am guessing that is the problem. I suggest to change
> boot method to psci, to do that you need to modify the dts (device tree
> source), see the attached patch for Linux. Then you can rebuild the dtb
> (device tree binary) with:

I am using --enable-psci and it works for me because bootwrapper is 
re-writing the CPUs node.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[GSOC] Xen on ARM: create multiple guests from device tree

[Denis Obrezkov]

Hello all,

I would like to participate in GSoC 2018 with the project Xen on ARM
related project. I have some previous experience with GSoC:
https://summerofcode.withgoogle.com/archive/2017/projects/4780624749527040/

Could you give me more details on the project?

I have RPi3 and BBB boards, or should this work be done in emulator?

-- 
Regards, Denis Obrezkov

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [GSOC] Xen on ARM: create multiple guests from device tree

[Denis Obrezkov]

2018-02-04 9:53 GMT+03:00 Denis Obrezkov <denisobrezkov@gmail.com>:
> Hello all,
>
> I would like to participate in GSoC 2018 with the project Xen on ARM
> related project. I have some previous experience with GSoC:
> https://summerofcode.withgoogle.com/archive/2017/projects/4780624749527040/
>
> Could you give me more details on the project?
>
> I have RPi3 and BBB boards, or should this work be done in emulator?
>
> --
> Regards, Denis Obrezkov
I am also very interested in GPU virtualization on ARM devices.


-- 
Regards, Denis Obrezkov

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [GSOC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

CC'ing Edgar who is co-mentoring this project

On Sun, 4 Feb 2018, Denis Obrezkov wrote:
> Hello all,
> 
> I would like to participate in GSoC 2018 with the project Xen on ARM
> related project. I have some previous experience with GSoC:
> https://summerofcode.withgoogle.com/archive/2017/projects/4780624749527040/
> 
> Could you give me more details on the project?
> 
> I have RPi3 and BBB boards, or should this work be done in emulator?

Hello Denis,

it is great to see interest in Xen on ARM and this project!

Unfortunately RPi3 can't run Xen as far as I know due to their non-ARM
interrupt controller without virtualization support. Otherwise it would
have been a good dev board. The BeagleBoard doesn't have processors with
virtualization support so it cannot run Xen either (it needs an Cortex
A7 or A15).

But that's not a problem, because the latest QEMU (2.11) can run Xen
just fine. Build QEMU with --target-list=aarch64-softmmu, then you can
run it with:

qemu-system-aarch64 -machine virt,gic_version=3 \
    -machine virtualization=true \
    -cpu cortex-a57 -machine type=virt \
    -smp 4 -m 2048 \
    -serial stdio -monitor none \
    -bios /path/QEMU_EFI.fd \
    -netdev user,id=hostnet0 -device virtio-net-device,netdev=hostnet0 \
    -drive if=none,file=$DISK1,id=hd0 -device virtio-blk-device,drive=hd0

Where DISK1 is your EFI ready disk image and QEMU_EFI.fd can be
downloaded from:

http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_GCC5/QEMU_EFI.fd

See the following for more detailed information:

https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64

Give it a try and let me know if you have any issues.

Cheers,

Stefano

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [GSOC] Xen on ARM: create multiple guests from device tree

[Denis Obrezkov]

> Hello Denis,
>
Hello Stefano,
> it is great to see interest in Xen on ARM and this project!
>
> Unfortunately RPi3 can't run Xen as far as I know due to their non-ARM
> interrupt controller without virtualization support. Otherwise it would
> have been a good dev board. The BeagleBoard doesn't have processors with
> virtualization support so it cannot run Xen either (it needs an Cortex
> A7 or A15).
I have RPi2 and it has Cortex A7 AFAIK.
>
> But that's not a problem, because the latest QEMU (2.11) can run Xen
> just fine. Build QEMU with --target-list=aarch64-softmmu, then you can
> run it with:
>
> qemu-system-aarch64 -machine virt,gic_version=3 \
>     -machine virtualization=true \
>     -cpu cortex-a57 -machine type=virt \
>     -smp 4 -m 2048 \
>     -serial stdio -monitor none \
>     -bios /path/QEMU_EFI.fd \
>     -netdev user,id=hostnet0 -device virtio-net-device,netdev=hostnet0 \
>     -drive if=none,file=$DISK1,id=hd0 -device virtio-blk-device,drive=hd0
>
> Where DISK1 is your EFI ready disk image and QEMU_EFI.fd can be
> downloaded from:
>
> http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_GCC5/QEMU_EFI.fd
>
> See the following for more detailed information:
>
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
>
> Give it a try and let me know if you have any issues.
>
> Cheers,
>
> Stefano

I was able to boot to uefi shell. What can I do further? What is my
overall goal? To build and run several instances of Linux? Make a
patch?

I have also proposed to make a port of xen for qemu-system-riscv (it
should be ready in Q2.2018) to people from riscv but I haven't
received any answer.Anyway, I would like to work with xen on embedded
platforms.


-- 
Regards, Denis Obrezkov

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [GSOC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

On Tue, 6 Feb 2018, Denis Obrezkov wrote:
> > Hello Denis,
> >
> Hello Stefano,
> > it is great to see interest in Xen on ARM and this project!
> >
> > Unfortunately RPi3 can't run Xen as far as I know due to their non-ARM
> > interrupt controller without virtualization support. Otherwise it would
> > have been a good dev board. The BeagleBoard doesn't have processors with
> > virtualization support so it cannot run Xen either (it needs an Cortex
> > A7 or A15).
> I have RPi2 and it has Cortex A7 AFAIK.
> >
> > But that's not a problem, because the latest QEMU (2.11) can run Xen
> > just fine. Build QEMU with --target-list=aarch64-softmmu, then you can
> > run it with:
> >
> > qemu-system-aarch64 -machine virt,gic_version=3 \
> >     -machine virtualization=true \
> >     -cpu cortex-a57 -machine type=virt \
> >     -smp 4 -m 2048 \
> >     -serial stdio -monitor none \
> >     -bios /path/QEMU_EFI.fd \
> >     -netdev user,id=hostnet0 -device virtio-net-device,netdev=hostnet0 \
> >     -drive if=none,file=$DISK1,id=hd0 -device virtio-blk-device,drive=hd0
> >
> > Where DISK1 is your EFI ready disk image and QEMU_EFI.fd can be
> > downloaded from:
> >
> > http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_GCC5/QEMU_EFI.fd
> >
> > See the following for more detailed information:
> >
> > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
> >
> > Give it a try and let me know if you have any issues.
> >
> > Cheers,
> >
> > Stefano
> 
> I was able to boot to uefi shell. What can I do further? What is my
> overall goal? To build and run several instances of Linux? Make a
> patch?

Good! Please compile Xen for ARM64 and try to boot Xen and Dom0 with it.
Once you have Xen and Dom0 up and running, you can try to create a small
guest and start that as well. It will help you setup your build and test
environments.

To build Xen for ARM64, you can use native compilation inside the
qemu-system-aarch64 emulation, but it is slow. Otherwise you can use
chroot and qemu-aarch64-static (the qemu-user aarch64 target, statically
compiled). For example give a look at:

https://wiki.debian.org/Arm64Qemu


The end goal of the project is to be able to boot two domains directly
from Xen. Typically, Xen boots Dom0, then only once Dom0 is fully up and
running, a second domain can be created and that is done via the Xen
tools (xl). However, it is not always necessary to wait until Dom0 is
fully booted before starting a second guest. In many embedded
environments, you only have two or three guests in total to deal with.
It would be better to create them all in parallel directly from Xen. It
would drastically shorten the boot time.

Device tree is used to describe the hardware available on the platform.
It comes into play in this project because it is the best way to pass
the required information to Xen, so that Xen knows it needs to boot a
second guest in addition to Dom0.

Before we start the main project though, we usually ask candidates to
send a patch to fix a small issue just to show that they managed to
setup a dev and test environment correctly. We'll come back with a list
of potential small issues to fix shortly.


> I have also proposed to make a port of xen for qemu-system-riscv (it
> should be ready in Q2.2018) to people from riscv but I haven't
> received any answer.Anyway, I would like to work with xen on embedded
> platforms.

That would be awesome, but I don't think that a project like porting Xen
to riscv would fit in a GSoC project :-)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [GSOC] Xen on ARM: create multiple guests from device tree

[Denis Obrezkov]

> Good! Please compile Xen for ARM64 and try to boot Xen and Dom0 with it.
> Once you have Xen and Dom0 up and running, you can try to create a small
> guest and start that as well. It will help you setup your build and test
> environments.
>
> To build Xen for ARM64, you can use native compilation inside the
> qemu-system-aarch64 emulation, but it is slow. Otherwise you can use
> chroot and qemu-aarch64-static (the qemu-user aarch64 target, statically
> compiled). For example give a look at:
>
> https://wiki.debian.org/Arm64Qemu
>
>
> The end goal of the project is to be able to boot two domains directly
> from Xen. Typically, Xen boots Dom0, then only once Dom0 is fully up and
> running, a second domain can be created and that is done via the Xen
> tools (xl). However, it is not always necessary to wait until Dom0 is
> fully booted before starting a second guest. In many embedded
> environments, you only have two or three guests in total to deal with.
> It would be better to create them all in parallel directly from Xen. It
> would drastically shorten the boot time.
>
> Device tree is used to describe the hardware available on the platform.
> It comes into play in this project because it is the best way to pass
> the required information to Xen, so that Xen knows it needs to boot a
> second guest in addition to Dom0.
>
> Before we start the main project though, we usually ask candidates to
> send a patch to fix a small issue just to show that they managed to
> setup a dev and test environment correctly. We'll come back with a list
> of potential small issues to fix shortly.
>
>
>> I have also proposed to make a port of xen for qemu-system-riscv (it
>> should be ready in Q2.2018) to people from riscv but I haven't
>> received any answer.Anyway, I would like to work with xen on embedded
>> platforms.
>
> That would be awesome, but I don't think that a project like porting Xen
> to riscv would fit in a GSoC project :-)
Thanks, I will try to do it this weekend.

Though, I want to note, that I probably won't participate in GSoC if
Google continue their discrimination payment policy. For example, last
year I've made a port of RTEMS for RISC-V and received $3600. At the
same time one of the RTEMS project participants did almost nothing and
was expelled from the program before the final evaluation and
received...$3600. That was unfair! Could you imagine what my fellow
from India felt who made an excellent project and received only $2400.

P.S. Is xen-project going to participate in Outreachy?


-- 
Regards, Denis Obrezkov

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [GSOC] Xen on ARM: create multiple guests from device tree

[Stefano Stabellini]

On Thu, 8 Feb 2018, Denis Obrezkov wrote:
> > Good! Please compile Xen for ARM64 and try to boot Xen and Dom0 with it.
> > Once you have Xen and Dom0 up and running, you can try to create a small
> > guest and start that as well. It will help you setup your build and test
> > environments.
> >
> > To build Xen for ARM64, you can use native compilation inside the
> > qemu-system-aarch64 emulation, but it is slow. Otherwise you can use
> > chroot and qemu-aarch64-static (the qemu-user aarch64 target, statically
> > compiled). For example give a look at:
> >
> > https://wiki.debian.org/Arm64Qemu
> >
> >
> > The end goal of the project is to be able to boot two domains directly
> > from Xen. Typically, Xen boots Dom0, then only once Dom0 is fully up and
> > running, a second domain can be created and that is done via the Xen
> > tools (xl). However, it is not always necessary to wait until Dom0 is
> > fully booted before starting a second guest. In many embedded
> > environments, you only have two or three guests in total to deal with.
> > It would be better to create them all in parallel directly from Xen. It
> > would drastically shorten the boot time.
> >
> > Device tree is used to describe the hardware available on the platform.
> > It comes into play in this project because it is the best way to pass
> > the required information to Xen, so that Xen knows it needs to boot a
> > second guest in addition to Dom0.
> >
> > Before we start the main project though, we usually ask candidates to
> > send a patch to fix a small issue just to show that they managed to
> > setup a dev and test environment correctly. We'll come back with a list
> > of potential small issues to fix shortly.
> >
> >
> >> I have also proposed to make a port of xen for qemu-system-riscv (it
> >> should be ready in Q2.2018) to people from riscv but I haven't
> >> received any answer.Anyway, I would like to work with xen on embedded
> >> platforms.
> >
> > That would be awesome, but I don't think that a project like porting Xen
> > to riscv would fit in a GSoC project :-)
> Thanks, I will try to do it this weekend.
> 
> Though, I want to note, that I probably won't participate in GSoC if
> Google continue their discrimination payment policy. For example, last
> year I've made a port of RTEMS for RISC-V and received $3600. At the
> same time one of the RTEMS project participants did almost nothing and
> was expelled from the program before the final evaluation and
> received...$3600. That was unfair! Could you imagine what my fellow
> from India felt who made an excellent project and received only $2400.
> 
> P.S. Is xen-project going to participate in Outreachy?

Yes, Xen Project is participating in Outreachy, and this specific Xen on
ARM project will also be available there.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel