Re: [GSoC] Xen on ARM: create multiple guests from device tree

[Luca Miccio <lucmiccio@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 7230 bytes --]


> Il giorno 23 mar 2017, alle ore 01:55, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
> 
> On Thu, 23 Mar 2017, Luca Miccio wrote:
>>> Il giorno 23 mar 2017, alle ore 01:27, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
>>> 
>>> On Thu, 23 Mar 2017, Luca Miccio wrote:
>>>> Hi Stefano and Julien,
>>>> 
>>>>> Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini <sstabellini@kernel.org> ha scritto:
>>>>> 
>>>>> Hi Luca,
>>>>> 
>>>>> please don't use HTML emails.
>>>>> 
>>>>> 
>>>> Sorry for that.
>>>> 
>>>>> On Wed, 22 Mar 2017, Julien Grall wrote:
>>>>>> On 22/03/2017 19:45, Luca Miccio wrote:
>>>>>>> Hi Stefano,
>>>>>> 
>>>>>> Hello Luca,
>>>>>> 
>>>>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>>>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>>>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN
>>>>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>>>>>> -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000
>>>>>>> -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image
>>>>>>> -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S
>>>>>>> aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000
>>>>>>> -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000
>>>>>>> -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S
>>>>>>> aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8
>>>>>>> -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000
>>>>>>> -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E
>>>>>>> -P -C -o model.lds model.lds.S
>>>>>>> aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds
>>>>>>> aarch64-linux-gnu-ld: section .xen LMA
>>>>>>> [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA
>>>>>>> [0000000080080000,0000000080f009ff]
>>>>>>> Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito
>>>>>>> make: *** [xen-system.axf] Errore 1\
>>>>>>> 
>>>>>>> Clearly there is a problem with the offset of xen section but i can't
>>>>>>> figure out how to solve it.
>>>>>> 
>>>>>> I would recommend to use the latest version of bootwrapper which is now
>>>>>> including support for Xen. I haven't yet updated the wiki with the runes
>>>>>> but here a quick summary of the step I am using:
>>>>>> 
>>>>>> * Building bootwrapper
>>>>>> 
>>>>>> 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git
>>>>>> 42sh> autoreconf -i
>>>>>> 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir=<kernel-dir> --with-dtb=<dtb> \
>>>>>> --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \
>>>>>> --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \
>>>>>> --with-xen=<xen-bin> --with-cpu-ids=0,1,2,3
>>>>>> 42sh> make
>>>>>> 
>>>>>> Where:
>>>>>> * <kernel-dir> is the path to the kernel repository
>>>>>> * <dtb> is the path to the device-tree. I am using the one shipped with Linux
>>>>>>  in arch/arm64/boot/dts/arm/foundation-armv8.dtbs
>>>>>>  It will be built if you did make all in Linux
>>>>>> * <xen-bin> is the path to Xen binary (xen/xen in the repo)
>>>>>> 
>>>>>> * Command line for the foundation model
>>>>>> 42sh> Foundation_Platform --image=<image> --block-device=<rootfs> --cores=4
>>>>>> 
>>>>>> Where:
>>>>>> * <image> is the path to xen-system.axf produced by bootwrapper
>>>>>> * <rootfs> is the patch to your rootfs
>>>>>> 
>>>>>> Let me know if you need more details.
>>>>> 
>>>>> Thanks Julien, I successfully booted Xen and Linux following these
>>>>> steps. I quickly updated the wiki.
>>>>> 
>>>>> Luca, I took me a while to figure out that for root=/dev/vda to work
>>>>> properly, Linux requires not just all the filesystem and virtio options
>>>>> enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that,
>>>>> you should be able to boot Xen, Linux and mount the root filesystem. If
>>>>> you download one of the Ubuntu arm64 cloud images, the root device will
>>>>> be /dev/vda1.
>>>> 
>>>> First of all, thank you for your help.
>>>> 
>>>> I think that i keep on doing somenthing wrong because now the bootwrapper produces
>>>> with no errors the xen-system.axf  with the configuration that Julien provided. 
>>>> But when i launch the emulator it simply stops at:
>>>> (XEN) Bringing up CPU1
>>>> 
>>>> To help you understand, this is what i have done:
>>>> - First i compiled the xen source with the following command:
>>>> $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel
>>>> ( I am using the staging branch)
>>>> 
>>>> - Get the linux source with git: 
>>>> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>>>> 
>>>> -Checkout on the v4.4 branch 
>>>> 
>>>> - Follow the steps described in the blog and, as you said, i set in my .config file:
>>>> CONFIG_VIRTIO=y
>>>> CONFIG_VIRTIO_MMIO=y
>>>> CONFIG_VIRTIO_BLK=y
>>>> I also checked all the filesystem and virtio options. 
>>>> 
>>>> - Compile the linux kernel with the toolchain:
>>>> $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file)
>>>> 
>>>> - Configured the bootwrapper with the command that Julian wrote and then launched the 
>>>> foundation model with the simple command:
>>>> $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4
>>> 
>>> Try removing --cores=4, does it work?
>>> 
>> I have already tried this option with no results
> 
> The problem is that no matter the --cores=4 option, the device tree
> (foundation-v8.dtb) has 4 cpu cores by default. It also says that the
> "enable-method" is "spin-table". At the same time Julien wrote to pass
> --enable-psci to the bootwrapper, which is a different way to start
> secondary cpus. I am guessing that is the problem. I suggest to change
> boot method to psci, to do that you need to modify the dts (device tree
> source), see the attached patch for Linux. Then you can rebuild the dtb
> (device tree binary) with:
> 
> make dtbs
> 
> Finally rebuild again the bootwrapper and run the model with --cores 4.
> 
> For me, that works. Great way to get familiar with device tree ;-)

I tried also this method but i had the same error. So i came up with the idea of removing the
psci method from the configure and also from the dtsi file that i have changed. 

With this configuration the model booted up correctly, even if the process took about 10 minutes.
Just for my personal knowledge: it is normal that the boot is very slow? Is the rootfs that i’m using?


Thank you for your response,
Luca


> 
>>>> The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/ <https://cloud-images.ubuntu.com/releases/16.04/release/>
>>>> 
>>>> But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device
>>>> properly to /dev/vda1.
>>>> 
>>>> Maybe my kernel configuration is wrong?
> <1.txt>




[-- Attachment #1.2: Type: text/html, Size: 20119 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel