From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefano Stabellini Subject: Re: [GSoC] Xen on ARM: create multiple guests from device tree Date: Thu, 23 Mar 2017 11:07:35 -0700 (PDT) Message-ID: References: <58D17A85.5050709@gmail.com> <2A15827A-D2E8-4D7C-96F2-076A0C6AD8EC@gmail.com> <172A126F-34AD-48BC-A305-0C3F43623E76@gmail.com> <075C6339-D4FD-4518-BE67-B63D2B4960EF@gmail.com> Mime-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="8323329-1137988001-1490292323=:8001" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cr79C-0005ph-Ft for xen-devel@lists.xenproject.org; Thu, 23 Mar 2017 18:07:42 +0000 In-Reply-To: <075C6339-D4FD-4518-BE67-B63D2B4960EF@gmail.com> Content-ID: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Luca Miccio Cc: Stefano Stabellini , andre.przywara@arm.com, Dario.faggioli@citrix.com, Stefano Stabellini , Julien Grall , xen-devel@lists.xenproject.org, nd@arm.com List-Id: xen-devel@lists.xenproject.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-1137988001-1490292323=:8001 Content-Type: TEXT/PLAIN; CHARSET=UTF-8 Content-Transfer-Encoding: 8BIT Content-ID: You are still using HTML emails On Thu, 23 Mar 2017, Luca Miccio wrote: > > Il giorno 23 mar 2017, alle ore 01:55, Stefano Stabellini ha scritto: > > On Thu, 23 Mar 2017, Luca Miccio wrote: > Il giorno 23 mar 2017, alle ore 01:27, Stefano Stabellini ha scritto: > > On Thu, 23 Mar 2017, Luca Miccio wrote: > Hi Stefano and Julien, > > Il giorno 22 mar 2017, alle ore 22:38, Stefano Stabellini ha scritto: > > Hi Luca, > > please don't use HTML emails. > > > Sorry for that. > > On Wed, 22 Mar 2017, Julien Grall wrote: > On 22/03/2017 19:45, Luca Miccio wrote: > Hi Stefano, > > > Hello Luca, > > aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000 > -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000 > -DGIC_CPU_BASE=0x2c002000 -c -o boot.xen.o boot.S -DXEN > aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8 > -DBOOT=boot.xen.o -DXEN_OFFSET=0xA00000 -DKERNEL_OFFSET=0x80000 > -DFDT_OFFSET=0x08000000 -DFS_OFFSET=0x10000000 -DXEN=Xen -DKERNEL=Image > -DFILESYSTEM=filesystem.cpio.gz -E -P -C -o model.xen.lds model.lds.S > aarch64-linux-gnu-gcc  -DCNTFRQ=0x01800000-DUART_BASE=0x1c090000 > -DSYSREGS_BASE=0x1c010000 -DGIC_DIST_BASE=0x2c001000 > -DGIC_CPU_BASE=0x2c002000 -c -o boot.o boot.S > aarch64-linux-gnu-gcc  -DPHYS_OFFSET=0x80000000 -DMBOX_OFFSET=0xfff8 > -DBOOT=boot.o -DKERNEL_OFFSET=0x80000 -DFDT_OFFSET=0x08000000 > -DFS_OFFSET=0x10000000 -DKERNEL=Image -DFILESYSTEM=filesystem.cpio.gz -E > -P -C -o model.lds model.lds.S > aarch64-linux-gnu-ld -o xen-system.axf --script=model.xen.lds > aarch64-linux-gnu-ld: section .xen LMA > [0000000080a00000,0000000080ac814f] overlaps section .kernel LMA > [0000000080080000,0000000080f009ff] > Makefile:78: set di istruzioni per l'obiettivo "xen-system.axf" non riuscito > make: *** [xen-system.axf] Errore 1\ > > Clearly there is a problem with the offset of xen section but i can't > figure out how to solve it. > > > I would recommend to use the latest version of bootwrapper which is now > including support for Xen. I haven't yet updated the wiki with the runes > but here a quick summary of the step I am using: > > * Building bootwrapper > > 42sh> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git > 42sh> autoreconf -i > 42sh> ./configure --host=aarch64-linux-gnu --with-kernel-dir= --with-dtb= \ > --with-cmdline="console=hvc0 earlycon=pl011,0x1c090000 root=/dev/vda rw" --enable-psci \ > --with-xen-cmdline="dtuart=serial0 console=dtuart no-bootscrub dom0_mem=512M" \ > --with-xen= --with-cpu-ids=0,1,2,3 > 42sh> make > > Where: > * is the path to the kernel repository > * is the path to the device-tree. I am using the one shipped with Linux >  in arch/arm64/boot/dts/arm/foundation-armv8.dtbs >  It will be built if you did make all in Linux > * is the path to Xen binary (xen/xen in the repo) > > * Command line for the foundation model > 42sh> Foundation_Platform --image= --block-device= --cores=4 > > Where: > * is the path to xen-system.axf produced by bootwrapper > * is the patch to your rootfs > > Let me know if you need more details. > > > Thanks Julien, I successfully booted Xen and Linux following these > steps. I quickly updated the wiki. > > Luca, I took me a while to figure out that for root=/dev/vda to work > properly, Linux requires not just all the filesystem and virtio options > enabled in the kernel config, but also CONFIG_VIRTIO_MMIO=y. With that, > you should be able to boot Xen, Linux and mount the root filesystem. If > you download one of the Ubuntu arm64 cloud images, the root device will > be /dev/vda1. > > > First of all, thank you for your help. > > I think that i keep on doing somenthing wrong because now the bootwrapper produces > with no errors the xen-system.axf  with the configuration that Julien provided.  > But when i launch the emulator it simply stops at: > (XEN) Bringing up CPU1 > > To help you understand, this is what i have done: > - First i compiled the xen source with the following command: > $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINT=fastmodel > ( I am using the staging branch) > > - Get the linux source with git:  > $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > > -Checkout on the v4.4 branch  > > - Follow the steps described in the blog and, as you said, i set in my .config file: > CONFIG_VIRTIO=y > CONFIG_VIRTIO_MMIO=y > CONFIG_VIRTIO_BLK=y > I also checked all the filesystem and virtio options.  > > - Compile the linux kernel with the toolchain: > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- all (to get also the .dtb file) > > - Configured the bootwrapper with the command that Julian wrote and then launched the  > foundation model with the simple command: > $ Foundation_Platform --image=/path/to/the/xen-system.axf --block-device=xenial-server-cloudimg-arm64.img --cores=4 > > > Try removing --cores=4, does it work? > > I have already tried this option with no results > > > The problem is that no matter the --cores=4 option, the device tree > (foundation-v8.dtb) has 4 cpu cores by default. It also says that the > "enable-method" is "spin-table". At the same time Julien wrote to pass > --enable-psci to the bootwrapper, which is a different way to start > secondary cpus. I am guessing that is the problem. I suggest to change > boot method to psci, to do that you need to modify the dts (device tree > source), see the attached patch for Linux. Then you can rebuild the dtb > (device tree binary) with: > > make dtbs > > Finally rebuild again the bootwrapper and run the model with --cores 4. > > For me, that works. Great way to get familiar with device tree ;-) > > > I tried also this method but i had the same error. So i came up with the idea of removing the > psci method from the configure and also from the dtsi file that i have changed.  > > With this configuration the model booted up correctly, even if the process took about 10 minutes. > Just for my personal knowledge: it is normal that the boot is very slow? Is the rootfs that i’m using? > > > Thank you for your response, > Luca > > > > The image was downloaded at : https://cloud-images.ubuntu.com/releases/16.04/release/ > > But, as I said, the emulator blocks at boot. I tried different rootfs images and i have configured the root device > properly to /dev/vda1. > > Maybe my kernel configuration is wrong? > > <1.txt> > > > > > --8323329-1137988001-1490292323=:8001 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --8323329-1137988001-1490292323=:8001--