* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 2:43 ` Tamas K Lengyel
@ 2021-02-01 2:54 ` Tamas K Lengyel
2021-02-01 3:06 ` Tamas K Lengyel
2021-02-01 5:54 ` Elliott Mitchell
2 siblings, 0 replies; 19+ messages in thread
From: Tamas K Lengyel @ 2021-02-01 2:54 UTC (permalink / raw)
To: Elliott Mitchell
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Sun, Jan 31, 2021 at 9:43 PM Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
>
> On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> >
> > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > >
> > > > On Sun, Jan 31, 2021 at 02:06:17PM -0500, Tamas K Lengyel wrote:
> > > > > (XEN) Unable to retrieve address 0 for /scb/pcie@7d500000/pci@1,0/usb@1,0
> > > > > (XEN) Device tree generation failed (-22).
> > > >
> > > > > Does anyone have an idea what might be going wrong here? I tried
> > > > > building the dtb without using the dtb overlay but it didn't seem to
> > > > > do anything.
> > > >
> > > > If you go to line 1412 of the file xen/arch/arm/domain_build.c and
> > > > replace the "return res;" with "continue;" that will bypass the issue.
> > > > The 3 people I'm copying on this message though may wish to ask questions
> > > > about the state of your build tree.
> > >
> > > I'll try that but it's a pretty hacky work-around ;)
> >
> > Actually no, it simply causes Xen to ignore these entries. The patch
> > I've got ready to submit to this list also adjusts the error message to
> > avoid misinterpretation, but does pretty well exactly this.
> >
> > My only concern is whether it should ignore the entries only for Domain 0
> > or should always ignore them.
> >
> >
> > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > point to that last being touched last year. Their tree is at
> > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > >
> > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > amount of work that went into fixing Xen on RPI4, which got merged
> > > into 5.9 and I would like to be able to build upstream everything
> > > without the custom patches coming with the rpixen script repo.
> >
> > Please keep track of where your kernel source is checked out at since
> > there was a desire to figure out what was going on with the device-trees.
> >
> >
> > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > kernel command-line should ensure you get output from the kernel if it
> > manages to start (yes, Linux does support having multiple consoles at the
> > same time).
>
> No output from dom0 received even with the added console options
> (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> with 4.19 next.
The dtb overlay is giving me the following error with both 4.19 and 5.10:
arch/arm64/boot/dts/overlays/pi4-64-xen.dtbo: Warning (pci_bridge):
/fragment@1/__overlay__: node name is not "pci" or "pcie"
arch/arm64/boot/dts/overlays/pi4-64-xen.dtbo: Warning (pci_bridge):
/fragment@1/__overlay__: missing ranges for PCI bridge (or not a
bridge)
arch/arm64/boot/dts/overlays/pi4-64-xen.dtbo: Warning (pci_bridge):
/fragment@1/__overlay__: incorrect #address-cells for PCI bridge
arch/arm64/boot/dts/overlays/pi4-64-xen.dtbo: Warning (pci_bridge):
/fragment@1/__overlay__: incorrect #size-cells for PCI bridge
arch/arm64/boot/dts/overlays/pi4-64-xen.dtbo: Warning
(pci_device_bus_num): Failed prerequisite 'pci_bridge'
The overlays are defined in
https://github.com/dornerworks/xen-rpi4-builder/blob/master/patches/linux/0001-Add-Xen-overlay-for-the-Pi-4.patch
as:
+/dts-v1/;
+/plugin/;
+
+/ {
+ compatible = "brcm,bcm2711";
+
+ fragment@0 {
+ target-path = "/chosen";
+ __overlay__ {
+ #address-cells = <0x1>;
+ #size-cells = <0x1>;
+ xen,xen-bootargs = "console=dtuart dtuart=/soc/serial@7e215040
sync_console dom0_mem=512M dom0_mem=512M bootscrub=0";
+
+ dom0 {
+ compatible = "xen,linux-zimage", "xen,multiboot-module";
+ reg = <0x00400000 0x01000000>;
+ };
+ };
+ };
+
+ fragment@1 {
+ target-path = "/scb/pcie@7d500000";
+ __overlay__ {
+ device_type = "pci";
+ };
+ };
+};
+// Xen configuration for Pi 4
Don't really know what those warnings mean or how to fix them but
perhaps they are relevant to why Xen also complains?
Tamas
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 2:43 ` Tamas K Lengyel
2021-02-01 2:54 ` Tamas K Lengyel
@ 2021-02-01 3:06 ` Tamas K Lengyel
2021-02-01 18:51 ` Stefano Stabellini
2021-02-01 5:54 ` Elliott Mitchell
2 siblings, 1 reply; 19+ messages in thread
From: Tamas K Lengyel @ 2021-02-01 3:06 UTC (permalink / raw)
To: Elliott Mitchell
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Sun, Jan 31, 2021 at 9:43 PM Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
>
> On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> >
> > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > >
> > > > On Sun, Jan 31, 2021 at 02:06:17PM -0500, Tamas K Lengyel wrote:
> > > > > (XEN) Unable to retrieve address 0 for /scb/pcie@7d500000/pci@1,0/usb@1,0
> > > > > (XEN) Device tree generation failed (-22).
> > > >
> > > > > Does anyone have an idea what might be going wrong here? I tried
> > > > > building the dtb without using the dtb overlay but it didn't seem to
> > > > > do anything.
> > > >
> > > > If you go to line 1412 of the file xen/arch/arm/domain_build.c and
> > > > replace the "return res;" with "continue;" that will bypass the issue.
> > > > The 3 people I'm copying on this message though may wish to ask questions
> > > > about the state of your build tree.
> > >
> > > I'll try that but it's a pretty hacky work-around ;)
> >
> > Actually no, it simply causes Xen to ignore these entries. The patch
> > I've got ready to submit to this list also adjusts the error message to
> > avoid misinterpretation, but does pretty well exactly this.
> >
> > My only concern is whether it should ignore the entries only for Domain 0
> > or should always ignore them.
> >
> >
> > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > point to that last being touched last year. Their tree is at
> > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > >
> > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > amount of work that went into fixing Xen on RPI4, which got merged
> > > into 5.9 and I would like to be able to build upstream everything
> > > without the custom patches coming with the rpixen script repo.
> >
> > Please keep track of where your kernel source is checked out at since
> > there was a desire to figure out what was going on with the device-trees.
> >
> >
> > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > kernel command-line should ensure you get output from the kernel if it
> > manages to start (yes, Linux does support having multiple consoles at the
> > same time).
>
> No output from dom0 received even with the added console options
> (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> with 4.19 next.
With rpi-4.19.y kernel and dtbs
(cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
previous error is not present. I get the boot log on the serial with
just console=hvc0 from dom0 but the kernel ends up in a panic down the
line:
(XEN) traps.c:1983:d0v0 HSR=0x93860046 pc=0xffffff80085ac97c
gva=0xffffff800b096000 gpa=0x0000003e330000
[ 1.242863] Unhandled fault at 0xffffff800b096000
[ 1.242871] Mem abort info:
[ 1.242879] ESR = 0x96000000
[ 1.242893] Exception class = DABT (current EL), IL = 32 bits
[ 1.242922] SET = 0, FnV = 0
[ 1.242928] EA = 0, S1PTW = 0
[ 1.242934] Data abort info:
[ 1.242941] ISV = 0, ISS = 0x00000000
[ 1.242948] CM = 0, WnR = 0
[ 1.242958] swapper pgtable: 4k pages, 39-bit VAs, pgdp = (____ptrval____)
[ 1.242965] [ffffff800b096000] pgd=0000000033ffe003,
pud=0000000033ffe003, pmd=000000003230a003, pte=006800003e33070f
[ 1.242989] Internal error: ttbr address size fault: 96000000 [#1]
PREEMPT SMP
[ 1.242995] Modules linked in:
[ 1.243005] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
[ 1.243014] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.127-v8+ #1
[ 1.243019] Hardware name: Raspberry Pi 4 Model B Rev 1.1 (DT)
[ 1.243026] pstate: 20000005 (nzCv daif -PAN -UAO)
[ 1.243044] pc : cfb_imageblit+0x58c/0x820
[ 1.243054] lr : bcm2708_fb_imageblit+0x2c/0x40
[ 1.243059] sp : ffffff800802b4e0
[ 1.243063] x29: ffffff800802b4e0 x28: 00000000ffffffff
[ 1.243073] x27: 0000000000000010 x26: ffffffc03212c000
[ 1.243081] x25: 0000000000000020 x24: ffffffc0322c7d80
[ 1.243088] x23: 0000000000000008 x22: ffffffc03212a118
[ 1.243095] x21: 0000000000000000 x20: ffffff800b096000
[ 1.243102] x19: 0000000000000000 x18: 00000000fffffffc
[ 1.243109] x17: 0000000000000000 x16: ffffff800b096000
[ 1.243116] x15: 0000000000000001 x14: 0000000000001e00
[ 1.243124] x13: 0000000000000010 x12: 0000000000000000
[ 1.243131] x11: 0000000000000020 x10: 0000000000000001
[ 1.243138] x9 : 0000000000000008 x8 : ffffff800b096020
[ 1.243145] x7 : ffffffc03212c001 x6 : 0000000000000000
[ 1.243152] x5 : ffffff80089e2f78 x4 : 0000000000000000
[ 1.243159] x3 : ffffff800b096000 x2 : ffffffc03212c000
[ 1.243166] x1 : 0000000000000000 x0 : 0000000000000000
[ 1.243173] Call trace:
[ 1.243182] cfb_imageblit+0x58c/0x820
[ 1.243190] bcm2708_fb_imageblit+0x2c/0x40
[ 1.243197] soft_cursor+0x16c/0x200
[ 1.243204] bit_cursor+0x30c/0x53c
[ 1.243211] fbcon_cursor+0x13c/0x1a0
[ 1.243220] hide_cursor+0x44/0xb0
[ 1.243228] redraw_screen+0x218/0x28c
[ 1.243234] fbcon_prepare_logo+0x380/0x3ec
[ 1.243241] fbcon_init+0x364/0x550
[ 1.243248] visual_init+0xbc/0x110
[ 1.243256] do_bind_con_driver.isra.0+0x1c4/0x3a0
[ 1.243264] do_take_over_console+0x148/0x204
[ 1.243270] do_fbcon_takeover+0x7c/0xe4
[ 1.243277] fbcon_event_notify+0x6d4/0x850
[ 1.243288] blocking_notifier_call_chain+0x90/0xc0
[ 1.243297] fb_notifier_call_chain+0x34/0x40
[ 1.243303] register_framebuffer+0x21c/0x300
[ 1.243311] bcm2708_fb_probe+0x340/0x770
[ 1.243319] platform_drv_probe+0x5c/0xb0
[ 1.243325] really_probe+0x290/0x3a4
[ 1.243331] driver_probe_device+0x60/0xf4
[ 1.243337] __driver_attach+0x118/0x13c
[ 1.243347] bus_for_each_dev+0x84/0xe0
[ 1.243352] driver_attach+0x34/0x40
[ 1.243358] bus_add_driver+0x1a8/0x21c
[ 1.243364] driver_register+0x7c/0x124
[ 1.243371] __platform_driver_register+0x58/0x64
[ 1.243382] bcm2708_fb_init+0x24/0x2c
[ 1.243390] do_one_initcall+0x54/0x248
[ 1.243399] kernel_init_freeable+0x2e4/0x384
[ 1.243408] kernel_init+0x1c/0x118
[ 1.243415] ret_from_fork+0x10/0x18
[ 1.243425] Code: 8b0608a6 b94050c6 0a060026 4a0400c6 (b9000066)
[ 1.243437] ---[ end trace b74230fc2252e944 ]---
[ 1.243452] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 1.243452]
[ 1.243464] SMP: stopping secondary CPUs
[ 1.243512] Kernel Offset: disabled
[ 1.243519] CPU features: 0x0,61006000
[ 1.243523] Memory Limit: none
[ 1.590409] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b
[ 1.590409] ]---
This seems to have been caused by a monitor being attached to the HDMI
port, with HDMI unplugged dom0 boots OK.
Tamas
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 2:43 ` Tamas K Lengyel
2021-02-01 2:54 ` Tamas K Lengyel
2021-02-01 3:06 ` Tamas K Lengyel
@ 2021-02-01 5:54 ` Elliott Mitchell
2021-02-01 15:23 ` Tamas K Lengyel
2 siblings, 1 reply; 19+ messages in thread
From: Elliott Mitchell @ 2021-02-01 5:54 UTC (permalink / raw)
To: Tamas K Lengyel
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> With rpi-4.19.y kernel and dtbs
> (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> previous error is not present. I get the boot log on the serial with
> just console=hvc0 from dom0 but the kernel ends up in a panic down the
> line:
> This seems to have been caused by a monitor being attached to the HDMI
> port, with HDMI unplugged dom0 boots OK.
The balance of reports seem to suggest 5.10 is the way to go if you want
graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
RP4.
On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> >
> > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > point to that last being touched last year. Their tree is at
> > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > >
> > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > amount of work that went into fixing Xen on RPI4, which got merged
> > > into 5.9 and I would like to be able to build upstream everything
> > > without the custom patches coming with the rpixen script repo.
> >
> > Please keep track of where your kernel source is checked out at since
> > there was a desire to figure out what was going on with the device-trees.
> >
> >
> > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > kernel command-line should ensure you get output from the kernel if it
> > manages to start (yes, Linux does support having multiple consoles at the
> > same time).
>
> No output from dom0 received even with the added console options
> (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> with 4.19 next.
So, their current HEAD. This reads like you've got a problematic kernel
configuration. What procedure are you following to generate the
configuration you use?
Using their upstream as a base and then adding the configuration options
for Xen has worked fairly well for me (`make bcm2711_defconfig`,
`make menuconfig`, `make zImage`).
Notably the options:
CONFIG_PARAVIRT
CONFIG_XEN_DOM0
CONFIG_XEN
CONFIG_XEN_BLKDEV_BACKEND
CONFIG_XEN_NETDEV_BACKEND
CONFIG_HVC_XEN
CONFIG_HVC_XEN_FRONTEND
Should be set to "y".
--
(\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/)
\BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) /
\_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 5:54 ` Elliott Mitchell
@ 2021-02-01 15:23 ` Tamas K Lengyel
2021-02-01 19:33 ` Tamas K Lengyel
2021-02-01 21:24 ` Elliott Mitchell
0 siblings, 2 replies; 19+ messages in thread
From: Tamas K Lengyel @ 2021-02-01 15:23 UTC (permalink / raw)
To: Elliott Mitchell
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
>
> On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > With rpi-4.19.y kernel and dtbs
> > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > previous error is not present. I get the boot log on the serial with
> > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > line:
>
> > This seems to have been caused by a monitor being attached to the HDMI
> > port, with HDMI unplugged dom0 boots OK.
>
> The balance of reports seem to suggest 5.10 is the way to go if you want
> graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
> RP4.
>
>
> On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > >
> > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > point to that last being touched last year. Their tree is at
> > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > >
> > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > into 5.9 and I would like to be able to build upstream everything
> > > > without the custom patches coming with the rpixen script repo.
> > >
> > > Please keep track of where your kernel source is checked out at since
> > > there was a desire to figure out what was going on with the device-trees.
> > >
> > >
> > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > kernel command-line should ensure you get output from the kernel if it
> > > manages to start (yes, Linux does support having multiple consoles at the
> > > same time).
> >
> > No output from dom0 received even with the added console options
> > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > with 4.19 next.
>
> So, their current HEAD. This reads like you've got a problematic kernel
> configuration. What procedure are you following to generate the
> configuration you use?
>
> Using their upstream as a base and then adding the configuration options
> for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> `make menuconfig`, `make zImage`).
>
> Notably the options:
> CONFIG_PARAVIRT
> CONFIG_XEN_DOM0
> CONFIG_XEN
> CONFIG_XEN_BLKDEV_BACKEND
> CONFIG_XEN_NETDEV_BACKEND
> CONFIG_HVC_XEN
> CONFIG_HVC_XEN_FRONTEND
>
> Should be set to "y".
Yes, these configs are all set the same way for all Linux builds by the script:
make O=.build-arm64 ARCH=arm64
CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
dom0. So far only 4.19 boots.
Tamas
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 15:23 ` Tamas K Lengyel
@ 2021-02-01 19:33 ` Tamas K Lengyel
2021-02-02 1:40 ` Stefano Stabellini
2021-02-01 21:24 ` Elliott Mitchell
1 sibling, 1 reply; 19+ messages in thread
From: Tamas K Lengyel @ 2021-02-01 19:33 UTC (permalink / raw)
To: Elliott Mitchell
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
>
> On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> >
> > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > With rpi-4.19.y kernel and dtbs
> > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > previous error is not present. I get the boot log on the serial with
> > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > line:
> >
> > > This seems to have been caused by a monitor being attached to the HDMI
> > > port, with HDMI unplugged dom0 boots OK.
> >
> > The balance of reports seem to suggest 5.10 is the way to go if you want
> > graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
> > RP4.
> >
> >
> > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > >
> > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > > point to that last being touched last year. Their tree is at
> > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > >
> > > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > without the custom patches coming with the rpixen script repo.
> > > >
> > > > Please keep track of where your kernel source is checked out at since
> > > > there was a desire to figure out what was going on with the device-trees.
> > > >
> > > >
> > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > > kernel command-line should ensure you get output from the kernel if it
> > > > manages to start (yes, Linux does support having multiple consoles at the
> > > > same time).
> > >
> > > No output from dom0 received even with the added console options
> > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > with 4.19 next.
> >
> > So, their current HEAD. This reads like you've got a problematic kernel
> > configuration. What procedure are you following to generate the
> > configuration you use?
> >
> > Using their upstream as a base and then adding the configuration options
> > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > `make menuconfig`, `make zImage`).
> >
> > Notably the options:
> > CONFIG_PARAVIRT
> > CONFIG_XEN_DOM0
> > CONFIG_XEN
> > CONFIG_XEN_BLKDEV_BACKEND
> > CONFIG_XEN_NETDEV_BACKEND
> > CONFIG_HVC_XEN
> > CONFIG_HVC_XEN_FRONTEND
> >
> > Should be set to "y".
>
> Yes, these configs are all set the same way for all Linux builds by the script:
> make O=.build-arm64 ARCH=arm64
> CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
>
> I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> dom0. So far only 4.19 boots.
rpi-5.4.y boots but ends up in yet another different kernel panic:
(XEN) d0v1: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
(XEN) d0v2: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
(XEN) d0v3: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
[ 0.354800] Detected PIPT I-cache on CPU1
[ 0.360473] Xen: initializing cpu1
[ 0.360508] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[ 0.361674] Detected PIPT I-cache on CPU2
[ 0.367323] Xen: initializing cpu2
[ 0.367357] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[ 0.368460] Detected PIPT I-cache on CPU3
[ 0.374110] Xen: initializing cpu3
[ 0.374144] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[ 0.374357] smp: Brought up 1 node, 4 CPUs
[ 0.421250] SMP: Total of 4 processors activated.
[ 0.426051] CPU features: detected: 32-bit EL0 Support
[ 0.431344] CPU features: detected: CRC32 instructions
[ 0.459837] ------------[ cut here ]------------
[ 0.463848] CPU: CPUs started in inconsistent modes
[ 0.463925] Internal error: aarch64 BRK: f2000800 [#1] PREEMPT SMP
[ 0.475143] Modules linked in:
[ 0.478308] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.4.83-v8+ #1
[ 0.484712] Hardware name: Raspberry Pi 4 Model B Rev 1.1 (DT)
[ 0.490686] pstate: 60000005 (nZCv daif -PAN -UAO)
[ 0.495615] pc : smp_cpus_done+0x74/0x98
[ 0.499642] lr : smp_cpus_done+0x74/0x98
[ 0.503687] sp : ffffffc01002bdf0
[ 0.507102] x29: ffffffc01002bdf0 x28: 0000000000000000
[ 0.512545] x27: 0000000000000000 x26: ffffffc010f08958
[ 0.517989] x25: ffffffc010f286c8 x24: 0000000000000040
[ 0.523442] x23: ffffffc010f08000 x22: ffffffc010f28000
[ 0.528876] x21: ffffffc010f08918 x20: ffffffc010f08aa0
[ 0.534331] x19: 0000000000000100 x18: 0000000000000000
[ 0.539764] x17: 000000004d4bcabc x16: 00000000deadbeef
[ 0.545209] x15: 0000000000000030 x14: ffffffffffffffff
[ 0.550652] x13: ffffffc09002ba87 x12: ffffffc01002ba8f
[ 0.556096] x11: ffffffc01002bdf0 x10: ffffffc01002bdf0
[ 0.561540] x9 : 00000000ffffffc8 x8 : 636e69206e692064
[ 0.566984] x7 : 6574726174732073 x6 : ffffffc010f09000
[ 0.572427] x5 : ffffffc010f09098 x4 : ffffffc01002c000
[ 0.577871] x3 : 0000000000000000 x2 : 0000000000000000
[ 0.583315] x1 : 0000000000000000 x0 : ffffff8036948000
[ 0.588759] Call trace:
[ 0.591310] smp_cpus_done+0x74/0x98
[ 0.594999] smp_init+0xe4/0xfc
[ 0.598245] kernel_init_freeable+0x134/0x27c
[ 0.602726] kernel_init+0x1c/0x118
[ 0.606326] ret_from_fork+0x10/0x18
[ 0.610016] Code: 540000c0 90fff480 9125a000 97cb8545 (d4210000)
[ 0.616251] ---[ end trace 828ddf3cc765197a ]---
[ 0.620987] note: swapper/0[1] exited with preempt_count 1
[ 0.626610] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 0.634425] SMP: stopping secondary CPUs
[ 0.638511] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b ]---
Tamas
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 19:33 ` Tamas K Lengyel
@ 2021-02-02 1:40 ` Stefano Stabellini
2021-02-02 2:10 ` Roman Shaposhnik
0 siblings, 1 reply; 19+ messages in thread
From: Stefano Stabellini @ 2021-02-02 1:40 UTC (permalink / raw)
To: Tamas K Lengyel
Cc: Elliott Mitchell, Xen-devel, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk
On Mon, 1 Feb 2021, Tamas K Lengyel wrote:
> On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
> <tamas.k.lengyel@gmail.com> wrote:
> >
> > On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > >
> > > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > > With rpi-4.19.y kernel and dtbs
> > > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > > previous error is not present. I get the boot log on the serial with
> > > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > > line:
> > >
> > > > This seems to have been caused by a monitor being attached to the HDMI
> > > > port, with HDMI unplugged dom0 boots OK.
> > >
> > > The balance of reports seem to suggest 5.10 is the way to go if you want
> > > graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
> > > RP4.
> > >
> > >
> > > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > > >
> > > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > > > point to that last being touched last year. Their tree is at
> > > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > > >
> > > > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > > without the custom patches coming with the rpixen script repo.
> > > > >
> > > > > Please keep track of where your kernel source is checked out at since
> > > > > there was a desire to figure out what was going on with the device-trees.
> > > > >
> > > > >
> > > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > > > kernel command-line should ensure you get output from the kernel if it
> > > > > manages to start (yes, Linux does support having multiple consoles at the
> > > > > same time).
> > > >
> > > > No output from dom0 received even with the added console options
> > > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > > with 4.19 next.
> > >
> > > So, their current HEAD. This reads like you've got a problematic kernel
> > > configuration. What procedure are you following to generate the
> > > configuration you use?
> > >
> > > Using their upstream as a base and then adding the configuration options
> > > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > > `make menuconfig`, `make zImage`).
> > >
> > > Notably the options:
> > > CONFIG_PARAVIRT
> > > CONFIG_XEN_DOM0
> > > CONFIG_XEN
> > > CONFIG_XEN_BLKDEV_BACKEND
> > > CONFIG_XEN_NETDEV_BACKEND
> > > CONFIG_HVC_XEN
> > > CONFIG_HVC_XEN_FRONTEND
> > >
> > > Should be set to "y".
> >
> > Yes, these configs are all set the same way for all Linux builds by the script:
> > make O=.build-arm64 ARCH=arm64
> > CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
> >
> > I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> > dom0. So far only 4.19 boots.
>
> rpi-5.4.y boots but ends up in yet another different kernel panic:
That's an interesting error. However, I can tell you that I can boot
rpi-5.9.y just fine (without a monitor attached) with:
cd linux
KERNEL=kernel7l
make bcm2711_defconfig
As mentioned here:
https://www.raspberrypi.org/documentation/linux/kernel/building.md
and also taking the device tree from arch/arm64/boot/dts/broadcom/.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-02 1:40 ` Stefano Stabellini
@ 2021-02-02 2:10 ` Roman Shaposhnik
2021-02-02 2:52 ` Tamas K Lengyel
0 siblings, 1 reply; 19+ messages in thread
From: Roman Shaposhnik @ 2021-02-02 2:10 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Tamas K Lengyel, Elliott Mitchell, Xen-devel, Julien Grall,
Volodymyr Babchuk
On Mon, Feb 1, 2021 at 5:40 PM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Mon, 1 Feb 2021, Tamas K Lengyel wrote:
> > On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
> > <tamas.k.lengyel@gmail.com> wrote:
> > >
> > > On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > >
> > > > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > > > With rpi-4.19.y kernel and dtbs
> > > > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > > > previous error is not present. I get the boot log on the serial with
> > > > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > > > line:
> > > >
> > > > > This seems to have been caused by a monitor being attached to the HDMI
> > > > > port, with HDMI unplugged dom0 boots OK.
> > > >
> > > > The balance of reports seem to suggest 5.10 is the way to go if you want
> > > > graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
> > > > RP4.
> > > >
> > > >
> > > > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > > > >
> > > > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > > > > point to that last being touched last year. Their tree is at
> > > > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > > > >
> > > > > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > > > without the custom patches coming with the rpixen script repo.
> > > > > >
> > > > > > Please keep track of where your kernel source is checked out at since
> > > > > > there was a desire to figure out what was going on with the device-trees.
> > > > > >
> > > > > >
> > > > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > > > > kernel command-line should ensure you get output from the kernel if it
> > > > > > manages to start (yes, Linux does support having multiple consoles at the
> > > > > > same time).
> > > > >
> > > > > No output from dom0 received even with the added console options
> > > > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > > > with 4.19 next.
> > > >
> > > > So, their current HEAD. This reads like you've got a problematic kernel
> > > > configuration. What procedure are you following to generate the
> > > > configuration you use?
> > > >
> > > > Using their upstream as a base and then adding the configuration options
> > > > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > > > `make menuconfig`, `make zImage`).
> > > >
> > > > Notably the options:
> > > > CONFIG_PARAVIRT
> > > > CONFIG_XEN_DOM0
> > > > CONFIG_XEN
> > > > CONFIG_XEN_BLKDEV_BACKEND
> > > > CONFIG_XEN_NETDEV_BACKEND
> > > > CONFIG_HVC_XEN
> > > > CONFIG_HVC_XEN_FRONTEND
> > > >
> > > > Should be set to "y".
> > >
> > > Yes, these configs are all set the same way for all Linux builds by the script:
> > > make O=.build-arm64 ARCH=arm64
> > > CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
> > >
> > > I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> > > dom0. So far only 4.19 boots.
> >
> > rpi-5.4.y boots but ends up in yet another different kernel panic:
>
> That's an interesting error. However, I can tell you that I can boot
> rpi-5.9.y just fine (without a monitor attached) with:
>
> cd linux
> KERNEL=kernel7l
> make bcm2711_defconfig
>
> As mentioned here:
>
> https://www.raspberrypi.org/documentation/linux/kernel/building.md
>
> and also taking the device tree from arch/arm64/boot/dts/broadcom/.
FWIW: I see the same results with stock upstream 5.10.7 effectively
following the steps you're doing.
However, as I keep saying -- the combination of firmware and u-boot
(in my case) is a very sensitive combination -- hence we're relying
on a very particular set of bits for there in EVE and will refuse to work
with anything else.
It may be helpful to take that combination outside of EVE's context and
try it out in your experiments Tamas.
Thanks,
Roman.
P.S. I'm running into a DomU issue in certain places when running with 5.10.7
but that's a subject for a different thread.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-02 2:10 ` Roman Shaposhnik
@ 2021-02-02 2:52 ` Tamas K Lengyel
2021-02-02 3:00 ` Roman Shaposhnik
0 siblings, 1 reply; 19+ messages in thread
From: Tamas K Lengyel @ 2021-02-02 2:52 UTC (permalink / raw)
To: Roman Shaposhnik
Cc: Stefano Stabellini, Elliott Mitchell, Xen-devel, Julien Grall,
Volodymyr Babchuk
On Mon, Feb 1, 2021 at 9:10 PM Roman Shaposhnik <roman@zededa.com> wrote:
>
> On Mon, Feb 1, 2021 at 5:40 PM Stefano Stabellini
> <sstabellini@kernel.org> wrote:
> >
> > On Mon, 1 Feb 2021, Tamas K Lengyel wrote:
> > > On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
> > > <tamas.k.lengyel@gmail.com> wrote:
> > > >
> > > > On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > > >
> > > > > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > > > > With rpi-4.19.y kernel and dtbs
> > > > > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > > > > previous error is not present. I get the boot log on the serial with
> > > > > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > > > > line:
> > > > >
> > > > > > This seems to have been caused by a monitor being attached to the HDMI
> > > > > > port, with HDMI unplugged dom0 boots OK.
> > > > >
> > > > > The balance of reports seem to suggest 5.10 is the way to go if you want
> > > > > graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
> > > > > RP4.
> > > > >
> > > > >
> > > > > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > > > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > > > > >
> > > > > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > > > > > point to that last being touched last year. Their tree is at
> > > > > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > > > > >
> > > > > > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > > > > without the custom patches coming with the rpixen script repo.
> > > > > > >
> > > > > > > Please keep track of where your kernel source is checked out at since
> > > > > > > there was a desire to figure out what was going on with the device-trees.
> > > > > > >
> > > > > > >
> > > > > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > > > > > kernel command-line should ensure you get output from the kernel if it
> > > > > > > manages to start (yes, Linux does support having multiple consoles at the
> > > > > > > same time).
> > > > > >
> > > > > > No output from dom0 received even with the added console options
> > > > > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > > > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > > > > with 4.19 next.
> > > > >
> > > > > So, their current HEAD. This reads like you've got a problematic kernel
> > > > > configuration. What procedure are you following to generate the
> > > > > configuration you use?
> > > > >
> > > > > Using their upstream as a base and then adding the configuration options
> > > > > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > > > > `make menuconfig`, `make zImage`).
> > > > >
> > > > > Notably the options:
> > > > > CONFIG_PARAVIRT
> > > > > CONFIG_XEN_DOM0
> > > > > CONFIG_XEN
> > > > > CONFIG_XEN_BLKDEV_BACKEND
> > > > > CONFIG_XEN_NETDEV_BACKEND
> > > > > CONFIG_HVC_XEN
> > > > > CONFIG_HVC_XEN_FRONTEND
> > > > >
> > > > > Should be set to "y".
> > > >
> > > > Yes, these configs are all set the same way for all Linux builds by the script:
> > > > make O=.build-arm64 ARCH=arm64
> > > > CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
> > > >
> > > > I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> > > > dom0. So far only 4.19 boots.
> > >
> > > rpi-5.4.y boots but ends up in yet another different kernel panic:
> >
> > That's an interesting error. However, I can tell you that I can boot
> > rpi-5.9.y just fine (without a monitor attached) with:
> >
> > cd linux
> > KERNEL=kernel7l
> > make bcm2711_defconfig
> >
> > As mentioned here:
> >
> > https://www.raspberrypi.org/documentation/linux/kernel/building.md
> >
> > and also taking the device tree from arch/arm64/boot/dts/broadcom/.
>
> FWIW: I see the same results with stock upstream 5.10.7 effectively
> following the steps you're doing.
>
> However, as I keep saying -- the combination of firmware and u-boot
> (in my case) is a very sensitive combination -- hence we're relying
> on a very particular set of bits for there in EVE and will refuse to work
> with anything else.
>
> It may be helpful to take that combination outside of EVE's context and
> try it out in your experiments Tamas.
Well, I'm giving up on this for now. I ran out of ideas to try and I
don't see any useful suggestions on how to debug this further. Looks
like it's super fragile and works only under specific conditions
that's not well documented - if documented at all.
Tamas
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-02 2:52 ` Tamas K Lengyel
@ 2021-02-02 3:00 ` Roman Shaposhnik
0 siblings, 0 replies; 19+ messages in thread
From: Roman Shaposhnik @ 2021-02-02 3:00 UTC (permalink / raw)
To: Tamas K Lengyel
Cc: Stefano Stabellini, Elliott Mitchell, Xen-devel, Julien Grall,
Volodymyr Babchuk
On Mon, Feb 1, 2021 at 6:53 PM Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
>
> On Mon, Feb 1, 2021 at 9:10 PM Roman Shaposhnik <roman@zededa.com> wrote:
> >
> > On Mon, Feb 1, 2021 at 5:40 PM Stefano Stabellini
> > <sstabellini@kernel.org> wrote:
> > >
> > > On Mon, 1 Feb 2021, Tamas K Lengyel wrote:
> > > > On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
> > > > <tamas.k.lengyel@gmail.com> wrote:
> > > > >
> > > > > On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > > > >
> > > > > > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > > > > > With rpi-4.19.y kernel and dtbs
> > > > > > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > > > > > previous error is not present. I get the boot log on the serial with
> > > > > > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > > > > > line:
> > > > > >
> > > > > > > This seems to have been caused by a monitor being attached to the HDMI
> > > > > > > port, with HDMI unplugged dom0 boots OK.
> > > > > >
> > > > > > The balance of reports seem to suggest 5.10 is the way to go if you want
> > > > > > graphics on a RP4 with Xen. Even without Xen 4.19 is looking rickety on
> > > > > > RP4.
> > > > > >
> > > > > >
> > > > > > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > > > > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > > > > > >
> > > > > > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@m5p.com> wrote:
> > > > > > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > > > > > > point to that last being touched last year. Their tree is at
> > > > > > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > > > > > >
> > > > > > > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > > > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > > > > > without the custom patches coming with the rpixen script repo.
> > > > > > > >
> > > > > > > > Please keep track of where your kernel source is checked out at since
> > > > > > > > there was a desire to figure out what was going on with the device-trees.
> > > > > > > >
> > > > > > > >
> > > > > > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > > > > > > kernel command-line should ensure you get output from the kernel if it
> > > > > > > > manages to start (yes, Linux does support having multiple consoles at the
> > > > > > > > same time).
> > > > > > >
> > > > > > > No output from dom0 received even with the added console options
> > > > > > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > > > > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > > > > > with 4.19 next.
> > > > > >
> > > > > > So, their current HEAD. This reads like you've got a problematic kernel
> > > > > > configuration. What procedure are you following to generate the
> > > > > > configuration you use?
> > > > > >
> > > > > > Using their upstream as a base and then adding the configuration options
> > > > > > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > > > > > `make menuconfig`, `make zImage`).
> > > > > >
> > > > > > Notably the options:
> > > > > > CONFIG_PARAVIRT
> > > > > > CONFIG_XEN_DOM0
> > > > > > CONFIG_XEN
> > > > > > CONFIG_XEN_BLKDEV_BACKEND
> > > > > > CONFIG_XEN_NETDEV_BACKEND
> > > > > > CONFIG_HVC_XEN
> > > > > > CONFIG_HVC_XEN_FRONTEND
> > > > > >
> > > > > > Should be set to "y".
> > > > >
> > > > > Yes, these configs are all set the same way for all Linux builds by the script:
> > > > > make O=.build-arm64 ARCH=arm64
> > > > > CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
> > > > >
> > > > > I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> > > > > dom0. So far only 4.19 boots.
> > > >
> > > > rpi-5.4.y boots but ends up in yet another different kernel panic:
> > >
> > > That's an interesting error. However, I can tell you that I can boot
> > > rpi-5.9.y just fine (without a monitor attached) with:
> > >
> > > cd linux
> > > KERNEL=kernel7l
> > > make bcm2711_defconfig
> > >
> > > As mentioned here:
> > >
> > > https://www.raspberrypi.org/documentation/linux/kernel/building.md
> > >
> > > and also taking the device tree from arch/arm64/boot/dts/broadcom/.
> >
> > FWIW: I see the same results with stock upstream 5.10.7 effectively
> > following the steps you're doing.
> >
> > However, as I keep saying -- the combination of firmware and u-boot
> > (in my case) is a very sensitive combination -- hence we're relying
> > on a very particular set of bits for there in EVE and will refuse to work
> > with anything else.
> >
> > It may be helpful to take that combination outside of EVE's context and
> > try it out in your experiments Tamas.
>
> Well, I'm giving up on this for now. I ran out of ideas to try and I
> don't see any useful suggestions on how to debug this further. Looks
> like it's super fragile and works only under specific conditions
> that's not well documented - if documented at all.
That's fair -- at the same time I honestly don't see how any other approach
but documenting that a BOM of versions is known to work together can be
really practical.
I mean -- at the end of the day that's why no user (well no sane user ;-)) takes
kernel from kernel.org directly -- most of them come through Linux distro BOM.
Thanks,
Roman.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 15:23 ` Tamas K Lengyel
2021-02-01 19:33 ` Tamas K Lengyel
@ 2021-02-01 21:24 ` Elliott Mitchell
2021-02-01 22:13 ` Tamas K Lengyel
1 sibling, 1 reply; 19+ messages in thread
From: Elliott Mitchell @ 2021-02-01 21:24 UTC (permalink / raw)
To: Tamas K Lengyel
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Mon, Feb 01, 2021 at 10:23:34AM -0500, Tamas K Lengyel wrote:
> On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > No output from dom0 received even with the added console options
> > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > with 4.19 next.
> >
> > So, their current HEAD. This reads like you've got a problematic kernel
> > configuration. What procedure are you following to generate the
> > configuration you use?
> >
> > Using their upstream as a base and then adding the configuration options
> > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > `make menuconfig`, `make zImage`).
> >
> > Notably the options:
> > CONFIG_PARAVIRT
> > CONFIG_XEN_DOM0
> > CONFIG_XEN
> > CONFIG_XEN_BLKDEV_BACKEND
> > CONFIG_XEN_NETDEV_BACKEND
> > CONFIG_HVC_XEN
> > CONFIG_HVC_XEN_FRONTEND
> >
> > Should be set to "y".
>
> Yes, these configs are all set the same way for all Linux builds by the script:
> make O=.build-arm64 ARCH=arm64
> CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
>
> I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> dom0. So far only 4.19 boots.
So you're using a scripted procedure to generate the configuration. The
actual kernel configuration is saved in the file ".config" in the build
directory. Could you confirm whether those are actually being set?
Try running `grep -eCONFIG_PARAVIRT -eCONFIG_XEN_DOM0 -eCONFIG_XEN
-eCONFIG_HVC_XEN -eCONFIG_HVC_XEN_FRONTEND .config`, those 5 must
be "=y". Various kernel configuration options depend upon others, so
there could be potential you need to set one before those get enabled.
--
(\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/)
\BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) /
\_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Xen 4.14.1 on RPI4: device tree generation failed
2021-02-01 21:24 ` Elliott Mitchell
@ 2021-02-01 22:13 ` Tamas K Lengyel
0 siblings, 0 replies; 19+ messages in thread
From: Tamas K Lengyel @ 2021-02-01 22:13 UTC (permalink / raw)
To: Elliott Mitchell
Cc: Xen-devel, Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On Mon, Feb 1, 2021 at 4:24 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
>
> On Mon, Feb 01, 2021 at 10:23:34AM -0500, Tamas K Lengyel wrote:
> > On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@m5p.com> wrote:
> > > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > > No output from dom0 received even with the added console options
> > > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > > with 4.19 next.
> > >
> > > So, their current HEAD. This reads like you've got a problematic kernel
> > > configuration. What procedure are you following to generate the
> > > configuration you use?
> > >
> > > Using their upstream as a base and then adding the configuration options
> > > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > > `make menuconfig`, `make zImage`).
> > >
> > > Notably the options:
> > > CONFIG_PARAVIRT
> > > CONFIG_XEN_DOM0
> > > CONFIG_XEN
> > > CONFIG_XEN_BLKDEV_BACKEND
> > > CONFIG_XEN_NETDEV_BACKEND
> > > CONFIG_HVC_XEN
> > > CONFIG_HVC_XEN_FRONTEND
> > >
> > > Should be set to "y".
> >
> > Yes, these configs are all set the same way for all Linux builds by the script:
> > make O=.build-arm64 ARCH=arm64
> > CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
> >
> > I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> > dom0. So far only 4.19 boots.
>
> So you're using a scripted procedure to generate the configuration. The
> actual kernel configuration is saved in the file ".config" in the build
> directory. Could you confirm whether those are actually being set?
>
> Try running `grep -eCONFIG_PARAVIRT -eCONFIG_XEN_DOM0 -eCONFIG_XEN
> -eCONFIG_HVC_XEN -eCONFIG_HVC_XEN_FRONTEND .config`, those 5 must
> be "=y". Various kernel configuration options depend upon others, so
> there could be potential you need to set one before those get enabled.
These options are all set, it was one of the first things I did was to
confirm in the actual config file. There is no output from 5.9 or
5.10. With 4.19 and 5.4 there is, but only 4.19 actually manages to
boot to a workable state.
Tamas
^ permalink raw reply [flat|nested] 19+ messages in thread