[PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
---
Changes in v2: Drop unneeded rcar3.c
        
 docs/misc/arm/early-printk.txt    |  1 +
 xen/arch/arm/Rules.mk             |  1 +
 xen/arch/arm/arm64/debug-scif.inc | 54 +++++++++++++++++++++++++++++++++++++++
 xen/drivers/char/Kconfig          |  2 +-
 4 files changed, 57 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/arm/arm64/debug-scif.inc

diff --git a/docs/misc/arm/early-printk.txt b/docs/misc/arm/early-printk.txt
index 41b528b..20acbc7 100644
--- a/docs/misc/arm/early-printk.txt
+++ b/docs/misc/arm/early-printk.txt
@@ -41,6 +41,7 @@ the name of the machine:
   - lager: printk with SCIF0 on Renesas R-Car H2 processors
   - midway: printk with the pl011 on Calxeda Midway processors
   - omap5432: printk with UART3 on TI OMAP5432 processors
+  - salvator-x: printk with SCIF2 on Renesas R-Car H3 processors
   - seattle: printk with pl011 for AMD Seattle processor
   - sun6i: printk with 8250 on Allwinner A31 processors
   - sun7i: printk with 8250 on Allwinner A20 processors
diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
index 93304be..3003e7b 100644
--- a/xen/arch/arm/Rules.mk
+++ b/xen/arch/arm/Rules.mk
@@ -36,6 +36,7 @@ EARLY_PRINTK_juno           := pl011,0x7ff80000
 EARLY_PRINTK_lager          := scif,0xe6e60000
 EARLY_PRINTK_midway         := pl011,0xfff36000
 EARLY_PRINTK_omap5432       := 8250,0x48020000,2
+EARLY_PRINTK_salvator-x     := scif,0xe6e88000
 EARLY_PRINTK_seattle        := pl011,0xe1010000
 EARLY_PRINTK_sun6i          := 8250,0x01c28000,2
 EARLY_PRINTK_sun7i          := 8250,0x01c28000,2
diff --git a/xen/arch/arm/arm64/debug-scif.inc b/xen/arch/arm/arm64/debug-scif.inc
new file mode 100644
index 0000000..ff91c32
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-scif.inc
@@ -0,0 +1,54 @@
+/*
+ * xen/arch/arm/arm64/debug-scif.inc
+ *
+ * SCIF specific debug code
+ *
+ * Based on xen/arch/arm/arm32/debug-scif.inc by
+ * Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>
+ * Copyright (C) 2014, Globallogic.
+ *
+ * Port to AARCH64 by
+ * Dirk Behme <dirk.behme@de.bosch.com>
+ * Copyright (C) 2016, Robert Bosch Car Multimedia
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <asm/scif-uart.h>
+
+/* SCIF UART wait UART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+        ldrh   w\c, [\xb, #SCIF_SCFSR]   /* <- SCFSR (status register) */
+        tst    w\c, #SCFSR_TDFE          /* Check TDFE bit */
+        beq    1b                        /* Wait for the UART to be ready */
+.endm
+
+/* SCIF UART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit
+ */
+.macro early_uart_transmit xb, wt
+        strb   \wt, [\xb, #SCIF_SCFTDR]                  /* -> SCFTDR (data register) */
+        ldrh   \wt, [\xb, #SCIF_SCFSR]                   /* <- SCFSR (status register) */
+        and    \wt, \wt, #(~(SCFSR_TEND | SCFSR_TDFE))   /* Clear TEND and TDFE bits */
+        strh   \wt, [\xb, #SCIF_SCFSR]                   /* -> SCFSR (status register) */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 08973cf..703a311 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -45,7 +45,7 @@ config HAS_OMAP
 config HAS_SCIF
 	bool
 	default y
-	depends on ARM_32
+	depends on ARM
 	help
 	  This selects the SuperH SCI(F) UART. If you have a SuperH based board,
 	  say Y.
-- 
2.8.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Julien Grall]

Hi Dirk,

On 05/07/16 07:37, Dirk Behme wrote:
> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>

This patch looks good to me, however I would like to see the 
documentation on the wiki page (see [1]) before giving any formal ack.

Regards,

[1] 
https://lists.xenproject.org/archives/html/xen-devel/2016-07/msg00109.html

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

Hi Julien,

On 05.07.2016 13:39, Julien Grall wrote:
> Hi Dirk,
>
> On 05/07/16 07:37, Dirk Behme wrote:
>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>
> This patch looks good to me, however I would like to see the
> documentation on the wiki page (see [1]) before giving any formal ack.


Ok, many thanks for your review!

Yes, I understood that something more is needed.


I just wonder if we keep the patch on the mailing list for a moment. 
With this it's publically available and we can see how's the public 
interest for this board.

Additionally, getting Xen running on this board and describe all this in 
the wiki isn't that easy. You either need to modify the flashed 
firmware. Or, like me, load everything via a JTAG debugger ...


> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2016-07/msg00109.html


Best regards

Dirk


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Julien Grall]

On 05/07/16 13:09, Dirk Behme wrote:
> Hi Julien,
>
> On 05.07.2016 13:39, Julien Grall wrote:
>> Hi Dirk,
>>
>> On 05/07/16 07:37, Dirk Behme wrote:
>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>
>> This patch looks good to me, however I would like to see the
>> documentation on the wiki page (see [1]) before giving any formal ack.
>
>
> Ok, many thanks for your review!
>
> Yes, I understood that something more is needed.
>
>
> I just wonder if we keep the patch on the mailing list for a moment.
> With this it's publically available and we can see how's the public
> interest for this board.
>
> Additionally, getting Xen running on this board and describe all this in
> the wiki isn't that easy. You either need to modify the flashed
> firmware. Or, like me, load everything via a JTAG debugger ...

Can you details why you think it is not easy? Why do you have to modify 
the firmware? Is it because it does not boot the hypervisor in EL2?

Also, having a page at least mentioning in few bullet points the version 
of Linux/U-boot used is still useful for potential users of this board.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Stefano Stabellini]

On Tue, 5 Jul 2016, Dirk Behme wrote:
> Hi Julien,
> 
> On 05.07.2016 13:39, Julien Grall wrote:
> > Hi Dirk,
> > 
> > On 05/07/16 07:37, Dirk Behme wrote:
> > > Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
> > 
> > This patch looks good to me, however I would like to see the
> > documentation on the wiki page (see [1]) before giving any formal ack.
> 
> 
> Ok, many thanks for your review!
> 
> Yes, I understood that something more is needed.
> 
> 
> I just wonder if we keep the patch on the mailing list for a moment. With this
> it's publically available and we can see how's the public interest for this
> board.
> 
> Additionally, getting Xen running on this board and describe all this in the
> wiki isn't that easy. You either need to modify the flashed firmware. Or, like
> me, load everything via a JTAG debugger ...

That's fine. The purpose of writing down the requirements on the wiki is
to avoid misleading our users. We don't want them to think that Xen
works out of the box on Renesas RCar Gen3 H3 Salvator-X, when some
specific extra steps might be required. Even something as cumbersome as
using JTAG to load the firmware is fine, as long as it is documented.
With that in place, both Julien and I would be happy to have this patch
in the tree.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

On 05.07.2016 14:45, Julien Grall wrote:
>
>
> On 05/07/16 13:09, Dirk Behme wrote:
>> Hi Julien,
>>
>> On 05.07.2016 13:39, Julien Grall wrote:
>>> Hi Dirk,
>>>
>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>
>>> This patch looks good to me, however I would like to see the
>>> documentation on the wiki page (see [1]) before giving any formal ack.
>>
>>
>> Ok, many thanks for your review!
>>
>> Yes, I understood that something more is needed.
>>
>>
>> I just wonder if we keep the patch on the mailing list for a moment.
>> With this it's publically available and we can see how's the public
>> interest for this board.
>>
>> Additionally, getting Xen running on this board and describe all this in
>> the wiki isn't that easy. You either need to modify the flashed
>> firmware. Or, like me, load everything via a JTAG debugger ...
>
> Can you details why you think it is not easy? Why do you have to modify
> the firmware? Is it because it does not boot the hypervisor in EL2?


The board boots via ATF, which starts U-Boot in EL1, then. You have to 
find a way to load Xen into memory (e.g. U-Boot TFTP) and then switch to 
EL2 to start Xen.

To give a pointer, I would recommend

https://github.com/ARM-software/tf-issues/issues/353

Best regards

Dirk



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Julien Grall]

(CC Andre)

On 05/07/16 14:04, Dirk Behme wrote:
> On 05.07.2016 14:45, Julien Grall wrote:
>>
>>
>> On 05/07/16 13:09, Dirk Behme wrote:
>>> Hi Julien,
>>>
>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>> Hi Dirk,
>>>>
>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>
>>>> This patch looks good to me, however I would like to see the
>>>> documentation on the wiki page (see [1]) before giving any formal ack.
>>>
>>>
>>> Ok, many thanks for your review!
>>>
>>> Yes, I understood that something more is needed.
>>>
>>>
>>> I just wonder if we keep the patch on the mailing list for a moment.
>>> With this it's publically available and we can see how's the public
>>> interest for this board.
>>>
>>> Additionally, getting Xen running on this board and describe all this in
>>> the wiki isn't that easy. You either need to modify the flashed
>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>
>> Can you details why you think it is not easy? Why do you have to modify
>> the firmware? Is it because it does not boot the hypervisor in EL2?
>
>
> The board boots via ATF, which starts U-Boot in EL1, then. You have to
> find a way to load Xen into memory (e.g. U-Boot TFTP) and then switch to
> EL2 to start Xen.

But ATF is running in EL3, right? If so, can ATF just starts U-boot in EL2?

I have a board at home (pine64) which also uses ATF and U-Boot and is 
able to boot Xen without any SMC call because the U-Boot is entered in EL2.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Andre Przywara]

Hi,

On 05/07/16 14:34, Julien Grall wrote:
> (CC Andre)
> 
> On 05/07/16 14:04, Dirk Behme wrote:
>> On 05.07.2016 14:45, Julien Grall wrote:
>>>
>>>
>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>> Hi Julien,
>>>>
>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>> Hi Dirk,
>>>>>
>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>
>>>>> This patch looks good to me, however I would like to see the
>>>>> documentation on the wiki page (see [1]) before giving any formal ack.
>>>>
>>>>
>>>> Ok, many thanks for your review!
>>>>
>>>> Yes, I understood that something more is needed.
>>>>
>>>>
>>>> I just wonder if we keep the patch on the mailing list for a moment.
>>>> With this it's publically available and we can see how's the public
>>>> interest for this board.
>>>>
>>>> Additionally, getting Xen running on this board and describe all
>>>> this in
>>>> the wiki isn't that easy. You either need to modify the flashed
>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>
>>> Can you details why you think it is not easy? Why do you have to modify
>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>
>>
>> The board boots via ATF, which starts U-Boot in EL1, then. You have to
>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then switch to
>> EL2 to start Xen.
> 
> But ATF is running in EL3, right? If so, can ATF just starts U-boot in EL2?
> 
> I have a board at home (pine64) which also uses ATF and U-Boot and is
> able to boot Xen without any SMC call because the U-Boot is entered in EL2.

From having a very quick look into the rcar ATF port on github[1] I see:

+#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
+	return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX, DISABLE_ALL_EXCEPTIONS);

So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
should enter U-Boot in EL2.

The fix for the Pine64 was equally simple and works fine since then.

Cheers,
Andre.

[1]
https://github.com/renesas-rcar/meta-renesas/tree/jethro/meta-rcar-gen3/recipes-bsp/arm-trusted-firmware/arm-trusted-firmware

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

On 05.07.2016 15:45, Andre Przywara wrote:
> Hi,
>
> On 05/07/16 14:34, Julien Grall wrote:
>> (CC Andre)
>>
>> On 05/07/16 14:04, Dirk Behme wrote:
>>> On 05.07.2016 14:45, Julien Grall wrote:
>>>>
>>>>
>>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>>> Hi Julien,
>>>>>
>>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>>> Hi Dirk,
>>>>>>
>>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>>
>>>>>> This patch looks good to me, however I would like to see the
>>>>>> documentation on the wiki page (see [1]) before giving any formal ack.
>>>>>
>>>>>
>>>>> Ok, many thanks for your review!
>>>>>
>>>>> Yes, I understood that something more is needed.
>>>>>
>>>>>
>>>>> I just wonder if we keep the patch on the mailing list for a moment.
>>>>> With this it's publically available and we can see how's the public
>>>>> interest for this board.
>>>>>
>>>>> Additionally, getting Xen running on this board and describe all
>>>>> this in
>>>>> the wiki isn't that easy. You either need to modify the flashed
>>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>>
>>>> Can you details why you think it is not easy? Why do you have to modify
>>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>>
>>>
>>> The board boots via ATF, which starts U-Boot in EL1, then. You have to
>>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then switch to
>>> EL2 to start Xen.
>>
>> But ATF is running in EL3, right? If so, can ATF just starts U-boot in EL2?
>>
>> I have a board at home (pine64) which also uses ATF and U-Boot and is
>> able to boot Xen without any SMC call because the U-Boot is entered in EL2.
>
>  From having a very quick look into the rcar ATF port on github[1] I see:
>
> +#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
> +	return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX, DISABLE_ALL_EXCEPTIONS);
>
> So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
> should enter U-Boot in EL2.
>
> The fix for the Pine64 was equally simple and works fine since then.


This is an other solution most probably I meant when talking about 
"modifying the firmware" ;)

I'm somehow unsure about the pros and cons running U-Boot at EL1 
versus EL2, though.

Best regards

Dirk


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Andre Przywara]

Hi,

On 05/07/16 15:22, Dirk Behme wrote:
> On 05.07.2016 15:45, Andre Przywara wrote:
>> Hi,
>>
>> On 05/07/16 14:34, Julien Grall wrote:
>>> (CC Andre)
>>>
>>> On 05/07/16 14:04, Dirk Behme wrote:
>>>> On 05.07.2016 14:45, Julien Grall wrote:
>>>>>
>>>>>
>>>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>>>> Hi Julien,
>>>>>>
>>>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>>>> Hi Dirk,
>>>>>>>
>>>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>>>
>>>>>>> This patch looks good to me, however I would like to see the
>>>>>>> documentation on the wiki page (see [1]) before giving any formal
>>>>>>> ack.
>>>>>>
>>>>>>
>>>>>> Ok, many thanks for your review!
>>>>>>
>>>>>> Yes, I understood that something more is needed.
>>>>>>
>>>>>>
>>>>>> I just wonder if we keep the patch on the mailing list for a moment.
>>>>>> With this it's publically available and we can see how's the public
>>>>>> interest for this board.
>>>>>>
>>>>>> Additionally, getting Xen running on this board and describe all
>>>>>> this in
>>>>>> the wiki isn't that easy. You either need to modify the flashed
>>>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>>>
>>>>> Can you details why you think it is not easy? Why do you have to
>>>>> modify
>>>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>>>
>>>>
>>>> The board boots via ATF, which starts U-Boot in EL1, then. You have to
>>>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then
>>>> switch to
>>>> EL2 to start Xen.
>>>
>>> But ATF is running in EL3, right? If so, can ATF just starts U-boot
>>> in EL2?
>>>
>>> I have a board at home (pine64) which also uses ATF and U-Boot and is
>>> able to boot Xen without any SMC call because the U-Boot is entered
>>> in EL2.
>>
>>  From having a very quick look into the rcar ATF port on github[1] I see:
>>
>> +#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
>> +    return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX,
>> DISABLE_ALL_EXCEPTIONS);
>>
>> So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
>> should enter U-Boot in EL2.
>>
>> The fix for the Pine64 was equally simple and works fine since then.
> 
> 
> This is an other solution most probably I meant when talking about
> "modifying the firmware" ;)
> 
> I'm somehow unsure about the pros and cons running U-Boot at EL1 versus
> EL2, though.

It shouldn't matter, really. U-Boot on AArch64 (called armv8 here) is
able to run in any exception level (in contrast to ARMv7).
Actually running in EL2 is the architecturally recommended way, even
with ATF (by default it drops into EL2 when returning to non-secure
world). On Juno (and the Pine64) is works fine this way.

As U-Boot only uses an identity mapping, many differences between EL2
and EL1 don't matter.

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

Hi Andre,

On 05.07.2016 16:29, Andre Przywara wrote:
> Hi,
>
> On 05/07/16 15:22, Dirk Behme wrote:
>> On 05.07.2016 15:45, Andre Przywara wrote:
>>> Hi,
>>>
>>> On 05/07/16 14:34, Julien Grall wrote:
>>>> (CC Andre)
>>>>
>>>> On 05/07/16 14:04, Dirk Behme wrote:
>>>>> On 05.07.2016 14:45, Julien Grall wrote:
>>>>>>
>>>>>>
>>>>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>>>>> Hi Julien,
>>>>>>>
>>>>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>>>>> Hi Dirk,
>>>>>>>>
>>>>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>>>>
>>>>>>>> This patch looks good to me, however I would like to see the
>>>>>>>> documentation on the wiki page (see [1]) before giving any formal
>>>>>>>> ack.
>>>>>>>
>>>>>>>
>>>>>>> Ok, many thanks for your review!
>>>>>>>
>>>>>>> Yes, I understood that something more is needed.
>>>>>>>
>>>>>>>
>>>>>>> I just wonder if we keep the patch on the mailing list for a moment.
>>>>>>> With this it's publically available and we can see how's the public
>>>>>>> interest for this board.
>>>>>>>
>>>>>>> Additionally, getting Xen running on this board and describe all
>>>>>>> this in
>>>>>>> the wiki isn't that easy. You either need to modify the flashed
>>>>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>>>>
>>>>>> Can you details why you think it is not easy? Why do you have to
>>>>>> modify
>>>>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>>>>
>>>>>
>>>>> The board boots via ATF, which starts U-Boot in EL1, then. You have to
>>>>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then
>>>>> switch to
>>>>> EL2 to start Xen.
>>>>
>>>> But ATF is running in EL3, right? If so, can ATF just starts U-boot
>>>> in EL2?
>>>>
>>>> I have a board at home (pine64) which also uses ATF and U-Boot and is
>>>> able to boot Xen without any SMC call because the U-Boot is entered
>>>> in EL2.
>>>
>>>  From having a very quick look into the rcar ATF port on github[1] I see:
>>>
>>> +#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
>>> +    return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX,
>>> DISABLE_ALL_EXCEPTIONS);
>>>
>>> So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
>>> should enter U-Boot in EL2.
>>>
>>> The fix for the Pine64 was equally simple and works fine since then.
>>
>>
>> This is an other solution most probably I meant when talking about
>> "modifying the firmware" ;)
>>
>> I'm somehow unsure about the pros and cons running U-Boot at EL1 versus
>> EL2, though.
>
> It shouldn't matter, really. U-Boot on AArch64 (called armv8 here) is
> able to run in any exception level (in contrast to ARMv7).
> Actually running in EL2 is the architecturally recommended way, even
> with ATF (by default it drops into EL2 when returning to non-secure
> world). On Juno (and the Pine64) is works fine this way.


Could you share the U-Boot commands how you load and esp. start Xen? For 
loading you use TFTP? How do you start Xen with U-Boot, then? I think we 
have to pass the device tree address in x0 and the Linux kernel image 
address in x2. How do you do this with an U-Boot command?

Best regards

Dirk



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Julien Grall]

Hi Dirk,

On 06/07/16 07:33, Dirk Behme wrote:
> Could you share the U-Boot commands how you load and esp. start Xen? For
> loading you use TFTP? How do you start Xen with U-Boot, then? I think we
> have to pass the device tree address in x0 and the Linux kernel image
> address in x2. How do you do this with an U-Boot command?

U-boot can load Xen from TFTP or from the SD-card. This is the same as 
booting a baremetal kernel with U-boot.

There is a section on the wiki page to explain how to create the 
device-tree node for the boot modules [1] and the allwinner page [2] 
gives a full example how to boot Xen with U-boot via tftp (Note that it 
could easily be adapted to load from the SD-Card).

Regards,

[1] 
http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#Boot_Modules
[2] 
http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwinner#Boot_script

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

On 06.07.2016 15:17, Julien Grall wrote:
> Hi Dirk,
>
> On 06/07/16 07:33, Dirk Behme wrote:
>> Could you share the U-Boot commands how you load and esp. start Xen? For
>> loading you use TFTP? How do you start Xen with U-Boot, then? I think we
>> have to pass the device tree address in x0 and the Linux kernel image
>> address in x2. How do you do this with an U-Boot command?
>
> U-boot can load Xen from TFTP or from the SD-card. This is the same as
> booting a baremetal kernel with U-boot.
>
> There is a section on the wiki page to explain how to create the
> device-tree node for the boot modules [1] and the allwinner page [2]
> gives a full example how to boot Xen with U-boot via tftp (Note that it
> could easily be adapted to load from the SD-Card).
>
> Regards,
>
> [1]
> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#Boot_Modules
>
> [2]
> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwinner#Boot_script


Hmm, sorry, I still seem to miss anything :(

I've loaded xen/xen with U-Boot to 0x4A000000 and then try to start it 
as described in [2] above:

=> bootz 0x4A000000 - 0x48000000
Bad Linux ARM zImage magic!

I'm not sure why xen/xen is assumed to be a zImage?

To verify that the address is correct, I can use U-Boot's go command 
(which then will fail, later, as I can't pass the device tree address):

=> go 0x4A000000
## Starting application at 0x4A000000 ...
- UART enabled -
- CPU 00000000 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) No valid device tree
(XEN)
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...


Best regards

Dirk









_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Julien Grall]

On 06/07/16 15:03, Dirk Behme wrote:
> On 06.07.2016 15:17, Julien Grall wrote:
>> Hi Dirk,
>>
>> On 06/07/16 07:33, Dirk Behme wrote:
>>> Could you share the U-Boot commands how you load and esp. start Xen? For
>>> loading you use TFTP? How do you start Xen with U-Boot, then? I think we
>>> have to pass the device tree address in x0 and the Linux kernel image
>>> address in x2. How do you do this with an U-Boot command?
>>
>> U-boot can load Xen from TFTP or from the SD-card. This is the same as
>> booting a baremetal kernel with U-boot.
>>
>> There is a section on the wiki page to explain how to create the
>> device-tree node for the boot modules [1] and the allwinner page [2]
>> gives a full example how to boot Xen with U-boot via tftp (Note that it
>> could easily be adapted to load from the SD-Card).
>>
>> Regards,
>>
>> [1]
>> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#Boot_Modules
>>
>>
>> [2]
>> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwinner#Boot_script
>>
>
>
> Hmm, sorry, I still seem to miss anything :(
>
> I've loaded xen/xen with U-Boot to 0x4A000000 and then try to start it
> as described in [2] above:
>
> => bootz 0x4A000000 - 0x48000000
> Bad Linux ARM zImage magic!
>
> I'm not sure why xen/xen is assumed to be a zImage?

Oh, sorry I meant to say that bootz should be replaced by booti but 
forgot to write it down.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

Hi Julien,

On 06.07.2016 16:21, Julien Grall wrote:
>
>
> On 06/07/16 15:03, Dirk Behme wrote:
>> On 06.07.2016 15:17, Julien Grall wrote:
>>> Hi Dirk,
>>>
>>> On 06/07/16 07:33, Dirk Behme wrote:
>>>> Could you share the U-Boot commands how you load and esp. start Xen?
>>>> For
>>>> loading you use TFTP? How do you start Xen with U-Boot, then? I
>>>> think we
>>>> have to pass the device tree address in x0 and the Linux kernel image
>>>> address in x2. How do you do this with an U-Boot command?
>>>
>>> U-boot can load Xen from TFTP or from the SD-card. This is the same as
>>> booting a baremetal kernel with U-boot.
>>>
>>> There is a section on the wiki page to explain how to create the
>>> device-tree node for the boot modules [1] and the allwinner page [2]
>>> gives a full example how to boot Xen with U-boot via tftp (Note that it
>>> could easily be adapted to load from the SD-Card).
>>>
>>> Regards,
>>>
>>> [1]
>>> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#Boot_Modules
>>>
>>>
>>>
>>> [2]
>>> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwinner#Boot_script
>>>
>>>
>>
>>
>> Hmm, sorry, I still seem to miss anything :(
>>
>> I've loaded xen/xen with U-Boot to 0x4A000000 and then try to start it
>> as described in [2] above:
>>
>> => bootz 0x4A000000 - 0x48000000
>> Bad Linux ARM zImage magic!
>>
>> I'm not sure why xen/xen is assumed to be a zImage?
>
> Oh, sorry I meant to say that bootz should be replaced by booti but
> forgot to write it down.


Ok, thanks, booti does work [1].


Most probably just cosmetics, but again our friend image_size:

Image lacks image_size field, assuming 16MiB

Is there any special reason why we don't write the image size to the xen 
header?

Or should we try to fix it? E.g. like the Linux kernel is doing it?

Best regards

Dirk

[1]

=> booti 0x48000000 - 0x4A000000
Image lacks image_size field, assuming 16MiB
## Flattened Device Tree blob at 4a000000
    Booting using the fdt blob at 0x4a000000
    Using Device Tree in place at 000000004a000000, end 000000004a00c767

Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
....

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Julien Grall]

Hi Dirk,

On 07/07/16 11:39, Dirk Behme wrote:
> On 06.07.2016 16:21, Julien Grall wrote:
>> On 06/07/16 15:03, Dirk Behme wrote:
>>> On 06.07.2016 15:17, Julien Grall wrote:
>>>> Hi Dirk,
>>>>
>>>> On 06/07/16 07:33, Dirk Behme wrote:
>>>>> Could you share the U-Boot commands how you load and esp. start Xen?
>>>>> For
>>>>> loading you use TFTP? How do you start Xen with U-Boot, then? I
>>>>> think we
>>>>> have to pass the device tree address in x0 and the Linux kernel image
>>>>> address in x2. How do you do this with an U-Boot command?
>>>>
>>>> U-boot can load Xen from TFTP or from the SD-card. This is the same as
>>>> booting a baremetal kernel with U-boot.
>>>>
>>>> There is a section on the wiki page to explain how to create the
>>>> device-tree node for the boot modules [1] and the allwinner page [2]
>>>> gives a full example how to boot Xen with U-boot via tftp (Note that it
>>>> could easily be adapted to load from the SD-Card).
>>>>
>>>> Regards,
>>>>
>>>> [1]
>>>> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#Boot_Modules
>>>>
>>>>
>>>>
>>>>
>>>> [2]
>>>> http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwinner#Boot_script
>>>>
>>>>
>>>>
>>>
>>>
>>> Hmm, sorry, I still seem to miss anything :(
>>>
>>> I've loaded xen/xen with U-Boot to 0x4A000000 and then try to start it
>>> as described in [2] above:
>>>
>>> => bootz 0x4A000000 - 0x48000000
>>> Bad Linux ARM zImage magic!
>>>
>>> I'm not sure why xen/xen is assumed to be a zImage?
>>
>> Oh, sorry I meant to say that bootz should be replaced by booti but
>> forgot to write it down.
>
>
> Ok, thanks, booti does work [1].

Good to know!

> Most probably just cosmetics, but again our friend image_size:
>
> Image lacks image_size field, assuming 16MiB
>
> Is there any special reason why we don't write the image size to the xen
> header?
>
> Or should we try to fix it? E.g. like the Linux kernel is doing it?

The support of Xen ARM64 was added before the size was extended with the 
size. We did not update the header since then.

I had the same warning a couple of weeks ago and have a patch in my xen 
tree at home. I will try to send it tonight.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

Hi Andre,

On 05.07.2016 16:29, Andre Przywara wrote:
> Hi,
>
> On 05/07/16 15:22, Dirk Behme wrote:
>> On 05.07.2016 15:45, Andre Przywara wrote:
>>> Hi,
>>>
>>> On 05/07/16 14:34, Julien Grall wrote:
>>>> (CC Andre)
>>>>
>>>> On 05/07/16 14:04, Dirk Behme wrote:
>>>>> On 05.07.2016 14:45, Julien Grall wrote:
>>>>>>
>>>>>>
>>>>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>>>>> Hi Julien,
>>>>>>>
>>>>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>>>>> Hi Dirk,
>>>>>>>>
>>>>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>>>>
>>>>>>>> This patch looks good to me, however I would like to see the
>>>>>>>> documentation on the wiki page (see [1]) before giving any formal
>>>>>>>> ack.
>>>>>>>
>>>>>>>
>>>>>>> Ok, many thanks for your review!
>>>>>>>
>>>>>>> Yes, I understood that something more is needed.
>>>>>>>
>>>>>>>
>>>>>>> I just wonder if we keep the patch on the mailing list for a moment.
>>>>>>> With this it's publically available and we can see how's the public
>>>>>>> interest for this board.
>>>>>>>
>>>>>>> Additionally, getting Xen running on this board and describe all
>>>>>>> this in
>>>>>>> the wiki isn't that easy. You either need to modify the flashed
>>>>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>>>>
>>>>>> Can you details why you think it is not easy? Why do you have to
>>>>>> modify
>>>>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>>>>
>>>>>
>>>>> The board boots via ATF, which starts U-Boot in EL1, then. You have to
>>>>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then
>>>>> switch to
>>>>> EL2 to start Xen.
>>>>
>>>> But ATF is running in EL3, right? If so, can ATF just starts U-boot
>>>> in EL2?
>>>>
>>>> I have a board at home (pine64) which also uses ATF and U-Boot and is
>>>> able to boot Xen without any SMC call because the U-Boot is entered
>>>> in EL2.
>>>
>>>  From having a very quick look into the rcar ATF port on github[1] I see:
>>>
>>> +#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
>>> +    return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX,
>>> DISABLE_ALL_EXCEPTIONS);
>>>
>>> So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
>>> should enter U-Boot in EL2.
>>>
>>> The fix for the Pine64 was equally simple and works fine since then.
>>
>>
>> This is an other solution most probably I meant when talking about
>> "modifying the firmware" ;)
>>
>> I'm somehow unsure about the pros and cons running U-Boot at EL1 versus
>> EL2, though.
>
> It shouldn't matter, really. U-Boot on AArch64 (called armv8 here) is
> able to run in any exception level (in contrast to ARMv7).
> Actually running in EL2 is the architecturally recommended way, even
> with ATF (by default it drops into EL2 when returning to non-secure
> world). On Juno (and the Pine64) is works fine this way.
>
> As U-Boot only uses an identity mapping, many differences between EL2
> and EL1 don't matter.

An understanding question regarding this: We found that running U-Boot 
at EL2 works fine and starting Xen works fine, then, too. So many thanks 
for that hint!

But we found that the Linux native boot case (without Xen) does fail, 
then. I.e. running U-Boot in EL1 starts the native Linux kernel 
successfully. While running U-Boot in EL2 starting the native Linux 
kernel fails, then.

Any idea regarding this?

Who is responsible to switch to EL1 for the native Linux boot case if 
U-Boot runs at EL2?

Best regards

Dirk


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Andre Przywara]

Hi Dirk,

On 08/07/16 12:38, Dirk Behme wrote:
> Hi Andre,
> 
> On 05.07.2016 16:29, Andre Przywara wrote:
>> Hi,
>>
>> On 05/07/16 15:22, Dirk Behme wrote:
>>> On 05.07.2016 15:45, Andre Przywara wrote:
>>>> Hi,
>>>>
>>>> On 05/07/16 14:34, Julien Grall wrote:
>>>>> (CC Andre)
>>>>>
>>>>> On 05/07/16 14:04, Dirk Behme wrote:
>>>>>> On 05.07.2016 14:45, Julien Grall wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>>>>>> Hi Julien,
>>>>>>>>
>>>>>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>>>>>> Hi Dirk,
>>>>>>>>>
>>>>>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>>>>>
>>>>>>>>> This patch looks good to me, however I would like to see the
>>>>>>>>> documentation on the wiki page (see [1]) before giving any formal
>>>>>>>>> ack.
>>>>>>>>
>>>>>>>>
>>>>>>>> Ok, many thanks for your review!
>>>>>>>>
>>>>>>>> Yes, I understood that something more is needed.
>>>>>>>>
>>>>>>>>
>>>>>>>> I just wonder if we keep the patch on the mailing list for a
>>>>>>>> moment.
>>>>>>>> With this it's publically available and we can see how's the public
>>>>>>>> interest for this board.
>>>>>>>>
>>>>>>>> Additionally, getting Xen running on this board and describe all
>>>>>>>> this in
>>>>>>>> the wiki isn't that easy. You either need to modify the flashed
>>>>>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>>>>>
>>>>>>> Can you details why you think it is not easy? Why do you have to
>>>>>>> modify
>>>>>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>>>>>
>>>>>>
>>>>>> The board boots via ATF, which starts U-Boot in EL1, then. You
>>>>>> have to
>>>>>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then
>>>>>> switch to
>>>>>> EL2 to start Xen.
>>>>>
>>>>> But ATF is running in EL3, right? If so, can ATF just starts U-boot
>>>>> in EL2?
>>>>>
>>>>> I have a board at home (pine64) which also uses ATF and U-Boot and is
>>>>> able to boot Xen without any SMC call because the U-Boot is entered
>>>>> in EL2.
>>>>
>>>>  From having a very quick look into the rcar ATF port on github[1] I
>>>> see:
>>>>
>>>> +#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
>>>> +    return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX,
>>>> DISABLE_ALL_EXCEPTIONS);
>>>>
>>>> So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
>>>> should enter U-Boot in EL2.
>>>>
>>>> The fix for the Pine64 was equally simple and works fine since then.
>>>
>>>
>>> This is an other solution most probably I meant when talking about
>>> "modifying the firmware" ;)
>>>
>>> I'm somehow unsure about the pros and cons running U-Boot at EL1 versus
>>> EL2, though.
>>
>> It shouldn't matter, really. U-Boot on AArch64 (called armv8 here) is
>> able to run in any exception level (in contrast to ARMv7).
>> Actually running in EL2 is the architecturally recommended way, even
>> with ATF (by default it drops into EL2 when returning to non-secure
>> world). On Juno (and the Pine64) is works fine this way.
>>
>> As U-Boot only uses an identity mapping, many differences between EL2
>> and EL1 don't matter.
> 
> An understanding question regarding this: We found that running U-Boot
> at EL2 works fine and starting Xen works fine, then, too. So many thanks
> for that hint!
> 
> But we found that the Linux native boot case (without Xen) does fail,
> then. I.e. running U-Boot in EL1 starts the native Linux kernel
> successfully. While running U-Boot in EL2 starting the native Linux
> kernel fails, then.

That's rather odd, can you provide some debug information?

> Any idea regarding this?
> 
> Who is responsible to switch to EL1 for the native Linux boot case if
> U-Boot runs at EL2?

That's well supported kernel code in arch/arm64/kernel/head.S. If it
sees the kernel to be entered in EL2 (which is highly recommended, btw),
it installs the KVM HYP stub and drops into EL1 (after some register setup).
You might want to check the code in there to see if something springs to
mind or use some debug UART output to see where it gets stuck (if it
gets stuck). I usually do something like "mov x1, #UART_TR; mov w0,
#'1'; str w0, [x1]" to get an idea where low level code hangs.

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen: arm64: Add support for Renesas RCar Gen3 H3 Salvator-X platform

[Dirk Behme]

Hi,

On 11.07.2016 11:25, Andre Przywara wrote:
> Hi Dirk,
>
> On 08/07/16 12:38, Dirk Behme wrote:
>> Hi Andre,
>>
>> On 05.07.2016 16:29, Andre Przywara wrote:
>>> Hi,
>>>
>>> On 05/07/16 15:22, Dirk Behme wrote:
>>>> On 05.07.2016 15:45, Andre Przywara wrote:
>>>>> Hi,
>>>>>
>>>>> On 05/07/16 14:34, Julien Grall wrote:
>>>>>> (CC Andre)
>>>>>>
>>>>>> On 05/07/16 14:04, Dirk Behme wrote:
>>>>>>> On 05.07.2016 14:45, Julien Grall wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 05/07/16 13:09, Dirk Behme wrote:
>>>>>>>>> Hi Julien,
>>>>>>>>>
>>>>>>>>> On 05.07.2016 13:39, Julien Grall wrote:
>>>>>>>>>> Hi Dirk,
>>>>>>>>>>
>>>>>>>>>> On 05/07/16 07:37, Dirk Behme wrote:
>>>>>>>>>>> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
>>>>>>>>>>
>>>>>>>>>> This patch looks good to me, however I would like to see the
>>>>>>>>>> documentation on the wiki page (see [1]) before giving any formal
>>>>>>>>>> ack.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ok, many thanks for your review!
>>>>>>>>>
>>>>>>>>> Yes, I understood that something more is needed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I just wonder if we keep the patch on the mailing list for a
>>>>>>>>> moment.
>>>>>>>>> With this it's publically available and we can see how's the public
>>>>>>>>> interest for this board.
>>>>>>>>>
>>>>>>>>> Additionally, getting Xen running on this board and describe all
>>>>>>>>> this in
>>>>>>>>> the wiki isn't that easy. You either need to modify the flashed
>>>>>>>>> firmware. Or, like me, load everything via a JTAG debugger ...
>>>>>>>>
>>>>>>>> Can you details why you think it is not easy? Why do you have to
>>>>>>>> modify
>>>>>>>> the firmware? Is it because it does not boot the hypervisor in EL2?
>>>>>>>
>>>>>>>
>>>>>>> The board boots via ATF, which starts U-Boot in EL1, then. You
>>>>>>> have to
>>>>>>> find a way to load Xen into memory (e.g. U-Boot TFTP) and then
>>>>>>> switch to
>>>>>>> EL2 to start Xen.
>>>>>>
>>>>>> But ATF is running in EL3, right? If so, can ATF just starts U-boot
>>>>>> in EL2?
>>>>>>
>>>>>> I have a board at home (pine64) which also uses ATF and U-Boot and is
>>>>>> able to boot Xen without any SMC call because the U-Boot is entered
>>>>>> in EL2.
>>>>>
>>>>>  From having a very quick look into the rcar ATF port on github[1] I
>>>>> see:
>>>>>
>>>>> +#elif (RCAR_BL33_EXECUTION_EL == BL33_EL2)
>>>>> +    return (uint32_t)SPSR_64(MODE_EL2, MODE_SP_ELX,
>>>>> DISABLE_ALL_EXCEPTIONS);
>>>>>
>>>>> So if you rebuild ATF with RCAR_BL33_EXECUTION_EL set to BL33_EL2 that
>>>>> should enter U-Boot in EL2.
>>>>>
>>>>> The fix for the Pine64 was equally simple and works fine since then.
>>>>
>>>>
>>>> This is an other solution most probably I meant when talking about
>>>> "modifying the firmware" ;)
>>>>
>>>> I'm somehow unsure about the pros and cons running U-Boot at EL1 versus
>>>> EL2, though.
>>>
>>> It shouldn't matter, really. U-Boot on AArch64 (called armv8 here) is
>>> able to run in any exception level (in contrast to ARMv7).
>>> Actually running in EL2 is the architecturally recommended way, even
>>> with ATF (by default it drops into EL2 when returning to non-secure
>>> world). On Juno (and the Pine64) is works fine this way.
>>>
>>> As U-Boot only uses an identity mapping, many differences between EL2
>>> and EL1 don't matter.
>>
>> An understanding question regarding this: We found that running U-Boot
>> at EL2 works fine and starting Xen works fine, then, too. So many thanks
>> for that hint!
>>
>> But we found that the Linux native boot case (without Xen) does fail,
>> then. I.e. running U-Boot in EL1 starts the native Linux kernel
>> successfully. While running U-Boot in EL2 starting the native Linux
>> kernel fails, then.
>
> That's rather odd, can you provide some debug information?
>
>> Any idea regarding this?
>>
>> Who is responsible to switch to EL1 for the native Linux boot case if
>> U-Boot runs at EL2?
>
> That's well supported kernel code in arch/arm64/kernel/head.S. If it
> sees the kernel to be entered in EL2 (which is highly recommended, btw),
> it installs the KVM HYP stub and drops into EL1 (after some register setup).
> You might want to check the code in there to see if something springs to
> mind or use some debug UART output to see where it gets stuck (if it
> gets stuck). I usually do something like "mov x1, #UART_TR; mov w0,
> #'1'; str w0, [x1]" to get an idea where low level code hangs.


Just for the logs, it seems that the kernel I was using missed

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/arch/arm64/boot/dts/renesas?id=4c811edf65e809e6ac6ec35f4818efba2b1c6163

resulting in EL2 interrupt misbehavior.


Thanks!

Best regards

Dirk

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel