[RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Oleksandr Tyshchenko]

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Don't use hardcoded address, retrieve it from device-tree instead.

And besides, this patch fixes the memory error when running
on top of Xen hypervisor:

(XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
      gpa=0x000000e6080000

Which shows that VCPU0 in Dom0 is trying to access an address in memory
it is not allowed to access (0x000000e6080000).
Put simply, Xen doesn't know that it is a device's register memory
since it wasn't described in a host device tree (which Xen parses)
and as the result this memory region wasn't assigned to Dom0 at
domain creation time.

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

---

This patch is meant to get feedback from the community before
proceeding further. If we decide to go this direction, all Gen2
device-trees should be updated (add memory region) before
this patch going in.

e.g. r8a7790.dtsi:

...
timer {
	compatible = "arm,armv7-timer";
	interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
			      <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
			      <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
			      <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
+	 reg = <0 0xe6080000 0 0x1000>;
};
...

---
 arch/arm/mach-shmobile/setup-rcar-gen2.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/arch/arm/mach-shmobile/setup-rcar-gen2.c b/arch/arm/mach-shmobile/setup-rcar-gen2.c
index 35dda21..153e3f5 100644
--- a/arch/arm/mach-shmobile/setup-rcar-gen2.c
+++ b/arch/arm/mach-shmobile/setup-rcar-gen2.c
@@ -15,6 +15,7 @@
 #include <linux/kernel.h>
 #include <linux/memblock.h>
 #include <linux/of.h>
+#include <linux/of_address.h>
 #include <linux/of_fdt.h>
 #include <linux/of_platform.h>
 #include <asm/mach/arch.h>
@@ -61,6 +62,8 @@ static unsigned int __init get_extal_freq(void)
 
 void __init rcar_gen2_timer_init(void)
 {
+	struct device_node *np;
+	struct resource res;
 	void __iomem *base;
 	u32 freq;
 
@@ -72,6 +75,13 @@ void __init rcar_gen2_timer_init(void)
 	if (!psci_smp_available())
 		secure_cntvoff_init();
 
+	np = of_find_compatible_node(NULL, NULL, "arm,armv7-timer");
+	if (!np)
+		goto out;
+
+	if (of_address_to_resource(np, 0, &res))
+		goto out;
+
 	if (of_machine_is_compatible("renesas,r8a7745") ||
 	    of_machine_is_compatible("renesas,r8a77470") ||
 	    of_machine_is_compatible("renesas,r8a7792") ||
@@ -88,7 +98,9 @@ void __init rcar_gen2_timer_init(void)
 	}
 
 	/* Remap "armgcnt address map" space */
-	base = ioremap(0xe6080000, PAGE_SIZE);
+	base = ioremap(res.start, resource_size(&res));
+	if (!base)
+		goto out;
 
 	/*
 	 * Update the timer if it is either not running, or is not at the
@@ -109,6 +121,9 @@ void __init rcar_gen2_timer_init(void)
 
 	iounmap(base);
 
+out:
+	of_node_put(np);
+
 	of_clk_init(NULL);
 	timer_probe();
 }
-- 
2.7.4

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Julien Grall]

Hi,

On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Don't use hardcoded address, retrieve it from device-tree instead.
> 
> And besides, this patch fixes the memory error when running
> on top of Xen hypervisor:
> 
> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
>        gpa=0x000000e6080000
> 
> Which shows that VCPU0 in Dom0 is trying to access an address in memory
> it is not allowed to access (0x000000e6080000).
> Put simply, Xen doesn't know that it is a device's register memory
> since it wasn't described in a host device tree (which Xen parses)
> and as the result this memory region wasn't assigned to Dom0 at
> domain creation time.
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> ---
> 
> This patch is meant to get feedback from the community before
> proceeding further. If we decide to go this direction, all Gen2
> device-trees should be updated (add memory region) before
> this patch going in.
> 
> e.g. r8a7790.dtsi:
> 
> ...
> timer {
> 	compatible = "arm,armv7-timer";
> 	interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> 			      <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> 			      <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> 			      <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
> +	 reg = <0 0xe6080000 0 0x1000>;

This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you 
the possibility to specify an MMIO region. This makes sense because it 
is meant to describe the Arch timer that is only access via co-processor 
registers.

Looking at the code, I think the MMIO region corresponds to the 
coresight (based on the register name). So you may want to describe the 
coresight in the Device-Tree.

Also, AFAICT, the code is configuring and turning on the timer if it has 
not been done yet. If you are here running on Xen, then you have 
probably done something wrong. Indeed, it means Xen would not be able to 
use the timer until Dom0 has booted. But, shouldn't newer U-boot do it 
for you?

Cheers,

-- 
Julien Grall

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Geert Uytterhoeven]

Hi Julien,

On Mon, May 13, 2019 at 11:20 AM Julien Grall <julien.grall@arm.com> wrote:
> On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
> > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >
> > Don't use hardcoded address, retrieve it from device-tree instead.
> >
> > And besides, this patch fixes the memory error when running
> > on top of Xen hypervisor:
> >
> > (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> >        gpa=0x000000e6080000
> >
> > Which shows that VCPU0 in Dom0 is trying to access an address in memory
> > it is not allowed to access (0x000000e6080000).
> > Put simply, Xen doesn't know that it is a device's register memory
> > since it wasn't described in a host device tree (which Xen parses)
> > and as the result this memory region wasn't assigned to Dom0 at
> > domain creation time.
> >
> > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >
> > ---
> >
> > This patch is meant to get feedback from the community before
> > proceeding further. If we decide to go this direction, all Gen2
> > device-trees should be updated (add memory region) before
> > this patch going in.
> >
> > e.g. r8a7790.dtsi:
> >
> > ...
> > timer {
> >       compatible = "arm,armv7-timer";
> >       interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> >                             <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> >                             <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> >                             <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
> > +      reg = <0 0xe6080000 0 0x1000>;
>
> This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you
> the possibility to specify an MMIO region. This makes sense because it
> is meant to describe the Arch timer that is only access via co-processor
> registers.
>
> Looking at the code, I think the MMIO region corresponds to the
> coresight (based on the register name). So you may want to describe the
> coresight in the Device-Tree.

This is the "counter module", not the ARM v7 timer, cfr. Mark Rutland's
response in an earlier discussion about describing this in DT in
"Re: Architecture Timer on R-Car Gen2"
https://lore.kernel.org/linux-renesas-soc/20170705113335.GE25115@leverpostej/

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Oleksandr]

On 13.05.19 12:19, Julien Grall wrote:
> Hi,

Hi, Julien, Geert


>
> On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> Don't use hardcoded address, retrieve it from device-tree instead.
>>
>> And besides, this patch fixes the memory error when running
>> on top of Xen hypervisor:
>>
>> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
>>        gpa=0x000000e6080000
>>
>> Which shows that VCPU0 in Dom0 is trying to access an address in memory
>> it is not allowed to access (0x000000e6080000).
>> Put simply, Xen doesn't know that it is a device's register memory
>> since it wasn't described in a host device tree (which Xen parses)
>> and as the result this memory region wasn't assigned to Dom0 at
>> domain creation time.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> ---
>>
>> This patch is meant to get feedback from the community before
>> proceeding further. If we decide to go this direction, all Gen2
>> device-trees should be updated (add memory region) before
>> this patch going in.
>>
>> e.g. r8a7790.dtsi:
>>
>> ...
>> timer {
>>     compatible = "arm,armv7-timer";
>>     interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | 
>> IRQ_TYPE_LEVEL_LOW)>,
>>                   <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | 
>> IRQ_TYPE_LEVEL_LOW)>,
>>                   <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | 
>> IRQ_TYPE_LEVEL_LOW)>,
>>                   <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | 
>> IRQ_TYPE_LEVEL_LOW)>;
>> +     reg = <0 0xe6080000 0 0x1000>;
>
> This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you 
> the possibility to specify an MMIO region. This makes sense because it 
> is meant to describe the Arch timer that is only access via 
> co-processor registers.
>
> Looking at the code, I think the MMIO region corresponds to the 
> coresight (based on the register name). So you may want to describe 
> the coresight in the Device-Tree.
>
> Also, AFAICT, the code is configuring and turning on the timer if it 
> has not been done yet. If you are here running on Xen, then you have 
> probably done something wrong. Indeed, it means Xen would not be able 
> to use the timer until Dom0 has booted. But, shouldn't newer U-boot do 
> it for you?

Let me elaborate a bit more on this...

Indeed, my PSCI patch series for U-Boot includes a patch [1] for 
configuring that "counter module". So, if PSCI is available 
(psci_smp_available() == true), then most likely we are running on 
PSCI-enabled
U-Boot which, we assume, has already taken care of configuring timer (as 
well as resetting CNTVOFF). So, when running on Xen, the timer was 
configured beforehand in U-Boot, and Xen is able to use it from the very 
beginning, these is no need to wait for Dom0 to configure it.

(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 10000 KHz

So, the code in brackets won't be called when using PSCI/running Xen, 
since the timer is already both enabled and configured:

if ((ioread32(base + CNTCR) & 1) == 0 ||
         ioread32(base + CNTFID0) != freq) {
         /* Update registers with correct frequency */
         iowrite32(freq, base + CNTFID0);
         asm volatile("mcr p15, 0, %0, c14, c0, 0" : : "r" (freq));

         /* make sure arch timer is started by setting bit 0 of CNTCR */
         iowrite32(1, base + CNTCR);
}

But, the problem here is the first read access from timer register (when 
we check whether the timer requires enabling) results in hypervisor trap:

(XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000 
gpa=0x000000e6080000

So, if the DT bindings for the counter module is not an option (if I 
correctly understood a discussion pointed by Geert in another letter), 
we should probably prevent all timer code here from being executed if 
PSCI is in use.
What I mean is to return to [2], but with the modification to use 
psci_smp_available() helper as an indicator of PSCI usage.

Julien, Geert, what do you think?


[1] https://marc.info/?l=u-boot&m=154895714510154&w=2

[2] https://lkml.org/lkml/2019/4/17/810


>
> Cheers,
>
-- 
Regards,

Oleksandr Tyshchenko

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Geert Uytterhoeven]

Hi Oleksandr,

On Mon, May 13, 2019 at 4:22 PM Oleksandr <olekstysh@gmail.com> wrote:
> On 13.05.19 12:19, Julien Grall wrote:
> > On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
> >> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >>
> >> Don't use hardcoded address, retrieve it from device-tree instead.
> >>
> >> And besides, this patch fixes the memory error when running
> >> on top of Xen hypervisor:
> >>
> >> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> >>        gpa=0x000000e6080000
> >>
> >> Which shows that VCPU0 in Dom0 is trying to access an address in memory
> >> it is not allowed to access (0x000000e6080000).
> >> Put simply, Xen doesn't know that it is a device's register memory
> >> since it wasn't described in a host device tree (which Xen parses)
> >> and as the result this memory region wasn't assigned to Dom0 at
> >> domain creation time.
> >>
> >> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >>
> >> ---
> >>
> >> This patch is meant to get feedback from the community before
> >> proceeding further. If we decide to go this direction, all Gen2
> >> device-trees should be updated (add memory region) before
> >> this patch going in.
> >>
> >> e.g. r8a7790.dtsi:
> >>
> >> ...
> >> timer {
> >>     compatible = "arm,armv7-timer";
> >>     interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>,
> >>                   <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>,
> >>                   <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>,
> >>                   <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>;
> >> +     reg = <0 0xe6080000 0 0x1000>;
> >
> > This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you
> > the possibility to specify an MMIO region. This makes sense because it
> > is meant to describe the Arch timer that is only access via
> > co-processor registers.
> >
> > Looking at the code, I think the MMIO region corresponds to the
> > coresight (based on the register name). So you may want to describe
> > the coresight in the Device-Tree.
> >
> > Also, AFAICT, the code is configuring and turning on the timer if it
> > has not been done yet. If you are here running on Xen, then you have
> > probably done something wrong. Indeed, it means Xen would not be able
> > to use the timer until Dom0 has booted. But, shouldn't newer U-boot do
> > it for you?
>
> Let me elaborate a bit more on this...
>
> Indeed, my PSCI patch series for U-Boot includes a patch [1] for
> configuring that "counter module". So, if PSCI is available
> (psci_smp_available() == true), then most likely we are running on
> PSCI-enabled
> U-Boot which, we assume, has already taken care of configuring timer (as
> well as resetting CNTVOFF). So, when running on Xen, the timer was
> configured beforehand in U-Boot, and Xen is able to use it from the very
> beginning, these is no need to wait for Dom0 to configure it.
>
> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 10000 KHz
>
> So, the code in brackets won't be called when using PSCI/running Xen,
> since the timer is already both enabled and configured:
>
> if ((ioread32(base + CNTCR) & 1) == 0 ||
>          ioread32(base + CNTFID0) != freq) {
>          /* Update registers with correct frequency */
>          iowrite32(freq, base + CNTFID0);
>          asm volatile("mcr p15, 0, %0, c14, c0, 0" : : "r" (freq));
>
>          /* make sure arch timer is started by setting bit 0 of CNTCR */
>          iowrite32(1, base + CNTCR);
> }
>
> But, the problem here is the first read access from timer register (when
> we check whether the timer requires enabling) results in hypervisor trap:
>
> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> gpa=0x000000e6080000
>
> So, if the DT bindings for the counter module is not an option (if I
> correctly understood a discussion pointed by Geert in another letter),
> we should probably prevent all timer code here from being executed if
> PSCI is in use.
> What I mean is to return to [2], but with the modification to use
> psci_smp_available() helper as an indicator of PSCI usage.
>
> Julien, Geert, what do you think?

Yes, that sounds good to me.

Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
so checking for that is not sufficient to avoid crashes when running a
uniprocessor kernel on a PSCI-enabled system.


Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Oleksandr]

On 13.05.19 18:13, Geert Uytterhoeven wrote:
> Hi Oleksandr,

Hi Geert


>> So, if the DT bindings for the counter module is not an option (if I
>> correctly understood a discussion pointed by Geert in another letter),
>> we should probably prevent all timer code here from being executed if
>> PSCI is in use.
>> What I mean is to return to [2], but with the modification to use
>> psci_smp_available() helper as an indicator of PSCI usage.
>>
>> Julien, Geert, what do you think?
> Yes, that sounds good to me.
>
> Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
> so checking for that is not sufficient to avoid crashes when running a
> uniprocessor kernel on a PSCI-enabled system.

Indeed, you are right.


Nothing than just check for psci_ops.cpu_on == NULL directly comes to 
mind...

Have already checked with CONFIG_SMP=n, it works.

Sounds ok?


-- 
Regards,

Oleksandr Tyshchenko

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Geert Uytterhoeven]

Hi Oleksandr,

On Mon, May 13, 2019 at 6:00 PM Oleksandr <olekstysh@gmail.com> wrote:
> On 13.05.19 18:13, Geert Uytterhoeven wrote:
> >> So, if the DT bindings for the counter module is not an option (if I
> >> correctly understood a discussion pointed by Geert in another letter),
> >> we should probably prevent all timer code here from being executed if
> >> PSCI is in use.
> >> What I mean is to return to [2], but with the modification to use
> >> psci_smp_available() helper as an indicator of PSCI usage.
> >>
> >> Julien, Geert, what do you think?
> > Yes, that sounds good to me.
> >
> > Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
> > so checking for that is not sufficient to avoid crashes when running a
> > uniprocessor kernel on a PSCI-enabled system.
>
> Indeed, you are right.
>
>
> Nothing than just check for psci_ops.cpu_on == NULL directly comes to
> mind...
>
> Have already checked with CONFIG_SMP=n, it works.
>
> Sounds ok?

Fine for me, thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [RFC PATCH] ARM: mach-shmobile: Parse DT to get ARCH timer memory region

[Oleksandr]

On 14.05.19 10:16, Geert Uytterhoeven wrote:
> Hi Oleksandr,

Hi Geert


>
> On Mon, May 13, 2019 at 6:00 PM Oleksandr <olekstysh@gmail.com> wrote:
>> On 13.05.19 18:13, Geert Uytterhoeven wrote:
>>>> So, if the DT bindings for the counter module is not an option (if I
>>>> correctly understood a discussion pointed by Geert in another letter),
>>>> we should probably prevent all timer code here from being executed if
>>>> PSCI is in use.
>>>> What I mean is to return to [2], but with the modification to use
>>>> psci_smp_available() helper as an indicator of PSCI usage.
>>>>
>>>> Julien, Geert, what do you think?
>>> Yes, that sounds good to me.
>>>
>>> Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
>>> so checking for that is not sufficient to avoid crashes when running a
>>> uniprocessor kernel on a PSCI-enabled system.
>> Indeed, you are right.
>>
>>
>> Nothing than just check for psci_ops.cpu_on == NULL directly comes to
>> mind...
>>
>> Have already checked with CONFIG_SMP=n, it works.
>>
>> Sounds ok?
> Fine for me, thanks!

Great, I will send new version.


>
> Gr{oetje,eeting}s,
>
>                          Geert
>
-- 
Regards,

Oleksandr Tyshchenko