[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[Atish Patra]

Hi All,

I noticed following issues around U-Boot fdt location in Unleashed and
Qemu virt machine.

OpenSBI copies the FDT to following addresses for respective platforms

Qemu: 		0x82200000
Unleashed:	0x88000000


As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to
gd->fdt_blob and then gets relocated to a different address(gd-
>new_fdt) in function reloc_fdt.

As a result, FDT is present at two locations.

1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this
address.
2. gd->new_fdt: U-Boot relocated the fdt to this address.
fdtcontroladdr will also point to this address.

However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to
SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to
0x88000000 which is wrong as nobody copies fdt to above address in
Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr. 

As a result, fdtcontroladdr won't work on any platform and fdt_addr_r
will work only for Unleashed but not for Qemu. 

I am not sure what should be the ideal solution to avoid these kind of
fdt placement issues in future. Here are the few possible ones.

1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for
Qemu as well. This won't work as Qemu copies initrd to that address. I
guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot
config for Qemu accordingly.

2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use
fdt only from fdt_addr_r not fdtcontroladdr.

@david: What was the reason behind changing it for Qemu ?

3. Fix gd->new_fdt computation. This may affect every board which is
not a very good idea either.

4. Mandate loading fdt only from tftp or sdcard which is the safest
option and will avoid these kind of complications. However, I think a
default booting method without tftp server should at least work. Let me
know if that is not a sane request. In that case, we should update
documentation to clearly say that only tftpboot or sdcard loading
method works.


-- 
Regards,
Atish

[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[Anup Patel]

Hi,

I think keeping FDT placement in-sync between U-Boot and OpenSBI
across platforms is going to be painful.

I suggest that for all platforms U-Boot explicitly load FDT from somewhere
so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly
copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command

I also suspect that in-future for certain platforms FDT passed to U-Boot and
FDT passed to Linux might be little different due to U-Boot specific changes
in DTS.

Thoughts ??

Regards,
Anup

> -----Original Message-----
> From: Atish Patra
> Sent: Sunday, September 8, 2019 5:40 PM
> To: david.abdurachmanov at sifive.com; Alistair Francis
> <Alistair.Francis@wdc.com>; Anup Patel <Anup.Patel@wdc.com>
> Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
> Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V
> 
> Hi All,
> 
> I noticed following issues around U-Boot fdt location in Unleashed and Qemu
> virt machine.
> 
> OpenSBI copies the FDT to following addresses for respective platforms
> 
> Qemu: 		0x82200000
> Unleashed:	0x88000000
> 
> 
> As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to
> gd->fdt_blob and then gets relocated to a different address(gd-
> >new_fdt) in function reloc_fdt.
> 
> As a result, FDT is present at two locations.
> 
> 1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this
> address.
> 2. gd->new_fdt: U-Boot relocated the fdt to this address.
> fdtcontroladdr will also point to this address.
> 
> However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to
> SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to
> 0x88000000 which is wrong as nobody copies fdt to above address in Qemu.
> Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.
> 
> As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will
> work only for Unleashed but not for Qemu.
> 
> I am not sure what should be the ideal solution to avoid these kind of fdt
> placement issues in future. Here are the few possible ones.
> 
> 1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for
> Qemu as well. This won't work as Qemu copies initrd to that address. I guess
> best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for
> Qemu accordingly.
> 
> 2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use
> fdt only from fdt_addr_r not fdtcontroladdr.
> 
> @david: What was the reason behind changing it for Qemu ?
> 
> 3. Fix gd->new_fdt computation. This may affect every board which is not a
> very good idea either.
> 
> 4. Mandate loading fdt only from tftp or sdcard which is the safest option and
> will avoid these kind of complications. However, I think a default booting
> method without tftp server should at least work. Let me know if that is not a
> sane request. In that case, we should update documentation to clearly say
> that only tftpboot or sdcard loading method works.
> 
> 
> --
> Regards,
> Atish

[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[David Abdurachmanov]

On Sun, Sep 8, 2019 at 3:10 PM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> Hi All,
>
> I noticed following issues around U-Boot fdt location in Unleashed and
> Qemu virt machine.
>
> OpenSBI copies the FDT to following addresses for respective platforms
>
> Qemu:           0x82200000
> Unleashed:      0x88000000
>
>
> As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to
> gd->fdt_blob and then gets relocated to a different address(gd-
> >new_fdt) in function reloc_fdt.
>
> As a result, FDT is present at two locations.
>
> 1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this
> address.
> 2. gd->new_fdt: U-Boot relocated the fdt to this address.
> fdtcontroladdr will also point to this address.
>
> However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to
> SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to
> 0x88000000 which is wrong as nobody copies fdt to above address in
> Qemu.
> Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.
>
> As a result, fdtcontroladdr won't work on any platform and fdt_addr_r
> will work only for Unleashed but not for Qemu.
>
> I am not sure what should be the ideal solution to avoid these kind of
> fdt placement issues in future. Here are the few possible ones.
>
> 1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for
> Qemu as well. This won't work as Qemu copies initrd to that address. I
> guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot
> config for Qemu accordingly.
>
> 2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use
> fdt only from fdt_addr_r not fdtcontroladdr.
>
> @david: What was the reason behind changing it for Qemu ?

Not enough space for the kernel. Thus it was moved from 16M to 64M
(which was a  common thing for some ARM boards, e.g. 96Boards).

I that point I didn't know about QEMU limitation and initrd placement
(IIUC only if you use -initrd).
>
> 3. Fix gd->new_fdt computation. This may affect every board which is
> not a very good idea either.
>
> 4. Mandate loading fdt only from tftp or sdcard which is the safest
> option and will avoid these kind of complications. However, I think a
> default booting method without tftp server should at least work. Let me
> know if that is not a sane request. In that case, we should update
> documentation to clearly say that only tftpboot or sdcard loading
> method works.

I don't think we should require this. FDT should be part of the firmware
(FSBL, U-Boot SPI, OpenSBI, U-Boot, etc.). There should be a way to
override the default one from the firmware (e.g. via tfptboot, PXE or
EXTLINUX configs), but that's optional as majority people will not do
that.

david

>
>
> --
> Regards,
> Atish
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> https://lists.denx.de/listinfo/u-boot

[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[David Abdurachmanov]

On Mon, Sep 9, 2019 at 8:05 AM Anup Patel <Anup.Patel@wdc.com> wrote:
>
> Hi,
>
> I think keeping FDT placement in-sync between U-Boot and OpenSBI
> across platforms is going to be painful.
>
> I suggest that for all platforms U-Boot explicitly load FDT from somewhere
> so:
> 1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
> 2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
> 3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly
> copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command
>
> I also suspect that in-future for certain platforms FDT passed to U-Boot and
> FDT passed to Linux might be little different due to U-Boot specific changes
> in DTS.
>
> Thoughts ??

Do not forget PXE and EXTLINUX boot options. These options must
always be able to override DTB from previous stages. See below what
PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus
fdt_addr needs to be set if DTB is coming from somewhere in the firmware.
This is why we had CONFIG_PREBOOT to set it.

[..]
* Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is an fdt_addr specified, pass it along to
* bootm, and adjust argc appropriately.
*
* Scenario 3: fdt blob is not available.
[..]

david

>
> Regards,
> Anup
>
> > -----Original Message-----
> > From: Atish Patra
> > Sent: Sunday, September 8, 2019 5:40 PM
> > To: david.abdurachmanov at sifive.com; Alistair Francis
> > <Alistair.Francis@wdc.com>; Anup Patel <Anup.Patel@wdc.com>
> > Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
> > Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V
> >
> > Hi All,
> >
> > I noticed following issues around U-Boot fdt location in Unleashed and Qemu
> > virt machine.
> >
> > OpenSBI copies the FDT to following addresses for respective platforms
> >
> > Qemu:                 0x82200000
> > Unleashed:    0x88000000
> >
> >
> > As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to
> > gd->fdt_blob and then gets relocated to a different address(gd-
> > >new_fdt) in function reloc_fdt.
> >
> > As a result, FDT is present at two locations.
> >
> > 1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this
> > address.
> > 2. gd->new_fdt: U-Boot relocated the fdt to this address.
> > fdtcontroladdr will also point to this address.
> >
> > However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to
> > SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to
> > 0x88000000 which is wrong as nobody copies fdt to above address in Qemu.
> > Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.
> >
> > As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will
> > work only for Unleashed but not for Qemu.
> >
> > I am not sure what should be the ideal solution to avoid these kind of fdt
> > placement issues in future. Here are the few possible ones.
> >
> > 1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for
> > Qemu as well. This won't work as Qemu copies initrd to that address. I guess
> > best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for
> > Qemu accordingly.
> >
> > 2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use
> > fdt only from fdt_addr_r not fdtcontroladdr.
> >
> > @david: What was the reason behind changing it for Qemu ?
> >
> > 3. Fix gd->new_fdt computation. This may affect every board which is not a
> > very good idea either.
> >
> > 4. Mandate loading fdt only from tftp or sdcard which is the safest option and
> > will avoid these kind of complications. However, I think a default booting
> > method without tftp server should at least work. Let me know if that is not a
> > sane request. In that case, we should update documentation to clearly say
> > that only tftpboot or sdcard loading method works.
> >
> >
> > --
> > Regards,
> > Atish
> _______________________________________________
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[Atish Patra]

On Mon, 2019-09-09 at 13:22 +0300, David Abdurachmanov wrote:
> On Mon, Sep 9, 2019 at 8:05 AM Anup Patel <Anup.Patel@wdc.com> wrote:
> > Hi,
> > 
> > I think keeping FDT placement in-sync between U-Boot and OpenSBI
> > across platforms is going to be painful.
> > 
> > I suggest that for all platforms U-Boot explicitly load FDT from
> > somewhere
> > so:
> > 1. U-Boot ${fdt_addr_r} default value will be recommended location
> > of FDT
> > 2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed
> > by OpenSBI
> > 3. To forward FDT passed by OpenSBI to Linux, U-Boot users can
> > always explicitly
> > copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy
> > command
> > 
> > I also suspect that in-future for certain platforms FDT passed to
> > U-Boot and
> > FDT passed to Linux might be little different due to U-Boot
> > specific changes
> > in DTS.
> > 
> > Thoughts ??
> 
> Do not forget PXE and EXTLINUX boot options. These options must
> always be able to override DTB from previous stages. See below what
> PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus
> fdt_addr needs to be set if DTB is coming from somewhere in the
> firmware.
> This is why we had CONFIG_PREBOOT to set it.
> 
> [..]
> * Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
> * pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
> * and adjust argc appropriately.
> *
> * Scenario 2: If there is an fdt_addr specified, pass it along to
> * bootm, and adjust argc appropriately.
> *

How about if we do this in PREBOOT ? 

1. copy fdt from fdtcontroladdr to fdt_addr_r
2. setenv fdt_addr $fdt_addr_r

In this way, U-Boot will not have any direct dependancies on OpenSBI.
As long as U-Boot is configured with a fdt_addr_r, it should work. It
will be still valid for loading fdt from tftp server to fdt_addr_r as
well.

> * Scenario 3: fdt blob is not available.
> [..]
> 
> david
> 
> > Regards,
> > Anup
> > 
> > > -----Original Message-----
> > > From: Atish Patra
> > > Sent: Sunday, September 8, 2019 5:40 PM
> > > To: david.abdurachmanov at sifive.com; Alistair Francis
> > > <Alistair.Francis@wdc.com>; Anup Patel <Anup.Patel@wdc.com>
> > > Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
> > > Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue
> > > for RISC-V
> > > 
> > > Hi All,
> > > 
> > > I noticed following issues around U-Boot fdt location in
> > > Unleashed and Qemu
> > > virt machine.
> > > 
> > > OpenSBI copies the FDT to following addresses for respective
> > > platforms
> > > 
> > > Qemu:                 0x82200000
> > > Unleashed:    0x88000000
> > > 
> > > 
> > > As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first
> > > copied to
> > > gd->fdt_blob and then gets relocated to a different address(gd-
> > > > new_fdt) in function reloc_fdt.
> > > 
> > > As a result, FDT is present at two locations.
> > > 
> > > 1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to
> > > this
> > > address.
> > > 2. gd->new_fdt: U-Boot relocated the fdt to this address.
> > > fdtcontroladdr will also point to this address.
> > > 
> > > However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to
> > > SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points
> > > to
> > > 0x88000000 which is wrong as nobody copies fdt to above address
> > > in Qemu.
> > > Also, 5.3-rc+ kernels overwrite the address pointed by
> > > fdtcontroladdr.
> > > 
> > > As a result, fdtcontroladdr won't work on any platform and
> > > fdt_addr_r will
> > > work only for Unleashed but not for Qemu.
> > > 
> > > I am not sure what should be the ideal solution to avoid these
> > > kind of fdt
> > > placement issues in future. Here are the few possible ones.
> > > 
> > > 1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB)
> > > for
> > > Qemu as well. This won't work as Qemu copies initrd to that
> > > address. I guess
> > > best next option is to copy to 0x84000000(RAM+64MB) and U-boot
> > > config for
> > > Qemu accordingly.
> > > 
> > > 2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation
> > > to use
> > > fdt only from fdt_addr_r not fdtcontroladdr.
> > > 
> > > @david: What was the reason behind changing it for Qemu ?
> > > 
> > > 3. Fix gd->new_fdt computation. This may affect every board which
> > > is not a
> > > very good idea either.
> > > 
> > > 4. Mandate loading fdt only from tftp or sdcard which is the
> > > safest option and
> > > will avoid these kind of complications. However, I think a
> > > default booting
> > > method without tftp server should at least work. Let me know if
> > > that is not a
> > > sane request. In that case, we should update documentation to
> > > clearly say
> > > that only tftpboot or sdcard loading method works.
> > > 
> > > 
> > > --
> > > Regards,
> > > Atish
> > _______________________________________________
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi

-- 
Regards,
Atish

[U-Boot] FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

[Tom Rini]

On Tue, Sep 10, 2019 at 09:59:38AM +0000, Atish Patra wrote:
> On Mon, 2019-09-09 at 13:22 +0300, David Abdurachmanov wrote:
> > On Mon, Sep 9, 2019 at 8:05 AM Anup Patel <Anup.Patel@wdc.com> wrote:
> > > Hi,
> > > 
> > > I think keeping FDT placement in-sync between U-Boot and OpenSBI
> > > across platforms is going to be painful.
> > > 
> > > I suggest that for all platforms U-Boot explicitly load FDT from
> > > somewhere
> > > so:
> > > 1. U-Boot ${fdt_addr_r} default value will be recommended location
> > > of FDT
> > > 2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed
> > > by OpenSBI
> > > 3. To forward FDT passed by OpenSBI to Linux, U-Boot users can
> > > always explicitly
> > > copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy
> > > command
> > > 
> > > I also suspect that in-future for certain platforms FDT passed to
> > > U-Boot and
> > > FDT passed to Linux might be little different due to U-Boot
> > > specific changes
> > > in DTS.
> > > 
> > > Thoughts ??
> > 
> > Do not forget PXE and EXTLINUX boot options. These options must
> > always be able to override DTB from previous stages. See below what
> > PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus
> > fdt_addr needs to be set if DTB is coming from somewhere in the
> > firmware.
> > This is why we had CONFIG_PREBOOT to set it.
> > 
> > [..]
> > * Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
> > * pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
> > * and adjust argc appropriately.
> > *
> > * Scenario 2: If there is an fdt_addr specified, pass it along to
> > * bootm, and adjust argc appropriately.
> > *
> 
> How about if we do this in PREBOOT ? 
> 
> 1. copy fdt from fdtcontroladdr to fdt_addr_r
> 2. setenv fdt_addr $fdt_addr_r
> 
> In this way, U-Boot will not have any direct dependancies on OpenSBI.
> As long as U-Boot is configured with a fdt_addr_r, it should work. It
> will be still valid for loading fdt from tftp server to fdt_addr_r as
> well.

Since, arg, it looks like we're already seeing some of the original
mistakes from ARM land (fdt address in a bad place and getting stomped
by ...), lets take a look at how this is being handled today there and
mirror it?  We should for example pick the fdt-in-memory location such
that we have enough room between it and the maximum kernel + bss, round
up and place it there.  Then round up again for an overly large fdt to
say that's where ramdisk should go.  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190910/5bb22a12/attachment.sig>