incompatible device trees between u-boot and linux

incompatible device trees between u-boot and linux

[Michael Walle]

Hi,

I noticed that there is a fallback to the u-boot device tree for linux
(esp. EFI boot) if no other device tree was found, see [1]. It seems 
this
is working fine for imx devices, for example, where you can just boot a
stock installer iso via EFI. It will just work and it is quite a nice
feature as a fallback.

Now for the layerscape architecture, the ls1028a in my case, things are
more difficult because the bindings differ between u-boot and linux - 
one
which comes to mind is DSA and ethernet.

Which begs the general question, is it encouraged to have both bindings
diverge? To me it seems, that most bindings in u-boot are ad-hoc and 
there
is no real review or alignment but just added as needed, which is ok if
they are local to u-boot. But since they are nowadays passed to linux
(by default!) I'm not so sure anymore.

OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
they should (could?) be shared between linux and u-boot.

-michael

[1] 
https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471

Re: incompatible device trees between u-boot and linux

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]

On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:

> Hi,
> 
> I noticed that there is a fallback to the u-boot device tree for linux
> (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> is working fine for imx devices, for example, where you can just boot a
> stock installer iso via EFI. It will just work and it is quite a nice
> feature as a fallback.
> 
> Now for the layerscape architecture, the ls1028a in my case, things are
> more difficult because the bindings differ between u-boot and linux - one
> which comes to mind is DSA and ethernet.
> 
> Which begs the general question, is it encouraged to have both bindings
> diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> is no real review or alignment but just added as needed, which is ok if
> they are local to u-boot. But since they are nowadays passed to linux
> (by default!) I'm not so sure anymore.
> 
> OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> they should (could?) be shared between linux and u-boot.
> 
> -michael
> 
> [1]
> https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471

The U-Boot device tree is supposed to be able to be passed on to Linux
and Just Work.  The bindings are not supposed to be different between
the two (except for when we take the binding while it's being hashed out
upstream BUT THEN RESYNCED).  Incompatible device trees / bindings are a
bug that needs to be fixed.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: incompatible device trees between u-boot and linux

[Vladimir Oltean]

On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
>
> > Hi,
> >
> > I noticed that there is a fallback to the u-boot device tree for linux
> > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > is working fine for imx devices, for example, where you can just boot a
> > stock installer iso via EFI. It will just work and it is quite a nice
> > feature as a fallback.
> >
> > Now for the layerscape architecture, the ls1028a in my case, things are
> > more difficult because the bindings differ between u-boot and linux - one
> > which comes to mind is DSA and ethernet.
> >
> > Which begs the general question, is it encouraged to have both bindings
> > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > is no real review or alignment but just added as needed, which is ok if
> > they are local to u-boot. But since they are nowadays passed to linux
> > (by default!) I'm not so sure anymore.
> >
> > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > they should (could?) be shared between linux and u-boot.
> >
> > -michael
> >
> > [1]
> > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
>
> The U-Boot device tree is supposed to be able to be passed on to Linux
> and Just Work.  The bindings are not supposed to be different between
> the two (except for when we take the binding while it's being hashed out
> upstream BUT THEN RESYNCED).

You might need to spell that out a bit clearer.

You are saying that both U-Boot and Linux are allowed to have their own
custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
for Linux), as long as the device tree files themselves are in sync, and
the subset of the device tree blob understood by Linux (i.e. the U-Boot
blob sans the U-Boot specifics) is compatible with the Linux DT blob?

To expand even further on that, it means we should put 'managed = "in-band-status"'
in U-Boot, which is a Linux phylink device tree property, even if U-Boot
does not use phylink?

> Incompatible device trees / bindings are a
> bug that needs to be fixed.

Curious that Michael mentions Ethernet and DSA on LS1028A.

I decompiled the two:

dtc -I dtb -O dts < arch/arm/dts/fsl-ls1028a-rdb.dtb > u-boot.dts
dtc -I dtb -O dts < arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dtb > linux.dts

and analyzed the Ethernet portion.

U-Boot looks like this:

/dts-v1/;

/ {
	compatible = "fsl,ls1028a-rdb", "fsl,ls1028a";
	interrupt-parent = <0x1>;
	#address-cells = <0x2>;
	#size-cells = <0x2>;
	model = "NXP Layerscape 1028a RDB Board";

	pcie@1f0000000 {
		compatible = "pci-host-ecam-generic";
		bus-range = <0x0 0x0>;
		reg = <0x1 0xf0000000 0x0 0x100000>;
		#address-cells = <0x3>;
		#size-cells = <0x2>;
		device_type = "pci";
		ranges = <0x82000000 0x0 0x0 0x1 0xf8000000 0x0 0x160000>;

		pci@0,0 {
			reg = <0x0 0x0 0x0 0x0 0x0>;
			status = "okay";
			phy-mode = "sgmii";
			phy-handle = <0x4>;
			phandle = <0x11>;
		};

		pci@0,1 {
			reg = <0x100 0x0 0x0 0x0 0x0>;
			status = "disabled";
			phandle = <0x12>;
		};

		pci@0,2 {
			reg = <0x200 0x0 0x0 0x0 0x0>;
			status = "okay";
			phy-mode = "internal";
			phandle = <0x9>;

			fixed-link {
				speed = <0x9c4>;
				full-duplex;
			};
		};

		pci@0,3 {
			#address-cells = <0x0>;
			#size-cells = <0x1>;
			reg = <0x300 0x0 0x0 0x0 0x0>;
			status = "okay";
			phandle = <0x13>;

			fixed-link {
				speed = <0x3e8>;
				full-duplex;
			};

			phy@2 {
				reg = <0x2>;
				phandle = <0x4>;
			};

			phy@10 {
				reg = <0x10>;
				phandle = <0x5>;
			};

			phy@11 {
				reg = <0x11>;
				phandle = <0x6>;
			};

			phy@12 {
				reg = <0x12>;
				phandle = <0x7>;
			};

			phy@13 {
				reg = <0x13>;
				phandle = <0x8>;
			};
		};

		pci@0,5 {
			reg = <0x500 0x0 0x0 0x0 0x0>;
			status = "okay";
			phandle = <0x14>;

			ports {
				#address-cells = <0x1>;
				#size-cells = <0x0>;

				port@0 {
					reg = <0x0>;
					status = "okay";
					label = "swp0";
					phy-handle = <0x5>;
					phy-mode = "qsgmii";
					phandle = <0x15>;
				};

				port@1 {
					reg = <0x1>;
					status = "okay";
					label = "swp1";
					phy-handle = <0x6>;
					phy-mode = "qsgmii";
					phandle = <0x16>;
				};

				port@2 {
					reg = <0x2>;
					status = "okay";
					label = "swp2";
					phy-handle = <0x7>;
					phy-mode = "qsgmii";
					phandle = <0x17>;
				};

				port@3 {
					reg = <0x3>;
					status = "okay";
					label = "swp3";
					phy-handle = <0x8>;
					phy-mode = "qsgmii";
					phandle = <0x18>;
				};

				port@4 {
					reg = <0x4>;
					phy-mode = "internal";
					status = "okay";
					ethernet = <0x9>;
					phandle = <0x19>;

					fixed-link {
						speed = <0x9c4>;
						full-duplex;
					};
				};

				port@5 {
					reg = <0x5>;
					phy-mode = "internal";
					status = "disabled";
					phandle = <0x1a>;

					fixed-link {
						speed = <0x3e8>;
						full-duplex;
					};
				};
			};
		};

		pci@0,6 {
			reg = <0x600 0x0 0x0 0x0 0x0>;
			status = "disabled";
			phy-mode = "internal";
			phandle = <0x1b>;
		};
	};
};

While Linux looks like this:

/dts-v1/;

/ {
	soc {
		compatible = "simple-bus";
		#address-cells = <0x2>;
		#size-cells = <0x2>;
		ranges;

		pcie@1f0000000 {
			compatible = "pci-host-ecam-generic";
			reg = <0x1 0xf0000000 0x0 0x100000>;
			#address-cells = <0x3>;
			#size-cells = <0x2>;
			msi-parent = <0x11>;
			device_type = "pci";
			bus-range = <0x0 0x0>;
			dma-coherent;
			msi-map = <0x0 0x11 0x17 0xe>;
			iommu-map = <0x0 0x12 0x17 0xe>;
			ranges = <0x82000000 0x1 0xf8000000 0x1 0xf8000000 0x0 0x160000 0xc2000000 0x1 0xf8160000 0x1 0xf8160000 0x0 0x70000 0x82000000 0x1 0xf81d0000 0x1 0xf81d0000 0x0 0x20000 0xc2000000 0x1 0xf81f0000 0x1 0xf81f0000 0x0 0x20000 0x82000000 0x1 0xf8210000 0x1 0xf8210000 0x0 0x20000 0xc2000000 0x1 0xf8230000 0x1 0xf8230000 0x0 0x20000 0x82000000 0x1 0xfc000000 0x1 0xfc000000 0x0 0x400000>;

			ethernet@0,0 {
				compatible = "fsl,enetc";
				reg = <0x0 0x0 0x0 0x0 0x0>;
				status = "okay";
				phy-handle = <0x13>;
				phy-connection-type = "sgmii";
				managed = "in-band-status";

				mdio {
					#address-cells = <0x1>;
					#size-cells = <0x0>;

					ethernet-phy@2 {
						reg = <0x2>;
						phandle = <0x13>;
					};
				};
			};

			ethernet@0,1 {
				compatible = "fsl,enetc";
				reg = <0x100 0x0 0x0 0x0 0x0>;
				status = "disabled";
			};

			ethernet@0,2 {
				compatible = "fsl,enetc";
				reg = <0x200 0x0 0x0 0x0 0x0>;
				phy-mode = "internal";
				status = "okay";
				phandle = <0x18>;

				fixed-link {
					speed = <0x9c4>;
					full-duplex;
				};
			};

			mdio@0,3 {
				compatible = "fsl,enetc-mdio";
				reg = <0x300 0x0 0x0 0x0 0x0>;
				#address-cells = <0x1>;
				#size-cells = <0x0>;

				ethernet-phy@10 {
					reg = <0x10>;
					phandle = <0x14>;
				};

				ethernet-phy@11 {
					reg = <0x11>;
					phandle = <0x15>;
				};

				ethernet-phy@12 {
					reg = <0x12>;
					phandle = <0x16>;
				};

				ethernet-phy@13 {
					reg = <0x13>;
					phandle = <0x17>;
				};
			};

			ethernet@0,4 {
				compatible = "fsl,enetc-ptp";
				reg = <0x400 0x0 0x0 0x0 0x0>;
				clocks = <0x2 0x2 0x3>;
				little-endian;
				fsl,extts-fifo;
			};

			ethernet-switch@0,5 {
				reg = <0x500 0x0 0x0 0x0 0x0>;
				interrupts = <0x0 0x5f 0x4>;
				status = "okay";

				ports {
					#address-cells = <0x1>;
					#size-cells = <0x0>;

					port@0 {
						reg = <0x0>;
						status = "okay";
						label = "swp0";
						managed = "in-band-status";
						phy-handle = <0x14>;
						phy-mode = "qsgmii";
					};

					port@1 {
						reg = <0x1>;
						status = "okay";
						label = "swp1";
						managed = "in-band-status";
						phy-handle = <0x15>;
						phy-mode = "qsgmii";
					};

					port@2 {
						reg = <0x2>;
						status = "okay";
						label = "swp2";
						managed = "in-band-status";
						phy-handle = <0x16>;
						phy-mode = "qsgmii";
					};

					port@3 {
						reg = <0x3>;
						status = "okay";
						label = "swp3";
						managed = "in-band-status";
						phy-handle = <0x17>;
						phy-mode = "qsgmii";
					};

					port@4 {
						reg = <0x4>;
						phy-mode = "internal";
						status = "okay";
						ethernet = <0x18>;

						fixed-link {
							speed = <0x9c4>;
							full-duplex;
						};
					};

					port@5 {
						reg = <0x5>;
						phy-mode = "internal";
						status = "disabled";

						fixed-link {
							speed = <0x3e8>;
							full-duplex;
						};
					};
				};
			};

			ethernet@0,6 {
				compatible = "fsl,enetc";
				reg = <0x600 0x0 0x0 0x0 0x0>;
				phy-mode = "internal";
				status = "disabled";

				fixed-link {
					speed = <0x3e8>;
					full-duplex;
				};
			};

			rcec@1f,0 {
				reg = <0xf800 0x0 0x0 0x0 0x0>;
				interrupts = <0x0 0x5e 0x4>;
			};
		};
	};
};

I mean, they look pretty similar to me? The biggest difference is that
the ENETC ECAM is under the /soc node in Linux, but under / in U-Boot,
as well as some BAR memory areas that are not marked as prefetchable or
non-prefetchable in the U-Boot device tree. But excuse me, that isn't an
Ethernet/DSA problem, is it?

Re: incompatible device trees between u-boot and linux

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 11208 bytes --]

On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> >
> > > Hi,
> > >
> > > I noticed that there is a fallback to the u-boot device tree for linux
> > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > > is working fine for imx devices, for example, where you can just boot a
> > > stock installer iso via EFI. It will just work and it is quite a nice
> > > feature as a fallback.
> > >
> > > Now for the layerscape architecture, the ls1028a in my case, things are
> > > more difficult because the bindings differ between u-boot and linux - one
> > > which comes to mind is DSA and ethernet.
> > >
> > > Which begs the general question, is it encouraged to have both bindings
> > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > > is no real review or alignment but just added as needed, which is ok if
> > > they are local to u-boot. But since they are nowadays passed to linux
> > > (by default!) I'm not so sure anymore.
> > >
> > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > > they should (could?) be shared between linux and u-boot.
> > >
> > > -michael
> > >
> > > [1]
> > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> >
> > The U-Boot device tree is supposed to be able to be passed on to Linux
> > and Just Work.  The bindings are not supposed to be different between
> > the two (except for when we take the binding while it's being hashed out
> > upstream BUT THEN RESYNCED).
> 
> You might need to spell that out a bit clearer.
> 
> You are saying that both U-Boot and Linux are allowed to have their own
> custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> for Linux), as long as the device tree files themselves are in sync, and
> the subset of the device tree blob understood by Linux (i.e. the U-Boot
> blob sans the U-Boot specifics) is compatible with the Linux DT blob?

I don't know what about the Linux example makes it Linux specific.  But
yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
logic (as much as it can be used is that device trees are device trees
and describe the hardware and developers don't need to write a device
tree for Linux and a device tree for U-Boot and a device tree for
FreeBSD and ...  So yes, you're supposed to use the device tree for a
platform and it works here and there and every where.

> To expand even further on that, it means we should put 'managed = "in-band-status"'
> in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> does not use phylink?

We should be able to drop in the device trees from Linux and use them.
Custodians should be re-syncing them periodically.  Some are, even.

> > Incompatible device trees / bindings are a
> > bug that needs to be fixed.
> 
> Curious that Michael mentions Ethernet and DSA on LS1028A.
> 
> I decompiled the two:
> 
> dtc -I dtb -O dts < arch/arm/dts/fsl-ls1028a-rdb.dtb > u-boot.dts
> dtc -I dtb -O dts < arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dtb > linux.dts
> 
> and analyzed the Ethernet portion.
> 
> U-Boot looks like this:
> 
> /dts-v1/;
> 
> / {
> 	compatible = "fsl,ls1028a-rdb", "fsl,ls1028a";
> 	interrupt-parent = <0x1>;
> 	#address-cells = <0x2>;
> 	#size-cells = <0x2>;
> 	model = "NXP Layerscape 1028a RDB Board";
> 
> 	pcie@1f0000000 {
> 		compatible = "pci-host-ecam-generic";
> 		bus-range = <0x0 0x0>;
> 		reg = <0x1 0xf0000000 0x0 0x100000>;
> 		#address-cells = <0x3>;
> 		#size-cells = <0x2>;
> 		device_type = "pci";
> 		ranges = <0x82000000 0x0 0x0 0x1 0xf8000000 0x0 0x160000>;
> 
> 		pci@0,0 {
> 			reg = <0x0 0x0 0x0 0x0 0x0>;
> 			status = "okay";
> 			phy-mode = "sgmii";
> 			phy-handle = <0x4>;
> 			phandle = <0x11>;
> 		};
> 
> 		pci@0,1 {
> 			reg = <0x100 0x0 0x0 0x0 0x0>;
> 			status = "disabled";
> 			phandle = <0x12>;
> 		};
> 
> 		pci@0,2 {
> 			reg = <0x200 0x0 0x0 0x0 0x0>;
> 			status = "okay";
> 			phy-mode = "internal";
> 			phandle = <0x9>;
> 
> 			fixed-link {
> 				speed = <0x9c4>;
> 				full-duplex;
> 			};
> 		};
> 
> 		pci@0,3 {
> 			#address-cells = <0x0>;
> 			#size-cells = <0x1>;
> 			reg = <0x300 0x0 0x0 0x0 0x0>;
> 			status = "okay";
> 			phandle = <0x13>;
> 
> 			fixed-link {
> 				speed = <0x3e8>;
> 				full-duplex;
> 			};
> 
> 			phy@2 {
> 				reg = <0x2>;
> 				phandle = <0x4>;
> 			};
> 
> 			phy@10 {
> 				reg = <0x10>;
> 				phandle = <0x5>;
> 			};
> 
> 			phy@11 {
> 				reg = <0x11>;
> 				phandle = <0x6>;
> 			};
> 
> 			phy@12 {
> 				reg = <0x12>;
> 				phandle = <0x7>;
> 			};
> 
> 			phy@13 {
> 				reg = <0x13>;
> 				phandle = <0x8>;
> 			};
> 		};
> 
> 		pci@0,5 {
> 			reg = <0x500 0x0 0x0 0x0 0x0>;
> 			status = "okay";
> 			phandle = <0x14>;
> 
> 			ports {
> 				#address-cells = <0x1>;
> 				#size-cells = <0x0>;
> 
> 				port@0 {
> 					reg = <0x0>;
> 					status = "okay";
> 					label = "swp0";
> 					phy-handle = <0x5>;
> 					phy-mode = "qsgmii";
> 					phandle = <0x15>;
> 				};
> 
> 				port@1 {
> 					reg = <0x1>;
> 					status = "okay";
> 					label = "swp1";
> 					phy-handle = <0x6>;
> 					phy-mode = "qsgmii";
> 					phandle = <0x16>;
> 				};
> 
> 				port@2 {
> 					reg = <0x2>;
> 					status = "okay";
> 					label = "swp2";
> 					phy-handle = <0x7>;
> 					phy-mode = "qsgmii";
> 					phandle = <0x17>;
> 				};
> 
> 				port@3 {
> 					reg = <0x3>;
> 					status = "okay";
> 					label = "swp3";
> 					phy-handle = <0x8>;
> 					phy-mode = "qsgmii";
> 					phandle = <0x18>;
> 				};
> 
> 				port@4 {
> 					reg = <0x4>;
> 					phy-mode = "internal";
> 					status = "okay";
> 					ethernet = <0x9>;
> 					phandle = <0x19>;
> 
> 					fixed-link {
> 						speed = <0x9c4>;
> 						full-duplex;
> 					};
> 				};
> 
> 				port@5 {
> 					reg = <0x5>;
> 					phy-mode = "internal";
> 					status = "disabled";
> 					phandle = <0x1a>;
> 
> 					fixed-link {
> 						speed = <0x3e8>;
> 						full-duplex;
> 					};
> 				};
> 			};
> 		};
> 
> 		pci@0,6 {
> 			reg = <0x600 0x0 0x0 0x0 0x0>;
> 			status = "disabled";
> 			phy-mode = "internal";
> 			phandle = <0x1b>;
> 		};
> 	};
> };
> 
> While Linux looks like this:
> 
> /dts-v1/;
> 
> / {
> 	soc {
> 		compatible = "simple-bus";
> 		#address-cells = <0x2>;
> 		#size-cells = <0x2>;
> 		ranges;
> 
> 		pcie@1f0000000 {
> 			compatible = "pci-host-ecam-generic";
> 			reg = <0x1 0xf0000000 0x0 0x100000>;
> 			#address-cells = <0x3>;
> 			#size-cells = <0x2>;
> 			msi-parent = <0x11>;
> 			device_type = "pci";
> 			bus-range = <0x0 0x0>;
> 			dma-coherent;
> 			msi-map = <0x0 0x11 0x17 0xe>;
> 			iommu-map = <0x0 0x12 0x17 0xe>;
> 			ranges = <0x82000000 0x1 0xf8000000 0x1 0xf8000000 0x0 0x160000 0xc2000000 0x1 0xf8160000 0x1 0xf8160000 0x0 0x70000 0x82000000 0x1 0xf81d0000 0x1 0xf81d0000 0x0 0x20000 0xc2000000 0x1 0xf81f0000 0x1 0xf81f0000 0x0 0x20000 0x82000000 0x1 0xf8210000 0x1 0xf8210000 0x0 0x20000 0xc2000000 0x1 0xf8230000 0x1 0xf8230000 0x0 0x20000 0x82000000 0x1 0xfc000000 0x1 0xfc000000 0x0 0x400000>;
> 
> 			ethernet@0,0 {
> 				compatible = "fsl,enetc";
> 				reg = <0x0 0x0 0x0 0x0 0x0>;
> 				status = "okay";
> 				phy-handle = <0x13>;
> 				phy-connection-type = "sgmii";
> 				managed = "in-band-status";
> 
> 				mdio {
> 					#address-cells = <0x1>;
> 					#size-cells = <0x0>;
> 
> 					ethernet-phy@2 {
> 						reg = <0x2>;
> 						phandle = <0x13>;
> 					};
> 				};
> 			};
> 
> 			ethernet@0,1 {
> 				compatible = "fsl,enetc";
> 				reg = <0x100 0x0 0x0 0x0 0x0>;
> 				status = "disabled";
> 			};
> 
> 			ethernet@0,2 {
> 				compatible = "fsl,enetc";
> 				reg = <0x200 0x0 0x0 0x0 0x0>;
> 				phy-mode = "internal";
> 				status = "okay";
> 				phandle = <0x18>;
> 
> 				fixed-link {
> 					speed = <0x9c4>;
> 					full-duplex;
> 				};
> 			};
> 
> 			mdio@0,3 {
> 				compatible = "fsl,enetc-mdio";
> 				reg = <0x300 0x0 0x0 0x0 0x0>;
> 				#address-cells = <0x1>;
> 				#size-cells = <0x0>;
> 
> 				ethernet-phy@10 {
> 					reg = <0x10>;
> 					phandle = <0x14>;
> 				};
> 
> 				ethernet-phy@11 {
> 					reg = <0x11>;
> 					phandle = <0x15>;
> 				};
> 
> 				ethernet-phy@12 {
> 					reg = <0x12>;
> 					phandle = <0x16>;
> 				};
> 
> 				ethernet-phy@13 {
> 					reg = <0x13>;
> 					phandle = <0x17>;
> 				};
> 			};
> 
> 			ethernet@0,4 {
> 				compatible = "fsl,enetc-ptp";
> 				reg = <0x400 0x0 0x0 0x0 0x0>;
> 				clocks = <0x2 0x2 0x3>;
> 				little-endian;
> 				fsl,extts-fifo;
> 			};
> 
> 			ethernet-switch@0,5 {
> 				reg = <0x500 0x0 0x0 0x0 0x0>;
> 				interrupts = <0x0 0x5f 0x4>;
> 				status = "okay";
> 
> 				ports {
> 					#address-cells = <0x1>;
> 					#size-cells = <0x0>;
> 
> 					port@0 {
> 						reg = <0x0>;
> 						status = "okay";
> 						label = "swp0";
> 						managed = "in-band-status";
> 						phy-handle = <0x14>;
> 						phy-mode = "qsgmii";
> 					};
> 
> 					port@1 {
> 						reg = <0x1>;
> 						status = "okay";
> 						label = "swp1";
> 						managed = "in-band-status";
> 						phy-handle = <0x15>;
> 						phy-mode = "qsgmii";
> 					};
> 
> 					port@2 {
> 						reg = <0x2>;
> 						status = "okay";
> 						label = "swp2";
> 						managed = "in-band-status";
> 						phy-handle = <0x16>;
> 						phy-mode = "qsgmii";
> 					};
> 
> 					port@3 {
> 						reg = <0x3>;
> 						status = "okay";
> 						label = "swp3";
> 						managed = "in-band-status";
> 						phy-handle = <0x17>;
> 						phy-mode = "qsgmii";
> 					};
> 
> 					port@4 {
> 						reg = <0x4>;
> 						phy-mode = "internal";
> 						status = "okay";
> 						ethernet = <0x18>;
> 
> 						fixed-link {
> 							speed = <0x9c4>;
> 							full-duplex;
> 						};
> 					};
> 
> 					port@5 {
> 						reg = <0x5>;
> 						phy-mode = "internal";
> 						status = "disabled";
> 
> 						fixed-link {
> 							speed = <0x3e8>;
> 							full-duplex;
> 						};
> 					};
> 				};
> 			};
> 
> 			ethernet@0,6 {
> 				compatible = "fsl,enetc";
> 				reg = <0x600 0x0 0x0 0x0 0x0>;
> 				phy-mode = "internal";
> 				status = "disabled";
> 
> 				fixed-link {
> 					speed = <0x3e8>;
> 					full-duplex;
> 				};
> 			};
> 
> 			rcec@1f,0 {
> 				reg = <0xf800 0x0 0x0 0x0 0x0>;
> 				interrupts = <0x0 0x5e 0x4>;
> 			};
> 		};
> 	};
> };
> 
> I mean, they look pretty similar to me? The biggest difference is that
> the ENETC ECAM is under the /soc node in Linux, but under / in U-Boot,
> as well as some BAR memory areas that are not marked as prefetchable or
> non-prefetchable in the U-Boot device tree. But excuse me, that isn't an
> Ethernet/DSA problem, is it?

I'll leave this part to Michael.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: incompatible device trees between u-boot and linux

[Vladimir Oltean]

On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> > >
> > > > Hi,
> > > >
> > > > I noticed that there is a fallback to the u-boot device tree for linux
> > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > > > is working fine for imx devices, for example, where you can just boot a
> > > > stock installer iso via EFI. It will just work and it is quite a nice
> > > > feature as a fallback.
> > > >
> > > > Now for the layerscape architecture, the ls1028a in my case, things are
> > > > more difficult because the bindings differ between u-boot and linux - one
> > > > which comes to mind is DSA and ethernet.
> > > >
> > > > Which begs the general question, is it encouraged to have both bindings
> > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > > > is no real review or alignment but just added as needed, which is ok if
> > > > they are local to u-boot. But since they are nowadays passed to linux
> > > > (by default!) I'm not so sure anymore.
> > > >
> > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > > > they should (could?) be shared between linux and u-boot.
> > > >
> > > > -michael
> > > >
> > > > [1]
> > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> > >
> > > The U-Boot device tree is supposed to be able to be passed on to Linux
> > > and Just Work.  The bindings are not supposed to be different between
> > > the two (except for when we take the binding while it's being hashed out
> > > upstream BUT THEN RESYNCED).
> >
> > You might need to spell that out a bit clearer.
> >
> > You are saying that both U-Boot and Linux are allowed to have their own
> > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> > for Linux), as long as the device tree files themselves are in sync, and
> > the subset of the device tree blob understood by Linux (i.e. the U-Boot
> > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
>
> I don't know what about the Linux example makes it Linux specific.  But
> yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> logic (as much as it can be used is that device trees are device trees
        ^
        I don't think this parenthesis ever closes...

> and describe the hardware and developers don't need to write a device
> tree for Linux and a device tree for U-Boot and a device tree for
> FreeBSD and ...  So yes, you're supposed to use the device tree for a
                ^
                so I never get the answer to "the whole reason is...".

> platform and it works here and there and every where.

The fact that only Linux uses it makes it Linux specific.

> > To expand even further on that, it means we should put 'managed = "in-band-status"'
> > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> > does not use phylink?
>
> We should be able to drop in the device trees from Linux and use them.
> Custodians should be re-syncing them periodically.  Some are, even.

Are you ready to take up device tree bindings for PTP timers, PCIe root
complex event collectors, cascaded interrupt controllers, things which
U-Boot will never ever need to support?

At least in Linux there is a policy to not add device tree nodes that do
not have drivers. Is the same policy not true for U-Boot? At least your
./scripts/checkpatch.pl does have the same "check for DT compatible
documentation" section as Linux. You might consider removing it if you
want people to not strip the DTs they submit to U-Boot.

And why do we even maintain the device tree bindings in Linux at all?
It seems rather counter-productive for both ends to do that, if it is
expected that the kernel works with DT blobs provided by third parties too,
and if all third parties need to resync with it (there are other boot
loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
doesn't feel right for the reference to be the Linux kernel. Maybe this
is something that needs to be brought up with higher-level Linux maintainers.

I have no problem at all with structuring the device tree in the same
way in U-Boot as in Linux, as long as that proves to not be a foolish
endeavor.

Re: incompatible device trees between u-boot and linux

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 5154 bytes --]

On Wed, Aug 25, 2021 at 06:12:20PM +0300, Vladimir Oltean wrote:
> On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I noticed that there is a fallback to the u-boot device tree for linux
> > > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > > > > is working fine for imx devices, for example, where you can just boot a
> > > > > stock installer iso via EFI. It will just work and it is quite a nice
> > > > > feature as a fallback.
> > > > >
> > > > > Now for the layerscape architecture, the ls1028a in my case, things are
> > > > > more difficult because the bindings differ between u-boot and linux - one
> > > > > which comes to mind is DSA and ethernet.
> > > > >
> > > > > Which begs the general question, is it encouraged to have both bindings
> > > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > > > > is no real review or alignment but just added as needed, which is ok if
> > > > > they are local to u-boot. But since they are nowadays passed to linux
> > > > > (by default!) I'm not so sure anymore.
> > > > >
> > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > > > > they should (could?) be shared between linux and u-boot.
> > > > >
> > > > > -michael
> > > > >
> > > > > [1]
> > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> > > >
> > > > The U-Boot device tree is supposed to be able to be passed on to Linux
> > > > and Just Work.  The bindings are not supposed to be different between
> > > > the two (except for when we take the binding while it's being hashed out
> > > > upstream BUT THEN RESYNCED).
> > >
> > > You might need to spell that out a bit clearer.
> > >
> > > You are saying that both U-Boot and Linux are allowed to have their own
> > > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> > > for Linux), as long as the device tree files themselves are in sync, and
> > > the subset of the device tree blob understood by Linux (i.e. the U-Boot
> > > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
> >
> > I don't know what about the Linux example makes it Linux specific.  But
> > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> > by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> > logic (as much as it can be used is that device trees are device trees
>         ^
>         I don't think this parenthesis ever closes...

Ah, whoops.  Should have been "(as much as it can be used)" because it
does get #included instead in some cases, for reasons.

> 
> > and describe the hardware and developers don't need to write a device
> > tree for Linux and a device tree for U-Boot and a device tree for
> > FreeBSD and ...  So yes, you're supposed to use the device tree for a
>                 ^
>                 so I never get the answer to "the whole reason is...".
> 
> > platform and it works here and there and every where.
> 
> The fact that only Linux uses it makes it Linux specific.
> 
> > > To expand even further on that, it means we should put 'managed = "in-band-status"'
> > > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> > > does not use phylink?
> >
> > We should be able to drop in the device trees from Linux and use them.
> > Custodians should be re-syncing them periodically.  Some are, even.
> 
> Are you ready to take up device tree bindings for PTP timers, PCIe root
> complex event collectors, cascaded interrupt controllers, things which
> U-Boot will never ever need to support?
> 
> At least in Linux there is a policy to not add device tree nodes that do
> not have drivers. Is the same policy not true for U-Boot? At least your
> ./scripts/checkpatch.pl does have the same "check for DT compatible
> documentation" section as Linux. You might consider removing it if you
> want people to not strip the DTs they submit to U-Boot.
> 
> And why do we even maintain the device tree bindings in Linux at all?
> It seems rather counter-productive for both ends to do that, if it is
> expected that the kernel works with DT blobs provided by third parties too,
> and if all third parties need to resync with it (there are other boot
> loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
> doesn't feel right for the reference to be the Linux kernel. Maybe this
> is something that needs to be brought up with higher-level Linux maintainers.
> 
> I have no problem at all with structuring the device tree in the same
> way in U-Boot as in Linux, as long as that proves to not be a foolish
> endeavor.

DT is ABI is hardware description and OS-agnostic has been the rule for
10+ years.  If that's no longer the case, can someone please tell me?

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: incompatible device trees between u-boot and linux

[Vladimir Oltean]

On Wed, Aug 25, 2021 at 11:24:08AM -0400, Tom Rini wrote:
> On Wed, Aug 25, 2021 at 06:12:20PM +0300, Vladimir Oltean wrote:
> > On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> > > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> > > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I noticed that there is a fallback to the u-boot device tree for linux
> > > > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > > > > > is working fine for imx devices, for example, where you can just boot a
> > > > > > stock installer iso via EFI. It will just work and it is quite a nice
> > > > > > feature as a fallback.
> > > > > >
> > > > > > Now for the layerscape architecture, the ls1028a in my case, things are
> > > > > > more difficult because the bindings differ between u-boot and linux - one
> > > > > > which comes to mind is DSA and ethernet.
> > > > > >
> > > > > > Which begs the general question, is it encouraged to have both bindings
> > > > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > > > > > is no real review or alignment but just added as needed, which is ok if
> > > > > > they are local to u-boot. But since they are nowadays passed to linux
> > > > > > (by default!) I'm not so sure anymore.
> > > > > >
> > > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > > > > > they should (could?) be shared between linux and u-boot.
> > > > > >
> > > > > > -michael
> > > > > >
> > > > > > [1]
> > > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> > > > >
> > > > > The U-Boot device tree is supposed to be able to be passed on to Linux
> > > > > and Just Work.  The bindings are not supposed to be different between
> > > > > the two (except for when we take the binding while it's being hashed out
> > > > > upstream BUT THEN RESYNCED).
> > > >
> > > > You might need to spell that out a bit clearer.
> > > >
> > > > You are saying that both U-Boot and Linux are allowed to have their own
> > > > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> > > > for Linux), as long as the device tree files themselves are in sync, and
> > > > the subset of the device tree blob understood by Linux (i.e. the U-Boot
> > > > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
> > >
> > > I don't know what about the Linux example makes it Linux specific.  But
> > > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> > > by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> > > logic (as much as it can be used is that device trees are device trees
> >         ^
> >         I don't think this parenthesis ever closes...
>
> Ah, whoops.  Should have been "(as much as it can be used)" because it
> does get #included instead in some cases, for reasons.
>
> >
> > > and describe the hardware and developers don't need to write a device
> > > tree for Linux and a device tree for U-Boot and a device tree for
> > > FreeBSD and ...  So yes, you're supposed to use the device tree for a
> >                 ^
> >                 so I never get the answer to "the whole reason is...".
> >
> > > platform and it works here and there and every where.
> >
> > The fact that only Linux uses it makes it Linux specific.
> >
> > > > To expand even further on that, it means we should put 'managed = "in-band-status"'
> > > > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> > > > does not use phylink?
> > >
> > > We should be able to drop in the device trees from Linux and use them.
> > > Custodians should be re-syncing them periodically.  Some are, even.
> >
> > Are you ready to take up device tree bindings for PTP timers, PCIe root
> > complex event collectors, cascaded interrupt controllers, things which
> > U-Boot will never ever need to support?
> >
> > At least in Linux there is a policy to not add device tree nodes that do
> > not have drivers. Is the same policy not true for U-Boot? At least your
> > ./scripts/checkpatch.pl does have the same "check for DT compatible
> > documentation" section as Linux. You might consider removing it if you
> > want people to not strip the DTs they submit to U-Boot.
> >
> > And why do we even maintain the device tree bindings in Linux at all?
> > It seems rather counter-productive for both ends to do that, if it is
> > expected that the kernel works with DT blobs provided by third parties too,
> > and if all third parties need to resync with it (there are other boot
> > loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
> > doesn't feel right for the reference to be the Linux kernel. Maybe this
> > is something that needs to be brought up with higher-level Linux maintainers.
> >
> > I have no problem at all with structuring the device tree in the same
> > way in U-Boot as in Linux, as long as that proves to not be a foolish
> > endeavor.
>
> DT is ABI is hardware description and OS-agnostic has been the rule for
> 10+ years.  If that's no longer the case, can someone please tell me?

So if Michael's board with DT provided by U-Boot doesn't work for some
stupid reason like "Linux expects the pcie node to be under /soc", or
"Linux wants all PCIe BARs of a RCIEP ECAM to be spelled out in the
'ranges' property, because it's too dumb to detect them itself", or
something like that, I've got no argument against that, let's go ahead
and resync U-Boot with Linux.

But "DT is ABI is hardware description" is a pretty vague truism that
does not actually help here.

Will you accept device trees with devices for which a driver will never
probe in U-Boot, and will you remove the checkpatch warnings about those,
to not discourage people from submitting them prior to the actual public
review?

If a U-Boot driver will be written for a device that does not have a
driver yet in Linux, then the Linux driver will be written but with
DT bindings incompatible with what was established in U-Boot, will you
wake up the U-Boot developer from the grave and ask them to resync the
driver to follow Linux? Or will you accept drivers at all for hardware
that is not supported by Linux?

I also think there are various degrees of what it means "to work" with a
device tree provided directly by U-Boot. Distros like Arch Linux ARM
have a package for device tree blobs, and it is expected that these are
updated by the distro, and U-Boot just loads them. As Michael points out,
the DT provided by U-Boot is just a fallback. The OS should boot to
prompt and have a network connection to set itself up properly again.
But we need to draw a harder line on what we _actually_ desire to work
beyond that.

Re: incompatible device trees between u-boot and linux

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 9098 bytes --]

On Wed, Aug 25, 2021 at 06:43:23PM +0300, Vladimir Oltean wrote:
> On Wed, Aug 25, 2021 at 11:24:08AM -0400, Tom Rini wrote:
> > On Wed, Aug 25, 2021 at 06:12:20PM +0300, Vladimir Oltean wrote:
> > > On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> > > > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> > > > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > > > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I noticed that there is a fallback to the u-boot device tree for linux
> > > > > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > > > > > > is working fine for imx devices, for example, where you can just boot a
> > > > > > > stock installer iso via EFI. It will just work and it is quite a nice
> > > > > > > feature as a fallback.
> > > > > > >
> > > > > > > Now for the layerscape architecture, the ls1028a in my case, things are
> > > > > > > more difficult because the bindings differ between u-boot and linux - one
> > > > > > > which comes to mind is DSA and ethernet.
> > > > > > >
> > > > > > > Which begs the general question, is it encouraged to have both bindings
> > > > > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > > > > > > is no real review or alignment but just added as needed, which is ok if
> > > > > > > they are local to u-boot. But since they are nowadays passed to linux
> > > > > > > (by default!) I'm not so sure anymore.
> > > > > > >
> > > > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > > > > > > they should (could?) be shared between linux and u-boot.
> > > > > > >
> > > > > > > -michael
> > > > > > >
> > > > > > > [1]
> > > > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> > > > > >
> > > > > > The U-Boot device tree is supposed to be able to be passed on to Linux
> > > > > > and Just Work.  The bindings are not supposed to be different between
> > > > > > the two (except for when we take the binding while it's being hashed out
> > > > > > upstream BUT THEN RESYNCED).
> > > > >
> > > > > You might need to spell that out a bit clearer.
> > > > >
> > > > > You are saying that both U-Boot and Linux are allowed to have their own
> > > > > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> > > > > for Linux), as long as the device tree files themselves are in sync, and
> > > > > the subset of the device tree blob understood by Linux (i.e. the U-Boot
> > > > > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
> > > >
> > > > I don't know what about the Linux example makes it Linux specific.  But
> > > > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> > > > by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> > > > logic (as much as it can be used is that device trees are device trees
> > >         ^
> > >         I don't think this parenthesis ever closes...
> >
> > Ah, whoops.  Should have been "(as much as it can be used)" because it
> > does get #included instead in some cases, for reasons.
> >
> > >
> > > > and describe the hardware and developers don't need to write a device
> > > > tree for Linux and a device tree for U-Boot and a device tree for
> > > > FreeBSD and ...  So yes, you're supposed to use the device tree for a
> > >                 ^
> > >                 so I never get the answer to "the whole reason is...".
> > >
> > > > platform and it works here and there and every where.
> > >
> > > The fact that only Linux uses it makes it Linux specific.
> > >
> > > > > To expand even further on that, it means we should put 'managed = "in-band-status"'
> > > > > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> > > > > does not use phylink?
> > > >
> > > > We should be able to drop in the device trees from Linux and use them.
> > > > Custodians should be re-syncing them periodically.  Some are, even.
> > >
> > > Are you ready to take up device tree bindings for PTP timers, PCIe root
> > > complex event collectors, cascaded interrupt controllers, things which
> > > U-Boot will never ever need to support?
> > >
> > > At least in Linux there is a policy to not add device tree nodes that do
> > > not have drivers. Is the same policy not true for U-Boot? At least your
> > > ./scripts/checkpatch.pl does have the same "check for DT compatible
> > > documentation" section as Linux. You might consider removing it if you
> > > want people to not strip the DTs they submit to U-Boot.
> > >
> > > And why do we even maintain the device tree bindings in Linux at all?
> > > It seems rather counter-productive for both ends to do that, if it is
> > > expected that the kernel works with DT blobs provided by third parties too,
> > > and if all third parties need to resync with it (there are other boot
> > > loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
> > > doesn't feel right for the reference to be the Linux kernel. Maybe this
> > > is something that needs to be brought up with higher-level Linux maintainers.
> > >
> > > I have no problem at all with structuring the device tree in the same
> > > way in U-Boot as in Linux, as long as that proves to not be a foolish
> > > endeavor.
> >
> > DT is ABI is hardware description and OS-agnostic has been the rule for
> > 10+ years.  If that's no longer the case, can someone please tell me?
> 
> So if Michael's board with DT provided by U-Boot doesn't work for some
> stupid reason like "Linux expects the pcie node to be under /soc", or
> "Linux wants all PCIe BARs of a RCIEP ECAM to be spelled out in the
> 'ranges' property, because it's too dumb to detect them itself", or
> something like that, I've got no argument against that, let's go ahead
> and resync U-Boot with Linux.
> 
> But "DT is ABI is hardware description" is a pretty vague truism that
> does not actually help here.

I'm saying that because it's what's been said for what feels like 10+
years.  I don't want to think how many countless hours have been spent
on that point at conferences over the years.  It's not even a Linux
thing.  I would swear you can (or could, unless it got broken) take the
same DTB for some platforms and boot Linux or FreeBSD or some other BSD
or maybe even VxWorks and it works.

> Will you accept device trees with devices for which a driver will never
> probe in U-Boot,

Yes, I will absolutely take device trees that have devices we don't need
in U-Boot since the point is, and many SoC vendors are doing (and the
ones that aren't are, I am not happy about / with) right now.

> and will you remove the checkpatch warnings about those,
> to not discourage people from submitting them prior to the actual public
> review?

With respect to checkpatch.pl, maybe I'm just missing the line in
question?  Or maybe it's a kernel-related warning we need to disable in
our .checkpatch.conf.  But I don't want to side-track over this part.

> If a U-Boot driver will be written for a device that does not have a
> driver yet in Linux, then the Linux driver will be written but with
> DT bindings incompatible with what was established in U-Boot, will you
> wake up the U-Boot developer from the grave and ask them to resync the
> driver to follow Linux? Or will you accept drivers at all for hardware
> that is not supported by Linux?

What I've said for years (but yes, I've missed changes, maybe the yaml
dt binding stuff would help so I could make CI fail or at least require
manual override?) is that U-Boot will take immature bindings but it's on
the developer to re-sync once the bindings are fully reviewed.  This is
to help with the chicken-and-egg problem.  But old bindings are not
intended to be supported, once it's finalized.  That is part of the
bargain.

> I also think there are various degrees of what it means "to work" with a
> device tree provided directly by U-Boot. Distros like Arch Linux ARM
> have a package for device tree blobs, and it is expected that these are
> updated by the distro, and U-Boot just loads them. As Michael points out,
> the DT provided by U-Boot is just a fallback. The OS should boot to
> prompt and have a network connection to set itself up properly again.
> But we need to draw a harder line on what we _actually_ desire to work
> beyond that.

Every distribution has a package for device tree binaries, because
that's how you get a device tree on the still vast majority of platforms
that don't ship one in-flash (RPis being the modern exception) and to my
knowledge none of them are happy about having to build and pass and make
sure the right one is used on a given board at boot.  So yes, U-Boot
being able to pass a device tree on to the next stage is one of those
things to help everything Just Work and be boring.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: incompatible device trees between u-boot and linux

[Vladimir Oltean]

On Wed, Aug 25, 2021 at 04:09:50PM -0400, Tom Rini wrote:
> I'm saying that because it's what's been said for what feels like 10+
> years.  I don't want to think how many countless hours have been spent
> on that point at conferences over the years.  It's not even a Linux
> thing.  I would swear you can (or could, unless it got broken) take the
> same DTB for some platforms and boot Linux or FreeBSD or some other BSD
> or maybe even VxWorks and it works.

So I absolutely do not oppose the greater goal, and if you say that
other silicon vendors do it, then shame on us really, NXP should step up
their game and be way stricter during internal review and such for things
that matter.

I'm afraid it's something that must trickle down from the management and
maintainership level before it could be effective.

In any case, it doesn't sound absurd at all, with a bit of passion it
could be done on all Layerscapes. I would be absolutely glad to help on
the Ethernet / DSA side of things (which I believe is the reason why
Michael summoned me into this thread), but I don't believe that's where
the problem is right now. When I added the DM_DSA uclass to U-Boot I did
my best to pick a reasonable subset of Linux DSA, and with compatible
device tree bindings. Also maintaining the Linux side of things, I did
provide feedback to Tim Harvey for the Microchip KSZ switches as to
what is the subset supported by U-Boot that would also be DT-compatible
with Linux. If it turns out that I failed at that, I am willing to
rework what we have.

I have been known on a few occasions to say "U-Boot does not parse this
part of the device tree, you can just strip it away", but I will keep my
mouth shut from now on.

Re: incompatible device trees between u-boot and linux

[Michael Walle]

Am 2021-08-26 01:03, schrieb Vladimir Oltean:
> On Wed, Aug 25, 2021 at 04:09:50PM -0400, Tom Rini wrote:
> In any case, it doesn't sound absurd at all, with a bit of passion it
> could be done on all Layerscapes. I would be absolutely glad to help on
> the Ethernet / DSA side of things (which I believe is the reason why
> Michael summoned me into this thread),

;) and because I thought you might be interested in the answer to the
initial question. After all, you also worked on the device trees in 
linux
and u-boot.

> but I don't believe that's where
> the problem is right now. When I added the DM_DSA uclass to U-Boot I 
> did
> my best to pick a reasonable subset of Linux DSA, and with compatible
> device tree bindings. Also maintaining the Linux side of things, I did
> provide feedback to Tim Harvey for the Microchip KSZ switches as to
> what is the subset supported by U-Boot that would also be DT-compatible
> with Linux. If it turns out that I failed at that, I am willing to
> rework what we have.

I started to convert the u-boot device tree yesterday - and it doesn't
look too bad for now. I was already able to copy the kernel soc dtsi
and u-boot is still booting and working.

Theres still one catch at the moment, AFAIK in linux you can put the
PHYs either in the mdio controller node or in a "mdio" subnode within
the ethernet controller node. I'm not sure wether the latter works in
u-boot, but [1] looks promising. At least, linux dtbs are using the
mdio subnodes and u-boot put the phys into the mdio controller node.

Maybe sharing the device tree between linux and u-boot isn't that hard
for the ls1028a after all and its just that nobody did it for now.
Renaming the reference here and there and introducing the linux
compatible strings may do it. I'll come back to you if there are
problems with ethernet (or DSA).

-michael

> I have been known on a few occasions to say "U-Boot does not parse this
> part of the device tree, you can just strip it away", but I will keep 
> my
> mouth shut from now on.

[1] https://lists.denx.de/pipermail/u-boot/2020-May/410169.html

Re: incompatible device trees between u-boot and linux

[Vladimir Oltean]

On Thu, Aug 26, 2021 at 09:35:12AM +0200, Michael Walle wrote:
> Am 2021-08-26 01:03, schrieb Vladimir Oltean:
> > On Wed, Aug 25, 2021 at 04:09:50PM -0400, Tom Rini wrote:
> > In any case, it doesn't sound absurd at all, with a bit of passion it
> > could be done on all Layerscapes. I would be absolutely glad to help on
> > the Ethernet / DSA side of things (which I believe is the reason why
> > Michael summoned me into this thread),
> 
> ;) and because I thought you might be interested in the answer to the
> initial question. After all, you also worked on the device trees in linux
> and u-boot.
> 
> > but I don't believe that's where
> > the problem is right now. When I added the DM_DSA uclass to U-Boot I did
> > my best to pick a reasonable subset of Linux DSA, and with compatible
> > device tree bindings. Also maintaining the Linux side of things, I did
> > provide feedback to Tim Harvey for the Microchip KSZ switches as to
> > what is the subset supported by U-Boot that would also be DT-compatible
> > with Linux. If it turns out that I failed at that, I am willing to
> > rework what we have.
> 
> I started to convert the u-boot device tree yesterday - and it doesn't
> look too bad for now. I was already able to copy the kernel soc dtsi
> and u-boot is still booting and working.
> 
> Theres still one catch at the moment, AFAIK in linux you can put the
> PHYs either in the mdio controller node or in a "mdio" subnode within
> the ethernet controller node. I'm not sure wether the latter works in
> u-boot, but [1] looks promising. At least, linux dtbs are using the
> mdio subnodes and u-boot put the phys into the mdio controller node.

Please change Linux for that, move the PHYs from the per-port MDIO node
to the PF 3 central MDIO controller node. Due to hardware reasons, the
per-port MDIO controller registers are in fact de-featured and should be
hidden from new LS1028A reference manuals.

> Maybe sharing the device tree between linux and u-boot isn't that hard
> for the ls1028a after all and its just that nobody did it for now.
> Renaming the reference here and there and introducing the linux
> compatible strings may do it. I'll come back to you if there are
> problems with ethernet (or DSA).
> 
> -michael
> 
> > I have been known on a few occasions to say "U-Boot does not parse this
> > part of the device tree, you can just strip it away", but I will keep my
> > mouth shut from now on.
> 
> [1] https://lists.denx.de/pipermail/u-boot/2020-May/410169.html

Re: incompatible device trees between u-boot and linux

[Michael Walle]

Am 2021-08-26 18:32, schrieb Vladimir Oltean:
> On Thu, Aug 26, 2021 at 09:35:12AM +0200, Michael Walle wrote:
>> Theres still one catch at the moment, AFAIK in linux you can put the
>> PHYs either in the mdio controller node or in a "mdio" subnode within
>> the ethernet controller node. I'm not sure wether the latter works in
>> u-boot, but [1] looks promising. At least, linux dtbs are using the
>> mdio subnodes and u-boot put the phys into the mdio controller node.
> 
> Please change Linux for that, move the PHYs from the per-port MDIO node
> to the PF 3 central MDIO controller node. Due to hardware reasons, the
> per-port MDIO controller registers are in fact de-featured and should 
> be
> hidden from new LS1028A reference manuals.

Care to share some more details? There should be some more information
besides "for hardware reasons" which should go into the commit message.
Is there an erratum?

In fact, I guess in Rev0 of the RM it has already been removed, at least
from the port memory map (in the RevB RM it was still present).

-michael

Re: incompatible device trees between u-boot and linux

[Vladimir Oltean]

On Sun, Aug 29, 2021 at 01:12:30AM +0200, Michael Walle wrote:
> Am 2021-08-26 18:32, schrieb Vladimir Oltean:
> > On Thu, Aug 26, 2021 at 09:35:12AM +0200, Michael Walle wrote:
> > > Theres still one catch at the moment, AFAIK in linux you can put the
> > > PHYs either in the mdio controller node or in a "mdio" subnode within
> > > the ethernet controller node. I'm not sure wether the latter works in
> > > u-boot, but [1] looks promising. At least, linux dtbs are using the
> > > mdio subnodes and u-boot put the phys into the mdio controller node.
> >
> > Please change Linux for that, move the PHYs from the per-port MDIO node
> > to the PF 3 central MDIO controller node. Due to hardware reasons, the
> > per-port MDIO controller registers are in fact de-featured and should be
> > hidden from new LS1028A reference manuals.
>
> Care to share some more details? There should be some more information
> besides "for hardware reasons" which should go into the commit message.
> Is there an erratum?
>
> In fact, I guess in Rev0 of the RM it has already been removed, at least
> from the port memory map (in the RevB RM it was still present).
>
> -michael

It's a long and boring explanation, PM sent.

Re: incompatible device trees between u-boot and linux

[Mark Kettenis]

> Date: Wed, 25 Aug 2021 18:12:20 +0300
> From: Vladimir Oltean <olteanv@gmail.com>
> 
> On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I noticed that there is a fallback to the u-boot device tree
> > > > > for linux (esp. EFI boot) if no other device tree was found,
> > > > > see [1]. It seems this is working fine for imx devices, for
> > > > > example, where you can just boot a stock installer iso via
> > > > > EFI. It will just work and it is quite a nice feature as a
> > > > > fallback.
> > > > >
> > > > > Now for the layerscape architecture, the ls1028a in my case,
> > > > > things are more difficult because the bindings differ
> > > > > between u-boot and linux - one which comes to mind is DSA
> > > > > and ethernet.
> > > > >
> > > > > Which begs the general question, is it encouraged to have
> > > > > both bindings diverge? To me it seems, that most bindings in
> > > > > u-boot are ad-hoc and there is no real review or alignment
> > > > > but just added as needed, which is ok if they are local to
> > > > > u-boot. But since they are nowadays passed to linux (by
> > > > > default!) I'm not so sure anymore.
> > > > >
> > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi
> > > > > looks like they should (could?) be shared between linux and
> > > > > u-boot.
> > > > >
> > > > > -michael
> > > > >
> > > > > [1]
> > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> > > >
> > > > The U-Boot device tree is supposed to be able to be passed on
> > > > to Linux and Just Work.  The bindings are not supposed to be
> > > > different between the two (except for when we take the binding
> > > > while it's being hashed out upstream BUT THEN RESYNCED).
> > >
> > > You might need to spell that out a bit clearer.
> > >
> > > You are saying that both U-Boot and Linux are allowed to have
> > > their own custom properties (like 'u-boot,dm-spl' for U-Boot,
> > > and 'managed = "in-band-status"' for Linux), as long as the
> > > device tree files themselves are in sync, and the subset of the
> > > device tree blob understood by Linux (i.e. the U-Boot blob sans
> > > the U-Boot specifics) is compatible with the Linux DT blob?
> >
> > I don't know what about the Linux example makes it Linux specific.  But
> > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> > by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> > logic (as much as it can be used is that device trees are device trees
>         ^
>         I don't think this parenthesis ever closes...
> 
> > and describe the hardware and developers don't need to write a device
> > tree for Linux and a device tree for U-Boot and a device tree for
> > FreeBSD and ...  So yes, you're supposed to use the device tree for a
>                 ^
>                 so I never get the answer to "the whole reason is...".
> 
> > platform and it works here and there and every where.
> 
> The fact that only Linux uses it makes it Linux specific.

Huh?  Other OSes use device trees as well on various platforms.  There
are (somewhat) linux-specific properties in those device trees in the
"linux," namespace.

For OpenBSD we don't maintain our own set of device trees.  Instead we
rely on the device tree provided by U-Boot and distribute device trees
built from the Linux tree as an alternative that users can install if
the U-Boot device tree is a bit behind.

> > > To expand even further on that, it means we should put 'managed
> > > = "in-band-status"' in U-Boot, which is a Linux phylink device
> > > tree property, even if U-Boot does not use phylink?
> >
> > We should be able to drop in the device trees from Linux and use them.
> > Custodians should be re-syncing them periodically.  Some are, even.
> 
> Are you ready to take up device tree bindings for PTP timers, PCIe root
> complex event collectors, cascaded interrupt controllers, things which
> U-Boot will never ever need to support?

The policy is that U-Boot uses a verbatim copy of the device tree
source files from Linux (.dts and .dtsi files) and augments that
device tree if necessary by using a *-u-boot.dts files that ideally
only add properties in the "u-boot," namespace.

That policy isn't always followed though.  In some cases support for
certain hardware is added to U-Boot early, before "official" DT
bindings have been established.  These should be replaced with the
"official" device tree from Linux as soon as possible, but that
doesn't always happen in a timely fashion.

Unfortunately what happens quite often is that hardware vendors
provide patches with preliminary bindings from their own Linux fork
before their devices are fully supported in the mainline Linux tree.

What also happens is that the DT bindings in mainline Linux evolve
over time.  Sometimes there is a good reason for that, but in other
cases maintainers seem to be a little bit too eager to "clean things"
up in a way that breaks backwards compatibility.

> At least in Linux there is a policy to not add device tree nodes that do
> not have drivers. Is the same policy not true for U-Boot? At least your
> ./scripts/checkpatch.pl does have the same "check for DT compatible
> documentation" section as Linux. You might consider removing it if you
> want people to not strip the DTs they submit to U-Boot.

There is no such policy in U-Boot.  And even in Linux the devicetree
maintainers seem to be willing to accept DT bindings and nodes for
things that have a driver elsewhere.

> And why do we even maintain the device tree bindings in Linux at
> all?  It seems rather counter-productive for both ends to do that,
> if it is expected that the kernel works with DT blobs provided by
> third parties too, and if all third parties need to resync with it
> (there are other boot loaders too beyond U-Boot, and other kernels
> beyond Linux). Somehow it doesn't feel right for the reference to be
> the Linux kernel. Maybe this is something that needs to be brought
> up with higher-level Linux maintainers.

IMHO it would be a good idea if the device trees would be maintained
in a separate source tree from the Linux kernel.

> I have no problem at all with structuring the device tree in the same
> way in U-Boot as in Linux, as long as that proves to not be a foolish
> endeavor.

As I said above, U-Boot should simply use a verbatim copy of the Linux
device tree.

Re: incompatible device trees between u-boot and linux

[Rob Herring]

On Wed, Aug 25, 2021 at 10:12 AM Vladimir Oltean <olteanv@gmail.com> wrote:
>
> On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I noticed that there is a fallback to the u-boot device tree for linux
> > > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> > > > > is working fine for imx devices, for example, where you can just boot a
> > > > > stock installer iso via EFI. It will just work and it is quite a nice
> > > > > feature as a fallback.
> > > > >
> > > > > Now for the layerscape architecture, the ls1028a in my case, things are
> > > > > more difficult because the bindings differ between u-boot and linux - one
> > > > > which comes to mind is DSA and ethernet.
> > > > >
> > > > > Which begs the general question, is it encouraged to have both bindings
> > > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> > > > > is no real review or alignment but just added as needed, which is ok if
> > > > > they are local to u-boot. But since they are nowadays passed to linux
> > > > > (by default!) I'm not so sure anymore.
> > > > >
> > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> > > > > they should (could?) be shared between linux and u-boot.
> > > > >
> > > > > -michael
> > > > >
> > > > > [1]
> > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> > > >
> > > > The U-Boot device tree is supposed to be able to be passed on to Linux
> > > > and Just Work.  The bindings are not supposed to be different between
> > > > the two (except for when we take the binding while it's being hashed out
> > > > upstream BUT THEN RESYNCED).
> > >
> > > You might need to spell that out a bit clearer.
> > >
> > > You are saying that both U-Boot and Linux are allowed to have their own
> > > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> > > for Linux), as long as the device tree files themselves are in sync, and
> > > the subset of the device tree blob understood by Linux (i.e. the U-Boot
> > > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
> >
> > I don't know what about the Linux example makes it Linux specific.  But
> > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> > by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> > logic (as much as it can be used is that device trees are device trees
>         ^
>         I don't think this parenthesis ever closes...
>
> > and describe the hardware and developers don't need to write a device
> > tree for Linux and a device tree for U-Boot and a device tree for
> > FreeBSD and ...  So yes, you're supposed to use the device tree for a
>                 ^
>                 so I never get the answer to "the whole reason is...".
>
> > platform and it works here and there and every where.
>
> The fact that only Linux uses it makes it Linux specific.
>
> > > To expand even further on that, it means we should put 'managed = "in-band-status"'
> > > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> > > does not use phylink?
> >
> > We should be able to drop in the device trees from Linux and use them.
> > Custodians should be re-syncing them periodically.  Some are, even.
>
> Are you ready to take up device tree bindings for PTP timers, PCIe root
> complex event collectors, cascaded interrupt controllers, things which
> U-Boot will never ever need to support?
>
> At least in Linux there is a policy to not add device tree nodes that do
> not have drivers.

That is not the policy. The policy is DT nodes must have binding
(schema) documentation and the binding should be complete as possible
(not what some driver currently uses). However, for complex bindings,
it might be difficult to write the binding in absence of a driver.

> Is the same policy not true for U-Boot? At least your
> ./scripts/checkpatch.pl does have the same "check for DT compatible
> documentation" section as Linux. You might consider removing it if you
> want people to not strip the DTs they submit to U-Boot.

That hacky checkpatch.pl check is going away. While somewhat
effective, it will take 'the' or any other word in a binding doc as a
valid compatible. With the schemas now, we have an exact list of what
compatibles are documented and that is checked on dts files. Though
anything with a free form text binding will be reported as
undocumented, but I view that as a todo list.

> And why do we even maintain the device tree bindings in Linux at all?
> It seems rather counter-productive for both ends to do that, if it is
> expected that the kernel works with DT blobs provided by third parties too,
> and if all third parties need to resync with it (there are other boot
> loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
> doesn't feel right for the reference to be the Linux kernel. Maybe this
> is something that needs to be brought up with higher-level Linux maintainers.

This has been discussed to death, but the short answer is convenience.
Where does the largest collection of h/w support and knowledge live?
The Linux kernel. By far. u-boot is probably second, but there's lots
of h/w on platforms u-boot doesn't care about. We could move portions
or all of DT stuff out, but then who is going to review it and check
whether bindings are accepted before a driver or dts uses them? I'm
sure some kernel or u-boot subsystem maintainers might sign up
initially, but are they really going to stay around? In the kernel,
they have to pay attention (to varying levels).

Rob

Re: incompatible device trees between u-boot and linux

[Sean Anderson]

On 8/31/21 9:35 AM, Rob Herring wrote:
> On Wed, Aug 25, 2021 at 10:12 AM Vladimir Oltean <olteanv@gmail.com> wrote:
>>
>> On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
>>> On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
>>>> On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
>>>>> On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I noticed that there is a fallback to the u-boot device tree for linux
>>>>>> (esp. EFI boot) if no other device tree was found, see [1]. It seems this
>>>>>> is working fine for imx devices, for example, where you can just boot a
>>>>>> stock installer iso via EFI. It will just work and it is quite a nice
>>>>>> feature as a fallback.
>>>>>>
>>>>>> Now for the layerscape architecture, the ls1028a in my case, things are
>>>>>> more difficult because the bindings differ between u-boot and linux - one
>>>>>> which comes to mind is DSA and ethernet.
>>>>>>
>>>>>> Which begs the general question, is it encouraged to have both bindings
>>>>>> diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
>>>>>> is no real review or alignment but just added as needed, which is ok if
>>>>>> they are local to u-boot. But since they are nowadays passed to linux
>>>>>> (by default!) I'm not so sure anymore.
>>>>>>
>>>>>> OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
>>>>>> they should (could?) be shared between linux and u-boot.
>>>>>>
>>>>>> -michael
>>>>>>
>>>>>> [1]
>>>>>> https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
>>>>>
>>>>> The U-Boot device tree is supposed to be able to be passed on to Linux
>>>>> and Just Work.  The bindings are not supposed to be different between
>>>>> the two (except for when we take the binding while it's being hashed out
>>>>> upstream BUT THEN RESYNCED).
>>>>
>>>> You might need to spell that out a bit clearer.
>>>>
>>>> You are saying that both U-Boot and Linux are allowed to have their own
>>>> custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
>>>> for Linux), as long as the device tree files themselves are in sync, and
>>>> the subset of the device tree blob understood by Linux (i.e. the U-Boot
>>>> blob sans the U-Boot specifics) is compatible with the Linux DT blob?
>>>
>>> I don't know what about the Linux example makes it Linux specific.  But
>>> yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
>>> by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
>>> logic (as much as it can be used is that device trees are device trees
>>          ^
>>          I don't think this parenthesis ever closes...
>>
>>> and describe the hardware and developers don't need to write a device
>>> tree for Linux and a device tree for U-Boot and a device tree for
>>> FreeBSD and ...  So yes, you're supposed to use the device tree for a
>>                  ^
>>                  so I never get the answer to "the whole reason is...".
>>
>>> platform and it works here and there and every where.
>>
>> The fact that only Linux uses it makes it Linux specific.
>>
>>>> To expand even further on that, it means we should put 'managed = "in-band-status"'
>>>> in U-Boot, which is a Linux phylink device tree property, even if U-Boot
>>>> does not use phylink?
>>>
>>> We should be able to drop in the device trees from Linux and use them.
>>> Custodians should be re-syncing them periodically.  Some are, even.
>>
>> Are you ready to take up device tree bindings for PTP timers, PCIe root
>> complex event collectors, cascaded interrupt controllers, things which
>> U-Boot will never ever need to support?
>>
>> At least in Linux there is a policy to not add device tree nodes that do
>> not have drivers.
> 
> That is not the policy. The policy is DT nodes must have binding
> (schema) documentation and the binding should be complete as possible
> (not what some driver currently uses). However, for complex bindings,
> it might be difficult to write the binding in absence of a driver.

It is effective policy for some arches...

When the K210 patches were submitted, any bindings for devices without
enabled drivers were requested to be (and subsequently were) removed,
even though many of those bindings were based off of existing
documentation. This is the primary cause of divergence between the Linux
and U-Boot devicetrees for this platform. It is also the main reason
that I have not bothered putting together a sync patch.

--Sean

Re: incompatible device trees between u-boot and linux

[Rob Herring]

On Tue, Aug 31, 2021 at 9:21 AM Sean Anderson <seanga2@gmail.com> wrote:
>
> On 8/31/21 9:35 AM, Rob Herring wrote:
> > On Wed, Aug 25, 2021 at 10:12 AM Vladimir Oltean <olteanv@gmail.com> wrote:
> >>
> >> On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
> >>> On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
> >>>> On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
> >>>>> On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I noticed that there is a fallback to the u-boot device tree for linux
> >>>>>> (esp. EFI boot) if no other device tree was found, see [1]. It seems this
> >>>>>> is working fine for imx devices, for example, where you can just boot a
> >>>>>> stock installer iso via EFI. It will just work and it is quite a nice
> >>>>>> feature as a fallback.
> >>>>>>
> >>>>>> Now for the layerscape architecture, the ls1028a in my case, things are
> >>>>>> more difficult because the bindings differ between u-boot and linux - one
> >>>>>> which comes to mind is DSA and ethernet.
> >>>>>>
> >>>>>> Which begs the general question, is it encouraged to have both bindings
> >>>>>> diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
> >>>>>> is no real review or alignment but just added as needed, which is ok if
> >>>>>> they are local to u-boot. But since they are nowadays passed to linux
> >>>>>> (by default!) I'm not so sure anymore.
> >>>>>>
> >>>>>> OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
> >>>>>> they should (could?) be shared between linux and u-boot.
> >>>>>>
> >>>>>> -michael
> >>>>>>
> >>>>>> [1]
> >>>>>> https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
> >>>>>
> >>>>> The U-Boot device tree is supposed to be able to be passed on to Linux
> >>>>> and Just Work.  The bindings are not supposed to be different between
> >>>>> the two (except for when we take the binding while it's being hashed out
> >>>>> upstream BUT THEN RESYNCED).
> >>>>
> >>>> You might need to spell that out a bit clearer.
> >>>>
> >>>> You are saying that both U-Boot and Linux are allowed to have their own
> >>>> custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
> >>>> for Linux), as long as the device tree files themselves are in sync, and
> >>>> the subset of the device tree blob understood by Linux (i.e. the U-Boot
> >>>> blob sans the U-Boot specifics) is compatible with the Linux DT blob?
> >>>
> >>> I don't know what about the Linux example makes it Linux specific.  But
> >>> yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
> >>> by Linux.  The whole reason we have the -u-boot.dtsi automatic drop-in
> >>> logic (as much as it can be used is that device trees are device trees
> >>          ^
> >>          I don't think this parenthesis ever closes...
> >>
> >>> and describe the hardware and developers don't need to write a device
> >>> tree for Linux and a device tree for U-Boot and a device tree for
> >>> FreeBSD and ...  So yes, you're supposed to use the device tree for a
> >>                  ^
> >>                  so I never get the answer to "the whole reason is...".
> >>
> >>> platform and it works here and there and every where.
> >>
> >> The fact that only Linux uses it makes it Linux specific.
> >>
> >>>> To expand even further on that, it means we should put 'managed = "in-band-status"'
> >>>> in U-Boot, which is a Linux phylink device tree property, even if U-Boot
> >>>> does not use phylink?
> >>>
> >>> We should be able to drop in the device trees from Linux and use them.
> >>> Custodians should be re-syncing them periodically.  Some are, even.
> >>
> >> Are you ready to take up device tree bindings for PTP timers, PCIe root
> >> complex event collectors, cascaded interrupt controllers, things which
> >> U-Boot will never ever need to support?
> >>
> >> At least in Linux there is a policy to not add device tree nodes that do
> >> not have drivers.
> >
> > That is not the policy. The policy is DT nodes must have binding
> > (schema) documentation and the binding should be complete as possible
> > (not what some driver currently uses). However, for complex bindings,
> > it might be difficult to write the binding in absence of a driver.
>
> It is effective policy for some arches...
>
> When the K210 patches were submitted, any bindings for devices without
> enabled drivers were requested to be (and subsequently were) removed,
> even though many of those bindings were based off of existing
> documentation. This is the primary cause of divergence between the Linux
> and U-Boot devicetrees for this platform. It is also the main reason
> that I have not bothered putting together a sync patch.

Well, that's a perfect example of why it shouldn't be policy. We need
to make sync'ing easier, not harder.

Rob

Re: incompatible device trees between u-boot and linux

[Michael Walle]

Am 2021-08-25 16:18, schrieb Vladimir Oltean:
> Curious that Michael mentions Ethernet and DSA on LS1028A.

[..]

> I mean, they look pretty similar to me? The biggest difference is that
> the ENETC ECAM is under the /soc node in Linux, but under / in U-Boot,
> as well as some BAR memory areas that are not marked as prefetchable or
> non-prefetchable in the U-Boot device tree. But excuse me, that isn't 
> an
> Ethernet/DSA problem, is it?

Mh, you're right. I had something in mind you've mentioned during the
initial DSA patches for u-boot. I must have remembered incorrectly.

My intention was to have a clear message if the u-boot device tree
should be following the linux one before I'm digging deeper into what
is different between these two and try to sync both.

-michael

incompatible device trees between u-boot and linux

[François Ozog]

Hi,

Trying to respond to a U-Boot digest manually, please accept apologies
as I don't know how to do it properly...

>On Wed, Aug 25, 2021 at 06:43:23PM +0300, Vladimir Oltean wrote:
>> On Wed, Aug 25, 2021 at 11:24:08AM -0400, Tom Rini wrote:
>> > On Wed, Aug 25, 2021 at 06:12:20PM +0300, Vladimir Oltean wrote:
>> > > On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
>> > > > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
>> > > > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
>> > > > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > I noticed that there is a fallback to the u-boot device tree for linux
>> > > > > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
>> > > > > > > is working fine for imx devices, for example, where you can just boot a
>> > > > > > > stock installer iso via EFI. It will just work and it is quite a nice
>> > > > > > > feature as a fallback.
>> > > > > > >
>> > > > > > > Now for the layerscape architecture, the ls1028a in my case, things are
>> > > > > > > more difficult because the bindings differ between u-boot and linux - one
>> > > > > > > which comes to mind is DSA and ethernet.
>> > > > > > >
>> > > > > > > Which begs the general question, is it encouraged to have both bindings
>> > > > > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
>> > > > > > > is no real review or alignment but just added as needed, which is ok if
>> > > > > > > they are local to u-boot. But since they are nowadays passed to linux
>> > > > > > > (by default!) I'm not so sure anymore.
>> > > > > > >
>> > > > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
>> > > > > > > they should (could?) be shared between linux and u-boot.
>> > > > > > >
>> > > > > > > -michael
>> > > > > > >
>> > > > > > > [1]
>> > > > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
>> > > > > >
>> > > > > > The U-Boot device tree is supposed to be able to be passed on to Linux
>> > > > > > and Just Work. The bindings are not supposed to be different between
>> > > > > > the two (except for when we take the binding while it's being hashed out
>> > > > > > upstream BUT THEN RESYNCED).
>> > > > >
>> > > > > You might need to spell that out a bit clearer.
>> > > > >
>> > > > > You are saying that both U-Boot and Linux are allowed to have their own
>> > > > > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
>> > > > > for Linux), as long as the device tree files themselves are in sync, and
>> > > > > the subset of the device tree blob understood by Linux (i.e. the U-Boot
>> > > > > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
>> > > >
>> > > > I don't know what about the Linux example makes it Linux specific. But
>> > > > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
>> > > > by Linux. The whole reason we have the -u-boot.dtsi automatic drop-in
>> > > > logic (as much as it can be used is that device trees are device trees
>> > > ^
>> > > I don't think this parenthesis ever closes...
>> >
>> > Ah, whoops. Should have been "(as much as it can be used)" because it
>> > does get #included instead in some cases, for reasons.
>> >
>> > >
>> > > > and describe the hardware and developers don't need to write a device
>> > > > tree for Linux and a device tree for U-Boot and a device tree for
>> > > > FreeBSD and ... So yes, you're supposed to use the device tree for a
>> > > ^
>> > > so I never get the answer to "the whole reason is...".
>> > >
>> > > > platform and it works here and there and every where.
>> > >
>> > > The fact that only Linux uses it makes it Linux specific.
>> > >
>> > > > > To expand even further on that, it means we should put 'managed = "in-band-status"'
>> > > > > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
>> > > > > does not use phylink?
>> > > >
>> > > > We should be able to drop in the device trees from Linux and use them.
>> > > > Custodians should be re-syncing them periodically. Some are, even.
>> > >
>> > > Are you ready to take up device tree bindings for PTP timers, PCIe root
>> > > complex event collectors, cascaded interrupt controllers, things which
>> > > U-Boot will never ever need to support?
>> > >
>> > > At least in Linux there is a policy to not add device tree nodes that do
>> > > not have drivers. Is the same policy not true for U-Boot? At least your
>> > > ./scripts/checkpatch.pl does have the same "check for DT compatible
>> > > documentation" section as Linux. You might consider removing it if you
>> > > want people to not strip the DTs they submit to U-Boot.
>> > >
>> > > And why do we even maintain the device tree bindings in Linux at all?
>> > > It seems rather counter-productive for both ends to do that, if it is
>> > > expected that the kernel works with DT blobs provided by third parties too,
>> > > and if all third parties need to resync with it (there are other boot
>> > > loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
>> > > doesn't feel right for the reference to be the Linux kernel. Maybe this
>> > > is something that needs to be brought up with higher-level Linux maintainers.
>> > >
>> > > I have no problem at all with structuring the device tree in the same
>> > > way in U-Boot as in Linux, as long as that proves to not be a foolish
>> > > endeavor.
>> >
>> > DT is ABI is hardware description and OS-agnostic has been the rule for
>> > 10+ years. If that's no longer the case, can someone please tell me?
>>
>> So if Michael's board with DT provided by U-Boot doesn't work for some
>> stupid reason like "Linux expects the pcie node to be under /soc", or
>> "Linux wants all PCIe BARs of a RCIEP ECAM to be spelled out in the
>> 'ranges' property, because it's too dumb to detect them itself", or
>> something like that, I've got no argument against that, let's go ahead
>> and resync U-Boot with Linux.
>>
>> But "DT is ABI is hardware description" is a pretty vague truism that
>> does not actually help here.
>
>I'm saying that because it's what's been said for what feels like 10+
>years. I don't want to think how many countless hours have been spent
>on that point at conferences over the years. It's not even a Linux
>thing. I would swear you can (or could, unless it got broken) take the
>same DTB for some platforms and boot Linux or FreeBSD or some other BSD
>or maybe even VxWorks and it works.
>
>
I cannot agree more.
For historical reasons in the embedded market, the product maker was
the board maker, the distro producer and the firmware producer. That
led to a situation where the Linux kernel developper was stating the
hardware description and that led to...problems. The best analogy I
have to describe how wrong this is: As a French driver, my Device Tree
description about the car is that the steering wheel is on the left
hand side of the car; when entering an English car, I just complain
that the wheel is not where it should be.
Now the value chain is going more "mature" and your may end up booting
a Fedora IoT edition on a bunch of platforms that have been designed
to boot any distro; and the Trusted Applications may come from the OEM
and not the board maker.

Organizing the software value chain along side of the hardware value
chain (Silicon provider, SoM provider, Carrier provider, Integrator,
Car manufacturer for instance) need DT technology and lifecycle
re-boot. (softwsystem up firmware such as Trusted Firmware, boot
firmware such as U-Boot and operating systems)

Some elements of discussion can be found here:
https://linaro.atlassian.net/wiki/spaces/DTE/overview
https://docs.google.com/document/d/1CLkhLRaz_zcCq44DLGmPZQFPbYHOC6nzPowaL0XmRk0/edit?usp=sharing

If we take the https://www.solid-run.com/arm-servers-networking-platforms/honeycomb-workstation/
board as an example to stay on NXP SoCs: what is the ideal way to deal
with device tree should we start from a blank sheet?

A key element are SerDes pins: from a SoC perspective they can be made
PCI lanes or Ethernet lanes. This is chosen by the board vendor. An
operating system should never manage the SerDes, it will see either
the PCI lanes or the Ethernet port (not exactly true yet because of
DPAA2 - but let's keep it high level). It is the responsibility of the
firmware to actually match the SoC configuration with board layout
(PCI slot, PCI device, Ethernet port).

In my mind, the ideal situation is: The board vendor takes the SoC DT
as input and applies fixups (at design time or runtime) to produce the
board DT. The board DT should be independent from the firmware stack
(Coreboot, TF-A+U-Boot, TF-A + LinuxBoot) and the booted OS: it is
just describing the hardware!!!
Firmware elements can still apply fixes at runtime: TF-A role is to
pass detected DRAM, U-Boot may have a parameter (saved as U-Boot
variable) to define active cores and ensures the DT is fixed up to
match that desire, OP-TEE can expose the amount of Secure DRAM it has
carved out through FF-A API.



>> Will you accept device trees with devices for which a driver will never
>> probe in U-Boot,
>
>Yes, I will absolutely take device trees that have devices we don't need
>in U-Boot since the point is, and many SoC vendors are doing (and the
>ones that aren't are, I am not happy about / with) right now.
>
>> and will you remove the checkpatch warnings about those,
>> to not discourage people from submitting them prior to the actual public
>> review?
>
>With respect to checkpatch.pl, maybe I'm just missing the line in
>question? Or maybe it's a kernel-related warning we need to disable in
>our .checkpatch.conf. But I don't want to side-track over this part.
>
>> If a U-Boot driver will be written for a device that does not have a
>> driver yet in Linux, then the Linux driver will be written but with
>> DT bindings incompatible with what was established in U-Boot, will you
>> wake up the U-Boot developer from the grave and ask them to resync the
>> driver to follow Linux? Or will you accept drivers at all for hardware
>> that is not supported by Linux?
>
>What I've said for years (but yes, I've missed changes, maybe the yaml
>dt binding stuff would help so I could make CI fail or at least require
>manual override?) is that U-Boot will take immature bindings but it's on
>the developer to re-sync once the bindings are fully reviewed. This is
>to help with the chicken-and-egg problem. But old bindings are not
>intended to be supported, once it's finalized. That is part of the
>bargain.
>
>> I also think there are various degrees of what it means "to work" with a
>> device tree provided directly by U-Boot. Distros like Arch Linux ARM
>> have a package for device tree blobs, and it is expected that these are
>> updated by the distro, and U-Boot just loads them. As Michael points out,
>> the DT provided by U-Boot is just a fallback. The OS should boot to
>> prompt and have a network connection to set itself up properly again.
>> But we need to draw a harder line on what we _actually_ desire to work
>> beyond that.
>
>Every distribution has a package for device tree binaries, because
>that's how you get a device tree on the still vast majority of platforms
>that don't ship one in-flash (RPis being the modern exception) and to my
>knowledge none of them are happy about having to build and pass and make
>sure the right one is used on a given board at boot. So yes, U-Boot
>being able to pass a device tree on to the next stage is one of those
>things to help everything Just Work and be boring.
>
>--
>Tom