[PATCH 0/9] arm64: dts: rockchip: Initial Toybrick TB-RK1808M0 support

[PATCH 0/9] arm64: dts: rockchip: Initial Toybrick TB-RK1808M0 support

[Andreas Färber]

Hello Heiko et al.,

It seems linux-rockchip list only saw two RK1808 patches for ASoC in 2019.
Following up on a SUSE Hackweek 20 project of mine, here's some patches that
allow me to start booting into the TB-RK1808M0 mPCIe card's eMMC.

Tested using its USB adapter, which allows to connect a serial cable and a
USB storage device that I load kernel+dtb from. It has a reset button, and
Ctrl+C allows to enter a U-Boot prompt (without EBBR/UEFI support though).

Patches are based on the shipping toybrick.dtb file.
http://t.rock-chips.com/en/wiki.php?mod=view&id=110 gives instructions for
compiling sources, but no source download or link is actually provided.

I encountered a hang: earlycon revealed it being related to KVM and vGIC.
Disabling KVM in Kconfig works around it, as does removing the vGIC irq in DT.
I've already tried low and high for the vGIC interrupt, so no clue what might
cause it. On an mPCIe card with 1 GiB of RAM I figured KVM is not going to be
a major use case, so if we find no other solution, we could just delete the
interrupts property in its .dts, as demonstrated here.

The TB-96AIoT 96Boards SoM would be another RK1808 platform someone might
test these patches on. For the TB-RK1808S0 USB stick there's at least no
documented way to access a serial console.

Have a lot of fun!

Cheers,
Andreas

Cc: devicetree@vger.kernel.org
Cc: Rob Herring <robh+dt@kernel.org>
Cc: Marc Zyngier <maz@kernel.org>

Andreas Färber (9):
  dt-bindings: arm: rockchip: Add Rockchip RK1808 and TB-RK1808M0
  dt-bindings: serial: snps-dw-apb-uart: Add Rockchip RK1808
  arm64: dts: rockchip: Prepare Rockchip RK1808
  arm64: dts: rockchip: Add Rockchip TB-RK1808M0
  arm64: dts: rockchip: rk1808k-toybrick-m0: Suppress vGIC interrupt
  dt-bindings: mmc: rockchip-dw-mshc: Add Rockchip RK1808
  arm64: dts: rockchip: rk1808: Prepare eMMC node
  arm64: dts: rockchip: rk1808k-toybrick-m0: Enable eMMC
  arm64: dts: rockchip: rk1808: Add CPU operating points

 .../devicetree/bindings/arm/rockchip.yaml     |   5 +
 .../bindings/mmc/rockchip-dw-mshc.yaml        |   1 +
 .../bindings/serial/snps-dw-apb-uart.yaml     |   1 +
 arch/arm64/boot/dts/rockchip/Makefile         |   1 +
 arch/arm64/boot/dts/rockchip/rk1808.dtsi      | 276 ++++++++++++++++++
 .../boot/dts/rockchip/rk1808k-toybrick-m0.dts |  97 ++++++
 6 files changed, 381 insertions(+)
 create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi
 create mode 100644 arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts

-- 
2.31.1

[PATCH 1/9] dt-bindings: arm: rockchip: Add Rockchip RK1808 and TB-RK1808M0

[Andreas Färber]

Define SoC and board compatibles for RK1808 SoC and Toybrick mPCIe card.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 Documentation/devicetree/bindings/arm/rockchip.yaml | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/devicetree/bindings/arm/rockchip.yaml b/Documentation/devicetree/bindings/arm/rockchip.yaml
index 4a6f772c1043..829f67f659d1 100644
--- a/Documentation/devicetree/bindings/arm/rockchip.yaml
+++ b/Documentation/devicetree/bindings/arm/rockchip.yaml
@@ -580,6 +580,11 @@ properties:
           - const: rockchip,rv1108-evb
           - const: rockchip,rv1108
 
+      - description: Rockchip Toybrick TB-RK1808M0 mPCIe card
+        items:
+          - const: rockchip,tb-rk1808m0
+          - const: rockchip,rk1808
+
       - description: Theobroma Systems RK3368-uQ7 with Haikou baseboard
         items:
           - const: tsd,rk3368-lion-haikou
-- 
2.31.1

Re: [PATCH 1/9] dt-bindings: arm: rockchip: Add Rockchip RK1808 and TB-RK1808M0

[Rob Herring]

On Mon, 17 May 2021 01:05:43 +0200, Andreas Färber wrote:
> Define SoC and board compatibles for RK1808 SoC and Toybrick mPCIe card.
> 
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  Documentation/devicetree/bindings/arm/rockchip.yaml | 5 +++++
>  1 file changed, 5 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

[PATCH 2/9] dt-bindings: serial: snps-dw-apb-uart: Add Rockchip RK1808

[Andreas Färber]

Add a Rockchip RK1808 compatible.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml b/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml
index 87ef1e218152..68545e421bbe 100644
--- a/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml
+++ b/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml
@@ -23,6 +23,7 @@ properties:
       - items:
           - enum:
               - rockchip,px30-uart
+              - rockchip,rk1808-uart
               - rockchip,rk3036-uart
               - rockchip,rk3066-uart
               - rockchip,rk3188-uart
-- 
2.31.1

Re: [PATCH 2/9] dt-bindings: serial: snps-dw-apb-uart: Add Rockchip RK1808

[Rob Herring]

On Mon, 17 May 2021 01:05:44 +0200, Andreas Färber wrote:
> Add a Rockchip RK1808 compatible.
> 
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml | 1 +
>  1 file changed, 1 insertion(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

[PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Andreas Färber]

Add an initial Device Tree for Rockchip RK1808 SoC.
Based on shipping TB-RK1808M0 DTB.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 arch/arm64/boot/dts/rockchip/rk1808.dtsi | 203 +++++++++++++++++++++++
 1 file changed, 203 insertions(+)
 create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi

diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
new file mode 100644
index 000000000000..af2b51afda7d
--- /dev/null
+++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright (c) 2021 Andreas Färber
+ */
+
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/interrupt-controller/irq.h>
+
+/ {
+	compatible = "rockchip,rk1808";
+	interrupt-parent = <&gic>;
+	#address-cells = <1>;
+	#size-cells = <1>;
+
+	aliases {
+		serial0 = &uart0;
+		serial1 = &uart1;
+		serial2 = &uart2;
+		serial3 = &uart3;
+		serial4 = &uart4;
+		serial5 = &uart5;
+		serial6 = &uart6;
+		serial7 = &uart7;
+	};
+
+	cpus {
+		#address-cells = <2>;
+		#size-cells = <0>;
+
+		cpu0: cpu@0 {
+			device_type = "cpu";
+			compatible = "arm,cortex-a35";
+			reg = <0x0 0x0>;
+			enable-method = "psci";
+			cpu-idle-states = <&CPU_SLEEP>;
+		};
+
+		cpu1: cpu@1 {
+			device_type = "cpu";
+			compatible = "arm,cortex-a35";
+			reg = <0x0 0x1>;
+			enable-method = "psci";
+			cpu-idle-states = <&CPU_SLEEP>;
+		};
+
+		idle-states {
+			entry-method = "psci";
+
+			CPU_SLEEP: cpu-sleep {
+				compatible = "arm,idle-state";
+				local-timer-stop;
+				arm,psci-suspend-param = <0x10000>;
+				entry-latency-us = <120>;
+				exit-latency-us = <250>;
+				min-residency-us = <900>;
+			};
+		};
+	};
+
+	arm-pmu {
+		compatible = "arm,cortex-a35-pmu";
+		interrupts = <GIC_SPI 228 IRQ_TYPE_LEVEL_HIGH>,
+			     <GIC_SPI 229 IRQ_TYPE_LEVEL_HIGH>;
+		interrupt-affinity = <&cpu0>, <&cpu1>;
+	};
+
+	timer {
+		compatible = "arm,armv8-timer";
+		interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_PPI 10 IRQ_TYPE_LEVEL_LOW>;
+		arm,no-tick-in-suspend;
+	};
+
+	xin24m: xin24m {
+		compatible = "fixed-clock";
+		clock-frequency = <24000000>;
+		#clock-cells = <0>;
+		clock-output-names = "xin24m";
+	};
+
+	firmware {
+		psci {
+			compatible = "arm,psci-1.0";
+			method = "smc";
+		};
+
+		tee {
+			compatible = "linaro,optee-tz";
+			method = "smc";
+		};
+	};
+
+	soc {
+		compatible = "simple-bus";
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		system_sram: sram@fec00000 {
+			compatible = "mmio-sram";
+			reg = <0xfec00000 0x200000>;
+			#address-cells = <1>;
+			#size-cells = <1>;
+			ranges = <0x0 0xfec00000 0x200000>;
+		};
+
+		gic: interrupt-controller@ff100000 {
+			compatible = "arm,gic-v3";
+			reg = <0xff100000 0x10000>, /* GICD */
+			      <0xff140000 0xc0000>, /* GICR */
+			      <0xff300000 0x10000>, /* GICC */
+			      <0xff310000 0x10000>, /* GICH */
+			      <0xff320000 0x10000>; /* GICV */
+			interrupt-controller;
+			#interrupt-cells = <3>;
+			interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
+			#address-cells = <1>;
+			#size-cells = <1>;
+			ranges;
+
+			gic_its: msi-controller@ff120000 {
+				compatible = "arm,gic-v3-its";
+				reg = <0xff120000 0x20000>;
+				msi-controller;
+				#msi-cells = <1>;
+			};
+		};
+
+		uart0: serial@ff430000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff430000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart1: serial@ff540000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff540000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart2: serial@ff550000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff550000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart3: serial@ff560000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff560000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart4: serial@ff570000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff570000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart5: serial@ff5a0000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff5a0000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart6: serial@ff5b0000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff5b0000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		uart7: serial@ff5c0000 {
+			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
+			reg = <0xff5c0000 0x100>;
+			reg-shift = <2>;
+			reg-io-width = <4>;
+			interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+	};
+};
-- 
2.31.1

Re: [PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Johan Jonker]

Hi Andreas,

Send the complete serie to all maintainers and mail lists.

===
Heiko's sort rules:

compatible
reg
interrupts
[alphabetical]
status [if needed]

===
My incomplete list:

For nodes:
If exists on top: model, compatible and chosen.
Sort things without reg alphabetical first,
then sort the rest by reg address.

Inside nodes:
If exists on top: compatible, reg and interrupts.
In alphabetical order the required properties.
Then in alphabetical order the other properties.
And as last things that start with '#' in alphabetical order.
Add status below all other properties for soc internal components with
any board-specifics.
Keep an empty line between properties and nodes.

Exceptions:
Sort pinctrl-0 above pinctrl-names, so it stays in line with clock-names
and dma-names.
Sort simple-audio-card,name above other simple-audio-card properties.
Sort regulator-name above other regulator properties.
Sort regulator-min-microvolt above regulator-max-microvolt.

===

Fix complete dtsi for property sort order!
Add more drivers. (cru, pinctrl)

Johan

On 5/17/21 1:05 AM, Andreas Färber wrote:
> Add an initial Device Tree for Rockchip RK1808 SoC.
> Based on shipping TB-RK1808M0 DTB.
> 
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  arch/arm64/boot/dts/rockchip/rk1808.dtsi | 203 +++++++++++++++++++++++
>  1 file changed, 203 insertions(+)
>  create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> new file mode 100644
> index 000000000000..af2b51afda7d
> --- /dev/null
> +++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> @@ -0,0 +1,203 @@
> +// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
> +/*
> + * Copyright (c) 2021 Andreas Färber
> + */
> +

#include <dt-bindings/clock/rk1808-cru.h>

> +#include <dt-bindings/interrupt-controller/arm-gic.h>
> +#include <dt-bindings/interrupt-controller/irq.h>
> +
> +/ {
> +	compatible = "rockchip,rk1808";
> +	interrupt-parent = <&gic>;

> +	#address-cells = <1>;
> +	#size-cells = <1>;


	#address-cells = <2>;
	#size-cells = <2>;

64 bit ??

> +
> +	aliases {
> +		serial0 = &uart0;
> +		serial1 = &uart1;
> +		serial2 = &uart2;
> +		serial3 = &uart3;
> +		serial4 = &uart4;
> +		serial5 = &uart5;
> +		serial6 = &uart6;
> +		serial7 = &uart7;
> +	};
> +
> +	cpus {
> +		#address-cells = <2>;
> +		#size-cells = <0>;
> +
> +		cpu0: cpu@0 {
> +			device_type = "cpu";
> +			compatible = "arm,cortex-a35";
> +			reg = <0x0 0x0>;
> +			enable-method = "psci";
> +			cpu-idle-states = <&CPU_SLEEP>;
> +		};
> +
> +		cpu1: cpu@1 {
> +			device_type = "cpu";
> +			compatible = "arm,cortex-a35";
> +			reg = <0x0 0x1>;
> +			enable-method = "psci";
> +			cpu-idle-states = <&CPU_SLEEP>;
> +		};
> +
> +		idle-states {
> +			entry-method = "psci";
> +
> +			CPU_SLEEP: cpu-sleep {
> +				compatible = "arm,idle-state";
> +				local-timer-stop;
> +				arm,psci-suspend-param = <0x10000>;
> +				entry-latency-us = <120>;
> +				exit-latency-us = <250>;
> +				min-residency-us = <900>;
> +			};
> +		};
> +	};
> +

> +	arm-pmu {

sort node names

> +		compatible = "arm,cortex-a35-pmu";
> +		interrupts = <GIC_SPI 228 IRQ_TYPE_LEVEL_HIGH>,
> +			     <GIC_SPI 229 IRQ_TYPE_LEVEL_HIGH>;
> +		interrupt-affinity = <&cpu0>, <&cpu1>;
> +	};
> +
> +	timer {
> +		compatible = "arm,armv8-timer";
> +		interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_PPI 10 IRQ_TYPE_LEVEL_LOW>;
> +		arm,no-tick-in-suspend;
> +	};
> +
> +	xin24m: xin24m {
> +		compatible = "fixed-clock";
> +		clock-frequency = <24000000>;
> +		#clock-cells = <0>;
> +		clock-output-names = "xin24m";
> +	};
> +

> +	firmware {

sort node names

> +		psci {
> +			compatible = "arm,psci-1.0";
> +			method = "smc";
> +		};
> +
> +		tee {
> +			compatible = "linaro,optee-tz";
> +			method = "smc";
> +		};
> +	};
> +

> +	soc {
> +		compatible = "simple-bus";
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;

Remove, use 64bit reg.
See:

https://github.com/rockchip-linux/kernel/blob/develop-4.19/arch/arm64/boot/dts/rockchip/rk1808.dtsi

> +
> +		system_sram: sram@fec00000 {
> +			compatible = "mmio-sram";
> +			reg = <0xfec00000 0x200000>;
> +			#address-cells = <1>;
> +			#size-cells = <1>;
> +			ranges = <0x0 0xfec00000 0x200000>;
> +		};
> +
> +		gic: interrupt-controller@ff100000 {
> +			compatible = "arm,gic-v3";
> +			reg = <0xff100000 0x10000>, /* GICD */
> +			      <0xff140000 0xc0000>, /* GICR */
> +			      <0xff300000 0x10000>, /* GICC */
> +			      <0xff310000 0x10000>, /* GICH */
> +			      <0xff320000 0x10000>; /* GICV */
> +			interrupt-controller;
> +			#interrupt-cells = <3>;
> +			interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
> +			#address-cells = <1>;
> +			#size-cells = <1>;
> +			ranges;
> +
> +			gic_its: msi-controller@ff120000 {
> +				compatible = "arm,gic-v3-its";
> +				reg = <0xff120000 0x20000>;
> +				msi-controller;
> +				#msi-cells = <1>;
> +			};
> +		};
> +

> +		uart0: serial@ff430000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff430000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};

From manufacturer tree:

	uart0: serial@ff430000 {
		compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
		reg = <0x0 0xff430000 0x0 0x100>;
		interrupts = <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>;
		clocks = <&cru SCLK_UART0_PMU>, <&cru PCLK_UART0_PMU>;
		clock-names = "baudclk", "apb_pclk";
		dmas = <&dmac 0>, <&dmac 1>;
		pinctrl-0 = <&uart0_xfer &uart0_cts &uart0_rts>;
		pinctrl-names = "default";
		reg-io-width = <4>;
		reg-shift = <2>;
		status = "disabled";
	};

Sort all uart nodes.

Does this work without SCLK_UART0_PMU, PCLK_UART0_PMU and pinctrl??
Add clk-rk1808.c rk1808-cru.h

In mainline pinctrl and gpio are WIP and split elsewhere now.

pinctrl: rockchip: add support for rk1808 SoCs

https://github.com/rockchip-linux/kernel/commit/b2828bc4417c9669fdaca3b8ef392f41850c86e7

> +
> +		uart1: serial@ff540000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff540000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +
> +		uart2: serial@ff550000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff550000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +
> +		uart3: serial@ff560000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff560000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +
> +		uart4: serial@ff570000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff570000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +
> +		uart5: serial@ff5a0000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff5a0000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +
> +		uart6: serial@ff5b0000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff5b0000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +
> +		uart7: serial@ff5c0000 {
> +			compatible = "rockchip,rk1808-uart", "snps,dw-apb-uart";
> +			reg = <0xff5c0000 0x100>;
> +			reg-shift = <2>;
> +			reg-io-width = <4>;
> +			interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
> +			status = "disabled";
> +		};
> +	};
> +};
> 

Re: [PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Andreas Färber]

Hi Johan,

On 17.05.21 03:29, Johan Jonker wrote:
> Send the complete serie to all maintainers and mail lists.

That's unhelpful - all patches went to linux-rockchip, LAKML and LKML,
and get_maintainers.pl was used. The cover letter was explicitly copied
to DTML as get_maintainers.pl doesn't catch it. You'll find them here:

https://lore.kernel.org/linux-rockchip/20210516230551.12469-1-afaerber@suse.de/

Which mailing list or maintainer do you think all should've gone to in
addition? You're not listed anywhere in linux-next MAINTAINERS. If you
want to be CC'ed, you can nicely ask me to for a v2 (and explaining why
would help for the next series), or you can send patches against
MAINTAINERS yourself.

Copying all maintainers and lists would likely not be appreciated by the
colleagues and may get mails flagged as spam.

> ===
> Heiko's sort rules:
> 
> compatible
> reg
> interrupts
> [alphabetical]
> status [if needed]
> 
> ===
> My incomplete list:
> 
> For nodes:
> If exists on top: model, compatible and chosen.
> Sort things without reg alphabetical first,
> then sort the rest by reg address.
> 
> Inside nodes:
> If exists on top: compatible, reg and interrupts.
> In alphabetical order the required properties.
> Then in alphabetical order the other properties.
> And as last things that start with '#' in alphabetical order.
> Add status below all other properties for soc internal components with
> any board-specifics.
> Keep an empty line between properties and nodes.
> 
> Exceptions:
> Sort pinctrl-0 above pinctrl-names, so it stays in line with clock-names
> and dma-names.
> Sort simple-audio-card,name above other simple-audio-card properties.
> Sort regulator-name above other regulator properties.
> Sort regulator-min-microvolt above regulator-max-microvolt.
> 
> ===


Here's a rule for you: No top-posting on kernel lists.


> 
> Fix complete dtsi for property sort order!
> Add more drivers. (cru, pinctrl)

-ETONE! I don't work for Rockchip, so don't command me to add drivers
for them that I have no source code for (cover letter) nor complete TRM.
You're welcome that I did this service to the kernel community in my
spare time... If you look up the date of Hackweek 20, you'll find that
it took me two months to get this patchset binding-documented and
cleaned up to this point, so by extrapolation it's unrealistic to expect
much more of me here anytime soon. Rudeness certainly does not motivate.

Heiko knows me - if he has any comments, he should be well capable of
voicing them himself inline. I will not follow nonsensical rules that
diverge from other mainline arm-soc vendors - unless I've missed some
new directive from Rob or arm-soc maintainers that would apply to all.

Thanks for nothing,

Andreas

-- 
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer
HRB 36809 (AG Nürnberg)

Re: [PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Marc Zyngier]

On Mon, 17 May 2021 00:05:45 +0100,
Andreas Färber <afaerber@suse.de> wrote:
> 
> Add an initial Device Tree for Rockchip RK1808 SoC.
> Based on shipping TB-RK1808M0 DTB.
> 
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  arch/arm64/boot/dts/rockchip/rk1808.dtsi | 203 +++++++++++++++++++++++
>  1 file changed, 203 insertions(+)
>  create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> new file mode 100644
> index 000000000000..af2b51afda7d
> --- /dev/null
> +++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> @@ -0,0 +1,203 @@
> +// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
> +/*
> + * Copyright (c) 2021 Andreas Färber
> + */
> +
> +#include <dt-bindings/interrupt-controller/arm-gic.h>
> +#include <dt-bindings/interrupt-controller/irq.h>
> +
> +/ {
> +	compatible = "rockchip,rk1808";
> +	interrupt-parent = <&gic>;
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +
> +	aliases {
> +		serial0 = &uart0;
> +		serial1 = &uart1;
> +		serial2 = &uart2;
> +		serial3 = &uart3;
> +		serial4 = &uart4;
> +		serial5 = &uart5;
> +		serial6 = &uart6;
> +		serial7 = &uart7;
> +	};
> +
> +	cpus {
> +		#address-cells = <2>;
> +		#size-cells = <0>;
> +
> +		cpu0: cpu@0 {
> +			device_type = "cpu";
> +			compatible = "arm,cortex-a35";
> +			reg = <0x0 0x0>;
> +			enable-method = "psci";
> +			cpu-idle-states = <&CPU_SLEEP>;
> +		};
> +
> +		cpu1: cpu@1 {
> +			device_type = "cpu";
> +			compatible = "arm,cortex-a35";
> +			reg = <0x0 0x1>;
> +			enable-method = "psci";
> +			cpu-idle-states = <&CPU_SLEEP>;
> +		};
> +
> +		idle-states {
> +			entry-method = "psci";
> +
> +			CPU_SLEEP: cpu-sleep {
> +				compatible = "arm,idle-state";
> +				local-timer-stop;
> +				arm,psci-suspend-param = <0x10000>;
> +				entry-latency-us = <120>;
> +				exit-latency-us = <250>;
> +				min-residency-us = <900>;
> +			};
> +		};
> +	};
> +
> +	arm-pmu {
> +		compatible = "arm,cortex-a35-pmu";
> +		interrupts = <GIC_SPI 228 IRQ_TYPE_LEVEL_HIGH>,
> +			     <GIC_SPI 229 IRQ_TYPE_LEVEL_HIGH>;
> +		interrupt-affinity = <&cpu0>, <&cpu1>;
> +	};
> +
> +	timer {
> +		compatible = "arm,armv8-timer";
> +		interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_PPI 10 IRQ_TYPE_LEVEL_LOW>;
> +		arm,no-tick-in-suspend;

Another facepalm moment...

> +	};
> +
> +	xin24m: xin24m {
> +		compatible = "fixed-clock";
> +		clock-frequency = <24000000>;
> +		#clock-cells = <0>;
> +		clock-output-names = "xin24m";
> +	};
> +
> +	firmware {
> +		psci {
> +			compatible = "arm,psci-1.0";
> +			method = "smc";
> +		};
> +
> +		tee {
> +			compatible = "linaro,optee-tz";
> +			method = "smc";
> +		};
> +	};
> +
> +	soc {
> +		compatible = "simple-bus";
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		system_sram: sram@fec00000 {
> +			compatible = "mmio-sram";
> +			reg = <0xfec00000 0x200000>;
> +			#address-cells = <1>;
> +			#size-cells = <1>;
> +			ranges = <0x0 0xfec00000 0x200000>;
> +		};
> +
> +		gic: interrupt-controller@ff100000 {
> +			compatible = "arm,gic-v3";
> +			reg = <0xff100000 0x10000>, /* GICD */
> +			      <0xff140000 0xc0000>, /* GICR */

This is obviously wrong. You have two CPUs, and yet describe a range
that spans 6. I guess this is a copy paste from rk3399 again?

> +			      <0xff300000 0x10000>, /* GICC */
> +			      <0xff310000 0x10000>, /* GICH */
> +			      <0xff320000 0x10000>; /* GICV */
> +			interrupt-controller;
> +			#interrupt-cells = <3>;
> +			interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
> +			#address-cells = <1>;
> +			#size-cells = <1>;
> +			ranges;
> +
> +			gic_its: msi-controller@ff120000 {
> +				compatible = "arm,gic-v3-its";
> +				reg = <0xff120000 0x20000>;
> +				msi-controller;
> +				#msi-cells = <1>;
> +			};

What uses the ITS?

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Andreas Färber]

On 17.05.21 11:21, Marc Zyngier wrote:
> On Mon, 17 May 2021 00:05:45 +0100,
> Andreas Färber <afaerber@suse.de> wrote:
>>
>> Add an initial Device Tree for Rockchip RK1808 SoC.
>> Based on shipping TB-RK1808M0 DTB.
>>
>> Signed-off-by: Andreas Färber <afaerber@suse.de>
>> ---
>>  arch/arm64/boot/dts/rockchip/rk1808.dtsi | 203 +++++++++++++++++++++++
>>  1 file changed, 203 insertions(+)
>>  create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
>> new file mode 100644
>> index 000000000000..af2b51afda7d
>> --- /dev/null
>> +++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
[...]
>> +		gic: interrupt-controller@ff100000 {
>> +			compatible = "arm,gic-v3";
>> +			reg = <0xff100000 0x10000>, /* GICD */
>> +			      <0xff140000 0xc0000>, /* GICR */
> 
> This is obviously wrong. You have two CPUs, and yet describe a range
> that spans 6. I guess this is a copy paste from rk3399 again?

Not on my part at least. As indicated, these numbers are what ships in
the DTB on the RK1808 card, as per dtc -I dtb -O dts. Could be a mistake
by Rockchip, of course.

Are you suggesting 0xc0000/6*2 = 0x40000 for two CPUs here?
Works as bad as before - investigation still ongoing with latest next.

As for "obviously": The GICv3 YAML binding has no description for me to
validate those numbers: "GIC Redistributors (GICR), one range per
redistributor region" - says nothing about correlation to number of CPUs
or size per CPU, and the examples are not explaining either: 0x200000
has no number of CPUs associated, and by my calculation 0x800000 for 32
CPUs results in 0x40000 per CPU; but then again the examples also have
GICC etc. at diverging 0x2000 size.

>> +			      <0xff300000 0x10000>, /* GICC */
>> +			      <0xff310000 0x10000>, /* GICH */
>> +			      <0xff320000 0x10000>; /* GICV */
>> +			interrupt-controller;
>> +			#interrupt-cells = <3>;
>> +			interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
>> +			#address-cells = <1>;
>> +			#size-cells = <1>;
>> +			ranges;
>> +
>> +			gic_its: msi-controller@ff120000 {
>> +				compatible = "arm,gic-v3-its";
>> +				reg = <0xff120000 0x20000>;
>> +				msi-controller;
>> +				#msi-cells = <1>;
>> +			};
> 
> What uses the ITS?

DT-wise seemingly only the __symbols__ table (named just "its" there, I
notice), so we could drop (or rename) the label if you prefer.

Regards,
Andreas

-- 
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer
HRB 36809 (AG Nürnberg)

Re: [PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Marc Zyngier]

On Mon, 24 May 2021 14:32:41 +0100,
Andreas Färber <afaerber@suse.de> wrote:
> 
> On 17.05.21 11:21, Marc Zyngier wrote:
> > On Mon, 17 May 2021 00:05:45 +0100,
> > Andreas Färber <afaerber@suse.de> wrote:
> >>
> >> Add an initial Device Tree for Rockchip RK1808 SoC.
> >> Based on shipping TB-RK1808M0 DTB.
> >>
> >> Signed-off-by: Andreas Färber <afaerber@suse.de>
> >> ---
> >>  arch/arm64/boot/dts/rockchip/rk1808.dtsi | 203 +++++++++++++++++++++++
> >>  1 file changed, 203 insertions(+)
> >>  create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi
> >>
> >> diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> >> new file mode 100644
> >> index 000000000000..af2b51afda7d
> >> --- /dev/null
> >> +++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> [...]
> >> +		gic: interrupt-controller@ff100000 {
> >> +			compatible = "arm,gic-v3";
> >> +			reg = <0xff100000 0x10000>, /* GICD */
> >> +			      <0xff140000 0xc0000>, /* GICR */
> > 
> > This is obviously wrong. You have two CPUs, and yet describe a range
> > that spans 6. I guess this is a copy paste from rk3399 again?
> 
> Not on my part at least. As indicated, these numbers are what ships in
> the DTB on the RK1808 card, as per dtc -I dtb -O dts. Could be a mistake
> by Rockchip, of course.
> 
> Are you suggesting 0xc0000/6*2 = 0x40000 for two CPUs here?  Works
> as bad as before - investigation still ongoing with latest next.
> 
> As for "obviously": The GICv3 YAML binding has no description for me to
> validate those numbers: "GIC Redistributors (GICR), one range per
> redistributor region" - says nothing about correlation to number of CPUs
> or size per CPU, and the examples are not explaining either: 0x200000
> has no number of CPUs associated, and by my calculation 0x800000 for 32
> CPUs results in 0x40000 per CPU; but then again the examples also have
> GICC etc. at diverging 0x2000 size.

The GICv3/v4 architecture spec does apply, and you should really have
a look at what these sizes mean. What is the value of copy-pasting
things without understanding it the first place?

> 
> >> +			      <0xff300000 0x10000>, /* GICC */
> >> +			      <0xff310000 0x10000>, /* GICH */
> >> +			      <0xff320000 0x10000>; /* GICV */
> >> +			interrupt-controller;
> >> +			#interrupt-cells = <3>;
> >> +			interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
> >> +			#address-cells = <1>;
> >> +			#size-cells = <1>;
> >> +			ranges;
> >> +
> >> +			gic_its: msi-controller@ff120000 {
> >> +				compatible = "arm,gic-v3-its";
> >> +				reg = <0xff120000 0x20000>;
> >> +				msi-controller;
> >> +				#msi-cells = <1>;
> >> +			};
> > 
> > What uses the ITS?
> 
> DT-wise seemingly only the __symbols__ table (named just "its" there, I
> notice), so we could drop (or rename) the label if you prefer.

No, I am asking *what* uses the ITS. Is it just dangling without any
user? No PCI bus making use of it?

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 3/9] arm64: dts: rockchip: Prepare Rockchip RK1808

[Heiko Stübner]

Am Montag, 24. Mai 2021, 17:21:41 CEST schrieb Marc Zyngier:
> On Mon, 24 May 2021 14:32:41 +0100,
> Andreas Färber <afaerber@suse.de> wrote:
> > 
> > On 17.05.21 11:21, Marc Zyngier wrote:
> > > On Mon, 17 May 2021 00:05:45 +0100,
> > > Andreas Färber <afaerber@suse.de> wrote:
> > >>
> > >> Add an initial Device Tree for Rockchip RK1808 SoC.
> > >> Based on shipping TB-RK1808M0 DTB.
> > >>
> > >> Signed-off-by: Andreas Färber <afaerber@suse.de>
> > >> ---
> > >>  arch/arm64/boot/dts/rockchip/rk1808.dtsi | 203 +++++++++++++++++++++++
> > >>  1 file changed, 203 insertions(+)
> > >>  create mode 100644 arch/arm64/boot/dts/rockchip/rk1808.dtsi
> > >>
> > >> diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> > >> new file mode 100644
> > >> index 000000000000..af2b51afda7d
> > >> --- /dev/null
> > >> +++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
> > [...]
> > >> +		gic: interrupt-controller@ff100000 {
> > >> +			compatible = "arm,gic-v3";
> > >> +			reg = <0xff100000 0x10000>, /* GICD */
> > >> +			      <0xff140000 0xc0000>, /* GICR */
> > > 
> > > This is obviously wrong. You have two CPUs, and yet describe a range
> > > that spans 6. I guess this is a copy paste from rk3399 again?
> > 
> > Not on my part at least. As indicated, these numbers are what ships in
> > the DTB on the RK1808 card, as per dtc -I dtb -O dts. Could be a mistake
> > by Rockchip, of course.
> > 
> > Are you suggesting 0xc0000/6*2 = 0x40000 for two CPUs here?  Works
> > as bad as before - investigation still ongoing with latest next.
> > 
> > As for "obviously": The GICv3 YAML binding has no description for me to
> > validate those numbers: "GIC Redistributors (GICR), one range per
> > redistributor region" - says nothing about correlation to number of CPUs
> > or size per CPU, and the examples are not explaining either: 0x200000
> > has no number of CPUs associated, and by my calculation 0x800000 for 32
> > CPUs results in 0x40000 per CPU; but then again the examples also have
> > GICC etc. at diverging 0x2000 size.
> 
> The GICv3/v4 architecture spec does apply, and you should really have
> a look at what these sizes mean. What is the value of copy-pasting
> things without understanding it the first place?
> 
> > 
> > >> +			      <0xff300000 0x10000>, /* GICC */
> > >> +			      <0xff310000 0x10000>, /* GICH */
> > >> +			      <0xff320000 0x10000>; /* GICV */
> > >> +			interrupt-controller;
> > >> +			#interrupt-cells = <3>;
> > >> +			interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
> > >> +			#address-cells = <1>;
> > >> +			#size-cells = <1>;
> > >> +			ranges;
> > >> +
> > >> +			gic_its: msi-controller@ff120000 {
> > >> +				compatible = "arm,gic-v3-its";
> > >> +				reg = <0xff120000 0x20000>;
> > >> +				msi-controller;
> > >> +				#msi-cells = <1>;
> > >> +			};
> > > 
> > > What uses the ITS?
> > 
> > DT-wise seemingly only the __symbols__ table (named just "its" there, I
> > notice), so we could drop (or rename) the label if you prefer.
> 
> No, I am asking *what* uses the ITS. Is it just dangling without any
> user? No PCI bus making use of it?

just 2ct, as far as I remember the rk1808 does have a PCIe controller.
And the datasheet [0] does agree with my memory it seems


Heiko

[0] http://opensource.rock-chips.com/images/4/43/Rockchip_RK1808_Datasheet_V1.2_20190527.pdf

[PATCH 4/9] arm64: dts: rockchip: Add Rockchip TB-RK1808M0

[Andreas Färber]

Add an initial Device Tree for Rockchip Toybrick TB-RK1808M0 mPCIe card.
Based on shipping TB-RK1808M0 DTB.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 arch/arm64/boot/dts/rockchip/Makefile         |  1 +
 .../boot/dts/rockchip/rk1808k-toybrick-m0.dts | 55 +++++++++++++++++++
 2 files changed, 56 insertions(+)
 create mode 100644 arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts

diff --git a/arch/arm64/boot/dts/rockchip/Makefile b/arch/arm64/boot/dts/rockchip/Makefile
index c3e00c0e2db7..d5a3837ccb7c 100644
--- a/arch/arm64/boot/dts/rockchip/Makefile
+++ b/arch/arm64/boot/dts/rockchip/Makefile
@@ -3,6 +3,7 @@ dtb-$(CONFIG_ARCH_ROCKCHIP) += px30-evb.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += px30-engicam-px30-core-ctouch2.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += px30-engicam-px30-core-ctouch2-of10.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += px30-engicam-px30-core-edimm2.2.dtb
+dtb-$(CONFIG_ARCH_ROCKCHIP) += rk1808k-toybrick-m0.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3308-evb.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3308-roc-cc.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3318-a95x-z2.dtb
diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
new file mode 100644
index 000000000000..2f8075d2391c
--- /dev/null
+++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright (c) 2021 Andreas Färber
+ */
+
+/dts-v1/;
+
+#include "rk1808.dtsi"
+
+/ {
+	compatible = "rockchip,tb-rk1808m0", "rockchip,rk1808";
+	model = "Rockchip Toybrick TB-RK1808M0";
+
+	chosen {
+		stdout-path = "serial2:1500000n8";
+	};
+
+	memory@0 {
+		device_type = "memory";
+		reg = <0x0 0x40000000>;
+	};
+
+	vcc3v3_pcie: vcc3v3-pcie {
+		compatible = "regulator-fixed";
+		regulator-name = "vcc3v3_pcie";
+		regulator-always-on;
+		regulator-boot-on;
+		regulator-min-microvolt = <3300000>;
+		regulator-max-microvolt = <3300000>;
+	};
+
+	vdd_cpu: vdd-cpu {
+		compatible = "regulator-fixed";
+		regulator-name = "vdd_cpu";
+		regulator-always-on;
+		regulator-boot-on;
+		regulator-min-microvolt = <850000>;
+		regulator-max-microvolt = <850000>;
+		vin-supply = <&vcc3v3_pcie>;
+	};
+};
+
+&cpu0 {
+	cpu-supply = <&vdd_cpu>;
+};
+
+&cpu1 {
+	cpu-supply = <&vdd_cpu>;
+};
+
+&uart2 {
+	status = "okay";
+	clocks = <&xin24m>;
+	clock-names = "baudclk";
+};
-- 
2.31.1

[PATCH RFC 5/9] arm64: dts: rockchip: rk1808k-toybrick-m0: Suppress vGIC interrupt

[Andreas Färber]

Avoid the kernel getting stuck after:

[    1.175956] kvm [1]: IPA Size Limit: 40 bits
[    1.177164] kvm [1]: vgic-v2@ff320000
[    1.177545] kvm [1]: GIC system register CPU interface enabled

or when dropping GICV reg entry:

[    1.176001] kvm [1]: IPA Size Limit: 40 bits
[    1.177191] kvm [1]: GICv3: no GICV resource entry
[    1.177664] kvm [1]: disabling GICv2 emulation
[    1.178115] kvm [1]: GIC system register CPU interface enabled

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
index 2f8075d2391c..15293a8576c6 100644
--- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
+++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
@@ -48,6 +48,10 @@ &cpu1 {
 	cpu-supply = <&vdd_cpu>;
 };
 
+&gic {
+	/delete-property/ interrupts;
+};
+
 &uart2 {
 	status = "okay";
 	clocks = <&xin24m>;
-- 
2.31.1

Re: [PATCH RFC 5/9] arm64: dts: rockchip: rk1808k-toybrick-m0: Suppress vGIC interrupt

[Marc Zyngier]

On Mon, 17 May 2021 00:05:47 +0100,
Andreas Färber <afaerber@suse.de> wrote:
> 
> Avoid the kernel getting stuck after:
> 
> [    1.175956] kvm [1]: IPA Size Limit: 40 bits
> [    1.177164] kvm [1]: vgic-v2@ff320000
> [    1.177545] kvm [1]: GIC system register CPU interface enabled
> 
> or when dropping GICV reg entry:
> 
> [    1.176001] kvm [1]: IPA Size Limit: 40 bits
> [    1.177191] kvm [1]: GICv3: no GICV resource entry
> [    1.177664] kvm [1]: disabling GICv2 emulation
> [    1.178115] kvm [1]: GIC system register CPU interface enabled
> 
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> index 2f8075d2391c..15293a8576c6 100644
> --- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> +++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> @@ -48,6 +48,10 @@ &cpu1 {
>  	cpu-supply = <&vdd_cpu>;
>  };
>  
> +&gic {
> +	/delete-property/ interrupts;
> +};
> +
>  &uart2 {
>  	status = "okay";
>  	clocks = <&xin24m>;

As I said in my reply to the cover letter, this is not an acceptable
outcome. Please add some debug to kvm_vgic_hyp_init() to understand
where this is hanging and why.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH RFC 5/9] arm64: dts: rockchip: rk1808k-toybrick-m0: Suppress vGIC interrupt

[Andreas Färber]

On 17.05.21 11:29, Marc Zyngier wrote:
> On Mon, 17 May 2021 00:05:47 +0100,
> Andreas Färber <afaerber@suse.de> wrote:
>>
>> Avoid the kernel getting stuck after:
>>
>> [    1.175956] kvm [1]: IPA Size Limit: 40 bits
>> [    1.177164] kvm [1]: vgic-v2@ff320000
>> [    1.177545] kvm [1]: GIC system register CPU interface enabled
>>
>> or when dropping GICV reg entry:
>>
>> [    1.176001] kvm [1]: IPA Size Limit: 40 bits
>> [    1.177191] kvm [1]: GICv3: no GICV resource entry
>> [    1.177664] kvm [1]: disabling GICv2 emulation
>> [    1.178115] kvm [1]: GIC system register CPU interface enabled
>>
>> Signed-off-by: Andreas Färber <afaerber@suse.de>
>> ---
>>  arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
>> index 2f8075d2391c..15293a8576c6 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
>> +++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
>> @@ -48,6 +48,10 @@ &cpu1 {
>>  	cpu-supply = <&vdd_cpu>;
>>  };
>>  
>> +&gic {
>> +	/delete-property/ interrupts;
>> +};
>> +
>>  &uart2 {
>>  	status = "okay";
>>  	clocks = <&xin24m>;
> 
> As I said in my reply to the cover letter, this is not an acceptable
> outcome. Please add some debug to kvm_vgic_hyp_init() to understand
> where this is hanging and why.

Many thanks for that pointer.

So, as alternative to dropping the DT interrupts property above, I could
also work around this issue by commenting out
vgic-init.c:vgic_init_cpu_starting()'s enable_percpu_irq() call.

Otherwise I am seeing the following call flow:

cpuhp_setup_state() -> __cpuhp_setup_state_cpuslocked() ->
cpuhp_issue_call() -> cpuhp_invoke_ap_callback() -> __cpuhp_kick_ap() ->
wait_for_ap_thread() -> wait_for_completion() --- doesn't return

With kvm_info() / printk():

[    1.244079] kvm [1]: IPA Size Limit: 40 bits

[    1.245205] kvm [1]: vgic-v2@ff320000

[    1.245584] kvm [1]: GIC system register CPU interface enabled

[    1.246177] kvm [1]: before cpuhp_setup_state

[    1.246605] __cpuhp_setup_state_cpuslocked: kvm/arm/vgic:starting

[    1.247198] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    1.247933] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    1.248745] cpuhp_issue_call: before invoke

[    1.249154] cpuhp_issue_call: before AP invoke

[    1.249585] cpuhp_invoke_ap_callback

[    1.249936] cpuhp_invoke_ap_callback: after cpu_online

[    1.250435] cpuhp_invoke_ap_callback: before st->thread

[    1.250944] cpuhp_invoke_ap_callback: after st->thread

[    1.251445] __cpuhp_kick_ap

[    1.251731] __cpuhp_kick_ap: not returned

[    1.252140] vgic_init_cpu_starting: 9

[    1.252507] vgic_init_cpu_starting: done

[    1.255538] __cpuhp_kick_ap: wait_for_ap_thread


Full serial output below.

Regards,
Andreas

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd042]

[    0.000000] Linux version 5.13.0-rc3-next-20210524+ (andreas@ryzen)
(aarch64-suse-linux-gcc (SUSE Linux) 10.3.0, GNU ld (GNU Binutils;
openSUSE Tumbleweed) 2.36.1.20210326-3) #17 SMP PREEMPT Mon May 24
16:21:21 CEST 2021

[    0.000000] Machine model: Rockchip Toybrick TB-RK1808M0

[    0.000000] earlycon: uart8250 at MMIO32 0x00000000ff550000 (options '')

[    0.000000] printk: bootconsole [uart8250] enabled

[    0.000000] efi: UEFI not found.

[    0.000000] Zone ranges:

[    0.000000]   DMA      [mem 0x0000000000600000-0x000000003fffffff]

[    0.000000]   DMA32    empty

[    0.000000]   Normal   empty

[    0.000000] Movable zone start for each node

[    0.000000] Early memory node ranges

[    0.000000]   node   0: [mem 0x0000000000600000-0x000000003fffffff]

[    0.000000] Initmem setup node 0 [mem
0x0000000000600000-0x000000003fffffff]

[    0.000000] cma: Reserved 16 MiB at 0x000000003dc00000

[    0.000000] psci: probing for conduit method from DT.

[    0.000000] psci: PSCIv1.0 detected in firmware.

[    0.000000] psci: Using standard PSCI v0.2 function IDs

[    0.000000] psci: Trusted OS migration not required

[    0.000000] psci: SMC Calling Convention v1.0

[    0.000000] percpu: Embedded 23 pages/cpu s55016 r8192 d31000 u94208

[    0.000000] Detected VIPT I-cache on CPU0

[    0.000000] CPU features: detected: GIC system register CPU interface

[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 256536

[    0.000000] __cpuhp_setup_state_cpuslocked: mm/page_alloc:dead

[    0.000000] Kernel command line: earlycon=uart8250,mmio32,0xff550000
console=ttyS2,1500000n8 rootwait

[    0.000000] Dentry cache hash table entries: 131072 (order: 8,
1048576 bytes, linear)

[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288
bytes, linear)

[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off

[    0.000000] Memory: 984916K/1042432K available (7168K kernel code,
770K rwdata, 2964K rodata, 1344K init, 331K bss, 41132K reserved, 16384K
cma-reserved)

[    0.000000] __cpuhp_setup_state_cpuslocked: slub:dead

[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1

[    0.000000] __cpuhp_setup_state_cpuslocked: lib/radix:dead

[    0.000000] rcu: Preemptible hierarchical RCU implementation.

[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=64 to
nr_cpu_ids=2.

[    0.000000]  Trampoline variant of Tasks RCU enabled.

[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay
is 25 jiffies.

[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2

[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0

[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode

[    0.000000] GICv3: 256 SPIs implemented

[    0.000000] GICv3: 0 Extended SPIs implemented

[    0.000000] GICv3: Distributor has no Range Selector support

[    0.000000] Root IRQ handler: gic_handle_irq

[    0.000000] GICv3: 16 PPIs implemented

[    0.000000] GICv3: CPU0: found redistributor 0 region
0:0x00000000ff140000

[    0.000000] __cpuhp_setup_state_cpuslocked: irqchip/arm/gicv3:starting

[    0.000000] ITS [mem 0xff120000-0xff13ffff]

[    0.000000] ITS@0x00000000ff120000: allocated 65536 Devices @1880000
(flat, esz 8, psz 64K, shr 0)

[    0.000000] ITS: using cache flushing for cmd queue

[    0.000000] GICv3: using LPI property table @0x0000000001830000

[    0.000000] GIC: using cache flushing for LPI property table

[    0.000000] GICv3: CPU0: using allocated LPI pending table
@0x0000000001840000

[    0.000000] random: get_random_bytes called from
start_kernel+0x494/0x614 with crng_init=0

[    0.000000] __cpuhp_setup_state_cpuslocked:
clockevents/arm/arch_timer:starting

[    0.000000] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    0.000000] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    0.000000] cpuhp_issue_call: before invoke

[    0.000000] cpuhp_issue_call: before AP invoke

[    0.000000] cpuhp_invoke_ap_callback

[    0.000000] cpuhp_invoke_ap_callback: after cpu_online

[    0.000000] cpuhp_invoke_ap_callback: before st->thread

[    0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys).

[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff
max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns

[    0.000000] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps
every 4398046511097ns

[    0.000968] Console: colour dummy device 80x25

[    0.001441] Calibrating delay loop (skipped), value calculated using
timer frequency.. 48.00 BogoMIPS (lpj=96000)

[    0.002476] pid_max: default: 32768 minimum: 301

[    0.002964] __cpuhp_setup_state_cpuslocked: fork:vm_stack_cache

[    0.003593] LSM: Security Framework initializing

[    0.004129] Mount-cache hash table entries: 2048 (order: 2, 16384
bytes, linear)

[    0.004857] Mountpoint-cache hash table entries: 2048 (order: 2,
16384 bytes, linear)

[    0.005933] __cpuhp_setup_state_cpuslocked: mm/writeback:online

[    0.006558] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    0.007254] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    0.008064] cpuhp_issue_call: before invoke

[    0.008473] cpuhp_issue_call: before AP invoke

[    0.008905] cpuhp_invoke_ap_callback

[    0.009256] cpuhp_invoke_ap_callback: after cpu_online

[    0.009754] cpuhp_invoke_ap_callback: before st->thread

[    0.010268] __cpuhp_setup_state_cpuslocked: mm/writeback:dead

[    0.012315] __cpuhp_setup_state_cpuslocked: mm/vmstat:dead

[    0.012868] __cpuhp_setup_state_cpuslocked: mm/vmstat:online

[    0.013611] __cpuhp_setup_state_cpuslocked: softirq:dead

[    0.014275] rcu: Hierarchical SRCU implementation.

[    0.015066] dyndbg: Ignore empty _ddebug table in a
CONFIG_DYNAMIC_DEBUG_CORE build

[    0.015882] Platform MSI: msi-controller@ff120000 domain created

[    0.016519] PCI/MSI:
/soc/interrupt-controller@ff100000/msi-controller@ff120000 domain created

[    0.017376] EFI services will not be available.

[    0.017819] __cpuhp_setup_state_cpuslocked:
clockevents/dummy_timer:starting

[    0.018505] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    0.019267] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    0.020079] cpuhp_issue_call: before invoke

[    0.020488] cpuhp_issue_call: before AP invoke

[    0.020920] cpuhp_invoke_ap_callback

[    0.021271] cpuhp_invoke_ap_callback: after cpu_online

[    0.021771] cpuhp_invoke_ap_callback: before st->thread

[    0.022283] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
state 0

[    0.023281] smp: Bringing up secondary CPUs ...

[    0.024485] Detected VIPT I-cache on CPU1

[    0.024531] GICv3: CPU1: found redistributor 1 region
0:0x00000000ff160000

[    0.024551] GICv3: CPU1: using allocated LPI pending table
@0x0000000001850000

[    0.024602] CPU1: Booted secondary processor 0x0000000001 [0x410fd042]

[    0.024654] __cpuhp_kick_ap

[    0.027331] __cpuhp_kick_ap: not returned

[    0.027728] __cpuhp_kick_ap: wait_for_ap_thread

[    0.028171] __cpuhp_kick_ap: done

[    0.028502] smp: Brought up 1 node, 2 CPUs

[    0.028907] SMP: Total of 2 processors activated.

[    0.029368] CPU features: detected: 32-bit EL0 Support

[    0.029870] CPU features: detected: 32-bit EL1 Support

[    0.030372] CPU features: detected: CRC32 instructions

[    0.031717] CPU: All CPU(s) started at EL2

[    0.032156] alternatives: patching kernel code

[    0.032935] __cpuhp_setup_state_cpuslocked: fs/buffer:dead

[    0.034270] devtmpfs: initialized

[    0.037183] __cpuhp_setup_state_cpuslocked: arm64/fpsimd:dead

[    0.038017] clocksource: jiffies: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 7645041785100000 ns

[    0.038982] futex hash table entries: 512 (order: 3, 32768 bytes, linear)

[    0.040375] pinctrl core: initialized pinctrl subsystem

[    0.041738] DMI not present or invalid.

[    0.042723] NET: Registered protocol family 16

[    0.043303] __cpuhp_setup_state_cpuslocked: arm64/debug_monitors:starting

[    0.043972] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    0.044665] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    0.045500] cpuhp_issue_call: before invoke

[    0.045910] cpuhp_issue_call: before AP invoke

[    0.046344] cpuhp_invoke_ap_callback

[    0.046694] cpuhp_invoke_ap_callback: after cpu_online

[    0.047193] cpuhp_invoke_ap_callback: before st->thread

[    0.047702] cpuhp_invoke_ap_callback: after st->thread

[    0.048202] __cpuhp_kick_ap

[    0.048477] __cpuhp_kick_ap: not returned

[    0.048880] __cpuhp_kick_ap: wait_for_ap_thread

[    0.049323] __cpuhp_kick_ap: done

[    0.049658] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
state 225

[    0.050352] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
before cpuhp_issue_call

[    0.051161] cpuhp_issue_call: before invoke

[    0.051569] cpuhp_issue_call: before AP invoke

[    0.052002] cpuhp_invoke_ap_callback

[    0.052352] cpuhp_invoke_ap_callback: after cpu_online

[    0.052851] cpuhp_invoke_ap_callback: before st->thread

[    0.053359] cpuhp_invoke_ap_callback: after st->thread

[    0.053867] __cpuhp_kick_ap

[    0.054142] __cpuhp_kick_ap: not returned

[    0.054556] __cpuhp_kick_ap: wait_for_ap_thread

[    0.054998] __cpuhp_kick_ap: done

[    0.056849] DMA: preallocated 128 KiB GFP_KERNEL pool for atomic
allocations

[    0.057739] DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA pool for
atomic allocations

[    0.058708] DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for
atomic allocations

[    0.059573] audit: initializing netlink subsys (disabled)

[    0.060384] audit: type=2000 audit(0.060:1): state=initialized
audit_enabled=0 res=1

[    0.061259] thermal_sys: Registered thermal governor 'step_wise'

[    0.061274] thermal_sys: Registered thermal governor 'power_allocator'

[    0.061967] cpuidle: using governor menu

[    0.063094] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.

[    0.063791] __cpuhp_setup_state_cpuslocked:
perf/arm64/hw_breakpoint:starting

[    0.064488] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    0.065181] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    0.065992] cpuhp_issue_call: before invoke

[    0.066401] cpuhp_issue_call: before AP invoke

[    0.066833] cpuhp_invoke_ap_callback

[    0.067184] cpuhp_invoke_ap_callback: after cpu_online

[    0.067682] cpuhp_invoke_ap_callback: before st->thread

[    0.068206] cpuhp_invoke_ap_callback: after st->thread

[    0.068707] __cpuhp_kick_ap

[    0.068981] __cpuhp_kick_ap: not returned

[    0.069402] __cpuhp_kick_ap: wait_for_ap_thread

[    0.069846] __cpuhp_kick_ap: done

[    0.070172] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
state 225

[    0.070864] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
before cpuhp_issue_call

[    0.071673] cpuhp_issue_call: before invoke

[    0.072081] cpuhp_issue_call: before AP invoke

[    0.072526] cpuhp_invoke_ap_callback

[    0.072876] cpuhp_invoke_ap_callback: after cpu_online

[    0.073377] cpuhp_invoke_ap_callback: before st->thread

[    0.073886] cpuhp_invoke_ap_callback: after st->thread

[    0.074384] __cpuhp_kick_ap

[    0.074658] __cpuhp_kick_ap: not returned

[    0.075058] __cpuhp_kick_ap: wait_for_ap_thread

[    0.075499] __cpuhp_kick_ap: done

[    0.075828] ASID allocator initialised with 65536 entries

[    0.076502] Serial: AMBA PL011 UART driver

[    0.076990] __cpuhp_setup_state_cpuslocked: iommu/iova:dead

[    0.087186] __cpuhp_setup_state_cpuslocked: mm/compaction:online

[    0.087978] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages

[    0.088643] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages

[    0.089294] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages

[    0.089946] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages

[    0.091249] __cpuhp_setup_state_cpuslocked: mm/memctrl:dead

[    0.091867] __cpuhp_setup_state_cpuslocked: io-wq/online

[    0.095436] __cpuhp_setup_state_cpuslocked: block/softirq:dead

[    0.096023] __cpuhp_setup_state_cpuslocked: block/mq:dead

[    0.096549] __cpuhp_setup_state_cpuslocked: block/mq:online

[    0.165312] raid6: neonx8   gen()  1270 MB/s

[    0.233458] raid6: neonx8   xor()  1007 MB/s

[    0.301610] raid6: neonx4   gen()  1289 MB/s

[    0.369761] raid6: neonx4   xor()   991 MB/s

[    0.437926] raid6: neonx2   gen()  1208 MB/s

[    0.506077] raid6: neonx2   xor()   932 MB/s

[    0.574228] raid6: neonx1   gen()  1068 MB/s

[    0.642354] raid6: neonx1   xor()   786 MB/s

[    0.710510] raid6: int64x8  gen()   470 MB/s

[    0.778636] raid6: int64x8  xor()   289 MB/s

[    0.846831] raid6: int64x4  gen()   602 MB/s

[    0.914893] raid6: int64x4  xor()   327 MB/s

[    0.983048] raid6: int64x2  gen()   788 MB/s

[    1.051164] raid6: int64x2  xor()   425 MB/s

[    1.119322] raid6: int64x1  gen()   652 MB/s

[    1.187434] raid6: int64x1  xor()   347 MB/s

[    1.187857] raid6: using algorithm neonx4 gen() 1289 MB/s

[    1.188383] raid6: .... xor() 991 MB/s, rmw enabled

[    1.188861] raid6: using neon recovery algorithm

[    1.190118] vdd_cpu: supplied by vcc3v3_pcie

[    1.190933] iommu: Default domain type: Translated

[    1.191731] vgaarb: loaded

[    1.192430] SCSI subsystem initialized

[    1.193060] usbcore: registered new interface driver usbfs

[    1.193644] usbcore: registered new interface driver hub

[    1.194208] usbcore: registered new device driver usb

[    1.194850] pps_core: LinuxPPS API ver. 1 registered

[    1.195336] pps_core: Software ver. 5.3.6 - Copyright 2005-2007
Rodolfo Giometti <giometti@linux.it>

[    1.196291] PTP clock support registered

[    1.197222] __cpuhp_setup_state_cpuslocked: perf/arm/pmu:starting

[    1.198409] __cpuhp_setup_state_cpuslocked: net/dev:dead

[    1.199515] clocksource: Switched to clocksource arch_sys_counter

[    1.200460] VFS: Disk quotas dquot_6.6.0

[    1.200916] VFS: Dquot-cache hash table entries: 512 (order 0, 4096
bytes)

[    1.213721] NET: Registered protocol family 2

[    1.214283] IP idents hash table entries: 16384 (order: 5, 131072
bytes, linear)

[    1.216289] tcp_listen_portaddr_hash hash table entries: 512 (order:
2, 20480 bytes, linear)

[    1.217219] TCP established hash table entries: 8192 (order: 4, 65536
bytes, linear)

[    1.218045] TCP bind hash table entries: 8192 (order: 6, 262144
bytes, linear)

[    1.218924] TCP: Hash tables configured (established 8192 bind 8192)

[    1.219783] UDP hash table entries: 512 (order: 3, 49152 bytes, linear)

[    1.220483] UDP-Lite hash table entries: 512 (order: 3, 49152 bytes,
linear)

[    1.221370] NET: Registered protocol family 1

[    1.221831] PCI: CLS 0 bytes, default 64

[    1.222307] __cpuhp_setup_state_cpuslocked: arm64/cpuinfo:online

[    1.222898] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    1.223629] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    1.224445] cpuhp_issue_call: before invoke

[    1.224854] cpuhp_issue_call: before AP invoke

[    1.225287] cpuhp_invoke_ap_callback

[    1.225637] cpuhp_invoke_ap_callback: after cpu_online

[    1.226136] cpuhp_invoke_ap_callback: before st->thread

[    1.226645] cpuhp_invoke_ap_callback: after st->thread

[    1.227145] __cpuhp_kick_ap

[    1.227421] __cpuhp_kick_ap: not returned

[    1.227896] __cpuhp_kick_ap: wait_for_ap_thread

[    1.228345] __cpuhp_kick_ap: done

[    1.228672] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
state 225

[    1.229365] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 1:
before cpuhp_issue_call

[    1.230174] cpuhp_issue_call: before invoke

[    1.230582] cpuhp_issue_call: before AP invoke

[    1.231015] cpuhp_invoke_ap_callback

[    1.231366] cpuhp_invoke_ap_callback: after cpu_online

[    1.231894] cpuhp_invoke_ap_callback: before st->thread

[    1.232404] cpuhp_invoke_ap_callback: after st->thread

[    1.232905] __cpuhp_kick_ap

[    1.233178] __cpuhp_kick_ap: not returned

[    1.233576] __cpuhp_kick_ap: wait_for_ap_thread

[    1.234018] __cpuhp_kick_ap: done

[    1.234645] cpuhp_issue_call: before invoke

[    1.235062] cpuhp_issue_call: before AP invoke

[    1.235536] cpuhp_invoke_ap_callback

[    1.235890] cpuhp_invoke_ap_callback: after cpu_online

[    1.236391] cpuhp_invoke_ap_callback: before st->thread

[    1.236899] cpuhp_invoke_ap_callback: after st->thread

[    1.237399] __cpuhp_kick_ap

[    1.237672] __cpuhp_kick_ap: not returned

[    1.238100] __cpuhp_kick_ap: wait_for_ap_thread

[    1.238544] __cpuhp_kick_ap: done

[    1.238870] cpuhp_issue_call: before invoke

[    1.239277] cpuhp_issue_call: before AP invoke

[    1.239759] cpuhp_invoke_ap_callback

[    1.240111] cpuhp_invoke_ap_callback: after cpu_online

[    1.240609] cpuhp_invoke_ap_callback: before st->thread

[    1.241117] cpuhp_invoke_ap_callback: after st->thread

[    1.241617] __cpuhp_kick_ap

[    1.241891] __cpuhp_kick_ap: not returned

[    1.242289] __cpuhp_kick_ap: wait_for_ap_thread

[    1.242732] __cpuhp_kick_ap: done

[    1.243083] hw perfevents: enabled with armv8_cortex_a35 PMU driver,
7 counters available

[    1.244079] kvm [1]: IPA Size Limit: 40 bits

[    1.245205] kvm [1]: vgic-v2@ff320000

[    1.245584] kvm [1]: GIC system register CPU interface enabled

[    1.246177] kvm [1]: before cpuhp_setup_state

[    1.246605] __cpuhp_setup_state_cpuslocked: kvm/arm/vgic:starting

[    1.247198] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
state 225

[    1.247933] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
before cpuhp_issue_call

[    1.248745] cpuhp_issue_call: before invoke

[    1.249154] cpuhp_issue_call: before AP invoke

[    1.249585] cpuhp_invoke_ap_callback

[    1.249936] cpuhp_invoke_ap_callback: after cpu_online

[    1.250435] cpuhp_invoke_ap_callback: before st->thread

[    1.250944] cpuhp_invoke_ap_callback: after st->thread

[    1.251445] __cpuhp_kick_ap

[    1.251731] __cpuhp_kick_ap: not returned

[    1.252140] vgic_init_cpu_starting: 9

[    1.252507] vgic_init_cpu_starting: done

[    1.255538] __cpuhp_kick_ap: wait_for_ap_thread


-- 
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer
HRB 36809 (AG Nürnberg)

Re: [PATCH RFC 5/9] arm64: dts: rockchip: rk1808k-toybrick-m0: Suppress vGIC interrupt

[Marc Zyngier]

On Mon, 24 May 2021 15:40:22 +0100,
Andreas Färber <afaerber@suse.de> wrote:
> 
> On 17.05.21 11:29, Marc Zyngier wrote:
> > On Mon, 17 May 2021 00:05:47 +0100,
> > Andreas Färber <afaerber@suse.de> wrote:
> >>
> >> Avoid the kernel getting stuck after:
> >>
> >> [    1.175956] kvm [1]: IPA Size Limit: 40 bits
> >> [    1.177164] kvm [1]: vgic-v2@ff320000
> >> [    1.177545] kvm [1]: GIC system register CPU interface enabled
> >>
> >> or when dropping GICV reg entry:
> >>
> >> [    1.176001] kvm [1]: IPA Size Limit: 40 bits
> >> [    1.177191] kvm [1]: GICv3: no GICV resource entry
> >> [    1.177664] kvm [1]: disabling GICv2 emulation
> >> [    1.178115] kvm [1]: GIC system register CPU interface enabled
> >>
> >> Signed-off-by: Andreas Färber <afaerber@suse.de>
> >> ---
> >>  arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts | 4 ++++
> >>  1 file changed, 4 insertions(+)
> >>
> >> diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> >> index 2f8075d2391c..15293a8576c6 100644
> >> --- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> >> +++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> >> @@ -48,6 +48,10 @@ &cpu1 {
> >>  	cpu-supply = <&vdd_cpu>;
> >>  };
> >>  
> >> +&gic {
> >> +	/delete-property/ interrupts;
> >> +};
> >> +
> >>  &uart2 {
> >>  	status = "okay";
> >>  	clocks = <&xin24m>;
> > 
> > As I said in my reply to the cover letter, this is not an acceptable
> > outcome. Please add some debug to kvm_vgic_hyp_init() to understand
> > where this is hanging and why.
> 
> Many thanks for that pointer.
> 
> So, as alternative to dropping the DT interrupts property above, I could
> also work around this issue by commenting out
> vgic-init.c:vgic_init_cpu_starting()'s enable_percpu_irq() call.
> 
> Otherwise I am seeing the following call flow:
> 
> cpuhp_setup_state() -> __cpuhp_setup_state_cpuslocked() ->
> cpuhp_issue_call() -> cpuhp_invoke_ap_callback() -> __cpuhp_kick_ap() ->
> wait_for_ap_thread() -> wait_for_completion() --- doesn't return
> 
> With kvm_info() / printk():
> 
> [    1.244079] kvm [1]: IPA Size Limit: 40 bits
> 
> [    1.245205] kvm [1]: vgic-v2@ff320000
> 
> [    1.245584] kvm [1]: GIC system register CPU interface enabled
> 
> [    1.246177] kvm [1]: before cpuhp_setup_state
> 
> [    1.246605] __cpuhp_setup_state_cpuslocked: kvm/arm/vgic:starting
> 
> [    1.247198] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
> state 225
> 
> [    1.247933] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
> before cpuhp_issue_call
> 
> [    1.248745] cpuhp_issue_call: before invoke
> 
> [    1.249154] cpuhp_issue_call: before AP invoke
> 
> [    1.249585] cpuhp_invoke_ap_callback
> 
> [    1.249936] cpuhp_invoke_ap_callback: after cpu_online
> 
> [    1.250435] cpuhp_invoke_ap_callback: before st->thread
> 
> [    1.250944] cpuhp_invoke_ap_callback: after st->thread
> 
> [    1.251445] __cpuhp_kick_ap
> 
> [    1.251731] __cpuhp_kick_ap: not returned
> 
> [    1.252140] vgic_init_cpu_starting: 9
> 
> [    1.252507] vgic_init_cpu_starting: done
> 
> [    1.255538] __cpuhp_kick_ap: wait_for_ap_thread

And you never see any RCU stall after that? It looks like a CPU has
disappeared in the weeds after enabling the per-CPU interrupt.

Please instrument what happens in
drivers/irqchip/irq-gic-v3.c::gic_unmask_irq() when d->hwirq == 9, a
well as vgic_maintenance_handler(), just in case it gets called...

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

[PATCH 6/9] dt-bindings: mmc: rockchip-dw-mshc: Add Rockchip RK1808

[Andreas Färber]

Add a compatible string for Rockchip RK1808 SoC.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml b/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml
index eaa3b0ef24f6..54fb59820d2b 100644
--- a/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml
+++ b/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml
@@ -30,6 +30,7 @@ properties:
       - items:
           - enum:
               - rockchip,px30-dw-mshc
+              - rockchip,rk1808-dw-mshc
               - rockchip,rk3036-dw-mshc
               - rockchip,rk3228-dw-mshc
               - rockchip,rk3308-dw-mshc
-- 
2.31.1

Re: [PATCH 6/9] dt-bindings: mmc: rockchip-dw-mshc: Add Rockchip RK1808

[Rob Herring]

On Mon, 17 May 2021 01:05:48 +0200, Andreas Färber wrote:
> Add a compatible string for Rockchip RK1808 SoC.
> 
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml | 1 +
>  1 file changed, 1 insertion(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH 6/9] dt-bindings: mmc: rockchip-dw-mshc: Add Rockchip RK1808

[Ulf Hansson]

On Mon, 17 May 2021 at 01:06, Andreas Färber <afaerber@suse.de> wrote:
>
> Add a compatible string for Rockchip RK1808 SoC.
>
> Signed-off-by: Andreas Färber <afaerber@suse.de>

Applied for next, thanks!

Kind regards
Uffe


> ---
>  Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml b/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml
> index eaa3b0ef24f6..54fb59820d2b 100644
> --- a/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml
> +++ b/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.yaml
> @@ -30,6 +30,7 @@ properties:
>        - items:
>            - enum:
>                - rockchip,px30-dw-mshc
> +              - rockchip,rk1808-dw-mshc
>                - rockchip,rk3036-dw-mshc
>                - rockchip,rk3228-dw-mshc
>                - rockchip,rk3308-dw-mshc
> --
> 2.31.1
>

[PATCH 7/9] arm64: dts: rockchip: rk1808: Prepare eMMC node

[Andreas Färber]

Add an eMMC DT node.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 arch/arm64/boot/dts/rockchip/rk1808.dtsi | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
index af2b51afda7d..b4a71c5c8be7 100644
--- a/arch/arm64/boot/dts/rockchip/rk1808.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
@@ -199,5 +199,14 @@ uart7: serial@ff5c0000 {
 			interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
 			status = "disabled";
 		};
+
+		emmc: mmc@ffd00000 {
+			compatible = "rockchip,rk1808-dw-mshc", "rockchip,rk3288-dw-mshc";
+			reg = <0xffd00000 0x4000>;
+			interrupts = <GIC_SPI 47 IRQ_TYPE_LEVEL_HIGH>;
+			max-frequency = <150000000>;
+			fifo-depth = <0x100>;
+			status = "disabled";
+		};
 	};
 };
-- 
2.31.1

[PATCH 8/9] arm64: dts: rockchip: rk1808k-toybrick-m0: Enable eMMC

[Andreas Färber]

Add fake clocks (based on downstream kernel's debugfs clk_summary)
and enable eMMC.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 .../boot/dts/rockchip/rk1808k-toybrick-m0.dts | 38 +++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
index 15293a8576c6..aabe7a7034b2 100644
--- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
+++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
@@ -38,6 +38,34 @@ vdd_cpu: vdd-cpu {
 		regulator-max-microvolt = <850000>;
 		vin-supply = <&vcc3v3_pcie>;
 	};
+
+	hclk_emmc: hclk-emmc {
+		compatible = "fixed-clock";
+		clock-frequency = <198000000>;
+		#clock-cells = <0>;
+		clock-output-names = "hclk_emmc";
+	};
+
+	sclk_emmc: sclk-emmc {
+		compatible = "fixed-clock";
+		clock-frequency = <297000000>;
+		#clock-cells = <0>;
+		clock-output-names = "sclk_emmc";
+	};
+
+	sclk_emmc_drv: sclk-emmc-drv {
+		compatible = "fixed-clock";
+		clock-frequency = <148500000>;
+		#clock-cells = <0>;
+		clock-output-names = "sclk_emmc_drv";
+	};
+
+	sclk_emmc_sample: sclk-emmc-sample {
+		compatible = "fixed-clock";
+		clock-frequency = <148500000>;
+		#clock-cells = <0>;
+		clock-output-names = "sclk_emmc_sample";
+	};
 };
 
 &cpu0 {
@@ -48,6 +76,16 @@ &cpu1 {
 	cpu-supply = <&vdd_cpu>;
 };
 
+&emmc {
+	status = "okay";
+	clocks = <&hclk_emmc>, <&sclk_emmc>, <&sclk_emmc_drv>, <&sclk_emmc_sample>;
+	clock-names = "biu", "ciu", "ciu-drive", "ciu-sample";
+	cap-mmc-highspeed;
+	mmc-hs200-1_2v;
+	non-removable;
+	bus-width = <8>;
+};
+
 &gic {
 	/delete-property/ interrupts;
 };
-- 
2.31.1

[PATCH 9/9] arm64: dts: rockchip: rk1808: Add CPU operating points

[Andreas Färber]

Associate operating points with both CPU nodes.
Data source is the shipping TB-RK1808M0 DTB.

Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 arch/arm64/boot/dts/rockchip/rk1808.dtsi | 64 ++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk1808.dtsi b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
index b4a71c5c8be7..82614c47f144 100644
--- a/arch/arm64/boot/dts/rockchip/rk1808.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk1808.dtsi
@@ -32,7 +32,10 @@ cpu0: cpu@0 {
 			compatible = "arm,cortex-a35";
 			reg = <0x0 0x0>;
 			enable-method = "psci";
+			dynamic-power-coefficient = <74>;
+			operating-points-v2 = <&cpu0_opp_table>;
 			cpu-idle-states = <&CPU_SLEEP>;
+			#cooling-cells = <2>;
 		};
 
 		cpu1: cpu@1 {
@@ -40,6 +43,7 @@ cpu1: cpu@1 {
 			compatible = "arm,cortex-a35";
 			reg = <0x0 0x1>;
 			enable-method = "psci";
+			operating-points-v2 = <&cpu0_opp_table>;
 			cpu-idle-states = <&CPU_SLEEP>;
 		};
 
@@ -57,6 +61,66 @@ CPU_SLEEP: cpu-sleep {
 		};
 	};
 
+	cpu0_opp_table: cpu0-opp-table {
+		compatible = "operating-points-v2";
+		opp-shared;
+
+		opp-408000000 {
+			opp-hz = /bits/ 64 <408000000>;
+			opp-microvolt = <750000 750000 950000>;
+			clock-latency-ns = <40000>;
+			opp-suspend;
+		};
+
+		opp-600000000 {
+			opp-hz = /bits/ 64 <600000000>;
+			opp-microvolt = <750000 750000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-816000000 {
+			opp-hz = /bits/ 64 <816000000>;
+			opp-microvolt = <750000 750000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-1008000000 {
+			opp-hz = /bits/ 64 <1008000000>;
+			opp-microvolt = <750000 750000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-1200000000 {
+			opp-hz = /bits/ 64 <1200000000>;
+			opp-microvolt = <800000 800000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-1296000000 {
+			opp-hz = /bits/ 64 <1296000000>;
+			opp-microvolt = <825000 825000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-1416000000 {
+			opp-hz = /bits/ 64 <1416000000>;
+			opp-microvolt = <850000 850000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-1512000000 {
+			opp-hz = /bits/ 64 <1512000000>;
+			opp-microvolt = <875000 875000 950000>;
+			clock-latency-ns = <40000>;
+		};
+
+		opp-1608000000 {
+			opp-hz = /bits/ 64 <1608000000>;
+			opp-microvolt = <900000 900000 950000>;
+			clock-latency-ns = <40000>;
+		};
+	};
+
 	arm-pmu {
 		compatible = "arm,cortex-a35-pmu";
 		interrupts = <GIC_SPI 228 IRQ_TYPE_LEVEL_HIGH>,
-- 
2.31.1

Re: [PATCH 0/9] arm64: dts: rockchip: Initial Toybrick TB-RK1808M0 support

[Marc Zyngier]

On Mon, 17 May 2021 00:05:42 +0100,
Andreas Färber <afaerber@suse.de> wrote:
> 
> Hello Heiko et al.,
> 
> It seems linux-rockchip list only saw two RK1808 patches for ASoC in 2019.
> Following up on a SUSE Hackweek 20 project of mine, here's some patches that
> allow me to start booting into the TB-RK1808M0 mPCIe card's eMMC.
> 
> Tested using its USB adapter, which allows to connect a serial cable and a
> USB storage device that I load kernel+dtb from. It has a reset button, and
> Ctrl+C allows to enter a U-Boot prompt (without EBBR/UEFI support though).
> 
> Patches are based on the shipping toybrick.dtb file.
> http://t.rock-chips.com/en/wiki.php?mod=view&id=110 gives instructions for
> compiling sources, but no source download or link is actually provided.
> 
> I encountered a hang: earlycon revealed it being related to KVM and
> vGIC.  Disabling KVM in Kconfig works around it, as does removing
> the vGIC irq in DT.  I've already tried low and high for the vGIC
> interrupt, so no clue what might cause it. On an mPCIe card with 1
> GiB of RAM I figured KVM is not going to be a major use case, so if
> we find no other solution, we could just delete the interrupts
> property in its .dts, as demonstrated here.

I think you figured it out wrong, for a number of reasons:

- KVM hanging is usually a sign that you have described the platform
  the wrong way. Either you are stepping over reserved memory regions,
  or you have badly described the GIC itself.

- It could also be a bug in KVM, which will need to be fixed. If
  that's because the HW is broken, we need to be able to detect it.

- You cannot be prescriptive of what a user is going to run. People
  have been running KVM on systems with less memory than that.

So no, we don't paper over these issues. We work out what is going
wrong and we fix it.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 0/9] arm64: dts: rockchip: Initial Toybrick TB-RK1808M0 support

[Andreas Färber]

Hi Marc,

On 17.05.21 11:02, Marc Zyngier wrote:
> On Mon, 17 May 2021 00:05:42 +0100,
> Andreas Färber <afaerber@suse.de> wrote:
>> Patches are based on the shipping toybrick.dtb file.

>> http://t.rock-chips.com/en/wiki.php?mod=view&id=110 gives instructions for

>> compiling sources, but no source download or link is actually provided.

>> 

>> I encountered a hang: earlycon revealed it being related to KVM and
>> vGIC.  Disabling KVM in Kconfig works around it, as does removing
>> the vGIC irq in DT.  I've already tried low and high for the vGIC
>> interrupt, so no clue what might cause it. On an mPCIe card with 1
>> GiB of RAM I figured KVM is not going to be a major use case, so if
>> we find no other solution, we could just delete the interrupts
>> property in its .dts, as demonstrated here.
> 
> I think you figured it out wrong,

Did I? I identified that an issue resulting in no serial console was
dependent on CONFIG_KVM being enabled and specifically to the vGIC
interrupt being specified in my DT. That's all I said.

I never claimed KVM code was to blame, you should know me better by now!

> for a number of reasons:
> 
> - KVM hanging is usually a sign that you have described the platform
>   the wrong way. Either you are stepping over reserved memory regions,
>   or you have badly described the GIC itself.

This whole series is about a new DT hardware description, so yes, that
is the most likely source of the problem I'm observing. Without further
hints how to verify what may cause it, you're just stating the obvious.

The only /reserved-memory entries in the shipping DTB are drm-logo of
size 0 and ramoops - the latter I could try to test, but I'd assume that
to just be a software convention that for lack of oops should not affect
KVM here?

And why would reserved memory affect the vGIC but no other driver doing
allocations? Any way to narrow it down, does vGIC allocate specially?

Only other issue I'm seeing is Debian failing to mount partitions that I
checked I do have drivers built in for and ends up failing to provide an
emergency shell. In order to boot a clean openSUSE rootfs for comparison
I'd first need to figure out adding any USB host nodes and clocks.

> 
> - It could also be a bug in KVM, which will need to be fixed. If
>   that's because the HW is broken, we need to be able to detect it.
> 
> - You cannot be prescriptive of what a user is going to run. People
>   have been running KVM on systems with less memory than that.
> 
> So no, we don't paper over these issues.

As you can see in patch 3, it does include the vGIC interrupt, so that
anyone with access to the TB-96AIoT or any EVB can test KVM and report
success or failure. Thus I don't see me as papering over something here.

However, patch 5 is needed to test this patchset on at least M0 - to
have serial and eMMC rootfs working - until a better fix is found.

> We work out what is going
> wrong and we fix it.

Thanks. You were specifically copied to advise on
how to figure out what might cause it, so that we/I can fix it properly. :)

As I mentioned, I already tried changing the interrupt between high and
low (which was a likely bug source on Realtek RK1319 (where I'm still
waiting on them to confirm a ~year later...)).
I don't have a data source other than the downstream .dtb to check the
interrupt number - mainline PX30/RK3308/RK3328/RK3368/RK3399 do all use
9 and high consistently though, so I figured it's likely correct.

What I was wondering is whether the vGIC, similar to arch timer, might
need some initialization in the bootloader? (Note: No U-Boot sources
either at the link.)
Unfortunately I'm seeing a recurring pattern (cf. Realtek) that vendors
in their BSPs don't enable KVM and thus don't validate their hardware
description against KVM; their shipping 4.4 based kernel here does not
seem to have KVM enabled.

Or is it possible for vendors to actually have a Cortex-A35 without the
Armv8 Virtualization Extensions in silicon? If so, how could one verify?

Thanks,
Andreas

-- 
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer
HRB 36809 (AG Nürnberg)

Re: [PATCH 0/9] arm64: dts: rockchip: Initial Toybrick TB-RK1808M0 support

[Marc Zyngier]

Andreas,

On Mon, 17 May 2021 13:22:27 +0100,
Andreas Färber <afaerber@suse.de> wrote:
> 
> Hi Marc,
> 
> On 17.05.21 11:02, Marc Zyngier wrote:
> > On Mon, 17 May 2021 00:05:42 +0100,
> > Andreas Färber <afaerber@suse.de> wrote:
> >> Patches are based on the shipping toybrick.dtb file.
> 
> >> http://t.rock-chips.com/en/wiki.php?mod=view&id=110 gives instructions for
> 
> >> compiling sources, but no source download or link is actually provided.
> 
> >> 
> 
> >> I encountered a hang: earlycon revealed it being related to KVM and
> >> vGIC.  Disabling KVM in Kconfig works around it, as does removing
> >> the vGIC irq in DT.  I've already tried low and high for the vGIC
> >> interrupt, so no clue what might cause it. On an mPCIe card with 1
> >> GiB of RAM I figured KVM is not going to be a major use case, so if
> >> we find no other solution, we could just delete the interrupts
> >> property in its .dts, as demonstrated here.
> > 
> > I think you figured it out wrong,
> 
> Did I? I identified that an issue resulting in no serial console was
> dependent on CONFIG_KVM being enabled and specifically to the vGIC
> interrupt being specified in my DT. That's all I said.

I guess we have a different way to approach these issues. Rather than
disabling a feature, I would have reached out to narrow the problem
down *before* posting a series.

> I never claimed KVM code was to blame, you should know me better by
> now!

Maybe it *is* to blame, and I'd really like to know.

> > for a number of reasons:
> > 
> > - KVM hanging is usually a sign that you have described the platform
> >   the wrong way. Either you are stepping over reserved memory regions,
> >   or you have badly described the GIC itself.
> 
> This whole series is about a new DT hardware description, so yes, that
> is the most likely source of the problem I'm observing. Without further
> hints how to verify what may cause it, you're just stating the obvious.
>
> The only /reserved-memory entries in the shipping DTB are drm-logo of
> size 0 and ramoops - the latter I could try to test, but I'd assume that
> to just be a software convention that for lack of oops should not affect
> KVM here?
> 
> And why would reserved memory affect the vGIC but no other driver doing
> allocations? Any way to narrow it down, does vGIC allocate specially?

Not an existing reserved memory, but instead the lack of a reserved
memory description in the DT, on which KVM would happily step as part
of its own allocations. Having a working vGIC adds a substantial
amount of code paths and (surprise!) interrupt handling.

> Only other issue I'm seeing is Debian failing to mount partitions that I
> checked I do have drivers built in for and ends up failing to provide an
> emergency shell. In order to boot a clean openSUSE rootfs for comparison
> I'd first need to figure out adding any USB host nodes and clocks.
> 
> > 
> > - It could also be a bug in KVM, which will need to be fixed. If
> >   that's because the HW is broken, we need to be able to detect it.
> > 
> > - You cannot be prescriptive of what a user is going to run. People
> >   have been running KVM on systems with less memory than that.
> > 
> > So no, we don't paper over these issues.
> 
> As you can see in patch 3, it does include the vGIC interrupt, so that
> anyone with access to the TB-96AIoT or any EVB can test KVM and report
> success or failure. Thus I don't see me as papering over something here.
> 
> However, patch 5 is needed to test this patchset on at least M0 - to
> have serial and eMMC rootfs working - until a better fix is found.

And that's not papering over the problem? OK, nevermind. Not to
mention that the GIC node has some obvious mistakes which result from
copy-paste.

> > We work out what is going
> > wrong and we fix it.
> 
> Thanks. You were specifically copied to advise on
> how to figure out what might cause it, so that we/I can fix it properly. :)
> 
> As I mentioned, I already tried changing the interrupt between high and
> low (which was a likely bug source on Realtek RK1319 (where I'm still
> waiting on them to confirm a ~year later...)).

Which has no influence since the GIC-500 PPIs are not configurable in
SW, and the presence of this attribute in the DT is just for
documentation.

> I don't have a data source other than the downstream .dtb to check the
> interrupt number - mainline PX30/RK3308/RK3328/RK3368/RK3399 do all use
> 9 and high consistently though, so I figured it's likely correct.
> 
> What I was wondering is whether the vGIC, similar to arch timer, might
> need some initialization in the bootloader? (Note: No U-Boot sources
> either at the link.)

As long as the PPIs are set as group-1NS, this is enough. You can find
out by dumping the redistributors' GICR_IGROUPR0 registers. Nothing
else is required for the GIC to behave.

> Unfortunately I'm seeing a recurring pattern (cf. Realtek) that vendors
> in their BSPs don't enable KVM and thus don't validate their hardware
> description against KVM; their shipping 4.4 based kernel here does not
> seem to have KVM enabled.
> 
> Or is it possible for vendors to actually have a Cortex-A35 without the
> Armv8 Virtualization Extensions in silicon? If so, how could one verify?

There is no "Armv8 Virtualization Extensions". There is only EL2, and
you are already booting at that exception level, or KVM wouldn't even
try to initialise.

It would probably help if you posted a full dmesg as well as added
some basic tracing in the vgic init code so that we can figure out
*what* is going wrong, so that we can all stop making idle guesses.

	M.

-- 
Without deviation from the norm, progress is not possible.