RISC-V Device Tree Documentation

RISC-V Device Tree Documentation

[Palmer Dabbelt]

As part of submitting the RISC-V Linux port it was suggested that we write some
documentation that describes our device tree bindings.  Our port is still in
progress, but it was suggested that we start sending patches to the relevant
parties.  There is one outstanding issue to resolve with the PLIC bindings, but
I think the other two patches should be sane.

[PATCH 1/3] dts: RISC-V vendor prefix

[Palmer Dabbelt]

RISC-V systems use device tree to specify the memory layout of the
system.  This patch reserves the "riscv" vendor prefix, which will be
used for devices that are specified by the various RISC-V ISA
specifications.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
---
 Documentation/devicetree/bindings/vendor-prefixes.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/devicetree/bindings/vendor-prefixes.txt b/Documentation/devicetree/bindings/vendor-prefixes.txt
index c03d20140366..be6c01a68094 100644
--- a/Documentation/devicetree/bindings/vendor-prefixes.txt
+++ b/Documentation/devicetree/bindings/vendor-prefixes.txt
@@ -266,6 +266,7 @@ renesas	Renesas Electronics Corporation
 richtek	Richtek Technology Corporation
 ricoh	Ricoh Co. Ltd.
 rikomagic	Rikomagic Tech Corp. Ltd
+riscv	RISC-V Foundation
 rockchip	Fuzhou Rockchip Electronics Co., Ltd
 rohm	ROHM Semiconductor Co., Ltd
 samsung	Samsung Semiconductor
-- 
2.13.0

[PATCH 2/3] dts: RISC-V local interrupt controller docs

[Palmer Dabbelt]

This patch adds documentation on the RISC-V local interrupt controller,
which is a per-hart interrupt controller that manages all interrupts
entering a RISC-V hart.  This interrupt controller is present on all
RISC-V systems.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
---
 .../interrupt-controller/riscv,cpu-intc.txt        | 70 ++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt

diff --git a/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
new file mode 100644
index 000000000000..f4906f49a1b4
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
@@ -0,0 +1,70 @@
+RISC-V Hart-Level Interrupt Controller (HLIC)
+---------------------------------------------
+
+RISC-V cores include Control Status Registers (CSRs) which are local to each
+hart and can be read or written by software. Some of these CSRs are used to
+control local interrupts connected to the core.  Every interrupt is ultimately
+routed through a hart's HLIC before it interrupts that hart.
+
+The RISC-V supervisor ISA manual specifices three interrupt sources that are
+attached to every HLIC: software interrupts, the timer interrupt, and external
+interrupts.  Software interrupts are used to send IPIs between cores.  The
+timer interrupt comes from an architectually mandated real-time timer that is
+controller via SBI calls and CSR reads.  External interrupts connect all other
+device interrupts to the HLIC, which are routed via the platforw-level
+interrupt contrellor (PLIC).
+
+All RISC-V systems that conform to the supervisor ISA specification are
+required to have a HLIC with these three interrupt sources present.
+
+Required properties:
+- compatible : "riscv,cpu-intc"
+- #interrupt-cells : should be <1>
+- interrupt-controller : Identifies the node as an interrupt controller
+
+Furthermore, this interrupt-controller MUST be embedded inside the cpu
+definition of the hart whose CSRs control these local interrupts.
+
+An example of a device tree entry for a standard RISC-V hart is as follows:
+
+	cpu1: cpu@1 {
+		clock-frequency = <1600000000>;
+		compatible = "riscv";
+		d-cache-block-size = <64>;
+		d-cache-sets = <64>;
+		d-cache-size = <16384>;
+		d-tlb-sets = <1>;
+		d-tlb-size = <32>;
+		device_type = "cpu";
+		i-cache-block-size = <64>;
+		i-cache-sets = <64>;
+		i-cache-size = <16384>;
+		i-tlb-sets = <1>;
+		i-tlb-size = <32>;
+		mmu-type = "riscv,sv39";
+		next-level-cache = <&L2>;
+		reg = <1>;
+		riscv,isa = "rv64imac";
+		status = "okay";
+		tlb-split;
+		cpu1-intc: interrupt-controller {
+			#interrupt-cells = <1>;
+			compatible = "riscv,cpu-intc";
+			interrupt-controller;
+		};
+	};
+
+The sections specifically relevant to the HLIC are
+
+	cpu1: cpu@1 {
+		cpu1-intc: interrupt-controller {
+			#interrupt-cells = <1>;
+			compatible = "riscv,cpu-intc";
+			interrupt-controller;
+		};
+	};
+
+As the HLIC is essentially a static interrupt controller, this device tree
+entry essentially just indicates the presence of a HLIC.  All HLIC interrupts
+are level-triggered, so there is no need to specify this as part of the device
+tree.
-- 
2.13.0

[PATCH 3/3] dts: RISC-V PLIC documentation

[Palmer Dabbelt]

This patch adds documentation for the platform-level interrupt
controller (PLIC) found in all RISC-V systems.  This interrupt
controller routes interrupts from all the devices in the system to each
hart-local interrupt controller.

Note: the DTS bindings for the PLIC aren't set in stone yet, as we might
want to change how we're specifying holes in the hart list.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
---
 .../bindings/interrupt-controller/riscv,plic0.txt  | 63 ++++++++++++++++++++++
 1 file changed, 63 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt

diff --git a/Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt b/Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt
new file mode 100644
index 000000000000..b51a948d9acc
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt
@@ -0,0 +1,63 @@
+RISC-V Platform-Level Interrupt Controller (PLIC)
+-------------------------------------------------
+
+The RISC-V supervisor ISA specification allows for the presence of a
+platform-level interrupt contrellor (PLIC).   The PLIC connects all external
+interrupts in the system to all hart contexts in the system, via the external
+interrupt source in each hart's hart-local interrupt controller (HLIC).  A hart
+context is a priviledge mode in a hardware execution thread.  For example, in
+an 4 core system with 2-way SMT, you have 8 harts and probably at least two
+priviledge modes per hart; machine mode and supervisor mode.
+
+Each interrupt can be enabled on per-context basis. Any context can claim
+a pending enabled interrupt and then release it once it has been handled.
+
+Each interrupt has a configurable priority. Higher priority interrupts are
+serviced firs. Each context can specify a priority threshold. Interrupts
+with priority below this threshold will not cause the PLIC to raise its
+interrupt line leading to the context.
+
+While the PLIC supports both edge-triggered and level-triggered interrupts,
+interrupt handlers are oblivious to this distinction and therefor it is not
+specific in the PLIC device-tree binding.
+
+FIXME: I'm going to see if it's viable to change this.
+On RISC-V systems there is no physical hart ID availiable to programs running
+in supervisor mode, only a logical hart ID that is set by the bootloader.  As
+such, hart contexts are expected to be mostly contiguous.
+
+Required properties:
+- compatible : "riscv,plic0"
+- #address-cells : should be <0>
+- #interrupt-cells : should be <1>
+- interrupt-controller : Identifies the node as an interrupt controller
+- reg : Should contain 1 register range (address and length)
+- riscv,ndev : Specifies the maximum number of devices this PLIC is capable of
+  recieving interrupts from.  While this may usually be the same as the number
+  of devices the PLIC is actually connected to, there may be fewer devices
+  actually connected.  Software can be largely oblivious of this, as devices
+  that are not connected will never fire an interrupt.
+- interrupts-extended : Specifies which contexts are connected to the PLIC,
+  with "-1" specifying that a context is not present.
+
+Example:
+
+	plic: interrupt-controller@c000000 {
+		#address-cells = <0>;
+		#interrupt-cells = <1>;
+		compatible = "riscv,plic0";
+		interrupt-controller;
+		interrupts-extended = <
+			&cpu0-intc 11
+			&cpu1-intc 11 &cpu1-intc 9
+			&cpu2-intc 11 &cpu2-intc 9
+			&cpu3-intc 11 &cpu3-intc 9
+			&cpu4-intc 11 &cpu4-intc 9>;
+		reg = <0xc000000 0x4000000>;
+		riscv,ndev = <10>;
+	};
+
+While the RISC-V ISA doesn't specify a memory layout for the PLIC, the
+"riscv,plic0" device is a concrete implementation of the PLIC that contains a
+specific memory layout.  More details about the memory layout of the
+"riscv,plic0" device can be found as a comment in the device driver.
-- 
2.13.0

Re: [PATCH 2/3] dts: RISC-V local interrupt controller docs

[Rob Herring]

On Mon, Jun 26, 2017 at 10:21:23PM -0700, Palmer Dabbelt wrote:
> This patch adds documentation on the RISC-V local interrupt controller,
> which is a per-hart interrupt controller that manages all interrupts
> entering a RISC-V hart.  This interrupt controller is present on all
> RISC-V systems.

"dt-bindings: interrupt-controller: ..." for the subject. Similar for 
the other patches.

> 
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> ---
>  .../interrupt-controller/riscv,cpu-intc.txt        | 70 ++++++++++++++++++++++
>  1 file changed, 70 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> 
> diff --git a/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> new file mode 100644
> index 000000000000..f4906f49a1b4
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> @@ -0,0 +1,70 @@
> +RISC-V Hart-Level Interrupt Controller (HLIC)
> +---------------------------------------------
> +
> +RISC-V cores include Control Status Registers (CSRs) which are local to each
> +hart and can be read or written by software. Some of these CSRs are used to

What's a hart? Please define.

> +control local interrupts connected to the core.  Every interrupt is ultimately
> +routed through a hart's HLIC before it interrupts that hart.
> +
> +The RISC-V supervisor ISA manual specifices three interrupt sources that are
> +attached to every HLIC: software interrupts, the timer interrupt, and external
> +interrupts.  Software interrupts are used to send IPIs between cores.  The
> +timer interrupt comes from an architectually mandated real-time timer that is
> +controller via SBI calls and CSR reads.  External interrupts connect all other
> +device interrupts to the HLIC, which are routed via the platforw-level

s/platforw/platform/

> +interrupt contrellor (PLIC).

another typo

> +
> +All RISC-V systems that conform to the supervisor ISA specification are
> +required to have a HLIC with these three interrupt sources present.
> +
> +Required properties:
> +- compatible : "riscv,cpu-intc"

Only one version? 

> +- #interrupt-cells : should be <1>
> +- interrupt-controller : Identifies the node as an interrupt controller
> +
> +Furthermore, this interrupt-controller MUST be embedded inside the cpu
> +definition of the hart whose CSRs control these local interrupts.
> +
> +An example of a device tree entry for a standard RISC-V hart is as follows:
> +
> +	cpu1: cpu@1 {
> +		clock-frequency = <1600000000>;
> +		compatible = "riscv";

This is not really specific enough. You need cpu node documentation 
for this as well.

> +		d-cache-block-size = <64>;
> +		d-cache-sets = <64>;
> +		d-cache-size = <16384>;
> +		d-tlb-sets = <1>;
> +		d-tlb-size = <32>;
> +		device_type = "cpu";
> +		i-cache-block-size = <64>;
> +		i-cache-sets = <64>;
> +		i-cache-size = <16384>;
> +		i-tlb-sets = <1>;
> +		i-tlb-size = <32>;
> +		mmu-type = "riscv,sv39";

This and ...

> +		next-level-cache = <&L2>;
> +		reg = <1>;
> +		riscv,isa = "rv64imac";

... this should be implied by specific compatible strings.

> +		status = "okay";
> +		tlb-split;
> +		cpu1-intc: interrupt-controller {
> +			#interrupt-cells = <1>;
> +			compatible = "riscv,cpu-intc";
> +			interrupt-controller;
> +		};
> +	};
> +
> +The sections specifically relevant to the HLIC are
> +
> +	cpu1: cpu@1 {

Just remove the above and add this which is enough for the example:

		compatible = "riscv";
		...

> +		cpu1-intc: interrupt-controller {
> +			#interrupt-cells = <1>;
> +			compatible = "riscv,cpu-intc";
> +			interrupt-controller;
> +		};
> +	};
> +
> +As the HLIC is essentially a static interrupt controller, this device tree
> +entry essentially just indicates the presence of a HLIC.  All HLIC interrupts

Won't it also show the connections to the timer, PLIC, etc.?

> +are level-triggered, so there is no need to specify this as part of the device
> +tree.
> -- 
> 2.13.0
> 

Re: [PATCH 1/3] dts: RISC-V vendor prefix

[Rob Herring]

On Mon, Jun 26, 2017 at 10:21:22PM -0700, Palmer Dabbelt wrote:
> RISC-V systems use device tree to specify the memory layout of the
> system.  This patch reserves the "riscv" vendor prefix, which will be
> used for devices that are specified by the various RISC-V ISA
> specifications.
> 
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> ---
>  Documentation/devicetree/bindings/vendor-prefixes.txt | 1 +
>  1 file changed, 1 insertion(+)

Applied with subject fixup.

Rob

Re: [PATCH 3/3] dts: RISC-V PLIC documentation

[Rob Herring]

On Mon, Jun 26, 2017 at 10:21:24PM -0700, Palmer Dabbelt wrote:
> This patch adds documentation for the platform-level interrupt
> controller (PLIC) found in all RISC-V systems.  This interrupt
> controller routes interrupts from all the devices in the system to each
> hart-local interrupt controller.
> 
> Note: the DTS bindings for the PLIC aren't set in stone yet, as we might
> want to change how we're specifying holes in the hart list.
> 
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> ---
>  .../bindings/interrupt-controller/riscv,plic0.txt  | 63 ++++++++++++++++++++++
>  1 file changed, 63 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt
> 
> diff --git a/Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt b/Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt
> new file mode 100644
> index 000000000000..b51a948d9acc
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/interrupt-controller/riscv,plic0.txt
> @@ -0,0 +1,63 @@
> +RISC-V Platform-Level Interrupt Controller (PLIC)
> +-------------------------------------------------
> +
> +The RISC-V supervisor ISA specification allows for the presence of a
> +platform-level interrupt contrellor (PLIC).   The PLIC connects all external
> +interrupts in the system to all hart contexts in the system, via the external
> +interrupt source in each hart's hart-local interrupt controller (HLIC).  A hart
> +context is a priviledge mode in a hardware execution thread.  For example, in
> +an 4 core system with 2-way SMT, you have 8 harts and probably at least two
> +priviledge modes per hart; machine mode and supervisor mode.
> +
> +Each interrupt can be enabled on per-context basis. Any context can claim
> +a pending enabled interrupt and then release it once it has been handled.
> +
> +Each interrupt has a configurable priority. Higher priority interrupts are
> +serviced firs. Each context can specify a priority threshold. Interrupts

s/firs/first/

> +with priority below this threshold will not cause the PLIC to raise its
> +interrupt line leading to the context.
> +
> +While the PLIC supports both edge-triggered and level-triggered interrupts,
> +interrupt handlers are oblivious to this distinction and therefor it is not
> +specific in the PLIC device-tree binding.

s/specific/specified/

Quite a few typos in your series. Please spell check your stuff so I 
don't have to.

> +
> +FIXME: I'm going to see if it's viable to change this.
> +On RISC-V systems there is no physical hart ID availiable to programs running
> +in supervisor mode, only a logical hart ID that is set by the bootloader.  As
> +such, hart contexts are expected to be mostly contiguous.

How does this affect the binding? Or if this changes, how will it change 
the binding?

> +
> +Required properties:
> +- compatible : "riscv,plic0"
> +- #address-cells : should be <0>
> +- #interrupt-cells : should be <1>
> +- interrupt-controller : Identifies the node as an interrupt controller
> +- reg : Should contain 1 register range (address and length)
> +- riscv,ndev : Specifies the maximum number of devices this PLIC is capable of
> +  recieving interrupts from.  While this may usually be the same as the number
> +  of devices the PLIC is actually connected to, there may be fewer devices
> +  actually connected.  Software can be largely oblivious of this, as devices
> +  that are not connected will never fire an interrupt.

We generally don't have this sort of property for irqchips because of 
the reason in the last sentence. So drop this.

> +- interrupts-extended : Specifies which contexts are connected to the PLIC,
> +  with "-1" specifying that a context is not present.
> +
> +Example:
> +
> +	plic: interrupt-controller@c000000 {
> +		#address-cells = <0>;
> +		#interrupt-cells = <1>;
> +		compatible = "riscv,plic0";
> +		interrupt-controller;
> +		interrupts-extended = <
> +			&cpu0-intc 11
> +			&cpu1-intc 11 &cpu1-intc 9
> +			&cpu2-intc 11 &cpu2-intc 9
> +			&cpu3-intc 11 &cpu3-intc 9
> +			&cpu4-intc 11 &cpu4-intc 9>;
> +		reg = <0xc000000 0x4000000>;
> +		riscv,ndev = <10>;
> +	};
> +
> +While the RISC-V ISA doesn't specify a memory layout for the PLIC, the
> +"riscv,plic0" device is a concrete implementation of the PLIC that contains a
> +specific memory layout.  More details about the memory layout of the
> +"riscv,plic0" device can be found as a comment in the device driver.

Move this above required properties. Is there not a reference manual or 
something that's not kernel dependent? Bindings need to stand on their 
own without the kernel. 

Is "0" the best you can come up with for the specific implementation?

Rob

Re: [PATCH 1/3] dts: RISC-V vendor prefix

[Palmer Dabbelt]

On Thu, 29 Jun 2017 13:35:15 PDT (-0700), robh@kernel.org wrote:
> On Mon, Jun 26, 2017 at 10:21:22PM -0700, Palmer Dabbelt wrote:
>> RISC-V systems use device tree to specify the memory layout of the
>> system.  This patch reserves the "riscv" vendor prefix, which will be
>> used for devices that are specified by the various RISC-V ISA
>> specifications.
>>
>> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
>> ---
>>  Documentation/devicetree/bindings/vendor-prefixes.txt | 1 +
>>  1 file changed, 1 insertion(+)
>
> Applied with subject fixup.

Thanks!

Re: [PATCH 2/3] dts: RISC-V local interrupt controller docs

[Mark Rutland]

On Mon, Jun 26, 2017 at 10:21:23PM -0700, Palmer Dabbelt wrote:
> This patch adds documentation on the RISC-V local interrupt controller,
> which is a per-hart interrupt controller that manages all interrupts
> entering a RISC-V hart.  This interrupt controller is present on all
> RISC-V systems.
> 
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> ---
>  .../interrupt-controller/riscv,cpu-intc.txt        | 70 ++++++++++++++++++++++
>  1 file changed, 70 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> 
> diff --git a/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> new file mode 100644
> index 000000000000..f4906f49a1b4
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> @@ -0,0 +1,70 @@
> +RISC-V Hart-Level Interrupt Controller (HLIC)
> +---------------------------------------------
> +
> +RISC-V cores include Control Status Registers (CSRs) which are local to each
> +hart and can be read or written by software. Some of these CSRs are used to
> +control local interrupts connected to the core.  Every interrupt is ultimately
> +routed through a hart's HLIC before it interrupts that hart.
> +
> +The RISC-V supervisor ISA manual specifices three interrupt sources that are
> +attached to every HLIC: software interrupts, the timer interrupt, and external
> +interrupts.  Software interrupts are used to send IPIs between cores.  The
> +timer interrupt comes from an architectually mandated real-time timer that is
> +controller via SBI calls and CSR reads.  External interrupts connect all other
> +device interrupts to the HLIC, which are routed via the platforw-level
> +interrupt contrellor (PLIC).
> +
> +All RISC-V systems that conform to the supervisor ISA specification are
> +required to have a HLIC with these three interrupt sources present.
> +
> +Required properties:
> +- compatible : "riscv,cpu-intc"
> +- #interrupt-cells : should be <1>

I know this'll sound trivial, but it would be worth defining what that
cell encodes. I guess it corresponds to a particular HW(?) notion of
interrupt ID -- it would be worth calling out which one.

> +- interrupt-controller : Identifies the node as an interrupt controller
> +
> +Furthermore, this interrupt-controller MUST be embedded inside the cpu
> +definition of the hart whose CSRs control these local interrupts.
> +
> +An example of a device tree entry for a standard RISC-V hart is as follows:
> +
> +	cpu1: cpu@1 {
> +		clock-frequency = <1600000000>;
> +		compatible = "riscv";
> +		d-cache-block-size = <64>;
> +		d-cache-sets = <64>;
> +		d-cache-size = <16384>;
> +		d-tlb-sets = <1>;
> +		d-tlb-size = <32>;
> +		device_type = "cpu";
> +		i-cache-block-size = <64>;
> +		i-cache-sets = <64>;
> +		i-cache-size = <16384>;
> +		i-tlb-sets = <1>;
> +		i-tlb-size = <32>;
> +		mmu-type = "riscv,sv39";
> +		next-level-cache = <&L2>;
> +		reg = <1>;
> +		riscv,isa = "rv64imac";
> +		status = "okay";
> +		tlb-split;
> +		cpu1-intc: interrupt-controller {
> +			#interrupt-cells = <1>;
> +			compatible = "riscv,cpu-intc";
> +			interrupt-controller;
> +		};
> +	};

As with Rob's comments, you'll need to document the RISC-V cpu binding,
as we do for ARM in Documentation/devicetree/bindings/arm/cpus.txt. That
should describe the format of your reg entry, which common properties
are meaningful, and any RISC-V specific properties or values.

I also don't see anything like an enable-method. Is your SMP bringup
mechanism defined by the architecture?

Thanks,
Mark.