[PATCH] arm64: tegra: only map accessible sysram

[PATCH] arm64: tegra: only map accessible sysram

[Mian Yousaf Kaukab]

Most of the SysRAM is secure and only accessible by TF-A.
Don't map this inaccessible memory in kernel. Only map pages
used by bpmp driver.

Signed-off-by: Mian Yousaf Kaukab <ykaukab@suse.de>
---
Only tegra186 is tested. Tested on Jetson TX2.

 arch/arm64/boot/dts/nvidia/tegra186.dtsi | 14 +++++++-------
 arch/arm64/boot/dts/nvidia/tegra194.dtsi | 14 +++++++-------
 2 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/arch/arm64/boot/dts/nvidia/tegra186.dtsi b/arch/arm64/boot/dts/nvidia/tegra186.dtsi
index 47cd831fcf44..a861f46125fd 100644
--- a/arch/arm64/boot/dts/nvidia/tegra186.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra186.dtsi
@@ -1174,23 +1174,23 @@
 		power-domains = <&bpmp TEGRA186_POWER_DOMAIN_GPU>;
 	};
 
-	sysram@30000000 {
+	sysram@3004e000 {
 		compatible = "nvidia,tegra186-sysram", "mmio-sram";
-		reg = <0x0 0x30000000 0x0 0x50000>;
+		reg = <0x0 0x3004e000 0x0 0x2000>;
 		#address-cells = <2>;
 		#size-cells = <2>;
-		ranges = <0 0x0 0x0 0x30000000 0x0 0x50000>;
+		ranges = <0 0x0 0x0 0x3004e000 0x0 0x2000>;
 
-		cpu_bpmp_tx: shmem@4e000 {
+		cpu_bpmp_tx: shmem@e000 {
 			compatible = "nvidia,tegra186-bpmp-shmem";
-			reg = <0x0 0x4e000 0x0 0x1000>;
+			reg = <0x0 0x0 0x0 0x1000>;
 			label = "cpu-bpmp-tx";
 			pool;
 		};
 
-		cpu_bpmp_rx: shmem@4f000 {
+		cpu_bpmp_rx: shmem@f000 {
 			compatible = "nvidia,tegra186-bpmp-shmem";
-			reg = <0x0 0x4f000 0x0 0x1000>;
+			reg = <0x0 0x1000 0x0 0x1000>;
 			label = "cpu-bpmp-rx";
 			pool;
 		};
diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
index 3c0cf54f0aab..98b9399d6b7f 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
@@ -1430,23 +1430,23 @@
 			  0x82000000 0x0  0x40000000 0x1f 0x40000000 0x0 0xc0000000>; /* non-prefetchable memory (3GB) */
 	};
 
-	sysram@40000000 {
+	sysram@4004e000 {
 		compatible = "nvidia,tegra194-sysram", "mmio-sram";
-		reg = <0x0 0x40000000 0x0 0x50000>;
+		reg = <0x0 0x4004e000 0x0 0x2000>;
 		#address-cells = <1>;
 		#size-cells = <1>;
-		ranges = <0x0 0x0 0x40000000 0x50000>;
+		ranges = <0x0 0x0 0x4004e000 0x2000>;
 
-		cpu_bpmp_tx: shmem@4e000 {
+		cpu_bpmp_tx: shmem@e000 {
 			compatible = "nvidia,tegra194-bpmp-shmem";
-			reg = <0x4e000 0x1000>;
+			reg = <0x0 0x1000>;
 			label = "cpu-bpmp-tx";
 			pool;
 		};
 
-		cpu_bpmp_rx: shmem@4f000 {
+		cpu_bpmp_rx: shmem@f000 {
 			compatible = "nvidia,tegra194-bpmp-shmem";
-			reg = <0x4f000 0x1000>;
+			reg = <0x1000 0x1000>;
 			label = "cpu-bpmp-rx";
 			pool;
 		};
-- 
2.16.4

Re: [PATCH] arm64: tegra: only map accessible sysram

[Stephen Warren]

On 9/29/19 2:08 PM, Mian Yousaf Kaukab wrote:
> Most of the SysRAM is secure and only accessible by TF-A.
> Don't map this inaccessible memory in kernel. Only map pages
> used by bpmp driver.

I don't believe this change is correct. The actual patch doesn't
implement mapping a subset of the RAM (a software issue), but rather it
changes the DT representation of the SYSRAM hardware. The SYSRAM
hardware always does start at 0x30000000, even if a subset of the
address range is dedicated to a specific purpose. If the kernel must map
only part of the RAM, then some additional property should indicate
this. Also, I believe it's incorrect to hard-code into the kernel's DT
the range of addresses used by the secure monitor/OS, since this can
vary depending on what the user actually chooses to install as the
secure monitor/OS. Any indication of such regions should be filled in at
runtime by some boot firmware or the secure monitor/OS itself, or
retrieved using some runtime API rather than DT.

Re: [PATCH] arm64: tegra: only map accessible sysram

[Mian Yousaf Kaukab]

On Sun, Sep 29, 2019 at 11:28:43PM -0600, Stephen Warren wrote:
> On 9/29/19 2:08 PM, Mian Yousaf Kaukab wrote:
> > Most of the SysRAM is secure and only accessible by TF-A.
> > Don't map this inaccessible memory in kernel. Only map pages
> > used by bpmp driver.
> 
> I don't believe this change is correct. The actual patch doesn't
> implement mapping a subset of the RAM (a software issue), but rather it
> changes the DT representation of the SYSRAM hardware. The SYSRAM
> hardware always does start at 0x30000000, even if a subset of the
> address range is dedicated to a specific purpose. If the kernel must map
> only part of the RAM, then some additional property should indicate
> this.[...]
I agree the hardware description becomes inaccurate with this change.

In the current setup complete 0x3000_0000 to 0x3005_0000 range is being mapped
as normal memory (MT_NORMAL_NC). Though only 0x3004_E000 to 0x3005_0000 are
accessible by the kernel. I am seeing an issue where a read access (which I
believe is speculative) to inaccessible range causes an SError. Another 
solution for this problem could be to add "no-memory-wc" to SysRAM node so that
it is mapped as device memory (MT_DEVICE_nGnRE). Would that be acceptable?

> [...] Also, I believe it's incorrect to hard-code into the kernel's DT
> the range of addresses used by the secure monitor/OS, since this can
> vary depending on what the user actually chooses to install as the
> secure monitor/OS. Any indication of such regions should be filled in at
> runtime by some boot firmware or the secure monitor/OS itself, or
> retrieved using some runtime API rather than DT.
Secure-OS addresses are not of interest here. SysRAM is partitioned
between secure-OS and BPMP and kernel is only interested in the BPMP
part. The firmware can update these addresses in the device-tree if it
wants to. Would you prefer something similar implemented in u-boot so
that it updates SysRAM node to only expose kernel accessible part of it
to the kernel?

Can u-boot dynamically figure out the Secure-OS vs BPMP partition?

BR,
Yousaf

Re: [PATCH] arm64: tegra: only map accessible sysram

[Stephen Warren]

On 9/30/19 4:02 AM, Mian Yousaf Kaukab wrote:
> On Sun, Sep 29, 2019 at 11:28:43PM -0600, Stephen Warren wrote:
>> On 9/29/19 2:08 PM, Mian Yousaf Kaukab wrote:
>>> Most of the SysRAM is secure and only accessible by TF-A.
>>> Don't map this inaccessible memory in kernel. Only map pages
>>> used by bpmp driver.
>>
>> I don't believe this change is correct. The actual patch doesn't
>> implement mapping a subset of the RAM (a software issue), but rather it
>> changes the DT representation of the SYSRAM hardware. The SYSRAM
>> hardware always does start at 0x30000000, even if a subset of the
>> address range is dedicated to a specific purpose. If the kernel must map
>> only part of the RAM, then some additional property should indicate
>> this.[...]
> I agree the hardware description becomes inaccurate with this change.
> 
> In the current setup complete 0x3000_0000 to 0x3005_0000 range is being mapped
> as normal memory (MT_NORMAL_NC). Though only 0x3004_E000 to 0x3005_0000 are
> accessible by the kernel.

Nit: I expect that a much larger region than that is *accessible*, 
although it's quite plausible that only that region is actually 
*accessed*/used right now.

> I am seeing an issue where a read access (which I
> believe is speculative) to inaccessible range causes an SError. Another
> solution for this problem could be to add "no-memory-wc" to SysRAM node so that
> it is mapped as device memory (MT_DEVICE_nGnRE). Would that be acceptable?

Why does the driver blindly map the entire memory at all? Surely it 
should only map the portions of RAM that other drivers request/use? And 
surely the BPMP driver or DT node is already providing that information?

But yes, changing the mapping type to avoid speculation might be an 
acceptable solution for now, although I think we'd want to work things 
out better later. I don't know if there would be any impact to the BPMP 
driver related to the slower SRAM access due to this change. Best 
consult a BPMP expert or Tegra maintainer about that.

>> [...] Also, I believe it's incorrect to hard-code into the kernel's DT
>> the range of addresses used by the secure monitor/OS, since this can
>> vary depending on what the user actually chooses to install as the
>> secure monitor/OS. Any indication of such regions should be filled in at
>> runtime by some boot firmware or the secure monitor/OS itself, or
>> retrieved using some runtime API rather than DT.
> Secure-OS addresses are not of interest here. SysRAM is partitioned
> between secure-OS and BPMP and kernel is only interested in the BPMP
> part. The firmware can update these addresses in the device-tree if it
> wants to. Would you prefer something similar implemented in u-boot so
> that it updates SysRAM node to only expose kernel accessible part of it
> to the kernel?
> 
> Can u-boot dynamically figure out the Secure-OS vs BPMP partition?
> 
> BR,
> Yousaf
>