External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

[Bryan O'Donoghue]

Hi.

I've been poking a problem on an ipq6018 board I have. Specifically it 
pertains to the address range for GPIO # 20, touching any address in the 
range 0x14000 - 0x14FFC results in data-abort.

I initially encountered this with the upstream kernel here:
https://paste.debian.net/1225390/

and assumed that I was doing something wrong, so went hunting for 
evidence that the downstream kernel touched the registers in that range, 
my assumption that something extra was happening the downstream kernel 
to make the register read kosher, turned out to be wrong, assuming 
something like the transition from arm32 to arm64 or a trustzone setting 
getting in the way.

Downstream doesn't touch that register with the default drivers and 
shipping DTS files.

Modifying the downstream kernel to do a simple read of each ctrl_reg 
makes downstream choke on the address too.

This is the only bootable version of the downstream kernel reading every 
ctrl_reg in the range of gpio0 - gpio79 - skipping gpio20 since even 
downstream the address range 0x14000 - 0x14FFC is broken.

In other words I can prove the problem is only with the physical range 
0x01014000 - 0x01014FFC

gpio19 @ 01013000 - 01013FFC is fine
gpio21 @ 01015000 - 01015FFC is fine

the issue is limited to gpio20 @ 0x01014000 - 0x01014FFC

Downstream kernel:
https://paste.debian.net/1225388/

This is also the case when doing a dumb dump in u-boot
Stock u-boot:

https://paste.debian.net/1225389/
https://paste.debian.net/1225396/

Is there a sku of this SoC which perhaps reserves 0x14000 - 0x14FFC or a 
version of the firmware that does so ?

I've already discussed this offline with Baruch who suggested firmware 
versions based on previous feedback working with this part, however the 
firmware version on the board is newer than what was suggested here

https://lore.kernel.org/all/5ba36dcd7c50b640e0bdaf5b9c2248f3@codeaurora.org/

"Please try with TZ version >= TZ.WNS.5.1-00084."

admin@OpenWrt:~$ cat /proc/device-tree/tz_version
CRM-TZ.WNS.5.1-00113
admin@OpenWrt:~$ cat /proc/device-tree/tz_version
CRM-TZ.WNS.5.1-00113
admin@OpenWrt:~$ cat /proc/device-tree/rpm_version
CRM-RPM.BF.2.4.1-00089
admin@OpenWrt:~$  cat /proc/device-tree/boot_version
crm-ubuntu47-BOOT.XF.0.3-00090-IPQ60xxLZB-2

I'm not far off writing a patch based on a "compatible" string to mark 
gpio # 20 as bad - any other ideas or insights welcome though.

---
bod

Re: External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

[Bjorn Andersson]

On Fri 31 Dec 09:05 PST 2021, Bryan O'Donoghue wrote:

> Hi.
> 
> I've been poking a problem on an ipq6018 board I have. Specifically it
> pertains to the address range for GPIO # 20, touching any address in the
> range 0x14000 - 0x14FFC results in data-abort.
> 

This problem exists on pretty much all Qualcomm platforms.

> I initially encountered this with the upstream kernel here:
> https://paste.debian.net/1225390/
> 
> and assumed that I was doing something wrong, so went hunting for evidence
> that the downstream kernel touched the registers in that range, my
> assumption that something extra was happening the downstream kernel to make
> the register read kosher, turned out to be wrong, assuming something like
> the transition from arm32 to arm64 or a trustzone setting getting in the
> way.
> 
> Downstream doesn't touch that register with the default drivers and shipping
> DTS files.
> 

Iirc the change that introduced this "regression" is 3edfb7bd76bd
("gpiolib: Show correct direction from the beginning"), which seems to
have been introduced in 4.20. So any downstream older than that will
typically boot just fine.

But I believe cat /sys/kernel/debug/gpio will crash the system still.

> Modifying the downstream kernel to do a simple read of each ctrl_reg makes
> downstream choke on the address too.
> 
> This is the only bootable version of the downstream kernel reading every
> ctrl_reg in the range of gpio0 - gpio79 - skipping gpio20 since even
> downstream the address range 0x14000 - 0x14FFC is broken.
> 
> In other words I can prove the problem is only with the physical range
> 0x01014000 - 0x01014FFC
> 
> gpio19 @ 01013000 - 01013FFC is fine
> gpio21 @ 01015000 - 01015FFC is fine
> 
> the issue is limited to gpio20 @ 0x01014000 - 0x01014FFC
> 
> Downstream kernel:
> https://paste.debian.net/1225388/
> 
> This is also the case when doing a dumb dump in u-boot
> Stock u-boot:
> 
> https://paste.debian.net/1225389/
> https://paste.debian.net/1225396/
> 
> Is there a sku of this SoC which perhaps reserves 0x14000 - 0x14FFC or a
> version of the firmware that does so ?
> 
> I've already discussed this offline with Baruch who suggested firmware
> versions based on previous feedback working with this part, however the
> firmware version on the board is newer than what was suggested here
> 
> https://lore.kernel.org/all/5ba36dcd7c50b640e0bdaf5b9c2248f3@codeaurora.org/
> 
> "Please try with TZ version >= TZ.WNS.5.1-00084."
> 

The reason for seeing this issue is typically that the pins are related
to some I2C bus that is used for some secure application, i.e. they are
reserved for TZ to use.

As such I don't think you can use the >= comparator to determine if the
firmware exposes you to this problem or not, because it will depend on
if the particular release branch has such features enabled or not.

> admin@OpenWrt:~$ cat /proc/device-tree/tz_version
> CRM-TZ.WNS.5.1-00113
> admin@OpenWrt:~$ cat /proc/device-tree/tz_version
> CRM-TZ.WNS.5.1-00113
> admin@OpenWrt:~$ cat /proc/device-tree/rpm_version
> CRM-RPM.BF.2.4.1-00089
> admin@OpenWrt:~$  cat /proc/device-tree/boot_version
> crm-ubuntu47-BOOT.XF.0.3-00090-IPQ60xxLZB-2
> 
> I'm not far off writing a patch based on a "compatible" string to mark gpio
> # 20 as bad - any other ideas or insights welcome though.
> 

Unfortunately I don't know how to determine the list of protected GPIOs,
other than trial and error.

But once you have the list, you can use "gpio-reserved-ranges" (a series
of offset, count pairs) in the TLMM DT node to mark off these gpios as
reserved.

Regards,
Bjorn

Re: External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

[Linus Walleij]

On Fri, Dec 31, 2021 at 10:17 PM Bjorn Andersson
<bjorn.andersson@linaro.org> wrote:
> On Fri 31 Dec 09:05 PST 2021, Bryan O'Donoghue wrote:

> The reason for seeing this issue is typically that the pins are related
> to some I2C bus that is used for some secure application, i.e. they are
> reserved for TZ to use.

In the best of all worlds TZ should be telling us what it is using
for itself.

> Unfortunately I don't know how to determine the list of protected GPIOs,
> other than trial and error.
>
> But once you have the list, you can use "gpio-reserved-ranges" (a series
> of offset, count pairs) in the TLMM DT node to mark off these gpios as
> reserved.

This is the right thing to do.

If I'm not mistaken the TZ can modify the device tree during boot just
as well as the boot loader can, or it can tell the boot loader to modify
the device tree, and add reserved GPIOs to the
gpio-reserved-ranges.

I think the real problem is that some system manager didn't think
about resource reservation even though we have mechanisms in
place to do them.

Yours,
Linus Walleij

Re: External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

[Bryan O'Donoghue]

On 31/12/2021 21:18, Bjorn Andersson wrote:
> Unfortunately I don't know how to determine the list of protected GPIOs,
> other than trial and error.
> 
> But once you have the list, you can use "gpio-reserved-ranges" (a series
> of offset, count pairs) in the TLMM DT node to mark off these gpios as
> reserved.

Ah nice, worked first time.

gpio-reserved-ranges = <20 4>;

Well I guess its good to know there's no more esoteric explanation than 
"trustzone made it explode" - it only seems to be the one GPIO too, 
quite what use there is for TZ in this GPIO

PINGROUP(20, pwm20, atest_char2, _, _, _, _, _, _, _)

is anybody's guess.

Good enough for my purposes.

Thanks !

---
bod

Re: External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

[Kathiravan T]

On 2022-01-01 21:17, Bryan O'Donoghue wrote:
> On 31/12/2021 21:18, Bjorn Andersson wrote:
>> Unfortunately I don't know how to determine the list of protected 
>> GPIOs,
>> other than trial and error.
>> 
>> But once you have the list, you can use "gpio-reserved-ranges" (a 
>> series
>> of offset, count pairs) in the TLMM DT node to mark off these gpios as
>> reserved.
> 
> Ah nice, worked first time.
> 
> gpio-reserved-ranges = <20 4>;


You want to protect only one GPIO right? I understand that it should be 
just <20 1>. Isn't the case?

You are using a CP01-C4 variant(#define CPU_IPQ6005 453, machid 
0x8030003). In this variant alone, GPIO20 is protected by TZ for some of 
its operation since there is no PMIC. In other variants of IPQ6018, you 
shouldn't be seeing this issue.

> 
> Well I guess its good to know there's no more esoteric explanation
> than "trustzone made it explode" - it only seems to be the one GPIO
> too, quite what use there is for TZ in this GPIO
> 
> PINGROUP(20, pwm20, atest_char2, _, _, _, _, _, _, _)
> 
> is anybody's guess.
> 
> Good enough for my purposes.
> 
> Thanks !
> 
> ---
> bod

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member of Code Aurora Forum, hosted by The Linux Foundation

Re: External data abort on ipq6018 gpio # 20 ~ offset 0x14000 any ideas ?

[Bryan O'Donoghue]

On 03/01/2022 05:47, Kathiravan T wrote:
> You want to protect only one GPIO right? I understand that it should be 
> just <20 1>. Isn't the case?
> 
> You are using a CP01-C4 variant(#define CPU_IPQ6005 453, machid 
> 0x8030003). In this variant alone, GPIO20 is protected by TZ for some of 
> its operation since there is no PMIC. In other variants of IPQ6018, you 
> shouldn't be seeing this issue.

Ah awesome.

Thanks for the extra information - yes the PCB is a CP01-C4 derived part.

Appreciate the confirmation.

---
bod