i.MX 7 boot freeze with 4.13-rc3

i.MX 7 boot freeze with 4.13-rc3

[Stefan Agner]

Hi,

Linux 4.13-rc3 seems to freeze on a Colibri iMX7:

...
8021q: 802.1Q VLAN Support v1.8
Key type dns_resolver registered
Registering SWP/SWPB emulation handler
vdd1p0d: supplied by DCDC3
asoc-simple-card sound: sgtl5000 <-> 308a0000.sai mapping ok
<freeze>

I started a bisect run and I had a hard time to get properly through
imx-dt-4.13. On the imx-dt branch the i.MX 7 GPC changes seem to cause
the UART to only output garbage:

...
io scheduler mq-deadline registered
io scheduler kyber registered
imx7-pgc imx7-pgc-domain.1: Failed to get domain's regulator
�������������������������������������������������~88����������������

It turned out at the merge point 2cb6115deba0 (Merge tag 'imx-dt-4.13'
of...) the behavior changed to the freeze seen in 4.13-rc3.

Reverting 0f90b43ac71b (ARM: dts: imx7s: Add node for GPC) helped in
both cases, and when using v4.13-rc3 (I also had to revert a816d5750edf
and 34adfaa3d404 which depend on this GPC change).

I assume that the merge adds driver changes which changes the behavior
of the GPC driver, hence freeze vs. garbage?

At the moment it is unclear to me why the GPC change leads to freezes on
our platform. On which platform have this changes been tested? Any other
i.MX 7 platforms seeing freezes?

--
Stefan

Re: i.MX 7 boot freeze with 4.13-rc3

[Andrey Smirnov]

On Tue, Aug 1, 2017 at 7:38 PM, Stefan Agner <stefan@agner.ch> wrote:
> Hi,
>
> Linux 4.13-rc3 seems to freeze on a Colibri iMX7:
>

Hi Stefan, sorry to hear about that.

> ...
> 8021q: 802.1Q VLAN Support v1.8
> Key type dns_resolver registered
> Registering SWP/SWPB emulation handler
> vdd1p0d: supplied by DCDC3

>From looking briefly at the Colibir iMX7's device tree file and given
the log line above it seems that the board has slightly different
"vdd1p0d" power supply configuration, so it might be a good starting
point to investigate.

> asoc-simple-card sound: sgtl5000 <-> 308a0000.sai mapping ok
> <freeze>
>

Not sure, but this might be a point where GPC driver will get probed
(drivers/soc/imx/gpcv2.c) and it's possible that some of the code I
wrote does not handle failure cases well.

> I started a bisect run and I had a hard time to get properly through
> imx-dt-4.13. On the imx-dt branch the i.MX 7 GPC changes seem to cause
> the UART to only output garbage:
>

I don't really have a good theory WRT UART, unfortunately.

> ...
> io scheduler mq-deadline registered
> io scheduler kyber registered
> imx7-pgc imx7-pgc-domain.1: Failed to get domain's regulator
>

The line above would be another clue in favor of differences in
"vdd1p0d" configuration somehow affecting things, because
"imx7-pgc-domain.1' is a PCIe power domain and it uses "vdd1p0d" as a
power supply.

�������������������������������������������������~88����������������
>
> It turned out at the merge point 2cb6115deba0 (Merge tag 'imx-dt-4.13'
> of...) the behavior changed to the freeze seen in 4.13-rc3.
>
> Reverting 0f90b43ac71b (ARM: dts: imx7s: Add node for GPC) helped in
> both cases, and when using v4.13-rc3 (I also had to revert a816d5750edf
> and 34adfaa3d404 which depend on this GPC change).
>
> I assume that the merge adds driver changes which changes the behavior
> of the GPC driver, hence freeze vs. garbage?
>
> At the moment it is unclear to me why the GPC change leads to freezes on
> our platform. On which platform have this changes been tested?

AFAIK it was tested with imx7-sdb as well as imx7d-cl-som.

> Any other i.MX 7 platforms seeing freezes?
>

I don't have a PSCI enabled bootloader handy right now, so I couldn't
test it in a true SMP configuration, but I ran 4.13-rc3 on my SabreSD
board and I don't think I am affected by the issue.

Hope this helps.

Thanks,
Andrey Smirnov

Re: i.MX 7 boot freeze with 4.13-rc3

[Stefan Agner]

Hi Andrey,

On 2017-08-02 09:57, Andrey Smirnov wrote:
> On Tue, Aug 1, 2017 at 7:38 PM, Stefan Agner <stefan@agner.ch> wrote:
>> Hi,
>>
>> Linux 4.13-rc3 seems to freeze on a Colibri iMX7:
>>
> 
> Hi Stefan, sorry to hear about that.
> 
>> ...
>> 8021q: 802.1Q VLAN Support v1.8
>> Key type dns_resolver registered
>> Registering SWP/SWPB emulation handler
>> vdd1p0d: supplied by DCDC3
> 
> From looking briefly at the Colibir iMX7's device tree file and given
> the log line above it seems that the board has slightly different
> "vdd1p0d" power supply configuration, so it might be a good starting
> point to investigate.
> 

That was a helpful hint, removing the vdd1p0d supply-in seems to fix the
issue. The supply seems to cause deferred probe which seems to create
the issue.

>> asoc-simple-card sound: sgtl5000 <-> 308a0000.sai mapping ok
>> <freeze>
>>
> 
> Not sure, but this might be a point where GPC driver will get probed
> (drivers/soc/imx/gpcv2.c) and it's possible that some of the code I
> wrote does not handle failure cases well.
> 
>> I started a bisect run and I had a hard time to get properly through
>> imx-dt-4.13. On the imx-dt branch the i.MX 7 GPC changes seem to cause
>> the UART to only output garbage:
>>
> 
> I don't really have a good theory WRT UART, unfortunately.
> 
>> ...
>> io scheduler mq-deadline registered
>> io scheduler kyber registered
>> imx7-pgc imx7-pgc-domain.1: Failed to get domain's regulator
>>
> 
> The line above would be another clue in favor of differences in
> "vdd1p0d" configuration somehow affecting things, because
> "imx7-pgc-domain.1' is a PCIe power domain and it uses "vdd1p0d" as a
> power supply.
> 
> �������������������������������������������������~88����������������
>>
>> It turned out at the merge point 2cb6115deba0 (Merge tag 'imx-dt-4.13'
>> of...) the behavior changed to the freeze seen in 4.13-rc3.
>>
>> Reverting 0f90b43ac71b (ARM: dts: imx7s: Add node for GPC) helped in
>> both cases, and when using v4.13-rc3 (I also had to revert a816d5750edf
>> and 34adfaa3d404 which depend on this GPC change).
>>
>> I assume that the merge adds driver changes which changes the behavior
>> of the GPC driver, hence freeze vs. garbage?
>>
>> At the moment it is unclear to me why the GPC change leads to freezes on
>> our platform. On which platform have this changes been tested?
> 
> AFAIK it was tested with imx7-sdb as well as imx7d-cl-som.

Ok, since both do not use a supply-in that makes sense.

Further debugging unveiled that imx7_pgc_domain_probe executes
pm_genpd_init twice when the regulator requests deferred probing. The
easiest fix seems to reorder pm_genpd_init vs.
devm_regulator_get_optional. With that I can keep boot with supply-in
specified. I will send out a patch soon.

Thanks!

--
Stefan