OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[David Russell]

Good morning,

I have a legacy project that runs on an OMAP2430 and is based on the linux-omap repo (tag v2.6.28-omap1).  I am in the process of upgrading the project to mainline v5.10; I have been able to get most issues resolved (e.g., USB DMA still fails, but PIO works for now), but have a difficult problem that I am hoping someone might lend an idea to.  We have a custom DSP driver that is responsible for resetting the DSP, downloading the firmware, setting up the mailbox and then managing the communication between the ARM and DSP.  I have the driver mostly ported (i.e., compiling) and am working through the behavior.  At probe-time there are several ioremap’s of IVA2.1 memory (0x5c800000 and 0x5d000000 — the internal SRAM and the MMU).  The ioremap’s succeed without error, but as soon as I simply try to read the revision register at 0x5d000000 the ARM hangs (i.e., not even a kernel panic message is presented, the console just freezes).  I assume that I am missing a clock or some module is powered-down, but am not sure.  Interestingly, in arch/arm/mach-omap2/iomap.h it is explicitly noted that the 2430 IVA2.1 is not mapped — I attempted to add these mappings to the 2430 table which resulted in zero console output of the kernel boot (i.e., I can see the x-loader console output getting ready to launch the kernel, and then nothing).

This is a challenging problem with the console simply freezing on the memory access.  I would very much appreciate any ideas as to this issue.

Regards,

Dave

RE: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[Woodruff, Richard]

Guess: this bit in JTAG script used for IVA tests probably is missing and needs to be worked in.  The generic linux-omap clock code probably handles the IVA clock but maybe not the resets.

   /*  Enable IVA-ss functional clock (set bit 0) */
   (*(int*)0x49006800) |= 0x1;

   /* Release l3s_idle_req  */
   (*(int*)0x49006810) |= (1 << 1);

   /* Release L3S reset and power-on reset (clear bit 1) at the same time */
   (*(int*)0x49006850) &= ~(( 1 << 1));

Regards,
Richard W.

-----Original Message-----
From: David Russell <david.russell73@gmail.com> 
Sent: Monday, June 07, 2021 10:01 AM
To: tony@atomide.com
Cc: linux-omap@vger.kernel.org
Subject: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

Good morning,

I have a legacy project that runs on an OMAP2430 and is based on the linux-omap repo (tag v2.6.28-omap1).  I am in the process of upgrading the project to mainline v5.10; I have been able to get most issues resolved (e.g., USB DMA still fails, but PIO works for now), but have a difficult problem that I am hoping someone might lend an idea to.  We have a custom DSP driver that is responsible for resetting the DSP, downloading the firmware, setting up the mailbox and then managing the communication between the ARM and DSP.  I have the driver mostly ported (i.e., compiling) and am working through the behavior.  At probe-time there are several ioremap’s of IVA2.1 memory (0x5c800000 and 0x5d000000 — the internal SRAM and the MMU).  The ioremap’s succeed without error, but as soon as I simply try to read the revision register at 0x5d000000 the ARM hangs (i.e., not even a kernel panic message is presented, the console just freezes).  I assume that I am missing a clock or some module is powered-down, but am not sure.  Interestingly, in arch/arm/mach-omap2/iomap.h it is explicitly noted that the 2430 IVA2.1 is not mapped — I attempted to add these mappings to the 2430 table which resulted in zero console output of the kernel boot (i.e., I can see the x-loader console output getting ready to launch the kernel, and then nothing).

This is a challenging problem with the console simply freezing on the memory access.  I would very much appreciate any ideas as to this issue.

Regards,

Dave

Re: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[Tony Lindgren]

Hi,

* Woodruff, Richard <r-woodruff2@ti.com> [210607 15:40]:
> Guess: this bit in JTAG script used for IVA tests probably is missing and needs to be worked in.  The generic linux-omap clock code probably handles the IVA clock but maybe not the resets.
> 
>    /*  Enable IVA-ss functional clock (set bit 0) */
>    (*(int*)0x49006800) |= 0x1;
> 
>    /* Release l3s_idle_req  */
>    (*(int*)0x49006810) |= (1 << 1);
> 
>    /* Release L3S reset and power-on reset (clear bit 1) at the same time */
>    (*(int*)0x49006850) &= ~(( 1 << 1));

Heh and I thought nobody is using 2430 any longer :)

FYI, the current mainline kernel actually can deal with all that using
reset driver and genpd, see for example commits:

ae57d1558908 ("ARM: dts: Configure interconnect target module for dra7 iva")
effe89e40037 ("soc: ti: omap-prm: Fix occasional abort on reset deassert for dra7 iva")

Similar setup should also work for 2430 but needs the power domains
configured for drivers/soc/ti/omap_prm.c at least for iva.

David, I think what you're seeing is iva getting released from reset with
an unconfigured MMU, and then the system will hang.

Regards,

Tony

Re: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[David Russell]

Good morning (where I am anyway),

Yeah it is definitely a dated part; it turns out the actual part is an OMAP2530, but it is effectively the same as the 2430 (not sure what the difference is but from a Linux perspective the 2430 works).  As I mentioned, the product has been around for a while (i.e., mid-2009) with a mature application (both ARM and DSP) that runs on Linux (2.6.28-omap1).  The company is moving forward with a new iteration of hardware that is still based on the 2530 (not sure why honestly) and they have requirements of BT and Wifi among other things.  Instead of trying to back port that functionality I thought it would be better to upgrade the kernel to mainline LTS.  In so doing I *have* noticed that the OMAP2xxx has some holes in the mainline, and I just figured that not many people use this part any more so regression testing of drivers has not been great.  I have been able to get past everything else (e.g., RFBI missing from mainline DISPC, USB DMA not working, device-tree mistakes on USB, etc.).

I am extremely grateful for this mailing list and people willing to respond to my question; thank you!  It turns out this particular problem was a bone-headed mistake on my part when porting the old DSP driver; previously there were calls to omap_readl() and omap_writel() to generically read or write some peripheral registers — these no longer exist in the mainline so I had to ioremap the base of SYSC and PRCM.  I then created #define’s for some registers.  The original driver referenced the initialization from a GEL file (code-composer studio) similar to what Richard had suggested.  It turns out my #defines were off so the calls to reset the DSP were incorrect.

In the event anyone is interested, the sequence that works is:

        /* Configure boot mode 0 and boot address at the beginning of SDRAM */
        write_sysc_reg(e, SYSC_OCP1_IVA2_BOOTADDR, DSP_VADDR_START);
        write_sysc_reg(e, SYSC_OCP1_IVA2_BOOTMOD, 0x0);

        /* Release L3S reset and perhaps power-on reset (clear bit 1) at the same time */
        write_prcm_reg(e, PRCM_RM_RSTCTRL_DSP, 0x1);

This allowed me to initialize the DSP MMU, then download the DSP binary via request firmware, followed by:

        /* Release DSP from reset */
        write_prcm_reg(e, PRCM_RM_RSTCTRL_DSP, 0x0);

        val = read_prcm_reg(e, PRCM_CM_IDLEST_DSP);

        if (val & 1) {
                dev_info(dev, "DSP is active\n");
        } else {
                dev_info(dev, "DSP is NOT active\n");
        }

Again, thank you for your quick responses; very appreciated.  I intend on submitting some patches in the near future to address some of the previously mentioned 2430-holes, especially affecting the USB.

Regards,

Dave

> On Jun 8, 2021, at 11:20 PM, Tony Lindgren <tony@atomide.com> wrote:
> 
> Hi,
> 
> * Woodruff, Richard <r-woodruff2@ti.com> [210607 15:40]:
>> Guess: this bit in JTAG script used for IVA tests probably is missing and needs to be worked in.  The generic linux-omap clock code probably handles the IVA clock but maybe not the resets.
>> 
>>   /*  Enable IVA-ss functional clock (set bit 0) */
>>   (*(int*)0x49006800) |= 0x1;
>> 
>>   /* Release l3s_idle_req  */
>>   (*(int*)0x49006810) |= (1 << 1);
>> 
>>   /* Release L3S reset and power-on reset (clear bit 1) at the same time */
>>   (*(int*)0x49006850) &= ~(( 1 << 1));
> 
> Heh and I thought nobody is using 2430 any longer :)
> 
> FYI, the current mainline kernel actually can deal with all that using
> reset driver and genpd, see for example commits:
> 
> ae57d1558908 ("ARM: dts: Configure interconnect target module for dra7 iva")
> effe89e40037 ("soc: ti: omap-prm: Fix occasional abort on reset deassert for dra7 iva")
> 
> Similar setup should also work for 2430 but needs the power domains
> configured for drivers/soc/ti/omap_prm.c at least for iva.
> 
> David, I think what you're seeing is iva getting released from reset with
> an unconfigured MMU, and then the system will hang.
> 
> Regards,
> 
> Tony

Re: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[Tony Lindgren]

* David Russell <david.russell73@gmail.com> [210609 12:11]:
> Again, thank you for your quick responses; very appreciated.  I intend on submitting some patches in the near future to address some of the previously mentioned 2430-holes, especially affecting the USB.

OK great, please do send patches.

Regards,

Tony

Re: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[David Russell]

Good Morning,

I have been upgrading a legacy project that runs on an OMAP2430 based on the linux-omap repo (tag v2.6.28-omap1) to mainline v5.10; I have been able to get the main application running, however I am concerned there are potentially many underlying issues I have not corrected nor even seen yet.  I recalled Tony’s comment that the part is dated (of course there is no argument from me there) to the point that I am not sure to what extent anyone has used the latest mainline on an OMAP2xxx part (i.e., regression testing).

I am hopeful someone has a good suggestion to the following question:  what mainline kernel version would be considered the most stable in regards to an OMAP2430 processor that may also include the TI WL18xx wifi driver?  Or if they do not overlap, simply what is the latest mainline kernel version that would be most stable for OMAP2xxx?  I am just afraid the v5.10 and v4.19 are so new that there are underlying issues that would take significant time to chase down; but if I can roll-back to an older version (yet newer than 2.6.28), that might get me closer to where I need to be.

Thanks for any suggestions,

Dave

> On Jun 8, 2021, at 11:20 PM, Tony Lindgren <tony@atomide.com> wrote:
> 
> Hi,
> 
> * Woodruff, Richard <r-woodruff2@ti.com> [210607 15:40]:
>> Guess: this bit in JTAG script used for IVA tests probably is missing and needs to be worked in.  The generic linux-omap clock code probably handles the IVA clock but maybe not the resets.
>> 
>>   /*  Enable IVA-ss functional clock (set bit 0) */
>>   (*(int*)0x49006800) |= 0x1;
>> 
>>   /* Release l3s_idle_req  */
>>   (*(int*)0x49006810) |= (1 << 1);
>> 
>>   /* Release L3S reset and power-on reset (clear bit 1) at the same time */
>>   (*(int*)0x49006850) &= ~(( 1 << 1));
> 
> Heh and I thought nobody is using 2430 any longer :)
> 
> FYI, the current mainline kernel actually can deal with all that using
> reset driver and genpd, see for example commits:
> 
> ae57d1558908 ("ARM: dts: Configure interconnect target module for dra7 iva")
> effe89e40037 ("soc: ti: omap-prm: Fix occasional abort on reset deassert for dra7 iva")
> 
> Similar setup should also work for 2430 but needs the power domains
> configured for drivers/soc/ti/omap_prm.c at least for iva.
> 
> David, I think what you're seeing is iva getting released from reset with
> an unconfigured MMU, and then the system will hang.
> 
> Regards,
> 
> Tony

RE: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[Woodruff, Richard]

Hello Dave,

It depends on what features you need to satisfy your use case (kernel + base + application).  Stable older kernels can do some tasks well enough if they have the features you need in place.  If you need newer features then what an older kernel provided, it can be a lot of work to back port.  Also if the kernel is used in open networks then how to handle security is also something to consider.

I've not booted my OMAP2430 board for several years, however, I do have some old tablets and phones I start from time to time.  The last kernel's I used on it were around the time frame of what you are using. It is very likely some newer snapshots were done on a per project basis.

Probably a good minimal kernel with basic drivers can be made at multiple points.  I'm less sure about the state of open drivers with WL18xx.  I'd suggest exploring that a bit more to see if it ends up dominating.  For early WLAN, sometimes firmware and fragile DMA integrations were hard to work around.

Regards,
Richard W.

-----Original Message-----
From: David Russell <david.russell73@gmail.com> 
Sent: Wednesday, August 11, 2021 7:54 AM
To: Tony Lindgren <tony@atomide.com>
Cc: Woodruff, Richard <r-woodruff2@ti.com>; linux-omap@vger.kernel.org
Subject: Re: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

Good Morning,

I have been upgrading a legacy project that runs on an OMAP2430 based on the linux-omap repo (tag v2.6.28-omap1) to mainline v5.10; I have been able to get the main application running, however I am concerned there are potentially many underlying issues I have not corrected nor even seen yet.  I recalled Tony’s comment that the part is dated (of course there is no argument from me there) to the point that I am not sure to what extent anyone has used the latest mainline on an OMAP2xxx part (i.e., regression testing).

I am hopeful someone has a good suggestion to the following question:  what mainline kernel version would be considered the most stable in regards to an OMAP2430 processor that may also include the TI WL18xx wifi driver?  Or if they do not overlap, simply what is the latest mainline kernel version that would be most stable for OMAP2xxx?  I am just afraid the v5.10 and v4.19 are so new that there are underlying issues that would take significant time to chase down; but if I can roll-back to an older version (yet newer than 2.6.28), that might get me closer to where I need to be.

Thanks for any suggestions,

Dave

> On Jun 8, 2021, at 11:20 PM, Tony Lindgren <tony@atomide.com> wrote:
> 
> Hi,
> 
> * Woodruff, Richard <r-woodruff2@ti.com> [210607 15:40]:
>> Guess: this bit in JTAG script used for IVA tests probably is missing and needs to be worked in.  The generic linux-omap clock code probably handles the IVA clock but maybe not the resets.
>> 
>>   /*  Enable IVA-ss functional clock (set bit 0) */
>>   (*(int*)0x49006800) |= 0x1;
>> 
>>   /* Release l3s_idle_req  */
>>   (*(int*)0x49006810) |= (1 << 1);
>> 
>>   /* Release L3S reset and power-on reset (clear bit 1) at the same time */
>>   (*(int*)0x49006850) &= ~(( 1 << 1));
> 
> Heh and I thought nobody is using 2430 any longer :)
> 
> FYI, the current mainline kernel actually can deal with all that using 
> reset driver and genpd, see for example commits:
> 
> ae57d1558908 ("ARM: dts: Configure interconnect target module for dra7 
> iva")
> effe89e40037 ("soc: ti: omap-prm: Fix occasional abort on reset 
> deassert for dra7 iva")
> 
> Similar setup should also work for 2430 but needs the power domains 
> configured for drivers/soc/ti/omap_prm.c at least for iva.
> 
> David, I think what you're seeing is iva getting released from reset 
> with an unconfigured MMU, and then the system will hang.
> 
> Regards,
> 
> Tony

Re: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[Tony Lindgren]

Hi,

* Woodruff, Richard <r-woodruff2@ti.com> [210811 14:18]:
> Probably a good minimal kernel with basic drivers can be made at multiple points.  I'm less sure about the state of open drivers with WL18xx.  I'd suggest exploring that a bit more to see if it ends up dominating.  For early WLAN, sometimes firmware and fragile DMA integrations were hard to work around.

The wl17xx wl18xx driver is behaving OK with various SoCs with the mainline
kernel at least as tested with SDIO.

For the Linux kernel, often the most recent long term kernel is used, that
is linux-5.10.y currently and might be linux-5.15.y at the end of the year.

If you just need basic SoC support and wl18xx, then I don't see issues
using current stable kernels. The SoC is fairly similar for peripherals
for various omap3 devices that are still being actively used :)

If you need to use out of tree hardware accelerators and and power
management, then things might get a bit trickier.

Regards,

Tony

RE: [EXTERNAL] OMAP2430 kernel hangs on ioremap of IVA2.1 addresses

[Woodruff, Richard]

Hello Tony,

> From: Tony Lindgren <tony@atomide.com>
> Sent: Thursday, August 12, 2021 1:32 AM

> The wl17xx wl18xx driver is behaving OK with various SoCs with the mainline
> kernel at least as tested with SDIO.
> 
> For the Linux kernel, often the most recent long term kernel is used, that is
> linux-5.10.y currently and might be linux-5.15.y at the end of the year.
> 
> If you just need basic SoC support and wl18xx, then I don't see issues using
> current stable kernels. The SoC is fairly similar for peripherals for various
> omap3 devices that are still being actively used :)
> 
> If you need to use out of tree hardware accelerators and and power
> management, then things might get a bit trickier.

Your recommendation makes sense to me.  Thanks for the good summary of the current usage.

Regards,
Richard W.