Question/issues about i.MX6 DDR configuration

Question/issues about i.MX6 DDR configuration

[Francesco Dolcini]

Hello Fabio, Tim and all,
in the last few weeks I have been debugging some sporadic i.MX6 board
boot failures (2020.07 U-Boot, if that matters) and we have some loose
indication that they could be RAM related. The effect is that SPL is not
able to load U-Boot from the eMMC to DDR and dqs|write_level calibration
is failing.

We do write the memory configuration in a pretty simple way, SPL is just
iterating thought a list of register address/value, in a very similar
way to what was done using the DCD table. Today most of the boards
however use a more programmatic approach as introduced by Tim in 2014 [1].

Contrary to that, however, Fabio moved away from this new approach to
just raw registers writing for the sabre board [2][3].

I'm a little bit puzzled at the moment, according to the iMX6 reference
manual[4], 44.4.2 MMDC initialization, a specific sequence is required
to be followed and this is implemented by the `mx6_dram_cfg()`[5]
function, but according to what Fabio wrote the raw initialization of
registers was just more reliable for mx6sabresd. Fabio, what was the
reason?

I would expect that using `mx6_dram_cfg()` is the correct approach and I
would expect issue if doing in a different way from what is described in
the reference manual. Regarding that I found also this note in the
manual interesting:

"A Precharge All command must be issued prior to the
MRW command to ensure robust DDR initialization. This
command is required to be issued to both chip selects if two
chip selects are utilized in the system."

In my tests adding 1ms delay after each MMDC register write seems to
have a positive effect and this is going into the direction that using
`mx6_dram_cfg()` is the way to go.

Any suggestion? Do I miss something?

Francesco

[1] https://lore.kernel.org/all/1401750807-5975-7-git-send-email-tharvey@gateworks.com/
[2] https://lore.kernel.org/all/1474892066-32005-1-git-send-email-festevam@gmail.com/
[3] https://source.denx.de/u-boot/u-boot/-/commit/3b30eece271cfc4096c2d20048c89e8bed0bbbfd
[4] https://www.nxp.com/webapp/Download?colCode=IMX6DQRM
[5] https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/mach-imx/mx6/ddr.c#L1672

Re: Question/issues about i.MX6 DDR configuration

[Fabio Estevam]

Hi Francesco,

On Thu, Dec 2, 2021 at 1:14 PM Francesco Dolcini
<francesco.dolcini@toradex.com> wrote:
>
> Hello Fabio, Tim and all,
> in the last few weeks I have been debugging some sporadic i.MX6 board
> boot failures (2020.07 U-Boot, if that matters) and we have some loose
> indication that they could be RAM related. The effect is that SPL is not
> able to load U-Boot from the eMMC to DDR and dqs|write_level calibration
> is failing.
>
> We do write the memory configuration in a pretty simple way, SPL is just
> iterating thought a list of register address/value, in a very similar
> way to what was done using the DCD table. Today most of the boards
> however use a more programmatic approach as introduced by Tim in 2014 [1].
>
> Contrary to that, however, Fabio moved away from this new approach to
> just raw registers writing for the sabre board [2][3].
>
> I'm a little bit puzzled at the moment, according to the iMX6 reference
> manual[4], 44.4.2 MMDC initialization, a specific sequence is required
> to be followed and this is implemented by the `mx6_dram_cfg()`[5]
> function, but according to what Fabio wrote the raw initialization of
> registers was just more reliable for mx6sabresd. Fabio, what was the
> reason?

The reason is that I wanted to keep the DDR initialization 100% the
same in U-Boot
mainline versus NXP U-Boot for mx6sabresd board.

The mx6_ddr_sysinfo approach is more elegant, for sure.

It is just that I wanted to keep 100% in sync with the initialization
done by the NXP hardware team.

Regards,

Fabio Estevam

Re: Question/issues about i.MX6 DDR configuration

[Michael Nazzareno Trimarchi]

Hi Fabio

On Thu, Dec 2, 2021 at 9:14 PM Fabio Estevam <festevam@gmail.com> wrote:
>
> Hi Francesco,
>
> On Thu, Dec 2, 2021 at 1:14 PM Francesco Dolcini
> <francesco.dolcini@toradex.com> wrote:
> >
> > Hello Fabio, Tim and all,
> > in the last few weeks I have been debugging some sporadic i.MX6 board
> > boot failures (2020.07 U-Boot, if that matters) and we have some loose
> > indication that they could be RAM related. The effect is that SPL is not
> > able to load U-Boot from the eMMC to DDR and dqs|write_level calibration
> > is failing.
> >
> > We do write the memory configuration in a pretty simple way, SPL is just
> > iterating thought a list of register address/value, in a very similar
> > way to what was done using the DCD table. Today most of the boards
> > however use a more programmatic approach as introduced by Tim in 2014 [1].
> >
> > Contrary to that, however, Fabio moved away from this new approach to
> > just raw registers writing for the sabre board [2][3].
> >
> > I'm a little bit puzzled at the moment, according to the iMX6 reference
> > manual[4], 44.4.2 MMDC initialization, a specific sequence is required
> > to be followed and this is implemented by the `mx6_dram_cfg()`[5]
> > function, but according to what Fabio wrote the raw initialization of
> > registers was just more reliable for mx6sabresd. Fabio, what was the
> > reason?
>
> The reason is that I wanted to keep the DDR initialization 100% the
> same in U-Boot
> mainline versus NXP U-Boot for mx6sabresd board.
>
> The mx6_ddr_sysinfo approach is more elegant, for sure.
>

The bootrom loads the dcd using some logic and you write the register
in sequence.
You don't respect the ddr initialization or this delay on MMDC
according to 44.4.2.
Is that not necessary?

Michael

> It is just that I wanted to keep 100% in sync with the initialization
> done by the NXP hardware team.
>
> Regards,
>
> Fabio Estevam

Re: Question/issues about i.MX6 DDR configuration

[Fabio Estevam]

Hi Michael,

On Thu, Dec 2, 2021 at 5:36 PM Michael Nazzareno Trimarchi
<michael@amarulasolutions.com> wrote:

> The bootrom loads the dcd using some logic and you write the register
> in sequence.
> You don't respect the ddr initialization or this delay on MMDC
> according to 44.4.2.
> Is that not necessary?

I don't see in 44.4.2 where it mentions the requirement of delay
between register writes.

The part that Francesco quoted:

"A Precharge All command must be issued prior to the
MRW command to ensure robust DDR initialization. This
command is required to be issued to both chip selects if two
chip selects are utilized in the system."

Does not apply to mx6sabresd as it only has one chip select.

Regards,

Fabio Estevam

Re: Question/issues about i.MX6 DDR configuration

[Francesco Dolcini]

On Thu, Dec 02, 2021 at 05:56:38PM -0300, Fabio Estevam wrote:
> The part that Francesco quoted:
> 
> "A Precharge All command must be issued prior to the
> MRW command to ensure robust DDR initialization. This
> command is required to be issued to both chip selects if two
> chip selects are utilized in the system."
> 
> Does not apply to mx6sabresd as it only has one chip select.

I think that this applies even with just one chip select, it is just
prescribing a procedure and explicitly saying that it must be done for
all the chip select in use, either 1 or 2.

Not sure if this is the reason for the sporadic failures I'm
debugging, what triggered my attention was that this is supposed to be
needed for a more "robust" initialization, whatever that means ...

Francesco

Re: Question/issues about i.MX6 DDR configuration

[Fabio Estevam]

Hi Francesco,

On Fri, Dec 3, 2021 at 5:47 AM Francesco Dolcini
<francesco.dolcini@toradex.com> wrote:

> I think that this applies even with just one chip select, it is just
> prescribing a procedure and explicitly saying that it must be done for
> all the chip select in use, either 1 or 2.

According to the NXP application note:
https://www.voipac.com/downloads/imx/test/nxp_doc/AN/AN4467_DDR_Calibration.pdf

"To issue a Precharge-All to CS0, write 0x04000050 to MDSCR."

but such setting is not done in NXP U-Boot for mx6sabresd:
https://source.codeaurora.org/external/imx/uboot-imx/tree/board/freescale/mx6sabresd/mx6q_4x_mt41j128.cfg?h=lf_v2021.04

Back to your original instability issue: I suppose you are talking
about colibri_imx6.c.

Does it work well if you convert it to the mx6_dram_cfg() scheme?

Re: Question/issues about i.MX6 DDR configuration

[Francesco Dolcini]

Hello Fabio,

On Sat, Dec 04, 2021 at 11:29:23AM -0300, Fabio Estevam wrote:
> Back to your original instability issue: I suppose you are talking
> about colibri_imx6.c.
Both colibri/apalis imx6 are somehow affected.

> Does it work well if you convert it to the mx6_dram_cfg() scheme?
I have not tried yet, I was trying to understand first if there are
reasons to believe that doing the MMDC initialization this way is
supposed to produce better results or it is going to be just different.

One of the problems is that the issue is not that easy to reproduce, it
seems related to some specific module/condition, I would like to avoid
doing a change that seems to solve the problem, but that in the end is
just going to hide it for a few months. I'd like to really get to the
bottom of it.

I also had a look at a NXP presentation from 2020 [1] on MMDC
calibration for mass production, but we already did something like that,
moreover we do calibrate for each board at power-up and this
should be just enough. I do not believe that this is the issue.

Francesco

[1] https://community.nxp.com/pwmxy87654/attachments/pwmxy87654/imx-processors%40tkb/335/1/i.MX6%20DDR%20calibration_stress%20for%20Mass%20Production.pdf

Re: Question/issues about i.MX6 DDR configuration

[Francesco Dolcini]

Hello Fabio and Michael,

On Thu, Dec 02, 2021 at 09:36:44PM +0100, Michael Nazzareno Trimarchi wrote:
> On Thu, Dec 2, 2021 at 9:14 PM Fabio Estevam <festevam@gmail.com> wrote:
> > On Thu, Dec 2, 2021 at 1:14 PM Francesco Dolcini
> > <francesco.dolcini@toradex.com> wrote:
> > > I'm a little bit puzzled at the moment, according to the iMX6 reference
> > > manual[4], 44.4.2 MMDC initialization, a specific sequence is required
> > > to be followed and this is implemented by the `mx6_dram_cfg()`[5]
> > > function, but according to what Fabio wrote the raw initialization of
> > > registers was just more reliable for mx6sabresd. Fabio, what was the
> > > reason?
> >
> > The mx6_ddr_sysinfo approach is more elegant, for sure.
> 
> The bootrom loads the dcd using some logic and you write the register
> in sequence.
> You don't respect the ddr initialization or this delay on MMDC
> according to 44.4.2.
> Is that not necessary?

This is my main doubt, is the bootrom loading the DCD using some logic
or it is really equivalent to the raw register writing we are doing in
SPL? I'm not thinking only at delays (if any), but the complete MMDC
initialization flow as documented in the reference manual.

> > It is just that I wanted to keep 100% in sync with the initialization
> > done by the NXP hardware team.

Thanks Fabio for the clarification.

Francesco

Re: Question/issues about i.MX6 DDR configuration

[Francesco Dolcini]

Hello Fabio and everybody,
just a quick update on this old email thread.

On Thu, Dec 02, 2021 at 05:14:28PM +0100, Francesco Dolcini wrote:
> In my tests adding 1ms delay after each MMDC register write seems to
> have a positive effect and this is going into the direction that using
> `mx6_dram_cfg()` is the way to go.

On Sat, Dec 04, 2021 at 11:29:23AM -0300, Fabio Estevam wrote:
> Does it work well if you convert it to the mx6_dram_cfg() scheme?

I did test converting to mx6_dram_cfg() in a couple of problematic
samples, and the results are the same as with the "raw" register writes.

If I enable the debug prints everything works fine, exactly as the
previous "raw" memory configuration after adding some mdelay(1)
in-between each register write.

I'm inclined to exclude any issue on the memory calibration and on the
memory timing at the moment (I did triple check those and they should
not be affected by any delay on the memory controller
register writes).

There should be some kind of requirement on the memory configuration
procedure, I wonder if there is some constraints in writing the DDR3
mode registers (there are tMRD and tMOD, for instance ...).

Any advise appreciated!

Thanks
Francesco