[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Mingkai Hu <mingkai.hu@nxp.com>]

> -----Original Message-----
> From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> Sent: Thursday, September 07, 2017 2:55 PM
> To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> York Sun <york.sun@nxp.com>
> Cc: u-boot at lists.denx.de
> Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> 
> On Thu, 2017-09-07 at 06:45 +0000, Mingkai Hu wrote:
> > > -----Original Message-----
> > > From: Mingkai Hu
> > > Sent: Wednesday, September 06, 2017 5:37 PM
> > > To: 'Joakim Tjernlund' <Joakim.Tjernlund@infinera.com>; Roy Zang
> > > <roy.zang@nxp.com>; York Sun <york.sun@nxp.com>
> > > Cc: u-boot at lists.denx.de
> > > Subject: RE: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> > > > Sent: Tuesday, September 05, 2017 8:45 PM
> > > > To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang
> <roy.zang@nxp.com>;
> > >
> > > York
> > > > Sun <york.sun@nxp.com>
> > > > Cc: u-boot at lists.denx.de
> > > > Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > > >
> > > > On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > > > > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > > > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > > > > +Roy Zang to comment on PCIe clock source
> > > > > > >
> > > > > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > > > > As we are looking at PCI stuff ATM I would like to
> > > > > > > > > > > ask about PEX_GCLK_RATIO in E500 CPUs. I cannot find
> > > > > > > > > > > this is setup at all for E500 but I THINK this is required.
> > > > > > > > > > >
> > > > > > > > > > > In 83xx one do:
> > > > > > > > > > > get_clocks();
> > > > > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > > > > >
> > > > > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > > > > >
> > > > > > > > > > > (((bus ? gd->arch.pciexp2_clk :
> > > > > > > > > > > gd->arch.pciexp1_clk) /
> > > > > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > > > > >
> > > > > > > > > > > Any clues?
> > > > > > > > > > >
> > > > > > > > > > >     Jocke
> > > > > > > > > >
> > > > > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > > > > >
> > > > > > > > > > Anyhow, I put together this patch:
> > > > > > > > > >
> > > > > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct
> > > > > > > > > > pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > > > > > >
> > > > > > > > > >            pci_setup_indirect(hose, cfg_addr,
> > > > > > > > > > cfg_data);
> > > > > > > > > >
> > > > > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > > > > > > > > + (gd-
> > > > >
> > > > > bus_clk / 333333333) * 16);
> > > > > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > > > > +
> > > > > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > > > > >
> > > > > > > > > > Any chance this will work for all supported FSL PCIe
> controllers?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Jocke,
> > > > > > > > >
> > > > > > > > > You don't need to program this register if the actual
> > > > > > > > > PCIe clock is the same as default. Since SerDes
> > > > > > > > > reference clock has to be 100MHz for PCIe protocol, my
> > > > > > > > > guess is the PCIe clock is always correct. The bus_clk
> > > > > > > > > you are referring is not the PCIe clock. Again, I am not
> > > > > > > > > a PCIe expert, I could be wrong. Since PCIe (SerDes) has
> > > > > > > > > been working on multiple platforms, I guess the
> > > >
> > > > clock is correct.
> > > > > > > >
> > > > > > > > I don't think so. Here is what T1042 says about this:
> > > > > > > >
> > > > > > > > The PCI Express controller clock frequency is one-half the
> > > > > > > > platform
> > > >
> > > > clock frequency.
> > > > > > > >
> > > > > > > > The PCI Express controller core clock ratio register is
> > > > > > > > used to program the ratio of the actual PCI Express
> > > > > > > > controller clock frequency to the default controller core
> frequency ( 333 MHz ).
> > > > > > > > This is required only when a PCI Express controller clock
> > > > > > > > frequency
> > > >
> > > > other than the default 333 MHz has to be used.
> > > > > > > > As an example of programming PEX_GCLK_RATIO, consider the
> > > > > > > > case where the actual PCI Express controller clock is 250
> > > > > > > > MHz, the ratio of the actual clock to the default clock (
> > > > > > > > 333 MHz) is 3:4. that is, the default core clock has to be
> > > > > > > > multiplied by the ratio (3/4, which is equivalent to
> > > > > > > > 12/16). So the register has to be
> > > >
> > > > programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > > > > >
> > > > > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > > > > >
> > > > > > > > I found this on Google too:
> > > > > > > > The PEX controller hardware requires for timing tuning in
> > > > > > > > order to operate properly at given CCB/2 clock frequency.
> > > > > > > > PEX_GCLK_RATIO register controls this tuning with
> > > > > > > > 333MHz/16 granularities. So for given CCB frequency Fccb
> > > > > > > > [MHz] we must write to
> > > >
> > > > PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for
> > > > Fccb=375 we have to write 18.
> > > > > > >
> > > > > > > Jocke,
> > > > > > >
> > > > > > > Basically I agree with you, if the clock is different from
> > > > > > > default, you need to set the ratio. T104x indeed uses 1/2
> > > > > > > platform clock as PCIe clock. I am not sure if P2010 uses
> > > > > > > the same way. Let me add Roy to this thread.
> > > > > > >
> > > > > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx)
> SoCs?
> > > > > > > Don't we need to set PEX_GCLK_RATIO register? It is not even
> > > > > > > in the
> > > >
> > > > header file.
> > > > > >
> > > > > > Patch was off, this works:
> > > > > > diff --git a/drivers/pci/fsl_pci_init.c
> > > > > > b/drivers/pci/fsl_pci_init.c index 52792dcd59..e7f2499784
> > > > > > 100644
> > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller
> > > > > > *hose, struct fsl_pci_info *pci_info)
> > > > > >
> > > > > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > >
> > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > > >
> > > > > pci_hose_write_config_dword() writes to config space, doesn't it?
> > > > > 0x440 is not in config space.
> > > > >
> > > > > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > > > > +       /* udelay(1000000) needed here ?*/
> > > > >
> > > > > Let us not to add delay if we don't know yet.
> > > > >
> > > > > > +
> > > > > >          block_rev = in_be32(&pci->block_rev1);
> > > > > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > > > > >
> > > > > >
> > > > > > Any news on this ?
> > > > > > I know both T1042 and P2010 has the same setting/formula, a
> > > > > > good start :)
> > > > >
> > > > > +Mingkai to comment on PCIe clock
> > > > >
> > > > > York
> > >
> > > This is the clock to PCIe controller which is not different from the
> > > clock to PCIe SerDes lane.
> > > My understanding is that it should be configured according to the
> > > platform frequency.
> > > But there is no specific and consistent description for all the
> > > platforms. I am checking with the design team to see if there is
> > > hardware fixed clock to PCIe controller for the platforms which
> > > don't have clear description. Will keep you all updated.
> > >
> >
> > This ratio will be used by GPEX only to calculate the actual timer
> > values to be used for Ack Latency and Replay timeout values and the FC
> > update timer value under traffic condition. It's applicable for all
> > PowerPC controllers except T4 and T2 family, so would you please help to
> update the patch to set the register according to platform clock (or
> platform/2, need to check chip RM) excluding T4/T2 platform?
> 
> I think NXP is in a better position to adjust the patch w.r.t actual PCIe clock. I
> just want to known, for now, what PCIe clock is for T1042 and P2010, CCB or
> CCB/2, so we can set the correct value and continue with our trouble
> shooting.
> 
> T1040RM actually has this writing in 28.11.19:
> The PCI Express controller clock frequency is one-half the platform clock
> frequency.
> ....
> That indicates the PCIe clock is CCB/2. P2010 does not have that text.
> Please advice.
> 

For P2010/P2020, CCB clock is routed to PCIe controller.

Mingkai