[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

As we are looking at PCI stuff ATM I would like to ask
about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
at all for E500 but I THINK this is required.

In 83xx one do:
get_clocks();
/* Configure the PCIE controller core clock ratio */
out_le32(hose_cfg_base + PEX_GCLK_RATIO,
(((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
/ 1000000) * 16) / 333);
udelay(1000000);

Any clues?

 Jocke

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> As we are looking at PCI stuff ATM I would like to ask
> about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
> at all for E500 but I THINK this is required.
> 
> In 83xx one do:
> get_clocks();
> /* Configure the PCIE controller core clock ratio */
> out_le32(hose_cfg_base + PEX_GCLK_RATIO,

A bit strange with out_le32 instead of out_be32 ?

> (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
> / 1000000) * 16) / 333);
> udelay(1000000);
> 
> Any clues?
> 
>  Jocke
Seems like only 83xx is setting this parameter.

Anyhow, I put together this patch:

--- a/drivers/pci/fsl_pci_init.c
+++ b/drivers/pci/fsl_pci_init.c
@@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
 
        pci_setup_indirect(hose, cfg_addr, cfg_data);
 
+       /* Configure the PCIE controller core clock ratio */
+       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
+       /* udelay(1000000) needed here ?*/
+
        block_rev = in_be32(&pci->block_rev1);
        if (PEX_IP_BLK_REV_2_2 <= block_rev) {
                pi = &pci->pit[2];      /* 0xDC0 */

Any chance this will work for all supported FSL PCIe controllers?

 Jocke

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[York Sun]

On 08/29/2017 03:47 AM, Joakim Tjernlund wrote:
> As we are looking at PCI stuff ATM I would like to ask
> about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
> at all for E500 but I THINK this is required.
> 
> In 83xx one do:
> get_clocks();
> /* Configure the PCIE controller core clock ratio */
> out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
> / 1000000) * 16) / 333);
> udelay(1000000);
> 
> Any clues?
> 

Jocke,

P2010 has SerDes. With PCIe, SerDes uses 100MHz reference clock. Please 
see reference manual "SerDes reference clock configuration".

York

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[York Sun]

On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
>> As we are looking at PCI stuff ATM I would like to ask
>> about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
>> at all for E500 but I THINK this is required.
>>
>> In 83xx one do:
>> get_clocks();
>> /* Configure the PCIE controller core clock ratio */
>> out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> 
> A bit strange with out_le32 instead of out_be32 ?
> 
>> (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
>> / 1000000) * 16) / 333);
>> udelay(1000000);
>>
>> Any clues?
>>
>>   Jocke
> Seems like only 83xx is setting this parameter.
> 
> Anyhow, I put together this patch:
> 
> --- a/drivers/pci/fsl_pci_init.c
> +++ b/drivers/pci/fsl_pci_init.c
> @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
>   
>          pci_setup_indirect(hose, cfg_addr, cfg_data);
>   
> +       /* Configure the PCIE controller core clock ratio */
> +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
> +       /* udelay(1000000) needed here ?*/
> +
>          block_rev = in_be32(&pci->block_rev1);
>          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
>                  pi = &pci->pit[2];      /* 0xDC0 */
> 
> Any chance this will work for all supported FSL PCIe controllers?
> 

Jocke,

You don't need to program this register if the actual PCIe clock is the 
same as default. Since SerDes reference clock has to be 100MHz for PCIe 
protocol, my guess is the PCIe clock is always correct. The bus_clk you 
are referring is not the PCIe clock. Again, I am not a PCIe expert, I 
could be wrong. Since PCIe (SerDes) has been working on multiple 
platforms, I guess the clock is correct.

York

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > As we are looking at PCI stuff ATM I would like to ask
> > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
> > > at all for E500 but I THINK this is required.
> > > 
> > > In 83xx one do:
> > > get_clocks();
> > > /* Configure the PCIE controller core clock ratio */
> > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > 
> > A bit strange with out_le32 instead of out_be32 ?
> > 
> > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
> > > / 1000000) * 16) / 333);
> > > udelay(1000000);
> > > 
> > > Any clues?
> > > 
> > >   Jocke
> > 
> > Seems like only 83xx is setting this parameter.
> > 
> > Anyhow, I put together this patch:
> > 
> > --- a/drivers/pci/fsl_pci_init.c
> > +++ b/drivers/pci/fsl_pci_init.c
> > @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
> >   
> >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> >   
> > +       /* Configure the PCIE controller core clock ratio */
> > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
> > +       /* udelay(1000000) needed here ?*/
> > +
> >          block_rev = in_be32(&pci->block_rev1);
> >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> >                  pi = &pci->pit[2];      /* 0xDC0 */
> > 
> > Any chance this will work for all supported FSL PCIe controllers?
> > 
> 
> Jocke,
> 
> You don't need to program this register if the actual PCIe clock is the 
> same as default. Since SerDes reference clock has to be 100MHz for PCIe 
> protocol, my guess is the PCIe clock is always correct. The bus_clk you 
> are referring is not the PCIe clock. Again, I am not a PCIe expert, I 
> could be wrong. Since PCIe (SerDes) has been working on multiple 
> platforms, I guess the clock is correct.

I don't think so. Here is what T1042 says about this:

The PCI Express controller clock frequency is one-half the platform clock frequency.

The PCI Express controller core clock ratio register is used to program the ratio of the
actual PCI Express controller clock frequency to the default controller core frequency
( 333 MHz ). This is required only when a PCI Express controller clock frequency other
than the default 333 MHz has to be used.
As an example of programming PEX_GCLK_RATIO, consider the case where the actual
PCI Express controller clock is 250 MHz, the ratio of the actual clock to the default clock
( 333 MHz) is 3:4. that is, the default core clock has to be multiplied by the ratio (3/4,
which is equivalent to 12/16). So the register has to be programmed with the decimal
numerator value 12 or 0x0000_000C.

Our CCB is 250 MHz so this should be set to 0xc.

I found this on Google too:
The PEX controller hardware requires for timing tuning in order to operate properly at given CCB/2 clock
frequency. PEX_GCLK_RATIO register controls this tuning with 333MHz/16 granularities. So for given CCB
frequency Fccb [MHz] we must write to PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
we have to write 18.

Jocke

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[York Sun]

+Roy Zang to comment on PCIe clock source

On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
>> On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
>>> On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
>>>> As we are looking at PCI stuff ATM I would like to ask
>>>> about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
>>>> at all for E500 but I THINK this is required.
>>>>
>>>> In 83xx one do:
>>>> get_clocks();
>>>> /* Configure the PCIE controller core clock ratio */
>>>> out_le32(hose_cfg_base + PEX_GCLK_RATIO,
>>>
>>> A bit strange with out_le32 instead of out_be32 ?
>>>
>>>> (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
>>>> / 1000000) * 16) / 333);
>>>> udelay(1000000);
>>>>
>>>> Any clues?
>>>>
>>>>    Jocke
>>>
>>> Seems like only 83xx is setting this parameter.
>>>
>>> Anyhow, I put together this patch:
>>>
>>> --- a/drivers/pci/fsl_pci_init.c
>>> +++ b/drivers/pci/fsl_pci_init.c
>>> @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
>>>    
>>>           pci_setup_indirect(hose, cfg_addr, cfg_data);
>>>    
>>> +       /* Configure the PCIE controller core clock ratio */
>>> +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
>>> +       /* udelay(1000000) needed here ?*/
>>> +
>>>           block_rev = in_be32(&pci->block_rev1);
>>>           if (PEX_IP_BLK_REV_2_2 <= block_rev) {
>>>                   pi = &pci->pit[2];      /* 0xDC0 */
>>>
>>> Any chance this will work for all supported FSL PCIe controllers?
>>>
>>
>> Jocke,
>>
>> You don't need to program this register if the actual PCIe clock is the
>> same as default. Since SerDes reference clock has to be 100MHz for PCIe
>> protocol, my guess is the PCIe clock is always correct. The bus_clk you
>> are referring is not the PCIe clock. Again, I am not a PCIe expert, I
>> could be wrong. Since PCIe (SerDes) has been working on multiple
>> platforms, I guess the clock is correct.
> 
> I don't think so. Here is what T1042 says about this:
> 
> The PCI Express controller clock frequency is one-half the platform clock frequency.
> 
> The PCI Express controller core clock ratio register is used to program the ratio of the
> actual PCI Express controller clock frequency to the default controller core frequency
> ( 333 MHz ). This is required only when a PCI Express controller clock frequency other
> than the default 333 MHz has to be used.
> As an example of programming PEX_GCLK_RATIO, consider the case where the actual
> PCI Express controller clock is 250 MHz, the ratio of the actual clock to the default clock
> ( 333 MHz) is 3:4. that is, the default core clock has to be multiplied by the ratio (3/4,
> which is equivalent to 12/16). So the register has to be programmed with the decimal
> numerator value 12 or 0x0000_000C.
> 
> Our CCB is 250 MHz so this should be set to 0xc.
> 
> I found this on Google too:
> The PEX controller hardware requires for timing tuning in order to operate properly at given CCB/2 clock
> frequency. PEX_GCLK_RATIO register controls this tuning with 333MHz/16 granularities. So for given CCB
> frequency Fccb [MHz] we must write to PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> we have to write 18.

Jocke,

Basically I agree with you, if the clock is different from default, you 
need to set the ratio. T104x indeed uses 1/2 platform clock as PCIe 
clock. I am not sure if P2010 uses the same way. Let me add Roy to this 
thread.

Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs? Don't 
we need to set PEX_GCLK_RATIO register? It is not even in the header file.

York

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> +Roy Zang to comment on PCIe clock source
> 
> On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
> > > > > at all for E500 but I THINK this is required.
> > > > > 
> > > > > In 83xx one do:
> > > > > get_clocks();
> > > > > /* Configure the PCIE controller core clock ratio */
> > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > 
> > > > A bit strange with out_le32 instead of out_be32 ?
> > > > 
> > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
> > > > > / 1000000) * 16) / 333);
> > > > > udelay(1000000);
> > > > > 
> > > > > Any clues?
> > > > > 
> > > > >    Jocke
> > > > 
> > > > Seems like only 83xx is setting this parameter.
> > > > 
> > > > Anyhow, I put together this patch:
> > > > 
> > > > --- a/drivers/pci/fsl_pci_init.c
> > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
> > > >    
> > > >           pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > >    
> > > > +       /* Configure the PCIE controller core clock ratio */
> > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
> > > > +       /* udelay(1000000) needed here ?*/
> > > > +
> > > >           block_rev = in_be32(&pci->block_rev1);
> > > >           if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > >                   pi = &pci->pit[2];      /* 0xDC0 */
> > > > 
> > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > 
> > > 
> > > Jocke,
> > > 
> > > You don't need to program this register if the actual PCIe clock is the
> > > same as default. Since SerDes reference clock has to be 100MHz for PCIe
> > > protocol, my guess is the PCIe clock is always correct. The bus_clk you
> > > are referring is not the PCIe clock. Again, I am not a PCIe expert, I
> > > could be wrong. Since PCIe (SerDes) has been working on multiple
> > > platforms, I guess the clock is correct.
> > 
> > I don't think so. Here is what T1042 says about this:
> > 
> > The PCI Express controller clock frequency is one-half the platform clock frequency.
> > 
> > The PCI Express controller core clock ratio register is used to program the ratio of the
> > actual PCI Express controller clock frequency to the default controller core frequency
> > ( 333 MHz ). This is required only when a PCI Express controller clock frequency other
> > than the default 333 MHz has to be used.
> > As an example of programming PEX_GCLK_RATIO, consider the case where the actual
> > PCI Express controller clock is 250 MHz, the ratio of the actual clock to the default clock
> > ( 333 MHz) is 3:4. that is, the default core clock has to be multiplied by the ratio (3/4,
> > which is equivalent to 12/16). So the register has to be programmed with the decimal
> > numerator value 12 or 0x0000_000C.
> > 
> > Our CCB is 250 MHz so this should be set to 0xc.
> > 
> > I found this on Google too:
> > The PEX controller hardware requires for timing tuning in order to operate properly at given CCB/2 clock
> > frequency. PEX_GCLK_RATIO register controls this tuning with 333MHz/16 granularities. So for given CCB
> > frequency Fccb [MHz] we must write to PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> > we have to write 18.
> 
> Jocke,
> 
> Basically I agree with you, if the clock is different from default, you 
> need to set the ratio. T104x indeed uses 1/2 platform clock as PCIe 
> clock. I am not sure if P2010 uses the same way. Let me add Roy to this 
> thread.
> 
> Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs? Don't 
> we need to set PEX_GCLK_RATIO register? It is not even in the header file.

Patch was off, this works:
diff --git a/drivers/pci/fsl_pci_init.c b/drivers/pci/fsl_pci_init.c
index 52792dcd59..e7f2499784 100644
--- a/drivers/pci/fsl_pci_init.c
+++ b/drivers/pci/fsl_pci_init.c
@@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
 
        pci_setup_indirect(hose, cfg_addr, cfg_data);
 
+       /* Configure the PCIE controller core clock ratio */
+       pci_hose_write_config_dword(hose, dev, 0x440,
+                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
+       /* udelay(1000000) needed here ?*/
+
        block_rev = in_be32(&pci->block_rev1);
        if (PEX_IP_BLK_REV_2_2 <= block_rev) {
                pi = &pci->pit[2];      /* 0xDC0 */


Any news on this ?
I know both T1042 and P2010 has the same setting/formula, a good start :)

 Jocke

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[York Sun]

On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
>> +Roy Zang to comment on PCIe clock source
>>
>> On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
>>> On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
>>>> On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
>>>>> On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
>>>>>> As we are looking at PCI stuff ATM I would like to ask
>>>>>> about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
>>>>>> at all for E500 but I THINK this is required.
>>>>>>
>>>>>> In 83xx one do:
>>>>>> get_clocks();
>>>>>> /* Configure the PCIE controller core clock ratio */
>>>>>> out_le32(hose_cfg_base + PEX_GCLK_RATIO,
>>>>>
>>>>> A bit strange with out_le32 instead of out_be32 ?
>>>>>
>>>>>> (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
>>>>>> / 1000000) * 16) / 333);
>>>>>> udelay(1000000);
>>>>>>
>>>>>> Any clues?
>>>>>>
>>>>>>     Jocke
>>>>>
>>>>> Seems like only 83xx is setting this parameter.
>>>>>
>>>>> Anyhow, I put together this patch:
>>>>>
>>>>> --- a/drivers/pci/fsl_pci_init.c
>>>>> +++ b/drivers/pci/fsl_pci_init.c
>>>>> @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
>>>>>     
>>>>>            pci_setup_indirect(hose, cfg_addr, cfg_data);
>>>>>     
>>>>> +       /* Configure the PCIE controller core clock ratio */
>>>>> +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
>>>>> +       /* udelay(1000000) needed here ?*/
>>>>> +
>>>>>            block_rev = in_be32(&pci->block_rev1);
>>>>>            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
>>>>>                    pi = &pci->pit[2];      /* 0xDC0 */
>>>>>
>>>>> Any chance this will work for all supported FSL PCIe controllers?
>>>>>
>>>>
>>>> Jocke,
>>>>
>>>> You don't need to program this register if the actual PCIe clock is the
>>>> same as default. Since SerDes reference clock has to be 100MHz for PCIe
>>>> protocol, my guess is the PCIe clock is always correct. The bus_clk you
>>>> are referring is not the PCIe clock. Again, I am not a PCIe expert, I
>>>> could be wrong. Since PCIe (SerDes) has been working on multiple
>>>> platforms, I guess the clock is correct.
>>>
>>> I don't think so. Here is what T1042 says about this:
>>>
>>> The PCI Express controller clock frequency is one-half the platform clock frequency.
>>>
>>> The PCI Express controller core clock ratio register is used to program the ratio of the
>>> actual PCI Express controller clock frequency to the default controller core frequency
>>> ( 333 MHz ). This is required only when a PCI Express controller clock frequency other
>>> than the default 333 MHz has to be used.
>>> As an example of programming PEX_GCLK_RATIO, consider the case where the actual
>>> PCI Express controller clock is 250 MHz, the ratio of the actual clock to the default clock
>>> ( 333 MHz) is 3:4. that is, the default core clock has to be multiplied by the ratio (3/4,
>>> which is equivalent to 12/16). So the register has to be programmed with the decimal
>>> numerator value 12 or 0x0000_000C.
>>>
>>> Our CCB is 250 MHz so this should be set to 0xc.
>>>
>>> I found this on Google too:
>>> The PEX controller hardware requires for timing tuning in order to operate properly at given CCB/2 clock
>>> frequency. PEX_GCLK_RATIO register controls this tuning with 333MHz/16 granularities. So for given CCB
>>> frequency Fccb [MHz] we must write to PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
>>> we have to write 18.
>>
>> Jocke,
>>
>> Basically I agree with you, if the clock is different from default, you
>> need to set the ratio. T104x indeed uses 1/2 platform clock as PCIe
>> clock. I am not sure if P2010 uses the same way. Let me add Roy to this
>> thread.
>>
>> Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs? Don't
>> we need to set PEX_GCLK_RATIO register? It is not even in the header file.
> 
> Patch was off, this works:
> diff --git a/drivers/pci/fsl_pci_init.c b/drivers/pci/fsl_pci_init.c
> index 52792dcd59..e7f2499784 100644
> --- a/drivers/pci/fsl_pci_init.c
> +++ b/drivers/pci/fsl_pci_init.c
> @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
>   
>          pci_setup_indirect(hose, cfg_addr, cfg_data);
>   
> +       /* Configure the PCIE controller core clock ratio */
> +       pci_hose_write_config_dword(hose, dev, 0x440,

pci_hose_write_config_dword() writes to config space, doesn't it? 0x440 
is not in config space.

> +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> +       /* udelay(1000000) needed here ?*/

Let us not to add delay if we don't know yet.

> +
>          block_rev = in_be32(&pci->block_rev1);
>          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
>                  pi = &pci->pit[2];      /* 0xDC0 */
> 
> 
> Any news on this ?
> I know both T1042 and P2010 has the same setting/formula, a good start :)

+Mingkai to comment on PCIe clock

York

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > +Roy Zang to comment on PCIe clock source
> > > 
> > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
> > > > > > > at all for E500 but I THINK this is required.
> > > > > > > 
> > > > > > > In 83xx one do:
> > > > > > > get_clocks();
> > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > 
> > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > 
> > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
> > > > > > > / 1000000) * 16) / 333);
> > > > > > > udelay(1000000);
> > > > > > > 
> > > > > > > Any clues?
> > > > > > > 
> > > > > > >     Jocke
> > > > > > 
> > > > > > Seems like only 83xx is setting this parameter.
> > > > > > 
> > > > > > Anyhow, I put together this patch:
> > > > > > 
> > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > >     
> > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > >     
> > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
> > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > +
> > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > 
> > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > 
> > > > > 
> > > > > Jocke,
> > > > > 
> > > > > You don't need to program this register if the actual PCIe clock is the
> > > > > same as default. Since SerDes reference clock has to be 100MHz for PCIe
> > > > > protocol, my guess is the PCIe clock is always correct. The bus_clk you
> > > > > are referring is not the PCIe clock. Again, I am not a PCIe expert, I
> > > > > could be wrong. Since PCIe (SerDes) has been working on multiple
> > > > > platforms, I guess the clock is correct.
> > > > 
> > > > I don't think so. Here is what T1042 says about this:
> > > > 
> > > > The PCI Express controller clock frequency is one-half the platform clock frequency.
> > > > 
> > > > The PCI Express controller core clock ratio register is used to program the ratio of the
> > > > actual PCI Express controller clock frequency to the default controller core frequency
> > > > ( 333 MHz ). This is required only when a PCI Express controller clock frequency other
> > > > than the default 333 MHz has to be used.
> > > > As an example of programming PEX_GCLK_RATIO, consider the case where the actual
> > > > PCI Express controller clock is 250 MHz, the ratio of the actual clock to the default clock
> > > > ( 333 MHz) is 3:4. that is, the default core clock has to be multiplied by the ratio (3/4,
> > > > which is equivalent to 12/16). So the register has to be programmed with the decimal
> > > > numerator value 12 or 0x0000_000C.
> > > > 
> > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > 
> > > > I found this on Google too:
> > > > The PEX controller hardware requires for timing tuning in order to operate properly at given CCB/2 clock
> > > > frequency. PEX_GCLK_RATIO register controls this tuning with 333MHz/16 granularities. So for given CCB
> > > > frequency Fccb [MHz] we must write to PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> > > > we have to write 18.
> > > 
> > > Jocke,
> > > 
> > > Basically I agree with you, if the clock is different from default, you
> > > need to set the ratio. T104x indeed uses 1/2 platform clock as PCIe
> > > clock. I am not sure if P2010 uses the same way. Let me add Roy to this
> > > thread.
> > > 
> > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs? Don't
> > > we need to set PEX_GCLK_RATIO register? It is not even in the header file.
> > 
> > Patch was off, this works:
> > diff --git a/drivers/pci/fsl_pci_init.c b/drivers/pci/fsl_pci_init.c
> > index 52792dcd59..e7f2499784 100644
> > --- a/drivers/pci/fsl_pci_init.c
> > +++ b/drivers/pci/fsl_pci_init.c
> > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
> >   
> >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> >   
> > +       /* Configure the PCIE controller core clock ratio */
> > +       pci_hose_write_config_dword(hose, dev, 0x440,
> 
> pci_hose_write_config_dword() writes to config space, doesn't it? 0x440 
> is not in config space.

I am still learning PCI but "controller core clock ratio" is listed under
"PCI Express extended configuration space", the same chapter as LTSSM(0x404)
so I THINK this is correct, you tell me:)

One odd thing though is that we have:
u16 ltssm;
#define PCI_LTSSM       0x404
 and
pci_hose_read_config_word(hose, dev, PCI_LTSSM, &ltssm);

LTSSM in my RM is 32 bits, yet the code does a 16 bits access
on 404 address. Is there some little endian magic fixing this?


> 
> > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > +       /* udelay(1000000) needed here ?*/
> 
> Let us not to add delay if we don't know yet.

Sure, will drop that.

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > +Roy Zang to comment on PCIe clock source
> > > 
> > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is setup
> > > > > > > at all for E500 but I THINK this is required.
> > > > > > > 
> > > > > > > In 83xx one do:
> > > > > > > get_clocks();
> > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > 
> > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > 
> > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk)
> > > > > > > / 1000000) * 16) / 333);
> > > > > > > udelay(1000000);
> > > > > > > 
> > > > > > > Any clues?
> > > > > > > 
> > > > > > >     Jocke
> > > > > > 
> > > > > > Seems like only 83xx is setting this parameter.
> > > > > > 
> > > > > > Anyhow, I put together this patch:
> > > > > > 
> > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > >     
> > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > >     
> > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd->bus_clk / 333333333) * 16);
> > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > +
> > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > 
> > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > 
> > > > > 
> > > > > Jocke,
> > > > > 
> > > > > You don't need to program this register if the actual PCIe clock is the
> > > > > same as default. Since SerDes reference clock has to be 100MHz for PCIe
> > > > > protocol, my guess is the PCIe clock is always correct. The bus_clk you
> > > > > are referring is not the PCIe clock. Again, I am not a PCIe expert, I
> > > > > could be wrong. Since PCIe (SerDes) has been working on multiple
> > > > > platforms, I guess the clock is correct.
> > > > 
> > > > I don't think so. Here is what T1042 says about this:
> > > > 
> > > > The PCI Express controller clock frequency is one-half the platform clock frequency.
> > > > 
> > > > The PCI Express controller core clock ratio register is used to program the ratio of the
> > > > actual PCI Express controller clock frequency to the default controller core frequency
> > > > ( 333 MHz ). This is required only when a PCI Express controller clock frequency other
> > > > than the default 333 MHz has to be used.
> > > > As an example of programming PEX_GCLK_RATIO, consider the case where the actual
> > > > PCI Express controller clock is 250 MHz, the ratio of the actual clock to the default clock
> > > > ( 333 MHz) is 3:4. that is, the default core clock has to be multiplied by the ratio (3/4,
> > > > which is equivalent to 12/16). So the register has to be programmed with the decimal
> > > > numerator value 12 or 0x0000_000C.
> > > > 
> > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > 
> > > > I found this on Google too:
> > > > The PEX controller hardware requires for timing tuning in order to operate properly at given CCB/2 clock
> > > > frequency. PEX_GCLK_RATIO register controls this tuning with 333MHz/16 granularities. So for given CCB
> > > > frequency Fccb [MHz] we must write to PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> > > > we have to write 18.
> > > 
> > > Jocke,
> > > 
> > > Basically I agree with you, if the clock is different from default, you
> > > need to set the ratio. T104x indeed uses 1/2 platform clock as PCIe
> > > clock. I am not sure if P2010 uses the same way. Let me add Roy to this
> > > thread.
> > > 
> > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs? Don't
> > > we need to set PEX_GCLK_RATIO register? It is not even in the header file.
> > 
> > Patch was off, this works:
> > diff --git a/drivers/pci/fsl_pci_init.c b/drivers/pci/fsl_pci_init.c
> > index 52792dcd59..e7f2499784 100644
> > --- a/drivers/pci/fsl_pci_init.c
> > +++ b/drivers/pci/fsl_pci_init.c
> > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller *hose, struct fsl_pci_info *pci_info)
> >   
> >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> >   
> > +       /* Configure the PCIE controller core clock ratio */
> > +       pci_hose_write_config_dword(hose, dev, 0x440,
> 
> pci_hose_write_config_dword() writes to config space, doesn't it? 0x440 
> is not in config space.
> 
> > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > +       /* udelay(1000000) needed here ?*/
> 
> Let us not to add delay if we don't know yet.
> 
> > +
> >          block_rev = in_be32(&pci->block_rev1);
> >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> >                  pi = &pci->pit[2];      /* 0xDC0 */
> > 
> > 
> > Any news on this ?
> > I know both T1042 and P2010 has the same setting/formula, a good start :)
> 
> +Mingkai to comment on PCIe clock
> 
> York

Ping ?

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Mingkai Hu]

> -----Original Message-----
> From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> Sent: Tuesday, September 05, 2017 8:45 PM
> To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> York Sun <york.sun@nxp.com>
> Cc: u-boot at lists.denx.de
> Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> 
> On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > +Roy Zang to comment on PCIe clock source
> > > >
> > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is
> > > > > > > > setup at all for E500 but I THINK this is required.
> > > > > > > >
> > > > > > > > In 83xx one do:
> > > > > > > > get_clocks();
> > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > >
> > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > >
> > > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk) /
> > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > >
> > > > > > > > Any clues?
> > > > > > > >
> > > > > > > >     Jocke
> > > > > > >
> > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > >
> > > > > > > Anyhow, I put together this patch:
> > > > > > >
> > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller
> > > > > > > *hose, struct fsl_pci_info *pci_info)
> > > > > > >
> > > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > > >
> > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd-
> >bus_clk / 333333333) * 16);
> > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > +
> > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > >
> > > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > >
> > > > > >
> > > > > > Jocke,
> > > > > >
> > > > > > You don't need to program this register if the actual PCIe
> > > > > > clock is the same as default. Since SerDes reference clock has
> > > > > > to be 100MHz for PCIe protocol, my guess is the PCIe clock is
> > > > > > always correct. The bus_clk you are referring is not the PCIe
> > > > > > clock. Again, I am not a PCIe expert, I could be wrong. Since
> > > > > > PCIe (SerDes) has been working on multiple platforms, I guess the
> clock is correct.
> > > > >
> > > > > I don't think so. Here is what T1042 says about this:
> > > > >
> > > > > The PCI Express controller clock frequency is one-half the platform
> clock frequency.
> > > > >
> > > > > The PCI Express controller core clock ratio register is used to
> > > > > program the ratio of the actual PCI Express controller clock
> > > > > frequency to the default controller core frequency ( 333 MHz ).
> > > > > This is required only when a PCI Express controller clock frequency
> other than the default 333 MHz has to be used.
> > > > > As an example of programming PEX_GCLK_RATIO, consider the case
> > > > > where the actual PCI Express controller clock is 250 MHz, the
> > > > > ratio of the actual clock to the default clock ( 333 MHz) is
> > > > > 3:4. that is, the default core clock has to be multiplied by the
> > > > > ratio (3/4, which is equivalent to 12/16). So the register has to be
> programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > >
> > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > >
> > > > > I found this on Google too:
> > > > > The PEX controller hardware requires for timing tuning in order
> > > > > to operate properly at given CCB/2 clock frequency.
> > > > > PEX_GCLK_RATIO register controls this tuning with 333MHz/16
> > > > > granularities. So for given CCB frequency Fccb [MHz] we must write to
> PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375 we
> have to write 18.
> > > >
> > > > Jocke,
> > > >
> > > > Basically I agree with you, if the clock is different from
> > > > default, you need to set the ratio. T104x indeed uses 1/2 platform
> > > > clock as PCIe clock. I am not sure if P2010 uses the same way. Let
> > > > me add Roy to this thread.
> > > >
> > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs?
> > > > Don't we need to set PEX_GCLK_RATIO register? It is not even in the
> header file.
> > >
> > > Patch was off, this works:
> > > diff --git a/drivers/pci/fsl_pci_init.c b/drivers/pci/fsl_pci_init.c
> > > index 52792dcd59..e7f2499784 100644
> > > --- a/drivers/pci/fsl_pci_init.c
> > > +++ b/drivers/pci/fsl_pci_init.c
> > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller *hose,
> > > struct fsl_pci_info *pci_info)
> > >
> > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > >
> > > +       /* Configure the PCIE controller core clock ratio */
> > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> >
> > pci_hose_write_config_dword() writes to config space, doesn't it?
> > 0x440 is not in config space.
> >
> > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > +       /* udelay(1000000) needed here ?*/
> >
> > Let us not to add delay if we don't know yet.
> >
> > > +
> > >          block_rev = in_be32(&pci->block_rev1);
> > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > >
> > >
> > > Any news on this ?
> > > I know both T1042 and P2010 has the same setting/formula, a good
> > > start :)
> >
> > +Mingkai to comment on PCIe clock
> >
> > York
> 

This is the clock to PCIe controller which is not different from the clock to PCIe SerDes lane.
My understanding is that it should be configured according to the platform frequency.
But there is no specific and consistent description for all the platforms. I am checking with
the design team to see if there is hardware fixed clock to PCIe controller for the platforms
which don't have clear description. Will keep you all updated.

Regards,
Mingkai

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Wed, 2017-09-06 at 09:36 +0000, Mingkai Hu wrote:
> > -----Original Message-----
> > From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> > Sent: Tuesday, September 05, 2017 8:45 PM
> > To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> > York Sun <york.sun@nxp.com>
> > Cc: u-boot at lists.denx.de
> > Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > 
> > On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > > +Roy Zang to comment on PCIe clock source
> > > > > 
> > > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is
> > > > > > > > > setup at all for E500 but I THINK this is required.
> > > > > > > > > 
> > > > > > > > > In 83xx one do:
> > > > > > > > > get_clocks();
> > > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > > > 
> > > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > > > 
> > > > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk) /
> > > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > > > 
> > > > > > > > > Any clues?
> > > > > > > > > 
> > > > > > > > >     Jocke
> > > > > > > > 
> > > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > > > 
> > > > > > > > Anyhow, I put together this patch:
> > > > > > > > 
> > > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct pci_controller
> > > > > > > > *hose, struct fsl_pci_info *pci_info)
> > > > > > > > 
> > > > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > > > > 
> > > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd-
> > > 
> > > bus_clk / 333333333) * 16);
> > > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > > +
> > > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > > > 
> > > > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > > > 
> > > > > > > 
> > > > > > > Jocke,
> > > > > > > 
> > > > > > > You don't need to program this register if the actual PCIe
> > > > > > > clock is the same as default. Since SerDes reference clock has
> > > > > > > to be 100MHz for PCIe protocol, my guess is the PCIe clock is
> > > > > > > always correct. The bus_clk you are referring is not the PCIe
> > > > > > > clock. Again, I am not a PCIe expert, I could be wrong. Since
> > > > > > > PCIe (SerDes) has been working on multiple platforms, I guess the
> > 
> > clock is correct.
> > > > > > 
> > > > > > I don't think so. Here is what T1042 says about this:
> > > > > > 
> > > > > > The PCI Express controller clock frequency is one-half the platform
> > 
> > clock frequency.
> > > > > > 
> > > > > > The PCI Express controller core clock ratio register is used to
> > > > > > program the ratio of the actual PCI Express controller clock
> > > > > > frequency to the default controller core frequency ( 333 MHz ).
> > > > > > This is required only when a PCI Express controller clock frequency
> > 
> > other than the default 333 MHz has to be used.
> > > > > > As an example of programming PEX_GCLK_RATIO, consider the case
> > > > > > where the actual PCI Express controller clock is 250 MHz, the
> > > > > > ratio of the actual clock to the default clock ( 333 MHz) is
> > > > > > 3:4. that is, the default core clock has to be multiplied by the
> > > > > > ratio (3/4, which is equivalent to 12/16). So the register has to be
> > 
> > programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > > > 
> > > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > > > 
> > > > > > I found this on Google too:
> > > > > > The PEX controller hardware requires for timing tuning in order
> > > > > > to operate properly at given CCB/2 clock frequency.
> > > > > > PEX_GCLK_RATIO register controls this tuning with 333MHz/16
> > > > > > granularities. So for given CCB frequency Fccb [MHz] we must write to
> > 
> > PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375 we
> > have to write 18.
> > > > > 
> > > > > Jocke,
> > > > > 
> > > > > Basically I agree with you, if the clock is different from
> > > > > default, you need to set the ratio. T104x indeed uses 1/2 platform
> > > > > clock as PCIe clock. I am not sure if P2010 uses the same way. Let
> > > > > me add Roy to this thread.
> > > > > 
> > > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs?
> > > > > Don't we need to set PEX_GCLK_RATIO register? It is not even in the
> > 
> > header file.
> > > > 
> > > > Patch was off, this works:
> > > > diff --git a/drivers/pci/fsl_pci_init.c b/drivers/pci/fsl_pci_init.c
> > > > index 52792dcd59..e7f2499784 100644
> > > > --- a/drivers/pci/fsl_pci_init.c
> > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller *hose,
> > > > struct fsl_pci_info *pci_info)
> > > > 
> > > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > 
> > > > +       /* Configure the PCIE controller core clock ratio */
> > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > 
> > > pci_hose_write_config_dword() writes to config space, doesn't it?
> > > 0x440 is not in config space.
> > > 
> > > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > > +       /* udelay(1000000) needed here ?*/
> > > 
> > > Let us not to add delay if we don't know yet.
> > > 
> > > > +
> > > >          block_rev = in_be32(&pci->block_rev1);
> > > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > > > 
> > > > 
> > > > Any news on this ?
> > > > I know both T1042 and P2010 has the same setting/formula, a good
> > > > start :)
> > > 
> > > +Mingkai to comment on PCIe clock
> > > 
> > > York
> 
> This is the clock to PCIe controller which is not different from the clock to PCIe SerDes lane.
> My understanding is that it should be configured according to the platform frequency.
> But there is no specific and consistent description for all the platforms. I am checking with
> the design team to see if there is hardware fixed clock to PCIe controller for the platforms
> which don't have clear description. Will keep you all updated.

Hi Mingkai

Do you agree that my calculation is correct? This is not described clearly in RM:
pci_hose_write_config_dword(hose, dev, 0x440,
                                ((gd->bus_clk / 1000000) * 16/2) / 333);
Where bus_clk is CCB clock in Hz.

 Jocke

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Mingkai Hu]

> -----Original Message-----
> From: Mingkai Hu
> Sent: Wednesday, September 06, 2017 5:37 PM
> To: 'Joakim Tjernlund' <Joakim.Tjernlund@infinera.com>; Roy Zang
> <roy.zang@nxp.com>; York Sun <york.sun@nxp.com>
> Cc: u-boot at lists.denx.de
> Subject: RE: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> 
> 
> 
> > -----Original Message-----
> > From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> > Sent: Tuesday, September 05, 2017 8:45 PM
> > To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> York
> > Sun <york.sun@nxp.com>
> > Cc: u-boot at lists.denx.de
> > Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> >
> > On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > > +Roy Zang to comment on PCIe clock source
> > > > >
> > > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is
> > > > > > > > > setup at all for E500 but I THINK this is required.
> > > > > > > > >
> > > > > > > > > In 83xx one do:
> > > > > > > > > get_clocks();
> > > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > > >
> > > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > > >
> > > > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk) /
> > > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > > >
> > > > > > > > > Any clues?
> > > > > > > > >
> > > > > > > > >     Jocke
> > > > > > > >
> > > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > > >
> > > > > > > > Anyhow, I put together this patch:
> > > > > > > >
> > > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct
> > > > > > > > pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > > > >
> > > > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > > > >
> > > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd-
> > >bus_clk / 333333333) * 16);
> > > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > > +
> > > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > > >
> > > > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > > >
> > > > > > >
> > > > > > > Jocke,
> > > > > > >
> > > > > > > You don't need to program this register if the actual PCIe
> > > > > > > clock is the same as default. Since SerDes reference clock
> > > > > > > has to be 100MHz for PCIe protocol, my guess is the PCIe
> > > > > > > clock is always correct. The bus_clk you are referring is
> > > > > > > not the PCIe clock. Again, I am not a PCIe expert, I could
> > > > > > > be wrong. Since PCIe (SerDes) has been working on multiple
> > > > > > > platforms, I guess the
> > clock is correct.
> > > > > >
> > > > > > I don't think so. Here is what T1042 says about this:
> > > > > >
> > > > > > The PCI Express controller clock frequency is one-half the
> > > > > > platform
> > clock frequency.
> > > > > >
> > > > > > The PCI Express controller core clock ratio register is used
> > > > > > to program the ratio of the actual PCI Express controller
> > > > > > clock frequency to the default controller core frequency ( 333 MHz ).
> > > > > > This is required only when a PCI Express controller clock
> > > > > > frequency
> > other than the default 333 MHz has to be used.
> > > > > > As an example of programming PEX_GCLK_RATIO, consider the case
> > > > > > where the actual PCI Express controller clock is 250 MHz, the
> > > > > > ratio of the actual clock to the default clock ( 333 MHz) is
> > > > > > 3:4. that is, the default core clock has to be multiplied by
> > > > > > the ratio (3/4, which is equivalent to 12/16). So the register
> > > > > > has to be
> > programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > > >
> > > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > > >
> > > > > > I found this on Google too:
> > > > > > The PEX controller hardware requires for timing tuning in
> > > > > > order to operate properly at given CCB/2 clock frequency.
> > > > > > PEX_GCLK_RATIO register controls this tuning with 333MHz/16
> > > > > > granularities. So for given CCB frequency Fccb [MHz] we must
> > > > > > write to
> > PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> > we have to write 18.
> > > > >
> > > > > Jocke,
> > > > >
> > > > > Basically I agree with you, if the clock is different from
> > > > > default, you need to set the ratio. T104x indeed uses 1/2
> > > > > platform clock as PCIe clock. I am not sure if P2010 uses the
> > > > > same way. Let me add Roy to this thread.
> > > > >
> > > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs?
> > > > > Don't we need to set PEX_GCLK_RATIO register? It is not even in
> > > > > the
> > header file.
> > > >
> > > > Patch was off, this works:
> > > > diff --git a/drivers/pci/fsl_pci_init.c
> > > > b/drivers/pci/fsl_pci_init.c index 52792dcd59..e7f2499784 100644
> > > > --- a/drivers/pci/fsl_pci_init.c
> > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller
> > > > *hose, struct fsl_pci_info *pci_info)
> > > >
> > > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > >
> > > > +       /* Configure the PCIE controller core clock ratio */
> > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > >
> > > pci_hose_write_config_dword() writes to config space, doesn't it?
> > > 0x440 is not in config space.
> > >
> > > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > > +       /* udelay(1000000) needed here ?*/
> > >
> > > Let us not to add delay if we don't know yet.
> > >
> > > > +
> > > >          block_rev = in_be32(&pci->block_rev1);
> > > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > > >
> > > >
> > > > Any news on this ?
> > > > I know both T1042 and P2010 has the same setting/formula, a good
> > > > start :)
> > >
> > > +Mingkai to comment on PCIe clock
> > >
> > > York
> >
> 
> This is the clock to PCIe controller which is not different from the clock to
> PCIe SerDes lane.
> My understanding is that it should be configured according to the platform
> frequency.
> But there is no specific and consistent description for all the platforms. I am
> checking with the design team to see if there is hardware fixed clock to PCIe
> controller for the platforms which don't have clear description. Will keep you
> all updated.
> 

This ratio will be used by GPEX only to calculate the actual timer values to be used for Ack Latency and
Replay timeout values and the FC update timer value under traffic condition. It's applicable for all PowerPC
controllers except T4 and T2 family, so would you please help to update the patch to set the register
according to platform clock (or platform/2, need to check chip RM) excluding T4/T2 platform?

Regards,
Mingkai

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Joakim Tjernlund]

On Thu, 2017-09-07 at 06:45 +0000, Mingkai Hu wrote:
> > -----Original Message-----
> > From: Mingkai Hu
> > Sent: Wednesday, September 06, 2017 5:37 PM
> > To: 'Joakim Tjernlund' <Joakim.Tjernlund@infinera.com>; Roy Zang
> > <roy.zang@nxp.com>; York Sun <york.sun@nxp.com>
> > Cc: u-boot at lists.denx.de
> > Subject: RE: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> > > Sent: Tuesday, September 05, 2017 8:45 PM
> > > To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> > 
> > York
> > > Sun <york.sun@nxp.com>
> > > Cc: u-boot at lists.denx.de
> > > Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > > 
> > > On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > > > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > > > +Roy Zang to comment on PCIe clock source
> > > > > > 
> > > > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is
> > > > > > > > > > setup at all for E500 but I THINK this is required.
> > > > > > > > > > 
> > > > > > > > > > In 83xx one do:
> > > > > > > > > > get_clocks();
> > > > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > > > > 
> > > > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > > > > 
> > > > > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk) /
> > > > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > > > > 
> > > > > > > > > > Any clues?
> > > > > > > > > > 
> > > > > > > > > >     Jocke
> > > > > > > > > 
> > > > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > > > > 
> > > > > > > > > Anyhow, I put together this patch:
> > > > > > > > > 
> > > > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct
> > > > > > > > > pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > > > > > 
> > > > > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > > > > > 
> > > > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd-
> > > > 
> > > > bus_clk / 333333333) * 16);
> > > > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > > > +
> > > > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > > > > 
> > > > > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Jocke,
> > > > > > > > 
> > > > > > > > You don't need to program this register if the actual PCIe
> > > > > > > > clock is the same as default. Since SerDes reference clock
> > > > > > > > has to be 100MHz for PCIe protocol, my guess is the PCIe
> > > > > > > > clock is always correct. The bus_clk you are referring is
> > > > > > > > not the PCIe clock. Again, I am not a PCIe expert, I could
> > > > > > > > be wrong. Since PCIe (SerDes) has been working on multiple
> > > > > > > > platforms, I guess the
> > > 
> > > clock is correct.
> > > > > > > 
> > > > > > > I don't think so. Here is what T1042 says about this:
> > > > > > > 
> > > > > > > The PCI Express controller clock frequency is one-half the
> > > > > > > platform
> > > 
> > > clock frequency.
> > > > > > > 
> > > > > > > The PCI Express controller core clock ratio register is used
> > > > > > > to program the ratio of the actual PCI Express controller
> > > > > > > clock frequency to the default controller core frequency ( 333 MHz ).
> > > > > > > This is required only when a PCI Express controller clock
> > > > > > > frequency
> > > 
> > > other than the default 333 MHz has to be used.
> > > > > > > As an example of programming PEX_GCLK_RATIO, consider the case
> > > > > > > where the actual PCI Express controller clock is 250 MHz, the
> > > > > > > ratio of the actual clock to the default clock ( 333 MHz) is
> > > > > > > 3:4. that is, the default core clock has to be multiplied by
> > > > > > > the ratio (3/4, which is equivalent to 12/16). So the register
> > > > > > > has to be
> > > 
> > > programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > > > > 
> > > > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > > > > 
> > > > > > > I found this on Google too:
> > > > > > > The PEX controller hardware requires for timing tuning in
> > > > > > > order to operate properly at given CCB/2 clock frequency.
> > > > > > > PEX_GCLK_RATIO register controls this tuning with 333MHz/16
> > > > > > > granularities. So for given CCB frequency Fccb [MHz] we must
> > > > > > > write to
> > > 
> > > PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> > > we have to write 18.
> > > > > > 
> > > > > > Jocke,
> > > > > > 
> > > > > > Basically I agree with you, if the clock is different from
> > > > > > default, you need to set the ratio. T104x indeed uses 1/2
> > > > > > platform clock as PCIe clock. I am not sure if P2010 uses the
> > > > > > same way. Let me add Roy to this thread.
> > > > > > 
> > > > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs?
> > > > > > Don't we need to set PEX_GCLK_RATIO register? It is not even in
> > > > > > the
> > > 
> > > header file.
> > > > > 
> > > > > Patch was off, this works:
> > > > > diff --git a/drivers/pci/fsl_pci_init.c
> > > > > b/drivers/pci/fsl_pci_init.c index 52792dcd59..e7f2499784 100644
> > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller
> > > > > *hose, struct fsl_pci_info *pci_info)
> > > > > 
> > > > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > 
> > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > > 
> > > > pci_hose_write_config_dword() writes to config space, doesn't it?
> > > > 0x440 is not in config space.
> > > > 
> > > > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > > > +       /* udelay(1000000) needed here ?*/
> > > > 
> > > > Let us not to add delay if we don't know yet.
> > > > 
> > > > > +
> > > > >          block_rev = in_be32(&pci->block_rev1);
> > > > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > > > > 
> > > > > 
> > > > > Any news on this ?
> > > > > I know both T1042 and P2010 has the same setting/formula, a good
> > > > > start :)
> > > > 
> > > > +Mingkai to comment on PCIe clock
> > > > 
> > > > York
> > 
> > This is the clock to PCIe controller which is not different from the clock to
> > PCIe SerDes lane.
> > My understanding is that it should be configured according to the platform
> > frequency.
> > But there is no specific and consistent description for all the platforms. I am
> > checking with the design team to see if there is hardware fixed clock to PCIe
> > controller for the platforms which don't have clear description. Will keep you
> > all updated.
> > 
> 
> This ratio will be used by GPEX only to calculate the actual timer values to be used for Ack Latency and
> Replay timeout values and the FC update timer value under traffic condition. It's applicable for all PowerPC
> controllers except T4 and T2 family, so would you please help to update the patch to set the register
> according to platform clock (or platform/2, need to check chip RM) excluding T4/T2 platform?

I think NXP is in a better position to adjust the patch w.r.t actual PCIe clock. I just
want to known, for now, what PCIe clock is for T1042 and P2010, CCB or CCB/2, so we can set
the correct value and continue with our trouble shooting.

T1040RM actually has this writing in 28.11.19:
The PCI Express controller clock frequency is one-half the platform clock frequency.
....
That indicates the PCIe clock is CCB/2. P2010 does not have that text. 
Please advice.

 Jocke

[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

[Mingkai Hu]

> -----Original Message-----
> From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> Sent: Thursday, September 07, 2017 2:55 PM
> To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>;
> York Sun <york.sun@nxp.com>
> Cc: u-boot at lists.denx.de
> Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> 
> On Thu, 2017-09-07 at 06:45 +0000, Mingkai Hu wrote:
> > > -----Original Message-----
> > > From: Mingkai Hu
> > > Sent: Wednesday, September 06, 2017 5:37 PM
> > > To: 'Joakim Tjernlund' <Joakim.Tjernlund@infinera.com>; Roy Zang
> > > <roy.zang@nxp.com>; York Sun <york.sun@nxp.com>
> > > Cc: u-boot at lists.denx.de
> > > Subject: RE: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> > > > Sent: Tuesday, September 05, 2017 8:45 PM
> > > > To: Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang
> <roy.zang@nxp.com>;
> > >
> > > York
> > > > Sun <york.sun@nxp.com>
> > > > Cc: u-boot at lists.denx.de
> > > > Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > > >
> > > > On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > > > > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > > > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > > > > +Roy Zang to comment on PCIe clock source
> > > > > > >
> > > > > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > > > > As we are looking at PCI stuff ATM I would like to
> > > > > > > > > > > ask about PEX_GCLK_RATIO in E500 CPUs. I cannot find
> > > > > > > > > > > this is setup at all for E500 but I THINK this is required.
> > > > > > > > > > >
> > > > > > > > > > > In 83xx one do:
> > > > > > > > > > > get_clocks();
> > > > > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > > > > >
> > > > > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > > > > >
> > > > > > > > > > > (((bus ? gd->arch.pciexp2_clk :
> > > > > > > > > > > gd->arch.pciexp1_clk) /
> > > > > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > > > > >
> > > > > > > > > > > Any clues?
> > > > > > > > > > >
> > > > > > > > > > >     Jocke
> > > > > > > > > >
> > > > > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > > > > >
> > > > > > > > > > Anyhow, I put together this patch:
> > > > > > > > > >
> > > > > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct
> > > > > > > > > > pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > > > > > >
> > > > > > > > > >            pci_setup_indirect(hose, cfg_addr,
> > > > > > > > > > cfg_data);
> > > > > > > > > >
> > > > > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > > > > > > > > + (gd-
> > > > >
> > > > > bus_clk / 333333333) * 16);
> > > > > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > > > > +
> > > > > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > > > > >
> > > > > > > > > > Any chance this will work for all supported FSL PCIe
> controllers?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Jocke,
> > > > > > > > >
> > > > > > > > > You don't need to program this register if the actual
> > > > > > > > > PCIe clock is the same as default. Since SerDes
> > > > > > > > > reference clock has to be 100MHz for PCIe protocol, my
> > > > > > > > > guess is the PCIe clock is always correct. The bus_clk
> > > > > > > > > you are referring is not the PCIe clock. Again, I am not
> > > > > > > > > a PCIe expert, I could be wrong. Since PCIe (SerDes) has
> > > > > > > > > been working on multiple platforms, I guess the
> > > >
> > > > clock is correct.
> > > > > > > >
> > > > > > > > I don't think so. Here is what T1042 says about this:
> > > > > > > >
> > > > > > > > The PCI Express controller clock frequency is one-half the
> > > > > > > > platform
> > > >
> > > > clock frequency.
> > > > > > > >
> > > > > > > > The PCI Express controller core clock ratio register is
> > > > > > > > used to program the ratio of the actual PCI Express
> > > > > > > > controller clock frequency to the default controller core
> frequency ( 333 MHz ).
> > > > > > > > This is required only when a PCI Express controller clock
> > > > > > > > frequency
> > > >
> > > > other than the default 333 MHz has to be used.
> > > > > > > > As an example of programming PEX_GCLK_RATIO, consider the
> > > > > > > > case where the actual PCI Express controller clock is 250
> > > > > > > > MHz, the ratio of the actual clock to the default clock (
> > > > > > > > 333 MHz) is 3:4. that is, the default core clock has to be
> > > > > > > > multiplied by the ratio (3/4, which is equivalent to
> > > > > > > > 12/16). So the register has to be
> > > >
> > > > programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > > > > >
> > > > > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > > > > >
> > > > > > > > I found this on Google too:
> > > > > > > > The PEX controller hardware requires for timing tuning in
> > > > > > > > order to operate properly at given CCB/2 clock frequency.
> > > > > > > > PEX_GCLK_RATIO register controls this tuning with
> > > > > > > > 333MHz/16 granularities. So for given CCB frequency Fccb
> > > > > > > > [MHz] we must write to
> > > >
> > > > PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for
> > > > Fccb=375 we have to write 18.
> > > > > > >
> > > > > > > Jocke,
> > > > > > >
> > > > > > > Basically I agree with you, if the clock is different from
> > > > > > > default, you need to set the ratio. T104x indeed uses 1/2
> > > > > > > platform clock as PCIe clock. I am not sure if P2010 uses
> > > > > > > the same way. Let me add Roy to this thread.
> > > > > > >
> > > > > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx)
> SoCs?
> > > > > > > Don't we need to set PEX_GCLK_RATIO register? It is not even
> > > > > > > in the
> > > >
> > > > header file.
> > > > > >
> > > > > > Patch was off, this works:
> > > > > > diff --git a/drivers/pci/fsl_pci_init.c
> > > > > > b/drivers/pci/fsl_pci_init.c index 52792dcd59..e7f2499784
> > > > > > 100644
> > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller
> > > > > > *hose, struct fsl_pci_info *pci_info)
> > > > > >
> > > > > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > >
> > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > > >
> > > > > pci_hose_write_config_dword() writes to config space, doesn't it?
> > > > > 0x440 is not in config space.
> > > > >
> > > > > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > > > > +       /* udelay(1000000) needed here ?*/
> > > > >
> > > > > Let us not to add delay if we don't know yet.
> > > > >
> > > > > > +
> > > > > >          block_rev = in_be32(&pci->block_rev1);
> > > > > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > > > > >
> > > > > >
> > > > > > Any news on this ?
> > > > > > I know both T1042 and P2010 has the same setting/formula, a
> > > > > > good start :)
> > > > >
> > > > > +Mingkai to comment on PCIe clock
> > > > >
> > > > > York
> > >
> > > This is the clock to PCIe controller which is not different from the
> > > clock to PCIe SerDes lane.
> > > My understanding is that it should be configured according to the
> > > platform frequency.
> > > But there is no specific and consistent description for all the
> > > platforms. I am checking with the design team to see if there is
> > > hardware fixed clock to PCIe controller for the platforms which
> > > don't have clear description. Will keep you all updated.
> > >
> >
> > This ratio will be used by GPEX only to calculate the actual timer
> > values to be used for Ack Latency and Replay timeout values and the FC
> > update timer value under traffic condition. It's applicable for all
> > PowerPC controllers except T4 and T2 family, so would you please help to
> update the patch to set the register according to platform clock (or
> platform/2, need to check chip RM) excluding T4/T2 platform?
> 
> I think NXP is in a better position to adjust the patch w.r.t actual PCIe clock. I
> just want to known, for now, what PCIe clock is for T1042 and P2010, CCB or
> CCB/2, so we can set the correct value and continue with our trouble
> shooting.
> 
> T1040RM actually has this writing in 28.11.19:
> The PCI Express controller clock frequency is one-half the platform clock
> frequency.
> ....
> That indicates the PCIe clock is CCB/2. P2010 does not have that text.
> Please advice.
> 

For P2010/P2020, CCB clock is routed to PCIe controller.

Mingkai