Re: [PATCH v4 4/4] PCIe: qcom: Add support to control pipe clk src

[Bjorn Andersson <bjorn.andersson@linaro.org>]

On Fri 16 Jul 16:39 CDT 2021, Stephen Boyd wrote:

> Quoting Bjorn Andersson (2021-07-16 13:31:55)
> > On Fri 16 Jul 14:37 CDT 2021, Stephen Boyd wrote:
> >
> > > Quoting Prasad Malisetty (2021-07-16 06:58:47)
> > > > This is a new requirement for sc7280 SoC.
> > > > To enable gdsc gcc_pcie_1_pipe_clk_src should be TCXO.
> > >
> > > Why? Can you add that detail here? Presumably it's something like the
> > > GDSC needs a running clk to send a reset through the flops or something
> > > like that.
> > >
> >
> > Which presumably means that we need to "park" gcc_pcie_N_pipe_clk_src
> > whenever the PHY pipe is paused due to a suspend or runtime suspend.
> >
> > I find this part of the commit message to primarily describing the next
> > patch (that is yet to be posted).
> 
> Ah I see. So there will be another patch to do the park and unpark over
> suspend/resume?
> 

That's my understanding.

> >
> > > > after PHY initialization gcc_pcie_1_pipe_clk_src needs
> > > > to switch from TCXO to gcc_pcie_1_pipe_clk.
> > > >
> > > > Signed-off-by: Prasad Malisetty <pmaliset@codeaurora.org>
> > > > ---
> > > >  drivers/pci/controller/dwc/pcie-qcom.c | 22 ++++++++++++++++++++++
> > > >  1 file changed, 22 insertions(+)
> > > >
> > > > diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
> > > > index 8a7a300..9e0e4ab 100644
> > > > --- a/drivers/pci/controller/dwc/pcie-qcom.c
> > > > +++ b/drivers/pci/controller/dwc/pcie-qcom.c
> > > > @@ -1167,6 +1170,20 @@ static int qcom_pcie_get_resources_2_7_0(struct qcom_pcie *pcie)
> > > >         if (ret < 0)
> > > >                 return ret;
> > > >
> > > > +       if (of_device_is_compatible(dev->of_node, "qcom,pcie-sc7280")) {
> > > > +               res->gcc_pcie_1_pipe_clk_src = devm_clk_get(dev, "pipe_mux");
> > > > +               if (IS_ERR(res->gcc_pcie_1_pipe_clk_src))
> > > > +                       return PTR_ERR(res->gcc_pcie_1_pipe_clk_src);
> > > > +
> > > > +               res->phy_pipe_clk = devm_clk_get(dev, "phy_pipe");
> > > > +               if (IS_ERR(res->phy_pipe_clk))
> > > > +                       return PTR_ERR(res->phy_pipe_clk);
> > > > +
> > > > +               res->ref_clk_src = devm_clk_get(dev, "ref");
> > > > +               if (IS_ERR(res->ref_clk_src))
> > > > +                       return PTR_ERR(res->ref_clk_src);
> > > > +       }
> > > > +
> > > >         res->pipe_clk = devm_clk_get(dev, "pipe");
> > > >         return PTR_ERR_OR_ZERO(res->pipe_clk);
> > > >  }
> > > > @@ -1255,6 +1272,11 @@ static void qcom_pcie_deinit_2_7_0(struct qcom_pcie *pcie)
> > > >  static int qcom_pcie_post_init_2_7_0(struct qcom_pcie *pcie)
> > > >  {
> > > >         struct qcom_pcie_resources_2_7_0 *res = &pcie->res.v2_7_0;
> > > > +       struct dw_pcie *pci = pcie->pci;
> > > > +       struct device *dev = pci->dev;
> > > > +
> > > > +       if (of_device_is_compatible(dev->of_node, "qcom,pcie-sc7280"))
> > > > +               clk_set_parent(res->gcc_pcie_1_pipe_clk_src, res->phy_pipe_clk);
> > >
> > > Is anything wrong if we call clk_set_parent() here when this driver is
> > > running on previous SoCs where the parent is assigned via DT?
> >
> > We don't assign the parent on previous platforms, we apparently just
> > rely on the reset value (afaict).
> 
> Oh sheesh. I thought that was being done already. It looks like at least
> on sdm845 that there is only one parent for this clk so we don't need to
> call clk_set_parent to set it there.
> 

I'll have to check the documentation on that...

> >
> > So I think it makes sense for all platforms to explicitly mux
> > pipe_clk_src to phy::pipe_clk, one the PHY is up and running.
> 
> Sure, except some platforms don't have a mux?
> 
> >
> > But I was under the impression that we have the BRANCH_HALT_SKIP on the
> > pipe_clk because there was some sort of feedback loop to the PHY's
> > calibration... What this patch indicates is that we should park
> > pipe_clk_src onto XO at boot time, then after the PHY starts ticking we
> > should enable and reparent the clk_src - at which point I don't see why
> > we need the HALT_SKIP.
> 
> I recall that qcom folks kept saying they needed to enable the
> pipe_clk_src clk branch in GCC before enabling the phy. So they required
> the halt skip flag so that the clk_prepare_enable() call would
> effectively set the enable bit in GCC and move on without caring. Then
> they could enable the upstream clk source in the phy without having to
> stop halfway through to enable the branch in GCC. The whole design here
> is pretty insane.
> 
> In fact, I think we discussed this whole topic in late 2019[1] and we
> concluded that we could just slam the clk on forever and deal with the
> clk_set_parent() when the clk became a mux+gate instead of a pure gate.
> 

That's exactly what I asked Prasad about, because per the description
and content of this patch the parent pipe_clk_src will remain XO until
the PHY is initialized. So either the PHY no longer need gcc in the loop
to calibrate the pipe clock or it used to, but no longer does.


Thanks for the link, we definitely should clean that up, but I think at
this point it might be worth waiting a little bit longer to see what
actually going to happen in the suspend/resume (system and runtime)
paths...

> >
> > > Also, shouldn't we make sure the parent is XO at driver probe time so
> > > that powering on the GDSC works properly?
> > >
> > > It all feels like a kludge though given that the GDSC is the one that
> > > requires the clock to be running at XO and we're working around that in
> > > the pcie driver instead of sticking that logic into the GDSC. What do we
> > > do if the GDSC is already enabled out of boot instead of being the power
> > > on reset (POR) configuration?
> > >
> >
> > What happens if we boot the device out of NVME...
> 
> I guess it's fine? The GDSC will be on and the parent clk will already
> be set so things are a no-op.
> 

Yes, if the pipe_clk_src is parked nicely in late initcall, so that when the
pd late init cuts the GDSC things will end up in a clean state.

Regards,
Bjorn

> >
> >
> > PS. Are we certain that it's the PCIe driver and not the PHY that should
> > do this dance? I really would like to see the continuation of this patch
> > to see the full picture...
> >
> 
> [1] https://lore.kernel.org/linux-clk/eba920f5-f5a2-53d5-2227-529b5ea99d32@codeaurora.org/