Re: [Patch v4 01/10] dt-bindings: memory: tegra: add bpmp ref in tegra234-mc node

From: Thierry Reding <thierry.reding@gmail.com>
To: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Cc: Sumit Gupta <sumitg@nvidia.com>,
	treding@nvidia.com, dmitry.osipenko@collabora.com,
	viresh.kumar@linaro.org, rafael@kernel.org, jonathanh@nvidia.com,
	robh+dt@kernel.org, lpieralisi@kernel.org,
	linux-kernel@vger.kernel.org, linux-tegra@vger.kernel.org,
	linux-pm@vger.kernel.org, devicetree@vger.kernel.org,
	linux-pci@vger.kernel.org, mmaddireddy@nvidia.com, kw@linux.com,
	bhelgaas@google.com, vidyas@nvidia.com, sanjayc@nvidia.com,
	ksitaraman@nvidia.com, ishah@nvidia.com, bbasu@nvidia.com
Subject: Re: [Patch v4 01/10] dt-bindings: memory: tegra: add bpmp ref in tegra234-mc node
Date: Tue, 4 Apr 2023 13:44:31 +0200	[thread overview]
Message-ID: <ZCwNn0xaBTKfZJmV@orome> (raw)
In-Reply-To: <8a22aea9-5027-e8a4-db80-ce79f3830e10@linaro.org>

[-- Attachment #1: Type: text/plain, Size: 4220 bytes --]

On Sun, Apr 02, 2023 at 12:47:01PM +0200, Krzysztof Kozlowski wrote:
> On 29/03/2023 19:12, Sumit Gupta wrote:
> > 
> > 
> > On 28/03/23 18:18, Thierry Reding wrote:
> >> On Tue, Mar 28, 2023 at 01:22:26PM +0200, Krzysztof Kozlowski wrote:
> >>> On 28/03/2023 12:48, Thierry Reding wrote:
> >>>> On Tue, Mar 28, 2023 at 09:23:04AM +0200, Krzysztof Kozlowski wrote:
> >>>>> On 27/03/2023 18:14, Sumit Gupta wrote:
> >>>>>> For Tegra234, add the "nvidia,bpmp" property within the Memory
> >>>>>> Controller (MC) node to reference BPMP node. This is needed in
> >>>>>> the MC driver to pass the client info to the BPMP-FW when memory
> >>>>>> interconnect support is available.
> >>>>>>
> >>>>>> Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
> >>>>>> ---
> >>>>>>   .../bindings/memory-controllers/nvidia,tegra186-mc.yaml    | 7 +++++++
> >>>>>>   1 file changed, 7 insertions(+)
> >>>>>>
> >>>>>> diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml
> >>>>>> index 935d63d181d9..398d27bb2373 100644
> >>>>>> --- a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml
> >>>>>> +++ b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml
> >>>>>> @@ -58,6 +58,10 @@ properties:
> >>>>>>     "#interconnect-cells":
> >>>>>>       const: 1
> >>>>>>   
> >>>>>> +  nvidia,bpmp:
> >>>>>> +    $ref: /schemas/types.yaml#/definitions/phandle
> >>>>>> +    description: phandle of the node representing the BPMP
> >>>>>
> >>>>> Why do you need this multiple times? Both in parent and all external-mc
> >>>>> children?
> >>>>
> >>>> We've had nvidia,bpmp in the external memory controller node since
> >>>> basically the beginning because we've always needed it there. For newer
> >>>> chips we now also need it for the memory controller.
> >>>>
> >>>> Ideally I think we would only have this in the MC and have the EMC
> >>>> driver reference it via the EMC's parent (i.e. MC), but that would break
> >>>> backwards-compatibility. Reaching into the EMC's DT node from the MC was
> >>>> another option that we discussed internally, but it didn't look right
> >>>> given how this is also needed by the MC.
> >>>>
> >>>> One thing we could potentially do is deprecate the nvidia,bpmp phandle
> >>>> in the EMC and only keep it as a fallback in the drivers in case the
> >>>> parent MC doesn't find it's own in the DT.
> >>>
> >>> Yes, deprecation would answer to my question.
> >>
> >> Okay, great. Sumit, you can resolve this by adding a "deprecated: true"
> >> to the EMC's nvidia,bpmp property schema. In the driver we can then try
> >> to look at the MC's ->bpmp and if it exists reuse that. If it doesn't
> >> exist, we can keep the existing lookup as a fallback for device trees
> >> that haven't been updated yet.
> > 
> > We can't use MC's->bpmp in the EMC driver's probe as it will be NULL. 
> > This is because MC driver uses "arch_initcall" and gets probed earlier 
> > than BPMP. We can do this in another way as below change. This way we 
> > can use the existing "nvidia,bpmp" property from EMC node and don't need 
> > to move it to the MC node. Please share if this change sounds OK.
> 
> Then rather it sounds like time to fix these
> orderings/arch_initcall/missing defer.

We can't fix this because there's a circular dependency between MC and
BPMP. Essentially BPMP requires the IOMMU for mappings and the IOMMU
needs the MC to do some register programming in order for the mappings
to take effect.

The MC programming isn't required for BPMP, but it is required for other
devices that are hooked up to an IOMMU. The dependency from MC on BPMP
is also a different area of functionality, so we know that there's no
actual problem.

If there was something like a "transitive" dependency we might be able
to resolve this (i.e. if creating the IOMMU mapping itself would trigger
the probe deferral chain, rather than the driver binding). However, all
dependencies are modelled between devices, so not much we can do in this
case other than try to work around it.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]