[PATCH v7 0/6] iommu/arm-smmu: Enable split pagetable support

[PATCH v7 0/6] iommu/arm-smmu: Enable split pagetable support

[Jordan Crouse]

Another iteration of the split-pagetable support for arm-smmu and the Adreno GPU
SMMU. After email discussions [1] we opted to make a arm-smmu implementation for
specifically for the Adreno GPU and use that to enable split pagetable support
and later other implementation specific bits that we need.

On the hardware side this is very close to the same code from before [2] only
the TTBR1 quirk is turned on by the implementation and not a domain attribute.
In drm/msm we use the returned size of the aperture as a clue to let us know
which virtual address space we should use for global memory objects.

There are two open items that you should be aware of. First, in the
implementation specific code we have to check the compatible string of the
device so that we only enable TTBR1 for the GPU (SID 0) and not the GMU (SID 4).
I went back and forth trying to decide if I wanted to use the compatbile string
or the SID as the filter and settled on the compatible string but I could be
talked out of it.

The other open item is that in drm/msm the hardware only uses 49 bits of the
address space but arm-smmu expects the address to be sign extended all the way
to 64 bits. This isn't a problem normally unless you look at the hardware
registers that contain a IOVA and then the upper bits will be zero. I opted to
restrict the internal drm/msm IOVA range to only 49 bits and then sign extend
right before calling iommu_map / iommu_unmap. This is a bit wonky but I thought
that matching the hardware would be less confusing when debugging a hang.

[1] https://lists.linuxfoundation.org/pipermail/iommu/2020-May/044537.html
[2] https://patchwork.kernel.org/patch/11482591/


Jordan Crouse (6):
  iommu/arm-smmu: Pass io-pgtable config to implementation specific
    function
  iommu/arm-smmu: Add support for split pagetables
  dt-bindings: arm-smmu: Add compatible string for Adreno GPU SMMU
  iommu/arm-smmu: Add implementation for the adreno GPU SMMU
  drm/msm: Set the global virtual address range from the IOMMU domain
  arm6: dts: qcom: sm845: Set the compatible string for the GPU SMMU

 .../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
 arch/arm64/boot/dts/qcom/sdm845.dtsi          |  2 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c       | 13 ++++++-
 drivers/gpu/drm/msm/msm_iommu.c               |  7 ++++
 drivers/iommu/arm-smmu-impl.c                 |  6 ++-
 drivers/iommu/arm-smmu-qcom.c                 | 38 ++++++++++++++++++-
 drivers/iommu/arm-smmu.c                      | 32 +++++++++++-----
 drivers/iommu/arm-smmu.h                      | 29 ++++++++++----
 8 files changed, 108 insertions(+), 23 deletions(-)

-- 
2.17.1

[PATCH v7 3/6] dt-bindings: arm-smmu: Add compatible string for Adreno GPU SMMU

[Jordan Crouse]

Every Qcom Adreno GPU has an embedded SMMU for its own use. These
devices depend on unique features such as split pagetables,
different stall/halt requirements and other settings. Identify them
with a compatible string so that they can be identified in the
arm-smmu implementation specific code.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
---

 Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.yaml b/Documentation/devicetree/bindings/iommu/arm,smmu.yaml
index d7ceb4c34423..e52a1b146c97 100644
--- a/Documentation/devicetree/bindings/iommu/arm,smmu.yaml
+++ b/Documentation/devicetree/bindings/iommu/arm,smmu.yaml
@@ -38,6 +38,10 @@ properties:
               - qcom,sc7180-smmu-500
               - qcom,sdm845-smmu-500
           - const: arm,mmu-500
+      - description: Qcom Adreno GPUs implementing "arm,smmu-v2"
+        items:
+          - const: qcom,adreno-smmu
+          - const: qcom,smmu-v2
       - items:
           - const: arm,mmu-500
           - const: arm,smmu-v2
-- 
2.17.1

[PATCH v7 6/6] arm6: dts: qcom: sm845: Set the compatible string for the GPU SMMU

[Jordan Crouse]

Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
split pagetables.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
---

 arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
index 8eb5a31346d2..8b15cd74e9ba 100644
--- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
@@ -3556,7 +3556,7 @@
 		};
 
 		adreno_smmu: iommu@5040000 {
-			compatible = "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
+			compatible = "qcom,adreno-smmu", "qcom,smmu-v2";
 			reg = <0 0x5040000 0 0x10000>;
 			#iommu-cells = <1>;
 			#global-interrupts = <2>;
-- 
2.17.1

Re: [Freedreno] [PATCH v7 6/6] arm6: dts: qcom: sm845: Set the compatible string for the GPU SMMU

[Rob Clark]

On Thu, Jun 4, 2020 at 1:57 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
>
> Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
> split pagetables.
>
> Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
> ---
>
>  arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> index 8eb5a31346d2..8b15cd74e9ba 100644
> --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
> +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> @@ -3556,7 +3556,7 @@
>                 };
>
>                 adreno_smmu: iommu@5040000 {
> -                       compatible = "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> +                       compatible = "qcom,adreno-smmu", "qcom,smmu-v2";

Should this be something like:

   compatible = "qcom,adreno-smmu", "qcom,sdm845-smmu-v2", "qcom,smmu-v2";

so that new dtb still works with old kernel?

BR,
-R

>                         reg = <0 0x5040000 0 0x10000>;
>                         #iommu-cells = <1>;
>                         #global-interrupts = <2>;
> --
> 2.17.1
>
> _______________________________________________
> Freedreno mailing list
> Freedreno@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/freedreno

Re: [Freedreno] [PATCH v7 6/6] arm6: dts: qcom: sm845: Set the compatible string for the GPU SMMU

[Jordan Crouse]

On Thu, Jun 04, 2020 at 02:27:21PM -0700, Rob Clark wrote:
> On Thu, Jun 4, 2020 at 1:57 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
> >
> > Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
> > split pagetables.
> >
> > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
> > ---
> >
> >  arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > index 8eb5a31346d2..8b15cd74e9ba 100644
> > --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > @@ -3556,7 +3556,7 @@
> >                 };
> >
> >                 adreno_smmu: iommu@5040000 {
> > -                       compatible = "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> > +                       compatible = "qcom,adreno-smmu", "qcom,smmu-v2";
> 
> Should this be something like:
> 
>    compatible = "qcom,adreno-smmu", "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> 
> so that new dtb still works with old kernel?

As far as I know qcom,sdm845-smmu-v2 never was actually used unless we were
doing the parsing trick somewhere.

Jordan

> >                         reg = <0 0x5040000 0 0x10000>;
> >                         #iommu-cells = <1>;
> >                         #global-interrupts = <2>;
> > --
> > 2.17.1
> >
> > _______________________________________________
> > Freedreno mailing list
> > Freedreno@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/freedreno
> _______________________________________________
> Freedreno mailing list
> Freedreno@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/freedreno

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Freedreno] [PATCH v7 6/6] arm6: dts: qcom: sm845: Set the compatible string for the GPU SMMU

[Rob Clark]

On Thu, Jun 4, 2020 at 3:02 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
>
> On Thu, Jun 04, 2020 at 02:27:21PM -0700, Rob Clark wrote:
> > On Thu, Jun 4, 2020 at 1:57 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
> > >
> > > Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
> > > split pagetables.
> > >
> > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
> > > ---
> > >
> > >  arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > > index 8eb5a31346d2..8b15cd74e9ba 100644
> > > --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > > +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > > @@ -3556,7 +3556,7 @@
> > >                 };
> > >
> > >                 adreno_smmu: iommu@5040000 {
> > > -                       compatible = "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> > > +                       compatible = "qcom,adreno-smmu", "qcom,smmu-v2";
> >
> > Should this be something like:
> >
> >    compatible = "qcom,adreno-smmu", "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> >
> > so that new dtb still works with old kernel?
>
> As far as I know qcom,sdm845-smmu-v2 never was actually used unless we were
> doing the parsing trick somewhere.

I *think* it has a use queued up for v5.8 for the
select-direct-mapping stuff?  I'm not 100% sure of the status of that
off the top of my head.

BR,
-R

> Jordan
>
> > >                         reg = <0 0x5040000 0 0x10000>;
> > >                         #iommu-cells = <1>;
> > >                         #global-interrupts = <2>;
> > > --
> > > 2.17.1
> > >
> > > _______________________________________________
> > > Freedreno mailing list
> > > Freedreno@lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/freedreno
> > _______________________________________________
> > Freedreno mailing list
> > Freedreno@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/freedreno
>
> --
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> a Linux Foundation Collaborative Project

Re: [Freedreno] [PATCH v7 6/6] arm6: dts: qcom: sm845: Set the compatible string for the GPU SMMU

[Jordan Crouse]

On Thu, Jun 04, 2020 at 03:38:28PM -0700, Rob Clark wrote:
> On Thu, Jun 4, 2020 at 3:02 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
> >
> > On Thu, Jun 04, 2020 at 02:27:21PM -0700, Rob Clark wrote:
> > > On Thu, Jun 4, 2020 at 1:57 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
> > > >
> > > > Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
> > > > split pagetables.
> > > >
> > > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
> > > > ---
> > > >
> > > >  arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > > > index 8eb5a31346d2..8b15cd74e9ba 100644
> > > > --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > > > +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
> > > > @@ -3556,7 +3556,7 @@
> > > >                 };
> > > >
> > > >                 adreno_smmu: iommu@5040000 {
> > > > -                       compatible = "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> > > > +                       compatible = "qcom,adreno-smmu", "qcom,smmu-v2";
> > >
> > > Should this be something like:
> > >
> > >    compatible = "qcom,adreno-smmu", "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
> > >
> > > so that new dtb still works with old kernel?
> >
> > As far as I know qcom,sdm845-smmu-v2 never was actually used unless we were
> > doing the parsing trick somewhere.
> 
> I *think* it has a use queued up for v5.8 for the
> select-direct-mapping stuff?  I'm not 100% sure of the status of that
> off the top of my head.

That uses the compatible string of the consumer device.  I can add
it back in but I didn't think we needed to complicate the bindings any more if
we didn't have to.

Jordan

> BR,
> -R
> 
> > Jordan
> >
> > > >                         reg = <0 0x5040000 0 0x10000>;
> > > >                         #iommu-cells = <1>;
> > > >                         #global-interrupts = <2>;
> > > > --
> > > > 2.17.1
> > > >
> > > > _______________________________________________
> > > > Freedreno mailing list
> > > > Freedreno@lists.freedesktop.org
> > > > https://lists.freedesktop.org/mailman/listinfo/freedreno
> > > _______________________________________________
> > > Freedreno mailing list
> > > Freedreno@lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/freedreno
> >
> > --
> > The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> > a Linux Foundation Collaborative Project
> _______________________________________________
> Freedreno mailing list
> Freedreno@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/freedreno

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project