[PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[Tom St Denis]

The function amdgpu_mm_wreg_mmio_rlc() is used by debugfs to write to
MMIO registers.  It didn't support registers beyond the BAR mapped MMIO
space.  This adds pcie indirect write support.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index c38e0e87090b..53a04095a6db 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -552,7 +552,7 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,
 }
 
 /**
- * amdgpu_mm_wreg_mmio_rlc -  write register either with mmio or with RLC path if in range
+ * amdgpu_mm_wreg_mmio_rlc -  write register either with direct/indirect mmio or with RLC path if in range
  *
  * this function is invoked only the debugfs register access
  */
@@ -567,6 +567,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
 	    adev->gfx.rlc.funcs->is_rlcg_access_range) {
 		if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
 			return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0);
+	} else if ((reg * 4) >= adev->rmmio_size) {
+		adev->pcie_wreg(adev, reg * 4, v);
 	} else {
 		writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
 	}
-- 
2.32.0

Re: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[Deucher, Alexander]

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

[AMD Official Use Only]

Acked-by: Alex Deucher <alexander.deucher@amd.com>
________________________________
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Tom St Denis <tom.stdenis@amd.com>
Sent: Friday, January 7, 2022 7:07 AM
To: amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>
Cc: StDenis, Tom <Tom.StDenis@amd.com>
Subject: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

The function amdgpu_mm_wreg_mmio_rlc() is used by debugfs to write to
MMIO registers.  It didn't support registers beyond the BAR mapped MMIO
space.  This adds pcie indirect write support.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index c38e0e87090b..53a04095a6db 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -552,7 +552,7 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,
 }

 /**
- * amdgpu_mm_wreg_mmio_rlc -  write register either with mmio or with RLC path if in range
+ * amdgpu_mm_wreg_mmio_rlc -  write register either with direct/indirect mmio or with RLC path if in range
  *
  * this function is invoked only the debugfs register access
  */
@@ -567,6 +567,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
             adev->gfx.rlc.funcs->is_rlcg_access_range) {
                 if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
                         return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0);
+       } else if ((reg * 4) >= adev->rmmio_size) {
+               adev->pcie_wreg(adev, reg * 4, v);
         } else {
                 writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
         }
--
2.32.0


[-- Attachment #2: Type: text/html, Size: 3542 bytes --]

RE: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[Zhang, Hawking]

[AMD Official Use Only]

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>

Regards,
Hawking
-----Original Message-----
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Tom St Denis
Sent: Friday, January 7, 2022 20:08
To: amd-gfx@lists.freedesktop.org
Cc: StDenis, Tom <Tom.StDenis@amd.com>
Subject: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

The function amdgpu_mm_wreg_mmio_rlc() is used by debugfs to write to MMIO registers.  It didn't support registers beyond the BAR mapped MMIO space.  This adds pcie indirect write support.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index c38e0e87090b..53a04095a6db 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -552,7 +552,7 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,  }
 
 /**
- * amdgpu_mm_wreg_mmio_rlc -  write register either with mmio or with RLC path if in range
+ * amdgpu_mm_wreg_mmio_rlc -  write register either with 
+ direct/indirect mmio or with RLC path if in range
  *
  * this function is invoked only the debugfs register access
  */
@@ -567,6 +567,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
 	    adev->gfx.rlc.funcs->is_rlcg_access_range) {
 		if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
 			return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0);
+	} else if ((reg * 4) >= adev->rmmio_size) {
+		adev->pcie_wreg(adev, reg * 4, v);
 	} else {
 		writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
 	}
--
2.32.0

Re: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[Alex Deucher]

On Fri, Jan 7, 2022 at 7:07 AM Tom St Denis <tom.stdenis@amd.com> wrote:
>
> The function amdgpu_mm_wreg_mmio_rlc() is used by debugfs to write to
> MMIO registers.  It didn't support registers beyond the BAR mapped MMIO
> space.  This adds pcie indirect write support.
>
> Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index c38e0e87090b..53a04095a6db 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -552,7 +552,7 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,
>  }
>
>  /**
> - * amdgpu_mm_wreg_mmio_rlc -  write register either with mmio or with RLC path if in range
> + * amdgpu_mm_wreg_mmio_rlc -  write register either with direct/indirect mmio or with RLC path if in range
>   *
>   * this function is invoked only the debugfs register access
>   */
> @@ -567,6 +567,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
>             adev->gfx.rlc.funcs->is_rlcg_access_range) {
>                 if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
>                         return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0);
> +       } else if ((reg * 4) >= adev->rmmio_size) {
> +               adev->pcie_wreg(adev, reg * 4, v);

Actually, for older asics, we shouldn't we be using mmINDEX/mmDATA
rather than the pcie indirect registers?  Or is that handled already
somehow?

Alex

>         } else {
>                 writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
>         }
> --
> 2.32.0
>

RE: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[Zhang, Hawking]

[AMD Official Use Only]

RE - Actually, for older asics, we shouldn't we be using mmINDEX/mmDATA rather than the pcie indirect registers?  Or is that handled already somehow?

I remember we checked this with hw team before (might two years ago when make the change in amdgpu_device_rreg/wreg helper). The answer was it is safe to retire mmINDEX/DATA approach for SI and onwards. PCIE_INDEX/DATA should be good enough for indirect access in amdgpu driver. For radeon driver, mmINDEX/mmDATA shall still be kept.

Regards,
Hawking
-----Original Message-----
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Alex Deucher
Sent: Sunday, January 9, 2022 05:13
To: StDenis, Tom <Tom.StDenis@amd.com>
Cc: amd-gfx list <amd-gfx@lists.freedesktop.org>
Subject: Re: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

On Fri, Jan 7, 2022 at 7:07 AM Tom St Denis <tom.stdenis@amd.com> wrote:
>
> The function amdgpu_mm_wreg_mmio_rlc() is used by debugfs to write to 
> MMIO registers.  It didn't support registers beyond the BAR mapped 
> MMIO space.  This adds pcie indirect write support.
>
> Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index c38e0e87090b..53a04095a6db 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -552,7 +552,7 @@ void amdgpu_device_wreg(struct amdgpu_device 
> *adev,  }
>
>  /**
> - * amdgpu_mm_wreg_mmio_rlc -  write register either with mmio or with 
> RLC path if in range
> + * amdgpu_mm_wreg_mmio_rlc -  write register either with 
> + direct/indirect mmio or with RLC path if in range
>   *
>   * this function is invoked only the debugfs register access
>   */
> @@ -567,6 +567,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
>             adev->gfx.rlc.funcs->is_rlcg_access_range) {
>                 if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
>                         return adev->gfx.rlc.funcs->sriov_wreg(adev, 
> reg, v, 0, 0);
> +       } else if ((reg * 4) >= adev->rmmio_size) {
> +               adev->pcie_wreg(adev, reg * 4, v);

Actually, for older asics, we shouldn't we be using mmINDEX/mmDATA rather than the pcie indirect registers?  Or is that handled already somehow?

Alex

>         } else {
>                 writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
>         }
> --
> 2.32.0
>

Re: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()

[Alex Deucher]

On Sun, Jan 9, 2022 at 5:27 AM Zhang, Hawking <Hawking.Zhang@amd.com> wrote:
>
> [AMD Official Use Only]
>
> RE - Actually, for older asics, we shouldn't we be using mmINDEX/mmDATA rather than the pcie indirect registers?  Or is that handled already somehow?
>
> I remember we checked this with hw team before (might two years ago when make the change in amdgpu_device_rreg/wreg helper). The answer was it is safe to retire mmINDEX/DATA approach for SI and onwards. PCIE_INDEX/DATA should be good enough for indirect access in amdgpu driver. For radeon driver, mmINDEX/mmDATA shall still be kept.
>

Thanks for the confirmation.

Alex

> Regards,
> Hawking
> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Alex Deucher
> Sent: Sunday, January 9, 2022 05:13
> To: StDenis, Tom <Tom.StDenis@amd.com>
> Cc: amd-gfx list <amd-gfx@lists.freedesktop.org>
> Subject: Re: [PATCH] drm/amd/amdgpu: Add pcie indirect support to amdgpu_mm_wreg_mmio_rlc()
>
> On Fri, Jan 7, 2022 at 7:07 AM Tom St Denis <tom.stdenis@amd.com> wrote:
> >
> > The function amdgpu_mm_wreg_mmio_rlc() is used by debugfs to write to
> > MMIO registers.  It didn't support registers beyond the BAR mapped
> > MMIO space.  This adds pcie indirect write support.
> >
> > Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
> > ---
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > index c38e0e87090b..53a04095a6db 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > @@ -552,7 +552,7 @@ void amdgpu_device_wreg(struct amdgpu_device
> > *adev,  }
> >
> >  /**
> > - * amdgpu_mm_wreg_mmio_rlc -  write register either with mmio or with
> > RLC path if in range
> > + * amdgpu_mm_wreg_mmio_rlc -  write register either with
> > + direct/indirect mmio or with RLC path if in range
> >   *
> >   * this function is invoked only the debugfs register access
> >   */
> > @@ -567,6 +567,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
> >             adev->gfx.rlc.funcs->is_rlcg_access_range) {
> >                 if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
> >                         return adev->gfx.rlc.funcs->sriov_wreg(adev,
> > reg, v, 0, 0);
> > +       } else if ((reg * 4) >= adev->rmmio_size) {
> > +               adev->pcie_wreg(adev, reg * 4, v);
>
> Actually, for older asics, we shouldn't we be using mmINDEX/mmDATA rather than the pcie indirect registers?  Or is that handled already somehow?
>
> Alex
>
> >         } else {
> >                 writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
> >         }
> > --
> > 2.32.0
> >