[PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[Guchun Chen]

Fall back to MMIO to read registers as rlcg read is not
available for gfx v9 in SRIOV configration. Otherwise,
gmc_v9_0_flush_gpu_tlb will always complain timeout and
finally breaks driver load.

Fixes: 0dc4a7e75581("drm/amdgpu: switch to get_rlcg_reg_access_flag for gfx9")
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..a3274fa1c7e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -37,6 +37,16 @@
 		vf2pf_info->ucode_info[ucode].version = ver; \
 	} while (0)
 
+static bool amdgpu_virt_is_rlcg_read_supported(struct amdgpu_device *adev)
+{
+	/* rlcg read is not support in SRIOV with gfx v9 */
+	if ((adev->ip_versions[MP0_HWIP][0] == IP_VERSION(9, 0, 0)) ||
+		(adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1)))
+		return false;
+
+	return true;
+}
+
 bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev)
 {
 	/* By now all MMIO pages except mailbox are blocked */
@@ -957,7 +967,8 @@ u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
 	u32 rlcg_flag;
 
 	if (!amdgpu_sriov_runtime(adev) &&
-	    amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
+		amdgpu_virt_is_rlcg_read_supported(adev) &&
+		amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
 		return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag);
 
 	if (acc_flags & AMDGPU_REGS_NO_KIQ)
-- 
2.17.1

RE: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[Zhang, Hawking]

[AMD Official Use Only]

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>

Regards,
Hawking
-----Original Message-----
From: Chen, Guchun <Guchun.Chen@amd.com> 
Sent: Thursday, February 10, 2022 14:40
To: amd-gfx@lists.freedesktop.org; Zhang, Hawking <Hawking.Zhang@amd.com>; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Cc: Chen, Guchun <Guchun.Chen@amd.com>
Subject: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

Fall back to MMIO to read registers as rlcg read is not available for gfx v9 in SRIOV configration. Otherwise, gmc_v9_0_flush_gpu_tlb will always complain timeout and finally breaks driver load.

Fixes: 0dc4a7e75581("drm/amdgpu: switch to get_rlcg_reg_access_flag for gfx9")
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..a3274fa1c7e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -37,6 +37,16 @@
 		vf2pf_info->ucode_info[ucode].version = ver; \
 	} while (0)
 
+static bool amdgpu_virt_is_rlcg_read_supported(struct amdgpu_device 
+*adev) {
+	/* rlcg read is not support in SRIOV with gfx v9 */
+	if ((adev->ip_versions[MP0_HWIP][0] == IP_VERSION(9, 0, 0)) ||
+		(adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1)))
+		return false;
+
+	return true;
+}
+
 bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev)  {
 	/* By now all MMIO pages except mailbox are blocked */ @@ -957,7 +967,8 @@ u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
 	u32 rlcg_flag;
 
 	if (!amdgpu_sriov_runtime(adev) &&
-	    amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
+		amdgpu_virt_is_rlcg_read_supported(adev) &&
+		amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, 
+&rlcg_flag))
 		return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag);
 
 	if (acc_flags & AMDGPU_REGS_NO_KIQ)
--
2.17.1

RE: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[Skvortsov, Victor]

[AMD Official Use Only]

Hi Guchun,

RLCG read is available on Aldebaran if amdgpu_sriov_reg_indirect_gc() flag is set. Instead of adding a new function, I think we should simply add a check inside amdgpu_virt_get_rlcg_reg_access_flag():


diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..1ee600e90312 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -836,7 +836,7 @@ static bool amdgpu_virt_get_rlcg_reg_access_flag(struct amdgpu_device *adev,
                /* only in new version, AMDGPU_REGS_NO_KIQ and
                 * AMDGPU_REGS_RLC are enabled simultaneously */
                } else if ((acc_flags & AMDGPU_REGS_RLC) &&
-                          !(acc_flags & AMDGPU_REGS_NO_KIQ)) {
+                          !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) {
                        *rlcg_flag = AMDGPU_RLCG_GC_WRITE_LEGACY;
                        ret = true;
                }

Thanks,
Victor

-----Original Message-----
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Zhang, Hawking
Sent: Thursday, February 10, 2022 5:02 AM
To: Chen, Guchun <Guchun.Chen@amd.com>; amd-gfx@lists.freedesktop.org; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Subject: RE: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[CAUTION: External Email]

[AMD Official Use Only]

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>

Regards,
Hawking
-----Original Message-----
From: Chen, Guchun <Guchun.Chen@amd.com>
Sent: Thursday, February 10, 2022 14:40
To: amd-gfx@lists.freedesktop.org; Zhang, Hawking <Hawking.Zhang@amd.com>; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Cc: Chen, Guchun <Guchun.Chen@amd.com>
Subject: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

Fall back to MMIO to read registers as rlcg read is not available for gfx v9 in SRIOV configration. Otherwise, gmc_v9_0_flush_gpu_tlb will always complain timeout and finally breaks driver load.

Fixes: 0dc4a7e75581("drm/amdgpu: switch to get_rlcg_reg_access_flag for gfx9")
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..a3274fa1c7e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -37,6 +37,16 @@
                vf2pf_info->ucode_info[ucode].version = ver; \
        } while (0)

+static bool amdgpu_virt_is_rlcg_read_supported(struct amdgpu_device
+*adev) {
+       /* rlcg read is not support in SRIOV with gfx v9 */
+       if ((adev->ip_versions[MP0_HWIP][0] == IP_VERSION(9, 0, 0)) ||
+               (adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1)))
+               return false;
+
+       return true;
+}
+
 bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev)  {
        /* By now all MMIO pages except mailbox are blocked */ @@ -957,7 +967,8 @@ u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
        u32 rlcg_flag;

        if (!amdgpu_sriov_runtime(adev) &&
-           amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
+               amdgpu_virt_is_rlcg_read_supported(adev) &&
+               amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false,
+&rlcg_flag))
                return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag);

        if (acc_flags & AMDGPU_REGS_NO_KIQ)
--
2.17.1

Re: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[Deucher, Alexander]

[-- Attachment #1: Type: text/plain, Size: 2492 bytes --]

[AMD Official Use Only]

For better future proofing maybe adjust the check to look for pre-gfx10 rather than checking for specific IP versions?  E.g.,

adev->ip_versions[MP0_HWIP][0] < IP_VERSION(10, 0, 0)
________________________________
From: Chen, Guchun <Guchun.Chen@amd.com>
Sent: Thursday, February 10, 2022 1:40 AM
To: amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>; Zhang, Hawking <Hawking.Zhang@amd.com>; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Cc: Chen, Guchun <Guchun.Chen@amd.com>
Subject: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

Fall back to MMIO to read registers as rlcg read is not
available for gfx v9 in SRIOV configration. Otherwise,
gmc_v9_0_flush_gpu_tlb will always complain timeout and
finally breaks driver load.

Fixes: 0dc4a7e75581("drm/amdgpu: switch to get_rlcg_reg_access_flag for gfx9")
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..a3274fa1c7e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -37,6 +37,16 @@
                 vf2pf_info->ucode_info[ucode].version = ver; \
         } while (0)

+static bool amdgpu_virt_is_rlcg_read_supported(struct amdgpu_device *adev)
+{
+       /* rlcg read is not support in SRIOV with gfx v9 */
+       if ((adev->ip_versions[MP0_HWIP][0] == IP_VERSION(9, 0, 0)) ||
+               (adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1)))
+               return false;
+
+       return true;
+}
+
 bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev)
 {
         /* By now all MMIO pages except mailbox are blocked */
@@ -957,7 +967,8 @@ u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
         u32 rlcg_flag;

         if (!amdgpu_sriov_runtime(adev) &&
-           amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
+               amdgpu_virt_is_rlcg_read_supported(adev) &&
+               amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
                 return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag);

         if (acc_flags & AMDGPU_REGS_NO_KIQ)
--
2.17.1


[-- Attachment #2: Type: text/html, Size: 4708 bytes --]

RE: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[Chen, Guchun]

[Public]

Hi Victor,

I thought about that before sending the patch, as there is indeed a hack in gfx9 old code. So my idea is not to impact asics with gfx10, so add the check in the caller.

Anyway, I will double verify your suggestion on ASICs with gfx10. If it's fine, I will submit a v2 patch.

Regards,
Guchun

-----Original Message-----
From: Skvortsov, Victor <Victor.Skvortsov@amd.com> 
Sent: Thursday, February 10, 2022 6:50 PM
To: Zhang, Hawking <Hawking.Zhang@amd.com>; Chen, Guchun <Guchun.Chen@amd.com>; amd-gfx@lists.freedesktop.org; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Subject: RE: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[AMD Official Use Only]

Hi Guchun,

RLCG read is available on Aldebaran if amdgpu_sriov_reg_indirect_gc() flag is set. Instead of adding a new function, I think we should simply add a check inside amdgpu_virt_get_rlcg_reg_access_flag():


diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..1ee600e90312 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -836,7 +836,7 @@ static bool amdgpu_virt_get_rlcg_reg_access_flag(struct amdgpu_device *adev,
                /* only in new version, AMDGPU_REGS_NO_KIQ and
                 * AMDGPU_REGS_RLC are enabled simultaneously */
                } else if ((acc_flags & AMDGPU_REGS_RLC) &&
-                          !(acc_flags & AMDGPU_REGS_NO_KIQ)) {
+                          !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) {
                        *rlcg_flag = AMDGPU_RLCG_GC_WRITE_LEGACY;
                        ret = true;
                }

Thanks,
Victor

-----Original Message-----
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Zhang, Hawking
Sent: Thursday, February 10, 2022 5:02 AM
To: Chen, Guchun <Guchun.Chen@amd.com>; amd-gfx@lists.freedesktop.org; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Subject: RE: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

[CAUTION: External Email]

[AMD Official Use Only]

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>

Regards,
Hawking
-----Original Message-----
From: Chen, Guchun <Guchun.Chen@amd.com>
Sent: Thursday, February 10, 2022 14:40
To: amd-gfx@lists.freedesktop.org; Zhang, Hawking <Hawking.Zhang@amd.com>; Zhou, Peng Ju <PengJu.Zhou@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>
Cc: Chen, Guchun <Guchun.Chen@amd.com>
Subject: [PATCH] drm/amdgpu: no rlcg read access in SRIOV case for gfx v9

Fall back to MMIO to read registers as rlcg read is not available for gfx v9 in SRIOV configration. Otherwise, gmc_v9_0_flush_gpu_tlb will always complain timeout and finally breaks driver load.

Fixes: 0dc4a7e75581("drm/amdgpu: switch to get_rlcg_reg_access_flag for gfx9")
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index e1288901beb6..a3274fa1c7e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -37,6 +37,16 @@
                vf2pf_info->ucode_info[ucode].version = ver; \
        } while (0)

+static bool amdgpu_virt_is_rlcg_read_supported(struct amdgpu_device
+*adev) {
+       /* rlcg read is not support in SRIOV with gfx v9 */
+       if ((adev->ip_versions[MP0_HWIP][0] == IP_VERSION(9, 0, 0)) ||
+               (adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1)))
+               return false;
+
+       return true;
+}
+
 bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev)  {
        /* By now all MMIO pages except mailbox are blocked */ @@ -957,7 +967,8 @@ u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
        u32 rlcg_flag;

        if (!amdgpu_sriov_runtime(adev) &&
-           amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
+               amdgpu_virt_is_rlcg_read_supported(adev) &&
+               amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false,
+&rlcg_flag))
                return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag);

        if (acc_flags & AMDGPU_REGS_NO_KIQ)
--
2.17.1