RE: [PATCH] amd/amdgpu/sriov enable onevf mode for ARCTURUS VF

["Zhang, Jack (Jian)" <Jack.Zhang1@amd.com>]

[-- Attachment #1.1: Type: text/plain, Size: 9242 bytes --]

Hi, Kevin,

Thank you for your code review.

  1.  I updated the patch to refine pm_enbable logic (attached).


  1.  For the following comments. “* Copy pptable bo in the vram…”, it is the original bare-metal comments. I didn’t change it.


  1.  “could you describe the function of pp_one_vf and sriov_vf ?”

  1.  amdgpu_sriov_vf  marks if it is sriov or bare-metal. While amdgpu_sriov_is_pp_one_vf is a mode of sriov- It means there is only one VF  generated by host driver.
  2.  When host driver is loaded, host administrator can determine “vf number”. If vf_num =1, host driver will let guest driver know it is under one vf mode-pp_one_vf return true. Otherwise, pp_one_vf return false. Without unloading guest driver and host driver, vf_num cannot be changed. So it is a static process.
  3.  Under  pp_one_vf mode, guest driver will do hw_init for smu, the purpose of it is to enable guest driver to “talk” with smu by sending authorized smu messages. This will help user mode app to dump info like clks, temperature, GPU usage…. Currently we don’t support guest driver to write value to smu. Only support read permission to dump smu infos.

Besides, as host driver has already initialized smu hw, some hw init steps need to skip in guest hw_init function of smu block,such as write pptable, load smc firmware.

  1.  pp_one_vf mode need smu some firmware changes to open permission for certain messages in VF.

B.R.
Jack

From: Wang, Kevin(Yang) <Kevin1.Wang@amd.com>
Sent: Monday, December 23, 2019 6:11 PM
To: Zhang, Jack (Jian) <Jack.Zhang1@amd.com>; Feng, Kenneth <Kenneth.Feng@amd.com>; Tao, Yintian <Yintian.Tao@amd.com>; amd-gfx@lists.freedesktop.org; Deng, Emily <Emily.Deng@amd.com>
Cc: Quan, Evan <Evan.Quan@amd.com>
Subject: Re: [PATCH] amd/amdgpu/sriov enable onevf mode for ARCTURUS VF


add @Quan, Evan<mailto:Evan.Quan@amd.com> to support arcturus asic.
comment inline.
________________________________
From: Zhang, Jack (Jian) <Jack.Zhang1@amd.com<mailto:Jack.Zhang1@amd.com>>
Sent: Monday, December 23, 2019 4:42 PM
To: Feng, Kenneth <Kenneth.Feng@amd.com<mailto:Kenneth.Feng@amd.com>>; Wang, Kevin(Yang) <Kevin1.Wang@amd.com<mailto:Kevin1.Wang@amd.com>>; Tao, Yintian <Yintian.Tao@amd.com<mailto:Yintian.Tao@amd.com>>; amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org> <amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org>>; Deng, Emily <Emily.Deng@amd.com<mailto:Emily.Deng@amd.com>>
Cc: Zhang, Jack (Jian) <Jack.Zhang1@amd.com<mailto:Jack.Zhang1@amd.com>>
Subject: RE: [PATCH] amd/amdgpu/sriov enable onevf mode for ARCTURUS VF



-----Original Message-----
From: Jack Zhang <Jack.Zhang1@amd.com<mailto:Jack.Zhang1@amd.com>>
Sent: Monday, December 23, 2019 4:40 PM
To: amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org>
Cc: Zhang, Jack (Jian) <Jack.Zhang1@amd.com<mailto:Jack.Zhang1@amd.com>>
Subject: [PATCH] amd/amdgpu/sriov enable onevf mode for ARCTURUS VF

Before, initialization of smu ip block would be skipped for sriov ASICs. But if there's only one VF being used, guest driver should be able to dump some HW info such as clks, temperature,etc.

To solve this, now after onevf mode is enabled, host driver will notify guest. If it's onevf mode, guest will do smu hw_init and skip some steps in normal smu hw_init flow because host driver has already done it for smu.

With this fix, guest app can talk with smu and dump hw information from smu.

Signed-off-by: Jack Zhang <Jack.Zhang1@amd.com<mailto:Jack.Zhang1@amd.com>>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c    |  3 +-
 drivers/gpu/drm/amd/amdgpu/soc15.c         |  3 +-
 drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 49 ++++++++++++++++++------------
 3 files changed, 33 insertions(+), 22 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 8469834..08130a6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -1448,7 +1448,8 @@ static int psp_np_fw_load(struct psp_context *psp)
                     || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_G
                     || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_CNTL
                     || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_GPM_MEM
-                   || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM))
+                   || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM
+                   || ucode->ucode_id == AMDGPU_UCODE_ID_SMC))
                         /*skip ucode loading in SRIOV VF */
                         continue;

diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index b53d401..a271496 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -827,8 +827,7 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
                         amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block);
                 amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block);
                 amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block);
-               if (!amdgpu_sriov_vf(adev))
-                       amdgpu_device_ip_block_add(adev, &smu_v11_0_ip_block);
+               amdgpu_device_ip_block_add(adev, &smu_v11_0_ip_block);

                 if (amdgpu_sriov_vf(adev)) {
                         if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP)) diff --git a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c b/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
index 936c682..c07fb26 100644
--- a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
@@ -531,10 +531,14 @@ bool is_support_sw_smu(struct amdgpu_device *adev)
         if (adev->asic_type == CHIP_VEGA20)
                 return (amdgpu_dpm == 2) ? true : false;
         else if (adev->asic_type >= CHIP_ARCTURUS) {
-               if (amdgpu_sriov_vf(adev))
-                       return false;
-               else
+               if (amdgpu_sriov_vf(adev)) {
+                       if(amdgpu_sriov_is_pp_one_vf(adev))
+                               return true;
+                       else
+                               return false;
+               } else {
                         return true;
+               }
         } else
                 return false;
 }
@@ -1062,20 +1066,19 @@ static int smu_smc_table_hw_init(struct smu_context *smu,
         }

         /* smu_dump_pptable(smu); */
+       if(amdgpu_sriov_vf(adev) && !amdgpu_sriov_is_pp_one_vf(adev)){
+               /*
+                * Copy pptable bo in the vram to smc with SMU MSGs such as
+                * SetDriverDramAddr and TransferTableDram2Smu.
+                */
[kevin]: this comment is not neccessary in smu module.
and could you describe the function of pp_one_vf and sriov_vf ?
it is useful to help us understand your patch.
thanks.

+               ret = smu_write_pptable(smu);
+               if (ret)
+                       return ret;

-       /*
-        * Copy pptable bo in the vram to smc with SMU MSGs such as
-        * SetDriverDramAddr and TransferTableDram2Smu.
-        */
-       ret = smu_write_pptable(smu);
-       if (ret)
-               return ret;
-
-       /* issue Run*Btc msg */
-       ret = smu_run_btc(smu);
-       if (ret)
-               return ret;
-
+               /* issue Run*Btc msg */
+               ret = smu_run_btc(smu);
+               if (ret)
+                       return ret;
         ret = smu_feature_set_allowed_mask(smu);
         if (ret)
                 return ret;
@@ -1083,7 +1086,7 @@ static int smu_smc_table_hw_init(struct smu_context *smu,
         ret = smu_system_features_control(smu, true);
         if (ret)
                 return ret;
-
+       }
         if (adev->asic_type != CHIP_ARCTURUS) {
                 ret = smu_notify_display_change(smu);
                 if (ret)
@@ -1136,8 +1139,9 @@ static int smu_smc_table_hw_init(struct smu_context *smu,
         /*
          * Set PMSTATUSLOG table bo address with SetToolsDramAddr MSG for tools.
          */
-       ret = smu_set_tool_table_location(smu);
-
+       if(amdgpu_sriov_vf(adev) && !amdgpu_sriov_is_pp_one_vf(adev)){
+               ret = smu_set_tool_table_location(smu);
+       }
         if (!smu_is_dpm_running(smu))
                 pr_info("dpm has been disabled\n");

@@ -1249,6 +1253,13 @@ static int smu_hw_init(void *handle)
                 smu_set_gfx_cgpg(&adev->smu, true);
         }

+       if (amdgpu_sriov_vf(adev)) {
+               if(amdgpu_sriov_is_pp_one_vf(adev))
+                       smu->pm_enabled = true;
+               else
+                       smu->pm_enabled = false;
+       }
+

[kevin]:
the variable of "smu->pm_enabeld" is initialize in smu_eary_init(), it is only depend on module param amdgpu_dpm.
after initialized, this variable should not be changed arbitrarily.
so i hope you can refine the above code logic.

         if (!smu->pm_enabled)
                 return 0;

--
2.7.4

[-- Attachment #1.2: Type: text/html, Size: 24992 bytes --]

[-- Attachment #2: 0001-amd-amdgpu-sriov-enable-onevf-mode-for-ARCTURUS-VF.patch --]
[-- Type: application/octet-stream, Size: 4883 bytes --]

From 1ffca7069de4d665440725e049c016ce7c466049 Mon Sep 17 00:00:00 2001
From: Jack Zhang <Jack.Zhang1@amd.com>
Date: Tue, 10 Dec 2019 10:51:01 +0800
Subject: [PATCH] amd/amdgpu/sriov enable onevf mode for ARCTURUS VF

Before, initialization of smu ip block would be skipped
for sriov ASICs. But if there's only one VF being used,
guest driver should be able to dump some HW info such as
clks, temperature,etc.

To solve this, now after onevf mode is enabled, host
driver will notify guest. If it's onevf mode, guest will
do smu hw_init and skip some steps in normal smu hw_init
flow because host driver has already done it for smu.

With this fix, guest app can talk with smu and dump hw
information from smu.

v2: refine the logic for pm_enabled.Skip hw_init by not
changing pm_enabled.

Signed-off-by: Jack Zhang <Jack.Zhang1@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c    |  3 +-
 drivers/gpu/drm/amd/amdgpu/soc15.c         |  3 +-
 drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 45 +++++++++++++++++-------------
 3 files changed, 29 insertions(+), 22 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 8469834..08130a6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -1448,7 +1448,8 @@ static int psp_np_fw_load(struct psp_context *psp)
                     || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_G
 	            || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_CNTL
 	            || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_GPM_MEM
-	            || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM))
+	            || ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM
+	            || ucode->ucode_id == AMDGPU_UCODE_ID_SMC))
 			/*skip ucode loading in SRIOV VF */
 			continue;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index b53d401..a271496 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -827,8 +827,7 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
 			amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block);
 		amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block);
 		amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block);
-		if (!amdgpu_sriov_vf(adev))
-			amdgpu_device_ip_block_add(adev, &smu_v11_0_ip_block);
+		amdgpu_device_ip_block_add(adev, &smu_v11_0_ip_block);
 
 		if (amdgpu_sriov_vf(adev)) {
 			if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP))
diff --git a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c b/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
index 936c682..42c0a6d 100644
--- a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
@@ -531,10 +531,14 @@ bool is_support_sw_smu(struct amdgpu_device *adev)
 	if (adev->asic_type == CHIP_VEGA20)
 		return (amdgpu_dpm == 2) ? true : false;
 	else if (adev->asic_type >= CHIP_ARCTURUS) {
-		if (amdgpu_sriov_vf(adev))
-			return false;
-		else
+		if (amdgpu_sriov_vf(adev)) {
+			if(amdgpu_sriov_is_pp_one_vf(adev))
+				return true;
+			else
+				return false;
+		} else {
 			return true;
+		}
 	} else
 		return false;
 }
@@ -1062,20 +1066,19 @@ static int smu_smc_table_hw_init(struct smu_context *smu,
 	}
 
 	/* smu_dump_pptable(smu); */
+	if(amdgpu_sriov_vf(adev) && !amdgpu_sriov_is_pp_one_vf(adev)){
+		/*
+		 * Copy pptable bo in the vram to smc with SMU MSGs such as
+		 * SetDriverDramAddr and TransferTableDram2Smu.
+		 */
+		ret = smu_write_pptable(smu);
+		if (ret)
+			return ret;
 
-	/*
-	 * Copy pptable bo in the vram to smc with SMU MSGs such as
-	 * SetDriverDramAddr and TransferTableDram2Smu.
-	 */
-	ret = smu_write_pptable(smu);
-	if (ret)
-		return ret;
-
-	/* issue Run*Btc msg */
-	ret = smu_run_btc(smu);
-	if (ret)
-		return ret;
-
+		/* issue Run*Btc msg */
+		ret = smu_run_btc(smu);
+		if (ret)
+			return ret;
 	ret = smu_feature_set_allowed_mask(smu);
 	if (ret)
 		return ret;
@@ -1083,7 +1086,7 @@ static int smu_smc_table_hw_init(struct smu_context *smu,
 	ret = smu_system_features_control(smu, true);
 	if (ret)
 		return ret;
-
+	}
 	if (adev->asic_type != CHIP_ARCTURUS) {
 		ret = smu_notify_display_change(smu);
 		if (ret)
@@ -1136,8 +1139,9 @@ static int smu_smc_table_hw_init(struct smu_context *smu,
 	/*
 	 * Set PMSTATUSLOG table bo address with SetToolsDramAddr MSG for tools.
 	 */
-	ret = smu_set_tool_table_location(smu);
-
+	if(amdgpu_sriov_vf(adev) && !amdgpu_sriov_is_pp_one_vf(adev)){
+		ret = smu_set_tool_table_location(smu);
+	}
 	if (!smu_is_dpm_running(smu))
 		pr_info("dpm has been disabled\n");
 
@@ -1249,6 +1253,9 @@ static int smu_hw_init(void *handle)
 		smu_set_gfx_cgpg(&adev->smu, true);
 	}
 
+	if (amdgpu_sriov_vf(adev)&& !amdgpu_sriov_is_pp_one_vf(adev))
+		return 0;
+
 	if (!smu->pm_enabled)
 		return 0;
 
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx