[PATCH] drm/amdgpu: use vm_update_mode=0 as default in sriov for gfx10.3 onwards

[PATCH] drm/amdgpu: use vm_update_mode=0 as default in sriov for gfx10.3 onwards

[Danijel Slivka]

Apply this rule to all newer asics in sriov case.
For asic with VF MMIO access protection avoid using CPU for VM table updates.
CPU pagetable updates have issues with HDP flush as VF MMIO access protection
blocks write to BIF_BX_DEV0_EPF0_VF0_HDP_MEM_COHERENCY_FLUSH_CNTL register
during sriov runtime.

Signed-off-by: Danijel Slivka <danijel.slivka@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 7 +++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c   | 6 ------
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 12dc71a6b5db..59ee902a1eaa 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4072,6 +4072,13 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 	/* Enable TMZ based on IP_VERSION */
 	amdgpu_gmc_tmz_set(adev);
 
+	if (amdgpu_sriov_vf(adev) &&
+		(amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(10, 3, 0)))
+		/* VF MMIO access (except mailbox range) from CPU
+		 * will be blocked during sriov runtime
+		 */
+		adev->virt.caps |= AMDGPU_VF_MMIO_ACCESS_PROTECT;
+
 	amdgpu_gmc_noretry_set(adev);
 	/* Need to get xgmi info early to decide the reset behavior*/
 	if (adev->gmc.xgmi.supported) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index aed60aaf1a55..6f01de220c44 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -724,12 +724,6 @@ void amdgpu_detect_virtualization(struct amdgpu_device *adev)
 			adev->virt.caps |= AMDGPU_PASSTHROUGH_MODE;
 	}
 
-	if (amdgpu_sriov_vf(adev) && adev->asic_type == CHIP_SIENNA_CICHLID)
-		/* VF MMIO access (except mailbox range) from CPU
-		 * will be blocked during sriov runtime
-		 */
-		adev->virt.caps |= AMDGPU_VF_MMIO_ACCESS_PROTECT;
-
 	/* we have the ability to check now */
 	if (amdgpu_sriov_vf(adev)) {
 		switch (adev->asic_type) {
-- 
2.25.1

Re: [PATCH] drm/amdgpu: use vm_update_mode=0 as default in sriov for gfx10.3 onwards

[Felix Kuehling]

On 2024-03-28 13:59, Danijel Slivka wrote:
> Apply this rule to all newer asics in sriov case.
> For asic with VF MMIO access protection avoid using CPU for VM table updates.
> CPU pagetable updates have issues with HDP flush as VF MMIO access protection
> blocks write to BIF_BX_DEV0_EPF0_VF0_HDP_MEM_COHERENCY_FLUSH_CNTL register
> during sriov runtime.

Please mention that you moved the check to amdgpu_device_init to ensure 
that it runs after amdgpu_device_ip_early_init where the IP versions are 
discovered.


>
> Signed-off-by: Danijel Slivka <danijel.slivka@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 7 +++++++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c   | 6 ------
>   2 files changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 12dc71a6b5db..59ee902a1eaa 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -4072,6 +4072,13 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>   	/* Enable TMZ based on IP_VERSION */
>   	amdgpu_gmc_tmz_set(adev);
>   
> +	if (amdgpu_sriov_vf(adev) &&
> +		(amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(10, 3, 0)))

Please fix the indentation. The second line should be aligned with the 
open parenthesis from the previous line. You could also remove the extra 
parentheses around the comparison. They're not needed, and IMO they make 
the code less readable.

With that fixed, the patch is

Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>


> +		/* VF MMIO access (except mailbox range) from CPU
> +		 * will be blocked during sriov runtime
> +		 */
> +		adev->virt.caps |= AMDGPU_VF_MMIO_ACCESS_PROTECT;
> +
>   	amdgpu_gmc_noretry_set(adev);
>   	/* Need to get xgmi info early to decide the reset behavior*/
>   	if (adev->gmc.xgmi.supported) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
> index aed60aaf1a55..6f01de220c44 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
> @@ -724,12 +724,6 @@ void amdgpu_detect_virtualization(struct amdgpu_device *adev)
>   			adev->virt.caps |= AMDGPU_PASSTHROUGH_MODE;
>   	}
>   
> -	if (amdgpu_sriov_vf(adev) && adev->asic_type == CHIP_SIENNA_CICHLID)
> -		/* VF MMIO access (except mailbox range) from CPU
> -		 * will be blocked during sriov runtime
> -		 */
> -		adev->virt.caps |= AMDGPU_VF_MMIO_ACCESS_PROTECT;
> -
>   	/* we have the ability to check now */
>   	if (amdgpu_sriov_vf(adev)) {
>   		switch (adev->asic_type) {