* [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
@ 2022-01-19 3:16 Yang Wang
2022-01-20 8:46 ` Wang, Yang(Kevin)
0 siblings, 1 reply; 7+ messages in thread
From: Yang Wang @ 2022-01-19 3:16 UTC (permalink / raw)
To: amd-gfx; +Cc: horace.chen, Frank.Min, Yang Wang
when mmio protection feature is enabled in hypervisor,
it will cause guest OS can't R/W HDP regiters,
and using cpu to update page table is not working well.
force using sdma to update page table when mmio is blocked.
Signed-off-by: Yang Wang <KevinYang.Wang@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index b23cb463b106..0f86f0b2e183 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm)
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_GFX);
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
@@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm)
/* Update VM state */
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_COMPUTE);
+
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
2022-01-19 3:16 [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked Yang Wang
@ 2022-01-20 8:46 ` Wang, Yang(Kevin)
2022-01-20 8:58 ` Christian König
0 siblings, 1 reply; 7+ messages in thread
From: Wang, Yang(Kevin) @ 2022-01-20 8:46 UTC (permalink / raw)
To: amd-gfx, Koenig, Christian, Deucher, Alexander, Liu, Monk
Cc: Chen, Horace, Min, Frank
[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]
[AMD Official Use Only]
ping...
add @Liu, Monk<mailto:Monk.Liu@amd.com> @Koenig, Christian<mailto:Christian.Koenig@amd.com> @Deucher, Alexander<mailto:Alexander.Deucher@amd.com>
Best Regards,
Kevin
________________________________
From: Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
Sent: Wednesday, January 19, 2022 11:16 AM
To: amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>
Cc: Min, Frank <Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com>; Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
Subject: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
when mmio protection feature is enabled in hypervisor,
it will cause guest OS can't R/W HDP regiters,
and using cpu to update page table is not working well.
force using sdma to update page table when mmio is blocked.
Signed-off-by: Yang Wang <KevinYang.Wang@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index b23cb463b106..0f86f0b2e183 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm)
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_GFX);
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
@@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm)
/* Update VM state */
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_COMPUTE);
+
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
--
2.25.1
[-- Attachment #2: Type: text/html, Size: 5402 bytes --]
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
2022-01-20 8:46 ` Wang, Yang(Kevin)
@ 2022-01-20 8:58 ` Christian König
2022-01-20 9:07 ` Wang, Yang(Kevin)
0 siblings, 1 reply; 7+ messages in thread
From: Christian König @ 2022-01-20 8:58 UTC (permalink / raw)
To: Wang, Yang(Kevin), amd-gfx, Deucher, Alexander, Liu, Monk
Cc: Chen, Horace, Min, Frank
[-- Attachment #1: Type: text/plain, Size: 2886 bytes --]
Well NAK.
Even when we can't R/W HDP registers we need a way to invalidate the HDP
or quite a bunch of functions won't work correctly.
Blocking CPU base page table updates only works around the symptoms, but
doesn't really solve anything.
Regards,
Christian.
Am 20.01.22 um 09:46 schrieb Wang, Yang(Kevin):
>
> [AMD Official Use Only]
>
>
> ping...
>
> add @Liu, Monk <mailto:Monk.Liu@amd.com> @Koenig, Christian
> <mailto:Christian.Koenig@amd.com> @Deucher, Alexander
> <mailto:Alexander.Deucher@amd.com>
>
> Best Regards,
> Kevin
> ------------------------------------------------------------------------
> *From:* Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
> *Sent:* Wednesday, January 19, 2022 11:16 AM
> *To:* amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>
> *Cc:* Min, Frank <Frank.Min@amd.com>; Chen, Horace
> <Horace.Chen@amd.com>; Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
> *Subject:* [PATCH] drm/amdgpu: force using sdma to update vm page
> table when mmio is blocked
> when mmio protection feature is enabled in hypervisor,
> it will cause guest OS can't R/W HDP regiters,
> and using cpu to update page table is not working well.
>
> force using sdma to update page table when mmio is blocked.
>
> Signed-off-by: Yang Wang <KevinYang.Wang@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index b23cb463b106..0f86f0b2e183 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev,
> struct amdgpu_vm *vm)
> vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
> AMDGPU_VM_USE_CPU_FOR_GFX);
>
> + if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) &&
> amdgpu_virt_mmio_blocked(adev))
> + vm->use_cpu_for_update = false;
> +
> DRM_DEBUG_DRIVER("VM update mode is %s\n",
> vm->use_cpu_for_update ? "CPU" : "SDMA");
> WARN_ONCE((vm->use_cpu_for_update &&
> @@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct amdgpu_device
> *adev, struct amdgpu_vm *vm)
> /* Update VM state */
> vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
> AMDGPU_VM_USE_CPU_FOR_COMPUTE);
> +
> + if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) &&
> amdgpu_virt_mmio_blocked(adev))
> + vm->use_cpu_for_update = false;
> +
> DRM_DEBUG_DRIVER("VM update mode is %s\n",
> vm->use_cpu_for_update ? "CPU" : "SDMA");
> WARN_ONCE((vm->use_cpu_for_update &&
> --
> 2.25.1
>
[-- Attachment #2: Type: text/html, Size: 7805 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
2022-01-20 8:58 ` Christian König
@ 2022-01-20 9:07 ` Wang, Yang(Kevin)
2022-01-20 9:11 ` Christian König
0 siblings, 1 reply; 7+ messages in thread
From: Wang, Yang(Kevin) @ 2022-01-20 9:07 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx, Deucher, Alexander, Liu, Monk
Cc: Chen, Horace, Min, Frank
[-- Attachment #1: Type: text/plain, Size: 3693 bytes --]
[AMD Official Use Only]
Hi Chris,
yes, I agree with your point.
and another way is that we can use KIQ to write HDP register to resolve HDP can't R/W issue.
but it will cause some performance drop if use KIQ to programing register.
what is your ideas?
Best Regards,
Kevin
________________________________
From: Koenig, Christian <Christian.Koenig@amd.com>
Sent: Thursday, January 20, 2022 4:58 PM
To: Wang, Yang(Kevin) <KevinYang.Wang@amd.com>; amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>; Deucher, Alexander <Alexander.Deucher@amd.com>; Liu, Monk <Monk.Liu@amd.com>
Cc: Min, Frank <Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com>
Subject: Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
Well NAK.
Even when we can't R/W HDP registers we need a way to invalidate the HDP or quite a bunch of functions won't work correctly.
Blocking CPU base page table updates only works around the symptoms, but doesn't really solve anything.
Regards,
Christian.
Am 20.01.22 um 09:46 schrieb Wang, Yang(Kevin):
[AMD Official Use Only]
ping...
add @Liu, Monk<mailto:Monk.Liu@amd.com> @Koenig, Christian<mailto:Christian.Koenig@amd.com> @Deucher, Alexander<mailto:Alexander.Deucher@amd.com>
Best Regards,
Kevin
________________________________
From: Wang, Yang(Kevin) <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>
Sent: Wednesday, January 19, 2022 11:16 AM
To: amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org> <amd-gfx@lists.freedesktop.org><mailto:amd-gfx@lists.freedesktop.org>
Cc: Min, Frank <Frank.Min@amd.com><mailto:Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com><mailto:Horace.Chen@amd.com>; Wang, Yang(Kevin) <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>
Subject: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
when mmio protection feature is enabled in hypervisor,
it will cause guest OS can't R/W HDP regiters,
and using cpu to update page table is not working well.
force using sdma to update page table when mmio is blocked.
Signed-off-by: Yang Wang <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index b23cb463b106..0f86f0b2e183 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm)
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_GFX);
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
@@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm)
/* Update VM state */
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_COMPUTE);
+
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
--
2.25.1
[-- Attachment #2: Type: text/html, Size: 8888 bytes --]
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
2022-01-20 9:07 ` Wang, Yang(Kevin)
@ 2022-01-20 9:11 ` Christian König
2022-01-20 9:39 ` Wang, Yang(Kevin)
0 siblings, 1 reply; 7+ messages in thread
From: Christian König @ 2022-01-20 9:11 UTC (permalink / raw)
To: Wang, Yang(Kevin), amd-gfx, Deucher, Alexander, Liu, Monk
Cc: Chen, Horace, Min, Frank
[-- Attachment #1: Type: text/plain, Size: 4583 bytes --]
Hi Kevin,
well at least the HDP flush function has to work correctly or otherwise
the driver won't work correctly.
If the registers are not accessible any more we need to find a proper
workaround for this.
One possibility would be to use the KIQ, another is a dummy write/read
to make sure the HDP is flushed (check the hardware docs).
The third option would be to question if blocking the HDP registers is
really a good idea.
The solution is up to you, but a workaround like proposed below doesn't
really help in any way.
Regards,
Christian.
Am 20.01.22 um 10:07 schrieb Wang, Yang(Kevin):
>
> [AMD Official Use Only]
>
>
> Hi Chris,
>
> yes, I agree with your point.
> and another way is that we can use KIQ to write HDP register to
> resolve HDP can't R/W issue.
> but it will cause some performance drop if use KIQ to programing register.
>
> what is your ideas?
>
> Best Regards,
> Kevin
> ------------------------------------------------------------------------
> *From:* Koenig, Christian <Christian.Koenig@amd.com>
> *Sent:* Thursday, January 20, 2022 4:58 PM
> *To:* Wang, Yang(Kevin) <KevinYang.Wang@amd.com>;
> amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>;
> Deucher, Alexander <Alexander.Deucher@amd.com>; Liu, Monk
> <Monk.Liu@amd.com>
> *Cc:* Min, Frank <Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com>
> *Subject:* Re: [PATCH] drm/amdgpu: force using sdma to update vm page
> table when mmio is blocked
> Well NAK.
>
> Even when we can't R/W HDP registers we need a way to invalidate the
> HDP or quite a bunch of functions won't work correctly.
>
> Blocking CPU base page table updates only works around the symptoms,
> but doesn't really solve anything.
>
> Regards,
> Christian.
>
> Am 20.01.22 um 09:46 schrieb Wang, Yang(Kevin):
>>
>> [AMD Official Use Only]
>>
>>
>> ping...
>>
>> add @Liu, Monk <mailto:Monk.Liu@amd.com> @Koenig, Christian
>> <mailto:Christian.Koenig@amd.com> @Deucher, Alexander
>> <mailto:Alexander.Deucher@amd.com>
>>
>> Best Regards,
>> Kevin
>> ------------------------------------------------------------------------
>> *From:* Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
>> <mailto:KevinYang.Wang@amd.com>
>> *Sent:* Wednesday, January 19, 2022 11:16 AM
>> *To:* amd-gfx@lists.freedesktop.org
>> <mailto:amd-gfx@lists.freedesktop.org>
>> <amd-gfx@lists.freedesktop.org> <mailto:amd-gfx@lists.freedesktop.org>
>> *Cc:* Min, Frank <Frank.Min@amd.com> <mailto:Frank.Min@amd.com>;
>> Chen, Horace <Horace.Chen@amd.com> <mailto:Horace.Chen@amd.com>;
>> Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
>> <mailto:KevinYang.Wang@amd.com>
>> *Subject:* [PATCH] drm/amdgpu: force using sdma to update vm page
>> table when mmio is blocked
>> when mmio protection feature is enabled in hypervisor,
>> it will cause guest OS can't R/W HDP regiters,
>> and using cpu to update page table is not working well.
>>
>> force using sdma to update page table when mmio is blocked.
>>
>> Signed-off-by: Yang Wang <KevinYang.Wang@amd.com>
>> <mailto:KevinYang.Wang@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
>> 1 file changed, 7 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> index b23cb463b106..0f86f0b2e183 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> @@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev,
>> struct amdgpu_vm *vm)
>> vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
>> AMDGPU_VM_USE_CPU_FOR_GFX);
>>
>> + if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) &&
>> amdgpu_virt_mmio_blocked(adev))
>> + vm->use_cpu_for_update = false;
>> +
>> DRM_DEBUG_DRIVER("VM update mode is %s\n",
>> vm->use_cpu_for_update ? "CPU" : "SDMA");
>> WARN_ONCE((vm->use_cpu_for_update &&
>> @@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct
>> amdgpu_device *adev, struct amdgpu_vm *vm)
>> /* Update VM state */
>> vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
>> AMDGPU_VM_USE_CPU_FOR_COMPUTE);
>> +
>> + if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) &&
>> amdgpu_virt_mmio_blocked(adev))
>> + vm->use_cpu_for_update = false;
>> +
>> DRM_DEBUG_DRIVER("VM update mode is %s\n",
>> vm->use_cpu_for_update ? "CPU" : "SDMA");
>> WARN_ONCE((vm->use_cpu_for_update &&
>> --
>> 2.25.1
>>
>
[-- Attachment #2: Type: text/html, Size: 13449 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
2022-01-20 9:11 ` Christian König
@ 2022-01-20 9:39 ` Wang, Yang(Kevin)
2022-01-25 1:48 ` Felix Kuehling
0 siblings, 1 reply; 7+ messages in thread
From: Wang, Yang(Kevin) @ 2022-01-20 9:39 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx, Deucher, Alexander, Liu, Monk
Cc: Chen, Horace, Min, Frank
[-- Attachment #1: Type: text/plain, Size: 5163 bytes --]
[AMD Official Use Only]
Thanks @Koenig, Christian<mailto:Christian.Koenig@amd.com>.
Best Regards,
Kevin
________________________________
From: Koenig, Christian <Christian.Koenig@amd.com>
Sent: Thursday, January 20, 2022 5:11 PM
To: Wang, Yang(Kevin) <KevinYang.Wang@amd.com>; amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>; Deucher, Alexander <Alexander.Deucher@amd.com>; Liu, Monk <Monk.Liu@amd.com>
Cc: Min, Frank <Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com>
Subject: Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
Hi Kevin,
well at least the HDP flush function has to work correctly or otherwise the driver won't work correctly.
If the registers are not accessible any more we need to find a proper workaround for this.
One possibility would be to use the KIQ, another is a dummy write/read to make sure the HDP is flushed (check the hardware docs).
The third option would be to question if blocking the HDP registers is really a good idea.
The solution is up to you, but a workaround like proposed below doesn't really help in any way.
Regards,
Christian.
Am 20.01.22 um 10:07 schrieb Wang, Yang(Kevin):
[AMD Official Use Only]
Hi Chris,
yes, I agree with your point.
and another way is that we can use KIQ to write HDP register to resolve HDP can't R/W issue.
but it will cause some performance drop if use KIQ to programing register.
what is your ideas?
Best Regards,
Kevin
________________________________
From: Koenig, Christian <Christian.Koenig@amd.com><mailto:Christian.Koenig@amd.com>
Sent: Thursday, January 20, 2022 4:58 PM
To: Wang, Yang(Kevin) <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>; amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org> <amd-gfx@lists.freedesktop.org><mailto:amd-gfx@lists.freedesktop.org>; Deucher, Alexander <Alexander.Deucher@amd.com><mailto:Alexander.Deucher@amd.com>; Liu, Monk <Monk.Liu@amd.com><mailto:Monk.Liu@amd.com>
Cc: Min, Frank <Frank.Min@amd.com><mailto:Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com><mailto:Horace.Chen@amd.com>
Subject: Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
Well NAK.
Even when we can't R/W HDP registers we need a way to invalidate the HDP or quite a bunch of functions won't work correctly.
Blocking CPU base page table updates only works around the symptoms, but doesn't really solve anything.
Regards,
Christian.
Am 20.01.22 um 09:46 schrieb Wang, Yang(Kevin):
[AMD Official Use Only]
ping...
add @Liu, Monk<mailto:Monk.Liu@amd.com> @Koenig, Christian<mailto:Christian.Koenig@amd.com> @Deucher, Alexander<mailto:Alexander.Deucher@amd.com>
Best Regards,
Kevin
________________________________
From: Wang, Yang(Kevin) <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>
Sent: Wednesday, January 19, 2022 11:16 AM
To: amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org> <amd-gfx@lists.freedesktop.org><mailto:amd-gfx@lists.freedesktop.org>
Cc: Min, Frank <Frank.Min@amd.com><mailto:Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com><mailto:Horace.Chen@amd.com>; Wang, Yang(Kevin) <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>
Subject: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
when mmio protection feature is enabled in hypervisor,
it will cause guest OS can't R/W HDP regiters,
and using cpu to update page table is not working well.
force using sdma to update page table when mmio is blocked.
Signed-off-by: Yang Wang <KevinYang.Wang@amd.com><mailto:KevinYang.Wang@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index b23cb463b106..0f86f0b2e183 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm)
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_GFX);
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
@@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm)
/* Update VM state */
vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
AMDGPU_VM_USE_CPU_FOR_COMPUTE);
+
+ if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) && amdgpu_virt_mmio_blocked(adev))
+ vm->use_cpu_for_update = false;
+
DRM_DEBUG_DRIVER("VM update mode is %s\n",
vm->use_cpu_for_update ? "CPU" : "SDMA");
WARN_ONCE((vm->use_cpu_for_update &&
--
2.25.1
[-- Attachment #2: Type: text/html, Size: 11828 bytes --]
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked
2022-01-20 9:39 ` Wang, Yang(Kevin)
@ 2022-01-25 1:48 ` Felix Kuehling
0 siblings, 0 replies; 7+ messages in thread
From: Felix Kuehling @ 2022-01-25 1:48 UTC (permalink / raw)
To: Wang, Yang(Kevin),
Koenig, Christian, amd-gfx, Deucher, Alexander, Liu, Monk
Cc: Chen, Horace, Min, Frank
I made a related fix for HDP flushing under SRIOV a while ago, and that
was confirmed to be working at the time. What changed since then? Is
this on different HW or with a different BIOS? Or are you missing this
patch? For reference:
commit 6295c4064d8d18c4480636076dbd5afb48885c02
Author: Felix Kuehling <Felix.Kuehling@amd.com>
Date: Thu Nov 4 16:15:43 2021 -0400
drm/amdgpu: Fix MMIO HDP flush on SRIOV
Disable HDP register remapping on SRIOV and set rmmio_remap.reg_offset
to the fixed address of the VF register for hdp_v*_flush_hdp.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Tested-by: Bokun Zhang <bokun.zhang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Regards,
Felix
Am 2022-01-20 um 04:39 schrieb Wang, Yang(Kevin):
>
> [AMD Official Use Only]
>
>
> [AMD Official Use Only]
>
>
> Thanks @Koenig, Christian <mailto:Christian.Koenig@amd.com>.
>
> Best Regards,
> Kevin
> ------------------------------------------------------------------------
> *From:* Koenig, Christian <Christian.Koenig@amd.com>
> *Sent:* Thursday, January 20, 2022 5:11 PM
> *To:* Wang, Yang(Kevin) <KevinYang.Wang@amd.com>;
> amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>;
> Deucher, Alexander <Alexander.Deucher@amd.com>; Liu, Monk
> <Monk.Liu@amd.com>
> *Cc:* Min, Frank <Frank.Min@amd.com>; Chen, Horace <Horace.Chen@amd.com>
> *Subject:* Re: [PATCH] drm/amdgpu: force using sdma to update vm page
> table when mmio is blocked
> Hi Kevin,
>
> well at least the HDP flush function has to work correctly or
> otherwise the driver won't work correctly.
>
> If the registers are not accessible any more we need to find a proper
> workaround for this.
>
> One possibility would be to use the KIQ, another is a dummy write/read
> to make sure the HDP is flushed (check the hardware docs).
>
> The third option would be to question if blocking the HDP registers is
> really a good idea.
>
> The solution is up to you, but a workaround like proposed below
> doesn't really help in any way.
>
> Regards,
> Christian.
>
> Am 20.01.22 um 10:07 schrieb Wang, Yang(Kevin):
>>
>> [AMD Official Use Only]
>>
>>
>> Hi Chris,
>>
>> yes, I agree with your point.
>> and another way is that we can use KIQ to write HDP register to
>> resolve HDP can't R/W issue.
>> but it will cause some performance drop if use KIQ to programing
>> register.
>>
>> what is your ideas?
>>
>> Best Regards,
>> Kevin
>> ------------------------------------------------------------------------
>> *From:* Koenig, Christian <Christian.Koenig@amd.com>
>> <mailto:Christian.Koenig@amd.com>
>> *Sent:* Thursday, January 20, 2022 4:58 PM
>> *To:* Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
>> <mailto:KevinYang.Wang@amd.com>; amd-gfx@lists.freedesktop.org
>> <mailto:amd-gfx@lists.freedesktop.org>
>> <amd-gfx@lists.freedesktop.org>
>> <mailto:amd-gfx@lists.freedesktop.org>; Deucher, Alexander
>> <Alexander.Deucher@amd.com> <mailto:Alexander.Deucher@amd.com>; Liu,
>> Monk <Monk.Liu@amd.com> <mailto:Monk.Liu@amd.com>
>> *Cc:* Min, Frank <Frank.Min@amd.com> <mailto:Frank.Min@amd.com>;
>> Chen, Horace <Horace.Chen@amd.com> <mailto:Horace.Chen@amd.com>
>> *Subject:* Re: [PATCH] drm/amdgpu: force using sdma to update vm page
>> table when mmio is blocked
>> Well NAK.
>>
>> Even when we can't R/W HDP registers we need a way to invalidate the
>> HDP or quite a bunch of functions won't work correctly.
>>
>> Blocking CPU base page table updates only works around the symptoms,
>> but doesn't really solve anything.
>>
>> Regards,
>> Christian.
>>
>> Am 20.01.22 um 09:46 schrieb Wang, Yang(Kevin):
>>>
>>> [AMD Official Use Only]
>>>
>>>
>>> ping...
>>>
>>> add @Liu, Monk <mailto:Monk.Liu@amd.com> @Koenig, Christian
>>> <mailto:Christian.Koenig@amd.com> @Deucher, Alexander
>>> <mailto:Alexander.Deucher@amd.com>
>>>
>>> Best Regards,
>>> Kevin
>>> ------------------------------------------------------------------------
>>> *From:* Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
>>> <mailto:KevinYang.Wang@amd.com>
>>> *Sent:* Wednesday, January 19, 2022 11:16 AM
>>> *To:* amd-gfx@lists.freedesktop.org
>>> <mailto:amd-gfx@lists.freedesktop.org>
>>> <amd-gfx@lists.freedesktop.org> <mailto:amd-gfx@lists.freedesktop.org>
>>> *Cc:* Min, Frank <Frank.Min@amd.com> <mailto:Frank.Min@amd.com>;
>>> Chen, Horace <Horace.Chen@amd.com> <mailto:Horace.Chen@amd.com>;
>>> Wang, Yang(Kevin) <KevinYang.Wang@amd.com>
>>> <mailto:KevinYang.Wang@amd.com>
>>> *Subject:* [PATCH] drm/amdgpu: force using sdma to update vm page
>>> table when mmio is blocked
>>> when mmio protection feature is enabled in hypervisor,
>>> it will cause guest OS can't R/W HDP regiters,
>>> and using cpu to update page table is not working well.
>>>
>>> force using sdma to update page table when mmio is blocked.
>>>
>>> Signed-off-by: Yang Wang <KevinYang.Wang@amd.com>
>>> <mailto:KevinYang.Wang@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +++++++
>>> 1 file changed, 7 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> index b23cb463b106..0f86f0b2e183 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> @@ -2959,6 +2959,9 @@ int amdgpu_vm_init(struct amdgpu_device *adev,
>>> struct amdgpu_vm *vm)
>>> vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
>>> AMDGPU_VM_USE_CPU_FOR_GFX);
>>>
>>> + if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) &&
>>> amdgpu_virt_mmio_blocked(adev))
>>> + vm->use_cpu_for_update = false;
>>> +
>>> DRM_DEBUG_DRIVER("VM update mode is %s\n",
>>> vm->use_cpu_for_update ? "CPU" : "SDMA");
>>> WARN_ONCE((vm->use_cpu_for_update &&
>>> @@ -3094,6 +3097,10 @@ int amdgpu_vm_make_compute(struct
>>> amdgpu_device *adev, struct amdgpu_vm *vm)
>>> /* Update VM state */
>>> vm->use_cpu_for_update = !!(adev->vm_manager.vm_update_mode &
>>> AMDGPU_VM_USE_CPU_FOR_COMPUTE);
>>> +
>>> + if (vm->use_cpu_for_update && amdgpu_sriov_vf(adev) &&
>>> amdgpu_virt_mmio_blocked(adev))
>>> + vm->use_cpu_for_update = false;
>>> +
>>> DRM_DEBUG_DRIVER("VM update mode is %s\n",
>>> vm->use_cpu_for_update ? "CPU" : "SDMA");
>>> WARN_ONCE((vm->use_cpu_for_update &&
>>> --
>>> 2.25.1
>>>
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-01-25 1:48 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-19 3:16 [PATCH] drm/amdgpu: force using sdma to update vm page table when mmio is blocked Yang Wang
2022-01-20 8:46 ` Wang, Yang(Kevin)
2022-01-20 8:58 ` Christian König
2022-01-20 9:07 ` Wang, Yang(Kevin)
2022-01-20 9:11 ` Christian König
2022-01-20 9:39 ` Wang, Yang(Kevin)
2022-01-25 1:48 ` Felix Kuehling
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.