[PATCH] drm/amdgpu: band aid validating VM PTs

[PATCH] drm/amdgpu: band aid validating VM PTs

[Christian König]

Always validating the VM PTs takes to much time. Only always validate
the per VM BOs for now.

Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 819949418495..7c30451ba897 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1082,7 +1082,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
 					   struct amdgpu_vm_bo_base,
 					   vm_status);
 		bo_base->moved = false;
-		list_move(&bo_base->vm_status, &vm->idle);
+		list_del_init(&bo_base->vm_status);
 
 		bo = bo_base->bo->parent;
 		if (!bo)
-- 
2.14.1

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: band aid validating VM PTs

[Huang Rui]

On Tue, Jun 19, 2018 at 02:57:00PM +0200, Christian König wrote:
> Always validating the VM PTs takes to much time. Only always validate
> the per VM BOs for now.

Christian, you delete PTE BO instead of moving them into idle list. The
intention is to avoid them do evction back when do vm_validate_pt_bos,
right?

But in that function, I just see that it will walk over the idle list and
move the bo into lru list, and didn't find the evction behaviour called
explicitly. So why will it save the performance cost?

Thanks,
Ray

> 
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 819949418495..7c30451ba897 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1082,7 +1082,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
>  					   struct amdgpu_vm_bo_base,
>  					   vm_status);
>  		bo_base->moved = false;
> -		list_move(&bo_base->vm_status, &vm->idle);
> +		list_del_init(&bo_base->vm_status);
>  
>  		bo = bo_base->bo->parent;
>  		if (!bo)
> -- 
> 2.14.1
> 
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: band aid validating VM PTs

[Christian König]

Am 20.06.2018 um 13:25 schrieb Huang Rui:
> On Tue, Jun 19, 2018 at 02:57:00PM +0200, Christian König wrote:
>> Always validating the VM PTs takes to much time. Only always validate
>> the per VM BOs for now.
> Christian, you delete PTE BO instead of moving them into idle list. The
> intention is to avoid them do evction back when do vm_validate_pt_bos,
> right?

No, the intention is to avoid the extra command submission overhead with 
moving them on the LRU.

This messes the LRU for VMs up a bit, but that shouldn't matter much 
because when we need to evict PTs the performance is not relevant 
anymore anyway.

Christian.

>
> But in that function, I just see that it will walk over the idle list and
> move the bo into lru list, and didn't find the evction behaviour called
> explicitly. So why will it save the performance cost?
>
> Thanks,
> Ray
>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> index 819949418495..7c30451ba897 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> @@ -1082,7 +1082,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
>>   					   struct amdgpu_vm_bo_base,
>>   					   vm_status);
>>   		bo_base->moved = false;
>> -		list_move(&bo_base->vm_status, &vm->idle);
>> +		list_del_init(&bo_base->vm_status);
>>   
>>   		bo = bo_base->bo->parent;
>>   		if (!bo)
>> -- 
>> 2.14.1
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: band aid validating VM PTs

[Huang Rui]

On Wed, Jun 20, 2018 at 10:45:44PM +0800, Christian König wrote:
> Am 20.06.2018 um 13:25 schrieb Huang Rui:
> > On Tue, Jun 19, 2018 at 02:57:00PM +0200, Christian König wrote:
> >> Always validating the VM PTs takes to much time. Only always validate
> >> the per VM BOs for now.
> > Christian, you delete PTE BO instead of moving them into idle list. The
> > intention is to avoid them do evction back when do vm_validate_pt_bos,
> > right?
> 
> No, the intention is to avoid the extra command submission overhead with 
> moving them on the LRU.
> 

Thanks. I only see it's to decrease bo numbers in the lru list (man->lru).
Is the extra command submission overhead caused by many times of evcition
or bo validation cost (like the purpose of per-vm that to decrease the work
of bo list parser).

Thanks,
Ray

> This messes the LRU for VMs up a bit, but that shouldn't matter much 
> because when we need to evict PTs the performance is not relevant 
> anymore anyway.
> 
> Christian.
> 
> >
> > But in that function, I just see that it will walk over the idle list and
> > move the bo into lru list, and didn't find the evction behaviour called
> > explicitly. So why will it save the performance cost?
> >
> > Thanks,
> > Ray
> >
> >> Signed-off-by: Christian König <christian.koenig@amd.com>
> >> ---
> >>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> >> index 819949418495..7c30451ba897 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> >> @@ -1082,7 +1082,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
> >>   					   struct amdgpu_vm_bo_base,
> >>   					   vm_status);
> >>   		bo_base->moved = false;
> >> -		list_move(&bo_base->vm_status, &vm->idle);
> >> +		list_del_init(&bo_base->vm_status);
> >>   
> >>   		bo = bo_base->bo->parent;
> >>   		if (!bo)
> >> -- 
> >> 2.14.1
> >>
> >> _______________________________________________
> >> amd-gfx mailing list
> >> amd-gfx@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> 
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: band aid validating VM PTs

[Christian König]

Am 21.06.2018 um 15:15 schrieb Huang Rui:
> On Wed, Jun 20, 2018 at 10:45:44PM +0800, Christian König wrote:
>> Am 20.06.2018 um 13:25 schrieb Huang Rui:
>>> On Tue, Jun 19, 2018 at 02:57:00PM +0200, Christian König wrote:
>>>> Always validating the VM PTs takes to much time. Only always validate
>>>> the per VM BOs for now.
>>> Christian, you delete PTE BO instead of moving them into idle list. The
>>> intention is to avoid them do evction back when do vm_validate_pt_bos,
>>> right?
>> No, the intention is to avoid the extra command submission overhead with
>> moving them on the LRU.
>>
> Thanks. I only see it's to decrease bo numbers in the lru list (man->lru).

Well, it doesn't decrease the number of BOs on the LRU list, but rather 
the number of times we move the BOs on the LRU.

> Is the extra command submission overhead caused by many times of evcition
> or bo validation cost (like the purpose of per-vm that to decrease the work
> of bo list parser).

Neither, it was caused by the many BOs we moved to the end of the LRU.

I should probably take another look into bulk moving them on the LRU 
instead of one by one.

Christian.

>
> Thanks,
> Ray
>
>> This messes the LRU for VMs up a bit, but that shouldn't matter much
>> because when we need to evict PTs the performance is not relevant
>> anymore anyway.
>>
>> Christian.
>>
>>> But in that function, I just see that it will walk over the idle list and
>>> move the bo into lru list, and didn't find the evction behaviour called
>>> explicitly. So why will it save the performance cost?
>>>
>>> Thanks,
>>> Ray
>>>
>>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>>> ---
>>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>>> index 819949418495..7c30451ba897 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>>> @@ -1082,7 +1082,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
>>>>    					   struct amdgpu_vm_bo_base,
>>>>    					   vm_status);
>>>>    		bo_base->moved = false;
>>>> -		list_move(&bo_base->vm_status, &vm->idle);
>>>> +		list_del_init(&bo_base->vm_status);
>>>>    
>>>>    		bo = bo_base->bo->parent;
>>>>    		if (!bo)
>>>> -- 
>>>> 2.14.1
>>>>
>>>> _______________________________________________
>>>> amd-gfx mailing list
>>>> amd-gfx@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: band aid validating VM PTs

[Zhang, Jerry (Junwei)]

On 06/19/2018 08:57 PM, Christian König wrote:
> Always validating the VM PTs takes to much time. Only always validate
> the per VM BOs for now.
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>

> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 819949418495..7c30451ba897 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1082,7 +1082,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
>   					   struct amdgpu_vm_bo_base,
>   					   vm_status);
>   		bo_base->moved = false;
> -		list_move(&bo_base->vm_status, &vm->idle);
> +		list_del_init(&bo_base->vm_status);
>
>   		bo = bo_base->bo->parent;
>   		if (!bo)
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx