Re: [PATCH] drm/amdkfd: only allow heavy-weight TLB flush on some ASICs for SVM too

[Paul Menzel <pmenzel@molgen.mpg.de>]

Dear Lang,


Am 15.04.22 um 05:20 schrieb Lang Yu:
> On 04/14/ , Paul Menzel wrote:

>> Am 14.04.22 um 10:19 schrieb Lang Yu:
>>> The idea is from commit a50fe7078035 ("drm/amdkfd: Only apply heavy-weight
>>> TLB flush on Aldebaran") and commit f61c40c0757a ("drm/amdkfd: enable
>>> heavy-weight TLB flush on Arcturus"). Otherwise, we will run into problems
>>> on some ASICs when running SVM applications.
>>
>> Please list the ASICs, you know of having problems, and even add how to
>> reproduce this.
> 
> Actually, this is ported from previous commits. You can find more details
> from the commits I mentioned. At the moment the ASICs except Aldebaran
> and Arcturus probably have the problem.

I think, it’s always good to make it as easy as possible for reviewers 
and, later, people reading a commit, and include the necessary 
information directly in the commit message. It’d be great if you amended 
the commit message.

> And running a SVM application could reproduce the issue.

Thanks. How will it fail though?

(Also, a small implementation note would be nice to have. Maybe: Move 
the helper function into the header `kfd_priv.h`, and use in 
`svm_range_unmap_from_gpus()`.)


Kind regards,

Paul


>>> Signed-off-by: Lang Yu <Lang.Yu@amd.com>
>>> ---
>>>    drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 8 --------
>>>    drivers/gpu/drm/amd/amdkfd/kfd_priv.h    | 8 ++++++++
>>>    drivers/gpu/drm/amd/amdkfd/kfd_svm.c     | 4 +++-
>>>    3 files changed, 11 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> index 91f82a9ccdaf..459f59e3d0ed 100644
>>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> @@ -1128,14 +1128,6 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep,
>>>    	return ret;
>>>    }
>>> -static bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev)
>>> -{
>>> -	return KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2) ||
>>> -		(KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 1) &&
>>> -		dev->adev->sdma.instance[0].fw_version >= 18) ||
>>> -		KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 0);
>>> -}
>>> -
>>>    static int kfd_ioctl_map_memory_to_gpu(struct file *filep,
>>>    					struct kfd_process *p, void *data)
>>>    {
>>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>>> index 8a43def1f638..aff6f598ff2c 100644
>>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>>> @@ -1328,6 +1328,14 @@ void kfd_signal_poison_consumed_event(struct kfd_dev *dev, u32 pasid);
>>>    void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type);
>>> +static inline bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev)
>>> +{
>>> +	return KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2) ||
>>> +	       (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 1) &&
>>> +	       dev->adev->sdma.instance[0].fw_version >= 18) ||
>>> +	       KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 0);
>>> +}
>>> +
>>>    bool kfd_is_locked(void);
>>>    /* Compute profile */
>>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>>> index 459fa07a3bcc..5afe216cf099 100644
>>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>>> @@ -1229,7 +1229,9 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start,
>>>    			if (r)
>>>    				break;
>>>    		}
>>> -		kfd_flush_tlb(pdd, TLB_FLUSH_HEAVYWEIGHT);
>>> +
>>> +		if (kfd_flush_tlb_after_unmap(pdd->dev))
>>> +			kfd_flush_tlb(pdd, TLB_FLUSH_HEAVYWEIGHT);
>>>    	}
>>>    	return r;