* Re: [PATCH 2/2] drm/amdkfd: set new unregistered range accessible by GPU
2021-05-05 17:56 ` [PATCH 2/2] drm/amdkfd: set new unregistered range accessible by GPU Philip Yang
@ 2021-05-05 20:03 ` Felix Kuehling
2021-05-07 19:07 ` [PATCH 2/2] drm/amdkfd: unregistered range accessible by all GPUs Philip Yang
2021-05-11 15:15 ` [PATCH v2 2/2] drm/amdkfd: new " Philip Yang
2 siblings, 0 replies; 10+ messages in thread
From: Felix Kuehling @ 2021-05-05 20:03 UTC (permalink / raw)
To: amd-gfx, Yang, Philip
Am 2021-05-05 um 1:56 p.m. schrieb Philip Yang:
> GPU retry vm fault on unregistered address, new range is created to
> recover the retry vm fault. Instead of setting new range perferred_loc
> to GPU, add GPU to new range access_bitmap, to be able to update GPU
> page table after new range migrate to VRAM. The new range preferred_loc
> is default value KFD_IOCTL_SVM_LOCATION_UNDEFINED.
>
> Correct one typo.
>
> Signed-off-by: Philip Yang <Philip.Yang@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> index 3b640772d9c5..5a486e0fbf18 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> @@ -2243,7 +2243,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
>
> prange = svm_range_new(&p->svms, start, last);
> if (!prange) {
> - pr_debug("Failed to create prange in address [0x%llx]\\n", addr);
> + pr_debug("Failed to create prange in address [0x%llx]\n", addr);
> return NULL;
> }
> if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) {
> @@ -2251,9 +2251,8 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
> svm_range_free(prange);
> return NULL;
> }
> - prange->preferred_loc = gpuid;
> - prange->actual_loc = 0;
> - /* Gurantee prange is migrate it */
> +
> + bitmap_set(prange->bitmap_access, gpuidx, 1);
By default, with XNACK enabled, the default should be that all GPUs have
access to memory (but not access in place).
Regards,
Felix
> svm_range_add_to_svms(prange);
> svm_range_add_notifier_locked(mm, prange);
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 2/2] drm/amdkfd: unregistered range accessible by all GPUs
2021-05-05 17:56 ` [PATCH 2/2] drm/amdkfd: set new unregistered range accessible by GPU Philip Yang
2021-05-05 20:03 ` Felix Kuehling
@ 2021-05-07 19:07 ` Philip Yang
2021-05-10 12:56 ` Felix Kuehling
2021-05-11 15:15 ` [PATCH v2 2/2] drm/amdkfd: new " Philip Yang
2 siblings, 1 reply; 10+ messages in thread
From: Philip Yang @ 2021-05-07 19:07 UTC (permalink / raw)
To: amd-gfx; +Cc: Philip Yang
New range is created to recover retry vm fault, set all GPUs have access
to the range. The new range preferred_loc is default value
KFD_IOCTL_SVM_LOCATION_UNDEFINED.
Correct one typo.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index d9111fea724b..537b12e75f54 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -2243,7 +2243,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
prange = svm_range_new(&p->svms, start, last);
if (!prange) {
- pr_debug("Failed to create prange in address [0x%llx]\\n", addr);
+ pr_debug("Failed to create prange in address [0x%llx]\n", addr);
return NULL;
}
if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) {
@@ -2251,9 +2251,8 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
svm_range_free(prange);
return NULL;
}
- prange->preferred_loc = gpuid;
- prange->actual_loc = 0;
- /* Gurantee prange is migrate it */
+
+ bitmap_fill(prange->bitmap_access, MAX_GPU_INSTANCE);
svm_range_add_to_svms(prange);
svm_range_add_notifier_locked(mm, prange);
--
2.17.1
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 2/2] drm/amdkfd: unregistered range accessible by all GPUs
2021-05-07 19:07 ` [PATCH 2/2] drm/amdkfd: unregistered range accessible by all GPUs Philip Yang
@ 2021-05-10 12:56 ` Felix Kuehling
2021-05-11 15:14 ` philip yang
0 siblings, 1 reply; 10+ messages in thread
From: Felix Kuehling @ 2021-05-10 12:56 UTC (permalink / raw)
To: Philip Yang, amd-gfx
Am 2021-05-07 um 3:07 p.m. schrieb Philip Yang:
> New range is created to recover retry vm fault, set all GPUs have access
> to the range. The new range preferred_loc is default value
> KFD_IOCTL_SVM_LOCATION_UNDEFINED.
>
> Correct one typo.
>
> Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Would it be better to move this into svm_range_new, conditional on
p->xnack_enabled? That way it would correctly apply to ranges created
through SVM API calls (e.g. in svm_range_handle_overlap or
svm_range_add) as well?
Regards,
Felix
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> index d9111fea724b..537b12e75f54 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> @@ -2243,7 +2243,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
>
> prange = svm_range_new(&p->svms, start, last);
> if (!prange) {
> - pr_debug("Failed to create prange in address [0x%llx]\\n", addr);
> + pr_debug("Failed to create prange in address [0x%llx]\n", addr);
> return NULL;
> }
> if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) {
> @@ -2251,9 +2251,8 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
> svm_range_free(prange);
> return NULL;
> }
> - prange->preferred_loc = gpuid;
> - prange->actual_loc = 0;
> - /* Gurantee prange is migrate it */
> +
> + bitmap_fill(prange->bitmap_access, MAX_GPU_INSTANCE);
> svm_range_add_to_svms(prange);
> svm_range_add_notifier_locked(mm, prange);
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 2/2] drm/amdkfd: new range accessible by all GPUs
2021-05-05 17:56 ` [PATCH 2/2] drm/amdkfd: set new unregistered range accessible by GPU Philip Yang
2021-05-05 20:03 ` Felix Kuehling
2021-05-07 19:07 ` [PATCH 2/2] drm/amdkfd: unregistered range accessible by all GPUs Philip Yang
@ 2021-05-11 15:15 ` Philip Yang
2021-05-11 15:39 ` Felix Kuehling
2 siblings, 1 reply; 10+ messages in thread
From: Philip Yang @ 2021-05-11 15:15 UTC (permalink / raw)
To: amd-gfx; +Cc: Philip Yang
If xnack is on, new range is created to recover retry vm fault or
created by SVM API calls, set all GPUs have access to the range.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index d9111fea724b..6a677bdfcadb 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -258,6 +258,7 @@ svm_range *svm_range_new(struct svm_range_list *svms, uint64_t start,
{
uint64_t size = last - start + 1;
struct svm_range *prange;
+ struct kfd_process *p;
prange = kzalloc(sizeof(*prange), GFP_KERNEL);
if (!prange)
@@ -277,6 +278,11 @@ svm_range *svm_range_new(struct svm_range_list *svms, uint64_t start,
prange->validate_timestamp = 0;
mutex_init(&prange->migrate_mutex);
mutex_init(&prange->lock);
+
+ p = container_of(svms, struct kfd_process, svms);
+ if (p->xnack_enabled)
+ bitmap_fill(prange->bitmap_access, MAX_GPU_INSTANCE);
+
svm_range_set_default_attributes(&prange->preferred_loc,
&prange->prefetch_loc,
&prange->granularity, &prange->flags);
@@ -2243,7 +2249,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
prange = svm_range_new(&p->svms, start, last);
if (!prange) {
- pr_debug("Failed to create prange in address [0x%llx]\\n", addr);
+ pr_debug("Failed to create prange in address [0x%llx]\n", addr);
return NULL;
}
if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) {
@@ -2251,9 +2257,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
svm_range_free(prange);
return NULL;
}
- prange->preferred_loc = gpuid;
- prange->actual_loc = 0;
- /* Gurantee prange is migrate it */
+
svm_range_add_to_svms(prange);
svm_range_add_notifier_locked(mm, prange);
--
2.17.1
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2 2/2] drm/amdkfd: new range accessible by all GPUs
2021-05-11 15:15 ` [PATCH v2 2/2] drm/amdkfd: new " Philip Yang
@ 2021-05-11 15:39 ` Felix Kuehling
0 siblings, 0 replies; 10+ messages in thread
From: Felix Kuehling @ 2021-05-11 15:39 UTC (permalink / raw)
To: Philip Yang, amd-gfx
Am 2021-05-11 um 11:15 a.m. schrieb Philip Yang:
> If xnack is on, new range is created to recover retry vm fault or
> created by SVM API calls, set all GPUs have access to the range.
>
> Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 12 ++++++++----
> 1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> index d9111fea724b..6a677bdfcadb 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> @@ -258,6 +258,7 @@ svm_range *svm_range_new(struct svm_range_list *svms, uint64_t start,
> {
> uint64_t size = last - start + 1;
> struct svm_range *prange;
> + struct kfd_process *p;
>
> prange = kzalloc(sizeof(*prange), GFP_KERNEL);
> if (!prange)
> @@ -277,6 +278,11 @@ svm_range *svm_range_new(struct svm_range_list *svms, uint64_t start,
> prange->validate_timestamp = 0;
> mutex_init(&prange->migrate_mutex);
> mutex_init(&prange->lock);
> +
> + p = container_of(svms, struct kfd_process, svms);
> + if (p->xnack_enabled)
> + bitmap_fill(prange->bitmap_access, MAX_GPU_INSTANCE);
> +
> svm_range_set_default_attributes(&prange->preferred_loc,
> &prange->prefetch_loc,
> &prange->granularity, &prange->flags);
> @@ -2243,7 +2249,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
>
> prange = svm_range_new(&p->svms, start, last);
> if (!prange) {
> - pr_debug("Failed to create prange in address [0x%llx]\\n", addr);
> + pr_debug("Failed to create prange in address [0x%llx]\n", addr);
> return NULL;
> }
> if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) {
> @@ -2251,9 +2257,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
> svm_range_free(prange);
> return NULL;
> }
> - prange->preferred_loc = gpuid;
> - prange->actual_loc = 0;
> - /* Gurantee prange is migrate it */
> +
> svm_range_add_to_svms(prange);
> svm_range_add_notifier_locked(mm, prange);
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 10+ messages in thread