From: "Christian König" <ckoenig.leichtzumerken@gmail.com>
To: Felix Kuehling <felix.kuehling@amd.com>,
amd-gfx@lists.freedesktop.org, "Yang,
Philip" <Philip.Yang@amd.com>
Subject: Re: [PATCH 4/6] drm/amdgpu: address remove from fault filter
Date: Wed, 21 Apr 2021 09:55:59 +0200 [thread overview]
Message-ID: <818c1295-2340-a6fe-9b64-e7a5e74d411e@gmail.com> (raw)
In-Reply-To: <d9ff9130-667f-1b5f-83b9-d7dc77743db3@amd.com>
Am 21.04.21 um 03:20 schrieb Felix Kuehling:
> Am 2021-04-20 um 4:21 p.m. schrieb Philip Yang:
>> Add interface to remove address from fault filter ring by resetting
>> fault ring entry of the fault address timestamp to 0, then future vm
>> fault on the address will be processed to recover.
>>
>> Check fault address from fault ring, add address into fault ring and
>> remove address from fault ring are serialized in same interrupt deferred
>> work, don't have race condition.
>>
>> Signed-off-by: Philip Yang <Philip.Yang@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 24 ++++++++++++++++++++++++
>> drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 2 ++
>> 2 files changed, 26 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
>> index c39ed9eb0987..338e45fa66cb 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
>> @@ -387,6 +387,30 @@ bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr,
>> return false;
>> }
>>
>> +/**
>> + * amdgpu_gmc_filter_faults_remove - remove address from VM faults filter
>> + *
>> + * @adev: amdgpu device structure
>> + * @addr: address of the VM fault
>> + * @pasid: PASID of the process causing the fault
>> + *
>> + * Remove the address from fault filter, then future vm fault on this address
>> + * will pass to retry fault handler to recover.
>> + */
>> +void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr,
>> + uint16_t pasid)
>> +{
>> + struct amdgpu_gmc *gmc = &adev->gmc;
>> +
>> + uint64_t key = addr << 4 | pasid;
>> + struct amdgpu_gmc_fault *fault;
>> + uint32_t hash;
>> +
>> + hash = hash_64(key, AMDGPU_GMC_FAULT_HASH_ORDER);
>> + fault = &gmc->fault_ring[gmc->fault_hash[hash].idx];
> You need to loop over the fault ring to find a fault with the matching
> key since there may be hash collisions.
>
> You also need to make sure you don't break the single link list of keys
> with the same hash when you remove an entry. I think the easier way to
> remove an entry without breaking this ring+closed hashing structure is
> to reset the fault->key rather than the fault->timestamp.
>
> Finally, you need to add locking to the fault ring structure. Currently
> it's not protected by any locks because only one thread (the interrupt
> handler) accesses it. Now you have another thread that can remove
> entries, so you need to protect it with a lock. If you are handling
> retry faults, you know that the interrupt handler is really a worker
> thread, so you can use a mutex or a spin-lock, but it doesn't need to be
> interrupt-safe.
I don't think you need a lock at all.
Just using cmpxchg() to update the key should do it.
Something like this:
hash = hash_64(key, AMDGPU_GMC_FAULT_HASH_ORDER);
fault = &gmc->fault_ring[gmc->fault_hash[hash].idx];
while (fault->timestamp >= stamp) {
uint64_t tmp;
cmpxchg(&fault->key, key, 0);
tmp = fault->timestamp;
fault = &gmc->fault_ring[fault->next];
/* Check if the entry was reused */
if (fault->timestamp >= tmp)
break;
}
Regards,
Christian.
>
> Regards,
> Felix
>
>
>> + fault->timestamp = 0;
>> +}
>> +
>> int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev)
>> {
>> int r;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
>> index 9d11c02a3938..498a7a0d5a9e 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
>> @@ -318,6 +318,8 @@ void amdgpu_gmc_agp_location(struct amdgpu_device *adev,
>> struct amdgpu_gmc *mc);
>> bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr,
>> uint16_t pasid, uint64_t timestamp);
>> +void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr,
>> + uint16_t pasid);
>> int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev);
>> void amdgpu_gmc_ras_fini(struct amdgpu_device *adev);
>> int amdgpu_gmc_allocate_vm_inv_eng(struct amdgpu_device *adev);
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-04-21 7:56 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-20 20:21 [PATCH 1/6] drm/amdkfd: retry validation to recover range Philip Yang
2021-04-20 20:21 ` [PATCH 2/6] drm/amdgpu: return IH ring drain finished if ring is empty Philip Yang
2021-04-20 20:21 ` [PATCH 3/6] drm/amdkfd: handle stale retry fault Philip Yang
2021-04-20 20:21 ` [PATCH 4/6] drm/amdgpu: address remove from fault filter Philip Yang
2021-04-21 1:20 ` Felix Kuehling
2021-04-21 7:55 ` Christian König [this message]
2021-04-21 15:29 ` Felix Kuehling
2021-04-23 1:52 ` philip yang
2021-04-21 7:22 ` Christian König
2021-04-23 2:00 ` philip yang
2021-04-20 20:21 ` [PATCH 5/6] drm/amdkfd: enable subsequent retry fault Philip Yang
2021-04-21 1:22 ` Felix Kuehling
2021-04-21 2:08 ` philip yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=818c1295-2340-a6fe-9b64-e7a5e74d411e@gmail.com \
--to=ckoenig.leichtzumerken@gmail.com \
--cc=Philip.Yang@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=felix.kuehling@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).