Re: [PATCH] drm/amdkfd: Flush the process wq before creating a kfd_process

[Felix Kuehling <felix.kuehling@amd.com>]

On 2024-04-26 14:55, Lancelot SIX wrote:
> There is a race condition when re-creating a kfd_process for a process.
> This has been observed when a process under the debugger executes
> exec(3).  In this scenario:
> - The process executes exec.
>   - This will eventually release the process's mm, which will cause the
>     kfd_process object associated with the process to be freed
>     (kfd_process_free_notifier decrements the reference count to the
>     kfd_process to 0).  This causes kfd_process_ref_release to enqueue
>     kfd_process_wq_release to the kfd_process_wq.
> - The debugger receives the PTRACE_EVENT_EXEC notification, and tries to
>    re-enable AMDGPU traps (KFD_IOC_DBG_TRAP_ENABLE).
>   - When handling this request, KFD tries to re-create a kfd_process.
>     This eventually calls kfd_create_process and kobject_init_and_add.
>
> At this point the call to kobject_init_and_add can fail because the
> old kfd_process.kobj has not been freed yet by kfd_process_wq_release.
>
> This patch proposes to avoid this race by making sure to drain
> kfd_process_wq before creating a new kfd_process object.  This way, we
> know that any cleanup task is done executing when we reach
> kobject_init_and_add.
>
> Signed-off-by: Lancelot SIX <lancelot.six@amd.com>

Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>


> ---
>   drivers/gpu/drm/amd/amdkfd/kfd_process.c | 8 ++++++++
>   1 file changed, 8 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 58c1fe542193..451bb058cc62 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -829,6 +829,14 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
>   	if (process) {
>   		pr_debug("Process already found\n");
>   	} else {
> +		/* If the process just called exec(3), it is possible that the
> +		 * cleanup of the kfd_process (following the release of the mm
> +		 * of the old process image) is still in the cleanup work queue.
> +		 * Make sure to drain any job before trying to recreate any
> +		 * resource for this process.
> +		 */
> +		flush_workqueue(kfd_process_wq);
> +
>   		process = create_process(thread);
>   		if (IS_ERR(process))
>   			goto out;
>
> base-commit: cf743996352e327f483dc7d66606c90276f57380