From: Andrey Grodzovsky <Andrey.Grodzovsky@amd.com> To: christian.koenig@amd.com, Oleg Nesterov <oleg@redhat.com> Cc: David.Panariti@amd.com, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, "Eric W. Biederman" <ebiederm@xmission.com>, Alexander.Deucher@amd.com, akpm@linux-foundation.org Subject: Re: [PATCH 2/3] drm/scheduler: Don't call wait_event_killable for signaled process. Date: Mon, 30 Apr 2018 15:28:42 -0400 [thread overview] Message-ID: <79b2ce10-2cd7-b6f2-551e-0b4ae21072af@amd.com> (raw) In-Reply-To: <bb224134-7ccb-cc87-9a71-3ef1743eb074@gmail.com> On 04/30/2018 02:29 PM, Christian König wrote: > Am 30.04.2018 um 18:10 schrieb Andrey Grodzovsky: >> >> >> On 04/30/2018 12:00 PM, Oleg Nesterov wrote: >>> On 04/30, Andrey Grodzovsky wrote: >>>> What about changing PF_SIGNALED to PF_EXITING in >>>> drm_sched_entity_do_release >>>> >>>> - if ((current->flags & PF_SIGNALED) && current->exit_code == >>>> SIGKILL) >>>> + if ((current->flags & PF_EXITING) && current->exit_code == >>>> SIGKILL) >>> let me repeat, please don't use task->exit_code. And in fact this >>> check is racy >>> >>> But this doesn't matter. Say, we can trivially add >>> SIGNAL_GROUP_KILLED_BY_SIGKILL, >>> or do something else, >> >> >> Can you explain where is the race and what is a possible alternative >> then ? > > The race is that the release doesn't necessarily comes from the > process/context which used the fd. > > E.g. it is just called when the last reference count goes away, but > that can be anywhere not related to the original process using it, > e.g. in a kernel thread or a debugger etc... I still don't see how it is a problem, if release comes from another task, then our process (let's say Firefox who received SIGKILL) won't even get here since fput will not call .release so it will die instantly, the last process who holds the reference (let's say the debugger) when finish will just go to wait_event_timeout and wait for SW queue to be empty from jobs (if any). So all the jobs will have their chance to get to HW anyway. > > The approach with the flush is indeed a really nice idea and I bite > myself to not had that previously as well. Regarding your request from another email to investigate more on .flush Looked at the code and did some reading - From LDD3 "The flush operation is invoked when a process closes its copy of a file descriptor for a device; it should execute (and wait for) any outstanding operations on the device" From printing back trace from dummy .flush hook in our driver - Normal exit (process terminates on it's own) [ 295.586130 < 0.000006>] dump_stack+0x5c/0x78 [ 295.586273 < 0.000143>] my_flush+0xa/0x10 [amdgpu] [ 295.586283 < 0.000010>] filp_close+0x4a/0x90 [ 295.586288 < 0.000005>] SyS_close+0x2d/0x60 [ 295.586295 < 0.000003>] do_syscall_64+0xee/0x270 Exit triggered by fatal signal (not handled signal, including SIGKILL) [ 356.551456 < 0.000008>] dump_stack+0x5c/0x78 [ 356.551592 < 0.000136>] my_flush+0xa/0x10 [amdgpu] [ 356.551597 < 0.000005>] filp_close+0x4a/0x90 [ 356.551605 < 0.000008>] put_files_struct+0xaf/0x120 [ 356.551615 < 0.000010>] do_exit+0x468/0x1280 [ 356.551669 < 0.000009>] do_group_exit+0x89/0x140 [ 356.551679 < 0.000010>] get_signal+0x375/0x8f0 [ 356.551696 < 0.000017>] do_signal+0x79/0xaa0 [ 356.551756 < 0.000014>] exit_to_usermode_loop+0x83/0xd0 [ 356.551764 < 0.000008>] do_syscall_64+0x244/0x270 So as it was said here before, it will be called for every process closing his FD to the file. But again, I don't quire see yet what we earn by using .flush, is it that you force every process holding reference to DRM file not die until all jobs are submitted to HW (as long as the process not being killed by a signal) ? Andrey > > Christian. The idea here is that any task still referencing this file and putting down the reference and is not exiting due to SIGKILL will just have to go through the slow path - wait for jobs completion on GPU (with some TO). > >> >>> but I fail to understand what are you trying to do. Suppose >>> that the check above is correct in that it is true iff the task is >>> exiting and >>> it was killed by SIGKILL. What about the "else" branch which does >>> >>> r = wait_event_killable(sched->job_scheduled, ...) >>> >>> ? >>> >>> Once again, fatal_signal_pending() (or even signal_pending()) is not >>> well defined >>> after the exiting task passes exit_signals(). >>> >>> So wait_event_killable() can fail because fatal_signal_pending() is >>> true; and this >>> can happen even if it was not killed. >>> >>> Or it can block and SIGKILL won't be able to wake it up. >>> >>>> If SIGINT was sent then it's SIGINT, >>> Yes, but see above. in this case fatal_signal_pending() will be >>> likely true so >>> wait_event_killable() will fail unless condition is already true. >> >> My bad, I didn't show the full intended fix, it was just a snippet to >> address the differentiation between exiting >> do to SIGKILL and any other exit, I also intended to change >> wait_event_killable to wait_event_timeout. >> >> Andrey >> >>> >>> Oleg. >>> >> >> _______________________________________________ >> amd-gfx mailing list >> amd-gfx@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/amd-gfx >
WARNING: multiple messages have this Message-ID (diff)
From: Andrey Grodzovsky <Andrey.Grodzovsky-5C7GfCeVMHo@public.gmane.org> To: christian.koenig-5C7GfCeVMHo@public.gmane.org, Oleg Nesterov <oleg-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Cc: David.Panariti-5C7GfCeVMHo@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>, Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org Subject: Re: [PATCH 2/3] drm/scheduler: Don't call wait_event_killable for signaled process. Date: Mon, 30 Apr 2018 15:28:42 -0400 [thread overview] Message-ID: <79b2ce10-2cd7-b6f2-551e-0b4ae21072af@amd.com> (raw) In-Reply-To: <bb224134-7ccb-cc87-9a71-3ef1743eb074-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> On 04/30/2018 02:29 PM, Christian König wrote: > Am 30.04.2018 um 18:10 schrieb Andrey Grodzovsky: >> >> >> On 04/30/2018 12:00 PM, Oleg Nesterov wrote: >>> On 04/30, Andrey Grodzovsky wrote: >>>> What about changing PF_SIGNALED to PF_EXITING in >>>> drm_sched_entity_do_release >>>> >>>> - if ((current->flags & PF_SIGNALED) && current->exit_code == >>>> SIGKILL) >>>> + if ((current->flags & PF_EXITING) && current->exit_code == >>>> SIGKILL) >>> let me repeat, please don't use task->exit_code. And in fact this >>> check is racy >>> >>> But this doesn't matter. Say, we can trivially add >>> SIGNAL_GROUP_KILLED_BY_SIGKILL, >>> or do something else, >> >> >> Can you explain where is the race and what is a possible alternative >> then ? > > The race is that the release doesn't necessarily comes from the > process/context which used the fd. > > E.g. it is just called when the last reference count goes away, but > that can be anywhere not related to the original process using it, > e.g. in a kernel thread or a debugger etc... I still don't see how it is a problem, if release comes from another task, then our process (let's say Firefox who received SIGKILL) won't even get here since fput will not call .release so it will die instantly, the last process who holds the reference (let's say the debugger) when finish will just go to wait_event_timeout and wait for SW queue to be empty from jobs (if any). So all the jobs will have their chance to get to HW anyway. > > The approach with the flush is indeed a really nice idea and I bite > myself to not had that previously as well. Regarding your request from another email to investigate more on .flush Looked at the code and did some reading - From LDD3 "The flush operation is invoked when a process closes its copy of a file descriptor for a device; it should execute (and wait for) any outstanding operations on the device" From printing back trace from dummy .flush hook in our driver - Normal exit (process terminates on it's own) [ 295.586130 < 0.000006>] dump_stack+0x5c/0x78 [ 295.586273 < 0.000143>] my_flush+0xa/0x10 [amdgpu] [ 295.586283 < 0.000010>] filp_close+0x4a/0x90 [ 295.586288 < 0.000005>] SyS_close+0x2d/0x60 [ 295.586295 < 0.000003>] do_syscall_64+0xee/0x270 Exit triggered by fatal signal (not handled signal, including SIGKILL) [ 356.551456 < 0.000008>] dump_stack+0x5c/0x78 [ 356.551592 < 0.000136>] my_flush+0xa/0x10 [amdgpu] [ 356.551597 < 0.000005>] filp_close+0x4a/0x90 [ 356.551605 < 0.000008>] put_files_struct+0xaf/0x120 [ 356.551615 < 0.000010>] do_exit+0x468/0x1280 [ 356.551669 < 0.000009>] do_group_exit+0x89/0x140 [ 356.551679 < 0.000010>] get_signal+0x375/0x8f0 [ 356.551696 < 0.000017>] do_signal+0x79/0xaa0 [ 356.551756 < 0.000014>] exit_to_usermode_loop+0x83/0xd0 [ 356.551764 < 0.000008>] do_syscall_64+0x244/0x270 So as it was said here before, it will be called for every process closing his FD to the file. But again, I don't quire see yet what we earn by using .flush, is it that you force every process holding reference to DRM file not die until all jobs are submitted to HW (as long as the process not being killed by a signal) ? Andrey > > Christian. The idea here is that any task still referencing this file and putting down the reference and is not exiting due to SIGKILL will just have to go through the slow path - wait for jobs completion on GPU (with some TO). > >> >>> but I fail to understand what are you trying to do. Suppose >>> that the check above is correct in that it is true iff the task is >>> exiting and >>> it was killed by SIGKILL. What about the "else" branch which does >>> >>> r = wait_event_killable(sched->job_scheduled, ...) >>> >>> ? >>> >>> Once again, fatal_signal_pending() (or even signal_pending()) is not >>> well defined >>> after the exiting task passes exit_signals(). >>> >>> So wait_event_killable() can fail because fatal_signal_pending() is >>> true; and this >>> can happen even if it was not killed. >>> >>> Or it can block and SIGKILL won't be able to wake it up. >>> >>>> If SIGINT was sent then it's SIGINT, >>> Yes, but see above. in this case fatal_signal_pending() will be >>> likely true so >>> wait_event_killable() will fail unless condition is already true. >> >> My bad, I didn't show the full intended fix, it was just a snippet to >> address the differentiation between exiting >> do to SIGKILL and any other exit, I also intended to change >> wait_event_killable to wait_event_timeout. >> >> Andrey >> >>> >>> Oleg. >>> >> >> _______________________________________________ >> amd-gfx mailing list >> amd-gfx@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/amd-gfx > _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2018-04-30 19:34 UTC|newest] Thread overview: 122+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-04-24 15:30 Avoid uninterruptible sleep during process exit Andrey Grodzovsky 2018-04-24 15:30 ` Andrey Grodzovsky 2018-04-24 15:30 ` [PATCH 1/3] signals: Allow generation of SIGKILL to exiting task Andrey Grodzovsky 2018-04-24 15:30 ` Andrey Grodzovsky 2018-04-24 16:10 ` Eric W. Biederman 2018-04-24 16:10 ` Eric W. Biederman 2018-04-24 16:42 ` Eric W. Biederman 2018-04-24 16:42 ` Eric W. Biederman 2018-04-24 16:51 ` Andrey Grodzovsky 2018-04-24 16:51 ` Andrey Grodzovsky 2018-04-24 17:29 ` Eric W. Biederman 2018-04-25 13:13 ` Oleg Nesterov 2018-04-24 15:30 ` [PATCH 2/3] drm/scheduler: Don't call wait_event_killable for signaled process Andrey Grodzovsky 2018-04-24 15:30 ` Andrey Grodzovsky 2018-04-24 15:46 ` Michel Dänzer 2018-04-24 15:51 ` Andrey Grodzovsky 2018-04-24 15:51 ` Andrey Grodzovsky 2018-04-24 15:52 ` Andrey Grodzovsky 2018-04-24 15:52 ` Andrey Grodzovsky 2018-04-24 19:44 ` Daniel Vetter 2018-04-24 19:44 ` Daniel Vetter 2018-04-24 21:00 ` Eric W. Biederman 2018-04-24 21:02 ` Andrey Grodzovsky 2018-04-24 21:02 ` Andrey Grodzovsky 2018-04-24 21:21 ` Eric W. Biederman 2018-04-24 21:37 ` Andrey Grodzovsky 2018-04-24 21:37 ` Andrey Grodzovsky 2018-04-24 22:11 ` Eric W. Biederman 2018-04-25 7:14 ` Daniel Vetter 2018-04-25 13:08 ` Andrey Grodzovsky 2018-04-25 13:08 ` Andrey Grodzovsky 2018-04-25 15:29 ` Eric W. Biederman 2018-04-25 16:13 ` Andrey Grodzovsky 2018-04-25 16:31 ` Eric W. Biederman 2018-04-24 21:40 ` Daniel Vetter 2018-04-24 21:40 ` Daniel Vetter 2018-04-25 13:22 ` Oleg Nesterov 2018-04-25 13:36 ` Daniel Vetter 2018-04-25 14:18 ` Oleg Nesterov 2018-04-25 14:18 ` Oleg Nesterov 2018-04-25 13:43 ` Andrey Grodzovsky 2018-04-25 13:43 ` Andrey Grodzovsky 2018-04-24 16:23 ` Eric W. Biederman 2018-04-24 16:23 ` Eric W. Biederman 2018-04-24 16:43 ` Andrey Grodzovsky 2018-04-24 16:43 ` Andrey Grodzovsky 2018-04-24 17:12 ` Eric W. Biederman 2018-04-25 13:55 ` Oleg Nesterov 2018-04-25 14:21 ` Andrey Grodzovsky 2018-04-25 14:21 ` Andrey Grodzovsky 2018-04-25 17:17 ` Oleg Nesterov 2018-04-25 18:40 ` Andrey Grodzovsky 2018-04-25 18:40 ` Andrey Grodzovsky 2018-04-26 0:01 ` Eric W. Biederman 2018-04-26 12:34 ` Andrey Grodzovsky 2018-04-26 12:34 ` Andrey Grodzovsky 2018-04-26 12:52 ` Andrey Grodzovsky 2018-04-26 12:52 ` Andrey Grodzovsky 2018-04-26 15:57 ` Eric W. Biederman 2018-04-26 20:43 ` Andrey Grodzovsky 2018-04-26 20:43 ` Andrey Grodzovsky 2018-04-30 12:08 ` Christian König 2018-04-30 12:08 ` Christian König 2018-04-30 14:32 ` Andrey Grodzovsky 2018-04-30 14:32 ` Andrey Grodzovsky 2018-04-30 15:25 ` Christian König 2018-04-30 15:25 ` Christian König 2018-04-30 16:00 ` Oleg Nesterov 2018-04-30 16:10 ` Andrey Grodzovsky 2018-04-30 16:10 ` Andrey Grodzovsky 2018-04-30 18:29 ` Christian König 2018-04-30 18:29 ` Christian König 2018-04-30 19:28 ` Andrey Grodzovsky [this message] 2018-04-30 19:28 ` Andrey Grodzovsky 2018-05-02 11:48 ` Christian König 2018-05-02 11:48 ` Christian König 2018-05-17 11:18 ` Andrey Grodzovsky 2018-05-17 14:48 ` Michel Dänzer 2018-05-17 15:33 ` Andrey Grodzovsky 2018-05-17 15:52 ` Michel Dänzer 2018-05-17 19:05 ` Andrey Grodzovsky 2018-05-18 8:46 ` Michel Dänzer 2018-05-18 9:42 ` Christian König 2018-05-18 14:44 ` Michel Dänzer 2018-05-18 14:50 ` Christian König 2018-05-18 15:02 ` Andrey Grodzovsky 2018-05-22 12:58 ` Christian König 2018-05-22 15:49 ` Andrey Grodzovsky 2018-05-22 16:09 ` Michel Dänzer 2018-05-22 16:30 ` Andrey Grodzovsky 2018-05-22 16:33 ` Michel Dänzer 2018-05-22 16:37 ` Andrey Grodzovsky 2018-05-01 14:35 ` Oleg Nesterov 2018-05-23 15:08 ` Andrey Grodzovsky 2018-05-23 15:08 ` Andrey Grodzovsky 2018-04-30 15:29 ` Oleg Nesterov 2018-04-30 16:25 ` Eric W. Biederman 2018-04-30 17:18 ` Andrey Grodzovsky 2018-04-30 17:18 ` Andrey Grodzovsky 2018-04-25 13:05 ` Oleg Nesterov 2018-04-24 15:30 ` [PATCH 3/3] drm/amdgpu: Switch to interrupted wait to recover from ring hang Andrey Grodzovsky 2018-04-24 15:30 ` Andrey Grodzovsky 2018-04-24 15:52 ` Panariti, David 2018-04-24 15:52 ` Panariti, David 2018-04-24 15:58 ` Andrey Grodzovsky 2018-04-24 15:58 ` Andrey Grodzovsky 2018-04-24 16:20 ` Panariti, David 2018-04-24 16:20 ` Panariti, David 2018-04-24 16:30 ` Eric W. Biederman 2018-04-24 16:30 ` Eric W. Biederman 2018-04-25 17:17 ` Andrey Grodzovsky 2018-04-25 17:17 ` Andrey Grodzovsky 2018-04-25 20:55 ` Eric W. Biederman 2018-04-25 20:55 ` Eric W. Biederman 2018-04-26 12:28 ` Andrey Grodzovsky 2018-04-26 12:28 ` Andrey Grodzovsky 2018-04-24 16:14 ` Eric W. Biederman 2018-04-24 16:14 ` Eric W. Biederman 2018-04-24 16:38 ` Andrey Grodzovsky 2018-04-24 16:38 ` Andrey Grodzovsky 2018-04-30 11:34 ` Christian König 2018-04-30 11:34 ` Christian König
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=79b2ce10-2cd7-b6f2-551e-0b4ae21072af@amd.com \ --to=andrey.grodzovsky@amd.com \ --cc=Alexander.Deucher@amd.com \ --cc=David.Panariti@amd.com \ --cc=akpm@linux-foundation.org \ --cc=amd-gfx@lists.freedesktop.org \ --cc=christian.koenig@amd.com \ --cc=ebiederm@xmission.com \ --cc=linux-kernel@vger.kernel.org \ --cc=oleg@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.