* [PATCH 0/1] kernel/module.c: don't allow modprobe to hang forever on a module load
@ 2019-11-13 9:29 Konstantin Khorenko
2019-11-13 9:29 ` [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload Konstantin Khorenko
0 siblings, 1 reply; 4+ messages in thread
From: Konstantin Khorenko @ 2019-11-13 9:29 UTC (permalink / raw)
To: Jessica Yu, Prarit Bhargava, Barret Rhoden
Cc: Konstantin Khorenko, Andrey Ryabinin, linux-kernel, David Arcari,
Heiko Carstens
After commit 6e6de3dee51a ("kernel/module.c: Only return -EEXIST
for modules that have finished loading")
the simple test leads to hanged modprobe process (INTERRUPTIBLE):
1. Make sure nft_ct module is not used by your firewall rules.
2. Run 3 copies of
# export i=0; while true; if [[ $(($i % 100)) -eq 0 ]] ; then \
echo "i=$i"; fi; do modprobe nft_ct; i=$(($i + 1)); done
3. Run 2 copies of
# while true; do rmmod nft_ct; done
Hanged "modprobe" process will appear in ~ 10 seconds.
# cat /proc/30184/stack
[<0>] load_module+0x53f/0x2060
[<0>] __do_sys_finit_module+0xd2/0x100
[<0>] do_syscall_64+0x5b/0x1c0
[<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Konstantin Khorenko (1):
kernel/module.c: wakeup processes in module_wq on module unload
kernel/module.c | 2 ++
1 file changed, 2 insertions(+)
--
2.15.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload
2019-11-13 9:29 [PATCH 0/1] kernel/module.c: don't allow modprobe to hang forever on a module load Konstantin Khorenko
@ 2019-11-13 9:29 ` Konstantin Khorenko
2019-11-13 13:51 ` Prarit Bhargava
2019-11-15 10:28 ` Jessica Yu
0 siblings, 2 replies; 4+ messages in thread
From: Konstantin Khorenko @ 2019-11-13 9:29 UTC (permalink / raw)
To: Jessica Yu, Prarit Bhargava, Barret Rhoden
Cc: Konstantin Khorenko, Andrey Ryabinin, linux-kernel, David Arcari,
Heiko Carstens
Fix the race between load and unload a kernel module.
sys_delete_module()
try_stop_module()
mod->state = _GOING
add_unformed_module()
old = find_module_all()
(old->state == _GOING =>
wait_event_interruptible())
During pre-condition
finished_loading() rets 0
schedule()
(never gets waken up later)
free_module()
mod->state = _UNFORMED
list_del_rcu(&mod->list)
(dels mod from "modules" list)
return
The race above leads to modprobe hanging forever on loading
a module.
Error paths on loading module call wake_up_all(&module_wq) after
freeing module, so let's do the same on straight module unload.
Fixes: 6e6de3dee51a ("kernel/module.c: Only return -EEXIST
for modules that have finished loading")
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
kernel/module.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/module.c b/kernel/module.c
index ff2d7359a418..cb09a5f37a5f 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1033,6 +1033,8 @@ SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
strlcpy(last_unloaded_module, mod->name, sizeof(last_unloaded_module));
free_module(mod);
+ /* someone could wait for the module in add_unformed_module() */
+ wake_up_all(&module_wq);
return 0;
out:
mutex_unlock(&module_mutex);
--
2.15.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload
2019-11-13 9:29 ` [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload Konstantin Khorenko
@ 2019-11-13 13:51 ` Prarit Bhargava
2019-11-15 10:28 ` Jessica Yu
1 sibling, 0 replies; 4+ messages in thread
From: Prarit Bhargava @ 2019-11-13 13:51 UTC (permalink / raw)
To: Konstantin Khorenko, Jessica Yu, Barret Rhoden
Cc: Andrey Ryabinin, linux-kernel, David Arcari, Heiko Carstens
On 11/13/19 4:29 AM, Konstantin Khorenko wrote:
> Fix the race between load and unload a kernel module.
>
> sys_delete_module()
> try_stop_module()
> mod->state = _GOING
> add_unformed_module()
> old = find_module_all()
> (old->state == _GOING =>
> wait_event_interruptible())
>
> During pre-condition
> finished_loading() rets 0
> schedule()
> (never gets waken up later)
> free_module()
> mod->state = _UNFORMED
> list_del_rcu(&mod->list)
> (dels mod from "modules" list)
>
> return
>
> The race above leads to modprobe hanging forever on loading
> a module.
>
> Error paths on loading module call wake_up_all(&module_wq) after
> freeing module, so let's do the same on straight module unload.
>
> Fixes: 6e6de3dee51a ("kernel/module.c: Only return -EEXIST
> for modules that have finished loading")
>
> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
> ---
> kernel/module.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/kernel/module.c b/kernel/module.c
> index ff2d7359a418..cb09a5f37a5f 100644
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -1033,6 +1033,8 @@ SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
> strlcpy(last_unloaded_module, mod->name, sizeof(last_unloaded_module));
>
> free_module(mod);
> + /* someone could wait for the module in add_unformed_module() */
> + wake_up_all(&module_wq);
free_module() acts on that *this* module so it makes sense to include the
wake_up_all(&module_wq) call in delete_module. IIRC I did try a module load &
unload stress test but it looks like my testing wasn't hard enough :(
In any case,
Reviewed-by: Prarit Bhargava <prarit@redhat.com>
P.
> return 0;
> out:
> mutex_unlock(&module_mutex);
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload
2019-11-13 9:29 ` [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload Konstantin Khorenko
2019-11-13 13:51 ` Prarit Bhargava
@ 2019-11-15 10:28 ` Jessica Yu
1 sibling, 0 replies; 4+ messages in thread
From: Jessica Yu @ 2019-11-15 10:28 UTC (permalink / raw)
To: Konstantin Khorenko
Cc: Prarit Bhargava, Barret Rhoden, Andrey Ryabinin, linux-kernel,
David Arcari, Heiko Carstens
+++ Konstantin Khorenko [13/11/19 12:29 +0300]:
>Fix the race between load and unload a kernel module.
>
>sys_delete_module()
> try_stop_module()
> mod->state = _GOING
> add_unformed_module()
> old = find_module_all()
> (old->state == _GOING =>
> wait_event_interruptible())
>
> During pre-condition
> finished_loading() rets 0
> schedule()
> (never gets waken up later)
> free_module()
> mod->state = _UNFORMED
> list_del_rcu(&mod->list)
> (dels mod from "modules" list)
>
>return
>
>The race above leads to modprobe hanging forever on loading
>a module.
>
>Error paths on loading module call wake_up_all(&module_wq) after
>freeing module, so let's do the same on straight module unload.
>
>Fixes: 6e6de3dee51a ("kernel/module.c: Only return -EEXIST
>for modules that have finished loading")
>
>Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
Thanks Konstantin for catching this. I've applied this to
modules-next.
Jessica
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-11-15 10:28 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-13 9:29 [PATCH 0/1] kernel/module.c: don't allow modprobe to hang forever on a module load Konstantin Khorenko
2019-11-13 9:29 ` [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload Konstantin Khorenko
2019-11-13 13:51 ` Prarit Bhargava
2019-11-15 10:28 ` Jessica Yu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).