* [Bug 208081] New: Memory leak in kvm_async_pf_task_wake
@ 2020-06-06 0:35 bugzilla-daemon
2020-06-06 0:37 ` [Bug 208081] " bugzilla-daemon
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-06 0:35 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
Bug ID: 208081
Summary: Memory leak in kvm_async_pf_task_wake
Product: Virtualization
Version: unspecified
Kernel Version: 5.6.14
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: kvm
Assignee: virtualization_kvm@kernel-bugs.osdl.org
Reporter: sites+kernel@d.sb
Regression: No
I have several KVM virtual servers at a number of hosting providers. On just
one of them, the unreclaimable slab memory is growing linearly over time, until
it hits a maximum (when the server's memory is 100% allocated). All the memory
is allocated in kmalloc-64 slabs.
After enabling slab debugging using slub_debug=U,
/sys/kernel/slab/kmalloc-64/alloc_calls says that most of the allocations are
coming from kvm_async_pf_task_wake
This looks very similar to this blog post:
https://darkimmortal.com/debian-10-kernel-slab-memory-leak/. Also see my post
on ServerFault:
https://serverfault.com/questions/1020241/debugging-kmalloc-64-slab-allocations-memory-leak
Any suggestions on how to debug this? It seems like it could be a kernel bug.
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
@ 2020-06-06 0:37 ` bugzilla-daemon
2020-06-08 0:46 ` bugzilla-daemon
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-06 0:37 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
--- Comment #1 from Daniel Lo Nigro (sites+kernel@d.sb) ---
I forgot to mention that this VPS is running Debian Bullseye (testing)
root@lux01:~# uname -a
Linux lux01 5.6.0-2-cloud-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64
GNU/Linux
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
2020-06-06 0:37 ` [Bug 208081] " bugzilla-daemon
@ 2020-06-08 0:46 ` bugzilla-daemon
2020-06-09 7:17 ` bugzilla-daemon
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-08 0:46 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
Wanpeng Li (wanpeng.li@hotmail.com) changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |wanpeng.li@hotmail.com
--- Comment #2 from Wanpeng Li (wanpeng.li@hotmail.com) ---
Could you apply below to the guest kernel and have a try again?
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index d6f22a3..93d267e 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -156,6 +156,7 @@ static void apf_task_wake_one(struct kvm_task_sleep_node
*n)
hlist_del_init(&n->link);
if (swq_has_sleeper(&n->wq))
swake_up_one(&n->wq);
+ kfree(n);
}
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
2020-06-06 0:37 ` [Bug 208081] " bugzilla-daemon
2020-06-08 0:46 ` bugzilla-daemon
@ 2020-06-09 7:17 ` bugzilla-daemon
2020-06-09 13:00 ` bugzilla-daemon
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-09 7:17 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
--- Comment #3 from Daniel Lo Nigro (sites+kernel@d.sb) ---
@Wanpeng Li - I'll try that out and let you know how it goes.
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
` (2 preceding siblings ...)
2020-06-09 7:17 ` bugzilla-daemon
@ 2020-06-09 13:00 ` bugzilla-daemon
2020-06-09 17:09 ` bugzilla-daemon
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-09 13:00 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
--- Comment #4 from Paolo Bonzini (bonzini@gnu.org) ---
That patch won't work, most APFs have a node that comes from the stack. The
issue must be arising when you enter this branch of kvm_async_pf_task_wake:
/*
* async PF was not yet handled.
* Add dummy entry for the token.
*/
n = kzalloc(sizeof(*n), GFP_ATOMIC);
but it should be handled here in kvm_async_pf_task_wait:
if (e) {
/* dummy entry exist -> wake up was delivered ahead of PF */
hlist_del(&e->link);
raw_spin_unlock(&b->lock);
kfree(e);
return false;
}
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
` (3 preceding siblings ...)
2020-06-09 13:00 ` bugzilla-daemon
@ 2020-06-09 17:09 ` bugzilla-daemon
2020-06-11 1:03 ` bugzilla-daemon
2023-01-24 7:54 ` bugzilla-daemon
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-09 17:09 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
--- Comment #5 from Daniel Lo Nigro (sites+kernel@d.sb) ---
Yeah, after applying that patch, I ended up getting a kernel panic on the
kfree() call.
A blog post I read
(https://darkimmortal.com/debian-10-kernel-slab-memory-leak/) mentioned adding
"no-kvmapf" to the kernel command line as a workaround for this issue... Are
there any major issues that would occur as a result of doing that? Currently
this issue is totally filling this server's memory after a few days of uptime.
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
` (4 preceding siblings ...)
2020-06-09 17:09 ` bugzilla-daemon
@ 2020-06-11 1:03 ` bugzilla-daemon
2023-01-24 7:54 ` bugzilla-daemon
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2020-06-11 1:03 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
--- Comment #6 from Paolo Bonzini (bonzini@gnu.org) ---
No particular issue apart from a latency increase if the host is being
overcommitted. It will be fixed in 5.8, but it's a host-side bug so you need
to inform your hosting provider or add no-kvmapf yourself.
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Bug 208081] Memory leak in kvm_async_pf_task_wake
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
` (5 preceding siblings ...)
2020-06-11 1:03 ` bugzilla-daemon
@ 2023-01-24 7:54 ` bugzilla-daemon
6 siblings, 0 replies; 8+ messages in thread
From: bugzilla-daemon @ 2023-01-24 7:54 UTC (permalink / raw)
To: kvm
https://bugzilla.kernel.org/show_bug.cgi?id=208081
BarinaHoppe (BarinaHoppe@proton.me) changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |BarinaHoppe@proton.me
--- Comment #7 from BarinaHoppe (BarinaHoppe@proton.me) ---
If you are having a problem with the Bosch Dishwasher, it can be frustrating in
your day-to-day life. To solve most of the problems in the Bosch Dishwasher, we
can reset the Bosch Dishwasher. Visit - https://appliancebytes.com/
--
You may reply to this email to add a comment.
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-01-24 7:55 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-06 0:35 [Bug 208081] New: Memory leak in kvm_async_pf_task_wake bugzilla-daemon
2020-06-06 0:37 ` [Bug 208081] " bugzilla-daemon
2020-06-08 0:46 ` bugzilla-daemon
2020-06-09 7:17 ` bugzilla-daemon
2020-06-09 13:00 ` bugzilla-daemon
2020-06-09 17:09 ` bugzilla-daemon
2020-06-11 1:03 ` bugzilla-daemon
2023-01-24 7:54 ` bugzilla-daemon
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.