* reading/writing btrfs volume regularly freezes system
@ 2019-04-22 20:37 Nathan Dehnel
2019-04-23 2:38 ` Chris Murphy
` (3 more replies)
0 siblings, 4 replies; 12+ messages in thread
From: Nathan Dehnel @ 2019-04-22 20:37 UTC (permalink / raw)
To: linux-btrfs
I have a raid10 volume that frequently locks up when I try to write to
it or delete things. Any command that touches it will hang (and can't
be killed) and I have to start a new ssh session to get into the
computer again. Nothing fixes it besides a reboot, and the volume will
fail to unmount while the computer is shutting down.
[ 302.360912] sysrq: SysRq : Show Blocked State
[ 302.360951] task PC stack pid father
[ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
[ 302.360993] Call Trace:
[ 302.361007] ? __schedule+0x59d/0x5f1
[ 302.361012] schedule+0x6a/0x85
[ 302.361019] btrfs_commit_transaction+0x219/0x7ac
[ 302.361027] ? wait_woken+0x6d/0x6d
[ 302.361031] transaction_kthread+0xc9/0x135
[ 302.361036] ? btrfs_cleanup_transaction+0x4c7/0x4c7
[ 302.361041] kthread+0x115/0x11d
[ 302.361046] ? kthread_park+0x76/0x76
[ 302.361050] ret_from_fork+0x35/0x40
[ 302.361064] nfsd D 0 2292 2 0x80000000
[ 302.361067] Call Trace:
[ 302.361072] ? __schedule+0x59d/0x5f1
[ 302.361077] schedule+0x6a/0x85
[ 302.361120] wait_current_trans+0x9b/0xd8
[ 302.361126] ? wait_woken+0x6d/0x6d
[ 302.361131] start_transaction+0x1ae/0x38e
[ 302.361135] btrfs_create+0x59/0x1d0
[ 302.361142] vfs_create+0xbf/0xef
[ 302.361160] do_nfsd_create+0x2be/0x41d [nfsd]
[ 302.361214] nfsd4_open+0x223/0x578 [nfsd]
[ 302.361229] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[ 302.361240] nfsd_dispatch+0xb9/0x16e [nfsd]
[ 302.361258] svc_process+0x524/0x6e2 [sunrpc]
[ 302.361270] ? nfsd_destroy+0x5f/0x5f [nfsd]
[ 302.361278] nfsd+0xf9/0x150 [nfsd]
[ 302.361284] kthread+0x115/0x11d
[ 302.361289] ? kthread_park+0x76/0x76
[ 302.361292] ret_from_fork+0x35/0x40
[ 302.361297] nfsd D 0 2293 2 0x80000000
[ 302.361300] Call Trace:
[ 302.361305] ? __schedule+0x59d/0x5f1
[ 302.361309] schedule+0x6a/0x85
[ 302.361314] rwsem_down_write_failed+0x1af/0x210
[ 302.361325] ? nfsd_permission+0xa3/0xe8 [nfsd]
[ 302.361330] call_rwsem_down_write_failed+0x13/0x20
[ 302.361335] down_write+0x20/0x2e
[ 302.361345] nfsd_unlink+0xb1/0x16b [nfsd]
[ 302.361359] nfsd4_remove+0x4e/0x10a [nfsd]
[ 302.361371] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[ 302.361381] nfsd_dispatch+0xb9/0x16e [nfsd]
[ 302.361395] svc_process+0x524/0x6e2 [sunrpc]
[ 302.361401] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
[ 302.361410] ? nfsd_destroy+0x5f/0x5f [nfsd]
[ 302.361419] nfsd+0xf9/0x150 [nfsd]
[ 302.361424] kthread+0x115/0x11d
[ 302.361428] ? kthread_park+0x76/0x76
[ 302.361434] ret_from_fork+0x35/0x40
[ 302.361441] rm D 0 2388 2334 0x00000004
[ 302.361444] Call Trace:
[ 302.361449] ? __schedule+0x59d/0x5f1
[ 302.361453] schedule+0x6a/0x85
[ 302.361457] wait_current_trans+0x9b/0xd8
[ 302.361462] ? wait_woken+0x6d/0x6d
[ 302.361466] start_transaction+0x1ae/0x38e
[ 302.361471] btrfs_start_transaction_fallback_global_rsv+0x32/0x127
[ 302.361475] btrfs_unlink+0x30/0xc0
[ 302.361478] vfs_unlink+0xd2/0x147
[ 302.361482] do_unlinkat+0x112/0x223
[ 302.361488] do_syscall_64+0x7e/0x133
[ 302.361492] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 302.361496] RIP: 0033:0x7f681509b5d7
[ 302.361504] Code: Bad RIP value.
[ 302.361506] RSP: 002b:00007fffb1aed668 EFLAGS: 00000202 ORIG_RAX:
0000000000000107
[ 302.361510] RAX: ffffffffffffffda RBX: 000055672760c6c0 RCX: 00007f681509b5d7
[ 302.361512] RDX: 0000000000000000 RSI: 000055672760b490 RDI: 00000000ffffff9c
[ 302.361514] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000
[ 302.361516] R10: fffffffffffff12b R11: 0000000000000202 R12: 00007fffb1aed848
[ 302.361518] R13: 000055672760b400 R14: 0000000000000002 R15: 0000000000000000
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-22 20:37 reading/writing btrfs volume regularly freezes system Nathan Dehnel
@ 2019-04-23 2:38 ` Chris Murphy
2019-04-24 4:21 ` Nathan Dehnel
2019-04-23 6:51 ` Nikolay Borisov
` (2 subsequent siblings)
3 siblings, 1 reply; 12+ messages in thread
From: Chris Murphy @ 2019-04-23 2:38 UTC (permalink / raw)
To: Nathan Dehnel; +Cc: Btrfs BTRFS
On Mon, Apr 22, 2019 at 2:38 PM Nathan Dehnel <ncdehnel@gmail.com> wrote:
>
> I have a raid10 volume that frequently locks up when I try to write to
> it or delete things. Any command that touches it will hang (and can't
> be killed) and I have to start a new ssh session to get into the
> computer again. Nothing fixes it besides a reboot, and the volume will
> fail to unmount while the computer is shutting down.
>
> [ 302.360912] sysrq: SysRq : Show Blocked State
> [ 302.360951] task PC stack pid father
> [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> [ 302.360993] Call Trace:
> [ 302.361007] ? __schedule+0x59d/0x5f1
> [ 302.361012] schedule+0x6a/0x85
> [ 302.361019] btrfs_commit_transaction+0x219/0x7ac
> [ 302.361027] ? wait_woken+0x6d/0x6d
> [ 302.361031] transaction_kthread+0xc9/0x135
> [ 302.361036] ? btrfs_cleanup_transaction+0x4c7/0x4c7
> [ 302.361041] kthread+0x115/0x11d
> [ 302.361046] ? kthread_park+0x76/0x76
> [ 302.361050] ret_from_fork+0x35/0x40
> [ 302.361064] nfsd D 0 2292 2 0x80000000
> [ 302.361067] Call Trace:
> [ 302.361072] ? __schedule+0x59d/0x5f1
> [ 302.361077] schedule+0x6a/0x85
> [ 302.361120] wait_current_trans+0x9b/0xd8
> [ 302.361126] ? wait_woken+0x6d/0x6d
> [ 302.361131] start_transaction+0x1ae/0x38e
> [ 302.361135] btrfs_create+0x59/0x1d0
> [ 302.361142] vfs_create+0xbf/0xef
> [ 302.361160] do_nfsd_create+0x2be/0x41d [nfsd]
> [ 302.361214] nfsd4_open+0x223/0x578 [nfsd]
> [ 302.361229] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> [ 302.361240] nfsd_dispatch+0xb9/0x16e [nfsd]
> [ 302.361258] svc_process+0x524/0x6e2 [sunrpc]
> [ 302.361270] ? nfsd_destroy+0x5f/0x5f [nfsd]
> [ 302.361278] nfsd+0xf9/0x150 [nfsd]
> [ 302.361284] kthread+0x115/0x11d
> [ 302.361289] ? kthread_park+0x76/0x76
> [ 302.361292] ret_from_fork+0x35/0x40
> [ 302.361297] nfsd D 0 2293 2 0x80000000
> [ 302.361300] Call Trace:
> [ 302.361305] ? __schedule+0x59d/0x5f1
> [ 302.361309] schedule+0x6a/0x85
> [ 302.361314] rwsem_down_write_failed+0x1af/0x210
> [ 302.361325] ? nfsd_permission+0xa3/0xe8 [nfsd]
> [ 302.361330] call_rwsem_down_write_failed+0x13/0x20
> [ 302.361335] down_write+0x20/0x2e
> [ 302.361345] nfsd_unlink+0xb1/0x16b [nfsd]
> [ 302.361359] nfsd4_remove+0x4e/0x10a [nfsd]
> [ 302.361371] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> [ 302.361381] nfsd_dispatch+0xb9/0x16e [nfsd]
> [ 302.361395] svc_process+0x524/0x6e2 [sunrpc]
> [ 302.361401] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
> [ 302.361410] ? nfsd_destroy+0x5f/0x5f [nfsd]
> [ 302.361419] nfsd+0xf9/0x150 [nfsd]
> [ 302.361424] kthread+0x115/0x11d
> [ 302.361428] ? kthread_park+0x76/0x76
> [ 302.361434] ret_from_fork+0x35/0x40
> [ 302.361441] rm D 0 2388 2334 0x00000004
> [ 302.361444] Call Trace:
> [ 302.361449] ? __schedule+0x59d/0x5f1
> [ 302.361453] schedule+0x6a/0x85
> [ 302.361457] wait_current_trans+0x9b/0xd8
> [ 302.361462] ? wait_woken+0x6d/0x6d
> [ 302.361466] start_transaction+0x1ae/0x38e
> [ 302.361471] btrfs_start_transaction_fallback_global_rsv+0x32/0x127
> [ 302.361475] btrfs_unlink+0x30/0xc0
> [ 302.361478] vfs_unlink+0xd2/0x147
> [ 302.361482] do_unlinkat+0x112/0x223
> [ 302.361488] do_syscall_64+0x7e/0x133
> [ 302.361492] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 302.361496] RIP: 0033:0x7f681509b5d7
> [ 302.361504] Code: Bad RIP value.
> [ 302.361506] RSP: 002b:00007fffb1aed668 EFLAGS: 00000202 ORIG_RAX:
> 0000000000000107
> [ 302.361510] RAX: ffffffffffffffda RBX: 000055672760c6c0 RCX: 00007f681509b5d7
> [ 302.361512] RDX: 0000000000000000 RSI: 000055672760b490 RDI: 00000000ffffff9c
> [ 302.361514] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000
> [ 302.361516] R10: fffffffffffff12b R11: 0000000000000202 R12: 00007fffb1aed848
> [ 302.361518] R13: 000055672760b400 R14: 0000000000000002 R15: 0000000000000000
What kernel version? Is it reproducible with something current? i.e.
5.0.6 or ideally 5.1rc6?
And is this actually writes/deletes to NFS as an intermediate to the
Btrfs volume? I can't really tell from the call trace if this is an
issue in nfsd or use case specific problem with NFS on Btrfs. You're
able to directly write/delete with this Btrfs volume?
Since you're getting some information out of the system when this
happens (call trace) I'm wondering if you can issue sysrq+t during the
hang? I find setting up sysrq and writing out the trigger command in a
console (either a tty if you have physical access, or netconsole),
then reproduce the hang, and then hit return on the console with the
pre-typed sysrq command. Sometimes the sysrq output is quite a lot for
the kernel buffer and will overflow dmesg, so you'll either need to
use `log_buf_len=1M` boot parameter, or you can get sysrq output from
journalctl if it's a system system.
--
Chris Murphy
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-22 20:37 reading/writing btrfs volume regularly freezes system Nathan Dehnel
2019-04-23 2:38 ` Chris Murphy
@ 2019-04-23 6:51 ` Nikolay Borisov
2019-04-23 7:29 ` Juergen Sauer
2019-04-24 8:04 ` Qu Wenruo
3 siblings, 0 replies; 12+ messages in thread
From: Nikolay Borisov @ 2019-04-23 6:51 UTC (permalink / raw)
To: Nathan Dehnel, linux-btrfs
On 22.04.19 г. 23:37 ч., Nathan Dehnel wrote:
> I have a raid10 volume that frequently locks up when I try to write to
> it or delete things. Any command that touches it will hang (and can't
> be killed) and I have to start a new ssh session to get into the
> computer again. Nothing fixes it besides a reboot, and the volume will
> fail to unmount while the computer is shutting down.
>
> [ 302.360912] sysrq: SysRq : Show Blocked State
> [ 302.360951] task PC stack pid father
> [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> [ 302.360993] Call Trace:
> [ 302.361007] ? __schedule+0x59d/0x5f1
> [ 302.361012] schedule+0x6a/0x85
> [ 302.361019] btrfs_commit_transaction+0x219/0x7ac
> [ 302.361027] ? wait_woken+0x6d/0x6d
> [ 302.361031] transaction_kthread+0xc9/0x135
> [ 302.361036] ? btrfs_cleanup_transaction+0x4c7/0x4c7
> [ 302.361041] kthread+0x115/0x11d
> [ 302.361046] ? kthread_park+0x76/0x76
> [ 302.361050] ret_from_fork+0x35/0x40
BTRFS is waiting to commit its transaction
> [ 302.361064] nfsd D 0 2292 2 0x80000000
> [ 302.361067] Call Trace:
> [ 302.361072] ? __schedule+0x59d/0x5f1
> [ 302.361077] schedule+0x6a/0x85
> [ 302.361120] wait_current_trans+0x9b/0xd8
> [ 302.361126] ? wait_woken+0x6d/0x6d
> [ 302.361131] start_transaction+0x1ae/0x38e
> [ 302.361135] btrfs_create+0x59/0x1d0
> [ 302.361142] vfs_create+0xbf/0xef
> [ 302.361160] do_nfsd_create+0x2be/0x41d [nfsd]
> [ 302.361214] nfsd4_open+0x223/0x578 [nfsd]
> [ 302.361229] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> [ 302.361240] nfsd_dispatch+0xb9/0x16e [nfsd]
> [ 302.361258] svc_process+0x524/0x6e2 [sunrpc]
> [ 302.361270] ? nfsd_destroy+0x5f/0x5f [nfsd]
> [ 302.361278] nfsd+0xf9/0x150 [nfsd]
> [ 302.361284] kthread+0x115/0x11d
> [ 302.361289] ? kthread_park+0x76/0x76
> [ 302.361292] ret_from_fork+0x35/0x40
Here it seems btrfs is exposed via NFS and a client requested a file to
be created and it's waiting for current transaction to finish.
> [ 302.361297] nfsd D 0 2293 2 0x80000000
> [ 302.361300] Call Trace:
> [ 302.361305] ? __schedule+0x59d/0x5f1
> [ 302.361309] schedule+0x6a/0x85
> [ 302.361314] rwsem_down_write_failed+0x1af/0x210
> [ 302.361325] ? nfsd_permission+0xa3/0xe8 [nfsd]
> [ 302.361330] call_rwsem_down_write_failed+0x13/0x20
> [ 302.361335] down_write+0x20/0x2e
> [ 302.361345] nfsd_unlink+0xb1/0x16b [nfsd]
> [ 302.361359] nfsd4_remove+0x4e/0x10a [nfsd]
> [ 302.361371] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> [ 302.361381] nfsd_dispatch+0xb9/0x16e [nfsd]
> [ 302.361395] svc_process+0x524/0x6e2 [sunrpc]
> [ 302.361401] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
> [ 302.361410] ? nfsd_destroy+0x5f/0x5f [nfsd]
> [ 302.361419] nfsd+0xf9/0x150 [nfsd]
> [ 302.361424] kthread+0x115/0x11d
> [ 302.361428] ? kthread_park+0x76/0x76
> [ 302.361434] ret_from_fork+0x35/0x40
Here NFSD is waiting on a lock of its own, presumably acquired by PID
2292, which in turn is waiting for btrfs pid 2187
> [ 302.361441] rm D 0 2388 2334 0x00000004
> [ 302.361444] Call Trace:
> [ 302.361449] ? __schedule+0x59d/0x5f1
> [ 302.361453] schedule+0x6a/0x85
> [ 302.361457] wait_current_trans+0x9b/0xd8
> [ 302.361462] ? wait_woken+0x6d/0x6d
> [ 302.361466] start_transaction+0x1ae/0x38e
> [ 302.361471] btrfs_start_transaction_fallback_global_rsv+0x32/0x127
> [ 302.361475] btrfs_unlink+0x30/0xc0
> [ 302.361478] vfs_unlink+0xd2/0x147
> [ 302.361482] do_unlinkat+0x112/0x223
> [ 302.361488] do_syscall_64+0x7e/0x133
> [ 302.361492] entry_SYSCALL_64_after_hwframe+0x44/0xa9
This rm is waiting again waiting for btrfs' current transaction to finish.
> [ 302.361496] RIP: 0033:0x7f681509b5d7
> [ 302.361504] Code: Bad RIP value.
> [ 302.361506] RSP: 002b:00007fffb1aed668 EFLAGS: 00000202 ORIG_RAX:
> 0000000000000107
> [ 302.361510] RAX: ffffffffffffffda RBX: 000055672760c6c0 RCX: 00007f681509b5d7
> [ 302.361512] RDX: 0000000000000000 RSI: 000055672760b490 RDI: 00000000ffffff9c
> [ 302.361514] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000
> [ 302.361516] R10: fffffffffffff12b R11: 0000000000000202 R12: 00007fffb1aed848
> [ 302.361518] R13: 000055672760b400 R14: 0000000000000002 R15: 0000000000000000
>
There isn't a lot to be done with the information you have provided. At
the very least:
1. Provide backtrace of all threads on the system via "echo t >
/proc/sysrq-trigger"
2. Provide source code line number of
btrfs_commit_transaction+0x219/0x7ac . This can be done by executing the
./faddr2line[0] vmlinux btrfs_commit_transaction+0x219/0x7ac
3. State your kernel version
Of course you will need the unstripped vmlinux image of your kernel.
[0]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/faddr2line
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-22 20:37 reading/writing btrfs volume regularly freezes system Nathan Dehnel
2019-04-23 2:38 ` Chris Murphy
2019-04-23 6:51 ` Nikolay Borisov
@ 2019-04-23 7:29 ` Juergen Sauer
2019-04-24 4:35 ` Nathan Dehnel
2019-04-24 8:04 ` Qu Wenruo
3 siblings, 1 reply; 12+ messages in thread
From: Juergen Sauer @ 2019-04-23 7:29 UTC (permalink / raw)
To: Nathan Dehnel, linux-btrfs
[-- Attachment #1.1.1: Type: text/plain, Size: 1724 bytes --]
Am 22.04.19 um 22:37 schrieb Nathan Dehnel:
> I have a raid10 volume that frequently locks up when I try to write to
> it or delete things. Any command that touches it will hang (and can't
> be killed) and I have to start a new ssh session to get into the
> computer again. Nothing fixes it besides a reboot, and the volume will
> fail to unmount while the computer is shutting down.
>
> [ 302.360912] sysrq: SysRq : Show Blocked State
> [ 302.360951] task PC stack pid father
> [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> [ 302.360993] Call Trace:
[...]
Nathan,
I presume you are using a kernel before 4.19.8 and after 4.0.9.x.
Your Post does not mention the kernel version, you use.
Could you please post the output of "uname -a" ?
During the rework of the kernel scheduler (multi queue schechduler) by
the kernel team around Linus, there were huge modification done, which
effected nearly all filesystems. Those rewritings caused data loss in
ext4, complete crashes in xfs and so on.
I was hit by this issue on BTRFS, so your description was somewhat a
dejavu for me.
Your can try to use the old mono-queued schedulers in an interim kernel,
or you may upgrade to an recent kernel.
The means disable "kyber" scheduler, wihch is default after 4.18.x or
so, and reactivate cfq, bfq or deadline or noop scheduler. If your
problems are solved it would be a great idea to upgrade the kernel.
We use BTRFS as raid5 (not advised, experimaental), but we use it with
kernel 5.x in arch linux.
Even the multi device code is under heavy development,
BTRFS likes the most recent kernels!
mit freundlichen Grüßen
Jürgen Sauer
[-- Attachment #1.1.2: juergen_sauer.vcf --]
[-- Type: text/x-vcard, Size: 389 bytes --]
begin:vcard
fn;quoted-printable:J=C3=BCrgen Sauer
n;quoted-printable:Sauer;J=C3=BCrgen
org:automatiX GmbH
adr:;;Neue Str. 11;Schwanewede;Niedersachsen;28790;Deutschland
email;internet:juergen.sauer@automatix.de
tel;work:+49 4209 4699
tel;fax:+49 4209 4644
tel;home:+49 4209 4653
tel;cell:+49 162 9699 259
x-mozilla-html:FALSE
url:https://automatix.de
version:2.1
end:vcard
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-23 2:38 ` Chris Murphy
@ 2019-04-24 4:21 ` Nathan Dehnel
0 siblings, 0 replies; 12+ messages in thread
From: Nathan Dehnel @ 2019-04-24 4:21 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
>What kernel version? Is it reproducible with something current? i.e.
5.0.6 or ideally 5.1rc6?
4.19.27-gentoo-r1, haven't tried newer.
>And is this actually writes/deletes to NFS as an intermediate to the
Btrfs volume? I can't really tell from the call trace if this is an
issue in nfsd or use case specific problem with NFS on Btrfs. You're
able to directly write/delete with this Btrfs volume?
This happens when directly writing to the volume locally. The volume
is also being used as an NFS share concurrently though. Killing
nfs-server doesn't seem to have any effect.
>I'm wondering if you can issue sysrq+t during the
hang?
It happens randomly. If it happens again soon I'll try this.
On Mon, Apr 22, 2019 at 9:39 PM Chris Murphy <lists@colorremedies.com> wrote:
>
> On Mon, Apr 22, 2019 at 2:38 PM Nathan Dehnel <ncdehnel@gmail.com> wrote:
> >
> > I have a raid10 volume that frequently locks up when I try to write to
> > it or delete things. Any command that touches it will hang (and can't
> > be killed) and I have to start a new ssh session to get into the
> > computer again. Nothing fixes it besides a reboot, and the volume will
> > fail to unmount while the computer is shutting down.
> >
> > [ 302.360912] sysrq: SysRq : Show Blocked State
> > [ 302.360951] task PC stack pid father
> > [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> > [ 302.360993] Call Trace:
> > [ 302.361007] ? __schedule+0x59d/0x5f1
> > [ 302.361012] schedule+0x6a/0x85
> > [ 302.361019] btrfs_commit_transaction+0x219/0x7ac
> > [ 302.361027] ? wait_woken+0x6d/0x6d
> > [ 302.361031] transaction_kthread+0xc9/0x135
> > [ 302.361036] ? btrfs_cleanup_transaction+0x4c7/0x4c7
> > [ 302.361041] kthread+0x115/0x11d
> > [ 302.361046] ? kthread_park+0x76/0x76
> > [ 302.361050] ret_from_fork+0x35/0x40
> > [ 302.361064] nfsd D 0 2292 2 0x80000000
> > [ 302.361067] Call Trace:
> > [ 302.361072] ? __schedule+0x59d/0x5f1
> > [ 302.361077] schedule+0x6a/0x85
> > [ 302.361120] wait_current_trans+0x9b/0xd8
> > [ 302.361126] ? wait_woken+0x6d/0x6d
> > [ 302.361131] start_transaction+0x1ae/0x38e
> > [ 302.361135] btrfs_create+0x59/0x1d0
> > [ 302.361142] vfs_create+0xbf/0xef
> > [ 302.361160] do_nfsd_create+0x2be/0x41d [nfsd]
> > [ 302.361214] nfsd4_open+0x223/0x578 [nfsd]
> > [ 302.361229] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> > [ 302.361240] nfsd_dispatch+0xb9/0x16e [nfsd]
> > [ 302.361258] svc_process+0x524/0x6e2 [sunrpc]
> > [ 302.361270] ? nfsd_destroy+0x5f/0x5f [nfsd]
> > [ 302.361278] nfsd+0xf9/0x150 [nfsd]
> > [ 302.361284] kthread+0x115/0x11d
> > [ 302.361289] ? kthread_park+0x76/0x76
> > [ 302.361292] ret_from_fork+0x35/0x40
> > [ 302.361297] nfsd D 0 2293 2 0x80000000
> > [ 302.361300] Call Trace:
> > [ 302.361305] ? __schedule+0x59d/0x5f1
> > [ 302.361309] schedule+0x6a/0x85
> > [ 302.361314] rwsem_down_write_failed+0x1af/0x210
> > [ 302.361325] ? nfsd_permission+0xa3/0xe8 [nfsd]
> > [ 302.361330] call_rwsem_down_write_failed+0x13/0x20
> > [ 302.361335] down_write+0x20/0x2e
> > [ 302.361345] nfsd_unlink+0xb1/0x16b [nfsd]
> > [ 302.361359] nfsd4_remove+0x4e/0x10a [nfsd]
> > [ 302.361371] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> > [ 302.361381] nfsd_dispatch+0xb9/0x16e [nfsd]
> > [ 302.361395] svc_process+0x524/0x6e2 [sunrpc]
> > [ 302.361401] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
> > [ 302.361410] ? nfsd_destroy+0x5f/0x5f [nfsd]
> > [ 302.361419] nfsd+0xf9/0x150 [nfsd]
> > [ 302.361424] kthread+0x115/0x11d
> > [ 302.361428] ? kthread_park+0x76/0x76
> > [ 302.361434] ret_from_fork+0x35/0x40
> > [ 302.361441] rm D 0 2388 2334 0x00000004
> > [ 302.361444] Call Trace:
> > [ 302.361449] ? __schedule+0x59d/0x5f1
> > [ 302.361453] schedule+0x6a/0x85
> > [ 302.361457] wait_current_trans+0x9b/0xd8
> > [ 302.361462] ? wait_woken+0x6d/0x6d
> > [ 302.361466] start_transaction+0x1ae/0x38e
> > [ 302.361471] btrfs_start_transaction_fallback_global_rsv+0x32/0x127
> > [ 302.361475] btrfs_unlink+0x30/0xc0
> > [ 302.361478] vfs_unlink+0xd2/0x147
> > [ 302.361482] do_unlinkat+0x112/0x223
> > [ 302.361488] do_syscall_64+0x7e/0x133
> > [ 302.361492] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> > [ 302.361496] RIP: 0033:0x7f681509b5d7
> > [ 302.361504] Code: Bad RIP value.
> > [ 302.361506] RSP: 002b:00007fffb1aed668 EFLAGS: 00000202 ORIG_RAX:
> > 0000000000000107
> > [ 302.361510] RAX: ffffffffffffffda RBX: 000055672760c6c0 RCX: 00007f681509b5d7
> > [ 302.361512] RDX: 0000000000000000 RSI: 000055672760b490 RDI: 00000000ffffff9c
> > [ 302.361514] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000
> > [ 302.361516] R10: fffffffffffff12b R11: 0000000000000202 R12: 00007fffb1aed848
> > [ 302.361518] R13: 000055672760b400 R14: 0000000000000002 R15: 0000000000000000
>
>
> What kernel version? Is it reproducible with something current? i.e.
> 5.0.6 or ideally 5.1rc6?
>
> And is this actually writes/deletes to NFS as an intermediate to the
> Btrfs volume? I can't really tell from the call trace if this is an
> issue in nfsd or use case specific problem with NFS on Btrfs. You're
> able to directly write/delete with this Btrfs volume?
>
> Since you're getting some information out of the system when this
> happens (call trace) I'm wondering if you can issue sysrq+t during the
> hang? I find setting up sysrq and writing out the trigger command in a
> console (either a tty if you have physical access, or netconsole),
> then reproduce the hang, and then hit return on the console with the
> pre-typed sysrq command. Sometimes the sysrq output is quite a lot for
> the kernel buffer and will overflow dmesg, so you'll either need to
> use `log_buf_len=1M` boot parameter, or you can get sysrq output from
> journalctl if it's a system system.
>
>
> --
> Chris Murphy
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-23 7:29 ` Juergen Sauer
@ 2019-04-24 4:35 ` Nathan Dehnel
0 siblings, 0 replies; 12+ messages in thread
From: Nathan Dehnel @ 2019-04-24 4:35 UTC (permalink / raw)
To: Juergen Sauer; +Cc: Btrfs BTRFS
>Could you please post the output of "uname -a" ?
Linux gentooserver 4.19.27-gentoo-r1 #1 SMP PREEMPT Tue Apr 2 01:01:13
CDT 2019 x86_64 Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz GenuineIntel
GNU/Linux
>The means disable "kyber" scheduler, wihch is default after 4.18.x or
so, and reactivate cfq, bfq or deadline or noop scheduler. If your
problems are solved it would be a great idea to upgrade the kernel.
It says CONFIG_DEFAULT_IOSCHED="cfq".
On Tue, Apr 23, 2019 at 2:29 AM Juergen Sauer
<juergen.sauer@automatix.de> wrote:
>
> Am 22.04.19 um 22:37 schrieb Nathan Dehnel:
> > I have a raid10 volume that frequently locks up when I try to write to
> > it or delete things. Any command that touches it will hang (and can't
> > be killed) and I have to start a new ssh session to get into the
> > computer again. Nothing fixes it besides a reboot, and the volume will
> > fail to unmount while the computer is shutting down.
> >
> > [ 302.360912] sysrq: SysRq : Show Blocked State
> > [ 302.360951] task PC stack pid father
> > [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> > [ 302.360993] Call Trace:
> [...]
>
> Nathan,
> I presume you are using a kernel before 4.19.8 and after 4.0.9.x.
> Your Post does not mention the kernel version, you use.
> Could you please post the output of "uname -a" ?
>
> During the rework of the kernel scheduler (multi queue schechduler) by
> the kernel team around Linus, there were huge modification done, which
> effected nearly all filesystems. Those rewritings caused data loss in
> ext4, complete crashes in xfs and so on.
>
> I was hit by this issue on BTRFS, so your description was somewhat a
> dejavu for me.
>
>
> Your can try to use the old mono-queued schedulers in an interim kernel,
> or you may upgrade to an recent kernel.
>
> The means disable "kyber" scheduler, wihch is default after 4.18.x or
> so, and reactivate cfq, bfq or deadline or noop scheduler. If your
> problems are solved it would be a great idea to upgrade the kernel.
>
> We use BTRFS as raid5 (not advised, experimaental), but we use it with
> kernel 5.x in arch linux.
>
> Even the multi device code is under heavy development,
> BTRFS likes the most recent kernels!
>
>
>
> mit freundlichen Grüßen
> Jürgen Sauer
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-22 20:37 reading/writing btrfs volume regularly freezes system Nathan Dehnel
` (2 preceding siblings ...)
2019-04-23 7:29 ` Juergen Sauer
@ 2019-04-24 8:04 ` Qu Wenruo
2019-05-11 10:50 ` Nathan Dehnel
3 siblings, 1 reply; 12+ messages in thread
From: Qu Wenruo @ 2019-04-24 8:04 UTC (permalink / raw)
To: Nathan Dehnel, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 4542 bytes --]
On 2019/4/23 上午4:37, Nathan Dehnel wrote:
> I have a raid10 volume that frequently locks up when I try to write to
> it or delete things. Any command that touches it will hang (and can't
> be killed) and I have to start a new ssh session to get into the
> computer again. Nothing fixes it besides a reboot, and the volume will
> fail to unmount while the computer is shutting down.
>
> [ 302.360912] sysrq: SysRq : Show Blocked State
> [ 302.360951] task PC stack pid father
> [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> [ 302.360993] Call Trace:
> [ 302.361007] ? __schedule+0x59d/0x5f1
> [ 302.361012] schedule+0x6a/0x85
> [ 302.361019] btrfs_commit_transaction+0x219/0x7ac
Btrfs is waiting other transaction to be committed.
At that freeze, would you check which thread is taking CPU time?
Are you utilizing backgroup balance or qgroup?
If you have tons of snapshots, they may hugely slow down qgroup.
Or you just have a running balance with qgroup enabled, it will almost
hang your system.
It's going to be fixed in v5.2. While during 4.19 to v5.1 we have a lot
of optimizations to make balance + qgroup much faster.
Thanks,
Qu
> [ 302.361027] ? wait_woken+0x6d/0x6d
> [ 302.361031] transaction_kthread+0xc9/0x135
> [ 302.361036] ? btrfs_cleanup_transaction+0x4c7/0x4c7
> [ 302.361041] kthread+0x115/0x11d
> [ 302.361046] ? kthread_park+0x76/0x76
> [ 302.361050] ret_from_fork+0x35/0x40
> [ 302.361064] nfsd D 0 2292 2 0x80000000
> [ 302.361067] Call Trace:
> [ 302.361072] ? __schedule+0x59d/0x5f1
> [ 302.361077] schedule+0x6a/0x85
> [ 302.361120] wait_current_trans+0x9b/0xd8
> [ 302.361126] ? wait_woken+0x6d/0x6d
> [ 302.361131] start_transaction+0x1ae/0x38e
> [ 302.361135] btrfs_create+0x59/0x1d0
> [ 302.361142] vfs_create+0xbf/0xef
> [ 302.361160] do_nfsd_create+0x2be/0x41d [nfsd]
> [ 302.361214] nfsd4_open+0x223/0x578 [nfsd]
> [ 302.361229] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> [ 302.361240] nfsd_dispatch+0xb9/0x16e [nfsd]
> [ 302.361258] svc_process+0x524/0x6e2 [sunrpc]
> [ 302.361270] ? nfsd_destroy+0x5f/0x5f [nfsd]
> [ 302.361278] nfsd+0xf9/0x150 [nfsd]
> [ 302.361284] kthread+0x115/0x11d
> [ 302.361289] ? kthread_park+0x76/0x76
> [ 302.361292] ret_from_fork+0x35/0x40
> [ 302.361297] nfsd D 0 2293 2 0x80000000
> [ 302.361300] Call Trace:
> [ 302.361305] ? __schedule+0x59d/0x5f1
> [ 302.361309] schedule+0x6a/0x85
> [ 302.361314] rwsem_down_write_failed+0x1af/0x210
> [ 302.361325] ? nfsd_permission+0xa3/0xe8 [nfsd]
> [ 302.361330] call_rwsem_down_write_failed+0x13/0x20
> [ 302.361335] down_write+0x20/0x2e
> [ 302.361345] nfsd_unlink+0xb1/0x16b [nfsd]
> [ 302.361359] nfsd4_remove+0x4e/0x10a [nfsd]
> [ 302.361371] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> [ 302.361381] nfsd_dispatch+0xb9/0x16e [nfsd]
> [ 302.361395] svc_process+0x524/0x6e2 [sunrpc]
> [ 302.361401] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
> [ 302.361410] ? nfsd_destroy+0x5f/0x5f [nfsd]
> [ 302.361419] nfsd+0xf9/0x150 [nfsd]
> [ 302.361424] kthread+0x115/0x11d
> [ 302.361428] ? kthread_park+0x76/0x76
> [ 302.361434] ret_from_fork+0x35/0x40
> [ 302.361441] rm D 0 2388 2334 0x00000004
> [ 302.361444] Call Trace:
> [ 302.361449] ? __schedule+0x59d/0x5f1
> [ 302.361453] schedule+0x6a/0x85
> [ 302.361457] wait_current_trans+0x9b/0xd8
> [ 302.361462] ? wait_woken+0x6d/0x6d
> [ 302.361466] start_transaction+0x1ae/0x38e
> [ 302.361471] btrfs_start_transaction_fallback_global_rsv+0x32/0x127
> [ 302.361475] btrfs_unlink+0x30/0xc0
> [ 302.361478] vfs_unlink+0xd2/0x147
> [ 302.361482] do_unlinkat+0x112/0x223
> [ 302.361488] do_syscall_64+0x7e/0x133
> [ 302.361492] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 302.361496] RIP: 0033:0x7f681509b5d7
> [ 302.361504] Code: Bad RIP value.
> [ 302.361506] RSP: 002b:00007fffb1aed668 EFLAGS: 00000202 ORIG_RAX:
> 0000000000000107
> [ 302.361510] RAX: ffffffffffffffda RBX: 000055672760c6c0 RCX: 00007f681509b5d7
> [ 302.361512] RDX: 0000000000000000 RSI: 000055672760b490 RDI: 00000000ffffff9c
> [ 302.361514] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000
> [ 302.361516] R10: fffffffffffff12b R11: 0000000000000202 R12: 00007fffb1aed848
> [ 302.361518] R13: 000055672760b400 R14: 0000000000000002 R15: 0000000000000000
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-04-24 8:04 ` Qu Wenruo
@ 2019-05-11 10:50 ` Nathan Dehnel
2019-05-11 11:21 ` Qu Wenruo
0 siblings, 1 reply; 12+ messages in thread
From: Nathan Dehnel @ 2019-05-11 10:50 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Btrfs BTRFS
It happened again, and I managed to get a backtrace.
[362108.290770] ? _raw_spin_unlock+0x12/0x23
[362108.290776] ? nlmsvc_retry_blocked+0x1d3/0x1e3 [lockd]
[362108.290782] ? grace_ender+0xe/0xe [lockd]
[362108.290787] lockd+0xcd/0x129 [lockd]
[362108.290794] kthread+0x115/0x11d
[362108.290798] ? kthread_park+0x76/0x76
[362108.290801] ret_from_fork+0x35/0x40
[362108.290806] nfsd S 0 2307 2 0x80000000
[362108.290809] Call Trace:
[362108.290813] ? __schedule+0x59d/0x5f1
[362108.290816] ? preempt_count_add+0x8a/0x9c
[362108.290820] schedule+0x6a/0x85
[362108.290825] schedule_timeout+0x2f7/0x33c
[362108.290829] ? collect_expired_timers+0x97/0x97
[362108.290842] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.290855] svc_recv+0x291/0x7da [sunrpc]
[362108.290860] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
[362108.290871] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.290881] nfsd+0xe7/0x150 [nfsd]
[362108.290887] kthread+0x115/0x11d
[362108.290891] ? kthread_park+0x76/0x76
[362108.290895] ret_from_fork+0x35/0x40
[362108.290898] nfsd D 0 2308 2 0x80000000
[362108.290901] Call Trace:
[362108.290906] ? __schedule+0x59d/0x5f1
[362108.290910] schedule+0x6a/0x85
[362108.290916] btrfs_tree_read_lock+0xbb/0xf1
[362108.290921] ? wait_woken+0x6d/0x6d
[362108.290924] btrfs_read_lock_root_node+0x1d/0x3b
[362108.290928] btrfs_search_slot+0x181/0x812
[362108.290934] ? crc32c+0x66/0x90
[362108.290938] btrfs_lookup_dir_item+0x7c/0xc4
[362108.290942] btrfs_lookup_dentry+0xae/0x443
[362108.290946] ? preempt_count_add+0x7d/0x9c
[362108.290950] ? d_alloc_parallel+0x44b/0x4a3
[362108.290955] ? btrfs_getattr+0xaa/0xdf
[362108.290958] btrfs_lookup+0xe/0x2c
[362108.290963] __lookup_slow+0xec/0x148
[362108.290967] lookup_one_len+0x61/0x7c
[362108.290978] nfsd_lookup_dentry+0x226/0x31e [nfsd]
[362108.290993] ? write_bytes_to_xdr_buf+0x39/0xc5 [sunrpc]
[362108.291005] nfsd_lookup+0x62/0x123 [nfsd]
[362108.291020] ? nfsd4_encode_operation+0xf0/0x18e [nfsd]
[362108.291033] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.291045] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.291060] svc_process+0x524/0x6e2 [sunrpc]
[362108.291067] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
[362108.291078] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.291087] nfsd+0xf9/0x150 [nfsd]
[362108.291093] kthread+0x115/0x11d
[362108.291097] ? kthread_park+0x76/0x76
[362108.291100] ret_from_fork+0x35/0x40
[362108.291104] nfsd D 0 2309 2 0x80000000
[362108.291107] Call Trace:
[362108.291112] ? __schedule+0x59d/0x5f1
[362108.291117] schedule+0x6a/0x85
[362108.291121] btrfs_tree_read_lock+0xbb/0xf1
[362108.291126] ? wait_woken+0x6d/0x6d
[362108.291129] btrfs_read_lock_root_node+0x1d/0x3b
[362108.291133] btrfs_search_slot+0x181/0x812
[362108.291137] ? crc32c+0x66/0x90
[362108.291140] btrfs_lookup_dir_item+0x7c/0xc4
[362108.291144] btrfs_lookup_dentry+0xae/0x443
[362108.291148] ? preempt_count_add+0x7d/0x9c
[362108.291152] ? d_alloc_parallel+0x44b/0x4a3
[362108.291156] ? btrfs_getattr+0xaa/0xdf
[362108.291159] btrfs_lookup+0xe/0x2c
[362108.291171] __lookup_slow+0xec/0x148
[362108.291175] lookup_one_len+0x61/0x7c
[362108.291186] nfsd_lookup_dentry+0x226/0x31e [nfsd]
[362108.291201] ? write_bytes_to_xdr_buf+0x39/0xc5 [sunrpc]
[362108.291212] nfsd_lookup+0x62/0x123 [nfsd]
[362108.291225] ? nfsd4_encode_operation+0xf0/0x18e [nfsd]
[362108.291238] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.291250] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.291265] svc_process+0x524/0x6e2 [sunrpc]
[362108.291271] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
[362108.291281] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.291291] nfsd+0xf9/0x150 [nfsd]
[362108.291297] kthread+0x115/0x11d
[362108.291301] ? kthread_park+0x76/0x76
[362108.291304] ret_from_fork+0x35/0x40
[362108.291308] nfsd D 0 2310 2 0x80000000
[362108.291311] Call Trace:
[362108.291316] ? __schedule+0x59d/0x5f1
[362108.291320] schedule+0x6a/0x85
[362108.291325] rwsem_down_write_failed+0x1af/0x210
[362108.291330] call_rwsem_down_write_failed+0x13/0x20
[362108.291335] down_write+0x20/0x2e
[362108.291346] nfsd_lookup_dentry+0x209/0x31e [nfsd]
[362108.291360] ? write_bytes_to_xdr_buf+0x39/0xc5 [sunrpc]
[362108.291371] nfsd_lookup+0x62/0x123 [nfsd]
[362108.291385] ? nfsd4_encode_operation+0xf0/0x18e [nfsd]
[362108.291397] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.291409] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.291423] svc_process+0x524/0x6e2 [sunrpc]
[362108.291428] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
[362108.291439] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.291448] nfsd+0xf9/0x150 [nfsd]
[362108.291454] kthread+0x115/0x11d
[362108.291458] ? kthread_park+0x76/0x76
[362108.291461] ret_from_fork+0x35/0x40
[362108.291465] nfsd D 0 2311 2 0x80000000
[362108.291468] Call Trace:
[362108.291473] ? __schedule+0x59d/0x5f1
[362108.291477] schedule+0x6a/0x85
[362108.291482] rwsem_down_write_failed+0x1af/0x210
[362108.291487] call_rwsem_down_write_failed+0x13/0x20
[362108.291492] down_write+0x20/0x2e
[362108.291502] nfsd_lookup_dentry+0x209/0x31e [nfsd]
[362108.291516] ? write_bytes_to_xdr_buf+0x39/0xc5 [sunrpc]
[362108.291527] nfsd_lookup+0x62/0x123 [nfsd]
[362108.291540] ? nfsd4_encode_operation+0xf0/0x18e [nfsd]
[362108.291553] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.291564] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.291578] svc_process+0x524/0x6e2 [sunrpc]
[362108.291590] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.291599] nfsd+0xf9/0x150 [nfsd]
[362108.291605] kthread+0x115/0x11d
[362108.291609] ? kthread_park+0x76/0x76
[362108.291612] ret_from_fork+0x35/0x40
[362108.291616] nfsd D 0 2312 2 0x80000000
[362108.291619] Call Trace:
[362108.291623] ? __schedule+0x59d/0x5f1
[362108.291627] schedule+0x6a/0x85
[362108.291632] rwsem_down_write_failed+0x1af/0x210
[362108.291637] call_rwsem_down_write_failed+0x13/0x20
[362108.291642] down_write+0x20/0x2e
[362108.291652] nfsd_lookup_dentry+0x209/0x31e [nfsd]
[362108.291666] ? write_bytes_to_xdr_buf+0x39/0xc5 [sunrpc]
[362108.291677] nfsd_lookup+0x62/0x123 [nfsd]
[362108.291690] ? nfsd4_encode_operation+0xf0/0x18e [nfsd]
[362108.291702] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.291714] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.291728] svc_process+0x524/0x6e2 [sunrpc]
[362108.291739] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.291749] nfsd+0xf9/0x150 [nfsd]
[362108.291755] kthread+0x115/0x11d
[362108.291759] ? kthread_park+0x76/0x76
[362108.291762] ret_from_fork+0x35/0x40
[362108.291766] nfsd D 0 2313 2 0x80000000
[362108.291769] Call Trace:
[362108.291773] ? __schedule+0x59d/0x5f1
[362108.291777] schedule+0x6a/0x85
[362108.291782] btrfs_tree_read_lock+0xbb/0xf1
[362108.291787] ? wait_woken+0x6d/0x6d
[362108.291790] btrfs_read_lock_root_node+0x1d/0x3b
[362108.291794] btrfs_search_slot+0x181/0x812
[362108.291798] ? crc32c+0x66/0x90
[362108.291801] btrfs_lookup_dir_item+0x7c/0xc4
[362108.291805] btrfs_lookup_dentry+0xae/0x443
[362108.291808] ? preempt_count_add+0x7d/0x9c
[362108.291813] ? d_alloc_parallel+0x44b/0x4a3
[362108.291817] ? btrfs_getattr+0xaa/0xdf
[362108.291820] btrfs_lookup+0xe/0x2c
[362108.291825] __lookup_slow+0xec/0x148
[362108.291829] lookup_one_len+0x61/0x7c
[362108.291840] nfsd_lookup_dentry+0x226/0x31e [nfsd]
[362108.291853] ? write_bytes_to_xdr_buf+0x39/0xc5 [sunrpc]
[362108.291864] nfsd_lookup+0x62/0x123 [nfsd]
[362108.291877] ? nfsd4_encode_operation+0xf0/0x18e [nfsd]
[362108.291890] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.291901] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.291915] svc_process+0x524/0x6e2 [sunrpc]
[362108.291926] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.291936] nfsd+0xf9/0x150 [nfsd]
[362108.291941] kthread+0x115/0x11d
[362108.291946] ? kthread_park+0x76/0x76
[362108.291949] ret_from_fork+0x35/0x40
[362108.291953] nfsd D 0 2314 2 0x80000000
[362108.291955] Call Trace:
[362108.291960] ? __schedule+0x59d/0x5f1
[362108.291964] schedule+0x6a/0x85
[362108.291969] btrfs_tree_read_lock+0xbb/0xf1
[362108.291973] ? wait_woken+0x6d/0x6d
[362108.291978] find_parent_nodes+0x91d/0x12b8
[362108.291985] ? btrfs_find_all_roots_safe+0x9c/0x107
[362108.291988] btrfs_find_all_roots_safe+0x9c/0x107
[362108.291992] btrfs_find_all_roots+0x57/0x75
[362108.291997] btrfs_qgroup_trace_extent_post+0x37/0x7c
[362108.292002] btrfs_add_delayed_tree_ref+0x305/0x32b
[362108.292007] ? btrfs_issue_discard+0x6c/0x11e
[362108.292010] btrfs_inc_extent_ref+0x73/0xe1
[362108.292014] ? add_pinned_bytes+0x38/0x38
[362108.292018] __btrfs_mod_ref+0x1e0/0x20e
[362108.292023] update_ref_for_cow+0x16c/0x28b
[362108.292027] __btrfs_cow_block+0x217/0x4b1
[362108.292031] btrfs_cow_block+0x107/0x17f
[362108.292035] btrfs_search_slot+0x31b/0x812
[362108.292040] btrfs_insert_empty_items+0x53/0x96
[362108.292044] insert_with_overflow+0x44/0xfb
[362108.292048] btrfs_insert_dir_item+0xd8/0x205
[362108.292052] btrfs_add_link+0x11a/0x36f
[362108.292057] ? __btrfs_release_delayed_node+0x165/0x1bb
[362108.292061] btrfs_add_nondir+0x1d/0x28
[362108.292064] btrfs_create+0x154/0x1d0
[362108.292070] vfs_create+0xbf/0xef
[362108.292082] do_nfsd_create+0x2be/0x41d [nfsd]
[362108.292096] nfsd4_open+0x223/0x578 [nfsd]
[362108.292109] nfsd4_proc_compound+0x44a/0x562 [nfsd]
[362108.292121] nfsd_dispatch+0xb9/0x16e [nfsd]
[362108.292134] svc_process+0x524/0x6e2 [sunrpc]
[362108.292146] ? nfsd_destroy+0x5f/0x5f [nfsd]
[362108.292155] nfsd+0xf9/0x150 [nfsd]
[362108.292161] kthread+0x115/0x11d
[362108.292174] ? kthread_park+0x76/0x76
[362108.292177] ret_from_fork+0x35/0x40
[362108.292182] anvil S 0 2347 2296 0x00000100
[362108.292184] Call Trace:
[362108.292189] ? __schedule+0x59d/0x5f1
[362108.292194] schedule+0x6a/0x85
[362108.292197] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.292200] ? _raw_spin_lock_irq+0x14/0x30
[362108.292202] ? _raw_spin_unlock_irq+0x13/0x24
[362108.292207] ? ep_scan_ready_list.constprop.3+0x187/0x1bb
[362108.292212] ep_poll+0x21b/0x31f
[362108.292217] ? wake_up_q+0x4d/0x4d
[362108.292221] do_epoll_wait+0x92/0xb2
[362108.292226] __x64_sys_epoll_wait+0x1a/0x1d
[362108.292229] do_syscall_64+0x7e/0x133
[362108.292233] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292235] RIP: 0033:0x7fa7b34ac7ab
[362108.292240] Code: Bad RIP value.
[362108.292241] RSP: 002b:00007ffe34927d10 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[362108.292245] RAX: ffffffffffffffda RBX: 0000557846d908c0 RCX:
00007fa7b34ac7ab
[362108.292247] RDX: 0000000000000007 RSI: 0000557846d91e90 RDI:
000000000000000c
[362108.292249] RBP: 0000557846d89f50 R08: 00000000ffffffd0 R09:
1999999999999998
[362108.292251] R10: 00000000ffffffff R11: 0000000000000246 R12:
00000000ffffffff
[362108.292253] R13: 0000000000000054 R14: 000000000000000c R15:
0000000000000000
[362108.292256] log S 0 2348 2296 0x00000100
[362108.292260] Call Trace:
[362108.292264] ? __schedule+0x59d/0x5f1
[362108.292269] schedule+0x6a/0x85
[362108.292271] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.292277] ? __seccomp_filter+0x3e/0x1f2
[362108.292281] ep_poll+0x21b/0x31f
[362108.292286] ? wake_up_q+0x4d/0x4d
[362108.292290] do_epoll_wait+0x92/0xb2
[362108.292295] __x64_sys_epoll_wait+0x1a/0x1d
[362108.292298] do_syscall_64+0x7e/0x133
[362108.292302] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292304] RIP: 0033:0x7f32e91647ab
[362108.292308] Code: Bad RIP value.
[362108.292309] RSP: 002b:00007ffdba3b1180 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[362108.292313] RAX: ffffffffffffffda RBX: 0000559d73277da0 RCX:
00007f32e91647ab
[362108.292315] RDX: 000000000000001b RSI: 0000559d73277e10 RDI:
0000000000000023
[362108.292316] RBP: 0000559d73271f30 R08: 0000559d73279a50 R09:
0000559d73279a00
[362108.292318] R10: 00000000ffffffff R11: 0000000000000246 R12:
00000000ffffffff
[362108.292320] R13: 0000000000000144 R14: 000000000000000c R15:
0000000000000000
[362108.292324] config S 0 2349 2296 0x00000100
[362108.292326] Call Trace:
[362108.292331] ? __schedule+0x59d/0x5f1
[362108.292335] schedule+0x6a/0x85
[362108.292338] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.292343] ? __seccomp_filter+0x3e/0x1f2
[362108.292348] ep_poll+0x21b/0x31f
[362108.292351] ? _raw_spin_unlock+0x12/0x23
[362108.292355] ? wake_up_q+0x4d/0x4d
[362108.292359] do_epoll_wait+0x92/0xb2
[362108.292364] __x64_sys_epoll_wait+0x1a/0x1d
[362108.292367] do_syscall_64+0x7e/0x133
[362108.292371] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292373] RIP: 0033:0x7fc472a707ab
[362108.292377] Code: Bad RIP value.
[362108.292378] RSP: 002b:00007ffc28445540 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[362108.292381] RAX: ffffffffffffffda RBX: 000055df0fbc8720 RCX:
00007fc472a707ab
[362108.292383] RDX: 0000000000000004 RSI: 000055df0fb469a0 RDI:
000000000000000a
[362108.292385] RBP: 000055df0fb00ec0 R08: 000055df0fb47530 R09:
000055df0fb474e0
[362108.292386] R10: 00000000ffffffff R11: 0000000000000246 R12:
00000000ffffffff
[362108.292388] R13: 0000000000000030 R14: 000000000000000c R15:
0000000000000000
[362108.292392] clamd S 0 2357 1 0x00000000
[362108.292394] Call Trace:
[362108.292399] ? __schedule+0x59d/0x5f1
[362108.292404] ? hrtimer_try_to_cancel+0x29/0x119
[362108.292408] schedule+0x6a/0x85
[362108.292410] schedule_hrtimeout_range_clock+0xb0/0xf4
[362108.292414] ? hrtimer_init+0xf4/0xf4
[362108.292419] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.292423] do_sys_poll+0x383/0x42b
[362108.292428] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.292433] ? __wake_up_common_lock+0x8b/0xd0
[362108.292440] ? merge_state+0x95/0x10a
[362108.292442] ? preempt_count_add+0x8a/0x9c
[362108.292445] ? _raw_spin_lock+0x13/0x2f
[362108.292447] ? _raw_spin_unlock+0x12/0x23
[362108.292451] ? block_rsv_release_bytes+0x96/0x2a2
[362108.292456] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.292460] ? btrfs_buffered_write+0x554/0x5ff
[362108.292464] ? preempt_count_add+0x8a/0x9c
[362108.292466] ? _raw_spin_lock+0x13/0x2f
[362108.292469] ? _raw_spin_unlock+0x12/0x23
[362108.292472] ? btrfs_file_write_iter+0x419/0x49c
[362108.292476] ? _copy_to_user+0x22/0x28
[362108.292480] ? read_null+0x8/0x8
[362108.292484] ? timespec64_add_safe+0x36/0x82
[362108.292488] __se_sys_poll+0x5a/0xd6
[362108.292492] do_syscall_64+0x7e/0x133
[362108.292495] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292497] RIP: 0033:0x7f305a4080a3
[362108.292502] Code: Bad RIP value.
[362108.292503] RSP: 002b:00007ffc419b38a0 EFLAGS: 00000293 ORIG_RAX:
0000000000000007
[362108.292506] RAX: ffffffffffffffda RBX: ffffffffffffffff RCX:
00007f305a4080a3
[362108.292508] RDX: 00000000000927c0 RSI: 0000000000000001 RDI:
000055dd3d819a80
[362108.292510] RBP: 00007ffc419b39d0 R08: 0000000000000000 R09:
00007ffc419b2b40
[362108.292511] R10: 00007f305a497e80 R11: 0000000000000293 R12:
000055dd3bb4d350
[362108.292513] R13: 00007ffc419b4540 R14: 0000000000000000 R15:
0000000000000000
[362108.292516] clamd S 0 2358 1 0x00000000
[362108.292519] Call Trace:
[362108.292524] ? __schedule+0x59d/0x5f1
[362108.292529] schedule+0x6a/0x85
[362108.292531] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.292534] ? preempt_count_add+0x8a/0x9c
[362108.292537] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.292539] ? preempt_count_add+0x8a/0x9c
[362108.292542] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.292546] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.292550] do_sys_poll+0x383/0x42b
[362108.292558] ? intel_pstate_update_util+0x20/0x3b4
[362108.292562] ? __update_load_avg_cfs_rq+0xf1/0x182
[362108.292566] ? enqueue_entity+0x535/0x684
[362108.292569] ? select_task_rq_fair+0xb33/0xb45
[362108.292573] ? enqueue_task_fair+0xc2/0x5fd
[362108.292577] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.292581] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.292585] ? pollwake+0x5b/0x75
[362108.292588] ? wake_up_q+0x4d/0x4d
[362108.292592] ? __wake_up_common+0xaa/0x126
[362108.292595] ? preempt_count_add+0x7d/0x9c
[362108.292600] ? __mnt_drop_write+0x25/0x37
[362108.292603] ? file_update_time+0xd6/0xfa
[362108.292606] ? __sb_end_write+0x4b/0x5d
[362108.292610] ? pipe_write+0x357/0x369
[362108.292616] ? vfs_write+0x121/0x16b
[362108.292620] __se_sys_poll+0x5a/0xd6
[362108.292623] do_syscall_64+0x7e/0x133
[362108.292627] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292629] RIP: 0033:0x7f305a4080a3
[362108.292633] Code: Bad RIP value.
[362108.292634] RSP: 002b:00007f302c21e8d0 EFLAGS: 00000293 ORIG_RAX:
0000000000000007
[362108.292637] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f305a4080a3
[362108.292639] RDX: 00000000ffffffff RSI: 0000000000000002 RDI:
00007f3024000b20
[362108.292640] RBP: 00007f302c21ea00 R08: 0000000000000000 R09:
00007f30240008d0
[362108.292642] R10: 00007f3024000080 R11: 0000000000000293 R12:
00007ffc419b395e
[362108.292644] R13: 00007ffc419b395f R14: 0000000000802000 R15:
00007ffc419b3960
[362108.292647] /usr/sbin/amavi S 0 2359 1 0x00000100
[362108.292650] Call Trace:
[362108.292655] ? __schedule+0x59d/0x5f1
[362108.292659] ? hrtimer_try_to_cancel+0x29/0x119
[362108.292663] schedule+0x6a/0x85
[362108.292665] schedule_hrtimeout_range_clock+0xb0/0xf4
[362108.292683] ? hrtimer_init+0xf4/0xf4
[362108.292684] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.292685] do_select+0x5f1/0x643
[362108.292688] ? __switch_to_asm+0x34/0x70
[362108.292688] ? __switch_to_asm+0x40/0x70
[362108.292689] ? __switch_to_asm+0x34/0x70
[362108.292690] ? __switch_to_asm+0x40/0x70
[362108.292691] ? __switch_to_asm+0x34/0x70
[362108.292692] ? __switch_to_asm+0x40/0x70
[362108.292693] ? __switch_to_asm+0x34/0x70
[362108.292694] ? __switch_to_asm+0x40/0x70
[362108.292694] ? __switch_to_asm+0x34/0x70
[362108.292696] ? __switch_to_asm+0x40/0x70
[362108.292696] ? __switch_to_asm+0x34/0x70
[362108.292697] ? __switch_to_asm+0x40/0x70
[362108.292698] ? __switch_to_asm+0x34/0x70
[362108.292699] ? __switch_to_asm+0x40/0x70
[362108.292700] ? __switch_to_asm+0x34/0x70
[362108.292701] ? __switch_to_asm+0x40/0x70
[362108.292702] ? __switch_to_asm+0x34/0x70
[362108.292703] ? __switch_to_asm+0x40/0x70
[362108.292704] ? __switch_to_asm+0x34/0x70
[362108.292704] ? __switch_to_asm+0x40/0x70
[362108.292705] ? _raw_spin_unlock_irq+0x13/0x24
[362108.292706] ? finish_task_switch+0x181/0x269
[362108.292708] ? __schedule+0x5a5/0x5f1
[362108.292709] ? __alloc_pages_nodemask+0x14c/0xca7
[362108.292710] ? preempt_schedule_irq+0x59/0x7e
[362108.292712] ? apic_timer_interrupt+0xa/0x20
[362108.292714] ? ___bpf_prog_run+0x15/0xe22
[362108.292715] ? ___bpf_prog_run+0xe19/0xe22
[362108.292717] ? __bpf_prog_run32+0x39/0x59
[362108.292718] core_sys_select+0x196/0x248
[362108.292720] ? seccomp_run_filters+0xf5/0x134
[362108.292722] ? apic_timer_interrupt+0xa/0x20
[362108.292723] ? __seccomp_filter+0x3e/0x1f2
[362108.292724] ? group_send_sig_info+0x31/0x59
[362108.292725] ? kill_pid_info+0x51/0x69
[362108.292726] ? timespec64_add_safe+0x36/0x82
[362108.292728] kern_select+0x97/0xcf
[362108.292729] __x64_sys_select+0x20/0x23
[362108.292730] do_syscall_64+0x7e/0x133
[362108.292731] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292732] RIP: 0033:0x7f4cbcf53e9b
[362108.292734] Code: Bad RIP value.
[362108.292734] RSP: 002b:00007ffffa950a40 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[362108.292735] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f4cbcf53e9b
[362108.292736] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
0000000000000000
[362108.292737] RBP: 000055e3abb31580 R08: 00007ffffa950ac0 R09:
0000000000000080
[362108.292737] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffffa950af8
[362108.292738] R13: 00007ffffa950ad0 R14: 0000000000000000 R15:
0000000000000000
[362108.292740] /usr/sbin/amavi S 0 2361 2359 0x00000100
[362108.292741] Call Trace:
[362108.292743] ? __schedule+0x59d/0x5f1
[362108.292744] schedule+0x6a/0x85
[362108.292745] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.292746] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.292748] ? add_wait_queue+0x15/0x3e
[362108.292749] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.292750] ? tcp_poll+0x3a/0x1fe
[362108.292752] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.292753] do_select+0x5f1/0x643
[362108.292756] ? uncompress_inline+0x13a/0x13a
[362108.292758] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.292759] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.292760] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.292761] ? __alloc_pages_nodemask+0x14c/0xca7
[362108.292763] ? mem_cgroup_event_ratelimit.isra.10+0x36/0x86
[362108.292765] ? memcg_check_events+0x2a/0x172
[362108.292767] ? mem_cgroup_commit_charge+0xaa/0x10d
[362108.292768] ? ___bpf_prog_run+0xe19/0xe22
[362108.292769] ? __bpf_prog_run32+0x39/0x59
[362108.292770] ? _raw_spin_unlock+0x12/0x23
[362108.292771] core_sys_select+0x196/0x248
[362108.292774] ? __seccomp_filter+0x3e/0x1f2
[362108.292775] ? handle_mm_fault+0x176/0x1c7
[362108.292777] kern_select+0x97/0xcf
[362108.292778] __x64_sys_select+0x20/0x23
[362108.292779] do_syscall_64+0x7e/0x133
[362108.292781] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292781] RIP: 0033:0x7f4cbcf53e9b
[362108.292783] Code: Bad RIP value.
[362108.292783] RSP: 002b:00007ffffa950a40 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[362108.292784] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f4cbcf53e9b
[362108.292785] RDX: 0000000000000000 RSI: 000055e3ad3b5530 RDI:
0000000000000008
[362108.292785] RBP: 000055e3abb31580 R08: 0000000000000000 R09:
0000000000000080
[362108.292786] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffffa950af8
[362108.292787] R13: 00007ffffa950ad0 R14: 0000000000000001 R15:
000055e3ad3b5530
[362108.292788] /usr/sbin/amavi S 0 2362 2359 0x00000100
[362108.292789] Call Trace:
[362108.292791] ? __schedule+0x59d/0x5f1
[362108.292793] schedule+0x6a/0x85
[362108.292795] locks_lock_inode_wait+0xf6/0x124
[362108.292797] ? wait_woken+0x6d/0x6d
[362108.292798] __se_sys_flock+0x10f/0x136
[362108.292799] do_syscall_64+0x7e/0x133
[362108.292800] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292801] RIP: 0033:0x7f4cbcf4cf67
[362108.292803] Code: Bad RIP value.
[362108.292803] RSP: 002b:00007ffffa950b08 EFLAGS: 00000202 ORIG_RAX:
0000000000000049
[362108.292804] RAX: ffffffffffffffda RBX: 000055e3abb31570 RCX:
00007f4cbcf4cf67
[362108.292805] RDX: 000055e3a803c8e0 RSI: 0000000000000002 RDI:
0000000000000007
[362108.292806] RBP: 000055e3a900e748 R08: 0000000000000000 R09:
000055e3ad505e30
[362108.292806] R10: 0000000000000000 R11: 0000000000000202 R12:
00007f4cbd615b60
[362108.292807] R13: 00007f4cbd614f78 R14: 000055e3abb31568 R15:
0000000000000002
[362108.292808] apache2 S 0 2371 2251 0x00000100
[362108.292810] Call Trace:
[362108.292811] ? __schedule+0x59d/0x5f1
[362108.292813] ? __radix_tree_lookup+0x6a/0xa3
[362108.292814] schedule+0x6a/0x85
[362108.292816] do_semtimedop+0x7d0/0x975
[362108.292818] ? inet_recvmsg+0xb6/0xe5
[362108.292820] ? seccomp_run_filters+0xf5/0x134
[362108.292822] ? __inode_wait_for_writeback+0x76/0xc9
[362108.292824] ? __seccomp_filter+0x3e/0x1f2
[362108.292825] ? __vfs_read+0x101/0x139
[362108.292827] ? syscall_trace_enter+0x16c/0x216
[362108.292828] do_syscall_64+0x7e/0x133
[362108.292829] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292830] RIP: 0033:0x7f9082fa171b
[362108.292831] Code: Bad RIP value.
[362108.292832] RSP: 002b:00007fff57fd1f90 EFLAGS: 00000246 ORIG_RAX:
0000000000000041
[362108.292833] RAX: ffffffffffffffda RBX: 0000559c133a9a98 RCX:
00007f9082fa171b
[362108.292833] RDX: 0000000000000001 RSI: 00007f9083efed6c RDI:
0000000000010000
[362108.292834] RBP: 00007f9083efed6c R08: 0000559c135536d8 R09:
0000000000000000
[362108.292835] R10: 0000559c135544c8 R11: 0000000000000246 R12:
00007fff57fd2020
[362108.292835] R13: 0000559c13556180 R14: 0000559c132ec848 R15:
0000000000000000
[362108.292836] apache2 S 0 2724 2251 0x00000100
[362108.292837] Call Trace:
[362108.292839] ? __schedule+0x59d/0x5f1
[362108.292841] ? __radix_tree_lookup+0x6a/0xa3
[362108.292842] schedule+0x6a/0x85
[362108.292844] do_semtimedop+0x7d0/0x975
[362108.292846] ? inet_recvmsg+0xb6/0xe5
[362108.292847] ? seccomp_run_filters+0xf5/0x134
[362108.292849] ? __inode_wait_for_writeback+0x76/0xc9
[362108.292851] ? __seccomp_filter+0x3e/0x1f2
[362108.292852] ? __vfs_read+0x101/0x139
[362108.292853] ? syscall_trace_enter+0x16c/0x216
[362108.292855] do_syscall_64+0x7e/0x133
[362108.292856] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292857] RIP: 0033:0x7f9082fa171b
[362108.292858] Code: Bad RIP value.
[362108.292859] RSP: 002b:00007fff57fd1f90 EFLAGS: 00000246 ORIG_RAX:
0000000000000041
[362108.292860] RAX: ffffffffffffffda RBX: 0000559c133a9a98 RCX:
00007f9082fa171b
[362108.292860] RDX: 0000000000000001 RSI: 00007f9083efed6c RDI:
0000000000010000
[362108.292861] RBP: 00007f9083efed6c R08: 0000559c135536d8 R09:
0000000000000000
[362108.292861] R10: 0000559c135544c8 R11: 0000000000000246 R12:
00007fff57fd2020
[362108.292862] R13: 0000559c13556180 R14: 0000559c132ec848 R15:
0000000000000000
[362108.292864] apache2 S 0 2725 2251 0x00000100
[362108.292865] Call Trace:
[362108.292867] ? __schedule+0x59d/0x5f1
[362108.292868] ? __radix_tree_lookup+0x6a/0xa3
[362108.292869] schedule+0x6a/0x85
[362108.292871] do_semtimedop+0x7d0/0x975
[362108.292873] ? inet_recvmsg+0xb6/0xe5
[362108.292874] ? seccomp_run_filters+0xf5/0x134
[362108.292876] ? __inode_wait_for_writeback+0x76/0xc9
[362108.292878] ? __seccomp_filter+0x3e/0x1f2
[362108.292879] ? __vfs_read+0x101/0x139
[362108.292881] ? syscall_trace_enter+0x16c/0x216
[362108.292882] do_syscall_64+0x7e/0x133
[362108.292883] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292884] RIP: 0033:0x7f9082fa171b
[362108.292885] Code: Bad RIP value.
[362108.292886] RSP: 002b:00007fff57fd1f90 EFLAGS: 00000246 ORIG_RAX:
0000000000000041
[362108.292887] RAX: ffffffffffffffda RBX: 0000559c133a9a98 RCX:
00007f9082fa171b
[362108.292887] RDX: 0000000000000001 RSI: 00007f9083efed6c RDI:
0000000000010000
[362108.292888] RBP: 00007f9083efed6c R08: 0000559c135536d8 R09:
0000000000000000
[362108.292888] R10: 0000559c135544c8 R11: 0000000000000246 R12:
00007fff57fd2020
[362108.292889] R13: 0000559c13556180 R14: 0000559c132ec848 R15:
0000000000000000
[362108.292890] apache2 S 0 2726 2251 0x00000100
[362108.292891] Call Trace:
[362108.292894] ? __schedule+0x59d/0x5f1
[362108.292895] ? __radix_tree_lookup+0x6a/0xa3
[362108.292896] schedule+0x6a/0x85
[362108.292898] do_semtimedop+0x7d0/0x975
[362108.292900] ? inet_recvmsg+0xb6/0xe5
[362108.292902] ? seccomp_run_filters+0xf5/0x134
[362108.292903] ? __inode_wait_for_writeback+0x76/0xc9
[362108.292905] ? __seccomp_filter+0x3e/0x1f2
[362108.292906] ? __vfs_read+0x101/0x139
[362108.292908] ? syscall_trace_enter+0x16c/0x216
[362108.292909] do_syscall_64+0x7e/0x133
[362108.292910] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.292911] RIP: 0033:0x7f9082fa171b
[362108.292912] Code: Bad RIP value.
[362108.292913] RSP: 002b:00007fff57fd1f90 EFLAGS: 00000246 ORIG_RAX:
0000000000000041
[362108.292914] RAX: ffffffffffffffda RBX: 0000559c133a9a98 RCX:
00007f9082fa171b
[362108.292914] RDX: 0000000000000001 RSI: 00007f9083efed6c RDI:
0000000000010000
[362108.292915] RBP: 00007f9083efed6c R08: 0000559c135536d8 R09:
0000000000000000
[362108.292916] R10: 0000559c135544c8 R11: 0000000000000246 R12:
00007fff57fd2020
[362108.292917] R13: 0000559c13556180 R14: 0000559c132ec848 R15:
0000000000000000
[362108.292918] kworker/7:2 I 0 1161 2 0x80000000
[362108.292921] Workqueue: (null) (events_freezable_power_)
[362108.292922] Call Trace:
[362108.292924] ? __schedule+0x59d/0x5f1
[362108.292926] ? cancel_delayed_work_sync+0xf/0xf
[362108.292927] schedule+0x6a/0x85
[362108.292929] worker_thread+0x24e/0x297
[362108.292930] kthread+0x115/0x11d
[362108.292932] ? kthread_park+0x76/0x76
[362108.292933] ret_from_fork+0x35/0x40
[362108.292935] kworker/6:1 I 0 3373 2 0x80000000
[362108.292938] Workqueue: (null) (mm_percpu_wq)
[362108.292939] Call Trace:
[362108.292940] ? __schedule+0x59d/0x5f1
[362108.292941] ? cancel_delayed_work_sync+0xf/0xf
[362108.292942] schedule+0x6a/0x85
[362108.292944] worker_thread+0x24e/0x297
[362108.292945] kthread+0x115/0x11d
[362108.292947] ? kthread_park+0x76/0x76
[362108.292948] ret_from_fork+0x35/0x40
[362108.292949] kworker/1:2 I 0 832 2 0x80000000
[362108.292952] Workqueue: (null) (mm_percpu_wq)
[362108.292953] Call Trace:
[362108.292955] ? __schedule+0x59d/0x5f1
[362108.292956] ? cancel_delayed_work_sync+0xf/0xf
[362108.292957] schedule+0x6a/0x85
[362108.292958] worker_thread+0x24e/0x297
[362108.292960] kthread+0x115/0x11d
[362108.292961] ? kthread_park+0x76/0x76
[362108.292962] ret_from_fork+0x35/0x40
[362108.292964] kworker/0:2 I 0 3008 2 0x80000000
[362108.292967] Workqueue: (null) (events_power_efficient)
[362108.292968] Call Trace:
[362108.292970] ? __schedule+0x59d/0x5f1
[362108.292971] ? cancel_delayed_work_sync+0xf/0xf
[362108.292972] schedule+0x6a/0x85
[362108.292973] worker_thread+0x24e/0x297
[362108.292975] kthread+0x115/0x11d
[362108.292976] ? kthread_park+0x76/0x76
[362108.292977] ret_from_fork+0x35/0x40
[362108.292979] kworker/6:2 I 0 7996 2 0x80000000
[362108.292982] Workqueue: (null) (cgroup_destroy)
[362108.292983] Call Trace:
[362108.292985] ? __schedule+0x59d/0x5f1
[362108.292986] ? cancel_delayed_work_sync+0xf/0xf
[362108.292988] schedule+0x6a/0x85
[362108.292989] worker_thread+0x24e/0x297
[362108.292990] kthread+0x115/0x11d
[362108.292992] ? kthread_park+0x76/0x76
[362108.292993] ret_from_fork+0x35/0x40
[362108.292995] kworker/u17:0 I 0 9115 2 0x80000000
[362108.292998] Workqueue: (null) (btrfs-worker-high)
[362108.292999] Call Trace:
[362108.293001] ? __schedule+0x59d/0x5f1
[362108.293002] ? cancel_delayed_work_sync+0xf/0xf
[362108.293004] schedule+0x6a/0x85
[362108.293005] worker_thread+0x24e/0x297
[362108.293006] kthread+0x115/0x11d
[362108.293008] ? kthread_park+0x76/0x76
[362108.293009] ret_from_fork+0x35/0x40
[362108.293010] kworker/4:2 I 0 12354 2 0x80000000
[362108.293013] Workqueue: (null) (events_freezable_power_)
[362108.293015] Call Trace:
[362108.293016] ? __schedule+0x59d/0x5f1
[362108.293017] ? cancel_delayed_work_sync+0xf/0xf
[362108.293019] schedule+0x6a/0x85
[362108.293020] worker_thread+0x24e/0x297
[362108.293022] kthread+0x115/0x11d
[362108.293023] ? kthread_park+0x76/0x76
[362108.293024] ret_from_fork+0x35/0x40
[362108.293026] kworker/3:2 I 0 13247 2 0x80000000
[362108.293029] Workqueue: (null) (events_freezable_power_)
[362108.293029] Call Trace:
[362108.293031] ? __schedule+0x59d/0x5f1
[362108.293032] ? cancel_delayed_work_sync+0xf/0xf
[362108.293034] schedule+0x6a/0x85
[362108.293035] worker_thread+0x24e/0x297
[362108.293036] kthread+0x115/0x11d
[362108.293038] ? kthread_park+0x76/0x76
[362108.293039] ret_from_fork+0x35/0x40
[362108.293041] kworker/1:0 I 0 16200 2 0x80000000
[362108.293044] Workqueue: (null) (cgroup_destroy)
[362108.293045] Call Trace:
[362108.293047] ? __schedule+0x59d/0x5f1
[362108.293048] ? cancel_delayed_work_sync+0xf/0xf
[362108.293049] schedule+0x6a/0x85
[362108.293050] worker_thread+0x24e/0x297
[362108.293052] kthread+0x115/0x11d
[362108.293053] ? kthread_park+0x76/0x76
[362108.293054] ret_from_fork+0x35/0x40
[362108.293056] systemd-helper S 0 18170 1 0x00000000
[362108.293058] Call Trace:
[362108.293059] ? __schedule+0x59d/0x5f1
[362108.293061] schedule+0x6a/0x85
[362108.293062] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293063] ? preempt_count_add+0x8a/0x9c
[362108.293064] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.293066] ? add_wait_queue+0x15/0x3e
[362108.293066] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293068] ? unix_poll+0x87/0x96
[362108.293070] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.293071] do_sys_poll+0x383/0x42b
[362108.293073] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293074] ? ep_poll_callback+0x188/0x1fd
[362108.293077] ? __wake_up_common+0xaa/0x126
[362108.293078] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293079] ? __wake_up_common_lock+0x8b/0xd0
[362108.293081] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293083] ? ___sys_sendmsg+0x206/0x230
[362108.293085] ? mem_cgroup_commit_charge+0xaa/0x10d
[362108.293086] ? __lru_cache_add+0x74/0x87
[362108.293088] ? _raw_spin_unlock+0x12/0x23
[362108.293088] ? __handle_mm_fault+0x955/0xeb4
[362108.293090] ? __sys_sendmsg+0x5e/0x94
[362108.293092] __se_sys_poll+0x5a/0xd6
[362108.293094] do_syscall_64+0x7e/0x133
[362108.293095] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293095] RIP: 0033:0x7f9f90195058
[362108.293097] Code: Bad RIP value.
[362108.293097] RSP: 002b:00007ffe1d98fa40 EFLAGS: 00000246 ORIG_RAX:
0000000000000007
[362108.293099] RAX: ffffffffffffffda RBX: 0000564ba5378420 RCX:
00007f9f90195058
[362108.293099] RDX: 00000000ffffffff RSI: 0000000000000001 RDI:
00007ffe1d98fa90
[362108.293100] RBP: 0000000000000006 R08: 0000000000000000 R09:
00007ffe1d9bd080
[362108.293100] R10: 0000000011f5d60a R11: 0000000000000246 R12:
0000000000000006
[362108.293101] R13: 0000000000000004 R14: 00000000ffffffff R15:
00007ffe1d98fa90
[362108.293103] snapperd S 0 18174 1 0x00000000
[362108.293104] Call Trace:
[362108.293106] ? __schedule+0x59d/0x5f1
[362108.293107] schedule+0x6a/0x85
[362108.293109] futex_wait_queue_me+0xc1/0x106
[362108.293110] futex_wait+0xdb/0x1ea
[362108.293111] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293112] ? try_to_wake_up+0x34f/0x375
[362108.293113] ? get_futex_key+0x94/0x2c2
[362108.293115] do_futex+0xdb/0x904
[362108.293117] __se_sys_futex+0x13e/0x163
[362108.293119] do_syscall_64+0x7e/0x133
[362108.293120] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293121] RIP: 0033:0x7ff6797acd7c
[362108.293123] Code: Bad RIP value.
[362108.293123] RSP: 002b:00007ffc9c00b700 EFLAGS: 00000246 ORIG_RAX:
00000000000000ca
[362108.293124] RAX: ffffffffffffffda RBX: 000055ff308f7bd0 RCX:
00007ff6797acd7c
[362108.293125] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000055ff308f7bf8
[362108.293125] RBP: 0000000000000000 R08: 0000000000000000 R09:
000055ff321deb00
[362108.293126] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[362108.293127] R13: 000055ff308f7ba8 R14: 0000000000000000 R15:
000055ff308f7bf8
[362108.293128] snapperd D 0 18178 1 0x00000000
[362108.293129] Call Trace:
[362108.293131] ? __schedule+0x59d/0x5f1
[362108.293133] ? terminate_walk+0x20/0x7e
[362108.293134] schedule+0x6a/0x85
[362108.293136] btrfs_tree_read_lock+0xbb/0xf1
[362108.293138] ? wait_woken+0x6d/0x6d
[362108.293140] btrfs_read_lock_root_node+0x1d/0x3b
[362108.293141] btrfs_search_slot+0x181/0x812
[362108.293142] ? cp_new_stat+0x150/0x17f
[362108.293144] btrfs_listxattr+0x9b/0x25b
[362108.293147] ? __se_sys_newfstat+0x41/0x64
[362108.293148] listxattr+0x5d/0xb4
[362108.293149] __se_sys_flistxattr+0x3e/0x5a
[362108.293151] do_syscall_64+0x7e/0x133
[362108.293152] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293152] RIP: 0033:0x7ff678598fe7
[362108.293154] Code: Bad RIP value.
[362108.293154] RSP: 002b:00007ff675f3db98 EFLAGS: 00000202 ORIG_RAX:
00000000000000c4
[362108.293155] RAX: ffffffffffffffda RBX: 00007ff675f3e2c8 RCX:
00007ff678598fe7
[362108.293156] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
0000000000000006
[362108.293156] RBP: 00007ff675f3dfc8 R08: 0000000000000000 R09:
0000000000000400
[362108.293157] R10: 0000000000000000 R11: 0000000000000202 R12:
00007ff675f3e2c8
[362108.293158] R13: 00007ff670007d28 R14: 00007ff670007d30 R15:
00007ff67000bdb0
[362108.293159] kworker/2:2 I 0 20398 2 0x80000000
[362108.293166] Workqueue: (null) (events_freezable_power_)
[362108.293167] Call Trace:
[362108.293169] ? __schedule+0x59d/0x5f1
[362108.293171] ? cancel_delayed_work_sync+0xf/0xf
[362108.293172] schedule+0x6a/0x85
[362108.293173] worker_thread+0x24e/0x297
[362108.293174] kthread+0x115/0x11d
[362108.293176] ? kthread_park+0x76/0x76
[362108.293177] ret_from_fork+0x35/0x40
[362108.293179] kworker/u17:1 I 0 20546 2 0x80000000
[362108.293182] Workqueue: (null) (btrfs-worker-high)
[362108.293183] Call Trace:
[362108.293185] ? __schedule+0x59d/0x5f1
[362108.293187] ? cancel_delayed_work_sync+0xf/0xf
[362108.293188] schedule+0x6a/0x85
[362108.293189] worker_thread+0x24e/0x297
[362108.293190] kthread+0x115/0x11d
[362108.293192] ? kthread_park+0x76/0x76
[362108.293193] ret_from_fork+0x35/0x40
[362108.293195] kworker/u16:0 I 0 20920 2 0x80000000
[362108.293198] Workqueue: (null) (bond1)
[362108.293199] Call Trace:
[362108.293200] ? __schedule+0x59d/0x5f1
[362108.293202] ? cancel_delayed_work_sync+0xf/0xf
[362108.293203] schedule+0x6a/0x85
[362108.293204] worker_thread+0x24e/0x297
[362108.293205] kthread+0x115/0x11d
[362108.293207] ? kthread_park+0x76/0x76
[362108.293208] ret_from_fork+0x35/0x40
[362108.293209] kworker/u16:2 I 0 20995 2 0x80000000
[362108.293212] Workqueue: (null) (events_unbound)
[362108.293213] Call Trace:
[362108.293214] ? __schedule+0x59d/0x5f1
[362108.293216] ? cancel_delayed_work_sync+0xf/0xf
[362108.293217] schedule+0x6a/0x85
[362108.293218] worker_thread+0x24e/0x297
[362108.293220] kthread+0x115/0x11d
[362108.293221] ? kthread_park+0x76/0x76
[362108.293222] ret_from_fork+0x35/0x40
[362108.293224] sshd S 0 20997 2260 0x00000000
[362108.293225] Call Trace:
[362108.293227] ? __schedule+0x59d/0x5f1
[362108.293229] schedule+0x6a/0x85
[362108.293230] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293231] ? preempt_count_add+0x8a/0x9c
[362108.293232] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.293233] ? add_wait_queue+0x15/0x3e
[362108.293235] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293236] ? unix_poll+0x87/0x96
[362108.293237] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.293238] do_sys_poll+0x383/0x42b
[362108.293241] ? __kmalloc_reserve.isra.7+0x2d/0x6f
[362108.293242] ? __alloc_skb+0xb3/0x192
[362108.293244] ? __follow_mount_rcu+0x5b/0xc5
[362108.293245] ? lookup_fast+0xff/0x297
[362108.293247] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293248] ? cdev_put+0x1e/0x1e
[362108.293249] ? dput.part.4+0x29/0xfd
[362108.293251] ? mntput_no_expire+0x11/0x185
[362108.293253] ? terminate_walk+0x20/0x7e
[362108.293254] ? path_openat+0xb78/0xc1b
[362108.293255] ? do_filp_open+0x88/0xae
[362108.293256] ? sock_write_iter+0x83/0xae
[362108.293258] ? preempt_count_add+0x7d/0x9c
[362108.293259] ? __fd_install+0xa5/0xbe
[362108.293260] __se_sys_poll+0x5a/0xd6
[362108.293262] ? filp_close+0x62/0x69
[362108.293263] do_syscall_64+0x7e/0x133
[362108.293264] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293265] RIP: 0033:0x7efddf36b058
[362108.293266] Code: Bad RIP value.
[362108.293267] RSP: 002b:00007ffe28b572f0 EFLAGS: 00000246 ORIG_RAX:
0000000000000007
[362108.293268] RAX: ffffffffffffffda RBX: 00005642588e01a0 RCX:
00007efddf36b058
[362108.293269] RDX: 00000000ffffffff RSI: 0000000000000001 RDI:
00007ffe28b57338
[362108.293270] RBP: 00007ffe28b57338 R08: 00005642590868b0 R09:
0000564259086870
[362108.293270] R10: 0000000000000000 R11: 0000000000000246 R12:
0000564259079880
[362108.293271] R13: 0000000000000000 R14: 0000000000005207 R15:
0000000000000014
[362108.293272] kworker/u16:5 I 0 20998 2 0x80000000
[362108.293275] Workqueue: (null) (btrfs-endio-write)
[362108.293276] Call Trace:
[362108.293278] ? __schedule+0x59d/0x5f1
[362108.293279] ? cancel_delayed_work_sync+0xf/0xf
[362108.293280] schedule+0x6a/0x85
[362108.293281] worker_thread+0x24e/0x297
[362108.293283] kthread+0x115/0x11d
[362108.293285] ? kthread_park+0x76/0x76
[362108.293286] ret_from_fork+0x35/0x40
[362108.293287] systemd S 0 21001 1 0x00000000
[362108.293289] Call Trace:
[362108.293290] ? __schedule+0x59d/0x5f1
[362108.293292] schedule+0x6a/0x85
[362108.293293] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293294] ? _raw_spin_lock_irq+0x14/0x30
[362108.293295] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293296] ? ep_scan_ready_list.constprop.3+0x187/0x1bb
[362108.293298] ep_poll+0x21b/0x31f
[362108.293300] ? wake_up_q+0x4d/0x4d
[362108.293302] do_epoll_wait+0x92/0xb2
[362108.293303] __x64_sys_epoll_wait+0x1a/0x1d
[362108.293304] do_syscall_64+0x7e/0x133
[362108.293306] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293306] RIP: 0033:0x7f7439e817ab
[362108.293308] Code: Bad RIP value.
[362108.293308] RSP: 002b:00007ffc3b1119f0 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[362108.293309] RAX: ffffffffffffffda RBX: 00005569bced2490 RCX:
00007f7439e817ab
[362108.293310] RDX: 0000000000000010 RSI: 00007ffc3b111a40 RDI:
0000000000000004
[362108.293311] RBP: 00007ffc3b111bf0 R08: 0000000000000010 R09:
00007ffc3b119080
[362108.293311] R10: 00000000ffffffff R11: 0000000000000246 R12:
00007ffc3b111a40
[362108.293312] R13: 0000000000000001 R14: ffffffffffffffff R15:
0000000000000000
[362108.293313] (sd-pam) S 0 21002 21001 0x00000000
[362108.293314] Call Trace:
[362108.293317] ? __schedule+0x59d/0x5f1
[362108.293318] schedule+0x6a/0x85
[362108.293319] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293320] ? __handle_mm_fault+0xbc3/0xeb4
[362108.293324] ? recalc_sigpending+0x17/0x42
[362108.293325] ? dequeue_signal+0x177/0x19d
[362108.293327] do_sigtimedwait.isra.7+0x13e/0x1f5
[362108.293329] __se_sys_rt_sigtimedwait+0x84/0xcb
[362108.293331] ? __do_page_fault+0x36d/0x3ff
[362108.293332] do_syscall_64+0x7e/0x133
[362108.293334] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293335] RIP: 0033:0x7fa707a9f72b
[362108.293336] Code: Bad RIP value.
[362108.293337] RSP: 002b:00007ffc1ffe6820 EFLAGS: 00000246 ORIG_RAX:
0000000000000080
[362108.293337] RAX: ffffffffffffffda RBX: 00007ffc1ffe6801 RCX:
00007fa707a9f72b
[362108.293338] RDX: 0000000000000000 RSI: 00007ffc1ffe6860 RDI:
00007ffc1ffe69f8
[362108.293339] RBP: 00007ffc1ffe6860 R08: 0000000000000000 R09:
0000562ff7984700
[362108.293340] R10: 0000000000000008 R11: 0000000000000246 R12:
00007ffc1ffe6958
[362108.293340] R13: 0000000000005209 R14: 00007ffc1ffe6958 R15:
0000000000008000
[362108.293342] kworker/5:2 I 0 21006 2 0x80000000
[362108.293345] Workqueue: (null) (cgroup_destroy)
[362108.293346] Call Trace:
[362108.293347] ? __schedule+0x59d/0x5f1
[362108.293349] ? cancel_delayed_work_sync+0xf/0xf
[362108.293350] schedule+0x6a/0x85
[362108.293351] worker_thread+0x24e/0x297
[362108.293353] kthread+0x115/0x11d
[362108.293354] ? kthread_park+0x76/0x76
[362108.293355] ret_from_fork+0x35/0x40
[362108.293357] sshd S 0 21007 20997 0x00000000
[362108.293358] Call Trace:
[362108.293360] ? __schedule+0x59d/0x5f1
[362108.293361] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293362] schedule+0x6a/0x85
[362108.293363] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293364] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.293367] ? pty_stop+0x70/0x70
[362108.293368] ? n_tty_poll+0x17c/0x18d
[362108.293370] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.293371] do_select+0x5f1/0x643
[362108.293374] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293375] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293377] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293378] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293379] ? preempt_count_add+0x8a/0x9c
[362108.293381] ? get_nohz_timer_target+0x1b/0xea
[362108.293382] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293383] ? mod_timer+0x287/0x2af
[362108.293384] ? sk_reset_timer+0x14/0x27
[362108.293386] ? tcp_schedule_loss_probe+0x102/0x10a
[362108.293388] ? tcp_write_xmit+0x9be/0xcd8
[362108.293390] ? _copy_from_iter_full+0x1dc/0x1ed
[362108.293391] ? __tcp_push_pending_frames+0x30/0x95
[362108.293392] core_sys_select+0x196/0x248
[362108.293395] ? sock_write_iter+0x83/0xae
[362108.293397] kern_select+0x97/0xcf
[362108.293398] ? _copy_to_user+0x22/0x28
[362108.293400] ? put_timespec64+0x3c/0x61
[362108.293401] __x64_sys_select+0x20/0x23
[362108.293402] do_syscall_64+0x7e/0x133
[362108.293404] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293404] RIP: 0033:0x7efddf36de9b
[362108.293406] Code: Bad RIP value.
[362108.293407] RSP: 002b:00007ffe28b57260 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[362108.293408] RAX: ffffffffffffffda RBX: 000056425907b4d0 RCX:
00007efddf36de9b
[362108.293408] RDX: 00005642590797b0 RSI: 0000564259074e00 RDI:
000000000000000d
[362108.293409] RBP: 0000000000000000 R08: 0000000000000000 R09:
00007ffe28bb1080
[362108.293409] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[362108.293410] R13: 0000000000000003 R14: 0000000000000003 R15:
00005642588e0a20
[362108.293412] bash S 0 21008 21007 0x00000000
[362108.293413] Call Trace:
[362108.293415] ? __schedule+0x59d/0x5f1
[362108.293416] ? __tty_check_change+0xdd/0x118
[362108.293418] ? __radix_tree_lookup+0x6a/0xa3
[362108.293419] schedule+0x6a/0x85
[362108.293420] do_wait+0x1d5/0x237
[362108.293422] kernel_wait4+0xbe/0x111
[362108.293424] ? task_stopped_code+0x3a/0x3a
[362108.293425] __se_sys_wait4+0x3c/0x8b
[362108.293427] ? __do_page_fault+0x36d/0x3ff
[362108.293427] ? preempt_count_add+0x8a/0x9c
[362108.293429] ? recalc_sigpending+0x17/0x42
[362108.293430] ? __set_current_blocked+0x3d/0x55
[362108.293431] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293432] ? sigprocmask+0x6e/0x8a
[362108.293433] ? _copy_to_user+0x22/0x28
[362108.293434] ? __se_sys_rt_sigprocmask+0x93/0xbb
[362108.293435] do_syscall_64+0x7e/0x133
[362108.293436] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293437] RIP: 0033:0x7f86d8cba21b
[362108.293439] Code: Bad RIP value.
[362108.293439] RSP: 002b:00007fff7ec6caf0 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[362108.293441] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f86d8cba21b
[362108.293441] RDX: 000000000000000a RSI: 00007fff7ec6cb30 RDI:
00000000ffffffff
[362108.293442] RBP: 000000000000000a R08: 0000000000000000 R09:
00005651dfba3280
[362108.293442] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[362108.293443] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000001
[362108.293445] su S 0 21065 21008 0x00000000
[362108.293446] Call Trace:
[362108.293448] ? __schedule+0x59d/0x5f1
[362108.293449] schedule+0x6a/0x85
[362108.293450] do_wait+0x1d5/0x237
[362108.293452] kernel_wait4+0xbe/0x111
[362108.293453] ? task_stopped_code+0x3a/0x3a
[362108.293454] __se_sys_wait4+0x3c/0x8b
[362108.293455] ? _raw_spin_lock_irq+0x14/0x30
[362108.293456] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293457] ? do_sigaction+0xb6/0x18c
[362108.293458] ? preempt_count_add+0x8a/0x9c
[362108.293460] ? recalc_sigpending+0x17/0x42
[362108.293461] ? __set_current_blocked+0x3d/0x55
[362108.293462] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293462] ? sigprocmask+0x6e/0x8a
[362108.293464] ? __se_sys_rt_sigprocmask+0x72/0xbb
[362108.293465] do_syscall_64+0x7e/0x133
[362108.293466] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293467] RIP: 0033:0x7f09e1fd721b
[362108.293468] Code: Bad RIP value.
[362108.293468] RSP: 002b:00007ffdfa21c640 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[362108.293469] RAX: ffffffffffffffda RBX: 00007ffdfa21c690 RCX:
00007f09e1fd721b
[362108.293470] RDX: 0000000000000002 RSI: 00007ffdfa21c68c RDI:
00000000ffffffff
[362108.293471] RBP: 00007ffdfa21c68c R08: 0000000000000000 R09:
00007f09e28f4b80
[362108.293471] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffdfa21c908
[362108.293472] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[362108.293473] bash S 0 21066 21065 0x00000000
[362108.293474] Call Trace:
[362108.293477] ? __schedule+0x59d/0x5f1
[362108.293478] ? preempt_count_add+0x8a/0x9c
[362108.293479] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.293480] schedule+0x6a/0x85
[362108.293481] do_wait+0x1d5/0x237
[362108.293482] kernel_wait4+0xbe/0x111
[362108.293484] ? task_stopped_code+0x3a/0x3a
[362108.293485] __se_sys_wait4+0x3c/0x8b
[362108.293486] ? __do_page_fault+0x36d/0x3ff
[362108.293487] ? preempt_count_add+0x8a/0x9c
[362108.293488] ? recalc_sigpending+0x17/0x42
[362108.293489] ? __set_current_blocked+0x3d/0x55
[362108.293490] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293491] ? sigprocmask+0x6e/0x8a
[362108.293492] ? _copy_to_user+0x22/0x28
[362108.293494] ? __se_sys_rt_sigprocmask+0x93/0xbb
[362108.293495] do_syscall_64+0x7e/0x133
[362108.293496] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293496] RIP: 0033:0x7f7e4cb5821b
[362108.293498] Code: Bad RIP value.
[362108.293499] RSP: 002b:00007ffc82f3aec0 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[362108.293500] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f7e4cb5821b
[362108.293500] RDX: 000000000000000a RSI: 00007ffc82f3af00 RDI:
00000000ffffffff
[362108.293501] RBP: 000000000000000a R08: 0000000000000000 R09:
0000558b380e7230
[362108.293501] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[362108.293502] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[362108.293503] ls D 0 21079 21066 0x00000000
[362108.293504] Call Trace:
[362108.293506] ? __schedule+0x59d/0x5f1
[362108.293508] schedule+0x6a/0x85
[362108.293510] rwsem_down_read_failed_killable+0x17c/0x1d9
[362108.293511] ? _raw_spin_unlock+0x12/0x23
[362108.293513] call_rwsem_down_read_failed_killable+0x14/0x30
[362108.293515] down_read_killable+0x13/0x36
[362108.293516] iterate_dir+0x59/0x131
[362108.293518] ksys_getdents64+0x8f/0x104
[362108.293520] ? filldir+0xef/0xef
[362108.293521] __x64_sys_getdents64+0x16/0x19
[362108.293522] do_syscall_64+0x7e/0x133
[362108.293523] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293524] RIP: 0033:0x7f4d00d1dadb
[362108.293525] Code: Bad RIP value.
[362108.293526] RSP: 002b:00007ffc7a066140 EFLAGS: 00000246 ORIG_RAX:
00000000000000d9
[362108.293527] RAX: ffffffffffffffda RBX: 000055766371c130 RCX:
00007f4d00d1dadb
[362108.293528] RDX: 0000000000008000 RSI: 000055766371c160 RDI:
0000000000000003
[362108.293528] RBP: 000055766371c160 R08: 0000000000000003 R09:
0000557663714010
[362108.293529] R10: ffffffffffffff6d R11: 0000000000000246 R12:
ffffffffffffff80
[362108.293530] R13: 0000000000000000 R14: 0000000000000000 R15:
0000000000000000
[362108.293531] kworker/3:1 I 0 21087 2 0x80000000
[362108.293534] Workqueue: (null) (events)
[362108.293535] Call Trace:
[362108.293537] ? __schedule+0x59d/0x5f1
[362108.293538] ? cancel_delayed_work_sync+0xf/0xf
[362108.293539] schedule+0x6a/0x85
[362108.293540] worker_thread+0x24e/0x297
[362108.293542] kthread+0x115/0x11d
[362108.293544] ? kthread_park+0x76/0x76
[362108.293545] ret_from_fork+0x35/0x40
[362108.293546] kworker/2:1 I 0 21156 2 0x80000000
[362108.293549] Workqueue: (null) (mm_percpu_wq)
[362108.293550] Call Trace:
[362108.293552] ? __schedule+0x59d/0x5f1
[362108.293553] ? cancel_delayed_work_sync+0xf/0xf
[362108.293554] schedule+0x6a/0x85
[362108.293555] worker_thread+0x24e/0x297
[362108.293557] kthread+0x115/0x11d
[362108.293558] ? kthread_park+0x76/0x76
[362108.293559] ret_from_fork+0x35/0x40
[362108.293561] kworker/4:0 I 0 21230 2 0x80000000
[362108.293564] Workqueue: (null) (mm_percpu_wq)
[362108.293565] Call Trace:
[362108.293567] ? __schedule+0x59d/0x5f1
[362108.293568] ? cancel_delayed_work_sync+0xf/0xf
[362108.293569] schedule+0x6a/0x85
[362108.293570] worker_thread+0x24e/0x297
[362108.293572] kthread+0x115/0x11d
[362108.293573] ? kthread_park+0x76/0x76
[362108.293574] ret_from_fork+0x35/0x40
[362108.293576] kworker/0:0 I 0 21237 2 0x80000000
[362108.293579] Workqueue: (null) (events_power_efficient)
[362108.293580] Call Trace:
[362108.293582] ? __schedule+0x59d/0x5f1
[362108.293583] ? cancel_delayed_work_sync+0xf/0xf
[362108.293584] schedule+0x6a/0x85
[362108.293586] worker_thread+0x24e/0x297
[362108.293588] kthread+0x115/0x11d
[362108.293589] ? kthread_park+0x76/0x76
[362108.293590] ret_from_fork+0x35/0x40
[362108.293592] kworker/2:0 I 0 21346 2 0x80000000
[362108.293595] Workqueue: (null) (mm_percpu_wq)
[362108.293596] Call Trace:
[362108.293597] ? __schedule+0x59d/0x5f1
[362108.293599] ? cancel_delayed_work_sync+0xf/0xf
[362108.293600] schedule+0x6a/0x85
[362108.293601] worker_thread+0x24e/0x297
[362108.293603] kthread+0x115/0x11d
[362108.293604] ? kthread_park+0x76/0x76
[362108.293605] ret_from_fork+0x35/0x40
[362108.293607] kworker/3:0 I 0 21347 2 0x80000000
[362108.293610] Workqueue: (null) (mm_percpu_wq)
[362108.293611] Call Trace:
[362108.293612] ? __schedule+0x59d/0x5f1
[362108.293613] ? cancel_delayed_work_sync+0xf/0xf
[362108.293615] schedule+0x6a/0x85
[362108.293616] worker_thread+0x24e/0x297
[362108.293618] kthread+0x115/0x11d
[362108.293619] ? kthread_park+0x76/0x76
[362108.293620] ret_from_fork+0x35/0x40
[362108.293622] kworker/u17:2 I 0 21361 2 0x80000000
[362108.293625] Workqueue: (null) (btrfs-worker-high)
[362108.293626] Call Trace:
[362108.293628] ? __schedule+0x59d/0x5f1
[362108.293629] ? cancel_delayed_work_sync+0xf/0xf
[362108.293630] schedule+0x6a/0x85
[362108.293631] worker_thread+0x24e/0x297
[362108.293633] kthread+0x115/0x11d
[362108.293635] ? kthread_park+0x76/0x76
[362108.293636] ret_from_fork+0x35/0x40
[362108.293638] sshd S 0 21421 2260 0x00000000
[362108.293639] Call Trace:
[362108.293641] ? __schedule+0x59d/0x5f1
[362108.293642] schedule+0x6a/0x85
[362108.293643] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293645] ? preempt_count_add+0x8a/0x9c
[362108.293645] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.293647] ? add_wait_queue+0x15/0x3e
[362108.293648] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293649] ? unix_poll+0x87/0x96
[362108.293650] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.293652] do_sys_poll+0x383/0x42b
[362108.293654] ? __kmalloc_reserve.isra.7+0x2d/0x6f
[362108.293655] ? __alloc_skb+0xb3/0x192
[362108.293657] ? __follow_mount_rcu+0x5b/0xc5
[362108.293659] ? lookup_fast+0xff/0x297
[362108.293660] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293661] ? cdev_put+0x1e/0x1e
[362108.293663] ? dput.part.4+0x29/0xfd
[362108.293664] ? mntput_no_expire+0x11/0x185
[362108.293665] ? terminate_walk+0x20/0x7e
[362108.293666] ? path_openat+0xb78/0xc1b
[362108.293668] ? do_filp_open+0x88/0xae
[362108.293669] ? sock_write_iter+0x83/0xae
[362108.293671] ? preempt_count_add+0x7d/0x9c
[362108.293672] ? __fd_install+0xa5/0xbe
[362108.293673] __se_sys_poll+0x5a/0xd6
[362108.293674] ? filp_close+0x62/0x69
[362108.293676] do_syscall_64+0x7e/0x133
[362108.293677] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293678] RIP: 0033:0x7f4bff3cf058
[362108.293680] Code: Bad RIP value.
[362108.293680] RSP: 002b:00007fff1cb24cb0 EFLAGS: 00000246 ORIG_RAX:
0000000000000007
[362108.293681] RAX: ffffffffffffffda RBX: 00005575e56bc1a0 RCX:
00007f4bff3cf058
[362108.293682] RDX: 00000000ffffffff RSI: 0000000000000001 RDI:
00007fff1cb24cf8
[362108.293682] RBP: 00007fff1cb24cf8 R08: 00005575e7659900 R09:
00005575e76598c0
[362108.293683] R10: 0000000000000000 R11: 0000000000000246 R12:
00005575e764f620
[362108.293683] R13: 0000000000000000 R14: 00000000000053ae R15:
0000000000000014
[362108.293685] sshd S 0 21423 21421 0x00000000
[362108.293686] Call Trace:
[362108.293688] ? __schedule+0x59d/0x5f1
[362108.293689] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293690] schedule+0x6a/0x85
[362108.293691] schedule_hrtimeout_range_clock+0x48/0xf4
[362108.293692] ? _raw_spin_lock_irqsave+0x17/0x36
[362108.293694] ? pty_stop+0x70/0x70
[362108.293695] ? n_tty_poll+0x17c/0x18d
[362108.293696] poll_schedule_timeout.constprop.3+0x43/0x5c
[362108.293697] do_select+0x5f1/0x643
[362108.293700] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293701] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293702] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293704] ? compat_poll_select_copy_remaining+0x11d/0x11d
[362108.293705] ? preempt_count_add+0x8a/0x9c
[362108.293706] ? get_nohz_timer_target+0x1b/0xea
[362108.293708] ? internal_add_timer+0x27/0x34
[362108.293708] ? _raw_spin_unlock_irqrestore+0x14/0x25
[362108.293709] ? mod_timer+0x287/0x2af
[362108.293711] ? sk_reset_timer+0x14/0x27
[362108.293712] ? tcp_schedule_loss_probe+0x102/0x10a
[362108.293713] ? tcp_write_xmit+0x9be/0xcd8
[362108.293714] ? _copy_from_iter_full+0x1dc/0x1ed
[362108.293715] ? __tcp_push_pending_frames+0x30/0x95
[362108.293717] core_sys_select+0x196/0x248
[362108.293718] ? sock_write_iter+0x83/0xae
[362108.293720] kern_select+0x97/0xcf
[362108.293722] ? _copy_to_user+0x22/0x28
[362108.293724] ? put_timespec64+0x3c/0x61
[362108.293725] __x64_sys_select+0x20/0x23
[362108.293726] do_syscall_64+0x7e/0x133
[362108.293727] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293728] RIP: 0033:0x7f4bff3d1e9b
[362108.293730] Code: Bad RIP value.
[362108.293730] RSP: 002b:00007fff1cb24c20 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[362108.293731] RAX: ffffffffffffffda RBX: 00005575e764e770 RCX:
00007f4bff3d1e9b
[362108.293732] RDX: 00005575e764c9b0 RSI: 00005575e7648110 RDI:
000000000000000d
[362108.293732] RBP: 0000000000000000 R08: 0000000000000000 R09:
00007fff1cbc8080
[362108.293733] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[362108.293734] R13: 0000000000000003 R14: 0000000000000003 R15:
00005575e56bca20
[362108.293735] bash S 0 21424 21423 0x00000000
[362108.293736] Call Trace:
[362108.293738] ? __schedule+0x59d/0x5f1
[362108.293739] ? __tty_check_change+0xdd/0x118
[362108.293741] ? __radix_tree_lookup+0x6a/0xa3
[362108.293742] schedule+0x6a/0x85
[362108.293743] do_wait+0x1d5/0x237
[362108.293744] kernel_wait4+0xbe/0x111
[362108.293746] ? task_stopped_code+0x3a/0x3a
[362108.293747] __se_sys_wait4+0x3c/0x8b
[362108.293748] ? __do_page_fault+0x36d/0x3ff
[362108.293749] ? preempt_count_add+0x8a/0x9c
[362108.293751] ? recalc_sigpending+0x17/0x42
[362108.293752] ? __set_current_blocked+0x3d/0x55
[362108.293753] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293753] ? sigprocmask+0x6e/0x8a
[362108.293755] ? _copy_to_user+0x22/0x28
[362108.293756] ? __se_sys_rt_sigprocmask+0x93/0xbb
[362108.293757] do_syscall_64+0x7e/0x133
[362108.293758] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293759] RIP: 0033:0x7f116010d21b
[362108.293760] Code: Bad RIP value.
[362108.293761] RSP: 002b:00007fff0dc8e940 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[362108.293762] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f116010d21b
[362108.293762] RDX: 000000000000000a RSI: 00007fff0dc8e980 RDI:
00000000ffffffff
[362108.293763] RBP: 000000000000000a R08: 0000000000000000 R09:
00005577f1e973f0
[362108.293764] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[362108.293765] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[362108.293766] su S 0 21427 21424 0x00000000
[362108.293767] Call Trace:
[362108.293769] ? __schedule+0x59d/0x5f1
[362108.293770] schedule+0x6a/0x85
[362108.293771] do_wait+0x1d5/0x237
[362108.293773] kernel_wait4+0xbe/0x111
[362108.293774] ? task_stopped_code+0x3a/0x3a
[362108.293775] __se_sys_wait4+0x3c/0x8b
[362108.293776] ? _raw_spin_lock_irq+0x14/0x30
[362108.293777] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293778] ? do_sigaction+0xb6/0x18c
[362108.293779] ? preempt_count_add+0x8a/0x9c
[362108.293780] ? recalc_sigpending+0x17/0x42
[362108.293782] ? __set_current_blocked+0x3d/0x55
[362108.293783] ? _raw_spin_unlock_irq+0x13/0x24
[362108.293783] ? sigprocmask+0x6e/0x8a
[362108.293784] ? __se_sys_rt_sigprocmask+0x72/0xbb
[362108.293785] do_syscall_64+0x7e/0x133
[362108.293786] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293787] RIP: 0033:0x7f9c3cd9721b
[362108.293789] Code: Bad RIP value.
[362108.293789] RSP: 002b:00007ffea71ad750 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[362108.293790] RAX: ffffffffffffffda RBX: 00007ffea71ad7a0 RCX:
00007f9c3cd9721b
[362108.293791] RDX: 0000000000000002 RSI: 00007ffea71ad79c RDI:
00000000ffffffff
[362108.293791] RBP: 00007ffea71ad79c R08: 0000000000000000 R09:
00007f9c3d6b4b80
[362108.293792] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffea71ada18
[362108.293793] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[362108.293794] bash R running task 0 21434 21427 0x80000000
[362108.293795] Call Trace:
[362108.293797] sched_show_task+0xea/0x11c
[362108.293799] show_state_filter+0x91/0xa5
[362108.293802] sysrq_handle_showstate+0xc/0x11
[362108.293804] __handle_sysrq+0x86/0x107
[362108.293806] write_sysrq_trigger+0x2b/0x30
[362108.293808] proc_reg_write+0x3e/0x5c
[362108.293809] ? proc_reg_poll+0x55/0x55
[362108.293811] __vfs_write+0x33/0x141
[362108.293812] ? handle_mm_fault+0x176/0x1c7
[362108.293814] vfs_write+0xc8/0x16b
[362108.293816] ksys_write+0x5d/0xab
[362108.293817] do_syscall_64+0x7e/0x133
[362108.293819] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[362108.293819] RIP: 0033:0x7f6ba3c63a08
[362108.293820] Code: 00 90 48 83 ec 38 64 48 8b 04 25 28 00 00 00 48
89 44 24 28 31 c0 48 8d 05 d5 5c 2d 00 8b 00 85 c0 75 27 b8 01 00 00
00 0f 05 <48> 3d 00 f0 ff ff 77 60 48 8b 4c 24 28 64 48 33 0c 25 28 00
00 00
[362108.293821] RSP: 002b:00007ffc3305df70 EFLAGS: 00000246 ORIG_RAX:
0000000000000001
[362108.293822] RAX: ffffffffffffffda RBX: 0000000000000002 RCX:
00007f6ba3c63a08
[362108.293822] RDX: 0000000000000002 RSI: 0000563a722ce800 RDI:
0000000000000001
[362108.293823] RBP: 0000563a722ce800 R08: 000000000000000a R09:
00007f6ba438e740
[362108.293824] R10: fffffffffffffd8e R11: 0000000000000246 R12:
00007f6ba3f35760
[362108.293824] R13: 0000000000000002 R14: 00007f6ba3f30760 R15:
0000000000000002
[362108.293826] kworker/0:1 I 0 21445 2 0x80000000
[362108.293829] Workqueue: (null) (events_power_efficient)
[362108.293830] Call Trace:
[362108.293832] ? __schedule+0x59d/0x5f1
[362108.293833] ? cancel_delayed_work_sync+0xf/0xf
[362108.293834] schedule+0x6a/0x85
[362108.293836] worker_thread+0x24e/0x297
[362108.293838] kthread+0x115/0x11d
[362108.293839] ? kthread_park+0x76/0x76
[362108.293840] ret_from_fork+0x35/0x40
[362108.293842] kworker/4:1 I 0 21452 2 0x80000000
[362108.293845] Workqueue: (null) (mm_percpu_wq)
[362108.293846] Call Trace:
[362108.293848] ? __schedule+0x59d/0x5f1
[362108.293849] ? cancel_delayed_work_sync+0xf/0xf
[362108.293850] schedule+0x6a/0x85
[362108.293851] worker_thread+0x24e/0x297
[362108.293853] kthread+0x115/0x11d
[362108.293855] ? kthread_park+0x76/0x76
[362108.293856] ret_from_fork+0x35/0x40
[362108.293858] kworker/u16:1 I 0 21454 2 0x80000000
[362108.293861] Workqueue: (null) (btrfs-endio-write)
[362108.293861] Call Trace:
[362108.293863] ? __schedule+0x59d/0x5f1
[362108.293865] ? cancel_delayed_work_sync+0xf/0xf
[362108.293866] schedule+0x6a/0x85
[362108.293867] worker_thread+0x24e/0x297
[362108.293869] kthread+0x115/0x11d
[362108.293870] ? kthread_park+0x76/0x76
[362108.293871] ret_from_fork+0x35/0x40
[362108.293873] kworker/u16:3 I 0 21455 2 0x80000000
[362108.293876] Workqueue: (null) (btrfs-endio-write)
[362108.293877] Call Trace:
[362108.293879] ? __schedule+0x59d/0x5f1
[362108.293880] ? cancel_delayed_work_sync+0xf/0xf
[362108.293881] schedule+0x6a/0x85
[362108.293883] worker_thread+0x24e/0x297
[362108.293884] kthread+0x115/0x11d
[362108.293886] ? kthread_park+0x76/0x76
[362108.293887] ret_from_fork+0x35/0x40
[362108.293889] Showing busy workqueues and worker pools:
root@gentooserver /usr/src/linux/scripts # ./faddr2line ../vmlinux
btrfs_commit_transaction+0x219/0x7ac
btrfs_commit_transaction+0x219/0x7ac:
btrfs_commit_transaction at ??:?
On Wed, Apr 24, 2019 at 3:04 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2019/4/23 上午4:37, Nathan Dehnel wrote:
> > I have a raid10 volume that frequently locks up when I try to write to
> > it or delete things. Any command that touches it will hang (and can't
> > be killed) and I have to start a new ssh session to get into the
> > computer again. Nothing fixes it besides a reboot, and the volume will
> > fail to unmount while the computer is shutting down.
> >
> > [ 302.360912] sysrq: SysRq : Show Blocked State
> > [ 302.360951] task PC stack pid father
> > [ 302.360987] btrfs-transacti D 0 2187 2 0x80000000
> > [ 302.360993] Call Trace:
> > [ 302.361007] ? __schedule+0x59d/0x5f1
> > [ 302.361012] schedule+0x6a/0x85
> > [ 302.361019] btrfs_commit_transaction+0x219/0x7ac
>
> Btrfs is waiting other transaction to be committed.
>
> At that freeze, would you check which thread is taking CPU time?
>
> Are you utilizing backgroup balance or qgroup?
> If you have tons of snapshots, they may hugely slow down qgroup.
> Or you just have a running balance with qgroup enabled, it will almost
> hang your system.
>
> It's going to be fixed in v5.2. While during 4.19 to v5.1 we have a lot
> of optimizations to make balance + qgroup much faster.
>
> Thanks,
> Qu
>
>
>
> > [ 302.361027] ? wait_woken+0x6d/0x6d
> > [ 302.361031] transaction_kthread+0xc9/0x135
> > [ 302.361036] ? btrfs_cleanup_transaction+0x4c7/0x4c7
> > [ 302.361041] kthread+0x115/0x11d
> > [ 302.361046] ? kthread_park+0x76/0x76
> > [ 302.361050] ret_from_fork+0x35/0x40
> > [ 302.361064] nfsd D 0 2292 2 0x80000000
> > [ 302.361067] Call Trace:
> > [ 302.361072] ? __schedule+0x59d/0x5f1
> > [ 302.361077] schedule+0x6a/0x85
> > [ 302.361120] wait_current_trans+0x9b/0xd8
> > [ 302.361126] ? wait_woken+0x6d/0x6d
> > [ 302.361131] start_transaction+0x1ae/0x38e
> > [ 302.361135] btrfs_create+0x59/0x1d0
> > [ 302.361142] vfs_create+0xbf/0xef
> > [ 302.361160] do_nfsd_create+0x2be/0x41d [nfsd]
> > [ 302.361214] nfsd4_open+0x223/0x578 [nfsd]
> > [ 302.361229] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> > [ 302.361240] nfsd_dispatch+0xb9/0x16e [nfsd]
> > [ 302.361258] svc_process+0x524/0x6e2 [sunrpc]
> > [ 302.361270] ? nfsd_destroy+0x5f/0x5f [nfsd]
> > [ 302.361278] nfsd+0xf9/0x150 [nfsd]
> > [ 302.361284] kthread+0x115/0x11d
> > [ 302.361289] ? kthread_park+0x76/0x76
> > [ 302.361292] ret_from_fork+0x35/0x40
> > [ 302.361297] nfsd D 0 2293 2 0x80000000
> > [ 302.361300] Call Trace:
> > [ 302.361305] ? __schedule+0x59d/0x5f1
> > [ 302.361309] schedule+0x6a/0x85
> > [ 302.361314] rwsem_down_write_failed+0x1af/0x210
> > [ 302.361325] ? nfsd_permission+0xa3/0xe8 [nfsd]
> > [ 302.361330] call_rwsem_down_write_failed+0x13/0x20
> > [ 302.361335] down_write+0x20/0x2e
> > [ 302.361345] nfsd_unlink+0xb1/0x16b [nfsd]
> > [ 302.361359] nfsd4_remove+0x4e/0x10a [nfsd]
> > [ 302.361371] nfsd4_proc_compound+0x44a/0x562 [nfsd]
> > [ 302.361381] nfsd_dispatch+0xb9/0x16e [nfsd]
> > [ 302.361395] svc_process+0x524/0x6e2 [sunrpc]
> > [ 302.361401] ? __mutex_unlock_slowpath.isra.6+0x1e8/0x20a
> > [ 302.361410] ? nfsd_destroy+0x5f/0x5f [nfsd]
> > [ 302.361419] nfsd+0xf9/0x150 [nfsd]
> > [ 302.361424] kthread+0x115/0x11d
> > [ 302.361428] ? kthread_park+0x76/0x76
> > [ 302.361434] ret_from_fork+0x35/0x40
> > [ 302.361441] rm D 0 2388 2334 0x00000004
> > [ 302.361444] Call Trace:
> > [ 302.361449] ? __schedule+0x59d/0x5f1
> > [ 302.361453] schedule+0x6a/0x85
> > [ 302.361457] wait_current_trans+0x9b/0xd8
> > [ 302.361462] ? wait_woken+0x6d/0x6d
> > [ 302.361466] start_transaction+0x1ae/0x38e
> > [ 302.361471] btrfs_start_transaction_fallback_global_rsv+0x32/0x127
> > [ 302.361475] btrfs_unlink+0x30/0xc0
> > [ 302.361478] vfs_unlink+0xd2/0x147
> > [ 302.361482] do_unlinkat+0x112/0x223
> > [ 302.361488] do_syscall_64+0x7e/0x133
> > [ 302.361492] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> > [ 302.361496] RIP: 0033:0x7f681509b5d7
> > [ 302.361504] Code: Bad RIP value.
> > [ 302.361506] RSP: 002b:00007fffb1aed668 EFLAGS: 00000202 ORIG_RAX:
> > 0000000000000107
> > [ 302.361510] RAX: ffffffffffffffda RBX: 000055672760c6c0 RCX: 00007f681509b5d7
> > [ 302.361512] RDX: 0000000000000000 RSI: 000055672760b490 RDI: 00000000ffffff9c
> > [ 302.361514] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000
> > [ 302.361516] R10: fffffffffffff12b R11: 0000000000000202 R12: 00007fffb1aed848
> > [ 302.361518] R13: 000055672760b400 R14: 0000000000000002 R15: 0000000000000000
> >
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-05-11 10:50 ` Nathan Dehnel
@ 2019-05-11 11:21 ` Qu Wenruo
2019-07-22 4:25 ` Nathan Dehnel
0 siblings, 1 reply; 12+ messages in thread
From: Qu Wenruo @ 2019-05-11 11:21 UTC (permalink / raw)
To: Nathan Dehnel; +Cc: Btrfs BTRFS
[-- Attachment #1.1: Type: text/plain, Size: 843 bytes --]
> [362108.291969] btrfs_tree_read_lock+0xbb/0xf1
> [362108.291973] ? wait_woken+0x6d/0x6d
> [362108.291978] find_parent_nodes+0x91d/0x12b8
> [362108.291985] ? btrfs_find_all_roots_safe+0x9c/0x107
> [362108.291988] btrfs_find_all_roots_safe+0x9c/0x107
> [362108.291992] btrfs_find_all_roots+0x57/0x75
> [362108.291997] btrfs_qgroup_trace_extent_post+0x37/0x7c
It's qgroup.
We have upstream fix for it, 38e3eebff643 ("btrfs: honor
path->skip_locking in backref code").
It's supported to be backported for all kernels after v4.14.
But I'm not sure if it's backported for your kernel.
As you're gentoo user, it shouldn't be hard to check the kernel source
to find if the fix is backported.
If not, feel free to backport for your kernel.
Thanks,
Qu
> [362108.292002] btrfs_add_delayed_tree_ref+0x305/0x32b
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-05-11 11:21 ` Qu Wenruo
@ 2019-07-22 4:25 ` Nathan Dehnel
2019-07-22 5:02 ` Qu Wenruo
0 siblings, 1 reply; 12+ messages in thread
From: Nathan Dehnel @ 2019-07-22 4:25 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Btrfs BTRFS
I'm still experiencing freezes with kernel 5.2. Here's a backtrace:
[1124738.601136] gitea S 0 12750 1 0x00000000
[1124738.601138] Call Trace:
[1124738.601142] ? __schedule+0x413/0x464
[1124738.601146] schedule+0x80/0xab
[1124738.601149] futex_wait_queue_me+0xc1/0x106
[1124738.601153] futex_wait+0xdb/0x1ea
[1124738.601156] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.601159] ? try_to_wake_up+0x358/0x37e
[1124738.601163] ? get_futex_key+0x94/0x2c2
[1124738.601168] do_futex+0x139/0x98b
[1124738.601170] ? __switch_to_asm+0x34/0x70
[1124738.601173] ? __switch_to_asm+0x34/0x70
[1124738.601175] ? __switch_to_asm+0x40/0x70
[1124738.601177] ? __switch_to_asm+0x34/0x70
[1124738.601179] ? __switch_to_asm+0x34/0x70
[1124738.601182] ? __switch_to_asm+0x40/0x70
[1124738.601184] ? __switch_to_asm+0x34/0x70
[1124738.601186] ? __switch_to_asm+0x40/0x70
[1124738.601188] ? __switch_to_asm+0x34/0x70
[1124738.601190] ? __switch_to_asm+0x40/0x70
[1124738.601192] ? __switch_to_asm+0x34/0x70
[1124738.601194] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.601197] ? finish_task_switch+0x17d/0x26b
[1124738.601201] __se_sys_futex+0x137/0x15c
[1124738.601204] do_syscall_64+0x7e/0x152
[1124738.601207] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601209] RIP: 0033:0x4625e3
[1124738.601214] Code: Bad RIP value.
[1124738.601215] RSP: 002b:00007fc6baffcca0 EFLAGS: 00000286 ORIG_RAX:
00000000000000ca
[1124738.601217] RAX: ffffffffffffffda RBX: 000000c00006d500 RCX:
00000000004625e3
[1124738.601218] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000c00006d648
[1124738.601220] RBP: 00007fc6baffcce8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601221] R10: 0000000000000000 R11: 0000000000000286 R12:
0000000000000040
[1124738.601223] R13: 00000000028aba20 R14: 0000000000000000 R15:
0000000000000000
[1124738.601226] gitea S 0 12751 1 0x00000000
[1124738.601228] Call Trace:
[1124738.601232] ? __schedule+0x413/0x464
[1124738.601235] schedule+0x80/0xab
[1124738.601239] futex_wait_queue_me+0xc1/0x106
[1124738.601243] futex_wait+0xdb/0x1ea
[1124738.601245] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.601249] ? try_to_wake_up+0x358/0x37e
[1124738.601252] ? get_futex_key+0x94/0x2c2
[1124738.601257] do_futex+0x139/0x98b
[1124738.601261] ? cgroup_base_stat_cputime_account_end.isra.0+0x24/0x36
[1124738.601264] ? update_curr+0x113/0x173
[1124738.601267] ? pick_next_entity+0xd1/0xdb
[1124738.601271] ? pick_next_task_fair+0x9c/0x54c
[1124738.601274] ? rcu_preempt_need_deferred_qs+0x15/0x3d
[1124738.601278] __se_sys_futex+0x137/0x15c
[1124738.601282] do_syscall_64+0x7e/0x152
[1124738.601284] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601286] RIP: 0033:0x4625e3
[1124738.601291] Code: Bad RIP value.
[1124738.601292] RSP: 002b:00007fc6ba7fbca0 EFLAGS: 00000286 ORIG_RAX:
00000000000000ca
[1124738.601294] RAX: ffffffffffffffda RBX: 000000c00006d880 RCX:
00000000004625e3
[1124738.601296] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000c00006d9c8
[1124738.601297] RBP: 00007fc6ba7fbce8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601299] R10: 0000000000000000 R11: 0000000000000286 R12:
00000000003fffed
[1124738.601300] R13: 000000000000001a R14: 0000000000000019 R15:
0000000000000200
[1124738.601303] gitea S 0 12752 1 0x00000000
[1124738.601305] Call Trace:
[1124738.601309] ? __schedule+0x413/0x464
[1124738.601313] schedule+0x80/0xab
[1124738.601316] futex_wait_queue_me+0xc1/0x106
[1124738.601320] futex_wait+0xdb/0x1ea
[1124738.601323] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.601327] ? try_to_wake_up+0x358/0x37e
[1124738.601330] ? get_futex_key+0x94/0x2c2
[1124738.601335] do_futex+0x139/0x98b
[1124738.601338] ? __switch_to_asm+0x34/0x70
[1124738.601340] ? __switch_to_asm+0x34/0x70
[1124738.601342] ? __switch_to_asm+0x40/0x70
[1124738.601344] ? __switch_to_asm+0x34/0x70
[1124738.601347] ? __switch_to_asm+0x34/0x70
[1124738.601349] ? __switch_to_asm+0x40/0x70
[1124738.601351] ? __switch_to_asm+0x34/0x70
[1124738.601353] ? __switch_to_asm+0x40/0x70
[1124738.601355] ? __switch_to_asm+0x34/0x70
[1124738.601357] ? __switch_to_asm+0x40/0x70
[1124738.601359] ? __switch_to_asm+0x34/0x70
[1124738.601361] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.601364] ? finish_task_switch+0x17d/0x26b
[1124738.601368] __se_sys_futex+0x137/0x15c
[1124738.601372] do_syscall_64+0x7e/0x152
[1124738.601375] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601376] RIP: 0033:0x4625e3
[1124738.601381] Code: Bad RIP value.
[1124738.601382] RSP: 002b:00007fc6b9ffaca0 EFLAGS: 00000286 ORIG_RAX:
00000000000000ca
[1124738.601384] RAX: ffffffffffffffda RBX: 000000c002332000 RCX:
00000000004625e3
[1124738.601386] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000c002332148
[1124738.601387] RBP: 00007fc6b9fface8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601389] R10: 0000000000000000 R11: 0000000000000286 R12:
0000000000000055
[1124738.601390] R13: 00000000028aba20 R14: 0000000000000000 R15:
0000000000000000
[1124738.601393] gitea S 0 12753 1 0x00000000
[1124738.601395] Call Trace:
[1124738.601399] ? __schedule+0x413/0x464
[1124738.601402] schedule+0x80/0xab
[1124738.601406] futex_wait_queue_me+0xc1/0x106
[1124738.601410] futex_wait+0xdb/0x1ea
[1124738.601414] ? hrtimer_init+0xf4/0xf4
[1124738.601419] do_futex+0x139/0x98b
[1124738.601424] ? hrtimer_cancel+0x11/0x1b
[1124738.601428] ? do_nanosleep+0xba/0x161
[1124738.601430] ? __switch_to_asm+0x34/0x70
[1124738.601434] __se_sys_futex+0x137/0x15c
[1124738.601438] do_syscall_64+0x7e/0x152
[1124738.601441] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601443] RIP: 0033:0x4625e3
[1124738.601447] Code: Bad RIP value.
[1124738.601448] RSP: 002b:000000c003062ea0 EFLAGS: 00000206 ORIG_RAX:
00000000000000ca
[1124738.601451] RAX: ffffffffffffffda RBX: 0000000005f5b74e RCX:
00000000004625e3
[1124738.601452] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000000393e940
[1124738.601454] RBP: 000000c003062ee8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601455] R10: 000000c003062ed8 R11: 0000000000000206 R12:
00000000003fffed
[1124738.601457] R13: 0000000000000178 R14: 0000000000000177 R15:
0000000000000200
[1124738.601459] gitea S 0 12762 1 0x00000000
[1124738.601461] Call Trace:
[1124738.601465] ? __schedule+0x413/0x464
[1124738.601469] schedule+0x80/0xab
[1124738.601473] futex_wait_queue_me+0xc1/0x106
[1124738.601476] futex_wait+0xdb/0x1ea
[1124738.601479] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.601482] ? try_to_wake_up+0x358/0x37e
[1124738.601486] ? get_futex_key+0x94/0x2c2
[1124738.601491] do_futex+0x139/0x98b
[1124738.601496] ? hrtimer_cancel+0x11/0x1b
[1124738.601499] ? do_nanosleep+0xba/0x161
[1124738.601501] ? __switch_to_asm+0x34/0x70
[1124738.601506] __se_sys_futex+0x137/0x15c
[1124738.601509] do_syscall_64+0x7e/0x152
[1124738.601512] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601513] RIP: 0033:0x4625e3
[1124738.601518] Code: Bad RIP value.
[1124738.601519] RSP: 002b:00007fc6b8ff8ca0 EFLAGS: 00000286 ORIG_RAX:
00000000000000ca
[1124738.601522] RAX: ffffffffffffffda RBX: 000000c0015ac000 RCX:
00000000004625e3
[1124738.601523] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000c0015ac148
[1124738.601525] RBP: 00007fc6b8ff8ce8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601526] R10: 0000000000000000 R11: 0000000000000286 R12:
0000000000000001
[1124738.601528] R13: 000000c00293a540 R14: 000000c004279080 R15:
0000000000000200
[1124738.601531] gitea S 0 13585 1 0x00000000
[1124738.601533] Call Trace:
[1124738.601537] ? __schedule+0x413/0x464
[1124738.601541] schedule+0x80/0xab
[1124738.601545] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.601548] ? sock_read_iter+0x79/0xad
[1124738.601552] ep_poll+0x263/0x360
[1124738.601556] ? wake_up_q+0x53/0x53
[1124738.601559] do_epoll_wait+0x92/0xb2
[1124738.601563] __x64_sys_epoll_pwait+0x61/0x9d
[1124738.601566] do_syscall_64+0x7e/0x152
[1124738.601569] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601571] RIP: 0033:0x462790
[1124738.601575] Code: Bad RIP value.
[1124738.601576] RSP: 002b:00007fc69bffe6f8 EFLAGS: 00000202 ORIG_RAX:
0000000000000119
[1124738.601579] RAX: ffffffffffffffda RBX: 00000000ffffffff RCX:
0000000000462790
[1124738.601580] RDX: 0000000000000080 RSI: 00007fc69bffe738 RDI:
0000000000000004
[1124738.601582] RBP: 00007fc69bffed38 R08: 0000000000000000 R09:
0000000000000020
[1124738.601584] R10: 00000000ffffffff R11: 0000000000000202 R12:
ffffffffffffffff
[1124738.601585] R13: 0000000000000024 R14: 0000000000000023 R15:
0000000000000200
[1124738.601588] gitea S 0 13871 1 0x00000000
[1124738.601590] Call Trace:
[1124738.601594] ? __schedule+0x413/0x464
[1124738.601598] schedule+0x80/0xab
[1124738.601601] futex_wait_queue_me+0xc1/0x106
[1124738.601605] futex_wait+0xdb/0x1ea
[1124738.601609] ? hrtimer_init+0xf4/0xf4
[1124738.601614] do_futex+0x139/0x98b
[1124738.601617] ? __switch_to_asm+0x34/0x70
[1124738.601622] __se_sys_futex+0x137/0x15c
[1124738.601635] do_syscall_64+0x7e/0x152
[1124738.601636] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601637] RIP: 0033:0x4625e3
[1124738.601638] Code: Bad RIP value.
[1124738.601639] RSP: 002b:000000c0005faea0 EFLAGS: 00000206 ORIG_RAX:
00000000000000ca
[1124738.601640] RAX: ffffffffffffffda RBX: 0000000015a1ddaa RCX:
00000000004625e3
[1124738.601640] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000000393e640
[1124738.601641] RBP: 000000c0005faee8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601642] R10: 000000c0005faed8 R11: 0000000000000206 R12:
0003fef6f4a5027a
[1124738.601642] R13: 0000000000000002 R14: 0000000000000090 R15:
0000000000000200
[1124738.601643] gitea S 0 4260 1 0x00000000
[1124738.601644] Call Trace:
[1124738.601645] ? __schedule+0x413/0x464
[1124738.601647] schedule+0x80/0xab
[1124738.601648] futex_wait_queue_me+0xc1/0x106
[1124738.601649] futex_wait+0xdb/0x1ea
[1124738.601650] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.601651] ? try_to_wake_up+0x358/0x37e
[1124738.601652] ? get_futex_key+0x94/0x2c2
[1124738.601654] do_futex+0x139/0x98b
[1124738.601656] ? hrtimer_cancel+0x11/0x1b
[1124738.601657] ? do_nanosleep+0xba/0x161
[1124738.601659] __se_sys_futex+0x137/0x15c
[1124738.601660] do_syscall_64+0x7e/0x152
[1124738.601661] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601661] RIP: 0033:0x4625e3
[1124738.601663] Code: Bad RIP value.
[1124738.601663] RSP: 002b:00007fc69affcca0 EFLAGS: 00000286 ORIG_RAX:
00000000000000ca
[1124738.601664] RAX: ffffffffffffffda RBX: 000000c001b27880 RCX:
00000000004625e3
[1124738.601665] RDX: 0000000000000000 RSI: 0000000000000080 RDI:
000000c001b279c8
[1124738.601665] RBP: 00007fc69affcce8 R08: 0000000000000000 R09:
0000000000000000
[1124738.601666] R10: 0000000000000000 R11: 0000000000000286 R12:
0000000000000055
[1124738.601666] R13: 00000000000000a1 R14: 00000000000000a0 R15:
0000000000000200
[1124738.601667] apache2 S 0 18555 11440 0x00000100
[1124738.601668] Call Trace:
[1124738.601670] ? __schedule+0x413/0x464
[1124738.601671] ? preempt_count_add+0x7d/0x9c
[1124738.601672] schedule+0x80/0xab
[1124738.601673] schedule_timeout+0x3a/0x2d9
[1124738.601674] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601675] ? preempt_count_add+0x8a/0x9c
[1124738.601677] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601678] inet_csk_accept+0xe9/0x286
[1124738.601679] ? wait_woken+0x6d/0x6d
[1124738.601680] inet_accept+0x43/0x139
[1124738.601681] __sys_accept4+0xfe/0x1a8
[1124738.601683] ? syscall_trace_enter+0x16c/0x216
[1124738.601684] __x64_sys_accept4+0x1a/0x1d
[1124738.601685] do_syscall_64+0x7e/0x152
[1124738.601686] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601687] RIP: 0033:0x7f317699b52b
[1124738.601688] Code: Bad RIP value.
[1124738.601689] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601689] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601690] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601691] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601691] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601692] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601693] apache2 S 0 25274 11440 0x00000100
[1124738.601694] Call Trace:
[1124738.601696] ? __schedule+0x413/0x464
[1124738.601697] ? preempt_count_add+0x7d/0x9c
[1124738.601698] schedule+0x80/0xab
[1124738.601699] schedule_timeout+0x3a/0x2d9
[1124738.601700] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601701] ? preempt_count_add+0x8a/0x9c
[1124738.601703] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601704] inet_csk_accept+0xe9/0x286
[1124738.601705] ? wait_woken+0x6d/0x6d
[1124738.601706] inet_accept+0x43/0x139
[1124738.601707] __sys_accept4+0xfe/0x1a8
[1124738.601708] ? syscall_trace_enter+0x16c/0x216
[1124738.601710] __x64_sys_accept4+0x1a/0x1d
[1124738.601711] do_syscall_64+0x7e/0x152
[1124738.601712] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601712] RIP: 0033:0x7f317699b52b
[1124738.601714] Code: Bad RIP value.
[1124738.601714] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601715] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601716] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601716] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601717] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601718] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601719] apache2 S 0 25278 11440 0x00000100
[1124738.601720] Call Trace:
[1124738.601721] ? __schedule+0x413/0x464
[1124738.601722] ? preempt_count_add+0x7d/0x9c
[1124738.601723] schedule+0x80/0xab
[1124738.601725] schedule_timeout+0x3a/0x2d9
[1124738.601726] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601726] ? preempt_count_add+0x8a/0x9c
[1124738.601728] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601729] inet_csk_accept+0xe9/0x286
[1124738.601730] ? wait_woken+0x6d/0x6d
[1124738.601731] inet_accept+0x43/0x139
[1124738.601732] __sys_accept4+0xfe/0x1a8
[1124738.601733] ? syscall_trace_enter+0x16c/0x216
[1124738.601735] __x64_sys_accept4+0x1a/0x1d
[1124738.601735] do_syscall_64+0x7e/0x152
[1124738.601736] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601737] RIP: 0033:0x7f317699b52b
[1124738.601738] Code: Bad RIP value.
[1124738.601739] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601740] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601740] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601741] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601741] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601742] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601743] apache2 S 0 6081 11440 0x00000100
[1124738.601744] Call Trace:
[1124738.601746] ? __schedule+0x413/0x464
[1124738.601747] ? preempt_count_add+0x7d/0x9c
[1124738.601748] schedule+0x80/0xab
[1124738.601749] schedule_timeout+0x3a/0x2d9
[1124738.601750] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601751] ? preempt_count_add+0x8a/0x9c
[1124738.601752] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601753] inet_csk_accept+0xe9/0x286
[1124738.601754] ? wait_woken+0x6d/0x6d
[1124738.601756] inet_accept+0x43/0x139
[1124738.601757] __sys_accept4+0xfe/0x1a8
[1124738.601758] ? syscall_trace_enter+0x16c/0x216
[1124738.601759] __x64_sys_accept4+0x1a/0x1d
[1124738.601760] do_syscall_64+0x7e/0x152
[1124738.601761] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601762] RIP: 0033:0x7f317699b52b
[1124738.601764] Code: Bad RIP value.
[1124738.601764] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601765] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601766] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601766] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601767] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601767] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601769] apache2 S 0 6083 11440 0x00000100
[1124738.601769] Call Trace:
[1124738.601771] ? __schedule+0x413/0x464
[1124738.601772] ? preempt_count_add+0x7d/0x9c
[1124738.601773] schedule+0x80/0xab
[1124738.601774] schedule_timeout+0x3a/0x2d9
[1124738.601775] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601776] ? preempt_count_add+0x8a/0x9c
[1124738.601778] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601779] inet_csk_accept+0xe9/0x286
[1124738.601780] ? wait_woken+0x6d/0x6d
[1124738.601781] inet_accept+0x43/0x139
[1124738.601782] __sys_accept4+0xfe/0x1a8
[1124738.601784] ? syscall_trace_enter+0x16c/0x216
[1124738.601785] __x64_sys_accept4+0x1a/0x1d
[1124738.601786] do_syscall_64+0x7e/0x152
[1124738.601787] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601787] RIP: 0033:0x7f317699b52b
[1124738.601789] Code: Bad RIP value.
[1124738.601789] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601790] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601791] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601791] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601792] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601792] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601794] apache2 S 0 6122 11440 0x00000100
[1124738.601794] Call Trace:
[1124738.601796] ? __schedule+0x413/0x464
[1124738.601797] ? preempt_count_add+0x7d/0x9c
[1124738.601798] schedule+0x80/0xab
[1124738.601800] schedule_timeout+0x3a/0x2d9
[1124738.601801] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601802] ? preempt_count_add+0x8a/0x9c
[1124738.601803] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601804] inet_csk_accept+0xe9/0x286
[1124738.601805] ? wait_woken+0x6d/0x6d
[1124738.601806] inet_accept+0x43/0x139
[1124738.601807] __sys_accept4+0xfe/0x1a8
[1124738.601808] ? syscall_trace_enter+0x16c/0x216
[1124738.601810] __x64_sys_accept4+0x1a/0x1d
[1124738.601811] do_syscall_64+0x7e/0x152
[1124738.601812] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601812] RIP: 0033:0x7f317699b52b
[1124738.601814] Code: Bad RIP value.
[1124738.601815] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601815] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601816] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601817] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601817] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601818] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601819] apache2 S 0 8529 11440 0x00000100
[1124738.601820] Call Trace:
[1124738.601821] ? __schedule+0x413/0x464
[1124738.601822] ? preempt_count_add+0x7d/0x9c
[1124738.601824] schedule+0x80/0xab
[1124738.601825] schedule_timeout+0x3a/0x2d9
[1124738.601826] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601827] ? preempt_count_add+0x8a/0x9c
[1124738.601828] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601830] inet_csk_accept+0xe9/0x286
[1124738.601831] ? wait_woken+0x6d/0x6d
[1124738.601832] inet_accept+0x43/0x139
[1124738.601833] __sys_accept4+0xfe/0x1a8
[1124738.601834] ? syscall_trace_enter+0x16c/0x216
[1124738.601836] __x64_sys_accept4+0x1a/0x1d
[1124738.601836] do_syscall_64+0x7e/0x152
[1124738.601837] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601838] RIP: 0033:0x7f317699b52b
[1124738.601839] Code: Bad RIP value.
[1124738.601840] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601841] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601841] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601842] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601843] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601843] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601844] systemd-journal R running task 0 11195 1
0x00000100
[1124738.601845] Call Trace:
[1124738.601848] ? irq_work_queue+0xa/0x45
[1124738.601850] ? wake_up_klogd+0x4f/0x61
[1124738.601851] ? vprintk_emit+0x1a3/0x1b4
[1124738.601852] ? apic_timer_interrupt+0xa/0x20
[1124738.601853] ? preempt_count_add+0x8a/0x9c
[1124738.601854] ? _raw_spin_trylock+0x13/0x2e
[1124738.601855] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.601856] ? ___ratelimit+0xbb/0xc3
[1124738.601858] ? devkmsg_write+0x98/0x16a
[1124738.601858] ? ___bpf_prog_run+0x89b/0x1233
[1124738.601860] ? __bpf_prog_run32+0x39/0x59
[1124738.601861] ? preempt_count_add+0x7d/0x9c
[1124738.601862] ? __seccomp_filter+0x1be/0x3c5
[1124738.601864] ? preempt_count_add+0x8a/0x9c
[1124738.601865] ? _raw_spin_unlock+0x12/0x23
[1124738.601866] ? devkmsg_read+0x1ec/0x214
[1124738.601867] ? syscall_trace_enter+0x16c/0x216
[1124738.601868] ? do_syscall_64+0x55/0x152
[1124738.601869] ? entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601870] apache2 S 0 26040 11440 0x00000100
[1124738.601871] Call Trace:
[1124738.601873] ? __schedule+0x413/0x464
[1124738.601874] ? preempt_count_add+0x7d/0x9c
[1124738.601875] schedule+0x80/0xab
[1124738.601877] schedule_timeout+0x3a/0x2d9
[1124738.601877] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601879] ? preempt_count_add+0x8a/0x9c
[1124738.601880] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601881] inet_csk_accept+0xe9/0x286
[1124738.601882] ? wait_woken+0x6d/0x6d
[1124738.601883] inet_accept+0x43/0x139
[1124738.601885] __sys_accept4+0xfe/0x1a8
[1124738.601886] ? syscall_trace_enter+0x16c/0x216
[1124738.601887] __x64_sys_accept4+0x1a/0x1d
[1124738.601888] do_syscall_64+0x7e/0x152
[1124738.601889] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601890] RIP: 0033:0x7f317699b52b
[1124738.601891] Code: Bad RIP value.
[1124738.601892] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601892] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601893] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601894] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601894] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601895] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601896] apache2 S 0 26697 11440 0x00000100
[1124738.601897] Call Trace:
[1124738.601898] ? __schedule+0x413/0x464
[1124738.601900] ? preempt_count_add+0x7d/0x9c
[1124738.601901] schedule+0x80/0xab
[1124738.601902] schedule_timeout+0x3a/0x2d9
[1124738.601904] ? percpu_counter_add_batch+0x87/0xa0
[1124738.601905] ? preempt_count_add+0x8a/0x9c
[1124738.601906] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.601907] inet_csk_accept+0xe9/0x286
[1124738.601908] ? wait_woken+0x6d/0x6d
[1124738.601909] inet_accept+0x43/0x139
[1124738.601910] __sys_accept4+0xfe/0x1a8
[1124738.601912] ? syscall_trace_enter+0x16c/0x216
[1124738.601913] __x64_sys_accept4+0x1a/0x1d
[1124738.601914] do_syscall_64+0x7e/0x152
[1124738.601915] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601916] RIP: 0033:0x7f317699b52b
[1124738.601917] Code: Bad RIP value.
[1124738.601918] RSP: 002b:00007ffe7fa6cfc0 EFLAGS: 00000246 ORIG_RAX:
0000000000000120
[1124738.601919] RAX: ffffffffffffffda RBX: 00007ffe7fa6d100 RCX:
00007f317699b52b
[1124738.601919] RDX: 00007ffe7fa6d028 RSI: 00007ffe7fa6d048 RDI:
0000000000000004
[1124738.601920] RBP: 000055e43f44da50 R08: 000055e43f71ccc8 R09:
0000000000000000
[1124738.601920] R10: 0000000000080000 R11: 0000000000000246 R12:
000055e43f71f508
[1124738.601921] R13: 000055e43f71f770 R14: 000055e43f451848 R15:
0000000000000000
[1124738.601922] postgres S 0 22573 2048 0x00000000
[1124738.601923] Call Trace:
[1124738.601925] ? __schedule+0x413/0x464
[1124738.601926] ? ep_send_events_proc+0xcc/0x1b6
[1124738.601927] schedule+0x80/0xab
[1124738.601929] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.601930] ? _raw_write_unlock_irq+0x13/0x24
[1124738.601931] ? ep_scan_ready_list.constprop.3+0x196/0x1cb
[1124738.601932] ep_poll+0x263/0x360
[1124738.601934] ? wake_up_q+0x53/0x53
[1124738.601935] do_epoll_wait+0x92/0xb2
[1124738.601936] __x64_sys_epoll_wait+0x1a/0x1d
[1124738.601937] do_syscall_64+0x7e/0x152
[1124738.601938] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601939] RIP: 0033:0x7fdf57acbbfb
[1124738.601940] Code: Bad RIP value.
[1124738.601941] RSP: 002b:00007fff27bbaf40 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[1124738.601942] RAX: ffffffffffffffda RBX: 0000557d77581418 RCX:
00007fdf57acbbfb
[1124738.601942] RDX: 0000000000000001 RSI: 0000557d77581488 RDI:
0000000000000003
[1124738.601943] RBP: 00007fff27bbb030 R08: 0000000006000000 R09:
0000000000000000
[1124738.601943] R10: 00000000ffffffff R11: 0000000000000246 R12:
ffffffffffffffff
[1124738.601944] R13: 0000000000002000 R14: 00007fff27bbb02c R15:
0000557d77556400
[1124738.601945] postgres S 0 22574 2048 0x00000000
[1124738.601946] Call Trace:
[1124738.601947] ? __schedule+0x413/0x464
[1124738.601948] ? ep_send_events_proc+0xcc/0x1b6
[1124738.601950] schedule+0x80/0xab
[1124738.601951] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.601952] ? _raw_write_unlock_irq+0x13/0x24
[1124738.601953] ? ep_scan_ready_list.constprop.3+0x196/0x1cb
[1124738.601954] ep_poll+0x263/0x360
[1124738.601956] ? wake_up_q+0x53/0x53
[1124738.601957] do_epoll_wait+0x92/0xb2
[1124738.601958] __x64_sys_epoll_wait+0x1a/0x1d
[1124738.601959] do_syscall_64+0x7e/0x152
[1124738.601960] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601961] RIP: 0033:0x7fdf57acbbfb
[1124738.601962] Code: Bad RIP value.
[1124738.601963] RSP: 002b:00007fff27bbaf40 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[1124738.601964] RAX: ffffffffffffffda RBX: 0000557d77581418 RCX:
00007fdf57acbbfb
[1124738.601964] RDX: 0000000000000001 RSI: 0000557d77581488 RDI:
0000000000000003
[1124738.601965] RBP: 00007fff27bbb030 R08: 0000000006000000 R09:
0000000000000000
[1124738.601965] R10: 00000000ffffffff R11: 0000000000000246 R12:
ffffffffffffffff
[1124738.601966] R13: 0000000000002000 R14: 00007fff27bbb02c R15:
0000557d77556400
[1124738.601967] systemd S 0 23212 1 0x00000000
[1124738.601968] Call Trace:
[1124738.601969] ? __schedule+0x413/0x464
[1124738.601970] ? ep_send_events_proc+0xcc/0x1b6
[1124738.601972] schedule+0x80/0xab
[1124738.601973] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.601974] ? _raw_write_unlock_irq+0x13/0x24
[1124738.601975] ? ep_scan_ready_list.constprop.3+0x196/0x1cb
[1124738.601976] ep_poll+0x263/0x360
[1124738.601978] ? wake_up_q+0x53/0x53
[1124738.601979] do_epoll_wait+0x92/0xb2
[1124738.601980] __x64_sys_epoll_wait+0x1a/0x1d
[1124738.601981] do_syscall_64+0x7e/0x152
[1124738.601982] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.601983] RIP: 0033:0x7ffb41a44bfb
[1124738.601984] Code: Bad RIP value.
[1124738.601985] RSP: 002b:00007ffee5c19f00 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[1124738.601986] RAX: ffffffffffffffda RBX: 000055b728cf8570 RCX:
00007ffb41a44bfb
[1124738.601986] RDX: 0000000000000010 RSI: 00007ffee5c19f40 RDI:
0000000000000004
[1124738.601987] RBP: 00007ffee5c1a100 R08: 0000000000000010 R09:
00007ffee5d1d080
[1124738.601987] R10: 00000000ffffffff R11: 0000000000000246 R12:
00007ffee5c19f40
[1124738.601988] R13: 0000000000000001 R14: ffffffffffffffff R15:
0000000000000000
[1124738.601989] (sd-pam) S 0 23213 23212 0x00000000
[1124738.601990] Call Trace:
[1124738.601992] ? __schedule+0x413/0x464
[1124738.601993] schedule+0x80/0xab
[1124738.601994] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.601995] ? recalc_sigpending+0x17/0x42
[1124738.601996] ? dequeue_signal+0x177/0x19d
[1124738.601997] do_sigtimedwait.isra.8+0x13e/0x1f5
[1124738.601999] __se_sys_rt_sigtimedwait+0x78/0xb9
[1124738.602001] ? switch_fpu_return+0x2d/0x11e
[1124738.602002] do_syscall_64+0x7e/0x152
[1124738.602003] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602004] RIP: 0033:0x7efe262a7f2c
[1124738.602005] Code: Bad RIP value.
[1124738.602006] RSP: 002b:00007ffecfed8a80 EFLAGS: 00000293 ORIG_RAX:
0000000000000080
[1124738.602006] RAX: ffffffffffffffda RBX: 00007ffecfed8a01 RCX:
00007efe262a7f2c
[1124738.602007] RDX: 0000000000000000 RSI: 00007ffecfed8ac0 RDI:
00007ffecfed8c58
[1124738.602007] RBP: 00007ffecfed8ac0 R08: 0000000000000000 R09:
0000557e458a7630
[1124738.602008] R10: 0000000000000008 R11: 0000000000000293 R12:
00007ffecfed8bb8
[1124738.602009] R13: 0000000000005aac R14: 00007ffecfed8bb8 R15:
0000000000008000
[1124738.602010] screen S 0 26041 1 0x00000000
[1124738.602011] Call Trace:
[1124738.602012] ? __schedule+0x413/0x464
[1124738.602014] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602015] schedule+0x80/0xab
[1124738.602016] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602018] ? pty_stop+0x70/0x70
[1124738.602019] ? n_tty_poll+0x17c/0x18d
[1124738.602021] poll_schedule_timeout.constprop.4+0x43/0x5c
[1124738.602022] do_select+0x5f1/0x643
[1124738.602025] ? poll_select_copy_remaining+0x156/0x156
[1124738.602027] ? poll_select_copy_remaining+0x156/0x156
[1124738.602028] ? poll_select_copy_remaining+0x156/0x156
[1124738.602030] ? lookup_fast+0xff/0x297
[1124738.602031] ? __accumulate_pelt_segments+0x29/0x3a
[1124738.602034] ? intel_pstate_update_util+0x20/0x3c2
[1124738.602035] ? __update_load_avg_cfs_rq+0xee/0x180
[1124738.602036] ? enqueue_entity+0x529/0x678
[1124738.602037] ? legitimize_path.isra.7+0x3c/0x4d
[1124738.602038] ? __switch_to_asm+0x40/0x70
[1124738.602038] ? __switch_to_asm+0x34/0x70
[1124738.602039] ? __switch_to_asm+0x40/0x70
[1124738.602040] ? __switch_to_asm+0x34/0x70
[1124738.602041] ? __switch_to_asm+0x40/0x70
[1124738.602041] ? __switch_to_asm+0x34/0x70
[1124738.602042] ? __switch_to_asm+0x40/0x70
[1124738.602043] ? __switch_to_asm+0x34/0x70
[1124738.602043] ? __switch_to_asm+0x40/0x70
[1124738.602044] ? __switch_to_asm+0x34/0x70
[1124738.602045] ? __switch_to_asm+0x34/0x70
[1124738.602046] core_sys_select+0x196/0x248
[1124738.602048] ? __schedule+0x41b/0x464
[1124738.602049] ? hrtimer_try_to_cancel+0x29/0x119
[1124738.602051] ? hrtimer_cancel+0x11/0x1b
[1124738.602052] ? do_nanosleep+0xba/0x161
[1124738.602054] kern_select+0x97/0xcc
[1124738.602055] __x64_sys_select+0x20/0x23
[1124738.602056] do_syscall_64+0x7e/0x152
[1124738.602057] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602058] RIP: 0033:0x7f4cc57a556b
[1124738.602060] Code: Bad RIP value.
[1124738.602060] RSP: 002b:00007ffeed5bd5e0 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[1124738.602061] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f4cc57a556b
[1124738.602062] RDX: 00007ffeed5bd6d8 RSI: 00007ffeed5bd658 RDI:
0000000000000400
[1124738.602062] RBP: 0000000000000000 R08: 0000000000000000 R09:
00005590928afa00
[1124738.602063] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffeed5bd648
[1124738.602063] R13: 00007ffeed5bd658 R14: 00007ffeed5bd6d8 R15:
0000000000000040
[1124738.602064] bash S 0 26042 26041 0x00000000
[1124738.602065] Call Trace:
[1124738.602067] ? __schedule+0x413/0x464
[1124738.602068] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602069] schedule+0x80/0xab
[1124738.602071] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602072] ? pty_stop+0x70/0x70
[1124738.602073] ? n_tty_poll+0x17c/0x18d
[1124738.602075] poll_schedule_timeout.constprop.4+0x43/0x5c
[1124738.602076] do_select+0x5f1/0x643
[1124738.602079] ? poll_select_copy_remaining+0x156/0x156
[1124738.602080] ? poll_select_copy_remaining+0x156/0x156
[1124738.602081] ? __inode_wait_for_writeback+0x76/0xc9
[1124738.602082] ? fsnotify_grab_connector+0x71/0x79
[1124738.602084] ? reuse_swap_page+0x6c/0x2f7
[1124738.602085] ? __accumulate_pelt_segments+0x29/0x3a
[1124738.602086] ? intel_pstate_update_util+0x20/0x3c2
[1124738.602087] ? __update_load_avg_cfs_rq+0xee/0x180
[1124738.602088] ? enqueue_entity+0x529/0x678
[1124738.602090] ? enqueue_task_fair+0x6c8/0x78d
[1124738.602091] ? tty_set_termios+0x176/0x196
[1124738.602092] ? resched_curr+0x41/0xac
[1124738.602093] ? check_preempt_curr+0x4c/0x69
[1124738.602094] ? ttwu_do_wakeup.isra.10+0x14/0xe2
[1124738.602096] core_sys_select+0x196/0x248
[1124738.602097] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.602098] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.602099] ? __wake_up_common_lock+0x8b/0xd0
[1124738.602101] __se_sys_pselect6+0x106/0x15f
[1124738.602103] do_syscall_64+0x7e/0x152
[1124738.602104] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602104] RIP: 0033:0x7fbf7de2867a
[1124738.602106] Code: Bad RIP value.
[1124738.602106] RSP: 002b:00007fffa31a6600 EFLAGS: 00000246 ORIG_RAX:
000000000000010e
[1124738.602108] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007fbf7de2867a
[1124738.602108] RDX: 0000000000000000 RSI: 00007fffa31a6708 RDI:
0000000000000001
[1124738.602109] RBP: 00007fbf7def5a00 R08: 0000000000000000 R09:
00007fffa31a6640
[1124738.602109] R10: 0000000000000000 R11: 0000000000000246 R12:
00007fffa31a6708
[1124738.602110] R13: 00007fffa31a6688 R14: 0000000000000000 R15:
0000000000000000
[1124738.602115] screen S 0 28157 1 0x00000000
[1124738.602116] Call Trace:
[1124738.602118] ? __schedule+0x413/0x464
[1124738.602120] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602121] schedule+0x80/0xab
[1124738.602122] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602124] ? pty_stop+0x70/0x70
[1124738.602124] ? n_tty_poll+0x17c/0x18d
[1124738.602126] poll_schedule_timeout.constprop.4+0x43/0x5c
[1124738.602127] do_select+0x5f1/0x643
[1124738.602129] ? enqueue_entity+0x529/0x678
[1124738.602131] ? poll_select_copy_remaining+0x156/0x156
[1124738.602133] ? poll_select_copy_remaining+0x156/0x156
[1124738.602134] ? poll_select_copy_remaining+0x156/0x156
[1124738.602135] ? __switch_to_asm+0x40/0x70
[1124738.602136] ? lookup_fast+0xff/0x297
[1124738.602138] ? __accumulate_pelt_segments+0x29/0x3a
[1124738.602139] ? __update_load_avg_se+0xec/0x19e
[1124738.602140] ? enqueue_entity+0x529/0x678
[1124738.602141] ? __switch_to_asm+0x40/0x70
[1124738.602142] ? __switch_to_asm+0x34/0x70
[1124738.602142] ? __switch_to_asm+0x40/0x70
[1124738.602143] ? __switch_to_asm+0x34/0x70
[1124738.602144] ? __switch_to_asm+0x40/0x70
[1124738.602145] ? __switch_to_asm+0x34/0x70
[1124738.602145] ? __switch_to_asm+0x40/0x70
[1124738.602146] ? __switch_to_asm+0x34/0x70
[1124738.602147] ? __switch_to_asm+0x40/0x70
[1124738.602147] ? __switch_to_asm+0x34/0x70
[1124738.602148] ? __switch_to_asm+0x34/0x70
[1124738.602150] core_sys_select+0x196/0x248
[1124738.602151] ? __schedule+0x41b/0x464
[1124738.602153] ? hrtimer_try_to_cancel+0x29/0x119
[1124738.602154] ? hrtimer_cancel+0x11/0x1b
[1124738.602155] ? do_nanosleep+0xba/0x161
[1124738.602157] kern_select+0x97/0xcc
[1124738.602158] __x64_sys_select+0x20/0x23
[1124738.602159] do_syscall_64+0x7e/0x152
[1124738.602160] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602161] RIP: 0033:0x7fd5847a556b
[1124738.602163] Code: Bad RIP value.
[1124738.602163] RSP: 002b:00007ffd2ef75720 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[1124738.602164] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007fd5847a556b
[1124738.602164] RDX: 00007ffd2ef75818 RSI: 00007ffd2ef75798 RDI:
0000000000000400
[1124738.602165] RBP: 0000000000000000 R08: 0000000000000000 R09:
000055aba1c38a00
[1124738.602166] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffd2ef75788
[1124738.602166] R13: 00007ffd2ef75798 R14: 00007ffd2ef75818 R15:
0000000000000040
[1124738.602168] bash S 0 28158 28157 0x00000000
[1124738.602169] Call Trace:
[1124738.602170] ? __schedule+0x413/0x464
[1124738.602171] ? preempt_count_add+0x8a/0x9c
[1124738.602172] schedule+0x80/0xab
[1124738.602173] do_wait+0x1d5/0x237
[1124738.602175] kernel_wait4+0xc5/0x11a
[1124738.602176] ? __se_compat_sys_waitid+0xf5/0xf5
[1124738.602178] __se_sys_wait4+0x3c/0x8b
[1124738.602179] ? handle_mm_fault+0x133/0x182
[1124738.602180] ? preempt_count_add+0x8a/0x9c
[1124738.602181] ? recalc_sigpending+0x17/0x42
[1124738.602182] ? __set_current_blocked+0x3d/0x55
[1124738.602183] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.602184] ? sigprocmask+0x6e/0x8a
[1124738.602185] ? _copy_to_user+0x22/0x28
[1124738.602186] ? __se_sys_rt_sigprocmask+0x93/0xbb
[1124738.602187] do_syscall_64+0x7e/0x152
[1124738.602188] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602188] RIP: 0033:0x7f0b335fe67b
[1124738.602190] Code: Bad RIP value.
[1124738.602190] RSP: 002b:00007ffc9d272300 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[1124738.602191] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f0b335fe67b
[1124738.602192] RDX: 000000000000000a RSI: 00007ffc9d272340 RDI:
00000000ffffffff
[1124738.602192] RBP: 000000000000000a R08: 0000000000000000 R09:
0000557394e76d70
[1124738.602193] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[1124738.602193] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[1124738.602195] su S 0 28393 28158 0x00000000
[1124738.602195] Call Trace:
[1124738.602197] ? __schedule+0x413/0x464
[1124738.602198] schedule+0x80/0xab
[1124738.602199] do_wait+0x1d5/0x237
[1124738.602200] ? do_wp_page+0x497/0x4b5
[1124738.602202] kernel_wait4+0xc5/0x11a
[1124738.602203] ? __se_compat_sys_waitid+0xf5/0xf5
[1124738.602204] __se_sys_wait4+0x3c/0x8b
[1124738.602205] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602206] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.602207] ? do_sigaction+0xad/0x184
[1124738.602208] ? preempt_count_add+0x8a/0x9c
[1124738.602209] ? recalc_sigpending+0x17/0x42
[1124738.602210] ? __set_current_blocked+0x3d/0x55
[1124738.602210] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.602211] ? sigprocmask+0x6e/0x8a
[1124738.602212] ? __se_sys_rt_sigprocmask+0x72/0xbb
[1124738.602213] do_syscall_64+0x7e/0x152
[1124738.602215] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602215] RIP: 0033:0x7f184842267b
[1124738.602217] Code: Bad RIP value.
[1124738.602217] RSP: 002b:00007ffeeabb96f0 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[1124738.602218] RAX: ffffffffffffffda RBX: 00007ffeeabb9740 RCX:
00007f184842267b
[1124738.602219] RDX: 0000000000000002 RSI: 00007ffeeabb973c RDI:
00000000ffffffff
[1124738.602219] RBP: 00007ffeeabb973c R08: 0000000000000000 R09:
00007f184834bb80
[1124738.602220] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffeeabb99b8
[1124738.602221] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[1124738.602222] bash S 0 28394 28393 0x00000000
[1124738.602223] Call Trace:
[1124738.602224] ? __schedule+0x413/0x464
[1124738.602226] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602227] schedule+0x80/0xab
[1124738.602228] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602230] ? pty_stop+0x70/0x70
[1124738.602230] ? n_tty_poll+0x17c/0x18d
[1124738.602232] poll_schedule_timeout.constprop.4+0x43/0x5c
[1124738.602233] do_select+0x5f1/0x643
[1124738.602236] ? poll_select_copy_remaining+0x156/0x156
[1124738.602237] ? poll_select_copy_remaining+0x156/0x156
[1124738.602239] ? page_counter_try_charge+0x52/0x88
[1124738.602240] ? number+0x14d/0x254
[1124738.602241] ? lookup_fast+0xff/0x297
[1124738.602243] ? __accumulate_pelt_segments+0x29/0x3a
[1124738.602244] ? intel_pstate_update_util+0x20/0x3c2
[1124738.602245] ? __update_load_avg_cfs_rq+0xee/0x180
[1124738.602246] ? enqueue_entity+0x529/0x678
[1124738.602248] ? enqueue_task_fair+0x6c8/0x78d
[1124738.602248] ? tty_set_termios+0x176/0x196
[1124738.602250] ? resched_curr+0x41/0xac
[1124738.602251] ? check_preempt_curr+0x4c/0x69
[1124738.602252] ? ttwu_do_wakeup.isra.10+0x14/0xe2
[1124738.602253] core_sys_select+0x196/0x248
[1124738.602255] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.602256] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.602257] ? __wake_up_common_lock+0x8b/0xd0
[1124738.602259] __se_sys_pselect6+0x106/0x15f
[1124738.602260] do_syscall_64+0x7e/0x152
[1124738.602261] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602262] RIP: 0033:0x7f44ced9a67a
[1124738.602263] Code: Bad RIP value.
[1124738.602264] RSP: 002b:00007ffc83eb9ea0 EFLAGS: 00000246 ORIG_RAX:
000000000000010e
[1124738.602265] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f44ced9a67a
[1124738.602265] RDX: 0000000000000000 RSI: 00007ffc83eb9fa8 RDI:
0000000000000001
[1124738.602266] RBP: 00007f44cee67a00 R08: 0000000000000000 R09:
00007ffc83eb9ee0
[1124738.602266] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffc83eb9fa8
[1124738.602267] R13: 00007ffc83eb9f28 R14: 0000000000000000 R15:
0000000000000000
[1124738.602268] kworker/6:1 I 0 20084 2 0x80004000
[1124738.602272] Workqueue: 0x0 (events_power_efficient)
[1124738.602273] Call Trace:
[1124738.602274] ? __schedule+0x413/0x464
[1124738.602275] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602277] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602278] schedule+0x80/0xab
[1124738.602279] worker_thread+0x24e/0x297
[1124738.602280] kthread+0x115/0x11d
[1124738.602281] ? kthread_park+0x76/0x76
[1124738.602282] ret_from_fork+0x35/0x40
[1124738.602284] kworker/0:2 I 0 32011 2 0x80004000
[1124738.602287] Workqueue: 0x0 (events_power_efficient)
[1124738.602288] Call Trace:
[1124738.602289] ? __schedule+0x413/0x464
[1124738.602290] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602292] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602293] schedule+0x80/0xab
[1124738.602294] worker_thread+0x24e/0x297
[1124738.602295] kthread+0x115/0x11d
[1124738.602296] ? kthread_park+0x76/0x76
[1124738.602297] ret_from_fork+0x35/0x40
[1124738.602298] kworker/u17:0 I 0 2604 2 0x80004000
[1124738.602301] Workqueue: 0x0 (btrfs-worker-high)
[1124738.602302] Call Trace:
[1124738.602304] ? __schedule+0x413/0x464
[1124738.602305] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602306] schedule+0x80/0xab
[1124738.602307] worker_thread+0x24e/0x297
[1124738.602308] kthread+0x115/0x11d
[1124738.602309] ? kthread_park+0x76/0x76
[1124738.602310] ret_from_fork+0x35/0x40
[1124738.602311] kworker/5:0 I 0 4557 2 0x80004000
[1124738.602315] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602315] Call Trace:
[1124738.602317] ? __schedule+0x413/0x464
[1124738.602318] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602319] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602320] schedule+0x80/0xab
[1124738.602321] worker_thread+0x24e/0x297
[1124738.602323] kthread+0x115/0x11d
[1124738.602323] ? kthread_park+0x76/0x76
[1124738.602324] ret_from_fork+0x35/0x40
[1124738.602326] kworker/1:0 I 0 14039 2 0x80004000
[1124738.602329] Workqueue: 0x0 (events_freezable_power_)
[1124738.602329] Call Trace:
[1124738.602331] ? __schedule+0x413/0x464
[1124738.602332] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602333] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602334] schedule+0x80/0xab
[1124738.602336] worker_thread+0x24e/0x297
[1124738.602337] kthread+0x115/0x11d
[1124738.602337] ? kthread_park+0x76/0x76
[1124738.602338] ret_from_fork+0x35/0x40
[1124738.602340] kworker/2:1 I 0 22643 2 0x80004000
[1124738.602343] Workqueue: 0x0 (events_freezable_power_)
[1124738.602344] Call Trace:
[1124738.602345] ? __schedule+0x413/0x464
[1124738.602347] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602348] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602349] schedule+0x80/0xab
[1124738.602350] worker_thread+0x24e/0x297
[1124738.602351] kthread+0x115/0x11d
[1124738.602352] ? kthread_park+0x76/0x76
[1124738.602353] ret_from_fork+0x35/0x40
[1124738.602354] kworker/3:2 I 0 3070 2 0x80004000
[1124738.602358] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602358] Call Trace:
[1124738.602360] ? __schedule+0x413/0x464
[1124738.602361] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602362] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602363] schedule+0x80/0xab
[1124738.602364] worker_thread+0x24e/0x297
[1124738.602365] kthread+0x115/0x11d
[1124738.602366] ? kthread_park+0x76/0x76
[1124738.602367] ret_from_fork+0x35/0x40
[1124738.602369] kworker/3:0 I 0 13409 2 0x80004000
[1124738.602372] Workqueue: 0x0 (cgroup_destroy)
[1124738.602373] Call Trace:
[1124738.602374] ? __schedule+0x413/0x464
[1124738.602376] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602377] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602378] schedule+0x80/0xab
[1124738.602379] worker_thread+0x24e/0x297
[1124738.602380] kthread+0x115/0x11d
[1124738.602381] ? kthread_park+0x76/0x76
[1124738.602382] ret_from_fork+0x35/0x40
[1124738.602383] kworker/0:0 I 0 13460 2 0x80004000
[1124738.602386] Workqueue: 0x0 (cgroup_destroy)
[1124738.602387] Call Trace:
[1124738.602389] ? __schedule+0x413/0x464
[1124738.602390] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602391] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602392] schedule+0x80/0xab
[1124738.602393] worker_thread+0x24e/0x297
[1124738.602394] kthread+0x115/0x11d
[1124738.602395] ? kthread_park+0x76/0x76
[1124738.602396] ret_from_fork+0x35/0x40
[1124738.602397] kworker/6:0 I 0 19672 2 0x80004000
[1124738.602401] Workqueue: 0x0 (cgroup_destroy)
[1124738.602401] Call Trace:
[1124738.602403] ? __schedule+0x413/0x464
[1124738.602404] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602405] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602406] schedule+0x80/0xab
[1124738.602407] worker_thread+0x24e/0x297
[1124738.602409] kthread+0x115/0x11d
[1124738.602410] ? kthread_park+0x76/0x76
[1124738.602411] ret_from_fork+0x35/0x40
[1124738.602412] kworker/u17:1 I 0 20661 2 0x80004000
[1124738.602415] Workqueue: 0x0 (btrfs-worker-high)
[1124738.602416] Call Trace:
[1124738.602417] ? __schedule+0x413/0x464
[1124738.602419] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602420] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602421] schedule+0x80/0xab
[1124738.602422] worker_thread+0x24e/0x297
[1124738.602423] kthread+0x115/0x11d
[1124738.602424] ? kthread_park+0x76/0x76
[1124738.602425] ret_from_fork+0x35/0x40
[1124738.602427] kworker/4:3 I 0 21762 2 0x80004000
[1124738.602430] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602430] Call Trace:
[1124738.602432] ? __schedule+0x413/0x464
[1124738.602433] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602434] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602435] schedule+0x80/0xab
[1124738.602436] worker_thread+0x24e/0x297
[1124738.602437] kthread+0x115/0x11d
[1124738.602438] ? kthread_park+0x76/0x76
[1124738.602439] ret_from_fork+0x35/0x40
[1124738.602441] kworker/7:3 I 0 22070 2 0x80004000
[1124738.602444] Workqueue: 0x0 (events_freezable_power_)
[1124738.602444] Call Trace:
[1124738.602446] ? __schedule+0x413/0x464
[1124738.602447] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602448] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602449] schedule+0x80/0xab
[1124738.602450] worker_thread+0x24e/0x297
[1124738.602452] kthread+0x115/0x11d
[1124738.602453] ? kthread_park+0x76/0x76
[1124738.602453] ret_from_fork+0x35/0x40
[1124738.602455] pickup S 0 23399 2375 0x00000100
[1124738.602456] Call Trace:
[1124738.602458] ? __schedule+0x413/0x464
[1124738.602459] schedule+0x80/0xab
[1124738.602461] schedule_hrtimeout_range_clock+0xb0/0xf4
[1124738.602462] ? hrtimer_init+0xf4/0xf4
[1124738.602463] ep_poll+0x263/0x360
[1124738.602465] ? wake_up_q+0x53/0x53
[1124738.602466] do_epoll_wait+0x92/0xb2
[1124738.602468] __x64_sys_epoll_wait+0x1a/0x1d
[1124738.602469] do_syscall_64+0x7e/0x152
[1124738.602470] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602470] RIP: 0033:0x7f1298a3bbfb
[1124738.602472] Code: Bad RIP value.
[1124738.602472] RSP: 002b:00007ffed6056330 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[1124738.602473] RAX: ffffffffffffffda RBX: 00007ffed6056388 RCX:
00007f1298a3bbfb
[1124738.602474] RDX: 0000000000000064 RSI: 00007ffed6056388 RDI:
0000000000000008
[1124738.602474] RBP: 00000000ffffffff R08: 000000000000000f R09:
0000556e3b2b07f0
[1124738.602475] R10: 00000000000186a0 R11: 0000000000000246 R12:
00007f1298d4f454
[1124738.602476] R13: 00007f1298d4f3c0 R14: 00007ffed6056ec0 R15:
0000000000000000
[1124738.602477] kworker/u16:3 I 0 23585 2 0x80004000
[1124738.602480] Workqueue: 0x0 (btrfs-endio)
[1124738.602480] Call Trace:
[1124738.602481] ? __schedule+0x413/0x464
[1124738.602483] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602484] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602485] schedule+0x80/0xab
[1124738.602486] worker_thread+0x24e/0x297
[1124738.602487] kthread+0x115/0x11d
[1124738.602488] ? kthread_park+0x76/0x76
[1124738.602489] ret_from_fork+0x35/0x40
[1124738.602490] kworker/7:1 I 0 23636 2 0x80004000
[1124738.602493] Workqueue: 0x0 (events_freezable_power_)
[1124738.602494] Call Trace:
[1124738.602495] ? __schedule+0x413/0x464
[1124738.602497] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602498] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602499] schedule+0x80/0xab
[1124738.602500] worker_thread+0x24e/0x297
[1124738.602501] kthread+0x115/0x11d
[1124738.602502] ? kthread_park+0x76/0x76
[1124738.602503] ret_from_fork+0x35/0x40
[1124738.602505] kworker/5:1 I 0 23887 2 0x80004000
[1124738.602508] Workqueue: 0x0 (cgroup_destroy)
[1124738.602508] Call Trace:
[1124738.602510] ? __schedule+0x413/0x464
[1124738.602511] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602512] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602513] schedule+0x80/0xab
[1124738.602514] worker_thread+0x24e/0x297
[1124738.602515] kthread+0x115/0x11d
[1124738.602516] ? kthread_park+0x76/0x76
[1124738.602517] ret_from_fork+0x35/0x40
[1124738.602519] kworker/4:0 I 0 23888 2 0x80004000
[1124738.602522] Call Trace:
[1124738.602523] ? __schedule+0x413/0x464
[1124738.602525] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602526] schedule+0x80/0xab
[1124738.602527] worker_thread+0x24e/0x297
[1124738.602528] kthread+0x115/0x11d
[1124738.602529] ? kthread_park+0x76/0x76
[1124738.602530] ret_from_fork+0x35/0x40
[1124738.602531] kworker/u16:4 I 0 23902 2 0x80004000
[1124738.602534] Workqueue: 0x0 (btrfs-endio)
[1124738.602534] Call Trace:
[1124738.602536] ? __schedule+0x413/0x464
[1124738.602537] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602538] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602539] schedule+0x80/0xab
[1124738.602540] worker_thread+0x24e/0x297
[1124738.602541] kthread+0x115/0x11d
[1124738.602542] ? kthread_park+0x76/0x76
[1124738.602543] ret_from_fork+0x35/0x40
[1124738.602545] kworker/7:2 I 0 24095 2 0x80004000
[1124738.602548] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602548] Call Trace:
[1124738.602550] ? __schedule+0x413/0x464
[1124738.602551] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602552] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602553] schedule+0x80/0xab
[1124738.602555] worker_thread+0x24e/0x297
[1124738.602556] kthread+0x115/0x11d
[1124738.602556] ? kthread_park+0x76/0x76
[1124738.602557] ret_from_fork+0x35/0x40
[1124738.602559] sshd S 0 24150 2049 0x00000000
[1124738.602560] Call Trace:
[1124738.602561] ? __schedule+0x413/0x464
[1124738.602563] ? __mod_memcg_state+0xa6/0xba
[1124738.602564] schedule+0x80/0xab
[1124738.602565] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602566] ? preempt_count_add+0x8a/0x9c
[1124738.602568] ? _raw_spin_lock_irqsave+0x17/0x34
[1124738.602569] ? add_wait_queue+0x15/0x3e
[1124738.602570] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.602571] ? unix_poll+0x87/0x96
[1124738.602573] poll_schedule_timeout.constprop.4+0x43/0x5c
[1124738.602574] do_sys_poll+0x383/0x431
[1124738.602576] ? __kmalloc_reserve.isra.9+0x2d/0x70
[1124738.602577] ? __alloc_skb+0xb3/0x18c
[1124738.602578] ? __follow_mount_rcu+0x5b/0xc5
[1124738.602579] ? lookup_fast+0xff/0x297
[1124738.602581] ? poll_select_copy_remaining+0x156/0x156
[1124738.602582] ? cdev_put+0x1e/0x1e
[1124738.602583] ? dput.part.5+0x29/0xfd
[1124738.602584] ? mntput_no_expire+0x11/0x185
[1124738.602585] ? terminate_walk+0x20/0x7e
[1124738.602586] ? path_openat+0xb78/0xc1b
[1124738.602588] ? do_filp_open+0x88/0xae
[1124738.602590] ? new_sync_write+0x109/0x142
[1124738.602591] ? preempt_count_add+0x7d/0x9c
[1124738.602592] ? __fd_install+0xa5/0xbe
[1124738.602593] __se_sys_poll+0x5a/0xd6
[1124738.602595] do_syscall_64+0x7e/0x152
[1124738.602596] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602596] RIP: 0033:0x7f6aa5be2728
[1124738.602598] Code: Bad RIP value.
[1124738.602598] RSP: 002b:00007ffebe1bd450 EFLAGS: 00000246 ORIG_RAX:
0000000000000007
[1124738.602599] RAX: ffffffffffffffda RBX: 000055ff5ba891a0 RCX:
00007f6aa5be2728
[1124738.602600] RDX: 00000000ffffffff RSI: 0000000000000001 RDI:
00007ffebe1bd498
[1124738.602600] RBP: 00007ffebe1bd498 R08: 000055ff5d8038b0 R09:
000055ff5d803870
[1124738.602601] R10: 0000000000000000 R11: 0000000000000246 R12:
000055ff5d7f6880
[1124738.602601] R13: 0000000000000000 R14: 0000000000005e57 R15:
0000000000000014
[1124738.602603] systemd S 0 24154 1 0x00000000
[1124738.602604] Call Trace:
[1124738.602605] ? __schedule+0x413/0x464
[1124738.602606] ? ep_send_events_proc+0xcc/0x1b6
[1124738.602607] schedule+0x80/0xab
[1124738.602609] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602610] ? _raw_write_unlock_irq+0x13/0x24
[1124738.602611] ? ep_scan_ready_list.constprop.3+0x196/0x1cb
[1124738.602612] ep_poll+0x263/0x360
[1124738.602614] ? wake_up_q+0x53/0x53
[1124738.602615] do_epoll_wait+0x92/0xb2
[1124738.602616] __x64_sys_epoll_wait+0x1a/0x1d
[1124738.602617] do_syscall_64+0x7e/0x152
[1124738.602618] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602619] RIP: 0033:0x7fd387c34bfb
[1124738.602621] Code: Bad RIP value.
[1124738.602621] RSP: 002b:00007ffd9ba9df20 EFLAGS: 00000246 ORIG_RAX:
00000000000000e8
[1124738.602622] RAX: ffffffffffffffda RBX: 000055845d562500 RCX:
00007fd387c34bfb
[1124738.602623] RDX: 0000000000000010 RSI: 00007ffd9ba9df60 RDI:
0000000000000004
[1124738.602623] RBP: 00007ffd9ba9e120 R08: 0000000000000010 R09:
00007ffd9bbdf080
[1124738.602624] R10: 00000000ffffffff R11: 0000000000000246 R12:
00007ffd9ba9df60
[1124738.602624] R13: 0000000000000001 R14: ffffffffffffffff R15:
0000000000000000
[1124738.602626] (sd-pam) S 0 24155 24154 0x00000000
[1124738.602626] Call Trace:
[1124738.602628] ? __schedule+0x413/0x464
[1124738.602629] schedule+0x80/0xab
[1124738.602631] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602632] ? recalc_sigpending+0x17/0x42
[1124738.602632] ? dequeue_signal+0x177/0x19d
[1124738.602634] do_sigtimedwait.isra.8+0x13e/0x1f5
[1124738.602635] __se_sys_rt_sigtimedwait+0x78/0xb9
[1124738.602637] ? switch_fpu_return+0x2d/0x11e
[1124738.602638] do_syscall_64+0x7e/0x152
[1124738.602639] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602639] RIP: 0033:0x7efe262a7f2c
[1124738.602641] Code: Bad RIP value.
[1124738.602641] RSP: 002b:00007ffecfed8a80 EFLAGS: 00000293 ORIG_RAX:
0000000000000080
[1124738.602642] RAX: ffffffffffffffda RBX: 00007ffecfed8a01 RCX:
00007efe262a7f2c
[1124738.602643] RDX: 0000000000000000 RSI: 00007ffecfed8ac0 RDI:
00007ffecfed8c58
[1124738.602643] RBP: 00007ffecfed8ac0 R08: 0000000000000000 R09:
0000557e457450f0
[1124738.602644] R10: 0000000000000008 R11: 0000000000000293 R12:
00007ffecfed8bb8
[1124738.602644] R13: 0000000000005e5a R14: 00007ffecfed8bb8 R15:
0000000000008000
[1124738.602646] sshd S 0 24159 24150 0x00000000
[1124738.602646] Call Trace:
[1124738.602648] ? __schedule+0x413/0x464
[1124738.602649] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602650] schedule+0x80/0xab
[1124738.602652] schedule_hrtimeout_range_clock+0x48/0xf4
[1124738.602653] ? pty_stop+0x70/0x70
[1124738.602654] ? n_tty_poll+0x17c/0x18d
[1124738.602655] poll_schedule_timeout.constprop.4+0x43/0x5c
[1124738.602657] do_select+0x5f1/0x643
[1124738.602659] ? __dev_queue_xmit+0x827/0x84f
[1124738.602660] ? poll_select_copy_remaining+0x156/0x156
[1124738.602662] ? poll_select_copy_remaining+0x156/0x156
[1124738.602663] ? poll_select_copy_remaining+0x156/0x156
[1124738.602664] ? poll_select_copy_remaining+0x156/0x156
[1124738.602666] ? internal_add_timer+0x27/0x34
[1124738.602667] ? _raw_spin_unlock_irqrestore+0x14/0x25
[1124738.602668] ? mod_timer+0x219/0x241
[1124738.602670] ? sk_reset_timer+0x14/0x27
[1124738.602671] ? tcp_schedule_loss_probe+0x12d/0x135
[1124738.602672] ? tcp_write_xmit+0x822/0xced
[1124738.602674] ? __tcp_push_pending_frames+0x31/0xb7
[1124738.602675] ? tcp_sendmsg_locked+0x9c1/0xb90
[1124738.602676] core_sys_select+0x196/0x248
[1124738.602678] ? new_sync_write+0x109/0x142
[1124738.602680] kern_select+0x97/0xcc
[1124738.602681] ? _copy_to_user+0x22/0x28
[1124738.602682] ? put_timespec64+0x3c/0x61
[1124738.602683] __x64_sys_select+0x20/0x23
[1124738.602684] do_syscall_64+0x7e/0x152
[1124738.602685] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602686] RIP: 0033:0x7f6aa5be556b
[1124738.602688] Code: Bad RIP value.
[1124738.602688] RSP: 002b:00007ffebe1bd3c0 EFLAGS: 00000246 ORIG_RAX:
0000000000000017
[1124738.602689] RAX: ffffffffffffffda RBX: 000055ff5d7f84d0 RCX:
00007f6aa5be556b
[1124738.602690] RDX: 000055ff5d7f67b0 RSI: 000055ff5d7f1e00 RDI:
000000000000000d
[1124738.602690] RBP: 0000000000000000 R08: 0000000000000000 R09:
00007ffebe1bd440
[1124738.602691] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[1124738.602691] R13: 0000000000000003 R14: 0000000000000003 R15:
000055ff5ba89a20
[1124738.602693] bash S 0 24160 24159 0x00000000
[1124738.602693] Call Trace:
[1124738.602695] ? __schedule+0x413/0x464
[1124738.602696] schedule+0x80/0xab
[1124738.602697] do_wait+0x1d5/0x237
[1124738.602699] kernel_wait4+0xc5/0x11a
[1124738.602700] ? __se_compat_sys_waitid+0xf5/0xf5
[1124738.602701] __se_sys_wait4+0x3c/0x8b
[1124738.602703] ? handle_mm_fault+0x133/0x182
[1124738.602704] ? preempt_count_add+0x8a/0x9c
[1124738.602705] ? recalc_sigpending+0x17/0x42
[1124738.602706] ? __set_current_blocked+0x3d/0x55
[1124738.602707] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.602708] ? sigprocmask+0x6e/0x8a
[1124738.602708] ? _copy_to_user+0x22/0x28
[1124738.602709] ? __se_sys_rt_sigprocmask+0x93/0xbb
[1124738.602710] do_syscall_64+0x7e/0x152
[1124738.602711] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602712] RIP: 0033:0x7f1ff00d567b
[1124738.602713] Code: Bad RIP value.
[1124738.602714] RSP: 002b:00007ffd62a92890 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[1124738.602715] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f1ff00d567b
[1124738.602715] RDX: 000000000000000a RSI: 00007ffd62a928d0 RDI:
00000000ffffffff
[1124738.602716] RBP: 000000000000000a R08: 0000000000000000 R09:
00005615fea643f0
[1124738.602716] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[1124738.602717] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[1124738.602718] su S 0 24163 24160 0x00000000
[1124738.602719] Call Trace:
[1124738.602721] ? __schedule+0x413/0x464
[1124738.602722] schedule+0x80/0xab
[1124738.602723] do_wait+0x1d5/0x237
[1124738.602724] ? do_wp_page+0x497/0x4b5
[1124738.602725] kernel_wait4+0xc5/0x11a
[1124738.602726] ? __se_compat_sys_waitid+0xf5/0xf5
[1124738.602728] __se_sys_wait4+0x3c/0x8b
[1124738.602729] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602730] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.602731] ? do_sigaction+0xad/0x184
[1124738.602732] ? preempt_count_add+0x8a/0x9c
[1124738.602733] ? recalc_sigpending+0x17/0x42
[1124738.602733] ? __set_current_blocked+0x3d/0x55
[1124738.602734] ? _raw_spin_unlock_irq+0x13/0x24
[1124738.602735] ? sigprocmask+0x6e/0x8a
[1124738.602736] ? __se_sys_rt_sigprocmask+0x72/0xbb
[1124738.602737] do_syscall_64+0x7e/0x152
[1124738.602738] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602739] RIP: 0033:0x7f0b662d767b
[1124738.602740] Code: Bad RIP value.
[1124738.602741] RSP: 002b:00007ffdc4d82200 EFLAGS: 00000246 ORIG_RAX:
000000000000003d
[1124738.602741] RAX: ffffffffffffffda RBX: 00007ffdc4d82250 RCX:
00007f0b662d767b
[1124738.602742] RDX: 0000000000000002 RSI: 00007ffdc4d8224c RDI:
00000000ffffffff
[1124738.602743] RBP: 00007ffdc4d8224c R08: 0000000000000000 R09:
00007f0b66200b80
[1124738.602743] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffdc4d824c8
[1124738.602744] R13: 00000000ffffffff R14: 0000000000000000 R15:
0000000000000000
[1124738.602745] bash R running task 0 24164 24163
0x80000000
[1124738.602746] Call Trace:
[1124738.602747] sched_show_task+0x10b/0x13d
[1124738.602748] show_state_filter+0x91/0xa5
[1124738.602750] sysrq_handle_showstate+0xc/0x11
[1124738.602752] __handle_sysrq+0x8d/0x114
[1124738.602753] write_sysrq_trigger+0x2b/0x30
[1124738.602755] proc_reg_write+0x3e/0x5c
[1124738.602756] ? proc_reg_poll+0x55/0x55
[1124738.602757] vfs_write+0xc8/0x16f
[1124738.602759] ksys_write+0x6b/0xc0
[1124738.602760] do_syscall_64+0x7e/0x152
[1124738.602761] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1124738.602762] RIP: 0033:0x7f870198c0d8
[1124738.602763] Code: 00 90 48 83 ec 38 64 48 8b 04 25 28 00 00 00 48
89 44 24 28 31 c0 48 8d 05 05 96 0d 00 8b 00 85 c0 75 27 b8 01 00 00
00 0f 05 <48> 3d 00 f0 ff ff 77 60 48 8b 4c 24 28 64 48 33 0c 25 28 00
00 00
[1124738.602763] RSP: 002b:00007ffc2c78eac0 EFLAGS: 00000246 ORIG_RAX:
0000000000000001
[1124738.602764] RAX: ffffffffffffffda RBX: 0000000000000002 RCX:
00007f870198c0d8
[1124738.602765] RDX: 0000000000000002 RSI: 000055cf6a6a19b0 RDI:
0000000000000001
[1124738.602765] RBP: 000055cf6a6a19b0 R08: 000000000000000a R09:
00007f8701647740
[1124738.602766] R10: fffffffffffffbff R11: 0000000000000246 R12:
00007f8701a61760
[1124738.602766] R13: 0000000000000002 R14: 00007f8701a5c760 R15:
0000000000000002
[1124738.602767] kworker/u16:0 I 0 24175 2 0x80004000
[1124738.602770] Workqueue: 0x0 (btrfs-endio)
[1124738.602771] Call Trace:
[1124738.602773] ? __schedule+0x413/0x464
[1124738.602774] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602775] schedule+0x80/0xab
[1124738.602776] worker_thread+0x24e/0x297
[1124738.602777] kthread+0x115/0x11d
[1124738.602778] ? kthread_park+0x76/0x76
[1124738.602779] ret_from_fork+0x35/0x40
[1124738.602781] kworker/u16:5 I 0 24177 2 0x80004000
[1124738.602783] Workqueue: 0x0 (btrfs-endio)
[1124738.602784] Call Trace:
[1124738.602785] ? __schedule+0x413/0x464
[1124738.602787] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602788] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602789] schedule+0x80/0xab
[1124738.602790] worker_thread+0x24e/0x297
[1124738.602791] kthread+0x115/0x11d
[1124738.602792] ? kthread_park+0x76/0x76
[1124738.602793] ret_from_fork+0x35/0x40
[1124738.602794] kworker/2:0 I 0 24181 2 0x80004000
[1124738.602797] Workqueue: 0x0 (events_power_efficient)
[1124738.602798] Call Trace:
[1124738.602799] ? __schedule+0x413/0x464
[1124738.602801] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602802] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602803] schedule+0x80/0xab
[1124738.602804] worker_thread+0x24e/0x297
[1124738.602805] kthread+0x115/0x11d
[1124738.602806] ? kthread_park+0x76/0x76
[1124738.602807] ret_from_fork+0x35/0x40
[1124738.602809] kworker/1:2 I 0 24286 2 0x80004000
[1124738.602812] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602812] Call Trace:
[1124738.602814] ? __schedule+0x413/0x464
[1124738.602815] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602816] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602817] schedule+0x80/0xab
[1124738.602819] worker_thread+0x24e/0x297
[1124738.602820] kthread+0x115/0x11d
[1124738.602821] ? kthread_park+0x76/0x76
[1124738.602822] ret_from_fork+0x35/0x40
[1124738.602823] kworker/7:0 I 0 24307 2 0x80004000
[1124738.602826] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602827] Call Trace:
[1124738.602829] ? __schedule+0x413/0x464
[1124738.602830] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602831] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602832] schedule+0x80/0xab
[1124738.602833] worker_thread+0x24e/0x297
[1124738.602834] kthread+0x115/0x11d
[1124738.602835] ? kthread_park+0x76/0x76
[1124738.602836] ret_from_fork+0x35/0x40
[1124738.602837] kworker/2:2 I 0 24377 2 0x80004000
[1124738.602840] Workqueue: 0x0 (mm_percpu_wq)
[1124738.602841] Call Trace:
[1124738.602843] ? __schedule+0x413/0x464
[1124738.602844] ? _raw_spin_lock_irq+0x14/0x2e
[1124738.602845] ? pwq_unbound_release_workfn+0xbc/0xbc
[1124738.602846] schedule+0x80/0xab
[1124738.602847] worker_thread+0x24e/0x297
[1124738.602848] kthread+0x115/0x11d
[1124738.602849] ? kthread_park+0x76/0x76
[1124738.602850] ret_from_fork+0x35/0x40
[1124738.602852] Showing busy workqueues and worker pools:
On Sat, May 11, 2019 at 6:21 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> > [362108.291969] btrfs_tree_read_lock+0xbb/0xf1
> > [362108.291973] ? wait_woken+0x6d/0x6d
> > [362108.291978] find_parent_nodes+0x91d/0x12b8
> > [362108.291985] ? btrfs_find_all_roots_safe+0x9c/0x107
> > [362108.291988] btrfs_find_all_roots_safe+0x9c/0x107
> > [362108.291992] btrfs_find_all_roots+0x57/0x75
> > [362108.291997] btrfs_qgroup_trace_extent_post+0x37/0x7c
>
> It's qgroup.
>
> We have upstream fix for it, 38e3eebff643 ("btrfs: honor
> path->skip_locking in backref code").
> It's supported to be backported for all kernels after v4.14.
> But I'm not sure if it's backported for your kernel.
>
> As you're gentoo user, it shouldn't be hard to check the kernel source
> to find if the fix is backported.
> If not, feel free to backport for your kernel.
>
> Thanks,
> Qu
>
>
> > [362108.292002] btrfs_add_delayed_tree_ref+0x305/0x32b
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
2019-07-22 4:25 ` Nathan Dehnel
@ 2019-07-22 5:02 ` Qu Wenruo
[not found] ` <CAEEhgEsoHhR=newte0mqvQo99y2WbJjr2MH1DTWpkqMnJ3Vgig@mail.gmail.com>
0 siblings, 1 reply; 12+ messages in thread
From: Qu Wenruo @ 2019-07-22 5:02 UTC (permalink / raw)
To: Nathan Dehnel; +Cc: Btrfs BTRFS
[-- Attachment #1.1: Type: text/plain, Size: 1313 bytes --]
On 2019/7/22 下午12:25, Nathan Dehnel wrote:
> I'm still experiencing freezes with kernel 5.2. Here's a backtrace:
>
[snip]
This time, there is no backtrace of btrfs functions at all, nor nfs ones.
Is the backtrace complete? If so, it would be something else causing the
problem.
Thanks,
Qu
>
>
> On Sat, May 11, 2019 at 6:21 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>>> [362108.291969] btrfs_tree_read_lock+0xbb/0xf1
>>> [362108.291973] ? wait_woken+0x6d/0x6d
>>> [362108.291978] find_parent_nodes+0x91d/0x12b8
>>> [362108.291985] ? btrfs_find_all_roots_safe+0x9c/0x107
>>> [362108.291988] btrfs_find_all_roots_safe+0x9c/0x107
>>> [362108.291992] btrfs_find_all_roots+0x57/0x75
>>> [362108.291997] btrfs_qgroup_trace_extent_post+0x37/0x7c
>>
>> It's qgroup.
>>
>> We have upstream fix for it, 38e3eebff643 ("btrfs: honor
>> path->skip_locking in backref code").
>> It's supported to be backported for all kernels after v4.14.
>> But I'm not sure if it's backported for your kernel.
>>
>> As you're gentoo user, it shouldn't be hard to check the kernel source
>> to find if the fix is backported.
>> If not, feel free to backport for your kernel.
>>
>> Thanks,
>> Qu
>>
>>
>>> [362108.292002] btrfs_add_delayed_tree_ref+0x305/0x32b
>>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: reading/writing btrfs volume regularly freezes system
[not found] ` <CAEEhgEsoHhR=newte0mqvQo99y2WbJjr2MH1DTWpkqMnJ3Vgig@mail.gmail.com>
@ 2019-07-22 6:48 ` Qu Wenruo
0 siblings, 0 replies; 12+ messages in thread
From: Qu Wenruo @ 2019-07-22 6:48 UTC (permalink / raw)
To: Nathan Dehnel; +Cc: Btrfs BTRFS
[-- Attachment #1.1: Type: text/plain, Size: 2201 bytes --]
On 2019/7/22 下午2:15, Nathan Dehnel wrote:
> How do I get a longer backtrace?
If your dmesg output includes regular message other than the backtraces,
it should be complete.
If not, some dmesg may lost, you need to either check your system
journal (if it catches all dmesg), or enlarge kernel message buffer
(log_buf_len=<size> kernel parameter), or find a way to catch the
message early enough to avoid unnecessary noise.
Thanks,
Qu
>
> On Mon, Jul 22, 2019, 12:02 AM Qu Wenruo <quwenruo.btrfs@gmx.com
> <mailto:quwenruo.btrfs@gmx.com>> wrote:
>
>
>
> On 2019/7/22 下午12:25, Nathan Dehnel wrote:
> > I'm still experiencing freezes with kernel 5.2. Here's a backtrace:
> >
> [snip]
>
> This time, there is no backtrace of btrfs functions at all, nor nfs
> ones.
>
> Is the backtrace complete? If so, it would be something else causing the
> problem.
>
> Thanks,
> Qu
>
>
> >
> >
> > On Sat, May 11, 2019 at 6:21 AM Qu Wenruo <quwenruo.btrfs@gmx.com
> <mailto:quwenruo.btrfs@gmx.com>> wrote:
> >>
> >>
> >>> [362108.291969] btrfs_tree_read_lock+0xbb/0xf1
> >>> [362108.291973] ? wait_woken+0x6d/0x6d
> >>> [362108.291978] find_parent_nodes+0x91d/0x12b8
> >>> [362108.291985] ? btrfs_find_all_roots_safe+0x9c/0x107
> >>> [362108.291988] btrfs_find_all_roots_safe+0x9c/0x107
> >>> [362108.291992] btrfs_find_all_roots+0x57/0x75
> >>> [362108.291997] btrfs_qgroup_trace_extent_post+0x37/0x7c
> >>
> >> It's qgroup.
> >>
> >> We have upstream fix for it, 38e3eebff643 ("btrfs: honor
> >> path->skip_locking in backref code").
> >> It's supported to be backported for all kernels after v4.14.
> >> But I'm not sure if it's backported for your kernel.
> >>
> >> As you're gentoo user, it shouldn't be hard to check the kernel
> source
> >> to find if the fix is backported.
> >> If not, feel free to backport for your kernel.
> >>
> >> Thanks,
> >> Qu
> >>
> >>
> >>> [362108.292002] btrfs_add_delayed_tree_ref+0x305/0x32b
> >>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-07-22 6:48 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-22 20:37 reading/writing btrfs volume regularly freezes system Nathan Dehnel
2019-04-23 2:38 ` Chris Murphy
2019-04-24 4:21 ` Nathan Dehnel
2019-04-23 6:51 ` Nikolay Borisov
2019-04-23 7:29 ` Juergen Sauer
2019-04-24 4:35 ` Nathan Dehnel
2019-04-24 8:04 ` Qu Wenruo
2019-05-11 10:50 ` Nathan Dehnel
2019-05-11 11:21 ` Qu Wenruo
2019-07-22 4:25 ` Nathan Dehnel
2019-07-22 5:02 ` Qu Wenruo
[not found] ` <CAEEhgEsoHhR=newte0mqvQo99y2WbJjr2MH1DTWpkqMnJ3Vgig@mail.gmail.com>
2019-07-22 6:48 ` Qu Wenruo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).