* [PATCH] btrfs: don't hold CPU for too long when defragging a file
@ 2022-01-30 12:53 Qu Wenruo
2022-02-01 14:53 ` David Sterba
0 siblings, 1 reply; 6+ messages in thread
From: Qu Wenruo @ 2022-01-30 12:53 UTC (permalink / raw)
To: linux-btrfs
There is a user report about "btrfs filesystem defrag" causing 120s
timeout problem.
For btrfs_defrag_file() it will iterate all file extents if called from
defrag ioctl, thus it can take a long time.
There is no reason not to release the CPU during such a long operation.
Add cond_resched() after defragged one cluster.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
This is a long existing bug, and it should also be applied to older
kernels.
But at v5.16 we have a large rework on defrag (which caused quite some
regression though), thus there will be a specific backport for kernels
older than v5.16.
---
fs/btrfs/ioctl.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 961c38660b55..bf90e809f3b2 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -1695,6 +1695,7 @@ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra,
ret = 0;
break;
}
+ cond_resched();
}
if (ra_allocated)
--
2.35.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: don't hold CPU for too long when defragging a file
2022-01-30 12:53 [PATCH] btrfs: don't hold CPU for too long when defragging a file Qu Wenruo
@ 2022-02-01 14:53 ` David Sterba
2022-02-01 15:13 ` piorunz
2022-02-02 0:00 ` Qu Wenruo
0 siblings, 2 replies; 6+ messages in thread
From: David Sterba @ 2022-02-01 14:53 UTC (permalink / raw)
To: Qu Wenruo; +Cc: linux-btrfs
On Sun, Jan 30, 2022 at 08:53:15PM +0800, Qu Wenruo wrote:
> There is a user report about "btrfs filesystem defrag" causing 120s
> timeout problem.
Do you have link of the report please?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: don't hold CPU for too long when defragging a file
2022-02-01 14:53 ` David Sterba
@ 2022-02-01 15:13 ` piorunz
2022-02-02 15:42 ` David Sterba
2022-02-02 0:00 ` Qu Wenruo
1 sibling, 1 reply; 6+ messages in thread
From: piorunz @ 2022-02-01 15:13 UTC (permalink / raw)
To: dsterba, Qu Wenruo, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 2965 bytes --]
On 01/02/2022 14:53, David Sterba wrote:
> On Sun, Jan 30, 2022 at 08:53:15PM +0800, Qu Wenruo wrote:
>> There is a user report about "btrfs filesystem defrag" causing 120s
>> timeout problem.
>
> Do you have link of the report please?
That would be most likely my topic here on this group. I had
conversation with Qu while discussing the topic and he commented 120s
kernel timeouts while defragging.
Topic is here on this group: "fstab autodegrag with 5.10 & 5.15 kernels,
Debian?" (typo is hilarious haha)
Link:
https://lore.kernel.org/linux-btrfs/94244e1a-7ccd-6f68-9052-5c01876b3939@gmx.com/T/#t
This is filesystem in question:
$ sudo btrfs filesystem usage /home
Overall:
Device size: 7.03TiB
Device allocated: 3.93TiB
Device unallocated: 3.10TiB
Device missing: 0.00B
Used: 3.90TiB
Free (estimated): 1.56TiB (min: 1.56TiB)
Free (statfs, df): 1.56TiB
Data ratio: 2.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)
Multiple profiles: no
Data,RAID10: Size:1.96TiB, Used:1.94TiB (99.46%)
/dev/sdd3 1001.00GiB
/dev/sde3 1001.00GiB
/dev/sda4 1001.00GiB
/dev/sdf4 1001.00GiB
Metadata,RAID10: Size:9.00GiB, Used:3.73GiB (41.50%)
/dev/sdd3 4.50GiB
/dev/sde3 4.50GiB
/dev/sda4 4.50GiB
/dev/sdf4 4.50GiB
System,RAID10: Size:64.00MiB, Used:240.00KiB (0.37%)
/dev/sdd3 32.00MiB
/dev/sde3 32.00MiB
/dev/sda4 32.00MiB
/dev/sdf4 32.00MiB
Unallocated:
/dev/sdd3 793.48GiB
/dev/sde3 793.48GiB
/dev/sda4 793.48GiB
/dev/sdf4 793.48GiB
Btrfs RAID10, consists of 4x 2TB spinners.
Mount options are:
/home btrfs noatime,space_cache=v2,compress-force=zstd:3 0 2
I never had any timeouts for last 2 years on this server, performance is
very good for amount of load I have on them. I only experienced timeouts
once I decided to defrag.
Snippet from sudo journalctl --utc -k -b -1 has been attached.
Please notice how different tasks and pids are being timeouted while
defrag took entire filesystem for itself, depriving everything else from
accessing. Errors continued after that but I just cut this for you so
you have the idea. Ideally, defrag should not take entire I/O for
itself, but filesystem should continue to work as usual. Like SMART Long
background test. It does not prevent use of the drive. That's how I feel
defrag should work.
I will be happy to provide more details if required.
--
With kindest regards, Piotr.
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀⠀⠀
[-- Attachment #2: journal.log --]
[-- Type: text/x-log, Size: 13020 bytes --]
Jan 25 23:22:48 ttbit.mine.bz kernel: INFO: task btrfs-transacti:1390 blocked for more than 120 seconds.
Jan 25 23:22:48 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 25 23:22:48 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 25 23:22:48 ttbit.mine.bz kernel: task:btrfs-transacti state:D stack: 0 pid: 1390 ppid: 2 flags:0x00004000
Jan 25 23:22:48 ttbit.mine.bz kernel: Call Trace:
Jan 25 23:22:48 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 25 23:22:48 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 25 23:22:48 ttbit.mine.bz kernel: wait_for_commit+0x58/0x80 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? add_wait_queue_exclusive+0x70/0x70
Jan 25 23:22:48 ttbit.mine.bz kernel: btrfs_commit_transaction+0x907/0xb40 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? start_transaction+0xd2/0x580 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? usleep_range+0x80/0x80
Jan 25 23:22:48 ttbit.mine.bz kernel: transaction_kthread+0x14c/0x170 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? btrfs_cleanup_transaction.isra.0+0x590/0x590 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: kthread+0x11b/0x140
Jan 25 23:22:48 ttbit.mine.bz kernel: ? __kthread_bind_mask+0x60/0x60
Jan 25 23:22:48 ttbit.mine.bz kernel: ret_from_fork+0x22/0x30
Jan 25 23:22:48 ttbit.mine.bz kernel: INFO: task StreamT~ns #743:2328701 blocked for more than 120 seconds.
Jan 25 23:22:48 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 25 23:22:48 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 25 23:22:48 ttbit.mine.bz kernel: task:StreamT~ns #743 state:D stack: 0 pid:2328701 ppid: 1 flags:0x00004000
Jan 25 23:22:48 ttbit.mine.bz kernel: Call Trace:
Jan 25 23:22:48 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 25 23:22:48 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 25 23:22:48 ttbit.mine.bz kernel: wait_for_commit+0x58/0x80 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? add_wait_queue_exclusive+0x70/0x70
Jan 25 23:22:48 ttbit.mine.bz kernel: btrfs_commit_transaction+0x907/0xb40 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? _cond_resched+0x16/0x40
Jan 25 23:22:48 ttbit.mine.bz kernel: ? dput+0xca/0x2f0
Jan 25 23:22:48 ttbit.mine.bz kernel: btrfs_sync_file+0x399/0x500 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: __x64_sys_fsync+0x34/0x60
Jan 25 23:22:48 ttbit.mine.bz kernel: do_syscall_64+0x33/0x80
Jan 25 23:22:48 ttbit.mine.bz kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 25 23:22:48 ttbit.mine.bz kernel: RIP: 0033:0x7fb3b4bcc5eb
Jan 25 23:22:48 ttbit.mine.bz kernel: RSP: 002b:00007fb39118a900 EFLAGS: 00000293 ORIG_RAX: 000000000000004a
Jan 25 23:22:48 ttbit.mine.bz kernel: RAX: ffffffffffffffda RBX: 00007fb36aa80280 RCX: 00007fb3b4bcc5eb
Jan 25 23:22:48 ttbit.mine.bz kernel: RDX: 00007fb3b4bd6360 RSI: 0000000000000001 RDI: 0000000000000057
Jan 25 23:22:48 ttbit.mine.bz kernel: RBP: 000000000000096e R08: 0000000000000000 R09: 00007fb36ad551c0
Jan 25 23:22:48 ttbit.mine.bz kernel: R10: 0000000000000040 R11: 0000000000000293 R12: 00007fb3b08e27a9
Jan 25 23:22:48 ttbit.mine.bz kernel: R13: 000000000000010b R14: 00007fb36aeb17c0 R15: 00007fb36aafb3d0
Jan 25 23:22:48 ttbit.mine.bz kernel: INFO: task monerod:2211804 blocked for more than 120 seconds.
Jan 25 23:22:48 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 25 23:22:48 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 25 23:22:48 ttbit.mine.bz kernel: task:monerod state:D stack: 0 pid:2211804 ppid:2211786 flags:0x00004000
Jan 25 23:22:48 ttbit.mine.bz kernel: Call Trace:
Jan 25 23:22:48 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 25 23:22:48 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 25 23:22:48 ttbit.mine.bz kernel: wait_for_commit+0x58/0x80 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? add_wait_queue_exclusive+0x70/0x70
Jan 25 23:22:48 ttbit.mine.bz kernel: btrfs_commit_transaction+0x907/0xb40 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: ? btrfs_wait_ordered_range+0x190/0x210 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: btrfs_sync_file+0x399/0x500 [btrfs]
Jan 25 23:22:48 ttbit.mine.bz kernel: __x64_sys_fdatasync+0x47/0x80
Jan 25 23:22:48 ttbit.mine.bz kernel: do_syscall_64+0x33/0x80
Jan 25 23:22:48 ttbit.mine.bz kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 25 23:22:48 ttbit.mine.bz kernel: RIP: 0033:0x7f1717b80b7b
Jan 25 23:22:48 ttbit.mine.bz kernel: RSP: 002b:00007ef4e7ffcd90 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
Jan 25 23:22:48 ttbit.mine.bz kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f1717b80b7b
Jan 25 23:22:48 ttbit.mine.bz kernel: RDX: 0000000001e94617 RSI: 0000000000000000 RDI: 0000000000000005
Jan 25 23:22:48 ttbit.mine.bz kernel: RBP: 00007ef4d4dbf9d8 R08: 0000000000000000 R09: 00007ef4d8504a60
Jan 25 23:22:48 ttbit.mine.bz kernel: R10: 0000001e91e3a000 R11: 0000000000000293 R12: 00007ef4e7ffce30
Jan 25 23:22:48 ttbit.mine.bz kernel: R13: 00000000000001a0 R14: 0000561ac126d9d0 R15: 0000561ac126f1e0
Jan 26 01:47:48 ttbit.mine.bz kernel: INFO: task btrfs-transacti:1390 blocked for more than 120 seconds.
Jan 26 01:47:48 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 26 01:47:48 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 26 01:47:48 ttbit.mine.bz kernel: task:btrfs-transacti state:D stack: 0 pid: 1390 ppid: 2 flags:0x00004000
Jan 26 01:47:48 ttbit.mine.bz kernel: Call Trace:
Jan 26 01:47:48 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 26 01:47:48 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 26 01:47:48 ttbit.mine.bz kernel: wait_for_commit+0x58/0x80 [btrfs]
Jan 26 01:47:48 ttbit.mine.bz kernel: ? add_wait_queue_exclusive+0x70/0x70
Jan 26 01:47:48 ttbit.mine.bz kernel: btrfs_commit_transaction+0x907/0xb40 [btrfs]
Jan 26 01:47:48 ttbit.mine.bz kernel: ? start_transaction+0xd2/0x580 [btrfs]
Jan 26 01:47:48 ttbit.mine.bz kernel: ? usleep_range+0x80/0x80
Jan 26 01:47:48 ttbit.mine.bz kernel: transaction_kthread+0x14c/0x170 [btrfs]
Jan 26 01:47:48 ttbit.mine.bz kernel: ? btrfs_cleanup_transaction.isra.0+0x590/0x590 [btrfs]
Jan 26 01:47:48 ttbit.mine.bz kernel: kthread+0x11b/0x140
Jan 26 01:47:48 ttbit.mine.bz kernel: ? __kthread_bind_mask+0x60/0x60
Jan 26 01:47:48 ttbit.mine.bz kernel: ret_from_fork+0x22/0x30
Jan 26 08:54:44 ttbit.mine.bz kernel: INFO: task worker:2371140 blocked for more than 120 seconds.
Jan 26 08:54:44 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 26 08:54:44 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 26 08:54:44 ttbit.mine.bz kernel: task:worker state:D stack: 0 pid:2371140 ppid: 1 flags:0x00000320
Jan 26 08:54:44 ttbit.mine.bz kernel: Call Trace:
Jan 26 08:54:44 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 26 08:54:44 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 26 08:54:44 ttbit.mine.bz kernel: rwsem_down_write_slowpath+0x242/0x4c0
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __aa_label_next_not_in_set+0x58/0xb0
Jan 26 08:54:44 ttbit.mine.bz kernel: btrfs_file_write_iter+0x72/0x650 [btrfs]
Jan 26 08:54:44 ttbit.mine.bz kernel: ? iovec_from_user+0x9e/0x170
Jan 26 08:54:44 ttbit.mine.bz kernel: do_iter_readv_writev+0x152/0x1b0
Jan 26 08:54:44 ttbit.mine.bz kernel: do_iter_write+0x7c/0x1b0
Jan 26 08:54:44 ttbit.mine.bz kernel: vfs_writev+0xa4/0x140
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __seccomp_filter+0x7f/0x680
Jan 26 08:54:44 ttbit.mine.bz kernel: ? _copy_from_user+0x28/0x60
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __fget_files+0x97/0xc0
Jan 26 08:54:44 ttbit.mine.bz kernel: __x64_sys_pwritev+0xad/0xf0
Jan 26 08:54:44 ttbit.mine.bz kernel: do_syscall_64+0x33/0x80
Jan 26 08:54:44 ttbit.mine.bz kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 26 08:54:44 ttbit.mine.bz kernel: RIP: 0033:0x7f1a385d2f40
Jan 26 08:54:44 ttbit.mine.bz kernel: RSP: 002b:00007f197c9dd5a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000128
Jan 26 08:54:44 ttbit.mine.bz kernel: RAX: ffffffffffffffda RBX: 00007f19285f6570 RCX: 00007f1a385d2f40
Jan 26 08:54:44 ttbit.mine.bz kernel: RDX: 0000000000000022 RSI: 00007f19e0041930 RDI: 0000000000000016
Jan 26 08:54:44 ttbit.mine.bz kernel: RBP: 000056490422c6b0 R08: 0000000000000000 R09: 0000000000000000
Jan 26 08:54:44 ttbit.mine.bz kernel: R10: 0000000d69610000 R11: 0000000000000246 R12: 000056490386f492
Jan 26 08:54:44 ttbit.mine.bz kernel: R13: 000056490422c718 R14: 00007f19e0041b60 R15: 00007f1984000b90
Jan 26 08:54:44 ttbit.mine.bz kernel: INFO: task worker:2371141 blocked for more than 120 seconds.
Jan 26 08:54:44 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 26 08:54:44 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 26 08:54:44 ttbit.mine.bz kernel: task:worker state:D stack: 0 pid:2371141 ppid: 1 flags:0x00000320
Jan 26 08:54:44 ttbit.mine.bz kernel: Call Trace:
Jan 26 08:54:44 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 26 08:54:44 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 26 08:54:44 ttbit.mine.bz kernel: rwsem_down_write_slowpath+0x242/0x4c0
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __aa_label_next_not_in_set+0x58/0xb0
Jan 26 08:54:44 ttbit.mine.bz kernel: btrfs_file_write_iter+0x72/0x650 [btrfs]
Jan 26 08:54:44 ttbit.mine.bz kernel: ? iovec_from_user+0x9e/0x170
Jan 26 08:54:44 ttbit.mine.bz kernel: do_iter_readv_writev+0x152/0x1b0
Jan 26 08:54:44 ttbit.mine.bz kernel: do_iter_write+0x7c/0x1b0
Jan 26 08:54:44 ttbit.mine.bz kernel: vfs_writev+0xa4/0x140
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __seccomp_filter+0x7f/0x680
Jan 26 08:54:44 ttbit.mine.bz kernel: ? _copy_from_user+0x28/0x60
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __fget_files+0x97/0xc0
Jan 26 08:54:44 ttbit.mine.bz kernel: __x64_sys_pwritev+0xad/0xf0
Jan 26 08:54:44 ttbit.mine.bz kernel: do_syscall_64+0x33/0x80
Jan 26 08:54:44 ttbit.mine.bz kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 26 08:54:44 ttbit.mine.bz kernel: RIP: 0033:0x7f1a385d2f40
Jan 26 08:54:44 ttbit.mine.bz kernel: RSP: 002b:00007f199effc5a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000128
Jan 26 08:54:44 ttbit.mine.bz kernel: RAX: ffffffffffffffda RBX: 00007f19d49d4570 RCX: 00007f1a385d2f40
Jan 26 08:54:44 ttbit.mine.bz kernel: RDX: 0000000000000022 RSI: 00007f19e0132ce0 RDI: 0000000000000016
Jan 26 08:54:44 ttbit.mine.bz kernel: RBP: 000056490422c6b0 R08: 0000000000000000 R09: 0000000000000000
Jan 26 08:54:44 ttbit.mine.bz kernel: R10: 0000000d69640000 R11: 0000000000000246 R12: 000056490386f492
Jan 26 08:54:44 ttbit.mine.bz kernel: R13: 000056490422c718 R14: 00007f19e0132f10 R15: 00007f1950000b90
Jan 26 08:54:44 ttbit.mine.bz kernel: INFO: task worker:2371142 blocked for more than 120 seconds.
Jan 26 08:54:44 ttbit.mine.bz kernel: Not tainted 5.10.0-11-amd64 #1 Debian 5.10.92-1
Jan 26 08:54:44 ttbit.mine.bz kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 26 08:54:44 ttbit.mine.bz kernel: task:worker state:D stack: 0 pid:2371142 ppid: 1 flags:0x00000320
Jan 26 08:54:44 ttbit.mine.bz kernel: Call Trace:
Jan 26 08:54:44 ttbit.mine.bz kernel: __schedule+0x282/0x870
Jan 26 08:54:44 ttbit.mine.bz kernel: ? rwsem_spin_on_owner+0x74/0xd0
Jan 26 08:54:44 ttbit.mine.bz kernel: schedule+0x46/0xb0
Jan 26 08:54:44 ttbit.mine.bz kernel: rwsem_down_write_slowpath+0x242/0x4c0
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __aa_label_next_not_in_set+0x58/0xb0
Jan 26 08:54:44 ttbit.mine.bz kernel: btrfs_file_write_iter+0x72/0x650 [btrfs]
Jan 26 08:54:44 ttbit.mine.bz kernel: do_iter_readv_writev+0x152/0x1b0
Jan 26 08:54:44 ttbit.mine.bz kernel: do_iter_write+0x7c/0x1b0
Jan 26 08:54:44 ttbit.mine.bz kernel: vfs_writev+0xa4/0x140
Jan 26 08:54:44 ttbit.mine.bz kernel: ? _copy_from_user+0x28/0x60
Jan 26 08:54:44 ttbit.mine.bz kernel: ? __fget_files+0x97/0xc0
Jan 26 08:54:44 ttbit.mine.bz kernel: __x64_sys_pwritev+0xad/0xf0
Jan 26 08:54:44 ttbit.mine.bz kernel: do_syscall_64+0x33/0x80
Jan 26 08:54:44 ttbit.mine.bz kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 26 08:54:44 ttbit.mine.bz kernel: RIP: 0033:0x7f1a385d2f40
Jan 26 08:54:44 ttbit.mine.bz kernel: RSP: 002b:00007f19b77fd5a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000128
Jan 26 08:54:44 ttbit.mine.bz kernel: RAX: ffffffffffffffda RBX: 00007f19b4aedb90 RCX: 00007f1a385d2f40
Jan 26 08:54:44 ttbit.mine.bz kernel: RDX: 0000000000000004 RSI: 00007f19e0003120 RDI: 0000000000000016
Jan 26 08:54:44 ttbit.mine.bz kernel: RBP: 000056490422c6b0 R08: 0000000000000000 R09: 0000000000000000
Jan 26 08:54:44 ttbit.mine.bz kernel: R10: 0000000d69480000 R11: 0000000000000246 R12: 000056490386f492
Jan 26 08:54:44 ttbit.mine.bz kernel: R13: 000056490422c718 R14: 00007f19e0042b00 R15: 00005649044cb430
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: don't hold CPU for too long when defragging a file
2022-02-01 14:53 ` David Sterba
2022-02-01 15:13 ` piorunz
@ 2022-02-02 0:00 ` Qu Wenruo
2022-02-02 14:39 ` David Sterba
1 sibling, 1 reply; 6+ messages in thread
From: Qu Wenruo @ 2022-02-02 0:00 UTC (permalink / raw)
To: dsterba, Qu Wenruo, linux-btrfs
On 2022/2/1 22:53, David Sterba wrote:
> On Sun, Jan 30, 2022 at 08:53:15PM +0800, Qu Wenruo wrote:
>> There is a user report about "btrfs filesystem defrag" causing 120s
>> timeout problem.
>
> Do you have link of the report please?
Here it is, not really a formal one, but just mentjioned in a thread:
https://lore.kernel.org/linux-btrfs/10e51417-2203-f0a4-2021-86c8511cc367@gmx.com/
Thanks,
Qu
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: don't hold CPU for too long when defragging a file
2022-02-02 0:00 ` Qu Wenruo
@ 2022-02-02 14:39 ` David Sterba
0 siblings, 0 replies; 6+ messages in thread
From: David Sterba @ 2022-02-02 14:39 UTC (permalink / raw)
To: Qu Wenruo; +Cc: dsterba, Qu Wenruo, linux-btrfs
On Wed, Feb 02, 2022 at 08:00:45AM +0800, Qu Wenruo wrote:
>
>
> On 2022/2/1 22:53, David Sterba wrote:
> > On Sun, Jan 30, 2022 at 08:53:15PM +0800, Qu Wenruo wrote:
> >> There is a user report about "btrfs filesystem defrag" causing 120s
> >> timeout problem.
> >
> > Do you have link of the report please?
>
> Here it is, not really a formal one, but just mentjioned in a thread:
>
> https://lore.kernel.org/linux-btrfs/10e51417-2203-f0a4-2021-86c8511cc367@gmx.com/
Thanks, that's much better than a vague "there's a report".
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: don't hold CPU for too long when defragging a file
2022-02-01 15:13 ` piorunz
@ 2022-02-02 15:42 ` David Sterba
0 siblings, 0 replies; 6+ messages in thread
From: David Sterba @ 2022-02-02 15:42 UTC (permalink / raw)
To: piorunz; +Cc: dsterba, Qu Wenruo, linux-btrfs
On Tue, Feb 01, 2022 at 03:13:33PM +0000, piorunz wrote:
> On 01/02/2022 14:53, David Sterba wrote:
> > On Sun, Jan 30, 2022 at 08:53:15PM +0800, Qu Wenruo wrote:
> >> There is a user report about "btrfs filesystem defrag" causing 120s
> >> timeout problem.
> >
> > Do you have link of the report please?
>
> That would be most likely my topic here on this group. I had
> conversation with Qu while discussing the topic and he commented 120s
> kernel timeouts while defragging.
>
> Topic is here on this group: "fstab autodegrag with 5.10 & 5.15 kernels,
> Debian?" (typo is hilarious haha)
>
> Link:
> https://lore.kernel.org/linux-btrfs/94244e1a-7ccd-6f68-9052-5c01876b3939@gmx.com/T/#t
>
> This is filesystem in question:
> $ sudo btrfs filesystem usage /home
> Overall:
> Device size: 7.03TiB
> Device allocated: 3.93TiB
> Device unallocated: 3.10TiB
> Device missing: 0.00B
> Used: 3.90TiB
> Free (estimated): 1.56TiB (min: 1.56TiB)
> Free (statfs, df): 1.56TiB
> Data ratio: 2.00
> Metadata ratio: 2.00
> Global reserve: 512.00MiB (used: 0.00B)
> Multiple profiles: no
>
> Data,RAID10: Size:1.96TiB, Used:1.94TiB (99.46%)
> /dev/sdd3 1001.00GiB
> /dev/sde3 1001.00GiB
> /dev/sda4 1001.00GiB
> /dev/sdf4 1001.00GiB
>
> Metadata,RAID10: Size:9.00GiB, Used:3.73GiB (41.50%)
> /dev/sdd3 4.50GiB
> /dev/sde3 4.50GiB
> /dev/sda4 4.50GiB
> /dev/sdf4 4.50GiB
>
> System,RAID10: Size:64.00MiB, Used:240.00KiB (0.37%)
> /dev/sdd3 32.00MiB
> /dev/sde3 32.00MiB
> /dev/sda4 32.00MiB
> /dev/sdf4 32.00MiB
>
> Unallocated:
> /dev/sdd3 793.48GiB
> /dev/sde3 793.48GiB
> /dev/sda4 793.48GiB
> /dev/sdf4 793.48GiB
>
>
> Btrfs RAID10, consists of 4x 2TB spinners.
> Mount options are:
> /home btrfs noatime,space_cache=v2,compress-force=zstd:3 0 2
>
> I never had any timeouts for last 2 years on this server, performance is
> very good for amount of load I have on them. I only experienced timeouts
> once I decided to defrag.
>
> Snippet from sudo journalctl --utc -k -b -1 has been attached.
> Please notice how different tasks and pids are being timeouted while
> defrag took entire filesystem for itself, depriving everything else from
> accessing. Errors continued after that but I just cut this for you so
> you have the idea. Ideally, defrag should not take entire I/O for
> itself, but filesystem should continue to work as usual. Like SMART Long
> background test. It does not prevent use of the drive. That's how I feel
> defrag should work.
>
> I will be happy to provide more details if required.
Thanks, not necessary. I missed the original discussion and there were
several defrag bugs floating around so this is for my reference and to
save it in the patch too.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-02-02 15:42 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-30 12:53 [PATCH] btrfs: don't hold CPU for too long when defragging a file Qu Wenruo
2022-02-01 14:53 ` David Sterba
2022-02-01 15:13 ` piorunz
2022-02-02 15:42 ` David Sterba
2022-02-02 0:00 ` Qu Wenruo
2022-02-02 14:39 ` David Sterba
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.