* [bug] scrub on low disk space leads to Transaction aborted
@ 2021-01-09 8:53 Anand Jain
2021-01-11 0:38 ` Qu Wenruo
0 siblings, 1 reply; 3+ messages in thread
From: Anand Jain @ 2021-01-09 8:53 UTC (permalink / raw)
To: linux-btrfs
I ran scrub when disk space was low on a btrfs with some raid1 csum
errors, on a system with kernel 5.11.0-rc2+. It lead to transaction
aborted and rdonly FS.
[ 2365.314375] ------------[ cut here ]------------
[ 2365.314385] BTRFS: Transaction aborted (error -28)
[ 2365.314470] WARNING: CPU: 2 PID: 4672 at fs/btrfs/inode.c:2823
btrfs_finish_ordered_io+0x8e6/0x950 [btrfs]
<snap>
[ 2365.314921] CPU: 2 PID: 4672 Comm: kworker/u8:11 Not tainted
5.11.0-rc2+ #3
[ 2365.314933] Hardware name: LENOVO 20FAS6PN00/20FAS6PN00, BIOS
N1CET47W (1.15 ) 08/08/2016
[ 2365.314939] Workqueue: btrfs-endio-write btrfs_work_helper [btrfs]
[ 2365.315093] RIP: 0010:btrfs_finish_ordered_io+0x8e6/0x950 [btrfs]
<snap>
[ 2365.315290] Call Trace:
[ 2365.315309] BTRFS warning (device sda6):
btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
[ 2365.315305] finish_ordered_fn+0x15/0x20 [btrfs]
[ 2365.315532] btrfs_work_helper+0xcc/0x2f0 [btrfs]
[ 2365.315763] process_one_work+0x1ee/0x390
[ 2365.315781] worker_thread+0x50/0x3d0
[ 2365.315797] kthread+0x114/0x150
[ 2365.315811] ? process_one_work+0x390/0x390
[ 2365.315824] ? kthread_park+0x90/0x90
[ 2365.315836] ret_from_fork+0x22/0x30
[ 2365.315861] ---[ end trace b0e289a2c3d424e8 ]---
[ 2365.315871] BTRFS warning (device sda6):
btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
[ 2365.316713] BTRFS warning (device sda6):
btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
[ 2365.316796] BTRFS warning (device sda6): Skipping commit of aborted
transaction.
[ 2365.316809] BTRFS: error (device sda6) in cleanup_transaction:1938:
errno=-30 Readonly filesystem
[ 2365.316824] BTRFS info (device sda6): forced readonly
[ 2365.359195] BTRFS info (device sda6): scrub: not finished on devid 2
with status: -125
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [bug] scrub on low disk space leads to Transaction aborted
2021-01-09 8:53 [bug] scrub on low disk space leads to Transaction aborted Anand Jain
@ 2021-01-11 0:38 ` Qu Wenruo
2021-01-11 3:30 ` Anand Jain
0 siblings, 1 reply; 3+ messages in thread
From: Qu Wenruo @ 2021-01-11 0:38 UTC (permalink / raw)
To: Anand Jain, linux-btrfs
On 2021/1/9 下午4:53, Anand Jain wrote:
>
> I ran scrub when disk space was low on a btrfs with some raid1 csum
> errors, on a system with kernel 5.11.0-rc2+. It lead to transaction
> aborted and rdonly FS.
>
Would you please provide `btrfs fi usage` output?
There is a bug in recent kernel that global rsv is not increased
properly so that we have a much higher chance to exhaust metadata space.
Thanks,
Qu
> [ 2365.314375] ------------[ cut here ]------------
> [ 2365.314385] BTRFS: Transaction aborted (error -28)
> [ 2365.314470] WARNING: CPU: 2 PID: 4672 at fs/btrfs/inode.c:2823
> btrfs_finish_ordered_io+0x8e6/0x950 [btrfs]
> <snap>
> [ 2365.314921] CPU: 2 PID: 4672 Comm: kworker/u8:11 Not tainted
> 5.11.0-rc2+ #3
> [ 2365.314933] Hardware name: LENOVO 20FAS6PN00/20FAS6PN00, BIOS
> N1CET47W (1.15 ) 08/08/2016
> [ 2365.314939] Workqueue: btrfs-endio-write btrfs_work_helper [btrfs]
> [ 2365.315093] RIP: 0010:btrfs_finish_ordered_io+0x8e6/0x950 [btrfs]
> <snap>
> [ 2365.315290] Call Trace:
> [ 2365.315309] BTRFS warning (device sda6):
> btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
> [ 2365.315305] finish_ordered_fn+0x15/0x20 [btrfs]
> [ 2365.315532] btrfs_work_helper+0xcc/0x2f0 [btrfs]
> [ 2365.315763] process_one_work+0x1ee/0x390
> [ 2365.315781] worker_thread+0x50/0x3d0
> [ 2365.315797] kthread+0x114/0x150
> [ 2365.315811] ? process_one_work+0x390/0x390
> [ 2365.315824] ? kthread_park+0x90/0x90
> [ 2365.315836] ret_from_fork+0x22/0x30
> [ 2365.315861] ---[ end trace b0e289a2c3d424e8 ]---
> [ 2365.315871] BTRFS warning (device sda6):
> btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
> [ 2365.316713] BTRFS warning (device sda6):
> btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
> [ 2365.316796] BTRFS warning (device sda6): Skipping commit of aborted
> transaction.
> [ 2365.316809] BTRFS: error (device sda6) in cleanup_transaction:1938:
> errno=-30 Readonly filesystem
> [ 2365.316824] BTRFS info (device sda6): forced readonly
> [ 2365.359195] BTRFS info (device sda6): scrub: not finished on devid 2
> with status: -125
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [bug] scrub on low disk space leads to Transaction aborted
2021-01-11 0:38 ` Qu Wenruo
@ 2021-01-11 3:30 ` Anand Jain
0 siblings, 0 replies; 3+ messages in thread
From: Anand Jain @ 2021-01-11 3:30 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
On 11/1/21 8:38 am, Qu Wenruo wrote:
>
>
> On 2021/1/9 下午4:53, Anand Jain wrote:
>>
>> I ran scrub when disk space was low on a btrfs with some raid1 csum
>> errors, on a system with kernel 5.11.0-rc2+. It lead to transaction
>> aborted and rdonly FS.
>>
>
> Would you please provide `btrfs fi usage` output?
>
> There is a bug in recent kernel that global rsv is not increased
> properly so that we have a much higher chance to exhaust metadata space.
>
Now I have made some space in the filesystem. So the usage below may not
be relevant at the time of the problem.
# btrfs fi usage /Volumes/
Overall:
Device size: 154.97GiB
Device allocated: 153.12GiB
Device unallocated: 1.85GiB
Device missing: 76.56GiB
Used: 147.63GiB
Free (estimated): 3.35GiB (min: 3.35GiB)
Data ratio: 2.00
Metadata ratio: 2.00
Global reserve: 181.56MiB (used: 0.00B)
Data,RAID1: Size:75.53GiB, Used:73.11GiB (96.79%)
/dev/sda6 75.53GiB
/dev/sdb2 75.53GiB
Metadata,RAID1: Size:1.00GiB, Used:724.14MiB (70.72%)
/dev/sda6 1.00GiB
/dev/sdb2 1.00GiB
System,RAID1: Size:32.00MiB, Used:16.00KiB (0.05%)
/dev/sda6 32.00MiB
/dev/sdb2 32.00MiB
Unallocated:
/dev/sda6 1.85GiB
/dev/sdb2 1.00MiB
> Thanks,
> Qu
>
>> [ 2365.314375] ------------[ cut here ]------------
>> [ 2365.314385] BTRFS: Transaction aborted (error -28)
>> [ 2365.314470] WARNING: CPU: 2 PID: 4672 at fs/btrfs/inode.c:2823
>> btrfs_finish_ordered_io+0x8e6/0x950 [btrfs]
>> <snap>
>> [ 2365.314921] CPU: 2 PID: 4672 Comm: kworker/u8:11 Not tainted
>> 5.11.0-rc2+ #3
>> [ 2365.314933] Hardware name: LENOVO 20FAS6PN00/20FAS6PN00, BIOS
>> N1CET47W (1.15 ) 08/08/2016
>> [ 2365.314939] Workqueue: btrfs-endio-write btrfs_work_helper [btrfs]
>> [ 2365.315093] RIP: 0010:btrfs_finish_ordered_io+0x8e6/0x950 [btrfs]
>> <snap>
>> [ 2365.315290] Call Trace:
>> [ 2365.315309] BTRFS warning (device sda6):
>> btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
>> [ 2365.315305] finish_ordered_fn+0x15/0x20 [btrfs]
>> [ 2365.315532] btrfs_work_helper+0xcc/0x2f0 [btrfs]
>> [ 2365.315763] process_one_work+0x1ee/0x390
>> [ 2365.315781] worker_thread+0x50/0x3d0
>> [ 2365.315797] kthread+0x114/0x150
>> [ 2365.315811] ? process_one_work+0x390/0x390
>> [ 2365.315824] ? kthread_park+0x90/0x90
>> [ 2365.315836] ret_from_fork+0x22/0x30
>> [ 2365.315861] ---[ end trace b0e289a2c3d424e8 ]---
>> [ 2365.315871] BTRFS warning (device sda6):
>> btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
>> [ 2365.316713] BTRFS warning (device sda6):
>> btrfs_finish_ordered_io:2823: Aborting unused transaction(No space left).
>> [ 2365.316796] BTRFS warning (device sda6): Skipping commit of aborted
>> transaction.
>> [ 2365.316809] BTRFS: error (device sda6) in cleanup_transaction:1938:
>> errno=-30 Readonly filesystem
>> [ 2365.316824] BTRFS info (device sda6): forced readonly
>> [ 2365.359195] BTRFS info (device sda6): scrub: not finished on devid 2
>> with status: -125
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-01-11 3:31 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-09 8:53 [bug] scrub on low disk space leads to Transaction aborted Anand Jain
2021-01-11 0:38 ` Qu Wenruo
2021-01-11 3:30 ` Anand Jain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).