* df shows no available space in 5.4.1
@ 2019-12-06 21:26 Martin Raiber
2019-12-06 22:35 ` Chris Murphy
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: Martin Raiber @ 2019-12-06 21:26 UTC (permalink / raw)
To: linux-btrfs
Hi,
with kernel 5.4.1 I have the problem that df shows 100% space used. I
can still write to the btrfs volume, but my software looks at the
available space and starts deleting stuff if statfs() says there is a
low amount of available space.
# df -h
Filesystem Size Used Avail
Use% Mounted on
...
/dev/loop0 7.4T 623G 0
100% /media/backup
...
statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
# btrfs fi usage /media/backup
Overall:
Device size: 7.40TiB
Device allocated: 671.02GiB
Device unallocated: 6.74TiB
Device missing: 0.00B
Used: 622.49GiB
Free (estimated): 6.79TiB (min: 6.79TiB)
Data ratio: 1.00
Metadata ratio: 1.00
Global reserve: 512.00MiB (used: 0.00B)
Data,single: Size:666.01GiB, Used:617.95GiB
/dev/loop0 666.01GiB
Metadata,single: Size:5.01GiB, Used:4.54GiB
/dev/loop0 5.01GiB
System,single: Size:4.00MiB, Used:96.00KiB
/dev/loop0 4.00MiB
Unallocated:
/dev/loop0 6.74TiB
# btrfs fi df /media/backup
Data, single: total=666.01GiB, used=617.95GiB
System, single: total=4.00MiB, used=96.00KiB
Metadata, single: total=5.01GiB, used=4.54GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
# mount
...
/dev/loop0 on /media/backup type btrfs
(rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/)
...
(I remounted with enospc_debug and the available space did not change...)
Regards,
Martin Raiber
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-06 21:26 df shows no available space in 5.4.1 Martin Raiber
@ 2019-12-06 22:35 ` Chris Murphy
2019-12-06 22:51 ` Martin Raiber
2019-12-08 18:12 ` Zygo Blaxell
2019-12-07 7:28 ` Qu Wenruo
` (2 subsequent siblings)
3 siblings, 2 replies; 13+ messages in thread
From: Chris Murphy @ 2019-12-06 22:35 UTC (permalink / raw)
To: Martin Raiber, Tomasz Chmielewski; +Cc: linux-btrfs, Qu Wenruo
On Fri, Dec 6, 2019 at 2:26 PM Martin Raiber <martin@urbackup.org> wrote:
>
> Hi,
>
> with kernel 5.4.1 I have the problem that df shows 100% space used. I
> can still write to the btrfs volume, but my software looks at the
> available space and starts deleting stuff if statfs() says there is a
> low amount of available space.
This is the second bug like this reported in as many days against 5.4.1.
Does this happen with an older kernel? Any 5.3 kernel or 5.2.15+ or
any 5.1 kernel? Or heck, even 5.4? :P
>
> # df -h
> Filesystem Size Used Avail
> Use% Mounted on
> ...
> /dev/loop0 7.4T 623G 0
> 100% /media/backup
> ...
>
> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
f_bavail=0 seems wrong to me.
What distro and what version of coreutils?
It's the same questions for Tomasz in yesterday's thread with similar subject.
--
Chris Murphy
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-06 22:35 ` Chris Murphy
@ 2019-12-06 22:51 ` Martin Raiber
2019-12-08 18:12 ` Zygo Blaxell
1 sibling, 0 replies; 13+ messages in thread
From: Martin Raiber @ 2019-12-06 22:51 UTC (permalink / raw)
To: Chris Murphy, Tomasz Chmielewski; +Cc: linux-btrfs, Qu Wenruo
On 06.12.2019 23:35 Chris Murphy wrote:
> On Fri, Dec 6, 2019 at 2:26 PM Martin Raiber <martin@urbackup.org> wrote:
>> Hi,
>>
>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>> can still write to the btrfs volume, but my software looks at the
>> available space and starts deleting stuff if statfs() says there is a
>> low amount of available space.
> This is the second bug like this reported in as many days against 5.4.1.
>
> Does this happen with an older kernel? Any 5.3 kernel or 5.2.15+ or
> any 5.1 kernel? Or heck, even 5.4? :P
Sorry, didn't see the other thread. Looks like the same issue.
Unfortunately, I was previously using 4.19.x, so I can't pinpoint it. I
think it did not occur when I was testing 5.4-rc7, but it does
(randomly?) take a few days of runtime to start occuring, so it could
have just not occurred then.
>> # df -h
>> Filesystem Size Used Avail
>> Use% Mounted on
>> ...
>> /dev/loop0 7.4T 623G 0
>> 100% /media/backup
>> ...
>>
>> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
>> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
>> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
>> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
>
> f_bavail=0 seems wrong to me.
>
> What distro and what version of coreutils?
It's debian stretch coreutils 8.26-3, glibc 2.29-3. But that's an
excerpt of the strace output so that shouldn't matter.
Thanks!
>
> It's the same questions for Tomasz in yesterday's thread with similar subject.
>
> --
> Chris Murphy
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-06 21:26 df shows no available space in 5.4.1 Martin Raiber
2019-12-06 22:35 ` Chris Murphy
@ 2019-12-07 7:28 ` Qu Wenruo
2019-12-09 18:56 ` Martin Raiber
2019-12-13 16:02 ` David Sterba
2019-12-13 20:03 ` Chris Murphy
3 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2019-12-07 7:28 UTC (permalink / raw)
To: Martin Raiber, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 4332 bytes --]
On 2019/12/7 上午5:26, Martin Raiber wrote:
> Hi,
>
> with kernel 5.4.1 I have the problem that df shows 100% space used. I
> can still write to the btrfs volume, but my software looks at the
> available space and starts deleting stuff if statfs() says there is a
> low amount of available space.
If the bug still happens, mind to try the snippet to see why this happened?
You will need to:
- Apply the patch to your kernel code
- Recompile the kernel or btrfs module
So this needs some experience in kernel compile.
- Reboot to newly compiled kernel or load the debug btrfs module
Thanks,
Qu
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 23aa630f04c9..cf34c05b16d7 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
{
struct btrfs_root *reloc_root;
- if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
+ if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
+ test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
return 0;
reloc_root = root->reloc_root;
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index f452a94abdc3..c2b70d97a63b 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
struct kstatfs *buf)
found->disk_used;
}
+ pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
+ __func__, found->flags, found->disk_used, factor);
total_used += found->disk_used;
}
@@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
struct kstatfs *buf)
buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
factor);
buf->f_blocks >>= bits;
+ pr_info("%s: super_total_bytes=%llu total_used=%llu
factor=%d\n", __func__,
+ btrfs_super_total_bytes(disk_super), total_used, factor);
buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
bits);
/* Account global block reserve as used, it's in logical size
already */
>
> # df -h
> Filesystem Size Used Avail
> Use% Mounted on
> ...
> /dev/loop0 7.4T 623G 0
> 100% /media/backup
> ...
>
> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
>
> # btrfs fi usage /media/backup
> Overall:
> Device size: 7.40TiB
> Device allocated: 671.02GiB
> Device unallocated: 6.74TiB
> Device missing: 0.00B
> Used: 622.49GiB
> Free (estimated): 6.79TiB (min: 6.79TiB)
> Data ratio: 1.00
> Metadata ratio: 1.00
> Global reserve: 512.00MiB (used: 0.00B)
>
> Data,single: Size:666.01GiB, Used:617.95GiB
> /dev/loop0 666.01GiB
>
> Metadata,single: Size:5.01GiB, Used:4.54GiB
> /dev/loop0 5.01GiB
>
> System,single: Size:4.00MiB, Used:96.00KiB
> /dev/loop0 4.00MiB
>
> Unallocated:
> /dev/loop0 6.74TiB
>
> # btrfs fi df /media/backup
> Data, single: total=666.01GiB, used=617.95GiB
> System, single: total=4.00MiB, used=96.00KiB
> Metadata, single: total=5.01GiB, used=4.54GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> # mount
>
> ...
> /dev/loop0 on /media/backup type btrfs
> (rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/)
> ...
>
> (I remounted with enospc_debug and the available space did not change...)
>
> Regards,
> Martin Raiber
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-06 22:35 ` Chris Murphy
2019-12-06 22:51 ` Martin Raiber
@ 2019-12-08 18:12 ` Zygo Blaxell
1 sibling, 0 replies; 13+ messages in thread
From: Zygo Blaxell @ 2019-12-08 18:12 UTC (permalink / raw)
To: Chris Murphy; +Cc: Martin Raiber, Tomasz Chmielewski, linux-btrfs, Qu Wenruo
[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]
On Fri, Dec 06, 2019 at 03:35:59PM -0700, Chris Murphy wrote:
> On Fri, Dec 6, 2019 at 2:26 PM Martin Raiber <martin@urbackup.org> wrote:
> >
> > Hi,
> >
> > with kernel 5.4.1 I have the problem that df shows 100% space used. I
> > can still write to the btrfs volume, but my software looks at the
> > available space and starts deleting stuff if statfs() says there is a
> > low amount of available space.
>
> This is the second bug like this reported in as many days against 5.4.1.
>
> Does this happen with an older kernel? Any 5.3 kernel or 5.2.15+ or
> any 5.1 kernel? Or heck, even 5.4? :P
I've noticed very different (lower) df space estimations while testing
5.3 kernels, compared to 5.0 or 5.2, on a draid5/mraid1 array.
df estimated about 1926 GB free on 5.0 and 5.2, 964 GB free on 5.3.
I'm guessing about 2GB was used in the time between those measurements,
which accounts for the discrepancy 1926 != 964 * 2.
My array has 2 large disks and 1 small one. The small one is full,
so only 2 drives are available for allocation, and at the time they
both had 954 GB of unallocated space + 10 GB in allocated/unused space.
The correct amount of free space is closer to 964 GB (5.3 value) than
1926 GB (5.0/5.2 values).
>
>
> >
> > # df -h
> > Filesystem Size Used Avail
> > Use% Mounted on
> > ...
> > /dev/loop0 7.4T 623G 0
> > 100% /media/backup
> > ...
> >
> > statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
> > f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
> > f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
> > f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
>
>
> f_bavail=0 seems wrong to me.
>
> What distro and what version of coreutils?
>
> It's the same questions for Tomasz in yesterday's thread with similar subject.
>
> --
> Chris Murphy
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-07 7:28 ` Qu Wenruo
@ 2019-12-09 18:56 ` Martin Raiber
2019-12-09 19:26 ` Martin Raiber
2019-12-10 0:52 ` Qu Wenruo
0 siblings, 2 replies; 13+ messages in thread
From: Martin Raiber @ 2019-12-09 18:56 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
On 07.12.2019 08:28 Qu Wenruo wrote:
>
> On 2019/12/7 上午5:26, Martin Raiber wrote:
>> Hi,
>>
>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>> can still write to the btrfs volume, but my software looks at the
>> available space and starts deleting stuff if statfs() says there is a
>> low amount of available space.
> If the bug still happens, mind to try the snippet to see why this happened?
>
> You will need to:
> - Apply the patch to your kernel code
> - Recompile the kernel or btrfs module
> So this needs some experience in kernel compile.
> - Reboot to newly compiled kernel or load the debug btrfs module
>
> Thanks,
> Qu
>
> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
> index 23aa630f04c9..cf34c05b16d7 100644
> --- a/fs/btrfs/relocation.c
> +++ b/fs/btrfs/relocation.c
> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
> {
> struct btrfs_root *reloc_root;
>
> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
> return 0;
>
> reloc_root = root->reloc_root;
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index f452a94abdc3..c2b70d97a63b 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
> struct kstatfs *buf)
> found->disk_used;
> }
>
> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
> + __func__, found->flags, found->disk_used, factor);
> total_used += found->disk_used;
> }
>
> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
> struct kstatfs *buf)
>
> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
> factor);
> buf->f_blocks >>= bits;
> + pr_info("%s: super_total_bytes=%llu total_used=%llu
> factor=%d\n", __func__,
> + btrfs_super_total_bytes(disk_super), total_used, factor);
> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
> bits);
>
> /* Account global block reserve as used, it's in logical size
> already */
>
Applied. It's currently 100% used directly after reboot, and I am
getting this log output:
[...]
[ 241.245150] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 241.904824] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 241.976082] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 241.976082] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 241.976082] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 241.976082] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 241.976083] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 241.976083] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 241.976083] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 241.976083] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 242.245301] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 242.245301] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 242.245301] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 242.245301] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 242.904977] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 242.904977] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 242.904977] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 242.904977] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 242.976105] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 242.976105] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 242.976105] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 242.976105] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 242.977777] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 242.977777] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 242.977777] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 242.977777] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 243.245041] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 243.245041] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 243.245041] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 243.245041] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 243.252079] btrfs_statfs: found type=0x1 disk_used=30026579968 factor=1
[ 243.252079] btrfs_statfs: found type=0x4 disk_used=957202432 factor=1
[ 243.252079] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 243.252079] btrfs_statfs: super_total_bytes=49999998976
total_used=30983798784 factor=1
[ 243.252080] btrfs_statfs: found type=0x1 disk_used=30026579968 factor=1
[ 243.252080] btrfs_statfs: found type=0x4 disk_used=957202432 factor=1
[ 243.252080] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 243.252080] btrfs_statfs: super_total_bytes=49999998976
total_used=30983798784 factor=1
[ 243.904085] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 243.904086] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 243.904086] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 243.904086] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 243.976096] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 243.976096] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 243.976096] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 243.976096] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 243.976257] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 243.976257] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 243.976257] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 243.976258] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 244.081959] btrfs_statfs: found type=0x1 disk_used=665652932608 factor=1
[ 244.081959] btrfs_statfs: found type=0x4 disk_used=4902944768 factor=1
[ 244.081959] btrfs_statfs: found type=0x2 disk_used=98304 factor=1
[ 244.081959] btrfs_statfs: super_total_bytes=8133881348096
total_used=670555975680 factor=1
[ 244.084110] btrfs_statfs: found type=0x1 disk_used=665652932608 factor=1
[ 244.084110] btrfs_statfs: found type=0x4 disk_used=4902944768 factor=1
[ 244.084110] btrfs_statfs: found type=0x2 disk_used=98304 factor=1
[ 244.084110] btrfs_statfs: super_total_bytes=8133881348096
total_used=670555975680 factor=1
[ 244.245470] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 244.245470] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 244.245470] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 244.245470] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 244.904170] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 244.904170] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 244.904170] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 244.904170] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 244.977624] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 244.977624] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 244.977624] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 244.977624] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 244.977387] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 244.977387] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 244.977387] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 244.977387] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
[ 245.244089] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
[ 245.244089] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
[ 245.244089] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[ 245.244089] btrfs_statfs: super_total_bytes=128835387392
total_used=93778841600 factor=1
>
>> # df -h
>> Filesystem Size Used Avail
>> Use% Mounted on
>> ...
>> /dev/loop0 7.4T 623G 0
>> 100% /media/backup
>> ...
>>
>> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
>> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
>> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
>> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
>>
>> # btrfs fi usage /media/backup
>> Overall:
>> Device size: 7.40TiB
>> Device allocated: 671.02GiB
>> Device unallocated: 6.74TiB
>> Device missing: 0.00B
>> Used: 622.49GiB
>> Free (estimated): 6.79TiB (min: 6.79TiB)
>> Data ratio: 1.00
>> Metadata ratio: 1.00
>> Global reserve: 512.00MiB (used: 0.00B)
>>
>> Data,single: Size:666.01GiB, Used:617.95GiB
>> /dev/loop0 666.01GiB
>>
>> Metadata,single: Size:5.01GiB, Used:4.54GiB
>> /dev/loop0 5.01GiB
>>
>> System,single: Size:4.00MiB, Used:96.00KiB
>> /dev/loop0 4.00MiB
>>
>> Unallocated:
>> /dev/loop0 6.74TiB
>>
>> # btrfs fi df /media/backup
>> Data, single: total=666.01GiB, used=617.95GiB
>> System, single: total=4.00MiB, used=96.00KiB
>> Metadata, single: total=5.01GiB, used=4.54GiB
>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>
>> # mount
>>
>> ...
>> /dev/loop0 on /media/backup type btrfs
>> (rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/)
>> ...
>>
>> (I remounted with enospc_debug and the available space did not change...)
>>
>> Regards,
>> Martin Raiber
>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-09 18:56 ` Martin Raiber
@ 2019-12-09 19:26 ` Martin Raiber
2019-12-10 0:52 ` Qu Wenruo
1 sibling, 0 replies; 13+ messages in thread
From: Martin Raiber @ 2019-12-09 19:26 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
On 09.12.2019 19:56 Martin Raiber wrote:
> On 07.12.2019 08:28 Qu Wenruo wrote:
>> On 2019/12/7 上午5:26, Martin Raiber wrote:
>>> Hi,
>>>
>>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>>> can still write to the btrfs volume, but my software looks at the
>>> available space and starts deleting stuff if statfs() says there is a
>>> low amount of available space.
>> If the bug still happens, mind to try the snippet to see why this happened?
>>
>> You will need to:
>> - Apply the patch to your kernel code
>> - Recompile the kernel or btrfs module
>> So this needs some experience in kernel compile.
>> - Reboot to newly compiled kernel or load the debug btrfs module
>>
>> Thanks,
>> Qu
>>
>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>> index 23aa630f04c9..cf34c05b16d7 100644
>> --- a/fs/btrfs/relocation.c
>> +++ b/fs/btrfs/relocation.c
>> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
>> {
>> struct btrfs_root *reloc_root;
>>
>> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
>> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
>> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
>> return 0;
>>
>> reloc_root = root->reloc_root;
>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>> index f452a94abdc3..c2b70d97a63b 100644
>> --- a/fs/btrfs/super.c
>> +++ b/fs/btrfs/super.c
>> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
>> struct kstatfs *buf)
>> found->disk_used;
>> }
>>
>> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
>> + __func__, found->flags, found->disk_used, factor);
>> total_used += found->disk_used;
>> }
>>
>> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
>> struct kstatfs *buf)
>>
>> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
>> factor);
>> buf->f_blocks >>= bits;
>> + pr_info("%s: super_total_bytes=%llu total_used=%llu
>> factor=%d\n", __func__,
>> + btrfs_super_total_bytes(disk_super), total_used, factor);
>> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
>> bits);
>>
>> /* Account global block reserve as used, it's in logical size
>> already */
>>
> Applied. It's currently 100% used directly after reboot, and I am
> getting this log output:
>
> [...]
> [ 241.245150] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 241.904824] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 241.976082] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 241.976082] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 241.976082] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 241.976082] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 241.976083] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 241.976083] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 241.976083] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 241.976083] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 242.245301] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 242.245301] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 242.245301] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 242.245301] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 242.904977] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 242.904977] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 242.904977] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 242.904977] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 242.976105] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 242.976105] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 242.976105] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 242.976105] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 242.977777] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 242.977777] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 242.977777] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 242.977777] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 243.245041] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 243.245041] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 243.245041] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 243.245041] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 243.252079] btrfs_statfs: found type=0x1 disk_used=30026579968 factor=1
> [ 243.252079] btrfs_statfs: found type=0x4 disk_used=957202432 factor=1
> [ 243.252079] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 243.252079] btrfs_statfs: super_total_bytes=49999998976
> total_used=30983798784 factor=1
> [ 243.252080] btrfs_statfs: found type=0x1 disk_used=30026579968 factor=1
> [ 243.252080] btrfs_statfs: found type=0x4 disk_used=957202432 factor=1
> [ 243.252080] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 243.252080] btrfs_statfs: super_total_bytes=49999998976
> total_used=30983798784 factor=1
> [ 243.904085] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 243.904086] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 243.904086] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 243.904086] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 243.976096] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 243.976096] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 243.976096] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 243.976096] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 243.976257] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 243.976257] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 243.976257] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 243.976258] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 244.081959] btrfs_statfs: found type=0x1 disk_used=665652932608 factor=1
> [ 244.081959] btrfs_statfs: found type=0x4 disk_used=4902944768 factor=1
> [ 244.081959] btrfs_statfs: found type=0x2 disk_used=98304 factor=1
> [ 244.081959] btrfs_statfs: super_total_bytes=8133881348096
> total_used=670555975680 factor=1
> [ 244.084110] btrfs_statfs: found type=0x1 disk_used=665652932608 factor=1
> [ 244.084110] btrfs_statfs: found type=0x4 disk_used=4902944768 factor=1
> [ 244.084110] btrfs_statfs: found type=0x2 disk_used=98304 factor=1
> [ 244.084110] btrfs_statfs: super_total_bytes=8133881348096
> total_used=670555975680 factor=1
> [ 244.245470] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 244.245470] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 244.245470] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 244.245470] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 244.904170] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 244.904170] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 244.904170] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 244.904170] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 244.977624] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 244.977624] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 244.977624] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 244.977624] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 244.977387] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 244.977387] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 244.977387] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 244.977387] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 245.244089] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 245.244089] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 245.244089] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 245.244089] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
It went back to 9% afterwards (without reboot, sync() perhaps?). I hope
it didn't go back while the above was logged. I'll keep an eye on it.
>>> # df -h
>>> Filesystem Size Used Avail
>>> Use% Mounted on
>>> ...
>>> /dev/loop0 7.4T 623G 0
>>> 100% /media/backup
>>> ...
>>>
>>> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
>>> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
>>> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
>>> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
>>>
>>> # btrfs fi usage /media/backup
>>> Overall:
>>> Device size: 7.40TiB
>>> Device allocated: 671.02GiB
>>> Device unallocated: 6.74TiB
>>> Device missing: 0.00B
>>> Used: 622.49GiB
>>> Free (estimated): 6.79TiB (min: 6.79TiB)
>>> Data ratio: 1.00
>>> Metadata ratio: 1.00
>>> Global reserve: 512.00MiB (used: 0.00B)
>>>
>>> Data,single: Size:666.01GiB, Used:617.95GiB
>>> /dev/loop0 666.01GiB
>>>
>>> Metadata,single: Size:5.01GiB, Used:4.54GiB
>>> /dev/loop0 5.01GiB
>>>
>>> System,single: Size:4.00MiB, Used:96.00KiB
>>> /dev/loop0 4.00MiB
>>>
>>> Unallocated:
>>> /dev/loop0 6.74TiB
>>>
>>> # btrfs fi df /media/backup
>>> Data, single: total=666.01GiB, used=617.95GiB
>>> System, single: total=4.00MiB, used=96.00KiB
>>> Metadata, single: total=5.01GiB, used=4.54GiB
>>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>>
>>> # mount
>>>
>>> ...
>>> /dev/loop0 on /media/backup type btrfs
>>> (rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/)
>>> ...
>>>
>>> (I remounted with enospc_debug and the available space did not change...)
>>>
>>> Regards,
>>> Martin Raiber
>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-09 18:56 ` Martin Raiber
2019-12-09 19:26 ` Martin Raiber
@ 2019-12-10 0:52 ` Qu Wenruo
2019-12-10 1:19 ` Qu Wenruo
1 sibling, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2019-12-10 0:52 UTC (permalink / raw)
To: Martin Raiber, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 3978 bytes --]
On 2019/12/10 上午2:56, Martin Raiber wrote:
> On 07.12.2019 08:28 Qu Wenruo wrote:
>>
>> On 2019/12/7 上午5:26, Martin Raiber wrote:
>>> Hi,
>>>
>>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>>> can still write to the btrfs volume, but my software looks at the
>>> available space and starts deleting stuff if statfs() says there is a
>>> low amount of available space.
>> If the bug still happens, mind to try the snippet to see why this happened?
>>
>> You will need to:
>> - Apply the patch to your kernel code
>> - Recompile the kernel or btrfs module
>> So this needs some experience in kernel compile.
>> - Reboot to newly compiled kernel or load the debug btrfs module
>>
>> Thanks,
>> Qu
>>
>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>> index 23aa630f04c9..cf34c05b16d7 100644
>> --- a/fs/btrfs/relocation.c
>> +++ b/fs/btrfs/relocation.c
>> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
>> {
>> struct btrfs_root *reloc_root;
>>
>> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
>> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
>> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
>> return 0;
>>
>> reloc_root = root->reloc_root;
>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>> index f452a94abdc3..c2b70d97a63b 100644
>> --- a/fs/btrfs/super.c
>> +++ b/fs/btrfs/super.c
>> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
>> struct kstatfs *buf)
>> found->disk_used;
>> }
>>
>> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
>> + __func__, found->flags, found->disk_used, factor);
>> total_used += found->disk_used;
>> }
>>
>> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
>> struct kstatfs *buf)
>>
>> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
>> factor);
>> buf->f_blocks >>= bits;
>> + pr_info("%s: super_total_bytes=%llu total_used=%llu
>> factor=%d\n", __func__,
>> + btrfs_super_total_bytes(disk_super), total_used, factor);
>> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
>> bits);
>>
>> /* Account global block reserve as used, it's in logical size
>> already */
>>
> Applied. It's currently 100% used directly after reboot, and I am
> getting this log output:
Thank you a lot for the debug output!
>
> [...]
> [ 241.245150] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
> [ 241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
> [ 241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
> [ 241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [ 241.904824] btrfs_statfs: super_total_bytes=128835387392
> total_used=93778841600 factor=1
This proves the on-disk numbers are all correct, so far so good.
The remaining problem is the block_rsv part. Which matches with the new
ticket system introduced in v5.4.
Mind to test the new debug snippet?
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index f452a94abdc3..516969534095 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -2076,6 +2076,8 @@ static int btrfs_statfs(struct dentry *dentry,
struct kstatfs *buf)
/* Account global block reserve as used, it's in logical size
already */
spin_lock(&block_rsv->lock);
/* Mixed block groups accounting is not byte-accurate, avoid
overflow */
+ pr_info("%s: block_rsv->size=%llu block_rsv->reserved=%llu\n",
__func__,
+ block_rsv->size, block_rsv->reserved);
if (buf->f_bfree >= block_rsv->size >> bits)
buf->f_bfree -= block_rsv->size >> bits;
else
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-10 0:52 ` Qu Wenruo
@ 2019-12-10 1:19 ` Qu Wenruo
2019-12-11 13:11 ` Martin Raiber
0 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2019-12-10 1:19 UTC (permalink / raw)
To: Martin Raiber, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 5053 bytes --]
On 2019/12/10 上午8:52, Qu Wenruo wrote:
>
>
> On 2019/12/10 上午2:56, Martin Raiber wrote:
>> On 07.12.2019 08:28 Qu Wenruo wrote:
>>>
>>> On 2019/12/7 上午5:26, Martin Raiber wrote:
>>>> Hi,
>>>>
>>>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>>>> can still write to the btrfs volume, but my software looks at the
>>>> available space and starts deleting stuff if statfs() says there is a
>>>> low amount of available space.
>>> If the bug still happens, mind to try the snippet to see why this happened?
>>>
>>> You will need to:
>>> - Apply the patch to your kernel code
>>> - Recompile the kernel or btrfs module
>>> So this needs some experience in kernel compile.
>>> - Reboot to newly compiled kernel or load the debug btrfs module
>>>
>>> Thanks,
>>> Qu
>>>
>>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>>> index 23aa630f04c9..cf34c05b16d7 100644
>>> --- a/fs/btrfs/relocation.c
>>> +++ b/fs/btrfs/relocation.c
>>> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
>>> {
>>> struct btrfs_root *reloc_root;
>>>
>>> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
>>> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
>>> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
>>> return 0;
>>>
>>> reloc_root = root->reloc_root;
>>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>>> index f452a94abdc3..c2b70d97a63b 100644
>>> --- a/fs/btrfs/super.c
>>> +++ b/fs/btrfs/super.c
>>> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>> struct kstatfs *buf)
>>> found->disk_used;
>>> }
>>>
>>> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
>>> + __func__, found->flags, found->disk_used, factor);
>>> total_used += found->disk_used;
>>> }
>>>
>>> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>> struct kstatfs *buf)
>>>
>>> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
>>> factor);
>>> buf->f_blocks >>= bits;
>>> + pr_info("%s: super_total_bytes=%llu total_used=%llu
>>> factor=%d\n", __func__,
>>> + btrfs_super_total_bytes(disk_super), total_used, factor);
>>> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
>>> bits);
>>>
>>> /* Account global block reserve as used, it's in logical size
>>> already */
>>>
>> Applied. It's currently 100% used directly after reboot, and I am
>> getting this log output:
>
> Thank you a lot for the debug output!
>
>>
>> [...]
>> [ 241.245150] btrfs_statfs: super_total_bytes=128835387392
>> total_used=93778841600 factor=1
>> [ 241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
>> [ 241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
>> [ 241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
>> [ 241.904824] btrfs_statfs: super_total_bytes=128835387392
>> total_used=93778841600 factor=1
>
> This proves the on-disk numbers are all correct, so far so good.
>
> The remaining problem is the block_rsv part. Which matches with the new
> ticket system introduced in v5.4.
>
> Mind to test the new debug snippet?
>
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index f452a94abdc3..516969534095 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -2076,6 +2076,8 @@ static int btrfs_statfs(struct dentry *dentry,
> struct kstatfs *buf)
> /* Account global block reserve as used, it's in logical size
> already */
> spin_lock(&block_rsv->lock);
> /* Mixed block groups accounting is not byte-accurate, avoid
> overflow */
> + pr_info("%s: block_rsv->size=%llu block_rsv->reserved=%llu\n",
> __func__,
> + block_rsv->size, block_rsv->reserved);
> if (buf->f_bfree >= block_rsv->size >> bits)
> buf->f_bfree -= block_rsv->size >> bits;
> else
>
And this extra snippet for available space.
Thanks,
Qu
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index f452a94abdc3..f1a3e01a0ef5 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1911,6 +1911,7 @@ static inline int
btrfs_calc_avail_data_space(struct btrfs_fs_info *fs_info,
* We aren't under the device list lock, so this is racy-ish,
but good
* enough for our purposes.
*/
+ pr_info("%s: original_free_bytes=%llu\n", __func__, *free_bytes);
nr_devices = fs_info->fs_devices->open_devices;
if (!nr_devices) {
smp_mb();
@@ -2005,6 +2006,7 @@ static inline int
btrfs_calc_avail_data_space(struct btrfs_fs_info *fs_info,
kfree(devices_info);
*free_bytes = avail_space;
+ pr_info("%s: calculated_bytes=%llu\n", __func__, avail_space);
return 0;
}
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-10 1:19 ` Qu Wenruo
@ 2019-12-11 13:11 ` Martin Raiber
2019-12-16 5:04 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Martin Raiber @ 2019-12-11 13:11 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
On 10.12.2019 02:19 Qu Wenruo wrote:
>
> On 2019/12/10 上午8:52, Qu Wenruo wrote:
>>
>> On 2019/12/10 上午2:56, Martin Raiber wrote:
>>> On 07.12.2019 08:28 Qu Wenruo wrote:
>>>> On 2019/12/7 上午5:26, Martin Raiber wrote:
>>>>> Hi,
>>>>>
>>>>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>>>>> can still write to the btrfs volume, but my software looks at the
>>>>> available space and starts deleting stuff if statfs() says there is a
>>>>> low amount of available space.
>>>> If the bug still happens, mind to try the snippet to see why this happened?
>>>>
>>>> You will need to:
>>>> - Apply the patch to your kernel code
>>>> - Recompile the kernel or btrfs module
>>>> So this needs some experience in kernel compile.
>>>> - Reboot to newly compiled kernel or load the debug btrfs module
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>>>> index 23aa630f04c9..cf34c05b16d7 100644
>>>> --- a/fs/btrfs/relocation.c
>>>> +++ b/fs/btrfs/relocation.c
>>>> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
>>>> {
>>>> struct btrfs_root *reloc_root;
>>>>
>>>> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
>>>> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
>>>> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
>>>> return 0;
>>>>
>>>> reloc_root = root->reloc_root;
>>>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>>>> index f452a94abdc3..c2b70d97a63b 100644
>>>> --- a/fs/btrfs/super.c
>>>> +++ b/fs/btrfs/super.c
>>>> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>>> struct kstatfs *buf)
>>>> found->disk_used;
>>>> }
>>>>
>>>> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
>>>> + __func__, found->flags, found->disk_used, factor);
>>>> total_used += found->disk_used;
>>>> }
>>>>
>>>> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>>> struct kstatfs *buf)
>>>>
>>>> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
>>>> factor);
>>>> buf->f_blocks >>= bits;
>>>> + pr_info("%s: super_total_bytes=%llu total_used=%llu
>>>> factor=%d\n", __func__,
>>>> + btrfs_super_total_bytes(disk_super), total_used, factor);
>>>> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
>>>> bits);
>>>>
>>>> /* Account global block reserve as used, it's in logical size
>>>> already */
>>>>
>>> Applied. It's currently 100% used directly after reboot, and I am
>>> getting this log output:
>> Thank you a lot for the debug output!
>>
>>> [...]
>>> [ 241.245150] btrfs_statfs: super_total_bytes=128835387392
>>> total_used=93778841600 factor=1
>>> [ 241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
>>> [ 241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
>>> [ 241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
>>> [ 241.904824] btrfs_statfs: super_total_bytes=128835387392
>>> total_used=93778841600 factor=1
>> This proves the on-disk numbers are all correct, so far so good.
>>
>> The remaining problem is the block_rsv part. Which matches with the new
>> ticket system introduced in v5.4.
>>
>> Mind to test the new debug snippet?
>>
>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>> index f452a94abdc3..516969534095 100644
>> --- a/fs/btrfs/super.c
>> +++ b/fs/btrfs/super.c
>> @@ -2076,6 +2076,8 @@ static int btrfs_statfs(struct dentry *dentry,
>> struct kstatfs *buf)
>> /* Account global block reserve as used, it's in logical size
>> already */
>> spin_lock(&block_rsv->lock);
>> /* Mixed block groups accounting is not byte-accurate, avoid
>> overflow */
>> + pr_info("%s: block_rsv->size=%llu block_rsv->reserved=%llu\n",
>> __func__,
>> + block_rsv->size, block_rsv->reserved);
>> if (buf->f_bfree >= block_rsv->size >> bits)
>> buf->f_bfree -= block_rsv->size >> bits;
>> else
>>
> And this extra snippet for available space.
>
> Thanks,
> Qu
>
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index f452a94abdc3..f1a3e01a0ef5 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -1911,6 +1911,7 @@ static inline int
> btrfs_calc_avail_data_space(struct btrfs_fs_info *fs_info,
> * We aren't under the device list lock, so this is racy-ish,
> but good
> * enough for our purposes.
> */
> + pr_info("%s: original_free_bytes=%llu\n", __func__, *free_bytes);
> nr_devices = fs_info->fs_devices->open_devices;
> if (!nr_devices) {
> smp_mb();
> @@ -2005,6 +2006,7 @@ static inline int
> btrfs_calc_avail_data_space(struct btrfs_fs_info *fs_info,
>
> kfree(devices_info);
> *free_bytes = avail_space;
> + pr_info("%s: calculated_bytes=%llu\n", __func__, avail_space);
> return 0;
> }
>
Now logs this at 100% used:
[90273.353449] btrfs_calc_avail_data_space: original_free_bytes=23583420416
[90273.353449] btrfs_calc_avail_data_space: calculated_bytes=13662945280
[90273.369508] btrfs_statfs: found type=0x1 disk_used=90233212928 factor=1
[90273.369536] btrfs_statfs: found type=0x1 disk_used=90233212928 factor=1
[90273.369536] btrfs_statfs: found type=0x4 disk_used=339361792 factor=1
[90273.369508] btrfs_statfs: found type=0x4 disk_used=339361792 factor=1
[90273.369508] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[90273.369536] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
[90273.369508] btrfs_statfs: super_total_bytes=128835387392
total_used=90572591104 factor=1
[90273.369508] btrfs_statfs: block_rsv->size=147554304
block_rsv->reserved=147554304
[90273.369537] btrfs_statfs: super_total_bytes=128835387392
total_used=90572591104 factor=1
[90273.369509] btrfs_calc_avail_data_space: original_free_bytes=23583420416
[90273.369537] btrfs_statfs: block_rsv->size=147554304
block_rsv->reserved=147554304
[90273.369537] btrfs_calc_avail_data_space: original_free_bytes=23583420416
[90273.369509] btrfs_calc_avail_data_space: calculated_bytes=13662945280
[90273.369537] btrfs_calc_avail_data_space: calculated_bytes=13662945280
[90273.400227] btrfs_statfs: found type=0x1 disk_used=726834307072 factor=1
[90273.400227] btrfs_statfs: found type=0x4 disk_used=4908548096 factor=1
[90273.400227] btrfs_statfs: found type=0x2 disk_used=98304 factor=1
[90273.400227] btrfs_statfs: super_total_bytes=8133881348096
total_used=731742953472 factor=1
[90273.400227] btrfs_statfs: block_rsv->size=536870912
block_rsv->reserved=536821760
[90273.400227] btrfs_calc_avail_data_space: original_free_bytes=1171038208
[90273.400227] btrfs_calc_avail_data_space: calculated_bytes=7400493613056
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-06 21:26 df shows no available space in 5.4.1 Martin Raiber
2019-12-06 22:35 ` Chris Murphy
2019-12-07 7:28 ` Qu Wenruo
@ 2019-12-13 16:02 ` David Sterba
2019-12-13 20:03 ` Chris Murphy
3 siblings, 0 replies; 13+ messages in thread
From: David Sterba @ 2019-12-13 16:02 UTC (permalink / raw)
To: Martin Raiber; +Cc: linux-btrfs
On Fri, Dec 06, 2019 at 09:26:05PM +0000, Martin Raiber wrote:
> # df -h
> Filesystem Size Used Avail
> Use% Mounted on
> ...
> /dev/loop0 7.4T 623G 0
> 100% /media/backup
> ...
>
> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096,
> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0,
> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255,
> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0
>
> # btrfs fi usage /media/backup
> Overall:
> Device size: 7.40TiB
> Device allocated: 671.02GiB
> Device unallocated: 6.74TiB
> Device missing: 0.00B
> Used: 622.49GiB
> Free (estimated): 6.79TiB (min: 6.79TiB)
> Data ratio: 1.00
> Metadata ratio: 1.00
> Global reserve: 512.00MiB (used: 0.00B)
>
> Data,single: Size:666.01GiB, Used:617.95GiB
> /dev/loop0 666.01GiB
>
> Metadata,single: Size:5.01GiB, Used:4.54GiB
> /dev/loop0 5.01GiB
Here's the cause for 0 for available data: there's a special case in
statfs that checks for remaining metadata space and if thre's less than
some threshold then the value becomes 0.
The global block reserve needs to be accounted there to so
4.54G + 512M ~ 5.01G
> System,single: Size:4.00MiB, Used:96.00KiB
> /dev/loop0 4.00MiB
>
> Unallocated:
> /dev/loop0 6.74TiB
Enough unallocated data for more metadata chunks, that are usually
allocated in advance so the above should not happen.
> # btrfs fi df /media/backup
> Data, single: total=666.01GiB, used=617.95GiB
> System, single: total=4.00MiB, used=96.00KiB
> Metadata, single: total=5.01GiB, used=4.54GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> # mount
>
> ...
> /dev/loop0 on /media/backup type btrfs
> (rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/)
commit=86400 is kind extreme but this should affect only unwritten data
possibly buffered in memory before write and sync/commit can be started
for other reasons too.
So it looks to me that metadata chunks don't get allocated as before. A
workaround could be the mount option metadata_ratio, but finding the
root cause is desired as this looks like a regression.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-06 21:26 df shows no available space in 5.4.1 Martin Raiber
` (2 preceding siblings ...)
2019-12-13 16:02 ` David Sterba
@ 2019-12-13 20:03 ` Chris Murphy
3 siblings, 0 replies; 13+ messages in thread
From: Chris Murphy @ 2019-12-13 20:03 UTC (permalink / raw)
To: Martin Raiber; +Cc: linux-btrfs
On Fri, Dec 6, 2019 at 2:26 PM Martin Raiber <martin@urbackup.org> wrote:
>
> /dev/loop0 on /media/backup type btrfs
> (rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/)
I missed the 24 hour commit time. I don't know that it's at all
related to the problem, but I don't think you're gaining anything that
can't be achieved with a 1 minute or even 5 minute interval. Has
testing demonstrated the 30 second default is causing some kind of
problem with this workload?
--
Chris Murphy
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: df shows no available space in 5.4.1
2019-12-11 13:11 ` Martin Raiber
@ 2019-12-16 5:04 ` Qu Wenruo
0 siblings, 0 replies; 13+ messages in thread
From: Qu Wenruo @ 2019-12-16 5:04 UTC (permalink / raw)
To: Martin Raiber, linux-btrfs
[-- Attachment #1.1.1: Type: text/plain, Size: 8832 bytes --]
On 2019/12/11 下午9:11, Martin Raiber wrote:
> On 10.12.2019 02:19 Qu Wenruo wrote:
>>
>> On 2019/12/10 上午8:52, Qu Wenruo wrote:
>>>
>>> On 2019/12/10 上午2:56, Martin Raiber wrote:
>>>> On 07.12.2019 08:28 Qu Wenruo wrote:
>>>>> On 2019/12/7 上午5:26, Martin Raiber wrote:
>>>>>> Hi,
>>>>>>
>>>>>> with kernel 5.4.1 I have the problem that df shows 100% space used. I
>>>>>> can still write to the btrfs volume, but my software looks at the
>>>>>> available space and starts deleting stuff if statfs() says there is a
>>>>>> low amount of available space.
>>>>> If the bug still happens, mind to try the snippet to see why this happened?
>>>>>
>>>>> You will need to:
>>>>> - Apply the patch to your kernel code
>>>>> - Recompile the kernel or btrfs module
>>>>> So this needs some experience in kernel compile.
>>>>> - Reboot to newly compiled kernel or load the debug btrfs module
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>>
>>>>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>>>>> index 23aa630f04c9..cf34c05b16d7 100644
>>>>> --- a/fs/btrfs/relocation.c
>>>>> +++ b/fs/btrfs/relocation.c
>>>>> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root)
>>>>> {
>>>>> struct btrfs_root *reloc_root;
>>>>>
>>>>> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state))
>>>>> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
>>>>> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state))
>>>>> return 0;
>>>>>
>>>>> reloc_root = root->reloc_root;
>>>>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>>>>> index f452a94abdc3..c2b70d97a63b 100644
>>>>> --- a/fs/btrfs/super.c
>>>>> +++ b/fs/btrfs/super.c
>>>>> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>>>> struct kstatfs *buf)
>>>>> found->disk_used;
>>>>> }
>>>>>
>>>>> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n",
>>>>> + __func__, found->flags, found->disk_used, factor);
>>>>> total_used += found->disk_used;
>>>>> }
>>>>>
>>>>> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>>>> struct kstatfs *buf)
>>>>>
>>>>> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super),
>>>>> factor);
>>>>> buf->f_blocks >>= bits;
>>>>> + pr_info("%s: super_total_bytes=%llu total_used=%llu
>>>>> factor=%d\n", __func__,
>>>>> + btrfs_super_total_bytes(disk_super), total_used, factor);
>>>>> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >>
>>>>> bits);
>>>>>
>>>>> /* Account global block reserve as used, it's in logical size
>>>>> already */
>>>>>
>>>> Applied. It's currently 100% used directly after reboot, and I am
>>>> getting this log output:
>>> Thank you a lot for the debug output!
>>>
>>>> [...]
>>>> [ 241.245150] btrfs_statfs: super_total_bytes=128835387392
>>>> total_used=93778841600 factor=1
>>>> [ 241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1
>>>> [ 241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1
>>>> [ 241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
>>>> [ 241.904824] btrfs_statfs: super_total_bytes=128835387392
>>>> total_used=93778841600 factor=1
>>> This proves the on-disk numbers are all correct, so far so good.
>>>
>>> The remaining problem is the block_rsv part. Which matches with the new
>>> ticket system introduced in v5.4.
>>>
>>> Mind to test the new debug snippet?
>>>
>>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>>> index f452a94abdc3..516969534095 100644
>>> --- a/fs/btrfs/super.c
>>> +++ b/fs/btrfs/super.c
>>> @@ -2076,6 +2076,8 @@ static int btrfs_statfs(struct dentry *dentry,
>>> struct kstatfs *buf)
>>> /* Account global block reserve as used, it's in logical size
>>> already */
>>> spin_lock(&block_rsv->lock);
>>> /* Mixed block groups accounting is not byte-accurate, avoid
>>> overflow */
>>> + pr_info("%s: block_rsv->size=%llu block_rsv->reserved=%llu\n",
>>> __func__,
>>> + block_rsv->size, block_rsv->reserved);
>>> if (buf->f_bfree >= block_rsv->size >> bits)
>>> buf->f_bfree -= block_rsv->size >> bits;
>>> else
>>>
>> And this extra snippet for available space.
>>
>> Thanks,
>> Qu
>>
>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>> index f452a94abdc3..f1a3e01a0ef5 100644
>> --- a/fs/btrfs/super.c
>> +++ b/fs/btrfs/super.c
>> @@ -1911,6 +1911,7 @@ static inline int
>> btrfs_calc_avail_data_space(struct btrfs_fs_info *fs_info,
>> * We aren't under the device list lock, so this is racy-ish,
>> but good
>> * enough for our purposes.
>> */
>> + pr_info("%s: original_free_bytes=%llu\n", __func__, *free_bytes);
>> nr_devices = fs_info->fs_devices->open_devices;
>> if (!nr_devices) {
>> smp_mb();
>> @@ -2005,6 +2006,7 @@ static inline int
>> btrfs_calc_avail_data_space(struct btrfs_fs_info *fs_info,
>>
>> kfree(devices_info);
>> *free_bytes = avail_space;
>> + pr_info("%s: calculated_bytes=%llu\n", __func__, avail_space);
>> return 0;
>> }
>>
Sorry for the date reply, was busy firefighting some bugs.
> Now logs this at 100% used:
>
> [90273.353449] btrfs_calc_avail_data_space: original_free_bytes=23583420416
> [90273.353449] btrfs_calc_avail_data_space: calculated_bytes=13662945280
This marks the beginning of one statefs call.
> [90273.369508] btrfs_statfs: found type=0x1 disk_used=90233212928 factor=1
> [90273.369536] btrfs_statfs: found type=0x1 disk_used=90233212928 factor=1
> [90273.369536] btrfs_statfs: found type=0x4 disk_used=339361792 factor=1
> [90273.369508] btrfs_statfs: found type=0x4 disk_used=339361792 factor=1
> [90273.369508] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [90273.369536] btrfs_statfs: found type=0x2 disk_used=16384 factor=1
> [90273.369508] btrfs_statfs: super_total_bytes=128835387392
> total_used=90572591104 factor=1
So far so good. All SINGLE chunks, total disk bytes are ~120GiB.
While totally used bytes are ~84GiB.
In theory, we should give ~36GiB.
> [90273.369508] btrfs_statfs: block_rsv->size=147554304
> block_rsv->reserved=147554304
block_rsv is tiny, just ~140 MiB, shouldn't cause much difference.
> [90273.369537] btrfs_statfs: super_total_bytes=128835387392
> total_used=90572591104 factor=1
So at this stage, f_bfree should be 74732024 - 288192 blocks.
^^^^^^^^ ^^^- block_rsv / 512
|- (total_bytes - total_used ) / 512
At least, f_bfree looks OK.
> [90273.369509] btrfs_calc_avail_data_space: original_free_bytes=23583420416
> [90273.369537] btrfs_statfs: block_rsv->size=147554304
> block_rsv->reserved=147554304
> [90273.369537] btrfs_calc_avail_data_space: original_free_bytes=23583420416
Still good, we have around ~21.9GiB unused data space across all
allocated data chunks.
All this ~21.9GiB should contribute to f_bavail.
Although it means you have some fragments, it's not a big deal at all.
> [90273.369509] btrfs_calc_avail_data_space: calculated_bytes=13662945280
> [90273.369537] btrfs_calc_avail_data_space: calculated_bytes=13662945280
And btrfs_calc_avail_data_space() find that we can allocate around
12.7GiB new data chunks.
This 12.7GiB also going to be part of f_bavail.
This means, you should have ~34GiB free space, before we do the
offending check:
if (!mixed && total_free_meta - thresh < block_rsv->size)
buf->f_bavail = 0;
This check is pretty old, from 2015, while recently we allow aggressive
metadata over-committing, thus we can have a lot of metadata reserved
space without really allocating new metadata chunks.
I'll try to find out a better calculation to co-operate with metadata
over-committing.
Feel free to remove all debugg snippets, and if you want some dirty
fixes, please try the attached diff.
Thanks,
Qu
> [90273.400227] btrfs_statfs: found type=0x1 disk_used=726834307072 factor=1
> [90273.400227] btrfs_statfs: found type=0x4 disk_used=4908548096 factor=1
> [90273.400227] btrfs_statfs: found type=0x2 disk_used=98304 factor=1
> [90273.400227] btrfs_statfs: super_total_bytes=8133881348096
> total_used=731742953472 factor=1
> [90273.400227] btrfs_statfs: block_rsv->size=536870912
> block_rsv->reserved=536821760
> [90273.400227] btrfs_calc_avail_data_space: original_free_bytes=1171038208
> [90273.400227] btrfs_calc_avail_data_space: calculated_bytes=7400493613056
>
[-- Attachment #1.1.2: diff --]
[-- Type: text/plain, Size: 1516 bytes --]
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 1b151af25772..b8b67ab05f72 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -2032,7 +2032,6 @@ static int btrfs_statfs(struct dentry *dentry, struct kstatfs *buf)
unsigned factor = 1;
struct btrfs_block_rsv *block_rsv = &fs_info->global_block_rsv;
int ret;
- u64 thresh = 0;
int mixed = 0;
rcu_read_lock();
@@ -2085,26 +2084,9 @@ static int btrfs_statfs(struct dentry *dentry, struct kstatfs *buf)
if (ret)
return ret;
buf->f_bavail += div_u64(total_free_data, factor);
+ buf->f_bavail -= block_rsv->size;
buf->f_bavail = buf->f_bavail >> bits;
- /*
- * We calculate the remaining metadata space minus global reserve. If
- * this is (supposedly) smaller than zero, there's no space. But this
- * does not hold in practice, the exhausted state happens where's still
- * some positive delta. So we apply some guesswork and compare the
- * delta to a 4M threshold. (Practically observed delta was ~2M.)
- *
- * We probably cannot calculate the exact threshold value because this
- * depends on the internal reservations requested by various
- * operations, so some operations that consume a few metadata will
- * succeed even if the Avail is zero. But this is better than the other
- * way around.
- */
- thresh = SZ_4M;
-
- if (!mixed && total_free_meta - thresh < block_rsv->size)
- buf->f_bavail = 0;
-
buf->f_type = BTRFS_SUPER_MAGIC;
buf->f_bsize = dentry->d_sb->s_blocksize;
buf->f_namelen = BTRFS_NAME_LEN;
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2019-12-16 5:05 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-06 21:26 df shows no available space in 5.4.1 Martin Raiber
2019-12-06 22:35 ` Chris Murphy
2019-12-06 22:51 ` Martin Raiber
2019-12-08 18:12 ` Zygo Blaxell
2019-12-07 7:28 ` Qu Wenruo
2019-12-09 18:56 ` Martin Raiber
2019-12-09 19:26 ` Martin Raiber
2019-12-10 0:52 ` Qu Wenruo
2019-12-10 1:19 ` Qu Wenruo
2019-12-11 13:11 ` Martin Raiber
2019-12-16 5:04 ` Qu Wenruo
2019-12-13 16:02 ` David Sterba
2019-12-13 20:03 ` Chris Murphy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.