100% disk usage reported by "df"

100% disk usage reported by "df"

[Russell Mosemann]

df displays 100% disk usage when 119GB is used out of 9TB on a freeshly-installed system. The issue surfaces when disk usage is somewhere over the low 100GBs. Existing systems with lots of data that were upgraded to kernel 5.4 do not exhibit this problem. Other freshly-installed systems with less than 100GB do not show this problem. It is unknown if they will exhibit the problem, as disk usage increases. btrfs reports the correct disk space, and the system is writable. I wiped the drive, recreated the file system and the problem reappeared after a couple of days, when disk usage exceeded 100GB. Files are being copied with --reflink every day. Snapshots are not being made, and there are no subvolumes.

# uname -a
Linux vhost361 5.4.0-0.bpo.3-amd64 #1 SMP Debian 5.4.13-1~bpo10+1 (2020-02-07) x86_64 GNU/Linux

# cat /etc/fstab
UUID=913... /usr/local/data/datastore2 btrfs   noatime,compress-force=zstd   0  0

# df
Filesystem      1K-blocks      Used  Available Use% Mounted on
/dev/sdc1      9766434816 123744020          0 100% /usr/local/data/datastore2

# btrfs subvolume list /usr/local/data/datastore2
#

# btrfs fi show
Label: 'datastore2'  uuid: 91361c03-cb4f-4d8d-954d-5a1e983daabd
       Total devices 1 FS bytes used 116.77GiB
       devid    1 size 9.09TiB used 120.02GiB path /dev/sdc1

# btrfs fi df /usr/local/data/datastore2
Data, single: total=118.00GiB, used=115.96GiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=1.00GiB, used=827.66MiB
GlobalReserve, single: total=444.61MiB, used=0.00B

# btrfs fi usage /usr/local/data/datastore2
Overall:
   Device size:                   9.09TiB
   Device allocated:            120.02GiB
   Device unallocated:            8.98TiB
   Device missing:                  0.00B
   Used:                        117.58GiB
   Free (estimated):              8.98TiB      (min: 4.49TiB)
   Data ratio:                       1.00
   Metadata ratio:                   2.00
   Global reserve:              444.61MiB      (used: 0.00B)

Data,single: Size:118.00GiB, Used:115.96GiB
  /dev/sdc1     118.00GiB

Metadata,DUP: Size:1.00GiB, Used:827.66MiB
  /dev/sdc1       2.00GiB

System,DUP: Size:8.00MiB, Used:16.00KiB
  /dev/sdc1      16.00MiB

Unallocated:
  /dev/sdc1       8.98TiB

# btrfsck /dev/sdc1
Opening filesystem to check...
Checking filesystem on /dev/sdc1
UUID: 91361c03-cb4f-4d8d-954d-5a1e983daabd
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 125379792896 bytes used, no error found
total csum bytes: 121563204
total tree bytes: 867876864
total fs tree bytes: 401555456
total extent tree bytes: 317325312
btree space waste bytes: 104952067
file data blocks allocated: 265984520192
referenced 747687165952

--
Russell Mosemann

Re: 100% disk usage reported by "df"

[Chris Murphy]

On Sun, Mar 15, 2020 at 1:46 PM Russell Mosemann
<rmosemann@futurefoam.com> wrote:
>
> df displays 100% disk usage when 119GB is used out of 9TB on a freeshly-installed system. The issue surfaces when disk usage is somewhere over the low 100GBs. Existing systems with lots of data that were upgraded to kernel 5.4 do not exhibit this problem. Other freshly-installed systems with less than 100GB do not show this problem. It is unknown if they will exhibit the problem, as disk usage increases. btrfs reports the correct disk space, and the system is writable. I wiped the drive, recreated the file system and the problem reappeared after a couple of days, when disk usage exceeded 100GB. Files are being copied with --reflink every day. Snapshots are not being made, and there are no subvolumes.
>
> # uname -a
> Linux vhost361 5.4.0-0.bpo.3-amd64 #1 SMP Debian 5.4.13-1~bpo10+1 (2020-02-07) x86_64 GNU/Linux


Fixed since 5.5.2, and 5.4.18.



-- 
Chris Murphy