Re: Again, no space left on device while rebalancing and recipe doesnt work

[Qu Wenruo <quwenruo@cn.fujitsu.com>]

Marc Haber wrote on 2016/02/27 22:14 +0100:
> Hi,
>
> I have again the issue of no space left on device while rebalancing
> (with btrfs-tools 4.4.1 on kernel 4.4.2 on Debian unstable):
>
> mh@fan:~$ sudo btrfs balance start /mnt/fanbtr
> ERROR: error during balancing '/mnt/fanbtr': No space left on device

It seems that, only when balancing all chunks, ENOSPC error happens.

And did you run any other heavy IO at background?

BTW, is there any kernel log when the ENOSPC happens?

> mh@fan:~$ sudo btrfs fi show /mnt/fanbtr
> mh@fan:~$ sudo btrfs fi show -m
> Label: 'fanbtr'  uuid: 4198d1bc-e3ce-40df-a7ee-44a2d120bff3
>          Total devices 1 FS bytes used 116.49GiB
>          devid    1 size 417.19GiB used 177.06GiB path /dev/mapper/fanbtr
> mh@fan:~$ sudo btrfs fi df /mnt/fanbtr
> Data, single: total=113.00GiB, used=112.77GiB
> System, DUP: total=32.00MiB, used=48.00KiB
> Metadata, DUP: total=32.00GiB, used=3.72GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
> mh@fan:~$
>
> The filesystem was recently resized from 300 GB to 420 GB.
>
> Why does btrfs fi show /mnt/fanbtr not give any output? Wy does btrfs
> fi df /mnt/fanbtr say that my data space is only 113 GiB large?
>
> btrfs balance start -dusage=5 works up to -dusage=100:
>
> mh@fan:~$ sudo btrfs balance start -dusage=100 /mnt/fanbtr
> Done, had to relocate 111 out of 179 chunks
> mh@fan:~$ sudo btrfs balance start -dusage=100 /mnt/fanbtr
> Done, had to relocate 111 out of 179 chunks
> mh@fan:~$ sudo btrfs balance start -dusage=100 /mnt/fanbtr
> Done, had to relocate 110 out of 179 chunks
> mh@fan:~$ sudo btrfs balance start -dusage=100 /mnt/fanbtr
> Done, had to relocate 109 out of 179 chunks
> mh@fan:~$ sudo btrfs balance start /mnt/fanbtr
> ERROR: error during balancing '/mnt/fanbtr': No space left on device
> mh@fan:~$

Would you please try the following commands to see which one caused the 
problem?
And would you please provide the dmesg of them?

# btrfs balance start -dprofiles=single /mnt/fanbtr
# btrfs balance start -mprofile=dup /mnt/fanbtr
# btrfs balance start -sprofile=dup /mnt/fanbtr

The above three commands should be equal to your "btrfs balance start" 
command without any parameter, but do them separately, so it should tell 
you which chunk type caused the ENOSPC error.

 From your above -dusage=100 parameter, I think data should be OK, and 
since the metadata chunk is larger than 1G, and there are a lot of free 
metadata space, I assume it's the single system chunk causing the problem.

And if my assumption is right, you have nothing to worry as data and 
metadata should be OK to allocate new chunks, and 32M system chunks is 
definitely large enough.
But in that case, we should have a clearer clue to trace and fix the bug.

Thanks,
Qu

>
> What is going on here? How do I get away from here?
>
> Greetings
> Marc
>
>