On Thu, 2 Mar 2023 20:34:27 -0800
Matt Corallo <blnxfsl@bluematt.me> wrote:

> I have a ~seven year old BTRFS filesystem who's performance has slowly degraded to unusability.
> 
> ...
>
> This has led to a lot of metadata:
> Metadata,RAID1C3: Size:1.48TiB, Used:1.46TiB (98.73%)
> 
> ...
>
> I recently started adding some I/O to the machine, writing 1MB/s or two of writes from openstack 
> swift, which has now racked up a million or three files itself (in a directory tree two layers of 
> ~1000-folder directories deep). This has made the filesystem largely unusable.
> 
> ...
> 
> Thanks,
> Matt

Hi,
I suspect lots of inline files are bloating your metadata. Especially
from openstack swift, given that each object is stored as it's own file:
https://docs.openstack.org/swift/latest/overview_architecture.html#object-server
By default, btrfs will store all files smaller than 2048 bytes inline
(i.e. directly in the metadata). You can change that with the
"max_inline" mount option

You can count the number of inline files with something like:

find /mnt/hdd -type f -print0 | xargs -0 filefrag -v | grep inline | wc -l

Regards,
Lukas Straub

--