All of lore.kernel.org
 help / color / mirror / Atom feed
From: Filipe Manana <fdmanana@kernel.org>
To: Ivan Shapovalov <intelfx@intelfx.name>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: [6.7 regression] [BISECTED] 28270e25c69a causes overallocation of metadata space
Date: Wed, 17 Jan 2024 11:28:32 +0000	[thread overview]
Message-ID: <CAL3q7H5UaYcAYHijBO+QTnnpruVQXvdirg05_X94KRKrKnXDZw@mail.gmail.com> (raw)
In-Reply-To: <9cdbf0ca9cdda1b4c84e15e548af7d7f9f926382.camel@intelfx.name>

On Wed, Jan 17, 2024 at 6:04 AM Ivan Shapovalov <intelfx@intelfx.name> wrote:
>
> Hi all,
>
> Starting from v6.7 I've noticed severe overallocation of metadata space
> on both of my btrfs root filesystems (both NVMe SSDs, both use
> snapshots):
>
> ```
> # mount | grep -w /
> rw,noatime,compress=zstd:1,ssd,discard=async,space_cache=v2,subvolid=<...>
>
> # btrfs fi usage /
> <...>
>
> Data,single: Size:550.00GiB, Used:497.12GiB (90.39%)
>    /dev/mapper/root      550.00GiB
>
> Metadata,DUP: Size:72.00GiB, Used:8.38GiB (11.64%)
>    /dev/mapper/root      144.00GiB
>
> <...>
> ```
>
> Running a full metadata balance (`btrfs balance start -m /`) or a
> "staged" balance
> (e. g. `for u in {0..90..5}; do btrfs balance start -musage=$u /`)
> does not help / makes the overallocation worse.
>
> Bisect log:
> ```
> # bad: [0dd3ee31125508cd67f7e7172247f05b7fd1753a] Linux 6.7
> # good: [ffc253263a1375a65fa6c9f62a893e9767fbebfa] Linux 6.6
> git bisect start 'v6.7' 'v6.6'
> # bad: [deefd5024f0772cf56052ace9a8c347dc70bcaf3] Merge tag 'vfio-v6.7-rc1' of https://github.com/awilliam/linux-vfio
> git bisect bad deefd5024f0772cf56052ace9a8c347dc70bcaf3
> # bad: [5a6a09e97199d6600d31383055f9d43fbbcbe86f] Merge tag 'cgroup-for-6.7' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
> git bisect bad 5a6a09e97199d6600d31383055f9d43fbbcbe86f
> # good: [17047fbced563cf5abe5aa546f6a92af48900b69] bcachefs: Fix incorrectly freeing btree_path in alloc path
> git bisect good 17047fbced563cf5abe5aa546f6a92af48900b69
> # good: [b827ac419721a106ae2fccaa40576b0594edad92] exportfs: Change bcachefs fid_type enum to avoid conflicts
> git bisect good b827ac419721a106ae2fccaa40576b0594edad92
> # bad: [9ab021a1b57007a22761f6f41d91eb4aae10d145] Merge tag 'x86_cache_for_6.7_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
> git bisect bad 9ab021a1b57007a22761f6f41d91eb4aae10d145
> # good: [8b16da681eb0c9b9cb2f9abd0dade67559cfb48d] Merge tag 'nfsd-6.7' of git://git.kernel.org/pub/scm/linux/kernel/git/cel/linux
> git bisect good 8b16da681eb0c9b9cb2f9abd0dade67559cfb48d
> # bad: [07a274a8862dba86a270ced2a4c5ff3f7a01b66a] btrfs: remove redundant root argument from btrfs_update_inode_item()
> git bisect bad 07a274a8862dba86a270ced2a4c5ff3f7a01b66a
> # good: [3ee56a58ad8921cb43c49d56347a8e270871844c] btrfs: reserve space for delayed refs on a per ref basis
> git bisect good 3ee56a58ad8921cb43c49d56347a8e270871844c
> # bad: [9f9918a8017b7925da2fad16b4ccf6a14630f03e] btrfs: sysfs: announce presence of raid-stripe-tree
> git bisect bad 9f9918a8017b7925da2fad16b4ccf6a14630f03e
> # bad: [87463f7e0250d471fac41e7c9c45ae21d83b5f85] btrfs: zoned: factor out DUP bg handling from btrfs_load_block_group_zone_info
> git bisect bad 87463f7e0250d471fac41e7c9c45ae21d83b5f85
> # bad: [50564b651d01c19ce732819c5b3c3fd60707188e] btrfs: abort transaction on generation mismatch when marking eb as dirty
> git bisect bad 50564b651d01c19ce732819c5b3c3fd60707188e
> # bad: [28270e25c69a2c76ea1ed0922095bffb9b9a4f98] btrfs: always reserve space for delayed refs when starting transaction
> git bisect bad 28270e25c69a2c76ea1ed0922095bffb9b9a4f98
> # good: [adb86dbe426f9a54843d70092819deca220a224d] btrfs: stop doing excessive space reservation for csum deletion
> git bisect good adb86dbe426f9a54843d70092819deca220a224d
> # first bad commit: [28270e25c69a2c76ea1ed0922095bffb9b9a4f98] btrfs: always reserve space for delayed refs when starting transaction

This sounds like the generally more pessimistic metadata reservation
is triggering allocation of many metadata block groups
that never get used and then unused and therefore not
reclaimed/deleted. Not something impossible to happen before that,
but much more likely due to more reserved space.

I'll send some fixes.

Did you actually run into -ENOSPC issues?

Thanks.

> ```
>
> Thanks,
> --
> Ivan Shapovalov / intelfx /
>

  reply	other threads:[~2024-01-17 11:29 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-17  6:04 [6.7 regression] [BISECTED] 28270e25c69a causes overallocation of metadata space Ivan Shapovalov
2024-01-17 11:28 ` Filipe Manana [this message]
2024-01-17 14:21   ` Ivan Shapovalov
2024-01-20 14:46 ` Linux regression tracking #adding (Thorsten Leemhuis)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAL3q7H5UaYcAYHijBO+QTnnpruVQXvdirg05_X94KRKrKnXDZw@mail.gmail.com \
    --to=fdmanana@kernel.org \
    --cc=intelfx@intelfx.name \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.