All of lore.kernel.org
 help / color / mirror / Atom feed
From: Josef Bacik <josef@toxicpanda.com>
To: kernel-team@fb.com, linux-btrfs@vger.kernel.org
Subject: [PATCH 0/5][v2] Fix global reserve size and can overcommit
Date: Thu, 22 Aug 2019 15:18:59 -0400	[thread overview]
Message-ID: <20190822191904.13939-1-josef@toxicpanda.com> (raw)

This didn't series didn't change much, but I did move things around a little bit
in previous series so these needed to be updated.

 fs/btrfs/block-rsv.c  | 36 ++++++++++++++++++++++++++----------
 fs/btrfs/space-info.c | 51 ++++++++++++++++++++++++++-------------------------
 2 files changed, 52 insertions(+), 35 deletions(-)

v1->v2:
- Rebased with the new name changes from previous patches.
- Added the reviewed-by's.
- Added a few lockdep_assert_held()'s.
- Patch " btrfs: use btrfs_try_granting_tickets in update_global_rsv" fixed to
  just use btrfs_try_granting_tickets directly since we're already holding the
  space_info->lock.

-- Original email --
We hit a pretty crappy corner case in production that resulted in boxes slowing
down to a crawl.

can_overcommit() will not allow us to overcommit if there is not enough "real"
space to satisfy the global reserve.  This is for hysterical raisins, we used to
not be able to allocate block groups a transaction commit time, so running out
of real space for the global reserve would result in an aborted transaction.

However that limitation was fixed years ago, so we no longer need to impose that
limitation on ourselves and can just overcommit with reckless abandon.

But this change exposed a bunch of corner cases in how we deal with very small
file systems.  A lot of enospc related xfstests make very tiny file systems.
Btrfs by default gives you an 8mib metadata chunk.  We default to having a
minimum global reserve size of 16mib.  This is problematic.

Before we would allocate a new metadata chunk, which actually meant we were
"passing" these tests with 24mib metadata chunks but 256kib of used metadata,
and drastically less data space than was intended.  With my change we started
failing these tests because we were no longer allocating the extra metadata
chunks.

The changes to the global reserve are to make it so we pass these xfstests, as
well as add some real hard logic to the minimum size of the global reserve,
rather than just an arbitrary 16mib.  In practice those changes won't affect
real users because real users use more than 8mib of metadata and will get full
chunks.

These patches fix the original problem I intended to fix, and also have the nice
side-effect of making a bunch of the enospc xfstests finish _much_ faster.  The
diffstat is as follows

 fs/btrfs/block-rsv.c  | 43 ++++++++++++++++++++++++++++++-------------
 fs/btrfs/space-info.c | 51 +++++++++++++++++++++++++--------------------------
 2 files changed, 55 insertions(+), 39 deletions(-)

Thanks,

Josef


             reply	other threads:[~2019-08-22 19:19 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-22 19:18 Josef Bacik [this message]
2019-08-22 19:19 ` [PATCH 1/5] btrfs: change the minimum global reserve size Josef Bacik
2019-08-22 19:19 ` [PATCH 2/5] btrfs: always reserve our entire size for the global reserve Josef Bacik
2019-08-22 19:19 ` [PATCH 3/5] btrfs: use btrfs_try_granting_tickets in update_global_rsv Josef Bacik
2019-08-22 19:19 ` [PATCH 4/5] btrfs: do not account global reserve in can_overcommit Josef Bacik
2020-01-23 16:14   ` Anand Jain
2020-01-24  0:48     ` Josef Bacik
2019-08-22 19:19 ` [PATCH 5/5] btrfs: add enospc debug messages for ticket failure Josef Bacik
2019-08-23 13:17 ` [PATCH 0/5][v2] Fix global reserve size and can overcommit David Sterba
2019-08-28 18:03   ` David Sterba

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190822191904.13939-1-josef@toxicpanda.com \
    --to=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.