All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: djwong@kernel.org
Cc: Brian Foster <bfoster@redhat.com>, Christoph Hellwig <hch@lst.de>,
	linux-xfs@vger.kernel.org, hch@infradead.org,
	david@fromorbit.com, bfoster@redhat.com
Subject: [PATCHSET v6 00/12] xfs: try harder to reclaim space when we run out
Date: Thu, 28 Jan 2021 18:17:46 -0800	[thread overview]
Message-ID: <161188666613.1943978.971196931920996596.stgit@magnolia> (raw)

Hi all,

Historically, when users ran out of space or quota when trying to write
to the filesystem, XFS didn't try very hard to reclaim space that it
might have speculatively allocated for the purpose of speeding up
front-end filesystem operations (appending writes, cow staging).  The
upcoming deferred inactivation series will greatly increase the amount
of allocated space that isn't actively being used to store user data.

Therefore, try to reduce the circumstances where we return EDQUOT or
ENOSPC to userspace by teaching the write paths to try to clear space
and retry the operation one time before giving up.

Previous iterations of this patchset made massive changes to the
codebase, but thanks to the transaction allocation helpers that Brian
pushed for (in the previous patchset) this is mostly no longer
necessary. :)

v2: clean up and rebase against 5.11.
v3: restructure the retry loops per dchinner suggestion
v4: simplify the calling convention of xfs_trans_reserve_quota_nblks
v5: constrain the open-coded 'goto retry' loops to the helpers created
    in the previous patchset
v6: move "xfs: try worst case space reservation upfront in
    xfs_reflink_remap_extent" to this series, open-code the qretry
    helpers in the (now very few) places they are used

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=reclaim-space-harder-5.12
---
 fs/xfs/xfs_file.c        |   24 +++---
 fs/xfs/xfs_icache.c      |  184 ++++++++++++++++++++++++++++++----------------
 fs/xfs/xfs_icache.h      |    7 +-
 fs/xfs/xfs_ioctl.c       |   12 +++
 fs/xfs/xfs_iops.c        |   11 +++
 fs/xfs/xfs_quota.h       |   16 ++--
 fs/xfs/xfs_reflink.c     |   30 +++++++-
 fs/xfs/xfs_trace.c       |    1 
 fs/xfs/xfs_trace.h       |   42 +++++++++++
 fs/xfs/xfs_trans.c       |   34 ++++++++-
 fs/xfs/xfs_trans_dquot.c |  116 ++++++++++++++++++++++++++---
 11 files changed, 370 insertions(+), 107 deletions(-)


             reply	other threads:[~2021-01-29  2:18 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-29  2:17 Darrick J. Wong [this message]
2021-01-29  2:17 ` [PATCH 01/12] xfs: trigger all block gc scans when low on quota space Darrick J. Wong
2021-01-29  2:17 ` [PATCH 02/12] xfs: don't stall cowblocks scan if we can't take locks Darrick J. Wong
2021-01-29  2:18 ` [PATCH 03/12] xfs: xfs_inode_free_quota_blocks should scan project quota Darrick J. Wong
2021-01-29  2:18 ` [PATCH 04/12] xfs: move and rename xfs_inode_free_quota_blocks to avoid conflicts Darrick J. Wong
2021-01-29  2:18 ` [PATCH 05/12] xfs: pass flags and return gc errors from xfs_blockgc_free_quota Darrick J. Wong
2021-01-29  2:18 ` [PATCH 06/12] xfs: try worst case space reservation upfront in xfs_reflink_remap_extent Darrick J. Wong
2021-01-29  2:18 ` [PATCH 07/12] xfs: flush eof/cowblocks if we can't reserve quota for file blocks Darrick J. Wong
     [not found]   ` <20210129160944.GE2665284@bfoster>
2021-01-29 17:47     ` Darrick J. Wong
2021-01-29  2:18 ` [PATCH 08/12] xfs: flush eof/cowblocks if we can't reserve quota for inode creation Darrick J. Wong
2021-01-29  2:18 ` [PATCH 09/12] xfs: flush eof/cowblocks if we can't reserve quota for chown Darrick J. Wong
2021-01-29  2:18 ` [PATCH 10/12] xfs: add a tracepoint for blockgc scans Darrick J. Wong
2021-01-29  2:18 ` [PATCH 11/12] xfs: refactor xfs_icache_free_{eof,cow}blocks call sites Darrick J. Wong
2021-01-29  2:18 ` [PATCH 12/12] xfs: flush speculative space allocations when we run out of space Darrick J. Wong
     [not found]   ` <20210129161047.GH2665284@bfoster>
2021-01-29 17:34     ` Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=161188666613.1943978.971196931920996596.stgit@magnolia \
    --to=djwong@kernel.org \
    --cc=bfoster@redhat.com \
    --cc=david@fromorbit.com \
    --cc=hch@infradead.org \
    --cc=hch@lst.de \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.