All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCHSET v7 00/12] xfs: try harder to reclaim space when we run out
@ 2021-02-01  2:05 Darrick J. Wong
  2021-02-01  2:05 ` [PATCH 01/12] xfs: trigger all block gc scans when low on quota space Darrick J. Wong
                   ` (11 more replies)
  0 siblings, 12 replies; 26+ messages in thread
From: Darrick J. Wong @ 2021-02-01  2:05 UTC (permalink / raw)
  To: djwong; +Cc: Brian Foster, Christoph Hellwig, linux-xfs, hch, david, bfoster

Hi all,

Historically, when users ran out of space or quota when trying to write
to the filesystem, XFS didn't try very hard to reclaim space that it
might have speculatively allocated for the purpose of speeding up
front-end filesystem operations (appending writes, cow staging).  The
upcoming deferred inactivation series will greatly increase the amount
of allocated space that isn't actively being used to store user data.

Therefore, try to reduce the circumstances where we return EDQUOT or
ENOSPC to userspace by teaching the write paths to try to clear space
and retry the operation one time before giving up.

Previous iterations of this patchset made massive changes to the
codebase, but thanks to the transaction allocation helpers that Brian
pushed for (in the previous patchset) this is mostly no longer
necessary. :)

v2: clean up and rebase against 5.11.
v3: restructure the retry loops per dchinner suggestion
v4: simplify the calling convention of xfs_trans_reserve_quota_nblks
v5: constrain the open-coded 'goto retry' loops to the helpers created
    in the previous patchset
v6: move "xfs: try worst case space reservation upfront in
    xfs_reflink_remap_extent" to this series, open-code the qretry
    helpers in the (now very few) places they are used
v7: rebase chown stuff to use new helpers introduced in quota cleanups

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=reclaim-space-harder-5.12
---
 fs/xfs/xfs_file.c    |   24 +++----
 fs/xfs/xfs_icache.c  |  184 +++++++++++++++++++++++++++++++++-----------------
 fs/xfs/xfs_icache.h  |    7 +-
 fs/xfs/xfs_ioctl.c   |    2 +
 fs/xfs/xfs_reflink.c |   28 +++++++-
 fs/xfs/xfs_trace.c   |    1 
 fs/xfs/xfs_trace.h   |   42 +++++++++++
 fs/xfs/xfs_trans.c   |   38 ++++++++++
 8 files changed, 244 insertions(+), 82 deletions(-)


^ permalink raw reply	[flat|nested] 26+ messages in thread
* [PATCHSET v6 00/12] xfs: try harder to reclaim space when we run out
@ 2021-01-29  2:17 Darrick J. Wong
  2021-01-29  2:18 ` [PATCH 09/12] xfs: flush eof/cowblocks if we can't reserve quota for chown Darrick J. Wong
  0 siblings, 1 reply; 26+ messages in thread
From: Darrick J. Wong @ 2021-01-29  2:17 UTC (permalink / raw)
  To: djwong; +Cc: Brian Foster, Christoph Hellwig, linux-xfs, hch, david, bfoster

Hi all,

Historically, when users ran out of space or quota when trying to write
to the filesystem, XFS didn't try very hard to reclaim space that it
might have speculatively allocated for the purpose of speeding up
front-end filesystem operations (appending writes, cow staging).  The
upcoming deferred inactivation series will greatly increase the amount
of allocated space that isn't actively being used to store user data.

Therefore, try to reduce the circumstances where we return EDQUOT or
ENOSPC to userspace by teaching the write paths to try to clear space
and retry the operation one time before giving up.

Previous iterations of this patchset made massive changes to the
codebase, but thanks to the transaction allocation helpers that Brian
pushed for (in the previous patchset) this is mostly no longer
necessary. :)

v2: clean up and rebase against 5.11.
v3: restructure the retry loops per dchinner suggestion
v4: simplify the calling convention of xfs_trans_reserve_quota_nblks
v5: constrain the open-coded 'goto retry' loops to the helpers created
    in the previous patchset
v6: move "xfs: try worst case space reservation upfront in
    xfs_reflink_remap_extent" to this series, open-code the qretry
    helpers in the (now very few) places they are used

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=reclaim-space-harder-5.12
---
 fs/xfs/xfs_file.c        |   24 +++---
 fs/xfs/xfs_icache.c      |  184 ++++++++++++++++++++++++++++++----------------
 fs/xfs/xfs_icache.h      |    7 +-
 fs/xfs/xfs_ioctl.c       |   12 +++
 fs/xfs/xfs_iops.c        |   11 +++
 fs/xfs/xfs_quota.h       |   16 ++--
 fs/xfs/xfs_reflink.c     |   30 +++++++-
 fs/xfs/xfs_trace.c       |    1 
 fs/xfs/xfs_trace.h       |   42 +++++++++++
 fs/xfs/xfs_trans.c       |   34 ++++++++-
 fs/xfs/xfs_trans_dquot.c |  116 ++++++++++++++++++++++++++---
 11 files changed, 370 insertions(+), 107 deletions(-)


^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2021-02-02 16:56 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-01  2:05 [PATCHSET v7 00/12] xfs: try harder to reclaim space when we run out Darrick J. Wong
2021-02-01  2:05 ` [PATCH 01/12] xfs: trigger all block gc scans when low on quota space Darrick J. Wong
2021-02-01  2:05 ` [PATCH 02/12] xfs: don't stall cowblocks scan if we can't take locks Darrick J. Wong
2021-02-01  2:05 ` [PATCH 03/12] xfs: xfs_inode_free_quota_blocks should scan project quota Darrick J. Wong
2021-02-01  2:05 ` [PATCH 04/12] xfs: move and rename xfs_inode_free_quota_blocks to avoid conflicts Darrick J. Wong
2021-02-01  2:05 ` [PATCH 05/12] xfs: pass flags and return gc errors from xfs_blockgc_free_quota Darrick J. Wong
2021-02-01  2:06 ` [PATCH 06/12] xfs: try worst case space reservation upfront in xfs_reflink_remap_extent Darrick J. Wong
2021-02-01 12:24   ` Christoph Hellwig
2021-02-01  2:06 ` [PATCH 07/12] xfs: flush eof/cowblocks if we can't reserve quota for file blocks Darrick J. Wong
2021-02-01 12:32   ` Christoph Hellwig
2021-02-01 19:01     ` Darrick J. Wong
2021-02-02 15:38   ` Brian Foster
2021-02-02 16:53     ` Darrick J. Wong
2021-02-01  2:06 ` [PATCH 08/12] xfs: flush eof/cowblocks if we can't reserve quota for inode creation Darrick J. Wong
2021-02-01 12:35   ` Christoph Hellwig
2021-02-01 19:03     ` Darrick J. Wong
2021-02-02 15:39   ` Brian Foster
2021-02-01  2:06 ` [PATCH 09/12] xfs: flush eof/cowblocks if we can't reserve quota for chown Darrick J. Wong
2021-02-01 12:36   ` Christoph Hellwig
2021-02-01 19:12     ` Darrick J. Wong
2021-02-02 15:39   ` Brian Foster
2021-02-01  2:06 ` [PATCH 10/12] xfs: add a tracepoint for blockgc scans Darrick J. Wong
2021-02-01  2:06 ` [PATCH 11/12] xfs: refactor xfs_icache_free_{eof,cow}blocks call sites Darrick J. Wong
2021-02-01  2:06 ` [PATCH 12/12] xfs: flush speculative space allocations when we run out of space Darrick J. Wong
2021-02-02 15:39   ` Brian Foster
  -- strict thread matches above, loose matches on Subject: below --
2021-01-29  2:17 [PATCHSET v6 00/12] xfs: try harder to reclaim space when we run out Darrick J. Wong
2021-01-29  2:18 ` [PATCH 09/12] xfs: flush eof/cowblocks if we can't reserve quota for chown Darrick J. Wong

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.