All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: djwong@kernel.org
Cc: linux-xfs@vger.kernel.org
Subject: [PATCH 4/7] xfs: automatic freeing of freshly allocated unwritten space
Date: Fri, 24 Nov 2023 15:47:56 -0800	[thread overview]
Message-ID: <170086926192.2768790.6648304136876480355.stgit@frogsfrogsfrogs> (raw)
In-Reply-To: <170086926113.2768790.10021834422326302654.stgit@frogsfrogsfrogs>

From: Darrick J. Wong <djwong@kernel.org>

As mentioned in the previous commit, online repair wants to allocate
space to write out a new metadata structure, and it also wants to hedge
against system crashes during repairs by logging (and later cancelling)
EFIs to free the space if we crash before committing the new data
structure.

Therefore, create a trio of functions to schedule automatic reaping of
freshly allocated unwritten space.  xfs_alloc_schedule_autoreap creates
a paused EFI representing the space we just allocated.  Once the
allocations are made and the autoreaps scheduled, we can start writing
to disk.

If the writes succeed, xfs_alloc_cancel_autoreap marks the EFI work
items as stale and unpauses the pending deferred work item.  Assuming
that's done in the same transaction that commits the new structure into
the filesystem, we guarantee that either the new object is fully
visible, or that all the space gets reclaimed.

If the writes succeed but only part of an extent was used, repair must
call the same _cancel_autoreap function to kill the first EFI and then
log a new EFI to free the unused space.  The first EFI is already
committed, so it cannot be changed.

For full extents that aren't used, xfs_alloc_commit_autoreap will
unpause the EFI, which results in the space being freed during the next
_defer_finish cycle.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 fs/xfs/libxfs/xfs_alloc.c |  104 +++++++++++++++++++++++++++++++++++++++++++--
 fs/xfs/libxfs/xfs_alloc.h |   12 +++++
 fs/xfs/xfs_extfree_item.c |   10 +++-
 3 files changed, 118 insertions(+), 8 deletions(-)


diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index c35224ad9428a..4940f9377f21a 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -2522,14 +2522,15 @@ xfs_defer_agfl_block(
  * Add the extent to the list of extents to be free at transaction end.
  * The list is maintained sorted (by block number).
  */
-int
-xfs_free_extent_later(
+static int
+xfs_defer_extent_free(
 	struct xfs_trans		*tp,
 	xfs_fsblock_t			bno,
 	xfs_filblks_t			len,
 	const struct xfs_owner_info	*oinfo,
 	enum xfs_ag_resv_type		type,
-	bool				skip_discard)
+	bool				skip_discard,
+	struct xfs_defer_pending	**dfpp)
 {
 	struct xfs_extent_free_item	*xefi;
 	struct xfs_mount		*mp = tp->t_mountp;
@@ -2577,10 +2578,105 @@ xfs_free_extent_later(
 			XFS_FSB_TO_AGBNO(tp->t_mountp, bno), len);
 
 	xfs_extent_free_get_group(mp, xefi);
-	xfs_defer_add(tp, XFS_DEFER_OPS_TYPE_FREE, &xefi->xefi_list);
+	*dfpp = xfs_defer_add(tp, XFS_DEFER_OPS_TYPE_FREE, &xefi->xefi_list);
 	return 0;
 }
 
+int
+xfs_free_extent_later(
+	struct xfs_trans		*tp,
+	xfs_fsblock_t			bno,
+	xfs_filblks_t			len,
+	const struct xfs_owner_info	*oinfo,
+	enum xfs_ag_resv_type		type,
+	bool				skip_discard)
+{
+	struct xfs_defer_pending	*dontcare = NULL;
+
+	return xfs_defer_extent_free(tp, bno, len, oinfo, type, skip_discard,
+			&dontcare);
+}
+
+/*
+ * Set up automatic freeing of unwritten space in the filesystem.
+ *
+ * This function attached a paused deferred extent free item to the
+ * transaction.  Pausing means that the EFI will be logged in the next
+ * transaction commit, but the pending EFI will not be finished until the
+ * pending item is unpaused.
+ *
+ * If the system goes down after the EFI has been persisted to the log but
+ * before the pending item is unpaused, log recovery will find the EFI, fail to
+ * find the EFD, and free the space.
+ *
+ * If the pending item is unpaused, the next transaction commit will log an EFD
+ * without freeing the space.
+ *
+ * Caller must ensure that the tp, fsbno, len, oinfo, and resv flags of the
+ * @args structure are set to the relevant values.
+ */
+int
+xfs_alloc_schedule_autoreap(
+	const struct xfs_alloc_arg	*args,
+	bool				skip_discard,
+	struct xfs_alloc_autoreap	*aarp)
+{
+	int				error;
+
+	error = xfs_defer_extent_free(args->tp, args->fsbno, args->len,
+			&args->oinfo, args->resv, skip_discard, &aarp->dfp);
+	if (error)
+		return error;
+
+	xfs_defer_item_pause(args->tp, aarp->dfp);
+	return 0;
+}
+
+/*
+ * Cancel automatic freeing of unwritten space in the filesystem.
+ *
+ * Earlier, we created a paused deferred extent free item and attached it to
+ * this transaction so that we could automatically roll back a new space
+ * allocation if the system went down.  Now we want to cancel the paused work
+ * item by marking the EFI stale so we don't actually free the space, unpausing
+ * the pending item and logging an EFD.
+ *
+ * The caller generally should have already mapped the space into the ondisk
+ * filesystem.  If the reserved space was partially used, the caller must call
+ * xfs_free_extent_later to create a new EFI to free the unused space.
+ */
+void
+xfs_alloc_cancel_autoreap(
+	struct xfs_trans		*tp,
+	struct xfs_alloc_autoreap	*aarp)
+{
+	struct xfs_defer_pending	*dfp = aarp->dfp;
+	struct xfs_extent_free_item	*xefi;
+
+	if (!dfp)
+		return;
+
+	list_for_each_entry(xefi, &dfp->dfp_work, xefi_list)
+		xefi->xefi_flags |= XFS_EFI_CANCELLED;
+
+	xfs_defer_item_unpause(tp, dfp);
+}
+
+/*
+ * Commit automatic freeing of unwritten space in the filesystem.
+ *
+ * This unpauses an earlier _schedule_autoreap and commits to freeing the
+ * allocated space.  Call this if none of the reserved space was used.
+ */
+void
+xfs_alloc_commit_autoreap(
+	struct xfs_trans		*tp,
+	struct xfs_alloc_autoreap	*aarp)
+{
+	if (aarp->dfp)
+		xfs_defer_item_unpause(tp, aarp->dfp);
+}
+
 #ifdef DEBUG
 /*
  * Check if an AGF has a free extent record whose length is equal to
diff --git a/fs/xfs/libxfs/xfs_alloc.h b/fs/xfs/libxfs/xfs_alloc.h
index 6b95d1d8a8537..851cafbd64494 100644
--- a/fs/xfs/libxfs/xfs_alloc.h
+++ b/fs/xfs/libxfs/xfs_alloc.h
@@ -255,6 +255,18 @@ void xfs_extent_free_get_group(struct xfs_mount *mp,
 #define XFS_EFI_SKIP_DISCARD	(1U << 0) /* don't issue discard */
 #define XFS_EFI_ATTR_FORK	(1U << 1) /* freeing attr fork block */
 #define XFS_EFI_BMBT_BLOCK	(1U << 2) /* freeing bmap btree block */
+#define XFS_EFI_CANCELLED	(1U << 3) /* dont actually free the space */
+
+struct xfs_alloc_autoreap {
+	struct xfs_defer_pending	*dfp;
+};
+
+int xfs_alloc_schedule_autoreap(const struct xfs_alloc_arg *args,
+		bool skip_discard, struct xfs_alloc_autoreap *aarp);
+void xfs_alloc_cancel_autoreap(struct xfs_trans *tp,
+		struct xfs_alloc_autoreap *aarp);
+void xfs_alloc_commit_autoreap(struct xfs_trans *tp,
+		struct xfs_alloc_autoreap *aarp);
 
 extern struct kmem_cache	*xfs_extfree_item_cache;
 
diff --git a/fs/xfs/xfs_extfree_item.c b/fs/xfs/xfs_extfree_item.c
index 9e7b58f3566c0..4af5b3b338b19 100644
--- a/fs/xfs/xfs_extfree_item.c
+++ b/fs/xfs/xfs_extfree_item.c
@@ -381,7 +381,7 @@ xfs_trans_free_extent(
 	uint				next_extent;
 	xfs_agblock_t			agbno = XFS_FSB_TO_AGBNO(mp,
 							xefi->xefi_startblock);
-	int				error;
+	int				error = 0;
 
 	oinfo.oi_owner = xefi->xefi_owner;
 	if (xefi->xefi_flags & XFS_EFI_ATTR_FORK)
@@ -392,9 +392,11 @@ xfs_trans_free_extent(
 	trace_xfs_bmap_free_deferred(tp->t_mountp, xefi->xefi_pag->pag_agno, 0,
 			agbno, xefi->xefi_blockcount);
 
-	error = __xfs_free_extent(tp, xefi->xefi_pag, agbno,
-			xefi->xefi_blockcount, &oinfo, xefi->xefi_agresv,
-			xefi->xefi_flags & XFS_EFI_SKIP_DISCARD);
+	if (!(xefi->xefi_flags & XFS_EFI_CANCELLED))
+		error = __xfs_free_extent(tp, xefi->xefi_pag, agbno,
+				xefi->xefi_blockcount, &oinfo,
+				xefi->xefi_agresv,
+				xefi->xefi_flags & XFS_EFI_SKIP_DISCARD);
 
 	/*
 	 * Mark the transaction dirty, even on error. This ensures the


  parent reply	other threads:[~2023-11-24 23:47 UTC|newest]

Thread overview: 155+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-24 23:39 [MEGAPATCHSET v28] xfs: online repair, second part of part 1 Darrick J. Wong
2023-11-24 23:44 ` [PATCHSET v28.0 0/1] xfs: prevent livelocks in xchk_iget Darrick J. Wong
2023-11-24 23:46   ` [PATCH 1/1] xfs: make xchk_iget safer in the presence of corrupt inode btrees Darrick J. Wong
2023-11-25  4:57     ` Christoph Hellwig
2023-11-27 21:55       ` Darrick J. Wong
2023-11-24 23:45 ` [PATCHSET v28.0 0/7] xfs: reserve disk space for online repairs Darrick J. Wong
2023-11-24 23:47   ` [PATCH 1/7] xfs: don't append work items to logged xfs_defer_pending objects Darrick J. Wong
2023-11-25  5:04     ` Christoph Hellwig
2023-11-24 23:47   ` [PATCH 2/7] xfs: allow pausing of pending deferred work items Darrick J. Wong
2023-11-25  5:05     ` Christoph Hellwig
2023-11-24 23:47   ` [PATCH 3/7] xfs: remove __xfs_free_extent_later Darrick J. Wong
2023-11-25  5:05     ` Christoph Hellwig
2023-11-24 23:47   ` Darrick J. Wong [this message]
2023-11-25  5:06     ` [PATCH 4/7] xfs: automatic freeing of freshly allocated unwritten space Christoph Hellwig
2023-11-24 23:48   ` [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging Darrick J. Wong
2023-11-26 13:14     ` Christoph Hellwig
2023-11-27 22:34       ` Darrick J. Wong
2023-11-28  5:41         ` Christoph Hellwig
2023-11-28 17:02           ` Darrick J. Wong
2023-11-24 23:48   ` [PATCH 6/7] xfs: log EFIs for all btree blocks being used to stage a btree Darrick J. Wong
2023-11-26 13:15     ` Christoph Hellwig
2023-11-24 23:48   ` [PATCH 7/7] xfs: force small EFIs for reaping btree extents Darrick J. Wong
2023-11-25  5:13     ` Christoph Hellwig
2023-11-27 22:46       ` Darrick J. Wong
2023-11-24 23:45 ` [PATCHSET v28.0 0/4] xfs: prepare repair for bulk loading Darrick J. Wong
2023-11-24 23:48   ` [PATCH 1/4] xfs: force all buffers to be written during btree bulk load Darrick J. Wong
2023-11-25  5:49     ` Christoph Hellwig
2023-11-28  1:50       ` Darrick J. Wong
2023-11-28  7:13         ` Christoph Hellwig
2023-11-28 15:18           ` Christoph Hellwig
2023-11-28 17:07             ` Darrick J. Wong
2023-11-30  4:33               ` Christoph Hellwig
2023-11-24 23:49   ` [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors Darrick J. Wong
2023-11-25  5:50     ` Christoph Hellwig
2023-11-28  1:44       ` Darrick J. Wong
2023-11-28  5:42         ` Christoph Hellwig
2023-11-28 17:07           ` Darrick J. Wong
2023-11-24 23:49   ` [PATCH 3/4] xfs: move btree bulkload record initialization to ->get_record implementations Darrick J. Wong
2023-11-25  5:51     ` Christoph Hellwig
2023-11-24 23:49   ` [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree Darrick J. Wong
2023-11-25  5:53     ` Christoph Hellwig
2023-11-27 22:56       ` Darrick J. Wong
2023-11-28 20:11         ` Darrick J. Wong
2023-11-29  5:50           ` Christoph Hellwig
2023-11-29  5:57             ` Darrick J. Wong
2023-11-24 23:45 ` [PATCHSET v28.0 0/5] xfs: online repair of AG btrees Darrick J. Wong
2023-11-24 23:50   ` [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps Darrick J. Wong
2023-11-25  5:57     ` Christoph Hellwig
2023-11-28  1:34       ` Darrick J. Wong
2023-11-28  5:43         ` Christoph Hellwig
2023-11-24 23:50   ` [PATCH 2/5] xfs: roll the scrub transaction after completing a repair Darrick J. Wong
2023-11-25  6:05     ` Christoph Hellwig
2023-11-28  1:29       ` Darrick J. Wong
2023-11-24 23:50   ` [PATCH 3/5] xfs: repair free space btrees Darrick J. Wong
2023-11-25  6:11     ` Christoph Hellwig
2023-11-28  1:05       ` Darrick J. Wong
2023-11-28 15:10     ` Christoph Hellwig
2023-11-28 21:13       ` Darrick J. Wong
2023-11-29  5:56         ` Christoph Hellwig
2023-11-29  6:18           ` Darrick J. Wong
2023-11-29  6:24             ` Christoph Hellwig
2023-11-29  6:26               ` Darrick J. Wong
2023-11-24 23:50   ` [PATCH 4/5] xfs: repair inode btrees Darrick J. Wong
2023-11-25  6:12     ` Christoph Hellwig
2023-11-28  1:09       ` Darrick J. Wong
2023-11-28 15:57     ` Christoph Hellwig
2023-11-28 21:37       ` Darrick J. Wong
2023-11-24 23:51   ` [PATCH 5/5] xfs: repair refcount btrees Darrick J. Wong
2023-11-28 16:07     ` Christoph Hellwig
2023-11-24 23:45 ` [PATCHSET v28.0 0/7] xfs: online repair of inodes and forks Darrick J. Wong
2023-11-24 23:51   ` [PATCH 1/7] xfs: disable online repair quota helpers when quota not enabled Darrick J. Wong
2023-11-25  6:13     ` Christoph Hellwig
2023-11-24 23:51   ` [PATCH 2/7] xfs: try to attach dquots to files before repairing them Darrick J. Wong
2023-11-25  6:14     ` Christoph Hellwig
2023-11-24 23:51   ` [PATCH 3/7] xfs: repair inode records Darrick J. Wong
2023-11-28 17:08     ` Christoph Hellwig
2023-11-28 23:08       ` Darrick J. Wong
2023-11-29  6:02         ` Christoph Hellwig
2023-12-05 23:08           ` Darrick J. Wong
2023-12-06  5:16             ` Christoph Hellwig
2023-11-24 23:52   ` [PATCH 4/7] xfs: zap broken inode forks Darrick J. Wong
2023-11-30  4:44     ` Christoph Hellwig
2023-11-30 21:08       ` Darrick J. Wong
2023-12-04  4:39         ` Christoph Hellwig
2023-12-04 20:43           ` Darrick J. Wong
2023-12-05  4:28             ` Christoph Hellwig
2023-11-24 23:52   ` [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory Darrick J. Wong
2023-11-30  4:47     ` Christoph Hellwig
2023-11-30 21:37       ` Darrick J. Wong
2023-12-04  4:41         ` Christoph Hellwig
2023-12-04 20:44           ` Darrick J. Wong
2023-11-24 23:52   ` [PATCH 6/7] xfs: skip the rmapbt search on an empty attr fork unless we know it was zapped Darrick J. Wong
2023-11-24 23:52   ` [PATCH 7/7] xfs: repair obviously broken inode modes Darrick J. Wong
2023-11-30  4:49     ` Christoph Hellwig
2023-11-30 21:18       ` Darrick J. Wong
2023-12-04  4:42         ` Christoph Hellwig
2023-11-24 23:46 ` [PATCHSET v28.0 0/5] xfs: online repair of file fork mappings Darrick J. Wong
2023-11-24 23:53   ` [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents Darrick J. Wong
2023-11-30  4:53     ` Christoph Hellwig
2023-11-30 21:48       ` Darrick J. Wong
2023-12-04  4:42         ` Christoph Hellwig
2023-11-24 23:53   ` [PATCH 2/5] xfs: repair inode fork block mapping data structures Darrick J. Wong
2023-11-30  5:07     ` Christoph Hellwig
2023-12-01  1:38       ` Darrick J. Wong
2023-11-24 23:53   ` [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper Darrick J. Wong
2023-11-28 14:20     ` Christoph Hellwig
2023-11-29  5:42       ` Darrick J. Wong
2023-11-29  6:03         ` Christoph Hellwig
2023-11-24 23:53   ` [PATCH 4/5] xfs: create a ranged query function for refcount btrees Darrick J. Wong
2023-11-28 13:59     ` Christoph Hellwig
2023-11-24 23:54   ` [PATCH 5/5] xfs: repair problems in CoW forks Darrick J. Wong
2023-11-30  5:10     ` Christoph Hellwig
2023-11-24 23:46 ` [PATCHSET v28.0 0/6] xfs: online repair of rt bitmap file Darrick J. Wong
2023-11-24 23:54   ` [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly Darrick J. Wong
2023-11-28 14:04     ` Christoph Hellwig
2023-11-28 23:27       ` Darrick J. Wong
2023-11-29  6:05         ` Christoph Hellwig
2023-11-29  6:20           ` Darrick J. Wong
2023-11-24 23:54   ` [PATCH 2/6] xfs: check rt summary " Darrick J. Wong
2023-11-28 14:05     ` Christoph Hellwig
2023-11-28 23:30       ` Darrick J. Wong
2023-11-29  1:23         ` Darrick J. Wong
2023-11-29  6:05         ` Christoph Hellwig
2023-11-29  6:21           ` Darrick J. Wong
2023-11-29  6:23             ` Christoph Hellwig
2023-11-30  0:10               ` Darrick J. Wong
2023-11-24 23:54   ` [PATCH 3/6] xfs: always check the rtbitmap and rtsummary files Darrick J. Wong
2023-11-28 14:06     ` Christoph Hellwig
2023-11-24 23:55   ` [PATCH 4/6] xfs: repair the inode core and forks of a metadata inode Darrick J. Wong
2023-11-30  5:12     ` Christoph Hellwig
2023-11-24 23:55   ` [PATCH 5/6] xfs: create a new inode fork block unmap helper Darrick J. Wong
2023-11-25  6:17     ` Christoph Hellwig
2023-11-24 23:55   ` [PATCH 6/6] xfs: online repair of realtime bitmaps Darrick J. Wong
2023-11-30  5:16     ` Christoph Hellwig
2023-11-24 23:46 ` [PATCHSET v28.0 0/5] xfs: online repair of quota and rt metadata files Darrick J. Wong
2023-11-24 23:56   ` [PATCH 1/5] xfs: check the ondisk space mapping behind a dquot Darrick J. Wong
2023-11-30  5:17     ` Christoph Hellwig
2023-11-24 23:56   ` [PATCH 2/5] xfs: check dquot resource timers Darrick J. Wong
2023-11-30  5:17     ` Christoph Hellwig
2023-11-24 23:56   ` [PATCH 3/5] xfs: pull xfs_qm_dqiterate back into scrub Darrick J. Wong
2023-11-30  5:22     ` Christoph Hellwig
2023-11-24 23:56   ` [PATCH 4/5] xfs: improve dquot iteration for scrub Darrick J. Wong
2023-11-30  5:25     ` Christoph Hellwig
2023-11-24 23:57   ` [PATCH 5/5] xfs: repair quotas Darrick J. Wong
2023-11-30  5:33     ` Christoph Hellwig
2023-11-30 22:10       ` Darrick J. Wong
2023-12-04  4:48         ` Christoph Hellwig
2023-12-04 20:52           ` Darrick J. Wong
2023-12-05  4:27             ` Christoph Hellwig
2023-12-05  5:20               ` Darrick J. Wong
2023-12-04  4:49         ` Christoph Hellwig
  -- strict thread matches above, loose matches on Subject: below --
2023-09-26 23:29 [PATCHSET v27.0 0/7] xfs: reserve disk space for online repairs Darrick J. Wong
2023-09-26 23:32 ` [PATCH 4/7] xfs: automatic freeing of freshly allocated unwritten space Darrick J. Wong
2023-10-05  3:47   ` Dave Chinner
2023-10-06  5:12     ` Darrick J. Wong
2023-11-24 23:32       ` Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=170086926192.2768790.6648304136876480355.stgit@frogsfrogsfrogs \
    --to=djwong@kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.