[PATCH] ext4: fix bigalloc cluster freeing when hole punching under load

[PATCH] ext4: fix bigalloc cluster freeing when hole punching under load

[Eric Whitney]

Ext4 may not free clusters correctly when punching holes in bigalloc
file systems under high load conditions.  If it's not possible to
extend and restart the journal in ext4_ext_rm_leaf() when preparing to
remove blocks from a punched region, a retry of the entire punch
operation is triggered in ext4_ext_remove_space().  This causes a
partial cluster to be set to the first cluster in the extent found to
the right of the punched region.  However, if the punch operation
prior to the retry had made enough progress to delete one or more
extents and a partial cluster candidate for freeing had already been
recorded, the retry would overwrite the partial cluster.  The loss of
this information makes it impossible to correctly free the original
partial cluster in all cases.

This bug can cause generic/476 to fail when run as part of
xfstests-bld's bigalloc and bigalloc_1k test cases.  The failure is
reported when e2fsck detects bad iblocks counts greater than expected
in units of whole clusters and also detects a number of negative block
bitmap differences equal to the iblocks discrepancy in cluster units.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
---
 fs/ext4/extents.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 240b6dea5441..252bbbb5a2f4 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -2956,14 +2956,17 @@ int ext4_ext_remove_space(struct inode *inode, ext4_lblk_t start,
 			if (err < 0)
 				goto out;
 
-		} else if (sbi->s_cluster_ratio > 1 && end >= ex_end) {
+		} else if (sbi->s_cluster_ratio > 1 && end >= ex_end &&
+			   partial.state == initial) {
 			/*
-			 * If there's an extent to the right its first cluster
-			 * contains the immediate right boundary of the
-			 * truncated/punched region.  Set partial_cluster to
-			 * its negative value so it won't be freed if shared
-			 * with the current extent.  The end < ee_block case
-			 * is handled in ext4_ext_rm_leaf().
+			 * If we're punching, there's an extent to the right.
+			 * If the partial cluster hasn't been set, set it to
+			 * that extent's first cluster and its state to nofree
+			 * so it won't be freed should it contain blocks to be
+			 * removed. If it's already set (tofree/nofree), we're
+			 * retrying and keep the original partial cluster info
+			 * so a cluster marked tofree as a result of earlier
+			 * extent removal is not lost.
 			 */
 			lblk = ex_end + 1;
 			err = ext4_ext_search_right(inode, path, &lblk, &pblk,
-- 
2.11.0

Re: [PATCH] ext4: fix bigalloc cluster freeing when hole punching under load

[Theodore Y. Ts'o]

On Wed, Feb 27, 2019 at 05:02:04PM -0500, Eric Whitney wrote:
> Ext4 may not free clusters correctly when punching holes in bigalloc
> file systems under high load conditions.  If it's not possible to
> extend and restart the journal in ext4_ext_rm_leaf() when preparing to
> remove blocks from a punched region, a retry of the entire punch
> operation is triggered in ext4_ext_remove_space().  This causes a
> partial cluster to be set to the first cluster in the extent found to
> the right of the punched region.  However, if the punch operation
> prior to the retry had made enough progress to delete one or more
> extents and a partial cluster candidate for freeing had already been
> recorded, the retry would overwrite the partial cluster.  The loss of
> this information makes it impossible to correctly free the original
> partial cluster in all cases.
> 
> This bug can cause generic/476 to fail when run as part of
> xfstests-bld's bigalloc and bigalloc_1k test cases.  The failure is
> reported when e2fsck detects bad iblocks counts greater than expected
> in units of whole clusters and also detects a number of negative block
> bitmap differences equal to the iblocks discrepancy in cluster units.
> 
> Signed-off-by: Eric Whitney <enwlinux@gmail.com>

Thanks, applied.

							- Ted