[PATCH V8 00/19] xfs: Extend per-inode extent counters

[PATCH V8 00/19] xfs: Extend per-inode extent counters

[Chandan Babu R]

The commit xfs: fix inode fork extent count overflow
(3f8a4f1d876d3e3e49e50b0396eaffcc4ba71b08) mentions that 10 billion
data fork extents should be possible to create. However the
corresponding on-disk field has a signed 32-bit type. Hence this
patchset extends the per-inode data fork extent counter to 64 bits
(out of which 48 bits are used to store the extent count).

Also, XFS has an attribute fork extent counter which is 16 bits
wide. A workload that,
1. Creates 1 million 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to insert 400,000 new 255-byte sized xattrs
   causes the xattr extent counter to overflow.

Dave tells me that there are instances where a single file has more
than 100 million hardlinks. With parent pointers being stored in
xattrs, we will overflow the signed 16-bits wide attribute extent
counter when large number of hardlinks are created. Hence this
patchset extends the on-disk field to 32-bits.

The following changes are made to accomplish this,
1. A 64-bit inode field is carved out of existing di_pad and
   di_flushiter fields to hold the 64-bit data fork extent counter.
2. The existing 32-bit inode data fork extent counter will be used to
   hold the attribute fork extent counter.
3. A new incompat superblock flag to prevent older kernels from mounting
   the filesystem.

The patchset has been tested by executing xfstests with the following
mkfs.xfs options,
1. -m crc=0 -b size=1k
2. -m crc=0 -b size=4k
3. -m crc=0 -b size=512
4. -m rmapbt=1,reflink=1 -b size=1k
5. -m rmapbt=1,reflink=1 -b size=4k

Each of the above test scenarios were executed on the following
combinations (For V4 FS test scenario, the last combination was
omitted).
|---------------------------+-----------|
| Xfsprogs                  | Kernel    |
|---------------------------+-----------|
| Unpatched                 | Patched   |
| Patched (disable nrext64) | Unpatched |
| Patched (disable nrext64) | Patched   |
| Patched (enable nrext64)  | Patched   |
|---------------------------+-----------|

I have also written tests to check if the correct extent counter
fields are updated with/without the new incompat flag and to verify
upgrading older fs instances to support large extent counters. I have
also fixed xfs/270 test to work with the new code base.

These patches can also be obtained from
https://github.com/chandanr/linux.git at branch
xfs-incompat-extend-extcnt-v8.

Changelog:
V7 -> V8:
1. Do not roll a transaction after upgrading an inode to "Large extent
   counter" feature. Any transaction which can cause an inode's extent counter
   to change, will have included the space required to log the inode in its
   transaction reservation calculation.  
   This means that the patch "xfs: xfs_growfs_rt_alloc: Unlock inode
   explicitly rather than through iop_committing()" is no longer required.
2. Use XFS_MAX_EXTCNT_DATA_FORK_LARGE & XFS_MAX_EXTCNT_ATTR_FORK_LARGE to
   represent large extent counter limits. Similarly, use
   XFS_MAX_EXTCNT_DATA_FORK_SMALL & XFS_MAX_EXTCNT_ATTR_FORK_SMALL to
   represent previously defined extent counter limits.
3. Decouple XFS_IBULK flags from XFS_IWALK flags in a separate patch.
4. Bulkstat operation now returns XFS_MAX_EXTCNT_DATA_FORK_SMALL as the extent
   count if data fork extent count exceeds XFS_MAX_EXTCNT_DATA_FORK_SMALL and
   userspace program isn't aware of large extent counters.

V6 -> V7:
1. Address the following review comments from V6,
   - Revert xfs_ibulk->flags to "unsigned int" type.
   - Fix definition of XFS_IBULK_NREXT64 to be independent of IWALK flags.
   - Fix possible double free of transaction handle in xfs_growfs_rt_alloc().

V5 -> V6:
1. Rebase on Linux-v5.17-rc4.
2. Upgrade inodes to use large extent counters from within a
   transaction context.

V4 -> V5:
1. Rebase on xfs-linux/for-next.
2. Use howmany_64() to compute height of maximum bmbt tree.
3. Rename disk and log inode's di_big_dextcnt to di_big_nextents.
4. Rename disk and log inode's di_big_aextcnt to di_big_anextents.
5. Since XFS_IBULK_NREXT64 is not associated with inode walking
   functionality, define it as the 32nd bit and mask it when passing
   xfs_ibulk->flags to xfs_iwalk() function. 

V3 -> V4:
1. Rebase patchset on xfs-linux/for-next branch.
2. Carve out a 64-bit inode field out of the existing di_pad and
   di_flushiter fields to hold the 64-bit data fork extent counter.
3. Use the existing 32-bit inode data fork extent counter to hold the
   attr fork extent counter.
4. Verify the contents of newly introduced inode fields immediately
   after the inode has been read from the disk.
5. Upgrade inodes to be able to hold large extent counters when
   reading them from disk.
6. Use XFS_BULK_IREQ_NREXT64 as the flag that userspace can use to
   indicate that it can read 64-bit data fork extent counter.
7. Bulkstat ioctl returns -EOVERFLOW when userspace is not capable of
   working with large extent counters and inode's data fork extent
   count is larger than INT32_MAX.

V2 -> V3:
1. Define maximum extent length as a function of
   BMBT_BLOCKCOUNT_BITLEN.
2. Introduce xfs_iext_max_nextents() function in the patch series
   before renaming MAXEXTNUM/MAXAEXTNUM. This is done to reduce
   proliferation of macros indicating maximum extent count for data
   and attribute forks.
3. Define xfs_dfork_nextents() as an inline function.
4. Use xfs_rfsblock_t as the data type for variables that hold block
   count.
5. xfs_dfork_nextents() now returns -EFSCORRUPTED when an invalid fork
   is passed as an argument.
6. The following changes are done to enable bulkstat ioctl to report
   64-bit extent counters,
   - Carve out a new 64-bit field xfs_bulkstat->bs_extents64 from
     xfs_bulkstat->bs_pad[].
   - Carve out a new 64-bit field xfs_bulk_ireq->bulkstat_flags from
     xfs_bulk_ireq->reserved[] to hold bulkstat specific operational
     flags. Introduce XFS_IBULK_NREXT64 flag to indicate that
     userspace has the necessary infrastructure to receive 64-bit
     extent counters.
   - Define the new flag XFS_BULK_IREQ_BULKSTAT for userspace to
     indicate that xfs_bulk_ireq->bulkstat_flags has valid flags set.
7. Rename the incompat flag from XFS_SB_FEAT_INCOMPAT_EXTCOUNT_64BIT
   to XFS_SB_FEAT_INCOMPAT_NREXT64.
8. Add a new helper function xfs_inode_to_disk_iext_counters() to
   convert from incore inode extent counters to ondisk inode extent
   counters.
9. Reuse XFS_ERRTAG_REDUCE_MAX_IEXTENTS error tag to skip reporting
   inodes with more than 10 extents when bulkstat ioctl is invoked by
   userspace.
10. Introduce the new per-inode XFS_DIFLAG2_NREXT64 flag to indicate
    that the inode uses 64-bit extent counter. This is used to allow
    administrators to upgrade existing filesystems.
11. Export presence of XFS_SB_FEAT_INCOMPAT_NREXT64 feature to
    userspace via XFS_IOC_FSGEOMETRY ioctl.

V1 -> V2:
1. Rebase patches on top of Darrick's btree-dynamic-depth branch.
2. Add new bulkstat ioctl version to support 64-bit data fork extent
   counter field.
3. Introduce new error tag to verify if the old bulkstat ioctls skip
   reporting inodes with large data fork extent counters.

Chandan Babu R (19):
  xfs: Move extent count limits to xfs_format.h
  xfs: Define max extent length based on on-disk format definition
  xfs: Introduce xfs_iext_max_nextents() helper
  xfs: Use xfs_extnum_t instead of basic data types
  xfs: Introduce xfs_dfork_nextents() helper
  xfs: Use basic types to define xfs_log_dinode's di_nextents and
    di_anextents
  xfs: Promote xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits
    respectively
  xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs
    feature bit
  xfs: Introduce XFS_FSOP_GEOM_FLAGS_NREXT64
  xfs: Introduce XFS_DIFLAG2_NREXT64 and associated helpers
  xfs: Use uint64_t to count maximum blocks that can be used by BMBT
  xfs: Introduce macros to represent new maximum extent counts for
    data/attr forks
  xfs: Replace numbered inode recovery error messages with descriptive
    ones
  xfs: Introduce per-inode 64-bit extent counters
  xfs: Directory's data fork extent counter can never overflow
  xfs: Conditionally upgrade existing inodes to use large extent
    counters
  xfs: Decouple XFS_IBULK flags from XFS_IWALK flags
  xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters
  xfs: Add XFS_SB_FEAT_INCOMPAT_NREXT64 to the list of supported flags

 fs/xfs/libxfs/xfs_alloc.c       |   2 +-
 fs/xfs/libxfs/xfs_attr.c        |   9 ++-
 fs/xfs/libxfs/xfs_bmap.c        | 112 +++++++++++++-------------
 fs/xfs/libxfs/xfs_bmap_btree.c  |   3 +-
 fs/xfs/libxfs/xfs_format.h      |  80 +++++++++++++++---
 fs/xfs/libxfs/xfs_fs.h          |  21 ++++-
 fs/xfs/libxfs/xfs_ialloc.c      |   2 +
 fs/xfs/libxfs/xfs_inode_buf.c   |  80 ++++++++++++++----
 fs/xfs/libxfs/xfs_inode_fork.c  |  42 ++++++++--
 fs/xfs/libxfs/xfs_inode_fork.h  |  63 ++++++++++++++-
 fs/xfs/libxfs/xfs_log_format.h  |  33 +++++++-
 fs/xfs/libxfs/xfs_sb.c          |   5 ++
 fs/xfs/libxfs/xfs_trans_resv.c  |  11 +--
 fs/xfs/libxfs/xfs_types.h       |  11 +--
 fs/xfs/scrub/bmap.c             |   2 +-
 fs/xfs/scrub/inode.c            |  20 ++---
 fs/xfs/xfs_bmap_item.c          |   8 +-
 fs/xfs/xfs_bmap_util.c          |  57 ++++++++++---
 fs/xfs/xfs_dquot.c              |   9 ++-
 fs/xfs/xfs_inode.c              |  36 ++++++++-
 fs/xfs/xfs_inode.h              |   5 ++
 fs/xfs/xfs_inode_item.c         |  23 +++++-
 fs/xfs/xfs_inode_item_recover.c | 138 +++++++++++++++++++++++---------
 fs/xfs/xfs_ioctl.c              |   3 +
 fs/xfs/xfs_iomap.c              |  45 +++++++----
 fs/xfs/xfs_itable.c             |  19 ++++-
 fs/xfs/xfs_itable.h             |   4 +-
 fs/xfs/xfs_iwalk.h              |   2 +-
 fs/xfs/xfs_mount.h              |   2 +
 fs/xfs/xfs_reflink.c            |  17 +++-
 fs/xfs/xfs_rtalloc.c            |   9 ++-
 fs/xfs/xfs_symlink.c            |   8 ++
 fs/xfs/xfs_trace.h              |   4 +-
 33 files changed, 679 insertions(+), 206 deletions(-)

-- 
2.30.2

[PATCH V8 01/19] xfs: Move extent count limits to xfs_format.h

[Chandan Babu R]

Maximum values associated with extent counters i.e. Maximum extent length,
Maximum data extents and Maximum xattr extents are dictated by the on-disk
format. Hence move these definitions over to xfs_format.h.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_format.h | 7 +++++++
 fs/xfs/libxfs/xfs_types.h  | 7 -------
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index d665c04e69dd..d75e5b16da7e 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -869,6 +869,13 @@ enum xfs_dinode_fmt {
 	{ XFS_DINODE_FMT_BTREE,		"btree" }, \
 	{ XFS_DINODE_FMT_UUID,		"uuid" }
 
+/*
+ * Max values for extlen, extnum, aextnum.
+ */
+#define	MAXEXTLEN	((xfs_extlen_t)0x001fffff)	/* 21 bits */
+#define	MAXEXTNUM	((xfs_extnum_t)0x7fffffff)	/* signed int */
+#define	MAXAEXTNUM	((xfs_aextnum_t)0x7fff)		/* signed short */
+
 /*
  * Inode minimum and maximum sizes.
  */
diff --git a/fs/xfs/libxfs/xfs_types.h b/fs/xfs/libxfs/xfs_types.h
index b6da06b40989..794a54cbd0de 100644
--- a/fs/xfs/libxfs/xfs_types.h
+++ b/fs/xfs/libxfs/xfs_types.h
@@ -56,13 +56,6 @@ typedef void *		xfs_failaddr_t;
 #define	NULLFSINO	((xfs_ino_t)-1)
 #define	NULLAGINO	((xfs_agino_t)-1)
 
-/*
- * Max values for extlen, extnum, aextnum.
- */
-#define	MAXEXTLEN	((xfs_extlen_t)0x001fffff)	/* 21 bits */
-#define	MAXEXTNUM	((xfs_extnum_t)0x7fffffff)	/* signed int */
-#define	MAXAEXTNUM	((xfs_aextnum_t)0x7fff)		/* signed short */
-
 /*
  * Minimum and maximum blocksize and sectorsize.
  * The blocksize upper limit is pretty much arbitrary.
-- 
2.30.2

[PATCH V8 02/19] xfs: Define max extent length based on on-disk format definition

[Chandan Babu R]

The maximum extent length depends on maximum block count that can be stored in
a BMBT record. Hence this commit defines MAXEXTLEN based on
BMBT_BLOCKCOUNT_BITLEN.

While at it, the commit also renames MAXEXTLEN to XFS_MAX_BMBT_EXTLEN.

Suggested-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_alloc.c      |  2 +-
 fs/xfs/libxfs/xfs_bmap.c       | 57 +++++++++++++++++-----------------
 fs/xfs/libxfs/xfs_format.h     |  5 +--
 fs/xfs/libxfs/xfs_inode_buf.c  |  4 +--
 fs/xfs/libxfs/xfs_trans_resv.c | 11 ++++---
 fs/xfs/scrub/bmap.c            |  2 +-
 fs/xfs/xfs_bmap_util.c         | 14 +++++----
 fs/xfs/xfs_iomap.c             | 28 ++++++++---------
 8 files changed, 64 insertions(+), 59 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index 353e53b892e6..3f9b9cbfef43 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -2493,7 +2493,7 @@ __xfs_free_extent_later(
 
 	ASSERT(bno != NULLFSBLOCK);
 	ASSERT(len > 0);
-	ASSERT(len <= MAXEXTLEN);
+	ASSERT(len <= XFS_MAX_BMBT_EXTLEN);
 	ASSERT(!isnullstartblock(bno));
 	agno = XFS_FSB_TO_AGNO(mp, bno);
 	agbno = XFS_FSB_TO_AGBNO(mp, bno);
diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index 74198dd82b03..00b8e6e1c404 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -1452,7 +1452,7 @@ xfs_bmap_add_extent_delay_real(
 	    LEFT.br_startoff + LEFT.br_blockcount == new->br_startoff &&
 	    LEFT.br_startblock + LEFT.br_blockcount == new->br_startblock &&
 	    LEFT.br_state == new->br_state &&
-	    LEFT.br_blockcount + new->br_blockcount <= MAXEXTLEN)
+	    LEFT.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN)
 		state |= BMAP_LEFT_CONTIG;
 
 	/*
@@ -1470,13 +1470,13 @@ xfs_bmap_add_extent_delay_real(
 	    new_endoff == RIGHT.br_startoff &&
 	    new->br_startblock + new->br_blockcount == RIGHT.br_startblock &&
 	    new->br_state == RIGHT.br_state &&
-	    new->br_blockcount + RIGHT.br_blockcount <= MAXEXTLEN &&
+	    new->br_blockcount + RIGHT.br_blockcount <= XFS_MAX_BMBT_EXTLEN &&
 	    ((state & (BMAP_LEFT_CONTIG | BMAP_LEFT_FILLING |
 		       BMAP_RIGHT_FILLING)) !=
 		      (BMAP_LEFT_CONTIG | BMAP_LEFT_FILLING |
 		       BMAP_RIGHT_FILLING) ||
 	     LEFT.br_blockcount + new->br_blockcount + RIGHT.br_blockcount
-			<= MAXEXTLEN))
+			<= XFS_MAX_BMBT_EXTLEN))
 		state |= BMAP_RIGHT_CONTIG;
 
 	error = 0;
@@ -2000,7 +2000,7 @@ xfs_bmap_add_extent_unwritten_real(
 	    LEFT.br_startoff + LEFT.br_blockcount == new->br_startoff &&
 	    LEFT.br_startblock + LEFT.br_blockcount == new->br_startblock &&
 	    LEFT.br_state == new->br_state &&
-	    LEFT.br_blockcount + new->br_blockcount <= MAXEXTLEN)
+	    LEFT.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN)
 		state |= BMAP_LEFT_CONTIG;
 
 	/*
@@ -2018,13 +2018,13 @@ xfs_bmap_add_extent_unwritten_real(
 	    new_endoff == RIGHT.br_startoff &&
 	    new->br_startblock + new->br_blockcount == RIGHT.br_startblock &&
 	    new->br_state == RIGHT.br_state &&
-	    new->br_blockcount + RIGHT.br_blockcount <= MAXEXTLEN &&
+	    new->br_blockcount + RIGHT.br_blockcount <= XFS_MAX_BMBT_EXTLEN &&
 	    ((state & (BMAP_LEFT_CONTIG | BMAP_LEFT_FILLING |
 		       BMAP_RIGHT_FILLING)) !=
 		      (BMAP_LEFT_CONTIG | BMAP_LEFT_FILLING |
 		       BMAP_RIGHT_FILLING) ||
 	     LEFT.br_blockcount + new->br_blockcount + RIGHT.br_blockcount
-			<= MAXEXTLEN))
+			<= XFS_MAX_BMBT_EXTLEN))
 		state |= BMAP_RIGHT_CONTIG;
 
 	/*
@@ -2510,15 +2510,15 @@ xfs_bmap_add_extent_hole_delay(
 	 */
 	if ((state & BMAP_LEFT_VALID) && (state & BMAP_LEFT_DELAY) &&
 	    left.br_startoff + left.br_blockcount == new->br_startoff &&
-	    left.br_blockcount + new->br_blockcount <= MAXEXTLEN)
+	    left.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN)
 		state |= BMAP_LEFT_CONTIG;
 
 	if ((state & BMAP_RIGHT_VALID) && (state & BMAP_RIGHT_DELAY) &&
 	    new->br_startoff + new->br_blockcount == right.br_startoff &&
-	    new->br_blockcount + right.br_blockcount <= MAXEXTLEN &&
+	    new->br_blockcount + right.br_blockcount <= XFS_MAX_BMBT_EXTLEN &&
 	    (!(state & BMAP_LEFT_CONTIG) ||
 	     (left.br_blockcount + new->br_blockcount +
-	      right.br_blockcount <= MAXEXTLEN)))
+	      right.br_blockcount <= XFS_MAX_BMBT_EXTLEN)))
 		state |= BMAP_RIGHT_CONTIG;
 
 	/*
@@ -2661,17 +2661,17 @@ xfs_bmap_add_extent_hole_real(
 	    left.br_startoff + left.br_blockcount == new->br_startoff &&
 	    left.br_startblock + left.br_blockcount == new->br_startblock &&
 	    left.br_state == new->br_state &&
-	    left.br_blockcount + new->br_blockcount <= MAXEXTLEN)
+	    left.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN)
 		state |= BMAP_LEFT_CONTIG;
 
 	if ((state & BMAP_RIGHT_VALID) && !(state & BMAP_RIGHT_DELAY) &&
 	    new->br_startoff + new->br_blockcount == right.br_startoff &&
 	    new->br_startblock + new->br_blockcount == right.br_startblock &&
 	    new->br_state == right.br_state &&
-	    new->br_blockcount + right.br_blockcount <= MAXEXTLEN &&
+	    new->br_blockcount + right.br_blockcount <= XFS_MAX_BMBT_EXTLEN &&
 	    (!(state & BMAP_LEFT_CONTIG) ||
 	     left.br_blockcount + new->br_blockcount +
-	     right.br_blockcount <= MAXEXTLEN))
+	     right.br_blockcount <= XFS_MAX_BMBT_EXTLEN))
 		state |= BMAP_RIGHT_CONTIG;
 
 	error = 0;
@@ -2906,15 +2906,15 @@ xfs_bmap_extsize_align(
 
 	/*
 	 * For large extent hint sizes, the aligned extent might be larger than
-	 * MAXEXTLEN. In that case, reduce the size by an extsz so that it pulls
-	 * the length back under MAXEXTLEN. The outer allocation loops handle
-	 * short allocation just fine, so it is safe to do this. We only want to
-	 * do it when we are forced to, though, because it means more allocation
-	 * operations are required.
+	 * XFS_BMBT_MAX_EXTLEN. In that case, reduce the size by an extsz so
+	 * that it pulls the length back under XFS_BMBT_MAX_EXTLEN. The outer
+	 * allocation loops handle short allocation just fine, so it is safe to
+	 * do this. We only want to do it when we are forced to, though, because
+	 * it means more allocation operations are required.
 	 */
-	while (align_alen > MAXEXTLEN)
+	while (align_alen > XFS_MAX_BMBT_EXTLEN)
 		align_alen -= extsz;
-	ASSERT(align_alen <= MAXEXTLEN);
+	ASSERT(align_alen <= XFS_MAX_BMBT_EXTLEN);
 
 	/*
 	 * If the previous block overlaps with this proposed allocation
@@ -3004,9 +3004,9 @@ xfs_bmap_extsize_align(
 			return -EINVAL;
 	} else {
 		ASSERT(orig_off >= align_off);
-		/* see MAXEXTLEN handling above */
+		/* see XFS_BMBT_MAX_EXTLEN handling above */
 		ASSERT(orig_end <= align_off + align_alen ||
-		       align_alen + extsz > MAXEXTLEN);
+		       align_alen + extsz > XFS_MAX_BMBT_EXTLEN);
 	}
 
 #ifdef DEBUG
@@ -3971,7 +3971,7 @@ xfs_bmapi_reserve_delalloc(
 	 * Cap the alloc length. Keep track of prealloc so we know whether to
 	 * tag the inode before we return.
 	 */
-	alen = XFS_FILBLKS_MIN(len + prealloc, MAXEXTLEN);
+	alen = XFS_FILBLKS_MIN(len + prealloc, XFS_MAX_BMBT_EXTLEN);
 	if (!eof)
 		alen = XFS_FILBLKS_MIN(alen, got->br_startoff - aoff);
 	if (prealloc && alen >= len)
@@ -4104,7 +4104,7 @@ xfs_bmapi_allocate(
 		if (!xfs_iext_peek_prev_extent(ifp, &bma->icur, &bma->prev))
 			bma->prev.br_startoff = NULLFILEOFF;
 	} else {
-		bma->length = XFS_FILBLKS_MIN(bma->length, MAXEXTLEN);
+		bma->length = XFS_FILBLKS_MIN(bma->length, XFS_MAX_BMBT_EXTLEN);
 		if (!bma->eof)
 			bma->length = XFS_FILBLKS_MIN(bma->length,
 					bma->got.br_startoff - bma->offset);
@@ -4424,8 +4424,8 @@ xfs_bmapi_write(
 			 * xfs_extlen_t and therefore 32 bits. Hence we have to
 			 * check for 32-bit overflows and handle them here.
 			 */
-			if (len > (xfs_filblks_t)MAXEXTLEN)
-				bma.length = MAXEXTLEN;
+			if (len > (xfs_filblks_t)XFS_MAX_BMBT_EXTLEN)
+				bma.length = XFS_MAX_BMBT_EXTLEN;
 			else
 				bma.length = len;
 
@@ -4560,7 +4560,8 @@ xfs_bmapi_convert_delalloc(
 	bma.ip = ip;
 	bma.wasdel = true;
 	bma.offset = bma.got.br_startoff;
-	bma.length = max_t(xfs_filblks_t, bma.got.br_blockcount, MAXEXTLEN);
+	bma.length = max_t(xfs_filblks_t, bma.got.br_blockcount,
+			XFS_MAX_BMBT_EXTLEN);
 	bma.minleft = xfs_bmapi_minleft(tp, ip, whichfork);
 
 	/*
@@ -4641,7 +4642,7 @@ xfs_bmapi_remap(
 
 	ifp = XFS_IFORK_PTR(ip, whichfork);
 	ASSERT(len > 0);
-	ASSERT(len <= (xfs_filblks_t)MAXEXTLEN);
+	ASSERT(len <= (xfs_filblks_t)XFS_MAX_BMBT_EXTLEN);
 	ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
 	ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC |
 			   XFS_BMAPI_NORMAP)));
@@ -5641,7 +5642,7 @@ xfs_bmse_can_merge(
 	if ((left->br_startoff + left->br_blockcount != startoff) ||
 	    (left->br_startblock + left->br_blockcount != got->br_startblock) ||
 	    (left->br_state != got->br_state) ||
-	    (left->br_blockcount + got->br_blockcount > MAXEXTLEN))
+	    (left->br_blockcount + got->br_blockcount > XFS_MAX_BMBT_EXTLEN))
 		return false;
 
 	return true;
diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index d75e5b16da7e..66594853a88b 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -870,9 +870,8 @@ enum xfs_dinode_fmt {
 	{ XFS_DINODE_FMT_UUID,		"uuid" }
 
 /*
- * Max values for extlen, extnum, aextnum.
+ * Max values for extnum and aextnum.
  */
-#define	MAXEXTLEN	((xfs_extlen_t)0x001fffff)	/* 21 bits */
 #define	MAXEXTNUM	((xfs_extnum_t)0x7fffffff)	/* signed int */
 #define	MAXAEXTNUM	((xfs_aextnum_t)0x7fff)		/* signed short */
 
@@ -1603,6 +1602,8 @@ typedef struct xfs_bmdr_block {
 #define BMBT_STARTOFF_MASK	((1ULL << BMBT_STARTOFF_BITLEN) - 1)
 #define BMBT_BLOCKCOUNT_MASK	((1ULL << BMBT_BLOCKCOUNT_BITLEN) - 1)
 
+#define XFS_MAX_BMBT_EXTLEN	((xfs_extlen_t)(BMBT_BLOCKCOUNT_MASK))
+
 /*
  * bmbt records have a file offset (block) field that is 54 bits wide, so this
  * is the largest xfs_fileoff_t that we ever expect to see.
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index cae9708c8587..87781a5d5a45 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -639,7 +639,7 @@ xfs_inode_validate_extsize(
 	if (extsize_bytes % blocksize_bytes)
 		return __this_address;
 
-	if (extsize > MAXEXTLEN)
+	if (extsize > XFS_MAX_BMBT_EXTLEN)
 		return __this_address;
 
 	if (!rt_flag && extsize > mp->m_sb.sb_agblocks / 2)
@@ -696,7 +696,7 @@ xfs_inode_validate_cowextsize(
 	if (cowextsize_bytes % mp->m_sb.sb_blocksize)
 		return __this_address;
 
-	if (cowextsize > MAXEXTLEN)
+	if (cowextsize > XFS_MAX_BMBT_EXTLEN)
 		return __this_address;
 
 	if (cowextsize > mp->m_sb.sb_agblocks / 2)
diff --git a/fs/xfs/libxfs/xfs_trans_resv.c b/fs/xfs/libxfs/xfs_trans_resv.c
index 6f83d9b306ee..8e1d09e8cc9a 100644
--- a/fs/xfs/libxfs/xfs_trans_resv.c
+++ b/fs/xfs/libxfs/xfs_trans_resv.c
@@ -199,8 +199,8 @@ xfs_calc_inode_chunk_res(
 /*
  * Per-extent log reservation for the btree changes involved in freeing or
  * allocating a realtime extent.  We have to be able to log as many rtbitmap
- * blocks as needed to mark inuse MAXEXTLEN blocks' worth of realtime extents,
- * as well as the realtime summary block.
+ * blocks as needed to mark inuse XFS_BMBT_MAX_EXTLEN blocks' worth of realtime
+ * extents, as well as the realtime summary block.
  */
 static unsigned int
 xfs_rtalloc_log_count(
@@ -210,7 +210,7 @@ xfs_rtalloc_log_count(
 	unsigned int		blksz = XFS_FSB_TO_B(mp, 1);
 	unsigned int		rtbmp_bytes;
 
-	rtbmp_bytes = (MAXEXTLEN / mp->m_sb.sb_rextsize) / NBBY;
+	rtbmp_bytes = (XFS_MAX_BMBT_EXTLEN / mp->m_sb.sb_rextsize) / NBBY;
 	return (howmany(rtbmp_bytes, blksz) + 1) * num_ops;
 }
 
@@ -247,7 +247,7 @@ xfs_rtalloc_log_count(
  *    the inode's bmap btree: max depth * block size
  *    the agfs of the ags from which the extents are allocated: 2 * sector
  *    the superblock free block counter: sector size
- *    the realtime bitmap: ((MAXEXTLEN / rtextsize) / NBBY) bytes
+ *    the realtime bitmap: ((XFS_BMBT_MAX_EXTLEN / rtextsize) / NBBY) bytes
  *    the realtime summary: 1 block
  *    the allocation btrees: 2 trees * (2 * max depth - 1) * block size
  * And the bmap_finish transaction can free bmap blocks in a join (t3):
@@ -299,7 +299,8 @@ xfs_calc_write_reservation(
  *    the agf for each of the ags: 2 * sector size
  *    the agfl for each of the ags: 2 * sector size
  *    the super block to reflect the freed blocks: sector size
- *    the realtime bitmap: 2 exts * ((MAXEXTLEN / rtextsize) / NBBY) bytes
+ *    the realtime bitmap:
+ *		2 exts * ((XFS_BMBT_MAX_EXTLEN / rtextsize) / NBBY) bytes
  *    the realtime summary: 2 exts * 1 block
  *    worst case split in allocation btrees per extent assuming 2 extents:
  *		2 exts * 2 trees * (2 * max depth - 1) * block size
diff --git a/fs/xfs/scrub/bmap.c b/fs/xfs/scrub/bmap.c
index a4cbbc346f60..c357593e0a02 100644
--- a/fs/xfs/scrub/bmap.c
+++ b/fs/xfs/scrub/bmap.c
@@ -350,7 +350,7 @@ xchk_bmap_iextent(
 				irec->br_startoff);
 
 	/* Make sure the extent points to a valid place. */
-	if (irec->br_blockcount > MAXEXTLEN)
+	if (irec->br_blockcount > XFS_MAX_BMBT_EXTLEN)
 		xchk_fblock_set_corrupt(info->sc, info->whichfork,
 				irec->br_startoff);
 	if (info->is_rt &&
diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c
index eb2e387ba528..18c1b99311a8 100644
--- a/fs/xfs/xfs_bmap_util.c
+++ b/fs/xfs/xfs_bmap_util.c
@@ -119,14 +119,14 @@ xfs_bmap_rtalloc(
 	 */
 	ralen = ap->length / mp->m_sb.sb_rextsize;
 	/*
-	 * If the old value was close enough to MAXEXTLEN that
+	 * If the old value was close enough to XFS_BMBT_MAX_EXTLEN that
 	 * we rounded up to it, cut it back so it's valid again.
 	 * Note that if it's a really large request (bigger than
-	 * MAXEXTLEN), we don't hear about that number, and can't
+	 * XFS_BMBT_MAX_EXTLEN), we don't hear about that number, and can't
 	 * adjust the starting point to match it.
 	 */
-	if (ralen * mp->m_sb.sb_rextsize >= MAXEXTLEN)
-		ralen = MAXEXTLEN / mp->m_sb.sb_rextsize;
+	if (ralen * mp->m_sb.sb_rextsize >= XFS_MAX_BMBT_EXTLEN)
+		ralen = XFS_MAX_BMBT_EXTLEN / mp->m_sb.sb_rextsize;
 
 	/*
 	 * Lock out modifications to both the RT bitmap and summary inodes
@@ -839,9 +839,11 @@ xfs_alloc_file_space(
 		 * count, hence we need to limit the number of blocks we are
 		 * trying to reserve to avoid an overflow. We can't allocate
 		 * more than @nimaps extents, and an extent is limited on disk
-		 * to MAXEXTLEN (21 bits), so use that to enforce the limit.
+		 * to XFS_BMBT_MAX_EXTLEN (21 bits), so use that to enforce the
+		 * limit.
 		 */
-		resblks = min_t(xfs_fileoff_t, (e - s), (MAXEXTLEN * nimaps));
+		resblks = min_t(xfs_fileoff_t, (e - s),
+				(XFS_MAX_BMBT_EXTLEN * nimaps));
 		if (unlikely(rt)) {
 			dblocks = XFS_DIOSTRAT_SPACE_RES(mp, 0);
 			rblocks = resblks;
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index e552ce541ec2..87e1cf5060bd 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -402,7 +402,7 @@ xfs_iomap_prealloc_size(
 	 */
 	plen = prev.br_blockcount;
 	while (xfs_iext_prev_extent(ifp, &ncur, &got)) {
-		if (plen > MAXEXTLEN / 2 ||
+		if (plen > XFS_MAX_BMBT_EXTLEN / 2 ||
 		    isnullstartblock(got.br_startblock) ||
 		    got.br_startoff + got.br_blockcount != prev.br_startoff ||
 		    got.br_startblock + got.br_blockcount != prev.br_startblock)
@@ -414,23 +414,23 @@ xfs_iomap_prealloc_size(
 	/*
 	 * If the size of the extents is greater than half the maximum extent
 	 * length, then use the current offset as the basis.  This ensures that
-	 * for large files the preallocation size always extends to MAXEXTLEN
-	 * rather than falling short due to things like stripe unit/width
-	 * alignment of real extents.
+	 * for large files the preallocation size always extends to
+	 * XFS_BMBT_MAX_EXTLEN rather than falling short due to things like stripe
+	 * unit/width alignment of real extents.
 	 */
 	alloc_blocks = plen * 2;
-	if (alloc_blocks > MAXEXTLEN)
+	if (alloc_blocks > XFS_MAX_BMBT_EXTLEN)
 		alloc_blocks = XFS_B_TO_FSB(mp, offset);
 	qblocks = alloc_blocks;
 
 	/*
-	 * MAXEXTLEN is not a power of two value but we round the prealloc down
-	 * to the nearest power of two value after throttling. To prevent the
-	 * round down from unconditionally reducing the maximum supported
-	 * prealloc size, we round up first, apply appropriate throttling,
-	 * round down and cap the value to MAXEXTLEN.
+	 * XFS_BMBT_MAX_EXTLEN is not a power of two value but we round the prealloc
+	 * down to the nearest power of two value after throttling. To prevent
+	 * the round down from unconditionally reducing the maximum supported
+	 * prealloc size, we round up first, apply appropriate throttling, round
+	 * down and cap the value to XFS_BMBT_MAX_EXTLEN.
 	 */
-	alloc_blocks = XFS_FILEOFF_MIN(roundup_pow_of_two(MAXEXTLEN),
+	alloc_blocks = XFS_FILEOFF_MIN(roundup_pow_of_two(XFS_MAX_BMBT_EXTLEN),
 				       alloc_blocks);
 
 	freesp = percpu_counter_read_positive(&mp->m_fdblocks);
@@ -478,14 +478,14 @@ xfs_iomap_prealloc_size(
 	 */
 	if (alloc_blocks)
 		alloc_blocks = rounddown_pow_of_two(alloc_blocks);
-	if (alloc_blocks > MAXEXTLEN)
-		alloc_blocks = MAXEXTLEN;
+	if (alloc_blocks > XFS_MAX_BMBT_EXTLEN)
+		alloc_blocks = XFS_MAX_BMBT_EXTLEN;
 
 	/*
 	 * If we are still trying to allocate more space than is
 	 * available, squash the prealloc hard. This can happen if we
 	 * have a large file on a small filesystem and the above
-	 * lowspace thresholds are smaller than MAXEXTLEN.
+	 * lowspace thresholds are smaller than XFS_BMBT_MAX_EXTLEN.
 	 */
 	while (alloc_blocks && alloc_blocks >= freesp)
 		alloc_blocks >>= 4;
-- 
2.30.2

[PATCH V8 03/19] xfs: Introduce xfs_iext_max_nextents() helper

[Chandan Babu R]

xfs_iext_max_nextents() returns the maximum number of extents possible for one
of data, cow or attribute fork. This helper will be extended further in a
future commit when maximum extent counts associated with data/attribute forks
are increased.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_bmap.c       | 9 ++++-----
 fs/xfs/libxfs/xfs_inode_buf.c  | 8 +++-----
 fs/xfs/libxfs/xfs_inode_fork.c | 2 +-
 fs/xfs/libxfs/xfs_inode_fork.h | 8 ++++++++
 4 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index 00b8e6e1c404..a713bc7242a4 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -74,13 +74,12 @@ xfs_bmap_compute_maxlevels(
 	 * ATTR2 we have to assume the worst case scenario of a minimum size
 	 * available.
 	 */
-	if (whichfork == XFS_DATA_FORK) {
-		maxleafents = MAXEXTNUM;
+	maxleafents = xfs_iext_max_nextents(whichfork);
+	if (whichfork == XFS_DATA_FORK)
 		sz = XFS_BMDR_SPACE_CALC(MINDBTPTRS);
-	} else {
-		maxleafents = MAXAEXTNUM;
+	else
 		sz = XFS_BMDR_SPACE_CALC(MINABTPTRS);
-	}
+
 	maxrootrecs = xfs_bmdr_maxrecs(sz, 0);
 	minleafrecs = mp->m_bmap_dmnr[0];
 	minnoderecs = mp->m_bmap_dmnr[1];
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index 87781a5d5a45..b1c37a82ddce 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -337,6 +337,7 @@ xfs_dinode_verify_fork(
 	int			whichfork)
 {
 	uint32_t		di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork);
+	xfs_extnum_t		max_extents;
 
 	switch (XFS_DFORK_FORMAT(dip, whichfork)) {
 	case XFS_DINODE_FMT_LOCAL:
@@ -358,12 +359,9 @@ xfs_dinode_verify_fork(
 			return __this_address;
 		break;
 	case XFS_DINODE_FMT_BTREE:
-		if (whichfork == XFS_ATTR_FORK) {
-			if (di_nextents > MAXAEXTNUM)
-				return __this_address;
-		} else if (di_nextents > MAXEXTNUM) {
+		max_extents = xfs_iext_max_nextents(whichfork);
+		if (di_nextents > max_extents)
 			return __this_address;
-		}
 		break;
 	default:
 		return __this_address;
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index 9149f4f796fc..e136c29a0ec1 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -744,7 +744,7 @@ xfs_iext_count_may_overflow(
 	if (whichfork == XFS_COW_FORK)
 		return 0;
 
-	max_exts = (whichfork == XFS_ATTR_FORK) ? MAXAEXTNUM : MAXEXTNUM;
+	max_exts = xfs_iext_max_nextents(whichfork);
 
 	if (XFS_TEST_ERROR(false, ip->i_mount, XFS_ERRTAG_REDUCE_MAX_IEXTENTS))
 		max_exts = 10;
diff --git a/fs/xfs/libxfs/xfs_inode_fork.h b/fs/xfs/libxfs/xfs_inode_fork.h
index 3d64a3acb0ed..2605f7ff8fc1 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.h
+++ b/fs/xfs/libxfs/xfs_inode_fork.h
@@ -133,6 +133,14 @@ static inline int8_t xfs_ifork_format(struct xfs_ifork *ifp)
 	return ifp->if_format;
 }
 
+static inline xfs_extnum_t xfs_iext_max_nextents(int whichfork)
+{
+	if (whichfork == XFS_DATA_FORK || whichfork == XFS_COW_FORK)
+		return MAXEXTNUM;
+
+	return MAXAEXTNUM;
+}
+
 struct xfs_ifork *xfs_ifork_alloc(enum xfs_dinode_fmt format,
 				xfs_extnum_t nextents);
 struct xfs_ifork *xfs_iext_state_to_fork(struct xfs_inode *ip, int state);
-- 
2.30.2

[PATCH V8 04/19] xfs: Use xfs_extnum_t instead of basic data types

[Chandan Babu R]

xfs_extnum_t is the type to use to declare variables which have values
obtained from xfs_dinode->di_[a]nextents. This commit replaces basic
types (e.g. uint32_t) with xfs_extnum_t for such variables.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_bmap.c       | 2 +-
 fs/xfs/libxfs/xfs_inode_buf.c  | 2 +-
 fs/xfs/libxfs/xfs_inode_fork.c | 2 +-
 fs/xfs/scrub/inode.c           | 2 +-
 fs/xfs/xfs_trace.h             | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index a713bc7242a4..cc15981b1793 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -54,7 +54,7 @@ xfs_bmap_compute_maxlevels(
 {
 	int		level;		/* btree level */
 	uint		maxblocks;	/* max blocks at this level */
-	uint		maxleafents;	/* max leaf entries possible */
+	xfs_extnum_t	maxleafents;	/* max leaf entries possible */
 	int		maxrootrecs;	/* max records in root block */
 	int		minleafrecs;	/* min records in leaf block */
 	int		minnoderecs;	/* min records in node block */
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index b1c37a82ddce..7cad307840b3 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -336,7 +336,7 @@ xfs_dinode_verify_fork(
 	struct xfs_mount	*mp,
 	int			whichfork)
 {
-	uint32_t		di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork);
+	xfs_extnum_t		di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork);
 	xfs_extnum_t		max_extents;
 
 	switch (XFS_DFORK_FORMAT(dip, whichfork)) {
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index e136c29a0ec1..a17c4d87520a 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -105,7 +105,7 @@ xfs_iformat_extents(
 	struct xfs_mount	*mp = ip->i_mount;
 	struct xfs_ifork	*ifp = XFS_IFORK_PTR(ip, whichfork);
 	int			state = xfs_bmap_fork_to_state(whichfork);
-	int			nex = XFS_DFORK_NEXTENTS(dip, whichfork);
+	xfs_extnum_t		nex = XFS_DFORK_NEXTENTS(dip, whichfork);
 	int			size = nex * sizeof(xfs_bmbt_rec_t);
 	struct xfs_iext_cursor	icur;
 	struct xfs_bmbt_rec	*dp;
diff --git a/fs/xfs/scrub/inode.c b/fs/xfs/scrub/inode.c
index eac15af7b08c..87925761e174 100644
--- a/fs/xfs/scrub/inode.c
+++ b/fs/xfs/scrub/inode.c
@@ -232,7 +232,7 @@ xchk_dinode(
 	size_t			fork_recs;
 	unsigned long long	isize;
 	uint64_t		flags2;
-	uint32_t		nextents;
+	xfs_extnum_t		nextents;
 	prid_t			prid;
 	uint16_t		flags;
 	uint16_t		mode;
diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h
index 4a8076ef8cb4..3153db29de40 100644
--- a/fs/xfs/xfs_trace.h
+++ b/fs/xfs/xfs_trace.h
@@ -2169,7 +2169,7 @@ DECLARE_EVENT_CLASS(xfs_swap_extent_class,
 		__field(int, which)
 		__field(xfs_ino_t, ino)
 		__field(int, format)
-		__field(int, nex)
+		__field(xfs_extnum_t, nex)
 		__field(int, broot_size)
 		__field(int, fork_off)
 	),
-- 
2.30.2

[PATCH V8 05/19] xfs: Introduce xfs_dfork_nextents() helper

[Chandan Babu R]

This commit replaces the macro XFS_DFORK_NEXTENTS() with the helper function
xfs_dfork_nextents(). As of this commit, xfs_dfork_nextents() returns the same
value as XFS_DFORK_NEXTENTS(). A future commit which extends inode's extent
counter fields will add more logic to this helper.

This commit also replaces direct accesses to xfs_dinode->di_[a]nextents
with calls to xfs_dfork_nextents().

No functional changes have been made.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_format.h     |  4 ----
 fs/xfs/libxfs/xfs_inode_buf.c  | 17 ++++++++++++-----
 fs/xfs/libxfs/xfs_inode_fork.c |  8 ++++----
 fs/xfs/libxfs/xfs_inode_fork.h | 32 ++++++++++++++++++++++++++++++++
 fs/xfs/scrub/inode.c           | 18 ++++++++++--------
 5 files changed, 58 insertions(+), 21 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index 66594853a88b..b5e9256d6d32 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -924,10 +924,6 @@ enum xfs_dinode_fmt {
 	((w) == XFS_DATA_FORK ? \
 		(dip)->di_format : \
 		(dip)->di_aformat)
-#define XFS_DFORK_NEXTENTS(dip,w) \
-	((w) == XFS_DATA_FORK ? \
-		be32_to_cpu((dip)->di_nextents) : \
-		be16_to_cpu((dip)->di_anextents))
 
 /*
  * For block and character special files the 32bit dev_t is stored at the
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index 7cad307840b3..f0e063835318 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -336,9 +336,11 @@ xfs_dinode_verify_fork(
 	struct xfs_mount	*mp,
 	int			whichfork)
 {
-	xfs_extnum_t		di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork);
+	xfs_extnum_t		di_nextents;
 	xfs_extnum_t		max_extents;
 
+	di_nextents = xfs_dfork_nextents(dip, whichfork);
+
 	switch (XFS_DFORK_FORMAT(dip, whichfork)) {
 	case XFS_DINODE_FMT_LOCAL:
 		/*
@@ -405,6 +407,9 @@ xfs_dinode_verify(
 	uint16_t		flags;
 	uint64_t		flags2;
 	uint64_t		di_size;
+	xfs_extnum_t		nextents;
+	xfs_extnum_t		naextents;
+	xfs_filblks_t		nblocks;
 
 	if (dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC))
 		return __this_address;
@@ -435,10 +440,12 @@ xfs_dinode_verify(
 	if ((S_ISLNK(mode) || S_ISDIR(mode)) && di_size == 0)
 		return __this_address;
 
+	nextents = xfs_dfork_data_extents(dip);
+	naextents = xfs_dfork_attr_extents(dip);
+	nblocks = be64_to_cpu(dip->di_nblocks);
+
 	/* Fork checks carried over from xfs_iformat_fork */
-	if (mode &&
-	    be32_to_cpu(dip->di_nextents) + be16_to_cpu(dip->di_anextents) >
-			be64_to_cpu(dip->di_nblocks))
+	if (mode && nextents + naextents > nblocks)
 		return __this_address;
 
 	if (mode && XFS_DFORK_BOFF(dip) > mp->m_sb.sb_inodesize)
@@ -495,7 +502,7 @@ xfs_dinode_verify(
 		default:
 			return __this_address;
 		}
-		if (dip->di_anextents)
+		if (naextents)
 			return __this_address;
 	}
 
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index a17c4d87520a..1cf48cee45e3 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -105,7 +105,7 @@ xfs_iformat_extents(
 	struct xfs_mount	*mp = ip->i_mount;
 	struct xfs_ifork	*ifp = XFS_IFORK_PTR(ip, whichfork);
 	int			state = xfs_bmap_fork_to_state(whichfork);
-	xfs_extnum_t		nex = XFS_DFORK_NEXTENTS(dip, whichfork);
+	xfs_extnum_t		nex = xfs_dfork_nextents(dip, whichfork);
 	int			size = nex * sizeof(xfs_bmbt_rec_t);
 	struct xfs_iext_cursor	icur;
 	struct xfs_bmbt_rec	*dp;
@@ -230,7 +230,7 @@ xfs_iformat_data_fork(
 	 * depend on it.
 	 */
 	ip->i_df.if_format = dip->di_format;
-	ip->i_df.if_nextents = be32_to_cpu(dip->di_nextents);
+	ip->i_df.if_nextents = xfs_dfork_data_extents(dip);
 
 	switch (inode->i_mode & S_IFMT) {
 	case S_IFIFO:
@@ -295,14 +295,14 @@ xfs_iformat_attr_fork(
 	struct xfs_inode	*ip,
 	struct xfs_dinode	*dip)
 {
+	xfs_extnum_t		naextents = xfs_dfork_attr_extents(dip);
 	int			error = 0;
 
 	/*
 	 * Initialize the extent count early, as the per-format routines may
 	 * depend on it.
 	 */
-	ip->i_afp = xfs_ifork_alloc(dip->di_aformat,
-				be16_to_cpu(dip->di_anextents));
+	ip->i_afp = xfs_ifork_alloc(dip->di_aformat, naextents);
 
 	switch (ip->i_afp->if_format) {
 	case XFS_DINODE_FMT_LOCAL:
diff --git a/fs/xfs/libxfs/xfs_inode_fork.h b/fs/xfs/libxfs/xfs_inode_fork.h
index 2605f7ff8fc1..7ed2ecb51bca 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.h
+++ b/fs/xfs/libxfs/xfs_inode_fork.h
@@ -141,6 +141,38 @@ static inline xfs_extnum_t xfs_iext_max_nextents(int whichfork)
 	return MAXAEXTNUM;
 }
 
+static inline xfs_extnum_t
+xfs_dfork_data_extents(
+	struct xfs_dinode	*dip)
+{
+	return be32_to_cpu(dip->di_nextents);
+}
+
+static inline xfs_extnum_t
+xfs_dfork_attr_extents(
+	struct xfs_dinode	*dip)
+{
+	return be16_to_cpu(dip->di_anextents);
+}
+
+static inline xfs_extnum_t
+xfs_dfork_nextents(
+	struct xfs_dinode	*dip,
+	int			whichfork)
+{
+	switch (whichfork) {
+	case XFS_DATA_FORK:
+		return xfs_dfork_data_extents(dip);
+	case XFS_ATTR_FORK:
+		return xfs_dfork_attr_extents(dip);
+	default:
+		ASSERT(0);
+		break;
+	}
+
+	return 0;
+}
+
 struct xfs_ifork *xfs_ifork_alloc(enum xfs_dinode_fmt format,
 				xfs_extnum_t nextents);
 struct xfs_ifork *xfs_iext_state_to_fork(struct xfs_inode *ip, int state);
diff --git a/fs/xfs/scrub/inode.c b/fs/xfs/scrub/inode.c
index 87925761e174..51820b40ab1c 100644
--- a/fs/xfs/scrub/inode.c
+++ b/fs/xfs/scrub/inode.c
@@ -233,6 +233,7 @@ xchk_dinode(
 	unsigned long long	isize;
 	uint64_t		flags2;
 	xfs_extnum_t		nextents;
+	xfs_extnum_t		naextents;
 	prid_t			prid;
 	uint16_t		flags;
 	uint16_t		mode;
@@ -390,8 +391,10 @@ xchk_dinode(
 
 	xchk_inode_extsize(sc, dip, ino, mode, flags);
 
+	nextents = xfs_dfork_data_extents(dip);
+	naextents = xfs_dfork_attr_extents(dip);
+
 	/* di_nextents */
-	nextents = be32_to_cpu(dip->di_nextents);
 	fork_recs =  XFS_DFORK_DSIZE(dip, mp) / sizeof(struct xfs_bmbt_rec);
 	switch (dip->di_format) {
 	case XFS_DINODE_FMT_EXTENTS:
@@ -411,7 +414,7 @@ xchk_dinode(
 	/* di_forkoff */
 	if (XFS_DFORK_APTR(dip) >= (char *)dip + mp->m_sb.sb_inodesize)
 		xchk_ino_set_corrupt(sc, ino);
-	if (dip->di_anextents != 0 && dip->di_forkoff == 0)
+	if (naextents != 0 && dip->di_forkoff == 0)
 		xchk_ino_set_corrupt(sc, ino);
 	if (dip->di_forkoff == 0 && dip->di_aformat != XFS_DINODE_FMT_EXTENTS)
 		xchk_ino_set_corrupt(sc, ino);
@@ -423,19 +426,18 @@ xchk_dinode(
 		xchk_ino_set_corrupt(sc, ino);
 
 	/* di_anextents */
-	nextents = be16_to_cpu(dip->di_anextents);
 	fork_recs =  XFS_DFORK_ASIZE(dip, mp) / sizeof(struct xfs_bmbt_rec);
 	switch (dip->di_aformat) {
 	case XFS_DINODE_FMT_EXTENTS:
-		if (nextents > fork_recs)
+		if (naextents > fork_recs)
 			xchk_ino_set_corrupt(sc, ino);
 		break;
 	case XFS_DINODE_FMT_BTREE:
-		if (nextents <= fork_recs)
+		if (naextents <= fork_recs)
 			xchk_ino_set_corrupt(sc, ino);
 		break;
 	default:
-		if (nextents != 0)
+		if (naextents != 0)
 			xchk_ino_set_corrupt(sc, ino);
 	}
 
@@ -513,14 +515,14 @@ xchk_inode_xref_bmap(
 			&nextents, &count);
 	if (!xchk_should_check_xref(sc, &error, NULL))
 		return;
-	if (nextents < be32_to_cpu(dip->di_nextents))
+	if (nextents < xfs_dfork_data_extents(dip))
 		xchk_ino_xref_set_corrupt(sc, sc->ip->i_ino);
 
 	error = xfs_bmap_count_blocks(sc->tp, sc->ip, XFS_ATTR_FORK,
 			&nextents, &acount);
 	if (!xchk_should_check_xref(sc, &error, NULL))
 		return;
-	if (nextents != be16_to_cpu(dip->di_anextents))
+	if (nextents != xfs_dfork_attr_extents(dip))
 		xchk_ino_xref_set_corrupt(sc, sc->ip->i_ino);
 
 	/* Check nblocks against the inode. */
-- 
2.30.2

[PATCH V8 06/19] xfs: Use basic types to define xfs_log_dinode's di_nextents and di_anextents

[Chandan Babu R]

A future commit will increase the width of xfs_extnum_t in order to facilitate
larger per-inode extent counters. Hence this patch now uses basic types to
define xfs_log_dinode->[di_nextents|dianextents].

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_log_format.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_log_format.h b/fs/xfs/libxfs/xfs_log_format.h
index b322db523d65..fd66e70248f7 100644
--- a/fs/xfs/libxfs/xfs_log_format.h
+++ b/fs/xfs/libxfs/xfs_log_format.h
@@ -396,8 +396,8 @@ struct xfs_log_dinode {
 	xfs_fsize_t	di_size;	/* number of bytes in file */
 	xfs_rfsblock_t	di_nblocks;	/* # of direct & btree blocks used */
 	xfs_extlen_t	di_extsize;	/* basic/minimum extent size for file */
-	xfs_extnum_t	di_nextents;	/* number of extents in data fork */
-	xfs_aextnum_t	di_anextents;	/* number of extents in attribute fork*/
+	uint32_t	di_nextents;	/* number of extents in data fork */
+	uint16_t	di_anextents;	/* number of extents in attribute fork*/
 	uint8_t		di_forkoff;	/* attr fork offs, <<3 for 64b align */
 	int8_t		di_aformat;	/* format of attr fork's data */
 	uint32_t	di_dmevmask;	/* DMIG event mask */
-- 
2.30.2

[PATCH V8 07/19] xfs: Promote xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits respectively

[Chandan Babu R]

A future commit will introduce a 64-bit on-disk data extent counter and a
32-bit on-disk attr extent counter. This commit promotes xfs_extnum_t and
xfs_aextnum_t to 64 and 32-bits in order to correctly handle in-core versions
of these quantities.

Reported-by: kernel test robot <lkp@intel.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_bmap.c       | 4 ++--
 fs/xfs/libxfs/xfs_inode_fork.c | 4 ++--
 fs/xfs/libxfs/xfs_inode_fork.h | 2 +-
 fs/xfs/libxfs/xfs_types.h      | 4 ++--
 fs/xfs/xfs_inode.c             | 4 ++--
 fs/xfs/xfs_trace.h             | 2 +-
 6 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index cc15981b1793..9f38e33d6ce2 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -83,7 +83,7 @@ xfs_bmap_compute_maxlevels(
 	maxrootrecs = xfs_bmdr_maxrecs(sz, 0);
 	minleafrecs = mp->m_bmap_dmnr[0];
 	minnoderecs = mp->m_bmap_dmnr[1];
-	maxblocks = (maxleafents + minleafrecs - 1) / minleafrecs;
+	maxblocks = howmany_64(maxleafents, minleafrecs);
 	for (level = 1; maxblocks > 1; level++) {
 		if (maxblocks <= maxrootrecs)
 			maxblocks = 1;
@@ -467,7 +467,7 @@ xfs_bmap_check_leaf_extents(
 	if (bp_release)
 		xfs_trans_brelse(NULL, bp);
 error_norelse:
-	xfs_warn(mp, "%s: BAD after btree leaves for %d extents",
+	xfs_warn(mp, "%s: BAD after btree leaves for %llu extents",
 		__func__, i);
 	xfs_err(mp, "%s: CORRUPTED BTREE OR SOMETHING", __func__);
 	xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE);
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index 1cf48cee45e3..004b205d87b8 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -117,8 +117,8 @@ xfs_iformat_extents(
 	 * we just bail out rather than crash in kmem_alloc() or memcpy() below.
 	 */
 	if (unlikely(size < 0 || size > XFS_DFORK_SIZE(dip, mp, whichfork))) {
-		xfs_warn(ip->i_mount, "corrupt inode %Lu ((a)extents = %d).",
-			(unsigned long long) ip->i_ino, nex);
+		xfs_warn(ip->i_mount, "corrupt inode %llu ((a)extents = %llu).",
+			ip->i_ino, nex);
 		xfs_inode_verifier_error(ip, -EFSCORRUPTED,
 				"xfs_iformat_extents(1)", dip, sizeof(*dip),
 				__this_address);
diff --git a/fs/xfs/libxfs/xfs_inode_fork.h b/fs/xfs/libxfs/xfs_inode_fork.h
index 7ed2ecb51bca..4a8b77d425df 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.h
+++ b/fs/xfs/libxfs/xfs_inode_fork.h
@@ -21,9 +21,9 @@ struct xfs_ifork {
 		void		*if_root;	/* extent tree root */
 		char		*if_data;	/* inline file data */
 	} if_u1;
+	xfs_extnum_t		if_nextents;	/* # of extents in this fork */
 	short			if_broot_bytes;	/* bytes allocated for root */
 	int8_t			if_format;	/* format of this fork */
-	xfs_extnum_t		if_nextents;	/* # of extents in this fork */
 };
 
 /*
diff --git a/fs/xfs/libxfs/xfs_types.h b/fs/xfs/libxfs/xfs_types.h
index 794a54cbd0de..373f64a492a4 100644
--- a/fs/xfs/libxfs/xfs_types.h
+++ b/fs/xfs/libxfs/xfs_types.h
@@ -12,8 +12,8 @@ typedef uint32_t	xfs_agblock_t;	/* blockno in alloc. group */
 typedef uint32_t	xfs_agino_t;	/* inode # within allocation grp */
 typedef uint32_t	xfs_extlen_t;	/* extent length in blocks */
 typedef uint32_t	xfs_agnumber_t;	/* allocation group number */
-typedef int32_t		xfs_extnum_t;	/* # of extents in a file */
-typedef int16_t		xfs_aextnum_t;	/* # extents in an attribute fork */
+typedef uint64_t	xfs_extnum_t;	/* # of extents in a file */
+typedef uint32_t	xfs_aextnum_t;	/* # extents in an attribute fork */
 typedef int64_t		xfs_fsize_t;	/* bytes in a file */
 typedef uint64_t	xfs_ufsize_t;	/* unsigned bytes in a file */
 
diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index 04bf467b1090..6810c4feaa45 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -3495,8 +3495,8 @@ xfs_iflush(
 	if (XFS_TEST_ERROR(ip->i_df.if_nextents + xfs_ifork_nextents(ip->i_afp) >
 				ip->i_nblocks, mp, XFS_ERRTAG_IFLUSH_5)) {
 		xfs_alert_tag(mp, XFS_PTAG_IFLUSH,
-			"%s: detected corrupt incore inode %Lu, "
-			"total extents = %d, nblocks = %Ld, ptr "PTR_FMT,
+			"%s: detected corrupt incore inode %llu, "
+			"total extents = %llu nblocks = %lld, ptr "PTR_FMT,
 			__func__, ip->i_ino,
 			ip->i_df.if_nextents + xfs_ifork_nextents(ip->i_afp),
 			ip->i_nblocks, ip);
diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h
index 3153db29de40..6b4a7f197308 100644
--- a/fs/xfs/xfs_trace.h
+++ b/fs/xfs/xfs_trace.h
@@ -2182,7 +2182,7 @@ DECLARE_EVENT_CLASS(xfs_swap_extent_class,
 		__entry->broot_size = ip->i_df.if_broot_bytes;
 		__entry->fork_off = XFS_IFORK_BOFF(ip);
 	),
-	TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %d, "
+	TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %llu, "
 		  "broot size %d, forkoff 0x%x",
 		  MAJOR(__entry->dev), MINOR(__entry->dev),
 		  __entry->ino,
-- 
2.30.2

[PATCH V8 08/19] xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs feature bit

[Chandan Babu R]

XFS_SB_FEAT_INCOMPAT_NREXT64 incompat feature bit will be set on filesystems
which support large per-inode extent counters. This commit defines the new
incompat feature bit and the corresponding per-fs feature bit (along with
inline functions to work on it).

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_format.h | 1 +
 fs/xfs/libxfs/xfs_sb.c     | 3 +++
 fs/xfs/xfs_mount.h         | 2 ++
 3 files changed, 6 insertions(+)

diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index b5e9256d6d32..64ff0c310696 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -372,6 +372,7 @@ xfs_sb_has_ro_compat_feature(
 #define XFS_SB_FEAT_INCOMPAT_META_UUID	(1 << 2)	/* metadata UUID */
 #define XFS_SB_FEAT_INCOMPAT_BIGTIME	(1 << 3)	/* large timestamps */
 #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)	/* needs xfs_repair */
+#define XFS_SB_FEAT_INCOMPAT_NREXT64	(1 << 5)	/* large extent counters */
 #define XFS_SB_FEAT_INCOMPAT_ALL \
 		(XFS_SB_FEAT_INCOMPAT_FTYPE|	\
 		 XFS_SB_FEAT_INCOMPAT_SPINODES|	\
diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c
index f4e84aa1d50a..bd632389ae92 100644
--- a/fs/xfs/libxfs/xfs_sb.c
+++ b/fs/xfs/libxfs/xfs_sb.c
@@ -124,6 +124,9 @@ xfs_sb_version_to_features(
 		features |= XFS_FEAT_BIGTIME;
 	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR)
 		features |= XFS_FEAT_NEEDSREPAIR;
+	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NREXT64)
+		features |= XFS_FEAT_NREXT64;
+
 	return features;
 }
 
diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h
index 00720a02e761..a008e8f983d8 100644
--- a/fs/xfs/xfs_mount.h
+++ b/fs/xfs/xfs_mount.h
@@ -276,6 +276,7 @@ typedef struct xfs_mount {
 #define XFS_FEAT_INOBTCNT	(1ULL << 23)	/* inobt block counts */
 #define XFS_FEAT_BIGTIME	(1ULL << 24)	/* large timestamps */
 #define XFS_FEAT_NEEDSREPAIR	(1ULL << 25)	/* needs xfs_repair */
+#define XFS_FEAT_NREXT64	(1ULL << 26)	/* large extent counters */
 
 /* Mount features */
 #define XFS_FEAT_NOATTR2	(1ULL << 48)	/* disable attr2 creation */
@@ -338,6 +339,7 @@ __XFS_HAS_FEAT(realtime, REALTIME)
 __XFS_HAS_FEAT(inobtcounts, INOBTCNT)
 __XFS_HAS_FEAT(bigtime, BIGTIME)
 __XFS_HAS_FEAT(needsrepair, NEEDSREPAIR)
+__XFS_HAS_FEAT(large_extent_counts, NREXT64)
 
 /*
  * Mount features
-- 
2.30.2

[PATCH V8 09/19] xfs: Introduce XFS_FSOP_GEOM_FLAGS_NREXT64

[Chandan Babu R]

XFS_FSOP_GEOM_FLAGS_NREXT64 indicates that the current filesystem instance
supports 64-bit per-inode extent counters.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_fs.h | 1 +
 fs/xfs/libxfs/xfs_sb.c | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/fs/xfs/libxfs/xfs_fs.h b/fs/xfs/libxfs/xfs_fs.h
index 505533c43a92..1f7238db35cc 100644
--- a/fs/xfs/libxfs/xfs_fs.h
+++ b/fs/xfs/libxfs/xfs_fs.h
@@ -236,6 +236,7 @@ typedef struct xfs_fsop_resblks {
 #define XFS_FSOP_GEOM_FLAGS_REFLINK	(1 << 20) /* files can share blocks */
 #define XFS_FSOP_GEOM_FLAGS_BIGTIME	(1 << 21) /* 64-bit nsec timestamps */
 #define XFS_FSOP_GEOM_FLAGS_INOBTCNT	(1 << 22) /* inobt btree counter */
+#define XFS_FSOP_GEOM_FLAGS_NREXT64	(1 << 23) /* large extent counters */
 
 /*
  * Minimum and maximum sizes need for growth checks.
diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c
index bd632389ae92..e292a1914a5b 100644
--- a/fs/xfs/libxfs/xfs_sb.c
+++ b/fs/xfs/libxfs/xfs_sb.c
@@ -1138,6 +1138,8 @@ xfs_fs_geometry(
 	} else {
 		geo->logsectsize = BBSIZE;
 	}
+	if (xfs_has_large_extent_counts(mp))
+		geo->flags |= XFS_FSOP_GEOM_FLAGS_NREXT64;
 	geo->rtsectsize = sbp->sb_blocksize;
 	geo->dirblocksize = xfs_dir2_dirblock_bytes(sbp);
 
-- 
2.30.2

[PATCH V8 10/19] xfs: Introduce XFS_DIFLAG2_NREXT64 and associated helpers

[Chandan Babu R]

This commit adds the new per-inode flag XFS_DIFLAG2_NREXT64 to indicate that
an inode supports 64-bit extent counters. This flag is also enabled by default
on newly created inodes when the corresponding filesystem has large extent
counter feature bit (i.e. XFS_FEAT_NREXT64) set.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_format.h      | 11 ++++++++++-
 fs/xfs/libxfs/xfs_ialloc.c      |  2 ++
 fs/xfs/xfs_inode.h              |  5 +++++
 fs/xfs/xfs_inode_item_recover.c |  7 +++++++
 4 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index 64ff0c310696..57b24744a7c2 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -991,15 +991,17 @@ static inline void xfs_dinode_put_rdev(struct xfs_dinode *dip, xfs_dev_t rdev)
 #define XFS_DIFLAG2_REFLINK_BIT	1	/* file's blocks may be shared */
 #define XFS_DIFLAG2_COWEXTSIZE_BIT   2  /* copy on write extent size hint */
 #define XFS_DIFLAG2_BIGTIME_BIT	3	/* big timestamps */
+#define XFS_DIFLAG2_NREXT64_BIT 4	/* large extent counters */
 
 #define XFS_DIFLAG2_DAX		(1 << XFS_DIFLAG2_DAX_BIT)
 #define XFS_DIFLAG2_REFLINK     (1 << XFS_DIFLAG2_REFLINK_BIT)
 #define XFS_DIFLAG2_COWEXTSIZE  (1 << XFS_DIFLAG2_COWEXTSIZE_BIT)
 #define XFS_DIFLAG2_BIGTIME	(1 << XFS_DIFLAG2_BIGTIME_BIT)
+#define XFS_DIFLAG2_NREXT64	(1 << XFS_DIFLAG2_NREXT64_BIT)
 
 #define XFS_DIFLAG2_ANY \
 	(XFS_DIFLAG2_DAX | XFS_DIFLAG2_REFLINK | XFS_DIFLAG2_COWEXTSIZE | \
-	 XFS_DIFLAG2_BIGTIME)
+	 XFS_DIFLAG2_BIGTIME | XFS_DIFLAG2_NREXT64)
 
 static inline bool xfs_dinode_has_bigtime(const struct xfs_dinode *dip)
 {
@@ -1007,6 +1009,13 @@ static inline bool xfs_dinode_has_bigtime(const struct xfs_dinode *dip)
 	       (dip->di_flags2 & cpu_to_be64(XFS_DIFLAG2_BIGTIME));
 }
 
+static inline bool xfs_dinode_has_large_extent_counts(
+	const struct xfs_dinode *dip)
+{
+	return dip->di_version >= 3 &&
+	       (dip->di_flags2 & cpu_to_be64(XFS_DIFLAG2_NREXT64));
+}
+
 /*
  * Inode number format:
  * low inopblog bits - offset in block
diff --git a/fs/xfs/libxfs/xfs_ialloc.c b/fs/xfs/libxfs/xfs_ialloc.c
index b418fe0c0679..cdf8b63fcb22 100644
--- a/fs/xfs/libxfs/xfs_ialloc.c
+++ b/fs/xfs/libxfs/xfs_ialloc.c
@@ -2772,6 +2772,8 @@ xfs_ialloc_setup_geometry(
 	igeo->new_diflags2 = 0;
 	if (xfs_has_bigtime(mp))
 		igeo->new_diflags2 |= XFS_DIFLAG2_BIGTIME;
+	if (xfs_has_large_extent_counts(mp))
+		igeo->new_diflags2 |= XFS_DIFLAG2_NREXT64;
 
 	/* Compute inode btree geometry. */
 	igeo->agino_log = sbp->sb_inopblog + sbp->sb_agblklog;
diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h
index b7e8f14d9fca..3d28daba88c9 100644
--- a/fs/xfs/xfs_inode.h
+++ b/fs/xfs/xfs_inode.h
@@ -218,6 +218,11 @@ static inline bool xfs_inode_has_bigtime(struct xfs_inode *ip)
 	return ip->i_diflags2 & XFS_DIFLAG2_BIGTIME;
 }
 
+static inline bool xfs_inode_has_large_extent_counts(struct xfs_inode *ip)
+{
+	return ip->i_diflags2 & XFS_DIFLAG2_NREXT64;
+}
+
 /*
  * Return the buftarg used for data allocations on a given inode.
  */
diff --git a/fs/xfs/xfs_inode_item_recover.c b/fs/xfs/xfs_inode_item_recover.c
index 239dd2e3384e..44b90614859e 100644
--- a/fs/xfs/xfs_inode_item_recover.c
+++ b/fs/xfs/xfs_inode_item_recover.c
@@ -142,6 +142,13 @@ xfs_log_dinode_to_disk_ts(
 	return ts;
 }
 
+static inline bool xfs_log_dinode_has_large_extent_counts(
+		const struct xfs_log_dinode *ld)
+{
+	return ld->di_version >= 3 &&
+	       (ld->di_flags2 & XFS_DIFLAG2_NREXT64);
+}
+
 STATIC void
 xfs_log_dinode_to_disk(
 	struct xfs_log_dinode	*from,
-- 
2.30.2

[PATCH V8 11/19] xfs: Use uint64_t to count maximum blocks that can be used by BMBT

[Chandan Babu R]

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_bmap.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index 9f38e33d6ce2..b317226fb4ba 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -52,9 +52,9 @@ xfs_bmap_compute_maxlevels(
 	xfs_mount_t	*mp,		/* file system mount structure */
 	int		whichfork)	/* data or attr fork */
 {
-	int		level;		/* btree level */
-	uint		maxblocks;	/* max blocks at this level */
+	uint64_t	maxblocks;	/* max blocks at this level */
 	xfs_extnum_t	maxleafents;	/* max leaf entries possible */
+	int		level;		/* btree level */
 	int		maxrootrecs;	/* max records in root block */
 	int		minleafrecs;	/* min records in leaf block */
 	int		minnoderecs;	/* min records in node block */
@@ -88,7 +88,7 @@ xfs_bmap_compute_maxlevels(
 		if (maxblocks <= maxrootrecs)
 			maxblocks = 1;
 		else
-			maxblocks = (maxblocks + minnoderecs - 1) / minnoderecs;
+			maxblocks = howmany_64(maxblocks, minnoderecs);
 	}
 	mp->m_bm_maxlevels[whichfork] = level;
 	ASSERT(mp->m_bm_maxlevels[whichfork] <= xfs_bmbt_maxlevels_ondisk());
-- 
2.30.2

[PATCH V8 12/19] xfs: Introduce macros to represent new maximum extent counts for data/attr forks

[Chandan Babu R]

This commit defines new macros to represent maximum extent counts allowed by
filesystems which have support for large per-inode extent counters.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_bmap.c       |  9 ++++-----
 fs/xfs/libxfs/xfs_bmap_btree.c |  3 ++-
 fs/xfs/libxfs/xfs_format.h     | 24 ++++++++++++++++++++++--
 fs/xfs/libxfs/xfs_inode_buf.c  |  4 +++-
 fs/xfs/libxfs/xfs_inode_fork.c |  3 ++-
 fs/xfs/libxfs/xfs_inode_fork.h | 21 +++++++++++++++++----
 6 files changed, 50 insertions(+), 14 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index b317226fb4ba..1254d4d4821e 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -61,10 +61,8 @@ xfs_bmap_compute_maxlevels(
 	int		sz;		/* root block size */
 
 	/*
-	 * The maximum number of extents in a file, hence the maximum number of
-	 * leaf entries, is controlled by the size of the on-disk extent count,
-	 * either a signed 32-bit number for the data fork, or a signed 16-bit
-	 * number for the attr fork.
+	 * The maximum number of extents in a fork, hence the maximum number of
+	 * leaf entries, is controlled by the size of the on-disk extent count.
 	 *
 	 * Note that we can no longer assume that if we are in ATTR1 that the
 	 * fork offset of all the inodes will be
@@ -74,7 +72,8 @@ xfs_bmap_compute_maxlevels(
 	 * ATTR2 we have to assume the worst case scenario of a minimum size
 	 * available.
 	 */
-	maxleafents = xfs_iext_max_nextents(whichfork);
+	maxleafents = xfs_iext_max_nextents(xfs_has_large_extent_counts(mp),
+				whichfork);
 	if (whichfork == XFS_DATA_FORK)
 		sz = XFS_BMDR_SPACE_CALC(MINDBTPTRS);
 	else
diff --git a/fs/xfs/libxfs/xfs_bmap_btree.c b/fs/xfs/libxfs/xfs_bmap_btree.c
index 453309fc85f2..7aabeccea9ab 100644
--- a/fs/xfs/libxfs/xfs_bmap_btree.c
+++ b/fs/xfs/libxfs/xfs_bmap_btree.c
@@ -611,7 +611,8 @@ xfs_bmbt_maxlevels_ondisk(void)
 	minrecs[1] = xfs_bmbt_block_maxrecs(blocklen, false) / 2;
 
 	/* One extra level for the inode root. */
-	return xfs_btree_compute_maxlevels(minrecs, MAXEXTNUM) + 1;
+	return xfs_btree_compute_maxlevels(minrecs,
+			XFS_MAX_EXTCNT_DATA_FORK_LARGE) + 1;
 }
 
 /*
diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index 57b24744a7c2..eb85bc9b229b 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -872,9 +872,29 @@ enum xfs_dinode_fmt {
 
 /*
  * Max values for extnum and aextnum.
+ *
+ * The original on-disk extent counts were held in signed fields, resulting in
+ * maximum extent counts of 2^31 and 2^15 for the data and attr forks
+ * respectively. Similarly the maximum extent length is limited to 2^21 blocks
+ * by the 21-bit wide blockcount field of a BMBT extent record.
+ *
+ * The newly introduced data fork extent counter can hold a 64-bit value,
+ * however the maximum number of extents in a file is also limited to 2^54
+ * extents by the 54-bit wide startoff field of a BMBT extent record.
+ *
+ * It is further limited by the maximum supported file size of 2^63
+ * *bytes*. This leads to a maximum extent count for maximally sized filesystem
+ * blocks (64kB) of:
+ *
+ * 2^63 bytes / 2^16 bytes per block = 2^47 blocks
+ *
+ * Rounding up 47 to the nearest multiple of bits-per-byte results in 48. Hence
+ * 2^48 was chosen as the maximum data fork extent count.
  */
-#define	MAXEXTNUM	((xfs_extnum_t)0x7fffffff)	/* signed int */
-#define	MAXAEXTNUM	((xfs_aextnum_t)0x7fff)		/* signed short */
+#define XFS_MAX_EXTCNT_DATA_FORK_LARGE	((xfs_extnum_t)((1ULL << 48) - 1))
+#define XFS_MAX_EXTCNT_ATTR_FORK_LARGE	((xfs_extnum_t)((1ULL << 32) - 1))
+#define XFS_MAX_EXTCNT_DATA_FORK_SMALL	((xfs_extnum_t)((1ULL << 31) - 1))
+#define XFS_MAX_EXTCNT_ATTR_FORK_SMALL	((xfs_extnum_t)((1ULL << 15) - 1))
 
 /*
  * Inode minimum and maximum sizes.
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index f0e063835318..e0d3140c3622 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -361,7 +361,9 @@ xfs_dinode_verify_fork(
 			return __this_address;
 		break;
 	case XFS_DINODE_FMT_BTREE:
-		max_extents = xfs_iext_max_nextents(whichfork);
+		max_extents = xfs_iext_max_nextents(
+					xfs_dinode_has_large_extent_counts(dip),
+					whichfork);
 		if (di_nextents > max_extents)
 			return __this_address;
 		break;
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index 004b205d87b8..bb5d841aac58 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -744,7 +744,8 @@ xfs_iext_count_may_overflow(
 	if (whichfork == XFS_COW_FORK)
 		return 0;
 
-	max_exts = xfs_iext_max_nextents(whichfork);
+	max_exts = xfs_iext_max_nextents(xfs_inode_has_large_extent_counts(ip),
+				whichfork);
 
 	if (XFS_TEST_ERROR(false, ip->i_mount, XFS_ERRTAG_REDUCE_MAX_IEXTENTS))
 		max_exts = 10;
diff --git a/fs/xfs/libxfs/xfs_inode_fork.h b/fs/xfs/libxfs/xfs_inode_fork.h
index 4a8b77d425df..967837a88860 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.h
+++ b/fs/xfs/libxfs/xfs_inode_fork.h
@@ -133,12 +133,25 @@ static inline int8_t xfs_ifork_format(struct xfs_ifork *ifp)
 	return ifp->if_format;
 }
 
-static inline xfs_extnum_t xfs_iext_max_nextents(int whichfork)
+static inline xfs_extnum_t xfs_iext_max_nextents(bool has_large_extent_counts,
+				int whichfork)
 {
-	if (whichfork == XFS_DATA_FORK || whichfork == XFS_COW_FORK)
-		return MAXEXTNUM;
+	switch (whichfork) {
+	case XFS_DATA_FORK:
+	case XFS_COW_FORK:
+		if (has_large_extent_counts)
+			return XFS_MAX_EXTCNT_DATA_FORK_LARGE;
+		return XFS_MAX_EXTCNT_DATA_FORK_SMALL;
+
+	case XFS_ATTR_FORK:
+		if (has_large_extent_counts)
+			return XFS_MAX_EXTCNT_ATTR_FORK_LARGE;
+		return XFS_MAX_EXTCNT_ATTR_FORK_SMALL;
 
-	return MAXAEXTNUM;
+	default:
+		ASSERT(0);
+		return 0;
+	}
 }
 
 static inline xfs_extnum_t
-- 
2.30.2

[PATCH V8 13/19] xfs: Replace numbered inode recovery error messages with descriptive ones

[Chandan Babu R]

This commit also prints inode fields with invalid values instead of printing
addresses of inode and buffer instances.

Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Suggested-by: Dave Chinner <dchinner@redhat.com>
---
 fs/xfs/xfs_inode_item_recover.c | 52 ++++++++++++++-------------------
 1 file changed, 22 insertions(+), 30 deletions(-)

diff --git a/fs/xfs/xfs_inode_item_recover.c b/fs/xfs/xfs_inode_item_recover.c
index 44b90614859e..96b222e18b0f 100644
--- a/fs/xfs/xfs_inode_item_recover.c
+++ b/fs/xfs/xfs_inode_item_recover.c
@@ -324,13 +324,12 @@ xlog_recover_inode_commit_pass2(
 	if (unlikely(S_ISREG(ldip->di_mode))) {
 		if ((ldip->di_format != XFS_DINODE_FMT_EXTENTS) &&
 		    (ldip->di_format != XFS_DINODE_FMT_BTREE)) {
-			XFS_CORRUPTION_ERROR("xlog_recover_inode_pass2(3)",
-					 XFS_ERRLEVEL_LOW, mp, ldip,
-					 sizeof(*ldip));
+			XFS_CORRUPTION_ERROR(
+				"Bad log dinode data fork format for regular file",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
 			xfs_alert(mp,
-		"%s: Bad regular inode log record, rec ptr "PTR_FMT", "
-		"ino ptr = "PTR_FMT", ino bp = "PTR_FMT", ino %Ld",
-				__func__, item, dip, bp, in_f->ilf_ino);
+				"Bad inode 0x%llx, data fork format 0x%x",
+				in_f->ilf_ino, ldip->di_format);
 			error = -EFSCORRUPTED;
 			goto out_release;
 		}
@@ -338,49 +337,42 @@ xlog_recover_inode_commit_pass2(
 		if ((ldip->di_format != XFS_DINODE_FMT_EXTENTS) &&
 		    (ldip->di_format != XFS_DINODE_FMT_BTREE) &&
 		    (ldip->di_format != XFS_DINODE_FMT_LOCAL)) {
-			XFS_CORRUPTION_ERROR("xlog_recover_inode_pass2(4)",
-					     XFS_ERRLEVEL_LOW, mp, ldip,
-					     sizeof(*ldip));
+			XFS_CORRUPTION_ERROR(
+				"Bad log dinode data fork format for directory",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
 			xfs_alert(mp,
-		"%s: Bad dir inode log record, rec ptr "PTR_FMT", "
-		"ino ptr = "PTR_FMT", ino bp = "PTR_FMT", ino %Ld",
-				__func__, item, dip, bp, in_f->ilf_ino);
+				"Bad inode 0x%llx, data fork format 0x%x",
+				in_f->ilf_ino, ldip->di_format);
 			error = -EFSCORRUPTED;
 			goto out_release;
 		}
 	}
 	if (unlikely(ldip->di_nextents + ldip->di_anextents > ldip->di_nblocks)){
-		XFS_CORRUPTION_ERROR("xlog_recover_inode_pass2(5)",
-				     XFS_ERRLEVEL_LOW, mp, ldip,
-				     sizeof(*ldip));
+		XFS_CORRUPTION_ERROR("Bad log dinode extent counts",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
 		xfs_alert(mp,
-	"%s: Bad inode log record, rec ptr "PTR_FMT", dino ptr "PTR_FMT", "
-	"dino bp "PTR_FMT", ino %Ld, total extents = %d, nblocks = %Ld",
-			__func__, item, dip, bp, in_f->ilf_ino,
-			ldip->di_nextents + ldip->di_anextents,
+			"Bad inode 0x%llx, nextents 0x%x, anextents 0x%x, nblocks 0x%llx",
+			in_f->ilf_ino, ldip->di_nextents, ldip->di_anextents,
 			ldip->di_nblocks);
 		error = -EFSCORRUPTED;
 		goto out_release;
 	}
 	if (unlikely(ldip->di_forkoff > mp->m_sb.sb_inodesize)) {
-		XFS_CORRUPTION_ERROR("xlog_recover_inode_pass2(6)",
-				     XFS_ERRLEVEL_LOW, mp, ldip,
-				     sizeof(*ldip));
+		XFS_CORRUPTION_ERROR("Bad log dinode fork offset",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
 		xfs_alert(mp,
-	"%s: Bad inode log record, rec ptr "PTR_FMT", dino ptr "PTR_FMT", "
-	"dino bp "PTR_FMT", ino %Ld, forkoff 0x%x", __func__,
-			item, dip, bp, in_f->ilf_ino, ldip->di_forkoff);
+			"Bad inode 0x%llx, di_forkoff 0x%x",
+			in_f->ilf_ino, ldip->di_forkoff);
 		error = -EFSCORRUPTED;
 		goto out_release;
 	}
 	isize = xfs_log_dinode_size(mp);
 	if (unlikely(item->ri_buf[1].i_len > isize)) {
-		XFS_CORRUPTION_ERROR("xlog_recover_inode_pass2(7)",
-				     XFS_ERRLEVEL_LOW, mp, ldip,
-				     sizeof(*ldip));
+		XFS_CORRUPTION_ERROR("Bad log dinode size", XFS_ERRLEVEL_LOW,
+				     mp, ldip, sizeof(*ldip));
 		xfs_alert(mp,
-			"%s: Bad inode log record length %d, rec ptr "PTR_FMT,
-			__func__, item->ri_buf[1].i_len, item);
+			"Bad inode 0x%llx log dinode size 0x%x",
+			in_f->ilf_ino, item->ri_buf[1].i_len);
 		error = -EFSCORRUPTED;
 		goto out_release;
 	}
-- 
2.30.2

[PATCH V8 14/19] xfs: Introduce per-inode 64-bit extent counters

[Chandan Babu R]

This commit introduces new fields in the on-disk inode format to support
64-bit data fork extent counters and 32-bit attribute fork extent
counters. The new fields will be used only when an inode has
XFS_DIFLAG2_NREXT64 flag set. Otherwise we continue to use the regular 32-bit
data fork extent counters and 16-bit attribute fork extent counters.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Suggested-by: Dave Chinner <dchinner@redhat.com>
---
 fs/xfs/libxfs/xfs_format.h      | 33 +++++++++++--
 fs/xfs/libxfs/xfs_inode_buf.c   | 49 +++++++++++++++++--
 fs/xfs/libxfs/xfs_inode_fork.h  |  6 +++
 fs/xfs/libxfs/xfs_log_format.h  | 33 +++++++++++--
 fs/xfs/xfs_inode_item.c         | 23 +++++++--
 fs/xfs/xfs_inode_item_recover.c | 87 ++++++++++++++++++++++++++++-----
 6 files changed, 202 insertions(+), 29 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index eb85bc9b229b..82b404c99b80 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -792,16 +792,41 @@ struct xfs_dinode {
 	__be32		di_nlink;	/* number of links to file */
 	__be16		di_projid_lo;	/* lower part of owner's project id */
 	__be16		di_projid_hi;	/* higher part owner's project id */
-	__u8		di_pad[6];	/* unused, zeroed space */
-	__be16		di_flushiter;	/* incremented on flush */
+	union {
+		/* Number of data fork extents if NREXT64 is set */
+		__be64	di_big_nextents;
+
+		/* Padding for V3 inodes without NREXT64 set. */
+		__be64	di_v3_pad;
+
+		/* Padding and inode flush counter for V2 inodes. */
+		struct {
+			__u8	di_v2_pad[6];
+			__be16	di_flushiter;
+		};
+	};
 	xfs_timestamp_t	di_atime;	/* time last accessed */
 	xfs_timestamp_t	di_mtime;	/* time last modified */
 	xfs_timestamp_t	di_ctime;	/* time created/inode modified */
 	__be64		di_size;	/* number of bytes in file */
 	__be64		di_nblocks;	/* # of direct & btree blocks used */
 	__be32		di_extsize;	/* basic/minimum extent size for file */
-	__be32		di_nextents;	/* number of extents in data fork */
-	__be16		di_anextents;	/* number of extents in attribute fork*/
+	union {
+		/*
+		 * For V2 inodes and V3 inodes without NREXT64 set, this
+		 * is the number of data and attr fork extents.
+		 */
+		struct {
+			__be32	di_nextents;
+			__be16	di_anextents;
+		} __packed;
+
+		/* Number of attr fork extents if NREXT64 is set. */
+		struct {
+			__be32	di_big_anextents;
+			__be16	di_nrext64_pad;
+		} __packed;
+	} __packed;
 	__u8		di_forkoff;	/* attr fork offs, <<3 for 64b align */
 	__s8		di_aformat;	/* format of attr fork's data */
 	__be32		di_dmevmask;	/* DMIG event mask */
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index e0d3140c3622..ee8d4eb7d048 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -279,6 +279,25 @@ xfs_inode_to_disk_ts(
 	return ts;
 }
 
+static inline void
+xfs_inode_to_disk_iext_counters(
+	struct xfs_inode	*ip,
+	struct xfs_dinode	*to)
+{
+	if (xfs_inode_has_large_extent_counts(ip)) {
+		to->di_big_nextents = cpu_to_be64(xfs_ifork_nextents(&ip->i_df));
+		to->di_big_anextents = cpu_to_be32(xfs_ifork_nextents(ip->i_afp));
+		/*
+		 * We might be upgrading the inode to use larger extent counters
+		 * than was previously used. Hence zero the unused field.
+		 */
+		to->di_nrext64_pad = cpu_to_be16(0);
+	} else {
+		to->di_nextents = cpu_to_be32(xfs_ifork_nextents(&ip->i_df));
+		to->di_anextents = cpu_to_be16(xfs_ifork_nextents(ip->i_afp));
+	}
+}
+
 void
 xfs_inode_to_disk(
 	struct xfs_inode	*ip,
@@ -296,7 +315,6 @@ xfs_inode_to_disk(
 	to->di_projid_lo = cpu_to_be16(ip->i_projid & 0xffff);
 	to->di_projid_hi = cpu_to_be16(ip->i_projid >> 16);
 
-	memset(to->di_pad, 0, sizeof(to->di_pad));
 	to->di_atime = xfs_inode_to_disk_ts(ip, inode->i_atime);
 	to->di_mtime = xfs_inode_to_disk_ts(ip, inode->i_mtime);
 	to->di_ctime = xfs_inode_to_disk_ts(ip, inode->i_ctime);
@@ -307,8 +325,6 @@ xfs_inode_to_disk(
 	to->di_size = cpu_to_be64(ip->i_disk_size);
 	to->di_nblocks = cpu_to_be64(ip->i_nblocks);
 	to->di_extsize = cpu_to_be32(ip->i_extsize);
-	to->di_nextents = cpu_to_be32(xfs_ifork_nextents(&ip->i_df));
-	to->di_anextents = cpu_to_be16(xfs_ifork_nextents(ip->i_afp));
 	to->di_forkoff = ip->i_forkoff;
 	to->di_aformat = xfs_ifork_format(ip->i_afp);
 	to->di_flags = cpu_to_be16(ip->i_diflags);
@@ -323,11 +339,14 @@ xfs_inode_to_disk(
 		to->di_lsn = cpu_to_be64(lsn);
 		memset(to->di_pad2, 0, sizeof(to->di_pad2));
 		uuid_copy(&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid);
-		to->di_flushiter = 0;
+		to->di_v3_pad = 0;
 	} else {
 		to->di_version = 2;
 		to->di_flushiter = cpu_to_be16(ip->i_flushiter);
+		memset(to->di_v2_pad, 0, sizeof(to->di_v2_pad));
 	}
+
+	xfs_inode_to_disk_iext_counters(ip, to);
 }
 
 static xfs_failaddr_t
@@ -398,6 +417,24 @@ xfs_dinode_verify_forkoff(
 	return NULL;
 }
 
+static xfs_failaddr_t
+xfs_dinode_verify_nrext64(
+	struct xfs_mount	*mp,
+	struct xfs_dinode	*dip)
+{
+	if (xfs_dinode_has_large_extent_counts(dip)) {
+		if (!xfs_has_large_extent_counts(mp))
+			return __this_address;
+		if (dip->di_nrext64_pad != 0)
+			return __this_address;
+	} else if (dip->di_version >= 3) {
+		if (dip->di_v3_pad != 0)
+			return __this_address;
+	}
+
+	return NULL;
+}
+
 xfs_failaddr_t
 xfs_dinode_verify(
 	struct xfs_mount	*mp,
@@ -442,6 +479,10 @@ xfs_dinode_verify(
 	if ((S_ISLNK(mode) || S_ISDIR(mode)) && di_size == 0)
 		return __this_address;
 
+	fa = xfs_dinode_verify_nrext64(mp, dip);
+	if (fa)
+		return fa;
+
 	nextents = xfs_dfork_data_extents(dip);
 	naextents = xfs_dfork_attr_extents(dip);
 	nblocks = be64_to_cpu(dip->di_nblocks);
diff --git a/fs/xfs/libxfs/xfs_inode_fork.h b/fs/xfs/libxfs/xfs_inode_fork.h
index 967837a88860..fd5c3c2d77e0 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.h
+++ b/fs/xfs/libxfs/xfs_inode_fork.h
@@ -158,6 +158,9 @@ static inline xfs_extnum_t
 xfs_dfork_data_extents(
 	struct xfs_dinode	*dip)
 {
+	if (xfs_dinode_has_large_extent_counts(dip))
+		return be64_to_cpu(dip->di_big_nextents);
+
 	return be32_to_cpu(dip->di_nextents);
 }
 
@@ -165,6 +168,9 @@ static inline xfs_extnum_t
 xfs_dfork_attr_extents(
 	struct xfs_dinode	*dip)
 {
+	if (xfs_dinode_has_large_extent_counts(dip))
+		return be32_to_cpu(dip->di_big_anextents);
+
 	return be16_to_cpu(dip->di_anextents);
 }
 
diff --git a/fs/xfs/libxfs/xfs_log_format.h b/fs/xfs/libxfs/xfs_log_format.h
index fd66e70248f7..12234a880e94 100644
--- a/fs/xfs/libxfs/xfs_log_format.h
+++ b/fs/xfs/libxfs/xfs_log_format.h
@@ -388,16 +388,41 @@ struct xfs_log_dinode {
 	uint32_t	di_nlink;	/* number of links to file */
 	uint16_t	di_projid_lo;	/* lower part of owner's project id */
 	uint16_t	di_projid_hi;	/* higher part of owner's project id */
-	uint8_t		di_pad[6];	/* unused, zeroed space */
-	uint16_t	di_flushiter;	/* incremented on flush */
+	union {
+		/* Number of data fork extents if NREXT64 is set */
+		uint64_t	di_big_nextents;
+
+		/* Padding for V3 inodes without NREXT64 set. */
+		uint64_t	di_v3_pad;
+
+		/* Padding and inode flush counter for V2 inodes. */
+		struct {
+			uint8_t	di_v2_pad[6];	/* V2 inode zeroed space */
+			uint16_t di_flushiter;	/* V2 inode incremented on flush */
+		};
+	};
 	xfs_log_timestamp_t di_atime;	/* time last accessed */
 	xfs_log_timestamp_t di_mtime;	/* time last modified */
 	xfs_log_timestamp_t di_ctime;	/* time created/inode modified */
 	xfs_fsize_t	di_size;	/* number of bytes in file */
 	xfs_rfsblock_t	di_nblocks;	/* # of direct & btree blocks used */
 	xfs_extlen_t	di_extsize;	/* basic/minimum extent size for file */
-	uint32_t	di_nextents;	/* number of extents in data fork */
-	uint16_t	di_anextents;	/* number of extents in attribute fork*/
+	union {
+		/*
+		 * For V2 inodes and V3 inodes without NREXT64 set, this
+		 * is the number of data and attr fork extents.
+		 */
+		struct {
+			uint32_t  di_nextents;
+			uint16_t  di_anextents;
+		} __packed;
+
+		/* Number of attr fork extents if NREXT64 is set. */
+		struct {
+			uint32_t  di_big_anextents;
+			uint16_t  di_nrext64_pad;
+		} __packed;
+	} __packed;
 	uint8_t		di_forkoff;	/* attr fork offs, <<3 for 64b align */
 	int8_t		di_aformat;	/* format of attr fork's data */
 	uint32_t	di_dmevmask;	/* DMIG event mask */
diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c
index 90d8e591baf8..5a513eb4a631 100644
--- a/fs/xfs/xfs_inode_item.c
+++ b/fs/xfs/xfs_inode_item.c
@@ -358,6 +358,21 @@ xfs_copy_dm_fields_to_log_dinode(
 	}
 }
 
+static inline void
+xfs_inode_to_log_dinode_iext_counters(
+	struct xfs_inode	*ip,
+	struct xfs_log_dinode	*to)
+{
+	if (xfs_inode_has_large_extent_counts(ip)) {
+		to->di_big_nextents = xfs_ifork_nextents(&ip->i_df);
+		to->di_big_anextents = xfs_ifork_nextents(ip->i_afp);
+		to->di_nrext64_pad = 0;
+	} else {
+		to->di_nextents = xfs_ifork_nextents(&ip->i_df);
+		to->di_anextents = xfs_ifork_nextents(ip->i_afp);
+	}
+}
+
 static void
 xfs_inode_to_log_dinode(
 	struct xfs_inode	*ip,
@@ -373,7 +388,6 @@ xfs_inode_to_log_dinode(
 	to->di_projid_lo = ip->i_projid & 0xffff;
 	to->di_projid_hi = ip->i_projid >> 16;
 
-	memset(to->di_pad, 0, sizeof(to->di_pad));
 	memset(to->di_pad3, 0, sizeof(to->di_pad3));
 	to->di_atime = xfs_inode_to_log_dinode_ts(ip, inode->i_atime);
 	to->di_mtime = xfs_inode_to_log_dinode_ts(ip, inode->i_mtime);
@@ -385,8 +399,6 @@ xfs_inode_to_log_dinode(
 	to->di_size = ip->i_disk_size;
 	to->di_nblocks = ip->i_nblocks;
 	to->di_extsize = ip->i_extsize;
-	to->di_nextents = xfs_ifork_nextents(&ip->i_df);
-	to->di_anextents = xfs_ifork_nextents(ip->i_afp);
 	to->di_forkoff = ip->i_forkoff;
 	to->di_aformat = xfs_ifork_format(ip->i_afp);
 	to->di_flags = ip->i_diflags;
@@ -406,11 +418,14 @@ xfs_inode_to_log_dinode(
 		to->di_lsn = lsn;
 		memset(to->di_pad2, 0, sizeof(to->di_pad2));
 		uuid_copy(&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid);
-		to->di_flushiter = 0;
+		to->di_v3_pad = 0;
 	} else {
 		to->di_version = 2;
 		to->di_flushiter = ip->i_flushiter;
+		memset(to->di_v2_pad, 0, sizeof(to->di_v2_pad));
 	}
+
+	xfs_inode_to_log_dinode_iext_counters(ip, to);
 }
 
 /*
diff --git a/fs/xfs/xfs_inode_item_recover.c b/fs/xfs/xfs_inode_item_recover.c
index 96b222e18b0f..d6f536cd8d4b 100644
--- a/fs/xfs/xfs_inode_item_recover.c
+++ b/fs/xfs/xfs_inode_item_recover.c
@@ -149,6 +149,22 @@ static inline bool xfs_log_dinode_has_large_extent_counts(
 	       (ld->di_flags2 & XFS_DIFLAG2_NREXT64);
 }
 
+static inline void
+xfs_log_dinode_to_disk_iext_counters(
+	struct xfs_log_dinode	*from,
+	struct xfs_dinode	*to)
+{
+	if (xfs_log_dinode_has_large_extent_counts(from)) {
+		to->di_big_nextents = cpu_to_be64(from->di_big_nextents);
+		to->di_big_anextents = cpu_to_be32(from->di_big_anextents);
+		to->di_nrext64_pad = cpu_to_be16(from->di_nrext64_pad);
+	} else {
+		to->di_nextents = cpu_to_be32(from->di_nextents);
+		to->di_anextents = cpu_to_be16(from->di_anextents);
+	}
+
+}
+
 STATIC void
 xfs_log_dinode_to_disk(
 	struct xfs_log_dinode	*from,
@@ -165,7 +181,6 @@ xfs_log_dinode_to_disk(
 	to->di_nlink = cpu_to_be32(from->di_nlink);
 	to->di_projid_lo = cpu_to_be16(from->di_projid_lo);
 	to->di_projid_hi = cpu_to_be16(from->di_projid_hi);
-	memcpy(to->di_pad, from->di_pad, sizeof(to->di_pad));
 
 	to->di_atime = xfs_log_dinode_to_disk_ts(from, from->di_atime);
 	to->di_mtime = xfs_log_dinode_to_disk_ts(from, from->di_mtime);
@@ -174,8 +189,6 @@ xfs_log_dinode_to_disk(
 	to->di_size = cpu_to_be64(from->di_size);
 	to->di_nblocks = cpu_to_be64(from->di_nblocks);
 	to->di_extsize = cpu_to_be32(from->di_extsize);
-	to->di_nextents = cpu_to_be32(from->di_nextents);
-	to->di_anextents = cpu_to_be16(from->di_anextents);
 	to->di_forkoff = from->di_forkoff;
 	to->di_aformat = from->di_aformat;
 	to->di_dmevmask = cpu_to_be32(from->di_dmevmask);
@@ -193,10 +206,63 @@ xfs_log_dinode_to_disk(
 		to->di_lsn = cpu_to_be64(lsn);
 		memcpy(to->di_pad2, from->di_pad2, sizeof(to->di_pad2));
 		uuid_copy(&to->di_uuid, &from->di_uuid);
-		to->di_flushiter = 0;
+		to->di_v3_pad = from->di_v3_pad;
 	} else {
 		to->di_flushiter = cpu_to_be16(from->di_flushiter);
+		memcpy(to->di_v2_pad, from->di_v2_pad, sizeof(to->di_v2_pad));
+	}
+
+	xfs_log_dinode_to_disk_iext_counters(from, to);
+}
+
+STATIC int
+xlog_dinode_verify_extent_counts(
+	struct xfs_mount	*mp,
+	struct xfs_log_dinode	*ldip)
+{
+	xfs_extnum_t		nextents;
+	xfs_aextnum_t		anextents;
+
+	if (xfs_log_dinode_has_large_extent_counts(ldip)) {
+		if (!xfs_has_large_extent_counts(mp) ||
+		    (ldip->di_nrext64_pad != 0)) {
+			XFS_CORRUPTION_ERROR(
+				"Bad log dinode large extent count format",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
+			xfs_alert(mp,
+				"Bad inode 0x%llx, nrext64 %d, padding 0x%x",
+				ldip->di_ino, xfs_has_large_extent_counts(mp),
+				ldip->di_nrext64_pad);
+			return -EFSCORRUPTED;
+		}
+
+		nextents = ldip->di_big_nextents;
+		anextents = ldip->di_big_anextents;
+	} else {
+		if (ldip->di_version == 3 && ldip->di_v3_pad != 0) {
+			XFS_CORRUPTION_ERROR(
+				"Bad log dinode di_v3_pad",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
+			xfs_alert(mp,
+				"Bad inode 0x%llx, di_v3_pad %llu",
+				ldip->di_ino, ldip->di_v3_pad);
+			return -EFSCORRUPTED;
+		}
+
+		nextents = ldip->di_nextents;
+		anextents = ldip->di_anextents;
+	}
+
+	if (unlikely(nextents + anextents > ldip->di_nblocks)) {
+		XFS_CORRUPTION_ERROR("Bad log dinode extent counts",
+				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
+		xfs_alert(mp,
+			"Bad inode 0x%llx, nextents 0x%llx, anextents 0x%x, nblocks 0x%llx",
+			ldip->di_ino, nextents, anextents, ldip->di_nblocks);
+		return -EFSCORRUPTED;
 	}
+
+	return 0;
 }
 
 STATIC int
@@ -347,16 +413,11 @@ xlog_recover_inode_commit_pass2(
 			goto out_release;
 		}
 	}
-	if (unlikely(ldip->di_nextents + ldip->di_anextents > ldip->di_nblocks)){
-		XFS_CORRUPTION_ERROR("Bad log dinode extent counts",
-				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
-		xfs_alert(mp,
-			"Bad inode 0x%llx, nextents 0x%x, anextents 0x%x, nblocks 0x%llx",
-			in_f->ilf_ino, ldip->di_nextents, ldip->di_anextents,
-			ldip->di_nblocks);
-		error = -EFSCORRUPTED;
+
+	error = xlog_dinode_verify_extent_counts(mp, ldip);
+	if (error)
 		goto out_release;
-	}
+
 	if (unlikely(ldip->di_forkoff > mp->m_sb.sb_inodesize)) {
 		XFS_CORRUPTION_ERROR("Bad log dinode fork offset",
 				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
-- 
2.30.2

[PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Chandan Babu R]

The maximum file size that can be represented by the data fork extent counter
in the worst case occurs when all extents are 1 block in length and each block
is 1KB in size.

With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
1KB sized blocks, a file can reach upto,
(2^31) * 1KB = 2TB

This is much larger than the theoretical maximum size of a directory
i.e. 32GB * 3 = 96GB.

Since a directory's inode can never overflow its data fork extent counter,
this commit replaces checking the return value of
xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).

Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_bmap.c | 19 +++++++++----------
 fs/xfs/xfs_inode.c       | 32 ++++++++++++++++++++++++++++++++
 fs/xfs/xfs_symlink.c     |  8 ++++++++
 3 files changed, 49 insertions(+), 10 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index 1254d4d4821e..5a089674c666 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -5148,24 +5148,23 @@ xfs_bmap_del_extent_real(
 		 */
 
 		/*
-		 * For directories, -ENOSPC is returned since a directory entry
-		 * remove operation must not fail due to low extent count
-		 * availability. -ENOSPC will be handled by higher layers of XFS
+		 * Extent count overflow during directory entry remove/rename
+		 * operation ideally should result in -ENOSPC being returned.
+		 * This error will eventually be handled by higher layers of XFS
 		 * by letting the corresponding empty Data/Free blocks to linger
 		 * until a future remove operation. Dabtree blocks would be
 		 * swapped with the last block in the leaf space and then the
 		 * new last block will be unmapped.
 		 *
-		 * The above logic also applies to the source directory entry of
-		 * a rename operation.
+		 * However, a directory inode can never overflow its data fork
+		 * extent counter because of reasons provided in the definition
+		 * of xfs_create().
 		 */
 		error = xfs_iext_count_may_overflow(ip, whichfork, 1);
-		if (error) {
-			ASSERT(S_ISDIR(VFS_I(ip)->i_mode) &&
-				whichfork == XFS_DATA_FORK);
-			error = -ENOSPC;
+		ASSERT(XFS_TEST_ERROR(false, ip->i_mount,
+				XFS_ERRTAG_REDUCE_MAX_IEXTENTS) || error == 0);
+		if (error)
 			goto done;
-		}
 
 		old = got;
 
diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index 6810c4feaa45..6016013658ff 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -1023,8 +1023,25 @@ xfs_create(
 	xfs_ilock(dp, XFS_ILOCK_EXCL | XFS_ILOCK_PARENT);
 	unlock_dp_on_error = true;
 
+	/*
+	 * The maximum file size that can be represented by the data fork extent
+	 * counter in the worst case occurs when all extents are 1 block in
+	 * length and each block is 1KB in size.
+	 *
+	 * With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count
+	 * and with 1KB sized blocks, a file can reach upto,
+	 * 1KB * (2^31) = 2TB
+	 *
+	 * This is much larger than the theoretical maximum size of a directory
+	 * i.e. 32GB * 3 = 96GB.
+	 *
+	 * Hence, a directory inode can never overflow its data fork extent
+	 * counter.
+	 */
 	error = xfs_iext_count_may_overflow(dp, XFS_DATA_FORK,
 			XFS_IEXT_DIR_MANIP_CNT(mp));
+	ASSERT(XFS_TEST_ERROR(false, dp->i_mount,
+			XFS_ERRTAG_REDUCE_MAX_IEXTENTS) || error == 0);
 	if (error)
 		goto out_trans_cancel;
 
@@ -1249,8 +1266,15 @@ xfs_link(
 	xfs_trans_ijoin(tp, sip, XFS_ILOCK_EXCL);
 	xfs_trans_ijoin(tp, tdp, XFS_ILOCK_EXCL);
 
+	/*
+	 * Please refer to the comment in xfs_create() for an explaination of
+	 * why a directory inode cannot have its data fork extent counter to
+	 * overflow.
+	 */
 	error = xfs_iext_count_may_overflow(tdp, XFS_DATA_FORK,
 			XFS_IEXT_DIR_MANIP_CNT(mp));
+	ASSERT(XFS_TEST_ERROR(false, tdp->i_mount,
+			XFS_ERRTAG_REDUCE_MAX_IEXTENTS) || error == 0);
 	if (error)
 		goto error_return;
 
@@ -3232,9 +3256,17 @@ xfs_rename(
 			if (error)
 				goto out_trans_cancel;
 		} else {
+			/*
+			 * Please refer to the comment in xfs_create() for an
+			 * explaination of why a directory inode cannot have its
+			 * data fork extent counter to overflow.
+			 */
 			error = xfs_iext_count_may_overflow(target_dp,
 					XFS_DATA_FORK,
 					XFS_IEXT_DIR_MANIP_CNT(mp));
+			ASSERT(XFS_TEST_ERROR(false, target_dp->i_mount,
+					XFS_ERRTAG_REDUCE_MAX_IEXTENTS) ||
+			       error == 0);
 			if (error)
 				goto out_trans_cancel;
 		}
diff --git a/fs/xfs/xfs_symlink.c b/fs/xfs/xfs_symlink.c
index affbedf78160..0e5acb00c59f 100644
--- a/fs/xfs/xfs_symlink.c
+++ b/fs/xfs/xfs_symlink.c
@@ -23,6 +23,7 @@
 #include "xfs_trans.h"
 #include "xfs_ialloc.h"
 #include "xfs_error.h"
+#include "xfs_errortag.h"
 
 /* ----- Kernel only functions below ----- */
 int
@@ -226,8 +227,15 @@ xfs_symlink(
 		goto out_trans_cancel;
 	}
 
+	/*
+	 * Please refer to the comment in xfs_create() for an explaination of
+	 * why a directory inode cannot have its data fork extent counter to
+	 * overflow.
+	 */
 	error = xfs_iext_count_may_overflow(dp, XFS_DATA_FORK,
 			XFS_IEXT_DIR_MANIP_CNT(mp));
+	ASSERT(XFS_TEST_ERROR(false, dp->i_mount,
+			XFS_ERRTAG_REDUCE_MAX_IEXTENTS) || error == 0);
 	if (error)
 		goto out_trans_cancel;
 
-- 
2.30.2

[PATCH V8 16/19] xfs: Conditionally upgrade existing inodes to use large extent counters

[Chandan Babu R]

This commit enables upgrading existing inodes to use large extent counters
provided that underlying filesystem's superblock has large extent counter
feature enabled.

Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_attr.c       |  9 ++++++-
 fs/xfs/libxfs/xfs_bmap.c       | 10 ++++++--
 fs/xfs/libxfs/xfs_inode_fork.c | 27 +++++++++++++++++++++
 fs/xfs/libxfs/xfs_inode_fork.h |  2 ++
 fs/xfs/xfs_bmap_item.c         |  8 ++++++-
 fs/xfs/xfs_bmap_util.c         | 43 ++++++++++++++++++++++++++++++----
 fs/xfs/xfs_dquot.c             |  9 ++++++-
 fs/xfs/xfs_iomap.c             | 17 ++++++++++++--
 fs/xfs/xfs_reflink.c           | 17 ++++++++++++--
 fs/xfs/xfs_rtalloc.c           |  9 ++++++-
 10 files changed, 136 insertions(+), 15 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c
index 23523b802539..6e56aa17fd82 100644
--- a/fs/xfs/libxfs/xfs_attr.c
+++ b/fs/xfs/libxfs/xfs_attr.c
@@ -776,8 +776,15 @@ xfs_attr_set(
 	if (args->value || xfs_inode_hasattr(dp)) {
 		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
 				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
-		if (error)
+		if (error && error != -EFBIG)
 			goto out_trans_cancel;
+
+		if (error == -EFBIG) {
+			error = xfs_iext_count_upgrade(args->trans, dp,
+					XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
+			if (error)
+				goto out_trans_cancel;
+		}
 	}
 
 	error = xfs_attr_lookup(args);
diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index 5a089674c666..0cb915bf8285 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -4524,13 +4524,19 @@ xfs_bmapi_convert_delalloc(
 		return error;
 
 	xfs_ilock(ip, XFS_ILOCK_EXCL);
+	xfs_trans_ijoin(tp, ip, 0);
 
 	error = xfs_iext_count_may_overflow(ip, whichfork,
 			XFS_IEXT_ADD_NOSPLIT_CNT);
-	if (error)
+	if (error && error != -EFBIG)
 		goto out_trans_cancel;
 
-	xfs_trans_ijoin(tp, ip, 0);
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip,
+				XFS_IEXT_ADD_NOSPLIT_CNT);
+		if (error)
+			goto out_trans_cancel;
+	}
 
 	if (!xfs_iext_lookup_extent(ip, ifp, offset_fsb, &bma.icur, &bma.got) ||
 	    bma.got.br_startoff > offset_fsb) {
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index bb5d841aac58..aff9242db829 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -756,3 +756,30 @@ xfs_iext_count_may_overflow(
 
 	return 0;
 }
+
+int
+xfs_iext_count_upgrade(
+	struct xfs_trans	*tp,
+	struct xfs_inode	*ip,
+	int			nr_to_add)
+{
+	if (!xfs_has_large_extent_counts(ip->i_mount) ||
+	    (ip->i_diflags2 & XFS_DIFLAG2_NREXT64) ||
+	    XFS_TEST_ERROR(false, ip->i_mount, XFS_ERRTAG_REDUCE_MAX_IEXTENTS))
+		return -EFBIG;
+
+	ip->i_diflags2 |= XFS_DIFLAG2_NREXT64;
+	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+
+	/*
+	 * The value of nr_to_add cannot be larger than 2^17
+	 *
+	 * - XFS_MAX_EXTCNT_ATTR_FORK_LARGE - XFS_MAX_EXTCNT_ATTR_FORK_SMALL
+	 *   i.e. 2^32 - 2^15
+	 * - XFS_MAX_EXTCNT_DATA_FORK_LARGE - XFS_MAX_EXTCNT_DATA_FORK_SMALL
+	 *   i.e. 2^48 - 2^31
+	 */
+	ASSERT(nr_to_add <= (1 << 17));
+
+	return 0;
+}
diff --git a/fs/xfs/libxfs/xfs_inode_fork.h b/fs/xfs/libxfs/xfs_inode_fork.h
index fd5c3c2d77e0..f2fe513e7252 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.h
+++ b/fs/xfs/libxfs/xfs_inode_fork.h
@@ -288,6 +288,8 @@ int xfs_ifork_verify_local_data(struct xfs_inode *ip);
 int xfs_ifork_verify_local_attr(struct xfs_inode *ip);
 int xfs_iext_count_may_overflow(struct xfs_inode *ip, int whichfork,
 		int nr_to_add);
+int xfs_iext_count_upgrade(struct xfs_trans *tp, struct xfs_inode *ip,
+		int nr_to_add);
 
 /* returns true if the fork has extents but they are not read in yet. */
 static inline bool xfs_need_iread_extents(struct xfs_ifork *ifp)
diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c
index e1f4d7d5a011..e8729abc15ab 100644
--- a/fs/xfs/xfs_bmap_item.c
+++ b/fs/xfs/xfs_bmap_item.c
@@ -506,9 +506,15 @@ xfs_bui_item_recover(
 		iext_delta = XFS_IEXT_PUNCH_HOLE_CNT;
 
 	error = xfs_iext_count_may_overflow(ip, whichfork, iext_delta);
-	if (error)
+	if (error && error != -EFBIG)
 		goto err_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip, iext_delta);
+		if (error)
+			goto err_cancel;
+	}
+
 	count = bmap->me_len;
 	error = xfs_trans_log_finish_bmap_update(tp, budp, bui_type, ip,
 			whichfork, bmap->me_startoff, bmap->me_startblock,
diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c
index 18c1b99311a8..9f7112625934 100644
--- a/fs/xfs/xfs_bmap_util.c
+++ b/fs/xfs/xfs_bmap_util.c
@@ -859,9 +859,16 @@ xfs_alloc_file_space(
 
 		error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK,
 				XFS_IEXT_ADD_NOSPLIT_CNT);
-		if (error)
+		if (error && error != -EFBIG)
 			goto error;
 
+		if (error == -EFBIG) {
+			error = xfs_iext_count_upgrade(tp, ip,
+					XFS_IEXT_ADD_NOSPLIT_CNT);
+			if (error)
+				goto error;
+		}
+
 		error = xfs_bmapi_write(tp, ip, startoffset_fsb,
 				allocatesize_fsb, XFS_BMAPI_PREALLOC, 0, imapp,
 				&nimaps);
@@ -914,9 +921,15 @@ xfs_unmap_extent(
 
 	error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK,
 			XFS_IEXT_PUNCH_HOLE_CNT);
-	if (error)
+	if (error && error != -EFBIG)
 		goto out_trans_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip, XFS_IEXT_PUNCH_HOLE_CNT);
+		if (error)
+			goto out_trans_cancel;
+	}
+
 	error = xfs_bunmapi(tp, ip, startoffset_fsb, len_fsb, 0, 2, done);
 	if (error)
 		goto out_trans_cancel;
@@ -1195,9 +1208,15 @@ xfs_insert_file_space(
 
 	error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK,
 			XFS_IEXT_PUNCH_HOLE_CNT);
-	if (error)
+	if (error && error != -EFBIG)
 		goto out_trans_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip, XFS_IEXT_PUNCH_HOLE_CNT);
+		if (error)
+			goto out_trans_cancel;
+	}
+
 	/*
 	 * The extent shifting code works on extent granularity. So, if stop_fsb
 	 * is not the starting block of extent, we need to split the extent at
@@ -1423,16 +1442,30 @@ xfs_swap_extent_rmap(
 				error = xfs_iext_count_may_overflow(ip,
 						XFS_DATA_FORK,
 						XFS_IEXT_SWAP_RMAP_CNT);
-				if (error)
+				if (error && error != -EFBIG)
 					goto out;
+
+				if (error == -EFBIG) {
+					error = xfs_iext_count_upgrade(tp, ip,
+							XFS_IEXT_SWAP_RMAP_CNT);
+					if (error)
+						goto out;
+				}
 			}
 
 			if (xfs_bmap_is_real_extent(&irec)) {
 				error = xfs_iext_count_may_overflow(tip,
 						XFS_DATA_FORK,
 						XFS_IEXT_SWAP_RMAP_CNT);
-				if (error)
+				if (error && error != -EFBIG)
 					goto out;
+
+				if (error == -EFBIG) {
+					error = xfs_iext_count_upgrade(tp, ip,
+							XFS_IEXT_SWAP_RMAP_CNT);
+					if (error)
+						goto out;
+				}
 			}
 
 			/* Remove the mapping from the donor file. */
diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c
index 5afedcbc78c7..8b8fc98a01eb 100644
--- a/fs/xfs/xfs_dquot.c
+++ b/fs/xfs/xfs_dquot.c
@@ -322,9 +322,16 @@ xfs_dquot_disk_alloc(
 
 	error = xfs_iext_count_may_overflow(quotip, XFS_DATA_FORK,
 			XFS_IEXT_ADD_NOSPLIT_CNT);
-	if (error)
+	if (error && error != -EFBIG)
 		goto err_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, quotip,
+				XFS_IEXT_ADD_NOSPLIT_CNT);
+		if (error)
+			goto err_cancel;
+	}
+
 	/* Create the block mapping. */
 	error = xfs_bmapi_write(tp, quotip, dqp->q_fileoffset,
 			XFS_DQUOT_CLUSTER_SIZE_FSB, XFS_BMAPI_METADATA, 0, &map,
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 87e1cf5060bd..57c3c2ccb07b 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -251,9 +251,15 @@ xfs_iomap_write_direct(
 		return error;
 
 	error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK, nr_exts);
-	if (error)
+	if (error && error != -EFBIG)
 		goto out_trans_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip, nr_exts);
+		if (error)
+			goto out_trans_cancel;
+	}
+
 	/*
 	 * From this point onwards we overwrite the imap pointer that the
 	 * caller gave to us.
@@ -555,9 +561,16 @@ xfs_iomap_write_unwritten(
 
 		error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK,
 				XFS_IEXT_WRITE_UNWRITTEN_CNT);
-		if (error)
+		if (error && error != -EFBIG)
 			goto error_on_bmapi_transaction;
 
+		if (error == -EFBIG) {
+			error = xfs_iext_count_upgrade(tp, ip,
+					XFS_IEXT_WRITE_UNWRITTEN_CNT);
+			if (error)
+				goto error_on_bmapi_transaction;
+		}
+
 		/*
 		 * Modify the unwritten extent state of the buffer.
 		 */
diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c
index db70060e7bf6..04717be330d7 100644
--- a/fs/xfs/xfs_reflink.c
+++ b/fs/xfs/xfs_reflink.c
@@ -617,9 +617,16 @@ xfs_reflink_end_cow_extent(
 
 	error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK,
 			XFS_IEXT_REFLINK_END_COW_CNT);
-	if (error)
+	if (error && error != -EFBIG)
 		goto out_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip,
+				XFS_IEXT_REFLINK_END_COW_CNT);
+		if (error)
+			goto out_cancel;
+	}
+
 	/*
 	 * In case of racing, overlapping AIO writes no COW extents might be
 	 * left by the time I/O completes for the loser of the race.  In that
@@ -1118,9 +1125,15 @@ xfs_reflink_remap_extent(
 		++iext_delta;
 
 	error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK, iext_delta);
-	if (error)
+	if (error && error != -EFBIG)
 		goto out_cancel;
 
+	if (error == -EFBIG) {
+		error = xfs_iext_count_upgrade(tp, ip, iext_delta);
+		if (error)
+			goto out_cancel;
+	}
+
 	if (smap_real) {
 		/*
 		 * If the extent we're unmapping is backed by storage (written
diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c
index b8c79ee791af..8cab153201d0 100644
--- a/fs/xfs/xfs_rtalloc.c
+++ b/fs/xfs/xfs_rtalloc.c
@@ -806,9 +806,16 @@ xfs_growfs_rt_alloc(
 
 		error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK,
 				XFS_IEXT_ADD_NOSPLIT_CNT);
-		if (error)
+		if (error && error != -EFBIG)
 			goto out_trans_cancel;
 
+		if (error == -EFBIG) {
+			error = xfs_iext_count_upgrade(tp, ip,
+					XFS_IEXT_ADD_NOSPLIT_CNT);
+			if (error)
+				goto out_trans_cancel;
+		}
+
 		/*
 		 * Allocate blocks to the bitmap file.
 		 */
-- 
2.30.2

[PATCH V8 17/19] xfs: Decouple XFS_IBULK flags from XFS_IWALK flags

[Chandan Babu R]

A future commit will add a new XFS_IBULK flag which will not have a
corresponding XFS_IWALK flag. In preparation for the change, this commit
separates XFS_IBULK_* flags from XFS_IWALK_* flags.

Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/xfs_itable.c | 6 +++++-
 fs/xfs/xfs_itable.h | 2 +-
 fs/xfs/xfs_iwalk.h  | 2 +-
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/fs/xfs/xfs_itable.c b/fs/xfs/xfs_itable.c
index c08c79d9e311..71ed4905f206 100644
--- a/fs/xfs/xfs_itable.c
+++ b/fs/xfs/xfs_itable.c
@@ -256,6 +256,7 @@ xfs_bulkstat(
 		.breq		= breq,
 	};
 	struct xfs_trans	*tp;
+	unsigned int		iwalk_flags = 0;
 	int			error;
 
 	if (breq->mnt_userns != &init_user_ns) {
@@ -279,7 +280,10 @@ xfs_bulkstat(
 	if (error)
 		goto out;
 
-	error = xfs_iwalk(breq->mp, tp, breq->startino, breq->flags,
+	if (breq->flags & XFS_IBULK_SAME_AG)
+		iwalk_flags |= XFS_IWALK_SAME_AG;
+
+	error = xfs_iwalk(breq->mp, tp, breq->startino, iwalk_flags,
 			xfs_bulkstat_iwalk, breq->icount, &bc);
 	xfs_trans_cancel(tp);
 out:
diff --git a/fs/xfs/xfs_itable.h b/fs/xfs/xfs_itable.h
index 7078d10c9b12..2cf3872fcd2f 100644
--- a/fs/xfs/xfs_itable.h
+++ b/fs/xfs/xfs_itable.h
@@ -17,7 +17,7 @@ struct xfs_ibulk {
 };
 
 /* Only iterate within the same AG as startino */
-#define XFS_IBULK_SAME_AG	(XFS_IWALK_SAME_AG)
+#define XFS_IBULK_SAME_AG	(1 << 0)
 
 /*
  * Advance the user buffer pointer by one record of the given size.  If the
diff --git a/fs/xfs/xfs_iwalk.h b/fs/xfs/xfs_iwalk.h
index 37a795f03267..3a68766fd909 100644
--- a/fs/xfs/xfs_iwalk.h
+++ b/fs/xfs/xfs_iwalk.h
@@ -26,7 +26,7 @@ int xfs_iwalk_threaded(struct xfs_mount *mp, xfs_ino_t startino,
 		unsigned int inode_records, bool poll, void *data);
 
 /* Only iterate inodes within the same AG as @startino. */
-#define XFS_IWALK_SAME_AG	(0x1)
+#define XFS_IWALK_SAME_AG	(1 << 0)
 
 #define XFS_IWALK_FLAGS_ALL	(XFS_IWALK_SAME_AG)
 
-- 
2.30.2

[PATCH V8 18/19] xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters

[Chandan Babu R]

The following changes are made to enable userspace to obtain 64-bit extent
counters,
1. Carve out a new 64-bit field xfs_bulkstat->bs_extents64 from
   xfs_bulkstat->bs_pad[] to hold 64-bit extent counter.
2. Define the new flag XFS_BULK_IREQ_BULKSTAT for userspace to indicate that
   it is capable of receiving 64-bit extent counters.

Suggested-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_fs.h | 20 ++++++++++++++++----
 fs/xfs/xfs_ioctl.c     |  3 +++
 fs/xfs/xfs_itable.c    | 13 ++++++++++++-
 fs/xfs/xfs_itable.h    |  2 ++
 4 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_fs.h b/fs/xfs/libxfs/xfs_fs.h
index 1f7238db35cc..2a42bfb85c3b 100644
--- a/fs/xfs/libxfs/xfs_fs.h
+++ b/fs/xfs/libxfs/xfs_fs.h
@@ -378,7 +378,7 @@ struct xfs_bulkstat {
 	uint32_t	bs_extsize_blks; /* extent size hint, blocks	*/
 
 	uint32_t	bs_nlink;	/* number of links		*/
-	uint32_t	bs_extents;	/* number of extents		*/
+	uint32_t	bs_extents;	/* 32-bit data fork extent counter */
 	uint32_t	bs_aextents;	/* attribute number of extents	*/
 	uint16_t	bs_version;	/* structure version		*/
 	uint16_t	bs_forkoff;	/* inode fork offset in bytes	*/
@@ -387,8 +387,9 @@ struct xfs_bulkstat {
 	uint16_t	bs_checked;	/* checked inode metadata	*/
 	uint16_t	bs_mode;	/* type and mode		*/
 	uint16_t	bs_pad2;	/* zeroed			*/
+	uint64_t	bs_extents64;	/* 64-bit data fork extent counter */
 
-	uint64_t	bs_pad[7];	/* zeroed			*/
+	uint64_t	bs_pad[6];	/* zeroed			*/
 };
 
 #define XFS_BULKSTAT_VERSION_V1	(1)
@@ -469,8 +470,19 @@ struct xfs_bulk_ireq {
  */
 #define XFS_BULK_IREQ_SPECIAL	(1 << 1)
 
-#define XFS_BULK_IREQ_FLAGS_ALL	(XFS_BULK_IREQ_AGNO | \
-				 XFS_BULK_IREQ_SPECIAL)
+/*
+ * Return data fork extent count via xfs_bulkstat->bs_extents64 field and assign
+ * 0 to xfs_bulkstat->bs_extents when the flag is set.  Otherwise, use
+ * xfs_bulkstat->bs_extents for returning data fork extent count and set
+ * xfs_bulkstat->bs_extents64 to 0. In the second case, return -EOVERFLOW and
+ * assign 0 to xfs_bulkstat->bs_extents if data fork extent count is larger than
+ * XFS_MAX_EXTCNT_DATA_FORK_OLD.
+ */
+#define XFS_BULK_IREQ_NREXT64	(1 << 2)
+
+#define XFS_BULK_IREQ_FLAGS_ALL	(XFS_BULK_IREQ_AGNO |	 \
+				 XFS_BULK_IREQ_SPECIAL | \
+				 XFS_BULK_IREQ_NREXT64)
 
 /* Operate on the root directory inode. */
 #define XFS_BULK_IREQ_SPECIAL_ROOT	(1)
diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c
index 2515fe8299e1..22947c5ffd34 100644
--- a/fs/xfs/xfs_ioctl.c
+++ b/fs/xfs/xfs_ioctl.c
@@ -813,6 +813,9 @@ xfs_bulk_ireq_setup(
 	if (XFS_INO_TO_AGNO(mp, breq->startino) >= mp->m_sb.sb_agcount)
 		return -ECANCELED;
 
+	if (hdr->flags & XFS_BULK_IREQ_NREXT64)
+		breq->flags |= XFS_IBULK_NREXT64;
+
 	return 0;
 }
 
diff --git a/fs/xfs/xfs_itable.c b/fs/xfs/xfs_itable.c
index 71ed4905f206..847f03f75a38 100644
--- a/fs/xfs/xfs_itable.c
+++ b/fs/xfs/xfs_itable.c
@@ -64,6 +64,7 @@ xfs_bulkstat_one_int(
 	struct xfs_inode	*ip;		/* incore inode pointer */
 	struct inode		*inode;
 	struct xfs_bulkstat	*buf = bc->buf;
+	xfs_extnum_t		nextents;
 	int			error = -EINVAL;
 
 	if (xfs_internal_inum(mp, ino))
@@ -102,7 +103,17 @@ xfs_bulkstat_one_int(
 
 	buf->bs_xflags = xfs_ip2xflags(ip);
 	buf->bs_extsize_blks = ip->i_extsize;
-	buf->bs_extents = xfs_ifork_nextents(&ip->i_df);
+
+	nextents = xfs_ifork_nextents(&ip->i_df);
+	if (!(bc->breq->flags & XFS_IBULK_NREXT64)) {
+		if (nextents > XFS_MAX_EXTCNT_DATA_FORK_SMALL)
+			buf->bs_extents = XFS_MAX_EXTCNT_DATA_FORK_SMALL;
+		else
+			buf->bs_extents = nextents;
+	} else {
+		buf->bs_extents64 = nextents;
+	}
+
 	xfs_bulkstat_health(ip, buf);
 	buf->bs_aextents = xfs_ifork_nextents(ip->i_afp);
 	buf->bs_forkoff = XFS_IFORK_BOFF(ip);
diff --git a/fs/xfs/xfs_itable.h b/fs/xfs/xfs_itable.h
index 2cf3872fcd2f..0150fd53d18e 100644
--- a/fs/xfs/xfs_itable.h
+++ b/fs/xfs/xfs_itable.h
@@ -19,6 +19,8 @@ struct xfs_ibulk {
 /* Only iterate within the same AG as startino */
 #define XFS_IBULK_SAME_AG	(1 << 0)
 
+#define XFS_IBULK_NREXT64	(1 << 1)
+
 /*
  * Advance the user buffer pointer by one record of the given size.  If the
  * buffer is now full, return the appropriate error code.
-- 
2.30.2

[PATCH V8 19/19] xfs: Add XFS_SB_FEAT_INCOMPAT_NREXT64 to the list of supported flags

[Chandan Babu R]

This commit enables XFS module to work with fs instances having 64-bit
per-inode extent counters by adding XFS_SB_FEAT_INCOMPAT_NREXT64 flag to the
list of supported incompat feature flags.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
---
 fs/xfs/libxfs/xfs_format.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
index 82b404c99b80..25a157b615ce 100644
--- a/fs/xfs/libxfs/xfs_format.h
+++ b/fs/xfs/libxfs/xfs_format.h
@@ -378,7 +378,8 @@ xfs_sb_has_ro_compat_feature(
 		 XFS_SB_FEAT_INCOMPAT_SPINODES|	\
 		 XFS_SB_FEAT_INCOMPAT_META_UUID| \
 		 XFS_SB_FEAT_INCOMPAT_BIGTIME| \
-		 XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR)
+		 XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \
+		 XFS_SB_FEAT_INCOMPAT_NREXT64)
 
 #define XFS_SB_FEAT_INCOMPAT_UNKNOWN	~XFS_SB_FEAT_INCOMPAT_ALL
 static inline bool
-- 
2.30.2

Re: [PATCH V8 05/19] xfs: Introduce xfs_dfork_nextents() helper

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:36AM +0530, Chandan Babu R wrote:
> This commit replaces the macro XFS_DFORK_NEXTENTS() with the helper function
> xfs_dfork_nextents(). As of this commit, xfs_dfork_nextents() returns the same
> value as XFS_DFORK_NEXTENTS(). A future commit which extends inode's extent
> counter fields will add more logic to this helper.
> 
> This commit also replaces direct accesses to xfs_dinode->di_[a]nextents
> with calls to xfs_dfork_nextents().
> 
> No functional changes have been made.
> 
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>

Looks good now.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 07/19] xfs: Promote xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits respectively

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:38AM +0530, Chandan Babu R wrote:
> A future commit will introduce a 64-bit on-disk data extent counter and a
> 32-bit on-disk attr extent counter. This commit promotes xfs_extnum_t and
> xfs_aextnum_t to 64 and 32-bits in order to correctly handle in-core versions
> of these quantities.
> 
> Reported-by: kernel test robot <lkp@intel.com>
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>

Looks ok.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 08/19] xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs feature bit

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:39AM +0530, Chandan Babu R wrote:
> XFS_SB_FEAT_INCOMPAT_NREXT64 incompat feature bit will be set on filesystems
> which support large per-inode extent counters. This commit defines the new
> incompat feature bit and the corresponding per-fs feature bit (along with
> inline functions to work on it).
> 
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
>  fs/xfs/libxfs/xfs_format.h | 1 +
>  fs/xfs/libxfs/xfs_sb.c     | 3 +++
>  fs/xfs/xfs_mount.h         | 2 ++
>  3 files changed, 6 insertions(+)
> 
> diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
> index b5e9256d6d32..64ff0c310696 100644
> --- a/fs/xfs/libxfs/xfs_format.h
> +++ b/fs/xfs/libxfs/xfs_format.h
> @@ -372,6 +372,7 @@ xfs_sb_has_ro_compat_feature(
>  #define XFS_SB_FEAT_INCOMPAT_META_UUID	(1 << 2)	/* metadata UUID */
>  #define XFS_SB_FEAT_INCOMPAT_BIGTIME	(1 << 3)	/* large timestamps */
>  #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)	/* needs xfs_repair */
> +#define XFS_SB_FEAT_INCOMPAT_NREXT64	(1 << 5)	/* large extent counters */
>  #define XFS_SB_FEAT_INCOMPAT_ALL \
>  		(XFS_SB_FEAT_INCOMPAT_FTYPE|	\
>  		 XFS_SB_FEAT_INCOMPAT_SPINODES|	\
> diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c
> index f4e84aa1d50a..bd632389ae92 100644
> --- a/fs/xfs/libxfs/xfs_sb.c
> +++ b/fs/xfs/libxfs/xfs_sb.c
> @@ -124,6 +124,9 @@ xfs_sb_version_to_features(
>  		features |= XFS_FEAT_BIGTIME;
>  	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR)
>  		features |= XFS_FEAT_NEEDSREPAIR;
> +	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NREXT64)
> +		features |= XFS_FEAT_NREXT64;
> +

Shouldn't enabling the feature be the last patch in the series, once
all the infrastructure to support the feature is in place? i.e. if
someone does a bisect on a XFS_FEAT_NREXT64 enabled filesystem, they
can land in the middle of this series where the fs tries to use
XFS_FEAT_NREXT64 but the functionality is not complete.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 10/19] xfs: Introduce XFS_DIFLAG2_NREXT64 and associated helpers

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:41AM +0530, Chandan Babu R wrote:
> This commit adds the new per-inode flag XFS_DIFLAG2_NREXT64 to indicate that
> an inode supports 64-bit extent counters. This flag is also enabled by default
> on newly created inodes when the corresponding filesystem has large extent
> counter feature bit (i.e. XFS_FEAT_NREXT64) set.
> 
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>

Looks good. The code is much more readable than the previous
version.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 08/19] xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs feature bit

[Darrick J. Wong]

On Fri, Mar 25, 2022 at 08:37:19AM +1100, Dave Chinner wrote:
> On Mon, Mar 21, 2022 at 10:47:39AM +0530, Chandan Babu R wrote:
> > XFS_SB_FEAT_INCOMPAT_NREXT64 incompat feature bit will be set on filesystems
> > which support large per-inode extent counters. This commit defines the new
> > incompat feature bit and the corresponding per-fs feature bit (along with
> > inline functions to work on it).
> > 
> > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> > Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> > ---
> >  fs/xfs/libxfs/xfs_format.h | 1 +
> >  fs/xfs/libxfs/xfs_sb.c     | 3 +++
> >  fs/xfs/xfs_mount.h         | 2 ++
> >  3 files changed, 6 insertions(+)
> > 
> > diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
> > index b5e9256d6d32..64ff0c310696 100644
> > --- a/fs/xfs/libxfs/xfs_format.h
> > +++ b/fs/xfs/libxfs/xfs_format.h
> > @@ -372,6 +372,7 @@ xfs_sb_has_ro_compat_feature(
> >  #define XFS_SB_FEAT_INCOMPAT_META_UUID	(1 << 2)	/* metadata UUID */
> >  #define XFS_SB_FEAT_INCOMPAT_BIGTIME	(1 << 3)	/* large timestamps */
> >  #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)	/* needs xfs_repair */
> > +#define XFS_SB_FEAT_INCOMPAT_NREXT64	(1 << 5)	/* large extent counters */
> >  #define XFS_SB_FEAT_INCOMPAT_ALL \
> >  		(XFS_SB_FEAT_INCOMPAT_FTYPE|	\
> >  		 XFS_SB_FEAT_INCOMPAT_SPINODES|	\
> > diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c
> > index f4e84aa1d50a..bd632389ae92 100644
> > --- a/fs/xfs/libxfs/xfs_sb.c
> > +++ b/fs/xfs/libxfs/xfs_sb.c
> > @@ -124,6 +124,9 @@ xfs_sb_version_to_features(
> >  		features |= XFS_FEAT_BIGTIME;
> >  	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR)
> >  		features |= XFS_FEAT_NEEDSREPAIR;
> > +	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NREXT64)
> > +		features |= XFS_FEAT_NREXT64;
> > +
> 
> Shouldn't enabling the feature be the last patch in the series, once
> all the infrastructure to support the feature is in place? i.e. if
> someone does a bisect on a XFS_FEAT_NREXT64 enabled filesystem, they
> can land in the middle of this series where the fs tries to use
> XFS_FEAT_NREXT64 but the functionality is not complete.

The last patch of the series does exactly that, by adding NREXT64 to
XFS_SB_FEAT_INCOMPAT_ALL.

--D

> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH V8 11/19] xfs: Use uint64_t to count maximum blocks that can be used by BMBT

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:42AM +0530, Chandan Babu R wrote:
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
>  fs/xfs/libxfs/xfs_bmap.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
> index 9f38e33d6ce2..b317226fb4ba 100644
> --- a/fs/xfs/libxfs/xfs_bmap.c
> +++ b/fs/xfs/libxfs/xfs_bmap.c
> @@ -52,9 +52,9 @@ xfs_bmap_compute_maxlevels(
>  	xfs_mount_t	*mp,		/* file system mount structure */
>  	int		whichfork)	/* data or attr fork */
>  {
> -	int		level;		/* btree level */
> -	uint		maxblocks;	/* max blocks at this level */
> +	uint64_t	maxblocks;	/* max blocks at this level */
>  	xfs_extnum_t	maxleafents;	/* max leaf entries possible */
> +	int		level;		/* btree level */
>  	int		maxrootrecs;	/* max records in root block */
>  	int		minleafrecs;	/* min records in leaf block */
>  	int		minnoderecs;	/* min records in node block */
> @@ -88,7 +88,7 @@ xfs_bmap_compute_maxlevels(
>  		if (maxblocks <= maxrootrecs)
>  			maxblocks = 1;
>  		else
> -			maxblocks = (maxblocks + minnoderecs - 1) / minnoderecs;
> +			maxblocks = howmany_64(maxblocks, minnoderecs);
>  	}
>  	mp->m_bm_maxlevels[whichfork] = level;
>  	ASSERT(mp->m_bm_maxlevels[whichfork] <= xfs_bmbt_maxlevels_ondisk());

Hmmmm. Shouldn't this be rolled up into the earlier patch that
converts a seperate part of this function to use howmany_64()?
That was done in "[PATCH V8 07/19] xfs: Promote xfs_extnum_t and
xfs_aextnum_t to 64 and 32-bits respectively" - it seems to me like
this should definitely be part of the type size extension rather
than a stand-alone change.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 13/19] xfs: Replace numbered inode recovery error messages with descriptive ones

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:44AM +0530, Chandan Babu R wrote:
> This commit also prints inode fields with invalid values instead of printing
> addresses of inode and buffer instances.
> 
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> Suggested-by: Dave Chinner <dchinner@redhat.com>

Looks good.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 14/19] xfs: Introduce per-inode 64-bit extent counters

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:45AM +0530, Chandan Babu R wrote:
> This commit introduces new fields in the on-disk inode format to support
> 64-bit data fork extent counters and 32-bit attribute fork extent
> counters. The new fields will be used only when an inode has
> XFS_DIFLAG2_NREXT64 flag set. Otherwise we continue to use the regular 32-bit
> data fork extent counters and 16-bit attribute fork extent counters.
> 
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> Suggested-by: Dave Chinner <dchinner@redhat.com>

Looks good, minor nits below.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

> +STATIC int
> +xlog_dinode_verify_extent_counts(
> +	struct xfs_mount	*mp,
> +	struct xfs_log_dinode	*ldip)
> +{
> +	xfs_extnum_t		nextents;
> +	xfs_aextnum_t		anextents;
> +
> +	if (xfs_log_dinode_has_large_extent_counts(ldip)) {
> +		if (!xfs_has_large_extent_counts(mp) ||
> +		    (ldip->di_nrext64_pad != 0)) {
> +			XFS_CORRUPTION_ERROR(
> +				"Bad log dinode large extent count format",
> +				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
> +			xfs_alert(mp,
> +				"Bad inode 0x%llx, nrext64 %d, padding 0x%x",

"large extent counts" rather than "nrext64"?

> +				ldip->di_ino, xfs_has_large_extent_counts(mp),
> +				ldip->di_nrext64_pad);
> +			return -EFSCORRUPTED;
> +		}
> +
> +		nextents = ldip->di_big_nextents;
> +		anextents = ldip->di_big_anextents;
> +	} else {
> +		if (ldip->di_version == 3 && ldip->di_v3_pad != 0) {
> +			XFS_CORRUPTION_ERROR(
> +				"Bad log dinode di_v3_pad",
> +				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
> +			xfs_alert(mp,
> +				"Bad inode 0x%llx, di_v3_pad %llu",
> +				ldip->di_ino, ldip->di_v3_pad);

the padding should be dumped as a hexadecimal value - it's much
easier to see the bit corruption patterns in hex dumps than decimal
output.

> +			return -EFSCORRUPTED;
> +		}
> +
> +		nextents = ldip->di_nextents;
> +		anextents = ldip->di_anextents;
> +	}
> +
> +	if (unlikely(nextents + anextents > ldip->di_nblocks)) {
> +		XFS_CORRUPTION_ERROR("Bad log dinode extent counts",
> +				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
> +		xfs_alert(mp,
> +			"Bad inode 0x%llx, nextents 0x%llx, anextents 0x%x, nblocks 0x%llx",
> +			ldip->di_ino, nextents, anextents, ldip->di_nblocks);
> +		return -EFSCORRUPTED;

Maybe should indicate large extent count status here, too.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
> The maximum file size that can be represented by the data fork extent counter
> in the worst case occurs when all extents are 1 block in length and each block
> is 1KB in size.
> 
> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
> 1KB sized blocks, a file can reach upto,
> (2^31) * 1KB = 2TB
> 
> This is much larger than the theoretical maximum size of a directory
> i.e. 32GB * 3 = 96GB.
> 
> Since a directory's inode can never overflow its data fork extent counter,
> this commit replaces checking the return value of
> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).

I'd really prefer that we don't add noise like this to a bunch of
call sites.  If directories can't overflow the extent count in
normal operation, then why are we even calling
xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
be a sign of an inode corruption, and we should have flagged that
long before we do an operation that might overflow the extent count.

So, really, I think you should document the directory size
constraints at the site where we define all the large extent count
values in xfs_format.h, remove the xfs_iext_count_may_overflow()
checks from the directory code and replace them with a simple inode
verifier check that we haven't got more than 100GB worth of
individual extents in the data fork for directory inodes....

Then all this directory specific "can't possibly overflow" overflow
checks can go away completely.  The best code is no code :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 16/19] xfs: Conditionally upgrade existing inodes to use large extent counters

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:47AM +0530, Chandan Babu R wrote:
> This commit enables upgrading existing inodes to use large extent counters
> provided that underlying filesystem's superblock has large extent counter
> feature enabled.
> 
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
> ---
>  fs/xfs/libxfs/xfs_attr.c       |  9 ++++++-
>  fs/xfs/libxfs/xfs_bmap.c       | 10 ++++++--
>  fs/xfs/libxfs/xfs_inode_fork.c | 27 +++++++++++++++++++++
>  fs/xfs/libxfs/xfs_inode_fork.h |  2 ++
>  fs/xfs/xfs_bmap_item.c         |  8 ++++++-
>  fs/xfs/xfs_bmap_util.c         | 43 ++++++++++++++++++++++++++++++----
>  fs/xfs/xfs_dquot.c             |  9 ++++++-
>  fs/xfs/xfs_iomap.c             | 17 ++++++++++++--
>  fs/xfs/xfs_reflink.c           | 17 ++++++++++++--
>  fs/xfs/xfs_rtalloc.c           |  9 ++++++-
>  10 files changed, 136 insertions(+), 15 deletions(-)
> 
> diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c
> index 23523b802539..6e56aa17fd82 100644
> --- a/fs/xfs/libxfs/xfs_attr.c
> +++ b/fs/xfs/libxfs/xfs_attr.c
> @@ -776,8 +776,15 @@ xfs_attr_set(
>  	if (args->value || xfs_inode_hasattr(dp)) {
>  		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
>  				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
> -		if (error)
> +		if (error && error != -EFBIG)
>  			goto out_trans_cancel;
> +
> +		if (error == -EFBIG) {
> +			error = xfs_iext_count_upgrade(args->trans, dp,
> +					XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
> +			if (error)
> +				goto out_trans_cancel;
> +		}

Neater and more compact to do this by checking explicitly for
-EFBIG:

		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
		if (error == -EFBIG)
			error = xfs_iext_count_upgrade(args->trans, dp,
					XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
		if (error)
			goto out_trans_cancel;
	}
>  
>  	error = xfs_attr_lookup(args);
> diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
> index 5a089674c666..0cb915bf8285 100644
> --- a/fs/xfs/libxfs/xfs_bmap.c
> +++ b/fs/xfs/libxfs/xfs_bmap.c
> @@ -4524,13 +4524,19 @@ xfs_bmapi_convert_delalloc(
>  		return error;
>  
>  	xfs_ilock(ip, XFS_ILOCK_EXCL);
> +	xfs_trans_ijoin(tp, ip, 0);
>  
>  	error = xfs_iext_count_may_overflow(ip, whichfork,
>  			XFS_IEXT_ADD_NOSPLIT_CNT);
> -	if (error)
> +	if (error && error != -EFBIG)
>  		goto out_trans_cancel;
>  
> -	xfs_trans_ijoin(tp, ip, 0);
> +	if (error == -EFBIG) {
> +		error = xfs_iext_count_upgrade(tp, ip,
> +				XFS_IEXT_ADD_NOSPLIT_CNT);
> +		if (error)
> +			goto out_trans_cancel;
> +	}
>  
>  	if (!xfs_iext_lookup_extent(ip, ifp, offset_fsb, &bma.icur, &bma.got) ||
>  	    bma.got.br_startoff > offset_fsb) {
> diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
> index bb5d841aac58..aff9242db829 100644
> --- a/fs/xfs/libxfs/xfs_inode_fork.c
> +++ b/fs/xfs/libxfs/xfs_inode_fork.c
> @@ -756,3 +756,30 @@ xfs_iext_count_may_overflow(
>  
>  	return 0;
>  }
> +
> +int
> +xfs_iext_count_upgrade(
> +	struct xfs_trans	*tp,
> +	struct xfs_inode	*ip,
> +	int			nr_to_add)

nr_to_add can only be positive, so should be unsigned.

> +{
> +	if (!xfs_has_large_extent_counts(ip->i_mount) ||
> +	    (ip->i_diflags2 & XFS_DIFLAG2_NREXT64) ||
> +	    XFS_TEST_ERROR(false, ip->i_mount, XFS_ERRTAG_REDUCE_MAX_IEXTENTS))
> +		return -EFBIG;
> +
> +	ip->i_diflags2 |= XFS_DIFLAG2_NREXT64;
> +	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
> +
> +	/*
> +	 * The value of nr_to_add cannot be larger than 2^17
> +	 *
> +	 * - XFS_MAX_EXTCNT_ATTR_FORK_LARGE - XFS_MAX_EXTCNT_ATTR_FORK_SMALL
> +	 *   i.e. 2^32 - 2^15
> +	 * - XFS_MAX_EXTCNT_DATA_FORK_LARGE - XFS_MAX_EXTCNT_DATA_FORK_SMALL
> +	 *   i.e. 2^48 - 2^31
> +	 */
> +	ASSERT(nr_to_add <= (1 << 17));

That's a comment for the function head and/or the format
documentation in xfs_format.h, not hidden in the code itself as a
magic number. i.e. it is a format definition because it is bound by
on-disk format constants, not by code constratints. Hence this
should probably be defined in xfs_format.h alongside the large/small
extent counts, such as:

#define XFS_MAX_EXTCNT_UPGRADE_NR	\
	min(XFS_MAX_EXTCNT_ATTR_FORK_LARGE - XFS_MAX_EXTCNT_ATTR_FORK_SMALL, \
	    XFS_MAX_EXTCNT_DATA_FORK_LARGE - XFS_MAX_EXTCNT_DATA_FORK_SMALL)

And the ASSERT checking the incoming nr_to_add placed right at the
top of the function because the assert then documents API
constraints and always catches violations of them.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 17/19] xfs: Decouple XFS_IBULK flags from XFS_IWALK flags

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:48AM +0530, Chandan Babu R wrote:
> A future commit will add a new XFS_IBULK flag which will not have a
> corresponding XFS_IWALK flag. In preparation for the change, this commit
> separates XFS_IBULK_* flags from XFS_IWALK_* flags.
> 
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>

looks good.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 18/19] xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters

[Dave Chinner]

On Mon, Mar 21, 2022 at 10:47:49AM +0530, Chandan Babu R wrote:
> The following changes are made to enable userspace to obtain 64-bit extent
> counters,
> 1. Carve out a new 64-bit field xfs_bulkstat->bs_extents64 from
>    xfs_bulkstat->bs_pad[] to hold 64-bit extent counter.
> 2. Define the new flag XFS_BULK_IREQ_BULKSTAT for userspace to indicate that
>    it is capable of receiving 64-bit extent counters.
> 
> Suggested-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>

Much simpler! :)

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 08/19] xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs feature bit

[Chandan Babu R]

On 25 Mar 2022 at 03:10, Darrick J. Wong wrote:
> On Fri, Mar 25, 2022 at 08:37:19AM +1100, Dave Chinner wrote:
>> On Mon, Mar 21, 2022 at 10:47:39AM +0530, Chandan Babu R wrote:
>> > XFS_SB_FEAT_INCOMPAT_NREXT64 incompat feature bit will be set on filesystems
>> > which support large per-inode extent counters. This commit defines the new
>> > incompat feature bit and the corresponding per-fs feature bit (along with
>> > inline functions to work on it).
>> > 
>> > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>> > Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> > ---
>> >  fs/xfs/libxfs/xfs_format.h | 1 +
>> >  fs/xfs/libxfs/xfs_sb.c     | 3 +++
>> >  fs/xfs/xfs_mount.h         | 2 ++
>> >  3 files changed, 6 insertions(+)
>> > 
>> > diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h
>> > index b5e9256d6d32..64ff0c310696 100644
>> > --- a/fs/xfs/libxfs/xfs_format.h
>> > +++ b/fs/xfs/libxfs/xfs_format.h
>> > @@ -372,6 +372,7 @@ xfs_sb_has_ro_compat_feature(
>> >  #define XFS_SB_FEAT_INCOMPAT_META_UUID	(1 << 2)	/* metadata UUID */
>> >  #define XFS_SB_FEAT_INCOMPAT_BIGTIME	(1 << 3)	/* large timestamps */
>> >  #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)	/* needs xfs_repair */
>> > +#define XFS_SB_FEAT_INCOMPAT_NREXT64	(1 << 5)	/* large extent counters */
>> >  #define XFS_SB_FEAT_INCOMPAT_ALL \
>> >  		(XFS_SB_FEAT_INCOMPAT_FTYPE|	\
>> >  		 XFS_SB_FEAT_INCOMPAT_SPINODES|	\
>> > diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c
>> > index f4e84aa1d50a..bd632389ae92 100644
>> > --- a/fs/xfs/libxfs/xfs_sb.c
>> > +++ b/fs/xfs/libxfs/xfs_sb.c
>> > @@ -124,6 +124,9 @@ xfs_sb_version_to_features(
>> >  		features |= XFS_FEAT_BIGTIME;
>> >  	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR)
>> >  		features |= XFS_FEAT_NEEDSREPAIR;
>> > +	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NREXT64)
>> > +		features |= XFS_FEAT_NREXT64;
>> > +
>> 
>> Shouldn't enabling the feature be the last patch in the series, once
>> all the infrastructure to support the feature is in place? i.e. if
>> someone does a bisect on a XFS_FEAT_NREXT64 enabled filesystem, they
>> can land in the middle of this series where the fs tries to use
>> XFS_FEAT_NREXT64 but the functionality is not complete.
>
> The last patch of the series does exactly that, by adding NREXT64 to
> XFS_SB_FEAT_INCOMPAT_ALL.
>

If a bisection lands on the current patch, the "Unknown incompat feature"
check would cause the filesystem mount to fail. Hence, I think the code is
safe since it does not allow partially implemented NREXT64 feature to be
executed.

-- 
chandan

Re: [PATCH V8 11/19] xfs: Use uint64_t to count maximum blocks that can be used by BMBT

[Chandan Babu R]

On 25 Mar 2022 at 03:12, Dave Chinner wrote:
> On Mon, Mar 21, 2022 at 10:47:42AM +0530, Chandan Babu R wrote:
>> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>>  fs/xfs/libxfs/xfs_bmap.c | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>> 
>> diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
>> index 9f38e33d6ce2..b317226fb4ba 100644
>> --- a/fs/xfs/libxfs/xfs_bmap.c
>> +++ b/fs/xfs/libxfs/xfs_bmap.c
>> @@ -52,9 +52,9 @@ xfs_bmap_compute_maxlevels(
>>  	xfs_mount_t	*mp,		/* file system mount structure */
>>  	int		whichfork)	/* data or attr fork */
>>  {
>> -	int		level;		/* btree level */
>> -	uint		maxblocks;	/* max blocks at this level */
>> +	uint64_t	maxblocks;	/* max blocks at this level */
>>  	xfs_extnum_t	maxleafents;	/* max leaf entries possible */
>> +	int		level;		/* btree level */
>>  	int		maxrootrecs;	/* max records in root block */
>>  	int		minleafrecs;	/* min records in leaf block */
>>  	int		minnoderecs;	/* min records in node block */
>> @@ -88,7 +88,7 @@ xfs_bmap_compute_maxlevels(
>>  		if (maxblocks <= maxrootrecs)
>>  			maxblocks = 1;
>>  		else
>> -			maxblocks = (maxblocks + minnoderecs - 1) / minnoderecs;
>> +			maxblocks = howmany_64(maxblocks, minnoderecs);
>>  	}
>>  	mp->m_bm_maxlevels[whichfork] = level;
>>  	ASSERT(mp->m_bm_maxlevels[whichfork] <= xfs_bmbt_maxlevels_ondisk());
>
> Hmmmm. Shouldn't this be rolled up into the earlier patch that
> converts a seperate part of this function to use howmany_64()?
> That was done in "[PATCH V8 07/19] xfs: Promote xfs_extnum_t and
> xfs_aextnum_t to 64 and 32-bits respectively" - it seems to me like
> this should definitely be part of the type size extension rather
> than a stand-alone change.

The goal of the earlier patch was to extend the basic data types underlying
xfs_extnum_t and xfs_aextnum_t. The "maxblocks" data type change was included
in a separate patch since the change does not fall into the purview of
extending either xfs_extnum_t or xfs_aextnum_t.

-- 
chandan

Re: [PATCH V8 14/19] xfs: Introduce per-inode 64-bit extent counters

[Chandan Babu R]

On 25 Mar 2022 at 03:23, Dave Chinner wrote:
> On Mon, Mar 21, 2022 at 10:47:45AM +0530, Chandan Babu R wrote:
>> This commit introduces new fields in the on-disk inode format to support
>> 64-bit data fork extent counters and 32-bit attribute fork extent
>> counters. The new fields will be used only when an inode has
>> XFS_DIFLAG2_NREXT64 flag set. Otherwise we continue to use the regular 32-bit
>> data fork extent counters and 16-bit attribute fork extent counters.
>> 
>> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> Suggested-by: Dave Chinner <dchinner@redhat.com>
>
> Looks good, minor nits below.
>
> Reviewed-by: Dave Chinner <dchinner@redhat.com>
>
>> +STATIC int
>> +xlog_dinode_verify_extent_counts(
>> +	struct xfs_mount	*mp,
>> +	struct xfs_log_dinode	*ldip)
>> +{
>> +	xfs_extnum_t		nextents;
>> +	xfs_aextnum_t		anextents;
>> +
>> +	if (xfs_log_dinode_has_large_extent_counts(ldip)) {
>> +		if (!xfs_has_large_extent_counts(mp) ||
>> +		    (ldip->di_nrext64_pad != 0)) {
>> +			XFS_CORRUPTION_ERROR(
>> +				"Bad log dinode large extent count format",
>> +				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
>> +			xfs_alert(mp,
>> +				"Bad inode 0x%llx, nrext64 %d, padding 0x%x",
>
> "large extent counts" rather than "nrext64"?
>
>> +				ldip->di_ino, xfs_has_large_extent_counts(mp),
>> +				ldip->di_nrext64_pad);
>> +			return -EFSCORRUPTED;
>> +		}
>> +
>> +		nextents = ldip->di_big_nextents;
>> +		anextents = ldip->di_big_anextents;
>> +	} else {
>> +		if (ldip->di_version == 3 && ldip->di_v3_pad != 0) {
>> +			XFS_CORRUPTION_ERROR(
>> +				"Bad log dinode di_v3_pad",
>> +				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
>> +			xfs_alert(mp,
>> +				"Bad inode 0x%llx, di_v3_pad %llu",
>> +				ldip->di_ino, ldip->di_v3_pad);
>
> the padding should be dumped as a hexadecimal value - it's much
> easier to see the bit corruption patterns in hex dumps than decimal
> output.
>

I agree. I will make the suggested change.

>> +			return -EFSCORRUPTED;
>> +		}
>> +
>> +		nextents = ldip->di_nextents;
>> +		anextents = ldip->di_anextents;
>> +	}
>> +
>> +	if (unlikely(nextents + anextents > ldip->di_nblocks)) {
>> +		XFS_CORRUPTION_ERROR("Bad log dinode extent counts",
>> +				XFS_ERRLEVEL_LOW, mp, ldip, sizeof(*ldip));
>> +		xfs_alert(mp,
>> +			"Bad inode 0x%llx, nextents 0x%llx, anextents 0x%x, nblocks 0x%llx",
>> +			ldip->di_ino, nextents, anextents, ldip->di_nblocks);
>> +		return -EFSCORRUPTED;
>
> Maybe should indicate large extent count status here, too.
>

Yes, I think it will make it easier to determine the inode fields of the two
extent counts.

-- 
chandan

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Chandan Babu R]

On 25 Mar 2022 at 03:44, Dave Chinner wrote:
> On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
>> The maximum file size that can be represented by the data fork extent counter
>> in the worst case occurs when all extents are 1 block in length and each block
>> is 1KB in size.
>> 
>> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
>> 1KB sized blocks, a file can reach upto,
>> (2^31) * 1KB = 2TB
>> 
>> This is much larger than the theoretical maximum size of a directory
>> i.e. 32GB * 3 = 96GB.
>> 
>> Since a directory's inode can never overflow its data fork extent counter,
>> this commit replaces checking the return value of
>> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
>
> I'd really prefer that we don't add noise like this to a bunch of
> call sites.  If directories can't overflow the extent count in
> normal operation, then why are we even calling
> xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
> be a sign of an inode corruption, and we should have flagged that
> long before we do an operation that might overflow the extent count.
>
> So, really, I think you should document the directory size
> constraints at the site where we define all the large extent count
> values in xfs_format.h, remove the xfs_iext_count_may_overflow()
> checks from the directory code and replace them with a simple inode
> verifier check that we haven't got more than 100GB worth of
> individual extents in the data fork for directory inodes....
>
> Then all this directory specific "can't possibly overflow" overflow
> checks can go away completely.  The best code is no code :)

I had retained the directory extent count overflow checks for the sake of
completeness i.e. The code checks for overflow before every fs operation that
could cause extent count to increase. However, I think your suggestion makes
more sense. I will include this change in the next version of the patchset.

-- 
chandan

Re: [PATCH V8 16/19] xfs: Conditionally upgrade existing inodes to use large extent counters

[Chandan Babu R]

On 25 Mar 2022 at 03:58, Dave Chinner wrote:
> On Mon, Mar 21, 2022 at 10:47:47AM +0530, Chandan Babu R wrote:
>> This commit enables upgrading existing inodes to use large extent counters
>> provided that underlying filesystem's superblock has large extent counter
>> feature enabled.
>> 
>> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
>> ---
>>  fs/xfs/libxfs/xfs_attr.c       |  9 ++++++-
>>  fs/xfs/libxfs/xfs_bmap.c       | 10 ++++++--
>>  fs/xfs/libxfs/xfs_inode_fork.c | 27 +++++++++++++++++++++
>>  fs/xfs/libxfs/xfs_inode_fork.h |  2 ++
>>  fs/xfs/xfs_bmap_item.c         |  8 ++++++-
>>  fs/xfs/xfs_bmap_util.c         | 43 ++++++++++++++++++++++++++++++----
>>  fs/xfs/xfs_dquot.c             |  9 ++++++-
>>  fs/xfs/xfs_iomap.c             | 17 ++++++++++++--
>>  fs/xfs/xfs_reflink.c           | 17 ++++++++++++--
>>  fs/xfs/xfs_rtalloc.c           |  9 ++++++-
>>  10 files changed, 136 insertions(+), 15 deletions(-)
>> 
>> diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c
>> index 23523b802539..6e56aa17fd82 100644
>> --- a/fs/xfs/libxfs/xfs_attr.c
>> +++ b/fs/xfs/libxfs/xfs_attr.c
>> @@ -776,8 +776,15 @@ xfs_attr_set(
>>  	if (args->value || xfs_inode_hasattr(dp)) {
>>  		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
>>  				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
>> -		if (error)
>> +		if (error && error != -EFBIG)
>>  			goto out_trans_cancel;
>> +
>> +		if (error == -EFBIG) {
>> +			error = xfs_iext_count_upgrade(args->trans, dp,
>> +					XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
>> +			if (error)
>> +				goto out_trans_cancel;
>> +		}
>
> Neater and more compact to do this by checking explicitly for
> -EFBIG:
>
> 		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
> 				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
> 		if (error == -EFBIG)
> 			error = xfs_iext_count_upgrade(args->trans, dp,
> 					XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
> 		if (error)
> 			goto out_trans_cancel;
> 	}

I agree.

>>  
>>  	error = xfs_attr_lookup(args);
>> diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
>> index 5a089674c666..0cb915bf8285 100644
>> --- a/fs/xfs/libxfs/xfs_bmap.c
>> +++ b/fs/xfs/libxfs/xfs_bmap.c
>> @@ -4524,13 +4524,19 @@ xfs_bmapi_convert_delalloc(
>>  		return error;
>>  
>>  	xfs_ilock(ip, XFS_ILOCK_EXCL);
>> +	xfs_trans_ijoin(tp, ip, 0);
>>  
>>  	error = xfs_iext_count_may_overflow(ip, whichfork,
>>  			XFS_IEXT_ADD_NOSPLIT_CNT);
>> -	if (error)
>> +	if (error && error != -EFBIG)
>>  		goto out_trans_cancel;
>>  
>> -	xfs_trans_ijoin(tp, ip, 0);
>> +	if (error == -EFBIG) {
>> +		error = xfs_iext_count_upgrade(tp, ip,
>> +				XFS_IEXT_ADD_NOSPLIT_CNT);
>> +		if (error)
>> +			goto out_trans_cancel;
>> +	}
>>  
>>  	if (!xfs_iext_lookup_extent(ip, ifp, offset_fsb, &bma.icur, &bma.got) ||
>>  	    bma.got.br_startoff > offset_fsb) {
>> diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
>> index bb5d841aac58..aff9242db829 100644
>> --- a/fs/xfs/libxfs/xfs_inode_fork.c
>> +++ b/fs/xfs/libxfs/xfs_inode_fork.c
>> @@ -756,3 +756,30 @@ xfs_iext_count_may_overflow(
>>  
>>  	return 0;
>>  }
>> +
>> +int
>> +xfs_iext_count_upgrade(
>> +	struct xfs_trans	*tp,
>> +	struct xfs_inode	*ip,
>> +	int			nr_to_add)
>
> nr_to_add can only be positive, so should be unsigned.
>

I will change this.

>> +{
>> +	if (!xfs_has_large_extent_counts(ip->i_mount) ||
>> +	    (ip->i_diflags2 & XFS_DIFLAG2_NREXT64) ||
>> +	    XFS_TEST_ERROR(false, ip->i_mount, XFS_ERRTAG_REDUCE_MAX_IEXTENTS))
>> +		return -EFBIG;
>> +
>> +	ip->i_diflags2 |= XFS_DIFLAG2_NREXT64;
>> +	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
>> +
>> +	/*
>> +	 * The value of nr_to_add cannot be larger than 2^17
>> +	 *
>> +	 * - XFS_MAX_EXTCNT_ATTR_FORK_LARGE - XFS_MAX_EXTCNT_ATTR_FORK_SMALL
>> +	 *   i.e. 2^32 - 2^15
>> +	 * - XFS_MAX_EXTCNT_DATA_FORK_LARGE - XFS_MAX_EXTCNT_DATA_FORK_SMALL
>> +	 *   i.e. 2^48 - 2^31
>> +	 */
>> +	ASSERT(nr_to_add <= (1 << 17));
>
> That's a comment for the function head and/or the format
> documentation in xfs_format.h, not hidden in the code itself as a
> magic number. i.e. it is a format definition because it is bound by
> on-disk format constants, not by code constratints. Hence this
> should probably be defined in xfs_format.h alongside the large/small
> extent counts, such as:
>
> #define XFS_MAX_EXTCNT_UPGRADE_NR	\
> 	min(XFS_MAX_EXTCNT_ATTR_FORK_LARGE - XFS_MAX_EXTCNT_ATTR_FORK_SMALL, \
> 	    XFS_MAX_EXTCNT_DATA_FORK_LARGE - XFS_MAX_EXTCNT_DATA_FORK_SMALL)
>
> And the ASSERT checking the incoming nr_to_add placed right at the
> top of the function because the assert then documents API
> constraints and always catches violations of them.
>

Ok. I will implement the change suggested above.

-- 
chandan

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Chandan Babu R]

On 25 Mar 2022 at 11:32, Chandan Babu R wrote:
> On 25 Mar 2022 at 03:44, Dave Chinner wrote:
>> On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
>>> The maximum file size that can be represented by the data fork extent counter
>>> in the worst case occurs when all extents are 1 block in length and each block
>>> is 1KB in size.
>>> 
>>> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
>>> 1KB sized blocks, a file can reach upto,
>>> (2^31) * 1KB = 2TB
>>> 
>>> This is much larger than the theoretical maximum size of a directory
>>> i.e. 32GB * 3 = 96GB.
>>> 
>>> Since a directory's inode can never overflow its data fork extent counter,
>>> this commit replaces checking the return value of
>>> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
>>
>> I'd really prefer that we don't add noise like this to a bunch of
>> call sites.  If directories can't overflow the extent count in
>> normal operation, then why are we even calling
>> xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
>> be a sign of an inode corruption, and we should have flagged that
>> long before we do an operation that might overflow the extent count.
>>
>> So, really, I think you should document the directory size
>> constraints at the site where we define all the large extent count
>> values in xfs_format.h, remove the xfs_iext_count_may_overflow()
>> checks from the directory code and replace them with a simple inode
>> verifier check that we haven't got more than 100GB worth of
>> individual extents in the data fork for directory inodes....
>>
>> Then all this directory specific "can't possibly overflow" overflow
>> checks can go away completely.  The best code is no code :)
>
> I had retained the directory extent count overflow checks for the sake of
> completeness i.e. The code checks for overflow before every fs operation that
> could cause extent count to increase. However, I think your suggestion makes
> more sense. I will include this change in the next version of the patchset.

Also, Removing directory extent counter overflow checks would also mean that
the test xfs/533 (Verify that XFS does not cause inode fork's extent count to
overflow when adding/removing directory entries) has to be removed as well. I
will post a patch to do the same if no objections are raised.

-- 
chandan

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Chandan Babu R]

On 25 Mar 2022 at 03:44, Dave Chinner wrote:
> On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
>> The maximum file size that can be represented by the data fork extent counter
>> in the worst case occurs when all extents are 1 block in length and each block
>> is 1KB in size.
>> 
>> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
>> 1KB sized blocks, a file can reach upto,
>> (2^31) * 1KB = 2TB
>> 
>> This is much larger than the theoretical maximum size of a directory
>> i.e. 32GB * 3 = 96GB.
>> 
>> Since a directory's inode can never overflow its data fork extent counter,
>> this commit replaces checking the return value of
>> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
>
> I'd really prefer that we don't add noise like this to a bunch of
> call sites.  If directories can't overflow the extent count in
> normal operation, then why are we even calling
> xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
> be a sign of an inode corruption, and we should have flagged that
> long before we do an operation that might overflow the extent count.
>
> So, really, I think you should document the directory size
> constraints at the site where we define all the large extent count
> values in xfs_format.h, remove the xfs_iext_count_may_overflow()
> checks from the directory code and replace them with a simple inode
> verifier check that we haven't got more than 100GB worth of
> individual extents in the data fork for directory inodes....

I don't think that we could trivially verify if the extents in a directory's
data fork add up to more than 96GB.

xfs_dinode->di_size tracks the size of XFS_DIR2_DATA_SPACE. This also includes
holes that could be created by freeing directory entries in a single directory
block. Also, there is no easy method to determine the space occupied by
XFS_DIR2_LEAF_SPACE and XFS_DIR2_FREE_SPACE segments of a directory.

May be the following can be added to xfs_dinode_verify(),

	if (S_ISDIR(mode) && ((xfs_dinode->di_size + 2 * 32GB) > 96GB))
    		return __this_address

as a rough check against possible directory inode corruption.

-- 
chandan

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Dave Chinner]

On Tue, Mar 29, 2022 at 10:52:04AM +0530, Chandan Babu R wrote:
> On 25 Mar 2022 at 03:44, Dave Chinner wrote:
> > On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
> >> The maximum file size that can be represented by the data fork extent counter
> >> in the worst case occurs when all extents are 1 block in length and each block
> >> is 1KB in size.
> >> 
> >> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
> >> 1KB sized blocks, a file can reach upto,
> >> (2^31) * 1KB = 2TB
> >> 
> >> This is much larger than the theoretical maximum size of a directory
> >> i.e. 32GB * 3 = 96GB.
> >> 
> >> Since a directory's inode can never overflow its data fork extent counter,
> >> this commit replaces checking the return value of
> >> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
> >
> > I'd really prefer that we don't add noise like this to a bunch of
> > call sites.  If directories can't overflow the extent count in
> > normal operation, then why are we even calling
> > xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
> > be a sign of an inode corruption, and we should have flagged that
> > long before we do an operation that might overflow the extent count.
> >
> > So, really, I think you should document the directory size
> > constraints at the site where we define all the large extent count
> > values in xfs_format.h, remove the xfs_iext_count_may_overflow()
> > checks from the directory code and replace them with a simple inode
> > verifier check that we haven't got more than 100GB worth of
> > individual extents in the data fork for directory inodes....
> 
> I don't think that we could trivially verify if the extents in a directory's
> data fork add up to more than 96GB.

dip->di_nextents tells us how many extents there are in the data
fork, we know what the block size of the filesystem is, so it should
be pretty easy to calculate a maximum extent count for 96GB of
space. i.e. absolute maximum valid dir data fork extent count
is (96GB / blocksize).

> 
> xfs_dinode->di_size tracks the size of XFS_DIR2_DATA_SPACE. This also includes
> holes that could be created by freeing directory entries in a single directory
> block. Also, there is no easy method to determine the space occupied by
> XFS_DIR2_LEAF_SPACE and XFS_DIR2_FREE_SPACE segments of a directory.

Sure there is. We do this sort of calc for things like transaction
reservations via definitions like XFS_DA_NODE_MAXDEPTH. That tells us
immediately how many blocks can be in the XFS_DIR2_LEAF_SPACE
segement....

We also know the maximum number of individual directory blocks in
the 32GB segment (fixed at 32GB / dir block size), so the free space
array is also a fixed size at (32GB / dir block size / free space
entries per block).

It's easy to just use (96GB / block size) and that will catch most
corruptions with no risk of a false positive detection, but we could
quite easily refine this to something like:

data	(32GB +				
leaf	 btree blocks(XFS_DA_NODE_MAXDEPTH) +
freesp	 (32GB / free space records per block))
frags					/ filesystem block size

> May be the following can be added to xfs_dinode_verify(),
> 
> 	if (S_ISDIR(mode) && ((xfs_dinode->di_size + 2 * 32GB) > 96GB))
>     		return __this_address

That doesn't validate that the on disk or in-memory di_nextents
value is withing the known valid range or not. We can do that
directly (as per above), so we shouldn't need a hueristic like this.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Darrick J. Wong]

On Tue, Mar 29, 2022 at 05:23:40PM +1100, Dave Chinner wrote:
> On Tue, Mar 29, 2022 at 10:52:04AM +0530, Chandan Babu R wrote:
> > On 25 Mar 2022 at 03:44, Dave Chinner wrote:
> > > On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
> > >> The maximum file size that can be represented by the data fork extent counter
> > >> in the worst case occurs when all extents are 1 block in length and each block
> > >> is 1KB in size.
> > >> 
> > >> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
> > >> 1KB sized blocks, a file can reach upto,
> > >> (2^31) * 1KB = 2TB
> > >> 
> > >> This is much larger than the theoretical maximum size of a directory
> > >> i.e. 32GB * 3 = 96GB.
> > >> 
> > >> Since a directory's inode can never overflow its data fork extent counter,
> > >> this commit replaces checking the return value of
> > >> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
> > >
> > > I'd really prefer that we don't add noise like this to a bunch of
> > > call sites.  If directories can't overflow the extent count in
> > > normal operation, then why are we even calling
> > > xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
> > > be a sign of an inode corruption, and we should have flagged that
> > > long before we do an operation that might overflow the extent count.
> > >
> > > So, really, I think you should document the directory size
> > > constraints at the site where we define all the large extent count
> > > values in xfs_format.h, remove the xfs_iext_count_may_overflow()
> > > checks from the directory code and replace them with a simple inode
> > > verifier check that we haven't got more than 100GB worth of
> > > individual extents in the data fork for directory inodes....
> > 
> > I don't think that we could trivially verify if the extents in a directory's
> > data fork add up to more than 96GB.
> 
> dip->di_nextents tells us how many extents there are in the data
> fork, we know what the block size of the filesystem is, so it should
> be pretty easy to calculate a maximum extent count for 96GB of
> space. i.e. absolute maximum valid dir data fork extent count
> is (96GB / blocksize).
> 
> > 
> > xfs_dinode->di_size tracks the size of XFS_DIR2_DATA_SPACE. This also includes
> > holes that could be created by freeing directory entries in a single directory
> > block. Also, there is no easy method to determine the space occupied by
> > XFS_DIR2_LEAF_SPACE and XFS_DIR2_FREE_SPACE segments of a directory.
> 
> Sure there is. We do this sort of calc for things like transaction
> reservations via definitions like XFS_DA_NODE_MAXDEPTH. That tells us

Hmmm.  Seeing as I just replaced XFS_BTREE_MAXLEVELS with dynamic limits
set for each filesytem, is XFS_DA_NODE_MAXDEPTH even appropriate for
modern filesystems?  We're about to start allowing far more extended
attributes in the form of parent pointers, so we should be careful about
this.

For a directory, there can be at most 32GB of directory entries, so the
maximum number of directory entries is...

32GB / (directory block size) * (max entries per dir block)

The dabtree stores (u32 hash, u32 offset) records, so I guess it
wouldn't be so hard to compute the number of blocks needed for each node
level until we only need one block, and that's our real
XFS_DA_NODE_MAXEPTH.

But then the second question is: what's the maximum height of a dabtree
that indexes an xattr structure?  I don't think there's any maximum
limit within XFS on the number of attrs you can set on a file, is there?
At least until you hit the iext_max_count check.  I think the VFS
institutes its own limit of 64k for the llistxattr buffer, but that's
about all I can think of.

I suppose right now the xattr structure can't grow larger than 2^(16+21)
blocks in size, which is 2^49 bytes, but that's a mix of attr leaves and
dabtree blocks, unlike directories, right?

> immediately how many blocks can be in the XFS_DIR2_LEAF_SPACE
> segement....
> 
> We also know the maximum number of individual directory blocks in
> the 32GB segment (fixed at 32GB / dir block size), so the free space
> array is also a fixed size at (32GB / dir block size / free space
> entries per block).
> 
> It's easy to just use (96GB / block size) and that will catch most
> corruptions with no risk of a false positive detection, but we could
> quite easily refine this to something like:
> 
> data	(32GB +				
> leaf	 btree blocks(XFS_DA_NODE_MAXDEPTH) +
> freesp	 (32GB / free space records per block))
> frags					/ filesystem block size

I think we ought to do a more careful study of XFS_DA_NODE_MAXDEPTH, but
it could become more involved than we think.  In the interest of keeping
this series moving, can we start with a new verifier check that
(di_nextents < that formula from above) and then refine that based on
whatever improvements we may or may not come up with for
XFS_DA_NODE_MAXDEPTH?

> 
> > May be the following can be added to xfs_dinode_verify(),
> > 
> > 	if (S_ISDIR(mode) && ((xfs_dinode->di_size + 2 * 32GB) > 96GB))
> >     		return __this_address
> 
> That doesn't validate that the on disk or in-memory di_nextents
> value is withing the known valid range or not. We can do that
> directly (as per above), so we shouldn't need a hueristic like this.

Indeed, inode size is not a good proxy variable for extent count.

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Chandan Babu R]

On 30 Mar 2022 at 09:13, Darrick J. Wong wrote:
> On Tue, Mar 29, 2022 at 05:23:40PM +1100, Dave Chinner wrote:
>> On Tue, Mar 29, 2022 at 10:52:04AM +0530, Chandan Babu R wrote:
>> > On 25 Mar 2022 at 03:44, Dave Chinner wrote:
>> > > On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
>> > >> The maximum file size that can be represented by the data fork extent counter
>> > >> in the worst case occurs when all extents are 1 block in length and each block
>> > >> is 1KB in size.
>> > >> 
>> > >> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
>> > >> 1KB sized blocks, a file can reach upto,
>> > >> (2^31) * 1KB = 2TB
>> > >> 
>> > >> This is much larger than the theoretical maximum size of a directory
>> > >> i.e. 32GB * 3 = 96GB.
>> > >> 
>> > >> Since a directory's inode can never overflow its data fork extent counter,
>> > >> this commit replaces checking the return value of
>> > >> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
>> > >
>> > > I'd really prefer that we don't add noise like this to a bunch of
>> > > call sites.  If directories can't overflow the extent count in
>> > > normal operation, then why are we even calling
>> > > xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
>> > > be a sign of an inode corruption, and we should have flagged that
>> > > long before we do an operation that might overflow the extent count.
>> > >
>> > > So, really, I think you should document the directory size
>> > > constraints at the site where we define all the large extent count
>> > > values in xfs_format.h, remove the xfs_iext_count_may_overflow()
>> > > checks from the directory code and replace them with a simple inode
>> > > verifier check that we haven't got more than 100GB worth of
>> > > individual extents in the data fork for directory inodes....
>> > 
>> > I don't think that we could trivially verify if the extents in a directory's
>> > data fork add up to more than 96GB.
>> 
>> dip->di_nextents tells us how many extents there are in the data
>> fork, we know what the block size of the filesystem is, so it should
>> be pretty easy to calculate a maximum extent count for 96GB of
>> space. i.e. absolute maximum valid dir data fork extent count
>> is (96GB / blocksize).
>> 
>> > 
>> > xfs_dinode->di_size tracks the size of XFS_DIR2_DATA_SPACE. This also includes
>> > holes that could be created by freeing directory entries in a single directory
>> > block. Also, there is no easy method to determine the space occupied by
>> > XFS_DIR2_LEAF_SPACE and XFS_DIR2_FREE_SPACE segments of a directory.
>> 
>> Sure there is. We do this sort of calc for things like transaction
>> reservations via definitions like XFS_DA_NODE_MAXDEPTH. That tells us
>
> Hmmm.  Seeing as I just replaced XFS_BTREE_MAXLEVELS with dynamic limits
> set for each filesytem, is XFS_DA_NODE_MAXDEPTH even appropriate for
> modern filesystems?  We're about to start allowing far more extended
> attributes in the form of parent pointers, so we should be careful about
> this.
>
> For a directory, there can be at most 32GB of directory entries, so the
> maximum number of directory entries is...
>
> 32GB / (directory block size) * (max entries per dir block)
>
> The dabtree stores (u32 hash, u32 offset) records, so I guess it
> wouldn't be so hard to compute the number of blocks needed for each node
> level until we only need one block, and that's our real
> XFS_DA_NODE_MAXEPTH.
>
> But then the second question is: what's the maximum height of a dabtree
> that indexes an xattr structure?  I don't think there's any maximum
> limit within XFS on the number of attrs you can set on a file, is there?
> At least until you hit the iext_max_count check.  I think the VFS
> institutes its own limit of 64k for the llistxattr buffer, but that's
> about all I can think of.
>
> I suppose right now the xattr structure can't grow larger than 2^(16+21)
> blocks in size, which is 2^49 bytes, but that's a mix of attr leaves and
> dabtree blocks, unlike directories, right?
>
>> immediately how many blocks can be in the XFS_DIR2_LEAF_SPACE
>> segement....
>> 
>> We also know the maximum number of individual directory blocks in
>> the 32GB segment (fixed at 32GB / dir block size), so the free space
>> array is also a fixed size at (32GB / dir block size / free space
>> entries per block).
>> 
>> It's easy to just use (96GB / block size) and that will catch most
>> corruptions with no risk of a false positive detection, but we could
>> quite easily refine this to something like:
>> 
>> data	(32GB +				
>> leaf	 btree blocks(XFS_DA_NODE_MAXDEPTH) +
>> freesp	 (32GB / free space records per block))
>> frags					/ filesystem block size
>
> I think we ought to do a more careful study of XFS_DA_NODE_MAXDEPTH, but
> it could become more involved than we think.  In the interest of keeping
> this series moving, can we start with a new verifier check that
> (di_nextents < that formula from above) and then refine that based on
> whatever improvements we may or may not come up with for
> XFS_DA_NODE_MAXDEPTH?
>

Are you referring to (dip->di_nextents <= (96GB / blocksize)) check?

-- 
chandan

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Darrick J. Wong]

On Wed, Mar 30, 2022 at 09:09:00PM +0530, Chandan Babu R wrote:
> On 30 Mar 2022 at 09:13, Darrick J. Wong wrote:
> > On Tue, Mar 29, 2022 at 05:23:40PM +1100, Dave Chinner wrote:
> >> On Tue, Mar 29, 2022 at 10:52:04AM +0530, Chandan Babu R wrote:
> >> > On 25 Mar 2022 at 03:44, Dave Chinner wrote:
> >> > > On Mon, Mar 21, 2022 at 10:47:46AM +0530, Chandan Babu R wrote:
> >> > >> The maximum file size that can be represented by the data fork extent counter
> >> > >> in the worst case occurs when all extents are 1 block in length and each block
> >> > >> is 1KB in size.
> >> > >> 
> >> > >> With XFS_MAX_EXTCNT_DATA_FORK_SMALL representing maximum extent count and with
> >> > >> 1KB sized blocks, a file can reach upto,
> >> > >> (2^31) * 1KB = 2TB
> >> > >> 
> >> > >> This is much larger than the theoretical maximum size of a directory
> >> > >> i.e. 32GB * 3 = 96GB.
> >> > >> 
> >> > >> Since a directory's inode can never overflow its data fork extent counter,
> >> > >> this commit replaces checking the return value of
> >> > >> xfs_iext_count_may_overflow() with calls to ASSERT(error == 0).
> >> > >
> >> > > I'd really prefer that we don't add noise like this to a bunch of
> >> > > call sites.  If directories can't overflow the extent count in
> >> > > normal operation, then why are we even calling
> >> > > xfs_iext_count_may_overflow() in these paths? i.e. an overflow would
> >> > > be a sign of an inode corruption, and we should have flagged that
> >> > > long before we do an operation that might overflow the extent count.
> >> > >
> >> > > So, really, I think you should document the directory size
> >> > > constraints at the site where we define all the large extent count
> >> > > values in xfs_format.h, remove the xfs_iext_count_may_overflow()
> >> > > checks from the directory code and replace them with a simple inode
> >> > > verifier check that we haven't got more than 100GB worth of
> >> > > individual extents in the data fork for directory inodes....
> >> > 
> >> > I don't think that we could trivially verify if the extents in a directory's
> >> > data fork add up to more than 96GB.
> >> 
> >> dip->di_nextents tells us how many extents there are in the data
> >> fork, we know what the block size of the filesystem is, so it should
> >> be pretty easy to calculate a maximum extent count for 96GB of
> >> space. i.e. absolute maximum valid dir data fork extent count
> >> is (96GB / blocksize).
> >> 
> >> > 
> >> > xfs_dinode->di_size tracks the size of XFS_DIR2_DATA_SPACE. This also includes
> >> > holes that could be created by freeing directory entries in a single directory
> >> > block. Also, there is no easy method to determine the space occupied by
> >> > XFS_DIR2_LEAF_SPACE and XFS_DIR2_FREE_SPACE segments of a directory.
> >> 
> >> Sure there is. We do this sort of calc for things like transaction
> >> reservations via definitions like XFS_DA_NODE_MAXDEPTH. That tells us
> >
> > Hmmm.  Seeing as I just replaced XFS_BTREE_MAXLEVELS with dynamic limits
> > set for each filesytem, is XFS_DA_NODE_MAXDEPTH even appropriate for
> > modern filesystems?  We're about to start allowing far more extended
> > attributes in the form of parent pointers, so we should be careful about
> > this.
> >
> > For a directory, there can be at most 32GB of directory entries, so the
> > maximum number of directory entries is...
> >
> > 32GB / (directory block size) * (max entries per dir block)
> >
> > The dabtree stores (u32 hash, u32 offset) records, so I guess it
> > wouldn't be so hard to compute the number of blocks needed for each node
> > level until we only need one block, and that's our real
> > XFS_DA_NODE_MAXEPTH.
> >
> > But then the second question is: what's the maximum height of a dabtree
> > that indexes an xattr structure?  I don't think there's any maximum
> > limit within XFS on the number of attrs you can set on a file, is there?
> > At least until you hit the iext_max_count check.  I think the VFS
> > institutes its own limit of 64k for the llistxattr buffer, but that's
> > about all I can think of.
> >
> > I suppose right now the xattr structure can't grow larger than 2^(16+21)
> > blocks in size, which is 2^49 bytes, but that's a mix of attr leaves and
> > dabtree blocks, unlike directories, right?
> >
> >> immediately how many blocks can be in the XFS_DIR2_LEAF_SPACE
> >> segement....
> >> 
> >> We also know the maximum number of individual directory blocks in
> >> the 32GB segment (fixed at 32GB / dir block size), so the free space
> >> array is also a fixed size at (32GB / dir block size / free space
> >> entries per block).
> >> 
> >> It's easy to just use (96GB / block size) and that will catch most
> >> corruptions with no risk of a false positive detection, but we could
> >> quite easily refine this to something like:
> >> 
> >> data	(32GB +				
> >> leaf	 btree blocks(XFS_DA_NODE_MAXDEPTH) +
> >> freesp	 (32GB / free space records per block))
> >> frags					/ filesystem block size
> >
> > I think we ought to do a more careful study of XFS_DA_NODE_MAXDEPTH, but
> > it could become more involved than we think.  In the interest of keeping
> > this series moving, can we start with a new verifier check that
> > (di_nextents < that formula from above) and then refine that based on
> > whatever improvements we may or may not come up with for
> > XFS_DA_NODE_MAXDEPTH?
> >
> 
> Are you referring to (dip->di_nextents <= (96GB / blocksize)) check?

Yup.

--D

> -- 
> chandan

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Dave Chinner]

On Tue, Mar 29, 2022 at 08:43:33PM -0700, Darrick J. Wong wrote:
> But then the second question is: what's the maximum height of a dabtree
> that indexes an xattr structure?  I don't think there's any maximum
> limit within XFS on the number of attrs you can set on a file, is there?

Nope. But the attr btree is a different beast to the dirv2
structure - it's a hashed index btree with the xattr records in the
leaf, it's not an index of a fixed max size external data structure.

> At least until you hit the iext_max_count check.  I think the VFS
> institutes its own limit of 64k for the llistxattr buffer, but that's
> about all I can think of.


> I suppose right now the xattr structure can't grow larger than 2^(16+21)
> blocks in size, which is 2^49 bytes, but that's a mix of attr leaves and
> dabtree blocks, unlike directories, right?

Yes. ALso remote xattr blocks, which aren't stored in the dabtree
at all, but are indexed by the attr fork BMBT address space.

So I think for XFS_DA_NODE_MAXDEPTH = 5, the xattr tree with a 4kB
block size (minimum allowed, IIRC), we can index approximately
(500^4) individual xattrs in the btree index (assuming 4 levels of
index and 1 level of leaf nodes containing xattrs).

My math calculates that to be about 62.5 billion xattrs before we
run out of index space in a 5 level tree with a 4kB block size.
Hence I suspect we'll run out of attr fork extents even on a 32 bit
extent count before we run out of xattr index space.

Also, 62.5 billion xattrs is more links than we can have to a single
inode (4billion), so we're not going to exhaust the xattr index
space just with parent pointers...

> > immediately how many blocks can be in the XFS_DIR2_LEAF_SPACE
> > segement....
> > 
> > We also know the maximum number of individual directory blocks in
> > the 32GB segment (fixed at 32GB / dir block size), so the free space
> > array is also a fixed size at (32GB / dir block size / free space
> > entries per block).
> > 
> > It's easy to just use (96GB / block size) and that will catch most
> > corruptions with no risk of a false positive detection, but we could
> > quite easily refine this to something like:
> > 
> > data	(32GB +				
> > leaf	 btree blocks(XFS_DA_NODE_MAXDEPTH) +
> > freesp	 (32GB / free space records per block))
> > frags					/ filesystem block size
> 
> I think we ought to do a more careful study of XFS_DA_NODE_MAXDEPTH,

*nod*

ISTR that Chandan had already done some of this when he
characterised the attr fork btree behaviour w.r.t. suitabililty for
large numbers of parent pointers before starting this extent count
work.

> but
> it could become more involved than we think.  In the interest of keeping
> this series moving, can we start with a new verifier check that
> (di_nextents < that formula from above) and then refine that based on
> whatever improvements we may or may not come up with for
> XFS_DA_NODE_MAXDEPTH?

Yup, that sounds like a good plan.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH V8 15/19] xfs: Directory's data fork extent counter can never overflow

[Chandan Babu R]

On 01 Apr 2022 at 06:57, Dave Chinner wrote:
> On Tue, Mar 29, 2022 at 08:43:33PM -0700, Darrick J. Wong wrote:
>> But then the second question is: what's the maximum height of a dabtree
>> that indexes an xattr structure?  I don't think there's any maximum
>> limit within XFS on the number of attrs you can set on a file, is there?
>
> Nope. But the attr btree is a different beast to the dirv2
> structure - it's a hashed index btree with the xattr records in the
> leaf, it's not an index of a fixed max size external data structure.
>
>> At least until you hit the iext_max_count check.  I think the VFS
>> institutes its own limit of 64k for the llistxattr buffer, but that's
>> about all I can think of.
>
>
>> I suppose right now the xattr structure can't grow larger than 2^(16+21)
>> blocks in size, which is 2^49 bytes, but that's a mix of attr leaves and
>> dabtree blocks, unlike directories, right?
>
> Yes. ALso remote xattr blocks, which aren't stored in the dabtree
> at all, but are indexed by the attr fork BMBT address space.
>
> So I think for XFS_DA_NODE_MAXDEPTH = 5, the xattr tree with a 4kB
> block size (minimum allowed, IIRC), we can index approximately
> (500^4) individual xattrs in the btree index (assuming 4 levels of
> index and 1 level of leaf nodes containing xattrs).
>
> My math calculates that to be about 62.5 billion xattrs before we
> run out of index space in a 5 level tree with a 4kB block size.
> Hence I suspect we'll run out of attr fork extents even on a 32 bit
> extent count before we run out of xattr index space.
>
> Also, 62.5 billion xattrs is more links than we can have to a single
> inode (4billion), so we're not going to exhaust the xattr index
> space just with parent pointers...
>

Attr dabtree blocks can be of 1k block size. My analysis of dabtree w.r.t
parent pointers had led me to this,

An inode could have 2^32 hard links with each link having 255 byte sized name.

- Size of one xattr
  Name length + Value length = 16 + 255 = 271 bytes
  16 comes from the size of the following structure,
      struct xfs_parent_name_rec {
      	__be64  p_ino;
      	__be32  p_gen;
      	__be32  p_diroffset;
      };

- sizeof(xfs_attr_leaf_hdr_t) = 32
- sizeof(xfs_attr_leaf_entry_t) = 8
- Number of entries in a 1k leaf block
  (1024 - sizeof(xfs_attr_leaf_hdr_t)) / (8 + 271)
  = (1024 - 32) / 279 = 992 / 279
  = floor(3.55)
  = 3

- Nr leaves = (2^32 / 3) * 3 (magicpct) = ~4.3 billion
- Nr entries per node
  = (1024 - sizeof(struct xfs_da3_node_hdr)) / sizeof(struct xfs_da_node_entry)
  = (1024 - 64) / 8
  = 120 entries

- Nr entries at level (n - 1) = 4.3 billion / 120 = 36 million
- Nr entries at level (n - 2) = 36 million / 120 = 300k
- Nr entries at level (n - 3) = 300k / 120 = 2.5k
- Nr entries at level (n - 4) = 2.5k / 120 = 20
- Nr entries at level (n - 5) = 20 / 120 = 1

Hence with 1024 block size, we could require 1 leaf level and 5 non-leaf
levels. This breaches the maximum height (i.e. XFS_DA_NODE_MAXDEPTH) allowed
for a dabtree.

>> > immediately how many blocks can be in the XFS_DIR2_LEAF_SPACE
>> > segement....
>> > 
>> > We also know the maximum number of individual directory blocks in
>> > the 32GB segment (fixed at 32GB / dir block size), so the free space
>> > array is also a fixed size at (32GB / dir block size / free space
>> > entries per block).
>> > 
>> > It's easy to just use (96GB / block size) and that will catch most
>> > corruptions with no risk of a false positive detection, but we could
>> > quite easily refine this to something like:
>> > 
>> > data	(32GB +				
>> > leaf	 btree blocks(XFS_DA_NODE_MAXDEPTH) +
>> > freesp	 (32GB / free space records per block))
>> > frags					/ filesystem block size
>> 
>> I think we ought to do a more careful study of XFS_DA_NODE_MAXDEPTH,
>
> *nod*
>
> ISTR that Chandan had already done some of this when he
> characterised the attr fork btree behaviour w.r.t. suitabililty for
> large numbers of parent pointers before starting this extent count
> work.
>

Sorry, I had forgotten about that. I searched through my notes and found that
we had reached the same conclusion w.r.t 4k block size. But with 1k block size,
we could breach the limit set by XFS_DA_NODE_MAXDEPTH.

-- 
chandan