[PATCH v2 00/10] xfsprogs-5.3: various fixes

[PATCH v2 00/10] xfsprogs-5.3: various fixes

[Darrick J. Wong]

Hi all,

Fix various problems in xfsprogs:

Patch 1 fixes libxfs-diff to handle files that are in libxfs/ in xfsprogs
but still in fs/xfs/ in the kernel.

Patch 2 moves the topology function declarations into a separate header
file since they're no longer libxcmd functionality.

Patch 3 teaches the xfs geometry wrapper function to try the v4 ioctl
calls if the v5 one fails.

Patches 4-7 document the ioctls introduced in 5.2.

Patch 8 removes the nearly empty "convert.h" file from db/ to eliminate
the possibility of confusion with include/convert.h.

Patch 9 adds a new "btheight" command to xfs_db so that we can calculate
the size of each level of a theoretical btree.

Patches 10-11 refactor db and repair to use inode geometry values instead
of recalculating them.

Patch 12 quiets down repair with regards to clearing reflink flags.

Patch 13 fixes repair rmap rebuilding transactions when the primary sb
has to be restored from a backup.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=xfsprogs-5.3-fixes

[PATCH 01/10] man: document the new v5 fs geometry ioctl structures

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Amend the fs geometry ioctl documentation to cover the new v5 structure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 man/man2/ioctl_xfs_fsop_geometry.2 |    8 ++++++++
 1 file changed, 8 insertions(+)


diff --git a/man/man2/ioctl_xfs_fsop_geometry.2 b/man/man2/ioctl_xfs_fsop_geometry.2
index 68e3387d..365bda8b 100644
--- a/man/man2/ioctl_xfs_fsop_geometry.2
+++ b/man/man2/ioctl_xfs_fsop_geometry.2
@@ -12,6 +12,8 @@ ioctl_xfs_fsop_geometry \- report XFS filesystem layout and features
 .PP
 .BI "int ioctl(int " fd ", XFS_IOC_FSOP_GEOMETRY, struct xfs_fsop_geom*" arg );
 .br
+.BI "int ioctl(int " fd ", XFS_IOC_FSOP_GEOMETRY_V4, struct xfs_fsop_geom_v4 *" arg );
+.br
 .BI "int ioctl(int " fd ", XFS_IOC_FSOP_GEOMETRY_V1, struct xfs_fsop_geom_v1 *" arg );
 .SH DESCRIPTION
 Report the details of an XFS filesystem layout, features, and other descriptive items.
@@ -43,6 +45,9 @@ struct xfs_fsop_geom {
 	/* struct xfs_fsop_geom_v1 stops here. */
 
 	__u32         logsunit;
+	/* struct xfs_fsop_geom_v4 stops here. */
+
+	__u64         reserved[18];
 };
 .fi
 .in
@@ -124,6 +129,9 @@ underlying log device, in filesystem blocks.
 This field is meaningful only if the flag
 .B  XFS_FSOP_GEOM_FLAGS_LOGV2
 is set.
+.PP
+.I reserved
+is set to zero.
 .SH FILESYSTEM FEATURE FLAGS
 Filesystem features are reported to userspace as a combination the following
 flags:

[PATCH 02/10] man: document new fs summary counter scrub command

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Update the scrub ioctl documentation to include the new fs summary
counter scrubber.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 man/man2/ioctl_xfs_scrub_metadata.2 |    5 +++++
 1 file changed, 5 insertions(+)


diff --git a/man/man2/ioctl_xfs_scrub_metadata.2 b/man/man2/ioctl_xfs_scrub_metadata.2
index 1e80ee71..046e3e36 100644
--- a/man/man2/ioctl_xfs_scrub_metadata.2
+++ b/man/man2/ioctl_xfs_scrub_metadata.2
@@ -159,6 +159,11 @@ corruption.
 .TP
 .B XFS_SCRUB_TYPE_PQUOTA
 Examine all user, group, or project quota records for corruption.
+
+.TP
+.B XFS_SCRUB_TYPE_FSCOUNTERS
+Examine all filesystem summary counters (free blocks, inode count, free inode
+count) for errors.
 .RE
 
 .PD 1

[PATCH 03/10] man: document the new allocation group geometry ioctl

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Document the new ioctl to describe an allocation group's geometry.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 man/man2/ioctl_xfs_ag_geometry.2 |   82 ++++++++++++++++++++++++++++++++++++++
 man/man3/xfsctl.3                |    6 +++
 2 files changed, 88 insertions(+)
 create mode 100644 man/man2/ioctl_xfs_ag_geometry.2


diff --git a/man/man2/ioctl_xfs_ag_geometry.2 b/man/man2/ioctl_xfs_ag_geometry.2
new file mode 100644
index 00000000..ddd54265
--- /dev/null
+++ b/man/man2/ioctl_xfs_ag_geometry.2
@@ -0,0 +1,82 @@
+.\" Copyright (c) 2019, Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(GPLv2+_DOC_FULL)
+.\" SPDX-License-Identifier: GPL-2.0+
+.\" %%%LICENSE_END
+.TH IOCTL-XFS-AG-GEOMETRY 2 2019-08-30 "XFS"
+.SH NAME
+ioctl_xfs_ag_geometry \- query XFS allocation group geometry information
+.SH SYNOPSIS
+.br
+.B #include <xfs/xfs_fs.h>
+.PP
+.BI "int ioctl(int " fd ", XFS_IOC_AG_GEOMETRY, struct xfs_ag_geometry *" arg );
+.SH DESCRIPTION
+This XFS ioctl retrieves the geometry information for a given allocation group.
+The geometry information is conveyed in a structure of the following form:
+.PP
+.in +4n
+.nf
+struct xfs_ag_geometry {
+	uint32_t  ag_number;
+	uint32_t  ag_length;
+	uint32_t  ag_freeblks;
+	uint32_t  ag_icount;
+	uint32_t  ag_ifree;
+	uint32_t  ag_sick;
+	uint32_t  ag_checked;
+	uint32_t  ag_flags;
+	uint64_t  ag_reserved[12];
+};
+.fi
+.in
+.TP
+.I ag_number
+The caller must set this field to the index of the allocation group that the
+caller wishes to learn about.
+.TP
+.I ag_length
+The length of the allocation group is returned in this field, in units of
+filesystem blocks.
+.TP
+.I ag_freeblks
+The number of free blocks in the allocation group is returned in this field, in
+units of filesystem blocks.
+.TP
+.I ag_icount
+The number of inode records allocated in this allocation group is returned in
+this field.
+.TP
+.I ag_ifree
+The number of unused inode records (of the space allocated) in this allocation
+group is returned in this field.
+.TP
+.I ag_flags
+The caller can set this field to change the operational behavior of the ioctl.
+Currently no flags are defined, so this field must be zero.
+.TP
+.IR ag_reserved
+All reserved fields will be set to zero on return.
+.SH RETURN VALUE
+On error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B EFSBADCRC
+Metadata checksum validation failed while performing the query.
+.TP
+.B EFSCORRUPTED
+Metadata corruption was encountered while performing the query.
+.TP
+.B EINVAL
+The specified allocation group number is not valid for this filesystem.
+.TP
+.B EIO
+An I/O error was encountered while performing the query.
+.SH CONFORMING TO
+This API is specific to XFS filesystem on the Linux kernel.
+.SH SEE ALSO
+.BR ioctl (2)
diff --git a/man/man3/xfsctl.3 b/man/man3/xfsctl.3
index 7e6588b8..dfebd12d 100644
--- a/man/man3/xfsctl.3
+++ b/man/man3/xfsctl.3
@@ -336,6 +336,12 @@ See
 .BR ioctl_xfs_fsop_geometry (2)
 for more information.
 
+.TP
+.B XFS_IOC_AG_GEOMETRY
+See
+.BR ioctl_xfs_ag_geometry (2)
+for more information.
+
 .TP
 .BR XFS_IOC_FSBULKSTAT " or " XFS_IOC_FSBULKSTAT_SINGLE
 See

[PATCH 04/10] man: document the new health reporting fields in various ioctls

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Update the manpages to conver the new health reporting fields in the
fs geometry, ag geometry, and bulkstat ioctls.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
---
 man/man2/ioctl_xfs_ag_geometry.2   |   48 +++++++++++++++++++++++++++++++
 man/man2/ioctl_xfs_fsbulkstat.2    |   52 +++++++++++++++++++++++++++++++++
 man/man2/ioctl_xfs_fsop_geometry.2 |   56 +++++++++++++++++++++++++++++++++++-
 3 files changed, 154 insertions(+), 2 deletions(-)


diff --git a/man/man2/ioctl_xfs_ag_geometry.2 b/man/man2/ioctl_xfs_ag_geometry.2
index ddd54265..80ebcb76 100644
--- a/man/man2/ioctl_xfs_ag_geometry.2
+++ b/man/man2/ioctl_xfs_ag_geometry.2
@@ -57,6 +57,54 @@ Currently no flags are defined, so this field must be zero.
 .TP
 .IR ag_reserved
 All reserved fields will be set to zero on return.
+.PP
+The fields
+.IR ag_sick " and " ag_checked
+indicate the relative health of various allocation group metadata:
+.IP \[bu] 2
+If a given sick flag is set in
+.IR ag_sick ,
+then that piece of metadata has been observed to be damaged.
+The same bit will be set in
+.IR ag_checked .
+.IP \[bu]
+If a given sick flag is set in
+.I ag_checked
+and is not set in
+.IR ag_sick ,
+then that piece of metadata has been checked and is not faulty.
+.IP \[bu]
+If a given sick flag is not set in
+.IR ag_checked ,
+then no conclusion can be made.
+.PP
+The following flags apply to these fields:
+.RS 0.4i
+.TP
+.B XFS_AG_GEOM_SICK_SB
+Allocation group superblock.
+.TP
+.B XFS_AG_GEOM_SICK_AGF
+Free space header.
+.TP
+.B XFS_AG_GEOM_SICK_AGFL
+Free space reserve list.
+.TP
+.B XFS_AG_GEOM_SICK_AGI
+Inode header.
+.TP
+.BR XFS_AG_GEOM_SICK_BNOBT " or " XFS_AG_GEOM_SICK_CNTBT
+Free space btrees.
+.TP
+.BR XFS_AG_GEOM_SICK_INOBT " or " XFS_AG_GEOM_SICK_FINOBT
+Inode btrees.
+.TP
+.B XFS_AG_GEOM_SICK_RMAPBT
+Reverse mapping btree.
+.TP
+.B XFS_AG_GEOM_SICK_REFCNTBT
+Reference count btree.
+.RE
 .SH RETURN VALUE
 On error, \-1 is returned, and
 .I errno
diff --git a/man/man2/ioctl_xfs_fsbulkstat.2 b/man/man2/ioctl_xfs_fsbulkstat.2
index a8b22dc4..3e13cfa8 100644
--- a/man/man2/ioctl_xfs_fsbulkstat.2
+++ b/man/man2/ioctl_xfs_fsbulkstat.2
@@ -94,7 +94,9 @@ struct xfs_bstat {
 	__u16             bs_projid_lo;
 	__u16             bs_forkoff;
 	__u16             bs_projid_hi;
-	unsigned char     bs_pad[6];
+	uint16_t          bs_sick;
+	uint16_t          bs_checked;
+	unsigned char     bs_pad[2];
 	__u32             bs_cowextsize;
 	__u32             bs_dmevmask;
 	__u16             bs_dmstate;
@@ -184,6 +186,54 @@ is unused on Linux.
 .I bs_aextents
 is the number of storage mappings associated with this file's extended
 attributes.
+.PP
+The fields
+.IR bs_sick " and " bs_checked
+indicate the relative health of various allocation group metadata:
+.IP \[bu] 2
+If a given sick flag is set in
+.IR bs_sick ,
+then that piece of metadata has been observed to be damaged.
+The same bit should be set in
+.IR bs_checked .
+.IP \[bu]
+If a given sick flag is set in
+.I bs_checked
+but is not set in
+.IR bs_sick ,
+then that piece of metadata has been checked and is not faulty.
+.IP \[bu]
+If a given sick flag is not set in
+.IR bs_checked ,
+then no conclusion can be made.
+.PP
+The following flags apply to these fields:
+.RS 0.4i
+.TP
+.B XFS_BS_SICK_INODE
+The inode's record itself.
+.TP
+.B XFS_BS_SICK_BMBTD
+File data extent mappings.
+.TP
+.B XFS_BS_SICK_BMBTA
+Extended attribute extent mappings.
+.TP
+.B XFS_BS_SICK_BMBTC
+Copy on Write staging extent mappings.
+.TP
+.B XFS_BS_SICK_DIR
+Directory information.
+.TP
+.B XFS_BS_SICK_XATTR
+Extended attribute data.
+.TP
+.B XFS_BS_SICK_SYMLINK
+Symbolic link target.
+.TP
+.B XFS_BS_SICK_PARENT
+Parent pointers.
+.RE
 .SH RETURN VALUE
 On error, \-1 is returned, and
 .I errno
diff --git a/man/man2/ioctl_xfs_fsop_geometry.2 b/man/man2/ioctl_xfs_fsop_geometry.2
index 365bda8b..a35bbaeb 100644
--- a/man/man2/ioctl_xfs_fsop_geometry.2
+++ b/man/man2/ioctl_xfs_fsop_geometry.2
@@ -47,7 +47,9 @@ struct xfs_fsop_geom {
 	__u32         logsunit;
 	/* struct xfs_fsop_geom_v4 stops here. */
 
-	__u64         reserved[18];
+	__u32         sick;
+	__u32         checked;
+	__u64         reserved[17];
 };
 .fi
 .in
@@ -130,6 +132,13 @@ This field is meaningful only if the flag
 .B  XFS_FSOP_GEOM_FLAGS_LOGV2
 is set.
 .PP
+The fields
+.IR sick " and " checked
+indicate the relative health of various whole-filesystem metadata.
+Please see the section
+.B XFS METADATA HEALTH REPORTING
+for more details.
+.PP
 .I reserved
 is set to zero.
 .SH FILESYSTEM FEATURE FLAGS
@@ -203,6 +212,51 @@ Filesystem stores reverse mappings of blocks to owners.
 .B XFS_FSOP_GEOM_FLAGS_REFLINK
 Filesystem supports sharing blocks between files.
 .RE
+.SH XFS METADATA HEALTH REPORTING
+.PP
+The online filesystem checking utility scans metadata and records what it
+finds in the kernel incore state.
+The following scheme is used for userspace to read the incore health status
+of the filesystem:
+
+.IP \[bu] 2
+If a given sick flag is set in
+.IR sick ,
+then that piece of metadata has been observed to be damaged.
+The same bit should be set in
+.IR checked .
+.IP \[bu]
+If a given sick flag is set in
+.I checked
+but is not set in
+.IR sick ,
+then that piece of metadata has been checked and is not faulty.
+.IP \[bu]
+If a given sick flag is not set in
+.IR checked ,
+then no conclusion can be made.
+.PP
+The following flags apply to these fields:
+.RS 0.4i
+.TP
+.B XFS_FSOP_GEOM_SICK_COUNTERS
+Inode and space summary counters.
+.TP
+.B XFS_FSOP_GEOM_SICK_UQUOTA
+User quota information.
+.TP
+.B XFS_FSOP_GEOM_SICK_GQUOTA
+Group quota information.
+.TP
+.B XFS_FSOP_GEOM_SICK_PQUOTA
+Project quota information.
+.TP
+.B XFS_FSOP_GEOM_SICK_RT_BITMAP
+Free space bitmap for the realtime device.
+.TP
+.B XFS_FSOP_GEOM_SICK_RT_SUMMARY
+Free space summary for the realtime device.
+.RE
 
 .SH RETURN VALUE
 On error, \-1 is returned, and

[PATCH 05/10] xfs_db: remove db/convert.h

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

db/convert.h conflicts with include/convert.h and since the former only
has one declaration in it anyway, just get rid of it.  We'll need this
in the next patch to avoid an ugly include mess.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
---
 db/Makefile  |    4 ++--
 db/command.c |    1 -
 db/command.h |    1 +
 db/convert.c |    1 -
 db/convert.h |    7 -------
 5 files changed, 3 insertions(+), 11 deletions(-)
 delete mode 100644 db/convert.h


diff --git a/db/Makefile b/db/Makefile
index 8fecfc1c..0941b32e 100644
--- a/db/Makefile
+++ b/db/Makefile
@@ -8,13 +8,13 @@ include $(TOPDIR)/include/builddefs
 LTCOMMAND = xfs_db
 
 HFILES = addr.h agf.h agfl.h agi.h attr.h attrshort.h bit.h block.h bmap.h \
-	btblock.h bmroot.h check.h command.h convert.h crc.h debug.h \
+	btblock.h bmroot.h check.h command.h crc.h debug.h \
 	dir2.h dir2sf.h dquot.h echo.h faddr.h field.h \
 	flist.h fprint.h frag.h freesp.h hash.h help.h init.h inode.h input.h \
 	io.h logformat.h malloc.h metadump.h output.h print.h quit.h sb.h \
 	sig.h strvec.h text.h type.h write.h attrset.h symlink.h fsmap.h \
 	fuzz.h
-CFILES = $(HFILES:.h=.c) btdump.c info.c
+CFILES = $(HFILES:.h=.c) btdump.c convert.c info.c
 LSRCFILES = xfs_admin.sh xfs_ncheck.sh xfs_metadump.sh
 
 LLDLIBS	= $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD)
diff --git a/db/command.c b/db/command.c
index c7c52342..89a78f03 100644
--- a/db/command.c
+++ b/db/command.c
@@ -11,7 +11,6 @@
 #include "bmap.h"
 #include "check.h"
 #include "command.h"
-#include "convert.h"
 #include "debug.h"
 #include "type.h"
 #include "echo.h"
diff --git a/db/command.h b/db/command.h
index eacfd465..2f9a7e16 100644
--- a/db/command.h
+++ b/db/command.h
@@ -28,5 +28,6 @@ extern int		command(int argc, char **argv);
 extern const cmdinfo_t	*find_command(const char *cmd);
 extern void		init_commands(void);
 
+extern void		convert_init(void);
 extern void		btdump_init(void);
 extern void		info_init(void);
diff --git a/db/convert.c b/db/convert.c
index 01a08823..e1466057 100644
--- a/db/convert.c
+++ b/db/convert.c
@@ -6,7 +6,6 @@
 
 #include "libxfs.h"
 #include "command.h"
-#include "convert.h"
 #include "output.h"
 #include "init.h"
 
diff --git a/db/convert.h b/db/convert.h
deleted file mode 100644
index 3660cabe..00000000
--- a/db/convert.h
+++ /dev/null
@@ -1,7 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Copyright (c) 2000-2001,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- */
-
-extern void	convert_init(void);

[PATCH 06/10] xfs_db: add a function to compute btree geometry

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Add a new command to xfs_db that uses a btree type and a record count
to report the best and worst case height and level size.  This can be
used to estimate how much overhead a metadata index will add to a
filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
---
 db/Makefile              |    2 
 db/btheight.c            |  308 ++++++++++++++++++++++++++++++++++++++++++++++
 db/command.c             |    1 
 db/command.h             |    1 
 libxfs/libxfs_api_defs.h |    2 
 5 files changed, 313 insertions(+), 1 deletion(-)
 create mode 100644 db/btheight.c


diff --git a/db/Makefile b/db/Makefile
index 0941b32e..ed9f56c2 100644
--- a/db/Makefile
+++ b/db/Makefile
@@ -14,7 +14,7 @@ HFILES = addr.h agf.h agfl.h agi.h attr.h attrshort.h bit.h block.h bmap.h \
 	io.h logformat.h malloc.h metadump.h output.h print.h quit.h sb.h \
 	sig.h strvec.h text.h type.h write.h attrset.h symlink.h fsmap.h \
 	fuzz.h
-CFILES = $(HFILES:.h=.c) btdump.c convert.c info.c
+CFILES = $(HFILES:.h=.c) btdump.c btheight.c convert.c info.c
 LSRCFILES = xfs_admin.sh xfs_ncheck.sh xfs_metadump.sh
 
 LLDLIBS	= $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD)
diff --git a/db/btheight.c b/db/btheight.c
new file mode 100644
index 00000000..289e5d84
--- /dev/null
+++ b/db/btheight.c
@@ -0,0 +1,308 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2019 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <darrick.wong@oracle.com>
+ */
+#include "libxfs.h"
+#include "command.h"
+#include "output.h"
+#include "init.h"
+#include "io.h"
+#include "type.h"
+#include "input.h"
+#include "libfrog/convert.h"
+
+static int refc_maxrecs(struct xfs_mount *mp, int blocklen, int leaf)
+{
+	return libxfs_refcountbt_maxrecs(blocklen, leaf != 0);
+}
+
+static int rmap_maxrecs(struct xfs_mount *mp, int blocklen, int leaf)
+{
+	return libxfs_rmapbt_maxrecs(blocklen, leaf);
+}
+
+struct btmap {
+	const char	*tag;
+	int		(*maxrecs)(struct xfs_mount *mp, int blocklen,
+				   int leaf);
+} maps[] = {
+	{"bnobt", libxfs_allocbt_maxrecs},
+	{"cntbt", libxfs_allocbt_maxrecs},
+	{"inobt", libxfs_inobt_maxrecs},
+	{"finobt", libxfs_inobt_maxrecs},
+	{"bmapbt", libxfs_bmbt_maxrecs},
+	{"refcountbt", refc_maxrecs},
+	{"rmapbt", rmap_maxrecs},
+};
+
+static void
+btheight_help(void)
+{
+	struct btmap	*m;
+	int		i;
+
+	dbprintf(_(
+"\n"
+" For a given number of btree records and a btree type, report the number of\n"
+" records and blocks for each level of the btree, and the total btree size.\n"
+" The btree type must be given after the options.  A raw btree geometry can\n"
+" be provided in the format \"record_bytes:key_bytes:ptr_bytes:header_type\"\n"
+" where header_type is one of \"short\", \"long\", \"shortcrc\", or \"longcrc\".\n"
+"\n"
+" Options:\n"
+"   -b -- Override the btree block size.\n"
+"   -n -- Number of records we want to store.\n"
+"   -w max -- Show only the best case scenario.\n"
+"   -w min -- Show only the worst case scenario.\n"
+"\n"
+" Supported btree types:\n"
+"   "
+));
+	for (i = 0, m = maps; i < ARRAY_SIZE(maps); i++, m++)
+		printf("%s ", m->tag);
+	printf("\n");
+}
+
+static void
+calc_height(
+	unsigned long long	nr_records,
+	uint			*records_per_block)
+{
+	unsigned int		level = 0;
+	unsigned long long	total_blocks = 0;
+	unsigned long long	blocks;
+	char			*levels_suffix = "s";
+	char			*totblocks_suffix = "s";
+
+	while (nr_records) {
+		unsigned int	level_rpb = records_per_block[level != 0];
+		char		*recs_suffix = "s";
+		char		*blocks_suffix = "s";
+
+		blocks = (nr_records + level_rpb - 1) / level_rpb;
+
+		if (nr_records == 1)
+			recs_suffix = "";
+		if (blocks == 1)
+			blocks_suffix = "";
+
+		printf(_("level %u: %llu record%s, %llu block%s\n"),
+				level, nr_records, recs_suffix, blocks,
+				blocks_suffix);
+
+		total_blocks += blocks;
+		nr_records = blocks == 1 ? 0 : blocks;
+		level++;
+	}
+
+	if (level == 1)
+		levels_suffix = "";
+	if (total_blocks == 1)
+		totblocks_suffix = "";
+
+	printf(_("%u level%s, %llu block%s total\n"), level, levels_suffix,
+			total_blocks, totblocks_suffix);
+}
+
+static int
+construct_records_per_block(
+	char		*tag,
+	int		blocksize,
+	unsigned int	*records_per_block)
+{
+	char		*toktag;
+	struct btmap	*m;
+	unsigned int	record_size, key_size, ptr_size;
+	char		*p;
+	int		i, ret;
+
+	for (i = 0, m = maps; i < ARRAY_SIZE(maps); i++, m++) {
+		if (!strcmp(m->tag, tag)) {
+			records_per_block[0] = m->maxrecs(mp, blocksize, 1);
+			records_per_block[1] = m->maxrecs(mp, blocksize, 0);
+			return 0;
+		}
+	}
+
+	toktag = strdup(tag);
+	ret = -1;
+
+	p = strtok(toktag, ":");
+	if (!p) {
+		fprintf(stderr, _("%s: record size not found.\n"), tag);
+		goto out;
+	}
+	record_size = cvt_u16(p, 0);
+	if (errno) {
+		perror(p);
+		goto out;
+	}
+
+	p = strtok(NULL, ":");
+	if (!p) {
+		fprintf(stderr, _("%s: key size not found.\n"), tag);
+		goto out;
+	}
+	key_size = cvt_u16(p, 0);
+	if (errno) {
+		perror(p);
+		goto out;
+	}
+
+	p = strtok(NULL, ":");
+	if (!p) {
+		fprintf(stderr, _("%s: pointer size not found.\n"), tag);
+		goto out;
+	}
+	ptr_size = cvt_u16(p, 0);
+	if (errno) {
+		perror(p);
+		goto out;
+	}
+
+	p = strtok(NULL, ":");
+	if (!p) {
+		fprintf(stderr, _("%s: header type not found.\n"), tag);
+		goto out;
+	}
+	if (!strcmp(p, "short"))
+		blocksize -= XFS_BTREE_SBLOCK_LEN;
+	else if (!strcmp(p, "shortcrc"))
+		blocksize -= XFS_BTREE_SBLOCK_CRC_LEN;
+	else if (!strcmp(p, "long"))
+		blocksize -= XFS_BTREE_LBLOCK_LEN;
+	else if (!strcmp(p, "longcrc"))
+		blocksize -= XFS_BTREE_LBLOCK_CRC_LEN;
+	else {
+		fprintf(stderr, _("%s: unrecognized btree header type."),
+				p);
+		goto out;
+	}
+
+	p = strtok(NULL, ":");
+	if (p) {
+		fprintf(stderr,
+			_("%s: unrecognized raw btree geometry."),
+				tag);
+		goto out;
+	}
+
+	records_per_block[0] = blocksize / record_size;
+	records_per_block[1] = blocksize / (key_size + ptr_size);
+	ret = 0;
+out:
+	free(toktag);
+	return ret;
+}
+
+#define REPORT_DEFAULT	(-1U)
+#define REPORT_MAX	(1 << 0)
+#define REPORT_MIN	(1 << 1)
+
+static void
+report(
+	char			*tag,
+	unsigned int		report_what,
+	unsigned long long	nr_records,
+	unsigned int		blocksize)
+{
+	unsigned int		records_per_block[2];
+	int			ret;
+
+	ret = construct_records_per_block(tag, blocksize, records_per_block);
+	if (ret) {
+		printf(_("%s: Unable to determine records per block.\n"),
+				tag);
+		return;
+	}
+
+	if (report_what & REPORT_MAX) {
+		printf(
+_("%s: best case per %u-byte block: %u records (leaf) / %u keyptrs (node)\n"),
+				tag, blocksize, records_per_block[0],
+				records_per_block[1]);
+
+		calc_height(nr_records, records_per_block);
+	}
+
+	if (report_what & REPORT_MIN) {
+		records_per_block[0] /= 2;
+		records_per_block[1] /= 2;
+
+		printf(
+_("%s: worst case per %u-byte block: %u records (leaf) / %u keyptrs (node)\n"),
+				tag, blocksize, records_per_block[0],
+				records_per_block[1]);
+
+		calc_height(nr_records, records_per_block);
+	}
+}
+
+static int
+btheight_f(
+	int		argc,
+	char		**argv)
+{
+	long long	blocksize = mp->m_sb.sb_blocksize;
+	uint64_t	nr_records = 0;
+	int		report_what = REPORT_DEFAULT;
+	int		i, c;
+
+	while ((c = getopt(argc, argv, "b:n:w:")) != -1) {
+		switch (c) {
+		case 'b':
+			errno = 0;
+			blocksize = cvtnum(mp->m_sb.sb_blocksize,
+					mp->m_sb.sb_sectsize,
+					optarg);
+			if (errno) {
+				perror(optarg);
+				return 0;
+			}
+			break;
+		case 'n':
+			nr_records = cvt_u64(optarg, 0);
+			if (errno) {
+				perror(optarg);
+				return 0;
+			}
+			break;
+		case 'w':
+			if (!strcmp(optarg, "min"))
+				report_what = REPORT_MIN;
+			else if (!strcmp(optarg, "max"))
+				report_what = REPORT_MAX;
+			else {
+				btheight_help();
+				return 0;
+			}
+			break;
+		default:
+			btheight_help();
+			return 0;
+		}
+	}
+
+	if (argc == optind || blocksize <= 0 || blocksize > INT_MAX ||
+	    nr_records == 0) {
+		btheight_help();
+		return 0;
+	}
+
+	for (i = optind; i < argc; i++)
+		report(argv[i], report_what, nr_records, blocksize);
+
+	return 0;
+}
+
+static const cmdinfo_t btheight_cmd =
+	{ "btheight", "b", btheight_f, 1, -1, 0,
+	  "[-b blksz] [-n recs] [-w max|-w min] btree types...",
+	  N_("compute btree heights"), btheight_help };
+
+void
+btheight_init(void)
+{
+	add_command(&btheight_cmd);
+}
diff --git a/db/command.c b/db/command.c
index 89a78f03..0fb44efa 100644
--- a/db/command.c
+++ b/db/command.c
@@ -113,6 +113,7 @@ init_commands(void)
 	block_init();
 	bmap_init();
 	btdump_init();
+	btheight_init();
 	check_init();
 	convert_init();
 	crc_init();
diff --git a/db/command.h b/db/command.h
index 2f9a7e16..b8499de0 100644
--- a/db/command.h
+++ b/db/command.h
@@ -31,3 +31,4 @@ extern void		init_commands(void);
 extern void		convert_init(void);
 extern void		btdump_init(void);
 extern void		info_init(void);
+extern void		btheight_init(void);
diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h
index 7bd373f7..0ae21318 100644
--- a/libxfs/libxfs_api_defs.h
+++ b/libxfs/libxfs_api_defs.h
@@ -125,6 +125,8 @@
 #define xfs_dir_ino_validate		libxfs_dir_ino_validate
 #define xfs_initialize_perag_data	libxfs_initialize_perag_data
 #define xfs_inobt_maxrecs		libxfs_inobt_maxrecs
+#define xfs_rmapbt_maxrecs		libxfs_rmapbt_maxrecs
+#define xfs_refcountbt_maxrecs		libxfs_refcountbt_maxrecs
 #define xfs_iread_extents		libxfs_iread_extents
 #define xfs_log_calc_minimum_size	libxfs_log_calc_minimum_size
 #define xfs_perag_get			libxfs_perag_get

[PATCH 07/10] xfs_db: use precomputed inode geometry values

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Use the precomputed inode geometry values instead of open-coding them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
---
 db/inode.c      |    8 +++-----
 repair/dinode.c |    2 +-
 2 files changed, 4 insertions(+), 6 deletions(-)


diff --git a/db/inode.c b/db/inode.c
index 73dd118d..d8d69ffb 100644
--- a/db/inode.c
+++ b/db/inode.c
@@ -657,16 +657,14 @@ set_cur_inode(
 	    igeo->inoalign_mask) {
 		xfs_agblock_t	chunk_agbno;
 		xfs_agblock_t	offset_agbno;
-		int		blks_per_cluster;
 
-		blks_per_cluster = igeo->inode_cluster_size >>
-							mp->m_sb.sb_blocklog;
 		offset_agbno = agbno & igeo->inoalign_mask;
 		chunk_agbno = agbno - offset_agbno;
 		cluster_agbno = chunk_agbno +
-			((offset_agbno / blks_per_cluster) * blks_per_cluster);
+			((offset_agbno / M_IGEO(mp)->blocks_per_cluster) *
+			 M_IGEO(mp)->blocks_per_cluster);
 		offset += ((agbno - cluster_agbno) * mp->m_sb.sb_inopblock);
-		numblks = XFS_FSB_TO_BB(mp, blks_per_cluster);
+		numblks = XFS_FSB_TO_BB(mp, M_IGEO(mp)->blocks_per_cluster);
 	} else
 		cluster_agbno = agbno;
 
diff --git a/repair/dinode.c b/repair/dinode.c
index 56992dd2..f5e88cc3 100644
--- a/repair/dinode.c
+++ b/repair/dinode.c
@@ -768,7 +768,7 @@ get_agino_buf(
 	 * we must find the buffer for its cluster, add the appropriate
 	 * offset, and return that.
 	 */
-	cluster_size = max(igeo->inode_cluster_size, mp->m_sb.sb_blocksize);
+	cluster_size = igeo->inode_cluster_size;
 	ino_per_cluster = cluster_size / mp->m_sb.sb_inodesize;
 	cluster_agino = agino & ~(ino_per_cluster - 1);
 	cluster_blks = XFS_FSB_TO_DADDR(mp, max(1,

[PATCH 08/10] xfs_repair: use precomputed inode geometry values

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Use the precomputed inode geometry values instead of open-coding them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
---
 repair/dino_chunks.c |   22 +++++++++++-----------
 repair/dinode.c      |   13 ++++---------
 repair/globals.c     |    1 -
 repair/globals.h     |    1 -
 repair/prefetch.c    |   22 ++++++++++------------
 repair/xfs_repair.c  |    2 --
 6 files changed, 25 insertions(+), 36 deletions(-)


diff --git a/repair/dino_chunks.c b/repair/dino_chunks.c
index 323a355e..00b67468 100644
--- a/repair/dino_chunks.c
+++ b/repair/dino_chunks.c
@@ -608,7 +608,6 @@ process_inode_chunk(
 	xfs_ino_t		ino;
 	int			dirty = 0;
 	int			isa_dir = 0;
-	int			blks_per_cluster;
 	int			cluster_count;
 	int			bp_index;
 	int			cluster_offset;
@@ -620,10 +619,7 @@ process_inode_chunk(
 	*bogus = 0;
 	ASSERT(igeo->ialloc_blks > 0);
 
-	blks_per_cluster = M_IGEO(mp)->inode_cluster_size >> mp->m_sb.sb_blocklog;
-	if (blks_per_cluster == 0)
-		blks_per_cluster = 1;
-	cluster_count = XFS_INODES_PER_CHUNK / inodes_per_cluster;
+	cluster_count = XFS_INODES_PER_CHUNK / M_IGEO(mp)->inodes_per_cluster;
 	if (cluster_count == 0)
 		cluster_count = 1;
 
@@ -662,13 +658,16 @@ process_inode_chunk(
 
 		bplist[bp_index] = libxfs_readbuf(mp->m_dev,
 					XFS_AGB_TO_DADDR(mp, agno, agbno),
-					XFS_FSB_TO_BB(mp, blks_per_cluster), 0,
+					XFS_FSB_TO_BB(mp,
+						M_IGEO(mp)->blocks_per_cluster),
+					0,
 					&xfs_inode_buf_ops);
 		if (!bplist[bp_index]) {
 			do_warn(_("cannot read inode %" PRIu64 ", disk block %" PRId64 ", cnt %d\n"),
 				XFS_AGINO_TO_INO(mp, agno, first_irec->ino_startnum),
 				XFS_AGB_TO_DADDR(mp, agno, agbno),
-				XFS_FSB_TO_BB(mp, blks_per_cluster));
+				XFS_FSB_TO_BB(mp,
+					M_IGEO(mp)->blocks_per_cluster));
 			while (bp_index > 0) {
 				bp_index--;
 				libxfs_putbuf(bplist[bp_index]);
@@ -684,8 +683,9 @@ process_inode_chunk(
 		bplist[bp_index]->b_ops = &xfs_inode_buf_ops;
 
 next_readbuf:
-		irec_offset += mp->m_sb.sb_inopblock * blks_per_cluster;
-		agbno += blks_per_cluster;
+		irec_offset += mp->m_sb.sb_inopblock *
+				M_IGEO(mp)->blocks_per_cluster;
+		agbno += M_IGEO(mp)->blocks_per_cluster;
 	}
 	agbno = XFS_AGINO_TO_AGBNO(mp, first_irec->ino_startnum);
 
@@ -745,7 +745,7 @@ process_inode_chunk(
 				ASSERT(ino_rec->ino_startnum == agino + 1);
 				irec_offset = 0;
 			}
-			if (cluster_offset == inodes_per_cluster) {
+			if (cluster_offset == M_IGEO(mp)->inodes_per_cluster) {
 				bp_index++;
 				cluster_offset = 0;
 			}
@@ -964,7 +964,7 @@ process_inode_chunk(
 			ASSERT(ino_rec->ino_startnum == agino + 1);
 			irec_offset = 0;
 		}
-		if (cluster_offset == inodes_per_cluster) {
+		if (cluster_offset == M_IGEO(mp)->inodes_per_cluster) {
 			bp_index++;
 			cluster_offset = 0;
 		}
diff --git a/repair/dinode.c b/repair/dinode.c
index f5e88cc3..8af2cb25 100644
--- a/repair/dinode.c
+++ b/repair/dinode.c
@@ -755,8 +755,6 @@ get_agino_buf(
 	struct xfs_dinode	**dipp)
 {
 	struct xfs_buf		*bp;
-	int			cluster_size;
-	int			ino_per_cluster;
 	xfs_agino_t		cluster_agino;
 	xfs_daddr_t		cluster_daddr;
 	xfs_daddr_t		cluster_blks;
@@ -768,18 +766,15 @@ get_agino_buf(
 	 * we must find the buffer for its cluster, add the appropriate
 	 * offset, and return that.
 	 */
-	cluster_size = igeo->inode_cluster_size;
-	ino_per_cluster = cluster_size / mp->m_sb.sb_inodesize;
-	cluster_agino = agino & ~(ino_per_cluster - 1);
-	cluster_blks = XFS_FSB_TO_DADDR(mp, max(1,
-			igeo->inode_cluster_size >> mp->m_sb.sb_blocklog));
+	cluster_agino = agino & ~(igeo->inodes_per_cluster - 1);
+	cluster_blks = XFS_FSB_TO_DADDR(mp, igeo->blocks_per_cluster);
 	cluster_daddr = XFS_AGB_TO_DADDR(mp, agno,
 			XFS_AGINO_TO_AGBNO(mp, cluster_agino));
 
 #ifdef XR_INODE_TRACE
 	printf("cluster_size %d ipc %d clusagino %d daddr %lld sectors %lld\n",
-		cluster_size, ino_per_cluster, cluster_agino, cluster_daddr,
-		cluster_blks);
+		M_IGEO(mp)->inode_cluster_size, M_IGEO(mp)->inodes_per_cluster,
+		cluster_agino, cluster_daddr, cluster_blks);
 #endif
 
 	bp = libxfs_readbuf(mp->m_dev, cluster_daddr, cluster_blks,
diff --git a/repair/globals.c b/repair/globals.c
index ae9d55b4..dcd79ea4 100644
--- a/repair/globals.c
+++ b/repair/globals.c
@@ -81,7 +81,6 @@ xfs_agblock_t	inobt_root;
 /* configuration vars -- fs geometry dependent */
 
 int		inodes_per_block;
-int		inodes_per_cluster;
 unsigned int	glob_agcount;
 int		chunks_pblock;	/* # of 64-ino chunks per allocation */
 int		max_symlink_blocks;
diff --git a/repair/globals.h b/repair/globals.h
index 05121d4f..008bdd90 100644
--- a/repair/globals.h
+++ b/repair/globals.h
@@ -122,7 +122,6 @@ extern xfs_agblock_t	inobt_root;
 /* configuration vars -- fs geometry dependent */
 
 extern int		inodes_per_block;
-extern int		inodes_per_cluster;
 extern unsigned int	glob_agcount;
 extern int		chunks_pblock;	/* # of 64-ino chunks per allocation */
 extern int		max_symlink_blocks;
diff --git a/repair/prefetch.c b/repair/prefetch.c
index 2fecfd68..beb36cd6 100644
--- a/repair/prefetch.c
+++ b/repair/prefetch.c
@@ -710,16 +710,14 @@ pf_queuing_worker(
 	int			num_inos;
 	ino_tree_node_t		*irec;
 	ino_tree_node_t		*cur_irec;
-	int			blks_per_cluster;
 	xfs_agblock_t		bno;
 	int			i;
 	int			err;
 	uint64_t		sparse;
 	struct xfs_ino_geometry	*igeo = M_IGEO(mp);
+	unsigned long long	cluster_mask;
 
-	blks_per_cluster = igeo->inode_cluster_size >> mp->m_sb.sb_blocklog;
-	if (blks_per_cluster == 0)
-		blks_per_cluster = 1;
+	cluster_mask = (1ULL << igeo->inodes_per_cluster) - 1;
 
 	for (i = 0; i < PF_THREAD_COUNT; i++) {
 		err = pthread_create(&args->io_threads[i], NULL,
@@ -786,21 +784,22 @@ pf_queuing_worker(
 			struct xfs_buf_map	map;
 
 			map.bm_bn = XFS_AGB_TO_DADDR(mp, args->agno, bno);
-			map.bm_len = XFS_FSB_TO_BB(mp, blks_per_cluster);
+			map.bm_len = XFS_FSB_TO_BB(mp,
+					igeo->blocks_per_cluster);
 
 			/*
 			 * Queue I/O for each non-sparse cluster. We can check
 			 * sparse state in cluster sized chunks as cluster size
 			 * is the min. granularity of sparse irec regions.
 			 */
-			if ((sparse & ((1ULL << inodes_per_cluster) - 1)) == 0)
+			if ((sparse & cluster_mask) == 0)
 				pf_queue_io(args, &map, 1,
 					    (cur_irec->ino_isa_dir != 0) ?
 					     B_DIR_INODE : B_INODE);
 
-			bno += blks_per_cluster;
-			num_inos += inodes_per_cluster;
-			sparse >>= inodes_per_cluster;
+			bno += igeo->blocks_per_cluster;
+			num_inos += igeo->inodes_per_cluster;
+			sparse >>= igeo->inodes_per_cluster;
 		} while (num_inos < igeo->ialloc_inos);
 	}
 
@@ -903,9 +902,8 @@ start_inode_prefetch(
 
 	max_queue = libxfs_bcache->c_maxcount / thread_count / 8;
 	if (igeo->inode_cluster_size > mp->m_sb.sb_blocksize)
-		max_queue = max_queue *
-			(igeo->inode_cluster_size >> mp->m_sb.sb_blocklog) /
-			igeo->ialloc_blks;
+		max_queue = max_queue * igeo->blocks_per_cluster /
+				igeo->ialloc_blks;
 
 	sem_init(&args->ra_count, 0, max_queue);
 
diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c
index b11b7448..7e810ef4 100644
--- a/repair/xfs_repair.c
+++ b/repair/xfs_repair.c
@@ -763,8 +763,6 @@ main(int argc, char **argv)
 
 	chunks_pblock = mp->m_sb.sb_inopblock / XFS_INODES_PER_CHUNK;
 	max_symlink_blocks = libxfs_symlink_blocks(mp, XFS_SYMLINK_MAXLEN);
-	inodes_per_cluster = max(mp->m_sb.sb_inopblock,
-			igeo->inode_cluster_size >> mp->m_sb.sb_inodelog);
 
 	/*
 	 * Automatic striding for high agcount filesystems.

[PATCH 09/10] xfs_repair: reduce the amount of "clearing reflink flag" messages

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Clearing the reflink flag on files that don't share blocks is an
optimization, not a repair, so it's not critical to log a message every
single time we clear a flag.  Only log one message that we're clearing
these flags unless verbose mode is enabled.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
---
 repair/rmap.c |   34 +++++++++++++++++++++++++++++++---
 1 file changed, 31 insertions(+), 3 deletions(-)


diff --git a/repair/rmap.c b/repair/rmap.c
index 5dd6557a..b907383e 100644
--- a/repair/rmap.c
+++ b/repair/rmap.c
@@ -1170,6 +1170,36 @@ record_inode_reflink_flag(
 		(unsigned long long)lino, (unsigned long long)irec->ino_was_rl);
 }
 
+/*
+ * Inform the user that we're clearing the reflink flag on an inode that
+ * doesn't actually share any blocks.  This is an optimization (the kernel
+ * skips refcount checks for non-reflink files) and not a corruption repair,
+ * so we don't need to log every time we clear a flag unless verbose mode is
+ * enabled.
+ */
+static void
+warn_clearing_reflink(
+	xfs_ino_t		ino)
+{
+	static bool		warned = false;
+	static pthread_mutex_t	lock = PTHREAD_MUTEX_INITIALIZER;
+
+	if (verbose) {
+		do_warn(_("clearing reflink flag on inode %"PRIu64"\n"), ino);
+		return;
+	}
+
+	if (warned)
+		return;
+
+	pthread_mutex_lock(&lock);
+	if (!warned) {
+		do_warn(_("clearing reflink flag on inodes when possible\n"));
+		warned = true;
+	}
+	pthread_mutex_unlock(&lock);
+}
+
 /*
  * Fix an inode's reflink flag.
  */
@@ -1188,9 +1218,7 @@ fix_inode_reflink_flag(
 _("setting reflink flag on inode %"PRIu64"\n"),
 			XFS_AGINO_TO_INO(mp, agno, agino));
 	else if (!no_modify) /* && !set */
-		do_warn(
-_("clearing reflink flag on inode %"PRIu64"\n"),
-			XFS_AGINO_TO_INO(mp, agno, agino));
+		warn_clearing_reflink(XFS_AGINO_TO_INO(mp, agno, agino));
 	if (no_modify)
 		return 0;
 

[PATCH 10/10] xfs_repair: add AG btree rmaps into the filesystem after syncing sb

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

In rmap_store_ag_btree_rec(), we try to reserve 16 blocks to handle
adding all the AG btree rmaps to the rmap record.  Unfortunately, at
that point in phase5 we haven't yet reinitialied sb_fdblocks, so
reserving blocks can fail if repair reconstructed the primary sb from a
secondary sb.  Even if the function succeeds, this still leads to
incorrect fdblocks because phase 5 resets sb_fdblocks after running the
rmap transactions.

To avoid all this, move the rmap_store_ag_btree_rec call to after the sb
has been reset.  xfs/350 was helpful in finding cases where xfs_repair
errored out while repairing the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 repair/phase5.c |   24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)


diff --git a/repair/phase5.c b/repair/phase5.c
index 2e18cc69..7f7d3d18 100644
--- a/repair/phase5.c
+++ b/repair/phase5.c
@@ -2233,7 +2233,6 @@ phase5_func(
 #endif
 	xfs_agblock_t	num_extents;
 	struct agi_stat	agi_stat = {0,};
-	int		error;
 
 	if (verbose)
 		do_log(_("        - agno = %d\n"), agno);
@@ -2426,14 +2425,6 @@ phase5_func(
 			finish_cursor(&fino_btree_curs);
 		finish_cursor(&bcnt_btree_curs);
 
-		/*
-		 * Put the per-AG btree rmap data into the rmapbt
-		 */
-		error = rmap_store_ag_btree_rec(mp, agno);
-		if (error)
-			do_error(
-_("unable to add AG %u reverse-mapping data to btree.\n"), agno);
-
 		/*
 		 * release the incore per-AG bno/bcnt trees so
 		 * the extent nodes can be recycled
@@ -2561,6 +2552,21 @@ phase5(xfs_mount_t *mp)
 	 */
 	sync_sb(mp);
 
+	/*
+	 * Put the per-AG btree rmap data into the rmapbt now that we've reset
+	 * the superblock counters.
+	 */
+	for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) {
+		error = rmap_store_ag_btree_rec(mp, agno);
+		if (error)
+			do_error(
+_("unable to add AG %u reverse-mapping data to btree.\n"), agno);
+	}
+
+	/*
+	 * Put blocks that were unnecessarily reserved for btree
+	 * reconstruction back into the filesystem free space data.
+	 */
 	error = inject_lost_blocks(mp, lost_fsb);
 	if (error)
 		do_error(_("Unable to reinsert lost blocks into filesystem.\n"));

Re: [PATCH 01/10] man: document the new v5 fs geometry ioctl structures

[Eric Sandeen]

On 9/3/19 11:37 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Amend the fs geometry ioctl documentation to cover the new v5 structure.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Seems fine; I wonder if we should have a some details in here about how
we rev this interface as it's a bit unusual, but this follows history
I guess.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  man/man2/ioctl_xfs_fsop_geometry.2 |    8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> 
> diff --git a/man/man2/ioctl_xfs_fsop_geometry.2 b/man/man2/ioctl_xfs_fsop_geometry.2
> index 68e3387d..365bda8b 100644
> --- a/man/man2/ioctl_xfs_fsop_geometry.2
> +++ b/man/man2/ioctl_xfs_fsop_geometry.2
> @@ -12,6 +12,8 @@ ioctl_xfs_fsop_geometry \- report XFS filesystem layout and features
>  .PP
>  .BI "int ioctl(int " fd ", XFS_IOC_FSOP_GEOMETRY, struct xfs_fsop_geom*" arg );
>  .br
> +.BI "int ioctl(int " fd ", XFS_IOC_FSOP_GEOMETRY_V4, struct xfs_fsop_geom_v4 *" arg );
> +.br
>  .BI "int ioctl(int " fd ", XFS_IOC_FSOP_GEOMETRY_V1, struct xfs_fsop_geom_v1 *" arg );
>  .SH DESCRIPTION
>  Report the details of an XFS filesystem layout, features, and other descriptive items.
> @@ -43,6 +45,9 @@ struct xfs_fsop_geom {
>  	/* struct xfs_fsop_geom_v1 stops here. */
>  
>  	__u32         logsunit;
> +	/* struct xfs_fsop_geom_v4 stops here. */
> +
> +	__u64         reserved[18];
>  };
>  .fi
>  .in
> @@ -124,6 +129,9 @@ underlying log device, in filesystem blocks.
>  This field is meaningful only if the flag
>  .B  XFS_FSOP_GEOM_FLAGS_LOGV2
>  is set.
> +.PP
> +.I reserved
> +is set to zero.
>  .SH FILESYSTEM FEATURE FLAGS
>  Filesystem features are reported to userspace as a combination the following
>  flags:
> 

Re: [PATCH 02/10] man: document new fs summary counter scrub command

[Eric Sandeen]

On 9/3/19 11:37 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Update the scrub ioctl documentation to include the new fs summary
> counter scrubber.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  man/man2/ioctl_xfs_scrub_metadata.2 |    5 +++++
>  1 file changed, 5 insertions(+)
> 
> 
> diff --git a/man/man2/ioctl_xfs_scrub_metadata.2 b/man/man2/ioctl_xfs_scrub_metadata.2
> index 1e80ee71..046e3e36 100644
> --- a/man/man2/ioctl_xfs_scrub_metadata.2
> +++ b/man/man2/ioctl_xfs_scrub_metadata.2
> @@ -159,6 +159,11 @@ corruption.
>  .TP
>  .B XFS_SCRUB_TYPE_PQUOTA
>  Examine all user, group, or project quota records for corruption.
> +
> +.TP
> +.B XFS_SCRUB_TYPE_FSCOUNTERS
> +Examine all filesystem summary counters (free blocks, inode count, free inode
> +count) for errors.
>  .RE
>  
>  .PD 1
> 

Re: [PATCH 03/10] man: document the new allocation group geometry ioctl

[Eric Sandeen]

On 9/3/19 11:37 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Document the new ioctl to describe an allocation group's geometry.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Ok - it's kind of interesting to get back EFSCORRUPTED failure from an
ioctl that is designed to report corruption details, but that detail can
be fleshed out more later.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  man/man2/ioctl_xfs_ag_geometry.2 |   82 ++++++++++++++++++++++++++++++++++++++
>  man/man3/xfsctl.3                |    6 +++
>  2 files changed, 88 insertions(+)
>  create mode 100644 man/man2/ioctl_xfs_ag_geometry.2
> 
> 
> diff --git a/man/man2/ioctl_xfs_ag_geometry.2 b/man/man2/ioctl_xfs_ag_geometry.2
> new file mode 100644
> index 00000000..ddd54265
> --- /dev/null
> +++ b/man/man2/ioctl_xfs_ag_geometry.2
> @@ -0,0 +1,82 @@
> +.\" Copyright (c) 2019, Oracle.  All rights reserved.
> +.\"
> +.\" %%%LICENSE_START(GPLv2+_DOC_FULL)
> +.\" SPDX-License-Identifier: GPL-2.0+
> +.\" %%%LICENSE_END
> +.TH IOCTL-XFS-AG-GEOMETRY 2 2019-08-30 "XFS"
> +.SH NAME
> +ioctl_xfs_ag_geometry \- query XFS allocation group geometry information
> +.SH SYNOPSIS
> +.br
> +.B #include <xfs/xfs_fs.h>
> +.PP
> +.BI "int ioctl(int " fd ", XFS_IOC_AG_GEOMETRY, struct xfs_ag_geometry *" arg );
> +.SH DESCRIPTION
> +This XFS ioctl retrieves the geometry information for a given allocation group.
> +The geometry information is conveyed in a structure of the following form:
> +.PP
> +.in +4n
> +.nf
> +struct xfs_ag_geometry {
> +	uint32_t  ag_number;
> +	uint32_t  ag_length;
> +	uint32_t  ag_freeblks;
> +	uint32_t  ag_icount;
> +	uint32_t  ag_ifree;
> +	uint32_t  ag_sick;
> +	uint32_t  ag_checked;
> +	uint32_t  ag_flags;
> +	uint64_t  ag_reserved[12];
> +};
> +.fi
> +.in
> +.TP
> +.I ag_number
> +The caller must set this field to the index of the allocation group that the
> +caller wishes to learn about.
> +.TP
> +.I ag_length
> +The length of the allocation group is returned in this field, in units of
> +filesystem blocks.
> +.TP
> +.I ag_freeblks
> +The number of free blocks in the allocation group is returned in this field, in
> +units of filesystem blocks.
> +.TP
> +.I ag_icount
> +The number of inode records allocated in this allocation group is returned in
> +this field.
> +.TP
> +.I ag_ifree
> +The number of unused inode records (of the space allocated) in this allocation
> +group is returned in this field.
> +.TP
> +.I ag_flags
> +The caller can set this field to change the operational behavior of the ioctl.
> +Currently no flags are defined, so this field must be zero.
> +.TP
> +.IR ag_reserved
> +All reserved fields will be set to zero on return.
> +.SH RETURN VALUE
> +On error, \-1 is returned, and
> +.I errno
> +is set to indicate the error.
> +.PP
> +.SH ERRORS
> +Error codes can be one of, but are not limited to, the following:
> +.TP
> +.B EFSBADCRC
> +Metadata checksum validation failed while performing the query.
> +.TP
> +.B EFSCORRUPTED
> +Metadata corruption was encountered while performing the query.
> +.TP
> +.B EINVAL
> +The specified allocation group number is not valid for this filesystem.
> +.TP
> +.B EIO
> +An I/O error was encountered while performing the query.
> +.SH CONFORMING TO
> +This API is specific to XFS filesystem on the Linux kernel.
> +.SH SEE ALSO
> +.BR ioctl (2)
> diff --git a/man/man3/xfsctl.3 b/man/man3/xfsctl.3
> index 7e6588b8..dfebd12d 100644
> --- a/man/man3/xfsctl.3
> +++ b/man/man3/xfsctl.3
> @@ -336,6 +336,12 @@ See
>  .BR ioctl_xfs_fsop_geometry (2)
>  for more information.
>  
> +.TP
> +.B XFS_IOC_AG_GEOMETRY
> +See
> +.BR ioctl_xfs_ag_geometry (2)
> +for more information.
> +
>  .TP
>  .BR XFS_IOC_FSBULKSTAT " or " XFS_IOC_FSBULKSTAT_SINGLE
>  See
> 

Re: [PATCH 04/10] man: document the new health reporting fields in various ioctls

[Eric Sandeen]

On 9/3/19 11:37 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Update the manpages to conver the new health reporting fields in the
> fs geometry, ag geometry, and bulkstat ioctls.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Reviewed-by: Dave Chinner <dchinner@redhat.com>
> ---
>  man/man2/ioctl_xfs_ag_geometry.2   |   48 +++++++++++++++++++++++++++++++
>  man/man2/ioctl_xfs_fsbulkstat.2    |   52 +++++++++++++++++++++++++++++++++
>  man/man2/ioctl_xfs_fsop_geometry.2 |   56 +++++++++++++++++++++++++++++++++++-
>  3 files changed, 154 insertions(+), 2 deletions(-)
> 
> 
...

> diff --git a/man/man2/ioctl_xfs_fsbulkstat.2 b/man/man2/ioctl_xfs_fsbulkstat.2
> index a8b22dc4..3e13cfa8 100644
> --- a/man/man2/ioctl_xfs_fsbulkstat.2
> +++ b/man/man2/ioctl_xfs_fsbulkstat.2
> @@ -94,7 +94,9 @@ struct xfs_bstat {
>  	__u16             bs_projid_lo;
>  	__u16             bs_forkoff;
>  	__u16             bs_projid_hi;
> -	unsigned char     bs_pad[6];
> +	uint16_t          bs_sick;
> +	uint16_t          bs_checked;
> +	unsigned char     bs_pad[2];
>  	__u32             bs_cowextsize;
>  	__u32             bs_dmevmask;
>  	__u16             bs_dmstate;
> @@ -184,6 +186,54 @@ is unused on Linux.
>  .I bs_aextents
>  is the number of storage mappings associated with this file's extended
>  attributes.
> +.PP
> +The fields
> +.IR bs_sick " and " bs_checked
> +indicate the relative health of various allocation group metadata:


This should probably say "inode metadata?" 

I can fix that on the way in, the rest looks ok

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

Re: [PATCH 05/10] xfs_db: remove db/convert.h

[Eric Sandeen]

On 9/3/19 11:37 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> db/convert.h conflicts with include/convert.h and since the former only
> has one declaration in it anyway, just get rid of it.  We'll need this
> in the next patch to avoid an ugly include mess.

larf ok :)

> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Reviewed-by: Dave Chinner <dchinner@redhat.com>

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  db/Makefile  |    4 ++--
>  db/command.c |    1 -
>  db/command.h |    1 +
>  db/convert.c |    1 -
>  db/convert.h |    7 -------
>  5 files changed, 3 insertions(+), 11 deletions(-)
>  delete mode 100644 db/convert.h
> 
> 
> diff --git a/db/Makefile b/db/Makefile
> index 8fecfc1c..0941b32e 100644
> --- a/db/Makefile
> +++ b/db/Makefile
> @@ -8,13 +8,13 @@ include $(TOPDIR)/include/builddefs
>  LTCOMMAND = xfs_db
>  
>  HFILES = addr.h agf.h agfl.h agi.h attr.h attrshort.h bit.h block.h bmap.h \
> -	btblock.h bmroot.h check.h command.h convert.h crc.h debug.h \
> +	btblock.h bmroot.h check.h command.h crc.h debug.h \
>  	dir2.h dir2sf.h dquot.h echo.h faddr.h field.h \
>  	flist.h fprint.h frag.h freesp.h hash.h help.h init.h inode.h input.h \
>  	io.h logformat.h malloc.h metadump.h output.h print.h quit.h sb.h \
>  	sig.h strvec.h text.h type.h write.h attrset.h symlink.h fsmap.h \
>  	fuzz.h
> -CFILES = $(HFILES:.h=.c) btdump.c info.c
> +CFILES = $(HFILES:.h=.c) btdump.c convert.c info.c
>  LSRCFILES = xfs_admin.sh xfs_ncheck.sh xfs_metadump.sh
>  
>  LLDLIBS	= $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD)
> diff --git a/db/command.c b/db/command.c
> index c7c52342..89a78f03 100644
> --- a/db/command.c
> +++ b/db/command.c
> @@ -11,7 +11,6 @@
>  #include "bmap.h"
>  #include "check.h"
>  #include "command.h"
> -#include "convert.h"
>  #include "debug.h"
>  #include "type.h"
>  #include "echo.h"
> diff --git a/db/command.h b/db/command.h
> index eacfd465..2f9a7e16 100644
> --- a/db/command.h
> +++ b/db/command.h
> @@ -28,5 +28,6 @@ extern int		command(int argc, char **argv);
>  extern const cmdinfo_t	*find_command(const char *cmd);
>  extern void		init_commands(void);
>  
> +extern void		convert_init(void);
>  extern void		btdump_init(void);
>  extern void		info_init(void);
> diff --git a/db/convert.c b/db/convert.c
> index 01a08823..e1466057 100644
> --- a/db/convert.c
> +++ b/db/convert.c
> @@ -6,7 +6,6 @@
>  
>  #include "libxfs.h"
>  #include "command.h"
> -#include "convert.h"
>  #include "output.h"
>  #include "init.h"
>  
> diff --git a/db/convert.h b/db/convert.h
> deleted file mode 100644
> index 3660cabe..00000000
> --- a/db/convert.h
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * Copyright (c) 2000-2001,2005 Silicon Graphics, Inc.
> - * All Rights Reserved.
> - */
> -
> -extern void	convert_init(void);
> 

Re: [PATCH 10/10] xfs_repair: add AG btree rmaps into the filesystem after syncing sb

[Eric Sandeen]

On 9/3/19 11:38 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> In rmap_store_ag_btree_rec(), we try to reserve 16 blocks to handle
> adding all the AG btree rmaps to the rmap record.  Unfortunately, at
> that point in phase5 we haven't yet reinitialied sb_fdblocks, so
> reserving blocks can fail if repair reconstructed the primary sb from a
> secondary sb.  Even if the function succeeds, this still leads to
> incorrect fdblocks because phase 5 resets sb_fdblocks after running the
> rmap transactions.
> 
> To avoid all this, move the rmap_store_ag_btree_rec call to after the sb
> has been reset.  xfs/350 was helpful in finding cases where xfs_repair
> errored out while repairing the filesystem.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

makes sense to me; would be nice to know what this failure looks like at
repair time tho, for inclusion in the changelog, perhaps.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  repair/phase5.c |   24 +++++++++++++++---------
>  1 file changed, 15 insertions(+), 9 deletions(-)
> 
> 
> diff --git a/repair/phase5.c b/repair/phase5.c
> index 2e18cc69..7f7d3d18 100644
> --- a/repair/phase5.c
> +++ b/repair/phase5.c
> @@ -2233,7 +2233,6 @@ phase5_func(
>  #endif
>  	xfs_agblock_t	num_extents;
>  	struct agi_stat	agi_stat = {0,};
> -	int		error;
>  
>  	if (verbose)
>  		do_log(_("        - agno = %d\n"), agno);
> @@ -2426,14 +2425,6 @@ phase5_func(
>  			finish_cursor(&fino_btree_curs);
>  		finish_cursor(&bcnt_btree_curs);
>  
> -		/*
> -		 * Put the per-AG btree rmap data into the rmapbt
> -		 */
> -		error = rmap_store_ag_btree_rec(mp, agno);
> -		if (error)
> -			do_error(
> -_("unable to add AG %u reverse-mapping data to btree.\n"), agno);
> -
>  		/*
>  		 * release the incore per-AG bno/bcnt trees so
>  		 * the extent nodes can be recycled
> @@ -2561,6 +2552,21 @@ phase5(xfs_mount_t *mp)
>  	 */
>  	sync_sb(mp);
>  
> +	/*
> +	 * Put the per-AG btree rmap data into the rmapbt now that we've reset
> +	 * the superblock counters.
> +	 */
> +	for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) {
> +		error = rmap_store_ag_btree_rec(mp, agno);
> +		if (error)
> +			do_error(
> +_("unable to add AG %u reverse-mapping data to btree.\n"), agno);
> +	}
> +
> +	/*
> +	 * Put blocks that were unnecessarily reserved for btree
> +	 * reconstruction back into the filesystem free space data.
> +	 */
>  	error = inject_lost_blocks(mp, lost_fsb);
>  	if (error)
>  		do_error(_("Unable to reinsert lost blocks into filesystem.\n"));
> 

Re: [PATCH 04/10] man: document the new health reporting fields in various ioctls

[Darrick J. Wong]

On Mon, Sep 09, 2019 at 02:19:42PM -0500, Eric Sandeen wrote:
> On 9/3/19 11:37 PM, Darrick J. Wong wrote:
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > 
> > Update the manpages to conver the new health reporting fields in the
> > fs geometry, ag geometry, and bulkstat ioctls.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > Reviewed-by: Dave Chinner <dchinner@redhat.com>
> > ---
> >  man/man2/ioctl_xfs_ag_geometry.2   |   48 +++++++++++++++++++++++++++++++
> >  man/man2/ioctl_xfs_fsbulkstat.2    |   52 +++++++++++++++++++++++++++++++++
> >  man/man2/ioctl_xfs_fsop_geometry.2 |   56 +++++++++++++++++++++++++++++++++++-
> >  3 files changed, 154 insertions(+), 2 deletions(-)
> > 
> > 
> ...
> 
> > diff --git a/man/man2/ioctl_xfs_fsbulkstat.2 b/man/man2/ioctl_xfs_fsbulkstat.2
> > index a8b22dc4..3e13cfa8 100644
> > --- a/man/man2/ioctl_xfs_fsbulkstat.2
> > +++ b/man/man2/ioctl_xfs_fsbulkstat.2
> > @@ -94,7 +94,9 @@ struct xfs_bstat {
> >  	__u16             bs_projid_lo;
> >  	__u16             bs_forkoff;
> >  	__u16             bs_projid_hi;
> > -	unsigned char     bs_pad[6];
> > +	uint16_t          bs_sick;
> > +	uint16_t          bs_checked;
> > +	unsigned char     bs_pad[2];
> >  	__u32             bs_cowextsize;
> >  	__u32             bs_dmevmask;
> >  	__u16             bs_dmstate;
> > @@ -184,6 +186,54 @@ is unused on Linux.
> >  .I bs_aextents
> >  is the number of storage mappings associated with this file's extended
> >  attributes.
> > +.PP
> > +The fields
> > +.IR bs_sick " and " bs_checked
> > +indicate the relative health of various allocation group metadata:
> 
> 
> This should probably say "inode metadata?" 

Yes it should have, thanks for fixing that.

--D

> I can fix that on the way in, the rest looks ok
> 
> Reviewed-by: Eric Sandeen <sandeen@redhat.com>
>