[PATCH 00/42] mkfs: factor the crap out of the code

[PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

Everyone who tries to modify mkfs quickly learns that it is a pile
of spaghetti, the only difference in opinion is whether it is a
steaming, cold or rotten pile. This patchset attempts to untangle
the ball of pasta and turn it into a set of clear, obvious
operations that lead to a filesystem being formatted correctly.

The patch series is really in three parts, splitting the code up
into roughly three modules. The first part introduces a mkfs
parameters structure and factors the on-disk formatting code to use
only information in that structure. The second part introduces a
command line input structure and factors the input parsing to use
it. This requires a bunch of temporary code to keep the rest of
the code working. The third part is factoring the input validation
and geometry calculation code to use the input structure and put
the output into the mkfs parameter structure and to remove all the
temporary support code.

The result is three modules - input parsing, validation+calculations
and formatting - with well defined data flow between them. This also
paves the way to supporting config files to set defaults via a
separate (new) module. The overall data flow now looks like this:

Build defaults --\
                  ---> mkfs_default_params -> CLI -> mkfs_params
config file -----/

It is not worth spending a lot of time reviewing the temporary code
that is added - it gets removed before the end of the series. No
attempt has been made to ensure that mkfs works 100% correctly after
each patch is applied - the only guarantee is that it will build
cleanly. It /should/ work if a bisect lands in the middle of the
series, but trying to exhaustively test each patch is OK would take
more effort than it is worth. As such, testing has only been
performed on the whole series.

The new output from mkfs to indicate where it has sourced the
defaults from causes xfstests to have conniptions. This requires
some updates to the mkfs output filters that are already in place
but it is a fairly trivial update. Test xfs/191 has a couple of new
failures, but that is because the new code now correctly parses
things like agsize so that block and sector size based
specifications work with default mkfs values. This will require test
updates.

Future work will be to split the xfs_mkfs.c file into a file per
module (i.e. seperate files for CLI parsing, mkfs formating,
validation+calculation and, finally, one for config file support),
but otherwise the majority of the factoring work is now complete.

Comments, flames, etc all welcome.

-Dave.

Overall Diffstat:
 include/libxfs.h |    2 
 mkfs/xfs_mkfs.c  | 4615 ++++++++++++++++++++++++++++++-------------------------
 2 files changed, 2578 insertions(+), 2039 deletions(-)

[PATCH 01/42] mkfs: can't specify sector size of internal log

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Because it's fixed by the underlying device size.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 7bb6408f9b77..e16e73f86cbc 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -428,6 +428,8 @@ struct opt_params lopts = {
 		{ .index = L_INTERNAL,
 		  .conflicts = { L_FILE,
 				 L_DEV,
+				 L_SECTLOG,
+				 L_SECTSIZE,
 				 LAST_CONFLICT },
 		  .minval = 0,
 		  .maxval = 1,
@@ -469,6 +471,7 @@ struct opt_params lopts = {
 		},
 		{ .index = L_SECTLOG,
 		  .conflicts = { L_SECTSIZE,
+				 L_INTERNAL,
 				 LAST_CONFLICT },
 		  .minval = XFS_MIN_SECTORSIZE_LOG,
 		  .maxval = XFS_MAX_SECTORSIZE_LOG,
@@ -476,6 +479,7 @@ struct opt_params lopts = {
 		},
 		{ .index = L_SECTSIZE,
 		  .conflicts = { L_SECTLOG,
+				 L_INTERNAL,
 				 LAST_CONFLICT },
 		  .convert = true,
 		  .is_power_2 = true,
-- 
2.13.3

[PATCH 02/42] mkfs: make subopt table const

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Use const for all the tables to remove most of the (char **) casts.
This adds a couple of temporary (const char **) casts that go away
as the input parsing is factored.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index e16e73f86cbc..43f631a647b4 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -24,11 +24,11 @@
 /*
  * Prototypes for internal functions.
  */
-static void conflict(char opt, char *tab[], int oldidx, int newidx);
+static void conflict(char opt, const char *tab[], int oldidx, int newidx);
 static void illegal(const char *value, const char *opt);
 static __attribute__((noreturn)) void usage (void);
-static __attribute__((noreturn)) void reqval(char opt, char *tab[], int idx);
-static void respec(char opt, char *tab[], int idx);
+static __attribute__((noreturn)) void reqval(char opt, const char *tab[], int idx);
+static void respec(char opt, const char *tab[], int idx);
 static void unknown(char opt, char *s);
 static int  ispow2(unsigned int i);
 
@@ -1287,7 +1287,7 @@ check_opt(
 		fprintf(stderr,
 	("Developer screwed up option parsing (%d/%d)! Please report!\n"),
 			sp->index, index);
-		reqval(opts->name, (char **)opts->subopts, index);
+		reqval(opts->name, opts->subopts, index);
 	}
 
 	/*
@@ -1300,11 +1300,11 @@ check_opt(
 	 */
 	if (!str_seen) {
 		if (sp->seen)
-			respec(opts->name, (char **)opts->subopts, index);
+			respec(opts->name, opts->subopts, index);
 		sp->seen = true;
 	} else {
 		if (sp->str_seen)
-			respec(opts->name, (char **)opts->subopts, index);
+			respec(opts->name, opts->subopts, index);
 		sp->str_seen = true;
 	}
 
@@ -1316,7 +1316,7 @@ check_opt(
 			break;
 		if (opts->subopt_params[conflict_opt].seen ||
 		    opts->subopt_params[conflict_opt].str_seen)
-			conflict(opts->name, (char **)opts->subopts,
+			conflict(opts->name, opts->subopts,
 				 conflict_opt, index);
 	}
 }
@@ -1334,7 +1334,7 @@ getnum(
 	/* empty strings might just return a default value */
 	if (!str || *str == '\0') {
 		if (sp->defaultval == SUBOPT_NEEDS_VAL)
-			reqval(opts->name, (char **)opts->subopts, index);
+			reqval(opts->name, opts->subopts, index);
 		return sp->defaultval;
 	}
 
@@ -1390,7 +1390,7 @@ getstr(
 
 	/* empty strings for string options are not valid */
 	if (!str || *str == '\0')
-		reqval(opts->name, (char **)opts->subopts, index);
+		reqval(opts->name, opts->subopts, index);
 	return str;
 }
 
@@ -1786,7 +1786,7 @@ main(
 					break;
 				case M_UUID:
 					if (!value || *value == '\0')
-						reqval('m', subopts, M_UUID);
+						reqval('m', (const char **)subopts, M_UUID);
 					if (platform_uuid_parse(value, &uuid))
 						illegal(optarg, "m uuid");
 					break;
@@ -1900,7 +1900,7 @@ main(
 				case S_LOG:
 				case S_SECTLOG:
 					if (lssflag)
-						conflict('s', subopts,
+						conflict('s', (const char **)subopts,
 							 S_SECTSIZE, S_SECTLOG);
 					sectorlog = getnum(value, &sopts,
 							   S_SECTLOG);
@@ -1912,7 +1912,7 @@ main(
 				case S_SIZE:
 				case S_SECTSIZE:
 					if (lslflag)
-						conflict('s', subopts, S_SECTLOG,
+						conflict('s', (const char **)subopts, S_SECTLOG,
 							 S_SECTSIZE);
 					sectorsize = getnum(value, &sopts,
 							    S_SECTSIZE);
@@ -3368,7 +3368,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 static void
 conflict(
 	char		opt,
-	char		*tab[],
+	const char	*tab[],
 	int		oldidx,
 	int		newidx)
 {
@@ -3397,7 +3397,7 @@ ispow2(
 static void __attribute__((noreturn))
 reqval(
 	char		opt,
-	char		*tab[],
+	const char	*tab[],
 	int		idx)
 {
 	fprintf(stderr, _("-%c %s option requires a value\n"), opt, tab[idx]);
@@ -3407,7 +3407,7 @@ reqval(
 static void
 respec(
 	char		opt,
-	char		*tab[],
+	const char	*tab[],
 	int		idx)
 {
 	fprintf(stderr, "-%c ", opt);
-- 
2.13.3

[PATCH 03/42] mkfs: introduce a structure to hold CLI options

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

We need to hold the values set from command line options so they can
later be validated and discriminated from the default values that
might be set. This structure will form a connector between th einput
parsing and the rest of the mkfs code.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 include/libxfs.h |   2 +-
 mkfs/xfs_mkfs.c  | 107 ++++++++++++++++++++++++++++++++++++++++++++++---------
 2 files changed, 91 insertions(+), 18 deletions(-)

diff --git a/include/libxfs.h b/include/libxfs.h
index e5e152378c2c..b77819b1adcc 100644
--- a/include/libxfs.h
+++ b/include/libxfs.h
@@ -95,7 +95,7 @@ extern uint32_t crc32c_le(uint32_t crc, unsigned char const *p, size_t len);
 /*
  * Argument structure for libxfs_init().
  */
-typedef struct {
+typedef struct libxfs_xinit {
 				/* input parameters */
 	char            *volname;       /* pathname of volume */
 	char            *dname;         /* pathname of data "subvolume" */
diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 43f631a647b4..2b264ec32974 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -709,6 +709,96 @@ struct opt_params mopts = {
 	},
 };
 
+/* quick way of checking is a parameter was set on the CLI */
+static bool
+cli_opt_set(
+	struct opt_params	*opts,
+	int			subopt)
+{
+	return opts->subopt_params[subopt].seen ||
+	       opts->subopt_params[subopt].str_seen;
+}
+
+/*
+ * Options configured on the command line.
+ *
+ * This stores all the specific config parameters the user sets on the command
+ * line. We do not use these values directly - they are inputs to the mkfs
+ * geometry validation and override any default configuration value we have.
+ *
+ * We don't keep flags to indicate what parameters are set - it is invalid to
+ * set values of -1 for any input, so we initialise values that can be set to
+ * zero to this, otherwise they are initialised to zero. Some parameters are
+ * stored as strings for later parsing (e.g. after block size and sector size
+ * have been validated), so they get initialised to NULL.
+ *
+ * This allows us to check that values have been set without needing separate
+ * flags for each value, and hence avoids needing to record and check for each
+ * specific option that can set the value later on in the code. In the cases
+ * where we don't have a cli_params structure around, the above cli_opt_set()
+ * function can be used.
+ */
+struct sb_feat_args {
+	int	log_version;
+	int	attr_version;
+	int	dir_version;
+	bool	inode_align;
+	bool	nci;
+	bool	lazy_sb_counters;
+	bool	projid16bit;
+	bool	crcs_enabled;
+	bool	dirftype;
+	bool	finobt;
+	bool	spinodes;
+	bool	rmapbt;
+	bool	reflink;
+	bool	parent_pointers;
+	bool	nodalign;
+	bool	nortalign;
+	uuid_t	m_uuid;
+};
+
+struct cli_params {
+	int	sectorsize;
+	int	blocksize;
+
+	/* parameters that depend on sector/block size being validated. */
+	char	*dsize;
+	char	*agsize;
+	char	*dsu;
+	char	*dirblocksize;
+	char	*logsize;
+	char	*lsu;
+	char	*rtextsize;
+	char	*rtsize;
+
+	/* parameters where 0 is a valid CLI value */
+	int	dsunit;
+	int	dswidth;
+	int	dsw;
+	int64_t	logagno;
+	int	loginternal;
+	int	lsunit;
+
+	/* parameters where 0 is not a valid value */
+	int64_t	agcount;
+	int	dirblocklog;
+	int	inodesize;
+	int	inopblock;
+	int	imaxpct;
+	int	lsectorsize;
+	uuid_t	uuid;
+
+	/* feature flags that are set */
+	struct sb_feat_args	sb_feat;
+
+	/* root inode characteristics */
+	struct fsxattr		fsx;
+
+	/* libxfs device setup */
+	struct libxfs_xinit	*xi;
+};
+
 #define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
 #define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
 #define MEGABYTES(count, blog)	((uint64_t)(count) << (20 - (blog)))
@@ -1150,23 +1240,6 @@ discard_blocks(dev_t dev, uint64_t nsectors)
 		platform_discard_blocks(fd, 0, nsectors << 9);
 }
 
-struct sb_feat_args {
-	int	log_version;
-	int	attr_version;
-	int	dir_version;
-	int	spinodes;
-	int	finobt;
-	bool	inode_align;
-	bool	nci;
-	bool	lazy_sb_counters;
-	bool	projid16bit;
-	bool	crcs_enabled;
-	bool	dirftype;
-	bool	parent_pointers;
-	bool	rmapbt;
-	bool	reflink;
-};
-
 static void
 sb_set_features(
 	struct xfs_sb		*sbp,
-- 
2.13.3

[PATCH 04/42] mkfs: add generic subopt parsing table

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Abstract out the common subopt parsing code into a common function
and type table so we can factor the parsing code. Add the function
stubs in preparation for factoring.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 130 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 2b264ec32974..022fb84016f6 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1467,6 +1467,136 @@ getstr(
 	return str;
 }
 
+static int
+block_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+data_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+inode_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+log_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+meta_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+naming_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+rtdev_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+static int
+sector_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	return 0;
+}
+
+struct subopts {
+	char		opt;
+	struct opt_params *opts;
+	int		(*parser)();
+} subopt_tab[] = {
+	{ 'b', &bopts, block_opts_parser },
+	{ 'd', &dopts, data_opts_parser },
+	{ 'i', &iopts, inode_opts_parser },
+	{ 'l', &lopts, log_opts_parser },
+	{ 'm', &mopts, meta_opts_parser },
+	{ 'n', &nopts, naming_opts_parser },
+	{ 'r', &ropts, rtdev_opts_parser },
+	{ 's', &sopts, sector_opts_parser },
+	{ '\0', NULL, NULL },
+};
+
+static void
+parse_subopts(
+	char		opt,
+	char		*arg,
+	struct cli_params *cli)
+{
+	struct subopts	*sop = &subopt_tab[0];
+	char		*p;
+	int		ret = 0;
+
+	while (sop->opts) {
+		if (sop->opt == opt)
+			break;
+		sop++;
+	}
+
+	/* should never happen */
+	if (!sop->opts)
+		return;
+
+	p = arg;
+	while (*p != '\0') {
+		char	**subopts = (char **)sop->opts->subopts;
+		char	*value;
+		int	subopt;
+
+		subopt = getsubopt(&p, subopts, &value);
+
+		ret = (sop->parser)(sop->opts, subopt, value, cli);
+		if (ret)
+			unknown(opt, value);
+	}
+}
+
 int
 main(
 	int			argc,
-- 
2.13.3

[PATCH 05/42] mkfs: factor block subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 43 ++++++++++++++++++++++---------------------
 1 file changed, 22 insertions(+), 21 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 022fb84016f6..40de7ea9bfa8 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1474,6 +1474,19 @@ block_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	int			blocklog;
+
+	switch (subopt) {
+	case B_LOG:
+		blocklog = getnum(value, opts, B_LOG);
+		cli->blocksize = 1 << blocklog;
+		break;
+	case B_SIZE:
+		cli->blocksize = getnum(value, opts, B_SIZE);
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -1706,6 +1719,7 @@ main(
 		.rmapbt = false,
 		.reflink = false,
 	};
+	struct cli_params	cli = {};
 
 	platform_uuid_generate(&uuid);
 	progname = basename(argv[0]);
@@ -1745,27 +1759,14 @@ main(
 			force_overwrite = 1;
 			break;
 		case 'b':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)bopts.subopts;
-				char	*value;
-
-				switch (getsubopt(&p, subopts, &value)) {
-				case B_LOG:
-					blocklog = getnum(value, &bopts, B_LOG);
-					blocksize = 1 << blocklog;
-					blflag = 1;
-					break;
-				case B_SIZE:
-					blocksize = getnum(value, &bopts,
-							   B_SIZE);
-					blocklog = libxfs_highbit32(blocksize);
-					bsflag = 1;
-					break;
-				default:
-					unknown('b', value);
-				}
-			}
+			parse_subopts(c, optarg, &cli);
+
+			/* temp don't break code */
+			blocksize = cli.blocksize;
+			blocklog = libxfs_highbit32(blocksize);
+			blflag = cli_opt_set(&bopts, B_LOG);
+			bsflag = cli_opt_set(&bopts, B_SIZE);
+			/* end temp don't break code */
 			break;
 		case 'd':
 			p = optarg;
-- 
2.13.3

[PATCH 06/42] mkfs: factor data subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 171 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 91 insertions(+), 80 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 40de7ea9bfa8..c92abc9a6bf3 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1497,6 +1497,65 @@ data_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	int			sectorlog;
+
+	switch (subopt) {
+	case D_AGCOUNT:
+		cli->agcount = getnum(value, opts, D_AGCOUNT);
+		break;
+	case D_AGSIZE:
+		cli->agsize = getstr(value, opts, D_AGSIZE);
+		break;
+	case D_FILE:
+		cli->xi->disfile = getnum(value, opts, D_FILE);
+		break;
+	case D_NAME:
+		cli->xi->dname = getstr(value, opts, D_NAME);
+		break;
+	case D_SIZE:
+		cli->dsize = getstr(value, opts, D_SIZE);
+		break;
+	case D_SUNIT:
+		cli->dsunit = getnum(value, opts, D_SUNIT);
+		break;
+	case D_SWIDTH:
+		cli->dswidth = getnum(value, opts, D_SWIDTH);
+		break;
+	case D_SU:
+		cli->dsu = getstr(value, opts, D_SU);
+		break;
+	case D_SW:
+		cli->dsw = getnum(value, opts, D_SW);
+		break;
+	case D_NOALIGN:
+		cli->sb_feat.nodalign = getnum(value, opts, D_NOALIGN);
+		break;
+	case D_SECTLOG:
+		if (cli->sectorsize)
+			conflict('d', opts->subopts, D_SECTSIZE, D_SECTLOG);
+		sectorlog = getnum(value, opts, D_SECTLOG);
+		cli->sectorsize = 1 << sectorlog;
+		break;
+	case D_SECTSIZE:
+		if (cli->sectorsize)
+			conflict('d', opts->subopts, D_SECTSIZE, D_SECTLOG);
+		cli->sectorsize = getnum(value, opts, D_SECTSIZE);
+		break;
+	case D_RTINHERIT:
+		if (getnum(value, opts, D_RTINHERIT))
+			cli->fsx.fsx_xflags |= XFS_DIFLAG_RTINHERIT;
+		break;
+	case D_PROJINHERIT:
+		cli->fsx.fsx_projid = getnum(value, opts, D_PROJINHERIT);
+		cli->fsx.fsx_xflags |= XFS_DIFLAG_PROJINHERIT;
+		break;
+	case D_EXTSZINHERIT:
+		cli->fsx.fsx_extsize = getnum(value, opts, D_EXTSZINHERIT);
+		cli->fsx.fsx_xflags |= XFS_DIFLAG_EXTSZINHERIT;
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -1719,7 +1778,9 @@ main(
 		.rmapbt = false,
 		.reflink = false,
 	};
-	struct cli_params	cli = {};
+	struct cli_params	cli = {
+		.xi = &xi,
+	};
 
 	platform_uuid_generate(&uuid);
 	progname = basename(argv[0]);
@@ -1769,87 +1830,37 @@ main(
 			/* end temp don't break code */
 			break;
 		case 'd':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)dopts.subopts;
-				char	*value;
+			parse_subopts(c, optarg, &cli);
 
-				switch (getsubopt(&p, subopts, &value)) {
-				case D_AGCOUNT:
-					agcount = getnum(value, &dopts,
-							 D_AGCOUNT);
-					daflag = 1;
-					break;
-				case D_AGSIZE:
-					agsize = getnum(value, &dopts, D_AGSIZE);
-					dasize = 1;
-					break;
-				case D_FILE:
-					xi.disfile = getnum(value, &dopts,
-							    D_FILE);
-					break;
-				case D_NAME:
-					xi.dname = getstr(value, &dopts, D_NAME);
-					break;
-				case D_SIZE:
-					dsize = getstr(value, &dopts, D_SIZE);
-					break;
-				case D_SUNIT:
-					dsunit = getnum(value, &dopts, D_SUNIT);
-					dsflag = 1;
-					break;
-				case D_SWIDTH:
-					dswidth = getnum(value, &dopts,
-							 D_SWIDTH);
-					dsflag = 1;
-					break;
-				case D_SU:
-					dsu = getnum(value, &dopts, D_SU);
-					dsflag = 1;
-					break;
-				case D_SW:
-					dsw = getnum(value, &dopts, D_SW);
-					dsflag = 1;
-					break;
-				case D_NOALIGN:
-					nodsflag = getnum(value, &dopts,
-								D_NOALIGN);
-					break;
-				case D_SECTLOG:
-					sectorlog = getnum(value, &dopts,
-							   D_SECTLOG);
-					sectorsize = 1 << sectorlog;
-					slflag = 1;
-					break;
-				case D_SECTSIZE:
-					sectorsize = getnum(value, &dopts,
-							    D_SECTSIZE);
-					sectorlog =
-						libxfs_highbit32(sectorsize);
-					ssflag = 1;
-					break;
-				case D_RTINHERIT:
-					c = getnum(value, &dopts, D_RTINHERIT);
-					if (c)
-						fsx.fsx_xflags |=
-							XFS_DIFLAG_RTINHERIT;
-					break;
-				case D_PROJINHERIT:
-					fsx.fsx_projid = getnum(value, &dopts,
-								D_PROJINHERIT);
-					fsx.fsx_xflags |=
-						XFS_DIFLAG_PROJINHERIT;
-					break;
-				case D_EXTSZINHERIT:
-					fsx.fsx_extsize = getnum(value, &dopts,
-								 D_EXTSZINHERIT);
-					fsx.fsx_xflags |=
-						XFS_DIFLAG_EXTSZINHERIT;
-					break;
-				default:
-					unknown('d', value);
-				}
+			/* temp don't break code */
+			agcount = cli.agcount;
+			if (cli_opt_set(&dopts, D_AGSIZE)) {
+				agsize = getnum(cli.agsize, &dopts, D_AGSIZE);
+				dasize = 1;
+			}
+			daflag = cli_opt_set(&dopts, D_AGCOUNT);
+
+			dsunit = cli.dsunit;
+			dswidth = cli.dswidth;
+			dsw = cli.dsw;
+			if (cli_opt_set(&dopts, D_SU)) {
+				dsu = getnum(cli.dsu, &dopts, D_SU);
+				dsflag = 1;
 			}
+			dsflag |= cli_opt_set(&dopts, D_SW) ||
+				  cli_opt_set(&dopts, D_SUNIT) ||
+				  cli_opt_set(&dopts, D_SWIDTH);
+			nodsflag = cli_opt_set(&dopts, D_NOALIGN);
+
+			sectorsize = cli.sectorsize;
+			sectorlog = libxfs_highbit32(sectorsize);
+			slflag = cli_opt_set(&dopts, D_SECTLOG);
+			ssflag = cli_opt_set(&dopts, D_SECTSIZE);
+
+			fsx.fsx_xflags |= cli.fsx.fsx_xflags;
+			fsx.fsx_projid = cli.fsx.fsx_projid;
+			fsx.fsx_extsize = cli.fsx.fsx_extsize;
+			/* end temp don't break code */
 			break;
 		case 'i':
 			p = optarg;
-- 
2.13.3

[PATCH 07/42] mkfs: factor inode subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 94 +++++++++++++++++++++++++++++----------------------------
 1 file changed, 48 insertions(+), 46 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index c92abc9a6bf3..d702010c2854 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1566,6 +1566,37 @@ inode_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	int			inodelog;
+
+	switch (subopt) {
+	case I_ALIGN:
+		cli->sb_feat.inode_align = getnum(value, &iopts, I_ALIGN);
+		break;
+	case I_LOG:
+		inodelog = getnum(value, &iopts, I_LOG);
+		cli->inodesize = 1 << inodelog;
+		break;
+	case I_MAXPCT:
+		cli->imaxpct = getnum(value, &iopts, I_MAXPCT);
+		break;
+	case I_PERBLOCK:
+		cli->inopblock = getnum(value, &iopts, I_PERBLOCK);
+		break;
+	case I_SIZE:
+		cli->inodesize = getnum(value, &iopts, I_SIZE);
+		break;
+	case I_ATTR:
+		cli->sb_feat.attr_version = getnum(value, &iopts, I_ATTR);
+		break;
+	case I_PROJID32BIT:
+		cli->sb_feat.projid16bit = !getnum(value, &iopts, I_PROJID32BIT);
+		break;
+	case I_SPINODES:
+		cli->sb_feat.spinodes = getnum(value, &iopts, I_SPINODES);
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -1780,6 +1811,7 @@ main(
 	};
 	struct cli_params	cli = {
 		.xi = &xi,
+		.sb_feat = sb_feat,
 	};
 
 	platform_uuid_generate(&uuid);
@@ -1863,53 +1895,19 @@ main(
 			/* end temp don't break code */
 			break;
 		case 'i':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)iopts.subopts;
-				char	*value;
+			parse_subopts(c, optarg, &cli);
 
-				switch (getsubopt(&p, subopts, &value)) {
-				case I_ALIGN:
-					sb_feat.inode_align = getnum(value,
-								&iopts, I_ALIGN);
-					break;
-				case I_LOG:
-					inodelog = getnum(value, &iopts, I_LOG);
-					isize = 1 << inodelog;
-					ilflag = 1;
-					break;
-				case I_MAXPCT:
-					imaxpct = getnum(value, &iopts,
-							 I_MAXPCT);
-					imflag = 1;
-					break;
-				case I_PERBLOCK:
-					inopblock = getnum(value, &iopts,
-							   I_PERBLOCK);
-					ipflag = 1;
-					break;
-				case I_SIZE:
-					isize = getnum(value, &iopts, I_SIZE);
-					inodelog = libxfs_highbit32(isize);
-					isflag = 1;
-					break;
-				case I_ATTR:
-					sb_feat.attr_version =
-						getnum(value, &iopts, I_ATTR);
-					break;
-				case I_PROJID32BIT:
-					sb_feat.projid16bit =
-						!getnum(value, &iopts,
-							I_PROJID32BIT);
-					break;
-				case I_SPINODES:
-					sb_feat.spinodes = getnum(value,
-							&iopts, I_SPINODES);
-					break;
-				default:
-					unknown('i', value);
-				}
-			}
+			/* temp don't break code */
+			isize = cli.inodesize;
+			inodelog = libxfs_highbit32(isize);
+			inopblock = cli.inopblock;
+			ilflag = cli_opt_set(&iopts, I_LOG);
+			isflag = cli_opt_set(&iopts, I_SIZE);
+			ipflag = cli_opt_set(&iopts, I_PERBLOCK);
+
+			imaxpct = cli.imaxpct;
+			imflag = cli_opt_set(&iopts, I_MAXPCT);
+			/* end temp don't break code */
 			break;
 		case 'l':
 			p = optarg;
@@ -2157,6 +2155,10 @@ main(
 	} else
 		dfile = xi.dname;
 
+	/* temp don't break code */
+	sb_feat = cli.sb_feat;
+	/* end temp don't break code */
+
 	/*
 	 * Blocksize and sectorsize first, other things depend on them
 	 * For RAID4/5/6 we want to align sector size and block size,
-- 
2.13.3

[PATCH 08/42] mkfs: factor log subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 127 +++++++++++++++++++++++++++++---------------------------
 1 file changed, 65 insertions(+), 62 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index d702010c2854..8e42fe0301fb 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1607,6 +1607,48 @@ log_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	int			lsectorlog;
+
+	switch (subopt) {
+	case L_AGNUM:
+		cli->logagno = getnum(value, &lopts, L_AGNUM);
+		break;
+	case L_FILE:
+		cli->xi->lisfile = getnum(value, &lopts, L_FILE);
+		break;
+	case L_INTERNAL:
+		cli->loginternal = getnum(value, &lopts, L_INTERNAL);
+		break;
+	case L_SU:
+		cli->lsu = getstr(value, &lopts, L_SU);
+		break;
+	case L_SUNIT:
+		cli->lsunit = getnum(value, &lopts, L_SUNIT);
+		break;
+	case L_NAME:
+	case L_DEV:
+		cli->xi->logname = getstr(value, &lopts, L_NAME);
+		cli->loginternal = 0;
+		break;
+	case L_VERSION:
+		cli->sb_feat.log_version = getnum(value, &lopts, L_VERSION);
+		break;
+	case L_SIZE:
+		cli->logsize = getstr(value, &lopts, L_SIZE);
+		break;
+	case L_SECTLOG:
+		lsectorlog = getnum(value, &lopts, L_SECTLOG);
+		cli->lsectorsize = 1 << lsectorlog;
+		break;
+	case L_SECTSIZE:
+		cli->lsectorsize = getnum(value, &lopts, L_SECTSIZE);
+		break;
+	case L_LAZYSBCNTR:
+		cli->sb_feat.lazy_sb_counters = getnum(value, &lopts, L_LAZYSBCNTR);
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -1910,70 +1952,31 @@ main(
 			/* end temp don't break code */
 			break;
 		case 'l':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)lopts.subopts;
-				char	*value;
+			parse_subopts(c, optarg, &cli);
 
-				switch (getsubopt(&p, subopts, &value)) {
-				case L_AGNUM:
-					logagno = getnum(value, &lopts, L_AGNUM);
-					laflag = 1;
-					break;
-				case L_FILE:
-					xi.lisfile = getnum(value, &lopts,
-							    L_FILE);
-					break;
-				case L_INTERNAL:
-					loginternal = getnum(value, &lopts,
-							     L_INTERNAL);
-					liflag = 1;
-					break;
-				case L_SU:
-					lsu = getnum(value, &lopts, L_SU);
-					lsuflag = 1;
-					break;
-				case L_SUNIT:
-					lsunit = getnum(value, &lopts, L_SUNIT);
-					lsunitflag = 1;
-					break;
-				case L_NAME:
-				case L_DEV:
-					logfile = getstr(value, &lopts, L_NAME);
-					xi.logname = logfile;
-					ldflag = 1;
-					loginternal = 0;
-					break;
-				case L_VERSION:
-					sb_feat.log_version =
-						getnum(value, &lopts, L_VERSION);
-					lvflag = 1;
-					break;
-				case L_SIZE:
-					logsize = getstr(value, &lopts, L_SIZE);
-					break;
-				case L_SECTLOG:
-					lsectorlog = getnum(value, &lopts,
-							    L_SECTLOG);
-					lsectorsize = 1 << lsectorlog;
-					lslflag = 1;
-					break;
-				case L_SECTSIZE:
-					lsectorsize = getnum(value, &lopts,
-							     L_SECTSIZE);
-					lsectorlog =
-						libxfs_highbit32(lsectorsize);
-					lssflag = 1;
-					break;
-				case L_LAZYSBCNTR:
-					sb_feat.lazy_sb_counters =
-							getnum(value, &lopts,
-							       L_LAZYSBCNTR);
-					break;
-				default:
-					unknown('l', value);
-				}
+			/* temp don't break code */
+			logagno = cli.logagno;
+			loginternal = cli.loginternal;
+			logfile = xi.logname;
+			logsize = cli.logsize;
+			lsectorsize = cli.lsectorsize;
+			lsectorlog = libxfs_highbit32(lsectorsize);
+
+			lsunit = cli.lsunit;
+			lsunitflag = cli_opt_set(&lopts, L_SUNIT);
+			if (cli_opt_set(&lopts, L_SU)) {
+				lsu = getnum(cli.lsu, &lopts, L_SU);
+				lsuflag = 1;
 			}
+
+			laflag = cli_opt_set(&lopts, L_AGNUM);
+			liflag = cli_opt_set(&lopts, L_INTERNAL);
+			ldflag = cli_opt_set(&lopts, L_NAME) ||
+				 cli_opt_set(&lopts, L_DEV);
+			lvflag = cli_opt_set(&lopts, L_VERSION);
+			lslflag = cli_opt_set(&lopts, L_SECTLOG);
+			lssflag = cli_opt_set(&lopts, L_SECTSIZE);
+			/* end temp don't break code */
 			break;
 		case 'L':
 			if (strlen(optarg) > sizeof(sbp->sb_fname))
-- 
2.13.3

[PATCH 09/42] mkfs: factor meta subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 61 ++++++++++++++++++++++++++-------------------------------
 1 file changed, 28 insertions(+), 33 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 8e42fe0301fb..7b967ffe6893 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1659,6 +1659,30 @@ meta_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	switch (subopt) {
+	case M_CRC:
+		cli->sb_feat.crcs_enabled = getnum(value, &mopts, M_CRC);
+		if (cli->sb_feat.crcs_enabled)
+			cli->sb_feat.dirftype = true;
+		break;
+	case M_FINOBT:
+		cli->sb_feat.finobt = getnum(value, &mopts, M_FINOBT);
+		break;
+	case M_UUID:
+		if (!value || *value == '\0')
+			reqval('m', opts->subopts, M_UUID);
+		if (platform_uuid_parse(value, &cli->uuid))
+			illegal(value, "m uuid");
+		break;
+	case M_RMAPBT:
+		cli->sb_feat.rmapbt = getnum(value, &mopts, M_RMAPBT);
+		break;
+	case M_REFLINK:
+		cli->sb_feat.reflink = getnum(value, &mopts, M_REFLINK);
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -1984,40 +2008,11 @@ main(
 			label = optarg;
 			break;
 		case 'm':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)mopts.subopts;
-				char	*value;
+			parse_subopts(c, optarg, &cli);
 
-				switch (getsubopt(&p, subopts, &value)) {
-				case M_CRC:
-					sb_feat.crcs_enabled =
-						getnum(value, &mopts, M_CRC);
-					if (sb_feat.crcs_enabled)
-						sb_feat.dirftype = true;
-					break;
-				case M_FINOBT:
-					sb_feat.finobt = getnum(
-						value, &mopts, M_FINOBT);
-					break;
-				case M_UUID:
-					if (!value || *value == '\0')
-						reqval('m', (const char **)subopts, M_UUID);
-					if (platform_uuid_parse(value, &uuid))
-						illegal(optarg, "m uuid");
-					break;
-				case M_RMAPBT:
-					sb_feat.rmapbt = getnum(
-						value, &mopts, M_RMAPBT);
-					break;
-				case M_REFLINK:
-					sb_feat.reflink = getnum(
-						value, &mopts, M_REFLINK);
-					break;
-				default:
-					unknown('m', value);
-				}
-			}
+			/* temp don't break code */
+			platform_uuid_copy(&uuid, &cli.uuid);
+			/* end temp don't break code */
 			break;
 		case 'n':
 			p = optarg;
-- 
2.13.3

[PATCH 10/42] mkfs: factor naming subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 68 +++++++++++++++++++++++++--------------------------------
 1 file changed, 30 insertions(+), 38 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 7b967ffe6893..e3ee1f1c87fc 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1693,6 +1693,28 @@ naming_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	switch (subopt) {
+	case N_LOG:
+		cli->dirblocklog = getnum(value, opts, N_LOG);
+		break;
+	case N_SIZE:
+		cli->dirblocksize = getstr(value, opts, N_SIZE);
+		break;
+	case N_VERSION:
+		value = getstr(value, &nopts, N_VERSION);
+		if (!strcasecmp(value, "ci")) {
+			/* ASCII CI mode */
+			cli->sb_feat.nci = true;
+		} else {
+			cli->sb_feat.dir_version = getnum(value, opts, N_VERSION);
+		}
+		break;
+	case N_FTYPE:
+		cli->sb_feat.dirftype = getnum(value, opts, N_FTYPE);
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -2015,45 +2037,15 @@ main(
 			/* end temp don't break code */
 			break;
 		case 'n':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)nopts.subopts;
-				char	*value;
+			parse_subopts(c, optarg, &cli);
 
-				switch (getsubopt(&p, subopts, &value)) {
-				case N_LOG:
-					dirblocklog = getnum(value, &nopts,
-							     N_LOG);
-					dirblocksize = 1 << dirblocklog;
-					nlflag = 1;
-					break;
-				case N_SIZE:
-					dirblocksize = getnum(value, &nopts,
-							      N_SIZE);
-					dirblocklog =
-						libxfs_highbit32(dirblocksize);
-					nsflag = 1;
-					break;
-				case N_VERSION:
-					value = getstr(value, &nopts, N_VERSION);
-					if (!strcasecmp(value, "ci")) {
-						/* ASCII CI mode */
-						sb_feat.nci = true;
-					} else {
-						sb_feat.dir_version =
-							getnum(value, &nopts,
-							       N_VERSION);
-					}
-					nvflag = 1;
-					break;
-				case N_FTYPE:
-					sb_feat.dirftype = getnum(value, &nopts,
-								  N_FTYPE);
-					break;
-				default:
-					unknown('n', value);
-				}
-			}
+			/* temp don't break code */
+			if ((nsflag = cli_opt_set(&nopts, N_SIZE)))
+				dirblocksize = getnum(cli.dirblocksize, &nopts, N_SIZE);
+			dirblocklog = cli.dirblocklog;
+
+			nlflag = cli_opt_set(&nopts, N_LOG);
+			/* end temp don't break code */
 			break;
 		case 'N':
 			Nflag = 1;
-- 
2.13.3

[PATCH 11/42] mkfs: factor rt subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 55 ++++++++++++++++++++++++++-----------------------------
 1 file changed, 26 insertions(+), 29 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index e3ee1f1c87fc..87505c61aa79 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1725,6 +1725,26 @@ rtdev_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	switch (subopt) {
+	case R_EXTSIZE:
+		cli->rtextsize = getstr(value, &ropts, R_EXTSIZE);
+		break;
+	case R_FILE:
+		cli->xi->risfile = getnum(value, &ropts, R_FILE);
+		break;
+	case R_NAME:
+	case R_DEV:
+		cli->xi->rtname = getstr(value, &ropts, R_NAME);
+		break;
+	case R_SIZE:
+		cli->rtsize = getstr(value, &ropts, R_SIZE);
+		break;
+	case R_NOALIGN:
+		cli->sb_feat.nortalign = getnum(value, &ropts, R_NOALIGN);
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -2062,36 +2082,13 @@ main(
 			qflag = 1;
 			break;
 		case 'r':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)ropts.subopts;
-				char	*value;
+			parse_subopts(c, optarg, &cli);
 
-				switch (getsubopt(&p, subopts, &value)) {
-				case R_EXTSIZE:
-					rtextsize = getstr(value, &ropts,
-							   R_EXTSIZE);
-					break;
-				case R_FILE:
-					xi.risfile = getnum(value, &ropts,
-							    R_FILE);
-					break;
-				case R_NAME:
-				case R_DEV:
-					xi.rtname = getstr(value, &ropts,
-							   R_NAME);
-					break;
-				case R_SIZE:
-					rtsize = getstr(value, &ropts, R_SIZE);
-					break;
-				case R_NOALIGN:
-					norsflag = getnum(value, &ropts,
-								R_NOALIGN);
-					break;
-				default:
-					unknown('r', value);
-				}
-			}
+			/* temp don't break code */
+			rtextsize = cli.rtextsize;
+			rtsize = cli.rtsize;
+			norsflag = cli.sb_feat.nortalign;
+			/* end temp don't break code */
 			break;
 		case 's':
 			p = optarg;
-- 
2.13.3

[PATCH 12/42] mkfs: factor sector subopts parser

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 69 +++++++++++++++++++++++++++------------------------------
 1 file changed, 33 insertions(+), 36 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 87505c61aa79..5a646b89cd0f 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1755,6 +1755,27 @@ sector_opts_parser(
 	char			*value,
 	struct cli_params	*cli)
 {
+	int			sectorlog;
+
+	switch (subopt) {
+	case S_LOG:
+	case S_SECTLOG:
+		if (cli->sectorsize)
+			conflict('s', opts->subopts, S_SECTSIZE, S_SECTLOG);
+		sectorlog = getnum(value, &sopts, S_SECTLOG);
+		cli->sectorsize = 1 << sectorlog;
+		cli->lsectorsize = cli->sectorsize;
+		break;
+	case S_SIZE:
+	case S_SECTSIZE:
+		if (cli->sectorsize)
+			conflict('s', opts->subopts, S_SECTLOG, S_SECTSIZE);
+		cli->sectorsize = getnum(value, &sopts, S_SECTSIZE);
+		cli->lsectorsize = cli->sectorsize;
+		break;
+	default:
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -1881,7 +1902,6 @@ main(
 	int			nvflag;
 	int			Nflag;
 	int			discard = 1;
-	char			*p;
 	char			*protofile;
 	char			*protostring;
 	int			qflag;
@@ -2091,41 +2111,18 @@ main(
 			/* end temp don't break code */
 			break;
 		case 's':
-			p = optarg;
-			while (*p != '\0') {
-				char	**subopts = (char **)sopts.subopts;
-				char	*value;
-
-				switch (getsubopt(&p, subopts, &value)) {
-				case S_LOG:
-				case S_SECTLOG:
-					if (lssflag)
-						conflict('s', (const char **)subopts,
-							 S_SECTSIZE, S_SECTLOG);
-					sectorlog = getnum(value, &sopts,
-							   S_SECTLOG);
-					lsectorlog = sectorlog;
-					sectorsize = 1 << sectorlog;
-					lsectorsize = sectorsize;
-					lslflag = slflag = 1;
-					break;
-				case S_SIZE:
-				case S_SECTSIZE:
-					if (lslflag)
-						conflict('s', (const char **)subopts, S_SECTLOG,
-							 S_SECTSIZE);
-					sectorsize = getnum(value, &sopts,
-							    S_SECTSIZE);
-					lsectorsize = sectorsize;
-					sectorlog =
-						libxfs_highbit32(sectorsize);
-					lsectorlog = sectorlog;
-					lssflag = ssflag = 1;
-					break;
-				default:
-					unknown('s', value);
-				}
-			}
+			parse_subopts(c, optarg, &cli);
+
+			/* temp don't break code */
+			sectorsize = cli.sectorsize;
+			lsectorlog = libxfs_highbit32(sectorsize);
+			lsectorsize = cli.lsectorsize;
+			lsectorlog = libxfs_highbit32(lsectorsize);
+			lslflag = slflag = cli_opt_set(&sopts, S_LOG) ||
+					   cli_opt_set(&sopts, S_SECTLOG);
+
+			lssflag = ssflag = cli_opt_set(&sopts, S_SIZE) ||
+					   cli_opt_set(&sopts, S_SECTSIZE);
 			break;
 		case 'V':
 			printf(_("%s version %s\n"), progname, VERSION);
-- 
2.13.3

[PATCH 13/42] mkfs: Introduce mkfs configuration structure

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Formatting the on disk XFS structures requires a certain set of
validated and calculated parameters. By the time we start writing
information to diskm this has all be done. Abstract this information
out into a separate structure and initialise it with all the
calculated parameters so we can factor the mkfs formatting code
to use it.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 5a646b89cd0f..b92c3316358f 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -799,6 +799,58 @@ struct cli_params {
 	struct libxfs_xinit	*xi;
 };
 
+/*
+ * Calculated filesystem feature and geometry information.
+ *
+ * This structure contains the information we will use to create the on-disk
+ * filesystem from. The validation and calculation code uses it to store all the
+ * temporary and final config state for the filesystem.
+ *
+ * The information in this structure will contain a mix of validated CLI input
+ * variables, default feature state and calculated values that are needed to
+ * construct the superblock and other on disk features. These are all in one
+ * place so that we don't have to pass handfuls of seemingly arbitrary variables
+ * around to different functions to do teh work we need to do.
+ */
+struct mkfs_params {
+	int		blocksize;
+	int		blocklog;
+	int		sectorsize;
+	int		sectorlog;
+	int		lsectorsize;
+	int		lsectorlog;
+	int		dirblocksize;
+	int		dirblocklog;
+	int		inodesize;
+	int		inodelog;
+	int		inopblock;
+
+	uint64_t	dblocks;
+	uint64_t	logblocks;
+	uint64_t	rtblocks;
+	uint64_t	rtextblocks;
+	uint64_t	rtextents;
+	uint64_t	rtbmblocks;	/* rt bitmap blocks */
+
+	int		dsunit;		/* in FSBs */
+	int		dswidth;	/* in FSBs */
+	int		lsunit;		/* in FSBs */
+
+	uint64_t	agsize;
+	uint64_t	agcount;
+
+	int		imaxpct;
+
+	bool		loginternal;
+	uint64_t	logstart;
+	uint64_t	logagno;
+
+	uuid_t		uuid;
+	char		*label;
+
+	struct sb_feat_args	sb_feat;
+};
+
 #define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
 #define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
 #define MEGABYTES(count, blog)	((uint64_t)(count) << (20 - (blog)))
@@ -1941,6 +1993,7 @@ main(
 		.xi = &xi,
 		.sb_feat = sb_feat,
 	};
+	struct mkfs_params	cfg = {};
 
 	platform_uuid_generate(&uuid);
 	progname = basename(argv[0]);
@@ -2965,6 +3018,39 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	}
 	validate_log_size(logblocks, blocklog, min_logblocks);
 
+	/* Temp support code  to set up mkfs cfg parameters */
+	cfg.blocksize = blocksize;
+	cfg.blocklog = blocklog;
+	cfg.sectorsize = sectorsize;
+	cfg.sectorlog = sectorlog;
+	cfg.lsectorsize = lsectorsize;
+	cfg.lsectorlog = lsectorlog;
+	cfg.dirblocksize = dirblocksize;
+	cfg.dirblocklog = dirblocklog;
+	cfg.inodesize = isize;
+	cfg.inodelog = inodelog;
+	cfg.inopblock = inopblock;
+
+	cfg.dblocks = dblocks;
+	cfg.logblocks = logblocks;
+	cfg.rtblocks = rtblocks;
+	cfg.rtextblocks = rtextblocks;
+	cfg.rtextents = rtextents;
+	cfg.rtbmblocks = nbmblocks;
+	cfg.dsunit = dsunit;
+	cfg.dswidth = dswidth;
+	cfg.lsunit = lsunit;
+	cfg.agsize = agsize;
+	cfg.agcount = agcount;
+	cfg.imaxpct = imaxpct;
+	cfg.loginternal = loginternal;
+	cfg.logstart = logstart;
+	cfg.logagno = logagno;
+	cfg.label = label;
+	platform_uuid_copy(&cfg.uuid, &uuid);
+	memcpy(&cfg.sb_feat, &sb_feat, sizeof(sb_feat));
+	/* end temp support code */
+
 	if (!qflag || Nflag) {
 		printf(_(
 		   "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n"
@@ -2994,6 +3080,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 			exit(0);
 	}
 
+
 	if (label)
 		strncpy(sbp->sb_fname, label, sizeof(sbp->sb_fname));
 	sbp->sb_magicnum = XFS_SB_MAGIC;
-- 
2.13.3

[PATCH 14/42] mkfs: factor printing of mkfs config

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 60 +++++++++++++++++++++++++++++++++------------------------
 1 file changed, 35 insertions(+), 25 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index b92c3316358f..4c67062047d9 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1881,6 +1881,40 @@ parse_subopts(
 	}
 }
 
+static void
+print_mkfs_cfg(
+	struct mkfs_params	*cfg,
+	char			*dfile,
+	char			*logfile,
+	char			*rtfile)
+{
+	struct sb_feat_args	*fp = &cfg->sb_feat;
+
+	printf(_(
+"meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n"
+"         =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n"
+"         =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"
+"data     =%-22s bsize=%-6u blocks=%llu, imaxpct=%u\n"
+"         =%-22s sunit=%-6u swidth=%u blks\n"
+"naming   =version %-14u bsize=%-6u ascii-ci=%d ftype=%d\n"
+"log      =%-22s bsize=%-6d blocks=%lld, version=%d\n"
+"         =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n"
+"realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"),
+		dfile, cfg->inodesize, (long long)cfg->agcount,
+			(long long)cfg->agsize,
+		"", cfg->sectorsize, fp->attr_version, !fp->projid16bit,
+		"", fp->crcs_enabled, fp->finobt, fp->spinodes, fp->rmapbt,
+			fp->reflink,
+		"", cfg->blocksize, (long long)cfg->dblocks, cfg->imaxpct,
+		"", cfg->dsunit, cfg->dswidth,
+		fp->dir_version, cfg->dirblocksize, fp->nci, fp->dirftype,
+		logfile, cfg->blocksize, (long long)cfg->logblocks,
+			fp->log_version,
+		"", cfg->lsectorsize, cfg->lsunit, fp->lazy_sb_counters,
+		rtfile, (int)cfg->rtextblocks << cfg->blocklog,
+			(long long)cfg->rtblocks, (long long)cfg->rtextents);
+}
+
 int
 main(
 	int			argc,
@@ -3052,35 +3086,11 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	/* end temp support code */
 
 	if (!qflag || Nflag) {
-		printf(_(
-		   "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n"
-		   "         =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n"
-		   "         =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"
-		   "data     =%-22s bsize=%-6u blocks=%llu, imaxpct=%u\n"
-		   "         =%-22s sunit=%-6u swidth=%u blks\n"
-		   "naming   =version %-14u bsize=%-6u ascii-ci=%d ftype=%d\n"
-		   "log      =%-22s bsize=%-6d blocks=%lld, version=%d\n"
-		   "         =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n"
-		   "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"),
-			dfile, isize, (long long)agcount, (long long)agsize,
-			"", sectorsize, sb_feat.attr_version,
-				    !sb_feat.projid16bit,
-			"", sb_feat.crcs_enabled, sb_feat.finobt, sb_feat.spinodes,
-			sb_feat.rmapbt, sb_feat.reflink,
-			"", blocksize, (long long)dblocks, imaxpct,
-			"", dsunit, dswidth,
-			sb_feat.dir_version, dirblocksize, sb_feat.nci,
-				sb_feat.dirftype,
-			logfile, 1 << blocklog, (long long)logblocks,
-			sb_feat.log_version, "", lsectorsize, lsunit,
-				sb_feat.lazy_sb_counters,
-			rtfile, rtextblocks << blocklog,
-			(long long)rtblocks, (long long)rtextents);
+		print_mkfs_cfg(&cfg, dfile, logfile, rtfile);
 		if (Nflag)
 			exit(0);
 	}
 
-
 	if (label)
 		strncpy(sbp->sb_fname, label, sizeof(sbp->sb_fname));
 	sbp->sb_magicnum = XFS_SB_MAGIC;
-- 
2.13.3

[PATCH 15/42] mkfs: factor in memory superblock setup

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 141 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 82 insertions(+), 59 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 4c67062047d9..0e115bf3cf4d 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1915,6 +1915,84 @@ print_mkfs_cfg(
 			(long long)cfg->rtblocks, (long long)cfg->rtextents);
 }
 
+/*
+ * Format everything from the generated config into the superblock that
+ * will be used to initialise the on-disk superblock. This is the in-memory
+ * copy, so no need to care about endian swapping here.
+ */
+static void
+setup_superblock(
+	struct mkfs_params	*cfg,
+	struct xfs_mount	*mp,
+	struct xfs_sb		*sbp)
+{
+	if (cfg->label)
+		strncpy(sbp->sb_fname, cfg->label, sizeof(sbp->sb_fname));
+
+	sbp->sb_magicnum = XFS_SB_MAGIC;
+	sbp->sb_blocksize = cfg->blocksize;
+	sbp->sb_dblocks = cfg->dblocks;
+	sbp->sb_rblocks = cfg->rtblocks;
+	sbp->sb_rextents = cfg->rtextents;
+	platform_uuid_copy(&sbp->sb_uuid, &cfg->uuid);
+	/* Only in memory; libxfs expects this as if read from disk */
+	platform_uuid_copy(&sbp->sb_meta_uuid, &cfg->uuid);
+	sbp->sb_logstart = cfg->logstart;
+	sbp->sb_rootino = sbp->sb_rbmino = sbp->sb_rsumino = NULLFSINO;
+	sbp->sb_rextsize = cfg->rtextblocks;
+	sbp->sb_agcount = (xfs_agnumber_t)cfg->agcount;
+	sbp->sb_rbmblocks = cfg->rtbmblocks;
+	sbp->sb_logblocks = (xfs_extlen_t)cfg->logblocks;
+	sbp->sb_sectsize = (uint16_t)cfg->sectorsize;
+	sbp->sb_inodesize = (uint16_t)cfg->inodesize;
+	sbp->sb_inopblock = (uint16_t)(cfg->blocksize / cfg->inodesize);
+	sbp->sb_sectlog = (uint8_t)cfg->sectorlog;
+	sbp->sb_inodelog = (uint8_t)cfg->inodelog;
+	sbp->sb_inopblog = (uint8_t)(cfg->blocklog - cfg->inodelog);
+	sbp->sb_rextslog = (uint8_t)(cfg->rtextents ?
+			libxfs_highbit32((unsigned int)cfg->rtextents) : 0);
+	sbp->sb_inprogress = 1;	/* mkfs is in progress */
+	sbp->sb_imax_pct = cfg->imaxpct;
+	sbp->sb_icount = 0;
+	sbp->sb_ifree = 0;
+	sbp->sb_fdblocks = cfg->dblocks -
+			   cfg->agcount * libxfs_prealloc_blocks(mp) -
+			   (cfg->loginternal ? cfg->logblocks : 0);
+	sbp->sb_frextents = 0;	/* will do a free later */
+	sbp->sb_uquotino = sbp->sb_gquotino = sbp->sb_pquotino = 0;
+	sbp->sb_qflags = 0;
+	sbp->sb_unit = cfg->dsunit;
+	sbp->sb_width = cfg->dswidth;
+	sbp->sb_dirblklog = cfg->dirblocklog - cfg->blocklog;
+
+	/*
+	 * log stripe unit is stored in bytes on disk and cannot be zero
+	 * for v2 logs.
+	 */
+	if (cfg->sb_feat.log_version == 2) {
+		int	lsunit = (cfg->lsunit == 0) ? 1 : cfg->lsunit;
+
+		sbp->sb_logsunit = XFS_FSB_TO_B(mp, lsunit);;
+	} else
+		sbp->sb_logsunit = 0;
+
+	if (cfg->sb_feat.inode_align) {
+		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
+		if (cfg->sb_feat.crcs_enabled)
+			cluster_size *= cfg->inodesize / XFS_DINODE_MIN_SIZE;
+		sbp->sb_inoalignmt = cluster_size >> cfg->blocklog;
+	} else
+		sbp->sb_inoalignmt = 0;
+
+	if (cfg->lsectorsize != BBSIZE || cfg->sectorsize != BBSIZE) {
+		sbp->sb_logsectlog = (uint8_t)cfg->lsectorlog;
+		sbp->sb_logsectsize = (uint16_t)cfg->lsectorsize;
+	} else {
+		sbp->sb_logsectlog = 0;
+		sbp->sb_logsectsize = 0;
+	}
+}
+
 int
 main(
 	int			argc,
@@ -3091,65 +3169,10 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 			exit(0);
 	}
 
-	if (label)
-		strncpy(sbp->sb_fname, label, sizeof(sbp->sb_fname));
-	sbp->sb_magicnum = XFS_SB_MAGIC;
-	sbp->sb_blocksize = blocksize;
-	sbp->sb_dblocks = dblocks;
-	sbp->sb_rblocks = rtblocks;
-	sbp->sb_rextents = rtextents;
-	platform_uuid_copy(&sbp->sb_uuid, &uuid);
-	/* Only in memory; libxfs expects this as if read from disk */
-	platform_uuid_copy(&sbp->sb_meta_uuid, &uuid);
-	sbp->sb_logstart = logstart;
-	sbp->sb_rootino = sbp->sb_rbmino = sbp->sb_rsumino = NULLFSINO;
-	sbp->sb_rextsize = rtextblocks;
-	sbp->sb_agcount = (xfs_agnumber_t)agcount;
-	sbp->sb_rbmblocks = nbmblocks;
-	sbp->sb_logblocks = (xfs_extlen_t)logblocks;
-	sbp->sb_sectsize = (uint16_t)sectorsize;
-	sbp->sb_inodesize = (uint16_t)isize;
-	sbp->sb_inopblock = (uint16_t)(blocksize / isize);
-	sbp->sb_sectlog = (uint8_t)sectorlog;
-	sbp->sb_inodelog = (uint8_t)inodelog;
-	sbp->sb_inopblog = (uint8_t)(blocklog - inodelog);
-	sbp->sb_rextslog =
-		(uint8_t)(rtextents ?
-			libxfs_highbit32((unsigned int)rtextents) : 0);
-	sbp->sb_inprogress = 1;	/* mkfs is in progress */
-	sbp->sb_imax_pct = imaxpct;
-	sbp->sb_icount = 0;
-	sbp->sb_ifree = 0;
-	sbp->sb_fdblocks = dblocks - agcount * libxfs_prealloc_blocks(mp) -
-		(loginternal ? logblocks : 0);
-	sbp->sb_frextents = 0;	/* will do a free later */
-	sbp->sb_uquotino = sbp->sb_gquotino = sbp->sb_pquotino = 0;
-	sbp->sb_qflags = 0;
-	sbp->sb_unit = dsunit;
-	sbp->sb_width = dswidth;
-	sbp->sb_dirblklog = dirblocklog - blocklog;
-	if (sb_feat.log_version == 2) {	/* This is stored in bytes */
-		lsunit = (lsunit == 0) ? 1 : XFS_FSB_TO_B(mp, lsunit);
-		sbp->sb_logsunit = lsunit;
-	} else
-		sbp->sb_logsunit = 0;
-	if (sb_feat.inode_align) {
-		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
-		if (sb_feat.crcs_enabled)
-			cluster_size *= isize / XFS_DINODE_MIN_SIZE;
-		sbp->sb_inoalignmt = cluster_size >> blocklog;
-		sb_feat.inode_align = sbp->sb_inoalignmt != 0;
-	} else
-		sbp->sb_inoalignmt = 0;
-	if (lsectorsize != BBSIZE || sectorsize != BBSIZE) {
-		sbp->sb_logsectlog = (uint8_t)lsectorlog;
-		sbp->sb_logsectsize = (uint16_t)lsectorsize;
-	} else {
-		sbp->sb_logsectlog = 0;
-		sbp->sb_logsectsize = 0;
-	}
-
-	sb_set_features(&mp->m_sb, &sb_feat, sectorsize, lsectorsize, dsunit);
+	/*
+	 * Finish setting up the superblock state ready for formatting.
+	 */
+	setup_superblock(&cfg, mp, sbp);
 
 	if (force_overwrite)
 		zero_old_xfs_structures(&xi, sbp);
-- 
2.13.3

[PATCH 16/42] mkfs: factor out device preparation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Prior to formating the device(s), we have to take several steps to
prepare them and check that they are appropriate for the formatting
that is about to take place. Pull all this into a single function
that is run before mounting the libxfs infrastructure.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 167 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 96 insertions(+), 71 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 0e115bf3cf4d..2438e50d3f33 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1993,6 +1993,97 @@ setup_superblock(
 	}
 }
 
+/*
+ * Sanitise the data and log devices and prepare them so libxfs can mount the
+ * device successfully. Also check we can access the rt device if configured.
+ */
+static void
+prepare_devices(
+	struct mkfs_params	*cfg,
+	struct libxfs_xinit	*xi,
+	struct xfs_mount	*mp,
+	struct xfs_sb		*sbp,
+	bool			clear_stale)
+{
+	struct xfs_buf		*buf;
+	int			whack_blks = BTOBB(WHACK_SIZE);
+
+	/*
+	 * If there's an old XFS filesystem on the device with enough intact
+	 * information that we can parse the superblock, there's enough
+	 * information on disk to confuse a future xfs_repair call. To avoid
+	 * this, whack all the old secondary superblocks that we can find.
+	 */
+	if (clear_stale)
+		zero_old_xfs_structures(xi, sbp);
+
+	/*
+	 * If the data device is a file, grow out the file to its final size if
+	 * needed so that the reads for the end of the device in the mount code
+	 * will succeed.
+	 */
+	if (xi->disfile &&
+	    xi->dsize * xi->dbsize < cfg->dblocks * cfg->blocksize) {
+		if (ftruncate(xi->dfd, cfg->dblocks * cfg->blocksize) < 0) {
+			fprintf(stderr,
+				_("%s: Growing the data section failed\n"),
+				progname);
+			exit(1);
+		}
+
+		/* update size to be able to whack blocks correctly */
+		xi->dsize = BTOBB(cfg->dblocks * cfg->blocksize);
+	}
+
+	/*
+	 * Zero out the end to obliterate any old MD RAID (or other) metadata at
+	 * the end of the device.  (MD sb is ~64k from the end, take out a wider
+	 * swath to be sure)
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp, (xi->dsize - whack_blks),
+			    whack_blks);
+	memset(XFS_BUF_PTR(buf), 0, WHACK_SIZE);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	libxfs_purgebuf(buf);
+
+	/*
+	 * Now zero out the beginning of the device, to obliterate any old
+	 * filesystem signatures out there.  This should take care of
+	 * swap (somewhere around the page size), jfs (32k),
+	 * ext[2,3] and reiserfs (64k) - and hopefully all else.
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp, 0, whack_blks);
+	memset(XFS_BUF_PTR(buf), 0, WHACK_SIZE);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	libxfs_purgebuf(buf);
+
+	/* OK, now write the superblock... */
+	buf = libxfs_getbuf(mp->m_ddev_targp, XFS_SB_DADDR, XFS_FSS_TO_BB(mp, 1));
+	buf->b_ops = &xfs_sb_buf_ops;
+	memset(XFS_BUF_PTR(buf), 0, cfg->sectorsize);
+	libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	libxfs_purgebuf(buf);
+
+	/* ...and zero the log.... */
+	libxfs_log_clear(mp->m_logdev_targp, NULL,
+			 XFS_FSB_TO_DADDR(mp, cfg->logstart),
+			 (xfs_extlen_t)XFS_FSB_TO_BB(mp, cfg->logblocks),
+			 &sbp->sb_uuid, cfg->sb_feat.log_version,
+			 sbp->sb_logsunit, XLOG_FMT, XLOG_INIT_CYCLE, false);
+
+	/* finally, check we can write the last block in the realtime area */
+	if (mp->m_rtdev_targp->dev && cfg->rtblocks > 0) {
+		buf = libxfs_getbuf(mp->m_rtdev_targp,
+				    XFS_FSB_TO_BB(mp, cfg->rtblocks - 1LL),
+				    BTOBB(cfg->blocksize));
+		memset(XFS_BUF_PTR(buf), 0, cfg->blocksize);
+		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+		libxfs_purgebuf(buf);
+	}
+
+}
+
 int
 main(
 	int			argc,
@@ -3174,65 +3265,17 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	 */
 	setup_superblock(&cfg, mp, sbp);
 
-	if (force_overwrite)
-		zero_old_xfs_structures(&xi, sbp);
-
 	/*
-	 * Zero out the beginning of the device, to obliterate any old
-	 * filesystem signatures out there.  This should take care of
-	 * swap (somewhere around the page size), jfs (32k),
-	 * ext[2,3] and reiserfs (64k) - and hopefully all else.
+	 * we need the libxfs buffer cache from here on in.
 	 */
 	libxfs_buftarg_init(mp, xi.ddev, xi.logdev, xi.rtdev);
-	buf = libxfs_getbuf(mp->m_ddev_targp, 0, BTOBB(WHACK_SIZE));
-	memset(XFS_BUF_PTR(buf), 0, WHACK_SIZE);
-	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-	libxfs_purgebuf(buf);
-
-	/* OK, now write the superblock */
-	buf = libxfs_getbuf(mp->m_ddev_targp, XFS_SB_DADDR, XFS_FSS_TO_BB(mp, 1));
-	buf->b_ops = &xfs_sb_buf_ops;
-	memset(XFS_BUF_PTR(buf), 0, sectorsize);
-	libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp);
-	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-	libxfs_purgebuf(buf);
-
-	/*
-	 * If the data area is a file, then grow it out to its final size
-	 * if needed so that the reads for the end of the device in the mount
-	 * code will succeed.
-	 */
-	if (xi.disfile && xi.dsize * xi.dbsize < dblocks * blocksize) {
-		if (ftruncate(xi.dfd, dblocks * blocksize) < 0) {
-			fprintf(stderr,
-				_("%s: Growing the data section failed\n"),
-				progname);
-			exit(1);
-		}
-	}
-
-	/*
-	 * Zero out the end of the device, to obliterate any
-	 * old MD RAID (or other) metadata at the end of the device.
-	 * (MD sb is ~64k from the end, take out a wider swath to be sure)
-	 */
-	if (!xi.disfile) {
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				    (xi.dsize - BTOBB(WHACK_SIZE)),
-				    BTOBB(WHACK_SIZE));
-		memset(XFS_BUF_PTR(buf), 0, WHACK_SIZE);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-		libxfs_purgebuf(buf);
-	}
 
 	/*
-	 * Zero the log....
+	 * Before we mount the filesystem we need to make sure the devices have
+	 * enough of the filesystem structure on them that allows libxfs to
+	 * mount.
 	 */
-	libxfs_log_clear(mp->m_logdev_targp, NULL,
-		XFS_FSB_TO_DADDR(mp, logstart),
-		(xfs_extlen_t)XFS_FSB_TO_BB(mp, logblocks),
-		&sbp->sb_uuid, sb_feat.log_version, lsunit, XLOG_FMT, XLOG_INIT_CYCLE, false);
-
+	prepare_devices(&cfg, &xi, mp, sbp, force_overwrite);
 	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 0);
 	if (mp == NULL) {
 		fprintf(stderr, _("%s: filesystem failed to initialize\n"),
@@ -3576,24 +3619,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	}
 
 	/*
-	 * Touch last block, make fs the right size if it's a file.
-	 */
-	buf = libxfs_getbuf(mp->m_ddev_targp,
-		(xfs_daddr_t)XFS_FSB_TO_BB(mp, dblocks - 1LL), bsize);
-	memset(XFS_BUF_PTR(buf), 0, blocksize);
-	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-	/*
-	 * Make sure we can write the last block in the realtime area.
-	 */
-	if (mp->m_rtdev_targp->dev && rtblocks > 0) {
-		buf = libxfs_getbuf(mp->m_rtdev_targp,
-				XFS_FSB_TO_BB(mp, rtblocks - 1LL), bsize);
-		memset(XFS_BUF_PTR(buf), 0, blocksize);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-	}
-
-	/*
 	 * BNO, CNT free block list
 	 */
 	for (agno = 0; agno < agcount; agno++) {
-- 
2.13.3

[PATCH 17/42] mkfs: factor writing AG headers

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

There are some slight changes to the way log alignment is calculated
in the change. Instead of using a flag, it checks the log start
block to see if it's different to the first free block in the log
AG, and if it is different then does the aligned setup. This means
we no longer have to care if the log is aligned or not, the code
will do the right thing in all cases.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 750 +++++++++++++++++++++++++++++---------------------------
 1 file changed, 390 insertions(+), 360 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 2438e50d3f33..13af1ef1cd41 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2084,22 +2084,401 @@ prepare_devices(
 
 }
 
+/*
+ * XXX: this code is mostly common with the kernel growfs code.
+ * These initialisations should be pulled into libxfs to keep the
+ * kernel/userspace header initialisation code the same.
+ */
+static void
+initialise_ag_headers(
+	struct mkfs_params	*cfg,
+	struct xfs_mount	*mp,
+	struct xfs_sb		*sbp,
+	xfs_agnumber_t		agno,
+	int			*worst_freelist)
+{
+	struct xfs_perag	*pag = libxfs_perag_get(mp, agno);
+	struct xfs_agfl		*agfl;
+	struct xfs_agf		*agf;
+	struct xfs_agi		*agi;
+	struct xfs_buf		*buf;
+	struct xfs_btree_block	*block;
+	struct xfs_alloc_rec	*arec;
+	struct xfs_alloc_rec	*nrec;
+	int			bucket;
+	uint64_t		agsize = cfg->agsize;
+	xfs_agblock_t		agblocks;
+	bool			is_log_ag = false;
+	int			c;
+
+	if (cfg->loginternal && agno == cfg->logagno)
+		is_log_ag = true;
+
+	/*
+	 * Superblock.
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AG_DADDR(mp, agno, XFS_SB_DADDR),
+			XFS_FSS_TO_BB(mp, 1));
+	buf->b_ops = &xfs_sb_buf_ops;
+	memset(XFS_BUF_PTR(buf), 0, cfg->sectorsize);
+	libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * AG header block: freespace
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AG_DADDR(mp, agno, XFS_AGF_DADDR(mp)),
+			XFS_FSS_TO_BB(mp, 1));
+	buf->b_ops = &xfs_agf_buf_ops;
+	agf = XFS_BUF_TO_AGF(buf);
+	memset(agf, 0, cfg->sectorsize);
+	if (agno == cfg->agcount - 1)
+		agsize = cfg->dblocks - (xfs_rfsblock_t)(agno * agsize);
+	agf->agf_magicnum = cpu_to_be32(XFS_AGF_MAGIC);
+	agf->agf_versionnum = cpu_to_be32(XFS_AGF_VERSION);
+	agf->agf_seqno = cpu_to_be32(agno);
+	agf->agf_length = cpu_to_be32(agsize);
+	agf->agf_roots[XFS_BTNUM_BNOi] = cpu_to_be32(XFS_BNO_BLOCK(mp));
+	agf->agf_roots[XFS_BTNUM_CNTi] = cpu_to_be32(XFS_CNT_BLOCK(mp));
+	agf->agf_levels[XFS_BTNUM_BNOi] = cpu_to_be32(1);
+	agf->agf_levels[XFS_BTNUM_CNTi] = cpu_to_be32(1);
+	pag->pagf_levels[XFS_BTNUM_BNOi] = 1;
+	pag->pagf_levels[XFS_BTNUM_CNTi] = 1;
+
+	if (xfs_sb_version_hasrmapbt(sbp)) {
+		agf->agf_roots[XFS_BTNUM_RMAPi] = cpu_to_be32(XFS_RMAP_BLOCK(mp));
+		agf->agf_levels[XFS_BTNUM_RMAPi] = cpu_to_be32(1);
+		agf->agf_rmap_blocks = cpu_to_be32(1);
+	}
+
+	if (xfs_sb_version_hasreflink(sbp)) {
+		agf->agf_refcount_root = cpu_to_be32(libxfs_refc_block(mp));
+		agf->agf_refcount_level = cpu_to_be32(1);
+		agf->agf_refcount_blocks = cpu_to_be32(1);
+	}
+
+	agf->agf_flfirst = 0;
+	agf->agf_fllast = cpu_to_be32(XFS_AGFL_SIZE(mp) - 1);
+	agf->agf_flcount = 0;
+	agblocks = (xfs_agblock_t)(agsize - libxfs_prealloc_blocks(mp));
+	agf->agf_freeblks = cpu_to_be32(agblocks);
+	agf->agf_longest = cpu_to_be32(agblocks);
+
+	if (xfs_sb_version_hascrc(sbp))
+		platform_uuid_copy(&agf->agf_uuid, &sbp->sb_uuid);
+
+	if (is_log_ag) {
+		be32_add_cpu(&agf->agf_freeblks, -(int64_t)cfg->logblocks);
+		agf->agf_longest = cpu_to_be32(agsize -
+			XFS_FSB_TO_AGBNO(mp, cfg->logstart) - cfg->logblocks);
+	}
+	if (libxfs_alloc_min_freelist(mp, pag) > *worst_freelist)
+		*worst_freelist = libxfs_alloc_min_freelist(mp, pag);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * AG freelist header block
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AG_DADDR(mp, agno, XFS_AGFL_DADDR(mp)),
+			XFS_FSS_TO_BB(mp, 1));
+	buf->b_ops = &xfs_agfl_buf_ops;
+	agfl = XFS_BUF_TO_AGFL(buf);
+	/* setting to 0xff results in initialisation to NULLAGBLOCK */
+	memset(agfl, 0xff, cfg->sectorsize);
+	if (xfs_sb_version_hascrc(sbp)) {
+		agfl->agfl_magicnum = cpu_to_be32(XFS_AGFL_MAGIC);
+		agfl->agfl_seqno = cpu_to_be32(agno);
+		platform_uuid_copy(&agfl->agfl_uuid, &sbp->sb_uuid);
+		for (bucket = 0; bucket < XFS_AGFL_SIZE(mp); bucket++)
+			agfl->agfl_bno[bucket] = cpu_to_be32(NULLAGBLOCK);
+	}
+
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * AG header block: inodes
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AG_DADDR(mp, agno, XFS_AGI_DADDR(mp)),
+			XFS_FSS_TO_BB(mp, 1));
+	agi = XFS_BUF_TO_AGI(buf);
+	buf->b_ops = &xfs_agi_buf_ops;
+	memset(agi, 0, cfg->sectorsize);
+	agi->agi_magicnum = cpu_to_be32(XFS_AGI_MAGIC);
+	agi->agi_versionnum = cpu_to_be32(XFS_AGI_VERSION);
+	agi->agi_seqno = cpu_to_be32(agno);
+	agi->agi_length = cpu_to_be32(agsize);
+	agi->agi_count = 0;
+	agi->agi_root = cpu_to_be32(XFS_IBT_BLOCK(mp));
+	agi->agi_level = cpu_to_be32(1);
+	if (xfs_sb_version_hasfinobt(sbp)) {
+		agi->agi_free_root = cpu_to_be32(XFS_FIBT_BLOCK(mp));
+		agi->agi_free_level = cpu_to_be32(1);
+	}
+	agi->agi_freecount = 0;
+	agi->agi_newino = cpu_to_be32(NULLAGINO);
+	agi->agi_dirino = cpu_to_be32(NULLAGINO);
+	if (xfs_sb_version_hascrc(sbp))
+		platform_uuid_copy(&agi->agi_uuid, &sbp->sb_uuid);
+	for (c = 0; c < XFS_AGI_UNLINKED_BUCKETS; c++)
+		agi->agi_unlinked[c] = cpu_to_be32(NULLAGINO);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * BNO btree root block
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AGB_TO_DADDR(mp, agno, XFS_BNO_BLOCK(mp)),
+			BTOBB(cfg->blocksize));
+	buf->b_ops = &xfs_allocbt_buf_ops;
+	block = XFS_BUF_TO_BLOCK(buf);
+	memset(block, 0, cfg->blocksize);
+	libxfs_btree_init_block(mp, buf, XFS_BTNUM_BNO, 0, 1, agno, 0);
+
+	arec = XFS_ALLOC_REC_ADDR(mp, block, 1);
+	arec->ar_startblock = cpu_to_be32(libxfs_prealloc_blocks(mp));
+	if (is_log_ag) {
+		xfs_agblock_t	start = XFS_FSB_TO_AGBNO(mp, cfg->logstart);
+
+		ASSERT(start >= libxfs_prealloc_blocks(mp));
+		if (start != libxfs_prealloc_blocks(mp)) {
+			/*
+			 * Modify first record to pad stripe align of log
+			 */
+			arec->ar_blockcount = cpu_to_be32(start -
+						libxfs_prealloc_blocks(mp));
+			nrec = arec + 1;
+			/*
+			 * Insert second record at start of internal log
+			 * which then gets trimmed.
+			 */
+			nrec->ar_startblock = cpu_to_be32(
+					be32_to_cpu(arec->ar_startblock) +
+					be32_to_cpu(arec->ar_blockcount));
+			arec = nrec;
+			be16_add_cpu(&block->bb_numrecs, 1);
+		}
+		/*
+		 * Change record start to after the internal log
+		 */
+		be32_add_cpu(&arec->ar_startblock, cfg->logblocks);
+	}
+	/*
+	 * Calculate the record block count and check for the case where
+	 * the log might have consumed all available space in the AG. If
+	 * so, reset the record count to 0 to avoid exposure of an invalid
+	 * record start block.
+	 */
+	arec->ar_blockcount = cpu_to_be32(agsize -
+					  be32_to_cpu(arec->ar_startblock));
+	if (!arec->ar_blockcount)
+		block->bb_numrecs = 0;
+
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * CNT btree root block
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AGB_TO_DADDR(mp, agno, XFS_CNT_BLOCK(mp)),
+			BTOBB(cfg->blocksize));
+	buf->b_ops = &xfs_allocbt_buf_ops;
+	block = XFS_BUF_TO_BLOCK(buf);
+	memset(block, 0, cfg->blocksize);
+	libxfs_btree_init_block(mp, buf, XFS_BTNUM_CNT, 0, 1, agno, 0);
+
+	arec = XFS_ALLOC_REC_ADDR(mp, block, 1);
+	arec->ar_startblock = cpu_to_be32(libxfs_prealloc_blocks(mp));
+	if (is_log_ag) {
+		xfs_agblock_t	start = XFS_FSB_TO_AGBNO(mp, cfg->logstart);
+
+		ASSERT(start >= libxfs_prealloc_blocks(mp));
+		if (start != libxfs_prealloc_blocks(mp)) {
+			arec->ar_blockcount = cpu_to_be32(start -
+					libxfs_prealloc_blocks(mp));
+			nrec = arec + 1;
+			nrec->ar_startblock = cpu_to_be32(
+					be32_to_cpu(arec->ar_startblock) +
+					be32_to_cpu(arec->ar_blockcount));
+			arec = nrec;
+			be16_add_cpu(&block->bb_numrecs, 1);
+		}
+		be32_add_cpu(&arec->ar_startblock, cfg->logblocks);
+	}
+	/*
+	 * Calculate the record block count and check for the case where
+	 * the log might have consumed all available space in the AG. If
+	 * so, reset the record count to 0 to avoid exposure of an invalid
+	 * record start block.
+	 */
+	arec->ar_blockcount = cpu_to_be32(agsize -
+					  be32_to_cpu(arec->ar_startblock));
+	if (!arec->ar_blockcount)
+		block->bb_numrecs = 0;
+
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * refcount btree root block
+	 */
+	if (xfs_sb_version_hasreflink(sbp)) {
+		buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AGB_TO_DADDR(mp, agno, libxfs_refc_block(mp)),
+			BTOBB(cfg->blocksize));
+		buf->b_ops = &xfs_refcountbt_buf_ops;
+
+		block = XFS_BUF_TO_BLOCK(buf);
+		memset(block, 0, cfg->blocksize);
+		libxfs_btree_init_block(mp, buf, XFS_BTNUM_REFC, 0, 0, agno, 0);
+		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	}
+
+	/*
+	 * INO btree root block
+	 */
+	buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AGB_TO_DADDR(mp, agno, XFS_IBT_BLOCK(mp)),
+			BTOBB(cfg->blocksize));
+	buf->b_ops = &xfs_inobt_buf_ops;
+	block = XFS_BUF_TO_BLOCK(buf);
+	memset(block, 0, cfg->blocksize);
+	libxfs_btree_init_block(mp, buf, XFS_BTNUM_INO, 0, 0, agno, 0);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/*
+	 * Free INO btree root block
+	 */
+	if (xfs_sb_version_hasfinobt(sbp)) {
+		buf = libxfs_getbuf(mp->m_ddev_targp,
+				XFS_AGB_TO_DADDR(mp, agno, XFS_FIBT_BLOCK(mp)),
+				BTOBB(cfg->blocksize));
+		buf->b_ops = &xfs_inobt_buf_ops;
+		block = XFS_BUF_TO_BLOCK(buf);
+		memset(block, 0, cfg->blocksize);
+		libxfs_btree_init_block(mp, buf, XFS_BTNUM_FINO, 0, 0, agno, 0);
+		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	}
+
+	/* RMAP btree root block */
+	if (xfs_sb_version_hasrmapbt(sbp)) {
+		struct xfs_rmap_rec	*rrec;
+
+		buf = libxfs_getbuf(mp->m_ddev_targp,
+			XFS_AGB_TO_DADDR(mp, agno, XFS_RMAP_BLOCK(mp)),
+			BTOBB(cfg->blocksize));
+		buf->b_ops = &xfs_rmapbt_buf_ops;
+		block = XFS_BUF_TO_BLOCK(buf);
+		memset(block, 0, cfg->blocksize);
+
+		libxfs_btree_init_block(mp, buf, XFS_BTNUM_RMAP, 0, 0, agno, 0);
+
+		/*
+		 * mark the AG header regions as static metadata
+		 * The BNO btree block is the first block after the
+		 * headers, so it's location defines the size of region
+		 * the static metadata consumes.
+		 */
+		rrec = XFS_RMAP_REC_ADDR(block, 1);
+		rrec->rm_startblock = 0;
+		rrec->rm_blockcount = cpu_to_be32(XFS_BNO_BLOCK(mp));
+		rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_FS);
+		rrec->rm_offset = 0;
+		be16_add_cpu(&block->bb_numrecs, 1);
+
+		/* account freespace btree root blocks */
+		rrec = XFS_RMAP_REC_ADDR(block, 2);
+		rrec->rm_startblock = cpu_to_be32(XFS_BNO_BLOCK(mp));
+		rrec->rm_blockcount = cpu_to_be32(2);
+		rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_AG);
+		rrec->rm_offset = 0;
+		be16_add_cpu(&block->bb_numrecs, 1);
+
+		/* account inode btree root blocks */
+		rrec = XFS_RMAP_REC_ADDR(block, 3);
+		rrec->rm_startblock = cpu_to_be32(XFS_IBT_BLOCK(mp));
+		rrec->rm_blockcount = cpu_to_be32(XFS_RMAP_BLOCK(mp) -
+						XFS_IBT_BLOCK(mp));
+		rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_INOBT);
+		rrec->rm_offset = 0;
+		be16_add_cpu(&block->bb_numrecs, 1);
+
+		/* account for rmap btree root */
+		rrec = XFS_RMAP_REC_ADDR(block, 4);
+		rrec->rm_startblock = cpu_to_be32(XFS_RMAP_BLOCK(mp));
+		rrec->rm_blockcount = cpu_to_be32(1);
+		rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_AG);
+		rrec->rm_offset = 0;
+		be16_add_cpu(&block->bb_numrecs, 1);
+
+		/* account for refcount btree root */
+		if (xfs_sb_version_hasreflink(sbp)) {
+			rrec = XFS_RMAP_REC_ADDR(block, 5);
+			rrec->rm_startblock = cpu_to_be32(libxfs_refc_block(mp));
+			rrec->rm_blockcount = cpu_to_be32(1);
+			rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_REFC);
+			rrec->rm_offset = 0;
+			be16_add_cpu(&block->bb_numrecs, 1);
+		}
+
+		/* account for the log space */
+		if (is_log_ag) {
+			rrec = XFS_RMAP_REC_ADDR(block,
+					be16_to_cpu(block->bb_numrecs) + 1);
+			rrec->rm_startblock = cpu_to_be32(
+					XFS_FSB_TO_AGBNO(mp, cfg->logstart));
+			rrec->rm_blockcount = cpu_to_be32(cfg->logblocks);
+			rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_LOG);
+			rrec->rm_offset = 0;
+			be16_add_cpu(&block->bb_numrecs, 1);
+		}
+
+		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	}
+
+	libxfs_perag_put(pag);
+}
+
+static void
+initialise_ag_freespace(
+	struct xfs_mount	*mp,
+	xfs_agnumber_t		agno,
+	int			worst_freelist)
+{
+	struct xfs_alloc_arg	args;
+	struct xfs_trans	*tp;
+	struct xfs_trans_res tres = {0};
+	int			c;
+
+	c = libxfs_trans_alloc(mp, &tres, worst_freelist, 0, 0, &tp);
+	if (c)
+		res_failed(c);
+
+	memset(&args, 0, sizeof(args));
+	args.tp = tp;
+	args.mp = mp;
+	args.agno = agno;
+	args.alignment = 1;
+	args.pag = libxfs_perag_get(mp, agno);
+
+	libxfs_alloc_fix_freelist(&args, 0);
+	libxfs_perag_put(args.pag);
+	libxfs_trans_commit(tp);
+}
+
 int
 main(
 	int			argc,
 	char			**argv)
 {
 	uint64_t		agcount;
-	xfs_agf_t		*agf;
-	xfs_agi_t		*agi;
 	xfs_agnumber_t		agno;
 	uint64_t		agsize;
-	xfs_alloc_rec_t		*arec;
-	struct xfs_btree_block	*block;
 	int			blflag;
 	int			blocklog;
 	int			bsflag;
-	int			bsize;
 	xfs_buf_t		*buf;
 	int			c;
 	int			daflag;
@@ -2152,7 +2531,6 @@ main(
 	int			nlflag;
 	int			nodsflag;
 	int			norsflag;
-	xfs_alloc_rec_t		*nrec;
 	int			nsflag;
 	int			nvflag;
 	int			Nflag;
@@ -3153,7 +3531,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	validate_log_size(logblocks, blocklog, min_logblocks);
 
 	protostring = setup_proto(protofile);
-	bsize = 1 << (blocklog - BBSHIFT);
 	mp = &mbuf;
 	sbp = &mp->m_sb;
 	memset(mp, 0, sizeof(xfs_mount_t));
@@ -3284,363 +3661,16 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	}
 
 	/*
-	 * XXX: this code is effectively shared with the kernel growfs code.
-	 * These initialisations should be pulled into libxfs to keep the
-	 * kernel/userspace header initialisation code the same.
+	 * Initialise all the static on disk metadata.
 	 */
-	for (agno = 0; agno < agcount; agno++) {
-		struct xfs_agfl	*agfl;
-		int		bucket;
-		struct xfs_perag *pag = libxfs_perag_get(mp, agno);
-
-		/*
-		 * Superblock.
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AG_DADDR(mp, agno, XFS_SB_DADDR),
-				XFS_FSS_TO_BB(mp, 1));
-		buf->b_ops = &xfs_sb_buf_ops;
-		memset(XFS_BUF_PTR(buf), 0, sectorsize);
-		libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * AG header block: freespace
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AG_DADDR(mp, agno, XFS_AGF_DADDR(mp)),
-				XFS_FSS_TO_BB(mp, 1));
-		buf->b_ops = &xfs_agf_buf_ops;
-		agf = XFS_BUF_TO_AGF(buf);
-		memset(agf, 0, sectorsize);
-		if (agno == agcount - 1)
-			agsize = dblocks - (xfs_rfsblock_t)(agno * agsize);
-		agf->agf_magicnum = cpu_to_be32(XFS_AGF_MAGIC);
-		agf->agf_versionnum = cpu_to_be32(XFS_AGF_VERSION);
-		agf->agf_seqno = cpu_to_be32(agno);
-		agf->agf_length = cpu_to_be32(agsize);
-		agf->agf_roots[XFS_BTNUM_BNOi] = cpu_to_be32(XFS_BNO_BLOCK(mp));
-		agf->agf_roots[XFS_BTNUM_CNTi] = cpu_to_be32(XFS_CNT_BLOCK(mp));
-		agf->agf_levels[XFS_BTNUM_BNOi] = cpu_to_be32(1);
-		agf->agf_levels[XFS_BTNUM_CNTi] = cpu_to_be32(1);
-		pag->pagf_levels[XFS_BTNUM_BNOi] = 1;
-		pag->pagf_levels[XFS_BTNUM_CNTi] = 1;
-		if (xfs_sb_version_hasrmapbt(&mp->m_sb)) {
-			agf->agf_roots[XFS_BTNUM_RMAPi] =
-						cpu_to_be32(XFS_RMAP_BLOCK(mp));
-			agf->agf_levels[XFS_BTNUM_RMAPi] = cpu_to_be32(1);
-			agf->agf_rmap_blocks = cpu_to_be32(1);
-		}
-		if (xfs_sb_version_hasreflink(&mp->m_sb)) {
-			agf->agf_refcount_root = cpu_to_be32(
-					libxfs_refc_block(mp));
-			agf->agf_refcount_level = cpu_to_be32(1);
-			agf->agf_refcount_blocks = cpu_to_be32(1);
-		}
-		agf->agf_flfirst = 0;
-		agf->agf_fllast = cpu_to_be32(XFS_AGFL_SIZE(mp) - 1);
-		agf->agf_flcount = 0;
-		nbmblocks = (xfs_extlen_t)(agsize - libxfs_prealloc_blocks(mp));
-		agf->agf_freeblks = cpu_to_be32(nbmblocks);
-		agf->agf_longest = cpu_to_be32(nbmblocks);
-		if (xfs_sb_version_hascrc(&mp->m_sb))
-			platform_uuid_copy(&agf->agf_uuid, &mp->m_sb.sb_uuid);
-
-		if (loginternal && agno == logagno) {
-			be32_add_cpu(&agf->agf_freeblks, -logblocks);
-			agf->agf_longest = cpu_to_be32(agsize -
-				XFS_FSB_TO_AGBNO(mp, logstart) - logblocks);
-		}
-		if (libxfs_alloc_min_freelist(mp, pag) > worst_freelist)
-			worst_freelist = libxfs_alloc_min_freelist(mp, pag);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * AG freelist header block
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AG_DADDR(mp, agno, XFS_AGFL_DADDR(mp)),
-				XFS_FSS_TO_BB(mp, 1));
-		buf->b_ops = &xfs_agfl_buf_ops;
-		agfl = XFS_BUF_TO_AGFL(buf);
-		/* setting to 0xff results in initialisation to NULLAGBLOCK */
-		memset(agfl, 0xff, sectorsize);
-		if (xfs_sb_version_hascrc(&mp->m_sb)) {
-			agfl->agfl_magicnum = cpu_to_be32(XFS_AGFL_MAGIC);
-			agfl->agfl_seqno = cpu_to_be32(agno);
-			platform_uuid_copy(&agfl->agfl_uuid, &mp->m_sb.sb_uuid);
-			for (bucket = 0; bucket < XFS_AGFL_SIZE(mp); bucket++)
-				agfl->agfl_bno[bucket] = cpu_to_be32(NULLAGBLOCK);
-		}
-
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * AG header block: inodes
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AG_DADDR(mp, agno, XFS_AGI_DADDR(mp)),
-				XFS_FSS_TO_BB(mp, 1));
-		agi = XFS_BUF_TO_AGI(buf);
-		buf->b_ops = &xfs_agi_buf_ops;
-		memset(agi, 0, sectorsize);
-		agi->agi_magicnum = cpu_to_be32(XFS_AGI_MAGIC);
-		agi->agi_versionnum = cpu_to_be32(XFS_AGI_VERSION);
-		agi->agi_seqno = cpu_to_be32(agno);
-		agi->agi_length = cpu_to_be32((xfs_agblock_t)agsize);
-		agi->agi_count = 0;
-		agi->agi_root = cpu_to_be32(XFS_IBT_BLOCK(mp));
-		agi->agi_level = cpu_to_be32(1);
-		if (sb_feat.finobt) {
-			agi->agi_free_root = cpu_to_be32(XFS_FIBT_BLOCK(mp));
-			agi->agi_free_level = cpu_to_be32(1);
-		}
-		agi->agi_freecount = 0;
-		agi->agi_newino = cpu_to_be32(NULLAGINO);
-		agi->agi_dirino = cpu_to_be32(NULLAGINO);
-		if (xfs_sb_version_hascrc(&mp->m_sb))
-			platform_uuid_copy(&agi->agi_uuid, &mp->m_sb.sb_uuid);
-		for (c = 0; c < XFS_AGI_UNLINKED_BUCKETS; c++)
-			agi->agi_unlinked[c] = cpu_to_be32(NULLAGINO);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * BNO btree root block
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AGB_TO_DADDR(mp, agno, XFS_BNO_BLOCK(mp)),
-				bsize);
-		buf->b_ops = &xfs_allocbt_buf_ops;
-		block = XFS_BUF_TO_BLOCK(buf);
-		memset(block, 0, blocksize);
-		libxfs_btree_init_block(mp, buf, XFS_BTNUM_BNO, 0, 1, agno, 0);
-
-		arec = XFS_ALLOC_REC_ADDR(mp, block, 1);
-		arec->ar_startblock = cpu_to_be32(libxfs_prealloc_blocks(mp));
-		if (loginternal && agno == logagno) {
-			if (lalign) {
-				/*
-				 * Have to insert two records
-				 * Insert pad record for stripe align of log
-				 */
-				arec->ar_blockcount = cpu_to_be32(
-					XFS_FSB_TO_AGBNO(mp, logstart) -
-					be32_to_cpu(arec->ar_startblock));
-				nrec = arec + 1;
-				/*
-				 * Insert record at start of internal log
-				 */
-				nrec->ar_startblock = cpu_to_be32(
-					be32_to_cpu(arec->ar_startblock) +
-					be32_to_cpu(arec->ar_blockcount));
-				arec = nrec;
-				be16_add_cpu(&block->bb_numrecs, 1);
-			}
-			/*
-			 * Change record start to after the internal log
-			 */
-			be32_add_cpu(&arec->ar_startblock, logblocks);
-		}
-		/*
-		 * Calculate the record block count and check for the case where
-		 * the log might have consumed all available space in the AG. If
-		 * so, reset the record count to 0 to avoid exposure of an invalid
-		 * record start block.
-		 */
-		arec->ar_blockcount = cpu_to_be32(agsize -
-					be32_to_cpu(arec->ar_startblock));
-		if (!arec->ar_blockcount)
-			block->bb_numrecs = 0;
-
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * CNT btree root block
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AGB_TO_DADDR(mp, agno, XFS_CNT_BLOCK(mp)),
-				bsize);
-		buf->b_ops = &xfs_allocbt_buf_ops;
-		block = XFS_BUF_TO_BLOCK(buf);
-		memset(block, 0, blocksize);
-		libxfs_btree_init_block(mp, buf, XFS_BTNUM_CNT, 0, 1, agno, 0);
-
-		arec = XFS_ALLOC_REC_ADDR(mp, block, 1);
-		arec->ar_startblock = cpu_to_be32(libxfs_prealloc_blocks(mp));
-		if (loginternal && agno == logagno) {
-			if (lalign) {
-				arec->ar_blockcount = cpu_to_be32(
-					XFS_FSB_TO_AGBNO(mp, logstart) -
-					be32_to_cpu(arec->ar_startblock));
-				nrec = arec + 1;
-				nrec->ar_startblock = cpu_to_be32(
-					be32_to_cpu(arec->ar_startblock) +
-					be32_to_cpu(arec->ar_blockcount));
-				arec = nrec;
-				be16_add_cpu(&block->bb_numrecs, 1);
-			}
-			be32_add_cpu(&arec->ar_startblock, logblocks);
-		}
-		/*
-		 * Calculate the record block count and check for the case where
-		 * the log might have consumed all available space in the AG. If
-		 * so, reset the record count to 0 to avoid exposure of an invalid
-		 * record start block.
-		 */
-		arec->ar_blockcount = cpu_to_be32(agsize -
-					be32_to_cpu(arec->ar_startblock));
-		if (!arec->ar_blockcount)
-			block->bb_numrecs = 0;
-
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * refcount btree root block
-		 */
-		if (xfs_sb_version_hasreflink(&mp->m_sb)) {
-			buf = libxfs_getbuf(mp->m_ddev_targp,
-					XFS_AGB_TO_DADDR(mp, agno,
-						libxfs_refc_block(mp)),
-					bsize);
-			buf->b_ops = &xfs_refcountbt_buf_ops;
-
-			block = XFS_BUF_TO_BLOCK(buf);
-			memset(block, 0, blocksize);
-			libxfs_btree_init_block(mp, buf, XFS_BTNUM_REFC, 0,
-						0, agno, 0);
-
-			libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-		}
-
-		/*
-		 * INO btree root block
-		 */
-		buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AGB_TO_DADDR(mp, agno, XFS_IBT_BLOCK(mp)),
-				bsize);
-		buf->b_ops = &xfs_inobt_buf_ops;
-		block = XFS_BUF_TO_BLOCK(buf);
-		memset(block, 0, blocksize);
-		libxfs_btree_init_block(mp, buf, XFS_BTNUM_INO, 0, 0, agno, 0);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
-		/*
-		 * Free INO btree root block
-		 */
-		if (sb_feat.finobt) {
-			buf = libxfs_getbuf(mp->m_ddev_targp,
-					XFS_AGB_TO_DADDR(mp, agno, XFS_FIBT_BLOCK(mp)),
-					bsize);
-			buf->b_ops = &xfs_inobt_buf_ops;
-			block = XFS_BUF_TO_BLOCK(buf);
-			memset(block, 0, blocksize);
-			libxfs_btree_init_block(mp, buf, XFS_BTNUM_FINO, 0, 0, agno, 0);
-			libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-		}
-
-		/* RMAP btree root block */
-		if (xfs_sb_version_hasrmapbt(&mp->m_sb)) {
-			struct xfs_rmap_rec	*rrec;
-
-			buf = libxfs_getbuf(mp->m_ddev_targp,
-				XFS_AGB_TO_DADDR(mp, agno, XFS_RMAP_BLOCK(mp)),
-				bsize);
-			buf->b_ops = &xfs_rmapbt_buf_ops;
-			block = XFS_BUF_TO_BLOCK(buf);
-			memset(block, 0, blocksize);
-
-			libxfs_btree_init_block(mp, buf, XFS_BTNUM_RMAP, 0, 0, agno, 0);
-
-			/*
-			 * mark the AG header regions as static metadata
-			 * The BNO btree block is the first block after the
-			 * headers, so it's location defines the size of region
-			 * the static metadata consumes.
-			 */
-			rrec = XFS_RMAP_REC_ADDR(block, 1);
-			rrec->rm_startblock = 0;
-			rrec->rm_blockcount = cpu_to_be32(XFS_BNO_BLOCK(mp));
-			rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_FS);
-			rrec->rm_offset = 0;
-			be16_add_cpu(&block->bb_numrecs, 1);
-
-			/* account freespace btree root blocks */
-			rrec = XFS_RMAP_REC_ADDR(block, 2);
-			rrec->rm_startblock = cpu_to_be32(XFS_BNO_BLOCK(mp));
-			rrec->rm_blockcount = cpu_to_be32(2);
-			rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_AG);
-			rrec->rm_offset = 0;
-			be16_add_cpu(&block->bb_numrecs, 1);
-
-			/* account inode btree root blocks */
-			rrec = XFS_RMAP_REC_ADDR(block, 3);
-			rrec->rm_startblock = cpu_to_be32(XFS_IBT_BLOCK(mp));
-			rrec->rm_blockcount = cpu_to_be32(XFS_RMAP_BLOCK(mp) -
-							XFS_IBT_BLOCK(mp));
-			rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_INOBT);
-			rrec->rm_offset = 0;
-			be16_add_cpu(&block->bb_numrecs, 1);
-
-			/* account for rmap btree root */
-			rrec = XFS_RMAP_REC_ADDR(block, 4);
-			rrec->rm_startblock = cpu_to_be32(XFS_RMAP_BLOCK(mp));
-			rrec->rm_blockcount = cpu_to_be32(1);
-			rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_AG);
-			rrec->rm_offset = 0;
-			be16_add_cpu(&block->bb_numrecs, 1);
-
-			/* account for refcount btree root */
-			if (xfs_sb_version_hasreflink(&mp->m_sb)) {
-				rrec = XFS_RMAP_REC_ADDR(block, 5);
-				rrec->rm_startblock = cpu_to_be32(
-							libxfs_refc_block(mp));
-				rrec->rm_blockcount = cpu_to_be32(1);
-				rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_REFC);
-				rrec->rm_offset = 0;
-				be16_add_cpu(&block->bb_numrecs, 1);
-			}
-
-			/* account for the log space */
-			if (loginternal && agno == logagno) {
-				rrec = XFS_RMAP_REC_ADDR(block,
-					be16_to_cpu(block->bb_numrecs) + 1);
-				rrec->rm_startblock = cpu_to_be32(
-						XFS_FSB_TO_AGBNO(mp, logstart));
-				rrec->rm_blockcount = cpu_to_be32(logblocks);
-				rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_LOG);
-				rrec->rm_offset = 0;
-				be16_add_cpu(&block->bb_numrecs, 1);
-			}
-
-			libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-		}
-
-		libxfs_perag_put(pag);
-	}
+	for (agno = 0; agno < cfg.agcount; agno++)
+		initialise_ag_headers(&cfg, mp, sbp, agno, &worst_freelist);
 
 	/*
-	 * BNO, CNT free block list
+	 * Initialise the freespace freelists (i.e. AGFLs) in each AG.
 	 */
-	for (agno = 0; agno < agcount; agno++) {
-		xfs_alloc_arg_t	args;
-		xfs_trans_t	*tp;
-		struct xfs_trans_res tres = {0};
-
-		c = libxfs_trans_alloc(mp, &tres, worst_freelist, 0, 0, &tp);
-		if (c)
-			res_failed(c);
-
-		memset(&args, 0, sizeof(args));
-		args.tp = tp;
-		args.mp = mp;
-		args.agno = agno;
-		args.alignment = 1;
-		args.pag = libxfs_perag_get(mp,agno);
-
-		libxfs_alloc_fix_freelist(&args, 0);
-		libxfs_perag_put(args.pag);
-		libxfs_trans_commit(tp);
-	}
+	for (agno = 0; agno < cfg.agcount; agno++)
+		initialise_ag_freespace(mp, agno, worst_freelist);
 
 	/*
 	 * Allocate the root inode and anything else in the proto file.
-- 
2.13.3

[PATCH 18/42] mkfs: factor secondary superblock updates

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 63 ++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 36 insertions(+), 27 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 13af1ef1cd41..1fd5108be520 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2468,6 +2468,39 @@ initialise_ag_freespace(
 	libxfs_trans_commit(tp);
 }
 
+/*
+ * rewrite several secondary superblocks with the root inode number filled out.
+ * This can help repair recovery from a trashed primary superblock without
+ * losing the root inode.
+ */
+static void
+rewrite_secondary_superblocks(
+	struct xfs_mount	*mp)
+{
+	struct xfs_buf		*buf;
+
+	/* rewrite the last superblock */
+	buf = libxfs_readbuf(mp->m_dev,
+			XFS_AGB_TO_DADDR(mp, mp->m_sb.sb_agcount - 1,
+				XFS_SB_DADDR),
+			XFS_FSS_TO_BB(mp, 1),
+			LIBXFS_EXIT_ON_FAILURE, &xfs_sb_buf_ops);
+	XFS_BUF_TO_SBP(buf)->sb_rootino = cpu_to_be64(mp->m_sb.sb_rootino);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+
+	/* and one in the middle for luck if there's enough AGs for that */
+	if (mp->m_sb.sb_agcount <= 2)
+		return;
+
+	buf = libxfs_readbuf(mp->m_dev,
+			XFS_AGB_TO_DADDR(mp, (mp->m_sb.sb_agcount - 1) / 2,
+				XFS_SB_DADDR),
+			XFS_FSS_TO_BB(mp, 1),
+			LIBXFS_EXIT_ON_FAILURE, &xfs_sb_buf_ops);
+	XFS_BUF_TO_SBP(buf)->sb_rootino = cpu_to_be64(mp->m_sb.sb_rootino);
+	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+}
+
 int
 main(
 	int			argc,
@@ -3688,34 +3721,10 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	}
 
 	/*
-	 * Write out multiple secondary superblocks with rootinode field set
+	 * Re-write multiple secondary superblocks with rootinode field set
 	 */
-	if (mp->m_sb.sb_agcount > 1) {
-		/*
-		 * the last superblock
-		 */
-		buf = libxfs_readbuf(mp->m_dev,
-				XFS_AGB_TO_DADDR(mp, mp->m_sb.sb_agcount-1,
-					XFS_SB_DADDR),
-				XFS_FSS_TO_BB(mp, 1),
-				LIBXFS_EXIT_ON_FAILURE, &xfs_sb_buf_ops);
-		XFS_BUF_TO_SBP(buf)->sb_rootino = cpu_to_be64(
-							mp->m_sb.sb_rootino);
-		libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-		/*
-		 * and one in the middle for luck
-		 */
-		if (mp->m_sb.sb_agcount > 2) {
-			buf = libxfs_readbuf(mp->m_dev,
-				XFS_AGB_TO_DADDR(mp, (mp->m_sb.sb_agcount-1)/2,
-					XFS_SB_DADDR),
-				XFS_FSS_TO_BB(mp, 1),
-				LIBXFS_EXIT_ON_FAILURE, &xfs_sb_buf_ops);
-			XFS_BUF_TO_SBP(buf)->sb_rootino = cpu_to_be64(
-							mp->m_sb.sb_rootino);
-			libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-		}
-	}
+	if (mp->m_sb.sb_agcount > 1)
+		rewrite_secondary_superblocks(mp);
 
 	/*
 	 * Dump all inodes and buffers before marking us all done.
-- 
2.13.3

[PATCH 19/42] mkfs: introduce default configuration structure

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

mkfs has lots of options that require default values. Some of these
are centralised, but others aren't. Introduce a new structure
designed to hold default values for all the parameters that need
defaults in one place.

This structure also provides a mechanism for providing mkfs defaults
from a config file. This is not implemented in this series, but a
comment is left where it is expected this functionality will hook
in.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 77 +++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 61 insertions(+), 16 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 1fd5108be520..e53e56a1bd47 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -851,6 +851,27 @@ struct mkfs_params {
 	struct sb_feat_args	sb_feat;
 };
 
+/*
+ * Default filesystem features and configuration values
+ *
+ * This structure contains the default mkfs values that are to be used when
+ * a user does not specify the option on the command line. We do not use these
+ * values directly - they are inputs to the mkfs geometry validation and
+ * calculations.
+ */
+struct mkfs_default_params {
+	char	*source;	/* where the defaults came from */
+
+	int	sectorsize;
+	int	blocksize;
+
+	/* feature flags that are set */
+	struct sb_feat_args	sb_feat;
+
+	/* root inode characteristics */
+	struct fsxattr		fsx;
+};
+
 #define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
 #define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
 #define MEGABYTES(count, blog)	((uint64_t)(count) << (20 - (blog)))
@@ -2587,25 +2608,33 @@ main(
 	int			worst_freelist;
 	libxfs_init_t		xi;
 	struct fs_topology	ft;
-	struct sb_feat_args	sb_feat = {
-		.finobt = 1,
-		.spinodes = 0,
-		.log_version = 2,
-		.attr_version = 2,
-		.dir_version = XFS_DFL_DIR_VERSION,
-		.inode_align = XFS_IFLAG_ALIGN,
-		.nci = false,
-		.lazy_sb_counters = true,
-		.projid16bit = false,
-		.crcs_enabled = true,
-		.dirftype = true,
-		.parent_pointers = false,
-		.rmapbt = false,
-		.reflink = false,
+	struct sb_feat_args	sb_feat;
+	/* build time defaults */
+	struct mkfs_default_params	dft = {
+		.source = "package build definitions",
+		.sectorsize = XFS_MIN_SECTORSIZE,
+		.blocksize = 1 << XFS_DFL_BLOCKSIZE_LOG,
+		.sb_feat = {
+			.log_version = 2,
+			.attr_version = 2,
+			.dir_version = XFS_DFL_DIR_VERSION,
+			.inode_align = XFS_IFLAG_ALIGN,
+			.nci = false,
+			.lazy_sb_counters = true,
+			.projid16bit = false,
+			.crcs_enabled = true,
+			.dirftype = true,
+			.finobt = true,
+			.spinodes = false,
+			.rmapbt = false,
+			.reflink = false,
+			.parent_pointers = false,
+			.nodalign = false,
+			.nortalign = false,
+		},
 	};
 	struct cli_params	cli = {
 		.xi = &xi,
-		.sb_feat = sb_feat,
 	};
 	struct mkfs_params	cfg = {};
 
@@ -2615,6 +2644,22 @@ main(
 	bindtextdomain(PACKAGE, LOCALEDIR);
 	textdomain(PACKAGE);
 
+	/*
+	 * TODO: Sourcing defaults from a config file
+	 *
+	 * Before anything else, see if there's a config file with different
+	 * defaults. If a file exists in <package location>, read in the new
+	 * default values and overwrite them in the &dft structure. This way the
+	 * new defaults will apply before we parse the CLI, and the CLI will
+	 * still be able to override them. Emit a message to indicate where the
+	 * defaults being used came from.
+	 */
+	printf(_("Default configuration sourced from %s\n"), dft.source);
+
+	/* copy new defaults into CLI parsing structure */
+	memcpy(&cli.sb_feat, &dft.sb_feat, sizeof(cli.sb_feat));
+	memcpy(&cli.fsx, &dft.fsx, sizeof(cli.fsx));
+
 	blflag = bsflag = slflag = ssflag = lslflag = lssflag = 0;
 	blocklog = blocksize = 0;
 	sectorlog = lsectorlog = 0;
-- 
2.13.3

[PATCH 20/42] mkfs: rename top level CLI parameters

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Names like "qflag" and "Nflag" don't tell us what the flag is
supposed to do, so rename them to be obvious in function.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index e53e56a1bd47..7b71f59dc654 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2587,11 +2587,11 @@ main(
 	int			norsflag;
 	int			nsflag;
 	int			nvflag;
-	int			Nflag;
+	int			dry_run;
 	int			discard = 1;
 	char			*protofile;
 	char			*protostring;
-	int			qflag;
+	int			quiet;
 	xfs_rfsblock_t		rtblocks;
 	xfs_extlen_t		rtextblocks;
 	xfs_rtblock_t		rtextents;
@@ -2669,9 +2669,9 @@ main(
 	liflag = laflag = lsflag = lsuflag = lsunitflag = ldflag = lvflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
-	Nflag = nlflag = nsflag = nvflag = 0;
+	dry_run = nlflag = nsflag = nvflag = 0;
 	dirblocklog = dirblocksize = 0;
-	qflag = 0;
+	quiet = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
 	dsize = logsize = rtsize = rtextsize = protofile = NULL;
@@ -2800,7 +2800,7 @@ main(
 			/* end temp don't break code */
 			break;
 		case 'N':
-			Nflag = 1;
+			dry_run = 1;
 			break;
 		case 'K':
 			discard = 0;
@@ -2811,7 +2811,7 @@ main(
 			protofile = optarg;
 			break;
 		case 'q':
-			qflag = 1;
+			quiet = 1;
 			break;
 		case 'r':
 			parse_subopts(c, optarg, &cli);
@@ -2896,14 +2896,14 @@ _("Minimum block size for CRC enabled filesystems is %d bytes.\n"),
 	 * host filesystem.
 	 */
 	check_device_type(dfile, &xi.disfile, !dsize, !dfile,
-			  Nflag ? NULL : &xi.dcreat, force_overwrite, "d");
+			  dry_run ? NULL : &xi.dcreat, force_overwrite, "d");
 	if (!loginternal)
 		check_device_type(xi.logname, &xi.lisfile, !logsize, !xi.logname,
-				  Nflag ? NULL : &xi.lcreat,
+				  dry_run ? NULL : &xi.lcreat,
 				  force_overwrite, "l");
 	if (xi.rtname)
 		check_device_type(xi.rtname, &xi.risfile, !rtsize, !xi.rtname,
-				  Nflag ? NULL : &xi.rcreat,
+				  dry_run ? NULL : &xi.rcreat,
 				  force_overwrite, "r");
 	if (xi.disfile || xi.lisfile || xi.risfile)
 		xi.isdirect = 0;
@@ -3259,7 +3259,7 @@ _("rmapbt not supported with realtime devices\n"));
 
 
 	/* don't do discards on print-only runs or on files */
-	if (discard && !Nflag) {
+	if (discard && !dry_run) {
 		if (!xi.disfile)
 			discard_blocks(xi.ddev, xi.dsize);
 		if (xi.rtdev && !xi.risfile)
@@ -3709,9 +3709,9 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	memcpy(&cfg.sb_feat, &sb_feat, sizeof(sb_feat));
 	/* end temp support code */
 
-	if (!qflag || Nflag) {
+	if (!quiet || dry_run) {
 		print_mkfs_cfg(&cfg, dfile, logfile, rtfile);
-		if (Nflag)
+		if (dry_run)
 			exit(0);
 	}
 
-- 
2.13.3

[PATCH 21/42] mkfs: factor sectorsize validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Start factoring all the sector size validation code into a
function that takes cli, dft and cfg structures. This starts
removing option flags and some of the temporary code in the input
parsing structures.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 207 +++++++++++++++++++++++++++++++-------------------------
 1 file changed, 114 insertions(+), 93 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 7b71f59dc654..a6d2dd516319 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1903,6 +1903,107 @@ parse_subopts(
 }
 
 static void
+validate_sectorsize(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	struct mkfs_default_params *dft,
+	struct fs_topology	*ft,
+	char			*dfile,
+	int			dry_run,
+	int			force_overwrite)
+{
+	/* set configured sector sizes in preparation for checks */
+	if (!cli->sectorsize) {
+		cfg->sectorsize = dft->sectorsize;
+	} else {
+		cfg->sectorsize = cli->sectorsize;
+	}
+	cfg->sectorlog = libxfs_highbit32(cfg->sectorsize);
+
+	/*
+	 * Before anything else, verify that we are correctly operating on
+	 * files or block devices and set the control parameters correctly.
+	 */
+	check_device_type(dfile, &cli->xi->disfile, !cli->dsize, !dfile,
+			  dry_run ? NULL : &cli->xi->dcreat,
+			  force_overwrite, "d");
+	if (!cli->loginternal)
+		check_device_type(cli->xi->logname, &cli->xi->lisfile,
+				  !cli->logsize, !cli->xi->logname,
+				  dry_run ? NULL : &cli->xi->lcreat,
+				  force_overwrite, "l");
+	if (cli->xi->rtname)
+		check_device_type(cli->xi->rtname, &cli->xi->risfile,
+				  !cli->rtsize, !cli->xi->rtname,
+				  dry_run ? NULL : &cli->xi->rcreat,
+				  force_overwrite, "r");
+
+	/*
+	 * Explicitly disable direct IO for image files so we don't error out on
+	 * sector size mismatches between the new filesystem and the underlying
+	 * host filesystem.
+	 */
+	if (cli->xi->disfile || cli->xi->lisfile || cli->xi->risfile)
+		cli->xi->isdirect = 0;
+
+	memset(ft, 0, sizeof(*ft));
+	get_topology(cli->xi, ft, force_overwrite);
+
+	if (!cli->sectorsize) {
+		/*
+		 * Unless specified manually on the command line use the
+		 * advertised sector size of the device.  We use the physical
+		 * sector size unless the requested block size is smaller
+		 * than that, then we can use logical, but warn about the
+		 * inefficiency.
+		 *
+		 * Set the topology sectors if they were not probed to the
+		 * minimum supported sector size.
+		 */
+
+		if (!ft->lsectorsize)
+			ft->lsectorsize = XFS_MIN_SECTORSIZE;
+
+		/* Older kernels may not have physical/logical distinction */
+		if (!ft->psectorsize)
+			ft->psectorsize = ft->lsectorsize;
+
+		cfg->sectorsize = ft->psectorsize;
+		if (cfg->blocksize < cfg->sectorsize &&
+		    cfg->blocksize >= ft->lsectorsize) {
+			fprintf(stderr,
+_("specified blocksize %d is less than device physical sector size %d\n"
+  "switching to logical sector size %d\n"),
+				cfg->blocksize, ft->psectorsize,
+				ft->lsectorsize);
+			cfg->sectorsize = ft->lsectorsize;
+		}
+
+		cfg->sectorlog = libxfs_highbit32(cfg->sectorsize);
+	}
+
+	/* validate specified/probed sector size */
+	if (cfg->sectorsize < XFS_MIN_SECTORSIZE ||
+	    cfg->sectorsize > XFS_MAX_SECTORSIZE) {
+		fprintf(stderr, _("illegal sector size %d\n"), cfg->sectorsize);
+		usage();
+	}
+
+	if (cfg->blocksize < cfg->sectorsize) {
+		fprintf(stderr,
+_("block size %d cannot be smaller than sector size %d\n"),
+			cfg->blocksize, cfg->sectorsize);
+		usage();
+	}
+
+	if (cfg->sectorsize < ft->lsectorsize) {
+		fprintf(stderr, _("illegal sector size %d; hw sector is %d\n"),
+			cfg->sectorsize, ft->lsectorsize);
+		usage();
+	}
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -2601,8 +2702,6 @@ main(
 	xfs_sb_t		*sbp;
 	int			sectorlog;
 	uint64_t		sector_mask;
-	int			slflag;
-	int			ssflag;
 	uint64_t		tmp_agsize;
 	uuid_t			uuid;
 	int			worst_freelist;
@@ -2660,10 +2759,10 @@ main(
 	memcpy(&cli.sb_feat, &dft.sb_feat, sizeof(cli.sb_feat));
 	memcpy(&cli.fsx, &dft.fsx, sizeof(cli.fsx));
 
-	blflag = bsflag = slflag = ssflag = lslflag = lssflag = 0;
+	blflag = bsflag = lslflag = lssflag = 0;
 	blocklog = blocksize = 0;
-	sectorlog = lsectorlog = 0;
-	sectorsize = lsectorsize = 0;
+	lsectorlog = 0;
+	lsectorsize = 0;
 	agsize = daflag = dasize = dblocks = 0;
 	ilflag = imflag = ipflag = isflag = 0;
 	liflag = laflag = lsflag = lsuflag = lsunitflag = ldflag = lvflag = 0;
@@ -2724,11 +2823,6 @@ main(
 				  cli_opt_set(&dopts, D_SWIDTH);
 			nodsflag = cli_opt_set(&dopts, D_NOALIGN);
 
-			sectorsize = cli.sectorsize;
-			sectorlog = libxfs_highbit32(sectorsize);
-			slflag = cli_opt_set(&dopts, D_SECTLOG);
-			ssflag = cli_opt_set(&dopts, D_SECTSIZE);
-
 			fsx.fsx_xflags |= cli.fsx.fsx_xflags;
 			fsx.fsx_projid = cli.fsx.fsx_projid;
 			fsx.fsx_extsize = cli.fsx.fsx_extsize;
@@ -2826,14 +2920,12 @@ main(
 			parse_subopts(c, optarg, &cli);
 
 			/* temp don't break code */
-			sectorsize = cli.sectorsize;
-			lsectorlog = libxfs_highbit32(sectorsize);
 			lsectorsize = cli.lsectorsize;
 			lsectorlog = libxfs_highbit32(lsectorsize);
-			lslflag = slflag = cli_opt_set(&sopts, S_LOG) ||
+			lslflag = cli_opt_set(&sopts, S_LOG) ||
 					   cli_opt_set(&sopts, S_SECTLOG);
 
-			lssflag = ssflag = cli_opt_set(&sopts, S_SIZE) ||
+			lssflag = cli_opt_set(&sopts, S_SIZE) ||
 					   cli_opt_set(&sopts, S_SECTSIZE);
 			break;
 		case 'V':
@@ -2879,89 +2971,18 @@ _("Minimum block size for CRC enabled filesystems is %d bytes.\n"),
 		usage();
 	}
 
-	if (!slflag && !ssflag) {
-		sectorlog = XFS_MIN_SECTORSIZE_LOG;
-		sectorsize = XFS_MIN_SECTORSIZE;
-	}
-	if (!lslflag && !lssflag) {
-		lsectorlog = sectorlog;
-		lsectorsize = sectorsize;
-	}
-
 	/*
-	 * Before anything else, verify that we are correctly operating on
-	 * files or block devices and set the control parameters correctly.
-	 * Explicitly disable direct IO for image files so we don't error out on
-	 * sector size mismatches between the new filesystem and the underlying
-	 * host filesystem.
+	 * Extract as much of the valid config as we can from the CLI input
+	 * before opening the libxfs devices.
 	 */
-	check_device_type(dfile, &xi.disfile, !dsize, !dfile,
-			  dry_run ? NULL : &xi.dcreat, force_overwrite, "d");
-	if (!loginternal)
-		check_device_type(xi.logname, &xi.lisfile, !logsize, !xi.logname,
-				  dry_run ? NULL : &xi.lcreat,
-				  force_overwrite, "l");
-	if (xi.rtname)
-		check_device_type(xi.rtname, &xi.risfile, !rtsize, !xi.rtname,
-				  dry_run ? NULL : &xi.rcreat,
-				  force_overwrite, "r");
-	if (xi.disfile || xi.lisfile || xi.risfile)
-		xi.isdirect = 0;
-
-	memset(&ft, 0, sizeof(ft));
-	get_topology(&xi, &ft, force_overwrite);
-
-	if (!ssflag) {
-		/*
-		 * Unless specified manually on the command line use the
-		 * advertised sector size of the device.  We use the physical
-		 * sector size unless the requested block size is smaller
-		 * than that, then we can use logical, but warn about the
-		 * inefficiency.
-		 */
-
-		/* Older kernels may not have physical/logical distinction */
-		if (!ft.psectorsize)
-			ft.psectorsize = ft.lsectorsize;
-
-		sectorsize = ft.psectorsize ? ft.psectorsize :
-					      XFS_MIN_SECTORSIZE;
-
-		if ((blocksize < sectorsize) && (blocksize >= ft.lsectorsize)) {
-			fprintf(stderr,
-_("specified blocksize %d is less than device physical sector size %d\n"),
-				blocksize, ft.psectorsize);
-			fprintf(stderr,
-_("switching to logical sector size %d\n"),
-				ft.lsectorsize);
-			sectorsize = ft.lsectorsize ? ft.lsectorsize :
-						      XFS_MIN_SECTORSIZE;
-		}
-	}
+	validate_sectorsize(&cfg, &cli, &dft, &ft, dfile, dry_run,
+			    force_overwrite);
 
-	if (!ssflag) {
-		sectorlog = libxfs_highbit32(sectorsize);
-		if (loginternal) {
-			lsectorsize = sectorsize;
-			lsectorlog = sectorlog;
-		}
-	}
+	/* temp don't break code */
+	sectorsize = cfg.sectorsize;
+	sectorlog = cfg.sectorlog;
+	/* end temp don't break code */
 
-	if (sectorsize < XFS_MIN_SECTORSIZE ||
-	    sectorsize > XFS_MAX_SECTORSIZE || sectorsize > blocksize) {
-		if (ssflag)
-			fprintf(stderr, _("illegal sector size %d\n"), sectorsize);
-		else
-			fprintf(stderr,
-_("block size %d cannot be smaller than logical sector size %d\n"),
-				blocksize, ft.lsectorsize);
-		usage();
-	}
-	if (sectorsize < ft.lsectorsize) {
-		fprintf(stderr, _("illegal sector size %d; hw sector is %d\n"),
-			sectorsize, ft.lsectorsize);
-		usage();
-	}
 	if (lsectorsize < XFS_MIN_SECTORSIZE ||
 	    lsectorsize > XFS_MAX_SECTORSIZE || lsectorsize > blocksize) {
 		fprintf(stderr, _("illegal log sector size %d\n"), lsectorsize);
-- 
2.13.3

[PATCH 22/42] mkfs: factor blocksize validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 68 ++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 38 insertions(+), 30 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index a6d2dd516319..4e256942173f 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2004,6 +2004,40 @@ _("block size %d cannot be smaller than sector size %d\n"),
 }
 
 static void
+validate_blocksize(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	struct mkfs_default_params *dft)
+{
+	/*
+	 * Blocksize and sectorsize first, other things depend on them
+	 * For RAID4/5/6 we want to align sector size and block size,
+	 * so we need to start with the device geometry extraction too.
+	 */
+	if (!cli->blocksize)
+		cfg->blocksize = dft->blocksize;
+	else
+		cfg->blocksize = cli->blocksize;
+	cfg->blocklog = libxfs_highbit32(cfg->blocksize);
+
+	/* validate block sizes are in range */
+	if (cfg->blocksize < XFS_MIN_BLOCKSIZE ||
+	    cfg->blocksize > XFS_MAX_BLOCKSIZE) {
+		fprintf(stderr, _("illegal block size %d\n"), cfg->blocksize);
+		usage();
+	}
+
+	if (cli->sb_feat.crcs_enabled &&
+	    cfg->blocksize < XFS_MIN_CRC_BLOCKSIZE) {
+		fprintf(stderr,
+_("Minimum block size for CRC enabled filesystems is %d bytes.\n"),
+			XFS_MIN_CRC_BLOCKSIZE);
+		usage();
+	}
+
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -2631,9 +2665,7 @@ main(
 	uint64_t		agcount;
 	xfs_agnumber_t		agno;
 	uint64_t		agsize;
-	int			blflag;
 	int			blocklog;
-	int			bsflag;
 	xfs_buf_t		*buf;
 	int			c;
 	int			daflag;
@@ -2759,8 +2791,7 @@ main(
 	memcpy(&cli.sb_feat, &dft.sb_feat, sizeof(cli.sb_feat));
 	memcpy(&cli.fsx, &dft.fsx, sizeof(cli.fsx));
 
-	blflag = bsflag = lslflag = lssflag = 0;
-	blocklog = blocksize = 0;
+	lslflag = lssflag = 0;
 	lsectorlog = 0;
 	lsectorsize = 0;
 	agsize = daflag = dasize = dblocks = 0;
@@ -2792,13 +2823,6 @@ main(
 			break;
 		case 'b':
 			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			blocksize = cli.blocksize;
-			blocklog = libxfs_highbit32(blocksize);
-			blflag = cli_opt_set(&bopts, B_LOG);
-			bsflag = cli_opt_set(&bopts, B_SIZE);
-			/* end temp don't break code */
 			break;
 		case 'd':
 			parse_subopts(c, optarg, &cli);
@@ -2947,25 +2971,6 @@ main(
 	sb_feat = cli.sb_feat;
 	/* end temp don't break code */
 
-	/*
-	 * Blocksize and sectorsize first, other things depend on them
-	 * For RAID4/5/6 we want to align sector size and block size,
-	 * so we need to start with the device geometry extraction too.
-	 */
-	if (!blflag && !bsflag) {
-		blocklog = XFS_DFL_BLOCKSIZE_LOG;
-		blocksize = 1 << XFS_DFL_BLOCKSIZE_LOG;
-	}
-	if (blocksize < XFS_MIN_BLOCKSIZE || blocksize > XFS_MAX_BLOCKSIZE) {
-		fprintf(stderr, _("illegal block size %d\n"), blocksize);
-		usage();
-	}
-	if (sb_feat.crcs_enabled && blocksize < XFS_MIN_CRC_BLOCKSIZE) {
-		fprintf(stderr,
-_("Minimum block size for CRC enabled filesystems is %d bytes.\n"),
-			XFS_MIN_CRC_BLOCKSIZE);
-		usage();
-	}
 	if (sb_feat.crcs_enabled && !sb_feat.dirftype) {
 		fprintf(stderr, _("cannot disable ftype with crcs enabled\n"));
 		usage();
@@ -2975,12 +2980,15 @@ _("Minimum block size for CRC enabled filesystems is %d bytes.\n"),
 	 * Extract as much of the valid config as we can from the CLI input
 	 * before opening the libxfs devices.
 	 */
+	validate_blocksize(&cfg, &cli, &dft);
 	validate_sectorsize(&cfg, &cli, &dft, &ft, dfile, dry_run,
 			    force_overwrite);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
+	blocksize = cfg.blocksize;
+	blocklog = cfg.blocklog;
 	/* end temp don't break code */
 
 	if (lsectorsize < XFS_MIN_SECTORSIZE ||
-- 
2.13.3

[PATCH 23/42] mkfs: factor log sector size validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

This also adds a check that disallows changing the sector size on
internal logs.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 101 ++++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 65 insertions(+), 36 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 4e256942173f..64542442148c 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2037,6 +2037,59 @@ _("Minimum block size for CRC enabled filesystems is %d bytes.\n"),
 
 }
 
+/*
+ * Grab log sector size and validate.
+ *
+ * XXX: should we probe sector size on external log device rather than using
+ * the data device sector size?
+ */
+static void
+validate_log_sectorsize(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	struct mkfs_default_params *dft)
+{
+
+	if (cli->loginternal && cli->lsectorsize &&
+	    cli->lsectorsize != cfg->sectorsize) {
+		fprintf(stderr,
+_("Can't change sector size on internal log!\n"));
+		usage();
+	}
+
+	if (!cli->lsectorsize)
+		cfg->lsectorsize = dft->sectorsize;
+	else
+		cfg->lsectorsize = cli->lsectorsize;
+	cfg->lsectorlog = libxfs_highbit32(cfg->lsectorsize);
+
+	if (cfg->lsectorsize < XFS_MIN_SECTORSIZE ||
+	    cfg->lsectorsize > XFS_MAX_SECTORSIZE ||
+	    cfg->lsectorsize > cfg->blocksize) {
+		fprintf(stderr, _("illegal log sector size %d\n"),
+			cfg->lsectorsize);
+		usage();
+	}
+	if (cfg->lsectorsize > XFS_MIN_SECTORSIZE) {
+		if (cli->sb_feat.log_version < 2) {
+			/* user specified non-default log version */
+			fprintf(stderr,
+_("Version 1 logs do not support sector size %d\n"),
+				cfg->lsectorsize);
+			usage();
+		}
+	}
+
+	/* if lsu or lsunit was specified, automatically use v2 logs */
+	if ((cli_opt_set(&lopts, L_SU) || cli_opt_set(&lopts, L_SUNIT)) &&
+	    cli->sb_feat.log_version == 1) {
+		fprintf(stderr,
+_("log stripe unit specified, using v2 logs\n"));
+		cli->sb_feat.log_version = 2;
+	}
+
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -2707,8 +2760,6 @@ main(
 	int			lsunitflag;
 	int			lsectorlog;
 	int			lsectorsize;
-	int			lslflag;
-	int			lssflag;
 	int			lsu;
 	int			lsunit;
 	int			min_logblocks;
@@ -2791,9 +2842,13 @@ main(
 	memcpy(&cli.sb_feat, &dft.sb_feat, sizeof(cli.sb_feat));
 	memcpy(&cli.fsx, &dft.fsx, sizeof(cli.fsx));
 
-	lslflag = lssflag = 0;
-	lsectorlog = 0;
-	lsectorsize = 0;
+	/*
+	 * Initialise cli parameters that can be set to zero to an appropriate
+	 * value so we can tell if they have been set or or changed from the
+	 * default value.
+	 */
+	cli.loginternal = 1;	/* internal by default */
+
 	agsize = daflag = dasize = dblocks = 0;
 	ilflag = imflag = ipflag = isflag = 0;
 	liflag = laflag = lsflag = lsuflag = lsunitflag = ldflag = lvflag = 0;
@@ -2822,6 +2877,7 @@ main(
 			force_overwrite = 1;
 			break;
 		case 'b':
+		case 's':
 			parse_subopts(c, optarg, &cli);
 			break;
 		case 'd':
@@ -2875,8 +2931,6 @@ main(
 			loginternal = cli.loginternal;
 			logfile = xi.logname;
 			logsize = cli.logsize;
-			lsectorsize = cli.lsectorsize;
-			lsectorlog = libxfs_highbit32(lsectorsize);
 
 			lsunit = cli.lsunit;
 			lsunitflag = cli_opt_set(&lopts, L_SUNIT);
@@ -2890,8 +2944,6 @@ main(
 			ldflag = cli_opt_set(&lopts, L_NAME) ||
 				 cli_opt_set(&lopts, L_DEV);
 			lvflag = cli_opt_set(&lopts, L_VERSION);
-			lslflag = cli_opt_set(&lopts, L_SECTLOG);
-			lssflag = cli_opt_set(&lopts, L_SECTSIZE);
 			/* end temp don't break code */
 			break;
 		case 'L':
@@ -2940,17 +2992,6 @@ main(
 			norsflag = cli.sb_feat.nortalign;
 			/* end temp don't break code */
 			break;
-		case 's':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			lsectorsize = cli.lsectorsize;
-			lsectorlog = libxfs_highbit32(lsectorsize);
-			lslflag = cli_opt_set(&sopts, S_LOG) ||
-					   cli_opt_set(&sopts, S_SECTLOG);
-
-			lssflag = cli_opt_set(&sopts, S_SIZE) ||
-					   cli_opt_set(&sopts, S_SECTSIZE);
 			break;
 		case 'V':
 			printf(_("%s version %s\n"), progname, VERSION);
@@ -2983,23 +3024,18 @@ main(
 	validate_blocksize(&cfg, &cli, &dft);
 	validate_sectorsize(&cfg, &cli, &dft, &ft, dfile, dry_run,
 			    force_overwrite);
+	validate_log_sectorsize(&cfg, &cli, &dft);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
 	blocksize = cfg.blocksize;
 	blocklog = cfg.blocklog;
+	lsectorsize = cfg.lsectorsize;
+	lsectorlog = cfg.lsectorlog;
+	sb_feat = cfg.sb_feat;
 	/* end temp don't break code */
 
-	if (lsectorsize < XFS_MIN_SECTORSIZE ||
-	    lsectorsize > XFS_MAX_SECTORSIZE || lsectorsize > blocksize) {
-		fprintf(stderr, _("illegal log sector size %d\n"), lsectorsize);
-		usage();
-	} else if (lsectorsize > XFS_MIN_SECTORSIZE && !lsu && !lsunit) {
-		lsu = blocksize;
-		sb_feat.log_version = 2;
-	}
-
 	/*
 	 * Now we have blocks and sector sizes set up, check parameters that are
 	 * no longer optional for CRC enabled filesystems.  Catch them up front
@@ -3244,13 +3280,6 @@ _("rmapbt not supported with realtime devices\n"));
 		exit(1);
 	}
 
-	/* if lsu or lsunit was specified, automatically use v2 logs */
-	if ((lsu || lsunit) && sb_feat.log_version == 1) {
-		fprintf(stderr,
-			_("log stripe unit specified, using v2 logs\n"));
-		sb_feat.log_version = 2;
-	}
-
 	calc_stripe_factors(dsu, dsw, sectorsize, lsu, lsectorsize,
 				&dsunit, &dswidth, &lsunit);
 
-- 
2.13.3

[PATCH 24/42] mkfs: factor superblock feature validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 228 +++++++++++++++++++++++++++++++-------------------------
 1 file changed, 127 insertions(+), 101 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 64542442148c..1c697dd5d4e1 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2090,6 +2090,130 @@ _("log stripe unit specified, using v2 logs\n"));
 
 }
 
+/*
+ * Check that the incoming features make sense. The CLI structure was
+ * initialised with the default values before parsing, so we can just
+ * check it and copy it straight across to the cfg structure if it
+ * checks out.
+ */
+static void
+validate_sb_features(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli)
+{
+	/*
+	 * Now we have blocks and sector sizes set up, check parameters that are
+	 * no longer optional for CRC enabled filesystems.  Catch them up front
+	 * here before doing anything else.
+	 */
+	if (cli->sb_feat.crcs_enabled) {
+		/* minimum inode size is 512 bytes, rest checked later */
+		if (cli->inodesize &&
+		    cli->inodesize < (1 << XFS_DINODE_DFL_CRC_LOG)) {
+			fprintf(stderr,
+_("Minimum inode size for CRCs is %d bytes\n"),
+				1 << XFS_DINODE_DFL_CRC_LOG);
+			usage();
+		}
+
+		/* inodes always aligned */
+		if (!cli->sb_feat.inode_align) {
+			fprintf(stderr,
+_("Inodes always aligned for CRC enabled filesytems\n"));
+			usage();
+		}
+
+		/* lazy sb counters always on */
+		if (!cli->sb_feat.lazy_sb_counters) {
+			fprintf(stderr,
+_("Lazy superblock counted always enabled for CRC enabled filesytems\n"));
+			usage();
+		}
+
+		/* version 2 logs always on */
+		if (cli->sb_feat.log_version != 2) {
+			fprintf(stderr,
+_("V2 logs always enabled for CRC enabled filesytems\n"));
+			usage();
+		}
+
+		/* attr2 always on */
+		if (cli->sb_feat.attr_version != 2) {
+			fprintf(stderr,
+_("V2 attribute format always enabled on CRC enabled filesytems\n"));
+			usage();
+		}
+
+		/* 32 bit project quota always on */
+		/* attr2 always on */
+		if (cli->sb_feat.projid16bit) {
+			fprintf(stderr,
+_("32 bit Project IDs always enabled on CRC enabled filesytems\n"));
+			usage();
+		}
+
+		/* ftype always on */
+		if (!cli->sb_feat.dirftype) {
+			fprintf(stderr,
+_("Directory ftype field always enabled on CRC enabled filesytems\n"));
+			usage();
+		}
+
+	} else {
+		/*
+		 * The kernel doesn't currently support crc=0,finobt=1
+		 * filesystems. If crcs are not enabled and the user has not
+		 * explicitly turned finobt on, then silently turn it off to
+		 * avoid an unnecessary warning.
+		 * If the user explicitly tried to use crc=0,finobt=1,
+		 * then issue an error.
+		 * The same is also for sparse inodes.
+		 */
+		if (cli->sb_feat.finobt && cli_opt_set(&mopts, M_FINOBT)) {
+			fprintf(stderr,
+_("finobt not supported without CRC support\n"));
+			usage();
+		}
+		cli->sb_feat.finobt = false;
+
+		if (cli->sb_feat.spinodes) {
+			fprintf(stderr,
+_("sparse inodes not supported without CRC support\n"));
+			usage();
+		}
+		cli->sb_feat.spinodes = false;
+
+		if (cli->sb_feat.rmapbt) {
+			fprintf(stderr,
+_("rmapbt not supported without CRC support\n"));
+			usage();
+		}
+		cli->sb_feat.rmapbt = false;
+
+		if (cli->sb_feat.reflink) {
+			fprintf(stderr,
+_("reflink not supported without CRC support\n"));
+			usage();
+		}
+		cli->sb_feat.reflink = false;
+	}
+
+
+	if (cli->sb_feat.rmapbt && cli->xi->rtname) {
+		fprintf(stderr,
+_("rmapbt not supported with realtime devices\n"));
+		usage();
+		cli->sb_feat.rmapbt = false;
+	}
+
+	/*
+	 * Copy features across to config structure now.
+	 */
+	cfg->sb_feat = cli->sb_feat;
+	if (!platform_uuid_is_null(&cli->uuid))
+		platform_uuid_copy(&cfg->uuid, &cli->uuid);
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -2820,7 +2944,7 @@ main(
 	};
 	struct mkfs_params	cfg = {};
 
-	platform_uuid_generate(&uuid);
+	platform_uuid_generate(&cli.uuid);
 	progname = basename(argv[0]);
 	setlocale(LC_ALL, "");
 	bindtextdomain(PACKAGE, LOCALEDIR);
@@ -3012,11 +3136,6 @@ main(
 	sb_feat = cli.sb_feat;
 	/* end temp don't break code */
 
-	if (sb_feat.crcs_enabled && !sb_feat.dirftype) {
-		fprintf(stderr, _("cannot disable ftype with crcs enabled\n"));
-		usage();
-	}
-
 	/*
 	 * Extract as much of the valid config as we can from the CLI input
 	 * before opening the libxfs devices.
@@ -3025,6 +3144,7 @@ main(
 	validate_sectorsize(&cfg, &cli, &dft, &ft, dfile, dry_run,
 			    force_overwrite);
 	validate_log_sectorsize(&cfg, &cli, &dft);
+	validate_sb_features(&cfg, &cli);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3034,103 +3154,9 @@ main(
 	lsectorsize = cfg.lsectorsize;
 	lsectorlog = cfg.lsectorlog;
 	sb_feat = cfg.sb_feat;
+	platform_uuid_copy(&uuid, &cfg.uuid);
 	/* end temp don't break code */
 
-	/*
-	 * Now we have blocks and sector sizes set up, check parameters that are
-	 * no longer optional for CRC enabled filesystems.  Catch them up front
-	 * here before doing anything else.
-	 */
-	if (sb_feat.crcs_enabled) {
-		/* minimum inode size is 512 bytes, ipflag checked later */
-		if ((isflag || ilflag) && inodelog < XFS_DINODE_DFL_CRC_LOG) {
-			fprintf(stderr,
-_("Minimum inode size for CRCs is %d bytes\n"),
-				1 << XFS_DINODE_DFL_CRC_LOG);
-			usage();
-		}
-
-		/* inodes always aligned */
-		if (!sb_feat.inode_align) {
-			fprintf(stderr,
-_("Inodes always aligned for CRC enabled filesytems\n"));
-			usage();
-		}
-
-		/* lazy sb counters always on */
-		if (!sb_feat.lazy_sb_counters) {
-			fprintf(stderr,
-_("Lazy superblock counted always enabled for CRC enabled filesytems\n"));
-			usage();
-		}
-
-		/* version 2 logs always on */
-		if (sb_feat.log_version != 2) {
-			fprintf(stderr,
-_("V2 logs always enabled for CRC enabled filesytems\n"));
-			usage();
-		}
-
-		/* attr2 always on */
-		if (sb_feat.attr_version != 2) {
-			fprintf(stderr,
-_("V2 attribute format always enabled on CRC enabled filesytems\n"));
-			usage();
-		}
-
-		/* 32 bit project quota always on */
-		/* attr2 always on */
-		if (sb_feat.projid16bit) {
-			fprintf(stderr,
-_("32 bit Project IDs always enabled on CRC enabled filesytems\n"));
-			usage();
-		}
-	} else {
-		/*
-		 * The kernel doesn't currently support crc=0,finobt=1
-		 * filesystems. If crcs are not enabled and the user has not
-		 * explicitly turned finobt on, then silently turn it off to
-		 * avoid an unnecessary warning.
-		 * If the user explicitly tried to use crc=0,finobt=1,
-		 * then issue an error.
-		 * The same is also for sparse inodes.
-		 */
-		if (sb_feat.finobt && mopts.subopt_params[M_FINOBT].seen) {
-			fprintf(stderr,
-_("finobt not supported without CRC support\n"));
-			usage();
-		}
-		sb_feat.finobt = 0;
-
-		if (sb_feat.spinodes) {
-			fprintf(stderr,
-_("sparse inodes not supported without CRC support\n"));
-			usage();
-		}
-		sb_feat.spinodes = 0;
-
-		if (sb_feat.rmapbt) {
-			fprintf(stderr,
-_("rmapbt not supported without CRC support\n"));
-			usage();
-		}
-		sb_feat.rmapbt = false;
-
-		if (sb_feat.reflink) {
-			fprintf(stderr,
-_("reflink not supported without CRC support\n"));
-			usage();
-		}
-		sb_feat.reflink = false;
-	}
-
-
-	if (sb_feat.rmapbt && xi.rtname) {
-		fprintf(stderr,
-_("rmapbt not supported with realtime devices\n"));
-		usage();
-		sb_feat.rmapbt = false;
-	}
 
 	if (nsflag || nlflag) {
 		if (dirblocksize < blocksize ||
-- 
2.13.3

[PATCH 25/42] mkfs: factor directory blocksize validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 78 +++++++++++++++++++++++++++++++--------------------------
 1 file changed, 42 insertions(+), 36 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 1c697dd5d4e1..72b21d715613 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2215,6 +2215,36 @@ _("rmapbt not supported with realtime devices\n"));
 }
 
 static void
+validate_dirblocksize(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli)
+{
+
+	if (cli->dirblocksize)
+		cfg->dirblocksize = getnum(cli->dirblocksize, &nopts, N_SIZE);
+	if (cli->dirblocklog)
+		cfg->dirblocksize = 1 << cli->dirblocklog;
+
+	if (cfg->dirblocksize) {
+		if (cfg->dirblocksize < cfg->blocksize ||
+		    cfg->dirblocksize > XFS_MAX_BLOCKSIZE) {
+			fprintf(stderr, _("illegal directory block size %d\n"),
+				cfg->dirblocksize);
+			usage();
+		}
+		cfg->dirblocklog = libxfs_highbit32(cfg->dirblocksize);
+		return;
+	}
+
+	/* use default size based on current block size */
+	if (cfg->blocksize < (1 << XFS_MIN_REC_DIRSIZE))
+		cfg->dirblocklog = XFS_MIN_REC_DIRSIZE;
+	else
+		cfg->dirblocklog = cfg->blocklog;
+	cfg->dirblocksize = 1 << cfg->dirblocklog;
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -2890,16 +2920,13 @@ main(
 	xfs_mount_t		*mp;
 	xfs_mount_t		mbuf;
 	xfs_extlen_t		nbmblocks;
-	int			nlflag;
 	int			nodsflag;
 	int			norsflag;
-	int			nsflag;
-	int			nvflag;
-	int			dry_run;
+	int			dry_run = 0;
 	int			discard = 1;
 	char			*protofile;
 	char			*protostring;
-	int			quiet;
+	int			quiet = 0;
 	xfs_rfsblock_t		rtblocks;
 	xfs_extlen_t		rtextblocks;
 	xfs_rtblock_t		rtextents;
@@ -2978,9 +3005,6 @@ main(
 	liflag = laflag = lsflag = lsuflag = lsunitflag = ldflag = lvflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
-	dry_run = nlflag = nsflag = nvflag = 0;
-	dirblocklog = dirblocksize = 0;
-	quiet = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
 	dsize = logsize = rtsize = rtextsize = protofile = NULL;
@@ -3001,6 +3025,7 @@ main(
 			force_overwrite = 1;
 			break;
 		case 'b':
+		case 'n':
 		case 's':
 			parse_subopts(c, optarg, &cli);
 			break;
@@ -3082,17 +3107,6 @@ main(
 			platform_uuid_copy(&uuid, &cli.uuid);
 			/* end temp don't break code */
 			break;
-		case 'n':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			if ((nsflag = cli_opt_set(&nopts, N_SIZE)))
-				dirblocksize = getnum(cli.dirblocksize, &nopts, N_SIZE);
-			dirblocklog = cli.dirblocklog;
-
-			nlflag = cli_opt_set(&nopts, N_LOG);
-			/* end temp don't break code */
-			break;
 		case 'N':
 			dry_run = 1;
 			break;
@@ -3146,6 +3160,13 @@ main(
 	validate_log_sectorsize(&cfg, &cli, &dft);
 	validate_sb_features(&cfg, &cli);
 
+	/*
+	 * we've now completed basic validation of the features, sector and
+	 * block sizes, so from this point onwards we use the values found in
+	 * the cfg structure for them, not the command line structure.
+	 */
+	validate_dirblocksize(&cfg, &cli);
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3155,25 +3176,10 @@ main(
 	lsectorlog = cfg.lsectorlog;
 	sb_feat = cfg.sb_feat;
 	platform_uuid_copy(&uuid, &cfg.uuid);
+	dirblocksize = cfg.dirblocksize;
+	dirblocklog = cfg.dirblocklog;
 	/* end temp don't break code */
 
-
-	if (nsflag || nlflag) {
-		if (dirblocksize < blocksize ||
-					dirblocksize > XFS_MAX_BLOCKSIZE) {
-			fprintf(stderr, _("illegal directory block size %d\n"),
-				dirblocksize);
-			usage();
-		}
-	} else {
-		if (blocksize < (1 << XFS_MIN_REC_DIRSIZE))
-			dirblocklog = XFS_MIN_REC_DIRSIZE;
-		else
-			dirblocklog = blocklog;
-		dirblocksize = 1 << dirblocklog;
-	}
-
-
 	if (dsize) {
 		uint64_t dbytes;
 
-- 
2.13.3

[PATCH 26/42] mkfs: factor inode size validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 101 +++++++++++++++++++++++++++++---------------------------
 1 file changed, 53 insertions(+), 48 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 72b21d715613..0062380254c9 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2245,6 +2245,54 @@ validate_dirblocksize(
 }
 
 static void
+validate_inodesize(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli)
+{
+
+	if (cli->inopblock)
+		cfg->inodelog = cfg->blocklog - libxfs_highbit32(cli->inopblock);
+	else if (cli->inodesize)
+		cfg->inodelog = libxfs_highbit32(cli->inodesize);
+	else if (cfg->sb_feat.crcs_enabled)
+		cfg->inodelog = XFS_DINODE_DFL_CRC_LOG;
+	else
+		cfg->inodelog = XFS_DINODE_DFL_LOG;
+
+	cfg->inodesize = 1 << cfg->inodelog;
+	cfg->inopblock = cfg->blocksize / cfg->inodesize;
+
+	/* input parsing has already validated non-crc inode size range */
+	if (cfg->sb_feat.crcs_enabled &&
+	    cfg->inodelog < XFS_DINODE_DFL_CRC_LOG) {
+		fprintf(stderr,
+		_("Minimum inode size for CRCs is %d bytes\n"),
+			1 << XFS_DINODE_DFL_CRC_LOG);
+		usage();
+	}
+
+	if (cfg->inodesize > cfg->blocksize / XFS_MIN_INODE_PERBLOCK ||
+	    cfg->inopblock < XFS_MIN_INODE_PERBLOCK ||
+	    cfg->inodesize < XFS_DINODE_MIN_SIZE ||
+	    cfg->inodesize > XFS_DINODE_MAX_SIZE) {
+		int	maxsz;
+
+		fprintf(stderr, _("illegal inode size %d\n"), cfg->inodesize);
+		maxsz = MIN(cfg->blocksize / XFS_MIN_INODE_PERBLOCK,
+			    XFS_DINODE_MAX_SIZE);
+		if (XFS_DINODE_MIN_SIZE == maxsz)
+			fprintf(stderr,
+			_("allowable inode size with %d byte blocks is %d\n"),
+				cfg->blocksize, XFS_DINODE_MIN_SIZE);
+		else
+			fprintf(stderr,
+	_("allowable inode size with %d byte blocks is between %d and %d\n"),
+				cfg->blocksize, XFS_DINODE_MIN_SIZE, maxsz);
+		exit(1);
+	}
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -2889,13 +2937,10 @@ main(
 	int			dsflag;
 	int			force_overwrite;
 	struct fsxattr		fsx;
-	int			ilflag;
 	int			imaxpct;
 	int			imflag;
 	int			inodelog;
 	int			inopblock;
-	int			ipflag;
-	int			isflag;
 	int			isize;
 	char			*label = NULL;
 	int			laflag;
@@ -3001,7 +3046,7 @@ main(
 	cli.loginternal = 1;	/* internal by default */
 
 	agsize = daflag = dasize = dblocks = 0;
-	ilflag = imflag = ipflag = isflag = 0;
+	imflag = 0;
 	liflag = laflag = lsflag = lsuflag = lsunitflag = ldflag = lvflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
@@ -3061,13 +3106,6 @@ main(
 			parse_subopts(c, optarg, &cli);
 
 			/* temp don't break code */
-			isize = cli.inodesize;
-			inodelog = libxfs_highbit32(isize);
-			inopblock = cli.inopblock;
-			ilflag = cli_opt_set(&iopts, I_LOG);
-			isflag = cli_opt_set(&iopts, I_SIZE);
-			ipflag = cli_opt_set(&iopts, I_PERBLOCK);
-
 			imaxpct = cli.imaxpct;
 			imflag = cli_opt_set(&iopts, I_MAXPCT);
 			/* end temp don't break code */
@@ -3166,6 +3204,7 @@ main(
 	 * the cfg structure for them, not the command line structure.
 	 */
 	validate_dirblocksize(&cfg, &cli);
+	validate_inodesize(&cfg, &cli);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3178,6 +3217,9 @@ main(
 	platform_uuid_copy(&uuid, &cfg.uuid);
 	dirblocksize = cfg.dirblocksize;
 	dirblocklog = cfg.dirblocklog;
+	isize = cfg.inodesize;
+	inodelog = cfg.inodelog;
+	inopblock = cfg.inopblock;
 	/* end temp don't break code */
 
 	if (dsize) {
@@ -3197,21 +3239,6 @@ main(
 				(long long)dbytes, blocksize,
 				(long long)(dblocks << blocklog));
 	}
-	if (ipflag) {
-		inodelog = blocklog - libxfs_highbit32(inopblock);
-		isize = 1 << inodelog;
-	} else if (!ilflag && !isflag) {
-		inodelog = sb_feat.crcs_enabled ? XFS_DINODE_DFL_CRC_LOG
-						: XFS_DINODE_DFL_LOG;
-		isize = 1 << inodelog;
-	}
-	if (sb_feat.crcs_enabled && inodelog < XFS_DINODE_DFL_CRC_LOG) {
-		fprintf(stderr,
-		_("Minimum inode size for CRCs is %d bytes\n"),
-			1 << XFS_DINODE_DFL_CRC_LOG);
-		usage();
-	}
-
 	if (logsize) {
 		uint64_t logbytes;
 
@@ -3290,28 +3317,6 @@ main(
 	}
 	ASSERT(rtextblocks);
 
-	/*
-	 * Check some argument sizes against mins, maxes.
-	 */
-	if (isize > blocksize / XFS_MIN_INODE_PERBLOCK ||
-	    isize < XFS_DINODE_MIN_SIZE ||
-	    isize > XFS_DINODE_MAX_SIZE) {
-		int	maxsz;
-
-		fprintf(stderr, _("illegal inode size %d\n"), isize);
-		maxsz = MIN(blocksize / XFS_MIN_INODE_PERBLOCK,
-			    XFS_DINODE_MAX_SIZE);
-		if (XFS_DINODE_MIN_SIZE == maxsz)
-			fprintf(stderr,
-			_("allowable inode size with %d byte blocks is %d\n"),
-				blocksize, XFS_DINODE_MIN_SIZE);
-		else
-			fprintf(stderr,
-	_("allowable inode size with %d byte blocks is between %d and %d\n"),
-				blocksize, XFS_DINODE_MIN_SIZE, maxsz);
-		exit(1);
-	}
-
 	calc_stripe_factors(dsu, dsw, sectorsize, lsu, lsectorsize,
 				&dsunit, &dswidth, &lsunit);
 
-- 
2.13.3

[PATCH 27/42] mkfs: factor out device size calculations

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 94 ++++++++++++++++++++++++++-------------------------------
 1 file changed, 43 insertions(+), 51 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 0062380254c9..e7e366cc518c 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2292,6 +2292,37 @@ validate_inodesize(
 	}
 }
 
+static xfs_rfsblock_t
+calc_dev_size(
+	char			*size,
+	struct mkfs_params	*cfg,
+	struct opt_params	*opts,
+	int			sizeopt,
+	char			*type)
+{
+	uint64_t		dbytes;
+	xfs_rfsblock_t		dblocks;
+
+	if (!size)
+		return 0;
+
+	dbytes = getnum(size, opts, sizeopt);
+	if (dbytes % XFS_MIN_BLOCKSIZE) {
+		fprintf(stderr,
+		_("illegal %s length %lld, not a multiple of %d\n"),
+			type, (long long)dbytes, XFS_MIN_BLOCKSIZE);
+		usage();
+	}
+	dblocks = (xfs_rfsblock_t)(dbytes >> cfg->blocklog);
+	if (dbytes % cfg->blocksize) {
+		fprintf(stderr,
+_("warning: %s length %lld not a multiple of %d, truncated to %lld\n"),
+			type, (long long)dbytes, cfg->blocksize,
+			(long long)(dblocks << cfg->blocklog));
+	}
+	return dblocks;
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -3206,6 +3237,15 @@ main(
 	validate_dirblocksize(&cfg, &cli);
 	validate_inodesize(&cfg, &cli);
 
+	/*
+	 * if the device size was specified convert it to a block count
+	 * now we have a valid block size. These will be set to zero if
+	 * nothing was specified, indicating we should use the full device.
+	 */
+	cfg.dblocks = calc_dev_size(cli.dsize, &cfg, &dopts, D_SIZE, "data");
+	cfg.logblocks = calc_dev_size(cli.logsize, &cfg, &lopts, L_SIZE, "log");
+	cfg.rtblocks = calc_dev_size(cli.rtsize, &cfg, &ropts, R_SIZE, "rt");
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3220,59 +3260,11 @@ main(
 	isize = cfg.inodesize;
 	inodelog = cfg.inodelog;
 	inopblock = cfg.inopblock;
+	dblocks = cfg.dblocks;
+	logblocks = cfg.logblocks;
+	rtblocks = cfg.rtblocks;
 	/* end temp don't break code */
 
-	if (dsize) {
-		uint64_t dbytes;
-
-		dbytes = getnum(dsize, &dopts, D_SIZE);
-		if (dbytes % XFS_MIN_BLOCKSIZE) {
-			fprintf(stderr,
-			_("illegal data length %lld, not a multiple of %d\n"),
-				(long long)dbytes, XFS_MIN_BLOCKSIZE);
-			usage();
-		}
-		dblocks = (xfs_rfsblock_t)(dbytes >> blocklog);
-		if (dbytes % blocksize)
-			fprintf(stderr, _("warning: "
-	"data length %lld not a multiple of %d, truncated to %lld\n"),
-				(long long)dbytes, blocksize,
-				(long long)(dblocks << blocklog));
-	}
-	if (logsize) {
-		uint64_t logbytes;
-
-		logbytes = getnum(logsize, &lopts, L_SIZE);
-		if (logbytes % XFS_MIN_BLOCKSIZE) {
-			fprintf(stderr,
-			_("illegal log length %lld, not a multiple of %d\n"),
-				(long long)logbytes, XFS_MIN_BLOCKSIZE);
-			usage();
-		}
-		logblocks = (xfs_rfsblock_t)(logbytes >> blocklog);
-		if (logbytes % blocksize)
-			fprintf(stderr,
-	_("warning: log length %lld not a multiple of %d, truncated to %lld\n"),
-				(long long)logbytes, blocksize,
-				(long long)(logblocks << blocklog));
-	}
-	if (rtsize) {
-		uint64_t rtbytes;
-
-		rtbytes = getnum(rtsize, &ropts, R_SIZE);
-		if (rtbytes % XFS_MIN_BLOCKSIZE) {
-			fprintf(stderr,
-			_("illegal rt length %lld, not a multiple of %d\n"),
-				(long long)rtbytes, XFS_MIN_BLOCKSIZE);
-			usage();
-		}
-		rtblocks = (xfs_rfsblock_t)(rtbytes >> blocklog);
-		if (rtbytes % blocksize)
-			fprintf(stderr,
-	_("warning: rt length %lld not a multiple of %d, truncated to %lld\n"),
-				(long long)rtbytes, blocksize,
-				(long long)(rtblocks << blocklog));
-	}
 	/*
 	 * If specified, check rt extent size against its constraints.
 	 */
-- 
2.13.3

[PATCH 28/42] mkfs: fix hidden parameter in DTOBT()

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Hiding references to variables inside macros instead of passing them
as parameters is just plain nasty. Fix it before going any further.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index e7e366cc518c..1f41d5f813f0 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -879,7 +879,7 @@ struct mkfs_default_params {
 /*
  * Use this macro before we have superblock and mount structure
  */
-#define	DTOBT(d)	((xfs_rfsblock_t)((d) >> (blocklog - BBSHIFT)))
+#define	DTOBT(d, bl)	((xfs_rfsblock_t)((d) >> ((bl) - BBSHIFT)))
 
 /*
  * Use this for block reservations needed for mkfs's conditions
@@ -3295,7 +3295,7 @@ main(
 
 		/* check that rswidth is a multiple of fs blocksize */
 		if (!norsflag && rswidth && !(BBTOB(rswidth) % blocksize)) {
-			rswidth = DTOBT(rswidth);
+			rswidth = DTOBT(rswidth, blocklog);
 			rtextbytes = rswidth << blocklog;
 			if (XFS_MIN_RTEXTSIZE <= rtextbytes &&
 			    (rtextbytes <= XFS_MAX_RTEXTSIZE)) {
@@ -3374,14 +3374,14 @@ main(
 		rtfile = _("volume rt");
 	else if (!xi.rtdev)
 		rtfile = _("none");
-	if (dsize && xi.dsize > 0 && dblocks > DTOBT(xi.dsize)) {
+	if (dsize && xi.dsize > 0 && dblocks > DTOBT(xi.dsize, blocklog)) {
 		fprintf(stderr,
 			_("size %s specified for data subvolume is too large, "
 			"maximum is %lld blocks\n"),
-			dsize, (long long)DTOBT(xi.dsize));
+			dsize, (long long)DTOBT(xi.dsize, blocklog));
 		usage();
 	} else if (!dsize && xi.dsize > 0)
-		dblocks = DTOBT(xi.dsize);
+		dblocks = DTOBT(xi.dsize, blocklog);
 	else if (!dsize) {
 		fprintf(stderr, _("can't get size of data subvolume\n"));
 		usage();
@@ -3422,14 +3422,14 @@ reported by the device (%u).\n"),
 			sectorsize, xi.rtbsize);
 	}
 
-	if (rtsize && xi.rtsize > 0 && rtblocks > DTOBT(xi.rtsize)) {
+	if (rtsize && xi.rtsize > 0 && rtblocks > DTOBT(xi.rtsize, blocklog)) {
 		fprintf(stderr,
 			_("size %s specified for rt subvolume is too large, "
 			"maximum is %lld blocks\n"),
-			rtsize, (long long)DTOBT(xi.rtsize));
+			rtsize, (long long)DTOBT(xi.rtsize, blocklog));
 		usage();
 	} else if (!rtsize && xi.rtsize > 0)
-		rtblocks = DTOBT(xi.rtsize);
+		rtblocks = DTOBT(xi.rtsize, blocklog);
 	else if (rtsize && !xi.rtdev) {
 		fprintf(stderr,
 			_("size specified for non-existent rt subvolume\n"));
@@ -3494,8 +3494,8 @@ reported by the device (%u).\n"),
 	    dswidth && !(BBTOB(dswidth) % blocksize)) {
 
 		/* convert from 512 byte blocks to fs blocksize */
-		dsunit = DTOBT(dsunit);
-		dswidth = DTOBT(dswidth);
+		dsunit = DTOBT(dsunit, blocklog);
+		dswidth = DTOBT(dswidth, blocklog);
 
 		/*
 		 * agsize is not a multiple of dsunit
@@ -3610,7 +3610,7 @@ an AG size that is one stripe unit smaller, for example %llu.\n"),
 
 	if (lsunit) {
 		/* convert from 512 byte blocks to fs blocks */
-		lsunit = DTOBT(lsunit);
+		lsunit = DTOBT(lsunit, blocklog);
 	} else if (sb_feat.log_version == 2 && loginternal && dsunit) {
 		/* lsunit and dsunit now in fs blocks */
 		lsunit = dsunit;
@@ -3638,13 +3638,13 @@ an AG size that is one stripe unit smaller, for example %llu.\n"),
 	min_logblocks = MAX(XFS_MIN_LOG_BLOCKS, min_logblocks);
 	if (!logsize && dblocks >= (1024*1024*1024) >> blocklog)
 		min_logblocks = MAX(min_logblocks, XFS_MIN_LOG_BYTES>>blocklog);
-	if (logsize && xi.logBBsize > 0 && logblocks > DTOBT(xi.logBBsize)) {
+	if (logsize && xi.logBBsize > 0 && logblocks > DTOBT(xi.logBBsize, blocklog)) {
 		fprintf(stderr,
 _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
-			logsize, (long long)DTOBT(xi.logBBsize));
+			logsize, (long long)DTOBT(xi.logBBsize, blocklog));
 		usage();
 	} else if (!logsize && xi.logBBsize > 0) {
-		logblocks = DTOBT(xi.logBBsize);
+		logblocks = DTOBT(xi.logBBsize, blocklog);
 	} else if (logsize && !xi.logdev && !loginternal) {
 		fprintf(stderr,
 			_("size specified for non-existent log subvolume\n"));
-- 
2.13.3

[PATCH 29/42] mkfs: factor rtdev extent size validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 108 +++++++++++++++++++++++++++++++-------------------------
 1 file changed, 59 insertions(+), 49 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 1f41d5f813f0..7595e378ad44 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2324,6 +2324,60 @@ _("warning: %s length %lld not a multiple of %d, truncated to %lld\n"),
 }
 
 static void
+validate_rtextsize(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	struct fs_topology	*ft)
+{
+	uint64_t		rtextbytes;
+
+	/*
+	 * If specified, check rt extent size against its constraints.
+	 */
+	if (cli->rtextsize) {
+
+		rtextbytes = getnum(cli->rtextsize, &ropts, R_EXTSIZE);
+		if (rtextbytes % cfg->blocksize) {
+			fprintf(stderr,
+		_("illegal rt extent size %lld, not a multiple of %d\n"),
+				(long long)rtextbytes, cfg->blocksize);
+			usage();
+		}
+		cfg->rtextblocks = (xfs_extlen_t)(rtextbytes >> cfg->blocklog);
+	} else {
+		/*
+		 * If realtime extsize has not been specified by the user,
+		 * and the underlying volume is striped, then set rtextblocks
+		 * to the stripe width.
+		 */
+		uint64_t	rswidth;
+
+		if (!cfg->sb_feat.nortalign && !cli->xi->risfile &&
+		    !(!cli->rtsize && cli->xi->disfile))
+			rswidth = ft->rtswidth;
+		else
+			rswidth = 0;
+
+		/* check that rswidth is a multiple of fs blocksize */
+		if (!cfg->sb_feat.nortalign && rswidth &&
+		    !(BBTOB(rswidth) % cfg->blocksize)) {
+			rswidth = DTOBT(rswidth, cfg->blocklog);
+			rtextbytes = rswidth << cfg->blocklog;
+			if (rtextbytes > XFS_MIN_RTEXTSIZE &&
+			    rtextbytes <= XFS_MAX_RTEXTSIZE) {
+				cfg->rtextblocks = rswidth;
+			}
+		}
+		if (!cfg->rtextblocks) {
+			cfg->rtextblocks = (cfg->blocksize < XFS_MIN_RTEXTSIZE)
+					? XFS_MIN_RTEXTSIZE >> cfg->blocklog
+					: 1;
+		}
+	}
+	ASSERT(cfg->rtextblocks);
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -2997,7 +3051,6 @@ main(
 	xfs_mount_t		mbuf;
 	xfs_extlen_t		nbmblocks;
 	int			nodsflag;
-	int			norsflag;
 	int			dry_run = 0;
 	int			discard = 1;
 	char			*protofile;
@@ -3006,7 +3059,6 @@ main(
 	xfs_rfsblock_t		rtblocks;
 	xfs_extlen_t		rtextblocks;
 	xfs_rtblock_t		rtextents;
-	char			*rtextsize;
 	char			*rtfile;
 	char			*rtsize;
 	xfs_sb_t		*sbp;
@@ -3083,9 +3135,9 @@ main(
 	logagno = logblocks = rtblocks = rtextblocks = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
-	dsize = logsize = rtsize = rtextsize = protofile = NULL;
+	dsize = logsize = rtsize = protofile = NULL;
 	dsu = dsw = dsunit = dswidth = lalign = lsu = lsunit = 0;
-	dsflag = nodsflag = norsflag = 0;
+	dsflag = nodsflag = 0;
 	force_overwrite = 0;
 	worst_freelist = 0;
 	memset(&fsx, 0, sizeof(fsx));
@@ -3194,9 +3246,7 @@ main(
 			parse_subopts(c, optarg, &cli);
 
 			/* temp don't break code */
-			rtextsize = cli.rtextsize;
 			rtsize = cli.rtsize;
-			norsflag = cli.sb_feat.nortalign;
 			/* end temp don't break code */
 			break;
 			break;
@@ -3246,6 +3296,8 @@ main(
 	cfg.logblocks = calc_dev_size(cli.logsize, &cfg, &lopts, L_SIZE, "log");
 	cfg.rtblocks = calc_dev_size(cli.rtsize, &cfg, &ropts, R_SIZE, "rt");
 
+	validate_rtextsize(&cfg, &cli, &ft);
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3263,51 +3315,9 @@ main(
 	dblocks = cfg.dblocks;
 	logblocks = cfg.logblocks;
 	rtblocks = cfg.rtblocks;
+	rtextblocks = cfg.rtextblocks;
 	/* end temp don't break code */
 
-	/*
-	 * If specified, check rt extent size against its constraints.
-	 */
-	if (rtextsize) {
-		uint64_t rtextbytes;
-
-		rtextbytes = getnum(rtextsize, &ropts, R_EXTSIZE);
-		if (rtextbytes % blocksize) {
-			fprintf(stderr,
-		_("illegal rt extent size %lld, not a multiple of %d\n"),
-				(long long)rtextbytes, blocksize);
-			usage();
-		}
-		rtextblocks = (xfs_extlen_t)(rtextbytes >> blocklog);
-	} else {
-		/*
-		 * If realtime extsize has not been specified by the user,
-		 * and the underlying volume is striped, then set rtextblocks
-		 * to the stripe width.
-		 */
-		uint64_t	rswidth;
-		uint64_t	rtextbytes;
-
-		if (!norsflag && !xi.risfile && !(!rtsize && xi.disfile))
-			rswidth = ft.rtswidth;
-		else
-			rswidth = 0;
-
-		/* check that rswidth is a multiple of fs blocksize */
-		if (!norsflag && rswidth && !(BBTOB(rswidth) % blocksize)) {
-			rswidth = DTOBT(rswidth, blocklog);
-			rtextbytes = rswidth << blocklog;
-			if (XFS_MIN_RTEXTSIZE <= rtextbytes &&
-			    (rtextbytes <= XFS_MAX_RTEXTSIZE)) {
-				rtextblocks = rswidth;
-			}
-		}
-		if (!rtextblocks) {
-			rtextblocks = (blocksize < XFS_MIN_RTEXTSIZE) ?
-					XFS_MIN_RTEXTSIZE >> blocklog : 1;
-		}
-	}
-	ASSERT(rtextblocks);
 
 	calc_stripe_factors(dsu, dsw, sectorsize, lsu, lsectorsize,
 				&dsunit, &dswidth, &lsunit);
-- 
2.13.3

[PATCH 30/42] mkfs: rework stripe calculations

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

The data and log stripe calculations a spaghettied all over the mkfs
code. This patch pulls all of the different chunks of code together
into calc_stripe_factors() and removes all the redundant/repeated
checks and calculations that are made.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 330 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 180 insertions(+), 150 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 7595e378ad44..fe380d2b388d 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -896,74 +896,6 @@ struct mkfs_default_params {
  */
 #define WHACK_SIZE (128 * 1024)
 
-/*
- * Convert lsu to lsunit for 512 bytes blocks and check validity of the values.
- */
-static void
-calc_stripe_factors(
-	int		dsu,
-	int		dsw,
-	int		dsectsz,
-	int		lsu,
-	int		lsectsz,
-	int		*dsunit,
-	int		*dswidth,
-	int		*lsunit)
-{
-	/* Handle data sunit/swidth options */
-	if ((*dsunit && !*dswidth) || (!*dsunit && *dswidth)) {
-		fprintf(stderr,
-			_("both data sunit and data swidth options "
-			"must be specified\n"));
-		usage();
-	}
-
-	if (dsu || dsw) {
-		if ((dsu && !dsw) || (!dsu && dsw)) {
-			fprintf(stderr,
-				_("both data su and data sw options "
-				"must be specified\n"));
-			usage();
-		}
-
-		if (dsu % dsectsz) {
-			fprintf(stderr,
-				_("data su must be a multiple of the "
-				"sector size (%d)\n"), dsectsz);
-			usage();
-		}
-
-		*dsunit  = (int)BTOBBT(dsu);
-		*dswidth = *dsunit * dsw;
-	}
-
-	if (*dsunit && (*dswidth % *dsunit != 0)) {
-		fprintf(stderr,
-			_("data stripe width (%d) must be a multiple of the "
-			"data stripe unit (%d)\n"), *dswidth, *dsunit);
-		usage();
-	}
-
-	/* Handle log sunit options */
-
-	if (lsu)
-		*lsunit = (int)BTOBBT(lsu);
-
-	/* verify if lsu/lsunit is a multiple block size */
-	if (lsu % blocksize != 0) {
-		fprintf(stderr,
-_("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
-		lsu, blocksize);
-		exit(1);
-	}
-	if ((BBTOB(*lsunit) % blocksize != 0)) {
-		fprintf(stderr,
-_("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
-		BBTOB(*lsunit), blocksize);
-		exit(1);
-	}
-}
-
 static void
 check_device_type(
 	const char	*name,
@@ -2377,6 +2309,178 @@ validate_rtextsize(
 	ASSERT(cfg->rtextblocks);
 }
 
+/*
+ * Validate the configured stripe geometry, or is none is specified, pull
+ * the configuration from the underlying device.
+ *
+ * CLI parameters come in as different units, go out as filesystem blocks.
+ */
+static void
+calc_stripe_factors(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	struct fs_topology	*ft)
+{
+	int		dsunit = 0;
+	int		dswidth = 0;
+	int		lsunit = 0;
+	int		dsu = 0;
+	int		dsw = 0;
+	int		lsu = 0;
+	bool		use_dev = false;
+
+	if (cli_opt_set(&dopts, D_SUNIT))
+		dsunit = cli->dsunit;
+	if (cli_opt_set(&dopts, D_SWIDTH))
+		dswidth = cli->dswidth;
+
+	if (cli_opt_set(&dopts, D_SU))
+		dsu = getnum(cli->dsu, &dopts, D_SU);
+	if (cli_opt_set(&dopts, D_SW))
+		dsw = cli->dsw;
+
+	/* data sunit/swidth options */
+	if ((dsunit && !dswidth) || (!dsunit && dswidth)) {
+		fprintf(stderr,
+_("both data sunit and data swidth options must be specified\n"));
+		usage();
+	}
+
+	/* convert dsu/dsw to dsunit/dswidth and use them from now on */
+	if (dsu || dsw) {
+		if ((dsu && !dsw) || (!dsu && dsw)) {
+			fprintf(stderr,
+_("both data su and data sw options must be specified\n"));
+			usage();
+		}
+
+		if (dsu % cfg->sectorsize) {
+			fprintf(stderr,
+_("data su must be a multiple of the sector size (%d)\n"), cfg->sectorsize);
+			usage();
+		}
+
+		dsunit  = (int)BTOBBT(dsu);
+		dswidth = dsunit * dsw;
+	}
+
+	if (dsunit && (dswidth % dsunit != 0)) {
+		fprintf(stderr,
+_("data stripe width (%d) must be a multiple of the data stripe unit (%d)\n"),
+			dswidth, dsunit);
+		usage();
+	}
+
+	/* If sunit & swidth were manually specified as 0, same as noalign */
+	if ((cli_opt_set(&dopts, D_SUNIT) || cli_opt_set(&dopts, D_SU)) &&
+	    !dsunit && !dswidth)
+		cfg->sb_feat.nodalign = true;
+
+	/* if we are not using alignment, don't apply device defaults */
+	if (cfg->sb_feat.nodalign) {
+		cfg->dsunit = 0;
+		cfg->dswidth = 0;
+		goto check_lsunit;
+	}
+
+	/* if no stripe config set, use the device default */
+	if (!dsunit) {
+		dsunit = ft->dsunit;
+		dswidth = ft->dswidth;
+		use_dev = true;
+	} else {
+		/* check and warn is alignment is sub-optimal */
+		if (ft->dsunit && ft->dsunit != dsunit) {
+			fprintf(stderr,
+_("%s: Specified data stripe unit %d is not the same as the volume stripe unit %d\n"),
+				progname, dsunit, ft->dsunit);
+		}
+		if (ft->dswidth && ft->dswidth != dswidth) {
+			fprintf(stderr,
+_("%s: Specified data stripe width %d is not the same as the volume stripe width %d\n"),
+				progname, dswidth, ft->dswidth);
+		}
+	}
+
+	/*
+	 * now we have our stripe config, check it's a multiple of block
+	 * size.
+	 */
+	if ((BBTOB(dsunit) % cfg->blocksize) ||
+	    (BBTOB(dswidth) % cfg->blocksize)) {
+		/*
+		 * If we are using device defaults, just clear them and we're
+		 * good to go. Otherwise bail out with an error.
+		 */
+		if (!use_dev) {
+			fprintf(stderr,
+_("%s: Stripe unit(%d) or stripe width(%d) is not a multiple of the block size(%d)\n"),
+				progname, BBTOB(dsunit), BBTOB(dswidth),
+				cfg->blocksize);
+			exit(1);
+		}
+		dsunit = 0;
+		dswidth = 0;
+		cfg->sb_feat.nodalign = true;
+	}
+
+	/* convert from 512 byte blocks to fs blocksize */
+	cfg->dsunit = DTOBT(dsunit, cfg->blocklog);
+	cfg->dswidth = DTOBT(dswidth, cfg->blocklog);
+
+check_lsunit:
+	/* log sunit options */
+	if (cli_opt_set(&lopts, L_SUNIT))
+		lsunit = cli->lsunit;
+	else if (cli_opt_set(&lopts, L_SU))
+		lsu = getnum(cli->lsu, &lopts, L_SU);
+	else /* set default log stripe unit if not set on CLI */
+		lsu = cfg->blocksize;
+
+	if (lsu) {
+		/* verify if lsu is a multiple block size */
+		if (lsu % cfg->blocksize != 0) {
+			fprintf(stderr,
+	_("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
+				lsu, cfg->blocksize);
+			usage();
+		}
+		lsunit = (int)BTOBBT(lsu);
+	}
+	if (BBTOB(lsunit) % cfg->blocksize != 0) {
+		fprintf(stderr,
+_("log stripe unit (%d) must be a multiple of the block size (%d)\n"),
+			BBTOB(lsunit), cfg->blocksize);
+		usage();
+	}
+
+	/*
+	 * check that log sunit is modulo fsblksize or default it to dsunit.
+	 */
+	if (lsunit) {
+		/* convert from 512 byte blocks to fs blocks */
+		cfg->lsunit = DTOBT(lsunit, cfg->blocklog);
+	} else if (cfg->sb_feat.log_version == 2 &&
+		   cfg->loginternal && cfg->dsunit) {
+		/* lsunit and dsunit now in fs blocks */
+		cfg->lsunit = cfg->dsunit;
+	}
+
+	if (cfg->sb_feat.log_version == 2 &&
+	    cfg->lsunit * cfg->blocksize > 256 * 1024) {
+		/* Warn only if specified on commandline */
+		if (cli->lsu || cli->lsunit != -1) {
+			fprintf(stderr,
+_("log stripe unit (%d bytes) is too large (maximum is 256KiB)\n"
+  "log stripe unit adjusted to 32KiB\n"),
+				(cfg->lsunit * cfg->blocksize));
+		}
+		/* XXX: 64k block size? */
+		cfg->lsunit = (32 * 1024) / cfg->blocksize;
+	}
+
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -3015,11 +3119,8 @@ main(
 	int			dirblocklog;
 	int			dirblocksize;
 	char			*dsize;
-	int			dsu;
-	int			dsw;
 	int			dsunit;
 	int			dswidth;
-	int			dsflag;
 	int			force_overwrite;
 	struct fsxattr		fsx;
 	int			imaxpct;
@@ -3040,11 +3141,8 @@ main(
 	xfs_fsblock_t		logstart;
 	int			lvflag;
 	int			lsflag;
-	int			lsuflag;
-	int			lsunitflag;
 	int			lsectorlog;
 	int			lsectorsize;
-	int			lsu;
 	int			lsunit;
 	int			min_logblocks;
 	xfs_mount_t		*mp;
@@ -3130,14 +3228,14 @@ main(
 
 	agsize = daflag = dasize = dblocks = 0;
 	imflag = 0;
-	liflag = laflag = lsflag = lsuflag = lsunitflag = ldflag = lvflag = 0;
+	liflag = laflag = lsflag = ldflag = lvflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
 	dsize = logsize = rtsize = protofile = NULL;
-	dsu = dsw = dsunit = dswidth = lalign = lsu = lsunit = 0;
-	dsflag = nodsflag = 0;
+	dsunit = dswidth = lalign = lsunit = 0;
+	nodsflag = 0;
 	force_overwrite = 0;
 	worst_freelist = 0;
 	memset(&fsx, 0, sizeof(fsx));
@@ -3168,18 +3266,6 @@ main(
 			}
 			daflag = cli_opt_set(&dopts, D_AGCOUNT);
 
-			dsunit = cli.dsunit;
-			dswidth = cli.dswidth;
-			dsw = cli.dsw;
-			if (cli_opt_set(&dopts, D_SU)) {
-				dsu = getnum(cli.dsu, &dopts, D_SU);
-				dsflag = 1;
-			}
-			dsflag |= cli_opt_set(&dopts, D_SW) ||
-				  cli_opt_set(&dopts, D_SUNIT) ||
-				  cli_opt_set(&dopts, D_SWIDTH);
-			nodsflag = cli_opt_set(&dopts, D_NOALIGN);
-
 			fsx.fsx_xflags |= cli.fsx.fsx_xflags;
 			fsx.fsx_projid = cli.fsx.fsx_projid;
 			fsx.fsx_extsize = cli.fsx.fsx_extsize;
@@ -3202,13 +3288,6 @@ main(
 			logfile = xi.logname;
 			logsize = cli.logsize;
 
-			lsunit = cli.lsunit;
-			lsunitflag = cli_opt_set(&lopts, L_SUNIT);
-			if (cli_opt_set(&lopts, L_SU)) {
-				lsu = getnum(cli.lsu, &lopts, L_SU);
-				lsuflag = 1;
-			}
-
 			laflag = cli_opt_set(&lopts, L_AGNUM);
 			liflag = cli_opt_set(&lopts, L_INTERNAL);
 			ldflag = cli_opt_set(&lopts, L_NAME) ||
@@ -3297,6 +3376,7 @@ main(
 	cfg.rtblocks = calc_dev_size(cli.rtsize, &cfg, &ropts, R_SIZE, "rt");
 
 	validate_rtextsize(&cfg, &cli, &ft);
+	calc_stripe_factors(&cfg, &cli, &ft);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3316,16 +3396,13 @@ main(
 	logblocks = cfg.logblocks;
 	rtblocks = cfg.rtblocks;
 	rtextblocks = cfg.rtextblocks;
+	dsunit = cfg.dsunit;
+	dswidth = cfg.dswidth;
+	lsunit = cfg.lsunit;
+	nodsflag = cfg.sb_feat.nodalign;
 	/* end temp don't break code */
 
 
-	calc_stripe_factors(dsu, dsw, sectorsize, lsu, lsectorsize,
-				&dsunit, &dswidth, &lsunit);
-
-	/* If sunit & swidth were manually specified as 0, same as noalign */
-	if (dsflag && !dsunit && !dswidth)
-		nodsflag = 1;
-
 	xi.setblksize = sectorsize;
 
 	/*
@@ -3453,29 +3530,6 @@ reported by the device (%u).\n"),
 		nbmblocks = 0;
 	}
 
-	if (!nodsflag) {
-		if (dsunit) {
-			if (ft.dsunit && ft.dsunit != dsunit) {
-				fprintf(stderr,
-					_("%s: Specified data stripe unit %d "
-					"is not the same as the volume stripe "
-					"unit %d\n"),
-					progname, dsunit, ft.dsunit);
-			}
-			if (ft.dswidth && ft.dswidth != dswidth) {
-				fprintf(stderr,
-					_("%s: Specified data stripe width %d "
-					"is not the same as the volume stripe "
-					"width %d\n"),
-					progname, dswidth, ft.dswidth);
-			}
-		} else {
-			dsunit = ft.dsunit;
-			dswidth = ft.dswidth;
-			nodsflag = 1;
-		}
-	} /* else dsunit & dswidth can't be set if nodsflag is set */
-
 	if (dasize) {		/* User-specified AG size */
 		/*
 		 * Check specified agsize is a multiple of blocksize.
@@ -3614,30 +3668,6 @@ an AG size that is one stripe unit smaller, for example %llu.\n"),
 	if (!imflag)
 		imaxpct = calc_default_imaxpct(blocklog, dblocks);
 
-	/*
-	 * check that log sunit is modulo fsblksize or default it to dsunit.
-	 */
-
-	if (lsunit) {
-		/* convert from 512 byte blocks to fs blocks */
-		lsunit = DTOBT(lsunit, blocklog);
-	} else if (sb_feat.log_version == 2 && loginternal && dsunit) {
-		/* lsunit and dsunit now in fs blocks */
-		lsunit = dsunit;
-	}
-
-	if (sb_feat.log_version == 2 && (lsunit * blocksize) > 256 * 1024) {
-		/* Warn only if specified on commandline */
-		if (lsuflag || lsunitflag) {
-			fprintf(stderr,
-	_("log stripe unit (%d bytes) is too large (maximum is 256KiB)\n"),
-				(lsunit * blocksize));
-			fprintf(stderr,
-	_("log stripe unit adjusted to 32KiB\n"));
-		}
-		lsunit = (32 * 1024) >> blocklog;
-	}
-
 	min_logblocks = max_trans_res(agsize,
 				   sb_feat.crcs_enabled, sb_feat.dir_version,
 				   sectorlog, blocklog, inodelog, dirblocklog,
-- 
2.13.3

[PATCH 31/42] mkfs: factor device opening

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 92 ++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 51 insertions(+), 41 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index fe380d2b388d..e8fa53fea8e0 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2482,6 +2482,52 @@ _("log stripe unit (%d bytes) is too large (maximum is 256KiB)\n"
 }
 
 static void
+open_devices(
+	struct mkfs_params	*cfg,
+	struct libxfs_xinit	*xi,
+	bool			discard)
+{
+	uint64_t		sector_mask;
+
+	/*
+	 * Initialize.  This will open the log and rt devices as well.
+	 */
+	xi->setblksize = cfg->sectorsize;
+	if (!libxfs_init(xi))
+		usage();
+	if (!xi->ddev) {
+		fprintf(stderr, _("no device name given in argument list\n"));
+		usage();
+	}
+
+	/*
+	 * Ok, Linux only has a 1024-byte resolution on device _size_,
+	 * and the sizes below are in basic 512-byte blocks,
+	 * so if we have (size % 2), on any partition, we can't get
+	 * to the last 512 bytes.  The same issue exists for larger
+	 * sector sizes - we cannot write past the last sector.
+	 *
+	 * So, we reduce the size (in basic blocks) to a perfect
+	 * multiple of the sector size, or 1024, whichever is larger.
+	 */
+	sector_mask = (uint64_t)-1 << (MAX(cfg->sectorlog, 10) - BBSHIFT);
+	xi->dsize &= sector_mask;
+	xi->rtsize &= sector_mask;
+	xi->logBBsize &= (uint64_t)-1 << (MAX(cfg->lsectorlog, 10) - BBSHIFT);
+
+
+	if (!discard)
+		return;
+
+	if (!xi->disfile)
+		discard_blocks(xi->ddev, xi->dsize);
+	if (xi->rtdev && !xi->risfile)
+		discard_blocks(xi->rtdev, xi->rtsize);
+	if (xi->logdev && xi->logdev != xi->ddev && !xi->lisfile)
+		discard_blocks(xi->logdev, xi->logBBsize);
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -3161,7 +3207,6 @@ main(
 	char			*rtsize;
 	xfs_sb_t		*sbp;
 	int			sectorlog;
-	uint64_t		sector_mask;
 	uint64_t		tmp_agsize;
 	uuid_t			uuid;
 	int			worst_freelist;
@@ -3378,6 +3423,11 @@ main(
 	validate_rtextsize(&cfg, &cli, &ft);
 	calc_stripe_factors(&cfg, &cli, &ft);
 
+	/*
+	 * Open and validate the device configurations
+	 */
+	open_devices(&cfg, &xi, (discard && !dry_run));
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3402,46 +3452,6 @@ main(
 	nodsflag = cfg.sb_feat.nodalign;
 	/* end temp don't break code */
 
-
-	xi.setblksize = sectorsize;
-
-	/*
-	 * Initialize.  This will open the log and rt devices as well.
-	 */
-	if (!libxfs_init(&xi))
-		usage();
-	if (!xi.ddev) {
-		fprintf(stderr, _("no device name given in argument list\n"));
-		usage();
-	}
-
-	/*
-	 * Ok, Linux only has a 1024-byte resolution on device _size_,
-	 * and the sizes below are in basic 512-byte blocks,
-	 * so if we have (size % 2), on any partition, we can't get
-	 * to the last 512 bytes.  The same issue exists for larger
-	 * sector sizes - we cannot write past the last sector.
-	 *
-	 * So, we reduce the size (in basic blocks) to a perfect
-	 * multiple of the sector size, or 1024, whichever is larger.
-	 */
-
-	sector_mask = (uint64_t)-1 << (MAX(sectorlog, 10) - BBSHIFT);
-	xi.dsize &= sector_mask;
-	xi.rtsize &= sector_mask;
-	xi.logBBsize &= (uint64_t)-1 << (MAX(lsectorlog, 10) - BBSHIFT);
-
-
-	/* don't do discards on print-only runs or on files */
-	if (discard && !dry_run) {
-		if (!xi.disfile)
-			discard_blocks(xi.ddev, xi.dsize);
-		if (xi.rtdev && !xi.risfile)
-			discard_blocks(xi.rtdev, xi.rtsize);
-		if (xi.logdev && xi.logdev != xi.ddev && !xi.lisfile)
-			discard_blocks(xi.logdev, xi.logBBsize);
-	}
-
 	if (!liflag && !ldflag)
 		loginternal = xi.logdev == 0;
 	if (xi.logname)
-- 
2.13.3

[PATCH 32/42] mkfs: factor data device validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 75 +++++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 49 insertions(+), 26 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index e8fa53fea8e0..81f1e12a941e 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2528,6 +2528,53 @@ open_devices(
 }
 
 static void
+validate_datadev(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli)
+{
+	struct libxfs_xinit	*xi = cli->xi;
+
+	if (!xi->dsize) {
+		/*
+		 * if the device is a file, we can't validate the size here.
+		 * Instead, the file will be truncated to the correct length
+		 * later on. if it's not a file, we've got a dud device.
+		 */
+		if (!xi->disfile) {
+			fprintf(stderr, _("can't get size of data subvolume\n"));
+			usage();
+		}
+		ASSERT(cfg->dblocks);
+	} else if (cfg->dblocks) {
+		/* check the size fits into the underlying device */
+		if (cfg->dblocks > DTOBT(xi->dsize, cfg->blocklog)) {
+			fprintf(stderr,
+_("size %s specified for data subvolume is too large, maximum is %lld blocks\n"),
+				cli->dsize,
+				(long long)DTOBT(xi->dsize, cfg->blocklog));
+			usage();
+		}
+	} else {
+		/* no user size, so use the full block device */
+		cfg->dblocks = DTOBT(xi->dsize, cfg->blocklog);
+	}
+
+	if (cfg->dblocks < XFS_MIN_DATA_BLOCKS) {
+		fprintf(stderr,
+_("size %lld of data subvolume is too small, minimum %d blocks\n"),
+			(long long)cfg->dblocks, XFS_MIN_DATA_BLOCKS);
+		usage();
+	}
+
+	if (xi->dbsize > cfg->sectorsize) {
+		fprintf(stderr, _(
+"Warning: the data subvolume sector size %u is less than the sector size \n\
+reported by the device (%u).\n"),
+			cfg->sectorsize, xi->dbsize);
+	}
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -3164,7 +3211,6 @@ main(
 	char			*dfile;
 	int			dirblocklog;
 	int			dirblocksize;
-	char			*dsize;
 	int			dsunit;
 	int			dswidth;
 	int			force_overwrite;
@@ -3278,7 +3324,7 @@ main(
 	logagno = logblocks = rtblocks = rtextblocks = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
-	dsize = logsize = rtsize = protofile = NULL;
+	logsize = rtsize = protofile = NULL;
 	dsunit = dswidth = lalign = lsunit = 0;
 	nodsflag = 0;
 	force_overwrite = 0;
@@ -3427,6 +3473,7 @@ main(
 	 * Open and validate the device configurations
 	 */
 	open_devices(&cfg, &xi, (discard && !dry_run));
+	validate_datadev(&cfg, &cli);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3471,24 +3518,6 @@ main(
 		rtfile = _("volume rt");
 	else if (!xi.rtdev)
 		rtfile = _("none");
-	if (dsize && xi.dsize > 0 && dblocks > DTOBT(xi.dsize, blocklog)) {
-		fprintf(stderr,
-			_("size %s specified for data subvolume is too large, "
-			"maximum is %lld blocks\n"),
-			dsize, (long long)DTOBT(xi.dsize, blocklog));
-		usage();
-	} else if (!dsize && xi.dsize > 0)
-		dblocks = DTOBT(xi.dsize, blocklog);
-	else if (!dsize) {
-		fprintf(stderr, _("can't get size of data subvolume\n"));
-		usage();
-	}
-	if (dblocks < XFS_MIN_DATA_BLOCKS) {
-		fprintf(stderr,
-	_("size %lld of data subvolume is too small, minimum %d blocks\n"),
-			(long long)dblocks, XFS_MIN_DATA_BLOCKS);
-		usage();
-	}
 
 	if (loginternal && xi.logdev) {
 		fprintf(stderr,
@@ -3500,12 +3529,6 @@ main(
 		usage();
 	}
 
-	if (xi.dbsize > sectorsize) {
-		fprintf(stderr, _(
-"Warning: the data subvolume sector size %u is less than the sector size \n\
-reported by the device (%u).\n"),
-			sectorsize, xi.dbsize);
-	}
 	if (!loginternal && xi.lbsize > lsectorsize) {
 		fprintf(stderr, _(
 "Warning: the log subvolume sector size %u is less than the sector size\n\
-- 
2.13.3

[PATCH 33/42] mkfs: factor log device validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 147 ++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 89 insertions(+), 58 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 81f1e12a941e..2fcc0d39a0a8 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2574,6 +2574,92 @@ reported by the device (%u).\n"),
 	}
 }
 
+/*
+ * This is more complex than it needs to be because we still support volume
+ * based external logs. They are only discovered *after* the devices have been
+ * opened, hence the crazy "is this really an internal log" checks here.
+ */
+static void
+validate_logdev(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	char			**devname)
+{
+	struct libxfs_xinit	*xi = cli->xi;
+
+	*devname = NULL;
+
+	/* check for volume log first */
+	if (cli->loginternal && xi->volname && xi->logdev) {
+		*devname = _("volume log");
+		cfg->loginternal = false;
+	} else
+		cfg->loginternal = cli->loginternal;
+
+	/* now run device checks */
+	if (cfg->loginternal) {
+		if (xi->logdev) {
+			fprintf(stderr,
+_("can't have both external and internal logs\n"));
+			usage();
+		}
+
+		/*
+		 * if no sector size has been specified on the command line,
+		 * use what has been configured and validated for the data
+		 * device.
+		 */
+		if (!cli->lsectorsize) {
+			cfg->lsectorsize = cfg->sectorsize;
+			cfg->lsectorlog = cfg->sectorlog;
+		}
+
+		if (cfg->sectorsize != cfg->lsectorsize) {
+			fprintf(stderr,
+_("data and log sector sizes must be equal for internal logs\n"));
+			usage();
+		}
+		if (cli->logsize && cfg->logblocks >= cfg->dblocks) {
+			fprintf(stderr,
+_("log size %lld too large for internal log\n"),
+				(long long)cfg->logblocks);
+			usage();
+		}
+		*devname = _("internal log");
+		return;
+	}
+
+	/* External/log subvolume checks */
+	if (xi->logname)
+		*devname = xi->logname;
+	if (!*devname || !xi->logdev) {
+		fprintf(stderr, _("no log subvolume or external log.\n"));
+		usage();
+	}
+
+	if (!cfg->logblocks) {
+		if (xi->logBBsize == 0) {
+			fprintf(stderr,
+_("unable to get size of the log subvolume.\n"));
+			usage();
+		}
+		cfg->logblocks = DTOBT(xi->logBBsize, cfg->blocklog);
+	} else if (cfg->logblocks > DTOBT(xi->logBBsize, cfg->blocklog)) {
+		fprintf(stderr,
+_("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
+			cli->logsize,
+			(long long)DTOBT(xi->logBBsize, cfg->blocklog));
+		usage();
+	}
+
+	if (xi->lbsize > cfg->lsectorsize) {
+		fprintf(stderr, _(
+"Warning: the log subvolume sector size %u is less than the sector size\n\
+reported by the device (%u).\n"),
+			cfg->lsectorsize, xi->lbsize);
+	}
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -3223,15 +3309,12 @@ main(
 	char			*label = NULL;
 	int			laflag;
 	int			lalign;
-	int			ldflag;
-	int			liflag;
 	xfs_agnumber_t		logagno;
 	xfs_rfsblock_t		logblocks;
 	char			*logfile;
 	int			loginternal;
 	char			*logsize;
 	xfs_fsblock_t		logstart;
-	int			lvflag;
 	int			lsflag;
 	int			lsectorlog;
 	int			lsectorsize;
@@ -3319,7 +3402,7 @@ main(
 
 	agsize = daflag = dasize = dblocks = 0;
 	imflag = 0;
-	liflag = laflag = lsflag = ldflag = lvflag = 0;
+	laflag = lsflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
@@ -3375,15 +3458,7 @@ main(
 
 			/* temp don't break code */
 			logagno = cli.logagno;
-			loginternal = cli.loginternal;
-			logfile = xi.logname;
-			logsize = cli.logsize;
-
 			laflag = cli_opt_set(&lopts, L_AGNUM);
-			liflag = cli_opt_set(&lopts, L_INTERNAL);
-			ldflag = cli_opt_set(&lopts, L_NAME) ||
-				 cli_opt_set(&lopts, L_DEV);
-			lvflag = cli_opt_set(&lopts, L_VERSION);
 			/* end temp don't break code */
 			break;
 		case 'L':
@@ -3474,6 +3549,7 @@ main(
 	 */
 	open_devices(&cfg, &xi, (discard && !dry_run));
 	validate_datadev(&cfg, &cli);
+	validate_logdev(&cfg, &cli, &logfile);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3499,18 +3575,6 @@ main(
 	nodsflag = cfg.sb_feat.nodalign;
 	/* end temp don't break code */
 
-	if (!liflag && !ldflag)
-		loginternal = xi.logdev == 0;
-	if (xi.logname)
-		logfile = xi.logname;
-	else if (loginternal)
-		logfile = _("internal log");
-	else if (xi.volname && xi.logdev)
-		logfile = _("volume log");
-	else if (!ldflag) {
-		fprintf(stderr, _("no log subvolume or internal log\n"));
-		usage();
-	}
 	if (xi.rtname)
 		rtfile = xi.rtname;
 	else
@@ -3519,22 +3583,6 @@ main(
 	else if (!xi.rtdev)
 		rtfile = _("none");
 
-	if (loginternal && xi.logdev) {
-		fprintf(stderr,
-			_("can't have both external and internal logs\n"));
-		usage();
-	} else if (loginternal && sectorsize != lsectorsize) {
-		fprintf(stderr,
-	_("data and log sector sizes must be equal for internal logs\n"));
-		usage();
-	}
-
-	if (!loginternal && xi.lbsize > lsectorsize) {
-		fprintf(stderr, _(
-"Warning: the log subvolume sector size %u is less than the sector size\n\
-reported by the device (%u).\n"),
-			lsectorsize, xi.lbsize);
-	}
 	if (rtsize && xi.rtsize > 0 && xi.rtbsize > sectorsize) {
 		fprintf(stderr, _(
 "Warning: the realtime subvolume sector size %u is less than the sector size\n\
@@ -3711,24 +3759,7 @@ an AG size that is one stripe unit smaller, for example %llu.\n"),
 	min_logblocks = MAX(XFS_MIN_LOG_BLOCKS, min_logblocks);
 	if (!logsize && dblocks >= (1024*1024*1024) >> blocklog)
 		min_logblocks = MAX(min_logblocks, XFS_MIN_LOG_BYTES>>blocklog);
-	if (logsize && xi.logBBsize > 0 && logblocks > DTOBT(xi.logBBsize, blocklog)) {
-		fprintf(stderr,
-_("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
-			logsize, (long long)DTOBT(xi.logBBsize, blocklog));
-		usage();
-	} else if (!logsize && xi.logBBsize > 0) {
-		logblocks = DTOBT(xi.logBBsize, blocklog);
-	} else if (logsize && !xi.logdev && !loginternal) {
-		fprintf(stderr,
-			_("size specified for non-existent log subvolume\n"));
-		usage();
-	} else if (loginternal && logsize && logblocks >= dblocks) {
-		fprintf(stderr, _("size %lld too large for internal log\n"),
-			(long long)logblocks);
-		usage();
-	} else if (!loginternal && !xi.logdev) {
-		logblocks = 0;
-	} else if (loginternal && !logsize) {
+	if (loginternal && !logsize) {
 
 		if (dblocks < GIGABYTES(1, blocklog)) {
 			/* tiny filesystems get minimum sized logs. */
-- 
2.13.3

[PATCH 34/42] mkfs: factor rt device validation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 109 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 63 insertions(+), 46 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 2fcc0d39a0a8..c6032e4af4c5 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2661,6 +2661,64 @@ reported by the device (%u).\n"),
 }
 
 static void
+validate_rtdev(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	char			**devname)
+{
+	struct libxfs_xinit	*xi = cli->xi;
+
+	*devname = NULL;
+
+	if (!xi->rtdev) {
+		if (cli->rtsize) {
+			fprintf(stderr,
+_("size specified for non-existent rt subvolume\n"));
+			usage();
+		}
+
+		*devname = _("none");
+		cfg->rtblocks = 0;
+		cfg->rtextents = 0;
+		cfg->rtbmblocks = 0;
+		return;
+	}
+	if (!xi->rtsize) {
+		fprintf(stderr, _("Invalid zero length rt subvolume found\n"));
+		usage();
+	}
+
+	/* volume rtdev */
+	if (xi->volname)
+		*devname = _("volume rt");
+	else
+		*devname = xi->rtname;
+
+	if (cli->rtsize) {
+		if (cfg->rtblocks > DTOBT(xi->rtsize, cfg->blocklog)) {
+			fprintf(stderr,
+_("size %s specified for rt subvolume is too large, maxi->um is %lld blocks\n"),
+				cli->rtsize,
+				(long long)DTOBT(xi->rtsize, cfg->blocklog));
+			usage();
+		}
+		if (xi->rtbsize > cfg->sectorsize) {
+			fprintf(stderr, _(
+"Warning: the realtime subvolume sector size %u is less than the sector size\n\
+reported by the device (%u).\n"),
+				cfg->sectorsize, xi->rtbsize);
+		}
+	} else {
+		/* grab volume size */
+		cfg->rtblocks = DTOBT(xi->rtsize, cfg->blocklog);
+	}
+
+	cfg->rtextents = cfg->rtblocks / cfg->rtextblocks;
+	cfg->rtbmblocks = (xfs_extlen_t)howmany(cfg->rtextents,
+						NBBY * cfg->blocksize);
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -3333,7 +3391,6 @@ main(
 	xfs_extlen_t		rtextblocks;
 	xfs_rtblock_t		rtextents;
 	char			*rtfile;
-	char			*rtsize;
 	xfs_sb_t		*sbp;
 	int			sectorlog;
 	uint64_t		tmp_agsize;
@@ -3407,7 +3464,7 @@ main(
 	logagno = logblocks = rtblocks = rtextblocks = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
-	logsize = rtsize = protofile = NULL;
+	logsize = protofile = NULL;
 	dsunit = dswidth = lalign = lsunit = 0;
 	nodsflag = 0;
 	force_overwrite = 0;
@@ -3426,6 +3483,7 @@ main(
 			break;
 		case 'b':
 		case 'n':
+		case 'r':
 		case 's':
 			parse_subopts(c, optarg, &cli);
 			break;
@@ -3487,14 +3545,6 @@ main(
 		case 'q':
 			quiet = 1;
 			break;
-		case 'r':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			rtsize = cli.rtsize;
-			/* end temp don't break code */
-			break;
-			break;
 		case 'V':
 			printf(_("%s version %s\n"), progname, VERSION);
 			exit(0);
@@ -3550,6 +3600,7 @@ main(
 	open_devices(&cfg, &xi, (discard && !dry_run));
 	validate_datadev(&cfg, &cli);
 	validate_logdev(&cfg, &cli, &logfile);
+	validate_rtdev(&cfg, &cli, &rtfile);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3569,48 +3620,14 @@ main(
 	logblocks = cfg.logblocks;
 	rtblocks = cfg.rtblocks;
 	rtextblocks = cfg.rtextblocks;
+	nbmblocks = cfg.rtbmblocks;
+	rtextents = cfg.rtextents;
 	dsunit = cfg.dsunit;
 	dswidth = cfg.dswidth;
 	lsunit = cfg.lsunit;
 	nodsflag = cfg.sb_feat.nodalign;
 	/* end temp don't break code */
 
-	if (xi.rtname)
-		rtfile = xi.rtname;
-	else
-	if (xi.volname && xi.rtdev)
-		rtfile = _("volume rt");
-	else if (!xi.rtdev)
-		rtfile = _("none");
-
-	if (rtsize && xi.rtsize > 0 && xi.rtbsize > sectorsize) {
-		fprintf(stderr, _(
-"Warning: the realtime subvolume sector size %u is less than the sector size\n\
-reported by the device (%u).\n"),
-			sectorsize, xi.rtbsize);
-	}
-
-	if (rtsize && xi.rtsize > 0 && rtblocks > DTOBT(xi.rtsize, blocklog)) {
-		fprintf(stderr,
-			_("size %s specified for rt subvolume is too large, "
-			"maximum is %lld blocks\n"),
-			rtsize, (long long)DTOBT(xi.rtsize, blocklog));
-		usage();
-	} else if (!rtsize && xi.rtsize > 0)
-		rtblocks = DTOBT(xi.rtsize, blocklog);
-	else if (rtsize && !xi.rtdev) {
-		fprintf(stderr,
-			_("size specified for non-existent rt subvolume\n"));
-		usage();
-	}
-	if (xi.rtdev) {
-		rtextents = rtblocks / rtextblocks;
-		nbmblocks = (xfs_extlen_t)howmany(rtextents, NBBY * blocksize);
-	} else {
-		rtextents = rtblocks = 0;
-		nbmblocks = 0;
-	}
-
 	if (dasize) {		/* User-specified AG size */
 		/*
 		 * Check specified agsize is a multiple of blocksize.
-- 
2.13.3

[PATCH 35/42] mkfs: factor AG geometry calculations

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 61 +++++++++++++++++++++++++++++++++++++++------------------
 1 file changed, 42 insertions(+), 19 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index c6032e4af4c5..a0dcea2609a6 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2719,6 +2719,38 @@ reported by the device (%u).\n"),
 }
 
 static void
+calculate_initial_ag_geometry(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli)
+{
+	if (cli->agsize) {		/* User-specified AG size */
+		cfg->agsize = getnum(cli->agsize, &dopts, D_AGSIZE);
+
+		/*
+		 * Check specified agsize is a multiple of blocksize.
+		 */
+		if (cfg->agsize % cfg->blocksize) {
+			fprintf(stderr,
+_("agsize (%s) not a multiple of fs blk size (%d)\n"),
+				cli->agsize, cfg->blocksize);
+			usage();
+		}
+		cfg->agsize /= cfg->blocksize;
+		cfg->agcount = cfg->dblocks / cfg->agsize +
+				(cfg->dblocks % cfg->agsize != 0);
+
+	} else if (cli->agcount) {	/* User-specified AG count */
+		cfg->agcount = cli->agcount;
+		cfg->agsize = cfg->dblocks / cfg->agcount +
+				(cfg->dblocks % cfg->agcount != 0);
+	} else {
+		calc_default_ag_geometry(cfg->blocklog, cfg->dblocks,
+					 cfg->dsunit, &cfg->agsize,
+					 &cfg->agcount);
+	}
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -3602,6 +3634,14 @@ main(
 	validate_logdev(&cfg, &cli, &logfile);
 	validate_rtdev(&cfg, &cli, &rtfile);
 
+	/*
+	 * At this point when know exactly what size all the devices are,
+	 * so we can start validating and calculating layout options that are
+	 * dependent on device sizes. Once calculated, make sure everything
+	 * aligns to device geometry correctly.
+	 */
+	calculate_initial_ag_geometry(&cfg, &cli);
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3626,27 +3666,10 @@ main(
 	dswidth = cfg.dswidth;
 	lsunit = cfg.lsunit;
 	nodsflag = cfg.sb_feat.nodalign;
+	agsize = cfg.agsize;
+	agcount = cfg.agcount;
 	/* end temp don't break code */
 
-	if (dasize) {		/* User-specified AG size */
-		/*
-		 * Check specified agsize is a multiple of blocksize.
-		 */
-		if (agsize % blocksize) {
-			fprintf(stderr,
-		_("agsize (%lld) not a multiple of fs blk size (%d)\n"),
-				(long long)agsize, blocksize);
-			usage();
-		}
-		agsize /= blocksize;
-		agcount = dblocks / agsize + (dblocks % agsize != 0);
-
-	} else if (daflag) {	/* User-specified AG count */
-		agsize = dblocks / agcount + (dblocks % agcount != 0);
-	} else {
-		calc_default_ag_geometry(blocklog, dblocks,
-				dsunit | dswidth, &agsize, &agcount);
-	}
 
 	/*
 	 * If dsunit is a multiple of fs blocksize, then check that is a
-- 
2.13.3

[PATCH 36/42] mkfs: factor AG alignment

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 246 +++++++++++++++++++++++++++-----------------------------
 1 file changed, 117 insertions(+), 129 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index a0dcea2609a6..1ab9e98b8b02 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2750,6 +2750,121 @@ _("agsize (%s) not a multiple of fs blk size (%d)\n"),
 	}
 }
 
+/*
+ * Align the AG size to stripe geometry. If this fails and we are using
+ * discovered stripe geometry, tell the caller to clear the stripe geometry.
+ * Otherwise, set the aligned geometry (valid or invalid!) so that the
+ * validation call will fail and exit.
+ */
+static void
+align_ag_geometry(
+	struct mkfs_params	*cfg)
+{
+	uint64_t	tmp_agsize;
+	int		dsunit = cfg->dsunit;
+
+	if (!dsunit)
+		return;
+
+	/*
+	 * agsize is not a multiple of dsunit
+	 */
+	if ((cfg->agsize % dsunit) != 0) {
+		/*
+		 * Round up to stripe unit boundary. Also make sure
+		 * that agsize is still larger than
+		 * XFS_AG_MIN_BLOCKS(blocklog)
+		 */
+		tmp_agsize = ((cfg->agsize + dsunit - 1) / dsunit) * dsunit;
+		/*
+		 * Round down to stripe unit boundary if rounding up
+		 * created an AG size that is larger than the AG max.
+		 */
+		if (tmp_agsize > XFS_AG_MAX_BLOCKS(cfg->blocklog))
+			tmp_agsize = (cfg->agsize / dsunit) * dsunit;
+
+		if (tmp_agsize < XFS_AG_MIN_BLOCKS(cfg->blocklog) &&
+		    tmp_agsize > XFS_AG_MAX_BLOCKS(cfg->blocklog)) {
+
+			/*
+			 * If the AG size is invalid and we are using device
+			 * probed stripe alignment, just clear the alignment
+			 * and continue on.
+			 */
+			if (!cli_opt_set(&dopts, D_SUNIT) &&
+			    !cli_opt_set(&dopts, D_SU)) {
+				cfg->dsunit = 0;
+				cfg->dswidth = 0;
+				goto validate;
+			}
+			/*
+			 * set the agsize to the invalid value so the following
+			 * validation of the ag will fail and print a nice error
+			 * and exit.
+			 */
+			cfg->agsize = tmp_agsize;
+			goto validate;
+		}
+
+		/* update geometry to be stripe unit aligned */
+		cfg->agsize = tmp_agsize;
+		if (!cli_opt_set(&dopts, D_AGCOUNT))
+			cfg->agcount = cfg->dblocks / cfg->agsize +
+					(cfg->dblocks % cfg->agsize != 0);
+		if (cli_opt_set(&dopts, D_AGSIZE))
+			fprintf(stderr,
+_("agsize rounded to %lld, sunit = %d\n"),
+				(long long)cfg->agsize, dsunit);
+	}
+
+	if ((cfg->agsize % cfg->dswidth) == 0 && cfg->agcount > 1) {
+
+		if (cli_opt_set(&dopts, D_AGCOUNT) ||
+		    cli_opt_set(&dopts, D_AGSIZE)) {
+			fprintf(stderr, _(
+"Warning: AG size is a multiple of stripe width.  This can cause performance\n\
+problems by aligning all AGs on the same disk.  To avoid this, run mkfs with\n\
+an AG size that is one stripe unit smaller or larger, for example %llu.\n"),
+				(unsigned long long)cfg->agsize - dsunit);
+			goto validate;
+		}
+
+		/*
+		 * This is a non-optimal configuration because all AGs start on
+		 * the same disk in the stripe.  Changing the AG size by one
+		 * sunit will guarantee that this does not happen.
+		 */
+		tmp_agsize = cfg->agsize - dsunit;
+		if (tmp_agsize < XFS_AG_MIN_BLOCKS(cfg->blocklog)) {
+			tmp_agsize = cfg->agsize + dsunit;
+			if (cfg->dblocks < cfg->agsize) {
+				/* oh well, nothing to do */
+				tmp_agsize = cfg->agsize;
+			}
+		}
+
+		cfg->agsize = tmp_agsize;
+		cfg->agcount = cfg->dblocks / cfg->agsize +
+				(cfg->dblocks % cfg->agsize != 0);
+	}
+
+	/*
+	 * If the last AG is too small, reduce the filesystem size
+	 * and drop the blocks.
+	 */
+	if (cfg->dblocks % cfg->agsize != 0 &&
+	     (cfg->dblocks % cfg->agsize < XFS_AG_MIN_BLOCKS(cfg->blocklog))) {
+		ASSERT(!cli_opt_set(&dopts, D_AGCOUNT));
+		cfg->dblocks = (xfs_rfsblock_t)((cfg->agcount - 1) * cfg->agsize);
+		cfg->agcount--;
+		ASSERT(cfg->agcount != 0);
+	}
+
+validate:
+	validate_ag_geometry(cfg->blocklog, cfg->dblocks,
+			     cfg->agsize, cfg->agcount);
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -3381,8 +3496,6 @@ main(
 	int			blocklog;
 	xfs_buf_t		*buf;
 	int			c;
-	int			daflag;
-	int			dasize;
 	xfs_rfsblock_t		dblocks;
 	char			*dfile;
 	int			dirblocklog;
@@ -3413,7 +3526,6 @@ main(
 	xfs_mount_t		*mp;
 	xfs_mount_t		mbuf;
 	xfs_extlen_t		nbmblocks;
-	int			nodsflag;
 	int			dry_run = 0;
 	int			discard = 1;
 	char			*protofile;
@@ -3425,7 +3537,6 @@ main(
 	char			*rtfile;
 	xfs_sb_t		*sbp;
 	int			sectorlog;
-	uint64_t		tmp_agsize;
 	uuid_t			uuid;
 	int			worst_freelist;
 	libxfs_init_t		xi;
@@ -3489,7 +3600,7 @@ main(
 	 */
 	cli.loginternal = 1;	/* internal by default */
 
-	agsize = daflag = dasize = dblocks = 0;
+	agsize = dblocks = 0;
 	imflag = 0;
 	laflag = lsflag = 0;
 	loginternal = 1;
@@ -3498,7 +3609,6 @@ main(
 	dfile = logfile = rtfile = NULL;
 	logsize = protofile = NULL;
 	dsunit = dswidth = lalign = lsunit = 0;
-	nodsflag = 0;
 	force_overwrite = 0;
 	worst_freelist = 0;
 	memset(&fsx, 0, sizeof(fsx));
@@ -3523,13 +3633,6 @@ main(
 			parse_subopts(c, optarg, &cli);
 
 			/* temp don't break code */
-			agcount = cli.agcount;
-			if (cli_opt_set(&dopts, D_AGSIZE)) {
-				agsize = getnum(cli.agsize, &dopts, D_AGSIZE);
-				dasize = 1;
-			}
-			daflag = cli_opt_set(&dopts, D_AGCOUNT);
-
 			fsx.fsx_xflags |= cli.fsx.fsx_xflags;
 			fsx.fsx_projid = cli.fsx.fsx_projid;
 			fsx.fsx_extsize = cli.fsx.fsx_extsize;
@@ -3641,6 +3744,7 @@ main(
 	 * aligns to device geometry correctly.
 	 */
 	calculate_initial_ag_geometry(&cfg, &cli);
+	align_ag_geometry(&cfg);
 
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
@@ -3665,127 +3769,11 @@ main(
 	dsunit = cfg.dsunit;
 	dswidth = cfg.dswidth;
 	lsunit = cfg.lsunit;
-	nodsflag = cfg.sb_feat.nodalign;
 	agsize = cfg.agsize;
 	agcount = cfg.agcount;
 	/* end temp don't break code */
 
 
-	/*
-	 * If dsunit is a multiple of fs blocksize, then check that is a
-	 * multiple of the agsize too
-	 */
-	if (dsunit && !(BBTOB(dsunit) % blocksize) &&
-	    dswidth && !(BBTOB(dswidth) % blocksize)) {
-
-		/* convert from 512 byte blocks to fs blocksize */
-		dsunit = DTOBT(dsunit, blocklog);
-		dswidth = DTOBT(dswidth, blocklog);
-
-		/*
-		 * agsize is not a multiple of dsunit
-		 */
-		if ((agsize % dsunit) != 0) {
-			/*
-			 * Round up to stripe unit boundary. Also make sure
-			 * that agsize is still larger than
-			 * XFS_AG_MIN_BLOCKS(blocklog)
-		 	 */
-			tmp_agsize = ((agsize + (dsunit - 1))/ dsunit) * dsunit;
-			/*
-			 * Round down to stripe unit boundary if rounding up
-			 * created an AG size that is larger than the AG max.
-			 */
-			if (tmp_agsize > XFS_AG_MAX_BLOCKS(blocklog))
-				tmp_agsize = ((agsize) / dsunit) * dsunit;
-
-			if ((tmp_agsize >= XFS_AG_MIN_BLOCKS(blocklog)) &&
-			    (tmp_agsize <= XFS_AG_MAX_BLOCKS(blocklog))) {
-				agsize = tmp_agsize;
-				if (!daflag)
-					agcount = dblocks/agsize +
-						(dblocks % agsize != 0);
-				if (dasize)
-					fprintf(stderr,
-				_("agsize rounded to %lld, swidth = %d\n"),
-						(long long)agsize, dswidth);
-			} else {
-				if (nodsflag) {
-					dsunit = dswidth = 0;
-				} else {
-					/*
-					 * agsize is out of bounds, this will
-					 * print nice details & exit.
-					 */
-					validate_ag_geometry(blocklog, dblocks,
-							    agsize, agcount);
-					exit(1);
-				}
-			}
-		}
-		if (dswidth && ((agsize % dswidth) == 0) && (agcount > 1)) {
-			/* This is a non-optimal configuration because all AGs
-			 * start on the same disk in the stripe.  Changing
-			 * the AG size by one sunit will guarantee that this
-			 * does not happen.
-			 */
-			tmp_agsize = agsize - dsunit;
-			if (tmp_agsize < XFS_AG_MIN_BLOCKS(blocklog)) {
-				tmp_agsize = agsize + dsunit;
-				if (dblocks < agsize) {
-					/* oh well, nothing to do */
-					tmp_agsize = agsize;
-				}
-			}
-			if (daflag || dasize) {
-				fprintf(stderr, _(
-"Warning: AG size is a multiple of stripe width.  This can cause performance\n\
-problems by aligning all AGs on the same disk.  To avoid this, run mkfs with\n\
-an AG size that is one stripe unit smaller, for example %llu.\n"),
-					(unsigned long long)tmp_agsize);
-			} else {
-				agsize = tmp_agsize;
-				agcount = dblocks/agsize + (dblocks % agsize != 0);
-				/*
-				 * If the last AG is too small, reduce the
-				 * filesystem size and drop the blocks.
-				 */
-				if ( dblocks % agsize != 0 &&
-				    (dblocks % agsize <
-				    XFS_AG_MIN_BLOCKS(blocklog))) {
-					dblocks = (xfs_rfsblock_t)((agcount - 1) * agsize);
-					agcount--;
-					ASSERT(agcount != 0);
-				}
-			}
-		}
-	} else {
-		if (nodsflag)
-			dsunit = dswidth = 0;
-		else {
-			fprintf(stderr,
-				_("%s: Stripe unit(%d) or stripe width(%d) is "
-				"not a multiple of the block size(%d)\n"),
-				progname, BBTOB(dsunit), BBTOB(dswidth),
-				blocksize);
-			exit(1);
-		}
-	}
-
-	/*
-	 * If the last AG is too small, reduce the filesystem size
-	 * and drop the blocks.
-	 */
-	if ( dblocks % agsize != 0 &&
-	     (dblocks % agsize < XFS_AG_MIN_BLOCKS(blocklog))) {
-		ASSERT(!daflag);
-		dblocks = (xfs_rfsblock_t)((agcount - 1) * agsize);
-		agcount--;
-		ASSERT(agcount != 0);
-	}
-
-	validate_ag_geometry(blocklog, dblocks, agsize, agcount);
-
 	if (!imflag)
 		imaxpct = calc_default_imaxpct(blocklog, dblocks);
 
-- 
2.13.3

Re: [PATCH 36/42] mkfs: factor AG alignment

[Dave Chinner]

On Wed, Aug 30, 2017 at 09:50:46AM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
....

This patch needs this delta fix, otherwise large filesystems will be
configured wrongly.

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com


agcount wrong when at full 1TB sized AGs:
$ sudo ~/packages/mkfs.xfs -f -d size=4t,file,name=/mnt/test/foo
Default configuration sourced from package build definitions
meta-data=/mnt/test/foo          isize=512    agcount=5, agsize=268435455 blks

---
 mkfs/xfs_mkfs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index bec12d473099..21394a1ae946 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2707,7 +2707,7 @@ align_ag_geometry(
 	int		dsunit = cfg->dsunit;
 
 	if (!dsunit)
-		return;
+		goto validate;
 
 	/*
 	 * agsize is not a multiple of dsunit
@@ -2791,6 +2791,7 @@ an AG size that is one stripe unit smaller or larger, for example %llu.\n"),
 				(cfg->dblocks % cfg->agsize != 0);
 	}
 
+validate:
 	/*
 	 * If the last AG is too small, reduce the filesystem size
 	 * and drop the blocks.
@@ -2803,7 +2804,6 @@ an AG size that is one stripe unit smaller or larger, for example %llu.\n"),
 		ASSERT(cfg->agcount != 0);
 	}
 
-validate:
 	validate_ag_geometry(cfg->blocklog, cfg->dblocks,
 			     cfg->agsize, cfg->agcount);
 }

[PATCH 37/42] mkfs: rework imaxpct calculation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 65 +++++++++++++++++++++++++--------------------------------
 1 file changed, 29 insertions(+), 36 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 1ab9e98b8b02..ae6514cb0a1f 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1055,28 +1055,6 @@ validate_log_size(uint64_t logblocks, int blocklog, int min_logblocks)
 	}
 }
 
-static int
-calc_default_imaxpct(
-	int		blocklog,
-	uint64_t	dblocks)
-{
-	/*
-	 * This returns the % of the disk space that is used for
-	 * inodes, it changes relatively to the FS size:
-	 *  - over  50 TB, use 1%,
-	 *  - 1TB - 50 TB, use 5%,
-	 *  - under  1 TB, use XFS_DFL_IMAXIMUM_PCT (25%).
-	 */
-
-	if (dblocks < TERABYTES(1, blocklog)) {
-		return XFS_DFL_IMAXIMUM_PCT;
-	} else if (dblocks < TERABYTES(50, blocklog)) {
-		return 5;
-	}
-
-	return 1;
-}
-
 static void
 validate_ag_geometry(
 	int		blocklog,
@@ -2866,6 +2844,31 @@ validate:
 }
 
 static void
+calculate_imaxpct(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli)
+{
+	cfg->imaxpct = cli->imaxpct;
+	if (cfg->imaxpct)
+		return;
+
+	/*
+	 * This returns the % of the disk space that is used for
+	 * inodes, it changes relatively to the FS size:
+	 *  - over  50 TB, use 1%,
+	 *  - 1TB - 50 TB, use 5%,
+	 *  - under  1 TB, use XFS_DFL_IMAXIMUM_PCT (25%).
+	 */
+
+	if (cfg->dblocks < TERABYTES(1, cfg->blocklog))
+		cfg->imaxpct = XFS_DFL_IMAXIMUM_PCT;
+	else if (cfg->dblocks < TERABYTES(50, cfg->blocklog))
+		cfg->imaxpct = 5;
+	else
+		cfg->imaxpct = 1;
+}
+
+static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
 	char			*dfile,
@@ -3505,7 +3508,6 @@ main(
 	int			force_overwrite;
 	struct fsxattr		fsx;
 	int			imaxpct;
-	int			imflag;
 	int			inodelog;
 	int			inopblock;
 	int			isize;
@@ -3601,7 +3603,6 @@ main(
 	cli.loginternal = 1;	/* internal by default */
 
 	agsize = dblocks = 0;
-	imflag = 0;
 	laflag = lsflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
@@ -3624,6 +3625,7 @@ main(
 			force_overwrite = 1;
 			break;
 		case 'b':
+		case 'i':
 		case 'n':
 		case 'r':
 		case 's':
@@ -3638,14 +3640,6 @@ main(
 			fsx.fsx_extsize = cli.fsx.fsx_extsize;
 			/* end temp don't break code */
 			break;
-		case 'i':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			imaxpct = cli.imaxpct;
-			imflag = cli_opt_set(&iopts, I_MAXPCT);
-			/* end temp don't break code */
-			break;
 		case 'l':
 			parse_subopts(c, optarg, &cli);
 
@@ -3746,6 +3740,8 @@ main(
 	calculate_initial_ag_geometry(&cfg, &cli);
 	align_ag_geometry(&cfg);
 
+	calculate_imaxpct(&cfg, &cli);
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3771,12 +3767,9 @@ main(
 	lsunit = cfg.lsunit;
 	agsize = cfg.agsize;
 	agcount = cfg.agcount;
+	imaxpct = cfg.imaxpct;
 	/* end temp don't break code */
 
-
-	if (!imflag)
-		imaxpct = calc_default_imaxpct(blocklog, dblocks);
-
 	min_logblocks = max_trans_res(agsize,
 				   sb_feat.crcs_enabled, sb_feat.dir_version,
 				   sectorlog, blocklog, inodelog, dirblocklog,
-- 
2.13.3

[PATCH 38/42] mkfs: factor initial mount setup

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 265 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 141 insertions(+), 124 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index ae6514cb0a1f..3fc485524d99 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -1223,96 +1223,6 @@ discard_blocks(dev_t dev, uint64_t nsectors)
 		platform_discard_blocks(fd, 0, nsectors << 9);
 }
 
-static void
-sb_set_features(
-	struct xfs_sb		*sbp,
-	struct sb_feat_args	*fp,
-	int			sectsize,
-	int			lsectsize,
-	int			dsunit)
-{
-
-	sbp->sb_versionnum = XFS_DFL_SB_VERSION_BITS;
-	if (fp->crcs_enabled)
-		sbp->sb_versionnum |= XFS_SB_VERSION_5;
-	else
-		sbp->sb_versionnum |= XFS_SB_VERSION_4;
-
-	if (fp->inode_align)
-		sbp->sb_versionnum |= XFS_SB_VERSION_ALIGNBIT;
-	if (dsunit)
-		sbp->sb_versionnum |= XFS_SB_VERSION_DALIGNBIT;
-	if (fp->log_version == 2)
-		sbp->sb_versionnum |= XFS_SB_VERSION_LOGV2BIT;
-	if (fp->attr_version == 1)
-		sbp->sb_versionnum |= XFS_SB_VERSION_ATTRBIT;
-	if (sectsize > BBSIZE || lsectsize > BBSIZE)
-		sbp->sb_versionnum |= XFS_SB_VERSION_SECTORBIT;
-	if (fp->nci)
-		sbp->sb_versionnum |= XFS_SB_VERSION_BORGBIT;
-
-
-	sbp->sb_features2 = 0;
-	if (fp->lazy_sb_counters)
-		sbp->sb_features2 |= XFS_SB_VERSION2_LAZYSBCOUNTBIT;
-	if (!fp->projid16bit)
-		sbp->sb_features2 |= XFS_SB_VERSION2_PROJID32BIT;
-	if (fp->parent_pointers)
-		sbp->sb_features2 |= XFS_SB_VERSION2_PARENTBIT;
-	if (fp->crcs_enabled)
-		sbp->sb_features2 |= XFS_SB_VERSION2_CRCBIT;
-	if (fp->attr_version == 2)
-		sbp->sb_features2 |= XFS_SB_VERSION2_ATTR2BIT;
-
-	/* v5 superblocks have their own feature bit for dirftype */
-	if (fp->dirftype && !fp->crcs_enabled)
-		sbp->sb_features2 |= XFS_SB_VERSION2_FTYPE;
-
-	/* update whether extended features are in use */
-	if (sbp->sb_features2 != 0)
-		sbp->sb_versionnum |= XFS_SB_VERSION_MOREBITSBIT;
-
-	/*
-	 * Due to a structure alignment issue, sb_features2 ended up in one
-	 * of two locations, the second "incorrect" location represented by
-	 * the sb_bad_features2 field. To avoid older kernels mounting
-	 * filesystems they shouldn't, set both field to the same value.
-	 */
-	sbp->sb_bad_features2 = sbp->sb_features2;
-
-	if (!fp->crcs_enabled)
-		return;
-
-	/* default features for v5 filesystems */
-	sbp->sb_features_compat = 0;
-	sbp->sb_features_ro_compat = 0;
-	sbp->sb_features_incompat = XFS_SB_FEAT_INCOMPAT_FTYPE;
-	sbp->sb_features_log_incompat = 0;
-
-	if (fp->finobt)
-		sbp->sb_features_ro_compat = XFS_SB_FEAT_RO_COMPAT_FINOBT;
-	if (fp->rmapbt)
-		sbp->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_RMAPBT;
-	if (fp->reflink)
-		sbp->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_REFLINK;
-
-	/*
-	 * Sparse inode chunk support has two main inode alignment requirements.
-	 * First, sparse chunk alignment must match the cluster size. Second,
-	 * full chunk alignment must match the inode chunk size.
-	 *
-	 * Copy the already calculated/scaled inoalignmt to spino_align and
-	 * update the former to the full inode chunk size.
-	 */
-	if (fp->spinodes) {
-		sbp->sb_spino_align = sbp->sb_inoalignmt;
-		sbp->sb_inoalignmt = XFS_INODES_PER_CHUNK *
-			sbp->sb_inodesize >> sbp->sb_blocklog;
-		sbp->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_SPINODES;
-	}
-
-}
-
 static __attribute__((noreturn)) void
 illegal_option(
 	const char		*value,
@@ -2868,6 +2778,137 @@ calculate_imaxpct(
 		cfg->imaxpct = 1;
 }
 
+/*
+ * Set up the initial state of the superblock so we can start using the
+ * libxfs geometry macros.
+ */
+static void
+sb_set_features(
+	struct mkfs_params	*cfg,
+	struct xfs_sb		*sbp)
+{
+	struct sb_feat_args	*fp = &cfg->sb_feat;
+
+	sbp->sb_versionnum = XFS_DFL_SB_VERSION_BITS;
+	if (fp->crcs_enabled)
+		sbp->sb_versionnum |= XFS_SB_VERSION_5;
+	else
+		sbp->sb_versionnum |= XFS_SB_VERSION_4;
+
+	if (fp->inode_align) {
+		int     cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
+
+		sbp->sb_versionnum |= XFS_SB_VERSION_ALIGNBIT;
+		if (cfg->sb_feat.crcs_enabled)
+			cluster_size *= cfg->inodesize / XFS_DINODE_MIN_SIZE;
+		sbp->sb_inoalignmt = cluster_size >> cfg->blocklog;
+	} else
+		sbp->sb_inoalignmt = 0;
+
+	if (cfg->dsunit)
+		sbp->sb_versionnum |= XFS_SB_VERSION_DALIGNBIT;
+	if (fp->log_version == 2)
+		sbp->sb_versionnum |= XFS_SB_VERSION_LOGV2BIT;
+	if (fp->attr_version == 1)
+		sbp->sb_versionnum |= XFS_SB_VERSION_ATTRBIT;
+	if (fp->nci)
+		sbp->sb_versionnum |= XFS_SB_VERSION_BORGBIT;
+
+	if (cfg->sectorsize > BBSIZE || cfg->lsectorsize > BBSIZE) {
+		sbp->sb_versionnum |= XFS_SB_VERSION_SECTORBIT;
+		sbp->sb_logsectlog = (uint8_t)cfg->lsectorlog;
+		sbp->sb_logsectsize = (uint16_t)cfg->lsectorsize;
+	} else {
+		sbp->sb_logsectlog = 0;
+		sbp->sb_logsectsize = 0;
+	}
+
+	sbp->sb_features2 = 0;
+	if (fp->lazy_sb_counters)
+		sbp->sb_features2 |= XFS_SB_VERSION2_LAZYSBCOUNTBIT;
+	if (!fp->projid16bit)
+		sbp->sb_features2 |= XFS_SB_VERSION2_PROJID32BIT;
+	if (fp->parent_pointers)
+		sbp->sb_features2 |= XFS_SB_VERSION2_PARENTBIT;
+	if (fp->crcs_enabled)
+		sbp->sb_features2 |= XFS_SB_VERSION2_CRCBIT;
+	if (fp->attr_version == 2)
+		sbp->sb_features2 |= XFS_SB_VERSION2_ATTR2BIT;
+
+	/* v5 superblocks have their own feature bit for dirftype */
+	if (fp->dirftype && !fp->crcs_enabled)
+		sbp->sb_features2 |= XFS_SB_VERSION2_FTYPE;
+
+	/* update whether extended features are in use */
+	if (sbp->sb_features2 != 0)
+		sbp->sb_versionnum |= XFS_SB_VERSION_MOREBITSBIT;
+
+	/*
+	 * Due to a structure alignment issue, sb_features2 ended up in one
+	 * of two locations, the second "incorrect" location represented by
+	 * the sb_bad_features2 field. To avoid older kernels mounting
+	 * filesystems they shouldn't, set both field to the same value.
+	 */
+	sbp->sb_bad_features2 = sbp->sb_features2;
+
+	if (!fp->crcs_enabled)
+		return;
+
+	/* default features for v5 filesystems */
+	sbp->sb_features_compat = 0;
+	sbp->sb_features_ro_compat = 0;
+	sbp->sb_features_incompat = XFS_SB_FEAT_INCOMPAT_FTYPE;
+	sbp->sb_features_log_incompat = 0;
+
+	if (fp->finobt)
+		sbp->sb_features_ro_compat = XFS_SB_FEAT_RO_COMPAT_FINOBT;
+	if (fp->rmapbt)
+		sbp->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_RMAPBT;
+	if (fp->reflink)
+		sbp->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_REFLINK;
+
+	/*
+	 * Sparse inode chunk support has two main inode alignment requirements.
+	 * First, sparse chunk alignment must match the cluster size. Second,
+	 * full chunk alignment must match the inode chunk size.
+	 *
+	 * Copy the already calculated/scaled inoalignmt to spino_align and
+	 * update the former to the full inode chunk size.
+	 */
+	if (fp->spinodes) {
+		sbp->sb_spino_align = sbp->sb_inoalignmt;
+		sbp->sb_inoalignmt = XFS_INODES_PER_CHUNK *
+				cfg->inodesize >> cfg->blocklog;
+		sbp->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_SPINODES;
+	}
+
+}
+
+/*
+ * Set up mount and superblock with the minimum parameters required for
+ * the libxfs macros needed by the log sizing code to run successfully.
+ */
+static void
+initialise_mount(
+	struct mkfs_params	*cfg,
+	struct xfs_mount	*mp,
+	struct xfs_sb		*sbp)
+{
+	sbp->sb_blocklog = (uint8_t)cfg->blocklog;
+	sbp->sb_sectlog = (uint8_t)cfg->sectorlog;
+	sbp->sb_agblklog = (uint8_t)libxfs_log2_roundup(cfg->agsize);
+	sbp->sb_agblocks = (xfs_agblock_t)cfg->agsize;
+	sbp->sb_agcount = (xfs_agnumber_t)cfg->agcount;
+	mp->m_blkbb_log = sbp->sb_blocklog - BBSHIFT;
+	mp->m_sectbb_log = sbp->sb_sectlog - BBSHIFT;
+
+	/*
+	 * sb_versionnum, finobt and rmapbt flags must be set before we use
+	 * libxfs_prealloc_blocks().
+	 */
+	sb_set_features(cfg, sbp);
+}
+
 static void
 print_mkfs_cfg(
 	struct mkfs_params	*cfg,
@@ -2963,21 +3004,6 @@ setup_superblock(
 	} else
 		sbp->sb_logsunit = 0;
 
-	if (cfg->sb_feat.inode_align) {
-		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
-		if (cfg->sb_feat.crcs_enabled)
-			cluster_size *= cfg->inodesize / XFS_DINODE_MIN_SIZE;
-		sbp->sb_inoalignmt = cluster_size >> cfg->blocklog;
-	} else
-		sbp->sb_inoalignmt = 0;
-
-	if (cfg->lsectorsize != BBSIZE || cfg->sectorsize != BBSIZE) {
-		sbp->sb_logsectlog = (uint8_t)cfg->lsectorlog;
-		sbp->sb_logsectsize = (uint16_t)cfg->lsectorsize;
-	} else {
-		sbp->sb_logsectlog = 0;
-		sbp->sb_logsectsize = 0;
-	}
 }
 
 /*
@@ -3525,8 +3551,6 @@ main(
 	int			lsectorsize;
 	int			lsunit;
 	int			min_logblocks;
-	xfs_mount_t		*mp;
-	xfs_mount_t		mbuf;
 	xfs_extlen_t		nbmblocks;
 	int			dry_run = 0;
 	int			discard = 1;
@@ -3537,11 +3561,13 @@ main(
 	xfs_extlen_t		rtextblocks;
 	xfs_rtblock_t		rtextents;
 	char			*rtfile;
-	xfs_sb_t		*sbp;
 	int			sectorlog;
 	uuid_t			uuid;
 	int			worst_freelist;
 	libxfs_init_t		xi;
+	struct xfs_mount	mbuf = {};
+	struct xfs_mount	*mp = &mbuf;
+	struct xfs_sb		*sbp = &mp->m_sb;
 	struct fs_topology	ft;
 	struct sb_feat_args	sb_feat;
 	/* build time defaults */
@@ -3742,6 +3768,13 @@ main(
 
 	calculate_imaxpct(&cfg, &cli);
 
+	/*
+	 * Set up the basic superblock parameters now so that we can use
+	 * the geometry information we've already validated in libxfs
+	 * provided functions to determine on-disk format information.
+	 */
+	initialise_mount(&cfg, mp, sbp);
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3821,22 +3854,6 @@ main(
 	validate_log_size(logblocks, blocklog, min_logblocks);
 
 	protostring = setup_proto(protofile);
-	mp = &mbuf;
-	sbp = &mp->m_sb;
-	memset(mp, 0, sizeof(xfs_mount_t));
-	sbp->sb_blocklog = (uint8_t)blocklog;
-	sbp->sb_sectlog = (uint8_t)sectorlog;
-	sbp->sb_agblklog = (uint8_t)libxfs_log2_roundup((unsigned int)agsize);
-	sbp->sb_agblocks = (xfs_agblock_t)agsize;
-	mp->m_blkbb_log = sbp->sb_blocklog - BBSHIFT;
-	mp->m_sectbb_log = sbp->sb_sectlog - BBSHIFT;
-
-	/*
-	 * sb_versionnum, finobt and rmapbt flags must be set before we use
-	 * libxfs_prealloc_blocks().
-	 */
-	sb_set_features(&mp->m_sb, &sb_feat, sectorsize, lsectorsize, dsunit);
-
 
 	if (loginternal) {
 		/*
-- 
2.13.3

[PATCH 39/42] mkfs: factor log size calculations

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 420 +++++++++++++++++++++++++++++---------------------------
 1 file changed, 217 insertions(+), 203 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 3fc485524d99..cbba679ba06d 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -971,91 +971,6 @@ check_device_type(
 }
 
 static void
-fixup_log_stripe_unit(
-	int		lsflag,
-	int		sunit,
-	xfs_rfsblock_t	*logblocks,
-	int		blocklog)
-{
-	uint64_t	tmp_logblocks;
-
-	/*
-	 * Make sure that the log size is a multiple of the stripe unit
-	 */
-	if ((*logblocks % sunit) != 0) {
-		if (!lsflag) {
-			tmp_logblocks = ((*logblocks + (sunit - 1))
-						/ sunit) * sunit;
-			/*
-			 * If the log is too large, round down
-			 * instead of round up
-			 */
-			if ((tmp_logblocks > XFS_MAX_LOG_BLOCKS) ||
-			    ((tmp_logblocks << blocklog) > XFS_MAX_LOG_BYTES)) {
-				tmp_logblocks = (*logblocks / sunit) * sunit;
-			}
-			*logblocks = tmp_logblocks;
-		} else {
-			fprintf(stderr, _("log size %lld is not a multiple "
-					  "of the log stripe unit %d\n"),
-				(long long) *logblocks, sunit);
-			usage();
-		}
-	}
-}
-
-static xfs_fsblock_t
-fixup_internal_log_stripe(
-	xfs_mount_t	*mp,
-	int		lsflag,
-	xfs_fsblock_t	logstart,
-	uint64_t	agsize,
-	int		sunit,
-	xfs_rfsblock_t	*logblocks,
-	int		blocklog,
-	int		*lalign)
-{
-	if ((logstart % sunit) != 0) {
-		logstart = ((logstart + (sunit - 1))/sunit) * sunit;
-		*lalign = 1;
-	}
-
-	fixup_log_stripe_unit(lsflag, sunit, logblocks, blocklog);
-
-	if (*logblocks > agsize - XFS_FSB_TO_AGBNO(mp, logstart)) {
-		fprintf(stderr,
-			_("Due to stripe alignment, the internal log size "
-			"(%lld) is too large.\n"), (long long) *logblocks);
-		fprintf(stderr, _("Must fit within an allocation group.\n"));
-		usage();
-	}
-	return logstart;
-}
-
-void
-validate_log_size(uint64_t logblocks, int blocklog, int min_logblocks)
-{
-	if (logblocks < min_logblocks) {
-		fprintf(stderr,
-	_("log size %lld blocks too small, minimum size is %d blocks\n"),
-			(long long)logblocks, min_logblocks);
-		usage();
-	}
-	if (logblocks > XFS_MAX_LOG_BLOCKS) {
-		fprintf(stderr,
-	_("log size %lld blocks too large, maximum size is %lld blocks\n"),
-			(long long)logblocks, XFS_MAX_LOG_BLOCKS);
-		usage();
-	}
-	if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES) {
-		fprintf(stderr,
-	_("log size %lld bytes too large, maximum size is %lld bytes\n"),
-			(long long)(logblocks << blocklog), XFS_MAX_LOG_BYTES);
-		usage();
-	}
-}
-
-static void
 validate_ag_geometry(
 	int		blocklog,
 	uint64_t	dblocks,
@@ -2885,6 +2800,210 @@ sb_set_features(
 }
 
 /*
+ * Make sure that the log size is a multiple of the stripe unit
+ */
+static void
+align_log_size(
+	struct mkfs_params	*cfg,
+	int			sunit)
+{
+	uint64_t	tmp_logblocks;
+
+	/* nothing to do if it's already aligned. */
+	if ((cfg->logblocks % sunit) == 0)
+		return;
+
+	if (cli_opt_set(&lopts, L_SIZE)) {
+		fprintf(stderr,
+_("log size %lld is not a multiple of the log stripe unit %d\n"),
+			(long long) cfg->logblocks, sunit);
+		usage();
+	}
+
+	tmp_logblocks = ((cfg->logblocks + (sunit - 1)) / sunit) * sunit;
+
+	/* If the log is too large, round down instead of round up */
+	if ((tmp_logblocks > XFS_MAX_LOG_BLOCKS) ||
+	    ((tmp_logblocks << cfg->blocklog) > XFS_MAX_LOG_BYTES)) {
+		tmp_logblocks = (cfg->logblocks / sunit) * sunit;
+	}
+	cfg->logblocks = tmp_logblocks;
+}
+
+/*
+ * Make sure that the internal log is correctly aligned to the specified
+ * stripe unit.
+ */
+static void
+align_internal_log(
+	struct mkfs_params	*cfg,
+	struct xfs_mount	*mp,
+	int			sunit)
+{
+	uint64_t	agspace;
+
+	/* round up log start if necessary */
+	if ((cfg->logstart % sunit) != 0)
+		cfg->logstart = ((cfg->logstart + (sunit - 1)) / sunit) * sunit;
+
+	/* round up/down the log size now */
+	align_log_size(cfg, sunit);
+
+	/* check the aligned log still fits in an AG. */
+	agspace = cfg->agsize - libxfs_prealloc_blocks(mp);
+	if (cfg->logblocks > agspace - XFS_FSB_TO_AGBNO(mp, cfg->logstart)) {
+		fprintf(stderr,
+_("Due to stripe alignment, the internal log size (%lld) is too large.\n"
+  "Must fit within an allocation group.\n"),
+			(long long) cfg->logblocks);
+		usage();
+	}
+}
+
+void
+validate_log_size(uint64_t logblocks, int blocklog, int min_logblocks)
+{
+	if (logblocks < min_logblocks) {
+		fprintf(stderr,
+	_("log size %lld blocks too small, minimum size is %d blocks\n"),
+			(long long)logblocks, min_logblocks);
+		usage();
+	}
+	if (logblocks > XFS_MAX_LOG_BLOCKS) {
+		fprintf(stderr,
+	_("log size %lld blocks too large, maximum size is %lld blocks\n"),
+			(long long)logblocks, XFS_MAX_LOG_BLOCKS);
+		usage();
+	}
+	if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES) {
+		fprintf(stderr,
+	_("log size %lld bytes too large, maximum size is %lld bytes\n"),
+			(long long)(logblocks << blocklog), XFS_MAX_LOG_BYTES);
+		usage();
+	}
+}
+
+static void
+calculate_log_size(
+	struct mkfs_params	*cfg,
+	struct cli_params	*cli,
+	struct xfs_mount	*mp)
+{
+	struct sb_feat_args	*fp = &cfg->sb_feat;
+	struct xfs_sb		*sbp = &mp->m_sb;
+	int			min_logblocks;
+
+	min_logblocks = max_trans_res(sbp->sb_agblocks, fp->crcs_enabled,
+				      fp->dir_version, cfg->sectorlog,
+				      cfg->blocklog, cfg->inodelog,
+				      cfg->dirblocklog, fp->log_version,
+				      cfg->lsunit, fp->finobt, fp->rmapbt,
+				      fp->reflink, fp->inode_align);
+
+	ASSERT(min_logblocks);
+	min_logblocks = MAX(XFS_MIN_LOG_BLOCKS, min_logblocks);
+
+	/* if we have lots of blocks, check against XFS_MIN_LOG_BYTES, too */
+	if (!cli->logsize &&
+	    cfg->dblocks >= (1024*1024*1024) >> cfg->blocklog)
+		min_logblocks = MAX(min_logblocks,
+				    XFS_MIN_LOG_BYTES >> cfg->blocklog);
+
+	/*
+	 * external logs will have a device and size by now, so all we have
+	 * to do is validate it against minimum size and align it.
+	 */
+	if (!cfg->loginternal) {
+		if (min_logblocks > cfg->logblocks) {
+			fprintf(stderr,
+_("external log device %lld too small, must be at least %lld blocks\n"),
+				(long long)cfg->logblocks,
+				(long long)min_logblocks);
+			usage();
+		}
+		cfg->logstart = 0;
+		cfg->logagno = 0;
+		if (cfg->lsunit)
+			align_log_size(cfg, cfg->lsunit);
+
+		validate_log_size(cfg->logblocks, cfg->blocklog, min_logblocks);
+		return;
+	}
+
+	/* internal log - if no size specified, calculate automatically */
+	if (!cfg->logblocks) {
+		if (cfg->dblocks < GIGABYTES(1, cfg->blocklog)) {
+			/* tiny filesystems get minimum sized logs. */
+			cfg->logblocks = min_logblocks;
+		} else if (cfg->dblocks < GIGABYTES(16, cfg->blocklog)) {
+
+			/*
+			 * For small filesystems, we want to use the
+			 * XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
+			 * at all possible, ramping up to 128MB at 256GB.
+			 */
+			cfg->logblocks = MIN(XFS_MIN_LOG_BYTES >> cfg->blocklog,
+					min_logblocks * XFS_DFL_LOG_FACTOR);
+		} else {
+			/*
+			 * With a 2GB max log size, default to maximum size
+			 * at 4TB. This keeps the same ratio from the older
+			 * max log size of 128M at 256GB fs size. IOWs,
+			 * the ratio of fs size to log size is 2048:1.
+			 */
+			cfg->logblocks = (cfg->dblocks << cfg->blocklog) / 2048;
+			cfg->logblocks = cfg->logblocks >> cfg->blocklog;
+		}
+
+		/* Ensure the chosen size meets minimum log size requirements */
+		cfg->logblocks = MAX(min_logblocks, cfg->logblocks);
+
+		/* make sure the log fits wholly within an AG */
+		cfg->logblocks = MIN(cfg->logblocks,
+				     libxfs_alloc_ag_max_usable(mp));
+
+		/* and now clamp the size to the maximum supported size */
+		cfg->logblocks = MIN(cfg->logblocks, XFS_MAX_LOG_BLOCKS);
+		if ((cfg->logblocks << cfg->blocklog) > XFS_MAX_LOG_BYTES)
+			cfg->logblocks = XFS_MAX_LOG_BYTES >> cfg->blocklog;
+
+		validate_log_size(cfg->logblocks, cfg->blocklog, min_logblocks);
+	}
+
+	if (cfg->logblocks > sbp->sb_agblocks - libxfs_prealloc_blocks(mp)) {
+		fprintf(stderr,
+_("internal log size %lld too large, must fit in allocation group\n"),
+			(long long)cfg->logblocks);
+		usage();
+	}
+
+	if (cli_opt_set(&lopts, L_AGNUM)) {
+		if (cli->logagno >= sbp->sb_agcount) {
+			fprintf(stderr,
+_("log ag number %lld too large, must be less than %lld\n"),
+				(long long)cli->logagno,
+				(long long)sbp->sb_agcount);
+			usage();
+		}
+		cfg->logagno = cli->logagno;
+	} else
+		cfg->logagno = (xfs_agnumber_t)(sbp->sb_agcount / 2);
+
+	cfg->logstart = XFS_AGB_TO_FSB(mp, cfg->logagno,
+				       libxfs_prealloc_blocks(mp));
+
+	/*
+	 * Align the logstart at stripe unit boundary.
+	 */
+	if (cfg->lsunit) {
+		align_internal_log(cfg, mp, cfg->lsunit);
+	} else if (cfg->dsunit) {
+		align_internal_log(cfg, mp, cfg->dsunit);
+	}
+	validate_log_size(cfg->logblocks, cfg->blocklog, min_logblocks);
+}
+
+/*
  * Set up mount and superblock with the minimum parameters required for
  * the libxfs macros needed by the log sizing code to run successfully.
  */
@@ -3538,19 +3657,14 @@ main(
 	int			inopblock;
 	int			isize;
 	char			*label = NULL;
-	int			laflag;
-	int			lalign;
 	xfs_agnumber_t		logagno;
 	xfs_rfsblock_t		logblocks;
 	char			*logfile;
 	int			loginternal;
-	char			*logsize;
 	xfs_fsblock_t		logstart;
-	int			lsflag;
 	int			lsectorlog;
 	int			lsectorsize;
 	int			lsunit;
-	int			min_logblocks;
 	xfs_extlen_t		nbmblocks;
 	int			dry_run = 0;
 	int			discard = 1;
@@ -3629,13 +3743,12 @@ main(
 	cli.loginternal = 1;	/* internal by default */
 
 	agsize = dblocks = 0;
-	laflag = lsflag = 0;
 	loginternal = 1;
 	logagno = logblocks = rtblocks = rtextblocks = 0;
 	imaxpct = inodelog = inopblock = isize = 0;
 	dfile = logfile = rtfile = NULL;
-	logsize = protofile = NULL;
-	dsunit = dswidth = lalign = lsunit = 0;
+	protofile = NULL;
+	dsunit = dswidth = lsunit = 0;
 	force_overwrite = 0;
 	worst_freelist = 0;
 	memset(&fsx, 0, sizeof(fsx));
@@ -3652,6 +3765,7 @@ main(
 			break;
 		case 'b':
 		case 'i':
+		case 'l':
 		case 'n':
 		case 'r':
 		case 's':
@@ -3666,14 +3780,6 @@ main(
 			fsx.fsx_extsize = cli.fsx.fsx_extsize;
 			/* end temp don't break code */
 			break;
-		case 'l':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			logagno = cli.logagno;
-			laflag = cli_opt_set(&lopts, L_AGNUM);
-			/* end temp don't break code */
-			break;
 		case 'L':
 			if (strlen(optarg) > sizeof(sbp->sb_fname))
 				illegal(optarg, "L");
@@ -3775,6 +3881,14 @@ main(
 	 */
 	initialise_mount(&cfg, mp, sbp);
 
+	/*
+	 * With the mount set up, we can finally calculate the log size
+	 * constraints and do default size calculations and final validation
+	 */
+	calculate_log_size(&cfg, &cli, mp);
+
+	protostring = setup_proto(protofile);
+
 	/* temp don't break code */
 	sectorsize = cfg.sectorsize;
 	sectorlog = cfg.sectorlog;
@@ -3801,110 +3915,10 @@ main(
 	agsize = cfg.agsize;
 	agcount = cfg.agcount;
 	imaxpct = cfg.imaxpct;
+	logagno = cfg.logagno;
+	logstart = cfg.logstart;
 	/* end temp don't break code */
 
-	min_logblocks = max_trans_res(agsize,
-				   sb_feat.crcs_enabled, sb_feat.dir_version,
-				   sectorlog, blocklog, inodelog, dirblocklog,
-				   sb_feat.log_version, lsunit, sb_feat.finobt,
-				   sb_feat.rmapbt, sb_feat.reflink,
-				   sb_feat.inode_align);
-	ASSERT(min_logblocks);
-	min_logblocks = MAX(XFS_MIN_LOG_BLOCKS, min_logblocks);
-	if (!logsize && dblocks >= (1024*1024*1024) >> blocklog)
-		min_logblocks = MAX(min_logblocks, XFS_MIN_LOG_BYTES>>blocklog);
-	if (loginternal && !logsize) {
-
-		if (dblocks < GIGABYTES(1, blocklog)) {
-			/* tiny filesystems get minimum sized logs. */
-			logblocks = min_logblocks;
-		} else if (dblocks < GIGABYTES(16, blocklog)) {
-
-			/*
-			 * For small filesystems, we want to use the
-			 * XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
-			 * at all possible, ramping up to 128MB at 256GB.
-			 */
-			logblocks = MIN(XFS_MIN_LOG_BYTES >> blocklog,
-					min_logblocks * XFS_DFL_LOG_FACTOR);
-		} else {
-			/*
-			 * With a 2GB max log size, default to maximum size
-			 * at 4TB. This keeps the same ratio from the older
-			 * max log size of 128M at 256GB fs size. IOWs,
-			 * the ratio of fs size to log size is 2048:1.
-			 */
-			logblocks = (dblocks << blocklog) / 2048;
-			logblocks = logblocks >> blocklog;
-		}
-
-		/* Ensure the chosen size meets minimum log size requirements */
-		logblocks = MAX(min_logblocks, logblocks);
-
-		/* make sure the log fits wholly within an AG */
-		if (logblocks >= agsize)
-			logblocks = min_logblocks;
-
-		/* and now clamp the size to the maximum supported size */
-		logblocks = MIN(logblocks, XFS_MAX_LOG_BLOCKS);
-		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES)
-			logblocks = XFS_MAX_LOG_BYTES >> blocklog;
-
-	}
-	validate_log_size(logblocks, blocklog, min_logblocks);
-
-	protostring = setup_proto(protofile);
-
-	if (loginternal) {
-		/*
-		 * Readjust the log size to fit within an AG if it was sized
-		 * automatically.
-		 */
-		if (!logsize) {
-			logblocks = MIN(logblocks,
-					libxfs_alloc_ag_max_usable(mp));
-
-			/* revalidate the log size is valid if we changed it */
-			validate_log_size(logblocks, blocklog, min_logblocks);
-		}
-		if (logblocks > agsize - libxfs_prealloc_blocks(mp)) {
-			fprintf(stderr,
-	_("internal log size %lld too large, must fit in allocation group\n"),
-				(long long)logblocks);
-			usage();
-		}
-
-		if (laflag) {
-			if (logagno >= agcount) {
-				fprintf(stderr,
-		_("log ag number %d too large, must be less than %lld\n"),
-					logagno, (long long)agcount);
-				usage();
-			}
-		} else
-			logagno = (xfs_agnumber_t)(agcount / 2);
-
-		logstart = XFS_AGB_TO_FSB(mp, logagno, libxfs_prealloc_blocks(mp));
-		/*
-		 * Align the logstart at stripe unit boundary.
-		 */
-		if (lsunit) {
-			logstart = fixup_internal_log_stripe(mp,
-					lsflag, logstart, agsize, lsunit,
-					&logblocks, blocklog, &lalign);
-		} else if (dsunit) {
-			logstart = fixup_internal_log_stripe(mp,
-					lsflag, logstart, agsize, dsunit,
-					&logblocks, blocklog, &lalign);
-		}
-	} else {
-		logstart = 0;
-		if (lsunit)
-			fixup_log_stripe_unit(lsflag, lsunit,
-					&logblocks, blocklog);
-	}
-	validate_log_size(logblocks, blocklog, min_logblocks);
-
 	/* Temp support code  to set up mkfs cfg parameters */
 	cfg.blocksize = blocksize;
 	cfg.blocklog = blocklog;
-- 
2.13.3

Re: [PATCH 39/42] mkfs: factor log size calculations

[Dave Chinner]

On Wed, Aug 30, 2017 at 09:50:49AM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Signed-Off-By: Dave Chinner <dchinner@redhat.com>
> ---
....
> +static void
> +align_internal_log(
> +	struct mkfs_params	*cfg,
> +	struct xfs_mount	*mp,
> +	int			sunit)
> +{
> +	uint64_t	agspace;
> +
> +	/* round up log start if necessary */
> +	if ((cfg->logstart % sunit) != 0)
> +		cfg->logstart = ((cfg->logstart + (sunit - 1)) / sunit) * sunit;
> +
> +	/* round up/down the log size now */
> +	align_log_size(cfg, sunit);
> +
> +	/* check the aligned log still fits in an AG. */
> +	agspace = cfg->agsize - libxfs_prealloc_blocks(mp);
> +	if (cfg->logblocks > agspace - XFS_FSB_TO_AGBNO(mp, cfg->logstart)) {

This breaks xfs/250. Should be:

	if (cfg->logblocks > cfg->agsize - XFS_FSB_TO_AGBNO(mp, cfg->logstart)) {

and the agspace variable goes away.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

[PATCH 40/42] mkfs: cleanup redundant temporary code

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Now the factoring is complete, we can remove the remaining temporary
code that was used to isolate the factoring from the rest of the
code.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 185 +++++++++++---------------------------------------------
 1 file changed, 34 insertions(+), 151 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index cbba679ba06d..9d087bc068a7 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -33,8 +33,8 @@ static void unknown(char opt, char *s);
 static int  ispow2(unsigned int i);
 
 /*
- * The configured block and sector sizes are defined as global variables so
- * that they don't need to be passed to functions that require them.
+ * XXX: The configured block and sector sizes are defined as global variables so
+ * that they don't need to be passed to getnum/cvtnum().
  */
 unsigned int		blocksize;
 unsigned int		sectorsize;
@@ -3638,52 +3638,34 @@ main(
 	int			argc,
 	char			**argv)
 {
-	uint64_t		agcount;
 	xfs_agnumber_t		agno;
-	uint64_t		agsize;
-	int			blocklog;
 	xfs_buf_t		*buf;
 	int			c;
-	xfs_rfsblock_t		dblocks;
-	char			*dfile;
-	int			dirblocklog;
-	int			dirblocksize;
-	int			dsunit;
-	int			dswidth;
-	int			force_overwrite;
-	struct fsxattr		fsx;
-	int			imaxpct;
-	int			inodelog;
-	int			inopblock;
-	int			isize;
-	char			*label = NULL;
-	xfs_agnumber_t		logagno;
-	xfs_rfsblock_t		logblocks;
-	char			*logfile;
-	int			loginternal;
-	xfs_fsblock_t		logstart;
-	int			lsectorlog;
-	int			lsectorsize;
-	int			lsunit;
-	xfs_extlen_t		nbmblocks;
+	char			*dfile = NULL;
+	char			*logfile = NULL;
+	char			*rtfile = NULL;
 	int			dry_run = 0;
 	int			discard = 1;
-	char			*protofile;
-	char			*protostring;
+	int			force_overwrite = 0;
 	int			quiet = 0;
-	xfs_rfsblock_t		rtblocks;
-	xfs_extlen_t		rtextblocks;
-	xfs_rtblock_t		rtextents;
-	char			*rtfile;
-	int			sectorlog;
-	uuid_t			uuid;
-	int			worst_freelist;
-	libxfs_init_t		xi;
+	char			*protofile = NULL;
+	char			*protostring = NULL;
+	int			worst_freelist = 0;
+
+	struct libxfs_xinit	xi = {
+		.isdirect = LIBXFS_DIRECT,
+		.isreadonly = LIBXFS_EXCLUSIVELY,
+	};
 	struct xfs_mount	mbuf = {};
 	struct xfs_mount	*mp = &mbuf;
 	struct xfs_sb		*sbp = &mp->m_sb;
-	struct fs_topology	ft;
-	struct sb_feat_args	sb_feat;
+	struct fs_topology	ft = {};
+	struct cli_params	cli = {
+		.xi = &xi,
+		.loginternal = 1,
+	};
+	struct mkfs_params	cfg = {};
+
 	/* build time defaults */
 	struct mkfs_default_params	dft = {
 		.source = "package build definitions",
@@ -3708,10 +3690,6 @@ main(
 			.nortalign = false,
 		},
 	};
-	struct cli_params	cli = {
-		.xi = &xi,
-	};
-	struct mkfs_params	cfg = {};
 
 	platform_uuid_generate(&cli.uuid);
 	progname = basename(argv[0]);
@@ -3735,28 +3713,6 @@ main(
 	memcpy(&cli.sb_feat, &dft.sb_feat, sizeof(cli.sb_feat));
 	memcpy(&cli.fsx, &dft.fsx, sizeof(cli.fsx));
 
-	/*
-	 * Initialise cli parameters that can be set to zero to an appropriate
-	 * value so we can tell if they have been set or or changed from the
-	 * default value.
-	 */
-	cli.loginternal = 1;	/* internal by default */
-
-	agsize = dblocks = 0;
-	loginternal = 1;
-	logagno = logblocks = rtblocks = rtextblocks = 0;
-	imaxpct = inodelog = inopblock = isize = 0;
-	dfile = logfile = rtfile = NULL;
-	protofile = NULL;
-	dsunit = dswidth = lsunit = 0;
-	force_overwrite = 0;
-	worst_freelist = 0;
-	memset(&fsx, 0, sizeof(fsx));
-
-	memset(&xi, 0, sizeof(xi));
-	xi.isdirect = LIBXFS_DIRECT;
-	xi.isreadonly = LIBXFS_EXCLUSIVELY;
-
 	while ((c = getopt(argc, argv, "b:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) {
 		switch (c) {
 		case 'C':
@@ -3764,33 +3720,19 @@ main(
 			force_overwrite = 1;
 			break;
 		case 'b':
+		case 'd':
 		case 'i':
 		case 'l':
+		case 'm':
 		case 'n':
 		case 'r':
 		case 's':
 			parse_subopts(c, optarg, &cli);
 			break;
-		case 'd':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			fsx.fsx_xflags |= cli.fsx.fsx_xflags;
-			fsx.fsx_projid = cli.fsx.fsx_projid;
-			fsx.fsx_extsize = cli.fsx.fsx_extsize;
-			/* end temp don't break code */
-			break;
 		case 'L':
 			if (strlen(optarg) > sizeof(sbp->sb_fname))
 				illegal(optarg, "L");
-			label = optarg;
-			break;
-		case 'm':
-			parse_subopts(c, optarg, &cli);
-
-			/* temp don't break code */
-			platform_uuid_copy(&uuid, &cli.uuid);
-			/* end temp don't break code */
+			cfg.label = optarg;
 			break;
 		case 'N':
 			dry_run = 1;
@@ -3821,9 +3763,7 @@ main(
 	} else
 		dfile = xi.dname;
 
-	/* temp don't break code */
-	sb_feat = cli.sb_feat;
-	/* end temp don't break code */
+	protostring = setup_proto(protofile);
 
 	/*
 	 * Extract as much of the valid config as we can from the CLI input
@@ -3832,6 +3772,14 @@ main(
 	validate_blocksize(&cfg, &cli, &dft);
 	validate_sectorsize(&cfg, &cli, &dft, &ft, dfile, dry_run,
 			    force_overwrite);
+
+	/*
+	 * XXX: we still need to set block size and sector size global variables
+	 * so that getnum/cvtnum works correctly
+	 */
+	blocksize = cfg.blocksize;
+	sectorsize = cfg.sectorsize;
+
 	validate_log_sectorsize(&cfg, &cli, &dft);
 	validate_sb_features(&cfg, &cli);
 
@@ -3887,71 +3835,6 @@ main(
 	 */
 	calculate_log_size(&cfg, &cli, mp);
 
-	protostring = setup_proto(protofile);
-
-	/* temp don't break code */
-	sectorsize = cfg.sectorsize;
-	sectorlog = cfg.sectorlog;
-	blocksize = cfg.blocksize;
-	blocklog = cfg.blocklog;
-	lsectorsize = cfg.lsectorsize;
-	lsectorlog = cfg.lsectorlog;
-	sb_feat = cfg.sb_feat;
-	platform_uuid_copy(&uuid, &cfg.uuid);
-	dirblocksize = cfg.dirblocksize;
-	dirblocklog = cfg.dirblocklog;
-	isize = cfg.inodesize;
-	inodelog = cfg.inodelog;
-	inopblock = cfg.inopblock;
-	dblocks = cfg.dblocks;
-	logblocks = cfg.logblocks;
-	rtblocks = cfg.rtblocks;
-	rtextblocks = cfg.rtextblocks;
-	nbmblocks = cfg.rtbmblocks;
-	rtextents = cfg.rtextents;
-	dsunit = cfg.dsunit;
-	dswidth = cfg.dswidth;
-	lsunit = cfg.lsunit;
-	agsize = cfg.agsize;
-	agcount = cfg.agcount;
-	imaxpct = cfg.imaxpct;
-	logagno = cfg.logagno;
-	logstart = cfg.logstart;
-	/* end temp don't break code */
-
-	/* Temp support code  to set up mkfs cfg parameters */
-	cfg.blocksize = blocksize;
-	cfg.blocklog = blocklog;
-	cfg.sectorsize = sectorsize;
-	cfg.sectorlog = sectorlog;
-	cfg.lsectorsize = lsectorsize;
-	cfg.lsectorlog = lsectorlog;
-	cfg.dirblocksize = dirblocksize;
-	cfg.dirblocklog = dirblocklog;
-	cfg.inodesize = isize;
-	cfg.inodelog = inodelog;
-	cfg.inopblock = inopblock;
-
-	cfg.dblocks = dblocks;
-	cfg.logblocks = logblocks;
-	cfg.rtblocks = rtblocks;
-	cfg.rtextblocks = rtextblocks;
-	cfg.rtextents = rtextents;
-	cfg.rtbmblocks = nbmblocks;
-	cfg.dsunit = dsunit;
-	cfg.dswidth = dswidth;
-	cfg.lsunit = lsunit;
-	cfg.agsize = agsize;
-	cfg.agcount = agcount;
-	cfg.imaxpct = imaxpct;
-	cfg.loginternal = loginternal;
-	cfg.logstart = logstart;
-	cfg.logagno = logagno;
-	cfg.label = label;
-	platform_uuid_copy(&cfg.uuid, &uuid);
-	memcpy(&cfg.sb_feat, &sb_feat, sizeof(sb_feat));
-	/* end temp support code */
-
 	if (!quiet || dry_run) {
 		print_mkfs_cfg(&cfg, dfile, logfile, rtfile);
 		if (dry_run)
@@ -3996,7 +3879,7 @@ main(
 	/*
 	 * Allocate the root inode and anything else in the proto file.
 	 */
-	parse_proto(mp, &fsx, &protostring);
+	parse_proto(mp, &cli.fsx, &protostring);
 
 	/*
 	 * Protect ourselves against possible stupidity
-- 
2.13.3

[PATCH 41/42] mkfs: move error functions

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Move all the error and helper functions like conflict() to the top
of the file so we can get rid of the forward declarations.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 324 +++++++++++++++++++++++++++-----------------------------
 1 file changed, 157 insertions(+), 167 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 9d087bc068a7..c01dc11882d9 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -21,16 +21,6 @@
 #include "xfs_multidisk.h"
 #include "libxcmd.h"
 
-/*
- * Prototypes for internal functions.
- */
-static void conflict(char opt, const char *tab[], int oldidx, int newidx);
-static void illegal(const char *value, const char *opt);
-static __attribute__((noreturn)) void usage (void);
-static __attribute__((noreturn)) void reqval(char opt, const char *tab[], int idx);
-static void respec(char opt, const char *tab[], int idx);
-static void unknown(char opt, char *s);
-static int  ispow2(unsigned int i);
 
 /*
  * XXX: The configured block and sector sizes are defined as global variables so
@@ -896,6 +886,163 @@ struct mkfs_default_params {
  */
 #define WHACK_SIZE (128 * 1024)
 
+static void __attribute__((noreturn))
+usage( void )
+{
+	fprintf(stderr, _("Usage: %s\n\
+/* blocksize */		[-b log=n|size=num]\n\
+/* metadata */		[-m crc=0|1,finobt=0|1,uuid=xxx,rmapbt=0|1,reflink=0|1]\n\
+/* data subvol */	[-d agcount=n,agsize=n,file,name=xxx,size=num,\n\
+			    (sunit=value,swidth=value|su=num,sw=num|noalign),\n\
+			    sectlog=n|sectsize=num\n\
+/* force overwrite */	[-f]\n\
+/* inode size */	[-i log=n|perblock=n|size=num,maxpct=n,attr=0|1|2,\n\
+			    projid32bit=0|1,sparse=0|1]\n\
+/* no discard */	[-K]\n\
+/* log subvol */	[-l agnum=n,internal,size=num,logdev=xxx,version=n\n\
+			    sunit=value|su=num,sectlog=n|sectsize=num,\n\
+			    lazy-count=0|1]\n\
+/* label */		[-L label (maximum 12 characters)]\n\
+/* naming */		[-n log=n|size=num,version=2|ci,ftype=0|1]\n\
+/* no-op info only */	[-N]\n\
+/* prototype file */	[-p fname]\n\
+/* quiet */		[-q]\n\
+/* realtime subvol */	[-r extsize=num,size=num,rtdev=xxx]\n\
+/* sectorsize */	[-s log=n|size=num]\n\
+/* version */		[-V]\n\
+			devicename\n\
+<devicename> is required unless -d name=xxx is given.\n\
+<num> is xxx (bytes), xxxs (sectors), xxxb (fs blocks), xxxk (xxx KiB),\n\
+      xxxm (xxx MiB), xxxg (xxx GiB), xxxt (xxx TiB) or xxxp (xxx PiB).\n\
+<value> is xxx (512 byte blocks).\n"),
+		progname);
+	exit(1);
+}
+
+static void
+conflict(
+	char		opt,
+	const char	*tab[],
+	int		oldidx,
+	int		newidx)
+{
+	fprintf(stderr, _("Cannot specify both -%c %s and -%c %s\n"),
+		opt, tab[oldidx], opt, tab[newidx]);
+	usage();
+}
+
+
+static void
+illegal(
+	const char	*value,
+	const char	*opt)
+{
+	fprintf(stderr, _("Illegal value %s for -%s option\n"), value, opt);
+	usage();
+}
+
+static int
+ispow2(
+	unsigned int	i)
+{
+	return (i & (i - 1)) == 0;
+}
+
+static void __attribute__((noreturn))
+reqval(
+	char		opt,
+	const char	*tab[],
+	int		idx)
+{
+	fprintf(stderr, _("-%c %s option requires a value\n"), opt, tab[idx]);
+	usage();
+}
+
+static void
+respec(
+	char		opt,
+	const char	*tab[],
+	int		idx)
+{
+	fprintf(stderr, "-%c ", opt);
+	if (tab)
+		fprintf(stderr, "%s ", tab[idx]);
+	fprintf(stderr, _("option respecified\n"));
+	usage();
+}
+
+static void
+unknown(
+	char		opt,
+	char		*s)
+{
+	fprintf(stderr, _("unknown option -%c %s\n"), opt, s);
+	usage();
+}
+
+long long
+cvtnum(
+	unsigned int	blksize,
+	unsigned int	sectsize,
+	const char	*s)
+{
+	long long	i;
+	char		*sp;
+	int		c;
+
+	i = strtoll(s, &sp, 0);
+	if (i == 0 && sp == s)
+		return -1LL;
+	if (*sp == '\0')
+		return i;
+
+	if (sp[1] != '\0')
+		return -1LL;
+
+	if (*sp == 'b') {
+		if (!blksize) {
+			fprintf(stderr,
+_("Blocksize must be provided prior to using 'b' suffix.\n"));
+			usage();
+		} else {
+			return i * blksize;
+		}
+	}
+	if (*sp == 's') {
+		if (!sectsize) {
+			fprintf(stderr,
+_("Sectorsize must be specified prior to using 's' suffix.\n"));
+			usage();
+		} else {
+			return i * sectsize;
+		}
+	}
+
+	c = tolower(*sp);
+	switch (c) {
+	case 'e':
+		i *= 1024LL;
+		/* fall through */
+	case 'p':
+		i *= 1024LL;
+		/* fall through */
+	case 't':
+		i *= 1024LL;
+		/* fall through */
+	case 'g':
+		i *= 1024LL;
+		/* fall through */
+	case 'm':
+		i *= 1024LL;
+		/* fall through */
+	case 'k':
+		return i * 1024LL;
+	default:
+		break;
+	}
+	return -1LL;
+}
+
 static void
 check_device_type(
 	const char	*name,
@@ -3920,160 +4067,3 @@ main(
 
 	return 0;
 }
-
-static void
-conflict(
-	char		opt,
-	const char	*tab[],
-	int		oldidx,
-	int		newidx)
-{
-	fprintf(stderr, _("Cannot specify both -%c %s and -%c %s\n"),
-		opt, tab[oldidx], opt, tab[newidx]);
-	usage();
-}
-
-
-static void
-illegal(
-	const char	*value,
-	const char	*opt)
-{
-	fprintf(stderr, _("Illegal value %s for -%s option\n"), value, opt);
-	usage();
-}
-
-static int
-ispow2(
-	unsigned int	i)
-{
-	return (i & (i - 1)) == 0;
-}
-
-static void __attribute__((noreturn))
-reqval(
-	char		opt,
-	const char	*tab[],
-	int		idx)
-{
-	fprintf(stderr, _("-%c %s option requires a value\n"), opt, tab[idx]);
-	usage();
-}
-
-static void
-respec(
-	char		opt,
-	const char	*tab[],
-	int		idx)
-{
-	fprintf(stderr, "-%c ", opt);
-	if (tab)
-		fprintf(stderr, "%s ", tab[idx]);
-	fprintf(stderr, _("option respecified\n"));
-	usage();
-}
-
-static void
-unknown(
-	char		opt,
-	char		*s)
-{
-	fprintf(stderr, _("unknown option -%c %s\n"), opt, s);
-	usage();
-}
-
-long long
-cvtnum(
-	unsigned int	blksize,
-	unsigned int	sectsize,
-	const char	*s)
-{
-	long long	i;
-	char		*sp;
-	int		c;
-
-	i = strtoll(s, &sp, 0);
-	if (i == 0 && sp == s)
-		return -1LL;
-	if (*sp == '\0')
-		return i;
-
-	if (sp[1] != '\0')
-		return -1LL;
-
-	if (*sp == 'b') {
-		if (!blksize) {
-			fprintf(stderr,
-_("Blocksize must be provided prior to using 'b' suffix.\n"));
-			usage();
-		} else {
-			return i * blksize;
-		}
-	}
-	if (*sp == 's') {
-		if (!sectsize) {
-			fprintf(stderr,
-_("Sectorsize must be specified prior to using 's' suffix.\n"));
-			usage();
-		} else {
-			return i * sectsize;
-		}
-	}
-
-	c = tolower(*sp);
-	switch (c) {
-	case 'e':
-		i *= 1024LL;
-		/* fall through */
-	case 'p':
-		i *= 1024LL;
-		/* fall through */
-	case 't':
-		i *= 1024LL;
-		/* fall through */
-	case 'g':
-		i *= 1024LL;
-		/* fall through */
-	case 'm':
-		i *= 1024LL;
-		/* fall through */
-	case 'k':
-		return i * 1024LL;
-	default:
-		break;
-	}
-	return -1LL;
-}
-
-static void __attribute__((noreturn))
-usage( void )
-{
-	fprintf(stderr, _("Usage: %s\n\
-/* blocksize */		[-b log=n|size=num]\n\
-/* metadata */		[-m crc=0|1,finobt=0|1,uuid=xxx,rmapbt=0|1,reflink=0|1]\n\
-/* data subvol */	[-d agcount=n,agsize=n,file,name=xxx,size=num,\n\
-			    (sunit=value,swidth=value|su=num,sw=num|noalign),\n\
-			    sectlog=n|sectsize=num\n\
-/* force overwrite */	[-f]\n\
-/* inode size */	[-i log=n|perblock=n|size=num,maxpct=n,attr=0|1|2,\n\
-			    projid32bit=0|1,sparse=0|1]\n\
-/* no discard */	[-K]\n\
-/* log subvol */	[-l agnum=n,internal,size=num,logdev=xxx,version=n\n\
-			    sunit=value|su=num,sectlog=n|sectsize=num,\n\
-			    lazy-count=0|1]\n\
-/* label */		[-L label (maximum 12 characters)]\n\
-/* naming */		[-n log=n|size=num,version=2|ci,ftype=0|1]\n\
-/* no-op info only */	[-N]\n\
-/* prototype file */	[-p fname]\n\
-/* quiet */		[-q]\n\
-/* realtime subvol */	[-r extsize=num,size=num,rtdev=xxx]\n\
-/* sectorsize */	[-s log=n|size=num]\n\
-/* version */		[-V]\n\
-			devicename\n\
-<devicename> is required unless -d name=xxx is given.\n\
-<num> is xxx (bytes), xxxs (sectors), xxxb (fs blocks), xxxk (xxx KiB),\n\
-      xxxm (xxx MiB), xxxg (xxx GiB), xxxt (xxx TiB) or xxxp (xxx PiB).\n\
-<value> is xxx (512 byte blocks).\n"),
-		progname);
-	exit(1);
-}
-- 
2.13.3

[PATCH 42/42] mkfs: tidy up definitions

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Put all the internal macro definitions at the top of the file so
they can be easily found and remove teh ones that aren't used
anymore.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 41 +++++++++++++++++------------------------
 1 file changed, 17 insertions(+), 24 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index c01dc11882d9..bec12d473099 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -22,6 +22,23 @@
 #include "libxcmd.h"
 
 
+
+#define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
+#define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
+#define MEGABYTES(count, blog)	((uint64_t)(count) << (20 - (blog)))
+
+/*
+ * Use this macro before we have superblock and mount structure to
+ * convert from basic blocks to filesystem blocks.
+ */
+#define	DTOBT(d, bl)	((xfs_rfsblock_t)((d) >> ((bl) - BBSHIFT)))
+
+/*
+ * amount (in bytes) we zero at the beginning and end of the device to
+ * remove traces of other filesystems, raid superblocks, etc.
+ */
+#define WHACK_SIZE (128 * 1024)
+
 /*
  * XXX: The configured block and sector sizes are defined as global variables so
  * that they don't need to be passed to getnum/cvtnum().
@@ -862,30 +879,6 @@ struct mkfs_default_params {
 	struct fsxattr		fsx;
 };
 
-#define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
-#define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
-#define MEGABYTES(count, blog)	((uint64_t)(count) << (20 - (blog)))
-
-/*
- * Use this macro before we have superblock and mount structure
- */
-#define	DTOBT(d, bl)	((xfs_rfsblock_t)((d) >> ((bl) - BBSHIFT)))
-
-/*
- * Use this for block reservations needed for mkfs's conditions
- * (basically no fragmentation).
- */
-#define	MKFS_BLOCKRES_INODE	\
-	((uint)(mp->m_ialloc_blks + (mp->m_in_maxlevels - 1)))
-#define	MKFS_BLOCKRES(rb)	\
-	((uint)(MKFS_BLOCKRES_INODE + XFS_DA_NODE_MAXDEPTH + \
-	(XFS_BM_MAXLEVELS(mp, XFS_DATA_FORK) - 1) + (rb)))
-
-/* amount (in bytes) we zero at the beginning and end of the device to
- * remove traces of other filesystems, raid superblocks, etc.
- */
-#define WHACK_SIZE (128 * 1024)
-
 static void __attribute__((noreturn))
 usage( void )
 {
-- 
2.13.3

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Darrick J. Wong]

On Wed, Aug 30, 2017 at 09:50:10AM +1000, Dave Chinner wrote:
> Everyone who tries to modify mkfs quickly learns that it is a pile
> of spaghetti, the only difference in opinion is whether it is a
> steaming, cold or rotten pile. This patchset attempts to untangle
> the ball of pasta and turn it into a set of clear, obvious
> operations that lead to a filesystem being formatted correctly.
> 
> The patch series is really in three parts, splitting the code up
> into roughly three modules. The first part introduces a mkfs
> parameters structure and factors the on-disk formatting code to use
> only information in that structure. The second part introduces a
> command line input structure and factors the input parsing to use
> it. This requires a bunch of temporary code to keep the rest of
> the code working. The third part is factoring the input validation
> and geometry calculation code to use the input structure and put
> the output into the mkfs parameter structure and to remove all the
> temporary support code.
> 
> The result is three modules - input parsing, validation+calculations
> and formatting - with well defined data flow between them. This also
> paves the way to supporting config files to set defaults via a
> separate (new) module. The overall data flow now looks like this:
> 
> Build defaults --\
>                   ---> mkfs_default_params -> CLI -> mkfs_params
> config file -----/
> 
> It is not worth spending a lot of time reviewing the temporary code
> that is added - it gets removed before the end of the series. No
> attempt has been made to ensure that mkfs works 100% correctly after
> each patch is applied - the only guarantee is that it will build
> cleanly. It /should/ work if a bisect lands in the middle of the
> series, but trying to exhaustively test each patch is OK would take
> more effort than it is worth. As such, testing has only been
> performed on the whole series.
> 
> The new output from mkfs to indicate where it has sourced the
> defaults from causes xfstests to have conniptions. This requires
> some updates to the mkfs output filters that are already in place
> but it is a fairly trivial update. Test xfs/191 has a couple of new
> failures, but that is because the new code now correctly parses
> things like agsize so that block and sector size based
> specifications work with default mkfs values. This will require test
> updates.
> 
> Future work will be to split the xfs_mkfs.c file into a file per
> module (i.e. seperate files for CLI parsing, mkfs formating,
> validation+calculation and, finally, one for config file support),
> but otherwise the majority of the factoring work is now complete.
> 
> Comments, flames, etc all welcome.

A couple of questions right off the bat--

I've noticed that max_trans_res in mkfs.xfs constructs a fake xfs_mount
so that it can use libxfs_log_calc_minimum_size to calculate the minimum
log size.  We've had at least a couple of mkfs bug reports where mkfs
fails to model enough of the proposed filesystem to calculate the
correct transaction reservation sizes and therefore the minimum log
size.  I've not looked into all 42 patches, but is there a way to
eliminate the duplicate xfs_mount constructions (there's another one
around line 2690 in xfs_mkfs.c) because I keep screwing them up? :)

Second: is it possible to refactor the geometry printing code in mkfs
and xfs_info to use the same function?  They're awfully similar, but not
/quite/ the same.

(Ofc I can already see the objections -- "some script could be parsing
the output and will break!" <cough>xfstests<cough>...)

--D

> 
> -Dave.
> 
> Overall Diffstat:
>  include/libxfs.h |    2 
>  mkfs/xfs_mkfs.c  | 4615 ++++++++++++++++++++++++++++++-------------------------
>  2 files changed, 2578 insertions(+), 2039 deletions(-)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Tue, Aug 29, 2017 at 06:23:30PM -0700, Darrick J. Wong wrote:
> On Wed, Aug 30, 2017 at 09:50:10AM +1000, Dave Chinner wrote:
> > Everyone who tries to modify mkfs quickly learns that it is a pile
> > of spaghetti, the only difference in opinion is whether it is a
> > steaming, cold or rotten pile. This patchset attempts to untangle
> > the ball of pasta and turn it into a set of clear, obvious
> > operations that lead to a filesystem being formatted correctly.
> > 
> > The patch series is really in three parts, splitting the code up
> > into roughly three modules. The first part introduces a mkfs
> > parameters structure and factors the on-disk formatting code to use
> > only information in that structure. The second part introduces a
> > command line input structure and factors the input parsing to use
> > it. This requires a bunch of temporary code to keep the rest of
> > the code working. The third part is factoring the input validation
> > and geometry calculation code to use the input structure and put
> > the output into the mkfs parameter structure and to remove all the
> > temporary support code.
> > 
> > The result is three modules - input parsing, validation+calculations
> > and formatting - with well defined data flow between them. This also
> > paves the way to supporting config files to set defaults via a
> > separate (new) module. The overall data flow now looks like this:
> > 
> > Build defaults --\
> >                   ---> mkfs_default_params -> CLI -> mkfs_params
> > config file -----/
> > 
> > It is not worth spending a lot of time reviewing the temporary code
> > that is added - it gets removed before the end of the series. No
> > attempt has been made to ensure that mkfs works 100% correctly after
> > each patch is applied - the only guarantee is that it will build
> > cleanly. It /should/ work if a bisect lands in the middle of the
> > series, but trying to exhaustively test each patch is OK would take
> > more effort than it is worth. As such, testing has only been
> > performed on the whole series.
> > 
> > The new output from mkfs to indicate where it has sourced the
> > defaults from causes xfstests to have conniptions. This requires
> > some updates to the mkfs output filters that are already in place
> > but it is a fairly trivial update. Test xfs/191 has a couple of new
> > failures, but that is because the new code now correctly parses
> > things like agsize so that block and sector size based
> > specifications work with default mkfs values. This will require test
> > updates.
> > 
> > Future work will be to split the xfs_mkfs.c file into a file per
> > module (i.e. seperate files for CLI parsing, mkfs formating,
> > validation+calculation and, finally, one for config file support),
> > but otherwise the majority of the factoring work is now complete.
> > 
> > Comments, flames, etc all welcome.
> 
> A couple of questions right off the bat--
> 
> I've noticed that max_trans_res in mkfs.xfs constructs a fake xfs_mount
> so that it can use libxfs_log_calc_minimum_size to calculate the minimum
> log size.  We've had at least a couple of mkfs bug reports where mkfs
> fails to model enough of the proposed filesystem to calculate the
> correct transaction reservation sizes and therefore the minimum log
> size.  I've not looked into all 42 patches, but is there a way to
> eliminate the duplicate xfs_mount constructions (there's another one
> around line 2690 in xfs_mkfs.c) because I keep screwing them up? :)

First of all, that's not a problem I'm trying to solve in this
patchset. What I'm trying to do is make it clear what order the
operations are done in mkfs so that we can quickly and clearly
analyse whether such things are possible.

So, if you look at the final code in main(), we now have:

        /*
         * Set up the basic superblock parameters now so that we can use
         * the geometry information we've already validated in libxfs
         * provided functions to determine on-disk format information.
         */
        initialise_mount(&cfg, mp, sbp);

        /*
         * With the mount set up, we can finally calculate the log size
         * constraints and do default size calculations and final validation
         */
        calculate_log_size(&cfg, &cli, mp);

        if (!quiet || dry_run) {
                print_mkfs_cfg(&cfg, dfile, logfile, rtfile);
                if (dry_run)
                        exit(0);
        }

        /*
         * Finish setting up the superblock state ready for formatting.
         */
        setup_superblock(&cfg, mp, sbp);

and the max_trans_res() call is inside calculate_log_size(). IOWs,
the superblock setup is split around parts of calculating the log
size as it was before the factoring, but the log size calculations
are no longer split into two parts around initialising the
superblock.

IOWs, it is clear now is that the only thing that actually
changes between the mount/sb being initialised and the final setup
is a couple of log parameters. Hence there's no real reason why we
can't set all of the superblock up in initialise_mount(), clean up
max_trans_res() to use the mkfs provided mount/sb and then update
the superblock log fields in finish_sb_setup() (aka
setup_superblock())....

So, yeah, it's now clear that this can be done and it's relatively
easy to do. It most certainly wasn't easily to see or do before the
factoring; this is a clear demonstration of why I've wanted mkfs to
be factored for years....

> Second: is it possible to refactor the geometry printing code in mkfs
> and xfs_info to use the same function?  They're awfully similar, but not
> /quite/ the same.

Sure, as long as it's the growfs output that changes. But, again,
that's outside the scope of this patchset....

> (Ofc I can already see the objections -- "some script could be parsing
> the output and will break!" <cough>xfstests<cough>...)

mkfs, yes. growfs, not so much. :P

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Luis R. Rodriguez]

On Wed, Aug 30, 2017 at 09:50:10AM +1000, Dave Chinner wrote:
> Everyone who tries to modify mkfs quickly learns that it is a pile
> of spaghetti, the only difference in opinion is whether it is a
> steaming, cold or rotten pile. This patchset attempts to untangle
> the ball of pasta and turn it into a set of clear, obvious
> operations that lead to a filesystem being formatted correctly.

Yay.

> The patch series is really in three parts, splitting the code up
> into roughly three modules.

Any reason you ended up with 3 instead of 4 as originally envisioned?

> The first part introduces a mkfs
> parameters structure and factors the on-disk formatting code to use
> only information in that structure.

Is there no validation on defaults prior to moving on to a next step?
I ask as config changes will modify defaults.

> The second part introduces a
> command line input structure and factors the input parsing to use
> it. This requires a bunch of temporary code to keep the rest of
> the code working. The third part is factoring the input validation
> and geometry calculation code to use the input structure and put
> the output into the mkfs parameter structure and to remove all the
> temporary support code.
> 
> The result is three modules - input parsing, validation+calculations
> and formatting - with well defined data flow between them. This also
> paves the way to supporting config files to set defaults via a
> separate (new) module. The overall data flow now looks like this:
> 
> Build defaults --\
>                   ---> mkfs_default_params -> CLI -> mkfs_params
> config file -----/
> 
> It is not worth spending a lot of time reviewing the temporary code
> that is added - it gets removed before the end of the series. No
> attempt has been made to ensure that mkfs works 100% correctly after
> each patch is applied - the only guarantee is that it will build
> cleanly. It /should/ work if a bisect lands in the middle of the
> series, but trying to exhaustively test each patch is OK would take
> more effort than it is worth. As such, testing has only been
> performed on the whole series.

Amen.

However this still begs the question of what tests we should run
prior and after the full set, and if we had any missing test what
we should add, or if we've considered that.

> The new output from mkfs to indicate where it has sourced the
> defaults from causes xfstests to have conniptions. This requires
> some updates to the mkfs output filters that are already in place
> but it is a fairly trivial update. Test xfs/191 has a couple of new
> failures, but that is because the new code now correctly parses
> things like agsize so that block and sector size based
> specifications work with default mkfs values. This will require test
> updates.
> 
> Future work will be to split the xfs_mkfs.c file into a file per
> module (i.e. seperate files for CLI parsing, mkfs formating,
> validation+calculation and, 

Great! I was hoping for this!

> finally, one for config file support),
> but otherwise the majority of the factoring work is now complete.
> 
> Comments, flames, etc all welcome.

Just one thing, got a git tree I can use? I honestly can't be bothered
reviewing the delta in between, I just want to move on with life. Thanks
for cleaning up the manure pile buttress.

  Luis

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Wed, Aug 30, 2017 at 06:16:35AM +0200, Luis R. Rodriguez wrote:
> On Wed, Aug 30, 2017 at 09:50:10AM +1000, Dave Chinner wrote:
> > Everyone who tries to modify mkfs quickly learns that it is a pile
> > of spaghetti, the only difference in opinion is whether it is a
> > steaming, cold or rotten pile. This patchset attempts to untangle
> > the ball of pasta and turn it into a set of clear, obvious
> > operations that lead to a filesystem being formatted correctly.
> 
> Yay.
> 
> > The patch series is really in three parts, splitting the code up
> > into roughly three modules.
> 
> Any reason you ended up with 3 instead of 4 as originally envisioned?

Because the 4th module - config file support - doesn't exist yet.
This patchset only supplies the hook point for that module to be
interface with the existing code.

> > The first part introduces a mkfs
> > parameters structure and factors the on-disk formatting code to use
> > only information in that structure.
> 
> Is there no validation on defaults prior to moving on to a next step?
> I ask as config changes will modify defaults.

Never has been - it's been assumed that the built in defaults are
sane and valid because they've been set by the XFS developers. This
patchset is not changing functionality, defaults, or how defaults
are used, anything config files might require is outside the scope
of this work.

> > The second part introduces a
> > command line input structure and factors the input parsing to use
> > it. This requires a bunch of temporary code to keep the rest of
> > the code working. The third part is factoring the input validation
> > and geometry calculation code to use the input structure and put
> > the output into the mkfs parameter structure and to remove all the
> > temporary support code.
> > 
> > The result is three modules - input parsing, validation+calculations
> > and formatting - with well defined data flow between them. This also
> > paves the way to supporting config files to set defaults via a
> > separate (new) module. The overall data flow now looks like this:
> > 
> > Build defaults --\
> >                   ---> mkfs_default_params -> CLI -> mkfs_params
> > config file -----/
> > 
> > It is not worth spending a lot of time reviewing the temporary code
> > that is added - it gets removed before the end of the series. No
> > attempt has been made to ensure that mkfs works 100% correctly after
> > each patch is applied - the only guarantee is that it will build
> > cleanly. It /should/ work if a bisect lands in the middle of the
> > series, but trying to exhaustively test each patch is OK would take
> > more effort than it is worth. As such, testing has only been
> > performed on the whole series.
> 
> Amen.
> 
> However this still begs the question of what tests we should run
> prior and after the full set, and if we had any missing test what
> we should add, or if we've considered that.

We have nothing that actually tests that the on-disk filesystem
format values match what is specified on the command line, so
testing/validation has all been completely manual. I'm not sure we
can automate it, given the math needed to check that all the fields
in the superblock are correct and sane for arbitrary mkfs
configurations on arbitrary block devices.

In the end I basically ran some configs with an old mkfs, dumped the
superblock with xfs_db for each config, then did the same with the
new mkfs. I then diffed the output to see if anything came out
differently. I fixed all the things I found that were different.  I
ran the checks on 4k and 512 byte sector devices, just to make sure
nothing went wrong with using probed device sector sizes by default.
Then I diffed logprint output for different log options (e.g. sector
size, lsunit, etc) to make sure everything was correct, etc. I
mounted each filesystem, ran repair on them, etc so that both the
kernel and repair code could validate the format, too (the kernel
found the first log formatting bug via CRC errors).  And, of course,
I ran xfstests over a variety of configs - different block sizes,
rmap, DAX, etc.

Maybe you can come up with a way of automating this, but for a
one-off piece of work that affects a point-in-time snapshot of mkfs
functionality, I'm not sure it's worth the effort to try to make a
generic test to do this sort of thing.

> > finally, one for config file support),
> > but otherwise the majority of the factoring work is now complete.
> > 
> > Comments, flames, etc all welcome.
> 
> Just one thing, got a git tree I can use? I honestly can't be bothered
> reviewing the delta in between, I just want to move on with life. Thanks
> for cleaning up the manure pile buttress.

Nope, not right now. Tag all the patches, save them to an mbox
file, run 'git-am <mbox-file>' to apply them all. Takes all of 20s
to do with mutt....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Luis R. Rodriguez]

On Wed, Aug 30, 2017 at 03:44:20PM +1000, Dave Chinner wrote:
> On Wed, Aug 30, 2017 at 06:16:35AM +0200, Luis R. Rodriguez wrote:
> > On Wed, Aug 30, 2017 at 09:50:10AM +1000, Dave Chinner wrote:
> > > Everyone who tries to modify mkfs quickly learns that it is a pile
> > > of spaghetti, the only difference in opinion is whether it is a
> > > steaming, cold or rotten pile. This patchset attempts to untangle
> > > the ball of pasta and turn it into a set of clear, obvious
> > > operations that lead to a filesystem being formatted correctly.
> > 
> > Yay.
> > 
> > > The patch series is really in three parts, splitting the code up
> > > into roughly three modules.
> > 
> > Any reason you ended up with 3 instead of 4 as originally envisioned?
> 
> Because the 4th module - config file support - doesn't exist yet.

To be fair you had itemized before:

  1) Settings default - struct mkfs_default_params
  2) CLI parsing - struct cli_params
  3) Validation + calculation - struct mkfs_params
  4) On disk formatting

> > > The result is three modules - input parsing, validation+calculations
> > > and formatting - with well defined data flow between them.

And you instead ended up with:

  1) Input parsing - struct mkfs_default_params and I guess struct cli_params
  2) Validation + calculation - struct mkfs_params
  3) On disk formatting

Hence my question.

> Maybe you can come up with a way of automating this, but for a
> one-off piece of work that affects a point-in-time snapshot of mkfs
> functionality, I'm not sure it's worth the effort to try to make a
> generic test to do this sort of thing.

In Dave we trust!

> > > finally, one for config file support),
> > > but otherwise the majority of the factoring work is now complete.
> > > 
> > > Comments, flames, etc all welcome.
> > 
> > Just one thing, got a git tree I can use? I honestly can't be bothered
> > reviewing the delta in between, I just want to move on with life. Thanks
> > for cleaning up the manure pile buttress.
> 
> Nope, not right now. Tag all the patches, save them to an mbox
> file, run 'git-am <mbox-file>' to apply them all. Takes all of 20s
> to do with mutt....

Turns out mutt re-orders tagged messages in what I think may be the order you
got them in so the order on the input output filename may differ from the patch
order intent. Even when I manually sort them and apply them, the patches failed
on both origin/master and origin/for-next, so I must be doing something wrong
or using an incorrect branch or commit ID. What branch and commit ID should I
use?

It also seems I didn't get patch #20 in my inbox, could you resend?

On the latest origin/for-next applying your patches fails at patch #3, you need
git am -3, but then after that it fails at patch #9 even with git am -3.

  Luis

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Thu, Aug 31, 2017 at 12:10:56AM +0200, Luis R. Rodriguez wrote:
> On Wed, Aug 30, 2017 at 03:44:20PM +1000, Dave Chinner wrote:
> > On Wed, Aug 30, 2017 at 06:16:35AM +0200, Luis R. Rodriguez wrote:
> > > On Wed, Aug 30, 2017 at 09:50:10AM +1000, Dave Chinner wrote:
> > > > Everyone who tries to modify mkfs quickly learns that it is a pile
> > > > of spaghetti, the only difference in opinion is whether it is a
> > > > steaming, cold or rotten pile. This patchset attempts to untangle
> > > > the ball of pasta and turn it into a set of clear, obvious
> > > > operations that lead to a filesystem being formatted correctly.
> > > 
> > > Yay.
> > > 
> > > > The patch series is really in three parts, splitting the code up
> > > > into roughly three modules.
> > > 
> > > Any reason you ended up with 3 instead of 4 as originally envisioned?
> > 
> > Because the 4th module - config file support - doesn't exist yet.
> 
> To be fair you had itemized before:
> 
>   1) Settings default - struct mkfs_default_params

As I said: this module doesn't exist yet, so it's output struct
mkfs_default_params is statically defined (i.e. built in defaults)
to provide input to the next module. Hence there are only three
modules in this patchset:

>   2) CLI parsing - struct cli_params
>   3) Validation + calculation - struct mkfs_params
>   4) On disk formatting

[....]

> > Maybe you can come up with a way of automating this, but for a
> > one-off piece of work that affects a point-in-time snapshot of mkfs
> > functionality, I'm not sure it's worth the effort to try to make a
> > generic test to do this sort of thing.
> 
> In Dave we trust!

No, definitely don't do that.  Don't trust a damn thing I do - I'm
full of dangerous ideas, I write shit code and should be kept on a
short leash at all times.....

> > > > finally, one for config file support),
> > > > but otherwise the majority of the factoring work is now complete.
> > > > 
> > > > Comments, flames, etc all welcome.
> > > 
> > > Just one thing, got a git tree I can use? I honestly can't be bothered
> > > reviewing the delta in between, I just want to move on with life. Thanks
> > > for cleaning up the manure pile buttress.
> > 
> > Nope, not right now. Tag all the patches, save them to an mbox
> > file, run 'git-am <mbox-file>' to apply them all. Takes all of 20s
> > to do with mutt....
> 
> Turns out mutt re-orders tagged messages

It does? I've never come across that. I always tag the entire thread
they've always come out in the correct sent order for me. And I've
been doing that for many, many years....

> in what I think may be the order you got them in so the order on
> the input output filename may differ from the patch order intent.
> Even when I manually sort them and apply them, the patches failed
> on both origin/master and origin/for-next, so I must be doing
> something wrong or using an incorrect branch or commit ID. What
> branch and commit ID should I use?

Applies to:

$ glo -n 1 origin/for-next
3540b418ba48 xfs_db: btdump should avoid eval for push and pop of cursor
$

I forgot to update the branch before my last pass over the
patchset...

> It also seems I didn't get patch #20 in my inbox, could you
> resend?

Probably best in one-off cases like this is to grab it from the
archive - the whole patchset made it back to me from the list, so it
should be in the archive. If it's not, then we've got a more general
list delivery problem rather than it just being a problem somewhere
in your mail delivery path...


Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Luis R. Rodriguez]

On Thu, Aug 31, 2017 at 09:22:16AM +1000, Dave Chinner wrote:
> On Thu, Aug 31, 2017 at 12:10:56AM +0200, Luis R. Rodriguez wrote:
> > On Wed, Aug 30, 2017 at 03:44:20PM +1000, Dave Chinner wrote:
> > > Maybe you can come up with a way of automating this, but for a
> > > one-off piece of work that affects a point-in-time snapshot of mkfs
> > > functionality, I'm not sure it's worth the effort to try to make a
> > > generic test to do this sort of thing.
> > 
> > In Dave we trust!
> 
> No, definitely don't do that.  Don't trust a damn thing I do - I'm
> full of dangerous ideas, I write shit code and should be kept on a
> short leash at all times.....

Well its great you have a job then!

> > > > > finally, one for config file support),
> > > > > but otherwise the majority of the factoring work is now complete.
> > > > > 
> > > > > Comments, flames, etc all welcome.
> > > > 
> > > > Just one thing, got a git tree I can use? I honestly can't be bothered
> > > > reviewing the delta in between, I just want to move on with life. Thanks
> > > > for cleaning up the manure pile buttress.
> > > 
> > > Nope, not right now. Tag all the patches, save them to an mbox
> > > file, run 'git-am <mbox-file>' to apply them all. Takes all of 20s
> > > to do with mutt....
> > 
> > Turns out mutt re-orders tagged messages
> 
> It does? I've never come across that. I always tag the entire thread
> they've always come out in the correct sent order for me. And I've
> been doing that for many, many years....

Yeah the order also comes out different on the thread on mutt for me,
it turns out I have:

set sort_aux=last-date-received

And you need:

set sort_aux=date

which is the default.

> > in what I think may be the order you got them in so the order on
> > the input output filename may differ from the patch order intent.
> > Even when I manually sort them and apply them, the patches failed
> > on both origin/master and origin/for-next, so I must be doing
> > something wrong or using an incorrect branch or commit ID. What
> > branch and commit ID should I use?
> 
> Applies to:
> 
> $ glo -n 1 origin/for-next
> 3540b418ba48 xfs_db: btdump should avoid eval for push and pop of cursor
> $
> 
> I forgot to update the branch before my last pass over the
> patchset...
> 
> > It also seems I didn't get patch #20 in my inbox, could you
> > resend?
> 
> Probably best in one-off cases like this is to grab it from the
> archive - the whole patchset made it back to me from the list, so it
> should be in the archive. If it's not, then we've got a more general
> list delivery problem rather than it just being a problem somewhere
> in your mail delivery path...

One-off, I definitely don't have that patch but it is in the archive.

For those lazy:

https://gitlab.com/mcgrof/xfsprogs-dev/tree/20170830-dave-mkfs

  Luis

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Jan Tulak]

On Wed, Aug 30, 2017 at 7:44 AM, Dave Chinner <david@fromorbit.com> wrote:
> On Wed, Aug 30, 2017 at 06:16:35AM +0200, Luis R. Rodriguez wrote:
>>
>> However this still begs the question of what tests we should run
>> prior and after the full set, and if we had any missing test what
>> we should add, or if we've considered that.
>
> We have nothing that actually tests that the on-disk filesystem
> format values match what is specified on the command line, so
> testing/validation has all been completely manual. I'm not sure we
> can automate it, given the math needed to check that all the fields
> in the superblock are correct and sane for arbitrary mkfs
> configurations on arbitrary block devices.
>
> In the end I basically ran some configs with an old mkfs, dumped the
> superblock with xfs_db for each config, then did the same with the
> new mkfs. I then diffed the output to see if anything came out
> differently. I fixed all the things I found that were different.  I
> ran the checks on 4k and 512 byte sector devices, just to make sure
> nothing went wrong with using probed device sector sizes by default.
> Then I diffed logprint output for different log options (e.g. sector
> size, lsunit, etc) to make sure everything was correct, etc. I
> mounted each filesystem, ran repair on them, etc so that both the
> kernel and repair code could validate the format, too (the kernel
> found the first log formatting bug via CRC errors).  And, of course,
> I ran xfstests over a variety of configs - different block sizes,
> rmap, DAX, etc.
>
> Maybe you can come up with a way of automating this, but for a
> one-off piece of work that affects a point-in-time snapshot of mkfs
> functionality, I'm not sure it's worth the effort to try to make a
> generic test to do this sort of thing.
>

I actually tried to automate this kind of testing. Something similar
to xfs/191, but with testing if the physically written fs is valid and
expected. But I didn't find a way that would work with xfstests
without hacking it to use some other version of mkfs.xfs manually, and
without doing all the math craziness that would be so error-prone...

Anyway, I'm on the sea until 10th September. I might be able to
continue my review and write something tomorrow (Friday) in the
morning before I depart, but more likely I will keep all my comments
until after I return, and see what changes in the meantime. But so
far, it looks like a good job, Dave. :-)

Cheers,
Jan

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Martin Steigerwald]

Dave Chinner - 30.08.17, 09:50:
> Everyone who tries to modify mkfs quickly learns that it is a pile
> of spaghetti, the only difference in opinion is whether it is a
> steaming, cold or rotten pile. This patchset attempts to untangle
> the ball of pasta and turn it into a set of clear, obvious
> operations that lead to a filesystem being formatted correctly.

I am thankful that all the XFS filesystems I created up to now *somehow* 
worked. By pure accident maybe :)

Thanks for the commit description which gave me a quite a bit of hearty 
laughing.

Thanks
-- 
Martin

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Chandan Rajendra]

On Wednesday, August 30, 2017 5:20:10 AM IST Dave Chinner wrote:
> Everyone who tries to modify mkfs quickly learns that it is a pile
> of spaghetti, the only difference in opinion is whether it is a
> steaming, cold or rotten pile. This patchset attempts to untangle
> the ball of pasta and turn it into a set of clear, obvious
> operations that lead to a filesystem being formatted correctly.
> 
> The patch series is really in three parts, splitting the code up
> into roughly three modules. The first part introduces a mkfs
> parameters structure and factors the on-disk formatting code to use
> only information in that structure. The second part introduces a
> command line input structure and factors the input parsing to use
> it. This requires a bunch of temporary code to keep the rest of
> the code working. The third part is factoring the input validation
> and geometry calculation code to use the input structure and put
> the output into the mkfs parameter structure and to remove all the
> temporary support code.
> 
> The result is three modules - input parsing, validation+calculations
> and formatting - with well defined data flow between them. This also
> paves the way to supporting config files to set defaults via a
> separate (new) module. The overall data flow now looks like this:
> 
> Build defaults --\
>                   ---> mkfs_default_params -> CLI -> mkfs_params
> config file -----/
> 
> It is not worth spending a lot of time reviewing the temporary code
> that is added - it gets removed before the end of the series. No
> attempt has been made to ensure that mkfs works 100% correctly after
> each patch is applied - the only guarantee is that it will build
> cleanly. It /should/ work if a bisect lands in the middle of the
> series, but trying to exhaustively test each patch is OK would take
> more effort than it is worth. As such, testing has only been
> performed on the whole series.
> 
> The new output from mkfs to indicate where it has sourced the
> defaults from causes xfstests to have conniptions. This requires
> some updates to the mkfs output filters that are already in place
> but it is a fairly trivial update. Test xfs/191 has a couple of new
> failures, but that is because the new code now correctly parses
> things like agsize so that block and sector size based
> specifications work with default mkfs values. This will require test
> updates.
> 
> Future work will be to split the xfs_mkfs.c file into a file per
> module (i.e. seperate files for CLI parsing, mkfs formating,
> validation+calculation and, finally, one for config file support),
> but otherwise the majority of the factoring work is now complete.
> 
> Comments, flames, etc all welcome.
Hi Dave,

I am trying to test the "mkfs refactor" patchset on a ppc64 machine. But "git
am ~/mkfs-xfs-rewrite/\[PATCH\ 03_42\]\ mkfs_introduce\ a\ structure\ to\
hold\ CLI\ options.mbox" fails to apply cleanly on top of xfsprogs/master
(with patches 01 and 02 already applied) branch. Can you please let me know
the id of the topmost commit on which the "mkfs refactor" patches are applied? 

-- 
chandan

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Eric Sandeen]

On 9/4/17 7:31 AM, Chandan Rajendra wrote:
> On Wednesday, August 30, 2017 5:20:10 AM IST Dave Chinner wrote:
>> Everyone who tries to modify mkfs quickly learns that it is a pile
>> of spaghetti, the only difference in opinion is whether it is a
>> steaming, cold or rotten pile. This patchset attempts to untangle
>> the ball of pasta and turn it into a set of clear, obvious
>> operations that lead to a filesystem being formatted correctly.
>>
>> The patch series is really in three parts, splitting the code up
>> into roughly three modules. The first part introduces a mkfs
>> parameters structure and factors the on-disk formatting code to use
>> only information in that structure. The second part introduces a
>> command line input structure and factors the input parsing to use
>> it. This requires a bunch of temporary code to keep the rest of
>> the code working. The third part is factoring the input validation
>> and geometry calculation code to use the input structure and put
>> the output into the mkfs parameter structure and to remove all the
>> temporary support code.
>>
>> The result is three modules - input parsing, validation+calculations
>> and formatting - with well defined data flow between them. This also
>> paves the way to supporting config files to set defaults via a
>> separate (new) module. The overall data flow now looks like this:
>>
>> Build defaults --\
>>                   ---> mkfs_default_params -> CLI -> mkfs_params
>> config file -----/
>>
>> It is not worth spending a lot of time reviewing the temporary code
>> that is added - it gets removed before the end of the series. No
>> attempt has been made to ensure that mkfs works 100% correctly after
>> each patch is applied - the only guarantee is that it will build
>> cleanly. It /should/ work if a bisect lands in the middle of the
>> series, but trying to exhaustively test each patch is OK would take
>> more effort than it is worth. As such, testing has only been
>> performed on the whole series.
>>
>> The new output from mkfs to indicate where it has sourced the
>> defaults from causes xfstests to have conniptions. This requires
>> some updates to the mkfs output filters that are already in place
>> but it is a fairly trivial update. Test xfs/191 has a couple of new
>> failures, but that is because the new code now correctly parses
>> things like agsize so that block and sector size based
>> specifications work with default mkfs values. This will require test
>> updates.
>>
>> Future work will be to split the xfs_mkfs.c file into a file per
>> module (i.e. seperate files for CLI parsing, mkfs formating,
>> validation+calculation and, finally, one for config file support),
>> but otherwise the majority of the factoring work is now complete.
>>
>> Comments, flames, etc all welcome.
> Hi Dave,
> 
> I am trying to test the "mkfs refactor" patchset on a ppc64 machine. But "git
> am ~/mkfs-xfs-rewrite/\[PATCH\ 03_42\]\ mkfs_introduce\ a\ structure\ to\
> hold\ CLI\ options.mbox" fails to apply cleanly on top of xfsprogs/master
> (with patches 01 and 02 already applied) branch. Can you please let me know
> the id of the topmost commit on which the "mkfs refactor" patches are applied? 

I think that
02c3e10 mkfs: add missing translation
is the problem, sorry.

-Eric

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Mon, Sep 04, 2017 at 06:01:02PM +0530, Chandan Rajendra wrote:
> On Wednesday, August 30, 2017 5:20:10 AM IST Dave Chinner wrote:
> > Everyone who tries to modify mkfs quickly learns that it is a pile
> > of spaghetti, the only difference in opinion is whether it is a
> > steaming, cold or rotten pile. This patchset attempts to untangle
> > the ball of pasta and turn it into a set of clear, obvious
> > operations that lead to a filesystem being formatted correctly.
> > 
> > The patch series is really in three parts, splitting the code up
> > into roughly three modules. The first part introduces a mkfs
> > parameters structure and factors the on-disk formatting code to use
> > only information in that structure. The second part introduces a
> > command line input structure and factors the input parsing to use
> > it. This requires a bunch of temporary code to keep the rest of
> > the code working. The third part is factoring the input validation
> > and geometry calculation code to use the input structure and put
> > the output into the mkfs parameter structure and to remove all the
> > temporary support code.
> > 
> > The result is three modules - input parsing, validation+calculations
> > and formatting - with well defined data flow between them. This also
> > paves the way to supporting config files to set defaults via a
> > separate (new) module. The overall data flow now looks like this:
> > 
> > Build defaults --\
> >                   ---> mkfs_default_params -> CLI -> mkfs_params
> > config file -----/
> > 
> > It is not worth spending a lot of time reviewing the temporary code
> > that is added - it gets removed before the end of the series. No
> > attempt has been made to ensure that mkfs works 100% correctly after
> > each patch is applied - the only guarantee is that it will build
> > cleanly. It /should/ work if a bisect lands in the middle of the
> > series, but trying to exhaustively test each patch is OK would take
> > more effort than it is worth. As such, testing has only been
> > performed on the whole series.
> > 
> > The new output from mkfs to indicate where it has sourced the
> > defaults from causes xfstests to have conniptions. This requires
> > some updates to the mkfs output filters that are already in place
> > but it is a fairly trivial update. Test xfs/191 has a couple of new
> > failures, but that is because the new code now correctly parses
> > things like agsize so that block and sector size based
> > specifications work with default mkfs values. This will require test
> > updates.
> > 
> > Future work will be to split the xfs_mkfs.c file into a file per
> > module (i.e. seperate files for CLI parsing, mkfs formating,
> > validation+calculation and, finally, one for config file support),
> > but otherwise the majority of the factoring work is now complete.
> > 
> > Comments, flames, etc all welcome.
> Hi Dave,
> 
> I am trying to test the "mkfs refactor" patchset on a ppc64 machine. But "git
> am ~/mkfs-xfs-rewrite/\[PATCH\ 03_42\]\ mkfs_introduce\ a\ structure\ to\
> hold\ CLI\ options.mbox" fails to apply cleanly on top of xfsprogs/master
> (with patches 01 and 02 already applied) branch. Can you please let me know
> the id of the topmost commit on which the "mkfs refactor" patches are applied? 

As I've already posted in the thread:

	Applies to:

	$ glo -n 1 origin/for-next
	3540b418ba48 xfs_db: btdump should avoid eval for push and pop of cursor
	$

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Chandan Rajendra]

On Wednesday, August 30, 2017 5:20:10 AM IST Dave Chinner wrote:
> Everyone who tries to modify mkfs quickly learns that it is a pile
> of spaghetti, the only difference in opinion is whether it is a
> steaming, cold or rotten pile. This patchset attempts to untangle
> the ball of pasta and turn it into a set of clear, obvious
> operations that lead to a filesystem being formatted correctly.
> 
> The patch series is really in three parts, splitting the code up
> into roughly three modules. The first part introduces a mkfs
> parameters structure and factors the on-disk formatting code to use
> only information in that structure. The second part introduces a
> command line input structure and factors the input parsing to use
> it. This requires a bunch of temporary code to keep the rest of
> the code working. The third part is factoring the input validation
> and geometry calculation code to use the input structure and put
> the output into the mkfs parameter structure and to remove all the
> temporary support code.
> 
> The result is three modules - input parsing, validation+calculations
> and formatting - with well defined data flow between them. This also
> paves the way to supporting config files to set defaults via a
> separate (new) module. The overall data flow now looks like this:
> 
> Build defaults --\
>                   ---> mkfs_default_params -> CLI -> mkfs_params
> config file -----/
> 
> It is not worth spending a lot of time reviewing the temporary code
> that is added - it gets removed before the end of the series. No
> attempt has been made to ensure that mkfs works 100% correctly after
> each patch is applied - the only guarantee is that it will build
> cleanly. It /should/ work if a bisect lands in the middle of the
> series, but trying to exhaustively test each patch is OK would take
> more effort than it is worth. As such, testing has only been
> performed on the whole series.
> 
> The new output from mkfs to indicate where it has sourced the
> defaults from causes xfstests to have conniptions. This requires
> some updates to the mkfs output filters that are already in place
> but it is a fairly trivial update. Test xfs/191 has a couple of new
> failures, but that is because the new code now correctly parses
> things like agsize so that block and sector size based
> specifications work with default mkfs values. This will require test
> updates.
> 
> Future work will be to split the xfs_mkfs.c file into a file per
> module (i.e. seperate files for CLI parsing, mkfs formating,
> validation+calculation and, finally, one for config file support),
> but otherwise the majority of the factoring work is now complete.
> 
> Comments, flames, etc all welcome.
> 

Hi Dave,

For 4k blocksized xfs filesystem on ppc64, xfs/058 fails with mkfs refactor 
patchset applied.

[root@localhost xfstests-dev]# ./check xfs/058
FSTYP         -- xfs (debug)
PLATFORM      -- Linux/ppc64 localhost 4.13.0-next-20170905-00001-g9730219
MKFS_OPTIONS  -- -f -bsize=4096 /dev/loop1
MOUNT_OPTIONS -- /dev/loop1 /mnt/btrfs-xfstest-scratch

xfs/058 2s ... - output mismatch (see /root/repos/xfstests-dev/results//xfs/058.out.bad)
    --- tests/xfs/058.out       2017-09-03 02:23:13.432063287 -0500
    +++ /root/repos/xfstests-dev/results//xfs/058.out.bad       2017-09-07 05:07:49.850183977 -0500
    @@ -13,16 +13,16 @@
     fdblocks = 9223372036854775807
     Test verb middlebit
     Allowing fuzz of corrupted data with good CRC
    -fdblocks = 9223372034707292159
    +fdblocks = 9223372036854775807
     Test verb lastbit
     Allowing fuzz of corrupted data with good CRC
    ...
    (Run 'diff -u tests/xfs/058.out /root/repos/xfstests-dev/results//xfs/058.out.bad'  to see the entire diff)
Ran: xfs/058
Failures: xfs/058
Failed 1 of 1 tests

I will debug this further and let you know about the results.

Also, xfs/206 fails because the line 
"Default configuration sourced from package build definitions" isn't getting 
filtered out.

-- 
chandan

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Thu, Sep 07, 2017 at 04:01:57PM +0530, Chandan Rajendra wrote:
> On Wednesday, August 30, 2017 5:20:10 AM IST Dave Chinner wrote:
> > Everyone who tries to modify mkfs quickly learns that it is a pile
> > of spaghetti, the only difference in opinion is whether it is a
> > steaming, cold or rotten pile. This patchset attempts to untangle
> > the ball of pasta and turn it into a set of clear, obvious
> > operations that lead to a filesystem being formatted correctly.
> > 
> > The patch series is really in three parts, splitting the code up
> > into roughly three modules. The first part introduces a mkfs
> > parameters structure and factors the on-disk formatting code to use
> > only information in that structure. The second part introduces a
> > command line input structure and factors the input parsing to use
> > it. This requires a bunch of temporary code to keep the rest of
> > the code working. The third part is factoring the input validation
> > and geometry calculation code to use the input structure and put
> > the output into the mkfs parameter structure and to remove all the
> > temporary support code.
> > 
> > The result is three modules - input parsing, validation+calculations
> > and formatting - with well defined data flow between them. This also
> > paves the way to supporting config files to set defaults via a
> > separate (new) module. The overall data flow now looks like this:
> > 
> > Build defaults --\
> >                   ---> mkfs_default_params -> CLI -> mkfs_params
> > config file -----/
> > 
> > It is not worth spending a lot of time reviewing the temporary code
> > that is added - it gets removed before the end of the series. No
> > attempt has been made to ensure that mkfs works 100% correctly after
> > each patch is applied - the only guarantee is that it will build
> > cleanly. It /should/ work if a bisect lands in the middle of the
> > series, but trying to exhaustively test each patch is OK would take
> > more effort than it is worth. As such, testing has only been
> > performed on the whole series.
> > 
> > The new output from mkfs to indicate where it has sourced the
> > defaults from causes xfstests to have conniptions. This requires
> > some updates to the mkfs output filters that are already in place
> > but it is a fairly trivial update. Test xfs/191 has a couple of new
> > failures, but that is because the new code now correctly parses
> > things like agsize so that block and sector size based
> > specifications work with default mkfs values. This will require test
> > updates.
> > 
> > Future work will be to split the xfs_mkfs.c file into a file per
> > module (i.e. seperate files for CLI parsing, mkfs formating,
> > validation+calculation and, finally, one for config file support),
> > but otherwise the majority of the factoring work is now complete.
> > 
> > Comments, flames, etc all welcome.
> > 
> 
> Hi Dave,
> 
> For 4k blocksized xfs filesystem on ppc64, xfs/058 fails with mkfs refactor 
> patchset applied.
> 
> [root@localhost xfstests-dev]# ./check xfs/058
> FSTYP         -- xfs (debug)
> PLATFORM      -- Linux/ppc64 localhost 4.13.0-next-20170905-00001-g9730219
> MKFS_OPTIONS  -- -f -bsize=4096 /dev/loop1
> MOUNT_OPTIONS -- /dev/loop1 /mnt/btrfs-xfstest-scratch
> 
> xfs/058 2s ... - output mismatch (see /root/repos/xfstests-dev/results//xfs/058.out.bad)
>     --- tests/xfs/058.out       2017-09-03 02:23:13.432063287 -0500
>     +++ /root/repos/xfstests-dev/results//xfs/058.out.bad       2017-09-07 05:07:49.850183977 -0500
>     @@ -13,16 +13,16 @@
>      fdblocks = 9223372036854775807
>      Test verb middlebit
>      Allowing fuzz of corrupted data with good CRC
>     -fdblocks = 9223372034707292159
>     +fdblocks = 9223372036854775807
>      Test verb lastbit
>      Allowing fuzz of corrupted data with good CRC
>     ...
>     (Run 'diff -u tests/xfs/058.out /root/repos/xfstests-dev/results//xfs/058.out.bad'  to see the entire diff)
> Ran: xfs/058
> Failures: xfs/058
> Failed 1 of 1 tests

That's a bug in xfs_db, not my patchset. Fixed by commit
592c10154f99 ("xfs_db: bit fuzzing should read the right bit when
flipping").

> Also, xfs/206 fails because the line 
> "Default configuration sourced from package build definitions" isn't getting 
> filtered out.

Lots of tests fail that way. I have a local xfstests patch that
filters these out that I haven't sent out yet. Attached below.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

---
 common/xfs    | 3 ++-
 tests/xfs/206 | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/common/xfs b/common/xfs
index 2c34e0887747..3969582866cf 100644
--- a/common/xfs
+++ b/common/xfs
@@ -81,7 +81,8 @@ _scratch_mkfs_xfs()
 {
 	local mkfs_cmd="`_scratch_mkfs_xfs_opts`"
 	local mkfs_filter="sed -e '/less than device physical sector/d' \
-			       -e '/switching to logical sector/d'"
+			       -e '/switching to logical sector/d' \
+			       -e '/Default configuration/d'"
 	local tmp=`mktemp -u`
 	local mkfs_status
 
diff --git a/tests/xfs/206 b/tests/xfs/206
index 70997e3fe83e..01782b7b93a9 100755
--- a/tests/xfs/206
+++ b/tests/xfs/206
@@ -84,7 +84,8 @@ mkfs_filter()
 	    -e "s/\(sectsz\)\(=[0-9]* *\)/\1=512   /" \
 	    -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \
 	    -e "s/, lazy-count=[0-9]//" \
-	    -e "/.*crc=/d"
+	    -e "/.*crc=/d" \
+	    -e "/^Default configuration/d"
 }
 
 # mkfs slightly smaller than that, small log for speed.

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Chandan Rajendra]

On Wednesday, August 30, 2017 5:20:10 AM IST Dave Chinner wrote:
> Everyone who tries to modify mkfs quickly learns that it is a pile
> of spaghetti, the only difference in opinion is whether it is a
> steaming, cold or rotten pile. This patchset attempts to untangle
> the ball of pasta and turn it into a set of clear, obvious
> operations that lead to a filesystem being formatted correctly.
> 
> The patch series is really in three parts, splitting the code up
> into roughly three modules. The first part introduces a mkfs
> parameters structure and factors the on-disk formatting code to use
> only information in that structure. The second part introduces a
> command line input structure and factors the input parsing to use
> it. This requires a bunch of temporary code to keep the rest of
> the code working. The third part is factoring the input validation
> and geometry calculation code to use the input structure and put
> the output into the mkfs parameter structure and to remove all the
> temporary support code.
> 
> The result is three modules - input parsing, validation+calculations
> and formatting - with well defined data flow between them. This also
> paves the way to supporting config files to set defaults via a
> separate (new) module. The overall data flow now looks like this:
> 
> Build defaults --\
>                   ---> mkfs_default_params -> CLI -> mkfs_params
> config file -----/
> 
> It is not worth spending a lot of time reviewing the temporary code
> that is added - it gets removed before the end of the series. No
> attempt has been made to ensure that mkfs works 100% correctly after
> each patch is applied - the only guarantee is that it will build
> cleanly. It /should/ work if a bisect lands in the middle of the
> series, but trying to exhaustively test each patch is OK would take
> more effort than it is worth. As such, testing has only been
> performed on the whole series.
> 
> The new output from mkfs to indicate where it has sourced the
> defaults from causes xfstests to have conniptions. This requires
> some updates to the mkfs output filters that are already in place
> but it is a fairly trivial update. Test xfs/191 has a couple of new
> failures, but that is because the new code now correctly parses
> things like agsize so that block and sector size based
> specifications work with default mkfs values. This will require test
> updates.
> 
> Future work will be to split the xfs_mkfs.c file into a file per
> module (i.e. seperate files for CLI parsing, mkfs formating,
> validation+calculation and, finally, one for config file support),
> but otherwise the majority of the factoring work is now complete.
> 
> Comments, flames, etc all welcome.
> 

I executed xfstests on ppc64 with 4k, 32k & 64k block sizes and did not notice
any regressions.

Tested-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>


-- 
chandan

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Jan Tulak]

On Wed, Aug 30, 2017 at 1:50 AM, Dave Chinner <david@fromorbit.com> wrote:
>
> Future work will be to split the xfs_mkfs.c file into a file per
> module (i.e. seperate files for CLI parsing, mkfs formating,
> validation+calculation and, finally, one for config file support),
> but otherwise the majority of the factoring work is now complete.
>
> Comments, flames, etc all welcome.
>

In my review, I didn't saw any issues. It works, it certainly looks
way better than what I was trying to do, and I didn't notice any
regression, although my testing was not as thorough as Chandan's. It
seems that everybody likes these changes, so is there anything that
prevents it to be merged once xfstests are updated as well?

And thinking about the future, I wonder if there is any point in
making the suboption tables into one hierarchical like I wanted
before. The main reason for my attempt was to allow cross-option
conflicts and the mentioned changes also added conditional conflicts.
But now with validate_sb_features() containing almost all these
conflicts, I'm not sure if it would help or if it would rather make it
more complex and obscure.

Cheers,
Jan

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Fri, Sep 15, 2017 at 11:42:44AM +0200, Jan Tulak wrote:
> On Wed, Aug 30, 2017 at 1:50 AM, Dave Chinner <david@fromorbit.com> wrote:
> >
> > Future work will be to split the xfs_mkfs.c file into a file per
> > module (i.e. seperate files for CLI parsing, mkfs formating,
> > validation+calculation and, finally, one for config file support),
> > but otherwise the majority of the factoring work is now complete.
> >
> > Comments, flames, etc all welcome.
> >
> 
> In my review, I didn't saw any issues. It works, it certainly looks
> way better than what I was trying to do, and I didn't notice any
> regression, although my testing was not as thorough as Chandan's. It
> seems that everybody likes these changes, so is there anything that
> prevents it to be merged once xfstests are updated as well?

Yeah, the bugs I found last week to do with log sizingi and log
stripe units. I'll fix them and send out the xfstests mods next
week. Then I'll wait for Eric to release 4.13.0 and send him a
pull request....

> And thinking about the future, I wonder if there is any point in
> making the suboption tables into one hierarchical like I wanted
> before. The main reason for my attempt was to allow cross-option
> conflicts and the mentioned changes also added conditional
> conflicts.  But now with validate_sb_features() containing almost
> all these conflicts, I'm not sure if it would help or if it would
> rather make it more complex and obscure.

Don't know. But let's focus on getting the features in that people
want first, then worry about further cleanups...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Eric Sandeen]

Ok more review, sorry this is so dispersed.

[PATCH 14/42] mkfs: factor printing of mkfs config

Oh good.  :)

[PATCH 15/42] mkfs: factor in memory superblock setup

Heh, I read that as factor in vs factor out ;)

This seems to change logic slightly, is it intentional?

from:

-	if (sb_feat.inode_align) {
-		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
-		if (sb_feat.crcs_enabled)
-			cluster_size *= isize / XFS_DINODE_MIN_SIZE;
-		sbp->sb_inoalignmt = cluster_size >> blocklog;
-		sb_feat.inode_align = sbp->sb_inoalignmt != 0;
-	}

to

+	if (cfg->sb_feat.inode_align) {
+		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
+		if (cfg->sb_feat.crcs_enabled)
+			cluster_size *= cfg->inodesize / XFS_DINODE_MIN_SIZE;
+		sbp->sb_inoalignmt = cluster_size >> cfg->blocklog;
+	}

it loses the test for inalignmt being 0 and turning off inode_align...
if you deemed that impossible, a commit log comment would be helpful.
If not, maybe it's a bug.  :)  The test & set goes back to the mists
of time, so I'm not sure of its purpose.

Re: [PATCH 00/42] mkfs: factor the crap out of the code

[Dave Chinner]

On Mon, Oct 23, 2017 at 10:00:26PM -0500, Eric Sandeen wrote:
> Ok more review, sorry this is so dispersed.
> 
> [PATCH 14/42] mkfs: factor printing of mkfs config
> 
> Oh good.  :)
> 
> [PATCH 15/42] mkfs: factor in memory superblock setup
> 
> Heh, I read that as factor in vs factor out ;)
> 
> This seems to change logic slightly, is it intentional?
> 
> from:
> 
> -	if (sb_feat.inode_align) {
> -		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
> -		if (sb_feat.crcs_enabled)
> -			cluster_size *= isize / XFS_DINODE_MIN_SIZE;
> -		sbp->sb_inoalignmt = cluster_size >> blocklog;
> -		sb_feat.inode_align = sbp->sb_inoalignmt != 0;
> -	}
> 
> to
> 
> +	if (cfg->sb_feat.inode_align) {
> +		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
> +		if (cfg->sb_feat.crcs_enabled)
> +			cluster_size *= cfg->inodesize / XFS_DINODE_MIN_SIZE;
> +		sbp->sb_inoalignmt = cluster_size >> cfg->blocklog;
> +	}
> 
> it loses the test for inalignmt being 0 and turning off inode_align...

Ok, so there's a few things here. When you look at the kernel and
it decides on inode alignment, it does:

int
xfs_ialloc_cluster_alignment(
        struct xfs_mount        *mp)
{
        if (xfs_sb_version_hasalign(&mp->m_sb) &&
            mp->m_sb.sb_inoalignmt >= xfs_icluster_size_fsb(mp))
                return mp->m_sb.sb_inoalignmt;
        return 1;
}

xfs_set_inoalignment() has a similar check before using
sb_inoalignmt to create the alignment mask, so the kernel doesn't
actually care that the alignment is 0 because it doesn't align
inodes if it's less than the cluster buffer size....

The tricky one is sparse inodes, because it requires sb_inoalignmt
to be equal to the size of the chunk. The kernel does this at mount
time:

        /*
         * Full inode chunks must be aligned to inode chunk size when
         * sparse inodes are enabled to support the sparse chunk
         * allocation algorithm and prevent overlapping inode records.
         */
        if (xfs_sb_version_hassparseinodes(sbp)) {
                uint32_t        align;

                align = XFS_INODES_PER_CHUNK * sbp->sb_inodesize
                                >> sbp->sb_blocklog;
                if (sbp->sb_inoalignmt != align) {
                        xfs_warn(mp,
"Inode block alignment (%u) must match chunk size (%u) for sparse inodes.",
                                 sbp->sb_inoalignmt, align);
                        return -EINVAL;
                }
        }


If we take the case of 64k block size, we have blocklog = 16, and so:

	align	= (64 * 512) >> 16
		= 2^15 >> 16
		= 0

So sparse inodes requires sbp->sb_inoalignmt == 0 when a 64k block
size is in use. It doesn't care about the align feature bit, though.

What I just noticed is this in xfs_align_sparse_ino():

        agbno = XFS_AGINO_TO_AGBNO(mp, *startino);
        mod = agbno % mp->m_sb.sb_inoalignmt;
        if (!mod)
                return;

if sb_inoalignmt is zero - and it will be for 64k block sizes and
inode sizes <= 512 bytes - then the result here should be a divide
by zero error. The only reason we haven't seen this is that we
can't allocate a sparse inode cluster on 64k block size filesystems
when the inode size <= 1k.....

So AFAICT, the inode alignment feature bit just doesn't matter when
the alignment is set to zero because the block size is larger than
the inode cluster size.

> if you deemed that impossible, a commit log comment would be helpful.
> If not, maybe it's a bug.  :)  The test & set goes back to the mists
> of time, so I'm not sure of its purpose.

Oh, and of course, I forgot the simple reason: we always use aligned
inodes on v5 filesystems, and we've defaulted to setting it on v4 fs
for donkey's years, so AFAIC there's no reason it should ever not be
set because the kernel has to handle alignment < cluster size sanely
anyway....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com