[PATCH 0/2] xfsprogs: mkfs fixes

[PATCH 0/2] xfsprogs: mkfs fixes

[Dave Chinner]

Hi folks,

The first patch is an update to the patch I sent a while back to
reduce the default log size of small filesystems back to new to the
3.1.x sizes. I've incorporated the changes Brian an I discussed, and
it makes all the xfstets that expect specific log sizes for certain
sized filesystems pass again.

The second patch fixes an oversight when adding dirent ftype support
to mkfs - the protofile code that creates files in the filesystem at
mkfs time does nto set the ftype value in the directory structures,
so xfs_repair complains on a freshly made filesystem that the ftypes
aren't correct.

Cheers,

Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 1/2] mkfs: default log size for small filesystems too large

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Recent changes to the log size scaling have resulted in using the
default size multiplier for the log size even on small filesystems.
Commit 88cd79b ("xfs: Add xfs_log_rlimit.c") changed the calculation
of the maximum transaction size that the kernel would issues and
that significantly increased the minimum size of the default log.
As such the size of the log on small filesystems was typically
larger than the prefious default, even though the previous default
was still larger than the minimum needed.

Rework the default log size calculation such that it will use the
original log size default if it is larger than the minimum log size
required, and only use a larger log if the configuration of the
filesystem requires it.

This is especially obvious in xfs/216, where the default log size is
10MB all the way up to 16GB filesystems. The current mkfs selects a
log size of 50MB for the same size filesystems and this is
unnecessarily large.

Return the scaling of the log size for small filesystems to
something similar to what xfs/216 expects.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 mkfs/xfs_mkfs.c | 48 ++++++++++++++++++++++++++++--------------------
 1 file changed, 28 insertions(+), 20 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index d82128c..f7cf394 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -2366,32 +2366,40 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	} else if (!loginternal && !xi.logdev) {
 		logblocks = 0;
 	} else if (loginternal && !logsize) {
-		/*
-		 * With a 2GB max log size, default to maximum size
-		 * at 4TB. This keeps the same ratio from the older
-		 * max log size of 128M at 256GB fs size. IOWs,
-		 * the ratio of fs size to log size is 2048:1.
-		 */
-		logblocks = (dblocks << blocklog) / 2048;
-		logblocks = logblocks >> blocklog;
-		logblocks = MAX(min_logblocks, logblocks);
 
-		/*
-		 * If the default log size doesn't fit in the AG size, use the
-		 * minimum log size instead. This ensures small filesystems
-		 * don't use excessive amounts of space for the log.
-		 */
-		if (min_logblocks * XFS_DFL_LOG_FACTOR >= agsize) {
+		if (dblocks < GIGABYTES(1, blocklog)) {
+			/* tiny filesystems get minimum sized logs. */
 			logblocks = min_logblocks;
+		} else if (dblocks < GIGABYTES(16, blocklog)) {
+
+			/*
+			 * For small filesystems, we want to use the
+			 * XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
+			 * at all possible, ramping up to 128MB at 256GB.
+			 */
+			logblocks = MIN(XFS_MIN_LOG_BYTES >> blocklog,
+					min_logblocks * XFS_DFL_LOG_FACTOR);
 		} else {
-			logblocks = MAX(logblocks,
-				MAX(XFS_DFL_LOG_SIZE,
-					min_logblocks * XFS_DFL_LOG_FACTOR));
+			/*
+			 * With a 2GB max log size, default to maximum size
+			 * at 4TB. This keeps the same ratio from the older
+			 * max log size of 128M at 256GB fs size. IOWs,
+			 * the ratio of fs size to log size is 2048:1.
+			 */
+			logblocks = (dblocks << blocklog) / 2048;
+			logblocks = logblocks >> blocklog;
+			logblocks = MAX(min_logblocks, logblocks);
 		}
+
+		/* make sure the log fits wholly within an AG */
+		if (logblocks >= agsize)
+			logblocks = min_logblocks;
+
+		/* and now clamp the size to the maximum supported size */
 		logblocks = MIN(logblocks, XFS_MAX_LOG_BLOCKS);
-		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES) {
+		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES)
 			logblocks = XFS_MAX_LOG_BYTES >> blocklog;
-		}
+
 	}
 	validate_log_size(logblocks, blocklog, min_logblocks);
 
-- 
1.8.4.rc3

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 2/2] mkfs: proto file creation does not set ftype correctly

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Hence running xfs_repair on a ftype enable filesystem that has
contents created by a proto file will throw warnings on mismatched
ftype entries and correct them. xfs/031 fails due to this problem.

Fix it by settin gup the xname structure with the correct type
fields.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 mkfs/proto.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/mkfs/proto.c b/mkfs/proto.c
index 4cc0df6..4d3680d 100644
--- a/mkfs/proto.c
+++ b/mkfs/proto.c
@@ -438,6 +438,7 @@ parseproto(
 	creds.cr_gid = (int)getnum(pp);
 	xname.name = (uchar_t *)name;
 	xname.len = name ? strlen(name) : 0;
+	xname.type = 0;
 	tp = libxfs_trans_alloc(mp, 0);
 	flags = XFS_ILOG_CORE;
 	xfs_bmap_init(&flist, &first);
@@ -453,6 +454,7 @@ parseproto(
 		if (buf)
 			free(buf);
 		libxfs_trans_ijoin(tp, pip, 0);
+		xname.type = XFS_DIR3_FT_REG_FILE;
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
 		libxfs_trans_ihold(tp, pip);
 		break;
@@ -469,6 +471,7 @@ parseproto(
 
 		libxfs_trans_ijoin(tp, pip, 0);
 
+		xname.type = XFS_DIR3_FT_REG_FILE;
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
 		libxfs_trans_ihold(tp, pip);
 		libxfs_trans_log_inode(tp, ip, flags);
@@ -490,6 +493,7 @@ parseproto(
 			fail(_("Inode allocation failed"), error);
 		}
 		libxfs_trans_ijoin(tp, pip, 0);
+		xname.type = XFS_DIR3_FT_BLKDEV;
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
 		libxfs_trans_ihold(tp, pip);
 		flags |= XFS_ILOG_DEV;
@@ -504,6 +508,7 @@ parseproto(
 		if (error)
 			fail(_("Inode allocation failed"), error);
 		libxfs_trans_ijoin(tp, pip, 0);
+		xname.type = XFS_DIR3_FT_CHRDEV;
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
 		libxfs_trans_ihold(tp, pip);
 		flags |= XFS_ILOG_DEV;
@@ -516,6 +521,7 @@ parseproto(
 		if (error)
 			fail(_("Inode allocation failed"), error);
 		libxfs_trans_ijoin(tp, pip, 0);
+		xname.type = XFS_DIR3_FT_FIFO;
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
 		libxfs_trans_ihold(tp, pip);
 		break;
@@ -529,6 +535,7 @@ parseproto(
 			fail(_("Inode allocation failed"), error);
 		flags |= newfile(tp, ip, &flist, &first, 1, 1, buf, len);
 		libxfs_trans_ijoin(tp, pip, 0);
+		xname.type = XFS_DIR3_FT_SYMLINK;
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
 		libxfs_trans_ihold(tp, pip);
 		break;
@@ -546,6 +553,7 @@ parseproto(
 			isroot = 1;
 		} else {
 			libxfs_trans_ijoin(tp, pip, 0);
+			xname.type = XFS_DIR3_FT_DIR;
 			newdirent(mp, tp, pip, &xname, ip->i_ino,
 				  &first, &flist);
 			pip->i_d.di_nlink++;
-- 
1.8.4.rc3

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/2] mkfs: proto file creation does not set ftype correctly

[Eric Sandeen]

On 2/27/14, 3:45 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Hence running xfs_repair on a ftype enable filesystem that has
> contents created by a proto file will throw warnings on mismatched
> ftype entries and correct them. xfs/031 fails due to this problem.
> 
> Fix it by settin gup the xname structure with the correct type

"setting up"

> fields.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>

huh, surprised that hung out there for so long ;)

I guess the root inode is handled by core mkfs code, right?
(around line 544)

And I notice that "r" filetypes aren't documented in the
manpage but that's a different issue.

As long as I'm right about the root inode,

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  mkfs/proto.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/mkfs/proto.c b/mkfs/proto.c
> index 4cc0df6..4d3680d 100644
> --- a/mkfs/proto.c
> +++ b/mkfs/proto.c
> @@ -438,6 +438,7 @@ parseproto(
>  	creds.cr_gid = (int)getnum(pp);
>  	xname.name = (uchar_t *)name;
>  	xname.len = name ? strlen(name) : 0;
> +	xname.type = 0;
>  	tp = libxfs_trans_alloc(mp, 0);
>  	flags = XFS_ILOG_CORE;
>  	xfs_bmap_init(&flist, &first);
> @@ -453,6 +454,7 @@ parseproto(
>  		if (buf)
>  			free(buf);
>  		libxfs_trans_ijoin(tp, pip, 0);
> +		xname.type = XFS_DIR3_FT_REG_FILE;
>  		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
>  		libxfs_trans_ihold(tp, pip);
>  		break;
> @@ -469,6 +471,7 @@ parseproto(
>  
>  		libxfs_trans_ijoin(tp, pip, 0);
>  
> +		xname.type = XFS_DIR3_FT_REG_FILE;
>  		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
>  		libxfs_trans_ihold(tp, pip);
>  		libxfs_trans_log_inode(tp, ip, flags);
> @@ -490,6 +493,7 @@ parseproto(
>  			fail(_("Inode allocation failed"), error);
>  		}
>  		libxfs_trans_ijoin(tp, pip, 0);
> +		xname.type = XFS_DIR3_FT_BLKDEV;
>  		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
>  		libxfs_trans_ihold(tp, pip);
>  		flags |= XFS_ILOG_DEV;
> @@ -504,6 +508,7 @@ parseproto(
>  		if (error)
>  			fail(_("Inode allocation failed"), error);
>  		libxfs_trans_ijoin(tp, pip, 0);
> +		xname.type = XFS_DIR3_FT_CHRDEV;
>  		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
>  		libxfs_trans_ihold(tp, pip);
>  		flags |= XFS_ILOG_DEV;
> @@ -516,6 +521,7 @@ parseproto(
>  		if (error)
>  			fail(_("Inode allocation failed"), error);
>  		libxfs_trans_ijoin(tp, pip, 0);
> +		xname.type = XFS_DIR3_FT_FIFO;
>  		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
>  		libxfs_trans_ihold(tp, pip);
>  		break;
> @@ -529,6 +535,7 @@ parseproto(
>  			fail(_("Inode allocation failed"), error);
>  		flags |= newfile(tp, ip, &flist, &first, 1, 1, buf, len);
>  		libxfs_trans_ijoin(tp, pip, 0);
> +		xname.type = XFS_DIR3_FT_SYMLINK;
>  		newdirent(mp, tp, pip, &xname, ip->i_ino, &first, &flist);
>  		libxfs_trans_ihold(tp, pip);
>  		break;
> @@ -546,6 +553,7 @@ parseproto(
>  			isroot = 1;
>  		} else {
>  			libxfs_trans_ijoin(tp, pip, 0);
> +			xname.type = XFS_DIR3_FT_DIR;
>  			newdirent(mp, tp, pip, &xname, ip->i_ino,
>  				  &first, &flist);
>  			pip->i_d.di_nlink++;
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/2] mkfs: proto file creation does not set ftype correctly

[Dave Chinner]

On Thu, Feb 27, 2014 at 08:17:43AM -0600, Eric Sandeen wrote:
> On 2/27/14, 3:45 AM, Dave Chinner wrote:
> > From: Dave Chinner <dchinner@redhat.com>
> > 
> > Hence running xfs_repair on a ftype enable filesystem that has
> > contents created by a proto file will throw warnings on mismatched
> > ftype entries and correct them. xfs/031 fails due to this problem.
> > 
> > Fix it by settin gup the xname structure with the correct type
> 
> "setting up"
> 
> > fields.
> > 
> > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> 
> huh, surprised that hung out there for so long ;)

Just shows how long it took my backlog down far enough to deal with
non-critical xfstests failures on CRC enabled filesystems...

> I guess the root inode is handled by core mkfs code, right?
> (around line 544)

Yes - and the root inode is indexed in a directory (it's indexed by
the superblock), so there's not ftype entry to be set up.

> And I notice that "r" filetypes aren't documented in the
> manpage but that's a different issue.

Patch? ;)

> As long as I'm right about the root inode,
> 
> Reviewed-by: Eric Sandeen <sandeen@redhat.com>

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/2] mkfs: default log size for small filesystems too large

[Eric Sandeen]

On 2/27/14, 3:45 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Recent changes to the log size scaling have resulted in using the
> default size multiplier for the log size even on small filesystems.
> Commit 88cd79b ("xfs: Add xfs_log_rlimit.c") changed the calculation
> of the maximum transaction size that the kernel would issues and
> that significantly increased the minimum size of the default log.
> As such the size of the log on small filesystems was typically
> larger than the prefious default, even though the previous default
> was still larger than the minimum needed.
> 
> Rework the default log size calculation such that it will use the
> original log size default if it is larger than the minimum log size
> required, and only use a larger log if the configuration of the
> filesystem requires it.
> 
> This is especially obvious in xfs/216, where the default log size is
> 10MB all the way up to 16GB filesystems. The current mkfs selects a
> log size of 50MB for the same size filesystems and this is
> unnecessarily large.
> 
> Return the scaling of the log size for small filesystems to
> something similar to what xfs/216 expects.

I can confirm that this fixes xfs/216, but I've lost the thread on
why log size scaling was changed at all in 88cd79b, and why we now
end up with something different in mkfs.xfs than what we originally
had....  Are there resulting functional changes?  Cosmetic?  What's
going on with log scaling, and who moved my cheese?  :)

-Eric

> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>  mkfs/xfs_mkfs.c | 48 ++++++++++++++++++++++++++++--------------------
>  1 file changed, 28 insertions(+), 20 deletions(-)
> 
> diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
> index d82128c..f7cf394 100644
> --- a/mkfs/xfs_mkfs.c
> +++ b/mkfs/xfs_mkfs.c
> @@ -2366,32 +2366,40 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
>  	} else if (!loginternal && !xi.logdev) {
>  		logblocks = 0;
>  	} else if (loginternal && !logsize) {
> -		/*
> -		 * With a 2GB max log size, default to maximum size
> -		 * at 4TB. This keeps the same ratio from the older
> -		 * max log size of 128M at 256GB fs size. IOWs,
> -		 * the ratio of fs size to log size is 2048:1.
> -		 */
> -		logblocks = (dblocks << blocklog) / 2048;
> -		logblocks = logblocks >> blocklog;
> -		logblocks = MAX(min_logblocks, logblocks);
>  
> -		/*
> -		 * If the default log size doesn't fit in the AG size, use the
> -		 * minimum log size instead. This ensures small filesystems
> -		 * don't use excessive amounts of space for the log.
> -		 */
> -		if (min_logblocks * XFS_DFL_LOG_FACTOR >= agsize) {
> +		if (dblocks < GIGABYTES(1, blocklog)) {
> +			/* tiny filesystems get minimum sized logs. */
>  			logblocks = min_logblocks;
> +		} else if (dblocks < GIGABYTES(16, blocklog)) {
> +
> +			/*
> +			 * For small filesystems, we want to use the
> +			 * XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
> +			 * at all possible, ramping up to 128MB at 256GB.
> +			 */
> +			logblocks = MIN(XFS_MIN_LOG_BYTES >> blocklog,
> +					min_logblocks * XFS_DFL_LOG_FACTOR);
>  		} else {
> -			logblocks = MAX(logblocks,
> -				MAX(XFS_DFL_LOG_SIZE,
> -					min_logblocks * XFS_DFL_LOG_FACTOR));
> +			/*
> +			 * With a 2GB max log size, default to maximum size
> +			 * at 4TB. This keeps the same ratio from the older
> +			 * max log size of 128M at 256GB fs size. IOWs,
> +			 * the ratio of fs size to log size is 2048:1.
> +			 */
> +			logblocks = (dblocks << blocklog) / 2048;
> +			logblocks = logblocks >> blocklog;
> +			logblocks = MAX(min_logblocks, logblocks);
>  		}
> +
> +		/* make sure the log fits wholly within an AG */
> +		if (logblocks >= agsize)
> +			logblocks = min_logblocks;
> +
> +		/* and now clamp the size to the maximum supported size */
>  		logblocks = MIN(logblocks, XFS_MAX_LOG_BLOCKS);
> -		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES) {
> +		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES)
>  			logblocks = XFS_MAX_LOG_BYTES >> blocklog;
> -		}
> +
>  	}
>  	validate_log_size(logblocks, blocklog, min_logblocks);
>  
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/2] mkfs: default log size for small filesystems too large

[Dave Chinner]

On Thu, Feb 27, 2014 at 04:29:03PM -0600, Eric Sandeen wrote:
> On 2/27/14, 3:45 AM, Dave Chinner wrote:
> > From: Dave Chinner <dchinner@redhat.com>
> > 
> > Recent changes to the log size scaling have resulted in using the
> > default size multiplier for the log size even on small filesystems.
> > Commit 88cd79b ("xfs: Add xfs_log_rlimit.c") changed the calculation
> > of the maximum transaction size that the kernel would issues and
> > that significantly increased the minimum size of the default log.
> > As such the size of the log on small filesystems was typically
> > larger than the prefious default, even though the previous default
> > was still larger than the minimum needed.
> > 
> > Rework the default log size calculation such that it will use the
> > original log size default if it is larger than the minimum log size
> > required, and only use a larger log if the configuration of the
> > filesystem requires it.
> > 
> > This is especially obvious in xfs/216, where the default log size is
> > 10MB all the way up to 16GB filesystems. The current mkfs selects a
> > log size of 50MB for the same size filesystems and this is
> > unnecessarily large.
> > 
> > Return the scaling of the log size for small filesystems to
> > something similar to what xfs/216 expects.
> 
> I can confirm that this fixes xfs/216, but I've lost the thread on
> why log size scaling was changed at all in 88cd79b, and why we now
> end up with something different in mkfs.xfs than what we originally
> had....  Are there resulting functional changes?  Cosmetic?  What's
> going on with log scaling, and who moved my cheese?  :)

tl;dr: I screwed up with the original change that caused the log
size to increase. It wasn't caught by review, it caused quite a few
problems for xfstests, and some of the things it was doing made no
sense. So this patch  is reverting the behaviour back to -almost-
the same as the 3.1.x series.

The only difference is that now the minimum log size takes into
account sunit padding on log writes IOWs, the log sizes should be
identical to 3.1.x for filesystems with lsu=0 and that's why tests
like xfs/216 pass again....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/2] mkfs: default log size for small filesystems too large

[Eric Sandeen]

On 2/27/14, 3:45 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Recent changes to the log size scaling have resulted in using the
> default size multiplier for the log size even on small filesystems.
> Commit 88cd79b ("xfs: Add xfs_log_rlimit.c") changed the calculation
> of the maximum transaction size that the kernel would issues and
> that significantly increased the minimum size of the default log.
> As such the size of the log on small filesystems was typically
> larger than the prefious default, even though the previous default
> was still larger than the minimum needed.
> 
> Rework the default log size calculation such that it will use the
> original log size default if it is larger than the minimum log size
> required, and only use a larger log if the configuration of the
> filesystem requires it.
> 
> This is especially obvious in xfs/216, where the default log size is
> 10MB all the way up to 16GB filesystems. The current mkfs selects a
> log size of 50MB for the same size filesystems and this is
> unnecessarily large.
> 
> Return the scaling of the log size for small filesystems to
> something similar to what xfs/216 expects.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>

Looks good.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> ---
>  mkfs/xfs_mkfs.c | 48 ++++++++++++++++++++++++++++--------------------
>  1 file changed, 28 insertions(+), 20 deletions(-)
> 
> diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
> index d82128c..f7cf394 100644
> --- a/mkfs/xfs_mkfs.c
> +++ b/mkfs/xfs_mkfs.c
> @@ -2366,32 +2366,40 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
>  	} else if (!loginternal && !xi.logdev) {
>  		logblocks = 0;
>  	} else if (loginternal && !logsize) {
> -		/*
> -		 * With a 2GB max log size, default to maximum size
> -		 * at 4TB. This keeps the same ratio from the older
> -		 * max log size of 128M at 256GB fs size. IOWs,
> -		 * the ratio of fs size to log size is 2048:1.
> -		 */
> -		logblocks = (dblocks << blocklog) / 2048;
> -		logblocks = logblocks >> blocklog;
> -		logblocks = MAX(min_logblocks, logblocks);
>  
> -		/*
> -		 * If the default log size doesn't fit in the AG size, use the
> -		 * minimum log size instead. This ensures small filesystems
> -		 * don't use excessive amounts of space for the log.
> -		 */
> -		if (min_logblocks * XFS_DFL_LOG_FACTOR >= agsize) {
> +		if (dblocks < GIGABYTES(1, blocklog)) {
> +			/* tiny filesystems get minimum sized logs. */
>  			logblocks = min_logblocks;
> +		} else if (dblocks < GIGABYTES(16, blocklog)) {
> +
> +			/*
> +			 * For small filesystems, we want to use the
> +			 * XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
> +			 * at all possible, ramping up to 128MB at 256GB.
> +			 */
> +			logblocks = MIN(XFS_MIN_LOG_BYTES >> blocklog,
> +					min_logblocks * XFS_DFL_LOG_FACTOR);
>  		} else {
> -			logblocks = MAX(logblocks,
> -				MAX(XFS_DFL_LOG_SIZE,
> -					min_logblocks * XFS_DFL_LOG_FACTOR));
> +			/*
> +			 * With a 2GB max log size, default to maximum size
> +			 * at 4TB. This keeps the same ratio from the older
> +			 * max log size of 128M at 256GB fs size. IOWs,
> +			 * the ratio of fs size to log size is 2048:1.
> +			 */
> +			logblocks = (dblocks << blocklog) / 2048;
> +			logblocks = logblocks >> blocklog;
> +			logblocks = MAX(min_logblocks, logblocks);
>  		}
> +
> +		/* make sure the log fits wholly within an AG */
> +		if (logblocks >= agsize)
> +			logblocks = min_logblocks;
> +
> +		/* and now clamp the size to the maximum supported size */
>  		logblocks = MIN(logblocks, XFS_MAX_LOG_BLOCKS);
> -		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES) {
> +		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES)
>  			logblocks = XFS_MAX_LOG_BYTES >> blocklog;
> -		}
> +
>  	}
>  	validate_log_size(logblocks, blocklog, min_logblocks);
>  
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/2] mkfs: default log size for small filesystems too large

[Brian Foster]

On Thu, Feb 27, 2014 at 08:45:43PM +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Recent changes to the log size scaling have resulted in using the
> default size multiplier for the log size even on small filesystems.
> Commit 88cd79b ("xfs: Add xfs_log_rlimit.c") changed the calculation
> of the maximum transaction size that the kernel would issues and
> that significantly increased the minimum size of the default log.
> As such the size of the log on small filesystems was typically
> larger than the prefious default, even though the previous default
> was still larger than the minimum needed.
> 
> Rework the default log size calculation such that it will use the
> original log size default if it is larger than the minimum log size
> required, and only use a larger log if the configuration of the
> filesystem requires it.
> 
> This is especially obvious in xfs/216, where the default log size is
> 10MB all the way up to 16GB filesystems. The current mkfs selects a
> log size of 50MB for the same size filesystems and this is
> unnecessarily large.
> 
> Return the scaling of the log size for small filesystems to
> something similar to what xfs/216 expects.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---

The code for the <16GB case still looks funny to me, but we beat that
horse to death on irc. ;) I do like the breakdown of logic to select the
log size calculation based on the particular situation, and the end
result seems to be what we want. The additional comments help too. :)

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  mkfs/xfs_mkfs.c | 48 ++++++++++++++++++++++++++++--------------------
>  1 file changed, 28 insertions(+), 20 deletions(-)
> 
> diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
> index d82128c..f7cf394 100644
> --- a/mkfs/xfs_mkfs.c
> +++ b/mkfs/xfs_mkfs.c
> @@ -2366,32 +2366,40 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
>  	} else if (!loginternal && !xi.logdev) {
>  		logblocks = 0;
>  	} else if (loginternal && !logsize) {
> -		/*
> -		 * With a 2GB max log size, default to maximum size
> -		 * at 4TB. This keeps the same ratio from the older
> -		 * max log size of 128M at 256GB fs size. IOWs,
> -		 * the ratio of fs size to log size is 2048:1.
> -		 */
> -		logblocks = (dblocks << blocklog) / 2048;
> -		logblocks = logblocks >> blocklog;
> -		logblocks = MAX(min_logblocks, logblocks);
>  
> -		/*
> -		 * If the default log size doesn't fit in the AG size, use the
> -		 * minimum log size instead. This ensures small filesystems
> -		 * don't use excessive amounts of space for the log.
> -		 */
> -		if (min_logblocks * XFS_DFL_LOG_FACTOR >= agsize) {
> +		if (dblocks < GIGABYTES(1, blocklog)) {
> +			/* tiny filesystems get minimum sized logs. */
>  			logblocks = min_logblocks;
> +		} else if (dblocks < GIGABYTES(16, blocklog)) {
> +
> +			/*
> +			 * For small filesystems, we want to use the
> +			 * XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
> +			 * at all possible, ramping up to 128MB at 256GB.
> +			 */
> +			logblocks = MIN(XFS_MIN_LOG_BYTES >> blocklog,
> +					min_logblocks * XFS_DFL_LOG_FACTOR);
>  		} else {
> -			logblocks = MAX(logblocks,
> -				MAX(XFS_DFL_LOG_SIZE,
> -					min_logblocks * XFS_DFL_LOG_FACTOR));
> +			/*
> +			 * With a 2GB max log size, default to maximum size
> +			 * at 4TB. This keeps the same ratio from the older
> +			 * max log size of 128M at 256GB fs size. IOWs,
> +			 * the ratio of fs size to log size is 2048:1.
> +			 */
> +			logblocks = (dblocks << blocklog) / 2048;
> +			logblocks = logblocks >> blocklog;
> +			logblocks = MAX(min_logblocks, logblocks);
>  		}
> +
> +		/* make sure the log fits wholly within an AG */
> +		if (logblocks >= agsize)
> +			logblocks = min_logblocks;
> +
> +		/* and now clamp the size to the maximum supported size */
>  		logblocks = MIN(logblocks, XFS_MAX_LOG_BLOCKS);
> -		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES) {
> +		if ((logblocks << blocklog) > XFS_MAX_LOG_BYTES)
>  			logblocks = XFS_MAX_LOG_BYTES >> blocklog;
> -		}
> +
>  	}
>  	validate_log_size(logblocks, blocklog, min_logblocks);
>  
> -- 
> 1.8.4.rc3
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs