[PATCH 0/5] xfstests: fixes for the free inode btree

[PATCH 0/5] xfstests: fixes for the free inode btree

[Brian Foster]

Hi all,

This series is a few xfstests fixes and addons for the finobt. Patch 1
fixes xfs/030 to work correctly on finobt-enabled filesystems. Patches 2
and 3 add support for finobt-oriented tests via require functions and
repair filter updates. Patch 4 adds a new test for targeted repair of
finobt filesystems. Patch 5 adds a stress test that creates/modifies a
sparsely allocated set of inodes to effectively exercise the finobt in
conjunction with an fsstress workload.

xfs/010 runs very quickly. xfs/013 runs for 5-10 minutes on my smallish
VM running against a single spindle, so I've been back and forth on
whether it should be part of the auto group. Thoughts, reviews, flames
appreciated...

Brian

Brian Foster (5):
  xfs/030: filter out extra repair noise for finobt enabled fs'
  xfstests: add _require_xfs_[mkfs_]finobt() checks for finobt tests
  xfstests: filter agno/ino repair output for finobt
  xfs/010: test repair for finobt corruption
  xfs/013: stress the free inode btree

 common/rc         |  18 +++++++
 common/repair     |   2 +
 tests/xfs/010     | 133 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/010.out |  57 ++++++++++++++++++++
 tests/xfs/013     | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/013.out |   7 +++
 tests/xfs/030     |   3 +-
 tests/xfs/group   |   2 +
 8 files changed, 376 insertions(+), 1 deletion(-)
 create mode 100755 tests/xfs/010
 create mode 100644 tests/xfs/010.out
 create mode 100755 tests/xfs/013
 create mode 100644 tests/xfs/013.out

-- 
1.8.3.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 1/5] xfs/030: filter out extra repair noise for finobt enabled fs'

[Brian Foster]

xfs/030 nukes various on-disk data structures to test for repair. This
can result in extra output when testing finobt enabled filesystems. For
example, xfs_repair detects an invalid free inode btree root block when
the agi is zeroed.

Filter this output directly in xfs/030 such that the test passes for
finobt and non-finobt filesystems.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 tests/xfs/030 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/xfs/030 b/tests/xfs/030
index af03166..a43455f 100755
--- a/tests/xfs/030
+++ b/tests/xfs/030
@@ -56,7 +56,8 @@ _check_ag()
 	do
 		echo "Corrupting $structure - setting bits to $1"
 		_check_repair $1 "$structure" |
-			sed -e '/^error following ag 0 unlinked list$/d'
+			sed -e '/^error following ag 0 unlinked list$/d' \
+			    -e '/^bad agbno AGBNO for finobt/d'
 	done
 }
 
-- 
1.8.3.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 2/5] xfstests: add _require_xfs_[mkfs_]finobt() checks for finobt tests

[Brian Foster]

Free inode btree tests must ensure that the userspace and kernel bits
are compatible. Add a couple checks for the associated support.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 common/rc | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/common/rc b/common/rc
index 5c13db5..66a3cb8 100644
--- a/common/rc
+++ b/common/rc
@@ -1113,6 +1113,24 @@ _require_xfs_crc()
 	umount $SCRATCH_MNT
 }
 
+# this test requires the finobt feature to be available in mkfs.xfs
+#
+_require_xfs_mkfs_finobt()
+{
+	_scratch_mkfs_xfs_supported -m crc=1,finobt=1 >/dev/null 2>&1 \
+	   || _notrun "mkfs.xfs doesn't have finobt feature"
+}
+
+# this test requires the xfs kernel support finobt feature
+#
+_require_xfs_finobt()
+{
+	_scratch_mkfs_xfs -m crc=1,finobt=1 >/dev/null 2>&1
+	_scratch_mount >/dev/null 2>&1 \
+	   || _notrun "Kernel doesn't support finobt feature"
+	umount $SCRATCH_MNT
+}
+
 # this test requires that external log/realtime devices are not in use
 #
 _require_nonexternal()
-- 
1.8.3.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 3/5] xfstests: filter agno/ino repair output for finobt

[Brian Foster]

finobt enabled filesystems can generate new repair output. Update
_filter_repair() to ensure specific AG and inode numbers are filtered
from test output.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 common/repair | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/common/repair b/common/repair
index 3e73c1d..46ca9e0 100644
--- a/common/repair
+++ b/common/repair
@@ -83,6 +83,8 @@ s/\s+- \d+:\d\d:\d\d:.*\n//g;
 /^Metadata corruption detected/ && next;
 /^Metadata CRC error detected/ && next;
 /^agfl has bad CRC/ && next;
+# finobt enabled filesystem output
+s/(inode chunk) (\d+)\/(\d+)/AGNO\/INO/;
 	print;'
 }
 
-- 
1.8.3.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 4/5] xfs/010: test repair for finobt corruption

[Brian Foster]

The finobt creates a duplicate subset of inode allocation metadata from
the inobt. xfs_repair should detect and repair inconsistencies in the
finobt that could be caused by bugs or corruption. This test uses xfs_db
to cause targeted corruptions in the finobt and verify repair detects
and corrects the filesystem.

In particular, the test corrupts individual finobt records to cause
inconsistency between the inode allocation count fields as well as
causing the finobt to contain a record with no free inodes.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 tests/xfs/010     | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/010.out |  57 +++++++++++++++++++++++
 tests/xfs/group   |   1 +
 3 files changed, 191 insertions(+)
 create mode 100755 tests/xfs/010
 create mode 100644 tests/xfs/010.out

diff --git a/tests/xfs/010 b/tests/xfs/010
new file mode 100755
index 0000000..2b5ad00
--- /dev/null
+++ b/tests/xfs/010
@@ -0,0 +1,133 @@
+#!/bin/bash
+# FS QA Test No. xfs/010
+#
+# Test xfs_repair of the free inode btree (finobt). Make a couple targeted
+# corruptions and verify that xfs_repair detects and repairs the filesystem to
+# a consistent state.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/repair
+
+_cleanup()
+{
+	cd /
+	umount $SCRATCH_MNT 2>/dev/null
+	rm -f $tmp.*
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_sparse_inode_populate()
+{
+	dir=$1
+	count=$2
+
+	for i in $(seq 0 $count)
+	do
+		touch $dir/$i
+	done
+
+	# Inode chunks are allocated 64 inodes at a time. If we remove 1 out of
+	# every 32 we allocated above, we'll end up leaving an inode or two free
+	# in each chunk. This ensures that most records are inserted into the
+	# finobt.
+	for i in $(seq 0 32 $count)
+	do
+		rm -f $dir/$i
+	done
+}
+
+_filter_dbval()
+{
+	awk '{ print $3 }'
+}
+
+_corrupt_finobt_records()
+{
+	dev=$1
+
+	# determine the root block of the finobt
+	free_root=`$XFS_DB_PROG -c "agi 0" -c "p free_root" $dev |
+			_filter_dbval`
+
+	# Corrupt a freecount value. This should never exceed 64.
+	$XFS_DB_PROG -x -c "fsb $free_root" -c "type inobt" \
+		-c "write recs[1].freecount 70" $dev
+
+	# Create a corrupted non-free record, which should never appear in the
+	# finobt.
+	$XFS_DB_PROG -x -c "fsb $free_root" -c "type inobt" \
+		 -c "write recs[2].freecount 0" $dev
+	$XFS_DB_PROG -x -c "fsb $free_root" -c "type inobt" \
+		-c "write recs[2].free 0" $dev
+}
+
+_corrupt_finobt_root()
+{
+	dev=$1
+
+	# nuke the agi finobt root fields
+	$XFS_DB_PROG -x -c "agi 0" -c "write free_root 0" $dev
+	$XFS_DB_PROG -x -c "agi 0" -c "write free_level 0" $dev
+}
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+
+_require_scratch
+_require_xfs_mkfs_finobt
+_require_xfs_finobt
+
+rm -f $seqres.full
+
+_scratch_mkfs_xfs "-m crc=1,finobt=1 -d agcount=2" | _filter_mkfs 2>$seqres.full
+
+# sparsely populate the fs such that we create records with free inodes
+_scratch_mount
+_sparse_inode_populate $SCRATCH_MNT 999
+umount $SCRATCH_MNT
+
+# corrupt some finobt records
+_corrupt_finobt_records $SCRATCH_DEV
+
+# repair should detect the inconsistencies
+_scratch_xfs_repair 2>&1 | _filter_repair
+_check_scratch_fs
+
+# nuke the finobt root, repair will have to regenerate from the inobt
+_corrupt_finobt_root $SCRATCH_DEV
+
+_scratch_xfs_repair 2>&1 | _filter_repair
+_check_scratch_fs
+
+status=0
+exit
diff --git a/tests/xfs/010.out b/tests/xfs/010.out
new file mode 100644
index 0000000..ab5e5f3
--- /dev/null
+++ b/tests/xfs/010.out
@@ -0,0 +1,57 @@
+QA output created by 010
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+recs[1].freecount = 70
+recs[2].freecount = 0
+recs[2].free = 0
+Phase 1 - find and verify superblock...
+Phase 2 - using <TYPEOF> log
+        - zero log...
+        - scan filesystem freespace and inode maps...
+finobt ir_freecount/free mismatch, AGNO/INO, freecount 70 nfree 2
+finobt record with no free inodes, AGNO/INO
+        - found root inode chunk
+Phase 3 - for each AG...
+        - scan and clear agi unlinked lists...
+        - process known inodes and perform inode discovery...
+        - process newly discovered inodes...
+Phase 4 - check for duplicate blocks...
+        - setting up duplicate extent list...
+        - check for inodes claiming duplicate blocks...
+Phase 5 - rebuild AG headers and trees...
+        - reset superblock...
+Phase 6 - check inode connectivity...
+        - resetting contents of realtime bitmap and summary inodes
+        - traversing filesystem ...
+        - traversal finished ...
+        - moving disconnected inodes to lost+found ...
+Phase 7 - verify and correct link counts...
+done
+free_root = 0
+free_level = 0
+Phase 1 - find and verify superblock...
+Phase 2 - using <TYPEOF> log
+        - zero log...
+        - scan filesystem freespace and inode maps...
+bad agbno AGBNO for finobt root, agno 0
+        - found root inode chunk
+Phase 3 - for each AG...
+        - scan and clear agi unlinked lists...
+        - process known inodes and perform inode discovery...
+        - process newly discovered inodes...
+Phase 4 - check for duplicate blocks...
+        - setting up duplicate extent list...
+        - check for inodes claiming duplicate blocks...
+Phase 5 - rebuild AG headers and trees...
+        - reset superblock...
+Phase 6 - check inode connectivity...
+        - resetting contents of realtime bitmap and summary inodes
+        - traversing filesystem ...
+        - traversal finished ...
+        - moving disconnected inodes to lost+found ...
+Phase 7 - verify and correct link counts...
+done
diff --git a/tests/xfs/group b/tests/xfs/group
index 4624fc3..c228a63 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -7,6 +7,7 @@
 007 auto quota quick
 008 rw ioctl auto quick
 009 rw ioctl auto prealloc quick
+010 auto quick repair
 012 rw auto quick
 016 rw auto quick
 017 mount auto quick stress
-- 
1.8.3.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 5/5] xfs/013: stress the free inode btree

[Brian Foster]

Create a stress test for the free inode btree. Allocate a set of inodes
sequentually and run a hard link clone and random replacement algorithm
across the set. Background removal of the oldest directories creates a
sparse set of free inodes over time. Run an fsstress workload
concurrently to exercise the fs.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 tests/xfs/013     | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/013.out |   7 +++
 tests/xfs/group   |   1 +
 3 files changed, 163 insertions(+)
 create mode 100755 tests/xfs/013
 create mode 100644 tests/xfs/013.out

diff --git a/tests/xfs/013 b/tests/xfs/013
new file mode 100755
index 0000000..049f0dc
--- /dev/null
+++ b/tests/xfs/013
@@ -0,0 +1,155 @@
+#!/bin/bash
+# FS QA Test No. xfs/013
+#
+# Exercise the free inode btree (finobt). XFS allocates physical inodes in
+# chunks of 64. Inode records with at least one free inode are stored in the
+# finobt to optimize free inode lookup. This test runs a workload that creates
+# and modifies a sparsely allocated set of inodes in combination with an
+# fsstress workload.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_cleanup()
+{
+	killall fsstress 2>/dev/null
+	cd /
+	umount $SCRATCH_MNT 2>/dev/null
+	rm -f $tmp.*
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_create()
+{
+	dir=$1
+	count=$2
+
+	mkdir -p $dir
+	for i in $(seq 0 $count)
+	do
+		touch $dir/$i
+	done
+}
+
+_rand_replace()
+{
+	dir=$1
+	count=$2
+
+	# replace 5% of the dataset
+	for i in $(seq 0 $((count / 20)))
+	do
+		file=$((RANDOM % count))
+		rm -f $dir/$file
+		touch $dir/$file
+	done
+}
+
+_cleaner()
+{
+	dir=$1
+	iters=$2
+	mindirs=$3
+
+	iters=$((iters - mindirs))
+
+	for i in $(seq 1 $iters)
+	do
+		need=$dir/dir$((i + mindirs))
+		while [ ! -e $need ]
+		do
+			sleep 3
+		done
+
+		rm -rf $dir/dir$i
+	done
+}
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+
+_require_scratch
+_require_xfs_mkfs_finobt
+_require_xfs_finobt
+
+rm -f $seqres.full
+
+_scratch_mkfs_xfs "-m crc=1,finobt=1 -d agcount=2" | \
+	_filter_mkfs 2>> $seqres.full
+_scratch_mount
+
+COUNT=20000	# number of files per directory
+LOOPS=15	# last loop iteration
+MINDIRS=2	# number of dirs for the cleaner to leave trailing behind the
+		# most recent (no less than 2 to prevent an rm from trampling a
+		# clone)
+
+# create initial directory
+_create $SCRATCH_MNT/dir1 $COUNT
+
+# start background cleaner to remove old directories as new ones are created
+_cleaner $SCRATCH_MNT $LOOPS $MINDIRS &
+
+# start a background stress workload on the fs
+$FSSTRESS_PROG -d $SCRATCH_MNT/fsstress -w -n 9999999 -p 2 -S t \
+	>> $seqres.full 2>&1 &
+
+# Each cycle clones the current directory and makes a random file replacement
+# pass on the new directory. The directory is copied to the next using hard
+# links. The replacement pass then randomly removes and replaces ~5% of the
+# content in the directory. Files replaced as such are effectively marked to be
+# freed by the background cleaner as it moves forward and removes all of the
+# previous hard links to the inode. Over several iterations, this workload
+# creates a sparsely located set of a free inodes across the set and uses the
+# finobt to allocate new inodes for replacement.
+
+for i in $(seq 1 $LOOPS)
+do
+	# hard link the content of the current directory to the next
+	cp -Rl $SCRATCH_MNT/dir$i $SCRATCH_MNT/dir$((i+1))
+
+	# do a random replacement of files in the new directory
+	_rand_replace $SCRATCH_MNT/dir$((i+1)) $COUNT
+done
+
+killall fsstress
+wait
+
+# clean out the competing fsstress allocations, then everything else
+rm -rf $SCRATCH_MNT/fsstress
+rm -rf $SCRATCH_MNT/dir*
+
+umount $SCRATCH_MNT
+_check_scratch_fs
+
+status=0
+exit
diff --git a/tests/xfs/013.out b/tests/xfs/013.out
new file mode 100644
index 0000000..ae653ff
--- /dev/null
+++ b/tests/xfs/013.out
@@ -0,0 +1,7 @@
+QA output created by 013
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
diff --git a/tests/xfs/group b/tests/xfs/group
index c228a63..0fa6674 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -9,6 +9,7 @@
 009 rw ioctl auto prealloc quick
 010 auto quick repair
 012 rw auto quick
+013 stress
 016 rw auto quick
 017 mount auto quick stress
 018 deprecated # log logprint v2log
-- 
1.8.3.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/5] xfstests: fixes for the free inode btree

[Dave Chinner]

On Fri, May 02, 2014 at 01:13:57PM -0400, Brian Foster wrote:
> Hi all,
> 
> This series is a few xfstests fixes and addons for the finobt. Patch 1
> fixes xfs/030 to work correctly on finobt-enabled filesystems. Patches 2
> and 3 add support for finobt-oriented tests via require functions and
> repair filter updates. Patch 4 adds a new test for targeted repair of
> finobt filesystems. Patch 5 adds a stress test that creates/modifies a
> sparsely allocated set of inodes to effectively exercise the finobt in
> conjunction with an fsstress workload.
> 
> xfs/010 runs very quickly. xfs/013 runs for 5-10 minutes on my smallish
> VM running against a single spindle, so I've been back and forth on
> whether it should be part of the auto group. Thoughts, reviews, flames
> appreciated...

5-10 minutes is probably right at the edge for auto, but I think
that most people won't be testing this any time soon. Hence I'd
include it by default in the auto group, and if people complain
about the runtime when they start testing it, we can revist that
choice. FWIW, I'd also include it in the metadata group so that it
gets exercised when people run that group....

I had a quick eyeball of the changes, and nothing major stood out.
The only thing I noticed was a missing "wait" in the _cleanup
function of xfs/013 after killing all the fsstress processes. It
should probably using killall -9 as well. If we don't wait, then the
unmount will fail and if the fsstress processes don't die it will
affect every test after that...

I'll probably have more suggestions once I've run the tests ;)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/5] xfstests: fixes for the free inode btree

[Brian Foster]

On Sat, May 03, 2014 at 09:48:31AM +1000, Dave Chinner wrote:
> On Fri, May 02, 2014 at 01:13:57PM -0400, Brian Foster wrote:
> > Hi all,
> > 
> > This series is a few xfstests fixes and addons for the finobt. Patch 1
> > fixes xfs/030 to work correctly on finobt-enabled filesystems. Patches 2
> > and 3 add support for finobt-oriented tests via require functions and
> > repair filter updates. Patch 4 adds a new test for targeted repair of
> > finobt filesystems. Patch 5 adds a stress test that creates/modifies a
> > sparsely allocated set of inodes to effectively exercise the finobt in
> > conjunction with an fsstress workload.
> > 
> > xfs/010 runs very quickly. xfs/013 runs for 5-10 minutes on my smallish
> > VM running against a single spindle, so I've been back and forth on
> > whether it should be part of the auto group. Thoughts, reviews, flames
> > appreciated...
> 
> 5-10 minutes is probably right at the edge for auto, but I think
> that most people won't be testing this any time soon. Hence I'd
> include it by default in the auto group, and if people complain
> about the runtime when they start testing it, we can revist that
> choice. FWIW, I'd also include it in the metadata group so that it
> gets exercised when people run that group....
> 

Ok, sounds good. It actually runs closer to 5 minutes than 10 when I
simply move to a separate (still single) spindle, so it's probably not
that bad. IIRC, it's still probably not the longest running test I've
seen in auto. I believe you have an SSD test setup, so I'm curious how
the workload looks if you get a a chance to run it there. :)

> I had a quick eyeball of the changes, and nothing major stood out.
> The only thing I noticed was a missing "wait" in the _cleanup
> function of xfs/013 after killing all the fsstress processes. It
> should probably using killall -9 as well. If we don't wait, then the
> unmount will fail and if the fsstress processes don't die it will
> affect every test after that...
> 

Indeed, thanks.

> I'll probably have more suggestions once I've run the tests ;)
> 

Ok. FWIW, the parameters of the stress test (# files, # iters, etc.)
were mostly determined empirically to provide a good workout for the
finobt in a reasonable amount of time. The test started out as open
coded for loops to do linking and random new creations, but that turned
out far too slow when tuned to be effective. The hard link cp and
smaller file replacement loop after the fact turned out to be much
faster. fsstress was added after that as it appeared to amplify the
operations on the finobt without adding too much time to the test. Any
thoughts on the parameters or broader test(s) are appreciated...

Brian

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/5] xfstests: fixes for the free inode btree

[Dave Chinner]

On Mon, May 05, 2014 at 07:34:43AM -0400, Brian Foster wrote:
> On Sat, May 03, 2014 at 09:48:31AM +1000, Dave Chinner wrote:
> > On Fri, May 02, 2014 at 01:13:57PM -0400, Brian Foster wrote:
> > > Hi all,
> > > 
> > > This series is a few xfstests fixes and addons for the finobt. Patch 1
> > > fixes xfs/030 to work correctly on finobt-enabled filesystems. Patches 2
> > > and 3 add support for finobt-oriented tests via require functions and
> > > repair filter updates. Patch 4 adds a new test for targeted repair of
> > > finobt filesystems. Patch 5 adds a stress test that creates/modifies a
> > > sparsely allocated set of inodes to effectively exercise the finobt in
> > > conjunction with an fsstress workload.
> > > 
> > > xfs/010 runs very quickly. xfs/013 runs for 5-10 minutes on my smallish
> > > VM running against a single spindle, so I've been back and forth on
> > > whether it should be part of the auto group. Thoughts, reviews, flames
> > > appreciated...
> > 
> > 5-10 minutes is probably right at the edge for auto, but I think
> > that most people won't be testing this any time soon. Hence I'd
> > include it by default in the auto group, and if people complain
> > about the runtime when they start testing it, we can revist that
> > choice. FWIW, I'd also include it in the metadata group so that it
> > gets exercised when people run that group....
> > 
> 
> Ok, sounds good. It actually runs closer to 5 minutes than 10 when I
> simply move to a separate (still single) spindle, so it's probably not
> that bad. IIRC, it's still probably not the longest running test I've
> seen in auto. I believe you have an SSD test setup, so I'm curious how
> the workload looks if you get a a chance to run it there. :)

FWIW, just running xfs/013 on 2 sata drives in hw RAID1 takes 80-90s
to run xfs/013, so this is fine. However, it runs out of disk space
on a 4GB ramdisk, so it still might need some tweaking...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/5] xfstests: fixes for the free inode btree

[Brian Foster]

On Wed, May 21, 2014 at 10:20:37AM +1000, Dave Chinner wrote:
> On Mon, May 05, 2014 at 07:34:43AM -0400, Brian Foster wrote:
> > On Sat, May 03, 2014 at 09:48:31AM +1000, Dave Chinner wrote:
> > > On Fri, May 02, 2014 at 01:13:57PM -0400, Brian Foster wrote:
> > > > Hi all,
> > > > 
> > > > This series is a few xfstests fixes and addons for the finobt. Patch 1
> > > > fixes xfs/030 to work correctly on finobt-enabled filesystems. Patches 2
> > > > and 3 add support for finobt-oriented tests via require functions and
> > > > repair filter updates. Patch 4 adds a new test for targeted repair of
> > > > finobt filesystems. Patch 5 adds a stress test that creates/modifies a
> > > > sparsely allocated set of inodes to effectively exercise the finobt in
> > > > conjunction with an fsstress workload.
> > > > 
> > > > xfs/010 runs very quickly. xfs/013 runs for 5-10 minutes on my smallish
> > > > VM running against a single spindle, so I've been back and forth on
> > > > whether it should be part of the auto group. Thoughts, reviews, flames
> > > > appreciated...
> > > 
> > > 5-10 minutes is probably right at the edge for auto, but I think
> > > that most people won't be testing this any time soon. Hence I'd
> > > include it by default in the auto group, and if people complain
> > > about the runtime when they start testing it, we can revist that
> > > choice. FWIW, I'd also include it in the metadata group so that it
> > > gets exercised when people run that group....
> > > 
> > 
> > Ok, sounds good. It actually runs closer to 5 minutes than 10 when I
> > simply move to a separate (still single) spindle, so it's probably not
> > that bad. IIRC, it's still probably not the longest running test I've
> > seen in auto. I believe you have an SSD test setup, so I'm curious how
> > the workload looks if you get a a chance to run it there. :)
> 
> FWIW, just running xfs/013 on 2 sata drives in hw RAID1 takes 80-90s
> to run xfs/013, so this is fine. However, it runs out of disk space
> on a 4GB ramdisk, so it still might need some tweaking...
> 

Noted, I'll look into it. Thanks.

Brian

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs