* [PATCH 0/10] xfstests: rework large filesystem testing
@ 2012-07-26 8:39 ` Dave Chinner
2012-07-26 8:39 ` [PATCH 01/10] xfstests: add --largefs check option Dave Chinner
` (12 more replies)
0 siblings, 13 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
This is a resend of a series that has been sitting inmy local
workarea for months. It reworks the large filesystem testing
infrastructure for XFS to allow it to use xfs_repair by switching
the filesystem filling to using preallocation rather than xfs_db
hacks.
It also introduces support for ext4 testing, also via preallocation
to fill the filesystem. btrfs isn't yet supported because I haven't
worked out how to reliably get the size of the filesystem and
calculate how much space is actually available in it.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH 01/10] xfstests: add --largefs check option
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:00 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic Dave Chinner
` (11 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Make it easier to check large filesystems quickly by adding a
--large-fs option to check to turn on shortcuts for large scratch
device filesystem testing.
Also, reject invalid command line options with a usage message.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common | 63 +++++++++++++++++++++++++++++++++++++++------------------------
1 file changed, 39 insertions(+), 24 deletions(-)
diff --git a/common b/common
index 0723224..7d13078 100644
--- a/common
+++ b/common
@@ -27,6 +27,35 @@ _setenvironment()
export MSGVERB
}
+usage()
+{
+ echo "Usage: $0 [options] [testlist]"'
+
+common options
+ -v verbose
+
+check options
+ -xfs test XFS (default)
+ -udf test UDF
+ -nfs test NFS
+ -l line mode diff
+ -xdiff graphical mode diff
+ -udiff show unified diff (default)
+ -n show me, do not run tests
+ -q quick [deprecated]
+ -T output timestamps
+ -r randomize test order
+ --large-fs optimise scratch device for large filesystems
+
+testlist options
+ -g group[,group...] include tests from these groups
+ -x group[,group...] exclude tests from these groups
+ NNN include test NNN
+ NNN-NNN include test range (eg. 012-021)
+'
+ exit 0
+}
+
here=`pwd`
rm -f $here/$iam.out
_setenvironment
@@ -117,30 +146,7 @@ s/ .*//p
in
-\? | -h | --help) # usage
- echo "Usage: $0 [options] [testlist]"'
-
-common options
- -v verbose
-
-check options
- -xfs test XFS (default)
- -udf test UDF
- -nfs test NFS
- -l line mode diff
- -xdiff graphical mode diff
- -udiff show unified diff (default)
- -n show me, do not run tests
- -q quick [deprecated]
- -T output timestamps
- -r randomize test order
-
-testlist options
- -g group[,group...] include tests from these groups
- -x group[,group...] exclude tests from these groups
- NNN include test NNN
- NNN-NNN include test range (eg. 012-021)
-'
- exit 0
+ usage
;;
-udf) # -udf ... set FSTYP to udf
@@ -232,6 +238,15 @@ testlist options
fi
;;
+ --large-fs)
+ export USE_BIG_LOOPFS=yes
+ xpand=false
+ ;;
+
+ -*)
+ usage
+ ;;
+
*)
start=$r
end=$r
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
2012-07-26 8:39 ` [PATCH 01/10] xfstests: add --largefs check option Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:01 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES Dave Chinner
` (10 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
easily as loop devices. It really means we are testing a large scratch device
and that we should enable the special filesystem filling and checking options
that enable xfstests to be run sanely on large XFS filesystems.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
004 | 2 +-
015 | 2 +-
030 | 2 +-
031 | 2 +-
032 | 2 +-
033 | 4 ++--
041 | 2 +-
049 | 2 +-
083 | 2 +-
092 | 2 +-
148 | 2 +-
149 | 2 +-
common | 2 +-
common.rc | 12 ++++++------
setup | 5 +++--
15 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/004 b/004
index 9f28e17..23729da 100755
--- a/004
+++ b/004
@@ -64,7 +64,7 @@ _supported_os IRIX Linux
_need_to_be_root
_require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
rm -f $seq.full
diff --git a/015 b/015
index 4206b93..a99f1ed 100755
--- a/015
+++ b/015
@@ -52,7 +52,7 @@ _supported_fs generic
_supported_os IRIX Linux
_require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
_scratch_mkfs_sized `expr 50 \* 1024 \* 1024` >/dev/null 2>&1 \
|| _fail "mkfs failed"
diff --git a/030 b/030
index fe57372..a8fb92c 100755
--- a/030
+++ b/030
@@ -64,8 +64,8 @@ _check_ag()
_supported_fs xfs
_supported_os IRIX Linux
-_require_nobigloopfs
_require_scratch
+_require_no_large_scratch_dev
DSIZE="-dsize=100m,agcount=6"
diff --git a/031 b/031
index b062277..fb6f15b 100755
--- a/031
+++ b/031
@@ -97,8 +97,8 @@ EOF
_supported_fs xfs
_supported_os IRIX Linux
-_require_nobigloopfs
_require_scratch
+_require_no_large_scratch_dev
# sanity test - default + one root directory entry
# Note: must do this proto/mkfs now for later inode size calcs
diff --git a/032 b/032
index 4261ca2..d093b45 100755
--- a/032
+++ b/032
@@ -41,8 +41,8 @@ rm -f $seq.full
_supported_fs xfs
_supported_os Linux
-_require_nobigloopfs
_require_scratch
+_require_no_large_scratch_dev
echo "Silence is golden."
for fs in `echo /sbin/mkfs.* | sed -e 's/.sbin.mkfs.//g'`
diff --git a/033 b/033
index 9651f26..68a688e 100755
--- a/033
+++ b/033
@@ -76,9 +76,9 @@ _filter_bad_ids()
# real QA test starts here
_supported_fs xfs
_supported_os IRIX Linux
-
-_require_nobigloopfs
+
_require_scratch
+_require_no_large_scratch_dev
# devzero blows away 512byte blocks, so make 512byte inodes (at least)
_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
diff --git a/041 b/041
index 2800811..28dcb33 100755
--- a/041
+++ b/041
@@ -50,7 +50,7 @@ _supported_fs xfs
_supported_os IRIX Linux
_require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
umount $SCRATCH_DEV 2>/dev/null
_fill()
diff --git a/049 b/049
index c3065ad..149c4b5 100755
--- a/049
+++ b/049
@@ -60,9 +60,9 @@ _log()
echo "--- $*" >> $seq.full
}
-_require_nobigloopfs
_require_nonexternal
_require_scratch
+_require_no_large_scratch_dev
_require_loop
_require_ext2
diff --git a/083 b/083
index e0670b9..7a73f30 100755
--- a/083
+++ b/083
@@ -58,7 +58,7 @@ _supported_fs generic
_supported_os IRIX Linux
_require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
rm -f $seq.full
diff --git a/092 b/092
index 429fa80..02ccc71 100755
--- a/092
+++ b/092
@@ -48,7 +48,7 @@ _cleanup()
_supported_fs xfs
_supported_os IRIX Linux
_require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
MOUNT_OPTIONS="$MOUNT_OPTIONS -o inode64"
_scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
diff --git a/148 b/148
index 76cbf37..7bb1722 100755
--- a/148
+++ b/148
@@ -66,8 +66,8 @@ _check_ag()
_supported_fs xfs
_supported_os IRIX Linux
-_require_nobigloopfs
_require_scratch
+_require_no_large_scratch_dev
DSIZE="-dsize=100m"
diff --git a/149 b/149
index 5131a45..193e6d7 100755
--- a/149
+++ b/149
@@ -100,8 +100,8 @@ EOF
_supported_fs xfs
_supported_os IRIX Linux
-_require_nobigloopfs
_require_scratch
+_require_no_large_scratch_dev
# sanity test - default + one root directory entry
# Note: must do this proto/mkfs now for later inode size calcs
diff --git a/common b/common
index 7d13078..da86cd9 100644
--- a/common
+++ b/common
@@ -239,7 +239,7 @@ s/ .*//p
;;
--large-fs)
- export USE_BIG_LOOPFS=yes
+ export LARGE_SCRATCH_DEV=yes
xpand=false
;;
diff --git a/common.rc b/common.rc
index 602513a..9517904 100644
--- a/common.rc
+++ b/common.rc
@@ -310,7 +310,7 @@ _scratch_mkfs_xfs()
cat $tmp_dir.mkfsstd
rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
- if [ "$USE_BIG_LOOPFS" = yes ]; then
+ if [ "$LARGE_SCRATCH_DEV" = yes ]; then
[ -z "$RETAIN_AG_BYTES" ] && RETAIN_AG_BYTES=0
./tools/ag-wipe -q -r $RETAIN_AG_BYTES $SCRATCH_DEV
fi
@@ -432,7 +432,7 @@ _scratch_xfs_repair()
SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -r$SCRATCH_RTDEV"
- [ "$USE_BIG_LOOPFS" = yes ] && SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
+ [ "$LARGE_SCRATCH_DEV" = yes ] && SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
$XFS_REPAIR_PROG $SCRATCH_OPTIONS $* $SCRATCH_DEV
}
@@ -821,9 +821,9 @@ _require_ext2()
# this test requires that (large) loopback device files are not in use
#
-_require_nobigloopfs()
+_require_no_large_scratch_dev()
{
- [ "$USE_BIG_LOOPFS" = yes ] && \
+ [ "$LARGE_SCRATCH_DEV" = yes ] && \
_notrun "Large filesystem testing in progress, skipped this test"
}
@@ -1202,7 +1202,7 @@ _check_xfs_filesystem()
[ "$FSTYP" != xfs ] && return 0
testoption=""
- [ "$USE_BIG_LOOPFS" = yes ] && testoption=-t
+ [ "$LARGE_SCRATCH_DEV" = yes ] && testoption=-t
type=`_fs_type $device`
ok=1
@@ -1241,7 +1241,7 @@ _check_xfs_filesystem()
ok=0
fi
# repair doesn't scale massively at this stage, optionally skip it for now
- [ "$USE_BIG_LOOPFS" = yes ] || \
+ [ "$LARGE_SCRATCH_DEV" = yes ] || \
$XFS_REPAIR_PROG -n $extra_log_options $extra_rt_options $device >$tmp.repair 2>&1
if [ $? -ne 0 ]
then
diff --git a/setup b/setup
index 62798cc..5225951 100755
--- a/setup
+++ b/setup
@@ -23,7 +23,7 @@ fi
[ "$USE_EXTERNAL" = yes ] || USE_EXTERNAL=no
[ "$USE_LBD_PATCH" = yes ] || USE_LBD_PATCH=no
-[ "$USE_BIG_LOOPFS" = yes ] || USE_BIG_LOOPFS=no
+[ "$LARGE_SCRATCH_DEV" = yes ] || LARGE_SCRATCH_DEV=no
[ "$USE_ATTR_SECURE" = yes ] || USE_ATTR_SECURE=no
[ -z "$FSTYP" ] && FSTYP="xfs"
@@ -31,5 +31,6 @@ cat <<EOF
TEST: DIR=$TEST_DIR DEV=$TEST_DEV rt=[$TEST_RTDEV] log=[$TEST_LOGDEV]
TAPE: dev=[$TAPE_DEV] rmt=[$RMT_TAPE_DEV] rmtirix=[$RMT_TAPE_USER@$RMT_IRIXTAPE_DEV]
SCRATCH: MNT=$SCRATCH_MNT DEV=$SCRATCH_DEV rt=[$SCRATCH_RTDEV] log=[$SCRATCH_LOGDEV]
-VARIABLES: external=$USE_EXTERNAL largeblk=$USE_LBD_PATCH fstyp=$FSTYP bigloopfs=$USE_BIG_LOOPFS attrsecure=$USE_ATTR_SECURE
+VARIABLES: external=$USE_EXTERNAL largeblk=$USE_LBD_PATCH fstyp=$FSTYP
+ large_scratch_dev=$LARGE_SCRATCH_DEV attrsecure=$USE_ATTR_SECURE
EOF
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
2012-07-26 8:39 ` [PATCH 01/10] xfstests: add --largefs check option Dave Chinner
2012-07-26 8:39 ` [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:01 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 04/10] xfstests: use preallocation for ag-wiper Dave Chinner
` (9 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Rename the $RETAIN_AG_BYTES variable to be more generic so that it
reflects the fact that it is designed to retain a certain amount of
extra free space above the default amount in the filesystem when
doing large scratch device testing.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common.rc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/common.rc b/common.rc
index 9517904..b0af482 100644
--- a/common.rc
+++ b/common.rc
@@ -311,8 +311,8 @@ _scratch_mkfs_xfs()
rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
if [ "$LARGE_SCRATCH_DEV" = yes ]; then
- [ -z "$RETAIN_AG_BYTES" ] && RETAIN_AG_BYTES=0
- ./tools/ag-wipe -q -r $RETAIN_AG_BYTES $SCRATCH_DEV
+ [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+ ./tools/ag-wipe -q -r $SCRATCH_DEV_EMPTY_SPACE $SCRATCH_DEV
fi
return $mkfs_status
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 04/10] xfstests: use preallocation for ag-wiper
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (2 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:57 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 05/10] xfstests: use command line option for setting extra space Dave Chinner
` (8 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
To enable sane testing of large scale filesystems, the --large-fs
test option uses xfs_db magic to mark AGs full without doing any IO.
This leaves only a small amount of free space left in the filesystem
to stress the high AGs of the filesystem rather than the low AGs.
This method requires us to have special filesystem check options to
avoid free space checking in xfs_check, and we cannot current run
xfs_repair on such a filesystem at all. As it is, free space
checking on xfs_check does not scale, so we still need to avoid this
checking regardless of how we fill the filesystem.
We can acheive exactly the same fill behaviour by preallocating a
single large file in the filesystem immediately after creating it.
This is a filesystem independent manner of filling the filesystem,
and allows us to do large filesystem testing on more than just XFS.
Further, this preallocation method effectively adds a new "very
large file" test. It also enables us to run an unmodified xfs_repair
or filesystem specific fsck program to check the filesystem for
sanity, so we can now do full sanity checking of such large
filesystems.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common.rc | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 53 insertions(+), 5 deletions(-)
diff --git a/common.rc b/common.rc
index b0af482..7bb1133 100644
--- a/common.rc
+++ b/common.rc
@@ -276,6 +276,47 @@ _scratch_mkfs_options()
echo $SCRATCH_OPTIONS $MKFS_OPTIONS $* $SCRATCH_DEV
}
+
+_setup_large_xfs_fs()
+{
+ fs_size=$1
+ local tmp_dir=/tmp/
+
+ [ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+ [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+ [ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
+
+ # calculate the size of the file we need to allocate.
+ # Default free space in the FS is 50GB, but you can specify more via
+ # SCRATCH_DEV_EMPTY_SPACE
+ file_size=$(($fs_size - 50*1024*1024*1024))
+ file_size=$(($file_size - $SCRATCH_DEV_EMPTY_SPACE))
+
+ # mount the filesystem, create the file, unmount it
+ _scratch_mount 2>&1 >$tmp_dir/mnt.err
+ local status=$?
+ if [ $status -ne 0 ]; then
+ echo "mount failed"
+ cat $tmp_dir/mnt.err >&2
+ rm -f $tmp_dir/mnt.err
+ return $status
+ fi
+ rm -f $tmp_dir/mnt.err
+
+ xfs_io -F -f \
+ -c "truncate $file_size" \
+ -c "falloc -k 0 $file_size" \
+ $SCRATCH_MNT/.use_space 2>&1 > /dev/null
+ status=$?
+ umount $SCRATCH_MNT
+ if [ $status -ne 0 ]; then
+ echo "large file prealloc failed"
+ cat $tmp_dir/mnt.err >&2
+ return $status
+ fi
+ return 0
+}
+
_scratch_mkfs_xfs()
{
# extra mkfs options can be added by tests
@@ -305,16 +346,23 @@ _scratch_mkfs_xfs()
mkfs_status=$?
fi
+ if [ $mkfs_status -eq 0 -a "$LARGE_SCRATCH_DEV" = yes ]; then
+ # manually parse the mkfs output to get the fs size in bytes
+ local fs_size
+ fs_size=`cat $tmp_dir.mkfsstd | perl -ne '
+ if (/^data\s+=\s+bsize=(\d+)\s+blocks=(\d+)/) {
+ my $size = $1 * $2;
+ print STDOUT "$size\n";
+ }'`
+ _setup_large_xfs_fs $fs_size
+ mkfs_status=$?
+ fi
+
# output stored mkfs output
cat $tmp_dir.mkfserr >&2
cat $tmp_dir.mkfsstd
rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
- if [ "$LARGE_SCRATCH_DEV" = yes ]; then
- [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
- ./tools/ag-wipe -q -r $SCRATCH_DEV_EMPTY_SPACE $SCRATCH_DEV
- fi
-
return $mkfs_status
}
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 05/10] xfstests: use command line option for setting extra space
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (3 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 04/10] xfstests: use preallocation for ag-wiper Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:57 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing Dave Chinner
` (7 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Allow the extra free space to leave in large scratch filesystems to
be specified by a command line option rather than just via an
environment variable.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/common b/common
index da86cd9..78ac654 100644
--- a/common
+++ b/common
@@ -247,6 +247,11 @@ s/ .*//p
usage
;;
+ --extra-space=*)
+ export SCRATCH_DEV_EMPTY_SPACE=${r#*=}
+ xpand=false
+ ;;
+
*)
start=$r
end=$r
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (4 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 05/10] xfstests: use command line option for setting extra space Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:58 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 07/10] xfstests: always us test option when checking large scratch device Dave Chinner
` (6 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Now that large filesystem testing does not play free space games to
fill the space without IO, we can enable xfs_repair when running in
this mode. xfs_repair has had it's scalability problems solved, too,
so this is a safe thing to do.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common.rc | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/common.rc b/common.rc
index 7bb1133..a9252e0 100644
--- a/common.rc
+++ b/common.rc
@@ -1238,12 +1238,12 @@ _check_xfs_filesystem()
extra_mount_options=""
device=$1
if [ "$2" != "none" ]; then
- extra_log_options="-l$2"
+ extra_log_options="-l$2"
extra_mount_options="-ologdev=$2"
fi
if [ "$3" != "none" ]; then
- extra_rt_options="-r$3"
+ extra_rt_options="-r$3"
extra_mount_options=$extra_mount_options" -ortdev=$3"
fi
extra_mount_options=$extra_mount_options" $MOUNT_OPTIONS"
@@ -1288,8 +1288,7 @@ _check_xfs_filesystem()
ok=0
fi
- # repair doesn't scale massively at this stage, optionally skip it for now
- [ "$LARGE_SCRATCH_DEV" = yes ] || \
+
$XFS_REPAIR_PROG -n $extra_log_options $extra_rt_options $device >$tmp.repair 2>&1
if [ $? -ne 0 ]
then
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 07/10] xfstests: always us test option when checking large scratch device
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (5 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-07-26 17:21 ` Paulo Alcantara
` (2 more replies)
2012-07-26 8:39 ` [PATCH 08/10] xfstests: enable large fs testing on ext4 Dave Chinner
` (5 subsequent siblings)
12 siblings, 3 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Some tests call _check_scratch_device directly and when using a
large filesystem this needs to run with a -t option to avoid
consuming large amounts of memory. Make this happen in all cases
that the scratch device is checked.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
017 | 7 ++-----
common.rc | 2 ++
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/017 b/017
index 9ca0e72..0a3ede3 100755
--- a/017
+++ b/017
@@ -48,9 +48,6 @@ _supported_os Linux
_require_scratch
-checkopts=""
-[ "$USE_BIG_LOOPFS" = yes ] && checkopts=-t
-
echo "*** init FS"
rm -f $seq.full
@@ -81,8 +78,8 @@ do
echo "" >>$seq.full
echo "*** XFS_CHECK ***" >>$seq.full
echo "" >>$seq.full
- _scratch_xfs_check $checkopts >>$seq.full 2>&1 \
- || _fail "xfs_check $checkopts failed"
+ _scratch_xfs_check >>$seq.full 2>&1 \
+ || _fail "xfs_check failed"
_scratch_mount -o remount,rw \
|| _fail "remount rw failed"
done
diff --git a/common.rc b/common.rc
index a9252e0..df47e2a 100644
--- a/common.rc
+++ b/common.rc
@@ -470,6 +470,8 @@ _scratch_xfs_check()
SCRATCH_OPTIONS=""
[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
SCRATCH_OPTIONS="-l $SCRATCH_LOGDEV"
+ [ "$LARGE_SCRATCH_DEV" = yes ] && \
+ SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
$XFS_CHECK_PROG $SCRATCH_OPTIONS $* $SCRATCH_DEV
}
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 08/10] xfstests: enable large fs testing on ext4
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (6 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 07/10] xfstests: always us test option when checking large scratch device Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 20:03 ` Christoph Hellwig
2012-09-05 16:27 ` Rich Johnston
2012-07-26 8:39 ` [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems Dave Chinner
` (4 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Now that setting up large filesystem testing on sparse loopback
devices uses a generic method for filling the filesystem, extent
support to ext4 filesystems.
ext4 is slightly more complex to fill as it does not support files
larger than 16TB. Hence a slightly more complex method of using
multiple smaller files to fill the space is necessary.
WARNING: be prepeared for ext4 to take forever to prepare large
filesystems as allocation of large amounts of space (especially as
it approaches ENOSPC) is can take minutes rather than milliseconds
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common.rc | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 90 insertions(+)
diff --git a/common.rc b/common.rc
index df47e2a..1c3c605 100644
--- a/common.rc
+++ b/common.rc
@@ -366,6 +366,93 @@ _scratch_mkfs_xfs()
return $mkfs_status
}
+_setup_large_ext4_fs()
+{
+ fs_size=$1
+ local tmp_dir=/tmp/
+
+ [ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+ [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+ [ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
+
+ # Default free space in the FS is 50GB, but you can specify more via
+ # SCRATCH_DEV_EMPTY_SPACE
+ space_to_consume=$(($fs_size - 50*1024*1024*1024 - $SCRATCH_DEV_EMPTY_SPACE))
+
+ # mount the filesystem and create 16TB - 4KB files until we consume
+ # all the necessary space.
+ _scratch_mount 2>&1 >$tmp_dir/mnt.err
+ local status=$?
+ if [ $status -ne 0 ]; then
+ echo "mount failed"
+ cat $tmp_dir/mnt.err >&2
+ rm -f $tmp_dir/mnt.err
+ return $status
+ fi
+ rm -f $tmp_dir/mnt.err
+
+ file_size=$((16*1024*1024*1024*1024 - 4096))
+ nfiles=0
+ while [ $space_to_consume -gt $file_size ]; do
+
+ xfs_io -F -f \
+ -c "truncate $file_size" \
+ -c "falloc -k 0 $file_size" \
+ $SCRATCH_MNT/.use_space.$nfiles 2>&1
+ status=$?
+ if [ $status -ne 0 ]; then
+ break;
+ fi
+
+ space_to_consume=$(( $space_to_consume - $file_size ))
+ nfiles=$(($nfiles + 1))
+ done
+
+ # consume the remaining space.
+ if [ $space_to_consume -gt 0 ]; then
+ xfs_io -F -f \
+ -c "truncate $space_to_consume" \
+ -c "falloc -k 0 $space_to_consume" \
+ $SCRATCH_MNT/.use_space.$nfiles 2>&1
+ status=$?
+ fi
+
+ umount $SCRATCH_MNT
+ if [ $status -ne 0 ]; then
+ echo "large file prealloc failed"
+ cat $tmp_dir/mnt.err >&2
+ return $status
+ fi
+ return 0
+}
+_scratch_mkfs_ext4()
+{
+ local tmp_dir=/tmp/
+
+ /sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV \
+ 2>$tmp_dir.mkfserr 1>$tmp_dir.mkfsstd
+ local mkfs_status=$?
+
+ if [ $mkfs_status -eq 0 -a "$LARGE_SCRATCH_DEV" = yes ]; then
+ # manually parse the mkfs output to get the fs size in bytes
+ fs_size=`cat $tmp_dir.mkfsstd | awk ' \
+ /^Block size/ { split($2, a, "="); bs = a[2] ; } \
+ / inodes, / { blks = $3 } \
+ /reserved for the super user/ { resv = $1 } \
+ END { fssize = bs * blks - resv; print fssize }'`
+
+ _setup_large_ext4_fs $fs_size
+ mkfs_status=$?
+ fi
+
+ # output stored mkfs output
+ cat $tmp_dir.mkfserr >&2
+ cat $tmp_dir.mkfsstd
+ rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
+
+ return $mkfs_status
+}
+
_scratch_mkfs()
{
case $FSTYP in
@@ -381,6 +468,9 @@ _scratch_mkfs()
btrfs)
$MKFS_BTRFS_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null
;;
+ ext4)
+ _scratch_mkfs_ext4 $*
+ ;;
*)
/sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV
;;
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (7 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 08/10] xfstests: enable large fs testing on ext4 Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:03 ` Rich Johnston
2012-08-28 20:03 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 10/10] xfstests: exclude largefs fill files from dump tests Dave Chinner
` (3 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
Test 017 typically fails due to the OOM killer kicking in and
killing everything but xfs_db so panics the machine.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
017 | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/017 b/017
index 0a3ede3..b3a7f9c 100755
--- a/017
+++ b/017
@@ -48,6 +48,10 @@ _supported_os Linux
_require_scratch
+# xfs_db will OOM kill the machine if you don't have huge amounts of RAM, so
+# don't run this on large filesystems.
+_require_no_large_scratch_dev
+
echo "*** init FS"
rm -f $seq.full
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH 10/10] xfstests: exclude largefs fill files from dump tests
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (8 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems Dave Chinner
@ 2012-07-26 8:39 ` Dave Chinner
2012-08-28 14:03 ` Rich Johnston
2012-08-28 20:04 ` Christoph Hellwig
2012-08-14 21:40 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (2 subsequent siblings)
12 siblings, 2 replies; 38+ messages in thread
From: Dave Chinner @ 2012-07-26 8:39 UTC (permalink / raw)
To: xfs
From: Dave Chinner <dchinner@redhat.com>
The xfsdump/restore tests will see the space filling files and may
try to backthem up and restore them, consuming huge amounts of time
to do (espcially when diffing the results). Exclude the space
filling files by setting the no dump attributes on them and ensure
that xfsdump runs with the -e flag to exclude such files.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common.config | 3 ++-
common.dump | 2 +-
common.rc | 1 +
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/common.config b/common.config
index 7bed1c5..bb9af93 100644
--- a/common.config
+++ b/common.config
@@ -146,7 +146,8 @@ export XFS_GROWFS_PROG=`set_prog_path xfs_growfs`
export XFS_IO_PROG="`set_prog_path xfs_io`"
export XFS_PARALLEL_REPAIR_PROG="`set_prog_path xfs_prepair`"
export XFS_PARALLEL_REPAIR64_PROG="`set_prog_path xfs_prepair64`"
-export XFSDUMP_PROG="`set_prog_path xfsdump`"
+export __XFSDUMP_PROG="`set_prog_path xfsdump`"
+export XFSDUMP_PROG="$__XFSDUMP_PROG -e"
export XFSRESTORE_PROG="`set_prog_path xfsrestore`"
export XFSINVUTIL_PROG="`set_prog_path xfsinvutil`"
export DBENCH_PROG="`set_prog_path dbench`"
diff --git a/common.dump b/common.dump
index e89dedc..a33dc81 100644
--- a/common.dump
+++ b/common.dump
@@ -829,7 +829,7 @@ _dump_filter_main()
{
_filter_devchar |\
sed \
- -e "s#$XFSDUMP_PROG#xfsdump#" \
+ -e "s#$__XFSDUMP_PROG#xfsdump#" \
-e "s#$XFSRESTORE_PROG#xfsrestore#" \
-e "s#$XFSINVUTIL_PROG#xfsinvutil#" \
-e "s/`hostname`/HOSTNAME/" \
diff --git a/common.rc b/common.rc
index 1c3c605..1b23b66 100644
--- a/common.rc
+++ b/common.rc
@@ -306,6 +306,7 @@ _setup_large_xfs_fs()
xfs_io -F -f \
-c "truncate $file_size" \
-c "falloc -k 0 $file_size" \
+ -c "chattr +d" \
$SCRATCH_MNT/.use_space 2>&1 > /dev/null
status=$?
umount $SCRATCH_MNT
--
1.7.10
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 38+ messages in thread
* Re: [PATCH 07/10] xfstests: always us test option when checking large scratch device
2012-07-26 8:39 ` [PATCH 07/10] xfstests: always us test option when checking large scratch device Dave Chinner
@ 2012-07-26 17:21 ` Paulo Alcantara
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:58 ` Christoph Hellwig
2 siblings, 0 replies; 38+ messages in thread
From: Paulo Alcantara @ 2012-07-26 17:21 UTC (permalink / raw)
To: david; +Cc: xfs
Hi Dave,
From: Dave Chinner <david@fromorbit.com>
Date: Thu, 26 Jul 2012 18:39:46 +1000
> From: Dave Chinner <dchinner@redhat.com>
>
> Some tests call _check_scratch_device directly and when using a
> large filesystem this needs to run with a -t option to avoid
> consuming large amounts of memory. Make this happen in all cases
> that the scratch device is checked.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> 017 | 7 ++-----
> common.rc | 2 ++
> 2 files changed, 4 insertions(+), 5 deletions(-)
I guess you meant "xfstests: always use test option..." instead in commit
message.
Paulo
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 0/10] xfstests: rework large filesystem testing
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (9 preceding siblings ...)
2012-07-26 8:39 ` [PATCH 10/10] xfstests: exclude largefs fill files from dump tests Dave Chinner
@ 2012-08-14 21:40 ` Dave Chinner
2012-09-05 21:34 ` [PATCH 11/10] xfstests: rework large filesystem testing - add golden output Ben Myers
2012-09-05 22:26 ` Dave Chinner
12 siblings, 0 replies; 38+ messages in thread
From: Dave Chinner @ 2012-08-14 21:40 UTC (permalink / raw)
To: xfs
ping?
On Thu, Jul 26, 2012 at 06:39:39PM +1000, Dave Chinner wrote:
> This is a resend of a series that has been sitting inmy local
> workarea for months. It reworks the large filesystem testing
> infrastructure for XFS to allow it to use xfs_repair by switching
> the filesystem filling to using preallocation rather than xfs_db
> hacks.
>
> It also introduces support for ext4 testing, also via preallocation
> to fill the filesystem. btrfs isn't yet supported because I haven't
> worked out how to reliably get the size of the filesystem and
> calculate how much space is actually available in it.
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 01/10] xfstests: add --largefs check option
2012-07-26 8:39 ` [PATCH 01/10] xfstests: add --largefs check option Dave Chinner
@ 2012-08-28 14:00 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:00 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Make it easier to check large filesystems quickly by adding a
> --large-fs option to check to turn on shortcuts for large scratch
> device filesystem testing.
>
> Also, reject invalid command line options with a usage message.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common | 63 +++++++++++++++++++++++++++++++++++++++------------------------
> 1 file changed, 39 insertions(+), 24 deletions(-)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic
2012-07-26 8:39 ` [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic Dave Chinner
@ 2012-08-28 14:01 ` Rich Johnston
2012-08-31 23:30 ` Dave Chinner
2012-08-28 19:56 ` Christoph Hellwig
1 sibling, 1 reply; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:01 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
> easily as loop devices. It really means we are testing a large scratch device
> and that we should enable the special filesystem filling and checking options
> that enable xfstests to be run sanely on large XFS filesystems.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> 004 | 2 +-
> 015 | 2 +-
> 030 | 2 +-
> 031 | 2 +-
> 032 | 2 +-
> 033 | 4 ++--
> 041 | 2 +-
> 049 | 2 +-
> 083 | 2 +-
> 092 | 2 +-
> 148 | 2 +-
> 149 | 2 +-
> common | 2 +-
> common.rc | 12 ++++++------
> setup | 5 +++--
> 15 files changed, 23 insertions(+), 22 deletions(-)
>
> MOUNT_OPTIONS="$MOUNT_OPTIONS -o inode64"
> _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
> diff --git a/148 b/148
> index 76cbf37..7bb1722 100755
> --- a/148
> +++ b/148
> @@ -66,8 +66,8 @@ _check_ag()
> _supported_fs xfs
> _supported_os IRIX Linux
#RCJ As long as you are making changes to this file, please remove Linux
from the _supported_os as xfs_prepair64 is an IRIX specific binary.
>
> -_require_nobigloopfs
> _require_scratch
> +_require_no_large_scratch_dev
>
> DSIZE="-dsize=100m"
>
> diff --git a/149 b/149
> index 5131a45..193e6d7 100755
> --- a/149
> +++ b/149
> @@ -100,8 +100,8 @@ EOF
> _supported_fs xfs
> _supported_os IRIX Linux
#RCJ As long as you are making changes to this file, please remove Linux
from the _supported_os as xfs_prepair is an IRIX specific binary.
>
> -_require_nobigloopfs
> _require_scratch
> +_require_no_large_scratch_dev
>
> # sanity test - default + one root directory entry
> # Note: must do this proto/mkfs now for later inode size calcs
Otherwise it looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES
2012-07-26 8:39 ` [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES Dave Chinner
@ 2012-08-28 14:01 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:01 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Rename the $RETAIN_AG_BYTES variable to be more generic so that it
> reflects the fact that it is designed to retain a certain amount of
> extra free space above the default amount in the filesystem when
> doing large scratch device testing.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common.rc | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 04/10] xfstests: use preallocation for ag-wiper
2012-07-26 8:39 ` [PATCH 04/10] xfstests: use preallocation for ag-wiper Dave Chinner
@ 2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:57 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:02 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
[-- Attachment #1: Type: text/plain, Size: 2957 bytes --]
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> To enable sane testing of large scale filesystems, the --large-fs
> test option uses xfs_db magic to mark AGs full without doing any IO.
> This leaves only a small amount of free space left in the filesystem
> to stress the high AGs of the filesystem rather than the low AGs.
>
> This method requires us to have special filesystem check options to
> avoid free space checking in xfs_check, and we cannot current run
> xfs_repair on such a filesystem at all. As it is, free space
> checking on xfs_check does not scale, so we still need to avoid this
> checking regardless of how we fill the filesystem.
>
> We can acheive exactly the same fill behaviour by preallocating a
> single large file in the filesystem immediately after creating it.
> This is a filesystem independent manner of filling the filesystem,
> and allows us to do large filesystem testing on more than just XFS.
>
> Further, this preallocation method effectively adds a new "very
> large file" test. It also enables us to run an unmodified xfs_repair
> or filesystem specific fsck program to check the filesystem for
> sanity, so we can now do full sanity checking of such large
> filesystems.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common.rc | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 53 insertions(+), 5 deletions(-)
>
> diff --git a/common.rc b/common.rc
> index b0af482..7bb1133 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -276,6 +276,47 @@ _scratch_mkfs_options()
> echo $SCRATCH_OPTIONS $MKFS_OPTIONS $* $SCRATCH_DEV
> }
>
> +
> +_setup_large_xfs_fs()
> +{
> + fs_size=$1
> + local tmp_dir=/tmp/
> +
> + [ "$LARGE_SCRATCH_DEV" != yes ] && return 0
> + [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
> + [ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
> +
> + # calculate the size of the file we need to allocate.
> + # Default free space in the FS is 50GB, but you can specify more via
> + # SCRATCH_DEV_EMPTY_SPACE
> + file_size=$(($fs_size - 50*1024*1024*1024))
> + file_size=$(($file_size - $SCRATCH_DEV_EMPTY_SPACE))
> +
> + # mount the filesystem, create the file, unmount it
> + _scratch_mount 2>&1 >$tmp_dir/mnt.err
> + local status=$?
> + if [ $status -ne 0 ]; then
> + echo "mount failed"
> + cat $tmp_dir/mnt.err >&2
> + rm -f $tmp_dir/mnt.err
> + return $status
> + fi
> + rm -f $tmp_dir/mnt.err
> +
> + xfs_io -F -f \
> + -c "truncate $file_size" \
> + -c "falloc -k 0 $file_size" \
> + $SCRATCH_MNT/.use_space 2>&1 > /dev/null
Looks like patch 10 (exclude largefs fill files from dump tests) does
not ignore this file as it should when using xfsdump -e and chattr +d.
The following tests which are all in the auto group fail when using the
--large-fs option:
019 026-028 046 047 050 056 059 060 062-066
I have attached the specific error output.
Regards
--Rich
[-- Attachment #2: patch10.err --]
[-- Type: text/plain, Size: 35601 bytes --]
./check 019 026-028 046 047 050 056 059 060 062-066
FSTYP -- xfs (non-debug)
PLATFORM -- Linux/x86_64 cxfsxe4 3.6.0-rc1-0.9-default
MKFS_OPTIONS -- -f -bsize=4096 /dev/sdc1
MOUNT_OPTIONS -- /dev/sdc1 /xfs_scratch
019 10s
026 21s
027 22s
028 32s
046 20s
047 31s
050 81s
056 20s
059 31s
060 31s
062 10s
063 22s
064 73s
065 51s
066 10s
Ran: 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
Passed all 15 tests
./check --large-fs 019 026-028 046 047 050 056 059 060 062-066
FSTYP -- xfs (non-debug)
PLATFORM -- Linux/x86_64 cxfsxe4 3.6.0-rc1-0.9-default
MKFS_OPTIONS -- -f -bsize=4096 /dev/sdc1
MOUNT_OPTIONS -- /dev/sdc1 /xfs_scratch
019 10s ... - output mismatch (see 019.out.bad)
--- 019.out 2012-08-15 15:01:39.000000000 -0500
+++ 019.out.bad 2012-08-28 08:38:08.000000000 -0500
@@ -9,6 +9,11 @@
Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
Device: <DEVICE> Inode: <INODE> Links: 3
+ File: "./.use_space"
+ Size: 6312890368 Filetype: Regular File
+ Mode: (0600/-rw-------) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1
+
File: "./bigfile"
Size: 2097152 Filetype: Regular File
Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
026 21s ... - output mismatch (see 026.out.bad)
--- 026.out 2012-08-15 15:01:39.000000000 -0500
+++ 026.out.bad 2012-08-28 08:38:34.000000000 -0500
@@ -9,6 +9,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_026"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -28,12 +29,13 @@
xfsrestore: searching media for directory dump
xfsrestore: examining media file 0
xfsrestore: reading directories
-xfsrestore: 3 directories and 38 entries processed
+xfsrestore: 3 directories and 39 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
xfsrestore: Restore Status: SUCCESS
Comparing dump directory with restore directory
+Only in SCRATCH_MNT: .use_space
Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical
Files DUMP_DIR/small and RESTORE_DIR/DUMP_SUBDIR/small are identical
Files DUMP_DIR/sub/a and RESTORE_DIR/DUMP_SUBDIR/sub/a are identical
027 22s ... ... - output mismatch (see 027.out.bad)
--- 027.out 2012-08-15 15:01:39.000000000 -0500
+++ 027.out.bad 2012-08-28 08:39:00.000000000 -0500
@@ -19,7 +19,7 @@
xfsrestore: media ID: ID
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 3 directories and 39 entries processed
+xfsrestore: 3 directories and 40 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
028 32s ... - output mismatch (see 028.out.bad)
--- 028.out 2012-08-15 15:01:39.000000000 -0500
+++ 028.out.bad 2012-08-28 08:39:36.000000000 -0500
@@ -7,6 +7,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.0"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -27,6 +28,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.1"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -46,6 +48,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.2"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -65,6 +68,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.3"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -84,6 +88,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.4"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
046 20s ... - output mismatch (see 046.out.bad)
--- 046.out 2012-08-15 15:01:39.000000000 -0500
+++ 046.out.bad 2012-08-28 08:40:00.000000000 -0500
@@ -9,6 +9,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_046"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -28,7 +29,7 @@
xfsrestore: searching media for directory dump
xfsrestore: examining media file 0
xfsrestore: reading directories
-xfsrestore: 3 directories and 10 entries processed
+xfsrestore: 3 directories and 11 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
047 31s ... - output mismatch (see 047.out.bad)
--- 047.out 2012-08-15 15:01:39.000000000 -0500
+++ 047.out.bad 2012-08-28 08:40:37.000000000 -0500
@@ -7,6 +7,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.0"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -27,6 +28,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.1"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -46,6 +48,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.2"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -65,6 +68,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.3"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -84,6 +88,7 @@
xfsdump: session id: ID
xfsdump: session label: "session.4"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: creating dump session media file 0 (media 0, file 0)
050 81s ... - output mismatch (see 050.out.bad)
--- 050.out 2012-08-15 15:01:39.000000000 -0500
+++ 050.out.bad 2012-08-28 08:42:03.000000000 -0500
@@ -8,26 +8,26 @@
realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
*** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
*** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [7 days] 4 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
*** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] =OK= 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
*** unmount
@@ -40,26 +40,26 @@
realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
*** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
*** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [7 days] 4 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
*** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] =OK= 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
*** unmount
@@ -72,26 +72,26 @@
realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
*** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
*** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [--------] 4 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [--------] 16 4 10 00 [--none--] 0 0 0 00 [--------]
*** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 540 100 500 00 [--none--] 16 4 10 00 [--none--] 0 0 0 00 [--------]
*** unmount
@@ -104,26 +104,26 @@
realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
*** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
*** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [--------] 4 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [--------] 16 4 10 00 [--none--] 0 0 0 00 [--------]
*** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 540 100 500 00 [--none--] 16 4 10 00 [--none--] 0 0 0 00 [--------]
*** unmount
@@ -136,26 +136,26 @@
realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
*** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
*** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [7 days] 4 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [7 days] 9 4 10 00 [7 days] 0 0 0 00 [--------]
*** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] =OK= 100 500 00 [7 days] 9 4 10 00 [7 days] 0 0 0 00 [--------]
*** unmount
@@ -168,26 +168,26 @@
realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
*** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
*** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [--------] 4 4 10 00 [--------] 0 0 0 00 [--------]
*** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 140 100 500 00 [--------] 16 4 10 00 [--none--] 0 0 0 00 [--------]
*** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
[NAME] 540 100 500 00 [--none--] 16 4 10 00 [--none--] 0 0 0 00 [--------]
*** unmount
056 20s ... - output mismatch (see 056.out.bad)
--- 056.out 2012-08-15 15:01:39.000000000 -0500
+++ 056.out.bad 2012-08-28 08:42:28.000000000 -0500
@@ -9,6 +9,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_056"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -28,7 +29,7 @@
xfsrestore: searching media for directory dump
xfsrestore: examining media file 0
xfsrestore: reading directories
-xfsrestore: 7 directories and 11 entries processed
+xfsrestore: 7 directories and 12 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
059 31s ... - output mismatch (see 059.out.bad)
--- 059.out 2012-08-15 15:01:39.000000000 -0500
+++ 059.out.bad 2012-08-28 08:43:01.000000000 -0500
@@ -12,6 +12,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_059"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: stream 0: ino INO offset NUM to ino INO offset NUM
xfsdump: stream 1: ino INO offset NUM to ino INO offset NUM
xfsdump: stream 2: ino INO offset NUM to ino INO offset NUM
@@ -34,6 +35,7 @@
Comparing listing of dump directory with restore directory
Files TMP.dump_dir and TMP.restore_dir are identical
Comparing dump directory with restore directory
+Only in SCRATCH_MNT: .use_space
Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical
Files DUMP_DIR/large000 and RESTORE_DIR/DUMP_SUBDIR/large000 are identical
Files DUMP_DIR/large111 and RESTORE_DIR/DUMP_SUBDIR/large111 are identical
060 31s ... - output mismatch (see 060.out.bad)
--- 060.out 2012-08-15 15:01:39.000000000 -0500
+++ 060.out.bad 2012-08-28 08:43:33.000000000 -0500
@@ -12,6 +12,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_060"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: stream 0: ino INO offset NUM to ino INO offset NUM
xfsdump: stream 1: ino INO offset NUM to ino INO offset NUM
xfsdump: stream 2: ino INO offset NUM to ino INO offset NUM
@@ -42,7 +43,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 3 directories and 41 entries processed
+xfsrestore: 3 directories and 42 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: NOTE: restore interrupted: SECS seconds elapsed: may resume later using -R option
@@ -77,6 +78,7 @@
Comparing listing of dump directory with restore directory
Files TMP.dump_dir and TMP.restore_dir are identical
Comparing dump directory with restore directory
+Only in SCRATCH_MNT: .use_space
Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical
Files DUMP_DIR/large000 and RESTORE_DIR/DUMP_SUBDIR/large000 are identical
Files DUMP_DIR/large111 and RESTORE_DIR/DUMP_SUBDIR/large111 are identical
062 10s ... - output mismatch (see 062.out.bad)
--- 062.out 2012-08-15 15:01:39.000000000 -0500
+++ 062.out.bad 2012-08-28 08:43:47.000000000 -0500
@@ -1,6 +1,7 @@
QA output created by 062
*** create test bed
SCRATCH_MNT
+SCRATCH_MNT/.use_space
SCRATCH_MNT/dev
SCRATCH_MNT/dev/b
SCRATCH_MNT/dev/c
@@ -492,6 +493,7 @@
*** extend test bed
SCRATCH_MNT
+SCRATCH_MNT/.use_space
SCRATCH_MNT/descend
SCRATCH_MNT/descend/and
SCRATCH_MNT/descend/down
063 22s ... - output mismatch (see 063.out.bad)
--- 063.out 2012-08-15 15:01:39.000000000 -0500
+++ 063.out.bad 2012-08-28 08:44:13.000000000 -0500
@@ -19,6 +19,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_063"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -38,7 +39,7 @@
xfsrestore: searching media for directory dump
xfsrestore: examining media file 0
xfsrestore: reading directories
-xfsrestore: 4 directories and 21 entries processed
+xfsrestore: 4 directories and 22 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
064 73s ... - output mismatch (see 064.out.bad)
--- 064.out 2012-08-15 15:01:39.000000000 -0500
+++ 064.out.bad 2012-08-28 08:45:31.000000000 -0500
@@ -54,6 +54,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_064"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -314,7 +315,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -395,7 +396,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -428,7 +429,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -461,7 +462,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -494,7 +495,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -527,7 +528,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -560,7 +561,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -593,7 +594,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -626,7 +627,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -659,7 +660,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -693,7 +694,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -774,7 +775,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -855,7 +856,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -936,7 +937,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -1017,7 +1018,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -1098,7 +1099,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -1179,7 +1180,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -1260,7 +1261,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -1341,7 +1342,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -1422,7 +1423,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
065 51s ... - output mismatch (see 065.out.bad)
--- 065.out 2012-08-15 15:01:39.000000000 -0500
+++ 065.out.bad 2012-08-28 08:46:26.000000000 -0500
@@ -19,6 +19,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_065"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -257,7 +258,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 6 directories and 11 entries processed
+xfsrestore: 6 directories and 12 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -290,7 +291,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 6 entries processed
+xfsrestore: 2 directories and 7 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -317,7 +318,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 4 directories and 7 entries processed
+xfsrestore: 4 directories and 8 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -346,7 +347,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 3 directories and 11 entries processed
+xfsrestore: 3 directories and 12 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -380,7 +381,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 3 directories and 7 entries processed
+xfsrestore: 3 directories and 8 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -410,7 +411,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -441,7 +442,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
xfsrestore: directory post-processing
xfsrestore: reading non-directory files
xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -525,7 +526,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 6 directories and 11 entries processed
+xfsrestore: 6 directories and 12 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -562,7 +563,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 6 entries processed
+xfsrestore: 2 directories and 7 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -595,7 +596,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 4 directories and 7 entries processed
+xfsrestore: 4 directories and 8 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -628,7 +629,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 3 directories and 11 entries processed
+xfsrestore: 3 directories and 12 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -665,7 +666,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 3 directories and 7 entries processed
+xfsrestore: 3 directories and 8 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -698,7 +699,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
@@ -729,7 +730,7 @@
xfsrestore: using online session inventory
xfsrestore: searching media for directory dump
xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
066 10s ... - output mismatch (see 066.out.bad)
--- 066.out 2012-08-15 15:01:39.000000000 -0500
+++ 066.out.bad 2012-08-28 08:46:41.000000000 -0500
@@ -12,6 +12,7 @@
xfsdump: session id: ID
xfsdump: session label: "stress_066"
xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
xfsdump: ino map construction complete
xfsdump: estimated dump size: NUM bytes
xfsdump: /var/xfsdump/inventory created
@@ -31,7 +32,7 @@
xfsrestore: searching media for directory dump
xfsrestore: examining media file 0
xfsrestore: reading directories
-xfsrestore: 2 directories and 2 entries processed
+xfsrestore: 2 directories and 3 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: SECS seconds elapsed
Ran: 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
Failures: 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
Failed 15 of 15 tests
[-- Attachment #3: Type: text/plain, Size: 121 bytes --]
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 05/10] xfstests: use command line option for setting extra space
2012-07-26 8:39 ` [PATCH 05/10] xfstests: use command line option for setting extra space Dave Chinner
@ 2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:57 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:02 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Allow the extra free space to leave in large scratch filesystems to
> be specified by a command line option rather than just via an
> environment variable.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common | 5 +++++
> 1 file changed, 5 insertions(+)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing
2012-07-26 8:39 ` [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing Dave Chinner
@ 2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:58 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:02 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Now that large filesystem testing does not play free space games to
> fill the space without IO, we can enable xfs_repair when running in
> this mode. xfs_repair has had it's scalability problems solved, too,
> so this is a safe thing to do.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common.rc | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 07/10] xfstests: always us test option when checking large scratch device
2012-07-26 8:39 ` [PATCH 07/10] xfstests: always us test option when checking large scratch device Dave Chinner
2012-07-26 17:21 ` Paulo Alcantara
@ 2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:58 ` Christoph Hellwig
2 siblings, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:02 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Some tests call _check_scratch_device directly and when using a
> large filesystem this needs to run with a -t option to avoid
> consuming large amounts of memory. Make this happen in all cases
> that the scratch device is checked.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> 017 | 7 ++-----
> common.rc | 2 ++
> 2 files changed, 4 insertions(+), 5 deletions(-)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems
2012-07-26 8:39 ` [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems Dave Chinner
@ 2012-08-28 14:03 ` Rich Johnston
2012-08-28 20:03 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:03 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Test 017 typically fails due to the OOM killer kicking in and
> killing everything but xfs_db so panics the machine.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> 017 | 4 ++++
> 1 file changed, 4 insertions(+)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 10/10] xfstests: exclude largefs fill files from dump tests
2012-07-26 8:39 ` [PATCH 10/10] xfstests: exclude largefs fill files from dump tests Dave Chinner
@ 2012-08-28 14:03 ` Rich Johnston
2012-08-28 20:04 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-08-28 14:03 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> The xfsdump/restore tests will see the space filling files and may
> try to backthem up and restore them, consuming huge amounts of time
> to do (espcially when diffing the results). Exclude the space
> filling files by setting the no dump attributes on them and ensure
> that xfsdump runs with the -e flag to exclude such files.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common.config | 3 ++-
> common.dump | 2 +-
> common.rc | 1 +
> 3 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/common.config b/common.config
> index 7bed1c5..bb9af93 100644
> --- a/common.config
> +++ b/common.config
> @@ -146,7 +146,8 @@ export XFS_GROWFS_PROG=`set_prog_path xfs_growfs`
> export XFS_IO_PROG="`set_prog_path xfs_io`"
> export XFS_PARALLEL_REPAIR_PROG="`set_prog_path xfs_prepair`"
> export XFS_PARALLEL_REPAIR64_PROG="`set_prog_path xfs_prepair64`"
> -export XFSDUMP_PROG="`set_prog_path xfsdump`"
> +export __XFSDUMP_PROG="`set_prog_path xfsdump`"
> +export XFSDUMP_PROG="$__XFSDUMP_PROG -e"
> export XFSRESTORE_PROG="`set_prog_path xfsrestore`"
> export XFSINVUTIL_PROG="`set_prog_path xfsinvutil`"
> export DBENCH_PROG="`set_prog_path dbench`"
> diff --git a/common.dump b/common.dump
> index e89dedc..a33dc81 100644
> --- a/common.dump
> +++ b/common.dump
> @@ -829,7 +829,7 @@ _dump_filter_main()
> {
> _filter_devchar |\
> sed \
> - -e "s#$XFSDUMP_PROG#xfsdump#" \
> + -e "s#$__XFSDUMP_PROG#xfsdump#" \
> -e "s#$XFSRESTORE_PROG#xfsrestore#" \
> -e "s#$XFSINVUTIL_PROG#xfsinvutil#" \
> -e "s/`hostname`/HOSTNAME/" \
> diff --git a/common.rc b/common.rc
> index 1c3c605..1b23b66 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -306,6 +306,7 @@ _setup_large_xfs_fs()
> xfs_io -F -f \
> -c "truncate $file_size" \
> -c "falloc -k 0 $file_size" \
> + -c "chattr +d" \
> $SCRATCH_MNT/.use_space 2>&1 > /dev/null
> status=$?
> umount $SCRATCH_MNT
>
Ditto my comments from patch 4, this does not work as expected when
using the --large-fs option for the following tests:
019 026-028 046 047 050 056 059 060 062-066
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 01/10] xfstests: add --largefs check option
2012-07-26 8:39 ` [PATCH 01/10] xfstests: add --largefs check option Dave Chinner
2012-08-28 14:00 ` Rich Johnston
@ 2012-08-28 19:56 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:56 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, Jul 26, 2012 at 06:39:40PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Make it easier to check large filesystems quickly by adding a
> --large-fs option to check to turn on shortcuts for large scratch
> device filesystem testing.
>
> Also, reject invalid command line options with a usage message.
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic
2012-07-26 8:39 ` [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic Dave Chinner
2012-08-28 14:01 ` Rich Johnston
@ 2012-08-28 19:56 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:56 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, Jul 26, 2012 at 06:39:41PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
> easily as loop devices. It really means we are testing a large scratch device
> and that we should enable the special filesystem filling and checking options
> that enable xfstests to be run sanely on large XFS filesystems.
Looks fine, even if I don't like the new name too much either.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES
2012-07-26 8:39 ` [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES Dave Chinner
2012-08-28 14:01 ` Rich Johnston
@ 2012-08-28 19:56 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:56 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, Jul 26, 2012 at 06:39:42PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Rename the $RETAIN_AG_BYTES variable to be more generic so that it
> reflects the fact that it is designed to retain a certain amount of
> extra free space above the default amount in the filesystem when
> doing large scratch device testing.
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 04/10] xfstests: use preallocation for ag-wiper
2012-07-26 8:39 ` [PATCH 04/10] xfstests: use preallocation for ag-wiper Dave Chinner
2012-08-28 14:02 ` Rich Johnston
@ 2012-08-28 19:57 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:57 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 05/10] xfstests: use command line option for setting extra space
2012-07-26 8:39 ` [PATCH 05/10] xfstests: use command line option for setting extra space Dave Chinner
2012-08-28 14:02 ` Rich Johnston
@ 2012-08-28 19:57 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:57 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing
2012-07-26 8:39 ` [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing Dave Chinner
2012-08-28 14:02 ` Rich Johnston
@ 2012-08-28 19:58 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:58 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 07/10] xfstests: always us test option when checking large scratch device
2012-07-26 8:39 ` [PATCH 07/10] xfstests: always us test option when checking large scratch device Dave Chinner
2012-07-26 17:21 ` Paulo Alcantara
2012-08-28 14:02 ` Rich Johnston
@ 2012-08-28 19:58 ` Christoph Hellwig
2 siblings, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 19:58 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Looks good modulo the subject typo pointed our earlier.
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 08/10] xfstests: enable large fs testing on ext4
2012-07-26 8:39 ` [PATCH 08/10] xfstests: enable large fs testing on ext4 Dave Chinner
@ 2012-08-28 20:03 ` Christoph Hellwig
2012-09-05 16:27 ` Rich Johnston
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 20:03 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems
2012-07-26 8:39 ` [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems Dave Chinner
2012-08-28 14:03 ` Rich Johnston
@ 2012-08-28 20:03 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 20:03 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 10/10] xfstests: exclude largefs fill files from dump tests
2012-07-26 8:39 ` [PATCH 10/10] xfstests: exclude largefs fill files from dump tests Dave Chinner
2012-08-28 14:03 ` Rich Johnston
@ 2012-08-28 20:04 ` Christoph Hellwig
1 sibling, 0 replies; 38+ messages in thread
From: Christoph Hellwig @ 2012-08-28 20:04 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, Jul 26, 2012 at 06:39:49PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> The xfsdump/restore tests will see the space filling files and may
> try to backthem up and restore them, consuming huge amounts of time
> to do (espcially when diffing the results). Exclude the space
> filling files by setting the no dump attributes on them and ensure
> that xfsdump runs with the -e flag to exclude such files.
Looks reasonable,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic
2012-08-28 14:01 ` Rich Johnston
@ 2012-08-31 23:30 ` Dave Chinner
0 siblings, 0 replies; 38+ messages in thread
From: Dave Chinner @ 2012-08-31 23:30 UTC (permalink / raw)
To: Rich Johnston; +Cc: xfs
On Tue, Aug 28, 2012 at 09:01:35AM -0500, Rich Johnston wrote:
> On 07/26/2012 03:39 AM, Dave Chinner wrote:
> >From: Dave Chinner <dchinner@redhat.com>
> >
> >USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
> >easily as loop devices. It really means we are testing a large scratch device
> >and that we should enable the special filesystem filling and checking options
> >that enable xfstests to be run sanely on large XFS filesystems.
> >
> >Signed-off-by: Dave Chinner <dchinner@redhat.com>
> >---
> > 004 | 2 +-
> > 015 | 2 +-
> > 030 | 2 +-
> > 031 | 2 +-
> > 032 | 2 +-
> > 033 | 4 ++--
> > 041 | 2 +-
> > 049 | 2 +-
> > 083 | 2 +-
> > 092 | 2 +-
> > 148 | 2 +-
> > 149 | 2 +-
> > common | 2 +-
> > common.rc | 12 ++++++------
> > setup | 5 +++--
> > 15 files changed, 23 insertions(+), 22 deletions(-)
> >
>
> > MOUNT_OPTIONS="$MOUNT_OPTIONS -o inode64"
> > _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
> >diff --git a/148 b/148
> >index 76cbf37..7bb1722 100755
> >--- a/148
> >+++ b/148
> >@@ -66,8 +66,8 @@ _check_ag()
> > _supported_fs xfs
> > _supported_os IRIX Linux
>
> #RCJ As long as you are making changes to this file, please remove
> Linux from the _supported_os as xfs_prepair64 is an IRIX specific
> binary.
Such a change belongs in a separate patch, not intermingled with a
patch in a series that has nothing to do with xfs_prepair64....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 08/10] xfstests: enable large fs testing on ext4
2012-07-26 8:39 ` [PATCH 08/10] xfstests: enable large fs testing on ext4 Dave Chinner
2012-08-28 20:03 ` Christoph Hellwig
@ 2012-09-05 16:27 ` Rich Johnston
1 sibling, 0 replies; 38+ messages in thread
From: Rich Johnston @ 2012-09-05 16:27 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Now that setting up large filesystem testing on sparse loopback
> devices uses a generic method for filling the filesystem, extent
> support to ext4 filesystems.
>
> ext4 is slightly more complex to fill as it does not support files
> larger than 16TB. Hence a slightly more complex method of using
> multiple smaller files to fill the space is necessary.
>
> WARNING: be prepeared for ext4 to take forever to prepare large
> filesystems as allocation of large amounts of space (especially as
> it approaches ENOSPC) is can take minutes rather than milliseconds
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
> common.rc | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 90 insertions(+)
>
Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Regards
--Rich
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (10 preceding siblings ...)
2012-08-14 21:40 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
@ 2012-09-05 21:34 ` Ben Myers
2012-09-05 22:26 ` Dave Chinner
12 siblings, 0 replies; 38+ messages in thread
From: Ben Myers @ 2012-09-05 21:34 UTC (permalink / raw)
To: rjohnston; +Cc: xfs
Hey Rich,
On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
> Patch "rework large filesystem testing" introduces a new option --large-fs
> which creates a new file $SCRATCH_MNT/.use_space. If this 10 part patchset is
> applied, the following tests will fail:
> 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
>
> This patch accounts for the following new output when testing xfs filesystems with
> the --large-fs option by creating new output file to compare against
> ($seq.largefs.out):
>
> 1. The following four lines appear in test 019.
> File: "./.use_space"
> Size: 6312890368 Filetype: Regular File
> Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> Device: <DEVICE> Inode: <INODE> Links: 1
>
> 2. When the nodump attribute is set, the xfsdump -e option will cause the
> following additional lines to appear.
> xfsdump: NOTE: pruned 1 files: skip attribute set
> Only in SCRATCH_MNT: .use_space
> SCRATCH_MNT/.use_space
>
> 3. Number of files off by one.
> xfsrestore: # directories and (off by 1) entries processed
> [ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
>
> This patch also modifies check and common.quota to use the new output file
> $seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
> or $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
>
> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
This looks pretty good to me. .use_space changed some golden output and you've
fixed that up by adding additional output files for when this --large-fs option
is used. Dave will probably want to take a look too.
Reviewed-by: Ben Myers <bpm@sgi.com>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
` (11 preceding siblings ...)
2012-09-05 21:34 ` [PATCH 11/10] xfstests: rework large filesystem testing - add golden output Ben Myers
@ 2012-09-05 22:26 ` Dave Chinner
2012-09-06 12:57 ` Rich Johnston
12 siblings, 1 reply; 38+ messages in thread
From: Dave Chinner @ 2012-09-05 22:26 UTC (permalink / raw)
To: rjohnston; +Cc: xfs
On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
> Patch "rework large filesystem testing" introduces a new option --large-fs
> which creates a new file $SCRATCH_MNT/.use_space. If this 10 part patchset is
> applied, the following tests will fail:
> 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
That's a lot more tests than I see failing.
> This patch accounts for the following new output when testing xfs filesystems with
> the --large-fs option by creating new output file to compare against
> ($seq.largefs.out):
Creating new output files is the absolute last resort. Indeed, what
happens when you get different output for tests that already select
an output file based on, say, platform or some other criteria? We
get a combinatorial explosion of golden output files, and that is
simply not manageable.
The usual thing to do is update the necessary filters or change the
way the tests run to avoid trivial output file differences e.g. use
a subdir rather than SCRATCH_MNT directly. Or, for example the
filters that munge different standard error messages from different
platforms to be the same...
> 1. The following four lines appear in test 019.
> File: "./.use_space"
> Size: 6312890368 Filetype: Regular File
> Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> Device: <DEVICE> Inode: <INODE> Links: 1
This test doesn't really need to be run for large filesystems -
running it on large filesystems doesn't improve the coverage of or
our confidence in the code it is testing, so I'd just add a
_require_no_large_scratch_dev to it.
> 2. When the nodump attribute is set, the xfsdump -e option will cause the
> following additional lines to appear.
> xfsdump: NOTE: pruned 1 files: skip attribute set
> Only in SCRATCH_MNT: .use_space
> SCRATCH_MNT/.use_space
Ok, those are the errors I haven't seen - not sure why. I'll have to
look into that.
However, this is definitely a case of updating the dump output
filter to remove these messages from the output stream. The
alternative is to change the common dump code to use a subdirectory
rather than the root directory so it doesn't see these files at all.
> 3. Number of files off by one.
> xfsrestore: # directories and (off by 1) entries processed
That would be fixed by using a subdir for the dump tests. I don't
recommend that the number should be filtered, as having dump report
the correct number of files scanned is important.
> [ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
Perhaps the usre/group of the use_space file needs to be changed so
it doesn't impact on the test results. Alternatively, a filter could
be written/modified to fix the number appropriately.
> This patch also modifies check and common.quota to use the new output file
> $seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
> or $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
>
> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
>
> ---
> 019.largefs.out | 5 +++
> 026.largefs.out | 4 ++-
> 027.largefs.out | 2 -
> 028.largefs.out | 5 +++
> 046.largefs.out | 3 +-
> 047.largefs.out | 5 +++
> 050.largefs.out | 72 ++++++++++++++++++++++++++++----------------------------
> 056.largefs.out | 3 +-
> 059.largefs.out | 2 +
> 060.largefs.out | 4 ++-
> 062.largefs.out | 2 +
> 063.largefs.out | 3 +-
> 064.largefs.out | 41 ++++++++++++++++---------------
> 065.largefs.out | 29 +++++++++++-----------
> 066.largefs.out | 3 +-
> check | 12 +++++++--
> common.quota | 20 ++++++++++-----
> 17 files changed, 128 insertions(+), 87 deletions(-)
FWIW, this patch is supposed to add these *.largefs.out files, right? The
patch, however:
> Index: b/019.largefs.out
> ===================================================================
> --- a/019.largefs.out
> +++ b/019.largefs.out
> @@ -9,6 +9,11 @@ Wrote 2048.00Kb (value 0x2c)
> Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> Device: <DEVICE> Inode: <INODE> Links: 3
>
> + File: "./.use_space"
> + Size: 6312890368 Filetype: Regular File
> + Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1
> +
> File: "./bigfile"
> Size: 2097152 Filetype: Regular File
> Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
... assumes they already exist...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output
2012-09-05 22:26 ` Dave Chinner
@ 2012-09-06 12:57 ` Rich Johnston
2012-09-06 23:07 ` Dave Chinner
0 siblings, 1 reply; 38+ messages in thread
From: Rich Johnston @ 2012-09-06 12:57 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
Thanks for the comments.
On 09/05/2012 05:26 PM, Dave Chinner wrote:
> On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
>> Patch "rework large filesystem testing" introduces a new option --large-fs
>> which creates a new file $SCRATCH_MNT/.use_space. If this 10 part patchset is
>> applied, the following tests will fail:
>> 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
>
> That's a lot more tests than I see failing.
It is very repeatable for me.
>
>> This patch accounts for the following new output when testing xfs filesystems with
>> the --large-fs option by creating new output file to compare against
>> ($seq.largefs.out):
>
> Creating new output files is the absolute last resort. Indeed, what
> happens when you get different output for tests that already select
> an output file based on, say, platform or some other criteria? We
> get a combinatorial explosion of golden output files, and that is
> simply not manageable.
>
> The usual thing to do is update the necessary filters or change the
> way the tests run to avoid trivial output file differences e.g. use
> a subdir rather than SCRATCH_MNT directly. Or, for example the
> filters that munge different standard error messages from different
> platforms to be the same...
>
OK good to know.
>> 1. The following four lines appear in test 019.
>> File: "./.use_space"
>> Size: 6312890368 Filetype: Regular File
>> Mode: (0600/-rw-------) Uid: (0) Gid: (0)
>> Device: <DEVICE> Inode: <INODE> Links: 1
>
> This test doesn't really need to be run for large filesystems -
> running it on large filesystems doesn't improve the coverage of or
> our confidence in the code it is testing, so I'd just add a
> _require_no_large_scratch_dev to it.
>
Works for me.
>> 2. When the nodump attribute is set, the xfsdump -e option will cause the
>> following additional lines to appear.
>> xfsdump: NOTE: pruned 1 files: skip attribute set
>> Only in SCRATCH_MNT: .use_space
>> SCRATCH_MNT/.use_space
>
> Ok, those are the errors I haven't seen - not sure why. I'll have to
> look into that.
>
> However, this is definitely a case of updating the dump output
> filter to remove these messages from the output stream. The
> alternative is to change the common dump code to use a subdirectory
> rather than the root directory so it doesn't see these files at all.
>
Good suggestion
>> 3. Number of files off by one.
>> xfsrestore: # directories and (off by 1) entries processed
>
> That would be fixed by using a subdir for the dump tests. I don't
> recommend that the number should be filtered, as having dump report
> the correct number of files scanned is important.
I agree.
>
>> [ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
>
> Perhaps the usre/group of the use_space file needs to be changed so
> it doesn't impact on the test results. Alternatively, a filter could
> be written/modified to fix the number appropriately.
Sounds reasonable.
>
>> This patch also modifies check and common.quota to use the new output file
>> $seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
>> or $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
>>
>> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
>>
>> ---
>> 019.largefs.out | 5 +++
>> 026.largefs.out | 4 ++-
>> 027.largefs.out | 2 -
>> 028.largefs.out | 5 +++
>> 046.largefs.out | 3 +-
>> 047.largefs.out | 5 +++
>> 050.largefs.out | 72 ++++++++++++++++++++++++++++----------------------------
>> 056.largefs.out | 3 +-
>> 059.largefs.out | 2 +
>> 060.largefs.out | 4 ++-
>> 062.largefs.out | 2 +
>> 063.largefs.out | 3 +-
>> 064.largefs.out | 41 ++++++++++++++++---------------
>> 065.largefs.out | 29 +++++++++++-----------
>> 066.largefs.out | 3 +-
>> check | 12 +++++++--
>> common.quota | 20 ++++++++++-----
>> 17 files changed, 128 insertions(+), 87 deletions(-)
>
> FWIW, this patch is supposed to add these *.largefs.out files, right? The
> patch, however:
>
>> Index: b/019.largefs.out
>> ===================================================================
>> --- a/019.largefs.out
>> +++ b/019.largefs.out
>> @@ -9,6 +9,11 @@ Wrote 2048.00Kb (value 0x2c)
>> Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
>> Device: <DEVICE> Inode: <INODE> Links: 3
>>
>> + File: "./.use_space"
>> + Size: 6312890368 Filetype: Regular File
>> + Mode: (0600/-rw-------) Uid: (0) Gid: (0)
>> +Device: <DEVICE> Inode: <INODE> Links: 1
>> +
>> File: "./bigfile"
>> Size: 2097152 Filetype: Regular File
>> Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
>
> ... assumes they already exist...
>
Yup my bad, I only posted the differences from the original *.out files.
May I make the suggested changes, or as this is your patchset do you
want to make them?
Regards,
--Rich
> Cheers,
>
> Dave.
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output
2012-09-06 12:57 ` Rich Johnston
@ 2012-09-06 23:07 ` Dave Chinner
0 siblings, 0 replies; 38+ messages in thread
From: Dave Chinner @ 2012-09-06 23:07 UTC (permalink / raw)
To: Rich Johnston; +Cc: xfs
On Thu, Sep 06, 2012 at 07:57:18AM -0500, Rich Johnston wrote:
> Thanks for the comments.
>
> On 09/05/2012 05:26 PM, Dave Chinner wrote:
> >On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
> >>Patch "rework large filesystem testing" introduces a new option --large-fs
> >>which creates a new file $SCRATCH_MNT/.use_space. If this 10 part patchset is
> >>applied, the following tests will fail:
> >> 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
> >
> >That's a lot more tests than I see failing.
>
> It is very repeatable for me.
>
> >
> >>This patch accounts for the following new output when testing xfs filesystems with
> >>the --large-fs option by creating new output file to compare against
> >>($seq.largefs.out):
> >
> >Creating new output files is the absolute last resort. Indeed, what
> >happens when you get different output for tests that already select
> >an output file based on, say, platform or some other criteria? We
> >get a combinatorial explosion of golden output files, and that is
> >simply not manageable.
> >
> >The usual thing to do is update the necessary filters or change the
> >way the tests run to avoid trivial output file differences e.g. use
> >a subdir rather than SCRATCH_MNT directly. Or, for example the
> >filters that munge different standard error messages from different
> >platforms to be the same...
> >
>
> OK good to know.
>
> >>1. The following four lines appear in test 019.
> >> File: "./.use_space"
> >> Size: 6312890368 Filetype: Regular File
> >> Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> >> Device: <DEVICE> Inode: <INODE> Links: 1
> >
> >This test doesn't really need to be run for large filesystems -
> >running it on large filesystems doesn't improve the coverage of or
> >our confidence in the code it is testing, so I'd just add a
> >_require_no_large_scratch_dev to it.
> >
>
> Works for me.
>
> >>2. When the nodump attribute is set, the xfsdump -e option will cause the
> >> following additional lines to appear.
> >> xfsdump: NOTE: pruned 1 files: skip attribute set
> >> Only in SCRATCH_MNT: .use_space
> >> SCRATCH_MNT/.use_space
> >
> >Ok, those are the errors I haven't seen - not sure why. I'll have to
> >look into that.
> >
> >However, this is definitely a case of updating the dump output
> >filter to remove these messages from the output stream. The
> >alternative is to change the common dump code to use a subdirectory
> >rather than the root directory so it doesn't see these files at all.
> >
>
> Good suggestion
>
> >>3. Number of files off by one.
> >> xfsrestore: # directories and (off by 1) entries processed
> >
> >That would be fixed by using a subdir for the dump tests. I don't
> >recommend that the number should be filtered, as having dump report
> >the correct number of files scanned is important.
>
> I agree.
>
> >
> >> [ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
> >
> >Perhaps the usre/group of the use_space file needs to be changed so
> >it doesn't impact on the test results. Alternatively, a filter could
> >be written/modified to fix the number appropriately.
>
> Sounds reasonable.
>
> >
> >>This patch also modifies check and common.quota to use the new output file
> >>$seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
> >>or $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
> >>
> >>Signed-off-by: Rich Johnston <rjohnston@sgi.com>
> >>
> >>---
> >> 019.largefs.out | 5 +++
> >> 026.largefs.out | 4 ++-
> >> 027.largefs.out | 2 -
> >> 028.largefs.out | 5 +++
> >> 046.largefs.out | 3 +-
> >> 047.largefs.out | 5 +++
> >> 050.largefs.out | 72 ++++++++++++++++++++++++++++----------------------------
> >> 056.largefs.out | 3 +-
> >> 059.largefs.out | 2 +
> >> 060.largefs.out | 4 ++-
> >> 062.largefs.out | 2 +
> >> 063.largefs.out | 3 +-
> >> 064.largefs.out | 41 ++++++++++++++++---------------
> >> 065.largefs.out | 29 +++++++++++-----------
> >> 066.largefs.out | 3 +-
> >> check | 12 +++++++--
> >> common.quota | 20 ++++++++++-----
> >> 17 files changed, 128 insertions(+), 87 deletions(-)
> >
> >FWIW, this patch is supposed to add these *.largefs.out files, right? The
> >patch, however:
> >
> >>Index: b/019.largefs.out
> >>===================================================================
> >>--- a/019.largefs.out
> >>+++ b/019.largefs.out
> >>@@ -9,6 +9,11 @@ Wrote 2048.00Kb (value 0x2c)
> >> Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> >> Device: <DEVICE> Inode: <INODE> Links: 3
> >>
> >>+ File: "./.use_space"
> >>+ Size: 6312890368 Filetype: Regular File
> >>+ Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> >>+Device: <DEVICE> Inode: <INODE> Links: 1
> >>+
> >> File: "./bigfile"
> >> Size: 2097152 Filetype: Regular File
> >> Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
> >
> >... assumes they already exist...
> >
>
> Yup my bad, I only posted the differences from the original *.out files.
>
> May I make the suggested changes, or as this is your patchset do you
> want to make them?
I've got a couple of other fixes to make to the series, so I'll work
these in. Thanks for reporting them and prototyping fixes for them.
:)
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 38+ messages in thread
end of thread, other threads:[~2012-09-06 23:06 UTC | newest]
Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20120831194326.741195404@sgi.com>
2012-07-26 8:39 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
2012-07-26 8:39 ` [PATCH 01/10] xfstests: add --largefs check option Dave Chinner
2012-08-28 14:00 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic Dave Chinner
2012-08-28 14:01 ` Rich Johnston
2012-08-31 23:30 ` Dave Chinner
2012-08-28 19:56 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES Dave Chinner
2012-08-28 14:01 ` Rich Johnston
2012-08-28 19:56 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 04/10] xfstests: use preallocation for ag-wiper Dave Chinner
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:57 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 05/10] xfstests: use command line option for setting extra space Dave Chinner
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:57 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing Dave Chinner
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:58 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 07/10] xfstests: always us test option when checking large scratch device Dave Chinner
2012-07-26 17:21 ` Paulo Alcantara
2012-08-28 14:02 ` Rich Johnston
2012-08-28 19:58 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 08/10] xfstests: enable large fs testing on ext4 Dave Chinner
2012-08-28 20:03 ` Christoph Hellwig
2012-09-05 16:27 ` Rich Johnston
2012-07-26 8:39 ` [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems Dave Chinner
2012-08-28 14:03 ` Rich Johnston
2012-08-28 20:03 ` Christoph Hellwig
2012-07-26 8:39 ` [PATCH 10/10] xfstests: exclude largefs fill files from dump tests Dave Chinner
2012-08-28 14:03 ` Rich Johnston
2012-08-28 20:04 ` Christoph Hellwig
2012-08-14 21:40 ` [PATCH 0/10] xfstests: rework large filesystem testing Dave Chinner
2012-09-05 21:34 ` [PATCH 11/10] xfstests: rework large filesystem testing - add golden output Ben Myers
2012-09-05 22:26 ` Dave Chinner
2012-09-06 12:57 ` Rich Johnston
2012-09-06 23:07 ` Dave Chinner
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.