* [PATCH] shared: new test to use up free inodes
@ 2014-03-19 9:27 Eryu Guan
2014-03-20 0:14 ` Dave Chinner
2014-03-20 0:22 ` Eric Sandeen
0 siblings, 2 replies; 6+ messages in thread
From: Eryu Guan @ 2014-03-19 9:27 UTC (permalink / raw)
To: xfs; +Cc: Eryu Guan
Stress test fs by using up all inodes and check fs.
Also a regression test for xfsprogs commit
d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
Signed-off-by: Eryu Guan <eguan@redhat.com>
---
tests/shared/006 | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/shared/006.out | 2 ++
tests/shared/group | 1 +
3 files changed, 99 insertions(+)
create mode 100755 tests/shared/006
create mode 100644 tests/shared/006.out
diff --git a/tests/shared/006 b/tests/shared/006
new file mode 100755
index 0000000..a3b13b6
--- /dev/null
+++ b/tests/shared/006
@@ -0,0 +1,96 @@
+#! /bin/bash
+# FS QA Test No. shared/006
+#
+# Stress test fs by using up all inodes and check fs.
+#
+# Also a regression test for xfsprogs commit
+# d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Red Hat Inc. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+create_file()
+{
+ local dir=$1
+ local nr_file=$2
+ local prefix=$3
+ local i=0
+
+ while [ $i -lt $nr_file ]; do
+ touch $dir/${prefix}_${i}
+ let i=$i+1
+ done
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs ext4 ext3 ext2 xfs
+_supported_os Linux
+
+_require_scratch
+
+rm -f $seqres.full
+echo "Silence is golden"
+
+_scratch_mkfs_sized $((1024 * 1024 * 1024)) >>$seqres.full 2>&1
+_scratch_mount
+
+i=0
+free_inode=`df -iP $SCRATCH_MNT | tail -1 | awk '{print $2}'`
+loop=$((free_inode / 1000 + 1))
+mkdir -p $SCRATCH_MNT/testdir
+
+echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
+while [ $i -lt $loop ]; do
+ create_file $SCRATCH_MNT/testdir 1000 $i >>$seqres.full 2>&1 &
+ let i=$i+1
+done
+wait
+
+# log inode status in $seqres.full for debug purpose
+echo "Inode status after taking all inodes" >>$seqres.full
+df -i $SCRATCH_MNT >>$seqres.full
+
+_check_scratch_fs
+
+# Check again after removing all the files
+rm -rf $SCRATCH_MNT/testdir
+echo "Inode status after deleting all test files" >>$seqres.full
+df -i $SCRATCH_MNT >>$seqres.full
+_check_scratch_fs
+
+status=0
+exit
diff --git a/tests/shared/006.out b/tests/shared/006.out
new file mode 100644
index 0000000..675c1b7
--- /dev/null
+++ b/tests/shared/006.out
@@ -0,0 +1,2 @@
+QA output created by 006
+Silence is golden
diff --git a/tests/shared/group b/tests/shared/group
index 29f17b0..230e560 100644
--- a/tests/shared/group
+++ b/tests/shared/group
@@ -8,6 +8,7 @@
003 auto quick prealloc
004 auto quick prealloc
005 auto prealloc
+006 auto stress enospc
032 mkfs auto quick
051 acl udf auto quick
218 auto fsr quick
--
1.8.5.3
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] shared: new test to use up free inodes
2014-03-19 9:27 [PATCH] shared: new test to use up free inodes Eryu Guan
@ 2014-03-20 0:14 ` Dave Chinner
2014-03-20 4:05 ` Eryu Guan
2014-03-20 0:22 ` Eric Sandeen
1 sibling, 1 reply; 6+ messages in thread
From: Dave Chinner @ 2014-03-20 0:14 UTC (permalink / raw)
To: Eryu Guan; +Cc: xfs
On Wed, Mar 19, 2014 at 05:27:49PM +0800, Eryu Guan wrote:
> Stress test fs by using up all inodes and check fs.
>
> Also a regression test for xfsprogs commit
> d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
>
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
> tests/shared/006 | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/shared/006.out | 2 ++
> tests/shared/group | 1 +
> 3 files changed, 99 insertions(+)
> create mode 100755 tests/shared/006
> create mode 100644 tests/shared/006.out
>
> diff --git a/tests/shared/006 b/tests/shared/006
> new file mode 100755
> index 0000000..a3b13b6
> --- /dev/null
> +++ b/tests/shared/006
> @@ -0,0 +1,96 @@
> +#! /bin/bash
> +# FS QA Test No. shared/006
> +#
> +# Stress test fs by using up all inodes and check fs.
> +#
> +# Also a regression test for xfsprogs commit
> +# d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2014 Red Hat Inc. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +create_file()
> +{
> + local dir=$1
> + local nr_file=$2
> + local prefix=$3
> + local i=0
> +
> + while [ $i -lt $nr_file ]; do
> + touch $dir/${prefix}_${i}
echo -n > $dir/${prefix}_${i}
will create a zero length file without needing to fork/exec and so
will have much lower overhead and create the files significantly
faster.
> + let i=$i+1
> + done
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs ext4 ext3 ext2 xfs
> +_supported_os Linux
> +
> +_require_scratch
> +
> +rm -f $seqres.full
> +echo "Silence is golden"
> +
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >>$seqres.full 2>&1
> +_scratch_mount
If this is going to be a stress test, you should add a scale factor
into this.
> +
> +i=0
> +free_inode=`df -iP $SCRATCH_MNT | tail -1 | awk '{print $2}'`
$DF_PROG
> +loop=$((free_inode / 1000 + 1))
And probably a LOAD_FACTOR into this to scale parallelism.
> +mkdir -p $SCRATCH_MNT/testdir
> +
> +echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> +while [ $i -lt $loop ]; do
> + create_file $SCRATCH_MNT/testdir 1000 $i >>$seqres.full 2>&1 &
> + let i=$i+1
> +done
> +wait
On XFS, that will create at least 500 threads creating 1000 inodes each
all in the same directory. This doesn't give you any extra
parallelism at all over just creating $free_inode files in a single
directory with a single thread. Indeed, it will probably be slower
due to the contention on the directory mutex.
If you want to scale this in terms of parallelism to keep the
creation time down, each loop needs to write into a different
directory. i.e. something like:
echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
while [ $i -lt $loop ]; do
mkdir -p $SCRATCH_MNT/testdir/$i
create_file $SCRATCH_MNT/testdir/$i 1000 $i >>$seqres.full 2>&1 &
let i=$i+1
done
wait
And even then I'd suggest that you'd be much better off with 10,000
files to a sub-directory....
> +# log inode status in $seqres.full for debug purpose
> +echo "Inode status after taking all inodes" >>$seqres.full
> +df -i $SCRATCH_MNT >>$seqres.full
> +
> +_check_scratch_fs
> +
> +# Check again after removing all the files
> +rm -rf $SCRATCH_MNT/testdir
That can be parallelised as well when you have multiple subdirs:
for d in $SCRATCH_MNT/testdir/*; do
rm -rf $d &
done
wait
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] shared: new test to use up free inodes
2014-03-19 9:27 [PATCH] shared: new test to use up free inodes Eryu Guan
2014-03-20 0:14 ` Dave Chinner
@ 2014-03-20 0:22 ` Eric Sandeen
2014-03-20 4:17 ` Eryu Guan
1 sibling, 1 reply; 6+ messages in thread
From: Eric Sandeen @ 2014-03-20 0:22 UTC (permalink / raw)
To: Eryu Guan, xfs
On 3/19/14, 4:27 AM, Eryu Guan wrote:
> Stress test fs by using up all inodes and check fs.
>
> Also a regression test for xfsprogs commit
> d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
Thanks Eryu. I had started to write this test but ran into another bug
(where xfs blows past the maximum inode space %) so got distracted.
I see Dave's already picked apart the patch so I won't comment specifically
on it, but you might be aware of the possibility that xfs won't stop when
we hit the default imaxpct, and may actually fill the entire fs. That's
not necessarily a problem but it may take longer than you expect.
I also wonder what's necessary to actually test the sibling pointer problem;
on xfs, it's probably just a sufficiently large directory, not necessarily
a full fs?
-Eric
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
> tests/shared/006 | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/shared/006.out | 2 ++
> tests/shared/group | 1 +
> 3 files changed, 99 insertions(+)
> create mode 100755 tests/shared/006
> create mode 100644 tests/shared/006.out
>
> diff --git a/tests/shared/006 b/tests/shared/006
> new file mode 100755
> index 0000000..a3b13b6
> --- /dev/null
> +++ b/tests/shared/006
> @@ -0,0 +1,96 @@
> +#! /bin/bash
> +# FS QA Test No. shared/006
> +#
> +# Stress test fs by using up all inodes and check fs.
> +#
> +# Also a regression test for xfsprogs commit
> +# d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2014 Red Hat Inc. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +create_file()
> +{
> + local dir=$1
> + local nr_file=$2
> + local prefix=$3
> + local i=0
> +
> + while [ $i -lt $nr_file ]; do
> + touch $dir/${prefix}_${i}
> + let i=$i+1
> + done
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs ext4 ext3 ext2 xfs
> +_supported_os Linux
> +
> +_require_scratch
> +
> +rm -f $seqres.full
> +echo "Silence is golden"
> +
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >>$seqres.full 2>&1
> +_scratch_mount
> +
> +i=0
> +free_inode=`df -iP $SCRATCH_MNT | tail -1 | awk '{print $2}'`
> +loop=$((free_inode / 1000 + 1))
> +mkdir -p $SCRATCH_MNT/testdir
> +
> +echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> +while [ $i -lt $loop ]; do
> + create_file $SCRATCH_MNT/testdir 1000 $i >>$seqres.full 2>&1 &
> + let i=$i+1
> +done
> +wait
> +
> +# log inode status in $seqres.full for debug purpose
> +echo "Inode status after taking all inodes" >>$seqres.full
> +df -i $SCRATCH_MNT >>$seqres.full
> +
> +_check_scratch_fs
> +
> +# Check again after removing all the files
> +rm -rf $SCRATCH_MNT/testdir
> +echo "Inode status after deleting all test files" >>$seqres.full
> +df -i $SCRATCH_MNT >>$seqres.full
> +_check_scratch_fs
> +
> +status=0
> +exit
> diff --git a/tests/shared/006.out b/tests/shared/006.out
> new file mode 100644
> index 0000000..675c1b7
> --- /dev/null
> +++ b/tests/shared/006.out
> @@ -0,0 +1,2 @@
> +QA output created by 006
> +Silence is golden
> diff --git a/tests/shared/group b/tests/shared/group
> index 29f17b0..230e560 100644
> --- a/tests/shared/group
> +++ b/tests/shared/group
> @@ -8,6 +8,7 @@
> 003 auto quick prealloc
> 004 auto quick prealloc
> 005 auto prealloc
> +006 auto stress enospc
> 032 mkfs auto quick
> 051 acl udf auto quick
> 218 auto fsr quick
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] shared: new test to use up free inodes
2014-03-20 0:14 ` Dave Chinner
@ 2014-03-20 4:05 ` Eryu Guan
2014-03-20 4:53 ` Eryu Guan
0 siblings, 1 reply; 6+ messages in thread
From: Eryu Guan @ 2014-03-20 4:05 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, Mar 20, 2014 at 11:14:29AM +1100, Dave Chinner wrote:
> On Wed, Mar 19, 2014 at 05:27:49PM +0800, Eryu Guan wrote:
> > Stress test fs by using up all inodes and check fs.
> >
> > Also a regression test for xfsprogs commit
> > d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
> >
> > Signed-off-by: Eryu Guan <eguan@redhat.com>
> > ---
> > tests/shared/006 | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/shared/006.out | 2 ++
> > tests/shared/group | 1 +
> > 3 files changed, 99 insertions(+)
> > create mode 100755 tests/shared/006
> > create mode 100644 tests/shared/006.out
> >
> > diff --git a/tests/shared/006 b/tests/shared/006
> > new file mode 100755
> > index 0000000..a3b13b6
> > --- /dev/null
> > +++ b/tests/shared/006
> > @@ -0,0 +1,96 @@
> > +#! /bin/bash
> > +# FS QA Test No. shared/006
> > +#
> > +# Stress test fs by using up all inodes and check fs.
> > +#
> > +# Also a regression test for xfsprogs commit
> > +# d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
> > +#
> > +#-----------------------------------------------------------------------
> > +# Copyright (c) 2014 Red Hat Inc. All Rights Reserved.
> > +#
> > +# This program is free software; you can redistribute it and/or
> > +# modify it under the terms of the GNU General Public License as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it would be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program; if not, write the Free Software Foundation,
> > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> > +#-----------------------------------------------------------------------
> > +#
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +create_file()
> > +{
> > + local dir=$1
> > + local nr_file=$2
> > + local prefix=$3
> > + local i=0
> > +
> > + while [ $i -lt $nr_file ]; do
> > + touch $dir/${prefix}_${i}
>
> echo -n > $dir/${prefix}_${i}
>
> will create a zero length file without needing to fork/exec and so
> will have much lower overhead and create the files significantly
> faster.
This is better.
>
> > + let i=$i+1
> > + done
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs ext4 ext3 ext2 xfs
> > +_supported_os Linux
> > +
> > +_require_scratch
> > +
> > +rm -f $seqres.full
> > +echo "Silence is golden"
> > +
> > +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >>$seqres.full 2>&1
> > +_scratch_mount
>
> If this is going to be a stress test, you should add a scale factor
> into this.
I don't have strong feeling to make it a stress test, I'd like to
remove the stress group to make the test a simple regression test.
>
> > +
> > +i=0
> > +free_inode=`df -iP $SCRATCH_MNT | tail -1 | awk '{print $2}'`
>
> $DF_PROG
OK.
>
> > +loop=$((free_inode / 1000 + 1))
>
> And probably a LOAD_FACTOR into this to scale parallelism.
>
> > +mkdir -p $SCRATCH_MNT/testdir
> > +
> > +echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> > +while [ $i -lt $loop ]; do
> > + create_file $SCRATCH_MNT/testdir 1000 $i >>$seqres.full 2>&1 &
> > + let i=$i+1
> > +done
> > +wait
>
> On XFS, that will create at least 500 threads creating 1000 inodes each
> all in the same directory. This doesn't give you any extra
> parallelism at all over just creating $free_inode files in a single
> directory with a single thread. Indeed, it will probably be slower
> due to the contention on the directory mutex.
>
> If you want to scale this in terms of parallelism to keep the
> creation time down, each loop needs to write into a different
> directory. i.e. something like:
>
>
> echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> while [ $i -lt $loop ]; do
> mkdir -p $SCRATCH_MNT/testdir/$i
> create_file $SCRATCH_MNT/testdir/$i 1000 $i >>$seqres.full 2>&1 &
> let i=$i+1
> done
> wait
>
> And even then I'd suggest that you'd be much better off with 10,000
> files to a sub-directory....
Will do.
>
> > +# log inode status in $seqres.full for debug purpose
> > +echo "Inode status after taking all inodes" >>$seqres.full
> > +df -i $SCRATCH_MNT >>$seqres.full
> > +
> > +_check_scratch_fs
> > +
> > +# Check again after removing all the files
> > +rm -rf $SCRATCH_MNT/testdir
>
> That can be parallelised as well when you have multiple subdirs:
>
> for d in $SCRATCH_MNT/testdir/*; do
> rm -rf $d &
> done
> wait
Will do.
Thanks for the detailed review (as always)!
Eryu
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] shared: new test to use up free inodes
2014-03-20 0:22 ` Eric Sandeen
@ 2014-03-20 4:17 ` Eryu Guan
0 siblings, 0 replies; 6+ messages in thread
From: Eryu Guan @ 2014-03-20 4:17 UTC (permalink / raw)
To: Eric Sandeen; +Cc: xfs
On Wed, Mar 19, 2014 at 07:22:43PM -0500, Eric Sandeen wrote:
> On 3/19/14, 4:27 AM, Eryu Guan wrote:
> > Stress test fs by using up all inodes and check fs.
> >
> > Also a regression test for xfsprogs commit
> > d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
>
> Thanks Eryu. I had started to write this test but ran into another bug
> (where xfs blows past the maximum inode space %) so got distracted.
I didn't notice this. Do I need to add extra check to detect this in
my test? I want to cover this in one test case if possible.
>
> I see Dave's already picked apart the patch so I won't comment specifically
> on it, but you might be aware of the possibility that xfs won't stop when
> we hit the default imaxpct, and may actually fill the entire fs. That's
> not necessarily a problem but it may take longer than you expect.
Yes, I noticed that testing on xfs didn't give ENOSPC error as testing
on ext4 do. The number of extra files created is less than 1000, so I
think that's fine.
>
> I also wonder what's necessary to actually test the sibling pointer problem;
> on xfs, it's probably just a sufficiently large directory, not necessarily
> a full fs?
I've seen list corruption and then xfs force shutdown by running the
same test(at the time removing all created files) on earlier RHEL7
test kernels, then the bug was fixed by xfs update. So I'm not sure
it's large directory or taking all free inode trigger the bug, I think
I'd better still to fulfill the fs.
Thanks,
Eryu
>
> -Eric
>
>
> > Signed-off-by: Eryu Guan <eguan@redhat.com>
> > ---
> > tests/shared/006 | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/shared/006.out | 2 ++
> > tests/shared/group | 1 +
> > 3 files changed, 99 insertions(+)
> > create mode 100755 tests/shared/006
> > create mode 100644 tests/shared/006.out
> >
> > diff --git a/tests/shared/006 b/tests/shared/006
> > new file mode 100755
> > index 0000000..a3b13b6
> > --- /dev/null
> > +++ b/tests/shared/006
> > @@ -0,0 +1,96 @@
> > +#! /bin/bash
> > +# FS QA Test No. shared/006
> > +#
> > +# Stress test fs by using up all inodes and check fs.
> > +#
> > +# Also a regression test for xfsprogs commit
> > +# d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
> > +#
> > +#-----------------------------------------------------------------------
> > +# Copyright (c) 2014 Red Hat Inc. All Rights Reserved.
> > +#
> > +# This program is free software; you can redistribute it and/or
> > +# modify it under the terms of the GNU General Public License as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it would be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program; if not, write the Free Software Foundation,
> > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> > +#-----------------------------------------------------------------------
> > +#
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +create_file()
> > +{
> > + local dir=$1
> > + local nr_file=$2
> > + local prefix=$3
> > + local i=0
> > +
> > + while [ $i -lt $nr_file ]; do
> > + touch $dir/${prefix}_${i}
> > + let i=$i+1
> > + done
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs ext4 ext3 ext2 xfs
> > +_supported_os Linux
> > +
> > +_require_scratch
> > +
> > +rm -f $seqres.full
> > +echo "Silence is golden"
> > +
> > +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >>$seqres.full 2>&1
> > +_scratch_mount
> > +
> > +i=0
> > +free_inode=`df -iP $SCRATCH_MNT | tail -1 | awk '{print $2}'`
> > +loop=$((free_inode / 1000 + 1))
> > +mkdir -p $SCRATCH_MNT/testdir
> > +
> > +echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> > +while [ $i -lt $loop ]; do
> > + create_file $SCRATCH_MNT/testdir 1000 $i >>$seqres.full 2>&1 &
> > + let i=$i+1
> > +done
> > +wait
> > +
> > +# log inode status in $seqres.full for debug purpose
> > +echo "Inode status after taking all inodes" >>$seqres.full
> > +df -i $SCRATCH_MNT >>$seqres.full
> > +
> > +_check_scratch_fs
> > +
> > +# Check again after removing all the files
> > +rm -rf $SCRATCH_MNT/testdir
> > +echo "Inode status after deleting all test files" >>$seqres.full
> > +df -i $SCRATCH_MNT >>$seqres.full
> > +_check_scratch_fs
> > +
> > +status=0
> > +exit
> > diff --git a/tests/shared/006.out b/tests/shared/006.out
> > new file mode 100644
> > index 0000000..675c1b7
> > --- /dev/null
> > +++ b/tests/shared/006.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 006
> > +Silence is golden
> > diff --git a/tests/shared/group b/tests/shared/group
> > index 29f17b0..230e560 100644
> > --- a/tests/shared/group
> > +++ b/tests/shared/group
> > @@ -8,6 +8,7 @@
> > 003 auto quick prealloc
> > 004 auto quick prealloc
> > 005 auto prealloc
> > +006 auto stress enospc
> > 032 mkfs auto quick
> > 051 acl udf auto quick
> > 218 auto fsr quick
> >
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] shared: new test to use up free inodes
2014-03-20 4:05 ` Eryu Guan
@ 2014-03-20 4:53 ` Eryu Guan
0 siblings, 0 replies; 6+ messages in thread
From: Eryu Guan @ 2014-03-20 4:53 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, Mar 20, 2014 at 12:05:25PM +0800, Eryu Guan wrote:
> On Thu, Mar 20, 2014 at 11:14:29AM +1100, Dave Chinner wrote:
> > On Wed, Mar 19, 2014 at 05:27:49PM +0800, Eryu Guan wrote:
[snip]
> >
> > On XFS, that will create at least 500 threads creating 1000 inodes each
> > all in the same directory. This doesn't give you any extra
> > parallelism at all over just creating $free_inode files in a single
> > directory with a single thread. Indeed, it will probably be slower
> > due to the contention on the directory mutex.
> >
> > If you want to scale this in terms of parallelism to keep the
> > creation time down, each loop needs to write into a different
> > directory. i.e. something like:
> >
> >
> > echo "Create $((loop * 1000)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> > while [ $i -lt $loop ]; do
> > mkdir -p $SCRATCH_MNT/testdir/$i
> > create_file $SCRATCH_MNT/testdir/$i 1000 $i >>$seqres.full 2>&1 &
> > let i=$i+1
> > done
> > wait
It turns out that creating files in different dirs is unable to
reproduce the bug that commit
d586858 xfs_repair: fix sibling pointer tests in verify_dir2_path()
fixed. I'll keep the files in one testdir.
Thanks,
Eryu
> >
> > And even then I'd suggest that you'd be much better off with 10,000
> > files to a sub-directory....
>
> Will do.
>
> >
> > > +# log inode status in $seqres.full for debug purpose
> > > +echo "Inode status after taking all inodes" >>$seqres.full
> > > +df -i $SCRATCH_MNT >>$seqres.full
> > > +
> > > +_check_scratch_fs
> > > +
> > > +# Check again after removing all the files
> > > +rm -rf $SCRATCH_MNT/testdir
> >
> > That can be parallelised as well when you have multiple subdirs:
> >
> > for d in $SCRATCH_MNT/testdir/*; do
> > rm -rf $d &
> > done
> > wait
>
> Will do.
>
> Thanks for the detailed review (as always)!
>
> Eryu
> >
> > Cheers,
> >
> > Dave.
> > --
> > Dave Chinner
> > david@fromorbit.com
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-03-20 4:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-03-19 9:27 [PATCH] shared: new test to use up free inodes Eryu Guan
2014-03-20 0:14 ` Dave Chinner
2014-03-20 4:05 ` Eryu Guan
2014-03-20 4:53 ` Eryu Guan
2014-03-20 0:22 ` Eric Sandeen
2014-03-20 4:17 ` Eryu Guan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.