All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: david@fromorbit.com, darrick.wong@oracle.com
Cc: linux-btrfs@vger.kernel.org, fstests@vger.kernel.org, xfs@oss.sgi.com
Subject: [PATCH 21/32] reflink: test xfs cow behavior when the filesystem crashes
Date: Thu, 11 Feb 2016 15:41:42 -0800	[thread overview]
Message-ID: <20160211234142.2202.57779.stgit@birch.djwong.org> (raw)
In-Reply-To: <20160211233916.2202.40961.stgit@birch.djwong.org>

Use the extent size hint to force leftover CoW reservations then
crash the filesystem to see how recovery works.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/212     |   99 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/212.out |   13 +++++++
 tests/xfs/group   |    1 +
 3 files changed, 113 insertions(+)
 create mode 100755 tests/xfs/212
 create mode 100644 tests/xfs/212.out


diff --git a/tests/xfs/212 b/tests/xfs/212
new file mode 100755
index 0000000..4e3bcd4
--- /dev/null
+++ b/tests/xfs/212
@@ -0,0 +1,99 @@
+#! /bin/bash
+# FS QA Test No. 212
+#
+# Test recovery of "lost" CoW blocks after a crash:
+# - Create two reflinked files.  Set extsz hint on second file.
+# - Dirty one byte on the second file and fsync.
+# - Crash the FS to test recovery.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch_reflink
+_require_cp_reflink
+_require_fiemap
+
+rm -f $seqres.full
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+blksz=65536
+nr=16
+filesize=$((blksz * nr))
+bufnr=2
+bufsize=$((blksz * bufnr))
+
+_require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
+
+echo "Create the original files"
+$XFS_IO_PROG -f -c "pwrite -S 0x61 -b $bufsize 0 $filesize" $testdir/file1 >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x61 -b $bufsize 0 $filesize" $testdir/file2.chk >> $seqres.full
+$XFS_IO_PROG -f -c "extsize $bufsize" $testdir/file2
+_cp_reflink $testdir/file1 $testdir/file2 >> $seqres.full
+_scratch_remount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file2 | _filter_scratch
+md5sum $testdir/file2.chk | _filter_scratch
+
+echo "CoW and leave leftovers"
+$XFS_IO_PROG -f -c "extsize" $testdir/file2 >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x63 $((filesize - 1)) 1" -c "fsync" $testdir/file2 >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x63 $((filesize - 1)) 1" -c "fsync" $testdir/file2.chk >> $seqres.full
+sync
+
+echo "Crash and recover"
+$XFS_IO_PROG -x -c "shutdown" $testdir/file2 >> $seqres.full
+_scratch_remount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file2 | _filter_scratch
+md5sum $testdir/file2.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/212.out b/tests/xfs/212.out
new file mode 100644
index 0000000..97e8425
--- /dev/null
+++ b/tests/xfs/212.out
@@ -0,0 +1,13 @@
+QA output created by 212
+Format and mount
+Create the original files
+Compare files
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file1
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file2
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file2.chk
+CoW and leave leftovers
+Crash and recover
+Compare files
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file1
+83feff041c88d5c746837552399dc27d  SCRATCH_MNT/test-212/file2
+83feff041c88d5c746837552399dc27d  SCRATCH_MNT/test-212/file2.chk
diff --git a/tests/xfs/group b/tests/xfs/group
index 119e1fd..d4a0d59 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -209,6 +209,7 @@
 209 auto quick clone
 210 auto quick clone
 211 clone_stress
+212 auto quick clone
 216 log metadata auto quick
 217 log metadata auto
 220 auto quota quick


WARNING: multiple messages have this Message-ID (diff)
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: david@fromorbit.com, darrick.wong@oracle.com
Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, xfs@oss.sgi.com
Subject: [PATCH 21/32] reflink: test xfs cow behavior when the filesystem crashes
Date: Thu, 11 Feb 2016 15:41:42 -0800	[thread overview]
Message-ID: <20160211234142.2202.57779.stgit@birch.djwong.org> (raw)
In-Reply-To: <20160211233916.2202.40961.stgit@birch.djwong.org>

Use the extent size hint to force leftover CoW reservations then
crash the filesystem to see how recovery works.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/212     |   99 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/212.out |   13 +++++++
 tests/xfs/group   |    1 +
 3 files changed, 113 insertions(+)
 create mode 100755 tests/xfs/212
 create mode 100644 tests/xfs/212.out


diff --git a/tests/xfs/212 b/tests/xfs/212
new file mode 100755
index 0000000..4e3bcd4
--- /dev/null
+++ b/tests/xfs/212
@@ -0,0 +1,99 @@
+#! /bin/bash
+# FS QA Test No. 212
+#
+# Test recovery of "lost" CoW blocks after a crash:
+# - Create two reflinked files.  Set extsz hint on second file.
+# - Dirty one byte on the second file and fsync.
+# - Crash the FS to test recovery.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch_reflink
+_require_cp_reflink
+_require_fiemap
+
+rm -f $seqres.full
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+blksz=65536
+nr=16
+filesize=$((blksz * nr))
+bufnr=2
+bufsize=$((blksz * bufnr))
+
+_require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
+
+echo "Create the original files"
+$XFS_IO_PROG -f -c "pwrite -S 0x61 -b $bufsize 0 $filesize" $testdir/file1 >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x61 -b $bufsize 0 $filesize" $testdir/file2.chk >> $seqres.full
+$XFS_IO_PROG -f -c "extsize $bufsize" $testdir/file2
+_cp_reflink $testdir/file1 $testdir/file2 >> $seqres.full
+_scratch_remount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file2 | _filter_scratch
+md5sum $testdir/file2.chk | _filter_scratch
+
+echo "CoW and leave leftovers"
+$XFS_IO_PROG -f -c "extsize" $testdir/file2 >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x63 $((filesize - 1)) 1" -c "fsync" $testdir/file2 >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x63 $((filesize - 1)) 1" -c "fsync" $testdir/file2.chk >> $seqres.full
+sync
+
+echo "Crash and recover"
+$XFS_IO_PROG -x -c "shutdown" $testdir/file2 >> $seqres.full
+_scratch_remount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file2 | _filter_scratch
+md5sum $testdir/file2.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/212.out b/tests/xfs/212.out
new file mode 100644
index 0000000..97e8425
--- /dev/null
+++ b/tests/xfs/212.out
@@ -0,0 +1,13 @@
+QA output created by 212
+Format and mount
+Create the original files
+Compare files
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file1
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file2
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file2.chk
+CoW and leave leftovers
+Crash and recover
+Compare files
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-212/file1
+83feff041c88d5c746837552399dc27d  SCRATCH_MNT/test-212/file2
+83feff041c88d5c746837552399dc27d  SCRATCH_MNT/test-212/file2.chk
diff --git a/tests/xfs/group b/tests/xfs/group
index 119e1fd..d4a0d59 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -209,6 +209,7 @@
 209 auto quick clone
 210 auto quick clone
 211 clone_stress
+212 auto quick clone
 216 log metadata auto quick
 217 log metadata auto
 220 auto quota quick

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  parent reply	other threads:[~2016-02-11 23:41 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-02-11 23:39 [PATCH v4.2 00/32] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls Darrick J. Wong
2016-02-11 23:39 ` Darrick J. Wong
2016-02-11 23:39 ` [PATCH 01/32] generic/182: this is a dedupe test, check for dedupe Darrick J. Wong
2016-02-11 23:39   ` Darrick J. Wong
2016-02-11 23:39 ` [PATCH 02/32] xfstests: filter whitespace in 128 and 132 Darrick J. Wong
2016-02-11 23:39   ` Darrick J. Wong
2016-02-11 23:39 ` [PATCH 03/32] xfstests: make _scratch_mkfs_blocksized usable Darrick J. Wong
2016-02-11 23:39   ` Darrick J. Wong
2016-02-11 23:39 ` [PATCH 04/32] reflink: remove redundant filesystem checks from the end of the tests Darrick J. Wong
2016-02-11 23:39   ` Darrick J. Wong
2016-02-11 23:39 ` [PATCH 05/32] reflink: remove unnecessary umounts from test code Darrick J. Wong
2016-02-11 23:39   ` Darrick J. Wong
2016-02-11 23:39 ` [PATCH 06/32] common/dmerror: add some more dmerror routines Darrick J. Wong
2016-02-11 23:39   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 07/32] common: create _require_test_program to look for programs in src/ Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 08/32] common: provide a method to repair the scratch fs Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 09/32] reflink: refactor mixed block creation code Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 10/32] reflink: fix style problems in existing tests Darrick J. Wong
2016-02-11 23:40 ` [PATCH 11/32] reflink: add _require_odirect to the directio tests Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 12/32] dio: unwritten conversion bug tests Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-12  3:52   ` Dave Chinner
2016-02-12  3:52     ` Dave Chinner
2016-02-12 15:22     ` Theodore Ts'o
2016-02-12 15:22       ` Theodore Ts'o
2016-02-12 22:49       ` Dave Chinner
2016-02-12 22:49         ` Dave Chinner
2016-02-12 17:40     ` Darrick J. Wong
2016-02-12 17:40       ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 13/32] reflink: test intersecting CoW and falloc/fpunch/fzero/fcollapse/finsert/ftrunc Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 14/32] reflink: test CoW behavior with IO errors Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:40 ` [PATCH 15/32] reflink: test CoW operations against the source file Darrick J. Wong
2016-02-11 23:40   ` Darrick J. Wong
2016-02-11 23:41 ` [PATCH 16/32] reflink: ensure that we can handle reflinking a lot of extents Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:41 ` [PATCH 17/32] xfs: more reflink tests Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:41 ` [PATCH 18/32] xfs/122: support refcount/rmap data structures Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:41 ` [PATCH 19/32] xfs: test fragmentation characteristics of copy-on-write Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:41 ` [PATCH 20/32] reflink: high offset reflink and dedupe tests Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:41 ` Darrick J. Wong [this message]
2016-02-11 23:41   ` [PATCH 21/32] reflink: test xfs cow behavior when the filesystem crashes Darrick J. Wong
2016-02-11 23:41 ` [PATCH 22/32] reflink: test quota accounting Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:41 ` [PATCH 23/32] reflink: test CoW across a mixed range of block types with cowextsize set Darrick J. Wong
2016-02-11 23:41   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 24/32] xfs: test the automatic cowextsize extent garbage collector Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 25/32] xfs: test rmapbt functionality Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 26/32] reflink: test aio copy on write Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 27/32] xfs: aio cow tests Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 28/32] xfs: test xfs_getbmapx behavior with shared extents Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 29/32] reflink: test reflink+cow+enospc all at the same time Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 30/32] xfs: ensure that we don't crash when freeing the ag reservations on a ro mount Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 31/32] xfs: test source CoW across mixed block types with cowextsz set Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-11 23:42 ` [PATCH 32/32] reflink: kick the slow tests out of the 'quick' group Darrick J. Wong
2016-02-11 23:42   ` Darrick J. Wong
2016-02-12 17:41 ` [PATCH 33/32] aiocp: fix the gcc warnings Darrick J. Wong
2016-02-12 17:41   ` Darrick J. Wong
2016-02-13  1:26 ` [PATCH v4.2 00/32] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls Dave Chinner
2016-02-13  1:26   ` Dave Chinner
2016-02-13  4:38   ` Darrick J. Wong
2016-02-13  4:38     ` Darrick J. Wong
2016-02-13  8:03     ` Christoph Hellwig
2016-02-13  8:03       ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160211234142.2202.57779.stgit@birch.djwong.org \
    --to=darrick.wong@oracle.com \
    --cc=david@fromorbit.com \
    --cc=fstests@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.