* [PATCH] fstests: btrfs/150 regression test for reading compressed data
@ 2017-09-20 23:52 Liu Bo
2017-09-21 4:39 ` Dave Chinner
` (3 more replies)
0 siblings, 4 replies; 15+ messages in thread
From: Liu Bo @ 2017-09-20 23:52 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
We had a bug in btrfs compression code which could end up with a
kernel panic.
This is adding a regression test for the bug and I've also sent a
kernel patch to fix the bug.
The patch is "Btrfs: fix kernel oops while reading compressed data".
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
---
tests/btrfs/150 | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/150.out | 3 ++
tests/btrfs/group | 1 +
3 files changed, 106 insertions(+)
create mode 100755 tests/btrfs/150
create mode 100644 tests/btrfs/150.out
diff --git a/tests/btrfs/150 b/tests/btrfs/150
new file mode 100755
index 0000000..834be51
--- /dev/null
+++ b/tests/btrfs/150
@@ -0,0 +1,102 @@
+#! /bin/bash
+# FS QA Test btrfs/150
+#
+# This is a regression test which ends up with a kernel oops in btrfs.
+# It occurs when btrfs's read repair happens while reading a compressed
+# extent.
+# The patch for this is
+# xxxxx
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_fail_make_request
+_require_scratch_dev_pool 2
+
+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
+enable_io_failure()
+{
+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
+ echo 1 > $SYSFS_BDEV/make-it-fail
+}
+
+disable_io_failure()
+{
+ echo 0 > $SYSFS_BDEV/make-it-fail
+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
+}
+
+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
+
+# It doesn't matter which compression algorithm we use.
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
+
+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
+# the bad copy to trigger read-repair.
+while true; do
+ disable_io_failure
+ # invalidate the page cache
+ $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | _filter_xfs_io
+
+ enable_io_failure
+ od -x $SCRATCH_MNT/foobar > /dev/null &
+ pid=$!
+ wait
+ [ $((pid % 2)) == 1 ] && break
+done
+
+disable_io_failure
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
new file mode 100644
index 0000000..c492c24
--- /dev/null
+++ b/tests/btrfs/150.out
@@ -0,0 +1,3 @@
+QA output created by 150
+wrote 8192/8192 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 70c3f05..b70a122 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -152,3 +152,4 @@
147 auto quick send
148 auto quick rw
149 auto quick send compress
+150 auto quick
--
2.5.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data
2017-09-20 23:52 [PATCH] fstests: btrfs/150 regression test for reading compressed data Liu Bo
@ 2017-09-21 4:39 ` Dave Chinner
2017-09-22 19:33 ` Liu Bo
2017-09-21 7:03 ` Lu Fengqi
` (2 subsequent siblings)
3 siblings, 1 reply; 15+ messages in thread
From: Dave Chinner @ 2017-09-21 4:39 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote:
> We had a bug in btrfs compression code which could end up with a
> kernel panic.
>
> This is adding a regression test for the bug and I've also sent a
> kernel patch to fix the bug.
>
> The patch is "Btrfs: fix kernel oops while reading compressed data".
>
> Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> ---
> tests/btrfs/150 | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/150.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 106 insertions(+)
> create mode 100755 tests/btrfs/150
> create mode 100644 tests/btrfs/150.out
>
> diff --git a/tests/btrfs/150 b/tests/btrfs/150
> new file mode 100755
> index 0000000..834be51
> --- /dev/null
> +++ b/tests/btrfs/150
> @@ -0,0 +1,102 @@
> +#! /bin/bash
> +# FS QA Test btrfs/150
> +#
> +# This is a regression test which ends up with a kernel oops in btrfs.
group += dangerous
> +# It occurs when btrfs's read repair happens while reading a compressed
> +# extent.
> +# The patch for this is
> +# xxxxx
Incomplete?
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
You're signing off this patch an Oracle employee, but claiming
personal copyright. Please clarify who owns the copyright - if it's
your personal copyright then please sign off with a personal email
address, not your employer's...
Also, I note that these recently added tests from you:
tests/btrfs/140:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
tests/btrfs/141:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
tests/btrfs/142:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
tests/btrfs/143:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
tests/generic/406:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
all have this same ambiguity - personal copyright with employer
signoff in the commit. This definitely needs clarification and
fixing if it is wrong....
> +disable_io_failure()
> +{
> + echo 0 > $SYSFS_BDEV/make-it-fail
> + echo 0 > $DEBUGFS_MNT/fail_make_request/probability
> + echo 0 > $DEBUGFS_MNT/fail_make_request/times
> +}
> +
> +_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
> +
> +# It doesn't matter which compression algorithm we use.
> +_scratch_mount -ocompress
> +
> +# Create a file with all data being compressed
> +$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
needs an fsync to reach disk.
> +# Raid1 consists of two copies and btrfs decides which copy to read by reader's
> +# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
> +# the bad copy to trigger read-repair.
> +while true; do
> + disable_io_failure
> + # invalidate the page cache
> + $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | _filter_xfs_io
> +
> + enable_io_failure
> + od -x $SCRATCH_MNT/foobar > /dev/null &
why are you using od to read the data when the output is piped to
dev/null? why not just xfs_io -c "pread 0 8k" ?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data
2017-09-20 23:52 [PATCH] fstests: btrfs/150 regression test for reading compressed data Liu Bo
@ 2017-09-21 7:03 ` Lu Fengqi
2017-09-21 7:03 ` Lu Fengqi
` (2 subsequent siblings)
3 siblings, 0 replies; 15+ messages in thread
From: Lu Fengqi @ 2017-09-21 7:03 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote:
>We had a bug in btrfs compression code which could end up with a
>kernel panic.
>
>This is adding a regression test for the bug and I've also sent a
>kernel patch to fix the bug.
>
>The patch is "Btrfs: fix kernel oops while reading compressed data".
>
>Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
>---
> tests/btrfs/150 | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/150.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 106 insertions(+)
> create mode 100755 tests/btrfs/150
> create mode 100644 tests/btrfs/150.out
>
>diff --git a/tests/btrfs/150 b/tests/btrfs/150
>new file mode 100755
>index 0000000..834be51
>--- /dev/null
>+++ b/tests/btrfs/150
>@@ -0,0 +1,102 @@
>+#! /bin/bash
>+# FS QA Test btrfs/150
>+#
>+# This is a regression test which ends up with a kernel oops in btrfs.
>+# It occurs when btrfs's read repair happens while reading a compressed
>+# extent.
>+# The patch for this is
>+# xxxxx
>+#
>+#-----------------------------------------------------------------------
>+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
>+#
>+# This program is free software; you can redistribute it and/or
>+# modify it under the terms of the GNU General Public License as
>+# published by the Free Software Foundation.
>+#
>+# This program is distributed in the hope that it would be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>+# GNU General Public License for more details.
>+#
>+# You should have received a copy of the GNU General Public License
>+# along with this program; if not, write the Free Software Foundation,
>+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>+#-----------------------------------------------------------------------
>+#
>+
>+seq=`basename $0`
>+seqres=$RESULT_DIR/$seq
>+echo "QA output created by $seq"
>+
>+here=`pwd`
>+tmp=/tmp/$$
>+status=1 # failure is the default!
>+trap "_cleanup; exit \$status" 0 1 2 3 15
>+
>+_cleanup()
>+{
>+ cd /
>+ rm -f $tmp.*
>+}
>+
>+# get standard environment, filters and checks
>+. ./common/rc
>+. ./common/filter
>+
>+# remove previous $seqres.full before test
>+rm -f $seqres.full
>+
>+# real QA test starts here
>+
>+# Modify as appropriate.
>+_supported_fs btrfs
>+_supported_os Linux
>+_require_scratch
>+_require_fail_make_request
>+_require_scratch_dev_pool 2
>+
>+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
>+enable_io_failure()
>+{
>+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
What does 1000 mean? Enough failures?
Why not set times to -1?
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
>+ echo 1 > $SYSFS_BDEV/make-it-fail
>+}
>+
>+disable_io_failure()
>+{
>+ echo 0 > $SYSFS_BDEV/make-it-fail
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
>+}
>+
>+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
>+
>+# It doesn't matter which compression algorithm we use.
>+_scratch_mount -ocompress
>+
>+# Create a file with all data being compressed
>+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
>+
>+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
>+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
>+# the bad copy to trigger read-repair.
>+while true; do
>+ disable_io_failure
>+ # invalidate the page cache
>+ $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | _filter_xfs_io
>+
>+ enable_io_failure
>+ od -x $SCRATCH_MNT/foobar > /dev/null &
>+ pid=$!
>+ wait
>+ [ $((pid % 2)) == 1 ] && break
>+done
>+
>+disable_io_failure
>+
>+# success, all done
>+status=0
>+exit
>diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
>new file mode 100644
>index 0000000..c492c24
>--- /dev/null
>+++ b/tests/btrfs/150.out
>@@ -0,0 +1,3 @@
>+QA output created by 150
>+wrote 8192/8192 bytes at offset 0
>+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>diff --git a/tests/btrfs/group b/tests/btrfs/group
>index 70c3f05..b70a122 100644
>--- a/tests/btrfs/group
>+++ b/tests/btrfs/group
>@@ -152,3 +152,4 @@
> 147 auto quick send
> 148 auto quick rw
> 149 auto quick send compress
>+150 auto quick
>--
>2.5.0
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
--
Thanks,
Lu
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data
@ 2017-09-21 7:03 ` Lu Fengqi
0 siblings, 0 replies; 15+ messages in thread
From: Lu Fengqi @ 2017-09-21 7:03 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote:
>We had a bug in btrfs compression code which could end up with a
>kernel panic.
>
>This is adding a regression test for the bug and I've also sent a
>kernel patch to fix the bug.
>
>The patch is "Btrfs: fix kernel oops while reading compressed data".
>
>Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
>---
> tests/btrfs/150 | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/150.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 106 insertions(+)
> create mode 100755 tests/btrfs/150
> create mode 100644 tests/btrfs/150.out
>
>diff --git a/tests/btrfs/150 b/tests/btrfs/150
>new file mode 100755
>index 0000000..834be51
>--- /dev/null
>+++ b/tests/btrfs/150
>@@ -0,0 +1,102 @@
>+#! /bin/bash
>+# FS QA Test btrfs/150
>+#
>+# This is a regression test which ends up with a kernel oops in btrfs.
>+# It occurs when btrfs's read repair happens while reading a compressed
>+# extent.
>+# The patch for this is
>+# xxxxx
>+#
>+#-----------------------------------------------------------------------
>+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
>+#
>+# This program is free software; you can redistribute it and/or
>+# modify it under the terms of the GNU General Public License as
>+# published by the Free Software Foundation.
>+#
>+# This program is distributed in the hope that it would be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>+# GNU General Public License for more details.
>+#
>+# You should have received a copy of the GNU General Public License
>+# along with this program; if not, write the Free Software Foundation,
>+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>+#-----------------------------------------------------------------------
>+#
>+
>+seq=`basename $0`
>+seqres=$RESULT_DIR/$seq
>+echo "QA output created by $seq"
>+
>+here=`pwd`
>+tmp=/tmp/$$
>+status=1 # failure is the default!
>+trap "_cleanup; exit \$status" 0 1 2 3 15
>+
>+_cleanup()
>+{
>+ cd /
>+ rm -f $tmp.*
>+}
>+
>+# get standard environment, filters and checks
>+. ./common/rc
>+. ./common/filter
>+
>+# remove previous $seqres.full before test
>+rm -f $seqres.full
>+
>+# real QA test starts here
>+
>+# Modify as appropriate.
>+_supported_fs btrfs
>+_supported_os Linux
>+_require_scratch
>+_require_fail_make_request
>+_require_scratch_dev_pool 2
>+
>+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
>+enable_io_failure()
>+{
>+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
What does 1000 mean? Enough failures?
Why not set times to -1?
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
>+ echo 1 > $SYSFS_BDEV/make-it-fail
>+}
>+
>+disable_io_failure()
>+{
>+ echo 0 > $SYSFS_BDEV/make-it-fail
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
>+}
>+
>+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
>+
>+# It doesn't matter which compression algorithm we use.
>+_scratch_mount -ocompress
>+
>+# Create a file with all data being compressed
>+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
>+
>+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
>+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
>+# the bad copy to trigger read-repair.
>+while true; do
>+ disable_io_failure
>+ # invalidate the page cache
>+ $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | _filter_xfs_io
>+
>+ enable_io_failure
>+ od -x $SCRATCH_MNT/foobar > /dev/null &
>+ pid=$!
>+ wait
>+ [ $((pid % 2)) == 1 ] && break
>+done
>+
>+disable_io_failure
>+
>+# success, all done
>+status=0
>+exit
>diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
>new file mode 100644
>index 0000000..c492c24
>--- /dev/null
>+++ b/tests/btrfs/150.out
>@@ -0,0 +1,3 @@
>+QA output created by 150
>+wrote 8192/8192 bytes at offset 0
>+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>diff --git a/tests/btrfs/group b/tests/btrfs/group
>index 70c3f05..b70a122 100644
>--- a/tests/btrfs/group
>+++ b/tests/btrfs/group
>@@ -152,3 +152,4 @@
> 147 auto quick send
> 148 auto quick rw
> 149 auto quick send compress
>+150 auto quick
>--
>2.5.0
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
--
Thanks,
Lu
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data
2017-09-21 7:03 ` Lu Fengqi
(?)
@ 2017-09-22 19:28 ` Liu Bo
-1 siblings, 0 replies; 15+ messages in thread
From: Liu Bo @ 2017-09-22 19:28 UTC (permalink / raw)
To: Lu Fengqi; +Cc: fstests, linux-btrfs
On Thu, Sep 21, 2017 at 03:03:45PM +0800, Lu Fengqi wrote:
> On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote:
> >We had a bug in btrfs compression code which could end up with a
> >kernel panic.
> >
> >This is adding a regression test for the bug and I've also sent a
> >kernel patch to fix the bug.
> >
> >The patch is "Btrfs: fix kernel oops while reading compressed data".
> >
> >Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> >---
> > tests/btrfs/150 | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/150.out | 3 ++
> > tests/btrfs/group | 1 +
> > 3 files changed, 106 insertions(+)
> > create mode 100755 tests/btrfs/150
> > create mode 100644 tests/btrfs/150.out
> >
> >diff --git a/tests/btrfs/150 b/tests/btrfs/150
> >new file mode 100755
> >index 0000000..834be51
> >--- /dev/null
> >+++ b/tests/btrfs/150
> >@@ -0,0 +1,102 @@
> >+#! /bin/bash
> >+# FS QA Test btrfs/150
> >+#
> >+# This is a regression test which ends up with a kernel oops in btrfs.
> >+# It occurs when btrfs's read repair happens while reading a compressed
> >+# extent.
> >+# The patch for this is
> >+# xxxxx
> >+#
> >+#-----------------------------------------------------------------------
> >+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> >+#
> >+# This program is free software; you can redistribute it and/or
> >+# modify it under the terms of the GNU General Public License as
> >+# published by the Free Software Foundation.
> >+#
> >+# This program is distributed in the hope that it would be useful,
> >+# but WITHOUT ANY WARRANTY; without even the implied warranty of
> >+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >+# GNU General Public License for more details.
> >+#
> >+# You should have received a copy of the GNU General Public License
> >+# along with this program; if not, write the Free Software Foundation,
> >+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> >+#-----------------------------------------------------------------------
> >+#
> >+
> >+seq=`basename $0`
> >+seqres=$RESULT_DIR/$seq
> >+echo "QA output created by $seq"
> >+
> >+here=`pwd`
> >+tmp=/tmp/$$
> >+status=1 # failure is the default!
> >+trap "_cleanup; exit \$status" 0 1 2 3 15
> >+
> >+_cleanup()
> >+{
> >+ cd /
> >+ rm -f $tmp.*
> >+}
> >+
> >+# get standard environment, filters and checks
> >+. ./common/rc
> >+. ./common/filter
> >+
> >+# remove previous $seqres.full before test
> >+rm -f $seqres.full
> >+
> >+# real QA test starts here
> >+
> >+# Modify as appropriate.
> >+_supported_fs btrfs
> >+_supported_os Linux
> >+_require_scratch
> >+_require_fail_make_request
> >+_require_scratch_dev_pool 2
> >+
> >+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
> >+enable_io_failure()
> >+{
> >+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
> >+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
>
> What does 1000 mean? Enough failures?
> Why not set times to -1?
This was copied from another test, so I kept it as is.
As this test just submits a single 8K read after enabling fault
injection, 1000 is in fact as same as -1(no limit), I think 1000 is OK
to use.
thanks,
-liubo
>
> >+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
> >+ echo 1 > $SYSFS_BDEV/make-it-fail
> >+}
> >+
> >+disable_io_failure()
> >+{
> >+ echo 0 > $SYSFS_BDEV/make-it-fail
> >+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
> >+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
> >+}
> >+
> >+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
> >+
> >+# It doesn't matter which compression algorithm we use.
> >+_scratch_mount -ocompress
> >+
> >+# Create a file with all data being compressed
> >+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
> >+
> >+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
> >+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
> >+# the bad copy to trigger read-repair.
> >+while true; do
> >+ disable_io_failure
> >+ # invalidate the page cache
> >+ $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | _filter_xfs_io
> >+
> >+ enable_io_failure
> >+ od -x $SCRATCH_MNT/foobar > /dev/null &
> >+ pid=$!
> >+ wait
> >+ [ $((pid % 2)) == 1 ] && break
> >+done
> >+
> >+disable_io_failure
> >+
> >+# success, all done
> >+status=0
> >+exit
> >diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
> >new file mode 100644
> >index 0000000..c492c24
> >--- /dev/null
> >+++ b/tests/btrfs/150.out
> >@@ -0,0 +1,3 @@
> >+QA output created by 150
> >+wrote 8192/8192 bytes at offset 0
> >+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> >diff --git a/tests/btrfs/group b/tests/btrfs/group
> >index 70c3f05..b70a122 100644
> >--- a/tests/btrfs/group
> >+++ b/tests/btrfs/group
> >@@ -152,3 +152,4 @@
> > 147 auto quick send
> > 148 auto quick rw
> > 149 auto quick send compress
> >+150 auto quick
> >--
> >2.5.0
> >
> >--
> >To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> >
>
> --
> Thanks,
> Lu
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data
2017-09-21 4:39 ` Dave Chinner
@ 2017-09-22 19:33 ` Liu Bo
0 siblings, 0 replies; 15+ messages in thread
From: Liu Bo @ 2017-09-22 19:33 UTC (permalink / raw)
To: Dave Chinner; +Cc: fstests, linux-btrfs
On Thu, Sep 21, 2017 at 02:39:52PM +1000, Dave Chinner wrote:
> On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote:
> > We had a bug in btrfs compression code which could end up with a
> > kernel panic.
> >
> > This is adding a regression test for the bug and I've also sent a
> > kernel patch to fix the bug.
> >
> > The patch is "Btrfs: fix kernel oops while reading compressed data".
> >
> > Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> > ---
> > tests/btrfs/150 | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/150.out | 3 ++
> > tests/btrfs/group | 1 +
> > 3 files changed, 106 insertions(+)
> > create mode 100755 tests/btrfs/150
> > create mode 100644 tests/btrfs/150.out
> >
> > diff --git a/tests/btrfs/150 b/tests/btrfs/150
> > new file mode 100755
> > index 0000000..834be51
> > --- /dev/null
> > +++ b/tests/btrfs/150
> > @@ -0,0 +1,102 @@
> > +#! /bin/bash
> > +# FS QA Test btrfs/150
> > +#
> > +# This is a regression test which ends up with a kernel oops in btrfs.
>
> group += dangerous
OK.
>
> > +# It occurs when btrfs's read repair happens while reading a compressed
> > +# extent.
> > +# The patch for this is
> > +# xxxxx
>
> Incomplete?
Urr, thanks for pointing it out.
>
> > +#
> > +#-----------------------------------------------------------------------
> > +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
>
> You're signing off this patch an Oracle employee, but claiming
> personal copyright. Please clarify who owns the copyright - if it's
> your personal copyright then please sign off with a personal email
> address, not your employer's...
>
> Also, I note that these recently added tests from you:
>
> tests/btrfs/140:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> tests/btrfs/141:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> tests/btrfs/142:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> tests/btrfs/143:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> tests/generic/406:# Copyright (c) 2017 Liu Bo. All Rights Reserved.
>
> all have this same ambiguity - personal copyright with employer
> signoff in the commit. This definitely needs clarification and
> fixing if it is wrong....
>
All right, will fix all of them (in a separate one).
>
> > +disable_io_failure()
> > +{
> > + echo 0 > $SYSFS_BDEV/make-it-fail
> > + echo 0 > $DEBUGFS_MNT/fail_make_request/probability
> > + echo 0 > $DEBUGFS_MNT/fail_make_request/times
> > +}
> > +
> > +_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
> > +
> > +# It doesn't matter which compression algorithm we use.
> > +_scratch_mount -ocompress
> > +
> > +# Create a file with all data being compressed
> > +$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
>
> needs an fsync to reach disk.
'pwrite -W' has ensured that.
>
> > +# Raid1 consists of two copies and btrfs decides which copy to read by reader's
> > +# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
> > +# the bad copy to trigger read-repair.
> > +while true; do
> > + disable_io_failure
> > + # invalidate the page cache
> > + $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | _filter_xfs_io
> > +
> > + enable_io_failure
> > + od -x $SCRATCH_MNT/foobar > /dev/null &
>
> why are you using od to read the data when the output is piped to
> dev/null? why not just xfs_io -c "pread 0 8k" ?
Oh yes, that's better, will do.
thanks,
-liubo
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-20 23:52 [PATCH] fstests: btrfs/150 regression test for reading compressed data Liu Bo
2017-09-21 4:39 ` Dave Chinner
2017-09-21 7:03 ` Lu Fengqi
@ 2017-09-22 23:21 ` Liu Bo
2017-09-24 7:15 ` Lu Fengqi
2017-09-26 9:02 ` Eryu Guan
2017-09-27 16:35 ` [PATCH v3] " Liu Bo
3 siblings, 2 replies; 15+ messages in thread
From: Liu Bo @ 2017-09-22 23:21 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
We had a bug in btrfs compression code which could end up with a
kernel panic.
This is adding a regression test for the bug and I've also sent a
kernel patch to fix the bug.
The patch is "Btrfs: fix kernel oops while reading compressed data".
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
---
v2: - Fix ambiguous copyright.
- Use /proc/$pid/make-it-fail to specify IO failure
- Use bash -c to run test only when pid is odd.
- Add test to dangerous group.
tests/btrfs/150 | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/150.out | 3 ++
tests/btrfs/group | 1 +
3 files changed, 107 insertions(+)
create mode 100755 tests/btrfs/150
create mode 100644 tests/btrfs/150.out
diff --git a/tests/btrfs/150 b/tests/btrfs/150
new file mode 100755
index 0000000..8891c38
--- /dev/null
+++ b/tests/btrfs/150
@@ -0,0 +1,103 @@
+#! /bin/bash
+# FS QA Test btrfs/150
+#
+# This is a regression test which ends up with a kernel oops in btrfs.
+# It occurs when btrfs's read repair happens while reading a compressed
+# extent.
+# The patch to fix it is
+# Btrfs: fix kernel oops while reading compressed data
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Oracle. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_fail_make_request
+_require_scratch_dev_pool 2
+
+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
+enable_io_failure()
+{
+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
+ echo 1 > $SYSFS_BDEV/make-it-fail
+}
+
+disable_io_failure()
+{
+ echo 0 > $SYSFS_BDEV/make-it-fail
+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
+}
+
+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
+
+# It doesn't matter which compression algorithm we use.
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
+
+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
+# the bad copy to trigger read-repair.
+while [[ -z $result ]]; do
+ # invalidate the page cache
+ $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
+
+ enable_io_failure
+
+ result=$(bash -c "
+ if [ \$((\$\$ % 2)) == 1 ]; then
+ echo 1 > /proc/\$\$/make-it-fail
+ exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
+ fi")
+
+ disable_io_failure
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
new file mode 100644
index 0000000..c492c24
--- /dev/null
+++ b/tests/btrfs/150.out
@@ -0,0 +1,3 @@
+QA output created by 150
+wrote 8192/8192 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 70c3f05..e73bb1b 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -152,3 +152,4 @@
147 auto quick send
148 auto quick rw
149 auto quick send compress
+150 auto quick dangerous
--
2.5.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-22 23:21 ` [PATCH v2] " Liu Bo
@ 2017-09-24 7:15 ` Lu Fengqi
2017-09-26 9:02 ` Eryu Guan
1 sibling, 0 replies; 15+ messages in thread
From: Lu Fengqi @ 2017-09-24 7:15 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
>We had a bug in btrfs compression code which could end up with a
>kernel panic.
>
>This is adding a regression test for the bug and I've also sent a
>kernel patch to fix the bug.
>
>The patch is "Btrfs: fix kernel oops while reading compressed data".
>
>Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
>---
>v2: - Fix ambiguous copyright.
> - Use /proc/$pid/make-it-fail to specify IO failure
- /sys/kernel/debug/fail*/task-filter:
Format: { 'Y' | 'N' }
A value of 'N' disables filtering by process (default).
Any positive value limits failures to only processes indicated by
/proc/<pid>/make-it-fail==1.
--
Thanks,
Lu
> - Use bash -c to run test only when pid is odd.
> - Add test to dangerous group.
>
> tests/btrfs/150 | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/150.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 107 insertions(+)
> create mode 100755 tests/btrfs/150
> create mode 100644 tests/btrfs/150.out
>
>diff --git a/tests/btrfs/150 b/tests/btrfs/150
>new file mode 100755
>index 0000000..8891c38
>--- /dev/null
>+++ b/tests/btrfs/150
>@@ -0,0 +1,103 @@
>+#! /bin/bash
>+# FS QA Test btrfs/150
>+#
>+# This is a regression test which ends up with a kernel oops in btrfs.
>+# It occurs when btrfs's read repair happens while reading a compressed
>+# extent.
>+# The patch to fix it is
>+# Btrfs: fix kernel oops while reading compressed data
>+#
>+#-----------------------------------------------------------------------
>+# Copyright (c) 2017 Oracle. All Rights Reserved.
>+#
>+# This program is free software; you can redistribute it and/or
>+# modify it under the terms of the GNU General Public License as
>+# published by the Free Software Foundation.
>+#
>+# This program is distributed in the hope that it would be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>+# GNU General Public License for more details.
>+#
>+# You should have received a copy of the GNU General Public License
>+# along with this program; if not, write the Free Software Foundation,
>+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>+#-----------------------------------------------------------------------
>+#
>+
>+seq=`basename $0`
>+seqres=$RESULT_DIR/$seq
>+echo "QA output created by $seq"
>+
>+here=`pwd`
>+tmp=/tmp/$$
>+status=1 # failure is the default!
>+trap "_cleanup; exit \$status" 0 1 2 3 15
>+
>+_cleanup()
>+{
>+ cd /
>+ rm -f $tmp.*
>+}
>+
>+# get standard environment, filters and checks
>+. ./common/rc
>+. ./common/filter
>+
>+# remove previous $seqres.full before test
>+rm -f $seqres.full
>+
>+# real QA test starts here
>+
>+# Modify as appropriate.
>+_supported_fs btrfs
>+_supported_os Linux
>+_require_scratch
>+_require_fail_make_request
>+_require_scratch_dev_pool 2
>+
>+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
>+enable_io_failure()
>+{
>+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
>+ echo 1 > $SYSFS_BDEV/make-it-fail
>+}
>+
>+disable_io_failure()
>+{
>+ echo 0 > $SYSFS_BDEV/make-it-fail
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
>+}
>+
>+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
>+
>+# It doesn't matter which compression algorithm we use.
>+_scratch_mount -ocompress
>+
>+# Create a file with all data being compressed
>+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
>+
>+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
>+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
>+# the bad copy to trigger read-repair.
>+while [[ -z $result ]]; do
>+ # invalidate the page cache
>+ $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
>+
>+ enable_io_failure
>+
>+ result=$(bash -c "
>+ if [ \$((\$\$ % 2)) == 1 ]; then
>+ echo 1 > /proc/\$\$/make-it-fail
>+ exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
>+ fi")
>+
>+ disable_io_failure
>+done
>+
>+# success, all done
>+status=0
>+exit
>diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
>new file mode 100644
>index 0000000..c492c24
>--- /dev/null
>+++ b/tests/btrfs/150.out
>@@ -0,0 +1,3 @@
>+QA output created by 150
>+wrote 8192/8192 bytes at offset 0
>+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>diff --git a/tests/btrfs/group b/tests/btrfs/group
>index 70c3f05..e73bb1b 100644
>--- a/tests/btrfs/group
>+++ b/tests/btrfs/group
>@@ -152,3 +152,4 @@
> 147 auto quick send
> 148 auto quick rw
> 149 auto quick send compress
>+150 auto quick dangerous
>--
>2.5.0
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
@ 2017-09-24 7:15 ` Lu Fengqi
0 siblings, 0 replies; 15+ messages in thread
From: Lu Fengqi @ 2017-09-24 7:15 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
>We had a bug in btrfs compression code which could end up with a
>kernel panic.
>
>This is adding a regression test for the bug and I've also sent a
>kernel patch to fix the bug.
>
>The patch is "Btrfs: fix kernel oops while reading compressed data".
>
>Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
>---
>v2: - Fix ambiguous copyright.
> - Use /proc/$pid/make-it-fail to specify IO failure
- /sys/kernel/debug/fail*/task-filter:
Format: { 'Y' | 'N' }
A value of 'N' disables filtering by process (default).
Any positive value limits failures to only processes indicated by
/proc/<pid>/make-it-fail==1.
--
Thanks,
Lu
> - Use bash -c to run test only when pid is odd.
> - Add test to dangerous group.
>
> tests/btrfs/150 | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/150.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 107 insertions(+)
> create mode 100755 tests/btrfs/150
> create mode 100644 tests/btrfs/150.out
>
>diff --git a/tests/btrfs/150 b/tests/btrfs/150
>new file mode 100755
>index 0000000..8891c38
>--- /dev/null
>+++ b/tests/btrfs/150
>@@ -0,0 +1,103 @@
>+#! /bin/bash
>+# FS QA Test btrfs/150
>+#
>+# This is a regression test which ends up with a kernel oops in btrfs.
>+# It occurs when btrfs's read repair happens while reading a compressed
>+# extent.
>+# The patch to fix it is
>+# Btrfs: fix kernel oops while reading compressed data
>+#
>+#-----------------------------------------------------------------------
>+# Copyright (c) 2017 Oracle. All Rights Reserved.
>+#
>+# This program is free software; you can redistribute it and/or
>+# modify it under the terms of the GNU General Public License as
>+# published by the Free Software Foundation.
>+#
>+# This program is distributed in the hope that it would be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>+# GNU General Public License for more details.
>+#
>+# You should have received a copy of the GNU General Public License
>+# along with this program; if not, write the Free Software Foundation,
>+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>+#-----------------------------------------------------------------------
>+#
>+
>+seq=`basename $0`
>+seqres=$RESULT_DIR/$seq
>+echo "QA output created by $seq"
>+
>+here=`pwd`
>+tmp=/tmp/$$
>+status=1 # failure is the default!
>+trap "_cleanup; exit \$status" 0 1 2 3 15
>+
>+_cleanup()
>+{
>+ cd /
>+ rm -f $tmp.*
>+}
>+
>+# get standard environment, filters and checks
>+. ./common/rc
>+. ./common/filter
>+
>+# remove previous $seqres.full before test
>+rm -f $seqres.full
>+
>+# real QA test starts here
>+
>+# Modify as appropriate.
>+_supported_fs btrfs
>+_supported_os Linux
>+_require_scratch
>+_require_fail_make_request
>+_require_scratch_dev_pool 2
>+
>+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
>+enable_io_failure()
>+{
>+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
>+ echo 1 > $SYSFS_BDEV/make-it-fail
>+}
>+
>+disable_io_failure()
>+{
>+ echo 0 > $SYSFS_BDEV/make-it-fail
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
>+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
>+}
>+
>+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
>+
>+# It doesn't matter which compression algorithm we use.
>+_scratch_mount -ocompress
>+
>+# Create a file with all data being compressed
>+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
>+
>+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
>+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
>+# the bad copy to trigger read-repair.
>+while [[ -z $result ]]; do
>+ # invalidate the page cache
>+ $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
>+
>+ enable_io_failure
>+
>+ result=$(bash -c "
>+ if [ \$((\$\$ % 2)) == 1 ]; then
>+ echo 1 > /proc/\$\$/make-it-fail
>+ exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
>+ fi")
>+
>+ disable_io_failure
>+done
>+
>+# success, all done
>+status=0
>+exit
>diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
>new file mode 100644
>index 0000000..c492c24
>--- /dev/null
>+++ b/tests/btrfs/150.out
>@@ -0,0 +1,3 @@
>+QA output created by 150
>+wrote 8192/8192 bytes at offset 0
>+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>diff --git a/tests/btrfs/group b/tests/btrfs/group
>index 70c3f05..e73bb1b 100644
>--- a/tests/btrfs/group
>+++ b/tests/btrfs/group
>@@ -152,3 +152,4 @@
> 147 auto quick send
> 148 auto quick rw
> 149 auto quick send compress
>+150 auto quick dangerous
>--
>2.5.0
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-22 23:21 ` [PATCH v2] " Liu Bo
2017-09-24 7:15 ` Lu Fengqi
@ 2017-09-26 9:02 ` Eryu Guan
2017-09-26 23:37 ` Liu Bo
1 sibling, 1 reply; 15+ messages in thread
From: Eryu Guan @ 2017-09-26 9:02 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
> We had a bug in btrfs compression code which could end up with a
> kernel panic.
>
> This is adding a regression test for the bug and I've also sent a
> kernel patch to fix the bug.
>
> The patch is "Btrfs: fix kernel oops while reading compressed data".
>
> Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Hmm, I can't reproduce the panic with 4.13 kernel, which doesn't have
the fix applied. Can you please help confirm if it panics on your test
environment?
> ---
> v2: - Fix ambiguous copyright.
> - Use /proc/$pid/make-it-fail to specify IO failure
> - Use bash -c to run test only when pid is odd.
> - Add test to dangerous group.
>
> tests/btrfs/150 | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/150.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 107 insertions(+)
> create mode 100755 tests/btrfs/150
> create mode 100644 tests/btrfs/150.out
>
> diff --git a/tests/btrfs/150 b/tests/btrfs/150
> new file mode 100755
> index 0000000..8891c38
> --- /dev/null
> +++ b/tests/btrfs/150
> @@ -0,0 +1,103 @@
> +#! /bin/bash
> +# FS QA Test btrfs/150
> +#
> +# This is a regression test which ends up with a kernel oops in btrfs.
> +# It occurs when btrfs's read repair happens while reading a compressed
> +# extent.
> +# The patch to fix it is
> +# Btrfs: fix kernel oops while reading compressed data
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2017 Oracle. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_fail_make_request
> +_require_scratch_dev_pool 2
Trailing whitespace in above line.
> +
> +SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
> +enable_io_failure()
> +{
> + echo 100 > $DEBUGFS_MNT/fail_make_request/probability
> + echo 1000 > $DEBUGFS_MNT/fail_make_request/times
> + echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
> + echo 1 > $SYSFS_BDEV/make-it-fail
> +}
> +
> +disable_io_failure()
> +{
> + echo 0 > $SYSFS_BDEV/make-it-fail
> + echo 0 > $DEBUGFS_MNT/fail_make_request/probability
> + echo 0 > $DEBUGFS_MNT/fail_make_request/times
> +}
> +
> +_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
> +
> +# It doesn't matter which compression algorithm we use.
> +_scratch_mount -ocompress
> +
> +# Create a file with all data being compressed
> +$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
> +
> +# Raid1 consists of two copies and btrfs decides which copy to read by reader's
> +# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
> +# the bad copy to trigger read-repair.
> +while [[ -z $result ]]; do
> + # invalidate the page cache
> + $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
Does 'echo 3 > /proc/sys/vm/drop_caches' work?
Thanks,
Eryu
> +
> + enable_io_failure
> +
> + result=$(bash -c "
> + if [ \$((\$\$ % 2)) == 1 ]; then
> + echo 1 > /proc/\$\$/make-it-fail
> + exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
> + fi")
> +
> + disable_io_failure
> +done
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
> new file mode 100644
> index 0000000..c492c24
> --- /dev/null
> +++ b/tests/btrfs/150.out
> @@ -0,0 +1,3 @@
> +QA output created by 150
> +wrote 8192/8192 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 70c3f05..e73bb1b 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -152,3 +152,4 @@
> 147 auto quick send
> 148 auto quick rw
> 149 auto quick send compress
> +150 auto quick dangerous
> --
> 2.5.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-26 9:02 ` Eryu Guan
@ 2017-09-26 23:37 ` Liu Bo
2017-09-27 0:18 ` Liu Bo
0 siblings, 1 reply; 15+ messages in thread
From: Liu Bo @ 2017-09-26 23:37 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs
On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote:
> On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
> > We had a bug in btrfs compression code which could end up with a
> > kernel panic.
> >
> > This is adding a regression test for the bug and I've also sent a
> > kernel patch to fix the bug.
> >
> > The patch is "Btrfs: fix kernel oops while reading compressed data".
> >
> > Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
>
> Hmm, I can't reproduce the panic with 4.13 kernel, which doesn't have
> the fix applied. Can you please help confirm if it panics on your test
> environment?
>
Yes, it is reproducible on my box, hrm...I'll be running it more times
to double check.
> > ---
> > v2: - Fix ambiguous copyright.
> > - Use /proc/$pid/make-it-fail to specify IO failure
> > - Use bash -c to run test only when pid is odd.
> > - Add test to dangerous group.
> >
> > tests/btrfs/150 | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/150.out | 3 ++
> > tests/btrfs/group | 1 +
> > 3 files changed, 107 insertions(+)
> > create mode 100755 tests/btrfs/150
> > create mode 100644 tests/btrfs/150.out
> >
> > diff --git a/tests/btrfs/150 b/tests/btrfs/150
> > new file mode 100755
> > index 0000000..8891c38
> > --- /dev/null
> > +++ b/tests/btrfs/150
> > @@ -0,0 +1,103 @@
> > +#! /bin/bash
> > +# FS QA Test btrfs/150
> > +#
> > +# This is a regression test which ends up with a kernel oops in btrfs.
> > +# It occurs when btrfs's read repair happens while reading a compressed
> > +# extent.
> > +# The patch to fix it is
> > +# Btrfs: fix kernel oops while reading compressed data
> > +#
> > +#-----------------------------------------------------------------------
> > +# Copyright (c) 2017 Oracle. All Rights Reserved.
> > +#
> > +# This program is free software; you can redistribute it and/or
> > +# modify it under the terms of the GNU General Public License as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it would be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program; if not, write the Free Software Foundation,
> > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> > +#-----------------------------------------------------------------------
> > +#
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# remove previous $seqres.full before test
> > +rm -f $seqres.full
> > +
> > +# real QA test starts here
> > +
> > +# Modify as appropriate.
> > +_supported_fs btrfs
> > +_supported_os Linux
> > +_require_scratch
> > +_require_fail_make_request
> > +_require_scratch_dev_pool 2
>
> Trailing whitespace in above line.
>
> > +
> > +SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
> > +enable_io_failure()
> > +{
> > + echo 100 > $DEBUGFS_MNT/fail_make_request/probability
> > + echo 1000 > $DEBUGFS_MNT/fail_make_request/times
> > + echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
> > + echo 1 > $SYSFS_BDEV/make-it-fail
> > +}
> > +
> > +disable_io_failure()
> > +{
> > + echo 0 > $SYSFS_BDEV/make-it-fail
> > + echo 0 > $DEBUGFS_MNT/fail_make_request/probability
> > + echo 0 > $DEBUGFS_MNT/fail_make_request/times
> > +}
> > +
> > +_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
> > +
> > +# It doesn't matter which compression algorithm we use.
> > +_scratch_mount -ocompress
> > +
> > +# Create a file with all data being compressed
> > +$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
> > +
> > +# Raid1 consists of two copies and btrfs decides which copy to read by reader's
> > +# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
> > +# the bad copy to trigger read-repair.
> > +while [[ -z $result ]]; do
> > + # invalidate the page cache
> > + $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
>
> Does 'echo 3 > /proc/sys/vm/drop_caches' work?
>
Yes, it works, drop_caches is system-wide, while here I'm just
dropping caches on this single inode.
Or are you implying that it's 'fadvise' that makes the test fail to
show oops?
thanks,
-liubo
> Thanks,
> Eryu
>
> > +
> > + enable_io_failure
> > +
> > + result=$(bash -c "
> > + if [ \$((\$\$ % 2)) == 1 ]; then
> > + echo 1 > /proc/\$\$/make-it-fail
> > + exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
> > + fi")
> > +
> > + disable_io_failure
> > +done
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
> > new file mode 100644
> > index 0000000..c492c24
> > --- /dev/null
> > +++ b/tests/btrfs/150.out
> > @@ -0,0 +1,3 @@
> > +QA output created by 150
> > +wrote 8192/8192 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > diff --git a/tests/btrfs/group b/tests/btrfs/group
> > index 70c3f05..e73bb1b 100644
> > --- a/tests/btrfs/group
> > +++ b/tests/btrfs/group
> > @@ -152,3 +152,4 @@
> > 147 auto quick send
> > 148 auto quick rw
> > 149 auto quick send compress
> > +150 auto quick dangerous
> > --
> > 2.5.0
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-26 23:37 ` Liu Bo
@ 2017-09-27 0:18 ` Liu Bo
2017-09-27 9:46 ` Eryu Guan
0 siblings, 1 reply; 15+ messages in thread
From: Liu Bo @ 2017-09-27 0:18 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs
On Tue, Sep 26, 2017 at 04:37:52PM -0700, Liu Bo wrote:
> On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote:
> > On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
> > > We had a bug in btrfs compression code which could end up with a
> > > kernel panic.
> > >
> > > This is adding a regression test for the bug and I've also sent a
> > > kernel patch to fix the bug.
> > >
> > > The patch is "Btrfs: fix kernel oops while reading compressed data".
> > >
> > > Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> >
> > Hmm, I can't reproduce the panic with 4.13 kernel, which doesn't have
> > the fix applied. Can you please help confirm if it panics on your test
> > environment?
> >
>
> Yes, it is reproducible on my box, hrm...I'll be running it more times
> to double check.
>
It worked for me...both v4.13 and v4.14.0-rc2 have the following
messages[1].
This requires two config:
CONFIG_FAULT_INJECTION=y
CONFIG_FAULT_INJECTION_DEBUG_FS=y
Could you please check again?
[1]:
[ 135.982643] run fstests btrfs/150 at 2017-09-26 16:11:27
[ 136.839434] BTRFS: device fsid 9152fe7e-3006-47d5-a9b7-330af2809da7
devid 1 transid 5 /dev/sde
[ 136.842082] BTRFS: device fsid 9152fe7e-3006-47d5-a9b7-330af2809da7
devid 2 transid 5 /dev/sdc
[ 136.879626] BTRFS info (device sdc): use zlib compression
[ 136.880263] BTRFS info (device sdc): disk space caching is enabled
[ 136.880845] BTRFS info (device sdc): has skinny extents
[ 136.881386] BTRFS info (device sdc): flagging fs with big metadata
feature
[ 136.890763] BTRFS info (device sdc): creating UUID tree
[ 137.023210] BTRFS error (device sdc): bdev /dev/sde errs: wr 0, rd 1,
flush 0, corrupt 0, gen 0
[ 137.023959] BTRFS warning (device sdc): csum failed root 5 ino 257
off 136839168 csum 0x98f94189 expected csum 0xd9cece72 mirror 0
[ 137.025349] ------------[ cut here ]------------
[ 137.025735] kernel BUG at fs/btrfs/extent_io.c:2104!
[ 137.025800] ------------[ cut here ]------------
[ 137.025805] kernel BUG at fs/btrfs/extent_io.c:2104!
Thanks,
-liubo
> > > ---
> > > v2: - Fix ambiguous copyright.
> > > - Use /proc/$pid/make-it-fail to specify IO failure
> > > - Use bash -c to run test only when pid is odd.
> > > - Add test to dangerous group.
> > >
> > > tests/btrfs/150 | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > tests/btrfs/150.out | 3 ++
> > > tests/btrfs/group | 1 +
> > > 3 files changed, 107 insertions(+)
> > > create mode 100755 tests/btrfs/150
> > > create mode 100644 tests/btrfs/150.out
> > >
> > > diff --git a/tests/btrfs/150 b/tests/btrfs/150
> > > new file mode 100755
> > > index 0000000..8891c38
> > > --- /dev/null
> > > +++ b/tests/btrfs/150
> > > @@ -0,0 +1,103 @@
> > > +#! /bin/bash
> > > +# FS QA Test btrfs/150
> > > +#
> > > +# This is a regression test which ends up with a kernel oops in btrfs.
> > > +# It occurs when btrfs's read repair happens while reading a compressed
> > > +# extent.
> > > +# The patch to fix it is
> > > +# Btrfs: fix kernel oops while reading compressed data
> > > +#
> > > +#-----------------------------------------------------------------------
> > > +# Copyright (c) 2017 Oracle. All Rights Reserved.
> > > +#
> > > +# This program is free software; you can redistribute it and/or
> > > +# modify it under the terms of the GNU General Public License as
> > > +# published by the Free Software Foundation.
> > > +#
> > > +# This program is distributed in the hope that it would be useful,
> > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > > +# GNU General Public License for more details.
> > > +#
> > > +# You should have received a copy of the GNU General Public License
> > > +# along with this program; if not, write the Free Software Foundation,
> > > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> > > +#-----------------------------------------------------------------------
> > > +#
> > > +
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +
> > > +# remove previous $seqres.full before test
> > > +rm -f $seqres.full
> > > +
> > > +# real QA test starts here
> > > +
> > > +# Modify as appropriate.
> > > +_supported_fs btrfs
> > > +_supported_os Linux
> > > +_require_scratch
> > > +_require_fail_make_request
> > > +_require_scratch_dev_pool 2
> >
> > Trailing whitespace in above line.
> >
> > > +
> > > +SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
> > > +enable_io_failure()
> > > +{
> > > + echo 100 > $DEBUGFS_MNT/fail_make_request/probability
> > > + echo 1000 > $DEBUGFS_MNT/fail_make_request/times
> > > + echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
> > > + echo 1 > $SYSFS_BDEV/make-it-fail
> > > +}
> > > +
> > > +disable_io_failure()
> > > +{
> > > + echo 0 > $SYSFS_BDEV/make-it-fail
> > > + echo 0 > $DEBUGFS_MNT/fail_make_request/probability
> > > + echo 0 > $DEBUGFS_MNT/fail_make_request/times
> > > +}
> > > +
> > > +_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
> > > +
> > > +# It doesn't matter which compression algorithm we use.
> > > +_scratch_mount -ocompress
> > > +
> > > +# Create a file with all data being compressed
> > > +$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
> > > +
> > > +# Raid1 consists of two copies and btrfs decides which copy to read by reader's
> > > +# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
> > > +# the bad copy to trigger read-repair.
> > > +while [[ -z $result ]]; do
> > > + # invalidate the page cache
> > > + $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
> >
> > Does 'echo 3 > /proc/sys/vm/drop_caches' work?
> >
>
> Yes, it works, drop_caches is system-wide, while here I'm just
> dropping caches on this single inode.
>
> Or are you implying that it's 'fadvise' that makes the test fail to
> show oops?
>
> thanks,
>
> -liubo
>
> > Thanks,
> > Eryu
> >
> > > +
> > > + enable_io_failure
> > > +
> > > + result=$(bash -c "
> > > + if [ \$((\$\$ % 2)) == 1 ]; then
> > > + echo 1 > /proc/\$\$/make-it-fail
> > > + exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
> > > + fi")
> > > +
> > > + disable_io_failure
> > > +done
> > > +
> > > +# success, all done
> > > +status=0
> > > +exit
> > > diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
> > > new file mode 100644
> > > index 0000000..c492c24
> > > --- /dev/null
> > > +++ b/tests/btrfs/150.out
> > > @@ -0,0 +1,3 @@
> > > +QA output created by 150
> > > +wrote 8192/8192 bytes at offset 0
> > > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > > diff --git a/tests/btrfs/group b/tests/btrfs/group
> > > index 70c3f05..e73bb1b 100644
> > > --- a/tests/btrfs/group
> > > +++ b/tests/btrfs/group
> > > @@ -152,3 +152,4 @@
> > > 147 auto quick send
> > > 148 auto quick rw
> > > 149 auto quick send compress
> > > +150 auto quick dangerous
> > > --
> > > 2.5.0
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-27 0:18 ` Liu Bo
@ 2017-09-27 9:46 ` Eryu Guan
2017-09-27 17:10 ` Liu Bo
0 siblings, 1 reply; 15+ messages in thread
From: Eryu Guan @ 2017-09-27 9:46 UTC (permalink / raw)
To: Liu Bo; +Cc: fstests, linux-btrfs
On Tue, Sep 26, 2017 at 05:18:51PM -0700, Liu Bo wrote:
> On Tue, Sep 26, 2017 at 04:37:52PM -0700, Liu Bo wrote:
> > On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote:
> > > On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
> > > > We had a bug in btrfs compression code which could end up with a
> > > > kernel panic.
> > > >
> > > > This is adding a regression test for the bug and I've also sent a
> > > > kernel patch to fix the bug.
> > > >
> > > > The patch is "Btrfs: fix kernel oops while reading compressed data".
> > > >
> > > > Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> > >
> > > Hmm, I can't reproduce the panic with 4.13 kernel, which doesn't have
> > > the fix applied. Can you please help confirm if it panics on your test
> > > environment?
> > >
> >
> > Yes, it is reproducible on my box, hrm...I'll be running it more times
> > to double check.
> >
>
> It worked for me...both v4.13 and v4.14.0-rc2 have the following
> messages[1].
>
> This requires two config:
> CONFIG_FAULT_INJECTION=y
> CONFIG_FAULT_INJECTION_DEBUG_FS=y
>
> Could you please check again?
I re-compiled 4.14-rc2 kernel on my test vm with FAIL_MAKE_REQUEST
enabled (which requires FAULT_INJECTION), and I can reproduce the crash
now. It was so weired that previously I did have FAIL_MAKE_REQUEST
enabled and test ran normally without hitting the bug, but now I can hit
the bug quite reliably. Not sure what was happning in my previous test..
Thanks for confirming!
Eryu
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3] fstests: btrfs/150 regression test for reading compressed data
2017-09-20 23:52 [PATCH] fstests: btrfs/150 regression test for reading compressed data Liu Bo
` (2 preceding siblings ...)
2017-09-22 23:21 ` [PATCH v2] " Liu Bo
@ 2017-09-27 16:35 ` Liu Bo
3 siblings, 0 replies; 15+ messages in thread
From: Liu Bo @ 2017-09-27 16:35 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
We had a bug in btrfs compression code which could end up with a
kernel panic.
This is adding a regression test for the bug and I've also sent a
kernel patch to fix the bug.
The patch is "Btrfs: fix kernel oops while reading compressed data".
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
---
v3: - Enable task-filter in debugfs
v2: - Fix ambiguous copyright.
- Use /proc/$pid/make-it-fail to specify IO failure
- Use bash -c to run test only when pid is odd.
- Add test to dangerous group.
tests/btrfs/150 | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/150.out | 3 ++
tests/btrfs/group | 1 +
3 files changed, 109 insertions(+)
create mode 100755 tests/btrfs/150
create mode 100644 tests/btrfs/150.out
diff --git a/tests/btrfs/150 b/tests/btrfs/150
new file mode 100755
index 0000000..97041b6
--- /dev/null
+++ b/tests/btrfs/150
@@ -0,0 +1,105 @@
+#! /bin/bash
+# FS QA Test btrfs/150
+#
+# This is a regression test which ends up with a kernel oops in btrfs.
+# It occurs when btrfs's read repair happens while reading a compressed
+# extent.
+# The patch to fix it is
+# Btrfs: fix kernel oops while reading compressed data
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Oracle. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_fail_make_request
+_require_scratch_dev_pool 2
+
+SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV`
+enable_io_failure()
+{
+ echo 100 > $DEBUGFS_MNT/fail_make_request/probability
+ echo 1000 > $DEBUGFS_MNT/fail_make_request/times
+ echo 0 > $DEBUGFS_MNT/fail_make_request/verbose
+ echo 1 > $DEBUGFS_MNT/fail_make_request/task-filter
+ echo 1 > $SYSFS_BDEV/make-it-fail
+}
+
+disable_io_failure()
+{
+ echo 0 > $DEBUGFS_MNT/fail_make_request/probability
+ echo 0 > $DEBUGFS_MNT/fail_make_request/times
+ echo 0 > $DEBUGFS_MNT/fail_make_request/task-filter
+ echo 0 > $SYSFS_BDEV/make-it-fail
+}
+
+_scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
+
+# It doesn't matter which compression algorithm we use.
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -W 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
+
+# Raid1 consists of two copies and btrfs decides which copy to read by reader's
+# %pid. Now we inject errors to copy #1 and copy #0 is good. We want to read
+# the bad copy to trigger read-repair.
+while [[ -z $result ]]; do
+ # invalidate the page cache
+ $XFS_IO_PROG -f -c "fadvise -d 0 8K" $SCRATCH_MNT/foobar
+
+ enable_io_failure
+
+ result=$(bash -c "
+ if [ \$((\$\$ % 2)) == 1 ]; then
+ echo 1 > /proc/\$\$/make-it-fail
+ exec $XFS_IO_PROG -c \"pread 0 8K\" \$SCRATCH_MNT/foobar
+ fi")
+
+ disable_io_failure
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/150.out b/tests/btrfs/150.out
new file mode 100644
index 0000000..c492c24
--- /dev/null
+++ b/tests/btrfs/150.out
@@ -0,0 +1,3 @@
+QA output created by 150
+wrote 8192/8192 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 70c3f05..e73bb1b 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -152,3 +152,4 @@
147 auto quick send
148 auto quick rw
149 auto quick send compress
+150 auto quick dangerous
--
2.9.4
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data
2017-09-27 9:46 ` Eryu Guan
@ 2017-09-27 17:10 ` Liu Bo
0 siblings, 0 replies; 15+ messages in thread
From: Liu Bo @ 2017-09-27 17:10 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs
On Wed, Sep 27, 2017 at 05:46:44PM +0800, Eryu Guan wrote:
> On Tue, Sep 26, 2017 at 05:18:51PM -0700, Liu Bo wrote:
> > On Tue, Sep 26, 2017 at 04:37:52PM -0700, Liu Bo wrote:
> > > On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote:
> > > > On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote:
> > > > > We had a bug in btrfs compression code which could end up with a
> > > > > kernel panic.
> > > > >
> > > > > This is adding a regression test for the bug and I've also sent a
> > > > > kernel patch to fix the bug.
> > > > >
> > > > > The patch is "Btrfs: fix kernel oops while reading compressed data".
> > > > >
> > > > > Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> > > >
> > > > Hmm, I can't reproduce the panic with 4.13 kernel, which doesn't have
> > > > the fix applied. Can you please help confirm if it panics on your test
> > > > environment?
> > > >
> > >
> > > Yes, it is reproducible on my box, hrm...I'll be running it more times
> > > to double check.
> > >
> >
> > It worked for me...both v4.13 and v4.14.0-rc2 have the following
> > messages[1].
> >
> > This requires two config:
> > CONFIG_FAULT_INJECTION=y
> > CONFIG_FAULT_INJECTION_DEBUG_FS=y
> >
> > Could you please check again?
>
> I re-compiled 4.14-rc2 kernel on my test vm with FAIL_MAKE_REQUEST
> enabled (which requires FAULT_INJECTION), and I can reproduce the crash
> now. It was so weired that previously I did have FAIL_MAKE_REQUEST
> enabled and test ran normally without hitting the bug, but now I can hit
> the bug quite reliably. Not sure what was happning in my previous test..
>
> Thanks for confirming!
No problem at all, then I'll send a patch v3 with enabling task-filter
pointed out by Lu.
thanks,
-liubo
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2017-09-27 17:39 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-20 23:52 [PATCH] fstests: btrfs/150 regression test for reading compressed data Liu Bo
2017-09-21 4:39 ` Dave Chinner
2017-09-22 19:33 ` Liu Bo
2017-09-21 7:03 ` Lu Fengqi
2017-09-21 7:03 ` Lu Fengqi
2017-09-22 19:28 ` Liu Bo
2017-09-22 23:21 ` [PATCH v2] " Liu Bo
2017-09-24 7:15 ` Lu Fengqi
2017-09-24 7:15 ` Lu Fengqi
2017-09-26 9:02 ` Eryu Guan
2017-09-26 23:37 ` Liu Bo
2017-09-27 0:18 ` Liu Bo
2017-09-27 9:46 ` Eryu Guan
2017-09-27 17:10 ` Liu Bo
2017-09-27 16:35 ` [PATCH v3] " Liu Bo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.