* [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
@ 2016-04-19 22:25 Mark Fasheh
2016-04-20 0:48 ` Satoru Takeuchi
0 siblings, 1 reply; 7+ messages in thread
From: Mark Fasheh @ 2016-04-19 22:25 UTC (permalink / raw)
To: fstests, linux-btrfs; +Cc: Qu Wenruo
This has been broken since Linux v4.1. We may have worked out a solution on
the btrfs list but in the meantime sending a test to expose the issue seems
like a good idea.
Signed-off-by: Mark Fasheh <mfasheh@suse.de>
---
tests/btrfs/122 | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/group | 1 +
2 files changed, 89 insertions(+)
create mode 100755 tests/btrfs/122
diff --git a/tests/btrfs/122 b/tests/btrfs/122
new file mode 100755
index 0000000..b7e9e4b
--- /dev/null
+++ b/tests/btrfs/122
@@ -0,0 +1,88 @@
+#! /bin/bash
+# FS QA Test No. btrfs/122
+#
+# Test that qgroup counts are valid after snapshot creation. This has
+# been broken in btrfs since Linux v4.1
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+# Force a small leaf size to make it easier to blow out our root
+# subvolume tree
+_scratch_mkfs "--nodesize 16384"
+_scratch_mount
+_run_btrfs_util_prog quota enable $SCRATCH_MNT
+
+mkdir "$SCRATCH_MNT/snaps"
+
+# First make some simple snapshots - the bug was initially reproduced like this
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty2"
+
+# This forces the fs tree out past level 0, adding at least one tree
+# block which must be properly accounted for when we make our next
+# snapshots.
+mkdir "$SCRATCH_MNT/data"
+for i in `seq 0 640`; do
+ $XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 2>&1
+done;
+
+# Snapshot twice.
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2"
+
+_scratch_unmount
+
+# generate a qgroup report and look for inconsistent groups
+$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \
+ grep -q -E "Counts for qgroup.*are different"
+if [ $? -ne 0 ]; then
+ status=0
+fi
+
+exit
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 9403daa..f7e8cff 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -122,3 +122,4 @@
119 auto quick snapshot metadata qgroup
120 auto quick snapshot metadata
121 auto quick snapshot qgroup
+122 auto quick snapshot qgroup
--
2.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
2016-04-19 22:25 [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation Mark Fasheh
@ 2016-04-20 0:48 ` Satoru Takeuchi
0 siblings, 0 replies; 7+ messages in thread
From: Satoru Takeuchi @ 2016-04-20 0:48 UTC (permalink / raw)
To: Mark Fasheh, fstests, linux-btrfs; +Cc: Qu Wenruo
On 2016/04/20 7:25, Mark Fasheh wrote:
> This has been broken since Linux v4.1. We may have worked out a solution on
> the btrfs list but in the meantime sending a test to expose the issue seems
> like a good idea.
>
> Signed-off-by: Mark Fasheh <mfasheh@suse.de>
> ---
> tests/btrfs/122 | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/group | 1 +
You forgot to add tests/btrfs/122.out.
> 2 files changed, 89 insertions(+)
> create mode 100755 tests/btrfs/122
>
> diff --git a/tests/btrfs/122 b/tests/btrfs/122
> new file mode 100755
> index 0000000..b7e9e4b
> --- /dev/null
> +++ b/tests/btrfs/122
> @@ -0,0 +1,88 @@
> +#! /bin/bash
> +# FS QA Test No. btrfs/122
> +#
> +# Test that qgroup counts are valid after snapshot creation. This has
> +# been broken in btrfs since Linux v4.1
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +# Force a small leaf size to make it easier to blow out our root
> +# subvolume tree
> +_scratch_mkfs "--nodesize 16384"
nodesize 16384 is the default value. Do you
intend other value, for example 4096?
> +_scratch_mount
> +_run_btrfs_util_prog quota enable $SCRATCH_MNT
> +
> +mkdir "$SCRATCH_MNT/snaps"
> +
> +# First make some simple snapshots - the bug was initially reproduced like this
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty1"
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty2"
> +
> +# This forces the fs tree out past level 0, adding at least one tree
> +# block which must be properly accounted for when we make our next
> +# snapshots.
> +mkdir "$SCRATCH_MNT/data"
> +for i in `seq 0 640`; do
> + $XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 2>&1
> +done;
";" after "done" is not necessary.
Thanks,
Satoru
> +
> +# Snapshot twice.
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1"
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2"
> +
> +_scratch_unmount
> +
> +# generate a qgroup report and look for inconsistent groups
> +$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \
> + grep -q -E "Counts for qgroup.*are different"
> +if [ $? -ne 0 ]; then
> + status=0
> +fi
> +
> +exit
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 9403daa..f7e8cff 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -122,3 +122,4 @@
> 119 auto quick snapshot metadata qgroup
> 120 auto quick snapshot metadata
> 121 auto quick snapshot qgroup
> +122 auto quick snapshot qgroup
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
@ 2016-04-20 0:48 ` Satoru Takeuchi
0 siblings, 0 replies; 7+ messages in thread
From: Satoru Takeuchi @ 2016-04-20 0:48 UTC (permalink / raw)
To: Mark Fasheh, fstests, linux-btrfs; +Cc: Qu Wenruo
On 2016/04/20 7:25, Mark Fasheh wrote:
> This has been broken since Linux v4.1. We may have worked out a solution on
> the btrfs list but in the meantime sending a test to expose the issue seems
> like a good idea.
>
> Signed-off-by: Mark Fasheh <mfasheh@suse.de>
> ---
> tests/btrfs/122 | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/group | 1 +
You forgot to add tests/btrfs/122.out.
> 2 files changed, 89 insertions(+)
> create mode 100755 tests/btrfs/122
>
> diff --git a/tests/btrfs/122 b/tests/btrfs/122
> new file mode 100755
> index 0000000..b7e9e4b
> --- /dev/null
> +++ b/tests/btrfs/122
> @@ -0,0 +1,88 @@
> +#! /bin/bash
> +# FS QA Test No. btrfs/122
> +#
> +# Test that qgroup counts are valid after snapshot creation. This has
> +# been broken in btrfs since Linux v4.1
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +# Force a small leaf size to make it easier to blow out our root
> +# subvolume tree
> +_scratch_mkfs "--nodesize 16384"
nodesize 16384 is the default value. Do you
intend other value, for example 4096?
> +_scratch_mount
> +_run_btrfs_util_prog quota enable $SCRATCH_MNT
> +
> +mkdir "$SCRATCH_MNT/snaps"
> +
> +# First make some simple snapshots - the bug was initially reproduced like this
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty1"
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty2"
> +
> +# This forces the fs tree out past level 0, adding at least one tree
> +# block which must be properly accounted for when we make our next
> +# snapshots.
> +mkdir "$SCRATCH_MNT/data"
> +for i in `seq 0 640`; do
> + $XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 2>&1
> +done;
";" after "done" is not necessary.
Thanks,
Satoru
> +
> +# Snapshot twice.
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1"
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2"
> +
> +_scratch_unmount
> +
> +# generate a qgroup report and look for inconsistent groups
> +$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \
> + grep -q -E "Counts for qgroup.*are different"
> +if [ $? -ne 0 ]; then
> + status=0
> +fi
> +
> +exit
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 9403daa..f7e8cff 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -122,3 +122,4 @@
> 119 auto quick snapshot metadata qgroup
> 120 auto quick snapshot metadata
> 121 auto quick snapshot qgroup
> +122 auto quick snapshot qgroup
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
2016-04-20 0:48 ` Satoru Takeuchi
(?)
@ 2016-04-21 23:53 ` Mark Fasheh
2016-04-22 0:26 ` Qu Wenruo
-1 siblings, 1 reply; 7+ messages in thread
From: Mark Fasheh @ 2016-04-21 23:53 UTC (permalink / raw)
To: Satoru Takeuchi; +Cc: fstests, linux-btrfs, Qu Wenruo
Thank you for the review, comments are below.
On Wed, Apr 20, 2016 at 09:48:54AM +0900, Satoru Takeuchi wrote:
> On 2016/04/20 7:25, Mark Fasheh wrote:
> >+# Force a small leaf size to make it easier to blow out our root
> >+# subvolume tree
> >+_scratch_mkfs "--nodesize 16384"
>
> nodesize 16384 is the default value. Do you
> intend other value, for example 4096?
"future proofing" I suppose - if we up the default, the for loop below may
not create a level 1 tree.
If we force it smaller than 16K I believe that may mean we can't run this
test on some kernels with page size larger than the typical 4k.
--Mark
--
Mark Fasheh
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
2016-04-21 23:53 ` Mark Fasheh
@ 2016-04-22 0:26 ` Qu Wenruo
0 siblings, 0 replies; 7+ messages in thread
From: Qu Wenruo @ 2016-04-22 0:26 UTC (permalink / raw)
To: Mark Fasheh, Satoru Takeuchi; +Cc: fstests, linux-btrfs
Mark Fasheh wrote on 2016/04/21 16:53 -0700:
> Thank you for the review, comments are below.
>
> On Wed, Apr 20, 2016 at 09:48:54AM +0900, Satoru Takeuchi wrote:
>> On 2016/04/20 7:25, Mark Fasheh wrote:
>>> +# Force a small leaf size to make it easier to blow out our root
>>> +# subvolume tree
>>> +_scratch_mkfs "--nodesize 16384"
>>
>> nodesize 16384 is the default value. Do you
>> intend other value, for example 4096?
>
> "future proofing" I suppose - if we up the default, the for loop below may
> not create a level 1 tree.
>
> If we force it smaller than 16K I believe that may mean we can't run this
> test on some kernels with page size larger than the typical 4k.
> --Mark
>
>
> --
> Mark Fasheh
>
>
Sorry for the late reply.
Unfortunately, for system with 64K page size, it will fail(mount and
mkfs) if we use 16K nodesize.
IIRC, like some other btrfs qgroup test case, we use 64K nodesize as the
safest nodesize.
And for level 1 tree create, the idea is to use inline file extents to
rapidly create level 1 tree.
16 4K files should create a level 1 tree.
Although in this case, max_inline=4096 would be added to mount option
though.
Thanks,
Qu
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
@ 2016-04-22 0:26 ` Qu Wenruo
0 siblings, 0 replies; 7+ messages in thread
From: Qu Wenruo @ 2016-04-22 0:26 UTC (permalink / raw)
To: Mark Fasheh, Satoru Takeuchi; +Cc: fstests, linux-btrfs
Mark Fasheh wrote on 2016/04/21 16:53 -0700:
> Thank you for the review, comments are below.
>
> On Wed, Apr 20, 2016 at 09:48:54AM +0900, Satoru Takeuchi wrote:
>> On 2016/04/20 7:25, Mark Fasheh wrote:
>>> +# Force a small leaf size to make it easier to blow out our root
>>> +# subvolume tree
>>> +_scratch_mkfs "--nodesize 16384"
>>
>> nodesize 16384 is the default value. Do you
>> intend other value, for example 4096?
>
> "future proofing" I suppose - if we up the default, the for loop below may
> not create a level 1 tree.
>
> If we force it smaller than 16K I believe that may mean we can't run this
> test on some kernels with page size larger than the typical 4k.
> --Mark
>
>
> --
> Mark Fasheh
>
>
Sorry for the late reply.
Unfortunately, for system with 64K page size, it will fail(mount and
mkfs) if we use 16K nodesize.
IIRC, like some other btrfs qgroup test case, we use 64K nodesize as the
safest nodesize.
And for level 1 tree create, the idea is to use inline file extents to
rapidly create level 1 tree.
16 4K files should create a level 1 tree.
Although in this case, max_inline=4096 would be added to mount option
though.
Thanks,
Qu
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
2016-04-22 0:26 ` Qu Wenruo
(?)
@ 2016-04-22 20:47 ` Mark Fasheh
-1 siblings, 0 replies; 7+ messages in thread
From: Mark Fasheh @ 2016-04-22 20:47 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Satoru Takeuchi, fstests, linux-btrfs
On Fri, Apr 22, 2016 at 08:26:33AM +0800, Qu Wenruo wrote:
>
>
> Mark Fasheh wrote on 2016/04/21 16:53 -0700:
> >Thank you for the review, comments are below.
> >
> >On Wed, Apr 20, 2016 at 09:48:54AM +0900, Satoru Takeuchi wrote:
> >>On 2016/04/20 7:25, Mark Fasheh wrote:
> >>>+# Force a small leaf size to make it easier to blow out our root
> >>>+# subvolume tree
> >>>+_scratch_mkfs "--nodesize 16384"
> >>
> >>nodesize 16384 is the default value. Do you
> >>intend other value, for example 4096?
> >
> >"future proofing" I suppose - if we up the default, the for loop below may
> >not create a level 1 tree.
> >
> >If we force it smaller than 16K I believe that may mean we can't run this
> >test on some kernels with page size larger than the typical 4k.
> > --Mark
> >
> >
> >--
> >Mark Fasheh
> >
> >
>
> Sorry for the late reply.
>
> Unfortunately, for system with 64K page size, it will fail(mount and
> mkfs) if we use 16K nodesize.
>
> IIRC, like some other btrfs qgroup test case, we use 64K nodesize as
> the safest nodesize.
>
> And for level 1 tree create, the idea is to use inline file extents
> to rapidly create level 1 tree.
>
> 16 4K files should create a level 1 tree.
> Although in this case, max_inline=4096 would be added to mount
> option though.
That all sounds good, thanks. The only thing about filling it completely
with inline extents though is that we should be exercising qgroups a little
harder. But maybe we can blow out the tree with inline extents and then add
some actual data extents after that.
--Mark
--
Mark Fasheh
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-04-22 20:47 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-19 22:25 [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation Mark Fasheh
2016-04-20 0:48 ` Satoru Takeuchi
2016-04-20 0:48 ` Satoru Takeuchi
2016-04-21 23:53 ` Mark Fasheh
2016-04-22 0:26 ` Qu Wenruo
2016-04-22 0:26 ` Qu Wenruo
2016-04-22 20:47 ` Mark Fasheh
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.