Linux-BTRFS Archive on lore.kernel.org
 help / Atom feed
* [PATCH] btrfs: Test if btrfs hits EDQUOT without trying to reclaim some space
@ 2019-01-28  2:04 Qu Wenruo
  2019-01-29 11:36 ` Filipe Manana
  0 siblings, 1 reply; 2+ messages in thread
From: Qu Wenruo @ 2019-01-28  2:04 UTC (permalink / raw)
  To: linux-btrfs; +Cc: fstests

Commit a514d63882c3 ("btrfs: qgroup: Commit transaction in advance to reduce
early EDQUOT") is no longer forcing transaction commit to reclaim space,
and only commits transaction asynchronously in advance to address it.

However the criteria used in async transaction commit is not
comprehensive, thus it doesn't reclaim space automatically.

This test case will check the behavior by:
1) Falloc a large padding file
   This file will take 90% of the qgroup limit

2) Sync the fs
   To reflect the qgroup changes

3) Delete the file
   Qgroup won't reclaim the space until transaction committed.

4) Try to write a file
   If kernel not fixed, qgroup will not automatically commit transaction
   to reclaim the freed space and hit EDQUOT.

This bug is going to be fxied by a patch for kernel titled
"btrfs: qgroup: Make qgroup async transaction commit more aggressive".

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/btrfs/180     | 62 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/180.out |  3 +++
 tests/btrfs/group   |  1 +
 3 files changed, 66 insertions(+)
 create mode 100755 tests/btrfs/180
 create mode 100644 tests/btrfs/180.out

diff --git a/tests/btrfs/180 b/tests/btrfs/180
new file mode 100755
index 00000000..df876310
--- /dev/null
+++ b/tests/btrfs/180
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 180
+#
+# Test if btrfs hits EDQUOT without reclaim already freed extents when quota
+# is enabled.
+#
+# This bug is going to be fxied by a patch for kernel titled
+# "btrfs: qgroup: Make qgroup async transaction commit more aggressive"
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_xfs_io_command falloc
+
+_scratch_mkfs > /dev/null
+_scratch_mount
+
+$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" > /dev/null
+$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" > /dev/null
+$BTRFS_UTIL_PROG qgroup limit -e 1G "$SCRATCH_MNT"
+
+$XFS_IO_PROG -f -c "falloc 0 900M" "$SCRATCH_MNT/padding" | _filter_xfs_io
+
+# Commit transaction to reflect the quota usage
+sync
+
+rm "$SCRATCH_MNT/padding"
+
+# Without the kernel fix, this will trigger EDQUOT.
+_pwrite_byte 0xcd 0 512M "$SCRATCH_MNT/real_file" | _filter_xfs_io
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/180.out b/tests/btrfs/180.out
new file mode 100644
index 00000000..17187477
--- /dev/null
+++ b/tests/btrfs/180.out
@@ -0,0 +1,3 @@
+QA output created by 180
+wrote 536870912/536870912 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 46dd3c95..03eb62f9 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -182,3 +182,4 @@
 177 auto quick swap balance
 178 auto quick send
 179 auto qgroup dangerous
+180 auto quick qgroup limit
-- 
2.18.0


^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [PATCH] btrfs: Test if btrfs hits EDQUOT without trying to reclaim some space
  2019-01-28  2:04 [PATCH] btrfs: Test if btrfs hits EDQUOT without trying to reclaim some space Qu Wenruo
@ 2019-01-29 11:36 ` Filipe Manana
  0 siblings, 0 replies; 2+ messages in thread
From: Filipe Manana @ 2019-01-29 11:36 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs, fstests

On Mon, Jan 28, 2019 at 2:05 AM Qu Wenruo <wqu@suse.com> wrote:
>
> Commit a514d63882c3 ("btrfs: qgroup: Commit transaction in advance to reduce
> early EDQUOT") is no longer forcing transaction commit to reclaim space,
> and only commits transaction asynchronously in advance to address it.
>
> However the criteria used in async transaction commit is not
> comprehensive, thus it doesn't reclaim space automatically.
>
> This test case will check the behavior by:
> 1) Falloc a large padding file
>    This file will take 90% of the qgroup limit
>
> 2) Sync the fs
>    To reflect the qgroup changes
>
> 3) Delete the file
>    Qgroup won't reclaim the space until transaction committed.
>
> 4) Try to write a file
>    If kernel not fixed, qgroup will not automatically commit transaction
>    to reclaim the freed space and hit EDQUOT.
>
> This bug is going to be fxied by a patch for kernel titled

fxied -> fixed

titled -> titled:

> "btrfs: qgroup: Make qgroup async transaction commit more aggressive".
>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  tests/btrfs/180     | 62 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/180.out |  3 +++
>  tests/btrfs/group   |  1 +
>  3 files changed, 66 insertions(+)
>  create mode 100755 tests/btrfs/180
>  create mode 100644 tests/btrfs/180.out
>
> diff --git a/tests/btrfs/180 b/tests/btrfs/180
> new file mode 100755
> index 00000000..df876310
> --- /dev/null
> +++ b/tests/btrfs/180
> @@ -0,0 +1,62 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 180
> +#
> +# Test if btrfs hits EDQUOT without reclaim already freed extents when quota

reclaim -> reclaiming

> +# is enabled.
> +#
> +# This bug is going to be fxied by a patch for kernel titled
> +# "btrfs: qgroup: Make qgroup async transaction commit more aggressive"
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1       # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_xfs_io_command falloc
> +
> +_scratch_mkfs > /dev/null
> +_scratch_mount
> +
> +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" > /dev/null
> +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" > /dev/null
> +$BTRFS_UTIL_PROG qgroup limit -e 1G "$SCRATCH_MNT"
> +
> +$XFS_IO_PROG -f -c "falloc 0 900M" "$SCRATCH_MNT/padding" | _filter_xfs_io

No need to use the _filter_xfs_io for falloc (it doesn't output
throughput values such as the write, reflink and dedupe commands).

> +
> +# Commit transaction to reflect the quota usage
> +sync
> +
> +rm "$SCRATCH_MNT/padding"

I would pass -f to rm. It's not uncommon to have setups with "alias rm='rm -i'".

Other than that, it looks good.

Reviewed-by: Filipe Manana <fdmanana@suse.com>

Thanks.

> +
> +# Without the kernel fix, this will trigger EDQUOT.
> +_pwrite_byte 0xcd 0 512M "$SCRATCH_MNT/real_file" | _filter_xfs_io
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/180.out b/tests/btrfs/180.out
> new file mode 100644
> index 00000000..17187477
> --- /dev/null
> +++ b/tests/btrfs/180.out
> @@ -0,0 +1,3 @@
> +QA output created by 180
> +wrote 536870912/536870912 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 46dd3c95..03eb62f9 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -182,3 +182,4 @@
>  177 auto quick swap balance
>  178 auto quick send
>  179 auto qgroup dangerous
> +180 auto quick qgroup limit
> --
> 2.18.0
>


-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, back to index

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-28  2:04 [PATCH] btrfs: Test if btrfs hits EDQUOT without trying to reclaim some space Qu Wenruo
2019-01-29 11:36 ` Filipe Manana

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox