FSTests Archive on lore.kernel.org
 help / color / Atom feed
* [PATCHSET 0/1] fstests: consolidate posteof and cowblocks cleanup
@ 2021-03-31  1:08 Darrick J. Wong
  2021-03-31  1:09 ` [PATCH 1/1] xfs/23[12]: abstractify the XFS cow prealloc trimming interval Darrick J. Wong
  0 siblings, 1 reply; 3+ messages in thread
From: Darrick J. Wong @ 2021-03-31  1:08 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

Hi all,

Currently, we treat the garbage collection of post-EOF preallocations
and copy-on-write preallocations as totally separate tasks -- different
incore inode tags, different workqueues, etc.  This is wasteful of radix
tree tags and workqueue resources since we effectively have parallel
code paths to do the same thing.

Therefore, consolidate both functions under one radix tree bit and one
workqueue function that scans an inode for both things at the same time.
At the end of the series we make the scanning per-AG instead of per-fs
so that the scanning can run in parallel.

Note that this is a fixup for a single sysfs change that appeared in
5.12.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=eofblocks-consolidation

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=eofblocks-consolidation
---
 common/xfs    |   23 +++++++++++++++++++++++
 tests/xfs/231 |   13 +++++++------
 tests/xfs/232 |   13 +++++++------
 3 files changed, 37 insertions(+), 12 deletions(-)


^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH 1/1] xfs/23[12]: abstractify the XFS cow prealloc trimming interval
  2021-03-31  1:08 [PATCHSET 0/1] fstests: consolidate posteof and cowblocks cleanup Darrick J. Wong
@ 2021-03-31  1:09 ` Darrick J. Wong
  2021-03-31 17:43   ` Brian Foster
  0 siblings, 1 reply; 3+ messages in thread
From: Darrick J. Wong @ 2021-03-31  1:09 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

From: Darrick J. Wong <djwong@kernel.org>

Create a couple of helper functions to get and set the XFS copy on write
preallocation garbage collection interval.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs    |   23 +++++++++++++++++++++++
 tests/xfs/231 |   13 +++++++------
 tests/xfs/232 |   13 +++++++------
 3 files changed, 37 insertions(+), 12 deletions(-)


diff --git a/common/xfs b/common/xfs
index c430b3ac..ade76d5a 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1165,3 +1165,26 @@ _require_xfs_scratch_bigtime()
 		_notrun "bigtime feature not advertised on mount?"
 	_scratch_unmount
 }
+
+__xfs_cowgc_interval_knob1="/proc/sys/fs/xfs/speculative_cow_prealloc_lifetime"
+__xfs_cowgc_interval_knob2="/proc/sys/fs/xfs/speculative_prealloc_lifetime"
+
+_xfs_set_cowgc_interval() {
+	if [ -w $__xfs_cowgc_interval_knob1 ]; then
+		echo "$@" > $__xfs_cowgc_interval_knob1
+	elif [ -w $__xfs_cowgc_interval_knob2 ]; then
+		echo "$@" > $__xfs_cowgc_interval_knob2
+	else
+		_fail "Can't find cowgc interval procfs knob?"
+	fi
+}
+
+_xfs_get_cowgc_interval() {
+	if [ -w $__xfs_cowgc_interval_knob1 ]; then
+		cat $__xfs_cowgc_interval_knob1
+	elif [ -w $__xfs_cowgc_interval_knob2 ]; then
+		cat $__xfs_cowgc_interval_knob2
+	else
+		_fail "Can't find cowgc interval procfs knob?"
+	fi
+}
diff --git a/tests/xfs/231 b/tests/xfs/231
index 5930339d..119a71bb 100755
--- a/tests/xfs/231
+++ b/tests/xfs/231
@@ -22,9 +22,10 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _cleanup()
 {
-    cd /
-    echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
-    rm -rf $tmp.*
+	cd /
+	test -n "$old_cowgc_interval" && \
+		_xfs_set_cowgc_interval $old_cowgc_interval
+	rm -rf $tmp.*
 }
 
 # get standard environment, filters and checks
@@ -40,7 +41,7 @@ _require_xfs_io_command "cowextsize"
 _require_xfs_io_command "falloc"
 _require_xfs_io_command "fiemap"
 
-old_cow_lifetime=$(cat /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime)
+old_cowgc_interval=$(_xfs_get_cowgc_interval)
 
 rm -f $seqres.full
 
@@ -74,7 +75,7 @@ md5sum $testdir/file2 | _filter_scratch
 md5sum $testdir/file2.chk | _filter_scratch
 
 echo "CoW and leave leftovers"
-echo 2 > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
+_xfs_set_cowgc_interval 2
 seq 2 2 $((nr - 1)) | while read f; do
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2 >> $seqres.full
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2.chk >> $seqres.full
@@ -91,7 +92,7 @@ done
 $XFS_IO_PROG -f -c "falloc 0 $filesize" $testdir/junk >> $seqres.full
 
 echo "CoW and leave leftovers"
-echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
+_xfs_set_cowgc_interval $old_cowgc_interval
 seq 2 2 $((nr - 1)) | while read f; do
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2 >> $seqres.full
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2.chk >> $seqres.full
diff --git a/tests/xfs/232 b/tests/xfs/232
index e56eb3aa..909f921c 100755
--- a/tests/xfs/232
+++ b/tests/xfs/232
@@ -23,9 +23,10 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _cleanup()
 {
-    cd /
-    echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
-    rm -rf $tmp.*
+	cd /
+	test -n "$old_cowgc_interval" && \
+		_xfs_set_cowgc_interval $old_cowgc_interval
+	rm -rf $tmp.*
 }
 
 # get standard environment, filters and checks
@@ -41,7 +42,7 @@ _require_cp_reflink
 _require_xfs_io_command "falloc"
 _require_xfs_io_command "fiemap"
 
-old_cow_lifetime=$(cat /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime)
+old_cowgc_interval=$(_xfs_get_cowgc_interval)
 
 rm -f $seqres.full
 
@@ -75,7 +76,7 @@ md5sum $testdir/file2 | _filter_scratch
 md5sum $testdir/file2.chk | _filter_scratch
 
 echo "CoW and leave leftovers"
-echo 2 > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
+_xfs_set_cowgc_interval 2
 seq 2 2 $((nr - 1)) | while read f; do
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2 >> $seqres.full
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2.chk >> $seqres.full
@@ -93,7 +94,7 @@ done
 $XFS_IO_PROG -f -c "falloc 0 $filesize" $testdir/junk >> $seqres.full
 
 echo "CoW and leave leftovers"
-echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
+_xfs_set_cowgc_interval $old_cowgc_interval
 seq 2 2 $((nr - 1)) | while read f; do
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2 >> $seqres.full
 	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2.chk >> $seqres.full


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH 1/1] xfs/23[12]: abstractify the XFS cow prealloc trimming interval
  2021-03-31  1:09 ` [PATCH 1/1] xfs/23[12]: abstractify the XFS cow prealloc trimming interval Darrick J. Wong
@ 2021-03-31 17:43   ` Brian Foster
  0 siblings, 0 replies; 3+ messages in thread
From: Brian Foster @ 2021-03-31 17:43 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan

On Tue, Mar 30, 2021 at 06:09:05PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Create a couple of helper functions to get and set the XFS copy on write
> preallocation garbage collection interval.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---

Seems reasonable:

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  common/xfs    |   23 +++++++++++++++++++++++
>  tests/xfs/231 |   13 +++++++------
>  tests/xfs/232 |   13 +++++++------
>  3 files changed, 37 insertions(+), 12 deletions(-)
> 
> 
> diff --git a/common/xfs b/common/xfs
> index c430b3ac..ade76d5a 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -1165,3 +1165,26 @@ _require_xfs_scratch_bigtime()
>  		_notrun "bigtime feature not advertised on mount?"
>  	_scratch_unmount
>  }
> +
> +__xfs_cowgc_interval_knob1="/proc/sys/fs/xfs/speculative_cow_prealloc_lifetime"
> +__xfs_cowgc_interval_knob2="/proc/sys/fs/xfs/speculative_prealloc_lifetime"
> +
> +_xfs_set_cowgc_interval() {
> +	if [ -w $__xfs_cowgc_interval_knob1 ]; then
> +		echo "$@" > $__xfs_cowgc_interval_knob1
> +	elif [ -w $__xfs_cowgc_interval_knob2 ]; then
> +		echo "$@" > $__xfs_cowgc_interval_knob2
> +	else
> +		_fail "Can't find cowgc interval procfs knob?"
> +	fi
> +}
> +
> +_xfs_get_cowgc_interval() {
> +	if [ -w $__xfs_cowgc_interval_knob1 ]; then
> +		cat $__xfs_cowgc_interval_knob1
> +	elif [ -w $__xfs_cowgc_interval_knob2 ]; then
> +		cat $__xfs_cowgc_interval_knob2
> +	else
> +		_fail "Can't find cowgc interval procfs knob?"
> +	fi
> +}
> diff --git a/tests/xfs/231 b/tests/xfs/231
> index 5930339d..119a71bb 100755
> --- a/tests/xfs/231
> +++ b/tests/xfs/231
> @@ -22,9 +22,10 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
>  
>  _cleanup()
>  {
> -    cd /
> -    echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
> -    rm -rf $tmp.*
> +	cd /
> +	test -n "$old_cowgc_interval" && \
> +		_xfs_set_cowgc_interval $old_cowgc_interval
> +	rm -rf $tmp.*
>  }
>  
>  # get standard environment, filters and checks
> @@ -40,7 +41,7 @@ _require_xfs_io_command "cowextsize"
>  _require_xfs_io_command "falloc"
>  _require_xfs_io_command "fiemap"
>  
> -old_cow_lifetime=$(cat /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime)
> +old_cowgc_interval=$(_xfs_get_cowgc_interval)
>  
>  rm -f $seqres.full
>  
> @@ -74,7 +75,7 @@ md5sum $testdir/file2 | _filter_scratch
>  md5sum $testdir/file2.chk | _filter_scratch
>  
>  echo "CoW and leave leftovers"
> -echo 2 > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
> +_xfs_set_cowgc_interval 2
>  seq 2 2 $((nr - 1)) | while read f; do
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2 >> $seqres.full
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2.chk >> $seqres.full
> @@ -91,7 +92,7 @@ done
>  $XFS_IO_PROG -f -c "falloc 0 $filesize" $testdir/junk >> $seqres.full
>  
>  echo "CoW and leave leftovers"
> -echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
> +_xfs_set_cowgc_interval $old_cowgc_interval
>  seq 2 2 $((nr - 1)) | while read f; do
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2 >> $seqres.full
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2.chk >> $seqres.full
> diff --git a/tests/xfs/232 b/tests/xfs/232
> index e56eb3aa..909f921c 100755
> --- a/tests/xfs/232
> +++ b/tests/xfs/232
> @@ -23,9 +23,10 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
>  
>  _cleanup()
>  {
> -    cd /
> -    echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
> -    rm -rf $tmp.*
> +	cd /
> +	test -n "$old_cowgc_interval" && \
> +		_xfs_set_cowgc_interval $old_cowgc_interval
> +	rm -rf $tmp.*
>  }
>  
>  # get standard environment, filters and checks
> @@ -41,7 +42,7 @@ _require_cp_reflink
>  _require_xfs_io_command "falloc"
>  _require_xfs_io_command "fiemap"
>  
> -old_cow_lifetime=$(cat /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime)
> +old_cowgc_interval=$(_xfs_get_cowgc_interval)
>  
>  rm -f $seqres.full
>  
> @@ -75,7 +76,7 @@ md5sum $testdir/file2 | _filter_scratch
>  md5sum $testdir/file2.chk | _filter_scratch
>  
>  echo "CoW and leave leftovers"
> -echo 2 > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
> +_xfs_set_cowgc_interval 2
>  seq 2 2 $((nr - 1)) | while read f; do
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2 >> $seqres.full
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f - 1)) 1" $testdir/file2.chk >> $seqres.full
> @@ -93,7 +94,7 @@ done
>  $XFS_IO_PROG -f -c "falloc 0 $filesize" $testdir/junk >> $seqres.full
>  
>  echo "CoW and leave leftovers"
> -echo $old_cow_lifetime > /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime
> +_xfs_set_cowgc_interval $old_cowgc_interval
>  seq 2 2 $((nr - 1)) | while read f; do
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2 >> $seqres.full
>  	$XFS_IO_PROG -f -c "pwrite -S 0x63 $((blksz * f)) 1" $testdir/file2.chk >> $seqres.full
> 


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-31  1:08 [PATCHSET 0/1] fstests: consolidate posteof and cowblocks cleanup Darrick J. Wong
2021-03-31  1:09 ` [PATCH 1/1] xfs/23[12]: abstractify the XFS cow prealloc trimming interval Darrick J. Wong
2021-03-31 17:43   ` Brian Foster

FSTests Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/fstests/0 fstests/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 fstests fstests/ https://lore.kernel.org/fstests \
		fstests@vger.kernel.org
	public-inbox-index fstests

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.fstests


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git