* [PATCH 0/2] Section handling patch and huge sparse file generic/618
@ 2020-12-04 10:43 Ritesh Harjani
2020-12-04 10:43 ` [PATCH 1/2] check: source common/rc again if TEST_DEV was recreated Ritesh Harjani
2020-12-04 10:43 ` [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device Ritesh Harjani
0 siblings, 2 replies; 6+ messages in thread
From: Ritesh Harjani @ 2020-12-04 10:43 UTC (permalink / raw)
To: fstests; +Cc: guan, linux-ext4, anju, Ritesh Harjani
Patch-1 was suggested from Eryu. This does fixes the multiple section handling
issue which was discussed here [1]. Hence sending it for review and
inclusion in fstests.
Patch-2 - addressed the review comments from [2] w.r.t. ext4 MKFS_OPTIONS
for bs < 4k.
[1]: https://patchwork.kernel.org/project/fstests/cover/cover.1604000570.git.riteshh@linux.ibm.com/
[2]: https://patchwork.kernel.org/patch/11864921
Eryu Guan (1):
check: source common/rc again if TEST_DEV was recreated
Ritesh Harjani (1):
generic: Add test to check for mounting a huge sparse dm device
check | 4 +++
common/rc | 10 +++++++
tests/generic/618 | 70 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/618.out | 3 ++
tests/generic/group | 1 +
5 files changed, 88 insertions(+)
create mode 100755 tests/generic/618
create mode 100644 tests/generic/618.out
--
2.26.2
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/2] check: source common/rc again if TEST_DEV was recreated
2020-12-04 10:43 [PATCH 0/2] Section handling patch and huge sparse file generic/618 Ritesh Harjani
@ 2020-12-04 10:43 ` Ritesh Harjani
2020-12-04 10:43 ` [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device Ritesh Harjani
1 sibling, 0 replies; 6+ messages in thread
From: Ritesh Harjani @ 2020-12-04 10:43 UTC (permalink / raw)
To: fstests; +Cc: guan, linux-ext4, anju, Eryu Guan, Ritesh Harjani
From: Eryu Guan <eguan@redhat.com>
If TEST_DEV is recreated by check, FSTYP derived from TEST_DEV
previously could be changed too and might not reflect the reality.
So source common/rc again with correct FSTYP to get fs-specific
configs, e.g. common/xfs.
For example, using this config-section config file, and run section
ext4 first then xfs, you can see:
our local _scratch_mkfs routine ...
./common/rc: line 825: _scratch_mkfs_xfs: command not found
check: failed to mkfs $SCRATCH_DEV using specified options
local.config:
[default]
RECREATE_TEST_DEV=true
TEST_DEV=/dev/sda5
SCRATCH_DEV=/dev/sda6
TEST_DIR=/mnt/test
SCRATCH_MNT=/mnt/scratch
[ext4]
MKFS_OPTIONS="-b 4096"
FSTYP=ext4
[xfs]
FSTYP=xfs
MKFS_OPTIONS="-f -b size=4k"
Tested-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
---
check | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/check b/check
index 83f6fc8bdf3e..c6ad1d6c0733 100755
--- a/check
+++ b/check
@@ -630,6 +630,10 @@ function run_section()
status=1
exit
fi
+ # TEST_DEV has been recreated, previous FSTYP derived from
+ # TEST_DEV could be changed, source common/rc again with
+ # correct FSTYP to get FSTYP specific configs, e.g. common/xfs
+ . common/rc
_prepare_test_list
elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then
_test_unmount 2> /dev/null
--
2.26.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device
2020-12-04 10:43 [PATCH 0/2] Section handling patch and huge sparse file generic/618 Ritesh Harjani
2020-12-04 10:43 ` [PATCH 1/2] check: source common/rc again if TEST_DEV was recreated Ritesh Harjani
@ 2020-12-04 10:43 ` Ritesh Harjani
2020-12-06 13:50 ` Eryu Guan
1 sibling, 1 reply; 6+ messages in thread
From: Ritesh Harjani @ 2020-12-04 10:43 UTC (permalink / raw)
To: fstests; +Cc: guan, linux-ext4, anju, Ritesh Harjani
Add this test to check for regression which was reported when ext4 bmap
aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
from fs/inode.c which was failing since iomap_bmap() implementation earlier
returned 0 for block addr > INT_MAX.
This regression was fixed with following kernel commit [1]
commit b75dfde1212991b24b220c3995101c60a7b8ae74
("fibmap: Warn and return an error in case of block > INT_MAX")
[1]: https://patchwork.ozlabs.org/patch/1279914
w/o the kernel fix we get below errors and mount fails
[ 1461.988701] run fstests generic/613 at 2020-10-27 19:57:34
[ 1530.406645] ------------[ cut here ]------------
[ 1530.407332] would truncate bmap result
[ 1530.408956] WARNING: CPU: 0 PID: 6401 at fs/iomap/fiemap.c:116
iomap_bmap_actor+0x43/0x50
[ 1530.410607] Modules linked in:
[ 1530.411024] CPU: 0 PID: 6401 Comm: mount Tainted: G W
<...>
1530.511978] jbd2_journal_init_inode: Cannot locate journal superblock
[ 1530.513310] EXT4-fs (dm-1): Could not load journal inode
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
---
common/rc | 10 +++++++
tests/generic/618 | 70 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/618.out | 3 ++
tests/generic/group | 1 +
4 files changed, 84 insertions(+)
create mode 100755 tests/generic/618
create mode 100644 tests/generic/618.out
diff --git a/common/rc b/common/rc
index b5a504e0dcb4..128d75226958 100644
--- a/common/rc
+++ b/common/rc
@@ -1608,6 +1608,16 @@ _require_scratch_size()
[ $devsize -lt $1 ] && _notrun "scratch dev too small"
}
+# require a scratch dev of a minimum size (in kb) and should not be checked
+# post test
+_require_scratch_size_nocheck()
+{
+ [ $# -eq 1 ] || _fail "_require_scratch_size: expected size param"
+
+ _require_scratch_nocheck
+ local devsize=`_get_device_size $SCRATCH_DEV`
+ [ $devsize -lt $1 ] && _notrun "scratch dev too small"
+}
# this test needs a test partition - check we're ok & mount it
#
diff --git a/tests/generic/618 b/tests/generic/618
new file mode 100755
index 000000000000..45c14da80c06
--- /dev/null
+++ b/tests/generic/618
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Christian Kujau. All Rights Reserved.
+# Copyright (c) 2020 Ritesh Harjani. All Rights Reserved.
+#
+# FS QA Test generic/618
+#
+# Since the test is not specific to ext4, hence adding it to generic.
+# Add this test to check for regression which was reported when ext4 bmap
+# aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
+# from fs/inode.c which was failing since iomap_bmap() implementation earlier
+# returned 0 for block addr > INT_MAX.
+# This regression was fixed with following kernel commit [1]
+# commit b75dfde1212991b24b220c3995101c60a7b8ae74
+# ("fibmap: Warn and return an error in case of block > INT_MAX")
+# [1]: https://patchwork.ozlabs.org/patch/1279914
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ _dmhugedisk_cleanup
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmhugedisk
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# Modify as appropriate.
+_supported_fs generic
+_require_dmhugedisk
+_require_scratch_size_nocheck $((4 * 1024 * 1024)) #kB
+
+# For 1k bs with ext4, mkfs was failing due to size limitation and also it
+# becomes too slow when doing an mkfs on a huge sparse ext4 FS with 1k bs.
+# Hence on ext4 run only for 4K bs.
+if [ "$FSTYP" == "ext4" ]; then
+ _scratch_mkfs > /dev/null 2>&1
+ blksz=$(sudo debugfs -R stats $SCRATCH_DEV 2> /dev/null |grep "Block size" |cut -d ':' -f 2)
+ test $blksz -lt 4096 && _notrun "This test requires ext4 with minimum 4k bs"
+fi
+
+# 17TB dm huge-test-zer0 device
+# (in terms of 512 sectors)
+sectors=$((2*1024*1024*1024*17))
+chunk_size=128
+
+_dmhugedisk_init $sectors $chunk_size
+_mkfs_dev $DMHUGEDISK_DEV
+_mount $DMHUGEDISK_DEV $SCRATCH_MNT || _fail "mount failed for $DMHUGEDISK_DEV $SCRATCH_MNT"
+testfile=$SCRATCH_MNT/testfile-$seq
+
+$XFS_IO_PROG -fc "pwrite -S 0xaa 0 1m" -c "fsync" $testfile | _filter_xfs_io
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/618.out b/tests/generic/618.out
new file mode 100644
index 000000000000..b920fe4d907a
--- /dev/null
+++ b/tests/generic/618.out
@@ -0,0 +1,3 @@
+QA output created by 618
+wrote 1048576/1048576 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/generic/group b/tests/generic/group
index 94e860b8c380..39e3ffb224a9 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -620,3 +620,4 @@
615 auto rw
616 auto rw io_uring stress
617 auto rw io_uring stress
+618 auto mount quick
--
2.26.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device
2020-12-04 10:43 ` [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device Ritesh Harjani
@ 2020-12-06 13:50 ` Eryu Guan
2020-12-06 14:14 ` Eryu Guan
0 siblings, 1 reply; 6+ messages in thread
From: Eryu Guan @ 2020-12-06 13:50 UTC (permalink / raw)
To: Ritesh Harjani; +Cc: fstests, linux-ext4, anju
On Fri, Dec 04, 2020 at 04:13:54PM +0530, Ritesh Harjani wrote:
> Add this test to check for regression which was reported when ext4 bmap
> aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
> from fs/inode.c which was failing since iomap_bmap() implementation earlier
> returned 0 for block addr > INT_MAX.
> This regression was fixed with following kernel commit [1]
> commit b75dfde1212991b24b220c3995101c60a7b8ae74
> ("fibmap: Warn and return an error in case of block > INT_MAX")
> [1]: https://patchwork.ozlabs.org/patch/1279914
>
> w/o the kernel fix we get below errors and mount fails
>
> [ 1461.988701] run fstests generic/613 at 2020-10-27 19:57:34
> [ 1530.406645] ------------[ cut here ]------------
> [ 1530.407332] would truncate bmap result
> [ 1530.408956] WARNING: CPU: 0 PID: 6401 at fs/iomap/fiemap.c:116
> iomap_bmap_actor+0x43/0x50
> [ 1530.410607] Modules linked in:
> [ 1530.411024] CPU: 0 PID: 6401 Comm: mount Tainted: G W
> <...>
> 1530.511978] jbd2_journal_init_inode: Cannot locate journal superblock
> [ 1530.513310] EXT4-fs (dm-1): Could not load journal inode
>
> Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
> ---
> common/rc | 10 +++++++
> tests/generic/618 | 70 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/618.out | 3 ++
> tests/generic/group | 1 +
> 4 files changed, 84 insertions(+)
> create mode 100755 tests/generic/618
> create mode 100644 tests/generic/618.out
>
> diff --git a/common/rc b/common/rc
> index b5a504e0dcb4..128d75226958 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1608,6 +1608,16 @@ _require_scratch_size()
> [ $devsize -lt $1 ] && _notrun "scratch dev too small"
> }
>
> +# require a scratch dev of a minimum size (in kb) and should not be checked
> +# post test
> +_require_scratch_size_nocheck()
> +{
> + [ $# -eq 1 ] || _fail "_require_scratch_size: expected size param"
> +
> + _require_scratch_nocheck
> + local devsize=`_get_device_size $SCRATCH_DEV`
> + [ $devsize -lt $1 ] && _notrun "scratch dev too small"
> +}
>
> # this test needs a test partition - check we're ok & mount it
> #
> diff --git a/tests/generic/618 b/tests/generic/618
> new file mode 100755
> index 000000000000..45c14da80c06
> --- /dev/null
> +++ b/tests/generic/618
> @@ -0,0 +1,70 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Christian Kujau. All Rights Reserved.
> +# Copyright (c) 2020 Ritesh Harjani. All Rights Reserved.
> +#
> +# FS QA Test generic/618
> +#
> +# Since the test is not specific to ext4, hence adding it to generic.
> +# Add this test to check for regression which was reported when ext4 bmap
> +# aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
> +# from fs/inode.c which was failing since iomap_bmap() implementation earlier
> +# returned 0 for block addr > INT_MAX.
> +# This regression was fixed with following kernel commit [1]
> +# commit b75dfde1212991b24b220c3995101c60a7b8ae74
> +# ("fibmap: Warn and return an error in case of block > INT_MAX")
> +# [1]: https://patchwork.ozlabs.org/patch/1279914
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + _dmhugedisk_cleanup
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmhugedisk
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_require_dmhugedisk
> +_require_scratch_size_nocheck $((4 * 1024 * 1024)) #kB
> +
> +# For 1k bs with ext4, mkfs was failing due to size limitation and also it
> +# becomes too slow when doing an mkfs on a huge sparse ext4 FS with 1k bs.
> +# Hence on ext4 run only for 4K bs.
> +if [ "$FSTYP" == "ext4" ]; then
> + _scratch_mkfs > /dev/null 2>&1
> + blksz=$(sudo debugfs -R stats $SCRATCH_DEV 2> /dev/null |grep "Block size" |cut -d ':' -f 2)
> + test $blksz -lt 4096 && _notrun "This test requires ext4 with minimum 4k bs"
> +fi
As this is a generic test, the same check should be done with ext2 and
ext3. And actually this test requires > 16T fs support. So I'd suggest
add a new helper, maybe called _require_16T_support, to check if $FSTYP
supports filesystem size > 16T. And for now, we could only check for
extN.
Also, there's no need to run command with sudo in fstests, as all tests
are required to be run by root.
And there's a helper to get fs block size called _get_block_size, no
need to parse output of debugfs.
Thanks,
Eryu
P.S. I've applied patch 1 in the patchset, no need to resend in v2,
thanks!
> +
> +# 17TB dm huge-test-zer0 device
> +# (in terms of 512 sectors)
> +sectors=$((2*1024*1024*1024*17))
> +chunk_size=128
> +
> +_dmhugedisk_init $sectors $chunk_size
> +_mkfs_dev $DMHUGEDISK_DEV
> +_mount $DMHUGEDISK_DEV $SCRATCH_MNT || _fail "mount failed for $DMHUGEDISK_DEV $SCRATCH_MNT"
> +testfile=$SCRATCH_MNT/testfile-$seq
> +
> +$XFS_IO_PROG -fc "pwrite -S 0xaa 0 1m" -c "fsync" $testfile | _filter_xfs_io
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/618.out b/tests/generic/618.out
> new file mode 100644
> index 000000000000..b920fe4d907a
> --- /dev/null
> +++ b/tests/generic/618.out
> @@ -0,0 +1,3 @@
> +QA output created by 618
> +wrote 1048576/1048576 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> diff --git a/tests/generic/group b/tests/generic/group
> index 94e860b8c380..39e3ffb224a9 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -620,3 +620,4 @@
> 615 auto rw
> 616 auto rw io_uring stress
> 617 auto rw io_uring stress
> +618 auto mount quick
> --
> 2.26.2
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device
2020-12-06 13:50 ` Eryu Guan
@ 2020-12-06 14:14 ` Eryu Guan
2020-12-07 8:37 ` Ritesh Harjani
0 siblings, 1 reply; 6+ messages in thread
From: Eryu Guan @ 2020-12-06 14:14 UTC (permalink / raw)
To: Ritesh Harjani; +Cc: fstests, linux-ext4, anju
On Sun, Dec 06, 2020 at 09:50:43PM +0800, Eryu Guan wrote:
> On Fri, Dec 04, 2020 at 04:13:54PM +0530, Ritesh Harjani wrote:
> > Add this test to check for regression which was reported when ext4 bmap
> > aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
> > from fs/inode.c which was failing since iomap_bmap() implementation earlier
> > returned 0 for block addr > INT_MAX.
> > This regression was fixed with following kernel commit [1]
> > commit b75dfde1212991b24b220c3995101c60a7b8ae74
> > ("fibmap: Warn and return an error in case of block > INT_MAX")
> > [1]: https://patchwork.ozlabs.org/patch/1279914
> >
> > w/o the kernel fix we get below errors and mount fails
> >
> > [ 1461.988701] run fstests generic/613 at 2020-10-27 19:57:34
> > [ 1530.406645] ------------[ cut here ]------------
> > [ 1530.407332] would truncate bmap result
> > [ 1530.408956] WARNING: CPU: 0 PID: 6401 at fs/iomap/fiemap.c:116
> > iomap_bmap_actor+0x43/0x50
> > [ 1530.410607] Modules linked in:
> > [ 1530.411024] CPU: 0 PID: 6401 Comm: mount Tainted: G W
> > <...>
> > 1530.511978] jbd2_journal_init_inode: Cannot locate journal superblock
> > [ 1530.513310] EXT4-fs (dm-1): Could not load journal inode
> >
> > Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
> > ---
> > common/rc | 10 +++++++
> > tests/generic/618 | 70 +++++++++++++++++++++++++++++++++++++++++++
> > tests/generic/618.out | 3 ++
> > tests/generic/group | 1 +
> > 4 files changed, 84 insertions(+)
> > create mode 100755 tests/generic/618
> > create mode 100644 tests/generic/618.out
> >
> > diff --git a/common/rc b/common/rc
> > index b5a504e0dcb4..128d75226958 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -1608,6 +1608,16 @@ _require_scratch_size()
> > [ $devsize -lt $1 ] && _notrun "scratch dev too small"
> > }
> >
> > +# require a scratch dev of a minimum size (in kb) and should not be checked
> > +# post test
> > +_require_scratch_size_nocheck()
> > +{
> > + [ $# -eq 1 ] || _fail "_require_scratch_size: expected size param"
> > +
> > + _require_scratch_nocheck
> > + local devsize=`_get_device_size $SCRATCH_DEV`
> > + [ $devsize -lt $1 ] && _notrun "scratch dev too small"
> > +}
> >
> > # this test needs a test partition - check we're ok & mount it
> > #
> > diff --git a/tests/generic/618 b/tests/generic/618
> > new file mode 100755
> > index 000000000000..45c14da80c06
> > --- /dev/null
> > +++ b/tests/generic/618
> > @@ -0,0 +1,70 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2020 Christian Kujau. All Rights Reserved.
> > +# Copyright (c) 2020 Ritesh Harjani. All Rights Reserved.
> > +#
> > +# FS QA Test generic/618
> > +#
> > +# Since the test is not specific to ext4, hence adding it to generic.
> > +# Add this test to check for regression which was reported when ext4 bmap
> > +# aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
> > +# from fs/inode.c which was failing since iomap_bmap() implementation earlier
> > +# returned 0 for block addr > INT_MAX.
> > +# This regression was fixed with following kernel commit [1]
> > +# commit b75dfde1212991b24b220c3995101c60a7b8ae74
> > +# ("fibmap: Warn and return an error in case of block > INT_MAX")
> > +# [1]: https://patchwork.ozlabs.org/patch/1279914
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + _dmhugedisk_cleanup
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +. ./common/dmhugedisk
> > +
> > +# remove previous $seqres.full before test
> > +rm -f $seqres.full
> > +
> > +# Modify as appropriate.
> > +_supported_fs generic
> > +_require_dmhugedisk
> > +_require_scratch_size_nocheck $((4 * 1024 * 1024)) #kB
> > +
> > +# For 1k bs with ext4, mkfs was failing due to size limitation and also it
> > +# becomes too slow when doing an mkfs on a huge sparse ext4 FS with 1k bs.
> > +# Hence on ext4 run only for 4K bs.
> > +if [ "$FSTYP" == "ext4" ]; then
> > + _scratch_mkfs > /dev/null 2>&1
> > + blksz=$(sudo debugfs -R stats $SCRATCH_DEV 2> /dev/null |grep "Block size" |cut -d ':' -f 2)
> > + test $blksz -lt 4096 && _notrun "This test requires ext4 with minimum 4k bs"
> > +fi
>
> As this is a generic test, the same check should be done with ext2 and
> ext3. And actually this test requires > 16T fs support. So I'd suggest
> add a new helper, maybe called _require_16T_support, to check if $FSTYP
> supports filesystem size > 16T. And for now, we could only check for
> extN.
>
> Also, there's no need to run command with sudo in fstests, as all tests
> are required to be run by root.
>
> And there's a helper to get fs block size called _get_block_size, no
> need to parse output of debugfs.
Maybe something like (not tested)
_require_scratch_16T_support()
{
case $FSTYP in
ext2|ext3)
_notrun "$FSTYP doesn't support >16T filesystem"
ext4)
_scratch_mkfs >> $seqres.full 2>&1
_scratch_mount
local blocksize=$(_get_block_size $SCRATCH_MNT)
if [ $blocksize -lt 4096 ]; then
_notrun "This test requires >16T fs support"
fi
;;
*)
;;
esac
}
Thanks,
Eryu
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device
2020-12-06 14:14 ` Eryu Guan
@ 2020-12-07 8:37 ` Ritesh Harjani
0 siblings, 0 replies; 6+ messages in thread
From: Ritesh Harjani @ 2020-12-07 8:37 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-ext4, anju
On 12/6/20 7:44 PM, Eryu Guan wrote:
> On Sun, Dec 06, 2020 at 09:50:43PM +0800, Eryu Guan wrote:
>> On Fri, Dec 04, 2020 at 04:13:54PM +0530, Ritesh Harjani wrote:
>>> Add this test to check for regression which was reported when ext4 bmap
>>> aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
>>> from fs/inode.c which was failing since iomap_bmap() implementation earlier
>>> returned 0 for block addr > INT_MAX.
>>> This regression was fixed with following kernel commit [1]
>>> commit b75dfde1212991b24b220c3995101c60a7b8ae74
>>> ("fibmap: Warn and return an error in case of block > INT_MAX")
>>> [1]: https://patchwork.ozlabs.org/patch/1279914
>>>
>>> w/o the kernel fix we get below errors and mount fails
>>>
>>> [ 1461.988701] run fstests generic/613 at 2020-10-27 19:57:34
>>> [ 1530.406645] ------------[ cut here ]------------
>>> [ 1530.407332] would truncate bmap result
>>> [ 1530.408956] WARNING: CPU: 0 PID: 6401 at fs/iomap/fiemap.c:116
>>> iomap_bmap_actor+0x43/0x50
>>> [ 1530.410607] Modules linked in:
>>> [ 1530.411024] CPU: 0 PID: 6401 Comm: mount Tainted: G W
>>> <...>
>>> 1530.511978] jbd2_journal_init_inode: Cannot locate journal superblock
>>> [ 1530.513310] EXT4-fs (dm-1): Could not load journal inode
>>>
>>> Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
>>> ---
>>> common/rc | 10 +++++++
>>> tests/generic/618 | 70 +++++++++++++++++++++++++++++++++++++++++++
>>> tests/generic/618.out | 3 ++
>>> tests/generic/group | 1 +
>>> 4 files changed, 84 insertions(+)
>>> create mode 100755 tests/generic/618
>>> create mode 100644 tests/generic/618.out
>>>
>>> diff --git a/common/rc b/common/rc
>>> index b5a504e0dcb4..128d75226958 100644
>>> --- a/common/rc
>>> +++ b/common/rc
>>> @@ -1608,6 +1608,16 @@ _require_scratch_size()
>>> [ $devsize -lt $1 ] && _notrun "scratch dev too small"
>>> }
>>>
>>> +# require a scratch dev of a minimum size (in kb) and should not be checked
>>> +# post test
>>> +_require_scratch_size_nocheck()
>>> +{
>>> + [ $# -eq 1 ] || _fail "_require_scratch_size: expected size param"
>>> +
>>> + _require_scratch_nocheck
>>> + local devsize=`_get_device_size $SCRATCH_DEV`
>>> + [ $devsize -lt $1 ] && _notrun "scratch dev too small"
>>> +}
>>>
>>> # this test needs a test partition - check we're ok & mount it
>>> #
>>> diff --git a/tests/generic/618 b/tests/generic/618
>>> new file mode 100755
>>> index 000000000000..45c14da80c06
>>> --- /dev/null
>>> +++ b/tests/generic/618
>>> @@ -0,0 +1,70 @@
>>> +#! /bin/bash
>>> +# SPDX-License-Identifier: GPL-2.0
>>> +# Copyright (c) 2020 Christian Kujau. All Rights Reserved.
>>> +# Copyright (c) 2020 Ritesh Harjani. All Rights Reserved.
>>> +#
>>> +# FS QA Test generic/618
>>> +#
>>> +# Since the test is not specific to ext4, hence adding it to generic.
>>> +# Add this test to check for regression which was reported when ext4 bmap
>>> +# aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
>>> +# from fs/inode.c which was failing since iomap_bmap() implementation earlier
>>> +# returned 0 for block addr > INT_MAX.
>>> +# This regression was fixed with following kernel commit [1]
>>> +# commit b75dfde1212991b24b220c3995101c60a7b8ae74
>>> +# ("fibmap: Warn and return an error in case of block > INT_MAX")
>>> +# [1]: https://patchwork.ozlabs.org/patch/1279914
>>> +#
>>> +seq=`basename $0`
>>> +seqres=$RESULT_DIR/$seq
>>> +echo "QA output created by $seq"
>>> +
>>> +here=`pwd`
>>> +tmp=/tmp/$$
>>> +status=1 # failure is the default!
>>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>>> +
>>> +_cleanup()
>>> +{
>>> + _dmhugedisk_cleanup
>>> + cd /
>>> + rm -f $tmp.*
>>> +}
>>> +
>>> +# get standard environment, filters and checks
>>> +. ./common/rc
>>> +. ./common/filter
>>> +. ./common/dmhugedisk
>>> +
>>> +# remove previous $seqres.full before test
>>> +rm -f $seqres.full
>>> +
>>> +# Modify as appropriate.
>>> +_supported_fs generic
>>> +_require_dmhugedisk
>>> +_require_scratch_size_nocheck $((4 * 1024 * 1024)) #kB
>>> +
>>> +# For 1k bs with ext4, mkfs was failing due to size limitation and also it
>>> +# becomes too slow when doing an mkfs on a huge sparse ext4 FS with 1k bs.
>>> +# Hence on ext4 run only for 4K bs.
>>> +if [ "$FSTYP" == "ext4" ]; then
>>> + _scratch_mkfs > /dev/null 2>&1
>>> + blksz=$(sudo debugfs -R stats $SCRATCH_DEV 2> /dev/null |grep "Block size" |cut -d ':' -f 2)
>>> + test $blksz -lt 4096 && _notrun "This test requires ext4 with minimum 4k bs"
>>> +fi
>>
>> As this is a generic test, the same check should be done with ext2 and
>> ext3. And actually this test requires > 16T fs support. So I'd suggest
>> add a new helper, maybe called _require_16T_support, to check if $FSTYP
>> supports filesystem size > 16T. And for now, we could only check for
>> extN.
>>
>> Also, there's no need to run command with sudo in fstests, as all tests
>> are required to be run by root.
>>
>> And there's a helper to get fs block size called _get_block_size, no
>> need to parse output of debugfs. >
> Maybe something like (not tested)
>
> _require_scratch_16T_support()
> {
> case $FSTYP in
> ext2|ext3)
> _notrun "$FSTYP doesn't support >16T filesystem"
> ext4)
> _scratch_mkfs >> $seqres.full 2>&1
> _scratch_mount
> local blocksize=$(_get_block_size $SCRATCH_MNT)
> if [ $blocksize -lt 4096 ]; then
> _notrun "This test requires >16T fs support"
> fi
> ;;
> *)
> ;;
> esac
> }
Sure, agreed. Let me add this in v2.
Will add f2fs along with ext2/3 as not supported (as listed below).
https://android.googlesource.com/platform/external/f2fs-tools/+/refs/heads/master/lib/libf2fs.c#1183
Thanks
-ritesh
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-12-07 8:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-04 10:43 [PATCH 0/2] Section handling patch and huge sparse file generic/618 Ritesh Harjani
2020-12-04 10:43 ` [PATCH 1/2] check: source common/rc again if TEST_DEV was recreated Ritesh Harjani
2020-12-04 10:43 ` [PATCH 2/2] generic: Add test to check for mounting a huge sparse dm device Ritesh Harjani
2020-12-06 13:50 ` Eryu Guan
2020-12-06 14:14 ` Eryu Guan
2020-12-07 8:37 ` Ritesh Harjani
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.