[PATCH 1/3] btrfs-progs: test: fix error of test target of Makefile

[PATCH 1/3] btrfs-progs: test: fix error of test target of Makefile

[Tsutomu Itoh]

Add test-cli to test target of Makefile.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
---
 Makefile.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile.in b/Makefile.in
index 4bd3e63..6c78841 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -298,7 +298,7 @@ test-inst: all
 		$(MAKE) DESTDIR=$$tmpdest install && \
 		$(RM) -rf -- $$tmpdest
 
-test: test-fsck test-mkfs test-convert test-misc test-fuzz
+test: test-fsck test-mkfs test-convert test-misc test-fuzz test-cli
 
 #
 # NOTE: For static compiles, you need to have all the required libs
-- 
2.9.3

[PATCH 2/3] btrfs-progs: test: expand size of test device of fsck-tests 013

[Tsutomu Itoh]

In my test environment, size of /lib/modules/`uname -r`/* is
larger than 1GB, so test data can not copy to testdev.
Therefore we need expand the size of testdev.

# make test-fsck
    [TEST]   fsck-tests.sh
    [TEST/fsck]   013-extent-tree-rebuild
failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
test failed for case 013-extent-tree-rebuild
Makefile:272: recipe for target 'test-fsck' failed
make: *** [test-fsck] Error 1
#

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
---
 tests/fsck-tests/013-extent-tree-rebuild/test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/fsck-tests/013-extent-tree-rebuild/test.sh b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
index f678e29..92c5d04 100755
--- a/tests/fsck-tests/013-extent-tree-rebuild/test.sh
+++ b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
@@ -7,7 +7,7 @@ check_prereq mkfs.btrfs
 check_prereq btrfs
 
 setup_root_helper
-prepare_test_dev 1G
+prepare_test_dev 2G
 
 # test whether fsck can rebuild a corrupted extent tree
 test_extent_tree_rebuild()
-- 
2.9.3

[PATCH 3/3] btrfs-progs: test: fix convert-tests 004 failure

[Tsutomu Itoh]

convert-tests 004 failed because the argument to check_all_images
is not specified.

# make test-convert
    [TEST]   convert-tests.sh
    [TEST/conv]   004-ext2-backup-superblock-ranges
find: '': No such file or directory
#

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
---
 tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh b/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh
index c56650b..d764ed8 100755
--- a/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh
+++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh
@@ -39,4 +39,4 @@ function check_image() {
 	rm -f $TEST_DEV
 }
 
-check_all_images
+check_all_images `pwd`
-- 
2.9.3

Re: [PATCH 2/3] btrfs-progs: test: expand size of test device of fsck-tests 013

[Qu Wenruo]

On 11/18/2016 01:47 PM, Tsutomu Itoh wrote:
> In my test environment, size of /lib/modules/`uname -r`/* is
> larger than 1GB, so test data can not copy to testdev.
> Therefore we need expand the size of testdev.

IMHO the correct fix is to enhance populate_fs() to fill the fs into a 
given size, other than using the unreliable copy.

Thanks,
Qu

>
> # make test-fsck
>     [TEST]   fsck-tests.sh
>     [TEST/fsck]   013-extent-tree-rebuild
> failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
> test failed for case 013-extent-tree-rebuild
> Makefile:272: recipe for target 'test-fsck' failed
> make: *** [test-fsck] Error 1
> #
>
> Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
> ---
>  tests/fsck-tests/013-extent-tree-rebuild/test.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/fsck-tests/013-extent-tree-rebuild/test.sh b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
> index f678e29..92c5d04 100755
> --- a/tests/fsck-tests/013-extent-tree-rebuild/test.sh
> +++ b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
> @@ -7,7 +7,7 @@ check_prereq mkfs.btrfs
>  check_prereq btrfs
>
>  setup_root_helper
> -prepare_test_dev 1G
> +prepare_test_dev 2G
>
>  # test whether fsck can rebuild a corrupted extent tree
>  test_extent_tree_rebuild()
>

Re: [PATCH 3/3] btrfs-progs: test: fix convert-tests 004 failure

[David Sterba]

On Fri, Nov 18, 2016 at 02:49:51PM +0900, Tsutomu Itoh wrote:
> convert-tests 004 failed because the argument to check_all_images
> is not specified.
> 
> # make test-convert
>     [TEST]   convert-tests.sh
>     [TEST/conv]   004-ext2-backup-superblock-ranges
> find: '': No such file or directory
> #
> 
> Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
> ---
>  tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh b/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh
> index c56650b..d764ed8 100755
> --- a/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh
> +++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh
> @@ -39,4 +39,4 @@ function check_image() {
>  	rm -f $TEST_DEV
>  }
>  
> -check_all_images
> +check_all_images `pwd`

I'll add a fallback to use the current directory instead of forcing all
callsites to supply it.

Re: [PATCH 2/3] btrfs-progs: test: expand size of test device of fsck-tests 013

[David Sterba]

On Fri, Nov 18, 2016 at 02:36:28PM +0800, Qu Wenruo wrote:
> 
> 
> On 11/18/2016 01:47 PM, Tsutomu Itoh wrote:
> > In my test environment, size of /lib/modules/`uname -r`/* is
> > larger than 1GB, so test data can not copy to testdev.
> > Therefore we need expand the size of testdev.
> 
> IMHO the correct fix is to enhance populate_fs() to fill the fs into a 
> given size, other than using the unreliable copy.

Yeah that would be better, the contents of the modules dir could be
anything, but was enough to use for initial testing.

Re: [PATCH 1/3] btrfs-progs: test: fix error of test target of Makefile

[David Sterba]

On Fri, Nov 18, 2016 at 02:44:12PM +0900, Tsutomu Itoh wrote:
> Add test-cli to test target of Makefile.
> 
> Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>

Applied, thanks.

Re: [PATCH 2/3] btrfs-progs: test: expand size of test device of fsck-tests 013

[Tsutomu Itoh]

Hi, Qu and David,

On 2016/11/19 3:28, David Sterba wrote:
> On Fri, Nov 18, 2016 at 02:36:28PM +0800, Qu Wenruo wrote:
>>
>>
>> On 11/18/2016 01:47 PM, Tsutomu Itoh wrote:
>>> In my test environment, size of /lib/modules/`uname -r`/* is
>>> larger than 1GB, so test data can not copy to testdev.
>>> Therefore we need expand the size of testdev.
>>
>> IMHO the correct fix is to enhance populate_fs() to fill the fs into a 
>> given size, other than using the unreliable copy.
> 
> Yeah that would be better, the contents of the modules dir could be
> anything, but was enough to use for initial testing.

Thanks for your comments. I'll post v2 patch.

Thanks,
Tsutomu

[PATCH v2 2/3] btrfs-progs: test: fix how to make test files in fsck-tests 013

[Tsutomu Itoh]

In my test environment, following error was occurred because the size
of /lib/modules/`uname -r`/* is larger than 1GB.

# make test-fsck
    [TEST]   fsck-tests.sh
    [TEST/fsck]   013-extent-tree-rebuild
failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
test failed for case 013-extent-tree-rebuild
Makefile:272: recipe for target 'test-fsck' failed
make: *** [test-fsck] Error 1
#

In this test case, 'generate_dataset small' is enough for making the
test files, so I will use 'generate_dataset' instead of 'cp'.

For this, move 'generate_dataset()' from 'common.convert' to 'common'.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
---
v2: change how to make test files
---
 tests/common                                     | 89 ++++++++++++++++++++++++
 tests/common.convert                             | 89 ------------------------
 tests/fsck-tests/013-extent-tree-rebuild/test.sh |  2 +-
 3 files changed, 90 insertions(+), 90 deletions(-)

diff --git a/tests/common b/tests/common
index c61962d..8b760f7 100644
--- a/tests/common
+++ b/tests/common
@@ -309,6 +309,95 @@ check_kernel_support()
 	return 0
 }
 
+# how many files to create.
+DATASET_SIZE=50
+
+generate_dataset() {
+
+	dataset_type="$1"
+	dirpath=$TEST_MNT/$dataset_type
+	run_check $SUDO_HELPER mkdir -p "$dirpath"
+
+	case $dataset_type in
+		small)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER dd if=/dev/urandom of="$dirpath/$dataset_type.$num" bs=10K \
+				count=1 >/dev/null 2>&1
+			done
+			;;
+
+		hardlink)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num
+				run_check $SUDO_HELPER ln "$dirpath/$dataset_type.$num" "$dirpath/hlink.$num"
+			done
+			;;
+
+		fast_symlink)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num
+				run_check cd "$dirpath" && \
+					$SUDO_HELPER ln -s "$dataset_type.$num" "$dirpath/slink.$num" && \
+					cd /
+			done
+			;;
+
+		brokenlink)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER ln -s "$dirpath/$dataset_type.$num" "$dirpath/blink.$num"
+			done
+			;;
+
+		perm)
+			for modes in 777 775 755 750 700 666 664 644 640 600 444 440 400 000		\
+				1777 1775 1755 1750 1700 1666 1664 1644 1640 1600 1444 1440 1400 1000	\
+				2777 2775 2755 2750 2700 2666 2664 2644 2640 2600 2444 2440 2400 2000	\
+				4777 4775 4755 4750 4700 4666 4664 4644 4640 4600 4444 4440 4400 4000; do
+				if [[ "$modes" == *9* ]] || [[ "$modes" == *8* ]]
+				then
+					continue;
+				else
+					run_check $SUDO_HELPER touch "$dirpath/$dataset_type.$modes"
+					run_check $SUDO_HELPER chmod "$modes" "$dirpath/$dataset_type.$modes"
+				fi
+			done
+			;;
+
+		sparse)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER dd if=/dev/urandom of="$dirpath/$dataset_type.$num" bs=10K \
+				count=1 >/dev/null 2>&1
+				run_check $SUDO_HELPER truncate -s 500K "$dirpath/$dataset_type.$num"
+				run_check $SUDO_HELPER dd if=/dev/urandom of="$dirpath/$dataset_type.$num" bs=10K \
+				oflag=append conv=notrunc count=1 >/dev/null 2>&1
+				run_check $SUDO_HELPER truncate -s 800K "$dirpath/$dataset_type.$num"
+			done
+			;;
+
+		acls)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER touch "$dirpath/$dataset_type.$num"
+				run_check $SUDO_HELPER setfacl -m "u:root:x" "$dirpath/$dataset_type.$num"
+				run_check $SUDO_HELPER setfattr -n user.foo -v "bar$num" "$dirpath/$dataset_type.$num"
+			done
+			;;
+
+		fifo)
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER mkfifo "$dirpath/$dataset_type.$num"
+			done
+			;;
+
+		slow_symlink)
+			long_filename=`date +%s | sha256sum | cut -f1 -d'-'`
+			run_check $SUDO_HELPER touch "$dirpath/$long_filename"
+			for num in $(seq 1 "$DATASET_SIZE"); do
+				run_check $SUDO_HELPER ln -s "$dirpath/$long_filename" "$dirpath/slow_slink.$num"
+			done
+			;;
+	esac
+}
+
 init_env()
 {
 	TEST_MNT="${TEST_MNT:-$TOP/tests/mnt}"
diff --git a/tests/common.convert b/tests/common.convert
index 375d4b5..3481c4d 100644
--- a/tests/common.convert
+++ b/tests/common.convert
@@ -1,95 +1,6 @@
 #!/bin/bash
 # helpers for btrfs-convert tests
 
-# how many files to create.
-DATASET_SIZE=50
-
-generate_dataset() {
-
-	dataset_type="$1"
-	dirpath=$TEST_MNT/$dataset_type
-	run_check $SUDO_HELPER mkdir -p "$dirpath"
-
-	case $dataset_type in
-		small)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER dd if=/dev/urandom of="$dirpath/$dataset_type.$num" bs=10K \
-				count=1 >/dev/null 2>&1
-			done
-			;;
-
-		hardlink)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num
-				run_check $SUDO_HELPER ln "$dirpath/$dataset_type.$num" "$dirpath/hlink.$num"
-			done
-			;;
-
-		fast_symlink)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num
-				run_check cd "$dirpath" && \
-					$SUDO_HELPER ln -s "$dataset_type.$num" "$dirpath/slink.$num" && \
-					cd /
-			done
-			;;
-
-		brokenlink)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER ln -s "$dirpath/$dataset_type.$num" "$dirpath/blink.$num"
-			done
-			;;
-
-		perm)
-			for modes in 777 775 755 750 700 666 664 644 640 600 444 440 400 000		\
-				1777 1775 1755 1750 1700 1666 1664 1644 1640 1600 1444 1440 1400 1000	\
-				2777 2775 2755 2750 2700 2666 2664 2644 2640 2600 2444 2440 2400 2000	\
-				4777 4775 4755 4750 4700 4666 4664 4644 4640 4600 4444 4440 4400 4000; do
-				if [[ "$modes" == *9* ]] || [[ "$modes" == *8* ]]
-				then
-					continue;
-				else
-					run_check $SUDO_HELPER touch "$dirpath/$dataset_type.$modes"
-					run_check $SUDO_HELPER chmod "$modes" "$dirpath/$dataset_type.$modes"
-				fi
-			done
-			;;
-
-		sparse)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER dd if=/dev/urandom of="$dirpath/$dataset_type.$num" bs=10K \
-				count=1 >/dev/null 2>&1
-				run_check $SUDO_HELPER truncate -s 500K "$dirpath/$dataset_type.$num"
-				run_check $SUDO_HELPER dd if=/dev/urandom of="$dirpath/$dataset_type.$num" bs=10K \
-				oflag=append conv=notrunc count=1 >/dev/null 2>&1
-				run_check $SUDO_HELPER truncate -s 800K "$dirpath/$dataset_type.$num"
-			done
-			;;
-
-		acls)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER touch "$dirpath/$dataset_type.$num"
-				run_check $SUDO_HELPER setfacl -m "u:root:x" "$dirpath/$dataset_type.$num"
-				run_check $SUDO_HELPER setfattr -n user.foo -v "bar$num" "$dirpath/$dataset_type.$num"
-			done
-			;;
-
-		fifo)
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER mkfifo "$dirpath/$dataset_type.$num"
-			done
-			;;
-
-		slow_symlink)
-			long_filename=`date +%s | sha256sum | cut -f1 -d'-'`
-			run_check $SUDO_HELPER touch "$dirpath/$long_filename"
-			for num in $(seq 1 "$DATASET_SIZE"); do
-				run_check $SUDO_HELPER ln -s "$dirpath/$long_filename" "$dirpath/slow_slink.$num"
-			done
-			;;
-	esac
-}
-
 populate_fs() {
 
         for dataset_type in 'small' 'hardlink' 'fast_symlink' 'brokenlink' 'perm' 'sparse' 'acls' 'fifo' 'slow_symlink'; do
diff --git a/tests/fsck-tests/013-extent-tree-rebuild/test.sh b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
index f678e29..37bdcd9 100755
--- a/tests/fsck-tests/013-extent-tree-rebuild/test.sh
+++ b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
@@ -15,7 +15,7 @@ test_extent_tree_rebuild()
 	run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $TEST_DEV
 
 	run_check_mount_test_dev
-	run_check $SUDO_HELPER cp -aR /lib/modules/`uname -r`/ $TEST_MNT
+	generate_dataset small
 
 	for i in `seq 1 100`;do
 		run_check $SUDO_HELPER $TOP/btrfs sub snapshot $TEST_MNT \
-- 
2.9.3