[PATCH 0/3] iotests: Check for the possibility to create large files

[PATCH 0/3] iotests: Check for the possibility to create large files

[Thomas Huth]

Travis recently added the possibility to test on ppc64le, arm64 and s390x
hosts, too. However, the containers are very restricted there and do not
allow the creation of large files, so that the tests 060 and 079 are
currently failing there. So let's add some proper checks to these tests
first.

Thomas Huth (3):
  iotests: Provide a function for checking the creation of huge files
  iotests: Skip test 060 if it is not possible to create large files
  iotests: Skip test 079 if it is not possible to create large files

 tests/qemu-iotests/005       |  5 +----
 tests/qemu-iotests/060       |  3 +++
 tests/qemu-iotests/079       |  3 +++
 tests/qemu-iotests/220       |  6 ++----
 tests/qemu-iotests/common.rc | 10 ++++++++++
 5 files changed, 19 insertions(+), 8 deletions(-)

-- 
2.18.1

[PATCH 1/3] iotests: Provide a function for checking the creation of huge files

[Thomas Huth]

Some tests create huge (but sparse) files, and to be able to run those
tests in certain limited environments (like CI containers), we have to
check for the possibility to create such files first. Thus let's introduce
a common function to check for large files, and replace the already
existing checks in the iotests 005 and 220 with this function.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/005       |  5 +----
 tests/qemu-iotests/220       |  6 ++----
 tests/qemu-iotests/common.rc | 10 ++++++++++
 3 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index 58442762fe..b6d03ac37d 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -59,10 +59,7 @@ fi
 # Sanity check: For raw, we require a file system that permits the creation
 # of a HUGE (but very sparse) file. Check we can create it before continuing.
 if [ "$IMGFMT" = "raw" ]; then
-    if ! truncate --size=5T "$TEST_IMG"; then
-        _notrun "file system on $TEST_DIR does not support large enough files"
-    fi
-    rm "$TEST_IMG"
+    _require_large_file 5T
 fi
 
 echo
diff --git a/tests/qemu-iotests/220 b/tests/qemu-iotests/220
index 2d62c5dcac..15159270d3 100755
--- a/tests/qemu-iotests/220
+++ b/tests/qemu-iotests/220
@@ -42,10 +42,8 @@ echo "== Creating huge file =="
 
 # Sanity check: We require a file system that permits the creation
 # of a HUGE (but very sparse) file.  tmpfs works, ext4 does not.
-if ! truncate --size=513T "$TEST_IMG"; then
-    _notrun "file system on $TEST_DIR does not support large enough files"
-fi
-rm "$TEST_IMG"
+_require_large_file 513T
+
 IMGOPTS='cluster_size=2M,refcount_bits=1' _make_test_img 513T
 
 echo "== Populating refcounts =="
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 38e949cf69..91c0217e59 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -657,5 +657,15 @@ _require_devices()
     done
 }
 
+# Check that we have a file system that allows huge (but very sparse) files
+#
+_require_large_file()
+{
+    if ! truncate --size="$1" "$TEST_IMG"; then
+        _notrun "file system on $TEST_DIR does not support large enough files"
+    fi
+    rm "$TEST_IMG"
+}
+
 # make sure this script returns success
 true
-- 
2.18.1

Re: [PATCH 1/3] iotests: Provide a function for checking the creation of huge files

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> Some tests create huge (but sparse) files, and to be able to run those
> tests in certain limited environments (like CI containers), we have to
> check for the possibility to create such files first. Thus let's introduce
> a common function to check for large files, and replace the already
> existing checks in the iotests 005 and 220 with this function.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/qemu-iotests/005       |  5 +----
>  tests/qemu-iotests/220       |  6 ++----
>  tests/qemu-iotests/common.rc | 10 ++++++++++
>  3 files changed, 13 insertions(+), 8 deletions(-)
>
> diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
> index 58442762fe..b6d03ac37d 100755
> --- a/tests/qemu-iotests/005
> +++ b/tests/qemu-iotests/005
> @@ -59,10 +59,7 @@ fi
>  # Sanity check: For raw, we require a file system that permits the creation
>  # of a HUGE (but very sparse) file. Check we can create it before continuing.
>  if [ "$IMGFMT" = "raw" ]; then
> -    if ! truncate --size=5T "$TEST_IMG"; then
> -        _notrun "file system on $TEST_DIR does not support large enough files"
> -    fi
> -    rm "$TEST_IMG"
> +    _require_large_file 5T
>  fi
>  
>  echo
> diff --git a/tests/qemu-iotests/220 b/tests/qemu-iotests/220
> index 2d62c5dcac..15159270d3 100755
> --- a/tests/qemu-iotests/220
> +++ b/tests/qemu-iotests/220
> @@ -42,10 +42,8 @@ echo "== Creating huge file =="
>  
>  # Sanity check: We require a file system that permits the creation
>  # of a HUGE (but very sparse) file.  tmpfs works, ext4 does not.
> -if ! truncate --size=513T "$TEST_IMG"; then
> -    _notrun "file system on $TEST_DIR does not support large enough files"
> -fi
> -rm "$TEST_IMG"
> +_require_large_file 513T
> +
>  IMGOPTS='cluster_size=2M,refcount_bits=1' _make_test_img 513T
>  
>  echo "== Populating refcounts =="
> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
> index 38e949cf69..91c0217e59 100644
> --- a/tests/qemu-iotests/common.rc
> +++ b/tests/qemu-iotests/common.rc
> @@ -657,5 +657,15 @@ _require_devices()
>      done
>  }
>  
> +# Check that we have a file system that allows huge (but very sparse) files
> +#
> +_require_large_file()
> +{
> +    if ! truncate --size="$1" "$TEST_IMG"; then
> +        _notrun "file system on $TEST_DIR does not support large enough files"
> +    fi
> +    rm "$TEST_IMG"
> +}
> +
>  # make sure this script returns success
>  true


-- 
Alex Bennée

[PATCH 2/3] iotests: Skip test 060 if it is not possible to create large files

[Thomas Huth]

Test 060 fails in the arm64, s390x and ppc64le LXD containers on Travis
(which we will hopefully enable in our CI soon). These containers
apparently do not allow large files to be created. The repair process
in test 060 creates a file of 64 GiB, so test first whether such large
files are possible and skip the test if that's not the case.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/060 | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index b91d8321bb..d96f17a484 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -49,6 +49,9 @@ _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
 
+# The repair process will create a large file - so check for availability first
+_require_large_file 64G
+
 rt_offset=65536  # 0x10000 (XXX: just an assumption)
 rb_offset=131072 # 0x20000 (XXX: just an assumption)
 l1_offset=196608 # 0x30000 (XXX: just an assumption)
-- 
2.18.1

[PATCH 3/3] iotests: Skip test 079 if it is not possible to create large files

[Thomas Huth]

Test 079 fails in the arm64, s390x and ppc64le LXD containers on Travis
(which we will hopefully enable in our CI soon). These containers
apparently do not allow large files to be created. Test 079 tries to
create a 4G sparse file, which is apparently already too big for these
containers, so check first whether we can really create such files before
executing the test.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/079 | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/079 b/tests/qemu-iotests/079
index 81f0c21f53..78536d3bbf 100755
--- a/tests/qemu-iotests/079
+++ b/tests/qemu-iotests/079
@@ -39,6 +39,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file nfs
 
+# Some containers (e.g. non-x86 on Travis) do not allow large files
+_require_large_file 4G
+
 echo "=== Check option preallocation and cluster_size ==="
 echo
 cluster_sizes="16384 32768 65536 131072 262144 524288 1048576 2097152 4194304"
-- 
2.18.1

Re: [PATCH 0/3] iotests: Check for the possibility to create large files

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> Travis recently added the possibility to test on ppc64le, arm64 and s390x
> hosts, too. However, the containers are very restricted there and do not
> allow the creation of large files, so that the tests 060 and 079 are
> currently failing there. So let's add some proper checks to these tests
> first.

These look good to me, do you want them to go via my testing/next or are
Kevin and Max going to take it via their tree?

>
> Thomas Huth (3):
>   iotests: Provide a function for checking the creation of huge files
>   iotests: Skip test 060 if it is not possible to create large files
>   iotests: Skip test 079 if it is not possible to create large files
>
>  tests/qemu-iotests/005       |  5 +----
>  tests/qemu-iotests/060       |  3 +++
>  tests/qemu-iotests/079       |  3 +++
>  tests/qemu-iotests/220       |  6 ++----
>  tests/qemu-iotests/common.rc | 10 ++++++++++
>  5 files changed, 19 insertions(+), 8 deletions(-)


-- 
Alex Bennée

Re: [PATCH 0/3] iotests: Check for the possibility to create large files

[Kevin Wolf]

Am 02.12.2019 um 11:16 hat Thomas Huth geschrieben:
> Travis recently added the possibility to test on ppc64le, arm64 and s390x
> hosts, too. However, the containers are very restricted there and do not
> allow the creation of large files, so that the tests 060 and 079 are
> currently failing there. So let's add some proper checks to these tests
> first.

Thanks, applied to the block branch.

Kevin