* [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
@ 2019-05-13 15:52 Max Reitz
2019-05-13 16:31 ` Eric Blake
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Max Reitz @ 2019-05-13 15:52 UTC (permalink / raw)
To: qemu-block; +Cc: Nir Soffer, Kevin Wolf, Thomas Huth, qemu-devel, Max Reitz
It is possible for an empty file to take up blocks on a filesystem.
Make iotest 175 take this into account.
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
v2: [Nir]
- Use a function for filtering
- s/empty_blocks/extra_blocks/
---
tests/qemu-iotests/175 | 26 ++++++++++++++++++++++----
tests/qemu-iotests/175.out | 8 ++++----
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index d0ffc495c2..b5eb0aa856 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -28,10 +28,25 @@ status=1 # failure is the default!
_cleanup()
{
- _cleanup_test_img
+ _cleanup_test_img
+ rm -f "$TEST_DIR/empty"
}
trap "_cleanup; exit \$status" 0 1 2 3 15
+# Some file systems sometimes allocate extra blocks independently of
+# the file size. This function hides the resulting difference in the
+# stat -c '%b' output.
+# Parameter 1: Number of blocks an empty file occupies
+# Parameter 2: Image size in bytes
+_filter_blocks()
+{
+ extra_blocks=$1
+ img_size=$2
+
+ sed -e "s/blocks=$extra_blocks/nothing allocated/" \
+ -e "s/blocks=$((extra_blocks + img_size / 512))/everything allocated/"
+}
+
# get standard environment, filters and checks
. ./common.rc
. ./common.filter
@@ -40,18 +55,21 @@ _supported_fmt raw
_supported_proto file
_supported_os Linux
-size=1m
+size=$((1 * 1024 * 1024))
+
+touch "$TEST_DIR/empty"
+extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
echo
echo "== creating image with default preallocation =="
_make_test_img $size | _filter_imgfmt
-stat -c "size=%s, blocks=%b" $TEST_IMG
+stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
for mode in off full falloc; do
echo
echo "== creating image with preallocation $mode =="
IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
- stat -c "size=%s, blocks=%b" $TEST_IMG
+ stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
done
# success, all done
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index 76c02c6a57..6d9a5ed84e 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -2,17 +2,17 @@ QA output created by 175
== creating image with default preallocation ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, blocks=0
+size=1048576, nothing allocated
== creating image with preallocation off ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, blocks=0
+size=1048576, nothing allocated
== creating image with preallocation full ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, blocks=2048
+size=1048576, everything allocated
== creating image with preallocation falloc ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
-size=1048576, blocks=2048
+size=1048576, everything allocated
*** done
--
2.21.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
2019-05-13 15:52 [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information Max Reitz
@ 2019-05-13 16:31 ` Eric Blake
2019-05-13 19:31 ` Nir Soffer
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Eric Blake @ 2019-05-13 16:31 UTC (permalink / raw)
To: Max Reitz, qemu-block; +Cc: Nir Soffer, Kevin Wolf, Thomas Huth, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]
On 5/13/19 10:52 AM, Max Reitz wrote:
> It is possible for an empty file to take up blocks on a filesystem.
> Make iotest 175 take this into account.
>
> Reported-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> v2: [Nir]
> - Use a function for filtering
> - s/empty_blocks/extra_blocks/
> ---
> +# Some file systems sometimes allocate extra blocks independently of
> +# the file size. This function hides the resulting difference in the
> +# stat -c '%b' output.
> +# Parameter 1: Number of blocks an empty file occupies
> +# Parameter 2: Image size in bytes
> +_filter_blocks()
> +{
> + extra_blocks=$1
> + img_size=$2
> +
> + sed -e "s/blocks=$extra_blocks/nothing allocated/" \
> + -e "s/blocks=$((extra_blocks + img_size / 512))/everything allocated/"
> +}
That turned out nicely.
Reviewed-by: Eric Blake <eblake@redhat.com>
> -size=1m
> +size=$((1 * 1024 * 1024))
Had to change since bash doesn't understand '1m' inside $(()).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
2019-05-13 15:52 [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information Max Reitz
2019-05-13 16:31 ` Eric Blake
@ 2019-05-13 19:31 ` Nir Soffer
2019-05-15 4:17 ` Max Reitz
2019-05-16 11:03 ` Max Reitz
3 siblings, 0 replies; 7+ messages in thread
From: Nir Soffer @ 2019-05-13 19:31 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, Thomas Huth, QEMU Developers, qemu-block
On Mon, May 13, 2019, 18:52 Max Reitz <mreitz@redhat.com> wrote:
> It is possible for an empty file to take up blocks on a filesystem.
> Make iotest 175 take this into account.
>
> Reported-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> v2: [Nir]
> - Use a function for filtering
> - s/empty_blocks/extra_blocks/
> ---
> tests/qemu-iotests/175 | 26 ++++++++++++++++++++++----
> tests/qemu-iotests/175.out | 8 ++++----
> 2 files changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
> index d0ffc495c2..b5eb0aa856 100755
> --- a/tests/qemu-iotests/175
> +++ b/tests/qemu-iotests/175
> @@ -28,10 +28,25 @@ status=1 # failure is the default!
>
> _cleanup()
> {
> - _cleanup_test_img
> + _cleanup_test_img
> + rm -f "$TEST_DIR/empty"
> }
> trap "_cleanup; exit \$status" 0 1 2 3 15
>
> +# Some file systems sometimes allocate extra blocks independently of
> +# the file size. This function hides the resulting difference in the
> +# stat -c '%b' output.
> +# Parameter 1: Number of blocks an empty file occupies
> +# Parameter 2: Image size in bytes
> +_filter_blocks()
> +{
> + extra_blocks=$1
> + img_size=$2
> +
> + sed -e "s/blocks=$extra_blocks/nothing allocated/" \
> + -e "s/blocks=$((extra_blocks + img_size / 512))/everything
> allocated/"
> +}
> +
> # get standard environment, filters and checks
> . ./common.rc
> . ./common.filter
> @@ -40,18 +55,21 @@ _supported_fmt raw
> _supported_proto file
> _supported_os Linux
>
> -size=1m
> +size=$((1 * 1024 * 1024))
> +
> +touch "$TEST_DIR/empty"
> +extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
>
> echo
> echo "== creating image with default preallocation =="
> _make_test_img $size | _filter_imgfmt
> -stat -c "size=%s, blocks=%b" $TEST_IMG
> +stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks
> $size
>
> for mode in off full falloc; do
> echo
> echo "== creating image with preallocation $mode =="
> IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
> - stat -c "size=%s, blocks=%b" $TEST_IMG
> + stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks
> $size
> done
>
> # success, all done
> diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
> index 76c02c6a57..6d9a5ed84e 100644
> --- a/tests/qemu-iotests/175.out
> +++ b/tests/qemu-iotests/175.out
> @@ -2,17 +2,17 @@ QA output created by 175
>
> == creating image with default preallocation ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> -size=1048576, blocks=0
> +size=1048576, nothing allocated
>
> == creating image with preallocation off ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
> -size=1048576, blocks=0
> +size=1048576, nothing allocated
>
> == creating image with preallocation full ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
> -size=1048576, blocks=2048
> +size=1048576, everything allocated
>
> == creating image with preallocation falloc ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> preallocation=falloc
> -size=1048576, blocks=2048
> +size=1048576, everything allocated
> *** done
> --
> 2.21.0
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
2019-05-13 15:52 [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information Max Reitz
2019-05-13 16:31 ` Eric Blake
2019-05-13 19:31 ` Nir Soffer
@ 2019-05-15 4:17 ` Max Reitz
2019-05-16 11:03 ` Max Reitz
3 siblings, 0 replies; 7+ messages in thread
From: Max Reitz @ 2019-05-15 4:17 UTC (permalink / raw)
To: qemu-block; +Cc: Nir Soffer, Kevin Wolf, Thomas Huth, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 632 bytes --]
On 13.05.19 17:52, Max Reitz wrote:
> It is possible for an empty file to take up blocks on a filesystem.
> Make iotest 175 take this into account.
>
> Reported-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> v2: [Nir]
> - Use a function for filtering
> - s/empty_blocks/extra_blocks/
> ---
> tests/qemu-iotests/175 | 26 ++++++++++++++++++++++----
> tests/qemu-iotests/175.out | 8 ++++----
> 2 files changed, 26 insertions(+), 8 deletions(-)
Thanks for the reviews, applied to my block branch:
https://git.xanclic.moe/XanClic/qemu/commits/branch/block
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
2019-05-13 15:52 [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information Max Reitz
` (2 preceding siblings ...)
2019-05-15 4:17 ` Max Reitz
@ 2019-05-16 11:03 ` Max Reitz
2019-05-16 13:06 ` Nir Soffer
3 siblings, 1 reply; 7+ messages in thread
From: Max Reitz @ 2019-05-16 11:03 UTC (permalink / raw)
To: qemu-block; +Cc: Nir Soffer, Kevin Wolf, Thomas Huth, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1729 bytes --]
On 13.05.19 17:52, Max Reitz wrote:
> It is possible for an empty file to take up blocks on a filesystem.
> Make iotest 175 take this into account.
>
> Reported-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> v2: [Nir]
> - Use a function for filtering
> - s/empty_blocks/extra_blocks/
> ---
> tests/qemu-iotests/175 | 26 ++++++++++++++++++++++----
> tests/qemu-iotests/175.out | 8 ++++----
> 2 files changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
> index d0ffc495c2..b5eb0aa856 100755
> --- a/tests/qemu-iotests/175
> +++ b/tests/qemu-iotests/175
> @@ -28,10 +28,25 @@ status=1 # failure is the default!
>
> _cleanup()
> {
> - _cleanup_test_img
> + _cleanup_test_img
> + rm -f "$TEST_DIR/empty"
> }
> trap "_cleanup; exit \$status" 0 1 2 3 15
>
> +# Some file systems sometimes allocate extra blocks independently of
> +# the file size. This function hides the resulting difference in the
> +# stat -c '%b' output.
> +# Parameter 1: Number of blocks an empty file occupies
> +# Parameter 2: Image size in bytes
> +_filter_blocks()
> +{
> + extra_blocks=$1
> + img_size=$2
> +
> + sed -e "s/blocks=$extra_blocks/nothing allocated/" \
> + -e "s/blocks=$((extra_blocks + img_size / 512))/everything allocated/"
Only now got around to creating an FS with similar characteristics to
Thomas’s. Turns out this test still fails there because it prints
“blocks=nothing allocated050”...
Unless objections arise, I’ll just add a \$ to each pattern to fix that.
(I could also just swap the lines, but that isn’t really right.)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
2019-05-16 11:03 ` Max Reitz
@ 2019-05-16 13:06 ` Nir Soffer
2019-05-16 13:30 ` Max Reitz
0 siblings, 1 reply; 7+ messages in thread
From: Nir Soffer @ 2019-05-16 13:06 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, Thomas Huth, QEMU Developers, qemu-block
On Thu, May 16, 2019 at 2:03 PM Max Reitz <mreitz@redhat.com> wrote:
> On 13.05.19 17:52, Max Reitz wrote:
> > It is possible for an empty file to take up blocks on a filesystem.
> > Make iotest 175 take this into account.
> >
> > Reported-by: Thomas Huth <thuth@redhat.com>
> > Signed-off-by: Max Reitz <mreitz@redhat.com>
> > ---
> > v2: [Nir]
> > - Use a function for filtering
> > - s/empty_blocks/extra_blocks/
> > ---
> > tests/qemu-iotests/175 | 26 ++++++++++++++++++++++----
> > tests/qemu-iotests/175.out | 8 ++++----
> > 2 files changed, 26 insertions(+), 8 deletions(-)
> >
> > diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
> > index d0ffc495c2..b5eb0aa856 100755
> > --- a/tests/qemu-iotests/175
> > +++ b/tests/qemu-iotests/175
> > @@ -28,10 +28,25 @@ status=1 # failure is the default!
> >
> > _cleanup()
> > {
> > - _cleanup_test_img
> > + _cleanup_test_img
> > + rm -f "$TEST_DIR/empty"
> > }
> > trap "_cleanup; exit \$status" 0 1 2 3 15
> >
> > +# Some file systems sometimes allocate extra blocks independently of
> > +# the file size. This function hides the resulting difference in the
> > +# stat -c '%b' output.
> > +# Parameter 1: Number of blocks an empty file occupies
> > +# Parameter 2: Image size in bytes
> > +_filter_blocks()
> > +{
> > + extra_blocks=$1
> > + img_size=$2
> > +
> > + sed -e "s/blocks=$extra_blocks/nothing allocated/" \
> > + -e "s/blocks=$((extra_blocks + img_size / 512))/everything
> allocated/"
>
> Only now got around to creating an FS with similar characteristics to
> Thomas’s. Turns out this test still fails there because it prints
> “blocks=nothing allocated050”...
>
> Unless objections arise, I’ll just add a \$ to each pattern to fix that.
> (I could also just swap the lines, but that isn’t really right.)
>
Sounds right.
Can you share how to create file system which reproduces this?
Maybe adding the details in the test would be useful.
Nir
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information
2019-05-16 13:06 ` Nir Soffer
@ 2019-05-16 13:30 ` Max Reitz
0 siblings, 0 replies; 7+ messages in thread
From: Max Reitz @ 2019-05-16 13:30 UTC (permalink / raw)
To: Nir Soffer; +Cc: Kevin Wolf, Thomas Huth, QEMU Developers, qemu-block
[-- Attachment #1: Type: text/plain, Size: 2610 bytes --]
On 16.05.19 15:06, Nir Soffer wrote:
> On Thu, May 16, 2019 at 2:03 PM Max Reitz <mreitz@redhat.com
> <mailto:mreitz@redhat.com>> wrote:
>
> On 13.05.19 17:52, Max Reitz wrote:
> > It is possible for an empty file to take up blocks on a filesystem.
> > Make iotest 175 take this into account.
> >
> > Reported-by: Thomas Huth <thuth@redhat.com <mailto:thuth@redhat.com>>
> > Signed-off-by: Max Reitz <mreitz@redhat.com
> <mailto:mreitz@redhat.com>>
> > ---
> > v2: [Nir]
> > - Use a function for filtering
> > - s/empty_blocks/extra_blocks/
> > ---
> > tests/qemu-iotests/175 | 26 ++++++++++++++++++++++----
> > tests/qemu-iotests/175.out | 8 ++++----
> > 2 files changed, 26 insertions(+), 8 deletions(-)
> >
> > diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
> > index d0ffc495c2..b5eb0aa856 100755
> > --- a/tests/qemu-iotests/175
> > +++ b/tests/qemu-iotests/175
> > @@ -28,10 +28,25 @@ status=1 # failure is the default!
> >
> > _cleanup()
> > {
> > - _cleanup_test_img
> > + _cleanup_test_img
> > + rm -f "$TEST_DIR/empty"
> > }
> > trap "_cleanup; exit \$status" 0 1 2 3 15
> >
> > +# Some file systems sometimes allocate extra blocks independently of
> > +# the file size. This function hides the resulting difference in the
> > +# stat -c '%b' output.
> > +# Parameter 1: Number of blocks an empty file occupies
> > +# Parameter 2: Image size in bytes
> > +_filter_blocks()
> > +{
> > + extra_blocks=$1
> > + img_size=$2
> > +
> > + sed -e "s/blocks=$extra_blocks/nothing allocated/" \
> > + -e "s/blocks=$((extra_blocks + img_size /
> 512))/everything allocated/"
>
> Only now got around to creating an FS with similar characteristics to
> Thomas’s. Turns out this test still fails there because it prints
> “blocks=nothing allocated050”...
>
> Unless objections arise, I’ll just add a \$ to each pattern to fix that.
> (I could also just swap the lines, but that isn’t really right.)
>
>
> Sounds right.
>
> Can you share how to create file system which reproduces this?
> Maybe adding the details in the test would be useful.
Sure:
$ mkfs.ext4 -b 1024 -I 128 /dev/loop0
-I 128 is the important one. The block size just changes the number of
extra blocks (2 with 1024, 8 with 4096).
...I guess I’ll just a v3 now.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-05-16 13:32 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-13 15:52 [Qemu-devel] [PATCH v2] iotests: Filter 175's allocation information Max Reitz
2019-05-13 16:31 ` Eric Blake
2019-05-13 19:31 ` Nir Soffer
2019-05-15 4:17 ` Max Reitz
2019-05-16 11:03 ` Max Reitz
2019-05-16 13:06 ` Nir Soffer
2019-05-16 13:30 ` Max Reitz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).