* [Qemu-devel] Failing QEMU iotest 175
@ 2019-04-28 15:18 ` Thomas Huth
0 siblings, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2019-04-28 15:18 UTC (permalink / raw)
To: Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Max Reitz, Eric Blake, Nir Soffer
QEMU iotest 175 is failing for me when I run it with -raw:
$ ./check -raw 175
QEMU --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
-nodefaults -machine accel=qtest
QEMU_IMG --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-img"
QEMU_IO --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-io" --cache
writeback -f raw
QEMU_NBD --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- raw
IMGPROTO -- file
PLATFORM -- Linux/x86_64 thuth 3.10.0-957.10.1.el7.x86_64
TEST_DIR -- /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER --
/home/thuth/tmp/qemu-build/tests/qemu-iotests/socket_scm_helper
175 - output mismatch (see 175.out.bad)
--- /home/thuth/devel/qemu/tests/qemu-iotests/175.out 2019-04-23
16:43:12.000000000 +0200
+++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/175.out.bad 2019-04-28
17:17:32.000000000 +0200
@@ -2,17 +2,17 @@
== creating image with default preallocation ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, blocks=0
+size=1048576, blocks=2
== creating image with preallocation off ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, blocks=0
+size=1048576, blocks=2
== creating image with preallocation full ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, blocks=2048
+size=1048576, blocks=2050
== creating image with preallocation falloc ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
preallocation=falloc
-size=1048576, blocks=2048
+size=1048576, blocks=2050
*** done
Failures: 175
Failed 1 of 1 tests
Any ideas how to fix this?
Thomas
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Qemu-devel] Failing QEMU iotest 175
@ 2019-04-28 15:18 ` Thomas Huth
0 siblings, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2019-04-28 15:18 UTC (permalink / raw)
To: Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Nir Soffer, Max Reitz
QEMU iotest 175 is failing for me when I run it with -raw:
$ ./check -raw 175
QEMU --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
-nodefaults -machine accel=qtest
QEMU_IMG --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-img"
QEMU_IO --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-io" --cache
writeback -f raw
QEMU_NBD --
"/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- raw
IMGPROTO -- file
PLATFORM -- Linux/x86_64 thuth 3.10.0-957.10.1.el7.x86_64
TEST_DIR -- /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER --
/home/thuth/tmp/qemu-build/tests/qemu-iotests/socket_scm_helper
175 - output mismatch (see 175.out.bad)
--- /home/thuth/devel/qemu/tests/qemu-iotests/175.out 2019-04-23
16:43:12.000000000 +0200
+++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/175.out.bad 2019-04-28
17:17:32.000000000 +0200
@@ -2,17 +2,17 @@
== creating image with default preallocation ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, blocks=0
+size=1048576, blocks=2
== creating image with preallocation off ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, blocks=0
+size=1048576, blocks=2
== creating image with preallocation full ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, blocks=2048
+size=1048576, blocks=2050
== creating image with preallocation falloc ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
preallocation=falloc
-size=1048576, blocks=2048
+size=1048576, blocks=2050
*** done
Failures: 175
Failed 1 of 1 tests
Any ideas how to fix this?
Thomas
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-02 21:56 ` Eric Blake
0 siblings, 0 replies; 18+ messages in thread
From: Eric Blake @ 2019-05-02 21:56 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Max Reitz, Nir Soffer
[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]
On 4/28/19 10:18 AM, Thomas Huth wrote:
> QEMU iotest 175 is failing for me when I run it with -raw:
>
> == creating image with default preallocation ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> -size=1048576, blocks=0
> +size=1048576, blocks=2
What filesystem? It should be fairly obvious that 'stat -c blocks=%b' is
file-system dependent (some allocate slightly more or less space, based
on granularities and on predictions of future use), so we may need to
update the test to apply a filter or otherwise allow a bit of fuzz in
the answer. But 0/2 is definitely different than...
>
> == creating image with preallocation off ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
> -size=1048576, blocks=0
> +size=1048576, blocks=2
>
> == creating image with preallocation full ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
> -size=1048576, blocks=2048
> +size=1048576, blocks=2050
2048/2050, so we DO have some indication of whether the file is sparse
or fully allocated.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-02 21:56 ` Eric Blake
0 siblings, 0 replies; 18+ messages in thread
From: Eric Blake @ 2019-05-02 21:56 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Nir Soffer, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]
On 4/28/19 10:18 AM, Thomas Huth wrote:
> QEMU iotest 175 is failing for me when I run it with -raw:
>
> == creating image with default preallocation ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> -size=1048576, blocks=0
> +size=1048576, blocks=2
What filesystem? It should be fairly obvious that 'stat -c blocks=%b' is
file-system dependent (some allocate slightly more or less space, based
on granularities and on predictions of future use), so we may need to
update the test to apply a filter or otherwise allow a bit of fuzz in
the answer. But 0/2 is definitely different than...
>
> == creating image with preallocation off ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
> -size=1048576, blocks=0
> +size=1048576, blocks=2
>
> == creating image with preallocation full ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
> -size=1048576, blocks=2048
> +size=1048576, blocks=2050
2048/2050, so we DO have some indication of whether the file is sparse
or fully allocated.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-03 4:37 ` Thomas Huth
0 siblings, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2019-05-03 4:37 UTC (permalink / raw)
To: Eric Blake, Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Max Reitz, Nir Soffer
[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]
On 02/05/2019 23.56, Eric Blake wrote:
> On 4/28/19 10:18 AM, Thomas Huth wrote:
>> QEMU iotest 175 is failing for me when I run it with -raw:
>>
>
>> == creating image with default preallocation ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>> -size=1048576, blocks=0
>> +size=1048576, blocks=2
>
> What filesystem?
ext4
> It should be fairly obvious that 'stat -c blocks=%b' is
> file-system dependent (some allocate slightly more or less space, based
> on granularities and on predictions of future use), so we may need to
> update the test to apply a filter or otherwise allow a bit of fuzz in
> the answer. But 0/2 is definitely different than...
>>
>> == creating image with preallocation off ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
>> -size=1048576, blocks=0
>> +size=1048576, blocks=2
>>
>> == creating image with preallocation full ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
>> -size=1048576, blocks=2048
>> +size=1048576, blocks=2050
>
> 2048/2050, so we DO have some indication of whether the file is sparse
> or fully allocated.
Maybe we could check that the value after "blocks=" is a single digit in
the first case, and matches "blocks=20.." in the second case?
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-03 4:37 ` Thomas Huth
0 siblings, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2019-05-03 4:37 UTC (permalink / raw)
To: Eric Blake, Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Nir Soffer, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]
On 02/05/2019 23.56, Eric Blake wrote:
> On 4/28/19 10:18 AM, Thomas Huth wrote:
>> QEMU iotest 175 is failing for me when I run it with -raw:
>>
>
>> == creating image with default preallocation ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>> -size=1048576, blocks=0
>> +size=1048576, blocks=2
>
> What filesystem?
ext4
> It should be fairly obvious that 'stat -c blocks=%b' is
> file-system dependent (some allocate slightly more or less space, based
> on granularities and on predictions of future use), so we may need to
> update the test to apply a filter or otherwise allow a bit of fuzz in
> the answer. But 0/2 is definitely different than...
>>
>> == creating image with preallocation off ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
>> -size=1048576, blocks=0
>> +size=1048576, blocks=2
>>
>> == creating image with preallocation full ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
>> -size=1048576, blocks=2048
>> +size=1048576, blocks=2050
>
> 2048/2050, so we DO have some indication of whether the file is sparse
> or fully allocated.
Maybe we could check that the value after "blocks=" is a single digit in
the first case, and matches "blocks=20.." in the second case?
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-03 20:21 ` Eric Blake
0 siblings, 0 replies; 18+ messages in thread
From: Eric Blake @ 2019-05-03 20:21 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Max Reitz, Nir Soffer
[-- Attachment #1: Type: text/plain, Size: 2199 bytes --]
On 5/2/19 11:37 PM, Thomas Huth wrote:
> On 02/05/2019 23.56, Eric Blake wrote:
>> On 4/28/19 10:18 AM, Thomas Huth wrote:
>>> QEMU iotest 175 is failing for me when I run it with -raw:
>>>
>>
>>> == creating image with default preallocation ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>>> -size=1048576, blocks=0
>>> +size=1048576, blocks=2
>>
>> What filesystem?
>
> ext4
>
Hmm, it's passing for me on ext4, but that probably means we have
different configuration parameters. I'm not sure how to easily show what
parameters a particular ext4 partition uses to compare the differences
between your setup and mine (mine is tuned to whatever defaults Fedora's
installer chose on my behalf), so maybe someone else can chime in.
>> It should be fairly obvious that 'stat -c blocks=%b' is
>> file-system dependent (some allocate slightly more or less space, based
>> on granularities and on predictions of future use), so we may need to
>> update the test to apply a filter or otherwise allow a bit of fuzz in
>> the answer. But 0/2 is definitely different than...
>>>
>>> == creating image with preallocation off ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
>>> -size=1048576, blocks=0
>>> +size=1048576, blocks=2
>>>
>>> == creating image with preallocation full ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
>>> -size=1048576, blocks=2048
>>> +size=1048576, blocks=2050
>>
>> 2048/2050, so we DO have some indication of whether the file is sparse
>> or fully allocated.
>
> Maybe we could check that the value after "blocks=" is a single digit in
> the first case, and matches "blocks=20.." in the second case?
I wonder if 'qemu-img map --output=json $TEST_IMG' might be any more
reliable (at least for ignoring any extra block allocations associated
with the file, if it is some journaling option or xattr or other reason
why your files seem to occupy more disk sectors than just the size of
the file would imply).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-03 20:21 ` Eric Blake
0 siblings, 0 replies; 18+ messages in thread
From: Eric Blake @ 2019-05-03 20:21 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Nir Soffer, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 2199 bytes --]
On 5/2/19 11:37 PM, Thomas Huth wrote:
> On 02/05/2019 23.56, Eric Blake wrote:
>> On 4/28/19 10:18 AM, Thomas Huth wrote:
>>> QEMU iotest 175 is failing for me when I run it with -raw:
>>>
>>
>>> == creating image with default preallocation ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>>> -size=1048576, blocks=0
>>> +size=1048576, blocks=2
>>
>> What filesystem?
>
> ext4
>
Hmm, it's passing for me on ext4, but that probably means we have
different configuration parameters. I'm not sure how to easily show what
parameters a particular ext4 partition uses to compare the differences
between your setup and mine (mine is tuned to whatever defaults Fedora's
installer chose on my behalf), so maybe someone else can chime in.
>> It should be fairly obvious that 'stat -c blocks=%b' is
>> file-system dependent (some allocate slightly more or less space, based
>> on granularities and on predictions of future use), so we may need to
>> update the test to apply a filter or otherwise allow a bit of fuzz in
>> the answer. But 0/2 is definitely different than...
>>>
>>> == creating image with preallocation off ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
>>> -size=1048576, blocks=0
>>> +size=1048576, blocks=2
>>>
>>> == creating image with preallocation full ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
>>> -size=1048576, blocks=2048
>>> +size=1048576, blocks=2050
>>
>> 2048/2050, so we DO have some indication of whether the file is sparse
>> or fully allocated.
>
> Maybe we could check that the value after "blocks=" is a single digit in
> the first case, and matches "blocks=20.." in the second case?
I wonder if 'qemu-img map --output=json $TEST_IMG' might be any more
reliable (at least for ignoring any extra block allocations associated
with the file, if it is some journaling option or xattr or other reason
why your files seem to occupy more disk sectors than just the size of
the file would imply).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-03 21:31 ` Nir Soffer
0 siblings, 0 replies; 18+ messages in thread
From: Nir Soffer @ 2019-05-03 21:31 UTC (permalink / raw)
To: Eric Blake
Cc: Thomas Huth, qemu-block, QEMU Developers, Kevin Wolf, Max Reitz
On Fri, May 3, 2019, 23:21 Eric Blake <eblake@redhat.com> wrote:
> On 5/2/19 11:37 PM, Thomas Huth wrote:
> > On 02/05/2019 23.56, Eric Blake wrote:
> >> On 4/28/19 10:18 AM, Thomas Huth wrote:
> >>> QEMU iotest 175 is failing for me when I run it with -raw:
> >>>
> >>
> >>> == creating image with default preallocation ==
> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> >>> -size=1048576, blocks=0
> >>> +size=1048576, blocks=2
> >>
> >> What filesystem?
> >
> > ext4
> >
>
> Hmm, it's passing for me on ext4, but that probably means we have
> different configuration parameters. I'm not sure how to easily show what
> parameters a particular ext4 partition uses to compare the differences
> between your setup and mine (mine is tuned to whatever defaults Fedora's
> installer chose on my behalf), so maybe someone else can chime in.
>
> >> It should be fairly obvious that 'stat -c blocks=%b' is
> >> file-system dependent (some allocate slightly more or less space, based
> >> on granularities and on predictions of future use), so we may need to
> >> update the test to apply a filter or otherwise allow a bit of fuzz in
> >> the answer. But 0/2 is definitely different than...
> >>>
> >>> == creating image with preallocation off ==
> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> preallocation=off
> >>> -size=1048576, blocks=0
> >>> +size=1048576, blocks=2
> >>>
> >>> == creating image with preallocation full ==
> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> preallocation=full
> >>> -size=1048576, blocks=2048
> >>> +size=1048576, blocks=2050
> >>
> >> 2048/2050, so we DO have some indication of whether the file is sparse
> >> or fully allocated.
> >
> > Maybe we could check that the value after "blocks=" is a single digit in
> > the first case, and matches "blocks=20.." in the second case?
>
> I wonder if 'qemu-img map --output=json $TEST_IMG' might be any more
> reliable (at least for ignoring any extra block allocations associated
> with the file, if it is some journaling option or xattr or other reason
> why your files seem to occupy more disk sectors than just the size of
> the file would imply).
>
I think it should work better and is more correct, testing actual sparsness
instead of underlying file system implementation.
I can send a fix next week.
Nir
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc. +1-919-301-3226
> Virtualization: qemu.org | libvirt.org
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-03 21:31 ` Nir Soffer
0 siblings, 0 replies; 18+ messages in thread
From: Nir Soffer @ 2019-05-03 21:31 UTC (permalink / raw)
To: Eric Blake
Cc: Kevin Wolf, Thomas Huth, QEMU Developers, qemu-block, Max Reitz
On Fri, May 3, 2019, 23:21 Eric Blake <eblake@redhat.com> wrote:
> On 5/2/19 11:37 PM, Thomas Huth wrote:
> > On 02/05/2019 23.56, Eric Blake wrote:
> >> On 4/28/19 10:18 AM, Thomas Huth wrote:
> >>> QEMU iotest 175 is failing for me when I run it with -raw:
> >>>
> >>
> >>> == creating image with default preallocation ==
> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> >>> -size=1048576, blocks=0
> >>> +size=1048576, blocks=2
> >>
> >> What filesystem?
> >
> > ext4
> >
>
> Hmm, it's passing for me on ext4, but that probably means we have
> different configuration parameters. I'm not sure how to easily show what
> parameters a particular ext4 partition uses to compare the differences
> between your setup and mine (mine is tuned to whatever defaults Fedora's
> installer chose on my behalf), so maybe someone else can chime in.
>
> >> It should be fairly obvious that 'stat -c blocks=%b' is
> >> file-system dependent (some allocate slightly more or less space, based
> >> on granularities and on predictions of future use), so we may need to
> >> update the test to apply a filter or otherwise allow a bit of fuzz in
> >> the answer. But 0/2 is definitely different than...
> >>>
> >>> == creating image with preallocation off ==
> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> preallocation=off
> >>> -size=1048576, blocks=0
> >>> +size=1048576, blocks=2
> >>>
> >>> == creating image with preallocation full ==
> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> preallocation=full
> >>> -size=1048576, blocks=2048
> >>> +size=1048576, blocks=2050
> >>
> >> 2048/2050, so we DO have some indication of whether the file is sparse
> >> or fully allocated.
> >
> > Maybe we could check that the value after "blocks=" is a single digit in
> > the first case, and matches "blocks=20.." in the second case?
>
> I wonder if 'qemu-img map --output=json $TEST_IMG' might be any more
> reliable (at least for ignoring any extra block allocations associated
> with the file, if it is some journaling option or xattr or other reason
> why your files seem to occupy more disk sectors than just the size of
> the file would imply).
>
I think it should work better and is more correct, testing actual sparsness
instead of underlying file system implementation.
I can send a fix next week.
Nir
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc. +1-919-301-3226
> Virtualization: qemu.org | libvirt.org
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-04 6:51 ` Thomas Huth
0 siblings, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2019-05-04 6:51 UTC (permalink / raw)
To: Eric Blake, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Max Reitz, Nir Soffer, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 2856 bytes --]
On 03/05/2019 22.21, Eric Blake wrote:
> On 5/2/19 11:37 PM, Thomas Huth wrote:
>> On 02/05/2019 23.56, Eric Blake wrote:
>>> On 4/28/19 10:18 AM, Thomas Huth wrote:
>>>> QEMU iotest 175 is failing for me when I run it with -raw:
>>>>
>>>> == creating image with default preallocation ==
>>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>>>> -size=1048576, blocks=0
>>>> +size=1048576, blocks=2
>>>
>>> What filesystem?
>>
>> ext4
>
> Hmm, it's passing for me on ext4, but that probably means we have
> different configuration parameters. I'm not sure how to easily show what
> parameters a particular ext4 partition uses to compare the differences
> between your setup and mine (mine is tuned to whatever defaults Fedora's
> installer chose on my behalf), so maybe someone else can chime in.
$ sudo tune2fs -l /dev/mapper/Home
tune2fs 1.42.9 (28-Dec-2013)
Filesystem volume name: <none>
Last mounted on: /home
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr dir_index filetype
needs_recovery meta_bg extent 64bit flex_bg sparse_super large_file
huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 36700160
Block count: 146800640
Reserved block count: 5873663
Free blocks: 56266267
Free inodes: 35403275
First block: 1
Block size: 1024
Fragment size: 1024
Group descriptor size: 64
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 2048
Inode blocks per group: 256
First meta block group: 258
Flex block group size: 16
Filesystem created: Thu Apr 19 18:34:33 2018
Last mount time: Sat May 4 08:20:36 2019
Last write time: Sat May 4 08:20:36 2019
Mount count: 224
Maximum mount count: -1
Last checked: Thu Apr 19 18:34:33 2018
Check interval: 0 (<none>)
Lifetime writes: 1826 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 11076944
Default directory hash: half_md4
Directory Hash Seed: 08e1be04-c3a3-4c37-a059-cf54af5c4bc0
Journal backup: inode blocks
IIRC I talked to stefanha on IRC about this some weeks ago already, and
he was able to reproduce the problem when using a certain parameter to
create the file system. However, I fail to remember which parameter it
was. Maybe Stefan still remembers...
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
@ 2019-05-04 6:51 ` Thomas Huth
0 siblings, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2019-05-04 6:51 UTC (permalink / raw)
To: Eric Blake, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Stefan Hajnoczi, Nir Soffer, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 2856 bytes --]
On 03/05/2019 22.21, Eric Blake wrote:
> On 5/2/19 11:37 PM, Thomas Huth wrote:
>> On 02/05/2019 23.56, Eric Blake wrote:
>>> On 4/28/19 10:18 AM, Thomas Huth wrote:
>>>> QEMU iotest 175 is failing for me when I run it with -raw:
>>>>
>>>> == creating image with default preallocation ==
>>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>>>> -size=1048576, blocks=0
>>>> +size=1048576, blocks=2
>>>
>>> What filesystem?
>>
>> ext4
>
> Hmm, it's passing for me on ext4, but that probably means we have
> different configuration parameters. I'm not sure how to easily show what
> parameters a particular ext4 partition uses to compare the differences
> between your setup and mine (mine is tuned to whatever defaults Fedora's
> installer chose on my behalf), so maybe someone else can chime in.
$ sudo tune2fs -l /dev/mapper/Home
tune2fs 1.42.9 (28-Dec-2013)
Filesystem volume name: <none>
Last mounted on: /home
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr dir_index filetype
needs_recovery meta_bg extent 64bit flex_bg sparse_super large_file
huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 36700160
Block count: 146800640
Reserved block count: 5873663
Free blocks: 56266267
Free inodes: 35403275
First block: 1
Block size: 1024
Fragment size: 1024
Group descriptor size: 64
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 2048
Inode blocks per group: 256
First meta block group: 258
Flex block group size: 16
Filesystem created: Thu Apr 19 18:34:33 2018
Last mount time: Sat May 4 08:20:36 2019
Last write time: Sat May 4 08:20:36 2019
Mount count: 224
Maximum mount count: -1
Last checked: Thu Apr 19 18:34:33 2018
Check interval: 0 (<none>)
Lifetime writes: 1826 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 11076944
Default directory hash: half_md4
Directory Hash Seed: 08e1be04-c3a3-4c37-a059-cf54af5c4bc0
Journal backup: inode blocks
IIRC I talked to stefanha on IRC about this some weeks ago already, and
he was able to reproduce the problem when using a certain parameter to
create the file system. However, I fail to remember which parameter it
was. Maybe Stefan still remembers...
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
2019-05-04 6:51 ` Thomas Huth
(?)
@ 2019-05-06 17:44 ` Eric Blake
2019-05-15 14:56 ` Stefan Hajnoczi
-1 siblings, 1 reply; 18+ messages in thread
From: Eric Blake @ 2019-05-06 17:44 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers
Cc: Kevin Wolf, Stefan Hajnoczi, Nir Soffer, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 4839 bytes --]
On 5/4/19 1:51 AM, Thomas Huth wrote:
>> Hmm, it's passing for me on ext4, but that probably means we have
>> different configuration parameters. I'm not sure how to easily show what
>> parameters a particular ext4 partition uses to compare the differences
>> between your setup and mine (mine is tuned to whatever defaults Fedora's
>> installer chose on my behalf), so maybe someone else can chime in.
>
> $ sudo tune2fs -l /dev/mapper/Home
> tune2fs 1.42.9 (28-Dec-2013)
> Filesystem volume name: <none>
> Last mounted on: /home
> Filesystem magic number: 0xEF53
> Filesystem revision #: 1 (dynamic)
> Filesystem features: has_journal ext_attr dir_index filetype
> needs_recovery meta_bg extent 64bit flex_bg sparse_super large_file
> huge_file uninit_bg dir_nlink extra_isize
> Filesystem flags: signed_directory_hash
> Default mount options: user_xattr acl
> Filesystem state: clean
> Errors behavior: Continue
> Filesystem OS type: Linux
> Inode count: 36700160
> Block count: 146800640
> Reserved block count: 5873663
> Free blocks: 56266267
> Free inodes: 35403275
> First block: 1
> Block size: 1024
> Fragment size: 1024
> Group descriptor size: 64
> Blocks per group: 8192
> Fragments per group: 8192
> Inodes per group: 2048
> Inode blocks per group: 256
> First meta block group: 258
> Flex block group size: 16
> Filesystem created: Thu Apr 19 18:34:33 2018
> Last mount time: Sat May 4 08:20:36 2019
> Last write time: Sat May 4 08:20:36 2019
> Mount count: 224
> Maximum mount count: -1
> Last checked: Thu Apr 19 18:34:33 2018
> Check interval: 0 (<none>)
> Lifetime writes: 1826 GB
> Reserved blocks uid: 0 (user root)
> Reserved blocks gid: 0 (group root)
> First inode: 11
> Inode size: 128
> Journal inode: 8
> First orphan inode: 11076944
> Default directory hash: half_md4
> Directory Hash Seed: 08e1be04-c3a3-4c37-a059-cf54af5c4bc0
> Journal backup: inode blocks
>
# tune2fs -l /dev/mapper/fedora-home
tune2fs 1.44.6 (5-Mar-2019)
Filesystem volume name: home
Last mounted on: /home
Filesystem UUID: 3ef45c0b-b2a0-43da-a1d3-c4f726097eda
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype needs_recovery extent 64bit flex_bg sparse_super large_file
huge_file dir_nlink extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 13107200
Block count: 52428800
Reserved block count: 2621440
Free blocks: 27184765
Free inodes: 12049129
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64
Reserved GDT blocks: 1024
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Flex block group size: 16
Filesystem created: Thu Dec 6 16:17:23 2018
Last mount time: Wed Apr 3 10:19:05 2019
Last write time: Wed Apr 3 10:19:05 2019
Mount count: 12
Maximum mount count: -1
Last checked: Thu Dec 6 16:17:23 2018
Check interval: 0 (<none>)
Lifetime writes: 1962 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
First orphan inode: 5248434
Default directory hash: half_md4
Directory Hash Seed: d1bbea0e-dd2e-4df7-b7f0-f7300c524cc9
Journal backup: inode blocks
Checksum type: crc32c
Checksum: 0x3a8a8676
I'm definitely seeing some differences in the two configs (such as your
block size of 1k vs. mine at 4k), but not sure which are the most
important, nor how to easily recreate a setup that matches yours.
> IIRC I talked to stefanha on IRC about this some weeks ago already, and
> he was able to reproduce the problem when using a certain parameter to
> create the file system. However, I fail to remember which parameter it
> was. Maybe Stefan still remembers...
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
2019-04-28 15:18 ` Thomas Huth
(?)
(?)
@ 2019-05-10 13:54 ` Max Reitz
2019-05-10 16:42 ` Thomas Huth
-1 siblings, 1 reply; 18+ messages in thread
From: Max Reitz @ 2019-05-10 13:54 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Nir Soffer
[-- Attachment #1: Type: text/plain, Size: 2211 bytes --]
On 28.04.19 17:18, Thomas Huth wrote:
> QEMU iotest 175 is failing for me when I run it with -raw:
>
> $ ./check -raw 175
> QEMU --
> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
> -nodefaults -machine accel=qtest
> QEMU_IMG --
> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-img"
> QEMU_IO --
> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-io" --cache
> writeback -f raw
> QEMU_NBD --
> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-nbd"
> IMGFMT -- raw
> IMGPROTO -- file
> PLATFORM -- Linux/x86_64 thuth 3.10.0-957.10.1.el7.x86_64
> TEST_DIR -- /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch
> SOCKET_SCM_HELPER --
> /home/thuth/tmp/qemu-build/tests/qemu-iotests/socket_scm_helper
>
> 175 - output mismatch (see 175.out.bad)
> --- /home/thuth/devel/qemu/tests/qemu-iotests/175.out 2019-04-23
> 16:43:12.000000000 +0200
> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/175.out.bad 2019-04-28
> 17:17:32.000000000 +0200
> @@ -2,17 +2,17 @@
>
> == creating image with default preallocation ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> -size=1048576, blocks=0
> +size=1048576, blocks=2
>
> == creating image with preallocation off ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
> -size=1048576, blocks=0
> +size=1048576, blocks=2
>
> == creating image with preallocation full ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
> -size=1048576, blocks=2048
> +size=1048576, blocks=2050
>
> == creating image with preallocation falloc ==
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
> preallocation=falloc
> -size=1048576, blocks=2048
> +size=1048576, blocks=2050
> *** done
> Failures: 175
> Failed 1 of 1 tests
>
> Any ideas how to fix this?
Hm. What output does
$ touch foo
$ stat -c "size=%s, blocks=%b" foo
$ truncate -s 1M foo
$ stat -c "size=%s, blocks=%b" foo
give for you?
If any of that returns blocks=2, we can probably just use that operation
to fix the result, then.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
2019-05-10 13:54 ` Max Reitz
@ 2019-05-10 16:42 ` Thomas Huth
2019-05-10 17:39 ` Max Reitz
0 siblings, 1 reply; 18+ messages in thread
From: Thomas Huth @ 2019-05-10 16:42 UTC (permalink / raw)
To: Max Reitz, Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Nir Soffer
[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]
On 10/05/2019 15.54, Max Reitz wrote:
> On 28.04.19 17:18, Thomas Huth wrote:
>> QEMU iotest 175 is failing for me when I run it with -raw:
>>
>> $ ./check -raw 175
>> QEMU --
>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
>> -nodefaults -machine accel=qtest
>> QEMU_IMG --
>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-img"
>> QEMU_IO --
>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-io" --cache
>> writeback -f raw
>> QEMU_NBD --
>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-nbd"
>> IMGFMT -- raw
>> IMGPROTO -- file
>> PLATFORM -- Linux/x86_64 thuth 3.10.0-957.10.1.el7.x86_64
>> TEST_DIR -- /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch
>> SOCKET_SCM_HELPER --
>> /home/thuth/tmp/qemu-build/tests/qemu-iotests/socket_scm_helper
>>
>> 175 - output mismatch (see 175.out.bad)
>> --- /home/thuth/devel/qemu/tests/qemu-iotests/175.out 2019-04-23
>> 16:43:12.000000000 +0200
>> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/175.out.bad 2019-04-28
>> 17:17:32.000000000 +0200
>> @@ -2,17 +2,17 @@
>>
>> == creating image with default preallocation ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>> -size=1048576, blocks=0
>> +size=1048576, blocks=2
>>
>> == creating image with preallocation off ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
>> -size=1048576, blocks=0
>> +size=1048576, blocks=2
>>
>> == creating image with preallocation full ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
>> -size=1048576, blocks=2048
>> +size=1048576, blocks=2050
>>
>> == creating image with preallocation falloc ==
>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>> preallocation=falloc
>> -size=1048576, blocks=2048
>> +size=1048576, blocks=2050
>> *** done
>> Failures: 175
>> Failed 1 of 1 tests
>>
>> Any ideas how to fix this?
>
> Hm. What output does
>
> $ touch foo
> $ stat -c "size=%s, blocks=%b" foo
> $ truncate -s 1M foo
> $ stat -c "size=%s, blocks=%b" foo
>
> give for you?
>
> If any of that returns blocks=2, we can probably just use that operation
> to fix the result, then.
$ stat -c "size=%s, blocks=%b" foo
size=0, blocks=2
$ truncate -s 1M foo
$ stat -c "size=%s, blocks=%b" foo
size=1048576, blocks=2
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
2019-05-10 16:42 ` Thomas Huth
@ 2019-05-10 17:39 ` Max Reitz
0 siblings, 0 replies; 18+ messages in thread
From: Max Reitz @ 2019-05-10 17:39 UTC (permalink / raw)
To: Thomas Huth, Qemu-block, QEMU Developers; +Cc: Kevin Wolf, Nir Soffer
[-- Attachment #1: Type: text/plain, Size: 2613 bytes --]
On 10.05.19 18:42, Thomas Huth wrote:
> On 10/05/2019 15.54, Max Reitz wrote:
>> On 28.04.19 17:18, Thomas Huth wrote:
>>> QEMU iotest 175 is failing for me when I run it with -raw:
>>>
>>> $ ./check -raw 175
>>> QEMU --
>>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
>>> -nodefaults -machine accel=qtest
>>> QEMU_IMG --
>>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-img"
>>> QEMU_IO --
>>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-io" --cache
>>> writeback -f raw
>>> QEMU_NBD --
>>> "/home/thuth/tmp/qemu-build/tests/qemu-iotests/../../qemu-nbd"
>>> IMGFMT -- raw
>>> IMGPROTO -- file
>>> PLATFORM -- Linux/x86_64 thuth 3.10.0-957.10.1.el7.x86_64
>>> TEST_DIR -- /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch
>>> SOCKET_SCM_HELPER --
>>> /home/thuth/tmp/qemu-build/tests/qemu-iotests/socket_scm_helper
>>>
>>> 175 - output mismatch (see 175.out.bad)
>>> --- /home/thuth/devel/qemu/tests/qemu-iotests/175.out 2019-04-23
>>> 16:43:12.000000000 +0200
>>> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/175.out.bad 2019-04-28
>>> 17:17:32.000000000 +0200
>>> @@ -2,17 +2,17 @@
>>>
>>> == creating image with default preallocation ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>>> -size=1048576, blocks=0
>>> +size=1048576, blocks=2
>>>
>>> == creating image with preallocation off ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
>>> -size=1048576, blocks=0
>>> +size=1048576, blocks=2
>>>
>>> == creating image with preallocation full ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
>>> -size=1048576, blocks=2048
>>> +size=1048576, blocks=2050
>>>
>>> == creating image with preallocation falloc ==
>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>>> preallocation=falloc
>>> -size=1048576, blocks=2048
>>> +size=1048576, blocks=2050
>>> *** done
>>> Failures: 175
>>> Failed 1 of 1 tests
>>>
>>> Any ideas how to fix this?
>>
>> Hm. What output does
>>
>> $ touch foo
>> $ stat -c "size=%s, blocks=%b" foo
>> $ truncate -s 1M foo
>> $ stat -c "size=%s, blocks=%b" foo
>>
>> give for you?
>>
>> If any of that returns blocks=2, we can probably just use that operation
>> to fix the result, then.
>
> $ stat -c "size=%s, blocks=%b" foo
> size=0, blocks=2
> $ truncate -s 1M foo
> $ stat -c "size=%s, blocks=%b" foo
> size=1048576, blocks=2
Thanks, that should be useful, then.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [Qemu-block] Failing QEMU iotest 175
2019-05-03 21:31 ` Nir Soffer
(?)
@ 2019-05-10 21:15 ` Nir Soffer
-1 siblings, 0 replies; 18+ messages in thread
From: Nir Soffer @ 2019-05-10 21:15 UTC (permalink / raw)
To: Nir Soffer
Cc: Kevin Wolf, Thomas Huth, qemu-block, QEMU Developers, Max Reitz
On Sat, May 4, 2019 at 12:32 AM Nir Soffer <nirsof@gmail.com> wrote:
>
>
> On Fri, May 3, 2019, 23:21 Eric Blake <eblake@redhat.com> wrote:
>
>> ...
>> >>> == creating image with preallocation off ==
>> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>> preallocation=off
>> >>> -size=1048576, blocks=0
>> >>> +size=1048576, blocks=2
>> >>>
>> >>> == creating image with preallocation full ==
>> >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
>> preallocation=full
>> >>> -size=1048576, blocks=2048
>> >>> +size=1048576, blocks=2050
>> >>
>> >> 2048/2050, so we DO have some indication of whether the file is sparse
>> >> or fully allocated.
>> >
>> > Maybe we could check that the value after "blocks=" is a single digit in
>> > the first case, and matches "blocks=20.." in the second case?
>>
>> I wonder if 'qemu-img map --output=json $TEST_IMG' might be any more
>> reliable (at least for ignoring any extra block allocations associated
>> with the file, if it is some journaling option or xattr or other reason
>> why your files seem to occupy more disk sectors than just the size of
>> the file would imply).
>>
>
> I think it should work better and is more correct, testing actual
> sparsness instead of underlying file system implementation.
>
> I can send a fix next week.
>
I tested this change:
$ git diff
diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index d0ffc495c2..0e3faa50e4 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -43,17 +43,17 @@ _supported_os Linux
size=1m
echo
echo "== creating image with default preallocation =="
_make_test_img $size | _filter_imgfmt
-stat -c "size=%s, blocks=%b" $TEST_IMG
+$QEMU_IMG map -f $IMGFMT --output json "$TEST_IMG"
for mode in off full falloc; do
echo
echo "== creating image with preallocation $mode =="
IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
- stat -c "size=%s, blocks=%b" $TEST_IMG
+ $QEMU_IMG map -f $IMGFMT --output json "$TEST_IMG"
done
# success, all done
echo "*** done"
rm -f $seq.full
It almost works:
$ ./check -raw 175
QEMU --
"/home/nsoffer/src/qemu/build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
-nodefaults -machine accel=qtest
QEMU_IMG --
"/home/nsoffer/src/qemu/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO --
"/home/nsoffer/src/qemu/build/tests/qemu-iotests/../../qemu-io" --cache
writeback -f raw
QEMU_NBD --
"/home/nsoffer/src/qemu/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- raw
IMGPROTO -- file
PLATFORM -- Linux/x86_64 lean 5.0.11-100.fc28.x86_64
TEST_DIR -- /home/nsoffer/src/qemu/build/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER --
/home/nsoffer/src/qemu/build/tests/qemu-iotests/socket_scm_helper
175 - output mismatch (see 175.out.bad)
--- /home/nsoffer/src/qemu/tests/qemu-iotests/175.out 2019-03-23
18:35:17.788177871 +0200
+++ /home/nsoffer/src/qemu/build/tests/qemu-iotests/175.out.bad 2019-05-11
00:06:09.515873624 +0300
@@ -2,17 +2,17 @@
== creating image with default preallocation ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, blocks=0
+[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false,
"offset": 0}]
== creating image with preallocation off ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, blocks=0
+[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false,
"offset": 0}]
== creating image with preallocation full ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, blocks=2048
+[{ "start": 0, "length": 1048576, "depth": 0, "zero": false, "data": true,
"offset": 0}]
== creating image with preallocation falloc ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
preallocation=falloc
-size=1048576, blocks=2048
+[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false,
"offset": 0}]
The "falloc" test looks exactly like "off", qemu-img map does not report
the allocation
status.
Nir
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] Failing QEMU iotest 175
2019-05-06 17:44 ` Eric Blake
@ 2019-05-15 14:56 ` Stefan Hajnoczi
0 siblings, 0 replies; 18+ messages in thread
From: Stefan Hajnoczi @ 2019-05-15 14:56 UTC (permalink / raw)
To: Eric Blake
Cc: Kevin Wolf, Thomas Huth, Qemu-block, Nir Soffer, QEMU Developers,
Max Reitz
[-- Attachment #1: Type: text/plain, Size: 4897 bytes --]
On Mon, May 06, 2019 at 12:44:56PM -0500, Eric Blake wrote:
> On 5/4/19 1:51 AM, Thomas Huth wrote:
>
> >> Hmm, it's passing for me on ext4, but that probably means we have
> >> different configuration parameters. I'm not sure how to easily show what
> >> parameters a particular ext4 partition uses to compare the differences
> >> between your setup and mine (mine is tuned to whatever defaults Fedora's
> >> installer chose on my behalf), so maybe someone else can chime in.
> >
> > $ sudo tune2fs -l /dev/mapper/Home
> > tune2fs 1.42.9 (28-Dec-2013)
> > Filesystem volume name: <none>
> > Last mounted on: /home
> > Filesystem magic number: 0xEF53
> > Filesystem revision #: 1 (dynamic)
> > Filesystem features: has_journal ext_attr dir_index filetype
> > needs_recovery meta_bg extent 64bit flex_bg sparse_super large_file
> > huge_file uninit_bg dir_nlink extra_isize
> > Filesystem flags: signed_directory_hash
> > Default mount options: user_xattr acl
> > Filesystem state: clean
> > Errors behavior: Continue
> > Filesystem OS type: Linux
> > Inode count: 36700160
> > Block count: 146800640
> > Reserved block count: 5873663
> > Free blocks: 56266267
> > Free inodes: 35403275
> > First block: 1
> > Block size: 1024
> > Fragment size: 1024
> > Group descriptor size: 64
> > Blocks per group: 8192
> > Fragments per group: 8192
> > Inodes per group: 2048
> > Inode blocks per group: 256
> > First meta block group: 258
> > Flex block group size: 16
> > Filesystem created: Thu Apr 19 18:34:33 2018
> > Last mount time: Sat May 4 08:20:36 2019
> > Last write time: Sat May 4 08:20:36 2019
> > Mount count: 224
> > Maximum mount count: -1
> > Last checked: Thu Apr 19 18:34:33 2018
> > Check interval: 0 (<none>)
> > Lifetime writes: 1826 GB
> > Reserved blocks uid: 0 (user root)
> > Reserved blocks gid: 0 (group root)
> > First inode: 11
> > Inode size: 128
> > Journal inode: 8
> > First orphan inode: 11076944
> > Default directory hash: half_md4
> > Directory Hash Seed: 08e1be04-c3a3-4c37-a059-cf54af5c4bc0
> > Journal backup: inode blocks
> >
>
> # tune2fs -l /dev/mapper/fedora-home
> tune2fs 1.44.6 (5-Mar-2019)
> Filesystem volume name: home
> Last mounted on: /home
> Filesystem UUID: 3ef45c0b-b2a0-43da-a1d3-c4f726097eda
> Filesystem magic number: 0xEF53
> Filesystem revision #: 1 (dynamic)
> Filesystem features: has_journal ext_attr resize_inode dir_index
> filetype needs_recovery extent 64bit flex_bg sparse_super large_file
> huge_file dir_nlink extra_isize metadata_csum
> Filesystem flags: signed_directory_hash
> Default mount options: user_xattr acl
> Filesystem state: clean
> Errors behavior: Continue
> Filesystem OS type: Linux
> Inode count: 13107200
> Block count: 52428800
> Reserved block count: 2621440
> Free blocks: 27184765
> Free inodes: 12049129
> First block: 0
> Block size: 4096
> Fragment size: 4096
> Group descriptor size: 64
> Reserved GDT blocks: 1024
> Blocks per group: 32768
> Fragments per group: 32768
> Inodes per group: 8192
> Inode blocks per group: 512
> Flex block group size: 16
> Filesystem created: Thu Dec 6 16:17:23 2018
> Last mount time: Wed Apr 3 10:19:05 2019
> Last write time: Wed Apr 3 10:19:05 2019
> Mount count: 12
> Maximum mount count: -1
> Last checked: Thu Dec 6 16:17:23 2018
> Check interval: 0 (<none>)
> Lifetime writes: 1962 GB
> Reserved blocks uid: 0 (user root)
> Reserved blocks gid: 0 (group root)
> First inode: 11
> Inode size: 256
> Required extra isize: 32
> Desired extra isize: 32
> Journal inode: 8
> First orphan inode: 5248434
> Default directory hash: half_md4
> Directory Hash Seed: d1bbea0e-dd2e-4df7-b7f0-f7300c524cc9
> Journal backup: inode blocks
> Checksum type: crc32c
> Checksum: 0x3a8a8676
>
> I'm definitely seeing some differences in the two configs (such as your
> block size of 1k vs. mine at 4k), but not sure which are the most
> important, nor how to easily recreate a setup that matches yours.
Yes, previously when we had similar issues it was the block size that
caused the difference. It's worth trying it out with a test file system
on a loop device.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2019-05-15 15:06 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-28 15:18 [Qemu-devel] Failing QEMU iotest 175 Thomas Huth
2019-04-28 15:18 ` Thomas Huth
2019-05-02 21:56 ` Eric Blake
2019-05-02 21:56 ` Eric Blake
2019-05-03 4:37 ` Thomas Huth
2019-05-03 4:37 ` Thomas Huth
2019-05-03 20:21 ` Eric Blake
2019-05-03 20:21 ` Eric Blake
2019-05-03 21:31 ` Nir Soffer
2019-05-03 21:31 ` Nir Soffer
2019-05-10 21:15 ` [Qemu-devel] [Qemu-block] " Nir Soffer
2019-05-04 6:51 ` [Qemu-devel] " Thomas Huth
2019-05-04 6:51 ` Thomas Huth
2019-05-06 17:44 ` Eric Blake
2019-05-15 14:56 ` Stefan Hajnoczi
2019-05-10 13:54 ` Max Reitz
2019-05-10 16:42 ` Thomas Huth
2019-05-10 17:39 ` Max Reitz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.