Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again

[Max Reitz <mreitz@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3623 bytes --]

On 10.05.19 18:20, Thomas Huth wrote:
> On 10/05/2019 15.47, Max Reitz wrote:
>> On 10.05.19 15:36, Thomas Huth wrote:
>>> On 10/05/2019 15.34, Max Reitz wrote:
>>>> On 10.05.19 06:29, Thomas Huth wrote:
>>>>> On 09/05/2019 20.08, Max Reitz wrote:
>>>>>> On 02.05.19 10:45, Thomas Huth wrote:
>>>>>>> People often forget to run the iotests before submitting patches or
>>>>>>> pull requests - this is likely due to the fact that we do not run the
>>>>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>>>>> a proper "auto" group of iotests that should be fine to run in every
>>>>>>> environment, we can enable the iotests during "make check" again by
>>>>>>> running the "auto" tests by default from the check-block.sh script.
>>>>>>>
>>>>>>> Some cases still need to be checked first, though: iotests need bash
>>>>>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>>>>>> the output of some test cases causing them to fail. So if we detect
>>>>>>> that one of the required programs is missing or that gprof is enabled,
>>>>>>> we still have to skip the iotests to avoid failures.
>>>>>>>
>>>>>>> And finally, since we are using check-block.sh now again, this patch also
>>>>>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>>>>>> (and having two shell wrapper scripts around the block tests seem
>>>>>>> rather confusing than helpful).
>>>>>>>
>>>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>>>> ---
>>>>>>>  tests/Makefile.include      |  8 +++----
>>>>>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>>>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>>>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>>>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>>>>>
>>>>>> Can I interest you in a Makefile target that explicitly excludes
>>>>>> check-block?  I run the iotests anyway, but I also run make check.
>>>>>> Running some iotests twice would be a bit pointless.
>>>>>
>>>>> Can't you simply run
>>>>>
>>>>>  ./check -qcow2 -x auto
>>>>>
>>>>> instead?
>>>>
>>>> I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
>>>> qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.
>>>>
>>>> So for which of the protocol/format combinations do I exclude the auto
>>>> group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
>>>> may that not be subject to change?
>>>
>>> With my patch series, the auto group is only used for qcow2.
>>
>> And that is not subject to change?  Like, maybe someone wants to add nbd
>> in the future?
> 
> The current set of qcow2 auto tests takes already quite a while, so I
> don't think that this will change soon.
> And if the "normal" users want to run more tests, they can simply use
> "make check SPEED=slow" or SPEED=thorough, so IMHO no need to extend the
> quick default list right now.
> 
>> I mean, I have a test branch anyway which collects a number of patches
>> on top of master that make everything pass more or less reliably for me
>> (11 patches currently...).  I suppose I can just revert your patch on
>> top of that.  But that doesn’t feel right.
> 
> Hmm, sure, non-upstream patches are always a bad thing. But I still
> don't see why you really need an extra Makefile target for this. In the
> worst case, you could also simply change your script to run something
> this instead:
> 
>  make $(grep ^check: tests/Makefile.include | sed s/check-block//)
> 
> ?

Hm.  I didn’t think of that.  Thanks. :-)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]