Re: [PATCH v8 0/5] Rework iotests/check - Vladimir Sementsov-Ogievskiy

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: Kevin Wolf <kwolf@redhat.com>
Cc: qemu-block@nongnu.org, jsnow@redhat.com, qemu-devel@nongnu.org,
	mreitz@redhat.com, den@openvz.org
Subject: Re: [PATCH v8 0/5] Rework iotests/check
Date: Mon, 25 Jan 2021 19:23:35 +0300	[thread overview]
Message-ID: <fbca8b6f-6a9a-bdcc-c4da-bc9343149031@virtuozzo.com> (raw)
In-Reply-To: <20210125160820.GD7107@merkur.fritz.box>

25.01.2021 19:08, Kevin Wolf wrote:
> Am 23.01.2021 um 22:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> v8:
>>
>> about linters:
>>
>> I didn't modify 297, as Max already staged 297 modifications to test all files.
>>
>> Also, now I have two complains:
>> +************* Module testenv
>> +testenv.py:158:4: R0915: Too many statements (53/50) (too-many-statements)
>> +************* Module testrunner
>> +testrunner.py:222:4: R0911: Too many return statements (7/6) (too-many-return-statements)
>>   Success: no issues found in 5 source files
>>
>> And I feel, I'm tired to refactor it now.. Probably we can ignore them in 297. Probably I can
>> do some refactoring as a follow-up.
> 
> I don't think these warning are very helpful, I would agree with
> disabling them (even globally).
> 
> When testing this with the other image formats, I found some problems.
> 
> 1. The first one probably means that we have changed the order of some
>     checks: 150 and 178 have reference outputs for raw and qcow2, but no
>     other formats.
> 
>     Previously, the _supported_fmt line in the test would just skip the test:
> 
>     $ build/check -vhdx 150 178
>     150      not run    [16:45:46] [16:45:46]                    not suitable for this image format: vhdx
>     178      not run    [16:45:46] [16:45:46]                    not suitable for this image format: vhdx
> 
>     Now we seem to test first if a reference output exists and fail:
> 
>     150   fail       [16:49:18] [16:49:18]   ...                  No qualified output (expected /home/kwolf/source/qemu/tests/qemu-iotests/150.out)
>     178   fail       [16:49:18] [16:49:18]   ...                  No qualified output (expected /home/kwolf/source/qemu/tests/qemu-iotests/178.out)
> 
> 2. Test case 146 for vpc passed previously, it fails now. This seems to
>     be because of whitespace that is checked now.
> 
> 3. Skipped tests display either "..." or "0.1s" as the elapsed time. The
>     old check implementation didn't display any time for them. I don't
>     really mind either of the three ways, but a consistent result would
>     be nice.
> 

Thanks for testing! I should have done it myself actually :/

Will fix soon.

-- 
Best regards,
Vladimir