Re: [PULL 0/2] block: Fix iotests to respect configured Python binary - Vladimir Sementsov-Ogievskiy

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: Kevin Wolf <kwolf@redhat.com>, Peter Maydell <peter.maydell@linaro.org>
Cc: QEMU Developers <qemu-devel@nongnu.org>,
	Qemu-block <qemu-block@nongnu.org>
Subject: Re: [PULL 0/2] block: Fix iotests to respect configured Python binary
Date: Mon, 1 Feb 2021 11:36:53 +0300	[thread overview]
Message-ID: <5ef46395-f370-2de1-4ea4-9aa997964756@virtuozzo.com> (raw)
In-Reply-To: <20210129171728.GJ10888@merkur.fritz.box>

29.01.2021 20:17, Kevin Wolf wrote:
> Am 29.01.2021 um 17:13 hat Peter Maydell geschrieben:
>> On Fri, 29 Jan 2021 at 14:52, Kevin Wolf <kwolf@redhat.com> wrote:
>>>
>>> The following changes since commit 5101d00d2f1138a73344dc4833587f76d7a5fa5c:
>>>
>>>    Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-6.0-pull-request' into staging (2021-01-29 10:10:43 +0000)
>>>
>>> are available in the Git repository at:
>>>
>>>    git://repo.or.cz/qemu/kevin.git tags/for-upstream
>>>
>>> for you to fetch changes up to 4cea90be62f4f15a63e1a8f7d5d0958f79fdf290:
>>>
>>>    tests/Makefile.include: export PYTHON for check-block.sh (2021-01-29 12:32:36 +0100)
>>>
>>> ----------------------------------------------------------------
>>> Block layer patches:
>>>
>>> - Fix iotests to respect configured Python binary
>>>
>>> ----------------------------------------------------------------
>>
>> This is definitely better so I'm going to apply it, but it seems
>> to reveal a pile of iotest failures on FreeBSD:
>> [...]
>> Failures: 030 040 041 127 256
> 
> Hm... I did run 'make check-block' on a FreeBSD VM before sending the
> pull request, but it turns out I ran it on the wrong branch. *sigh*
> 
> After rerunning it with the right one (and manually so that I get all
> tests, not just those in 'make check-block'), however, while I'm seeing
> some failures, my list of failing cases and your list are completely
> disjoint:
> 
>      Failures: 115 125 145 182 286 298 300 migrate-bitmaps-postcopy-test
>      migrate-bitmaps-test
> 
> All of these are test cases that are not run during 'make check-block',
> so I would assume that they were already broken before.
> 
> So there seems to be something more specific to the environment that
> made your test cases fail than just FreeBSD.
> 
>> though the entire build goes on to succeed (which is probably
>> a test framework bug somewhere...)
> 
> This is not good, and it can easily be reproduced on Linux by simply
> changing the reference output of a test so that it fails. 'check'
> doesn't seem to return the right exit code any more.
> 

That's bad. I'll make a patch.

-- 
Best regards,
Vladimir