Re: [PATCH v4 4/4] meson: Warn when TCI is selected but TCG backend is available

From: "Alex Bennée" <alex.bennee@linaro.org>
To: "Daniel P. Berrangé" <berrange@redhat.com>
Cc: "Thomas Huth" <thuth@redhat.com>, "Stefan Weil" <sw@weilnetz.de>,
	"Richard Henderson" <richard.henderson@linaro.org>,
	qemu-devel@nongnu.org, "Paolo Bonzini" <pbonzini@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>
Subject: Re: [PATCH v4 4/4] meson: Warn when TCI is selected but TCG backend is available
Date: Wed, 27 Jan 2021 12:34:48 +0000	[thread overview]
Message-ID: <87zh0uwnyz.fsf@linaro.org> (raw)
In-Reply-To: <20210127100221.GD3653144@redhat.com>

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Tue, Jan 26, 2021 at 05:24:10PM +0000, Alex Bennée wrote:
>> 
>> Stefan Weil <sw@weilnetz.de> writes:
>> 
>> > Am 25.01.21 um 23:35 schrieb Richard Henderson:
>> >> On 1/25/21 11:02 AM, Stefan Weil wrote:
>> >>> Am 25.01.21 um 20:02 schrieb Richard Henderson:
>> >>>> On 1/25/21 8:58 AM, Stefan Weil wrote:
>> >>>>> I have no evidence that TCI is less reliable than TCG, so I would not write
>> >>>>> that.
>> >>>> It can't pass make check-tcg.
>> >>> Where does it fail? Maybe an expected timeout problem which can be solved by
>> >>> increasing the timeouts for TCI?
>> >>>
>> >>> I have just run a local test of `make check-tcg` with native TCG and with TCI
>> >>> and did not see a difference. But I noticed that in both cases many tests show
>> >>> "skipped".
>> >> You need to enable docker or podman for your development, so that you get all
>> >> of the cross-compilers.
>> >>
>> >> Then:
>> >>
>> >>    TEST    fcvt on arm
>> >> TODO ../qemu/tcg/tci.c:614: tcg_qemu_tb_exec()
>> >> ../qemu/tcg/tci.c:614: tcg fatal error
>> >> qemu: uncaught target signal 11 (Segmentation fault) - core dumped
>> >>
>> >>    TEST    float_convs on m68k
>> >> TODO ../qemu/tcg/tci.c:614: tcg_qemu_tb_exec()
>> >> ../qemu/tcg/tci.c:614: tcg fatal error
>> >> qemu: uncaught target signal 11 (Segmentation fault) - core dumped
>> >>
>> >> which is of course one of the TODO assertions.
>> >> It's positively criminal those still exist in the code.
>> >
>> >
>> > I installed podman and repeated `make check-tcg`. The log file still 
>> > shows 87 lines with "SKIPPED". There is also a gdb core dump, several 
>> > warnings, but nothing related to TCI. Both tests cited above seem to 
>> > work without a problem.
>> 
>> I'm going to go out on a limb and guess you have commit:
>> 
>>   23a77b2d18 (build-system: clean up TCG/TCI configury)
>> 
>> which temporarily has the effect of disabling TCI. See
>> 
>>   Subject: Re: [PATCH v4 1/4] configure: Fix --enable-tcg-interpreter
>>   From: Paolo Bonzini <pbonzini@redhat.com>
>>   Message-ID: <2b8b6291-b54c-b285-ae38-21f067a8497d@redhat.com>
>>   Date: Mon, 25 Jan 2021 17:36:42 +0100
>> 
>> with that fix fixed I see the same failures as Richard:
>> 
>>   ./qemu-arm ./tests/tcg/arm-linux-user/fcvt > /dev/null
>>   TODO ../../tcg/tci.c:614: tcg_qemu_tb_exec()
>>   ../../tcg/tci.c:614: tcg fatal error
>>   qemu: uncaught target signal 11 (Segmentation fault) - core dumped
>>   fish: “./qemu-arm ./tests/tcg/arm-linu…” terminated by signal SIGSEGV (Address boundary error)
>> 
>> which does raise the question before today when was the last time anyone
>> attempted to run check-tcg on this?
>> 
>> > The complete log file is available from 
>> > https://qemu.weilnetz.de/test/check-tcg.txt.
>> >
>> > Daniel, regarding your comment: TCI has 100 % test coverage for the 
>> > productive code lines.
>> 
>> By what tests? The fact you don't hit asserts in your day to day testing
>> doesn't mean there are features missing that are easily tripped up or
>> that TCI got it right.
>> 
>> > All code lines which were never tested raise an 
>> > assertion, so can easily be identified (and fixed as soon as there is a 
>> > test case which triggers such an assertion). The known deficits are 
>> > speed, missing TCG opcodes, unimplemented TCG opcodes because of missing 
>> > test cases and missing support for some host architectures.
>> 
>> Passing check-tcg would be a minimum for me.
>
> Passing check-tcg *in gitlab CI* would be the minimum to consider
> it on a par with TCG.
>
> The lack of automated GitLab CI for TCI is a reason my proposed wording
> described TCI as less reliable than native TCG. We can't claim it has
> equivalent reliability unless we have equiv automated testing of TCI.

I should point out that check-tcg is hardly a comprehensive test suite.
Most of our instruction testing for example tends to be done with RISU.
Any program that attempts to use vector instructions is likely to come a
cropper with TCI. I guess we don't even attempt to run check-acceptance
due to speed issues but it would be interesting to see how far it gets.

>
> Regards,
> Daniel

-- 
Alex Bennée