Re: [PATCH 0/5] tools/tests: More cleanup for automation improvements

From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "Ian Jackson" <iwj@xenproject.org>, "Wei Liu" <wl@xen.org>,
	"Roger Pau Monné" <roger.pau@citrix.com>,
	"Juergen Gross" <jgross@suse.com>,
	Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 0/5] tools/tests: More cleanup for automation improvements
Date: Wed, 16 Jun 2021 16:14:15 +0200	[thread overview]
Message-ID: <5f44c856-635c-cf46-2504-75451b1abac4@suse.com> (raw)
In-Reply-To: <5f786fa7-80eb-ea66-3fbc-2b556c1d3a73@citrix.com>

On 16.06.2021 15:22, Andrew Cooper wrote:
> On 16/06/2021 07:38, Jan Beulich wrote:
>> On 15.06.2021 18:19, Andrew Cooper wrote:
>>> Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC).  While they are unit
>>> tests, we still potentially want to run them in dom0 rather than the build
>>> environment - particularly for x86_emulator which is heavily CPUID based and
>>> wants to run on a wide set of hardware.  Any issues moving them off $(HOSTCC)?
>> Well, yes, I'm afraid: If anything, we may need to build two binaries,
>> or build the one binary two different ways: The "run" (and "run32" for
>> the emulator harness) target wants a binary built with HOSTCC. The
>> install target (which prior to your series does nothing) indeed wants
>> building with CC. So maybe we want something like
>>
>> install: HOSTCC:=$(CC)
>>
>> plus suitable detection of whether the opposite set of objects are
>> presently in the build area, requiring a full rebuild? (Of course this
>> will work only as long as HOSTCC isn't used for any build time helper
>> binaries. See "x86emul: test AMX insns" for when this starts not to be
>> the case anymore for the emulator harness. So we'd need yet another
>> variable to express this detail.)
> 
> Having slept on the problem overnight, I'm going to argue that HOSTCC is
> conceptually wrong to use here in the first place.
> 
> In an arm64 environment, cross-compiling x86_64, this will explode
> everywhere, and the fault is with using HOSTCC rather than CC.

In principle, if there wasn't the massive amount of inline assembly,
and if the emulator wasn't just re-executing the instructions it is
asked to emulate, building all of this on Arm ought to be possible.
But with the code we have this simply makes no sense.

> HOSTCC is specifically for compiling utilities executed as part of the
> build.  Tests, and particularly arch-specific ones like x86_emulate, are
> not in this category.

Hmm, right now they definitely are. Running them directly (which is
their only purpose right now, with the install targets doing nothing)
from the build tree puts them into this category. But they aren't
anymore as soon as you want to install them. Hence the need to have
two modes here.

>  Whether you happen to be able to run
> test_x86_emulator in the build environment is a property of whether
> you're cross-compiling.

In a way, yes. I'd consider the run32 target to also be cross-like,
yet that binary can then still be run from the build tree (if the
distro supports 32-bit binaries).

> For a non-cross-compiled builds, HOSTCC and CC are largely
> interchangeable, and won't impact the ability to run the binary in the
> build environment.

Not exactly, I think. If I override CC but not HOSTCC (which I do
normally), I still don't need to worry about that other CC finding
all the bits and pieces it needs for building a host binary. For the
C compiler this may not mean much, but the C++ compiler (if we used
it and hence would need to treat it similarly) wants e.g. its own
copies of the headers, which may not be readily available. I will
admit that my environment may be pretty non-standard, as I run non-
default compilers (and binutils) also directly from their build
trees. But things have been working fine this way for all sorts of
projects, so I expect the test harness here to not break in this
regard.

Jan