RE: [PATCH 01/20] Hexagon HVX (target/hexagon) README

From: Sid Manning <sidneym@quicinc.com>
To: Brian Cain <bcain@quicinc.com>, Rob Landley <rob@landley.net>,
	Taylor Simpson <tsimpson@quicinc.com>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>
Cc: "ale@rev.ng" <ale@rev.ng>,
	"peter.maydell@linaro.org" <peter.maydell@linaro.org>,
	"richard.henderson@linaro.org" <richard.henderson@linaro.org>,
	"philmd@redhat.com" <philmd@redhat.com>
Subject: RE: [PATCH 01/20] Hexagon HVX (target/hexagon) README
Date: Mon, 19 Jul 2021 16:19:37 +0000	[thread overview]
Message-ID: <BYAPR02MB55091DE5B976956075B46FADBEE19@BYAPR02MB5509.namprd02.prod.outlook.com> (raw)
In-Reply-To: <BN7PR02MB4194007550E66EBE877625A6B8E19@BN7PR02MB4194.namprd02.prod.outlook.com>

> -----Original Message-----
> From: Brian Cain <bcain@quicinc.com>
> Sent: Monday, July 19, 2021 8:40 AM
> To: Rob Landley <rob@landley.net>; Taylor Simpson
> <tsimpson@quicinc.com>; qemu-devel@nongnu.org; Sid Manning
> <sidneym@quicinc.com>
> Cc: ale@rev.ng; peter.maydell@linaro.org; richard.henderson@linaro.org;
> philmd@redhat.com
> Subject: RE: [EXT] Re: [PATCH 01/20] Hexagon HVX (target/hexagon)
> README
> 
> 
> 
> > -----Original Message-----
> > From: Rob Landley <rob@landley.net>
> ...
> > On 7/12/21 8:42 AM, Brian Cain wrote:
> ...
> > > and there's also a binary hexagon-linux cross toolchain that we
> > > shared for use by kernel developers.  The hexagon linux toolchain is
> > > built on Ubuntu 16.04.
> >
> > Where's that one?
> 
> https://codelinaro.jfrog.io/artifactory/codelinaro-qemu/2021-05-
> 12/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl.tar.xz -
> 	- Built on Ubuntu 16.04, similar dynamic dependencies as
> releases.llvm.org binaries
> 	- Manifest:
> 		- llvm+clang 12.0.0 tag
> 		- Linux 5.6.18
> 		- github.com/qemu/qemu
> 15106f7dc3290ff3254611f265849a314a93eb0e
> 		- github.com/quic/musl
> aff74b395fbf59cd7e93b3691905aa1af6c0778c
> 
> 
> > > But when building your toolchain, omitting LLVM_ENABLE_LLD should
> > > work
> > just fine.
> >
> > It did, thanks.
> >
> > Now I'm trying to figure out what all the extra CFLAGS are for.
> 
> +Sid for some perspective on the rationale of these flags.  Some of these
> flags may be workarounds for toolchain issues.
> 
> > The clang_rt build has CMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic
> > --target=hexagon-unknown-linux-musl" which
> > https://clang.llvm.org/docs/ClangCommandLineReference.html defines as:
> >
> > -G<size>
> >   Put objects of at most <size> bytes into small data section (MIPS /
> > Hexagon)
> >
> > -mlong-calls
> >   Generate branches with extended addressability, usually via indirect
> jumps.
> >
> > I don't understand why your libcc build needs no-pic? (Are we only
> > building a static libgcc.a instead of a dynamic one? I'm fine with
> > that if so, but this needs to be specified in the MAKE_ASM_FLAGS why?)
> >
> > Why is it saying --target=hexagon-random-nonsense-musl to a toolchain
> > we built with exactly one target type? How does it NOT default to
> hexagon?
> > (Is this related to the build writing a hexagon-potato-banana-musl.cfg
> > file in the bin directory, meaning the config file is in the $PATH?
> > Does clang only look for it in the same directory the clang executable
> > lives in?)
> >
> > And while we're at it, the CONTENTS of hexagon-gratuitous-gnu-format.cfg
> is:
> >
> > cat <<EOF > hexagon-unknown-linux-musl.cfg
> > -G0 --sysroot=${HEX_SYSROOT}
> > EOF
> >
> > Which is ALREADY saying -G0? (Why does it want to do that globally?
> > Some sort of bug workaround?) So why do we specify it again here?
> >
> > Next up build_musl_headers does CROSS_CFLAGS="-G0 -O0 -mv65
> > -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl"
> which:

I agree G0 is overplayed here.  G0 is the implied default on Linux.  On occasion we use a different configuration of clang where small data (G8) is the default so G0 is specified.

> >
> > -O0
> >   disable most of the optimizer

This should be changed.  This was added so I could factor out clang's floating point optimizations.  These optimizations caused musl-libc testsuite to fail some floating point accuracy tests.  I know at least some of those issues have now been resolved.

> >
> > -mv65
> >   -mtune for a specific hexagon generation.
> >   (Why? Does qemu only support that but not newer?)
Passing -mvXX it is now recommended practice.  A few years ago the default arch selected changed from the oldest support arch to the newest arch.  QEMU supports later architectures.

> >
> > -fno-builtin
> >   musl's ./configure already probes for this and will add it if
> >   the compiler supports it.
I did not know this, we can get rid of -fno-builtin if the driver is meeting musl's expectations.

> >
> > -fno-rounding-math
> >   the docs MENTION this, but do not explain it.

This was workaround and is no longer needed.  IIRC clang was asserting while building musl.

> >
> > And again with the -G0.
> >
> > These flags probably aren't needed _here_ because this is just the
> > headers install (which is basically a cp -a isn't it?). This looks
> > like it's copied verbatim from the musl library build. But that
> > library build happens in a bit, so relevant-ish I guess...
> >
> > (Also, why does building librt-but-not-really need the libc headers?
> > The libgcc build EXPLICITLY does not require that, because otherwise
> > you have this kind of BS circular dependency. Also, how do you EVER
> > build a bare metal ELF toolchain with that dependency in there?)

Getting cmake to agree to build compiler-rt might be better now.

> >
> > Next up build_kernel_headers has KBUILD_CFLAGS_KERNEL="-mlong-
> calls"
> > which
> > again, A) why does the compiler not do by default, B) can't be needed
> > here because you don't even have to specify a cross compiler when
> > doing headers_install. (I just confirmed this by diffing installs with
> > an without a cross compiler specified: they were identical. I
> > remembered this from
> > https://github.com/torvalds/linux/commit/e0e2fa4b515c but checked
> > again to be
> > sure.) Presumably this is more "shared with full kernel build".

-mlong-calls are not needed for header install.  -mlong-calls are needed when building the kernel source.  If this is removed the link step may fail with a relocation overflow depending on the version of the kernel source you are building.

> >
> > And then build_musl, covered above under the headers build: lotsa
> > flags, not sure why.
> >
> > > -Brian
> > >
> >
> > Rob
> >
> > P.S. It took me a while to figure out that clang_rt is NOT librt.a, I
> > think it's their libgcc? Especially confusing since librt.a has
> > existed for decades and was on solaris before it was on linux, and the
> > OBVIOUS name is libcc the same way "cc" is the generic compiler name
> instead of "gcc".
> > (In fact that was the posix compiler name until they decided to
> > replace it with "c99" and everybody ignored them the way tar->pax was
> > ignored, largely because make's $CC defaults to "cc" so it Just Works,
> > and yes the cross compiler should have that name but the prepackaged
> > clang tarball above does not. *shrug* I fix that up when making my
> > prefix symlinks. The android NDK guys at least have the excuse of
> > shipping NINE different x86_64-linux-android*-clang with API version
> > numbers and thus not wanting to pick a default to single out, so leave
> > making the -cc link as an exercise for the reader. I give instructions
> > for doing so on the toybox cross compiling page I linked above. :)