Re: [PATCH] rust: Respect HOSTCC when linking for host

From: Matthew Maurer <mmaurer@google.com>
To: Nick Desaulniers <ndesaulniers@google.com>
Cc: "Finn Behrens" <me@kloenk.dev>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Alex Gaynor" <alex.gaynor@gmail.com>,
	"Wedson Almeida Filho" <wedsonaf@gmail.com>,
	"Masahiro Yamada" <masahiroy@kernel.org>,
	"Boqun Feng" <boqun.feng@gmail.com>,
	"Gary Guo" <gary@garyguo.net>,
	"Benno Lossin" <benno.lossin@proton.me>,
	"Andreas Hindborg" <a.hindborg@samsung.com>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Nathan Chancellor" <nathan@kernel.org>,
	"Nicolas Schier" <nicolas@fjasle.eu>,
	rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-kbuild@vger.kernel.org
Subject: Re: [PATCH] rust: Respect HOSTCC when linking for host
Date: Mon, 18 Sep 2023 11:55:41 -0700	[thread overview]
Message-ID: <CAGSQo01jMFVeqa=99Ne7tDXeOShcHWHBPgyiwZ+tm6x9qRJOfg@mail.gmail.com> (raw)
In-Reply-To: <CAKwvOdnbbStoEeykYnx0jbnR=TAmmcdnOXbbA4Fx0BneFW8Fsg@mail.gmail.com>

On Mon, Sep 18, 2023 at 11:43 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Mon, Sep 18, 2023 at 9:38 AM Matthew Maurer <mmaurer@google.com> wrote:
> >
> > On Mon, Sep 18, 2023 at 8:25 AM Nick Desaulniers
> > <ndesaulniers@google.com> wrote:
> > >
> > > What happens if you invoke the linker directly?
> > Rust unfortunately currently doesn't support invoking the linker
> > directly: https://github.com/rust-lang/rust/issues/73632
>
> Wait; does Rust have its own linker? It doesn't use the system linker?
>  Perhaps that's necessary for the rust module format? If so, TIL.
It does use the system linker (this is what -C linker is controlling),
but the command line passed to the linker may change, extra object
files may be added to the command line, etc.
>
> > > Generally, the kernel either invokes the compiler or the linker
> > > directly. (For assembler, it is typically preprocessed, as are linker
> > > scripts!)  So invoking the linker directly is a common pattern in
> > > kbuild.  It also makes me slightly sad if the rust compiler ends up
> > > invoking a c compiler, even if it's simply to drive the linker.
> > As mentioned earlier, we could pass $HOSTLD, but if the linker isn't
> > named something accurate (e.g. if the linker is not named lld, but is
> > lld), we need to know how to pass a flavor:
> > https://doc.rust-lang.org/rustc/codegen-options/index.html#linker-flavor
> > Would it be appropriate to just assume the linker is named correctly?
>
> If it were not, what does failure look like?
That depends. I think it will usually look like "unrecognized flag:
blah blah", but that's not guaranteed.
>
> command not found: asdfadfasdf
This isn't about command-not-found, it's about "I set
HOSTLD=foo/bar/weirdname, and it is an lld-like linker. rustc invoked
it assuming it is an ld-like linker."
>
> Seems fine to me. If the user mis-specifies HOSTLD=, then they will
> get a similar error, which should be prescriptive enough for them to
> figure out how exactly they're "holding it wrong."
>
> > > For example, Android carries a downstream patch to set `-fuse-ld=lld`
> > > for $KBUILD_HOSTCFLAGS, because its build environment doesn't contain
> > > GNU binutils ("guilty, officer").
> > Oddly, the Android kernel environment (Kleaf) is the one that I needed
> > this patch to build in, but it seemed to be working without a manual
> > KBUILD_HOSTCFLAGS forwarding.
>
> Surprising that worked.
>
> > Overall, it sounds like you'd prefer if I set this to use
> > `KBUILD_HOSTLD` and `KBUILD_HOSTLDFLAGS`, and leave the linker flavor
> > to autodetect?
>
> Yes for the first two.
>
> Dunno, what precisely is a linker flavor?  Is that like a flavor of ice cream?
> Oh, right looking at your link:
> https://doc.rust-lang.org/rustc/codegen-options/index.html#linker-flavor
> Seems like if `LLVM=1` is set, or `LD=ld.lld`, or CONFIG_LD_IS_LLD, then
> the flavor should be set to ld.lld, else ld.  Then the
> KBUILD_HOSTLDFLAGS need to be passed, probably.
>
> But how are there "linker flavors" like ld or ld.lld if you just said
> "Rust unfortunately currently doesn't support invoking the linker
> directly: https://github.com/rust-lang/rust/issues/73632".  I'm having
> trouble reconciling the two.
Yes, what I meant by that is that *rustc* wants to invoke the linker,
rather than having the surrounding build system invoke the linker. The
exact command line passed to the linker in the final link, including
potential synthetic objects, is considered an internal detail.
>
> Can we do something more like the below?
>
> ifdef CONFIG_LD_IS_LLD
> hostrust_flags += -C linker-flavor=ld.lld
> else
> hostrust_flags += -C linker-flavor=ld
> endif
> hostrust_flags += -C linker=$(HOSTLD) <todo: figure out how to pass
> KBUILD_HOSTLDFLAGS>
Yes, I can make host linking use `$(HOSTLD)` and pass the flavor based
on CONFIG_LD_IS_LLD. I'll send a variant that does that this
afternoon.
> --
> Thanks,
> ~Nick Desaulniers