linux-kbuild.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2] rust: Respect HOSTCC when linking for host
@ 2023-09-18 23:43 Matthew Maurer
  2023-09-19 19:18 ` Martin Rodriguez Reboredo
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: Matthew Maurer @ 2023-09-18 23:43 UTC (permalink / raw)
  To: Miguel Ojeda, Alex Gaynor, Wedson Almeida Filho, Masahiro Yamada
  Cc: Matthew Maurer, Boqun Feng, Gary Guo, Björn Roy Baron,
	Benno Lossin, Andreas Hindborg, Alice Ryhl, Nathan Chancellor,
	Nick Desaulniers, Nicolas Schier, rust-for-linux, linux-kernel,
	linux-kbuild

Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
resulting in build failures in hermetic environments where `cc` does not
exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to `HOSTCC`, we set the linker flavor to
`gcc` explicitly.

Signed-off-by: Matthew Maurer <mmaurer@google.com>
---

Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS     
should be respected as well.

I did not switch it to use HOSTLD for two reasons:                     
* That variable is not globally defined - it is only available in two   
  subdirectories of tools/                                                                            
* C host scripts are linked by HOSTCC as well, even when linking a                                    
  collection of object files. It *prints* HOSTLD, but invokes HOSTCC.                                 
  See scripts/Makefile.host cmd_host-cmulti for an example.

 rust/Makefile         | 4 ++++
 scripts/Makefile.host | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/rust/Makefile b/rust/Makefile
index 87958e864be0..b60b7eb8c5a0 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
 $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
 	$(call if_changed,exports)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
       cmd_rustc_procmacro = \
 	$(RUSTC_OR_CLIPPY) $(rust_common_flags) \
+		-Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		-Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
 		--emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
 		--crate-type proc-macro \
 		--crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
diff --git a/scripts/Makefile.host b/scripts/Makefile.host
index 8f7f842b54f9..dc0410cae5ca 100644
--- a/scripts/Makefile.host
+++ b/scripts/Makefile.host
@@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
+		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
 
-- 
2.42.0.459.ge4e396fd5e-goog


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] rust: Respect HOSTCC when linking for host
  2023-09-18 23:43 [PATCH v2] rust: Respect HOSTCC when linking for host Matthew Maurer
@ 2023-09-19 19:18 ` Martin Rodriguez Reboredo
  2023-09-19 19:44 ` Nick Desaulniers
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: Martin Rodriguez Reboredo @ 2023-09-19 19:18 UTC (permalink / raw)
  To: Matthew Maurer, Miguel Ojeda, Alex Gaynor, Wedson Almeida Filho,
	Masahiro Yamada
  Cc: Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
	Andreas Hindborg, Alice Ryhl, Nathan Chancellor,
	Nick Desaulniers, Nicolas Schier, rust-for-linux, linux-kernel,
	linux-kbuild

On 9/18/23 20:43, Matthew Maurer wrote:
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
> 
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
> 
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
> ---
> 
> Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> should be respected as well.
> 
> I did not switch it to use HOSTLD for two reasons:
> * That variable is not globally defined - it is only available in two
>    subdirectories of tools/
> * C host scripts are linked by HOSTCC as well, even when linking a
>    collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
>    See scripts/Makefile.host cmd_host-cmulti for an example.
> [...]
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] rust: Respect HOSTCC when linking for host
  2023-09-18 23:43 [PATCH v2] rust: Respect HOSTCC when linking for host Matthew Maurer
  2023-09-19 19:18 ` Martin Rodriguez Reboredo
@ 2023-09-19 19:44 ` Nick Desaulniers
  2023-09-19 20:02   ` Nick Desaulniers
  2023-09-26 10:45 ` Alice Ryhl
  2023-09-26 17:02 ` Masahiro Yamada
  3 siblings, 1 reply; 6+ messages in thread
From: Nick Desaulniers @ 2023-09-19 19:44 UTC (permalink / raw)
  To: Matthew Maurer
  Cc: Miguel Ojeda, Alex Gaynor, Wedson Almeida Filho, Masahiro Yamada,
	Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
	Andreas Hindborg, Alice Ryhl, Nathan Chancellor, Nicolas Schier,
	rust-for-linux, linux-kernel, linux-kbuild

On Mon, Sep 18, 2023 at 4:44 PM Matthew Maurer <mmaurer@google.com> wrote:
>
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
>
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
> ---
>
> Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> should be respected as well.
>
> I did not switch it to use HOSTLD for two reasons:
> * That variable is not globally defined - it is only available in two
>   subdirectories of tools/
> * C host scripts are linked by HOSTCC as well, even when linking a
>   collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
>   See scripts/Makefile.host cmd_host-cmulti for an example.

Sure, that makes sense to me, thanks for pointing that out.

>
>  rust/Makefile         | 4 ++++
>  scripts/Makefile.host | 4 ++++
>  2 files changed, 8 insertions(+)
>
> diff --git a/rust/Makefile b/rust/Makefile
> index 87958e864be0..b60b7eb8c5a0 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
>  $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
>         $(call if_changed,exports)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'

I don't think we need to do this kind of escaping. If a linker flag is
passed to the linker directly as the driver has spaces such as `-z
relro`, if it is instead passed to the compiler as the driver will not
contain spaces (`-Wl,-z,relro`). As such, I don't think we need this
escaping (famous last words).  Mind submitting a v3 without it?

> +
>  quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
>        cmd_rustc_procmacro = \
>         $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
> +               -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> +               -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
>                 --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
>                 --crate-type proc-macro \
>                 --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
> diff --git a/scripts/Makefile.host b/scripts/Makefile.host
> index 8f7f842b54f9..dc0410cae5ca 100644
> --- a/scripts/Makefile.host
> +++ b/scripts/Makefile.host
> @@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
>                   $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
>                   $(HOSTCXXFLAGS_$(target-stem).o)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
> +
>  # `--out-dir` is required to avoid temporaries being created by `rustc` in the
>  # current working directory, which may be not accessible in the out-of-tree
>  # modules case.
>  hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
> +                -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> +                -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
>                   $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
>                   $(HOSTRUSTFLAGS_$(target-stem))
>
> --
> 2.42.0.459.ge4e396fd5e-goog
>


-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] rust: Respect HOSTCC when linking for host
  2023-09-19 19:44 ` Nick Desaulniers
@ 2023-09-19 20:02   ` Nick Desaulniers
  0 siblings, 0 replies; 6+ messages in thread
From: Nick Desaulniers @ 2023-09-19 20:02 UTC (permalink / raw)
  To: Matthew Maurer
  Cc: Miguel Ojeda, Alex Gaynor, Wedson Almeida Filho, Masahiro Yamada,
	Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
	Andreas Hindborg, Alice Ryhl, Nathan Chancellor, Nicolas Schier,
	rust-for-linux, linux-kernel, linux-kbuild

On Tue, Sep 19, 2023 at 12:44 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Mon, Sep 18, 2023 at 4:44 PM Matthew Maurer <mmaurer@google.com> wrote:
> >
> > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> > resulting in build failures in hermetic environments where `cc` does not
> > exist. This includes both hostprogs and proc-macros.
> >
> > Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> > `gcc` explicitly.
> >
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
> > ---
> >
> > Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> > should be respected as well.
> >
> > I did not switch it to use HOSTLD for two reasons:
> > * That variable is not globally defined - it is only available in two
> >   subdirectories of tools/
> > * C host scripts are linked by HOSTCC as well, even when linking a
> >   collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
> >   See scripts/Makefile.host cmd_host-cmulti for an example.
>
> Sure, that makes sense to me, thanks for pointing that out.
>
> >
> >  rust/Makefile         | 4 ++++
> >  scripts/Makefile.host | 4 ++++
> >  2 files changed, 8 insertions(+)
> >
> > diff --git a/rust/Makefile b/rust/Makefile
> > index 87958e864be0..b60b7eb8c5a0 100644
> > --- a/rust/Makefile
> > +++ b/rust/Makefile
> > @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
> >  $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
> >         $(call if_changed,exports)
> >
> > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
>
> I don't think we need to do this kind of escaping. If a linker flag is
> passed to the linker directly as the driver has spaces such as `-z
> relro`, if it is instead passed to the compiler as the driver will not
> contain spaces (`-Wl,-z,relro`). As such, I don't think we need this
> escaping (famous last words).  Mind submitting a v3 without it?

Thanks for clarifying off list that:
```
It's not escaping the spaces, it's escaping any quotes
example: -Clink-args='-Wl,-z,relro -Wl,-linker-plugin-lto'
Without the quote, the second one would go to rustc rather than the linker
```

Sorry for misunderstanding what exactly we were escaping here and why;
thanks for implementing my suggestion of passing along
KBUILD_HOSTLDFLAGS though.  That will avoid issues for Android
downstream.

Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

>
> > +
> >  quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
> >        cmd_rustc_procmacro = \
> >         $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
> > +               -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> > +               -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> >                 --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
> >                 --crate-type proc-macro \
> >                 --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
> > diff --git a/scripts/Makefile.host b/scripts/Makefile.host
> > index 8f7f842b54f9..dc0410cae5ca 100644
> > --- a/scripts/Makefile.host
> > +++ b/scripts/Makefile.host
> > @@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
> >                   $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
> >                   $(HOSTCXXFLAGS_$(target-stem).o)
> >
> > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
> > +
> >  # `--out-dir` is required to avoid temporaries being created by `rustc` in the
> >  # current working directory, which may be not accessible in the out-of-tree
> >  # modules case.
> >  hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
> > +                -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> > +                -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> >                   $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
> >                   $(HOSTRUSTFLAGS_$(target-stem))
> >
> > --
> > 2.42.0.459.ge4e396fd5e-goog
> >
>
>
> --
> Thanks,
> ~Nick Desaulniers



-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] rust: Respect HOSTCC when linking for host
  2023-09-18 23:43 [PATCH v2] rust: Respect HOSTCC when linking for host Matthew Maurer
  2023-09-19 19:18 ` Martin Rodriguez Reboredo
  2023-09-19 19:44 ` Nick Desaulniers
@ 2023-09-26 10:45 ` Alice Ryhl
  2023-09-26 17:02 ` Masahiro Yamada
  3 siblings, 0 replies; 6+ messages in thread
From: Alice Ryhl @ 2023-09-26 10:45 UTC (permalink / raw)
  To: mmaurer
  Cc: a.hindborg, alex.gaynor, aliceryhl, benno.lossin, bjorn3_gh,
	boqun.feng, gary, linux-kbuild, linux-kernel, masahiroy, nathan,
	ndesaulniers, nicolas, ojeda, rust-for-linux, wedsonaf

Matthew Maurer <mmaurer@google.com> writes:
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
> 
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
> 
> Signed-off-by: Matthew Maurer <mmaurer@google.com>

Tested-by: Alice Ryhl <aliceryhl@google.com>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] rust: Respect HOSTCC when linking for host
  2023-09-18 23:43 [PATCH v2] rust: Respect HOSTCC when linking for host Matthew Maurer
                   ` (2 preceding siblings ...)
  2023-09-26 10:45 ` Alice Ryhl
@ 2023-09-26 17:02 ` Masahiro Yamada
  3 siblings, 0 replies; 6+ messages in thread
From: Masahiro Yamada @ 2023-09-26 17:02 UTC (permalink / raw)
  To: Matthew Maurer
  Cc: Miguel Ojeda, Alex Gaynor, Wedson Almeida Filho, Boqun Feng,
	Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
	Alice Ryhl, Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
	rust-for-linux, linux-kernel, linux-kbuild

On Tue, Sep 19, 2023 at 8:44 AM Matthew Maurer <mmaurer@google.com> wrote:
>
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
>
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.


This sentence is unclear to me because it does not explain
why we need both despite the spec mentions
'linker-flavor' is inferred from the value of 'linker'.



The answer exists in this thread.
"
As mentioned earlier, we could pass $HOSTLD, but if the linker isn't
named something accurate (e.g. if the linker is not named lld, but is
lld), we need to know how to pass a flavor:
"

Maybe it is worthing menting that HOSTCC may not be properly named.







>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
> ---
>
> Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> should be respected as well.
>
> I did not switch it to use HOSTLD for two reasons:
> * That variable is not globally defined - it is only available in two
>   subdirectories of tools/
> * C host scripts are linked by HOSTCC as well, even when linking a
>   collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
>   See scripts/Makefile.host cmd_host-cmulti for an example.
>
>  rust/Makefile         | 4 ++++
>  scripts/Makefile.host | 4 ++++
>  2 files changed, 8 insertions(+)
>
> diff --git a/rust/Makefile b/rust/Makefile
> index 87958e864be0..b60b7eb8c5a0 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
>  $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
>         $(call if_changed,exports)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'



This is equivalent to a macro in scripts/Kbuild.include

  escsq = $(subst $(squote),'\$(squote)',$1)





I would write it directly in cmd_rustc_procmacro.





> +
>  quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
>        cmd_rustc_procmacro = \
>         $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
> +               -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> +               -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
>                 --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
>                 --crate-type proc-macro \
>                 --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
> diff --git a/scripts/Makefile.host b/scripts/Makefile.host
> index 8f7f842b54f9..dc0410cae5ca 100644
> --- a/scripts/Makefile.host
> +++ b/scripts/Makefile.host
> @@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
>                   $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
>                   $(HOSTCXXFLAGS_$(target-stem).o)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
> +
>  # `--out-dir` is required to avoid temporaries being created by `rustc` in the
>  # current working directory, which may be not accessible in the out-of-tree
>  # modules case.
>  hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
> +                -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> +                -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
>                   $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
>                   $(HOSTRUSTFLAGS_$(target-stem))
>
> --
> 2.42.0.459.ge4e396fd5e-goog
>


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2023-09-26 17:03 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-18 23:43 [PATCH v2] rust: Respect HOSTCC when linking for host Matthew Maurer
2023-09-19 19:18 ` Martin Rodriguez Reboredo
2023-09-19 19:44 ` Nick Desaulniers
2023-09-19 20:02   ` Nick Desaulniers
2023-09-26 10:45 ` Alice Ryhl
2023-09-26 17:02 ` Masahiro Yamada

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).