[RFC PATCH 0/1] rust: crates in other kernel directories

[RFC PATCH 0/1] rust: crates in other kernel directories

[Martin Rodriguez Reboredo]

This RFC provides makes possible to have bindings for kernel subsystems
that are compiled as modules.

Previously, if you wanted to have Rust bindings for a subsystem, like
AMBA for example, you had to put it under `rust/kernel/` so it came
part of the `kernel` crate, but this came with many downsides. Namely
if you compiled said subsystem as a module you've a dependency on it
from `kernel`, which is linked directly on `vmlinux`.

So instead of overpopulating `kernel` with a gazillion modules that
throws you into dire straits you should rather have the bindings in the
same directory as the subsystem you want to bind with and link it to
it.

For now I don't have a completely working example but I have an MWE as
following...

    # Add this line to drivers/usb/core/Kconfig
    config USB_RUST
    	bool "Rust USB bindings"
    	depends on USB && RUST
    	default n
    	help
    	  Enables Rust bindings for USB.

    # Add this line to drivers/usb/core/Makefile
    usbcore-$(CONFIG_USB_RUST)		+= usb.rlib

    # Create this file drivers/usb/core/usb.rs
    // SPDX-License-Identifier: GPL-2.0
    
    //! USB devices and drivers.
    //!
    //! C header: [`include/linux/usb.h`](../../../../include/linux/usb.h)
    
    use kernel::bindings;
    
    /// Check if USB is disabled.
    pub fn disabled() -> bool {
        // SAFETY: FFI call.
        unsafe { bindings::usb_disabled() != 0 }
    }

    # Compile the .rlib with
    make drivers/usb/core/usb.rlib LLVM=1

If everything went well then you should have
`drivers/usb/core/usb.rlib` ready to be linked.

As of now this PR won't compile the kernel fully as it serves as an
starting point and it's going to change in the future.

Martin Rodriguez Reboredo (1):
  scripts: Build per module Rust crates

 Makefile               |  4 ++--
 scripts/Makefile.build | 10 +++++++++-
 scripts/Makefile.lib   | 19 +++++++++++++------
 3 files changed, 24 insertions(+), 9 deletions(-)

-- 
2.42.0

[RFC PATCH 1/1] scripts: Build per module Rust crates

[Martin Rodriguez Reboredo]

Enables compiling Rust crates as dependencies of kernel modules.

When a composite object depends on an `.rlib` file, which by the way is
a current ar archive, Kbuild will compile it from its base Rust source
and link it.

This makes possible to have Rust bindings for a subsystem that is
compiled as a module.

Signed-off-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
---
 Makefile               |  4 ++--
 scripts/Makefile.build | 10 +++++++++-
 scripts/Makefile.lib   | 19 +++++++++++++------
 3 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/Makefile b/Makefile
index 7d6be12e4c3e..7774c97e8aa0 100644
--- a/Makefile
+++ b/Makefile
@@ -283,7 +283,7 @@ no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
 			headers_install modules_install modules_sign kernelrelease image_name
 no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
 			  image_name
-single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %/
+single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rlib %.rsi %.s %.symtypes %/
 
 config-build	:=
 mixed-build	:=
@@ -1919,7 +1919,7 @@ $(clean-dirs):
 clean: $(clean-dirs)
 	$(call cmd,rmfiles)
 	@find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
-		\( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \
+		\( -name '*.[aios]' -o -name '*.rlib' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \
 		-o -name '*.ko.*' \
 		-o -name '*.dtb' -o -name '*.dtbo' \
 		-o -name '*.dtb.S' -o -name '*.dtbo.S' \
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index da37bfa97211..627010518b27 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -246,7 +246,9 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
 # To make this rule robust against "Argument list too long" error,
 # ensure to add $(obj)/ prefix by a shell command.
 cmd_mod = printf '%s\n' $(call real-search, $*.o, .o, -objs -y -m) | \
-	$(AWK) '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
+	$(AWK) '!x[$$0]++ { print("$(obj)/"$$0) }' > $@ && \
+	printf '%s\n' $(call real-search, $*.rlib, .rlib, -objs -y -m) | \
+	$(AWK) '!x[$$0]++ { print("--library=$(obj)/"$$0) }' >> $@
 
 $(obj)/%.mod: FORCE
 	$(call if_changed,mod)
@@ -291,6 +293,12 @@ quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
 $(obj)/%.o: $(src)/%.rs FORCE
 	$(call if_changed_dep,rustc_o_rs)
 
+quiet_cmd_rustc_rlib_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
+      cmd_rustc_rlib_rs = $(rust_common_cmd) -Ccodegen-units=1 --emit=link=$@ $<
+
+$(obj)/%.rlib: $(src)/%.rs FORCE
+	$(call if_changed_dep,rustc_rlib_rs)
+
 quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
       cmd_rustc_rsi_rs = \
 	$(rust_common_cmd) -Zunpretty=expanded $< >$@; \
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 68d0134bdbf9..6e8cfbad015d 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -53,14 +53,18 @@ multi-search = $(sort $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -),
 real-search = $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $(call suffix-search, $m, $2, $3), $m))
 
 # If $(foo-objs), $(foo-y), $(foo-m), or $(foo-) exists, foo.o is a composite object
-multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y)
-multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m)
+multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y) \
+	$(call multi-search, $(obj-y), .rlib, -objs -y)
+multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m) \
+	$(call multi-search, $(obj-m), .rlib, -objs -y -m)
 multi-obj-ym := $(multi-obj-y) $(multi-obj-m)
 
 # Replace multi-part objects by their individual parts,
 # including built-in.a from subdirectories
-real-obj-y := $(call real-search, $(obj-y), .o, -objs -y)
-real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m)
+real-obj-y := $(call real-search, $(obj-y), .o, -objs -y) \
+	$(call real-search, $(obj-y), .rlib, -objs -y)
+real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m) \
+	$(call real-search, $(obj-m), .rlib, -objs -y -m)
 
 always-y += $(always-m)
 
@@ -107,7 +111,8 @@ endif
 # Finds the multi-part object the current object will be linked into.
 # If the object belongs to two or more multi-part objects, list them all.
 modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
-		$(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o=))))
+		$(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o=)) \
+		$(if $(filter $*.rlib, $(call suffix-search, $m, .rlib, -objs -y -m)),$(m:.rlib=))))
 
 __modname = $(or $(modname-multi),$(basetarget))
 
@@ -210,7 +215,9 @@ _cpp_flags += -I $(srctree)/$(src) -I $(objtree)/$(obj)
 endif
 endif
 
-part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
+part-of-module =                                             \
+	$(if $(or $(filter $(basename $@).o, $(real-obj-m)), \
+		$(filter $(basename $@).rlib, $(real-obj-m))),y)
 quiet_modtag = $(if $(part-of-module),[M],   )
 
 modkern_cflags =                                          \
-- 
2.42.0

Re: [RFC PATCH 0/1] rust: crates in other kernel directories

[Miguel Ojeda]

On Fri, Oct 6, 2023 at 5:57 PM Martin Rodriguez Reboredo
<yakoyoku@gmail.com> wrote:
>
> For now I don't have a completely working example but I have an MWE as
> following...

Thanks Martin, I appreciate it, but I don't think this would work in
the general case we want to solve.

I have been intending to work on this for a long time now, and I still
mean to. Hopefully I will have some news at LPC.

Cheers,
Miguel

Re: [RFC PATCH 0/1] rust: crates in other kernel directories

[Martin Rodriguez Reboredo]

On 10/6/23 18:13, Miguel Ojeda wrote:
> On Fri, Oct 6, 2023 at 5:57 PM Martin Rodriguez Reboredo
> <yakoyoku@gmail.com> wrote:
>>
>> For now I don't have a completely working example but I have an MWE as
>> following...
> 
> Thanks Martin, I appreciate it, but I don't think this would work in
> the general case we want to solve.

Well, there's the network PHY driver series that is stuck on this
topic, I (and probably others) have local work that have the same issue
and said general case might not arrive that soon. So it'd better to
start on a feature that will benefit API users right away and can be
updated to fit needs. That's what I think.

> I have been intending to work on this for a long time now, and I still
> mean to. Hopefully I will have some news at LPC.
> 
> Cheers,
> Miguel

Re: [RFC PATCH 0/1] rust: crates in other kernel directories

[Miguel Ojeda]

On Sat, Oct 7, 2023 at 1:07 AM Martin Rodriguez Reboredo
<yakoyoku@gmail.com> wrote:
>
> Well, there's the network PHY driver series that is stuck on this
> topic, I (and probably others) have local work that have the same issue

No, that series is not stuck on this.

> and said general case might not arrive that soon. So it'd better to
> start on a feature that will benefit API users right away and can be
> updated to fit needs. That's what I think.

If there is an in-tree use case that will really require a solution to
this very soon, then please focus on explaining that (e.g. what "local
work" do you have and what are your plans for landing it upstream?)
and we will do our best to reschedule priorities or we can provide a
workaround for the time being.

In any case, please note that this RFC breaks the kernel build and
does not really solve the overall issue.

Cheers,
Miguel

Re: [RFC PATCH 1/1] scripts: Build per module Rust crates

[Masahiro Yamada]

On Sat, Oct 7, 2023 at 12:57 AM Martin Rodriguez Reboredo
<yakoyoku@gmail.com> wrote:
>
> Enables compiling Rust crates as dependencies of kernel modules.
>
> When a composite object depends on an `.rlib` file, which by the way is
> a current ar archive, Kbuild will compile it from its base Rust source
> and link it.
>
> This makes possible to have Rust bindings for a subsystem that is
> compiled as a module.
>
> Signed-off-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>


I could not understand how this will work because
there is no explanation how to use *.rlib in the later
build steps.

If I understand correctly, does this intend to link *.rlib
as a part of modules?

In C, there was a discussion about
"that would be nice to be able to link static libraries",
but we do not do it any more. [1]

Following that discussion, linking libraries does not
seem what we want to do.


[1]: https://lore.kernel.org/lkml/20200106032324.3147-1-masahiroy@kernel.org/




> ---
>  Makefile               |  4 ++--
>  scripts/Makefile.build | 10 +++++++++-
>  scripts/Makefile.lib   | 19 +++++++++++++------
>  3 files changed, 24 insertions(+), 9 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 7d6be12e4c3e..7774c97e8aa0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -283,7 +283,7 @@ no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
>                         headers_install modules_install modules_sign kernelrelease image_name
>  no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
>                           image_name
> -single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %/
> +single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rlib %.rsi %.s %.symtypes %/
>
>  config-build   :=
>  mixed-build    :=
> @@ -1919,7 +1919,7 @@ $(clean-dirs):
>  clean: $(clean-dirs)
>         $(call cmd,rmfiles)
>         @find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
> -               \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \
> +               \( -name '*.[aios]' -o -name '*.rlib' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \
>                 -o -name '*.ko.*' \
>                 -o -name '*.dtb' -o -name '*.dtbo' \
>                 -o -name '*.dtb.S' -o -name '*.dtbo.S' \
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index da37bfa97211..627010518b27 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -246,7 +246,9 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
>  # To make this rule robust against "Argument list too long" error,
>  # ensure to add $(obj)/ prefix by a shell command.
>  cmd_mod = printf '%s\n' $(call real-search, $*.o, .o, -objs -y -m) | \
> -       $(AWK) '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
> +       $(AWK) '!x[$$0]++ { print("$(obj)/"$$0) }' > $@ && \
> +       printf '%s\n' $(call real-search, $*.rlib, .rlib, -objs -y -m) | \
> +       $(AWK) '!x[$$0]++ { print("--library=$(obj)/"$$0) }' >> $@
>
>  $(obj)/%.mod: FORCE
>         $(call if_changed,mod)
> @@ -291,6 +293,12 @@ quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
>  $(obj)/%.o: $(src)/%.rs FORCE
>         $(call if_changed_dep,rustc_o_rs)
>
> +quiet_cmd_rustc_rlib_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
> +      cmd_rustc_rlib_rs = $(rust_common_cmd) -Ccodegen-units=1 --emit=link=$@ $<
> +
> +$(obj)/%.rlib: $(src)/%.rs FORCE
> +       $(call if_changed_dep,rustc_rlib_rs)
> +
>  quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
>        cmd_rustc_rsi_rs = \
>         $(rust_common_cmd) -Zunpretty=expanded $< >$@; \
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 68d0134bdbf9..6e8cfbad015d 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -53,14 +53,18 @@ multi-search = $(sort $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -),
>  real-search = $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $(call suffix-search, $m, $2, $3), $m))
>
>  # If $(foo-objs), $(foo-y), $(foo-m), or $(foo-) exists, foo.o is a composite object
> -multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y)
> -multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m)
> +multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y) \
> +       $(call multi-search, $(obj-y), .rlib, -objs -y)
> +multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m) \
> +       $(call multi-search, $(obj-m), .rlib, -objs -y -m)
>  multi-obj-ym := $(multi-obj-y) $(multi-obj-m)
>
>  # Replace multi-part objects by their individual parts,
>  # including built-in.a from subdirectories
> -real-obj-y := $(call real-search, $(obj-y), .o, -objs -y)
> -real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m)
> +real-obj-y := $(call real-search, $(obj-y), .o, -objs -y) \
> +       $(call real-search, $(obj-y), .rlib, -objs -y)
> +real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m) \
> +       $(call real-search, $(obj-m), .rlib, -objs -y -m)
>
>  always-y += $(always-m)
>
> @@ -107,7 +111,8 @@ endif
>  # Finds the multi-part object the current object will be linked into.
>  # If the object belongs to two or more multi-part objects, list them all.
>  modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
> -               $(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o=))))
> +               $(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o=)) \
> +               $(if $(filter $*.rlib, $(call suffix-search, $m, .rlib, -objs -y -m)),$(m:.rlib=))))
>
>  __modname = $(or $(modname-multi),$(basetarget))
>
> @@ -210,7 +215,9 @@ _cpp_flags += -I $(srctree)/$(src) -I $(objtree)/$(obj)
>  endif
>  endif
>
> -part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
> +part-of-module =                                             \
> +       $(if $(or $(filter $(basename $@).o, $(real-obj-m)), \
> +               $(filter $(basename $@).rlib, $(real-obj-m))),y)
>  quiet_modtag = $(if $(part-of-module),[M],   )
>
>  modkern_cflags =                                          \
> --
> 2.42.0
>


--
Best Regards
Masahiro Yamada

Re: [RFC PATCH 1/1] scripts: Build per module Rust crates

[Miguel Ojeda]

On Sun, Oct 22, 2023 at 6:22 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> I could not understand how this will work because
> there is no explanation how to use *.rlib in the later
> build steps.

It seems linux-kbuild was not Cc'd in the cover letter -- I replied to
Martin there: https://lore.kernel.org/rust-for-linux/20231006155739.246381-1-yakoyoku@gmail.com/

Cheers,
Miguel

Re: [RFC PATCH 0/1] rust: crates in other kernel directories

[Masahiro Yamada]

On Mon, Oct 9, 2023 at 9:01 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Sat, Oct 7, 2023 at 1:07 AM Martin Rodriguez Reboredo
> <yakoyoku@gmail.com> wrote:
> >
> > Well, there's the network PHY driver series that is stuck on this
> > topic, I (and probably others) have local work that have the same issue
>
> No, that series is not stuck on this.
>
> > and said general case might not arrive that soon. So it'd better to
> > start on a feature that will benefit API users right away and can be
> > updated to fit needs. That's what I think.
>
> If there is an in-tree use case that will really require a solution to
> this very soon, then please focus on explaining that (e.g. what "local
> work" do you have and what are your plans for landing it upstream?)
> and we will do our best to reschedule priorities or we can provide a
> workaround for the time being.
>
> In any case, please note that this RFC breaks the kernel build and
> does not really solve the overall issue.
>
> Cheers,
> Miguel
>


Does this rely on the assumption that usb.rlib is small enough
because usb.rlib will be linked to every usb driver.

So, usb.rlib is just a binding, not able to write the
subsystem code in Rust, correct?







-- 
Best Regards
Masahiro Yamada

Re: [RFC PATCH 1/1] scripts: Build per module Rust crates

[Martin Rodriguez Reboredo]

On 10/22/23 13:21, Masahiro Yamada wrote:
> On Sat, Oct 7, 2023 at 12:57 AM Martin Rodriguez Reboredo
> <yakoyoku@gmail.com> wrote:
>>
>> Enables compiling Rust crates as dependencies of kernel modules.
>>
>> When a composite object depends on an `.rlib` file, which by the way is
>> a current ar archive, Kbuild will compile it from its base Rust source
>> and link it.
>>
>> This makes possible to have Rust bindings for a subsystem that is
>> compiled as a module.
>>
>> Signed-off-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
> 
> 
> I could not understand how this will work because
> there is no explanation how to use *.rlib in the later
> build steps.
> 
> If I understand correctly, does this intend to link *.rlib
> as a part of modules?

Yes, the purpose of this RFC is to link the *.rlib, its members
specifically, as part of modules to avoid the scenario of having
Rust bindings linked statically and their C counterparts dynamically.

I'll write a v2 that'll clarify its usage. I've made more progress
locally and had run tests with success.