All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing
@ 2022-02-24 15:45 Romain Naour
  2022-02-24 15:45 ` [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot Romain Naour
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Romain Naour @ 2022-02-24 15:45 UTC (permalink / raw)
  To: buildroot; +Cc: Eric Le Bihan, Romain Naour

TestRust and TestRustBin has been introduced at the time when there was
no cargo package infrastructure or any package using rust compiler
(Buildroot 2018.02).

Since then the ripgrep package has been introduced, initially using
the generic package infrastructure and converted later to the cargo
package infrastructure.

Due a recent change in rust/cargo removing the cargo config file [1]
the test TestRust and TestRustBin now fail to compile since they build
an hello-world crate outside of the cargo package infrastructure
without the correct environment for cross-compiling.

Replace the 'hello-world' crate by ripgrep package and check if it
can run properly in Qemu.

Fixes tests.package.test_rust.TestRustBin:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202545

But doesn't fixes tests.package.test_rust.TestRust due another bug:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544

[1] b6378631c2609742382984f6f7b93c1d9d2cdb78

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
---
v2: no changes
---
 support/testing/tests/package/test_rust.py | 48 ++--------------------
 1 file changed, 4 insertions(+), 44 deletions(-)

diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
index 884b40a4a3..a1abd20c09 100644
--- a/support/testing/tests/package/test_rust.py
+++ b/support/testing/tests/package/test_rust.py
@@ -8,9 +8,6 @@ import infra.basetest
 
 class TestRustBase(infra.basetest.BRTest):
 
-    target = 'armv7-unknown-linux-gnueabihf'
-    crate = 'hello-world'
-
     def login(self):
         img = os.path.join(self.builddir, "images", "rootfs.cpio")
         self.emulator.boot(arch="armv7",
@@ -18,43 +15,6 @@ class TestRustBase(infra.basetest.BRTest):
                            options=["-initrd", img])
         self.emulator.login()
 
-    def build_test_prog(self):
-        hostdir = os.path.join(self.builddir, 'host')
-        env = os.environ.copy()
-        env["USER"] = "br-user"
-        env["PATH"] = "{}:".format(os.path.join(hostdir, 'bin')) + env["PATH"]
-        env["CARGO_HOME"] = os.path.join(hostdir, 'usr', 'share', 'cargo')
-        env["RUST_TARGET_PATH"] = os.path.join(hostdir, 'etc', 'rustc')
-        cargo = os.path.join(hostdir, 'bin', 'cargo')
-        workdir = os.path.join(tempfile.mkdtemp(suffix='-br2-testing-rust'),
-                               self.crate)
-        manifest = os.path.join(workdir, 'Cargo.toml')
-        prog = os.path.join(workdir, 'target', self.target, 'debug', self.crate)
-
-        cmd = [cargo, 'init', '--bin', '--vcs', 'none', '-vv', workdir]
-        ret = subprocess.call(cmd,
-                              stdout=self.b.logfile,
-                              stderr=self.b.logfile,
-                              env=env)
-        if ret != 0:
-            raise SystemError("Cargo init failed")
-
-        cmd = [
-            cargo, 'build', '-vv', '--target', self.target,
-            '--manifest-path', manifest
-        ]
-        ret = subprocess.call(cmd,
-                              stdout=self.b.logfile,
-                              stderr=self.b.logfile,
-                              env=env)
-        if ret != 0:
-            raise SystemError("Cargo build failed")
-
-        shutil.copy(prog, os.path.join(self.builddir, 'target', 'usr', 'bin'))
-        self.b.build()
-        shutil.rmtree(workdir)
-
-
 class TestRustBin(TestRustBase):
     config = \
         """
@@ -68,12 +28,12 @@ class TestRustBin(TestRustBase):
         BR2_TARGET_ROOTFS_CPIO=y
         # BR2_TARGET_ROOTFS_TAR is not set
         BR2_PACKAGE_HOST_RUSTC=y
+        BR2_PACKAGE_RIPGREP=y
         """
 
     def test_run(self):
-        self.build_test_prog()
         self.login()
-        self.assertRunOk(self.crate)
+        self.assertRunOk("rg Buildroot /etc/issue")
 
 
 class TestRust(TestRustBase):
@@ -90,9 +50,9 @@ class TestRust(TestRustBase):
         # BR2_TARGET_ROOTFS_TAR is not set
         BR2_PACKAGE_HOST_RUSTC=y
         BR2_PACKAGE_HOST_RUST=y
+        BR2_PACKAGE_RIPGREP=y
         """
 
     def test_run(self):
-        self.build_test_prog()
         self.login()
-        self.assertRunOk(self.crate)
+        self.assertRunOk("rg Buildroot /etc/issue")
-- 
2.35.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot
  2022-02-24 15:45 [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Romain Naour
@ 2022-02-24 15:45 ` Romain Naour
  2022-03-12 20:34   ` Arnout Vandecappelle
  2022-03-20 17:35   ` Peter Korsgaard
  2022-03-12 20:30 ` [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Arnout Vandecappelle
  2022-03-15 21:13 ` Arnout Vandecappelle
  2 siblings, 2 replies; 8+ messages in thread
From: Romain Naour @ 2022-02-24 15:45 UTC (permalink / raw)
  To: buildroot; +Cc: Eric Le Bihan, Romain Naour

The test TestRust is currently broken with ripgrep package with
the following error:

error[E0514]: found crate `core` compiled by an incompatible version of rustc
  |
  = help: please recompile that crate using this compiler (rustc 1.58.1) (consider running `cargo clean` first)
  = note: the following crate versions were found:
          crate `core` compiled by rustc 1.58.1 (db9d1b20b 2022-01-20): TestRust/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6cfcec236d576603.rlib

error[E0514]: found crate `std` compiled by an incompatible version of rustc

The problem is not really a cross-compilation issue (we are building
for an armv7 target on x86_64 host) but a problem with rust-std libraries
(rlib).

We can notice that "rustc 1.58.1 (db9d1b20b 2022-01-20)" is the same
version as the prebuilt rustc used to bootstrap the build:

TestRust/host/bin/rustc --version
rustc 1.58.1

TestRustBin/host/bin/rustc --version
rustc 1.58.1 (db9d1b20b 2022-01-20)

Indeed we are using host-rust-bin to bootstrap the host-rust compiler
package built by Buildroot. The problem is that the
libcore-6cfcec236d576603.rlib file come from host-rust-bin (rust-std)
and is not removed before installing host-rust built by Buildroot.

We actually spent a lot of time to build host-rust with rust-std
and forget to install this important library HOST_DIR.

Looking at the host-rust build directory we can notice two installer
script "install.sh" (the same scripts used to install host-rust-bin):

TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust/x86_64-unknown-linux-gnu/rust-1.58.1-x86_64-unknown-linux-gnu/install.sh
TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust-std/armv7-unknown-linux-gnueabihf/rust-std-1.58.1-armv7-unknown-linux-gnueabihf/install.sh

The "tarball" directory is generated by the "python x.py dist" during
the install step, we have to keep it.

Replace "python x.py install" by theses two install scripts.
Installing rust-std with the install.sh script replace the rust-std
libraries installed by host-rust-bin.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
---
 package/rust/rust.mk | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/package/rust/rust.mk b/package/rust/rust.mk
index 46487fe4b7..4a6b04dc8c 100644
--- a/package/rust/rust.mk
+++ b/package/rust/rust.mk
@@ -67,9 +67,30 @@ define HOST_RUST_BUILD_CMDS
 	cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py build
 endef
 
+HOST_RUST_INSTALL_COMMON_OPTS = \
+	--prefix=$(HOST_DIR) \
+	--disable-ldconfig
+
+HOST_RUST_INSTALL_OPTS = \
+	$(HOST_RUST_INSTALL_COMMON_OPTS) \
+	--components=rustc,cargo,rust-std-$(RUSTC_HOST_NAME)
+
+define HOST_RUST_INSTALL_RUSTC
+	(cd $(@D)/build/tmp/tarball/rust/$(RUSTC_HOST_NAME)/rust-$(RUST_VERSION)-$(RUSTC_HOST_NAME); \
+		./install.sh $(HOST_RUST_INSTALL_OPTS))
+endef
+
+ifeq ($(BR2_PACKAGE_HOST_RUSTC_TARGET_ARCH_SUPPORTS),y)
+define HOST_RUST_INSTALL_LIBSTD_TARGET
+	(cd $(@D)/build/tmp/tarball/rust-std/$(RUSTC_TARGET_NAME)/rust-std-$(RUST_VERSION)-$(RUSTC_TARGET_NAME); \
+		./install.sh $(HOST_RUST_INSTALL_COMMON_OPTS))
+endef
+endif
+
 define HOST_RUST_INSTALL_CMDS
 	cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py dist
-	cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py install
+	$(HOST_RUST_INSTALL_RUSTC)
+	$(HOST_RUST_INSTALL_LIBSTD_TARGET)
 endef
 
 $(eval $(host-generic-package))
-- 
2.35.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing
  2022-02-24 15:45 [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Romain Naour
  2022-02-24 15:45 ` [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot Romain Naour
@ 2022-03-12 20:30 ` Arnout Vandecappelle
  2022-03-15 21:13 ` Arnout Vandecappelle
  2 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2022-03-12 20:30 UTC (permalink / raw)
  To: Romain Naour, buildroot; +Cc: Eric Le Bihan


On 24/02/2022 16:45, Romain Naour wrote:
> TestRust and TestRustBin has been introduced at the time when there was
> no cargo package infrastructure or any package using rust compiler
> (Buildroot 2018.02).
>
> Since then the ripgrep package has been introduced, initially using
> the generic package infrastructure and converted later to the cargo
> package infrastructure.
>
> Due a recent change in rust/cargo removing the cargo config file [1]
> the test TestRust and TestRustBin now fail to compile since they build
> an hello-world crate outside of the cargo package infrastructure
> without the correct environment for cross-compiling.
>
> Replace the 'hello-world' crate by ripgrep package and check if it
> can run properly in Qemu.
>
> Fixes tests.package.test_rust.TestRustBin:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202545
>
> But doesn't fixes tests.package.test_rust.TestRust due another bug:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
>
> [1] b6378631c2609742382984f6f7b93c1d9d2cdb78
>
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> ---
> v2: no changes
> ---
>   support/testing/tests/package/test_rust.py | 48 ++--------------------
>   1 file changed, 4 insertions(+), 44 deletions(-)
>
> diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
> index 884b40a4a3..a1abd20c09 100644
> --- a/support/testing/tests/package/test_rust.py
> +++ b/support/testing/tests/package/test_rust.py
> @@ -8,9 +8,6 @@ import infra.basetest
>   
>   class TestRustBase(infra.basetest.BRTest):
>   
> -    target = 'armv7-unknown-linux-gnueabihf'
> -    crate = 'hello-world'
> -
>       def login(self):
>           img = os.path.join(self.builddir, "images", "rootfs.cpio")
>           self.emulator.boot(arch="armv7",
> @@ -18,43 +15,6 @@ class TestRustBase(infra.basetest.BRTest):
>                              options=["-initrd", img])
>           self.emulator.login()
>   
> -    def build_test_prog(self):
> -        hostdir = os.path.join(self.builddir, 'host')
> -        env = os.environ.copy()
> -        env["USER"] = "br-user"
> -        env["PATH"] = "{}:".format(os.path.join(hostdir, 'bin')) + env["PATH"]
> -        env["CARGO_HOME"] = os.path.join(hostdir, 'usr', 'share', 'cargo')
> -        env["RUST_TARGET_PATH"] = os.path.join(hostdir, 'etc', 'rustc')
> -        cargo = os.path.join(hostdir, 'bin', 'cargo')
> -        workdir = os.path.join(tempfile.mkdtemp(suffix='-br2-testing-rust'),
> -                               self.crate)
> -        manifest = os.path.join(workdir, 'Cargo.toml')
> -        prog = os.path.join(workdir, 'target', self.target, 'debug', self.crate)
> -
> -        cmd = [cargo, 'init', '--bin', '--vcs', 'none', '-vv', workdir]
> -        ret = subprocess.call(cmd,
> -                              stdout=self.b.logfile,
> -                              stderr=self.b.logfile,
> -                              env=env)
> -        if ret != 0:
> -            raise SystemError("Cargo init failed")
> -
> -        cmd = [
> -            cargo, 'build', '-vv', '--target', self.target,
> -            '--manifest-path', manifest
> -        ]
> -        ret = subprocess.call(cmd,
> -                              stdout=self.b.logfile,
> -                              stderr=self.b.logfile,
> -                              env=env)
> -        if ret != 0:
> -            raise SystemError("Cargo build failed")
> -
> -        shutil.copy(prog, os.path.join(self.builddir, 'target', 'usr', 'bin'))
> -        self.b.build()
> -        shutil.rmtree(workdir)
> -
> -
>   class TestRustBin(TestRustBase):
>       config = \
>           """
> @@ -68,12 +28,12 @@ class TestRustBin(TestRustBase):
>           BR2_TARGET_ROOTFS_CPIO=y
>           # BR2_TARGET_ROOTFS_TAR is not set
>           BR2_PACKAGE_HOST_RUSTC=y
> +        BR2_PACKAGE_RIPGREP=y
>           """
>   
>       def test_run(self):
> -        self.build_test_prog()
>           self.login()
> -        self.assertRunOk(self.crate)
> +        self.assertRunOk("rg Buildroot /etc/issue")

  It now looks more like test_ripgrep than test_rust, but OK, we still have the 
rust-bin and full rust variants.


  Applied to master, thanks.

  Regards,
  Arnout

>   
>   
>   class TestRust(TestRustBase):
> @@ -90,9 +50,9 @@ class TestRust(TestRustBase):
>           # BR2_TARGET_ROOTFS_TAR is not set
>           BR2_PACKAGE_HOST_RUSTC=y
>           BR2_PACKAGE_HOST_RUST=y
> +        BR2_PACKAGE_RIPGREP=y
>           """
>   
>       def test_run(self):
> -        self.build_test_prog()
>           self.login()
> -        self.assertRunOk(self.crate)
> +        self.assertRunOk("rg Buildroot /etc/issue")
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot
  2022-02-24 15:45 ` [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot Romain Naour
@ 2022-03-12 20:34   ` Arnout Vandecappelle
  2022-03-19 14:30     ` Romain Naour
  2022-03-20 17:35   ` Peter Korsgaard
  1 sibling, 1 reply; 8+ messages in thread
From: Arnout Vandecappelle @ 2022-03-12 20:34 UTC (permalink / raw)
  To: Romain Naour, buildroot; +Cc: Eric Le Bihan



On 24/02/2022 16:45, Romain Naour wrote:
> The test TestRust is currently broken with ripgrep package with
> the following error:
> 
> error[E0514]: found crate `core` compiled by an incompatible version of rustc
>    |
>    = help: please recompile that crate using this compiler (rustc 1.58.1) (consider running `cargo clean` first)
>    = note: the following crate versions were found:
>            crate `core` compiled by rustc 1.58.1 (db9d1b20b 2022-01-20): TestRust/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6cfcec236d576603.rlib
> 
> error[E0514]: found crate `std` compiled by an incompatible version of rustc
> 
> The problem is not really a cross-compilation issue (we are building
> for an armv7 target on x86_64 host) but a problem with rust-std libraries
> (rlib).
> 
> We can notice that "rustc 1.58.1 (db9d1b20b 2022-01-20)" is the same
> version as the prebuilt rustc used to bootstrap the build:
> 
> TestRust/host/bin/rustc --version
> rustc 1.58.1
> 
> TestRustBin/host/bin/rustc --version
> rustc 1.58.1 (db9d1b20b 2022-01-20)
> 
> Indeed we are using host-rust-bin to bootstrap the host-rust compiler
> package built by Buildroot. The problem is that the
> libcore-6cfcec236d576603.rlib file come from host-rust-bin (rust-std)
> and is not removed before installing host-rust built by Buildroot.
> 
> We actually spent a lot of time to build host-rust with rust-std
> and forget to install this important library HOST_DIR.
> 
> Looking at the host-rust build directory we can notice two installer
> script "install.sh" (the same scripts used to install host-rust-bin):
> 
> TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust/x86_64-unknown-linux-gnu/rust-1.58.1-x86_64-unknown-linux-gnu/install.sh
> TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust-std/armv7-unknown-linux-gnueabihf/rust-std-1.58.1-armv7-unknown-linux-gnueabihf/install.sh
> 
> The "tarball" directory is generated by the "python x.py dist" during
> the install step, we have to keep it.
> 
> Replace "python x.py install" by theses two install scripts.
> Installing rust-std with the install.sh script replace the rust-std
> libraries installed by host-rust-bin.

  Excellent description, thanks. It's almost Yann-level :-)

> 
> Fixes:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> ---
>   package/rust/rust.mk | 23 ++++++++++++++++++++++-
>   1 file changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/package/rust/rust.mk b/package/rust/rust.mk
> index 46487fe4b7..4a6b04dc8c 100644
> --- a/package/rust/rust.mk
> +++ b/package/rust/rust.mk
> @@ -67,9 +67,30 @@ define HOST_RUST_BUILD_CMDS
>   	cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py build
>   endef
>   
> +HOST_RUST_INSTALL_COMMON_OPTS = \
> +	--prefix=$(HOST_DIR) \
> +	--disable-ldconfig
> +
> +HOST_RUST_INSTALL_OPTS = \

  This variable is used only once, which makes it not very useful. I instead 
added the --components directly in HOST_RUST_INSTALL_RUSTC. I also renamend the 
INSTALL_COMMON_OPTS to INSTALL_OPTS since there's only one variable now.

> +	$(HOST_RUST_INSTALL_COMMON_OPTS) \
> +	--components=rustc,cargo,rust-std-$(RUSTC_HOST_NAME)
> +
> +define HOST_RUST_INSTALL_RUSTC
> +	(cd $(@D)/build/tmp/tarball/rust/$(RUSTC_HOST_NAME)/rust-$(RUST_VERSION)-$(RUSTC_HOST_NAME); \

  The parenthesis are not necessary - make already starts a sub-shell for every 
line, there's no need to add another shell level. We still have them in a lot of 
.mk files, but we try to avoid adding them.


  Applied to master with those two changes, thanks.

  Regards,
  Arnout

> +		./install.sh $(HOST_RUST_INSTALL_OPTS))
> +endef
> +
> +ifeq ($(BR2_PACKAGE_HOST_RUSTC_TARGET_ARCH_SUPPORTS),y)
> +define HOST_RUST_INSTALL_LIBSTD_TARGET
> +	(cd $(@D)/build/tmp/tarball/rust-std/$(RUSTC_TARGET_NAME)/rust-std-$(RUST_VERSION)-$(RUSTC_TARGET_NAME); \
> +		./install.sh $(HOST_RUST_INSTALL_COMMON_OPTS))
> +endef
> +endif
> +
>   define HOST_RUST_INSTALL_CMDS
>   	cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py dist
> -	cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py install
> +	$(HOST_RUST_INSTALL_RUSTC)
> +	$(HOST_RUST_INSTALL_LIBSTD_TARGET)
>   endef
>   
>   $(eval $(host-generic-package))
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing
  2022-02-24 15:45 [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Romain Naour
  2022-02-24 15:45 ` [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot Romain Naour
  2022-03-12 20:30 ` [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Arnout Vandecappelle
@ 2022-03-15 21:13 ` Arnout Vandecappelle
  2022-03-19 14:33   ` Romain Naour
  2 siblings, 1 reply; 8+ messages in thread
From: Arnout Vandecappelle @ 2022-03-15 21:13 UTC (permalink / raw)
  To: Romain Naour, buildroot; +Cc: Eric Le Bihan


On 24/02/2022 16:45, Romain Naour wrote:
> TestRust and TestRustBin has been introduced at the time when there was
> no cargo package infrastructure or any package using rust compiler
> (Buildroot 2018.02).
>
> Since then the ripgrep package has been introduced, initially using
> the generic package infrastructure and converted later to the cargo
> package infrastructure.
>
> Due a recent change in rust/cargo removing the cargo config file [1]
> the test TestRust and TestRustBin now fail to compile since they build
> an hello-world crate outside of the cargo package infrastructure
> without the correct environment for cross-compiling.
>
> Replace the 'hello-world' crate by ripgrep package and check if it
> can run properly in Qemu.
>
> Fixes tests.package.test_rust.TestRustBin:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202545
>
> But doesn't fixes tests.package.test_rust.TestRust due another bug:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
>
> [1] b6378631c2609742382984f6f7b93c1d9d2cdb78
>
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>

  This introduced a few flake8 errors, which I fixed now. In the future: run 
flake8! (I also always forget :-))


  Regards,
  Arnout


> ---
> v2: no changes
> ---
>   support/testing/tests/package/test_rust.py | 48 ++--------------------
>   1 file changed, 4 insertions(+), 44 deletions(-)
>
> diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
> index 884b40a4a3..a1abd20c09 100644
> --- a/support/testing/tests/package/test_rust.py
> +++ b/support/testing/tests/package/test_rust.py
> @@ -8,9 +8,6 @@ import infra.basetest
>   
>   class TestRustBase(infra.basetest.BRTest):
>   
> -    target = 'armv7-unknown-linux-gnueabihf'
> -    crate = 'hello-world'
> -
>       def login(self):
>           img = os.path.join(self.builddir, "images", "rootfs.cpio")
>           self.emulator.boot(arch="armv7",
> @@ -18,43 +15,6 @@ class TestRustBase(infra.basetest.BRTest):
>                              options=["-initrd", img])
>           self.emulator.login()
>   
> -    def build_test_prog(self):
> -        hostdir = os.path.join(self.builddir, 'host')
> -        env = os.environ.copy()
> -        env["USER"] = "br-user"
> -        env["PATH"] = "{}:".format(os.path.join(hostdir, 'bin')) + env["PATH"]
> -        env["CARGO_HOME"] = os.path.join(hostdir, 'usr', 'share', 'cargo')
> -        env["RUST_TARGET_PATH"] = os.path.join(hostdir, 'etc', 'rustc')
> -        cargo = os.path.join(hostdir, 'bin', 'cargo')
> -        workdir = os.path.join(tempfile.mkdtemp(suffix='-br2-testing-rust'),
> -                               self.crate)
> -        manifest = os.path.join(workdir, 'Cargo.toml')
> -        prog = os.path.join(workdir, 'target', self.target, 'debug', self.crate)
> -
> -        cmd = [cargo, 'init', '--bin', '--vcs', 'none', '-vv', workdir]
> -        ret = subprocess.call(cmd,
> -                              stdout=self.b.logfile,
> -                              stderr=self.b.logfile,
> -                              env=env)
> -        if ret != 0:
> -            raise SystemError("Cargo init failed")
> -
> -        cmd = [
> -            cargo, 'build', '-vv', '--target', self.target,
> -            '--manifest-path', manifest
> -        ]
> -        ret = subprocess.call(cmd,
> -                              stdout=self.b.logfile,
> -                              stderr=self.b.logfile,
> -                              env=env)
> -        if ret != 0:
> -            raise SystemError("Cargo build failed")
> -
> -        shutil.copy(prog, os.path.join(self.builddir, 'target', 'usr', 'bin'))
> -        self.b.build()
> -        shutil.rmtree(workdir)
> -
> -
>   class TestRustBin(TestRustBase):
>       config = \
>           """
> @@ -68,12 +28,12 @@ class TestRustBin(TestRustBase):
>           BR2_TARGET_ROOTFS_CPIO=y
>           # BR2_TARGET_ROOTFS_TAR is not set
>           BR2_PACKAGE_HOST_RUSTC=y
> +        BR2_PACKAGE_RIPGREP=y
>           """
>   
>       def test_run(self):
> -        self.build_test_prog()
>           self.login()
> -        self.assertRunOk(self.crate)
> +        self.assertRunOk("rg Buildroot /etc/issue")
>   
>   
>   class TestRust(TestRustBase):
> @@ -90,9 +50,9 @@ class TestRust(TestRustBase):
>           # BR2_TARGET_ROOTFS_TAR is not set
>           BR2_PACKAGE_HOST_RUSTC=y
>           BR2_PACKAGE_HOST_RUST=y
> +        BR2_PACKAGE_RIPGREP=y
>           """
>   
>       def test_run(self):
> -        self.build_test_prog()
>           self.login()
> -        self.assertRunOk(self.crate)
> +        self.assertRunOk("rg Buildroot /etc/issue")
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot
  2022-03-12 20:34   ` Arnout Vandecappelle
@ 2022-03-19 14:30     ` Romain Naour
  0 siblings, 0 replies; 8+ messages in thread
From: Romain Naour @ 2022-03-19 14:30 UTC (permalink / raw)
  To: Arnout Vandecappelle, buildroot; +Cc: Eric Le Bihan, Yann E. MORIN

Hello Arnout,

Le 12/03/2022 à 21:34, Arnout Vandecappelle a écrit :
> 
> 
> On 24/02/2022 16:45, Romain Naour wrote:
>> The test TestRust is currently broken with ripgrep package with
>> the following error:
>>
>> error[E0514]: found crate `core` compiled by an incompatible version of rustc
>>    |
>>    = help: please recompile that crate using this compiler (rustc 1.58.1)
>> (consider running `cargo clean` first)
>>    = note: the following crate versions were found:
>>            crate `core` compiled by rustc 1.58.1 (db9d1b20b 2022-01-20):
>> TestRust/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6cfcec236d576603.rlib
>>
>>
>> error[E0514]: found crate `std` compiled by an incompatible version of rustc
>>
>> The problem is not really a cross-compilation issue (we are building
>> for an armv7 target on x86_64 host) but a problem with rust-std libraries
>> (rlib).
>>
>> We can notice that "rustc 1.58.1 (db9d1b20b 2022-01-20)" is the same
>> version as the prebuilt rustc used to bootstrap the build:
>>
>> TestRust/host/bin/rustc --version
>> rustc 1.58.1
>>
>> TestRustBin/host/bin/rustc --version
>> rustc 1.58.1 (db9d1b20b 2022-01-20)
>>
>> Indeed we are using host-rust-bin to bootstrap the host-rust compiler
>> package built by Buildroot. The problem is that the
>> libcore-6cfcec236d576603.rlib file come from host-rust-bin (rust-std)
>> and is not removed before installing host-rust built by Buildroot.
>>
>> We actually spent a lot of time to build host-rust with rust-std
>> and forget to install this important library HOST_DIR.
>>
>> Looking at the host-rust build directory we can notice two installer
>> script "install.sh" (the same scripts used to install host-rust-bin):
>>
>> TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust/x86_64-unknown-linux-gnu/rust-1.58.1-x86_64-unknown-linux-gnu/install.sh
>>
>> TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust-std/armv7-unknown-linux-gnueabihf/rust-std-1.58.1-armv7-unknown-linux-gnueabihf/install.sh
>>
>>
>> The "tarball" directory is generated by the "python x.py dist" during
>> the install step, we have to keep it.
>>
>> Replace "python x.py install" by theses two install scripts.
>> Installing rust-std with the install.sh script replace the rust-std
>> libraries installed by host-rust-bin.
> 
>  Excellent description, thanks. It's almost Yann-level :-)

Why "almost"? you meant Yann-level but without typos (kidding) :)

Thank you for your comment, It's highly appreciated.

> 
>>
>> Fixes:
>> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
>>
>> Signed-off-by: Romain Naour <romain.naour@gmail.com>
>> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
>> ---
>>   package/rust/rust.mk | 23 ++++++++++++++++++++++-
>>   1 file changed, 22 insertions(+), 1 deletion(-)
>>
>> diff --git a/package/rust/rust.mk b/package/rust/rust.mk
>> index 46487fe4b7..4a6b04dc8c 100644
>> --- a/package/rust/rust.mk
>> +++ b/package/rust/rust.mk
>> @@ -67,9 +67,30 @@ define HOST_RUST_BUILD_CMDS
>>       cd $(@D); $(HOST_MAKE_ENV)
>> $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py build
>>   endef
>>   +HOST_RUST_INSTALL_COMMON_OPTS = \
>> +    --prefix=$(HOST_DIR) \
>> +    --disable-ldconfig
>> +
>> +HOST_RUST_INSTALL_OPTS = \
> 
>  This variable is used only once, which makes it not very useful. I instead
> added the --components directly in HOST_RUST_INSTALL_RUSTC. I also renamend the
> INSTALL_COMMON_OPTS to INSTALL_OPTS since there's only one variable now.
> 
>> +    $(HOST_RUST_INSTALL_COMMON_OPTS) \
>> +    --components=rustc,cargo,rust-std-$(RUSTC_HOST_NAME)
>> +
>> +define HOST_RUST_INSTALL_RUSTC
>> +    (cd
>> $(@D)/build/tmp/tarball/rust/$(RUSTC_HOST_NAME)/rust-$(RUST_VERSION)-$(RUSTC_HOST_NAME);
>> \
> 
>  The parenthesis are not necessary - make already starts a sub-shell for every
> line, there's no need to add another shell level. We still have them in a lot of
> .mk files, but we try to avoid adding them.

Ok

> 
> 
>  Applied to master with those two changes, thanks.

Thanks!

Best regards,
Romain

> 
>  Regards,
>  Arnout
> 
>> +        ./install.sh $(HOST_RUST_INSTALL_OPTS))
>> +endef
>> +
>> +ifeq ($(BR2_PACKAGE_HOST_RUSTC_TARGET_ARCH_SUPPORTS),y)
>> +define HOST_RUST_INSTALL_LIBSTD_TARGET
>> +    (cd
>> $(@D)/build/tmp/tarball/rust-std/$(RUSTC_TARGET_NAME)/rust-std-$(RUST_VERSION)-$(RUSTC_TARGET_NAME);
>> \
>> +        ./install.sh $(HOST_RUST_INSTALL_COMMON_OPTS))
>> +endef
>> +endif
>> +
>>   define HOST_RUST_INSTALL_CMDS
>>       cd $(@D); $(HOST_MAKE_ENV)
>> $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR) x.py dist
>> -    cd $(@D); $(HOST_MAKE_ENV) $(HOST_DIR)/bin/python$(PYTHON3_VERSION_MAJOR)
>> x.py install
>> +    $(HOST_RUST_INSTALL_RUSTC)
>> +    $(HOST_RUST_INSTALL_LIBSTD_TARGET)
>>   endef
>>     $(eval $(host-generic-package))

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing
  2022-03-15 21:13 ` Arnout Vandecappelle
@ 2022-03-19 14:33   ` Romain Naour
  0 siblings, 0 replies; 8+ messages in thread
From: Romain Naour @ 2022-03-19 14:33 UTC (permalink / raw)
  To: Arnout Vandecappelle, buildroot; +Cc: Eric Le Bihan

Le 15/03/2022 à 22:13, Arnout Vandecappelle a écrit :
> 
> On 24/02/2022 16:45, Romain Naour wrote:
>> TestRust and TestRustBin has been introduced at the time when there was
>> no cargo package infrastructure or any package using rust compiler
>> (Buildroot 2018.02).
>>
>> Since then the ripgrep package has been introduced, initially using
>> the generic package infrastructure and converted later to the cargo
>> package infrastructure.
>>
>> Due a recent change in rust/cargo removing the cargo config file [1]
>> the test TestRust and TestRustBin now fail to compile since they build
>> an hello-world crate outside of the cargo package infrastructure
>> without the correct environment for cross-compiling.
>>
>> Replace the 'hello-world' crate by ripgrep package and check if it
>> can run properly in Qemu.
>>
>> Fixes tests.package.test_rust.TestRustBin:
>> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202545
>>
>> But doesn't fixes tests.package.test_rust.TestRust due another bug:
>> https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
>>
>> [1] b6378631c2609742382984f6f7b93c1d9d2cdb78
>>
>> Signed-off-by: Romain Naour <romain.naour@gmail.com>
>> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> 
>  This introduced a few flake8 errors, which I fixed now. In the future: run
> flake8! (I also always forget :-))

This is completely unexpected, sorry for that.

Best regards,
Romain


> 
> 
>  Regards,
>  Arnout
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot
  2022-02-24 15:45 ` [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot Romain Naour
  2022-03-12 20:34   ` Arnout Vandecappelle
@ 2022-03-20 17:35   ` Peter Korsgaard
  1 sibling, 0 replies; 8+ messages in thread
From: Peter Korsgaard @ 2022-03-20 17:35 UTC (permalink / raw)
  To: Romain Naour; +Cc: Eric Le Bihan, buildroot

>>>>> "Romain" == Romain Naour <romain.naour@gmail.com> writes:

 > The test TestRust is currently broken with ripgrep package with
 > the following error:

 > error[E0514]: found crate `core` compiled by an incompatible version of rustc
 >   |
 >   = help: please recompile that crate using this compiler (rustc 1.58.1) (consider running `cargo clean` first)
 >   = note: the following crate versions were found:
 >           crate `core` compiled by rustc 1.58.1 (db9d1b20b 2022-01-20): TestRust/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6cfcec236d576603.rlib

 > error[E0514]: found crate `std` compiled by an incompatible version of rustc

 > The problem is not really a cross-compilation issue (we are building
 > for an armv7 target on x86_64 host) but a problem with rust-std libraries
 > (rlib).

 > We can notice that "rustc 1.58.1 (db9d1b20b 2022-01-20)" is the same
 > version as the prebuilt rustc used to bootstrap the build:

 > TestRust/host/bin/rustc --version
 > rustc 1.58.1

 > TestRustBin/host/bin/rustc --version
 > rustc 1.58.1 (db9d1b20b 2022-01-20)

 > Indeed we are using host-rust-bin to bootstrap the host-rust compiler
 > package built by Buildroot. The problem is that the
 > libcore-6cfcec236d576603.rlib file come from host-rust-bin (rust-std)
 > and is not removed before installing host-rust built by Buildroot.

 > We actually spent a lot of time to build host-rust with rust-std
 > and forget to install this important library HOST_DIR.

 > Looking at the host-rust build directory we can notice two installer
 > script "install.sh" (the same scripts used to install host-rust-bin):

 > TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust/x86_64-unknown-linux-gnu/rust-1.58.1-x86_64-unknown-linux-gnu/install.sh
 > TestRust/build/host-rust-1.58.1/build/tmp/tarball/rust-std/armv7-unknown-linux-gnueabihf/rust-std-1.58.1-armv7-unknown-linux-gnueabihf/install.sh

 > The "tarball" directory is generated by the "python x.py dist" during
 > the install step, we have to keep it.

 > Replace "python x.py install" by theses two install scripts.
 > Installing rust-std with the install.sh script replace the rust-std
 > libraries installed by host-rust-bin.

 > Fixes:
 > https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544

 > Signed-off-by: Romain Naour <romain.naour@gmail.com>
 > Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>

Committed to 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2022-03-20 17:35 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-24 15:45 [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Romain Naour
2022-02-24 15:45 ` [Buildroot] [PATCHv2 2/2] package/rust: install rustc and rust-std built by Buildroot Romain Naour
2022-03-12 20:34   ` Arnout Vandecappelle
2022-03-19 14:30     ` Romain Naour
2022-03-20 17:35   ` Peter Korsgaard
2022-03-12 20:30 ` [Buildroot] [PATCHv2 1/2] support/testing: TestRust{Bin} use ripgrep package for testing Arnout Vandecappelle
2022-03-15 21:13 ` Arnout Vandecappelle
2022-03-19 14:33   ` Romain Naour

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.