All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Vinay Kumar" <vinay.m.engg@gmail.com>
To: openembedded-core@lists.openembedded.org, alex.kanavin@gmail.com
Cc: richard.purdie@linuxfoundation.org, kweihmann@outlook.com,
	rwmacleod@gmail.com, umesh.kalappa0@gmail.com,
	vinay.kumar@blackfigtech.com,
	Vinay Kumar <vinay.m.engg@gmail.com>
Subject: [PATCH v3 1/5] rust-testsuite.inc : Rust cross testing.
Date: Fri, 21 May 2021 06:29:10 -0700	[thread overview]
Message-ID: <20210521132914.140884-1-vinay.m.engg@gmail.com> (raw)
In-Reply-To: <CANNYZj-yODhEEFErtUxBeiw3O1YvEfV-hGOOW_uPNvo6x8mNYQ@mail.gmail.com>

Below list of functions are part of rust-testsuite.inc
setup_cargo_environment(): Build bootstrap and some early stage tools.
do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemutarget image.
do_check(): To execute testing on qmutarget image.

Signed-off-by: Vinay Kumar <vinay.m.engg@gmail.com>
---
 meta/recipes-devtools/rust/rust-testsuite.inc | 163 ++++++++++++++++++
 1 file changed, 163 insertions(+)
 create mode 100644 meta/recipes-devtools/rust/rust-testsuite.inc

diff --git a/meta/recipes-devtools/rust/rust-testsuite.inc b/meta/recipes-devtools/rust/rust-testsuite.inc
new file mode 100644
index 0000000000..88fb4e70d9
--- /dev/null
+++ b/meta/recipes-devtools/rust/rust-testsuite.inc
@@ -0,0 +1,163 @@
+SUMMARY = "Rust testing"
+HOMEPAGE = "https://rustc-dev-guide.rust-lang.org/tests/intro.html"
+SECTION = "test"
+LICENSE = "MIT | Apache-2.0"
+
+inherit rust
+inherit cargo_common
+
+DEPENDS += "file-native python3-native"
+EXCLUDE_FROM_WORLD = "1"
+
+# Path of target specification file "target-poky-linux.json"
+export RUST_TARGET_PATH="${STAGING_LIBDIR_NATIVE}/rustlib"
+
+export FORCE_CRATE_HASH="${BB_TASKHASH}"
+
+# We don't want to use bitbakes vendoring because the rust sources do their
+# own vendoring.
+CARGO_DISABLE_BITBAKE_VENDORING = "1"
+
+# We can't use RUST_BUILD_SYS here because that may be "musl" if
+# TCLIBC="musl". Snapshots are always -unknown-linux-gnu
+SNAPSHOT_BUILD_SYS = "${BUILD_ARCH}-unknown-linux-gnu"
+setup_cargo_environment () {
+    # The first step is to build bootstrap and some early stage tools,
+    # these are build for the same target as the snapshot, e.g.
+    # x86_64-unknown-linux-gnu.
+    # Later stages are build for the native target (i.e. target.x86_64-linux)
+    cargo_common_do_configure
+
+    printf '[target.%s]\n' "${SNAPSHOT_BUILD_SYS}" >> ${CARGO_HOME}/config
+    printf "linker = '%s'\n" "${RUST_BUILD_CCLD}" >> ${CARGO_HOME}/config
+}
+
+do_rust_setup_snapshot () {
+    for installer in "${WORKDIR}/rust-snapshot-components/"*"/install.sh"; do
+        "${installer}" --prefix="${WORKDIR}/rust-snapshot" --disable-ldconfig
+    done
+
+    # Some versions of rust (e.g. 1.18.0) tries to find cargo in stage0/bin/cargo
+    # and fail without it there.
+    mkdir -p ${RUSTSRC}/build/${BUILD_SYS}
+    ln -sf ${WORKDIR}/rust-snapshot/ ${RUSTSRC}/build/${BUILD_SYS}/stage0
+}
+addtask rust_setup_snapshot after do_unpack before do_configure
+do_rust_setup_snapshot[dirs] += "${WORKDIR}/rust-snapshot"
+
+python do_configure() {
+    import json
+    from distutils.version import LooseVersion
+    try:
+        import configparser
+    except ImportError:
+        import ConfigParser as configparser
+
+    # toml is rather similar to standard ini like format except it likes values
+    # that look more JSON like. So for our purposes simply escaping all values
+    # as JSON seem to work fine.
+
+    e = lambda s: json.dumps(s)
+
+    config = configparser.RawConfigParser()
+
+    # [target.ARCH-unknown-linux-gnu] in case of x86_64 [target.ARCH-poky-linux] 
+    target_section = "target.{}".format(d.getVar('TARGET_SYS', True))
+    config.add_section(target_section)
+
+    # Points to wrapper files which contain target specific compiler and linker commands. 
+    config.set(target_section, "cxx", e(d.expand("${RUST_TARGET_CXX}")))
+    config.set(target_section, "cc", e(d.expand("${RUST_TARGET_CC}")))
+    config.set(target_section, "linker", e(d.expand("${RUST_TARGET_CCLD}")))
+
+    # If we don't do this rust-native will compile it's own llvm for BUILD.
+    # [target.${BUILD_ARCH}-unknown-linux-gnu]
+    target_section = "target.{}".format(d.getVar('SNAPSHOT_BUILD_SYS', True))
+    config.add_section(target_section)
+
+    # Wrapper scripts of build system.
+    config.set(target_section, "cxx", e(d.expand("${RUST_BUILD_CXX}")))
+    config.set(target_section, "cc", e(d.expand("${RUST_BUILD_CC}")))
+
+    # [llvm]
+    config.add_section("llvm")
+    config.set("llvm", "targets", e("ARM;AArch64;Mips;PowerPC;RISCV;X86"))
+
+    # [rust]
+    config.add_section("rust")
+    config.set("rust", "rpath", e(True))
+    config.set("rust", "channel", e("stable"))
+
+    if LooseVersion(d.getVar("PV")) < LooseVersion("1.32.0"):
+        config.set("rust", "use-jemalloc", e(False))
+
+    # Whether or not to optimize the compiler and standard library
+    config.set("rust", "optimize", e(True))
+
+    # Emits extraneous output from tests to ensure that failures of the test
+    # harness are debuggable just from logfiles
+    config.set("rust", "verbose-tests", e(True))    
+
+    # Override default linker cc.
+    config.set("rust", "default-linker", e(d.expand("${RUST_BUILD_CCLD}")))
+
+    # [build]
+    config.add_section("build")
+    config.set("build", "submodules", e(False))
+    config.set("build", "docs", e(False))
+
+    rustc = d.expand("${WORKDIR}/rust-snapshot/bin/rustc")
+    config.set("build", "rustc", e(rustc))
+
+    cargo = d.expand("${WORKDIR}/rust-snapshot/bin/cargo")
+    config.set("build", "cargo", e(cargo))
+
+    config.set("build", "vendor", e(True))
+
+    targets = [d.getVar("TARGET_SYS", True)]
+    config.set("build", "target", e(targets))
+
+    hosts = [d.getVar("SNAPSHOT_BUILD_SYS", True)]
+    config.set("build", "host", e(hosts))
+
+    # We can't use BUILD_SYS since that is something the rust snapshot knows
+    # nothing about when trying to build some stage0 tools (like fabricate)
+    config.set("build", "build", e(d.getVar("SNAPSHOT_BUILD_SYS", True)))
+
+    with open("config.toml", "w") as f:
+        config.write(f)
+
+    # set up ${WORKDIR}/cargo_home
+    bb.build.exec_func("setup_cargo_environment", d)
+}
+
+
+rust_runx () {
+    echo "COMPILE ${PN}" "$@"
+
+    # CFLAGS, LDFLAGS, CXXFLAGS, CPPFLAGS are used by rust's build for a
+    # wide range of targets (not just TARGET). Yocto's settings for them will
+    # be inappropriate, avoid using.
+    unset CFLAGS
+    unset LDFLAGS
+    unset CXXFLAGS
+    unset CPPFLAGS
+
+    oe_cargo_fix_env
+
+    python3 src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose
+}
+rust_runx[vardepsexclude] += "PARALLEL_MAKE"
+
+do_compile () {
+
+    rust_runx build src/tools/remote-test-server --target "${TARGET_SYS}"
+}
+
+do_check[dirs] += "${B}"
+do_check[nostamp] = "1"
+do_check () {
+
+    rust_runx test --no-fail-fast --bless --target "${TARGET_SYS}" > summary.txt 2> /dev/null
+}
+addtask do_check after do_compile
-- 
2.17.1


  reply	other threads:[~2021-05-21 13:29 UTC|newest]

Thread overview: 90+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-09 11:59 [PATCH] Rust cross testing integration with oe-selftest Vinay Kumar
2021-04-10  9:50 ` [OE-core] " Alexander Kanavin
2021-04-15  9:32   ` [PATCH 1/8] We copied rust.inc to rust-test.inc file, since the rust.inc generates the rquired wrappers (which contains target compiler paths) in place. Which we can use in config.toml file for cross testing. We are trying to generate the config.toml through do_configure option in rust-test.inc file Vinay Kumar
2021-04-15  9:32     ` [PATCH 2/8] The function "do_compile" is to compile remote-test-server, and "do_check" to trigger the rust testing Vinay Kumar
2021-04-15  9:32     ` [PATCH 3/8] The rust.py is oe-selftest script to build remote-test-server, qemu-image and boot image in slirp mode. Once the image gets booted throgh "check" the testing starts by copying the "remote-test-server" in to the image Vinay Kumar
2021-04-15  9:32     ` [PATCH 4/8] In rust testing we have to boot image in slirp mode along with qemu monitor via telnet. Also, we are passing "-serial mon:stdio -serial" to fix runtime errors like, "failed with Connection reset by peer" observed during execution on image Vinay Kumar
2021-04-15  9:32     ` [PATCH 5/8] There are some error messages like, "error: linker `cc` not found" got fixed by adding "cc" in hosttools Vinay Kumar
2021-04-15  9:52       ` [OE-core] " Konrad Weihmann
2021-04-15 22:17       ` Richard Purdie
2021-05-17 10:35         ` [PATCH v2 1/7] rust-testsuite.inc : Rust cross testing Vinay Kumar
2021-05-17 10:35           ` [PATCH v2 2/7] Add rust testsuite for 1.46.0 Vinay Kumar
2021-05-17 10:35           ` [PATCH v2 3/7] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-17 10:35           ` [PATCH v2 4/7] runqemu: Add hostfwd for rust testing Vinay Kumar
2021-05-17 10:56             ` Alexander Kanavin
2021-05-17 10:35           ` [PATCH v2 5/7] oeqa/utils/qemurunner.py: Enable self.use_slirp Vinay Kumar
2021-05-17 10:45             ` Konrad Weihmann
2021-05-17 10:35           ` [PATCH v2 6/7] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-17 11:00             ` Alexander Kanavin
2021-05-17 11:34               ` Vinay Kumar
2021-05-17 11:46                 ` Alexander Kanavin
2021-05-17 13:20                   ` Vinay Kumar
2021-05-17 14:32                     ` Alexander Kanavin
2021-05-17 14:41                       ` Vinay Kumar
2021-05-17 15:27                         ` Vinay Kumar
     [not found]                         ` <167FE427910432AF.29970@lists.openembedded.org>
2021-05-20 10:06                           ` [OE-core] " Vinay Kumar
2021-05-20 20:29                             ` Alexander Kanavin
2021-05-20 20:30                               ` Alexander Kanavin
2021-05-21 13:29                                 ` Vinay Kumar [this message]
2021-05-21 13:29                                   ` [PATCH v3 2/5] Add rust testsuite for 1.46.0 Vinay Kumar
2021-05-21 13:29                                   ` [PATCH v3 3/5] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-21 13:29                                   ` [PATCH v3 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-22 16:34                                     ` Alexander Kanavin
2021-05-24  5:12                                       ` Vinay Kumar
2021-05-24 17:15                                       ` [PATCH v4 1/5] rust-testsuite.inc : Rust cross testing Vinay Kumar
2021-05-24 17:15                                         ` [PATCH v4 2/5] Add rust testsuite for 1.46.0 Vinay Kumar
2021-05-24 17:15                                         ` [PATCH v4 3/5] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-24 17:15                                         ` [PATCH v4 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-24 19:59                                           ` Alexander Kanavin
2021-05-26 20:43                                             ` Vinay Kumar
2021-05-27  8:10                                               ` [PATCH v5 1/5] rust-testsuite.inc : Rust cross testing Vinay Kumar
2021-05-27  8:10                                                 ` [PATCH v5 2/5] Add rust testsuite for 1.46.0 Vinay Kumar
2021-05-27  8:10                                                 ` [PATCH v5 3/5] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-27  8:10                                                 ` [PATCH v5 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-27  8:32                                                   ` Alexander Kanavin
     [not found]                                                   ` <1682DF4C7F380F13.5334@lists.openembedded.org>
2021-05-27  9:31                                                     ` [OE-core] " Alexander Kanavin
2021-05-27 12:03                                                   ` Alexander Kanavin
2021-05-27 12:18                                                     ` Vinay Kumar
2021-05-27 13:35                                                       ` Alexander Kanavin
2021-05-28 15:53                                                         ` [PATCH v6 1/5] rust-testsuite.inc : Rust cross testing Vinay Kumar
2021-05-28 15:53                                                           ` [PATCH v6 2/5] Add rust testsuite for 1.46.0 Vinay Kumar
2021-05-28 15:53                                                           ` [PATCH v6 3/5] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-28 15:53                                                           ` [PATCH v6 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-28 15:53                                                           ` [PATCH v6 5/5] rust.inc : Fix for aarch64 feature Vinay Kumar
2021-05-28 16:09                                                         ` [PATCH v7 1/5] rust-testsuite.inc : Rust cross testing Vinay Kumar
2021-05-28 16:09                                                           ` [PATCH v7 2/5] rust-testsuite_1.46.0.bb Vinay Kumar
2021-05-28 16:09                                                           ` [PATCH v7 3/5] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-28 16:09                                                           ` [PATCH v7 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-28 16:09                                                           ` [PATCH v7 5/5] rust.inc : Fix for aarch64 feature Vinay Kumar
2021-05-28 16:13                                                         ` [PATCH v5 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-28 21:10                                                           ` Alexander Kanavin
2021-05-29  9:18                                                             ` Vinay Kumar
2021-05-29  9:49                                                               ` Alexander Kanavin
2021-05-29 10:57                                                                 ` [PATCH v8 1/5] rust-testsuite.inc : Rust cross testing Vinay Kumar
2021-05-29 10:57                                                                   ` [PATCH v8 2/5] Add rust testsuite for 1.46.0 Vinay Kumar
2021-05-29 10:57                                                                   ` [PATCH v8 3/5] rust.inc : Changed target cpu for powerpc to "7400" Vinay Kumar
2021-05-29 10:57                                                                   ` [PATCH v8 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-29 10:57                                                                   ` [PATCH v8 5/5] While executing ui testing observed failures due to below warning, Vinay Kumar
2021-05-29 11:14                                                                 ` [PATCH v5 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-27  8:11                                                 ` [PATCH v5 5/5] rust.inc : Fix for aarch64 feature Vinay Kumar
2021-05-27  8:18                                               ` [PATCH v4 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-24 17:15                                         ` [PATCH v4 5/5] rust.inc : Fix for aarch64 feature Vinay Kumar
     [not found]                                       ` <1681E8AC31CF1367.29978@lists.openembedded.org>
2021-05-24 17:25                                         ` [OE-core] [PATCH v3 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-21 13:29                                   ` [PATCH v3 5/5] rust.inc : Fix for aarch64 feature Vinay Kumar
2021-05-22 16:11                                   ` [PATCH v3 1/5] rust-testsuite.inc : Rust cross testing Alexander Kanavin
2021-05-22 16:15                                     ` Alexander Kanavin
2021-05-21 13:46                                 ` [OE-core] [PATCH v2 6/7] oeqa/selftest/cases/rust.py: Rust oe-selftest script Vinay Kumar
2021-05-22 16:29                                   ` Alexander Kanavin
2021-05-17 10:35           ` [PATCH v2 7/7] rust.inc : Fix for aarch64 feature Vinay Kumar
2021-05-17 10:56           ` [PATCH v2 1/7] rust-testsuite.inc : Rust cross testing Alexander Kanavin
2021-05-17 11:04             ` Vinay Kumar
2021-05-17 11:05           ` Alexander Kanavin
2021-05-17 11:20             ` Vinay Kumar
2021-05-17 11:41               ` Alexander Kanavin
2021-05-17 22:18           ` [OE-core] " Khem Raj
2021-05-20 11:24             ` Vinay Kumar
2021-05-17 10:41         ` [OE-core] [PATCH 5/8] There are some error messages like, "error: linker `cc` not found" got fixed by adding "cc" in hosttools Vinay Kumar
2021-04-15  9:32     ` [PATCH 6/8] We are passing "hostfwd=tcp::12345-:12345" while booting image when slirp is selected. Also, we are using mapped port of ssh :22 to copy remote-test-server to image. We have added a logic to extract the port number whenever multiple instaces of qemu are running (ex., 2222 gets incremented by 1 if its already in use by another qemu) Vinay Kumar
2021-04-15  9:32     ` [PATCH 7/8] These changes are to get the rust target triplet "mips64-unknown-linux-gnuabi64" for mips64 Vinay Kumar
2021-04-15  9:32     ` [PATCH 8/8] In case of testing for "x86_64" Vinay Kumar
2021-04-15 14:30   ` [OE-core] [PATCH] Rust cross testing integration with oe-selftest Vinay Kumar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210521132914.140884-1-vinay.m.engg@gmail.com \
    --to=vinay.m.engg@gmail.com \
    --cc=alex.kanavin@gmail.com \
    --cc=kweihmann@outlook.com \
    --cc=openembedded-core@lists.openembedded.org \
    --cc=richard.purdie@linuxfoundation.org \
    --cc=rwmacleod@gmail.com \
    --cc=umesh.kalappa0@gmail.com \
    --cc=vinay.kumar@blackfigtech.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.