All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation
@ 2020-09-06 11:46 Alper Nebi Yasak
  2020-09-06 11:46 ` [PATCH v2 1/4] binman: Support cross-compiling test files to x86 Alper Nebi Yasak
                   ` (5 more replies)
  0 siblings, 6 replies; 11+ messages in thread
From: Alper Nebi Yasak @ 2020-09-06 11:46 UTC (permalink / raw)
  To: u-boot

Right now the 'binman test' command fails spectacularly on arm64 since
it cannot even setup the test environments properly due to errors during
setUpClass(). I can get a 100% coverage result with all tests passing if
I cross-compile things to x86 and run the cross-compiling versions of
some tools. This series tries to implement that solution in a general
way.

I think the thoroughly proper thing would be to make the tests and their
files portable. I don't really know how. Another alternative is to split
the test environments into multiple parts and skip the parts that can't
be prepared for an architecture, but that affects the test coverage
results. In any case, this series doesn't really prevent these other
solutions from being implemented on top of it.

Changes in v2:
- Fix detecting architecture (HOST_ARCH should've been HOSTARCH)
- Remove Makefile variables for unused tools
- Fix typo in an "e.g." (had used commas instead of dots)
- Add a table of target-specific versions in the docstring
- Add a table of host-specific versions in the docstring
- Add patch to document CROSS_COMPILE, CC, HOSTCC etc. in README
- Collect tags

v1: https://patchwork.ozlabs.org/project/uboot/list/?series=199707

Alper Nebi Yasak (4):
  binman: Support cross-compiling test files to x86
  binman: Use target-specific tools when cross-compiling
  binman: Allow resolving host-specific tools from env vars
  binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README

 tools/binman/README        |  24 +++++++
 tools/binman/elf.py        |   6 +-
 tools/binman/elf_test.py   |   4 +-
 tools/binman/test/Makefile |  15 ++++-
 tools/dtoc/fdt_util.py     |   9 +--
 tools/patman/tools.py      | 125 +++++++++++++++++++++++++++++++++++++
 6 files changed, 175 insertions(+), 8 deletions(-)

-- 
2.28.0

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 1/4] binman: Support cross-compiling test files to x86
  2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
@ 2020-09-06 11:46 ` Alper Nebi Yasak
  2020-09-06 13:58   ` Simon Glass
  2020-09-27  1:59   ` Simon Glass
  2020-09-06 11:46 ` [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling Alper Nebi Yasak
                   ` (4 subsequent siblings)
  5 siblings, 2 replies; 11+ messages in thread
From: Alper Nebi Yasak @ 2020-09-06 11:46 UTC (permalink / raw)
  To: u-boot

These test files are currently "intended for use on x86 hosts", but most
of the tests using them can still pass when cross-compiled to x86 on an
arm64 host.

This patch enables non-x86 hosts to run the tests by specifying a
cross-compiler via CROSS_COMPILE. The list of variables it sets is taken
from the top-level Makefile. It would be possible to automatically set
an x86 cross-compiler with a few blocks like:

    ifneq ($(shell i386-linux-gnu-gcc --version 2> /dev/null),)
    CROSS_COMPILE = i386-linux-gnu-
    endif

But it wouldn't propagate to the binman process calling this Makefile,
so it's better just raise an error and expect 'binman test' to be run
with a correct CROSS_COMPILE.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
---

Changes in v2:
- Fix detecting architecture (HOST_ARCH should've been HOSTARCH)
- Remove Makefile variables for unused tools

 tools/binman/test/Makefile | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/binman/test/Makefile b/tools/binman/test/Makefile
index e4fd97bb2e..0b19b7d993 100644
--- a/tools/binman/test/Makefile
+++ b/tools/binman/test/Makefile
@@ -7,6 +7,19 @@
 # SPDX-License-Identifier:      GPL-2.0+
 #
 
+HOSTARCH := $(shell uname -m | sed -e s/i.86/x86/ )
+ifeq ($(findstring $(HOSTARCH),"x86" "x86_64"),)
+ifeq ($(findstring $(MAKECMDGOALS),"help" "clean"),)
+ifndef CROSS_COMPILE
+$(error Binman tests need to compile to x86, but the CPU arch of your \
+	machine is $(HOSTARCH). Set CROSS_COMPILE to a suitable cross compiler)
+endif
+endif
+endif
+
+CC		= $(CROSS_COMPILE)gcc
+OBJCOPY		= $(CROSS_COMPILE)objcopy
+
 VPATH := $(SRC)
 CFLAGS := -march=i386 -m32 -nostdlib -I $(SRC)../../../include \
 	-Wl,--no-dynamic-linker
@@ -32,7 +45,7 @@ bss_data: CFLAGS += $(SRC)bss_data.lds
 bss_data: bss_data.c
 
 u_boot_binman_syms.bin: u_boot_binman_syms
-	objcopy -O binary $< -R .note.gnu.build-id $@
+	$(OBJCOPY) -O binary $< -R .note.gnu.build-id $@
 
 u_boot_binman_syms: CFLAGS += $(LDS_BINMAN)
 u_boot_binman_syms: u_boot_binman_syms.c
-- 
2.28.0

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling
  2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
  2020-09-06 11:46 ` [PATCH v2 1/4] binman: Support cross-compiling test files to x86 Alper Nebi Yasak
@ 2020-09-06 11:46 ` Alper Nebi Yasak
  2020-09-06 11:46 ` [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars Alper Nebi Yasak
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Alper Nebi Yasak @ 2020-09-06 11:46 UTC (permalink / raw)
  To: u-boot

Currently, binman always runs the compile tools like cc, objcopy, strip,
etc. using their literal name. Instead, this patch makes it use the
target-specific versions by default, derived from the tool-specific
environment variables (CC, OBJCOPY, STRIP, etc.) or from the
CROSS_COMPILE environment variable.

For example, the u-boot-elf etype directly uses 'strip'. Trying to run
the tests with 'CROSS_COMPILE=i686-linux-gnu- binman test' on an arm64
host results in the '097_elf_strip.dts' test to fail as the arm64
version of 'strip' can't understand the format of the x86 ELF file.

This also adjusts some command.Output() calls that caused test errors or
failures to use the target versions of the tools they call. After this,
patch, an arm64 host can run all tests with no errors or failures using
a correct CROSS_COMPILE value.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
---

Changes in v2:
- Fix typo in an "e.g." (had used commas instead of dots)
- Add a table of target-specific versions in the docstring
- Add tag: "Reviewed-by: Simon Glass <sjg@chromium.org>"

 tools/binman/elf.py      |  6 ++--
 tools/binman/elf_test.py |  4 ++-
 tools/dtoc/fdt_util.py   |  9 ++---
 tools/patman/tools.py    | 77 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+), 7 deletions(-)

diff --git a/tools/binman/elf.py b/tools/binman/elf.py
index f88031c2bf..5e566e56cb 100644
--- a/tools/binman/elf.py
+++ b/tools/binman/elf.py
@@ -234,8 +234,10 @@ SECTIONS
     #   text section at the start
     # -m32: Build for 32-bit x86
     # -T...: Specifies the link script, which sets the start address
-    stdout = command.Output('cc', '-static', '-nostdlib', '-Wl,--build-id=none',
-                            '-m32','-T', lds_file, '-o', elf_fname, s_file)
+    cc, args = tools.GetTargetCompileTool('cc')
+    args += ['-static', '-nostdlib', '-Wl,--build-id=none', '-m32', '-T',
+            lds_file, '-o', elf_fname, s_file]
+    stdout = command.Output(cc, *args)
     shutil.rmtree(outdir)
 
 def DecodeElf(data, location):
diff --git a/tools/binman/elf_test.py b/tools/binman/elf_test.py
index 37e1b423cf..e3d218a89e 100644
--- a/tools/binman/elf_test.py
+++ b/tools/binman/elf_test.py
@@ -186,7 +186,9 @@ class TestElf(unittest.TestCase):
         # Make an Elf file and then convert it to a fkat binary file. This
         # should produce the original data.
         elf.MakeElf(elf_fname, expected_text, expected_data)
-        stdout = command.Output('objcopy', '-O', 'binary', elf_fname, bin_fname)
+        objcopy, args = tools.GetTargetCompileTool('objcopy')
+        args += ['-O', 'binary', elf_fname, bin_fname]
+        stdout = command.Output(objcopy, *args)
         with open(bin_fname, 'rb') as fd:
             data = fd.read()
         self.assertEqual(expected_text + expected_data, data)
diff --git a/tools/dtoc/fdt_util.py b/tools/dtoc/fdt_util.py
index b040793772..37e96b9864 100644
--- a/tools/dtoc/fdt_util.py
+++ b/tools/dtoc/fdt_util.py
@@ -68,22 +68,23 @@ def EnsureCompiled(fname, tmpdir=None, capture_stderr=False):
 
     search_paths = [os.path.join(os.getcwd(), 'include')]
     root, _ = os.path.splitext(fname)
-    args = ['-E', '-P', '-x', 'assembler-with-cpp', '-D__ASSEMBLY__']
+    cc, args = tools.GetTargetCompileTool('cc')
+    args += ['-E', '-P', '-x', 'assembler-with-cpp', '-D__ASSEMBLY__']
     args += ['-Ulinux']
     for path in search_paths:
         args.extend(['-I', path])
     args += ['-o', dts_input, fname]
-    command.Run('cc', *args)
+    command.Run(cc, *args)
 
     # If we don't have a directory, put it in the tools tempdir
     search_list = []
     for path in search_paths:
         search_list.extend(['-i', path])
-    args = ['-I', 'dts', '-o', dtb_output, '-O', 'dtb',
+    dtc, args = tools.GetTargetCompileTool('dtc')
+    args += ['-I', 'dts', '-o', dtb_output, '-O', 'dtb',
             '-W', 'no-unit_address_vs_reg']
     args.extend(search_list)
     args.append(dts_input)
-    dtc = os.environ.get('DTC') or 'dtc'
     command.Run(dtc, *args, capture_stderr=capture_stderr)
     return dtb_output
 
diff --git a/tools/patman/tools.py b/tools/patman/tools.py
index d41115a22c..66f6ab7af0 100644
--- a/tools/patman/tools.py
+++ b/tools/patman/tools.py
@@ -188,6 +188,77 @@ def PathHasFile(path_spec, fname):
             return True
     return False
 
+def GetTargetCompileTool(name, cross_compile=None):
+    """Get the target-specific version for a compile tool
+
+    This first checks the environment variables that specify which
+    version of the tool should be used (e.g. ${CC}). If those aren't
+    specified, it checks the CROSS_COMPILE variable as a prefix for the
+    tool with some substitutions (e.g. "${CROSS_COMPILE}gcc" for cc).
+
+    The following table lists the target-specific versions of the tools
+    this function resolves to:
+
+        Compile Tool  | First choice   | Second choice
+        --------------+----------------+----------------------------
+        as            |  ${AS}         | ${CROSS_COMPILE}as
+        ld            |  ${LD}         | ${CROSS_COMPILE}ld.bfd
+                      |                |   or ${CROSS_COMPILE}ld
+        cc            |  ${CC}         | ${CROSS_COMPILE}gcc
+        cpp           |  ${CPP}        | ${CROSS_COMPILE}gcc -E
+        c++           |  ${CXX}        | ${CROSS_COMPILE}g++
+        ar            |  ${AR}         | ${CROSS_COMPILE}ar
+        nm            |  ${NM}         | ${CROSS_COMPILE}nm
+        ldr           |  ${LDR}        | ${CROSS_COMPILE}ldr
+        strip         |  ${STRIP}      | ${CROSS_COMPILE}strip
+        objcopy       |  ${OBJCOPY}    | ${CROSS_COMPILE}objcopy
+        objdump       |  ${OBJDUMP}    | ${CROSS_COMPILE}objdump
+        dtc           |  ${DTC}        | (no CROSS_COMPILE version)
+
+    Args:
+        name: Command name to run
+
+    Returns:
+        target_name: Exact command name to run instead
+        extra_args: List of extra arguments to pass
+    """
+    env = dict(os.environ)
+
+    target_name = None
+    extra_args = []
+    if name in ('as', 'ld', 'cc', 'cpp', 'ar', 'nm', 'ldr', 'strip',
+                'objcopy', 'objdump', 'dtc'):
+        target_name, *extra_args = env.get(name.upper(), '').split(' ')
+    elif name == 'c++':
+        target_name, *extra_args = env.get('CXX', '').split(' ')
+
+    if target_name:
+        return target_name, extra_args
+
+    if cross_compile is None:
+        cross_compile = env.get('CROSS_COMPILE', '')
+    if not cross_compile:
+        return name, []
+
+    if name in ('as', 'ar', 'nm', 'ldr', 'strip', 'objcopy', 'objdump'):
+        target_name = cross_compile + name
+    elif name == 'ld':
+        try:
+            if Run(cross_compile + 'ld.bfd', '-v'):
+                target_name = cross_compile + 'ld.bfd'
+        except:
+            target_name = cross_compile + 'ld'
+    elif name == 'cc':
+        target_name = cross_compile + 'gcc'
+    elif name == 'cpp':
+        target_name = cross_compile + 'gcc'
+        extra_args = ['-E']
+    elif name == 'c++':
+        target_name = cross_compile + 'g++'
+    else:
+        target_name = name
+    return target_name, extra_args
+
 def Run(name, *args, **kwargs):
     """Run a tool with some arguments
 
@@ -198,16 +269,22 @@ def Run(name, *args, **kwargs):
     Args:
         name: Command name to run
         args: Arguments to the tool
+        for_target: False to run the command as-is, without resolving it
+                   to the version for the compile target
 
     Returns:
         CommandResult object
     """
     try:
         binary = kwargs.get('binary')
+        for_target = kwargs.get('for_target', True)
         env = None
         if tool_search_paths:
             env = dict(os.environ)
             env['PATH'] = ':'.join(tool_search_paths) + ':' + env['PATH']
+        if for_target:
+            name, extra_args = GetTargetCompileTool(name)
+            args = tuple(extra_args) + args
         all_args = (name,) + args
         result = command.RunPipe([all_args], capture=True, capture_stderr=True,
                                  env=env, raise_on_error=False, binary=binary)
-- 
2.28.0

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars
  2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
  2020-09-06 11:46 ` [PATCH v2 1/4] binman: Support cross-compiling test files to x86 Alper Nebi Yasak
  2020-09-06 11:46 ` [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling Alper Nebi Yasak
@ 2020-09-06 11:46 ` Alper Nebi Yasak
  2020-09-06 11:46 ` [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README Alper Nebi Yasak
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Alper Nebi Yasak @ 2020-09-06 11:46 UTC (permalink / raw)
  To: u-boot

This patch lets tools.Run() use host-specific versions with the
for_host keyword argument, based on the host-specific environment
variables (HOSTCC, HOSTOBJCOPY, HOSTSTRIP, etc.).

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
---

Changes in v2:
- Add a table of host-specific versions in the docstring
- Add tag: "Reviewed-by: Simon Glass <sjg@chromium.org>"

 tools/patman/tools.py | 50 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/tools/patman/tools.py b/tools/patman/tools.py
index 66f6ab7af0..bbb157da87 100644
--- a/tools/patman/tools.py
+++ b/tools/patman/tools.py
@@ -188,6 +188,49 @@ def PathHasFile(path_spec, fname):
             return True
     return False
 
+def GetHostCompileTool(name):
+    """Get the host-specific version for a compile tool
+
+    This checks the environment variables that specify which version of
+    the tool should be used (e.g. ${HOSTCC}).
+
+    The following table lists the host-specific versions of the tools
+    this function resolves to:
+
+        Compile Tool  | Host version
+        --------------+----------------
+        as            |  ${HOSTAS}
+        ld            |  ${HOSTLD}
+        cc            |  ${HOSTCC}
+        cpp           |  ${HOSTCPP}
+        c++           |  ${HOSTCXX}
+        ar            |  ${HOSTAR}
+        nm            |  ${HOSTNM}
+        ldr           |  ${HOSTLDR}
+        strip         |  ${HOSTSTRIP}
+        objcopy       |  ${HOSTOBJCOPY}
+        objdump       |  ${HOSTOBJDUMP}
+        dtc           |  ${HOSTDTC}
+
+    Args:
+        name: Command name to run
+
+    Returns:
+        host_name: Exact command name to run instead
+        extra_args: List of extra arguments to pass
+    """
+    host_name = None
+    extra_args = []
+    if name in ('as', 'ld', 'cc', 'cpp', 'ar', 'nm', 'ldr', 'strip',
+                'objcopy', 'objdump', 'dtc'):
+        host_name, *host_args = env.get('HOST' + name.upper(), '').split(' ')
+    elif name == 'c++':
+        host_name, *host_args = env.get('HOSTCXX', '').split(' ')
+
+    if host_name:
+        return host_name, extra_args
+    return name, []
+
 def GetTargetCompileTool(name, cross_compile=None):
     """Get the target-specific version for a compile tool
 
@@ -269,6 +312,7 @@ def Run(name, *args, **kwargs):
     Args:
         name: Command name to run
         args: Arguments to the tool
+        for_host: True to resolve the command to the version for the host
         for_target: False to run the command as-is, without resolving it
                    to the version for the compile target
 
@@ -277,7 +321,8 @@ def Run(name, *args, **kwargs):
     """
     try:
         binary = kwargs.get('binary')
-        for_target = kwargs.get('for_target', True)
+        for_host = kwargs.get('for_host', False)
+        for_target = kwargs.get('for_target', not for_host)
         env = None
         if tool_search_paths:
             env = dict(os.environ)
@@ -285,6 +330,9 @@ def Run(name, *args, **kwargs):
         if for_target:
             name, extra_args = GetTargetCompileTool(name)
             args = tuple(extra_args) + args
+        elif for_host:
+            name, extra_args = GetHostCompileTool(name)
+            args = tuple(extra_args) + args
         all_args = (name,) + args
         result = command.RunPipe([all_args], capture=True, capture_stderr=True,
                                  env=env, raise_on_error=False, binary=binary)
-- 
2.28.0

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README
  2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
                   ` (2 preceding siblings ...)
  2020-09-06 11:46 ` [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars Alper Nebi Yasak
@ 2020-09-06 11:46 ` Alper Nebi Yasak
  2020-09-06 13:58   ` Simon Glass
  2020-09-27  1:59   ` Simon Glass
  2020-09-27  1:59 ` [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars Simon Glass
  2020-09-27  1:59 ` [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling Simon Glass
  5 siblings, 2 replies; 11+ messages in thread
From: Alper Nebi Yasak @ 2020-09-06 11:46 UTC (permalink / raw)
  To: u-boot

Explain that binman interprets these environment variables in the
"External tools" section to run target/host specific versions of the
tools, and add a new section on how to use CROSS_COMPILE to run the
tests on non-x86 machines.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
---

Changes in v2:
- Added this new patch

 tools/binman/README | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/tools/binman/README b/tools/binman/README
index 37ee3fc2d3..dfa0ed6a09 100644
--- a/tools/binman/README
+++ b/tools/binman/README
@@ -884,6 +884,12 @@ the 'tools' module's Run() method. The tools generally must exist on the PATH,
 but the --toolpath option can be used to specify additional search paths to
 use. This option can be specified multiple times to add more than one path.
 
+For some compile tools binman will use the versions specified by commonly-used
+environment variables like CC and HOSTCC for the C compiler, based on whether
+the tool's output will be used for the target or for the host machine. If those
+aren't given, it will also try to derive target-specific versions from the
+CROSS_COMPILE environment variable during a cross-compilation.
+
 
 Code coverage
 -------------
@@ -918,6 +924,24 @@ directories so they can be examined later. Use -X or --test-preserve-dirs for
 this.
 
 
+Running tests on non-x86 architectures
+--------------------------------------
+
+Binman's tests have been written under the assumption that they'll be run on a
+x86-like host and there hasn't been an attempt to make them portable yet.
+However, it's possible to run the tests by cross-compiling to x86.
+
+To install an x86 cross-compiler on Debian-type distributions (e.g. Ubuntu):
+
+  $ sudo apt-get install gcc-x86-64-linux-gnu
+
+Then, you can run the tests under cross-compilation:
+
+  $ CROSS_COMPILE=x86_64-linux-gnu- binman test -T
+
+You can also use gcc-i686-linux-gnu similar to the above.
+
+
 Advanced Features / Technical docs
 ----------------------------------
 
-- 
2.28.0

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 1/4] binman: Support cross-compiling test files to x86
  2020-09-06 11:46 ` [PATCH v2 1/4] binman: Support cross-compiling test files to x86 Alper Nebi Yasak
@ 2020-09-06 13:58   ` Simon Glass
  2020-09-27  1:59   ` Simon Glass
  1 sibling, 0 replies; 11+ messages in thread
From: Simon Glass @ 2020-09-06 13:58 UTC (permalink / raw)
  To: u-boot

On Sun, 6 Sep 2020 at 05:46, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote:
>
> These test files are currently "intended for use on x86 hosts", but most
> of the tests using them can still pass when cross-compiled to x86 on an
> arm64 host.
>
> This patch enables non-x86 hosts to run the tests by specifying a
> cross-compiler via CROSS_COMPILE. The list of variables it sets is taken
> from the top-level Makefile. It would be possible to automatically set
> an x86 cross-compiler with a few blocks like:
>
>     ifneq ($(shell i386-linux-gnu-gcc --version 2> /dev/null),)
>     CROSS_COMPILE = i386-linux-gnu-
>     endif
>
> But it wouldn't propagate to the binman process calling this Makefile,
> so it's better just raise an error and expect 'binman test' to be run
> with a correct CROSS_COMPILE.
>
> Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
> ---
>
> Changes in v2:
> - Fix detecting architecture (HOST_ARCH should've been HOSTARCH)
> - Remove Makefile variables for unused tools
>
>  tools/binman/test/Makefile | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README
  2020-09-06 11:46 ` [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README Alper Nebi Yasak
@ 2020-09-06 13:58   ` Simon Glass
  2020-09-27  1:59   ` Simon Glass
  1 sibling, 0 replies; 11+ messages in thread
From: Simon Glass @ 2020-09-06 13:58 UTC (permalink / raw)
  To: u-boot

On Sun, 6 Sep 2020 at 05:46, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote:
>
> Explain that binman interprets these environment variables in the
> "External tools" section to run target/host specific versions of the
> tools, and add a new section on how to use CROSS_COMPILE to run the
> tests on non-x86 machines.
>
> Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
> ---
>
> Changes in v2:
> - Added this new patch
>
>  tools/binman/README | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)

Reviewed-by: Simon Glass <sjg@chromium.org>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README
  2020-09-06 11:46 ` [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README Alper Nebi Yasak
  2020-09-06 13:58   ` Simon Glass
@ 2020-09-27  1:59   ` Simon Glass
  1 sibling, 0 replies; 11+ messages in thread
From: Simon Glass @ 2020-09-27  1:59 UTC (permalink / raw)
  To: u-boot

On Sun, 6 Sep 2020 at 05:46, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote:
>
> Explain that binman interprets these environment variables in the
> "External tools" section to run target/host specific versions of the
> tools, and add a new section on how to use CROSS_COMPILE to run the
> tests on non-x86 machines.
>
> Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
> ---
>
> Changes in v2:
> - Added this new patch
>
>  tools/binman/README | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)

Reviewed-by: Simon Glass <sjg@chromium.org>

Applied to u-boot-dm/next, thanks!

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars
  2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
                   ` (3 preceding siblings ...)
  2020-09-06 11:46 ` [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README Alper Nebi Yasak
@ 2020-09-27  1:59 ` Simon Glass
  2020-09-27  1:59 ` [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling Simon Glass
  5 siblings, 0 replies; 11+ messages in thread
From: Simon Glass @ 2020-09-27  1:59 UTC (permalink / raw)
  To: u-boot

This patch lets tools.Run() use host-specific versions with the
for_host keyword argument, based on the host-specific environment
variables (HOSTCC, HOSTOBJCOPY, HOSTSTRIP, etc.).

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
---

Changes in v2:
- Add a table of host-specific versions in the docstring
- Add tag: "Reviewed-by: Simon Glass <sjg@chromium.org>"

 tools/patman/tools.py | 50 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

Applied to u-boot-dm/next, thanks!

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling
  2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
                   ` (4 preceding siblings ...)
  2020-09-27  1:59 ` [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars Simon Glass
@ 2020-09-27  1:59 ` Simon Glass
  5 siblings, 0 replies; 11+ messages in thread
From: Simon Glass @ 2020-09-27  1:59 UTC (permalink / raw)
  To: u-boot

Currently, binman always runs the compile tools like cc, objcopy, strip,
etc. using their literal name. Instead, this patch makes it use the
target-specific versions by default, derived from the tool-specific
environment variables (CC, OBJCOPY, STRIP, etc.) or from the
CROSS_COMPILE environment variable.

For example, the u-boot-elf etype directly uses 'strip'. Trying to run
the tests with 'CROSS_COMPILE=i686-linux-gnu- binman test' on an arm64
host results in the '097_elf_strip.dts' test to fail as the arm64
version of 'strip' can't understand the format of the x86 ELF file.

This also adjusts some command.Output() calls that caused test errors or
failures to use the target versions of the tools they call. After this,
patch, an arm64 host can run all tests with no errors or failures using
a correct CROSS_COMPILE value.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
---

Changes in v2:
- Fix typo in an "e.g." (had used commas instead of dots)
- Add a table of target-specific versions in the docstring
- Add tag: "Reviewed-by: Simon Glass <sjg@chromium.org>"

 tools/binman/elf.py      |  6 ++--
 tools/binman/elf_test.py |  4 ++-
 tools/dtoc/fdt_util.py   |  9 ++---
 tools/patman/tools.py    | 77 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+), 7 deletions(-)

Applied to u-boot-dm/next, thanks!

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 1/4] binman: Support cross-compiling test files to x86
  2020-09-06 11:46 ` [PATCH v2 1/4] binman: Support cross-compiling test files to x86 Alper Nebi Yasak
  2020-09-06 13:58   ` Simon Glass
@ 2020-09-27  1:59   ` Simon Glass
  1 sibling, 0 replies; 11+ messages in thread
From: Simon Glass @ 2020-09-27  1:59 UTC (permalink / raw)
  To: u-boot

On Sun, 6 Sep 2020 at 05:46, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote:
>
> These test files are currently "intended for use on x86 hosts", but most
> of the tests using them can still pass when cross-compiled to x86 on an
> arm64 host.
>
> This patch enables non-x86 hosts to run the tests by specifying a
> cross-compiler via CROSS_COMPILE. The list of variables it sets is taken
> from the top-level Makefile. It would be possible to automatically set
> an x86 cross-compiler with a few blocks like:
>
>     ifneq ($(shell i386-linux-gnu-gcc --version 2> /dev/null),)
>     CROSS_COMPILE = i386-linux-gnu-
>     endif
>
> But it wouldn't propagate to the binman process calling this Makefile,
> so it's better just raise an error and expect 'binman test' to be run
> with a correct CROSS_COMPILE.
>
> Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
> ---
>
> Changes in v2:
> - Fix detecting architecture (HOST_ARCH should've been HOSTARCH)
> - Remove Makefile variables for unused tools
>
>  tools/binman/test/Makefile | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

Applied to u-boot-dm/next, thanks!

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2020-09-27  1:59 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-06 11:46 [PATCH v2 0/4] binman: Make tests work on non-x86 architectures via cross-compilation Alper Nebi Yasak
2020-09-06 11:46 ` [PATCH v2 1/4] binman: Support cross-compiling test files to x86 Alper Nebi Yasak
2020-09-06 13:58   ` Simon Glass
2020-09-27  1:59   ` Simon Glass
2020-09-06 11:46 ` [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling Alper Nebi Yasak
2020-09-06 11:46 ` [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars Alper Nebi Yasak
2020-09-06 11:46 ` [PATCH v2 4/4] binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README Alper Nebi Yasak
2020-09-06 13:58   ` Simon Glass
2020-09-27  1:59   ` Simon Glass
2020-09-27  1:59 ` [PATCH v2 3/4] binman: Allow resolving host-specific tools from env vars Simon Glass
2020-09-27  1:59 ` [PATCH v2 2/4] binman: Use target-specific tools when cross-compiling Simon Glass

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.