[PATCH 0/4] Fix 32-bit builds of CHIPSEC

[PATCH 0/4] Fix 32-bit builds of CHIPSEC

[Ricardo Neri]

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

Our build bot discovered broken builds of CHIPSEC in qemux86. This series
fixes these issues. In a nutshell, the CHIPSEC driver needs to be aware
of the CPU architecture to build the driver differently. Also, given
that CHIPSEC provides an out-of-tree module, Linux needs extra tasks performed
before CHIPSEC can be built.

Thanks and BR,
Ricardo

Ricardo Neri (4):
  chipsec: make chipsec recipe depend on make_scripts
  chipsec: Provide CHIPSEC's driver with the build architecture
  chipsec: distutils' build arguments
  chipsec: Ship 64-bit binary files only in 64-bit packages

 ...ve-CPU-architecture-to-the-driver-s-Makef.patch | 40 ++++++++++++++++++++++
 meta-luv/recipes-core/chipsec/chipsec_git.bb       | 14 +++++++-
 meta/lib/oe/terminal.py                            |  3 +-
 3 files changed, 55 insertions(+), 2 deletions(-)
 create mode 100644 meta-luv/recipes-core/chipsec/chipsec/0001-setup.py-give-CPU-architecture-to-the-driver-s-Makef.patch

-- 
2.9.3

[PATCH 1/4] chipsec: make chipsec recipe depend on make_scripts

[Ricardo Neri]

[-- Attachment #1: Type: text/plain, Size: 1687 bytes --]

In order to out-of-tree build modules properly, it is necessary to invoke
make prepare && make scripts in the Linux source tree. Otherwise, the
module will not know all the appropriate kernel configurations. The task
make_scripts, inherited from module-base.bbclass, takes care of it.

CHIPSEC includes an out-of-tree module. Thus, make_scripts should run
before our do_compile task. Since in Yocto the Linux build artifacts are
put in a separate directory, controlled by the kernel's do_shared_workdir
task, such task must run before make_scripts.

Lastly, we hold a lock to the kernel-scripts task.

The issue of not running make_scripts before do_compile manifests itself
as:

ERROR: Kernel configuration is invalid.
      include/generated/autoconf.h or include/config/auto.conf are missing.
      Run 'make oldconfig && make prepare' on kernel src to fix it.

Cc: Megha Dey <megha.dey@intel.com>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
 meta-luv/recipes-core/chipsec/chipsec_git.bb | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta-luv/recipes-core/chipsec/chipsec_git.bb b/meta-luv/recipes-core/chipsec/chipsec_git.bb
index d6c1bef..6c29231 100644
--- a/meta-luv/recipes-core/chipsec/chipsec_git.bb
+++ b/meta-luv/recipes-core/chipsec/chipsec_git.bb
@@ -28,6 +28,10 @@ inherit python-dir
 inherit distutils
 inherit luv-test
 
+addtask make_scripts after do_patch before do_compile
+do_make_scripts[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
+do_make_scripts[depends] += "virtual/kernel:do_shared_workdir"
+
 S = "${WORKDIR}/git"
 
 export INC = "-I${STAGING_INCDIR}/${PYTHON_DIR}"
-- 
2.9.3

[PATCH 2/4] chipsec: Provide CHIPSEC's driver with the build architecture

[Ricardo Neri]

[-- Attachment #1: Type: text/plain, Size: 4012 bytes --]

When cross-building for 32-bit x86, we need to make aware the CHIPSEC
driver that we intend to build the 32-bit version via the ARCH variable.
A quick fix is to define an environment varialble, CHIPSEC_ARCH, that
can be retrieved from setup.py and passed to the driver Makefile.

We no longer the EXTRA_OEMAKE variable as we rely on Python setuptools
to build CHIPSEC.

Cc: Megha Dey <megha.dey@intel.com>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
 ...ve-CPU-architecture-to-the-driver-s-Makef.patch | 40 ++++++++++++++++++++++
 meta-luv/recipes-core/chipsec/chipsec_git.bb       |  3 +-
 2 files changed, 42 insertions(+), 1 deletion(-)
 create mode 100644 meta-luv/recipes-core/chipsec/chipsec/0001-setup.py-give-CPU-architecture-to-the-driver-s-Makef.patch

diff --git a/meta-luv/recipes-core/chipsec/chipsec/0001-setup.py-give-CPU-architecture-to-the-driver-s-Makef.patch b/meta-luv/recipes-core/chipsec/chipsec/0001-setup.py-give-CPU-architecture-to-the-driver-s-Makef.patch
new file mode 100644
index 0000000..4fe7f10
--- /dev/null
+++ b/meta-luv/recipes-core/chipsec/chipsec/0001-setup.py-give-CPU-architecture-to-the-driver-s-Makef.patch
@@ -0,0 +1,40 @@
+From 4df09a6f905c9e1cf8c65235ff309ac7bebcc6eb Mon Sep 17 00:00:00 2001
+From: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
+Date: Wed, 15 Mar 2017 12:00:57 -0700
+Subject: [PATCH] setup.py: give CPU architecture to the driver's Makefile
+
+Driver is built differently depending on whether it is built for
+32-bit or 64-bit x86 archs. Thus, we need to pass the desired
+CPU architecture. This can be done by reading the environment
+variable CHIPSEC_ARCH. This could be done more nicely by adding
+a new variable to build_ext, though.
+
+Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
+---
+ setup.py | 5 +++--
+ 1 file changed, 3 insertions(+), 2 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index fb0da43..e09eaa6 100644
+--- a/setup.py
++++ b/setup.py
+@@ -68,13 +68,14 @@ class build_ext(_build_ext):
+         self.set_undefined_options("build", ("skip_driver", "skip_driver"))
+ 
+     def _build_linux_driver(self):
+-        log.info("building the linux driver")
++        arch = os.environ["CHIPSEC_ARCH"]
++        log.info("building the linux driver for arch " + arch)
+         build_driver = os.path.join(self.real_build_lib, "drivers", "linux")
+         ko_ext = os.path.join(build_driver, "chipsec.ko")
+         # We copy the drivers extension to the build directory.
+         self.copy_tree(os.path.join("drivers", "linux"), build_driver)
+         # Run the makefile there.
+-        subprocess.check_output(["make", "-C", build_driver])
++        subprocess.check_output(["make", "ARCH="+arch, "-C", build_driver])
+         # And copy the resulting .ko to the right place.
+         # That is to the source directory if we are in "develop" mode,
+         # otherwise to the helper subdirectory in the build directory.
+-- 
+2.9.3
+
diff --git a/meta-luv/recipes-core/chipsec/chipsec_git.bb b/meta-luv/recipes-core/chipsec/chipsec_git.bb
index 6c29231..1a209e7 100644
--- a/meta-luv/recipes-core/chipsec/chipsec_git.bb
+++ b/meta-luv/recipes-core/chipsec/chipsec_git.bb
@@ -11,6 +11,7 @@ SRC_URI = "git://github.com/chipsec/chipsec.git \
     file://chipsec file://luv-parser-chipsec \
     file://0001-chipsec-building-for-32-bit-systems.patch \
     file://0001-chipsec-do-not-ship-manual.patch \
+    file://0001-setup.py-give-CPU-architecture-to-the-driver-s-Makef.patch \
     "
 
 SRCREV="2fbb7a4eeb3752998dfbcbdf771cd3f8ed1985ae"
@@ -48,7 +49,7 @@ def get_target_arch(d):
  else:
     raise bb.parse.SkipPackage("TARGET_ARCH %s not supported!" % target)
 
-EXTRA_OEMAKE += "ARCH="${@get_target_arch(d)}""
+export CHIPSEC_ARCH = "${@get_target_arch(d)}"
 
 DISTUTILS_INSTALL_ARGS = "--root=${D}${PYTHON_SITEPACKAGES_DIR} \
     --install-data=${D}/${datadir}"
-- 
2.9.3

[PATCH 3/4] chipsec: distutils' build arguments

[Ricardo Neri]

[-- Attachment #1: Type: text/plain, Size: 1135 bytes --]

Utilize DISTUTILS_BUILD_ARGS variable to give the name of the setup.py
build library to use. This is mostly a cosmetic change: it looks odd
that the library name is linux-x86_64-2.7 when building for 32-bit.

Cc: Megha Dey <megha.dey@intel.com>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
 meta-luv/recipes-core/chipsec/chipsec_git.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta-luv/recipes-core/chipsec/chipsec_git.bb b/meta-luv/recipes-core/chipsec/chipsec_git.bb
index 1a209e7..4778a03 100644
--- a/meta-luv/recipes-core/chipsec/chipsec_git.bb
+++ b/meta-luv/recipes-core/chipsec/chipsec_git.bb
@@ -54,6 +54,8 @@ export CHIPSEC_ARCH = "${@get_target_arch(d)}"
 DISTUTILS_INSTALL_ARGS = "--root=${D}${PYTHON_SITEPACKAGES_DIR} \
     --install-data=${D}/${datadir}"
 
+DISTUTILS_BUILD_ARGS = "--build-lib=${TARGET_ARCH}"
+
 fix_mod_path() {
     sed -i -e "s:^INSTALL_MOD_PATH_PREFIX = .*:INSTALL_MOD_PATH_PREFIX = \"${PYTHON_SITEPACKAGES_DIR}\":" ${S}/chipsec_main.py
     sed -i -e "s:PYTHONPATH:${PYTHON_SITEPACKAGES_DIR}/chipsec:" ${WORKDIR}/chipsec
-- 
2.9.3

[PATCH 4/4] chipsec: Ship 64-bit binary files only in 64-bit packages

[Ricardo Neri]

[-- Attachment #1: Type: text/plain, Size: 3302 bytes --]

The files TianoCompress.bin and LzmaCompress.bin are distributed
in binary form. It's format is ELF64. Thus, only ship them for 64-bit
builds. They could not be used in 32-bit builds anyway. We also avoid this
package quality error:

ERROR: chipsec-1.2.5+-r0 do_package_qa: QA Issue: Architecture did not match
(x86-64, expected x86) on /work/qemux86-oe-linux/chipsec/1.2.5+-r0/packages-
split/chipsec/usr/lib/python2.7/site-packages/chipsec/chipsec_tools/linux/
TianoCompress.bin
Architecture did not match (x86-64, expected x86) on /work/qemux86-oe-linux/
chipsec/1.2.5+-r0/packages-split/chipsec/usr/lib/python2.7/site-packages/
chipsec/chipsec_tools/linux/LzmaCompress.bin [arch]
ERROR: chipsec-1.2.5+-r0 do_package_qa: QA Issue: Architecture did not match
(x86-64, expected x86) on /work/qemux86-oe-linux/chipsec/1.2.5+-r0/packages-
split/chipsec-dbg/usr/lib/python2.7/site-packages/chipsec/chipsec_tools/
linux/.debug/TianoCompress.bin
Architecture did not match (x86-64, expected x86) on /work/qemux86-oe-linux
/chipsec/1.2.5+-r0/packages-split/chipsec-dbg/usr/lib/python2.7/site-
packages/chipsec/chipsec_tools/linux/.debug/LzmaCompress.bin [arch]
ERROR: chipsec-1.2.5+-r0 do_package_qa: QA run found fatal errors.
Please consider fixing them.
ERROR: chipsec-1.2.5+-r0 do_package_qa: Function failed: do_package_qa
ERROR: Logfile of failure stored in: /data/code/luv-yocto-32/build/tmp/
work/qemux86-oe-linux/chipsec/1.2.5+-r0/temp/log.do_package_qa.108703
ERROR: Task (/data/code/luv-yocto-32/meta-luv/recipes-core/chipsec/
chipsec_git.bb:do_package_qa) failed with exit code '1'

Cc: Megha Dey <megha.dey@intel.com>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
 meta-luv/recipes-core/chipsec/chipsec_git.bb | 5 +++++
 meta/lib/oe/terminal.py                      | 3 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta-luv/recipes-core/chipsec/chipsec_git.bb b/meta-luv/recipes-core/chipsec/chipsec_git.bb
index 4778a03..a21c8bd 100644
--- a/meta-luv/recipes-core/chipsec/chipsec_git.bb
+++ b/meta-luv/recipes-core/chipsec/chipsec_git.bb
@@ -86,6 +86,11 @@ do_install_append() {
     cd ${D}${PYTHON_SITEPACKAGES_DIR}
     ls | grep -v chipsec | xargs rm -fr
     cd $OLDPWD
+
+    if [ ! "${TARGET_ARCH}" = "x86_64" ]; then
+        rm ${D}${PYTHON_SITEPACKAGES_DIR}/${PN}/chipsec_tools/linux/LzmaCompress.bin
+        rm ${D}${PYTHON_SITEPACKAGES_DIR}/${PN}/chipsec_tools/linux/TianoCompress.bin
+    fi
 }
 
 LUV_TEST_LOG_PARSER="luv-parser-chipsec"
diff --git a/meta/lib/oe/terminal.py b/meta/lib/oe/terminal.py
index 3901ad3..22cb5d7 100644
--- a/meta/lib/oe/terminal.py
+++ b/meta/lib/oe/terminal.py
@@ -67,7 +67,8 @@ class Gnome(XTerminal):
         import tempfile
         pidfile = tempfile.NamedTemporaryFile(delete = False).name
         try:
-            sh_cmd = "oe-gnome-terminal-phonehome " + pidfile + " " + sh_cmd
+            #sh_cmd = "oe-gnome-terminal-phonehome " + pidfile + " " + sh_cmd
+            sh_cmd = bb.utils.which(os.getenv('PATH'), "oe-gnome-terminal-phonehome") + " " + pidfile + " " + sh_cmd
             XTerminal.__init__(self, sh_cmd, title, env, d)
             while os.stat(pidfile).st_size <= 0:
                 continue
-- 
2.9.3