[PATCH 1/2] manifest.py: sort package list

[PATCH 1/2] manifest.py: sort package list

[Michael Blättler]

The entries of the created manifest file are always in a
different order. To ensure a deterministic build output
the entries are ordered alphabetically.

Signed-off-by: Michael Blättler <michael.blaettler@siemens.com>
---
 meta/lib/oe/manifest.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/manifest.py b/meta/lib/oe/manifest.py
index 60c49be0e9..674303c866 100644
--- a/meta/lib/oe/manifest.py
+++ b/meta/lib/oe/manifest.py
@@ -274,8 +274,8 @@ class OpkgManifest(Manifest):
                     if pkg_list is not None:
                         pkgs[self.var_maps[self.manifest_type][var]] = self.d.getVar(var)
 
-            for pkg_type in pkgs:
-                for pkg in pkgs[pkg_type].split():
+            for pkg_type in sorted(pkgs):
+                for pkg in sorted(pkgs[pkg_type].split()):
                     manifest.write("%s,%s\n" % (pkg_type, pkg))
 
     def create_final(self):
-- 
2.11.0

[PATCH 2/2] package.bbclass: variable to influence link style

[Michael Blättler]

When using separate debug file, gdb needs to match the debug file and
the binary. To match them, there can either be a .gnu_debuglink section
in the binary or a build-id embedded in the debug file and binary.
Until now, only the debuglink option was available. The problem with the
debuglink is, that it contains a checksum of the debug information.
Since the debug information depends on build path and many other things
reproducible binaries were not possible. The build-id can be set via
compiler flag and can be generated from ${PF} or other variables to
ensure reproducibility.

This commit introduces a new variable PACKAGE_DEBUG_SPLIT_LINK_STYLE
which specifies the link option between debug infos and binary. If
build-id is uesed as link option, the id can be defined with the
BUILD_ID variable.

Signed-off-by: Michael Blättler <michael.blaettler@siemens.com>
---
 meta/classes/package.bbclass          | 21 +++++++++++++++------
 meta/conf/bitbake.conf                |  9 ++++++++-
 meta/recipes-bsp/u-boot/u-boot.inc    |  3 +++
 meta/recipes-core/glibc/glibc_2.26.bb |  4 ++++
 4 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 7dc759699f..41b50c199d 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -345,7 +345,7 @@ def parse_debugsources_from_dwarfsrcfiles_output(dwarfsrcfiles_output):
     return debugfiles.keys()
 
 
-def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
+def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, create_debuglink, d):
     # Function to split a single file into two components, one is the stripped
     # target system binary, the other contains any debugging information. The
     # two files are linked to reference each other.
@@ -390,10 +390,11 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
         bb.fatal("objcopy failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
 
     # Set the debuglink to have the view of the file path on the target
-    cmd = "'%s' --add-gnu-debuglink='%s' '%s'" % (objcopy, debugfile, file)
-    (retval, output) = oe.utils.getstatusoutput(cmd)
-    if retval:
-        bb.fatal("objcopy failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
+    if create_debuglink:
+        cmd = "'%s' --add-gnu-debuglink='%s' '%s'" % (objcopy, debugfile, file)
+        (retval, output) = oe.utils.getstatusoutput(cmd)
+        if retval:
+            bb.fatal("objcopy failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
 
     if newmode:
         os.chmod(file, origmode)
@@ -906,6 +907,14 @@ python split_and_strip_files () {
         debuglibdir = ""
         debugsrcdir = "/usr/src/debug"
 
+    # Default to debuglink
+    if d.getVar('PACKAGE_DEBUG_SPLIT_LINK_STYLE') == 'debuglink':
+        create_debuglink = True
+    elif d.getVar('PACKAGE_DEBUG_SPLIT_LINK_STYLE') == 'build-id':
+        create_debuglink = False
+    else:  # default and 'both'
+        create_debuglink = True
+
     sourcefile = d.expand("${WORKDIR}/debugsources.list")
     bb.utils.remove(sourcefile)
 
@@ -1027,7 +1036,7 @@ python split_and_strip_files () {
             bb.utils.mkdirhier(os.path.dirname(fpath))
             #bb.note("Split %s -> %s" % (file, fpath))
             # Only store off the hard link reference if we successfully split!
-            splitdebuginfo(file, fpath, debugsrcdir, sourcefile, d)
+            splitdebuginfo(file, fpath, debugsrcdir, sourcefile, create_debuglink, d)
 
         # Hardlink our debug symbols to the other hardlink copies
         for ref in inodes:
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 93afb13166..fc2f77fb54 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -578,8 +578,15 @@ LINKER_HASH_STYLE_mipsarch = "sysv"
 
 TARGET_LINK_HASH_STYLE ?= "${@['-Wl,--hash-style=gnu',''][d.getVar('LINKER_HASH_STYLE') != 'gnu']}"
 
+BUILD_ID ??= "sha1"
+
+# Enable build-id if PACKAGE_DEBUG_SPLIT_LINK_STYLE equals 'build-id' or 'both'
+DEBUG_LINK_BUILDID ?= "${@['', '-Wl,--build-id=${BUILD_ID}']\
+                          [d.getVar('PACKAGE_DEBUG_SPLIT_LINK_STYLE') == 'build-id' or \
+                           d.getVar('PACKAGE_DEBUG_SPLIT_LINK_STYLE') == 'both']}"
+
 export LDFLAGS = "${TARGET_LDFLAGS}"
-export TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
+export TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE} ${DEBUG_LINK_BUILDID}"
 #export TARGET_LDFLAGS = "-L${STAGING_DIR_TARGET}${libdir} \
 #                         -Wl,-rpath-link,${STAGING_DIR_TARGET}${libdir} \
 #                         -Wl,-O1"
diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
index c2bcf99840..7281a54571 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -72,6 +72,9 @@ do_compile () {
 	unset CFLAGS
 	unset CPPFLAGS
 
+	# Set LDFLAGS again in case a build-id is used to link debug files and binary
+	export LDFLAGS="${DEBUG_LINK_BUILDID}"
+
 	if [ ! -e ${B}/.scmversion -a ! -e ${S}/.scmversion ]
 	then
 		echo ${UBOOT_LOCALVERSION} > ${B}/.scmversion
diff --git a/meta/recipes-core/glibc/glibc_2.26.bb b/meta/recipes-core/glibc/glibc_2.26.bb
index 04d97734b3..866e4c65ea 100644
--- a/meta/recipes-core/glibc/glibc_2.26.bb
+++ b/meta/recipes-core/glibc/glibc_2.26.bb
@@ -113,6 +113,10 @@ rpcsvc = "bootparam_prot.x nlm_prot.x rstat.x \
 do_compile () {
 	# -Wl,-rpath-link <staging>/lib in LDFLAGS can cause breakage if another glibc is in staging
 	unset LDFLAGS
+
+	# Set LDFLAGS again in case a build-id is used to link debug files and binary
+	export LDFLAGS="${DEBUG_LINK_BUILDID}"
+
 	base_do_compile
 	(
 		cd ${S}/sunrpc/rpcsvc
-- 
2.11.0

Re: [PATCH 2/2] package.bbclass: variable to influence link style

[Alexander Kanavin]

On 01/08/2018 09:42 AM, Michael Blättler wrote:

> This commit introduces a new variable PACKAGE_DEBUG_SPLIT_LINK_STYLE
> which specifies the link option between debug infos and binary. If
> build-id is uesed as link option, the id can be defined with the
> BUILD_ID variable.

How did you test this? Can you add a test to insane.bbclass please? 
Adding new options without having a way to test in an automated, regular 
fashion is not a great idea.

Alex

Re: [PATCH 2/2] package.bbclass: variable to influence link style

[Blaettler, Michael]

Hi Alex

I did the tests manually. But of course you're right in regard to automated testing. I'll add a test to "insane.bbclass" and send a new patch.

Regards
Michael

-----Ursprüngliche Nachricht-----
Von: Alexander Kanavin [mailto:alexander.kanavin@linux.intel.com] 
Gesendet: Montag, 8. Januar 2018 12:02
An: Blaettler, Michael (BT CPS R&D ZG FW ITW); openembedded-core@lists.openembedded.org
Betreff: Re: [OE-core] [PATCH 2/2] package.bbclass: variable to influence link style

On 01/08/2018 09:42 AM, Michael Blättler wrote:

> This commit introduces a new variable PACKAGE_DEBUG_SPLIT_LINK_STYLE 
> which specifies the link option between debug infos and binary. If 
> build-id is uesed as link option, the id can be defined with the 
> BUILD_ID variable.

How did you test this? Can you add a test to insane.bbclass please? 
Adding new options without having a way to test in an automated, regular fashion is not a great idea.

Alex