* [Buildroot] [PATCH v8 01/14] core: split variables definition related to in/out-of-tree build from O itself
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 02/14] core: re-enter make if $(CURDIR) or $(O) are not absolute canonical path Samuel Martin
` (12 subsequent siblings)
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
This change uncorrolates the CONFIG_DIR and NEED_WRAPPER definition from
the presence of the O variable in the command line.
Now, the condition used to set these variables is the value of O itself.
This change is a preparatory work since the O definition will need to
be moved around when we will make Buildroot run with absolute canonical
paths for both its root directory and the output location.
This will be addressed in a follow-up patch.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- new patch
---
Makefile | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index 0e4beb2..3d86c9b 100644
--- a/Makefile
+++ b/Makefile
@@ -110,10 +110,11 @@ comma := ,
empty :=
space := $(empty) $(empty)
+# Set O variable if not already done on the command line;
+# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
+# build by preventing it from being forwarded to sub-make calls.
ifneq ("$(origin O)", "command line")
O := output
-CONFIG_DIR := $(TOPDIR)
-NEED_WRAPPER =
else
# other packages might also support Linux-style out of tree builds
# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
@@ -126,9 +127,16 @@ MAKEOVERRIDES =
# Unfortunately some packages don't look at origin (E.G. uClibc 0.9.31+)
# To really make O go away, we have to override it.
override O := $(O)
-CONFIG_DIR := $(O)
# we need to pass O= everywhere we call back into the toplevel makefile
EXTRAMAKEARGS = O=$(O)
+endif
+
+# Set variables related to in-tree or out-of-tree build.
+ifeq ($(O),output)
+CONFIG_DIR := $(TOPDIR)
+NEED_WRAPPER =
+else
+CONFIG_DIR := $(O)
NEED_WRAPPER = y
endif
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 02/14] core: re-enter make if $(CURDIR) or $(O) are not absolute canonical path
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 01/14] core: split variables definition related to in/out-of-tree build from O itself Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 03/14] support/scripts: add fix-rpath script + a bunch of helpers Samuel Martin
` (11 subsequent siblings)
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
When $(CURDIR) or $(O) contain symlinks in their path, they can be
resolved differently, depending on each package build-system (whether it
uses the given paths or get the absolute canonical ones).
This will make easier tracking down host machine paths leaking into the
host, target or staging trees, the CURDIR and O variables are set to
their absolute canonical paths.
In order to recall the toplevel makefile with absolute canonical paths
for $(CURDIR) and $(O), we need to:
1- Move the O variable definition out of any if-block; so they are
always available.
2- Compute the absolute canonical paths for $(CURDIR) and $(O) that will
be passed to the sub-make. This is achieved using the 'realpath' make
primitive. However, some care must be taken when manipulating O:
- the out-of-tree makefile wrapper happens a trailing "/.", we need
to strip this part away to not break the comparison driving the
sub-make call;
- according to [1,2], realpath returns an empty string in case of
non-existing entry. So, to avoid passing an empty O= variable to
sub-make, it is necessary to define the output directory and create
it prior to call realpath on it (because on the first invocation,
$(O) usually does not yet exists), hence the trick doing the mkdir
right before calling realpath.
3- Update EXTRAMAKEARGS with the absobulte canonical $(O) and use it
when call recalling the toplevel makefile with umask and paths
correctly set.
4- Lastly, update the condition for setting the CONFIG_DIR and
NEED_WRAPPER variables.
Notes:
* This change takes care of the makefile wrapper installed in $(O) to
avoid unneeded make recursion.
* Now, only $(O) is strip away from MAKEOVERRIDES whatever the build is
done in- or out-of-tree (i.o.w. without or with O set in the command
line); wheares is the previous implementation, all variables set on
the command line were stripped away only in the case of out-of-tree
build.
[1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html
[2] http://man7.org/linux/man-pages/man3/realpath.3.html
Reported-by: Matthew Weber <matt@thewebers.ws>
Cc: Matthew Weber <matt@thewebers.ws>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- keep @ at the beginning of the command (Yann)
- make ifneq condition easier to read/parsed (Yann)
- fix O definition before re-entering make (Reported by Matthew)
- use EXTRAMAKEARGS when re-entering make
- update the condition for CONFIG_DIR and NEED_WRAPPER
changes v6->v7:
- none
changes v5->v6:
- new patch
---
Makefile | 86 +++++++++++++++++++++++++++++++++++++++++++---------------------
1 file changed, 58 insertions(+), 28 deletions(-)
diff --git a/Makefile b/Makefile
index 3d86c9b..a05b9e1 100644
--- a/Makefile
+++ b/Makefile
@@ -24,18 +24,68 @@
# You shouldn't need to mess with anything beyond this point...
#--------------------------------------------------------------
-# Trick for always running with a fixed umask
+# Set O variable if not already done on the command line;
+# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
+# build by preventing it from being forwarded to sub-make calls.
+ifneq ("$(origin O)", "command line")
+O := $(CURDIR)/output
+else
+# Other packages might also support Linux-style out of tree builds
+# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
+# forwards command line variable definitions those packages get very
+# confused. Fix this by telling make to not do so, only for O=..., but
+# keep all others (such as BR2_EXTERNAL, BR2_DL_DIR, etc).
+MAKEOVERRIDES := $(filter-out O=%,$(MAKEOVERRIDES))
+# Strangely enough O is still passed to submakes with MAKEOVERRIDES
+# (with make 3.81 atleast), the only thing that changes is the output
+# of the origin function (command line -> environment).
+# Unfortunately some packages don't look at origin (E.G. uClibc 0.9.31+)
+# To really make O go away, we have to override it.
+override O := $(O)
+endif
+
+# Check if the current Buildroot execution meets all the pre-requisites.
+# If they are not met, Buildroot will actually do its job in a sub-make meeting
+# its pre-requisites, which are:
+# 1- Permissive enough umask:
+# Wrong or too restrictive umask will prevent Buildroot and packages from
+# creating files and directories.
+# 2- Absolute canonical CWD (i.e. $(CURDIR)):
+# Otherwise, some packages will use CWD as-is, others will compute its
+# absolute canonical path. This makes harder tracking and fixing host
+# machine path leaks.
+# 3- Absolute canonical output location (i.e. $(O)):
+# For the same reason as the one for CWD.
+
+# Current state:
+CUR_UMASK := $(shell umask)
+# Remove the trailing '/.' from $(O) as it can be added by the makefile wrapper
+# installed in the $(O) directory.
+O := $(patsubst %/.,%,$(O))
+
+# Buildroot requirements:
UMASK = 0022
-ifneq ($(shell umask),$(UMASK))
+REAL_CURDIR := $(realpath $(CURDIR))
+# realpath needs the entry to exists, otherwise an empty string is returned.
+REAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))
+
+# Make sure O= is passed (with its absolute canonical path) everywhere the
+# toplevel makefile is called back.
+EXTRAMAKEARGS := O=$(REAL_O)
+
+# Check Buildroot execution pre-requisites here.
+ifneq ($(CUR_UMASK):$(CURDIR):$(O),$(UMASK):$(REAL_CURDIR):$(REAL_O))
.PHONY: _all $(MAKECMDGOALS)
$(MAKECMDGOALS): _all
@:
_all:
- @umask $(UMASK) && $(MAKE) --no-print-directory $(MAKECMDGOALS)
+ @umask $(UMASK) && \
+ $(MAKE) -C $(REAL_CURDIR) --no-print-directory \
+ $(MAKECMDGOALS) $(EXTRAMAKEARGS)
-else # umask
+else # umask / $(CURDIR) / $(O)
# This is our default rule, so must come first
all:
@@ -110,30 +160,10 @@ comma := ,
empty :=
space := $(empty) $(empty)
-# Set O variable if not already done on the command line;
-# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
-# build by preventing it from being forwarded to sub-make calls.
-ifneq ("$(origin O)", "command line")
-O := output
-else
-# other packages might also support Linux-style out of tree builds
-# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
-# forwards command line variable definitions those packages get very
-# confused. Fix this by telling make to not do so
-MAKEOVERRIDES =
-# strangely enough O is still passed to submakes with MAKEOVERRIDES
-# (with make 3.81 atleast), the only thing that changes is the output
-# of the origin function (command line -> environment).
-# Unfortunately some packages don't look at origin (E.G. uClibc 0.9.31+)
-# To really make O go away, we have to override it.
-override O := $(O)
-# we need to pass O= everywhere we call back into the toplevel makefile
-EXTRAMAKEARGS = O=$(O)
-endif
-
# Set variables related to in-tree or out-of-tree build.
-ifeq ($(O),output)
-CONFIG_DIR := $(TOPDIR)
+# Here, both $(O) and $(CURDIR) are absolute canonical paths.
+ifeq ($(O),$(CURDIR)/output)
+CONFIG_DIR := $(CURDIR)
NEED_WRAPPER =
else
CONFIG_DIR := $(O)
@@ -1017,4 +1047,4 @@ include docs/manual/manual.mk
.PHONY: $(noconfig_targets)
-endif #umask
+endif #umask / $(CURDIR) / $(O)
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 03/14] support/scripts: add fix-rpath script + a bunch of helpers
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 01/14] core: split variables definition related to in/out-of-tree build from O itself Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 02/14] core: re-enter make if $(CURDIR) or $(O) are not absolute canonical path Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
` (10 subsequent siblings)
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
This commit introduces a fix-rpath shell-script able to scan a tree,
detect ELF files, check their RPATH and fix it in a proper way.
Along to the fix-rpath script, it also adds a bunch of shell helper
functions grouped into modules. This will help writing scripts handling
RPATH and other things, while allowing to share and reuse these
functions between scripts.
These helpers are namespaced within the filename of the module in which
they are gathered.
This change adds 6 modules:
- source.sh: provides simple helper to easily source another module, taking
care of not sourcing again an already-sourced one;
- log.sh: provides logging helpers;
- utils.sh: provides simple functions to filter ELF files in a list;
- readelf.sh: provides functions retrieving ELF details from a file;
- patchelf.sh: provides function updating ELF files;
- sdk.sh: provides RPATH computation functions.
These 6 modules are used by the fix-rpath script.
Follow-up patches will make some scripts leveraging these modules.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- slightly rework log.error (return non-0 instead of exiting; exit
decision is deferred to the caller)
changes v6->v7 (includes changes started during BR dev. days):
- log module:
- debug env. var. renamed: DEBUG -> SHELL_DEBUG
- trace now prints the message level
- patchelf module:
- get rid of XRPATH/XORIGIN stuff (Arnout)
- set_rpath now handles ELF files with no .dynamic section
- cleanup and minor fix in sanitize_rpath
- readelf module:
- move {filter,is}_elf* functions from utils to readelf modules
- add list_sections and has_section functions (needed in
sanitize_rpath)
- sdk modules:
- get rid of XRPATH/XORIGIN stuff (Arnout)
- minor functions doc. fixes
- source module:
- move "Could not load module" error message in load_module (Arnout)
- utils module:
- move {filter,is}_elf* functions from utils to readelf modules
- add list_has function
- fix-rpath:
- runs with "set -e" (exit on error)
- improve help text
- slight refactoring because of modules changes
changes v5->v6:
- fully rewritten in shell
changes v4->v5:
- add verbose support
- rename shrink_rpath -> clear_rpath
- add sanitize_rpath function
changes v3->v4:
- fix typos and license (Baruch)
changes v2->v3:
- no change
---
support/scripts/fix-rpath | 116 +++++++++++++++++++++++++
support/scripts/shell/log.sh | 61 +++++++++++++
support/scripts/shell/patchelf.sh | 178 ++++++++++++++++++++++++++++++++++++++
support/scripts/shell/readelf.sh | 173 ++++++++++++++++++++++++++++++++++++
support/scripts/shell/sdk.sh | 68 +++++++++++++++
support/scripts/shell/source.sh | 77 +++++++++++++++++
support/scripts/shell/utils.sh | 60 +++++++++++++
7 files changed, 733 insertions(+)
create mode 100755 support/scripts/fix-rpath
create mode 100644 support/scripts/shell/log.sh
create mode 100644 support/scripts/shell/patchelf.sh
create mode 100644 support/scripts/shell/readelf.sh
create mode 100644 support/scripts/shell/sdk.sh
create mode 100644 support/scripts/shell/source.sh
create mode 100644 support/scripts/shell/utils.sh
diff --git a/support/scripts/fix-rpath b/support/scripts/fix-rpath
new file mode 100755
index 0000000..fb2bfeb
--- /dev/null
+++ b/support/scripts/fix-rpath
@@ -0,0 +1,116 @@
+#!/usr/bin/env bash
+
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+set -e
+
+usage() {
+ cat <<EOF >&2
+Usage: ${0} TREE_KIND TREE_ROOT
+
+Description:
+
+ This script scans a tree and sanitize ELF files' RPATH found in there.
+
+ Sanitization behaves the same whatever the kindd of the processed tree, but
+ the resulting RPATH differs.
+
+ Sanitization action:
+ - remove RPATH pointing outside of the tree
+ - for RPATH pointing in the tree:
+ - if they point to standard location (/lib, /usr/lib): remove them
+ - otherwise: make them relative using \$ORIGIN
+
+ For the target tree:
+ - scan the whole tree for sanitization
+
+ For the staging tree :
+ - scan the whole tree for sanitization
+
+ For the host tree:
+ - skip the staging tree for sanitization
+ - add \$HOST_DIR/{lib,usr/lib} to RPATH (as relative pathes)
+
+Arguments:
+
+ TREE_KIND Kind of tree to be processed.
+ Allowed values: host, target, staging
+
+ TREE_ROOT Path to the root of the tree to be scaned
+
+Environment:
+
+ HOST_READELF readelf program to use for host ELF files
+ (default: readelf)
+
+ HOST_READELF readelf program to use for host ELF files
+ (default: readelf)
+
+ PATCHELF patchelf program to use
+ (default: patchelf)
+EOF
+}
+
+source "${0%/*}/shell/source.sh"
+
+source.load_module readelf
+source.load_module patchelf
+
+: ${HOST_READELF:=readelf}
+: ${TARGET_READELF:=readelf}
+: ${PATCHELF:=patchelf}
+
+main() {
+ local tree="${1}"
+ local basedir="$(readlink -f "${2}")"
+
+ local find_args=( "${basedir}" )
+ local sanitize_extra_args=()
+ local readelf
+
+ case "${tree}" in
+ host)
+ # do not process the sysroot (only contains target binaries)
+ find_args+=( "-name" "sysroot" "-prune" "-o" )
+
+ # do not process the external toolchain installation directory to
+ # to avoid breaking it.
+ find_args+=( "-path" "*/opt/ext-toolchain" "-prune" "-o" )
+
+ # make sure RPATH will point to ${hostdir}/lib and ${hostdir}/usr/lib
+ sanitize_extra_args+=( "keep_lib_and_usr_lib" )
+
+ readelf="${HOST_READELF}"
+ ;;
+ staging|target)
+ readelf="${TARGET_READELF}"
+ ;;
+ *)
+ usage
+ exit 1
+ ;;
+ esac
+
+ find_args+=( "-type" "f" "-print" )
+
+ while read file ; do
+ READELF="${readelf}" PATCHELF="${PATCHELF}" \
+ patchelf.sanitize_rpath "${basedir}" "${file}" ${sanitize_extra_args[@]}
+ done < <(find ${find_args[@]} | readelf.filter_elf)
+}
+
+main ${@}
diff --git a/support/scripts/shell/log.sh b/support/scripts/shell/log.sh
new file mode 100644
index 0000000..efadb3f
--- /dev/null
+++ b/support/scripts/shell/log.sh
@@ -0,0 +1,61 @@
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# Logging helpers
+#
+# This module defines the following functions:
+# log.trace
+# log.debug
+# log.info
+# log.warn
+# log.errorN
+# log.error
+#
+# This module sets the following variables:
+# my_name
+#
+# This module is sensitive to the following environment variables:
+# DEBUG
+
+source.declare_module log
+
+# Debug level:
+# - 0 or empty: only show errors
+# - 1 : show errors and warnings
+# - 2 : show errors, warnings, and info
+# - 3 : show errors, warnings, info and debug
+: ${SHELL_DEBUG:=0}
+
+# Low level utility function
+log.trace() {
+ local level="${1}" msg="${2}"
+ shift 2
+ printf "[%-5s] %s: ${msg}" "${level:0:5}" "${my_name}" "${@}"
+}
+
+# Public logging functions
+log.debug() { :; }
+[ ${SHELL_DEBUG} -lt 3 ] || log.debug() { log.trace DEBUG "${@}" >&2; }
+log.info() { :; }
+[ ${SHELL_DEBUG} -lt 2 ] || log.info() { log.trace INFO "${@}" >&2; }
+log.warn() { :; }
+[ ${SHELL_DEBUG} -lt 1 ] || log.warn() { log.trace WARN "${@}" >&2; }
+log.errorN() { local ret="${1}" ; shift ; log.trace ERROR "${@}" ; return ${ret} ; }
+log.error() { log.errorN 1 "${@}"; }
+
+# Program name
+my_name="${0##*/}"
+
diff --git a/support/scripts/shell/patchelf.sh b/support/scripts/shell/patchelf.sh
new file mode 100644
index 0000000..a035305
--- /dev/null
+++ b/support/scripts/shell/patchelf.sh
@@ -0,0 +1,178 @@
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# Patchelf helpers
+#
+# This module defines the following functions:
+# patchelf.set_rpath
+# patchelf.update_rpath
+# patchelf.sanitize_rpath
+#
+# This module is sensitive to the following environment variables:
+# PATCHELF
+# READELF
+
+source.declare_module patchelf
+
+source.load_module log
+source.load_module sdk
+source.load_module utils
+source.load_module readelf
+
+: ${PATCHELF:=patchelf}
+
+# patchelf.set_xrpath file rpath...
+#
+# Set RPATH in $file.
+# Automatically join all RPATH with the correct separator.
+#
+# file : ELF file path
+# rpath : RPATH element
+#
+# environment:
+# PATCHELF: patchelf program path
+patchelf.set_rpath() {
+ local file="${1}"
+ shift
+ local rpath="$(sed -e 's/ +/:/g' <<<"${@}")"
+ # Sanity check: patchelf needs the ELF file to have a .dynamic section.
+ # So, check for it and behaves in a proper way:
+ # - returns immediatly if no .dynamic section, and RPATH is empty;
+ # - bail out if no .dynamic section and RPATH is not empty.
+ if ! readelf.has_section "${file}" '.dynamic' ; then
+ if test -z "${rpath}" ; then
+ return 0
+ else
+ local fmt="Trying to set a RPATH to a ELF file with no .dynamic section\n"
+ fmt="${fmt}\tfile : %s\n"
+ fmt="${fmt}\tRPATH: %s\n"
+ log.error "${fmt}" "${file}" "${rpath}" || return 1
+ fi
+ fi
+ "${PATCHELF}" --set-rpath "${rpath}" "${file}"
+}
+
+# patchelf.update_rpath basedir binary libdirs...
+#
+# Set RPATH in $binary computing them from the paths $libdirs (and $basedir).
+# Existing RPATH in $file will be overwritten if any.
+#
+# basedir : absolute path of the tree in which the $bindir and $libdirs must be
+# binary : ELF file absolute path
+# libdirs : list of library location (absolute paths)
+#
+# environment:
+# PATCHELF: patchelf program path
+patchelf.update_rpath() {
+ local basedir="${1}"
+ local binary="${2}"
+ shift 2
+ local libdirs=( ${@} )
+ log.debug " basedir: %s\n" "${basedir}"
+ log.debug " elf: %s\n" "${binary}"
+ log.debug " libdirs: %s\n" "${libdirs[*]}"
+ log.info " rpath: %s\n" \
+ "$(sdk.compute_rpath "${basedir}" "${binary%/*}" ${libdirs[@]})"
+ patchelf.set_rpath "${binary}" \
+ "$(sdk.compute_rpath "${basedir}" "${binary%/*}" ${libdirs[@]})"
+}
+
+# patchelf.sanitize_rpath basedir binary [keep_lib_usr_lib]
+#
+# Scan $binary's RPATH, remove any of them pointing outside of $basedir.
+# If $keep_lib_usr_lib in not empty, the library directories $basedir/lib and
+# $basedir/usr/lib will be added to the RPATH.
+#
+# Note:
+# Absolute paths is needed to correctly handle symlinks and or mount-bind in
+# the $basedir path.
+#
+# basedir : absolute path of the tree in which the $bindir and $libdirs
+# must be
+# binary : ELF file absolute path
+# keep_lib_usr_lib : add to RPATH $basedir/lib and $basedir/usr/lib
+#
+# environment:
+# PATCHELF: patchelf program path
+# READELF : readelf program path
+patchelf.sanitize_rpath() {
+ local basedir="$(readlink -f "${1}")"
+ local binary="${2}"
+ local keep_lib_usr_lib="${3}"
+
+ readelf.is_elf_shared_object "${binary}" ||
+ readelf.is_elf_executable "${binary}" ||
+ return 0
+
+ local path abspath rpath
+ local libdirs=()
+
+ if test -n "${keep_lib_usr_lib}" ; then
+ libdirs+=( "${basedir}/lib" "${basedir}/usr/lib" )
+ fi
+
+ log.info "ELF: %s\n" "${binary}"
+
+ local rpaths="$(readelf.get_rpath "${binary}")"
+
+ for rpath in ${rpaths//:/ } ; do
+ # figure out if we should keep or discard the path; there are several
+ # cases to handled:
+ # - $path starts with "$ORIGIN":
+ # The original build-system already took care of setting a relative
+ # RPATH, resolve it and test if it is worthwhile to keep it;
+ # - $basedir/$path exists:
+ # The original build-system already took care of setting an absolute
+ # RPATH (absolute in the final rootfs), resolve it and test if it is
+ # worthwhile to keep it;
+ # - $path start with $basedir:
+ # The original build-system added some absolute RPATH (absolute on
+ # the build machine). While this is wrong, it can still be fixed; so
+ # test if it is worthwhile to keep it;
+ # - $path points somewhere else:
+ # (can be anywhere: build trees, staging tree, host location,
+ # non-existing location, etc.)
+ # Just discard such a path.
+ if grep -q '^$ORIGIN/' <<<"${rpath}" ; then
+ path="${binary%/*}/${rpath#*ORIGIN/}"
+ elif test -e "${basedir}/${rpath}" ; then
+ path="${basedir}/${rpath}"
+ elif grep -q "^${basedir}/" <<<"$(readlink -f "${rpath}")" ; then
+ path="${rpath}"
+ else
+ log.debug "\tDROPPED [out-of-tree]: %s\n" "${rpath}"
+ continue
+ fi
+
+ abspath="$(readlink -f "${path}")"
+
+ # discard path pointing to default locations handled by ld-linux
+ if grep -qE "^${basedir}/(lib|usr/lib)$" <<<"${abspath}" ; then
+ log.debug \
+ "\tDROPPED [std libdirs]: %s (%s)\n" "${rpath}" "${abspath}"
+ continue
+ fi
+
+ log.debug "\tKEPT %s (%s)\n" "${rpath}" "${abspath}"
+
+ libdirs+=( "${abspath}" )
+
+ done
+
+ libdirs=( $(utils.list_reduce ${libdirs[@]}) )
+
+ patchelf.update_rpath "${basedir}" "${binary}" ${libdirs[@]}
+}
diff --git a/support/scripts/shell/readelf.sh b/support/scripts/shell/readelf.sh
new file mode 100644
index 0000000..c8ad38b
--- /dev/null
+++ b/support/scripts/shell/readelf.sh
@@ -0,0 +1,173 @@
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# Readelf helpers
+#
+# This module defines the following functions:
+# readelf._filter_elf_regexp
+# readelf.filter_elf
+# readelf.filter_elf_executable
+# readelf.filter_elf_shared_object
+# readelf.is_elf_executable
+# readelf.is_elf_shared_object
+# readelf.get_rpath
+# readelf.list_sections
+# readelf.has_section
+#
+# This module is sensitive to the following environment variables:
+# READELF
+source.declare_module readelf
+
+# When calling readelf(1) program, the user's locale will be overriden with the
+# C locale, so we are sure we can reliably parse its output.
+: ${READELF:=readelf}
+
+# readelf._filter_elf_regexp filter_cmd file...
+#
+# Filters ELF files WRT the given regular extended expression.
+# This funtion can take one or several files, or read them from stdin.
+#
+# filter_cmd : filter command (usually based on grep)
+# file : list of files to be filtered
+#
+# environment:
+# READELF: readelf program path
+readelf._filter_elf_regexp() {
+ local regexp="${1}"
+ shift
+ local in file
+ test ${#} -gt 0 && in='printf "%s\n" "${@}"' || in='dd 2>/dev/null'
+ eval "${in}" |
+ while read file ; do
+ LC_ALL=C ${READELF} -h "${file}" 2>/dev/null |
+ grep -qE "${regexp}" ||
+ continue
+ printf "%s\n" "${file}"
+ done
+}
+
+# readelf.filter_elf file...
+#
+# Filters ELF files; if $file is an ELF file, $file is printed, else it is
+# discarded.
+# This funtion can take one or several arguments, or read them from stdin.
+#
+# file : path of file to be filtered
+#
+# environment:
+# READELF: readelf program path
+readelf.filter_elf() {
+ readelf._filter_elf_regexp "Class:\s+ELF" "${@}"
+}
+
+# readelf.filter_elf_shared_object file...
+#
+# Filters ELF files; if $file is an ELF file, $file is printed, else it is
+# discarded.
+# This funtion can take one or several arguments, or read them from stdin.
+#
+# file : path of file to be filtered
+#
+# environment:
+# READELF: readelf program path
+readelf.filter_elf_shared_object() {
+ readelf._filter_elf_regexp "Type:\s+DYN\s\(Shared\sobject\sfile\)" "${@}"
+}
+
+# readelf.filter_elf_executable file...
+#
+# Filters ELF files; if $file is an ELF file, $file is printed, else it is
+# discarded.
+# This funtion can take one or several arguments, or read them from stdin.
+#
+# file : path of file to be filtered
+#
+# environment:
+# READELF: readelf program path
+readelf.filter_elf_executable() {
+ readelf._filter_elf_regexp "Type:\s+EXEC\s\(Executable\sfile\)" "${@}"
+}
+
+# readelf.is_elf_shared_object file
+#
+# Returns 0 if $file is an ELF file, non-0 otherwise.
+#
+# file : path of file to be tested
+#
+# environment:
+# READELF: readelf program path
+readelf.is_elf_shared_object() {
+ test "$(readelf.filter_elf_shared_object "${1}")" != ""
+}
+
+# readelf.is_elf_executable file
+#
+# Returns 0 if $file is an ELF file, non-0 otherwise.
+#
+# file : path of file to be tested
+#
+# environment:
+# READELF: readelf program path
+readelf.is_elf_executable() {
+ test "$(readelf.filter_elf_executable "${1}")" != ""
+}
+
+# readelf.get_rpath file
+#
+# Return the unsplitted RPATH/RUNPATH of $file.
+#
+# To split the returned RPATH string and store them in an array, do:
+#
+# paths=( $(readelf.get_rpath "${file}" | sed -e 's/:/ /g') )
+#
+# file : ELF file path
+#
+# environment:
+# READELF: readelf program path
+readelf.get_rpath() {
+ local file="${1}"
+ LC_ALL=C "${READELF}" --dynamic "${file}" |
+ sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d ; s//\3/'
+}
+
+# readelf.list_sections file
+#
+# Returns the list of ELF sections in $file.
+#
+# file : ELF file path
+#
+# environment:
+# READELF: readelf program path
+readelf.list_sections() {
+ local file="${1}"
+ LC_ALL=C "${READELF}" --sections "${file}" |
+ sed -re '/^ \[ *[0-9]+\] (\S+).*/!d ; s//\1/' |
+ sort
+}
+
+# readelf.has_section file section
+#
+# Return 0 if $file has a section named $section
+#
+# file : ELF file path
+# section : ELF section name
+#
+# environment:
+# READELF: readelf program path
+readelf.has_section() {
+ local file="${1}" section_name="${2}"
+ readelf.list_sections "${file}" | grep -q "^${section_name}$"
+}
diff --git a/support/scripts/shell/sdk.sh b/support/scripts/shell/sdk.sh
new file mode 100644
index 0000000..b2f699c
--- /dev/null
+++ b/support/scripts/shell/sdk.sh
@@ -0,0 +1,68 @@
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# SDK helpers
+#
+# This module defines the following functions:
+# sdk.compute_relative_path
+# sdk.compute_rpath
+
+source.declare_module sdk
+
+# sdk.compute_relative_path basedir path start
+#
+# Computes and prints the relative path between $start and $path within $basedir.
+#
+# basedir : absolute path of the tree in which the $path and $start must be
+# path : destination absolute path
+# start : origin absolute path
+sdk.compute_relative_path() {
+ local basedir="${1}"
+ local path="${2}"
+ local start="${3}"
+ # sanity checks: make sure $path and $start starts with $basedir
+ grep -q "^${basedir}" <<<"${path}" || return 1
+ grep -q "^${basedir}" <<<"${start}" || return 1
+ local i
+ local backward="${start#${basedir}}"
+ local relative=()
+ for i in ${backward//\// } ; do
+ # don't need to check for empty items they are already discarded
+ test "${i}" != '.' || continue
+ relative+=( ".." )
+ done
+ relative+=( ${path#${basedir}} )
+ sed -r -e 's:[ /]+:/:g' <<<"${relative[@]}"
+}
+
+# sdk.compute_rpath basedir bindir libdirs...
+#
+# Computes and prints the list of RPATH.
+#
+# basedir : absolute path of the tree in which the $bindir and $libdirs must be
+# bindir : binary directory absolute path
+# libdirs : list of library directories (absolute paths)
+sdk.compute_rpath() {
+ local basedir="${1}"
+ local bindir="${2}"
+ shift 2
+ local libdirs=( ${@} )
+ local rpath=()
+ for libdir in ${libdirs[@]} ; do
+ rpath+=( "\$ORIGIN/$(sdk.compute_relative_path "${basedir}" "${libdir}" "${bindir}")" )
+ done
+ sed -e 's/ /:/g' <<<"${rpath[@]}"
+}
diff --git a/support/scripts/shell/source.sh b/support/scripts/shell/source.sh
new file mode 100644
index 0000000..70b7fac
--- /dev/null
+++ b/support/scripts/shell/source.sh
@@ -0,0 +1,77 @@
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# Source helpers
+#
+# This module defines the following functions:
+# source.declare_module
+# source.load_module
+#
+# This module is sensitive to the following environment variables:
+# TOPDIR
+
+# Assuming the script sourcing this file is in support/scripts/
+: ${TOPDIR:=$(readlink -f "${0}" | sed -re 's:(/[^/]+){3}$::')}
+
+# source.declare_module module_name
+#
+# Declare a shell module.
+# Set the variable '_source_${module_name}'.
+# Should be called once per module, in the global scope.
+#
+# module_name : Module name (allowed char.: [_a-zA-Z0-9])
+source.declare_module() {
+ local module_name="${1}"
+ # use printf from bash to set the variable in the environment:
+ printf -v "_source_${module_name}" "%s" "${module_name}"
+}
+
+# source.load_module module_name
+#
+# Load the given shell module, making available all functions declared
+# in it, ensuring it is not reloaded if it already is.
+# Should be called in the global scope.
+# Need the TOPDIR environment variable.
+#
+# param module_name: Module name
+source.load_module() {
+ local module_name="${1}"
+ local loaded="loaded=\${_source_${module_name}}"
+ eval "${loaded}"
+ local module_file="${TOPDIR}/support/scripts/shell/${module_name}.sh"
+
+ if [ ! -f "${module_file}" ] ; then
+ cat <<EOF >&2
+error: Could load module '${module_name}',
+ ${module_file} does not exists.
+
+ Maybe TOPDIR does not point to Buildroot's '\$(TOPDIR)'.
+
+ Or this script '${0##*/}' is most not installed in Buildroot's
+ '\$(TOPDIR)/support/scripts' directory.
+
+ You can fix this by:
+ - either installing '${0##*/}' in the support/scripts/ directory;
+ - or setting the TOPDIR variable in the '${0##*/}' script, before
+ sourcing anything.
+EOF
+ exit 1
+ fi
+
+ test -n "${loaded}" || source "${module_file}"
+}
+
+source.declare_module source
diff --git a/support/scripts/shell/utils.sh b/support/scripts/shell/utils.sh
new file mode 100644
index 0000000..9e9aaab
--- /dev/null
+++ b/support/scripts/shell/utils.sh
@@ -0,0 +1,60 @@
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# Miscellaneous helpers
+#
+# This module defines the following functions:
+# utils.list_has
+# utils.list_reduce
+
+source.declare_module utils
+
+# utils.list_has value list_items...
+#
+# Returns 0 if $list_items contains $value, returns 1 otherwise.
+#
+# value : item to be checked if it is in the list
+# list_items : list of items
+utils.list_has() {
+ local key=$1
+ shift
+ for val in $@ ; do
+ if test "$val" = "$key" ; then
+ return 0
+ fi
+ done
+ return 1
+}
+
+# utils.list_reduce input_list
+#
+# Prints the $input_list list with duplicated items removed.
+# Order is preserved, WRT the first occurence of duplicated items.
+#
+# input_list : list of items
+utils.list_reduce() {
+ local -a lout # return list
+ local i
+
+ for i in ${@} ; do
+ if utils.list_has "${i}" ${lout[@]} ; then
+ continue
+ fi
+ lout+=( "${i}" )
+ done
+
+ echo ${lout[@]}
+}
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (2 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 03/14] support/scripts: add fix-rpath script + a bunch of helpers Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-18 11:33 ` Thomas Petazzoni
2016-04-17 21:38 ` [Buildroot] [PATCH v8 05/14] core: add {TARGET, STAGING}_SANITIZE_RPATH_HOOK " Samuel Martin
` (9 subsequent siblings)
13 siblings, 1 reply; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
This patch adds host-patchelf as a target-finalize dependency, and
introduces the HOST_SANITIZE_RPATH_HOOK hook to fix the ELF files'
RPATH from the HOST_DIR location (excluding the sysroot).
After running this hook, the RPATH from any host ELF files is relative to
the binary location itself.
Notes:
- we avoid to fix RPATH in the sysroot.
- we do not try to fix RPATH in the external toolchain installation
location as they may have been built in a way, this is already correct;
furthermore, fixing RPATH in those programs may result in breaking them.
- the whole host directory is processed because a number of
host-package install programs that could be useful in places
different from $(HOST_DIR)/{bin,sbin,usr/bin,usr/sbin}.
- the shared libraries are also processed in case they have a 'main'
function.
As a step toward a fully relocatable SDK, this change allows to get the
toolchain relocatable, but not yet the whole SDK.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- none
changes v6->v7:
- remove the DEBUG env. var. specific handling in Makefile, now just
place the variable in the Buildroot environment, to enable debug logs.
This is documented in a follow-up patch of this series.
changes v5->v6:
- update for the new script version
- add debug mode support
changes v4->v5:
- add verbose support
changes v3->v4:
- add host-patchelf to PACKAGES instead of target-finalize (Baruch)
- add comment
changes v2->v3:
- move hook in Makefile (Baruch)
---
Makefile | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/Makefile b/Makefile
index a05b9e1..e3574cf 100644
--- a/Makefile
+++ b/Makefile
@@ -638,6 +638,21 @@ endef
TARGET_FINALIZE_HOOKS += PURGE_LOCALES
endif
+# RPATH fixing
+# - The host hook sets RPATH in host ELF binaries, using relative paths to the
+# library locations.
+# - The target hook sanitizes RPATH in target ELF binaries, removing paths
+# pointing to package's build directories or the sysroot's libdirs.
+PACKAGES += host-patchelf
+
+define HOST_SANITIZE_RPATH_HOOK
+ PATCHELF=$(HOST_DIR)/usr/bin/patchelf \
+ READELF=readelf \
+ $(TOPDIR)/support/scripts/fix-rpath host $(HOST_DIR)
+endef
+
+TARGET_FINALIZE_HOOKS += HOST_SANITIZE_RPATH_HOOK
+
$(TARGETS_ROOTFS): target-finalize
target-finalize: $(PACKAGES)
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2016-04-17 21:38 ` [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
@ 2016-04-18 11:33 ` Thomas Petazzoni
2016-04-18 12:18 ` Samuel Martin
2016-04-18 23:38 ` Arnout Vandecappelle
0 siblings, 2 replies; 24+ messages in thread
From: Thomas Petazzoni @ 2016-04-18 11:33 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 17 Apr 2016 23:38:21 +0200, Samuel Martin wrote:
> +# RPATH fixing
> +# - The host hook sets RPATH in host ELF binaries, using relative paths to the
> +# library locations.
> +# - The target hook sanitizes RPATH in target ELF binaries, removing paths
> +# pointing to package's build directories or the sysroot's libdirs.
There is nothing in this patch that adds a target hook.
> +PACKAGES += host-patchelf
> +
> +define HOST_SANITIZE_RPATH_HOOK
> + PATCHELF=$(HOST_DIR)/usr/bin/patchelf \
> + READELF=readelf \
> + $(TOPDIR)/support/scripts/fix-rpath host $(HOST_DIR)
> +endef
> +
> +TARGET_FINALIZE_HOOKS += HOST_SANITIZE_RPATH_HOOK
I find it somewhat odd that this gets done as a "target finalize" hook
since it really has nothing to do with finalizing the target. It really
just needs to be done "at the end of the build".
We could add that as a new target, but it would complexify the
dependency chain, so maybe adding to TARGET_FINALIZE_HOOKS is the
easiest solution after all.
One tricky aspect though is that not all host packages are guaranteed
to be built by the time TARGET_FINALIZE_HOOKS is called. Indeed,
TARGET_FINALIZE_HOOKS is guaranteed to be called after all *target*
packages have been built. But if a host package is needed just to
build a filesystem image for example, it is not guaranteed that it will
be built before TARGET_FINALIZE_HOOKS is called.
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2016-04-18 11:33 ` Thomas Petazzoni
@ 2016-04-18 12:18 ` Samuel Martin
2016-04-18 23:38 ` Arnout Vandecappelle
1 sibling, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-18 12:18 UTC (permalink / raw)
To: buildroot
On Mon, Apr 18, 2016 at 1:33 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Sun, 17 Apr 2016 23:38:21 +0200, Samuel Martin wrote:
>
>> +# RPATH fixing
>> +# - The host hook sets RPATH in host ELF binaries, using relative paths to the
>> +# library locations.
>> +# - The target hook sanitizes RPATH in target ELF binaries, removing paths
>> +# pointing to package's build directories or the sysroot's libdirs.
>
> There is nothing in this patch that adds a target hook.
Oops, my bad! :-s
This is for the next patch in the series.
>
>> +PACKAGES += host-patchelf
>> +
>> +define HOST_SANITIZE_RPATH_HOOK
>> + PATCHELF=$(HOST_DIR)/usr/bin/patchelf \
>> + READELF=readelf \
>> + $(TOPDIR)/support/scripts/fix-rpath host $(HOST_DIR)
>> +endef
>> +
>> +TARGET_FINALIZE_HOOKS += HOST_SANITIZE_RPATH_HOOK
>
> I find it somewhat odd that this gets done as a "target finalize" hook
> since it really has nothing to do with finalizing the target. It really
> just needs to be done "at the end of the build".
>
> We could add that as a new target, but it would complexify the
> dependency chain, so maybe adding to TARGET_FINALIZE_HOOKS is the
> easiest solution after all.
>
> One tricky aspect though is that not all host packages are guaranteed
> to be built by the time TARGET_FINALIZE_HOOKS is called. Indeed,
> TARGET_FINALIZE_HOOKS is guaranteed to be called after all *target*
> packages have been built. But if a host package is needed just to
> build a filesystem image for example, it is not guaranteed that it will
> be built before TARGET_FINALIZE_HOOKS is called.
Good point!
I'll check and see if I can come up with something.
Regards,
--
Samuel
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2016-04-18 11:33 ` Thomas Petazzoni
2016-04-18 12:18 ` Samuel Martin
@ 2016-04-18 23:38 ` Arnout Vandecappelle
2016-04-19 7:21 ` Thomas Petazzoni
1 sibling, 1 reply; 24+ messages in thread
From: Arnout Vandecappelle @ 2016-04-18 23:38 UTC (permalink / raw)
To: buildroot
On 04/18/16 13:33, Thomas Petazzoni wrote:
> Hello,
>
> On Sun, 17 Apr 2016 23:38:21 +0200, Samuel Martin wrote:
>
>> +# RPATH fixing
>> +# - The host hook sets RPATH in host ELF binaries, using relative paths to the
>> +# library locations.
>> +# - The target hook sanitizes RPATH in target ELF binaries, removing paths
>> +# pointing to package's build directories or the sysroot's libdirs.
>
> There is nothing in this patch that adds a target hook.
>
>> +PACKAGES += host-patchelf
>> +
>> +define HOST_SANITIZE_RPATH_HOOK
>> + PATCHELF=$(HOST_DIR)/usr/bin/patchelf \
>> + READELF=readelf \
>> + $(TOPDIR)/support/scripts/fix-rpath host $(HOST_DIR)
>> +endef
>> +
>> +TARGET_FINALIZE_HOOKS += HOST_SANITIZE_RPATH_HOOK
>
> I find it somewhat odd that this gets done as a "target finalize" hook
> since it really has nothing to do with finalizing the target. It really
> just needs to be done "at the end of the build".
>
> We could add that as a new target, but it would complexify the
> dependency chain, so maybe adding to TARGET_FINALIZE_HOOKS is the
> easiest solution after all.
>
> One tricky aspect though is that not all host packages are guaranteed
> to be built by the time TARGET_FINALIZE_HOOKS is called. Indeed,
> TARGET_FINALIZE_HOOKS is guaranteed to be called after all *target*
> packages have been built. But if a host package is needed just to
> build a filesystem image for example, it is not guaranteed that it will
> be built before TARGET_FINALIZE_HOOKS is called.
Actually, it is run after all explicitly selected packages have been built. So
this is yet another reason to also add Kconfig options for host packages.
Regards,
Arnout
>
> Best regards,
>
> Thomas
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2016-04-18 23:38 ` Arnout Vandecappelle
@ 2016-04-19 7:21 ` Thomas Petazzoni
0 siblings, 0 replies; 24+ messages in thread
From: Thomas Petazzoni @ 2016-04-19 7:21 UTC (permalink / raw)
To: buildroot
Hello,
On Tue, 19 Apr 2016 01:38:19 +0200, Arnout Vandecappelle wrote:
> > One tricky aspect though is that not all host packages are guaranteed
> > to be built by the time TARGET_FINALIZE_HOOKS is called. Indeed,
> > TARGET_FINALIZE_HOOKS is guaranteed to be called after all *target*
> > packages have been built. But if a host package is needed just to
> > build a filesystem image for example, it is not guaranteed that it will
> > be built before TARGET_FINALIZE_HOOKS is called.
>
> Actually, it is run after all explicitly selected packages have been built. So
> this is yet another reason to also add Kconfig options for host packages.
Well, while I do like the fact that you try to find more arguments to
add Kconfig options for host packages, I am not sure this particular
one is a good reason. Indeed, as the name TARGET_FINALIZE_HOOKS
suggests, it finalizes the _target_, so there is no point in
guaranteeing that all host packages have been built *before* the hooks
in TARGET_FINALIZE_HOOKS are executed.
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 05/14] core: add {TARGET, STAGING}_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (3 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 04/14] core: add HOST_SANITIZE_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 06/14] package/speex: remove no longer needed hook Samuel Martin
` (8 subsequent siblings)
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
This patch introduces the TARGET_SANITIZE_RPATH_HOOK and
STAGING_SANITIZE_RPATH_HOOK hooks fixing the ELF files' RPATH of
binaries from, respectively, the TARGET_DIR and the STAGING_DIR
locations.
After running this hook, the RPATH from any target ELF files from both
the target and the staging locations won't contain any occurence of the
sysroot or some build locations.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- none
changes v6->v7:
- remove the DEBUG env. var. specific handling in Makefile
changes v5->v6:
- update for the new script version
- add debug mode support
changes v4->v5:
- target hook now sanitizes the rpath (Baruch)
- add verbose support
- update comment
changes v3->v4:
- rebase
- add comment
changes v2->v3:
- move hook in Makefile (Baruch)
---
Makefile | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/Makefile b/Makefile
index e3574cf..38d1076 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,24 @@ endef
TARGET_FINALIZE_HOOKS += HOST_SANITIZE_RPATH_HOOK
+# Function sanitizing target/staging ELF files' RPATH.
+# i.e. it removes paths pointing to the staging or build location from the ELF
+# files' RPATH.
+define TARGET_SANITIZE_RPATH_HOOK
+ PATCHELF=$(HOST_DIR)/usr/bin/patchelf \
+ READELF=$(TARGET_READELF) \
+ $(TOPDIR)/support/scripts/fix-rpath target $(TARGET_DIR)
+endef
+
+define STAGING_SANITIZE_RPATH_HOOK
+ PATCHELF=$(HOST_DIR)/usr/bin/patchelf \
+ READELF=$(TARGET_READELF) \
+ $(TOPDIR)/support/scripts/fix-rpath staging $(STAGING_DIR)
+endef
+
+TARGET_FINALIZE_HOOKS += TARGET_SANITIZE_RPATH_HOOK \
+ STAGING_SANITIZE_RPATH_HOOK
+
$(TARGETS_ROOTFS): target-finalize
target-finalize: $(PACKAGES)
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 06/14] package/speex: remove no longer needed hook
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (4 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 05/14] core: add {TARGET, STAGING}_SANITIZE_RPATH_HOOK " Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-18 19:33 ` Thomas Petazzoni
2016-04-17 21:38 ` [Buildroot] [PATCH v8 07/14] toolchain: add post-install hooks making the toolchain relocatable Samuel Martin
` (7 subsequent siblings)
13 siblings, 1 reply; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
Remove the LIBTOOL_FIXUP hook since RPATH are now sanitized in
target-finalize hooks.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- none
changes v6->v7:
- none
changes v5->v6:
- none
changes v4->v5:
- new patch (suggested by Baruch)
---
package/speex/speex.mk | 5 -----
1 file changed, 5 deletions(-)
diff --git a/package/speex/speex.mk b/package/speex/speex.mk
index a34bfac..24782ec 100644
--- a/package/speex/speex.mk
+++ b/package/speex/speex.mk
@@ -24,11 +24,6 @@ ifeq ($(BR2_PACKAGE_SPEEX_ARM5E),y)
SPEEX_CONF_OPTS += --enable-arm5e-asm
endif
-define SPEEX_LIBTOOL_FIXUP
- $(SED) 's|^hardcode_libdir_flag_spec=.*|hardcode_libdir_flag_spec=""|g' $(@D)/libtool
- $(SED) 's|^runpath_var=LD_RUN_PATH|runpath_var=DIE_RPATH_DIE|g' $(@D)/libtool
-endef
-
SPEEX_POST_CONFIGURE_HOOKS += SPEEX_LIBTOOL_FIXUP
define SPEEX_BUILD_CMDS
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 07/14] toolchain: add post-install hooks making the toolchain relocatable
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (5 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 06/14] package/speex: remove no longer needed hook Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-18 19:33 ` Thomas Petazzoni
2016-04-17 21:38 ` [Buildroot] [PATCH v8 08/14] support/scripts: update check-host-rpath to use the shell helpers Samuel Martin
` (6 subsequent siblings)
13 siblings, 1 reply; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
This change adds the required dependency and post-target-install hooks
to render the toolchain relocatable.
Notes:
- Although the toolchain package is a virtual package, it does support
hooks.
- This has to be post-target-install hooks because the toolchain package
is handled as a "target" package.
Ultimately, this patch allows to generate relocatable toolchains, just
by doing:
$ make menuconfig
$ make toolchain
Then, this toolchain can be used/shared/deployed anywhere*.
*: on any Linux system with an ABI-compatible set of system libraries
(usually including: glibc, libgcc, zlib, flex and libsigsegv).
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
---
changes v7->v8:
- none
changes v6->v7:
- new patch
---
toolchain/toolchain/toolchain.mk | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/toolchain/toolchain/toolchain.mk b/toolchain/toolchain/toolchain.mk
index c22713b..618aa64 100644
--- a/toolchain/toolchain/toolchain.mk
+++ b/toolchain/toolchain/toolchain.mk
@@ -12,6 +12,11 @@ endif
TOOLCHAIN_ADD_TOOLCHAIN_DEPENDENCY = NO
+# dependency and hooks needed to make the toolchain relocatable
+TOOLCHAIN_DEPENDENCIES += host-patchelf
+TOOLCHAIN_POST_INSTALL_TARGET_HOOKS += HOST_SANITIZE_RPATH_HOOK \
+ STAGING_SANITIZE_RPATH_HOOK
+
$(eval $(virtual-package))
toolchain: $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 07/14] toolchain: add post-install hooks making the toolchain relocatable
2016-04-17 21:38 ` [Buildroot] [PATCH v8 07/14] toolchain: add post-install hooks making the toolchain relocatable Samuel Martin
@ 2016-04-18 19:33 ` Thomas Petazzoni
0 siblings, 0 replies; 24+ messages in thread
From: Thomas Petazzoni @ 2016-04-18 19:33 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 17 Apr 2016 23:38:24 +0200, Samuel Martin wrote:
> This change adds the required dependency and post-target-install hooks
> to render the toolchain relocatable.
>
> Notes:
> - Although the toolchain package is a virtual package, it does support
> hooks.
> - This has to be post-target-install hooks because the toolchain package
> is handled as a "target" package.
>
> Ultimately, this patch allows to generate relocatable toolchains, just
> by doing:
>
> $ make menuconfig
> $ make toolchain
>
> Then, this toolchain can be used/shared/deployed anywhere*.
>
> *: on any Linux system with an ABI-compatible set of system libraries
> (usually including: glibc, libgcc, zlib, flex and libsigsegv).
>
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <peter@korsgaard.com>
Why is this needed? You already do such steps in the
TARGET_FINALIZE_HOOKS ?
Yes, it means after just a "make toolchain", you don't have a
relocatable toolchain. But Buildroot never guaranteed that a partial
build will yield fully usable results.
So I would prefer to not have this patch, at least in the initial
version of the series.
Again, this stuff is complicated. Please keep things as minimal as
possible in a first step.
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 08/14] support/scripts: update check-host-rpath to use the shell helpers
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (6 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 07/14] toolchain: add post-install hooks making the toolchain relocatable Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-18 19:36 ` Thomas Petazzoni
2016-04-17 21:38 ` [Buildroot] [PATCH v8 09/14] support/scripts/check-host-rpath: silent find command Samuel Martin
` (5 subsequent siblings)
13 siblings, 1 reply; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- none
changes v6->v7:
- minor updates after shell modules shell changes
changes v5->v6:
- new patch
---
support/scripts/check-host-rpath | 73 +++++++++++++++-------------------------
support/scripts/shell/readelf.sh | 69 +++++++++++++++++++++++++++++++++++++
2 files changed, 97 insertions(+), 45 deletions(-)
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
index 48d69da..45bece8 100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -1,12 +1,29 @@
#!/usr/bin/env bash
+# Copyright (C) 2015 Yann E. MORIN <yann.morin.1998@free.fr>
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
# This script scans $(HOST_DIR)/{bin,sbin} for all ELF files, and checks
# they have an RPATH to $(HOST_DIR)/usr/lib if they need libraries from
# there.
-# Override the user's locale so we are sure we can parse the output of
-# readelf(1) and file(1)
-export LC_ALL=C
+source "${0%/*}/shell/source.sh"
+
+source.load_module readelf
main() {
local pkg="${1}"
@@ -14,58 +31,24 @@ main() {
local file ret
# Remove duplicate and trailing '/' for proper match
- hostdir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${hostdir}" )"
+ hostdir="$(sed -r -e 's:/+:/:g; s:/$::;' <<<"${hostdir}")"
ret=0
while read file; do
- elf_needs_rpath "${file}" "${hostdir}" || continue
- check_elf_has_rpath "${file}" "${hostdir}" && continue
+ READELF=readelf readelf.needs_rpath "${file}" "${hostdir}" || continue
+ READELF=readelf readelf.has_rpath "${file}" "${hostdir}" && continue
if [ ${ret} -eq 0 ]; then
ret=1
printf "***\n"
- printf "*** ERROR: package %s installs executables without proper RPATH:\n" "${pkg}"
+ printf \
+ "*** ERROR: package %s installs executables without proper RPATH:\n" \
+ "${pkg}"
fi
printf "*** %s\n" "${file}"
- done < <( find "${hostdir}"/usr/{bin,sbin} -type f -exec file {} + 2>/dev/null \
- |sed -r -e '/^([^:]+):.*\<ELF\>.*\<executable\>.*/!d' \
- -e 's//\1/' \
- )
+ done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print |
+ readelf.filter_elf_executable)
return ${ret}
}
-elf_needs_rpath() {
- local file="${1}"
- local hostdir="${2}"
- local lib
-
- while read lib; do
- [ -e "${hostdir}/usr/lib/${lib}" ] && return 0
- done < <( readelf -d "${file}" \
- |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
- -e 's//\1/;' \
- )
-
- return 1
-}
-
-check_elf_has_rpath() {
- local file="${1}"
- local hostdir="${2}"
- local rpath dir
-
- while read rpath; do
- for dir in ${rpath//:/ }; do
- # Remove duplicate and trailing '/' for proper match
- dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )"
- [ "${dir}" = "${hostdir}/usr/lib" ] && return 0
- done
- done < <( readelf -d "${file}" \
- |sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \
- -e 's//\3/;' \
- )
-
- return 1
-}
-
main "${@}"
diff --git a/support/scripts/shell/readelf.sh b/support/scripts/shell/readelf.sh
index c8ad38b..0b7e8d2 100644
--- a/support/scripts/shell/readelf.sh
+++ b/support/scripts/shell/readelf.sh
@@ -24,11 +24,15 @@
# readelf.is_elf_executable
# readelf.is_elf_shared_object
# readelf.get_rpath
+# readelf.get_neededs
+# readelf.needs_rpath
+# readelf.has_rpath
# readelf.list_sections
# readelf.has_section
#
# This module is sensitive to the following environment variables:
# READELF
+
source.declare_module readelf
# When calling readelf(1) program, the user's locale will be overriden with the
@@ -143,6 +147,71 @@ readelf.get_rpath() {
sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d ; s//\3/'
}
+# readelf.get_neededs file
+#
+# Return the list of the NEEDED libraries of $file.
+#
+# file : ELF file path
+#
+# environment:
+# READELF: readelf program path
+readelf.get_neededs() {
+ local file="${1}"
+ LC_ALL=C "${READELF}" --dynamic "${file}" |
+ sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d ; s//\1/'
+}
+
+# readelf.needs_rpath file basedir
+#
+# Return 0 if $file needs to have RPATH set, 1 otherwise.
+#
+# file : path of file to be tested
+# basedir : path of the tree in which $basedir/lib and $basedir/usr/lib are
+# checked for belonging to RPATH
+#
+# environment:
+# READELF: readelf program path
+readelf.needs_rpath() {
+ local file="${1}"
+ local basedir="${2}"
+ local lib
+
+ while read lib; do
+ [ -e "${basedir}/lib/${lib}" ] && return 0
+ [ -e "${basedir}/usr/lib/${lib}" ] && return 0
+ done < <(readelf.get_neededs "${file}")
+ return 1
+}
+
+# readelf.has_rpath file basedir
+#
+# Return 0 if $file has RPATH already set to $basedir/lib or $basedir/usr/lib,
+# 1 otherwise.
+#
+# file : path of file to be tested
+# basedir : path of the tree in which $basedir/lib and $basedir/usr/lib are
+# checked for belonging to RPATH
+#
+# environment:
+# READELF: readelf program path
+readelf.has_rpath() {
+ local file="${1}"
+ local basedir="${2}"
+ local rpath dir
+
+ while read rpath; do
+ for dir in ${rpath//:/ }; do
+ # Remove duplicate and trailing '/' for proper match
+ dir="$(sed -r -e "s:/+:/:g; s:/$::" <<<"${dir}")"
+ [ "${dir}" = "${basedir}/lib" ] && return 0
+ [ "${dir}" = "${basedir}/usr/lib" ] && return 0
+ grep -q '^\$ORIGIN/' <<<"${dir}" && return 0
+ done
+ done < <(readelf.get_rpath "${file}")
+
+ return 1
+}
+
# readelf.list_sections file
#
# Returns the list of ELF sections in $file.
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 09/14] support/scripts/check-host-rpath: silent find command
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (7 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 08/14] support/scripts: update check-host-rpath to use the shell helpers Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-18 19:36 ` Thomas Petazzoni
2016-04-17 21:38 ` [Buildroot] [PATCH v8 10/14] support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin} Samuel Martin
` (4 subsequent siblings)
13 siblings, 1 reply; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
Silent find command when HOST_DIR/usr/sbin does not exist.
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- none
changes v6->v7:
- new patch
---
support/scripts/check-host-rpath | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
index 45bece8..3df70bd 100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -45,7 +45,7 @@ main() {
"${pkg}"
fi
printf "*** %s\n" "${file}"
- done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print |
+ done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print 2>/dev/null |
readelf.filter_elf_executable)
return ${ret}
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 09/14] support/scripts/check-host-rpath: silent find command
2016-04-17 21:38 ` [Buildroot] [PATCH v8 09/14] support/scripts/check-host-rpath: silent find command Samuel Martin
@ 2016-04-18 19:36 ` Thomas Petazzoni
0 siblings, 0 replies; 24+ messages in thread
From: Thomas Petazzoni @ 2016-04-18 19:36 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 17 Apr 2016 23:38:26 +0200, Samuel Martin wrote:
> - done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print |
> + done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print 2>/dev/null |
> readelf.filter_elf_executable)
This patch is useless. Your patch 08/14 is doing:
- done < <( find "${hostdir}"/usr/{bin,sbin} -type f -exec file {} + 2>/dev/null \
- |sed -r -e '/^([^:]+):.*\<ELF\>.*\<executable\>.*/!d' \
- -e 's//\1/' \
- )
+ done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print |
+ readelf.filter_elf_executable)
So essentially the 2>/dev/null was already here, and you remove it. And
then you add it again in your patch 09/14. Simply squash both patches.
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 10/14] support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (8 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 09/14] support/scripts/check-host-rpath: silent find command Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-18 19:37 ` Thomas Petazzoni
2016-04-17 21:38 ` [Buildroot] [PATCH v8 11/14] support/scripts: add check-host-leaks script + all needed helpers Samuel Martin
` (3 subsequent siblings)
13 siblings, 1 reply; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
At least syslinux is installing stuff in HOST_DIR/sbin.
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- none
changes v6->v7:
- new patch
---
support/scripts/check-host-rpath | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
index 3df70bd..022f01b 100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -45,7 +45,7 @@ main() {
"${pkg}"
fi
printf "*** %s\n" "${file}"
- done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print 2>/dev/null |
+ done < <(find "${hostdir}"/{,usr/}{bin,sbin} -type f -print 2>/dev/null |
readelf.filter_elf_executable)
return ${ret}
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 10/14] support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}
2016-04-17 21:38 ` [Buildroot] [PATCH v8 10/14] support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin} Samuel Martin
@ 2016-04-18 19:37 ` Thomas Petazzoni
0 siblings, 0 replies; 24+ messages in thread
From: Thomas Petazzoni @ 2016-04-18 19:37 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 17 Apr 2016 23:38:27 +0200, Samuel Martin wrote:
> At least syslinux is installing stuff in HOST_DIR/sbin.
>
> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
>
> ---
> changes v7->v8:
> - none
>
> changes v6->v7:
> - new patch
> ---
> support/scripts/check-host-rpath | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
> index 3df70bd..022f01b 100755
> --- a/support/scripts/check-host-rpath
> +++ b/support/scripts/check-host-rpath
> @@ -45,7 +45,7 @@ main() {
> "${pkg}"
> fi
> printf "*** %s\n" "${file}"
> - done < <(find "${hostdir}"/usr/{bin,sbin} -type f -print 2>/dev/null |
> + done < <(find "${hostdir}"/{,usr/}{bin,sbin} -type f -print 2>/dev/null |
> readelf.filter_elf_executable)
Please put this patch as PATCH 01 in your series, so that we can apply
it right now as a preparation fix. This way, you reduce the size of
your patch series, and things move forward.
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Buildroot] [PATCH v8 11/14] support/scripts: add check-host-leaks script + all needed helpers
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (9 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 10/14] support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin} Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 12/14] core: add check-leaks-in-{target, host, staging} targets Samuel Martin
` (2 subsequent siblings)
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- move sdk.check_host_leaks in the check-host-leaks script itself
(Arnout)
- drop leak classification (Arnout)
- rework readelf.is_elf_{static_library,object} (Arnout)
- add utils.assert_absolute_canonical_path
- drop utils.guess_gnu_target_name
- rework/simplify the check-host-leaks script command line
changes v6->v7:
- {filer,is}_elf* functions moved from utils to readelf module
- update sdk.check_host_leaks
changes v5->v6:
- new patch
---
support/scripts/check-host-leaks | 126 +++++++++++++++++++++++++++++++++++++++
support/scripts/shell/readelf.sh | 77 ++++++++++++++++++++++--
support/scripts/shell/sdk.sh | 7 +++
support/scripts/shell/utils.sh | 14 +++++
4 files changed, 218 insertions(+), 6 deletions(-)
create mode 100755 support/scripts/check-host-leaks
diff --git a/support/scripts/check-host-leaks b/support/scripts/check-host-leaks
new file mode 100755
index 0000000..9f363af
--- /dev/null
+++ b/support/scripts/check-host-leaks
@@ -0,0 +1,126 @@
+#!/usr/bin/env bash
+
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+usage() {
+ local ret="${1:-0}"
+ cat <<EOF
+
+Usage: ${0##*/} [options] ROOT_DIR LEAK_PATH...
+
+Description:
+
+ This script scans a tree for host paths leaked into it.
+ Prints out leaks alongside with the kind of leak.
+
+Arguments:
+
+ ROOT_DIR Path to the root of the tree to be scanned
+
+ LEAK_PATH Paths to check for occurence. Several paths can be
+ passed.
+ A LEAK_PATH must be an absolute canonical path.
+ In the Buildroot context, the LEAK_PATH are Buildroot's
+ TOPDIR, BASE_DIR, HOST_DIR and STAGING_DIR.
+
+Options:
+
+ --exclude=PATH
+ Excludes files from ROOT_DIR starting with PATH. Obviously, the
+ excluded paths should point to some sub-location of ROOT_DIR.
+ Can be set more than once.
+ An excluded path must be an absolute canonical path.
+
+
+EOF
+ return ${ret}
+}
+
+source "${0%/*}/shell/source.sh"
+
+source.load_module log
+source.load_module utils
+source.load_module sdk
+
+main() {
+ local root_dir
+ local class_leaks
+ local excluded=()
+ local leak_paths=()
+ while test ${#} -gt 0 ; do
+ case "${1}" in
+ --exclude=*) excluded+=( "${1#*=}" )
+ ;;
+ --exclude) shift ; excluded+=( "${1}" )
+ ;;
+ -h|--help)
+ usage
+ exit 0
+ ;;
+ *) if test -z "${root_dir}" ; then root_dir="${1}"
+ else leak_paths+=( "${1}" )
+ fi
+ ;;
+ esac
+ shift
+ done
+
+ excluded=( $(utils.list_reduce ${excluded[@]}) )
+ leak_paths=( $(utils.list_reduce ${leak_paths[@]}) )
+
+ # sanity checks
+ if test -z "${root_dir}" ; then
+ log.error "Not enough argument: no ROOT_DIR given.\n" ||
+ usage ${?} || exit ${?}
+ fi
+ if test ${#leak_paths[@]} -lt 1 ; then
+ log.error "Not enough argument. No LEAK_PATH given.\n" ||
+ usage ${?} || exit ${?}
+ fi
+
+ local path has_non_abspath
+ for path in ${leak_paths[@]} ; do
+ utils.assert_absolute_canonical_path "${path}" || has_non_abspath=y
+ done
+ if test -n "${has_non_abspath}" ; then
+ log.error "Some LEAK_PATH are not absolute canonical paths.\n" ||
+ usage ${?} || exit ${?}
+ fi
+ has_non_abspath=
+ for path in ${excluded[@]} ; do
+ utils.assert_absolute_canonical_path "${path}" || has_non_abspath=y
+ done
+ if test -n "${has_non_abspath}" ; then
+ log.error "Some excluded path are not absolute canonical paths.\n" ||
+ usage ${?} || exit ${?}
+ fi
+
+ local re_leaks="$(sed -re 's/^/(/ ; s/$/)/ ; s/ +/|/g' <<<"${leak_paths[*]}")"
+ local re_excl="$(sed -re 's/ +/|/g' <<<"${excluded[*]}")"
+ if test -n "${re_excl}" ; then
+ re_excl="\\:^(${re_excl}):d"
+ fi
+ pushd "${root_dir}" >/dev/null
+ local f
+ grep -raEl "${re_leaks}" . |
+ sed -re "${re_excl} ; s:^\.:${root_dir}:" |
+ while read f ; do
+ printf "%s\n" "${f}"
+ done | sort
+ popd >/dev/null
+}
+main "${@}"
diff --git a/support/scripts/shell/readelf.sh b/support/scripts/shell/readelf.sh
index 0b7e8d2..87599bb 100644
--- a/support/scripts/shell/readelf.sh
+++ b/support/scripts/shell/readelf.sh
@@ -17,18 +17,22 @@
# Readelf helpers
#
# This module defines the following functions:
+# readelf._match_elf_regexp
# readelf._filter_elf_regexp
# readelf.filter_elf
# readelf.filter_elf_executable
# readelf.filter_elf_shared_object
# readelf.is_elf_executable
# readelf.is_elf_shared_object
+# readelf.is_elf_static_library
+# readelf.is_elf_object
# readelf.get_rpath
# readelf.get_neededs
# readelf.needs_rpath
# readelf.has_rpath
# readelf.list_sections
# readelf.has_section
+# readelf.string_section
#
# This module is sensitive to the following environment variables:
# READELF
@@ -39,12 +43,28 @@ source.declare_module readelf
# C locale, so we are sure we can reliably parse its output.
: ${READELF:=readelf}
-# readelf._filter_elf_regexp filter_cmd file...
+# readelf._match_elf_regexp regexp file
#
-# Filters ELF files WRT the given regular extended expression.
+# Returns 0 if the ELF file matches the ELF type given in extended regular
+# expression, non-0 otherwise.
+#
+# regexp : extended regular expression
+# file : list of files to be filtered
+#
+# environment:
+# READELF: readelf program path
+readelf._match_elf_regexp() {
+ log._trace_func
+ local regexp="${1}" file="${2}"
+ LC_ALL=C ${READELF} -h "${file}" 2>/dev/null | grep -qE "${regexp}"
+}
+
+# readelf._filter_elf_regexp regexp file...
+#
+# Filters ELF files WRT the given extended regular expression.
# This funtion can take one or several files, or read them from stdin.
#
-# filter_cmd : filter command (usually based on grep)
+# regexp : extended regular expression
# file : list of files to be filtered
#
# environment:
@@ -56,9 +76,7 @@ readelf._filter_elf_regexp() {
test ${#} -gt 0 && in='printf "%s\n" "${@}"' || in='dd 2>/dev/null'
eval "${in}" |
while read file ; do
- LC_ALL=C ${READELF} -h "${file}" 2>/dev/null |
- grep -qE "${regexp}" ||
- continue
+ readelf._match_elf_regexp "${regexp}" "${file}" || continue
printf "%s\n" "${file}"
done
}
@@ -129,6 +147,39 @@ readelf.is_elf_executable() {
test "$(readelf.filter_elf_executable "${1}")" != ""
}
+# readelf.is_elf file
+#
+# Returns 0 if $file is an ELF file, non-0 otherwise.
+#
+# file : path of file to be tested
+#
+# environment:
+# READELF: readelf program path
+readelf.is_elf() {
+ test "$(readelf.filter_elf "${1}")" != ""
+}
+
+# readelf.is_elf_static_library file
+#
+# Return 0 if $file is a Linux static libraries, i.e. an ar-archive
+# containing *.o files.
+#
+# file : path of file to be tested
+readelf.is_elf_static_library() {
+ readelf._match_elf_regexp "Type:\s+REL\s\(Relocatable\sfile\)" "${@}" &&
+ readelf._match_elf_regexp "^File:\s+\S+\)$" "${@}"
+}
+
+# readelf.is_elf_object file
+#
+# Return 0 if $file is an ELF object file, i.e. a *.o (or *.ko) file.
+#
+# file : path of file to be tested
+readelf.is_elf_object() {
+ readelf._match_elf_regexp "Type:\s+REL\s\(Relocatable\sfile\)" "${@}" &&
+ ! readelf._match_elf_regexp "^File:\s+\S+\)$" "${@}"
+}
+
# readelf.get_rpath file
#
# Return the unsplitted RPATH/RUNPATH of $file.
@@ -240,3 +291,17 @@ readelf.has_section() {
local file="${1}" section_name="${2}"
readelf.list_sections "${file}" | grep -q "^${section_name}$"
}
+
+# readelf.string_section file section
+#
+# Return the given $section of $file.
+#
+# file : ELF file path
+# section : ELF section name
+#
+# environment:
+# READELF: readelf program path
+readelf.string_section() {
+ local file="${1}" section="${2}"
+ LC_ALL=C "${READELF}" --string-dump "${section}" "${file}" 2>/dev/null
+}
diff --git a/support/scripts/shell/sdk.sh b/support/scripts/shell/sdk.sh
index b2f699c..6cb3bbe 100644
--- a/support/scripts/shell/sdk.sh
+++ b/support/scripts/shell/sdk.sh
@@ -19,9 +19,16 @@
# This module defines the following functions:
# sdk.compute_relative_path
# sdk.compute_rpath
+# sdk.check_host_leaks
+#
+# This module is sensitive to the following environment variables:
+# READELF
source.declare_module sdk
+source.load_module utils
+source.load_module readelf
+
# sdk.compute_relative_path basedir path start
#
# Computes and prints the relative path between $start and $path within $basedir.
diff --git a/support/scripts/shell/utils.sh b/support/scripts/shell/utils.sh
index 9e9aaab..eac3158 100644
--- a/support/scripts/shell/utils.sh
+++ b/support/scripts/shell/utils.sh
@@ -19,6 +19,7 @@
# This module defines the following functions:
# utils.list_has
# utils.list_reduce
+# utils.assert_absolute_canonical_path
source.declare_module utils
@@ -58,3 +59,16 @@ utils.list_reduce() {
echo ${lout[@]}
}
+
+# utils.assert_absolute_canonical_path path
+#
+# Returns 0 if 'path' is the absolute canonical path, returns non-0
+# otherwise.
+#
+# If the test failed, an error message will be issued to stderr.
+#
+# path : path to be tested
+utils.assert_absolute_canonical_path() {
+ test "$(readlink -f "${1}")" = "${1}" ||
+ log.error "%s is not the absolute canonical path.\n" "${1}" >&2
+}
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 12/14] core: add check-leaks-in-{target, host, staging} targets
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (10 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 11/14] support/scripts: add check-host-leaks script + all needed helpers Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 13/14] support/scripts/check-host-leaks: add option to classify leaks Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 14/14] docs/manual: document how to debug shell script Samuel Martin
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
This new targets allow identifying what need to be fixed to fulfill the
ultimate goal of removing any trace of the host in the target filesystem
or the SDK.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- update the check-host-leak script call
changes v6->v7:
- none
changes v5->v6:
- new patch
---
Makefile | 15 +++++++++++++++
package/pkg-utils.mk | 5 +++++
2 files changed, 20 insertions(+)
diff --git a/Makefile b/Makefile
index 38d1076..8309eae 100644
--- a/Makefile
+++ b/Makefile
@@ -638,6 +638,21 @@ endef
TARGET_FINALIZE_HOOKS += PURGE_LOCALES
endif
+# Checks for build machine leaks into target filesystem, SDK
+check-leaks-in-target:
+ @$(call MESSAGE,Checking leaks in the target)
+ $(Q)$(call check-for-build-machine-leaks-in,$(TARGET_DIR))
+
+check-leaks-in-host:
+ @$(call MESSAGE,Checking leaks in the host)
+ $(Q)$(call check-for-build-machine-leaks-in,$(HOST_DIR),--exclude=$(STAGING_DIR))
+
+check-leaks-in-staging:
+ @$(call MESSAGE,Checking leaks in the staging)
+ $(Q)$(call check-for-build-machine-leaks-in,$(STAGING_DIR))
+
+check-leaks: check-leaks-in-target check-leaks-in-host check-leaks-in-staging
+
# RPATH fixing
# - The host hook sets RPATH in host ELF binaries, using relative paths to the
# library locations.
diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
index f88313a..8b80028 100644
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -150,3 +150,8 @@ define legal-license-file # pkg, filename, file-fullpath, {HOST|TARGET}
mkdir -p $(LICENSE_FILES_DIR_$(4))/$(1)/$(dir $(2)) && \
cp $(3) $(LICENSE_FILES_DIR_$(4))/$(1)/$(2)
endef
+
+define check-for-build-machine-leaks-in
+ $(TOPDIR)/support/scripts/check-host-leaks $(1) \
+ $(TOPDIR) $(BASE_DIR) $(HOST_DIR) $(STAGING_DIR) $(2)
+endef
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 13/14] support/scripts/check-host-leaks: add option to classify leaks
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (11 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 12/14] core: add check-leaks-in-{target, host, staging} targets Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
2016-04-17 21:38 ` [Buildroot] [PATCH v8 14/14] docs/manual: document how to debug shell script Samuel Martin
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- new patch. Move this feature in its own change-set as it seems a bit
controversial.
- make leaks classification optional
---
support/scripts/check-host-leaks | 59 ++++++++++++++++++++++++++++++++++++++--
support/scripts/shell/readelf.sh | 1 -
2 files changed, 57 insertions(+), 3 deletions(-)
diff --git a/support/scripts/check-host-leaks b/support/scripts/check-host-leaks
index 9f363af..bd92493 100755
--- a/support/scripts/check-host-leaks
+++ b/support/scripts/check-host-leaks
@@ -45,6 +45,12 @@ Options:
Can be set more than once.
An excluded path must be an absolute canonical path.
+ --classify-leaks
+ Enables leak classification. Categories are defined according to
+ the matched files' type or ELF kinds.
+ For ELF files, the sections containing the leaks will displayed.
+ Enabling this option substantially increases the execution time
+ of the script.
EOF
return ${ret}
@@ -56,9 +62,51 @@ source.load_module log
source.load_module utils
source.load_module sdk
+classify_leak() {
+ local f="${1}" regexp="${2}"
+ local leak
+ if test -h "${f}" ; then leak="symlink"
+ elif readelf.is_elf "${f}" ; then
+ if readelf.is_elf_executable "${f}" ; then leak="ELF/exe"
+ elif readelf.is_elf_shared_object "${f}" ; then leak="ELF/*.so"
+ elif readelf.is_elf_static_library "${f}" ; then leak="ELF/*.a"
+ elif readelf.is_elf_object "${f}" ; then
+ case "${f}" in
+ *.ko) leak="ELF/*.ko" ;;
+ *) leak="ELF/*.o" ;;
+ esac
+ else leak="ELF/?"
+ fi
+ local section
+ local sections=()
+ for section in $(readelf.list_sections "${f}") ; do
+ if readelf.string_section "${f}" "${section}" |
+ grep -qaE "${regexp}" ; then
+ if ! utils.list_has ${section} ${sections[@]} ; then
+ sections+=( "${section}" )
+ fi
+ fi
+ done
+ leak="${leak} [${sections[*]}]"
+ else
+ case "${f}" in
+ *-config) leak="*-config script" ;;
+ *.la) leak="*.la" ;;
+ *.pc) leak="*.pc" ;;
+ *.py) leak="*.py" ;;
+ *.pyc|*.pyo) leak="*.py[co]" ;;
+ esac
+ fi
+ if test -z "${leak}" ; then
+ leak="? [$(file -z "${f}" | sed -e 's/.*: //')]"
+ fi
+ printf "${leak}"
+}
+
+
main() {
local root_dir
- local class_leaks
+ local classify_leaks
local excluded=()
local leak_paths=()
while test ${#} -gt 0 ; do
@@ -67,6 +115,8 @@ main() {
;;
--exclude) shift ; excluded+=( "${1}" )
;;
+ --classify-leaks) classify_leaks=y
+ ;;
-h|--help)
usage
exit 0
@@ -119,7 +169,12 @@ main() {
grep -raEl "${re_leaks}" . |
sed -re "${re_excl} ; s:^\.:${root_dir}:" |
while read f ; do
- printf "%s\n" "${f}"
+ if test -n "${classify_leaks}" ; then
+ local leak="$(classify_leak "${f}" "${regexp}")"
+ printf "%-70s : %-120s\n" "${leak}" "${f}"
+ else
+ printf "%s\n" "${f}"
+ fi
done | sort
popd >/dev/null
}
diff --git a/support/scripts/shell/readelf.sh b/support/scripts/shell/readelf.sh
index 87599bb..acb522f 100644
--- a/support/scripts/shell/readelf.sh
+++ b/support/scripts/shell/readelf.sh
@@ -54,7 +54,6 @@ source.declare_module readelf
# environment:
# READELF: readelf program path
readelf._match_elf_regexp() {
- log._trace_func
local regexp="${1}" file="${2}"
LC_ALL=C ${READELF} -h "${file}" 2>/dev/null | grep -qE "${regexp}"
}
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Buildroot] [PATCH v8 14/14] docs/manual: document how to debug shell script
2016-04-17 21:38 [Buildroot] [PATCH v8 00/14] Relocatable SDK/Build machine leaks: RPATH fixing Samuel Martin
` (12 preceding siblings ...)
2016-04-17 21:38 ` [Buildroot] [PATCH v8 13/14] support/scripts/check-host-leaks: add option to classify leaks Samuel Martin
@ 2016-04-17 21:38 ` Samuel Martin
13 siblings, 0 replies; 24+ messages in thread
From: Samuel Martin @ 2016-04-17 21:38 UTC (permalink / raw)
To: buildroot
The method describes here is only true for scripts using the shell
modules from support/scripts/shell/*.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v7->v8:
- put instrumentation in a subsection (Arnout)
- minor rewording
changes v6->v7:
- new patch
---
docs/manual/debugging-buildroot.txt | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/docs/manual/debugging-buildroot.txt b/docs/manual/debugging-buildroot.txt
index f575fc4..b809c2d 100644
--- a/docs/manual/debugging-buildroot.txt
+++ b/docs/manual/debugging-buildroot.txt
@@ -5,6 +5,8 @@
== Debugging Buildroot
+=== Running scripts before and after each build step
+
It is possible to instrument the steps +Buildroot+ does when building
packages. Define the variable +BR2_INSTRUMENTATION_SCRIPTS+ to contain
the path of one or more scripts (or other executables), in a
@@ -41,3 +43,18 @@ The script has access to the following variables:
- +BINARIES_DIR+: the place where all binary files (aka images) are
stored
- +BASE_DIR+: the base output directory
+
+=== Debugging shell scripts in Buildroot
+
+For shell scripts using the _shell modules_ (available in
++support/scripts/shell/+), an environment variable is available to trace
+the scripts:
+
+ - +SHELL_DEBUG+ adjustes the debug level, from 0 to 3 (only errors to
+ all debug messages)
+
+To enable these debug outputs:
+
+----
+SHELL_DEBUG=3 make
+----
--
2.8.0
^ permalink raw reply related [flat|nested] 24+ messages in thread