* [Buildroot] [PATCH 0/2] support/check-bin-arch: add support for excluding arbitrary locations @ 2018-03-07 20:27 Yann E. MORIN 2018-03-07 20:27 ` [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths Yann E. MORIN 2018-03-07 20:27 ` [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr Yann E. MORIN 0 siblings, 2 replies; 8+ messages in thread From: Yann E. MORIN @ 2018-03-07 20:27 UTC (permalink / raw) To: buildroot Hello All! Users have started reporting issues with this check, when they have local packages (mostly out-of-tree, proprietary) that install binary blobs (like co-processor firmwares) in arbitrary locations, like /opt, which we currently do not exclude from the check. Regards, Yann E. MORIN. The following changes since commit 74295b02d4b380e5267357be0ae1281c7410cdc2 ipset: bump to version 6.36 (2018-03-06 15:44:18 +0100) are available in the git repository at: git://git.buildroot.org/~ymorin/git/buildroot.git for you to fetch changes up to ce786b49c9bbe6a3e2ebde9ca28b56de86e68b2e support/check-bin-arch: exclude kernel modules for merged /usr (2018-03-07 21:17:11 +0100) ---------------------------------------------------------------- Yann E. MORIN (2): spport/check-bin-arch: accept arbitrary ignore paths support/check-bin-arch: exclude kernel modules for merged /usr docs/manual/adding-packages-generic.txt | 7 ++++ package/pkg-generic.mk | 1 + support/scripts/check-bin-arch | 59 +++++++++++++++++++++------------ 3 files changed, 45 insertions(+), 22 deletions(-) -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths 2018-03-07 20:27 [Buildroot] [PATCH 0/2] support/check-bin-arch: add support for excluding arbitrary locations Yann E. MORIN @ 2018-03-07 20:27 ` Yann E. MORIN 2018-03-07 20:45 ` Peter Korsgaard 2018-03-07 20:27 ` [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr Yann E. MORIN 1 sibling, 1 reply; 8+ messages in thread From: Yann E. MORIN @ 2018-03-07 20:27 UTC (permalink / raw) To: buildroot Some packages (mostly, out-of-tree) may want to install binary blobs for another architecture, outside the locations we currently exclude, like in /opt or whatever... Add support in check-bin-arch to accept any arbitrary location, that individual package can each request to excude from the check, when they are installed. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- docs/manual/adding-packages-generic.txt | 7 ++++ package/pkg-generic.mk | 1 + support/scripts/check-bin-arch | 58 ++++++++++++++++++++------------- 3 files changed, 44 insertions(+), 22 deletions(-) diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt index 63ea51bf89..1f5d92d2c9 100644 --- a/docs/manual/adding-packages-generic.txt +++ b/docs/manual/adding-packages-generic.txt @@ -453,6 +453,13 @@ information is (assuming the package name is +libfoo+) : FLAT binary format is only 4k bytes. If the application consumes more stack, append the required number here. +* +LIBFOO_BIN_ARCH_EXCLUDE+ is a space-separated list of path (relative + to the target directory) to ignore when checking that the package + installs correctly cross-compiled binaries. You seldom need to set this + variable, unless the package installs installs binary blobs in the + non-default locations: `/lib/firmware`, `/usr/lib/firmware`, + `/lib/modules`, and `/usr/share`. + The recommended way to define these variables is to use the following syntax: diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index a2a12e7b56..9eddaeee57 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -112,6 +112,7 @@ define check_bin_arch $(if $(filter end-install-target,$(1)-$(2)),\ support/scripts/check-bin-arch -p $(3) \ -l $(BUILD_DIR)/packages-file-list.txt \ + $(foreach i,$($(PKG)_BIN_ARCH_EXCLUDE),-i "$(i)") \ -r $(TARGET_READELF) \ -a $(BR2_READELF_ARCH_NAME)) endef diff --git a/support/scripts/check-bin-arch b/support/scripts/check-bin-arch index f6a4569c62..922791ab89 100755 --- a/support/scripts/check-bin-arch +++ b/support/scripts/check-bin-arch @@ -1,18 +1,47 @@ #!/usr/bin/env bash -while getopts p:l:r:a: OPT ; do +# List of hardcoded paths that should be ignored, as they may +# contain binaries for an architecture different from the +# architecture of the target. +declare -a IGNORES=( + # Skip firmware files, they could be ELF files for other + # architectures + "/lib/firmware" + "/usr/lib/firmware" + + # Skip kernel modules + # When building a 32-bit userland on 64-bit architectures, the kernel + # and its modules may still be 64-bit. To keep the basic + # check-bin-arch logic simple, just skip this directory. + "/lib/modules" + + # Skip files in /usr/share, several packages (qemu, + # pru-software-support) legitimately install ELF binaries that + # are not for the target architecture + "/usr/share" +) + +while getopts p:l:r:a:i: OPT ; do case "${OPT}" in p) package="${OPTARG}";; l) pkg_list="${OPTARG}";; r) readelf="${OPTARG}";; a) arch_name="${OPTARG}";; + i) + # Ensure we do have single '/' as separators, + # and that we have a leading and a trailing one. + IGNORES+=( "$(sed -r -e 's:/+:/:g; s:^/*:/:; s:/*$:/:;' \ + <<<"${OPTARG}" + )" + ) + ;; :) error "option '%s' expects a mandatory argument\n" "${OPTARG}";; \?) error "unknown option '%s'\n" "${OPTARG}";; esac done if test -z "${package}" -o -z "${pkg_list}" -o -z "${readelf}" -o -z "${arch_name}" ; then - echo "Usage: $0 -p <pkg> -l <pkg-file-list> -r <readelf> -a <arch name>" + echo "Usage: $0 -p <pkg> -l <pkg-file-list> -r <readelf> -a <arch name> [-i PATTERN ...]" exit 1 fi @@ -23,26 +52,11 @@ IFS=" " while read f; do - # Skip firmware files, they could be ELF files for other - # architectures - if [[ "${f}" =~ ^/(usr/)?lib/firmware/.* ]]; then - continue - fi - - # Skip kernel modules - # When building a 32-bit userland on 64-bit architectures, the kernel - # and its modules may still be 64-bit. To keep the basic - # check-bin-arch logic simple, just skip this directory. - if [[ "${f}" =~ ^/lib/modules/.* ]]; then - continue - fi - - # Skip files in /usr/share, several packages (qemu, - # pru-software-support) legitimately install ELF binaries that - # are not for the target architecture - if [[ "${f}" =~ ^/usr/share/.* ]]; then - continue - fi + for ignore in "${IGNORES[@]}"; do + if [[ "${f}" =~ ^"${ignore}" ]]; then + continue 2 + fi + done # Skip symlinks. Some symlinks may have absolute paths as # target, pointing to host binaries while we're building. -- 2.14.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths 2018-03-07 20:27 ` [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths Yann E. MORIN @ 2018-03-07 20:45 ` Peter Korsgaard 2018-03-07 21:22 ` Yann E. MORIN 0 siblings, 1 reply; 8+ messages in thread From: Peter Korsgaard @ 2018-03-07 20:45 UTC (permalink / raw) To: buildroot >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes: > Some packages (mostly, out-of-tree) may want to install binary blobs for > another architecture, outside the locations we currently exclude, like > in /opt or whatever... > Add support in check-bin-arch to accept any arbitrary location, that > individual package can each request to excude from the check, when they > are installed. > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Peter Korsgaard <peter@korsgaard.com> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > --- > docs/manual/adding-packages-generic.txt | 7 ++++ > package/pkg-generic.mk | 1 + > support/scripts/check-bin-arch | 58 ++++++++++++++++++++------------- > 3 files changed, 44 insertions(+), 22 deletions(-) > diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt > index 63ea51bf89..1f5d92d2c9 100644 > --- a/docs/manual/adding-packages-generic.txt > +++ b/docs/manual/adding-packages-generic.txt > @@ -453,6 +453,13 @@ information is (assuming the package name is +libfoo+) : > FLAT binary format is only 4k bytes. If the application consumes more stack, > append the required number here. > +* +LIBFOO_BIN_ARCH_EXCLUDE+ is a space-separated list of path (relative s/path/paths/ > + to the target directory) to ignore when checking that the package > + installs correctly cross-compiled binaries. You seldom need to set this > + variable, unless the package installs installs binary blobs in the s/installs installs/installs/ > + non-default locations: `/lib/firmware`, `/usr/lib/firmware`, > + `/lib/modules`, and `/usr/share`. This sounds a bit confusing to me, E.G. like these are non default locations. Perhaps we should instead write: variable, unless the package installs binary blobs outside the default locations, which are `/lib/firmware`, `/usr/lib/firmware`, `/lib/modules` and '/usr/share`, as these locations are automatically excluded. > if test -z "${package}" -o -z "${pkg_list}" -o -z "${readelf}" -o -z "${arch_name}" ; then > - echo "Usage: $0 -p <pkg> -l <pkg-file-list> -r <readelf> -a <arch name>" > + echo "Usage: $0 -p <pkg> -l <pkg-file-list> -r <readelf> -a <arch name> [-i PATTERN ...]" We don't really document this as a pattern (even if it gets passed to ~= in the end), so perhaps it would be clearer to say [ -i PATH ]? Otherwise it looks good to me, thanks! -- Bye, Peter Korsgaard ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths 2018-03-07 20:45 ` Peter Korsgaard @ 2018-03-07 21:22 ` Yann E. MORIN 0 siblings, 0 replies; 8+ messages in thread From: Yann E. MORIN @ 2018-03-07 21:22 UTC (permalink / raw) To: buildroot Peter, All, On 2018-03-07 21:45 +0100, Peter Korsgaard spake thusly: > >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes: [--SNIP--] > > +* +LIBFOO_BIN_ARCH_EXCLUDE+ is a space-separated list of path (relative > s/path/paths/ > > + to the target directory) to ignore when checking that the package > > + installs correctly cross-compiled binaries. You seldom need to set this > > + variable, unless the package installs installs binary blobs in the > s/installs installs/installs/ Typoes fixed. > > + non-default locations: `/lib/firmware`, `/usr/lib/firmware`, > > + `/lib/modules`, and `/usr/share`. > > This sounds a bit confusing to me, E.G. like these are non default > locations. > > Perhaps we should instead write: > > variable, unless the package installs binary blobs outside the default > locations, which are `/lib/firmware`, `/usr/lib/firmware`, > `/lib/modules` and '/usr/share`, as these locations are automatically > excluded. Indeed, it was not clear (except maybe just in my twisted head). I've rephrased slightly differently from your suggestion, but if you don;t like it, we can go with yours instead. > > if test -z "${package}" -o -z "${pkg_list}" -o -z "${readelf}" -o -z "${arch_name}" ; then > > - echo "Usage: $0 -p <pkg> -l <pkg-file-list> -r <readelf> -a <arch name>" > > + echo "Usage: $0 -p <pkg> -l <pkg-file-list> -r <readelf> -a <arch name> [-i PATTERN ...]" > > We don't really document this as a pattern (even if it gets passed to ~= > in the end), so perhaps it would be clearer to say [ -i PATH ]? ACK. > Otherwise it looks good to me, thanks! Thanks! :-) Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr 2018-03-07 20:27 [Buildroot] [PATCH 0/2] support/check-bin-arch: add support for excluding arbitrary locations Yann E. MORIN 2018-03-07 20:27 ` [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths Yann E. MORIN @ 2018-03-07 20:27 ` Yann E. MORIN 2018-03-07 20:45 ` Peter Korsgaard 1 sibling, 1 reply; 8+ messages in thread From: Yann E. MORIN @ 2018-03-07 20:27 UTC (permalink / raw) To: buildroot When using a merged /usr, the kernel module path is really /usr/lib/modules, as /lib is a symlink to usr/lib . Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- support/scripts/check-bin-arch | 1 + 1 file changed, 1 insertion(+) diff --git a/support/scripts/check-bin-arch b/support/scripts/check-bin-arch index 922791ab89..258014b98f 100755 --- a/support/scripts/check-bin-arch +++ b/support/scripts/check-bin-arch @@ -14,6 +14,7 @@ declare -a IGNORES=( # and its modules may still be 64-bit. To keep the basic # check-bin-arch logic simple, just skip this directory. "/lib/modules" + "/usr/lib/modules" # Skip files in /usr/share, several packages (qemu, # pru-software-support) legitimately install ELF binaries that -- 2.14.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr 2018-03-07 20:27 ` [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr Yann E. MORIN @ 2018-03-07 20:45 ` Peter Korsgaard 2018-03-07 21:23 ` Yann E. MORIN 0 siblings, 1 reply; 8+ messages in thread From: Peter Korsgaard @ 2018-03-07 20:45 UTC (permalink / raw) To: buildroot >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes: > When using a merged /usr, the kernel module path is really > /usr/lib/modules, as /lib is a symlink to usr/lib . > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: Peter Korsgaard <peter@korsgaard.com> -- Bye, Peter Korsgaard ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr 2018-03-07 20:45 ` Peter Korsgaard @ 2018-03-07 21:23 ` Yann E. MORIN 2018-03-07 21:35 ` Peter Korsgaard 0 siblings, 1 reply; 8+ messages in thread From: Yann E. MORIN @ 2018-03-07 21:23 UTC (permalink / raw) To: buildroot Peter, All, On 2018-03-07 21:45 +0100, Peter Korsgaard spake thusly: > >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes: > > > When using a merged /usr, the kernel module path is really > > /usr/lib/modules, as /lib is a symlink to usr/lib . > > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > > Reviewed-by: Peter Korsgaard <peter@korsgaard.com> But I forgot to update the manual with this new location! ;-) Should I still cary your rev-tag in round 2? Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr 2018-03-07 21:23 ` Yann E. MORIN @ 2018-03-07 21:35 ` Peter Korsgaard 0 siblings, 0 replies; 8+ messages in thread From: Peter Korsgaard @ 2018-03-07 21:35 UTC (permalink / raw) To: buildroot >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes: > Peter, All, > On 2018-03-07 21:45 +0100, Peter Korsgaard spake thusly: >> >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes: >> >> > When using a merged /usr, the kernel module path is really >> > /usr/lib/modules, as /lib is a symlink to usr/lib . >> >> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> >> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> >> >> Reviewed-by: Peter Korsgaard <peter@korsgaard.com> > But I forgot to update the manual with this new location! ;-) > Should I still cary your rev-tag in round 2? Sure! -- Bye, Peter Korsgaard ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-03-07 21:35 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-03-07 20:27 [Buildroot] [PATCH 0/2] support/check-bin-arch: add support for excluding arbitrary locations Yann E. MORIN 2018-03-07 20:27 ` [Buildroot] [PATCH 1/2] spport/check-bin-arch: accept arbitrary ignore paths Yann E. MORIN 2018-03-07 20:45 ` Peter Korsgaard 2018-03-07 21:22 ` Yann E. MORIN 2018-03-07 20:27 ` [Buildroot] [PATCH 2/2] support/check-bin-arch: exclude kernel modules for merged /usr Yann E. MORIN 2018-03-07 20:45 ` Peter Korsgaard 2018-03-07 21:23 ` Yann E. MORIN 2018-03-07 21:35 ` Peter Korsgaard
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.