* [PATCH V2] busybox: Use CC instead of bare LD to be the Linker
@ 2015-09-25 4:59 Khem Raj
2015-09-25 4:59 ` [PATCH 1/4] powertop: Include right headers for timval struct Khem Raj
` (5 more replies)
0 siblings, 6 replies; 18+ messages in thread
From: Khem Raj @ 2015-09-25 4:59 UTC (permalink / raw)
To: openembedded-core
This patch was on mailing list, another patch to make sure -r is not
passed directly but via -Wl switch is added.
This was exposed when using clang and gold linker, clang does not have
-r switch to do relocatable objects and problem happens specific to OE
becuase we use LD = CC
now what happens is that busybox assumes that linker will be called
directly, and hence sprinkles linkers options in its kbuild system which
aggregate into LDFLAGS, some of these options are happily ignored by gcc
as well but it passes -r options rightly to linker so it all works,
however when using clang, this falls apart since -r is not known option
for clang so it drops this option and all obects which should be
partially linked becomes ET_EXEC and when they are added to final link
then gold starts to get confused
/mnt/home/kraj/work/angstrom/build/tmp-angstrom-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-ld:
error: applets/built-in.o: unsupported ELF file type 2
clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
...-linking-instead-of-LD-and-use-CFLAGS-and.patch | 114 +++++++++++++++++++++
.../busybox/0002-Passthrough-r-to-linker.patch | 32 ++++++
meta/recipes-core/busybox/busybox_1.23.2.bb | 2 +
3 files changed, 148 insertions(+)
create mode 100644 meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
create mode 100644 meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
diff --git a/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch b/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
new file mode 100644
index 0000000..2bf2b91
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
@@ -0,0 +1,114 @@
+From a9333eb6a7b8dbda735947cd5bc981ff9352a2c9 Mon Sep 17 00:00:00 2001
+From: Nathan Phillip Brink <ohnobinki@ohnopublishing.net>
+Date: Thu, 10 Mar 2011 00:27:08 -0500
+Subject: [PATCH 1/2] Use $(CC) when linking instead of $(LD) and use $(CFLAGS)
+ and $(EXTRA_CFLAGS) when linking.
+
+This fixes the issue where LDFLAGS escaped with -Wl are ignored during
+compilation. It also simplifies using CFLAGS or EXTRA_CFLAGS (such as
+-m32 on x86_64 or -flto) which apply to both compilation and linking
+situations.
+
+Signed-off-by: Nathan Phillip Brink <ohnobinki@ohnopublishing.net>
+---
+Upstream-Status: Pending
+
+ Makefile | 7 ++++---
+ scripts/Makefile.build | 8 ++++----
+ scripts/Makefile.lib | 13 +++----------
+ 3 files changed, 11 insertions(+), 17 deletions(-)
+
+Index: busybox-1.23.2/Makefile
+===================================================================
+--- busybox-1.23.2.orig/Makefile
++++ busybox-1.23.2/Makefile
+@@ -309,7 +309,8 @@ CHECKFLAGS := -D__linux__ -Dlinux -D
+ MODFLAGS = -DMODULE
+ CFLAGS_MODULE = $(MODFLAGS)
+ AFLAGS_MODULE = $(MODFLAGS)
+-LDFLAGS_MODULE = -r
++LDFLAGS_RELOCATABLE = -r -nostdlib
++LDFLAGS_MODULE = $(LDFLAGS_RELOCATABLE)
+ CFLAGS_KERNEL =
+ AFLAGS_KERNEL =
+
+@@ -331,7 +332,7 @@ KERNELVERSION = $(VERSION).$(PATCHLEVEL)
+ export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION \
+ ARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC \
+ CPP AR NM STRIP OBJCOPY OBJDUMP MAKE AWK GENKSYMS PERL UTS_MACHINE \
+- HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS
++ HOSTCXX HOSTCXXFLAGS LDFLAGS_RELOCATABLE LDFLAGS_MODULE CHECK CHECKFLAGS
+
+ export CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS
+ export CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
+@@ -610,7 +611,7 @@ quiet_cmd_busybox__ ?= LINK $@
+ cmd_busybox__ ?= $(srctree)/scripts/trylink \
+ "$@" \
+ "$(CC)" \
+- "$(CFLAGS) $(CFLAGS_busybox)" \
++ "$(CFLAGS) $(CFLAGS_busybox) $(EXTRA_CFLAGS)" \
+ "$(LDFLAGS) $(EXTRA_LDFLAGS)" \
+ "$(core-y)" \
+ "$(libs-y)" \
+Index: busybox-1.23.2/scripts/Makefile.build
+===================================================================
+--- busybox-1.23.2.orig/scripts/Makefile.build
++++ busybox-1.23.2/scripts/Makefile.build
+@@ -174,7 +174,7 @@ cmd_modversions = \
+ | $(GENKSYMS) -a $(ARCH) \
+ > $(@D)/.tmp_$(@F:.o=.ver); \
+ \
+- $(LD) $(LDFLAGS) -r -o $@ $(@D)/.tmp_$(@F) \
++ $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(@D)/.tmp_$(@F) \
+ -T $(@D)/.tmp_$(@F:.o=.ver); \
+ rm -f $(@D)/.tmp_$(@F) $(@D)/.tmp_$(@F:.o=.ver); \
+ else \
+@@ -257,7 +257,7 @@ quiet_cmd_link_o_target = LD $@
+ # If the list of objects to link is empty, just create an empty built-in.o
+ # -nostdlib is added to make "make LD=gcc ..." work (some people use that)
+ cmd_link_o_target = $(if $(strip $(obj-y)),\
+- $(LD) -nostdlib $(ld_flags) -r -o $@ $(filter $(obj-y), $^),\
++ $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(filter $(obj-y), $^),\
+ rm -f $@; $(AR) rcs $@)
+
+ $(builtin-target): $(obj-y) FORCE
+@@ -292,10 +292,10 @@ $($(subst $(obj)/,,$(@:.o=-objs))) \
+ $($(subst $(obj)/,,$(@:.o=-y)))), $^)
+
+ quiet_cmd_link_multi-y = LD $@
+-cmd_link_multi-y = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps)
++cmd_link_multi-y = $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(link_multi_deps)
+
+ quiet_cmd_link_multi-m = LD [M] $@
+-cmd_link_multi-m = $(LD) $(ld_flags) $(LDFLAGS_MODULE) -o $@ $(link_multi_deps)
++cmd_link_multi-m = $(CC) $(ld_flags) $(LDFLAGS_MODULE) -o $@ $(link_multi_deps)
+
+ # We would rather have a list of rules like
+ # foo.o: $(foo-objs)
+Index: busybox-1.23.2/scripts/Makefile.lib
+===================================================================
+--- busybox-1.23.2.orig/scripts/Makefile.lib
++++ busybox-1.23.2/scripts/Makefile.lib
+@@ -121,7 +121,8 @@ cpp_flags = -Wp,-MD,$(depfile) $(NO
+ # yet ld_flags is fed to ld.
+ #ld_flags = $(LDFLAGS) $(EXTRA_LDFLAGS)
+ # Remove the -Wl, prefix from linker options normally passed through gcc
+-ld_flags = $(filter-out -Wl$(comma)%,$(LDFLAGS) $(EXTRA_LDFLAGS))
++ld_flags = $(filter-out -Wl$(comma)%,$(LDFLAGS) $(EXTRA_LDFLAGS) $(CFLAGS) $(EXTRA_CFLAGS))
++ld_flags_partial = $($(filter-out -shared%, $(filter-out -pie%,$(ld_flags))))
+
+
+ # Finds the multi-part object the current object will be linked into
+@@ -151,10 +152,8 @@ $(obj)/%:: $(src)/%_shipped
+ # Linking
+ # ---------------------------------------------------------------------------
+
+-# TODO: LDFLAGS usually is supposed to contain gcc's flags, not ld's.
+-# but here we feed them to ld!
+-quiet_cmd_ld = LD $@
+-cmd_ld = $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) $(LDFLAGS_$(@F)) \
++quiet_cmd_ld = CC $@
++cmd_ld = $(CC) $(ld_flags) $(LDFLAGS_$(@F)) \
+ $(filter-out FORCE,$^) -o $@
+
+ # Objcopy
diff --git a/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch b/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
new file mode 100644
index 0000000..de286fb
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
@@ -0,0 +1,32 @@
+From df2cc76cdebc4773361477f3db203790f6986e3b Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Sat, 22 Aug 2015 23:42:40 -0700
+Subject: [PATCH 2/2] Passthrough -r to linker
+
+clang does not have -r switch and it does not pass it down to linker
+either, LDFLAGS_RELOCATABLE is used when CC is used for LD, so this
+should not cause side effects
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+---
+Upstream-Status: Pending
+
+ Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/Makefile b/Makefile
+index 9da02cb..10dd4a9 100644
+--- a/Makefile
++++ b/Makefile
+@@ -309,7 +309,7 @@ CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(C
+ MODFLAGS = -DMODULE
+ CFLAGS_MODULE = $(MODFLAGS)
+ AFLAGS_MODULE = $(MODFLAGS)
+-LDFLAGS_RELOCATABLE = -r -nostdlib
++LDFLAGS_RELOCATABLE = -Xlinker -r -nostdlib
+ LDFLAGS_MODULE = $(LDFLAGS_RELOCATABLE)
+ CFLAGS_KERNEL =
+ AFLAGS_KERNEL =
+--
+2.1.4
+
diff --git a/meta/recipes-core/busybox/busybox_1.23.2.bb b/meta/recipes-core/busybox/busybox_1.23.2.bb
index 5e02732..2559823 100644
--- a/meta/recipes-core/busybox/busybox_1.23.2.bb
+++ b/meta/recipes-core/busybox/busybox_1.23.2.bb
@@ -33,6 +33,8 @@ SRC_URI = "http://www.busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \
file://0001-Switch-to-POSIX-utmpx-API.patch \
file://0001-ifconfig-fix-double-free-fatal-error-in-INET_sprint.patch \
file://0001-chown-fix-help-text.patch \
+ file://0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch \
+ file://0002-Passthrough-r-to-linker.patch \
file://mount-via-label.cfg \
file://sha1sum.cfg \
file://sha256sum.cfg \
--
2.5.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 1/4] powertop: Include right headers for timval struct
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
@ 2015-09-25 4:59 ` Khem Raj
2015-09-25 9:28 ` Burton, Ross
2015-09-25 4:59 ` [PATCH] pxz: Add recipe and use it for xz image type Khem Raj
` (4 subsequent siblings)
5 siblings, 1 reply; 18+ messages in thread
From: Khem Raj @ 2015-09-25 4:59 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
| 45 ++++++++++++++++++++++
meta/recipes-kernel/powertop/powertop_2.7.bb | 4 +-
2 files changed, 48 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-kernel/powertop/powertop/0001-include-rquired-headers-for-typedefs.patch
--git a/meta/recipes-kernel/powertop/powertop/0001-include-rquired-headers-for-typedefs.patch b/meta/recipes-kernel/powertop/powertop/0001-include-rquired-headers-for-typedefs.patch
new file mode 100644
index 0000000..5df9961
--- /dev/null
+++ b/meta/recipes-kernel/powertop/powertop/0001-include-rquired-headers-for-typedefs.patch
@@ -0,0 +1,45 @@
+From 0856d8145d187a7e5a49625247abe43a13f95acc Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Tue, 15 Sep 2015 19:36:28 +0000
+Subject: [PATCH] include rquired headers for typedefs
+
+timeval struct needs to include sys/time.h and sprintf() usage requires
+to include stdio.h headers from system
+
+Fixes
+src/perf/perf_bundle.cpp:141:2: error: use of undeclared identifier 'sprintf'; did you mean 'vswprintf'?
+src/devices/devfreq.h:35:18: error: field has incomplete type 'struct timeval'
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+---
+ src/devices/devfreq.h | 1 +
+ src/perf/perf_bundle.cpp | 1 +
+ 2 files changed, 2 insertions(+)
+
+diff --git a/src/devices/devfreq.h b/src/devices/devfreq.h
+index 16a60fb..4bede7c 100644
+--- a/src/devices/devfreq.h
++++ b/src/devices/devfreq.h
+@@ -25,6 +25,7 @@
+ #ifndef _INCLUDE_GUARD_DEVFREQ_H
+ #define _INCLUDE_GUARD_DEVFREQ_H
+
++#include <sys/time.h>
+ #include "device.h"
+ #include "../parameters/parameters.h"
+
+diff --git a/src/perf/perf_bundle.cpp b/src/perf/perf_bundle.cpp
+index cf1ae11..232f894 100644
+--- a/src/perf/perf_bundle.cpp
++++ b/src/perf/perf_bundle.cpp
+@@ -27,6 +27,7 @@
+ #include <algorithm>
+ #include <string.h>
+ #include <stdint.h>
++#include <stdio.h>
+ #include <sys/types.h>
+ #include <sys/stat.h>
+ #include <fcntl.h>
+--
+2.5.2
+
diff --git a/meta/recipes-kernel/powertop/powertop_2.7.bb b/meta/recipes-kernel/powertop/powertop_2.7.bb
index 5ba07e9..4158dd2 100644
--- a/meta/recipes-kernel/powertop/powertop_2.7.bb
+++ b/meta/recipes-kernel/powertop/powertop_2.7.bb
@@ -6,7 +6,9 @@ DEPENDS = "ncurses libnl pciutils"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=12f884d2ae1ff87c09e5b7ccc2c4ca7e"
-SRC_URI = "http://01.org/sites/default/files/downloads/powertop/powertop-${PV}.tar.gz"
+SRC_URI = "http://01.org/sites/default/files/downloads/powertop/powertop-${PV}.tar.gz \
+ file://0001-include-rquired-headers-for-typedefs.patch \
+"
SRC_URI[md5sum] = "e0d686e47daaf7e9d89031f7763432ef"
SRC_URI[sha256sum] = "8d4b1490e2baad4467c0ded3c423db4472dcbf7b2dd8f8f2a928f54047c678ca"
--
2.5.3
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
2015-09-25 4:59 ` [PATCH 1/4] powertop: Include right headers for timval struct Khem Raj
@ 2015-09-25 4:59 ` Khem Raj
2015-09-25 5:01 ` Khem Raj
` (2 more replies)
2015-09-25 4:59 ` [PATCH 2/4] dhcp: Include sys/types.h for u_int* defs Khem Raj
` (3 subsequent siblings)
5 siblings, 3 replies; 18+ messages in thread
From: Khem Raj @ 2015-09-25 4:59 UTC (permalink / raw)
To: openembedded-core
pxz results in significant time saving when generating xz filetypes for
images due to parallelization support
a simple test on ubuntu build host with 16 CPUs
time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
real
23m42.299s
user 23m36.947s
sys 0m5.101s
time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
real2m59.666s
user 24m38.529s
sys 0m10.056s
Signed-off-by: Khem Raj <raj.khem@gmail.com
---
meta/classes/image_types.bbclass | 4 ++--
meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
2 files changed, 33 insertions(+), 2 deletions(-)
create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 306403e..d766cd2 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
-COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
+COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
COMPRESS_DEPENDS_lzma = "xz-native"
COMPRESS_DEPENDS_gz = ""
COMPRESS_DEPENDS_bz2 = "pbzip2-native"
-COMPRESS_DEPENDS_xz = "xz-native"
+COMPRESS_DEPENDS_xz = "pxz-native"
COMPRESS_DEPENDS_lz4 = "lz4-native"
COMPRESS_DEPENDS_sum = "mtd-utils-native"
diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
new file mode 100644
index 0000000..fe21be6
--- /dev/null
+++ b/meta/recipes-extended/pxz/pxz_git.bb
@@ -0,0 +1,31 @@
+# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
+# Released under the MIT license (see COPYING.MIT for the terms)
+
+SUMMARY = "Parallel LZMA compressor compatible with XZ"
+DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
+HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
+LICENSE = "GPL-2.0+"
+LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
+SECTION = "console/utils"
+DEPENDS = "xz"
+
+SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
+PV = "4.999.9beta+git${SRCPV}"
+SRC_URI = "git://github.com/jnovy/pxz.git"
+
+S = "${WORKDIR}/git"
+
+CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
+LDFLAGS_append = " -llzma"
+
+do_compile () {
+ oe_runmake
+}
+
+do_install () {
+ oe_runmake DESTDIR=${D} INSTALL="install -p"
+}
+
+deltask do_configure
+
+BBCLASSEXTEND = "native"
--
2.5.3
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 4:59 ` [PATCH] pxz: Add recipe and use it for xz image type Khem Raj
@ 2015-09-25 5:01 ` Khem Raj
2015-09-25 11:22 ` Gary Thomas
2015-09-25 18:24 ` Andre McCurdy
2 siblings, 0 replies; 18+ messages in thread
From: Khem Raj @ 2015-09-25 5:01 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
ignore this one from applying yet. its still premature but if folks can test it out would be appreciated.
> On Sep 24, 2015, at 9:59 PM, Khem Raj <raj.khem@gmail.com> wrote:
>
> pxz results in significant time saving when generating xz filetypes for
> images due to parallelization support
>
> a simple test on ubuntu build host with 16 CPUs
>
> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>
> real
> 23m42.299s
> user 23m36.947s
> sys 0m5.101s
>
> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>
> real2m59.666s
> user 24m38.529s
> sys 0m10.056s
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com
> ---
> meta/classes/image_types.bbclass | 4 ++--
> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+), 2 deletions(-)
> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index 306403e..d766cd2 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
> COMPRESS_DEPENDS_lzma = "xz-native"
> COMPRESS_DEPENDS_gz = ""
> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
> -COMPRESS_DEPENDS_xz = "xz-native"
> +COMPRESS_DEPENDS_xz = "pxz-native"
> COMPRESS_DEPENDS_lz4 = "lz4-native"
> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>
> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
> new file mode 100644
> index 0000000..fe21be6
> --- /dev/null
> +++ b/meta/recipes-extended/pxz/pxz_git.bb
> @@ -0,0 +1,31 @@
> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
> +# Released under the MIT license (see COPYING.MIT for the terms)
> +
> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
> +LICENSE = "GPL-2.0+"
> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
> +SECTION = "console/utils"
> +DEPENDS = "xz"
> +
> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
> +PV = "4.999.9beta+git${SRCPV}"
> +SRC_URI = "git://github.com/jnovy/pxz.git"
> +
> +S = "${WORKDIR}/git"
> +
> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
> +LDFLAGS_append = " -llzma"
> +
> +do_compile () {
> + oe_runmake
> +}
> +
> +do_install () {
> + oe_runmake DESTDIR=${D} INSTALL="install -p"
> +}
> +
> +deltask do_configure
> +
> +BBCLASSEXTEND = "native"
> --
> 2.5.3
>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 4:59 ` [PATCH] pxz: Add recipe and use it for xz image type Khem Raj
2015-09-25 5:01 ` Khem Raj
@ 2015-09-25 11:22 ` Gary Thomas
2015-09-25 13:25 ` Khem Raj
2015-09-25 18:24 ` Andre McCurdy
2 siblings, 1 reply; 18+ messages in thread
From: Gary Thomas @ 2015-09-25 11:22 UTC (permalink / raw)
To: openembedded-core
On 2015-09-24 22:59, Khem Raj wrote:
> pxz results in significant time saving when generating xz filetypes for
> images due to parallelization support
This adds another host tool dependency - on my Ubuntu (14.04) system, even
if 'xz' is installed, 'pxz' is not and I'd have to add it. Since *.xz seems
to be the new fashion/trend, perhaps this choice might be optional/configurable?
>
> a simple test on ubuntu build host with 16 CPUs
>
> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>
> real
> 23m42.299s
> user 23m36.947s
> sys 0m5.101s
>
> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>
> real2m59.666s
> user 24m38.529s
> sys 0m10.056s
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com
> ---
> meta/classes/image_types.bbclass | 4 ++--
> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+), 2 deletions(-)
> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index 306403e..d766cd2 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
> COMPRESS_DEPENDS_lzma = "xz-native"
> COMPRESS_DEPENDS_gz = ""
> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
> -COMPRESS_DEPENDS_xz = "xz-native"
> +COMPRESS_DEPENDS_xz = "pxz-native"
> COMPRESS_DEPENDS_lz4 = "lz4-native"
> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>
> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
> new file mode 100644
> index 0000000..fe21be6
> --- /dev/null
> +++ b/meta/recipes-extended/pxz/pxz_git.bb
> @@ -0,0 +1,31 @@
> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
> +# Released under the MIT license (see COPYING.MIT for the terms)
> +
> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
> +LICENSE = "GPL-2.0+"
> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
> +SECTION = "console/utils"
> +DEPENDS = "xz"
> +
> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
> +PV = "4.999.9beta+git${SRCPV}"
> +SRC_URI = "git://github.com/jnovy/pxz.git"
> +
> +S = "${WORKDIR}/git"
> +
> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
> +LDFLAGS_append = " -llzma"
> +
> +do_compile () {
> + oe_runmake
> +}
> +
> +do_install () {
> + oe_runmake DESTDIR=${D} INSTALL="install -p"
> +}
> +
> +deltask do_configure
> +
> +BBCLASSEXTEND = "native"
>
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 11:22 ` Gary Thomas
@ 2015-09-25 13:25 ` Khem Raj
2015-09-25 13:32 ` Gary Thomas
0 siblings, 1 reply; 18+ messages in thread
From: Khem Raj @ 2015-09-25 13:25 UTC (permalink / raw)
To: Gary Thomas; +Cc: openembedded-core
> On Sep 25, 2015, at 4:22 AM, Gary Thomas <gary@mlbassoc.com> wrote:
>
> On 2015-09-24 22:59, Khem Raj wrote:
>> pxz results in significant time saving when generating xz filetypes for
>> images due to parallelization support
>
> This adds another host tool dependency - on my Ubuntu (14.04) system, even
> if 'xz' is installed, 'pxz' is not and I'd have to add it. Since *.xz seems
> to be the new fashion/trend, perhaps this choice might be optional/configurable?
I provided pxz recipe as well.. so I don’t get it where is host tool dep.
>
>>
>> a simple test on ubuntu build host with 16 CPUs
>>
>> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>
>> real
>> 23m42.299s
>> user 23m36.947s
>> sys 0m5.101s
>>
>> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>
>> real2m59.666s
>> user 24m38.529s
>> sys 0m10.056s
>>
>> Signed-off-by: Khem Raj <raj.khem@gmail.com
>> ---
>> meta/classes/image_types.bbclass | 4 ++--
>> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
>> 2 files changed, 33 insertions(+), 2 deletions(-)
>> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>>
>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>> index 306403e..d766cd2 100644
>> --- a/meta/classes/image_types.bbclass
>> +++ b/meta/classes/image_types.bbclass
>> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
>> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
>> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
>> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
>> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
>> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
>> COMPRESS_DEPENDS_lzma = "xz-native"
>> COMPRESS_DEPENDS_gz = ""
>> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
>> -COMPRESS_DEPENDS_xz = "xz-native"
>> +COMPRESS_DEPENDS_xz = "pxz-native"
>> COMPRESS_DEPENDS_lz4 = "lz4-native"
>> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>>
>> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
>> new file mode 100644
>> index 0000000..fe21be6
>> --- /dev/null
>> +++ b/meta/recipes-extended/pxz/pxz_git.bb
>> @@ -0,0 +1,31 @@
>> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
>> +# Released under the MIT license (see COPYING.MIT for the terms)
>> +
>> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
>> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
>> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
>> +LICENSE = "GPL-2.0+"
>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
>> +SECTION = "console/utils"
>> +DEPENDS = "xz"
>> +
>> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
>> +PV = "4.999.9beta+git${SRCPV}"
>> +SRC_URI = "git://github.com/jnovy/pxz.git"
>> +
>> +S = "${WORKDIR}/git"
>> +
>> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
>> +LDFLAGS_append = " -llzma"
>> +
>> +do_compile () {
>> + oe_runmake
>> +}
>> +
>> +do_install () {
>> + oe_runmake DESTDIR=${D} INSTALL="install -p"
>> +}
>> +
>> +deltask do_configure
>> +
>> +BBCLASSEXTEND = "native"
>>
>
> --
> ------------------------------------------------------------
> Gary Thomas | Consulting for the
> MLB Associates | Embedded world
> ------------------------------------------------------------
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 13:25 ` Khem Raj
@ 2015-09-25 13:32 ` Gary Thomas
0 siblings, 0 replies; 18+ messages in thread
From: Gary Thomas @ 2015-09-25 13:32 UTC (permalink / raw)
To: Khem Raj; +Cc: openembedded-core
On 2015-09-25 07:25, Khem Raj wrote:
>
>> On Sep 25, 2015, at 4:22 AM, Gary Thomas <gary@mlbassoc.com> wrote:
>>
>> On 2015-09-24 22:59, Khem Raj wrote:
>>> pxz results in significant time saving when generating xz filetypes for
>>> images due to parallelization support
>>
>> This adds another host tool dependency - on my Ubuntu (14.04) system, even
>> if 'xz' is installed, 'pxz' is not and I'd have to add it. Since *.xz seems
>> to be the new fashion/trend, perhaps this choice might be optional/configurable?
>
> I provided pxz recipe as well.. so I don’t get it where is host tool dep.
Fair enough, I missed your additional recipe.
>
>>
>>>
>>> a simple test on ubuntu build host with 16 CPUs
>>>
>>> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>>
>>> real
>>> 23m42.299s
>>> user 23m36.947s
>>> sys 0m5.101s
>>>
>>> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>>
>>> real2m59.666s
>>> user 24m38.529s
>>> sys 0m10.056s
>>>
>>> Signed-off-by: Khem Raj <raj.khem@gmail.com
>>> ---
>>> meta/classes/image_types.bbclass | 4 ++--
>>> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
>>> 2 files changed, 33 insertions(+), 2 deletions(-)
>>> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>>>
>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>> index 306403e..d766cd2 100644
>>> --- a/meta/classes/image_types.bbclass
>>> +++ b/meta/classes/image_types.bbclass
>>> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
>>> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
>>> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
>>> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
>>> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>>> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>>> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
>>> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
>>> COMPRESS_DEPENDS_lzma = "xz-native"
>>> COMPRESS_DEPENDS_gz = ""
>>> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
>>> -COMPRESS_DEPENDS_xz = "xz-native"
>>> +COMPRESS_DEPENDS_xz = "pxz-native"
>>> COMPRESS_DEPENDS_lz4 = "lz4-native"
>>> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>>>
>>> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
>>> new file mode 100644
>>> index 0000000..fe21be6
>>> --- /dev/null
>>> +++ b/meta/recipes-extended/pxz/pxz_git.bb
>>> @@ -0,0 +1,31 @@
>>> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
>>> +# Released under the MIT license (see COPYING.MIT for the terms)
>>> +
>>> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
>>> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
>>> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
>>> +LICENSE = "GPL-2.0+"
>>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
>>> +SECTION = "console/utils"
>>> +DEPENDS = "xz"
>>> +
>>> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
>>> +PV = "4.999.9beta+git${SRCPV}"
>>> +SRC_URI = "git://github.com/jnovy/pxz.git"
>>> +
>>> +S = "${WORKDIR}/git"
>>> +
>>> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
>>> +LDFLAGS_append = " -llzma"
>>> +
>>> +do_compile () {
>>> + oe_runmake
>>> +}
>>> +
>>> +do_install () {
>>> + oe_runmake DESTDIR=${D} INSTALL="install -p"
>>> +}
>>> +
>>> +deltask do_configure
>>> +
>>> +BBCLASSEXTEND = "native"
>>>
>>
>> --
>> ------------------------------------------------------------
>> Gary Thomas | Consulting for the
>> MLB Associates | Embedded world
>> ------------------------------------------------------------
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 4:59 ` [PATCH] pxz: Add recipe and use it for xz image type Khem Raj
2015-09-25 5:01 ` Khem Raj
2015-09-25 11:22 ` Gary Thomas
@ 2015-09-25 18:24 ` Andre McCurdy
2015-09-25 18:34 ` Khem Raj
2 siblings, 1 reply; 18+ messages in thread
From: Andre McCurdy @ 2015-09-25 18:24 UTC (permalink / raw)
To: Khem Raj; +Cc: OE Core mailing list
On Thu, Sep 24, 2015 at 9:59 PM, Khem Raj <raj.khem@gmail.com> wrote:
> pxz results in significant time saving when generating xz filetypes for
> images due to parallelization support
It looks like the version of xz utils already packaged in oe-core
supports multiple threads via the -T option. See XZ_THREADS in
image_types.bbclass.
> a simple test on ubuntu build host with 16 CPUs
>
> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>
> real
> 23m42.299s
> user 23m36.947s
> sys 0m5.101s
>
> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>
> real2m59.666s
> user 24m38.529s
> sys 0m10.056s
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com
> ---
> meta/classes/image_types.bbclass | 4 ++--
> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+), 2 deletions(-)
> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index 306403e..d766cd2 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
> COMPRESS_DEPENDS_lzma = "xz-native"
> COMPRESS_DEPENDS_gz = ""
> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
> -COMPRESS_DEPENDS_xz = "xz-native"
> +COMPRESS_DEPENDS_xz = "pxz-native"
> COMPRESS_DEPENDS_lz4 = "lz4-native"
> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>
> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
> new file mode 100644
> index 0000000..fe21be6
> --- /dev/null
> +++ b/meta/recipes-extended/pxz/pxz_git.bb
> @@ -0,0 +1,31 @@
> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
> +# Released under the MIT license (see COPYING.MIT for the terms)
> +
> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
> +LICENSE = "GPL-2.0+"
> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
> +SECTION = "console/utils"
> +DEPENDS = "xz"
> +
> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
> +PV = "4.999.9beta+git${SRCPV}"
> +SRC_URI = "git://github.com/jnovy/pxz.git"
> +
> +S = "${WORKDIR}/git"
> +
> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
> +LDFLAGS_append = " -llzma"
> +
> +do_compile () {
> + oe_runmake
> +}
> +
> +do_install () {
> + oe_runmake DESTDIR=${D} INSTALL="install -p"
> +}
> +
> +deltask do_configure
> +
> +BBCLASSEXTEND = "native"
> --
> 2.5.3
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 18:24 ` Andre McCurdy
@ 2015-09-25 18:34 ` Khem Raj
2015-09-25 20:18 ` Andre McCurdy
0 siblings, 1 reply; 18+ messages in thread
From: Khem Raj @ 2015-09-25 18:34 UTC (permalink / raw)
To: Andre McCurdy; +Cc: OE Core mailing list
> On Sep 25, 2015, at 11:24 AM, Andre McCurdy <armccurdy@gmail.com> wrote:
>
> On Thu, Sep 24, 2015 at 9:59 PM, Khem Raj <raj.khem@gmail.com> wrote:
>> pxz results in significant time saving when generating xz filetypes for
>> images due to parallelization support
>
> It looks like the version of xz utils already packaged in oe-core
> supports multiple threads via the -T option. See XZ_THREADS in
> image_types.bbclass.
the option -T is there for long time but does it work ?
>
>> a simple test on ubuntu build host with 16 CPUs
>>
>> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>
>> real
>> 23m42.299s
>> user 23m36.947s
>> sys 0m5.101s
>>
>> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>
>> real2m59.666s
>> user 24m38.529s
>> sys 0m10.056s
>>
>> Signed-off-by: Khem Raj <raj.khem@gmail.com
>> ---
>> meta/classes/image_types.bbclass | 4 ++--
>> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
>> 2 files changed, 33 insertions(+), 2 deletions(-)
>> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>>
>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>> index 306403e..d766cd2 100644
>> --- a/meta/classes/image_types.bbclass
>> +++ b/meta/classes/image_types.bbclass
>> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
>> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
>> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
>> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
>> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
>> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
>> COMPRESS_DEPENDS_lzma = "xz-native"
>> COMPRESS_DEPENDS_gz = ""
>> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
>> -COMPRESS_DEPENDS_xz = "xz-native"
>> +COMPRESS_DEPENDS_xz = "pxz-native"
>> COMPRESS_DEPENDS_lz4 = "lz4-native"
>> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>>
>> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
>> new file mode 100644
>> index 0000000..fe21be6
>> --- /dev/null
>> +++ b/meta/recipes-extended/pxz/pxz_git.bb
>> @@ -0,0 +1,31 @@
>> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
>> +# Released under the MIT license (see COPYING.MIT for the terms)
>> +
>> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
>> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
>> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
>> +LICENSE = "GPL-2.0+"
>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
>> +SECTION = "console/utils"
>> +DEPENDS = "xz"
>> +
>> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
>> +PV = "4.999.9beta+git${SRCPV}"
>> +SRC_URI = "git://github.com/jnovy/pxz.git"
>> +
>> +S = "${WORKDIR}/git"
>> +
>> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
>> +LDFLAGS_append = " -llzma"
>> +
>> +do_compile () {
>> + oe_runmake
>> +}
>> +
>> +do_install () {
>> + oe_runmake DESTDIR=${D} INSTALL="install -p"
>> +}
>> +
>> +deltask do_configure
>> +
>> +BBCLASSEXTEND = "native"
>> --
>> 2.5.3
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH] pxz: Add recipe and use it for xz image type
2015-09-25 18:34 ` Khem Raj
@ 2015-09-25 20:18 ` Andre McCurdy
0 siblings, 0 replies; 18+ messages in thread
From: Andre McCurdy @ 2015-09-25 20:18 UTC (permalink / raw)
To: Khem Raj; +Cc: OE Core mailing list
On Fri, Sep 25, 2015 at 11:34 AM, Khem Raj <raj.khem@gmail.com> wrote:
>
>> On Sep 25, 2015, at 11:24 AM, Andre McCurdy <armccurdy@gmail.com> wrote:
>>
>> On Thu, Sep 24, 2015 at 9:59 PM, Khem Raj <raj.khem@gmail.com> wrote:
>>> pxz results in significant time saving when generating xz filetypes for
>>> images due to parallelization support
>>
>> It looks like the version of xz utils already packaged in oe-core
>> supports multiple threads via the -T option. See XZ_THREADS in
>> image_types.bbclass.
>
> the option -T is there for long time but does it work ?
I don't know. The -T option isn't supported by the version of xz
packaged in Ubuntu 14.04, so maybe it has issues (oe-core commit
cfd201ec suggests that xz -T 0 can use a lot of memory).
pxz just looks like a simple wrapper on top of liblzma though, so I'd
be surprised if it worked fundamentally better than xz.
How much memory did your pxz test use? From the pxz source it looks
like the -M option to limit memory usage is accepted but ignored...
>>> a simple test on ubuntu build host with 16 CPUs
>>>
>>> time xz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>>
>>> real
>>> 23m42.299s
>>> user 23m36.947s
>>> sys 0m5.101s
>>>
>>> time pxz -M 50% -f -k -c -e -9 --check=crc32 CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio >CX041AEI_PROD_default_20150610000105sdy-dbg.rootfs.cpio.xz
>>>
>>> real2m59.666s
>>> user 24m38.529s
>>> sys 0m10.056s
>>>
>>> Signed-off-by: Khem Raj <raj.khem@gmail.com
>>> ---
>>> meta/classes/image_types.bbclass | 4 ++--
>>> meta/recipes-extended/pxz/pxz_git.bb | 31 +++++++++++++++++++++++++++++++
>>> 2 files changed, 33 insertions(+), 2 deletions(-)
>>> create mode 100644 meta/recipes-extended/pxz/pxz_git.bb
>>>
>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>> index 306403e..d766cd2 100644
>>> --- a/meta/classes/image_types.bbclass
>>> +++ b/meta/classes/image_types.bbclass
>>> @@ -233,13 +233,13 @@ COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
>>> COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
>>> COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
>>> COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
>>> -COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>>> +COMPRESS_CMD_xz = "pxz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
>>> COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
>>> COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
>>> COMPRESS_DEPENDS_lzma = "xz-native"
>>> COMPRESS_DEPENDS_gz = ""
>>> COMPRESS_DEPENDS_bz2 = "pbzip2-native"
>>> -COMPRESS_DEPENDS_xz = "xz-native"
>>> +COMPRESS_DEPENDS_xz = "pxz-native"
>>> COMPRESS_DEPENDS_lz4 = "lz4-native"
>>> COMPRESS_DEPENDS_sum = "mtd-utils-native"
>>>
>>> diff --git a/meta/recipes-extended/pxz/pxz_git.bb b/meta/recipes-extended/pxz/pxz_git.bb
>>> new file mode 100644
>>> index 0000000..fe21be6
>>> --- /dev/null
>>> +++ b/meta/recipes-extended/pxz/pxz_git.bb
>>> @@ -0,0 +1,31 @@
>>> +# Copyright (C) 2015 Khem Raj <raj.khem@gmail.com>
>>> +# Released under the MIT license (see COPYING.MIT for the terms)
>>> +
>>> +SUMMARY = "Parallel LZMA compressor compatible with XZ"
>>> +DESCRIPTION = "Parallel XZ is a compression utility that takes advantage of running LZMA compression of different parts of an input file on multiple cores and processors simultaneously. Its primary goal is to utilize all resources to speed up compression time with minimal possible influence on compression ratio"
>>> +HOMEPAGE = "https://jnovy.fedorapeople.org/pxz/"
>>> +LICENSE = "GPL-2.0+"
>>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
>>> +SECTION = "console/utils"
>>> +DEPENDS = "xz"
>>> +
>>> +SRCREV = "ae808463c2950edfdedb8fb49f95006db0a18667"
>>> +PV = "4.999.9beta+git${SRCPV}"
>>> +SRC_URI = "git://github.com/jnovy/pxz.git"
>>> +
>>> +S = "${WORKDIR}/git"
>>> +
>>> +CFLAGS_append = " -fopenmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
>>> +LDFLAGS_append = " -llzma"
>>> +
>>> +do_compile () {
>>> + oe_runmake
>>> +}
>>> +
>>> +do_install () {
>>> + oe_runmake DESTDIR=${D} INSTALL="install -p"
>>> +}
>>> +
>>> +deltask do_configure
>>> +
>>> +BBCLASSEXTEND = "native"
>>> --
>>> 2.5.3
>>>
>>> --
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 2/4] dhcp: Include sys/types.h for u_int* defs
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
2015-09-25 4:59 ` [PATCH 1/4] powertop: Include right headers for timval struct Khem Raj
2015-09-25 4:59 ` [PATCH] pxz: Add recipe and use it for xz image type Khem Raj
@ 2015-09-25 4:59 ` Khem Raj
2015-09-25 4:59 ` [PATCH 3/4] blktrace: Include <sys/types.h for dev_t Khem Raj
` (2 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Khem Raj @ 2015-09-25 4:59 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
.../dhcp/0001-include-missing-sys-types.h.patch | 56 ++++++++++++++++++++++
meta/recipes-connectivity/dhcp/dhcp_4.3.2.bb | 1 +
2 files changed, 57 insertions(+)
create mode 100644 meta/recipes-connectivity/dhcp/dhcp/0001-include-missing-sys-types.h.patch
diff --git a/meta/recipes-connectivity/dhcp/dhcp/0001-include-missing-sys-types.h.patch b/meta/recipes-connectivity/dhcp/dhcp/0001-include-missing-sys-types.h.patch
new file mode 100644
index 0000000..6d40290
--- /dev/null
+++ b/meta/recipes-connectivity/dhcp/dhcp/0001-include-missing-sys-types.h.patch
@@ -0,0 +1,56 @@
+From 971491b6d7ab47d2f92f34269a129d0347fba15a Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Mon, 14 Sep 2015 23:51:38 +0000
+Subject: [PATCH] include missing sys/types.h
+
+Code uses plain typedefs defines in this header but does not include it
+Fixes
+
+error: unknown type name 'u_int16_t'; did you mean 'uint16_t'?
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+---
+ dst/dst_api.c | 1 +
+ dst/dst_support.c | 1 +
+ dst/hmac_link.c | 1 +
+ 3 files changed, 3 insertions(+)
+
+diff --git a/dst/dst_api.c b/dst/dst_api.c
+index a77abd2..7885538 100644
+--- a/dst/dst_api.c
++++ b/dst/dst_api.c
+@@ -49,6 +49,7 @@
+ #include <sys/param.h>
+ #include <sys/stat.h>
+ #include <sys/socket.h>
++#include <sys/types.h>
+ #include <netinet/in.h>
+
+ #include "cdefs.h"
+diff --git a/dst/dst_support.c b/dst/dst_support.c
+index 8e08a0c..f353ec6 100644
+--- a/dst/dst_support.c
++++ b/dst/dst_support.c
+@@ -25,6 +25,7 @@
+ #include <sys/stat.h>
+ #include <netinet/in.h>
+ #include <sys/socket.h>
++#include <sys/types.h>
+
+ #include "cdefs.h"
+ #include "osdep.h"
+diff --git a/dst/hmac_link.c b/dst/hmac_link.c
+index b812c1c..a7de622 100644
+--- a/dst/hmac_link.c
++++ b/dst/hmac_link.c
+@@ -31,6 +31,7 @@
+ #include <sys/time.h>
+ #include <netinet/in.h>
+ #include <sys/socket.h>
++#include <sys/types.h>
+
+ #include "cdefs.h"
+ #include "osdep.h"
+--
+2.5.2
+
diff --git a/meta/recipes-connectivity/dhcp/dhcp_4.3.2.bb b/meta/recipes-connectivity/dhcp/dhcp_4.3.2.bb
index b4a05fc..436a32e 100644
--- a/meta/recipes-connectivity/dhcp/dhcp_4.3.2.bb
+++ b/meta/recipes-connectivity/dhcp/dhcp_4.3.2.bb
@@ -6,6 +6,7 @@ SRC_URI += "file://dhcp-3.0.3-dhclient-dbus.patch;striplevel=0 \
file://fixsepbuild.patch \
file://dhclient-script-drop-resolv.conf.dhclient.patch \
file://replace-ifconfig-route.patch \
+ file://0001-include-missing-sys-types.h.patch \
"
SRC_URI[md5sum] = "5a284875dd2c12ddd388416d69156a67"
--
2.5.3
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 3/4] blktrace: Include <sys/types.h for dev_t
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
` (2 preceding siblings ...)
2015-09-25 4:59 ` [PATCH 2/4] dhcp: Include sys/types.h for u_int* defs Khem Raj
@ 2015-09-25 4:59 ` Khem Raj
2015-09-25 9:29 ` Burton, Ross
2015-09-25 4:59 ` [PATCH 4/4] waffle: Fix build with musl Khem Raj
2015-09-25 5:06 ` [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
5 siblings, 1 reply; 18+ messages in thread
From: Khem Raj @ 2015-09-25 4:59 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
...-include-sys-types.h-for-dev_t-definition.patch | 30 ++++++++++++++++++++++
meta/recipes-kernel/blktrace/blktrace_git.bb | 4 ++-
2 files changed, 33 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-kernel/blktrace/blktrace/0001-include-sys-types.h-for-dev_t-definition.patch
diff --git a/meta/recipes-kernel/blktrace/blktrace/0001-include-sys-types.h-for-dev_t-definition.patch b/meta/recipes-kernel/blktrace/blktrace/0001-include-sys-types.h-for-dev_t-definition.patch
new file mode 100644
index 0000000..f63868f
--- /dev/null
+++ b/meta/recipes-kernel/blktrace/blktrace/0001-include-sys-types.h-for-dev_t-definition.patch
@@ -0,0 +1,30 @@
+From 6b5bbdfaac7f216fe8a02c4cf29e5eb2aee5a409 Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Tue, 15 Sep 2015 00:01:00 +0000
+Subject: [PATCH] include sys/types.h for dev_t definition
+
+Avoids the build failures when sys/types.h does not get included
+indirectly through other headers.
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+---
+Status: Submitted
+
+ blktrace.h | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/blktrace.h b/blktrace.h
+index 380aec7..944fc08 100644
+--- a/blktrace.h
++++ b/blktrace.h
+@@ -5,6 +5,7 @@
+ #include <limits.h>
+ #include <byteswap.h>
+ #include <endian.h>
++#include <sys/types.h>
+
+ #include "blktrace_api.h"
+ #include "rbtree.h"
+--
+2.5.2
+
diff --git a/meta/recipes-kernel/blktrace/blktrace_git.bb b/meta/recipes-kernel/blktrace/blktrace_git.bb
index 5b0be42..957cb70 100644
--- a/meta/recipes-kernel/blktrace/blktrace_git.bb
+++ b/meta/recipes-kernel/blktrace/blktrace_git.bb
@@ -9,7 +9,9 @@ SRCREV = "43fc870ce04e963def45dfc0d1ed4ea21ef10d4b"
PV = "1.1.0+git${SRCPV}"
SRC_URI = "git://git.kernel.dk/blktrace.git \
- file://ldflags.patch"
+ file://ldflags.patch \
+ file://0001-include-sys-types.h-for-dev_t-definition.patch \
+"
S = "${WORKDIR}/git"
--
2.5.3
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 4/4] waffle: Fix build with musl
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
` (3 preceding siblings ...)
2015-09-25 4:59 ` [PATCH 3/4] blktrace: Include <sys/types.h for dev_t Khem Raj
@ 2015-09-25 4:59 ` Khem Raj
2015-09-25 5:06 ` [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
5 siblings, 0 replies; 18+ messages in thread
From: Khem Raj @ 2015-09-25 4:59 UTC (permalink / raw)
To: openembedded-core
Backport needed patches for compilation fixes on musl
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
...-threads-Use-PTHREAD_MUTEX_RECURSIVE-by-d.patch | 54 ++++++++++++++++++++++
meta/recipes-graphics/waffle/waffle_1.5.1.bb | 4 +-
2 files changed, 57 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-graphics/waffle/waffle/0001-third_party-threads-Use-PTHREAD_MUTEX_RECURSIVE-by-d.patch
diff --git a/meta/recipes-graphics/waffle/waffle/0001-third_party-threads-Use-PTHREAD_MUTEX_RECURSIVE-by-d.patch b/meta/recipes-graphics/waffle/waffle/0001-third_party-threads-Use-PTHREAD_MUTEX_RECURSIVE-by-d.patch
new file mode 100644
index 0000000..a0c826e
--- /dev/null
+++ b/meta/recipes-graphics/waffle/waffle/0001-third_party-threads-Use-PTHREAD_MUTEX_RECURSIVE-by-d.patch
@@ -0,0 +1,54 @@
+From 3b9b8f5f6d1b99af43e95ec0868404e552a85b73 Mon Sep 17 00:00:00 2001
+From: Emil Velikov <emil.l.velikov@gmail.com>
+Date: Thu, 19 Mar 2015 22:26:11 +0000
+Subject: [PATCH] third_party/threads: Use PTHREAD_MUTEX_RECURSIVE by default
+
+PTHREAD_MUTEX_RECURSIVE_NP was used for compatibility with old glibc.
+Although due to the_GNU_SOURCES define the portable,
+PTHREAD_MUTEX_RECURSIVE will be available for Linuxes since at least
+1998. Simplify things giving us compatibility with musl which
+apparently does not provide the non-portable define.
+
+Inspired by almost identical commit in mesa aead7fe2e2b(c11/threads: Use
+PTHREAD_MUTEX_RECURSIVE by default) by Felix Janda.
+
+Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
+Reviewed-by: Chad Versace <chad.versace@intel.com>
+---
+Upstream-Status: Backport
+
+ third_party/threads/threads_posix.c | 10 ++++------
+ 1 file changed, 4 insertions(+), 6 deletions(-)
+
+diff --git a/third_party/threads/threads_posix.c b/third_party/threads/threads_posix.c
+index 5835e43..e122bf9 100644
+--- a/third_party/threads/threads_posix.c
++++ b/third_party/threads/threads_posix.c
+@@ -26,6 +26,9 @@
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
++
++#define _GNU_SOURCE
++
+ #include <stdlib.h>
+ #ifndef assert
+ #include <assert.h>
+@@ -150,13 +153,8 @@ int mtx_init(mtx_t *mtx, int type)
+ && type != (mtx_try|mtx_recursive))
+ return thrd_error;
+ pthread_mutexattr_init(&attr);
+- if ((type & mtx_recursive) != 0) {
+-#if defined(__linux__) || defined(__linux)
+- pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP);
+-#else
++ if ((type & mtx_recursive) != 0)
+ pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
+-#endif
+- }
+ pthread_mutex_init(mtx, &attr);
+ pthread_mutexattr_destroy(&attr);
+ return thrd_success;
+--
+2.5.2
+
diff --git a/meta/recipes-graphics/waffle/waffle_1.5.1.bb b/meta/recipes-graphics/waffle/waffle_1.5.1.bb
index b8aa05a..af84020 100644
--- a/meta/recipes-graphics/waffle/waffle_1.5.1.bb
+++ b/meta/recipes-graphics/waffle/waffle_1.5.1.bb
@@ -3,7 +3,9 @@ LICENSE = "BSD-2-Clause"
LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=4c5154407c2490750dd461c50ad94797 \
file://include/waffle/waffle.h;endline=24;md5=61dbf8697f61c78645e75a93c585b1bf"
-SRC_URI = "http://waffle-gl.org/files/release/${BPN}-${PV}/${BPN}-${PV}.tar.xz"
+SRC_URI = "http://waffle-gl.org/files/release/${BPN}-${PV}/${BPN}-${PV}.tar.xz \
+ file://0001-third_party-threads-Use-PTHREAD_MUTEX_RECURSIVE-by-d.patch \
+ "
SRC_URI[md5sum] = "c0d802bc3d0aba87c51e423a3a8bdd69"
SRC_URI[sha256sum] = "cbab0e926515064e818bf089a5af04be33307e5f40d07659fb40d59b2bfe20aa"
--
2.5.3
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH V2] busybox: Use CC instead of bare LD to be the Linker
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
` (4 preceding siblings ...)
2015-09-25 4:59 ` [PATCH 4/4] waffle: Fix build with musl Khem Raj
@ 2015-09-25 5:06 ` Khem Raj
2015-09-25 9:33 ` Burton, Ross
5 siblings, 1 reply; 18+ messages in thread
From: Khem Raj @ 2015-09-25 5:06 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
this is a resend of the very same patch which was send 2-3 weeks ago
> On Sep 24, 2015, at 9:59 PM, Khem Raj <raj.khem@gmail.com> wrote:
>
> This patch was on mailing list, another patch to make sure -r is not
> passed directly but via -Wl switch is added.
>
> This was exposed when using clang and gold linker, clang does not have
> -r switch to do relocatable objects and problem happens specific to OE
> becuase we use LD = CC
>
> now what happens is that busybox assumes that linker will be called
> directly, and hence sprinkles linkers options in its kbuild system which
> aggregate into LDFLAGS, some of these options are happily ignored by gcc
> as well but it passes -r options rightly to linker so it all works,
> however when using clang, this falls apart since -r is not known option
> for clang so it drops this option and all obects which should be
> partially linked becomes ET_EXEC and when they are added to final link
> then gold starts to get confused
>
> /mnt/home/kraj/work/angstrom/build/tmp-angstrom-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-ld:
> error: applets/built-in.o: unsupported ELF file type 2
> clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
> ...-linking-instead-of-LD-and-use-CFLAGS-and.patch | 114 +++++++++++++++++++++
> .../busybox/0002-Passthrough-r-to-linker.patch | 32 ++++++
> meta/recipes-core/busybox/busybox_1.23.2.bb | 2 +
> 3 files changed, 148 insertions(+)
> create mode 100644 meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
> create mode 100644 meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
>
> diff --git a/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch b/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
> new file mode 100644
> index 0000000..2bf2b91
> --- /dev/null
> +++ b/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
> @@ -0,0 +1,114 @@
> +From a9333eb6a7b8dbda735947cd5bc981ff9352a2c9 Mon Sep 17 00:00:00 2001
> +From: Nathan Phillip Brink <ohnobinki@ohnopublishing.net>
> +Date: Thu, 10 Mar 2011 00:27:08 -0500
> +Subject: [PATCH 1/2] Use $(CC) when linking instead of $(LD) and use $(CFLAGS)
> + and $(EXTRA_CFLAGS) when linking.
> +
> +This fixes the issue where LDFLAGS escaped with -Wl are ignored during
> +compilation. It also simplifies using CFLAGS or EXTRA_CFLAGS (such as
> +-m32 on x86_64 or -flto) which apply to both compilation and linking
> +situations.
> +
> +Signed-off-by: Nathan Phillip Brink <ohnobinki@ohnopublishing.net>
> +---
> +Upstream-Status: Pending
> +
> + Makefile | 7 ++++---
> + scripts/Makefile.build | 8 ++++----
> + scripts/Makefile.lib | 13 +++----------
> + 3 files changed, 11 insertions(+), 17 deletions(-)
> +
> +Index: busybox-1.23.2/Makefile
> +===================================================================
> +--- busybox-1.23.2.orig/Makefile
> ++++ busybox-1.23.2/Makefile
> +@@ -309,7 +309,8 @@ CHECKFLAGS := -D__linux__ -Dlinux -D
> + MODFLAGS = -DMODULE
> + CFLAGS_MODULE = $(MODFLAGS)
> + AFLAGS_MODULE = $(MODFLAGS)
> +-LDFLAGS_MODULE = -r
> ++LDFLAGS_RELOCATABLE = -r -nostdlib
> ++LDFLAGS_MODULE = $(LDFLAGS_RELOCATABLE)
> + CFLAGS_KERNEL =
> + AFLAGS_KERNEL =
> +
> +@@ -331,7 +332,7 @@ KERNELVERSION = $(VERSION).$(PATCHLEVEL)
> + export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION \
> + ARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC \
> + CPP AR NM STRIP OBJCOPY OBJDUMP MAKE AWK GENKSYMS PERL UTS_MACHINE \
> +- HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS
> ++ HOSTCXX HOSTCXXFLAGS LDFLAGS_RELOCATABLE LDFLAGS_MODULE CHECK CHECKFLAGS
> +
> + export CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS
> + export CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
> +@@ -610,7 +611,7 @@ quiet_cmd_busybox__ ?= LINK $@
> + cmd_busybox__ ?= $(srctree)/scripts/trylink \
> + "$@" \
> + "$(CC)" \
> +- "$(CFLAGS) $(CFLAGS_busybox)" \
> ++ "$(CFLAGS) $(CFLAGS_busybox) $(EXTRA_CFLAGS)" \
> + "$(LDFLAGS) $(EXTRA_LDFLAGS)" \
> + "$(core-y)" \
> + "$(libs-y)" \
> +Index: busybox-1.23.2/scripts/Makefile.build
> +===================================================================
> +--- busybox-1.23.2.orig/scripts/Makefile.build
> ++++ busybox-1.23.2/scripts/Makefile.build
> +@@ -174,7 +174,7 @@ cmd_modversions = \
> + | $(GENKSYMS) -a $(ARCH) \
> + > $(@D)/.tmp_$(@F:.o=.ver); \
> + \
> +- $(LD) $(LDFLAGS) -r -o $@ $(@D)/.tmp_$(@F) \
> ++ $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(@D)/.tmp_$(@F) \
> + -T $(@D)/.tmp_$(@F:.o=.ver); \
> + rm -f $(@D)/.tmp_$(@F) $(@D)/.tmp_$(@F:.o=.ver); \
> + else \
> +@@ -257,7 +257,7 @@ quiet_cmd_link_o_target = LD $@
> + # If the list of objects to link is empty, just create an empty built-in.o
> + # -nostdlib is added to make "make LD=gcc ..." work (some people use that)
> + cmd_link_o_target = $(if $(strip $(obj-y)),\
> +- $(LD) -nostdlib $(ld_flags) -r -o $@ $(filter $(obj-y), $^),\
> ++ $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(filter $(obj-y), $^),\
> + rm -f $@; $(AR) rcs $@)
> +
> + $(builtin-target): $(obj-y) FORCE
> +@@ -292,10 +292,10 @@ $($(subst $(obj)/,,$(@:.o=-objs))) \
> + $($(subst $(obj)/,,$(@:.o=-y)))), $^)
> +
> + quiet_cmd_link_multi-y = LD $@
> +-cmd_link_multi-y = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps)
> ++cmd_link_multi-y = $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(link_multi_deps)
> +
> + quiet_cmd_link_multi-m = LD [M] $@
> +-cmd_link_multi-m = $(LD) $(ld_flags) $(LDFLAGS_MODULE) -o $@ $(link_multi_deps)
> ++cmd_link_multi-m = $(CC) $(ld_flags) $(LDFLAGS_MODULE) -o $@ $(link_multi_deps)
> +
> + # We would rather have a list of rules like
> + # foo.o: $(foo-objs)
> +Index: busybox-1.23.2/scripts/Makefile.lib
> +===================================================================
> +--- busybox-1.23.2.orig/scripts/Makefile.lib
> ++++ busybox-1.23.2/scripts/Makefile.lib
> +@@ -121,7 +121,8 @@ cpp_flags = -Wp,-MD,$(depfile) $(NO
> + # yet ld_flags is fed to ld.
> + #ld_flags = $(LDFLAGS) $(EXTRA_LDFLAGS)
> + # Remove the -Wl, prefix from linker options normally passed through gcc
> +-ld_flags = $(filter-out -Wl$(comma)%,$(LDFLAGS) $(EXTRA_LDFLAGS))
> ++ld_flags = $(filter-out -Wl$(comma)%,$(LDFLAGS) $(EXTRA_LDFLAGS) $(CFLAGS) $(EXTRA_CFLAGS))
> ++ld_flags_partial = $($(filter-out -shared%, $(filter-out -pie%,$(ld_flags))))
> +
> +
> + # Finds the multi-part object the current object will be linked into
> +@@ -151,10 +152,8 @@ $(obj)/%:: $(src)/%_shipped
> + # Linking
> + # ---------------------------------------------------------------------------
> +
> +-# TODO: LDFLAGS usually is supposed to contain gcc's flags, not ld's.
> +-# but here we feed them to ld!
> +-quiet_cmd_ld = LD $@
> +-cmd_ld = $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) $(LDFLAGS_$(@F)) \
> ++quiet_cmd_ld = CC $@
> ++cmd_ld = $(CC) $(ld_flags) $(LDFLAGS_$(@F)) \
> + $(filter-out FORCE,$^) -o $@
> +
> + # Objcopy
> diff --git a/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch b/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
> new file mode 100644
> index 0000000..de286fb
> --- /dev/null
> +++ b/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
> @@ -0,0 +1,32 @@
> +From df2cc76cdebc4773361477f3db203790f6986e3b Mon Sep 17 00:00:00 2001
> +From: Khem Raj <raj.khem@gmail.com>
> +Date: Sat, 22 Aug 2015 23:42:40 -0700
> +Subject: [PATCH 2/2] Passthrough -r to linker
> +
> +clang does not have -r switch and it does not pass it down to linker
> +either, LDFLAGS_RELOCATABLE is used when CC is used for LD, so this
> +should not cause side effects
> +
> +Signed-off-by: Khem Raj <raj.khem@gmail.com>
> +---
> +Upstream-Status: Pending
> +
> + Makefile | 2 +-
> + 1 file changed, 1 insertion(+), 1 deletion(-)
> +
> +diff --git a/Makefile b/Makefile
> +index 9da02cb..10dd4a9 100644
> +--- a/Makefile
> ++++ b/Makefile
> +@@ -309,7 +309,7 @@ CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(C
> + MODFLAGS = -DMODULE
> + CFLAGS_MODULE = $(MODFLAGS)
> + AFLAGS_MODULE = $(MODFLAGS)
> +-LDFLAGS_RELOCATABLE = -r -nostdlib
> ++LDFLAGS_RELOCATABLE = -Xlinker -r -nostdlib
> + LDFLAGS_MODULE = $(LDFLAGS_RELOCATABLE)
> + CFLAGS_KERNEL =
> + AFLAGS_KERNEL =
> +--
> +2.1.4
> +
> diff --git a/meta/recipes-core/busybox/busybox_1.23.2.bb b/meta/recipes-core/busybox/busybox_1.23.2.bb
> index 5e02732..2559823 100644
> --- a/meta/recipes-core/busybox/busybox_1.23.2.bb
> +++ b/meta/recipes-core/busybox/busybox_1.23.2.bb
> @@ -33,6 +33,8 @@ SRC_URI = "http://www.busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \
> file://0001-Switch-to-POSIX-utmpx-API.patch \
> file://0001-ifconfig-fix-double-free-fatal-error-in-INET_sprint.patch \
> file://0001-chown-fix-help-text.patch \
> + file://0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch \
> + file://0002-Passthrough-r-to-linker.patch \
> file://mount-via-label.cfg \
> file://sha1sum.cfg \
> file://sha256sum.cfg \
> --
> 2.5.1
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH V2] busybox: Use CC instead of bare LD to be the Linker
@ 2015-09-09 5:53 Khem Raj
0 siblings, 0 replies; 18+ messages in thread
From: Khem Raj @ 2015-09-09 5:53 UTC (permalink / raw)
To: openembedded-core
This patch was on mailing list, another patch to make sure -r is not
passed directly but via -Wl switch is added.
This was exposed when using clang and gold linker, clang does not have
-r switch to do relocatable objects and problem happens specific to OE
becuase we use LD = CC
now what happens is that busybox assumes that linker will be called
directly, and hence sprinkles linkers options in its kbuild system which
aggregate into LDFLAGS, some of these options are happily ignored by gcc
as well but it passes -r options rightly to linker so it all works,
however when using clang, this falls apart since -r is not known option
for clang so it drops this option and all obects which should be
partially linked becomes ET_EXEC and when they are added to final link
then gold starts to get confused
/mnt/home/kraj/work/angstrom/build/tmp-angstrom-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-ld:
error: applets/built-in.o: unsupported ELF file type 2
clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
...-linking-instead-of-LD-and-use-CFLAGS-and.patch | 114 +++++++++++++++++++++
.../busybox/0002-Passthrough-r-to-linker.patch | 32 ++++++
meta/recipes-core/busybox/busybox_1.23.2.bb | 2 +
3 files changed, 148 insertions(+)
create mode 100644 meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
create mode 100644 meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
diff --git a/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch b/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
new file mode 100644
index 0000000..2bf2b91
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch
@@ -0,0 +1,114 @@
+From a9333eb6a7b8dbda735947cd5bc981ff9352a2c9 Mon Sep 17 00:00:00 2001
+From: Nathan Phillip Brink <ohnobinki@ohnopublishing.net>
+Date: Thu, 10 Mar 2011 00:27:08 -0500
+Subject: [PATCH 1/2] Use $(CC) when linking instead of $(LD) and use $(CFLAGS)
+ and $(EXTRA_CFLAGS) when linking.
+
+This fixes the issue where LDFLAGS escaped with -Wl are ignored during
+compilation. It also simplifies using CFLAGS or EXTRA_CFLAGS (such as
+-m32 on x86_64 or -flto) which apply to both compilation and linking
+situations.
+
+Signed-off-by: Nathan Phillip Brink <ohnobinki@ohnopublishing.net>
+---
+Upstream-Status: Pending
+
+ Makefile | 7 ++++---
+ scripts/Makefile.build | 8 ++++----
+ scripts/Makefile.lib | 13 +++----------
+ 3 files changed, 11 insertions(+), 17 deletions(-)
+
+Index: busybox-1.23.2/Makefile
+===================================================================
+--- busybox-1.23.2.orig/Makefile
++++ busybox-1.23.2/Makefile
+@@ -309,7 +309,8 @@ CHECKFLAGS := -D__linux__ -Dlinux -D
+ MODFLAGS = -DMODULE
+ CFLAGS_MODULE = $(MODFLAGS)
+ AFLAGS_MODULE = $(MODFLAGS)
+-LDFLAGS_MODULE = -r
++LDFLAGS_RELOCATABLE = -r -nostdlib
++LDFLAGS_MODULE = $(LDFLAGS_RELOCATABLE)
+ CFLAGS_KERNEL =
+ AFLAGS_KERNEL =
+
+@@ -331,7 +332,7 @@ KERNELVERSION = $(VERSION).$(PATCHLEVEL)
+ export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION \
+ ARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC \
+ CPP AR NM STRIP OBJCOPY OBJDUMP MAKE AWK GENKSYMS PERL UTS_MACHINE \
+- HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS
++ HOSTCXX HOSTCXXFLAGS LDFLAGS_RELOCATABLE LDFLAGS_MODULE CHECK CHECKFLAGS
+
+ export CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS
+ export CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
+@@ -610,7 +611,7 @@ quiet_cmd_busybox__ ?= LINK $@
+ cmd_busybox__ ?= $(srctree)/scripts/trylink \
+ "$@" \
+ "$(CC)" \
+- "$(CFLAGS) $(CFLAGS_busybox)" \
++ "$(CFLAGS) $(CFLAGS_busybox) $(EXTRA_CFLAGS)" \
+ "$(LDFLAGS) $(EXTRA_LDFLAGS)" \
+ "$(core-y)" \
+ "$(libs-y)" \
+Index: busybox-1.23.2/scripts/Makefile.build
+===================================================================
+--- busybox-1.23.2.orig/scripts/Makefile.build
++++ busybox-1.23.2/scripts/Makefile.build
+@@ -174,7 +174,7 @@ cmd_modversions = \
+ | $(GENKSYMS) -a $(ARCH) \
+ > $(@D)/.tmp_$(@F:.o=.ver); \
+ \
+- $(LD) $(LDFLAGS) -r -o $@ $(@D)/.tmp_$(@F) \
++ $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(@D)/.tmp_$(@F) \
+ -T $(@D)/.tmp_$(@F:.o=.ver); \
+ rm -f $(@D)/.tmp_$(@F) $(@D)/.tmp_$(@F:.o=.ver); \
+ else \
+@@ -257,7 +257,7 @@ quiet_cmd_link_o_target = LD $@
+ # If the list of objects to link is empty, just create an empty built-in.o
+ # -nostdlib is added to make "make LD=gcc ..." work (some people use that)
+ cmd_link_o_target = $(if $(strip $(obj-y)),\
+- $(LD) -nostdlib $(ld_flags) -r -o $@ $(filter $(obj-y), $^),\
++ $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(filter $(obj-y), $^),\
+ rm -f $@; $(AR) rcs $@)
+
+ $(builtin-target): $(obj-y) FORCE
+@@ -292,10 +292,10 @@ $($(subst $(obj)/,,$(@:.o=-objs))) \
+ $($(subst $(obj)/,,$(@:.o=-y)))), $^)
+
+ quiet_cmd_link_multi-y = LD $@
+-cmd_link_multi-y = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps)
++cmd_link_multi-y = $(CC) $(ld_flags_partial) $(LDFLAGS_RELOCATABLE) -o $@ $(link_multi_deps)
+
+ quiet_cmd_link_multi-m = LD [M] $@
+-cmd_link_multi-m = $(LD) $(ld_flags) $(LDFLAGS_MODULE) -o $@ $(link_multi_deps)
++cmd_link_multi-m = $(CC) $(ld_flags) $(LDFLAGS_MODULE) -o $@ $(link_multi_deps)
+
+ # We would rather have a list of rules like
+ # foo.o: $(foo-objs)
+Index: busybox-1.23.2/scripts/Makefile.lib
+===================================================================
+--- busybox-1.23.2.orig/scripts/Makefile.lib
++++ busybox-1.23.2/scripts/Makefile.lib
+@@ -121,7 +121,8 @@ cpp_flags = -Wp,-MD,$(depfile) $(NO
+ # yet ld_flags is fed to ld.
+ #ld_flags = $(LDFLAGS) $(EXTRA_LDFLAGS)
+ # Remove the -Wl, prefix from linker options normally passed through gcc
+-ld_flags = $(filter-out -Wl$(comma)%,$(LDFLAGS) $(EXTRA_LDFLAGS))
++ld_flags = $(filter-out -Wl$(comma)%,$(LDFLAGS) $(EXTRA_LDFLAGS) $(CFLAGS) $(EXTRA_CFLAGS))
++ld_flags_partial = $($(filter-out -shared%, $(filter-out -pie%,$(ld_flags))))
+
+
+ # Finds the multi-part object the current object will be linked into
+@@ -151,10 +152,8 @@ $(obj)/%:: $(src)/%_shipped
+ # Linking
+ # ---------------------------------------------------------------------------
+
+-# TODO: LDFLAGS usually is supposed to contain gcc's flags, not ld's.
+-# but here we feed them to ld!
+-quiet_cmd_ld = LD $@
+-cmd_ld = $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) $(LDFLAGS_$(@F)) \
++quiet_cmd_ld = CC $@
++cmd_ld = $(CC) $(ld_flags) $(LDFLAGS_$(@F)) \
+ $(filter-out FORCE,$^) -o $@
+
+ # Objcopy
diff --git a/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch b/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
new file mode 100644
index 0000000..de286fb
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/0002-Passthrough-r-to-linker.patch
@@ -0,0 +1,32 @@
+From df2cc76cdebc4773361477f3db203790f6986e3b Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Sat, 22 Aug 2015 23:42:40 -0700
+Subject: [PATCH 2/2] Passthrough -r to linker
+
+clang does not have -r switch and it does not pass it down to linker
+either, LDFLAGS_RELOCATABLE is used when CC is used for LD, so this
+should not cause side effects
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+---
+Upstream-Status: Pending
+
+ Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/Makefile b/Makefile
+index 9da02cb..10dd4a9 100644
+--- a/Makefile
++++ b/Makefile
+@@ -309,7 +309,7 @@ CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(C
+ MODFLAGS = -DMODULE
+ CFLAGS_MODULE = $(MODFLAGS)
+ AFLAGS_MODULE = $(MODFLAGS)
+-LDFLAGS_RELOCATABLE = -r -nostdlib
++LDFLAGS_RELOCATABLE = -Xlinker -r -nostdlib
+ LDFLAGS_MODULE = $(LDFLAGS_RELOCATABLE)
+ CFLAGS_KERNEL =
+ AFLAGS_KERNEL =
+--
+2.1.4
+
diff --git a/meta/recipes-core/busybox/busybox_1.23.2.bb b/meta/recipes-core/busybox/busybox_1.23.2.bb
index 5e02732..2559823 100644
--- a/meta/recipes-core/busybox/busybox_1.23.2.bb
+++ b/meta/recipes-core/busybox/busybox_1.23.2.bb
@@ -33,6 +33,8 @@ SRC_URI = "http://www.busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \
file://0001-Switch-to-POSIX-utmpx-API.patch \
file://0001-ifconfig-fix-double-free-fatal-error-in-INET_sprint.patch \
file://0001-chown-fix-help-text.patch \
+ file://0001-Use-CC-when-linking-instead-of-LD-and-use-CFLAGS-and.patch \
+ file://0002-Passthrough-r-to-linker.patch \
file://mount-via-label.cfg \
file://sha1sum.cfg \
file://sha256sum.cfg \
--
2.5.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-09-25 20:18 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-25 4:59 [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
2015-09-25 4:59 ` [PATCH 1/4] powertop: Include right headers for timval struct Khem Raj
2015-09-25 9:28 ` Burton, Ross
2015-09-25 4:59 ` [PATCH] pxz: Add recipe and use it for xz image type Khem Raj
2015-09-25 5:01 ` Khem Raj
2015-09-25 11:22 ` Gary Thomas
2015-09-25 13:25 ` Khem Raj
2015-09-25 13:32 ` Gary Thomas
2015-09-25 18:24 ` Andre McCurdy
2015-09-25 18:34 ` Khem Raj
2015-09-25 20:18 ` Andre McCurdy
2015-09-25 4:59 ` [PATCH 2/4] dhcp: Include sys/types.h for u_int* defs Khem Raj
2015-09-25 4:59 ` [PATCH 3/4] blktrace: Include <sys/types.h for dev_t Khem Raj
2015-09-25 9:29 ` Burton, Ross
2015-09-25 4:59 ` [PATCH 4/4] waffle: Fix build with musl Khem Raj
2015-09-25 5:06 ` [PATCH V2] busybox: Use CC instead of bare LD to be the Linker Khem Raj
2015-09-25 9:33 ` Burton, Ross
-- strict thread matches above, loose matches on Subject: below --
2015-09-09 5:53 Khem Raj
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.