* [PATCH V5 00/16] fix buildpaths QA issue
@ 2016-03-22 12:12 Hongxu Jia
2016-03-22 12:12 ` [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
` (16 more replies)
0 siblings, 17 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Changed in V5:
- Fix dbg package not contain sources while -fdebug-prefix-map used;
- Tweak fix_buildpaths.bbclass to adapt the above fix;
- gconf: remove duplicated option;
Test case to verify patch 01/16:
1. Modify recipe service in meta-skeleton,
execute it caused segment fault
diff --git a/meta-skeleton/recipes-skeleton/service/service/skeleton_test.c b/meta-skeleton/recipes-skeleton/service/service/skeleton_test.c
index 96c67ac..0862882 100644
--- a/meta-skeleton/recipes-skeleton/service/service/skeleton_test.c
+++ b/meta-skeleton/recipes-skeleton/service/service/skeleton_test.c
@@ -1,12 +1,10 @@
-#include <unistd.h>
-
-/* This demo does nothing except for testing /etc/init.d/skeleton */
+#include <stdio.h>
int main(int argc, char *argv[])
{
- daemon(0, 0);
- while (1) {
- sleep(1);
- }
+ printf("hello world\n");
+
+ char *p = NULL;
+ *p = 1;
return 0;
}
diff --git a/meta-skeleton/recipes-skeleton/service/service_0.1.bb b/meta-skeleton/recipes-skeleton/service/service_0.1.bb
index c5cd7a9..c4b43f0 100644
--- a/meta-skeleton/recipes-skeleton/service/service_0.1.bb
+++ b/meta-skeleton/recipes-skeleton/service/service_0.1.bb
@@ -3,29 +3,16 @@ SECTION = "base"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://${WORKDIR}/COPYRIGHT;md5=349c872e0066155e1818b786938876a4"
-SRC_URI = "file://skeleton \
- file://skeleton_test.c \
+SRC_URI = "file://skeleton_test.c \
file://COPYRIGHT \
"
do_compile () {
- ${CC} ${WORKDIR}/skeleton_test.c -o ${WORKDIR}/skeleton-test
+ ${CC} ${WORKDIR}/skeleton_test.c ${DEBUG_FLAGS} -o ${WORKDIR}/skeleton-test
}
do_install () {
- install -d ${D}${sysconfdir}/init.d
- cat ${WORKDIR}/skeleton | \
- sed -e 's,/etc,${sysconfdir},g' \
- -e 's,/usr/sbin,${sbindir},g' \
- -e 's,/var,${localstatedir},g' \
- -e 's,/usr/bin,${bindir},g' \
- -e 's,/usr,${prefix},g' > ${D}${sysconfdir}/init.d/skeleton
- chmod a+x ${D}${sysconfdir}/init.d/skeleton
-
install -d ${D}${sbindir}
install -m 0755 ${WORKDIR}/skeleton-test ${D}${sbindir}/
}
-RDEPENDS_${PN} = "initscripts"
-
-CONFFILES_${PN} += "${sysconfdir}/init.d/skeleton"
2. Install service, gdb and *-dbg to core-image-minimal
vim local.conf
...
IMAGE_INSTALL_append = " service gdb"
EXTRA_IMAGE_FEATURES_append = " dbg-pkgs"
...
vim conf/bblayers.conf
...
BBLAYERS ?= " \
/buildarea/raid0/hjia/poky/meta \
/buildarea/raid0/hjia/poky/meta-poky \
/buildarea/raid0/hjia/poky/meta-skeleton \
/buildarea/raid0/hjia/poky/meta-yocto-bsp \
"
...
bitbake core-image-minimal
3. Run qemu and invoke gdb to execute skeleton-test,
while segment fault, the source printed correctly.
runqemu qemux86-64 slirp nographic
...
root@qemux86-64:~# gdb skeleton-test
(gdb) r
Starting program: /usr/sbin/skeleton-test
hello world
Program received signal SIGSEGV, Segmentation fault.
0x000000000040056b in main (argc=1, argv=0x7fffffffed08) at /usr/src/debug/skeleton_test.c:8
8 *p = 1;
(gdb) list
3 int main(int argc, char *argv[])
4 {
5 printf("hello world\n");
6
7 char *p = NULL;
8 *p = 1;
9 return 0;
10 }
...
//Hongxu
The following changes since commit 2f2f784727a1617c2b9ce0bb86730ba301238e06:
parselogs: add new whitelist entries to address 4.4.3 issues (2016-03-22 08:54:34 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib hongxu/fix-buildpath
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=hongxu/fix-buildpath
Hongxu Jia (16):
conf/bitbake.conf package.bbclass: fix dbg package not contain sources
while -fdebug-prefix-map used
kernel.bbclass: fix buildpath QA issue
dtc.inc: fix buildpaths QA issue
fix_buildpaths.bbclass: add bbclass to fix build path
icu: fix buildpaths QA issue
tcl: fix buildpath QA issue
python2/3: fix buildpath QA issue
bbclass distutils/distutils3: fix .pyc/.pyo buildpath
bbclass distutils/distutils3/setuptools/setuptools3: clean up
DISTUTILS_INSTALL_ARGS
python-setuptools/python3-setuptools: use old-style install
python3-pip: use old-style install
waf.bbclass: support do patch on extracted files
python-pycairo: fix buildpath QA issue
openssl: fix buildpath QA issue
epiphany: fix buildpaths QA issue
gconf: fix buildpaths QA issue
meta/classes/distutils.bbclass | 8 +--
meta/classes/distutils3.bbclass | 8 +--
meta/classes/fix_buildpaths.bbclass | 55 +++++++++++++++++++
meta/classes/kernel.bbclass | 4 +-
meta/classes/package.bbclass | 31 ++++++++---
meta/classes/setuptools.bbclass | 5 --
meta/classes/setuptools3.bbclass | 5 --
meta/classes/waf.bbclass | 11 +++-
meta/conf/bitbake.conf | 3 +-
.../openssl/remove-build-path-in-source-code.patch | 44 +++++++++++++++
.../recipes-connectivity/openssl/openssl_1.0.2g.bb | 7 +++
...trip-off-build-path-in-purported-filename.patch | 62 ++++++++++++++++++++++
.../python/python-pycairo_1.10.0.bb | 25 +++------
.../python/python-setuptools_19.4.bb | 2 -
meta/recipes-devtools/python/python3-pip_8.0.0.bb | 12 -----
.../python/python3-setuptools_19.4.bb | 8 +--
meta/recipes-devtools/python/python3_3.5.1.bb | 20 +++++--
meta/recipes-devtools/python/python_2.7.11.bb | 29 ++++++++--
meta/recipes-devtools/tcltk/tcl_8.6.4.bb | 9 +++-
meta/recipes-gnome/epiphany/epiphany_3.18.4.bb | 8 ++-
.../files/use-relative-path-for-build-dir.patch | 57 ++++++++++++++++++++
meta/recipes-gnome/gnome/gconf_3.2.6.bb | 3 +-
meta/recipes-kernel/dtc/dtc.inc | 3 ++
.../dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch | 30 +++++++++++
.../icu/icu/add-variable-U_CC-and-U_CXX.patch | 61 +++++++++++++++++++++
meta/recipes-support/icu/icu_56.1.bb | 12 +++++
26 files changed, 442 insertions(+), 80 deletions(-)
create mode 100644 meta/classes/fix_buildpaths.bbclass
create mode 100644 meta/recipes-connectivity/openssl/openssl/remove-build-path-in-source-code.patch
create mode 100644 meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch
create mode 100644 meta/recipes-gnome/epiphany/files/use-relative-path-for-build-dir.patch
create mode 100644 meta/recipes-kernel/dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch
create mode 100644 meta/recipes-support/icu/icu/add-variable-U_CC-and-U_CXX.patch
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 17:01 ` Mark Hatle
2016-03-22 12:12 ` [PATCH 02/16] kernel.bbclass: fix buildpath QA issue Hongxu Jia
` (15 subsequent siblings)
16 siblings, 1 reply; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Tweak DEBUG_FLAGS to use "/usr/src/debug" as source target path
in DWARF. While use gdb to debug binary, it could work with
sources in dbg package.
While -fdebug-prefix-map is used for compiling, we do not need
invoking debugedit to edit DWARF at do_package time, but list
where sources files are.
The copydebugsources uses the list to copy sources to dbg package.
It works whether -fdebug-prefix-map used or not.
[YOCTO #9305]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/package.bbclass | 31 +++++++++++++++++++++++++------
meta/conf/bitbake.conf | 3 +--
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index ef80e50..478a1d9 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -300,6 +300,15 @@ def get_conffiles(pkg, d):
os.chdir(cwd)
return conf_list
+def checkbuildpath(file, d):
+ tmpdir = d.getVar('TMPDIR', True)
+ with open(file) as f:
+ file_content = f.read()
+ if tmpdir in file_content:
+ return True
+
+ return False
+
def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
# Function to split a single file into two components, one is the stripped
# target system binary, the other contains any debugging information. The
@@ -313,7 +322,6 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
objcopy = d.getVar("OBJCOPY", True)
debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
workdir = d.getVar("WORKDIR", True)
- workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or os.path.dirname(os.path.dirname(workdir))
# We ignore kernel modules, we don't generate debug info files.
if file.find("/lib/modules/") != -1 and file.endswith(".ko"):
@@ -327,7 +335,7 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
# We need to extract the debug src information here...
if debugsrcdir:
- cmd = "'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (debugedit, workparentdir, debugsrcdir, sourcefile, file)
+ cmd = "'%s' -i -l '%s' '%s'" % (debugedit, sourcefile, file)
(retval, output) = oe.utils.getstatusoutput(cmd)
if retval:
bb.fatal("debugedit failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
@@ -363,8 +371,14 @@ def copydebugsources(debugsrcdir, d):
objcopy = d.getVar("OBJCOPY", True)
debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
workdir = d.getVar("WORKDIR", True)
- workparentdir = os.path.dirname(os.path.dirname(workdir))
- workbasedir = os.path.basename(os.path.dirname(workdir)) + "/" + os.path.basename(workdir)
+
+ # If build path exists in sourcefile, it means toolchain did not use
+ # -fdebug-prefix-map to compile
+ if checkbuildpath(sourcefile, d):
+ localsrc_word = workdir
+ else:
+ localsrc_word = "/usr/src/debug"
+ localsrc_prefix = localsrc_word + '/'
nosuchdir = []
basepath = dvar
@@ -379,17 +393,22 @@ def copydebugsources(debugsrcdir, d):
# We need to ignore files that are not actually ours
# we do this by only paying attention to items from this package
processdebugsrc += "fgrep -zw '%s' | "
+ # Remove prefix in the source paths
+ processdebugsrc += "sed 's#%s##g' | "
+
processdebugsrc += "(cd '%s' ; cpio -pd0mlL --no-preserve-owner '%s%s' 2>/dev/null)"
- cmd = processdebugsrc % (sourcefile, workbasedir, workparentdir, dvar, debugsrcdir)
+ cmd = processdebugsrc % (sourcefile, localsrc_word, localsrc_prefix, workdir, dvar, debugsrcdir)
(retval, output) = oe.utils.getstatusoutput(cmd)
+
# Can "fail" if internal headers/transient sources are attempted
#if retval:
# bb.fatal("debug source copy failed with exit code %s (cmd was %s)" % (retval, cmd))
# cpio seems to have a bug with -lL together and symbolic links are just copied, not dereferenced.
# Work around this by manually finding and copying any symbolic links that made it through.
- cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s' 2>/dev/null)" % (dvar, debugsrcdir, dvar, debugsrcdir, workparentdir, dvar, debugsrcdir)
+ cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s' 2>/dev/null)" % (dvar, debugsrcdir, dvar, debugsrcdir, workdir, dvar, debugsrcdir)
+
(retval, output) = oe.utils.getstatusoutput(cmd)
if retval:
bb.fatal("debugsrc symlink fixup failed with exit code %s (cmd was %s)" % (retval, cmd))
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 7ed5ffb..838ee81 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -550,8 +550,7 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
# Optimization flags.
##################################################################
DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
- -fdebug-prefix-map=${B}=/usr/src/${BPN} \
- -fdebug-prefix-map=${S}=/usr/src/${BPN} \
+ -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
-fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
-fdebug-prefix-map=${STAGING_DIR_HOST}= \
"
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
2016-03-22 12:12 ` [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:21 ` Bruce Ashfield
2016-03-22 12:12 ` [PATCH 03/16] dtc.inc: fix buildpaths " Hongxu Jia
` (14 subsequent siblings)
16 siblings, 1 reply; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable DEBUG_FLAGS could
not been passed to compiler, so we explicitly add DEBUG_FLAGS to CC to
replace build path with target path.
[YOCTO #7058]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/kernel.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index c3eab50..d357ccf 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -207,7 +207,7 @@ kernel_do_compile() {
copy_initramfs
use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
fi
- oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE} CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
+ oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE} CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" = "${KERNEL_IMAGETYPE}"; then
gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" > "${KERNEL_OUTPUT}"
fi
@@ -216,7 +216,7 @@ kernel_do_compile() {
do_compile_kernelmodules() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
- oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
+ oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
# Module.symvers gets updated during the
# building of the kernel modules. We need to
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 03/16] dtc.inc: fix buildpaths QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
2016-03-22 12:12 ` [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
2016-03-22 12:12 ` [PATCH 02/16] kernel.bbclass: fix buildpath QA issue Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 04/16] fix_buildpaths.bbclass: add bbclass to fix build path Hongxu Jia
` (13 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Previous Makefile define its own CFLAGS, pass DEBUG_FLAGS to
it could fix build path issue in DWARF.
[YOCTO #8457]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-kernel/dtc/dtc.inc | 3 +++
.../dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch | 30 ++++++++++++++++++++++
2 files changed, 33 insertions(+)
create mode 100644 meta/recipes-kernel/dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch
diff --git a/meta/recipes-kernel/dtc/dtc.inc b/meta/recipes-kernel/dtc/dtc.inc
index 0c409b0..a1a30fc 100644
--- a/meta/recipes-kernel/dtc/dtc.inc
+++ b/meta/recipes-kernel/dtc/dtc.inc
@@ -7,8 +7,11 @@ DEPENDS = "flex-native bison-native"
SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git \
file://make_install.patch \
"
+SRC_URI_append_class-target = " file://pass-DEBUG_FLAGS-to-CFLAGS.patch \
+"
EXTRA_OEMAKE='PREFIX="${prefix}" LIBDIR="${libdir}" DESTDIR="${D}"'
+EXTRA_OEMAKE_class-target += 'DEBUG_FLAGS="${DEBUG_FLAGS}"'
S = "${WORKDIR}/git"
diff --git a/meta/recipes-kernel/dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch b/meta/recipes-kernel/dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch
new file mode 100644
index 0000000..3f17ee9
--- /dev/null
+++ b/meta/recipes-kernel/dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch
@@ -0,0 +1,30 @@
+From 4019e29b0c1ec441fa43d9969197c24ad73f3b1a Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Fri, 26 Feb 2016 03:28:15 -0500
+Subject: [PATCH] Makefile: pass DEBUG_FLAGS to CFLAGS
+
+It fixes buildpaths issue while compiling.
+
+Upstream-Status: Inappropriate [openembedded specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/Makefile b/Makefile
+index 18838cc..4954f32 100644
+--- a/Makefile
++++ b/Makefile
+@@ -18,7 +18,7 @@ CONFIG_LOCALVERSION =
+ CPPFLAGS = -I libfdt -I .
+ WARNINGS = -Werror -Wall -Wpointer-arith -Wcast-qual -Wnested-externs \
+ -Wstrict-prototypes -Wmissing-prototypes -Wredundant-decls -Wshadow
+-CFLAGS = -g -Os -fPIC -Werror $(WARNINGS)
++CFLAGS = $(DEBUG_FLAGS) -Os -fPIC -Werror $(WARNINGS)
+
+ BISON = bison
+ LEX = flex
+--
+1.9.1
+
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 04/16] fix_buildpaths.bbclass: add bbclass to fix build path
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (2 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 03/16] dtc.inc: fix buildpaths " Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 05/16] icu: fix buildpaths QA issue Hongxu Jia
` (12 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Define variable PACKAGE_BUILDPATH_TEXT_FILES to list files that have
build paths and remove these paths at do_package time. It supports
package override while files exist in conditionally generated package.
(such as ${PN}-ptest is conditionally generated)
Define variable PACKAGE_BUILDPATH_TEXT_PATTERNS to list build path
patterns, which used by sed, it removes --sysroot and
-fdebug-prefix-map in text files by default.
Define python function to remove build path in variable.
Define python function to remove build path in python compiled
code.
[YOCTO #9169]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/fix_buildpaths.bbclass | 55 +++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 meta/classes/fix_buildpaths.bbclass
diff --git a/meta/classes/fix_buildpaths.bbclass b/meta/classes/fix_buildpaths.bbclass
new file mode 100644
index 0000000..eff0a30
--- /dev/null
+++ b/meta/classes/fix_buildpaths.bbclass
@@ -0,0 +1,55 @@
+PACKAGE_PREPROCESS_FUNCS += 'remove_buildpath_package_preprocess'
+
+def list_package_buildpath_text_files(d):
+ files = d.getVar('PACKAGE_BUILDPATH_TEXT_FILES', True) or ''
+
+ # Support package override
+ for pn in d.getVar('PACKAGES', True).split():
+ files += d.getVar('PACKAGE_BUILDPATH_TEXT_FILES_%s' % pn, True) or ''
+
+ return files
+
+remove_buildpath_package_preprocess () {
+ buildpath_files="${@list_package_buildpath_text_files(d)}"
+
+ # Remove build paths in text files
+ for file in $buildpath_files;do
+ sed -i ${PACKAGE_BUILDPATH_TEXT_PATTERNS} ${PKGD}$file
+ done
+}
+
+# List build path patterns, which used by sed, it removes
+# --sysroot and -fdebug-prefix-map by default.
+PACKAGE_BUILDPATH_TEXT_PATTERNS = " \
+ -e 's:--sysroot=${STAGING_DIR_TARGET}::g' \
+ -e 's:-fdebug-prefix-map=${WORKDIR}=/usr/src/debug::g' \
+ -e 's:-fdebug-prefix-map=${STAGING_DIR_NATIVE}=::g' \
+ -e 's:-fdebug-prefix-map=${STAGING_DIR_HOST}=::g' \
+"
+
+# List files which have build paths, and remove these paths.
+PACKAGE_BUILDPATH_TEXT_FILES ??= ""
+
+# Remove build path in variable
+def remove_buildpath_variable(d, var):
+ val = d.getVar(var, True) or ''
+
+ toolchain_opts = d.getVar('TOOLCHAIN_OPTIONS', True)
+ val = val.replace(toolchain_opts, '')
+
+ debug_flags = d.getVar('DEBUG_FLAGS', True).split()
+ for opt in debug_flags:
+ if '-fdebug-prefix-map=' in opt:
+ val = val.replace(opt, '')
+
+ return val
+
+# Remove build path in python compiled code which
+# located in root_path
+def remove_buildpath_bytecode(root_path, byte_code):
+ import py_compile
+
+ file = root_path + byte_code[0:-1]
+ dfile = byte_code[0:-1]
+ py_compile.compile(file, dfile=dfile)
+
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 05/16] icu: fix buildpaths QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (3 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 04/16] fix_buildpaths.bbclass: add bbclass to fix build path Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 06/16] tcl: fix buildpath " Hongxu Jia
` (11 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
- Define new variable U_CC and U_CXX which do not have build path.
The output of icuinfo will not have build path in it:
...
root@qemux86-64:~# icuinfo
<icuSystemParams type="icu4c">
[snip]
<param name="build.cc">x86_64-poky-linux-gcc -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse
<param name="build.cxx">x86_64-poky-linux-g++ -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse
[snip]
</icuSystemParams>
...
- Remove build paths in pkgdata.inc, Makefile.inc and
icu-config.
[YOCTO #7058]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
.../icu/icu/add-variable-U_CC-and-U_CXX.patch | 61 ++++++++++++++++++++++
meta/recipes-support/icu/icu_56.1.bb | 12 +++++
2 files changed, 73 insertions(+)
create mode 100644 meta/recipes-support/icu/icu/add-variable-U_CC-and-U_CXX.patch
diff --git a/meta/recipes-support/icu/icu/add-variable-U_CC-and-U_CXX.patch b/meta/recipes-support/icu/icu/add-variable-U_CC-and-U_CXX.patch
new file mode 100644
index 0000000..826d934
--- /dev/null
+++ b/meta/recipes-support/icu/icu/add-variable-U_CC-and-U_CXX.patch
@@ -0,0 +1,61 @@
+From 3221b108a18edeff5424bc5c66589eb20148d9b2 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Wed, 2 Mar 2016 21:18:34 -0500
+Subject: [PATCH] add variable U_CC and U_CXX
+
+Define new variable U_CC and U_CXX which do not have build path.
+The output of icuinfo will not have build path in it:
+...
+root@qemux86-64:~# icuinfo
+ <icuSystemParams type="icu4c">
+ [snip]
+ <param name="build.cc">x86_64-poky-linux-gcc -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse
+ <param name="build.cxx">x86_64-poky-linux-g++ -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse
+ [snip]
+ </icuSystemParams>
+...
+
+Upstream-Status: Inappropriate [openembedded specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ configure.ac | 9 +++++++++
+ tools/toolutil/Makefile.in | 2 +-
+ 2 files changed, 10 insertions(+), 1 deletion(-)
+
+diff --git a/configure.ac b/configure.ac
+index 595bd12..ec05c19 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -673,6 +673,15 @@ AC_SUBST(GENCCODE_ASSEMBLY)
+
+ AC_MSG_RESULT($GENCCODE_ASSEMBLY)
+
++# U_CC and U_CXX are used for icuinfo which remove
++# --sysroot for cross-compilation
++U_CC=${U_CC}
++AC_SUBST(U_CC)
++AC_MSG_RESULT(U_CC)
++U_CXX=${U_CXX}
++AC_SUBST(U_CXX)
++AC_MSG_RESULT(U_CXX)
++
+ # Checks for header files
+ AC_CHECK_HEADERS(inttypes.h)
+ if test $ac_cv_header_inttypes_h = no; then
+diff --git a/tools/toolutil/Makefile.in b/tools/toolutil/Makefile.in
+index 1fb42ed..a083522 100644
+--- a/tools/toolutil/Makefile.in
++++ b/tools/toolutil/Makefile.in
+@@ -46,7 +46,7 @@ CXXFLAGS += $(LIBCXXFLAGS)
+ CPPFLAGS += -I$(top_srcdir)/common -I$(top_srcdir)/i18n $(LIBCPPFLAGS)
+
+ # from icuinfo
+-CPPFLAGS+= "-DU_BUILD=\"@build@\"" "-DU_HOST=\"@host@\"" "-DU_CC=\"@CC@\"" "-DU_CXX=\"@CXX@\""
++CPPFLAGS+= "-DU_BUILD=\"@build@\"" "-DU_HOST=\"@host@\"" "-DU_CC=\"@U_CC@\"" "-DU_CXX=\"@U_CXX@\""
+ CPPFLAGS += -DUNISTR_FROM_CHAR_EXPLICIT=explicit -DUNISTR_FROM_STRING_EXPLICIT=explicit
+
+ DEFS += -DU_TOOLUTIL_IMPLEMENTATION
+--
+1.9.1
+
diff --git a/meta/recipes-support/icu/icu_56.1.bb b/meta/recipes-support/icu/icu_56.1.bb
index db0de10..1fd4dfc 100644
--- a/meta/recipes-support/icu/icu_56.1.bb
+++ b/meta/recipes-support/icu/icu_56.1.bb
@@ -17,6 +17,7 @@ SRC_URI = "${BASE_SRC_URI} \
file://icu-pkgdata-large-cmd.patch \
file://fix-install-manx.patch \
file://icu-release-56-1-flagparser-fix.patch \
+ file://add-variable-U_CC-and-U_CXX.patch \
"
SRC_URI_append_class-target = "\
@@ -27,3 +28,14 @@ SRC_URI[sha256sum] = "3a64e9105c734dcf631c0b3ed60404531bce6c0f5a64bfe1a6402a4cc2
UPSTREAM_CHECK_REGEX = "(?P<pver>\d+(\.\d+)+)/"
UPSTREAM_CHECK_URI = "http://download.icu-project.org/files/icu4c/"
+
+inherit fix_buildpaths
+CACHED_CONFIGUREVARS += "U_CC="${@remove_buildpath_variable(d, 'CC')}" \
+ U_CXX="${@remove_buildpath_variable(d, 'CXX')}" \
+"
+PACKAGE_BUILDPATH_TEXT_FILES_${PN}-dev ?= " \
+ ${libdir}/${BPN}/${PV}/pkgdata.inc \
+ ${libdir}/${BPN}/${PV}/Makefile.inc \
+ ${bindir}/icu-config \
+"
+
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 06/16] tcl: fix buildpath QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (4 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 05/16] icu: fix buildpaths QA issue Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 07/16] python2/3: " Hongxu Jia
` (10 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
- Tweak DEBUG_FLAGS, since ${S} is not normal format
- Remove build path in tclConfig.sh
- Tweak sed expression at tclConfig.sh to strict the pattern
[YOCTO #8457]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-devtools/tcltk/tcl_8.6.4.bb | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-devtools/tcltk/tcl_8.6.4.bb b/meta/recipes-devtools/tcltk/tcl_8.6.4.bb
index 8e92b3e..2aeb952 100644
--- a/meta/recipes-devtools/tcltk/tcl_8.6.4.bb
+++ b/meta/recipes-devtools/tcltk/tcl_8.6.4.bb
@@ -54,8 +54,8 @@ do_install() {
autotools_do_install install-private-headers
ln -sf ./tclsh${VER} ${D}${bindir}/tclsh
ln -sf tclsh8.6 ${D}${bindir}/tclsh${VER}
- sed -i "s+-L${B}+-L${STAGING_LIBDIR}+g" tclConfig.sh
- sed -i "s+${WORKDIR}+${STAGING_INCDIR}+g" tclConfig.sh
+ sed -i "s,-L${B},-L${STAGING_LIBDIR},g" tclConfig.sh
+ sed -i "s,='${WORKDIR},='${STAGING_INCDIR},g" tclConfig.sh
sed -i "s,-L${libdir},-L=${libdir},g" tclConfig.sh
sed -i "s,-I${includedir},-I=${includedir},g" tclConfig.sh
install -d ${D}${bindir_crossscripts}
@@ -73,6 +73,11 @@ tcl_sysroot_preprocess () {
sysroot_stage_dir ${D}${bindir_crossscripts} ${SYSROOT_DESTDIR}${bindir_crossscripts}
}
+inherit fix_buildpaths
+PACKAGE_BUILDPATH_TEXT_FILES ?= "${bindir_crossscripts}/tclConfig.sh \
+ ${libdir}/tclConfig.sh \
+"
+
PACKAGES =+ "tcl-lib"
FILES_tcl-lib = "${libdir}/libtcl8.6.so.*"
FILES_${PN} += "${libdir}/tcl${VER} ${libdir}/tcl8.6 ${libdir}/tcl8"
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 07/16] python2/3: fix buildpath QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (5 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 06/16] tcl: fix buildpath " Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 08/16] bbclass distutils/distutils3: fix .pyc/.pyo buildpath Hongxu Jia
` (9 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
- Remove build paths in target file
- Since _sysconfigdata.py was modified at do_package,
so update _sysconfigdata.pyc manually (Python2 only).
[YOCTO #8446]
[YOCTO #9169]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-devtools/python/python3_3.5.1.bb | 20 ++++++++++++++----
meta/recipes-devtools/python/python_2.7.11.bb | 29 ++++++++++++++++++++++-----
2 files changed, 40 insertions(+), 9 deletions(-)
diff --git a/meta/recipes-devtools/python/python3_3.5.1.bb b/meta/recipes-devtools/python/python3_3.5.1.bb
index 2fbb3cb..bfcea85 100644
--- a/meta/recipes-devtools/python/python3_3.5.1.bb
+++ b/meta/recipes-devtools/python/python3_3.5.1.bb
@@ -186,12 +186,24 @@ PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
py_package_preprocess () {
# copy back the old Makefile to fix target package
install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
- # Remove references to buildmachine paths in target Makefile and _sysconfigdata
- sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \
- ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile \
- ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py
}
+# Remove references to buildmachine paths in target
+inherit fix_buildpaths
+PACKAGE_BUILDPATH_TEXT_FILES ?= " \
+ ${libdir}/python${PYTHON_MAJMIN}/config/Makefile \
+ ${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}/Makefile \
+ ${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py \
+ ${bindir}/python${PYTHON_BINABI}-config \
+"
+PACKAGE_BUILDPATH_TEXT_PATTERNS += " \
+ -e 's:--with-libtool-sysroot=${STAGING_DIR_TARGET}::g' \
+ -e 's:${STAGING_DIR}/${MACHINE}${datadir}/pkgconfig:${datadir}/pkgconfig:g' \
+ -e 's:${STAGING_DIR_HOST}::g' \
+ -e 's:${STAGING_DIR_NATIVE}::g' \
+ -e 's:${WORKDIR}::g' \
+"
+
require python-${PYTHON_MAJMIN}-manifest.inc
# manual dependency additions
diff --git a/meta/recipes-devtools/python/python_2.7.11.bb b/meta/recipes-devtools/python/python_2.7.11.bb
index 0c00d6e..3cf8642 100644
--- a/meta/recipes-devtools/python/python_2.7.11.bb
+++ b/meta/recipes-devtools/python/python_2.7.11.bb
@@ -128,15 +128,34 @@ do_install_append_class-nativesdk () {
SSTATE_SCAN_FILES += "Makefile"
PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
-
py_package_preprocess () {
# copy back the old Makefile to fix target package
install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
+}
+
+# Remove references to buildmachine paths in target
+inherit fix_buildpaths
+PACKAGE_BUILDPATH_TEXT_FILES ?= " \
+ ${libdir}/python${PYTHON_MAJMIN}/config/Makefile \
+ ${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py \
+"
+PACKAGE_BUILDPATH_TEXT_FILES_${PN}-ptest ?= " \
+ ${PTEST_PATH}/Makefile \
+"
+PACKAGE_BUILDPATH_TEXT_PATTERNS += " \
+ -e 's:--with-libtool-sysroot=${STAGING_DIR_TARGET}::g' \
+ -e 's:${STAGING_DIR}/${MACHINE}${datadir}/pkgconfig:${datadir}/pkgconfig:g' \
+ -e 's:${STAGING_DIR_HOST}::g' \
+ -e 's:${STAGING_DIR_NATIVE}::g' \
+ -e 's:${WORKDIR}::g' \
+"
- # Remove references to buildmachine paths in target Makefile and _sysconfigdata
- sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \
- ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile \
- ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py
+PACKAGE_PREPROCESS_FUNCS += "update_pyc_package_preprocess"
+python update_pyc_package_preprocess () {
+ # Update _sysconfigdata.pyc
+ root_path = d.expand("${PKGD}")
+ byte_code = d.expand("${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.pyc")
+ remove_buildpath_bytecode(root_path, byte_code)
}
require python-${PYTHON_MAJMIN}-manifest.inc
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 08/16] bbclass distutils/distutils3: fix .pyc/.pyo buildpath
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (6 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 07/16] python2/3: " Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 09/16] bbclass distutils/distutils3/setuptools/setuptools3: clean up DISTUTILS_INSTALL_ARGS Hongxu Jia
` (8 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Get the "--root" directory supplied to the "install" command,
and use it as a prefix to strip off the purported filename
encoded in bytecode files.
Since --root added, we need to tweak --install-data,
--prefix and --install-lib to use relative path.
[YOCTO #8446]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/distutils.bbclass | 6 +++---
meta/classes/distutils3.bbclass | 6 +++---
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index cd06713..f054006 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -4,8 +4,8 @@ DISTUTILS_BUILD_ARGS ?= ""
DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}"
DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \
--install-data=${STAGING_DATADIR}"
-DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \
- --install-data=${D}/${datadir}"
+DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \
+ --install-data=${datadir}"
distutils_do_compile() {
STAGING_INCDIR=${STAGING_INCDIR} \
@@ -38,7 +38,7 @@ distutils_do_install() {
STAGING_LIBDIR=${STAGING_LIBDIR} \
PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
- ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
+ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
bbfatal "${PYTHON_PN} setup.py install execution failed."
# support filenames with *spaces*
diff --git a/meta/classes/distutils3.bbclass b/meta/classes/distutils3.bbclass
index 443bf3a..7962d40 100644
--- a/meta/classes/distutils3.bbclass
+++ b/meta/classes/distutils3.bbclass
@@ -5,8 +5,8 @@ DISTUTILS_BUILD_EXT_ARGS ?= ""
DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}"
DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \
--install-data=${STAGING_DATADIR}"
-DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \
- --install-data=${D}/${datadir}"
+DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \
+ --install-data=${datadir}"
distutils3_do_compile() {
if [ ${BUILD_SYS} != ${HOST_SYS} ]; then
@@ -63,7 +63,7 @@ distutils3_do_install() {
STAGING_LIBDIR=${STAGING_LIBDIR} \
PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
BUILD_SYS=${BUILD_SYS} HOST_SYS=${SYS} \
- ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
+ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
bbfatal "${PYTHON_PN} setup.py install execution failed."
# support filenames with *spaces*
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 09/16] bbclass distutils/distutils3/setuptools/setuptools3: clean up DISTUTILS_INSTALL_ARGS
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (7 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 08/16] bbclass distutils/distutils3: fix .pyc/.pyo buildpath Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 10/16] python-setuptools/python3-setuptools: use old-style install Hongxu Jia
` (7 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Since fix .pyc/.pyo buildpaths issue, we have the same
DISTUTILS_INSTALL_ARGS in these bbclass files, so clean
up the duplicate.
[YOCTO #8446]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/distutils.bbclass | 4 +++-
meta/classes/distutils3.bbclass | 4 +++-
meta/classes/setuptools.bbclass | 5 -----
meta/classes/setuptools3.bbclass | 5 -----
4 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index f054006..18cfb17 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -5,6 +5,8 @@ DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}"
DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \
--install-data=${STAGING_DATADIR}"
DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \
+ --root=${D} \
+ --install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
distutils_do_compile() {
@@ -38,7 +40,7 @@ distutils_do_install() {
STAGING_LIBDIR=${STAGING_LIBDIR} \
PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
- ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
+ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install ${DISTUTILS_INSTALL_ARGS} || \
bbfatal "${PYTHON_PN} setup.py install execution failed."
# support filenames with *spaces*
diff --git a/meta/classes/distutils3.bbclass b/meta/classes/distutils3.bbclass
index 7962d40..1e2924a 100644
--- a/meta/classes/distutils3.bbclass
+++ b/meta/classes/distutils3.bbclass
@@ -6,6 +6,8 @@ DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}"
DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \
--install-data=${STAGING_DATADIR}"
DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \
+ --root=${D} \
+ --install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
distutils3_do_compile() {
@@ -63,7 +65,7 @@ distutils3_do_install() {
STAGING_LIBDIR=${STAGING_LIBDIR} \
PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
BUILD_SYS=${BUILD_SYS} HOST_SYS=${SYS} \
- ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
+ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install ${DISTUTILS_INSTALL_ARGS} || \
bbfatal "${PYTHON_PN} setup.py install execution failed."
# support filenames with *spaces*
diff --git a/meta/classes/setuptools.bbclass b/meta/classes/setuptools.bbclass
index 56343b1..0a3f0f9 100644
--- a/meta/classes/setuptools.bbclass
+++ b/meta/classes/setuptools.bbclass
@@ -1,8 +1,3 @@
inherit distutils
DEPENDS += "python-distribute-native"
-
-DISTUTILS_INSTALL_ARGS = "--root=${D} \
- --prefix=${prefix} \
- --install-lib=${PYTHON_SITEPACKAGES_DIR} \
- --install-data=${datadir}"
diff --git a/meta/classes/setuptools3.bbclass b/meta/classes/setuptools3.bbclass
index de6dd94..38e175a 100644
--- a/meta/classes/setuptools3.bbclass
+++ b/meta/classes/setuptools3.bbclass
@@ -1,8 +1,3 @@
inherit distutils3
DEPENDS += "python3-setuptools-native"
-
-DISTUTILS_INSTALL_ARGS = "--root=${D} \
- --prefix=${prefix} \
- --install-lib=${PYTHON_SITEPACKAGES_DIR} \
- --install-data=${datadir}"
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 10/16] python-setuptools/python3-setuptools: use old-style install
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (8 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 09/16] bbclass distutils/distutils3/setuptools/setuptools3: clean up DISTUTILS_INSTALL_ARGS Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 11/16] python3-pip: " Hongxu Jia
` (6 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Since '--root' is added to 'setup.py install' to fix .pyc/.pyo
buildpaths issue. For python setuptools, the '--root' option
means explicit request for old-style install. There will be
no egg generated. And it do not need setuptools.pth any more.
Rename easy_install to easy_install3.
Ubuntu or Debian did the same thing like us:
https://packages.debian.org/sid/all/python3-setuptools/filelist
[YOCTO #8446]
[YOCTO #9162]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-devtools/python/python-setuptools_19.4.bb | 2 --
meta/recipes-devtools/python/python3-setuptools_19.4.bb | 8 ++------
2 files changed, 2 insertions(+), 8 deletions(-)
diff --git a/meta/recipes-devtools/python/python-setuptools_19.4.bb b/meta/recipes-devtools/python/python-setuptools_19.4.bb
index c172039..01763d4 100644
--- a/meta/recipes-devtools/python/python-setuptools_19.4.bb
+++ b/meta/recipes-devtools/python/python-setuptools_19.4.bb
@@ -7,8 +7,6 @@ DEPENDS_class-native += "python-native"
inherit distutils
-DISTUTILS_INSTALL_ARGS += "--install-lib=${D}${libdir}/${PYTHON_DIR}/site-packages"
-
RDEPENDS_${PN} = "\
python-stringold \
python-email \
diff --git a/meta/recipes-devtools/python/python3-setuptools_19.4.bb b/meta/recipes-devtools/python/python3-setuptools_19.4.bb
index fb2931c..12b0d93 100644
--- a/meta/recipes-devtools/python/python3-setuptools_19.4.bb
+++ b/meta/recipes-devtools/python/python3-setuptools_19.4.bb
@@ -5,13 +5,9 @@ DEPENDS_class-native += "python3-native"
inherit distutils3
-DISTUTILS_INSTALL_ARGS += "--install-lib=${D}${libdir}/${PYTHON_DIR}/site-packages"
-
-# The installer puts the wrong path in the setuptools.pth file. Correct it.
+# Rename easy_install to avoid confliction with python2's setuptools
do_install_append() {
- rm ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
- mv ${D}${bindir}/easy_install ${D}${bindir}/easy3_install
- echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
+ mv ${D}${bindir}/easy_install ${D}${bindir}/easy_install3
}
RDEPENDS_${PN} = "\
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 11/16] python3-pip: use old-style install
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (9 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 10/16] python-setuptools/python3-setuptools: use old-style install Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 12/16] waf.bbclass: support do patch on extracted files Hongxu Jia
` (5 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Since '--root' is added to 'setup.py install' to fix .pyc/.pyo
buildpaths issue. For python3-pip, the '--root' option
means there will be no eggs generated.
Ubuntu or Debian did the same thing like us:
https://packages.debian.org/sid/all/python3-pip/filelist
[YOCTO #8446]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-devtools/python/python3-pip_8.0.0.bb | 12 ------------
1 file changed, 12 deletions(-)
diff --git a/meta/recipes-devtools/python/python3-pip_8.0.0.bb b/meta/recipes-devtools/python/python3-pip_8.0.0.bb
index 834ecda..b45aeea 100644
--- a/meta/recipes-devtools/python/python3-pip_8.0.0.bb
+++ b/meta/recipes-devtools/python/python3-pip_8.0.0.bb
@@ -19,22 +19,10 @@ S = "${WORKDIR}/${SRCNAME}-${PV}"
inherit distutils3
-DISTUTILS_INSTALL_ARGS += "--install-lib=${D}${libdir}/${PYTHON_DIR}/site-packages"
-
-do_install_prepend() {
- install -d ${D}/${libdir}/${PYTHON_DIR}/site-packages
-}
-
# Use setuptools site.py instead, avoid shared state issue
do_install_append() {
- rm ${D}/${libdir}/${PYTHON_DIR}/site-packages/site.py
- rm ${D}/${libdir}/${PYTHON_DIR}/site-packages/__pycache__/site.cpython-*.pyc
-
# Install as pip3 and leave pip2 as default
rm ${D}/${bindir}/pip
-
- # Installed eggs need to be passed directly to the interpreter via a pth file
- echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/${SRCNAME}-${PV}.pth
}
RDEPENDS_${PN} = "\
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 12/16] waf.bbclass: support do patch on extracted files
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (10 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 11/16] python3-pip: " Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 13/16] python-pycairo: fix buildpath QA issue Hongxu Jia
` (4 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
Along with do_unpack, run './waf --help' to do the self extract,
so we could do patch on the extracted files later.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/waf.bbclass | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/meta/classes/waf.bbclass b/meta/classes/waf.bbclass
index 5e55833..a36bdd4 100644
--- a/meta/classes/waf.bbclass
+++ b/meta/classes/waf.bbclass
@@ -23,8 +23,17 @@ def get_waf_parallel_make(d):
return ""
+do_unpack[postfuncs] += "unpack_waf"
+do_unpack[vardeps] += "unpack_waf"
+unpack_waf() {
+ # Just self extract tarball embedded in waf, do not care help message.
+ # Do not generate bytecode, becuase we may need to patch python file
+ # later
+ PYTHONDONTWRITEBYTECODE=True ./waf --help >/dev/null
+}
+
waf_do_configure() {
- ${S}/waf configure --prefix=${prefix} ${EXTRA_OECONF}
+ ${CACHED_CONFIGUREVARS} ${S}/waf configure --prefix=${prefix} ${EXTRA_OECONF}
}
waf_do_compile() {
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 13/16] python-pycairo: fix buildpath QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (11 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 12/16] waf.bbclass: support do patch on extracted files Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 14/16] openssl: " Hongxu Jia
` (3 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
- Inherit waf.bbclass, remove duplicate tasks
- Get "--destdir" directory supplied to the "install" command,
and use it as a prefix to strip off the purported filename
encoded in bytecode files.
- Do not need to generate pycairo.pc manually
[YOCTO #8446]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
...trip-off-build-path-in-purported-filename.patch | 62 ++++++++++++++++++++++
.../python/python-pycairo_1.10.0.bb | 25 +++------
2 files changed, 68 insertions(+), 19 deletions(-)
create mode 100644 meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch
diff --git a/meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch b/meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch
new file mode 100644
index 0000000..647c4be
--- /dev/null
+++ b/meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch
@@ -0,0 +1,62 @@
+From 86faf604fc8a7f17320db48734cda464119da387 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Thu, 25 Feb 2016 04:22:07 -0500
+Subject: [PATCH] strip off build path in purported filename
+
+Get the "--destdir" directory supplied to the "install" command,
+and use it as a prefix to strip off the purported filename
+encoded in bytecode files.
+
+Upstream-Status: Inappropriate [openembedded specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+diff --git a/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py b/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py
+--- a/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py
++++ b/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py
+@@ -26,7 +26,7 @@ int main()
+ INST='''
+ import sys, py_compile
+ for pyfile in sys.argv[1:]:
+- py_compile.compile(pyfile, pyfile + %r)
++ py_compile.compile(pyfile, pyfile + %r, %r)
+ '''
+ def process_py(self,node):
+ try:
+@@ -39,6 +39,16 @@ def process_py(self,node):
+ def inst_py(ctx):
+ install_pyfile(self,node)
+ self.bld.add_post_fun(inst_py)
++
++def get_dfile(dfile):
++ if getattr(Options.options,'destdir'):
++ destdir = Options.options.destdir
++ if dfile[:len(destdir)] != destdir:
++ raise Errors.WafError("invalid destdir: filename %r doesn't start with %r" % ((dfile, destdir)))
++ dfile = dfile[len(destdir):]
++
++ return dfile
++
+ def install_pyfile(self,node):
+ tsk=self.bld.install_files(self.install_path,[node],postpone=False)
+ path=os.path.join(tsk.get_install_path(),node.name)
+@@ -53,12 +63,14 @@ def install_pyfile(self,node):
+ if self.env['PYC']or self.env['PYO']:
+ info("+ byte compiling %r"%path)
+ if self.env['PYC']:
+- argv=self.env['PYTHON']+['-c',INST%'c',path]
++ dfile = get_dfile(path+'c')
++ argv=self.env['PYTHON']+['-c',INST%('c',dfile),path]
+ ret=Utils.subprocess.Popen(argv).wait()
+ if ret:
+ raise Errors.WafError('pyc compilation failed %r'%path)
+ if self.env['PYO']:
+- argv=self.env['PYTHON']+[self.env['PYFLAGS_OPT'],'-c',INST%'o',path]
++ dfile = get_dfile(path+'o')
++ argv=self.env['PYTHON']+[self.env['PYFLAGS_OPT'],'-c',INST%('o',dfile),path]
+ ret=Utils.subprocess.Popen(argv).wait()
+ if ret:
+ raise Errors.WafError('pyo compilation failed %r'%path)
+--
+1.9.1
+
diff --git a/meta/recipes-devtools/python/python-pycairo_1.10.0.bb b/meta/recipes-devtools/python/python-pycairo_1.10.0.bb
index 70fe25c..2fbd033 100644
--- a/meta/recipes-devtools/python/python-pycairo_1.10.0.bb
+++ b/meta/recipes-devtools/python/python-pycairo_1.10.0.bb
@@ -11,31 +11,18 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=f2e071ab72978431b294a0d696327421 \
DEPENDS = "cairo"
PR = "r2"
-SRC_URI = "http://cairographics.org/releases/py2cairo-${PV}.tar.bz2"
+SRC_URI = "http://cairographics.org/releases/py2cairo-${PV}.tar.bz2 \
+ file://strip-off-build-path-in-purported-filename.patch \
+"
SRC_URI[md5sum] = "20337132c4ab06c1146ad384d55372c5"
SRC_URI[sha256sum] = "d30439f06c2ec1a39e27464c6c828b6eface3b22ee17b2de05dc409e429a7431"
S = "${WORKDIR}/py2cairo-${PV}"
-inherit distutils pkgconfig
+inherit distutils pkgconfig waf
BBCLASSEXTEND = "native"
-do_configure() {
- BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${D}${prefix} --libdir=${D}${libdir}
-}
-
-do_compile() {
- ./waf build ${PARALLEL_MAKE}
-}
-
-do_install() {
- ./waf install
- sed \
- -e 's:@prefix@:${prefix}:' \
- -e 's:@VERSION@:${PV}:' \
- -e 's:@includedir@:${includedir}:' \
- pycairo.pc.in > pycairo.pc
- install -m 0644 pycairo.pc ${D}${libdir}/pkgconfig/
-}
+CACHED_CONFIGUREVARS = "BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS}"
+EXTRA_OECONF += "--libdir=${libdir}"
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 14/16] openssl: fix buildpath QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (12 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 13/16] python-pycairo: fix buildpath QA issue Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 15/16] epiphany: fix buildpaths " Hongxu Jia
` (2 subsequent siblings)
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
- The openssl records $CC and $CFLAGS in source code which
caused buildpaths QA issue:
In crypto/cversion.c
...
if (t == SSLEAY_CFLAGS) {
return (CFLAGS);
return ("compiler: information not available");
}
...
We should remove build path in the above source code.
- Remove build paths in ${PTEST_PATH}/Makefile and
${PTEST_PATH}/Configure
[YOCTO #7058]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
.../openssl/remove-build-path-in-source-code.patch | 44 ++++++++++++++++++++++
.../recipes-connectivity/openssl/openssl_1.0.2g.bb | 7 ++++
2 files changed, 51 insertions(+)
create mode 100644 meta/recipes-connectivity/openssl/openssl/remove-build-path-in-source-code.patch
diff --git a/meta/recipes-connectivity/openssl/openssl/remove-build-path-in-source-code.patch b/meta/recipes-connectivity/openssl/openssl/remove-build-path-in-source-code.patch
new file mode 100644
index 0000000..fd5531b
--- /dev/null
+++ b/meta/recipes-connectivity/openssl/openssl/remove-build-path-in-source-code.patch
@@ -0,0 +1,44 @@
+From b937d7581a1793cdebbf5829bdaea787ed4128db Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Mon, 7 Mar 2016 01:11:42 -0500
+Subject: [PATCH] crypto/Makefile: remove build path in source code
+
+Openssl records $CC and $CFLAGS in source code which
+caused buildpaths QA issue:
+In crypto/cversion.c
+...
+ if (t == SSLEAY_CFLAGS) {
+ return (CFLAGS);
+ return ("compiler: information not available");
+ }
+...
+
+We should remove build path in the above source code.
+
+Upstream-Status: Inappropriate [embedded specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ crypto/Makefile | 5 ++++-
+ 1 file changed, 4 insertions(+), 1 deletion(-)
+
+diff --git a/crypto/Makefile b/crypto/Makefile
+index 17a87f8..ee630e8 100644
+--- a/crypto/Makefile
++++ b/crypto/Makefile
+@@ -54,8 +54,11 @@ top:
+
+ all: shared
+
++NO_BUILDPATH_CCFLAGS=`echo "$(CC) $(CFLAGS)" | \
++ sed -e 's:--sysroot=[^ ]*::g' \
++ -e 's:-fdebug-prefix-map=[^ ]*::g'`
+ buildinf.h: ../Makefile
+- $(PERL) $(TOP)/util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" >buildinf.h
++ $(PERL) $(TOP)/util/mkbuildinf.pl "$(NO_BUILDPATH_CCFLAGS)" "$(PLATFORM)" >buildinf.h
+
+ x86cpuid.s: x86cpuid.pl perlasm/x86asm.pl
+ $(PERL) x86cpuid.pl $(PERLASM_SCHEME) $(CFLAGS) $(PROCESSOR) > $@
+--
+1.9.1
+
diff --git a/meta/recipes-connectivity/openssl/openssl_1.0.2g.bb b/meta/recipes-connectivity/openssl/openssl_1.0.2g.bb
index 290f129..078f60e 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.0.2g.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.0.2g.bb
@@ -39,6 +39,7 @@ SRC_URI += "file://find.pl;subdir=${BP}/util/ \
file://ptest_makefile_deps.patch \
file://configure-musl-target.patch \
file://parallel.patch \
+ file://remove-build-path-in-source-code.patch \
"
SRC_URI[md5sum] = "f3c710c045cdee5fd114feb69feba7aa"
@@ -47,6 +48,12 @@ SRC_URI[sha256sum] = "b784b1b3907ce39abf4098702dade6365522a253ad1552e267a9a0e895
PACKAGES =+ "${PN}-engines"
FILES_${PN}-engines = "${libdir}/ssl/engines/*.so ${libdir}/engines"
+inherit fix_buildpaths
+PACKAGE_BUILDPATH_TEXT_FILES_${PN}-ptest ?= " \
+ ${PTEST_PATH}/Makefile \
+ ${PTEST_PATH}/Configure \
+"
+
# The crypto_use_bigint patch means that perl's bignum module needs to be
# installed, but some distributions (for example Fedora 23) don't ship it by
# default. As the resulting error is very misleading check for bignum before
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 15/16] epiphany: fix buildpaths QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (13 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 14/16] openssl: " Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 16/16] gconf: " Hongxu Jia
2016-03-23 6:15 ` [PATCH V6 00/15] " Hongxu Jia
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
In order to fix build path issue, we reverts the following commit
in https://github.com/GNOME/epiphany.git
...
commit 54a3122d8626f78338033a704d5849738f6dd2e6
Author: Xan Lopez <xan@igalia.com>
Date: Wed May 16 18:21:21 2012 +0900
ephy-profile-utils: use absolute path for build dir
That way we can run the tests from any directory in debug mode.
...
It did not consider cross-compile situation which we do not need
to run tests in host build.
BTW, the tests suits were compiled but not installed to ${D},
so we explicitly disable it.
[YOCTO #7058]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-gnome/epiphany/epiphany_3.18.4.bb | 8 ++-
.../files/use-relative-path-for-build-dir.patch | 57 ++++++++++++++++++++++
2 files changed, 63 insertions(+), 2 deletions(-)
create mode 100644 meta/recipes-gnome/epiphany/files/use-relative-path-for-build-dir.patch
diff --git a/meta/recipes-gnome/epiphany/epiphany_3.18.4.bb b/meta/recipes-gnome/epiphany/epiphany_3.18.4.bb
index 12d3b07..0b8335b 100644
--- a/meta/recipes-gnome/epiphany/epiphany_3.18.4.bb
+++ b/meta/recipes-gnome/epiphany/epiphany_3.18.4.bb
@@ -9,11 +9,15 @@ inherit gnomebase gsettings distro_features_check upstream-version-is-even
# libwnck3 is x11 only
REQUIRED_DISTRO_FEATURES = "x11"
-SRC_URI += "file://0001-yelp.m4-drop-the-check-for-itstool.patch"
+SRC_URI += "file://0001-yelp.m4-drop-the-check-for-itstool.patch \
+ file://use-relative-path-for-build-dir.patch \
+"
SRC_URI[archive.md5sum] = "172b78256100e8d3c629764abd0e1495"
SRC_URI[archive.sha256sum] = "be699d484371111abae754e669187215df73e21533f461e513b79537d7a1c1c1"
-EXTRA_OECONF += " --disable-nss --with-distributor-name=${DISTRO}"
+EXTRA_OECONF += " --disable-nss --with-distributor-name=${DISTRO} \
+ --disable-tests \
+"
do_configure_prepend() {
touch ${S}/gnome-doc-utils.make
diff --git a/meta/recipes-gnome/epiphany/files/use-relative-path-for-build-dir.patch b/meta/recipes-gnome/epiphany/files/use-relative-path-for-build-dir.patch
new file mode 100644
index 0000000..21c0912
--- /dev/null
+++ b/meta/recipes-gnome/epiphany/files/use-relative-path-for-build-dir.patch
@@ -0,0 +1,57 @@
+From 73e46aeea35c7bda7833e635f1bb2dbd39e2b43e Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Wed, 2 Mar 2016 02:00:23 -0500
+Subject: [PATCH] ephy-profile-utils: use relative path for build dir
+
+It reverts the following commit in https://github.com/GNOME/epiphany.git
+...
+commit 54a3122d8626f78338033a704d5849738f6dd2e6
+Author: Xan Lopez <xan@igalia.com>
+Date: Wed May 16 18:21:21 2012 +0900
+
+ ephy-profile-utils: use absolute path for build dir
+
+ That way we can run the tests from any directory in debug mode.
+...
+It did not consider cross-compile situation which we do not need
+to run tests in host build.
+
+It also caused build path issue.
+
+Upstream-Status: Inappropriate [openembedded specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ lib/Makefile.am | 2 +-
+ lib/ephy-profile-utils.c | 2 +-
+ 2 files changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/lib/Makefile.am b/lib/Makefile.am
+index 24d684b..849a07f 100644
+--- a/lib/Makefile.am
++++ b/lib/Makefile.am
+@@ -88,7 +88,7 @@ libephymisc_la_CPPFLAGS = \
+ -DDATADIR="\"$(datadir)\"" \
+ -DSHARE_DIR=\"$(pkgdatadir)\" \
+ -DTOP_SRC_DATADIR=\"$(top_srcdir)/data\" \
+- -DABS_TOP_BUILD_DIR=\"$(abs_top_builddir)\" \
++ -DTOP_BUILD_DIR=\"$(top_builddir)\" \
+ -DLOCALEDIR=\"$(localedir)\" \
+ $(AM_CPPFLAGS)
+
+diff --git a/lib/ephy-profile-utils.c b/lib/ephy-profile-utils.c
+index 0430582..a42ec4e 100644
+--- a/lib/ephy-profile-utils.c
++++ b/lib/ephy-profile-utils.c
+@@ -128,7 +128,7 @@ ephy_profile_utils_do_migration (const char *profile_directory, int test_to_run,
+ argv[i++] = NULL;
+
+ if (debug)
+- argv[0] = ABS_TOP_BUILD_DIR"/lib/"EPHY_PROFILE_MIGRATOR;
++ argv[0] = TOP_BUILD_DIR"/lib/"EPHY_PROFILE_MIGRATOR;
+
+ ret = g_spawn_sync (NULL, argv, envp, G_SPAWN_SEARCH_PATH,
+ NULL, NULL, NULL, NULL,
+--
+1.9.1
+
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 16/16] gconf: fix buildpaths QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (14 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 15/16] epiphany: fix buildpaths " Hongxu Jia
@ 2016-03-22 12:12 ` Hongxu Jia
2016-03-23 6:15 ` [PATCH V6 00/15] " Hongxu Jia
16 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-22 12:12 UTC (permalink / raw)
To: ross.burton, richard.purdie, raj.khem, mark.hatle; +Cc: openembedded-core
The build paths come from debug mode, and GConf is effectively
unmaintained these days anyway. So explicitly disable debug
to fix build paths QA issue.
The minimum still keeps Glib check and assert which can result in
crashes if bad data is passed to the API.
[YOCTO #7058]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-gnome/gnome/gconf_3.2.6.bb | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-gnome/gnome/gconf_3.2.6.bb b/meta/recipes-gnome/gnome/gconf_3.2.6.bb
index feaf080..76fca14 100644
--- a/meta/recipes-gnome/gnome/gconf_3.2.6.bb
+++ b/meta/recipes-gnome/gnome/gconf_3.2.6.bb
@@ -17,7 +17,7 @@ SRC_URI[archive.sha256sum] = "1912b91803ab09a5eed34d364bf09fe3a2a9c96751fde03a4e
S = "${WORKDIR}/GConf-${PV}"
-EXTRA_OECONF = "--enable-shared --disable-static --enable-debug=yes \
+EXTRA_OECONF = "--enable-shared --disable-static \
--disable-orbit --with-openldap=no --disable-gtk"
# Disable PolicyKit by default
@@ -27,6 +27,7 @@ PACKAGECONFIG_class-native = ""
PACKAGECONFIG_libc-uclibc = ""
PACKAGECONFIG[policykit] = "--enable-defaults-service,--disable-defaults-service,polkit"
+PACKAGECONFIG[debug] = "--enable-debug=yes, --enable-debug=minimum"
do_install_append() {
# this directory need to be created to avoid an Error 256 at gdm launch
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-22 12:12 ` [PATCH 02/16] kernel.bbclass: fix buildpath QA issue Hongxu Jia
@ 2016-03-22 12:21 ` Bruce Ashfield
2016-03-22 16:18 ` Mark Hatle
0 siblings, 1 reply; 31+ messages in thread
From: Bruce Ashfield @ 2016-03-22 12:21 UTC (permalink / raw)
To: Hongxu Jia; +Cc: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 2619 bytes --]
On Tue, Mar 22, 2016 at 8:12 AM, Hongxu Jia <hongxu.jia@windriver.com>
wrote:
> Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable DEBUG_FLAGS could
> not been passed to compiler, so we explicitly add DEBUG_FLAGS to CC to
> replace build path with target path.
>
Can you be more explicit here ? What is typically contained in DEBUG_FLAGS
? The
kernel builds its own compiler line and flags, so anything you are setting
from this
environment, should not be leaking into the kernel build.
.. which leaves me wondering, what exactly is in DEBUG_FLAGS, and how is it
actually fixing the QA issue ?
Bruce
>
> [YOCTO #7058]
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/classes/kernel.bbclass | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index c3eab50..d357ccf 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -207,7 +207,7 @@ kernel_do_compile() {
> copy_initramfs
>
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> fi
> - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
> CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" = "${KERNEL_IMAGETYPE}";
> then
> gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
> "${KERNEL_OUTPUT}"
> fi
> @@ -216,7 +216,7 @@ kernel_do_compile() {
> do_compile_kernelmodules() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
> - oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> + oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>
> # Module.symvers gets updated during the
> # building of the kernel modules. We need to
> --
> 1.9.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
--
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"
[-- Attachment #2: Type: text/html, Size: 3781 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-22 12:21 ` Bruce Ashfield
@ 2016-03-22 16:18 ` Mark Hatle
2016-03-22 16:49 ` Bruce Ashfield
0 siblings, 1 reply; 31+ messages in thread
From: Mark Hatle @ 2016-03-22 16:18 UTC (permalink / raw)
To: Bruce Ashfield, Hongxu Jia
Cc: Patches and discussions about the oe-core layer
On 3/22/16 7:21 AM, Bruce Ashfield wrote:
>
>
> On Tue, Mar 22, 2016 at 8:12 AM, Hongxu Jia <hongxu.jia@windriver.com
> <mailto:hongxu.jia@windriver.com>> wrote:
>
> Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable DEBUG_FLAGS could
> not been passed to compiler, so we explicitly add DEBUG_FLAGS to CC to
> replace build path with target path.
>
>
> Can you be more explicit here ? What is typically contained in DEBUG_FLAGS ? The
> kernel builds its own compiler line and flags, so anything you are setting from this
> environment, should not be leaking into the kernel build.
>
> .. which leaves me wondering, what exactly is in DEBUG_FLAGS, and how is it
> actually fixing the QA issue ?
The specific debug flag in question is being added to remap the on-disk path to
the path we want the items to look like from a debugging point of view.
The typical format of DEBUG_FLAGS is something like:
DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
This was recently updated to include the remapping:
DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
-fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
-fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
-fdebug-prefix-map=${STAGING_DIR_HOST}= \
The idea is the WORKDIR is replaced with '/usr/src/debug' inside of the binary
and any debug symbol references. This makes it much easier for the system to
work with supplied debug sources/symbols.
I'm not sure you want to use DEBUG_FLAGS since anything can be put in there....
Application space code should always use the DEBUG_FLAGS, or I think it's likely
a bug.
The kernel though should probably define it's own flag(s). Most likely you'll
want something simple like:
-fdebug-prefix-map=${WORKDIR}=/usr/src/debug
Then all of the internal references to WORKDIR will be replaced with
'/usr/src/debug' which cross gdb can 'map' or on-target will actually look in
that location. (A corresponding '-dbg' package should be provided with the
matching compilation sources at the local specified.)
--Mark
> Bruce
>
>
>
> [YOCTO #7058]
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com
> <mailto:hongxu.jia@windriver.com>>
> ---
> meta/classes/kernel.bbclass | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index c3eab50..d357ccf 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -207,7 +207,7 @@ kernel_do_compile() {
> copy_initramfs
>
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> fi
> - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
> + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
> CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" = "${KERNEL_IMAGETYPE}"; then
> gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" > "${KERNEL_OUTPUT}"
> fi
> @@ -216,7 +216,7 @@ kernel_do_compile() {
> do_compile_kernelmodules() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
> - oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> + oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC}
> ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>
> # Module.symvers gets updated during the
> # building of the kernel modules. We need to
> --
> 1.9.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> <mailto:Openembedded-core@lists.openembedded.org>
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee at its
> end"
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-22 16:18 ` Mark Hatle
@ 2016-03-22 16:49 ` Bruce Ashfield
2016-03-22 16:57 ` Mark Hatle
2016-03-23 1:29 ` Hongxu Jia
0 siblings, 2 replies; 31+ messages in thread
From: Bruce Ashfield @ 2016-03-22 16:49 UTC (permalink / raw)
To: Mark Hatle; +Cc: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 5316 bytes --]
On Tue, Mar 22, 2016 at 12:18 PM, Mark Hatle <mark.hatle@windriver.com>
wrote:
> On 3/22/16 7:21 AM, Bruce Ashfield wrote:
> >
> >
> > On Tue, Mar 22, 2016 at 8:12 AM, Hongxu Jia <hongxu.jia@windriver.com
> > <mailto:hongxu.jia@windriver.com>> wrote:
> >
> > Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable DEBUG_FLAGS
> could
> > not been passed to compiler, so we explicitly add DEBUG_FLAGS to CC
> to
> > replace build path with target path.
> >
> >
> > Can you be more explicit here ? What is typically contained in
> DEBUG_FLAGS ? The
> > kernel builds its own compiler line and flags, so anything you are
> setting from this
> > environment, should not be leaking into the kernel build.
> >
> > .. which leaves me wondering, what exactly is in DEBUG_FLAGS, and how is
> it
> > actually fixing the QA issue ?
>
> The specific debug flag in question is being added to remap the on-disk
> path to
> the path we want the items to look like from a debugging point of view.
>
> The typical format of DEBUG_FLAGS is something like:
>
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
>
> This was recently updated to include the remapping:
>
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
>
> The idea is the WORKDIR is replaced with '/usr/src/debug' inside of the
> binary
> and any debug symbol references. This makes it much easier for the system
> to
> work with supplied debug sources/symbols.
>
> I'm not sure you want to use DEBUG_FLAGS since anything can be put in
> there....
>
> Application space code should always use the DEBUG_FLAGS, or I think it's
> likely
> a bug.
>
> The kernel though should probably define it's own flag(s). Most likely
> you'll
> want something simple like:
>
> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug
>
I can see that.
What outputs of the kernel build need this remapping ? The userspace parts ?
Something else ?
The point is that we patch the kernel build, or make a specific Kconfig
option, or
something similar to that, versus leaking our build flags into the kernel
build or
"bad things can happen".
So I'm sure the solution in the build is the same, but passing flags via a
general
mechanism like I saw in the patch is generally not a good idea.
Bruce
>
> Then all of the internal references to WORKDIR will be replaced with
> '/usr/src/debug' which cross gdb can 'map' or on-target will actually look
> in
> that location. (A corresponding '-dbg' package should be provided with the
> matching compilation sources at the local specified.)
>
> --Mark
>
> > Bruce
> >
> >
> >
> > [YOCTO #7058]
> >
> > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com
> > <mailto:hongxu.jia@windriver.com>>
> > ---
> > meta/classes/kernel.bbclass | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/kernel.bbclass
> b/meta/classes/kernel.bbclass
> > index c3eab50..d357ccf 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -207,7 +207,7 @@ kernel_do_compile() {
> > copy_initramfs
> >
> >
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> > fi
> > - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE}
> ${KERNEL_ALT_IMAGETYPE}
> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> > + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE}
> ${KERNEL_ALT_IMAGETYPE}
> > CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}"
> ${KERNEL_EXTRA_ARGS}
> > $use_alternate_initrd
> > if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" =
> "${KERNEL_IMAGETYPE}"; then
> > gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
> "${KERNEL_OUTPUT}"
> > fi
> > @@ -216,7 +216,7 @@ kernel_do_compile() {
> > do_compile_kernelmodules() {
> > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> > if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
> > - oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> > + oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> CC="${KERNEL_CC}
> > ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> >
> > # Module.symvers gets updated during the
> > # building of the kernel modules. We need to
> > --
> > 1.9.1
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > <mailto:Openembedded-core@lists.openembedded.org>
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
> >
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its
> > end"
>
>
--
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"
[-- Attachment #2: Type: text/html, Size: 7376 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-22 16:49 ` Bruce Ashfield
@ 2016-03-22 16:57 ` Mark Hatle
2016-03-23 1:29 ` Hongxu Jia
1 sibling, 0 replies; 31+ messages in thread
From: Mark Hatle @ 2016-03-22 16:57 UTC (permalink / raw)
To: Bruce Ashfield; +Cc: Patches and discussions about the oe-core layer
On 3/22/16 11:49 AM, Bruce Ashfield wrote:
>
>
> On Tue, Mar 22, 2016 at 12:18 PM, Mark Hatle <mark.hatle@windriver.com
> <mailto:mark.hatle@windriver.com>> wrote:
>
> On 3/22/16 7:21 AM, Bruce Ashfield wrote:
> >
> >
> > On Tue, Mar 22, 2016 at 8:12 AM, Hongxu Jia <hongxu.jia@windriver.com <mailto:hongxu.jia@windriver.com>
> > <mailto:hongxu.jia@windriver.com <mailto:hongxu.jia@windriver.com>>> wrote:
> >
> > Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable DEBUG_FLAGS could
> > not been passed to compiler, so we explicitly add DEBUG_FLAGS to CC to
> > replace build path with target path.
> >
> >
> > Can you be more explicit here ? What is typically contained in DEBUG_FLAGS ? The
> > kernel builds its own compiler line and flags, so anything you are setting from this
> > environment, should not be leaking into the kernel build.
> >
> > .. which leaves me wondering, what exactly is in DEBUG_FLAGS, and how is it
> > actually fixing the QA issue ?
>
> The specific debug flag in question is being added to remap the on-disk path to
> the path we want the items to look like from a debugging point of view.
>
> The typical format of DEBUG_FLAGS is something like:
>
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
>
> This was recently updated to include the remapping:
>
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
>
> The idea is the WORKDIR is replaced with '/usr/src/debug' inside of the binary
> and any debug symbol references. This makes it much easier for the system to
> work with supplied debug sources/symbols.
>
> I'm not sure you want to use DEBUG_FLAGS since anything can be put in there....
>
> Application space code should always use the DEBUG_FLAGS, or I think it's likely
> a bug.
>
> The kernel though should probably define it's own flag(s). Most likely you'll
> want something simple like:
>
> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug
>
>
>
> I can see that.
>
> What outputs of the kernel build need this remapping ? The userspace parts ?
> Something else ?
The mapping affects __FILE__ references, internal dwarf symbols, etc.
So you need to have a mapping that covers the source and any intermediate
objects. In the general case using WORKDIR (as above) results in:
/usr/src/debug/<source>
/usr/src/debug/<build>
(This isn't right and I'll be commenting on the patch about that.. since it
should be wrapped in a package/version...)
Typically in YP 2.0 and before, the format was:
/usr/src/debug/<recipe>/<version>/[directories]
> The point is that we patch the kernel build, or make a specific Kconfig option, or
> something similar to that, versus leaking our build flags into the kernel build or
> "bad things can happen".
I agree.. this flag change needs to be unique to the kernel environment. The
key thing is that you want the kernel's internal file references __FILE__, dwarf
symbols, etc to have the format of an on-target filesystem location that can be
filled out.
> So I'm sure the solution in the build is the same, but passing flags via a general
> mechanism like I saw in the patch is generally not a good idea.
Agree.
--Mark
> Bruce
>
>
>
> Then all of the internal references to WORKDIR will be replaced with
> '/usr/src/debug' which cross gdb can 'map' or on-target will actually look in
> that location. (A corresponding '-dbg' package should be provided with the
> matching compilation sources at the local specified.)
>
> --Mark
>
> > Bruce
> >
> >
> >
> > [YOCTO #7058]
> >
> > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com <mailto:hongxu.jia@windriver.com>
> > <mailto:hongxu.jia@windriver.com <mailto:hongxu.jia@windriver.com>>>
> > ---
> > meta/classes/kernel.bbclass | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > index c3eab50..d357ccf 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -207,7 +207,7 @@ kernel_do_compile() {
> > copy_initramfs
> >
> >
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> > fi
> > - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> > + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
> > CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> > $use_alternate_initrd
> > if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" =
> "${KERNEL_IMAGETYPE}"; then
> > gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
> "${KERNEL_OUTPUT}"
> > fi
> > @@ -216,7 +216,7 @@ kernel_do_compile() {
> > do_compile_kernelmodules() {
> > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> > if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
> > - oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> > + oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> CC="${KERNEL_CC}
> > ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> >
> > # Module.symvers gets updated during the
> > # building of the kernel modules. We need to
> > --
> > 1.9.1
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> <mailto:Openembedded-core@lists.openembedded.org>
> > <mailto:Openembedded-core@lists.openembedded.org
> <mailto:Openembedded-core@lists.openembedded.org>>
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
> >
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its
> > end"
>
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee at its
> end"
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used
2016-03-22 12:12 ` [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
@ 2016-03-22 17:01 ` Mark Hatle
2016-03-23 1:11 ` Hongxu Jia
0 siblings, 1 reply; 31+ messages in thread
From: Mark Hatle @ 2016-03-22 17:01 UTC (permalink / raw)
To: Hongxu Jia, ross.burton, richard.purdie, raj.khem; +Cc: openembedded-core
On 3/22/16 7:12 AM, Hongxu Jia wrote:
> Tweak DEBUG_FLAGS to use "/usr/src/debug" as source target path
> in DWARF. While use gdb to debug binary, it could work with
> sources in dbg package.
>
> While -fdebug-prefix-map is used for compiling, we do not need
> invoking debugedit to edit DWARF at do_package time, but list
> where sources files are.
>
> The copydebugsources uses the list to copy sources to dbg package.
> It works whether -fdebug-prefix-map used or not.
>
> [YOCTO #9305]
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/classes/package.bbclass | 31 +++++++++++++++++++++++++------
> meta/conf/bitbake.conf | 3 +--
> 2 files changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index ef80e50..478a1d9 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -300,6 +300,15 @@ def get_conffiles(pkg, d):
> os.chdir(cwd)
> return conf_list
>
> +def checkbuildpath(file, d):
> + tmpdir = d.getVar('TMPDIR', True)
> + with open(file) as f:
> + file_content = f.read()
> + if tmpdir in file_content:
> + return True
> +
> + return False
> +
> def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
> # Function to split a single file into two components, one is the stripped
> # target system binary, the other contains any debugging information. The
> @@ -313,7 +322,6 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
> objcopy = d.getVar("OBJCOPY", True)
> debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
> workdir = d.getVar("WORKDIR", True)
> - workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or os.path.dirname(os.path.dirname(workdir))
>
> # We ignore kernel modules, we don't generate debug info files.
> if file.find("/lib/modules/") != -1 and file.endswith(".ko"):
> @@ -327,7 +335,7 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
>
> # We need to extract the debug src information here...
> if debugsrcdir:
> - cmd = "'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (debugedit, workparentdir, debugsrcdir, sourcefile, file)
> + cmd = "'%s' -i -l '%s' '%s'" % (debugedit, sourcefile, file)
> (retval, output) = oe.utils.getstatusoutput(cmd)
> if retval:
> bb.fatal("debugedit failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
> @@ -363,8 +371,14 @@ def copydebugsources(debugsrcdir, d):
> objcopy = d.getVar("OBJCOPY", True)
> debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
> workdir = d.getVar("WORKDIR", True)
> - workparentdir = os.path.dirname(os.path.dirname(workdir))
> - workbasedir = os.path.basename(os.path.dirname(workdir)) + "/" + os.path.basename(workdir)
> +
> + # If build path exists in sourcefile, it means toolchain did not use
> + # -fdebug-prefix-map to compile
> + if checkbuildpath(sourcefile, d):
> + localsrc_word = workdir
> + else:
> + localsrc_word = "/usr/src/debug"
> + localsrc_prefix = localsrc_word + '/'
>
> nosuchdir = []
> basepath = dvar
> @@ -379,17 +393,22 @@ def copydebugsources(debugsrcdir, d):
> # We need to ignore files that are not actually ours
> # we do this by only paying attention to items from this package
> processdebugsrc += "fgrep -zw '%s' | "
> + # Remove prefix in the source paths
> + processdebugsrc += "sed 's#%s##g' | "
> +
> processdebugsrc += "(cd '%s' ; cpio -pd0mlL --no-preserve-owner '%s%s' 2>/dev/null)"
>
> - cmd = processdebugsrc % (sourcefile, workbasedir, workparentdir, dvar, debugsrcdir)
> + cmd = processdebugsrc % (sourcefile, localsrc_word, localsrc_prefix, workdir, dvar, debugsrcdir)
> (retval, output) = oe.utils.getstatusoutput(cmd)
> +
> # Can "fail" if internal headers/transient sources are attempted
> #if retval:
> # bb.fatal("debug source copy failed with exit code %s (cmd was %s)" % (retval, cmd))
>
> # cpio seems to have a bug with -lL together and symbolic links are just copied, not dereferenced.
> # Work around this by manually finding and copying any symbolic links that made it through.
> - cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s' 2>/dev/null)" % (dvar, debugsrcdir, dvar, debugsrcdir, workparentdir, dvar, debugsrcdir)
> + cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s' 2>/dev/null)" % (dvar, debugsrcdir, dvar, debugsrcdir, workdir, dvar, debugsrcdir)
> +
> (retval, output) = oe.utils.getstatusoutput(cmd)
> if retval:
> bb.fatal("debugsrc symlink fixup failed with exit code %s (cmd was %s)" % (retval, cmd))
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 7ed5ffb..838ee81 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -550,8 +550,7 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
> # Optimization flags.
> ##################################################################
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
> - -fdebug-prefix-map=${B}=/usr/src/${BPN} \
> - -fdebug-prefix-map=${S}=/usr/src/${BPN} \
> + -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
The above is incorrect.
Replacing WORKDIR w/ /usr/src/debug will lead to file collisions. Since a lot
of 'workdir' directories have a 'build' directory. Any temporary files created
in the build will now end up in a shared build directory. In addition if there
are differences between different 'arch' source dirs, the same issue can occur
causing file conflicts.
In the past the /usr/src/debug location would have been:
/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}
That ensures that all recipes, even multiple recipes of different versions have
a uniquely named directory structure. (PN is important vs BPN, as it has the
remapped 'lib32-' or similar extension to prevent collision.)
--Mark
> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
> "
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used
2016-03-22 17:01 ` Mark Hatle
@ 2016-03-23 1:11 ` Hongxu Jia
0 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 1:11 UTC (permalink / raw)
To: Mark Hatle, ross.burton, richard.purdie, raj.khem; +Cc: openembedded-core
On 03/23/2016 01:01 AM, Mark Hatle wrote:
> On 3/22/16 7:12 AM, Hongxu Jia wrote:
>> Tweak DEBUG_FLAGS to use "/usr/src/debug" as source target path
>> in DWARF. While use gdb to debug binary, it could work with
>> sources in dbg package.
>>
>> While -fdebug-prefix-map is used for compiling, we do not need
>> invoking debugedit to edit DWARF at do_package time, but list
>> where sources files are.
>>
>> The copydebugsources uses the list to copy sources to dbg package.
>> It works whether -fdebug-prefix-map used or not.
>>
>> [YOCTO #9305]
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>> ---
>> meta/classes/package.bbclass | 31 +++++++++++++++++++++++++------
>> meta/conf/bitbake.conf | 3 +--
>> 2 files changed, 26 insertions(+), 8 deletions(-)
>>
>> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
>> index ef80e50..478a1d9 100644
>> --- a/meta/classes/package.bbclass
>> +++ b/meta/classes/package.bbclass
>> @@ -300,6 +300,15 @@ def get_conffiles(pkg, d):
>> os.chdir(cwd)
>> return conf_list
>>
>> +def checkbuildpath(file, d):
>> + tmpdir = d.getVar('TMPDIR', True)
>> + with open(file) as f:
>> + file_content = f.read()
>> + if tmpdir in file_content:
>> + return True
>> +
>> + return False
>> +
>> def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
>> # Function to split a single file into two components, one is the stripped
>> # target system binary, the other contains any debugging information. The
>> @@ -313,7 +322,6 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
>> objcopy = d.getVar("OBJCOPY", True)
>> debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
>> workdir = d.getVar("WORKDIR", True)
>> - workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or os.path.dirname(os.path.dirname(workdir))
>>
>> # We ignore kernel modules, we don't generate debug info files.
>> if file.find("/lib/modules/") != -1 and file.endswith(".ko"):
>> @@ -327,7 +335,7 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
>>
>> # We need to extract the debug src information here...
>> if debugsrcdir:
>> - cmd = "'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (debugedit, workparentdir, debugsrcdir, sourcefile, file)
>> + cmd = "'%s' -i -l '%s' '%s'" % (debugedit, sourcefile, file)
>> (retval, output) = oe.utils.getstatusoutput(cmd)
>> if retval:
>> bb.fatal("debugedit failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
>> @@ -363,8 +371,14 @@ def copydebugsources(debugsrcdir, d):
>> objcopy = d.getVar("OBJCOPY", True)
>> debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
>> workdir = d.getVar("WORKDIR", True)
>> - workparentdir = os.path.dirname(os.path.dirname(workdir))
>> - workbasedir = os.path.basename(os.path.dirname(workdir)) + "/" + os.path.basename(workdir)
>> +
>> + # If build path exists in sourcefile, it means toolchain did not use
>> + # -fdebug-prefix-map to compile
>> + if checkbuildpath(sourcefile, d):
>> + localsrc_word = workdir
>> + else:
>> + localsrc_word = "/usr/src/debug"
>> + localsrc_prefix = localsrc_word + '/'
>>
>> nosuchdir = []
>> basepath = dvar
>> @@ -379,17 +393,22 @@ def copydebugsources(debugsrcdir, d):
>> # We need to ignore files that are not actually ours
>> # we do this by only paying attention to items from this package
>> processdebugsrc += "fgrep -zw '%s' | "
>> + # Remove prefix in the source paths
>> + processdebugsrc += "sed 's#%s##g' | "
>> +
>> processdebugsrc += "(cd '%s' ; cpio -pd0mlL --no-preserve-owner '%s%s' 2>/dev/null)"
>>
>> - cmd = processdebugsrc % (sourcefile, workbasedir, workparentdir, dvar, debugsrcdir)
>> + cmd = processdebugsrc % (sourcefile, localsrc_word, localsrc_prefix, workdir, dvar, debugsrcdir)
>> (retval, output) = oe.utils.getstatusoutput(cmd)
>> +
>> # Can "fail" if internal headers/transient sources are attempted
>> #if retval:
>> # bb.fatal("debug source copy failed with exit code %s (cmd was %s)" % (retval, cmd))
>>
>> # cpio seems to have a bug with -lL together and symbolic links are just copied, not dereferenced.
>> # Work around this by manually finding and copying any symbolic links that made it through.
>> - cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s' 2>/dev/null)" % (dvar, debugsrcdir, dvar, debugsrcdir, workparentdir, dvar, debugsrcdir)
>> + cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s' 2>/dev/null)" % (dvar, debugsrcdir, dvar, debugsrcdir, workdir, dvar, debugsrcdir)
>> +
>> (retval, output) = oe.utils.getstatusoutput(cmd)
>> if retval:
>> bb.fatal("debugsrc symlink fixup failed with exit code %s (cmd was %s)" % (retval, cmd))
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index 7ed5ffb..838ee81 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -550,8 +550,7 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
>> # Optimization flags.
>> ##################################################################
>> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
>> - -fdebug-prefix-map=${B}=/usr/src/${BPN} \
>> - -fdebug-prefix-map=${S}=/usr/src/${BPN} \
>> + -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
> The above is incorrect.
>
> Replacing WORKDIR w/ /usr/src/debug will lead to file collisions. Since a lot
> of 'workdir' directories have a 'build' directory. Any temporary files created
> in the build will now end up in a shared build directory. In addition if there
> are differences between different 'arch' source dirs, the same issue can occur
> causing file conflicts.
>
> In the past the /usr/src/debug location would have been:
>
> /usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}
>
> That ensures that all recipes, even multiple recipes of different versions have
> a uniquely named directory structure. (PN is important vs BPN, as it has the
> remapped 'lib32-' or similar extension to prevent collision.)
Yes, you are right, I thought you should point it out (but not sure).
So I pulled this V1 for discussing, and I will change it as you
suggested in V2.
//Hongxu
> --Mark
>
>> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
>> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
>> "
>>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-22 16:49 ` Bruce Ashfield
2016-03-22 16:57 ` Mark Hatle
@ 2016-03-23 1:29 ` Hongxu Jia
2016-03-23 14:28 ` Bruce Ashfield
1 sibling, 1 reply; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 1:29 UTC (permalink / raw)
To: Bruce Ashfield, Mark Hatle
Cc: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 7206 bytes --]
On 03/23/2016 12:49 AM, Bruce Ashfield wrote:
>
>
> On Tue, Mar 22, 2016 at 12:18 PM, Mark Hatle <mark.hatle@windriver.com
> <mailto:mark.hatle@windriver.com>> wrote:
>
> On 3/22/16 7:21 AM, Bruce Ashfield wrote:
> >
> >
> > On Tue, Mar 22, 2016 at 8:12 AM, Hongxu Jia
> <hongxu.jia@windriver.com <mailto:hongxu.jia@windriver.com>
> > <mailto:hongxu.jia@windriver.com <mailto:hongxu.jia@windriver.com>>> wrote:
> >
> > Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable
> DEBUG_FLAGS could
> > not been passed to compiler, so we explicitly add
> DEBUG_FLAGS to CC to
> > replace build path with target path.
> >
> >
> > Can you be more explicit here ? What is typically contained in
> DEBUG_FLAGS ? The
> > kernel builds its own compiler line and flags, so anything you
> are setting from this
> > environment, should not be leaking into the kernel build.
> >
> > .. which leaves me wondering, what exactly is in DEBUG_FLAGS,
> and how is it
> > actually fixing the QA issue ?
>
> The specific debug flag in question is being added to remap the
> on-disk path to
> the path we want the items to look like from a debugging point of
> view.
>
> The typical format of DEBUG_FLAGS is something like:
>
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
>
> This was recently updated to include the remapping:
>
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
>
> The idea is the WORKDIR is replaced with '/usr/src/debug' inside
> of the binary
> and any debug symbol references. This makes it much easier for
> the system to
> work with supplied debug sources/symbols.
>
> I'm not sure you want to use DEBUG_FLAGS since anything can be put
> in there....
>
> Application space code should always use the DEBUG_FLAGS, or I
> think it's likely
> a bug.
>
> The kernel though should probably define it's own flag(s). Most
> likely you'll
> want something simple like:
>
> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug
>
>
>
> I can see that.
>
> What outputs of the kernel build need this remapping ? The userspace
> parts ?
> Something else ?
>
While building kernel, there are many build paths which caused by macro
__FILE__.
Once you invoke marco WARN_ON in kernel/module, there will be a __FILE__
used.
....
#define WARN_ON(condition) ({ \
int __ret_warn_on = !!(condition); \
if (unlikely(__ret_warn_on)) \
__WARN(); \
unlikely(__ret_warn_on); \
})
#define __WARN() warn_slowpath_null(__FILE__, __LINE__)
....
That's why I add -fdebug-prefix-map to kernel build. I original added a
new option
-ffile-prefix-map for gcc to remap __FILE__, and enhance -fdebug-prefix-map
to replace -ffile-prefix-map to compat with external toolchain.
> The point is that we patch the kernel build, or make a specific
> Kconfig option, or
> something similar to that, versus leaking our build flags into the
> kernel build or
> "bad things can happen".
>
If the leaking idea is bad, would you like to accept '-fdebug-prefix-map' as
kernel's private?
Or suggestions else to fix the build path in kernel?
//Hongxu
> So I'm sure the solution in the build is the same, but passing flags
> via a general
> mechanism like I saw in the patch is generally not a good idea.
>
> Bruce
>
>
> Then all of the internal references to WORKDIR will be replaced with
> '/usr/src/debug' which cross gdb can 'map' or on-target will
> actually look in
> that location. (A corresponding '-dbg' package should be provided
> with the
> matching compilation sources at the local specified.)
>
> --Mark
>
> > Bruce
> >
> >
> >
> > [YOCTO #7058]
> >
> > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com
> <mailto:hongxu.jia@windriver.com>
> > <mailto:hongxu.jia@windriver.com
> <mailto:hongxu.jia@windriver.com>>>
> > ---
> > meta/classes/kernel.bbclass | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/kernel.bbclass
> b/meta/classes/kernel.bbclass
> > index c3eab50..d357ccf 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -207,7 +207,7 @@ kernel_do_compile() {
> > copy_initramfs
> >
> >
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> > fi
> > - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE}
> ${KERNEL_ALT_IMAGETYPE}
> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> > + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE}
> ${KERNEL_ALT_IMAGETYPE}
> > CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}"
> ${KERNEL_EXTRA_ARGS}
> > $use_alternate_initrd
> > if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" =
> "${KERNEL_IMAGETYPE}"; then
> > gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
> "${KERNEL_OUTPUT}"
> > fi
> > @@ -216,7 +216,7 @@ kernel_do_compile() {
> > do_compile_kernelmodules() {
> > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> > if (grep -q -i -e '^CONFIG_MODULES=y$'
> ${B}/.config); then
> > - oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> > + oe_runmake -C ${B} ${PARALLEL_MAKE} modules
> CC="${KERNEL_CC}
> > ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> >
> > # Module.symvers gets updated during the
> > # building of the kernel modules. We need to
> > --
> > 1.9.1
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> <mailto:Openembedded-core@lists.openembedded.org>
> > <mailto:Openembedded-core@lists.openembedded.org
> <mailto:Openembedded-core@lists.openembedded.org>>
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
> >
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness
> await thee at its
> > end"
>
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end"
[-- Attachment #2: Type: text/html, Size: 13604 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH V6 00/15] fix buildpaths QA issue
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
` (15 preceding siblings ...)
2016-03-22 12:12 ` [PATCH 16/16] gconf: " Hongxu Jia
@ 2016-03-23 6:15 ` Hongxu Jia
2016-03-23 6:15 ` [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
2016-03-23 6:15 ` [PATCH 03/15] fix_buildpaths.bbclass: add bbclass to fix build path Hongxu Jia
16 siblings, 2 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 6:15 UTC (permalink / raw)
To: ross.burton, richard.purdie, bruce.ashfield, mark.hatle; +Cc: openembedded-core
Changed in V6:
- Drop kernel build path fix
- Tweak debug location to /usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}
to keep syn with past
- Tweak PACKAGE_BUILDPATH_TEXT_PATTERNS in fix_buildpaths.bbclass to
adapt the above fix;
Follow test case in V5:
-----------
root@qemux86-64:~# ls /usr/src/debug/
acl bash e2fsprogs glibc libcap libxcb ncurses service util-linux zlib
attr busybox expat kbd libx11 libxdmcp opkg-utils shadow v86d
base-passwd dbus gdb kmod libxau libxkbcommon readline systemd xz
root@qemux86-64:~# gdb skeleton-test
(gdb) r
Starting program: /usr/sbin/skeleton-test
hello world
Program received signal SIGSEGV, Segmentation fault.
0x000000000040056b in main (argc=1, argv=0x7fffffffed08) at /usr/src/debug/service/0.1-r0/skeleton_test.c:8
8 *p = 1;
(gdb) list
3 int main(int argc, char *argv[])
4 {
5 printf("hello world\n");
6
7 char *p = NULL;
8 *p = 1;
9 return 0;
10 }
-----------
//Hongxu
The following changes since commit 2f2f784727a1617c2b9ce0bb86730ba301238e06:
parselogs: add new whitelist entries to address 4.4.3 issues (2016-03-22 08:54:34 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib hongxu/fix-buildpath
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=hongxu/fix-buildpath
Hongxu Jia (15):
conf/bitbake.conf package.bbclass: fix dbg package not contain sources
while -fdebug-prefix-map used
dtc.inc: fix buildpaths QA issue
fix_buildpaths.bbclass: add bbclass to fix build path
icu: fix buildpaths QA issue
tcl: fix buildpath QA issue
python2/3: fix buildpath QA issue
bbclass distutils/distutils3: fix .pyc/.pyo buildpath
bbclass distutils/distutils3/setuptools/setuptools3: clean up
DISTUTILS_INSTALL_ARGS
python-setuptools/python3-setuptools: use old-style install
python3-pip: use old-style install
waf.bbclass: support do patch on extracted files
python-pycairo: fix buildpath QA issue
openssl: fix buildpath QA issue
epiphany: fix buildpaths QA issue
gconf: fix buildpaths QA issue
meta/classes/distutils.bbclass | 8 +--
meta/classes/distutils3.bbclass | 8 +--
meta/classes/fix_buildpaths.bbclass | 55 +++++++++++++++++++
meta/classes/package.bbclass | 24 +++++++--
meta/classes/setuptools.bbclass | 5 --
meta/classes/setuptools3.bbclass | 5 --
meta/classes/waf.bbclass | 11 +++-
meta/conf/bitbake.conf | 3 +-
.../openssl/remove-build-path-in-source-code.patch | 44 +++++++++++++++
.../recipes-connectivity/openssl/openssl_1.0.2g.bb | 7 +++
...trip-off-build-path-in-purported-filename.patch | 62 ++++++++++++++++++++++
.../python/python-pycairo_1.10.0.bb | 25 +++------
.../python/python-setuptools_19.4.bb | 2 -
meta/recipes-devtools/python/python3-pip_8.0.0.bb | 12 -----
.../python/python3-setuptools_19.4.bb | 8 +--
meta/recipes-devtools/python/python3_3.5.1.bb | 20 +++++--
meta/recipes-devtools/python/python_2.7.11.bb | 29 ++++++++--
meta/recipes-devtools/tcltk/tcl_8.6.4.bb | 9 +++-
meta/recipes-gnome/epiphany/epiphany_3.18.4.bb | 8 ++-
.../files/use-relative-path-for-build-dir.patch | 57 ++++++++++++++++++++
meta/recipes-gnome/gnome/gconf_3.2.6.bb | 3 +-
meta/recipes-kernel/dtc/dtc.inc | 3 ++
.../dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch | 30 +++++++++++
.../icu/icu/add-variable-U_CC-and-U_CXX.patch | 61 +++++++++++++++++++++
meta/recipes-support/icu/icu_56.1.bb | 12 +++++
25 files changed, 435 insertions(+), 76 deletions(-)
create mode 100644 meta/classes/fix_buildpaths.bbclass
create mode 100644 meta/recipes-connectivity/openssl/openssl/remove-build-path-in-source-code.patch
create mode 100644 meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch
create mode 100644 meta/recipes-gnome/epiphany/files/use-relative-path-for-build-dir.patch
create mode 100644 meta/recipes-kernel/dtc/dtc/pass-DEBUG_FLAGS-to-CFLAGS.patch
create mode 100644 meta/recipes-support/icu/icu/add-variable-U_CC-and-U_CXX.patch
--
1.9.1
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used
2016-03-23 6:15 ` [PATCH V6 00/15] " Hongxu Jia
@ 2016-03-23 6:15 ` Hongxu Jia
2016-03-23 13:40 ` Mark Hatle
2016-03-23 6:15 ` [PATCH 03/15] fix_buildpaths.bbclass: add bbclass to fix build path Hongxu Jia
1 sibling, 1 reply; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 6:15 UTC (permalink / raw)
To: ross.burton, richard.purdie, bruce.ashfield, mark.hatle; +Cc: openembedded-core
Tweak DEBUG_FLAGS to use "/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
as source target path in DWARF. While use gdb to debug binary, it could
work with sources in dbg package.
While -fdebug-prefix-map is used for compiling, we do not need invoking
debugedit to edit DWARF at do_package time, but list where sources files
are.
The copydebugsources uses the list to copy sources to dbg package. It
works whether -fdebug-prefix-map used or not.
[YOCTO #9305]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/package.bbclass | 24 ++++++++++++++++++++----
meta/conf/bitbake.conf | 3 +--
2 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index ef80e50..b2f5868 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -300,6 +300,15 @@ def get_conffiles(pkg, d):
os.chdir(cwd)
return conf_list
+def checkbuildpath(file, d):
+ tmpdir = d.getVar('TMPDIR', True)
+ with open(file) as f:
+ file_content = f.read()
+ if tmpdir in file_content:
+ return True
+
+ return False
+
def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
# Function to split a single file into two components, one is the stripped
# target system binary, the other contains any debugging information. The
@@ -312,8 +321,6 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
dvar = d.getVar('PKGD', True)
objcopy = d.getVar("OBJCOPY", True)
debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
- workdir = d.getVar("WORKDIR", True)
- workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or os.path.dirname(os.path.dirname(workdir))
# We ignore kernel modules, we don't generate debug info files.
if file.find("/lib/modules/") != -1 and file.endswith(".ko"):
@@ -327,7 +334,7 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
# We need to extract the debug src information here...
if debugsrcdir:
- cmd = "'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (debugedit, workparentdir, debugsrcdir, sourcefile, file)
+ cmd = "'%s' -i -l '%s' '%s'" % (debugedit, sourcefile, file)
(retval, output) = oe.utils.getstatusoutput(cmd)
if retval:
bb.fatal("debugedit failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
@@ -366,6 +373,13 @@ def copydebugsources(debugsrcdir, d):
workparentdir = os.path.dirname(os.path.dirname(workdir))
workbasedir = os.path.basename(os.path.dirname(workdir)) + "/" + os.path.basename(workdir)
+ # If build path exists in sourcefile, it means toolchain did not use
+ # -fdebug-prefix-map to compile
+ if checkbuildpath(sourcefile, d):
+ localsrc_prefix = workparentdir + "/"
+ else:
+ localsrc_prefix = "/usr/src/debug/"
+
nosuchdir = []
basepath = dvar
for p in debugsrcdir.split("/"):
@@ -379,9 +393,11 @@ def copydebugsources(debugsrcdir, d):
# We need to ignore files that are not actually ours
# we do this by only paying attention to items from this package
processdebugsrc += "fgrep -zw '%s' | "
+ # Remove prefix in the source paths
+ processdebugsrc += "sed 's#%s##g' | "
processdebugsrc += "(cd '%s' ; cpio -pd0mlL --no-preserve-owner '%s%s' 2>/dev/null)"
- cmd = processdebugsrc % (sourcefile, workbasedir, workparentdir, dvar, debugsrcdir)
+ cmd = processdebugsrc % (sourcefile, workbasedir, localsrc_prefix, workparentdir, dvar, debugsrcdir)
(retval, output) = oe.utils.getstatusoutput(cmd)
# Can "fail" if internal headers/transient sources are attempted
#if retval:
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 7ed5ffb..6bd6075 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -550,8 +550,7 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
# Optimization flags.
##################################################################
DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
- -fdebug-prefix-map=${B}=/usr/src/${BPN} \
- -fdebug-prefix-map=${S}=/usr/src/${BPN} \
+ -fdebug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
-fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
-fdebug-prefix-map=${STAGING_DIR_HOST}= \
"
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 03/15] fix_buildpaths.bbclass: add bbclass to fix build path
2016-03-23 6:15 ` [PATCH V6 00/15] " Hongxu Jia
2016-03-23 6:15 ` [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
@ 2016-03-23 6:15 ` Hongxu Jia
1 sibling, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 6:15 UTC (permalink / raw)
To: ross.burton, richard.purdie, bruce.ashfield, mark.hatle; +Cc: openembedded-core
Define variable PACKAGE_BUILDPATH_TEXT_FILES to list files that have
build paths and remove these paths at do_package time. It supports
package override while files exist in conditionally generated package.
(such as ${PN}-ptest is conditionally generated)
Define variable PACKAGE_BUILDPATH_TEXT_PATTERNS to list build path
patterns, which used by sed, it removes --sysroot and
-fdebug-prefix-map in text files by default.
Define python function to remove build path in variable.
Define python function to remove build path in python compiled
code.
[YOCTO #9169]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/fix_buildpaths.bbclass | 55 +++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 meta/classes/fix_buildpaths.bbclass
diff --git a/meta/classes/fix_buildpaths.bbclass b/meta/classes/fix_buildpaths.bbclass
new file mode 100644
index 0000000..440a1b2
--- /dev/null
+++ b/meta/classes/fix_buildpaths.bbclass
@@ -0,0 +1,55 @@
+PACKAGE_PREPROCESS_FUNCS += 'remove_buildpath_package_preprocess'
+
+def list_package_buildpath_text_files(d):
+ files = d.getVar('PACKAGE_BUILDPATH_TEXT_FILES', True) or ''
+
+ # Support package override
+ for pn in d.getVar('PACKAGES', True).split():
+ files += d.getVar('PACKAGE_BUILDPATH_TEXT_FILES_%s' % pn, True) or ''
+
+ return files
+
+remove_buildpath_package_preprocess () {
+ buildpath_files="${@list_package_buildpath_text_files(d)}"
+
+ # Remove build paths in text files
+ for file in $buildpath_files;do
+ sed -i ${PACKAGE_BUILDPATH_TEXT_PATTERNS} ${PKGD}$file
+ done
+}
+
+# List build path patterns, which used by sed, it removes
+# --sysroot and -fdebug-prefix-map by default.
+PACKAGE_BUILDPATH_TEXT_PATTERNS = " \
+ -e 's:--sysroot=${STAGING_DIR_TARGET}::g' \
+ -e 's:-fdebug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}::g' \
+ -e 's:-fdebug-prefix-map=${STAGING_DIR_NATIVE}=::g' \
+ -e 's:-fdebug-prefix-map=${STAGING_DIR_HOST}=::g' \
+"
+
+# List files which have build paths, and remove these paths.
+PACKAGE_BUILDPATH_TEXT_FILES ??= ""
+
+# Remove build path in variable
+def remove_buildpath_variable(d, var):
+ val = d.getVar(var, True) or ''
+
+ toolchain_opts = d.getVar('TOOLCHAIN_OPTIONS', True)
+ val = val.replace(toolchain_opts, '')
+
+ debug_flags = d.getVar('DEBUG_FLAGS', True).split()
+ for opt in debug_flags:
+ if '-fdebug-prefix-map=' in opt:
+ val = val.replace(opt, '')
+
+ return val
+
+# Remove build path in python compiled code which
+# located in root_path
+def remove_buildpath_bytecode(root_path, byte_code):
+ import py_compile
+
+ file = root_path + byte_code[0:-1]
+ dfile = byte_code[0:-1]
+ py_compile.compile(file, dfile=dfile)
+
--
1.9.1
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used
2016-03-23 6:15 ` [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
@ 2016-03-23 13:40 ` Mark Hatle
2016-03-23 14:15 ` Hongxu Jia
0 siblings, 1 reply; 31+ messages in thread
From: Mark Hatle @ 2016-03-23 13:40 UTC (permalink / raw)
To: Hongxu Jia, ross.burton, richard.purdie, bruce.ashfield; +Cc: openembedded-core
On 3/23/16 1:15 AM, Hongxu Jia wrote:
> Tweak DEBUG_FLAGS to use "/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
> as source target path in DWARF. While use gdb to debug binary, it could
> work with sources in dbg package.
>
> While -fdebug-prefix-map is used for compiling, we do not need invoking
> debugedit to edit DWARF at do_package time, but list where sources files
> are.
>
> The copydebugsources uses the list to copy sources to dbg package. It
> works whether -fdebug-prefix-map used or not.
The following looks right to me. I'd recommend this version be used.
However, I'm guessing the work is not done. Anything that has an ${S} outside
of the WORKDIR will not be remapped. I'm guessing this includes the toolchain
and kernel bits.... but other things might also have a shared work dir?
We probably should make it a requirement that those things have their own unique
mapping defined as well. I'm not sure if the existing tests will capture those
issues and alert the user.
(This is also a place in the developer guide we should probably explain what
this debug-prelix-map does and how it should be used to make on-target
debuggable items.)
--Mark
> [YOCTO #9305]
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/classes/package.bbclass | 24 ++++++++++++++++++++----
> meta/conf/bitbake.conf | 3 +--
> 2 files changed, 21 insertions(+), 6 deletions(-)
>
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index ef80e50..b2f5868 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -300,6 +300,15 @@ def get_conffiles(pkg, d):
> os.chdir(cwd)
> return conf_list
>
> +def checkbuildpath(file, d):
> + tmpdir = d.getVar('TMPDIR', True)
> + with open(file) as f:
> + file_content = f.read()
> + if tmpdir in file_content:
> + return True
> +
> + return False
> +
> def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
> # Function to split a single file into two components, one is the stripped
> # target system binary, the other contains any debugging information. The
> @@ -312,8 +321,6 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
> dvar = d.getVar('PKGD', True)
> objcopy = d.getVar("OBJCOPY", True)
> debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
> - workdir = d.getVar("WORKDIR", True)
> - workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or os.path.dirname(os.path.dirname(workdir))
>
> # We ignore kernel modules, we don't generate debug info files.
> if file.find("/lib/modules/") != -1 and file.endswith(".ko"):
> @@ -327,7 +334,7 @@ def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
>
> # We need to extract the debug src information here...
> if debugsrcdir:
> - cmd = "'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (debugedit, workparentdir, debugsrcdir, sourcefile, file)
> + cmd = "'%s' -i -l '%s' '%s'" % (debugedit, sourcefile, file)
> (retval, output) = oe.utils.getstatusoutput(cmd)
> if retval:
> bb.fatal("debugedit failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
> @@ -366,6 +373,13 @@ def copydebugsources(debugsrcdir, d):
> workparentdir = os.path.dirname(os.path.dirname(workdir))
> workbasedir = os.path.basename(os.path.dirname(workdir)) + "/" + os.path.basename(workdir)
>
> + # If build path exists in sourcefile, it means toolchain did not use
> + # -fdebug-prefix-map to compile
> + if checkbuildpath(sourcefile, d):
> + localsrc_prefix = workparentdir + "/"
> + else:
> + localsrc_prefix = "/usr/src/debug/"
> +
> nosuchdir = []
> basepath = dvar
> for p in debugsrcdir.split("/"):
> @@ -379,9 +393,11 @@ def copydebugsources(debugsrcdir, d):
> # We need to ignore files that are not actually ours
> # we do this by only paying attention to items from this package
> processdebugsrc += "fgrep -zw '%s' | "
> + # Remove prefix in the source paths
> + processdebugsrc += "sed 's#%s##g' | "
> processdebugsrc += "(cd '%s' ; cpio -pd0mlL --no-preserve-owner '%s%s' 2>/dev/null)"
>
> - cmd = processdebugsrc % (sourcefile, workbasedir, workparentdir, dvar, debugsrcdir)
> + cmd = processdebugsrc % (sourcefile, workbasedir, localsrc_prefix, workparentdir, dvar, debugsrcdir)
> (retval, output) = oe.utils.getstatusoutput(cmd)
> # Can "fail" if internal headers/transient sources are attempted
> #if retval:
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 7ed5ffb..6bd6075 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -550,8 +550,7 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
> # Optimization flags.
> ##################################################################
> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
> - -fdebug-prefix-map=${B}=/usr/src/${BPN} \
> - -fdebug-prefix-map=${S}=/usr/src/${BPN} \
> + -fdebug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
> "
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used
2016-03-23 13:40 ` Mark Hatle
@ 2016-03-23 14:15 ` Hongxu Jia
0 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 14:15 UTC (permalink / raw)
To: Mark Hatle, ross.burton, richard.purdie, bruce.ashfield; +Cc: openembedded-core
On 03/23/2016 09:40 PM, Mark Hatle wrote:
> The following looks right to me. I'd recommend this version be used.
>
> However, I'm guessing the work is not done. Anything that has an ${S} outside
> of the WORKDIR will not be remapped. I'm guessing this includes the toolchain
> and kernel bits.... but other things might also have a shared work dir?
Yes, we should consider this situation, I will do the investigation to
find the fix
>
> We probably should make it a requirement that those things have their own unique
> mapping defined as well. I'm not sure if the existing tests will capture those
> issues and alert the user.
OK, with some investigation done, I will try to collect the requirements
for discussing.
>
> (This is also a place in the developer guide we should probably explain what
> this debug-prelix-map does and how it should be used to make on-target
> debuggable items.)
Maybe we should introduce -debug-pprefix-map in reference/development
manual, I will file a doc requirement in bugzilla.
So it's just a start, we should continue to work on:
1) Support to remap ${S} outside of the WORKDIR (shared work dir)
2) Add -fdebug-prefix-map usage in manual
//Hongxu
>
> --Mark
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-23 1:29 ` Hongxu Jia
@ 2016-03-23 14:28 ` Bruce Ashfield
2016-03-23 15:50 ` Hongxu Jia
0 siblings, 1 reply; 31+ messages in thread
From: Bruce Ashfield @ 2016-03-23 14:28 UTC (permalink / raw)
To: Hongxu Jia; +Cc: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 7333 bytes --]
On Tue, Mar 22, 2016 at 9:29 PM, Hongxu Jia <hongxu.jia@windriver.com>
wrote:
> On 03/23/2016 12:49 AM, Bruce Ashfield wrote:
>
>
>
> On Tue, Mar 22, 2016 at 12:18 PM, Mark Hatle <mark.hatle@windriver.com>
> wrote:
>
>> On 3/22/16 7:21 AM, Bruce Ashfield wrote:
>> >
>> >
>> > On Tue, Mar 22, 2016 at 8:12 AM, Hongxu Jia <
>> <hongxu.jia@windriver.com>hongxu.jia@windriver.com
>> > <mailto: <hongxu.jia@windriver.com>hongxu.jia@windriver.com>> wrote:
>> >
>> > Since CFLAGS CPPFLAGS CXXFLAGS has been unset, variable DEBUG_FLAGS
>> could
>> > not been passed to compiler, so we explicitly add DEBUG_FLAGS to CC
>> to
>> > replace build path with target path.
>> >
>> >
>> > Can you be more explicit here ? What is typically contained in
>> DEBUG_FLAGS ? The
>> > kernel builds its own compiler line and flags, so anything you are
>> setting from this
>> > environment, should not be leaking into the kernel build.
>> >
>> > .. which leaves me wondering, what exactly is in DEBUG_FLAGS, and how
>> is it
>> > actually fixing the QA issue ?
>>
>> The specific debug flag in question is being added to remap the on-disk
>> path to
>> the path we want the items to look like from a debugging point of view.
>>
>> The typical format of DEBUG_FLAGS is something like:
>>
>> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
>>
>> This was recently updated to include the remapping:
>>
>> DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types \
>> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug \
>> -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
>> -fdebug-prefix-map=${STAGING_DIR_HOST}= \
>>
>> The idea is the WORKDIR is replaced with '/usr/src/debug' inside of the
>> binary
>> and any debug symbol references. This makes it much easier for the
>> system to
>> work with supplied debug sources/symbols.
>>
>> I'm not sure you want to use DEBUG_FLAGS since anything can be put in
>> there....
>>
>> Application space code should always use the DEBUG_FLAGS, or I think it's
>> likely
>> a bug.
>>
>> The kernel though should probably define it's own flag(s). Most likely
>> you'll
>> want something simple like:
>>
>> -fdebug-prefix-map=${WORKDIR}=/usr/src/debug
>>
>
>
> I can see that.
>
> What outputs of the kernel build need this remapping ? The userspace parts
> ?
> Something else ?
>
>
> While building kernel, there are many build paths which caused by macro
> __FILE__.
> Once you invoke marco WARN_ON in kernel/module, there will be a __FILE__
> used.
> ....
> #define WARN_ON(condition) ({ \
> int __ret_warn_on = !!(condition); \
> if (unlikely(__ret_warn_on)) \
> __WARN(); \
> unlikely(__ret_warn_on); \
> })
>
> #define __WARN() warn_slowpath_null(__FILE__, __LINE__)
> ....
>
Aha. Honestly, I've always gotten different paths from the kernel debug
info (or many
applications) via __FILE__, and never considered it much of a problem to go
and
find the source file :) .. but I digress.
>
> That's why I add -fdebug-prefix-map to kernel build. I original added a
> new option
> -ffile-prefix-map for gcc to remap __FILE__, and enhance -fdebug-prefix-map
> to replace -ffile-prefix-map to compat with external toolchain.
>
> The point is that we patch the kernel build, or make a specific Kconfig
> option, or
> something similar to that, versus leaking our build flags into the kernel
> build or
> "bad things can happen".
>
>
> If the leaking idea is bad, would you like to accept '-fdebug-prefix-map'
> as
> kernel's private?
>
Either you patch the kernel, and make it a Kconfig option for the mapping,
or we pass it
in as an intrinsic part of the CC definition, or some variant of those.
Bruce
>
> Or suggestions else to fix the build path in kernel?
>
> //Hongxu
>
>
> So I'm sure the solution in the build is the same, but passing flags via a
> general
> mechanism like I saw in the patch is generally not a good idea.
>
> Bruce
>
>
>>
>> Then all of the internal references to WORKDIR will be replaced with
>> '/usr/src/debug' which cross gdb can 'map' or on-target will actually
>> look in
>> that location. (A corresponding '-dbg' package should be provided with
>> the
>> matching compilation sources at the local specified.)
>>
>> --Mark
>>
>> > Bruce
>> >
>> >
>> >
>> > [YOCTO #7058]
>> >
>> > Signed-off-by: Hongxu Jia < <hongxu.jia@windriver.com>
>> hongxu.jia@windriver.com
>> > <mailto:hongxu.jia@windriver.com>>
>> > ---
>> > meta/classes/kernel.bbclass | 4 ++--
>> > 1 file changed, 2 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/meta/classes/kernel.bbclass
>> b/meta/classes/kernel.bbclass
>> > index c3eab50..d357ccf 100644
>> > --- a/meta/classes/kernel.bbclass
>> > +++ b/meta/classes/kernel.bbclass
>> > @@ -207,7 +207,7 @@ kernel_do_compile() {
>> > copy_initramfs
>> >
>> >
>> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>> > fi
>> > - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE}
>> ${KERNEL_ALT_IMAGETYPE}
>> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>> $use_alternate_initrd
>> > + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE}
>> ${KERNEL_ALT_IMAGETYPE}
>> > CC="${KERNEL_CC} ${DEBUG_FLAGS}" LD="${KERNEL_LD}"
>> ${KERNEL_EXTRA_ARGS}
>> > $use_alternate_initrd
>> > if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" =
>> "${KERNEL_IMAGETYPE}"; then
>> > gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
>> "${KERNEL_OUTPUT}"
>> > fi
>> > @@ -216,7 +216,7 @@ kernel_do_compile() {
>> > do_compile_kernelmodules() {
>> > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
>> > if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
>> > - oe_runmake -C ${B} ${PARALLEL_MAKE} modules
>> > CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>> > + oe_runmake -C ${B} ${PARALLEL_MAKE} modules
>> CC="${KERNEL_CC}
>> > ${DEBUG_FLAGS}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>> >
>> > # Module.symvers gets updated during the
>> > # building of the kernel modules. We need to
>> > --
>> > 1.9.1
>> >
>> > --
>> > _______________________________________________
>> > Openembedded-core mailing list
>> > Openembedded-core@lists.openembedded.org
>> > <mailto:Openembedded-core@lists.openembedded.org>
>> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
>> >
>> >
>> >
>> >
>> > --
>> > "Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its
>> > end"
>>
>>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"
>
>
>
--
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"
[-- Attachment #2: Type: text/html, Size: 14456 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/16] kernel.bbclass: fix buildpath QA issue
2016-03-23 14:28 ` Bruce Ashfield
@ 2016-03-23 15:50 ` Hongxu Jia
0 siblings, 0 replies; 31+ messages in thread
From: Hongxu Jia @ 2016-03-23 15:50 UTC (permalink / raw)
To: Bruce Ashfield; +Cc: Patches and discussions about the oe-core layer
On 03/23/2016 10:28 PM, Bruce Ashfield wrote:
> Either you patch the kernel, and make it a Kconfig option for the
> mapping, or we pass it
> in as an intrinsic part of the CC definition, or some variant of those.
>
OK, I will try to choose one way for review.
BWT, currently, package kernel-dbg (which contains sources) not
generated, should we support it?
If yes, I should do the investigation to fix it.
//Hongxu
> Bruce
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2016-03-23 15:50 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-22 12:12 [PATCH V5 00/16] fix buildpaths QA issue Hongxu Jia
2016-03-22 12:12 ` [PATCH 01/16] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
2016-03-22 17:01 ` Mark Hatle
2016-03-23 1:11 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 02/16] kernel.bbclass: fix buildpath QA issue Hongxu Jia
2016-03-22 12:21 ` Bruce Ashfield
2016-03-22 16:18 ` Mark Hatle
2016-03-22 16:49 ` Bruce Ashfield
2016-03-22 16:57 ` Mark Hatle
2016-03-23 1:29 ` Hongxu Jia
2016-03-23 14:28 ` Bruce Ashfield
2016-03-23 15:50 ` Hongxu Jia
2016-03-22 12:12 ` [PATCH 03/16] dtc.inc: fix buildpaths " Hongxu Jia
2016-03-22 12:12 ` [PATCH 04/16] fix_buildpaths.bbclass: add bbclass to fix build path Hongxu Jia
2016-03-22 12:12 ` [PATCH 05/16] icu: fix buildpaths QA issue Hongxu Jia
2016-03-22 12:12 ` [PATCH 06/16] tcl: fix buildpath " Hongxu Jia
2016-03-22 12:12 ` [PATCH 07/16] python2/3: " Hongxu Jia
2016-03-22 12:12 ` [PATCH 08/16] bbclass distutils/distutils3: fix .pyc/.pyo buildpath Hongxu Jia
2016-03-22 12:12 ` [PATCH 09/16] bbclass distutils/distutils3/setuptools/setuptools3: clean up DISTUTILS_INSTALL_ARGS Hongxu Jia
2016-03-22 12:12 ` [PATCH 10/16] python-setuptools/python3-setuptools: use old-style install Hongxu Jia
2016-03-22 12:12 ` [PATCH 11/16] python3-pip: " Hongxu Jia
2016-03-22 12:12 ` [PATCH 12/16] waf.bbclass: support do patch on extracted files Hongxu Jia
2016-03-22 12:12 ` [PATCH 13/16] python-pycairo: fix buildpath QA issue Hongxu Jia
2016-03-22 12:12 ` [PATCH 14/16] openssl: " Hongxu Jia
2016-03-22 12:12 ` [PATCH 15/16] epiphany: fix buildpaths " Hongxu Jia
2016-03-22 12:12 ` [PATCH 16/16] gconf: " Hongxu Jia
2016-03-23 6:15 ` [PATCH V6 00/15] " Hongxu Jia
2016-03-23 6:15 ` [PATCH V2 01/15] conf/bitbake.conf package.bbclass: fix dbg package not contain sources while -fdebug-prefix-map used Hongxu Jia
2016-03-23 13:40 ` Mark Hatle
2016-03-23 14:15 ` Hongxu Jia
2016-03-23 6:15 ` [PATCH 03/15] fix_buildpaths.bbclass: add bbclass to fix build path Hongxu Jia
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.