* [PATCH 0/3][RFC v2] Machine specific sysroot implementation
@ 2011-01-08 15:52 Dongxiao Xu
2011-01-08 15:52 ` [PATCH 1/3] staging: Use relative path in sysroot-destdir for target recipes Dongxiao Xu
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Dongxiao Xu @ 2011-01-08 15:52 UTC (permalink / raw)
To: poky
Hi Richard,
This RFC is the second version of machine specific sysroot implementation.
Please help to review.
I tested the patchset with a full build of poky-image-sdk and meta-toolchain-sdk for both qemuppc and mpc8315e-rdb machines that are powerpc architecture.
Changes from v1:
1) Re-implementation the mechanism adding ${MACHINE} into stamp file based on the latest bitbake. Also this can avoid adding unnecessary ${MACHINE} information for native/nativesdk/crosssdk/cross-canadian task stamps.
2) Change staging directories layout. Use relative path in sysroot-destdir for target recipes.
3) gcc-cross-intermediate will populate files into two sysroots. The modification in v1 patchset is incorrect. This version uses two sstate input/output dirs to handle it.
Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: dxu4/mach_sysroot_v2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/mach_sysroot_v2
Thanks,
Dongxiao Xu <dongxiao.xu@intel.com>
---
Dongxiao Xu (3):
staging: Use relative path in sysroot-destdir for target recipes
bitbake: Introduce stamp-extra-info into build stamp file
bitbake: machine specific sysroots implementation
bitbake/lib/bb/siggen.py | 14 +++++++++-
meta/classes/binconfig.bbclass | 6 ++--
meta/classes/cross-canadian.bbclass | 3 +-
meta/classes/cross.bbclass | 5 +++
meta/classes/kernel.bbclass | 2 +-
meta/classes/native.bbclass | 2 +
meta/classes/nativesdk.bbclass | 6 +++-
meta/classes/package.bbclass | 1 +
meta/classes/siteconfig.bbclass | 11 ++++---
meta/classes/sstate.bbclass | 6 +++-
meta/classes/staging.bbclass | 28 ++++++++++---------
meta/classes/toolchain-scripts.bbclass | 4 +-
meta/conf/bitbake.conf | 11 ++++---
meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb | 4 +-
.../gcc/gcc-cross-intermediate.inc | 3 ++
meta/recipes-devtools/libtool/libtool-cross_2.4.bb | 4 +-
.../libtool/libtool-nativesdk_2.4.bb | 4 +-
meta/recipes-devtools/libtool/libtool_2.4.bb | 6 ++--
meta/recipes-gnome/gtk+/gtk+.inc | 4 +-
meta/recipes-support/apr/apr_1.3.3.bb | 2 +-
20 files changed, 79 insertions(+), 47 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/3] staging: Use relative path in sysroot-destdir for target recipes
2011-01-08 15:52 [PATCH 0/3][RFC v2] Machine specific sysroot implementation Dongxiao Xu
@ 2011-01-08 15:52 ` Dongxiao Xu
2011-01-08 15:53 ` [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file Dongxiao Xu
2011-01-08 15:53 ` [PATCH 3/3] bitbake: machine specific sysroots implementation Dongxiao Xu
2 siblings, 0 replies; 10+ messages in thread
From: Dongxiao Xu @ 2011-01-08 15:52 UTC (permalink / raw)
To: poky
Original we used absolute path in sysroot-destdir for both native and
target recipes. This commit changes target recipes to use relative path
which is same as the image directory.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
meta/classes/binconfig.bbclass | 6 ++--
meta/classes/cross-canadian.bbclass | 2 +
meta/classes/kernel.bbclass | 2 +-
meta/classes/native.bbclass | 2 +
meta/classes/nativesdk.bbclass | 4 +-
meta/classes/siteconfig.bbclass | 5 ++-
meta/classes/staging.bbclass | 27 ++++++++++---------
meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb | 4 +-
.../gcc/gcc-cross-intermediate.inc | 3 ++
meta/recipes-devtools/libtool/libtool-cross_2.4.bb | 4 +-
.../libtool/libtool-nativesdk_2.4.bb | 4 +-
meta/recipes-devtools/libtool/libtool_2.4.bb | 6 ++--
meta/recipes-gnome/gtk+/gtk+.inc | 4 +-
meta/recipes-support/apr/apr_1.3.3.bb | 2 +-
14 files changed, 42 insertions(+), 33 deletions(-)
diff --git a/meta/classes/binconfig.bbclass b/meta/classes/binconfig.bbclass
index 73ca4d6..7c12bd3 100644
--- a/meta/classes/binconfig.bbclass
+++ b/meta/classes/binconfig.bbclass
@@ -47,8 +47,8 @@ SYSROOT_PREPROCESS_FUNCS += "binconfig_sysroot_preprocess"
binconfig_sysroot_preprocess () {
for config in `find ${S} -name '${BINCONFIG_GLOB}'`; do
configname=`basename $config`
- install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}
- cat $config | sed ${@get_binconfig_mangle(d)} > ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/$configname
- chmod u+x ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/$configname
+ install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts
+ cat $config | sed ${@get_binconfig_mangle(d)} > ${SYSROOT_DESTDIR}${bindir}/crossscripts/$configname
+ chmod u+x ${SYSROOT_DESTDIR}${bindir}/crossscripts/$configname
done
}
diff --git a/meta/classes/cross-canadian.bbclass b/meta/classes/cross-canadian.bbclass
index 3f3a24d..9897105 100644
--- a/meta/classes/cross-canadian.bbclass
+++ b/meta/classes/cross-canadian.bbclass
@@ -71,6 +71,8 @@ base_sbindir = "${bindir}"
libdir = "${exec_prefix}/lib/${OLD_MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
libexecdir = "${exec_prefix}/libexec/${OLD_MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}"
+
FILES_${PN} = "${prefix}"
FILES_${PN}-dbg += "${prefix}/.debug \
${prefix}/bin/.debug \
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 86a1923..c8303ae 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -185,7 +185,7 @@ kernel_do_install() {
}
sysroot_stage_all_append() {
- sysroot_stage_dir ${D}/kernel ${SYSROOT_DESTDIR}${STAGING_KERNEL_DIR}
+ sysroot_stage_dir ${D}/kernel ${SYSROOT_DESTDIR}/kernel
}
diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
index 3ca9d62..4355db2 100644
--- a/meta/classes/native.bbclass
+++ b/meta/classes/native.bbclass
@@ -63,6 +63,8 @@ exec_prefix = "${STAGING_DIR_NATIVE}${prefix_native}"
# Since we actually install these into situ there is no staging prefix
STAGING_DIR_HOST = ""
STAGING_DIR_TARGET = ""
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_NATIVE}"
+
SHLIBSDIR = "${STAGING_DIR_NATIVE}/shlibs"
PKG_CONFIG_DIR = "${libdir}/pkgconfig"
diff --git a/meta/classes/nativesdk.bbclass b/meta/classes/nativesdk.bbclass
index 154bd82..7ed5ca5 100644
--- a/meta/classes/nativesdk.bbclass
+++ b/meta/classes/nativesdk.bbclass
@@ -43,6 +43,8 @@ base_prefix = "${SDKPATHNATIVE}"
prefix = "${SDKPATHNATIVE}${prefix_nativesdk}"
exec_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}"
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}"
+
FILES_${PN} += "${prefix}"
FILES_${PN}-dbg += "${prefix}/.debug \
${prefix}/bin/.debug \
@@ -78,5 +80,3 @@ python __anonymous () {
bb.data.setVar("PROVIDES", provides, d)
bb.data.setVar("OVERRIDES", bb.data.getVar("OVERRIDES", d, False) + ":virtclass-nativesdk", d)
}
-
-
diff --git a/meta/classes/siteconfig.bbclass b/meta/classes/siteconfig.bbclass
index 37d910e..e7cc9ae 100644
--- a/meta/classes/siteconfig.bbclass
+++ b/meta/classes/siteconfig.bbclass
@@ -20,8 +20,9 @@ siteconfig_do_siteconfig_gencache () {
sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \
-e "/ac_cv_type_/p" -e "/ac_cv_header_/p" -e "/ac_cv_func_/p" \
< ${PN}_cache > ${PN}_config
- mkdir -p ${SYSROOT_DESTDIR}${SITECONFIG_SYSROOTCACHE}
- cp ${PN}_config ${SYSROOT_DESTDIR}${SITECONFIG_SYSROOTCACHE}
+ mkdir -p ${SYSROOT_DESTDIR}${datadir}/${TARGET_SYS}_config_site.d
+ cp ${PN}_config ${SYSROOT_DESTDIR}${datadir}/${TARGET_SYS}_config_site.d
+
}
do_populate_sysroot[sstate-interceptfuncs] += "do_siteconfig "
diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 8432565..3a48d27 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -45,25 +45,25 @@ sysroot_stage_dirs() {
from="$1"
to="$2"
- sysroot_stage_dir $from${includedir} $to${STAGING_INCDIR}
+ sysroot_stage_dir $from${includedir} $to${includedir}
if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then
- sysroot_stage_dir $from${bindir} $to${STAGING_DIR_HOST}${bindir}
- sysroot_stage_dir $from${sbindir} $to${STAGING_DIR_HOST}${sbindir}
- sysroot_stage_dir $from${base_bindir} $to${STAGING_DIR_HOST}${base_bindir}
- sysroot_stage_dir $from${base_sbindir} $to${STAGING_DIR_HOST}${base_sbindir}
- sysroot_stage_dir $from${libexecdir} $to${STAGING_DIR_HOST}${libexecdir}
- sysroot_stage_dir $from${sysconfdir} $to${STAGING_DIR_HOST}${sysconfdir}
- sysroot_stage_dir $from${localstatedir} $to${STAGING_DIR_HOST}${localstatedir}
+ sysroot_stage_dir $from${bindir} $to${bindir}
+ sysroot_stage_dir $from${sbindir} $to${sbindir}
+ sysroot_stage_dir $from${base_bindir} $to${base_bindir}
+ sysroot_stage_dir $from${base_sbindir} $to${base_sbindir}
+ sysroot_stage_dir $from${libexecdir} $to${libexecdir}
+ sysroot_stage_dir $from${sysconfdir} $to${sysconfdir}
+ sysroot_stage_dir $from${localstatedir} $to${localstatedir}
fi
if [ -d $from${libdir} ]
then
- sysroot_stage_libdir $from/${libdir} $to${STAGING_LIBDIR}
+ sysroot_stage_libdir $from/${libdir} $to${libdir}
fi
if [ -d $from${base_libdir} ]
then
- sysroot_stage_libdir $from${base_libdir} $to${STAGING_DIR_HOST}${base_libdir}
+ sysroot_stage_libdir $from${base_libdir} $to${base_libdir}
fi
- sysroot_stage_dir $from${datadir} $to${STAGING_DATADIR}
+ sysroot_stage_dir $from${datadir} $to${datadir}
}
sysroot_stage_all() {
@@ -76,6 +76,7 @@ do_populate_sysroot[dirs] = "${STAGING_DIR_TARGET}/${bindir} ${STAGING_DIR_TARGE
${STAGING_INCDIR_NATIVE} \
${STAGING_DATADIR} \
${SYSROOT_DESTDIR}${STAGING_DIR_TARGET} \
+ ${SYSROOT_DESTDIR}${base_prefix} \
${S} ${B}"
# Could be compile but populate_sysroot and do_install shouldn't run at the same time
@@ -103,8 +104,8 @@ python do_populate_sysroot () {
SSTATETASKS += "do_populate_sysroot"
do_populate_sysroot[sstate-name] = "populate-sysroot"
-do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR}"
-do_populate_sysroot[sstate-outputdirs] = "${TMPDIR}/sysroots"
+do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}"
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
python do_populate_sysroot_setscene () {
sstate_setscene(d)
diff --git a/meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb b/meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb
index 9beb46e..da9fc92 100644
--- a/meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb
+++ b/meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb
@@ -23,7 +23,7 @@ FILES_${PN}-dev += "${bindir}/gupnp-binding-tool"
SYSROOT_PREPROCESS_FUNCS += "gupnp_sysroot_preprocess"
gupnp_sysroot_preprocess () {
- install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
- install -m 755 ${D}${bindir}/gupnp-binding-tool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
+ install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/
+ install -m 755 ${D}${bindir}/gupnp-binding-tool ${SYSROOT_DESTDIR}${bindir}/crossscripts/
}
diff --git a/meta/recipes-devtools/gcc/gcc-cross-intermediate.inc b/meta/recipes-devtools/gcc/gcc-cross-intermediate.inc
index 72a4241..22bef73 100644
--- a/meta/recipes-devtools/gcc/gcc-cross-intermediate.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross-intermediate.inc
@@ -17,6 +17,9 @@ EXTRA_OECONF = "--with-local-prefix=${STAGING_DIR_TARGET}${target_prefix} \
${EXTRA_OECONF_INTERMEDIATE} \
${@get_gcc_fpu_setting(bb, d)}"
+do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR_HOST} ${SYSROOT_DESTDIR}/${STAGING_DIR_TARGET}"
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST} ${STAGING_DIR_TARGET}"
+
do_compile () {
oe_runmake
}
diff --git a/meta/recipes-devtools/libtool/libtool-cross_2.4.bb b/meta/recipes-devtools/libtool/libtool-cross_2.4.bb
index 6ea96a5..09a8693 100644
--- a/meta/recipes-devtools/libtool/libtool-cross_2.4.bb
+++ b/meta/recipes-devtools/libtool/libtool-cross_2.4.bb
@@ -29,6 +29,6 @@ do_install () {
SYSROOT_PREPROCESS_FUNCS += "libtoolcross_sysroot_preprocess"
libtoolcross_sysroot_preprocess () {
- install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
- install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/${HOST_SYS}-libtool
+ install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts
+ install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${bindir}/crossscripts/${HOST_SYS}-libtool
}
diff --git a/meta/recipes-devtools/libtool/libtool-nativesdk_2.4.bb b/meta/recipes-devtools/libtool/libtool-nativesdk_2.4.bb
index c9f24a9..c17023a 100644
--- a/meta/recipes-devtools/libtool/libtool-nativesdk_2.4.bb
+++ b/meta/recipes-devtools/libtool/libtool-nativesdk_2.4.bb
@@ -22,6 +22,6 @@ do_install () {
SYSROOT_PREPROCESS_FUNCS += "libtoolnativesdk_sysroot_preprocess"
libtoolnativesdk_sysroot_preprocess () {
- install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
- install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/${HOST_SYS}-libtool
+ install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/
+ install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${bindir}/crossscripts/${HOST_SYS}-libtool
}
diff --git a/meta/recipes-devtools/libtool/libtool_2.4.bb b/meta/recipes-devtools/libtool/libtool_2.4.bb
index b5ec33e..3f73732 100644
--- a/meta/recipes-devtools/libtool/libtool_2.4.bb
+++ b/meta/recipes-devtools/libtool/libtool_2.4.bb
@@ -26,9 +26,9 @@ SYSROOT_PREPROCESS_FUNCS += "libtool_sysroot_preprocess"
libtool_sysroot_preprocess () {
if [ "${PN}" == "libtool" ]; then
- rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${bindir}/*
- rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/aclocal/*
- rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/libtool/config/*
+ rm -rf ${SYSROOT_DESTDIR}${bindir}/*
+ rm -rf ${SYSROOT_DESTDIR}${datadir}/aclocal/*
+ rm -rf ${SYSROOT_DESTDIR}${datadir}/libtool/config/*
fi
}
diff --git a/meta/recipes-gnome/gtk+/gtk+.inc b/meta/recipes-gnome/gtk+/gtk+.inc
index 425a21e..bd6fb19 100644
--- a/meta/recipes-gnome/gtk+/gtk+.inc
+++ b/meta/recipes-gnome/gtk+/gtk+.inc
@@ -64,8 +64,8 @@ SYSROOT_PREPROCESS_FUNCS += "gtk_sysroot_preprocess"
gtk_sysroot_preprocess () {
if [ -e ${D}${bindir}/gtk-builder-convert ]; then
- install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
- install -m 755 ${D}${bindir}/gtk-builder-convert ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
+ install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/
+ install -m 755 ${D}${bindir}/gtk-builder-convert ${SYSROOT_DESTDIR}${bindir}/crossscripts/
fi
}
diff --git a/meta/recipes-support/apr/apr_1.3.3.bb b/meta/recipes-support/apr/apr_1.3.3.bb
index 26cc01f..fabaae4 100644
--- a/meta/recipes-support/apr/apr_1.3.3.bb
+++ b/meta/recipes-support/apr/apr_1.3.3.bb
@@ -24,7 +24,7 @@ do_configure_prepend() {
SYSROOT_PREPROCESS_FUNCS += "apr_sysroot_preprocess"
apr_sysroot_preprocess () {
- d=${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/apr
+ d=${SYSROOT_DESTDIR}${datadir}/apr
install -d $d/
cp ${S}/build/apr_rules.mk $d/
sed -i s,apr_builddir=.*,apr_builddir=,g $d/apr_rules.mk
--
1.6.3.3
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-08 15:52 [PATCH 0/3][RFC v2] Machine specific sysroot implementation Dongxiao Xu
2011-01-08 15:52 ` [PATCH 1/3] staging: Use relative path in sysroot-destdir for target recipes Dongxiao Xu
@ 2011-01-08 15:53 ` Dongxiao Xu
2011-01-09 22:41 ` Richard Purdie
2011-01-08 15:53 ` [PATCH 3/3] bitbake: machine specific sysroots implementation Dongxiao Xu
2 siblings, 1 reply; 10+ messages in thread
From: Dongxiao Xu @ 2011-01-08 15:53 UTC (permalink / raw)
To: poky
For certain tasks, we need additional information in build stamp file
except the task name and file name. stamp-extra-info is introduced as
an flag adding to the end of stamp file name.
We also implement a blacklist, in which extra stamp information will
not be added.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
bitbake/lib/bb/siggen.py | 14 +++++++++++++-
1 files changed, 13 insertions(+), 1 deletions(-)
diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
index 4dc09b3..6c5a227 100644
--- a/bitbake/lib/bb/siggen.py
+++ b/bitbake/lib/bb/siggen.py
@@ -43,7 +43,17 @@ class SignatureGenerator(object):
return
def stampfile(self, stampbase, file_name, taskname):
- return "%s.%s" % (stampbase, taskname)
+ stamp_extra_info = (self.data.getVarFlag(taskname, 'stamp-extra-info', True) or self.data.getVar('BB_STAMP_EXTRA', True) or "")
+
+ if self.stampblacklist:
+ self.sbl = re.compile(self.stampblacklist)
+ else:
+ self.sbl = None
+
+ if self.sbl and self.sbl.search(file_name):
+ return "%s.%s" % (stampbase, taskname)
+
+ return ("%s.%s.%s" % (stampbase, taskname, stamp_extra_info)).rstrip('.')
class SignatureGeneratorBasic(SignatureGenerator):
"""
@@ -59,6 +69,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
self.lookupcache = {}
self.basewhitelist = (data.getVar("BB_HASHBASE_WHITELIST", True) or "").split()
self.taskwhitelist = data.getVar("BB_HASHTASK_WHITELIST", True) or None
+ self.stampblacklist = data.getVar("BB_STAMPTASK_BLACKLIST", True) or None
+ self.data = data
if self.taskwhitelist:
self.twl = re.compile(self.taskwhitelist)
--
1.6.3.3
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/3] bitbake: machine specific sysroots implementation
2011-01-08 15:52 [PATCH 0/3][RFC v2] Machine specific sysroot implementation Dongxiao Xu
2011-01-08 15:52 ` [PATCH 1/3] staging: Use relative path in sysroot-destdir for target recipes Dongxiao Xu
2011-01-08 15:53 ` [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file Dongxiao Xu
@ 2011-01-08 15:53 ` Dongxiao Xu
2 siblings, 0 replies; 10+ messages in thread
From: Dongxiao Xu @ 2011-01-08 15:53 UTC (permalink / raw)
To: poky
This commit changes the sysroots path to be machine specific.
Changes includes:
1) STAGING_DIR_TARGET and STRAGING_DIR_HOST points to machine specific
paths.
2) task stamp files. Adding ${MACHINE} info into stamp files for
do_populate_sysroots and do_package tasks. Add a BB_STAMPTASK_BLACKLIST
to keep native, nativesdk, crosssdk, and cross-canadian stamp unchanged.
3) siteconfig path. Separate the site config path for different machines
to avoid one machine adopting the cache file of another machine.
4) sstate stamp file. Add machine name to sstate stamp file since some
prebuild result could not be shared after sysroots are implemented as
machine spedific. Keep native, nativesdk, crosssdk, and cross-canadian
unchanged.
5) toolchain scripts. Change the environment path to point to machine
specific sysroots in toolchain scripts bbclass.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
meta/classes/cross-canadian.bbclass | 1 -
meta/classes/cross.bbclass | 5 +++++
meta/classes/nativesdk.bbclass | 2 ++
meta/classes/package.bbclass | 1 +
meta/classes/siteconfig.bbclass | 6 +++---
meta/classes/sstate.bbclass | 6 ++++--
meta/classes/staging.bbclass | 1 +
meta/classes/toolchain-scripts.bbclass | 4 ++--
meta/conf/bitbake.conf | 11 ++++++-----
9 files changed, 24 insertions(+), 13 deletions(-)
diff --git a/meta/classes/cross-canadian.bbclass b/meta/classes/cross-canadian.bbclass
index 9897105..5b36533 100644
--- a/meta/classes/cross-canadian.bbclass
+++ b/meta/classes/cross-canadian.bbclass
@@ -30,7 +30,6 @@ MULTIMACH_TARGET_SYS = "${MULTIMACH_ARCH}${HOST_VENDOR}-${HOST_OS}"
INHIBIT_DEFAULT_DEPS = "1"
STAGING_DIR_HOST = "${STAGING_DIR}/${HOST_SYS}-nativesdk"
-STAGING_DIR_TARGET = "${STAGING_DIR}/${OLD_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR}/${HOST_ARCH}-nativesdk${HOST_VENDOR}-${HOST_OS}"
diff --git a/meta/classes/cross.bbclass b/meta/classes/cross.bbclass
index 7c20be0..085ce21 100644
--- a/meta/classes/cross.bbclass
+++ b/meta/classes/cross.bbclass
@@ -20,6 +20,11 @@ HOST_OS = "${BUILD_OS}"
HOST_PREFIX = "${BUILD_PREFIX}"
HOST_CC_ARCH = "${BUILD_CC_ARCH}"
+STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}"
+
+export PKG_CONFIG_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}${libdir}/pkgconfig"
+export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
+
CPPFLAGS = "${BUILD_CPPFLAGS}"
CFLAGS = "${BUILD_CFLAGS}"
CXXFLAGS = "${BUILD_CFLAGS}"
diff --git a/meta/classes/nativesdk.bbclass b/meta/classes/nativesdk.bbclass
index 7ed5ca5..6b689c4 100644
--- a/meta/classes/nativesdk.bbclass
+++ b/meta/classes/nativesdk.bbclass
@@ -19,6 +19,8 @@ python () {
#STAGING_DIR_HOST = "${STAGING_DIR}/${HOST_SYS}-nativesdk"
#STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}-nativesdk"
+STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}"
+STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}"
HOST_ARCH = "${SDK_ARCH}"
HOST_VENDOR = "${SDK_VENDOR}"
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index d39c694..7e901ec 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1091,6 +1091,7 @@ do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
do_package[sstate-lockfile] = "${PACKAGELOCK}"
+do_package[stamp-extra-info] = "${MACHINE}"
do_package_setscene[dirs] = "${STAGING_DIR}"
python do_package_setscene () {
diff --git a/meta/classes/siteconfig.bbclass b/meta/classes/siteconfig.bbclass
index e7cc9ae..c037bb5 100644
--- a/meta/classes/siteconfig.bbclass
+++ b/meta/classes/siteconfig.bbclass
@@ -11,10 +11,10 @@ python siteconfig_do_siteconfig () {
}
siteconfig_do_siteconfig_gencache () {
- mkdir -p ${WORKDIR}/site_config
+ mkdir -p ${WORKDIR}/site_config_${MACHINE}
gen-site-config ${FILE_DIRNAME}/site_config \
- >${WORKDIR}/site_config/configure.ac
- cd ${WORKDIR}/site_config
+ >${WORKDIR}/site_config_${MACHINE}/configure.ac
+ cd ${WORKDIR}/site_config_${MACHINE}
autoconf
CONFIG_SITE="" ./configure ${CONFIGUREOPTS} --cache-file ${PN}_cache
sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index b6e6c92..1a9805a 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -5,7 +5,7 @@ SSTATE_MANFILEBASE = "${SSTATE_MANIFESTS}/manifest-${SSTATE_PKGARCH}-"
SSTATE_MANFILEPREFIX = "${SSTATE_MANFILEBASE}${PN}"
-SSTATE_PKGARCH = "${MULTIMACH_ARCH}"
+SSTATE_PKGARCH = "${MULTIMACH_ARCH}_${MACHINE}"
SSTATE_PKGSPEC = "sstate-${PN}-${MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}-${PV}-${PR}-${SSTATE_PKGARCH}-${SSTATE_VERSION}-"
SSTATE_PKGNAME = "${SSTATE_PKGSPEC}${BB_TASKHASH}"
SSTATE_PKG = "${SSTATE_DIR}/${SSTATE_PKGNAME}"
@@ -17,7 +17,9 @@ BB_HASHFILENAME = "${SSTATE_PKGNAME}"
python () {
if bb.data.inherits_class('native', d):
bb.data.setVar('SSTATE_PKGARCH', bb.data.getVar('BUILD_ARCH', d), d)
- elif bb.data.inherits_class('cross', d) or bb.data.inherits_class('crosssdk', d):
+ elif bb.data.inherits_class('cross', d):
+ bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}_${MACHINE}", d), d)
+ elif bb.data.inherits_class('crosssdk', d):
bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
elif bb.data.inherits_class('nativesdk', d):
bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${SDK_ARCH}", d), d)
diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 3a48d27..a72935d 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -106,6 +106,7 @@ SSTATETASKS += "do_populate_sysroot"
do_populate_sysroot[sstate-name] = "populate-sysroot"
do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}"
do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
+do_populate_sysroot[stamp-extra-info] = "${MACHINE}"
python do_populate_sysroot_setscene () {
sstate_setscene(d)
diff --git a/meta/classes/toolchain-scripts.bbclass b/meta/classes/toolchain-scripts.bbclass
index a5b2bd1..462f3d8 100644
--- a/meta/classes/toolchain-scripts.bbclass
+++ b/meta/classes/toolchain-scripts.bbclass
@@ -49,8 +49,8 @@ toolchain_create_tree_env_script () {
echo 'export TARGET_PREFIX=${TARGET_PREFIX}' >> $script
echo 'export CONFIGURE_FLAGS="--target=${TARGET_SYS} --host=${TARGET_SYS} --build=${BUILD_SYS}"' >> $script
if [ "${TARGET_OS}" = "darwin8" ]; then
- echo 'export TARGET_CFLAGS="-I${STAGING_DIR}${TARGET_SYS}${includedir}"' >> $script
- echo 'export TARGET_LDFLAGS="-L${STAGING_DIR}${TARGET_SYS}${libdir}"' >> $script
+ echo 'export TARGET_CFLAGS="-I${STAGING_DIR}${MACHINE}${includedir}"' >> $script
+ echo 'export TARGET_LDFLAGS="-L${STAGING_DIR}${MACHINE}${libdir}"' >> $script
# Workaround darwin toolchain sysroot path problems
cd ${SDK_OUTPUT}${SDKTARGETSYSROOT}/usr
ln -s /usr/local local
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 30fcc79..cf9182b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -276,7 +276,7 @@ STAGING_DATADIR_NATIVE = "${STAGING_DIR_NATIVE}${datadir_native}"
# This should really be MULTIMACH_HOST_SYS but that breaks "all" and machine
# specific packages - hack around it for now.
-STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}"
+STAGING_DIR_HOST = "${STAGING_DIR}/${MACHINE}"
STAGING_BINDIR = "${STAGING_DIR_HOST}${bindir}"
STAGING_LIBDIR = "${STAGING_DIR_HOST}${libdir}"
STAGING_INCDIR = "${STAGING_DIR_HOST}${includedir}"
@@ -288,7 +288,7 @@ STAGING_PYDIR = "${STAGING_DIR}/lib/python2.4"
# This should really be MULTIMACH_TARGET_SYS but that breaks "all" and machine
# specific packages - hack around it for now.
-STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}"
+STAGING_DIR_TARGET = "${STAGING_DIR}/${MACHINE}"
# Setting DEPLOY_DIR outside of TMPDIR is helpful, when you are using
# packaged staging and/or multimachine.
@@ -311,7 +311,7 @@ SDKPATHNATIVE = "${SDKPATH}/sysroots/${SDK_SYS}"
##################################################################
OLDEST_KERNEL = "2.4.0"
-STAGING_KERNEL_DIR = "${STAGING_DIR}/${MULTIMACH_TARGET_SYS}/kernel"
+STAGING_KERNEL_DIR = "${STAGING_DIR}/${MACHINE}/kernel"
##################################################################
# Specific image creation and rootfs population info.
@@ -595,10 +595,10 @@ SLOT = "0"
# Other
-export PKG_CONFIG_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}${libdir}/pkgconfig"
+export PKG_CONFIG_DIR = "${STAGING_DIR}/${MACHINE}/${libdir}/pkgconfig"
export PKG_CONFIG_PATH = "${PKG_CONFIG_DIR}:${STAGING_DATADIR}/pkgconfig"
export PKG_CONFIG_LIBDIR = "${PKG_CONFIG_DIR}"
-export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
+export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${MACHINE}"
export PKG_CONFIG_DISABLE_UNINSTALLED = "yes"
export QMAKE_MKSPEC_PATH = "${STAGING_DATADIR_NATIVE}/qmake"
@@ -731,3 +731,4 @@ BB_SIGNATURE_HANDLER ?= "basic"
BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
+BB_STAMPTASK_BLACKLIST ?= "(.*-native_*|.*-nativesdk_*|^virtual:native:*|^virtual:nativesdk:*|.*-crosssdk_*|.*-cross-canadian-*)"
--
1.6.3.3
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-08 15:53 ` [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file Dongxiao Xu
@ 2011-01-09 22:41 ` Richard Purdie
2011-01-10 3:07 ` Xu, Dongxiao
2011-01-10 8:44 ` Xu, Dongxiao
0 siblings, 2 replies; 10+ messages in thread
From: Richard Purdie @ 2011-01-09 22:41 UTC (permalink / raw)
To: Dongxiao Xu; +Cc: poky
On Sat, 2011-01-08 at 23:53 +0800, Dongxiao Xu wrote:
> For certain tasks, we need additional information in build stamp file
> except the task name and file name. stamp-extra-info is introduced as
> an flag adding to the end of stamp file name.
>
> We also implement a blacklist, in which extra stamp information will
> not be added.
To be honest I much preferred the original version of this patch. The
problem with this approach is that the data dictionary you're storing in
the siggen code is just the global config data dict. This means that if
any recipe customises BB_STAMP_EXTRA or the stamp-extra-into flag, those
changes will not be seen by this code. This is why your original of
extracting the flag at parse time and caching it was better.
I appreciate the code in those areas has changed but the original patch
should translate relatively easily.
Also note that re.compile is expensive so you'd do that at init time,
not for each stampfile.
Cheers,
Richard
> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> ---
> bitbake/lib/bb/siggen.py | 14 +++++++++++++-
> 1 files changed, 13 insertions(+), 1 deletions(-)
>
> diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
> index 4dc09b3..6c5a227 100644
> --- a/bitbake/lib/bb/siggen.py
> +++ b/bitbake/lib/bb/siggen.py
> @@ -43,7 +43,17 @@ class SignatureGenerator(object):
> return
>
> def stampfile(self, stampbase, file_name, taskname):
> - return "%s.%s" % (stampbase, taskname)
> + stamp_extra_info = (self.data.getVarFlag(taskname, 'stamp-extra-info', True) or self.data.getVar('BB_STAMP_EXTRA', True) or "")
> +
> + if self.stampblacklist:
> + self.sbl = re.compile(self.stampblacklist)
> + else:
> + self.sbl = None
> +
> + if self.sbl and self.sbl.search(file_name):
> + return "%s.%s" % (stampbase, taskname)
> +
> + return ("%s.%s.%s" % (stampbase, taskname, stamp_extra_info)).rstrip('.')
>
> class SignatureGeneratorBasic(SignatureGenerator):
> """
> @@ -59,6 +69,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
> self.lookupcache = {}
> self.basewhitelist = (data.getVar("BB_HASHBASE_WHITELIST", True) or "").split()
> self.taskwhitelist = data.getVar("BB_HASHTASK_WHITELIST", True) or None
> + self.stampblacklist = data.getVar("BB_STAMPTASK_BLACKLIST", True) or None
> + self.data = data
>
> if self.taskwhitelist:
> self.twl = re.compile(self.taskwhitelist)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-09 22:41 ` Richard Purdie
@ 2011-01-10 3:07 ` Xu, Dongxiao
2011-01-10 8:44 ` Xu, Dongxiao
1 sibling, 0 replies; 10+ messages in thread
From: Xu, Dongxiao @ 2011-01-10 3:07 UTC (permalink / raw)
To: Richard Purdie; +Cc: poky
Richard Purdie wrote:
> On Sat, 2011-01-08 at 23:53 +0800, Dongxiao Xu wrote:
>> For certain tasks, we need additional information in build stamp file
>> except the task name and file name. stamp-extra-info is introduced as
>> an flag adding to the end of stamp file name.
>>
>> We also implement a blacklist, in which extra stamp information will
>> not be added.
>
> To be honest I much preferred the original version of this patch. The
> problem with this approach is that the data dictionary you're storing
> in the siggen code is just the global config data dict. This means
> that if any recipe customises BB_STAMP_EXTRA or the stamp-extra-into
> flag, those changes will not be seen by this code. This is why your
> original of extracting the flag at parse time and caching it was
> better.
Hi Richard,
I used the new method to handle stamp in bitbake because I could not avoid adding extra info for native stamps with previous implementation.
I noticed an strange issue with the previous code that, if I set:
do_populate_sysroot[stamp-extra-info] = "${MACHINE}" in staging.bbclass.
and set:
do_populate_sysroot[stamp-extra-info] = "" in native.bbclass.
The above code will not work. For native recipes, what stored in the data cache was tagged with the "${MACHINE}" info.
Is it because the cache data parsing is handled before the inherit of native bbclass?
Thanks,
Dongxiao
>
> I appreciate the code in those areas has changed but the original
> patch should translate relatively easily.
>
> Also note that re.compile is expensive so you'd do that at init time,
> not for each stampfile.
>
> Cheers,
>
> Richard
>
>> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
>> ---
>> bitbake/lib/bb/siggen.py | 14 +++++++++++++-
>> 1 files changed, 13 insertions(+), 1 deletions(-)
>>
>> diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
>> index 4dc09b3..6c5a227 100644 --- a/bitbake/lib/bb/siggen.py
>> +++ b/bitbake/lib/bb/siggen.py
>> @@ -43,7 +43,17 @@ class SignatureGenerator(object): return
>>
>> def stampfile(self, stampbase, file_name, taskname):
>> - return "%s.%s" % (stampbase, taskname)
>> + stamp_extra_info = (self.data.getVarFlag(taskname,
>> + 'stamp-extra-info', True) or self.data.getVar('BB_STAMP_EXTRA', +
>> True) or "") +
>> + if self.stampblacklist:
>> + self.sbl = re.compile(self.stampblacklist) +
>> else: + self.sbl = None
>> +
>> + if self.sbl and self.sbl.search(file_name):
>> + return "%s.%s" % (stampbase, taskname) +
>> + return ("%s.%s.%s" % (stampbase, taskname,
>> + stamp_extra_info)).rstrip('.')
>>
>> class SignatureGeneratorBasic(SignatureGenerator): """
>> @@ -59,6 +69,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
>> self.lookupcache = {}
>> self.basewhitelist = (data.getVar("BB_HASHBASE_WHITELIST",
>> True) or "").split() self.taskwhitelist =
>> data.getVar("BB_HASHTASK_WHITELIST",
>> True) or None
>> + self.stampblacklist = data.getVar("BB_STAMPTASK_BLACKLIST",
>> True) or None + self.data = data
>>
>> if self.taskwhitelist:
>> self.twl = re.compile(self.taskwhitelist)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-09 22:41 ` Richard Purdie
2011-01-10 3:07 ` Xu, Dongxiao
@ 2011-01-10 8:44 ` Xu, Dongxiao
1 sibling, 0 replies; 10+ messages in thread
From: Xu, Dongxiao @ 2011-01-10 8:44 UTC (permalink / raw)
To: Xu, Dongxiao, Richard Purdie; +Cc: poky
Xu, Dongxiao wrote:
> Richard Purdie wrote:
>> On Sat, 2011-01-08 at 23:53 +0800, Dongxiao Xu wrote:
>>> For certain tasks, we need additional information in build stamp
>>> file except the task name and file name. stamp-extra-info is
>>> introduced as an flag adding to the end of stamp file name.
>>>
>>> We also implement a blacklist, in which extra stamp information will
>>> not be added.
>>
>> To be honest I much preferred the original version of this patch. The
>> problem with this approach is that the data dictionary you're storing
>> in the siggen code is just the global config data dict. This means
>> that if any recipe customises BB_STAMP_EXTRA or the stamp-extra-into
>> flag, those changes will not be seen by this code. This is why your
>> original of extracting the flag at parse time and caching it was
>> better.
>
> Hi Richard,
>
> I used the new method to handle stamp in bitbake because I could not
> avoid adding extra info for native stamps with previous
> implementation.
>
> I noticed an strange issue with the previous code that, if I set:
>
> do_populate_sysroot[stamp-extra-info] = "${MACHINE}" in
> staging.bbclass.
>
> and set:
>
> do_populate_sysroot[stamp-extra-info] = "" in native.bbclass.
>
> The above code will not work. For native recipes, what stored in the
> data cache was tagged with the "${MACHINE}" info.
>
> Is it because the cache data parsing is handled before the inherit of
> native bbclass?
Hi Richard,
It seems that the above issue I met before disappeared when translated the original stamp extra info logic against latest bitbake.
I will send out a v3 patchset after the test is passed.
Thanks,
Dongxiao
>
> Thanks,
> Dongxiao
>
>>
>> I appreciate the code in those areas has changed but the original
>> patch should translate relatively easily.
>>
>> Also note that re.compile is expensive so you'd do that at init
>> time, not for each stampfile.
>>
>> Cheers,
>>
>> Richard
>>
>>> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
>>> ---
>>> bitbake/lib/bb/siggen.py | 14 +++++++++++++-
>>> 1 files changed, 13 insertions(+), 1 deletions(-)
>>>
>>> diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
>>> index 4dc09b3..6c5a227 100644 --- a/bitbake/lib/bb/siggen.py
>>> +++ b/bitbake/lib/bb/siggen.py
>>> @@ -43,7 +43,17 @@ class SignatureGenerator(object): return
>>>
>>> def stampfile(self, stampbase, file_name, taskname):
>>> - return "%s.%s" % (stampbase, taskname)
>>> + stamp_extra_info = (self.data.getVarFlag(taskname,
>>> + 'stamp-extra-info', True) or self.data.getVar('BB_STAMP_EXTRA', +
>>> True) or "") + + if self.stampblacklist:
>>> + self.sbl = re.compile(self.stampblacklist) +
>>> else: + self.sbl = None
>>> +
>>> + if self.sbl and self.sbl.search(file_name):
>>> + return "%s.%s" % (stampbase, taskname) +
>>> + return ("%s.%s.%s" % (stampbase, taskname,
>>> + stamp_extra_info)).rstrip('.')
>>>
>>> class SignatureGeneratorBasic(SignatureGenerator): """
>>> @@ -59,6 +69,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
>>> self.lookupcache = {}
>>> self.basewhitelist = (data.getVar("BB_HASHBASE_WHITELIST",
>>> True) or "").split() self.taskwhitelist =
>>> data.getVar("BB_HASHTASK_WHITELIST",
>>> True) or None
>>> + self.stampblacklist = data.getVar("BB_STAMPTASK_BLACKLIST",
>>> True) or None + self.data = data
>>>
>>> if self.taskwhitelist:
>>> self.twl = re.compile(self.taskwhitelist)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-11 12:11 ` Joshua Lock
@ 2011-01-12 11:48 ` Richard Purdie
0 siblings, 0 replies; 10+ messages in thread
From: Richard Purdie @ 2011-01-12 11:48 UTC (permalink / raw)
To: Joshua Lock; +Cc: poky
On Tue, 2011-01-11 at 12:11 +0000, Joshua Lock wrote:
> Hi Dongxiao,
>
> I've done a quick style review of this patch, comments inline.
>
> On Tue, 2011-01-11 at 14:18 +0800, Dongxiao Xu wrote:
> > For certain tasks, we need additional information in build stamp file
> > except the task name and file name. stamp-extra-info is introduced as
> > an flag adding to the end of stamp file name.
> >
> > Besides, if we need to add common flags for tasks, we can set value for
> > macro BB_STAMP_EXTRA.
> >
> > Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
I talked with Dongxiao about this patch and we agreed the best approach
will be to make bb.build.stampfile() the main entry point so most users
will be converted to use that and then the datacache vs d ugliness can
be contained in one place (stamp_internal) which calls into siggen.
Josh is right about the 0/1 vs. False/True though. Values of 0/1 are
used in many places in the code but we're trying to clean it up as we
update things.
Cheers,
Richard
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-11 6:18 ` [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file Dongxiao Xu
@ 2011-01-11 12:11 ` Joshua Lock
2011-01-12 11:48 ` Richard Purdie
0 siblings, 1 reply; 10+ messages in thread
From: Joshua Lock @ 2011-01-11 12:11 UTC (permalink / raw)
To: poky
Hi Dongxiao,
I've done a quick style review of this patch, comments inline.
On Tue, 2011-01-11 at 14:18 +0800, Dongxiao Xu wrote:
> For certain tasks, we need additional information in build stamp file
> except the task name and file name. stamp-extra-info is introduced as
> an flag adding to the end of stamp file name.
>
> Besides, if we need to add common flags for tasks, we can set value for
> macro BB_STAMP_EXTRA.
>
> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> ---
> bitbake/lib/bb/build.py | 10 +++-------
> bitbake/lib/bb/cache.py | 19 ++++++++++++++++++-
> bitbake/lib/bb/runqueue.py | 12 ++++++------
> bitbake/lib/bb/siggen.py | 20 ++++++++++++++++++--
> 4 files changed, 45 insertions(+), 16 deletions(-)
>
> diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
> index f127796..3b297b3 100644
> --- a/bitbake/lib/bb/build.py
> +++ b/bitbake/lib/bb/build.py
> @@ -383,15 +383,11 @@ def stamp_internal(taskname, d, file_name):
> When called in task context, d will be a data store, file_name will not be set
> """
> if file_name:
> - stamp = d.stamp[file_name]
> + is_dict = 0
> else:
> - stamp = d.getVar('STAMP', True)
> - file_name = d.getVar('BB_FILENAME', True)
> + is_dict = 1
>
> - if not stamp:
> - return
> -
> - stamp = bb.parse.siggen.stampfile(stamp, file_name, taskname)
> + stamp = bb.parse.siggen.stampfile(d, file_name, taskname, is_dict)
Nitpick, but could you please use True/False rather than 1/0 - IMHO it
makes the code more readable and the BitBake codebase has been gradually
moving in the direction of using the boolean types of late.
>
> bb.utils.mkdirhier(os.path.dirname(stamp))
>
> diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py
> index 9a2e2d5..2437d4a 100644
> --- a/bitbake/lib/bb/cache.py
> +++ b/bitbake/lib/bb/cache.py
> @@ -55,6 +55,8 @@ recipe_fields = (
> 'provides',
> 'task_deps',
> 'stamp',
> + 'stamp_extra',
> + 'stamp_extra_info',
> 'broken',
> 'not_world',
> 'skipped',
> @@ -102,6 +104,11 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
> for task in tasks)
>
> @classmethod
> + def stampvar(cls, var, tasks, stamp, metadata):
> + return dict((stamp + '.' + task, metadata.getVarFlag(task, var, True))
> + for task in tasks)
> +
> + @classmethod
> def getvar(cls, var, metadata):
> return metadata.getVar(var, True) or ''
>
> @@ -126,6 +133,7 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
> if not pn in packages:
> packages.append(pn)
>
> + stamp = cls.getvar('STAMP', metadata)
> return RecipeInfo(
> tasks = tasks,
> basetaskhashes = cls.taskvar('BB_BASEHASH', tasks, metadata),
> @@ -147,7 +155,9 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
> defaultpref = cls.intvar('DEFAULT_PREFERENCE', metadata),
> broken = cls.getvar('BROKEN', metadata),
> not_world = cls.getvar('EXCLUDE_FROM_WORLD', metadata),
> - stamp = cls.getvar('STAMP', metadata),
> + stamp = stamp,
> + stamp_extra = cls.getvar('BB_STAMP_EXTRA', metadata),
> + stamp_extra_info = cls.stampvar('stamp-extra-info', tasks, stamp, metadata),
> packages_dynamic = cls.listvar('PACKAGES_DYNAMIC', metadata),
> depends = cls.depvar('DEPENDS', metadata),
> provides = cls.depvar('PROVIDES', metadata),
> @@ -562,6 +572,8 @@ class CacheData(object):
> self.task_queues = {}
> self.task_deps = {}
> self.stamp = {}
> + self.stamp_extra = {}
> + self.stamp_extra_info = {}
> self.preferred = {}
> self.tasks = {}
> self.basetaskhash = {}
> @@ -577,12 +589,17 @@ class CacheData(object):
> self.bbfile_config_priorities = []
>
> def add_from_recipeinfo(self, fn, info):
> + tasks = info.tasks
> self.task_deps[fn] = info.task_deps
> self.pkg_fn[fn] = info.pn
> self.pkg_pn[info.pn].append(fn)
> self.pkg_pepvpr[fn] = (info.pe, info.pv, info.pr)
> self.pkg_dp[fn] = info.defaultpref
> self.stamp[fn] = info.stamp
> + self.stamp_extra[fn] = info.stamp_extra
> +
> + for task in tasks:
> + self.stamp_extra_info[self.stamp[fn] + '.' + task] = info.stamp_extra_info[self.stamp[fn] + '.' + task]
>
> provides = [info.pn]
> for provide in info.provides:
> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> index a465275..764ae7e 100644
> --- a/bitbake/lib/bb/runqueue.py
> +++ b/bitbake/lib/bb/runqueue.py
> @@ -794,7 +794,7 @@ class RunQueue:
> continue
> fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]]
> taskname = self.rqdata.runq_task[task]
> - stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
> + stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
False, rather than 0, here please.
> # If the stamp is missing its not current
> if not os.access(stampfile, os.F_OK):
> del unchecked[task]
> @@ -815,7 +815,7 @@ class RunQueue:
> if task in unchecked:
> fn = self.taskData.fn_index[self.rqdata.runq_fnid[task]]
> taskname = self.rqdata.runq_task[task]
> - stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
> + stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
And again.
> iscurrent = True
>
> t1 = os.stat(stampfile)[stat.ST_MTIME]
> @@ -823,7 +823,7 @@ class RunQueue:
> if iscurrent:
> fn2 = self.taskData.fn_index[self.rqdata.runq_fnid[dep]]
> taskname2 = self.rqdata.runq_task[dep]
> - stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
> + stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
And again
> if fn == fn2 or (fulldeptree and fn2 not in stampwhitelist):
> if dep in notcurrent:
> iscurrent = False
> @@ -875,7 +875,7 @@ class RunQueue:
> if taskname is None:
> taskname = self.rqdata.runq_task[task]
>
> - stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
> + stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
And again
>
> # If the stamp is missing its not current
> if not os.access(stampfile, os.F_OK):
> @@ -896,8 +896,8 @@ class RunQueue:
> if iscurrent:
> fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]]
> taskname2 = self.rqdata.runq_task[dep]
> - stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
> - stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2 + "_setscene")
> + stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
> + stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2 + "_setscene", 0)
And again, and again :-)
> t2 = get_timestamp(stampfile2)
> t3 = get_timestamp(stampfile3)
> if t3 and t3 > t2:
> diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
> index 4dc09b3..48fe6af 100644
> --- a/bitbake/lib/bb/siggen.py
> +++ b/bitbake/lib/bb/siggen.py
> @@ -42,8 +42,24 @@ class SignatureGenerator(object):
> def set_taskdata(self, hashes, deps):
> return
>
> - def stampfile(self, stampbase, file_name, taskname):
> - return "%s.%s" % (stampbase, taskname)
> + def stampfile(self, d, file_name, taskname, is_dict):
As you have a lot of calls to stampfile with is_dict set to False I'd
recommend you define the function with a default parameter of False for
is_dict:
def stampfile(self, d, file_name, taskname, is_dict=False):
that way you can avoid typing this parameter for most of the calls to
this function. 6 times, by my count.
> + if is_dict:
> + stamp = d.getVar('STAMP', True)
> + file_name = d.getVar('BB_FILENAME', True)
> + extra_info = taskname + '.' + (bb.data.getVarFlag(taskname, 'stamp-extra-info', d) or bb.data.getVar('BB_STAMP_EXTRA', d, True) or "")
> + extra_info = bb.data.expand(extra_info, d)
Haven't you already expanded this by passing True as the third argument
to getVar() above?
> + else:
> + stamp = d.stamp[file_name]
> + if stamp + "." + taskname in d.stamp_extra_info.keys():
> + extra_info = taskname + '.' + (d.stamp_extra_info[stamp + "." + taskname] or d.stamp_extra[file_name] or "")
> + else:
> + extra_info = taskname + '.' + (d.stamp_extra[file_name] or "")
> +
> + if not stamp:
> + return
> +
> + return "%s.%s" % (stamp, extra_info.rstrip('.'))
> +
>
> class SignatureGeneratorBasic(SignatureGenerator):
> """
Cheers,
Joshua
--
Joshua Lock
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file
2011-01-11 6:18 [PATCH 0/3][RFC] Machine specific sysroot implementation Dongxiao Xu
@ 2011-01-11 6:18 ` Dongxiao Xu
2011-01-11 12:11 ` Joshua Lock
0 siblings, 1 reply; 10+ messages in thread
From: Dongxiao Xu @ 2011-01-11 6:18 UTC (permalink / raw)
To: poky
For certain tasks, we need additional information in build stamp file
except the task name and file name. stamp-extra-info is introduced as
an flag adding to the end of stamp file name.
Besides, if we need to add common flags for tasks, we can set value for
macro BB_STAMP_EXTRA.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
bitbake/lib/bb/build.py | 10 +++-------
bitbake/lib/bb/cache.py | 19 ++++++++++++++++++-
bitbake/lib/bb/runqueue.py | 12 ++++++------
bitbake/lib/bb/siggen.py | 20 ++++++++++++++++++--
4 files changed, 45 insertions(+), 16 deletions(-)
diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
index f127796..3b297b3 100644
--- a/bitbake/lib/bb/build.py
+++ b/bitbake/lib/bb/build.py
@@ -383,15 +383,11 @@ def stamp_internal(taskname, d, file_name):
When called in task context, d will be a data store, file_name will not be set
"""
if file_name:
- stamp = d.stamp[file_name]
+ is_dict = 0
else:
- stamp = d.getVar('STAMP', True)
- file_name = d.getVar('BB_FILENAME', True)
+ is_dict = 1
- if not stamp:
- return
-
- stamp = bb.parse.siggen.stampfile(stamp, file_name, taskname)
+ stamp = bb.parse.siggen.stampfile(d, file_name, taskname, is_dict)
bb.utils.mkdirhier(os.path.dirname(stamp))
diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py
index 9a2e2d5..2437d4a 100644
--- a/bitbake/lib/bb/cache.py
+++ b/bitbake/lib/bb/cache.py
@@ -55,6 +55,8 @@ recipe_fields = (
'provides',
'task_deps',
'stamp',
+ 'stamp_extra',
+ 'stamp_extra_info',
'broken',
'not_world',
'skipped',
@@ -102,6 +104,11 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
for task in tasks)
@classmethod
+ def stampvar(cls, var, tasks, stamp, metadata):
+ return dict((stamp + '.' + task, metadata.getVarFlag(task, var, True))
+ for task in tasks)
+
+ @classmethod
def getvar(cls, var, metadata):
return metadata.getVar(var, True) or ''
@@ -126,6 +133,7 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
if not pn in packages:
packages.append(pn)
+ stamp = cls.getvar('STAMP', metadata)
return RecipeInfo(
tasks = tasks,
basetaskhashes = cls.taskvar('BB_BASEHASH', tasks, metadata),
@@ -147,7 +155,9 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
defaultpref = cls.intvar('DEFAULT_PREFERENCE', metadata),
broken = cls.getvar('BROKEN', metadata),
not_world = cls.getvar('EXCLUDE_FROM_WORLD', metadata),
- stamp = cls.getvar('STAMP', metadata),
+ stamp = stamp,
+ stamp_extra = cls.getvar('BB_STAMP_EXTRA', metadata),
+ stamp_extra_info = cls.stampvar('stamp-extra-info', tasks, stamp, metadata),
packages_dynamic = cls.listvar('PACKAGES_DYNAMIC', metadata),
depends = cls.depvar('DEPENDS', metadata),
provides = cls.depvar('PROVIDES', metadata),
@@ -562,6 +572,8 @@ class CacheData(object):
self.task_queues = {}
self.task_deps = {}
self.stamp = {}
+ self.stamp_extra = {}
+ self.stamp_extra_info = {}
self.preferred = {}
self.tasks = {}
self.basetaskhash = {}
@@ -577,12 +589,17 @@ class CacheData(object):
self.bbfile_config_priorities = []
def add_from_recipeinfo(self, fn, info):
+ tasks = info.tasks
self.task_deps[fn] = info.task_deps
self.pkg_fn[fn] = info.pn
self.pkg_pn[info.pn].append(fn)
self.pkg_pepvpr[fn] = (info.pe, info.pv, info.pr)
self.pkg_dp[fn] = info.defaultpref
self.stamp[fn] = info.stamp
+ self.stamp_extra[fn] = info.stamp_extra
+
+ for task in tasks:
+ self.stamp_extra_info[self.stamp[fn] + '.' + task] = info.stamp_extra_info[self.stamp[fn] + '.' + task]
provides = [info.pn]
for provide in info.provides:
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index a465275..764ae7e 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -794,7 +794,7 @@ class RunQueue:
continue
fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]]
taskname = self.rqdata.runq_task[task]
- stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+ stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
# If the stamp is missing its not current
if not os.access(stampfile, os.F_OK):
del unchecked[task]
@@ -815,7 +815,7 @@ class RunQueue:
if task in unchecked:
fn = self.taskData.fn_index[self.rqdata.runq_fnid[task]]
taskname = self.rqdata.runq_task[task]
- stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+ stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
iscurrent = True
t1 = os.stat(stampfile)[stat.ST_MTIME]
@@ -823,7 +823,7 @@ class RunQueue:
if iscurrent:
fn2 = self.taskData.fn_index[self.rqdata.runq_fnid[dep]]
taskname2 = self.rqdata.runq_task[dep]
- stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
+ stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
if fn == fn2 or (fulldeptree and fn2 not in stampwhitelist):
if dep in notcurrent:
iscurrent = False
@@ -875,7 +875,7 @@ class RunQueue:
if taskname is None:
taskname = self.rqdata.runq_task[task]
- stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+ stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
# If the stamp is missing its not current
if not os.access(stampfile, os.F_OK):
@@ -896,8 +896,8 @@ class RunQueue:
if iscurrent:
fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]]
taskname2 = self.rqdata.runq_task[dep]
- stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
- stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2 + "_setscene")
+ stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
+ stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2 + "_setscene", 0)
t2 = get_timestamp(stampfile2)
t3 = get_timestamp(stampfile3)
if t3 and t3 > t2:
diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
index 4dc09b3..48fe6af 100644
--- a/bitbake/lib/bb/siggen.py
+++ b/bitbake/lib/bb/siggen.py
@@ -42,8 +42,24 @@ class SignatureGenerator(object):
def set_taskdata(self, hashes, deps):
return
- def stampfile(self, stampbase, file_name, taskname):
- return "%s.%s" % (stampbase, taskname)
+ def stampfile(self, d, file_name, taskname, is_dict):
+ if is_dict:
+ stamp = d.getVar('STAMP', True)
+ file_name = d.getVar('BB_FILENAME', True)
+ extra_info = taskname + '.' + (bb.data.getVarFlag(taskname, 'stamp-extra-info', d) or bb.data.getVar('BB_STAMP_EXTRA', d, True) or "")
+ extra_info = bb.data.expand(extra_info, d)
+ else:
+ stamp = d.stamp[file_name]
+ if stamp + "." + taskname in d.stamp_extra_info.keys():
+ extra_info = taskname + '.' + (d.stamp_extra_info[stamp + "." + taskname] or d.stamp_extra[file_name] or "")
+ else:
+ extra_info = taskname + '.' + (d.stamp_extra[file_name] or "")
+
+ if not stamp:
+ return
+
+ return "%s.%s" % (stamp, extra_info.rstrip('.'))
+
class SignatureGeneratorBasic(SignatureGenerator):
"""
--
1.6.3.3
^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2011-01-12 11:49 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-08 15:52 [PATCH 0/3][RFC v2] Machine specific sysroot implementation Dongxiao Xu
2011-01-08 15:52 ` [PATCH 1/3] staging: Use relative path in sysroot-destdir for target recipes Dongxiao Xu
2011-01-08 15:53 ` [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file Dongxiao Xu
2011-01-09 22:41 ` Richard Purdie
2011-01-10 3:07 ` Xu, Dongxiao
2011-01-10 8:44 ` Xu, Dongxiao
2011-01-08 15:53 ` [PATCH 3/3] bitbake: machine specific sysroots implementation Dongxiao Xu
2011-01-11 6:18 [PATCH 0/3][RFC] Machine specific sysroot implementation Dongxiao Xu
2011-01-11 6:18 ` [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file Dongxiao Xu
2011-01-11 12:11 ` Joshua Lock
2011-01-12 11:48 ` Richard Purdie
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.