* [PATCH V2 0/3] Extensible SDK: 3 fixes
@ 2015-06-10 7:25 Chen Qi
2015-06-10 7:25 ` [PATCH V2 1/3] populate_sdk_ext: install the latest buildtools-tarball Chen Qi
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Chen Qi @ 2015-06-10 7:25 UTC (permalink / raw)
To: openembedded-core
Changes since V1:
*) Make use of bb.utils.edit_metadata.
*) Introduce a whitelist mechanism to allow exceptions.
The following changes since commit de6a26b95a7f7bd8f9dc47ab35d8b07ba671f4eb:
x264: use x86 over-ride instead of i586 (2015-06-08 17:32:46 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib ChenQi/ext-sdk-3-fixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/ext-sdk-3-fixes
Chen Qi (3):
populate_sdk_ext: install the latest buildtools-tarball
copy_buildsystem: make sure bitbake directory is copied
populate_sdk_ext: consider custom configuration in local.conf
meta/classes/populate_sdk_ext.bbclass | 21 ++++++++++++++++++++-
meta/lib/oe/copy_buildsystem.py | 7 +++----
2 files changed, 23 insertions(+), 5 deletions(-)
--
1.9.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH V2 1/3] populate_sdk_ext: install the latest buildtools-tarball
2015-06-10 7:25 [PATCH V2 0/3] Extensible SDK: 3 fixes Chen Qi
@ 2015-06-10 7:25 ` Chen Qi
2015-06-10 7:25 ` [PATCH V2 2/3] copy_buildsystem: make sure bitbake directory is copied Chen Qi
2015-06-10 7:25 ` [PATCH V2 3/3] populate_sdk_ext: consider custom configuration in local.conf Chen Qi
2 siblings, 0 replies; 5+ messages in thread
From: Chen Qi @ 2015-06-10 7:25 UTC (permalink / raw)
To: openembedded-core
If we do `bitbake buildtools-tarball' and then after one day do `bitbake
core-image-minimal -c populate_sdk_ext', we would meet errors like below.
| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/
poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone
-1.8+snapshot-20150429.sh': No such file or directory
The problem is that the output name for buildtools-tarball has ${DATE} in it.
So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt,
the above error appears.
Instead of hardcoding ${DISTRO_VERSION} which consists of ${DATE} in the
install_tools() function, we should find the latest buildtools-tarball based
on the modification time and install it.
[YOCTO #7674]
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/classes/populate_sdk_ext.bbclass | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index dc2c58e..2fc4c11 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -168,7 +168,9 @@ install_tools() {
ln -sr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool
touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
- install ${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKGARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH}
+ # find latest buildtools-tarball and install it
+ buildtools_path=`ls -t1 ${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKGARCH}-buildtools-nativesdk-standalone-*.sh | head -n1`
+ install $buildtools_path ${SDK_OUTPUT}/${SDKPATH}
install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2 ${SDK_OUTPUT}/${SDKPATH}
}
--
1.9.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH V2 2/3] copy_buildsystem: make sure bitbake directory is copied
2015-06-10 7:25 [PATCH V2 0/3] Extensible SDK: 3 fixes Chen Qi
2015-06-10 7:25 ` [PATCH V2 1/3] populate_sdk_ext: install the latest buildtools-tarball Chen Qi
@ 2015-06-10 7:25 ` Chen Qi
2015-06-10 7:25 ` [PATCH V2 3/3] populate_sdk_ext: consider custom configuration in local.conf Chen Qi
2 siblings, 0 replies; 5+ messages in thread
From: Chen Qi @ 2015-06-10 7:25 UTC (permalink / raw)
To: openembedded-core
The previous code assumes that bitbake/ directory is under the core layer.
This is the case for Yocto project. But users might clone oe-core and bitbake
separately. So we use bb.__file__ to locate the bitbake directory to make sure
it's copied into the extensible SDK.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oe/copy_buildsystem.py | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
index cf7fada..979578c 100644
--- a/meta/lib/oe/copy_buildsystem.py
+++ b/meta/lib/oe/copy_buildsystem.py
@@ -28,11 +28,10 @@ class BuildSystem(object):
layers.append(corebase)
corebase_files = self.d.getVar('COREBASE_FILES', True).split()
-
- # bitbake belongs in corebase so make sure it goes there
- if "bitbake" not in corebase_files:
- corebase_files.append("bitbake")
corebase_files = [corebase + '/' +x for x in corebase_files]
+ # Make sure bitbake goes in
+ bitbake_dir = bb.__file__.rsplit('/', 3)[0]
+ corebase_files.append(bitbake_dir)
for layer in layers:
layerconf = os.path.join(layer, 'conf', 'layer.conf')
--
1.9.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH V2 3/3] populate_sdk_ext: consider custom configuration in local.conf
2015-06-10 7:25 [PATCH V2 0/3] Extensible SDK: 3 fixes Chen Qi
2015-06-10 7:25 ` [PATCH V2 1/3] populate_sdk_ext: install the latest buildtools-tarball Chen Qi
2015-06-10 7:25 ` [PATCH V2 2/3] copy_buildsystem: make sure bitbake directory is copied Chen Qi
@ 2015-06-10 7:25 ` Chen Qi
2015-06-15 16:26 ` Paul Eggleton
2 siblings, 1 reply; 5+ messages in thread
From: Chen Qi @ 2015-06-10 7:25 UTC (permalink / raw)
To: openembedded-core
Copy the contents of local.conf under TOPDIR into the final generated
local.conf. In this way, custom settings are also made into the final
local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.
Before this change, installing extensible SDK would usually report failure
when preparing the build system if the user has custom configuration for
DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in local.conf
also don't get built correctly.
This patch solves the above problem by making use of the bb.utils.edit_metadata.
In addition, we check to avoid any setting that might lead to host path
bleeding into SDK's configuration. Basically, variables with values starting
with '/' are removed. A whitelist mechanism is introduced so that users could
specify variables that should not be ignored. The name of the whitelist is
SDK_LOCAL_CONF_WHITELIST.
[YOCTO #7616]
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/classes/populate_sdk_ext.bbclass | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index 2fc4c11..08130d4 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -16,6 +16,7 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"
SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"
SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"
+SDK_LOCAL_CONF_WHITELIST ?= ""
SDK_TARGETS ?= "${PN}"
OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"
@@ -108,12 +109,28 @@ python copy_buildsystem () {
f.write(' "\n')
# Create local.conf
+ local_conf_whitelist = d.getVar('SDK_LOCAL_CONF_WHITELIST', True).split()
+ def handle_var(varname, origvalue, op, newlines):
+ if origvalue.strip().startswith('/') and not varname in local_conf_whitelist:
+ newlines.append('# Removed original setting of %s\n' % varname)
+ return None, op, 0, True
+ else:
+ return origvalue, op, 0, True
+ varlist = ['[^#=+ ]*']
+ builddir = d.getVar('TOPDIR', True)
+ with open(builddir + '/conf/local.conf', 'r') as f:
+ oldlines = f.readlines()
+ (updated, newlines) = bb.utils.edit_metadata(oldlines, varlist, handle_var)
+
with open(baseoutpath + '/conf/local.conf', 'w') as f:
f.write('# WARNING: this configuration has been automatically generated and in\n')
f.write('# most cases should not be edited. If you need more flexibility than\n')
f.write('# this configuration provides, it is strongly suggested that you set\n')
f.write('# up a proper instance of the full build system and use that instead.\n\n')
+ for line in newlines:
+ f.write(line)
+
f.write('INHERIT += "%s"\n\n' % 'uninative')
f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))
--
1.9.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH V2 3/3] populate_sdk_ext: consider custom configuration in local.conf
2015-06-10 7:25 ` [PATCH V2 3/3] populate_sdk_ext: consider custom configuration in local.conf Chen Qi
@ 2015-06-15 16:26 ` Paul Eggleton
0 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2015-06-15 16:26 UTC (permalink / raw)
To: Chen Qi; +Cc: openembedded-core
On Wednesday 10 June 2015 15:25:24 Chen Qi wrote:
> Copy the contents of local.conf under TOPDIR into the final generated
> local.conf. In this way, custom settings are also made into the final
> local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.
>
> Before this change, installing extensible SDK would usually report failure
> when preparing the build system if the user has custom configuration for
> DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in
> local.conf also don't get built correctly.
>
> This patch solves the above problem by making use of the
> bb.utils.edit_metadata.
>
> In addition, we check to avoid any setting that might lead to host path
> bleeding into SDK's configuration. Basically, variables with values starting
> with '/' are removed. A whitelist mechanism is introduced so that users
> could specify variables that should not be ignored. The name of the
> whitelist is SDK_LOCAL_CONF_WHITELIST.
>
> [YOCTO #7616]
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
> meta/classes/populate_sdk_ext.bbclass | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
>
> diff --git a/meta/classes/populate_sdk_ext.bbclass
> b/meta/classes/populate_sdk_ext.bbclass index 2fc4c11..08130d4 100644
> --- a/meta/classes/populate_sdk_ext.bbclass
> +++ b/meta/classes/populate_sdk_ext.bbclass
> @@ -16,6 +16,7 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = "
> ${SDK_TARGETS}" SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"
>
> SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"
> +SDK_LOCAL_CONF_WHITELIST ?= ""
>
> SDK_TARGETS ?= "${PN}"
> OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"
> @@ -108,12 +109,28 @@ python copy_buildsystem () {
> f.write(' "\n')
>
> # Create local.conf
> + local_conf_whitelist = d.getVar('SDK_LOCAL_CONF_WHITELIST',
> True).split() + def handle_var(varname, origvalue, op, newlines):
> + if origvalue.strip().startswith('/') and not varname in
> local_conf_whitelist: + newlines.append('# Removed original
> setting of %s\n' % varname) + return None, op, 0, True
> + else:
> + return origvalue, op, 0, True
> + varlist = ['[^#=+ ]*']
> + builddir = d.getVar('TOPDIR', True)
> + with open(builddir + '/conf/local.conf', 'r') as f:
> + oldlines = f.readlines()
> + (updated, newlines) = bb.utils.edit_metadata(oldlines, varlist,
> handle_var) +
> with open(baseoutpath + '/conf/local.conf', 'w') as f:
> f.write('# WARNING: this configuration has been automatically
> generated and in\n') f.write('# most cases should not be edited. If you
> need more flexibility than\n') f.write('# this configuration provides, it
> is strongly suggested that you set\n') f.write('# up a proper instance of
> the full build system and use that instead.\n\n')
>
> + for line in newlines:
> + f.write(line)
> +
> f.write('INHERIT += "%s"\n\n' % 'uninative')
> f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))
OK, this looks good - but do we really need two whitelists now
(SDK_META_CONF_WHITELIST and SDK_LOCAL_CONF_WHITELIST)?
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-06-15 16:26 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-10 7:25 [PATCH V2 0/3] Extensible SDK: 3 fixes Chen Qi
2015-06-10 7:25 ` [PATCH V2 1/3] populate_sdk_ext: install the latest buildtools-tarball Chen Qi
2015-06-10 7:25 ` [PATCH V2 2/3] copy_buildsystem: make sure bitbake directory is copied Chen Qi
2015-06-10 7:25 ` [PATCH V2 3/3] populate_sdk_ext: consider custom configuration in local.conf Chen Qi
2015-06-15 16:26 ` Paul Eggleton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.