* [PATCH 0/7] rootfs generation improvements v3
@ 2012-07-25 18:42 Paul Eggleton
2012-07-25 18:42 ` [PATCH 1/7] package.bbclass: Create symlinks for packages with different packaged name Paul Eggleton
` (8 more replies)
0 siblings, 9 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
Fix up handling of "complementary" package installation (dev, dbg, doc,
etc.) and introduce a staticdev-pkgs IMAGE_FEATURES feature.
Changes since v2:
* Clear IMAGE_FEATURES in core-image-minimal so that it doesn't
have dev/dbg packages added (since the default value of IMAGE_FEATURES
contains dev-pkgs and dbg-pkgs and this will now apply to any image).
* Fix errors with the ipk and deb backends if no complementary packages
need to be installed
* Add DEPENDS on rpm to rpmresolve
The following changes since commit 4148bc80c008d25c8a536c7c7dfdeac1669a6662:
libgcc: Add missing dependency on libc:do_package (2012-07-24 10:12:08 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/rootfs2
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/rootfs2
Andrei Gherzan (1):
package.bbclass: Create symlinks for packages with different packaged
name
Paul Eggleton (6):
classes/rootfs_rpm: improve speed of RPM rootfs construction
classes/package_deb: add PackageArch field
Rework installation of dev, dbg, doc, and locale packages
core-image-minimal: clear IMAGE_FEATURES
classes/image: add staticdev-pkgs IMAGE_FEATURES feature
buildhistory: improve performance of image info collection
meta/classes/buildhistory.bbclass | 51 ++-
meta/classes/image.bbclass | 93 ++---
meta/classes/license.bbclass | 2 +-
meta/classes/package.bbclass | 6 +-
meta/classes/package_deb.bbclass | 1 +
meta/classes/package_rpm.bbclass | 183 +++++-----
meta/classes/populate_sdk_deb.bbclass | 2 +
meta/classes/populate_sdk_ipk.bbclass | 2 +
meta/classes/populate_sdk_rpm.bbclass | 3 +
meta/classes/rootfs_deb.bbclass | 51 ++-
meta/classes/rootfs_ipk.bbclass | 50 ++-
meta/classes/rootfs_rpm.bbclass | 69 ++--
meta/recipes-core/images/core-image-minimal.bb | 1 +
meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c | 376 +++++++++++++++++++++
meta/recipes-devtools/rpm/rpmresolve_1.0.bb | 22 ++
scripts/oe-pkgdata-util | 167 +++++++++
scripts/opkg-query-helper.py | 76 +++++
17 files changed, 890 insertions(+), 265 deletions(-)
create mode 100644 meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
create mode 100644 meta/recipes-devtools/rpm/rpmresolve_1.0.bb
create mode 100755 scripts/oe-pkgdata-util
create mode 100755 scripts/opkg-query-helper.py
--
1.7.9.5
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/7] package.bbclass: Create symlinks for packages with different packaged name
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 18:42 ` [PATCH 2/7] classes/rootfs_rpm: improve speed of RPM rootfs construction Paul Eggleton
` (7 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
From: Andrei Gherzan <andrei@gherzan.ro>
While generating license.manifest package information is searched in:
filename=`ls ${TMPDIR}/pkgdata/*/runtime/${pkg}| head -1`
This is ok as long as package name is the same as the package name
after packaging.
For example dbus is packaged as dbus-1. So, searching
ls ${TMPDIR}/pkgdata/*/runtime/dbus-1 will fail because there is no file
with this package name.
Create a symlink to the pkgdata file in a runtime-reverse directory so
that these reverse lookups are possible.
Fixes [YOCTO #2638].
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/license.bbclass | 2 +-
meta/classes/package.bbclass | 6 +++++-
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index a2721ab..231b4a3 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -86,7 +86,7 @@ license_create_manifest() {
# list of installed packages is broken for deb
for pkg in ${INSTALLED_PKGS}; do
# not the best way to do this but licenses are not arch dependant iirc
- filename=`ls ${TMPDIR}/pkgdata/*/runtime/${pkg}| head -1`
+ filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1`
pkged_pn="$(sed -n 's/^PN: //p' ${filename})"
pkged_lic="$(sed -n '/^LICENSE: /{ s/^LICENSE: //; s/[+|&()*]/ /g; s/ */ /g; p }' ${filename})"
pkged_pv="$(sed -n 's/^PV: //p' ${filename})"
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index fc83424..ea199d3 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1124,6 +1124,10 @@ python emit_pkgdata() {
sf.write('%s_%s: %s\n' % ('PKGSIZE', pkg, get_directory_size(pkgdest + "/%s" % pkg)))
sf.close()
+ # Symlinks needed for reverse lookups (from the final package name)
+ pkgval = d.getVar('PKG_%s' % (pkg), True)
+ subdata_sym = pkgdatadir + "/runtime-reverse/%s" % pkgval
+ oe.path.symlink("../runtime/%s" % pkg, subdata_sym, True)
allow_empty = d.getVar('ALLOW_EMPTY_%s' % pkg, True)
if not allow_empty:
@@ -1137,7 +1141,7 @@ python emit_pkgdata() {
bb.utils.unlockfile(lf)
}
-emit_pkgdata[dirs] = "${PKGDESTWORK}/runtime"
+emit_pkgdata[dirs] = "${PKGDESTWORK}/runtime ${PKGDESTWORK}/runtime-reverse"
ldconfig_postinst_fragment() {
if [ x"$D" = "x" ]; then
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 2/7] classes/rootfs_rpm: improve speed of RPM rootfs construction
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
2012-07-25 18:42 ` [PATCH 1/7] package.bbclass: Create symlinks for packages with different packaged name Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 18:42 ` [PATCH 3/7] classes/package_deb: add PackageArch field Paul Eggleton
` (6 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
Improve the performance of the RPM backend during do_rootfs by
performing most of the package name to file resolution in a separate
utility written in C, processing the entire list of packages at once
rather than running rpm on the command line which loads the RPM database
for every package.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/package_rpm.bbclass | 109 ++++----
meta/classes/rootfs_rpm.bbclass | 1 +
meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c | 273 +++++++++++++++++++++
meta/recipes-devtools/rpm/rpmresolve_1.0.bb | 22 ++
4 files changed, 341 insertions(+), 64 deletions(-)
create mode 100644 meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
create mode 100644 meta/recipes-devtools/rpm/rpmresolve_1.0.bb
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 983be4c..29018e9 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -197,6 +197,45 @@ rpm_update_pkg () {
fi
}
+process_pkg_list_rpm() {
+ local insttype=$1
+ shift
+ local pkgs="$@"
+ local confbase=${INSTALL_CONFBASE_RPM}
+
+ echo -n > ${target_rootfs}/install/base_archs.pkglist
+ echo -n > ${target_rootfs}/install/ml_archs.pkglist
+
+ for pkg in $pkgs; do
+ echo "Processing $pkg..."
+
+ archvar=base_archs
+ ml_pkg=$pkg
+ for i in ${MULTILIB_PREFIX_LIST} ; do
+ subst=${pkg#${i}-}
+ if [ $subst != $pkg ] ; then
+ ml_pkg=$subst
+ archvar=ml_archs
+ break
+ fi
+ done
+
+ echo $ml_pkg >> ${target_rootfs}/install/$archvar.pkglist
+ done
+
+ local manifestpfx="install"
+ local extraopt=""
+ if [ "$insttype" = "attemptonly" ] ; then
+ manifestpfx="install_attemptonly"
+ extraopt="-i"
+ fi
+
+ rpmresolve $extraopt ${confbase}-base_archs.conf ${target_rootfs}/install/base_archs.pkglist >> ${target_rootfs}/install/${manifestpfx}.manifest
+ if [ -s ${target_rootfs}/install/ml_archs.pkglist ] ; then
+ rpmresolve $extraopt ${confbase}-ml_archs.conf ${target_rootfs}/install/ml_archs.pkglist >> ${target_rootfs}/install/${manifestpfx}_multilib.manifest
+ fi
+}
+
#
# install a bunch of packages using rpm
# the following shell variables needs to be set before calling this func:
@@ -256,55 +295,12 @@ package_install_internal_rpm () {
# Uclibc builds don't provide this stuff...
if [ x${TARGET_OS} = "xlinux" ] || [ x${TARGET_OS} = "xlinux-gnueabi" ] ; then
if [ ! -z "${package_linguas}" ]; then
- for pkg in ${package_linguas}; do
- echo "Processing $pkg..."
-
- archvar=base_archs
- manifest=install.manifest
- ml_prefix=`echo ${pkg} | cut -d'-' -f1`
- ml_pkg=$pkg
- for i in ${MULTILIB_PREFIX_LIST} ; do
- if [ ${ml_prefix} = ${i} ]; then
- ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,")
- archvar=ml_archs
- manifest=install_multilib.manifest
- break
- fi
- done
-
- pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
- if [ -z "$pkg_name" ]; then
- echo "Unable to find package $pkg ($ml_pkg)!"
- exit 1
- fi
- echo $pkg_name >> ${target_rootfs}/install/${manifest}
- done
+ process_pkg_list_rpm linguas ${package_linguas}
fi
fi
- if [ ! -z "${package_to_install}" ]; then
- for pkg in ${package_to_install} ; do
- echo "Processing $pkg..."
- archvar=base_archs
- manifest=install.manifest
- ml_prefix=`echo ${pkg} | cut -d'-' -f1`
- ml_pkg=$pkg
- for i in ${MULTILIB_PREFIX_LIST} ; do
- if [ ${ml_prefix} = ${i} ]; then
- ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,")
- archvar=ml_archs
- manifest=install_multilib.manifest
- break
- fi
- done
-
- pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
- if [ -z "$pkg_name" ]; then
- echo "Unable to find package $pkg ($ml_pkg)!"
- exit 1
- fi
- echo $pkg_name >> ${target_rootfs}/install/${manifest}
- done
+ if [ ! -z "${package_to_install}" ]; then
+ process_pkg_list_rpm default ${package_to_install}
fi
# Normal package installation
@@ -324,24 +320,9 @@ package_install_internal_rpm () {
if [ ! -z "${package_attemptonly}" ]; then
echo "Adding attempt only packages..."
- for pkg in ${package_attemptonly} ; do
- echo "Processing $pkg..."
- archvar=base_archs
- ml_prefix=`echo ${pkg} | cut -d'-' -f1`
- ml_pkg=$pkg
- for i in ${MULTILIB_PREFIX_LIST} ; do
- if [ ${ml_prefix} = ${i} ]; then
- ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,")
- archvar=ml_archs
- break
- fi
- done
-
- pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
- if [ -z "$pkg_name" ]; then
- echo "Note: Unable to find package $pkg ($ml_pkg) -- PACKAGE_INSTALL_ATTEMPTONLY"
- continue
- fi
+ process_pkg_list_rpm attemptonly ${package_attemptonly}
+ cat ${target_rootfs}/install/install_attemptonly.manifest | while read pkg_name
+ do
echo "Attempting $pkg_name..." >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}"
${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index 4551f7a..cd9c5ab 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -11,6 +11,7 @@ IMAGE_ROOTFS_EXTRA_SPACE_append = "${@base_contains("PACKAGE_INSTALL", "zypper",
ROOTFS_PKGMANAGE_BOOTSTRAP = ""
do_rootfs[depends] += "rpm-native:do_populate_sysroot"
+do_rootfs[depends] += "rpmresolve-native:do_populate_sysroot"
# Needed for update-alternatives
do_rootfs[depends] += "opkg-native:do_populate_sysroot"
diff --git a/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c b/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
new file mode 100644
index 0000000..9f6cdf2
--- /dev/null
+++ b/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
@@ -0,0 +1,273 @@
+/* OpenEmbedded RPM resolver utility
+
+ Written by: Paul Eggleton <paul.eggleton@linux.intel.com>
+
+ Copyright 2012 Intel Corporation
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License version 2 as
+ published by the Free Software Foundation.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License along
+ with this program; if not, write to the Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+*/
+
+#include <ctype.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include <rpmdb.h>
+#include <rpmtypes.h>
+#include <rpmtag.h>
+#include <rpmts.h>
+#include <rpmmacro.h>
+#include <rpmcb.h>
+#include <rpmlog.h>
+#include <argv.h>
+#include <mire.h>
+
+int getPackageStr(rpmts ts, const char *NVRA, rpmTag tag, char **value)
+{
+ int rc = -1;
+ rpmmi mi = rpmtsInitIterator(ts, RPMTAG_NVRA, NVRA, 0);
+ Header h;
+ if ((h = rpmmiNext(mi)) != NULL) {
+ HE_t he = (HE_t) memset(alloca(sizeof(*he)), 0, sizeof(*he));
+ he->tag = tag;
+ rc = (headerGet(h, he, 0) != 1);
+ if(rc==0)
+ *value = strdup((char *)he->p.ptr);
+ }
+ (void)rpmmiFree(mi);
+ return rc;
+}
+
+int loadTs(rpmts **ts, int *tsct, const char *dblistfn)
+{
+ int count = 0;
+ int sz = 5;
+ int rc = 0;
+ int listfile = 1;
+ struct stat st_buf;
+
+ rc = stat(dblistfn, &st_buf);
+ if(rc != 0) {
+ perror("stat");
+ return 1;
+ }
+ if(S_ISDIR(st_buf.st_mode))
+ listfile = 0;
+
+ if(listfile) {
+ *ts = malloc(sz * sizeof(rpmts));
+ FILE *f = fopen(dblistfn, "r" );
+ if(f) {
+ char line[2048];
+ while(fgets(line, sizeof(line), f)) {
+ int len = strlen(line) - 1;
+ if(len > 0)
+ // Trim trailing whitespace
+ while(len > 0 && isspace(line[len]))
+ line[len--] = '\0';
+
+ if(len > 0) {
+ // Expand array if needed
+ if(count == sz) {
+ sz += 5;
+ *ts = (rpmts *)realloc(*ts, sz);
+ }
+
+ char *dbpathm = malloc(strlen(line) + 10);
+ sprintf(dbpathm, "_dbpath %s", line);
+ rpmDefineMacro(NULL, dbpathm, RMIL_CMDLINE);
+ free(dbpathm);
+
+ rpmts tsi = rpmtsCreate();
+ (*ts)[count] = tsi;
+ rc = rpmtsOpenDB(tsi, O_RDONLY);
+ if( rc ) {
+ fprintf(stderr, "Failed to open database %s\n", line);
+ rc = -1;
+ break;
+ }
+
+ count++;
+ }
+ }
+ fclose(f);
+ *tsct = count;
+ }
+ else {
+ perror(dblistfn);
+ rc = -1;
+ }
+ }
+ else {
+ // Load from single database
+ *ts = malloc(sizeof(rpmts));
+ char *dbpathm = malloc(strlen(dblistfn) + 10);
+ sprintf(dbpathm, "_dbpath %s", dblistfn);
+ rpmDefineMacro(NULL, dbpathm, RMIL_CMDLINE);
+ free(dbpathm);
+
+ rpmts tsi = rpmtsCreate();
+ (*ts)[0] = tsi;
+ rc = rpmtsOpenDB(tsi, O_RDONLY);
+ if( rc ) {
+ fprintf(stderr, "Failed to open database %s\n", dblistfn);
+ rc = -1;
+ }
+ *tsct = 1;
+ }
+
+ return rc;
+}
+
+int processPackages(rpmts *ts, int tscount, const char *packagelistfn, int ignoremissing)
+{
+ int rc = 0;
+ int count = 0;
+ int sz = 100;
+ int i = 0;
+ int missing = 0;
+
+ FILE *f = fopen(packagelistfn, "r" );
+ if(f) {
+ char line[255];
+ while(fgets(line, sizeof(line), f)) {
+ int len = strlen(line) - 1;
+ if(len > 0)
+ // Trim trailing whitespace
+ while(len > 0 && isspace(line[len]))
+ line[len--] = '\0';
+
+ if(len > 0) {
+ int found = 0;
+ for(i=0; i<tscount; i++) {
+ ARGV_t keys = NULL;
+ rpmdb db = rpmtsGetRdb(ts[i]);
+ rc = rpmdbMireApply(db, RPMTAG_NAME,
+ RPMMIRE_STRCMP, line, &keys);
+ if (keys) {
+ int nkeys = argvCount(keys);
+ if( nkeys == 1 ) {
+ char *value = NULL;
+ rc = getPackageStr(ts[i], keys[0], RPMTAG_PACKAGEORIGIN, &value);
+ if(rc == 0)
+ printf("%s\n", value);
+ else
+ fprintf(stderr, "Failed to get package origin for %s\n", line);
+ found = 1;
+ }
+ else if( nkeys > 1 ) {
+ fprintf(stderr, "Multiple matches for %s!\n", line);
+ }
+ }
+ if(found)
+ break;
+ }
+
+ if( !found ) {
+ if( ignoremissing ) {
+ fprintf(stderr, "unable to find package %s - ignoring\n", line);
+ }
+ else {
+ fprintf(stderr, "unable to find package %s\n", line);
+ missing = 1;
+ }
+ }
+ }
+ count++;
+ }
+ fclose(f);
+
+ if( missing ) {
+ fprintf(stderr, "ERROR: some packages were missing\n");
+ rc = 1;
+ }
+ }
+ else {
+ perror(packagelistfn);
+ rc = -1;
+ }
+
+ return rc;
+}
+
+void usage()
+{
+ fprintf(stderr, "OpenEmbedded rpm resolver utility\n");
+ fprintf(stderr, "syntax: rpmresolve [-i] <dblistfile> <packagelistfile>\n");
+}
+
+int main(int argc, char **argv)
+{
+ rpmts *ts = NULL;
+ int tscount = 0;
+ int rc = 0;
+ int i;
+ int c;
+ int ignoremissing = 0;
+
+ opterr = 0;
+ while ((c = getopt (argc, argv, "i")) != -1) {
+ switch (c) {
+ case 'i':
+ ignoremissing = 1;
+ break;
+ case '?':
+ if(isprint(optopt))
+ fprintf(stderr, "Unknown option `-%c'.\n", optopt);
+ else
+ fprintf(stderr, "Unknown option character `\\x%x'.\n",
+ optopt);
+ usage();
+ return 1;
+ default:
+ abort();
+ }
+ }
+
+ if( argc - optind < 1 ) {
+ usage();
+ return 1;
+ }
+
+ const char *dblistfn = argv[optind];
+
+ //rpmSetVerbosity(RPMLOG_DEBUG);
+
+ rpmReadConfigFiles( NULL, NULL );
+ rpmDefineMacro(NULL, "__dbi_txn create nofsync", RMIL_CMDLINE);
+
+ rc = loadTs(&ts, &tscount, dblistfn);
+ if( rc )
+ return 1;
+ if( tscount == 0 ) {
+ fprintf(stderr, "Please specify database list file or database location\n");
+ return 1;
+ }
+
+ if( argc - optind < 2 ) {
+ fprintf(stderr, "Please specify package list file\n");
+ return 1;
+ }
+ const char *pkglistfn = argv[optind+1];
+ rc = processPackages(ts, tscount, pkglistfn, ignoremissing);
+
+ for(i=0; i<tscount; i++)
+ (void) rpmtsCloseDB(ts[i]);
+ free(ts);
+
+ return rc;
+}
diff --git a/meta/recipes-devtools/rpm/rpmresolve_1.0.bb b/meta/recipes-devtools/rpm/rpmresolve_1.0.bb
new file mode 100644
index 0000000..f8750e0
--- /dev/null
+++ b/meta/recipes-devtools/rpm/rpmresolve_1.0.bb
@@ -0,0 +1,22 @@
+SUMMARY = "OpenEmbedded RPM resolver utility"
+DESCRIPTION = "OpenEmbedded RPM resolver - performs RPM database lookups in batches to avoid \
+ repeated invocations of rpm on the command line."
+DEPENDS = "rpm"
+LICENSE = "GPLv2"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6"
+PR = "r0"
+
+SRC_URI = "file://rpmresolve.c"
+
+S = "${WORKDIR}"
+
+do_compile() {
+ ${CC} ${CFLAGS} -ggdb -I${STAGING_INCDIR}/rpm ${LDFLAGS} rpmresolve.c -o rpmresolve -lrpmbuild -lrpm -lrpmio -lrpmdb -lpopt
+}
+
+do_install() {
+ install -d ${D}${bindir}
+ install -m 0755 rpmresolve ${D}${bindir}
+}
+
+BBCLASSEXTEND = "native"
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 3/7] classes/package_deb: add PackageArch field
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
2012-07-25 18:42 ` [PATCH 1/7] package.bbclass: Create symlinks for packages with different packaged name Paul Eggleton
2012-07-25 18:42 ` [PATCH 2/7] classes/rootfs_rpm: improve speed of RPM rootfs construction Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 18:42 ` [PATCH 4/7] Rework installation of dev, dbg, doc, and locale packages Paul Eggleton
` (5 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
If we want to query the PACKAGE_ARCH from the installed package (as we
do in order to be able to do a pkgdata lookup for example) then we need
to have this stored in its own field as this is not always the same as
the Architecture field for deb packages.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/package_deb.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index d09baea..48511df 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -290,6 +290,7 @@ python do_package_deb () {
fields.append(["Maintainer: %s\n", ['MAINTAINER']])
fields.append(["Architecture: %s\n", ['DPKG_ARCH']])
fields.append(["OE: %s\n", ['PN']])
+ fields.append(["PackageArch: %s\n", ['PACKAGE_ARCH']])
fields.append(["Homepage: %s\n", ['HOMEPAGE']])
# Package, Version, Maintainer, Description - mandatory
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 4/7] Rework installation of dev, dbg, doc, and locale packages
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
` (2 preceding siblings ...)
2012-07-25 18:42 ` [PATCH 3/7] classes/package_deb: add PackageArch field Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 18:42 ` [PATCH 5/7] core-image-minimal: clear IMAGE_FEATURES Paul Eggleton
` (4 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
Use a similar mechanism that was previously used to install locales at
rootfs generation time to install other "complementary" packages (e.g.
*-dev packages) - i.e. install all of the explicitly requested packages
and their dependencies, then get a list of the packages that were
installed, and use that list to install the complementary packages. This
has been implemented by using a list of globs which should make it
easier to extend in future.
The previous locale package installation code assumed that the locale
packages did not have any dependencies that were not already installed;
now that we are installing non-locale packages this is no longer
correct. In practice only the rpm backend actually made use of this
assumption, so it needed to be changed to call into the existing package
backend code to do the complementary package installation rather than
calling rpm directly.
This fixes the doc-pkgs IMAGE_FEATURES feature to work correctly, and
also ensures that all dev/dbg packages get installed for
dev-pkgs/dbg-pkgs respectively even if the dependency chains between
those packages was not ensuring that already.
The code has also been adapted to work correctly with the new
SDK-from-image functionality. To that end, an SDKIMAGE_FEATURES variable
has been added to allow specifying what extra image features should go
into the SDK (extra, because by virtue of installing all of the packages
in the image into the target part of the SDK, we already include all of
IMAGE_FEATURES) with a default value of "dev-pkgs dbg-pkgs".
Fixes [YOCTO #2614].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/image.bbclass | 89 ++++++++++--------
meta/classes/package_rpm.bbclass | 74 +++++++++------
meta/classes/populate_sdk_deb.bbclass | 2 +
meta/classes/populate_sdk_ipk.bbclass | 2 +
meta/classes/populate_sdk_rpm.bbclass | 3 +
meta/classes/rootfs_deb.bbclass | 27 +++---
meta/classes/rootfs_ipk.bbclass | 26 ++---
meta/classes/rootfs_rpm.bbclass | 35 +++----
scripts/oe-pkgdata-util | 167 +++++++++++++++++++++++++++++++++
scripts/opkg-query-helper.py | 76 +++++++++++++++
10 files changed, 390 insertions(+), 111 deletions(-)
create mode 100755 scripts/oe-pkgdata-util
create mode 100755 scripts/opkg-query-helper.py
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 1799bf1..2c605bf 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -6,7 +6,8 @@ inherit imagetest-${IMAGETEST}
inherit populate_sdk_base
TOOLCHAIN_TARGET_TASK += "${PACKAGE_INSTALL}"
-TOOLCHAIN_TARGET_TASK_ATTEMPTONLY += "${PACKAGE_INSTALL_ATTEMPTONLY} ${PACKAGE_GROUP_dev-pkgs} ${PACKAGE_GROUP_dbg-pkgs}"
+TOOLCHAIN_TARGET_TASK_ATTEMPTONLY += "${PACKAGE_INSTALL_ATTEMPTONLY}"
+POPULATE_SDK_POST_TARGET_COMMAND += "rootfs_install_complementary populate_sdk; "
inherit gzipnative
@@ -38,25 +39,23 @@ def normal_groups(d):
features = set(oe.data.typed_value('IMAGE_FEATURES', d))
return features.difference(extras)
-def normal_pkgs_to_install(d):
- import oe.packagedata
-
- to_install = oe.data.typed_value('IMAGE_INSTALL', d)
- features = normal_groups(d)
- required = list(oe.packagegroup.required_packages(features, d))
- optional = list(oe.packagegroup.optional_packages(features, d))
- all_packages = to_install + required + optional
-
- recipes = filter(None, [oe.packagedata.recipename(pkg, d) for pkg in all_packages])
-
- return all_packages + recipes
-
-PACKAGE_GROUP_dbg-pkgs = "${@' '.join('%s-dbg' % pkg for pkg in normal_pkgs_to_install(d))}"
-PACKAGE_GROUP_dbg-pkgs[optional] = "1"
-PACKAGE_GROUP_dev-pkgs = "${@' '.join('%s-dev' % pkg for pkg in normal_pkgs_to_install(d))}"
-PACKAGE_GROUP_dev-pkgs[optional] = "1"
-PACKAGE_GROUP_doc-pkgs = "${@' '.join('%s-doc' % pkg for pkg in normal_pkgs_to_install(d))}"
-PACKAGE_GROUP_doc-pkgs[optional] = "1"
+# Wildcards specifying complementary packages to install for every package that has been explicitly
+# installed into the rootfs
+def complementary_globs(featurevar, d):
+ globs = []
+ features = set((d.getVar(featurevar, True) or '').split())
+ for feature in features:
+ if feature == 'dev-pkgs':
+ globs.append('*-dev')
+ elif feature == 'doc-pkgs':
+ globs.append('*-doc')
+ elif feature == 'dbg-pkgs':
+ globs.append('*-dbg')
+ return ' '.join(globs)
+
+IMAGE_INSTALL_COMPLEMENTARY = '${@complementary_globs("IMAGE_FEATURES", d)}'
+SDKIMAGE_FEATURES ??= "dev-pkgs dbg-pkgs"
+SDKIMAGE_INSTALL_COMPLEMENTARY = '${@complementary_globs("SDKIMAGE_FEATURES", d)}'
# "export IMAGE_BASENAME" not supported at this time
IMAGE_INSTALL ?= ""
@@ -306,32 +305,44 @@ get_split_linguas() {
done | sort | uniq
}
-rootfs_install_all_locales() {
- # Generate list of installed packages for which additional locale packages might be available
- INSTALLED_PACKAGES=`list_installed_packages | egrep -v -- "(-locale-|^locale-base-|-dev$|-doc$|^kernel|^glibc|^ttf|^task|^perl|^python)"`
-
- # Generate a list of locale packages that exist
- SPLIT_LINGUAS=`get_split_linguas`
- PACKAGES_TO_INSTALL=""
- for lang in $SPLIT_LINGUAS; do
- for pkg in $INSTALLED_PACKAGES; do
- existing_pkg=`rootfs_check_package_exists $pkg-locale-$lang`
- if [ "$existing_pkg" != "" ]; then
- PACKAGES_TO_INSTALL="$PACKAGES_TO_INSTALL $existing_pkg"
- fi
+rootfs_install_complementary() {
+ # Install complementary packages based upon the list of currently installed packages
+ # e.g. locales, *-dev, *-dbg, etc. This will only attempt to install these packages,
+ # if they don't exist then no error will occur.
+ # Note: every backend needs to call this function explicitly after the normal
+ # package installation
+
+ # Get list of installed packages
+ list_installed_packages arch > ${WORKDIR}/installed_pkgs.txt
+
+ # Apply the globs to all the packages currently installed
+ if [ "$1" = "populate_sdk" ] ; then
+ GLOBS="${SDKIMAGE_INSTALL_COMPLEMENTARY}"
+ else
+ GLOBS="${IMAGE_INSTALL_COMPLEMENTARY}"
+ # Add locales
+ SPLIT_LINGUAS=`get_split_linguas`
+ PACKAGES_TO_INSTALL=""
+ for lang in $SPLIT_LINGUAS ; do
+ GLOBS="$GLOBS *-locale-$lang"
done
- done
+ fi
+
+ if [ "$GLOBS" != "" ] ; then
+ # Use the magic script to do all the work for us :)
+ oe-pkgdata-util glob ${TMPDIR}/pkgdata ${TARGET_VENDOR}-${TARGET_OS} ${WORKDIR}/installed_pkgs.txt "$GLOBS" > ${WORKDIR}/complementary_pkgs.txt
- # Install the packages, if any
- if [ "$PACKAGES_TO_INSTALL" != "" ]; then
- rootfs_install_packages $PACKAGES_TO_INSTALL
+ # Install the packages, if any
+ sed -i '/^$/d' ${WORKDIR}/complementary_pkgs.txt
+ if [ cat ${WORKDIR}/complementary_pkgs.txt ]; then
+ echo "Installing complementary packages"
+ rootfs_install_packages ${WORKDIR}/complementary_pkgs.txt
+ fi
fi
# Workaround for broken shell function dependencies
if false ; then
get_split_linguas
- list_installed_packages
- rootfs_check_package_exists
fi
}
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 29018e9..b4bc52e 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -248,6 +248,7 @@ process_pkg_list_rpm() {
# INSTALL_PACKAGES_LINGUAS_RPM - additional packages for uclibc
# INSTALL_PROVIDENAME_RPM - content for provide name
# INSTALL_TASK_RPM - task name
+# INSTALL_COMPLEMENTARY_RPM - 1 to enable complementary package install mode
package_install_internal_rpm () {
@@ -261,31 +262,35 @@ package_install_internal_rpm () {
local providename="${INSTALL_PROVIDENAME_RPM}"
local task="${INSTALL_TASK_RPM}"
- # Setup base system configuration
- mkdir -p ${target_rootfs}/etc/rpm/
- echo "${platform}${TARGET_VENDOR}-${TARGET_OS}" > ${target_rootfs}/etc/rpm/platform
- if [ ! -z "$platform_extra" ]; then
- for pt in $platform_extra ; do
- case $pt in
- noarch | any | all)
- os="`echo ${TARGET_OS} | sed "s,-.*,,"`.*"
- ;;
- *)
- os="${TARGET_OS}"
- ;;
- esac
- echo "$pt-.*-$os" >> ${target_rootfs}/etc/rpm/platform
- done
- fi
+ if [ "${INSTALL_COMPLEMENTARY_RPM}" != "1" ] ; then
+ # Setup base system configuration
+ mkdir -p ${target_rootfs}/etc/rpm/
+ echo "${platform}${TARGET_VENDOR}-${TARGET_OS}" > ${target_rootfs}/etc/rpm/platform
+ if [ ! -z "$platform_extra" ]; then
+ for pt in $platform_extra ; do
+ case $pt in
+ noarch | any | all)
+ os="`echo ${TARGET_OS} | sed "s,-.*,,"`.*"
+ ;;
+ *)
+ os="${TARGET_OS}"
+ ;;
+ esac
+ echo "$pt-.*-$os" >> ${target_rootfs}/etc/rpm/platform
+ done
+ fi
- # Tell RPM that the "/" directory exist and is available
- mkdir -p ${target_rootfs}/etc/rpm/sysinfo
- echo "/" >${target_rootfs}/etc/rpm/sysinfo/Dirnames
- if [ ! -z "$providename" ]; then
- cat /dev/null > ${target_rootfs}/etc/rpm/sysinfo/Providename
- for provide in $providename ; do
- echo $provide >> ${target_rootfs}/etc/rpm/sysinfo/Providename
- done
+ # Tell RPM that the "/" directory exist and is available
+ mkdir -p ${target_rootfs}/etc/rpm/sysinfo
+ echo "/" >${target_rootfs}/etc/rpm/sysinfo/Dirnames
+ if [ ! -z "$providename" ]; then
+ cat /dev/null > ${target_rootfs}/etc/rpm/sysinfo/Providename
+ for provide in $providename ; do
+ echo $provide >> ${target_rootfs}/etc/rpm/sysinfo/Providename
+ done
+ fi
+ else
+ mv ${target_rootfs}/install/total_solution.manifest ${target_rootfs}/install/original_solution.manifest
fi
# Setup manifest of packages to install...
@@ -480,13 +485,22 @@ mutex_set_max 163840
# ================ Replication
EOF
- # RPM is special. It can't handle dependencies and preinstall scripts correctly. Its
- # probably a feature. The only way to convince rpm to actually run the preinstall scripts
- # for base-passwd and shadow first before installing packages that depend on these packages
- # is to do two image installs, installing one set of packages, then the other.
- if [ "${INC_RPM_IMAGE_GEN}" = "1" -a -f "$pre_btmanifest" ]; then
- echo "Skipping pre install due to exisitng image"
+ if [ "${INSTALL_COMPLEMENTARY_RPM}" = "1" ] ; then
+ # Only install packages not already installed (dependency calculation will
+ # almost certainly have added some that have been)
+ sort ${target_rootfs}/install/original_solution.manifest > ${target_rootfs}/install/original_solution_sorted.manifest
+ sort ${target_rootfs}/install/total_solution.manifest > ${target_rootfs}/install/total_solution_sorted.manifest
+ comm -2 -3 ${target_rootfs}/install/total_solution_sorted.manifest \
+ ${target_rootfs}/install/original_solution_sorted.manifest | awk '{print $1}' > \
+ ${target_rootfs}/install/diff.manifest
+ mv ${target_rootfs}/install/diff.manifest ${target_rootfs}/install/total_solution.manifest
+ elif [ "${INC_RPM_IMAGE_GEN}" = "1" -a -f "$pre_btmanifest" ]; then
+ echo "Skipping pre install due to existing image"
else
+ # RPM is special. It can't handle dependencies and preinstall scripts correctly. Its
+ # probably a feature. The only way to convince rpm to actually run the preinstall scripts
+ # for base-passwd and shadow first before installing packages that depend on these packages
+ # is to do two image installs, installing one set of packages, then the other.
rm -f ${target_rootfs}/install/initial_install.manifest
echo "Installing base dependencies first (base-passwd, base-files and shadow) since rpm is special"
grep /base-passwd-[0-9] ${target_rootfs}/install/total_solution.manifest >> ${target_rootfs}/install/initial_install.manifest || true
diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass
index 9e9e1e1..6f89dcf 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -36,6 +36,8 @@ populate_sdk_deb () {
package_install_internal_deb
+ ${POPULATE_SDK_POST_TARGET_COMMAND}
+
populate_sdk_post_deb ${INSTALL_ROOTFS_DEB}
populate_sdk_log_check populate_sdk
diff --git a/meta/classes/populate_sdk_ipk.bbclass b/meta/classes/populate_sdk_ipk.bbclass
index 4321afb..65a95e7 100644
--- a/meta/classes/populate_sdk_ipk.bbclass
+++ b/meta/classes/populate_sdk_ipk.bbclass
@@ -29,6 +29,8 @@ populate_sdk_ipk() {
package_install_internal_ipk
+ ${POPULATE_SDK_POST_TARGET_COMMAND}
+
#install host
export INSTALL_ROOTFS_IPK="${SDK_OUTPUT}"
export INSTALL_CONF_IPK="${IPKGCONF_SDK}"
diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass
index 365a337..fac653b 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -37,6 +37,7 @@ populate_sdk_rpm () {
export INSTALL_PACKAGES_LINGUAS_RPM=""
export INSTALL_PROVIDENAME_RPM="/bin/sh /bin/bash /usr/bin/env /usr/bin/perl pkgconfig pkgconfig(pkg-config)"
export INSTALL_TASK_RPM="populate_sdk-target"
+ export INSTALL_COMPLEMENTARY_RPM=""
# Setup base system configuration
mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
@@ -74,6 +75,7 @@ EOF
export INSTALL_PLATFORM_EXTRA_RPM
package_install_internal_rpm
+ ${POPULATE_SDK_POST_TARGET_COMMAND}
populate_sdk_post_rpm ${INSTALL_ROOTFS_RPM}
## install nativesdk ##
@@ -86,6 +88,7 @@ EOF
export INSTALL_PACKAGES_LINGUAS_RPM=""
export INSTALL_PROVIDENAME_RPM="/bin/sh /bin/bash /usr/bin/env /usr/bin/perl pkgconfig libGL.so()(64bit) libGL.so"
export INSTALL_TASK_RPM="populate_sdk_rpm-nativesdk"
+ export INSTALL_COMPLEMENTARY_RPM=""
# List must be prefered to least preferred order
INSTALL_PLATFORM_EXTRA_RPM=""
diff --git a/meta/classes/rootfs_deb.bbclass b/meta/classes/rootfs_deb.bbclass
index 67871a9..a002b1e 100644
--- a/meta/classes/rootfs_deb.bbclass
+++ b/meta/classes/rootfs_deb.bbclass
@@ -10,7 +10,7 @@ do_rootfs[recrdeptask] += "do_package_write_deb"
do_rootfs[lockfiles] += "${WORKDIR}/deb.lock"
-DEB_POSTPROCESS_COMMANDS = "rootfs_install_all_locales; "
+DEB_POSTPROCESS_COMMANDS = ""
opkglibdir = "${localstatedir}/lib/opkg"
@@ -42,6 +42,8 @@ fakeroot rootfs_deb_do_rootfs () {
package_install_internal_deb
${DEB_POSTPROCESS_COMMANDS}
+ rootfs_install_complementary
+
export D=${IMAGE_ROOTFS}
export OFFLINE_ROOT=${IMAGE_ROOTFS}
export IPKG_OFFLINE_ROOT=${IMAGE_ROOTFS}
@@ -87,10 +89,16 @@ remove_packaging_data_files() {
rm -rf ${IMAGE_ROOTFS}/usr/dpkg/
}
-DPKG_QUERY_COMMAND = "${STAGING_BINDIR_NATIVE}/dpkg --admindir=${IMAGE_ROOTFS}/var/lib/dpkg"
+# This will of course only work after rootfs_deb_do_rootfs has been called
+DPKG_QUERY_COMMAND = "${STAGING_BINDIR_NATIVE}/dpkg-query --admindir=$INSTALL_ROOTFS_DEB/var/lib/dpkg"
list_installed_packages() {
- ${DPKG_QUERY_COMMAND} -l | grep ^ii | awk '{ print $2 }'
+ if [ "$1" = "arch" ] ; then
+ # Here we want the PACKAGE_ARCH not the deb architecture
+ ${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch}\n'
+ else
+ ${DPKG_QUERY_COMMAND} -W -f='${Package}\n'
+ fi
}
get_package_filename() {
@@ -110,16 +118,9 @@ list_package_recommends() {
${DPKG_QUERY_COMMAND} -s $1 | grep ^Recommends | sed -e 's/^Recommends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
}
-rootfs_check_package_exists() {
- if [ `apt-cache policy $1 | wc -l` -gt 4 ]; then
- echo $1
- fi
-}
-
rootfs_install_packages() {
- ${STAGING_BINDIR_NATIVE}/apt-get install $@ --force-yes --allow-unauthenticated
+ ${STAGING_BINDIR_NATIVE}/apt-get install `cat $1` --force-yes --allow-unauthenticated
- for pkg in $@ ; do
- deb_package_setflag installed $pkg
- done
+ # Mark all packages installed
+ sed -i -e "s/Status: install ok unpacked/Status: install ok installed/;" $INSTALL_ROOTFS_DEB/var/lib/dpkg/status
}
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index 9732385..7df97a0 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -15,10 +15,12 @@ do_rootfs[recrdeptask] += "do_package_write_ipk"
do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
IPKG_ARGS = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS} --force-overwrite"
+# The _POST version also works when constructing the matching SDK
+IPKG_ARGS_POST = "-f ${IPKGCONF_TARGET} -o $INSTALL_ROOTFS_IPK --force-overwrite"
OPKG_PREPROCESS_COMMANDS = "package_update_index_ipk; package_generate_ipkg_conf"
-OPKG_POSTPROCESS_COMMANDS = "ipk_insert_feed_uris; rootfs_install_all_locales; "
+OPKG_POSTPROCESS_COMMANDS = "ipk_insert_feed_uris; "
opkglibdir = "${localstatedir}/lib/opkg"
@@ -74,6 +76,8 @@ fakeroot rootfs_ipk_do_rootfs () {
#mkdir -p ${IMAGE_ROOTFS}/etc/opkg/
#grep "^arch" ${IPKGCONF_TARGET} >${IMAGE_ROOTFS}/etc/opkg/arch.conf
+ rootfs_install_complementary
+
${OPKG_POSTPROCESS_COMMANDS}
${ROOTFS_POSTINSTALL_COMMAND}
@@ -125,12 +129,16 @@ remove_packaging_data_files() {
}
list_installed_packages() {
- grep ^Package: ${IMAGE_ROOTFS}${opkglibdir}/status | sed "s/^Package: //"
+ if [ "$1" = "arch" ] ; then
+ opkg-cl ${IPKG_ARGS_POST} status | opkg-query-helper.py -a
+ else
+ opkg-cl ${IPKG_ARGS_POST} list_installed | awk '{ print $1 }'
+ fi
}
get_package_filename() {
set +x
- info=`opkg-cl ${IPKG_ARGS} info $1 | grep -B 7 -A 7 "^Status.* \(\(installed\)\|\(unpacked\)\)" || true`
+ info=`opkg-cl ${IPKG_ARGS_POST} info $1 | grep -B 7 -A 7 "^Status.* \(\(installed\)\|\(unpacked\)\)" || true`
name=`echo "${info}" | awk '/^Package/ {printf $2"_"}'`
name=$name`echo "${info}" | awk -F: '/^Version/ {printf $NF"_"}' | sed 's/^\s*//g'`
name=$name`echo "${info}" | awk '/^Archi/ {print $2".ipk"}'`
@@ -145,21 +153,15 @@ get_package_filename() {
}
list_package_depends() {
- opkg-cl ${IPKG_ARGS} info $1 | grep ^Depends | sed -e 's/^Depends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
+ opkg-cl ${IPKG_ARGS_POST} info $1 | grep ^Depends | sed -e 's/^Depends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
}
list_package_recommends() {
- opkg-cl ${IPKG_ARGS} info $1 | grep ^Recommends | sed -e 's/^Recommends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
-}
-
-rootfs_check_package_exists() {
- if [ `opkg-cl ${IPKG_ARGS} info $1 | wc -l` -gt 2 ]; then
- echo $1
- fi
+ opkg-cl ${IPKG_ARGS_POST} info $1 | grep ^Recommends | sed -e 's/^Recommends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
}
rootfs_install_packages() {
- opkg-cl ${IPKG_ARGS} install $PACKAGES_TO_INSTALL
+ opkg-cl ${IPKG_ARGS_POST} install `cat $1`
}
ipk_insert_feed_uris () {
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index cd9c5ab..1cc4a84 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -22,7 +22,7 @@ do_rootfs[depends] += "opkg-native:do_populate_sysroot"
do_rootfs[recrdeptask] += "do_package_write_rpm"
RPM_PREPROCESS_COMMANDS = "package_update_index_rpm; package_generate_rpm_conf; "
-RPM_POSTPROCESS_COMMANDS = "rootfs_install_all_locales; "
+RPM_POSTPROCESS_COMMANDS = ""
#
# Allow distributions to alter when [postponed] package install scripts are run
@@ -56,6 +56,7 @@ fakeroot rootfs_rpm_do_rootfs () {
export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
export INSTALL_PROVIDENAME_RPM=""
export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
+ export INSTALL_COMPLEMENTARY_RPM=""
# Setup base system configuration
mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
@@ -69,6 +70,8 @@ fakeroot rootfs_rpm_do_rootfs () {
package_install_internal_rpm
+ rootfs_install_complementary
+
export D=${IMAGE_ROOTFS}
export OFFLINE_ROOT=${IMAGE_ROOTFS}
export IPKG_OFFLINE_ROOT=${IMAGE_ROOTFS}
@@ -134,11 +137,15 @@ remove_packaging_data_files() {
rm -rf ${IMAGE_ROOTFS}${opkglibdir}
}
-RPM_QUERY_CMD = '${RPM} --root ${IMAGE_ROOTFS} -D "_dbpath ${rpmlibdir}" \
+RPM_QUERY_CMD = '${RPM} --root $INSTALL_ROOTFS_RPM -D "_dbpath ${rpmlibdir}" \
-D "__dbi_txn create nofsync private"'
list_installed_packages() {
- ${RPM_QUERY_CMD} -qa --qf "[%{NAME}\n]"
+ if [ "$1" = "arch" ] ; then
+ ${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH}\n]"
+ else
+ ${RPM_QUERY_CMD} -qa --qf "[%{NAME}\n]"
+ fi
}
get_package_filename() {
@@ -172,21 +179,15 @@ list_package_recommends() {
${RPM_QUERY_CMD} -q --suggests $1
}
-rootfs_check_package_exists() {
- resolve_package_rpm ${RPMCONF_TARGET_BASE}-base_archs.conf $1
-}
-
rootfs_install_packages() {
- # The pkg to be installed here is not controlled by the
- # package_install_internal_rpm, so it may have already been
- # installed(e.g, installed in the first time when generate the
- # rootfs), use '--replacepkgs' to always install them
- for pkg in $@; do
- ${RPM} --root ${IMAGE_ROOTFS} -D "_dbpath ${rpmlibdir}" \
- -D "__dbi_txn create nofsync private" \
- --noscripts --notriggers --noparentdirs --nolinktos \
- --replacepkgs -Uhv $pkg || true
- done
+ # Note - we expect the variables not set here to already have been set
+ export INSTALL_PACKAGES_RPM=""
+ export INSTALL_PACKAGES_ATTEMPTONLY_RPM="`cat $1`"
+ export INSTALL_PROVIDENAME_RPM=""
+ export INSTALL_TASK_RPM="rootfs_install_packages"
+ export INSTALL_COMPLEMENTARY_RPM="1"
+
+ package_install_internal_rpm
}
python () {
diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
new file mode 100755
index 0000000..2427f10
--- /dev/null
+++ b/scripts/oe-pkgdata-util
@@ -0,0 +1,167 @@
+#!/usr/bin/env python
+
+# OpenEmbedded pkgdata utility
+#
+# Written by: Paul Eggleton <paul.eggleton@linux.intel.com>
+#
+# Copyright 2012 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+#
+# Currently only has one function - mapping of packages to their dev/dbg/doc/locale etc.
+# counterparts ("glob" command). Could be extended in future to perform other useful querying
+# functions on the pkgdata though.
+#
+
+import sys
+import os
+import os.path
+import fnmatch
+import re
+
+def usage():
+ print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"");
+
+
+
+def glob(args):
+ if len(args) < 4:
+ usage()
+ sys.exit(1)
+
+ pkgdata_dir = args[0]
+ target_suffix = args[1]
+ pkglist_file = args[2]
+ globs = args[3].split()
+
+ if target_suffix.startswith("-"):
+ target_suffix = target_suffix[1:]
+
+ skipregex = re.compile("-locale-|^locale-base-|-dev$|-doc$|-dbg$|-staticdev$|^kernel-module-")
+
+ mappedpkgs = set()
+ with open(pkglist_file, 'r') as f:
+ for line in f:
+ fields = line.rstrip().split()
+ if len(fields) < 2:
+ continue
+ pkg = fields[0]
+ arch = fields[1]
+ multimach_target_sys = "%s-%s" % (arch, target_suffix)
+
+ # Skip packages for which there is no point applying globs
+ if skipregex.search(pkg):
+ if debug:
+ print("%s -> !!" % pkg)
+ continue
+
+ # Skip packages that already match the globs, so if e.g. a dev package
+ # is already installed and thus in the list, we don't process it any further
+ # Most of these will be caught by skipregex already, but just in case...
+ already = False
+ for g in globs:
+ if fnmatch.fnmatchcase(pkg, g):
+ already = True
+ break
+ if already:
+ if debug:
+ print("%s -> !" % pkg)
+ continue
+
+ # Define some functions
+ def revpkgdata(pkgn):
+ return os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkgn)
+ def fwdpkgdata(pkgn):
+ return os.path.join(pkgdata_dir, multimach_target_sys, "runtime", pkgn)
+ def readpn(pkgdata_file):
+ pn = ""
+ with open(pkgdata_file, 'r') as f:
+ for line in f:
+ if line.startswith("PN:"):
+ pn = line.split(': ')[1].rstrip()
+ return pn
+ def readrenamed(pkgdata_file):
+ renamed = ""
+ pn = os.path.basename(pkgdata_file)
+ with open(pkgdata_file, 'r') as f:
+ for line in f:
+ if line.startswith("PKG_%s:" % pn):
+ renamed = line.split(': ')[1].rstrip()
+ return renamed
+
+ # Main processing loop
+ for g in globs:
+ mappedpkg = ""
+ # First just try substitution (i.e. packagename -> packagename-dev)
+ newpkg = g.replace("*", pkg)
+ revlink = revpkgdata(newpkg)
+ if os.path.exists(revlink):
+ mappedpkg = os.path.basename(os.readlink(revlink))
+ fwdfile = fwdpkgdata(mappedpkg)
+ if os.path.exists(fwdfile):
+ mappedpkg = readrenamed(fwdfile)
+ else:
+ # That didn't work, so now get the PN, substitute that, then map in the other direction
+ revlink = revpkgdata(pkg)
+ if os.path.exists(revlink):
+ pn = readpn(revlink)
+ newpkg = g.replace("*", pn)
+ fwdfile = fwdpkgdata(newpkg)
+ if os.path.exists(fwdfile):
+ mappedpkg = readrenamed(fwdfile)
+ else:
+ # Package doesn't even exist...
+ if debug:
+ print "%s is not a valid package!" % (pkg)
+ break
+
+ if mappedpkg:
+ if debug:
+ print "%s (%s) -> %s" % (pkg, g, mappedpkg)
+ mappedpkgs.add(mappedpkg)
+ else:
+ if debug:
+ print "%s (%s) -> ?" % (pkg, g)
+
+ if debug:
+ print "------"
+
+ print("\n".join(mappedpkgs))
+
+
+
+# Too lazy to use getopt
+debug = False
+noopt = False
+args = []
+for arg in sys.argv[1:]:
+ if arg == "--":
+ noopt = True
+ else:
+ if not noopt:
+ if arg == "-d":
+ debug = True
+ continue
+ args.append(arg)
+
+if len(args) < 1:
+ usage()
+ sys.exit(1)
+
+if args[0] == "glob":
+ glob(args[1:])
+else:
+ usage()
+ sys.exit(1)
diff --git a/scripts/opkg-query-helper.py b/scripts/opkg-query-helper.py
new file mode 100755
index 0000000..b52284b
--- /dev/null
+++ b/scripts/opkg-query-helper.py
@@ -0,0 +1,76 @@
+#!/usr/bin/env python
+
+# OpenEmbedded opkg query helper utility
+#
+# Written by: Paul Eggleton <paul.eggleton@linux.intel.com>
+#
+# Copyright 2012 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+#
+
+
+import sys
+import fileinput
+import re
+
+archmode = False
+filemode = False
+
+args = []
+for arg in sys.argv[1:]:
+ if arg == '-a':
+ archmode = True
+ elif arg == '-f':
+ filemode = True
+ else:
+ args.append(arg)
+
+# Regex for removing version specs after dependency items
+verregex = re.compile(' \([=<>]* [^ )]*\)')
+
+pkg = ""
+ver = ""
+for line in fileinput.input(args):
+ line = line.rstrip()
+ if ': ' in line:
+ if line.startswith("Package:"):
+ pkg = line.split(": ")[1]
+ ver = ""
+ else:
+ if archmode:
+ if line.startswith("Architecture:"):
+ arch = line.split(": ")[1]
+ print("%s %s" % (pkg,arch))
+ elif filemode:
+ if line.startswith("Version:"):
+ ver = line.split(": ")[1]
+ elif line.startswith("Architecture:"):
+ arch = line.split(": ")[1]
+ print("%s %s_%s_%s.ipk" % (pkg,pkg,ver,arch))
+ else:
+ if line.startswith("Depends:"):
+ depval = line.split(": ")[1]
+ deps = depval.split(", ")
+ for dep in deps:
+ dep = verregex.sub('', dep)
+ print("%s|%s" % (pkg,dep))
+ elif line.startswith("Recommends:"):
+ recval = line.split(": ")[1]
+ recs = recval.split(", ")
+ for rec in recs:
+ rec = verregex.sub('', rec)
+ print("%s|%s [REC]" % (pkg, rec))
+
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 5/7] core-image-minimal: clear IMAGE_FEATURES
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
` (3 preceding siblings ...)
2012-07-25 18:42 ` [PATCH 4/7] Rework installation of dev, dbg, doc, and locale packages Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 18:42 ` [PATCH 6/7] classes/image: add staticdev-pkgs IMAGE_FEATURES feature Paul Eggleton
` (3 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
Since dbg/dev packages are no longer added via PACKAGE_GROUP_*, having
dev-pkgs/dbg-pkgs in IMAGE_FEATURES will now affect core-image-minimal.
Set IMAGE_FEATURES to "" in order to avoid this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/recipes-core/images/core-image-minimal.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
index 089a728..3d63a96 100644
--- a/meta/recipes-core/images/core-image-minimal.bb
+++ b/meta/recipes-core/images/core-image-minimal.bb
@@ -2,6 +2,7 @@ DESCRIPTION = "A small image just capable of allowing a device to boot."
IMAGE_INSTALL = "task-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP} ${CORE_IMAGE_EXTRA_INSTALL}"
+IMAGE_FEATURES = ""
IMAGE_LINGUAS = " "
LICENSE = "MIT"
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 6/7] classes/image: add staticdev-pkgs IMAGE_FEATURES feature
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
` (4 preceding siblings ...)
2012-07-25 18:42 ` [PATCH 5/7] core-image-minimal: clear IMAGE_FEATURES Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 18:42 ` [PATCH 7/7] buildhistory: improve performance of image info collection Paul Eggleton
` (2 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
Add a staticdev-pkgs feature that can be added to IMAGE_FEATURES in
order to install all staticdev packages.
Fixes [YOCTO #2531].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/image.bbclass | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 2c605bf..9bdb15e 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -35,7 +35,7 @@ NORMAL_FEATURE_INSTALL_OPTIONAL = "${@' '.join(oe.packagegroup.optional_packages
def normal_groups(d):
"""Return all the IMAGE_FEATURES, with the exception of our special package groups"""
- extras = set(['dev-pkgs', 'doc-pkgs', 'dbg-pkgs'])
+ extras = set(['dev-pkgs', 'staticdev-pkgs', 'doc-pkgs', 'dbg-pkgs'])
features = set(oe.data.typed_value('IMAGE_FEATURES', d))
return features.difference(extras)
@@ -47,6 +47,8 @@ def complementary_globs(featurevar, d):
for feature in features:
if feature == 'dev-pkgs':
globs.append('*-dev')
+ elif feature == 'staticdev-pkgs':
+ globs.append('*-staticdev')
elif feature == 'doc-pkgs':
globs.append('*-doc')
elif feature == 'dbg-pkgs':
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 7/7] buildhistory: improve performance of image info collection
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
` (5 preceding siblings ...)
2012-07-25 18:42 ` [PATCH 6/7] classes/image: add staticdev-pkgs IMAGE_FEATURES feature Paul Eggleton
@ 2012-07-25 18:42 ` Paul Eggleton
2012-07-25 19:08 ` [PATCH 0/7] rootfs generation improvements v3 Koen Kooi
2012-07-25 22:37 ` Paul Eggleton
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 18:42 UTC (permalink / raw)
To: openembedded-core
Reduce the number of calls to the packaging tool, especially in the case
of rpm, using helper utilities to gather the required information more
efficiently where possible.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/buildhistory.bbclass | 51 ++++-----
meta/classes/rootfs_deb.bbclass | 28 +++--
meta/classes/rootfs_ipk.bbclass | 34 +++---
meta/classes/rootfs_rpm.bbclass | 33 +-----
meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c | 117 +++++++++++++++++++--
5 files changed, 162 insertions(+), 101 deletions(-)
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index f0bf849..ddb76e8 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -285,48 +285,43 @@ buildhistory_get_image_installed() {
mkdir -p ${BUILDHISTORY_DIR_IMAGE}
# Get list of installed packages
- list_installed_packages | sort > ${BUILDHISTORY_DIR_IMAGE}/installed-package-names.txt
- INSTALLED_PKGS=`cat ${BUILDHISTORY_DIR_IMAGE}/installed-package-names.txt`
+ pkgcache="${BUILDHISTORY_DIR_IMAGE}/installed-packages.tmp"
+ list_installed_packages file | sort > $pkgcache
+
+ cat $pkgcache | awk '{ print $1 }' > ${BUILDHISTORY_DIR_IMAGE}/installed-package-names.txt
+ cat $pkgcache | awk '{ print $2 }' | xargs -n1 basename > ${BUILDHISTORY_DIR_IMAGE}/installed-packages.txt
+
+ # Produce dependency graph
+ # First, filter out characters that cause issues for dot
+ rootfs_list_installed_depends | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' > ${BUILDHISTORY_DIR_IMAGE}/depends.tmp
+ # Change delimiter from pipe to -> and set style for recommend lines
+ sed -i -e 's:|: -> :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' ${BUILDHISTORY_DIR_IMAGE}/depends.tmp
+ # Add header, sorted and de-duped contents and footer and then delete the temp file
+ echo -e "digraph depends {\n node [shape=plaintext]" > ${BUILDHISTORY_DIR_IMAGE}/depends.dot
+ cat ${BUILDHISTORY_DIR_IMAGE}/depends.tmp | sort | uniq >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
+ echo "}" >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
+ rm ${BUILDHISTORY_DIR_IMAGE}/depends.tmp
- # Produce installed package file and size lists and dependency graph
- echo -n > ${BUILDHISTORY_DIR_IMAGE}/installed-packages.txt
+ # Produce installed package sizes list
echo -n > ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.tmp
- echo -e "digraph depends {\n node [shape=plaintext]" > ${BUILDHISTORY_DIR_IMAGE}/depends.dot
- for pkg in $INSTALLED_PKGS; do
- pkgfile=`get_package_filename $pkg`
- echo `basename $pkgfile` >> ${BUILDHISTORY_DIR_IMAGE}/installed-packages.txt
+ cat $pkgcache | while read pkg pkgfile
+ do
if [ -f $pkgfile ] ; then
pkgsize=`du -k $pkgfile | head -n1 | awk '{ print $1 }'`
echo $pkgsize $pkg >> ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.tmp
fi
-
- deps=`list_package_depends $pkg`
- for dep in $deps ; do
- echo "$pkg OPP $dep;" | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' | sed 's:OPP:->:g'
- done
-
- recs=`list_package_recommends $pkg`
- for rec in $recs ; do
- echo "$pkg OPP $rec [style=dotted];" | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' | sed 's:OPP:->:g'
- done
- done | sort | uniq >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
- echo "}" >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
-
+ done
cat ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.txt
rm ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.tmp
+ # We're now done with the cache, delete it
+ rm $pkgcache
+
# Produce some cut-down graphs (for readability)
grep -v kernel_image ${BUILDHISTORY_DIR_IMAGE}/depends.dot | grep -v kernel_2 | grep -v kernel_3 > ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel.dot
grep -v libc6 ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel.dot | grep -v libgcc > ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel-nolibc.dot
grep -v update_ ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel-nolibc.dot > ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel-nolibc-noupdate.dot
grep -v kernel_module ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel-nolibc-noupdate.dot > ${BUILDHISTORY_DIR_IMAGE}/depends-nokernel-nolibc-noupdate-nomodules.dot
-
- # Workaround for broken shell function dependencies
- if false ; then
- get_package_filename
- list_package_depends
- list_package_recommends
- fi
}
buildhistory_get_imageinfo() {
diff --git a/meta/classes/rootfs_deb.bbclass b/meta/classes/rootfs_deb.bbclass
index a002b1e..750a8ca 100644
--- a/meta/classes/rootfs_deb.bbclass
+++ b/meta/classes/rootfs_deb.bbclass
@@ -96,26 +96,24 @@ list_installed_packages() {
if [ "$1" = "arch" ] ; then
# Here we want the PACKAGE_ARCH not the deb architecture
${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch}\n'
+ elif [ "$1" = "file" ] ; then
+ ${DPKG_QUERY_COMMAND} -W -f='${Package} ${Package}_${Version}_${Architecture}.deb\n' | while read pkg pkgfile
+ do
+ fullpath=`find ${DEPLOY_DIR_DEB} -name "$pkgfile" || true`
+ if [ "$fullpath" = "" ] ; then
+ echo "$pkg $pkgfile"
+ else
+ echo "$pkg $fullpath"
+ fi
+ done
else
${DPKG_QUERY_COMMAND} -W -f='${Package}\n'
fi
}
-get_package_filename() {
- fullname=`find ${DEPLOY_DIR_DEB} -name "$1_*.deb" || true`
- if [ "$fullname" = "" ] ; then
- echo $name
- else
- echo $fullname
- fi
-}
-
-list_package_depends() {
- ${DPKG_QUERY_COMMAND} -s $1 | grep ^Depends | sed -e 's/^Depends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
-}
-
-list_package_recommends() {
- ${DPKG_QUERY_COMMAND} -s $1 | grep ^Recommends | sed -e 's/^Recommends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
+rootfs_list_installed_depends() {
+ # Cheat here a little bit by using the opkg query helper util
+ ${DPKG_QUERY_COMMAND} -W -f='Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n' | opkg-query-helper.py
}
rootfs_install_packages() {
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index 7df97a0..6cdd8f6 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -131,33 +131,23 @@ remove_packaging_data_files() {
list_installed_packages() {
if [ "$1" = "arch" ] ; then
opkg-cl ${IPKG_ARGS_POST} status | opkg-query-helper.py -a
+ elif [ "$1" = "file" ] ; then
+ opkg-cl ${IPKG_ARGS_POST} status | opkg-query-helper.py -f | while read pkg pkgfile
+ do
+ fullpath=`find ${DEPLOY_DIR_IPK} -name "$pkgfile" || true`
+ if [ "$fullpath" = "" ] ; then
+ echo "$pkg $pkgfile"
+ else
+ echo "$pkg $fullpath"
+ fi
+ done
else
opkg-cl ${IPKG_ARGS_POST} list_installed | awk '{ print $1 }'
fi
}
-get_package_filename() {
- set +x
- info=`opkg-cl ${IPKG_ARGS_POST} info $1 | grep -B 7 -A 7 "^Status.* \(\(installed\)\|\(unpacked\)\)" || true`
- name=`echo "${info}" | awk '/^Package/ {printf $2"_"}'`
- name=$name`echo "${info}" | awk -F: '/^Version/ {printf $NF"_"}' | sed 's/^\s*//g'`
- name=$name`echo "${info}" | awk '/^Archi/ {print $2".ipk"}'`
- set -x
-
- fullname=`find ${DEPLOY_DIR_IPK} -name "$name" || true`
- if [ "$fullname" = "" ] ; then
- echo $name
- else
- echo $fullname
- fi
-}
-
-list_package_depends() {
- opkg-cl ${IPKG_ARGS_POST} info $1 | grep ^Depends | sed -e 's/^Depends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
-}
-
-list_package_recommends() {
- opkg-cl ${IPKG_ARGS_POST} info $1 | grep ^Recommends | sed -e 's/^Recommends: //' -e 's/,//g' -e 's:([=<>]* [^ )]*)::g'
+rootfs_list_installed_depends() {
+ opkg-cl ${IPKG_ARGS_POST} status | opkg-query-helper.py
}
rootfs_install_packages() {
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index 1cc4a84..c9258df 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -143,40 +143,15 @@ RPM_QUERY_CMD = '${RPM} --root $INSTALL_ROOTFS_RPM -D "_dbpath ${rpmlibdir}" \
list_installed_packages() {
if [ "$1" = "arch" ] ; then
${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH}\n]"
+ elif [ "$1" = "file" ] ; then
+ ${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{PACKAGEORIGIN}\n]"
else
${RPM_QUERY_CMD} -qa --qf "[%{NAME}\n]"
fi
}
-get_package_filename() {
- resolve_package_rpm ${RPMCONF_TARGET_BASE}-base_archs.conf $1
-}
-
-list_package_depends() {
- pkglist=`list_installed_packages`
-
- # REQUIRE* lists "soft" requirements (which we know as recommends and RPM refers to
- # as "suggests") so filter these out with the help of awk
- for req in `${RPM_QUERY_CMD} -q --qf "[%{REQUIRENAME} %{REQUIREFLAGS}\n]" $1 | awk '{ if( and($2, 0x80000) == 0) print $1 }'`; do
- if echo "$req" | grep -q "^rpmlib" ; then continue ; fi
-
- realpkg=""
- for dep in $pkglist; do
- if [ "$dep" = "$req" ] ; then
- realpkg="1"
- echo $req
- break
- fi
- done
-
- if [ "$realdep" = "" ] ; then
- ${RPM_QUERY_CMD} -q --whatprovides $req --qf "%{NAME}\n"
- fi
- done
-}
-
-list_package_recommends() {
- ${RPM_QUERY_CMD} -q --suggests $1
+rootfs_list_installed_depends() {
+ rpmresolve -d $INSTALL_ROOTFS_RPM/${rpmlibdir}
}
rootfs_install_packages() {
diff --git a/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c b/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
index 9f6cdf2..2d9ed14 100644
--- a/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
+++ b/meta/recipes-devtools/rpm/rpmresolve/rpmresolve.c
@@ -204,10 +204,104 @@ int processPackages(rpmts *ts, int tscount, const char *packagelistfn, int ignor
return rc;
}
+int lookupProvider(rpmts ts, const char *req, char **provider)
+{
+ int rc = 0;
+ rpmmi provmi = rpmtsInitIterator(ts, RPMTAG_PROVIDENAME, req, 0);
+ if(provmi) {
+ Header h;
+ if ((h = rpmmiNext(provmi)) != NULL) {
+ HE_t he = (HE_t) memset(alloca(sizeof(*he)), 0, sizeof(*he));
+ he->tag = RPMTAG_NAME;
+ rc = (headerGet(h, he, 0) != 1);
+ if(rc==0)
+ *provider = strdup((char *)he->p.ptr);
+ }
+ (void)rpmmiFree(provmi);
+ }
+ else {
+ rc = -1;
+ }
+ return rc;
+}
+
+int printDepList(rpmts *ts, int tscount)
+{
+ int rc = 0;
+
+ if( tscount > 1 )
+ printf(">1 database specified with dependency list, using first only\n");
+
+ /* Get list of names */
+ rpmdb db = rpmtsGetRdb(ts[0]);
+ ARGV_t names = NULL;
+ rc = rpmdbMireApply(db, RPMTAG_NAME,
+ RPMMIRE_STRCMP, NULL, &names);
+ int nnames = argvCount(names);
+
+ /* Get list of NVRAs */
+ ARGV_t keys = NULL;
+ rc = rpmdbMireApply(db, RPMTAG_NVRA,
+ RPMMIRE_STRCMP, NULL, &keys);
+ if (keys) {
+ int i, j;
+ HE_t he = (HE_t) memset(alloca(sizeof(*he)), 0, sizeof(*he));
+ int nkeys = argvCount(keys);
+ for(i=0; i<nkeys; i++) {
+ rpmmi mi = rpmtsInitIterator(ts[0], RPMTAG_NVRA, keys[i], 0);
+ Header h;
+ if ((h = rpmmiNext(mi)) != NULL) {
+ /* Get name of package */
+ he->tag = RPMTAG_NAME;
+ rc = (headerGet(h, he, 0) != 1);
+ char *name = strdup((char *)he->p.ptr);
+ /* Get its requires */
+ he->tag = RPMTAG_REQUIRENAME;
+ rc = (headerGet(h, he, 0) != 1);
+ ARGV_t reqs = (ARGV_t)he->p.ptr;
+ /* Get its requireflags */
+ he->tag = RPMTAG_REQUIREFLAGS;
+ rc = (headerGet(h, he, 0) != 1);
+ rpmuint32_t *reqflags = (rpmuint32_t *)he->p.ui32p;
+ for(j=0; j<he->c; j++) {
+ int k;
+ char *prov = NULL;
+ for(k=0; k<nnames; k++) {
+ if(strcmp(names[k], reqs[j]) == 0) {
+ prov = names[k];
+ break;
+ }
+ }
+ if(prov) {
+ if((int)reqflags[j] & 0x80000)
+ printf("%s|%s [REC]\n", name, prov);
+ else
+ printf("%s|%s\n", name, prov);
+ }
+ else {
+ rc = lookupProvider(ts[0], reqs[j], &prov);
+ if(rc==0 && prov) {
+ if((int)reqflags[j] & 0x80000)
+ printf("%s|%s [REC]\n", name, prov);
+ else
+ printf("%s|%s\n", name, prov);
+ free(prov);
+ }
+ }
+ }
+ free(name);
+ }
+ (void)rpmmiFree(mi);
+ }
+ }
+
+ return rc;
+}
+
void usage()
{
fprintf(stderr, "OpenEmbedded rpm resolver utility\n");
- fprintf(stderr, "syntax: rpmresolve [-i] <dblistfile> <packagelistfile>\n");
+ fprintf(stderr, "syntax: rpmresolve [-i] [-d] <dblistfile> <packagelistfile>\n");
}
int main(int argc, char **argv)
@@ -218,13 +312,17 @@ int main(int argc, char **argv)
int i;
int c;
int ignoremissing = 0;
+ int deplistmode = 0;
opterr = 0;
- while ((c = getopt (argc, argv, "i")) != -1) {
+ while ((c = getopt (argc, argv, "id")) != -1) {
switch (c) {
case 'i':
ignoremissing = 1;
break;
+ case 'd':
+ deplistmode = 1;
+ break;
case '?':
if(isprint(optopt))
fprintf(stderr, "Unknown option `-%c'.\n", optopt);
@@ -258,12 +356,17 @@ int main(int argc, char **argv)
return 1;
}
- if( argc - optind < 2 ) {
- fprintf(stderr, "Please specify package list file\n");
- return 1;
+ if(deplistmode) {
+ rc = printDepList(ts, tscount);
+ }
+ else {
+ if( argc - optind < 2 ) {
+ fprintf(stderr, "Please specify package list file\n");
+ return 1;
+ }
+ const char *pkglistfn = argv[optind+1];
+ rc = processPackages(ts, tscount, pkglistfn, ignoremissing);
}
- const char *pkglistfn = argv[optind+1];
- rc = processPackages(ts, tscount, pkglistfn, ignoremissing);
for(i=0; i<tscount; i++)
(void) rpmtsCloseDB(ts[i]);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 0/7] rootfs generation improvements v3
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
` (6 preceding siblings ...)
2012-07-25 18:42 ` [PATCH 7/7] buildhistory: improve performance of image info collection Paul Eggleton
@ 2012-07-25 19:08 ` Koen Kooi
2012-07-25 20:28 ` Paul Eggleton
2012-07-25 22:37 ` Paul Eggleton
8 siblings, 1 reply; 11+ messages in thread
From: Koen Kooi @ 2012-07-25 19:08 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Op 25 jul. 2012, om 20:42 heeft Paul Eggleton het volgende geschreven:
> Fix up handling of "complementary" package installation (dev, dbg, doc,
> etc.) and introduce a staticdev-pkgs IMAGE_FEATURES feature.
>
> Changes since v2:
>
> * Clear IMAGE_FEATURES in core-image-minimal so that it doesn't
> have dev/dbg packages added (since the default value of IMAGE_FEATURES
> contains dev-pkgs and dbg-pkgs and this will now apply to any image).
Wait, what?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 0/7] rootfs generation improvements v3
2012-07-25 19:08 ` [PATCH 0/7] rootfs generation improvements v3 Koen Kooi
@ 2012-07-25 20:28 ` Paul Eggleton
0 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 20:28 UTC (permalink / raw)
To: openembedded-core; +Cc: Koen Kooi
On Wednesday 25 July 2012 21:08:53 Koen Kooi wrote:
> Op 25 jul. 2012, om 20:42 heeft Paul Eggleton het volgende geschreven:
> > Fix up handling of "complementary" package installation (dev, dbg, doc,
> > etc.) and introduce a staticdev-pkgs IMAGE_FEATURES feature.
> >
> > Changes since v2:
> >
> > * Clear IMAGE_FEATURES in core-image-minimal so that it doesn't
> >
> > have dev/dbg packages added (since the default value of IMAGE_FEATURES
> > contains dev-pkgs and dbg-pkgs and this will now apply to any image).
>
> Wait, what?
Ah, right... not sure what I was thinking - there is no such default value. I
have dropped that patch from the branch.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 0/7] rootfs generation improvements v3
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
` (7 preceding siblings ...)
2012-07-25 19:08 ` [PATCH 0/7] rootfs generation improvements v3 Koen Kooi
@ 2012-07-25 22:37 ` Paul Eggleton
8 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2012-07-25 22:37 UTC (permalink / raw)
To: openembedded-core
On Wednesday 25 July 2012 19:42:50 Paul Eggleton wrote:
> * Fix errors with the ipk and deb backends if no complementary packages
> need to be installed
This change is totally broken as well. Shows what happens when you rush to get
something out before you leave for the evening. I might as well send out a v4
at this point.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2012-07-25 22:48 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-07-25 18:42 [PATCH 0/7] rootfs generation improvements v3 Paul Eggleton
2012-07-25 18:42 ` [PATCH 1/7] package.bbclass: Create symlinks for packages with different packaged name Paul Eggleton
2012-07-25 18:42 ` [PATCH 2/7] classes/rootfs_rpm: improve speed of RPM rootfs construction Paul Eggleton
2012-07-25 18:42 ` [PATCH 3/7] classes/package_deb: add PackageArch field Paul Eggleton
2012-07-25 18:42 ` [PATCH 4/7] Rework installation of dev, dbg, doc, and locale packages Paul Eggleton
2012-07-25 18:42 ` [PATCH 5/7] core-image-minimal: clear IMAGE_FEATURES Paul Eggleton
2012-07-25 18:42 ` [PATCH 6/7] classes/image: add staticdev-pkgs IMAGE_FEATURES feature Paul Eggleton
2012-07-25 18:42 ` [PATCH 7/7] buildhistory: improve performance of image info collection Paul Eggleton
2012-07-25 19:08 ` [PATCH 0/7] rootfs generation improvements v3 Koen Kooi
2012-07-25 20:28 ` Paul Eggleton
2012-07-25 22:37 ` Paul Eggleton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.