* [RFC] buildhistory: Record size of installed package not compressed archive
@ 2013-06-30 1:36 Martin Jansa
2013-07-01 16:53 ` Otavio Salvador
2013-07-04 6:53 ` Saul Wold
0 siblings, 2 replies; 7+ messages in thread
From: Martin Jansa @ 2013-06-30 1:36 UTC (permalink / raw)
To: openembedded-core
* usually it's more important to know how much space will each
package take on target device then size of compressed package
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/buildhistory.bbclass | 11 ++++----
scripts/oe-pkgdata-util | 59 ++++++++++++++++++++++++++++++++++++---
2 files changed, 61 insertions(+), 9 deletions(-)
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 36e7fe1..1ebe68e 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -317,6 +317,8 @@ buildhistory_get_installed() {
# Get list of installed packages
pkgcache="$1/installed-packages.tmp"
list_installed_packages file | sort > $pkgcache
+ pkgcachearch="$1/installed-packages-arch.tmp"
+ list_installed_packages arch | sort > $pkgcachearch
cat $pkgcache | awk '{ print $1 }' > $1/installed-package-names.txt
if [ -s $pkgcache ] ; then
@@ -338,18 +340,17 @@ buildhistory_get_installed() {
# Produce installed package sizes list
printf "" > $1/installed-package-sizes.tmp
- cat $pkgcache | while read pkg pkgfile
+ cat $pkgcachearch | while read pkg arch
do
- if [ -f $pkgfile ] ; then
- pkgsize=`du -k $pkgfile | head -n1 | awk '{ print $1 }'`
- echo $pkgsize $pkg >> $1/installed-package-sizes.tmp
- fi
+ size=`oe-pkgdata-util read_values ${TMPDIR}/pkgdata ${TARGET_VENDOR}-${TARGET_OS} "PKGSIZE" ${pkg}_${arch}`
+ echo "$size $pkg" >> $1/installed-package-sizes.tmp
done
cat $1/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > $1/installed-package-sizes.txt
rm $1/installed-package-sizes.tmp
# We're now done with the cache, delete it
rm $pkgcache
+ rm $pkgcachearch
if [ "$2" != "sdk" ] ; then
# Produce some cut-down graphs (for readability)
diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 629b2d5..88b8122 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -20,9 +20,12 @@
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
#
-# Currently only has one function - mapping of packages to their dev/dbg/doc/locale etc.
-# counterparts ("glob" command). Could be extended in future to perform other useful querying
-# functions on the pkgdata though.
+# Currently only has two functions
+# 1) glob - mapping of packages to their dev/dbg/doc/locale etc. counterparts.
+# 2) read_values - mapping of packagenames to their location in
+# pkgdata and then returns value of selected variable (e.g. PKGSIZE)
+# Could be extended in future to perform other useful querying functions on the
+# pkgdata though.
#
import sys
@@ -32,7 +35,8 @@ import fnmatch
import re
def usage():
- print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"");
+ print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"\n \
+ read_values [-d] <pkgdatadir> <vendor-os> <variable-name> \"<ipk filenames>\"");
@@ -151,7 +155,52 @@ def glob(args):
print("\n".join(mappedpkgs))
+def read_values(args):
+ if len(args) < 4:
+ usage()
+ sys.exit(1)
+
+ pkgdata_dir = args[0]
+ target_suffix = args[1]
+ var = args[2]
+ packages = args[3].split()
+ if target_suffix.startswith("-"):
+ target_suffix = target_suffix[1:]
+
+ def readvar(pkgdata_file, var):
+ val = ""
+ with open(pkgdata_file, 'r') as f:
+ for line in f:
+ if line.startswith(var + ":"):
+ val = line.split(': ')[1].rstrip()
+ return val
+
+ if debug:
+ print "read_values('%s', '%s', '%s' '%s'" % (pkgdata_dir, target_suffix, var, packages)
+ for package in packages:
+ pkg_split = package.split('_')
+ pkg_name = pkg_split[0]
+ pkg_arch = '_'.join(pkg_split[1:])
+ if debug:
+ print "package: name: '%s', arch: '%s'" % (pkg_name, pkg_arch)
+ multimach_target_sys = "%s-%s" % (pkg_arch, target_suffix)
+ revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
+ if debug:
+ print(revlink)
+ if not os.path.exists(revlink):
+ # [YOCTO #4227] try to drop -gnueabi from TARGET_OS
+ multimach_target_sys = '-'.join(multimach_target_sys.split('-')[:-1])
+ revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
+ if debug:
+ print(revlink)
+ if os.path.exists(revlink):
+ mappedpkg = os.path.basename(os.readlink(revlink))
+ qvar = var
+ if qvar == "PKGSIZE":
+ # append packagename
+ qvar = "%s_%s" % (var, mappedpkg)
+ print(readvar(revlink, qvar))
# Too lazy to use getopt
debug = False
@@ -173,6 +222,8 @@ if len(args) < 1:
if args[0] == "glob":
glob(args[1:])
+if args[0] == "read_values":
+ read_values(args[1:])
else:
usage()
sys.exit(1)
--
1.8.2.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [RFC] buildhistory: Record size of installed package not compressed archive
2013-06-30 1:36 [RFC] buildhistory: Record size of installed package not compressed archive Martin Jansa
@ 2013-07-01 16:53 ` Otavio Salvador
2013-07-04 6:53 ` Saul Wold
1 sibling, 0 replies; 7+ messages in thread
From: Otavio Salvador @ 2013-07-01 16:53 UTC (permalink / raw)
To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer
On Sat, Jun 29, 2013 at 10:36 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
> * usually it's more important to know how much space will each
> package take on target device then size of compressed package
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
I fully agree; this is a more useful information indeed.
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC] buildhistory: Record size of installed package not compressed archive
2013-06-30 1:36 [RFC] buildhistory: Record size of installed package not compressed archive Martin Jansa
2013-07-01 16:53 ` Otavio Salvador
@ 2013-07-04 6:53 ` Saul Wold
2013-07-04 8:30 ` Martin Jansa
1 sibling, 1 reply; 7+ messages in thread
From: Saul Wold @ 2013-07-04 6:53 UTC (permalink / raw)
To: Martin Jansa; +Cc: openembedded-core
On 06/29/2013 06:36 PM, Martin Jansa wrote:
> * usually it's more important to know how much space will each
> package take on target device then size of compressed package
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
> meta/classes/buildhistory.bbclass | 11 ++++----
> scripts/oe-pkgdata-util | 59 ++++++++++++++++++++++++++++++++++++---
> 2 files changed, 61 insertions(+), 9 deletions(-)
>
I am not sure how, but this patch is causing some failures in poky with
core-image-sato build, it seems that that the args coming from the
rootfs_install_complementary get goofed up somehow, not sure but I
bisected down to this patch.
the -poky seems to have a space added so it gets -poky -linux on the
command line for oe-pkgdata-utils.
Sau!
> diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> index 36e7fe1..1ebe68e 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -317,6 +317,8 @@ buildhistory_get_installed() {
> # Get list of installed packages
> pkgcache="$1/installed-packages.tmp"
> list_installed_packages file | sort > $pkgcache
> + pkgcachearch="$1/installed-packages-arch.tmp"
> + list_installed_packages arch | sort > $pkgcachearch
>
> cat $pkgcache | awk '{ print $1 }' > $1/installed-package-names.txt
> if [ -s $pkgcache ] ; then
> @@ -338,18 +340,17 @@ buildhistory_get_installed() {
>
> # Produce installed package sizes list
> printf "" > $1/installed-package-sizes.tmp
> - cat $pkgcache | while read pkg pkgfile
> + cat $pkgcachearch | while read pkg arch
> do
> - if [ -f $pkgfile ] ; then
> - pkgsize=`du -k $pkgfile | head -n1 | awk '{ print $1 }'`
> - echo $pkgsize $pkg >> $1/installed-package-sizes.tmp
> - fi
> + size=`oe-pkgdata-util read_values ${TMPDIR}/pkgdata ${TARGET_VENDOR}-${TARGET_OS} "PKGSIZE" ${pkg}_${arch}`
> + echo "$size $pkg" >> $1/installed-package-sizes.tmp
> done
> cat $1/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > $1/installed-package-sizes.txt
> rm $1/installed-package-sizes.tmp
>
> # We're now done with the cache, delete it
> rm $pkgcache
> + rm $pkgcachearch
>
> if [ "$2" != "sdk" ] ; then
> # Produce some cut-down graphs (for readability)
> diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
> index 629b2d5..88b8122 100755
> --- a/scripts/oe-pkgdata-util
> +++ b/scripts/oe-pkgdata-util
> @@ -20,9 +20,12 @@
> # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> #
> #
> -# Currently only has one function - mapping of packages to their dev/dbg/doc/locale etc.
> -# counterparts ("glob" command). Could be extended in future to perform other useful querying
> -# functions on the pkgdata though.
> +# Currently only has two functions
> +# 1) glob - mapping of packages to their dev/dbg/doc/locale etc. counterparts.
> +# 2) read_values - mapping of packagenames to their location in
> +# pkgdata and then returns value of selected variable (e.g. PKGSIZE)
> +# Could be extended in future to perform other useful querying functions on the
> +# pkgdata though.
> #
>
> import sys
> @@ -32,7 +35,8 @@ import fnmatch
> import re
>
> def usage():
> - print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"");
> + print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"\n \
> + read_values [-d] <pkgdatadir> <vendor-os> <variable-name> \"<ipk filenames>\"");
>
>
>
> @@ -151,7 +155,52 @@ def glob(args):
>
> print("\n".join(mappedpkgs))
>
> +def read_values(args):
> + if len(args) < 4:
> + usage()
> + sys.exit(1)
> +
> + pkgdata_dir = args[0]
> + target_suffix = args[1]
> + var = args[2]
> + packages = args[3].split()
>
> + if target_suffix.startswith("-"):
> + target_suffix = target_suffix[1:]
> +
> + def readvar(pkgdata_file, var):
> + val = ""
> + with open(pkgdata_file, 'r') as f:
> + for line in f:
> + if line.startswith(var + ":"):
> + val = line.split(': ')[1].rstrip()
> + return val
> +
> + if debug:
> + print "read_values('%s', '%s', '%s' '%s'" % (pkgdata_dir, target_suffix, var, packages)
> + for package in packages:
> + pkg_split = package.split('_')
> + pkg_name = pkg_split[0]
> + pkg_arch = '_'.join(pkg_split[1:])
> + if debug:
> + print "package: name: '%s', arch: '%s'" % (pkg_name, pkg_arch)
> + multimach_target_sys = "%s-%s" % (pkg_arch, target_suffix)
> + revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
> + if debug:
> + print(revlink)
> + if not os.path.exists(revlink):
> + # [YOCTO #4227] try to drop -gnueabi from TARGET_OS
> + multimach_target_sys = '-'.join(multimach_target_sys.split('-')[:-1])
> + revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
> + if debug:
> + print(revlink)
> + if os.path.exists(revlink):
> + mappedpkg = os.path.basename(os.readlink(revlink))
> + qvar = var
> + if qvar == "PKGSIZE":
> + # append packagename
> + qvar = "%s_%s" % (var, mappedpkg)
> + print(readvar(revlink, qvar))
>
> # Too lazy to use getopt
> debug = False
> @@ -173,6 +222,8 @@ if len(args) < 1:
>
> if args[0] == "glob":
> glob(args[1:])
> +if args[0] == "read_values":
> + read_values(args[1:])
> else:
> usage()
> sys.exit(1)
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC] buildhistory: Record size of installed package not compressed archive
2013-07-04 6:53 ` Saul Wold
@ 2013-07-04 8:30 ` Martin Jansa
2013-07-06 10:04 ` [PATCHv2] " Martin Jansa
0 siblings, 1 reply; 7+ messages in thread
From: Martin Jansa @ 2013-07-04 8:30 UTC (permalink / raw)
To: Saul Wold; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 6598 bytes --]
On Wed, Jul 03, 2013 at 11:53:06PM -0700, Saul Wold wrote:
> On 06/29/2013 06:36 PM, Martin Jansa wrote:
> > * usually it's more important to know how much space will each
> > package take on target device then size of compressed package
> >
> > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> > ---
> > meta/classes/buildhistory.bbclass | 11 ++++----
> > scripts/oe-pkgdata-util | 59 ++++++++++++++++++++++++++++++++++++---
> > 2 files changed, 61 insertions(+), 9 deletions(-)
> >
>
> I am not sure how, but this patch is causing some failures in poky with
> core-image-sato build, it seems that that the args coming from the
> rootfs_install_complementary get goofed up somehow, not sure but I
> bisected down to this patch.
>
> the -poky seems to have a space added so it gets -poky -linux on the
> command line for oe-pkgdata-utils.
I'm sorry I haven't replied to that patch yet, it was sent only as RFC
to see if people agree that we should just replace current sizes with
size when installed on target (I have different version which records
both, but is a bit slower and resulting file bigger without good
reasons)
I'm aware of the issue and will fix it soon.
> > diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> > index 36e7fe1..1ebe68e 100644
> > --- a/meta/classes/buildhistory.bbclass
> > +++ b/meta/classes/buildhistory.bbclass
> > @@ -317,6 +317,8 @@ buildhistory_get_installed() {
> > # Get list of installed packages
> > pkgcache="$1/installed-packages.tmp"
> > list_installed_packages file | sort > $pkgcache
> > + pkgcachearch="$1/installed-packages-arch.tmp"
> > + list_installed_packages arch | sort > $pkgcachearch
> >
> > cat $pkgcache | awk '{ print $1 }' > $1/installed-package-names.txt
> > if [ -s $pkgcache ] ; then
> > @@ -338,18 +340,17 @@ buildhistory_get_installed() {
> >
> > # Produce installed package sizes list
> > printf "" > $1/installed-package-sizes.tmp
> > - cat $pkgcache | while read pkg pkgfile
> > + cat $pkgcachearch | while read pkg arch
> > do
> > - if [ -f $pkgfile ] ; then
> > - pkgsize=`du -k $pkgfile | head -n1 | awk '{ print $1 }'`
> > - echo $pkgsize $pkg >> $1/installed-package-sizes.tmp
> > - fi
> > + size=`oe-pkgdata-util read_values ${TMPDIR}/pkgdata ${TARGET_VENDOR}-${TARGET_OS} "PKGSIZE" ${pkg}_${arch}`
> > + echo "$size $pkg" >> $1/installed-package-sizes.tmp
> > done
> > cat $1/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > $1/installed-package-sizes.txt
> > rm $1/installed-package-sizes.tmp
> >
> > # We're now done with the cache, delete it
> > rm $pkgcache
> > + rm $pkgcachearch
> >
> > if [ "$2" != "sdk" ] ; then
> > # Produce some cut-down graphs (for readability)
> > diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
> > index 629b2d5..88b8122 100755
> > --- a/scripts/oe-pkgdata-util
> > +++ b/scripts/oe-pkgdata-util
> > @@ -20,9 +20,12 @@
> > # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> > #
> > #
> > -# Currently only has one function - mapping of packages to their dev/dbg/doc/locale etc.
> > -# counterparts ("glob" command). Could be extended in future to perform other useful querying
> > -# functions on the pkgdata though.
> > +# Currently only has two functions
> > +# 1) glob - mapping of packages to their dev/dbg/doc/locale etc. counterparts.
> > +# 2) read_values - mapping of packagenames to their location in
> > +# pkgdata and then returns value of selected variable (e.g. PKGSIZE)
> > +# Could be extended in future to perform other useful querying functions on the
> > +# pkgdata though.
> > #
> >
> > import sys
> > @@ -32,7 +35,8 @@ import fnmatch
> > import re
> >
> > def usage():
> > - print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"");
> > + print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"\n \
> > + read_values [-d] <pkgdatadir> <vendor-os> <variable-name> \"<ipk filenames>\"");
> >
> >
> >
> > @@ -151,7 +155,52 @@ def glob(args):
> >
> > print("\n".join(mappedpkgs))
> >
> > +def read_values(args):
> > + if len(args) < 4:
> > + usage()
> > + sys.exit(1)
> > +
> > + pkgdata_dir = args[0]
> > + target_suffix = args[1]
> > + var = args[2]
> > + packages = args[3].split()
> >
> > + if target_suffix.startswith("-"):
> > + target_suffix = target_suffix[1:]
> > +
> > + def readvar(pkgdata_file, var):
> > + val = ""
> > + with open(pkgdata_file, 'r') as f:
> > + for line in f:
> > + if line.startswith(var + ":"):
> > + val = line.split(': ')[1].rstrip()
> > + return val
> > +
> > + if debug:
> > + print "read_values('%s', '%s', '%s' '%s'" % (pkgdata_dir, target_suffix, var, packages)
> > + for package in packages:
> > + pkg_split = package.split('_')
> > + pkg_name = pkg_split[0]
> > + pkg_arch = '_'.join(pkg_split[1:])
> > + if debug:
> > + print "package: name: '%s', arch: '%s'" % (pkg_name, pkg_arch)
> > + multimach_target_sys = "%s-%s" % (pkg_arch, target_suffix)
> > + revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
> > + if debug:
> > + print(revlink)
> > + if not os.path.exists(revlink):
> > + # [YOCTO #4227] try to drop -gnueabi from TARGET_OS
> > + multimach_target_sys = '-'.join(multimach_target_sys.split('-')[:-1])
> > + revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
> > + if debug:
> > + print(revlink)
> > + if os.path.exists(revlink):
> > + mappedpkg = os.path.basename(os.readlink(revlink))
> > + qvar = var
> > + if qvar == "PKGSIZE":
> > + # append packagename
> > + qvar = "%s_%s" % (var, mappedpkg)
> > + print(readvar(revlink, qvar))
> >
> > # Too lazy to use getopt
> > debug = False
> > @@ -173,6 +222,8 @@ if len(args) < 1:
> >
> > if args[0] == "glob":
> > glob(args[1:])
> > +if args[0] == "read_values":
> > + read_values(args[1:])
> > else:
> > usage()
> > sys.exit(1)
> >
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCHv2] buildhistory: Record size of installed package not compressed archive
2013-07-04 8:30 ` Martin Jansa
@ 2013-07-06 10:04 ` Martin Jansa
2013-07-08 15:47 ` Paul Eggleton
0 siblings, 1 reply; 7+ messages in thread
From: Martin Jansa @ 2013-07-06 10:04 UTC (permalink / raw)
To: openembedded-core
* usually it's more important to know how much space will each
package take on target device then size of compressed package
* example for libewebkit0 with 4 different architectures, interesting
that om_gta02 .ipk is bigger but it's smaller when installed
before:
MACHINE DEFAULTTUNE SIZE (.ipk file)
om_gta04 cortexa8t-neon 15996 KiB libewebkit0
qemux86_64 x86-64 16992 KiB libewebkit0
spitz xscale 16148 KiB libewebkit0
om_gta02 arm920t 16260 KiB libewebkit0
after:
MACHINE DEFAULTTUNE SIZE (installed)
om_gta04 cortexa8t-neon 60544 KiB libewebkit0
qemux86_64 x86-64 63720 KiB libewebkit0
spitz xscale 60588 KiB libewebkit0
om_gta02 arm920t 56268 KiB libewebkit0
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/buildhistory.bbclass | 11 ++++----
scripts/oe-pkgdata-util | 59 ++++++++++++++++++++++++++++++++++++---
2 files changed, 61 insertions(+), 9 deletions(-)
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 36e7fe1..1ebe68e 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -317,6 +317,8 @@ buildhistory_get_installed() {
# Get list of installed packages
pkgcache="$1/installed-packages.tmp"
list_installed_packages file | sort > $pkgcache
+ pkgcachearch="$1/installed-packages-arch.tmp"
+ list_installed_packages arch | sort > $pkgcachearch
cat $pkgcache | awk '{ print $1 }' > $1/installed-package-names.txt
if [ -s $pkgcache ] ; then
@@ -338,18 +340,17 @@ buildhistory_get_installed() {
# Produce installed package sizes list
printf "" > $1/installed-package-sizes.tmp
- cat $pkgcache | while read pkg pkgfile
+ cat $pkgcachearch | while read pkg arch
do
- if [ -f $pkgfile ] ; then
- pkgsize=`du -k $pkgfile | head -n1 | awk '{ print $1 }'`
- echo $pkgsize $pkg >> $1/installed-package-sizes.tmp
- fi
+ size=`oe-pkgdata-util read_values ${TMPDIR}/pkgdata ${TARGET_VENDOR}-${TARGET_OS} "PKGSIZE" ${pkg}_${arch}`
+ echo "$size $pkg" >> $1/installed-package-sizes.tmp
done
cat $1/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > $1/installed-package-sizes.txt
rm $1/installed-package-sizes.tmp
# We're now done with the cache, delete it
rm $pkgcache
+ rm $pkgcachearch
if [ "$2" != "sdk" ] ; then
# Produce some cut-down graphs (for readability)
diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 629b2d5..731e269 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -20,9 +20,12 @@
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
#
-# Currently only has one function - mapping of packages to their dev/dbg/doc/locale etc.
-# counterparts ("glob" command). Could be extended in future to perform other useful querying
-# functions on the pkgdata though.
+# Currently only has two functions
+# 1) glob - mapping of packages to their dev/dbg/doc/locale etc. counterparts.
+# 2) read_values - mapping of packagenames to their location in
+# pkgdata and then returns value of selected variable (e.g. PKGSIZE)
+# Could be extended in future to perform other useful querying functions on the
+# pkgdata though.
#
import sys
@@ -32,7 +35,8 @@ import fnmatch
import re
def usage():
- print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"");
+ print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"\n \
+ read_values [-d] <pkgdatadir> <vendor-os> <variable-name> \"<package-name>_<package_architecture>\"");
@@ -151,7 +155,52 @@ def glob(args):
print("\n".join(mappedpkgs))
+def read_values(args):
+ if len(args) < 4:
+ usage()
+ sys.exit(1)
+
+ pkgdata_dir = args[0]
+ target_suffix = args[1]
+ var = args[2]
+ packages = args[3].split()
+ if target_suffix.startswith("-"):
+ target_suffix = target_suffix[1:]
+
+ def readvar(pkgdata_file, var):
+ val = ""
+ with open(pkgdata_file, 'r') as f:
+ for line in f:
+ if line.startswith(var + ":"):
+ val = line.split(': ')[1].rstrip()
+ return val
+
+ if debug:
+ print "read_values('%s', '%s', '%s' '%s'" % (pkgdata_dir, target_suffix, var, packages)
+ for package in packages:
+ pkg_split = package.split('_')
+ pkg_name = pkg_split[0]
+ pkg_arch = '_'.join(pkg_split[1:])
+ if debug:
+ print "package: name: '%s', arch: '%s'" % (pkg_name, pkg_arch)
+ multimach_target_sys = "%s-%s" % (pkg_arch, target_suffix)
+ revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
+ if debug:
+ print(revlink)
+ if not os.path.exists(revlink):
+ # [YOCTO #4227] try to drop -gnueabi from TARGET_OS
+ multimach_target_sys = '-'.join(multimach_target_sys.split('-')[:-1])
+ revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
+ if debug:
+ print(revlink)
+ if os.path.exists(revlink):
+ mappedpkg = os.path.basename(os.readlink(revlink))
+ qvar = var
+ if qvar == "PKGSIZE":
+ # append packagename
+ qvar = "%s_%s" % (var, mappedpkg)
+ print(readvar(revlink, qvar))
# Too lazy to use getopt
debug = False
@@ -173,6 +222,8 @@ if len(args) < 1:
if args[0] == "glob":
glob(args[1:])
+elif args[0] == "read_values":
+ read_values(args[1:])
else:
usage()
sys.exit(1)
--
1.8.2.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCHv2] buildhistory: Record size of installed package not compressed archive
2013-07-06 10:04 ` [PATCHv2] " Martin Jansa
@ 2013-07-08 15:47 ` Paul Eggleton
2013-07-08 16:40 ` Martin Jansa
0 siblings, 1 reply; 7+ messages in thread
From: Paul Eggleton @ 2013-07-08 15:47 UTC (permalink / raw)
To: Martin Jansa; +Cc: openembedded-core
On Saturday 06 July 2013 12:04:04 Martin Jansa wrote:
> * usually it's more important to know how much space will each
> package take on target device then size of compressed package
> * example for libewebkit0 with 4 different architectures, interesting
> that om_gta02 .ipk is bigger but it's smaller when installed
>
> before:
> MACHINE DEFAULTTUNE SIZE (.ipk file)
> om_gta04 cortexa8t-neon 15996 KiB libewebkit0
> qemux86_64 x86-64 16992 KiB libewebkit0
> spitz xscale 16148 KiB libewebkit0
> om_gta02 arm920t 16260 KiB libewebkit0
>
> after:
> MACHINE DEFAULTTUNE SIZE (installed)
> om_gta04 cortexa8t-neon 60544 KiB libewebkit0
> qemux86_64 x86-64 63720 KiB libewebkit0
> spitz xscale 60588 KiB libewebkit0
> om_gta02 arm920t 56268 KiB libewebkit0
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
> meta/classes/buildhistory.bbclass | 11 ++++----
> scripts/oe-pkgdata-util | 59
> ++++++++++++++++++++++++++++++++++++--- 2 files changed, 61 insertions(+),
> 9 deletions(-)
>
> diff --git a/meta/classes/buildhistory.bbclass
> b/meta/classes/buildhistory.bbclass index 36e7fe1..1ebe68e 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -317,6 +317,8 @@ buildhistory_get_installed() {
> # Get list of installed packages
> pkgcache="$1/installed-packages.tmp"
> list_installed_packages file | sort > $pkgcache
> + pkgcachearch="$1/installed-packages-arch.tmp"
> + list_installed_packages arch | sort > $pkgcachearch
If possible I would prefer to avoid having to query the installed package list
twice; I know that would mean changing every backend. I can look at this if
you're short on time.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCHv2] buildhistory: Record size of installed package not compressed archive
2013-07-08 15:47 ` Paul Eggleton
@ 2013-07-08 16:40 ` Martin Jansa
0 siblings, 0 replies; 7+ messages in thread
From: Martin Jansa @ 2013-07-08 16:40 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 2268 bytes --]
On Mon, Jul 08, 2013 at 04:47:57PM +0100, Paul Eggleton wrote:
> On Saturday 06 July 2013 12:04:04 Martin Jansa wrote:
> > * usually it's more important to know how much space will each
> > package take on target device then size of compressed package
> > * example for libewebkit0 with 4 different architectures, interesting
> > that om_gta02 .ipk is bigger but it's smaller when installed
> >
> > before:
> > MACHINE DEFAULTTUNE SIZE (.ipk file)
> > om_gta04 cortexa8t-neon 15996 KiB libewebkit0
> > qemux86_64 x86-64 16992 KiB libewebkit0
> > spitz xscale 16148 KiB libewebkit0
> > om_gta02 arm920t 16260 KiB libewebkit0
> >
> > after:
> > MACHINE DEFAULTTUNE SIZE (installed)
> > om_gta04 cortexa8t-neon 60544 KiB libewebkit0
> > qemux86_64 x86-64 63720 KiB libewebkit0
> > spitz xscale 60588 KiB libewebkit0
> > om_gta02 arm920t 56268 KiB libewebkit0
> >
> > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> > ---
> > meta/classes/buildhistory.bbclass | 11 ++++----
> > scripts/oe-pkgdata-util | 59
> > ++++++++++++++++++++++++++++++++++++--- 2 files changed, 61 insertions(+),
> > 9 deletions(-)
> >
> > diff --git a/meta/classes/buildhistory.bbclass
> > b/meta/classes/buildhistory.bbclass index 36e7fe1..1ebe68e 100644
> > --- a/meta/classes/buildhistory.bbclass
> > +++ b/meta/classes/buildhistory.bbclass
> > @@ -317,6 +317,8 @@ buildhistory_get_installed() {
> > # Get list of installed packages
> > pkgcache="$1/installed-packages.tmp"
> > list_installed_packages file | sort > $pkgcache
> > + pkgcachearch="$1/installed-packages-arch.tmp"
> > + list_installed_packages arch | sort > $pkgcachearch
>
> If possible I would prefer to avoid having to query the installed package list
> twice; I know that would mean changing every backend. I can look at this if
> you're short on time.
I was thinking about adding "list_installed_packages filearch" or
something like that, but because of lack of time I could spend on this
one I decided to skip it. Feel free to improve it.
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2013-07-08 16:40 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-06-30 1:36 [RFC] buildhistory: Record size of installed package not compressed archive Martin Jansa
2013-07-01 16:53 ` Otavio Salvador
2013-07-04 6:53 ` Saul Wold
2013-07-04 8:30 ` Martin Jansa
2013-07-06 10:04 ` [PATCHv2] " Martin Jansa
2013-07-08 15:47 ` Paul Eggleton
2013-07-08 16:40 ` Martin Jansa
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.