[PATCH 0/1] buildhistory: files-in-package.txt

[PATCH 0/1] buildhistory: files-in-package.txt

[Juro Bystricky]

When comparing two build histories of the same image, but built using
hosts with different filesystems, I noticed that although the images/packages were
binary identical, the build histories differed. This was caused by the
fact that the generated files (i.e. files-in-package.txt) listed different
sizes for the same folder. This depends on the host build filesystem.
For example, on EXT4 we see:
 
drwxr-xr-x root       root             4096 ./etc
 
and on tmpfs (I use ramdisk when I can), I see:
 
drwxr-xr-x root       root              100 ./etc

This makes comparison of the build histories difficult, as virtually all
generated files can show up as different. Also, AFAIK, the "folder size"
is meaningless anyway, as it has no impact on the package/image size.

The attached patch addresses the problem by not printing the folder sizes.
The above example would become:

drwxr-xr-x root       root                  ./etc

This is achieved by piping the lines through awk code. The awk code simply
replaces the size by spaces, while keeping the formatting intact. I admit
the code is not pretty, but it gets the job done. The ugliness is mostly
due to the effort to preserve the original white spaces.

Juro Bystricky (1):
  buildhistory.bbclass: buildhistory_list_files: remove folder size

 meta/classes/buildhistory.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

-- 
2.7.4

[PATCH 1/1] buildhistory.bbclass: buildhistory_list_files: remove folder size

[Juro Bystricky]

"buildhistory_list_files" generates text files containing list of files
that go into various packages/images. The file list contains file sizes as
well. However, the listed "file size" of a directory entry depends
on the file system of the build host. So, for example, building on EXT4 the
folder size is always listed as 4096 bytes. The same folder is listed with different
size when building on tmpfs. This can make direct comparison of two build histories
difficult.
Considering that the listed folder size is meaningless anyway (it has no relation to
the size of the package/image), this patch addresses the problem by removing
the directory entry sizes entirely from the generated text files.

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
---
 meta/classes/buildhistory.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 601b29f..1cc7c44 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -508,9 +508,11 @@ buildhistory_list_files() {
 	# List the files in the specified directory, but exclude date/time etc.
 	# This awk script is somewhat messy, but handles where the size is not printed for device files under pseudo
 	if [ "$3" = "fakeroot" ] ; then
-		( cd $1 && ${FAKEROOTENV} ${FAKEROOTCMD} find . ! -path . -printf "%M %-10u %-10g %10s %p -> %l\n" | sort -k5 | sed 's/ * -> $//' > $2 )
+		( cd $1 && ${FAKEROOTENV} ${FAKEROOTCMD} find . ! -path . -printf "%M %-10u %-10g %10s %p -> %l\n" | sort -k5 | sed 's/ * -> $//' | \
+		awk '{if (substr($0,1,1) =="d"){split($0,a," ",b); print b[0]a[1]b[1]a[2]b[2]a[3]"                 "b[4]a[5]}else print}' > $2 )
 	else
-		( cd $1 && find . ! -path . -printf "%M %-10u %-10g %10s %p -> %l\n" | sort -k5 | sed 's/ * -> $//' > $2 )
+		( cd $1 && find . ! -path . -printf "%M %-10u %-10g %10s %p -> %l\n" | sort -k5 | sed 's/ * -> $//' | \
+		awk '{if (substr($0,1,1) =="d"){split($0,a," ",b); print b[0]a[1]b[1]a[2]b[2]a[3]"                 "b[4]a[5]}else print}' > $2)
 	fi
 }
 
-- 
2.7.4

Re: [PATCH 1/1] buildhistory.bbclass: buildhistory_list_files: remove folder size

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

On 22 June 2017 at 21:04, Juro Bystricky <juro.bystricky@intel.com> wrote:

> Considering that the listed folder size is meaningless anyway (it has no
> relation to
> the size of the package/image), this patch addresses the problem by
> removing
> the directory entry sizes entirely from the generated text files.
>

The alternative would be to show the size of the children of the
directory.  For example we manually calculate the size of a package (for
the PKGSIZE field in pkgdata) by adding up the size of files instead of
using du for the same reason.

Ross

[-- Attachment #2: Type: text/html, Size: 981 bytes --]