[PATCH 0/4] replace genext2fs with populate-extfs.sh

[PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

* The benefits:
  - Really support ext4
  - Support the sparse file (the sparse file became into the common file before)
  - Have a uniform code for ext2/3/4 generation
  - Remove the depends on genext2fs-native

* Impact
  - Build time:
    a) If we build fresh core-image-sato, there is nearly no impact.
    b) If we do the image generation, which means:
       $ bitbake core-image-sato
       $ bitbake core-image-sato -ccleansstate
       $ bitbake core-image-sato
   About 50 extra seconds are needed, here is my test result:
   Before the patches: 4m25s
   After the patches: 5m17s
   This is because the genext2fs is much faster than the 
   populate-extfs.sh, we will replace this script by the mke2fs when it
   supports create the filesystem from a initial directory.

  - Disk space (take core-image-sato as an example)
    a) The image file size is the same as before (529M)
    b) The disk usage is a little different: (du -sh)
              before       now 
       ext2:  364M         388M
       ext3:  381M         404M
       ext4:  380M         387M

   We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.

   I have done some simple runtime testing on core-image-sato and 
   core-image-minimal, they worked well.

// Robert

The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:

  grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib robert/ext4
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4

Robert Yang (4):
  e2fsprogs: the max length of debugfs argument is too short
  e2fsprogs: let debugfs do sparse copy
  e2fsprogs: add populate-extfs.sh
  image_types.bbclass: replace genext2fs with populate-extfs.sh

 meta/classes/image_types.bbclass                   |   46 ++++----
 .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
 .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
 .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
 .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
 5 files changed, 257 insertions(+), 28 deletions(-)
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch

-- 
1.7.10.4

[PATCH 1/4] e2fsprogs: the max length of debugfs argument is too short

[Robert Yang]

The max length of debugfs argument is 256 which is too short, the
arguments are two paths, the PATH_MAX is 4096 according to
/usr/include/linux/limits.h, use 2048 (4096 / 2) is a reasonable value.

[YOCTO #3848]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 ++++++++++++++++++++
 .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    1 +
 2 files changed, 29 insertions(+)
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch

diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
new file mode 100644
index 0000000..db6814d
--- /dev/null
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
@@ -0,0 +1,28 @@
+debugfs.c: the max length of debugfs argument is too short
+
+The max length of debugfs argument is 256 which is too short, the
+arguments are two paths, the PATH_MAX is 4096 according to
+/usr/include/linux/limits.h, use 2048 (4096 / 2) is a reasonable value.
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
+---
+ debugfs/debugfs.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
+--- a/debugfs/debugfs.c
++++ b/debugfs/debugfs.c
+@@ -2282,7 +2282,7 @@ void do_dump_mmp(int argc EXT2FS_ATTR((unused)), char *argv[])
+ static int source_file(const char *cmd_file, int ss_idx)
+ {
+ 	FILE		*f;
+-	char		buf[256];
++	char		buf[2048];
+ 	char		*cp;
+ 	int		exit_status = 0;
+ 	int		retval;
+-- 
+1.7.11.2
+
diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
index 3a4f5a8..898ee88 100644
--- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
@@ -4,6 +4,7 @@ PR = "r0"
 
 SRC_URI += "file://acinclude.m4 \
             file://remove.ldconfig.call.patch \
+            file://debugfs-too-short.patch \
 "
 
 SRC_URI[md5sum] = "a1ec22ef003688dae9f76c74881b22b9"
-- 
1.7.10.4

[PATCH 2/4] e2fsprogs: let debugfs do sparse copy

[Robert Yang]

Let debugfs do sparse copy when src is a sparse file, just like
"cp --sparse=auto"

[YOCTO #3848]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
 .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    1 +
 2 files changed, 115 insertions(+)
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch

diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
new file mode 100644
index 0000000..f20fcaf
--- /dev/null
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
@@ -0,0 +1,114 @@
+debugfs.c: do sparse copy when src is a sparse file
+
+Let debugfs do sparse copy when src is a sparse file, just like
+"cp --sparse=auto"
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
+---
+ debugfs/debugfs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++----------
+ 1 file changed, 44 insertions(+), 10 deletions(-)
+
+diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
+index 5cdc86a..015bfad 100644
+--- a/debugfs/debugfs.c
++++ b/debugfs/debugfs.c
+@@ -37,6 +37,16 @@ extern char *optarg;
+ #include "../version.h"
+ #include "jfs_user.h"
+ 
++/* 32KiB is the minimium blksize to best minimize system call overhead. */
++#ifndef IO_BUFSIZE
++#define IO_BUFSIZE 32*1024
++#endif
++
++/* Block size for `st_blocks'. */
++#ifndef S_BLKSIZE
++#define S_BLKSIZE 512
++#endif
++
+ ss_request_table *extra_cmds;
+ const char *debug_prog_name;
+ int sci_idx;
+@@ -1559,13 +1569,13 @@ void do_find_free_inode(int argc, char *argv[])
+ }
+ 
+ #ifndef READ_ONLY
+-static errcode_t copy_file(int fd, ext2_ino_t newfile)
++static errcode_t copy_file(int fd, ext2_ino_t newfile, int bufsiz, int make_holes)
+ {
+ 	ext2_file_t	e2_file;
+ 	errcode_t	retval;
+ 	int		got;
+ 	unsigned int	written;
+-	char		buf[8192];
++	char		buf[bufsiz];
+ 	char		*ptr;
+ 
+ 	retval = ext2fs_file_open(current_fs, newfile,
+@@ -1582,14 +1592,30 @@ static errcode_t copy_file(int fd, ext2_ino_t newfile)
+ 			goto fail;
+ 		}
+ 		ptr = buf;
++		char *cp = ptr;
+ 		while (got > 0) {
+-			retval = ext2fs_file_write(e2_file, ptr,
+-						   got, &written);
+-			if (retval)
+-				goto fail;
+-
+-			got -= written;
+-			ptr += written;
++			int count = got;
++			if (make_holes) {
++				/* Check whether all is zero */
++				while (count-- && *cp++ == 0) {
++					continue;
++				}
++				if (count < 0) {
++					/* The whole block is zero, make a hole */
++					retval = ext2fs_file_lseek(e2_file, got, EXT2_SEEK_CUR, 0);
++					if (retval)
++						goto fail;
++					got = 0;
++				}
++			}
++			/* Normal copy */
++			if (got > 0) {
++				retval = ext2fs_file_write(e2_file, ptr, got, &written);
++				if (retval)
++					goto fail;
++				got -= written;
++				ptr += written;
++			}
+ 		}
+ 	}
+ 	retval = ext2fs_file_close(e2_file);
+@@ -1608,6 +1634,8 @@ void do_write(int argc, char *argv[])
+ 	ext2_ino_t	newfile;
+ 	errcode_t	retval;
+ 	struct ext2_inode inode;
++	int make_holes = 0;
++	int bufsiz = IO_BUFSIZE;
+ 
+ 	if (common_args_process(argc, argv, 3, 3, "write",
+ 				"<native file> <new file>", CHECK_FS_RW))
+@@ -1672,7 +1700,13 @@ void do_write(int argc, char *argv[])
+ 		return;
+ 	}
+ 	if (LINUX_S_ISREG(inode.i_mode)) {
+-		retval = copy_file(fd, newfile);
++		if (statbuf.st_blocks < statbuf.st_size / S_BLKSIZE) {
++			make_holes = 1;
++			/* Use the I/O blocksize (st_blksize) as the buffer
++			 * size when copy sparse file */
++			bufsiz = statbuf.st_blksize;
++		}
++		retval = copy_file(fd, newfile, bufsiz, make_holes);
+ 		if (retval)
+ 			com_err("copy_file", retval, 0);
+ 	}
+-- 
+1.7.11.2
+
diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
index 898ee88..b547036 100644
--- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
@@ -5,6 +5,7 @@ PR = "r0"
 SRC_URI += "file://acinclude.m4 \
             file://remove.ldconfig.call.patch \
             file://debugfs-too-short.patch \
+            file://sparse_copy.patch \
 "
 
 SRC_URI[md5sum] = "a1ec22ef003688dae9f76c74881b22b9"
-- 
1.7.10.4

[PATCH 3/4] e2fsprogs: add populate-extfs.sh

[Robert Yang]

This script is originally from Darren Hart, it will be used for creating
the ext* filesystem from a given directory, which will replace the
genext2fs in image_types.bbclass at the moment, we may use the mke2fs to
replace this script again when it has the initial directory support.

Changes of the script:
* Rename it from mkdebugfs.sh to populate-extfs.sh
* Add a simple usage
* Add checking for the number of the parameters
* Add the "regular empty file" and "fifo" file type
* Set mode, uid and gid for the file
* Save the command lines to a file and batch run them
* Change the error message
* Improve the performance
* Add the support for hardlink

[YOCTO #3848]

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++++++
 .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    2 +
 2 files changed, 95 insertions(+)
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh

diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
new file mode 100644
index 0000000..5f2c307
--- /dev/null
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
@@ -0,0 +1,93 @@
+#!/bin/sh
+
+do_usage () {
+	cat << _EOF
+Usage: populate-extfs.sh <source> <device>
+Create an ext2/ext3/ext4 filesystem from a directory or file
+
+  source: The source directory or file
+  device: The target device
+
+_EOF
+	exit 1
+}
+
+[ $# -ne 2 ] && do_usage
+
+SRCDIR=${1%%/}
+DEVICE=$2
+DEBUGFS="debugfs"
+
+{
+	CWD="/"
+	find $SRCDIR | while read FILE; do
+                TGT="${FILE##*/}"
+                DIR="${FILE#$SRCDIR}"
+                DIR="${DIR%$TGT}"
+
+		# Skip the root dir
+		[ ! -z "$DIR" ] || continue
+		[ ! -z "$TGT" ] || continue
+
+		if [ "$DIR" != "$CWD" ]; then
+			echo "cd $DIR"
+			CWD="$DIR"
+		fi
+
+		# Only stat once since stat is a time consuming command
+		STAT=$(stat -c "TYPE=\"%F\";DEVNO=\"0x%t 0x%T\";MODE=\"%f\";U=\"%u\";G=\"%g\"" $FILE)
+		eval $STAT
+
+		case $TYPE in
+		"directory")
+			echo "mkdir $TGT"
+			;;
+		"regular file" | "regular empty file")
+			echo "write $FILE $TGT"
+			;;
+		"symbolic link")
+			LINK_TGT=$(readlink $FILE)
+			echo "symlink $TGT $LINK_TGT"
+			;;
+		"block special file" | "character special file")
+			echo "mknod $TGT b $DEVNO"
+			;;
+		"fifo")
+			echo "mknod $TGT p"
+			;;
+		*)
+			echo "Unknown/unhandled file type '$TYPE' file: $FILE" 1>&2
+			;;
+		esac
+
+		# Set the file mode
+		echo "sif $TGT mode 0x$MODE"
+
+		# Set uid and gid
+		echo "sif $TGT uid $U"
+		echo "sif $TGT gid $G"
+	done
+
+	# Handle the hard links.
+	# Save the hard links to a file, use the inode number as the filename, for example:
+	# If a and b's inode number is 6775928, save a and b to /tmp/tmp.VrCwHh5gdt/6775928.
+	INODE_DIR=`mktemp -d` || exit 1
+	for i in `find $SRCDIR -type f -links +1 -printf 'INODE=%i###FN=%p\n'`; do
+		eval `echo $i | sed 's$###$ $'`
+		echo ${FN#$SRCDIR} >>$INODE_DIR/$INODE
+	done
+	# Use the debugfs' ln and "sif links_count" to handle them.
+	for i in `ls $INODE_DIR`; do
+		# The link source
+		SRC=`head -1 $INODE_DIR/$i`
+		# Remove the files and link them again except the first one
+		for TGT in `sed -n -e '1!p' $INODE_DIR/$i`; do
+			echo "rm $TGT"
+			echo "ln $SRC $TGT"
+		done
+		LN_CNT=`cat $INODE_DIR/$i | wc -l`
+		# Set the links count
+		echo "sif $SRC links_count $LN_CNT"
+	done
+	rm -fr $INODE_DIR
+} | $DEBUGFS -w -f - $DEVICE
diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
index b547036..f2590f5 100644
--- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
@@ -6,6 +6,7 @@ SRC_URI += "file://acinclude.m4 \
             file://remove.ldconfig.call.patch \
             file://debugfs-too-short.patch \
             file://sparse_copy.patch \
+            file://populate-extfs.sh \
 "
 
 SRC_URI[md5sum] = "a1ec22ef003688dae9f76c74881b22b9"
@@ -43,6 +44,7 @@ do_install_append () {
 		mv ${D}${base_libdir}/e2initrd_helper ${D}${libdir}
 		mv ${D}${base_libdir}/pkgconfig ${D}${libdir}
 	fi
+	install -m 0755 ${WORKDIR}/populate-extfs.sh ${D}${bindir}
 }
 
 RDEPENDS_e2fsprogs = "e2fsprogs-badblocks"
-- 
1.7.10.4

[PATCH 4/4] image_types.bbclass: replace genext2fs with populate-extfs.sh

[Robert Yang]

* The benefits:
  - Really support ext4
  - Support the sparse file (we lost the sparse file in the image in the
    past, the sparse file became into the common file)
  - Have a uniform code for ext2/3/4 generation

* Comments from Darren Hart:
Basically, genext2fs doesn't support creating ext4 filesystems. It
creates, as I understand it, an ext2 filesystem, then adds a journal,
and sets some bits. It can't support the newer features like extents. So
what we end up with is a bit of a hack for a filesystem.

The ext tools (e2fsprogs) unfortunately don't provide an integrated
solution for generating prepopulated filesystem images as many other
mkfs* tools do. One thing missing was symlink support in libext2fs. I
added that support and demonstrated a script which uses the e2fsprogs
debugfs tool that can populate the newly formatted filesystem from a
directory and without root privileges.

This patches integrate this stage of development into OE-Core. We can
go about this in two ways. One is to just prototype this in a branch and
use it to validate the functionality and not make any changes to oe-core
image generation until mke2fs has initial directory support. The other
is to merge this and get broader testing of the concept and later move
to the full mke2fs implementation once it becomes available. I
understand the resistance to the latter, but long term I think it will
result in a more robust solution as we will have caught more of the
corner cases and have been able to do a better job integrating into
mke2fs the first time.

[YOCTO #3848]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/image_types.bbclass |   46 +++++++++++++++-----------------------
 1 file changed, 18 insertions(+), 28 deletions(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 94837ae..0476753 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -137,34 +137,24 @@ IMAGE_CMD_sum.jffs2 = "${IMAGE_CMD_jffs2} && sumtool -i ${DEPLOY_DIR_IMAGE}/${IM
 
 IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
 
-IMAGE_CMD_ext2 () {
-	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}
-	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2
-	mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2
-	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}
-}
+oe_mkext234fs () {
+	fstype=$1
+	extra_imagecmd=""
 
-IMAGE_CMD_ext3 () {
-	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
-	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
-}
+	if [ $# -gt 1 ]; then
+		shift
+		extra_imagecmd=$@
+	fi
 
-oe_mkext4fs () {
-	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1
-	tune2fs -O extents,uninit_bg,dir_index,has_journal $1
-	e2fsck -yfDC0 $1 || chk=$?
-	case $chk in
-	0|1|2)
-	    ;;
-	*)
-	    return $chk
-	    ;;
-	esac
+	# Create a sparse image block
+	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE count=0 bs=1k
+	yes | mkfs.$fstype $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype
+	populate-extfs.sh ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype
 }
 
-IMAGE_CMD_ext4 () {
-	oe_mkext4fs ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4
-}
+IMAGE_CMD_ext2 = "oe_mkext234fs ext2 ${EXTRA_IMAGECMD}"
+IMAGE_CMD_ext3 = "oe_mkext234fs ext3 ${EXTRA_IMAGECMD}"
+IMAGE_CMD_ext4 = "oe_mkext234fs ext4 ${EXTRA_IMAGECMD}"
 
 IMAGE_CMD_btrfs () {
 	mkfs.btrfs -b `expr ${ROOTFS_SIZE} \* 1024` ${EXTRA_IMAGECMD} -r ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.btrfs
@@ -214,7 +204,7 @@ JFFS2_ENDIANNESS ?= "${@base_conditional('SITEINFO_ENDIANNESS', 'le', '--little-
 JFFS2_ERASEBLOCK ?= "0x40000"
 EXTRA_IMAGECMD_jffs2 ?= "--pad ${JFFS2_ENDIANNESS} --eraseblock=${JFFS2_ERASEBLOCK} --no-cleanmarkers"
 
-# Change these if you want default genext2fs behavior (i.e. create minimal inode number)
+# Change these if you want default mkfs behavior (i.e. create minimal inode number)
 EXTRA_IMAGECMD_ext2 ?= "-i 8192"
 EXTRA_IMAGECMD_ext3 ?= "-i 8192"
 EXTRA_IMAGECMD_ext4 ?= "-i 8192"
@@ -225,9 +215,9 @@ IMAGE_DEPENDS = ""
 IMAGE_DEPENDS_jffs2 = "mtd-utils-native"
 IMAGE_DEPENDS_sum.jffs2 = "mtd-utils-native"
 IMAGE_DEPENDS_cramfs = "cramfs-native"
-IMAGE_DEPENDS_ext2 = "genext2fs-native"
-IMAGE_DEPENDS_ext3 = "genext2fs-native e2fsprogs-native"
-IMAGE_DEPENDS_ext4 = "genext2fs-native e2fsprogs-native"
+IMAGE_DEPENDS_ext2 = "e2fsprogs-native"
+IMAGE_DEPENDS_ext3 = "e2fsprogs-native"
+IMAGE_DEPENDS_ext4 = "e2fsprogs-native"
 IMAGE_DEPENDS_btrfs = "btrfs-tools-native"
 IMAGE_DEPENDS_squashfs = "squashfs-tools-native"
 IMAGE_DEPENDS_squashfs-xz = "squashfs-tools-native"
-- 
1.7.10.4

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Darren Hart]

Have these been reviewed on the ext mailing list already?

--
Darren

On 05/07/2013 02:48 AM, Robert Yang wrote:
> * The benefits:
>   - Really support ext4
>   - Support the sparse file (the sparse file became into the common file before)
>   - Have a uniform code for ext2/3/4 generation
>   - Remove the depends on genext2fs-native
> 
> * Impact
>   - Build time:
>     a) If we build fresh core-image-sato, there is nearly no impact.
>     b) If we do the image generation, which means:
>        $ bitbake core-image-sato
>        $ bitbake core-image-sato -ccleansstate
>        $ bitbake core-image-sato
>    About 50 extra seconds are needed, here is my test result:
>    Before the patches: 4m25s
>    After the patches: 5m17s
>    This is because the genext2fs is much faster than the 
>    populate-extfs.sh, we will replace this script by the mke2fs when it
>    supports create the filesystem from a initial directory.
> 
>   - Disk space (take core-image-sato as an example)
>     a) The image file size is the same as before (529M)
>     b) The disk usage is a little different: (du -sh)
>               before       now 
>        ext2:  364M         388M
>        ext3:  381M         404M
>        ext4:  380M         387M
> 
>    We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
> 
>    I have done some simple runtime testing on core-image-sato and 
>    core-image-minimal, they worked well.
> 
> // Robert
> 
> The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
> 
>   grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
> 
> are available in the git repository at:
> 
>   git://git.pokylinux.org/poky-contrib robert/ext4
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
> 
> Robert Yang (4):
>   e2fsprogs: the max length of debugfs argument is too short
>   e2fsprogs: let debugfs do sparse copy
>   e2fsprogs: add populate-extfs.sh
>   image_types.bbclass: replace genext2fs with populate-extfs.sh
> 
>  meta/classes/image_types.bbclass                   |   46 ++++----
>  .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>  .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>  .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>  5 files changed, 257 insertions(+), 28 deletions(-)
>  create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>  create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>  create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
> 

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

Hi Darren,

No, I haven't sent them to the ext mailing list, can we merge them in
oe-core and then back port to the ext community ? Or I should send them
to the ext mailing list first ?

// Robert

On 05/08/2013 06:48 AM, Darren Hart wrote:
> Have these been reviewed on the ext mailing list already?
>
> --
> Darren
>
> On 05/07/2013 02:48 AM, Robert Yang wrote:
>> * The benefits:
>>    - Really support ext4
>>    - Support the sparse file (the sparse file became into the common file before)
>>    - Have a uniform code for ext2/3/4 generation
>>    - Remove the depends on genext2fs-native
>>
>> * Impact
>>    - Build time:
>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>      b) If we do the image generation, which means:
>>         $ bitbake core-image-sato
>>         $ bitbake core-image-sato -ccleansstate
>>         $ bitbake core-image-sato
>>     About 50 extra seconds are needed, here is my test result:
>>     Before the patches: 4m25s
>>     After the patches: 5m17s
>>     This is because the genext2fs is much faster than the
>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>     supports create the filesystem from a initial directory.
>>
>>    - Disk space (take core-image-sato as an example)
>>      a) The image file size is the same as before (529M)
>>      b) The disk usage is a little different: (du -sh)
>>                before       now
>>         ext2:  364M         388M
>>         ext3:  381M         404M
>>         ext4:  380M         387M
>>
>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>
>>     I have done some simple runtime testing on core-image-sato and
>>     core-image-minimal, they worked well.
>>
>> // Robert
>>
>> The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>
>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>
>> are available in the git repository at:
>>
>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>
>> Robert Yang (4):
>>    e2fsprogs: the max length of debugfs argument is too short
>>    e2fsprogs: let debugfs do sparse copy
>>    e2fsprogs: add populate-extfs.sh
>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>
>>   meta/classes/image_types.bbclass                   |   46 ++++----
>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Darren Hart]

Always discuss them upstream first. If they are accepted upstream, THEN,
we can roll them into oe-core if we don't want to bring in everything
that landed before them too. But always discuss changes to a project to
the project before doing so in oe-core.

Thanks,

Darren

On 05/07/2013 06:49 PM, Robert Yang wrote:
> 
> Hi Darren,
> 
> No, I haven't sent them to the ext mailing list, can we merge them in
> oe-core and then back port to the ext community ? Or I should send them
> to the ext mailing list first ?
> 
> // Robert
> 
> On 05/08/2013 06:48 AM, Darren Hart wrote:
>> Have these been reviewed on the ext mailing list already?
>>
>> --
>> Darren
>>
>> On 05/07/2013 02:48 AM, Robert Yang wrote:
>>> * The benefits:
>>>    - Really support ext4
>>>    - Support the sparse file (the sparse file became into the common file before)
>>>    - Have a uniform code for ext2/3/4 generation
>>>    - Remove the depends on genext2fs-native
>>>
>>> * Impact
>>>    - Build time:
>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>      b) If we do the image generation, which means:
>>>         $ bitbake core-image-sato
>>>         $ bitbake core-image-sato -ccleansstate
>>>         $ bitbake core-image-sato
>>>     About 50 extra seconds are needed, here is my test result:
>>>     Before the patches: 4m25s
>>>     After the patches: 5m17s
>>>     This is because the genext2fs is much faster than the
>>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>>     supports create the filesystem from a initial directory.
>>>
>>>    - Disk space (take core-image-sato as an example)
>>>      a) The image file size is the same as before (529M)
>>>      b) The disk usage is a little different: (du -sh)
>>>                before       now
>>>         ext2:  364M         388M
>>>         ext3:  381M         404M
>>>         ext4:  380M         387M
>>>
>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>
>>>     I have done some simple runtime testing on core-image-sato and
>>>     core-image-minimal, they worked well.
>>>
>>> // Robert
>>>
>>> The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>
>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>
>>> are available in the git repository at:
>>>
>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>
>>> Robert Yang (4):
>>>    e2fsprogs: the max length of debugfs argument is too short
>>>    e2fsprogs: let debugfs do sparse copy
>>>    e2fsprogs: add populate-extfs.sh
>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>
>>>   meta/classes/image_types.bbclass                   |   46 ++++----
>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>
>>

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 05/08/2013 03:39 PM, Darren Hart wrote:
> Always discuss them upstream first. If they are accepted upstream, THEN,
> we can roll them into oe-core if we don't want to bring in everything
> that landed before them too. But always discuss changes to a project to
> the project before doing so in oe-core.
>

Thanks, got it, I will send the following 2 to ext mailing list if you are
fine with them.

[PATCH 1/4] e2fsprogs: the max length of debugfs argument is too short
[PATCH 2/4] e2fsprogs: let debugfs do sparse copy

// Robert

> Thanks,
>
> Darren
>
> On 05/07/2013 06:49 PM, Robert Yang wrote:
>>
>> Hi Darren,
>>
>> No, I haven't sent them to the ext mailing list, can we merge them in
>> oe-core and then back port to the ext community ? Or I should send them
>> to the ext mailing list first ?
>>
>> // Robert
>>
>> On 05/08/2013 06:48 AM, Darren Hart wrote:
>>> Have these been reviewed on the ext mailing list already?
>>>
>>> --
>>> Darren
>>>
>>> On 05/07/2013 02:48 AM, Robert Yang wrote:
>>>> * The benefits:
>>>>     - Really support ext4
>>>>     - Support the sparse file (the sparse file became into the common file before)
>>>>     - Have a uniform code for ext2/3/4 generation
>>>>     - Remove the depends on genext2fs-native
>>>>
>>>> * Impact
>>>>     - Build time:
>>>>       a) If we build fresh core-image-sato, there is nearly no impact.
>>>>       b) If we do the image generation, which means:
>>>>          $ bitbake core-image-sato
>>>>          $ bitbake core-image-sato -ccleansstate
>>>>          $ bitbake core-image-sato
>>>>      About 50 extra seconds are needed, here is my test result:
>>>>      Before the patches: 4m25s
>>>>      After the patches: 5m17s
>>>>      This is because the genext2fs is much faster than the
>>>>      populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>      supports create the filesystem from a initial directory.
>>>>
>>>>     - Disk space (take core-image-sato as an example)
>>>>       a) The image file size is the same as before (529M)
>>>>       b) The disk usage is a little different: (du -sh)
>>>>                 before       now
>>>>          ext2:  364M         388M
>>>>          ext3:  381M         404M
>>>>          ext4:  380M         387M
>>>>
>>>>      We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>
>>>>      I have done some simple runtime testing on core-image-sato and
>>>>      core-image-minimal, they worked well.
>>>>
>>>> // Robert
>>>>
>>>> The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>
>>>>     grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>
>>>> are available in the git repository at:
>>>>
>>>>     git://git.pokylinux.org/poky-contrib robert/ext4
>>>>     http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>
>>>> Robert Yang (4):
>>>>     e2fsprogs: the max length of debugfs argument is too short
>>>>     e2fsprogs: let debugfs do sparse copy
>>>>     e2fsprogs: add populate-extfs.sh
>>>>     image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>
>>>>    meta/classes/image_types.bbclass                   |   46 ++++----
>>>>    .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>>>    .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>>>    .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>>>>    .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>>>    5 files changed, 257 insertions(+), 28 deletions(-)
>>>>    create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>    create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>    create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>
>>>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Darren Hart]

On 05/08/2013 12:48 AM, Robert Yang wrote:
> 
> 
> On 05/08/2013 03:39 PM, Darren Hart wrote:
>> Always discuss them upstream first. If they are accepted upstream, THEN,
>> we can roll them into oe-core if we don't want to bring in everything
>> that landed before them too. But always discuss changes to a project to
>> the project before doing so in oe-core.
>>
> 
> Thanks, got it, I will send the following 2 to ext mailing list if you are
> fine with them.
> 
> [PATCH 1/4] e2fsprogs: the max length of debugfs argument is too short
> [PATCH 2/4] e2fsprogs: let debugfs do sparse copy


If you would like my review, I'll have a look at them tomorrow morning
and get you some feedback. Too tired to think straight any longer :-)

--
Darren


> 
> // Robert
> 
>> Thanks,
>>
>> Darren
>>
>> On 05/07/2013 06:49 PM, Robert Yang wrote:
>>>
>>> Hi Darren,
>>>
>>> No, I haven't sent them to the ext mailing list, can we merge them in
>>> oe-core and then back port to the ext community ? Or I should send them
>>> to the ext mailing list first ?
>>>
>>> // Robert
>>>
>>> On 05/08/2013 06:48 AM, Darren Hart wrote:
>>>> Have these been reviewed on the ext mailing list already?
>>>>
>>>> --
>>>> Darren
>>>>
>>>> On 05/07/2013 02:48 AM, Robert Yang wrote:
>>>>> * The benefits:
>>>>>     - Really support ext4
>>>>>     - Support the sparse file (the sparse file became into the common file before)
>>>>>     - Have a uniform code for ext2/3/4 generation
>>>>>     - Remove the depends on genext2fs-native
>>>>>
>>>>> * Impact
>>>>>     - Build time:
>>>>>       a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>       b) If we do the image generation, which means:
>>>>>          $ bitbake core-image-sato
>>>>>          $ bitbake core-image-sato -ccleansstate
>>>>>          $ bitbake core-image-sato
>>>>>      About 50 extra seconds are needed, here is my test result:
>>>>>      Before the patches: 4m25s
>>>>>      After the patches: 5m17s
>>>>>      This is because the genext2fs is much faster than the
>>>>>      populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>>      supports create the filesystem from a initial directory.
>>>>>
>>>>>     - Disk space (take core-image-sato as an example)
>>>>>       a) The image file size is the same as before (529M)
>>>>>       b) The disk usage is a little different: (du -sh)
>>>>>                 before       now
>>>>>          ext2:  364M         388M
>>>>>          ext3:  381M         404M
>>>>>          ext4:  380M         387M
>>>>>
>>>>>      We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>
>>>>>      I have done some simple runtime testing on core-image-sato and
>>>>>      core-image-minimal, they worked well.
>>>>>
>>>>> // Robert
>>>>>
>>>>> The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>
>>>>>     grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>
>>>>> are available in the git repository at:
>>>>>
>>>>>     git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>     http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>
>>>>> Robert Yang (4):
>>>>>     e2fsprogs: the max length of debugfs argument is too short
>>>>>     e2fsprogs: let debugfs do sparse copy
>>>>>     e2fsprogs: add populate-extfs.sh
>>>>>     image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>
>>>>>    meta/classes/image_types.bbclass                   |   46 ++++----
>>>>>    .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>>>>    .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>>>>    .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>>>>>    .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>>>>    5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>    create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>    create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>    create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>
>>>>
>>

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 05/08/2013 03:51 PM, Darren Hart wrote:
> On 05/08/2013 12:48 AM, Robert Yang wrote:
>>
>>
>> On 05/08/2013 03:39 PM, Darren Hart wrote:
>>> Always discuss them upstream first. If they are accepted upstream, THEN,
>>> we can roll them into oe-core if we don't want to bring in everything
>>> that landed before them too. But always discuss changes to a project to
>>> the project before doing so in oe-core.
>>>
>>
>> Thanks, got it, I will send the following 2 to ext mailing list if you are
>> fine with them.
>>
>> [PATCH 1/4] e2fsprogs: the max length of debugfs argument is too short
>> [PATCH 2/4] e2fsprogs: let debugfs do sparse copy
>
>
> If you would like my review, I'll have a look at them tomorrow morning
> and get you some feedback. Too tired to think straight any longer :-)
>

Yes, your review is very important, tomorrow is fine to me, thank you very
much, night.

// Robert

> --
> Darren
>
>
>>
>> // Robert
>>
>>> Thanks,
>>>
>>> Darren
>>>
>>> On 05/07/2013 06:49 PM, Robert Yang wrote:
>>>>
>>>> Hi Darren,
>>>>
>>>> No, I haven't sent them to the ext mailing list, can we merge them in
>>>> oe-core and then back port to the ext community ? Or I should send them
>>>> to the ext mailing list first ?
>>>>
>>>> // Robert
>>>>
>>>> On 05/08/2013 06:48 AM, Darren Hart wrote:
>>>>> Have these been reviewed on the ext mailing list already?
>>>>>
>>>>> --
>>>>> Darren
>>>>>
>>>>> On 05/07/2013 02:48 AM, Robert Yang wrote:
>>>>>> * The benefits:
>>>>>>      - Really support ext4
>>>>>>      - Support the sparse file (the sparse file became into the common file before)
>>>>>>      - Have a uniform code for ext2/3/4 generation
>>>>>>      - Remove the depends on genext2fs-native
>>>>>>
>>>>>> * Impact
>>>>>>      - Build time:
>>>>>>        a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>>        b) If we do the image generation, which means:
>>>>>>           $ bitbake core-image-sato
>>>>>>           $ bitbake core-image-sato -ccleansstate
>>>>>>           $ bitbake core-image-sato
>>>>>>       About 50 extra seconds are needed, here is my test result:
>>>>>>       Before the patches: 4m25s
>>>>>>       After the patches: 5m17s
>>>>>>       This is because the genext2fs is much faster than the
>>>>>>       populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>>>       supports create the filesystem from a initial directory.
>>>>>>
>>>>>>      - Disk space (take core-image-sato as an example)
>>>>>>        a) The image file size is the same as before (529M)
>>>>>>        b) The disk usage is a little different: (du -sh)
>>>>>>                  before       now
>>>>>>           ext2:  364M         388M
>>>>>>           ext3:  381M         404M
>>>>>>           ext4:  380M         387M
>>>>>>
>>>>>>       We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>>
>>>>>>       I have done some simple runtime testing on core-image-sato and
>>>>>>       core-image-minimal, they worked well.
>>>>>>
>>>>>> // Robert
>>>>>>
>>>>>> The following changes since commit 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>
>>>>>>      grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>>
>>>>>> are available in the git repository at:
>>>>>>
>>>>>>      git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>      http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>
>>>>>> Robert Yang (4):
>>>>>>      e2fsprogs: the max length of debugfs argument is too short
>>>>>>      e2fsprogs: let debugfs do sparse copy
>>>>>>      e2fsprogs: add populate-extfs.sh
>>>>>>      image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>
>>>>>>     meta/classes/image_types.bbclass                   |   46 ++++----
>>>>>>     .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>>>>>     .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>>>>>     .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>>>>>>     .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>>>>>     5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>     create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>>     create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>     create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>
>>>>>
>>>
>

Re: [PATCH 4/4] image_types.bbclass: replace genext2fs with populate-extfs.sh

[Jonathan Liu]

On 7/05/2013 7:48 PM, Robert Yang wrote:
> -oe_mkext4fs () {
> -	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1
> -	tune2fs -O extents,uninit_bg,dir_index,has_journal $1
> -	e2fsck -yfDC0 $1 || chk=$?
> -	case $chk in
> -	0|1|2)
> -	    ;;
> -	*)
> -	    return $chk
> -	    ;;
> -	esac
> +	# Create a sparse image block
> +	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE count=0 bs=1k
> +	yes | mkfs.$fstype $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype
yes | mkfs.$fstype can be simplified to mkfs.$fstype -F
> +	populate-extfs.sh ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype
>   }
Regards,
Jonathan

Re: [PATCH 1/4] e2fsprogs: the max length of debugfs argument is too short

[Darren Hart]

This one looks reasonable to me. Be sure to add your patch header to the
patch itself, add your Signed-off-by, and submit upstream for feedback
before merging with oe-core.

--
Darren

On 05/07/2013 02:48 AM, Robert Yang wrote:
> The max length of debugfs argument is 256 which is too short, the
> arguments are two paths, the PATH_MAX is 4096 according to
> /usr/include/linux/limits.h, use 2048 (4096 / 2) is a reasonable value.
> 
> [YOCTO #3848]
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 ++++++++++++++++++++
>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    1 +
>  2 files changed, 29 insertions(+)
>  create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
> 
> diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
> new file mode 100644
> index 0000000..db6814d
> --- /dev/null
> +++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
> @@ -0,0 +1,28 @@
> +debugfs.c: the max length of debugfs argument is too short
> +
> +The max length of debugfs argument is 256 which is too short, the
> +arguments are two paths, the PATH_MAX is 4096 according to
> +/usr/include/linux/limits.h, use 2048 (4096 / 2) is a reasonable value.
> +
> +Upstream-Status: Pending
> +
> +Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> +---
> + debugfs/debugfs.c | 2 +-
> + 1 file changed, 1 insertion(+), 1 deletion(-)
> +
> +diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
> +--- a/debugfs/debugfs.c
> ++++ b/debugfs/debugfs.c
> +@@ -2282,7 +2282,7 @@ void do_dump_mmp(int argc EXT2FS_ATTR((unused)), char *argv[])
> + static int source_file(const char *cmd_file, int ss_idx)
> + {
> + 	FILE		*f;
> +-	char		buf[256];
> ++	char		buf[2048];
> + 	char		*cp;
> + 	int		exit_status = 0;
> + 	int		retval;
> +-- 
> +1.7.11.2
> +
> diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
> index 3a4f5a8..898ee88 100644
> --- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
> +++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
> @@ -4,6 +4,7 @@ PR = "r0"
>  
>  SRC_URI += "file://acinclude.m4 \
>              file://remove.ldconfig.call.patch \
> +            file://debugfs-too-short.patch \
>  "
>  
>  SRC_URI[md5sum] = "a1ec22ef003688dae9f76c74881b22b9"
> 

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel

Re: [PATCH 2/4] e2fsprogs: let debugfs do sparse copy

[Darren Hart]

On 05/07/2013 02:48 AM, Robert Yang wrote:
> Let debugfs do sparse copy when src is a sparse file, just like
> "cp --sparse=auto"
> 
> [YOCTO #3848]
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    1 +
>  2 files changed, 115 insertions(+)
>  create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
> 
> diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
> new file mode 100644
> index 0000000..f20fcaf
> --- /dev/null
> +++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
> @@ -0,0 +1,114 @@
> +debugfs.c: do sparse copy when src is a sparse file
> +
> +Let debugfs do sparse copy when src is a sparse file, just like
> +"cp --sparse=auto"
> +
> +Upstream-Status: Pending
> +
> +Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> +---
> + debugfs/debugfs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++----------
> + 1 file changed, 44 insertions(+), 10 deletions(-)
> +
> +diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
> +index 5cdc86a..015bfad 100644
> +--- a/debugfs/debugfs.c
> ++++ b/debugfs/debugfs.c
> +@@ -37,6 +37,16 @@ extern char *optarg;
> + #include "../version.h"
> + #include "jfs_user.h"
> + 
> ++/* 32KiB is the minimium blksize to best minimize system call overhead. */

Why is that? Can you defend that to Ted and the ext4 developers?

> ++#ifndef IO_BUFSIZE
> ++#define IO_BUFSIZE 32*1024
> ++#endif
> ++
> ++/* Block size for `st_blocks'. */
> ++#ifndef S_BLKSIZE
> ++#define S_BLKSIZE 512
> ++#endif
> ++
> + ss_request_table *extra_cmds;
> + const char *debug_prog_name;
> + int sci_idx;
> +@@ -1559,13 +1569,13 @@ void do_find_free_inode(int argc, char *argv[])
> + }
> + 
> + #ifndef READ_ONLY
> +-static errcode_t copy_file(int fd, ext2_ino_t newfile)
> ++static errcode_t copy_file(int fd, ext2_ino_t newfile, int bufsiz, int make_holes)

Don't trim off single vowels in variable names. bufsize is fine.

> + {
> + 	ext2_file_t	e2_file;
> + 	errcode_t	retval;
> + 	int		got;
> + 	unsigned int	written;
> +-	char		buf[8192];
> ++	char		buf[bufsiz];
> + 	char		*ptr;
> + 
> + 	retval = ext2fs_file_open(current_fs, newfile,
> +@@ -1582,14 +1592,30 @@ static errcode_t copy_file(int fd, ext2_ino_t newfile)
> + 			goto fail;
> + 		}
> + 		ptr = buf;
> ++		char *cp = ptr;
> + 		while (got > 0) {
> +-			retval = ext2fs_file_write(e2_file, ptr,
> +-						   got, &written);
> +-			if (retval)
> +-				goto fail;
> +-
> +-			got -= written;
> +-			ptr += written;
> ++			int count = got;

Why declare this inside the while loop?

> ++			if (make_holes) {
> ++				/* Check whether all is zero */
> ++				while (count-- && *cp++ == 0) {
> ++					continue;
> ++				}


Could you improve this a bit by using larger chunks? u32 maybe? You
could ensure the bufsize rounds up to that, or just enforce it.


> ++				if (count < 0) {
> ++					/* The whole block is zero, make a hole */
> ++					retval = ext2fs_file_lseek(e2_file, got, EXT2_SEEK_CUR, 0);
> ++					if (retval)
> ++						goto fail;
> ++					got = 0;
> ++				}
> ++			}
> ++			/* Normal copy */
> ++			if (got > 0) {
> ++				retval = ext2fs_file_write(e2_file, ptr, got, &written);
> ++				if (retval)
> ++					goto fail;
> ++				got -= written;
> ++				ptr += written;
> ++			}
> + 		}
> + 	}
> + 	retval = ext2fs_file_close(e2_file);
> +@@ -1608,6 +1634,8 @@ void do_write(int argc, char *argv[])
> + 	ext2_ino_t	newfile;
> + 	errcode_t	retval;
> + 	struct ext2_inode inode;
> ++	int make_holes = 0;
> ++	int bufsiz = IO_BUFSIZE;

bufsize


> + 
> + 	if (common_args_process(argc, argv, 3, 3, "write",
> + 				"<native file> <new file>", CHECK_FS_RW))
> +@@ -1672,7 +1700,13 @@ void do_write(int argc, char *argv[])
> + 		return;
> + 	}
> + 	if (LINUX_S_ISREG(inode.i_mode)) {
> +-		retval = copy_file(fd, newfile);
> ++		if (statbuf.st_blocks < statbuf.st_size / S_BLKSIZE) {
> ++			make_holes = 1;
> ++			/* Use the I/O blocksize (st_blksize) as the buffer
> ++			 * size when copy sparse file */

"when copying sparse files"

Also, multiline comments should probably look like this:

 			/*
			 * Use I/O blocksize as buffer size when
			 * copying sparse files.
			 */

Although, honestly, this doesn't add any information. It is obvious that
is what the code does. It isn't obvious WHY that is necessary, a comment
to that effect might be worthwhile, but as is, this one can just be removed.

Thanks,

Darren


> ++			bufsiz = statbuf.st_blksize;
> ++		}
> ++		retval = copy_file(fd, newfile, bufsiz, make_holes);
> + 		if (retval)
> + 			com_err("copy_file", retval, 0);
> + 	}
> +-- 
> +1.7.11.2
> +
> diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
> index 898ee88..b547036 100644
> --- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
> +++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
> @@ -5,6 +5,7 @@ PR = "r0"
>  SRC_URI += "file://acinclude.m4 \
>              file://remove.ldconfig.call.patch \
>              file://debugfs-too-short.patch \
> +            file://sparse_copy.patch \
>  "
>  
>  SRC_URI[md5sum] = "a1ec22ef003688dae9f76c74881b22b9"
> 

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Jonathan Liu]

[-- Attachment #1: Type: text/plain, Size: 9173 bytes --]

Hi Robert,

I get the following errors doing filesystem check on the ext4 image:

# fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
e2fsck 1.42.7 (21-Jan-2013)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
3).
Fix? no

Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
3).
Fix? no

Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, should be
3).
Fix? no

Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
3).
Fix? no

Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
3).
Fix? no

Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
Fix? no

Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Inode 774 ref count is 9, should be 10.  Fix? no

Pass 5: Checking group summary information

tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
Filesystem still has errors **********

tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
non-contiguous), 131042/169656 blocks


After booting the image I get the following:
# dmesg | grep EXT4-fs
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs (sda2): re-mounted. Opts: (null)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
0(0), depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
depth 0(0)
EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
depth 0(0)


# systemctl --failed | cat
UNIT                     LOAD   ACTIVE SUB    DESCRIPTION
machineid.service        loaded failed failed Machine ID first boot
configure
systemd-journald.service loaded failed failed Journal Service
systemd-journald.socket  loaded failed failed Journal Socket

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

3 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.


I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".

Regards,
Jonathan


On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:

> * The benefits:
>   - Really support ext4
>   - Support the sparse file (the sparse file became into the common file
> before)
>   - Have a uniform code for ext2/3/4 generation
>   - Remove the depends on genext2fs-native
>
> * Impact
>   - Build time:
>     a) If we build fresh core-image-sato, there is nearly no impact.
>     b) If we do the image generation, which means:
>        $ bitbake core-image-sato
>        $ bitbake core-image-sato -ccleansstate
>        $ bitbake core-image-sato
>    About 50 extra seconds are needed, here is my test result:
>    Before the patches: 4m25s
>    After the patches: 5m17s
>    This is because the genext2fs is much faster than the
>    populate-extfs.sh, we will replace this script by the mke2fs when it
>    supports create the filesystem from a initial directory.
>
>   - Disk space (take core-image-sato as an example)
>     a) The image file size is the same as before (529M)
>     b) The disk usage is a little different: (du -sh)
>               before       now
>        ext2:  364M         388M
>        ext3:  381M         404M
>        ext4:  380M         387M
>
>    We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>
>    I have done some simple runtime testing on core-image-sato and
>    core-image-minimal, they worked well.
>
> // Robert
>
> The following changes since commit
> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>
>   grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>
> are available in the git repository at:
>
>   git://git.pokylinux.org/poky-contrib robert/ext4
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>
> Robert Yang (4):
>   e2fsprogs: the max length of debugfs argument is too short
>   e2fsprogs: let debugfs do sparse copy
>   e2fsprogs: add populate-extfs.sh
>   image_types.bbclass: replace genext2fs with populate-extfs.sh
>
>  meta/classes/image_types.bbclass                   |   46 ++++----
>  .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>  .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>  .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114
> ++++++++++++++++++++
>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>  5 files changed, 257 insertions(+), 28 deletions(-)
>  create mode 100644
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>  create mode 100644
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>  create mode 100644
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>
> --
> 1.7.10.4
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 12063 bytes --]

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

Hi Jonathan Liu,

What's your configuration, please, I tried a core-image-minimal building
just now, didn't notice this error. Did the error come after these patches
applied ?

My configuration:
MACHINE = "qemux86"
IMAGE_FSTYPES += "ext4"

// Robert

On 05/13/2013 12:59 PM, Jonathan Liu wrote:
> Hi Robert,
>
> I get the following errors doing filesystem check on the ext4 image:
>
> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
> e2fsck 1.42.7 (21-Jan-2013)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
> 3).
> Fix? no
>
> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
> 3).
> Fix? no
>
> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, should be
> 3).
> Fix? no
>
> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
> 3).
> Fix? no
>
> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
> 3).
> Fix? no
>
> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
> Fix? no
>
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Inode 774 ref count is 9, should be 10.  Fix? no
>
> Pass 5: Checking group summary information
>
> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
> Filesystem still has errors **********
>
> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
> non-contiguous), 131042/169656 blocks
>
>
> After booting the image I get the following:
> # dmesg | grep EXT4-fs
> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
> depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
> depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
> depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs (sda2): re-mounted. Opts: (null)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
> 0(0), depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
> depth 0(0)
> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
> depth 0(0)
>
>
> # systemctl --failed | cat
> UNIT                     LOAD   ACTIVE SUB    DESCRIPTION
> machineid.service        loaded failed failed Machine ID first boot
> configure
> systemd-journald.service loaded failed failed Journal Service
> systemd-journald.socket  loaded failed failed Journal Socket
>
> LOAD   = Reflects whether the unit definition was properly loaded.
> ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
> SUB    = The low-level unit activation state, values depend on unit type.
>
> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
> To show all installed unit files use 'systemctl list-unit-files'.
>
>
> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>
> Regards,
> Jonathan
>
>
> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>
>> * The benefits:
>>    - Really support ext4
>>    - Support the sparse file (the sparse file became into the common file
>> before)
>>    - Have a uniform code for ext2/3/4 generation
>>    - Remove the depends on genext2fs-native
>>
>> * Impact
>>    - Build time:
>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>      b) If we do the image generation, which means:
>>         $ bitbake core-image-sato
>>         $ bitbake core-image-sato -ccleansstate
>>         $ bitbake core-image-sato
>>     About 50 extra seconds are needed, here is my test result:
>>     Before the patches: 4m25s
>>     After the patches: 5m17s
>>     This is because the genext2fs is much faster than the
>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>     supports create the filesystem from a initial directory.
>>
>>    - Disk space (take core-image-sato as an example)
>>      a) The image file size is the same as before (529M)
>>      b) The disk usage is a little different: (du -sh)
>>                before       now
>>         ext2:  364M         388M
>>         ext3:  381M         404M
>>         ext4:  380M         387M
>>
>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>
>>     I have done some simple runtime testing on core-image-sato and
>>     core-image-minimal, they worked well.
>>
>> // Robert
>>
>> The following changes since commit
>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>
>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>
>> are available in the git repository at:
>>
>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>
>> Robert Yang (4):
>>    e2fsprogs: the max length of debugfs argument is too short
>>    e2fsprogs: let debugfs do sparse copy
>>    e2fsprogs: add populate-extfs.sh
>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>
>>   meta/classes/image_types.bbclass                   |   46 ++++----
>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114
>> ++++++++++++++++++++
>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>   create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>   create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>   create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>
>> --
>> 1.7.10.4
>>
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Jonathan Liu]

On 13/05/2013 7:23 PM, Robert Yang wrote:
>
> Hi Jonathan Liu,
>
> What's your configuration, please, I tried a core-image-minimal building
> just now, didn't notice this error. Did the error come after these 
> patches
> applied ?
Yes, the errors occured after applying your patches.

distro conf:
DISTRO_FEATURES_append = " largefile opengl systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_graphical_init_manager = ""
VIRTUAL-RUNTIME_init_manager = "systemd"
PACKAGE_CLASSES = "package_ipk"

image bb:
SYSLINUX_ROOT ?= "root=/dev/sda2"
SYSLINUX_PROMPT = "0"
SYSLINUX_TIMEOUT = "1"
SYSLINUX_LABELS = "boot"
LABELS_append = " ${SYSLINUX_LABELS} "
do_bootdirectdisk[depends] += "${PN}:do_rootfs"
APPEND += "rootfstype=ext4 rw"
ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
inherit boot-directdisk
IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
IMAGE_FSTYPES = "ext4"
IMAGE_LINGUAS = ""

>
> My configuration:
> MACHINE = "qemux86"
> IMAGE_FSTYPES += "ext4"
>
> // Robert
>
> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>> Hi Robert,
>>
>> I get the following errors doing filesystem check on the ext4 image:
>>
>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>> e2fsck 1.42.7 (21-Jan-2013)
>> Pass 1: Checking inodes, blocks, and sizes
>> Pass 2: Checking directory structure
>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, 
>> should be
>> 3).
>> Fix? no
>>
>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, 
>> should be
>> 3).
>> Fix? no
>>
>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, 
>> should be
>> 3).
>> Fix? no
>>
>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, 
>> should be
>> 3).
>> Fix? no
>>
>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, 
>> should be
>> 3).
>> Fix? no
>>
>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should 
>> be 3).
>> Fix? no
>>
>> Pass 3: Checking directory connectivity
>> Pass 4: Checking reference counts
>> Inode 774 ref count is 9, should be 10.  Fix? no
>>
>> Pass 5: Checking group summary information
>>
>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>> Filesystem still has errors **********
>>
>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
>> non-contiguous), 131042/169656 blocks
>>
>>
>> After booting the image I get the following:
>> # dmesg | grep EXT4-fs
>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 
>> 0(0),
>> depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 
>> 0(0),
>> depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 
>> 0(0),
>> depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs (sda2): re-mounted. Opts: (null)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 
>> 0, max
>> 0(0), depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>> depth 0(0)
>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>> depth 0(0)
>>
>>
>> # systemctl --failed | cat
>> UNIT                     LOAD   ACTIVE SUB    DESCRIPTION
>> machineid.service        loaded failed failed Machine ID first boot
>> configure
>> systemd-journald.service loaded failed failed Journal Service
>> systemd-journald.socket  loaded failed failed Journal Socket
>>
>> LOAD   = Reflects whether the unit definition was properly loaded.
>> ACTIVE = The high-level unit activation state, i.e. generalization of 
>> SUB.
>> SUB    = The low-level unit activation state, values depend on unit 
>> type.
>>
>> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
>> To show all installed unit files use 'systemctl list-unit-files'.
>>
>>
>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>
>> Regards,
>> Jonathan
>>
>>
>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>
>>> * The benefits:
>>>    - Really support ext4
>>>    - Support the sparse file (the sparse file became into the common 
>>> file
>>> before)
>>>    - Have a uniform code for ext2/3/4 generation
>>>    - Remove the depends on genext2fs-native
>>>
>>> * Impact
>>>    - Build time:
>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>      b) If we do the image generation, which means:
>>>         $ bitbake core-image-sato
>>>         $ bitbake core-image-sato -ccleansstate
>>>         $ bitbake core-image-sato
>>>     About 50 extra seconds are needed, here is my test result:
>>>     Before the patches: 4m25s
>>>     After the patches: 5m17s
>>>     This is because the genext2fs is much faster than the
>>>     populate-extfs.sh, we will replace this script by the mke2fs 
>>> when it
>>>     supports create the filesystem from a initial directory.
>>>
>>>    - Disk space (take core-image-sato as an example)
>>>      a) The image file size is the same as before (529M)
>>>      b) The disk usage is a little different: (du -sh)
>>>                before       now
>>>         ext2:  364M         388M
>>>         ext3:  381M         404M
>>>         ext4:  380M         387M
>>>
>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>
>>>     I have done some simple runtime testing on core-image-sato and
>>>     core-image-minimal, they worked well.
>>>
>>> // Robert
>>>
>>> The following changes since commit
>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>
>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>
>>> are available in the git repository at:
>>>
>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>
>>> Robert Yang (4):
>>>    e2fsprogs: the max length of debugfs argument is too short
>>>    e2fsprogs: let debugfs do sparse copy
>>>    e2fsprogs: add populate-extfs.sh
>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>
>>>   meta/classes/image_types.bbclass                   |   46 ++++----
>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 
>>> ++++++++++++++++
>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114
>>> ++++++++++++++++++++
>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>   create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch 
>>>
>>>   create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>   create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>
>>> -- 
>>> 1.7.10.4
>>>
>>>
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>
>>
Regards,
Jonathan

Re: [PATCH 2/4] e2fsprogs: let debugfs do sparse copy

[Robert Yang]

Hi Darren,

Thank you very much for your detailed review, I will fix them and send
to you and Saul again (but not to oe-core mailing list atm), then send
to ext4 mailing list, please see my comments inline.

On 05/11/2013 05:59 AM, Darren Hart wrote:
>
>
> On 05/07/2013 02:48 AM, Robert Yang wrote:
>> Let debugfs do sparse copy when src is a sparse file, just like
>> "cp --sparse=auto"
>>
>> [YOCTO #3848]
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114 ++++++++++++++++++++
>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    1 +
>>   2 files changed, 115 insertions(+)
>>   create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>
>> diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>> new file mode 100644
>> index 0000000..f20fcaf
>> --- /dev/null
>> +++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>> @@ -0,0 +1,114 @@
>> +debugfs.c: do sparse copy when src is a sparse file
>> +
>> +Let debugfs do sparse copy when src is a sparse file, just like
>> +"cp --sparse=auto"
>> +
>> +Upstream-Status: Pending
>> +
>> +Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> +---
>> + debugfs/debugfs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++----------
>> + 1 file changed, 44 insertions(+), 10 deletions(-)
>> +
>> +diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
>> +index 5cdc86a..015bfad 100644
>> +--- a/debugfs/debugfs.c
>> ++++ b/debugfs/debugfs.c
>> +@@ -37,6 +37,16 @@ extern char *optarg;
>> + #include "../version.h"
>> + #include "jfs_user.h"
>> +
>> ++/* 32KiB is the minimium blksize to best minimize system call overhead. */
>
> Why is that? Can you defend that to Ted and the ext4 developers?
>

I saw it from coreutils-8.13/src/ioblksize.h:

/* As of Mar 2009, 32KiB is determined to be the minimium
    blksize to best minimize system call overhead.
    This can be tested with this script with the results
    shown for a 1.7GHz pentium-m with 2GB of 400MHz DDR2 RAM:

    for i in $(seq 0 10); do
      size=$((8*1024**3)) #ensure this is big enough
      bs=$((1024*2**$i))
      printf "%7s=" $bs
      dd bs=$bs if=/dev/zero of=/dev/null count=$(($size/$bs)) 2>&1 |
      sed -n 's/.* \([0-9.]* [GM]B\/s\)/\1/p'
    done

       1024=734 MB/s
       2048=1.3 GB/s
       4096=2.4 GB/s
       8192=3.5 GB/s
      16384=3.9 GB/s
      32768=5.2 GB/s
      65536=5.3 GB/s
     131072=5.5 GB/s
     262144=5.7 GB/s
     524288=5.7 GB/s
    1048576=5.8 GB/s

    Note that this is to minimize system call overhead.
    Other values may be appropriate to minimize file system
    or disk overhead.  For example on my current GNU/Linux system
    the readahead setting is 128KiB which was read using:

    file="."
    device=$(df -P --local "$file" | tail -n1 | cut -d' ' -f1)
    echo $(( $(blockdev --getra $device) * 512 ))

    However there isn't a portable way to get the above.
    In the future we could use the above method if available
    and default to io_blksize() if not.
  */


>> ++#ifndef IO_BUFSIZE
>> ++#define IO_BUFSIZE 32*1024
>> ++#endif
>> ++
>> ++/* Block size for `st_blocks'. */
>> ++#ifndef S_BLKSIZE
>> ++#define S_BLKSIZE 512
>> ++#endif
>> ++
>> + ss_request_table *extra_cmds;
>> + const char *debug_prog_name;
>> + int sci_idx;
>> +@@ -1559,13 +1569,13 @@ void do_find_free_inode(int argc, char *argv[])
>> + }
>> +
>> + #ifndef READ_ONLY
>> +-static errcode_t copy_file(int fd, ext2_ino_t newfile)
>> ++static errcode_t copy_file(int fd, ext2_ino_t newfile, int bufsiz, int make_holes)
>
> Don't trim off single vowels in variable names. bufsize is fine.
>
>> + {
>> + 	ext2_file_t	e2_file;
>> + 	errcode_t	retval;
>> + 	int		got;
>> + 	unsigned int	written;
>> +-	char		buf[8192];
>> ++	char		buf[bufsiz];
>> + 	char		*ptr;
>> +
>> + 	retval = ext2fs_file_open(current_fs, newfile,
>> +@@ -1582,14 +1592,30 @@ static errcode_t copy_file(int fd, ext2_ino_t newfile)
>> + 			goto fail;
>> + 		}
>> + 		ptr = buf;
>> ++		char *cp = ptr;
>> + 		while (got > 0) {
>> +-			retval = ext2fs_file_write(e2_file, ptr,
>> +-						   got, &written);
>> +-			if (retval)
>> +-				goto fail;
>> +-
>> +-			got -= written;
>> +-			ptr += written;
>> ++			int count = got;
>
> Why declare this inside the while loop?
>

I think that you mean we should:

int count;
while got > 0) {
	...
	count = got;
}

I will update it.

>> ++			if (make_holes) {
>> ++				/* Check whether all is zero */
>> ++				while (count-- && *cp++ == 0) {
>> ++					continue;
>> ++				}
>
>
> Could you improve this a bit by using larger chunks? u32 maybe? You
> could ensure the bufsize rounds up to that, or just enforce it.
>

I think that you mean use a larger "count" ? Usually, the bufsize is 4096
(the I/O block size) when doing sparse copy, so I used "int", but use "u32"
seems reasonable since it is unsigned here. For the bufsize, the suggested
value is 32K, please see my comments in previous.

// Robert

>
>> ++				if (count < 0) {
>> ++					/* The whole block is zero, make a hole */
>> ++					retval = ext2fs_file_lseek(e2_file, got, EXT2_SEEK_CUR, 0);
>> ++					if (retval)
>> ++						goto fail;
>> ++					got = 0;
>> ++				}
>> ++			}
>> ++			/* Normal copy */
>> ++			if (got > 0) {
>> ++				retval = ext2fs_file_write(e2_file, ptr, got, &written);
>> ++				if (retval)
>> ++					goto fail;
>> ++				got -= written;
>> ++				ptr += written;
>> ++			}
>> + 		}
>> + 	}
>> + 	retval = ext2fs_file_close(e2_file);
>> +@@ -1608,6 +1634,8 @@ void do_write(int argc, char *argv[])
>> + 	ext2_ino_t	newfile;
>> + 	errcode_t	retval;
>> + 	struct ext2_inode inode;
>> ++	int make_holes = 0;
>> ++	int bufsiz = IO_BUFSIZE;
>
> bufsize
>
>
>> +
>> + 	if (common_args_process(argc, argv, 3, 3, "write",
>> + 				"<native file> <new file>", CHECK_FS_RW))
>> +@@ -1672,7 +1700,13 @@ void do_write(int argc, char *argv[])
>> + 		return;
>> + 	}
>> + 	if (LINUX_S_ISREG(inode.i_mode)) {
>> +-		retval = copy_file(fd, newfile);
>> ++		if (statbuf.st_blocks < statbuf.st_size / S_BLKSIZE) {
>> ++			make_holes = 1;
>> ++			/* Use the I/O blocksize (st_blksize) as the buffer
>> ++			 * size when copy sparse file */
>
> "when copying sparse files"
>
> Also, multiline comments should probably look like this:
>
>   			/*
> 			 * Use I/O blocksize as buffer size when
> 			 * copying sparse files.
> 			 */
>
> Although, honestly, this doesn't add any information. It is obvious that
> is what the code does. It isn't obvious WHY that is necessary, a comment
> to that effect might be worthwhile, but as is, this one can just be removed.
>
> Thanks,
>
> Darren
>
>
>> ++			bufsiz = statbuf.st_blksize;
>> ++		}
>> ++		retval = copy_file(fd, newfile, bufsiz, make_holes);
>> + 		if (retval)
>> + 			com_err("copy_file", retval, 0);
>> + 	}
>> +--
>> +1.7.11.2
>> +
>> diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
>> index 898ee88..b547036 100644
>> --- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
>> +++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb
>> @@ -5,6 +5,7 @@ PR = "r0"
>>   SRC_URI += "file://acinclude.m4 \
>>               file://remove.ldconfig.call.patch \
>>               file://debugfs-too-short.patch \
>> +            file://sparse_copy.patch \
>>   "
>>
>>   SRC_URI[md5sum] = "a1ec22ef003688dae9f76c74881b22b9"
>>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 05/13/2013 05:50 PM, Jonathan Liu wrote:
> On 13/05/2013 7:23 PM, Robert Yang wrote:
>>
>> Hi Jonathan Liu,
>>
>> What's your configuration, please, I tried a core-image-minimal building
>> just now, didn't notice this error. Did the error come after these patches
>> applied ?
> Yes, the errors occured after applying your patches.
>

Thanks, I will try a build with these configuration tonight, and see the
result tomorrow.

// Robert

> distro conf:
> DISTRO_FEATURES_append = " largefile opengl systemd"
> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
> VIRTUAL-RUNTIME_graphical_init_manager = ""
> VIRTUAL-RUNTIME_init_manager = "systemd"
> PACKAGE_CLASSES = "package_ipk"
>
> image bb:
> SYSLINUX_ROOT ?= "root=/dev/sda2"
> SYSLINUX_PROMPT = "0"
> SYSLINUX_TIMEOUT = "1"
> SYSLINUX_LABELS = "boot"
> LABELS_append = " ${SYSLINUX_LABELS} "
> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
> APPEND += "rootfstype=ext4 rw"
> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
> inherit boot-directdisk
> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
> IMAGE_FSTYPES = "ext4"
> IMAGE_LINGUAS = ""
>
>>
>> My configuration:
>> MACHINE = "qemux86"
>> IMAGE_FSTYPES += "ext4"
>>
>> // Robert
>>
>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>> Hi Robert,
>>>
>>> I get the following errors doing filesystem check on the ext4 image:
>>>
>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>> e2fsck 1.42.7 (21-Jan-2013)
>>> Pass 1: Checking inodes, blocks, and sizes
>>> Pass 2: Checking directory structure
>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
>>> 3).
>>> Fix? no
>>>
>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
>>> 3).
>>> Fix? no
>>>
>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, should be
>>> 3).
>>> Fix? no
>>>
>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
>>> 3).
>>> Fix? no
>>>
>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
>>> 3).
>>> Fix? no
>>>
>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>> Fix? no
>>>
>>> Pass 3: Checking directory connectivity
>>> Pass 4: Checking reference counts
>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>
>>> Pass 5: Checking group summary information
>>>
>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>> Filesystem still has errors **********
>>>
>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
>>> non-contiguous), 131042/169656 blocks
>>>
>>>
>>> After booting the image I get the following:
>>> # dmesg | grep EXT4-fs
>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>> depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>> depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>> depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>> 0(0), depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>> depth 0(0)
>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>> depth 0(0)
>>>
>>>
>>> # systemctl --failed | cat
>>> UNIT                     LOAD   ACTIVE SUB    DESCRIPTION
>>> machineid.service        loaded failed failed Machine ID first boot
>>> configure
>>> systemd-journald.service loaded failed failed Journal Service
>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>
>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>> ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
>>> SUB    = The low-level unit activation state, values depend on unit type.
>>>
>>> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>
>>>
>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>
>>> Regards,
>>> Jonathan
>>>
>>>
>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>
>>>> * The benefits:
>>>>    - Really support ext4
>>>>    - Support the sparse file (the sparse file became into the common file
>>>> before)
>>>>    - Have a uniform code for ext2/3/4 generation
>>>>    - Remove the depends on genext2fs-native
>>>>
>>>> * Impact
>>>>    - Build time:
>>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>>      b) If we do the image generation, which means:
>>>>         $ bitbake core-image-sato
>>>>         $ bitbake core-image-sato -ccleansstate
>>>>         $ bitbake core-image-sato
>>>>     About 50 extra seconds are needed, here is my test result:
>>>>     Before the patches: 4m25s
>>>>     After the patches: 5m17s
>>>>     This is because the genext2fs is much faster than the
>>>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>     supports create the filesystem from a initial directory.
>>>>
>>>>    - Disk space (take core-image-sato as an example)
>>>>      a) The image file size is the same as before (529M)
>>>>      b) The disk usage is a little different: (du -sh)
>>>>                before       now
>>>>         ext2:  364M         388M
>>>>         ext3:  381M         404M
>>>>         ext4:  380M         387M
>>>>
>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>
>>>>     I have done some simple runtime testing on core-image-sato and
>>>>     core-image-minimal, they worked well.
>>>>
>>>> // Robert
>>>>
>>>> The following changes since commit
>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>
>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>
>>>> are available in the git repository at:
>>>>
>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>
>>>> Robert Yang (4):
>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>    e2fsprogs: let debugfs do sparse copy
>>>>    e2fsprogs: add populate-extfs.sh
>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>
>>>>   meta/classes/image_types.bbclass                   |   46 ++++----
>>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       |   28 +++++
>>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   |   93 ++++++++++++++++
>>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   |  114
>>>> ++++++++++++++++++++
>>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb |    4 +
>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>   create mode 100644
>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>   create mode 100644
>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>   create mode 100644
>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>
>>>> --
>>>> 1.7.10.4
>>>>
>>>>
>>>> _______________________________________________
>>>> Openembedded-core mailing list
>>>> Openembedded-core@lists.openembedded.org
>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>
>>>
> Regards,
> Jonathan
>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Jonathan Liu]

On 13/05/2013 8:50 PM, Robert Yang wrote:
>
>
> On 05/13/2013 05:50 PM, Jonathan Liu wrote:
>> On 13/05/2013 7:23 PM, Robert Yang wrote:
>>>
>>> Hi Jonathan Liu,
>>>
>>> What's your configuration, please, I tried a core-image-minimal 
>>> building
>>> just now, didn't notice this error. Did the error come after these 
>>> patches
>>> applied ?
>> Yes, the errors occured after applying your patches.
>>
>
> Thanks, I will try a build with these configuration tonight, and see the
> result tomorrow.
>
> // Robert
I can reproduce the fsck errors using the following:
meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb

conf/local.conf:
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ??= "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES = "package_ipk"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
     STOPTASKS,${TMPDIR},1G,100K \
     STOPTASKS,${DL_DIR},1G,100K \
     STOPTASKS,${SSTATE_DIR},1G,100K \
     ABORT,${TMPDIR},100M,1K \
     ABORT,${DL_DIR},100M,1K \
     ABORT,${SSTATE_DIR},100M,1K"
CONF_VERSION = "1"
DISTRO_FEATURES_append = " systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
IMAGE_FSTYPES = "ext4"

# bitbake core-image-minimal
# fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4

Regards,
Jonathan
>
>> distro conf:
>> DISTRO_FEATURES_append = " largefile opengl systemd"
>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>> VIRTUAL-RUNTIME_init_manager = "systemd"
>> PACKAGE_CLASSES = "package_ipk"
>>
>> image bb:
>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>> SYSLINUX_PROMPT = "0"
>> SYSLINUX_TIMEOUT = "1"
>> SYSLINUX_LABELS = "boot"
>> LABELS_append = " ${SYSLINUX_LABELS} "
>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>> APPEND += "rootfstype=ext4 rw"
>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>> inherit boot-directdisk
>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
>> IMAGE_FSTYPES = "ext4"
>> IMAGE_LINGUAS = ""
>>
>>>
>>> My configuration:
>>> MACHINE = "qemux86"
>>> IMAGE_FSTYPES += "ext4"
>>>
>>> // Robert
>>>
>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>> Hi Robert,
>>>>
>>>> I get the following errors doing filesystem check on the ext4 image:
>>>>
>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>> Pass 1: Checking inodes, blocks, and sizes
>>>> Pass 2: Checking directory structure
>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should 
>>>> be 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should 
>>>> be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be
>>>> 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be
>>>> 3).
>>>> Fix? no
>>>>
>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be
>>>> 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be
>>>> 3).
>>>> Fix? no
>>>>
>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should 
>>>> be 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be
>>>> 3).
>>>> Fix? no
>>>>
>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, 
>>>> should be 3).
>>>> Fix? no
>>>>
>>>> Pass 3: Checking directory connectivity
>>>> Pass 4: Checking reference counts
>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>
>>>> Pass 5: Checking group summary information
>>>>
>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>>> Filesystem still has errors **********
>>>>
>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files 
>>>> (0.5%
>>>> non-contiguous), 131042/169656 blocks
>>>>
>>>>
>>>> After booting the image I get the following:
>>>> # dmesg | grep EXT4-fs
>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: 
>>>> (null)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 
>>>> 0(0),
>>>> depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 
>>>> 0(0),
>>>> depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 
>>>> 0(0),
>>>> depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-machine: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-machine: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: 
>>>> comm
>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>> entries 0, max
>>>> 0(0), depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: 
>>>> comm
>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 
>>>> 0(0),
>>>> depth 0(0)
>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: 
>>>> comm
>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>> depth 0(0)
>>>>
>>>>
>>>> # systemctl --failed | cat
>>>> UNIT                     LOAD   ACTIVE SUB    DESCRIPTION
>>>> machineid.service        loaded failed failed Machine ID first boot
>>>> configure
>>>> systemd-journald.service loaded failed failed Journal Service
>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>
>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>> ACTIVE = The high-level unit activation state, i.e. generalization 
>>>> of SUB.
>>>> SUB    = The low-level unit activation state, values depend on unit 
>>>> type.
>>>>
>>>> 3 loaded units listed. Pass --all to see loaded but inactive units, 
>>>> too.
>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>
>>>>
>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>
>>>> Regards,
>>>> Jonathan
>>>>
>>>>
>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>>
>>>>> * The benefits:
>>>>>    - Really support ext4
>>>>>    - Support the sparse file (the sparse file became into the 
>>>>> common file
>>>>> before)
>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>    - Remove the depends on genext2fs-native
>>>>>
>>>>> * Impact
>>>>>    - Build time:
>>>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>      b) If we do the image generation, which means:
>>>>>         $ bitbake core-image-sato
>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>         $ bitbake core-image-sato
>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>     Before the patches: 4m25s
>>>>>     After the patches: 5m17s
>>>>>     This is because the genext2fs is much faster than the
>>>>>     populate-extfs.sh, we will replace this script by the mke2fs 
>>>>> when it
>>>>>     supports create the filesystem from a initial directory.
>>>>>
>>>>>    - Disk space (take core-image-sato as an example)
>>>>>      a) The image file size is the same as before (529M)
>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>                before       now
>>>>>         ext2:  364M         388M
>>>>>         ext3:  381M         404M
>>>>>         ext4:  380M         387M
>>>>>
>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>
>>>>>     I have done some simple runtime testing on core-image-sato and
>>>>>     core-image-minimal, they worked well.
>>>>>
>>>>> // Robert
>>>>>
>>>>> The following changes since commit
>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>
>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>
>>>>> are available in the git repository at:
>>>>>
>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>
>>>>> Robert Yang (4):
>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>
>>>>>   meta/classes/image_types.bbclass                   | 46 ++++----
>>>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       | 28 +++++
>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   | 93 
>>>>> ++++++++++++++++
>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   | 114
>>>>> ++++++++++++++++++++
>>>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>   create mode 100644
>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch 
>>>>>
>>>>>   create mode 100644
>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>   create mode 100644
>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>
>>>>> -- 
>>>>> 1.7.10.4
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Openembedded-core mailing list
>>>>> Openembedded-core@lists.openembedded.org
>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>>
>>>>
>> Regards,
>> Jonathan
>>
>>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 05/13/2013 09:22 PM, Jonathan Liu wrote:
> On 13/05/2013 8:50 PM, Robert Yang wrote:
>>
>>
>> On 05/13/2013 05:50 PM, Jonathan Liu wrote:
>>> On 13/05/2013 7:23 PM, Robert Yang wrote:
>>>>
>>>> Hi Jonathan Liu,
>>>>
>>>> What's your configuration, please, I tried a core-image-minimal building
>>>> just now, didn't notice this error. Did the error come after these patches
>>>> applied ?
>>> Yes, the errors occured after applying your patches.
>>>
>>
>> Thanks, I will try a build with these configuration tonight, and see the
>> result tomorrow.
>>
>> // Robert
> I can reproduce the fsck errors using the following:

Thanks, I can reproduce it with your setting, will do more investigation on
it later.

// Robert

> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>
> conf/local.conf:
> BB_NUMBER_THREADS = "8"
> PARALLEL_MAKE = "-j 8"
> MACHINE ??= "qemux86"
> DISTRO ?= "poky"
> PACKAGE_CLASSES = "package_ipk"
> EXTRA_IMAGE_FEATURES = "debug-tweaks"
> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
> PATCHRESOLVE = "noop"
> BB_DISKMON_DIRS = "\
>      STOPTASKS,${TMPDIR},1G,100K \
>      STOPTASKS,${DL_DIR},1G,100K \
>      STOPTASKS,${SSTATE_DIR},1G,100K \
>      ABORT,${TMPDIR},100M,1K \
>      ABORT,${DL_DIR},100M,1K \
>      ABORT,${SSTATE_DIR},100M,1K"
> CONF_VERSION = "1"
> DISTRO_FEATURES_append = " systemd"
> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
> VIRTUAL-RUNTIME_init_manager = "systemd"
> IMAGE_FSTYPES = "ext4"
>
> # bitbake core-image-minimal
> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>
> Regards,
> Jonathan
>>
>>> distro conf:
>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>> PACKAGE_CLASSES = "package_ipk"
>>>
>>> image bb:
>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>> SYSLINUX_PROMPT = "0"
>>> SYSLINUX_TIMEOUT = "1"
>>> SYSLINUX_LABELS = "boot"
>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>> APPEND += "rootfstype=ext4 rw"
>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>> inherit boot-directdisk
>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
>>> IMAGE_FSTYPES = "ext4"
>>> IMAGE_LINGUAS = ""
>>>
>>>>
>>>> My configuration:
>>>> MACHINE = "qemux86"
>>>> IMAGE_FSTYPES += "ext4"
>>>>
>>>> // Robert
>>>>
>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>> Hi Robert,
>>>>>
>>>>> I get the following errors doing filesystem check on the ext4 image:
>>>>>
>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>> Pass 2: Checking directory structure
>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>> 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>> 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>> 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>> 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>> 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>> Fix? no
>>>>>
>>>>> Pass 3: Checking directory connectivity
>>>>> Pass 4: Checking reference counts
>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>
>>>>> Pass 5: Checking group summary information
>>>>>
>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>>>> Filesystem still has errors **********
>>>>>
>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
>>>>> non-contiguous), 131042/169656 blocks
>>>>>
>>>>>
>>>>> After booting the image I get the following:
>>>>> # dmesg | grep EXT4-fs
>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>> depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>> depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>> depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>> 0(0), depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>> depth 0(0)
>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>> depth 0(0)
>>>>>
>>>>>
>>>>> # systemctl --failed | cat
>>>>> UNIT                     LOAD   ACTIVE SUB    DESCRIPTION
>>>>> machineid.service        loaded failed failed Machine ID first boot
>>>>> configure
>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>
>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>> ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
>>>>> SUB    = The low-level unit activation state, values depend on unit type.
>>>>>
>>>>> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>
>>>>>
>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>
>>>>> Regards,
>>>>> Jonathan
>>>>>
>>>>>
>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>>>
>>>>>> * The benefits:
>>>>>>    - Really support ext4
>>>>>>    - Support the sparse file (the sparse file became into the common file
>>>>>> before)
>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>    - Remove the depends on genext2fs-native
>>>>>>
>>>>>> * Impact
>>>>>>    - Build time:
>>>>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>>      b) If we do the image generation, which means:
>>>>>>         $ bitbake core-image-sato
>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>         $ bitbake core-image-sato
>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>     Before the patches: 4m25s
>>>>>>     After the patches: 5m17s
>>>>>>     This is because the genext2fs is much faster than the
>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>>>     supports create the filesystem from a initial directory.
>>>>>>
>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>      a) The image file size is the same as before (529M)
>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>                before       now
>>>>>>         ext2:  364M         388M
>>>>>>         ext3:  381M         404M
>>>>>>         ext4:  380M         387M
>>>>>>
>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>>
>>>>>>     I have done some simple runtime testing on core-image-sato and
>>>>>>     core-image-minimal, they worked well.
>>>>>>
>>>>>> // Robert
>>>>>>
>>>>>> The following changes since commit
>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>
>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>>
>>>>>> are available in the git repository at:
>>>>>>
>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>
>>>>>> Robert Yang (4):
>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>
>>>>>>   meta/classes/image_types.bbclass                   | 46 ++++----
>>>>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       | 28 +++++
>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   | 93 ++++++++++++++++
>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   | 114
>>>>>> ++++++++++++++++++++
>>>>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>   create mode 100644
>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>>   create mode 100644
>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>   create mode 100644
>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>
>>>>>> --
>>>>>> 1.7.10.4
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Openembedded-core mailing list
>>>>>> Openembedded-core@lists.openembedded.org
>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>>>
>>>>>
>>> Regards,
>>> Jonathan
>>>
>>>
>
>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Jonathan Liu]

On 14/05/2013 3:32 PM, Robert Yang wrote:
>
>
> On 05/13/2013 09:22 PM, Jonathan Liu wrote:
>> On 13/05/2013 8:50 PM, Robert Yang wrote:
>>>
>>>
>>> On 05/13/2013 05:50 PM, Jonathan Liu wrote:
>>>> On 13/05/2013 7:23 PM, Robert Yang wrote:
>>>>>
>>>>> Hi Jonathan Liu,
>>>>>
>>>>> What's your configuration, please, I tried a core-image-minimal 
>>>>> building
>>>>> just now, didn't notice this error. Did the error come after these 
>>>>> patches
>>>>> applied ?
>>>> Yes, the errors occured after applying your patches.
>>>>
>>>
>>> Thanks, I will try a build with these configuration tonight, and see 
>>> the
>>> result tomorrow.
>>>
>>> // Robert
>> I can reproduce the fsck errors using the following:
>
> Thanks, I can reproduce it with your setting, will do more 
> investigation on
> it later.
>
> // Robert
Any updates?

Regards,
Jonathan
>
>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>>
>> conf/local.conf:
>> BB_NUMBER_THREADS = "8"
>> PARALLEL_MAKE = "-j 8"
>> MACHINE ??= "qemux86"
>> DISTRO ?= "poky"
>> PACKAGE_CLASSES = "package_ipk"
>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>> PATCHRESOLVE = "noop"
>> BB_DISKMON_DIRS = "\
>>      STOPTASKS,${TMPDIR},1G,100K \
>>      STOPTASKS,${DL_DIR},1G,100K \
>>      STOPTASKS,${SSTATE_DIR},1G,100K \
>>      ABORT,${TMPDIR},100M,1K \
>>      ABORT,${DL_DIR},100M,1K \
>>      ABORT,${SSTATE_DIR},100M,1K"
>> CONF_VERSION = "1"
>> DISTRO_FEATURES_append = " systemd"
>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>> VIRTUAL-RUNTIME_init_manager = "systemd"
>> IMAGE_FSTYPES = "ext4"
>>
>> # bitbake core-image-minimal
>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>>
>> Regards,
>> Jonathan
>>>
>>>> distro conf:
>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>> PACKAGE_CLASSES = "package_ipk"
>>>>
>>>> image bb:
>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>>> SYSLINUX_PROMPT = "0"
>>>> SYSLINUX_TIMEOUT = "1"
>>>> SYSLINUX_LABELS = "boot"
>>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>>> APPEND += "rootfstype=ext4 rw"
>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>>> inherit boot-directdisk
>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
>>>> IMAGE_FSTYPES = "ext4"
>>>> IMAGE_LINGUAS = ""
>>>>
>>>>>
>>>>> My configuration:
>>>>> MACHINE = "qemux86"
>>>>> IMAGE_FSTYPES += "ext4"
>>>>>
>>>>> // Robert
>>>>>
>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>>> Hi Robert,
>>>>>>
>>>>>> I get the following errors doing filesystem check on the ext4 image:
>>>>>>
>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>> Pass 2: Checking directory structure
>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be
>>>>>> 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be
>>>>>> 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be
>>>>>> 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be
>>>>>> 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be
>>>>>> 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>> should be 3).
>>>>>> Fix? no
>>>>>>
>>>>>> Pass 3: Checking directory connectivity
>>>>>> Pass 4: Checking reference counts
>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>>
>>>>>> Pass 5: Checking group summary information
>>>>>>
>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>>>>> Filesystem still has errors **********
>>>>>>
>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files 
>>>>>> (0.5%
>>>>>> non-contiguous), 131042/169656 blocks
>>>>>>
>>>>>>
>>>>>> After booting the image I get the following:
>>>>>> # dmesg | grep EXT4-fs
>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: 
>>>>>> (null)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>> max 0(0),
>>>>>> depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>> max 0(0),
>>>>>> depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>> max 0(0),
>>>>>> depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #883: comm
>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>> entries 0, max
>>>>>> 0(0), depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #810: comm
>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 
>>>>>> 0(0),
>>>>>> depth 0(0)
>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>> #810: comm
>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 
>>>>>> 0(0),
>>>>>> depth 0(0)
>>>>>>
>>>>>>
>>>>>> # systemctl --failed | cat
>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
>>>>>> machineid.service        loaded failed failed Machine ID first boot
>>>>>> configure
>>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>>
>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>>> ACTIVE = The high-level unit activation state, i.e. 
>>>>>> generalization of SUB.
>>>>>> SUB    = The low-level unit activation state, values depend on 
>>>>>> unit type.
>>>>>>
>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive 
>>>>>> units, too.
>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>>
>>>>>>
>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>>
>>>>>> Regards,
>>>>>> Jonathan
>>>>>>
>>>>>>
>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>>>>
>>>>>>> * The benefits:
>>>>>>>    - Really support ext4
>>>>>>>    - Support the sparse file (the sparse file became into the 
>>>>>>> common file
>>>>>>> before)
>>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>>    - Remove the depends on genext2fs-native
>>>>>>>
>>>>>>> * Impact
>>>>>>>    - Build time:
>>>>>>>      a) If we build fresh core-image-sato, there is nearly no 
>>>>>>> impact.
>>>>>>>      b) If we do the image generation, which means:
>>>>>>>         $ bitbake core-image-sato
>>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>>         $ bitbake core-image-sato
>>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>>     Before the patches: 4m25s
>>>>>>>     After the patches: 5m17s
>>>>>>>     This is because the genext2fs is much faster than the
>>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs 
>>>>>>> when it
>>>>>>>     supports create the filesystem from a initial directory.
>>>>>>>
>>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>>      a) The image file size is the same as before (529M)
>>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>>                before       now
>>>>>>>         ext2:  364M         388M
>>>>>>>         ext3:  381M         404M
>>>>>>>         ext4:  380M         387M
>>>>>>>
>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 
>>>>>>> 1.4.
>>>>>>>
>>>>>>>     I have done some simple runtime testing on core-image-sato and
>>>>>>>     core-image-minimal, they worked well.
>>>>>>>
>>>>>>> // Robert
>>>>>>>
>>>>>>> The following changes since commit
>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>>
>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>>>
>>>>>>> are available in the git repository at:
>>>>>>>
>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>>
>>>>>>> Robert Yang (4):
>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>>
>>>>>>>   meta/classes/image_types.bbclass                   | 46 ++++----
>>>>>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       | 28 +++++
>>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   | 93 
>>>>>>> ++++++++++++++++
>>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   | 114
>>>>>>> ++++++++++++++++++++
>>>>>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>>   create mode 100644
>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch 
>>>>>>>
>>>>>>>   create mode 100644
>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>>   create mode 100644
>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>>
>>>>>>> -- 
>>>>>>> 1.7.10.4
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Openembedded-core mailing list
>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core 
>>>>>>>
>>>>>>>
>>>>>>
>>>> Regards,
>>>> Jonathan
>>>>
>>>>
>>
>>
>>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 07/06/2013 10:49 PM, Jonathan Liu wrote:
> On 14/05/2013 3:32 PM, Robert Yang wrote:
>>
>>
>> On 05/13/2013 09:22 PM, Jonathan Liu wrote:
>>> On 13/05/2013 8:50 PM, Robert Yang wrote:
>>>>
>>>>
>>>> On 05/13/2013 05:50 PM, Jonathan Liu wrote:
>>>>> On 13/05/2013 7:23 PM, Robert Yang wrote:
>>>>>>
>>>>>> Hi Jonathan Liu,
>>>>>>
>>>>>> What's your configuration, please, I tried a core-image-minimal building
>>>>>> just now, didn't notice this error. Did the error come after these patches
>>>>>> applied ?
>>>>> Yes, the errors occured after applying your patches.
>>>>>
>>>>
>>>> Thanks, I will try a build with these configuration tonight, and see the
>>>> result tomorrow.
>>>>
>>>> // Robert
>>> I can reproduce the fsck errors using the following:
>>
>> Thanks, I can reproduce it with your setting, will do more investigation on
>> it later.
>>
>> // Robert
> Any updates?
>

Sorry, I have no update for it recently, I'm busy on other things, will come
back to it soon.

// Robert


> Regards,
> Jonathan
>>
>>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>>>
>>> conf/local.conf:
>>> BB_NUMBER_THREADS = "8"
>>> PARALLEL_MAKE = "-j 8"
>>> MACHINE ??= "qemux86"
>>> DISTRO ?= "poky"
>>> PACKAGE_CLASSES = "package_ipk"
>>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>> PATCHRESOLVE = "noop"
>>> BB_DISKMON_DIRS = "\
>>>      STOPTASKS,${TMPDIR},1G,100K \
>>>      STOPTASKS,${DL_DIR},1G,100K \
>>>      STOPTASKS,${SSTATE_DIR},1G,100K \
>>>      ABORT,${TMPDIR},100M,1K \
>>>      ABORT,${DL_DIR},100M,1K \
>>>      ABORT,${SSTATE_DIR},100M,1K"
>>> CONF_VERSION = "1"
>>> DISTRO_FEATURES_append = " systemd"
>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>> IMAGE_FSTYPES = "ext4"
>>>
>>> # bitbake core-image-minimal
>>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>>>
>>> Regards,
>>> Jonathan
>>>>
>>>>> distro conf:
>>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>>
>>>>> image bb:
>>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>>>> SYSLINUX_PROMPT = "0"
>>>>> SYSLINUX_TIMEOUT = "1"
>>>>> SYSLINUX_LABELS = "boot"
>>>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>>>> APPEND += "rootfstype=ext4 rw"
>>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>>>> inherit boot-directdisk
>>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
>>>>> IMAGE_FSTYPES = "ext4"
>>>>> IMAGE_LINGUAS = ""
>>>>>
>>>>>>
>>>>>> My configuration:
>>>>>> MACHINE = "qemux86"
>>>>>> IMAGE_FSTYPES += "ext4"
>>>>>>
>>>>>> // Robert
>>>>>>
>>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> I get the following errors doing filesystem check on the ext4 image:
>>>>>>>
>>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>>> Pass 2: Checking directory structure
>>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Pass 3: Checking directory connectivity
>>>>>>> Pass 4: Checking reference counts
>>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>>>
>>>>>>> Pass 5: Checking group summary information
>>>>>>>
>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>>>>>> Filesystem still has errors **********
>>>>>>>
>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
>>>>>>> non-contiguous), 131042/169656 blocks
>>>>>>>
>>>>>>>
>>>>>>> After booting the image I get the following:
>>>>>>> # dmesg | grep EXT4-fs
>>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>>
>>>>>>>
>>>>>>> # systemctl --failed | cat
>>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
>>>>>>> machineid.service        loaded failed failed Machine ID first boot
>>>>>>> configure
>>>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>>>
>>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>>>> ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
>>>>>>> SUB    = The low-level unit activation state, values depend on unit type.
>>>>>>>
>>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
>>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>>>
>>>>>>>
>>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jonathan
>>>>>>>
>>>>>>>
>>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>>>>>
>>>>>>>> * The benefits:
>>>>>>>>    - Really support ext4
>>>>>>>>    - Support the sparse file (the sparse file became into the common file
>>>>>>>> before)
>>>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>>>    - Remove the depends on genext2fs-native
>>>>>>>>
>>>>>>>> * Impact
>>>>>>>>    - Build time:
>>>>>>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>>>>      b) If we do the image generation, which means:
>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>>>     Before the patches: 4m25s
>>>>>>>>     After the patches: 5m17s
>>>>>>>>     This is because the genext2fs is much faster than the
>>>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>>>>>     supports create the filesystem from a initial directory.
>>>>>>>>
>>>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>>>      a) The image file size is the same as before (529M)
>>>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>>>                before       now
>>>>>>>>         ext2:  364M         388M
>>>>>>>>         ext3:  381M         404M
>>>>>>>>         ext4:  380M         387M
>>>>>>>>
>>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>>>>
>>>>>>>>     I have done some simple runtime testing on core-image-sato and
>>>>>>>>     core-image-minimal, they worked well.
>>>>>>>>
>>>>>>>> // Robert
>>>>>>>>
>>>>>>>> The following changes since commit
>>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>>>
>>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>>>>
>>>>>>>> are available in the git repository at:
>>>>>>>>
>>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>>>
>>>>>>>> Robert Yang (4):
>>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>>>
>>>>>>>>   meta/classes/image_types.bbclass                   | 46 ++++----
>>>>>>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       | 28 +++++
>>>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   | 93 ++++++++++++++++
>>>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   | 114
>>>>>>>> ++++++++++++++++++++
>>>>>>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>>>   create mode 100644
>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>>>>   create mode 100644
>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>>>   create mode 100644
>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>>>
>>>>>>>> --
>>>>>>>> 1.7.10.4
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Openembedded-core mailing list
>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>>>>>
>>>>>>>
>>>>> Regards,
>>>>> Jonathan
>>>>>
>>>>>
>>>
>>>
>>>
>
>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 07/06/2013 10:49 PM, Jonathan Liu wrote:
> Any updates?
>

Hi Jonathan,

I've updated the patches, now they are working well, the "fsck -fn" finds
no errors any more, but I have to send the patches to the ext4 mailing list
firstly, you can try it if you are interested in it atm.:

   git://git.pokylinux.org/poky-contrib robert/image
   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/image


Robert Yang (5):
   e2fsprogs: the max length of debugfs argument is too short
   e2fsprogs: let debugfs do sparse copy
   e2fsprogs: only update the icache for ext2_inode
   e2fsprogs: add populate-extfs.sh
   image_types.bbclass: replace genext2fs with populate-extfs.sh

  meta/classes/image_types.bbclass                   |  46 +++---
  .../e2fsprogs-1.42.8/debugfs-too-short.patch       |  28 ++++
  .../e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch    |  69 ++++++++
  .../e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh   |  96 +++++++++++
  .../e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch   | 177 +++++++++++++++++++++
  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb |   5 +
  6 files changed, 393 insertions(+), 28 deletions(-)
  create mode 100644 
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/debugfs-too-short.patch
  create mode 100644 
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch
  create mode 100644 
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh
  create mode 100644 
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch

// Robert

> Regards,
> Jonathan
>>
>>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>>>
>>> conf/local.conf:
>>> BB_NUMBER_THREADS = "8"
>>> PARALLEL_MAKE = "-j 8"
>>> MACHINE ??= "qemux86"
>>> DISTRO ?= "poky"
>>> PACKAGE_CLASSES = "package_ipk"
>>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>> PATCHRESOLVE = "noop"
>>> BB_DISKMON_DIRS = "\
>>>      STOPTASKS,${TMPDIR},1G,100K \
>>>      STOPTASKS,${DL_DIR},1G,100K \
>>>      STOPTASKS,${SSTATE_DIR},1G,100K \
>>>      ABORT,${TMPDIR},100M,1K \
>>>      ABORT,${DL_DIR},100M,1K \
>>>      ABORT,${SSTATE_DIR},100M,1K"
>>> CONF_VERSION = "1"
>>> DISTRO_FEATURES_append = " systemd"
>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>> IMAGE_FSTYPES = "ext4"
>>>
>>> # bitbake core-image-minimal
>>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>>>
>>> Regards,
>>> Jonathan
>>>>
>>>>> distro conf:
>>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>>
>>>>> image bb:
>>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>>>> SYSLINUX_PROMPT = "0"
>>>>> SYSLINUX_TIMEOUT = "1"
>>>>> SYSLINUX_LABELS = "boot"
>>>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>>>> APPEND += "rootfstype=ext4 rw"
>>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>>>> inherit boot-directdisk
>>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
>>>>> IMAGE_FSTYPES = "ext4"
>>>>> IMAGE_LINGUAS = ""
>>>>>
>>>>>>
>>>>>> My configuration:
>>>>>> MACHINE = "qemux86"
>>>>>> IMAGE_FSTYPES += "ext4"
>>>>>>
>>>>>> // Robert
>>>>>>
>>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> I get the following errors doing filesystem check on the ext4 image:
>>>>>>>
>>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>>> Pass 2: Checking directory structure
>>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>> 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>> Fix? no
>>>>>>>
>>>>>>> Pass 3: Checking directory connectivity
>>>>>>> Pass 4: Checking reference counts
>>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>>>
>>>>>>> Pass 5: Checking group summary information
>>>>>>>
>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>>>>>> Filesystem still has errors **********
>>>>>>>
>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
>>>>>>> non-contiguous), 131042/169656 blocks
>>>>>>>
>>>>>>>
>>>>>>> After booting the image I get the following:
>>>>>>> # dmesg | grep EXT4-fs
>>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>> 0(0), depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>> depth 0(0)
>>>>>>>
>>>>>>>
>>>>>>> # systemctl --failed | cat
>>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
>>>>>>> machineid.service        loaded failed failed Machine ID first boot
>>>>>>> configure
>>>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>>>
>>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>>>> ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
>>>>>>> SUB    = The low-level unit activation state, values depend on unit type.
>>>>>>>
>>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
>>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>>>
>>>>>>>
>>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jonathan
>>>>>>>
>>>>>>>
>>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>>>>>
>>>>>>>> * The benefits:
>>>>>>>>    - Really support ext4
>>>>>>>>    - Support the sparse file (the sparse file became into the common file
>>>>>>>> before)
>>>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>>>    - Remove the depends on genext2fs-native
>>>>>>>>
>>>>>>>> * Impact
>>>>>>>>    - Build time:
>>>>>>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>>>>      b) If we do the image generation, which means:
>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>>>     Before the patches: 4m25s
>>>>>>>>     After the patches: 5m17s
>>>>>>>>     This is because the genext2fs is much faster than the
>>>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>>>>>     supports create the filesystem from a initial directory.
>>>>>>>>
>>>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>>>      a) The image file size is the same as before (529M)
>>>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>>>                before       now
>>>>>>>>         ext2:  364M         388M
>>>>>>>>         ext3:  381M         404M
>>>>>>>>         ext4:  380M         387M
>>>>>>>>
>>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>>>>
>>>>>>>>     I have done some simple runtime testing on core-image-sato and
>>>>>>>>     core-image-minimal, they worked well.
>>>>>>>>
>>>>>>>> // Robert
>>>>>>>>
>>>>>>>> The following changes since commit
>>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>>>
>>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>>>>
>>>>>>>> are available in the git repository at:
>>>>>>>>
>>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>>>
>>>>>>>> Robert Yang (4):
>>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>>>
>>>>>>>>   meta/classes/image_types.bbclass                   | 46 ++++----
>>>>>>>>   .../e2fsprogs-1.42.7/debugfs-too-short.patch       | 28 +++++
>>>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh   | 93 ++++++++++++++++
>>>>>>>>   .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch   | 114
>>>>>>>> ++++++++++++++++++++
>>>>>>>>   .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>>>   create mode 100644
>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>>>>   create mode 100644
>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>>>   create mode 100644
>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>>>
>>>>>>>> --
>>>>>>>> 1.7.10.4
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Openembedded-core mailing list
>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>>>>>
>>>>>>>
>>>>> Regards,
>>>>> Jonathan
>>>>>
>>>>>
>>>
>>>
>>>
>
>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Jonathan Liu]

On 18/07/2013 10:32 PM, Robert Yang wrote:
>
> On 07/06/2013 10:49 PM, Jonathan Liu wrote:
>> Any updates?
>>
>
> Hi Jonathan,
>
> I've updated the patches, now they are working well, the "fsck -fn" finds
> no errors any more, but I have to send the patches to the ext4 mailing 
> list
> firstly, you can try it if you are interested in it atm.:
>
>   git://git.pokylinux.org/poky-contrib robert/image
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/image
>
>
> Robert Yang (5):
>   e2fsprogs: the max length of debugfs argument is too short
>   e2fsprogs: let debugfs do sparse copy
>   e2fsprogs: only update the icache for ext2_inode
>   e2fsprogs: add populate-extfs.sh
>   image_types.bbclass: replace genext2fs with populate-extfs.sh
>
>  meta/classes/image_types.bbclass                   |  46 +++---
>  .../e2fsprogs-1.42.8/debugfs-too-short.patch       |  28 ++++
>  .../e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch    |  69 ++++++++
>  .../e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh   |  96 +++++++++++
>  .../e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch   | 177 
> +++++++++++++++++++++
>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb |   5 +
>  6 files changed, 393 insertions(+), 28 deletions(-)
>  create mode 100644 
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/debugfs-too-short.patch
>  create mode 100644 
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch
>  create mode 100644 
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh
>  create mode 100644 
> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch
>
> // Robert
Thanks, I will test it tomorrow. "yes | mkfs.$fstype" could probably be 
replaced with "mkfs.$fstype -F".

Regards,
Jonathan
>
>> Regards,
>> Jonathan
>>>
>>>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>>>>
>>>> conf/local.conf:
>>>> BB_NUMBER_THREADS = "8"
>>>> PARALLEL_MAKE = "-j 8"
>>>> MACHINE ??= "qemux86"
>>>> DISTRO ?= "poky"
>>>> PACKAGE_CLASSES = "package_ipk"
>>>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>>> PATCHRESOLVE = "noop"
>>>> BB_DISKMON_DIRS = "\
>>>>      STOPTASKS,${TMPDIR},1G,100K \
>>>>      STOPTASKS,${DL_DIR},1G,100K \
>>>>      STOPTASKS,${SSTATE_DIR},1G,100K \
>>>>      ABORT,${TMPDIR},100M,1K \
>>>>      ABORT,${DL_DIR},100M,1K \
>>>>      ABORT,${SSTATE_DIR},100M,1K"
>>>> CONF_VERSION = "1"
>>>> DISTRO_FEATURES_append = " systemd"
>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>> IMAGE_FSTYPES = "ext4"
>>>>
>>>> # bitbake core-image-minimal
>>>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>>>>
>>>> Regards,
>>>> Jonathan
>>>>>
>>>>>> distro conf:
>>>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>>>
>>>>>> image bb:
>>>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>>>>> SYSLINUX_PROMPT = "0"
>>>>>> SYSLINUX_TIMEOUT = "1"
>>>>>> SYSLINUX_LABELS = "boot"
>>>>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>>>>> APPEND += "rootfstype=ext4 rw"
>>>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>>>>> inherit boot-directdisk
>>>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh 
>>>>>> x11"
>>>>>> IMAGE_FSTYPES = "ext4"
>>>>>> IMAGE_LINGUAS = ""
>>>>>>
>>>>>>>
>>>>>>> My configuration:
>>>>>>> MACHINE = "qemux86"
>>>>>>> IMAGE_FSTYPES += "ext4"
>>>>>>>
>>>>>>> // Robert
>>>>>>>
>>>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>>>>> Hi Robert,
>>>>>>>>
>>>>>>>> I get the following errors doing filesystem check on the ext4 
>>>>>>>> image:
>>>>>>>>
>>>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>>>> Pass 2: Checking directory structure
>>>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be
>>>>>>>> 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 
>>>>>>>> 4, should be
>>>>>>>> 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 
>>>>>>>> 4, should be
>>>>>>>> 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be
>>>>>>>> 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 
>>>>>>>> 4, should be
>>>>>>>> 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, 
>>>>>>>> should be 3).
>>>>>>>> Fix? no
>>>>>>>>
>>>>>>>> Pass 3: Checking directory connectivity
>>>>>>>> Pass 4: Checking reference counts
>>>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>>>>
>>>>>>>> Pass 5: Checking group summary information
>>>>>>>>
>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** 
>>>>>>>> WARNING:
>>>>>>>> Filesystem still has errors **********
>>>>>>>>
>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 
>>>>>>>> files (0.5%
>>>>>>>> non-contiguous), 131042/169656 blocks
>>>>>>>>
>>>>>>>>
>>>>>>>> After booting the image I get the following:
>>>>>>>> # dmesg | grep EXT4-fs
>>>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. 
>>>>>>>> Opts: (null)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>>>> max 0(0),
>>>>>>>> depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>>>> max 0(0),
>>>>>>>> depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>>>> max 0(0),
>>>>>>>> depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #883: comm
>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, 
>>>>>>>> entries 0, max
>>>>>>>> 0(0), depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #810: comm
>>>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>>>> max 0(0),
>>>>>>>> depth 0(0)
>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode 
>>>>>>>> #810: comm
>>>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, 
>>>>>>>> max 0(0),
>>>>>>>> depth 0(0)
>>>>>>>>
>>>>>>>>
>>>>>>>> # systemctl --failed | cat
>>>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
>>>>>>>> machineid.service        loaded failed failed Machine ID first 
>>>>>>>> boot
>>>>>>>> configure
>>>>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>>>>
>>>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>>>>> ACTIVE = The high-level unit activation state, i.e. 
>>>>>>>> generalization of SUB.
>>>>>>>> SUB    = The low-level unit activation state, values depend on 
>>>>>>>> unit type.
>>>>>>>>
>>>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive 
>>>>>>>> units, too.
>>>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>>>>
>>>>>>>>
>>>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Jonathan
>>>>>>>>
>>>>>>>>
>>>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> * The benefits:
>>>>>>>>>    - Really support ext4
>>>>>>>>>    - Support the sparse file (the sparse file became into the 
>>>>>>>>> common file
>>>>>>>>> before)
>>>>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>>>>    - Remove the depends on genext2fs-native
>>>>>>>>>
>>>>>>>>> * Impact
>>>>>>>>>    - Build time:
>>>>>>>>>      a) If we build fresh core-image-sato, there is nearly no 
>>>>>>>>> impact.
>>>>>>>>>      b) If we do the image generation, which means:
>>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>>>>     Before the patches: 4m25s
>>>>>>>>>     After the patches: 5m17s
>>>>>>>>>     This is because the genext2fs is much faster than the
>>>>>>>>>     populate-extfs.sh, we will replace this script by the 
>>>>>>>>> mke2fs when it
>>>>>>>>>     supports create the filesystem from a initial directory.
>>>>>>>>>
>>>>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>>>>      a) The image file size is the same as before (529M)
>>>>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>>>>                before       now
>>>>>>>>>         ext2:  364M         388M
>>>>>>>>>         ext3:  381M         404M
>>>>>>>>>         ext4:  380M         387M
>>>>>>>>>
>>>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 
>>>>>>>>> to 1.4.
>>>>>>>>>
>>>>>>>>>     I have done some simple runtime testing on core-image-sato 
>>>>>>>>> and
>>>>>>>>>     core-image-minimal, they worked well.
>>>>>>>>>
>>>>>>>>> // Robert
>>>>>>>>>
>>>>>>>>> The following changes since commit
>>>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>>>>
>>>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 
>>>>>>>>> +0100)
>>>>>>>>>
>>>>>>>>> are available in the git repository at:
>>>>>>>>>
>>>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>>>>
>>>>>>>>> Robert Yang (4):
>>>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>>>>
>>>>>>>>> meta/classes/image_types.bbclass | 46 ++++----
>>>>>>>>> .../e2fsprogs-1.42.7/debugfs-too-short.patch | 28 +++++
>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh | 93 
>>>>>>>>> ++++++++++++++++
>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch | 114
>>>>>>>>> ++++++++++++++++++++
>>>>>>>>> .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>>>>   create mode 100644
>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch 
>>>>>>>>>
>>>>>>>>>   create mode 100644
>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh 
>>>>>>>>>
>>>>>>>>>   create mode 100644
>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> 1.7.10.4
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Openembedded-core mailing list
>>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>> Regards,
>>>>>> Jonathan
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>
>>
>>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Robert Yang]

On 07/18/2013 09:13 PM, Jonathan Liu wrote:
> On 18/07/2013 10:32 PM, Robert Yang wrote:
>>
>> On 07/06/2013 10:49 PM, Jonathan Liu wrote:
>>> Any updates?
>>>
>>
>> Hi Jonathan,
>>
>> I've updated the patches, now they are working well, the "fsck -fn" finds
>> no errors any more, but I have to send the patches to the ext4 mailing list
>> firstly, you can try it if you are interested in it atm.:
>>
>>   git://git.pokylinux.org/poky-contrib robert/image
>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/image
>>
>>
>> Robert Yang (5):
>>   e2fsprogs: the max length of debugfs argument is too short
>>   e2fsprogs: let debugfs do sparse copy
>>   e2fsprogs: only update the icache for ext2_inode
>>   e2fsprogs: add populate-extfs.sh
>>   image_types.bbclass: replace genext2fs with populate-extfs.sh
>>
>>  meta/classes/image_types.bbclass                   |  46 +++---
>>  .../e2fsprogs-1.42.8/debugfs-too-short.patch       |  28 ++++
>>  .../e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch    |  69 ++++++++
>>  .../e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh   |  96 +++++++++++
>>  .../e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch   | 177 +++++++++++++++++++++
>>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb |   5 +
>>  6 files changed, 393 insertions(+), 28 deletions(-)
>>  create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/debugfs-too-short.patch
>>  create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch
>>  create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh
>>  create mode 100644
>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch
>>
>> // Robert
> Thanks, I will test it tomorrow. "yes | mkfs.$fstype" could probably be replaced
> with "mkfs.$fstype -F".
>

Yes, it should be:-)

// Robert

> Regards,
> Jonathan
>>
>>> Regards,
>>> Jonathan
>>>>
>>>>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>>>>>
>>>>> conf/local.conf:
>>>>> BB_NUMBER_THREADS = "8"
>>>>> PARALLEL_MAKE = "-j 8"
>>>>> MACHINE ??= "qemux86"
>>>>> DISTRO ?= "poky"
>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>>>> PATCHRESOLVE = "noop"
>>>>> BB_DISKMON_DIRS = "\
>>>>>      STOPTASKS,${TMPDIR},1G,100K \
>>>>>      STOPTASKS,${DL_DIR},1G,100K \
>>>>>      STOPTASKS,${SSTATE_DIR},1G,100K \
>>>>>      ABORT,${TMPDIR},100M,1K \
>>>>>      ABORT,${DL_DIR},100M,1K \
>>>>>      ABORT,${SSTATE_DIR},100M,1K"
>>>>> CONF_VERSION = "1"
>>>>> DISTRO_FEATURES_append = " systemd"
>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>> IMAGE_FSTYPES = "ext4"
>>>>>
>>>>> # bitbake core-image-minimal
>>>>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>>>>>
>>>>> Regards,
>>>>> Jonathan
>>>>>>
>>>>>>> distro conf:
>>>>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>>>>
>>>>>>> image bb:
>>>>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>>>>>> SYSLINUX_PROMPT = "0"
>>>>>>> SYSLINUX_TIMEOUT = "1"
>>>>>>> SYSLINUX_LABELS = "boot"
>>>>>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>>>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>>>>>> APPEND += "rootfstype=ext4 rw"
>>>>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>>>>>> inherit boot-directdisk
>>>>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh x11"
>>>>>>> IMAGE_FSTYPES = "ext4"
>>>>>>> IMAGE_LINGUAS = ""
>>>>>>>
>>>>>>>>
>>>>>>>> My configuration:
>>>>>>>> MACHINE = "qemux86"
>>>>>>>> IMAGE_FSTYPES += "ext4"
>>>>>>>>
>>>>>>>> // Robert
>>>>>>>>
>>>>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>>>>>> Hi Robert,
>>>>>>>>>
>>>>>>>>> I get the following errors doing filesystem check on the ext4 image:
>>>>>>>>>
>>>>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>>>>> Pass 2: Checking directory structure
>>>>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>> should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4, should be 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4, should
>>>>>>>>> be 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4, should be
>>>>>>>>> 3).
>>>>>>>>> Fix? no
>>>>>>>>>
>>>>>>>>> Pass 3: Checking directory connectivity
>>>>>>>>> Pass 4: Checking reference counts
>>>>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>>>>>
>>>>>>>>> Pass 5: Checking group summary information
>>>>>>>>>
>>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: ********** WARNING:
>>>>>>>>> Filesystem still has errors **********
>>>>>>>>>
>>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files (0.5%
>>>>>>>>> non-contiguous), 131042/169656 blocks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> After booting the image I get the following:
>>>>>>>>> # dmesg | grep EXT4-fs
>>>>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>>>> depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>>>> depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>>>> depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883: comm
>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>> max
>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>>>> depth 0(0)
>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810: comm
>>>>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max 0(0),
>>>>>>>>> depth 0(0)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> # systemctl --failed | cat
>>>>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
>>>>>>>>> machineid.service        loaded failed failed Machine ID first boot
>>>>>>>>> configure
>>>>>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>>>>>
>>>>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>>>>>> ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
>>>>>>>>> SUB    = The low-level unit activation state, values depend on unit type.
>>>>>>>>>
>>>>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive units, too.
>>>>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Jonathan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>>>>>>>
>>>>>>>>>> * The benefits:
>>>>>>>>>>    - Really support ext4
>>>>>>>>>>    - Support the sparse file (the sparse file became into the common file
>>>>>>>>>> before)
>>>>>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>>>>>    - Remove the depends on genext2fs-native
>>>>>>>>>>
>>>>>>>>>> * Impact
>>>>>>>>>>    - Build time:
>>>>>>>>>>      a) If we build fresh core-image-sato, there is nearly no impact.
>>>>>>>>>>      b) If we do the image generation, which means:
>>>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>>>>>     Before the patches: 4m25s
>>>>>>>>>>     After the patches: 5m17s
>>>>>>>>>>     This is because the genext2fs is much faster than the
>>>>>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs when it
>>>>>>>>>>     supports create the filesystem from a initial directory.
>>>>>>>>>>
>>>>>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>>>>>      a) The image file size is the same as before (529M)
>>>>>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>>>>>                before       now
>>>>>>>>>>         ext2:  364M         388M
>>>>>>>>>>         ext3:  381M         404M
>>>>>>>>>>         ext4:  380M         387M
>>>>>>>>>>
>>>>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to 1.4.
>>>>>>>>>>
>>>>>>>>>>     I have done some simple runtime testing on core-image-sato and
>>>>>>>>>>     core-image-minimal, they worked well.
>>>>>>>>>>
>>>>>>>>>> // Robert
>>>>>>>>>>
>>>>>>>>>> The following changes since commit
>>>>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>>>>>
>>>>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05 +0100)
>>>>>>>>>>
>>>>>>>>>> are available in the git repository at:
>>>>>>>>>>
>>>>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>>>>>
>>>>>>>>>> Robert Yang (4):
>>>>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>>>>>
>>>>>>>>>> meta/classes/image_types.bbclass | 46 ++++----
>>>>>>>>>> .../e2fsprogs-1.42.7/debugfs-too-short.patch | 28 +++++
>>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh | 93 ++++++++++++++++
>>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch | 114
>>>>>>>>>> ++++++++++++++++++++
>>>>>>>>>> .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>>>>>   create mode 100644
>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>>>>>>   create mode 100644
>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>>>>>   create mode 100644
>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> 1.7.10.4
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Openembedded-core mailing list
>>>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>>>>>>>
>>>>>>>>>
>>>>>>> Regards,
>>>>>>> Jonathan
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>
>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Jonathan Liu]

Hi Robert,

I tested your patches with my ext4 image on Intel Atom D2500HN board
and it is working well.
Previously I have been using custom image class which requires root
privileges as I need to be able to specify custom options to mkfs.ext4
which are not available with genext2fs and cannot be set with tune2fs
afterwards. Using your patches I just put the extra options in
EXTRA_IMAGECMD with ext4 image class and I no longer need root
privileges for building image. =)

Thanks for your efforts.

Regards,
Jonathan

On 19 July 2013 00:54, Robert Yang <liezhi.yang@windriver.com> wrote:
>
> Hi Jonathan,
>
> I've updated it a little, please make sure that you get the newest patches,
> I've payed most of the time on e2fsprogs itself atm, so I didn't do enough
> testing of the image (I will do it sooner)  thank you very much for focusing
> on this.
>
> // Robert
>
>
> On 07/18/2013 09:13 PM, Jonathan Liu wrote:
>>
>> On 18/07/2013 10:32 PM, Robert Yang wrote:
>>>
>>>
>>> On 07/06/2013 10:49 PM, Jonathan Liu wrote:
>>>>
>>>> Any updates?
>>>>
>>>
>>> Hi Jonathan,
>>>
>>> I've updated the patches, now they are working well, the "fsck -fn" finds
>>> no errors any more, but I have to send the patches to the ext4 mailing
>>> list
>>> firstly, you can try it if you are interested in it atm.:
>>>
>>>   git://git.pokylinux.org/poky-contrib robert/image
>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/image
>>>
>>>
>>> Robert Yang (5):
>>>   e2fsprogs: the max length of debugfs argument is too short
>>>   e2fsprogs: let debugfs do sparse copy
>>>   e2fsprogs: only update the icache for ext2_inode
>>>   e2fsprogs: add populate-extfs.sh
>>>   image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>
>>>  meta/classes/image_types.bbclass                   |  46 +++---
>>>  .../e2fsprogs-1.42.8/debugfs-too-short.patch       |  28 ++++
>>>  .../e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch    |  69 ++++++++
>>>  .../e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh   |  96 +++++++++++
>>>  .../e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch   | 177
>>> +++++++++++++++++++++
>>>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb |   5 +
>>>  6 files changed, 393 insertions(+), 28 deletions(-)
>>>  create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/debugfs-too-short.patch
>>>  create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch
>>>  create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh
>>>  create mode 100644
>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch
>>>
>>> // Robert
>>
>> Thanks, I will test it tomorrow. "yes | mkfs.$fstype" could probably be
>> replaced
>> with "mkfs.$fstype -F".
>>
>> Regards,
>> Jonathan
>>>
>>>
>>>> Regards,
>>>> Jonathan
>>>>>
>>>>>
>>>>>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
>>>>>>
>>>>>> conf/local.conf:
>>>>>> BB_NUMBER_THREADS = "8"
>>>>>> PARALLEL_MAKE = "-j 8"
>>>>>> MACHINE ??= "qemux86"
>>>>>> DISTRO ?= "poky"
>>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>>>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>>>>> PATCHRESOLVE = "noop"
>>>>>> BB_DISKMON_DIRS = "\
>>>>>>      STOPTASKS,${TMPDIR},1G,100K \
>>>>>>      STOPTASKS,${DL_DIR},1G,100K \
>>>>>>      STOPTASKS,${SSTATE_DIR},1G,100K \
>>>>>>      ABORT,${TMPDIR},100M,1K \
>>>>>>      ABORT,${DL_DIR},100M,1K \
>>>>>>      ABORT,${SSTATE_DIR},100M,1K"
>>>>>> CONF_VERSION = "1"
>>>>>> DISTRO_FEATURES_append = " systemd"
>>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>>> IMAGE_FSTYPES = "ext4"
>>>>>>
>>>>>> # bitbake core-image-minimal
>>>>>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
>>>>>>
>>>>>> Regards,
>>>>>> Jonathan
>>>>>>>
>>>>>>>
>>>>>>>> distro conf:
>>>>>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
>>>>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
>>>>>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
>>>>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
>>>>>>>> PACKAGE_CLASSES = "package_ipk"
>>>>>>>>
>>>>>>>> image bb:
>>>>>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
>>>>>>>> SYSLINUX_PROMPT = "0"
>>>>>>>> SYSLINUX_TIMEOUT = "1"
>>>>>>>> SYSLINUX_LABELS = "boot"
>>>>>>>> LABELS_append = " ${SYSLINUX_LABELS} "
>>>>>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
>>>>>>>> APPEND += "rootfstype=ext4 rw"
>>>>>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
>>>>>>>> inherit boot-directdisk
>>>>>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh
>>>>>>>> x11"
>>>>>>>> IMAGE_FSTYPES = "ext4"
>>>>>>>> IMAGE_LINGUAS = ""
>>>>>>>>
>>>>>>>>>
>>>>>>>>> My configuration:
>>>>>>>>> MACHINE = "qemux86"
>>>>>>>>> IMAGE_FSTYPES += "ext4"
>>>>>>>>>
>>>>>>>>> // Robert
>>>>>>>>>
>>>>>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Robert,
>>>>>>>>>>
>>>>>>>>>> I get the following errors doing filesystem check on the ext4
>>>>>>>>>> image:
>>>>>>>>>>
>>>>>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
>>>>>>>>>> e2fsck 1.42.7 (21-Jan-2013)
>>>>>>>>>> Pass 1: Checking inodes, blocks, and sizes
>>>>>>>>>> Pass 2: Checking directory structure
>>>>>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should
>>>>>>>>>> be 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4,
>>>>>>>>>> should be
>>>>>>>>>> 3).
>>>>>>>>>> Fix? no
>>>>>>>>>>
>>>>>>>>>> Pass 3: Checking directory connectivity
>>>>>>>>>> Pass 4: Checking reference counts
>>>>>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
>>>>>>>>>>
>>>>>>>>>> Pass 5: Checking group summary information
>>>>>>>>>>
>>>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: **********
>>>>>>>>>> WARNING:
>>>>>>>>>> Filesystem still has errors **********
>>>>>>>>>>
>>>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files
>>>>>>>>>> (0.5%
>>>>>>>>>> non-contiguous), 131042/169656 blocks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> After booting the image I get the following:
>>>>>>>>>> # dmesg | grep EXT4-fs
>>>>>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts:
>>>>>>>>>> (null)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>>> max 0(0),
>>>>>>>>>> depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>>> max 0(0),
>>>>>>>>>> depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0,
>>>>>>>>>> max 0(0),
>>>>>>>>>> depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
>>>>>>>>>> comm
>>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
>>>>>>>>>> entries 0,
>>>>>>>>>> max
>>>>>>>>>> 0(0), depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810:
>>>>>>>>>> comm
>>>>>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>>>>> 0(0),
>>>>>>>>>> depth 0(0)
>>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810:
>>>>>>>>>> comm
>>>>>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max
>>>>>>>>>> 0(0),
>>>>>>>>>> depth 0(0)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> # systemctl --failed | cat
>>>>>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
>>>>>>>>>> machineid.service        loaded failed failed Machine ID first
>>>>>>>>>> boot
>>>>>>>>>> configure
>>>>>>>>>> systemd-journald.service loaded failed failed Journal Service
>>>>>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
>>>>>>>>>>
>>>>>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
>>>>>>>>>> ACTIVE = The high-level unit activation state, i.e. generalization
>>>>>>>>>> of SUB.
>>>>>>>>>> SUB    = The low-level unit activation state, values depend on
>>>>>>>>>> unit type.
>>>>>>>>>>
>>>>>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive
>>>>>>>>>> units, too.
>>>>>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Jonathan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> * The benefits:
>>>>>>>>>>>    - Really support ext4
>>>>>>>>>>>    - Support the sparse file (the sparse file became into the
>>>>>>>>>>> common file
>>>>>>>>>>> before)
>>>>>>>>>>>    - Have a uniform code for ext2/3/4 generation
>>>>>>>>>>>    - Remove the depends on genext2fs-native
>>>>>>>>>>>
>>>>>>>>>>> * Impact
>>>>>>>>>>>    - Build time:
>>>>>>>>>>>      a) If we build fresh core-image-sato, there is nearly no
>>>>>>>>>>> impact.
>>>>>>>>>>>      b) If we do the image generation, which means:
>>>>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>>>>         $ bitbake core-image-sato -ccleansstate
>>>>>>>>>>>         $ bitbake core-image-sato
>>>>>>>>>>>     About 50 extra seconds are needed, here is my test result:
>>>>>>>>>>>     Before the patches: 4m25s
>>>>>>>>>>>     After the patches: 5m17s
>>>>>>>>>>>     This is because the genext2fs is much faster than the
>>>>>>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs
>>>>>>>>>>> when it
>>>>>>>>>>>     supports create the filesystem from a initial directory.
>>>>>>>>>>>
>>>>>>>>>>>    - Disk space (take core-image-sato as an example)
>>>>>>>>>>>      a) The image file size is the same as before (529M)
>>>>>>>>>>>      b) The disk usage is a little different: (du -sh)
>>>>>>>>>>>                before       now
>>>>>>>>>>>         ext2:  364M         388M
>>>>>>>>>>>         ext3:  381M         404M
>>>>>>>>>>>         ext4:  380M         387M
>>>>>>>>>>>
>>>>>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to
>>>>>>>>>>> 1.4.
>>>>>>>>>>>
>>>>>>>>>>>     I have done some simple runtime testing on core-image-sato
>>>>>>>>>>> and
>>>>>>>>>>>     core-image-minimal, they worked well.
>>>>>>>>>>>
>>>>>>>>>>> // Robert
>>>>>>>>>>>
>>>>>>>>>>> The following changes since commit
>>>>>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
>>>>>>>>>>>
>>>>>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05
>>>>>>>>>>> +0100)
>>>>>>>>>>>
>>>>>>>>>>> are available in the git repository at:
>>>>>>>>>>>
>>>>>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
>>>>>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
>>>>>>>>>>>
>>>>>>>>>>> Robert Yang (4):
>>>>>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
>>>>>>>>>>>    e2fsprogs: let debugfs do sparse copy
>>>>>>>>>>>    e2fsprogs: add populate-extfs.sh
>>>>>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
>>>>>>>>>>>
>>>>>>>>>>> meta/classes/image_types.bbclass | 46 ++++----
>>>>>>>>>>> .../e2fsprogs-1.42.7/debugfs-too-short.patch | 28 +++++
>>>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh | 93
>>>>>>>>>>> ++++++++++++++++
>>>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch | 114
>>>>>>>>>>> ++++++++++++++++++++
>>>>>>>>>>> .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
>>>>>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
>>>>>>>>>>>   create mode 100644
>>>>>>>>>>>
>>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
>>>>>>>>>>>   create mode 100644
>>>>>>>>>>>
>>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
>>>>>>>>>>>   create mode 100644
>>>>>>>>>>>
>>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> 1.7.10.4
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Openembedded-core mailing list
>>>>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>>>>>
>>>>>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Jonathan
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>
>>
>>
>

Re: [PATCH 0/4] replace genext2fs with populate-extfs.sh

[Darren Hart]

On Fri, 2013-07-19 at 11:31 +1000, Jonathan Liu wrote:
> Hi Robert,
> 
> I tested your patches with my ext4 image on Intel Atom D2500HN board
> and it is working well.
> Previously I have been using custom image class which requires root
> privileges as I need to be able to specify custom options to mkfs.ext4
> which are not available with genext2fs and cannot be set with tune2fs
> afterwards. Using your patches I just put the extra options in
> EXTRA_IMAGECMD with ext4 image class and I no longer need root
> privileges for building image. =)

That's fantastic. Thank you for testing Jonathan. Congrats Robert, great
to see it improving things for people.

--
Darren

> 
> Thanks for your efforts.
> 
> Regards,
> Jonathan
> 
> On 19 July 2013 00:54, Robert Yang <liezhi.yang@windriver.com> wrote:
> >
> > Hi Jonathan,
> >
> > I've updated it a little, please make sure that you get the newest patches,
> > I've payed most of the time on e2fsprogs itself atm, so I didn't do enough
> > testing of the image (I will do it sooner)  thank you very much for focusing
> > on this.
> >
> > // Robert
> >
> >
> > On 07/18/2013 09:13 PM, Jonathan Liu wrote:
> >>
> >> On 18/07/2013 10:32 PM, Robert Yang wrote:
> >>>
> >>>
> >>> On 07/06/2013 10:49 PM, Jonathan Liu wrote:
> >>>>
> >>>> Any updates?
> >>>>
> >>>
> >>> Hi Jonathan,
> >>>
> >>> I've updated the patches, now they are working well, the "fsck -fn" finds
> >>> no errors any more, but I have to send the patches to the ext4 mailing
> >>> list
> >>> firstly, you can try it if you are interested in it atm.:
> >>>
> >>>   git://git.pokylinux.org/poky-contrib robert/image
> >>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/image
> >>>
> >>>
> >>> Robert Yang (5):
> >>>   e2fsprogs: the max length of debugfs argument is too short
> >>>   e2fsprogs: let debugfs do sparse copy
> >>>   e2fsprogs: only update the icache for ext2_inode
> >>>   e2fsprogs: add populate-extfs.sh
> >>>   image_types.bbclass: replace genext2fs with populate-extfs.sh
> >>>
> >>>  meta/classes/image_types.bbclass                   |  46 +++---
> >>>  .../e2fsprogs-1.42.8/debugfs-too-short.patch       |  28 ++++
> >>>  .../e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch    |  69 ++++++++
> >>>  .../e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh   |  96 +++++++++++
> >>>  .../e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch   | 177
> >>> +++++++++++++++++++++
> >>>  .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb |   5 +
> >>>  6 files changed, 393 insertions(+), 28 deletions(-)
> >>>  create mode 100644
> >>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/debugfs-too-short.patch
> >>>  create mode 100644
> >>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/fix-icache.patch
> >>>  create mode 100644
> >>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/populate-extfs.sh
> >>>  create mode 100644
> >>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/sparse_copy.patch
> >>>
> >>> // Robert
> >>
> >> Thanks, I will test it tomorrow. "yes | mkfs.$fstype" could probably be
> >> replaced
> >> with "mkfs.$fstype -F".
> >>
> >> Regards,
> >> Jonathan
> >>>
> >>>
> >>>> Regards,
> >>>> Jonathan
> >>>>>
> >>>>>
> >>>>>> meta: f7afeeb75993b159bb8959e0309bc5eb3978a8fb
> >>>>>>
> >>>>>> conf/local.conf:
> >>>>>> BB_NUMBER_THREADS = "8"
> >>>>>> PARALLEL_MAKE = "-j 8"
> >>>>>> MACHINE ??= "qemux86"
> >>>>>> DISTRO ?= "poky"
> >>>>>> PACKAGE_CLASSES = "package_ipk"
> >>>>>> EXTRA_IMAGE_FEATURES = "debug-tweaks"
> >>>>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
> >>>>>> PATCHRESOLVE = "noop"
> >>>>>> BB_DISKMON_DIRS = "\
> >>>>>>      STOPTASKS,${TMPDIR},1G,100K \
> >>>>>>      STOPTASKS,${DL_DIR},1G,100K \
> >>>>>>      STOPTASKS,${SSTATE_DIR},1G,100K \
> >>>>>>      ABORT,${TMPDIR},100M,1K \
> >>>>>>      ABORT,${DL_DIR},100M,1K \
> >>>>>>      ABORT,${SSTATE_DIR},100M,1K"
> >>>>>> CONF_VERSION = "1"
> >>>>>> DISTRO_FEATURES_append = " systemd"
> >>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
> >>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
> >>>>>> IMAGE_FSTYPES = "ext4"
> >>>>>>
> >>>>>> # bitbake core-image-minimal
> >>>>>> # fsck.ext4 -fn tmp/deploy/images/core-image-minimal-qemux86.ext4
> >>>>>>
> >>>>>> Regards,
> >>>>>> Jonathan
> >>>>>>>
> >>>>>>>
> >>>>>>>> distro conf:
> >>>>>>>> DISTRO_FEATURES_append = " largefile opengl systemd"
> >>>>>>>> DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
> >>>>>>>> VIRTUAL-RUNTIME_graphical_init_manager = ""
> >>>>>>>> VIRTUAL-RUNTIME_init_manager = "systemd"
> >>>>>>>> PACKAGE_CLASSES = "package_ipk"
> >>>>>>>>
> >>>>>>>> image bb:
> >>>>>>>> SYSLINUX_ROOT ?= "root=/dev/sda2"
> >>>>>>>> SYSLINUX_PROMPT = "0"
> >>>>>>>> SYSLINUX_TIMEOUT = "1"
> >>>>>>>> SYSLINUX_LABELS = "boot"
> >>>>>>>> LABELS_append = " ${SYSLINUX_LABELS} "
> >>>>>>>> do_bootdirectdisk[depends] += "${PN}:do_rootfs"
> >>>>>>>> APPEND += "rootfstype=ext4 rw"
> >>>>>>>> ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext4"
> >>>>>>>> inherit boot-directdisk
> >>>>>>>> IMAGE_FEATURES += "hwcodecs package-management ssh-server-openssh
> >>>>>>>> x11"
> >>>>>>>> IMAGE_FSTYPES = "ext4"
> >>>>>>>> IMAGE_LINGUAS = ""
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> My configuration:
> >>>>>>>>> MACHINE = "qemux86"
> >>>>>>>>> IMAGE_FSTYPES += "ext4"
> >>>>>>>>>
> >>>>>>>>> // Robert
> >>>>>>>>>
> >>>>>>>>> On 05/13/2013 12:59 PM, Jonathan Liu wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi Robert,
> >>>>>>>>>>
> >>>>>>>>>> I get the following errors doing filesystem check on the ext4
> >>>>>>>>>> image:
> >>>>>>>>>>
> >>>>>>>>>> # fsck.ext4 -fn tmp/deploy/images/custom-image-custom.ext4.ext4
> >>>>>>>>>> e2fsck 1.42.7 (21-Jan-2013)
> >>>>>>>>>> Pass 1: Checking inodes, blocks, and sizes
> >>>>>>>>>> Pass 2: Checking directory structure
> >>>>>>>>>> Entry 'mtd7' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd4' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'fb0' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd5' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty6' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'random' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd1' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'urandom' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'zero' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'null' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'kmsg' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'kmem' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd6' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty7' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty4' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'apm_bios' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty5' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd0' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty0' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'ttySA0' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mem' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd2' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty8' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'console' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'ttyS0' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should
> >>>>>>>>>> be 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty2' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'mtd3' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty1' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Entry 'tty3' in /dev (2739) has an incorrect filetype (was 4,
> >>>>>>>>>> should be
> >>>>>>>>>> 3).
> >>>>>>>>>> Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Pass 3: Checking directory connectivity
> >>>>>>>>>> Pass 4: Checking reference counts
> >>>>>>>>>> Inode 774 ref count is 9, should be 10.  Fix? no
> >>>>>>>>>>
> >>>>>>>>>> Pass 5: Checking group summary information
> >>>>>>>>>>
> >>>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: **********
> >>>>>>>>>> WARNING:
> >>>>>>>>>> Filesystem still has errors **********
> >>>>>>>>>>
> >>>>>>>>>> tmp/deploy/images/custom-image-custom.ext4.ext4: 3899/21336 files
> >>>>>>>>>> (0.5%
> >>>>>>>>>> non-contiguous), 131042/169656 blocks
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> After booting the image I get the following:
> >>>>>>>>>> # dmesg | grep EXT4-fs
> >>>>>>>>>> EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts:
> >>>>>>>>>> (null)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0,
> >>>>>>>>>> max 0(0),
> >>>>>>>>>> depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0,
> >>>>>>>>>> max 0(0),
> >>>>>>>>>> depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd: bad header/extent: invalid magic - magic 0, entries 0,
> >>>>>>>>>> max 0(0),
> >>>>>>>>>> depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-machine: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs (sda2): re-mounted. Opts: (null)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #883:
> >>>>>>>>>> comm
> >>>>>>>>>> systemd-journal: bad header/extent: invalid magic - magic 0,
> >>>>>>>>>> entries 0,
> >>>>>>>>>> max
> >>>>>>>>>> 0(0), depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810:
> >>>>>>>>>> comm
> >>>>>>>>>> login: bad header/extent: invalid magic - magic 0, entries 0, max
> >>>>>>>>>> 0(0),
> >>>>>>>>>> depth 0(0)
> >>>>>>>>>> EXT4-fs error (device sda2): ext4_ext_check_inode:467: inode #810:
> >>>>>>>>>> comm
> >>>>>>>>>> sshd: bad header/extent: invalid magic - magic 0, entries 0, max
> >>>>>>>>>> 0(0),
> >>>>>>>>>> depth 0(0)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> # systemctl --failed | cat
> >>>>>>>>>> UNIT                     LOAD   ACTIVE SUB DESCRIPTION
> >>>>>>>>>> machineid.service        loaded failed failed Machine ID first
> >>>>>>>>>> boot
> >>>>>>>>>> configure
> >>>>>>>>>> systemd-journald.service loaded failed failed Journal Service
> >>>>>>>>>> systemd-journald.socket  loaded failed failed Journal Socket
> >>>>>>>>>>
> >>>>>>>>>> LOAD   = Reflects whether the unit definition was properly loaded.
> >>>>>>>>>> ACTIVE = The high-level unit activation state, i.e. generalization
> >>>>>>>>>> of SUB.
> >>>>>>>>>> SUB    = The low-level unit activation state, values depend on
> >>>>>>>>>> unit type.
> >>>>>>>>>>
> >>>>>>>>>> 3 loaded units listed. Pass --all to see loaded but inactive
> >>>>>>>>>> units, too.
> >>>>>>>>>> To show all installed unit files use 'systemctl list-unit-files'.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I am booting kernel with rootfstype=ext4. IMAGE_FSTYPES = "ext4".
> >>>>>>>>>>
> >>>>>>>>>> Regards,
> >>>>>>>>>> Jonathan
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 7 May 2013 19:48, Robert Yang <liezhi.yang@windriver.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> * The benefits:
> >>>>>>>>>>>    - Really support ext4
> >>>>>>>>>>>    - Support the sparse file (the sparse file became into the
> >>>>>>>>>>> common file
> >>>>>>>>>>> before)
> >>>>>>>>>>>    - Have a uniform code for ext2/3/4 generation
> >>>>>>>>>>>    - Remove the depends on genext2fs-native
> >>>>>>>>>>>
> >>>>>>>>>>> * Impact
> >>>>>>>>>>>    - Build time:
> >>>>>>>>>>>      a) If we build fresh core-image-sato, there is nearly no
> >>>>>>>>>>> impact.
> >>>>>>>>>>>      b) If we do the image generation, which means:
> >>>>>>>>>>>         $ bitbake core-image-sato
> >>>>>>>>>>>         $ bitbake core-image-sato -ccleansstate
> >>>>>>>>>>>         $ bitbake core-image-sato
> >>>>>>>>>>>     About 50 extra seconds are needed, here is my test result:
> >>>>>>>>>>>     Before the patches: 4m25s
> >>>>>>>>>>>     After the patches: 5m17s
> >>>>>>>>>>>     This is because the genext2fs is much faster than the
> >>>>>>>>>>>     populate-extfs.sh, we will replace this script by the mke2fs
> >>>>>>>>>>> when it
> >>>>>>>>>>>     supports create the filesystem from a initial directory.
> >>>>>>>>>>>
> >>>>>>>>>>>    - Disk space (take core-image-sato as an example)
> >>>>>>>>>>>      a) The image file size is the same as before (529M)
> >>>>>>>>>>>      b) The disk usage is a little different: (du -sh)
> >>>>>>>>>>>                before       now
> >>>>>>>>>>>         ext2:  364M         388M
> >>>>>>>>>>>         ext3:  381M         404M
> >>>>>>>>>>>         ext4:  380M         387M
> >>>>>>>>>>>
> >>>>>>>>>>>     We may need to adjust the IMAGE_OVERHEAD_FACTOR from 1.3 to
> >>>>>>>>>>> 1.4.
> >>>>>>>>>>>
> >>>>>>>>>>>     I have done some simple runtime testing on core-image-sato
> >>>>>>>>>>> and
> >>>>>>>>>>>     core-image-minimal, they worked well.
> >>>>>>>>>>>
> >>>>>>>>>>> // Robert
> >>>>>>>>>>>
> >>>>>>>>>>> The following changes since commit
> >>>>>>>>>>> 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8:
> >>>>>>>>>>>
> >>>>>>>>>>>    grub-efi-native: Cleanup whitespace (2013-05-03 16:37:05
> >>>>>>>>>>> +0100)
> >>>>>>>>>>>
> >>>>>>>>>>> are available in the git repository at:
> >>>>>>>>>>>
> >>>>>>>>>>>    git://git.pokylinux.org/poky-contrib robert/ext4
> >>>>>>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ext4
> >>>>>>>>>>>
> >>>>>>>>>>> Robert Yang (4):
> >>>>>>>>>>>    e2fsprogs: the max length of debugfs argument is too short
> >>>>>>>>>>>    e2fsprogs: let debugfs do sparse copy
> >>>>>>>>>>>    e2fsprogs: add populate-extfs.sh
> >>>>>>>>>>>    image_types.bbclass: replace genext2fs with populate-extfs.sh
> >>>>>>>>>>>
> >>>>>>>>>>> meta/classes/image_types.bbclass | 46 ++++----
> >>>>>>>>>>> .../e2fsprogs-1.42.7/debugfs-too-short.patch | 28 +++++
> >>>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh | 93
> >>>>>>>>>>> ++++++++++++++++
> >>>>>>>>>>> .../e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch | 114
> >>>>>>>>>>> ++++++++++++++++++++
> >>>>>>>>>>> .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.7.bb | 4 +
> >>>>>>>>>>>   5 files changed, 257 insertions(+), 28 deletions(-)
> >>>>>>>>>>>   create mode 100644
> >>>>>>>>>>>
> >>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/debugfs-too-short.patch
> >>>>>>>>>>>   create mode 100644
> >>>>>>>>>>>
> >>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/populate-extfs.sh
> >>>>>>>>>>>   create mode 100644
> >>>>>>>>>>>
> >>>>>>>>>>> meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.7/sparse_copy.patch
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> 1.7.10.4
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> _______________________________________________
> >>>>>>>>>>> Openembedded-core mailing list
> >>>>>>>>>>> Openembedded-core@lists.openembedded.org
> >>>>>>>>>>>
> >>>>>>>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Jonathan
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>
> >>
> >>
> >

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel