All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/6] Setup for VMDK to use Direct Disk
@ 2012-03-27  5:42 Saul Wold
  2012-03-27  5:42 ` [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source Saul Wold
                   ` (7 more replies)
  0 siblings, 8 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:42 UTC (permalink / raw)
  To: openembedded-core

Updated comments per Darren's request, added cleanup to
image-types to only use one -i (inode-count) parameter.

Sau!
The following changes since commit 644b7503c37fd73730dd3d7841463b158b8934ed:

  guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)

are available in the git repository at:
  git://git.openembedded.org/openembedded-core-contrib sgw/self
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=sgw/self

Dexuan Cui (1):
  self-hosted-image: pre-populate the builder user with poky source

Saul Wold (5):
  image-vmdk: Add symbolic link for a short named vmdk image
  boot-directdisk: Fix Block Calcuation
  vmdk: Update for direstdisk
  builder: Enable auto starting of Hob
  image_types: remove duplicate setting of -i for inode count

 meta/classes/boot-directdisk.bbclass               |   19 ++++++---
 meta/classes/image-vmdk.bbclass                    |   17 +++++++-
 meta/classes/image_types.bbclass                   |    5 +-
 meta/recipes-core/images/self-hosted-image.bb      |   41 +++++++++++++++++++-
 meta/recipes-graphics/builder/builder_0.1.bb       |    2 +-
 .../builder/files/builder_hob_start.sh             |   10 +++-
 6 files changed, 77 insertions(+), 17 deletions(-)

-- 
1.7.7.6




^ permalink raw reply	[flat|nested] 22+ messages in thread
* [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
@ 2012-03-27  5:42 ` Saul Wold
  2012-03-28 15:35   ` Cui, Dexuan
  2012-03-28 20:45   ` Paul Eggleton
  2012-03-27  5:42 ` [PATCH 2/6] image-vmdk: Add symbolic link for a short named vmdk image Saul Wold
                   ` (6 subsequent siblings)
  7 siblings, 2 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:42 UTC (permalink / raw)
  To: openembedded-core

From: Dexuan Cui <dexuan.cui@intel.com>

This patch installs the poky source into the /home/builder/poky/ of the
self-hosted-image.
This makes the user of self-hosted-image easier to start a build.

I think the recent poky master is stable enough, so I specify
a commit number by SRCREV -- we may want to update this number before
releasing 1.2.

This patch fixes [YOCTO #2065]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>

Added code for supporting target based pseudo
fixed indentation

Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
 meta/recipes-core/images/self-hosted-image.bb |   41 +++++++++++++++++++++++-
 1 files changed, 39 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/images/self-hosted-image.bb b/meta/recipes-core/images/self-hosted-image.bb
index d56c2cb..5aa670d 100644
--- a/meta/recipes-core/images/self-hosted-image.bb
+++ b/meta/recipes-core/images/self-hosted-image.bb
@@ -4,7 +4,7 @@ LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
                     file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
 
-PR = "r5"
+PR = "r6"
 
 CORE_IMAGE_EXTRA_INSTALL = "\
     task-self-hosted \
@@ -13,7 +13,10 @@ CORE_IMAGE_EXTRA_INSTALL = "\
 IMAGE_FEATURES += "x11-mini package-management"
 
 # Ensure there's enough space to do a core-image-minimal build, with rm_work enabled
-IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
+IMAGE_ROOTFS_EXTRA_SPACE = "1048576"
+#IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
+#IMAGE_ROOTFS_EXTRA_SPACE = "20971520"
+#IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
 
 # Do a quiet boot with limited console messages
 APPEND += "quiet"
@@ -21,3 +24,37 @@ APPEND += "quiet"
 IMAGE_FSTYPES = "vmdk"
 
 inherit core-image
+
+SRCREV = "26a46938d3ea1821e7bec4fa6cc8379babad238b"
+SRC_URI = "git://git.yoctoproject.org/poky;protocol=git"
+
+fakeroot do_populate_poky_src () {
+	# Because fetch2's git's unpack uses -s cloneflag, the unpacked git repo
+	# will become invalid in the target.
+	rm -rf ${WORKDIR}/git/.git
+	rm -f ${WORKDIR}/git/.gitignore
+
+	cp -Rp ${WORKDIR}/git ${IMAGE_ROOTFS}/home/builder/poky
+
+	mkdir -p ${IMAGE_ROOTFS}/home/builder/poky/build/conf
+	cp -Rp ${DL_DIR} ${IMAGE_ROOTFS}/home/builder/poky/build
+	echo "/usr/bin" > ${IMAGE_ROOTFS}/home/builder/poky/build/pseudodone
+	echo "BB_NO_NETWORK = \"1\"" > ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf
+	echo "INHERIT += \"rm_work\"" >> ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf
+        mkdir -p ${IMAGE_ROOTFS}/home/builder/pseudo
+        echo "export PSEUDO_PREFIX=/usr" >> ${IMAGE_ROOTFS}/home/builder/.bashrc
+        echo "export PSEUDO_LOCALSTATEDIR=/home/builder/pseudo" >> ${IMAGE_ROOTFS}/home/builder/.bashrc
+        echo "export PSEUDO_LIBDIR=/usr/lib/pseudo/lib64" >> ${IMAGE_ROOTFS}/home/builder/.bashrc
+
+        chown builder.builder ${IMAGE_ROOTFS}/home/builder/pseudo
+
+	chown -R builder.builder  ${IMAGE_ROOTFS}/home/builder/poky
+}
+
+IMAGE_PREPROCESS_COMMAND += "do_populate_poky_src; "
+
+python do_get_poky_src () {
+    bb.build.exec_func('base_do_fetch', d)
+    bb.build.exec_func('base_do_unpack', d)
+}
+addtask do_get_poky_src before do_rootfs
-- 
1.7.7.6




^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH 2/6] image-vmdk: Add symbolic link for a short named vmdk image
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
  2012-03-27  5:42 ` [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source Saul Wold
@ 2012-03-27  5:42 ` Saul Wold
  2012-03-27  5:42 ` [PATCH 3/6] boot-directdisk: Fix Block Calcuation Saul Wold
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:42 UTC (permalink / raw)
  To: openembedded-core

This adds a link from the date stamped vmdk image to a shortened
name file.

Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
 meta/classes/image-vmdk.bbclass |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/meta/classes/image-vmdk.bbclass b/meta/classes/image-vmdk.bbclass
index 3e63006..9d596cd 100644
--- a/meta/classes/image-vmdk.bbclass
+++ b/meta/classes/image-vmdk.bbclass
@@ -10,6 +10,8 @@ inherit image-live
 
 create_vmdk_image () {
 	qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk
+	ln -s ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk
+
 }
 
 python do_vmdkimg() {
-- 
1.7.7.6




^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH 3/6] boot-directdisk: Fix Block Calcuation
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
  2012-03-27  5:42 ` [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source Saul Wold
  2012-03-27  5:42 ` [PATCH 2/6] image-vmdk: Add symbolic link for a short named vmdk image Saul Wold
@ 2012-03-27  5:42 ` Saul Wold
  2012-03-27  5:42 ` [PATCH 4/6] vmdk: Update for direstdisk Saul Wold
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:42 UTC (permalink / raw)
  To: openembedded-core; +Cc: Darren Hart

This also changes the timeout to be settable

The block calcuation was not correctly rounding, see comment

Thanks to Darren Hart for fixing this.

Cc: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
 meta/classes/boot-directdisk.bbclass |   19 +++++++++++++------
 1 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/meta/classes/boot-directdisk.bbclass b/meta/classes/boot-directdisk.bbclass
index 893164f..e9e1ba3 100644
--- a/meta/classes/boot-directdisk.bbclass
+++ b/meta/classes/boot-directdisk.bbclass
@@ -37,9 +37,8 @@ BOOTDD_EXTRA_SPACE ?= "16384"
 # Get the build_syslinux_cfg() function from the syslinux class
 
 AUTO_SYSLINUXCFG = "1"
-LABELS = "boot"
 SYSLINUX_ROOT ?= "root=/dev/sda2"
-SYSLINUX_TIMEOUT = "10" # 1 second
+SYSLINUX_TIMEOUT ?= "10"
 
 SYSLINUXCFG  = "${HDDDIR}/syslinux.cfg"
 SYSLINUXMENU = "${HDDDIR}/menu"
@@ -55,15 +54,23 @@ build_boot_dd() {
 	install -m 444 ${STAGING_LIBDIR}/syslinux/ldlinux.sys ${HDDDIR}/ldlinux.sys
 
 	BLOCKS=`du -bks ${HDDDIR} | cut -f 1`
-	SIZE=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}`
+	BLOCKS=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}`
+
+	# Ensure total sectors is an integral number of sectors per
+	# track or mcopy will complain. Sectors are 512 bytes, and we
+	# generate images with 32 sectors per track. This calculation is
+	# done in blocks, thus the mod by 16 instead of 32.
+	BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16)))
+
+	mkdosfs -n ${BOOTDD_VOLUME_ID} -S 512 -C ${HDDIMG} $BLOCKS 
+	mcopy -i ${HDDIMG} -s ${HDDDIR}/* ::/
 
-	mkdosfs -n ${BOOTDD_VOLUME_ID} -d ${HDDDIR} -C ${HDDIMG} $SIZE 
 	syslinux ${HDDIMG}
 	chmod 644 ${HDDIMG}
 
 	ROOTFSBLOCKS=`du -Lbks ${ROOTFS} | cut -f 1`
-	TOTALSIZE=`expr $SIZE + $ROOTFSBLOCKS`
-	END1=`expr $SIZE \* 1024`
+	TOTALSIZE=`expr $BLOCKS + $ROOTFSBLOCKS`
+	END1=`expr $BLOCKS \* 1024`
 	END2=`expr $END1 + 512`
 	END3=`expr \( $ROOTFSBLOCKS \* 1024 \) + $END1`
 
-- 
1.7.7.6




^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH 4/6] vmdk: Update for direstdisk
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
                   ` (2 preceding siblings ...)
  2012-03-27  5:42 ` [PATCH 3/6] boot-directdisk: Fix Block Calcuation Saul Wold
@ 2012-03-27  5:42 ` Saul Wold
  2012-03-27  5:42 ` [PATCH 5/6] builder: Enable auto starting of Hob Saul Wold
                   ` (3 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:42 UTC (permalink / raw)
  To: openembedded-core

The conversion to directdisk is necessary due to the limitation
in the mkdosfs version we carry and larger filesystems.

Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
 meta/classes/image-vmdk.bbclass |   15 ++++++++++++---
 1 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/meta/classes/image-vmdk.bbclass b/meta/classes/image-vmdk.bbclass
index 9d596cd..736d1d7 100644
--- a/meta/classes/image-vmdk.bbclass
+++ b/meta/classes/image-vmdk.bbclass
@@ -1,15 +1,23 @@
 
 NOISO = "1"
+
+SYSLINUX_ROOT = "root=/dev/hda2 "
 SYSLINUX_PROMPT = "0"
 SYSLINUX_TIMEOUT = "1"
 SYSLINUX_LABELS = "boot"
+LABELS_append = " ${SYSLINUX_LABELS} "
+
+# need to define the dependency and the ROOTFS for directdisk
+do_bootdirectdisk[depends] += "${IMAGE_BASENAME}:do_rootfs"
+ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext3"
 
 # creating VMDK relies on having a live hddimg so ensure we
 # inherit it here.
-inherit image-live
+#inherit image-live
+inherit boot-directdisk
 
 create_vmdk_image () {
-	qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk
+	qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hdddirect ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk
 	ln -s ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk
 
 }
@@ -18,7 +26,8 @@ python do_vmdkimg() {
         bb.build.exec_func('create_vmdk_image', d)
 }
 
-addtask vmdkimg after do_bootimg before do_build
+#addtask vmdkimg after do_bootimg before do_build
+addtask vmdkimg after do_bootdirectdisk before do_build
 do_vmdkimg[nostamp] = "1"
 
 do_vmdkimg[depends] += "qemu-native:do_populate_sysroot" 
-- 
1.7.7.6




^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH 5/6] builder: Enable auto starting of Hob
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
                   ` (3 preceding siblings ...)
  2012-03-27  5:42 ` [PATCH 4/6] vmdk: Update for direstdisk Saul Wold
@ 2012-03-27  5:42 ` Saul Wold
  2012-03-27  5:43 ` [PATCH 6/6] image_types: remove duplicate setting of -i for inode count Saul Wold
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:42 UTC (permalink / raw)
  To: openembedded-core

Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
 meta/recipes-graphics/builder/builder_0.1.bb       |    2 +-
 .../builder/files/builder_hob_start.sh             |   10 +++++++---
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-graphics/builder/builder_0.1.bb b/meta/recipes-graphics/builder/builder_0.1.bb
index ceb9612..1324e5b 100644
--- a/meta/recipes-graphics/builder/builder_0.1.bb
+++ b/meta/recipes-graphics/builder/builder_0.1.bb
@@ -1,7 +1,7 @@
 SUMMARY = "New user to do specific job"
 DESCRIPTION = "This recipe create a new user named ${PN}, who is used for specific jobs like building. The task can be auto started via mini X"
 SECTION = "x11"
-PR = "r0"
+PR = "r1"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://builder_hob_start.sh;endline=5;md5=84796c3c41785d86100fdabcbdade00e"
 
diff --git a/meta/recipes-graphics/builder/files/builder_hob_start.sh b/meta/recipes-graphics/builder/files/builder_hob_start.sh
index fdafc4e..f43e6c5 100644
--- a/meta/recipes-graphics/builder/files/builder_hob_start.sh
+++ b/meta/recipes-graphics/builder/files/builder_hob_start.sh
@@ -4,8 +4,12 @@
 #be put here
 
 # start hob here
-#cd /intel/poky/poky
-#. ./oe-init-build-env
-#../scripts/hob
+export PSEUDO_PREFIX=/usr
+export PSEUDO_LOCALSTATEDIR=/home/builder/pseudo
+export PSEUDO_LIBDIR=/usr/lib/pseudo/lib64
+
+cd /home/builder/poky
+. ./oe-init-build-env
+hob &
 
 matchbox-terminal&
-- 
1.7.7.6




^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [PATCH 6/6] image_types: remove duplicate setting of -i for inode count
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
                   ` (4 preceding siblings ...)
  2012-03-27  5:42 ` [PATCH 5/6] builder: Enable auto starting of Hob Saul Wold
@ 2012-03-27  5:43 ` Saul Wold
  2012-03-27 14:37 ` [PATCH 0/6] Setup for VMDK to use Direct Disk Paul Eggleton
  2012-03-30 14:17 ` Paul Eggleton
  7 siblings, 0 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-27  5:43 UTC (permalink / raw)
  To: openembedded-core

inode count is already set in the EXTRA_IMAGECMD definition

Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
 meta/classes/image_types.bbclass |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 3aa3f17..b590ae1 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -133,12 +133,12 @@ IMAGE_CMD_ext2 () {
 }
 
 IMAGE_CMD_ext3 () {
-	genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
 	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
 }
 
 oe_mkext4fs () {
-	genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1
 	tune2fs -O extents,uninit_bg,dir_index,has_journal $1
 	e2fsck -yfDC0 $1 || chk=$?
 	case $chk in
@@ -192,6 +192,7 @@ EXTRA_IMAGECMD_jffs2 ?= "--pad --little-endian --eraseblock=0x40000"
 # Change these if you want default genext2fs behavior (i.e. create minimal inode number)
 EXTRA_IMAGECMD_ext2 ?= "-i 8192"
 EXTRA_IMAGECMD_ext3 ?= "-i 8192"
+EXTRA_IMAGECMD_ext4 ?= "-i 8192"
 EXTRA_IMAGECMD_btrfs ?= ""
 
 IMAGE_DEPENDS = ""
-- 
1.7.7.6




^ permalink raw reply related	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
                   ` (5 preceding siblings ...)
  2012-03-27  5:43 ` [PATCH 6/6] image_types: remove duplicate setting of -i for inode count Saul Wold
@ 2012-03-27 14:37 ` Paul Eggleton
  2012-03-27 18:16   ` Saul Wold
  2012-03-30 14:17 ` Paul Eggleton
  7 siblings, 1 reply; 22+ messages in thread
From: Paul Eggleton @ 2012-03-27 14:37 UTC (permalink / raw)
  To: openembedded-core

On Monday 26 March 2012 22:42:54 Saul Wold wrote:
> Updated comments per Darren's request, added cleanup to
> image-types to only use one -i (inode-count) parameter.

I tried this series on my build machine, but do_rootfs failed with "genext2fs: 
not enough memory for filesystem".

The machine has 6GB RAM and 8GB swap. The rootfs directory is 15GB. 

I guess this might have something to do with my DL_DIR not being clean 
beforehand (since it seems to be copying it into the rootfs in its entirety).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-27 14:37 ` [PATCH 0/6] Setup for VMDK to use Direct Disk Paul Eggleton
@ 2012-03-27 18:16   ` Saul Wold
  2012-03-27 20:21     ` Paul Eggleton
  0 siblings, 1 reply; 22+ messages in thread
From: Saul Wold @ 2012-03-27 18:16 UTC (permalink / raw)
  To: Paul Eggleton; +Cc: openembedded-core

On 03/27/2012 07:37 AM, Paul Eggleton wrote:
> On Monday 26 March 2012 22:42:54 Saul Wold wrote:
>> Updated comments per Darren's request, added cleanup to
>> image-types to only use one -i (inode-count) parameter.
>
> I tried this series on my build machine, but do_rootfs failed with "genext2fs:
> not enough memory for filesystem".

Yes, there is an issue with genext2fs needing memory to build the 
filesystem first.  I noted in another email thread that the Debian build 
system had so resort to dd, mkfs.ext3, mounting and copying. The issue 
with that is that it requires root privileges which is forbidden with 
bitbake.

genext2fs combines the making and copying of the fs into one process, if 
there is a way to populate the filesystem image with the bits without 
root privs that's what we need to look towards.

This is turning into a non-trivial problem.

I am beginning to think that we might have to have 2 vmdk images for the 
build appliance to work correctly. 1 for the rootfs and the second for 
the build space (10G-20G) with DL_DIR populated.

>
> The machine has 6GB RAM and 8GB swap. The rootfs directory is 15GB.
>
> I guess this might have something to do with my DL_DIR not being clean
> beforehand (since it seems to be copying it into the rootfs in its entirety).
>
I have created a clean DL_DIR to work with by doing a world fetch and 
then using that DL_DIR to build into the image.

> Cheers,
> Paul
>



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-27 18:16   ` Saul Wold
@ 2012-03-27 20:21     ` Paul Eggleton
  2012-03-27 20:45       ` João Henrique Freitas
  0 siblings, 1 reply; 22+ messages in thread
From: Paul Eggleton @ 2012-03-27 20:21 UTC (permalink / raw)
  To: Saul Wold; +Cc: openembedded-core

On Tuesday 27 March 2012 11:16:10 Saul Wold wrote:
> Yes, there is an issue with genext2fs needing memory to build the
> filesystem first.  I noted in another email thread that the Debian build
> system had so resort to dd, mkfs.ext3, mounting and copying. The issue
> with that is that it requires root privileges which is forbidden with
> bitbake.
> 
> genext2fs combines the making and copying of the fs into one process, if
> there is a way to populate the filesystem image with the bits without
> root privs that's what we need to look towards.

Yes, the mount alternative won't work for us. virt-make-fs from the libguestfs 
tools might be a possibility but its large set of dependencies could be an 
issue unless we can pare them down somehow.

> This is turning into a non-trivial problem.

Indeed :/

> I am beginning to think that we might have to have 2 vmdk images for the
> build appliance to work correctly. 1 for the rootfs and the second for
> the build space (10G-20G) with DL_DIR populated.

That might mitigate the problem, yes. If genext2fs requires the same amount of 
RAM as the target filesystem then maybe we need to do an available  RAM check 
first and bail out if there isn't enough.

> > I guess this might have something to do with my DL_DIR not being clean
> > beforehand (since it seems to be copying it into the rootfs in its
> > entirety).
>
> I have created a clean DL_DIR to work with by doing a world fetch and
> then using that DL_DIR to build into the image.

I guess we can just put that in the docs for now. For future improvement 
though I wonder if we can trick the fetcher into just pulling across the 
current files into the rootfs. I'll put that on my todo list for later.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-27 20:21     ` Paul Eggleton
@ 2012-03-27 20:45       ` João Henrique Freitas
  2012-03-28 15:21         ` Cui, Dexuan
  0 siblings, 1 reply; 22+ messages in thread
From: João Henrique Freitas @ 2012-03-27 20:45 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 604 bytes --]

Hi,

I guess we can just put that in the docs for now. For future improvement
> though I wonder if we can trick the fetcher into just pulling across the
> current files into the rootfs. I'll put that on my todo list for later.
>


The following thread suggest a patch to fix it:

http://sourceforge.net/mailarchive/forum.php?thread_name=1307458172.10974.64.camel%40skunk&forum_name=genext2fs-devel

Thanks.

-- 
-----------------------------------------------------------
João Henrique Freitas - joaohf_at_gmail.com
Campinas-SP-Brasil
BSD051283
LPI 1
http://www.joaohfreitas.eti.br

[-- Attachment #2: Type: text/html, Size: 1042 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-27 20:45       ` João Henrique Freitas
@ 2012-03-28 15:21         ` Cui, Dexuan
  0 siblings, 0 replies; 22+ messages in thread
From: Cui, Dexuan @ 2012-03-28 15:21 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

Hi joaohf,

Thank for the hint!

I have got the patches from genext2fs-devel mailing list and did some tests:

I can successfully create a .ext2 file of 8.5GB with genext2fs now!



I’ll send an update to the genext2fs recipe later.



Thanks,

-- Dexuan

From: openembedded-core-bounces@lists.openembedded.org [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Jo?o Henrique Freitas
Sent: Wednesday, March 28, 2012 4:46 AM
To: Patches and discussions about the oe-core layer
Subject: Re: [OE-core] [PATCH 0/6] Setup for VMDK to use Direct Disk


Hi,
I guess we can just put that in the docs for now. For future improvement
though I wonder if we can trick the fetcher into just pulling across the
current files into the rootfs. I'll put that on my todo list for later.


The following thread suggest a patch to fix it:

http://sourceforge.net/mailarchive/forum.php?thread_name=1307458172.10974.64.camel%40skunk&forum_name=genext2fs-devel

Thanks.

--
-----------------------------------------------------------
João Henrique Freitas - joaohf_at_gmail.com<http://joaohf_at_gmail.com>
Campinas-SP-Brasil
BSD051283
LPI 1
http://www.joaohfreitas.eti.br

[-- Attachment #2: Type: text/html, Size: 5400 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source
  2012-03-27  5:42 ` [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source Saul Wold
@ 2012-03-28 15:35   ` Cui, Dexuan
  2012-03-28 21:07     ` Saul Wold
  2012-03-28 20:45   ` Paul Eggleton
  1 sibling, 1 reply; 22+ messages in thread
From: Cui, Dexuan @ 2012-03-28 15:35 UTC (permalink / raw)
  To: Saul Wold, openembedded-core

Hi Saul,
Did you test "bitbake core-image-minimal" inside the vmware guest?
I got the following ERROR immediately:

------------------------
ERROR:  OE-core's config sanity checker detected a potential misconfiguration.
    Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
    Following is the list of potential problems / advisories:

    Pseudo is not functioning correctly, which will cause failures during package installation. Please check your configuration.

ERROR: Execution of event handler 'check_sanity_eventhandler' failed
------------------------

I suspect in the guest, pseudo is not setup properly?

Thanks,
-- Dexuan

> -----Original Message-----
> From: Saul Wold [mailto:sgw@linux.intel.com]
> Sent: Tuesday, March 27, 2012 1:43 PM
> To: openembedded-core@lists.openembedded.org
> Cc: Cui, Dexuan
> Subject: [PATCH 1/6] self-hosted-image: pre-populate the builder user with
> poky source
> 
> From: Dexuan Cui <dexuan.cui@intel.com>
> 
> This patch installs the poky source into the /home/builder/poky/ of the
> self-hosted-image.
> This makes the user of self-hosted-image easier to start a build.
> 
> I think the recent poky master is stable enough, so I specify a commit number
> by SRCREV -- we may want to update this number before releasing 1.2.
> 
> This patch fixes [YOCTO #2065]
> 
> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
> 
> Added code for supporting target based pseudo fixed indentation
> 
> Signed-off-by: Saul Wold <sgw@linux.intel.com>
> ---
>  meta/recipes-core/images/self-hosted-image.bb |   41
> +++++++++++++++++++++++-
>  1 files changed, 39 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-core/images/self-hosted-image.bb
> b/meta/recipes-core/images/self-hosted-image.bb
> index d56c2cb..5aa670d 100644
> --- a/meta/recipes-core/images/self-hosted-image.bb
> +++ b/meta/recipes-core/images/self-hosted-image.bb
> @@ -4,7 +4,7 @@ LICENSE = "MIT"
>  LIC_FILES_CHKSUM =
> "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
> 
> file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de
> 20420"
> 
> -PR = "r5"
> +PR = "r6"
> 
>  CORE_IMAGE_EXTRA_INSTALL = "\
>      task-self-hosted \
> @@ -13,7 +13,10 @@ CORE_IMAGE_EXTRA_INSTALL = "\  IMAGE_FEATURES
> += "x11-mini package-management"
> 
>  # Ensure there's enough space to do a core-image-minimal build, with
> rm_work enabled -IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
> +IMAGE_ROOTFS_EXTRA_SPACE = "1048576"
> +#IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
> +#IMAGE_ROOTFS_EXTRA_SPACE = "20971520"
> +#IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
> 
>  # Do a quiet boot with limited console messages  APPEND += "quiet"
> @@ -21,3 +24,37 @@ APPEND += "quiet"
>  IMAGE_FSTYPES = "vmdk"
> 
>  inherit core-image
> +
> +SRCREV = "26a46938d3ea1821e7bec4fa6cc8379babad238b"
> +SRC_URI = "git://git.yoctoproject.org/poky;protocol=git"
> +
> +fakeroot do_populate_poky_src () {
> +	# Because fetch2's git's unpack uses -s cloneflag, the unpacked git repo
> +	# will become invalid in the target.
> +	rm -rf ${WORKDIR}/git/.git
> +	rm -f ${WORKDIR}/git/.gitignore
> +
> +	cp -Rp ${WORKDIR}/git ${IMAGE_ROOTFS}/home/builder/poky
> +
> +	mkdir -p ${IMAGE_ROOTFS}/home/builder/poky/build/conf
> +	cp -Rp ${DL_DIR} ${IMAGE_ROOTFS}/home/builder/poky/build
> +	echo "/usr/bin" >
> ${IMAGE_ROOTFS}/home/builder/poky/build/pseudodone
> +	echo "BB_NO_NETWORK = \"1\"" >
> ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf
> +	echo "INHERIT += \"rm_work\"" >>
> ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf
> +        mkdir -p ${IMAGE_ROOTFS}/home/builder/pseudo
> +        echo "export PSEUDO_PREFIX=/usr" >>
> ${IMAGE_ROOTFS}/home/builder/.bashrc
> +        echo "export PSEUDO_LOCALSTATEDIR=/home/builder/pseudo" >>
> ${IMAGE_ROOTFS}/home/builder/.bashrc
> +        echo "export PSEUDO_LIBDIR=/usr/lib/pseudo/lib64" >>
> +${IMAGE_ROOTFS}/home/builder/.bashrc
> +
> +        chown builder.builder ${IMAGE_ROOTFS}/home/builder/pseudo
> +
> +	chown -R builder.builder  ${IMAGE_ROOTFS}/home/builder/poky }
> +
> +IMAGE_PREPROCESS_COMMAND += "do_populate_poky_src; "
> +
> +python do_get_poky_src () {
> +    bb.build.exec_func('base_do_fetch', d)
> +    bb.build.exec_func('base_do_unpack', d) } addtask do_get_poky_src
> +before do_rootfs
> --
> 1.7.7.6




^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source
  2012-03-27  5:42 ` [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source Saul Wold
  2012-03-28 15:35   ` Cui, Dexuan
@ 2012-03-28 20:45   ` Paul Eggleton
  2012-03-28 21:08     ` Saul Wold
  1 sibling, 1 reply; 22+ messages in thread
From: Paul Eggleton @ 2012-03-28 20:45 UTC (permalink / raw)
  To: Dexuan Cui, Saul Wold, openembedded-core

On Monday 26 March 2012 22:42:55 Saul Wold wrote:
> From: Dexuan Cui <dexuan.cui@intel.com>
> 
> This patch installs the poky source into the /home/builder/poky/ of the
> self-hosted-image.
> This makes the user of self-hosted-image easier to start a build.
> 
> I think the recent poky master is stable enough, so I specify
> a commit number by SRCREV -- we may want to update this number before
> releasing 1.2.
> 
> This patch fixes [YOCTO #2065]
> 
> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
> 
> Added code for supporting target based pseudo
> fixed indentation
> 
> Signed-off-by: Saul Wold <sgw@linux.intel.com>
> ---
>  meta/recipes-core/images/self-hosted-image.bb |   41
> +++++++++++++++++++++++- 1 files changed, 39 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-core/images/self-hosted-image.bb
> b/meta/recipes-core/images/self-hosted-image.bb index d56c2cb..5aa670d
> 100644
> --- a/meta/recipes-core/images/self-hosted-image.bb
> +++ b/meta/recipes-core/images/self-hosted-image.bb
> @@ -4,7 +4,7 @@ LICENSE = "MIT"
>  LIC_FILES_CHKSUM =
> "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
> file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
> 
> -PR = "r5"
> +PR = "r6"
> 
>  CORE_IMAGE_EXTRA_INSTALL = "\
>      task-self-hosted \
> @@ -13,7 +13,10 @@ CORE_IMAGE_EXTRA_INSTALL = "\
>  IMAGE_FEATURES += "x11-mini package-management"
> 
>  # Ensure there's enough space to do a core-image-minimal build, with
> rm_work enabled -IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
> +IMAGE_ROOTFS_EXTRA_SPACE = "1048576"
> +#IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
> +#IMAGE_ROOTFS_EXTRA_SPACE = "20971520"
> +#IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
> 
>  # Do a quiet boot with limited console messages
>  APPEND += "quiet"
> @@ -21,3 +24,37 @@ APPEND += "quiet"
>  IMAGE_FSTYPES = "vmdk"
> 
>  inherit core-image
> +
> +SRCREV = "26a46938d3ea1821e7bec4fa6cc8379babad238b"
> +SRC_URI = "git://git.yoctoproject.org/poky;protocol=git"
> +
> +fakeroot do_populate_poky_src () {
> +	# Because fetch2's git's unpack uses -s cloneflag, the unpacked git repo
> +	# will become invalid in the target.
> +	rm -rf ${WORKDIR}/git/.git
> +	rm -f ${WORKDIR}/git/.gitignore
> +
> +	cp -Rp ${WORKDIR}/git ${IMAGE_ROOTFS}/home/builder/poky
> +
> +	mkdir -p ${IMAGE_ROOTFS}/home/builder/poky/build/conf
> +	cp -Rp ${DL_DIR} ${IMAGE_ROOTFS}/home/builder/poky/build

Could we change this last line to:

mkdir ${IMAGE_ROOTFS}/home/builder/poky/build/downloads
cp -RpL  ${DL_DIR}/* ${IMAGE_ROOTFS}/home/builder/poky/build/downloads/

This does two things:

1) Handles if DL_DIR on the build machine is not called "downloads" (which by 
chance mine was not when I tested it)

2) I notice if you set up a separate downloads directory and then use own-
mirrors to fetch from your original downloads directory (possibly unorthodox, 
but it gets you a clean downloads directory without redownloading everything) 
you get symlinks into the original downloads dir rather than copied files. I 
don't know how many people will hit this but since we never want any symlinks 
in this directory in the self-hosted image, I think using -L is worth doing 
here.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source
  2012-03-28 15:35   ` Cui, Dexuan
@ 2012-03-28 21:07     ` Saul Wold
  2012-03-29  7:15       ` Cui, Dexuan
  0 siblings, 1 reply; 22+ messages in thread
From: Saul Wold @ 2012-03-28 21:07 UTC (permalink / raw)
  To: Cui, Dexuan; +Cc: openembedded-core

On 03/28/2012 08:35 AM, Cui, Dexuan wrote:
> Hi Saul,
> Did you test "bitbake core-image-minimal" inside the vmware guest?
> I got the following ERROR immediately:
>
This should be addressed by the 5/6 patch that adds the correct PSEUDO_* 
setup into the minix session script.

I guess that you tried to run this on the command line and as you might 
notice I modified the bashrc, but for some reason that did not get 
picked up when the shell started up, I think we need to force the 
builder user to get /bin/bash as a shell not /bin/sh

Sau!

> ------------------------
> ERROR:  OE-core's config sanity checker detected a potential misconfiguration.
>      Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
>      Following is the list of potential problems / advisories:
>
>      Pseudo is not functioning correctly, which will cause failures during package installation. Please check your configuration.
>
> ERROR: Execution of event handler 'check_sanity_eventhandler' failed
> ------------------------
>
> I suspect in the guest, pseudo is not setup properly?
>
> Thanks,
> -- Dexuan
>
>> -----Original Message-----
>> From: Saul Wold [mailto:sgw@linux.intel.com]
>> Sent: Tuesday, March 27, 2012 1:43 PM
>> To: openembedded-core@lists.openembedded.org
>> Cc: Cui, Dexuan
>> Subject: [PATCH 1/6] self-hosted-image: pre-populate the builder user with
>> poky source
>>
>> From: Dexuan Cui<dexuan.cui@intel.com>
>>
>> This patch installs the poky source into the /home/builder/poky/ of the
>> self-hosted-image.
>> This makes the user of self-hosted-image easier to start a build.
>>
>> I think the recent poky master is stable enough, so I specify a commit number
>> by SRCREV -- we may want to update this number before releasing 1.2.
>>
>> This patch fixes [YOCTO #2065]
>>
>> Signed-off-by: Dexuan Cui<dexuan.cui@intel.com>
>>
>> Added code for supporting target based pseudo fixed indentation
>>
>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>> ---
>>   meta/recipes-core/images/self-hosted-image.bb |   41
>> +++++++++++++++++++++++-
>>   1 files changed, 39 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-core/images/self-hosted-image.bb
>> b/meta/recipes-core/images/self-hosted-image.bb
>> index d56c2cb..5aa670d 100644
>> --- a/meta/recipes-core/images/self-hosted-image.bb
>> +++ b/meta/recipes-core/images/self-hosted-image.bb
>> @@ -4,7 +4,7 @@ LICENSE = "MIT"
>>   LIC_FILES_CHKSUM =
>> "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
>>
>> file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de
>> 20420"
>>
>> -PR = "r5"
>> +PR = "r6"
>>
>>   CORE_IMAGE_EXTRA_INSTALL = "\
>>       task-self-hosted \
>> @@ -13,7 +13,10 @@ CORE_IMAGE_EXTRA_INSTALL = "\  IMAGE_FEATURES
>> += "x11-mini package-management"
>>
>>   # Ensure there's enough space to do a core-image-minimal build, with
>> rm_work enabled -IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
>> +IMAGE_ROOTFS_EXTRA_SPACE = "1048576"
>> +#IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
>> +#IMAGE_ROOTFS_EXTRA_SPACE = "20971520"
>> +#IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
>>
>>   # Do a quiet boot with limited console messages  APPEND += "quiet"
>> @@ -21,3 +24,37 @@ APPEND += "quiet"
>>   IMAGE_FSTYPES = "vmdk"
>>
>>   inherit core-image
>> +
>> +SRCREV = "26a46938d3ea1821e7bec4fa6cc8379babad238b"
>> +SRC_URI = "git://git.yoctoproject.org/poky;protocol=git"
>> +
>> +fakeroot do_populate_poky_src () {
>> +	# Because fetch2's git's unpack uses -s cloneflag, the unpacked git repo
>> +	# will become invalid in the target.
>> +	rm -rf ${WORKDIR}/git/.git
>> +	rm -f ${WORKDIR}/git/.gitignore
>> +
>> +	cp -Rp ${WORKDIR}/git ${IMAGE_ROOTFS}/home/builder/poky
>> +
>> +	mkdir -p ${IMAGE_ROOTFS}/home/builder/poky/build/conf
>> +	cp -Rp ${DL_DIR} ${IMAGE_ROOTFS}/home/builder/poky/build
>> +	echo "/usr/bin">
>> ${IMAGE_ROOTFS}/home/builder/poky/build/pseudodone
>> +	echo "BB_NO_NETWORK = \"1\"">
>> ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf
>> +	echo "INHERIT += \"rm_work\"">>
>> ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf
>> +        mkdir -p ${IMAGE_ROOTFS}/home/builder/pseudo
>> +        echo "export PSEUDO_PREFIX=/usr">>
>> ${IMAGE_ROOTFS}/home/builder/.bashrc
>> +        echo "export PSEUDO_LOCALSTATEDIR=/home/builder/pseudo">>
>> ${IMAGE_ROOTFS}/home/builder/.bashrc
>> +        echo "export PSEUDO_LIBDIR=/usr/lib/pseudo/lib64">>
>> +${IMAGE_ROOTFS}/home/builder/.bashrc
>> +
>> +        chown builder.builder ${IMAGE_ROOTFS}/home/builder/pseudo
>> +
>> +	chown -R builder.builder  ${IMAGE_ROOTFS}/home/builder/poky }
>> +
>> +IMAGE_PREPROCESS_COMMAND += "do_populate_poky_src; "
>> +
>> +python do_get_poky_src () {
>> +    bb.build.exec_func('base_do_fetch', d)
>> +    bb.build.exec_func('base_do_unpack', d) } addtask do_get_poky_src
>> +before do_rootfs
>> --
>> 1.7.7.6
>
>



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source
  2012-03-28 20:45   ` Paul Eggleton
@ 2012-03-28 21:08     ` Saul Wold
  0 siblings, 0 replies; 22+ messages in thread
From: Saul Wold @ 2012-03-28 21:08 UTC (permalink / raw)
  To: Paul Eggleton; +Cc: openembedded-core

On 03/28/2012 01:45 PM, Paul Eggleton wrote:
> On Monday 26 March 2012 22:42:55 Saul Wold wrote:
>> From: Dexuan Cui<dexuan.cui@intel.com>
>>
>> This patch installs the poky source into the /home/builder/poky/ of the
>> self-hosted-image.
>> This makes the user of self-hosted-image easier to start a build.
>>
>> I think the recent poky master is stable enough, so I specify
>> a commit number by SRCREV -- we may want to update this number before
>> releasing 1.2.
>>
>> This patch fixes [YOCTO #2065]
>>
>> Signed-off-by: Dexuan Cui<dexuan.cui@intel.com>
>>
>> Added code for supporting target based pseudo
>> fixed indentation
>>
>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>> ---
>>   meta/recipes-core/images/self-hosted-image.bb |   41
>> +++++++++++++++++++++++- 1 files changed, 39 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-core/images/self-hosted-image.bb
>> b/meta/recipes-core/images/self-hosted-image.bb index d56c2cb..5aa670d
>> 100644
>> --- a/meta/recipes-core/images/self-hosted-image.bb
>> +++ b/meta/recipes-core/images/self-hosted-image.bb
>> @@ -4,7 +4,7 @@ LICENSE = "MIT"
>>   LIC_FILES_CHKSUM =
>> "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
>> file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
>>
>> -PR = "r5"
>> +PR = "r6"
>>
>>   CORE_IMAGE_EXTRA_INSTALL = "\
>>       task-self-hosted \
>> @@ -13,7 +13,10 @@ CORE_IMAGE_EXTRA_INSTALL = "\
>>   IMAGE_FEATURES += "x11-mini package-management"
>>
>>   # Ensure there's enough space to do a core-image-minimal build, with
>> rm_work enabled -IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
>> +IMAGE_ROOTFS_EXTRA_SPACE = "1048576"
>> +#IMAGE_ROOTFS_EXTRA_SPACE = "2621440"
>> +#IMAGE_ROOTFS_EXTRA_SPACE = "20971520"
>> +#IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
>>
>>   # Do a quiet boot with limited console messages
>>   APPEND += "quiet"
>> @@ -21,3 +24,37 @@ APPEND += "quiet"
>>   IMAGE_FSTYPES = "vmdk"
>>
>>   inherit core-image
>> +
>> +SRCREV = "26a46938d3ea1821e7bec4fa6cc8379babad238b"
>> +SRC_URI = "git://git.yoctoproject.org/poky;protocol=git"
>> +
>> +fakeroot do_populate_poky_src () {
>> +	# Because fetch2's git's unpack uses -s cloneflag, the unpacked git repo
>> +	# will become invalid in the target.
>> +	rm -rf ${WORKDIR}/git/.git
>> +	rm -f ${WORKDIR}/git/.gitignore
>> +
>> +	cp -Rp ${WORKDIR}/git ${IMAGE_ROOTFS}/home/builder/poky
>> +
>> +	mkdir -p ${IMAGE_ROOTFS}/home/builder/poky/build/conf
>> +	cp -Rp ${DL_DIR} ${IMAGE_ROOTFS}/home/builder/poky/build
>
> Could we change this last line to:
>
> mkdir ${IMAGE_ROOTFS}/home/builder/poky/build/downloads
> cp -RpL  ${DL_DIR}/* ${IMAGE_ROOTFS}/home/builder/poky/build/downloads/
>
> This does two things:
>
> 1) Handles if DL_DIR on the build machine is not called "downloads" (which by
> chance mine was not when I tested it)
>
> 2) I notice if you set up a separate downloads directory and then use own-
> mirrors to fetch from your original downloads directory (possibly unorthodox,
> but it gets you a clean downloads directory without redownloading everything)
> you get symlinks into the original downloads dir rather than copied files. I
> don't know how many people will hit this but since we never want any symlinks
> in this directory in the self-hosted image, I think using -L is worth doing
> here.
>
That seems like a completely reasonable change and makes good sense, 
your right I did assume DL_DIR would include downloads.

Sau!

> Cheers,
> Paul
>



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source
  2012-03-28 21:07     ` Saul Wold
@ 2012-03-29  7:15       ` Cui, Dexuan
  0 siblings, 0 replies; 22+ messages in thread
From: Cui, Dexuan @ 2012-03-29  7:15 UTC (permalink / raw)
  To: Saul Wold; +Cc: openembedded-core

Saul Wold wrote on 2012-03-29:
> On 03/28/2012 08:35 AM, Cui, Dexuan wrote:
>> Hi Saul,
>> Did you test "bitbake core-image-minimal" inside the vmware guest?
>> I got the following ERROR immediately:
>>      Pseudo is not functioning correctly, which will cause failures
>> I suspect in the guest, pseudo is not setup properly? 
> This should be addressed by the 5/6 patch that adds the correct
> PSEUDO_* setup into the minix session script.
> I guess that you tried to run this on the command line and as you
> might notice I modified the bashrc, but for some reason that did not
> get picked up when the shell started up, I think we need to force the
> builder user to get /bin/bash as a shell not /bin/sh
Hi Saul,
I'm sorry -- this is actually a false alarm -- I didn't use the updated
branch...
With the latest poky master, bitbake in cmd line can work fine.
Hob is automatically started and can work fine, too.

Thanks,
-- Dexuan



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
                   ` (6 preceding siblings ...)
  2012-03-27 14:37 ` [PATCH 0/6] Setup for VMDK to use Direct Disk Paul Eggleton
@ 2012-03-30 14:17 ` Paul Eggleton
  2012-03-30 21:16   ` Cui, Dexuan
  7 siblings, 1 reply; 22+ messages in thread
From: Paul Eggleton @ 2012-03-30 14:17 UTC (permalink / raw)
  To: openembedded-core

On Monday 26 March 2012 22:42:54 Saul Wold wrote:
> Updated comments per Darren's request, added cleanup to
> image-types to only use one -i (inode-count) parameter.
> 
> Sau!
> The following changes since commit 644b7503c37fd73730dd3d7841463b158b8934ed:
> 
>   guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)
> 
> are available in the git repository at:
>   git://git.openembedded.org/openembedded-core-contrib sgw/self
>  
> http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=sgw/
> self

So these patches have been merged now; I updated to latest master and re-ran 
bitbake self-hosted-image; unfortunately the output doesn't appear to be 
usable. I don't know what has gone wrong but during boot there are complaints 
that the filesystem is corrupt:

SYSLINUX 4.03 2010-10-22 EDD Copyright (C) 1994-2010 H. Peter Anvin et al
EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced: 581713
EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced: 610383
Kernel panic - not syncing: no init found.  Try passing init= option to 
kernel.

Running e2fsck -fn on the rootfs.ext3 file shows quite a number of errors. 
There were no unusual errors in the log.do_rootfs.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-30 14:17 ` Paul Eggleton
@ 2012-03-30 21:16   ` Cui, Dexuan
  2012-03-31  5:11     ` Cui, Dexuan
  0 siblings, 1 reply; 22+ messages in thread
From: Cui, Dexuan @ 2012-03-30 21:16 UTC (permalink / raw)
  To: Paul Eggleton, openembedded-core

Paul Eggleton wrote on 2012-03-30:
> On Monday 26 March 2012 22:42:54 Saul Wold wrote:
>> Updated comments per Darren's request, added cleanup to image-types
>> to only use one -i (inode-count) parameter.
>> 
>> Sau! The following changes since commit
>> 644b7503c37fd73730dd3d7841463b158b8934ed:
>> 
>>   guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)
>> are available in the git repository at:
>>   git://git.openembedded.org/openembedded-core-contrib sgw/self
>> http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/
>> ?h
>> =sgw/
>> self
> 
> So these patches have been merged now; I updated to latest master and
> re-ran bitbake self-hosted-image; unfortunately the output doesn't
> appear to be usable. I don't know what has gone wrong but during boot
> there are complaints that the filesystem is corrupt:
> 
> SYSLINUX 4.03 2010-10-22 EDD Copyright (C) 1994-2010 H. Peter Anvin et
> al EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced:
> 581713 EXT3-fs error (device hda2): ext3_lookup: deleted inode
> referenced: 610383 Kernel panic - not syncing: no init found.  Try passing init= option to kernel.
> 
> Running e2fsck -fn on the rootfs.ext3 file shows quite a number of errors.
> There were no unusual errors in the log.do_rootfs.
> 
Hi Paul,
I can reproduce the same I issue, too...
I'll try to look into this, but at the first glance, I don't know what has gone wrong, either...

Thanks,
-- Dexuan





^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-30 21:16   ` Cui, Dexuan
@ 2012-03-31  5:11     ` Cui, Dexuan
  2012-04-01 10:28       ` Cui, Dexuan
  0 siblings, 1 reply; 22+ messages in thread
From: Cui, Dexuan @ 2012-03-31  5:11 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer, Paul Eggleton

Cui, Dexuan wrote on 2012-03-31:
> Paul Eggleton wrote on 2012-03-30:
>> On Monday 26 March 2012 22:42:54 Saul Wold wrote:
>>> Updated comments per Darren's request, added cleanup to image-types
>>> to only use one -i (inode-count) parameter.
>>> 
>>> Sau! The following changes since commit
>>> 644b7503c37fd73730dd3d7841463b158b8934ed:
>>> 
>>>   guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)
>>>   are available in the git repository at:
>>>   git://git.openembedded.org/openembedded-core-contrib sgw/self
>>> http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log
>>> /
>>> ?h
>>> =sgw/
>>> self
>> 
>> So these patches have been merged now; I updated to latest master
>> and re-ran bitbake self-hosted-image; unfortunately the output
>> doesn't appear to be usable. I don't know what has gone wrong but
>> during boot there are complaints that the filesystem is corrupt:
>> 
>> SYSLINUX 4.03 2010-10-22 EDD Copyright (C) 1994-2010 H. Peter Anvin et
>> al EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced:
>> 581713 EXT3-fs error (device hda2): ext3_lookup: deleted inode
>> referenced: 610383 Kernel panic - not syncing: no init found.  Try
>> passing init= option to kernel.
>> 
>> Running e2fsck -fn on the rootfs.ext3 file shows quite a number of errors.
>> There were no unusual errors in the log.do_rootfs.
>> 
> Hi Paul,
> I can reproduce the same I issue, too...
> I'll try to look into this, but at the first glance, I don't know what
> has gone wrong, either...
Can we make a conclusion the current genext2fs is buggy here
when creating a big image(e.g., >4GB)?

Thanks,
-- Dexuan





^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-03-31  5:11     ` Cui, Dexuan
@ 2012-04-01 10:28       ` Cui, Dexuan
  2012-04-01 19:43         ` Paul Eggleton
  0 siblings, 1 reply; 22+ messages in thread
From: Cui, Dexuan @ 2012-04-01 10:28 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer, Paul Eggleton

Cui, Dexuan wrote on 2012-03-31:
> Cui, Dexuan wrote on 2012-03-31:
>> Paul Eggleton wrote on 2012-03-30:
>>> On Monday 26 March 2012 22:42:54 Saul Wold wrote:
>>>> Updated comments per Darren's request, added cleanup to image-types
>>>> to only use one -i (inode-count) parameter.
>>>> 
>>>> Sau! The following changes since commit
>>>> 644b7503c37fd73730dd3d7841463b158b8934ed:
>>>> 
>>>>   guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)
>>>>   are available in the git repository at:
>>>>   git://git.openembedded.org/openembedded-core-contrib sgw/self
>>>> http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log
>>>> /
>>>> ?h
>>>> =sgw/
>>>> self
>>> 
>>> So these patches have been merged now; I updated to latest master
>>> and re-ran bitbake self-hosted-image; unfortunately the output
>>> doesn't appear to be usable. I don't know what has gone wrong but
>>> during boot there are complaints that the filesystem is corrupt:
>>> 
>>> SYSLINUX 4.03 2010-10-22 EDD Copyright (C) 1994-2010 H. Peter Anvin et
>>> al EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced:
>>> 581713 EXT3-fs error (device hda2): ext3_lookup: deleted inode
>>> referenced: 610383 Kernel panic - not syncing: no init found.  Try
>>> passing init= option to kernel.
>>> 
>>> Running e2fsck -fn on the rootfs.ext3 file shows quite a number of errors.
>>> There were no unusual errors in the log.do_rootfs.
>>> 
>> Hi Paul,
>> I can reproduce the same I issue, too...
>> I'll try to look into this, but at the first glance, I don't know what
>> has gone wrong, either...
> Can we make a conclusion the current genext2fs is buggy here
> when creating a big image(e.g., >4GB)?
Hi Paul, I believe this is true:
genext2fs-1.4.1 can create a corrupt file system when
the size of the file system exceeds some limit.

e.g., when I create an image of 2.8GB, the image can be mounted
properly, but when I create an image of about 4.5GB(yes, I found
genext2fs-1.4.1 is actually able to create an image of 4.5GB; I also
found it's unable to create an image of about 5.5GB, always
reporting "couldn't allocate a block (no free space)"),
the generated image can show something like this when it's
booted:

EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced...

However, with Corey's patches (from the genext2fs's mailing list)
applied, I can't see the issues.

You can try the patch too to see if this would also works for you:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=dcui/master&id=c76f9db84c2eabe0bf704b253a957f695f421936 

Thanks,
-- Dexuan





^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [PATCH 0/6] Setup for VMDK to use Direct Disk
  2012-04-01 10:28       ` Cui, Dexuan
@ 2012-04-01 19:43         ` Paul Eggleton
  0 siblings, 0 replies; 22+ messages in thread
From: Paul Eggleton @ 2012-04-01 19:43 UTC (permalink / raw)
  To: Cui, Dexuan; +Cc: Patches and discussions about the oe-core layer

On Sunday 01 April 2012 10:28:35 Cui, Dexuan wrote:
> Hi Paul, I believe this is true:
> genext2fs-1.4.1 can create a corrupt file system when
> the size of the file system exceeds some limit.
> 
> e.g., when I create an image of 2.8GB, the image can be mounted
> properly, but when I create an image of about 4.5GB(yes, I found
> genext2fs-1.4.1 is actually able to create an image of 4.5GB; I also
> found it's unable to create an image of about 5.5GB, always
> reporting "couldn't allocate a block (no free space)"),
> the generated image can show something like this when it's
> booted:
> 
> EXT3-fs error (device hda2): ext3_lookup: deleted inode referenced...
> 
> However, with Corey's patches (from the genext2fs's mailing list)
> applied, I can't see the issues.

I can confirm this.

With these patches applied I also compared (a) using genext2fs to create a 
30GB rootfs, and (b) letting genext2fs create a filesystem with just enough 
space and then resizing it to 30GB with resize2fs. There was no benefit in the 
latter - it took 5 minutes longer.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 22+ messages in thread
end of thread, other threads:[~2012-04-01 19:52 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-03-27  5:42 [PATCH 0/6] Setup for VMDK to use Direct Disk Saul Wold
2012-03-27  5:42 ` [PATCH 1/6] self-hosted-image: pre-populate the builder user with poky source Saul Wold
2012-03-28 15:35   ` Cui, Dexuan
2012-03-28 21:07     ` Saul Wold
2012-03-29  7:15       ` Cui, Dexuan
2012-03-28 20:45   ` Paul Eggleton
2012-03-28 21:08     ` Saul Wold
2012-03-27  5:42 ` [PATCH 2/6] image-vmdk: Add symbolic link for a short named vmdk image Saul Wold
2012-03-27  5:42 ` [PATCH 3/6] boot-directdisk: Fix Block Calcuation Saul Wold
2012-03-27  5:42 ` [PATCH 4/6] vmdk: Update for direstdisk Saul Wold
2012-03-27  5:42 ` [PATCH 5/6] builder: Enable auto starting of Hob Saul Wold
2012-03-27  5:43 ` [PATCH 6/6] image_types: remove duplicate setting of -i for inode count Saul Wold
2012-03-27 14:37 ` [PATCH 0/6] Setup for VMDK to use Direct Disk Paul Eggleton
2012-03-27 18:16   ` Saul Wold
2012-03-27 20:21     ` Paul Eggleton
2012-03-27 20:45       ` João Henrique Freitas
2012-03-28 15:21         ` Cui, Dexuan
2012-03-30 14:17 ` Paul Eggleton
2012-03-30 21:16   ` Cui, Dexuan
2012-03-31  5:11     ` Cui, Dexuan
2012-04-01 10:28       ` Cui, Dexuan
2012-04-01 19:43         ` Paul Eggleton

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.