[V2 PATCH 0/2] Fix init-live.sh to use unionfs for live CD and remove questioned ISO image

[V2 PATCH 0/2] Fix init-live.sh to use unionfs for live CD and remove questioned ISO image

[Yang Shi]

Changelog V1 -> V2:
* Refined patch 1/2 log
* Fixed some spelling erros in patch 1/2
* Instead of removing questioned ISO images, just throw warning info in patch 2/2


When booting up with liveCD image, init scripts can't work well on read-only filesystem. Unionfs,
which is supported in Yocto kernel, allows a filesystem to appear as writeable, but without
actually allowing writes to change the filesystem.

Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
Set UNION_FS variable depending on kernel config, so that it can work with kernel which doesn't
have unionfs feature.

For the liveCD image, interactive bootup is needed, but psplash prevents from
booting interactively. In such case ISO image is not usable, so throw warning
info when building ISO image without unionfs enabled in kernel.

We tried a lot of different ways to achieve this:

1. Check if ISO image is built in initramfs-live-boot.bb or not, then prevent from generating ISO image.
But, we can't distinguish minimal image or sato image.

2. Check if x11 feature is included in initramfs-live-boot.bb or not, then prevent from generating ISO image.
But, x11 feature is contained in minimal image as well even though the package is not built.

3. Tried to append some check code in build_iso() in core-image-sato.bb, but poky can't support append code
in build_xxx functions like what we do for do_xxx funcitons.

So, we came up with current solution. Any new suggestion is appreciated.

One more thing is that if IMAGE_FSTYPE += "live" is not set in conf/local.conf with this fix, we will get below
error when building sato image:

ERROR: Running idle function
Traceback (most recent call last):
  File "/home/yshi/yocto/poky/bitbake/lib/bb/server/process.py", line
122, in ProcessServer.idle_commands(delay=0.1):
                 try:
    >                retval = function(self, data, False)
                     if retval is False:
  File "/home/yshi/yocto/poky/bitbake/lib/bb/cooker.py", line 1130, in
buildTargetsIdle(server=<ProcessServer(ProcessServer-1, started)>,
rq=<bb.runqueue.RunQueue instance at 0x74e4638>, abort=False):
                 try:
    >                retval = rq.execute_runqueue()
                 except runqueue.TaskFailure as exc:
  File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 947,
in RunQueue.execute_runqueue():
                 self.rqexe = RunQueueExecuteDummy(self)
    >            if self.rqdata.prepare() == 0:
                     self.state = runQueueComplete
  File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 719,
in RunQueueData.prepare():

procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." +
self.runq_task[dep])
    >                    self.runq_hash[task] =
bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]],
self.runq_task[task], procdep, self.dataCache)

  File "/home/yshi/yocto/poky/bitbake/lib/bb/siggen.py", line 153, in
SignatureGeneratorOEBasicHash.get_taskhash(fn='/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb',
task='do_bootimg', deps=[], dataCache=<bb.cache.CacheData object at
0x2148490>):
             k = fn + "." + task
    >        data = dataCache.basetaskhash[k]
             self.runtaskdeps[k] = []
KeyError: '/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb.do_bootimg'

NOTE: Preparing runqueue

Bruce pointed out there was a large discussion about this on hte list last week, so this should be a known issue in poky.

The following changes since commit f81b0593e74a31cb2d992df0583948ff57e3ed98:

  gdbm: Activate -enable-libgdbm-compat and add symlinks to headers in include/gdbm (2012-03-23 17:56:29 +0200)

are available in the git repository at:
  git://git.yoctoproject.org/poky-contrib yshi/1487v2
  http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=yshi/1487v2

Yang Shi (2):
      initrdscripts: fix init-live.sh and use unionfs
      sato:  Throw warning when building ISO image without unionfs enabled

 meta/recipes-core/initrdscripts/files/init-live.sh |   23 ++++++++++++++++---
 .../initrdscripts/initramfs-live-boot_1.0.bb       |    9 +++++++-
 meta/recipes-sato/images/core-image-sato.bb        |   14 ++++++++++++
 3 files changed, 41 insertions(+), 5 deletions(-)

[PATCH 1/2] initrdscripts: fix init-live.sh and use unionfs

[Yang Shi]

[YOCTO #1487]

When booting up with liveCD image, init scripts can't work well on read-only filesystem. Unionfs,
which is supported in Yocto kernel, allows a filesystem to appear as writeable, but without
actually allowing writes to change the filesystem.

Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
Set UNION_FS variable depending on kernel config, so that it can work with kernel which doesn't
have unionfs feature.

Signed-off-by: Yang Shi <yang.shi@windriver.com>
---
 meta/recipes-core/initrdscripts/files/init-live.sh |   23 ++++++++++++++++---
 .../initrdscripts/initramfs-live-boot_1.0.bb       |    9 +++++++-
 2 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-core/initrdscripts/files/init-live.sh b/meta/recipes-core/initrdscripts/files/init-live.sh
index eb5ab5b..737dae4 100644
--- a/meta/recipes-core/initrdscripts/files/init-live.sh
+++ b/meta/recipes-core/initrdscripts/files/init-live.sh
@@ -7,6 +7,7 @@ ROOT_IMAGE="rootfs.img"
 MOUNT="/bin/mount"
 UMOUNT="/bin/umount"
 ISOLINUX=""
+UNIONFS="no"
 
 early_setup() {
     mkdir /proc
@@ -89,17 +90,31 @@ case $label in
 	mkdir $ROOT_MOUNT
 	mknod /dev/loop0 b 7 0 2>/dev/null
 
-	if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE $ROOT_MOUNT ; then
-	    fatal "Couldnt mount rootfs image"
+	
+	if [ "$UNIONFS" = "yes" ]; then
+	    mkdir /rootfs-tmp
+
+	    if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE /rootfs-tmp ; then
+		fatal "Could not mount rootfs image"
+	    else
+		mkdir /cow
+		mount -t tmpfs -o rw,noatime,mode=755 tmpfs /cow
+		mount -t unionfs -o dirs=/cow:/rootfs-tmp=ro unionfs $ROOT_MOUNT
+		boot_live_root
+	    fi
 	else
-	    boot_live_root
+	    if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE $ROOT_MOUNT ; then
+		fatal "Could not mount rootfs image"
+	    else
+		boot_live_root
+	    fi
 	fi
 	;;
     install)
 	if [ -f /media/$i/$ISOLINUX/$ROOT_IMAGE ] ; then
 	    ./install.sh $i/$ISOLINUX $ROOT_IMAGE $video_mode $vga_mode
 	else
-	    fatal "Couldnt find install script"
+	    fatal "Could not find install script"
 	fi
 
 	# If we're getting here, we failed...
diff --git a/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb b/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
index e85a0e1..f7f0c9d 100644
--- a/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
+++ b/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
@@ -2,10 +2,17 @@ DESCRIPTION = "A live image init script"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
 RDEPENDS = "udev"
+DEPENDS = "virtual/kernel"
 SRC_URI = "file://init-live.sh"
 
-PR = "r7"
+PR = "r8"
 
+do_compile() {
+	if grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
+		sed -i 's/UNIONFS="no"/UNIONFS="yes"/g' ${WORKDIR}/init-live.sh
+	fi
+}
+ 
 do_install() {
         install -m 0755 ${WORKDIR}/init-live.sh ${D}/init
 }
-- 
1.7.5.4

[PATCH 2/2] sato: Throw warning when building ISO image without unionfs enabled

[Yang Shi]

[YOCTO #1487]

For the liveCD image, interactive bootup is needed, but psplash prevents from
booting interactively. In such case ISO image is not usable, so throw warning
info when building ISO image without unionfs enabled in kernel.

Signed-off-by: Yang Shi <yang.shi@windriver.com>
---
 meta/recipes-sato/images/core-image-sato.bb |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/meta/recipes-sato/images/core-image-sato.bb b/meta/recipes-sato/images/core-image-sato.bb
index 11c3318..871b227 100644
--- a/meta/recipes-sato/images/core-image-sato.bb
+++ b/meta/recipes-sato/images/core-image-sato.bb
@@ -9,3 +9,17 @@ IMAGE_FEATURES += "apps-console-core ${SATO_IMAGE_FEATURES}"
 LICENSE = "MIT"
 
 inherit core-image
+
+LIVE = "${@base_contains('IMAGE_FSTYPES', 'live', 'yes', 'no', d)}"
+
+do_check_unionfs() {
+        if [ "${NOISO}" = "1" ]; then
+                return
+        fi
+
+        if [ "${LIVE}" = "yes" ] && ! grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
+                bbwarn "Building LIVE CD without UNION FS enabled in kernel"
+        fi
+}
+
+addtask check_unionfs before do_build after do_bootimg
-- 
1.7.5.4

Re: [PATCH 1/2] initrdscripts: fix init-live.sh and use unionfs

[Darren Hart]

On 04/09/2012 03:15 PM, Yang Shi wrote:
> [YOCTO #1487]
> 
> When booting up with liveCD image, init scripts can't work well on read-only filesystem. Unionfs,
> which is supported in Yocto kernel, allows a filesystem to appear as writeable, but without
> actually allowing writes to change the filesystem.
> 
> Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
> Set UNION_FS variable depending on kernel config, so that it can work with kernel which doesn't
> have unionfs feature.
> 
> Signed-off-by: Yang Shi <yang.shi@windriver.com>

Acked-by: Darren Hart <dvhart@linux.intel.com>

> ---
>  meta/recipes-core/initrdscripts/files/init-live.sh |   23 ++++++++++++++++---
>  .../initrdscripts/initramfs-live-boot_1.0.bb       |    9 +++++++-
>  2 files changed, 27 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/recipes-core/initrdscripts/files/init-live.sh b/meta/recipes-core/initrdscripts/files/init-live.sh
> index eb5ab5b..737dae4 100644
> --- a/meta/recipes-core/initrdscripts/files/init-live.sh
> +++ b/meta/recipes-core/initrdscripts/files/init-live.sh
> @@ -7,6 +7,7 @@ ROOT_IMAGE="rootfs.img"
>  MOUNT="/bin/mount"
>  UMOUNT="/bin/umount"
>  ISOLINUX=""
> +UNIONFS="no"
>  
>  early_setup() {
>      mkdir /proc
> @@ -89,17 +90,31 @@ case $label in
>  	mkdir $ROOT_MOUNT
>  	mknod /dev/loop0 b 7 0 2>/dev/null
>  
> -	if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE $ROOT_MOUNT ; then
> -	    fatal "Couldnt mount rootfs image"
> +	
> +	if [ "$UNIONFS" = "yes" ]; then
> +	    mkdir /rootfs-tmp
> +
> +	    if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE /rootfs-tmp ; then
> +		fatal "Could not mount rootfs image"
> +	    else
> +		mkdir /cow
> +		mount -t tmpfs -o rw,noatime,mode=755 tmpfs /cow
> +		mount -t unionfs -o dirs=/cow:/rootfs-tmp=ro unionfs $ROOT_MOUNT
> +		boot_live_root
> +	    fi
>  	else
> -	    boot_live_root
> +	    if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE $ROOT_MOUNT ; then
> +		fatal "Could not mount rootfs image"
> +	    else
> +		boot_live_root
> +	    fi
>  	fi
>  	;;
>      install)
>  	if [ -f /media/$i/$ISOLINUX/$ROOT_IMAGE ] ; then
>  	    ./install.sh $i/$ISOLINUX $ROOT_IMAGE $video_mode $vga_mode
>  	else
> -	    fatal "Couldnt find install script"
> +	    fatal "Could not find install script"
>  	fi
>  
>  	# If we're getting here, we failed...
> diff --git a/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb b/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
> index e85a0e1..f7f0c9d 100644
> --- a/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
> +++ b/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
> @@ -2,10 +2,17 @@ DESCRIPTION = "A live image init script"
>  LICENSE = "MIT"
>  LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
>  RDEPENDS = "udev"
> +DEPENDS = "virtual/kernel"
>  SRC_URI = "file://init-live.sh"
>  
> -PR = "r7"
> +PR = "r8"
>  
> +do_compile() {
> +	if grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
> +		sed -i 's/UNIONFS="no"/UNIONFS="yes"/g' ${WORKDIR}/init-live.sh
> +	fi
> +}
> + 
>  do_install() {
>          install -m 0755 ${WORKDIR}/init-live.sh ${D}/init
>  }

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: [PATCH 2/2] sato: Throw warning when building ISO image without unionfs enabled

[Darren Hart]

On 04/09/2012 03:15 PM, Yang Shi wrote:
> [YOCTO #1487]
> 
> For the liveCD image, interactive bootup is needed, but psplash prevents from
> booting interactively. In such case ISO image is not usable, so throw warning
> info when building ISO image without unionfs enabled in kernel.
> 
> Signed-off-by: Yang Shi <yang.shi@windriver.com>

Acked-by: Darren Hart <dvhart@linux.intel.com>

> ---
>  meta/recipes-sato/images/core-image-sato.bb |   14 ++++++++++++++
>  1 files changed, 14 insertions(+), 0 deletions(-)
> 
> diff --git a/meta/recipes-sato/images/core-image-sato.bb b/meta/recipes-sato/images/core-image-sato.bb
> index 11c3318..871b227 100644
> --- a/meta/recipes-sato/images/core-image-sato.bb
> +++ b/meta/recipes-sato/images/core-image-sato.bb
> @@ -9,3 +9,17 @@ IMAGE_FEATURES += "apps-console-core ${SATO_IMAGE_FEATURES}"
>  LICENSE = "MIT"
>  
>  inherit core-image
> +
> +LIVE = "${@base_contains('IMAGE_FSTYPES', 'live', 'yes', 'no', d)}"
> +
> +do_check_unionfs() {
> +        if [ "${NOISO}" = "1" ]; then
> +                return
> +        fi
> +
> +        if [ "${LIVE}" = "yes" ] && ! grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
> +                bbwarn "Building LIVE CD without UNION FS enabled in kernel"
> +        fi
> +}
> +
> +addtask check_unionfs before do_build after do_bootimg

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: [PATCH 2/2] sato: Throw warning when building ISO image without unionfs enabled

[Saul Wold]

On 04/09/2012 03:15 PM, Yang Shi wrote:
> [YOCTO #1487]
>
> For the liveCD image, interactive bootup is needed, but psplash prevents from
> booting interactively. In such case ISO image is not usable, so throw warning
> info when building ISO image without unionfs enabled in kernel.
>
> Signed-off-by: Yang Shi<yang.shi@windriver.com>
> ---
>   meta/recipes-sato/images/core-image-sato.bb |   14 ++++++++++++++
>   1 files changed, 14 insertions(+), 0 deletions(-)
>
> diff --git a/meta/recipes-sato/images/core-image-sato.bb b/meta/recipes-sato/images/core-image-sato.bb
> index 11c3318..871b227 100644
> --- a/meta/recipes-sato/images/core-image-sato.bb
> +++ b/meta/recipes-sato/images/core-image-sato.bb
> @@ -9,3 +9,17 @@ IMAGE_FEATURES += "apps-console-core ${SATO_IMAGE_FEATURES}"
>   LICENSE = "MIT"
>
>   inherit core-image
> +
> +LIVE = "${@base_contains('IMAGE_FSTYPES', 'live', 'yes', 'no', d)}"
> +
> +do_check_unionfs() {
> +        if [ "${NOISO}" = "1" ]; then
> +                return
> +        fi
> +
> +        if [ "${LIVE}" = "yes" ]&&  ! grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
I think you are missing a "]" at the end of this "if [" test.
> +                bbwarn "Building LIVE CD without UNION FS enabled in kernel"
> +        fi
> +}
> +
> +addtask check_unionfs before do_build after do_bootimg

This change is causing a new failure:

Traceback (most recent call last):
   File 
"/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/server/process.py", 
line 122, in ProcessServer.idle_commands(delay=0.1):
                  try:
     >                retval = function(self, data, False)
                      if retval is False:
   File 
"/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/cooker.py", 
line 1134, in buildTargetsIdle(server=<ProcessServer(ProcessServer-1, 
started)>, rq=<bb.runqueue.RunQueue instance at 0xb6891b8>, abort=False):
                  try:
     >                retval = rq.execute_runqueue()
                  except runqueue.TaskFailure as exc:
   File 
"/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/runqueue.py", 
line 948, in RunQueue.execute_runqueue():
                  self.rqexe = RunQueueExecuteDummy(self)
     >            if self.rqdata.prepare() == 0:
                      self.state = runQueueComplete
   File 
"/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/runqueue.py", 
line 719, in RunQueueData.prepare():
 
procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." + 
self.runq_task[dep])
     >                    self.runq_hash[task] = 
bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]], 
self.runq_task[task], procdep, self.dataCache)

   File 
"/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/siggen.py", 
line 153, in 
SignatureGeneratorOEBasicHash.get_taskhash(fn='/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/meta/recipes-sato/images/core-image-sato.bb', 
task='do_bootimg', deps=[], dataCache=<bb.cache.CacheData object at 
0x2ede190>):
              k = fn + "." + task
     >        data = dataCache.basetaskhash[k]
              self.runtaskdeps[k] = []
KeyError: 
'/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/meta/recipes-sato/images/core-image-sato.bb.do_bootimg'

Re: [PATCH 2/2] sato: Throw warning when building ISO image without unionfs enabled

[Yang Shi]

On 4/10/2012 12:06 PM, Saul Wold wrote:
> On 04/09/2012 03:15 PM, Yang Shi wrote:
>> [YOCTO #1487]
>>
>> For the liveCD image, interactive bootup is needed, but psplash 
>> prevents from
>> booting interactively. In such case ISO image is not usable, so throw 
>> warning
>> info when building ISO image without unionfs enabled in kernel.
>>
>> Signed-off-by: Yang Shi<yang.shi@windriver.com>
>> ---
>>   meta/recipes-sato/images/core-image-sato.bb |   14 ++++++++++++++
>>   1 files changed, 14 insertions(+), 0 deletions(-)
>>
>> diff --git a/meta/recipes-sato/images/core-image-sato.bb 
>> b/meta/recipes-sato/images/core-image-sato.bb
>> index 11c3318..871b227 100644
>> --- a/meta/recipes-sato/images/core-image-sato.bb
>> +++ b/meta/recipes-sato/images/core-image-sato.bb
>> @@ -9,3 +9,17 @@ IMAGE_FEATURES += "apps-console-core 
>> ${SATO_IMAGE_FEATURES}"
>>   LICENSE = "MIT"
>>
>>   inherit core-image
>> +
>> +LIVE = "${@base_contains('IMAGE_FSTYPES', 'live', 'yes', 'no', d)}"
>> +
>> +do_check_unionfs() {
>> +        if [ "${NOISO}" = "1" ]; then
>> +                return
>> +        fi
>> +
>> +        if [ "${LIVE}" = "yes" ]&&  ! grep -q "CONFIG_UNION_FS=y" 
>> ${STAGING_KERNEL_DIR}/.config; then
> I think you are missing a "]" at the end of this "if [" test.

It is 'if [ "${LIVE}" = "yes" ]'.

>> +                bbwarn "Building LIVE CD without UNION FS enabled in 
>> kernel"
>> +        fi
>> +}
>> +
>> +addtask check_unionfs before do_build after do_bootimg
>
> This change is causing a new failure:

Yes, I mentioned this in my mail, the workaround is to add IMAGE_FSTYPE 
+= "live" in conf/local.conf

Bruce pointed out this is a known problem in poky.

Thanks,
Yang

>
> Traceback (most recent call last):
>   File 
> "/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/server/process.py", 
> line 122, in ProcessServer.idle_commands(delay=0.1):
>                  try:
> >                retval = function(self, data, False)
>                      if retval is False:
>   File 
> "/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/cooker.py", 
> line 1134, in buildTargetsIdle(server=<ProcessServer(ProcessServer-1, 
> started)>, rq=<bb.runqueue.RunQueue instance at 0xb6891b8>, abort=False):
>                  try:
> >                retval = rq.execute_runqueue()
>                  except runqueue.TaskFailure as exc:
>   File 
> "/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/runqueue.py", 
> line 948, in RunQueue.execute_runqueue():
>                  self.rqexe = RunQueueExecuteDummy(self)
> >            if self.rqdata.prepare() == 0:
>                      self.state = runQueueComplete
>   File 
> "/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/runqueue.py", 
> line 719, in RunQueueData.prepare():
>
> procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." + 
> self.runq_task[dep])
> >                    self.runq_hash[task] = 
> bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]], 
> self.runq_task[task], procdep, self.dataCache)
>
>   File 
> "/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/bitbake/lib/bb/siggen.py", 
> line 153, in 
> SignatureGeneratorOEBasicHash.get_taskhash(fn='/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/meta/recipes-sato/images/core-image-sato.bb', 
> task='do_bootimg', deps=[], dataCache=<bb.cache.CacheData object at 
> 0x2ede190>):
>              k = fn + "." + task
> >        data = dataCache.basetaskhash[k]
>              self.runtaskdeps[k] = []
> KeyError: 
> '/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-arm/build/meta/recipes-sato/images/core-image-sato.bb.do_bootimg'
>
>

Re: [V2 PATCH 0/2] Fix init-live.sh to use unionfs for live CD and remove questioned ISO image

[Saul Wold]

On 04/09/2012 03:15 PM, Yang Shi wrote:
>
> Changelog V1 ->  V2:
> * Refined patch 1/2 log
> * Fixed some spelling erros in patch 1/2
> * Instead of removing questioned ISO images, just throw warning info in patch 2/2
>
>
> When booting up with liveCD image, init scripts can't work well on read-only filesystem. Unionfs,
> which is supported in Yocto kernel, allows a filesystem to appear as writeable, but without
> actually allowing writes to change the filesystem.
>
> Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
> Set UNION_FS variable depending on kernel config, so that it can work with kernel which doesn't
> have unionfs feature.
>
> For the liveCD image, interactive bootup is needed, but psplash prevents from
> booting interactively. In such case ISO image is not usable, so throw warning
> info when building ISO image without unionfs enabled in kernel.
>
> We tried a lot of different ways to achieve this:
>
> 1. Check if ISO image is built in initramfs-live-boot.bb or not, then prevent from generating ISO image.
> But, we can't distinguish minimal image or sato image.
>
> 2. Check if x11 feature is included in initramfs-live-boot.bb or not, then prevent from generating ISO image.
> But, x11 feature is contained in minimal image as well even though the package is not built.
>
> 3. Tried to append some check code in build_iso() in core-image-sato.bb, but poky can't support append code
> in build_xxx functions like what we do for do_xxx funcitons.
>
> So, we came up with current solution. Any new suggestion is appreciated.
>
> One more thing is that if IMAGE_FSTYPE += "live" is not set in conf/local.conf with this fix, we will get below
> error when building sato image:
>
> ERROR: Running idle function
> Traceback (most recent call last):
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/server/process.py", line
> 122, in ProcessServer.idle_commands(delay=0.1):
>                   try:
>      >                 retval = function(self, data, False)
>                       if retval is False:
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/cooker.py", line 1130, in
> buildTargetsIdle(server=<ProcessServer(ProcessServer-1, started)>,
> rq=<bb.runqueue.RunQueue instance at 0x74e4638>, abort=False):
>                   try:
>      >                 retval = rq.execute_runqueue()
>                   except runqueue.TaskFailure as exc:
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 947,
> in RunQueue.execute_runqueue():
>                   self.rqexe = RunQueueExecuteDummy(self)
>      >             if self.rqdata.prepare() == 0:
>                       self.state = runQueueComplete
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 719,
> in RunQueueData.prepare():
>
> procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." +
> self.runq_task[dep])
>      >                     self.runq_hash[task] =
> bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]],
> self.runq_task[task], procdep, self.dataCache)
>
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/siggen.py", line 153, in
> SignatureGeneratorOEBasicHash.get_taskhash(fn='/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb',
> task='do_bootimg', deps=[], dataCache=<bb.cache.CacheData object at
> 0x2148490>):
>               k = fn + "." + task
>      >         data = dataCache.basetaskhash[k]
>               self.runtaskdeps[k] = []
> KeyError: '/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb.do_bootimg'
>
> NOTE: Preparing runqueue
>
> Bruce pointed out there was a large discussion about this on hte list last week, so this should be a known issue in poky.
>
I am not finding the discussion, maybe it's under some other title, can 
someone please point me at this large discussion about the above failure?


> The following changes since commit f81b0593e74a31cb2d992df0583948ff57e3ed98:
>
>    gdbm: Activate -enable-libgdbm-compat and add symlinks to headers in include/gdbm (2012-03-23 17:56:29 +0200)
>
> are available in the git repository at:
>    git://git.yoctoproject.org/poky-contrib yshi/1487v2
>    http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=yshi/1487v2
>
> Yang Shi (2):
>        initrdscripts: fix init-live.sh and use unionfs

merged this one in

>        sato:  Throw warning when building ISO image without unionfs enabled
>
This is not merged since in the normal case with out live in 
IMAGE_FSTYPES, the above error occurs, that is not the proper way to 
handle this.

The underlying issue needs to be addressed first.

>   meta/recipes-core/initrdscripts/files/init-live.sh |   23 ++++++++++++++++---
>   .../initrdscripts/initramfs-live-boot_1.0.bb       |    9 +++++++-
>   meta/recipes-sato/images/core-image-sato.bb        |   14 ++++++++++++
>   3 files changed, 41 insertions(+), 5 deletions(-)
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

Re: [yocto] [V2 PATCH 0/2] Fix init-live.sh to use unionfs for live CD and remove questioned ISO image

[Saul Wold]

On 04/09/2012 03:15 PM, Yang Shi wrote:
>
> Changelog V1 ->  V2:
> * Refined patch 1/2 log
> * Fixed some spelling erros in patch 1/2
> * Instead of removing questioned ISO images, just throw warning info in patch 2/2
>
>
> When booting up with liveCD image, init scripts can't work well on read-only filesystem. Unionfs,
> which is supported in Yocto kernel, allows a filesystem to appear as writeable, but without
> actually allowing writes to change the filesystem.
>
> Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
> Set UNION_FS variable depending on kernel config, so that it can work with kernel which doesn't
> have unionfs feature.
>
> For the liveCD image, interactive bootup is needed, but psplash prevents from
> booting interactively. In such case ISO image is not usable, so throw warning
> info when building ISO image without unionfs enabled in kernel.
>
> We tried a lot of different ways to achieve this:
>
> 1. Check if ISO image is built in initramfs-live-boot.bb or not, then prevent from generating ISO image.
> But, we can't distinguish minimal image or sato image.
>
> 2. Check if x11 feature is included in initramfs-live-boot.bb or not, then prevent from generating ISO image.
> But, x11 feature is contained in minimal image as well even though the package is not built.
>
> 3. Tried to append some check code in build_iso() in core-image-sato.bb, but poky can't support append code
> in build_xxx functions like what we do for do_xxx funcitons.
>
> So, we came up with current solution. Any new suggestion is appreciated.
>
> One more thing is that if IMAGE_FSTYPE += "live" is not set in conf/local.conf with this fix, we will get below
> error when building sato image:
>
> ERROR: Running idle function
> Traceback (most recent call last):
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/server/process.py", line
> 122, in ProcessServer.idle_commands(delay=0.1):
>                   try:
>      >                 retval = function(self, data, False)
>                       if retval is False:
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/cooker.py", line 1130, in
> buildTargetsIdle(server=<ProcessServer(ProcessServer-1, started)>,
> rq=<bb.runqueue.RunQueue instance at 0x74e4638>, abort=False):
>                   try:
>      >                 retval = rq.execute_runqueue()
>                   except runqueue.TaskFailure as exc:
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 947,
> in RunQueue.execute_runqueue():
>                   self.rqexe = RunQueueExecuteDummy(self)
>      >             if self.rqdata.prepare() == 0:
>                       self.state = runQueueComplete
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 719,
> in RunQueueData.prepare():
>
> procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." +
> self.runq_task[dep])
>      >                     self.runq_hash[task] =
> bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]],
> self.runq_task[task], procdep, self.dataCache)
>
>    File "/home/yshi/yocto/poky/bitbake/lib/bb/siggen.py", line 153, in
> SignatureGeneratorOEBasicHash.get_taskhash(fn='/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb',
> task='do_bootimg', deps=[], dataCache=<bb.cache.CacheData object at
> 0x2148490>):
>               k = fn + "." + task
>      >         data = dataCache.basetaskhash[k]
>               self.runtaskdeps[k] = []
> KeyError: '/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb.do_bootimg'
>
> NOTE: Preparing runqueue
>
> Bruce pointed out there was a large discussion about this on hte list last week, so this should be a known issue in poky.
>
> The following changes since commit f81b0593e74a31cb2d992df0583948ff57e3ed98:
>
>    gdbm: Activate -enable-libgdbm-compat and add symlinks to headers in include/gdbm (2012-03-23 17:56:29 +0200)
>
> are available in the git repository at:
>    git://git.yoctoproject.org/poky-contrib yshi/1487v2
>    http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=yshi/1487v2
>
> Yang Shi (2):
>        initrdscripts: fix init-live.sh and use unionfs
>        sato:  Throw warning when building ISO image without unionfs enabled
>
>   meta/recipes-core/initrdscripts/files/init-live.sh |   23 ++++++++++++++++---
>   .../initrdscripts/initramfs-live-boot_1.0.bb       |    9 +++++++-
>   meta/recipes-sato/images/core-image-sato.bb        |   14 ++++++++++++
>   3 files changed, 41 insertions(+), 5 deletions(-)
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

The first part of the patch set was merged, but then later disabled as 
there were problems with unionfs and images.

Please revisit this patch set

Thanks
	Sau!