[PATCH V2 0/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[PATCH V2 0/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

Changed in V2: 
- Strip the commit message within 80 characters a line

//Hongxu

The following changes since commit 52b788c6df9201764130ab744bf67b770b24b896:

  autogen-native: inherit pkgconfig to fix a build failure (2014-08-25 10:26:00 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib hongxu/bitbake
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/bitbake

Hongxu Jia (1):
  bitbake: runqueue: terminate build safely while
    RunQueueExecuteScenequeue init failed

 bitbake/lib/bb/runqueue.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

-- 
1.9.1

[PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

While RunQueueExecuteScenequeue init failed, the exit of build is mess.
Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
...
Traceback (most recent call last):
|  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py",
  line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue
  instance at 0x7f87dd7d5050>):
|                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname,
  "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
|    >            valid = bb.utils.better_eval(call, locs)

|    def better_eval(source, locals):
|    >    return eval(source, get_context(), locals)

|  File "<string>", line 1, in <module>
|  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102,
  in fatal:
|         logger.critical(''.join(args))
|    >    raise BBHandledException()

BBHandledException
...

We should terminate build safely while RunQueueExecuteScenequeue init failed.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 bitbake/lib/bb/runqueue.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index e13dc57..c7b1dc0 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1046,7 +1046,12 @@ class RunQueue:
                 self.state = runQueueComplete
             else:
                 self.start_worker()
-                self.rqexe = RunQueueExecuteScenequeue(self)
+                try:
+                    self.rqexe = RunQueueExecuteScenequeue(self)
+                except:
+                    # The RunQueueExecuteScenequeue init failure
+                    # terminated the build safely
+                    self.state = runQueueComplete
 
         if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
             self.dm.check(self)
-- 
1.9.1

Re: [PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Richard Purdie]

On Wed, 2014-08-27 at 18:44 +0800, Hongxu Jia wrote:
> While RunQueueExecuteScenequeue init failed, the exit of build is mess.
> Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
> ...
> Traceback (most recent call last):
> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py",
>   line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue
>   instance at 0x7f87dd7d5050>):
> |                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname,
>   "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
> |    >            valid = bb.utils.better_eval(call, locs)
> 
> |    def better_eval(source, locals):
> |    >    return eval(source, get_context(), locals)
> 
> |  File "<string>", line 1, in <module>
> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102,
>   in fatal:
> |         logger.critical(''.join(args))
> |    >    raise BBHandledException()
> 
> BBHandledException
> ...
> 
> We should terminate build safely while RunQueueExecuteScenequeue init failed.
> 
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  bitbake/lib/bb/runqueue.py | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> index e13dc57..c7b1dc0 100644
> --- a/bitbake/lib/bb/runqueue.py
> +++ b/bitbake/lib/bb/runqueue.py
> @@ -1046,7 +1046,12 @@ class RunQueue:
>                  self.state = runQueueComplete
>              else:
>                  self.start_worker()
> -                self.rqexe = RunQueueExecuteScenequeue(self)
> +                try:
> +                    self.rqexe = RunQueueExecuteScenequeue(self)
> +                except:
> +                    # The RunQueueExecuteScenequeue init failure
> +                    # terminated the build safely
> +                    self.state = runQueueComplete
>  

If the init fails, shouldn't we show a traceback and exit with an error
status? Does this happen in a normal build and if so, is there an
underlying cause we should fix?

I'm worried this would let errors go unnoticed. Except clauses in
general should catch specific errors not every exception so this really
can't merge in this form, sorry.

Cheers,

Richard

Re: [PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

On 08/27/2014 06:49 PM, Richard Purdie wrote:
> On Wed, 2014-08-27 at 18:44 +0800, Hongxu Jia wrote:
>> While RunQueueExecuteScenequeue init failed, the exit of build is mess.
>> Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
>> ...
>> Traceback (most recent call last):
>> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py",
>>    line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue
>>    instance at 0x7f87dd7d5050>):
>> |                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname,
>>    "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
>> |    >            valid = bb.utils.better_eval(call, locs)
>>
>> |    def better_eval(source, locals):
>> |    >    return eval(source, get_context(), locals)
>>
>> |  File "<string>", line 1, in <module>
>> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102,
>>    in fatal:
>> |         logger.critical(''.join(args))
>> |    >    raise BBHandledException()
>>
>> BBHandledException
>> ...
>>
>> We should terminate build safely while RunQueueExecuteScenequeue init failed.
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>> ---
>>   bitbake/lib/bb/runqueue.py | 7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
>> index e13dc57..c7b1dc0 100644
>> --- a/bitbake/lib/bb/runqueue.py
>> +++ b/bitbake/lib/bb/runqueue.py
>> @@ -1046,7 +1046,12 @@ class RunQueue:
>>                   self.state = runQueueComplete
>>               else:
>>                   self.start_worker()
>> -                self.rqexe = RunQueueExecuteScenequeue(self)
>> +                try:
>> +                    self.rqexe = RunQueueExecuteScenequeue(self)
>> +                except:
>> +                    # The RunQueueExecuteScenequeue init failure
>> +                    # terminated the build safely
>> +                    self.state = runQueueComplete
>>   
> If the init fails, shouldn't we show a traceback and exit with an error
> status? Does this happen in a normal build and if so, is there an
> underlying cause we should fix?

In my case, I want to catch the failure of self.rq.hashvalidate which
invoked in RunQueueExecuteScenequeue, and exit in a normal build.

>
> I'm worried this would let errors go unnoticed. Except clauses in
> general should catch specific errors not every exception so this really
> can't merge in this form, sorry.

Got it, I will add condition to catch the failure of self.rq.hashvalidate,
and don't affect other errors.

Thanks,
//Hongxu

> Cheers,
>
> Richard
>
>

Re: [PATCH V2 0/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

On Wed, Aug 27, 2014 at 06:44:40PM +0800, Hongxu Jia wrote:
> Changed in V2: 
> - Strip the commit message within 80 characters a line
> 
> //Hongxu
> 
> The following changes since commit 52b788c6df9201764130ab744bf67b770b24b896:
> 
>   autogen-native: inherit pkgconfig to fix a build failure (2014-08-25 10:26:00 +0100)
> 
> are available in the git repository at:
> 
>   git://git.pokylinux.org/poky-contrib hongxu/bitbake
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/bitbake

It would be nice to have patches for bitbake repository on bitbake-devel
ML.

> Hongxu Jia (1):
>   bitbake: runqueue: terminate build safely while
>     RunQueueExecuteScenequeue init failed
> 
>  bitbake/lib/bb/runqueue.py | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> -- 
> 1.9.1
> 
> -- 
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

[PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

While RunQueueExecuteScenequeue init failed, the exit of build is mess.
Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
...
Traceback (most recent call last):
  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py", line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue instance at 0x7f87dd7d5050>):
                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname, "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
    >            valid = bb.utils.better_eval(call, locs)

  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/utils.py", line 374, in better_eval(source='sstate_checkhashes(sq_fn, sq_task, sq_hash, sq_hashfn, d)', locals={'sq_fn': ['virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb'], 'sq_task': ['do_populate_sysroot', 'do_populate_lic', 'do_package_qa'], 'sq_hash': ['7d132e7705b5772accce5544cf785c70', '52387c5751c018ddfe2c6a5eecf59e36', '315bcde98908fd36442830f9ae7469c1'], 'sq_hashfn': ['Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:'], 'd': <bb.data_smart.DataSmart object at 0x7f87e15cde10>}):
     def better_eval(source, locals):
    >    return eval(source, get_context(), locals)

  File "<string>", line 1, in <module>
  File "sstate.bbclass", line 110, in sstate_checkhashes(sq_fn=['virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb'], sq_task=['do_populate_sysroot', 'do_populate_lic', 'do_package_qa'], sq_hash=['7d132e7705b5772accce5544cf785c70', '52387c5751c018ddfe2c6a5eecf59e36', '315bcde98908fd36442830f9ae7469c1'], sq_hashfn=['Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:'], d=<bb.data_smart.DataSmart object at 0x7f87e15cde10>)
  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102, in fatal:
         logger.critical(''.join(args))
    >    raise BBHandledException()

BBHandledException
...

We should terminate build safely while RunQueueExecuteScenequeue init failed.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 bitbake/lib/bb/runqueue.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index e13dc57..c7b1dc0 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1046,7 +1046,12 @@ class RunQueue:
                 self.state = runQueueComplete
             else:
                 self.start_worker()
-                self.rqexe = RunQueueExecuteScenequeue(self)
+                try:
+                    self.rqexe = RunQueueExecuteScenequeue(self)
+                except:
+                    # The RunQueueExecuteScenequeue init failure
+                    # terminated the build safely
+                    self.state = runQueueComplete
 
         if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
             self.dm.check(self)
-- 
1.9.1