[PATCH 0/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[PATCH 0/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

The following changes since commit 52b788c6df9201764130ab744bf67b770b24b896:

  autogen-native: inherit pkgconfig to fix a build failure (2014-08-25 10:26:00 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib hongxu/bitbake
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/readonly-sstatecache

Hongxu Jia (1):
  bitbake: runqueue: terminate build safely while
    RunQueueExecuteScenequeue init failed

 bitbake/lib/bb/runqueue.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

-- 
1.9.1

[PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

While RunQueueExecuteScenequeue init failed, the exit of build is mess.
Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
...
Traceback (most recent call last):
  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py", line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue instance at 0x7f87dd7d5050>):
                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname, "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
    >            valid = bb.utils.better_eval(call, locs)

  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/utils.py", line 374, in better_eval(source='sstate_checkhashes(sq_fn, sq_task, sq_hash, sq_hashfn, d)', locals={'sq_fn': ['virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb'], 'sq_task': ['do_populate_sysroot', 'do_populate_lic', 'do_package_qa'], 'sq_hash': ['7d132e7705b5772accce5544cf785c70', '52387c5751c018ddfe2c6a5eecf59e36', '315bcde98908fd36442830f9ae7469c1'], 'sq_hashfn': ['Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:'], 'd': <bb.data_smart.DataSmart object at 0x7f87e15cde10>}):
     def better_eval(source, locals):
    >    return eval(source, get_context(), locals)

  File "<string>", line 1, in <module>
  File "sstate.bbclass", line 110, in sstate_checkhashes(sq_fn=['virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb', 'virtual:native:/home/jiahongxu/yocto/poky/meta/recipes-support/db/db_6.0.30.bb'], sq_task=['do_populate_sysroot', 'do_populate_lic', 'do_package_qa'], sq_hash=['7d132e7705b5772accce5544cf785c70', '52387c5751c018ddfe2c6a5eecf59e36', '315bcde98908fd36442830f9ae7469c1'], sq_hashfn=['Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:', 'Ubuntu-14.04/ sstate:db-native:x86_64-linux:6.0.30:r0:x86_64:3: sstate:db::6.0.30:r0::3:'], d=<bb.data_smart.DataSmart object at 0x7f87e15cde10>)
  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102, in fatal:
         logger.critical(''.join(args))
    >    raise BBHandledException()

BBHandledException
...

We should terminate build safely while RunQueueExecuteScenequeue init failed.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 bitbake/lib/bb/runqueue.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index e13dc57..c7b1dc0 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1046,7 +1046,12 @@ class RunQueue:
                 self.state = runQueueComplete
             else:
                 self.start_worker()
-                self.rqexe = RunQueueExecuteScenequeue(self)
+                try:
+                    self.rqexe = RunQueueExecuteScenequeue(self)
+                except:
+                    # The RunQueueExecuteScenequeue init failure
+                    # terminated the build safely
+                    self.state = runQueueComplete
 
         if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
             self.dm.check(self)
-- 
1.9.1

Re: [PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

On 08/27/2014 06:49 PM, Richard Purdie wrote:
> On Wed, 2014-08-27 at 18:44 +0800, Hongxu Jia wrote:
>> While RunQueueExecuteScenequeue init failed, the exit of build is mess.
>> Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
>> ...
>> Traceback (most recent call last):
>> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py",
>>    line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue
>>    instance at 0x7f87dd7d5050>):
>> |                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname,
>>    "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
>> |    >            valid = bb.utils.better_eval(call, locs)
>>
>> |    def better_eval(source, locals):
>> |    >    return eval(source, get_context(), locals)
>>
>> |  File "<string>", line 1, in <module>
>> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102,
>>    in fatal:
>> |         logger.critical(''.join(args))
>> |    >    raise BBHandledException()
>>
>> BBHandledException
>> ...
>>
>> We should terminate build safely while RunQueueExecuteScenequeue init failed.
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>> ---
>>   bitbake/lib/bb/runqueue.py | 7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
>> index e13dc57..c7b1dc0 100644
>> --- a/bitbake/lib/bb/runqueue.py
>> +++ b/bitbake/lib/bb/runqueue.py
>> @@ -1046,7 +1046,12 @@ class RunQueue:
>>                   self.state = runQueueComplete
>>               else:
>>                   self.start_worker()
>> -                self.rqexe = RunQueueExecuteScenequeue(self)
>> +                try:
>> +                    self.rqexe = RunQueueExecuteScenequeue(self)
>> +                except:
>> +                    # The RunQueueExecuteScenequeue init failure
>> +                    # terminated the build safely
>> +                    self.state = runQueueComplete
>>   
> If the init fails, shouldn't we show a traceback and exit with an error
> status? Does this happen in a normal build and if so, is there an
> underlying cause we should fix?

In my case, I want to catch the failure of self.rq.hashvalidate which
invoked in RunQueueExecuteScenequeue, and exit in a normal build.

>
> I'm worried this would let errors go unnoticed. Except clauses in
> general should catch specific errors not every exception so this really
> can't merge in this form, sorry.

Got it, I will add condition to catch the failure of self.rq.hashvalidate,
and don't affect other errors.

Thanks,
//Hongxu

> Cheers,
>
> Richard
>
>

Re: [PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Richard Purdie]

On Wed, 2014-08-27 at 18:44 +0800, Hongxu Jia wrote:
> While RunQueueExecuteScenequeue init failed, the exit of build is mess.
> Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
> ...
> Traceback (most recent call last):
> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py",
>   line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue
>   instance at 0x7f87dd7d5050>):
> |                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname,
>   "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
> |    >            valid = bb.utils.better_eval(call, locs)
> 
> |    def better_eval(source, locals):
> |    >    return eval(source, get_context(), locals)
> 
> |  File "<string>", line 1, in <module>
> |  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102,
>   in fatal:
> |         logger.critical(''.join(args))
> |    >    raise BBHandledException()
> 
> BBHandledException
> ...
> 
> We should terminate build safely while RunQueueExecuteScenequeue init failed.
> 
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  bitbake/lib/bb/runqueue.py | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> index e13dc57..c7b1dc0 100644
> --- a/bitbake/lib/bb/runqueue.py
> +++ b/bitbake/lib/bb/runqueue.py
> @@ -1046,7 +1046,12 @@ class RunQueue:
>                  self.state = runQueueComplete
>              else:
>                  self.start_worker()
> -                self.rqexe = RunQueueExecuteScenequeue(self)
> +                try:
> +                    self.rqexe = RunQueueExecuteScenequeue(self)
> +                except:
> +                    # The RunQueueExecuteScenequeue init failure
> +                    # terminated the build safely
> +                    self.state = runQueueComplete
>  

If the init fails, shouldn't we show a traceback and exit with an error
status? Does this happen in a normal build and if so, is there an
underlying cause we should fix?

I'm worried this would let errors go unnoticed. Except clauses in
general should catch specific errors not every exception so this really
can't merge in this form, sorry.

Cheers,

Richard

[PATCH 1/1] bitbake: runqueue: terminate build safely while RunQueueExecuteScenequeue init failed

[Hongxu Jia]

While RunQueueExecuteScenequeue init failed, the exit of build is mess.
Here is the example while bb.fatal invoked in RunQueueExecuteScenequeue:
...
Traceback (most recent call last):
|  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/runqueue.py",
  line 1824, in RunQueueExecuteScenequeue.__init__(rq=<bb.runqueue.RunQueue
  instance at 0x7f87dd7d5050>):
|                 locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname,
  "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
|    >            valid = bb.utils.better_eval(call, locs)

|    def better_eval(source, locals):
|    >    return eval(source, get_context(), locals)

|  File "<string>", line 1, in <module>
|  File "/home/jiahongxu/yocto/poky/bitbake/lib/bb/__init__.py", line 102,
  in fatal:
|         logger.critical(''.join(args))
|    >    raise BBHandledException()

BBHandledException
...

We should terminate build safely while RunQueueExecuteScenequeue init failed.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 bitbake/lib/bb/runqueue.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index e13dc57..c7b1dc0 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1046,7 +1046,12 @@ class RunQueue:
                 self.state = runQueueComplete
             else:
                 self.start_worker()
-                self.rqexe = RunQueueExecuteScenequeue(self)
+                try:
+                    self.rqexe = RunQueueExecuteScenequeue(self)
+                except:
+                    # The RunQueueExecuteScenequeue init failure
+                    # terminated the build safely
+                    self.state = runQueueComplete
 
         if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
             self.dm.check(self)
-- 
1.9.1