[PATCH] bitbake-worker: Add DryRun event

[PATCH] bitbake-worker: Add DryRun event

[Mark Hatle]

There may be actions that a user wants to take, but the information is
only available during task execution.  Using the DryRun event you can prevent
the actual task execution, but get access to the associated data stores.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 bin/bitbake-worker | 2 ++
 lib/bb/event.py    | 5 +++++
 2 files changed, 7 insertions(+)

diff --git a/bin/bitbake-worker b/bin/bitbake-worker
index e925054b..e27fe7ac 100755
--- a/bin/bitbake-worker
+++ b/bin/bitbake-worker
@@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder, workerdata, fn, task, taskname, append
                 os._exit(1)
             try:
                 if dry_run:
+                    localdata = bb.build._task_data(fn, taskname, the_data)
+                    bb.event.fire(bb.event.DryRun(), localdata)
                     return 0
                 return bb.build.exec_task(fn, taskname, the_data, cfg.profile)
             except:
diff --git a/lib/bb/event.py b/lib/bb/event.py
index 5b1b094a..c03541a1 100644
--- a/lib/bb/event.py
+++ b/lib/bb/event.py
@@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
     def __init__(self, result):
         Event.__init__(self)
         self.result = result
+
+class DryRun(Event):
+    """
+    Event to fire during a dry run
+    """
-- 
2.16.0.rc2

Re: [PATCH] bitbake-worker: Add DryRun event

[Richard Purdie]

On Mon, 2018-10-01 at 21:37 -0400, Mark Hatle wrote:
> There may be actions that a user wants to take, but the information
> is only available during task execution.  Using the DryRun event you
> can prevent the actual task execution, but get access to the
> associated data stores.
> 
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
>  bin/bitbake-worker | 2 ++
>  lib/bb/event.py    | 5 +++++
>  2 files changed, 7 insertions(+)
> 
> diff --git a/bin/bitbake-worker b/bin/bitbake-worker
> index e925054b..e27fe7ac 100755
> --- a/bin/bitbake-worker
> +++ b/bin/bitbake-worker
> @@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder,
> workerdata, fn, task, taskname, append
>                  os._exit(1)
>              try:
>                  if dry_run:
> +                    localdata = bb.build._task_data(fn, taskname,
> the_data)
> +                    bb.event.fire(bb.event.DryRun(), localdata)
>                      return 0
>                  return bb.build.exec_task(fn, taskname, the_data,
> cfg.profile)
>              except:
> diff --git a/lib/bb/event.py b/lib/bb/event.py
> index 5b1b094a..c03541a1 100644
> --- a/lib/bb/event.py
> +++ b/lib/bb/event.py
> @@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
>      def __init__(self, result):
>          Event.__init__(self)
>          self.result = result
> +
> +class DryRun(Event):
> +    """
> +    Event to fire during a dry run
> +    """

I think we can add something like this but the DryRun event should
really be in build.py and subclass TaskBase, called something like
TaskDryRun?

I'm hoping that doesn't upset some of the other code that looks at
TaskBase events but its certainly logically where the dryrun event
belongs.

We may want to pass dry_run into exec_task as a parameter and put this
code at the top of exec_task to make things cleaner across the files?

Cheers,

Richard

Re: [bitbake-devel] [PATCH] bitbake-worker: Add DryRun event

[Randy MacLeod]

On 2018-10-04 4:57 a.m., Richard Purdie wrote:
> On Mon, 2018-10-01 at 21:37 -0400, Mark Hatle wrote:
>> There may be actions that a user wants to take, but the information
>> is only available during task execution.  Using the DryRun event you
>> can prevent the actual task execution, but get access to the
>> associated data stores.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>>   bin/bitbake-worker | 2 ++
>>   lib/bb/event.py    | 5 +++++
>>   2 files changed, 7 insertions(+)
>>
>> diff --git a/bin/bitbake-worker b/bin/bitbake-worker
>> index e925054b..e27fe7ac 100755
>> --- a/bin/bitbake-worker
>> +++ b/bin/bitbake-worker
>> @@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder,
>> workerdata, fn, task, taskname, append
>>                   os._exit(1)
>>               try:
>>                   if dry_run:
>> +                    localdata = bb.build._task_data(fn, taskname,
>> the_data)
>> +                    bb.event.fire(bb.event.DryRun(), localdata)
>>                       return 0
>>                   return bb.build.exec_task(fn, taskname, the_data,
>> cfg.profile)
>>               except:
>> diff --git a/lib/bb/event.py b/lib/bb/event.py
>> index 5b1b094a..c03541a1 100644
>> --- a/lib/bb/event.py
>> +++ b/lib/bb/event.py
>> @@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
>>       def __init__(self, result):
>>           Event.__init__(self)
>>           self.result = result
>> +
>> +class DryRun(Event):
>> +    """
>> +    Event to fire during a dry run
>> +    """
> 
> I think we can add something like this but the DryRun event should
> really be in build.py and subclass TaskBase, called something like
> TaskDryRun?
> 
> I'm hoping that doesn't upset some of the other code that looks at
> TaskBase events but its certainly logically where the dryrun event
> belongs.
> 
> We may want to pass dry_run into exec_task as a parameter and put this
> code at the top of exec_task to make things cleaner across the files?
> 
> Cheers,
> 
> Richard
> 


Hey Mark,

I was looking at local WR patches that we carry and noticed this one.

When I asked you privately about it in 2018, you said:

 > Randy said:
>>

>> Isn't this always firing?

>> It's mostly harmless but it's no longer really a completely dry-run

>> if you're logging PR info. Do you think anyone will object or

>> have you already discussed it with upstream folks?

>>

>> Why not add a new flag?

>
> If there is no event handler, then firing takes nearly no time at all.. I've

> sent the change to bitbake.. We'll see if RP or others have comments.

> 

> The change itself though is designed to be generic, in that we want any possible

> event handlers to work through this -- as it could be useful for collecting

> statistics and such as well -- without running a build.

> 

> --Mark

> 


so do you or Robert still care about this ?

I don't see it mentioned in the WR docs but
now that I think about it, it could be useful in a few use cases.
The one that was mentioned in the internal email was:

> Typically populating PR service takes a full build.  This is because the

> do_package task is what actually assigns the PR service values.  So we want

> a way that will allow us to generate the PR service values in significantly

> less time then a full build.. (minutes vs hours)



-- 
# Randy MacLeod
# Wind River Linux

Re: [bitbake-devel] [PATCH] bitbake-worker: Add DryRun event

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 4342 bytes --]

Use Mark's current email address.
../Randy

On 2021-12-03 1:55 p.m., Randy MacLeod wrote:
> On 2018-10-04 4:57 a.m., Richard Purdie wrote:
>> On Mon, 2018-10-01 at 21:37 -0400, Mark Hatle wrote:
>>> There may be actions that a user wants to take, but the information
>>> is only available during task execution.  Using the DryRun event you
>>> can prevent the actual task execution, but get access to the
>>> associated data stores.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>> ---
>>>   bin/bitbake-worker | 2 ++
>>>   lib/bb/event.py    | 5 +++++
>>>   2 files changed, 7 insertions(+)
>>>
>>> diff --git a/bin/bitbake-worker b/bin/bitbake-worker
>>> index e925054b..e27fe7ac 100755
>>> --- a/bin/bitbake-worker
>>> +++ b/bin/bitbake-worker
>>> @@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder,
>>> workerdata, fn, task, taskname, append
>>>                   os._exit(1)
>>>               try:
>>>                   if dry_run:
>>> +                    localdata = bb.build._task_data(fn, taskname,
>>> the_data)
>>> +                    bb.event.fire(bb.event.DryRun(), localdata)
>>>                       return 0
>>>                   return bb.build.exec_task(fn, taskname, the_data,
>>> cfg.profile)
>>>               except:
>>> diff --git a/lib/bb/event.py b/lib/bb/event.py
>>> index 5b1b094a..c03541a1 100644
>>> --- a/lib/bb/event.py
>>> +++ b/lib/bb/event.py
>>> @@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
>>>       def __init__(self, result):
>>>           Event.__init__(self)
>>>           self.result = result
>>> +
>>> +class DryRun(Event):
>>> +    """
>>> +    Event to fire during a dry run
>>> +    """
>>
>> I think we can add something like this but the DryRun event should
>> really be in build.py and subclass TaskBase, called something like
>> TaskDryRun?
>>
>> I'm hoping that doesn't upset some of the other code that looks at
>> TaskBase events but its certainly logically where the dryrun event
>> belongs.
>>
>> We may want to pass dry_run into exec_task as a parameter and put this
>> code at the top of exec_task to make things cleaner across the files?
>>
>> Cheers,
>>
>> Richard
>>
>
>
> Hey Mark,
>
> I was looking at local WR patches that we carry and noticed this one.
>
> When I asked you privately about it in 2018, you said:
>
> > Randy said:
>>>
>
>>> Isn't this always firing?
>
>>> It's mostly harmless but it's no longer really a completely dry-run
>
>>> if you're logging PR info. Do you think anyone will object or
>
>>> have you already discussed it with upstream folks?
>
>>>
>
>>> Why not add a new flag?
>
>>
>> If there is no event handler, then firing takes nearly no time at 
>> all.. I've
>
>> sent the change to bitbake.. We'll see if RP or others have comments.
>
>>
>
>> The change itself though is designed to be generic, in that we want 
>> any possible
>
>> event handlers to work through this -- as it could be useful for 
>> collecting
>
>> statistics and such as well -- without running a build.
>
>>
>
>> --Mark
>
>>
>
>
> so do you or Robert still care about this ?
>
> I don't see it mentioned in the WR docs but
> now that I think about it, it could be useful in a few use cases.
> The one that was mentioned in the internal email was:
>
>> Typically populating PR service takes a full build.  This is because the
>
>> do_package task is what actually assigns the PR service values.  So 
>> we want
>
>> a way that will allow us to generate the PR service values in 
>> significantly
>
>> less time then a full build.. (minutes vs hours)
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#13130): https://lists.openembedded.org/g/bitbake-devel/message/13130
> Mute This Topic: https://lists.openembedded.org/mt/72130803/3616765
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux


[-- Attachment #2: Type: text/html, Size: 9445 bytes --]