[master][zeus][PATCH] uihelper: Track scene tasks again

[master][zeus][PATCH] uihelper: Track scene tasks again

[Peter Kjellerstedt]

Track both scene tasks and run tasks to provide a complete progress
information for all tasks. This avoids the progress bar from stating
"Currently XX running tasks (0 of 0)" while progressing through
thousands of setscene tasks, obviously confusing the user.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 bitbake/lib/bb/ui/uihelper.py | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/ui/uihelper.py b/bitbake/lib/bb/ui/uihelper.py
index 48d808ae28..ac4ac5a882 100644
--- a/bitbake/lib/bb/ui/uihelper.py
+++ b/bitbake/lib/bb/ui/uihelper.py
@@ -16,6 +16,10 @@ class BBUIHelper:
         self.running_pids = []
         self.failed_tasks = []
         self.pidmap = {}
+        self.scenetasks_current = 0
+        self.scenetasks_total = 0
+        self.runtasks_current = 0
+        self.runtasks_total = 0
         self.tasknumber_current = 0
         self.tasknumber_total = 0
 
@@ -49,9 +53,17 @@ class BBUIHelper:
             tid = event._fn + ":" + event._task
             removetid(event.pid, tid)
             self.failed_tasks.append( { 'title' : "%s %s" % (event._package, event._task)})
+        elif isinstance(event, bb.runqueue.sceneQueueTaskStarted):
+            self.scenetasks_current = event.stats.completed + event.stats.active + event.stats.failed + 1
+            self.scenetasks_total = event.stats.total
+            self.tasknumber_current = self.scenetasks_current + self.runtasks_current
+            self.tasknumber_total = self.scenetasks_total + self.runtasks_total
+            self.needUpdate = True
         elif isinstance(event, bb.runqueue.runQueueTaskStarted):
-            self.tasknumber_current = event.stats.completed + event.stats.active + event.stats.failed + 1
-            self.tasknumber_total = event.stats.total
+            self.runtasks_current = event.stats.completed + event.stats.active + event.stats.failed + 1
+            self.runtasks_total = event.stats.total
+            self.tasknumber_current = self.scenetasks_current + self.runtasks_current
+            self.tasknumber_total = self.scenetasks_total + self.runtasks_total
             self.needUpdate = True
         elif isinstance(event, bb.build.TaskProgress):
             if event.pid > 0 and event.pid in self.pidmap:
-- 
2.21.1

Re: [master][zeus][PATCH] uihelper: Track scene tasks again

[Richard Purdie]

On Sun, 2020-02-09 at 09:50 +0100, Peter Kjellerstedt wrote:
> Track both scene tasks and run tasks to provide a complete progress
> information for all tasks. This avoids the progress bar from stating
> "Currently XX running tasks (0 of 0)" while progressing through
> thousands of setscene tasks, obviously confusing the user.

There was a reason this was removed, this can't work unfortunately. 

The code was changed such that setscene tasks can potentially be rerun
so we have no idea what "complete" looks like or how many will actually
run. I'm not happy about that but there wasn't an alternative.

Cheers,

Richard

Re: [master][zeus][PATCH] uihelper: Track scene tasks again

[Peter Kjellerstedt]

> -----Original Message-----
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Sent: den 10 februari 2020 15:18
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; bitbake-
> devel@lists.openembedded.org
> Subject: Re: [bitbake-devel] [master][zeus][PATCH] uihelper: Track scene
> tasks again
> 
> On Sun, 2020-02-09 at 09:50 +0100, Peter Kjellerstedt wrote:
> > Track both scene tasks and run tasks to provide a complete progress
> > information for all tasks. This avoids the progress bar from stating
> > "Currently XX running tasks (0 of 0)" while progressing through
> > thousands of setscene tasks, obviously confusing the user.
> 
> There was a reason this was removed, this can't work unfortunately.
> 
> The code was changed such that setscene tasks can potentially be rerun
> so we have no idea what "complete" looks like or how many will actually
> run. I'm not happy about that but there wasn't an alternative.
> 
> Cheers,
> 
> Richard

If a setscene task is rerun, won't that just mean that the counters 
don't change due to this? So in this case there won't be any progress 
to the total progress bar even if a task is running? That should still 
be better than the current situation when they are never counted.

//Peter

Re: [master][zeus][PATCH] uihelper: Track scene tasks again

[Richard Purdie]

On Mon, 2020-02-10 at 15:01 +0000, Peter Kjellerstedt wrote:
> If a setscene task is rerun, won't that just mean that the counters 
> don't change due to this? So in this case there won't be any
> progress  to the total progress bar even if a task is running? That
> should still  be better than the current situation when they are
> never counted.

No, that isn't how the code is written. You'd see "task 1000 of 500".
We got some pretty horrible numbers when testing this originally, I got
lots of complaints and its why it was removed.

The code has changed a bit since and we probably don't rerun as much
but we get complains when we print any invalid info...

Cheers,

Richard

Re: [master][zeus][PATCH] uihelper: Track scene tasks again

[Peter Kjellerstedt]

> -----Original Message-----
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Sent: den 10 februari 2020 16:18
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; bitbake-
> devel@lists.openembedded.org
> Subject: Re: [bitbake-devel] [master][zeus][PATCH] uihelper: Track scene
> tasks again
> 
> On Mon, 2020-02-10 at 15:01 +0000, Peter Kjellerstedt wrote:
> > If a setscene task is rerun, won't that just mean that the counters
> > don't change due to this? So in this case there won't be any
> > progress  to the total progress bar even if a task is running? That
> > should still  be better than the current situation when they are
> > never counted.
> 
> No, that isn't how the code is written. You'd see "task 1000 of 500".
> We got some pretty horrible numbers when testing this originally, I got
> lots of complaints and its why it was removed.
> 
> The code has changed a bit since and we probably don't rerun as much
> but we get complains when we print any invalid info...
> 
> Cheers,
> 
> Richard

Hmm, ok. I get loud complaints that we are printing "(0 of 0)" while 
thousands of setscene tasks are being processed, so that is unfortunately 
not much better... 

Would it be possible to change the accounting so that we do not end 
up with "task 1000 of 500"?

For the record, we are not using the hash equivalence server, so I 
guess we should not have any setscene tasks that are being rerun, but I 
have a feeling you do not want to treat the two cases differently.

//Peter