* [PATCH 0/4] Progress fixes
@ 2016-08-11 3:36 Paul Eggleton
2016-08-11 3:36 ` [PATCH 1/4] runqueue: fix two minor issues with the initialising tasks progress Paul Eggleton
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Paul Eggleton @ 2016-08-11 3:36 UTC (permalink / raw)
To: bitbake-devel
Fixes for minor issues in the progress bar display, plus a patch for
some unclosed files that I noticed at the same time as working on these.
The following changes since commit 0a9b5d7d9655dbb09d458fc6e330e932f0f9dab6:
toaster: buildinfohelper Add handling local layers (i.e. non-git) layers (2016-08-11 00:08:18 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib paule/bb-progress-fixes2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bb-progress-fixes2
Paul Eggleton (4):
runqueue: fix two minor issues with the initialising tasks progress
knotty: fix task progress bar not starting at 0%
knotty: don't show number of running tasks in quiet mode
siggen: properly close files rather than opening them inline
lib/bb/runqueue.py | 5 +++--
lib/bb/siggen.py | 15 +++++++++------
lib/bb/ui/knotty.py | 11 ++++++++---
3 files changed, 20 insertions(+), 11 deletions(-)
--
2.5.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/4] runqueue: fix two minor issues with the initialising tasks progress
2016-08-11 3:36 [PATCH 0/4] Progress fixes Paul Eggleton
@ 2016-08-11 3:36 ` Paul Eggleton
2016-08-11 3:36 ` [PATCH 2/4] knotty: fix task progress bar not starting at 0% Paul Eggleton
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Paul Eggleton @ 2016-08-11 3:36 UTC (permalink / raw)
To: bitbake-devel
A couple of fixes for the "Initialising tasks" progress bar behaviour:
* Properly finish the progress bar when using bitbake -S
* Finish the progress bar before calling BB_HASHCHECK_FUNCTION (so that
in OE when that shows its own "Checking sstate mirror object
availability" progress bar it gets shown on the next line as it
should).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
lib/bb/runqueue.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index aa939d0..3a593b6 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -1148,6 +1148,7 @@ class RunQueue:
if self.state is runQueueSceneInit:
dump = self.cooker.configuration.dump_signatures
if dump:
+ self.rqdata.init_progress_reporter.finish()
if 'printdiff' in dump:
invalidtasks = self.print_diffscenetasks()
self.dump_signatures(dump)
@@ -1967,6 +1968,8 @@ class RunQueueExecuteScenequeue(RunQueueExecute):
if len(self.sq_revdeps[tid]) == 0:
self.runq_buildable.add(tid)
+ self.rqdata.init_progress_reporter.finish()
+
self.outrightfail = []
if self.rq.hashvalidate:
sq_hash = []
@@ -2018,8 +2021,6 @@ class RunQueueExecuteScenequeue(RunQueueExecute):
logger.debug(2, 'No package found, so skipping setscene task %s', tid)
self.outrightfail.append(tid)
- self.rqdata.init_progress_reporter.finish()
-
logger.info('Executing SetScene Tasks')
self.rq.state = runQueueSceneRun
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/4] knotty: fix task progress bar not starting at 0%
2016-08-11 3:36 [PATCH 0/4] Progress fixes Paul Eggleton
2016-08-11 3:36 ` [PATCH 1/4] runqueue: fix two minor issues with the initialising tasks progress Paul Eggleton
@ 2016-08-11 3:36 ` Paul Eggleton
2016-08-11 3:36 ` [PATCH 3/4] knotty: don't show number of running tasks in quiet mode Paul Eggleton
2016-08-11 3:37 ` [PATCH 4/4] siggen: properly close files rather than opening them inline Paul Eggleton
3 siblings, 0 replies; 7+ messages in thread
From: Paul Eggleton @ 2016-08-11 3:36 UTC (permalink / raw)
To: bitbake-devel
If we have the task number here we need to subtract 1 to get the number
of tasks completed.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
lib/bb/ui/knotty.py | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index 1723a72..f89f7b3 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -260,13 +260,16 @@ class TerminalFilter(object):
content = "No currently running tasks (%s of %s)" % (self.helper.tasknumber_current, self.helper.tasknumber_total)
else:
content = "Currently %2s running tasks (%s of %s)" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total)
- maxtask = self.helper.tasknumber_total + 1
+ maxtask = self.helper.tasknumber_total
if not self.main_progress or self.main_progress.maxval != maxtask:
widgets = [' ', progressbar.Percentage(), ' ', progressbar.Bar()]
self.main_progress = BBProgress("Running tasks", maxtask, widgets=widgets)
self.main_progress.start(False)
self.main_progress.setmessage(content)
- self.main_progress.update(self.helper.tasknumber_current)
+ progress = self.helper.tasknumber_current - 1
+ if progress < 0:
+ progress = 0
+ self.main_progress.update(progress)
print('')
lines = 1 + int(len(content) / (self.columns + 1))
if not self.quiet:
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/4] knotty: don't show number of running tasks in quiet mode
2016-08-11 3:36 [PATCH 0/4] Progress fixes Paul Eggleton
2016-08-11 3:36 ` [PATCH 1/4] runqueue: fix two minor issues with the initialising tasks progress Paul Eggleton
2016-08-11 3:36 ` [PATCH 2/4] knotty: fix task progress bar not starting at 0% Paul Eggleton
@ 2016-08-11 3:36 ` Paul Eggleton
2016-08-11 3:37 ` [PATCH 4/4] siggen: properly close files rather than opening them inline Paul Eggleton
3 siblings, 0 replies; 7+ messages in thread
From: Paul Eggleton @ 2016-08-11 3:36 UTC (permalink / raw)
To: bitbake-devel
There's not a whole lot of point showing how many tasks are running when
we're in quiet mode, it just looks a bit strange particularly when it's
not running any tasks.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
lib/bb/ui/knotty.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index f89f7b3..b30135b 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -256,7 +256,9 @@ class TerminalFilter(object):
content = "Waiting for %s running tasks to finish:" % len(activetasks)
print(content)
else:
- if not len(activetasks):
+ if self.quiet:
+ content = "Running tasks (%s of %s)" % (self.helper.tasknumber_current, self.helper.tasknumber_total)
+ elif not len(activetasks):
content = "No currently running tasks (%s of %s)" % (self.helper.tasknumber_current, self.helper.tasknumber_total)
else:
content = "Currently %2s running tasks (%s of %s)" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total)
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/4] siggen: properly close files rather than opening them inline
2016-08-11 3:36 [PATCH 0/4] Progress fixes Paul Eggleton
` (2 preceding siblings ...)
2016-08-11 3:36 ` [PATCH 3/4] knotty: don't show number of running tasks in quiet mode Paul Eggleton
@ 2016-08-11 3:37 ` Paul Eggleton
2016-08-16 21:16 ` Peter Kjellerstedt
3 siblings, 1 reply; 7+ messages in thread
From: Paul Eggleton @ 2016-08-11 3:37 UTC (permalink / raw)
To: bitbake-devel
If you don't do this, with Python 3 you get a warning on exit under some
circumstances.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
lib/bb/siggen.py | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py
index 5d8a253..9b2f658 100644
--- a/lib/bb/siggen.py
+++ b/lib/bb/siggen.py
@@ -363,10 +363,12 @@ def clean_basepaths_list(a):
def compare_sigfiles(a, b, recursecb = None):
output = []
- p1 = pickle.Unpickler(open(a, "rb"))
- a_data = p1.load()
- p2 = pickle.Unpickler(open(b, "rb"))
- b_data = p2.load()
+ with open(a, 'rb') as f:
+ p1 = pickle.Unpickler(ff)
+ a_data = p1.load()
+ with open(b, 'rb') as f:
+ p2 = pickle.Unpickler(f)
+ b_data = p2.load()
def dict_diff(a, b, whitelist=set()):
sa = set(a.keys())
@@ -563,8 +565,9 @@ def calc_taskhash(sigdata):
def dump_sigfile(a):
output = []
- p1 = pickle.Unpickler(open(a, "rb"))
- a_data = p1.load()
+ with open(a, 'rb') as f:
+ p1 = pickle.Unpickler(f)
+ a_data = p1.load()
output.append("basewhitelist: %s" % (a_data['basewhitelist']))
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 4/4] siggen: properly close files rather than opening them inline
2016-08-11 3:37 ` [PATCH 4/4] siggen: properly close files rather than opening them inline Paul Eggleton
@ 2016-08-16 21:16 ` Peter Kjellerstedt
2016-08-16 22:30 ` Paul Eggleton
0 siblings, 1 reply; 7+ messages in thread
From: Peter Kjellerstedt @ 2016-08-16 21:16 UTC (permalink / raw)
To: Paul Eggleton, bitbake-devel
> -----Original Message-----
> From: bitbake-devel-bounces@lists.openembedded.org [mailto:bitbake-
> devel-bounces@lists.openembedded.org] On Behalf Of Paul Eggleton
> Sent: den 11 augusti 2016 05:37
> To: bitbake-devel@lists.openembedded.org
> Subject: [bitbake-devel] [PATCH 4/4] siggen: properly close files
> rather than opening them inline
>
> If you don't do this, with Python 3 you get a warning on exit under some
> circumstances.
>
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> ---
> lib/bb/siggen.py | 15 +++++++++------
> 1 file changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py
> index 5d8a253..9b2f658 100644
> --- a/lib/bb/siggen.py
> +++ b/lib/bb/siggen.py
> @@ -363,10 +363,12 @@ def clean_basepaths_list(a):
> def compare_sigfiles(a, b, recursecb = None):
> output = []
>
> - p1 = pickle.Unpickler(open(a, "rb"))
> - a_data = p1.load()
> - p2 = pickle.Unpickler(open(b, "rb"))
> - b_data = p2.load()
> + with open(a, 'rb') as f:
> + p1 = pickle.Unpickler(ff)
^^
Shouldn't that be:
p1 = pickle.Unpickler(f)
> + a_data = p1.load()
> + with open(b, 'rb') as f:
> + p2 = pickle.Unpickler(f)
> + b_data = p2.load()
>
> def dict_diff(a, b, whitelist=set()):
> sa = set(a.keys())
> @@ -563,8 +565,9 @@ def calc_taskhash(sigdata):
> def dump_sigfile(a):
> output = []
>
> - p1 = pickle.Unpickler(open(a, "rb"))
> - a_data = p1.load()
> + with open(a, 'rb') as f:
> + p1 = pickle.Unpickler(f)
> + a_data = p1.load()
>
> output.append("basewhitelist: %s" % (a_data['basewhitelist']))
>
> --
> 2.5.5
//Peter
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 4/4] siggen: properly close files rather than opening them inline
2016-08-16 21:16 ` Peter Kjellerstedt
@ 2016-08-16 22:30 ` Paul Eggleton
0 siblings, 0 replies; 7+ messages in thread
From: Paul Eggleton @ 2016-08-16 22:30 UTC (permalink / raw)
To: Peter Kjellerstedt; +Cc: bitbake-devel
On Tue, 16 Aug 2016 21:16:17 Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: bitbake-devel-bounces@lists.openembedded.org [mailto:bitbake-
> > devel-bounces@lists.openembedded.org] On Behalf Of Paul Eggleton
> > Sent: den 11 augusti 2016 05:37
> > To: bitbake-devel@lists.openembedded.org
> > Subject: [bitbake-devel] [PATCH 4/4] siggen: properly close files
> > rather than opening them inline
> >
> > If you don't do this, with Python 3 you get a warning on exit under some
> > circumstances.
> >
> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> > ---
> >
> > lib/bb/siggen.py | 15 +++++++++------
> > 1 file changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py
> > index 5d8a253..9b2f658 100644
> > --- a/lib/bb/siggen.py
> > +++ b/lib/bb/siggen.py
> >
> > @@ -363,10 +363,12 @@ def clean_basepaths_list(a):
> > def compare_sigfiles(a, b, recursecb = None):
> > output = []
> >
> > - p1 = pickle.Unpickler(open(a, "rb"))
> > - a_data = p1.load()
> > - p2 = pickle.Unpickler(open(b, "rb"))
> > - b_data = p2.load()
> > + with open(a, 'rb') as f:
> > + p1 = pickle.Unpickler(ff)
>
> ^^
> Shouldn't that be:
>
> p1 = pickle.Unpickler(f)
>
Oops - yes it should be. I guess it just proves that even "simple, obvious"
changes like this one do actually need to be tested. I'll send a v2.
Thanks,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-08-16 22:30 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-11 3:36 [PATCH 0/4] Progress fixes Paul Eggleton
2016-08-11 3:36 ` [PATCH 1/4] runqueue: fix two minor issues with the initialising tasks progress Paul Eggleton
2016-08-11 3:36 ` [PATCH 2/4] knotty: fix task progress bar not starting at 0% Paul Eggleton
2016-08-11 3:36 ` [PATCH 3/4] knotty: don't show number of running tasks in quiet mode Paul Eggleton
2016-08-11 3:37 ` [PATCH 4/4] siggen: properly close files rather than opening them inline Paul Eggleton
2016-08-16 21:16 ` Peter Kjellerstedt
2016-08-16 22:30 ` Paul Eggleton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.