* [yocto autobuilder][PATCH 0/3] Fix buildlogger
@ 2016-09-07 15:33 Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 1/3] buildlogger: remove old, incomplete, comment Joshua Lock
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Joshua Lock @ 2016-09-07 15:33 UTC (permalink / raw)
To: yocto
Since build 999 we haven't seen any builds logged to the BuildLogs wiki page,
that turns out to be because the JSON dicts keys are strings and thus the sort
order isn't as expected, i.e. 999 > 1001.
This series tidies up the buildlogger some and includes a fix for this issue by
converting the keys of the dicts to a list of ints and sorting that to determine
the most recent build.
The following changes since commit ba01fe19440a91091fe15544ecc2809a79f19401:
nightly-qa-systemd.conf: Add create_eventlog property (2016-08-31 21:22:11 +0100)
are available in the git repository at:
git://git.yoctoproject.org/yocto-autobuilder joshuagl/buildlogger
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/log/?h=joshuagl/buildlogger
Joshua Lock (3):
buildlogger: remove old, incomplete, comment
buildlogger: don't hardcode YP AB in generated URL
buildlogger: fix sorting of recent builds
bin/buildlogger | 23 +++++++++++++----------
etc/buildlogger.conf.example | 1 +
2 files changed, 14 insertions(+), 10 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 5+ messages in thread
* [yocto autobuilder][PATCH 1/3] buildlogger: remove old, incomplete, comment
2016-09-07 15:33 [yocto autobuilder][PATCH 0/3] Fix buildlogger Joshua Lock
@ 2016-09-07 15:33 ` Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 2/3] buildlogger: don't hardcode YP AB in generated URL Joshua Lock
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Joshua Lock @ 2016-09-07 15:33 UTC (permalink / raw)
To: yocto
The comment isn't complete and doesn't make much sense.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
bin/buildlogger | 6 ------
1 file changed, 6 deletions(-)
diff --git a/bin/buildlogger b/bin/buildlogger
index 97f67ec..7a529c0 100755
--- a/bin/buildlogger
+++ b/bin/buildlogger
@@ -11,12 +11,6 @@ __maintainer__ = "Joshua Lock"
__email__ = "joshua.g.lock@intel.com"
'''
-# We'd probably benefit from using some caching, but first we'd need the AB API
-# to include
-#
-# We can set repo url, branch & commit for a bunch of repositorys.
-# Do they all get built for nightly?
-
import codecs
try:
import configparser
--
2.7.4
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [yocto autobuilder][PATCH 2/3] buildlogger: don't hardcode YP AB in generated URL
2016-09-07 15:33 [yocto autobuilder][PATCH 0/3] Fix buildlogger Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 1/3] buildlogger: remove old, incomplete, comment Joshua Lock
@ 2016-09-07 15:33 ` Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 3/3] buildlogger: fix sorting of recent builds Joshua Lock
2016-09-08 12:07 ` [yocto autobuilder][PATCH 0/3] Fix buildlogger Beth 'pidge' Flanagan
3 siblings, 0 replies; 5+ messages in thread
From: Joshua Lock @ 2016-09-07 15:33 UTC (permalink / raw)
To: yocto
When generating text for a BuildLog wiki entry we were hard-coding
the Yocto Project AB URL in the entry's title, rather than that
add a new variable to buildlogger.conf and use the value of this
variable to construct the URL in the link.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
bin/buildlogger | 7 +++++--
etc/buildlogger.conf.example | 1 +
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/bin/buildlogger b/bin/buildlogger
index 7a529c0..1b31281 100755
--- a/bin/buildlogger
+++ b/bin/buildlogger
@@ -30,6 +30,7 @@ except ImportError:
sys.exit(1)
abapi = ''
+abwww = ''
# Wiki editing params
un = ''
pw = ''
@@ -49,6 +50,7 @@ def load_config(configfile):
global title
global wikiapi
global abapi
+ global abwww
success = False
if os.path.exists(configfile):
@@ -59,6 +61,7 @@ def load_config(configfile):
pw = config.get('wiki', 'password')
title = config.get('wiki', 'pagetitle')
wikiapi = config.get('wiki', 'apiuri')
+ abwww = config.get('builder', 'wwwuri')
abapi = config.get('builder', 'apiuri')
success = True
except configparser.Error as ex:
@@ -195,8 +198,8 @@ def ab_last_build_to_entry(build_json, build_id):
if prop[0] == 'commit_poky':
chash = prop[1]
- urlfmt = 'https://autobuilder.yoctoproject.org/main/builders/%s/builds/%s/'
- url = urlfmt % (builder, buildid)
+ urlfmt = '%s/builders/%s/builds/%s/'
+ url = urlfmt % (abwww, builder, buildid)
sectionfmt = '==[%s %s %s - %s %s]=='
section_title = sectionfmt % (url, builder, buildid, buildbranch, chash)
summaryfmt = 'Adding new BuildLog entry for build %s (%s)'
diff --git a/etc/buildlogger.conf.example b/etc/buildlogger.conf.example
index 459961c..330af56 100644
--- a/etc/buildlogger.conf.example
+++ b/etc/buildlogger.conf.example
@@ -5,4 +5,5 @@ pagetitle = BuildLogPageTitle
apiuri = https://wiki.yoctoproject.org/wiki/api.php
[builder]
+wwwuri = https://autobuilder.yoctoproject.org/main
apiuri = https://autobuilder.yoctoproject.org/main/json/builders/nightly/builds/_all
--
2.7.4
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [yocto autobuilder][PATCH 3/3] buildlogger: fix sorting of recent builds
2016-09-07 15:33 [yocto autobuilder][PATCH 0/3] Fix buildlogger Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 1/3] buildlogger: remove old, incomplete, comment Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 2/3] buildlogger: don't hardcode YP AB in generated URL Joshua Lock
@ 2016-09-07 15:33 ` Joshua Lock
2016-09-08 12:07 ` [yocto autobuilder][PATCH 0/3] Fix buildlogger Beth 'pidge' Flanagan
3 siblings, 0 replies; 5+ messages in thread
From: Joshua Lock @ 2016-09-07 15:33 UTC (permalink / raw)
To: yocto
Recent builds, >999, on the YP AutoBuilder were not being logged.
This is because we sort the builds based on a string representation
of their build number, yet string 1001 < string 999.
Resolve this by converting the build id's to an int, sorting them
and then determining which is the highest/most recent build id.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
bin/buildlogger | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/bin/buildlogger b/bin/buildlogger
index 1b31281..912f419 100755
--- a/bin/buildlogger
+++ b/bin/buildlogger
@@ -219,7 +219,7 @@ def write_last_build(buildid):
# Read last logged buildid from a file
def read_last_build():
- last_build = ''
+ last_build = '0'
try:
with open(cachefile, 'r') as fi:
last_build = fi.readline()
@@ -254,7 +254,13 @@ def watch_for_builds(configfile):
print("Failed to decode JSON: %s" % str(e))
continue
- last_build = sorted(build_json.keys())[-1]
+ # The keys of the JSON dict are unicode strings, which makes
+ # sorting to find the most recent difficult (i.e. for a string type
+ # 999 > 1001). Convert them to ints for a simpler sort.
+ buildids = map(int, build_json.keys())
+ buildids.sort()
+ last_build = str(buildids[-1])
+
# If a new build is detected, post a new entry to the BuildLog
if last_build != last_logged:
new_entry, summary = ab_last_build_to_entry(build_json, last_build)
--
2.7.4
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [yocto autobuilder][PATCH 0/3] Fix buildlogger
2016-09-07 15:33 [yocto autobuilder][PATCH 0/3] Fix buildlogger Joshua Lock
` (2 preceding siblings ...)
2016-09-07 15:33 ` [yocto autobuilder][PATCH 3/3] buildlogger: fix sorting of recent builds Joshua Lock
@ 2016-09-08 12:07 ` Beth 'pidge' Flanagan
3 siblings, 0 replies; 5+ messages in thread
From: Beth 'pidge' Flanagan @ 2016-09-08 12:07 UTC (permalink / raw)
To: Joshua Lock, yocto
merged into master. Thanks!
-b
On Wed, 2016-09-07 at 16:33 +0100, Joshua Lock wrote:
> Since build 999 we haven't seen any builds logged to the BuildLogs
> wiki page,
> that turns out to be because the JSON dicts keys are strings and thus
> the sort
> order isn't as expected, i.e. 999 > 1001.
>
> This series tidies up the buildlogger some and includes a fix for
> this issue by
> converting the keys of the dicts to a list of ints and sorting that
> to determine
> the most recent build.
>
> The following changes since commit
> ba01fe19440a91091fe15544ecc2809a79f19401:
>
> nightly-qa-systemd.conf: Add create_eventlog property (2016-08-31
> 21:22:11 +0100)
>
> are available in the git repository at:
>
> git://git.yoctoproject.org/yocto-autobuilder joshuagl/buildlogger
> http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/log/?h=joshu
> agl/buildlogger
>
> Joshua Lock (3):
> buildlogger: remove old, incomplete, comment
> buildlogger: don't hardcode YP AB in generated URL
> buildlogger: fix sorting of recent builds
>
> bin/buildlogger | 23 +++++++++++++----------
> etc/buildlogger.conf.example | 1 +
> 2 files changed, 14 insertions(+), 10 deletions(-)
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-09-08 12:15 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-07 15:33 [yocto autobuilder][PATCH 0/3] Fix buildlogger Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 1/3] buildlogger: remove old, incomplete, comment Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 2/3] buildlogger: don't hardcode YP AB in generated URL Joshua Lock
2016-09-07 15:33 ` [yocto autobuilder][PATCH 3/3] buildlogger: fix sorting of recent builds Joshua Lock
2016-09-08 12:07 ` [yocto autobuilder][PATCH 0/3] Fix buildlogger Beth 'pidge' Flanagan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.