* [PATCH 0/7] Fixes to oe-build-perf-report
@ 2017-09-15 12:54 Markus Lehtonen
2017-09-15 12:54 ` [PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats Markus Lehtonen
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
A collection of miscellanious small fixes to scripts/oe-build-perf-report. No
changes in the output of the script, except for a more valid html syntax.
The following changes since commit d72d116e0228cc0e4391d0558b2d32c3fd5e399c:
README.qemu: Add from meta-yocto as it belongs in core (2017-09-14 12:39:29 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/fixes
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/fixes
Markus Lehtonen (7):
scripts/oe-build-perf-report: fix dumping buildstats
scripts/oe-build-perf-report: add AggregateTestData class
scripts/oe-build-perf-report: fix handling of --history-length
scripts/oe-build-perf-report: use --hostname in --list
scripts/oe-build-perf-report: remove dead code
scripts/oe-build-perf-report: provide valid title in the html report
scripts/oe-build-perf-report: tidy up html syntax
scripts/lib/build_perf/html/report.html | 36 ++++++++++++++++++---------------
scripts/lib/build_perf/report.py | 5 ++++-
scripts/oe-build-perf-report | 33 ++++++++++++++++--------------
3 files changed, 42 insertions(+), 32 deletions(-)
--
2.12.3
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
2017-09-15 12:54 ` [PATCH 2/7] scripts/oe-build-perf-report: add AggregateTestData class Markus Lehtonen
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Fix a misbehavior when some of the buildstats are missing.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/oe-build-perf-report | 1 +
1 file changed, 1 insertion(+)
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index b5ad42bc8a..23081db173 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -422,6 +422,7 @@ def dump_buildstats(repo, outdir, notes_ref, revs):
'show', tag + '^0']))
except GitError:
log.warning("Buildstats not found for %s", tag)
+ bs_all = {}
missing = True
for measurement, buildstats in bs_all.items():
tag_base, run_id = tag.rsplit('/', 1)
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/7] scripts/oe-build-perf-report: add AggregateTestData class
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
2017-09-15 12:54 ` [PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
2017-09-15 12:54 ` [PATCH 3/7] scripts/oe-build-perf-report: fix handling of --history-length Markus Lehtonen
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Making the code a bit more readable.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/lib/build_perf/report.py | 5 ++++-
scripts/oe-build-perf-report | 16 +++++++++-------
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/build_perf/report.py b/scripts/lib/build_perf/report.py
index eb00ccca2d..d99a36797f 100644
--- a/scripts/lib/build_perf/report.py
+++ b/scripts/lib/build_perf/report.py
@@ -11,12 +11,15 @@
# more details.
#
"""Handling of build perf test reports"""
-from collections import OrderedDict, Mapping
+from collections import OrderedDict, Mapping, namedtuple
from datetime import datetime, timezone
from numbers import Number
from statistics import mean, stdev, variance
+AggregateTestData = namedtuple('AggregateTestData', ['metadata', 'results'])
+
+
def isofmt_to_timestamp(string):
"""Convert timestamp string in ISO 8601 format into unix timestamp"""
if '.' in string:
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 23081db173..3a76ab621d 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -29,7 +29,8 @@ sys.path.append(os.path.join(scripts_path, 'lib'))
import scriptpath
from build_perf import print_table
from build_perf.report import (metadata_xml_to_json, results_xml_to_json,
- aggregate_data, aggregate_metadata, measurement_stats)
+ aggregate_data, aggregate_metadata, measurement_stats,
+ AggregateTestData)
from build_perf import html
scriptpath.add_oe_lib_path()
@@ -337,13 +338,13 @@ def print_html_report(data, id_comp):
'hostname': {'title': 'Hostname', 'value': 'foobar'},
'commit': {'title': 'Commit', 'value': '1234'}
}
- metadata = metadata_diff(data[id_comp][0], data[-1][0])
+ metadata = metadata_diff(data[id_comp].metadata, data[-1].metadata)
# Generate list of tests
tests = []
- for test in data[-1][1]['tests'].keys():
- test_r = data[-1][1]['tests'][test]
+ for test in data[-1].results['tests'].keys():
+ test_r = data[-1].results['tests'][test]
new_test = {'name': test_r['name'],
'description': test_r['description'],
'status': test_r['status'],
@@ -576,7 +577,8 @@ def main(argv=None):
data = []
for raw_m, raw_d in raw_data:
- data.append((aggregate_metadata(raw_m), aggregate_data(raw_d)))
+ data.append(AggregateTestData(aggregate_metadata(raw_m),
+ aggregate_data(raw_d)))
# Re-map list indexes to the new table starting from index 0
index_r = index_r - index_0
@@ -584,8 +586,8 @@ def main(argv=None):
# Print report
if not args.html:
- print_diff_report(data[index_l][0], data[index_l][1],
- data[index_r][0], data[index_r][1])
+ print_diff_report(data[index_l].metadata, data[index_l].results,
+ data[index_r].metadata, data[index_r].results)
else:
print_html_report(data, index_l)
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/7] scripts/oe-build-perf-report: fix handling of --history-length
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
2017-09-15 12:54 ` [PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats Markus Lehtonen
2017-09-15 12:54 ` [PATCH 2/7] scripts/oe-build-perf-report: add AggregateTestData class Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
2017-09-15 12:54 ` [PATCH 4/7] scripts/oe-build-perf-report: use --hostname in --list Markus Lehtonen
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Don't crash if 'left' revision is older than the range of commits
specified with '--history-length'. In this case the 'left' revision
takes precedence.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/oe-build-perf-report | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 3a76ab621d..defd930b04 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -563,7 +563,7 @@ def main(argv=None):
xml = is_xml_format(repo, revs[index_r].tags[-1])
if args.html:
- index_0 = max(0, index_r - args.history_length)
+ index_0 = max(0, min(index_l, index_r - args.history_length))
rev_range = range(index_0, index_r + 1)
else:
# We do not need range of commits for text report (no graphs)
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 4/7] scripts/oe-build-perf-report: use --hostname in --list
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
` (2 preceding siblings ...)
2017-09-15 12:54 ` [PATCH 3/7] scripts/oe-build-perf-report: fix handling of --history-length Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
2017-09-15 12:54 ` [PATCH 5/7] scripts/oe-build-perf-report: remove dead code Markus Lehtonen
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Makes it possible to list test results for certain host only, instead of
always listing all results from all hosts.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/oe-build-perf-report | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index defd930b04..89d9d6f6b5 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -85,7 +85,9 @@ def get_test_runs(repo, tag_name, **kwargs):
def list_test_revs(repo, tag_name, verbosity, **kwargs):
"""Get list of all tested revisions"""
- fields, revs = get_test_runs(repo, tag_name, **kwargs)
+ valid_kwargs = dict([(k, v) for k, v in kwargs.items() if v is not None])
+
+ fields, revs = get_test_runs(repo, tag_name, **valid_kwargs)
ignore_fields = ['tag_number']
if verbosity < 2:
extra_fields = ['COMMITS', 'TEST RUNS']
@@ -509,7 +511,7 @@ def main(argv=None):
repo = GitRepo(args.repo)
if args.list:
- list_test_revs(repo, args.tag_name, args.list)
+ list_test_revs(repo, args.tag_name, args.list, hostname=args.hostname)
return 0
# Determine hostname which to use
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 5/7] scripts/oe-build-perf-report: remove dead code
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
` (3 preceding siblings ...)
2017-09-15 12:54 ` [PATCH 4/7] scripts/oe-build-perf-report: use --hostname in --list Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
2017-09-15 12:54 ` [PATCH 6/7] scripts/oe-build-perf-report: provide valid title in the html report Markus Lehtonen
2017-09-15 12:54 ` [PATCH 7/7] scripts/oe-build-perf-report: tidy up html syntax Markus Lehtonen
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Some leftover from an early prototype.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/oe-build-perf-report | 4 ----
1 file changed, 4 deletions(-)
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 89d9d6f6b5..4e067caba0 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -336,10 +336,6 @@ def print_diff_report(metadata_l, data_l, metadata_r, data_r):
def print_html_report(data, id_comp):
"""Print report in html format"""
# Handle metadata
- metadata = {'branch': {'title': 'Branch', 'value': 'master'},
- 'hostname': {'title': 'Hostname', 'value': 'foobar'},
- 'commit': {'title': 'Commit', 'value': '1234'}
- }
metadata = metadata_diff(data[id_comp].metadata, data[-1].metadata)
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 6/7] scripts/oe-build-perf-report: provide valid title in the html report
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
` (4 preceding siblings ...)
2017-09-15 12:54 ` [PATCH 5/7] scripts/oe-build-perf-report: remove dead code Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
2017-09-15 12:54 ` [PATCH 7/7] scripts/oe-build-perf-report: tidy up html syntax Markus Lehtonen
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/oe-build-perf-report | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 4e067caba0..8d730cd20a 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -396,7 +396,9 @@ def print_html_report(data, id_comp):
'max': get_data_item(data[-1][0], 'layers.meta.commit_count')}
}
- print(html.template.render(metadata=metadata, test_data=tests, chart_opts=chart_opts))
+ print(html.template.render(title="Build Perf Test Report",
+ metadata=metadata, test_data=tests,
+ chart_opts=chart_opts))
def dump_buildstats(repo, outdir, notes_ref, revs):
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 7/7] scripts/oe-build-perf-report: tidy up html syntax
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
` (5 preceding siblings ...)
2017-09-15 12:54 ` [PATCH 6/7] scripts/oe-build-perf-report: provide valid title in the html report Markus Lehtonen
@ 2017-09-15 12:54 ` Markus Lehtonen
6 siblings, 0 replies; 8+ messages in thread
From: Markus Lehtonen @ 2017-09-15 12:54 UTC (permalink / raw)
To: openembedded-core
Fix some problems in the html syntax of the generated report:
- prevent empty rows in the summary table
- add one missing column in the results table
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
scripts/lib/build_perf/html/report.html | 36 ++++++++++++++++++---------------
1 file changed, 20 insertions(+), 16 deletions(-)
diff --git a/scripts/lib/build_perf/html/report.html b/scripts/lib/build_perf/html/report.html
index 165cbb811c..f983a4a09c 100644
--- a/scripts/lib/build_perf/html/report.html
+++ b/scripts/lib/build_perf/html/report.html
@@ -118,29 +118,32 @@ h3 {
{% else %}
{% set row_style = 'style="background-color: #ffffff"' %}
{% endif %}
- <tr {{ row_style }}><td>{{ test.name }}: {{ test.description }}</td>
{% if test.status == 'SUCCESS' %}
{% for measurement in test.measurements %}
- {# add empty cell in place of the test name#}
- {% if loop.index > 1 %}<td></td>{% endif %}
- {% if measurement.absdiff > 0 %}
- {% set result_style = "color: red" %}
- {% elif measurement.absdiff == measurement.absdiff %}
- {% set result_style = "color: green" %}
- {% else %}
- {% set result_style = "color: orange" %}
- {%endif %}
- <td>{{ measurement.description }}</td>
- <td style="font-weight: bold">{{ measurement.value.mean }}</td>
- <td style="{{ result_style }}">{{ measurement.absdiff_str }}</td>
- <td style="{{ result_style }}">{{ measurement.reldiff }}</td>
- </tr><tr {{ row_style }}>
+ <tr {{ row_style }}>
+ {% if loop.index == 1 %}
+ <td>{{ test.name }}: {{ test.description }}</td>
+ {% else %}
+ {# add empty cell in place of the test name#}
+ <td></td>
+ {% endif %}
+ {% if measurement.absdiff > 0 %}
+ {% set result_style = "color: red" %}
+ {% elif measurement.absdiff == measurement.absdiff %}
+ {% set result_style = "color: green" %}
+ {% else %}
+ {% set result_style = "color: orange" %}
+ {%endif %}
+ <td>{{ measurement.description }}</td>
+ <td style="font-weight: bold">{{ measurement.value.mean }}</td>
+ <td style="{{ result_style }}">{{ measurement.absdiff_str }}</td>
+ <td style="{{ result_style }}">{{ measurement.reldiff }}</td>
+ </tr>
{% endfor %}
{% else %}
<td style="font-weight: bold; color: red;">{{test.status }}</td>
<td></td> <td></td> <td></td> <td></td>
{% endif %}
- </tr>
{% endfor %}
</table>
@@ -186,6 +189,7 @@ h3 {
<th>Stdev</th><td>{{ measurement.value.stdev }}</td>
</tr><tr>
<th><div id="{{ test.name }}_{{ measurement.name }}_chart_png"></div></th>
+ <td></td>
</tr>
</table>
</td>
--
2.12.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-09-15 12:55 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-15 12:54 [PATCH 0/7] Fixes to oe-build-perf-report Markus Lehtonen
2017-09-15 12:54 ` [PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats Markus Lehtonen
2017-09-15 12:54 ` [PATCH 2/7] scripts/oe-build-perf-report: add AggregateTestData class Markus Lehtonen
2017-09-15 12:54 ` [PATCH 3/7] scripts/oe-build-perf-report: fix handling of --history-length Markus Lehtonen
2017-09-15 12:54 ` [PATCH 4/7] scripts/oe-build-perf-report: use --hostname in --list Markus Lehtonen
2017-09-15 12:54 ` [PATCH 5/7] scripts/oe-build-perf-report: remove dead code Markus Lehtonen
2017-09-15 12:54 ` [PATCH 6/7] scripts/oe-build-perf-report: provide valid title in the html report Markus Lehtonen
2017-09-15 12:54 ` [PATCH 7/7] scripts/oe-build-perf-report: tidy up html syntax Markus Lehtonen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.