* [PATCH 1/8] oeqa.utils.metadata: re-organise host distro information
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2016-12-28 13:02 ` [PATCH 2/8] oeqa.utils.metadata: re-organise " Markus Lehtonen
` (6 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
Put all host distro data under one <host_distro> element. In addition
take the data directly from /etc/os-release instead of the "lsb API".
The /etc/os-release file is virtually ubiquitous, now, and using its
field names and values provides a more standardized and extensible
format.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 26 +++++++++++++++++++++-----
1 file changed, 21 insertions(+), 5 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index 5d8bf84..2316841 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -10,11 +10,22 @@ from collections.abc import MutableMapping
from xml.dom.minidom import parseString
from xml.etree.ElementTree import Element, tostring
-from oe.lsb import distro_identifier
from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars
metadata_vars = ['MACHINE', 'DISTRO', 'DISTRO_VERSION']
+def get_os_release():
+ """Get info from /etc/os-release as a dict"""
+ data = OrderedDict()
+ os_release_file = '/etc/os-release'
+ if not os.path.exists(os_release_file):
+ return None
+ with open(os_release_file) as fobj:
+ for line in fobj:
+ key, value = line.split('=', 1)
+ data[key.strip().lower()] = value.strip().strip('"')
+ return data
+
def metadata_from_bb():
""" Returns test's metadata as OrderedDict.
@@ -27,10 +38,15 @@ def metadata_from_bb():
data_dict = get_bb_vars(metadata_vars)
for var in metadata_vars:
info_dict[var.lower()] = data_dict[var]
- host_distro= distro_identifier()
- host_distro, _, host_distro_release = host_distro.partition('-')
- info_dict['host_distro'] = host_distro
- info_dict['host_distro_release'] = host_distro_release
+
+ # Host distro information
+ os_release = get_os_release()
+ if os_release:
+ info_dict['host_distro'] = OrderedDict()
+ for key in ('id', 'version_id', 'pretty_name'):
+ if key in os_release:
+ info_dict['host_distro'][key] = os_release[key]
+
info_dict['layers'] = get_layers(get_bb_var('BBLAYERS'))
return info_dict
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/8] oeqa.utils.metadata: re-organise distro information
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
2016-12-28 13:02 ` [PATCH 1/8] oeqa.utils.metadata: re-organise host distro information Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2016-12-28 13:02 ` [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements Markus Lehtonen
` (5 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
Use the same format, based on /etc/os-release, as for host distro
information.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 17 ++++++++++-------
scripts/oe-selftest | 4 ++--
2 files changed, 12 insertions(+), 9 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index 2316841..df6ed91 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -10,9 +10,7 @@ from collections.abc import MutableMapping
from xml.dom.minidom import parseString
from xml.etree.ElementTree import Element, tostring
-from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars
-
-metadata_vars = ['MACHINE', 'DISTRO', 'DISTRO_VERSION']
+from oeqa.utils.commands import runCmd, get_bb_vars
def get_os_release():
"""Get info from /etc/os-release as a dict"""
@@ -35,9 +33,14 @@ def metadata_from_bb():
info_dict = OrderedDict()
hostname = runCmd('hostname')
info_dict['hostname'] = hostname.output
- data_dict = get_bb_vars(metadata_vars)
- for var in metadata_vars:
- info_dict[var.lower()] = data_dict[var]
+ data_dict = get_bb_vars()
+
+ info_dict['machine'] = data_dict['MACHINE']
+
+ # Distro information
+ info_dict['distro'] = {'id': data_dict['DISTRO'],
+ 'version_id': data_dict['DISTRO_VERSION'],
+ 'pretty_name': '%s %s' % (data_dict['DISTRO'], data_dict['DISTRO_VERSION'])}
# Host distro information
os_release = get_os_release()
@@ -47,7 +50,7 @@ def metadata_from_bb():
if key in os_release:
info_dict['host_distro'][key] = os_release[key]
- info_dict['layers'] = get_layers(get_bb_var('BBLAYERS'))
+ info_dict['layers'] = get_layers(data_dict['BBLAYERS'])
return info_dict
def metadata_from_data_store(d):
diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index bfcea66..b4d911e 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -624,8 +624,8 @@ def main():
for layer, values in metadata['layers'].items():
layer_info = '%s%-17s = %s:%s\n' % (layer_info, layer,
values['branch'], values['revision'])
- msg = 'Selftest for build %s of %s %s for machine %s on %s\n\n%s' % (
- log_prefix[12:], metadata['distro'], metadata['distro_version'],
+ msg = 'Selftest for build %s of %s for machine %s on %s\n\n%s' % (
+ log_prefix[12:], metadata['distro']['pretty_name'],
metadata['machine'], metadata['hostname'], layer_info)
log.debug('Commiting results to local repository')
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
2016-12-28 13:02 ` [PATCH 1/8] oeqa.utils.metadata: re-organise host distro information Markus Lehtonen
2016-12-28 13:02 ` [PATCH 2/8] oeqa.utils.metadata: re-organise " Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2017-01-04 15:30 ` Mariano Lopez
2016-12-28 13:02 ` [PATCH 4/8] oeqa.utils.metadata: fix retrieval of git branch and revision Markus Lehtonen
` (4 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
It's better just to not have the xml elements than to have elements with
faux data. One could have git branch named 'unknown', for example.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 7 +++----
scripts/oe-selftest | 4 ++--
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index df6ed91..a389c6a 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -73,11 +73,10 @@ def get_layers(layers):
try:
repo = Repo(layer, search_parent_directories=True)
revision, branch = repo.head.object.name_rev.split()
- layer_dict[layer_name]['branch'] = branch
- layer_dict[layer_name]['revision'] = revision
except (InvalidGitRepositoryError, NoSuchPathError):
- layer_dict[layer_name]['branch'] = 'unknown'
- layer_dict[layer_name]['revision'] = 'unknown'
+ continue
+ layer_dict[layer_name]['branch'] = branch
+ layer_dict[layer_name]['revision'] = revision
return layer_dict
def write_metadata_file(file_path, metadata):
diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index b4d911e..51c52f2 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -595,7 +595,7 @@ def main():
r_branches = set(r_branches.replace('origin/', '').split())
l_branches = {str(branch) for branch in repo.branches}
branch = '%s/%s/%s' % (metadata['hostname'],
- metadata['layers']['meta']['branch'],
+ metadata['layers']['meta'].get('branch', '(nogit)'),
metadata['machine'])
if branch in l_branches:
@@ -623,7 +623,7 @@ def main():
layer_info = ''
for layer, values in metadata['layers'].items():
layer_info = '%s%-17s = %s:%s\n' % (layer_info, layer,
- values['branch'], values['revision'])
+ values.get('branch', '(nogit)'), values.get('revision', '0'*40))
msg = 'Selftest for build %s of %s for machine %s on %s\n\n%s' % (
log_prefix[12:], metadata['distro']['pretty_name'],
metadata['machine'], metadata['hostname'], layer_info)
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements
2016-12-28 13:02 ` [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements Markus Lehtonen
@ 2017-01-04 15:30 ` Mariano Lopez
2017-01-05 9:19 ` Markus Lehtonen
0 siblings, 1 reply; 15+ messages in thread
From: Mariano Lopez @ 2017-01-04 15:30 UTC (permalink / raw)
To: Markus Lehtonen, openembedded-core
On 28/12/16 07:02, Markus Lehtonen wrote:
> It's better just to not have the xml elements than to have elements with
> faux data. One could have git branch named 'unknown', for example.
>
>
I don't think is a good idea to completely remove the layer from the
metadata when such layer is not a repository, it would remove valuable
information for later analysis. And if you ask would do such a thing, I
would, I have my own debug layers that can mess with testing.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements
2017-01-04 15:30 ` Mariano Lopez
@ 2017-01-05 9:19 ` Markus Lehtonen
0 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2017-01-05 9:19 UTC (permalink / raw)
To: Mariano Lopez, openembedded-core
On 04/01/2017, 17.30, "Mariano Lopez" <mariano.lopez@linux.intel.com>
wrote:
>On 28/12/16 07:02, Markus Lehtonen wrote:
>> It's better just to not have the xml elements than to have elements with
>> faux data. One could have git branch named 'unknown', for example.
>>
>>
>
>I don't think is a good idea to completely remove the layer from the
>metadata when such layer is not a repository, it would remove valuable
>information for later analysis. And if you ask would do such a thing, I
>would, I have my own debug layers that can mess with testing.
I'm not removing the entire layer. I'm just removing the
<branch>unknown</branch> and <revisio>unknown</revision> elements. So the
layer elements are still there.
Thanks,
Markus
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 4/8] oeqa.utils.metadata: fix retrieval of git branch and revision
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
` (2 preceding siblings ...)
2016-12-28 13:02 ` [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2016-12-28 13:02 ` [PATCH 5/8] oeqa.utils.metadata: rename 'revision' to 'commit' Markus Lehtonen
` (3 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
Always return a valid branch name, or, '(nobranch)' if the current HEAD
is detached. Also, always return the hash of the commit object that HEAD
is pointing to. Previous code returned an incorrect branch name (or
crashed) e.g. in the case of detached HEAD.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index a389c6a..b732d37 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -72,11 +72,13 @@ def get_layers(layers):
layer_dict[layer_name] = OrderedDict()
try:
repo = Repo(layer, search_parent_directories=True)
- revision, branch = repo.head.object.name_rev.split()
except (InvalidGitRepositoryError, NoSuchPathError):
continue
- layer_dict[layer_name]['branch'] = branch
- layer_dict[layer_name]['revision'] = revision
+ layer_dict[layer_name]['revision'] = repo.head.commit.hexsha
+ try:
+ layer_dict[layer_name]['branch'] = repo.active_branch.name
+ except TypeError:
+ layer_dict[layer_name]['branch'] = '(nobranch)'
return layer_dict
def write_metadata_file(file_path, metadata):
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 5/8] oeqa.utils.metadata: rename 'revision' to 'commit'
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
` (3 preceding siblings ...)
2016-12-28 13:02 ` [PATCH 4/8] oeqa.utils.metadata: fix retrieval of git branch and revision Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2016-12-28 13:02 ` [PATCH 6/8] oeqa.utils.metadata: add commit count information Markus Lehtonen
` (2 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
Revision is a bit vague and could point to a tag, for example. Git
commit objects are unambiguous and persistent so be explicit that the
element should contain git commit hash.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 2 +-
scripts/oe-selftest | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index b732d37..2f7e8f2 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -74,7 +74,7 @@ def get_layers(layers):
repo = Repo(layer, search_parent_directories=True)
except (InvalidGitRepositoryError, NoSuchPathError):
continue
- layer_dict[layer_name]['revision'] = repo.head.commit.hexsha
+ layer_dict[layer_name]['commit'] = repo.head.commit.hexsha
try:
layer_dict[layer_name]['branch'] = repo.active_branch.name
except TypeError:
diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index 51c52f2..2092b09 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -623,7 +623,7 @@ def main():
layer_info = ''
for layer, values in metadata['layers'].items():
layer_info = '%s%-17s = %s:%s\n' % (layer_info, layer,
- values.get('branch', '(nogit)'), values.get('revision', '0'*40))
+ values.get('branch', '(nogit)'), values.get('commit', '0'*40))
msg = 'Selftest for build %s of %s for machine %s on %s\n\n%s' % (
log_prefix[12:], metadata['distro']['pretty_name'],
metadata['machine'], metadata['hostname'], layer_info)
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 6/8] oeqa.utils.metadata: add commit count information
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
` (4 preceding siblings ...)
2016-12-28 13:02 ` [PATCH 5/8] oeqa.utils.metadata: rename 'revision' to 'commit' Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2016-12-28 13:02 ` [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml Markus Lehtonen
2016-12-28 13:02 ` [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information Markus Lehtonen
7 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
Makes it easier to put the commits into a timeline.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index 2f7e8f2..d5cc290 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -63,7 +63,7 @@ def metadata_from_data_store(d):
pass
def get_layers(layers):
- """ Returns layer name, branch, and revision as OrderedDict. """
+ """Returns layer information in dict format"""
from git import Repo, InvalidGitRepositoryError, NoSuchPathError
layer_dict = OrderedDict()
@@ -75,6 +75,7 @@ def get_layers(layers):
except (InvalidGitRepositoryError, NoSuchPathError):
continue
layer_dict[layer_name]['commit'] = repo.head.commit.hexsha
+ layer_dict[layer_name]['commit_count'] = repo.head.commit.count()
try:
layer_dict[layer_name]['branch'] = repo.active_branch.name
except TypeError:
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
` (5 preceding siblings ...)
2016-12-28 13:02 ` [PATCH 6/8] oeqa.utils.metadata: add commit count information Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2017-01-04 15:43 ` Mariano Lopez
2016-12-28 13:02 ` [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information Markus Lehtonen
7 siblings, 1 reply; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
Have the layer name as an attribute instead of of the name of the
element itself. That is, have <layer name="layer_name"/> instead of
<layer_name/>. A bit better XML design.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index d5cc290..6331c21 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -90,12 +90,14 @@ def write_metadata_file(file_path, metadata):
with open(file_path, 'w') as f:
f.write(xml_doc.toprettyxml())
-def dict_to_XML(tag, dictionary):
+def dict_to_XML(tag, dictionary, **kwargs):
""" Return XML element converting dicts recursively. """
- elem = Element(tag)
+ elem = Element(tag, **kwargs)
for key, val in dictionary.items():
- if isinstance(val, MutableMapping):
+ if tag == 'layers':
+ child = (dict_to_XML('layer', val, name=key))
+ elif isinstance(val, MutableMapping):
child = (dict_to_XML(key, val))
else:
child = Element(key)
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml
2016-12-28 13:02 ` [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml Markus Lehtonen
@ 2017-01-04 15:43 ` Mariano Lopez
2017-01-05 9:35 ` Markus Lehtonen
0 siblings, 1 reply; 15+ messages in thread
From: Mariano Lopez @ 2017-01-04 15:43 UTC (permalink / raw)
To: Markus Lehtonen, openembedded-core
On 28/12/16 07:02, Markus Lehtonen wrote:
> -def dict_to_XML(tag, dictionary):
> +def dict_to_XML(tag, dictionary, **kwargs):
> """ Return XML element converting dicts recursively. """
>
> - elem = Element(tag)
> + elem = Element(tag, **kwargs)
> for key, val in dictionary.items():
> - if isinstance(val, MutableMapping):
> + if tag == 'layers':
> + child = (dict_to_XML('layer', val, name=key))
> + elif isinstance(val, MutableMapping):
> child = (dict_to_XML(key, val))
> else:
> child = Element(key)
I was thinking that this function would be more generic, adding a
comparison to a tag name surely won't keep the function generic, is
there another way to implement this keeping this function generic enough?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml
2017-01-04 15:43 ` Mariano Lopez
@ 2017-01-05 9:35 ` Markus Lehtonen
0 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2017-01-05 9:35 UTC (permalink / raw)
To: Mariano Lopez, openembedded-core
On 04/01/2017, 17.43, "Mariano Lopez" <mariano.lopez@linux.intel.com>
wrote:
>
>On 28/12/16 07:02, Markus Lehtonen wrote:
>> -def dict_to_XML(tag, dictionary):
>> +def dict_to_XML(tag, dictionary, **kwargs):
>> """ Return XML element converting dicts recursively. """
>>
>> - elem = Element(tag)
>> + elem = Element(tag, **kwargs)
>> for key, val in dictionary.items():
>> - if isinstance(val, MutableMapping):
>> + if tag == 'layers':
>> + child = (dict_to_XML('layer', val, name=key))
>> + elif isinstance(val, MutableMapping):
>> child = (dict_to_XML(key, val))
>> else:
>> child = Element(key)
>
>I was thinking that this function would be more generic, adding a
>comparison to a tag name surely won't keep the function generic, is
>there another way to implement this keeping this function generic enough?
I don't think it's possible to make a nice generic json to xml converter.
The formats are just so different. I think the current method just
generates bad xml. In the future, it would probably be better to just get
rid of the json-to-xml conversion completely and make metadata_from_bb()
to directly return an ElementTree.
Thanks,
Markus
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information
2016-12-28 13:02 [PATCH 0/8] oeqa.utils.metadata: update xml schema Markus Lehtonen
` (6 preceding siblings ...)
2016-12-28 13:02 ` [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml Markus Lehtonen
@ 2016-12-28 13:02 ` Markus Lehtonen
2017-01-04 15:45 ` Mariano Lopez
7 siblings, 1 reply; 15+ messages in thread
From: Markus Lehtonen @ 2016-12-28 13:02 UTC (permalink / raw)
To: openembedded-core
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/lib/oeqa/utils/metadata.py | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index 6331c21..23449fc 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -10,6 +10,8 @@ from collections.abc import MutableMapping
from xml.dom.minidom import parseString
from xml.etree.ElementTree import Element, tostring
+from git import Repo, InvalidGitRepositoryError, NoSuchPathError
+
from oeqa.utils.commands import runCmd, get_bb_vars
def get_os_release():
@@ -51,6 +53,7 @@ def metadata_from_bb():
info_dict['host_distro'][key] = os_release[key]
info_dict['layers'] = get_layers(data_dict['BBLAYERS'])
+ info_dict['bitbake'] = git_rev_info(os.path.dirname(bb.__file__))
return info_dict
def metadata_from_data_store(d):
@@ -62,24 +65,27 @@ def metadata_from_data_store(d):
# be useful when running within bitbake.
pass
+def git_rev_info(path):
+ """Get git revision information as a dict"""
+ info = OrderedDict()
+ try:
+ repo = Repo(path, search_parent_directories=True)
+ except (InvalidGitRepositoryError, NoSuchPathError):
+ return info
+ info['commit'] = repo.head.commit.hexsha
+ info['commit_count'] = repo.head.commit.count()
+ try:
+ info['branch'] = repo.active_branch.name
+ except TypeError:
+ info['branch'] = '(nobranch)'
+ return info
+
def get_layers(layers):
"""Returns layer information in dict format"""
- from git import Repo, InvalidGitRepositoryError, NoSuchPathError
-
layer_dict = OrderedDict()
for layer in layers.split():
layer_name = os.path.basename(layer)
- layer_dict[layer_name] = OrderedDict()
- try:
- repo = Repo(layer, search_parent_directories=True)
- except (InvalidGitRepositoryError, NoSuchPathError):
- continue
- layer_dict[layer_name]['commit'] = repo.head.commit.hexsha
- layer_dict[layer_name]['commit_count'] = repo.head.commit.count()
- try:
- layer_dict[layer_name]['branch'] = repo.active_branch.name
- except TypeError:
- layer_dict[layer_name]['branch'] = '(nobranch)'
+ layer_dict[layer_name] = git_rev_info(layer)
return layer_dict
def write_metadata_file(file_path, metadata):
--
2.6.6
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information
2016-12-28 13:02 ` [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information Markus Lehtonen
@ 2017-01-04 15:45 ` Mariano Lopez
2017-01-05 9:40 ` Markus Lehtonen
0 siblings, 1 reply; 15+ messages in thread
From: Mariano Lopez @ 2017-01-04 15:45 UTC (permalink / raw)
To: Markus Lehtonen, openembedded-core
On 28/12/16 07:02, Markus Lehtonen wrote:
> [YOCTO #10590]
>
> Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
> ---
> meta/lib/oeqa/utils/metadata.py | 32 +++++++++++++++++++-------------
> 1 file changed, 19 insertions(+), 13 deletions(-)
>
> diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
> index 6331c21..23449fc 100644
> --- a/meta/lib/oeqa/utils/metadata.py
> +++ b/meta/lib/oeqa/utils/metadata.py
> @@ -10,6 +10,8 @@ from collections.abc import MutableMapping
> from xml.dom.minidom import parseString
> from xml.etree.ElementTree import Element, tostring
>
> +from git import Repo, InvalidGitRepositoryError, NoSuchPathError
> +
It seems not every user running selftest appreciated the requirement of
gitpython, so there was a patch to keep this dependency out. This will
introduce the need of gitpython again.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information
2017-01-04 15:45 ` Mariano Lopez
@ 2017-01-05 9:40 ` Markus Lehtonen
0 siblings, 0 replies; 15+ messages in thread
From: Markus Lehtonen @ 2017-01-05 9:40 UTC (permalink / raw)
To: Mariano Lopez, openembedded-core
On 04/01/2017, 17.45, "Mariano Lopez" <mariano.lopez@linux.intel.com>
wrote:
>On 28/12/16 07:02, Markus Lehtonen wrote:
>> [YOCTO #10590]
>>
>> Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
>> ---
>> meta/lib/oeqa/utils/metadata.py | 32 +++++++++++++++++++-------------
>> 1 file changed, 19 insertions(+), 13 deletions(-)
>>
>> diff --git a/meta/lib/oeqa/utils/metadata.py
>>b/meta/lib/oeqa/utils/metadata.py
>> index 6331c21..23449fc 100644
>> --- a/meta/lib/oeqa/utils/metadata.py
>> +++ b/meta/lib/oeqa/utils/metadata.py
>> @@ -10,6 +10,8 @@ from collections.abc import MutableMapping
>> from xml.dom.minidom import parseString
>> from xml.etree.ElementTree import Element, tostring
>>
>> +from git import Repo, InvalidGitRepositoryError, NoSuchPathError
>> +
>
>It seems not every user running selftest appreciated the requirement of
>gitpython, so there was a patch to keep this dependency out. This will
>introduce the need of gitpython again.
Yes. I don't know why I moved this import to module level in the first
place. A corrected version of the patch in now found in my
oe-core-contrib branch
(git://git.openembedded.org/openembedded-core-contrib marquiz/oeqa-metaxml)
Thanks,
Markus
^ permalink raw reply [flat|nested] 15+ messages in thread