* [PATCH] yocto-compat-layer: improve error handling in signature creation
@ 2017-03-15 10:01 Patrick Ohly
2017-03-15 16:54 ` Aníbal Limón
0 siblings, 1 reply; 4+ messages in thread
From: Patrick Ohly @ 2017-03-15 10:01 UTC (permalink / raw)
To: openembedded-core; +Cc: saul.wold
When "bitbake -k -S none world" failed, the error printed by
yocto-compat-layer.py contained the stack trace multiple times and did not
contain the stderr output from bitbake, making the error hard to understand
and debug:
INFO: ======================================================================
INFO: ERROR: test_signatures (common.CommonCompatLayer)
INFO: ----------------------------------------------------------------------
INFO: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
stderr=subprocess.PIPE)
File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in get_signatures
raise RuntimeError(msg)
RuntimeError: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
stderr=subprocess.PIPE)
File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
Loading cache...done.
Loaded 1328 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
NOTE: Runtime target 'zlib-qat' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['zlib-qat']
...
Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
The yocto-compat-layer.log was incomplete, it only had the first part
without the command output.
stderr was missing due to stderr=subprocess.PIPE.
Instead of the complicated try/except construct it's better to check
the return code ourselves and raise just a single exception. The
output (both on stderr and in the yocto-compat-layer.log) now is:
INFO: ======================================================================
INFO: ERROR: test_signatures (common.CommonCompatLayer)
INFO: ----------------------------------------------------------------------
INFO: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in get_signatures
raise RuntimeError(msg)
RuntimeError: Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.
Command: bitbake -k -S none world
Output:
Loading cache...done.
Loaded 1328 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
ERROR: Nothing PROVIDES 'qat16' (but /fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb DEPENDS on or otherwise requires it)
ERROR: qat16 was skipped: incompatible with machine qemux86 (not in COMPATIBLE_MACHINE)
...
Missing or unbuildable dependency chain was: ['openssl-qat-dev']
...
Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
scripts/lib/compatlayer/__init__.py | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/scripts/lib/compatlayer/__init__.py b/scripts/lib/compatlayer/__init__.py
index b3a166aa9ab..a7eb8625310 100644
--- a/scripts/lib/compatlayer/__init__.py
+++ b/scripts/lib/compatlayer/__init__.py
@@ -135,17 +135,15 @@ def get_signatures(builddir, failsafe=False):
sigs = {}
- try:
- cmd = 'bitbake '
- if failsafe:
- cmd += '-k '
- cmd += '-S none world'
- output = subprocess.check_output(cmd, shell=True,
- stderr=subprocess.PIPE)
- except subprocess.CalledProcessError as e:
- import traceback
- exc = traceback.format_exc()
- msg = '%s\n%s\n' % (exc, e.output.decode('utf-8'))
+ cmd = 'bitbake '
+ if failsafe:
+ cmd += '-k '
+ cmd += '-S none world'
+ p = subprocess.Popen(cmd, shell=True,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+ output, _ = p.communicate()
+ if p.returncode:
+ msg = "Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.\nCommand: %s\nOutput:\n%s" % (cmd, output.decode('utf-8'))
raise RuntimeError(msg)
sigs_file = os.path.join(builddir, 'locked-sigs.inc')
--
2.11.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] yocto-compat-layer: improve error handling in signature creation
2017-03-15 10:01 [PATCH] yocto-compat-layer: improve error handling in signature creation Patrick Ohly
@ 2017-03-15 16:54 ` Aníbal Limón
0 siblings, 0 replies; 4+ messages in thread
From: Aníbal Limón @ 2017-03-15 16:54 UTC (permalink / raw)
To: Patrick Ohly, openembedded-core; +Cc: saul.wold
[-- Attachment #1: Type: text/plain, Size: 5396 bytes --]
Acked-by: Aníbal Limón <anibal.limon@linux.intel.com>
On 03/15/2017 04:01 AM, Patrick Ohly wrote:
> When "bitbake -k -S none world" failed, the error printed by
> yocto-compat-layer.py contained the stack trace multiple times and did not
> contain the stderr output from bitbake, making the error hard to understand
> and debug:
>
> INFO: ======================================================================
> INFO: ERROR: test_signatures (common.CommonCompatLayer)
> INFO: ----------------------------------------------------------------------
> INFO: Traceback (most recent call last):
> File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
> stderr=subprocess.PIPE)
> File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
> raise CalledProcessError(retcode, process.args, output=output)
> subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
> curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
> File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in get_signatures
> raise RuntimeError(msg)
> RuntimeError: Traceback (most recent call last):
> File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
> stderr=subprocess.PIPE)
> File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
> raise CalledProcessError(retcode, process.args, output=output)
> subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
>
> Loading cache...done.
> Loaded 1328 entries from dependency cache.
> NOTE: Resolving any missing task queue dependencies
> NOTE: Runtime target 'zlib-qat' is unbuildable, removing...
> Missing or unbuildable dependency chain was: ['zlib-qat']
> ...
> Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
>
> The yocto-compat-layer.log was incomplete, it only had the first part
> without the command output.
>
> stderr was missing due to stderr=subprocess.PIPE.
>
> Instead of the complicated try/except construct it's better to check
> the return code ourselves and raise just a single exception. The
> output (both on stderr and in the yocto-compat-layer.log) now is:
>
> INFO: ======================================================================
> INFO: ERROR: test_signatures (common.CommonCompatLayer)
> INFO: ----------------------------------------------------------------------
> INFO: Traceback (most recent call last):
> File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
> curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
> File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in get_signatures
> raise RuntimeError(msg)
> RuntimeError: Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.
> Command: bitbake -k -S none world
> Output:
> Loading cache...done.
> Loaded 1328 entries from dependency cache.
> NOTE: Resolving any missing task queue dependencies
> ERROR: Nothing PROVIDES 'qat16' (but /fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb DEPENDS on or otherwise requires it)
> ERROR: qat16 was skipped: incompatible with machine qemux86 (not in COMPATIBLE_MACHINE)
> ...
> Missing or unbuildable dependency chain was: ['openssl-qat-dev']
> ...
> Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
>
> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
> ---
> scripts/lib/compatlayer/__init__.py | 20 +++++++++-----------
> 1 file changed, 9 insertions(+), 11 deletions(-)
>
> diff --git a/scripts/lib/compatlayer/__init__.py b/scripts/lib/compatlayer/__init__.py
> index b3a166aa9ab..a7eb8625310 100644
> --- a/scripts/lib/compatlayer/__init__.py
> +++ b/scripts/lib/compatlayer/__init__.py
> @@ -135,17 +135,15 @@ def get_signatures(builddir, failsafe=False):
>
> sigs = {}
>
> - try:
> - cmd = 'bitbake '
> - if failsafe:
> - cmd += '-k '
> - cmd += '-S none world'
> - output = subprocess.check_output(cmd, shell=True,
> - stderr=subprocess.PIPE)
> - except subprocess.CalledProcessError as e:
> - import traceback
> - exc = traceback.format_exc()
> - msg = '%s\n%s\n' % (exc, e.output.decode('utf-8'))
> + cmd = 'bitbake '
> + if failsafe:
> + cmd += '-k '
> + cmd += '-S none world'
> + p = subprocess.Popen(cmd, shell=True,
> + stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
> + output, _ = p.communicate()
> + if p.returncode:
> + msg = "Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.\nCommand: %s\nOutput:\n%s" % (cmd, output.decode('utf-8'))
> raise RuntimeError(msg)
> sigs_file = os.path.join(builddir, 'locked-sigs.inc')
>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] yocto-compat-layer: improve error handling in signature creation
2017-03-15 9:48 Patrick Ohly
@ 2017-03-15 10:01 ` Patrick Ohly
0 siblings, 0 replies; 4+ messages in thread
From: Patrick Ohly @ 2017-03-15 10:01 UTC (permalink / raw)
To: openembedded-devel; +Cc: saul.wold
On Wed, 2017-03-15 at 10:48 +0100, Patrick Ohly wrote:
> When "bitbake -k -S none world" failed, the error printed by
> yocto-compat-layer.py contained the stack trace multiple times and did not
> contain the stderr output from bitbake, making the error hard to understand
> and debug
Sorry, wrong list. I closed the patch in patchwork, too.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH] yocto-compat-layer: improve error handling in signature creation
@ 2017-03-15 9:48 Patrick Ohly
2017-03-15 10:01 ` Patrick Ohly
0 siblings, 1 reply; 4+ messages in thread
From: Patrick Ohly @ 2017-03-15 9:48 UTC (permalink / raw)
To: openembedded-devel; +Cc: saul.wold
When "bitbake -k -S none world" failed, the error printed by
yocto-compat-layer.py contained the stack trace multiple times and did not
contain the stderr output from bitbake, making the error hard to understand
and debug:
INFO: ======================================================================
INFO: ERROR: test_signatures (common.CommonCompatLayer)
INFO: ----------------------------------------------------------------------
INFO: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
stderr=subprocess.PIPE)
File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in get_signatures
raise RuntimeError(msg)
RuntimeError: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
stderr=subprocess.PIPE)
File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
Loading cache...done.
Loaded 1328 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
NOTE: Runtime target 'zlib-qat' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['zlib-qat']
...
Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
The yocto-compat-layer.log was incomplete, it only had the first part
without the command output.
stderr was missing due to stderr=subprocess.PIPE.
Instead of the complicated try/except construct it's better to check
the return code ourselves and raise just a single exception. The
output (both on stderr and in the yocto-compat-layer.log) now is:
INFO: ======================================================================
INFO: ERROR: test_signatures (common.CommonCompatLayer)
INFO: ----------------------------------------------------------------------
INFO: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in get_signatures
raise RuntimeError(msg)
RuntimeError: Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.
Command: bitbake -k -S none world
Output:
Loading cache...done.
Loaded 1328 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
ERROR: Nothing PROVIDES 'qat16' (but /fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb DEPENDS on or otherwise requires it)
ERROR: qat16 was skipped: incompatible with machine qemux86 (not in COMPATIBLE_MACHINE)
...
Missing or unbuildable dependency chain was: ['openssl-qat-dev']
...
Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
scripts/lib/compatlayer/__init__.py | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/scripts/lib/compatlayer/__init__.py b/scripts/lib/compatlayer/__init__.py
index b3a166aa9ab..a7eb8625310 100644
--- a/scripts/lib/compatlayer/__init__.py
+++ b/scripts/lib/compatlayer/__init__.py
@@ -135,17 +135,15 @@ def get_signatures(builddir, failsafe=False):
sigs = {}
- try:
- cmd = 'bitbake '
- if failsafe:
- cmd += '-k '
- cmd += '-S none world'
- output = subprocess.check_output(cmd, shell=True,
- stderr=subprocess.PIPE)
- except subprocess.CalledProcessError as e:
- import traceback
- exc = traceback.format_exc()
- msg = '%s\n%s\n' % (exc, e.output.decode('utf-8'))
+ cmd = 'bitbake '
+ if failsafe:
+ cmd += '-k '
+ cmd += '-S none world'
+ p = subprocess.Popen(cmd, shell=True,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+ output, _ = p.communicate()
+ if p.returncode:
+ msg = "Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.\nCommand: %s\nOutput:\n%s" % (cmd, output.decode('utf-8'))
raise RuntimeError(msg)
sigs_file = os.path.join(builddir, 'locked-sigs.inc')
--
2.11.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-03-15 16:50 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-15 10:01 [PATCH] yocto-compat-layer: improve error handling in signature creation Patrick Ohly
2017-03-15 16:54 ` Aníbal Limón
-- strict thread matches above, loose matches on Subject: below --
2017-03-15 9:48 Patrick Ohly
2017-03-15 10:01 ` Patrick Ohly
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.