* [PATCH 2/3] data_smart: Improve error display for handled exceptions
2021-09-10 14:07 [PATCH 1/3] build: Catch and error upon circular task references Richard Purdie
@ 2021-09-10 14:07 ` Richard Purdie
2021-09-10 14:07 ` [PATCH 3/3] fetch2: Add recursion guard Richard Purdie
2021-09-10 14:42 ` [bitbake-devel] [PATCH 1/3] build: Catch and error upon circular task references Quentin Schulz
2 siblings, 0 replies; 7+ messages in thread
From: Richard Purdie @ 2021-09-10 14:07 UTC (permalink / raw)
To: bitbake-devel
We don't need tracebacks for BBHandledException. Reduces confusing output like:
ERROR: /meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio
ERROR: ExpansionError during parsing /meta/recipes-core/images/core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_qa', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
if task in deps:
> follow_chain(othertask, endtask, chain)
chain.pop()
File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
if task in deps:
> follow_chain(othertask, endtask, chain)
chain.pop()
File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_cpio', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
if task in deps:
> follow_chain(othertask, endtask, chain)
chain.pop()
File "/bitbake/lib/bb/build.py", line 1038, in follow_chain(task='do_image_complete', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
if task in chain:
> bb.fatal("Circular task dependencies as %s depends itself via the chain %s?!" % (task, " -> ".join(chain)))
chain.append(task)
File "/bitbake/lib/bb/__init__.py", line 165, in fatal:
mainlogger.critical(''.join(args), extra=kwargs)
> raise BBHandledException()
to the real error:
ERROR: /media/build1/poky/meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
lib/bb/data_smart.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 515d195fe8..e4afac64b2 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -418,6 +418,8 @@ class DataSmart(MutableMapping):
raise
except bb.parse.SkipRecipe:
raise
+ except bb.BBHandledException:
+ raise
except Exception as exc:
tb = sys.exc_info()[2]
raise ExpansionError(varname, s, exc).with_traceback(tb) from exc
--
2.32.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/3] fetch2: Add recursion guard
2021-09-10 14:07 [PATCH 1/3] build: Catch and error upon circular task references Richard Purdie
2021-09-10 14:07 ` [PATCH 2/3] data_smart: Improve error display for handled exceptions Richard Purdie
@ 2021-09-10 14:07 ` Richard Purdie
2021-09-10 14:48 ` [bitbake-devel] " Quentin Schulz
2021-09-10 14:42 ` [bitbake-devel] [PATCH 1/3] build: Catch and error upon circular task references Quentin Schulz
2 siblings, 1 reply; 7+ messages in thread
From: Richard Purdie @ 2021-09-10 14:07 UTC (permalink / raw)
To: bitbake-devel
Users sometimes put ${S} references in ${SRC_URI} without realising this can be
problematic. Improve the error messages if they accidentally do.
[YOCTO #11593]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
lib/bb/fetch2/__init__.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 47a4943369..d9e1599a05 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -754,6 +754,11 @@ def get_srcrev(d, method_name='sortable_revision'):
that fetcher provides a method with the given name and the same signature as sortable_revision.
"""
+ recursion = d.getVar("__BBINSRCREV")
+ if recursion:
+ raise FetchError("There are recursive references in fetcher variables, likely through SRC_URI")
+ d.setVar("__BBINSRCREV", True)
+
scms = []
fetcher = Fetch(d.getVar('SRC_URI').split(), d)
urldata = fetcher.ud
@@ -768,6 +773,7 @@ def get_srcrev(d, method_name='sortable_revision'):
autoinc, rev = getattr(urldata[scms[0]].method, method_name)(urldata[scms[0]], d, urldata[scms[0]].names[0])
if len(rev) > 10:
rev = rev[:10]
+ d.delVar("__BBINSRCREV")
if autoinc:
return "AUTOINC+" + rev
return rev
@@ -802,6 +808,7 @@ def get_srcrev(d, method_name='sortable_revision'):
if seenautoinc:
format = "AUTOINC+" + format
+ d.delVar("__BBINSRCREV")
return format
def localpath(url, d):
--
2.32.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [bitbake-devel] [PATCH 1/3] build: Catch and error upon circular task references
2021-09-10 14:07 [PATCH 1/3] build: Catch and error upon circular task references Richard Purdie
2021-09-10 14:07 ` [PATCH 2/3] data_smart: Improve error display for handled exceptions Richard Purdie
2021-09-10 14:07 ` [PATCH 3/3] fetch2: Add recursion guard Richard Purdie
@ 2021-09-10 14:42 ` Quentin Schulz
2021-09-10 14:54 ` Richard Purdie
2 siblings, 1 reply; 7+ messages in thread
From: Quentin Schulz @ 2021-09-10 14:42 UTC (permalink / raw)
To: Richard Purdie; +Cc: bitbake-devel
Hi Richard,
On Fri, Sep 10, 2021 at 03:07:38PM +0100, Richard Purdie wrote:
> If there are circular task references, error on them rather than show
> a recursion error. A simple reproducer is:
>
> """
> do_packageswu () {
> :
> }
>
> addtask do_packageswu after do_image_complete before do_image_qa
> """
>
> into image_types.bbclass. There is code in runqueue to detect these but
> we never get that far with the current codebase.
>
> [YOCTO #13140]
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> lib/bb/build.py | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/lib/bb/build.py b/lib/bb/build.py
> index 1e062adb51..4249c0be2b 100644
> --- a/lib/bb/build.py
> +++ b/lib/bb/build.py
> @@ -1034,6 +1034,8 @@ def tasksbetween(task_start, task_end, d):
> def follow_chain(task, endtask, chain=None):
> if not chain:
> chain = []
> + if task in chain:
> + bb.fatal("Circular task dependencies as %s depends itself via the chain %s" % (task, " -> ".join(chain)))
depends +on itself?
Cheers,
Quentin
^ permalink raw reply [flat|nested] 7+ messages in thread