[RFC][PATCH] ast.py: catch SyntaxError from imported modules as well

[RFC][PATCH] ast.py: catch SyntaxError from imported modules as well

[Martin Jansa]

* before addpylib directive possible syntax error resulted in output like this:
  $ echo "def foo()" >> oe-core/meta/lib/oe/qa.py
  $ bitbake -k zlib-native
  ERROR: Unable to parse Var <OE_IMPORTED[:=]>
  Traceback (most recent call last):
    File "Var <OE_IMPORTED[:=]>", line 1, in <module>
    File "oe-core/meta/classes-global/base.bbclass", line 35, in oe_import(d=<bb.data_smart.DataSmart object at 0x7fa09b142790>):
                   # Make a python object accessible from the metadata
      >            bb.utils._context[toimport.split(".", 1)[0]] = __import__(toimport)
               except AttributeError as e:
  bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception SyntaxError: expected ':' (qa.py, line 222)
  The variable dependency chain for the failure is: OE_IMPORTED[:=]

* now it wasn't showing any output at all when there was syntax error
* show error and name of the module which failed to be imported
  $ bitbake -k zlib-native
  ERROR: Error importing OE module qa: invalid syntax (qa.py, line 229)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 lib/bb/parse/ast.py | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/lib/bb/parse/ast.py b/lib/bb/parse/ast.py
index 375ba3cb..efdb9000 100644
--- a/lib/bb/parse/ast.py
+++ b/lib/bb/parse/ast.py
@@ -285,10 +285,10 @@ class PyLibNode(AstNode):
         libdir = data.expand(self.libdir)
         if libdir not in sys.path:
             sys.path.append(libdir)
-        try:
-            bb.utils._context[self.namespace] = __import__(self.namespace)
-            toimport = getattr(bb.utils._context[self.namespace], "BBIMPORTS", [])
-            for i in toimport:
+        bb.utils._context[self.namespace] = __import__(self.namespace)
+        toimport = getattr(bb.utils._context[self.namespace], "BBIMPORTS", [])
+        for i in toimport:
+            try:
                 bb.utils._context[self.namespace] = __import__(self.namespace + "." + i)
                 mod = getattr(bb.utils._context[self.namespace], i)
                 fn = getattr(mod, "__file__")
@@ -301,9 +301,10 @@ class PyLibNode(AstNode):
                         continue
                     funcs[f] = fcall
                 bb.codeparser.add_module_functions(fn, funcs, "%s.%s" % (self.namespace, i))
+            except (AttributeError, SyntaxError) as e:
+                bb.error("Error importing OE module %s: %s" % (i, str(e)))
+                raise
 
-        except AttributeError as e:
-            bb.error("Error importing OE modules: %s" % str(e))
 
 class InheritNode(AstNode):
     def __init__(self, filename, lineno, classes):
-- 
2.39.1

Re: [bitbake-devel] [RFC][PATCH] ast.py: catch SyntaxError from imported modules as well

[Richard Purdie]

On Wed, 2023-02-15 at 16:35 +0100, Martin Jansa wrote:
> * before addpylib directive possible syntax error resulted in output like this:
>   $ echo "def foo()" >> oe-core/meta/lib/oe/qa.py
>   $ bitbake -k zlib-native
>   ERROR: Unable to parse Var <OE_IMPORTED[:=]>
>   Traceback (most recent call last):
>     File "Var <OE_IMPORTED[:=]>", line 1, in <module>
>     File "oe-core/meta/classes-global/base.bbclass", line 35, in oe_import(d=<bb.data_smart.DataSmart object at 0x7fa09b142790>):
>                    # Make a python object accessible from the metadata
>       >            bb.utils._context[toimport.split(".", 1)[0]] = __import__(toimport)
>                except AttributeError as e:
>   bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception SyntaxError: expected ':' (qa.py, line 222)
>   The variable dependency chain for the failure is: OE_IMPORTED[:=]
> 
> * now it wasn't showing any output at all when there was syntax error
> * show error and name of the module which failed to be imported
>   $ bitbake -k zlib-native
>   ERROR: Error importing OE module qa: invalid syntax (qa.py, line 229)
> 
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  lib/bb/parse/ast.py | 13 +++++++------
>  1 file changed, 7 insertions(+), 6 deletions(-)

Thanks for reporting this. I looked into it a bit and in the end I've
sent a different patch which should fix this and a number of other
possible errors more generically.

I still don't know where the original errors are disappearing to though
:(.

Cheers,

Richard