[bitbake][dunfell][1.46][PATCH 00/14] Patch review

[bitbake][dunfell][1.46][PATCH 00/14] Patch review

[Steve Sakoman]

Please review this next set of patches for 1.46/dunfell and have comments
back by end of day Tuesday. Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/2639

The following changes since commit c2a3bda3a29e12472ef7862e424ea1552fab2959:

  providers: replace newly added logger.warn() with logger.warning() (2021-08-02 21:34:27 +0100)

are available in the Git repository at:

  git://git.openembedded.org/bitbake-contrib stable/1.46-nut
  http://cgit.openembedded.org/bitbake-contrib/log/?h=stable/1.46-nut

Joshua Watt (1):
  server: Fix early parsing errors preventing zombie bitbake

Marco Felsch (1):
  bitbake: bitbake-layers: add skip reason to output

Richard Purdie (12):
  ui/taskexp: Improve startup exception handling
  ui/taskexp: Fix to work with empty build directories
  build: Match markup to real function name
  build: Handle SystemExit in python tasks correctly
  process: Don't include logs in error message if piping them
  build: Avoid duplicating logs in verbose mode
  build: Catch and error upon circular task references
  data_smart: Improve error display for handled exceptions
  cookerdata: Improve missing core layer error message
  cookerdata: Show error for no BBLAYERS in bblayers.conf
  bitbake-worker: Improve error handling
  cookerdata: Show a readable error for invalid multiconfig name

 bin/bitbake-worker       | 10 ++++++----
 lib/bb/build.py          | 12 +++++++++---
 lib/bb/cookerdata.py     |  7 +++++++
 lib/bb/data_smart.py     |  2 ++
 lib/bb/process.py        |  3 +++
 lib/bb/server/process.py |  3 ++-
 lib/bb/ui/taskexp.py     |  5 +++++
 lib/bblayers/query.py    |  2 +-
 8 files changed, 35 insertions(+), 9 deletions(-)

-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 01/14] server: Fix early parsing errors preventing zombie bitbake

[Steve Sakoman]

From: Joshua Watt <JPEWhacker@gmail.com>

If the client process never sends cooker data, the server timeout will
be 0.0, not None. This will prevent the server from exiting, as it is
waiting for a new client. In particular, the client will disconnect with
a bad "INHERIT" line, such as:

    INHERIT += "this-class-does-not-exist"

Instead of checking explicitly for None, check for a false value, which
means either 0.0 or None.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 13e2855bff6a6ead6dbd33c5be4b988aafcd4afa)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/server/process.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py
index b66fbe0a..45f2e863 100644
--- a/lib/bb/server/process.py
+++ b/lib/bb/server/process.py
@@ -152,7 +152,8 @@ class ProcessServer(multiprocessing.Process):
                 conn = newconnections.pop(-1)
                 fds.append(conn)
                 self.controllersock = conn
-            elif self.timeout is None and not ready:
+
+            elif not self.timeout and not ready:
                 print("No timeout, exiting.")
                 self.quit = True
 
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 02/14] ui/taskexp: Improve startup exception handling

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

When an exception occurs at startup, show it to the user.

[YOCTO #14408]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cc1df1af67cfd3e223b39e2b7ea5f86b8cf78aee)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/ui/taskexp.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/bb/ui/taskexp.py b/lib/bb/ui/taskexp.py
index 2b246710..81392977 100644
--- a/lib/bb/ui/taskexp.py
+++ b/lib/bb/ui/taskexp.py
@@ -8,6 +8,7 @@
 #
 
 import sys
+import traceback
 
 try:
     import gi
@@ -218,6 +219,9 @@ def main(server, eventHandler, params):
     except client.Fault as x:
         print("XMLRPC Fault getting commandline:\n %s" % x)
         return
+    except Exception as e:
+        print("Exception in startup:\n %s" % traceback.format_exc())
+        return
 
     if gtkthread.quit.isSet():
         return
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 03/14] ui/taskexp: Fix to work with empty build directories

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If run on an empty build directory, taskexp wasn't working as it didn't
send the current environment to the server. This means HOSTTOOLS in oe-core
couldn't be built and gave an error. Add the missing updateToServer call in.

[YOCTO #14408]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 06a0bbe746f879ae539223e7fdb6f07d55d13719)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/ui/taskexp.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/bb/ui/taskexp.py b/lib/bb/ui/taskexp.py
index 81392977..c00eaf66 100644
--- a/lib/bb/ui/taskexp.py
+++ b/lib/bb/ui/taskexp.py
@@ -197,6 +197,7 @@ def main(server, eventHandler, params):
     gtkgui.start()
 
     try:
+        params.updateToServer(server, os.environ.copy())
         params.updateFromServer(server)
         cmdline = params.parseActions()
         if not cmdline:
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 04/14] bitbake: bitbake-layers: add skip reason to output

[Steve Sakoman]

From: Marco Felsch <m.felsch@pengutronix.de>

Currently we inform the user that some package/layer is skipped but we
don't print the reason albeit bitbake knows the reason. So currently it
looks like:

gtk+:
  meta-oe              2.24.32 (skipped)

With this change the output prints the skip reason which is very helpful
for debugging:

gtk+:
  meta-oe              2.24.32 (skipped: one of 'x11 directfb' needs to be in DISTRO_FEATURES)

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d43e72db4f7c8b47d91d99ed54ce30e9ee898de1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bblayers/query.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bblayers/query.py b/lib/bblayers/query.py
index e2cc3105..fb6f550e 100644
--- a/lib/bblayers/query.py
+++ b/lib/bblayers/query.py
@@ -150,7 +150,7 @@ skipped recipes will also be listed, with a " (skipped)" suffix.
         def print_item(f, pn, ver, layer, ispref):
             if not selected_layer or layer == selected_layer:
                 if not bare and f in skiplist:
-                    skipped = ' (skipped)'
+                    skipped = ' (skipped: %s)' % self.tinfoil.cooker.skiplist[f].skipreason
                 else:
                     skipped = ''
                 if show_filenames:
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 05/14] build: Match markup to real function name

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

The point of the injected text is to identify where the function comes from. Using
the correct function name would therefore be better.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 30c6ff8551c235254ab90663ab88f66bb0c71edb)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/build.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/bb/build.py b/lib/bb/build.py
index 23b6ee45..fa22a1b4 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -290,8 +290,8 @@ def exec_func_python(func, d, runfile, cwd=None):
         lineno = int(d.getVarFlag(func, "lineno", False))
         bb.methodpool.insert_method(func, text, fn, lineno - 1)
 
-        comp = utils.better_compile(code, func, "exec_python_func() autogenerated")
-        utils.better_exec(comp, {"d": d}, code, "exec_python_func() autogenerated")
+        comp = utils.better_compile(code, func, "exec_func_python() autogenerated")
+        utils.better_exec(comp, {"d": d}, code, "exec_func_python() autogenerated")
     finally:
         bb.debug(2, "Python function %s finished" % func)
 
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 06/14] build: Handle SystemExit in python tasks correctly

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If a python task fails with sys.exit(), we currently see no TaskFailed event.
The high level code does detect the exit code and fail the task but it can
leave the UI inconsistent with log output.

Fix this be intercepting SystemExit explicitly. This makes python
task failures consistent with shell task failures.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9eee9fd4f2f96789ad2b037e74d561bdc1426856)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/build.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bb/build.py b/lib/bb/build.py
index fa22a1b4..04971636 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -587,7 +587,7 @@ def _exec_task(fn, task, d, quieterr):
         except bb.BBHandledException:
             event.fire(TaskFailed(task, fn, logfn, localdata, True), localdata)
             return 1
-        except Exception as exc:
+        except (Exception, SystemExit) as exc:
             if quieterr:
                 event.fire(TaskFailedSilent(task, fn, logfn, localdata), localdata)
             else:
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 07/14] process: Don't include logs in error message if piping them

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If the caller is piping the logs, they likely don't want them in the error exception
as well. This removes duplicate output from the build output allowing the UI level
controls on whether to show logs to work correctly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fc58ad84a9deb2620ad90611684dad65dafedb11)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/process.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/bb/process.py b/lib/bb/process.py
index 2dc472a8..24c588e5 100644
--- a/lib/bb/process.py
+++ b/lib/bb/process.py
@@ -179,5 +179,8 @@ def run(cmd, input=None, log=None, extrafiles=None, **options):
             stderr = stderr.decode("utf-8")
 
     if pipe.returncode != 0:
+        if log:
+            # Don't duplicate the output in the exception if logging it
+            raise ExecutionError(cmd, pipe.returncode, None, None)
         raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
     return stdout, stderr
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 08/14] build: Avoid duplicating logs in verbose mode

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

With "bitbake -v", for task failures you'd see the log output twice. Avoid
this by using the existing "did we print info" switch.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/build.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/bb/build.py b/lib/bb/build.py
index 04971636..6e9c064e 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -592,6 +592,10 @@ def _exec_task(fn, task, d, quieterr):
                 event.fire(TaskFailedSilent(task, fn, logfn, localdata), localdata)
             else:
                 errprinted = errchk.triggered
+                # If the output is already on stdout, we've printed the information in the
+                # logs once already so don't duplicate
+                if verboseStdoutLogging:
+                    errprinted = True
                 logger.error(str(exc))
                 event.fire(TaskFailed(task, fn, logfn, localdata, errprinted), localdata)
             return 1
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 09/14] build: Catch and error upon circular task references

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If there are circular task references, error on them rather than show
a recursion error. A simple reproducer is:

"""
do_packageswu () {
       :
}

addtask do_packageswu after do_image_complete before do_image_qa
"""

into image_types.bbclass. There is code in runqueue to detect these but
we never get that far with the current codebase.

[YOCTO #13140]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 339d4d6be515a71311b81fb9e99742af0d8a5130)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/build.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/build.py b/lib/bb/build.py
index 6e9c064e..17c70d0a 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -905,6 +905,8 @@ def tasksbetween(task_start, task_end, d):
     def follow_chain(task, endtask, chain=None):
         if not chain:
             chain = []
+        if task in chain:
+            bb.fatal("Circular task dependencies as %s depends on itself via the chain %s" % (task, " -> ".join(chain)))
         chain.append(task)
         for othertask in tasks:
             if othertask == task:
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 10/14] data_smart: Improve error display for handled exceptions

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

We don't need tracebacks for BBHandledException. Reduces confusing output like:

ERROR: /meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio
ERROR: ExpansionError during parsing /meta/recipes-core/images/core-image-tiny-initramfs.bb
Traceback (most recent call last):
  File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_qa', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
                     if task in deps:
    >                    follow_chain(othertask, endtask, chain)
             chain.pop()
  File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
                     if task in deps:
    >                    follow_chain(othertask, endtask, chain)
             chain.pop()
  File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_cpio', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
                     if task in deps:
    >                    follow_chain(othertask, endtask, chain)
             chain.pop()
  File "/bitbake/lib/bb/build.py", line 1038, in follow_chain(task='do_image_complete', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']):
             if task in chain:
    >            bb.fatal("Circular task dependencies as %s depends itself via the chain %s?!" % (task, " -> ".join(chain)))
             chain.append(task)
  File "/bitbake/lib/bb/__init__.py", line 165, in fatal:
         mainlogger.critical(''.join(args), extra=kwargs)
    >    raise BBHandledException()

to the real error:

ERROR: /media/build1/poky/meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 551d4c0576a0a0c3406000029df9238b312f2263)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/data_smart.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 61b37cf3..68770832 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -411,6 +411,8 @@ class DataSmart(MutableMapping):
                 raise
             except bb.parse.SkipRecipe:
                 raise
+            except bb.BBHandledException:
+                raise
             except Exception as exc:
                 tb = sys.exc_info()[2]
                 raise ExpansionError(varname, s, exc).with_traceback(tb) from exc
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 11/14] cookerdata: Improve missing core layer error message

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If the core layer is missing from bblayers.conf, the message the user sees is
hard to understand. Improve it.

[YOCTO #14340]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5815a7258ebb8a989e0c6f5798853559d9413f02)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/cookerdata.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 472423fd..44af7b2a 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -399,6 +399,8 @@ class CookerDataBuilder(object):
                 if c in collections_tmp:
                     bb.fatal("Found duplicated BBFILE_COLLECTIONS '%s', check bblayers.conf or layer.conf to fix it." % c)
                 compat = set((data.getVar("LAYERSERIES_COMPAT_%s" % c) or "").split())
+                if compat and not layerseries:
+                    bb.fatal("No core layer found to work with layer '%s'. Missing entry in bblayers.conf?" % c)
                 if compat and not (compat & layerseries):
                     bb.fatal("Layer %s is not compatible with the core layer which only supports these series: %s (layer is compatible with %s)"
                               % (c, " ".join(layerseries), " ".join(compat)))
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 12/14] cookerdata: Show error for no BBLAYERS in bblayers.conf

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If there is no BBLAYERS set in bblayers.conf show a more helpful
error and exit.

[YOCTO #14340]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 97183e10faf9862b5d9489d6e2c27ac77c3b697d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/cookerdata.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 44af7b2a..26b9b0b4 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -348,6 +348,9 @@ class CookerDataBuilder(object):
             layers = (data.getVar('BBLAYERS') or "").split()
             broken_layers = []
 
+            if not layers:
+                bb.fatal("The bblayers.conf file doesn't contain any BBLAYERS definition")
+
             data = bb.data.createCopy(data)
             approved = bb.utils.approved_variables()
 
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 13/14] bitbake-worker: Improve error handling

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If bitbake-worker fails, return an error code showing that. Also
make the thread cleanup code explict in a finally clause as it would
otherwise hang.

[YOCTO #14393]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7e0af70fb53fb13f824ca954b8cc1dffee730233)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 bin/bitbake-worker | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/bin/bitbake-worker b/bin/bitbake-worker
index 97cc0fd6..9ef0d5a6 100755
--- a/bin/bitbake-worker
+++ b/bin/bitbake-worker
@@ -505,9 +505,11 @@ except BaseException as e:
         import traceback
         sys.stderr.write(traceback.format_exc())
         sys.stderr.write(str(e))
+finally:
+    worker_thread_exit = True
+    worker_thread.join()
 
-worker_thread_exit = True
-worker_thread.join()
-
-workerlog_write("exitting")
+workerlog_write("exiting")
+if not normalexit:
+    sys.exit(1)
 sys.exit(0)
-- 
2.25.1

[bitbake][dunfell][1.46][PATCH 14/14] cookerdata: Show a readable error for invalid multiconfig name

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If a multiconfig starts with a digit, users would see pages of
errors as we use the multiconfig as a python function name prefix
and python functions cannot start with a digit. We could avoid doing
that but it is easier just to ask users to name multiconfigs not
starting with digits.

This tweak ensures the user sees an easier to understand error.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f9cddaeef35b2ea0dadf717101ed896f6b857abd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/cookerdata.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 26b9b0b4..32a8798f 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -297,6 +297,8 @@ class CookerDataBuilder(object):
 
             multiconfig = (self.data.getVar("BBMULTICONFIG") or "").split()
             for config in multiconfig:
+                if config[0].isdigit():
+                    bb.fatal("Multiconfig name '%s' is invalid as multiconfigs cannot start with a digit" % config)
                 mcdata = self.parseConfigurationFiles(self.prefiles, self.postfiles, config)
                 bb.event.fire(bb.event.ConfigParsed(), mcdata)
                 self.mcdata[config] = mcdata
-- 
2.25.1

Re: [bitbake-devel] [bitbake][dunfell][1.46][PATCH 08/14] build: Avoid duplicating logs in verbose mode

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3474 bytes --]

I'm sorry for late review, but this patch won't work in 1.46 without this
commit from 1.48:

  commit 423c046f2173aaff3072dc3d0882d01b8a0b0212
  Author: Richard Purdie <richard.purdie@linuxfoundation.org>
  Date:   Mon Aug 24 12:28:27 2020 +0100

    build/msg: Cleanup verbose option handling

which defines verboseStdoutLogging

Without this when task fails it shows completely wrong error output e.g.:

ERROR: sysdig-0.26.6-r0 do_compile: Build of do_compile failed
ERROR: sysdig-0.26.6-r0 do_compile: Traceback (most recent call last):
  File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line
584, in _exec_task
    exec_func(task, localdata)
  File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line
251, in exec_func
    exec_func_shell(func, d, runfile, cwd=adir)
  File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line
452, in exec_func_shell
    bb.process.run(cmd, shell=False, stdin=stdin, log=logfile,
extrafiles=[(fifo,readfifo)])
  File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/process.py",
line 184, in run
    raise ExecutionError(cmd, pipe.returncode, None, None)
bb.process.ExecutionError: Execution of
'/jenkins/mjansa/build/webos/dunfell/BUILD/work/qemux86-webos-linux/sysdig/0.26.6-r0/temp/run.do_compile.69208'
failed with exit code 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line
649, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line
597, in _exec_task
    if verboseStdoutLogging:
NameError: name 'verboseStdoutLogging' is not defined

On Fri, Sep 24, 2021 at 4:22 PM Steve Sakoman <steve@sakoman.com> wrote:

> From: Richard Purdie <richard.purdie@linuxfoundation.org>
>
> With "bitbake -v", for task failures you'd see the log output twice. Avoid
> this by using the existing "did we print info" switch.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> (cherry picked from commit e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8)
> Signed-off-by: Steve Sakoman <steve@sakoman.com>
> ---
>  lib/bb/build.py | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/lib/bb/build.py b/lib/bb/build.py
> index 04971636..6e9c064e 100644
> --- a/lib/bb/build.py
> +++ b/lib/bb/build.py
> @@ -592,6 +592,10 @@ def _exec_task(fn, task, d, quieterr):
>                  event.fire(TaskFailedSilent(task, fn, logfn, localdata),
> localdata)
>              else:
>                  errprinted = errchk.triggered
> +                # If the output is already on stdout, we've printed the
> information in the
> +                # logs once already so don't duplicate
> +                if verboseStdoutLogging:
> +                    errprinted = True
>                  logger.error(str(exc))
>                  event.fire(TaskFailed(task, fn, logfn, localdata,
> errprinted), localdata)
>              return 1
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12697):
> https://lists.openembedded.org/g/bitbake-devel/message/12697
> Mute This Topic: https://lists.openembedded.org/mt/85840116/3617156
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 4989 bytes --]

Re: [bitbake-devel] [bitbake][dunfell][1.46][PATCH 08/14] build: Avoid duplicating logs in verbose mode

[Steve Sakoman]

On Thu, Sep 30, 2021 at 12:09 AM Martin Jansa <martin.jansa@gmail.com> wrote:
>
> I'm sorry for late review, but this patch won't work in 1.46 without this commit from 1.48:
>
>   commit 423c046f2173aaff3072dc3d0882d01b8a0b0212
>   Author: Richard Purdie <richard.purdie@linuxfoundation.org>
>   Date:   Mon Aug 24 12:28:27 2020 +0100
>
>     build/msg: Cleanup verbose option handling

Good catch!  Thanks for the review.

Richard hasn't taken the pull request yet, so I will add this patch,
test, and resubmit the series.

Thanks again!

Steve

>
> which defines verboseStdoutLogging
>
> Without this when task fails it shows completely wrong error output e.g.:
>
> ERROR: sysdig-0.26.6-r0 do_compile: Build of do_compile failed
> ERROR: sysdig-0.26.6-r0 do_compile: Traceback (most recent call last):
>   File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line 584, in _exec_task
>     exec_func(task, localdata)
>   File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line 251, in exec_func
>     exec_func_shell(func, d, runfile, cwd=adir)
>   File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line 452, in exec_func_shell
>     bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)])
>   File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/process.py", line 184, in run
>     raise ExecutionError(cmd, pipe.returncode, None, None)
> bb.process.ExecutionError: Execution of '/jenkins/mjansa/build/webos/dunfell/BUILD/work/qemux86-webos-linux/sysdig/0.26.6-r0/temp/run.do_compile.69208' failed with exit code 1
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>   File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line 649, in exec_task
>     return _exec_task(fn, task, d, quieterr)
>   File "/jenkins/mjansa/build/webos/dunfell/bitbake/lib/bb/build.py", line 597, in _exec_task
>     if verboseStdoutLogging:
> NameError: name 'verboseStdoutLogging' is not defined
>
> On Fri, Sep 24, 2021 at 4:22 PM Steve Sakoman <steve@sakoman.com> wrote:
>>
>> From: Richard Purdie <richard.purdie@linuxfoundation.org>
>>
>> With "bitbake -v", for task failures you'd see the log output twice. Avoid
>> this by using the existing "did we print info" switch.
>>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> (cherry picked from commit e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8)
>> Signed-off-by: Steve Sakoman <steve@sakoman.com>
>> ---
>>  lib/bb/build.py | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/lib/bb/build.py b/lib/bb/build.py
>> index 04971636..6e9c064e 100644
>> --- a/lib/bb/build.py
>> +++ b/lib/bb/build.py
>> @@ -592,6 +592,10 @@ def _exec_task(fn, task, d, quieterr):
>>                  event.fire(TaskFailedSilent(task, fn, logfn, localdata), localdata)
>>              else:
>>                  errprinted = errchk.triggered
>> +                # If the output is already on stdout, we've printed the information in the
>> +                # logs once already so don't duplicate
>> +                if verboseStdoutLogging:
>> +                    errprinted = True
>>                  logger.error(str(exc))
>>                  event.fire(TaskFailed(task, fn, logfn, localdata, errprinted), localdata)
>>              return 1
>> --
>> 2.25.1
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#12697): https://lists.openembedded.org/g/bitbake-devel/message/12697
>> Mute This Topic: https://lists.openembedded.org/mt/85840116/3617156
>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Martin.Jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>