All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Steve Sakoman" <steve@sakoman.com>
To: yocto@lists.yoctoproject.org
Subject: [yocto-autobuilder-helper][dunfell 12/23] scripts/run-config: Improve logfile naming
Date: Wed, 24 Mar 2021 14:39:21 -1000	[thread overview]
Message-ID: <0f8fe3adac44ef539c05ea503c9504a26b421984.1616625264.git.steve@sakoman.com> (raw)
In-Reply-To: <cover.1616625264.git.steve@sakoman.com>

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4a4c888f6618c3a7273c6dfe30b640e75e2b0de8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 scripts/run-config | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)

diff --git a/scripts/run-config b/scripts/run-config
index 116dd49..25a4296 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -131,11 +131,8 @@ def flush():
     sys.stdout.flush()
     sys.stderr.flush()
 
-lognum = 0
-def logname(path, stepnum, logsuffix):
-    global lognum
-    lognum += 1
-    return path + "/command.log.%s%s" % (stepnum, logsuffix)
+def logname(path, stepnum, stepname):
+    return path + "/command-%s-%s.log" % (stepnum, stepname)
 
 utils.mkdir(args.builddir)
 
@@ -146,10 +143,10 @@ utils.mkdir(errordir)
 
 errorlogs = set()
 
-def bitbakecmd(builddir, cmd, report, stepnum, logsuffix, oeenv=True):
+def bitbakecmd(builddir, cmd, report, stepnum, stepname, oeenv=True):
     global finalret
     flush()
-    log = logname(builddir, stepnum, logsuffix)
+    log = logname(builddir, stepnum, stepname)
     errordir = utils.errorreportdir(builddir)
     try:
         numreports = len(os.listdir(errordir))
@@ -176,10 +173,9 @@ def bitbakecmd(builddir, cmd, report, stepnum, logsuffix, oeenv=True):
 
     with subprocess.Popen(cmd, shell=True, cwd=builddir + "/..", stdout=subprocess.PIPE, stderr=subprocess.STDOUT, bufsize=1) as p, open(log, 'ab') as f:
         for line in p.stdout:
-            if not args.quietlogging:
-                sys.stdout.buffer.write(line)
-                sys.stdout.flush()
+            sys.stdout.buffer.write(line)
             f.write(line)
+            sys.stdout.flush()
             f.flush()
         ret = p.wait()
     if ret:
@@ -221,7 +217,7 @@ def handle_stepnum(stepnum):
             jsonconfig.append({"name" : "add-layers", "description" : "Adding layers %s" % str(layers), "phase" : str(stepnum)})
     elif args.stepname == "add-layers":
         for layer in layers:
-            bitbakecmd(args.builddir, "bitbake-layers add-layer %s" % layer, report, stepnum, 'a')
+            bitbakecmd(args.builddir, "bitbake-layers add-layer %s" % layer, report, stepnum, args.stepname)
 
     flush()
 
@@ -239,7 +235,7 @@ def handle_stepnum(stepnum):
             jsonconfig.append({"name" : "build-targets", "description" : "Building targets %s" % str(targets), "phase" : str(stepnum)})
         elif args.stepname == "build-targets":
             hp.printheader("Step %s/%s: Running bitbake %s" % (stepnum, maxsteps, targets))
-            bitbakecmd(args.builddir, "bitbake %s -k" % targets, report, stepnum, 'b')
+            bitbakecmd(args.builddir, "bitbake %s -k" % targets, report, stepnum, args.stepname)
 
     # Execute the sanity targets for this configuration
     sanitytargets = utils.getconfigvar("SANITYTARGETS", ourconfig, args.target, stepnum)
@@ -248,7 +244,7 @@ def handle_stepnum(stepnum):
             jsonconfig.append({"name" : "test-targets", "description" : "Running OEQA test targets %s" % str(sanitytargets), "phase" : str(stepnum)})
         elif args.stepname == "test-targets":
             hp.printheader("Step %s/%s: Running bitbake %s" % (stepnum, maxsteps, sanitytargets))
-            bitbakecmd(args.builddir, "%s/checkvnc; DISPLAY=:1 bitbake %s -k" % (scriptsdir, sanitytargets), report, stepnum, 'c')
+            bitbakecmd(args.builddir, "%s/checkvnc; DISPLAY=:1 bitbake %s -k" % (scriptsdir, sanitytargets), report, stepnum, args.stepname)
 
     # Run any extra commands specified
     cmds = utils.getconfiglist("EXTRACMDS", ourconfig, args.target, stepnum)
@@ -258,7 +254,7 @@ def handle_stepnum(stepnum):
     elif args.stepname == "cmds":
         for cmd in cmds:
             hp.printheader("Step %s/%s: Running command %s" % (stepnum, maxsteps, cmd))
-            bitbakecmd(args.builddir, cmd, report, stepnum, 'd')
+            bitbakecmd(args.builddir, cmd, report, stepnum, args.stepname)
 
     cmds = utils.getconfiglist("EXTRAPLAINCMDS", ourconfig, args.target, stepnum)
     if jcfg:
@@ -267,7 +263,7 @@ def handle_stepnum(stepnum):
     elif args.stepname == "plain-cmds":
         for cmd in cmds:
             hp.printheader("Step %s/%s: Running 'plain' command %s" % (stepnum, maxsteps, cmd))
-            bitbakecmd(args.builddir, cmd, report, stepnum, 'd', oeenv=False)
+            bitbakecmd(args.builddir, cmd, report, stepnum, args.stepname, oeenv=False)
 
     if jcfg:
         if layers:
@@ -275,7 +271,7 @@ def handle_stepnum(stepnum):
     elif args.stepname == "remove-layers":
         # Remove any layers we added in a reverse order
         for layer in reversed(layers):
-            bitbakecmd(args.builddir, "bitbake-layers remove-layer %s" % layer, report, stepnum, 'a')
+            bitbakecmd(args.builddir, "bitbake-layers remove-layer %s" % layer, report, stepnum, args.stepname)
 
     if not jcfg:
         sys.exit(finalret)
-- 
2.25.1


  parent reply	other threads:[~2021-03-25  0:40 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-25  0:39 [yocto-autobuilder-helper][dunfell 00/23] Patch review Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 01/23] scripts: Add runqemu-renice.c for renicing runqemu Steve Sakoman
2021-03-25 14:20   ` [yocto] " Richard Purdie
2021-03-25 15:31     ` Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 02/23] scripts/generate-testresult-index: Update after 'posttrigger' renaming broke the index generation Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 03/23] scripts/generate-testresult-index: Ensure backwards compatibility with older layout Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 04/23] scripts/generate-testresult-index.py: Ensure we're not always rerunning resulttool Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 05/23] scripts/generate-testresult-index: Improve index to list test reports, ptest and buildperf separately Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 06/23] scripts/generate-testresult-index: Reorder buildhistory to improve display Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 07/23] scripts/generate-testresult-index.py: Use bulma css to improve the look of the index Steve Sakoman
2021-03-25 14:04   ` [yocto] " Richard Purdie
2021-03-25 14:10     ` Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 08/23] scripts/run-config: Don't execute steps that don't exist! Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 09/23] scripts/run-config: Ensure stepnum has a value when there are no steps Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 10/23] scripts/run-config: If target is present default to 1 step Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 11/23] run-config: Adapt to two pass execution Steve Sakoman
2021-03-25  0:39 ` Steve Sakoman [this message]
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 13/23] scripts/run-config: Ensure logging to both logfile and stdout Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 14/23] config.json/run-config: Add human readable descriptions of steps Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 15/23] scripts/run-config: Remove redundant boilerplate json Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 16/23] scripts/shared-repo-unpack: Add flush call to update the output more regularly before buildtools Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 17/23] config.json/run-config: Add support for shortnames and descriptions Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 18/23] config.json: Unbreak qa-extras locked sigs test Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 19/23] config.json: Add further descriptions Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 20/23] config.json: Use buildtools tarball on debian9 Steve Sakoman
2021-03-25 14:02   ` [yocto] " Richard Purdie
2021-03-25 14:07     ` Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 21/23] scripts/run-config: Disable output buffering Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 22/23] config.json: Split reproduciblity tests into their own target Steve Sakoman
2021-03-25  0:39 ` [yocto-autobuilder-helper][dunfell 23/23] config: build and test SDKs when using package_deb Steve Sakoman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0f8fe3adac44ef539c05ea503c9504a26b421984.1616625264.git.steve@sakoman.com \
    --to=steve@sakoman.com \
    --cc=yocto@lists.yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.