* [PATCH 0/3] Fixes for tinfoil usage issues
@ 2017-03-27 0:17 Paul Eggleton
2017-03-27 0:17 ` [PATCH 1/3] fetch2/git: prevent recursion on getting latest revision Paul Eggleton
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Paul Eggleton @ 2017-03-27 0:17 UTC (permalink / raw)
To: bitbake-devel
Fixes for a recursion issue triggered by RSS in OE, and a logging issue
caused by the tinfoil2 rework.
The following changes since commit 751b06c25d22eea8240f9429cb49874082245e52:
bitbake-diffsigs: Add debug support (2017-03-22 14:52:39 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib paule/tinfoil-fixes-bb2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/tinfoil-fixes-bb2
Paul Eggleton (3):
fetch2/git: prevent recursion on getting latest revision
lib/bb/msg: introduce logger_create() function
tinfoil: enable client-side logger handling by default
bin/bitbake-diffsigs | 14 ++------------
bin/bitbake-dumpsig | 14 ++------------
bin/bitbake-layers | 25 +++++--------------------
lib/bb/fetch2/git.py | 28 ++++++++++++++++++++--------
lib/bb/msg.py | 15 +++++++++++++++
lib/bb/tinfoil.py | 6 +++++-
6 files changed, 49 insertions(+), 53 deletions(-)
--
2.9.3
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/3] fetch2/git: prevent recursion on getting latest revision
2017-03-27 0:17 [PATCH 0/3] Fixes for tinfoil usage issues Paul Eggleton
@ 2017-03-27 0:17 ` Paul Eggleton
2017-03-27 0:17 ` [PATCH 2/3] lib/bb/msg: introduce logger_create() function Paul Eggleton
2017-03-27 0:17 ` [PATCH 3/3] tinfoil: enable client-side logger handling by default Paul Eggleton
2 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2017-03-27 0:17 UTC (permalink / raw)
To: bitbake-devel
We call git ls-remote to get the latest revision from a git repository,
however by calling runfetchcmd() we can end up recursively running
git ls-remote a number of times with OE e.g. if ${SRCPV} is in PV, ${PV}
is in WORKDIR, and ${WORKDIR} is in PATH (as a result of recipe-specific
sysroots), our call to runfetchcmd() exports PATH so _lsremote() will
get called again - with the end result that we run git ls-remote 30
times in quick succession (!). Prevent that from happening by using a
guard variable and returning a dummy value if it's called recursively.
Fixes [YOCTO #11185].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
lib/bb/fetch2/git.py | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index a8be859..2550bde 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -379,14 +379,26 @@ class Git(FetchMethod):
"""
Run git ls-remote with the specified search string
"""
- repourl = self._get_repo_url(ud)
- cmd = "%s ls-remote %s %s" % \
- (ud.basecmd, repourl, search)
- if ud.proto.lower() != 'file':
- bb.fetch2.check_network_access(d, cmd, repourl)
- output = runfetchcmd(cmd, d, True)
- if not output:
- raise bb.fetch2.FetchError("The command %s gave empty output unexpectedly" % cmd, ud.url)
+ # Prevent recursion e.g. in OE if SRCPV is in PV, PV is in WORKDIR,
+ # and WORKDIR is in PATH (as a result of RSS), our call to
+ # runfetchcmd() exports PATH so this function will get called again (!)
+ # In this scenario the return call of the function isn't actually
+ # important - WORKDIR isn't needed in PATH to call git ls-remote
+ # anyway.
+ if d.getVar('_BB_GIT_IN_LSREMOTE', False):
+ return ''
+ d.setVar('_BB_GIT_IN_LSREMOTE', '1')
+ try:
+ repourl = self._get_repo_url(ud)
+ cmd = "%s ls-remote %s %s" % \
+ (ud.basecmd, repourl, search)
+ if ud.proto.lower() != 'file':
+ bb.fetch2.check_network_access(d, cmd, repourl)
+ output = runfetchcmd(cmd, d, True)
+ if not output:
+ raise bb.fetch2.FetchError("The command %s gave empty output unexpectedly" % cmd, ud.url)
+ finally:
+ d.delVar('_BB_GIT_IN_LSREMOTE')
return output
def _latest_revision(self, ud, d, name):
--
2.9.3
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/3] lib/bb/msg: introduce logger_create() function
2017-03-27 0:17 [PATCH 0/3] Fixes for tinfoil usage issues Paul Eggleton
2017-03-27 0:17 ` [PATCH 1/3] fetch2/git: prevent recursion on getting latest revision Paul Eggleton
@ 2017-03-27 0:17 ` Paul Eggleton
2017-03-27 0:17 ` [PATCH 3/3] tinfoil: enable client-side logger handling by default Paul Eggleton
2 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2017-03-27 0:17 UTC (permalink / raw)
To: bitbake-devel
We use this code to set up a logger with colour in a number of different
places, so create one function that does this and make some of bitbake's
utility scripts use it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
bin/bitbake-diffsigs | 14 ++------------
bin/bitbake-dumpsig | 14 ++------------
bin/bitbake-layers | 25 +++++--------------------
lib/bb/msg.py | 15 +++++++++++++++
4 files changed, 24 insertions(+), 44 deletions(-)
diff --git a/bin/bitbake-diffsigs b/bin/bitbake-diffsigs
index c087f99..1e3de09 100755
--- a/bin/bitbake-diffsigs
+++ b/bin/bitbake-diffsigs
@@ -30,19 +30,9 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
import bb.tinfoil
import bb.siggen
+import bb.msg
-def logger_create(name, output=sys.stderr):
- logger = logging.getLogger(name)
- console = logging.StreamHandler(output)
- format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
- if output.isatty():
- format.enable_color()
- console.setFormatter(format)
- logger.addHandler(console)
- logger.setLevel(logging.INFO)
- return logger
-
-logger = logger_create('bitbake-diffsigs')
+logger = bb.msg.logger_create('bitbake-diffsigs')
def find_compare_task(bbhandler, pn, taskname):
""" Find the most recent signature files for the specified PN/task and compare them """
diff --git a/bin/bitbake-dumpsig b/bin/bitbake-dumpsig
index 38efd22..95ebd93 100755
--- a/bin/bitbake-dumpsig
+++ b/bin/bitbake-dumpsig
@@ -29,19 +29,9 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
import bb.tinfoil
import bb.siggen
+import bb.msg
-def logger_create(name, output=sys.stderr):
- logger = logging.getLogger(name)
- console = logging.StreamHandler(output)
- format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
- if output.isatty():
- format.enable_color()
- console.setFormatter(format)
- logger.addHandler(console)
- logger.setLevel(logging.INFO)
- return logger
-
-logger = logger_create('bitbake-dumpsig')
+logger = bb.msg.logger_create('bitbake-dumpsig')
def find_siginfo_task(bbhandler, pn, taskname):
""" Find the most recent signature file for the specified PN/task """
diff --git a/bin/bitbake-layers b/bin/bitbake-layers
index 66fc7ca..390ad05 100755
--- a/bin/bitbake-layers
+++ b/bin/bitbake-layers
@@ -30,26 +30,9 @@ topdir = os.path.dirname(bindir)
sys.path[0:0] = [os.path.join(topdir, 'lib')]
import bb.tinfoil
+import bb.msg
-def logger_create(name, output=sys.stderr):
- logger = logging.getLogger(name)
- loggerhandler = logging.StreamHandler(output)
- loggerhandler.setFormatter(logging.Formatter("%(levelname)s: %(message)s"))
- logger.addHandler(loggerhandler)
- logger.setLevel(logging.INFO)
- return logger
-
-def logger_setup_color(logger, color='auto'):
- from bb.msg import BBLogFormatter
- console = logging.StreamHandler(sys.stdout)
- formatter = BBLogFormatter("%(levelname)s: %(message)s")
- console.setFormatter(formatter)
- logger.handlers = [console]
- if color == 'always' or (color == 'auto' and console.stream.isatty()):
- formatter.enable_color()
-
-
-logger = logger_create('bitbake-layers', sys.stdout)
+logger = bb.msg.logger_create('bitbake-layers', sys.stdout)
def main():
parser = argparse.ArgumentParser(
@@ -74,7 +57,9 @@ def main():
elif global_args.quiet:
logger.setLevel(logging.ERROR)
- logger_setup_color(logger, global_args.color)
+ # Need to re-run logger_create with color argument
+ # (will be the same logger since it has the same name)
+ bb.msg.logger_create('bitbake-layers', output=sys.stdout, color=global_args.color)
plugins = []
tinfoil = bb.tinfoil.Tinfoil(tracking=True)
diff --git a/lib/bb/msg.py b/lib/bb/msg.py
index b7c39fa..90b1582 100644
--- a/lib/bb/msg.py
+++ b/lib/bb/msg.py
@@ -201,3 +201,18 @@ def fatal(msgdomain, msg):
logger = logging.getLogger("BitBake")
logger.critical(msg)
sys.exit(1)
+
+def logger_create(name, output=sys.stderr, level=logging.INFO, preserve_handlers=False, color='auto'):
+ """Standalone logger creation function"""
+ logger = logging.getLogger(name)
+ console = logging.StreamHandler(output)
+ format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+ if color == 'always' or (color == 'auto' and output.isatty()):
+ format.enable_color()
+ console.setFormatter(format)
+ if preserve_handlers:
+ logger.addHandler(console)
+ else:
+ logger.handlers = [console]
+ logger.setLevel(level)
+ return logger
--
2.9.3
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/3] tinfoil: enable client-side logger handling by default
2017-03-27 0:17 [PATCH 0/3] Fixes for tinfoil usage issues Paul Eggleton
2017-03-27 0:17 ` [PATCH 1/3] fetch2/git: prevent recursion on getting latest revision Paul Eggleton
2017-03-27 0:17 ` [PATCH 2/3] lib/bb/msg: introduce logger_create() function Paul Eggleton
@ 2017-03-27 0:17 ` Paul Eggleton
2017-03-27 7:02 ` Paul Eggleton
2 siblings, 1 reply; 5+ messages in thread
From: Paul Eggleton @ 2017-03-27 0:17 UTC (permalink / raw)
To: bitbake-devel
If you had a script that uses tinfoil and it failed to connect to the
BitBake server, you did't see any of the expected messages - this was
because client-side logging wasn't being handled at all. Since you'll
almost always want this when using tinfoil, have it use the new
bb.msg.logger_create() function to enable client-side logging by
default.
Relates to [YOCTO #11185].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
lib/bb/tinfoil.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/bb/tinfoil.py b/lib/bb/tinfoil.py
index 19b41be..c2ee707 100644
--- a/lib/bb/tinfoil.py
+++ b/lib/bb/tinfoil.py
@@ -210,13 +210,17 @@ class TinfoilCookerAdapter:
class Tinfoil:
- def __init__(self, output=sys.stdout, tracking=False):
+ def __init__(self, output=sys.stdout, tracking=False, setup_logging=True):
self.logger = logging.getLogger('BitBake')
self.config_data = None
self.cooker = None
self.tracking = tracking
self.ui_module = None
self.server_connection = None
+ if setup_logging:
+ # This is the *client-side* logger, nothing to do with
+ # logging messages from the server
+ bb.msg.logger_create('BitBake', output)
def __enter__(self):
return self
--
2.9.3
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 3/3] tinfoil: enable client-side logger handling by default
2017-03-27 0:17 ` [PATCH 3/3] tinfoil: enable client-side logger handling by default Paul Eggleton
@ 2017-03-27 7:02 ` Paul Eggleton
0 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2017-03-27 7:02 UTC (permalink / raw)
To: bitbake-devel
On Monday, 27 March 2017 1:17:28 PM NZDT Paul Eggleton wrote:
> If you had a script that uses tinfoil and it failed to connect to the
> BitBake server, you did't see any of the expected messages - this was
> because client-side logging wasn't being handled at all. Since you'll
> almost always want this when using tinfoil, have it use the new
> bb.msg.logger_create() function to enable client-side logging by
> default.
Sigh... this causes a bunch of debug messages from the codeparser to get shown
when using devtool add. I am still tracking that issue down - perhaps it's
better than not seeing any messages at all though.
- Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-03-27 7:02 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-27 0:17 [PATCH 0/3] Fixes for tinfoil usage issues Paul Eggleton
2017-03-27 0:17 ` [PATCH 1/3] fetch2/git: prevent recursion on getting latest revision Paul Eggleton
2017-03-27 0:17 ` [PATCH 2/3] lib/bb/msg: introduce logger_create() function Paul Eggleton
2017-03-27 0:17 ` [PATCH 3/3] tinfoil: enable client-side logger handling by default Paul Eggleton
2017-03-27 7:02 ` Paul Eggleton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.