[PATCH 1/2] tinfoil/data_smart: Allow variable history emit() to function remotely

[PATCH 1/2] tinfoil/data_smart: Allow variable history emit() to function remotely

[Richard Purdie]

We can't access the emit() function of varhistory currently as the datastore parameter
isn't handled correctly, nor is the output stream. Add a custom wrapper for this
function which handles the two details correctly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/command.py | 12 ++++++++++++
 lib/bb/tinfoil.py |  4 ++++
 2 files changed, 16 insertions(+)

diff --git a/lib/bb/command.py b/lib/bb/command.py
index dd77cdd6e2..f530cf844b 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -20,6 +20,7 @@ Commands are queued in a CommandQueue
 
 from collections import OrderedDict, defaultdict
 
+import io
 import bb.event
 import bb.cooker
 import bb.remotedata
@@ -500,6 +501,17 @@ class CommandsSync:
         d = command.remotedatastores[dsindex].varhistory
         return getattr(d, method)(*args, **kwargs)
 
+    def dataStoreConnectorVarHistCmdEmit(self, command, params):
+        dsindex = params[0]
+        var = params[1]
+        oval = params[2]
+        val = params[3]
+        d = command.remotedatastores[params[4]]
+
+        o = io.StringIO()
+        command.remotedatastores[dsindex].varhistory.emit(var, oval, val, o, d)
+        return o.getvalue()
+
     def dataStoreConnectorIncHistCmd(self, command, params):
         dsindex = params[0]
         method = params[1]
diff --git a/lib/bb/tinfoil.py b/lib/bb/tinfoil.py
index 796a98f053..27a341541a 100644
--- a/lib/bb/tinfoil.py
+++ b/lib/bb/tinfoil.py
@@ -52,6 +52,10 @@ class TinfoilDataStoreConnectorVarHistory:
     def remoteCommand(self, cmd, *args, **kwargs):
         return self.tinfoil.run_command('dataStoreConnectorVarHistCmd', self.dsindex, cmd, args, kwargs)
 
+    def emit(self, var, oval, val, o, d):
+        ret = self.tinfoil.run_command('dataStoreConnectorVarHistCmdEmit', self.dsindex, var, oval, val, d.dsindex)
+        o.write(ret)
+
     def __getattr__(self, name):
         if not hasattr(bb.data_smart.VariableHistory, name):
             raise AttributeError("VariableHistory has no such method %s" % name)
-- 
2.30.2

[PATCH 2/2] bin/bitbake-getvar: Add a new command to query a variable value (with history)

[Richard Purdie]

We've talked about having this for long enough. Add a command which queries a single
variable value with history. This saves "bitbake -e | grep" and avoids the
various pitfalls that has.

It also provides a neat example of using tinfoil to make such a query.

Parameters to limit the output to just the value, to limit to a variable flag
and to not expand the output are provided.

[YOCTO #10748]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 bin/bitbake-getvar | 48 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100755 bin/bitbake-getvar

diff --git a/bin/bitbake-getvar b/bin/bitbake-getvar
new file mode 100755
index 0000000000..0eddfe294b
--- /dev/null
+++ b/bin/bitbake-getvar
@@ -0,0 +1,48 @@
+#! /usr/bin/env python3
+#
+# Copyright (C) 2021 Richard Purdie
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import argparse
+import io
+import os
+import sys
+
+bindir = os.path.dirname(__file__)
+topdir = os.path.dirname(bindir)
+sys.path[0:0] = [os.path.join(topdir, 'lib')]
+
+import bb.tinfoil
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Bitbake Query Variable")
+    parser.add_argument("variable", help="variable name to query")
+    parser.add_argument("-r", "--recipe", help="Recipe name to query", default=None, required=False)
+    parser.add_argument('-u', '--unexpand', help='Do not expand the value (with --value)', action="store_true")
+    parser.add_argument('-f', '--flag', help='Specify a variable flag to query (with --value)', default=None)
+    parser.add_argument('--value', help='Only report the value, no history and no variable name', action="store_true")
+    args = parser.parse_args()
+
+    if args.unexpand and not args.value:
+        print("--unexpand only makes sense with --value")
+        sys.exit(1)
+
+    if args.flag and not args.value:
+        print("--flag only makes sense with --value")
+        sys.exit(1)
+
+    with bb.tinfoil.Tinfoil(tracking=True) as tinfoil:
+        if not args.recipe:
+            tinfoil.prepare(quiet=2, config_only=True)
+            d = tinfoil.config_data
+        else:
+            tinfoil.prepare(quiet=2)
+            d = tinfoil.parse_recipe(args.recipe)
+        if args.flag:
+            print(str(d.getVarFlag(args.variable, args.flag, expand=(not args.unexpand))))        
+        elif args.value:
+            print(str(d.getVar(args.variable, expand=(not args.unexpand))))
+        else:
+            bb.data.emit_var(args.variable, d=d, all=True)
-- 
2.30.2

Re: [bitbake-devel] [PATCH 2/2] bin/bitbake-getvar: Add a new command to query a variable value (with history)

[Paul Eggleton]

On Saturday, 17 April 2021 05:11:26 NZST Richard Purdie wrote:
> We've talked about having this for long enough. Add a command which queries
> a single variable value with history. This saves "bitbake -e | grep" and
> avoids the various pitfalls that has.
> 
> It also provides a neat example of using tinfoil to make such a query.
> 
> Parameters to limit the output to just the value, to limit to a variable
> flag and to not expand the output are provided.
> 
> [YOCTO #10748]
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

This is great, I have wanted to see this for quite a while! About the only 
feedback would be to add a short option for  --value, though I wonder if -v is 
the best option to choose or not given that it's typically short for "verbose" 
with most tools.

Cheers
Paul

Re: [bitbake-devel] [PATCH 2/2] bin/bitbake-getvar: Add a new command to query a variable value (with history)

[Ross Burton]

On Mon, 19 Apr 2021 at 22:36, Paul Eggleton
<bluelightning@bluelightning.org> wrote:
> This is great, I have wanted to see this for quite a while! About the only
> feedback would be to add a short option for  --value, though I wonder if -v is
> the best option to choose or not given that it's typically short for "verbose"
> with most tools.

-V would be an alternative but I don't think this is a great problem.

Ross

Re: [bitbake-devel] [PATCH 2/2] bin/bitbake-getvar: Add a new command to query a variable value (with history)

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 729 bytes --]

-v or -V can often be version as well..

On Tue, Apr 20, 2021 at 8:13 AM Ross Burton <ross@burtonini.com> wrote:

> On Mon, 19 Apr 2021 at 22:36, Paul Eggleton
> <bluelightning@bluelightning.org> wrote:
> > This is great, I have wanted to see this for quite a while! About the
> only
> > feedback would be to add a short option for  --value, though I wonder if
> -v is
> > the best option to choose or not given that it's typically short for
> "verbose"
> > with most tools.
>
> -V would be an alternative but I don't think this is a great problem.
>
> Ross
>
> 
>
>

-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1302 bytes --]

Re: [bitbake-devel] [PATCH 2/2] bin/bitbake-getvar: Add a new command to query a variable value (with history)

[Ola x Nilsson]

On fri, apr 16 2021, Richard Purdie wrote:

> We've talked about having this for long enough. Add a command which queries a single
> variable value with history. This saves "bitbake -e | grep" and avoids the
> various pitfalls that has.
>
> It also provides a neat example of using tinfoil to make such a query.
>
> Parameters to limit the output to just the value, to limit to a variable flag
> and to not expand the output are provided.

I think this will be a very useful tool.  I have hacked (as in I really
don't want to show anyone the code) something similar together on top of
bitbake -e and I've found this commandline very useful

bitbake-getvar RECIPE VARIABLE [VARIABLE...]

so that I do not have to repeat the command when I want to examine more
than one variable.  Expanding with VARIABLE[FLAG] syntax would be nice.
Of course, if you want to examine several flags for a variable that will
mean a lot of extra typing.

I chose to add a -g flag for --global, as in no specific recipe so it is
kind of reverse of this command's options.

Just my 2 cents

/Ola