[PATCH v3] ConfHandler: Allow the '@' character in variable flag names

[PATCH v3] ConfHandler: Allow the '@' character in variable flag names

[James Raphael Tiovalen]

This patch enables the usage of the '@' character in variable flag
names.

One use case of variable flags is to assign the network namespaces of
some systemd services/targets to configure other parts of the build
process of some system. The filenames of systemd services/targets might
contain the '@' character if they are template unit files that can take
in a single parameter/argument and be instanced multiple times, as
indicated by systemd's official manual page.

This patch is successfully verified by creating a custom BitBake recipe
that sets and unsets the value of a variable flag with the '@' character
in its name and ensuring that no ParseError is being thrown. A
regression test has also been added to `bb.parse`.

`bin/bitbake-selftest` has also been successfully executed and all tests
passed.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
v3:
Improve commit message to better convey the intent of this patch.

v2:
Add regression test.
---
 lib/bb/parse/parse_py/ConfHandler.py |  4 ++--
 lib/bb/tests/parse.py                | 15 +++++++++++++++
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/lib/bb/parse/parse_py/ConfHandler.py b/lib/bb/parse/parse_py/ConfHandler.py
index 30760672..f2b8a2c9 100644
--- a/lib/bb/parse/parse_py/ConfHandler.py
+++ b/lib/bb/parse/parse_py/ConfHandler.py
@@ -21,7 +21,7 @@ __config_regexp__  = re.compile( r"""
     ^
     (?P<exp>export\s+)?
     (?P<var>[a-zA-Z0-9\-_+.${}/~:]+?)
-    (\[(?P<flag>[a-zA-Z0-9\-_+.]+)\])?
+    (\[(?P<flag>[a-zA-Z0-9\-_+.@]+)\])?
 
     \s* (
         (?P<colon>:=) |
@@ -45,7 +45,7 @@ __include_regexp__ = re.compile( r"include\s+(.+)" )
 __require_regexp__ = re.compile( r"require\s+(.+)" )
 __export_regexp__ = re.compile( r"export\s+([a-zA-Z0-9\-_+.${}/~]+)$" )
 __unset_regexp__ = re.compile( r"unset\s+([a-zA-Z0-9\-_+.${}/~]+)$" )
-__unset_flag_regexp__ = re.compile( r"unset\s+([a-zA-Z0-9\-_+.${}/~]+)\[([a-zA-Z0-9\-_+.]+)\]$" )
+__unset_flag_regexp__ = re.compile( r"unset\s+([a-zA-Z0-9\-_+.${}/~]+)\[([a-zA-Z0-9\-_+.@]+)\]$" )
 __addpylib_regexp__      = re.compile(r"addpylib\s+(.+)\s+(.+)" )
 
 def init(data):
diff --git a/lib/bb/tests/parse.py b/lib/bb/tests/parse.py
index ee7f2534..dba77292 100644
--- a/lib/bb/tests/parse.py
+++ b/lib/bb/tests/parse.py
@@ -218,3 +218,18 @@ VAR = " \\
         with self.assertRaises(bb.BBHandledException):
             d = bb.parse.handle(f.name, self.d)['']
 
+
+    at_sign_in_var_flag = """
+A[flag@.service] = "nonet"
+B[flag@.target] = "ntb"
+
+unset A[flag@.service]
+"""
+    def test_parse_at_sign_in_var_flag(self):
+        f = self.parsehelper(self.at_sign_in_var_flag)
+        d = bb.parse.handle(f.name, self.d)['']
+        self.assertEqual(d.getVar("A"), None)
+        self.assertEqual(d.getVar("B"), None)
+        self.assertEqual(d.getVarFlag("A","flag@.service"), None)
+        self.assertEqual(d.getVarFlag("B","flag@.target"), "ntb")
+
-- 
2.39.2

Re: [bitbake-devel] [PATCH v3] ConfHandler: Allow the '@' character in variable flag names

[Richard Purdie]

On Fri, 2023-03-17 at 00:57 +0800, James R T wrote:
> This patch enables the usage of the '@' character in variable flag
> names.
> 
> One use case of variable flags is to assign the network namespaces of
> some systemd services/targets to configure other parts of the build
> process of some system. The filenames of systemd services/targets might
> contain the '@' character if they are template unit files that can take
> in a single parameter/argument and be instanced multiple times, as
> indicated by systemd's official manual page.
> 
> This patch is successfully verified by creating a custom BitBake recipe
> that sets and unsets the value of a variable flag with the '@' character
> in its name and ensuring that no ParseError is being thrown. A
> regression test has also been added to `bb.parse`.
> 
> `bin/bitbake-selftest` has also been successfully executed and all tests
> passed.
> 
> Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>


I'm torn on this. I can see why some people would like it, equally,
restricting the character set does sometimes make parsing easier and
gives us options in future to extend the syntax.

After some though, I did wonder if we should change things so that the
first character has more restriction and @ is only allowed mid string
as a compromise?

Cheers,

Richard

Re: [bitbake-devel] [PATCH v3] ConfHandler: Allow the '@' character in variable flag names

[James R T]

On Fri, Mar 17, 2023 at 1:42 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> After some though, I did wonder if we should change things so that the
> first character has more restriction and @ is only allowed mid string
> as a compromise?

Sure, that sounds good. I can modify the regex to restrict the first
character in v4 of the patch. Will update the test as well.

Best regards,
James Raphael Tiovalen