[PATCHv2 0/5] Fixes for useradd-staticids.bbclass

[PATCHv2 0/5] Fixes for useradd-staticids.bbclass

[Peter Kjellerstedt]

The main change here is to fix the regression reported in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9777 which resulted
in setting USERADD_ERROR_DYNAMIC having no effect if the user or group
was not listed at all in a passwd or groups file.

In the ticket discussion there was also a suggestion to allow the
error to be turned into a warning. I have implemented this as well,
whereby one can now set USERADD_ERROR_DYNAMIC to either "error" or
"warn" to decide which behavior one wants. (It is of course still
possible to set it to "1" for backwards compatibility.)

The change to USERADD_ERROR_DYNAMIC requires some updates to the
documentation, which is provided in the final commit. This is intended
for the yocto-docs repository.

PATCHv2: Added a first commit to update the documentation of
USERADD_ERROR_DYNAMIC and USERADDEXTENSION per Mark Hatle's suggestion.
This is also intended for the yocto-docs repository.

//Peter

The following changes since commit 4799e1e89fd9b0f8b606e3c502fe89f32d45fe76:

  gtk+3: Add patch for --disable-opengl (2016-06-17 11:17:21 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib pkj/useradd-staticids-regression
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/useradd-staticids-regression

Peter Kjellerstedt (5):
  Update the documentation of USERADD_ERROR_DYNAMIC and USERADDEXTENSION
  documentation.conf: Add information about USERADD variables
  useradd-staticids.bbclass: Restore failure on missing UIDs/GIDs
  useradd-staticids.bbclass: Allow missing UIDs/GIDs to generate
    warnings
  Update the documentation of USERADD_ERROR_DYNAMIC

 documentation/ref-manual/ref-variables.xml | 27 +++++++++++++++------------
 meta/classes/useradd-staticids.bbclass     | 22 ++++++++++++++++------
 meta/conf/documentation.conf               |  4 ++++
 3 files changed, 35 insertions(+), 18 deletions(-)

-- 
2.9.0

[PATCHv2 1/5] Update the documentation of USERADD_ERROR_DYNAMIC and USERADDEXTENSION

[Peter Kjellerstedt]

Suggested-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 documentation/ref-manual/ref-variables.xml | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index 0252757..0b31566 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -14381,16 +14381,18 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
 
         <glossentry id='var-USERADD_ERROR_DYNAMIC'><glossterm>USERADD_ERROR_DYNAMIC</glossterm>
             <info>
-               USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files."
+               USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in any of the files listed in USERADD_UID_TABLES and USERADD_GID_TABLES."
             </info>
             <glossdef>
                 <para role="glossdeffirst">
 <!--                <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
-                    Forces the OpenEmbedded build system to produce an error
-                    if the user identification (<filename>uid</filename>) and
-                    group identification (<filename>gid</filename>) values
-                    are not defined in <filename>files/passwd</filename>
-                    and <filename>files/group</filename> files.
+
+                    Forces the OpenEmbedded build system to produce an error if
+                    the user identification (<filename>uid</filename>) and group
+                    identification (<filename>gid</filename>) values are not
+                    defined in any of the files listed
+                    in <link linkend='var-USERADD_UID_TABLES'><filename>USERADD_UID_TABLES</filename></link>
+                    and <link linkend='var-USERADD_GID_TABLES'><filename>USERADD_GID_TABLES</filename></link>.
                 </para>
 
                 <para>
@@ -14562,17 +14564,16 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
 
         <glossentry id='var-USERADDEXTENSION'><glossterm>USERADDEXTENSION</glossterm>
             <info>
-               USERADDEXTENSION[doc] = "When set to "useradd-staticids", causes the OpenEmbedded build system to base all user and group additions on a static passwd and group files found in BBPATH."
+               USERADDEXTENSION[doc] = "When set to 'useradd-staticids', causes the OpenEmbedded build system to base all user and group additions on files listed in USERADD_UID_TABLES and USERADD_GID_TABLES."
             </info>
             <glossdef>
                 <para role="glossdeffirst">
 <!--                <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
                     When set to "useradd-staticids", causes the
                     OpenEmbedded build system to base all user and group
-                    additions on a static
-                    <filename>passwd</filename> and
-                    <filename>group</filename> files found in
-                    <link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
+                    additions on files listed
+                    in <link linkend='var-USERADD_UID_TABLES'><filename>USERADD_UID_TABLES</filename></link>
+                    and <link linkend='var-USERADD_GID_TABLES'><filename>USERADD_GID_TABLES</filename></link>.
                 </para>
 
                 <para>
-- 
2.9.0

[PATCHv2 2/5] documentation.conf: Add information about USERADD variables

[Peter Kjellerstedt]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/conf/documentation.conf | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index c0025f5..010585a 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -452,8 +452,12 @@ UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension."
 UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
 USE_DEVFS[doc] = "When building images, specifies to populate or not /dev. This variable defaults to '1' (leave directory empty, surely because devtmpfs do the job). Set it to '0' to use makedevs (or consider using a custom file with IMAGE_DEVICE_TABLES)."
 USER_CLASSES[doc] = "List of additional classes to use when building images that enable extra features."
+USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in any of the files listed in USERADD_UID_TABLES and USERADD_GID_TABLES."
+USERADD_GID_TABLES[doc] = "Specifies a password file to use for obtaining static group identification (gid) values when the OpenEmbedded build system adds a group to the system during package installation."
 USERADD_PACKAGES[doc] = "When a recipe inherits the useradd class, this variable specifies the individual packages within the recipe that require users and/or groups to be added."
 USERADD_PARAM[doc] = "When a recipe inherits the useradd class, this variable specifies for a package what parameters should be passed to the useradd command if you wish to add a user to the system when the package is installed."
+USERADD_UID_TABLES[doc] = "Specifies a password file to use for obtaining static user identification (uid) values when the OpenEmbedded build system adds a user to the system during package installation."
+USERADDEXTENSION[doc] = "When set to 'useradd-staticids', causes the OpenEmbedded build system to base all user and group additions on files listed in USERADD_UID_TABLES and USERADD_GID_TABLES."
 
 #W
 
-- 
2.9.0

[PATCHv2 3/5] useradd-staticids.bbclass: Restore failure on missing UIDs/GIDs

[Peter Kjellerstedt]

A regression was introduced with commit 3149319a whereby setting
USERADD_ERROR_DYNAMIC no longer resulted in an error for users and
groups that were missing numeric UIDs and GIDs but were not mentioned
at all in any passwd or groups file.

[YOCTO #9777]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/useradd-staticids.bbclass | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass
index d963f28..90835cf 100644
--- a/meta/classes/useradd-staticids.bbclass
+++ b/meta/classes/useradd-staticids.bbclass
@@ -50,6 +50,11 @@ def update_useradd_static_config(d):
 
         return id_table
 
+    def handle_missing_id(id, type, pkg):
+        if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1':
+            #bb.error("Skipping recipe %s, package %s which adds %sname %s does not have a static ID defined." % (d.getVar('PN', True),  pkg, type, id))
+            raise bb.build.FuncFailed("%s - %s: %sname %s does not have a static ID defined." % (d.getVar('PN', True), pkg, type, id))
+
     # We parse and rewrite the useradd components
     def rewrite_useradd(params):
         # The following comes from --help on useradd from shadow
@@ -116,6 +121,8 @@ def update_useradd_static_config(d):
                 users = merge_files(get_passwd_list(d), 7)
 
             if uaargs.LOGIN not in users:
+                if not uaargs.uid or not uaargs.uid.isdigit() or not uaargs.gid:
+                    handle_missing_id(uaargs.LOGIN, 'user', pkg)
                 continue
 
             field = users[uaargs.LOGIN]
@@ -165,9 +172,8 @@ def update_useradd_static_config(d):
             uaargs.shell = field[6] or uaargs.shell
 
             # Should be an error if a specific option is set...
-            if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1' and not ((uaargs.uid and uaargs.uid.isdigit()) and uaargs.gid):
-                #bb.error("Skipping recipe %s, package %s which adds username %s does not have a static uid defined." % (d.getVar('PN', True),  pkg, uaargs.LOGIN))
-                raise bb.build.FuncFailed("%s - %s: Username %s does not have a static uid defined." % (d.getVar('PN', True), pkg, uaargs.LOGIN))
+            if not uaargs.uid or not uaargs.uid.isdigit() or not uaargs.gid:
+                 handle_missing_id(uaargs.LOGIN, 'user', pkg)
 
             # Reconstruct the args...
             newparam  = ['', ' --defaults'][uaargs.defaults]
@@ -248,6 +254,8 @@ def update_useradd_static_config(d):
                 groups = merge_files(get_group_list(d), 4)
 
             if gaargs.GROUP not in groups:
+                if not gaargs.gid or not gaargs.gid.isdigit():
+                    handle_missing_id(gaargs.GROUP, 'group', pkg)
                 continue
 
             field = groups[gaargs.GROUP]
@@ -257,9 +265,8 @@ def update_useradd_static_config(d):
                     bb.warn("%s: Changing groupname %s's gid from (%s) to (%s), verify configuration files!" % (d.getVar('PN', True), gaargs.GROUP, gaargs.gid, field[2]))
                 gaargs.gid = field[2]
 
-            if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1' and not (gaargs.gid and gaargs.gid.isdigit()):
-                #bb.error("Skipping recipe %s, package %s which adds groupname %s does not have a static gid defined." % (d.getVar('PN', True),  pkg, gaargs.GROUP))
-                raise bb.build.FuncFailed("%s - %s: Groupname %s does not have a static gid defined." % (d.getVar('PN', True), pkg, gaargs.GROUP))
+            if not gaargs.gid or not gaargs.gid.isdigit():
+                handle_missing_id(gaargs.GROUP, 'group', pkg)
 
             # Reconstruct the args...
             newparam  = ['', ' --force'][gaargs.force]
-- 
2.9.0

[PATCHv2 4/5] useradd-staticids.bbclass: Allow missing UIDs/GIDs to generate warnings

[Peter Kjellerstedt]

Previously when USERADD_ERROR_DYNAMIC was set to "1", an exception was
raised if no numeric UID/GID could be determined for a user/group. Now
it is possible to set it to either "error", which results in the old
behavior, or "warn" in which case a warning is issued instead.

For backwards compatibility reasons, it is still possible to set
USERADD_ERROR_DYNAMIC to "1" and get an exception in case of failure.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/useradd-staticids.bbclass | 5 ++++-
 meta/conf/documentation.conf           | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass
index 90835cf..b6e498c 100644
--- a/meta/classes/useradd-staticids.bbclass
+++ b/meta/classes/useradd-staticids.bbclass
@@ -51,9 +51,12 @@ def update_useradd_static_config(d):
         return id_table
 
     def handle_missing_id(id, type, pkg):
-        if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1':
+        # For backwards compatibility we accept "1" in addition to "error"
+        if d.getVar('USERADD_ERROR_DYNAMIC', True) == 'error' or d.getVar('USERADD_ERROR_DYNAMIC', True) == '1':
             #bb.error("Skipping recipe %s, package %s which adds %sname %s does not have a static ID defined." % (d.getVar('PN', True),  pkg, type, id))
             raise bb.build.FuncFailed("%s - %s: %sname %s does not have a static ID defined." % (d.getVar('PN', True), pkg, type, id))
+        elif d.getVar('USERADD_ERROR_DYNAMIC', True) == 'warn':
+            bb.warn("%s - %s: %sname %s does not have a static ID defined." % (d.getVar('PN', True), pkg, type, id))
 
     # We parse and rewrite the useradd components
     def rewrite_useradd(params):
diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 010585a..51c4116 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -452,7 +452,7 @@ UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension."
 UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
 USE_DEVFS[doc] = "When building images, specifies to populate or not /dev. This variable defaults to '1' (leave directory empty, surely because devtmpfs do the job). Set it to '0' to use makedevs (or consider using a custom file with IMAGE_DEVICE_TABLES)."
 USER_CLASSES[doc] = "List of additional classes to use when building images that enable extra features."
-USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in any of the files listed in USERADD_UID_TABLES and USERADD_GID_TABLES."
+USERADD_ERROR_DYNAMIC[doc] = "If set to 'error', forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in any of the files listed in USERADD_UID_TABLES and USERADD_GID_TABLES. If set to 'warn', a warning will be issued instead."
 USERADD_GID_TABLES[doc] = "Specifies a password file to use for obtaining static group identification (gid) values when the OpenEmbedded build system adds a group to the system during package installation."
 USERADD_PACKAGES[doc] = "When a recipe inherits the useradd class, this variable specifies the individual packages within the recipe that require users and/or groups to be added."
 USERADD_PARAM[doc] = "When a recipe inherits the useradd class, this variable specifies for a package what parameters should be passed to the useradd command if you wish to add a user to the system when the package is installed."
-- 
2.9.0

[PATCHv2 5/5] Update the documentation of USERADD_ERROR_DYNAMIC

[Peter Kjellerstedt]

Document that USERADD_ERROR_DYNAMIC can now take "error" or "warn" as
values, to either cause an exception or a warning in case no numeric
UID/GID can be determined for a user or group.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 documentation/ref-manual/ref-variables.xml | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index 0b31566..c2a5f29 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -14381,18 +14381,20 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
 
         <glossentry id='var-USERADD_ERROR_DYNAMIC'><glossterm>USERADD_ERROR_DYNAMIC</glossterm>
             <info>
-               USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in any of the files listed in USERADD_UID_TABLES and USERADD_GID_TABLES."
+               USERADD_ERROR_DYNAMIC[doc] = "If set to 'error', forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in any of the files listed in USERADD_UID_TABLES and USERADD_GID_TABLES. If set to 'warn', a warning will be issued instead."
             </info>
             <glossdef>
                 <para role="glossdeffirst">
 <!--                <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
 
-                    Forces the OpenEmbedded build system to produce an error if
-                    the user identification (<filename>uid</filename>) and group
-                    identification (<filename>gid</filename>) values are not
-                    defined in any of the files listed
+                    If set to "error", forces the OpenEmbedded build system to
+                    produce an error if the user identification
+                    (<filename>uid</filename>) and group identification
+                    (<filename>gid</filename>) values are not defined in any of
+                    the files listed
                     in <link linkend='var-USERADD_UID_TABLES'><filename>USERADD_UID_TABLES</filename></link>
-                    and <link linkend='var-USERADD_GID_TABLES'><filename>USERADD_GID_TABLES</filename></link>.
+                    and <link linkend='var-USERADD_GID_TABLES'><filename>USERADD_GID_TABLES</filename></link>. If
+                    set to "warn", a warning will be issued instead.
                 </para>
 
                 <para>
@@ -14408,7 +14410,7 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                     your <filename>local.conf</filename> file as
                     follows:
                     <literallayout class='monospaced'>
-     USERADD_ERROR_DYNAMIC = "1"
+     USERADD_ERROR_DYNAMIC = "error"
                     </literallayout>
                     Overriding the default behavior implies you are going to
                     also take steps to set static <filename>uid</filename> and
-- 
2.9.0